Multisite support (another approach)

Created on 22 May 2020, almost 5 years ago
Updated 13 August 2024, 8 months ago

Looking through the issues related to multisite support, I wasn't able to find a good solution. Removing robots.txt is not an option, as my site is on a platform, on which there are couple of dozens of other sites. This would require coordination between different teems, working withing different organizations. And, it feels there is not really much in the admin interface to provide flexibility.

So... The concept is simple: have each multisite marked in such a way that Apache or Nginx could recognize, and perform redirect to a URL on which the module will return that site's robots.txt file. This will require a change to rewrite rules. Marking/unmarking of the sites should be available from the admin interface. Additionally, we should be able to recognize aliases recorded in sites.php file and be able to respond to the aliases not recorded in sites.php.

Provided patch is a start, and allows all of the above, with exception of Nginx support - still working on that one.

Sites are marked via admin interface, in the following manner:

sites/mysite.com/.robotstxt
sites/mysite.alias.com.robotstxt            - for 'mysite.alias.com' alias
sites/mysite.another.alias.com.robotstxt    - for 'mysite.another.alias.com' alias

Apache mod_rewrite additions to .htaccess file, and Nginx rules updates are displayed in the admin interface as a suggestion.

Apache rules as follows:

RewriteCond %{REQUEST_URI} ^/robots.txt$
RewriteCond %{DOCUMENT_ROOT}/sites/%{HTTP_HOST}/.robotstxt -f [OR]
RewriteCond %{DOCUMENT_ROOT}/sites/%{SERVER_PORT}.%{HTTP_HOST}.robotstxt -f [OR]
RewriteCond %{DOCUMENT_ROOT}/sites/%{HTTP_HOST}.robotstxt -f
RewriteRule ^ index.php?q=robots.txt [L] 

Nginx rules as follows:

location = /robots.txt {
    allow all;
    log_not_found off;
    access_log off;
    if (-f $document_root/sites/$host/.robotstxt) {
        rewrite ^ /index.php?q=robots.txt;
    }
    if (-f $document_root/sites/$host.robotstxt) {
        rewrite ^ /index.php?q=robots.txt;
    }
    if (-f $document_root/sites/$server_port.$host.robotstxt) {
        rewrite ^ /index.php?q=robots.txt;
    }
}
✨ Feature request
Status

Postponed

Version

1.0

Component

Code

Created by

πŸ‡ΊπŸ‡ΈUnited States euk

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

Production build 0.71.5 2024