Contrib.social

Robots.txt module settings only work if there is no robots.txt file

Open on Drupal.org →
Created on 16 January 2025, 3 months ago

Problem/Motivation

The SEO tools recipe adds Robots.txt module to manage this via the UI. But the module only works if there is no robots.txt file in the web root. The file is provided by core and is needed by default, because the SEO tools recipe is optional.

Steps to reproduce

  1. Install Drupal CMS
  2. Apply the SEO tools recipe
  3. See the warning on the status page, and see that the changes in Robots.txt settings do not have any effect

Proposed resolution

??

🐛 Bug report
Status

Active

Component

Track: SEO

Created by

🇦🇺Australia pameeela

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @pameeela
  • Comment 3 months ago →
    🇺🇸United States thejimbirch Cape Cod, Massachusetts
  • Comment 3 months ago →
    🇬🇧United Kingdom catch

    I don't think the robots.txt file can be removed from Drupal CMS - even if this is removed from the base recipe, the robots.txt module could be uninstalled, and then the site would be left with no robots.txt at all. Fine for sites to do it themselves, that's why it's part of scaffolding, but Drupal CMS isn't a site.

  • Comment about 2 months ago →
    🇺🇸United States thejimbirch Cape Cod, Massachusetts

    Looks like there is an issue with a patch in the module's queue. Postponing on that.

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024