Turn robots.txt into config

Created on 6 October 2020, over 4 years ago

Updated 22 March 2023, about 2 years ago

robots.txt -> turn into config, see contrib module
- add to config_ignore
- deploy with hook_update first time?
- delete robots.txt
- work with infra to not deploy it anymore
Crawl-Delay: 10 remove from robots.txt
Test robots.txt module integration with sitemap

✨ Feature request

Status

Postponed

Version

6.0

Component

Code

Created by

🇧🇪Belgium Grayle

Live updates comments and jobs are added and updated live.

Incomplete comments