Blog
FAQ
Discussions
Search
Projects, issues, users, and merge requests.
Project ID, name, and description.
User nickname, name, and org.
Issue ID, title, and summary.
Merge request titles.
Contrib
.social
Feed
Live feed
Builds
Live builds
Core
Tags
Tags and Initiatives
RobotsTxt
Open on Drupal.org โ
Open on Drupal GitLab โ
Created on 11 March 2006,
over 19 years ago
Maintained by
๐บ๐ธ
United States
ccjjmartin
๐บ๐ธ
United States
david strauss
๐บ๐ธ
United States
mikeegoulding
๐บ๐ธ
United States
todd nienkerk
Clone
Clone with SSH
Clone with HTTPS
Clone with
doGit
๐บ๐ธ
United States
53%
๐ซ๐ท
France
25%
๐ณ๐ฌ
Nigeria
6%
๐ดโโ ๏ธ
6%
๐น๐ท
Turkey
3%
๐จ๐ด
Colombia
3%
๐บ๐ฆ
Ukraine
3%
๐ช๐ธ
Spain
3%
Top 10 contributors
Velir
36%
๐บ๐ธ
@kevinquillen
Smile
19%
๐ซ๐ท
@Grimreaper
Redfin Solutions, LLC
11%
๐บ๐ธ
@leslieg
Skillmatic Ace
6%
๐ณ๐ฌ
@chike
Insite
6%
๐ซ๐ท
@prudloff
Kanopi Studios
3%
๐บ๐ธ
@thejimbirch
GeoPerformans
3%
๐น๐ท
@orkut murat yฤฑlmaz
Urban Insight
3%
๐จ๐ด
@camilo.escobar
Drupal Ukraine Community
3%
๐บ๐ฆ
@rollins
Drupal France & Francophonie
๐ซ๐ท
@Grimreaper
+1
and 1 other organisations
Lemberg Solutions
๐บ๐ฆ
@rollins
and 3 individuals
( 11% )
@cilefen
๐บ๐ธ
@kentr
๐ช๐ธ
@alvarodemendoza
Follow
Sign in to follow projects
Merge Requests
More
!17
Include fixes from core robots.txt in default robots.txt in module
Open
Show issue
๐ซ๐ท
France
prudloff
updated
4 months ago
!16
Do not override existing config on install
Open
Show issue
๐ซ๐ท
France
Grimreaper
updated
4 months ago
!7
Provide options to add Simple Sitemap files to robot.txt
Open
Show issue
๐ช๐ธ
Spain
gxleano
updated
5 months ago
!15
Add X-Content-Type-Options header
Open
Show issue
Unnamed author
updated
10 months ago
!6
Add cache context url.site to robots.txt response
Merged
Show issue
๐ง๐ช
Belgium
JeroenT
updated
11 months ago
!10
Integration with Gitlab CI
Merged
Show issue
๐ท๐บ
Russia
zniki.ru
updated
11 months ago
More Merge Requests
Issues
๐ฌ
Bing can't find robots.txt
Active
Miscellaneous
Created
6 months ago
v1.6
๐จ๐ฆ
Canada
tneo
9 days ago
โจ
Provide options to add Simple Sitemap files to robot.txt
RTBC
Code
Created
about 4 years ago
v1.0
๐บ๐ธ
United States
damienmckenna
2 months ago
๐
Automatically removing robots.txt provided by Drupal core
Fixed
Documentation
Created
over 8 years ago
v1.0
๐ฌ๐ง
United Kingdom
marcelovani
3 months ago
โจ
Add option to "Invalidate robots.txt cache on each cron run"
Active
Code
Created
3 months ago
v1.6
๐จ๐ด
Colombia
camilo.escobar
3 months ago
๐
Robots.txt file not changed
Needs review
Code
Created
over 2 years ago
v1.4
๐ฒ๐ฉ
Moldova
nick.murza
4 months ago
๐
Include fixes from core robots.txt in default robots.txt in module
Active
Code
Created
5 months ago
v1.0
๐ซ๐ท
France
prudloff
4 months ago
๐
Do not override existing config on install
Active
Code
Created
4 months ago
v1.0
๐ซ๐ท
France
Grimreaper
4 months ago
๐ฌ
Coding Standards
Active
Code
Created
11 months ago
v1.5
๐จ๐ด
Colombia
andresalvarez
5 months ago
โจ
Disallow GPTBot by default
Active
Code
Created
over 1 year ago
v1.5
๐บ๐ธ
United States
kevinquillen
5 months ago
๐
Update categories on project page for compatibility with Project Browser
Active
Miscellaneous
Created
10 months ago
v1.0
๐บ๐ธ
United States
leslieg
10 months ago
๐
Update summary on project page for compatibility with Project Browser
Active
Miscellaneous
Created
10 months ago
v1.0
๐บ๐ธ
United States
leslieg
10 months ago
๐
Update logo for compatibility with Project Browser
Active
Miscellaneous
Created
10 months ago
v1.0
๐บ๐ธ
United States
leslieg
10 months ago
๐
Add X-Content-Type-Options header
Needs review
Code
Created
about 1 year ago
v1.0
๐ฐ๐ฟ
Kazakhstan
Dmitrii_Zadorozhnyi
10 months ago
โจ
Support for ai.txt?
Active
Miscellaneous
Created
over 1 year ago
v1.0
๐ฉ๐ฐ
Denmark
blymemikkel
10 months ago
๐
Add cache context url.site to robots.txt response
Fixed
Code
Created
about 3 years ago
v1.0
๐ง๐ช
Belgium
JeroenT
10 months ago
๐
Integration with Gitlab CI
Fixed
Code
Created
over 1 year ago
v1.0
๐ท๐บ
Russia
zniki.ru
10 months ago
๐
Automated Drupal 11 compatibility fixes for robotstxt
Fixed
Code
Created
over 1 year ago
v1.5
project update bot
10 months ago
โจ
Multisite support (another approach)
Postponed
Code
Created
about 5 years ago
v1.0
๐บ๐ธ
United States
euk
11 months ago
๐
LogicException: The controller result claims to be providing relevant cache metadata, but leaked metadata was detected
RTBC
Code
Created
over 5 years ago
v1.0
๐ท๐ธ
Serbia
Vladimir Tanovic
11 months ago
โจ
Support semantic versioning
Postponed
Code
Created
about 5 years ago
v1.0
๐บ๐ธ
United States
ccjjmartin
11 months ago
โจ
Allow using the Drupal core robots.txt file
Needs work
Code
Created
over 6 years ago
v1.0
๐จ๐ญ
Switzerland
drikc
about 1 year ago
๐
Symfony\Component\Routing\Exception\RouteNotFoundException: Route "robotstxt.admin_settings_form" does not exist. in Drupal\Core\Routing\RouteProvider->getRouteByName() (line 206 of core/lib/Drupal/Core/Routing/RouteProvider.php).
Postponed: needs info
Code
Created
about 1 year ago
v1.5
๐ต๐ฐ
Pakistan
isalmanhaider
about 1 year ago
โจ
Create hook_robotstxt_alter()
Needs work
Code
Created
almost 8 years ago
v1.0
๐บ๐ธ
United States
bappa.sarkar
about 1 year ago
โจ
Integrate with the Dark Visitors API
Active
Code
Created
about 1 year ago
v1.0
๐จ๐ฆ
Canada
sillygwailo
about 1 year ago
โจ
Enable usage of tokens
Needs work
Code
Created
over 5 years ago
v1.0
๐ณ๐ฑ
Netherlands
Ruuds
over 1 year ago
๐
Add Unit tests for Robots Txt
Needs review
Code
Created
over 4 years ago
v1.0
๐บ๐ธ
United States
mikelutz
over 1 year ago
๐
Update README.md as per README.md format
Fixed
Documentation
Created
over 2 years ago
v1.0
๐ฎ๐ณ
India
jay jangid
over 1 year ago
๐ฌ
How to use this module with auto scaffold robots.txt?
Fixed
Documentation
Created
about 5 years ago
v1.0
๐ท๐บ
Russia
vvs
over 1 year ago
contrib
.social
Blog
FAQ
Discussions
Production build 0.71.5
2024