Blog
FAQ
Discussions
Search
Projects, issues, users, and merge requests.
Project ID, name, and description.
User nickname, name, and org.
Issue ID, title, and summary.
Merge request titles.
Contrib
.social
Feed
Live feed
Builds
Live builds
Core
Tags
Tags and Initiatives
RobotsTxt
Open on Drupal.org โ
Open on Drupal GitLab โ
Created on 11 March 2006,
about 19 years ago
Maintained by
๐บ๐ธ
United States
ccjjmartin
๐บ๐ธ
United States
david strauss
๐บ๐ธ
United States
mikeegoulding
๐บ๐ธ
United States
todd nienkerk
Clone
Clone with SSH
Clone with HTTPS
Clone with
doGit
๐บ๐ธ
United States
58%
๐ซ๐ท
France
27%
๐ดโโ ๏ธ
6%
๐น๐ท
Turkey
3%
๐บ๐ฆ
Ukraine
3%
๐ช๐ธ
Spain
3%
Top 9 contributors
Velir
39%
๐บ๐ธ
@kevinquillen
Smile
21%
๐ซ๐ท
@Grimreaper
Redfin Solutions, LLC
12%
๐บ๐ธ
@leslieg
Insite
6%
๐ซ๐ท
@prudloff
Kanopi Studios
3%
๐บ๐ธ
@thejimbirch
GeoPerformans
3%
๐น๐ท
@orkut murat yฤฑlmaz
Drupal Ukraine Community
3%
๐บ๐ฆ
@rollins
Drupal France & Francophonie
๐ซ๐ท
@Grimreaper
Lemberg Solutions
๐บ๐ฆ
@rollins
and 3 individuals
( 12% )
@cilefen
๐บ๐ธ
@kentr
๐ช๐ธ
@alvarodemendoza
Follow
Sign in to follow projects
Merge Requests
More
!17
Include fixes from core robots.txt in default robots.txt in module
Open
Show issue
๐ซ๐ท
France
prudloff
updated
about 1 month ago
!16
Do not override existing config on install
Open
Show issue
๐ซ๐ท
France
Grimreaper
updated
about 1 month ago
!7
Provide options to add Simple Sitemap files to robot.txt
Open
Show issue
๐ช๐ธ
Spain
gxleano
updated
2 months ago
!15
Add X-Content-Type-Options header
Open
Show issue
Unnamed author
updated
7 months ago
!6
Add cache context url.site to robots.txt response
Merged
Show issue
๐ง๐ช
Belgium
JeroenT
updated
8 months ago
!10
Integration with Gitlab CI
Merged
Show issue
๐ท๐บ
Russia
zniki.ru
updated
8 months ago
More Merge Requests
Issues
๐
Robots.txt file not changed
Needs review
Code
Created
over 2 years ago
v1.4
๐ฒ๐ฉ
Moldova
nick.murza
about 1 month ago
๐
Include fixes from core robots.txt in default robots.txt in module
Active
Code
Created
2 months ago
v1.0
๐ซ๐ท
France
prudloff
about 1 month ago
๐
Do not override existing config on install
Active
Code
Created
about 1 month ago
v1.0
๐ซ๐ท
France
Grimreaper
about 1 month ago
๐ฌ
Coding Standards
Active
Code
Created
8 months ago
v1.5
๐จ๐ด
Colombia
andresalvarez
about 2 months ago
โจ
Provide options to add Simple Sitemap files to robot.txt
RTBC
Code
Created
almost 4 years ago
v1.0
๐บ๐ธ
United States
DamienMcKenna
2 months ago
โจ
Disallow GPTBot by default
Active
Code
Created
over 1 year ago
v1.5
๐บ๐ธ
United States
kevinquillen
2 months ago
๐ฌ
Bing can't find robots.txt
Active
Miscellaneous
Created
3 months ago
v1.6
๐จ๐ฆ
Canada
tneo
3 months ago
๐
Update categories on project page for compatibility with Project Browser
Active
Miscellaneous
Created
7 months ago
v1.0
๐บ๐ธ
United States
leslieg
7 months ago
๐
Update summary on project page for compatibility with Project Browser
Active
Miscellaneous
Created
7 months ago
v1.0
๐บ๐ธ
United States
leslieg
7 months ago
๐
Update logo for compatibility with Project Browser
Active
Miscellaneous
Created
7 months ago
v1.0
๐บ๐ธ
United States
leslieg
7 months ago
๐
Add X-Content-Type-Options header
Needs review
Code
Created
12 months ago
v1.0
๐ฐ๐ฟ
Kazakhstan
Dmitrii_Zadorozhnyi
7 months ago
โจ
Support for ai.txt?
Active
Miscellaneous
Created
over 1 year ago
v1.0
๐ฉ๐ฐ
Denmark
blymemikkel
7 months ago
๐
Add cache context url.site to robots.txt response
Fixed
Code
Created
almost 3 years ago
v1.0
๐ง๐ช
Belgium
JeroenT
7 months ago
๐
Integration with Gitlab CI
Fixed
Code
Created
over 1 year ago
v1.0
๐ท๐บ
Russia
zniki.ru
7 months ago
๐
Automated Drupal 11 compatibility fixes for robotstxt
Fixed
Code
Created
about 1 year ago
v1.5
project update bot
7 months ago
โจ
Multisite support (another approach)
Postponed
Code
Created
almost 5 years ago
v1.0
๐บ๐ธ
United States
euk
8 months ago
๐
LogicException: The controller result claims to be providing relevant cache metadata, but leaked metadata was detected
RTBC
Code
Created
about 5 years ago
v1.0
๐ท๐ธ
Serbia
Vladimir Tanovic
8 months ago
โจ
Support semantic versioning
Postponed
Code
Created
almost 5 years ago
v1.0
๐บ๐ธ
United States
ccjjmartin
8 months ago
โจ
Allow using the Drupal core robots.txt file
Needs work
Code
Created
about 6 years ago
v1.0
๐จ๐ญ
Switzerland
drikc
10 months ago
๐
Symfony\Component\Routing\Exception\RouteNotFoundException: Route "robotstxt.admin_settings_form" does not exist. in Drupal\Core\Routing\RouteProvider->getRouteByName() (line 206 of core/lib/Drupal/Core/Routing/RouteProvider.php).
Postponed: needs info
Code
Created
12 months ago
v1.5
๐ต๐ฐ
Pakistan
isalmanhaider
11 months ago
โจ
Create hook_robotstxt_alter()
Needs work
Code
Created
over 7 years ago
v1.0
๐บ๐ธ
United States
bappa.sarkar
11 months ago
โจ
Integrate with the Dark Visitors API
Active
Code
Created
11 months ago
v1.0
๐จ๐ฆ
Canada
sillygwailo
11 months ago
โจ
Enable usage of tokens
Needs work
Code
Created
about 5 years ago
v1.0
๐ณ๐ฑ
Netherlands
Ruuds
about 1 year ago
๐
Add Unit tests for Robots Txt
Needs review
Code
Created
about 4 years ago
v1.0
๐บ๐ธ
United States
mikelutz
about 1 year ago
๐
Update README.md as per README.md format
Fixed
Documentation
Created
about 2 years ago
v1.0
๐ฎ๐ณ
India
jay jangid
over 1 year ago
๐ฌ
How to use this module with auto scaffold robots.txt?
Fixed
Documentation
Created
almost 5 years ago
v1.0
๐ท๐บ
Russia
vvs
over 1 year ago
contrib
.social
Blog
FAQ
Discussions
Production build 0.71.5
2024