Google Sitemaps Ping Endpoint Deprecated

Created on 20 February 2024, 11 months ago
Updated 22 February 2024, 11 months ago

Problem/Motivation

Google has deprecated the sitemaps ping endpoint:

https://developers.google.com/search/blog/2023/06/sitemaps-lastmod-ping

This means that attempts to submit sitemaps using the ping endpoint are now rejected with an HTTP 404 response. Here's an example of the error that appeared in my server log:

Submitted the sitemap to https://www.google.com/ping?sitemap=https://www.mysite.org/sitemap.xml and received response 404.

Google says that "You can still submit your sitemaps through robots.txt and Search Console". That would means changes to this module.

Steps to reproduce

Submit a sitemap using the ping endpoint as follows:

https://www.google.com/ping?sitemap=https://www.mysite.org/sitemap.xml

This returns an HTTP 404 error response.

πŸ› Bug report
Status

Active

Version

1.5

Component

Code

Created by

πŸ‡ΊπŸ‡ΈUnited States sah62 US

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @sah62
  • πŸ‡¨πŸ‡¦Canada mparker17 UTC-4

    @sah62, thank you for the bug report!

    I apologize, but I've looked through the code, and I cannot find a feature to ping Google (or any search engine) with a link to the Sitemap generated by this module in the 8.x-1.5 version of this module (i.e.: the version that you filed this ticket against), nor the 8.x-1.x branch of this module, nor the 8.x-2.x branch of this module. (disclaimer: I am new to this module and most familiar with the 8.x-2.x branch).

    I suspect that the issue you're reporting here might actually be a bug in either the XML Sitemap module β†’ , or the Simple XML Sitemap module β†’ . But, I'd like to confirm a few things from you before I move this ticket into one of those queues:

    1. The log where you saw the "Submitted the sitemap to https://www.google.com/ping?sitemap=https://www.mysite.org/sitemap.xml and received response 404." error should have an associated "type" or "channel" with that message... could you tell me what that "type" or "channel" is?

    2. Are you aware of a configuration option to enable/disable pinging Google? If so, what is the URL of the page where you see that configuration option? (e.g.: https://mysite.com/admin/config/lorem/ipsum)

    3. Do you have either the XML Sitemap module β†’ , or the Simple XML Sitemap module β†’ downloaded on your site? If both are downloaded, does one or both of them have a check-mark next to their name at /admin/modules?

    4. Can you clarify what kind of changes you would like the module maintainers to make? That is to say, would you like your site to simply stop pinging Google, or would you like it to both stop pinging Google and modify robots.txt to include a Sitemap line, or something else? (even if it turns out that this is a bug with another module, the maintainers of the other module will want to know the answer to this question)

    (note you don't have to answer all of these in the same reply: in particular, question 4 seems like it could be harder to answer than questions 1, 2, and 3)

    As an aside, an additional clarification that might be useful for understanding how this module interacts with other modules is that this module defines a path /sitemap that displays a human-readable, HTML sitemap. But as far as Drupal's routing system is concerned, /sitemap.xml is a completely different path: this module doesn't define a /sitemap.xml path, but other modules (e.g.: the XML Sitemap module β†’ , or the Simple XML Sitemap module β†’ ) are free to define and own the path /sitemap.xml at the same time that this module can define and own the path at /sitemap.

    Thank you!

  • πŸ‡ΊπŸ‡ΈUnited States sah62 US

    Thanks for the reply! Yes, sorry, I believe this is an issue, or also an issue, for the XML Sitemap module. Please move it as appropriate.

    1. The type is "xmlsitemap".

    2. The URL I see on my site to enable or disable submission to Google is this one:

    https://www.mysite.com/admin/config/search/xmlsitemap/engines

    3. Yes, I'm using the XML Sitemap module. It's checked on the /admin/modules page.

    4. My preference would be for sitemap submission to work without error. Given what I read in the Google blog, I believe this means "stop pinging Google AND modify robots.txt to include a Sitemap line" as you described.

    In the meantime I can disable sitemap submission by uninstalling the XML Sitemap Engines module.

  • πŸ‡¨πŸ‡¦Canada mparker17 UTC-4

    Hi @sah62, thanks for the clarification! I will re-file this into the xmlsitemap module's queue.

    Taking a brief look at their issue queue, the following xmlsitemap issues might be relevant to your situation:

    You might also be interested in the RobotsTxt module β†’ , which gives you more control over your site's robots.txt file. I gather that either the xmlsitemap or the robotstxt module has this feature already, but I don't have first-hand experience setting this up myself (I should check it out though: it sounds useful for my site!)

Production build 0.71.5 2024