Exclude content with a "noindex" robots meta tag

One thing maybe to consider is also checking rewrites/redirects. In my current setup I have some URLs in the sitemap leading to nodes having noindex set. Is there reasonable any way to avoid that?

This patch is working for v4.1.2. I updated the return statement for the getMetatagFieldFor method which was given a PHP warning for content types that don't have any field of type "metatag".

@camvertiz : Why does your match contain much less code than the Merge request (https://git.drupalcode.org/project/simple_sitemap/-/merge_requests/52.patch)?

Find a patch attached, that applies against 4.1.6

The last submitted patch, 62: simple_sitemap-add_noindex_support-3080541-62.patch, failed testing. View results →
- codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

Ups, forgot something... next try.

Resolved Fatal error: Cannot redeclare simple_sitemap_update_8407() (previously declared modules/contrib/simple_sitemap/
simple_sitemap.install:948)

Metatag 2.x now uses JSON data, so the unserialize needs to be changed to json_decode if you want this to work with Metatag 2.x. This is exactly why this should probably be in its own module.

Given the wide ranging code changes, the performance hit, metatag's flaky API and given that several people agree this should be worked on in a 3rd party glue module, I'm closing this issue. Anyone interested please create a sandbox and let us know in this here issue. Thanks for all of your input!

Rerolling the patch from #67 without the double reference to $entityFieldManager;

Last patch still had a duplicate function in it.

I rerolled #71 for a setup with metatags v2 replacing the unserialize() calls by json_decode().

Rerolled #72 for simple_sitemap 4.1.9 fixing the issue

PHP Fatal error: Cannot redeclare simple_sitemap_update_8408() (previously declared in /var/www/html/app/web/modules/contrib/simple_sitemap/simple_sitemap.install:942) in /var/www/html/app/web/modules/contrib/simple_sitemap/simple_sitemap.install on line 967

Rerolled #71 for simple_sitemap 4.1.9 fixing the issue

PHP Fatal error:  Cannot redeclare simple_sitemap_update_8408() (previously declared in /var/www/html/app/web/modules/contrib/simple_sitemap/simple_sitemap.install:942) in /var/www/html/app/web/modules/contrib/simple_sitemap/simple_sitemap.install on line 967

WalkingDexter → closed merge request !9

WalkingDexter → closed merge request !23

WalkingDexter → closed merge request !52

Rerolled #73 for simple_sitemap 4.2.1

Rerolled #79 for simple_sitemap 4.2.2

The reroll in #80 is missing the SimplesitemapMetatagTest from #79

Rerolling #79 for simple_sitemap 4.2.2 to also include the new tests SimplesitemapMetatagTest.

Just a small contribution, we faced this issue and I wasn't comfortable with applying a patch.

If you have Cloudflare in front of your site, this would be a good option:

https://developers.cloudflare.com/rules/transform/request-header-modific...

Just add this snippet in a custom module:

/**
 * Implements hook_simple_sitemap_links_alter().
 */
function YOUR_MODULE_simple_sitemap_links_alter(&$links, \Drupal\simple_sitemap\Entity\Simplesitemap $sitemap): void {
  foreach ($links as $key => $link) {
    if (isset($link['meta']['entity_info']) && $link['meta']['entity_info']['entity_type'] == 'node') {
      $node = \Drupal\node\Entity\Node::load($link['meta']['entity_info']['id']);
      if ($node && $node->hasField('field_metatags')) {
        $metatags = $node->get('field_metatags');
        if ($metatags && str_contains($metatags->value, 'noindex')) {
          unset($links[$key]);
        }
      }
    }
  }
}