Make compatible with simple_sitemap 4.0 & domain_entity

Ran into issue mentioned by @betarobot in #29. Looks like cache ID /sitemap.xml does not take into consideration the domain. Quick workaround is to disable route cache for /sitemap.xml. Better would be to dig into how Domain handles this so we can get /sitemap.xml cached per domain.

Got same issue as in #29. `/sitemap.xml` page gets cached and is served for other domains.
Instead on disabling `/sitemap.xml` page cache completely as it is done in #30, added `url` cache context that means result will be cached for every url variation.

Patch is based on #20.

Hello there, 8.x-2.0-beta1 was released and the patch #31 doesn't apply. I've re-rolled it :)

Thanks @nachosalvador! Worked fine for me with 8.x-2.0-beta1.

I have encountered two issues while testing. I am using:

Domain Entity - 1.0.0-rc1
Domain Simple Sitemap - 2.0.0-beta1
Country Path - latest dev-1.x

Issue 1:
Entities using the domain_entity field that have no domains checked were not showing in the sitemap. On the domain_entity field, if no domains are checked, then the content is considered affiliated to all domains.

The domain_entity module alters queries to add conditions to determine whether or not an entity is allowed for a domain. That logic is not accounted for in this issue's patches. The patch only checks if the entity was explicitly affiliated to the domain with:

$query->condition(\Drupal\domain_entity\DomainEntityMapper::FIELD_NAME, $sitemap_domain_id);

I made a change that lets the domain_entity module handle this logic with its hook_query_alter() code.

Issue 2:
Patch #19 changes the way URLs are replaced when using the Country Path module → . The change in #19 resulted in URLs with 2 country prefixes (e.g https://example.com/us/us/en/product/1 instead of https://example.com/us/en/product/1).
I have reverted that change.

milos.kroulik → made their first commit to this issue’s fork.

Patch provided on #24 works as expected for me.

However, the Sitemap listing on admin/config/search/simplesitemap does not link properly to the sitemaps. I had to manually access the URL: /{sitemap_machine_name}/sitemap.xml to check the generated sitemap.xml

If I just click on the links, I get a 404 page not found.

Moving to "Needs work" to address the incorrect links on the listing page.

Adding a version of #34 combined with Drupal 10 fixes as the patch provided on https://www.drupal.org/project/domain_simple_sitemap/issues/3296977 📌 Automated Drupal 10 compatibility fixes Closed: duplicate can't apply with the #34.

I started with the branch /u/sonvir249 but did a bit of refactoring to bring this up to date. The changes are in https://git.drupalcode.org/project/domain_simple_sitemap/-/merge_request... and the diff can be seen there, but a key difference in the approach I took is that I did not set a "Simple Sitemap Type" entity for each individual Domain entity.

Instead, this MR utilizes a singular Domain "Simple Sitemap Type" entity, and allows each individual Domain to have their own "Simple Sitemap" entities of this type. Prior to this change, I was seeing both "Simple Sitemap" and "Simple Sitemap Type" entities for every Domain, and I couldn't tell what benefit that offered.

Also, I included an update hook to correspond with simple_sitemap_update_8403() (see simple_sitemap.install), which automatically creates a Domain simple_sitemap_type and converts existing Domain variants to new Domain "Simple Sitemap" entities.

I did end up patching Simple Sitemap to eliminate this line https://git.drupalcode.org/project/simple_sitemap/-/blob/4.x/simple_site..., which was deleting all simple_sitemap.variants.* config.

the getpath functions also needs to be changed in drupal 10, i edited it and tested it and for me its works

Fixed a call to setSitemaps which doesn't exist which killed the batching. Other than that this seems to be working ok for now

Some changes that I'm sharing that we needed internally, but not sure if they should be added to the MR and didn't want to muddy the waters more with more MRs.

This changes the URL generation so it works with Domain aliases again, something like what was done in #19 but hopefully without the side effects
And it restores the use of "Use node source instead of node access as filter" which doesn't do anything when turned on using the code in the MR.

If others feel these are changes we want to add, feel free to add them.

The last patch " https://www.drupal.org/files/issues/2023-08-04/3251227-make-compatible-w... → " didn't work properly for me, it still said
The "simple_sitemap" entity type does not exist.

but then i apllied the plain diff of the MR12 and it worked
https://git.drupalcode.org/project/domain_simple_sitemap/-/merge_request...

Hello everyone and thanks for the module, I am bit confused on how to make this update work, right now composer has all this versions available :
found drupal/domain_simple_sitemap[dev-8.x-1.x, dev-8.x-2.x, dev-3251227-make-compatible-with, dev-3251227-simple_sitemap-4x-compatibility-combined-with-D10, dev-3251227-simple_sitemap-4x-compatibility, dev-3251227-make-compatible-40, dev-3296977-automated-drupal-10-with-3251227, 2.0-RC1-3251227, 2.0-RC2-3251227, 2.0-RC3-3251227, 2.0-RC4-3251227]

I have tried 2.0-RC4-3251227 with simple_sitemap 416 but when I try Bulk creating sitemap variants for domain I get :
TypeError: Drupal\simple_sitemap\Entity\SimpleSitemap::getType(): Return value must be of type Drupal\simple_sitemap\Entity\SimpleSitemapTypeInterface, null returned in Drupal\simple_sitemap\Entity\SimpleSitemap->getType() (line 131 of /var/www/html/web/modules/contrib/simple_sitemap/src/Entity/SimpleSitemap.php).

Am I using the right branch ? Should I apply I patch on top of it (none seem to apply on 2.0-RC4-3251227...)? I am a bit confused...

Hi,
I'm still getting the "simple_sitemap" entity type does not exist error when trying the last → patch. I've tried other patches, applying the plain diff of the MR12 (as another user suggested) but still getting the same error.

Anyone any suggestions on what to try next?
I cannot find where the error is triggered. Could this indicate on a problem in the database?

Hi, everyone

I have created a new patch to fix issues, can you check if it's working for you?

Patch #51 → works for me. thanks!

I updated patch to make sitemap.xml for different domains, please, check

Thanks, is this meant to work with Domain v1 or 2? I get this when trying to require with composer. Am I missing something?

Your requirements could not be resolved to an installable set of packages.

Problem 1
- drupal/domain_simple_sitemap 1.0.0-beta2 requires drupal/domain ^1.0 -> found drupal/domain[1.0.0-beta1, ..., 1.0.0-beta8] but it conflicts with your root composer.json require (^2.0@beta).
- drupal/domain_simple_sitemap[dev-1.x, 1.0.0-beta3, ..., 1.x-dev] require drupal/core ~8.0 -> found drupal/core[8.0.0-beta6, ..., 8.9.x-dev] but the package is fixed to 10.1.4 (lock file version) by a partial update and that version does not match. Make sure you list it as an argument for the update command.
- drupal/domain_simple_sitemap[1.0.0-beta4, ..., 1.0.0-beta6] require drupal/core ^8 -> found drupal/core[8.0.0-beta6, ..., 8.9.x-dev] but the package is fixed to 10.1.4 (lock file version) by a partial update and that version does not match. Make sure you list it as an argument for the update command.
- drupal/domain_simple_sitemap[dev-2.x, 2.0.0-beta1, ..., 2.x-dev] require drupal/core ^8 || ^9 -> found drupal/core[8.0.0-beta6, ..., 8.9.x-dev, 9.0.0-alpha1, ..., 9.5.x-dev] but the package is fixed to 10.1.4 (lock file version) by a partial update and that version does not match. Make sure you list it as an argument for the update command.
- Root composer.json requires drupal/domain_simple_sitemap * -> satisfiable by drupal/domain_simple_sitemap[dev-1.x, dev-2.x, 1.0.0-beta2, ..., 1.x-dev, 2.0.0-beta1, 2.x-dev].

#53 worked great for me, finally. Thanks!

Hi @betarobot,

Im curious what version of core you are using and what steps you took to install with composer?

As mentioned above, I cannot get this installed to run the patch due to composer dependency issues.

Thanks.

I ended up following steps outlined in #2 to require this issue branch project directly. We have not tried the recent patches in #53

Had to use version 8.x-2.x-dev to get the patch to apply.

composer require 'drupal/domain_simple_sitemap:8.x-2.x-dev@dev' -W
composer config --json --merge extra.patches.drupal/domain_simple_sitemap '{"3251227 - Make compatible with simple_sitemap 4.0": "https://www.drupal.org/files/issues/2023-10-06/3251227-make-compatible-with-simple-sitemap-combined-with-D10-fixes-53.patch"}'

Thanks for your comment @the_g_bomb, are you using D9 or 10?

When running
composer require 'drupal/domain_simple_sitemap:2.x-dev@dev' -W

I get the below error, as 8.x-2.x-dev is still D8 || 9, not D10
https://www.drupal.org/project/domain_simple_sitemap/releases/8.x-2.x-dev →

-----
Your requirements could not be resolved to an installable set of packages.
Problem 1
- drupal/domain_simple_sitemap dev-2.x requires drupal/core ^8 || ^9 -> found drupal/core[8.0.0-beta6, ..., 8.9.x-dev, 9.0.0-alpha1, ..., 9.5.x-dev] but these were not loaded, likely because it conflicts with another require.
- drupal/domain_simple_sitemap 2.x-dev is an alias of drupal/domain_simple_sitemap dev-2.x and thus requires it to be installed too.
- Root composer.json requires drupal/domain_simple_sitemap 2.x-dev@dev -> satisfiable by drupal/domain_simple_sitemap[2.x-dev (alias of dev-2.x)].
----

Im clearly doing something wrong and dont have the knowledge of how to require a module and also apply a patch to that module that has requirements that dont meet my core.

My logic is
Required the module, then patch it

But the step of requiring it in the first place isnt possible, as core version of 2xdev is set to 8 || 9.

So what is the process to include a module, but also apply a patch to that module that will change the core version to meet your requirement?

Any advice here would be really appreciated and I apologise if this is a simple step that I just dont have the knowledge of and havent done before.

Thank you to anyway that can help.

@lindsay.wils it's unrelated, however you can use alias for core (e.g. "drupal/core": "10.1.4 as 9.5.11") or https://github.com/mglaman/composer-drupal-lenient.

Hi, I managed to install the D10 compatible version using the patches mentioned.
But I still experience a very important issue.
I have a site with 4 domains.

Reproduce:
- Create 4 domains
- Create 4 domain sitemap types for each domain
- Create 4 domain sitemaps & generate

When I check, my first sitemap is always correct, no matter which order I drag the sitemaps.
But after the first sitemap, all of the next sitemaps will also filter the urls on the domain of the first sitemap.

I've been looking into this problem now for few days, starting to look double now :D
Anyone else with this issue? And even better, with a solution?

@lindsay.wils. Hmm you may be right that there needs to be a slightly different process required for D9 and D10.
I think I used D9 initially, but was also attempting the update to D10.
In D10 you may need the lenient or try the method Alex has suggested.
You could try:

composer require mglaman/composer-drupal-lenient
composer config --merge --json extra.drupal-lenient.allowed-list '["drupal/domain_simple_sitemap"]'
composer require 'drupal/domain_simple_sitemap:8.x-2.x-dev@dev' -W
composer config --json --merge extra.patches.drupal/domain_simple_sitemap '{"3251227 - Make compatible with simple_sitemap 4.0": "https://www.drupal.org/files/issues/2023-10-06/3251227-make-compatible-with-simple-sitemap-combined-with-D10-fixes-53.patch"}'

Thanks for your comment #61 @segiergiancarlo

I can confirm that I have faced the issue that sitemap all sitemaps for other domains than on which the cron is running (of default domain if you run cron via drush) includes the filter of that domain. So it contains only links to content where both default domain and subdomain was checked.

I have found out that the problem makes that I have in the sitemaps (/admin/config/search/simplesitemap) also Default hreflang sitemap type. When I removed it and keep there only sitemaps for each domain, than the sitemaps are generated now correctly. Can you also tried if this is not your problem, or it fix the problem for you?

I have Drupal 9.5.11, Simple XML sitemap 4.1.6, Domain Access Simple XML Sitemap 8.x-2.0-beta1 with this #53 path:
https://www.drupal.org/files/issues/2023-10-06/3251227-make-compatible-w... →

Outsite of "bug" mentioned above I can confirm that the patch is working correctly.

I'm trying the latest patch (#53), and I'm wondering... am I supposed to have to recreate all our sitemaps?

(I'm on Drupal 9.5.11)

What I've done so far:

1. Add the patch in #53 (in our case, that means adding it to composer.patches.json)
2. Update modules:
   • drupal/domain (1.0.0-beta8 => 2.0.0-beta1)
   • drupal/domain_simple_sitemap (dev-2.x 198f364 => 2.0.0-beta1)
   • drupal/simple_sitemap (3.11.0 => 4.1.7)
3. Run DB updates
4. At that point, I saw no sitemaps or sitemap types.
5. Then, I did "generate domain's sitemap variants" here: /admin/config/domain/domain_simple_sitemap/config
6. Now, I see a "sitemap type" for all my domains.
7. Still there's only the "Sitemap Index" sitemap here: /admin/config/search/simplesitemap.
8. I tried clicking "rebuild queue" and "rebuild queue & generate" on that same admin page -- no dice. (Logs say "The XML sitemaps have been regenerated." and there are no errors.)

Do I need to recreate them all...? -- if yes, okay, but obviously I wanted to check first! (and obviously I'm hoping I'm mistaken 🤞🤞)

FWIW, we were running a pretty old version of domain_simple_sitemap dev-2.x -- I mean, it's still the 2.x branch, but maybe we missed an upgrade path along the way between dev-2.x 198f364 and 2.0.0-beta1...?

-------

P.S. Specific steps I took for updating the modules, in case it matters:

$ composer require drupal/domain:^2.0@beta drupal/simple_sitemap:^4.1 drupal/domain_simple_sitemap:^2.0@beta --no-update
$ composer update drupal/domain drupal/simple_sitemap drupal/domain_simple_sitemap

In a development environment -- specifically, a Pantheon multidev...

I tried manually recreating the default sitemap for this site, and sitemaps for two of the many (many) domains configured for this site, then I generated them.

1. Previously (see comment #64): I did "generate domain's sitemap variants" here: /admin/config/domain/domain_simple_sitemap/config -- and then I saw "sitemap types" for each of my domains.
2. Then (today), I added new sitemaps (one for the default domain, and two others) -- from here: /admin/config/search/simplesitemap
3. Then, I went to "Inclusion," and configured the "content" entity type to match the settings we have on prod (including/excluding various content types): /admin/config/search/simplesitemap/entities
4. Then, based on comment #63, I deleted the "Default hreflang sitemap" sitemap type.
5. Then, I queued/generated the sitemaps, here: /admin/config/search/simplesitemap

And, they did get generated, or, I think they did? There are WAY more links in each sitemap than how many links are in the sitemaps on the live site, and also, the sitemap links go to the prod URLs -- like, I see the three sitemaps I created when I go here (plus "Sitemap index" which is "disabled"):
/admin/config/search/simplesitemap
But the link counts for each are higher than the "max_links" setting (which is the default: 5000) -- and, each sitemap link goes to the prod domain URLs. Even if I manually navigate to the dev URLs, with "sitemap.xml" tacked onto the end, the links are prod URLs.

THAT SAID, I'm looking at another multidev, with the prior versions of domain/simple_sitemap/domain_simple_sitemap, and, while the sitemap links over there are the dev URLs, the links inside the sitemaps are all prod domain URLs -- so maybe that part is fine?? (I'm really hoping y'all have more experience with multiple domains + sitemaps than I do 🤞)

Even so, the number of links in the sitemaps is concerning, I think? (way higher than the live site, and the 5000 max setting) -- really, overall I just feel really unsure and apprehensive, like, if this is the kind of thing that simply doesn't quite work on a dev environment, I can mostly accept that, but I don't know what I should be seeing / I don't know how to know if things are going to work correctly on the live/prod site.
>> I did try the "remove_duplicates" checkbox, but after regenerating, the link counts per sitemap were still just as high.

And, I do still wonder if I'm supposed to have to rebuild everything manually, or if the sitemaps should've still existed after I upgraded the modules and ran database updates, y'know?

Sorry, I know I'm rambling, I just feel pretty lost 😖

-------
FWIW: I do know the dev subdomains are configured fine because I tested them after adding them to the Pantheon multidev -- I can get to the dev URLs for the two domains I configured on this environment, and I see the "test edits" to content that I made on that environment, when view content on these dev domains.

In a project using country path, domain and domain_language module, it took me so much time trying to get simple_sitemap module to work with this (domain_simple_sitemap) module, trying to generate sitemaps how I would like them to be that I suddenly realised: For this project it might be a whole lot easier to figure out how to generate a sitemap from scratch (creating a custom module) than to continue to try to get the simple_sitemap and the domain_simple_sitemap module working

So I did just that and I learned a lot and it was fixed in a very short amount of time.
Its very simple code now that I understand and as a bonus my custom module also generates a sitemap-styling.xsl file and adds it to the sitemap so the sitemap.xml file has great styling.

I'm just saying:
I hope issues get fixed so that domain_simple_sitemap works flawlessly.
For those that cannot get it working but you need working sitemaps for your project, it might be an option to see if you cant just create a custom module to generate your sitemaps instead of using the simple_sitemap and domain_simple_sitemap modules.

Thank you for the share @flyke! We might end up in that boat eventually, it's good to hear it was pretty doable for you.

-------

General update for the class:
It turns out, when I recreate the sitemaps (post upgrade), if I use the same machine name as what existed before the upgrade, the inclusion/bundle settings are right there when I go to /admin/config/search/simplesitemap/entities (more specifically: /admin/config/search/simplesitemap/entities/node -- I haven't tested any other entity types or custom links, we're only including nodes).

Soooooooo kinda uncomfortable to be honest, that the sitemaps were "gone" but the settings still "there" -- but I guess I'll take it??

Fixed URLs of non-default sitemaps at /admin/config/search/simplesitemap.

Hey all, has anyone got this working on D10 yet? We are set to launch the current project Dec 5th and trying to get a working solution.

I have followed instrctions provided by @the_g_bomb #62 (thanks)

But I still get core dependency issues

  Problem 1
    - drupal/domain_simple_sitemap dev-2.x requires drupal/core ^8 || ^9 -> found drupal/core[8.0.0-beta6, ..., 8.9.x-dev, 9.0.0-alpha1, ..., 9.5.x-dev] but the package is fixed to 10.1.6 (lock file version) by a partial update and that version does not match. Make sure you list it as an argument for the update command.
    - drupal/domain_simple_sitemap 2.x-dev is an alias of drupal/domain_simple_sitemap dev-2.x and thus requires it to be installed too.
    - Root composer.json requires drupal/domain_simple_sitemap 2.x-dev@dev -> satisfiable by drupal/domain_simple_sitemap[2.x-dev (alias of dev-2.x)].

My composer file includes this

"extra":{
 "drupal-lenient": {
            "allowed-list": ["drupal/domain_simple_sitemap"]
        }
}

really at a lose here to what can be done. Thanks for any advice or direction

Actually.. I am getting this error when running

composer require mglaman/composer-drupal-lenient

Anyone else experienced this?

[ParseError]
  syntax error, unexpected 'PackageRequiresAdjuster' (T_STRING), expecting function (T_FUNCTION) or const (T_CONST)

I'm going to commit #68 patch to 3.0.x branch

I'm going to commit #68 patch to 3.0.x branch

Committed and created a new version of the module - 3.0.0-beta1 → Let me know if you get any issues here.

Thank you very much for getting this released, appreciate the work.

I unfortunately am still having some issues as comment #65. I am using this latest version on D10.1.6

My sitemap on domain1, is returning lots of URLs from domain2, even though that node is set to only be available on domain2, using node access setting.

In the generation of the sitemap variants on this page
https://wsl.lndo.site/admin/config/domain/domain_simple_sitemap/config

I have tried both checking and not checking 'Use node source instead of node access as filter', rebuild, and get the same results.

I do not have domain source set on many nodes, only one content type, domain access is used throughout the site, but this seems to not be functioning as expected when generating the site maps.

Is anyone else seeing this?

Thank you.

Automatically closed - issue fixed for 2 weeks with no activity.

First off, nice work on getting support for Drupal 10 and Simple XML Sitemap!

Unfortunately, I'm really struggling to perform this update. After updating, I'm running into the same kind of issues as described in #65 and #75.
The main issue (for me at least) seems to be that the 'domain' sitemap type has been removed in favor of per-domain types and that the 'Generate domain sitemap variants' button on the config page generates new sitemap types but no sitemap variants.

Like #41 mentions, it seems a bit much to have a type and a variant for each domain. Are there any reasons why the code / direction from #41 was discontinued?

Any suggestions / insights on how to update would be greatly appreciated!

I am seeing the same issue mentioned above by lindsay.wils (comment #75).
Drupal 10.1.7
simple_sitemap 4.1.8
domain_simple_sitemap 3.0.0-beta1

Link count is very high for each domain because links from other domains are added in addition to the correct domain links.
Tried rebuild queue and reiteration again, but still same issue.
This is a blocker for our D10 upgrade.

Sorry to keep adding to a closed issue, but for the folks who commented earlier and anyone else who ends up here, removing the default "Entity URL Generator" from the domain specific sitemap types generated by domain simple sitemap got me the correct results. Make sure "Domain entity URL generator" is the only entity URL generator available in the domain specific sitemap types otherwise all entities from the default domain get added to all of the sitemaps for the other domains.

In my case I was only adding nodes as well as taxonomy terms assigned to a domain through the domain entity module → to the sitemaps.

This is with Simple XML Sitemap 4.1.8 and Domain Simple Sitemap 3.0.0-beta1.

Thanks very much jeni_dc, this comment helped me get this working finally after waiting months and months for a working solution. Very much appreciated.

Thank you jeni_dc, your solution in #79 solution worked for me as well. Updating all the sitemap types to only have a URL Generator type of "Domain entity URL generator" fixed all the issues I was running into. Running simple_sitemap 4.1.8 and domain_simple_sitemap 3.0.0-beta1 as well. Tested here with 7 different sitemaps.

The solution in #79 worked for me as well, thanks jeni_dc! If anyone else is wondering, you can leave any of the other URL generators selected just not any "Entity URL generator" (except for "Domain entity URL generator").

Thank you jeni_dc, we've been working arround this issue for weeks, you're a life savior =)
I have copied your solution on https://www.drupal.org/project/domain_simple_sitemap/issues/3216846#comm... 🐛 Doesn't generate nodes sitemap for other websites (Multisite) domain access. Only the first site. Needs review

Just to be clear, the idea is to create a sitemap and a sitemap type per domain to have a domain specific sitemap?

Thank you thank you thank you, @jeni_dc! I just spent a solid week wrestling with this and your comment helped me fix it. What a relief!