Missing language prefix in sitemap

Problem/Motivation

I'm using this domain_language module together with the simple_sitemap and domain_simple_sitemap modules.
My default domain has 2 languages enabled (nl & en), and another domains has 3 (nl, en & de).
Now when I generate the sitemap the "de" links in the non-default domain are missing their language prefix.
That is because this module removes the language prefix for links of languages that are not allowed on the domain.
It does that in the `/src/DomainLanguageOverrider.php` file.
Though that file, when generating the sitemap, assumes we're on the default domain. Though we're actually generating the sitemap for a non-default domain and would not expect that language prefix to have been removed.

I've looked through the code and I was able to fix this by disabling the language prefix removal, but I can imagine that will break other stuff.
For now I've just enabled the "de" language for the default domain. But it would be amazing if this could be fixed somehow.