cache problem after updated to 10.4.0

I don't understand this bug report very well as written. Is it possibly something to do with 🐛 Individual admin pages no longer accessible after update to 10.4 Active ?

I am having the same issue. I am including my log of the warning.

User warning: Trying to overwrite a cache redirect with one that has nothing in common, old one at address "languages:language_interface, theme, user.permissions, route.name.is_layout_builder_ui" was pointing to "context_stack:view:9449f4ef8e50a89825a4445c64e93b4a", new one points to "context_stack:view:ee1d86660efff497e0d4f88f265e687e". in Drupal\Core\Cache\VariationCache->set() (line 138 of .../web/core/lib/Drupal/Core/Cache/VariationCache.php)

#0 .../web/core/includes/bootstrap.inc(166): _drupal_error_handler_real(512, 'Trying to overw...', '/home/xxx...', 138)
#1 [internal function]: _drupal_error_handler(512, 'Trying to overw...', '/home/xxx...', 138)
#2 .../web/core/lib/Drupal/Core/Cache/VariationCache.php(138): trigger_error('Trying to overw...', 512)
#3 .../web/core/lib/Drupal/Core/Render/RenderCache.php(92): Drupal\Core\Cache\VariationCache->set(Array, Array, Object(Drupal\Core\Cache\CacheableMetadata), Object(Drupal\Core\Cache\CacheableMetadata))
#4 .../web/core/lib/Drupal/Core/Render/PlaceholderingRenderCache.php(127): Drupal\Core\Render\RenderCache->set(Array, Array)
#5 .../web/core/lib/Drupal/Core/Render/Renderer.php(584): Drupal\Core\Render\PlaceholderingRenderCache->set(Array, Array)
#6 .../web/core/lib/Drupal/Core/Render/Renderer.php(504): Drupal\Core\Render\Renderer->doRender(Array)
#7 .../web/core/lib/Drupal/Core/Render/Renderer.php(248): Drupal\Core\Render\Renderer->doRender(Array, false)
#8 .../web/core/lib/Drupal/Core/Template/TwigExtension.php(484): Drupal\Core\Render\Renderer->render(Array)
#9 .../web/sites/default/files/php/twig/6765f41a5db7e_page.html.twig_vgqHdACnoXdk7NRozFtijxvjO/LvotwseHwWQoOqxChHzF_p4uRLXDE_IdFmPB06YFbLY.php(329): Drupal\Core\Template\TwigExtension->escapeFilter(Object(Drupal\Core\Template\TwigEnvironment), Array, 'html', NULL, true)
#10 .../vendor/twig/twig/src/Template.php(387): __TwigTemplate_480925317b68064ddd31e111b10d8f9c->doDisplay(Array, Array)
#11 .../vendor/twig/twig/src/Template.php(343): Twig\Template->yield(Array, Array)
#12 .../vendor/twig/twig/src/Template.php(358): Twig\Template->display(Array)
#13 .../vendor/twig/twig/src/TemplateWrapper.php(35): Twig\Template->render(Array)
#14 .../web/core/themes/engines/twig/twig.engine(33): Twig\TemplateWrapper->render(Array)
#15 .../web/core/lib/Drupal/Core/Theme/ThemeManager.php(348): twig_render_template('themes/custom/n...', Array)
#16 .../web/core/lib/Drupal/Core/Render/Renderer.php(491): Drupal\Core\Theme\ThemeManager->render('page', Array)
#17 .../web/core/lib/Drupal/Core/Render/Renderer.php(248): Drupal\Core\Render\Renderer->doRender(Array, false)
#18 .../web/core/lib/Drupal/Core/Template/TwigExtension.php(484): Drupal\Core\Render\Renderer->render(Array)
#19 .../web/sites/default/files/php/twig/6765f41a5db7e_html.html.twig_O2cP37t0XjrjP--x4EKWg3wIm/-Y1_TmydklAhduRCKolylSEwdglGn2phDpZH-fgNzig.php(79): Drupal\Core\Template\TwigExtension->escapeFilter(Object(Drupal\Core\Template\TwigEnvironment), Array, 'html', NULL, true)
#20 .../vendor/twig/twig/src/Template.php(387): __TwigTemplate_cc8cca55ec5f9b87f8c4ee0fae94bf67->doDisplay(Array, Array)
#21 .../vendor/twig/twig/src/Template.php(343): Twig\Template->yield(Array, Array)
#22 .../vendor/twig/twig/src/Template.php(358): Twig\Template->display(Array)
#23 .../vendor/twig/twig/src/TemplateWrapper.php(35): Twig\Template->render(Array)
#24 .../web/core/themes/engines/twig/twig.engine(33): Twig\TemplateWrapper->render(Array)
#25 .../web/core/lib/Drupal/Core/Theme/ThemeManager.php(348): twig_render_template('themes/custom/n...', Array)
#26 .../web/core/lib/Drupal/Core/Render/Renderer.php(491): Drupal\Core\Theme\ThemeManager->render('html', Array)
#27 .../web/core/lib/Drupal/Core/Render/Renderer.php(248): Drupal\Core\Render\Renderer->doRender(Array, false)
#28 .../web/core/lib/Drupal/Core/Render/MainContent/HtmlRenderer.php(158): Drupal\Core\Render\Renderer->render(Array)
#29 .../web/core/lib/Drupal/Core/Render/Renderer.php(638): Drupal\Core\Render\MainContent\HtmlRenderer->Drupal\Core\Render\MainContent\{closure}()
#30 .../web/core/lib/Drupal/Core/Render/MainContent/HtmlRenderer.php(153): Drupal\Core\Render\Renderer->executeInRenderContext(Object(Drupal\Core\Render\RenderContext), Object(Closure))
#31 .../web/core/lib/Drupal/Core/EventSubscriber/MainContentViewSubscriber.php(90): Drupal\Core\Render\MainContent\HtmlRenderer->renderResponse(Array, Object(Symfony\Component\HttpFoundation\Request), Object(Drupal\Core\Routing\CurrentRouteMatch))
#32 [internal function]: Drupal\Core\EventSubscriber\MainContentViewSubscriber->onViewRenderArray(Object(Symfony\Component\HttpKernel\Event\ViewEvent), 'kernel.view', Object(Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher))
#33 .../web/core/lib/Drupal/Component/EventDispatcher/ContainerAwareEventDispatcher.php(111): call_user_func(Array, Object(Symfony\Component\HttpKernel\Event\ViewEvent), 'kernel.view', Object(Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher))
#34 .../vendor/symfony/http-kernel/HttpKernel.php(186): Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch(Object(Symfony\Component\HttpKernel\Event\ViewEvent), 'kernel.view')
#35 .../vendor/symfony/http-kernel/HttpKernel.php(76): Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object(Symfony\Component\HttpFoundation\Request), 1)
#36 .../web/core/lib/Drupal/Core/StackMiddleware/Session.php(53): Symfony\Component\HttpKernel\HttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#37 .../web/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(48): Drupal\Core\StackMiddleware\Session->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#38 .../web/core/lib/Drupal/Core/StackMiddleware/ContentLength.php(28): Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#39 .../web/core/modules/big_pipe/src/StackMiddleware/ContentLength.php(32): Drupal\Core\StackMiddleware\ContentLength->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#40 .../web/core/modules/page_cache/src/StackMiddleware/PageCache.php(201): Drupal\big_pipe\StackMiddleware\ContentLength->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#41 .../web/core/modules/page_cache/src/StackMiddleware/PageCache.php(138): Drupal\page_cache\StackMiddleware\PageCache->fetch(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#42 .../web/core/modules/page_cache/src/StackMiddleware/PageCache.php(87): Drupal\page_cache\StackMiddleware\PageCache->lookup(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#43 .../web/core/modules/ban/src/BanMiddleware.php(50): Drupal\page_cache\StackMiddleware\PageCache->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#44 .../web/modules/contrib/crowdsec/src/Middleware.php(96): Drupal\ban\BanMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#45 .../web/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(48): Drupal\crowdsec\Middleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#46 .../web/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(51): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#47 .../web/core/lib/Drupal/Core/StackMiddleware/AjaxPageState.php(36): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#48 .../web/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(51): Drupal\Core\StackMiddleware\AjaxPageState->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#49 .../web/core/lib/Drupal/Core/DrupalKernel.php(741): Drupal\Core\StackMiddleware\StackedHttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true)
#50 .../web/index.php(19): Drupal\Core\DrupalKernel->handle(Object(Symfony\Component\HttpFoundation\Request))
#51 {main}

Your backtrace shows a cache key with context_stack, which looks like it's from https://www.drupal.org/project/context_stack → which has a large notice on its homepage recommending to uninstall it.

Can you try uninstalling that module and see if the problem persists?

I'm seeing similar, but do not have the context_stack module.

User warning: Trying to overwrite a cache redirect with one that has nothing in common, old one at address "languages:language_interface, theme, user.permissions" was pointing to "url.path.parent, url.path.is_front, route", new one points to "url.path". in Drupal\Core\Cache\VariationCache->set() (line 138 of core/lib/Drupal/Core/Cache/VariationCache.php).

Drupal\Core\Cache\VariationCache->set() (Line: 92)
Drupal\Core\Render\RenderCache->set() (Line: 127)
Drupal\Core\Render\PlaceholderingRenderCache->set() (Line: 584)
Drupal\Core\Render\Renderer->doRender() (Line: 504)
Drupal\Core\Render\Renderer->doRender() (Line: 248)
Drupal\Core\Render\Renderer->render() (Line: 484)
Drupal\Core\Template\TwigExtension->escapeFilter() (Line: 135)
__TwigTemplate_7933b1592f1978df929103e8f2b9cd6d->doDisplay() (Line: 387)
Twig\Template->yield() (Line: 343)
Twig\Template->display() (Line: 358)
Twig\Template->render() (Line: 35)
Twig\TemplateWrapper->render() (Line: 33)
twig_render_template() (Line: 348)
Drupal\Core\Theme\ThemeManager->render() (Line: 491)
Drupal\Core\Render\Renderer->doRender() (Line: 248)
Drupal\Core\Render\Renderer->render() (Line: 484)
Drupal\Core\Template\TwigExtension->escapeFilter() (Line: 98)
__TwigTemplate_036e62bb102d1f6e378f9094f7809ad1->doDisplay() (Line: 387)
Twig\Template->yield() (Line: 343)
Twig\Template->display() (Line: 358)
Twig\Template->render() (Line: 35)
Twig\TemplateWrapper->render() (Line: 33)
twig_render_template() (Line: 348)
Drupal\Core\Theme\ThemeManager->render() (Line: 491)
Drupal\Core\Render\Renderer->doRender() (Line: 248)
Drupal\Core\Render\Renderer->render() (Line: 158)
Drupal\Core\Render\MainContent\HtmlRenderer->Drupal\Core\Render\MainContent\{closure}() (Line: 638)
Drupal\Core\Render\Renderer->executeInRenderContext() (Line: 153)
Drupal\Core\Render\MainContent\HtmlRenderer->renderResponse() (Line: 90)
Drupal\Core\EventSubscriber\MainContentViewSubscriber->onViewRenderArray()
call_user_func() (Line: 111)
Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch() (Line: 186)
Symfony\Component\HttpKernel\HttpKernel->handleRaw() (Line: 76)
Symfony\Component\HttpKernel\HttpKernel->handle() (Line: 53)
Drupal\Core\StackMiddleware\Session->handle() (Line: 48)
Drupal\Core\StackMiddleware\KernelPreHandle->handle() (Line: 28)
Drupal\Core\StackMiddleware\ContentLength->handle() (Line: 32)
Drupal\big_pipe\StackMiddleware\ContentLength->handle() (Line: 50)
Drupal\ban\BanMiddleware->handle() (Line: 48)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle() (Line: 51)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle() (Line: 36)
Drupal\Core\StackMiddleware\AjaxPageState->handle() (Line: 51)
Drupal\Core\StackMiddleware\StackedHttpKernel->handle() (Line: 741)
Drupal\Core\DrupalKernel->handle() (Line: 19)

Also hoping to track this one down. The author of 🐛 VariationCache needs to be more defensive about cache context manipulation to avoid broken redirects Fixed says:

...if a longstanding bugfix ( 📌 Cacheability information from route access checker access results are ignored by dynamic_page_cache Needs review ) gets in we might all of a sudden see bug reports and perhaps even security issues...

So looks like it's come to fruition? Thanks for the tip above about finding the potentially offending module's name in the stack. In my case I guess it's either `url.path` or `route.menu_active_trails`. Will update if I figure something out :)

I am also seeing this:

Error message: User warning: Trying to overwrite a cache redirect with one that has nothing in common, old one at address "languages:language_interface, theme, user.permissions" was pointing to "url.path.parent, url.path.is_front, route", new one points to "url". in Drupal\Core\Cache\VariationCache->set() (line 138 of core/lib/Drupal/Core/Cache/VariationCache.php).

... randomly in a fresh Drupal 10.4 install for CiviCRM.

$ drush pm:list --no-core
 ---------------- ----------------------------------------------------------------------- ---------- ------------- 
  Package          Name                                                                    Status     Version      
 ---------------- ----------------------------------------------------------------------- ---------- ------------- 
  Administration   Admin Toolbar (admin_toolbar)                                           Enabled    3.5.1        
  Administration   Admin Toolbar Links Access Filter (admin_toolbar_links_access_filter)   Disabled   3.5.1        
  Administration   Admin Toolbar Search (admin_toolbar_search)                             Disabled   3.5.1        
  Administration   Admin Toolbar Extra Tools (admin_toolbar_tools)                         Disabled   3.5.1        
  CiviCRM          CiviCRM Core (civicrm)                                                  Enabled                 
  CiviCRM          CiviCRM Theme (civicrmtheme)                                            Enabled                 
  CiviCRM          CiviCRM Entity (civicrm_entity)                                         Enabled    4.0.0-beta4  
  Core             Claro Compact (claro_compact)                                           Enabled    8.x-2.1      
 ---------------- ----------------------------------------------------------------------- ---------- ------------- 

I think there are one or two possible bugs in the logic throwing the exception.

From @ressa's example:

"languages:language_interface, theme, user.permissions" was pointing to "url.path.parent, url.path.is_front, route", new one points to "url". 

For render caching, we need to take into required cache contexts which aren't always explicitly set on the render array.

We don't want to do that in VariationCache itself, so we'd need to add them in before they're compared here.

I also think we might need to do something with CacheContextsManager::optimizeTokens() here for the situation where we're comparing url vs. url.query_args. Not sure exactly how it would work though, something like optimize both contexts, then merge the contexts, then optimize the result of that and compare before/after the final optimization and if there's a difference, don't throw the exception.

Any fixes will need to happen on the 11.x branch and be backported.

Put up an MR that removes the exception altogether.

We'll need a follow-up to add it back with some or all of the logic in #8, I also think it needs to be a trigger_error() until 12.x rather than an exception. Critical because this is user facing. I'm pretty sure #3 is an example of the exception working as it should though.

token tests did also run into a pretty weird version of this that caused test fails, but only tests are affected there: 🐛 Fixing test fails on 11.1 Active

Wait a minute no-one's triggering the exception here, only the warning, so my MR is useless.

Update the MR so it just adds the cache ID to the warning message, this will give us a better idea of what's going on.

Reading the comment here, I think I might also be wrong about CacheContextsManager::optimize() since the comment explicitly says this:

  // This usually occurs when using calculated cache contexts and the
            // author tried to manually optimize them. E.g.: When using
            // user.roles:anonymous and in one of the outcomes we end up varying
            // by user.roles. In that case, both user.roles:anonymous and
            // user.roles need to be present on the cacheable metadata, even
            // though they will eventually be optimized into user.roles. The
            // cache needs all the initial information to do its job and if an
            // author were to manually optimize this prematurely, it would be
            // impossible to properly store a redirect chain.

So let's get some more information about where this is happening.

Let's just improve the error message here, then open a follow-up to track cases of this that come up.

Wonder if steps and issue summary update could still happen please.

Testing 10.4.0 causes the same issue for us, and several projects currently on 10.3.x are impacted. Reverting to 10.3.10 resolves it. We can not proceed with the 10.4 upgrade until this is fixed.

Probably more people like myself, but I had custom code that had to be adjusted much like in book.module:

https://git.drupalcode.org/project/book/-/merge_requests/15/diffs

Hope that helps

Yes, the most common one for this are breadcrumb builders, happened in a number of projects.

I'm also not sure that throwing an exception is a good idea, but:

> We can not proceed with the 10.4 upgrade until this is fixed.

I assume you hit the exception, not the warning. This won't be "fixed". At best it will change the exception to an error or warning. The reason an exception was chosen is that current breadcrumb builders result in cache bugs. But I agree that an exception is too strong for that in a minor update, it should be a deprecation or maybe warning/error that gets logged but doesn't break the site.

> How can we help with troubleshooting?

If you get the warning, try the patch to get a better warning message that should help you identify what thing is unhappy.

If you get the exception, the best way forward is to fix it now, look for breadcrumb builders in contrib or custom projects, if none then you'll need to figure out what else is triggering this.

With the "add cid" patch in #15 I see the erroneous cid in my case is entity_view:block:****_breadcrumbs:[languages:language_interface]=en:[theme]=*****:[user.permissions]=is-admin. I updated the menu_breadcrumbs to latest 2.0.x-dev, and the message appears to have gone away. If I get to narrow down which of the latest dev commits actually fixed it, I'll update this comment.

In our case, the menu_breadcrumb was also involved. Thank you texas-bronious!.

In addition to deploying the latest 2.0.x-dev, I had to apply this patch too: https://www.drupal.org/project/menu_breadcrumb/issues/3495164 🐛 Missing route cachability metadata. Active

Then the issue was resolved.

I hope this helps someone out there.

As previously pointed out by others, breadcrumbs were a big offender in core and contrib and the linked fix for the Book module shows exactly how to alleviate the issue.

On how to improve the message: I actually had the CID in there before but chose to remove that to keep the message shorter. I am not at all opposed to adding back the CID as I agree it can help debugging.

MR is RTBC for me, will update IS a bit. Keep in mind the goal is to turn the E_USER_WARNING into an exception in future major versions of Drupal. Once ✨ Add a Production/Development Toggle To Core Needs work lands, we can already convert the warning into an exception for dev sites.

Committed and pushed 25b6571b63e to 11.x and c19f0d7139e to 11.1.x and ea9093d961d to 10.5.x and 0e3d2d669de to 10.4.x. Thanks!

Thanks this patch is great, and you get actually useful hints to the culprit now. Again, it was breadcrumb-related:

User warning: Trying to overwrite a cache redirect for "entity_view:block:claro_compact_breadcrumbs:[languages:language_interface]=en:[theme]=claro_compact:[user.permissions]=is-admin" with one that has nothing in common, old one at address "languages:language_interface, theme, user.permissions" was pointing to "url.path.parent, url.path.is_front, route", new one points to "url". in Drupal\Core\Cache\VariationCache->set() (line 138 of core/lib/Drupal/Core/Cache/VariationCache.php).

I created 🐛 Missing route cachability metadata in Claro Compact Active .

Adding a related issue, which contains a solution.

Really nice to see that this is uncovering and helping to fix bugs a lot faster, thanks for following up with the links!