Auto-placeholdering does not always happen if #lazy_builder did not specify cache keys

Created on 7 February 2024, 11 months ago
Updated 18 March 2024, 9 months ago

This is a theoretical bug at this stage, I have yet to write a test to confirm it. But from reading the code it sure seems to be the way I think it is.

Problem/Motivation

Assume the following render array:

$render = [
  'something_expensive' => [
    '#lazy_builder' => 'my_service:somethingExpensive',
    '#cache' => [
      'contexts' => ['foo'],
      'tags' => ['bar'],
    ],
  ],
];

Now imagine if my_service:somethingExpensive generated something that varies by the user cache context.

Normally, if something_expensive specified cache keys, then Renderer would run this code:

    // Cache the processed element if both $pre_bubbling_elements and $elements
    // have the metadata necessary to generate a cache ID.
    if (isset($pre_bubbling_elements['#cache']['keys']) && isset($elements['#cache']['keys'])) {
      if ($pre_bubbling_elements['#cache']['keys'] !== $elements['#cache']['keys']) {
        throw new \LogicException('Cache keys may not be changed after initial setup. Use the contexts property instead to bubble additional metadata.');
      }
      $this->renderCache->set($elements, $pre_bubbling_elements);
      // Add debug output to the renderable array on cache miss.
      if ($this->rendererConfig['debug'] === TRUE) {
        $render_stop = microtime(TRUE);
        $elements = $this->addDebugOutput($elements, FALSE, $pre_bubbling_elements, $render_stop - $render_start);
      }
      // Update the render context; the render cache implementation may update
      // the element, and it may have different bubbleable metadata now.
      // @see \Drupal\Core\Render\PlaceholderingRenderCache::set()
      $context->pop();
      $context->push(new BubbleableMetadata());
      $context->update($elements);
    }

In PlaceholderingRenderCache, it finds that a highly dynamic cache context got added by the lazy builder, placeholders the whole thing after all and makes it so the cache context does not bubble up. But this only happens if there were cache keys! Meaning that in the example above, PlaceholderingRenderCache is never called for something_expensive and all of this magic never happens.

The end result is that the really expensive thing that varies by user and the user cache context both make it to the parent render array elements, (Last bit is not true, HtmlRenderer takes care of that)

Examples that do work:

$render = [
  'something_expensive' => [
    '#lazy_builder' => 'my_service:somethingExpensive',
    '#cache' => [
      'contexts' => ['user'], // VARIES BY USER UPFRONT, GETS PLACEHOLDERED IMMEDIATELY
      'tags' => ['bar'],
    ],
  ],
];

$render = [
  'something_expensive' => [
    '#lazy_builder' => 'my_service:somethingExpensive', // SETS USER CACHE CONTEXT
    '#cache' => [
      'keys' => ['my', 'expensive', 'thing'], // TRIGGERS CACHE AND THUS LATE PLACEHOLDERING
      'contexts' => ['user'],
      'tags' => ['bar'],
    ],
  ],
];

Steps to reproduce

Use example above, witness that 'user' bubbles up to the page.

Proposed resolution

Late placeholder even if there are no cache keys set.

Merge request link

N/A

Remaining tasks

Fix

User interface changes

N/A

API changes

N/A

Data model changes

N/A

Release notes snippet

N/A

πŸ› Bug report
Status

Active

Version

11.0 πŸ”₯

Component
RenderΒ  β†’

Last updated 3 days ago

Created by

πŸ‡§πŸ‡ͺBelgium kristiaanvandeneynde Antwerp, Belgium

Live updates comments and jobs are added and updated live.
  • DrupalWTF

    Worse Than Failure. Approximates the unpleasant remark made by Drupal developers when they first encounter a particular (mis)feature.

Sign in to follow issues

Comments & Activities

  • Issue created by @kristiaanvandeneynde
  • πŸ‡§πŸ‡ͺBelgium kristiaanvandeneynde Antwerp, Belgium
  • πŸ‡§πŸ‡ͺBelgium wim leers Ghent πŸ‡§πŸ‡ͺπŸ‡ͺπŸ‡Ί

    πŸ€” Interesting. It's been too long for me to remember the details on this, but two immediate thoughts:

    • something can only be cached individually if and only if it has #cache[keys] set:
          // Try to fetch the prerendered element from cache, replace any placeholders
          // and return the final markup.
          if (isset($elements['#cache']['keys'])) {
      
    • … which I guess maybe (I do not recall!) implies that auto-placeholdering is impossible? But the detailed validation logic to ensure a reasonable DX does not check for that:
          // First validate the usage of #lazy_builder; both of the next if-statements
          // use it if available.
          if (isset($elements['#lazy_builder'])) {
            assert(is_array($elements['#lazy_builder']), 'The #lazy_builder property must have an array as a value.');
            assert(count($elements['#lazy_builder']) === 2, 'The #lazy_builder property must have an array as a value, containing two values: the callback, and the arguments for the callback.');
            assert(is_array($elements['#lazy_builder'][1]), 'The #lazy_builder argument for callback must have an array as a value.');
            assert(count($elements['#lazy_builder'][1]) === count(array_filter($elements['#lazy_builder'][1], function ($v) {
              return is_null($v) || is_scalar($v);
            })), "A #lazy_builder callback's context may only contain scalar values or NULL.");
            assert(!Element::children($elements), sprintf('When a #lazy_builder callback is specified, no children can exist; all children must be generated by the #lazy_builder callback. You specified the following children: %s.', implode(', ', Element::children($elements))));
            $supported_keys = [
              '#lazy_builder',
              '#cache',
              '#create_placeholder',
              '#lazy_builder_preview',
              '#preview',
              // The keys below are not actually supported, but these are added
              // automatically by the Renderer. Adding them as though they are
              // supported allows us to avoid throwing an exception 100% of the time.
              '#weight',
              '#printed',
            ];
            assert(empty(array_diff(array_keys($elements), $supported_keys)), sprintf('When a #lazy_builder callback is specified, no properties can exist; all properties must be generated by the #lazy_builder callback. You specified the following properties: %s.', implode(', ', array_diff(array_keys($elements), $supported_keys))));
          }
          // Determine whether to do auto-placeholdering.
          if ($this->placeholderGenerator->canCreatePlaceholder($elements) && $this->placeholderGenerator->shouldAutomaticallyPlaceholder($elements)) {
            $elements['#create_placeholder'] = TRUE;
          }
      

      There's no way to do the bubbling of contexts and update the cache redirect if there isn't any containing cacheable parent. I bet that that is the reason?

    So AFAICT you're right, and this is a bug. At minimum this is poor DX.

  • πŸ‡§πŸ‡ͺBelgium kristiaanvandeneynde Antwerp, Belgium

    Yeah, in a nutshell: Late placeholdering (i.e. lazy builder added a highly variable cache context) happens in the cache, but if the thing that specified the lazy builder has no cache keys, then the late placeholdering can never take place because it never gets to the cache.

  • πŸ‡§πŸ‡ͺBelgium wim leers Ghent πŸ‡§πŸ‡ͺπŸ‡ͺπŸ‡Ί

    πŸ™ˆ

Production build 0.71.5 2024