AssetResolver may load the same cache several time on a single page with bigpipe

Created on 5 February 2025, 2 months ago

Problem/Motivation

Don't fully understand why yet, but i noticed on our project with redis monitor that there are quite a few identical cache lookups:

"HGETALL" "prefix:data:css:THEME_NAMEenaxHQwg5UBhW9FfRUylgqlvQ43K67H9VuTW3QXt27iZU1"
"HGETALL" "prefix:data:js:THEME_NAMEen:D9AaXzT3MlnOQcjEJMKZEOqJpT9sVkpufRFf2FceqFM11"
"HGETALL" "prefix:data:js:THEME_NAMEen:NXhscRe0440PFpI5dSznEVgmauL25KojD7u4e9aZwOM11"
"HGETALL" "prefix:data:js:THEME_NAMEen:NXhscRe0440PFpI5dSznEVgmauL25KojD7u4e9aZwOM11"
"HGETALL" "prefix:data:js:THEME_NAMEen:NXhscRe0440PFpI5dSznEVgmauL25KojD7u4e9aZwOM11"
"HGETALL" "prefix:data:js:THEME_NAMEen:NXhscRe0440PFpI5dSznEVgmauL25KojD7u4e9aZwOM11"
"HGETALL" "prefix:data:js:THEME_NAMEen:NXhscRe0440PFpI5dSznEVgmauL25KojD7u4e9aZwOM11"
"HGETALL" "prefix:data:js:THEME_NAMEen:NXhscRe0440PFpI5dSznEVgmauL25KojD7u4e9aZwOM11"
"HGETALL" "prefix:data:js:THEME_NAMEen:NXhscRe0440PFpI5dSznEVgmauL25KojD7u4e9aZwOM11"
"HGETALL" "prefix:data:js:THEME_NAMEen:NXhscRe0440PFpI5dSznEVgmauL25KojD7u4e9aZwOM11"
"HGETALL" "prefix:data:js:THEME_NAMEen:NXhscRe0440PFpI5dSznEVgmauL25KojD7u4e9aZwOM11"
"HGETALL" "prefix:data:js:THEME_NAMEen:NXhscRe0440PFpI5dSznEVgmauL25KojD7u4e9aZwOM11"
"HGETALL" "prefix:data:js:THEME_NAMEen:NXhscRe0440PFpI5dSznEVgmauL25KojD7u4e9aZwOM11"
"HGETALL" "prefix:data:js:THEME_NAMEen:D9AaXzT3MlnOQcjEJMKZEOqJpT9sVkpufRFf2FceqFM11"

There are two unique hashes, the D9A... one is actually all-the-libraries, the NXh hash is an empty array of $libraries_to_load and settings.

Not sure how common this situation is, this also also on 10.4, haven't verified yet if 11.x changes anything.

I'm not sure what this is doing exactly with those assets. Took me a while to find where exactly it's looping, it's in \Drupal\big_pipe\Render\BigPipe::sendNoJsPlaceholders(), which has 23 fragments, all or most seem to be toolbar parts (classic toolbar with admin_toolbar).

Not sure what should be done in this scenario. statically cache the cache? if it's all identical, do we even need to process it or could we somehow return early somewhere?

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

πŸ“Œ Task
Status

Active

Version

11.0 πŸ”₯

Component

asset library system

Created by

πŸ‡¨πŸ‡­Switzerland berdir Switzerland

Live updates comments and jobs are added and updated live.
  • Performance

    It affects performance. It is often combined with the Needs profiling tag.

Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @berdir
  • πŸ‡¨πŸ‡­Switzerland berdir Switzerland

    I can reproduce this on 11.x by installing admin_toolbar_tools, but I assume this might also happen if you have lots of placeholders without that.

    I can see that this also happens out of the box on standard profile, for example with comments. To reproduce, create an article with a few comments, helps to filter redis monitor to just ":js:" then:

    ddev redis-cli monitor | grep ":js:"
    "HGETALL" "core:data:js:olivero:en:7zukb8t4ZfBhCx251RER8wVVOq1uHT4RK1bzyN-Zs4k10"
    "HGETALL" "core:data:js:olivero:en:NXhscRe0440PFpI5dSznEVgmauL25KojD7u4e9aZwOM10"
    "HGETALL" "core:data:js:olivero:en:NXhscRe0440PFpI5dSznEVgmauL25KojD7u4e9aZwOM10"
    "HGETALL" "core:data:js:olivero:en:7zukb8t4ZfBhCx251RER8wVVOq1uHT4RK1bzyN-Zs4k10"
    "HGETALL" "core:data:js:olivero:en:3tuqg80w5eAzmKR6QJQIP9lFpLGh3emPlFtUgncnLic10"
    "HGETALL" "core:data:js:olivero:en:NXhscRe0440PFpI5dSznEVgmauL25KojD7u4e9aZwOM10"
    

    7zukb8t4ZfBhCx251RER8wVVOq1uHT4RK1bzyN is repeated twice and NXhscRe0440PFpI5dSznEVgmauL25KojD7u4e9aZwOM10 three times, that's on an article with 4 comments or so. Adding more comments doesn't directly correlate to the number of repetitions it seems.

    Unsure how to proceed on this one. I could add a memory cache or something, but there might be something more useful to be done in this case?

  • πŸ‡¨πŸ‡­Switzerland berdir Switzerland

    I also tested with πŸ› AssetResolver::getJsAssets cache id doesn't consider attached settings, leads to ajax issues Active , doesn't really change this.

  • πŸ‡¨πŸ‡­Switzerland berdir Switzerland

    πŸ“Œ Render the navigation toolbar in a placeholder Active is also not helping here, because those placeholders aren't cacheable.

  • πŸ‡¬πŸ‡§United Kingdom catch

    I think I tried to get rid of this in πŸ› AssetResolver::getJsAssets cache id doesn't consider attached settings, leads to ajax issues Active by skipping the cache get on empty libraries altogether, but it is not entirely skippable due to alters etc.

    Statically caching result of the cache get only in the case of empty libraries is probably the simplest thing to do here.

  • Merge request !11121try a memory backend chain β†’ (Closed) created by berdir
  • Pipeline finished with Failed
    2 months ago
    Total: 554s
    #416252
  • Pipeline finished with Failed
    2 months ago
    Total: 90s
    #417269
  • πŸ‡¨πŸ‡­Switzerland berdir Switzerland

    Injecting the cache chain now and updated the test. This will of course conflict with several other issues.

  • Pipeline finished with Failed
    2 months ago
    Total: 681s
    #417275
  • Pipeline finished with Failed
    2 months ago
    Total: 332s
    #417285
  • Pipeline finished with Success
    2 months ago
    Total: 309s
    #417330
  • The Needs Review Queue Bot β†’ tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

    This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

    Consult the Drupal Contributor Guide β†’ to find step-by-step guides for working with issues.

  • πŸ‡¨πŸ‡­Switzerland berdir Switzerland

    Rebased.

  • Pipeline finished with Failed
    about 1 month ago
    Total: 351s
    #435657
  • Pipeline finished with Success
    about 1 month ago
    Total: 365s
    #435693
  • πŸ‡¬πŸ‡§United Kingdom catch

    This looks great, I don't see anything else to do.

  • The Needs Review Queue Bot β†’ tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

    This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

    Consult the Drupal Contributor Guide β†’ to find step-by-step guides for working with issues.

  • πŸ‡¨πŸ‡­Switzerland berdir Switzerland

    Rebased, minor conflict on CacheTagIsValidCount.

  • Pipeline finished with Failed
    about 1 month ago
    Total: 549s
    #442284
  • Pipeline finished with Success
    28 days ago
    Total: 3993s
    #445100
  • Pipeline finished with Success
    24 days ago
    Total: 629s
    #448716
  • Pipeline finished with Success
    23 days ago
    Total: 1711s
    #449512
  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Appears to need a rebase

    If you are another contributor eager to jump in, please allow the previous poster at least 48 hours to respond to feedback first, so they have the opportunity to finish what they started!

  • Pipeline finished with Success
    20 days ago
    Total: 608s
    #451621
  • πŸ‡¨πŸ‡­Switzerland berdir Switzerland

    Rebased and test assertions updated, green again.

  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Will add to my list for tomorrow but since it was previously RTBC wonder if it’s good to go back?

  • πŸ‡¨πŸ‡­Switzerland berdir Switzerland

    It should be, nothing changed except performance metric conflicts. One reason I didn't set it back myself is that there are a lot of performance issues right now and it's not as important/big of an impact as some others, but any improvement helps...

  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Okay in that case going to remark it

    • catch β†’ committed 407e7645 on 11.x
      Issue #3504559 by berdir: AssetResolver may load the same cache several...
  • πŸ‡¬πŸ‡§United Kingdom catch

    Committed/pushed to 11.x, thanks!

  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024