AssetResolver::getJsAssets() and AssetResolver::getCssAssets() can repeatedly try to calculate the same set of assets

Created on 12 January 2024, 12 months ago
Updated 13 May 2024, 8 months ago

Problem/Motivation

Found in πŸ“Œ Add authenticated user umami performance tests Fixed . AjaxResponseAttachmentsProcessor is used by big pipe responses, and calls AssetResolve::getJsAssets() and AssetResolver::getCssAssets() with the same arguments as the main HtmlResponseAttachmentsProcessor. The arguments are the same when there are no assets to get at all, so we can special-case that.

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

πŸ“Œ Task
Status

Fixed

Version

10.3 ✨

Component
Asset libraryΒ  β†’

Last updated 1 day ago

No maintainer
Created by

πŸ‡¬πŸ‡§United Kingdom catch

Live updates comments and jobs are added and updated live.
  • Performance

    It affects performance. It is often combined with the Needs profiling tag.

Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @catch
  • Status changed to Needs review 12 months ago
  • πŸ‡¬πŸ‡§United Kingdom catch

    Can just early return when there's no libraries to load. This will help any big pipe response that doesn't return unique css/js that's already loaded - makes a decent reduction in the number of cache operations in both standard and umami performance tests. Won't be a measurable speed improvement, but less to do is good.

  • πŸ‡¬πŸ‡§United Kingdom catch

    MR also tidies up the cache IDs to add colons between each part consistently. This doesn't resolve any bugs, it was just a debugging aid before I realised exactly what was going on. However I don't think it does any harm here.

  • Status changed to Needs work 12 months ago
  • πŸ‡¬πŸ‡§United Kingdom catch

    Apparently not that easy.

  • πŸ‡¬πŸ‡§United Kingdom catch
  • Status changed to Needs review 12 months ago
  • πŸ‡¬πŸ‡§United Kingdom catch
  • Status changed to RTBC 12 months ago
  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Seems like a good cleanup, reviewing the code for assetResolver and using the variable seems to still work. Number going down too was neat.

  • πŸ‡§πŸ‡ͺBelgium wim leers Ghent πŸ‡§πŸ‡ͺπŸ‡ͺπŸ‡Ί

    Nice β€” trivial improvement thanks to static caching, in a code path where I see no reason to ever need static cache clearing? Ship it! 🚒

    Do we have any numbers? YES WE DO! 🀩 Thanks to core/profiles/standard/tests/src/FunctionalJavascript/StandardPerformanceTest.php and core/profiles/demo_umami/tests/src/FunctionalJavascript/OpenTelemetryAuthenticatedPerformanceTest.php πŸ‘

    If we add a static cache, we save a couple of cache gets.

    Devil's advocate: why does this matter? Aren't we just spreading out the places where we "keep stuff"? The cache backend already does static caching, so why does this matter?

    It'd be more meaningful if this was about unique cache gets. Wouldn't it be more meaningful to only track cache gets for unique cache IDs? πŸ€”

  • Status changed to Needs review 11 months ago
  • πŸ‡¬πŸ‡§United Kingdom catch

    The cache backend already does static caching, so why does this matter?

    cache.data doesn't do any static caching, and it also doesn't use chained fast (which would go into apcu by default so be extremely cheap although not static as such) - so this is either a database query or a round trip to redis/memcache depending on the site.

    fwiw the situation where this actually seems to happen is for a list of empty assets (i.e. where the AJAX request either attaches nothing, or only things that have already been attached - in fact any other situation ought to be a unique cache ID). I tried an early return when the list is empty and that broke lots of things, but I think we could special case the static caching and do it only when there's an empty list - it would be a bit more logic, but we can easily tell from the tests whether it'll still work like that.

  • Status changed to Needs work 11 months ago
  • The Needs Review Queue Bot β†’ tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

    This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

    Consult the Drupal Contributor Guide β†’ to find step-by-step guides for working with issues.

  • Merge request !7666Return early if there's no CSS to load. β†’ (Open) created by catch
  • Pipeline finished with Failed
    8 months ago
    Total: 982s
    #154096
  • Pipeline finished with Success
    8 months ago
    #154113
  • Pipeline finished with Failed
    8 months ago
    Total: 1049s
    #154129
  • πŸ‡¬πŸ‡§United Kingdom catch

    catch β†’ changed the visibility of the branch 3414398-asset-resolving-runs to hidden.

  • Status changed to Needs review 8 months ago
  • πŸ‡¬πŸ‡§United Kingdom catch

    Figured out what was wrong with the early return in the first attempt - new MR with no static caching and early returns that work this time. Saves a lot of cache gets when bigpipe is enabled.

  • Pipeline finished with Canceled
    8 months ago
    Total: 1040s
    #154395
  • Status changed to RTBC 8 months ago
  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Code wise seems like a good cleanup. Not sure how else to best test so leaning on performance tests.

  • πŸ‡¬πŸ‡§United Kingdom catch
  • Pipeline finished with Canceled
    8 months ago
    Total: 816s
    #154426
  • Pipeline finished with Failed
    8 months ago
    Total: 957s
    #154453
  • Status changed to Needs work 8 months ago
  • πŸ‡¬πŸ‡§United Kingdom catch

    We need to do the early return and the static cache here to get the best results. Updated MR incoming.

  • Status changed to RTBC 8 months ago
  • πŸ‡¬πŸ‡§United Kingdom catch

    No it's a false alarm, sometimes we cache the same thing with different cache keys, but to figure that out, we'd have to run at least some of the logic we're caching, so just static caching doesn't help with that. The MR is relatively simple as it is and gets more complicated with static caching, and performance tests don't change when I implement it, so... moving back to RTBC.

  • Pipeline finished with Success
    8 months ago
    Total: 1082s
    #154776
  • Status changed to Needs review 8 months ago
  • πŸ‡¬πŸ‡§United Kingdom catch

    Going back on #19, I think there might be one more thing we can do here.

  • Pipeline finished with Success
    8 months ago
    Total: 1050s
    #154804
  • Status changed to RTBC 8 months ago
  • πŸ‡¬πŸ‡§United Kingdom catch

    Meh moving back to RTBC again, might open a follow-up for this last bit.

    What happens is when you have e.g. the announce toolbar library, which only has a single CSS file in it, the end result of that in ::getJsAssets() is the same as if there are no libraries to load at all.

    However we can only determine this by getting the full library definition from LibraryDiscovery::getLibraryByName(), which involves running some logic, which involves getting the library definitions from the cache collector, which.. potentially undermines caching in the first place. So changing that logic would need a fair bit of manual profiling I think.

    I reverted the change @Berdir asked about in the MR, since that's only removing something out of scope, back to RTBC (again).

  • πŸ‡¬πŸ‡§United Kingdom alexpott πŸ‡ͺπŸ‡ΊπŸŒ

    This looks great.

    Committed and pushed e2840d7270 to 11.x and 557022fcdd to 10.3.x. Thanks!

  • πŸ‡¨πŸ‡­Switzerland berdir Switzerland
    • alexpott β†’ committed 557022fc on 10.3.x
      Issue #3414398 by catch, Berdir: AssetResolver::getJsAssets() and...
  • Status changed to Fixed 8 months ago
  • Status changed to RTBC 8 months ago
  • πŸ‡¬πŸ‡§United Kingdom catch

    Got an idea how to resolve what I was trying to figure out above, but opened πŸ“Œ Optimize AssetResolver caching Fixed , will put a stacked MR on there since it'll build on the changes here.

  • Status changed to Fixed 8 months ago
  • πŸ‡¬πŸ‡§United Kingdom catch

    Whoops cross-post but that means I don't need to stack the other issue now!

  • πŸ‡¬πŸ‡§United Kingdom alexpott πŸ‡ͺπŸ‡ΊπŸŒ
  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024