Turbo: JS aggregation causes the RefreshLess JavaScript to evaluate more and more times on navigation

Created on 13 January 2024, 6 months ago
Updated 24 June 2024, 7 days ago

Problem/Motivation

This has only been tested on Drupal 10.1+ with its improved aggregation system,

Some relevant links:

Nope, that issue is due to hard coding in Turbo itself; see linked GitHub issue comments.

Steps to reproduce

Enable JavaScript aggregation. Navigate around and start seeing the console debug messages multiply, along with various errors when behaviours are double, triple, etc. attached:

RefreshLess: redirected /wiki/Special:Random → /wiki/2049/10/03/Adira js_hhgE22TbJoFEc0p2gGcm5NrkLElXp2JKD5t-ppI6IeU.js:1859:2492
RefreshLess: redirected /wiki/Special:Random → /wiki/2049/10/03/Adira js_91B8e_fr_qMZjN308me2y5aKFcx82T_6cWN5klDnuFc.js:1859:2492
RefreshLess: redirected /wiki/Special:Random → /wiki/2049/10/03/Adira js_hhgE22TbJoFEc0p2gGcm5NrkLElXp2JKD5t-ppI6IeU.js:1859:2492
RefreshLess: detaching behaviours js_hhgE22TbJoFEc0p2gGcm5NrkLElXp2JKD5t-ppI6IeU.js:1861:1450
RefreshLess: detaching behaviours js_91B8e_fr_qMZjN308me2y5aKFcx82T_6cWN5klDnuFc.js:1861:1450
RefreshLess: detaching behaviours js_hhgE22TbJoFEc0p2gGcm5NrkLElXp2JKD5t-ppI6IeU.js:1861:1450
RefreshLess: detaching behaviours js_hhgE22TbJoFEc0p2gGcm5NrkLElXp2JKD5t-ppI6IeU.js:1861:1450
RefreshLess: drupalSettings has been updated js_hhgE22TbJoFEc0p2gGcm5NrkLElXp2JKD5t-ppI6IeU.js:1859:631
RefreshLess: drupalSettings has been updated js_91B8e_fr_qMZjN308me2y5aKFcx82T_6cWN5klDnuFc.js:1859:631
RefreshLess: drupalSettings has been updated js_hhgE22TbJoFEc0p2gGcm5NrkLElXp2JKD5t-ppI6IeU.js:1859:631
RefreshLess: attaching behaviours js_hhgE22TbJoFEc0p2gGcm5NrkLElXp2JKD5t-ppI6IeU.js:1861:1286
RefreshLess: detaching behaviours js_hhgE22TbJoFEc0p2gGcm5NrkLElXp2JKD5t-ppI6IeU.js:1861:1450
RefreshLess: attaching behaviours js_hhgE22TbJoFEc0p2gGcm5NrkLElXp2JKD5t-ppI6IeU.js:1861:1286
RefreshLess: attaching behaviours js_hhgE22TbJoFEc0p2gGcm5NrkLElXp2JKD5t-ppI6IeU.js:1861:1286
RefreshLess: attaching behaviours js_91B8e_fr_qMZjN308me2y5aKFcx82T_6cWN5klDnuFc.js:1861:1286

Proposed resolution

This seems to be entirely due to how Drupal core handles aggregation (see 🐛 Aggregation of JavaScript files can cause redeclaration Active ) - since it cannot know every possible combination of libraries that may need to be aggregated across every request on a site, it builds aggregates using the combination of libraries for the current request, which means that different routes can and will have different aggregates referenced that are mostly the same in contents but differ in a few libraries not present in one and present in the other; this works well enough for full page loads (though isn't 100% efficient in terms of bandwidth and storage), but Turbo will happily evaluate aggregate JavaScript files as you navigate around that attempt to redeclare a lot of things, causing symptoms such as the above.

Some short term fixes have been implemented, such as setting preprocess: false on refreshless_turbo/js/refreshless.js (see commits in comments), but there are still some unexpected oddities, especially in administration areas with complex JavaScript interactions.

A long term fix should be to re-implement the additive library loading found in modules/refreshless_ajax/js/refreshless.js (originally from the only implementation in the 8.x-1.x branch). Ideally, we would find a way to build aggregates additively, but that may be a big headache potentially opening up security and performance issues. Further investigation will be needed.

Relevant may be core issues #3227837: Optimize aggregation grouping and [PP-1] Allow setting aggregation groups for js files in library definitions Postponed ; while not directly fixing our issue, #3224261: [PP-2] Front-end performance: concatenate CKEditor 5 JS assets for every unique configuration has some interesting and potentially useful ideas.

Original proposed resolution archived below but is no longer accurate:

Unsure. Figure out if it's on our end or if it's a Turbo issue.

A possibility on our end may be that it's caused by the 8.x-1.x code and/or core patch, so we should try moving everything not needed for Turbo into the refreshless_ajax submodule to see if that improves the situation.

If this is a bug with Turbo, work around it if possible - one hack that seems to work is wrapping the instantiation of our main class in a once() tied to the <html> element:

once('refreshless-turbo', 'html').forEach(function(element) {
  Drupal.RefreshLess.Turbo = new Turbo(
    Drupal.RefreshLess.classes.TurboBehaviours,
    Drupal.RefreshLess.classes.TurboDrupalSettings,
  );
});

...but this has a code smell that doesn't fix the underlying problem.

Remaining tasks

See above.

User interface changes

Stuff less borked.

API changes

None?

Data model changes

None.

🐛 Bug report
Status

Postponed

Version

2.0

Component

Code

Created by

🇨🇦Canada Ambient.Impact Toronto

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @Ambient.Impact
  • 🇨🇦Canada Ambient.Impact Toronto

    Added more detail, links to relevant Drupal core and Turbo code, and an example of one hack that seems to work around the issue (though not fixing the underlying problem).

  • Status changed to Postponed 4 months ago
  • 🇨🇦Canada Ambient.Impact Toronto

    Setting to postponed for now; we can clearly state that a known limitation of this Turbo implementation is that JavaScript aggregation should be off or weird things will happen which should be fine for a dev or alpha release; when we start getting to beta, we'll need to implement the longer term proposed resolution in some way.

  • 🇬🇧United Kingdom catch

    One possibility would be to track $ajax_page_state in session, and then maybe in a middleware, take what's in $_SESSION and fake the query parameter (probably merging with whatever's already in there just in case), then Drupal won't load any of those already-loaded libraries.

  • 🇨🇦Canada Ambient.Impact Toronto

    That may work, yeah, but I'm wondering if forcing a session for all users - including anonymous users - is going to have a noticeable performance hit. By default, Symfony doesn't start a session until there's something in the session to track (like in this case the Ajax page state) for performance reasons. That said, we do force a session for all users on Omnipedia to track the last few wiki pages a user has visited so that the random page controller knows not to pick something you've just seem, and that seems to perform well enough. If we do start sessions for all users, it would be a good idea to require the core `page_cache` module is uninstalled, because it'll prevent a session being started if a page is already cached - it took me far longer than I would have liked to figure out it was why our random page controller would sometimes redirect to the same page over and over for anonymous users.

  • 🇬🇧United Kingdom catch

    Might be possible to set it in a cookie instead as well, could still be read in a middleware.

  • 🇨🇦Canada Ambient.Impact Toronto

    I haven't messed around with any middleware as of yet so I'm not experienced with that but I like the idea. If anyone wants to create a basic prototype I'd be happy to test it out. Sorry about the slow replies; I'm busy with a bunch of stuff at the moment but checking in every so often.

  • 🇨🇦Canada Ambient.Impact Toronto

    So I tried to implement a solution based on @catch's idea and it sort of works but not quite. It doesn't yet have a cache context to vary by and it breaks Turbo since the Turbo JavaScript isn't found on the page so Turbo does a full page load, and I feel like piggybacking on ajaxPageState might not be worth the potential headaches down the road if that changes.

    To do list/ideas

    • Implement a cache context that varies based on a hash or compressed URL query string because we'll likely need to add that to the page #cache.
    • Port over some of the 1.x refreshless_page_state logic and don't spoof or use ajaxPageState.
    • Also don't forget RefreshlessRenderer::validatePreconditions() for security from the 1.x branch.
    • Turbo and RefreshLess stuff needs to be handled separately and excluded from being removed on subsequent pages because doing so causes Turbo to relinquish control and let a full page load occur. We may have to decorate another one of the core asset services to group the RefreshLess and Turbo stuff into its own aggregate since setting it all to preprocess: false isn't great.
    • Implement the PHP logic somewhere to exclude the already loaded libraries somewhere; our decorated AssetResolver would make the most sense since it's central to the whole process.

    Core stuff to use as reference

Production build 0.69.0 2024