Compress aggregate URL query strings

Created on 11 August 2022, almost 2 years ago
Updated 1 February 2023, over 1 year ago

Problem/Motivation

πŸ› Stampedes and cold cache performance issues with css/js aggregation Fixed changes aggregate URLs to a hash + query string with theme, delta, language and libraries.

The libraries list is the 'minimum representative set' (i.e. leaves of the dependency tree that are not dependents of anything else), but can still get quite long. user/1 visiting the front page of the standard profile ends up with query strings nearly 600 chars long. That gives us some headroom before it reaches 1,000, but it's not indefinite.

There is no RFC limit on URL or query string length, and we no longer support old versions of IE that used to have an arbitrary limit of around 2,000, but it's still possible for servers to set a maximum length (for example see discussion in https://stackoverflow.com/questions/812925/what-is-the-maximum-possible-...).

The most likely way someone would run into this is a site with 300 modules on quite a restrictive hosting platform, which is... well it's very possible really even if it'll be relatively uncommon.

@alexpott suggested compressing the query string if it's going to go over 950 characters.

Steps to reproduce

Proposed resolution

There are a couple of ways we could do this:
1. In the original issue, before we had the 'minimum representative set' option for libraries, #1014086-100: Stampedes and cold cache performance issues with css/js aggregation β†’ suggested keeping a lookup table, using a base64 increment or similar, so that each library could be represented by just one or two characters in the query string. We'd have to build that lookup table on library discovery and consult it both when building URLs and aggregates. Somewhat complex to implement but would be extremely efficent for the actual URL length which would end up looking something like A%2c0%2cf%2cbA instead of drupal/once%2cdrupal/backbone%2cdrupal/ajax

2. @alexpott suggested using gzcompress(), i.e.:

strlen(base64_encode(gzcompress("contextual/drupal.contextual-links%2Csystem/base%2Colivero/global-styling%2Ccore/drupal.active-link%2Colivero/powered-by-block%2Colivero/feed                                                                                                                                                       /drupal.debounce%2Ctoolbar/toolbar%2Cuser/drupal.user.icons%2Ccore/shepherd%2Ctour/tour-styling%2Ctour/tour%2Ccore/drupal.tabbingmanager%2Ccontextual/drupal.contextual-tool

This reduces 544 characters => 324 characters.

I think we should go with #2, at least until there's a compelling reason to go with #1, because #2 will be trivial to implement, and #1 will be hard.

A further complication is we've been hoping to use the URL information for ajaxPageState: πŸ“Œ Dynamically determine ajaxPageState based on libraries Active . I guess if we're only ever going to pass that list back to PHP at some point, maybe it's fine to just pass the encoded list around? But is it that simple?

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

πŸ“Œ Task
Status

Fixed

Version

10.1 ✨

Component
Asset libraryΒ  β†’

Last updated about 5 hours ago

No maintainer
Created by

πŸ‡¬πŸ‡§United Kingdom catch

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

Production build 0.69.0 2024