RoutePreloader loads a lot of routes

When disabling RoutePreloader and all render cache bins, I get 77 calls in total to preloadRoutes() for an anonymous user on my frontpage only 6 of them actually fetch additional routes (including the two from redirect), many of them already deliberately preload their specific routes. and mostly happening within cacheable bits. LocalTask does all the node local tasks at once, a few menus, some weird bits like .

Worth noting: Our site building approach almost exclusively uses nodes with paragraphs, we basically have no other linked routes such as views or something.

In total, less than 100kb memory reported through that method by Blackfire.

With render caches (but page and dynamic page cache disabled), I'm getting exactly 2 as anon user, those from redirect. As an admin, on the user edit form with navigation module I get 800 calls to preloadRoutes(), 13 cache lookups. On the frontpage, it's 11 with render caches on, 17 with them off. 500-600kb memory usage for admin.

(I'm optimistic that a lot of these will actually go away if we can fully cache the navigation)

Is there a third option? Can we just remove the RoutePreloader, do we really need this?

Somehow missed 🐛 RoutePreloader loads a lot of routes Active . Will keep this separate for now to do some more testing as the approach is quite different.

Doing some tests with a different implementation.

Basically, I disabled the route preloader for now completely. On a production system that's been running for a bit, I see around 110 variations of route preloader caches with the current approach. I assume that without preloading a ton of routes on every request, there will be a lot more of those variations. Right now, the cache is always for a bunch of routes, keyed by a hash of those.

I'm changing that to using getMultiple() so we don't get many overlapping variations of different sets of routes. I'm not sure what the performance difference between a multi-cache load and a single one is. It's multiple rows, multiple cache items to expand and check. On its own, possibly slower, but I plan to combine it with the second idea:

As mentioned, I'm playing with the idea of using a ChainedFast backend for either some (either hardcoded on not /admin or by still using the Preloader, but only to feed the list of "frontend" routes) or all. For the mix, it would split the routes, and load them from the fast or regular bin. At this point, getMultiple() a bunch of routes should be a lot faster than the regular query.

Right now we cache the serialized strings because we know we don't need all of them. Without the big preload, that's no longer true and we will use those routes quite reliable. So we could consider to stop doing that and benefit from improvements such as 📌 Make igbinary the default serializer if available, it saves 50% time on unserialize and memory footprint Active by caching the route objects and letting the cache backend/serializer optimize it. would also

For size comparison as to how big the router table is, on our install profile:

select count(*), sum(length(route)) from router;
+----------+--------------------+
| count(*) | sum(length(route)) |
+----------+--------------------+
|      876 |            1244156 |
+----------+--------------------+

select count(*), sum(length(route)) from router where path not like '/admin%';
+----------+--------------------+
| count(*) | sum(length(route)) |
+----------+--------------------+
|      175 |             231266 |
+----------+--------------------+

MariaDB [db]> select count(*), sum(length(data)) from config;
+----------+-------------------+
| count(*) | sum(length(data)) |
+----------+-------------------+
|     1078 |           2124343 |
+----------+-------------------+

MariaDB [db]> select count(*), sum(length(data)) from cache_discovery;
+----------+-------------------+
| count(*) | sum(length(data)) |
+----------+-------------------+
|       40 |           1454662 |
+----------+-------------------+

Full router table is around 1.2MB, similar to discovery cache and around half the size of the config table. Non-admin routes is only 200kb.

Sure, router could get a lot bigger, but every view that would be added there will also add a much bigger chunk to config.

Unsure how fancy we want to get with this, happy for some opinions.

I think in 5 you meant this 📌 Move route preloading into the UrlGenerator: don't simply preload on every HTML request Needs work

I tried to see how many routes we really have to load on most pages.

Made some small changes to @Berdir's MR to remove all caching, which is pushed to a draft MR.

Took these steps:

install standard
uninstall toolbar
install navigation

Left myself logged in as admin.

On a dynamic_page_cache hit, I get only get route lookups for the front page view route. This is for the is_front logic in preprocess + drupal settings. We could potentially cache that lookup independently in the bootstrap cache because it is done on every page. Would love to get rid of is_front altogether too but that's not easy unfortunately.

After truncating the dynamic page cache, I got , , the same views route, and bigpipe.no_js.

I think we could potentially handle the special and similar routes to circumvent the database/cache altogether too, not sure how yet but we never need to match against those routes, only load them.

Overall I think we could aim for zero route lookups when dynamic page cache is warm, one or two when it's cold. And then use @berdir's approach (without the serialization hack) to cache the individual routes.

I think I tested with disabled render caches, my concerns around the caching atm are how the impact is cold caches, agreed that warm caches should have very few lookups.

A variant of the caching idea above is that we could also only use the fast cache and just don't cache others. It should only happen in the backend and for admins, and it's essentially a 1:1 of a query lookup and cache lookup.

I pushed a proof of concept for that, still using the route preloader and a new method, didn't add to interface yet, route preloader is now a misleading name.

RouteProvider already implements events too. I wonder if we should just inline that cacheable routes stuff and make it an implementation detail, unsure.