Implement a caching strategy for the menu links

Created on 5 April 2024, 9 months ago

Problem/Motivation

From looking at https://git.drupalcode.org/project/navigation/-/commit/17970de1ed6b4129d... it looks like there is not yet a caching strategy for the menu links in the navigation module.

The existing toolbar has a very developed caching system already, this was implemented in #1814932: Caching strategy for D8 toolbar , #1137920-303: Fix toolbar on small screen sizes and redesign toolbar for desktop and #1805054: Cache localized, access filtered, URL resolved, and rendered menu trees and has not significantly changed since.

It was necessary to add this because rendering a full tree of admin menu links is very expensive both in terms of server-side processing (hundreds of milliseconds every request) and page weight (potentially hundreds of extra kb every request).

I think we need an issue to port this logic across from the existing toolbar.

by @catch 📌 New Toolbar Roadmap: Path to Beta & Stable Active .

Proposed resolution

Implement a caching strategy for the menu links.

Remaining tasks

User interface changes

API changes

Data model changes

📌 Task
Status

Active

Version

1.0

Component
Navigation 

Last updated about 14 hours ago

No maintainer
Created by

🇪🇸Spain ckrina Barcelona

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @ckrina
  • 🇬🇧United Kingdom catch

    ckrina credited catch .

  • 🇪🇸Spain ckrina Barcelona
  • Assigned to plopesc
  • 🇪🇸Spain plopesc Valladolid

    Working on it.

  • Pipeline finished with Failed
    8 months ago
    Total: 250s
    #154541
  • Pipeline finished with Failed
    8 months ago
    Total: 243s
    #155014
  • Pipeline finished with Failed
    8 months ago
    Total: 253s
    #155112
  • Issue was unassigned.
  • 🇪🇸Spain plopesc Valladolid

    Worked on a POC that use client-side cache for the whole navigation tree, while loading only the 1st level from the server.

    The approach is a cloned adapted from how Toolbar module works.

    My JS skills are limited, so would be great if someone could take a look into them.

    Also need to figure out how to avoid duplicated IDs when same block is placed more than once clean_unique_id twig filter does not work well for individual AJAX calls.

    My initial thought was to use the block ID, but I found this issue: 💬 Retrieve unique block ID or machine name in build() method of BlockBase class Active

    Feedback and ideas are welcome!

  • First commit to issue fork.
  • Pipeline finished with Failed
    8 months ago
    Total: 334s
    #155696
  • 🇺🇸United States bnjmnm Ann Arbor, MI

    Also need to figure out how to avoid duplicated IDs when same block is placed more than once clean_unique_id twig filter does not work well for individual AJAX calls.

    On 1.x it is already appending --<NUMBER> to avoid the duplicates if > 1 of the same Navigation Block is placed.

    I'm specifically looking at the ids that the MR modifies in templates/navigation-menu.html.twig, and those seem to be fine. If there are specific steps that lead to the duplicate ID issue I'm interested in knowing what they are.

  • 🇫🇮Finland lauriii Finland

    Let's make sure that any caching strategy that we implement is built in mind that we want to keep the navigation completely static in between page loads with the exception of the active marker moving to a different menu link. This means we want to avoid any flickering or even loading animations when you navigate within Drupal, including the username. The navigation is really close to being able to achieve this which is quite remarkable.

  • 🇪🇸Spain plopesc Valladolid

    Hi @bnjmnm

    The reason why it failed is because the --NUMBER suffixes added to the IDs modify the HTML markup, hence the hash generated from that markup is not consistent.

    This is because Html::getUniqueId() methods adds a random identifier for AJAX calls

    Here are the steps to reproduce it:

    • Replace clean_id with clean_unique_id in navigation-menu.html.twig
    • Clear Drupal cache
    • Clear browser local storage
    • Disable render cache from /admin/config/development/settings or config files
    • Reload the page and confirm that there's an AJAX error and navigation menu nested levels are not being loaded.

    Getting rid of the id suffix solves the issue in the meantime, even if the HTML markup is not correct. It has to be resolved somehow, though.

    Another option would be to get rid of id attributes in the template, but I'm not sure if that's possible for accessibility reasons.

  • 🇪🇸Spain ckrina Barcelona
  • Pipeline finished with Failed
    8 months ago
    Total: 618s
    #159574
  • 🇪🇸Spain plopesc Valladolid

    Closed MR against contrib navigation module. Created equivalent MR against core.

  • 🇬🇧United Kingdom catch

    I'm not 100% sure about this, it would be good to get confirmation from someone with more accessibility knowledge, but I think ID attributes are only required for form elements, and anything else should be OK with aria roles etc. - so it may just be fine to remove the ids, use data- attributes if necessary, and add aria markup if it's not already there.

    Toolbar flickering was fixed in 🐛 Toolbar tray rendering can result "flickering" resizing of content area to accommodate open trays Fixed / 🐛 Investigate better ways to add anti-flicker JS Fixed , so unless there is more flickering going on, that might not be an issue any more? That issue only landed in the past few months after many years of toolbar in core though.

  • Pipeline finished with Success
    8 months ago
    Total: 548s
    #159595
  • 🇪🇸Spain ckrina Barcelona
  • Pipeline finished with Success
    8 months ago
    Total: 3250s
    #160358
  • 🇺🇸United States itmaybejj

    On the accessibility question -- I don't anticipate an issue there. Assistive devices only need an HTML ID if something in the HTML is trying to reference the element by its ID, e.g. via a label's for= attribute or an aria-labelledby reference.

    Sometimes in nav elements it's nice to reference the inner heading to give the nav a unique name. But that could be done in the JS with a simple global counter.

  • 🇪🇸Spain plopesc Valladolid

    During Drupalcon sprints worked on an approach where ID attributes, and all the references to them are removed from the HTML markup before generating the hash, so the problem is gone, but at the cost of some extra calculations that would be great to avoid.

    Would be great to have some extra feedback about whether we can get rid of id attributes or should we explore other paths assuming that IDs could be there.

    On the other hand, if we remove IDs from the core templates, but a specific site overrides the template making use of them, the navigation would be broken.

    That's something that already would happen with current toolbar, but need to confirm if that scenario should be supported by navigation.

  • 🇬🇧United Kingdom catch

    #17 is pretty convincing to me that we can do without the ID attributes.

  • Pipeline finished with Skipped
    7 months ago
    #180513
  • Pipeline finished with Skipped
    7 months ago
    #181421
  • Pipeline finished with Success
    4 months ago
    Total: 405s
    #274674
  • Pipeline finished with Failed
    4 months ago
    Total: 411s
    #274783
  • Pipeline finished with Success
    4 months ago
    Total: 464s
    #275532
  • Pipeline finished with Failed
    4 months ago
    Total: 686s
    #275698
  • Pipeline finished with Failed
    4 months ago
    #275746
  • Pipeline finished with Running
    4 months ago
    #275772
  • Pipeline finished with Failed
    4 months ago
    Total: 837s
    #275817
  • Pipeline finished with Canceled
    4 months ago
    Total: 128s
    #275889
  • Pipeline finished with Success
    4 months ago
    Total: 433s
    #275891
  • Pipeline finished with Success
    3 months ago
    #277863
  • Pipeline finished with Success
    3 months ago
    Total: 661s
    #277885
  • Pipeline finished with Failed
    3 months ago
    Total: 804s
    #277932
  • Pipeline finished with Success
    3 months ago
    #277980
  • Pipeline finished with Failed
    3 months ago
    Total: 539s
    #278145
  • Pipeline finished with Failed
    3 months ago
    Total: 567s
    #278169
  • Pipeline finished with Success
    3 months ago
    Total: 548s
    #278183
  • Pipeline finished with Failed
    3 months ago
    Total: 525s
    #278218
  • Pipeline finished with Success
    3 months ago
    Total: 15824s
    #278400
  • Pipeline finished with Failed
    3 months ago
    Total: 570s
    #279003
  • Pipeline finished with Success
    3 months ago
    #279040
  • Pipeline finished with Success
    3 months ago
    Total: 461s
    #279089
  • Pipeline finished with Failed
    3 months ago
    Total: 444s
    #279149
  • Pipeline finished with Failed
    3 months ago
    #279166
  • Pipeline finished with Failed
    3 months ago
    Total: 445s
    #279249
  • Pipeline finished with Success
    3 months ago
    Total: 428s
    #279308
  • Pipeline finished with Canceled
    3 months ago
    Total: 458s
    #279316
  • Pipeline finished with Success
    3 months ago
    #279325
  • Pipeline finished with Success
    3 months ago
    Total: 82s
    #290211
  • Pipeline finished with Success
    3 months ago
    Total: 51s
    #290368
  • 🇪🇸Spain ckrina Barcelona

    Postponing this until back the ID and aria-labelled-by with JS Postponed is in. Discussing this with @plopesc.

  • Pipeline finished with Failed
    about 1 month ago
    Total: 327s
    #334461
  • Pipeline finished with Failed
    about 1 month ago
    Total: 352s
    #335115
  • Pipeline finished with Failed
    about 1 month ago
    Total: 336s
    #335152
  • Pipeline finished with Success
    about 1 month ago
    Total: 155s
    #336162
  • Pipeline finished with Skipped
    about 1 month ago
    #336170
  • Pipeline finished with Canceled
    about 1 month ago
    Total: 79s
    #336342
  • Pipeline finished with Success
    about 1 month ago
    Total: 326s
    #336344
  • Pipeline finished with Success
    about 1 month ago
    Total: 340s
    #336345
  • Pipeline finished with Success
    about 1 month ago
    Total: 519s
    #336835
  • Pipeline finished with Success
    about 1 month ago
    Total: 877s
    #337509
  • Pipeline finished with Success
    about 1 month ago
    Total: 466s
    #337933
  • Pipeline finished with Failed
    25 days ago
    Total: 735s
    #350982
  • Pipeline finished with Failed
    24 days ago
    Total: 732s
    #352311
  • Pipeline finished with Failed
    24 days ago
    Total: 787s
    #352324
  • Pipeline finished with Failed
    24 days ago
    Total: 860s
    #352357
  • Pipeline finished with Success
    24 days ago
    Total: 714s
    #352369
  • Pipeline finished with Success
    24 days ago
    Total: 745s
    #352383
  • Status changed to Postponed 3 days ago
  • 🇬🇧United Kingdom catch

    I think we might be better off doing 📌 Render the navigation toolbar in a placeholder Active (see some of the discussion in that issue).

    If we do that, then navigation needs to use a #lazy_builder for the navigation render array and core will handle efficiently rendering it via render caching / dynamic_page_cache / bigpipe depending on what's enabled. The HTML will be served directly, no need for local storage and hence no need to worry about the flickering issues that affected the toolbar module or custom js etc.

    This needs one improvement to core's placeholder rendering strategy (the CachedPlaceholderStrategy sketched out in that issue), but that improvement would benefit any situation where placeholders are used, not just Navigation.

    Not closing this as a duplicate yet but if it all works, then we'll be able to keep things simple in Navigation itself, and also improve overall core page serving performance at the same time.

  • 🇪🇸Spain plopesc Valladolid

    The approach proposed by @catchlooks great to me.

  • 🇬🇧United Kingdom catch

    📌 Add render caching for the navigation render array Active landed which unblocks 📌 Render the navigation toolbar in a placeholder Active .

    Marking this as PP-2 and added that issue to the issue summary here. I'll swap the navigation stable blocker tags over too. If we have to change direction again we can switch things around again, but pretty sure this will be both easier and better than adapting the toolbar tree caching implementation so let's try it!

Production build 0.71.5 2024