Avoid registering multiple batch sets with node_access_rebuild()'s batch mode

Created on 4 March 2025, 21 days ago

Problem/Motivation

When node_access_rebuild(TRUE) is called multiple times per request, it registers as many batch sets as the number of times the function was called. This can lead to unnecessary long node access rebuild processes, because triggering the operation once per request is enough. At least it's hard to imagine a scenario when multiple consecutive runs would have any benefit.

Example: Let's say you are rolling out some updates to your site and multiple modules request node access rebuild in their database update hooks via node_access_rebuild(TRUE);. Ideally, what should happen is that the rebuild process is performed only once, but it going to be triggered multiple times.

Steps to reproduce

  1. Implement two new database update hooks (not necessarily in different modules) which request node access rebuild by calling node_access_rebuild(TRUE);.
  2. Run the pending database updates either via Drush or from the administrative UI.

Proposed resolution

I can think of several possible solutions to address the problem:

  • Fix the implementation of node_access_rebuild()'s batch mode to register only one batch set even if the function is called multiple times.
  • As an alternative solution - if it turns out that there can be scenarios when running the rebuild process multiple times does make sense - provide a solution which ensures that the process is only registered once per request (e.g. node_access_rebuild_once()).
  • Provide a solution by which developers can check whether the rebuild process has been triggered already during the request.

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

πŸ› Bug report
Status

Active

Version

11.1 πŸ”₯

Component

node system

Created by

πŸ‡­πŸ‡ΊHungary balazswmann

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @balazswmann
  • πŸ‡­πŸ‡ΊHungary mxr576 Hungary
  • Pipeline finished with Failed
    20 days ago
    Total: 93s
    #440672
  • Pipeline finished with Success
    20 days ago
    Total: 2639s
    #440683
  • πŸ‡­πŸ‡ΊHungary mxr576 Hungary
  • πŸ‡­πŸ‡ΊHungary mxr576 Hungary

    Fix the implementation of node_access_rebuild()'s batch mode to register only one batch set even if the function is called multiple times.

    This is what I currently see in the MR.

    As an alternative solution - if it turns out that there can be scenarios when running the rebuild process multiple times....

    What would be those scenarios realistically? I cannot think of any at this moment... ideally there would be only one node access rebuild, triggered at the very at of the deploy process, only triggered when it is necessary.

  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Thanks for reporting, next step will probably be to add test coverage

  • Setting/detecting an arbitrary array property on the batch set feels strange, but I don't see a better way to detect whether a certain batch set is already registered, though.

    One small nit on the MR, but otherwise looks like only tests are outstanding.

  • πŸ‡¬πŸ‡§United Kingdom catch

    We could add a static in the function itself to ensure it's not added from there more than once. I guess the difference is someone could add it without calling this function.

  • We could add a static in the function itself to ensure it's not added from there more than once. I guess the difference is someone could add it without calling this function.

    By "add it," do you mean the static variable? If the variable is regular static and not drupal_static, then it can't be accessed outside the function. If you mean adding the batch set, if someone is doing that without using this function, then the id property might not get added, either. So the static variable might be a cleaner way to go?

    Alternatively, if a property in array is being used, it could be on the $batch array itself and not within the set, to save from having to loop:

    // Only recalculate if the site is using a node_access module.
    if (\Drupal::moduleHandler()->hasImplementations('node_grants')) {
      if ($batch_mode) {
        // Ensure that the batch set is only registered once.
        $batch = &batch_get();
        if (!empty($batch['has_node_access_rebuild'])) {
          return;
        }
        $batch['has_node_access_rebuild'] = TRUE;
        ...
    
  • First commit to issue fork.
  • Pipeline finished with Canceled
    17 days ago
    Total: 132s
    #443477
  • πŸ‡ΊπŸ‡ΈUnited States mikelutz Michigan, USA

    Switched to using a static variable, and added a test.

  • Pipeline finished with Canceled
    17 days ago
    Total: 70s
    #443479
  • πŸ‡ΊπŸ‡ΈUnited States mikelutz Michigan, USA
  • Pipeline finished with Failed
    17 days ago
    Total: 1322s
    #443480
  • Pipeline finished with Success
    17 days ago
    Total: 786s
    #443488
  • I wonder if there's value in generalizing an approach to prevent certain batch sets from being added twice. My thinking is to add a static array property to BatchBuilder. Something like:

    class BatchBuilder {
    
      protected static array $ids = [];
    
      public static function hasId(string $id): bool {
        return !empty(static::$ids[$id]);
      }
    
      public static function setId(string $id): void {
        static::$ids[$id] = TRUE;
      }
    
      ...
    }
    

    Then in node_access_rebuild(), it'd be something like:

    if ($batch_mode && !BatchBuilder::hasId('node_access_rebuild')) {
       ...
       batch_set($batch_builder->toArray());
       BatchBuilder::setId('node_access_rebuild');
       ...
    

    And this could be applied similarly for other uses.

  • πŸ‡ΊπŸ‡ΈUnited States mikelutz Michigan, USA

    ooh, I really like that. I had given thought to something more general in the batch system to prevent duplicate batches from being set, but I was thinking about thinks like having batch_set hash the definition array and preventing duplicates that way, but it would be a behavior change, and I can't be certain that someone doesn't have. a legitimate reason to duplicate a batch somewhere for some reason, so I kept the focus on the rebuild.

    This is a simple and elegant helper method in the helper class for code to ensure it doesn't add a duplicate batch if it doesn't want to.

  • Pipeline finished with Failed
    15 days ago
    Total: 89s
    #444544
  • Pipeline finished with Success
    15 days ago
    Total: 760s
    #444553
  • MR looks good. I have 1 minor comment, but it's personal code style preference, and I don't feel that strongly about it.

  • πŸ‡­πŸ‡ΊHungary balazswmann

    Thanks for the improvements! It's definitely much better now than my original fix. I was also thinking about a more general solution but I didn't want to touch the other parts of the system.

  • πŸ‡­πŸ‡ΊHungary mxr576 Hungary

    Awesome improvements indeed! Maybe announcing the new API worth a change record?

  • CR added: https://www.drupal.org/node/3512253 β†’ (@mikelutz provided helpful docblock that could be copied).

    Note to update the CR if it's decided that method names need changing.

  • πŸ‡ΊπŸ‡ΈUnited States mikelutz Michigan, USA

    NW to bikeshed method names. :-)

  • πŸ‡ΊπŸ‡ΈUnited States mikelutz Michigan, USA
  • πŸ‡ΊπŸ‡ΈUnited States mikelutz Michigan, USA

    Fixed/tweaked a few docs while I was in there, setting back to NR

  • Pipeline finished with Success
    10 days ago
    Total: 3181s
    #449048
  • πŸ‡¬πŸ‡§United Kingdom catch

    Looks good. Committed/pushed to 11.x, thanks!

    • catch β†’ committed 8f207214 on 11.x
      Issue #3510939 by mikelutz, balazswmann, godotislate, mxr576: Avoid...
  • πŸ‡¬πŸ‡§United Kingdom catch
  • πŸ‡¬πŸ‡§United Kingdom catch
  • πŸ‡­πŸ‡ΊHungary mxr576 Hungary

    Is this eligible for backporting?

    • catch β†’ committed b64382f3 on 10.5.x
      Issue #3510939 by mikelutz, balazswmann, godotislate, mxr576: Avoid...
  • πŸ‡¬πŸ‡§United Kingdom catch

    Yes - cherry-picked to 10.5.x, thanks!

  • πŸ‡­πŸ‡ΊHungary mxr576 Hungary

    Thank you, too!

Production build 0.71.5 2024