The amount of individual calculated permission items grow by the number of memberships

Created on 13 July 2024, about 2 months ago
Updated 27 August 2024, 12 days ago

Problem/Motivation

I'm investigating a huge performance problem on a website, which uses Group for fine-granular access control. Users may have thousands of group memberships.

Queries being built up using query access are exploding by their statement length using hundreds (sometimes thousands) of placeholders.

The first thing that I noticed was that Drupal\group\QueryAccess\EntityQueryAlter was calling up hasPermission() using flexible_permissions over than 2 million times on runtime. It was a first indicating that something is wrong there.

After some investigation I might have identified the problem, but feel free to correct me - it may well be that I'm on the wrong track.

The reason why the queries on the site I'm looking at are exploding, is that there is a huge amount of individual permissions being calculated by Drupal\group\Access\IndividualGroupPermissionCalculator and those are then being included as part of conditions within Drupal\group\QueryAccess\EntityQueryAlter.

On the website, users may have a lot of memberships (thousands). But whenever permissions are being re-calculated, all memberships are being loaded: https://git.drupalcode.org/project/group/-/blob/3.3.x/src/Access/Individ...

There are some major impacts due to this:

  • Whenever permissions need to be re-calculated, this slows down response times by a lot and exceeds some memory limits.
  • Website cannot scale with growing amoung of memberships per user
  • Query statements may grow and potentially exceed the SQL string length being passed to the database

There are for sure more impacts because of including all memberships for re-calculation. But as said maybe I'm on the wrong track ATM but currently this looks like the source of the problem.

Steps to reproduce

The problem possibly affects any site with users having many memberships (starting at around 500 where this may have an impact).
Then there are some factors that multiply that number:

  • Number of group content plugins enabled
  • Distinguish of "own" and "any" permissions
  • Number of times entity / views queries are to be invoked on a request that make use of access control (query access)

Proposed resolution

I don't know yet, just started looking at the problem. But this looks like a big one if I'm not on the wrong track.

Remaining tasks

  • Let's make sure this is the problem
  • Find a solution path

User interface changes

API changes

Data model changes

💬 Support request
Status

Active

Version

3.2

Component

Code

Created by

🇩🇪Germany mxh Offenburg

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @mxh
  • 🇩🇪Germany mxh Offenburg

    Queries being built up using query access are exploding by their statement length using hundreds (sometimes thousands) of placeholders.

    This case is true when the number of groups grow accordingly. Usually a user has one membership per group anyway.

    Another problem is the following query being built within \Drupal\group\QueryAccess\EntityQueryAlter::doAlter:

    <?php
    // ...
    $group_relationship_data_table = $this->entityTypeManager->getDefinition('group_relationship')->getDataTable();
    $plugin_ids_in_use = $this->database
      ->select($group_relationship_data_table, 'gc')
      ->fields('gc', ['plugin_id'])
      ->condition('plugin_id', $plugin_ids, 'IN')
      ->distinct()
      ->execute()
      ->fetchCol();
    // ...
    ?>

    Since the table may grow by hundreds of thousands of records, the distinct query affects performance. It gets noticeable when this method is called within one record up to 100 times (which is possible when rendering Views configurations).

Production build 0.71.5 2024