Merging cache metadata is slow with thousands of tags added.

Created on 19 October 2023, 8 months ago
Updated 20 October 2023, 8 months ago

Problem/Motivation

The situation where I see merging taking a long time is huge GraphQL requests (thousands of entities using https://www.drupal.org/project/graphql → ).

When GraphQL resolves fields they can add cache metadata to the object that holds metadata for the whole request, and a lot of tags can be added multiple times.
Every call to addCacheTags performs merging of these tags, so if there are already a few thousand tags each added pack of tags will call array_unique and array_merge with thousands of values.

In one of my tests, Cache::mergeTags was called 71503 times and the average amount of tags in these calls was 2099.

I would like to hear if Drupal itself can benefit from optimizing merge, I'm not sure if this is only a GraphQL issue or maybe just my edge case.

Added patch for sure will fail tests and included just if someone would like to do quick profiling.
In my case, it reduces this query https://pastebin.com/ULmCsr3L from 12-13s to 4-5s. (Without Xdebug/profiler, the profiler doesn't show the same relative difference).
Also in the patch I've changed \Drupal\Core\Access\AccessResult::inheritCacheability, Cache::mergeMaxAges call was redundant since it is already done by addCacheableDependency. It seems to be a safe change and can be a different issue, although it doesn't change much in terms of performance.

Steps to reproduce

  • Create CacheableMetadata object
  • Keep adding tags to it
  • Profile

Proposed resolution

Change addCacheTags/Contexts so that merge is only performed when needed, for example only when calling getCacheTags/Contexts

Remaining tasks

User interface changes

None

API changes

TBD

Data model changes

-

Release notes snippet

-

📌 Task
Status

Needs work

Version

11.0 🔥

Component
Cache  →

Last updated about 16 hours ago

Created by

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Production build 0.69.0 2024