Add caching to ConfigurableLanguageManager::getLanguages()

Created on 4 January 2025, 18 days ago

Problem/Motivation

See 🐛 Add persistent cache to CachedStorage::findByPrefix Active

This method causes an uncached query on every page request on translatable sites

Steps to reproduce

Proposed resolution

Add some sort of persistent caching to that method.

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

📌 Task
Status

Active

Version

11.0 🔥

Component

language system

Created by

🇨🇭Switzerland berdir Switzerland

Live updates comments and jobs are added and updated live.
  • Performance

    It affects performance. It is often combined with the Needs profiling tag.

Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @berdir
  • Merge request !10799Add caching to ConfigurableLanguageManager → (Open) created by berdir
  • 🇨🇭Switzerland berdir Switzerland

    Seeing two calls to that that aren't statically cached. Once with STATE_CONFIGURABLE, very early, in \Drupal\language\EventSubscriber\LanguageRequestSubscriber::setLanguageOverrides() after/as part of language negotation. And the second one is the EntityConverter param upcasting (through EntityRepository => ContentEntitBase::getLanguages(), which explicitly uses STATE_ALL.

    The second cache creates the same information, but mostly hits static caches (CachedStorage and ConfigFactory).

    Started with a basic implementation, lets see if this causes any test fails. Also didn't yet add static caching for $languages and no DI either.

  • Pipeline finished with Failed
    18 days ago
    Total: 557s
    #385785
  • Pipeline finished with Failed
    17 days ago
    Total: 636s
    #385914
  • Pipeline finished with Success
    17 days ago
    Total: 948s
    #385924
  • 🇨🇭Switzerland berdir Switzerland

    To get the tests to pass, this required a single manual cache clear in a test that's explicitly messing with the default language service in the container on the fly. I think that's fair.

    This saves an early bootstrap query on every single multilingual site requests that isn't an internal page cache hit. When using redis, that means the amount of queries go down from 6 to 5 on a dynamic page cache hit:

    What's strange is that \Drupal\Tests\demo_umami\FunctionalJavascript\OpenTelemetryAuthenticatedPerformanceTest::testFrontPageAuthenticatedWarmCache isn't failing, because that's exactly the query that should go away. Ah, I think those tests don't run by default. I also wasn't able to run it manually, go this error:

    This job is stuck because of one of the following problems. There are no active runners online, no runners for the protected branch , or no runners that match all of the job's tags: performance-test-runner 
    

    Locally, it 's failing, but not only because that query is gone, also because I'm getting a race condition on automated_cron. Reliably got this fail 3 times:

    1) Drupal\Tests\demo_umami\FunctionalJavascript\OpenTelemetryAuthenticatedPerformanceTest::testFrontPageAuthenticatedWarmCache
    Failed asserting that two arrays are identical.
    --- Expected
    +++ Actual
    @@ @@
         0 => 'SELECT "session" FROM "sessions" WHERE "sid" = "SESSION_ID" LIMIT 0, 1',
         1 => 'SELECT * FROM "users_field_data" "u" WHERE "u"."uid" = "10" AND "u"."default_langcode" = 1',
         2 => 'SELECT "roles_target_id" FROM "user__roles" WHERE "entity_id" = "10"',
    -    3 => 'SELECT "config"."name" AS "name" FROM "config" "config" WHERE ("collection" = "") AND ("name" LIKE "language.entity.%" ESCAPE '\\') ORDER BY "collection" ASC, "name" ASC',
    +    3 => 'INSERT INTO "semaphore" ("name", "value", "expire") VALUES ("cron", "LOCK_ID", "EXPIRE")',
    +    4 => 'SELECT "expire", "value" FROM "semaphore" WHERE "name" = "cron"',
    +    5 => 'INSERT INTO "watchdog" ("uid", "type", "message", "variables", "severity", "link", "location", "referer", "hostname", "timestamp") VALUES (0, "cron", "Attempting to re-run cron while it is already running.", "WATCHDOG_DATA", 4, "", "LOCATION", "REFERER", "CLIENT_IP", "TIMESTAMP")',
     ]
    

    I think the non-cron-on-installer change really did a number on this test. Others maybe too, didn't try to run them. The umami based test is special because it's first cron run is really slow. There's a lot to index there.

    I tried a number of things such as the wait for terminate event (adds another query for the lock and wasn't reliable enough either), explicit lock wait(), sleep(). The one thing that seems to reliably help is explicitly running cron before the first web request. Maybe the performance tests should just run without that module?

  • Pipeline finished with Failed
    11 days ago
    Total: 91s
    #392878
  • 🇬🇧United Kingdom catch

    What's strange is that \Drupal\Tests\demo_umami\FunctionalJavascript\OpenTelemetryAuthenticatedPerformanceTest::testFrontPageAuthenticatedWarmCache isn't failing, because that's exactly the query that should go away. Ah, I think those tests don't run by default. I also wasn't able to run it manually, go this error:

    That should run by default.

    The one thing that seems to reliably help is explicitly running cron before the first web request. Maybe the performance tests should just run without that module?

    Yeah that's a good plan, we can always enable it if we want to test it, or run cron manually if we want to test after a cron run.

  • Pipeline finished with Success
    11 days ago
    Total: 679s
    #392884
  • 🇨🇭Switzerland berdir Switzerland

    One question is whether we can avoid the cache tag. We have a variable amount of cache keys, but the amount of variations is limited, we might be able to do a bulk delete this. Not sure yet.

  • 🇬🇧United Kingdom catch

    Should we count cache gets that hit the database differently from fast chained? Would be useful to know how many additional db queries we actually have?

    It would be possible to separate out chained fast vs. non-chained fast bin gets and aggregate them separately and that might be worth doing. I'm not sure we'd necessarily find more issues that way but it would be closer to reality which is always good. Can add separate getters for these so it's only an API addition.

  • Pipeline finished with Success
    about 16 hours ago
    Total: 416s
    #401690
Production build 0.71.5 2024