Contrib.social

[META] Track: Proposal for performance testing

Open on Drupal.org →
Created on 4 November 2024, 5 months ago

Summary

Gander, the performance testing framework → , has been part of Drupal since 10.2. We are using it to test performance in Drupal core and it already helped us identify and resolve multiple performance issues ( 🐛 Flood database backend ::isAllowed() should call ::ensureTableExists() Fixed , 📌 Optimize user logins by avoiding duplicate entity queries Needs work , 📌 Improve Largest Contentful Paint in Umami front page Fixed , 📌 Avoid reading session from the database multiple times during a request Needs work , ...).

We think that we should use it for Drupal CMS in order to make it the most performant open source CMS on the market and to make sure performance is not degraded over time.

Work to be done

  • Agree on a set of performance-focused goals for Drupal CMS. They can use Core Web Vitals or some other metric.
  • Provide monthly reports on the actual metrics vs. goals.
  • Ensure current CI environment is ready to run performance tests (most likely already the case).
  • Plan and write initial performance tests. Focus on base recipe initially and expand on optional recipes afterwards.
  • Set up a performance dashboard, similar to the one we have for Drupal core. In order to secure reliable results for the dashboard we would need a dedicated test runner (sponsorship/donation needed).
  • Support community with expanding and writing additional performance tests on an ongoing basis.
  • Support community with identifying and fixing performance issues on an ongoing basis.

Out of scope

Actively looking for an fixing performance issues is not in scope (providing support for the community to do so is).

Target milestone

TBD

Skills required

  • Experience planning, writing and maintaining performance tests.
  • General performance expertise.

Blockers / dependencies

TBD

Track owner

TBD (Tag1 is offering to sponsor work in this track.)

Proposal / roadmap

TBD

📌 Task
Status

Needs review

Component

General

Created by

🇸🇮Slovenia slashrsm

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @slashrsm
  • Comment 5 months ago →
    🇬🇧United Kingdom catch

    Set up a performance dashboard, similar to the one we have for Drupal core. In order to secure reliable results for the dashboard we would need a dedicated test runner (sponsorship/donation needed).

    The core performance dashboard currently requires ~4 test runs per hour to populate the graphs properly, this is due to some kind of prometheus/grafana quirk in graph rendering. A core test run takes about 8.5 minutes.

    If we reduce the number of core test classes/methods, or if Drupal CMS performance tests can complete in ~5 minutes, we could run those tests on the same runner and still have results graphed - we just need to be able to run >= 8 tests per hour.

    Alternatively it might be a one-line fix somewhere to make the grafana graphs work with less frequent test runs, the problem is figuring out which line ( and that it's not very easy to test this locally since it take a fair amount of time to generate data). If we can fix that, we can have multiple core branches and potentially Drupal CMS stable vs. HEAD and things like that.

  • Status changed to Needs work 3 months ago
  • Comment 3 months ago →
    🇺🇸United States thejimbirch Cape Cod, Massachusetts

    Marking as Needs Work based on the comments in #2

  • Comment 3 months ago →
    🇬🇧United Kingdom catch

    Next steps here:

    📌 Switch the performance job to fail Active

    Also we should add performance tests for the optional recipes - either all at once or individually, need to talk to @phenaproxima about how to organise that.

  • Comment about 2 months ago →
    🇬🇧United Kingdom catch

    I've refactored the core dashboard (https://gander.tag1.io/) to require less frequent test runs. It's currently relying on hourly tests of 11.1.x and 10.4.x instead of every 15 minutes, but theoretically we could run tests every four hours or twelve hours and it should still work now.

    Next step for Drupal CMS is 📌 Send opentelemetry traces to the performance dashboard Active . Once that issue lands, we should be able to set up a pipeline schedule to populate the dashboard.

    The remaining step after this would be adding some additional test coverage for optional recipes. I'm wondering if we can enable every optional recipe at once. InteractiveInstallTest installs all the recipes shown in the UI installer, but this isn't every optional recipe shipped with Drupal CMS. Not sure if there's a clean way to install 'everything in the recipes folder'

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024