Performance tests need to run cron

Created on 17 January 2025, 5 days ago

Problem/Motivation

At least sometimes, performance tests are triggering cron in the 'warm up' requests, but cron is so expensive that it takes ages to finish, meaning those requests don't necessarily warm up enough to get consistent results.

We should add a cron run before the warm-up requests, maybe in PerformanceTestBase::setUp()

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

πŸ› Bug report
Status

Active

Version

11.0 πŸ”₯

Component

phpunit

Created by

πŸ‡¬πŸ‡§United Kingdom catch

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @catch
  • First commit to issue fork.
  • Pipeline finished with Success
    5 days ago
    Total: 475s
    #398751
  • πŸ‡ͺπŸ‡ΈSpain plopesc Valladolid

    Made the suggested change and PerformanceTest is passing again locally.

    Since the reported issue is not being reported in the CI jobs, not sure how to test it or whether this one could be merged as it is.

  • πŸ‡¬πŸ‡§United Kingdom catch

    I think it's more important that the tests tests what they're supposed to, than that we test that the tests test what they're supposed to.

    We could probably check cron_run_last and make sure it's not 0 - would mean doing an assertion inside ::setUp() but probably we do that elsewhere already.

  • πŸ‡¬πŸ‡§United Kingdom catch

    Or another option here - we could uninstall automated_cron. We don't need it for anything in the tests, we just need it not to run. That definitely doesn't need explicit test coverage then - would need to call ModuleInstaller::uninstall() in ::setUp() where it would otherwise run cron.

  • Pipeline finished with Failed
    5 days ago
    Total: 748s
    #398776
  • Pipeline finished with Success
    5 days ago
    Total: 636s
    #398793
  • πŸ‡ͺπŸ‡ΈSpain plopesc Valladolid

    Uninstalling automated_cron did the trick. It required to update a few cache get counters in a couple of Performance Tests, but I think this is OK now.

    It seems that we're doing exactly the opposite of what the IS suggests, though :)

  • πŸ‡¬πŸ‡§United Kingdom catch

    Cache gets are probably the automated_cron configuration.

    Updated the issue summary and title. Hoping someone else can RTBC so I can still commit this..

  • Pipeline finished with Failed
    4 days ago
    Total: 488s
    #399329
  • Pipeline finished with Success
    4 days ago
    #399335
  • πŸ‡ΊπŸ‡ΈUnited States nicxvan

    Looks good to me, all performance tests have been updated and this should make the queries like up in tests.

  • πŸ‡¨πŸ‡­Switzerland berdir Switzerland

    > Cache gets are probably the automated_cron configuration.

    Created πŸ“Œ Aggregate cache operations per bin to allow for more specific asserts Active so we have the ability to see this a bit more reliably and can be more specific with assertions when we want to.

  • πŸ‡¨πŸ‡­Switzerland berdir Switzerland

    Crosspost on the RTBC, sorry.

    • catch β†’ committed 18071b07 on 11.x
      Issue #3500489 by plopesc, catch: Performance tests should uninstall...
  • πŸ‡¬πŸ‡§United Kingdom catch

    A bit concerning that we're getting different results locally vs. gitlab, there's an existing issue about functional js test false negatives on gitlab. However this issue makes sense in itself and is a straight regression from when we removed the cron run in the installer, so let's do it.

    Committed/pushed to 11.x, thanks!

Production build 0.71.5 2024