Adjust performance tests to avoid random failures

Currently failing because we allow neither 38 nor 39, adding suggestions to MR

Not for StandardPerformanceTest, but rather OpenTelemetryAuthenticatedPerformanceTest: https://git.drupalcode.org/issue/drupal-3417362/-/jobs/709685

This one failed even though no real code was changed in that MR< only test code in unrelated tests. Should we also fix that here?

Let's do that here, would be nice to know what this is, seems infrequent though.

Looks good to me. Hopefully it stays with these 2.

Committed d3c96dd and pushed to 11.x. Thanks!

Doesn't apply cleanly to 10.2.x, I don't think the previous tightening of these tests was backported there.

longwave → closed merge request !6365

would be nice to know what this is,

Shouldn't we be able to figure this out by logging the requested URLs and matching them, i.e. defining which URL patterns are expected to be requested? 🤔

It's a variation in the number of database queries. What's happening is a small fraction (not sure how small yet) of these test runs record one extra database query compared to the others. There are some things in Drupal that are inherently indeterminate - the order that image derivatives and css/js aggregates return responses (and write to cache collectors and similar), things like session_write_interval, automated cron.

We could try to do the same for database queries that we're trying for cache IDs in 📌 Investigate variable database query counts in performance tests Active but it's not inherently straightforward because some things depend on the simpletest ID, so we don't actually get the same thing each time to compare against.

Just found more in https://git.drupalcode.org/project/drupal/-/jobs/712690 - will open a follow-up now.

will open a follow-up now.

Not sure if that happened, my search foo couldn't find it.

But I have hit another one, that's not the same as the ones mentioned in #14:

    Drupal\Tests\demo_umami\FunctionalJavascript\OpenTelemetryAuthenticatedPerformanceTest::testFrontPageAuthenticatedWarmCache
    Failed asserting that 17 is equal to 16 or is less than 16.
    
    /builds/issue/drupal-3347502/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php:121
    /builds/issue/drupal-3347502/vendor/phpunit/phpunit/src/Framework/Constraint/Constraint.php:55
    /builds/issue/drupal-3347502/core/profiles/demo_umami/tests/src/FunctionalJavascript/OpenTelemetryAuthenticatedPerformanceTest.php:39
    /builds/issue/drupal-3347502/vendor/phpunit/phpunit/src/Framework/TestResult.php:728

https://git.drupalcode.org/issue/drupal-3347502/-/jobs/740903

I nearly didn't find it when I looked for it to link here, but just found it again 📌 Add ranges to more performance test assertions Needs review .

Automatically closed - issue fixed for 2 weeks with no activity.