StandardPerformanceTest fails randomly on MySQL and consistently on Postgres

Added an MR based on failures in https://git.drupalcode.org/project/drupal/-/jobs/795888

Switch to a range makes sense, and know that was done on another ticket.

I swear, this specific test hates me: https://git.drupalcode.org/issue/drupal-3395986/-/jobs/805441#L780

Having said that, this MR would have prevent this failure.

I read the issue summary and the comments. The proposed resolution is explained in comment #7.

Reading the MR I think it should have comments to explain 1) why a range is needed and 2) how the numbers to use were picked. I think that information will be useful to those who come after us. However, I do not wish to hold this up since it does cause test failures. I am going to tag for a followup.

This very nearly fixes a critical regression in Postgres testing - head is failing due to this. See https://git.drupalcode.org/project/drupal/-/pipelines/96003

But unfortunately there's one more thing to fix in Postgres... will push a commit to MR once the current MR has finished testing on Postgres.

Yep as you can see from https://git.drupalcode.org/project/drupal/-/jobs/828846 this still does not fix postgres.

Fixing the issue title.

Re #9 I'm not that comments are really possible. These number really represent a stake in the ground - this is what Drupal currently does. If we introduce something that improves them then we want this test fail and the issue that makes the improvement should adjust the numbers. If we have an issue that degrades them then we need to carefully consider why and if it worth it or if there is a better way.

The latest commit on the MR is a nice colour of green for postgres... https://git.drupalcode.org/project/drupal/-/pipelines/96195

Wondering if an assertBetween() helper would make these slightly easier to read (and adjust in future).

After discussions with @catch we decided to skip core performance tests on Postgres and SQLite because this is not really fixing the extra queries on postgres because they are due to the driver. So I reverted the postgres specific fix and added a skip in PerformanceTestBase.

Looks great now.

Removing follow-up tag because of explanation in #12 📌 StandardPerformanceTest fails randomly on MySQL and consistently on Postgres RTBC .

Committed 46ed71c and pushed to 11.x. Thanks!

Did not cherry-pick cleanly to 10.2.x so not backporting this one.

longwave → closed merge request !6553

Automatically closed - issue fixed for 2 weeks with no activity.