Random test failure in Drupal\Tests\demo_umami\FunctionalJavascript\PerformanceTest

Well that didn't work, what about just leaving it at $lcp_count === 2.

That seems to work, but it will make the tests ridiculously slow on pages that only have a single largest contentful paint event, there's already a follow-up to optimize this though 📌 Optimize PerformanceTestBase largest contentful paint detection Active which should be pretty stable too (possibly more because it will stop polling only once paint and network events stop coming through in the logs).

Without the 100x stuff, and marking with @group #slow.

Looks like this comment needs an update?

diff --git a/core/tests/Drupal/Tests/PerformanceTestTrait.php b/core/tests/Drupal/Tests/PerformanceTestTrait.php
@@ -143,7 +143,7 @@ protected function processChromeDriverPerformanceLogs(?string $service_name): Pe
       }
       // Only check once if $service_name is not set, since
       // largestContentfulPaint is not currently asserted on.
-      if ($lcp_count === 2 || !isset($service_name)) {
+      if ($lcp_count === 2) {
         break;
       }
       sleep(1);

Also, do we want to fix a test by making it slower whilst waiting on another issue to make it better?
Personally this feels like we should rollback the original issue, but that's probably just me.

Started looking at 📌 Optimize PerformanceTestBase largest contentful paint detection Active and maybe it is this simple?

Apparently git add isn't simple enough for me... complete patch this time.

OK that approach is looking good and should result in much faster telemetry tests once we have an otel endpoint connected. Converted to an MR.

Got this on ManageFieldsFunctionalTest (which isn't functional javascript so not affected by any of these tests):

1)
Drupal\Tests\field_ui\Functional\ManageFieldsFunctionalTest::testDefaultValue
Drupal\Core\Installer\Exception\InstallerException: Resolve all issues
below to continue the installation. For help configuring your database
server, see the installation handbook, or contact your hosting
provider.Failed to connect to database. The database engine reports the
following message: SQLSTATE[HY000]: General error: 5 database is
locked.Does the database file exist?Does web server have permission to
write to the database file?Does the web server have permission to write to
the directory the database file should be created in?

Everything else is green so ready for review I think.

Tested locally with the otel stack and giving up as soon as the performance log is empty is too aggressive, but if we wait for at least one lcp event and then check for an empty log, it seems fine.

I think this is a good incremental improvement to get rid of the random test failure, but also will see if we can do more in 📌 Optimize PerformanceTestBase largest contentful paint detection Active like possibly checking for an equal number of http requests and responses instead of, or as well as, the LCP event.

I've been shot down for random-test-failure-fixing patches for the below:

I think this needs a test with the change running 8500-10.000x failure free, whilst at the same time running a test running the current situation (in this case even 100x will fail a lot on sqllite).

Also currently the MR (if that's what should be committed?) is running one test only, so no matter the above, this isn't NR at the moment, me thinks.

Let's see if we can get 200 runs out of drupalci.

Also fixed up the MR cruft.

Looks like 200 times passed

Additional check that the number of requests made and responses returned is the same. This is as robust as I can think of. Also expanded the code comments to explain the logic.

Seeing one or two test failures, but pretty sure these are generic random sqlite test failures the same as the one in #11, not related to this issue:

1) Drupal\Tests\demo_umami\FunctionalJavascript\PerformanceTest::testPagesAnonymous
Drupal\Core\Installer\Exception\InstallerException: Resolve all issues below to continue the installation. For help configuring your database server, see the installation handbook, or contact your hosting provider.Failed to connect to database. The database engine reports the following message: SQLSTATE[HY000]: General error: 5 database is locked.Does the database file exist?Does web server have permission to write to the database file?Does the web server have permission to write to the directory the database file should be created in?

i.e. it's not complaining about trying to rm a non-empty directory.

Updated 🐛 Occasional locks on SQLite Active .

Thanks @catch, looks good to me.

_Very_ unsure if 🐛 Occasional locks on SQLite Active is about the same (known) problem, that, when running a test multiple times on SQLite, the DB locks.
But that problem hasn't got anything to do with the random that was fixed here.
If I understand 🐛 Occasional locks on SQLite Active correctly, the lock happened on a normal test-run where all test are ran only once?

Anyway, this is RTBC for me.

@Spokje oh interesting I forgot there's a locking issue specific to multiple runs of the same test, however I also saw the same error on ManageFieldsFunctionalTest on an MR run here, so maybe it's very occasional on real MR runs and very common when running the same test multiple times (although even this doesn't necessarily mean it's exactly the same problem as that issue, but the fix might be the same).

Thanks @catch, a final(?) question: After this is committed, will 📌 Optimize PerformanceTestBase largest contentful paint detection Active still be relevant?

@Spokje with the latest MR, I don't have any further ideas, so I think 📌 Optimize PerformanceTestBase largest contentful paint detection Active will end up being a duplicate. At least until something else comes up which is not an impossibility with the chrome performance log..

Committed 93d24db and pushed to 11.x. Thanks!

Automatically closed - issue fixed for 2 weeks with no activity.