SettingsTrayBlockFormTest::testBlocks() fails locally 100% of the time and lots of times on Gitlab CI

The block opening isn't an AJAX request, which is why it was removed in the other issue, but it is a CSS animation or whatever, so the sleep makes sense. The wait on ajax request was making this pass randomly by at least waiting a bit, even if not for an AJAX request to complete.

Proper fix should be done in #3317520: Random failure in Drupal\Tests\settings_tray\FunctionalJavascript\SettingsTrayBlockFormTest::testEditModeEnableDisable → but I think a sleep(1) if fine for now if it makes it pass all the time.

@catch I think there is something else going on here... it passes if you only test stark but if you test both stark and stable9 then it'll fail on the second regardless of order.

I've tried clearing the cache after changing the theme but it made no difference...

Got a repeat job running on this patch - https://git.drupalcode.org/project/drupal/-/jobs/360205

...and HEAD - https://git.drupalcode.org/project/drupal/-/jobs/358222

What if we enable the css_disable_transitions_test module?

@larowlan thanks for trying! For others following along we already enable the css_disable_transitions_test module in \Drupal\FunctionalJavascriptTests\WebDriverTestBase::installModulesFromClassProperty()

So this gets odder... it is passing on HEAD when run 100 times - yet on other MRs it is failing lots - see https://git.drupalcode.org/project/drupal/-/jobs/360407 and it fails locally.

So that leaves me to pndering what at the differences between MR tests and prod tests and do they get different resources somehow.

So this gets odder... it is passing on HEAD when run 100 times - yet on other MRs it is failing lots - see https://git.drupalcode.org/project/drupal/-/jobs/360407 and it fails locally.

For at least some tests I assume we get issues due to resource usage during a particular test, which when it runs at the same time as a test that's timing dependent, then causes that test to fail. This would explain why running the same test 100 times is different to running it amongst other tests, but it's not really based on anything concrete.

Note that testEditModeEnableDisable() in the same test was skipped recently due to random failures, these particular random failures started happening after that was skipped I think, then even more after the ajax wait issue.

This passes locally for me...

Pushed a commit which still passes locally for me, but maybe it makes a difference elsewhere - just switching to the new assertWaitOnAjaxRequest() in OffCanvasTestBase.

Absolutely unsure if this is the root cause, but it certainly doesn't help that pressing the 'Save Site branding' in the overlay, when using any theme, returns you to the non-existing /2, instead of /user/2...

If the sleep(1) causes this to pass, I think we should commit that workaround to stop the bleeding - it's a more honest situation than the previous, fake, waitForAjaxRequest() call. Then we can continue in #3317520: Random failure in Drupal\Tests\settings_tray\FunctionalJavascript\SettingsTrayBlockFormTest::testEditModeEnableDisable → .

100ms sleep fixes this 100% pf the time for me.

I've updated the MR to have an @todo to https://www.drupal.org/project/drupal/issues/3317520 → and remove the assertion that @catch added that broke.

The problem is happening because we're clicking before the click handler that's added in core/modules/settings_tray/js/settings_tray.js:207 - I have no idea why it's the second theme that causes this 100% of the time locally for me.

I think the MR is committable.

I will add more detail about my findings to #3317520: Random failure in Drupal\Tests\settings_tray\FunctionalJavascript\SettingsTrayBlockFormTest::testEditModeEnableDisable →

lauriii → closed merge request !5464

Committing from needs review because it seems that this is now failing almost 100% of the time on Gitlab CI too.

Committed 4f47df3 and pushed to 11.x. Also cherry-picked to 10.2.x. Thanks!

Automatically closed - issue fixed for 2 weeks with no activity.