Investigate random failures in nightwatch and phpunit

Coming from Slack. I'm also seeing phpunit jobs failing in GitLab pipelines for contrib with the DB not alive yet. E.g. https://git.drupalcode.org/project/address/-/jobs/616657 -- I haven't been keeping super close track. I can paste more examples here as I see them if it's helpful. Not sure this is really the best project, or if we need a gitlab templates issue about it or something...

Created #3414252: Add CI_DEBUG_SERVICES to investigate DB failures → for contrib.

Not sure I can think of a reason this would be bad, so going to mark.

Failed nightwatch job with debug enabled
https://git.drupalcode.org/issue/drupal-2448545/-/jobs/633899

Another failed nightwatch job: https://git.drupalcode.org/issue/drupal-3415412/-/jobs/651717

  ️TEST FAILURE (1m 49s): 
   - 1 error during execution; 
   - 0 assertions failed; 1920 passed
   ✖ 3) Tests/ckEditor5CodeSyntaxTest
  - OTHER ERRORS:
  1)  Error
   Command failed: php ./scripts/test-site.php user-login 1 --site-path sites/simpletest/49243948
In Database.php line 455:
                                                             
  The specified database connection is not defined: default  
                                                             
user-login [--site-path SITE-PATH] [--] <uid>

All roads point to the DB credentials not showing up in settings.php (?) in time for the test suite to find them.

should this go back to NW then

No, it's just that dww reported one more of the failures, because we gather whatever we can find, in order to find the cause of it.

I have to say that the nightwatch failures do not feel like they have the same cause of failure with PHPUnit tests.. Let's fix them one by one. Also, since in #7 we have a nightwatch along with DB logs, we don't need to enable CI_DEBUG_SERVICES yet. Let's fix the nightwatch jobs, that seem pretty straightforward and then we can check the phpunit ones

Moving this back to needs review because it looks like we're getting enough information from the MR without a core commit.

Wondering if 📌 Replace deprecated functions in Nightwatch tests Fixed is the culprit, opened a revert MR to find out.

Changed the title so people can easily find this issue

Turned out the culprit for #8 was 📌 Enable modules through Nightwatch API when not testing module enabling Needs work which has been reverted and is in the process of being fixed.

Tried to come back to this one, but comparing the 2 MRs seem to be doing different things. Which one is supposed to be review? Does it diff from issue summary?

For now, 6126 can be closed.
We've found the cause for nightwatch, but we don't know what's happening with phpunit tests. If we see an increase in random phpunit failures, we could, in the end, commit the changes of 6126 to 11.x and monitor all jobs. But due to the fact phpunit failures are not that common, and usually a re-run of the job works, its priority can be considered minor

Automatic Updates is suffering from many of these failures!

We enabled CI_DEBUG_SERVICES in our .gitlab-ci.yml, so here are some failed job logs for the heroes working to fix this:

https://git.drupalcode.org/project/automatic_updates/-/jobs/712543
https://git.drupalcode.org/project/automatic_updates/-/jobs/712550
https://git.drupalcode.org/project/automatic_updates/-/jobs/712549
https://git.drupalcode.org/project/automatic_updates/-/jobs/712558

@dimitriskr told me in Slack that he couldn't access the logs, so here they are, as raw text.

I'm uploading one of the logs and make it more readable due to gitlab's raw output