Nightwatch tests failing due to lack of "capabilities" - need Selenium driver

@fjgarlin did point us here - https://docs.gitlab.com/runner/executors/kubernetes/#overwrite-container... - so we can see what's available.

After some discussion with @longwave, it seems like it's failing consistently on "next major" (Drupal 11) and passing on Drupal 10. Drupal 10 uses Nightwatch 2.4.2, but Drupal 11 now uses Nightwatch 3.7.

See 📌 Update Nightwatch to 3.x Fixed

I have worked on the solution in #3462681: Add W3C compliant JS testing → and am going to test that MR here.

The changes in the gitlab MR 248 made a difference. Previously the Nightwatch Next Major test failed with:

─────────────────────────────────
- Connecting to localhost on port 9515...
⚠ Failed to connect to Selenium Server on localhost with port 9515.
─────────────────────────────────
TEST FAILURE (484ms): 
   - 1 error during execution; 
   - 0 tests failed; 
   - 0/NA tests passed
   - 1 skipped
   ✖ 1) Tests/keyboardTest
  - OTHER ERRORS:
  Error
    An error occurred while creating a new Selenium Server session: [SessionNotCreatedError] session not created: Missing or invalid capabilities

But now the next major test gets further, you can see the new hostname 'selenium' and port '4444' and it fails with

 Error
   uncaughtException: browser.drupalRelativeURL(...).waitForElementVisible(...).updateValue(...).updateValue(...).element(...).click(...).waitForElementVisible is not a function
TypeError: browser.drupalRelativeURL(...).waitForElementVisible(...).updateValue(...).updateValue(...).element(...).click(...).waitForElementVisible is not a function
    at NightwatchAPI.<anonymous> (/builds/issue/project_browser-3469490/tests/src/Nightwatch/Tests/keyboardTest.js:59:10)

So it looks like the gitlab MR 248 is making some improvement.

The MR is good for the "Nightwatch Next Major" test, but the PHPUnit Next Major test fails with three ocurrences of an error like

Drupal\Tests\project_browser\FunctionalJavascript\ProjectBrowserUiTest::testAdvancedFiltering
WebDriver\Exception\JavaScriptError: javascript error:
{"status":60,"value":"[object HTMLOptionElement] has no size and location"}

This test passes when using the existing web driver. So what's not clear is whether there is something wrong with how we have implemented the new web driver for D11 W3C-compliant javascript testing, or whether the Project Browser is not yet W3C-compliant.

Maybe adding some options to the driver can help: https://stackoverflow.com/questions/78436451/i-am-facing-javascriptexcep...

No luck. Reverting the change in the issue.

Trying pinning the version as per 🐛 Nightwatch and Functional JavaScript fails since selenium/standalone-chrome:128 Fixed

@chrisfromredfin - we've changed the chrome driver to use for "next major" to match the one that Drupal core is following.

The "nightwatch (next major)" initially reported no longer happens, and there is an error that might be a legit one.
However, "phpunit (next major)" started failing.

See the history of changes made in here https://www.drupal.org/project/gitlab_templates/issues/3462681#comment-1... →

I wonder if going ahead with the new driver is a good idea and we try to address the issue of failing tests from Project Browser as a follow up. I know that we are at an edge case due to svelte/ajax, but it's hard to debug where the fault is at.

Right now, Drupal core latest "almost" forces the new chrome driver, but with that, we have fails that we don't with the legacy driver.

Now that #3462681: Add W3C compliant JS testing → has been completed and committed and a new default release of Gitlab Templates has been made, I have reverted from using MR242 and now use the default temlpate branch

The pipeline yesterday had failures for Javascript at Next Major and Previous Minor
https://git.drupalcode.org/issue/project_browser-3469490/-/pipelines/285477

I just rebased the MR (which added the commit from ✨ Change the project_browser.stage.require and project_browser.activate routes to accept a list of project IDs Active made on 16 Sept) and all variant PHPUnit jobs now pass green
https://git.drupalcode.org/issue/project_browser-3469490/-/pipelines/288081

The Nightwatch Next Major tests are no longer failing for 'lack of capabilities' because of the gitlab templates fixed issue mentioned in #14, and the new Selenium driver is being used for Drupal 11.

But as explaied in #9-13 we discovered that The PHPunit tests for Project Browser still need the lagacy driver at Drupal 11, and that is what MR573 is doing. So this is now ready for review.

So I'm not really sure why, but as you can see from latest 2.0.x HEAD - all the PHPUnit are passing even without these changes. Not sure why, but that's good... right?

https://git.drupalcode.org/project/project_browser/-/pipelines/294084

The only thing that we're getting still consistently, seems to be a little new - which is the nightwatch (next major) which is 11.0.x-dev right now, with a ".....waitForElementVisible is not defined."

jonathan1055 → closed merge request !573

The PHPUnit tests are passing because the changes in #3462681: Add W3C compliant JS testing → were committed and new default version of gitlab_templates was released, so that automatically get the fixes. Therefore there is nothing to be done here, so I have closed the MR.

For the actual Nightwatch test failures at 'next major' I suggest you start a new issue, as the work done here on this issue is complete.

For the record, 🐛 Nightwatch "next major" failing Active is the new issue.

Automatically closed - issue fixed for 2 weeks with no activity.