Expand the a11y nightwatch coverage in Core

Suggest first step is to use 'standard' install profile rather than the limited 'nightwatch_a11y_testing', even if we have to skip tests until the issues are fixed.

The failing test looks to the related to image resizing and maybe related to:
https://www.drupal.org/project/drupal/issues/3484845 🐛 [random test failure] ImageUrlProviderTest::testResize Active

@the_g_bomb Grepping the core codebase I notice there are not apparently any existing usages of the 'standard' Nightwatch profile showing up. What is the 'standard' profile? Is that a configuration that has been standardised for Drupal? Or is it the default configuration profile that Nighwatch itself provides whether Nighwatch is being used on Drupal, Wordpress etc?

It is just the standard installation profile, so it sets the site up as everyone else sets it up. Wasn't aware that other tests don't use that, but I suspect that without doing that we won't check the accessibility of many tools included by default.
If I can get this running, it may be worth setting up umami_demo as well.
Basically it should just run the installer using the profile at:
/profiles/standard

#8 Yes I was confused. I thought it was a configuration profile for Nightwatch itself. But I suppose umami could be useful at some point. The article edit url seems fine especially since there is an existing url for the 'Page' node type that uses the same query param.

Wondering if trying without the query param (I read something about the query param being required to POST the form?) which i think would just load the form? To try to isolate the cause: is it the query param in the url? I think the issue is currently isolated to the article edit url being added to the test. If we can isolate it to the query param that would be slight progress?

#8 I notice that core.entity_view_display.node.article.default.yml version in Nightwatch profile contains core.entity_view_display.node.article.default.yml
But it is not contained in the same file in standard profile. Havent looked at the umami version of the file yet.

¯\_(ツ)_/¯ Sometimes the tests don't work until you retry them.

@the_g_bomb I hate that when you go all around the houses. Life sucks sometimes : )

FYI, I think it would be a good idea to move the existing Nightwatch tests to either Playwright or PHPUnit as appropriate when they are available. Just keen to get the current tests working as optimally as possible in the meantime.

FYI, there was some concern about using the standard profile when the nightwatch a11y tests were created: #3293469-23: Automated A11y tests in Nightwatch →

Yeah, I did think there would be a reason for using a slimmed-down profile, but then as the a11y tests are testing the user experience rather than specific functions or features, using a controlled environment might not test what users actually experience.

In fact, there is a case that freshly installed Drupal may not have enough things in place to ensure everything is accessible. e.g. the homepage list of articles with pagination.

To be honest, I suspect we may want each profile to have its own set of tests. I would love to see umami_demo and the drupal cms out-of-the-box experience covered by some a11y tests.

Thanks for the feedback, looking forward to seeing how you get on with PHPUnit.