Problem/Motivation
Or more generically: Drupal's nightwatch.conf.js does not allow specifying a subdirectory to run tests from. It only supports two things:
- all tests in the entire Drupal site
- all tests in the entire Drupal site + tests one level higher
IOW: it is impossible to specify a directory inside the Drupal site whose tests to run. And this happens to be the strategy that the d.o GitLab CI template uses to run Nightwatch tests for contrib modules:
- yarn --cwd $_WEB_ROOT/core test:nightwatch $CI_PROJECT_DIR/$_WEB_ROOT/modules/custom/$CI_PROJECT_NAME/tests
Quoting core/.env.example:
# The path that Nightwatch searches for assumes the same directory structure as
# when you download Drupal core. If you have Drupal installed into a docroot
# folder, you can use the following folder structure to add integration tests
# for your project, outside of tests specifically for custom modules/themes/profiles.
#
# .
# ├── docroot
# │ ├── core
# ├── tests
# │ ├── Nightwatch
# │ │ ├── Tests
# │ │ │ ├── myTest.js
#
# and then set DRUPAL_NIGHTWATCH_SEARCH_DIRECTORY=../
#
#DRUPAL_NIGHTWATCH_SEARCH_DIRECTORY=
— committed in
#2869825: Leverage JS for JS testing (using nightwatch) →
and never updated since.
Steps to reproduce
On GitLab CI, if you are using the d.o GitLab template:
$ sudo BABEL_DISABLE_CACHE=1 DRUPAL_NIGHTWATCH_SEARCH_DIRECTORY=modules/custom -u www-data yarn --cwd ./core test:nightwatch
yarn run v1.22.19
$ cross-env BABEL_ENV=development node -r dotenv-safe/config -r @babel/register ./node_modules/.bin/nightwatch --config ./tests/Drupal/Nightwatch/nightwatch.conf.js
Browserslist: caniuse-lite is outdated. Please run:
npx update-browserslist-db@latest
Why you should do it regularly: https://github.com/browserslist/update-db#readme
Error: Cannot read test source location: /builds/project/acquia_migrate/docroot/core/modules/custom../CUSTOM_MODULE_NAME/tests/src/Nightwatch/Tests
The only work-around that is possible:
--tag=something
(Typically "something" would be the name of a specific module.)
Which requires your Nightwatch tests to contain:
…
module.exports = {
'@tags': ['something'],
…
Or on your local machine, run all Nightwatch tests of all installed contrib modules:
cd /path/to/drupal/root/that/contains/index.php
DRUPAL_NIGHTWATCH_SEARCH_DIRECTORY=modules/contrib yarn --cwd ./core test:nightwatch
Proposed resolution
Tweak the logic in nightwatch.conf.js to accept not only supersets (go up the directory tree) to find Nightwatch tests, but also subsets (go inside a directory tree, such as the contrib modules directory, the custom modules directory, or one specific module directory).
Hot fix on macOS:
sed -i '' 's/const key = \`\.\.\/.*/const moduleDirectory = m[1];/' core/tests/Drupal/Nightwatch/nightwatch.conf.js
sed -i '' 's/\${searchDirectory}\${key}/\.\.\/\${searchDirectory}\/\${moduleDirectory}/' core/tests/Drupal/Nightwatch/nightwatch.conf.js
Hotfix on Linux (and GitLab CI — which does not seem to like backticks):
sed -i 's/const key = \`\.\.\//const key = \`/' $CI_PROJECT_DIR/$_WEB_ROOT/core/tests/Drupal/Nightwatch/nightwatch.conf.js
sed -i 's/\${searchDirectory}\${key}/\.\.\/\${searchDirectory}\/\${key}/' $CI_PROJECT_DIR/$_WEB_ROOT/core/tests/Drupal/Nightwatch/nightwatch.conf.js
Remaining tasks
User interface changes
API changes
Data model changes
Release notes snippet
