Replace Nightwatch testing with another solution

Created on 25 February 2025, 2 months ago

Problem/Motivation

Nightwatch tests have proven to be too shaky for my liking and may not be a good long-term choice.

Details:

Random fails, such as: https://git.drupalcode.org/project/ckeditor5_paste_filter/-/jobs/3888711 and https://git.drupalcode.org/project/ckeditor5_paste_filter/-/jobs/3888757

Outputs errors, but tests still run normally such as: https://git.drupalcode.org/project/ckeditor5_paste_filter/-/jobs/3888793

    Error   Error while running .clearElementValue() protocol action: invalid element state
  (Session info: headless chrome=106.0.5249.103)
  (Driver info: chromedriver=106.0.5249.61 (511755355844955cd3e264779baf0dd38212a4d0-refs/branch-heads/5249@{#569}),platform=Linux 5.10.230-223.885.amzn2.x86_64 x86_64)
    Error   Error while running .clearElementValue() protocol action: invalid element state
  (Session info: headless chrome=106.0.5249.103)
  (Driver info: chromedriver=106.0.5249.61 (511755355844955cd3e264779baf0dd38212a4d0-refs/branch-heads/5249@{#569}),platform=Linux 5.10.230-223.885.amzn2.x86_64 x86_64)

Although Drupal core uses some Nightwatch tests, we don't need to restrict ourselves to the testing tools shipped with core.

Proposed resolution

Look into alternative testing solutions. These can be explored in sub-issues.

Remaining tasks

Try other testing solutions in separate branches
Try running the other tests in GitLab CI
Evaluate pros and cons
Decide on the testing solution for this project

User interface changes

n/a

API changes

n/a

Data model changes

n/a

🌱 Plan

Status

Active

Version

1.0

Component

Code

Created by

🇨🇦Canada star-szr

Live updates comments and jobs are added and updated live.

Comments & Activities

Issue created by @star-szr
Comment 2 months ago →
🇨🇦Canada star-szr
Comment 2 months ago →
🇨🇦Canada star-szr
More random fails:

https://git.drupalcode.org/project/ckeditor5_paste_filter/-/jobs/4490782

https://git.drupalcode.org/project/ckeditor5_paste_filter/-/jobs/4490484
Comment 2 months ago →
🇨🇦Canada star-szr
I did a quick search today and found that core is looking to go in the direction of Playwright, which based on my recent evaluations for a client project would be very high on my list as well.

Tools to check out:

https://github.com/Lullabot/ddev-playwright
https://github.com/julienloizelet/ddev-playwright
Comment 19 days ago →
🇨🇦Canada star-szr
At this point the path seems clear to use Playwright, of course things can change but converting to a task for now.

I am hoping to have some time available in the next week or two to continue this work.

contrib.social Blog FAQ Discussions

Production build 0.71.5 2024