Visual regression testing

A pro is - they can help to discover problems earlier.
A con is - they require planning, set up and maintenance resource.

The value comes in the scenarios where a) a developer is working on one component or global style and unintentionally affects something else that they did not think was in scope of the change. c) a component is used in a way not intended by the designer or developer and layout issues occur as a result.

Some names to throw into the discussion:

Backstop.js - and it would be great to support the project.
Another very good tool is Percy by Browserstack, has an Open Source programme I believe.
Storybook has ddev support and helps to maintain design system integrity. I believe it also ships with comparison tools.

Pragmatic solution: cypress. We are already using it.

Optimistic solution: playwright. We are already planning to use it in core at 📌 Consider dropping Nightwatch in favor of Functional Javascript tests Active , but we don't have the infra in place yet.

Both solutions might need to have the screenshots in the repo, which might make its size grow by a bit.

Other external options that would avoid the screenshots in the repo: tugboat. Wondering how the existing drupalorg integration might help here.

@penyaskito I must take tugboat for a spin!

With all these tools and different use cases, we should define which tools we intend to use for which purposes and what we mean by Visual Regression Testing.

+1 for backstop.js, we're using that for all our projects and can only tell good things about it.

Both solutions might need to have the screenshots in the repo, which might make its size grow by a bit.

For that I'd recommend Git LFS which is supported by GitLab and probably available as a git extension for most developers. It keeps the binaries out of the git repo and stores them in the file system. But to the developer, the process is completely transparent so that they don't have to bother about it at all.

I love visual regression testing so I hope this happens :)

Agreed! Once we've got the visuals locked in this should be top priority.

I am happy to help with providing Diffy (https://diffy.website) license for this initiative. We specialize on Drupal projects so have quite a few integrations already (DDEV, Tugboat). I am happy to assist with setting up the integration.

Here is how we are different from BackstopJS https://diffy.website/backstopjs

All the assets are stored in Amazon S3 so there won't be any issues with size of the screenshots.

How can I learn about the way you do development of the project to see how we can assist?