Add End to End (e2e) testing

Great choice, Cypress is a joy to use and very easy to get started with. I asked about testing last year in the Drupal Forum, and @jaypan kindly made me aware of Cypress → and how it works. Together, we created Browser testing using Cypress → .

I support this, and it's definitely within my domain, so self-assigning.

I experimented with this a bit and I found Cypress is very complicated to set up within DDEV if you want to see the tests actually executing in front of your eyes. It has to be able to talk to an X11 server running on the host machine, which has to be set up in a particular way and...sheesh. And that was just on a Mac. I don't know how onerous it would be on Windows.

So I think we might want to set some realistic expectations here. Testing on Cypress in CI is probably within reach; adding it to our local development is also reachable but having the tests be visible while they run is tricky at best, and I'm not sure we'll be able to make it non-tricky.

@phenaproxima not sure if that helps, but here is a snippet from our docker compose config, that we generate automatically with a composer plugin so that this works for each local dev project easily. These projects are not on DDEV, but in a docker compose project with Traefik as the frontend which routes the domain to the right container.

  cypress:
    image: 'cypress:latest'
    environment:
      - CYPRESS_baseUrl=https://eca.dev.jurgenhaas.de
      - CYPRESS_mailpitUrl=https://mailpit-eca.dev.jurgenhaas.de
      - PHP_CONTAINER=prjname-php-1
      - DISPLAY=:0
    network_mode: host
    working_dir: '/e2e'
    volumes:
      - /etc/timezone:/etc/timezone:ro
      - /etc/localtime:/etc/localtime:ro
      - /var/run/docker.sock:/var/run/docker.sock
      - /var/www/drupal/projects/prhname/tests/:/e2e
      - ~/.Xauthority:/root/.Xauthority:ro
    entrypoint: cypress open --project /e2e
    user: '1000:997'

@phenaproxima: As I wrote, it was easy to get Cypress up and running in Debian 12 (my OS) and see the tests run, following the simple steps in https://www.drupal.org/docs/develop/automated-testing/browser-testing-us... → ... There were no extra set up.

Perhaps it's because I am using Debian, and not Mac? I created https://github.com/tyler36/ddev-cypress/issues/43 to clarify this.

I don't think Cypress is more established. One caveat to Cypress over Playwright is that you write less "normal" scripting in your tests and are tied to their plugin and locator system. Playwright has more straightforward locators built in, such as getByLabel, getByTitle, getByRole.

Note: I don't mean to say "don't use it", I just wanted to at least share my opinion.

Welp. That took some doing, but I have the basic infrastructure for Cypress set up:

• It's now a dev dependency of the project. It gets automatically installed, if needed, when DDEV is started.
• The DDEV web container now includes Chromium and Firefox. Regular Chrome turned out to be a big pain in the ass to install, so I figured Chromium was close enough.
• I'm not using Cypress' Docker images because I was advised by members of the Experience Builder team that it doesn't work for our kinds of tests. They went down that road and it leads straight to hell.
• Cypress config and support code will be kept at the top level of the project (for now, anyway); tests should go in the individual recipes, like drupal_cms_blog/tests/e2e/something.cy.js. There's only one test right now and it's pretty useless, but it at least proves that Cypress runs properly.
• To run the Cypress tests in DDEV, it's ddev exec npx cypress run. You can also do it via the UI, as long as you've got an X11 server running on the host machine (such as XQuartz).
• I also now have a GitLab CI job specifically to run Cypress tests.

With this change, I think the decision is clear that we won't do end-to-end testing in PHP. At all. And it's just as well -- functional JS tests in PHP are flaky and tricky to write.

So, you wanna do non-JS functional testing, or testing things at the Drupal API level? PHPUnit is perfectly fine; we have functional, unit, and kernel tests. But for end-to-end stuff, Cypress is the way.

And, merged. Thanks, folks! This is one of those merge requests where every line was hard-won, but I think this will be worth it in the long run. The fact that we can collaborate with Experience Builder to build a great Drupal-specific framework for end-to-end testing in Cypress is just more evidence of how awesome the collaboration has been in the Starshot initiative. Onward!

This is great!

• Perhaps Cypress can get a short section, covering the basics in the README, like PHPUnit has?
  https://git.drupalcode.org/project/drupal_cms/-/wikis/Contributing-to-Dr...
• If it's not necessary to add the node_modules folder to Git, a Cypress .gitignore file could be added?
  https://www.drupal.org/docs/develop/automated-testing/browser-testing-us... →
• In stead of having Cypress-related files and folders in the root of the project (cluttering it), it could be placed in the /test folder?
  https://www.drupal.org/docs/develop/automated-testing/browser-testing-us... →

Automatically closed - issue fixed for 2 weeks with no activity.

As a huge proponent of Cypress, I am very happy to see this. I, without any knowledge of this action, put forth a a presentation proposal for Drupalcon 2025 to present Cypress and Drupal, so if my presentation is accepted, this will be excellent information to also relay as part of the presentation.

I experimented with this a bit and I found Cypress is very complicated to set up within DDEV if you want to see the tests actually executing in front of your eyes. It has to be able to talk to an X11 server running on the host machine, which has to be set up in a particular way and...sheesh. And that was just on a Mac. I don't know how onerous it would be on Windows.

I set up Cypress to use 'ddev drush' as the Drush command, and it has worked well.

That sound great @jaypan, I very much hope your Drupalcon 2025 presentation proposal gets accepted.

You could also consider making a "Using Cypress in Drupal" YouTube channel, going through the basic steps of setting Cypress up in Drupal with DDEV, and some short examples. I would certainly be watching the videos!

I asked Is it possible to clarify the benefits? #43 of the DDEV-cypress plugin originally used in this MR. @tyler36 confirmed that MacOS and Windows can have a hard time with X11, as can be seen in the work-in-progress README from Clarify benefits and use cases in README.md #44 , which will probably soon be committed.