Run phpunit tests from a single job in parallel

Created on 28 June 2023, almost 2 years ago

Problem/Motivation

Some phpunit jobs take really long because they execute dozens (or hundreds or...) of tests and assertions and the job takes too long, maybe even times out.

It'd be great to leverage parallel run within the same job. The library https://github.com/paratestphp/paratest offers exactly that with minimal configuration, so it'd be worth exploring it.

Steps to reproduce

If you want to test it, all you need to do is to add the following at the bottom of your .gitlab-ci.yml

.show-environment-variables: &show-environment-variables
    - |
      echo -e "\e[0Ksection_start:`date +%s`:show_env_vars[collapsed=true]\r\e[0KShow Environment Variables"
      env | sort
      echo -e "\e[0Ksection_end:`date +%s`:show_env_vars\r\e[0K"

.simpletest_db: &simpletest-db
  - |
    [[ $_TARGET_DB_TYPE == "sqlite" ]] && export SIMPLETEST_DB=sqlite://localhost/sites/default/files/db.sqlite
    [[ $_TARGET_DB_TYPE == "mysql" ]] && export SIMPLETEST_DB=mysql://$MYSQL_USER:$MYSQL_PASSWORD@database/$MYSQL_DATABASE
    [[ $_TARGET_DB_TYPE == "pgsql" ]] && export SIMPLETEST_DB=pgsql://$POSTGRES_USER:$POSTGRES_PASSWORD@database/$POSTGRES_DB

.phpunit-base:
  script:
    - *simpletest-db
    - *show-environment-variables
    - mkdir $_WEB_ROOT/sites/simpletest
    # @todo Use Apache instead of PHP web server.
    - cd $_WEB_ROOT && php -S 0.0.0.0:8888 .ht.router.php >> webserver.log 2>&1 &
    # Provide some context on the test run.
    - vendor/bin/drush status
    # Finally, execute tests.
    - composer require --dev brianium/paratest
    - vendor/bin/paratest --functional -v --bootstrap $PWD/web/core/tests/bootstrap.php $_WEB_ROOT/modules/custom --log-junit junit.xml $_PHPUNIT_EXTRA
    # - vendor/bin/phpunit --no-interaction --printer="\Drupal\Tests\Listeners\HtmlOutputPrinter" --bootstrap $PWD/web/core/tests/bootstrap.php $_WEB_ROOT/modules/custom --log-junit junit.xml $_PHPUNIT_EXTRA

90% of those lines are just copy/paste from the original template, the key lines that were changed were

- composer require --dev brianium/paratest
- vendor/bin/paratest --functional -v --bootstrap $PWD/web/core/tests/bootstrap.php $_WEB_ROOT/modules/custom --log-junit junit.xml $_PHPUNIT_EXTRA

Before (30 min): https://git.drupalcode.org/project/api/-/jobs/52555
After (18 min): https://git.drupalcode.org/project/api/-/jobs/53625

--

So far, I've only found one potential case that the library doesn't support, which is the use of custom phpunit printers, which we use here
https://git.drupalcode.org/project/gitlab_templates/-/blob/1.0.x/include...

I made a comment on an open issue to see if they'd allow for this option to be overriden: https://github.com/paratestphp/paratest/issues/771

Proposed resolution

Maybe consider adopting the new package as the way to run the tests to leverage multicore server runners. Not sure if this needs to bubble up to "core" for discussion.

Remaining tasks

User interface changes

API changes

Data model changes

✨ Feature request
Status

Active

Component

gitlab-ci

Created by

πŸ‡ͺπŸ‡ΈSpain fjgarlin

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Production build 0.71.5 2024