Rework database update tests so we don't have to ship database dumps in git

Created on 23 November 2023, 7 months ago
Updated 19 June 2024, 11 days ago

Problem/Motivation

See #3305003: Create standard steps for creating database dumps โ†’ and โœจ Improve DX of maintaining migration database fixtures: provide an option for creating per-table database fixtures in DbDumpcommand Needs work for some background.

Right now we ship database dumps compressed with core. Updating these is complicated and error prone - i.e. to remove a module from a dump you have to load it into the version of core that the database dump was created with, uninstall the module, then re-export the dump. But the re-exporting might need to be made with the 11.x version of the script, not the 9.4.x version of the script, to incorporate improvements to the dump script. It's equally complicated updating for example an 8.9.x dump to a 9.4.x dump when we remove database updates.

I don't have a fully developed idea for how this would work, but I think we can possibly replace this if we rely on gitlab a bit more. See also this discussion in slack:

https://drupal.slack.com/archives/C1BMUQ9U6/p1700048273974479

1. Have a gitlab pipeline that installs, via a build test, the version of Drupal we want to update from. In the build test, create the content that we want to update (save nodes, comments, taxonomy terms, users, create some extra fields of various types etc.).

2. Somewhere in the build test, export the populated database to a dump file.

3. Save the dump file either as an artifact of the file itself, or into a container.

4. In the actual update test, we then rely on the database dump file, load it and update from that.

This way steps #1-3 can happen once per week, once per month etc. instead of every single time that tests run - so even though it'll be a lot of processing going on creating the dump, it'll happen infrequently.

To modify the dump output, we can update the test to add extra/different content and/or update the test to install a different version of Drupal.

We can also produce multiple database exports - for example a 9.4.4 dump, but also a 9.4.4 dump upgraded to 9.5.11. Or a 9.5.11 dump from a fresh install. If the tests work across core versions, this could even be done via a data provider.

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

๐Ÿ“Œ Task
Status

Active

Version

11.0 ๐Ÿ”ฅ

Component
PHPUnitย  โ†’

Last updated about 19 hours ago

Created by

๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom catch

Live updates comments and jobs are added and updated live.
  • Security

    It is used for security vulnerabilities which do not need a security advisory. For example, security issues in projects which do not have security advisory coverage, or forward-porting a change already disclosed in a security advisory. See Drupalโ€™s security advisory policy for details. Be careful publicly disclosing security vulnerabilities! Use the โ€œReport a security vulnerabilityโ€ link in the project pageโ€™s sidebar. See how to report a security issue for details.

Sign in to follow issues

Comments & Activities

Production build 0.69.0 2024