- Issue created by @spokje
- Status changed to Needs review
over 1 year ago 8:22am 22 June 2023 - last update
over 1 year ago CI aborted - 🇬🇧United Kingdom longwave UK
Two groups of changes here:
- The zlib check dates back to Simpletest, and previously ran half the update tests then failed if zlib was not available. If zlib is not available we can't import the compressed database dumps, so we might as well just skip the test early.
- There are four other properties in UpdatePathTestBase that have been refactored away in the move from Simpletest to PHPUnit, so let's remove their declarations.
I previously thought we should refactor away the updateUrl property, but this pattern is used elsewhere in update tests, so I think it can be left alone here.
- last update
over 1 year ago 29,531 pass - 🇮🇹Italy mondrake 🇮🇹
Skip or fail? IMHO skipping make sense when you have 'alternatives'. For instance, you skip the SQLite-specific tests when you're running a different database. Or you skip testing an image format if you do not have the extension able to manage it. But here, if no zlib installed, a significant portion of the tests will simply be ignored - and who knows what a bot has really installed.
- 🇬🇧United Kingdom longwave UK
The "official" way of doing this is with the "@requires extension zlib" annotation, which also skips, but I don't think that works in base classes.
Overall I still think "skip" is appropriate. If you're running the whole test suite in a specific environment, then you expect tests to pass against what is available in that environment. If you're running individual tests or a small group, the report will tell you that the tests were skipped, and you can investigate to find out why.
In a similar way WebDriverTestBase skips instead of fails if it can't start Mink or connect to webdriver, which in some cases means that none of the functional JavaScript tests are actually run.
- 🇬🇧United Kingdom longwave UK
Note that I previously argued the other way for WebDriverTestBase but was convinced by @alexpott and the PHPUnit docs in #3187577: FunctionalJavascript tests should fail when ChromeDriver is not running → that skip is correct.
- 🇬🇧United Kingdom longwave UK
TBH after reading https://phpunit.de/manual/6.5/en/incomplete-and-skipped-tests.html again, maybe we should use "incomplete" to identify random fails and other non-working tests, and "skip" to identify environment specific issues...
- Status changed to RTBC
over 1 year ago 9:59am 22 June 2023 - 🇮🇹Italy mondrake 🇮🇹
Yeah, I assume we could debate forever...
Looks good to me.
- last update
over 1 year ago 29,553 pass - last update
over 1 year ago 29,554 pass - last update
over 1 year ago 29,562 pass - last update
over 1 year ago 29,566 pass - last update
over 1 year ago 29,571 pass - last update
over 1 year ago 29,801 pass - last update
over 1 year ago 29,802 pass - last update
over 1 year ago 29,802 pass - last update
over 1 year ago 29,805 pass - Status changed to Fixed
over 1 year ago 4:57pm 10 July 2023 Automatically closed - issue fixed for 2 weeks with no activity.