Contrib.social

[policy, maybe patch] Clarify backport policy for testing framework changes, including new BTB tests

Open on Drupal.org →
Created on 27 July 2016, almost 9 years ago
Updated 20 September 2023, almost 2 years ago

Problem/Motivation

Part of 🌱 [meta] Document @internal APIs both explicitly in phpdoc and implicitly in d.o documentation Active .

In general, we do not backport internal refactorings or API additions to stable releases (i.e., they are allowed in minor releases, but not patch releases). Internal refactorings are occasionally allowed at committer discretion (mainly to keep the branches from diverging, if it is non-disruptive and low-risk). API additions can be made in patch releases for experimental modules, but not stable code, for compliance with semver and because they have a small risk of disrupting stable code through method name collisions (etc.).

However, the testing framework is a bit of a special case, because (as far as I know) the testing infrastructure only allows contrib test suites to run against one core branch, which at present cannot even be configured per project. On the other hand, backporting changes to 8.1.x can also be disruptive, if tests that were passing suddenly start failing. https://www.drupal.org/node/2696421 → is the infrastructure policy about testing targets, but I believe it describes an end goal rather than the current implementation.

https://www.drupal.org/core/d8-bc-policy → considers the testing framework public API:

The contents of automated tests provided by core modules are never considered part of the API. While the testing framework itself may be treated as an API individual test classes are always internal.

There is also an interesting case with the new testing API, in BrowserTestBase and JavaScriptTestBase → . BTB was added late in the 8.0.x cycle as an exception to beta restrictions. JSTB was added as a priority feature for 8.1.x. Both APIs are still under active development. Neither, however, are considered experimental -- and we don't have a way to mark non-modules experimental anyway. On the other hand, converting core's test suite to rely on experimental code would also give me pause, so we should consider what we actually think the stability of those APIs is.

Proposed resolution

  • Discuss and document best practices for backporting test changes.
  • Clarify the "experimentalness" (or not) of BTB and JSTB.
🌱 Plan
Status

Closed: outdated

Version

11.0 🔥

Component
PHPUnit  →

Last updated about 4 hours ago

Created by

🇺🇸United States xjm

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024