[policy] What do our stability levels mean?

Created on 11 December 2024, 10 days ago

Drupal CMS is unlike most other Drupal things -- it has no APIs or update path. So what, exactly, does alpha/beta/RC/stable mean in the context of this project? We need to define these as a matter of policy.

To start discussion, I propose the following:

  • Alpha: not something we do. We don't ever have to tag an alpha; since we have no update paths or APIs, it doesn't mean much from a stability perspective.
  • Beta: "early preview" -- nothing in a beta is guaranteed to make it into the next release.
  • Release candidate: in an RC, the set of recipes in it is finalized. No recipes will added or removed except as indirect dependencies. Modules/themes may still be added or removed, and functionality may change to some degree during this phase.
  • Stable: no recipes or secondary dependencies are added or removed. Functionality doesn't change, apart from what's needed to fix buggy behavior.
πŸ“Œ Task
Status

Active

Component

General

Created by

πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @phenaproxima
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts
  • πŸ‡¬πŸ‡§United Kingdom longwave UK

    With my security team hat on I'd also like this policy to ideally define what this means in terms of security coverage. This means that although Drupal CMS 1.0 will soon have a stable release version, it seems unlikely that all of the components will actually be covered by the security team, because some dependencies currently only have alpha or beta releases. When evaluating Drupal CMS, it might be important for users to know that we provide no guarantees of security with some of the components, and we should certainly not be giving a false sense of security.

    Additionally, some organisations have policies that mean they cannot use alpha or beta quality software in production. So even though they might be interested in Drupal CMS 1.0, they will be unable to use it due to the stability of the dependencies.

    In the future I would really like us to aim to only use stable versions of dependencies, with security coverage, in stable versions of Drupal CMS - this isn't possible today but perhaps it is something we can aim for in the future, which would alleviate my concerns here.

  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts
  • πŸ‡¬πŸ‡§United Kingdom catch

    , it seems unlikely that all of the components will actually be covered by the security team, because some dependencies currently only have alpha or beta releases

    It's currently worse than this, because there are dependencies where the entire project has it not opted into security coverage, has never had a stable release, and is currently alpha. So even if those modules make stable releases they would also need to opt-in to security coverage.

    Drupal CMS is unlike most other Drupal things -- it has no APIs or update path.

    Sites need to be able to update from their initial stable Drupal CMS install to new versions of Drupal core and contrib.

    So if Drupal CMS includes a module at alpha1, and that module releases alpha2 with a config or database schema change and no upgrade path (which they can do because they're alpha), or completely abandons that branch and starts a new branch with a different schema/API again (which they can do because they're alpha), then those sites will break or get permanently stuck.

    And IMO that will not be the module's fault, because it clearly labelled itself as alpha and not for use on production sites, but Drupal CMS's fault, because it labelled itself as stable while including alpha code with no guarantee of an upgrade path.

    I can see arguments for including beta-stability code where that's the least worst option, say https://www.drupal.org/project/dashboard β†’ where there is a close relationship between Drupal CMS and the project maintainers, and (probably) a commitment to providing an upgrade path for sites. From a security perspective that could possibly be covered by widening the scope of security coverage to include some modules. But if an alpha module is at that point, it should just tag a beta anyway, and if it's not, then it shouldn't be included.

  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    I tend to agree that including alpha-stability dependencies willy-nilly is a bad idea.

    For the record, as of this writing, our current alpha dependencies are:

    • Webform: this is an enormously popular project and will take stability seriously; it was only tagged alpha since it's a new minor branch with Drupal 11 support. The maintainer is easy to reach and work with, so this is a pretty safe case of including an alpha dependency.
    • AI Agents, AI Image alt text, AI installer: These were all built by the track leads for the AI track, with whom we are working closely. So it's analogous to Dashboard; there's little risk here.
    • Project Browser: Another one where we have a very close relationship with the maintainers, and this is on a trajectory to being included with core, so it's relatively low-risk (although API breaks might still be necessary with this one, they is increasingly likely to be frowned upon).

    There are also some modules that are effectively stable, but tagged as alphas for various idiosyncratic reasons. I tend to agree with the concept that the maintainers should be willing to deliver a beta, with clearly defined expectations, as a condition of inclusion in Drupal CMS.

  • πŸ‡¬πŸ‡§United Kingdom longwave UK

    There are also some modules that are effectively stable, but tagged as alphas for various idiosyncratic reasons

    Similarly Gin has had 15 release candidates over two years, and Klaro has had 14 RCs over 18 months - surely it's time for a stable releases?

  • πŸ‡¬πŸ‡§United Kingdom catch

    So it's analogous to Dashboard; there's little risk here.

    Dashboard is beta and opted into security coverage, these modules are alpha and are not opted into security coverage.

  • πŸ‡©πŸ‡ͺGermany jurgenhaas Gottmadingen

    @longwave, yes, both Gin and Klaro ate working on stable releases before Drupal CMS 1.0. Same for for ECA 2.1

Production build 0.71.5 2024