Clarify/replace 'stage' terminology in Package Manager

Created on 17 September 2024, 8 months ago

Problem/Motivation

Follow-up to ✨ Add Alpha level Experimental Package Manager module Needs review .

Package manager currently uses the word 'stage' to mean several things.

1. A 'staging environment' as in composer stager. @phenaproxima mentioned changing this to sandbox in https://git.drupalcode.org/project/drupal/-/merge_requests/3608#note_376550

2. As a verb, i.e. to 'stage' the stage/sandbox environment.

3. As a synonym of 'phase' or 'step', or the object which manages the lifecycle of a staging directory.

For example:
StageEvent

"The stage event during which this exception is thrown."

"Stage operations"

I personally found this quite confusing when reviewing the code, it's hard to scan something and see what it's referring to at any one time.

Steps to reproduce

Proposed resolution

Not sure. I think when referring to a staging/sandbox directory, it would be good to consistently add 'directory' every time.

But if we can use actual different words for different things (sandbox, phase, step?) that would be better.

Remaining tasks

Opening this as a beta blocker because it is very likely to involve some class renames, even if the only consumers of the API are automatic updates and project browser - but that could get complicated with the contrib versions.

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

πŸ“Œ Task
Status

Active

Version

11.0 πŸ”₯

Component
BaseΒ  β†’

Last updated 42 minutes ago

Created by

πŸ‡¬πŸ‡§United Kingdom catch

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @catch
  • πŸ‡¬πŸ‡§United Kingdom catch
  • πŸ‡¦πŸ‡ΊAustralia larowlan πŸ‡¦πŸ‡ΊπŸ.au GMT+10
  • πŸ‡ΊπŸ‡ΈUnited States benjifisher Boston area

    We discussed this issue at πŸ“Œ Drupal Usability Meeting 2025-04-11 Active . That issue will have a link to a recording of the meeting.

    For the record, the attendees at the usability meeting were benjifisher, rkoller, and simohell.

    We are not familiar with the code base and the various ways that "stage" is used, but we tried to suggest some guidelines for naming things.

    First, we agree with the first suggestion in the Proposed Resolution. If adding "directory" is too long, then use "staging" or "staged", not "stage" in the context of the staging directory. For example: use StagingBase or StagingDirectoryBase instead of StageBase.

    Second, use "phase" or "step" instead of "stage" when you can. Use "step" if it is possible to move back, or "phase" if only one direction is possible.

    I hope those suggestions are helpful!

  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    A few proposals:

    • StageBase should become SandboxManagerBase, to more accurately describe what it does. Hopefully, this will completely disambiguate the concept of an object that manages the existence and lifecycle of a sandbox. Generically these will be referred to as "sandbox managers"
    • "Sandbox" should refer to the temporary copy of the Drupal project, as a whole. This is more common term than "stage", but refers to the same concept, and is less likely to be confused with a verb.
    • "Sandbox directory" specifically refers to the directory where the sandbox is located.
    • "Create", "require", "apply", "post-apply", and "destroy" can be consistently referred to as "phases", e.g. "the create phase", "the apply phase", etc.
  • πŸ‡ΊπŸ‡ΈUnited States tedbow Ithaca, NY, USA

    #5 sounds good to me

  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    This is a huge scope of work. I'd like to propose that we split up the work here into two issues.

    • In this one, which should block beta, we just rename the classes as appropriate, but let "stage" persist in comments.
    • In another one, which can be stable blocking, we change all comments, but no runtime code whatsoever, to use the updated terminology.
  • Pipeline finished with Failed
    22 days ago
    Total: 721s
    #476085
  • Pipeline finished with Failed
    22 days ago
    Total: 640s
    #476100
  • Pipeline finished with Canceled
    22 days ago
    Total: 298s
    #476103
  • Pipeline finished with Failed
    22 days ago
    Total: 116s
    #476107
  • Pipeline finished with Failed
    22 days ago
    Total: 117s
    #476111
  • Pipeline finished with Failed
    22 days ago
    Total: 404s
    #476113
  • Pipeline finished with Canceled
    22 days ago
    Total: 1758s
    #476129
  • Pipeline finished with Canceled
    22 days ago
    Total: 366s
    #476153
  • Pipeline finished with Success
    22 days ago
    Total: 721s
    #476160
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    I think this is ready for an initial review. It doesn't change any commentary, only runtime code.

    • StageBase is now SandboxManagerBase.
    • Events that have a $stage property now have a $sandboxManager property.
    • getStageDirectory() and stageDirectoryExists() are now getSandboxDirectory() and sandboxDirectoryExists().
    • Any parameters or local variables named $stage are now $sandbox_manager.
    • Exceptions that began with "Stage" now start with "Sandbox" (e.g. StageException is now SandboxException).
    • The idea of "operations" vs. "phases" is still fuzzier than I'd like -- I think that create, apply, and destroy should be considered "phases", and anything that happens between create and apply should be considered an "operation". But having said that, PreOperationStageEvent was named years ago, before we understood that the "pre" events are central to validation. But since that is their superpower (the ability to log errors), I have renamed that to SandboxValidationEvent to clarify its purpose.
    • A couple of tests and test methods were renamed, but generally the tests were not altered.
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    Re-titling for clarity. @catch endorsed the two-issues idea in Slack.

  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    Opened πŸ“Œ Update all comments and documentation in Package Manager to refer to "sandboxes" and related terminology Postponed to do the documentation and comment changes.

  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts
  • πŸ‡¬πŸ‡§United Kingdom catch

    #5 looks really good to me, it's immediately obvious what the new terms are likely to refer to, even when I'm not sure what the old terms refer to - so that's an instant improvement in terms of both self-documenting naming as well as disambiguation.

    Given the module is still alpha, and this is a blocker for beta, splitting this into code-naming + docs references seems fine to keep things reviewable. We can probably get both issues done quicker than trying to do it in one go.

  • Pipeline finished with Canceled
    22 days ago
    Total: 132s
    #476293
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    While discussing this with @catch in Slack, I realized that RequireEventTrait could be better-named. There is nothing about it that specifically has anything to do with a composer require operation -- if, say, we implemented a composer remove capability in the future, and wanted to dispatch a pre- and post-event for that, then we'd want to also use something like RequireEventTrait, but it would be confusingly named for that scenario.

    I've renamed it to the clearer and more accurate EventWithPackageListTrait. This could have been done in its own issue, but while we're getting everything ready to go with better permanent names in this one beta-blocking issue, I figured now was the easiest time to do it.

  • Pipeline finished with Success
    22 days ago
    #476294
  • πŸ‡¬πŸ‡§United Kingdom catch

    Went through the MR twice, there are no logic changes, just the renames. All the renames make sense to me. In some places you can see the mis-match with the docs that will be updated in the follow-up, but even with the mismatch it's clearer because it's more obvious what's being referred to.

    We won't completely remove the stage/stager concept because that's also in composer stager, but I think it's fine if we're 'staging sandboxes' using a Drupal adjacent third party library - composer stager has less concepts to balance than package manager / automatic updates / project browser.

    If we find other small naming issues we can always address those during beta or future minor releases after stable with deprecations, but the large number of changes here shows why this needs to happen during alpha.

    Automatic updates ships its own version of package_manager still, so should be able to handle any compatible issues smoothly by the release that removes the package_manager module also ensuring full compatibility with core package manager.

    For sites that include both automatic updates + project browser, the transition from contrib package manager to core package manager will also be handled by automatic updates, but project browser itself will likely need versions for before/after 11.2 - that will have to happen for a Drupal CMS release, and the quicker we do all of this the better so we don't have to do it when things are even further along.

    Moving this to RTBC which means I can't commit it, but if it's still in the RTBC queue without objections in a week or so, additional RTBCs would let me commit it in that case.

  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    Adjusting credit.

  • πŸ‡ΊπŸ‡ΈUnited States tedbow Ithaca, NY, USA

    @phenaproxima thanks for this. I think it is great improvement!

    @catch thanks for the detailed review.

    I still see a lot of variables that start with `$stage*` or are just `$stage` . and function names like `\Drupal\Tests\package_manager\Kernel\PackageManagerKernelTestBase::createStage`

    Since this issue just deals with class names and the follow-up πŸ“Œ Update all comments and documentation in Package Manager to refer to "sandboxes" and related terminology Postponed do we want to expand this issue update variable and function names or create a follow-up for that too?

  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    I think renaming local variables and test methods can be done in a follow-up. I'm pretty sure Package Manager's tests aren't part of its API, and therefore we have some latitude to change them later. But I'll try get them done before 11.2 beta1 anyway.

  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    Opened πŸ“Œ Remove minor usages of "stage" from Package Manager's runtime code Active to do more clean-up.

  • πŸ‡¦πŸ‡ΊAustralia larowlan πŸ‡¦πŸ‡ΊπŸ.au GMT+10

    I checked this out locally and searched for Stage in the package manager code base. There's still a lot of references. A lot are to the ComposerStager classes from the PHPTuf namespace and can be ignored. But there's still some class names and methods in runtime (non test) code that use Stage and not Sandbox. I think we should address at least the class names, constants and methods here. There are also some local variables. Not super fussed on those, that could be a low priority follow-up for a Novice - a lot of these are only in tests.

    • \Drupal\package_manager\Validator\StagedDBUpdateValidator has a method checkForStagedDatabaseUpdates but it appears to be checking the sandbox for database updates. Should it also be renamed?
    • \Drupal\package_manager\Validator\StageNotInActiveValidator is checking if a sandbox is part of the project root - should it be renamed too? It adds an error with text Stage directory is a subdirectory of the active directory. - that is neither a runtime class or a code comment so I'm not sure which phase that should be changed in - are we missing a third stage to this issue to update user facing strings or should we do that here?
    • \Drupal\package_manager\ComposerInspector::validateExecutable has a variable $stage_dir should that now be $sandbox_dir
    • Should we rename \Drupal\package_manager\SandboxManagerBase::TEMPSTORE_DESTROYED_STAGES_INFO_PREFIX?
    • \Drupal\package_manager\SandboxManagerBase::rethrowAsStageException throws a SandboxException - should we rename that method too?
    • \Drupal\package_manager\SandboxManagerBase::create has a variable $stage_dir that refers to a sandbox. So do \Drupal\package_manager\SandboxManagerBase::apply and \Drupal\package_manager\PathExcluder\SiteConfigurationExcluder::syncDefaultSiteDirectoryPermissions - rename?
    • \Drupal\package_manager\SandboxManagerBase::require has a local function in variable with name $do_stage should that be renamed? Inside that function it has a variable $stage_dir that refers to a sandbox, rename?
    • \Drupal\package_manager\Validator\ComposerPatchesValidator::validate has a number of local variables with stage in the name but that refer to sandboxes - rename? Same for \Drupal\package_manager\Validator\DuplicateInfoFileValidator::validate, \Drupal\package_manager\Validator\EnabledExtensionsValidator::validate and \Drupal\package_manager\Validator\LockFileValidator::validate about another half-dozen validator classes in that namespace.
    • \Drupal\package_manager_test_api\ApiController::create has a local variable $stage that refers to a sandbox manager.
    • \Drupal\package_manager_test_api\ApiController::__construct(stage) should this property be renamed? It contains a SandboxManager now
    • Numerous other instances of local variables named $stage in test code that refer to sandboxes.
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    \Drupal\package_manager\Validator\StagedDBUpdateValidator has a method checkForStagedDatabaseUpdates but it appears to be checking the sandbox for database updates. Should it also be renamed?

    Renamed to SandboxDatabaseUpdatesValidator, and its test-only version to TestSandboxDatabaseUpdatesValidator.

    \Drupal\package_manager\Validator\StageNotInActiveValidator is checking if a sandbox is part of the project root - should it be renamed too? It adds an error with text Stage directory is a subdirectory of the active directory. - that is neither a runtime class or a code comment so I'm not sure which phase that should be changed in - are we missing a third stage to this issue to update user facing strings or should we do that here?

    Renamed to SandboxDirectoryValidator, and changed the user-facing string.

    The rest of your comments seem to be about local variables and test code, which will be handled in
    πŸ“Œ Remove minor usages of "stage" from Package Manager's runtime code Active . Same goes for anything with private visibility, since none of those things are part of our API.

  • Pipeline finished with Failed
    17 days ago
    Total: 156s
    #479708
  • πŸ‡¬πŸ‡§United Kingdom catch

    I had to double check these two:

    Should we rename \Drupal\package_manager\SandboxManagerBase::TEMPSTORE_DESTROYED_STAGES_INFO_PREFIX?
    \Drupal\package_manager\SandboxManagerBase::rethrowAsStageException throws a SandboxException - should we rename that method too?
    

    Both of those are private, so could indeed be handled in πŸ“Œ Remove minor usages of "stage" from Package Manager's runtime code Active . The rest are more obviously in scope for that issue.

    The two new renames look good to me, so moving this back to RTBC.

  • Pipeline finished with Failed
    16 days ago
    Total: 606s
    #480165
  • πŸ‡¦πŸ‡ΊAustralia larowlan πŸ‡¦πŸ‡ΊπŸ.au GMT+10

    Committed to 11.x - thanks!

  • πŸ‡¦πŸ‡ΊAustralia larowlan πŸ‡¦πŸ‡ΊπŸ.au GMT+10

    Used some cli commands to knock up a rough change record https://www.drupal.org/node/3521441 β†’

    • larowlan β†’ committed c7908f76 on 11.x
      Issue #3474876 by phenaproxima, catch, tedbow, larowlan, benjifisher:...
Production build 0.71.5 2024