Make Package Manager more friendly to local development by allowing stages to operate directly on the codebase in some situations

Created on 31 January 2025, 5 days ago

Problem/Motivation

There are some cases where we would like people to be able to use Package Manager in a more permissive way that it behaves out of the box. For the purposes of this issue, consider the use case of a simple local development setup (i.e., php -S) of Drupal CMS, being used by a marketer who is not technical and cannot be expected to ever look at a command line.

Package Manager requires Composer and rsync. Currently, those must be detectable in the PATH environment variable, or anything built on Package Manager will crap out with a nasty error. You can configure the paths to these executables in config, but you have to know how to do that (spoiler: it involves the command line, so we've already lost our theoretical non-technical user). That's being improved over in ✨ When it is installed, Package Manager should try to detect the paths of Composer and rsync Active .

What if you don't have rsync, though? What if you don't even know what that is? You are completely prevented from installing modules into your site in a local environment, in which you are the only user, with Project Browser. You are shut out of the ecosystem.

This is a mission-critical problem for Drupal CMS.

Proposed resolution

There are two possible paths here.

  1. Explore alternative methods for copying the Drupal code base to and from the sandbox directory. Are there any reasonable alternatives to rsync that are available on 99.9% of Mac, Windows, and Linux installations?
  2. Allow Package Manager to operate directly on the code base if and only if certain conditions are present. Package Manager would, by default, continue to sandbox everything. But, if a developer set a particular setting (say Settings::get('package_manager_allow_optional_sandboxing') or similar), and a given StageBase subclass declared, via an immutable constant, that it was willing to operate outside of the sandbox, Package Manager would allow it to do so. This means that Project Browser's StageBase subclass could declare that it is willing to operate on the un-sandboxed code base, but Automatic Updates could continue to be stringent.

Remaining tasks

TBD

API changes

TBD

Data model changes

None anticipated.

Release notes snippet

TBD

✨ Feature request
Status

Active

Version

11.0 πŸ”₯

Component

package_manager.module

Created by

πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @phenaproxima
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts
  • πŸ‡¬πŸ‡§United Kingdom catch
  • πŸ‡¬πŸ‡§United Kingdom catch

    So if I'm reading correctly this would be adding a direct-write mode when a flag is set, but only for project browser. The idea being you can composer require new dependencies and the simple act of doing so won't blow up your site, whereas update might.

    Overall that sounds great I think.

    Is there any situation where indirect dependencies might be updated as part of a composer require or does composer always just complain in that case?

  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    Is there any situation where indirect dependencies might be updated as part of a composer require or does composer always just complain in that case?

    Yes, Composer might update indirect dependencies as part of a require, depending on what the dependency solver spits out. So there is an element of risk to this, but I would say that:

    • This is why the direct-write mode would be accessible only to developers.
    • Project Browser could take steps to prevent indirect dependencies from being updated (Composer now accepts a --minimal-changes option, which tries to minimize changes to indirect dependencies; Package Manager could be changed to ensure that Project Browser can pass this option, or maybe Package Manager could just always pass the option regardless).
    • If packages are properly respecting semantic versioning, and using sensible constraints for their own dependencies, then the risk is probably low enough to not be of concern to the non-technical user.
  • Merge request !11076Resolve #3503699 "Make package manager" β†’ (Open) created by phenaproxima
  • Pipeline finished with Failed
    5 days ago
    Total: 606s
    #411677
  • Pipeline finished with Failed
    5 days ago
    Total: 188s
    #411701
  • Pipeline finished with Failed
    5 days ago
    Total: 461s
    #411738
  • Pipeline finished with Failed
    5 days ago
    Total: 714s
    #411745
  • πŸ‡¬πŸ‡§United Kingdom catch

    If packages are properly respecting semantic versioning, and using sensible constraints for their own dependencies (which is true for core, and probably widely true enough for contrib),

    At least two fairly high usage contrib modules, maintained by different people, incremented hook_update_last_removed() in patch releases in the past month or so, preventing updates from e.g. 1.1.1 to 1.1.3.) Just the most recent example I can think of where a patch release led to a site that could not be updated.

    --mininal sounds great to avoid this though.

    Would it be possible to run --dry-run and detect if it includes updates? Would be annoyingly slow though, but maybe only when this mode is on?

  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    Would it be possible to run --dry-run and detect if it includes updates? Would be annoyingly slow though, but maybe only when this mode is on?

    Unfortunately, composer update doesn't give us any machine-readable output, with or without the --dry-run flag. The only way to detect what will happen, is to use the sandbox.

    So as I said, there's some risk involved here, which is why direct-write wouldn't be enabled by default. I think it also makes sense for Package Manager to always pass the --minimal-changes flag to Composer to minimize the possibility of breaking changes being introduced (regardless of whether direct-write is enabled), but we should probably do that in a separate, blocking issue.

    The reason Package Manager doesn't pass --minimal-changes is because I don't think it even existed when we were writing Package Manager. It was only introduced in Composer 2.7.0 (see https://getcomposer.org/changelog/2.7.0).

Production build 0.71.5 2024