Support non proc_open composer

Created on 20 September 2022, almost 3 years ago
Updated 14 January 2025, 6 months ago

Clicking on "Update" I got the following error message:

Error message
Your site does not pass some readiness checks for automatic updates. It cannot be automatically updated until further action is performed.
The Process class relies on proc_open, which is not available on your PHP installation. See the help page for information on how to configure the path to Composer.

I host this site on a private shared server that doesn't have proc_open enabled and this was caught in the readiness check.

Which hosting provider did you test Automatic Updates with? eg. BlueHost, SiteGround, etc.

I host this on a private cloud instance from Hetzner that is personally configured by our DevOps team.

What was your goal? For example, what versions of Drupal core were you trying to update from and to?

I was trying to update from version 9.4.0 to version 9.4.6

What were the steps you took, starting from when you added the Automatic Updates module to your Drupal project (see https://www.drupal.org/i/3275810#install-module β†’ for instructions)? Include anything that seems relevant, including commands you ran, links you clicked on, output logs, relevant config files, screenshots, screen recordings, etc.

I did exactly what was layed out in the instructions here 🌱 Automatic updates 8.x-2.x testing guide for hosting environments Closed: outdated and I only enabled proc_open after it prompted that it needed it.

What was your environment like?
Please answer as many of the following questions as you can.
Are you using Lando, DDEV, or another Docker-based environment?:

No

What operating system were you using?

Ubuntu 20.04.4 LTS with kernel version 5.4.0-94-generic

What web server are you using?

Nginx version 1.23.1

What PHP version are you using (see https://www.drupal.org/i/3275810#confirm-php-version β†’ )?

PHP 7.4.30 (fpm and cli)

What version of Composer are you using (see https://www.drupal.org/i/3275810#confirm-composer-version β†’ )?

Composer version 2.4.2

Did you accomplish your goal? Were the instructions clear? Did you observe any bugs or errors, or other issues? Did you need to do any workarounds?

I did not. Even after I solve the proc_open issue the update batch process results in a 500 error with a generic error of losing connection to the database. I am currently investigating whether this is a PHP memory limit issue that kills the process during the update.

Problem with these instructions? Anything else we should know?

None that I can think of.

✨ Feature request
Status

Active

Version

11.0 πŸ”₯

Component

ajax system

Created by

πŸ‡¨πŸ‡ΎCyprus EliasPapa

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • πŸ‡¬πŸ‡§United Kingdom catch

    Moving this to the core queue.

    I have no idea how widespread disabling proc_open on shared hosting (or other hosting environments) is in 2025, but I think it would be useful to try to get data from at least a few major hosting providers.

    Depending on that, and reports like the original one here from people trying to run automatic_updates/project_browser on hosting without proc_open support, we could then consider what to do.

    It would probably require a fallback that bypasses Symfony's Process component altogether - but unless we re-implement the process component or swap out some classes, that probably means calling composer API functions directly instead of via the cli.

  • πŸ‡¬πŸ‡§United Kingdom catch
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    An idea: I have no idea if this would actually work or be remotely feasible, but we can't use proc_open, then maybe we can take advantage of the fact that Composer is itself built on Symfony components. We could perhaps use Symfony Console's command testing harness to invoke the various Composer commands CLI-style, but without creating an actual process. This means we'd need to bring in Composer's PHP code base as a runtime dependency, but wouldn't necessarily need the Composer binary itself.

  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    So I tested out the idea in #14 locally and it seems to work. I'm attaching the tiny proof-of-concept module I created to implement this.

    If we implement this in core (doesn't seem like it'd be too hard), then combined with the addition of direct-write mode ✨ Make Package Manager more friendly to local development by allowing stages to operate directly on the codebase in some situations Active , it would be possible to do some Package Manager stuff without needing Composer to be preinstalled on the system. It could be a runtime dependency of the project.

  • Pipeline finished with Failed
    about 1 month ago
    Total: 226s
    #501148
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    @catch pointed out in Slack that doing this would not necessitate adding composer/composer to core's runtime dependencies. Package Manager would keep trying to use an external binary, unless it detects composer/composer in the local code base, in which case it would try to use that instead.

    This means we could add composer/composer as a runtime dependency of, say, drupal/cms (or maybe even core's project templates), but it would not actually need to be a true core dependency.

  • Pipeline finished with Failed
    about 1 month ago
    Total: 527s
    #501155
  • Pipeline finished with Failed
    about 1 month ago
    Total: 391s
    #501161
  • Pipeline finished with Running
    about 1 month ago
    #501172
  • Pipeline finished with Failed
    about 1 month ago
    Total: 260s
    #501386
  • Pipeline finished with Failed
    about 1 month ago
    Total: 143s
    #501391
  • Pipeline finished with Failed
    about 1 month ago
    Total: 235s
    #501393
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    I like the idea of this being a fallback. Re-titling to clarify that.

  • Pipeline finished with Failed
    about 1 month ago
    Total: 221s
    #501404
  • Pipeline finished with Canceled
    about 1 month ago
    Total: 218s
    #501406
  • Pipeline finished with Failed
    about 1 month ago
    Total: 1016s
    #501407
  • Pipeline finished with Canceled
    28 days ago
    Total: 400s
    #513255
  • Pipeline finished with Canceled
    28 days ago
    Total: 121s
    #513260
  • Pipeline finished with Failed
    28 days ago
    Total: 1092s
    #513262
  • Pipeline finished with Success
    28 days ago
    Total: 544s
    #513317
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts
  • Pipeline finished with Success
    28 days ago
    Total: 807s
    #513340
  • Pipeline finished with Canceled
    28 days ago
    Total: 229s
    #513363
  • Pipeline finished with Canceled
    28 days ago
    Total: 159s
    #513368
  • Pipeline finished with Success
    28 days ago
    Total: 488s
    #513371
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    Been thinking more about this. Now that I have a working patch, I am actually not sure if we want to do this at all, for a few reasons:

    • Having proc_open disabled seems to be, to at least some degree, an edge case. We haven't heard this complaint too much. We get far more complaints about Composer and rsync not being auto-detectable, which is something we are trying to ameliorate in other issues.
    • We are very clear here that proc_open is by far the best way to run Composer, since it prevents weird behavior and the heightened risk of OOM and timeout errors. We already have documentation in our hook_help that says what to do if proc_open is disabled.
    • If you have proc_open disabled, not only can you not use Composer; you can't use rsync either, and rsync is still required, in almost all cases, for Package Manager to work. It doesn't make any sense to allow a fallback method of executing Composer without having something similar for rsync -- but that brings us back to the "PHP file syncer" days, when Composer Stager tried to reimplement the relevant parts of rsync in PHP, and we ended up throwing out all that work (and it was a shit ton of work). Even if we ended up using a file syncer that was not rsync (like xcopy, for example), it'd still need proc_open.
    • This would be yet another example of the "having more than one way to do things" syndrome that core has historically suffered from. Maybe we don't want that.
    • The implementation (using a test harness as an execution driver) is clever, and we could rely on it, but it is kind of...sideways. I doubt Symfony would be expecting anyone to use their test harness for this purpose, so we are at slightly higher risk of breaking changes in Symfony Console.
    • I didn't spend too much time on this, and I am okay throwing out my work here if it doesn't, ultimately, make strategic and technical sense overall.

    I therefore leave this one up to @catch and other framework managers. Is there really a benefit to implementing this, or is it really a poison pill in disguise?

  • Pipeline finished with Success
    28 days ago
    Total: 552s
    #513431
  • πŸ‡¬πŸ‡§United Kingdom catch

    So composer/composer updates make this a bit frightening. But having to re-implement rsync tips this into 'is it worth it?' territory, and combined with the other points, I don't see a good reason to do this. At least we have good documentation for why not. Going to close this as won't fix (briefly discussed with @phenaproxima in slack too), but also adding credits.

Production build 0.71.5 2024