InstallException when trying to install Pathauto via the ProjectBrowser UI

@rkoller thanks for filing such a detailed issue!!!!!

Do you know if the set up that you described has worked with using Project Browser to install other modules and the problem only happens with path_auto? Or was path_auto the only module that was tested and you hit the this error right away?

I haven't tested the install functionality via the ui the last few weeks. but i've applied the steps to reproduce , setting up another fresh site, and tried to add and install the honeypot module instead this time. i've picked that one because as far as i know it doesnt have any dependencies installed alongside. in contrast pathauto has the token and ctools modules as required dependencies. but i ran into the exact same issue.

on the status report page:

<strong>Failed update detected</strong>
Staged changes failed to apply, and the site is in an indeterminate state. It is strongly recommended to restore the code and database from a backup.

the log message:

<strong>location:</strong> https://pbtwo.ddev.site/admin/modules/project_browser/install-apply/drupal/honeypot/ncmkeeX9enjpXrRuZhdR76yaQHknDa9O2Wz-Y-yXRxw
<strong>referrer:</strong> https://pbtwo.ddev.site/admin/modules/browse
<strong>message:</strong> Drupal\project_browser\Exception\InstallException: The install operation failed to apply. The install may have been partially applied. It is recommended that the site be restored from a code backup.. #0 /var/www/html/web/modules/contrib/project_browser/src/Controller/InstallerController.php(477): Drupal\project_browser\ComposerInstaller\Installer->apply() #1 [internal function]: Drupal\project_browser\Controller\InstallerController->apply('drupal', 'honeypot', 'ncmkeeX9enjpXrR...') #2 /var/www/html/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(123): call_user_func_array(Array, Array) #3 /var/www/html/web/core/lib/Drupal/Core/Render/Renderer.php(583): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() #4 /var/www/html/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(124): Drupal\Core\Render\Renderer->executeInRenderContext(Object(Drupal\Core\Render\RenderContext), Object(Closure)) #5 /var/www/html/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(97): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array) #6 /var/www/html/vendor/symfony/http-kernel/HttpKernel.php(163): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() #7 /var/www/html/vendor/symfony/http-kernel/HttpKernel.php(74): Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object(Symfony\Component\HttpFoundation\Request), 1) #8 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/Session.php(58): Symfony\Component\HttpKernel\HttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #9 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(48): Drupal\Core\StackMiddleware\Session->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #10 /var/www/html/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(106): Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #11 /var/www/html/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(85): Drupal\page_cache\StackMiddleware\PageCache->pass(Object(Symfony\Component\HttpFoundation\Request), 1, true) #12 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(48): Drupal\page_cache\StackMiddleware\PageCache->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #13 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(51): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #14 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(51): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #15 /var/www/html/web/core/lib/Drupal/Core/DrupalKernel.php(685): Drupal\Core\StackMiddleware\StackedHttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #16 /var/www/html/web/index.php(19): Drupal\Core\DrupalKernel->handle(Object(Symfony\Component\HttpFoundation\Request)) #17 {main}

out of curiosity i've then simply went to the cmd line and tried:

$> ddev composer require drupal/honeypot
./composer.json has been updated
Running composer update drupal/honeypot
Loading composer repositories with package information
Updating dependencies
Nothing to modify in lock file
Writing lock file
Installing dependencies from lock file (including require-dev)
Package operations: 1 install, 0 updates, 0 removals
  - Installing drupal/honeypot (2.1.2): Extracting archive
Generating autoload files
Hardening vendor directory with .htaccess and web.config files.
    Skipped installation of bin bin/composer for package composer/composer: file not found in package
86 packages you are using are looking for funding.
Use the `composer fund` command to find out more!
phpstan/extension-installer: Extensions installed
Cleaning installed packages.
No security vulnerability advisories found
Using version ^2.1 for drupal/honeypot

so the composer require in the command line worked without a problem. if i revisit /admin/modules/browse and search the honeypot card, its card button reflects the required honeypot module in the cmd line by showing the install button instead of the add and install button. and when i click the install button the honeypot module is installed without any error.

one additional observation. even though i was able to take the detour via composer require honeypot and then install it via the project browser ui if i try to click any other module via the add and install button i run into another error:

<strong>Location</strong>	https://pbtwo.ddev.site/admin/modules/project_browser/install-begin/drupal/features
<strong>Referrer</strong>	https://pbtwo.ddev.site/admin/modules/browse
<strong>Message: </strong>
Drupal\package_manager\Exception\ApplyFailedException: Staged changes failed to apply, and the site is in an indeterminate state. It is strongly recommended to restore the code and database from a backup.. #0 /var/www/html/web/modules/contrib/automatic_updates/package_manager/src/Stage.php(369): Drupal\package_manager\FailureMarker->assertNotExists() #1 /var/www/html/web/modules/contrib/project_browser/src/Controller/InstallerController.php(419): Drupal\package_manager\Stage->create() #2 [internal function]: Drupal\project_browser\Controller\InstallerController->begin('drupal', 'features') #3 /var/www/html/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(123): call_user_func_array(Array, Array) #4 /var/www/html/web/core/lib/Drupal/Core/Render/Renderer.php(583): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() #5 /var/www/html/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(124): Drupal\Core\Render\Renderer->executeInRenderContext(Object(Drupal\Core\Render\RenderContext), Object(Closure)) #6 /var/www/html/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(97): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array) #7 /var/www/html/vendor/symfony/http-kernel/HttpKernel.php(163): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() #8 /var/www/html/vendor/symfony/http-kernel/HttpKernel.php(74): Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object(Symfony\Component\HttpFoundation\Request), 1) #9 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/Session.php(58): Symfony\Component\HttpKernel\HttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #10 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(48): Drupal\Core\StackMiddleware\Session->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #11 /var/www/html/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(106): Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #12 /var/www/html/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(85): Drupal\page_cache\StackMiddleware\PageCache->pass(Object(Symfony\Component\HttpFoundation\Request), 1, true) #13 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(48): Drupal\page_cache\StackMiddleware\PageCache->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #14 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(51): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #15 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(51): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #16 /var/www/html/web/core/lib/Drupal/Core/DrupalKernel.php(685): Drupal\Core\StackMiddleware\StackedHttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #17 /var/www/html/web/index.php(19): Drupal\Core\DrupalKernel->handle(Object(Symfony\Component\HttpFoundation\Request)) #18 {main}

had another idea. again i've followed all the steps from the steps to reproduce section. only exception i haven't required drush nor composer patches (and therefore no vendor hardening was added as well). just wanted to make sure the error isn't related in any way with composer patches, drush or the vendor hardening. but same effect. same error when trying to add and install path auto. again the same log error it looks like:

Drupal\project_browser\Exception\InstallException: The install operation failed to apply. The install may have been partially applied. It is recommended that the site be restored from a code backup.. #0 /var/www/html/web/modules/contrib/project_browser/src/Controller/InstallerController.php(477): Drupal\project_browser\ComposerInstaller\Installer->apply() #1 [internal function]: Drupal\project_browser\Controller\InstallerController->apply('drupal', 'pathauto', 'oSdJ1BDIkCHB5Y3...') #2 /var/www/html/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(123): call_user_func_array(Array, Array) #3 /var/www/html/web/core/lib/Drupal/Core/Render/Renderer.php(583): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() #4 /var/www/html/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(121): Drupal\Core\Render\Renderer->executeInRenderContext(Object(Drupal\Core\Render\RenderContext), Object(Closure)) #5 /var/www/html/web/core/lib/Drupal/Core/EventSubscriber/EarlyRenderingControllerWrapperSubscriber.php(97): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array) #6 /var/www/html/vendor/symfony/http-kernel/HttpKernel.php(163): Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() #7 /var/www/html/vendor/symfony/http-kernel/HttpKernel.php(74): Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object(Symfony\Component\HttpFoundation\Request), 1) #8 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/Session.php(58): Symfony\Component\HttpKernel\HttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #9 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/KernelPreHandle.php(48): Drupal\Core\StackMiddleware\Session->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #10 /var/www/html/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(106): Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #11 /var/www/html/web/core/modules/page_cache/src/StackMiddleware/PageCache.php(85): Drupal\page_cache\StackMiddleware\PageCache->pass(Object(Symfony\Component\HttpFoundation\Request), 1, true) #12 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/ReverseProxyMiddleware.php(48): Drupal\page_cache\StackMiddleware\PageCache->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #13 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/NegotiationMiddleware.php(51): Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #14 /var/www/html/web/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(51): Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #15 /var/www/html/web/core/lib/Drupal/Core/DrupalKernel.php(685): Drupal\Core\StackMiddleware\StackedHttpKernel->handle(Object(Symfony\Component\HttpFoundation\Request), 1, true) #16 /var/www/html/web/index.php(19): Drupal\Core\DrupalKernel->handle(Object(Symfony\Component\HttpFoundation\Request)) #17 {main}

if you need any other details let me know. am out of ideas not what else i might be able to test and provide more input for the moment.

Can you run composer --version and share the output? 🙏

i am using the head version of ddev without any updates applied to the web containers. the test sites the outputs in the issue summary and comments are based on all are using composer 2.5.3

$> ddev composer --version
Composer version 2.5.3 2023-02-10 13:23:52

for a unknowing reason some of the other test sites i've set up yesterday show version 2.5.2. (still have to figure out why the ones referenced here use 2.5.3 and some others 2.5.2. i haven't changed anything in regards of the composer version.

@rkoller reported something similar in 🐛 Staged database updates messages is unclear Active .

In both cases, the site was running in ddev. This makes me think that there's something about ddev which breaks Package Manager.

We might want to escalate this.

I’ve revisited this issue after @chrisfromredfin posted a Gitlab link providing an instance with the latest automatic_updates and project_browser versions for testing purposes for folks at DrupalCon. Since DrupalPod uses DDEV I was worried people might run into the same issue described in this issue. The surprising part it turned out module installs via the Project Browser UI ran successfully through. The only apparent difference, since Gitpod runs on Linux, is that the Mutagen sync is disabled. I went ahead and disabled Mutagen on my local install (Colima 0.5.4 and the latest version of DDEV HEAD) and set up a new instance. Aside the fact that it took forever, the module install went through without any issue. But the interesting part even if Mutagen is enabled on Gitpod things still work there.

I’ve opened an issue in the DDEV issue queue https://github.com/ddev/ddev/issues/4938 and I’ve discussed and experimented with @rfay alongside on the DDEV Discord server. With the steps outlined in https://github.com/ddev/ddev/issues/4938#issuecomment-1556060005 the issue was consistently reproducible on MacOS with Colima and Mutagen enabled with the MountType sshfs and 9p, Docker Desktop and Randy was also able to reproduce the failure on Ubuntu with Mutagen enabled.

It was possible to get things working with ddev delete -Oy (deletes Mutagen sync and Docker volume) or a ddev mutagen reset after the base install is set up. And it might also work with a ddev restart as well.

According to Randy it has nothing to do with OS or even the Docker provider. It fails the same way on Docker Desktop, Colima and Linux.

Even though the problem is narrowed down now, for figuring out the root cause and understand the problem entirely he asked if it would be possible to get a brief summary what happens when a module is installed via the Project Browser UI? What directories are being used? And where is the code being staged? that would help with debugging and the problem could then be forwarded upstream to the Mutagen maintainer.

Randy added, when i ran the draft for this comment by him, the best might be if he could have a talk with a knowledgeable maintainer. It doesn't even seem to him to be something that happens during the installation, because once you fix it, it stays fixed. So the question is what happens before that? Pretty weird overall.

Composer has specifically had to work around problems on virtualized filesystems — for example: https://github.com/composer/composer/issues/9627

Can you try setting COMPOSER_RUNTIME_ENV=virtualbox?

I've created a new test setup and added

web_environment:
 - COMPOSER_RUNTIME_ENV=virtualbox

in the project's config.yaml and then went through the rest of the steps outlined in the issue summary in https://github.com/ddev/ddev/issues/4938. unfortunately that didn't fix the issue. i still ran into the identical installexception.

Thanks for eliminating that!

I'll cross post my answer from the issue on the ddev queue. I was able to narrow down the root cause further. I've remembered that there was the option to use rsync as the file syncer when @tedbow mentioned rsync in the corresponding thread on the drupal slack. i've followed the steps setting up a new test instance (https://github.com/ddev/ddev/issues/4938) except i've entered the following two lines between step 6 and 7 :

ddev drush config:set package_manager.settings file_syncer rsync
ddev drush config:set package_manager.settings executables.rsync /usr/bin/rsync

with rsync active i was able to install pathauto , webform and metatag in a row and no error happened. so the root cause for the error is somewhere in the php file syncer?

another detail @rfay pointed out and i just tested and am able to reconfirm. with the PHP file syncer active, after setting up the site, right before the last step installing a module, if you run ddev mutagen reset && ddev start, you are able to avoid the error with the PHP file syncer as well. perhaps that helps to narrow things further down.

Wow. There must be something about the order of file I/O operations then. Or perhaps rsync include special system calls to inform the operating system for the need of a sync of a virtual file system? 🤔

It is probably untwined now. @tedbow opened an issue for composer stager yesterday https://github.com/php-tuf/composer-stager/issues/162. in the corresponding thread on the drupal slack (https://drupal.slack.com/archives/C7QJNEY3E/p1684353557138949) he was uncertain if that issue covers only a single aspect or if it covers all problems listed in that thread. but @rfay confirmed yesterday in the following comment in the issue on the ddev queue that it probably actually does and that permissions are the key to the problem: https://github.com/ddev/ddev/issues/4938#issuecomment-1558201204 . problem is mutagen does not manage permissions. the details could be found in randy's comment as well as in the comments in composer stage issue ted opened explaining the different btw php and rsync file sync.