Staged database updates messages is unclear

What's interesting about this is that, as far as I know, these are two unrelated errors.

The "possible updates detected", in this situation, refers to the fact that system.install is different (i.e., differing SHA-256 hash) in the Drupal site than in the staged copy of it that contains the update to 10.0.3.

The second error, on the other hand, is a catastrophic error that we throw if, for some reason, the staged copy cannot be completely copied into the live Drupal site. It means that the codebase is potentially incomplete and not viable. It has nothing to do with the state of the database, or any database updates that need to be applied.

With regard to the first issue, I think it's time we re-evaluated that UX. IMHO the right thing to do here is only raise an error about it during an unattended (cron) update, since we do not want to run database updates during cron. But, in the UI, I'm not sure there's any reason we need to mention it. If there are database updates, we already automatically redirect to update.php when the update finishes.

As for the second error...we really gotta figure out how that happened. Personally I'd like to see some better logging put in so that we can actually get some insight into why the copy failed.

@rkoller, Thanks for filing this.

if you compare it with the previous screenshot of the status update page there where no necessary database indicated therefore it is unexpected to see a warning about possible database updates detected in the system extension. from a user perspective i don't know what to do in this situation, no database updates indicated on the status report page and getting a warning like this that leads me into a sort of freeze state not knowing what to do.

I think we need fix the database relate messages here.

The intention of this message is to let you know that your staged update has a DB update detected. The idea being that you should know that when you finish the update(if it had been successful) you will be redirected to update.php to run the updates.

Some users might not be prepared to run these updates if their site is being used in production. They might not continue the update knowing they will need apply updates and want to do it later.

Of course the current message doesn't convey this and we should fix this.

Also, I should mention that the 2.x branch of Automatic Updates is more or less abandoned at this point in favor of a Drupal core-targeted 3.0.x version. That's undergoing heavy refactoring right now. So I doubt this will be fixed in the short term, but it's good to have this feedback on record.

We still to fix this 3.0.x or in core if it gets merged first. For now I have marked this as core-post-mvp which means we could get it into core without this fix since it will still be experimental in core.

So not adding our sprint tag, but anybody can work on approving this message and we could review.

@phenaproxima oh they are unrelated? that is unexpected. interesting! if i can help and shall test anything in particular let me know. i am able to reproduce the error consistently.
and thanks for the headsup. i've taken the latest branch available on the project page. i've read about things moving over to 3.0.x but since i havent found any note in that regard on the project page i've tried 2.x first. but i've quickly spun up an ddev instance with 10.0.0 version with an identical setup now - only difference i've used the 3.0.x branch of automatic updates. but i ran into the identical error but with slightly different details, a few line numbers differ it looks like. but maybe the output is of any help:

An AJAX HTTP error occurred.
HTTP Result Code: 500
Debugging information follows.
Path: /batch?id=3&op=do_nojs&op=do
StatusText: error
ResponseText: The website encountered an unexpected error. Please try again later.Drupal\automatic_updates\Exception\UpdateException: The update operation failed to apply completely. All the files necessary to run Drupal correctly and securely are probably not present. It is strongly recommended to restore your site's code and database from a backup. in Drupal\automatic_updates\Updater->apply() (line 119 of modules/contrib/automatic_updates/src/Updater.php). PhpTuf\ComposerStager\Infrastructure\Service\Filesystem\Filesystem->copy(Object, Object) (Line: 151)
PhpTuf\ComposerStager\Infrastructure\Service\FileSyncer\PhpFileSyncer->copySourceFilesToDestination(Object, Object, Object) (Line: 48)
PhpTuf\ComposerStager\Infrastructure\Service\FileSyncer\PhpFileSyncer->sync(Object, Object, Object, NULL, 600) (Line: 37)
PhpTuf\ComposerStager\Domain\Core\Committer\Committer->commit(Object, Object, Object, NULL, 600) (Line: 399)
Drupal\package_manager\Stage->apply(600) (Line: 116)
Drupal\automatic_updates\Updater->apply() (Line: 131)
Drupal\automatic_updates\BatchProcessor::commit('wmfyuNqWwtTHGuVAwhL2LABlsbHVio33OMbBVrqwyXA', Array) (Line: 295)
_batch_process() (Line: 137)
_batch_do() (Line: 93)
_batch_page(Object) (Line: 55)
Drupal\system\Controller\BatchController->batchPage(Object)
call_user_func_array(Array, Array) (Line: 123)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 580)
Drupal\Core\Render\Renderer->executeInRenderContext(Object, Object) (Line: 121)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->wrapControllerExecutionInRenderContext(Array, Array) (Line: 97)
Drupal\Core\EventSubscriber\EarlyRenderingControllerWrapperSubscriber->Drupal\Core\EventSubscriber\{closure}() (Line: 163)
Symfony\Component\HttpKernel\HttpKernel->handleRaw(Object, 1) (Line: 74)
Symfony\Component\HttpKernel\HttpKernel->handle(Object, 1, 1) (Line: 58)
Drupal\Core\StackMiddleware\Session->handle(Object, 1, 1) (Line: 48)
Drupal\Core\StackMiddleware\KernelPreHandle->handle(Object, 1, 1) (Line: 106)
Drupal\page_cache\StackMiddleware\PageCache->pass(Object, 1, 1) (Line: 85)
Drupal\page_cache\StackMiddleware\PageCache->handle(Object, 1, 1) (Line: 48)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle(Object, 1, 1) (Line: 51)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle(Object, 1, 1) (Line: 51)
Drupal\Core\StackMiddleware\StackedHttpKernel->handle(Object, 1, 1) (Line: 675)
Drupal\Core\DrupalKernel->handle(Object) (Line: 19)

@tedbow ahhhh the message means that the update will entail mandatory database updates. for me the message sounded like the current database requires some database updates BEFORE the update can run!

i wonder if it is really necessary to name which extension (system) requires the mandatory database update? it doesnt really help me as someone who isn't a developer to know that the extension is system. or that it it is about extensions. the most important pieces of information are that this is not something that is blocking your update but instead it is a heads up for you after the update that a database update is required. and maybe a word what the user should be mindful about or plan for after the update.
I would added a remark and word of caution, something like: Before you proceed, be aware, after the update is successfully applied an immediate mandatory database update will be required.. That way for one you would communicate that this isn't a warning about an error preventing the update but instead a heads up. and by using mandatory and immediate it would communicate that there is no other option. with that message i would be prepared of the next upcoming step. in the current form if i would have run into it in a real life scenario i wouldnt have proceeded and did some research hunting for a potential error in my database.
But i would add a second sentence about the scenario you've described. i am just uncertain about the recommended approach. should the user be aware of some potential downtime in maintenance mode or would the recommendation be not to apply the update in production? i am just not sure about the best recommendation. and on a side note. could that update be applied unattended? 10.0.0 to 10.0.3 is a patch release so it would be updated automatically, isn't it? how would that work in combination with that mandatory database update?

The message you are suggesting looks good/perfect for this current situation(Database updates warning) but the message is generic for all the REQUIREMENT_WARNINGs, so we can't use the message you are suggesting.

re #9 🐛 Staged database updates messages is unclear Active : oh didn't knew that. what other scenarios would cause that this message is shown? if i know the other potential causes leading to that message then i might try to update the message accordingly

I am also able to reproduce the second error consistently.

Thanks for the research, @yash.rode!

That's enough for now, we need to focus on the core-mvp-tagged issues first, and in particular the 15 new ones at #3316368-23: Remove our runtime dependency on composer/composer: remove ComposerUtility → 🙏