Update module can get 'stuck' with 'no releases available'

I'm using #35 patch but it can still get stuck (D9.5.1; D9.5.3).
Delete all works each time: \Drupal::keyValue('update_fetch_task')->deleteAll();

The #35 🐛 Update module can get 'stuck' with 'no releases available' Needs work worked for me on the remote live env - So did the commands from #39 🐛 Update module can get 'stuck' with 'no releases available' Needs work & #41 🐛 Update module can get 'stuck' with 'no releases available' Needs work testing locally. Thanks :)

Updated #35 patch to be compatible with D10.1.5

So, did a review of this and the patch doesn't really fix the problem. If makes things work by accident. Ted's reviews highlight the test failure are correct, this test was working and the test was pointing out this patch wasn't correct.

The problem with the patch is, that during "createFetchTask", project is an array and the value in $this->fetchTasks[$project['name']] is an int. This always fails, again as Ted pointed out.

This means that the reason this "fixes" people's stuck updates is because every page load _always_ requests updates even if there are tasks in the queue. Not great.

So digging deeper I think I understand the problem. It boils down to storing update tasks in two places which can get out of sync, causing updates to no longer trigger.

$this->fetchQueue = $queue_factory->get('update_fetch_tasks');
$this->fetchTaskStore = $key_value_factory->get('update_fetch_task');

This happens when a queue is unable to finish. Because it never finishes, the queue task, is never able to clear itself from the task store.

$this->fetchTaskStore->delete($project_name);

After that happens, there is _no_ way for Drupal to recover and fetch another update for this module.

There may actually be other ways to trigger this? But as long as its possible for the lists to get out of sync, this will be an issue.

Here's some modifications to the test that demonstrate the problem.

Moved the test to the kernel tests as well so it was quicker to iterate on.

Running command drush php:eval "\Drupal::keyValue('update_fetch_task')->deleteAll();" on D 9.5.11 fixed my issue!

Fixed CCF in #47.

Kept running into this, so I made the fix automatic. Slight overkill. Presenting Update NARF! →

Having written that module, a solution dawned for Drupal core that does not involve trying to stop the orphan keyvalue:update_fetch_task.

There are two events which should be coded to delete the queue, and delete any orphans:
- in update_cron(), before running the main check every check.interval_days
- during the user initiated "Check manually" run

In both of these cases, there should be of no concern as to the current state of update_fetch_task as any left overs when either of those is initiated should be considered defunct. Safe to do when there aren't any gummed up tasks too. The keyvalue:update_fetch_task is acting as semaphore to prevent multiple fetch queue items being made, but in both cases here it needs to be guaranteed that every fetch task is run anew, and that semaphore is preventing it from happening. By clearing the queue, and the orphans, the process starts from a clean slate.

While the "Check manually" updates check will always complete due to the user initiated batch, the cron process may take several runs. In that instance, the clearing of the queue and orphans is done on the initial update_cron() run, and all subsequent processing is handled by the update queue runner that's not a queue runner called in update_fetch_data() (maybe that should be made a queue runner eh?). That queue processing should always be complete by the time check.interval_days comes around again.

This doesn't solve the reason that orphan keyvalue:update_fetch_task occur, which seems to be a much deeper intermittent issue related to timing and two data storage systems, but turns this a non-permanent failure which is easily solvable by admin via UI.

Updated the patch from #44 to work with 10.2.2

Both patch #51 and MR 782 seem to only partially fix the issue, it allowed to fetch 83 avor 104 projects for updates instead of 8, but few remained unchecked.

after running drush php:eval "\Drupal::keyValue('update_fetch_task')->deleteAll();"
thet all 104 projects were fetched.