#Starshot blocker

In this issue, we are not implementing a queue. That is probably the desired end behavior, but would have been too much here and is not really in scope.

That probably makes this comment out of scope but also reading it made me realise something.

If/when this moves to a 'real' queue, it might need to be a single queue item that holds the various operations to do, then the queue runner starts at the top of the operations, and updates (or replaces) the queue item, and if it can't finish, then what's left is in the queue item for another runner to pick up.

This is because if the operations were split into different queue items, they could end up getting processed out of order.

Manual testing (thus far):

1. The "Clear Selection" button no longer works
2. The count of projects being installed (bottom toolbar) does not return to 0 when install completes
3. (and Installs don't work after the first batch is installed)
4. Reloading the page very early after starting does not solve the issue (no sandbox ID yet)

Lets land 📌 Stop automatic storage creation of body field Active and see what's left here.

This one is going to need heavy manual testing.

This was hard to do and required a major overhaul of the installation system.

Here's the gist:

• In this issue, we are not implementing a queue. That is probably the desired end behavior, but would have been too much here and is not really in scope. Queue or no queue, the request is to make the installer pick up where it left off. A queue was not, strictly speaking, necessary for that. We can implement a true queue in a follow-up, as a minor release feature request. That should not block a stable release.
• The InstallState class, whose purpose was somewhat mysterious (in addition to being underused) has now come into its own as the critical backend piece of the installation system: it is the source of truth about what projects we are installing, and what phase of installation they are at (for which we use the ActivationStatus enum, as it turned out to be a good fit here). The backend always passes the installation state to the frontend in its success responses, and only accepts it from the frontend once -- when the user initiates the install process (at that point, the frontend is the source of truth since it is the only thing that knows what the user's intention is). That's why the "begin" route is now a POST request.
• The frontend now has a coherent InstallationManager service that coordinates selecting and installing projects. It dispatches three events that components can listen to: install-selection-changed (a project was added to the list, or removed from it), install-start, and install-end (self-explanatory). The code that makes HTTP requests to the backend was refactored and simplified for clarity.
• In order for the install to proceed, we need to keep the sandbox ID. That is kept in sessionStorage, so it will persist for the same tab (and thus survive page reloads), but not across tabs or in different windows. If we want that, we can use localStorage instead, but that's not in scope here.

, it should a progress response (I've done X of Y tasks you've asked me to do)

An issue here is the queue system doesn't have an API for finding out how many items there are in the queue, how many items were added by the current user or anything like this. This means either a generic 'still working...' message (which is probably the most accurate and safest option) or having to implement something between queue and batch API.

Using the queue system would be a big step towards unblocking ✨ Allow package_manager to work with non-web-writable directory permissions Active .

On running down the queue, neither of these actually have code unfortunately but ✨ Add an 'instant' queue runner Needs work and ✨ Add a non blocking batch runner Active may have ideas.

One possibility for project_browser would be when adding items to the queue, add something to the user's session saying they've added something to the queue.

Then when this session key exists, add an AJAX queue runner to the page that processes the queue one by one (using something like the process route that @phenaproxima describes). When it doesn't find anything in the queue, it would unset the session key so that there's no an AJAX request all the time.

Need to allow for the possibility that different browser tabs, or cron, run down the queue as well.

I agree this is a stable blocker, but I think it will necessitate an overhaul of how installs are handled on the backend, which will imply significant changes to the underlying frontend code too.

The behavior we seek to implement is a queue. When you press "Install", the projects you've chosen go into a queue. Drupal already has an API to handle queued tasks.

So let's say you put three projects into the queue and kick it off. As far as Project Browser is concerned, here's what has to happen:

1. A sandbox needs to be created (or reclaimed if one already exists).
2. A project has to be required into the sandbox.
3. Then another.
4. Then the last one.

When that's done, a couple of final tasks tasks:

1. Sandboxed changes must be applied.
2. The sandbox needs to be marked for cleanup (Package Manager already has a queue for this).

Each of these things is an item in a queue, and we do as many of them as we can safely do in any given request. That's how queues work in Drupal, and this is all relatively straightforward to implement.

Meanwhile, the frontend code needs to be able to keep asking Drupal: "hey, how's my queue going?" The backend needs to reply with one of:

• All done!
• Still N things to go.
• All done, here's what you need to know about what just took place (e.g., here are some AJAX commands for you to run).

So what does this look like from a technical perspective?

Well, first we need to refactor the "begin" route so that it creates a sandbox and initializes a queue. It will continue to return the sandbox ID, just as it does today.

Then we have two more routes:

• require: This adds another project to the queue and immediately returns.
• process: This tries to drain the queue. If it succeeds -- meaning everything is installed -- it has to apply the sandboxed changes, activate everything, and return a response with AJAX commands. If the queue still has things in it, it should a progress response (I've done X of Y tasks you've asked me to do) -- in which case, the frontend should wait a bit (5 seconds?) and call it again.

The "apply" route will go away entirely, and the "activate" route might not be necessary either -- we'll see.