Use symfony/runtime for less bespoke bootstrap/compatibility with varied runtime environments

Drupal core is moving towards using a “main” branch. As an interim step, a new 11.x branch has been opened → , as Drupal.org infrastructure cannot currently fully support a branch named main. New developments and disruptive changes should now be targeted for the 11.x branch, which currently accepts only minor-version allowed changes. For more information, see the Drupal core minor version schedule → and the Allowed changes during the Drupal core release cycle → .

It is required to run Drupal using https://github.com/dunglas/frankenphp-drupal/issues/1 to get the best perf

I created https://github.com/el7cosmos/drupal-runtime a while ago that works with non-persistent app servers at that time (e.g. php-fpm, nginx unit, apache httpd)

One more report of usage!

https://github.com/dunglas/frankenphp-drupal/issues/1#issuecomment-17501...

Another +1 for runtime.

I used it recently on https://www.drupal.org/project/sm → with success. It cleans up setting up Symfony Console commands remarkably https://git.drupalcode.org/project/sm/-/blob/1.x/bin/sm?ref_type=heads

+1 it would allow to use worker mode to boost performances considerably

Took some force-pushes (one day I'll get FunctionalTests working locally again) but the MR is now green across the board :raised_hands:

Updated the issue summary to help reviewers.

Forgot to snip a reformatted paragraph, whoops.

For a start this will likely need framework manager review.

I tried to understand how runtime works and I tried to contribute my fresh understanding: I submitted https://github.com/symfony/symfony-docs/pull/20909 and created https://app.infinitymaps.io/maps/mnJJ9FR4FND. I hope this helps someone.

For now this MR adds a dependency and a bunch of new code.
That new code looks quite complex if you are not familiar with the symfony/runtime package (which I am not, at this point).

The supposed payoff is that we might later be able to split logic out of DrupalKernel.
But the current MR does not make this obvious.

We should have a proof-of-concept MR that shows how this change would help us to refactor and simplify DrupalKernel.
(This is the only reason stated in this issue, I don't see other benefits mentioned)

We also need to evaluate if there are alternatives (as in custom code) that do not add a dependency.

The quickest example I can give you is the pull request for 📌 [PP-1] Ensure asynchronous tasks that run beyond the length of a request have the chance to complete before process exit Active . Other items I'm excited to work on, but they require a bit more puzzling. I hope that in addition I can add more information/rephrase my case and that can hopefully lead to a more concrete understanding (and possibly an updated issue summary).

DrupalKernel is currently a monolith and is responsible for:

1. Setting PHP environment configuration in bootEnvironment
2. Initializing settings and determining which multi-site if any we're in
3. Initializing global variables for application context
4. Handling requests

This worked for Drupal because, for the most part, the only use-case we were considering is the simple request response flow where there is a 1:1 mapping of process to request. However, in modern web applications we're seeing that that no longer holds true. A websocket server may have a long running application that handles many different requests. Similarly Drush might execute Drupal code and not have a request at all for its tasks. These kinds of use cases are discussed in #2218651: [meta] Make Drupal compatible with persistent app servers like ReactPHP, PHP-PM, PHPFastCGI, FrankenPHP, Swoole → .

For the adoption of Revolt and enabling of asynchronous applications we must introduce a call to EventLoop::run. By definition, this call must be "global" because in an application there can only be a single EventLoop. The EventLoop has the task to schedule different cooperative tasks and calling EventLoop::run from within an EventLoop::run call causes a fatal error because their scheduling would conflict.

This was discussed in 📌 [PP-1] Ensure asynchronous tasks that run beyond the length of a request have the chance to complete before process exit Active where I tried to find the best possible location. From Drupal's single-request perspective the most logical place would be in the Kernel at the end of handle and the end of terminate. However, this would cause a problem for developers who want to build multi-request applications, or applications in which a Drupal request handler may only be a small part. Their application can not function if EventLoop::run is buried within the Drupal Kernel. The logical conclusion for that then is that the EventLoop::run calls should be placed outside of the Kernel, but this would require it to be placed in index.php, update.php, the related scaffold files. Additionally, anyone that would build their own application that does not use those files would need to remember to include EventLoop::run in the right location for their calls of Drupal to work.

If we look at the list of responsibilities for DrupalKernel then we can see a similar tension for items 1 through 3. They're tasks that should really only happen once and they should happen in a specific manner for Drupal to work. The only consistent place that currently exists in Drupal to satisfy those requirements is DrupalKernel, because if we move them out, then we risk someone not doing the environment initialization separately.

To reduce the amount of responsibilities in DrupalKernel safely, we need a way to extract functionality from it while ensuring that it remains easy for ourselves (as contributors to Drupal Core) but also for application developers who integrate Drupal to properly set-up the environment that Drupal will run in.

We could write that code ourselves but symfony/runtime was created by the Symfony maintainers specifically to solve that problem:

What if we could decouple the bootstrapping logic of our apps from any global state?

This PR makes it possible via a new proposed symfony/runtime component.

The smart thing about the symfony/runtime component, which we would be hard-pressed to replicate ourselves without arriving at the same code, is that it decouples the runtime/global state and application in such a way that you can switch out either side. This puts Drupal in a position where Drupal can provide a runtime that would work in PHP-FPM, as I do in my proposed PR, but it can also provide a runtime that would let Drupal work within a console application, like the drupal CLI tool that is shipped with Drupal core.

Then anyone would be able to write a server or console application that uses Drupal and could use Drupal's out of the box runtime and be sure that their environment is set-up correctly. However, if they have specific needs and want the environment to be set-up in a different way then they can also change the runtime itself, while still use the Drupal PHP-FPM application or console tool.

We can already see an example of the necessity for a consistent environment in the Drupal CLI tool. We see that the InstallCommand and ServerCommand, both have a boot method which goes through steps 1-3 outlined at the start of this comment (and RecipeCommand and RecipeInfoCommand do the same through BootableCommandTrait).

I'm still puzzling a little bit of how to untangle DrupalKernel (though I think it's a worthwhile effort) but I'm confident that we can clean up the code for those commands as well.

1. Move DrupalKernel::bootEnvironment into DrupalRuntime::__construct (different environments can swap out the runtime if needed): 📌 Extract DrupalKernel::bootEnvironment into SAPI adapter Closed: duplicate
2. Move DrupalKernel::findSitePath to DrupalRuntime. This should most likely be requestable as an argument from the main function and passed into the kernel since it's required for the Kernel to boot. That standardises the logic, but allows other tools to easily overwrite it if needed.
3. Application root and initializeRequestGlobals would similarly be moved as per #2529170-104: [PP-1] Remove DrupalKernel::initializeRequestGlobals and replace base_root, base_url and base_path with a service →
4. By moving the site path detection out of the Kernel we can do the same for Settings::initialize from DrupalKernel::initializeSettings and make the settings available as front-controller argument from our DrupalRuntime:getArgument as well. That would make it trivial to write a Drush-like application that reads settings

I think there's more possible. For example some of the environment is now set based on drupal_valid_test_ua but we may want to switch out the runtime altogether in those. Some of that also requires untangling some existing global state before it becomes clearer.

I realise that's quite a long post which still assumes some knowledge of symfony/runtime. I think the main takeaway is the clear delineation and swapping mechanism that symfony/runtime provides between environment and application.

If environment is E and application is A, then the DrupalKernal is currently (EA), a very tightly coupled environment and application. With symfony/runtime this turns into

E
---- symfony/runtime
A

This makes it easy to change E for E' (in case you have different path requirements around file system structure or front-controller for example), but it also makes it easy to swap A for A' (in case of Drupal console or Drush for example).

The next steps that I'll work on:

1. Expand the PR for this issue to include updating the Drupal console script
2. Work on some of the related issues as stacked MRs to show how they would be implemented on top of the runtime.

The Needs Review Queue Bot → tested this issue. It no longer applies to Drupal core. Therefore, this issue status is now "Needs work".

This does not mean that the patch necessarily needs to be re-rolled or the MR rebased. Read the Issue Summary, the issue tags and the latest discussion here to determine what needs to be done.

Consult the Drupal Contributor Guide → to find step-by-step guides for working with issues.