[meta] Use Fibers for concurrency in core

#2218651: [meta] Make Drupal compatible with persistent app servers like ReactPHP, PHP-PM, PHPFastCGI → was mentioned in #5 but not added as related issue, doing so now

In my opinion #2218651: [meta] Make Drupal compatible with persistent app servers like ReactPHP, PHP-PM, PHPFastCGI → is the more concrete problem to tackle. Fibers feels like more of an implementation detail. In most cases I think Drupal should lean on libraries that abstract that away for us.

https://clue.engineering/2021/fibers-in-php is a good article on the topic for anyone not familiar with Fibers (added to the IS too).

Relevant: @matt glaman's experiments with making BigPipe concurrent using fibers https://mglaman.dev/blog/can-we-use-concurrency-speed-streamed-bigpipe-r...

Fibers doesn't provide parallelism, it's just coroutine in the same thread

It looks to me like the work Matt Glaman has done in the blog post and the mysqli driver could meet in the middle.

Following on from #9:

1. Render bigpipe placeholders as (or in?) fibers.
1b. Add a mysqli driver with async support
2. Add support to views to execute the main listing query in ::execute() async, it would call Fiber::suspend() immediately after doing so.
3. When the fiber is resumed, views checks if its query has come back, and if so keeps going, if not ::suspend() again.
4. We could also do this outside views in PagerSelectExtender::query() specifically, where we execute two queries, both the count and the listing query, that gives us two places to Fiber::suspend() then.

It looks like Fiber::getCurrent() would be enough to tell if we're inside a Fiber or not, which would allow us to run the non-async code path for those cases, that way nothing changes.

Going to be some steps in the middle like dealing with

$result->setFetchMode(\PDO::FETCH_CLASS, 'Drupal\views\ResultRow');

in views query execution.

The ::getConnection() method supports both replicas and a base_database setting on the view, this would allow us to set up a separate async connection for mysqli leaving all other queries using the PDO driver. Would mean configuring each individual view as to whether it should use the async connection or not though and we'd need a way to know if the driver supports async queries at all.

Solr also supports async, so we could take a similar approach in search_api contrib too.

@andypost indeed, that's why I mentioned concurrency, not parallelism (cf. https://go.dev/blog/waza-talk) . That's also mostly what Node.js does and it still usually provides a nice performance boost when multiple I/O related tasks are concurrent, like here (DB + rendering).

Re-titling now we actually require PHP 8.1

In slack @mglaman mentioned possibly using Fibers for post-response tasks. We'd need to find post-response things where we can actually use non-blocking APIs, but if we can that'd be a good idea.

I think Big Pipe + Views + mysql async is likely to be the most dramatic improvement here and also the simplest, since it's quite centralised, so have updated the issue summary with an outline of how it seems to be coming together.

Opened 📌 Add PHP Fibers support to BigPipe RTBC .

How about adding a ‘Fibers’ tag to group together all Fibers related issues?

Updated #3257725-12: Use Fibers to offload discovery builds during stampedes → with a more concrete proposal - no patch or anything but would be good to get more eyes on the idea. Let's add that tag yeah.

FYI latest xdebug release included flamegraph support

and using fibers in tests for it https://github.com/xdebug/xdebug/commit/41ce86ab55979f386c9e612bd377b6ed...