Try to use HTTP trailers or a real streaming response in Big Pipe.

Created on 30 May 2022, over 2 years ago
Updated 27 April 2024, 8 months ago

Problem/Motivation

Follow-up from #3276186: [upstream] BigPipe breaks with symfony/http-foundation 6.1 β†’ .

See also the upstream issue: https://github.com/symfony/symfony/issues/46449

Steps to reproduce

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

πŸ“Œ Task
Status

Needs work

Version

11.0 πŸ”₯

Component
BigPipeΒ  β†’

Last updated 17 days ago

Created by

πŸ‡¬πŸ‡§United Kingdom catch

Live updates comments and jobs are added and updated live.
  • Performance

    It affects performance. It is often combined with the Needs profiling tag.

Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • πŸ‡ΊπŸ‡ΈUnited States bradjones1 Digital Nomad Life

    This is important also because any post-response processing (e.g., destructable services) keeps the client "spinner" spinning when the server uses mod_php, even though all the content has been sent. See #3295790-69: Post-response task running (destructable services) are actually blocking; add test coverage and warn for common misconfiguration β†’ .

  • πŸ‡¬πŸ‡§United Kingdom catch

    Retitling to make it clearer what this is about.

  • πŸ‡§πŸ‡ͺBelgium wim leers Ghent πŸ‡§πŸ‡ͺπŸ‡ͺπŸ‡Ί

    Idea: if Symfony actually added support for HTTP trailers, then we could change BigPipe to rely on that instead β€” we wouldn't have to mess with Response::$content anymore!

  • πŸ‡¬πŸ‡§United Kingdom catch

    Huh interesting, my main interest here in using fully chunked encoding -

    Can we actually use Transfer-Encoding: chunked? We could probably shim the encoding into BigPipe::sendChunk() (also see comment on the original issue) to find out.

    so that we can send content length in a way the browser understands. The only drawback is that http2 works against this a bit so it might be already outdated to some extent.

    Does trailers help with this or is that mainly for the cache tags bit?

  • πŸ‡§πŸ‡ͺBelgium wim leers Ghent πŸ‡§πŸ‡ͺπŸ‡ͺπŸ‡Ί

    Quoting MDN:

    The Trailer response header allows the sender to include additional fields at the end of chunked messages in order to supply metadata that might be dynamically generated while the message body is sent, such as a message integrity check, digital signature, or post-processing status.

    IOW: it still requires chunked/streamed responses … but it just would fit better with Symfony's desire/intent to disallow manipulating Response::$content.

    That's it.

    No HTTP-level reason. Only a Symfony-level reason.

  • πŸ‡¬πŸ‡§United Kingdom catch

    Ahh gotcha, we might need both things then.

  • πŸ‡¬πŸ‡§United Kingdom longwave UK

    Found this after looking into BigPipe behind Amazon CloudFront, where I don't appear to be seeing the content streamed correctly - instead the response is delivered all in one go.

    CloudFront docs say:

    CloudFront supports only the chunked value of the Transfer-Encoding header. If your origin returns Transfer-Encoding: chunked, CloudFront returns the object to the client as the object is received at the edge location, and caches the object in chunked format for subsequent requests.

    BigPipe.php says

        // BigPipe sends responses using "Transfer-Encoding: chunked".
    

    but it appears this is not actually true?

  • πŸ‡ΊπŸ‡ΈUnited States mfb San Francisco

    As far as I've seen, Big Pipe does send chunked? All you have to do is print something and then call flush(); and the Transfer-Encoding: chunked header should be added automatically. HTTP/2 or HTTP/3 don't support it though, so you might not literally see a chunked transfer, depending on the stack.

  • πŸ‡¬πŸ‡§United Kingdom catch
    All you have to do is print something and then call flush(); and the Transfer-Encoding: chunked header should be added automatically. 
    

    OK yeah so PHP is supposed to send it if you call flush() as long as there's not a content length header (which there isn't for bigpipe), which makes this issue title a bit tautological. Good to figure that out before anything actually got going trying to reimplement the encoding in bigpipe though.

    But switching to using trailers would be really nice and maybe more compatible with http/2?

    Let's retitle. https://www.fastly.com/blog/supercharging-server-timing-http-trailers suggests some CDN support (although not for this use case at all).

  • πŸ‡ΊπŸ‡ΈUnited States mfb San Francisco

    re: compatible with HTTP/2, I'd hope that proxies convert HTTP/1.1 chunked transfer to HTTP/2 (or HTTP/3) data frames and pass them along, without excessive buffering, but not something I've analyzed

Production build 0.71.5 2024