BigPipe when server under heavy load results in PHP notices

Created on 24 January 2021, over 3 years ago
Updated 28 February 2023, over 1 year ago

Problem/Motivation

Steps to reproduce

I created a custom linkchecks module that uses javascript to crawl my site (checking for broken links) as fast as ajax can process the previous request, so one after another in sequence as fast as my javascript (web browser) can crawl. Doing this causes BigPipe to trigger an undefined offset error.

Proposed resolution

Fix is inspired by this idea:
https://stackoverflow.com/a/56971347
See patch β†’

Remaining tasks

Test coverage.

User interface changes

None

API changes

None.

Data model changes

N/A

Release notes snippet

None.

πŸ› Bug report
Status

Needs work

Version

10.1 ✨

Component
BigPipeΒ  β†’

Last updated 13 days ago

Created by

πŸ‡¨πŸ‡¦Canada joseph.olstad

Live updates comments and jobs are added and updated live.
  • Needs tests

    The change is currently missing an automated test that fails when run with the original code, and succeeds when the bug has been fixed.

Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • πŸ‡§πŸ‡ͺBelgium Wim Leers Ghent πŸ‡§πŸ‡ͺπŸ‡ͺπŸ‡Ί

    This still needs tests β€” once those exist, I'll be happy to RTBC 😊

  • πŸ‡¨πŸ‡¦Canada joseph.olstad

    Not sure how I'd write a test for this, basically I assume, load the test site web links via javascript with random links and some ?getparam=timestamp to break cache would probably do it.

    Here's the module I wrote and used that exposed the problem in BigPipe.

    basically using the https://www.drupal.org/project/linkchecker β†’ module to create a list of links to crawl

    from there rather than using Drupal to crawl, using javascript to crawl, the javascript is included in this module.

    basically you'll need heavy loads to expose the bug.

  • πŸ‡©πŸ‡ͺGermany jurgenhaas Gottmadingen

    This should pad for an empty string and not for NULL, because at the end of the method, there is a call to $this->sendChunk($scripts_bottom); where the given argument is asserted for being a string or HtmlResponse. I.e. if we passed NULL, this would throw an exception there.

    As for #24, here is a stack trace of what actually has happened in our case:

    AssertionError: assert(is_string($chunk) || $chunk instanceof HtmlResponse) in assert() (Zeile 259 in /var/www/html/web/core/modules/big_pipe/src/Render/BigPipe.php)
    #0 /var/www/html/web/core/modules/big_pipe/src/Render/BigPipe.php(259): assert()
    #1 /var/www/html/web/core/modules/big_pipe/src/Render/BigPipe.php(370): Drupal\big_pipe\Render\BigPipe->sendChunk()
    #2 /var/www/html/web/core/modules/big_pipe/src/Render/BigPipe.php(304): Drupal\big_pipe\Render\BigPipe->sendPreBody()
    #3 /var/www/html/web/core/modules/big_pipe/src/Render/BigPipeResponse.php(112): Drupal\big_pipe\Render\BigPipe->sendContent()
    #4 /var/www/html/vendor/symfony/http-foundation/Response.php(381): Drupal\big_pipe\Render\BigPipeResponse->sendContent()
    #5 /var/www/html/web/index.php(20): Symfony\Component\HttpFoundation\Response->send()
    #6 {main}
    
Production build 0.69.0 2024