BigPipe when server under heavy load results in PHP notices

Created on 24 January 2021, almost 4 years ago
Updated 15 February 2023, almost 2 years ago

Problem/Motivation

Steps to reproduce

I created a custom linkchecks module that uses javascript to crawl my site (checking for broken links) as fast as ajax can process the previous request, so one after another in sequence as fast as my javascript (web browser) can crawl. Doing this causes BigPipe to trigger an undefined offset error.

Proposed resolution

Fix is inspired by this idea:
https://stackoverflow.com/a/56971347
See patch

Remaining tasks

Test coverage.

User interface changes

None

API changes

None.

Data model changes

N/A

Release notes snippet

None.

🐛 Bug report
Status

Needs work

Version

10.1

Component
BigPipe 

Last updated 25 days ago

Created by

🇨🇦Canada joseph.olstad

Live updates comments and jobs are added and updated live.
  • Needs tests

    The change is currently missing an automated test that fails when run with the original code, and succeeds when the bug has been fixed.

Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • 🇧🇪Belgium wim leers Ghent 🇧🇪🇪🇺

    This still needs tests — once those exist, I'll be happy to RTBC 😊

  • 🇨🇦Canada joseph.olstad

    Not sure how I'd write a test for this, basically I assume, load the test site web links via javascript with random links and some ?getparam=timestamp to break cache would probably do it.

    Here's the module I wrote and used that exposed the problem in BigPipe.

    basically using the https://www.drupal.org/project/linkchecker module to create a list of links to crawl

    from there rather than using Drupal to crawl, using javascript to crawl, the javascript is included in this module.

    basically you'll need heavy loads to expose the bug.

  • 🇩🇪Germany jurgenhaas Gottmadingen

    This should pad for an empty string and not for NULL, because at the end of the method, there is a call to $this->sendChunk($scripts_bottom); where the given argument is asserted for being a string or HtmlResponse. I.e. if we passed NULL, this would throw an exception there.

    As for #24, here is a stack trace of what actually has happened in our case:

    AssertionError: assert(is_string($chunk) || $chunk instanceof HtmlResponse) in assert() (Zeile 259 in /var/www/html/web/core/modules/big_pipe/src/Render/BigPipe.php)
    #0 /var/www/html/web/core/modules/big_pipe/src/Render/BigPipe.php(259): assert()
    #1 /var/www/html/web/core/modules/big_pipe/src/Render/BigPipe.php(370): Drupal\big_pipe\Render\BigPipe->sendChunk()
    #2 /var/www/html/web/core/modules/big_pipe/src/Render/BigPipe.php(304): Drupal\big_pipe\Render\BigPipe->sendPreBody()
    #3 /var/www/html/web/core/modules/big_pipe/src/Render/BigPipeResponse.php(112): Drupal\big_pipe\Render\BigPipe->sendContent()
    #4 /var/www/html/vendor/symfony/http-foundation/Response.php(381): Drupal\big_pipe\Render\BigPipeResponse->sendContent()
    #5 /var/www/html/web/index.php(20): Symfony\Component\HttpFoundation\Response->send()
    #6 {main}
    
  • 🇫🇷France andypost

    I find #24 is a good idea about test and still not clear how server load affects chunks

  • First commit to issue fork.
  • Merge request !10382patch #30 as MR → (Open) created by sleitner
  • Pipeline finished with Success
    28 days ago
    Total: 987s
    #353364
  • 🇩🇪Germany sleitner

    @wim leers : I have this PHP warning in my logs:
    Warning: Undefined array key 2 in Drupal\big_pipe\Render\BigPipe->sendPreBody() (line 311 in /www/web/core/modules/big_pipe/src/Render/BigPipe.php)

    Created a MR for this issues, but tests are still needed.

Production build 0.71.5 2024