Out of memory issue since 3.3.0

Attaching temporary work around patches

@pookmish You are a life saver! I would've never figured this out. Confirmed that the patch for in #2 for hook_event_dispatcher 4.x resolves the error.

My dev process involves installing the site, so as to have an "empty" database for development and testing. Without the patch this fails due to a memory limit error connected to hook_event_dispatcher. Here's part of the stack trace:

11:32:28 Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 6433664 bytes) in /var/www/docroot/core/lib/Drupal/Core/Database/StatementWrapperIterator.php on line 110
11:32:28 
11:32:28 Fatal error: Uncaught PDOException: SQLSTATE[HY000]: General error: 2014 Cannot execute queries while other unbuffered queries are active.  Consider using PDOStatement::fetchAll().  Alternatively, if your code is only ever going to run against mysql, you may enable query buffering by setting the PDO::MYSQL_ATTR_USE_BUFFERED_QUERY attribute. in /var/www/docroot/core/lib/Drupal/Core/Database/StatementWrapperIterator.php:110
11:32:28 Stack trace:
11:32:28 #0 /var/www/docroot/core/lib/Drupal/Core/Database/StatementWrapperIterator.php(110): PDOStatement->execute(Array)
11:32:28 #1 /var/www/docroot/core/lib/Drupal/Core/Database/Query/Upsert.php(115): Drupal\Core\Database\StatementWrapperIterator->execute(Array, Array)
11:32:28 #2 /var/www/docroot/core/lib/Drupal/Core/Cache/DatabaseBackend.php(283): Drupal\Core\Database\Query\Upsert->execute()
11:32:28 #3 /var/www/docroot/core/lib/Drupal/Core/Cache/DatabaseBackend.php(210): Drupal\Core\Cache\DatabaseBackend->doSetMultiple(Array)
11:32:28 #4 /var/www/docroot/core/lib/Drupal/Core/Cache/DatabaseBackend.php(186): Drupal\Core\Cache\DatabaseBackend->setMultiple(Array)
11:32:28 #5 /var/www/docroot/core/lib/Drupal/Core/Extension/ModuleHandler.php(329): Drupal\Core\Cache\DatabaseBackend->set('module_implemen...', Array)
11:32:28 #6 /var/www/docroot/modules/contrib/hook_event_dispatcher/src/HookEventDispatcherModuleHandlerProxyTrait.php(121): Drupal\Core\Extension\ModuleHandler->writeCache()
11:32:28 #7 /var/www/docroot/core/lib/Drupal/Core/EventSubscriber/RequestCloseSubscriber.php(42): Drupal\hook_event_dispatcher\HookEventDispatcherModuleHandler->writeCache()
11:32:28 #8 [internal function]: Drupal\Core\EventSubscriber\RequestCloseSubscriber->onTerminate(Object(Symfony\Component\HttpKernel\Event\TerminateEvent), 'kernel.terminat...', Object(Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher))
11:32:28 #9 /var/www/docroot/core/lib/Drupal/Component/EventDispatcher/ContainerAwareEventDispatcher.php(111): call_user_func(Array, Object(Symfony\Component\HttpKernel\Event\TerminateEvent), 'kernel.terminat...', Object(Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher))
11:32:28 #10 /var/www/vendor/symfony/http-kernel/HttpKernel.php(115): Drupal\Component\EventDispatcher\ContainerAwareEventDispatcher->dispatch(Object(Symfony\Component\HttpKernel\Event\TerminateEvent), 'kernel.terminat...')
11:32:28 #11 /var/www/docroot/core/lib/Drupal/Core/StackMiddleware/StackedHttpKernel.php(63): Symfony\Component\HttpKernel\HttpKernel->terminate(Object(Symfony\Component\HttpFoundation\Request), Object(Drupal\Core\Render\HtmlResponse))
11:32:28 #12 /var/www/docroot/core/lib/Drupal/Core/DrupalKernel.php(688): Drupal\Core\StackMiddleware\StackedHttpKernel->terminate(Object(Symfony\Component\HttpFoundation\Request), Object(Drupal\Core\Render\HtmlResponse))
11:32:28 #13 /var/www/vendor/drush/drush/src/Boot/DrupalBoot8.php(394): Drupal\Core\DrupalKernel->terminate(Object(Symfony\Component\HttpFoundation\Request), Object(Drupal\Core\Render\HtmlResponse))
11:32:28 #14 [internal function]: Drush\Boot\DrupalBoot8->terminate()

@pookmish - Working great for us, allowed us to finally pass the memory issue on the Acquia pipeline. Thanks!

This is hard to test and reproduce from my end, can someone able to test if only one of two services that uses drupal_static can make any difference? I'm trying to understand if both services cause the issue or only one of them.

Since the above patches required re-rolling to support the latest version of the module, I double check the memory issue. It still occurs and happens with both services. I tried to leave each of the services, but the OOM would still happen.

@el7cosmos this is tough to test, but it seems like others are having similar issues. If you have an installation profile that can install from existing configuration, it should be replicated. Lower your CLI memory limit to something. For my situation, even 512M caused the OOM error.

This is because Drupal rebuilds the container each time a module is installed. Since this module doesn't provide events during site installation I think it makes sense to disable the module handler decoration.

Create an MR to disable the module handler decorator during site installation. Really appreciate it if anyone is able to review and test it.

Thanks for the reference to the Core issue, @el7cosmos. That makes more sense why this issue is happening. It looks like your MR works well for me. I was able to do a full site install with a memory limit of 512MB. To be thorough, I tested with 256MB memory limit. Both your MR and the patch in #7 ran OOM. I like your approach better as well since it maintains proper dependency injection in the service.

Thanks.

@el7cosmos @pookmish - I've had slightly different results after running full site installs following the normal setup with config-imports. Without patch #7 the Acquia pipeline runs out of resources during the setup phase. Here are the results trying to run each patch locally.

With no patch:
cim 1 = 1024 MB OOM
cim 2 = 1024 MB OOM
cim 3 = 231.42 MB

With #7 patch:
cim = 360.27 MB

With #9 MR patch:
cim 1 = 1024 MB OOM
cim 2 = 1024 MB OOM
cim 3 = 231.41 MB

As shown above using #7 I can install and run a single config-import without any memory issues.

@j-barnes the MR only affects when importing configuration during site installation, this won't handle running the config import after the site is installed