Kernel tests for recipes are significantly slower than Functional tests

Created on 5 August 2025, 1 day ago

Problem/Motivation

This is a follow-up to a performance issue reported on Drupal Slack with a reproducible test case.

When adding Kernel tests to Recipes following the standard transitional pattern from Drupal modules, we expected improved test execution speed compared to Functional tests. However, the results show the opposite performance characteristics:

  • Functional tests: ~17 seconds execution time
  • Kernel tests: ~60+ seconds execution time (3-4x slower)

This contradicts the expected performance hierarchy where Kernel tests should be faster than Functional tests since they don't require full browser simulation or complete Drupal bootstrap.

The recipes being tested install several modules as dependencies, which may be contributing to the performance issue.

Steps to reproduce

Compare the performance between:

  • \Drupal\Tests\system\Kernel\Recipe\GenericRecipeTestBase (Kernel test for Standard recipe)
  • \Drupal\Tests\system\Functional\Recipe\GenericRecipeTestBase (Functional test for Standard recipe)

The Standard recipe was chosen as it represents the most dependency-heavy recipe in Drupal core.

Proposed resolution

Remaining tasks

User interface changes

Introduced terminology

API changes

Data model changes

Release notes snippet

🐛 Bug report
Status

Active

Version

11.0 🔥

Component

recipe system

Created by

🇭🇺Hungary mxr576 Hungary

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @mxr576
  • Merge request !12916Draft: Add POC test for performance comparison → (Open) created by mxr576
  • 🇭🇺Hungary mxr576 Hungary

    Based on my investigation into the performance regression, I've identified several contributing factors to the slowness in Recipe Kernel tests:

    Both Functional and Kernel tests use MemoryCacheFactory as the default cache factory, but Kernel tests appear to be disproportionately impacted by its non-persistent nature. The frequent cache invalidations during module installation compound this issue.

    Related to 📌 RecipeRunner::processInstall() installs modules one by one Active - every module installation triggers multiple cache invalidations. Even when replacing the cache factory with a persistent one in Kernel tests, there's no significant performance improvement due to this cache churn.

    Attempted mitigation in our Kernel test:

    public function register(ContainerBuilder $container) {
      parent::register($container);
      // Change container to database cache backends.
      $container
        ->register('cache_factory', 'Drupal\Core\Cache\CacheFactory')
        ->addArgument(new Reference('settings'))
        ->addMethodCall('setContainer', [new Reference('service_container')]);
    }

    XHProf profiling reveals that attribute discovery is the most time-consuming operation during Recipe Kernel tests:

    • Majority of calls to getDefinitions() are pass-through calls from AttributeDiscoveryWithAnnotations::getDefinitions()
    • 80% of these calls originate from DerivativeDiscoveryDecorator::getDefinitions()
    • 80% of those calls come from DefaultPluginManager::findDefinitions()
  • Pipeline finished with Failed
    1 day ago
    Total: 1161s
    #565249
  • 🇭🇺Hungary mxr576 Hungary
  • 🇭🇺Hungary mxr576 Hungary

    For a more realistic comparison, I have removed idempotency check - so the second recipe application - from the GenericRecipeTestBase classes and captured the following with XHGui:

    web/core/recipes/standard/tests/src/Kernel/GenericTest.php

    web/core/recipes/standard/tests/src/Functional/GenericTest.php

Production build 0.71.5 2024