Add a cached normalizer service

Created on 27 June 2023, about 1 year ago
Updated 19 June 2024, 12 days ago

Problem/Motivation

The idea is to

  1. Provide a way to attach cacheability metadata to the return values of normalizers (as described in #3028080: Add CacheableNormalization for Normalizer to return Normalized value and Cacheablity → , but implemented just for this module)
  2. Provide a cached normalizer service that automatically caches the return values, using the format, context array and cacheability metadata attached to the return value as cache contexts.

This would be kind of like the dynamic page cache, improving performance in API endpoints where whole responses cannot be cached due to high cardinality cache contexts like session and user.

✨ Feature request
Status

Fixed

Version

1.0

Component

Code

Created by

🇧🇪Belgium DieterHolvoet Brussels

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @DieterHolvoet
  • 🇧🇪Belgium DieterHolvoet Brussels
  • Merge request !12Add a cached normalizer → (Merged) created by DieterHolvoet
  • Status changed to Needs review 6 months ago
  • 🇧🇪Belgium DieterHolvoet Brussels

    The MR has a working version now. Still testing, not yet using in a production environment.

  • 🇧🇪Belgium DieterHolvoet Brussels

    In Symfony 6 Symfony\Component\Serializer\Normalizer\NormalizerInterface got more function argument types, including one mixed type. In order to support both Symfony 5 and 6, we'll have to add those types to Drupal\api_toolkit\Normalizer\CachedNormalizer, which means that we'll have to bump the minimum PHP version to 8.0 if we want to add the mixed type. I think that's reasonable.

  • 🇧🇪Belgium DieterHolvoet Brussels

    We should recommend using a Permanent Cache Bin → backend for the normalizer cache, to make it more efficient. This needs to be added to settings.php:

    $settings['cache']['bins']['api_toolkit_normalizer'] = 'cache.backend.permanent_database';
    
  • 🇧🇪Belgium DieterHolvoet Brussels

    I added something new, an experiment: a way to do placeholdering with lazy builders, similar to how it's already possible in other places in Drupal using #lazy_builder. It can be used to add highly dynamic data to normalization results while keeping their cacheability. Here's an example in action:

    
    namespace Drupal\example_module\Normalizer;
    
    use Drupal\api_toolkit\Normalizer\Placeholder;
    use Drupal\Core\Cache\CacheableMetadata;
    use Drupal\Core\Security\TrustedCallbackInterface;
    use Drupal\example_module\Service\ActivityCounts;
    use Drupal\example_module\Entity\Node\UserGroup;
    use Drupal\serialization\Normalizer\NormalizerBase;
    
    class UserGroupNormalizer extends NormalizerBase implements TrustedCallbackInterface
    {
        protected $format = ['pro_api'];
        protected $supportedInterfaceOrClass = [UserGroup::class];
    
        public function __construct(
            protected ActivityCounts $activityCounts,
        ) {
        }
    
        /**
         * @param UserGroup $object
         * @param array{cacheability: CacheableMetadata|null} $context
         */
        public function normalize($object, $format = null, array $context = []): array
        {
            $context['cacheability'] ??= new CacheableMetadata();
            $context['cacheability']->addCacheableDependency($object);
    
            return [
                'uuid' => $object->uuid(),
                'created' => $object->getCreatedTime(),
                'title' => $object->getTitle(),
                'totalActivities' => new Placeholder([$this, 'getActivityCount']),
                'totalDistance' => new Placeholder([$this, 'getTotalDistance']),
            ];
        }
    
        public function getActivityCount(UserGroup $group): int
        {
            return $this->activityCounts->getActivityCount(group: $group);
        }
    
        public function getTotalDistance(UserGroup $group): int
        {
            return $this->activityCounts->getTotalDistance(group: $group);
        }
    
        public static function trustedCallbacks(): array
        {
            return ['getActivityCount', 'getTotalDistance'];
        }
    }
    
  • 🇧🇪Belgium DieterHolvoet Brussels

    The previous approach didn't replace placeholders in case of nested normalizers, so I rewrote part of the implementation. The end result is a lot simpler:

    namespace Drupal\example_module\Normalizer;
    
    use Drupal\api_toolkit\Normalizer\Placeholder\Placeholder;
    use Drupal\Core\Cache\CacheableMetadata;
    use Drupal\example_module\Service\ActivityCounts;
    use Drupal\example_module\Entity\Node\UserGroup;
    use Drupal\serialization\Normalizer\NormalizerBase;
    
    class UserGroupNormalizer extends NormalizerBase
    {
        protected $format = ['pro_api'];
        protected $supportedInterfaceOrClass = [UserGroup::class];
    
        public function __construct(
            protected ActivityCounts $activityCounts,
        ) {
        }
    
        /**
         * @param UserGroup $object
         * @param array{cacheability: CacheableMetadata|null} $context
         */
        public function normalize($object, $format = null, array $context = []): array
        {
            $context['cacheability'] ??= new CacheableMetadata();
            $context['cacheability']->addCacheableDependency($object);
    
            return [
                'uuid' => $object->uuid(),
                'created' => $object->getCreatedTime(),
                'title' => $object->getTitle(),
                'postalCodes' => new Placeholder([$this->serializer, 'normalize'], [$object->getPostalCodes(), $format, $context]),
                'totalActivities' => new Placeholder([$this->activityCounts, 'getActivityCount'], ['group' => $object]),
                'totalDistance' => new Placeholder([$this->activityCounts, 'getTotalDistance'], ['group' => $object]),
            ];
        }
    }
    

    Entity, field item list & field item arguments that are passed to placeholder callbacks are automatically normalized to a simple string format and the entity in question is reloaded from the database before placeholder replacement happens. A side result of the current implementation is that you can now also pass [$this->serializer, 'normalize'] to a placeholder, which will cause the nested normalizations to not be part of the cached normalization anymore, which should in turn decrease cache sizes in case of a lot of nested normalizations.

  • Pipeline finished with Skipped
    26 days ago
    #191907
  • Status changed to Fixed 26 days ago
  • 🇧🇪Belgium DieterHolvoet Brussels
  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.69.0 2024