Contrib.social

Improve fast-backend write performance of ChainedFastBackend by serializing upfront

Open on Drupal.org →
Created on 17 November 2018, about 6 years ago
Updated 10 September 2024, 4 months ago

Problem/Motivation

ChainedFast is a nice backend, which has it's problems as it needs to write back all data again and again to the fast backend.

One of the most CPU intensive things is the constant serialization and unserialization -- even if it's usually handled by the cache backends themselves.

One of the reasons is that Drupal uses DEEP arrays, and DEEP objects.

However if you do:

$item = (object) [
  'data' => serialize($data),
];
$store = serialize($data);

this is almost as fast as:

$item = (object) [
  'data' => $data,
];
$store = serialize($data);

because the one level for serialization is not responsible for a huge performance penalty.

And we can use this to our advantage!

Proposed resolution

If we serialize() the data before set() and unserialize() the data after get() / getMultiple(), then in practice it means that 90% of the data is already serialized, which we get back from the consistent backend!

Which means we save 90% of the serialization effort when saving data to the fast backend!

And that in the end means that things are 90% faster.

(And APCu is storing serialized data as well and no longer copying raw PHP objects into memory, so that's no longer a concern.)

Also: Because we do not need to unserialize the data coming from APCu, we can effectively save the unserialize() step when the data is invalid -- before we would unserialize twice.

Remaining tasks

- Create patch based on the following:

diff --git a/web/core/lib/Drupal/Core/Cache/ChainedFastBackend.php b/web/core/lib/Drupal/Core/Cache/ChainedFastBackend.php
--- a/web/core/lib/Drupal/Core/Cache/ChainedFastBackend.php
+++ b/web/core/lib/Drupal/Core/Cache/ChainedFastBackend.php
@@ -170,6 +170,10 @@ public function getMultiple(&$cids, $allow_invalid = FALSE) {
       }
     }
 
+    foreach ($cache as $cid => $item) {
+      $item->data = unserialize($item->data);
+    }
+
     return $cache;
   }
 
@@ -177,6 +181,8 @@ public function getMultiple(&$cids, $allow_invalid = FALSE) {
    * {@inheritdoc}
    */
   public function set($cid, $data, $expire = Cache::PERMANENT, array $tags = []) {
+    $data = serialize($data);
+
     $this->consistentBackend->set($cid, $data, $expire, $tags);
     $this->markAsOutdated();
     // Don't write the cache tags to the fast backend as any cache tag
@@ -188,6 +194,10 @@ public function set($cid, $data, $expire = Cache::PERMANENT, array $tags = []) {
    * {@inheritdoc}
    */
   public function setMultiple(array $items) {
+    foreach ($items as &$item) {
+      $item['data'] = serialize($item['data']);
+    }
+
     $this->consistentBackend->setMultiple($items);
     $this->markAsOutdated();
     // Don't write the cache tags to the fast backend as any cache tag

User interface changes

- None

API changes

- None

Data model changes

- None

📌 Task
Status

Needs work

Version

11.0 🔥

Component
Cache  →

Last updated about 11 hours ago

Created by

🇩🇪Germany Fabianx

Live updates comments and jobs are added and updated live.
  • Performance

    It affects performance. It is often combined with the Needs profiling tag.

  • Needs issue summary update

    Issue summaries save everyone time if they are kept up-to-date. See Update issue summary task instructions.

Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • Comment almost 2 years ago →
    🇨🇭Switzerland berdir Switzerland

    To add to that, even when just looking at the isolated performance of ChainedFastBackend, an all-miss situation looks like this:

    So writing back the cache entries only takes 25% of the time, so we save ~90% of 25%.

    Here's an attempt at a better issue title that reflects what it is actually about.

    One thing to keep in mind is also that this change is quite tricky to apply on a site, adding or removing the patch or updating if it were to be committed with any existing caches is not be handled and probably should be.

  • Status changed to Needs work almost 2 years ago
  • Comment almost 2 years ago →
    🇺🇸United States smustgrave

    #29 requested an issue summary update so moving to NW for that.

  • Merge request !8524Issue #3014521 by IT-Cru, _utsavsharma, Fabianx: Improve fast-backend write... → (Open) created by IT-Cru
  • Pipeline finished with Failed
    7 months ago
    Total: 658s
    #207010
  • Comment 7 months ago →
    🇩🇪Germany IT-Cru Munich

    Role back of patch from #26 as MR to apply against 11.x and 10.3.x again.

    One kernel and some functional unit tests are failing in Gitlab CI jobs now, but I have no clue why this wasn't previously the case. So I think this needs some more investigations.

  • Comment 4 months ago →
    🇯🇴Jordan Qusai Taha Amman

    Re-roll the patch to make it working with 10.3.2

  • Comment 4 months ago →
    🇺🇸United States SocialNicheGuru

    I was migrating a site from Drupal 10 to Drupal 10.3.

    I applied the patch in composer.json:
    "Improve fast-backend write performance of ChainedFastBackend by serializing upfront":" https://www.drupal.org/files/issues/2024-09-01/3014521-34.patch → ",

    I received the following error:

    Drush command terminated abnormally due to an unrecoverable error. Error: Uncaught TypeError: unserialize(): Argument #1 ($data) must be of type string, array given in drupal-10.3.x/html/core/lib/Drupal/Core/Cache/ChainedFastBackend.php:174 Stack trace:
    #0 drupal-10.3.x/html/core/lib/Drupal/Core/Cache/ChainedFastBackend.php(174): unserialize()
    #1 drupal-10.3.x/html/core/lib/Drupal/Core/Config/CachedStorage.php(87): Drupal\Core\Cache\ChainedFastBackend->getMultiple()
    #2 drupal-10.3.x/html/core/lib/Drupal/Core/Config/ConfigFactory.php(165): Drupal\Core\Config\CachedStorage->readMultiple()
    #3 drupal-10.3.x/html/core/lib/Drupal/Core/Config/ConfigFactory.php(104): Drupal\Core\Config\ConfigFactory->doLoadMultiple()
    #4 drupal-10.3.x/html/core/lib/Drupal/Core/Config/ConfigFactory.php(89): Drupal\Core\Config\ConfigFactory->doGet()
    #5 drupal-10.3.x/html/core/lib/Drupal.php(414): Drupal\Core\Config\ConfigFactory->get()
    #6 drupal-10.3.x/html/core/includes/errors.inc(323): Drupal::config()
    #7 drupal-10.3.x/html/core/includes/errors.inc(123): _drupal_get_error_level()
    #8 drupal-10.3.x/html/core/includes/bootstrap.inc(204): error_displayable()
    #9 drupal-10.3.x/html/core/includes/bootstrap.inc(188): _drupal_exception_handler_additional()
    #10 [internal function]: _drupal_exception_handler()
    #11 {main} thrown in drupal-10.3.x/html/core/lib/Drupal/Core/Cache/ChainedFastBackend.php, line 174

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024