Cache bin handling in multiple Redis DBs

Created on 18 December 2014, about 10 years ago
Updated 8 June 2023, over 1 year ago

Perhaps it would be nice to handle different cache bins in multiple Redis DBs.

Why? Currently Redis doesn't support wildcard delete so the Redis module have to clear all keys one by one for a cache clear of one cache bin. This could be possible failed with an PHP execution timeout, when you Redis Cache have many many keys. With a seperate Redis DB for one cache bin the module could use a flushdb to clear a hole cache bin.

Problem: How to configure this and are their any performance issues, when you have every time switch between the different Redis DBs?

Perhaps someone have any ideas on that. We have tried a patch with a LUA script handling for a wildcard delete, but then our Redis Server sometimes could fail after a flush all caches because it is also busy from the LUA script. Patch can be found here #2140897-43: cache_clear_all() not properly handled in PhpRedis.php

Feature request
Status

Needs review

Version

1.0

Component

Code

Created by

🇩🇪Germany IT-Cru Munich

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • 🇬🇧United Kingdom somersoft

    This patch enables the ability to use a multiple Redis services and configure which cache bin use which service based upon the cache bin name.
    The Redis report has been greatly modified to report information per service. It also lists which cache bins have been configured to use each service.
    The patch has been used with AWS services in production.
    It is fully backward compatible. for single service use and there are no changes to limit the configuration options.
    Documentation has also been updated to indicate how to add and configure multiple services.

  • 🇬🇧United Kingdom somersoft

    Updated the patch for redis 1.7
    Here is the list of patches from composer.patches.json

        "drupal/redis": {
          "Currently Drush Cr or Cache Clear UI does not flush Redis cache - https://www.drupal.org/project/redis/issues/2765895": "https://www.drupal.org/files/issues/redis-n2765895-16.patch",
          "Support TLS for Predis - https://www.drupal.org/project/redis/issues/3004561": "https://www.drupal.org/files/issues/2021-11-19/redis-support_tls_on_predis-3004561-37.patch",
          "Implement initial RedisCluster client integration - https://www.drupal.org/project/redis/issues/2900947": "https://www.drupal.org/files/issues/2024-01-15/2900947-60.patch",
          "Cache bin handling in multiple Redis DBs - https://www.drupal.org/project/redis/issues/2395255": "https://www.drupal.org/files/issues/2024-02-01/redis-Cache_bin_handling_in_multiple_Redis_DBs-2395255-5.patch"
        },
    
  • 🇬🇧United Kingdom somersoft

    Updated the patch for redis 1.8
    Here is the list of patches from composer[.patches].json

        "drupal/redis": {
          "Currently Drush Cr or Cache Clear UI does not flush Redis cache - https://www.drupal.org/project/redis/issues/2765895": "https://www.drupal.org/files/issues/redis-n2765895-16.patch",
          "Support TLS for Predis - https://www.drupal.org/project/redis/issues/3004561": "https://www.drupal.org/files/issues/2021-11-19/redis-support_tls_on_predis-3004561-37.patch",
          "Implement initial RedisCluster client integration - https://www.drupal.org/project/redis/issues/2900947": "https://www.drupal.org/files/issues/2024-10-15/2900947-82.patch",
          "Cache bin handling in multiple Redis DBs - https://www.drupal.org/project/redis/issues/2395255": "https://www.drupal.org/files/issues/2024-10-27/redis-Cache_bin_handling_in_multiple_Redis_DBs-2395255-6.patch"
        },
    
  • 🇳🇱Netherlands casey

    Nice. This is also useful if you want things like queues and persistent locks to be stored in redis instances with persistence enabled, while caches can be stored in redis instances without persistence.

    Note that I don't really like the name $settings['redis_client_bins']. I would go for $settings['redis.connections'] (or redis.servers) and $settings['redis.targets'] (or redis.bindings) and have the redis CacheBackendFactory look for "cache:$bin" and "cache". Maybe the same can be useful for queues.

    Also, the patch seems to create a client per cache bin, even if they are configured to use the same server. I guess this can be done more efficiently.

  • 🇳🇱Netherlands casey
  • 🇬🇧United Kingdom somersoft

    @casey
    The index redis.connection is already used. It is for connection details when there is only one and acts the fallback conection configuration for when no further server is configured for that cache bin. The change does not break any current configurations.
    FYI:
    cache bin is mentioned here core/lib/Drupal/Core/Cache/CacheFactory.php::24
    'bins' and 'servers' are the indexes used in the Memcache configuration. It seems they had the functionality fromearlier on. Wanted the configuration concepts for both to be very similar so moving between using different techonlogies was not difficult for evalution purposes.
    Feel free to make updates to the patch file and create one client per server rather than per bin by updating
    Drupal\redis\ClientFactory::getClientInterface() and/or ::getClient()

Production build 0.71.5 2024