Contrib.social

Exception when installing on a project that already uses redis

Open on Drupal.org →
Created on 3 September 2024, 8 months ago

Problem/Motivation

Installing this in a project that already uses redis causes the following exception:

The website encountered an unexpected error. Try again later.
CrowdSec\RemediationEngine\CacheStorage\CacheStorageException: Error when creating Redis cache adapter:Redis connection failed: No such file or directory in CrowdSec\RemediationEngine\CacheStorage\Redis->__construct() (line 35 of /vendor/crowdsec/remediation-engine/src/CacheStorage/Redis.php).

CrowdSec\RemediationEngine\CacheStorage\Redis->__construct() (Line: 183)
Drupal\crowdsec\Client->cache() (Line: 133)
Drupal\crowdsec\Client->remediation() (Line: 160)
Drupal\crowdsec\Client->verifyIp() (Line: 87)
Drupal\crowdsec\Middleware->handle() (Line: 48)
Drupal\Core\StackMiddleware\ReverseProxyMiddleware->handle() (Line: 51)
Drupal\Core\StackMiddleware\NegotiationMiddleware->handle() (Line: 36)
Drupal\Core\StackMiddleware\AjaxPageState->handle() (Line: 49)
Drupal\remove_http_headers\StackMiddleware\RemoveHttpHeadersMiddleware->handle() (Line: 51)
Drupal\Core\StackMiddleware\StackedHttpKernel->handle() (Line: 741)
Drupal\Core\DrupalKernel->handle() (Line: 19)

Steps to reproduce

See above and here's my settings.redis.php:

<?php

use Drupal\Core\Installer\InstallerKernel;

// Either copy the content of this file into an existing settings.php file or
// create as e.g. settings.redis.php and include it, then customize. See
// README.md and client specific README files for more information.

// Adjust extension check or remove, prevents configuring Redis before
// Drupal is installed or on an environment where the used client is not
// available.
if (!InstallerKernel::installationAttempted() && extension_loaded('redis')) {

  // Customize host and port.
  // $settings['redis.connection']['host'] = '127.0.0.1';
  // $settings['redis.connection']['port'] = 6379;

  // Old Hetzner redis via cron:
  // $settings['redis.connection']['host']      = '127.0.0.1';  // Your Redis instance hostname.
  // $settings['redis.connection']['port']      = '6379';  // Redis port

  // New Hetzner redis via UI:
  $settings['redis.connection']['host']      = '/run/redis_me/redis.sock';  // Your Redis instance hostname.
  $settings['redis.connection']['port']      = 0;  // Redis port


  // Customize used interface.
  // $settings['redis.connection']['interface'] = 'PhpRedis';

  // Set Redis as the default backend for any cache bin not otherwise specified.
  $settings['cache']['default'] = 'cache.backend.redis';

  // Per-bin configuration examples, bypass the default ChainedFastBackend.
  // *Only* use this when using Relay (see README.Relay.md) or when APCu is not
  // available.
  // $settings['cache']['bins']['config'] = 'cache.backend.redis';
  // $settings['cache']['bins']['discovery'] = 'cache.backend.redis';
  // $settings['cache']['bins']['bootstrap'] = 'cache.backend.redis';

  // Use compression for cache entries longer than the specified limit.
  $settings['redis_compress_length'] = 100;

  // Customize the prefix, a reliable but long fallback is used if not defined.
  // $settings['cache_prefix'] = 'prefix';
  $settings['cache_prefix'] = 'redisme_';

  // Apply changes to the container configuration to better leverage Redis.
  // This includes using Redis for the lock and flood control systems, as well
  // as the cache tag checksum. Alternatively, copy the contents of that file
  // to your project-specific services.yml file, modify as appropriate, and
  // remove this line.
  // $settings['container_yamls'][] = 'modules/contrib/redis/example.services.yml';
  $settings['container_yamls'][] = 'sites/default/redis.services.yml';

  // Allow the services to work before the Redis module itself is enabled.
  $settings['container_yamls'][] = 'modules/contrib/redis/redis.services.yml';

  // Manually add the classloader path, this is required for the container cache
  // bin definition below and allows to use it without the redis module being
  // enabled.
  $class_loader->addPsr4('Drupal\\redis\\', 'modules/contrib/redis/src');

  // Use redis for container cache.
  // The container cache is used to load the container definition itself, and
  // thus any configuration stored in the container itself is not available
  // yet. These lines force the container cache to use Redis rather than the
  // default SQL cache.
  $settings['bootstrap_container_definition'] = [
    'parameters' => [],
    'services' => [
      'redis.factory' => [
        'class' => 'Drupal\redis\ClientFactory',
      ],
      'cache.backend.redis' => [
        'class' => 'Drupal\redis\Cache\CacheBackendFactory',
        'arguments' => ['@redis.factory', '@cache_tags_provider.container', '@serialization.phpserialize'],
      ],
      'cache.container' => [
        'class' => '\Drupal\redis\Cache\PhpRedis',
        'factory' => ['@cache.backend.redis', 'get'],
        'arguments' => ['container'],
      ],
      'cache_tags_provider.container' => [
        'class' => 'Drupal\redis\Cache\RedisCacheTagsChecksum',
        'arguments' => ['@redis.factory'],
      ],
      'serialization.phpserialize' => [
        'class' => 'Drupal\Component\Serialization\PhpSerialize',
      ],
    ],
  ];
}

Proposed resolution

Remaining tasks

User interface changes

API changes

Data model changes

🐛 Bug report
Status

Active

Version

1.1

Component

Code

Created by

🇩🇪Germany Anybody Porta Westfalica

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @Anybody
  • Comment 8 months ago →
    🇩🇪Germany Anybody Porta Westfalica

    Is it perhaps because redis is used through socket?

    I didn't even have the chance to enable redis usage in the settings, as described on the module page.

    Any early ideas?

  • Comment 8 months ago →
    🇩🇪Germany Anybody Porta Westfalica
  • Comment 8 months ago →
    🇩🇪Germany jurgenhaas Gottmadingen

    You're right, this module assumes that the redis connection is set to a hostname and port. We never thought of sockets being used here. The code can be found \Drupal\crowdsec\Client::cache where we build redis dns in line 182:

    $dns = 'redis://' . ($redis['host'] ?? 'localhost') . ':' . ($redis['port'] ?? '6379');

    I'm not sure how the dns would look like with sockets and if the crowdsec even supports that.

    Do you know that dns? If so, you could give it a try at that point in the code and see if that works. We would then have to create an MR with a reasonable determination of the dns.

    I would have expected that Redis is not used until I enable it?

    No, it's expected to work in any case.

  • Comment 8 months ago →
    🇩🇪Germany Anybody Porta Westfalica

    Whao, thank you for the superfast reply @jurgenhaas! 🎉

    As background information: Since some months, Hetzner has a UI for the Redis caching for managed servers. They only publish the socket and that's the documented way to use it. So I think it would at least be risky to work around that?

    The way we use it complies with the redis module. See the background information here: ✨ Allow Socket Connections Needs review

    Just some ideas:
    Maybe for the future a flexible solution could be to detect the redis module and use it for the connection?

    Also for now, it would be good enough to be able to switch off the redis integration (or opt-in, as I'd read it from the module page), but honestly I have no idea if that would also be hard to implement.

  • Comment 8 months ago →
    🇩🇪Germany Anybody Porta Westfalica
  • Comment 8 months ago →
    🇩🇪Germany Anybody Porta Westfalica
  • Comment 8 months ago →
    🇩🇪Germany Anybody Porta Westfalica

    BTW I just saw that it should be called dsn not dns? Never seen that before... ;D

  • Comment 8 months ago →
    🇩🇪Germany Anybody Porta Westfalica

    I sadly still don't have an idea how we could at least disable redis support for this case to make the module easily usable, until there's a supported way to use it through socket. If anyone should have good ideas for a general fix or a switch to disable "auto-detection", please leave a note here :)

    (Of course it should be straightforward and clean, not a hack)

  • Comment 8 months ago →
    🇩🇪Germany jurgenhaas Gottmadingen

    Well, the only next step is to find the syntax for the socket DSN and try that out in code. Once we know that, we can easily support sockets with this module. I don't have Redis over sockets in place, so that should be researched and tested by someone who does.

  • Comment 8 months ago →
    🇩🇪Germany Anybody Porta Westfalica

    Thanks @jurgenhaas sorry I understood it's simply not possible that way. I'll try to find it out!

  • Comment 6 months ago →
    🇩🇪Germany jurgenhaas Gottmadingen

    @anybody any news on this?

  • Comment 6 months ago →
    🇩🇪Germany Anybody Porta Westfalica

    @jurgenhaas sorry had no time yet, I think I won't make it before 2025 - many projects on the finish line... :/ But of course still relevant!

    (e.g. for all Hetzner Hostings)

  • Status changed to Postponed: needs info 2 days ago
  • Comment 2 days ago →
    🇩🇪Germany jurgenhaas Gottmadingen
contrib.social Blog FAQ Discussions
Production build 0.71.5 2024