Contrib.social

MemcacheD with persistent connections

Open on Drupal.org →
Created on 30 May 2019, over 5 years ago
Updated 26 August 2024, 3 months ago

Problem/Motivation

There is no documented way to use persistent connections with this module and MemcacheD extension.
On further diving into the code I found this issue mentioned in and talks about the legacy Memcache extension #822316: receiving a PHP notice "Connection reset by peer .... dmemcache.inc on line 104" → .

On further reverse engineering, you reach to the actual connection factory method / class that is disregarding the persistent connection settings all together, so there are NO persistent connections on memcacheD extension. This is not documented anywhere.

Note that the legacy extension had persistent connections. On top of that it is already deprecated and not supported on PHP 7.0+. With new Drupal 8.8 dropping support of PHP 5.6 this should be a close to critical issue for people to be able to migrate to the memcacheD and still have persistent connections.

Official PHP docs: 

/* Create a regular instance */
$m = new Memcached();
echo get_class($m);

/* Create a persistent instance */
$m2 = new Memcached('story_pool');
$m3 = new Memcached('story_pool');

/* now $m2 and $m3 share the same connection */

In this module's case the implementation follows the non-persistent connections example.
I know it is durable and overall easier to maintain, but in case of:

  1. High latency networks (imagine ANY cloud provider), establishing a new connection could be too costly to have that overhead on every request.
  2. Huge cluster sizes (server counts) will multiply the TCP overhead on each HTTP request.
  3. Authenticated connections.

Proposed resolution

Implement the persistent connections in the module.

Have a way to respect the configuration that is already present from memcache era:

# We generate the ID on server side.
$settings['memcache']['persistent'] = TRUE;

# Or have something like, so the setting is exposed to the users of the system.
$settings['memcache']['persistent_id'] = 'drupal';

Remaining tasks

Discussion, Patch, Benchmarks, etc.

User interface changes

None expected.

API changes

None expected.
This is a pure feature addition and changes in implementation details only.

Data model changes

New setting to manage the enable / disable and configuration of the persistent connections.

Release notes snippet

Added persistent connections for the MemcacheD PHP extension.

✨ Feature request
Status

Needs review

Version

2.7

Component

Code

Created by

🇧🇬Bulgaria ndobromirov

Live updates comments and jobs are added and updated live.
  • Performance

    It affects performance. It is often combined with the Needs profiling tag.

Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • Comment over 1 year ago →
    🇪🇪Estonia hkirsman

    I'm not really sure if this works or not but one way seems to be to call isPersistent() method on the Memcached() object.

    I thought why not have it on the /admin/reports/memcache next to connections:

    Is it ok that the "Connections 6 open of 1 104 total" is still growing? On our dev server 1104 is 16 000.

  • Comment over 1 year ago →
    🇪🇪Estonia hkirsman
  • Comment over 1 year ago →
    🇪🇪Estonia hkirsman
  • Comment over 1 year ago →
    🇩🇪Germany jan kellermann

    Patch #17 works for me. Fine!

    Maybe you can choose $settings['memcache']['key_prefix']; as $settings['memcache']['persistent'] per default? In shared environment it could be great to have a pconnection for each drupal. Maybe this could be a hint in the README.

    Thank you for your work.

  • Comment over 1 year ago →
    🇫🇷France fgm Paris, France

    You have to be aware of the fact that the design hypotheses of libmemcached (on which the memcached extension is built) are not entirely compatible with PHP, and as a consequence, the behavior of any code using the memcached extension with persistent connections is often unexpected in the presence of server errors, which can lead to the feeling that bugs exist in memcached servers / libmemcached / the memcached extension / this module, while the underlying issue is in fact a hardly documented design choice regarding expectations of behaviour in the face of errors.

    I do not have the link at hand, but IIRC there were already a couple of tickets around these issues. The summary being: avoid using the memcached extension and stick with the memcache extension, which is tailor-made for the PHP design choices and behaves as you would expect in the face of errors.

    Also, since the memcache extension has been working quite nicely with PHP 8 since 2020 and version 8 of the extension, and features are almost never critical anyway, downgrading from critical to normal

  • Comment 3 months ago →
    anantaisaustin

    Changed ReadMe.txt to ReadMe.md

  • Comment 3 months ago →
    anantaisaustin
  • Merge request !36#3058121: Add persistence for memcache(d). → (Open) created by jan kellermann
  • Pipeline finished with Success
    3 months ago
    Total: 210s
    #259678
  • Comment 3 months ago →
    anantaisaustin

    Posting the static patch because using the MR doesn't allow pinning to a specific commit, so anyone can submit pretty much anything and inject it into our codebase IIRC. This also fixed our issue.

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024