Support for query splitting to Replica in core

Created on 29 May 2019, over 5 years ago
Updated 18 September 2024, about 2 months ago

I understand how to configure the $database['default']['replica'][] settings so that I can point to 1..N replica servers.

I do not understand how to get anything but my custom-written queries to leverage the Replica servers.

This seems to me like a core feature to supporting multiple databases in core, but I see no clear path to handling this logic, nor do I see a clear place to override this

I've even tried

 if (!isset($options['target']) && stripos($query, 'SELECT') === 0) {
        $options['target'] = 'replica';
      }
      return parent::query($query, $args, $options);

in
Drupal\Core\Database\Driver\mysql\Connection function query(
but whatever other processes are involved with loading any page in Drupal appear to mark the replica as invalid, so I never actually have any queries that execute against the replica in this scenario.

This seems to me like a core scaling issue for Drupal, so I'm not clear on if I'm just misunderstanding how this is supposed to work, or if the only recommendation is to use an external process such as proxySQL or other mechanism to split read/write queries across master/slave (primary/replica)

✨ Feature request
Status

Needs work

Version

11.0 πŸ”₯

Component
MySQL driverΒ  β†’

Last updated about 2 months ago

Created by

πŸ‡ΊπŸ‡ΈUnited States craigmc

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • The Needs Review Queue Bot β†’ tested this issue. It either no longer applies to Drupal core, or fails the Drupal core commit checks. Therefore, this issue status is now "Needs work".

    Apart from a re-roll or rebase, this issue may need more work to address feedback in the issue or MR comments. To progress an issue, incorporate this feedback as part of the process of updating the issue. This helps other contributors to know what is outstanding.

    Consult the Drupal Contributor Guide β†’ to find step-by-step guides for working with issues.

  • Open in Jenkins β†’ Open on Drupal.org β†’
    Environment: PHP 8.1 & MariaDB 10.3.22
    last update about 1 year ago
    Build Successful
  • last update about 1 year ago
    Patch Failed to Apply
  • last update about 1 year ago
    Patch Failed to Apply
  • πŸ‡―πŸ‡΄Jordan Anas_maw

    This is a very important issue, especially for high-traffic websites.

  • πŸ‡ͺπŸ‡ͺEstonia hkirsman

    +1

  • πŸ‡¬πŸ‡§United Kingdom matason

    Hi folks,

    I've had a look at the patch and it's pretty much the direction I was going in, I'd start work on it locally before checking the issue queue...

    Having reached the same point, I started to think about it bit more... regarding SqlContentEntityStorage, I think it could be a good idea to separate out the read operations from the write options.

    I'm thinking SqlContentEntityStorage could become a facade with two underlying objects, a reader and a writer. That way, instead of introducing more execution paths to the code in the class "if we're reading, use a replica, if we're writing, use default", we can just pass the database.replica to the reader, safe in the knowledge that it'll get default if no replica's are configured and pass default through to the writer.

    Before I go off and attempt to carve this out, I just thought I'd share the idea to see if sounds sensible to anyone else?

  • πŸ‡ΈπŸ‡ͺSweden emek

    We have a few high-traffic sites and could use this functionality.

Production build 0.71.5 2024