Contrib.social

Delete items from field data and field revision tables should be done in bulk

Open on Drupal.org →
Created on 17 June 2024, 6 months ago
Updated 18 June 2024, 6 months ago

Problem/Motivation

Deleting 100M content entities is slow.

Steps to reproduce

1. Create 100M content entities on an entity with 100 fields
2. Delete all the content entities
3. Look at query log and see unoptimized deletion:

Proposed resolution

In SqlContentEntityStorage::doDeleteFieldItems we delete data from the base and revision tables in bulk, but we call ::deleteBulkFromDedicatedTables one at a time:

    if ($this->revisionDataTable) {
      $this->database->delete($this->revisionDataTable)
        ->condition($this->idKey, $ids, 'IN')
        ->execute();
    }

    foreach ($entities as $entity) {
      $this->deleteFromDedicatedTables($entity);
    }

That results in a ton of separate SQL queries that could be grouped into one.

It seems pretty straightforward to do, as deleteFromDedicatedTables() is specific to SqlContentEntityStorage.

Making this change resulted in the duration going from several days to hours.

Remaining tasks

User interface changes

API changes

Data model changes

Release notes snippet

🐛 Bug report
Status

Needs work

Version

11.0 🔥

Component
Entity  →

Last updated about 17 hours ago

Created by

🇺🇸United States djdevin Philadelphia

Live updates comments and jobs are added and updated live.
  • Needs tests

    The change is currently missing an automated test that fails when run with the original code, and succeeds when the bug has been fixed.

Sign in to follow issues

Merge Requests

Comments & Activities

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024