acquia_contenthub_publisher_update_91002 makes MySQL go away

Created on 30 July 2024, 7 months ago

Problem/Motivation

acquia_contenthub_publisher_update_91002 makes MySQL go away when the acquia_contenthub_publisher_export_tracking table contains thousands of records. Our acquia_contenthub_publisher_export_tracking table contains >86k records.

Steps to reproduce

Update acquia_contenthub to v3.5.x

Proposed resolution

I'm developing a queue worker, which should help, although a queue worker creating that many items may also fail. If the hook update could somehow be made more efficient, that would be great. Suggestions are welcome.

Remaining tasks

  1. Get the hook update to work by either
  • Make acquia_contenthub_publisher_update_91002 more efficient.
  • or
  • Add a queue worker to the acquia_contenthub_publisher.
  • Test it to confirm it works.
  • User interface changes

    None

    API changes

    None

    Data model changes

    None

    πŸ› Bug report
    Status

    Active

    Version

    3.5

    Component

    Code

    Created by

    πŸ‡ΊπŸ‡ΈUnited States esod

    Live updates comments and jobs are added and updated live.
    Sign in to follow issues

    Comments & Activities

    • Issue created by @esod
    • πŸ‡ΊπŸ‡ΈUnited States esod

      A core bug is preventing the update from completing. The bug is an Annotation bug in Editor.php. Here is the bug.

      Doctrine\Common\Annotations\AnnotationException [Semantical Error] The class "Editor" is not annotated with @Annotation. Are you sure this class can be used as annotation? If so, then you need to add @Annotation to the _class_ doc comment of "Editor". If it is indeed no annotation, then you need to add @IgnoreAnnotation("Editor") to the _class_ doc comment of class @Drupal\editor\Annotation\Editor.

      core/modules/editor/src/Annotation/Editor.php includes @Annotation. But since the editor entity_id is for only 2 records in the acquia_contenthub_publisher_export_tracking table, my patch manually updates them so as to avoid the annotation issue with the Editor service class.

      However, even with the patch successfully applied to our DEV environment, MySQL still goes away after ~20k record updates, ~15 - 20 minutes.

      My patch is tending a bit towards a custom patch, but I'm going to add it here because I also don't think the issue I'm experiencing with the acquia_contenthub_publisher_update_91002() is unique. I think any large system is going to eventually have tens of thousands of records in its publisher export tracking table. The attached patch may encourage others to chime in. Otherwise, the continuing discussion on this issue will be with Acquia support.

    • πŸ‡ΊπŸ‡ΈUnited States esod

      Here's a better version of the patch. The server still times out after ~21k record updates.

    • πŸ‡ΊπŸ‡ΈUnited States esod

      We have >90,000 rows in the acquia_contenthub_publisher_export_tracking table at this point. I appreciate that an effort was made to address this issue with release v.3.5.2. Unfortunately, it's still failing:

      Fatal error: Allowed memory size of 1073741824 bytes exhausted (tried to allocate 163840 bytes) in /mnt/www/html/ojp01test/docroot/core/lib/Drupal/Core/Database/StatementWrapperIterator.php on line 113 
      

      After running the hook updates 3 times on our test environment.

      mysql> SELECT COUNT(*) FROM acquia_contenthub_publisher_export_tracking WHERE label IS NULL;
      +----------+
      | COUNT(*) |
      +----------+
      | 71680 |
      +----------+

      Same count after running the hook updates a 4th time.

    Production build 0.71.5 2024