Adding an index to the 'schemadotorg_types' table

Created on 7 March 2024, 4 months ago
Updated 30 March 2024, 3 months ago

Problem/Motivation

A few methods in the "schemadotorg.schema_type_manager" service search by the 'enumerationtype' column. When there are about 30k-60k users on the site, a decrease in response time on the page can be observed. Adding an index will speed up the search in the table.

Steps to reproduce

The presence of a large number of users on the site is required.

Proposed resolution

Add an index to the schemadotorg_types table.

Remaining tasks

n/a

User interface changes

n/a

API changes

n/a

Data model changes

n/a

πŸ“Œ Task
Status

Fixed

Version

1.0

Component

Code

Created by

πŸ‡΅πŸ‡±Poland damian.skiba

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @damian.skiba
  • πŸ‡ΊπŸ‡ΈUnited States jrockowitz Brooklyn, NY

    We can easily add indexing to the schemadotorg_type table.

    We might want to improve caching within \Drupal\schemadotorg\SchemaDotOrgSchemaTypeManager because this would reduce the number of database queries. The good news about caching Schema.org data is it will never change unless the Schema.org Blueprints module is updated.

  • πŸ‡΅πŸ‡±Poland damian.skiba

    Today I added drupal_static function and caching to the \Drupal\schemadotorg\SchemaDotOrgSchemaTypeManager::getDataTypes method. Before applying the patch, the \Drupal\schemadotorg_jsonld\SchemaDotOrgJsonLdBuilder::build method was generating 221 database queries. After applying the patch, the number of queries decreased to 17. On my local environment, there is a difference of 0.2 seconds.

  • πŸ‡ΊπŸ‡ΈUnited States jrockowitz Brooklyn, NY

    Yep, I think our first goal is to reduce the number of database queries for some key methods in \Drupal\schemadotorg\SchemaDotOrgSchemaTypeManager

  • πŸ‡ΊπŸ‡ΈUnited States g.paluch

    I want to confirm that the patch drastically improved the DB session load. I attached the before-and-after test results. We can still observe the load, but in our case, this is in range. Thank you, everyone, for your work on the solution!

    Before the patch

    After the patch

  • πŸ‡ΊπŸ‡ΈUnited States jrockowitz Brooklyn, NY

    The MR creates a cache storing information about schema.org types and properties. Right now, the patch still reduces the number of queries a little bit more than the MR.

    Below are stats for generating the same JSON-LD

    Current w/o JSONLD: Db queries 273 / Query time 0.04 ms
    Current: Db queries 1474 / Query time 0.22 ms
    Patch: Db queries 957 / Query time 0.12 ms
    MR: Db queries 978 / Query time 0.12 ms

    I might be able to make one more optimization.

    We should consider caching the generated JSON-LD via cache tags.

    For now, use the patch.

  • πŸ‡ΊπŸ‡ΈUnited States jrockowitz Brooklyn, NY

    The updated MR gets the queries down to

    Db queries 940 / Query time 0.12 ms

  • πŸ‡ΊπŸ‡ΈUnited States jrockowitz Brooklyn, NY

    πŸ“Œ Add caching to Schema.org JSON-LD via \BubbleableMetadata and render caching Active should address most performance issues.

    I would still like to reduce the overall number of database queries via this ticket.

  • πŸ‡ΊπŸ‡ΈUnited States jrockowitz Brooklyn, NY

    Attached is the simplest solution which assumes Data types can be hard coded because they never change.

  • Status changed to Fixed 3 months ago
  • πŸ‡ΊπŸ‡ΈUnited States jrockowitz Brooklyn, NY

    I went with the last patch.

    If you update to use #3427909, you will see a huge performance improvement, as JSON-LD only adds a single DB query.

    • 702f45c0 committed on 1.0.x
      Issue #3426422: Adding an index to the 'schemadotorg_types' table
      
  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.69.0 2024