Query count exceptionally high

Created on 3 May 2023, about 1 year ago
Updated 25 September 2023, 9 months ago

Problem/Motivation

When enabling this module, my query count for an uncached page jumps from ~700 to ~3600. After inspecting the queries it seems like there are A LOT of these (2900 of the 3600):

SELECT "base_table"."id" AS "id", "base_table"."id" AS "base_table_id" FROM "domain_path" "base_table" INNER JOIN "domain_path" "domain_path" ON "domain_path"."id" = "base_table"."id" WHERE ("domain_path"."alias" IN ('/node/3')) AND ("domain_path"."domain_id" IN ('my_domain'))

There seems to be no cache for these lookups, so if the same link is rendered multiple times on the same page, then each time a fresh query is done to the DB to look up domain path aliases.

Steps to reproduce

  • Install domain_path module
  • Inspect queries with webprofiler module.

Proposed resolution

Add some sort of caching? So that the domain_path custom paths can be looked up once in cache rebuild and re-used in memory throughout that page request.

Also, does the module still need to process outbound links for EVERYTHING? e.g. maybe admin menu links could be skipped to save some lookups?

πŸ› Bug report
Status

Needs review

Version

1.0

Component

Code

Created by

πŸ‡¬πŸ‡§United Kingdom mattjones86 πŸ‡¬πŸ‡§ GMT+0

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @mattjones86
  • πŸ‡¬πŸ‡§United Kingdom nicholasThompson

    Having the same issue in 1.3 too...

    @mattjones86 did you make any headway on this?

  • πŸ‡¬πŸ‡§United Kingdom nicholasThompson

    I've implemented the static caching for lookup and noPath (like the AliasManager) does...

    This brought the homepage down from 5788 queries to 3871 (and knocked 2 seconds off)... But thats still about 3,500 queries too many ;)

    The "extend" modules page was 18k queries and thats down to 11k now, too... but that take 45 seconds to load.

    (all the above is on a local ddev instance).

  • πŸ‡¬πŸ‡§United Kingdom nicholasThompson

    Very much work in progress....

  • πŸ‡¬πŸ‡§United Kingdom nicholasThompson

    I've also just hugely helped the DB by adding a new index on alias, domain_id and language. This didn't help the NUMBER of queries, but it has hugely helped the time; page is down to 1.5s now and DB query time from 2.5s to 0.5s.

    Looked onto per-page caching (which is the next massive win, really) and its non-trivial to add (needs new writeCache and key generation + an event listener to write the cache on page close).

    Will re-roll the patch in a bit...

  • πŸ‡¬πŸ‡§United Kingdom nicholasThompson

    Just looked at the query itself... its really odd... why does it return the same ID field twice in two different field names... and why does it join the domain_path table to itself by ID?

  • Assigned to nicholasThompson
  • Status changed to Needs review 12 months ago
  • πŸ‡¬πŸ‡§United Kingdom nicholasThompson

    Revised patch - this one includes the index.

  • πŸ‡¬πŸ‡§United Kingdom nicholasThompson

    Hmmm - the noAlias stuff isn't right and this has also included some language selection fixes we were using in https://www.drupal.org/project/domain_path/issues/3245281#comment-15102240 πŸ› Wrong Language is chosen and no node visibe under certian conditions Needs work but no longer seem to behave the same.

  • πŸ‡¬πŸ‡§United Kingdom nicholasThompson

    This, so far, seems more reliable for me. Fixed the issue where it was returning the wrong content for the domain.

    Also fixes the "no result" cache, which brings out homepage down to 2569 queries (from the original ~5,700).. Query time is down to 400ms and page render to 1.3s (on ddev locally as admin - which is doing a lot of other lookups for admin paths, etc).

    I guess we could do a separate module which adds the `/admin` path to the whitelist (I'm aware there is an issue open to rename this, I'm using it for consistency with the current codebase) - this might help further, but would mean you couldn't alias any admin URLs (which is probably a rare thing anyway).

  • πŸ‡©πŸ‡ͺGermany webflo
    +++ b/src/Entity/DomainPath.php
    @@ -23,6 +23,7 @@ use Drupal\domain_path\DomainPathInterface;
    + *     "storage_schema" = "Drupal\domain_path\DomainPathStorageSchema",
    

    Looks promising. But the DomainPathStorageSchema file is not included in your patch.

  • πŸ‡©πŸ‡ͺGermany webflo
Production build 0.69.0 2024