Add indexes to Voting API entity tables to speed up result calculations

Created on 22 March 2023, over 1 year ago
Updated 3 October 2023, 9 months ago

Problem/Motivation

If there are a lot of votes (and lot of vote results), then vote result recalculation takes more and more time. This isn't an issue if we have hundreds of thousands of entities, but e.g. if we have 600 million votes, only performing the result delete query in VoteResultFunctionManager::recalculateResults() takes noticeable amount of time.

Steps to reproduce

  1. Create a site with millions of votes.
  2. Create a vote on an entity which has 100k+ votes with the same vote type.
  3. Trigger vote result recalculation.

Proposed resolution

Add extra indexes both to the table of vote entities and vote result entities.

  1. Create schema handlers for both vote and vote_result entities.
  2. Add a new index to their base table. Since the query conditions are quite the same, the extra index is identical (entity_type, entity_id, (vote) type).
  3. Add an update hook that adds the new indexes.

Remaining tasks

TBD.

User interface changes

Nothing.

API changes

Only non-public changes.

Data model changes

Nothing.

✨ Feature request
Status

Needs work

Version

3.0

Component

Code

Created by

πŸ‡­πŸ‡ΊHungary huzooka Hungary πŸ‡­πŸ‡ΊπŸ‡ͺπŸ‡Ί

Live updates comments and jobs are added and updated live.
  • Needs tests

    The change is currently missing an automated test that fails when run with the original code, and succeeds when the bug has been fixed.

Sign in to follow issues

Comments & Activities

  • Issue created by @huzooka
  • @huzooka opened merge request.
  • Issue was unassigned.
  • Status changed to Needs review over 1 year ago
  • πŸ‡­πŸ‡ΊHungary huzooka Hungary πŸ‡­πŸ‡ΊπŸ‡ͺπŸ‡Ί

    Old-school patch attached.

  • πŸ‡­πŸ‡ΊHungary huzooka Hungary πŸ‡­πŸ‡ΊπŸ‡ͺπŸ‡Ί
  • πŸ‡¬πŸ‡§United Kingdom jaydenpearly

    I thought of applying the patch when it comes to the importantn issue and performance is concerned.
    I actually have no idea about how it indexes or what it is :))
    You probably know something. thanks.

  • πŸ‡¬πŸ‡§United Kingdom jaydenpearly

    I've been using it for a while. I didn't have a problem. Of course, since I do not have a large amount of data during this time, I cannot test it.

    My curiosity; Why was the patch on such an important issue not merched until now?
    Do you think we should use this patch on a prodcution site?

  • πŸ‡ΊπŸ‡ΈUnited States TR Cascadia

    Why has it not been merged? Because it is a patch proposed by one person and no one else has reviewed it to confirm it works. Because no one has written a test of the hook_update, and no one has written a test of the new storage class to confirm this patch works and doesn't break sites. We don't even have any benchmarking data to determine if this actually helps, or if it does help then how much.

    I'm not going to casually commit a patch that modifies the database of everyone using Voting API without some automated tests and without some feedback from the community. This is just basic good practice, and is a requirement for Drupal core issues.

    This may be a very useful patch, but so far no one has tried it out to confirm that. That's why the issue is still set to "Needs review".

  • πŸ‡¬πŸ‡§United Kingdom jaydenpearly

    Ok,I see. I'm new to the community. And I am just getting to know this working culture.
    Thanks for your reply.

  • Status changed to RTBC 9 months ago
  • heddn Nicaragua

    Tested the hook update and the indexes themselves. As mentioned in the OP, it doesn't help a lot for those of us with only a few thousand votes. My site has ~766K votes and ~3300 results. To manually calculate votes on 410 nodes for a total of 18K votes, with and without the patch it takes about ~19-20 seconds. But I imagine that is because there are only 3300 vote results in my DB. This proves that 1) performance doesn't suffer 2) the update hook is in a good state.

  • Status changed to Needs work 9 months ago
  • πŸ‡ΊπŸ‡ΈUnited States TR Cascadia
Production build 0.69.0 2024