Improve performance of search totals calculation

Created on 13 May 2008, almost 17 years ago

Updated 6 May 2025, 2 days ago

Not LIVE FROM THE MINNESOTA SEARCH SPRINT, but found on the plane ride home...

After cron completes indexing a chunk of nodes, search_update_totals() updates {search_total} with the new totals for each modified word. It performs some of the calculations in php and thus makes 3 SQL statements for each word. Since you can update 100's of nodes with 1000's of words, that's a lot of SQL to perform at the end of each cron run.

The attached patch greatly simplifies this by updating 500 words at a time. I arbitrarily limited it 500 words, not knowing what was a good limit to the SQL packet size.

The one concern I have about this approach is databases other than mysql. I'm relying on the non-standard function LOG10(). I believe that we use GREATEST() somewhere else, so I don't think that this is an issue. So, we need to check if pgsql supports LOG10().

And we should discuss how many words at a time are reasonable to update.

📌 Task

Status

Postponed: needs info

Version

11.0 🔥

Component

search.module

Created by

🇺🇸United States douggreen Winchester, VA

Live updates comments and jobs are added and updated live.

Performance
It affects performance. It is often combined with the Needs profiling tag.

Needs backport to D6
After being applied to the 7.x branch, it should be considered for backport to the 6.x branch. Note: This tag should generally remain even after the backport has been written, approved, and committed.

Needs backport to D7
After being applied to the 8.x branch, it should be considered for backport to the 7.x branch. Note: This tag should generally remain even after the backport has been written, approved, and committed.

stale-issue-cleanup
To track issues in the developing policy for closing stale issues, [Policy, no patch] closing older issues

Incomplete comments

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

Comment 2 days ago →
🇺🇸United States smustgrave
Thank you for creating this issue to improve Drupal.

We are working to decide if this task is still relevant to a currently supported version of Drupal. There hasn't been any discussion here for over 8 years which suggests that this has either been implemented or is no longer relevant. Your thoughts on this will allow a decision to be made.

Since we need more information to move forward with this issue, the status is now Postponed (maintainer needs more info). If we don't receive additional information to help with the issue, it may be closed after three months.

Thanks!

contrib.social Blog FAQ Discussions

Production build 0.71.5 2024