Contrib.social

Add a garbage collector to clean up the queue for failed sitemap batches

Open on Drupal.org →
Created on 8 February 2023, almost 2 years ago

Problem/Motivation

The problem is, I'm working on a website with more than 10k items to be indexed, and the items are all reindexed every day, after some point, the sitemap batch stopped to work properly, due to a custom module that we have here, the main problem was, it took some days to realize this kind of behavior, and after taking a look at the queue table, we noticed the table with more than 100k items, the sitemap was taking a long time to run. Anyway, after truncating the table the slowness problem was solved.

The idea of this issue is to implement a garbage collector for failed items. Drupal already provides this kind of stuff, if you take a look at the DatabaseQueue class , you'll find a garbage collector method, the problem is, this method expects a drupal_batch: pattern on the name of the queue item, but sitemap uses simple_sitemap_elements. Drupal will clean up the queue table with the correct after 10 days.

Proposed resolution

My idea here is to update the hook cron and add the properly clean up here.

We've could also add a config to configure how much do we want to keep the register on the table.

📌 Task
Status

Active

Version

4.0

Component

Code

Created by

🇧🇷Brazil murilohp

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024