- Issue created by @kufliievskyi
- Merge request !32Issue #3508480: Fixed the tome_static_cron module, added limit for... → (Open) created by Unnamed author
I'm using tome_static_cron on a site with ~60,000 paths. When I process the scheduled queue items, I see the performance issues. The exporting is slowing down after ~1 hour significantly.
The Tome Static Cron export process has a bottleneck due to excessive accumulation of invoke_paths in the key_value table. When processing scheduled paths, the `requestPath` method not only exports the required path but also discovers additional asset files (e.g., CSS, JS) and continuously adds them to the invoke_paths list. Over time, this list grows too large, leading to performance degradation when we merge new invoke_paths with the existing ones and set it back into the key_value table(option name= tome_static_cron.invoke_paths).
The code related to the described problem is placed in the ./tome_static/modules/tome_static_cron/src/Plugin/QueueWorker/TomeStaticQueueWorker.php file.
This query shows that invoke_paths become excessively large and take more than a second to process for each execution:
SELECT * FROM key_value WHERE name = 'tome_static_cron.invoke_paths';
The scheduling code in `tome/modules/tome_static/modules/tome_static_cron/tome_static_cron.module` indicates that `process_invoke_paths` is the final step of the export process. This suggests that the bottleneck lies in the handling of invoke_paths, and a solution is needed to prevent continuous accumulation and inefficient processing.
I think that this solution is a kind of workaround to the existing way of storing and processing the queue items.
Probably, the better solution could be to store `invoke_paths` in a separate database table and avoid exporting the same assets multiple times.
But this solution will require more changes and I would want to delegate it to the module maintainers or someone who has time.
- Enable Tome static export with scheduled path processing.
- Run an export process that includes dynamically discovered assets.
- Observe the increasing size of tome_static_cron.invoke_paths in the key_value table.
- Note the degradation in performance as the query time increases.
Modify the TomeStaticQueueWorker to process invoke_paths more efficiently:
No changes are expected.
No changes are expected.
Potentially adjusting how invoke_paths are stored and processed in the key_value table.
Active
1.0
Tome Static
It affects performance. It is often combined with the Needs profiling tag.