Docs: Clarify recommendation to use Purge's Late Runtime Processor

Created on 2 February 2024, 5 months ago
Updated 27 June 2024, about 23 hours ago

Problem/Motivation

The guidance to enable git purge_processor_lateruntime feels a little conflicting.

Purge module's description for Late Runtime Processor says (emphasis mine):

Process the queue on every request, this is only recommended on high latency configurations.

Acquia Purge module says purge_processor_lateruntime is necessary and further warns if purge_processor_lateruntime is not enabled (emphasis mine again):

For an optimal experience, you're recommended to enable the cron processor and the late runtime processors simultaneously. These two processors will complement each other and assure that the queue is processed as fast as possible.

This messaging seems a bit contradictory; Purge (which is "owner" of purge_processor_lateruntime) says only to use it on "high-latency configurations". Acquia says to use both. What a "high-latency configuration" means exactly is not qualified (that I can find).

Steps to reproduce

Refer above. First excerpt shows on modules page (/admin/modules) in Drupal UI, second on status report (/admin/reports/status) in Drupal UI.

Proposed resolution

Check that guidance is consistent, clarify if appropriate.

Remaining tasks

User interface changes

API changes

Data model changes

๐Ÿ“Œ Task
Status

Active

Version

1.0

Component

Documentation

Created by

๐Ÿ‡ณ๐Ÿ‡ฟNew Zealand xurizaemon ลŒtepoti, Aotearoa ๐Ÿ

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @xurizaemon
  • ๐Ÿ‡ณ๐Ÿ‡ฟNew Zealand xurizaemon ลŒtepoti, Aotearoa ๐Ÿ
  • ๐Ÿ‡ณ๐Ÿ‡ฟNew Zealand xurizaemon ลŒtepoti, Aotearoa ๐Ÿ
  • ๐Ÿ‡ณ๐Ÿ‡ฟNew Zealand xurizaemon ลŒtepoti, Aotearoa ๐Ÿ
  • ๐Ÿ‡ณ๐Ÿ‡ฟNew Zealand xurizaemon ลŒtepoti, Aotearoa ๐Ÿ
  • ๐Ÿ‡ฉ๐Ÿ‡ชGermany D34dMan Hamburg

    @xurizaemon,
    What is the latency that you experience if you use purge_processor_cron?

    For example if you run cron every 10 mins, and 10 minutes is defined as "SLOW"/high latency, then you can benefit from "purge_processor_lateruntime" almost instantaneous invalidation event.

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States dan612 Portland, Maine

    I would not use the lateruntime processor on any semi-large scale production site. It will hijack/piggyback a request and can cause performance issues. This is recommended from Acquia here - https://acquia.my.site.com/s/article/360005311513--Drupal-8-Acquia-Purge...

    Busy sites should DISABLE the lateruntime processor, because it runs when users are logged in, it can increase load on your site(s).

    And further emphasized by this issue to remove it from Acquia Purge - https://www.drupal.org/project/acquia_purge/issues/2925381 โ†’ .

    The issue then becomes, on Acquia Cloud (non-classic), you can only configure cron at 5 minute intervals. So if you set up your p:queue-work to run at this minimum it will mean editors looking for their changes will have to wait a variable amount of time (up to 5 mins) to see the changes. For some sites this may not be an acceptable in which case you would need to set up custom purging from actions on the site - https://docs.acquia.com/acquia-cloud-platform/performance/varnish/manual.... It becomes a tradeoff:

    1. Rely on late runtime processor and deal with some slow requests / performance issues + spikes
    2. Rely on cron to purge the queue every 5 mins
    3. Custom develop a solution to make BAN requests and invalidate necessary items in real time

    Maybe there are more strategies than listed above, but that is where I have netted out after dabbling with this!

  • ๐Ÿ‡ณ๐Ÿ‡ฟNew Zealand xurizaemon ลŒtepoti, Aotearoa ๐Ÿ

    What is the latency that you experience if you use purge_processor_cron?

    @D34dMan I don't understand this question, sorry, but I'll try to answer anyway :)

    I don't understand how "latency" is being used in the module documentation. I would not consider a site hosted on a managed platform, eg Acquia's, to be "high latency". If asked to define "high latency" I'd probably say "network round trip time over 250ms", which would hopefully exclude any professional hosting environment.

    However, it doesn't make sense to me to think of latency as ping time here, unless such latency was internal to the hosting environment. So I'm my conclusion is that I don't know how the writer is using "high latency" in this context.

    My approach for the one site we have using Acquia Purge is pretty much as Dan162 suggests, though I hadn't yet investigated the Purge Queues module.

    This issue is really about the guidance provided via module documentation, Acquia's docs, and the status report warning displayed. To me the guidance seems to conflict (as noted in ID above), and I am hoping that it can be clarified.

    My interest in understanding the recommended configuration is because of a site which has some issues which appear to relate to purge queues, detailed here (no issue link as yet; the cause seems to be Acquia Cohesion, which does not appear to have an issue queue). When we enabled the Late Runtime Purge Processor, we observed negative impact on site performance, and it did not process purge queues any better than cron operation. So we disabled it again.

    Thanks both for your input! It's appreciated.

  • ๐Ÿ‡ฉ๐Ÿ‡ชGermany D34dMan Hamburg

    > I don't understand how "latency" is being used in the module documentation

    I assumed the latency to be defined as "the time between the content is updated in Drupal and the request is made to the CDN to purge the cache".

    So if the cron is running every 15 mins, then a latency of upto 15 mins can be experienced, considering a single cron run is able to handle all the items in the queue. If not, then the latency would increase until you get to see the error that "Your queue exceeded...."

Production build 0.69.0 2024