Use the flood system to rate limit security advisories fetching during cron

Created on 3 May 2021, about 4 years ago
Updated 9 June 2025, 26 days ago

Problem/Motivation

While reviewing #3041885: Display relevant Security Advisories data for Drupal β†’ , @catch called out an implementation detail in https://git.drupalcode.org/project/drupal/-/merge_requests/284#note_22874 (emphasis mine, to show the potential benefit):

Relying on key value expire for the data to be refreshed means http requests on admin pages when it's empty (only once every 12 hours, but on a low-admin-traffic site that could be the first admin request every visit).
I think we could do the following - would be more robust in the case the feed is down and no impact on admin page views:

  1. In system_cron(), poll Drupal.org. Keep the fallback though in case the entry is empty.
  2. Use the flood system to rate limit the polling requests on cron, so that it respects the interval setting - so each hook_cron() call finishes early if flood acquire fails and still only does the actual work every 12 hours.
  3. When the interval is changed in config to a lower value, instead of deleting the data, we could clear the flood record - this will still force an update next cron run but means the current data is preserved in the meantime.
  4. We could set the expire in k/v to interval * 2 or interval * 1.5 so that it does still eventually expire if outbound requests consistently fail.

Steps to reproduce

N/A

Proposed resolution

Discuss if it's worth doing what @catch suggests, and implement it if so.

Remaining tasks

Decide if we want to make this change, and if we do, write a merge request with tests. Then commit it.

User interface changes

None.

API changes

None. This functionality is not an API.

Data model changes

None.

Release notes snippet

TBD, but likely none.

✨ Feature request
Status

Active

Version

11.0 πŸ”₯

Component

system.module

Created by

πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • πŸ‡¦πŸ‡ΊAustralia mstrelan

    In a discussion on the original MR @tedbow pointed out that \Drupal\system\SecurityAdvisories\SecurityAdvisoriesFetcher::getSecurityAdvisories is called with $allow_outgoing_request = FALSE in hook_page_top. When it's called from hook_cron and hook_requirements this param is TRUE.

    If I understand correctly, @catch originally suggested we add flood control to system_cron, but in light of the above I think we can put it in ::getSecurityAdvisories when the bool is TRUE.

  • Merge request !12327Use flood β†’ (Open) created by mstrelan
  • πŸ‡¦πŸ‡ΊAustralia mstrelan

    Made a start, needs some tests, should use DI too.

  • Pipeline finished with Failed
    26 days ago
    Total: 600s
    #517453
  • πŸ‡¬πŸ‡§United Kingdom catch
Production build 0.71.5 2024