Data cached by EnabledSourceHandler never expires

Created on 17 July 2025, 6 days ago

Problem/Motivation

EnabledSourceHandler caches query results in non-volatile storage. But those results never expire. Ever. You have to run a Drush command to clear it out; a regular cache rebuild won't do it, since they're not stored in the regular cache.

This is a problem because it means you can be stuck with outdated results indefinitely, with no way to resolve problem unless you have command-line access.

Proposed resolution

EnabledSourceHandler should use the expirable key-value service and cache items for a day by default. We don't need to worry about making the TTL configurable; if we want that, we can do it in a follow-up.

πŸ› Bug report
Status

Active

Version

2.1

Component

Code

Created by

πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @phenaproxima
  • Merge request !828Switch to KV expirable β†’ (Merged) created by phenaproxima
  • Pipeline finished with Canceled
    6 days ago
    Total: 480s
    #550518
  • Pipeline finished with Failed
    6 days ago
    Total: 347s
    #550527
  • Pipeline finished with Failed
    6 days ago
    Total: 360s
    #550533
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    Been thinking on this a bit more, and the problem comes down to the fact that EnabledSourceHandler stands in deep violation of the single-responsibility principle. It does three things:

    • It's a quasi-plugin manager, only letting us access enabled sources.
    • It's a caching layer for query results.
    • It's a stable, non-volatile repository of every project we have encountered while using Project Browser.

    Fully cleaning this up is not in scope here. But I think we can create a cleaner separation between points #2 and #3.

    Query results are just that: "when you give me these criteria, here are the projects I found". These should use a normal, volatile cache backend.

    Stored projects are relied upon by the activators, so they cannot be subject to volatile caching. Clearing the stored projects needs to be -- and is -- a separate operation from querying.

    So I think my proposed resolution is correct -- use the cache system for query results, and always update stored projects with whatever the source plugins return. Project object storage, though, should stay as it is, in non-expiring key-value stores.

  • Pipeline finished with Failed
    6 days ago
    Total: 595s
    #550643
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts
  • Pipeline finished with Canceled
    6 days ago
    Total: 267s
    #550646
  • Pipeline finished with Canceled
    6 days ago
    Total: 281s
    #550650
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    Assigning to @tim.plunkett specifically, since this kind of stuff is very much in his wheelhouse.

  • Pipeline finished with Failed
    6 days ago
    Total: 341s
    #550653
  • Pipeline finished with Failed
    6 days ago
    Total: 258s
    #550657
  • Pipeline finished with Success
    6 days ago
    Total: 345s
    #550658
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    This should be backported.

  • Pipeline finished with Failed
    5 days ago
    Total: 732s
    #551344
  • Pipeline finished with Failed
    5 days ago
    Total: 323s
    #551362
  • Pipeline finished with Failed
    5 days ago
    Total: 248s
    #551364
  • Pipeline finished with Failed
    5 days ago
    Total: 337s
    #551367
  • Pipeline finished with Canceled
    5 days ago
    Total: 257s
    #551386
  • Pipeline finished with Failed
    5 days ago
    Total: 321s
    #551391
  • Pipeline finished with Failed
    5 days ago
    Total: 411s
    #551398
  • Pipeline finished with Failed
    4 days ago
    Total: 314s
    #551559
  • Pipeline finished with Failed
    4 days ago
    Total: 870s
    #551562
  • Pipeline finished with Failed
    4 days ago
    Total: 492s
    #551574
  • Pipeline finished with Failed
    4 days ago
    Total: 438s
    #551582
  • Pipeline finished with Success
    4 days ago
    Total: 1365s
    #551589
  • Pipeline finished with Canceled
    4 days ago
    Total: 138s
    #551609
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    Decided to expand the scope here to encompass the full clean-up of EnabledSourceHandler into better-scoped, single-responsibility classes. Need to update the issue summary to reflect that.

  • Pipeline finished with Success
    4 days ago
    #551612
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts
  • Pipeline finished with Failed
    4 days ago
    Total: 378s
    #551637
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts
  • Pipeline finished with Canceled
    4 days ago
    Total: 215s
    #551640
  • Pipeline finished with Success
    4 days ago
    Total: 337s
    #551641
  • Pipeline finished with Failed
    4 days ago
    Total: 424s
    #551680
  • Pipeline finished with Success
    4 days ago
    Total: 433s
    #551682
  • Pipeline finished with Failed
    4 days ago
    Total: 383s
    #551684
  • Pipeline finished with Canceled
    4 days ago
    Total: 320s
    #551686
  • Pipeline finished with Success
    4 days ago
    #551687
  • πŸ‡ΊπŸ‡ΈUnited States tim.plunkett Philadelphia

    Walked through this with @phenaproxima and asked for one additional code-comment, but this is great work!

  • Pipeline finished with Success
    1 day ago
    Total: 430s
    #553272
  • Merge request !831Begin backport β†’ (Merged) created by phenaproxima
  • Pipeline finished with Failed
    1 day ago
    #553445
  • Pipeline finished with Failed
    1 day ago
    #553448
  • Pipeline finished with Failed
    1 day ago
    #553450
  • Pipeline finished with Failed
    1 day ago
    #553457
  • Pipeline finished with Success
    1 day ago
    #553474
  • Pipeline finished with Skipped
    about 12 hours ago
    #554355
  • First commit to issue fork.
  • Pipeline finished with Skipped
    about 12 hours ago
    #554356
  • πŸ‡ΊπŸ‡ΈUnited States chrisfromredfin Portland, Maine

    !831 merged to 2.1.x

  • Pipeline finished with Skipped
    about 12 hours ago
    #554358
  • πŸ‡ΊπŸ‡ΈUnited States chrisfromredfin Portland, Maine

    and !828 now in 2.0.x

Production build 0.71.5 2024