Create a centralized, non-volatile repository of all projects known to all sources

Created on 5 June 2024, 23 days ago
Updated 20 June 2024, 8 days ago

Problem/Motivation

✨ Make it possible for Project Browser to activate different kinds of projects Fixed is a game changer for Project Browser. Right now, source plugins are really just different sources of modules, but as of that issue, source plugins are actually different sources of projects, which are identified by a fully qualified ID consisting of $SOURCE_PLUGIN_ID/$SOME_LOCAL_ID.

The activators introduced in that issue expect to deal with full Project objects, but there is no way for a source to return a particular project object by name, except for us iterating over all the projects that the source returns, which could have serious performance implications.

Proposed resolution

Make EnabledSourceHandler the "source of truth" about what projects are available, and which ones match particular queries. It should use a non-volatile key-value storage to store project information and be able to return it without querying the source plugins again.

✨ Feature request
Status

Fixed

Version

2.0

Component

Code

Created by

πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @phenaproxima
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    Since this could have a major performance impact, I'm bumping it.

  • Pipeline finished with Failed
    15 days ago
    Total: 536s
    #198263
  • Pipeline finished with Failed
    15 days ago
    Total: 239s
    #198265
  • Pipeline finished with Failed
    15 days ago
    Total: 224s
    #198267
  • Pipeline finished with Failed
    15 days ago
    Total: 218s
    #198270
  • Pipeline finished with Canceled
    15 days ago
    Total: 159s
    #198272
  • Pipeline finished with Failed
    15 days ago
    #198274
  • Pipeline finished with Failed
    15 days ago
    Total: 301s
    #198278
  • Pipeline finished with Failed
    15 days ago
    Total: 280s
    #198285
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts
  • Pipeline finished with Failed
    15 days ago
    Total: 328s
    #198289
  • Pipeline finished with Failed
    14 days ago
    Total: 549s
    #198294
  • Pipeline finished with Failed
    14 days ago
    Total: 354s
    #198978
  • Pipeline finished with Failed
    14 days ago
    Total: 554s
    #198982
  • Pipeline finished with Canceled
    14 days ago
    Total: 110s
    #199032
  • Pipeline finished with Failed
    14 days ago
    Total: 620s
    #199033
  • Pipeline finished with Failed
    14 days ago
    Total: 515s
    #199081
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts
  • Pipeline finished with Failed
    14 days ago
    Total: 613s
    #199091
  • Pipeline finished with Failed
    14 days ago
    Total: 519s
    #199112
  • Pipeline finished with Failed
    14 days ago
    Total: 520s
    #199188
  • Pipeline finished with Failed
    14 days ago
    Total: 1807s
    #199222
  • Pipeline finished with Failed
    13 days ago
    Total: 529s
    #199255
  • Pipeline finished with Failed
    13 days ago
    Total: 519s
    #199273
  • Pipeline finished with Failed
    13 days ago
    Total: 176s
    #199277
  • Pipeline finished with Failed
    13 days ago
    Total: 593s
    #199286
  • Pipeline finished with Canceled
    13 days ago
    Total: 7s
    #199465
  • Pipeline finished with Failed
    13 days ago
    Total: 386s
    #199466
  • Pipeline finished with Failed
    13 days ago
    Total: 200s
    #199469
  • Pipeline finished with Canceled
    13 days ago
    Total: 139s
    #199471
  • Pipeline finished with Failed
    13 days ago
    Total: 366s
    #199475
  • Pipeline finished with Failed
    13 days ago
    Total: 448s
    #199484
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts
  • Pipeline finished with Failed
    11 days ago
    Total: 494s
    #200923
  • Pipeline finished with Success
    11 days ago
    Total: 548s
    #200933
  • Pipeline finished with Canceled
    11 days ago
    Total: 236s
    #200951
  • Status changed to Needs review 11 days ago
  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts
  • Pipeline finished with Canceled
    11 days ago
    Total: 91s
    #200953
  • Pipeline finished with Canceled
    11 days ago
    Total: 167s
    #200954
  • Pipeline finished with Success
    11 days ago
    Total: 409s
    #200956
  • Pipeline finished with Success
    11 days ago
    Total: 522s
    #200977
  • Pipeline finished with Success
    11 days ago
    Total: 3744s
    #201012
  • Pipeline finished with Failed
    11 days ago
    Total: 536s
    #201057
  • Pipeline finished with Canceled
    11 days ago
    Total: 175s
    #201069
  • Pipeline finished with Success
    11 days ago
    Total: 398s
    #201072
  • πŸ‡¦πŸ‡ΊAustralia sime Canberra

    If I'm reading the code correctly, every unique $SOURCE_PLUGIN_ID/$SOME_LOCAL_ID gets a separate key set? If this numbered into the thousands or tens of thousands I don't this kv is the best place, especially being the place I most associate with Redis, and these IDs are not values I assocaited with require super fast runtime read/write. Would it be better to use a cache bin?

  • πŸ‡ΊπŸ‡ΈUnited States phenaproxima Massachusetts

    @sime, the problem with a cache bin is that it can be wiped out at any time, and since this data has to persist across multiple requests, we need to guarantee that it can't just disappear because some other user did a cache clear while you were trying to install something. So I needed something "stickier", thus the key-value store.

  • πŸ‡¦πŸ‡ΊAustralia sime Canberra

    And it would be so easy to swap KV with something else if needed later.

    Everything else makes sense and it's worth a maintainer having a look - the key thing I see is changes to how the projects are uniquely identified and then all the services are adjusted to support that.

  • Pipeline finished with Failed
    10 days ago
    Total: 335s
    #202176
  • Pipeline finished with Failed
    10 days ago
    Total: 422s
    #202177
  • Pipeline finished with Failed
    10 days ago
    Total: 226s
    #202182
  • Pipeline finished with Success
    10 days ago
    Total: 661s
    #202189
  • Pipeline finished with Failed
    8 days ago
    Total: 471s
    #203809
  • Pipeline finished with Success
    8 days ago
    Total: 731s
    #203828
  • πŸ‡ΊπŸ‡ΈUnited States tim.plunkett Philadelphia
  • πŸ‡ΊπŸ‡ΈUnited States chrisfromredfin Portland, Maine
  • πŸ‡ΊπŸ‡ΈUnited States tim.plunkett Philadelphia
  • Status changed to RTBC 8 days ago
  • πŸ‡ΊπŸ‡ΈUnited States tim.plunkett Philadelphia
  • Pipeline finished with Skipped
    8 days ago
    #203904
  • Status changed to Fixed 8 days ago
  • πŸ‡ΊπŸ‡ΈUnited States tim.plunkett Philadelphia

    Merged !511 πŸŽ‰

Production build 0.69.0 2024