Project usage totals should exclude Drupal 7 branches

Created on 10 January 2025, 3 months ago

Problem/Motivation

Slack discussion here: https://drupal.slack.com/archives/C01UHB4QG12/p1736456868218749

https://www.drupal.org/jsonapi/index/project_modules?filter%5Bmachine_na... β†’

and also project home pages ( https://www.drupal.org/project/webform β†’ ) show total project usage.

This currently includes Drupal 7 (and 6 and 5 in some cases) branches.

In some cases (pathauto) Drupal 7 and 8+ usage are broadly proportionate, but this is not the case for modules like e.g. field_collection https://www.drupal.org/project/usage/field_collection β†’

Especially for project browser which orders by usage, it could make quite a big difference to what gets listed first, but it's also useful for manual browsers to the project page to be able to see what the usage is on supported core branches without EOL ones mixed in.

Steps to reproduce

Proposed resolution

Where total usage is calculated, only count branches that are compatible with Drupal 8+

For me it is nice to be able to see EOL branches on the project/usage pages so please keep this until it's no longer possible to tell that there are ~170 Drupal 5 sites still phoning home to d.o every week.

Remaining tasks

User interface changes

API changes

Data model changes

πŸ“Œ Task
Status

Active

Version

1.0

Component

Code

Created by

πŸ‡¬πŸ‡§United Kingdom catch

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @catch
  • πŸ‡ͺπŸ‡ΈSpain fjgarlin

    Checking compatibility of every single branch (branch name is not enough to determine compatibility with D8+, we’d need to check against updates.drupal.org XML and then only add some values.

    Question about this: what about D8, D9 branches? These are technically EOL as well.

  • πŸ‡¬πŸ‡§United Kingdom catch

    Hmm i would have thought branch name was enough.

    A 7.x compatible release has to start with the prefix 7.x-, e.g. webform 7.x-4.x

    An 8+ compatible branch has to start with either the prefix 8.x-, or be semver.

    Question about this: what about D8, D9 branches? These are technically EOL as well.

    This is impossible to tell from the branch - some modules, e.g. https://www.drupal.org/project/redirect β†’ have stayed on 8.x-1.x since 8.0.0, and progressively added and dropped Drupal major version compatibility over the years, and we actively encourage that when the only change is major version compatibility. So it would be really hard to filter out 8.x and 9.x usage, and also don't think that matters as much.

    There are some modules that exist in contrib in 8.x, but were absorbed into core in 9.x or 10.x (or otherwise became obsolete) and it would matter for them, but this is a tiny percentage of overall modules compared to 7.x vs. 8+. Also if we really wanted to filter out EOL core branches we'd have to do something like exclude 10.2 and (in six months) 11.0, but not 10.4 and 11.1.

    @fjgarlin a module can theoretically be compatible with 8.0-11.x, while that's unlikely at this point, 9.5-11.1 is not.

  • πŸ‡ͺπŸ‡ΈSpain fjgarlin

    Cool. As long as there is agreement on the logic then it should be fine.

    We can exclude 7.x and 6.x prefixes and add up all the others. If this logic sounds good then it should be easy.

  • πŸ‡©πŸ‡°Denmark ressa Copenhagen

    Perhaps it's possible to copy-paste the Slack discussion here, since not everyone is on Slack, and also the discussion might not be available at some point ... Thanks!

    If you're a free user in the current setup, Slack shows up to 90 days of messages and files you upload. You need a premium subscription to access the older history, which isn't deleted but kept hidden until you upgrade.

    https://www.neowin.net/news/slack-takes-away-free-lifetime-message-histo...

  • πŸ‡ͺπŸ‡ΈSpain fjgarlin

    The description of the issue is already bigger than the conversation itself.

    It was something like:

    @catch: Does anyone know off the top of their head - when project browser sorts by usage, is that Drupal 8/10+ usage or does it also include Drupal 7 usage?
    @fjgarlin: ...quick investigation... yes
    @catch: here is an issue https://www.drupal.org/project/drupalorg/issues/3498849
    
  • πŸ‡©πŸ‡°Denmark ressa Copenhagen

    Thanks for sharing, I couldn't tell if it was a big debate, or a short one.

  • πŸ‡ΊπŸ‡ΈUnited States drumm NY, US

    The best way to check is if the release has field_release_category === 'current'

    field_release_category for releases compatible with 6 or earlier is obsolete. Compatible with 7 is legacy

    This avoids pitfalls like core versions not having prefixes, and just avoiding parsing in general.

  • Merge request !319Only calculate usage on newer releases. β†’ (Merged) created by fjgarlin
  • πŸ‡ͺπŸ‡ΈSpain fjgarlin

    @drumm - I think that the changes in the MR should be enough. We're only setting entries in the array for releases that are "current", and the values from the returned array are what's used to calculate the totals.

  • πŸ‡ΊπŸ‡ΈUnited States drumm NY, US

    Looks good to me

  • Pipeline finished with Skipped
    2 months ago
    #402490
    • fjgarlin β†’ committed 33012e39 on 1.0.x
      Issue #3498849 by fjgarlin, catch, drumm: Project usage totals should...
  • πŸ‡ͺπŸ‡ΈSpain fjgarlin

    Merged. It will be deployed this week to the site and it might start reflecting in Project Browser in a week or two.

  • Automatically closed - issue fixed for 2 weeks with no activity.

  • πŸ‡ͺπŸ‡ΈSpain fjgarlin

    This should have already worked but the values are not updated, at least for the project linked in the issue description.

    I added some extra debug so I can check logs to make sure that everything is running as expected: https://git.drupalcode.org/project/drupalorg/-/commit/7dcb07197b2268a795...

    I'll continue monitoring it.

  • πŸ‡ͺπŸ‡ΈSpain fjgarlin

    Finally done. https://www.drupal.org/jsonapi/index/project_modules?filter%5Bmachine_na... β†’ it went from +360K to +220K for field_active_installs_total.

  • πŸ‡©πŸ‡°Denmark ressa Copenhagen

    Thanks @fjgarlin! The field_active_installs_total value is now based on Drupal installations above Drupal 7.

    "field_active_installs": "{\u00228.x-4.x\u0022:3,\u00228.x-5.x\u0022:20138,\u00226.x\u0022:244,\u00226.0.x\u0022:5864,\u00226.1.x\u0022:34571,\u00226.2.x\u0022:159276,\u00226.3.x\u0022:5458}",
    "field_active_installs_total": 225554,

    It looks like the current value (see for example 159276 under 6.2.x / 159,276) is from January 26:

    Week		6.0.x	6.1.x	6.2.x	6.3.x	Total
    February 2, 2025	5,847	33,843	155,775	5,954	365,040
    January 26, 2025	5,864	34,571	159,276	5,458	369,443
    [...]
    

    From https://www.drupal.org/project/usage/webform β†’

    I wonder if, for precision, a new field for the basis of the field_active_installs_total value could be added, something like this?

    "field_active_installs_total": 225554,
    "field_active_installs_total_date": "2025-01-26",
    

    Also, would it be possible to get older data points? Or is there only a single field_active_installs_total value available for each project?

  • πŸ‡ͺπŸ‡ΈSpain fjgarlin

    There is a single field_active_installs_total field, with just one value, usage can be tracked in https://www.drupal.org/project/usage/webform β†’ .

    The cron job runs weekly and grabs data from the previous week parsed. Right now we can see in https://www.drupal.org/project/usage/webform β†’ how the last week parsed was February 2nd, so we're grabbing the data from the week prior to that, as the data needs to be be synced from D7 to D10. This is a temporary situation while usage data is being managed by D7 drupal.org

Production build 0.71.5 2024