By default deprecate non-experimental modules that are used by less 5% of sites before the next major version

Created on 12 July 2020, about 4 years ago
Updated 13 February 2023, over 1 year ago

Problem/Motivation

  1. Drupal core has 80 modules.
  2. 10(more ?) modules were added to core in the Drupal 8 cycle.
  3. 1(? more) module, place_block, was removed from core in the Drupal 8 cycle
  4. MAINTAINERS.txt currently has 39 "?" denoting a subsystem or module that has no maintainer.
  5. There are 4 modules, aggregator, tracker, book, forum that were only used by 5% or fewer Drupal sites

We have a lack of maintainers and we have a few modules that are used by very few sites.

Even though these modules don't change much they do require some maintenance and they must be kept up with core deprecations. For instance we had to move all the tests from SimpleTest to PHPUnit. When we update versions of phpunit will have to update all of the tests in these modules.

There are many contrib modules that have 20x the usage of these core modules.

Proposed resolution

We should create a policy to deprecate and remove any module that is used in less than 5% of Drupal sites with each new version of Drupal. This would be the default action if no issue is made to not remove a module. This would move the burden from those that would like to advocate for a module to be deprecated to those that would like to advocate for a module to not be deprecated.

We will have to determine which modules should be exempt from the policy but possible criteria would be:

  1. Only remove modules that were not introduced in the previous major version of Drupal. Therefore for the Drupal 9 recycle we would not by default remove any module that was introduced in the Drupal 9 release cycle. This would give modules more time to gain traction
  2. Do not remove modules that are not recommended for production use or modules that are only needed temporally. These modules would not have usage reflect in the numbers. For instance, migrate_drupal_ui would only need to be installed for brief period of time.
  3. Do not remove experimental modules with this policy. These modules may be removed by the Experimental Modules policy โ†’

This policy would not prevent removing other modules from core that do not fall below the 5% usage level. Any module could still be removed from core. The only difference would be that for modules under the threshold it would be the default and an issue would have to be approved to keep them in. For modules above the threshold an issue would have to be approved to remove them(which is the current case).

Remaining tasks

  1. Create a process for getting core sub module stats
  2. Determine when the stats should be run to determine which modules should be removed. When a new major version is released?
  3. Determine what % usage we should use to determine if a module is removed. 5% was just a starting point for discussion.
  4. Determine what the process should be for creating an issue to keep a module even if it has lower usage. For example should you be able to advocate for module to stay in core even though it has less than 5% usage if it has no maintainer?

.

๐ŸŒฑ Plan
Status

Active

Component

Idea

Created by

๐Ÿ‡บ๐Ÿ‡ธUnited States tedbow Ithaca, NY, USA

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • ๐Ÿ‡ซ๐Ÿ‡ฎFinland lauriii Finland

    I wonder if there are some additional caveats to the data that we are not aware of yet. I don't know if I'm alone with this but it seems strange that only 93.4% have Field enabled, and that only 91.0% have Block enabled. But then Menu UI is enabled by 88.0% of the sites ๐Ÿค”.

    Even more glaring to me is that Claro + Seven have total of 17.4% usage which seems really low. The two of the most popular contrib admin themes have total of around 74500 usages reported which is around 17%. They both depend on either Seven or Claro. Do we think it's possible that there are only 0.4% of sites using Seven or Claro as their admin theme?

  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom catch

    @lauriii at one point tests with update status enabled were reporting back to Drupal.org, if this is still the case then that'd be a lot of sites every month with 'stark' enabled.

  • ๐Ÿ‡ง๐Ÿ‡ชBelgium wim leers Ghent ๐Ÿ‡ง๐Ÿ‡ช๐Ÿ‡ช๐Ÿ‡บ

    If that's true, then shouldn't we be ignoring all update status checks to d.o for sites with stark enabled? ๐Ÿ˜‡ I think only an incredibly small number of Drupal sites will eve have gone into production on stark ๐Ÿค“

  • ๐Ÿ‡ซ๐Ÿ‡ฎFinland lauriii Finland

    Looks like Stark usage is between 0.5% and 1.5%. Based on that, I'm not sure that #31 helps explain #30.

  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom catch

    @lauriii that's a good point and also reassuring :)

    The table in #23 is from March 2022, I think we could expect usage of Claro to have gone up significantly since then since it was marked stable and added to the standard profile in April 2022 (and then 9.4 was released in June). For me, that alone explains low Claro usage - too new to appear in the stats. Even with updated stats, it will probably still take some time to pick up - i.e. for existing sites to switch to Seven to Claro as part of a Drupal 10 upgrade, still a few months of that to go.

    This doesn't explain the combined low usage though. Seven was on 44.5% in Drupal 8 according to #23 which seems more what I'd expect although still quite low.

  • ๐Ÿ‡ซ๐Ÿ‡ฎFinland lauriii Finland

    Something else that seems off about the data is that between Drupal 8 and 9 Stable usage went from 45.0% to 5.5%, and Classy usage went from: 45.0% to 3.9%. Did this many people really remove their dependency on Classy? If we look at the contrib Classy usage and compare it to Drupal 10.0.x sites, around 13.3% of Drupal 10 sites have Classy installed.

  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom 2dareis2do

    What is the latest data on this?

  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom catch

    @2dareis2do the data is only available via querying the d.o database, we'd need to ask the DA for updated stats.

    The current list of modules that are actively being prepared for a move to contrib is in ๐ŸŒฑ Deprecate dependencies, libraries, modules, and themes that will be removed from Drupal 11 core Active .

    Also are there any plans to do the opposite? i.e. move contrib modules into core that are installed on more than 5% of sites

    There are individual issues like โœจ Pathauto in Core Needs work although e.g. pathauto is installed on more like 75% of sites. The main focus on making it easier to find and install modules is on project browser.

  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom 2dareis2do

    Thanks @catch

    Can I create an issue to make this data publicly available?

    Current data is nearly 2 years old. Drupal 10 has been released since then.

    Feedback loop is too long.

  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom catch

    @2dareis2do there is probably already a public issue, I would look in the 'drupalorg' issue queue first, and open an issue there if there's not one already.

  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom AaronMcHale Edinburgh, Scotland

    Also are there any plans to do the opposite? i.e. move contrib modules into core that are installed on more than 5% of sites

    With Project Browser and Recipes soon coming into Core, we might find that we actually don't need to bring more module into core, because the barrier to entry for getting those modules becomes so much lower.

    What I mean is that I can imagine a day where, when you setup Drupal, you don't install the standard profile, you pick from a curated set of recipes right in the installer, the recipe takes care of getting all of the modules it needs. Some of those recipes might not even be included with Core, they might life in contrib and the recipe you choose might then be downloaded during the installation. Maybe then core can become even smaller, and thus even more maintainable.

    Now, I'm not saying this is definitely where we should go, but what I am saying is that I can see this being a place that we are gradually moving towards.

  • ๐Ÿ‡ฌ๐Ÿ‡งUnited Kingdom 2dareis2do

    Project Browser reminds me a little of Drupal Gardens.

    Our goal is to make it easier to find and install modules for people that are (1) new to Drupal and that are (2) site builders*

    Moving popular contributed modules into core also acheives this aim.

  • ๐Ÿ‡บ๐Ÿ‡ธUnited States drumm NY, US

    Here is updated data.

    To clean the data, this is how I got the core components:

    SET group_concat_max_len = 10000000000; SELECT group_concat(DISTINCT pcc.name SEPARATOR '\\|') FROM project_composer_component pcc INNER JOIN field_data_field_release_project fdf_rp ON fdf_rp.entity_id = pcc.release_nid AND fdf_rp.field_release_project_target_id = 3060\G

  • ๐Ÿ‡ซ๐Ÿ‡ทFrance nod_ Lille

    tried to reformat it, double checking is needed.

    
    module	
    demo_umami_tour	0,00 %
    profile	0,00 %
    blog	0,00 %
    outside_in	0,00 %
    simpletest	0,01 %
    starterkit_theme	0,04 %
    block_place	0,05 %
    migrate_drupal_multilingual	0,06 %
    demo_umami_content	0,09 %
    umami	0,09 %
    demo_umami	0,09 %
    minimal	0,28 %
    help_topics	0,48 %
    stark	0,50 %
    navigation_top_bar	0,53 %
    workspaces	0,55 %
    navigation	0,81 %
    entity_reference	0,86 %
    aggregator	1,59 %
    sdc	1,65 %
    stable9	1,72 %
    tracker	1,85 %
    pgsql	1,92 %
    sqlite	2,63 %
    hal	2,95 %
    forum	3,03 %
    field_layout	3,64 %
    book	4,32 %
    settings_tray	4,57 %
    layout_builder_expose_all_field_blocks	4,79 %
    migrate_drupal_ui	5,32 %
    olivero	6,50 %
    classy	6,56 %
    bartik	6,93 %
    stable	7,10 %
    statistics	7,59 %
    announcements_feed	8,18 %
    claro	8,25 %
    seven	8,80 %
    action	9,95 %
    basic_auth	11,53 %
    migrate_drupal	13,36 %
    ban	13,43 %
    jsonapi	15,44 %
    standard	16,11 %
    syslog	17,12 %
    inline_form_errors	18,03 %
    quickedit	19,76 %
    color	22,31 %
    migrate	24,08 %
    layout_builder	25,32 %
    ckeditor	25,33 %
    content_moderation	25,42 %
    workflows	27,48 %
    datetime_range	27,55 %
    rdf	27,92 %
    config_translation	31,83 %
    content_translation	32,96 %
    rest	35,14 %
    telephone	35,54 %
    layout_discovery	40,51 %
    responsive_image	42,57 %
    phpass	44,01 %
    serialization	46,64 %
    media_library	47,43 %
    locale	52,71 %
    tour	57,91 %
    ckeditor5	60,10 %
    search	63,53 %
    language	63,60 %
    media	64,84 %
    big_pipe	66,08 %
    comment	67,34 %
    contact	67,53 %
    history	79,99 %
    mysql	80,24 %
    help	84,72 %
    shortcut	85,49 %
    automated_cron	86,34 %
    dblog	89,05 %
    page_cache	93,23 %
    views_ui	94,05 %
    contextual	94,29 %
    block_content	95,30 %
    dynamic_page_cache	95,83 %
    field_ui	96,26 %
    config	97,32 %
    datetime	97,56 %
    taxonomy	97,74 %
    options	98,28 %
    menu_ui	98,50 %
    path	98,52 %
    editor	98,56 %
    menu_link_content	98,59 %
    toolbar	98,72 %
    link	98,79 %
    image	98,83 %
    views	98,98 %
    breakpoint	99,02 %
    file	99,14 %
    node	99,32 %
    text	99,55 %
    filter	99,56 %
    block	99,57 %
    field	99,58 %
    path_alias	99,93 %
    update	99,98 %
    user	99,99 %
    system	100,00 %
    Total Rรฉsultat	4715,56 %
    
    
Production build 0.71.5 2024