Migrating "Categories" on Drupal.org Project pages

What the difference between "delete" and "delete after migrate"?

Also, as per @drumm via slack

Please be sure to move the issue, not make a new one. And in a format we can copy/paste/manipulate from

Are there descriptions for the categories that are being kept?

I updated the issue summary with stubs of the ideal format. For implementing this, it will be 3 phases, delete old categories, update categories being kept, merge categories that will be merged. It would be good to have the lists in that format, including the category descriptions.

I have updated the summary with nearly all of the information, but getting consensus from the group whether or not to keep decoupled-type-things in import/export or make it a separate term; and if separate, we still need to get a definition/description for that one, and update the import/export one.

The consensus for descriptions for Decoupled and "Import and export" is as follows:

Decoupled - Support the idea of separating front-end and backend concerns by integrating Drupal to external or third-party frameworks for display

Import and Export - Help transfer content and data into or out of the site, by migration, backup, or exposing data to external systems.

@drumm. I'll update the lists to be in the format you recommended above, with the descriptions included

Updated descriptions for Decoupled and Import and Export in the Issue Summary.

Are all changes documented and ready here?

Good question; but now looking and seeing what we did with Decoupled, I can confirm that YES, we're ready on this ticket. :)

Attached is a dry run of the removals, which will serve as a backup. Script used is:

$terms_to_remove = [
  56 => 'Community',
  88 => 'Content Construction Kit (CCK)',
  4654 => 'Drush',
  19440 => 'Examples',
  11478 => 'Features Package',
  7404 => 'Mobile',
  16190 => 'Novelty',
  90 => 'Organic Groups (OG)',
  51425 => 'Other',
  19984 => 'Project management',
  116 => 'RDF',
  73 => 'Theme Enhancements',
  89 => 'Views',
];
$tids_to_remove = array_keys($terms_to_remove);
$result = (new EntityFieldQuery())->entityCondition('entity_type',  'node')
->propertyCondition('type', 'project_module')
->fieldCondition('taxonomy_vocabulary_3', 'tid', $tids_to_remove)
->execute();
foreach (array_chunk(array_keys($result['node']), 50) as $nids) {
  foreach (node_load_multiple($nids) as $node) {
    print $node->nid . ' ' . $node->title . ': removing';
    $removed = [];
    foreach ($node->taxonomy_vocabulary_3[LANGUAGE_NONE] as $delta => $item) {
      if (in_array($item['tid'], $tids_to_remove)) {
        print ' ' . $item['tid'] . ':' . $terms_to_remove[$item['tid']];
        $removed[] = $terms_to_remove[$item['tid']];
        unset($node->taxonomy_vocabulary_3[LANGUAGE_NONE][$delta]);
      }
    }
    print PHP_EOL;
    $node->revision = TRUE;
    $node->log = t('Removing module categories, @removed, which are being discontinued, see https://www.drupal.org/project/drupalorg/issues/3383004', [
      '@removed' => implode(', ', $removed)
    ]);
    node_save($node);
  }
}

Content - Rename to "Content Editor Tools"

But the description is

Content Editing Experience - Enhance the editorial interface and improve the processes and workflows around creating, editing or removing content.

Please update the issue summary to correct one of those

Moving file management into merges, since there is already another media category.

The description for Education is missing

Updated summary:

1. Good catch. "Content Editor Tools" should be "Content Editor Experience" wherever that appears; have updated.
2. Education - mis-transcribed from the spreadsheet. That should just be a drop. Updated summary.

Question about the ecosystem filter plan from the issue summary:

Is that something that is going to be rolled out imminently, or do we just lose functionality for some unknown (possibly VERY lengthy) period of time?

Specifically, the "Rules" category used to be only for the Rules ecosystem. This is now renamed to "Automation" which means that a lot of modules and submodules unrelated to the Rules module are now lumped into the same category.

Automation now has 426 (!!) modules. Most of which are not part of the Rules ecosystem.

With no "ecosystem" filter, extensions to the Rules module now become a lot harder to find and you have to wade through hundreds of modules to find them.

This is a major step backwards. At least from the Rules standpoint. I feel like I need to now do all this searching myself then go and modify the Rules homepage to list all the compatible modules, like we used to have to do back in the Drupal 5 days, in order to help users find ecosystem modules.

Education is now being removed, the log is attached.

The "ecosystem" functionality is already on Drupal.org. On https://www.drupal.org/project/rules → , click "Projects that extend this" in the right sidebar. Each project can be updated to say it extends Rules. ("Ecosystem" is a bit of an abstract term, we should try to keep it out of UIs and elsewhere.)

We might want to pause and identify categories that should be migrated to ecosystems as well. We could update every project in the Automation, formerly Rules, category to say they extend rules.

Automation now has 426 (!!) modules. Most of which are not part of the Rules ecosystem.

To be clear, no merges have happened yet, so those 426 modules are the ones that were in the Rules category. We probably should not migrate them to the ecosystem field, since most of them are not related to Rules.

I didn’t see a category with good data that might be a candidate for migrating to ecosystems. Like the Rules/Automation example, categories use is just not restricted enough to match up to a single module.

The merges are now running, log of a dry run is attached. Using code:

$terms_to_merge = [
  74 => 13434,
  76 => 13434,
  8818 => 53,
  68 => 58,
  63 => 57,
  13158 => 59,
  101 => 59,
  26738 => 59,
  75 => 59,
  55 => 104,
  196950 => 64,
  70 => 64,
  66 => 52,
  119 => 52,
  7266 => 69,
  61 => 20224,
  65 => 20224,
  124 => 20224,
  71 => 20224,
  122 => 60,
  62 => 67,
];
$terms = taxonomy_term_load_multiple(array_merge(array_keys($terms_to_merge), $terms_to_merge));
foreach ($terms_to_merge as $key => $value) {
  print $terms[$key]->name . ' → ' . $terms[$value]->name . PHP_EOL;
}
$tids_to_merge = array_keys($terms_to_merge);
$result = (new EntityFieldQuery())->entityCondition('entity_type',  'node')
->propertyCondition('type', 'project_module')
->fieldCondition('taxonomy_vocabulary_3', 'tid', $tids_to_merge)
->execute();
foreach (array_chunk(array_keys($result['node']), 50) as $nids) {
  foreach (node_load_multiple($nids) as $node) {
    print $node->nid . ' ' . $node->title . ': merging';
    $merged = [];
    $existing_terms = array_column($node->taxonomy_vocabulary_3[LANGUAGE_NONE], 'tid');
    foreach ($node->taxonomy_vocabulary_3[LANGUAGE_NONE] as $delta => $item) {
      if (in_array($item['tid'], $tids_to_merge)) {
        print ' ' . $item['tid'] . ':' . $terms[$item['tid']]->name . '→' . $terms[$terms_to_merge[$item['tid']]]->name . '-';
        if (in_array($terms_to_merge[$item['tid']], $existing_terms)) {
          print 'exists';
          unset($node->taxonomy_vocabulary_3[LANGUAGE_NONE][$delta]);
        }
        else {
          print 'update';
          $node->taxonomy_vocabulary_3[LANGUAGE_NONE][$delta]['tid'] = $terms_to_merge[$item['tid']];
          $existing_terms = array_column($node->taxonomy_vocabulary_3[LANGUAGE_NONE], 'tid');
        }
        $merged[] = $terms[$item['tid']]->name . '→' . $terms[$terms_to_merge[$item['tid']]]->name;
      }
    }
    print PHP_EOL;
    $node->revision = TRUE;
    $node->log = t('Updating module categories, @merged, see https://www.drupal.org/project/drupalorg/issues/3383004', [
      '@merged' => implode(', ', $merged)
    ]);
    node_save($node);
  }
}

This is now all done. Search indexing is still catching up, I'll continue to monitor it and motivate it if needed tomorrow.

Thanks everyone!

The "ecosystem" functionality is already on Drupal.org. On https://www.drupal.org/project/rules → , click "Projects that extend this" in the right sidebar. Each project can be updated to say it extends Rules. ("Ecosystem" is a bit of an abstract term, we should try to keep it out of UIs and elsewhere.)

I have to disagree - that is fundamentally different. (BTW, these arguments apply to any ecosystem, not just Rules.)

"Projects that extend this" has been around for a while and is not very useful because it is an opt-in categorization made and controlled by other modules. That doesn't help when it comes to building ecosystems or finding modules that are part of an ecosystem.

In the case of Rules, Rules can't simply designate other modules that depend on Rules, and drupal.org does not automatically check dependencies to figure out which modules depend on Rules.

Regardless, with the Plugin system, modules that support Rules DO NOT have a dependency on Rules in order to provide Rules functionality. The same is true of Drush commands and a large part of Drupal 8+ functionality.

The problem remains: There is NO LONGER a way to find modules that work with Rules. And there's a similar problem (to a greater or lesser extent) with every other ecosystem.

I have nothing against the new categories, but I do have a problem with throwing away functionality that we've depended on for at least 10 years.

Hi TR, since we're looking at ecosystem as a separate thing, we'd welcome your constructive feedback on that related issue, so we can let this one close.

We'll have an opportunity to look at how to truly implement/re-implement ecosystem in an even better way!

https://www.drupal.org/project/project_browser/issues/3381187 🌱 [META] Plan/Proposal to Implement Ecosystem Active

(And, related more specifically to PB) -
https://www.drupal.org/project/project_browser/issues/3241544 ✨ Add filter by project dependencies (ecosystem) Active
https://www.drupal.org/project/project_browser/issues/3365593 ✨ Define how to implement Ecosystem for Project Browser Active

Both categories and ecosystem are updated by the downstream modules by editing their project page. The only difference is ecosystem is an autocomplete for the project, like rules, vs selecting a category from a list.

Before re-arranging categories, the former rules category did have 426 modules, many not related to or integrating with Rules module. The ecosystem selection has the potential to be more accurate, since maintainers aren’t looking at a list of categories, often not knowing what “rules” might mean as a category.

I'm just using "Rules" as an example - the problem is true of any of the ecosystems that contain more than just a few related modules. Commerce, for example.

Regardless, the "Automation" category still has 406 modules as of today - not an improvement - so the 20 or so modules that are *actually* related to Rules are totally lost in the noise.

@chrisfromredfin: Frankly, I *already* spent years dealing with this issue back in the D5/D6 days. I thought it was fixed. Spending another year or more hoping to get some new feature that approximates what was just lost doesn't appeal to me. I'll just go back to the way we used to have to do things - manually compile a list of ecosystem modules and post it on the project page, then try to keep it up to date.

Automatically closed - issue fixed for 2 weeks with no activity.