Module Project Categories out of sync?

Jsonapi URL: https://www.drupal.org/jsonapi/index/project_modules?filter%5Bmachine_na... →

In D7, the paragraph starting with "When using batch mode, it is important to sort the view..." was added recently, but it's not propagated to D10.

As we are working of a live DB "replica", I'm going to check there first, to see if the D7 data is correct.

Query and results in database:

select * from field_data_taxonomy_vocabulary_3 where entity_id=980666;
+-------------+----------------+---------+-----------+-------------+----------+-------+---------------------------+
| entity_type | bundle         | deleted | entity_id | revision_id | language | delta | taxonomy_vocabulary_3_tid |
+-------------+----------------+---------+-----------+-------------+----------+-------+---------------------------+
| node        | project_module |       0 |    980666 |    13428179 | und      |     0 |                        53 |
| node        | project_module |       0 |    980666 |    13428179 | und      |     1 |                        57 |
| node        | project_module |       0 |    980666 |    13428179 | und      |     2 |                        58 |
| node        | project_module |       0 |    980666 |    13428179 | und      |     3 |                        64 |
| node        | project_module |       0 |    980666 |    13428179 | und      |     4 |                        59 |
+-------------+----------------+---------+-----------+-------------+----------+-------+---------------------------+
5 rows in set (0.004 sec)

Yet I see via the UI 3 elements.

I will try next running the migration with the --update flag.

drush migrate:import drupalorg_migrate_project_module --update
Seems to have done the trick.

I can see only 3 categories here https://www.drupal.org/jsonapi/index/project_modules?filter%5Bmachine_na... → and also the update description.

I will take this one as a one-off, but if we have the problem again, we'll need to do a deeper investigation on why the records on D10 weren't updated. If that were the case, please reopen this issue or create a new one.

There was another module that had the same (cookie_compliance_checker or something like that), but if you re-ran the WHOLE migration with --update then that should have also fixed that one.

We'll let you know if we see it again!

Hey Fran -

Looking today, another example is /admin/modules/browse/drupalorg_jsonapi/drupal-flexslider-flexslider - (Flex Slider module) - it seems to have 5 categories. Leslie has also spotted several others.

Do the ongoing nightly migrations use `--update` always?

Thanks for reporting this Chris. The ongoing migrations do not use the --update flag as Drupal should recognize a change and then update the record, but it's clearly not doing it, so I'm going to change it to always use the --update flag.

I updated the migration to run with the --update flag.
Marking RTBC.

We should see, within one hour, only 3 categories instead of 5 in here: https://www.drupal.org/jsonapi/index/project_modules?filter%5Bmachine_na... →

Umm... that didn't fix the issue, which made me look into the data.

MariaDB [drupal]> select entity_id, taxonomy_vocabulary_3_tid from field_data_taxonomy_vocabulary_3 where entity_id=1316346;
+-----------+---------------------------+
| entity_id | taxonomy_vocabulary_3_tid |
+-----------+---------------------------+
|   1316346 |                        57 |
|   1316346 |                        58 |
|   1316346 |                     20224 |
|   1316346 |                        59 |
|   1316346 |                        67 |
+-----------+---------------------------+
5 rows in set (0.002 sec)

That's 5 results, so the migration is actually working. I need to dig a little deeper on this.

That query was in the DB replica. We also ran it in the live server and the result is the same. 5 results.
I will revert the update flag for now.

So the error might be in "drupalorg" D7. There are 5 values in the database, but only 3 displayed in the edit form.
I'm moving the issue to that queue instead.

https://www.drupal.org/project/drupalorg/issues/3382690#comment-15441835 📌 Limit number of "Categories" on the Drupal.org Project page Fixed

We can see that there are still a number of modules that have more than 3 categories saved. We cannot make a script where we just truncate them because the script won't have context about the module.

In this case, @chrisfromredfin, the best way to proceed is to contact the maintainers and to ask them to review the categories and save the node, even if they see only 3.

The number of allowed choices was limited to 3, but that's only on the UI. If modules have more categories, these will be kept.

That was my fear, and why we had planned to hard-truncate categories to the first three during the migration. Sad that we have these lingering. We will probably get popular, and certainly Top 100, modules to update and fix. Might have more of a struggle with the long tail. But as we notice them, we can open issues with the maintainers then. Thanks!

That was my fear, and why we had planned to hard-truncate categories to the first three during the migration. Sad that we have these lingering. We will probably get popular, and certainly Top 100, modules to update and fix. Might have more of a struggle with the long tail. But as we notice them, we can open issues with the maintainers then. Thanks!