Fix spellcheck:make-drupal-dict

@quietone the --wordsOnly option was added in #2972224: Add .cspell.json to automate spellchecking in Drupal core → - i.e. the patch that introduced cspell to core. The command certainly worked then. This was changed in https://github.com/streetsidesoftware/cspell/commit/aac621f46b6f1f60185e... - and https://www.drupal.org/project/drupal/issues/3343216 🌱 [meta] Update JS dependencies for Drupal 10.1 Closed: duplicate in introducing the breaking change so it should be done there.

If I do yarn run spellcheck:make-drupal-dict on HEAD it works just fine.

Hmmm... #4 is a bit wrong. Cspell tries to tell us that we should be using --words-only instead of --wordsOnly - kinda like a deprecation but this info ends up in the dictionary! Oops. So yeah there is something to do here!

The problem is happening because we're piping all output from yarn -s spellcheck:core --unique --words-only into the dictionary, including the error output :)

Reviewing the patch...

+++ b/core/misc/cspell/dictionary.txt
@@ -684,7 +681,7 @@ metapackage
-meφω
+meΦΩ

@@ -1480,7 +1475,8 @@ zwei
+Èxample
+Über
 åwesome
-èxample
 über
-ȅchȏ
+Ȅchȏ

These are changes are incorrect. It looks like the | tr '[:upper:]' '[:lower:]' is not working for @quietone for some reason. It does work for me.

It looks like locale is affecting the tr command too. I think we can fix that by standardising on en_US.utf8 like so...

The patch attached addresses #6

@alexpott, thanks for finding the cspell issue about wordsOnly.

I tested the patch in #7 and it did not fix the problem. Then I did some reading and searching.

The online 'tr' man page has this text, which is missing from man tr on my Debian host.

Full support is available only for safe single-byte locales, in
which every possible input byte represents a single character.
The C locale is safe in GNU systems, so you can avoid this issue
in the shell by running LC_ALL=C tr instead of plain tr.

Then I found How to make tr aware of non-ascii(unicode) characters?. That has links to the gnu tr bug reports about this, which are unfortunately have a severity of 'wishlist'. However, the page does provide alternative commands. Of those, I implemented the perl one and it does work. I don't know perl, I can only say that the command works.

📌 Fix spelling for words used once, beginning with 'n' -> 'p', inclusive Fixed was just committed so I expected the dictionary results would be different that in #7. What I did not expect is that "overrider's" is added to the list. Looking at that issue I see that the dictionary was not remade with the patch so this mistake wasn't noticed. I will make an issue for fixing that problem.

Instead of creating an issue, I reverted 📌 Fix spelling for words used once, beginning with 'n' -> 'p', inclusive Fixed

Update patch post the revert.

Applied patch #10
cd into core
ran spellcheck:make-drupal-dict without issue
ran spellcheck:core without issue
Also didn't cause any additional changes to the dictionary file.

@smustgrave, what results do you get when running on HEAD. Do you get the changes to the non-English words listed in #6?

Moving to review to remind myself to run the command .

When I run the command from #6 on 10.1 I don't get any changes.

Hmm, that is not what I meant. I meant do you get the results in #6 when running yarn:spellcheck:drupal-make-dict from HEAD.

It would be good to get confirmation of the results I get.

I ran the command locally and it results in the same dictionary therefore let's commit this and get the dictionary fixed. The new perl command for lowering the case works for me.

Committed 4858c4e and pushed to 10.1.x. Thanks!
Committed 5235770 and pushed to 10.0.x. Thanks!

9.5.x spellcheck does not have the --words-only vs --wordsOnly problem because it's not been updated. Therefore I left it alone. However 9.5.x does have incorrectly capitalised words in the dictionary so I do think that we should backport that part of this change. So leaving open against 9.5.x

Right, here is the 9.5.x patch.

+++ b/core/misc/cspell/dictionary.txt
@@ -766,6 +765,7 @@ montag
+mopen

@@ -828,6 +828,7 @@ nids
+nikic

When I run this on a clean build I don't get either of these.

I'm running from inside core after having done a yarn install and a composer install.

I've tried to work out why @quietone's run has these words and I can't see why. Mopen is only in package.json and we ignore that... and nikic is in the root composer.lock and that should be ignored too.

Sorry about that. Yes, I wasn't working on a clean setup. So, I sorted that out and rebuild the dictionary.

The last submitted patch, 19: 3355301-19.9.5.x.patch, failed testing. View results →

Tested #21 same way I did in #11 with no issues.

Committed 5339d05 and pushed to 9.5.x. Thanks!

Automatically closed - issue fixed for 2 weeks with no activity.