Broken auto creation of machine name field

Created on 15 March 2024, 9 months ago
Updated 5 September 2024, 4 months ago

Problem/Motivation

Automatic machine name creation fails.

Expected result

    Expected result of transformed text to machine name:
  1. Arbetsflöde -> arbetsflode
  2. Såga i träd -> saga_i_trad

Result returned of transformed text to machine name:

    Transformed text to machine name:
  1. Arbetsflöde -> arbetsfl Javascript error stop further functionality
  2. Såga i träd -> s Javascript error stop further functionality

Since I do not use other language(s)...

...beside Swedish on my keyboard I can only asume that same thing happens with other non-ASCII characters.

Steps to reproduce

  1. Create a entity, in my case it was a workflow. Name it something that contains swedish umlaut character (one of this: åäöÅÄÖ). I used the word "Flöde" (Flöde).
  2. "Machine name" ends with "Fl [Edit]"
  3. Javascript console contains:
    Uncaught SyntaxError: unmatched ) in regular expression - es.array.loin.js:17:74
    slugify es.array.join.js:17
    transliterate machine-name.js:311
    machineNameHandler machine-name.js:114
    jQuery 8
    dispatch
    handle
    add
    Le
    each
    each
    Le
    on
    attach machine-name.js:243
    attach machine-name.js:120
    attachBehaviors drupal.js:166
    attachBehaviors drupal.js:162
    big_pipe.js:153
    big_pipe.js:184
    4 es.array.join.js:17:74
    slugify es.array.join.js:17
    transliterate machine-name.js:311
    machineNameHandler machine-name.js:114
    jQuery 8
    dispatch
    handle
    (Async: EventListener.handleEvent)
    add
    Le
    each
    each
    Le
    on
    attach machine-name.js:243
    forEach self-hosted:203
    attach machine-name.js:120
    attachBehaviors drupal.js:166
    forEach self-hosted:203
    attachBehaviors drupal.js:162
    big_pipe.js:153
    big_pipe.js:184
  4. Proposed resolution

    Replace non-ACII character(s) (with REGEX) to approximated ACII character i.e. ö -> o, å -> a, š -> s etc..

    Remaining tasks

    All

    User interface changes

    None

    API changes

    Data model changes

    Release notes snippet

    Versions

    Drupal version: 10.2.4
    Webserver: Apache/2.4.41 (Ubuntu 20.04.6)
    PHP: 8.2.16 FPM/FastCGI
    PHP Modules: calendar, Core, ctype, date, dom, exif, FFI, fileinfo, filter, ftp, gd, gettext, hash, iconv, json, libxml, mbstring, openssl, pcntl, pcre, PDO, pdo_pgsql, pgsql, Phar, posix, random, readline, Reflection, session, shmop, SimpleXML, sockets, sodium, SPL, standard, sysvmsg, sysvsem, sysvshm, tokenizer, xml, xmlreader, xmlwriter, xsl, Zend OPcache, zlib
    Zend Modules: Zend OPcache
    Database version: 12.18 (Ubuntu 12.18-0ubuntu0.20.04.1)
    Database system: PostgreSQL (Ubuntu 20.04.6 on different server)
    Browser: Firefox 123.0.1 (64-bit)

🐛 Bug report
Status

Fixed

Version

10.3

Component
Base 

Last updated 1 day ago

Created by

🇸🇪Sweden UserOne.se

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • According to https://www.drupal.org/node/3367037 this is supposed to work and it is not Ajax.

    Is this bug replicated on a newly-installed and unmodified Drupal site?

    See the Issue tags field description for the reason I removed tags.

  • First commit to issue fork.
  • Status changed to Postponed: needs info 9 months ago
  • 🇫🇷France nod_ Lille

    Can you type in the JS console: console.log(drupalSettings.machineName) and post the result here please?

  • 🇫🇷France nod_ Lille
  • Status changed to Active 9 months ago
  • 🇸🇪Sweden UserOne.se

    #2, #4
    Yes, I created a new DB, new instance of Drupal 10 as a new project with "Workflow" as the only added module. All installations were performed as instructed on https://www.drupal.org/docs/getting-started/installing-drupal/get-the-code i.e.

    • composer create-project drupal/recommended-project newD10
    • Installation and configuration in browser "as usual"
    • composer require drupal/workflow
    • Enabling only workflow module and no submodules
    • Under Admin -> Configuration -> Workflow I choose (i.e. click on button) "+ Add workflow"
    • Textbox with labe "Label*" I type "Flö"
    • Text to the right of the textbox says "Machine name: fl [Edit] (see attached image "fl1.png"
    • Javascript console - see attached image "fl2.png"

    Please note:
    Since I reinstalled and rechecked... this might not be D10 issue,
    it could be Workflow related since correct behavior is discovered
    in other situation in D10 not related to Workflow.
    In such case this might need to be moved to Workflow issue queue - could you do it or should I?

  • 🇫🇷France nod_ Lille

    It happens on date format inputs too, thank you for the detailed steps.

    Downgrading to normal since there is a workaround (editing the machine name field directly) and the feature is for admin/developers, not regular site users. This is not related to ajax so moving to the base system.

  • 🇸🇪Sweden UserOne.se

    Oops, date formats? Did not see that one comming.

    You are welcome (regarding detailed reporting).
    Good job on spotting date format problem, excluding Ajax, downgrading and moving.

  • Assigned to jmaxant
  • Pipeline finished with Failed
    7 months ago
    Total: 168s
    #192620
  • 🇫🇷France jmaxant

    Hi,
    I just pushed a commit in mr !8316 fixing this error.

    First time contributing to the core, so please let me know if I didn't apply proper workflow, naming or anything else.

  • Issue was unassigned.
  • Pipeline finished with Failed
    7 months ago
    Total: 968s
    #192641
  • Status changed to Needs review 7 months ago
  • Pipeline finished with Success
    7 months ago
    Total: 3816s
    #192694
  • Status changed to Needs work 7 months ago
  • 🇺🇸United States smustgrave

    Ran test-only feature

    1) Drupal\Tests\system\FunctionalJavascript\System\DateFormatTest::testDateFormatXss
    Behat\Mink\Exception\ResponseTextException: The text "Machine name: label_with_special_characters" was not found anywhere in the text of the current page.
    /builds/issue/drupal-3428652/vendor/behat/mink/src/WebAssert.php:907
    /builds/issue/drupal-3428652/vendor/behat/mink/src/WebAssert.php:293
    /builds/issue/drupal-3428652/core/tests/Drupal/Tests/WebAssert.php:975
    /builds/issue/drupal-3428652/core/modules/system/tests/src/FunctionalJavascript/System/DateFormatTest.php:77
    FAILURES!
    Tests: 1, Assertions: 6, Failures: 1.
    

    Which shows coverage.

    Left 1 small comment on the MR though.

    Good job for a first MR.

  • 🇮🇳India prashant.c Dharamshala

    Prashant.c made their first commit to this issue’s fork.

  • Pipeline finished with Canceled
    7 months ago
    Total: 71s
    #193269
  • 🇮🇳India prashant.c Dharamshala
    1. Resolved this https://git.drupalcode.org/project/drupal/-/merge_requests/8316#note_323107 as the constant was not required because was not being used anywhere else
    2. Keeping the issue in NW as per https://www.drupal.org/project/drupal/issues/3428652#comment-15630065 🐛 Broken auto creation of machine name field Needs work
    3. Tested with content type fields machine names, special characters/non-ASCII characters are being replaced properly, tested with é, à, ö, ñ, etc.

    Thanks!

  • Pipeline finished with Success
    7 months ago
    Total: 552s
    #193270
  • 🇫🇷France jmaxant

    Weirdly enough, tests succeed on a ddev install with ddev/ddev-selenium-standalone-chrome for functionalJS tests.
    I'll have a deeper look later this week, maybe there is some configuration discrepancies on my env vs drupal-ci.

  • Pipeline finished with Success
    6 months ago
    Total: 628s
    #195308
  • Assigned to jmaxant
  • Issue was unassigned.
  • Status changed to Needs review 6 months ago
  • 🇫🇷France jmaxant

    Ok, so, after much contemplation and reading the docs, it seems the Test-only changes job is supposed to fail ? I feel a little silly tbh, but hey, I learnt something new !

    Changing the status to need review.

  • Status changed to RTBC 6 months ago
  • 🇺🇸United States smustgrave

    Feedback on the MR appears to be addressed.

    Re-ran the test-only job after the pattern change and it still fails as expected, so coverage is still good.

  • Merge request !8677Resolve #3428652 "Do not use replace pattern" → (Closed) created by nod_
  • Status changed to Needs review 6 months ago
  • 🇫🇷France nod_ Lille

    I think the problem is elsewhere. We're passing a regex to slugify for the allowedChars option where it expects a list of characters (so, not a regex). It's fine to do that because the replace pattern is used before sending the string to slugify.

    Tried a different approach, let's see what testbot says.

  • 🇫🇷France nod_ Lille

    the tests in machineNameTransliterationTest.js are not correct, it doesn't test how things are used in Drupal from the PHP

  • Status changed to Needs work 6 months ago
  • 🇺🇸United States smustgrave

    So sounds like the test needs to be corrected? Would almost say it should be separate from this issue and this one postponed but probably can roll the two together

  • Pipeline finished with Failed
    6 months ago
    Total: 636s
    #218509
  • First commit to issue fork.
  • 🇫🇷France nod_ Lille

    Well I wanted to keep that test and fix the code instead. There is no reason to have 2 underscores in the transformed name.

  • Pipeline finished with Success
    4 months ago
    Total: 1022s
    #258952
  • 🇫🇷France nod_ Lille

    added back the failing test

  • 🇮🇳India ankitv18

    Oh alright :)

  • Pipeline finished with Failed
    4 months ago
    #258969
  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    Oh I see... the problem is with date formats and workflow specifically because they have '([^a-z0-9_]+)|(^custom$)', and this is not going to work with what the allowedChars option does for slugify.

    My guess is that the workflow module has incorrectly copy pasted the date format code as I can't see why custom would not be allowed there. But regardless date format machine names are broken in the same way.

    This is going to be fun to fix in a way that supports both 'replace_pattern' => '([^a-z0-9_]+)|(^custom$)', and 'replace_pattern' => '[^a-z0-9_]+',

  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    It's turns out that 🐛 Machine name generation is way too slow Fixed was built on an incorrect premise. That replace_pattern was always compatible with allowedChars - that's just not true. We need to do the replacing ourselves. So we transliterate and then do the replacement and do not use slugify.

  • Status changed to Needs review 4 months ago
  • 🇫🇷France nod_ Lille

    well that was easier than expected :)

  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    alexpott changed the visibility of the branch fix/3428652d-date_format_machine_name_pattern_replacement to hidden.

  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    alexpott changed the visibility of the branch 3428652-broken-auto-creation to hidden.

  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    alexpott changed the visibility of the branch 11.x to hidden.

  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    @nod_ I agree with the latest fix. Think we should add test coverage of the bug exposed here.

  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    I checked that the transliteration per language is being tested and it is - in \Drupal\FunctionalJavascriptTests\MachineName\MachineNameTransliterationTest - so we have test coverage of the changes how we build the replace array. I slightly improved the test because it feels in scope.

    I this issue is ready and will commit once rtbc.

  • Status changed to RTBC 4 months ago
  • Changes look good.

    Tested locally on latest 11.x with standard profile install. No JS errors when adding these date formats:



  • Actually, one question/nit about this:

    (function ($, Drupal, drupalSettings, transliterateLibrary) {
    ...
      const transliteratedSource = transliterateLibrary(source, { replace });
    ...
    })(jQuery, Drupal, drupalSettings, transliterate);
    

    Can the "transliterate" global object be passed into the the self-invoking function as "transliterate" instead of "transliterateLibrary"? It could be helpful to be able to grep for "transliterate(" for uses in JS code.

  • 🇫🇷France nod_ Lille

    We have a method on the Drupal object called transliterate too, if you search for it now you'll find all the Drupal calls, it's to make it explicit this is not the same thing.

  • 🇫🇷France nod_ Lille
  • We have a method on the Drupal object called transliterate too, if you search for it now you'll find all the Drupal calls, it's to make it explicit this is not the same thing.

    Makes sense. Can stay at RTBC.

  • Status changed to Fixed 4 months ago
  • 🇬🇧United Kingdom alexpott 🇪🇺🌍

    Committed and pushed 73ec28542e to 11.x and adc4219ee6 to 11.0.x and 2ca2d354d0 to 10.4.x and 747c866073 to 10.3.x. Thanks!

    Backported this to 10.3.x as a bug fix with no API concerns.

    • alexpott committed 747c8660 on 10.3.x
      Issue #3428652 by nod_, jmaxant, alexpott, prashant.c, godotislate,...
    • alexpott committed 2ca2d354 on 10.4.x
      Issue #3428652 by nod_, jmaxant, alexpott, prashant.c, godotislate,...
    • alexpott committed adc4219e on 11.0.x
      Issue #3428652 by nod_, jmaxant, alexpott, prashant.c, godotislate,...
    • alexpott committed 73ec2854 on 11.x
      Issue #3428652 by nod_, jmaxant, alexpott, prashant.c, godotislate,...
  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024