Special character handling

Created on 5 March 2018, over 7 years ago

Updated 22 May 2025, 6 months ago

The module works fine but it does not handle special characters well.

For example "Lenny's" is output as "lenny039s".

My specific use case scenario: Automatic SKU generation for commerce product failed due to some product titles containing characters such as the example provided.

I fiddled with the module and did the following:

token_modifier.module: changed function token_modifier_tokens($type, $tokens, array $data = [], array $options = []) including the line:
$options['sanitize'] = FALSE;
Here the sanitize option is the one that messes up what is returned.

lowercase.inc: The module transliteration → does special char handling so using one of its functions I changed the following for my needs:

function token_modifier_token_modifier_lowercase($token, $data, $options) {
  // original string to be processed with transliteration module instead of strtolower
  $token_replacement = token_replace("[$token]", $data, $options);
  // return string in lowercase and underscores
  $token_replacement = transliteration_clean_filename($token_replacement);
  // remove underscores
  $token_replacement = str_ireplace('_', '', $token_replacement);
  return $token_replacement;
  // return strtolower(token_replace("[$token]", $data, $options));
}

Since I do not need more functionality other than the lowercase scenario I add this as a suggestion that may enhance the module and a possible fix for others that may encounter my same issue.

The latter would of course add a dependency to the transliteration → module but it is a good option to consider.

💬 Support request

Status

Closed: outdated

Version

1.1

Component

Code

Created by

🇨🇴Colombia jucedogi

Live updates comments and jobs are added and updated live.

Incomplete comments

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

Comment 6 months ago →
🇬🇧United Kingdom jofitz Coventry
Closing ticket for unsupported D7 version
Status changed to Active 3 months ago5:03pm 18 August 2025
Comment 3 months ago →
pearls
This point is still problem with token modifier 2.x and Drupal 11.X

When I use the auto entity label module (AEL) I don't have any problems with special characters.

But when I want to use the token modifier with AEL, a problem occurs.

For example; [token-modifier:length:50:node:body]

When the pattern for the label uses a Text (plain) field, it happens with apostrophes, quotation marks and ampersands:

' becomes '
" becomes "
& becomes &

Any idea,please.. Thanks

contrib.social Blog FAQ Discussions

Production build 0.71.5 2024