Contrib.social

Field locales_source.source is not suitable for long texts and huge config objects

Open on Drupal.org →
Created on 10 January 2013, almost 12 years ago
Updated 28 August 2024, 4 months ago

We have a feed import that fails (we have no clue why).
Looking at the "Log" tab did not help much as it crashed. It uses the locale() function to display the (translated) error text which in this case throws an PDOException:

PDOException: SQLSTATE[22001]: String data, right truncated: 1406 Data too long for column 'source' at row 1: INSERT INTO {locales_source} (location, version, source, context, textgroup) VALUES (:db_insert_placeholder_0, :db_insert_placeholder_1, :db_insert_placeholder_2, :db_insert_placeholder_3, :db_insert_placeholder_4); Array ( ..[very long].. ) ) in locale() (Zeile 714 von /html/drupal/modules/locale/locale.module).

(That is line 714 in /html/drupal/modules/locale/locale.module)

It is the "else" code block "We don't have the source string, cache this as untranslated...." with a call to "db_merge()->execute" which fails.

My dirty patch is to call this "else"-block only for texts with less than 128 Bytes (mb_strlen) and calling $locale_t[$langcode][$context][$string] = TRUE; only otherwise.

It would be better to catch this kind of PDOException with SQLSTATE[22001] and probably just forget it.
The "$locale_t... = TRUE;" is needed here, too, because drupal make calls to locale() over and over otherwise.

Another question is why to try to translate an exception's message as it might hide important details. Nevertheless there'a a bug in the locale() function or the database code used by it.

🐛 Bug report
Status

Postponed: needs info

Version

11.0 🔥

Component
Locale  →

Last updated about 5 hours ago

Created by

🇩🇪Germany tvierb

Live updates comments and jobs are added and updated live.
  • D8MI

    (Drupal 8 Multilingual Initiative) is the tag used by the multilingual initiative to mark core issues (and some contributed module issues). For versions other than Drupal 8, use the i18n (Internationalization) tag on issues which involve or affect multilingual / multinational support. That is preferred over Translation.

  • Needs tests

    The change is currently missing an automated test that fails when run with the original code, and succeeds when the bug has been fixed.

Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • Comment over 1 year ago →
    🇺🇸United States smustgrave

    Wonder if this should be reopened?

  • Comment over 1 year ago →
    🇺🇸United States emptyvoid

    I've got this issue in Drupal 9 and Drupal 10.
    Having the body field or content fields with both editorial content as well as full HTML on larger pages causes this local field validation to fail every single time.

    I have to patch each release so that the 10,000+ records and updates will actually complete in my builds.

    Most recent Patch attached.
    This commits to both Drupal 9.5.10 and Drupal 10.1.2

  • Comment 4 months ago →
    🇫🇷France andypost

    Maybe increasing size of blob for MySQL could help here as both Sqlite and Pgsql has limit for blobs 1GB but Mysql only 64k

  • Comment 4 months ago →
    🇨🇦Canada joseph.olstad

    It would be worthwhile looking into increasing the blob size. With that said, 64kb for a locale string is very large.

    When I manually replaced \r\n with \n and everything imported correctly with MySQL.
    The issue is that the ckeditor outputs \r\n, perhaps a filter should be modified for the locale processing that treats \r\n the same as \n.

    Ideally the common web interface tools should be handled and understood the locale system.

    It seems strange that we'd be translating strings of 64 kilobytes when we could have line items as was originally intended.

  • Comment 4 months ago →
    🇨🇦Canada joseph.olstad

    Alternatively, perhaps the config system could automatically convert \r\n to \n and then maybe the locale system perhaps would work correctly. Might be easy to create this patch.

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024