Order translations alphabetically

Created on 27 March 2023, over 2 years ago

Problem/Motivation

When exporting translations with drush_language β†’ , they are not sorted in a logical order.
This makes Git diffs of the .po file really hard to read.

Steps to reproduce

Call PoDatabaseReader::readItem() multiple times.
You get translations in a seemingly random order.

Proposed resolution

Sort translations alphabetically directly in the SQL query in StringDatabaseStorage::dbStringSelect().

✨ Feature request
Status

Active

Version

9.5

Component
LocaleΒ  β†’

Last updated 1 day ago

Created by

πŸ‡«πŸ‡·France prudloff Lille

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @prudloff
  • @prudloff opened merge request.
  • Status changed to Needs review over 2 years ago
  • πŸ‡«πŸ‡·France prudloff Lille
  • Status changed to Needs work over 2 years ago
  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Sounds like a valid use case.

    Can we add a kernel test maybe to make sure they come out alphabetically.

  • πŸ‡΅πŸ‡±Poland artemboiko

    Caught the same problem, thanx for saving time.)

  • First commit to issue fork.
  • Pipeline finished with Failed
    about 1 year ago
    Total: 168s
    #214051
  • Pipeline finished with Canceled
    about 1 year ago
    Total: 10s
    #214055
  • πŸ‡©πŸ‡ͺGermany tgauges

    I created a new branch against 11.x which first sorts the translations by their source string and then the context. The context is needed for equal source string with different context.

    Before review, the test should be extended to also test the case for equal source strings but different context.

  • Pipeline finished with Success
    about 1 year ago
    Total: 663s
    #214056
  • πŸ‡«πŸ‡·France prudloff Lille

    prudloff β†’ changed the visibility of the branch 3350704-order-translations-alphabetically to hidden.

  • Pipeline finished with Canceled
    4 months ago
    Total: 59s
    #442313
  • Pipeline finished with Success
    4 months ago
    Total: 728s
    #442314
  • πŸ‡«πŸ‡·France prudloff Lille
  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Removing the tests tag as https://git.drupalcode.org/issue/drupal-3350704/-/jobs/4589844 shows the coverage

    See no issue with ordering these.

  • πŸ‡¬πŸ‡§United Kingdom catch

    I think this might need a new database index - see note on the MR.

  • πŸ‡©πŸ‡ͺGermany tgauges
  • πŸ‡©πŸ‡ͺGermany tgauges
  • πŸ‡¦πŸ‡ΊAustralia larowlan πŸ‡¦πŸ‡ΊπŸ.au GMT+10

    We need to add an index then to avoid the `using filesort` - thanks @tgauges

  • πŸ‡©πŸ‡ͺGermany tgauges

    What kind of index needs to be added to "avoid the using filesort"?

    I read a bit of MySQL ORDER BY Optimization but my experiments did not result in the avoidance of using filesort when running EXPLAIN SELECT s.* FROM locales_source s ORDER BY s.source ASC, s.context ASC;.

    I tried the following on 10.11.11-MariaDB-ubu2204-log:

    • Change the existing index (source(30), context) to (source(2048), context).
    • Add a new index (source(2048)).
    • Add a new index (context).

    The largest context value in my database is 1452 queried with SELECT length(source) AS length FROM locales_source ORDER BY length DESC;.

  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Hello @tgauges first appreciate taking on this task!

    Since this seems to be still in discussion moving to NW, discussions can still happen without being in review

  • Pipeline finished with Failed
    17 days ago
    Total: 150s
    #530268
  • πŸ‡©πŸ‡ͺGermany tgauges

    @smustgrave sorry, my bad.

    Since a usable index might not be possible I adjusted the merge request to only add the ORDER BY statements when multiple strings are loaded.

    This too might not be acceptable. In that case, I propose a function API update so that callers can decide via parameter whether they want the translations sorted or not.

    Another option might be to make the \Drupal\locale\StringDatabaseStorage::dbStringSelect function public so that the query can be adjusted and executed at will.

Production build 0.71.5 2024