field_sql_storage_field_storage_load does use an unnecessary sort in the DB leading to a filesort

Created on 22 August 2018, over 6 years ago
Updated 7 November 2024, 5 months ago

field_sql_storage_field_storage_load does use an unnecessary sort in the DB leading to a:

using where, using filesort

A better approach is to just use array_multisort() in PHP.

Explain before:

mysql> explain
    -> SELECT t.*
    -> FROM 
    -> field_data_field_teams t
    -> WHERE  (entity_type = 'node') AND (entity_id IN  ('1150776', '1150796', '1151126', '1151136', '1151856', '1151911', '1151916', '1151936')) AND (language IN  ('und')) AND (deleted = '0') 
    -> ORDER BY delta ASC;
+----+-------------+-------+-------+------------------------------------------------+---------+---------+------+------+-----------------------------+
| id | select_type | table | type  | possible_keys                                  | key     | key_len | ref  | rows | Extra                       |
+----+-------------+-------+-------+------------------------------------------------+---------+---------+------+------+-----------------------------+
|  1 | SIMPLE      | t     | range | PRIMARY,entity_type,deleted,entity_id,language | PRIMARY | 391     | NULL |   13 | Using where; Using filesort |
+----+-------------+-------+-------+------------------------------------------------+---------+---------+------+------+-----------------------------+
1 row in set (0.00 sec)

After:

mysql> explain SELECT t.*
    -> FROM 
    -> field_data_field_teams t
    -> WHERE  (entity_type = 'node') AND (entity_id IN  ('1150776', '1150796', '1151126', '1151136', '1151856', '1151911', '1151916', '1151936')) AND (language IN  ('und')) AND (deleted = '0') 
    -> ;
+----+-------------+-------+-------+------------------------------------------------------------+---------+---------+------+------+-------------+
| id | select_type | table | type  | possible_keys                                              | key     | key_len | ref  | rows | Extra       |
+----+-------------+-------+-------+------------------------------------------------------------+---------+---------+------+------+-------------+
|  1 | SIMPLE      | t     | range | PRIMARY,entity_type,deleted,entity_id,language,idx_primary | PRIMARY | 391     | NULL |   13 | Using where |
+----+-------------+-------+-------+------------------------------------------------------------+---------+---------+------+------+-------------+
1 row in set (0.13 sec)
πŸ› Bug report
Status

Needs review

Version

11.0 πŸ”₯

Component

field system

Created by

πŸ‡©πŸ‡ͺGermany Fabianx

Live updates comments and jobs are added and updated live.
  • Needs backport to D7

    After being applied to the 8.x branch, it should be considered for backport to the 7.x branch. Note: This tag should generally remain even after the backport has been written, approved, and committed.

Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • πŸ‡³πŸ‡ΏNew Zealand quietone

    Based on the comment in #15 this should be a won't fix. I am setting to 'needs review' to get opinions.

  • πŸ‡¬πŸ‡§United Kingdom joachim

    This needs an IS update for the title at least, as on D11 it's surely not that function any more!

    I found an explanation of 'using filesort' on stack overflow, which says:

    > When you don't see "using filesort," it means the order you requested for the query result matches the natural order in which the data was read, so there was no extra work needed to sort the result.

    So I'm curious about this. Field values are surely inserted by delta order per-entity ID, so I assume that the reason we're getting a 'using filesort' is because we're loading multiple entities.

    What if we did 'order by ID, delta'? That would then probably match the read order.

  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Sounds like there may be more discussion for this. Could someone update the issue summary for how this could be an improvement? Not sure PNMI is the right status but NR didn't spark any conversation. So figured i'd put there before it gets closed out.

  • Status changed to Postponed: needs info 9 days ago
  • πŸ‡ΊπŸ‡ΈUnited States smustgrave

    Just following up if someone is able to update the summary? else do we close this.

  • πŸ‡¬πŸ‡§United Kingdom joachim

    This is still the case on D11.

    The query is:

    SELECT t.*
        FROM
        node__body t
        WHERE (entity_id IN ('1', 2)) AND (deleted = '0') AND (langcode IN ('en', 'und', 'zxx'))
        ORDER BY delta ASC
    

    and EXPLAINing it still shows a filesort.

    I was right in #19 that we could add the entity_id to the sort to get rid of the filesort. But better still, as in the OP, we don't need the sort at all: the values from this query are added to a $values array which already has the entity IDs (or revision IDs) and is already ordered.

    So we don't even need to replace the SQL ordering with PHP ordering -- no ordering is needed at all.

  • πŸ‡³πŸ‡±Netherlands daffie

    We can as an alternative solution add a new index on the delta column. Not sure if a multi-column index would be better.

  • πŸ‡¬πŸ‡§United Kingdom joachim

    We don't need to sort this query at all.

Production build 0.71.5 2024