Aggregated FullText fields need to have fielddata=true to be searchable

Created on 14 May 2021, almost 4 years ago
Updated 28 September 2023, over 1 year ago

Problem/Motivation

Hello,

I have added an aggregated field that is configured to be a union aggregation to the search index. No search results are generated and I get this error:

Elasticsearch\Common\Exceptions\BadRequest400Exception: {"error":{"root_cause":[{"type":"illegal_argument_exception","reason":"Fielddata is disabled on text fields by default. Set fielddata=true on [tags] in order to load fielddata in memory by uninverting the inverted index. Note that this can however use significant memory. Alternatively use a keyword field instead."}],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":

It appears that the field type should either be keyword or text with fielddata=true enabled.

Here is the field that is generated in the index:

          "tags" : {
            "type" : "text",
            "fields" : {
              "keyword" : {
                "type" : "keyword",
                "ignore_above" : 256
              }
            }
          },

Steps to reproduce

Setup the index:
Create an index with a General Aggregated Field that is a Union type.
Once added to the fields, ensure that it is a Fulltext type.
Build the index.

Build the View:
Create a view that has a keyword search filter and add the field you create above to it.
Try and search for a value that is in either field you aggregated into one (the one you added to the index)

At this point you should make two observations:

  1. Views wont show any results
  2. In the drupal log, you should see the error above

It appears that you can't switch field types after the index has been built. If i set fielddata=true, the search works and the error goes away.

curl -X PUT "YOUR_SERVER:9200/YOUR_INDEX/_mapping/_doc?pretty" -H 'Content-Type: application/json' -d'
{
  "properties": {
    "YOUR FIELD": { 
      "type":     "text",
      "fielddata": true
    }
  }
}
'

Proposed resolution

It seems like elastic search would prefer the fieldtype to be keyword for performance reasons over a fielddata enabled text.

Remaining tasks

User interface changes

API changes

Data model changes

🐛 Bug report
Status

Needs review

Component

Elasticsearch Connector

Created by

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

Production build 0.71.5 2024