"Fielddata is disabled on [field] in [index]" error when trying to add a facet on a String field in an index managed by Search API

Problem/Motivation

Drupal's contributed Facets module → lets you apply Elasticsearch bucket aggregations to searches on an Elasticsearch index.

When you assign Search API field data types to fields in an index manged by Search API → , for a field containing text data, you can assign a Type of Search-As-You-Type, Elasticsearch Spellcheck, Fulltext, or String. Most of these map to the Elasticsearch text field type; but Elasticsearch connector additionally creates a sub-field of type keyword (that is to say, Elasticsearch Connector creates a field of type "text with a .keyword sub-field", like the current documented behavior of Elasticsearch's dynamic field mapping)

(see also \Drupal\elasticsearch_connector\SearchAPI\FieldMapper::mapFieldProperty())

But, as noted in the Elasticsearch documentation on Terms bucket aggregation:

By default, you cannot run a terms aggregation on a text field. Use a keyword sub-field instead. Alternatively, you can enable fielddata on the text field to create buckets for the field’s analyzed terms. Enabling fielddata can significantly increase memory usage.

We discovered in #3465988-23: Error with facets "Expected [START_OBJECT] under [filter], but got a [START_ARRAY]..." → that when you set up a Facet on a field whose Search API data type is Search-As-You-Type, Elasticsearch Spellcheck, Fulltext, or String, it tries to aggregate on the main field whose Elasticsearch type is text, i.e.: not on the sub-field whose Elasticsearch type is keyword. This results in an error like...

Fielddata is disabled on [FIELD_NAME] in [INDEX_NAME]. Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [FIELD_NAME] in order to load field data by uninverting the inverted index. Note that this can use significant memory.

... and faceting (and the search) doesn't work.

If you're using something other than Drupal to make changes to the index (e.g.: Elastic web crawler), then you won't necessarily run into this issue, because that other software can create fields of type keyword at the top level.

Steps to reproduce

1. Set up a local environment → , including both the One-time setup, and the Setup for working on an issue (I ended up with drupal 11.2.4 → , and search_api 8.x-1.39 → ).
2. Install facets 3.0.1 → , and better_exposed_filters 7.0.5 →
3. Log in, if you aren't already
4. Go to /admin/modules, check Facets Exposed Filters (facets_exposed_filters), click Install
5. Go to /test-elasticsearch-index-search, verify the view is working
6. Go to /admin/config/search/search-api/server/elasticsearch_server/edit, check Enable debugging mode: log ElasticSearch network traffic, then click Save
7. Go to /admin/structure/views/view/test_elasticsearch_index_search. Under Filter criteria, click Add. In the Add filter criteria modal, filter by Category = Facets, check Keywords (keywords) in the Facets category, then click Add and configure filter criteria. In the Configure extra settings for filter criterion Facet: Keywords (keywords), click Apply and continue to accept the defaults. In the Configure filter criterion 2 of 2: Facet: Keywords (keywords) modal, click Apply to accept the defaults. Click Save to save the view.
8. Go to /test-elasticsearch-index-search.
   • Expected behavior: The view is still working; now there is a Keywords exposed filter.
   • Actual behavior: You see an Error message at the top of the page:
     

             Error querying index test_elasticsearch_index
             

9. Go to /admin/reports/dblog. Filter by Type = elasticsearch_connector_client. You should see a Response (retry 0): 400 message, and a message that begins with Headers: @"X-elastic-product":["Elasticsearch"] immediately afterwards. Click the message that begins with Headers: @"X-elastic-product":["Elasticsearch"].

   The full message is:

         Headers: @"X-elastic-product":["Elasticsearch"],"content-type":["application\/vnd.elasticsearch+json;compatible-with=8"],"content-length":["2127"] Body: {"error":{"root_cause":[@"type":"illegal_argument_exception","reason":"Fielddata is disabled on [keywords] in [test_elasticsearch_index]. Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [keywords] in order to load field data by uninverting the inverted index. Note that this can use significant memory."],"type":"search_phase_execution_exception","reason":"all shards failed","phase":"query","grouped":true,"failed_shards":[{"shard":0,"index":"test_elasticsearch_index","node":"CqbzakXzQHmnt5466RDOXA","reason":@"type":"illegal_argument_exception","reason":"Fielddata is disabled on [keywords] in [test_elasticsearch_index]. Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [keywords] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}],"caused_by":{"type":"illegal_argument_exception","reason":"Fielddata is disabled on [keywords] in [test_elasticsearch_index]. Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [keywords] in order to load field data by uninverting the inverted index. Note that this can use significant memory.","caused_by":@"type":"illegal_argument_exception","reason":"Fielddata is disabled on [keywords] in [test_elasticsearch_index]. Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead. Alternatively, set fielddata=true on [keywords] in order to load field data by uninverting the inverted index. Note that this can use significant memory."}},"status":400}
       

Proposed resolution

To be determined.

Remaining tasks

1. Determine a path forward and describe it in the issue summary
2. Write a merge request
3. Review and feedback
4. RTBC and feedback
5. Commit
6. Release

User interface changes

To be determined.

API changes

To be determined.

Data model changes

To be determined.