Indexing after deleting index leads to problematic index definition

Created on 19 June 2024, 6 months ago

Problem/Motivation

The index definition in ElasticSearch is different depending on if "Clear all indexed data" is executed before indexing an item.

The "type" property is either {"type":"keyword"} or {"type":"text","fields:{"keyword":{"type":"keyword","ignore_above":256}}}.

With the latter, using aggregation functions on the "type" property will fail with

Text fields are not optimised for operations that require per-document field data like aggregations and sorting, so these operations are disabled by default. Please use a keyword field instead.

(this are the errors reported in issue #3316267 and #2913535).

Steps to reproduce

Correct property definition:

  1. Delete ElasticSearch index: curl -XDELETE -s http://127.0.0.1:9200/elasticsearch_index_foo_content_index
  2. Run "Clear all indexed data" or run drush search-api:clear
  3. Check the property definition in the index:
    $ curl -s 127.0.0.1:9200/elasticsearch_index_foo_content_index/ | jq '.[].mappings.properties.type'
    {
      "type": "keyword"
    }
    

Broken property definition:

  1. Delete ElasticSearch index: curl -XDELETE -s http://127.0.0.1:9200/elasticsearch_index_foo_content_index
  2. Index a single document: drush search-api:index --limit=1
  3. Check the property definition in the index:
    $ curl -s 127.0.0.1:9200/elasticsearch_index_foo_content_index/ | jq '.[].mappings.properties.type'
    {
      "type": "text",
      "fields": {
        "keyword": {
          "type": "keyword",
          "ignore_above": 256
        }
      }
    }
    

"type" is not the only property that is different. "_language", "changed", "created" and others have the wrong definition.

Proposed resolution

Either check if the index exists before indexing, or document the problem and its solution.
Maybe automatically verify if the index is correct somehow.

🐛 Bug report
Status

Active

Version

7.0

Component

Elasticsearch Connector

Created by

🇩🇪Germany cweiske

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Production build 0.71.5 2024