Contrib.social

Index _mappings can be incorrect

Open on Drupal.org →
Created on 25 March 2025, 18 days ago

Problem/Motivation

There is an edge case where the _mappings for an index can be incorrect.

If you index from the command line and the index doesn't exist the BackendClient::indexItem() indexes the content anyway. It creates the packet to send to Opensearch with the data from the entities; and sends it. Opensearch, not having an index definition, is happy to accept the data (as it is designed to) and infers the type from the data sent to it.

There are everyday cases that can cause the index to be removed so I think we need to handle the case gracefully.

  • I've upgraded my Opensearch server and cron indexed items while I was still transferring data.
  • I've copied my database to a local development environment and ran drush sapi-i to get some test data.
  • I've created a test enviroment in the cloud to test my PR and it didn't specifically have to update the Index entity.
  • I've deleted the index intentionally and an automated index beat me to my next tasks.
  • Many others...

Steps to reproduce

A concrete example is that if you have a date field it will be indexed with the type 

{
  "type": "date",
   "format": "strict_date_optional_time||epoch_second"
}

. This _mapping will be created/updated on save or update of the Index entity.

If you remove the index from Opensearch and run drush sapi-i [index] the field will be indexed with the type

{
  "type": "long",
}

.

Proposed resolution

In `BackendClient::indexItems()` do a check if the index exists and if not, add it, before processing the items.

Remaining tasks

User interface changes

API changes

Data model changes

🐛 Bug report
Status

Active

Version

2.3

Component

Code

Created by

🇦🇺Australia nterbogt

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024