Prefix the document ID to enable multi-site indexing

Created on 17 May 2018, over 6 years ago
Updated 30 August 2024, 4 months ago

Usually a sites has its own backend server (i.e. elasticsearch) which It interacts with, having indexes isolated storing documents with ids taking the entities ids such as nid (i.e: 8, an Integer). These combined ids (i.e "_id":"entity:node/8:en") are useless when merging documents within indexes from different sites. The most evident issue is the overlap of documents in the merged index which could be avoid using a UUID instead of an combined id.

I could use a processor plugin to alter the documents IDs, doing this:

public function alterIndexedItems(array &$items) {
   foreach ($items as $item_id => $item) {
      $node = $item->getOriginalObject()->getValue();
      if (!($node instanceof NodeInterface)) {
        continue;
      }
      // create new item using the uuid
      $items[$node->uuid()] = $item; // -> the $item keeps the same id that has before.
     // Remove this one to leave the one with the uuid
      unset($items[$item_id]);
    }
}

This alter allows to switch the "_id" from the combined id to the uuid one in the elasticsearch index, almost happy until see that this has impact on tracking and seems to not be deleting the documents in the index when content is deleted.

🐛 Bug report
Status

Postponed: needs info

Version

7.0

Component

Code

Created by

🇨🇴Colombia alejomc

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • 🇨🇦Canada mparker17 UTC-4

    I think the version number wasn't updated when this moved from the search_api queue into the elasticsearch_connector queue: the elasticsearch_connector-8.x-1.x branch is no longer supported.

    @alejomc and @dakku, could you confirm which version you're using?

    (I am cleaning up old issues as part of a big review of open issues for 🌱 [Meta] Roadmap for a stable release with security coverage Active : thank you for your patience with me)

Production build 0.71.5 2024