Implementing Elasticsearch Completion Suggester

@ebeyrent could you maybe share how you did this. I managed to create a custom SearchApiDataType but I can't get the mapping to work on my server. I'm really struggling finding the right documentation for this.

A new index field type "completion" would be necessary:
https://www.elastic.co/guide/en/elasticsearch/reference/7.17/search-sugg...

{
  "mappings": {
    "properties": {
      "suggest": {
        "type": "completion"
      },
[...]

The elasticsearch_connector-8.x-1.x-dev branch is no longer maintained... development for Elasticsearch 8 is happening on 8.0.x-dev branch and development for Elasticsearch 7 is happening on the 8.x-7.x branch.

@cweiske you linked to Elasticsearch 7 documentation, does that mean you're using Elasticsearch 7?

I am indeed using ElasticSearch 7, but the completion type is available in both ES7 and ES8.

Okay! I will update the Version field to 8.x-7.x-dev.

Unfortunately, I'm not familiar with Elasticsearch 7 or the code/API in the 8.x-7.x-dev branch of this module, because I work exclusively with ES8, and I only work on the 8.0.x branch (which was a pretty extensive re-write — see 📌 Investigate search_api_opensearch as base for elasticsearch_connector Fixed for more information).

That being said, we use suggester queries in the 8.0.x ticket ✨ Support for Search API Spellcheck Active — but be aware that it is still a work-in-progress. As I understand it, ✨ Feature: Support for Search API Spellcheck Needs review is the corresponding ticket for the 8.x-7.x branch, and there appears to be a patch with a code that uses suggester queries in that ticket.

Other than 🐛 Search as you type field not working as expected Active (which has no patch or solution), and ✨ Feature: Support for Search API Spellcheck Needs work (which heavily inspired our work-in-progress 8.0.x spell-check code) I cannot find anything else that seems to be relevant in the Search API OpenSearch issue queue → .

Here's how I did it.

1. Implement some hooks:

function my_module_search_api_field_type_mapping_alter(array &$mapping) {
  $mapping['completion'] = 'completion';
}

function my_module_elasticsearch_connector_supported_data_types_alter(array &$data_types) {
  $data_types[] = 'completion';
}

2. Add a new search_api data type:

namespace Drupal\my_module\Plugin\search_api\data_type;

use Drupal\search_api\DataType\DataTypePluginBase;

/**
 * Provides a completion data type.
 *
 * @SearchApiDataType(
 *   id = "completion",
 *   label = @Translation("Autocomplete (suggest)"),
 *   description = @Translation("Indexes field using the index's suggest analyzer"),
 *   fallback_type = "object"
 * )
 */
class CompletionDataType extends DataTypePluginBase {

  /**
   * {@inheritdoc}
   */
  public function getValue($value) {
    // Implement your logic here.
  }

}

3. Subscribe to the indexing events:

namespace Drupal\my_module\EventSubscriber;

use Drupal\Core\Database\Connection;
use Drupal\Core\Entity\EntityInterface;
use Drupal\Core\Entity\EntityTypeManager;
use Drupal\elasticsearch_connector\Event\BuildIndexParamsEvent;
use Drupal\elasticsearch_connector\Event\PrepareIndexMappingEvent;
use Symfony\Component\EventDispatcher\EventSubscriberInterface;

/**
 * Class ElasticSearchIndexSubscriber.
 *
 * This subscriber gets called by the ElasticSearch Connector's IndexFactory
 * class.
 *
 * @see \Drupal\elasticsearch_connector\ElasticSearch\Parameters\Factory\IndexFactory::mapping()
 */
class ElasticSearchIndexSubscriber implements EventSubscriberInterface {

  /**
   * Entity type manager.
   *
   * @var \Drupal\Core\Entity\Entity
   */
  private $entityTypeManager;

  /**
   * Connection service.
   *
   * @var \Drupal\Core\Database\Connection
   */
  private $connection;

  /**
   * Constructs a new ElasticSearchIndexSubscriber object.
   *
   * @param \Drupal\Core\Entity\EntityTypeManager $entityTypeManager
   *   Entity type manager.
   * @param \Drupal\Core\Database\Connection $connection
   *   The connection service.
   */
  public function __construct(EntityTypeManager $entityTypeManager, Connection $connection) {
    $this->entityTypeManager = $entityTypeManager;
    $this->connection = $connection;
  }

  /**
   * {@inheritdoc}
   */
  public static function getSubscribedEvents() {
    // Map methods in this class to defined events provided by the
    // Elasticsearch Connector module.
    return [
      PrepareIndexMappingEvent::PREPARE_INDEX_MAPPING => ['doPrepareIndexMapping'],
      BuildIndexParamsEvent::BUILD_PARAMS => ['alterIndexParams'],
    ];
  }

  /**
   * Event listener.
   *
   * Called on elasticsearch_connector.build_params event.  This method
   * modifies the data to be indexed in ElasticSearch.  Specifically, this adds
   * handling for completion fields where the bulkIndex() method in
   * \Drupal\elasticsearch_connector\ElasticSearch\Parameters\Factory\IndexFactory
   * makes assumptions about how the fields' data should be represented.  In
   * the case of completion fields, the values are wrapped in an extra array,
   * which causes indexing to fail.  This method removes the array nesting so
   * that the data is correct.
   *
   * @param \Drupal\elasticsearch_connector\Event\BuildIndexParamsEvent $event
   *   The event.
   *
   * @see \Drupal\elasticsearch_connector\ElasticSearch\Parameters\Factory\IndexFactory::bulkIndex()
   */
  public function alterIndexParams(BuildIndexParamsEvent $event) {
   // Add your fields here.
    $completion_fields = [
      'multi_field_suggest',
    ];
    $data = $event->getElasticIndexParams();

    foreach ($data['body'] as $key => $param) {
      foreach ($completion_fields as $completion_field) {
        if (isset($param[$completion_field])) {
          $data['body'][$key][$completion_field] = array_shift($param[$completion_field]);
        }
      }
    }
    $event->setElasticIndexParams($data);
  }

  /**
   * Event listener.
   *
   * Called on elasticsearch_connector.prepare_index_mapping event. This method
   * sets the field type as "completion" on fields that are defined as
   * completion fields in Search API.
   *
   * @param \Drupal\elasticsearch_connector\Event\PrepareIndexMappingEvent $event
   *   The event.
   *
   * @see \Drupal\elasticsearch_connector\ElasticSearch\Parameters\Factory\MappingFactory::mappingFromField()
   * @see chemdb_search_setup_search_api_field_type_mapping_alter()
   */
  public function doPrepareIndexMapping(PrepareIndexMappingEvent $event) {
    $index = $this->loadIndexFromIndexName($event->getIndexName());
    $params = $event->getIndexMappingParams();
    foreach ($index->getFields() as $field_id => $field_data) {
      if ($field_data->getType() === 'completion') {
        $params['body']['properties'][$field_id]['type'] = 'completion';

        // Use the standard analyzer instead of the default simply analyzer.
        // @link https://www.elastic.co/guide/en/elasticsearch/reference/6.8/analysis-standard-analyzer.html
        $params['body']['properties'][$field_id]['analyzer'] = 'standard';
     
      }
    }
    $event->setIndexMappingParams($params);
  }

  /**
   * Calculates the Index entity id form the event.
   *
   * @param string $index_name
   *   The long index name as a string.
   *
   * @return string
   *   The id of the associated index entity.
   */
  private function getIndexIdFromIndexName($index_name):string {
    $last_underscore = strrpos($index_name, '_');
    if ($last_underscore) {
      $index_name = substr($index_name, $last_underscore + 1);
    }
    return $index_name;
  }

  /**
   * Loads the index entity associated with this event.
   *
   * @param string $index_name
   *   The long index name as a string.
   *
   * @return \Drupal\Core\Entity\EntityInterface|null
   *   The loaded index or NULL.
   */
  private function loadIndexFromIndexName($index_name):?EntityInterface {
    $index_id = $this->getIndexIdFromIndexName($index_name);
    $index_storage = $this->entityTypeManager->getStorage('search_api_index');
    return $index_storage->load($index_id);
  }

}