SearchApiException while adding Views handlers for field during profile install of index config

I resolved this issue by moving the config in question from config/install to config/optional.

Just as a note: This will never affect the fields in the index or anything like that, the errors only occur in the Views integration. Until you clear the index, search views might not work correctly, but other than that the error is harmless. Most importantly, it shouldn’t damage your site in any persistent way. Just clear the Views cache after installation and all should be fine.
However, if you have found a workaround that avoids the errors altogether, that’s of course even better.

I fear that I cannot really offer any more help on this, as this error looks to be very site-specific. I know for a fact that a lot of sites include search indexes in their installation profiles, without any such errors. Still, I’m leaving this open so people can find your hint about using config/optional, and maybe provide their own suggestions or insights.

Also encountered this error.

The cause is similar to what was discussed in the now closed https://www.drupal.org/project/search_api/issues/2939010#comment-12464723 →

TLDR from that issue is that the index depends on the field storage not field instances, where the field instances are what is needed.

So when the index is created, the field storage has been created but not necessarily the field instances.

Moving to optional is sensible as that runs after all the config provided by your module / profile.

The other option I briefly tried while I was debugging this was manually adding the field instances as dependencies to the index, but I like your approach of using the optional folder better.

Hey guys. So.. Do you have some news about this topic? I don't find the solution in any place.

I landed here via Google after encountering the same error. In my case:

I'm working on a Github action to build/test the site install prior to an automated deployment. I'm running:

drush si --existing-config

to install the site from its existing configuration. My action fails due these errors and I don't believe I can really place them into /optional because I'm installing directly from our config/sync folder (not from and install profile.)

@ broeker: Probably, manually (or via an alter hook/event) adding the field instances as dependencies to the index is your best bet here. Unless you delete fields on your site, this change should have no effect whatsoever on the running site, but might resolve this problem when deploying.

I've encountered on this same issue when I am trying to install a site from a profile.
Below is the the search API index

langcode: en
status: true
dependencies:
  config:
    - field.storage.node.field_about
    - field.storage.node.field_search_exclude
    - field.storage.node.field_suitable_for
    - search_api.server.database_search
    - core.entity_view_mode.node.search_index
  module:
    - node
    - search_api
id: content
name: Content
description: 'Default search index using the database search server'
read_only: false
field_settings:
  field_about:
    label: About
    datasource_id: 'entity:node'
    property_path: field_about
    type: integer
    dependencies:
      config:
        - field.storage.node.field_about
  field_search_exclude:
    label: 'Search exclude'
    datasource_id: 'entity:node'
    property_path: field_search_exclude
    type: boolean
    dependencies:
      config:
        - field.storage.node.field_search_exclude
  field_suitable_for:
    label: 'Suitable for'
    datasource_id: 'entity:node'
    property_path: field_suitable_for
    type: integer
    dependencies:
      config:
        - field.storage.node.field_suitable_for
  node_grants:
    label: 'Node access information'
    property_path: search_api_node_grants
    type: string
    indexed_locked: true
    type_locked: true
    hidden: true
  rendered_item:
    label: 'Rendered HTML output'
    property_path: rendered_item
    type: text
    configuration:
      roles:
        - anonymous
      view_mode:
        'entity:node':
          article: search_index
          audience: search_index
          casestudy: search_index
          page: search_index
          person: search_index
          topic: search_index
  status:
    label: Published
    datasource_id: 'entity:node'
    property_path: status
    type: boolean
    indexed_locked: true
    type_locked: true
    dependencies:
      module:
        - node
  title:
    label: Title
    datasource_id: 'entity:node'
    property_path: title
    type: string
    dependencies:
      module:
        - node
  type:
    label: 'Content type'
    datasource_id: 'entity:node'
    property_path: type
    type: string
    dependencies:
      module:
        - node
  uid:
    label: 'Authored by'
    datasource_id: 'entity:node'
    property_path: uid
    type: integer
    indexed_locked: true
    type_locked: true
    dependencies:
      module:
        - node
datasource_settings:
  'entity:node':
    bundles:
      default: true
      selected: {  }
    languages:
      default: true
      selected: {  }
processor_settings:
  add_url: {  }
  aggregated_field: {  }
  content_access:
    weights:
      preprocess_query: -30
  entity_status: {  }
  html_filter:
    weights:
      preprocess_index: -15
      preprocess_query: -15
    all_fields: false
    fields:
      - rendered_item
    title: true
    alt: true
    tags:
      b: 2
      em: 1
      h1: 5
      h2: 3
      h3: 2
      strong: 2
      u: 1
  ignorecase:
    weights:
      preprocess_index: -20
      preprocess_query: -20
    all_fields: true
    fields:
      - rendered_item
      - title
      - type
  language_with_fallback: {  }
  rendered_item: {  }
  tokenizer:
    weights:
      preprocess_index: -6
      preprocess_query: -6
    all_fields: true
    fields:
      - rendered_item
    spaces: ''
    ignored: ._-
    overlap_cjk: 1
    minimum_word_size: '3'
  transliteration:
    weights:
      preprocess_index: -20
      preprocess_query: -20
    all_fields: true
    fields:
      - rendered_item
      - title
      - type
tracker_settings:
  default:
    indexing_order: fifo
options:
  cron_limit: 50
  index_directly: true
  track_changes_in_references: true
server: database_search

This error occurs because the field storage e.g. field.storage.node.field_about does not have dependency on the content type or the bundle.
So during site install, Search API index using the field_about persuming that exist. However, its storage do exist but its bundle/content type does not exist.

Just to confirm the above, I added the additional dependency in the search api index

- field.field.node.article.field_about

Now the above error is gone. I haven't digged into it. If adding the dependency on the field works. How about adding the dependency on the field itself.

Thanks for your analysis, that really helps to understand what might cause this!

However, including only the field storage instead of the individual field instances was a deliberate choice we made, in #2541206: Consider field storage dependency removal on Index → . The reasoning is quickly given in #2472917: Make index configuration depend on field storage not instances → : Internally in the Search API we don’t care about the field instances at all, and we want to avoid having to react if a field instance is deleted (if the field storage remains, the field can remain on the search index). Furthermore, if a site has different content types but defines the same fields (e.g., common ones like field_tags) it’s nice to be able to just reuse a search index from another site.

On the other hand, of course, the issue reported here seems to be not too rare, it proves that some of our internals do depend on actual field instances being present (even if we don’t care which ones) and the use case is undoubtedly more conventional than wanting to import a search index to a slightly different site (which will always only work for very specific differences anyways). So, there is a strong argument for changing this, too.
Complicating this further, though, is that we currently have no way internally of declaring a particular field’s dependencies as “optional”. Therefore, removing a single field instance would always lead to the associated field being removed from the search index, even if the field is still present on other bundles. See the attached patch (which is expected to fail testing) for an illustration. Since that behavior would be a clear regression, we’d have to find a way around it, which wouldn’t be possible without changes to our framework/API as far as I can see.
A thorny issue, either way. Maybe we’re better off leaving things as-is and requiring people that run into this problem to manually add the required dependency on the field instance config.

Additional input very much welcome!

Also, side note: It’s preferred to use a YAML file attachment for posting configuration instead of including it in the text of the post.

The last submitted patch, 10: 3352150-10--add_field_instance_dependencies.patch, failed testing. View results →

I was working on a workaround for this before I stumbled upon this issue, and have subsequently pushed into an issue fork for reference.

Mine was caused because I had multiple custom modules which handled different functionality. And the "base" module had the search api index as well as the base field storage entities, but not the individual bundles (since those would be handled when the optional modules were installed later down the step).

Given that the issue only affected site installs for me, I added a condition to only load the base field storage properties during that state. However if it makes sense to remove that condition, then that's perfectly fine too.

I have the same issue while importing the configuration.

The rank field is created via weight module which can contain negative numbers.
I thought it was related to negative number.s Even though the problem is solved (it is fixed here - #3484686 💬 Support: Indexable negative number Active .), I still get an error while importing the config.

I don't know if the problem is related to the negative numbers or weight module, but the problem with importing the configuration still persists.

 [error]  Drupal\search_api\SearchApiException while adding Views handlers for field Content » Rank on index Default content index: Could not retrieve data definition for field 'Rank' on index 'Default content index'. in Drupal\search_api\Item\Field->getDataDefinition() (line 487 of /var/www/html/d10Site/web/modules/contrib/search_api/src/Item/Field.php).
 [error]  Drupal\search_api\SearchApiException while computing Views data for index Default content index: Could not retrieve data definition for field 'Rank' on index 'Default content index'. in Drupal\search_api\Item\Field->getDataDefinition() (line 487 of /var/www/html/d10Site/web/modules/contrib/search_api/src/Item/Field.php).