trouble indexing new/changed content

Created on 20 August 2021, almost 4 years ago
Updated 15 March 2023, about 2 years ago

Problem/Motivation

I have search_api happily running on my Drupal 8 (8.9.18) site, except for one issue: new nodes or changes to existing nodes only make their way into the index if I do a full rebuild of all tracking and rebuild of the index. In other sites that I've worked with, search_api automatically picked up such changes without requiring a full index rebuild.

When I look in the search_api cron logs, I see no problems. Cron-generated runs appear to terminate normally. If I rebuild the index completely, the log ends up showing some warnings about image styles, but no errors.

Any advice about where I might look to try to track down what's failing here? Thanks.

Steps to reproduce

Proposed resolution

Remaining tasks

🐛 Bug report
Status

Active

Version

1.20

Component

Framework

Created by

🇺🇸United States mefron

Live updates comments and jobs are added and updated live.
  • Needs tests

    The change is currently missing an automated test that fails when run with the original code, and succeeds when the bug has been fixed.

Sign in to follow issues

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • 🇯🇵Japan ultrabob Japan

    I'm sorry to have not replied yet. I haven't forgotten about this, it has just been difficult to find time to investigate. I have a ticket open for it in and intend to get to it before too much longer.

  • 🇹🇷Turkey orkut murat yılmaz Istanbul

    It still exists with the version 8.x-1.34.

  • 🇯🇵Japan hktang

    This case is specific to unpublished items with moderation state change, as described by @ultrabob on #7. And may not be related to the issue reported by the OP.

    Also, I should point out that our search index include unpublished content, which has multiple moderation states (draft, A, B, C... etc). We use a filter in the view configuration page to exclude unpublished content with the moderation state "draft" (but not others).

    I have observed the following:

    • The index includes all drafts, as we wanted ✅
    • The index updates immediately after saving a draft ✅
    • The index is correctly updated by entityUpdate() call, as evident by its timestamp ✅
    • The moderation state of the indexed item updates if the value is the same as that of the original (e.g. from A to A). ✅
    • The moderation state of the indexed item reverts to the initial state (draft) if the state values are different after updating (e.g. from A to B). As a result, the item did not show up in search, as we would want it to. ❌

    Workaround

    When I turn off “Index items immediately”, the moderation state of the item updates correctly after cron runs. I also reduce the cron frequency to 1 minute so that users can see the item more quickly.

  • 🇯🇵Japan hktang

    On a fresh install, I can no longer reproduce the issue described earlier. The search index updates _almost_ as expected, with a caveat:

    In order for the search index to update the moderation state value correctly, "Default revision" must be checked for any custom moderation state. Otherwise:

    If the item was Published and a user sets the moderation state to something else (such as "awaiting greenlight"), the search index will not update the moderation state. This can be problematic.

    However, as the fix seems to be in setting "Default revision" for a state in the Workflow configuration, so I wonder how the Search API module can take that into account.

    Sorry for going off topic, as my comment again is related to moderation states.

    Tested on:

    - Fresh install of Drupal 10.4
    - Install meilisearch (ddev)
    - Install and enable search_api module (1.38)
    - Install and enable search_api_meilisearch module (2.0)
    - Enable Workflows, Content Moderation
    - Set up basic workflows:
    - Moderation state: Awaiting greenlight
    - Transition: All states to "Awaiting greenlight"

Production build 0.71.5 2024