Add new Search API processor plugin for more generic data indexing

Created on 13 October 2022, about 2 years ago
Updated 9 September 2024, 2 months ago

Add a new Search API processor plugin (ProcessorPluginBase) that allows the rendered output of each meta tag to be indexed, separate to the existing typed data plugin that only indexes values from the optional entity field.

The plugin should list all available meta tags with checkboxes (or a multi-value select list?) to control which ones are indexed, with the items grouped.

Original issue summary

In https://www.drupal.org/project/metatag/issues/2901039 , Search API integration was added to the Metatag module to allow for the indexing of entity metatags to a Search API backend.

But, this is not working as expected.

The issue is that the current Search API integration with Metatag module relies heavily on this computed field:
https://git.drupalcode.org/project/metatag/-/blob/8.x-1.x/src/Plugin/Fie...

The main issue with this implementation is this logic done in a preSave() method, that checks if the metatag values at the node level differ from the metatag defaults defined for the content type. This is an issue because if the node level metatags are the same as the defaults, the metatag value is not saved to the metatag computed field for the entity. But, it seems that the Search API integration is using that metatag computed field to get the metatags and their values to pass along to the search API index during indexing.

When this happens, you will notice that backer class of IndividualTag will not trigger any XDebug breakpoints nor index any of the metatags as expected, in the scenario that the node level metatags are the same as the defaults.

If the node level metatag values differ from the defaults, then the breakpoints will work and the metatags will be indexed. If the node field has the same value as the default configuration, the value is removed from what is stored for that node.

The Metatag field storage doesn't store the system defaults in the field, so that the defaults can be changed later if needed without having to worry how to update all of the per-entity changes.

We might need to rework the Search API integration to not be driven by the Field API field itself but rather just work from supported entity types via a separate property.

Feature request
Status

Needs review

Version

2.0

Component

Integration with other module

Created by

🇺🇸United States joshua.boltz

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

Not all content is available!

It's likely this issue predates Contrib.social: some issue and comment data are missing.

  • 🇨🇭Switzerland berdir Switzerland

    Drive-by comment: We implemented a custom processor that skips entities that have noindex set on metatags, because in pretty much all cases that we've seen, content that isn't indexed publicly should also not be indexed in your internal search, things like thank you pages and so on.

    Probably doesn't really fit in this issue but I could look into creating an issue with our code if there's interested for that.

  • 🇺🇸United States DamienMcKenna NH, USA

    That's an interesting idea, it kinda skips the need for search_api_exclude by using the checkbox from Metatag.

    I'd be happy to include that, if you'd be willing to throw it into a new issue. Thank you!

  • 🇺🇸United States benabaird

    Thanks for the patch, it's working nicely. I did find an issue with the "all metatags including defaults" option, it looks like the config value isn't being read. Attached a patch and interdiff.

  • Status changed to Needs review over 1 year ago
  • Open in Jenkins → Open on Drupal.org →
    Core: 9.5.5 + Environment: PHP 7.3 & MySQL 5.7
    last update over 1 year ago
    372 pass, 2 fail
  • 🇺🇸United States DamienMcKenna NH, USA
  • The last submitted patch, 25: metatag-n3315049-24.patch, failed testing. View results
    - codesniffer_fixes.patch Interdiff of automated coding standards fixes only.

  • Status changed to Needs work over 1 year ago
  • A big thank you to everyone who's contributed to this issue! I (perhaps foolishly) upgraded directly from v1.22 to v2.0.0 and ran into a *lot* of search-api-integration issues with metatag fields, and this issue seemed to be the most current one to help me get back on track.

    I'm not sure if this is still the right place to follow for official post-v2.0.0 search-api integration? Issue #3326104 seems to suggest that this current issue will be used to re-implement search-api integration in 2.x, but the recent patches and activity in this issue seem to be focused on the 1.x branch, and consequently the patches don't work as-is with the 2.x branch code. But perhaps that will be coming with future activity?

    Regardless, for what it's worth I adapted the most recent patch from @benabird (metatag-n3315049-24.patch) to work with the 2.x branch. It seems to work as intended in a Drupal 9.5.10 (PHP 8.1.x) environment, but I suspect it has the same test failures that the #24 and previous patches triggered.

  • Open in Jenkins → Open on Drupal.org →
    Core: 10.1.x + Environment: PHP 8.1 & MySQL 8
    last update over 1 year ago
    113 pass, 2 fail
  • 🇺🇸United States Nuuou Lincoln, NE

    Ran into a similar situation as above, with a site that was previously using the old patches. Need to upgrade that site to the "new" metatags/Search API methodology eventually, but not today haha.

    I re-rolled this patch for Metatag 2.0.2 support, since #28 doesn't apply anymore.
    I did not include the README edits, since this approach shouldn't be supported anymore anyhow.

  • 🇬🇧United Kingdom 8bitplateau

    @nuuou what is the 'new methodology' you mention ? how do we achieve this without the patch ?

  • 🇺🇸United States Nuuou Lincoln, NE

    Y'know, I think was wrong about the "new methodology". There are quite a few related issues to this, I thought this was solved in a different way!

    Looks like this is the issue to follow going forward on this. [#3326104]

    Lemme re-roll that patch again with the README.
    Also, tagging this to 2.x since that's the recommended version now.

  • 🇺🇸United States Nuuou Lincoln, NE
  • Merge request !140Resolve #3315049 "Add new search" → (Open) created by Nuuou
  • Pipeline finished with Failed
    2 months ago
    Total: 312s
    #278170
  • 🇺🇸United States Nuuou Lincoln, NE

    Created an MR based on the above, against 2.0.x.
    Patch file created for folks who want that for Composer patches too.

  • Status changed to Needs review 2 months ago
  • 🇺🇸United States Nuuou Lincoln, NE
  • Pipeline finished with Failed
    2 months ago
    Total: 302s
    #278187
  • Pipeline finished with Failed
    2 months ago
    Total: 367s
    #278239
  • 🇨🇭Switzerland idflood

    Thanks @nuuou, the merge request worked perfectly for my use case. I was trying to index keywords, and this patch made it work nicely : )

Production build 0.71.5 2024