How to configure tokenizer

Created on 13 February 2025, about 2 months ago

I believe I successfully installed and enabled the normalizer module, but am unsure how to enable tokenizer.

It sounds like I'd like to use the TinySegmenter, which says "no additional installation required", but I don't see any corresponding options in the Search API processor settings. I've attached a screenshot below.

Do I need to download and add a library, or something else I'm missing here?

Similarly, how will I disable the default tokenizer once I get the TinySegmenter working correctly?

Thanks!

πŸ’¬ Support request
Status

Active

Version

1.0

Component

Documentation

Created by

πŸ‡ΊπŸ‡ΈUnited States w01f

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @w01f
  • πŸ‡ΊπŸ‡ΈUnited States w01f

    Update: Disregard - the new default module filter is too smart for my own good and just wasn't showing the submodules. I believe it's working correctly now - thanks again!

  • πŸ‡―πŸ‡΅Japan u7aro Japan

    Hi w01f,

    Thank you for your feedback. I’m glad you were able to find the submodule.

    Just in case, I’ll attach a screenshot of the recommended minimal processor configuration.

    I hope this module will be useful for your project.

  • πŸ‡―πŸ‡΅Japan u7aro Japan
Production build 0.71.5 2024