Named entity recognition and automatic categorization

Created on 2 November 2023, about 1 year ago
Updated 16 November 2023, about 1 year ago

I've seen the video https://workflows-of-ai.com/workflow/categorize-content.

Is there any way to do more "advanced" categorization.

Say I have 2 vocabs: Persons, Organizations

In the Persons vocab, each term is a name of a person.

In the Organizations vocab, each term is a name of an organization.

LLM's can do Named-entity recognition, so they can list all persons mentioned in the text in one list, and all the organizations in another list.

But how can AI Interpolator help us store this information in the corresponding vocabularies as terms?

Thanks for these modules!

Feature request
Status

Closed: won't fix

Version

1.0

Component

Code

Created by

🇩🇰Denmark lennart

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @lennart
  • 🇩🇪Germany marcus_johansson

    Hi Lennart,

    There is no simple way at the moment for the interpolator to do this in one request for two fields. Currently it only supports many-context-to-one-field or one-context-to-one field.

    What you can do is either:

    • that you create two different taxonomy fields for each of the vocabularies and then add a rule for each. This of course unfortunately means almost twice the cost for OpenAI queries.
    • or put both vocabularies in the same field

    Hope that answers the question.

  • Status changed to Needs review about 1 year ago
  • Status changed to Active about 1 year ago
  • 🇩🇰Denmark lennart

    Thank you!

    Is this something OpenAI's Function calling could help achieve down the line?

    And will AI Interpolator eventually be able to "parse" structured data returned as json?

  • 🇩🇪Germany marcus_johansson

    So it already work completely by parsing structured JSON data, basically anything that gets generated is returned as JSON. That's how it can work with complex fields like custom fields (see: https://workflows-of-ai.com/workflow/custom-complex-fields).

    This is superior to function calling for a no-code GUI solution, since the function calling needs you to do a very good spec of inputs and outputs. In this case we use one-shot learning to generate the (JSON) output, which is easy enough for an end user to generate and good enough to handle complex data.

    This won't however solve the problem of wanting to create multiple fields with the same prompt. That one we have a solution for in our company, but we are not ready to open source it yet.

  • 🇩🇰Denmark lennart

    Thank you very much for the detailed answer, Marcus.

  • Status changed to Closed: won't fix about 1 year ago
Production build 0.71.5 2024