Add inputter plugins

Created on 9 May 2024, 6 months ago
Updated 11 May 2024, 6 months ago

Problem/Motivation

Currently there are only two ways to input date - the base mode and the advanced mode (tokens). This was very good for generic solutions and base mode is still the best input type for complex fields like file, image etc, but there are more complex inputs that you might want to take when the input is text.

Think of the following scenario - you have a commerce website with possibility to review products based on user accounts with personal data. You have a product entity and you want to know the general sentiment based on the textual output from women aged 30-40. You need custom inputters.

The first that I will create, in an external module, will be:

  • Views inputter - This makes it possible to solve the above, it takes a view and uses tokens to fill in arguments for the view and takes the output of the view as input.
  • Rendered Entity inputter - This takes an entity fetched using tokens and uses a renrdered entity in a view mode of your choice and takes the output as input.
  • Embeddings - research which embeddings module to require, but the general idea is that you can write natural language what you want to use as context from your own database.

Proposed resolution

Add the rule.
Add the module with the 2-3 rules.

📌 Task
Status

Active

Version

1.0

Component

Code

Created by

🇩🇪Germany marcus_johansson

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @marcus_johansson
  • 🇱🇹Lithuania mindaugasd

    first that I will create, in an external module, will be:[...]

    It is better for you to create external module, or work with https://www.drupal.org/project/aiprompt → ?

    Views inputer:

    • aiprompt have views, except tokens support overall is in design stage. I spend quite a lot of thought on the tokens, I did many designs and prototypes, but don't have a final design how it will look like. In case aiprompt module would be used, I could create various issues for tokens support and continue from there.

    Rendered Entity inputter:

    Embeddings:

  • 🇱🇹Lithuania mindaugasd

    AI prompt engineering → module is storage independent (it already have different content and config entity storages), so it could have new "field" storage too. Having prompt inline as a field would neatly integrate within AI interpolator.

    But AI prompts are multipage and feature rich (for example, AI developer assistant → prompt UI will be huge, and don't fit in a small space, works better a separate page. Or another example - I attached views prompt segment picture. I made it highly feature rich taking a lot of space. And integrating tokens next will take even more space).

    Since I put a lot of work and thought into this, I want aiprompt to grow and I would love AI interpolator to use aiprompt instead.

    Options:

    • reference prompt (simplest)
    • reference prompt with inline_entity_form (and upgrade everything as ajax instead of separate pages)
    • create field storage for aiprompt and have it inline as field (and upgrade everything as ajax)
    • organize prompt in a modal
    • organize prompt in a sidebar →

    Maybe sidebar option would be best UX, as well as, simplest to implement?

  • 🇩🇪Germany marcus_johansson

    Lets do it like this, because I need to get to know the AI Prompt Engineering before I can see if it will work (mostly from UX point of view), I will start with just integrating the actual plugin manager and the views plugin because:

    1. I actually need it the view context in production rather soon, the two others were just ideas for now.
    2. I need an actual plugin working while working on how the plugin manager should work, otherwise I will end up missing things in the interface.

    The most important thing is that it doesn't create the typical Drupal crazy UX where you have to setup something that could be setup on one form on 4 different pages and that the fields where you set it up are actually intuitive - for Views it should for example, I think, be a select field with the view wanted and then arguments showing up when you select them, where you can insert hardcoded values or tokens and a prompt field where you can use that token/placeholder.

    If that is possible and doable in reasonable time and specially if its better, I'm very happy not to have to do it. Specially if you thought about stuff that I didn't really think about offering, like for instance prompting like this (views example still):

    How does this text:
    
    {views_results_1}
    
    differ from this text:
    
    {views_result_2}
    

    and probably 1000 other things you already though about.

    I think anyway it will offer things that any of the suggestions above doesn't, if it can combine certain of those roles or if it can also bind different Drupal internal AI tools together. I can also do the plugin integration, so you don't have to put effort into that, when I understand how it works.

  • 🇱🇹Lithuania mindaugasd

    The most important thing is that it doesn't create the typical Drupal crazy UX

    Agree, this is very important.

    Views it should for example, I think, be a select field with the view wanted and then arguments showing up when you select them

    I tried doing views implementation in the best way I ever seen in Drupal. There are actual detail explanations how things work. And "Views URL parameters" field is unique (never seen before) and extra powerful, basically enabling to override any config (not just arguments).

    "Selecting arguments" as you write would be unique too, never seen this before. I think it is not a requirement at this moment, maybe in the future simplifying for people new to Drupal. But this intuitiveness in general, can be an abstracted feature of Drupal, smth part of https://www.drupal.org/starshot → . Like simplifying views embed within Layout Builder, and by extension simplifying for all modules.

  • 🇱🇹Lithuania mindaugasd

    For next days my tasks are to explore:
    1) Fastest and best UX way to integrate AI Prompt with AI interpolator
    2) Passing variables to AI prompt
    3) Tokens for views, and tokens in general

  • 🇱🇹Lithuania mindaugasd

    I can also do the plugin integration

    After we make action plan clear, we will be able to split the work to achieve features you need.

  • 🇱🇹Lithuania mindaugasd

    There is fundamental difference between how AI interpolator writes prompts today, and how it would be writen using AI Prompt:

    This

    How does this text:

    {views_results_1}

    differ from this text:

    {views_result_2}

    is achieved by adding/arranging "prompt segments", like this:
    [text segment]
    [views segment]
    [text segment]
    [views segment]

    Its clicking in UI, instead of writing in a textfield. You can try it with combination with AI chat user interface → already and tell how you feel about this.

    I personally like it. I think its intuitive. You don't need to know the names of tokens, and just click, drag-and-drop, disable/enable segments etc. Disabling segments allows to have many drafts/variations of the same prompt for example. This is made for chat, where prompt is expected to be modified often.

  • 🇱🇹Lithuania mindaugasd

    For next days my tasks are

    Actually, how much time do I have? In case that is not weeks, then I pass this on, and you can proceed with your original plan without waiting for me.

  • 🇩🇪Germany marcus_johansson

    No, its a week or two before I have the plugin system at least. And I would probably need another week or two to do the views plugin.

    There is one other thing, but I don't think the plugin system will play a big role there anyway, this ticket ✨ Add events for important actions Active is prepared and has higher priority. It will enable a lot of external modules to do different powerful stuff, the main reason I want a preFieldSelection event is so that I/we can setup an AI evaluation platform, where for instance testing 20 different prompts in combination with different temperatures etc can be evaluated.

    I think anyone doing that would use the base field or token field anyway since evaluation prompts shouldn't be complex by their nature, but if there is a way to hook that action into any action you have for changing the prompt its good to have in mind.

  • 🇱🇹Lithuania mindaugasd

    Ok, I will know I have more time.

    do different powerful stuff

    Growing to be quite complex UX.

    Theoretical redesign:

    $entity1 = clone $entity;
    $entity1->setTemperature(5);
    $entity1->run();
    
    $entity2 = clone $entity;
    $entity2->setTemperature(6);
    $entity2->run();
    
    $entity3 = clone $entity;
    $entity3->setTemperature(7);
    $entity3->run();
    
Production build 0.71.5 2024