🇪🇸Spain @Nikro

Picking this up - but so far can't reliably set-up MCP connection (opencode <-> drupal) - still working on that.

Picking this up

Picking this up for tmw.

Great job, thanks for addressing those! :)

Hey, thanks for all the hard work, this looks really good.

Note: I didn't dive deep into the code, but running it locally, I like how it works & looks.

I'd like others to sample this idea and give some feedback, we can then, based on that feedback, polish-up some things and present it as a standalone module. It's a solid flow when someone wants just to install and configure an AI provider + an AI Recipe.

Sure, we don't have vectordb selection yet, or some other things - but those are smaller feature requests, basics things are in-place.

@abhisek - there voices have to be split into 2 groups: ones are only used with english, the others, with arabic :) [i.e. you can't use arabic voice with english model] - was that the issue you had?

Looks good to me, models showed up as expected.
:)

Hey, great catch :)
I actually mostly followed the implementation of ai_provider_openai - https://git.drupalcode.org/project/ai_provider_openai/-/blob/1.2.x/src/P...

But, I guess that's not ideal, people can upload any format really.

--

Also, while I was on vacation - PlayAI was acquired and shut down all their services, they moved to a new model. Baked in the fix 🤷
https://console.groq.com/docs/text-to-speech/orpheus - this is their new model, I adopted the code.

Merged, closing.

Merged - and because the errors/warnings in the pipeline were bugging me - opened https://www.drupal.org/project/ai_search_block/issues/3566247 📌 Get CI/CD back to no errors Active as a follow-up.

Okay now it's good again :)
https://git.drupalcode.org/project/ai_search_block/-/commit/b9aba9ba6088...

But someone needs to review to make sure I didn't break anything. Any quick review of the module would be good.

nikro → created an issue.

@joevagyok

Hey, left some comments in the MR.
I fixed the namings locally and yeah the ai_ckeditor pop-up shows up, does the fix, but doesn't replace it (probably as intended, at least in the code, for now).

Okay, I indeed forgot to adopt the tests to our new UI changes.
Now it should be fine, also sorted those annoying phpstan errors (apparently those hook-based errors are popular).

Alrighty - please use the ai_explorer (make sure you have latest 1.2.x-dev of ai module - as it relies on a fix that was just pushed yesterday).

Pushed the minimal changes.

This works well (I know it's a small change, but I never tested LiteLLM with image generation).
I proxied openai/gpt-image-1-mini as:

  - model_name: gpt-image
    litellm_params:
      model: openai/gpt-image-1-mini
      api_key: os.environ/OPENAI_API_KEY

It worked well.

So, a module blueprint / blueprint module? 🤔

Picking this up, thanks @joevagyok!

Uploading explainer for Drupal AI showcase.

→

Here's also an explainer:

If enabled (and it's important: if it's bi-directional encoder, meaning that we can send a special query-based embeddings).

Great find! I think I recall I had the same issue on one of our projects. Picking this up.

I'll pick this up and will test against our providers - I wonder if this fix will auto-magically work through-out everywhere.

Hi @anaconda777, I got you.

Hm, for us to be able to reproduce your issue, can you provide a bit more details: what version of ai / ai_agents are you using?
Any other detail will be super useful - i.e. what LiteLLM version did you use (because every version might have different responses) - I mean LiteLLM proxy service.

Alright alright, did some testing here and there and it seems to work well.

I'd like someone with DS's (or similar) - E5 Large Instruct embedding model to test this bit.

nikro → created an issue. See original summary → .

nikro → created an issue.

nikro → created an issue.

Done, the form compatibility - should be good now.

Hey, nice, great start!

I'd suggest to make sure we're basing most of the stuff on >= 1.2.x for the ai.module, but the rest is a good way to start.

Done, made a new alpha1 release with this in it.
Thanks!

I'm picking this one - because the main branch moved, and I need to make sure this one lands nicely, will spend a few minutes / hours testing this one out and polishing it a bit as well.

I'm picking this up again, will do an update PR today.

nikro → created an issue.

So last updates were as following:

- Kept unified terminate subscriber.
- But, removed the deepchat attempted integration - as it requires way more changes than I thought - especially SDK changes. I'll leave it for now as a separate ticket.

After anlyzing this live with Frederik Wouters, we discovered that it misbehaves when used under Deepchat.

More fixes done:

Termination Flow Consolidation

Removed duplicate KernelEvents::TERMINATE handler from langfuse_ai_logging. The root LangfuseSyncSubscriber in web/modules/custom/langfuse/src/EventSubscriber/LangfuseSyncSubscriber.php now exclusively finalizes traces, updates metadata (final timestamp/output), clears the active trace, and runs syncTraces().

DeepChat-Friendly Trace Management

• Introduced LangFuseTraceSessionStorageInterface with a session-backed implementation (LangFuseTraceSessionStorage). It tracks thread→trace IDs so multi-request workflows (DeepChat’s “submit” + “tool follow-up”) can re-use the same trace.
• The AI logging subscriber now injects this storage instead of reaching directly into the request stack. Whenever a generation finishes without pending tool calls, it clears the thread mapping; otherwise, it leaves it in place.
• LangfuseSyncSubscriber::finalizeActiveTrace() checks whether the current trace still has mapped threads. If so, it skips finalization and syncTraces(), allowing the next request to resume the same trace. Only when all thread mappings are cleared does it finalize and sync.
• DeepChat’s two-step requests therefore stay within one trace: request #1 leaves the mapping (because tool calls were requested), the terminate event defers syncing, request #2 re-uses the trace and clears the mapping once complete, and only then does termination finalize and send the trace to LangFuse.

More changes - as previous ones ignored some things:

I had to split the ai logging sub-module into 3 separate sub-modules: general ai, ai agents and search - because they actually depend on separate module:

- The trace manager now exposes high-level lifecycle methods (scheduleToolParent, claim/releaseToolParent, start/endDelegation, enter/leave/currentSpanContext). Also nuked some span-related tracking that we implemented earlier.
- LangFuseAiLoggingSubscriber registers active generations per thread ID, schedules tool parents when tool calls occur, and resolves parents via delegation/search contexts instead of scanning observations.
- LangFuseToolSpanSubscriber claims the queued generation parent, starts delegation when another agent is invoked, pushes/pops AI Search contexts, and releases claims using releaseToolParent() rather than a custom handle class.
- LangFuseSearchSpanSubscriber wraps Search API queries with span contexts so embeddings can attach to the retrieval span; contextual comments explain each step.
- LangFuseAgentLifecycleSubscriber documents that it registers parent runner relationships for delegation.

This structure keeps AI Agents and AI Search logging optional submodules (wiring only when those contrib modules are enabled) while preserving a single runner-context system the rest of the code can rely on.

nikro → created an issue.

nikro → created an issue.

We talked via slack - things seem to be fine now, closing this issue.

Hey, good start, I like how things work. 🎉

Here are some things we need to think about and build on top of what we have so far:

1. After the final form submit -> it redirects back to /ai-provider. Message states:
   “Litellm configuration has been saved successfully.” - but I see the selection again.
   It might not be an actual issue but just unfinished implementation as it's a concept. So, just a heads up.
2. Unfinished flow - Would be nice to continue the concept and add 2–3 recipes to select (next step).
   This will solidify the PoC as it'll prove that we not only can download the needed modules (as a dependency tree) -
   but we can also unpack all the right recipes (configs, etc).
   
3. Actual flow - would be nice instead of having 2 independent forms, to make a wizard-like behavior (multi-step form),
   where user can go “< Back” and “Next >”. As a wizard also add a touch i.e. Step X/Y: Configure AI Provider…
   
4. Final step - and then, after the recipe will be pulled - we need to add the final step, maybe we want to present some kind of summary
   of what happened as final (non-submittable) step: installed XYZ, created key, set defaults, etc. - the depth/complexity or simplicity of
   the final thing is mostly up to you.

Code-wise:

• Example folder + Interface - I think that was left-over from previous implementation? Might need some clean-up.
• AiProviderBatchOperations.php - use StringTranslationTrait and t()'s - gotta clean that up.
  And some other things here and there (full-name-spaced objects/methods inline, etc).
  
  • In static batch callbacks prefer TranslatableMarkup over t():
  Translating strings →
• Provider Config Form (aka: step 2) - we're copying elements, maybe we can use Subforms, i.e.
  Subform API → - to make sure that their validation / submission hooks get triggered (they might do some checks, like check if the key is valid, or other things) - but we also need to make sure that our “key swapping” happens before their validation/submission callbacks. We could leverage #element_validate for that - and do it on element level.
  
• KeyManager - there are some methods which I don't see being used - keyExists, getKeyId..
• KeyCreationException - it's basically same Exception - do we need it?
• $this->t(...)->render() - try to avoid these when using forms, just pass the TranslatableMarkup directly;
• For secrets rely on $form_state->setTemporaryValue() (vs ->set())
  
• AiSandboxPackageManagerInstaller - has a \Drupal::service() call
• AiSandboxPackageManagerInstaller - not sure how good it is to destroy a sandbox we didn't create - maybe reconsider that part?
  Potentially return a warning/msg - Another update was started? or smth along those lines
• AiSandboxPackageManagerInstaller - speaking of sandboxes we didn't create - if WE did create it, maybe use finally {}
  (in case of exceptions) and clean-up the sandbox via ->destroy()?
  

Done, I think I've addressed all of the issues. Let me know.

Picking this up

Le fix:
- After each failed validation we measure the actual density (tokens / characters) of the riskiest chunk, apply the existing variance buffer, and promote that value to the new guard density.
- We rebuild the chunking plan from this guard (character budget, overlap, stride) and scale the next character budget proportionally to the ratio effective_limit / tokens, ensuring the immediate retry targets a chunk size safely under the token limit.
- If the proportional scale does not shrink the budget, we fall back to the 0.9 reduction safeguard. The retry loop still validates only one chunk per attempt and does not perform additional sampling beyond the original probes.

Ran this on >5k docs, no issues.

nikro → created an issue.

Thanks for the review!

Added 1 change and merged it in!

Testing Instructions for TokenAware Text Chunking Implementation

Code Review - Overview

This MR introduces a token-aware text chunking system with custom tokenizer support for Dropsolid AI integrations. The implementation includes:

• New Files Added:
  • src/Service/TokenAwareTextChunker.php - Core chunking service with density-based probing
  • src/Service/TokenAwareTextChunkerInterface.php - Service interface
  • src/Plugin/EmbeddingStrategy/TokenAwareEmbeddingStrategy.php - Embedding strategy plugin
  • src/Tokenizer/DropsolidXlmRobertaTokenizer.php - Custom XLM-RoBERTa tokenizer
  • src/Tokenizer/DropsolidTokenizerDecorator.php - Tokenizer decorator for AI module integration
  • src/Form/TokenizerSettingsForm.php - Admin configuration UI
• Files Removed: Old RecursiveTextChunker and RecursiveCharacterTextSplitter implementations

Prerequisites

Required Access: You must have access to Dropsolid's infrastructure services:

• Dropsolid LiteLLM service (for HTTP-based tokenization)
• Dropsolid Vector database service
• It's assumed Dropsolid.ai (litellm etc) are configured and working...

Testing Steps

Step 1: Configure the Tokenizer

1. Navigate to /admin/config/ai/ai-dropsolid/tokenizer
2. Select "LiteLLM (HTTP) tokenizer – default" from the "Tokenizer mode" dropdown
3. Ensure your LiteLLM settings are configured (should already be set up if you have access)
4. Click "Save configuration"
5. Expected Result: You should see a success message showing demo tokenization with approximately 24 tokens counted for the test string

Step 2: Create Search API Server

1. Navigate to /admin/config/search/search-api/add-server
2. Enter a server name (e.g., "Dropsolid AI Server")
3. Select "AI Search" as the backend
4. In the backend configuration:
   • Embedding engine: Select eu-e5-large-embeddings-selfhosted
   • Tokenizer for token counting: Select Dropsolid AI XLM-Roberta-base from dropdown
5. Expand "Advanced: Embedding strategy configuration":
   • Embedding strategy: Select Token-Aware Embedding Strategy (Separator-Based Chunking)
   • Maximum chunk size: 480 or 500 tokens
   • Overlap: 70, 80, or 100 tokens (your choice)
   • Context content maximum percentage: 25% or 30%
6. Click "Save"

Step 3: Create Search API Index

1. Navigate to /admin/config/search/search-api/add-index
2. Enter an index name (e.g., "Dropsolid Content Index")
3. Select your newly created server from Step 2
4. Choose datasources (e.g., "Content" for nodes)
5. Configure which fields to index (e.g., Title, Body, etc.)
6. Click "Save"

Step 4: Test Chunking with Content Preview

1. Navigate to your index: /admin/config/search/search-api/index/[your_index_name]/fields
2. Click on the "Preview contents to be vectorized" tab
3. Use "Search for an item by title" field to find a content item
4. Select a document with substantial text content (long body field)
5. Expected Results:
   • Text should be chunked into multiple segments
   • Each chunk should be ≤ 480-500 tokens (as configured) - note - it's pretty hard to identify but you can at least see that links should not be cut-off between chunks (please use some content with a ton of URLs)
   • Consecutive chunks should have 70-100 tokens overlap (as configured)
   • Chunks should break on natural boundaries (paragraphs, sentences, words)
   • No URLs should be split mid-way

Step 5: Go wild and index stuff

Process:

• Start indexing content, use 5-10 nodes per run.
• You will have to wait for the job to finish, it will not be blazing fast; but not super slow either - eventually you should be able to see all the content indexed w/o any errors

Technical Notes

• Tokenization Efficiency: The new implementation uses only 3-4 tokenization API calls per document (density probes + optional validation) instead of N calls for N chunks
• Separator Hierarchy: Text splits on \n\n → \n → . → \t → in order of preference
• Density Probing: Samples text at 10%, 50%, and 90% positions to estimate token density before chunking

Troubleshooting

If tokenization test fails:

• Verify LiteLLM service credentials in /admin/config/ai/provider/litellm
• Check that you have network access to Dropsolid's LiteLLM endpoint
• Review recent log messages for API errors

I'll take this over - in a separate branch.

After several hours of research and not finding any PHP library, I concluded there are two possible approaches:

• - Re-use LiteLLM: They provide a /utils/token_counter endpoint specifically for this purpose.
  This approach will be slightly slower since it operates over HTTPS, but it will be accurate.
• - Build and use SentencePiece locally: For those who prefer a faster solution, they can compile and install SentencePiece on their system and configure it as an executable. This will offer the best performance but requires server-level changes.

We will need to implement both options to provide customers with flexibility.

Even more than that - we need to create a custom Tokenizer that is aware of the custom embeddings we use, particularly in the context of the Search API.

This tokenizer should be injected into the system - potentially via a decorator - so that we can override or “hijack” the default tokenizer and replace it with our own implementation when needed.

This approach will ensure that tokenization is consistent with the embedding model’s expectations and that downstream processes relying on accurate token boundaries continue to function correctly.

nikro → created an issue.

Thanks!

Code was pushed into 1.x - once I get a RTBC I'll republish this bit as an alpha1.

Closing - if someone has any complains - feel free to re-open! :)

nikro → created an issue.

nikro → created an issue.

nikro → created an issue.

Alright, after digging a bit, I realized that all my tests were NON-STREAMING and your bumped into the streaming use-case. I doesn't depend on my original assumptions that it's related to ai.module v1.2.x - but it's simply related to streaming.

Apparently the PostGeneration hook is called even for streaming events, but the contents are empty (as they are normally constructed in a later phase). Solution was to identify that we're actually using a streamed case, and bail early, and use a different event (with already properly prepared response) to track it.

I nuked (or tried nuking) 1.2.x branch, committed changes straight into the main branch.

@Niels, please pull, recheck - see if it works as expected.

nikro → created an issue.

What's the use-case:

- if we want to react in a subscriber to an embedding event specific to ai_search -> we'd do it thanks to this tag
- if we want to use the subscriber to collect analytics (which tokens were used for what feature) - and the embeddings are used for ai_search - we can use this tag to mark the call as ai_search call

nikro → created an issue.

Hey, so here are my 2 cents 👇

After talking to AI, analyzing stuff and the issues such as: https://www.drupal.org/project/ai/issues/3525303 ✨ Create Plugin Action for Recipes to check for installed default provider Active (that got merged btw).
I still think that going the recipe way, with custom actions - is not a good option (due to it being a bit of anti-pattern indeed).
Recipes should stay declarative and deterministic; validation/branching belongs in an orchestration layer (wizard or CLI) that can decide which recipes to apply (or not) based on site state and user choices.

Modifying the official installer and manually adding settings is also not the best way to go - AI modules evolve fast, providers appear fast as well, and their custom settings form change constantly (reasoning modes, showing/hiding reasoning, etc).

I did like the idea of having a custom form, as the final step of the install profile 🤔 so here are my 2 cents (from abhishek's segue):

another contrib module smth like "AI Experience Wizard" in contrib, launched right after standard install (or anytime later).

• Step 1 - Pick the infra (dynamic): User selects exactly one AI provider, optional Vector DB, optional observability. The wizard uses Package Manager to fetch/enable only those projects (similar how abhishek did in his PoC). No pre-bundled 30 modules, no guessing.

• Step 2 - Configure what you just installed (native forms extracted from the selected provider, not hand-rolled): Because the modules are now present, the wizard embeds each provider’s own config form(s) (and VDB/telemetry forms) inline. Secrets go via Key/env; we can validate connectivity. This eliminates brittle, custom per-provider fields in the installer and keeps up with rapidly changing provider options (reasoning, tools, images, etc.). We can also design sub-steps: AI Provider Config, VDB Config, Telemetry Config (like: https://www.drupal.org/project/langfuse → )

• Step 3 - Add "AI Experiences" (declarative recipes): Show tiles like SEO Writer, Translator, Chatbot, Agents, Page Builder. Only enable tiles whose capabilities are satisfied by the chosen infra. Apply just those recipes - small, deterministic bundles that wire UI, permissions, and defaults. No branching inside recipes; the wizard decides which to run.

I'm not sure about these:
- is it hard to tell the install profile where to redirect? can we already have the custom module installed before hand? -> maybe this might be the change we might request from the core/community; AI says that yes, but gotta confirm :)
- can we use the package manager dynamically from our module's form, to install stuff into Drupal? AI says that yes, but gotta confirm :)
- can we, on demand, install recipes (that build on top of existing providers)? AI says that yes, but gotta confirm :)
- can we build a tiny PoC (i.e. have 2 providers for now, 2 vdbs and 2 recipes?)

What do you think? ^_^

nikro → created an issue.

I'll pick this up, will try to cook a PoC over the weekend ^_^

I'll pick it up for a bit of research for today evening + weekend.

Alright, looks good to me.

I did a few changes:

• Renamed picked -> selected for consistency
• Added a missing interface - I think users will want to inject PollerService later so they might need it
• Ran multiple tests, and with a given UUID everything seems to be working nicely

As the backend (poller) at this moment lives separate from the to-be-implemented frontend(s) - maybe we can add a smaller addition to any existing .md files explaining how to use it (or better, create a smaller .md file specific for this) - what do you think? :)

valthebald → credited nikro → .

nikro → created an issue.

nikro → created an issue.

nikro → created an issue.

nikro → created an issue.

Okay now that we cleared - https://www.drupal.org/project/ai/issues/3525296 ✨ Add optional RAW vectors in RAG results Active
And https://www.drupal.org/project/ai_vdb_provider_postgres/issues/3525329 ✨ Implement exposing raw vector Active

--

Along the way of implementation, realized that https://www.drupal.org/project/ai/issues/3488114 ✨ Add support for rerank operation type Active - reranking operation wasn't done yet - so we can't proceed with the FULL reranking (cross-encoder) - so, for now we can just reuse the same embedding model (especially if it's "instruct" bi-encoder) - and use it.

I tested and it does help, it pulled some results closer (i.e. 20 candidates and 5 cut-off, those 5 usually weren't always the same as top 5 would have been). Difference is small but still.

NOTE: this is special for instruct-based embedding models.

nikro → created an issue.

Alright:

1. Added the method to the interface
3. Made sure base abstract method implements it with a NULL
4. Adjusted the search backend itself to use the column if checkbox is checked

nikro → created an issue.