Reasoning Models

Created on 27 March 2025, 23 days ago

Problem/Motivation

Reasoning models - are the juiciest models Groq provides, but
gets in a way - so I don't want just to hide it, but to provide a clever way, where per Provider Action, user might want to configure how to deal with the Reasoning tokens.

Groq provides a few ways to deal with that - https://console.groq.com/docs/reasoning

Proposed resolution

I propose to use a custom per-action setting of configuring the LLM's default interactions (i.e. reasoning).
Maybe also add a fallback global setting to the provider setting form as well (but per action-setting should override default global fallback).

Remaining tasks

Implement it all :D

Feature request
Status

Active

Version

1.0

Component

Code

Created by

🇪🇸Spain Nikro Benalmadena, Malaga

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Merge Requests

Comments & Activities

  • Issue created by @Nikro
  • 🇪🇸Spain Nikro Benalmadena, Malaga

    I'll work on this during Drupalcon ATL sprint.

  • 🇪🇸Spain Nikro Benalmadena, Malaga
  • 🇪🇸Spain Nikro Benalmadena, Malaga

    So here's what I did:

    • Simple notification in the Default Providers: I added a notification for all providers using Groq to inform them about the available settings to override defaults.
    • Expanded Groq Configuration Form: The Groq config form now includes defaults such as temperature, max tokens, reasoning, and JSON output options. Also we offer an override-per-operation option - to override these defaults (see below).
    • Operation-Specific Overrides: There are two operations - chat and chat_with_json - where users can globally override settings. Fallback settings have also been implemented.
    • Enhanced Groq Model Usage: I expanded the use of the Groq model(s). Chat with JSON is now designed so that any model can be used (it's the dev-user that should decide). I also removed models related to speech, keeping only the textual models for now (additional updates will follow in future tickets).
    • Configuration Persistence: I ensured that the configurations saved are indeed applied during runtime.

    Testing Process

    1. Enable the module, configure it, and set up the necessary API keys.
    2. Enable the assistant and chatbot modules.
    3. Use the chatbot with an assistant:
      • Select a reasoning model in the default settings (or assistant settings) to verify that the model's thought process is displayed.
      • Return to the Groq module settings, change the reasoning option to a hidden value, refresh the interface, and confirm that only the final results are shown - omitting the internal reasoning.

    + I also added extra settings in the api_defaults.yml - to make sure that assistants can override these (we do an array_merge to make sure it's good).

    NOTE: I did this against ai.module released version - NOT the DEV (I will need to make sure it aligns with DEV later).

  • Merge request !3Resolve #3515795 "Reasoning models" → (Merged) created by Nikro
  • Pipeline finished with Failed
    20 days ago
    Total: 166s
    #461463
  • Pipeline finished with Failed
    18 days ago
    Total: 342s
    #462920
  • 🇩🇪Germany marcus_johansson

    Changing to 1.1.x-dev - we could also backport it and create a first 1.0.0-rc1 if wanted?

  • Pipeline finished with Success
    18 days ago
    Total: 153s
    #463408
  • 🇩🇪Germany marcus_johansson

    Thanks, getting merged

  • Automatically closed - issue fixed for 2 weeks with no activity.

Production build 0.71.5 2024