Contrib.social

Max Input Tokens & Max Output Tokens

Open on Drupal.org →
Created on 29 April 2025, 5 months ago

Problem/Motivation

Max Input Tokens & Max Output Tokens default to 0, in /admin/config/ai/providers/chat/google_vertex/gemini20flashlite

Often 0 it this kind of config is essentially unlimited.

It would be good to have either better defaults or better errors.

I don't know if this is good:
Max Input Tokens: 1048576
Max Output Tokens: 8192

I kept running into this error:

An error occurred
The following error occurred while processing your request:
Error invoking model response: Unknown Error: Could not get a response from the model.
Of the following exception type:
Drupal\ai\Exception\AiRequestErrorException

Which does make sense, now that I realize that 0=0. It was just hard to debug when working with an unfamiliar system with lots of possible options for what I did wrong.

Steps to reproduce

Install the module from scratch, or set the tokens to:

Max Input Tokens: 0
Max Output Tokens: 0

Proposed resolution

Set some good defaults.

Remaining tasks

User interface changes

API changes

None

Data model changes

📌 Task
Status

Active

Version

1.0

Component

User interface

Created by

🇨🇦Canada mgifford Ottawa, Ontario

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

  • Issue created by @mgifford
  • Status changed to Closed: works as designed about 1 month ago
  • Comment about 1 month ago →
    🇭🇺Hungary yce

    Hi,

    The defaults seem to be coming from the ai module itself.
    The "Use Model Template" field has some pre-defined defaults set, each model can have different defaults.
    There are templates for google models.

    Regarding the error logging, the latest dev has an update for that: https://www.drupal.org/project/ai_provider_google_vertex/issues/3537588 🐛 ThrowError() method obscurs useful error messages Active

  • Comment about 1 month ago →
    System Message

    Now that this issue is closed, please review the contribution record.

    As a contributor, attribute any organization helped you, or if you volunteered your own time.

    Maintainers, please credit people who helped resolve this issue.

contrib.social Blog FAQ Discussions
Production build 0.71.5 2024