Add token usage to streamed chat

Created on 14 May 2025, about 17 hours ago

Problem/Motivation

Currently we have the possibility to abstract token usage for anyone using normal static chat, see Abstract token usage Active , however for the streamed chat message this will not happen, since we do not know token usage until the final chunk is strreamed.

We need to add this to the PostStreamingResponseEvent as well.

Steps to reproduce

Proposed resolution

  • Add so that you can collect tokens in the StreamedChatMessage, but do not add it to the interface (no breaking changes).
  • Add so we can collect it in the PostStreamingResponseEvent
  • Add so we inject it into the PostStreamingResponseEvent, but not in the constructor (no bkreaing changes)

Remaining tasks

User interface changes

API changes

Data model changes

📌 Task
Status

Active

Version

1.1

Component

AI Core module

Created by

🇩🇪Germany marcus_johansson

Live updates comments and jobs are added and updated live.
Sign in to follow issues

Comments & Activities

Production build 0.71.5 2024