- Issue created by @bbruno
- Merge request !769Issue #3536798: "Stream" Option no longer available for Deepchat block → (Open) created by Unnamed author
- 🇵🇹Portugal bbruno Poland
It would be nice to get some maintainer information first, perhaps this is not the intended behaviour and removing is not what we want. Although it does seem to be the case for now :)
- 🇩🇪Germany marcus_johansson
So this is on purpose - the problem is that the sequence goes like this for none-streaming stuff for agents/assistants:
1. You request something.
2. A response comes with a full payload.
3. The agents looks at this payload and check if it includes tool usage
4. If it includes tools, it will trigger those tools
5. A new loop starts with the tool results.
6. If no tool was used or if an exit tool was used, an answer is given backFor streaming the issue is that it works like this
1. You request something.
2. A response comes in empty
3. The first chunk is sent out to the browser - could be text or beginning of the tool usage
4. This continues until everything is sent out
5. If a tool result is there, we need to assemble it and run a new loop - this happens after the agent is done with its processing meaning we have to invoke a loop via an event here or a callback.
6. If no tool was used or if an exit tool was used, an answer is given backThe part 3-4 is part of this issue is here ✨ Allow tool calling in streamed chat Active and 5 we have to fix via the agents.
So it is completely on purpose - if we would allow tools streaming.
If there is an assistant that wouldn't use any tools/rag, we could allow it, but it would anyway be fixed correctly (hopefully in 1.2.0).