- Issue created by @NicholasS
- 🇺🇸United States NicholasS
So for other who stumble on this issue it seem the problem is the default "Stream" behavior, if you disable the stream the chatbot almost never does this anymore. Bug lies I think in the decoder sometimes the action comes in too late.
Looking at the code in PromptJsonDecoder.php lines 55-75, it only tests the first 10 chunks for JSON markers like {, [, etc. If those markers aren't in the first few chunks (because the AI started with conversational text), it assumes it's not JSON.
Simple Solution: Turn of streaming if your primarily using it for RAG.
- 🇺🇸United States NicholasS
Clarify title since I think I know what is going on as I experiment more with it.
- 🇺🇸United States NicholasS
So as usage continues turning off streaming helps but does not fully fix the issue, users still occasionally experience this issue.