Add a polling mechanism to get constant feedback on status the chatbot

Problem/Motivation

Currently we have the issue that you might be calling the assistants api via the chatbot and a lot of processes is happening in the background, but you only see the final answer once it is complete.

This can take 5-20 seconds depending on how time consuming the instructions are in output tokens, meaning that the end-user might be frustrated and think the loading icon broke.

What we want to do instead is a way to give constant feedback on what is happening. Since we have tool calling now, this should be an easy feedback to give, either via showing what tools are going to be called or showing the textual response that is given back when a tool is being used.

Proposed resolution

• Add a setting that you can choose between a determenistic answer like "Calling {tool_name}" or the textual response on the tool call.
• Add in the assistants api so its keeping track in a private session of all the tools or textual responses directly after the request is finished.
• Add in the assistants api so this private sessions key is being cleared on finished or on destroying the class.
• Add a controller/endpoint that you can poll for the latest status of the response.
• Add to the chatbot so it can poll with a distance of 200ms from the last finished request, to check for updates.
• Create a custom style for the loading/finished part, see: https://deepchat.dev/docs/messages/styles

FYI

This is the solution for 1.1.x, because it is a solution that does work fairly easily and works on any hosting environment, from scalability point of view its not great since each user ends up with two connections - one for the request and one for the poll.

The problem with other solutions is either hosting, time consuming or that they need garbage collectors if the users closes the browser.

A follow up issue will be created to research the use of:

• Javascript as the orchestrator with a garbage collector (or runner).
• Streamed response giving back what is happening using SSE and json-ld.
• Having a websocket orchestrator (nodejs or PHP) in front of the assistants, taking care of the connection.

On top of that we also need to research if we need to solve long running tasks - like deep research or heavy file moving jobs where the agent or even tool needs to run longer then the timeout of a single request.