In Tess AI's API, the concept of "stream" is about sending data in real time through a persistent connection. This means that instead of waiting for a full response after processing, the API can send information as it's generated or becomes available. Let's dig a bit deeper into what this means:
Persistent Connection:
When you enable the streaming parameter on an API call, the HTTP connection stays open while the agent runs. This lets the API send data continuously instead of waiting for the execution to finish.
Incremental Data:
While the agent is running, the API streams out parts of the response as soon as they’re ready. This includes:
Tokens or parts of the agent’s “thought process.”
Intermediate results from tool calls.
The final response split into parts, as it gets built.
Interactive Experience:
Streaming gives a way more dynamic experience for the end user. For example, in a chat interface, the user can start seeing replies show up in real time, which is way more engaging than waiting for everything to process before getting a full response.
Ideal Applications:
Streaming is especially handy in scenarios where:
Real-time interaction is crucial (like chats or virtual assistants).
Instant feedback is preferred (like in dashboards or progress displays).
Reduced Apparent Latency: Users get information quickly without having to wait for the entire response.
Interactivity: Makes the user experience better by making the app more responsive and engaging.
Real-Time Data: Lets you show results and progress while the agent is still processing info.
Implementation Complexity: The client has to handle receiving multiple chunks of data, which can be trickier than just waiting for a single response.
Connection Management: Keeping an open connection might need more server and network resources.
In a nutshell, "stream" in the API means sending data in a continuous, real-time way, letting users get info as it's being processed. This feature is perfect for apps that need interactivity and quick response.
