Advanced Usages

Streaming and Chunking

Overview

When an Agent generates a long response, sending the entire response at once can make users wait until generation is complete. Large responses can also be difficult for some clients to process reliably.

The Agent Chat Protocol supports streaming, allowing Agents to progressively send a response as it is generated. Chunking can also be used to split large responses into smaller pieces.

Streaming

A streamed response follows this sequence:

StartStreamContent -> TextContent -> TextContent -> EndStreamContent

StartStreamContent opens the stream and provides a stream_id. The Agent then sends one or more TextContent messages before closing the stream with EndStreamContent.

Use streaming when:

  • An LLM takes time to generate a response.
  • A response is long or generated progressively.
  • Users benefit from seeing the response before it is complete.

For short responses, a regular TextContent message is usually sufficient.

Example

A basic streaming pattern may look like this:

copy
1import uuid
2
3from uagents import Context
4from uagents_core.contrib.protocols.chat import (
5    ChatMessage,
6    EndStreamContent,
7    StartStreamContent,
8    TextContent,
9)
10
11
12async def stream_response(
13    ctx: Context,
14    sender: str,
15    chunks: list[str],
16):
17    stream_id = str(uuid.uuid4())
18
19    # Start the stream
20    await ctx.send(
21        sender,
22        ChatMessage(
23            content=[
24                StartStreamContent(stream_id=stream_id),
25            ],
26        ),
27    )
28
29    # Send response chunks
30    for chunk in chunks:
31        await ctx.send(
32            sender,
33            ChatMessage(
34                content=[
35                    TextContent(text=chunk),
36                ],
37            ),
38        )
39
40    # End the stream
41    await ctx.send(
42        sender,
43        ChatMessage(
44            content=[
45                EndStreamContent(stream_id=stream_id),
46            ],
47        ),
48    )

For LLM-powered Agents, chunks can be produced as the model generates its response.

Note: Avoid sending every individual token as a separate message. Buffer small pieces of generated text into reasonably sized chunks.

Chunking Large Responses

Chunking means splitting a large response into smaller pieces before sending it. Prefer natural boundaries such as:

  • Paragraphs.
  • Sentences.
  • Markdown sections.
  • List items.

For large responses that take time to generate, combine chunking with streaming:

StartStream
     ↓
Chunk 1
     ↓
Chunk 2
     ↓
Chunk 3
     ↓
EndStream

If progressive rendering is not required, large content can also be sent as multiple regular ChatMessage messages.

Best Practices

  • Stream long-running responses so users can start reading immediately.
  • Buffer small chunks instead of sending individually.
  • Use natural boundaries when splitting large responses.
  • Keep the same stream_id throughout a stream.
  • Preserve message ordering and follow the normal Chat Protocol acknowledgement rules.
  • Always close the stream with EndStreamContent.