mirror of
https://github.com/crewAIInc/crewAI.git
synced 2026-07-03 14:09:24 +00:00
Document streamed tool call arguments
This commit is contained in:
@@ -61,6 +61,35 @@ Frames are grouped into high-level channels:
|
||||
|
||||
The stream itself remains one ordered timeline. Channel projections let consumers focus on only part of that timeline.
|
||||
|
||||
### Tool Call Streaming
|
||||
|
||||
Tool calls appear in two places because there are two distinct phases:
|
||||
|
||||
| Phase | Channel | Event type | Meaning |
|
||||
|-------|---------|------------|---------|
|
||||
| Tool-call construction | `llm` | `llm_stream_chunk` | The model is streaming the tool name or arguments it intends to call |
|
||||
| Tool execution | `tools` | `tool_usage_started`, `tool_usage_finished`, `tool_usage_error` | CrewAI is executing the tool and reporting the result |
|
||||
|
||||
For streamed tool-call arguments, the latest provider delta is available as `frame.data["chunk"]`. The accumulated argument string is available at `frame.data["tool_call"]["function"]["arguments"]`.
|
||||
|
||||
```python
|
||||
with llm.stream_events("Check the weather in Paris.", tools=[weather_tool]) as stream:
|
||||
for frame in stream.llm:
|
||||
if frame.type != "llm_stream_chunk":
|
||||
continue
|
||||
|
||||
tool_call = frame.data.get("tool_call")
|
||||
if tool_call:
|
||||
function = tool_call["function"]
|
||||
print("Tool:", function["name"])
|
||||
print("Latest argument delta:", frame.data["chunk"])
|
||||
print("Arguments so far:", function["arguments"])
|
||||
elif frame.content:
|
||||
print(frame.content, end="", flush=True)
|
||||
```
|
||||
|
||||
Use the `tools` channel when you want to display that a tool actually started, completed, or failed. Use the `llm` channel when you want to observe the model constructing the tool call before execution.
|
||||
|
||||
```mermaid
|
||||
flowchart LR
|
||||
A["flow<br/>flow_started"] --> B["llm<br/>llm_call_started"]
|
||||
|
||||
@@ -70,6 +70,38 @@ result = stream.result
|
||||
|
||||
`frame.event` is the structured payload for the source event. Use it for metadata such as tool names, arguments, message roles, and runtime identifiers.
|
||||
|
||||
## Print Streamed Tool Call Arguments
|
||||
|
||||
When a model supports streamed tool calling, CrewAI emits partial tool-call arguments as `llm_stream_chunk` frames on the `llm` channel. These frames are different from tool execution events:
|
||||
|
||||
- `llm` channel: the model is constructing the tool call.
|
||||
- `tools` channel: CrewAI is executing the tool.
|
||||
|
||||
The current delta is stored in `frame.event["chunk"]`. The accumulated arguments are stored in `frame.event["tool_call"]["function"]["arguments"]`.
|
||||
|
||||
```python
|
||||
with llm.stream_events("Get the weather in Paris.", tools=[weather_tool]) as stream:
|
||||
for frame in stream.interleave(["llm", "tools"]):
|
||||
if frame.channel == "llm" and frame.type == "llm_stream_chunk":
|
||||
tool_call = frame.event.get("tool_call")
|
||||
if tool_call:
|
||||
function = tool_call["function"]
|
||||
print(f"\nPreparing tool: {function['name']}")
|
||||
print(f"Arguments so far: {function['arguments']}")
|
||||
elif frame.content:
|
||||
print(frame.content, end="", flush=True)
|
||||
|
||||
elif frame.channel == "tools" and frame.type == "tool_usage_started":
|
||||
print(f"\nRunning tool: {frame.event.get('tool_name')}")
|
||||
|
||||
elif frame.channel == "tools" and frame.type == "tool_usage_finished":
|
||||
print(f"\nTool finished: {frame.event.get('tool_name')}")
|
||||
|
||||
result = stream.result
|
||||
```
|
||||
|
||||
Some providers emit arguments in small JSON fragments. For display, prefer the accumulated argument string over the latest delta.
|
||||
|
||||
## Watch Flow Progress
|
||||
|
||||
Flow lifecycle and method execution frames arrive on the `flow` channel:
|
||||
|
||||
@@ -240,15 +240,18 @@ for chunk in streaming:
|
||||
|
||||
### TOOL_CALL Chunks
|
||||
|
||||
Information about tool calls being made:
|
||||
Information about tool calls being made. Depending on the provider, CrewAI may receive tool-call arguments incrementally. In that case, each `TOOL_CALL` chunk contains the latest streamed content in `chunk.content`, while `chunk.tool_call.arguments` contains the accumulated argument string so far.
|
||||
|
||||
```python Code
|
||||
for chunk in streaming:
|
||||
if chunk.chunk_type == StreamChunkType.TOOL_CALL:
|
||||
print(f"\nCalling tool: {chunk.tool_call.tool_name}")
|
||||
print(f"Arguments: {chunk.tool_call.arguments}")
|
||||
print(f"Latest argument delta: {chunk.content}")
|
||||
print(f"Arguments so far: {chunk.tool_call.arguments}")
|
||||
```
|
||||
|
||||
Actual tool execution is reported separately through tool usage events in the runtime event stream and through CrewAI's verbose console output. `TOOL_CALL` chunks represent the model constructing the tool request before or during execution.
|
||||
|
||||
## Practical Example: Building a UI with Streaming
|
||||
|
||||
Here's a complete example showing how to build an interactive application with streaming:
|
||||
@@ -381,4 +384,4 @@ except Exception as e:
|
||||
print("Streaming completed but an error occurred")
|
||||
```
|
||||
|
||||
By leveraging streaming, you can build more responsive and interactive applications with CrewAI, providing users with real-time visibility into agent execution and results.
|
||||
By leveraging streaming, you can build more responsive and interactive applications with CrewAI, providing users with real-time visibility into agent execution and results.
|
||||
|
||||
@@ -145,6 +145,33 @@ result = stream.result
|
||||
|
||||
`llm.stream_events(...)` temporarily enables streaming for the wrapped call and restores the LLM's previous `stream` setting afterward. Provider integrations continue to emit the underlying LLM stream events; this helper provides a common iterator API over those events for every LLM provider.
|
||||
|
||||
### Tool Call Argument Deltas
|
||||
|
||||
Native tool-call streaming uses `llm_stream_chunk` frames on the `llm` channel. This represents the model constructing a tool call. It is separate from the `tools` channel, which represents CrewAI executing that tool.
|
||||
|
||||
For a streamed tool call, the frame payload includes:
|
||||
|
||||
```python
|
||||
frame.type == "llm_stream_chunk"
|
||||
frame.channel == "llm"
|
||||
frame.event["call_type"] == "tool_call"
|
||||
frame.event["chunk"] # latest argument delta from the provider
|
||||
frame.event["tool_call"] # accumulated tool-call state
|
||||
```
|
||||
|
||||
The `tool_call` payload follows the OpenAI-style function call shape:
|
||||
|
||||
```python
|
||||
tool_call = frame.event["tool_call"]
|
||||
tool_call["id"]
|
||||
tool_call["index"]
|
||||
tool_call["type"] # "function"
|
||||
tool_call["function"]["name"]
|
||||
tool_call["function"]["arguments"] # accumulated JSON argument string
|
||||
```
|
||||
|
||||
Providers differ in granularity. OpenAI and Anthropic may stream arguments as small JSON fragments, while some providers emit the complete argument object in one chunk. Consumers should treat `frame.event["chunk"]` as the latest delta and `tool_call["function"]["arguments"]` as the current accumulated state.
|
||||
|
||||
## Conversational Turns
|
||||
|
||||
Conversational Flows can stream one user turn with `stream_turn()`:
|
||||
|
||||
Reference in New Issue
Block a user