mirror of
https://github.com/crewAIInc/crewAI.git
synced 2026-07-25 00:35:09 +00:00
fix: aggregate token usage across all LLM calls (#6122)
* feat: aggregate LLM token usage at the flow level Introduces `flow.usage_metrics`, a snapshot of every LLMCallCompletedEvent emitted under the flow's `current_flow_id` for the duration of one kickoff (or resume) call. Aggregation happens on the singleton event bus so it covers crews, direct `LLM.call`s, and nested listener calls — solving the mismatch where the SDK reported only the last crew's usage while the Enterprise UI showed the correct full total. Co-authored-by: Cursor <cursoragent@cursor.com> * refactor: centralize provider key normalization in UsageMetrics Add UsageMetrics.from_provider_dict to normalize raw LLM usage dicts across providers (LiteLLM, native Anthropic, native Gemini, OpenAI nested cached). BaseLLM._track_token_usage_internal and the flow-level aggregator now share this single source of truth, so `flow.usage_metrics` agrees with per-LLM totals on every provider — including the native Anthropic path that emits `input_tokens`/`output_tokens` instead of `prompt_tokens`/`completion_tokens`. * fix: flush event bus before reading aggregated usage_metrics `crewai_event_bus.emit` dispatches LLMCallCompletedEvent handlers on a ThreadPoolExecutor (fire-and-forget), so a flow whose last LLM call completes right before kickoff_async/resume_async returns can detach the usage listener while that handler is still queued, leaving its tokens off `flow.usage_metrics`. Match `Crew.kickoff()` and call `crewai_event_bus.flush()` in both finally blocks so every handler drains before the listener is detached. --------- Co-authored-by: Cursor <cursoragent@cursor.com>
This commit is contained in:
@@ -226,6 +226,49 @@ After the Flow has run, you can access the final state to see the updates made b
|
||||
By ensuring that the final method's output is returned and providing access to the state, CrewAI Flows make it easy to integrate the results of your AI workflows into larger applications or systems,
|
||||
while also maintaining and accessing the state throughout the Flow's execution.
|
||||
|
||||
## Flow Usage Metrics
|
||||
|
||||
After a Flow execution completes, you can access the `usage_metrics` property to view aggregated token usage across **every LLM call** made during the run — including calls from every Crew the Flow orchestrated, calls inside Agent tools, and bare `LLM.call(...)` invocations from Flow methods. This is the SDK-side equivalent of the totals shown in the CrewAI Enterprise UI.
|
||||
|
||||
```python Code
|
||||
from crewai import LLM
|
||||
from crewai.flow.flow import Flow, listen, start
|
||||
|
||||
class UsageMetricsFlow(Flow):
|
||||
@start()
|
||||
def run_first_crew(self):
|
||||
self.state.first_result = FirstCrew().crew().kickoff()
|
||||
|
||||
@listen(run_first_crew)
|
||||
def call_llm_directly(self):
|
||||
# Bare LLM call — still counted by flow.usage_metrics
|
||||
llm = LLM(model="openai/gpt-4o-mini")
|
||||
self.state.summary = llm.call("Summarize the key takeaways.")
|
||||
|
||||
@listen(call_llm_directly)
|
||||
def run_second_crew(self):
|
||||
self.state.second_result = SecondCrew().crew().kickoff()
|
||||
|
||||
flow = UsageMetricsFlow()
|
||||
flow.kickoff()
|
||||
|
||||
print(flow.usage_metrics)
|
||||
# UsageMetrics(total_tokens=8579, prompt_tokens=6210, completion_tokens=2369,
|
||||
# cached_prompt_tokens=0, reasoning_tokens=0,
|
||||
# cache_creation_tokens=0, successful_requests=5)
|
||||
```
|
||||
|
||||
<Note>
|
||||
`flow.usage_metrics` is **not** the same as `flow.kickoff().token_usage`. The
|
||||
latter returns the `CrewOutput.token_usage` of the **last** `@listen` method
|
||||
that returned a `CrewOutput`, which means it only reflects the final Crew and
|
||||
ignores prior Crews and bare `LLM.call(...)` invocations entirely. Use
|
||||
`flow.usage_metrics` whenever you need the **full** token rollup for the Flow
|
||||
execution.
|
||||
</Note>
|
||||
|
||||
Each entry in the returned [`UsageMetrics`](https://github.com/crewAIInc/crewAI/blob/main/lib/crewai/src/crewai/types/usage_metrics.py) is the sum across all LLM calls made within a single `flow.kickoff()` invocation. Counters reset on the next `kickoff()` call (or on each iteration of `kickoff_for_each`), so successive runs don't double-count. The property is safe to read at any point after `kickoff()` completes; reading it during execution returns the partial total accumulated so far.
|
||||
|
||||
## Flow State Management
|
||||
|
||||
Managing state effectively is crucial for building reliable and maintainable AI workflows. CrewAI Flows provides robust mechanisms for both unstructured and structured state management,
|
||||
|
||||
Reference in New Issue
Block a user