Commit Graph

3 Commits

Author SHA1 Message Date
Lucas Gomide
8565713a1a fix: seed _flow_match_id in from_pending for resume aggregation
A flow restored via Flow.from_pending() is a fresh instance whose
_flow_match_id starts at None. When resume_async then attaches the
LLMCallCompletedEvent listener, the handler filter
(current_flow_id.get() != flow._flow_match_id) either absorbs
unrelated events (when the contextvar is also None) or skips the
flow's own LLM calls (when set to a different id).

Seed instance._flow_match_id = instance.flow_id at the end of
from_pending so the resume-phase aggregator has a real id to match
against. The accumulator itself stays at zero on restore — any
usage from before the pause was only observable on the original
kickoff instance.

Add an end-to-end test that pauses a flow, restores it via
from_pending, emits one of its own LLM events and one belonging
to a sibling flow during resume, and asserts only its own is
counted.
2026-06-11 14:18:12 -03:00
Lucas Gomide
540f5df767 fix: address PR review on flow.usage_metrics
- Protect _aggregated_usage_metrics with a lock so concurrent
  LLMCallCompletedEvent handlers can't race the read-modify-write
  inside add_usage_metrics, and so usage_metrics snapshots are
  consistent.
- Wire the usage aggregation listener into resume_async so LLM
  calls during outcome collapsing and downstream crews continue
  to roll up into flow.usage_metrics after a paused-then-resumed
  kickoff. Restores current_flow_id to the original kickoff's
  match id when none is set, and detaches in finally.
- Guard against reentrant kickoff on the same Flow instance:
  only the outer kickoff captures _flow_match_id, resets the
  accumulator, and owns the listener lifecycle. Inner reentrant
  calls pass through and no longer wipe outer state or detach
  the shared handler.
- Rename test_snapshot_is_immutable to
  test_usage_metrics_returns_independent_copy to reflect that
  the property returns a copy of a (still-mutable) UsageMetrics.
- Extend test_handler_is_unregistered_after_kickoff to also
  cover the failure path, confirming the handler is removed
  when kickoff raises.
2026-06-11 13:39:46 -03:00
Lucas Gomide
c4476366ff fix: aggregate token usage across all LLM calls
`flow.kickoff().token_usage` only returned the last @listen method's
`CrewOutput.token_usage`, so multi-crew flows under-reported by a
factor of N and bare `LLM.call(...)` invocations were ignored
entirely. SDK totals therefore disagreed with the CrewAI Enterprise UI
(Wharf), which aggregates every LLM span.

Add a new `flow.usage_metrics` property that wires an
`LLMCallCompletedEvent` listener for the duration of `kickoff_async`.
The listener scopes to the active flow via the `current_flow_id`
contextvar (the event bus copies the context at emit time, so the
value the handler sees is the one set when the LLM call fired) and
normalizes the provider-specific usage dict into a `UsageMetrics`.
This covers every LLM call inside the flow — crew-led, tool-led, and
bare `LLM.call(...)` — and matches the UI totals 1:1.
2026-06-11 13:29:38 -03:00