* feat(flow): report flow outcome and human-in-the-loop signals
A flow reported only that it started. FlowFinishedEvent, FlowFailedEvent,
MethodExecutionFailedEvent, MethodExecutionPausedEvent and FlowPausedEvent all
reached the console formatter and stopped there, and FlowInputRequestedEvent,
FlowInputReceivedEvent and ConversationTurnFailedEvent had no listener at all -
so success rate, failure rate and every HITL pause were unmeasurable.
Adds flow:completed, flow:failed, flow:method_failed, flow:paused,
flow:hitl_paused, flow:input_requested, flow:input_received and
flow:conversation_turn_failed as feature-usage spans, which the existing
feature-usage aggregation already reads.
Deliberately does not hold the Flow Execution span open to measure duration:
flow_executions_daily_target counts those spans at start, so a run that never
finishes would disappear from the count entirely. Duration needs its own span.
Counts only - flow names, method names, error text and flow state are never
recorded.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH
* feat(flow): record how long a flow ran
Adds a Flow Completed span carrying flow_name, duration_ms and outcome,
emitted when a flow finishes or fails. Elapsed time comes from a monotonic
stamp taken at flow start and cleared on use.
Kept separate from the Flow Execution span rather than holding that one open:
it is emitted and closed at start and the daily aggregate counts it, so
holding it would drop every run that is killed or crashes from the execution
count. A killed run now simply has no Flow Completed row, and the count is
unaffected.
Elapsed time is an explicit duration_ms attribute rather than the span's own
duration, which the ingestion pipeline stores as a suffixed string
("0.0000184s") that downstream aggregation parses to zero.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH
* feat(flow): tag flow origin and report resumed runs
Two gaps found while testing the pause/resume path end to end.
Resumed runs were invisible. There is no resume event: a restored run re-enters
through kickoff(), so it looked identical to a fresh start. flow:resumed is
derived from _is_execution_resuming at flow start, which makes
flow:paused - flow:resumed the abandonment rate.
Flow counts are dominated by CrewAI's own AgentExecutor, which is itself a Flow
and runs once per agent execution - it is the top flow in the warehouse by a
wide margin. Nothing distinguished it from a user's flows except guessing at the
name. Both Flow Execution and Flow Completed now carry origin: "internal" when
the flow class is defined under crewai.*, "user" otherwise. Tagging only the new
span would have left the existing daily count unsplittable.
Both span methods take origin with a default, so their signatures stay
backward compatible.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH
* fix(flow): scope outcome and resume signals to user flows
Two findings from review, both confirmed against the code.
Outcome features counted CrewAI's own flows. The agent executor, memory
encoding and memory recall are all Flows and all set suppress_flow_events;
they run far more often than anything a user wrote, so flow:completed,
flow:failed and flow:method_failed were mostly bookkeeping. Those three are now
emitted only for flows the caller wrote. Internal outcomes are still recorded
on the Flow Completed span, which carries origin.
flow:resumed counted checkpoint restores. _is_execution_resuming is set both by
from_pending (a human pause) and by a checkpoint restore that never paused for
anyone, so resumes could exceed pauses and the abandonment rate was unusable.
Keyed off _pending_feedback_context instead, which only from_pending sets.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH
* fix(flow): declare internal flows instead of inferring them
Three findings from review, all confirmed against the code.
Gating on suppress_flow_events was wrong. That flag asks for console quiet and
is a public field, so a caller who set it on their own flow silently lost
flow:completed, flow:failed and flow:method_failed.
Deciding origin from the defining module was also wrong. Flow.from_declaration()
returns a Flow typed in crewai.flow.flow, so a caller's declarative flow was
reported as one of CrewAI's own - the inversion this split exists to prevent.
Both had the same root cause: the discriminator was inferred. Flow now declares
is_crewai_internal, set on the agent executor and the memory encoding/recall
flows, and one helper serves both origin and the outcome gate.
A failed conversational session was reported as completed. Its session closes
with FlowFinishedEvent whatever happened, so a failed turn produced
flow:conversation_turn_failed and flow:completed together. The turn failure is
now recorded on the flow and read back when the session finishes.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH
* refactor(flow): report flow lifecycle as spans, not feature usage
Flow start, completion, pause and method failure are lifecycle facts, and the
lifecycle is reported as spans everywhere else. Reporting them through
feature usage put them in a table that aggregates on the feature string alone -
it cannot carry origin, duration or outcome, so those signals could never be
split between a user's flows and the ones CrewAI runs for itself.
Adds Flow Paused and Flow Method Failed spans, and a resumed marker on Flow
Execution so a run restored from a pause is not counted as a second fresh
start. Removes the duplicate feature rows for completed, failed, method_failed,
paused and resumed - every one of those facts is now on a span, with more
attached to it than the feature row ever carried.
Feature usage keeps only genuine adoption signals: flow:hitl_paused,
flow:input_requested, flow:input_received and flow:conversation_turn_failed.
Also clears the conversational turn-failure flag on every terminal path. A turn
that failed without deferred finalization ends via FlowFailedEvent, and the flag
left set there marked the next run on that instance as failed.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH
* test(flow): update the flow_execution_span caller for the resumed argument
Adding the resumed marker changed a signature that tests/utilities/test_events.py
asserts on exactly, and that assertion was not re-run before pushing.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH
* test(flow): make the checkpoint-restore guard actually guard
The test asserted that flow:resumed was absent from feature usage, but that
signal moved onto the Flow Execution span. The assertion could no longer fail,
so a regression that mis-tagged checkpoint restores as resumes would have gone
unnoticed.
Now asserts the resumed attribute, and waits for the handlers: the manual emit
dispatches asynchronously, so the previous shape also read its result before the
listener had run.
Confirmed it discriminates - keying resumed off _is_execution_resuming again
fails it with [('RestoredFlow', True)] == [('RestoredFlow', False)].
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH
* fix(telemetry): record the resumed marker as a string
Verified end to end against the live collector and ClickHouse: the pipeline
encodes a boolean attribute as the presence of a vBool key, so false arrives as
the key simply being absent. That is invisible in the schema and easy to read
wrongly - crew_memory is extracted as "the attribute exists" and consequently
reports 1 for 99.8% of crews against a field that defaults to False.
A string leaves nothing to infer. Confirmed in the warehouse: the emitted span
reads resumed = "false".
Adds direct coverage for the attributes each flow span records, including both
resumed values, and resets the Telemetry singleton in the helper so more than
one span method can be exercised per session.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH
* feat: bump versions to 1.15.15
---------
Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat: update pyproject.toml to specify wheel targets
Added a new section to the pyproject.toml file to include only specific files in the wheel build, enhancing the packaging process. Updated tests to verify the inclusion of these targets.
* feat: add memory save event handling to activity log
Implemented event handlers for MemorySaveStartedEvent, MemorySaveCompletedEvent, and MemorySaveFailedEvent in the crew_run_tui module. This allows the application to log memory save operations, capturing their status and details in the activity log. Added corresponding tests to verify the correct logging behavior for successful and failed memory saves.
* feat: enhance memory save event handling in activity log
Added functionality to suppress nested memory save events and updated the handling of MemorySaveStartedEvent, MemorySaveCompletedEvent, and MemorySaveFailedEvent to improve logging accuracy. Introduced new tests to verify the correct behavior of memory save events, including scenarios for nested events and completion updates for timed-out entries.
* Fix memory save activity log handling
* Normalize alpha package versions
* Update scaffolded crew dependency
* feat: add button to copy setup instructions for CrewAI coding agents
Introduced a button in the documentation that allows users to easily copy setup instructions for CrewAI coding agents. The instructions include installation steps, environment setup, and best practices for using the CrewAI CLI. This enhancement aims to streamline the onboarding process for new users.
* Improve missing CrewAI install guidance
* fix: address pr review feedback
* fix: avoid mismatched memory save rows
* fix: wait for queued memory save events
* fix: avoid matching memory saves on missing ids
* chore: normalize prerelease version to 1.14.8a1
* feat(cli): introduce JSON crew project support and TUI enhancements
- Added support for creating and running JSON-defined crew projects, allowing users to scaffold projects with a new `create_json_crew.py` file.
- Implemented a full-screen Textual TUI for crew execution in `crew_run_tui.py`, enhancing user interaction with a two-column layout.
- Updated `run_crew.py` to prioritize JSON crew projects and added daemon mode for running without TUI.
- Introduced interactive pickers in `tui_picker.py` for improved CLI prompts.
- Enhanced validation for JSON crew files in `validate.py` to ensure proper structure and agent definitions.
- Updated `.gitignore` to exclude demo and crewai directories.
* feat: update LLM model references to gpt-5.4-mini
- Changed default LLM model from gpt-4o-mini to gpt-5.4-mini across various files, including CLI options, JSON crew configurations, and agent definitions.
- Enhanced benchmark and human feedback functionalities to utilize the new model.
- Improved user interface elements in the TUI for better interaction and feedback during execution.
- Added support for new skills directory in JSON crew project creation.
* feat(benchmark): add crew-level benchmarking functionality
- Introduced a new `benchmark` command in the CLI for crew-level benchmarking, allowing users to specify agents, models, and timeout settings.
- Implemented `CrewBenchmarkCase` to handle crew-level benchmark cases with inputs and criteria.
- Enhanced the benchmark runner to support progress tracking and detailed reporting of results for multiple models.
- Added tests for loading crew benchmark cases and validating their structure.
- Updated existing benchmark functions to accommodate the new crew-level execution model.
* feat(cli): enhance JSON crew project functionality and TUI improvements
- Added optional agent-level guardrails and advanced options in JSON crew configurations to improve output validation and flexibility.
- Updated the TUI to better handle plan step statuses, including visual indicators for task completion and failure.
- Introduced methods for parsing and managing step observation events, ensuring accurate updates to task statuses during execution.
- Enhanced validation for JSON crew projects, ensuring proper structure and error handling for agent and task definitions.
- Added comprehensive tests for new features and validation logic, ensuring robustness in JSON crew project handling.
* refactor(cli): streamline JSON crew project handling and improve validation
- Refactored JSON crew project loading and validation logic to enhance clarity and maintainability.
- Introduced utility functions for finding JSON crew files, improving code reuse across modules.
- Removed deprecated benchmark functionality and associated tests to simplify the codebase.
- Updated CLI commands to utilize the new JSON project structure, ensuring compatibility with recent changes.
- Enhanced test coverage for JSON crew project features, ensuring robust validation and error handling.
* feat(cli): enhance activity log navigation and focus management
- Added functionality to focus on the activity log when navigating through log entries.
- Implemented refresh logic for the log panel to ensure updates are displayed correctly during navigation.
- Improved keyboard navigation for log entries, allowing users to expand and scroll through logs seamlessly.
- Added tests to verify the correct behavior of log navigation and focus management in the TUI.
* feat(cli): enhance JSON crew project interaction and input handling
- Introduced a new function to enable prompt line editing for better user experience during input prompts.
- Updated the JSON crew project wizards to show interpolation hints for dynamic values, improving user guidance.
- Enhanced the handling of missing input placeholders by prompting users for required values during crew setup.
- Refactored the crew run logic to ensure proper loading and preparation of JSON-defined crews, including runtime input management.
- Added tests to verify the correct behavior of new input handling features and JSON crew project interactions.
* feat(cli): improve crew project input prompts and event handling
- Enhanced the `_prompt_text` function to allow for configurable spacing before prompts, improving user experience during input collection.
- Updated the wizards for agent and task creation to utilize the new prompt configuration, ensuring a more compact and streamlined interaction.
- Introduced new plan step lifecycle events (`PlanStepStartedEvent`, `PlanStepCompletedEvent`) to better track the execution status of plan steps.
- Refactored the step executor to emit these events during the execution of tasks, improving observability and debugging capabilities.
- Added tests to verify the correct behavior of new prompt handling and event emissions during crew project execution.
* fix: refine json-first crew interactions
* fix: prioritize common json crew tools
* fix: make json crew more tools expandable
* fix: show json crew tools by category
* feat(memory): update default embedder to OpenAI text-embedding-3-large and enhance memory compatibility
- Changed the default embedding model for Memory to OpenAI text-embedding-3-large, which uses 3072-dimensional vectors.
- Added warnings regarding compatibility issues with existing local memory stores created with 1536-dimensional embeddings.
- Updated documentation to reflect the new default embedder and its configuration options.
- Enhanced the CLI and codebase to support the new embedding model across various components, ensuring a seamless transition for users.
* fix: address PR review feedback for JSON-first crews
Review blockers:
- Forward trained_agents_file to JSON crews: crewai run -f now exports
CREWAI_TRAINED_AGENTS_FILE for the in-process JSON crew path
- Wizard agent picker: Esc/cancel now reprompts instead of silently
assigning the first agent
- JSON tool resolution hard-fails: unknown tool names, missing custom
tool files, and invalid custom tool modules raise JSONProjectError
with actionable messages instead of warn-and-continue
- Embedding dimension mismatch: LanceDB and Qdrant Edge storages raise
EmbeddingDimensionMismatchError with reset/pin guidance instead of
silently zero-filling vectors or returning empty search results
- Custom tool code execution documented in loader docstring and the
scaffolded project README
CI fixes:
- ruff format across lib/
- All 133 PR-introduced mypy errors fixed (llm.py lazy-litellm and
cli.py lazy command shims now use TYPE_CHECKING imports; textual
is_mounted misuse fixed; pick_many overloads; misc annotations)
Bot review comments:
- Empty except blocks now have explanatory comments or debug logging
- Removed unused _C_BG/_C_PANEL/_C_BORDER globals and redundant
import re; tests use a single import style for create_json_crew
Tests: trained-agents propagation, wizard cancel, tool resolution
failures, and dimension mismatch guidance.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: address second round of PR review comments
Cursor Bugbot:
- Wizard agent slugs: strip to [a-z0-9_] and fall back to agent_<n> so
symbol-only roles can't produce an empty agents/.jsonc filename
- Wizard task names: dedupe against prior task names and fall back to
task_<n> for symbol-only descriptions
CodeRabbit:
- Agent.message(): import Task explicitly at runtime instead of relying
on the namespace injection done by crewai/__init__
- Async executor: move the native-tools-unsupported fallback from
_ainvoke_loop_react (self-recursion) to _ainvoke_loop_native_tools,
mirroring the sync implementation
- StepExecutor downgrade: keep the in-step conversation and append the
text-tooling instructions instead of rebuilding messages, so completed
native tool calls are not re-executed
- crewai-files: extension-based MIME lookup now runs before byte
sniffing so csv/xml types are not degraded to text/plain
- Memory storages: validate every record in a save() batch against a
consistent embedding dimension (LanceDB previously checked only the
first record); added mixed-batch tests
- _print_post_tui_summary now typed against CrewRunApp
- Docs: Azure OpenAI default embedder change called out in the memory
migration warning and provider table
Code quality bots:
- Removed unused _C_YELLOW/_C_CYAN (crew_run_tui) and _GREEN (tui_picker)
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* feat(cli): accordion tool picker in JSON crew wizard
The flat tool list had grown to ~90 rows. The picker now shows:
- Common tools always visible at the top
- Every other category as a single expandable row with tool and
selection counts (e.g. "Search & Research (27 tools, 2 selected)")
- Expanding a category collapses the previously expanded one
- Selections persist across expand/collapse via new preselected
support in pick_many; cursor follows the toggled category row
tui_picker gains preselected + initial_cursor options on pick_many,
and Esc in multi-select now confirms the current selection instead of
discarding it (required so collapsing can't silently drop choices).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* refactor(cli): remove --daemon flag from crewai run
The flag only affected JSON crew projects — classic and flow projects
ignored it entirely, which made the behavior inconsistent. Removed the
option, the daemon code path (_run_json_crew_daemon), and its helper
(_load_json_crew_with_inputs).
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* test: update run command tests after --daemon removal
lib/crewai/tests/cli/test_run_crew.py still asserted the old
run_crew(trained_agents_file=..., daemon=False) call signature.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(cli): exit codes, mid-run quit, async statuses, hyphen placeholders
Addresses the latest Bugbot review round:
- Failed JSON crew runs now exit non-zero (SystemExit(1)) so scripts
and CI don't treat failures as success, mirroring the classic path
- Quitting the TUI mid-run now ends the process (os._exit(130));
kickoff runs in a thread worker that cannot be force-cancelled, so
letting the CLI return would leave LLM/tool work burning tokens in
the background
- Sidebar task statuses are now async-safe: completion/failure events
resolve the task's own row via identity instead of assuming the most
recently started task, and starting a task no longer blanket-marks
earlier active rows as done
- The runtime-input prompt regex now accepts hyphenated placeholder
names ({my-topic}), matching kickoff's interpolation pattern
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix: validation safety, custom tool sandboxing, TUI log integrity, memory error surfacing
- Deploy validation no longer executes project code: validation mode
checks tool declarations structurally (well-formed entries, custom
tool file exists) without importing or instantiating anything.
custom:<name> resolution only happens on the actual run path.
- custom:<name> is constrained to [A-Za-z_][A-Za-z0-9_]* and the
resolved path must stay inside the project's tools/ directory, so
custom:../foo or absolute-path names cannot execute code outside it.
Tool paths resolve relative to the crew project root, not cwd.
- TUI task logs are built from per-task state captured at task start
(idx, description, agent, start time); an out-of-order completion
takes its output from the event and no longer steals or resets the
current task's streamed steps/output.
- EmbeddingDimensionMismatchError now inherits ValueError instead of
RuntimeError so background saves surface it through
MemorySaveFailedEvent instead of silently dropping the save; the
shutdown catch in _background_encode_batch is narrowed to the
"cannot schedule new futures" case.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(cli): declared project type wins over crew.json presence
A flow project that also contains a crew.json(c) file now runs and
validates as the flow it declares in pyproject.toml instead of being
hijacked by the JSON crew path. Both crewai run (_has_json_crew) and
deploy validation (_is_json_crew) check tool.crewai.type; a missing or
unreadable pyproject still means a bare JSON crew project.
Also documents why StepObservationFailedEvent intentionally marks the
plan step "done": the event signals an observer failure, not a step
failure, and the executor continues past it.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
* fix(cli): type the declared_type locals so mypy stays clean
Comparing an Any-typed .get() chain returns Any, which tripped
no-any-return on the previous commit.
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
