31 Commits

Author	SHA1	Message	Date
João Moura	0374c63129	feat(tracing): task spans say the declared output format and what came out, agent spans carry the prompt and answer, tool spans say whether the cache answered (#7597) ... * feat(tracing): record the task's declared output format, the agent's prompt and answer, and the tool cache flag on their spans A reader of a run's OTel spans could see a task's raw output but not the format it declared, nor whether a Pydantic object or a JSON dict actually came out of it; could see an agent's goal, backstory and model but not the prompt it was handed or the answer it gave; and could see a tool's result but not whether the tool ran or the cache answered. execute task: crewai.task.output_format (json / pydantic / raw; from the declaration on start and failure, from the TaskOutput on completion), crewai.task.output_pydantic_produced, crewai.task.output_json_produced. execute agent: gen_ai.input.messages carries the task prompt and gen_ai.output.messages the answer, the spec shape the task span already uses for its own text, under the existing per-attribute byte cap with the .truncated / .original_size_bytes markers when cut. call tool: crewai.tool.from_cache. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * test(tracing): the agent's prompt and answer leave under the two standard message keys and no other Pins the review decision on #7597: the text travels as gen_ai.input.messages / gen_ai.output.messages — the keys the call llm span already exports its messages under — so a rule an exporter or a redaction processor applies to LLM content by key name applies to the agent span unchanged. A copy under a crewai.agent.* key would fail this. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>	2026-09-19 19:38:04 -03:00
Lorenze Jay	3831e8b6c8	ensure link is shown after finalizing traces (#7593)	2026-09-18 20:13:44 -03:00
João Moura	6b6a39d503	fix(events): Improve coding agents instructions (#7447) ... * feat: scaffold project with assistant instruction files - Updated project creation to include `CLAUDE.md` and `GEMINI.md` that import `AGENTS.md`, ensuring consistent guidance across coding assistants. - Implemented utility functions to copy assistant instruction files during project setup. - Enhanced documentation in `AGENTS.md` to emphasize the importance of keeping telemetry enabled for optimal performance. - Added tests to verify the correct scaffolding of assistant instruction files and their contents. * fix(cli): neutral observability guidance in scaffolded AGENTS.md - State the observability rule as the user's decision, never a fix for console warnings, speed, or a "clean" configuration - Rewrite the AMP section as built-in capabilities: no "free", "proactively", "sales pitch", or scripted pitches - Turn the research mandate into a list of sources to consult when version details matter - Retarget the scaffold tests to the new wording Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(cli): scaffold assistant files for JSON crews and harden telemetry tests - create_json_crew, the default `crewai create crew` path, now copies AGENTS.md, CLAUDE.md and GEMINI.md; AGENTS.md documents the JSON layout - span helper no longer depends on OTEL_SDK_DISABLED being popped by an earlier test; thread-scope test stops its worker before leaving the mock - single import style in the shutdown test Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>	2026-09-16 02:07:57 -03:00
Lorenze Jay	993a96c4e0	feat(tracing): port trace events sessions to OSS (#7464) ... * feat(tracing): port enterprise event sessions to OSS * fix(tracing): address review findings and verify concurrent exports * fix(tracing): keep redactor ownership in enterprise integrations * test(tracing): isolate intentional failures from cleanup assertions	2026-09-15 13:09:40 -07:00
Lorenze Jay	b0fd0a9fa3	feat(telemetry): track checkpoint runtime and CLI usage (#7348) ... * feat(cli): track checkpoint command and TUI usage * feat(telemetry): track runtime checkpoint operations * fix(telemetry): count prune usage after argument validation * style(cli): format checkpoint prune command --------- Co-authored-by: Vidit Ostwal <110953813+Vidit-Ostwal@users.noreply.github.com>	2026-09-09 22:13:11 +05:30
Vidit Ostwal	818f2624e8	[OSS-149] Accept 1/yes/on on telemetry disable flags (#7185) ... * fix(telemetry): accept 1/yes/on on disable flags CREWAI_DISABLE_TELEMETRY=1 was ignored because the gate only matched true, so telemetry stayed on with no warning. * fix(telemetry): warn once on unrecognized disable values Stop repeating the same invalid-flag warning on every telemetry check, and drop the undocumented CREWAI_DISABLE_TRACKING alias from docs. --------- Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com>	2026-09-01 11:41:20 -07:00
João Moura	cba6c03646	feat(events): record how a crew run ended, for every user (#7118) ... * feat(events): record how a crew run ended, for every user Crew was the one level with no ungated terminal record. `Crew Execution` and `end_crew` are both behind `share_crew`, which defaults False, so for essentially every run there is no end-of-crew span at all - not one with fields missing. Task outcomes ship ungated, flow outcomes ship ungated; crew being the exception looks like an accident of history rather than a decision. Adds `Crew Completed` carrying `outcome` and an explicit `duration_ms`, keyed by crew_key/crew_id so it joins the existing ungated `Crew Created`. Modelled directly on `flow_completed_span`, including its reasoning: a separate span rather than holding `Crew Execution` open, because that span is emitted and closed at start, so holding it would drop every run that is killed or crashes. `on_crew_failed` called no telemetry at all before this, so a failed crew produced nothing. It deliberately does not call `end_crew`, which writes onto the gated execution span a failed run may never have opened. Deliberately NOT included, each for a reason: - Tokens. `crew.token_usage` sums per-agent LLM counters, and two agents sharing one LLM object share one counter, so the total double-counts today. Putting it on a span would propagate a known-wrong number into a metric. The dedup keys on `id(llm._token_usage)`, not `id(llm)` - `Agent.copy()` shallow-copies the LLM - and it changes the value of public `Crew.calculate_usage_metrics`, so it earns its own change. - Models. Already on the ungated `Crew Created` span at 99.86% coverage; this joins to them by crew_id rather than duplicating. - Tool counts. The ungated `Tool Usage` span covers only the ReAct path, the plan/step path double-emits, and nested crews share one RuntimeState - the count needs a design decision on cache hits before it is worth emitting. - error_type. Needs 4.1's exception-class field factored out of task_events so both events share it, rather than duplicated hours after that merged. Tests use the exporter pattern this suite already uses rather than mocking `EventListener._telemetry`: EventListener is a singleton, so swapping that leaks a MagicMock into every later test. The listener tests assert on the stamp lifecycle instead. Verified order-independent over five randomized runs. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(telemetry): docstring the crew-completed helpers Eight of the thirteen functions in this file carried a docstring and five did not, which is an inconsistency inside the file this PR adds rather than anything inherited. Documents the two fixtures, the span lookup, the event-bus runner and the two test methods that were missing one. No behaviour change: 9 passed, and re-run under random ordering on two seeds to confirm order-independence. Deliberately not addressed: the reviewer's 52.17% docstring-coverage figure is dominated by event_listener.py, where 84 functions - nearly every pre-existing on_* handler - carry no docstring. That is the file's convention, and documenting them here would be an unrelated refactor. The public API this PR adds, Telemetry.crew_completed_span, is documented. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-28 13:06:51 -03:00
Lorenze Jay	f90d37b4ac	feat(flow): promote conversational flows to stable (#7107) ... Move the canonical API into crewai.flow while preserving experimental imports and declarative references through compatibility aliases. Co-authored-by: Cursor <cursoragent@cursor.com>	2026-08-27 23:06:57 +05:30
João Moura	4e0b2e2b15	fix(events): record task failures as failures, not as successes (#7073) ... * fix(telemetry): record task failures as failures, not as successes close_span() sets StatusCode.OK unconditionally, and TaskFailedEvent was routed to Telemetry.task_ended, which calls it. Every failed task was therefore exported as OK, which is why error_count downstream is not merely low but exactly zero: 240.0M task executions across 13 months in crew_task_executions_daily_target, error_count = 0 in every one of them. The same line had a second defect. The span was only ended when source.agent.crew was present, so a task failing without one was popped from the span map and then never closed - never ended, never exported, invisible rather than mislabelled. task_failed takes no crew (it reads nothing off one), so that condition disappears rather than being widened. Only the exception class name is recorded, never the message, which routinely contains prompts, model output, file paths and credentials. close_span_with_error drops any value failing str.isidentifier(), so a message cannot be recorded even if one is passed by mistake. This is the task half of closed PR #6781, re-cut onto main as that PR asked for. The crew half is deliberately left out: crew_execution_span() returns None unless share_crew=True, so crew._execution_span is None for nearly every user and a crew-failure handler would exit immediately for the default population. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RfV2uMqWRcdfufMvtdCVoN * fix(telemetry): take the exception class for error_type, not a free-form string cursor and CodeRabbit both flagged the sanitization, and they were right: the package already had a stronger convention and this change had not used it. Telemetry._safe_error_type takes the exception *class*, and its docstring says why in as many words - "a single-word message such as 'secret_token' is itself a valid identifier", so filtering a string with isidentifier() is not enough. The ported code predates that helper and reinvented the weaker check. TaskFailedEvent.error_type is now type[BaseException] | None, so pydantic itself rejects a message before any of our code runs, and task_failed routes it through _safe_error_type. The identifier check in close_span_with_error stays as the second gate on the derived name, which is the role _safe_error_type's docstring already describes. Also adds producer-level tests, which CodeRabbit correctly identified as missing: every earlier test constructed TaskFailedEvent directly, so a regression in the two emit sites this change touches in task.py would have passed the whole suite. The sync and async producers are driven through Task._execute_core and Task._aexecute_core with a distinctive exception class, and each patches a different agent method (execute_task vs aexecute_task), which is why they can regress independently. Verified by dropping error_type from both producers: all three new tests fail, and pass again when restored. Removes an unused `import os` left behind when the fixture was rewritten. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RfV2uMqWRcdfufMvtdCVoN * test(telemetry): capture producer failures at the emit boundary, not via the bus The producer tests subscribed a handler to crewai_event_bus and asserted on what it received. That passed this file in isolation and every randomized local run, then failed in CI inside a 621-test shard with zero events captured: FAILED tests/telemetry/test_task_failure_instrumentation.py:: test_sync_producer_puts_the_exception_class_on_the_event assert 0 == 1 + where 0 = len([]) task_failed is an "ending" event, and with an empty scope stack - there is no real kickoff in these tests - dispatch is conditional on event-context state that other tests in the same worker process can leave behind. Subscribing made the assertion depend on the bus choosing to dispatch, which is not what these tests are about: they are about what the producer in task.py constructs. Patching crewai_event_bus.emit records the event unconditionally at the point the producer hands it over, with no dispatch involved. Both producers ignore emit's return value, so returning None is faithful. Containment re-verified after the change: dropping error_type from both producers fails exactly these three tests and nothing else. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RfV2uMqWRcdfufMvtdCVoN * fix(events): keep TaskFailedEvent JSON-serializable with a class-valued error_type error_type holds an exception class, which is not a JSON type, so model_dump(mode="json") raised PydanticSerializationError for the whole event - not just that field. Two real consumers depend on it: the checkpoint listener dumps every event through EventRecord, and the tracing listener JSON-POSTs events to AMP. A single task failure therefore took out checkpointing. field_serializer with when_used="json" returns the class name. The "json" scope is load-bearing: event_listener hands the live class to Telemetry.task_failed, which needs it for _safe_error_type, so python-mode dumps must keep the class. The annotation is a module-level _ExceptionClass alias rather than an inline type[BaseException], because TaskFailedEvent declares a field named `type` which shadows the builtin for the rest of the class body - inline, it raises TypeError at import ("task_failed"[BaseException]) and mypy rejects it as "Variable ... is not valid as a type". Quoting satisfies neither tool: ruff flags UP037 and mypy still resolves it in the class scope. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RfV2uMqWRcdfufMvtdCVoN * fix(events): let a dumped error_type restore, instead of degrading the event The serializer added in the previous commit stopped model_dump(mode="json") from raising, but nothing accepted the class-name string back. _resolve_event (state/event_record.py:32-35) wraps cls.model_validate in a bare except and falls back to BaseEvent, so restoring a checkpoint after a task failure silently dropped the whole event -- including `error`, a plain string that would otherwise have survived. Traded a loud failure for a quiet one. Measured before: dumped error_type='ValueError' and error='boom', restored as BaseEvent with neither attribute. After: restores as TaskFailedEvent with error='boom' and error_type is ValueError. A BeforeValidator resolves a name against real exception classes only -- builtins first, then a walk of BaseException.__subclasses__(). So this does not reopen the hole the class-typed field closes: "secret_token" resolves to nothing, is returned unchanged, and is rejected by the field's own type. Asserted for secret_token, sk_live_1234, dict and os. A name whose class is not imported in this process still degrades, which is deliberate: synthesising a class from an arbitrary string is the injection risk this field exists to avoid. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RfV2uMqWRcdfufMvtdCVoN --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: Vidit Ostwal <110953813+Vidit-Ostwal@users.noreply.github.com>	2026-08-25 14:30:46 +05:30
João Moura	f4731f5025	feat(events): record whether a run had inputs, without recording the inputs (#7072) ... * feat(telemetry): record whether a run had inputs, without recording the inputs The `crew_inputs` payload is gated behind `share_crew` and stays that way, so the only way to tell a parameterised run from an unparameterised one was to read a gated key: it is present on roughly 0.02% of spans, all of them opt-in sharers. That is a measurement of people who opted into sharing, not of users. `crew_inputs_present` carries just the answer -- "true"/"false" -- on the already-ungated `Crew Created` span. The payload stays inside the `share_crew` branch, so nothing new about the contents of anyone's inputs is collected. A string, for the reason `crew_memory` is a string, and the encoding matters more here because the majority case is the empty one. Measured over a single day (312,424,709 spans): `vInt64='0'` occurs 0 times and `vBool='false'` occurs 0 times, while `vStr='0'` does occur. proto3 omits the zero value for ints as well as bools, so an integer key count would have silently dropped every unparameterised run -- and among sharers, 54.46% of runs pass `{}`. `{}` and `None` are both "false": an empty dict parameterises nothing, so truthiness is the question being asked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RfV2uMqWRcdfufMvtdCVoN * test(telemetry): assert input keys are absent too, not only input values The gating test checked only the input value. A regression that emitted the input keys - json.dumps(sorted(inputs)) or similar - would have passed it, and key names are user data as much as values are. Verified by injecting exactly that regression: the new assertion fails on it and passes once reverted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RfV2uMqWRcdfufMvtdCVoN --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-21 16:10:02 +05:30
João Moura	456c67d7c2	fix(telemetry): record crew_memory as a string, not a bool (#7064) ... This pipeline cannot carry a false boolean. Measured across 218,400,577 spans, not one carries vBool=false - proto3 omits the bool zero value, so false is never serialized and arrives as the key simply being absent. "Memory disabled" was therefore structurally unrepresentable, and presence had to stand in for the value, which is why crew_memory read 1 for 99.8% of crews against a field that defaults to False. The fix is the convention this file already documents and applies to `resumed` and `conversational`; crew_memory is the attribute those comments name as the outstanding case. It was the only remaining CrewAI-emitted boolean attribute - checked empirically: every other attribute appearing in vBool comes from third-party instrumentation. Truthiness rather than `is True`, per the decision that memory counts as enabled when set by any means: a Memory, MemoryScope or MemorySlice instance is enabled just as much as `memory=True`. None of those classes defines __bool__ or __len__, so an instance is always truthy. Tests cover all four inputs - True, False, None and an instance - and reuse the existing guard that no attribute is ever passed as a bare boolean. Verified they fail against the unpatched emitter. Claude-Session: https://claude.ai/code/session_01RfV2uMqWRcdfufMvtdCVoN Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-20 21:59:35 +00:00
João Moura	7c72d57b73	fix(events): always emit project_id so absent and empty stay distinct (#7056) ... * fix(telemetry): always emit project_id so absent and empty stay distinct common_span_attributes() stamped project_id onto every span only when the project declared one, and omitted the key otherwise. That makes two different situations indistinguishable downstream: a client too old to report a project id at all, and a current client whose project simply declares none. The consequence is not cosmetic. The share of clients that COULD have reported an id is the denominator of every attribution rate, and with both cases collapsed into "key absent" that denominator cannot be computed at all - it can only be inferred from a version floor, which is fragile and silently wrong for any client that backports or pins. The key is now always present and is the empty string when the project declares none. It still never invents an identity: get_project_id() remains read-only and minting stays with the CLI commands a user explicitly invoked. Two existing tests asserted the old contract and are updated rather than deleted, one of them renamed because its name described the behaviour that changed. A third test is added pinning the distinction itself. The test asserting that a foreign application's spans are never annotated is unaffected and still passes: this changes what our processor stamps, not where it is attached. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RfV2uMqWRcdfufMvtdCVoN * docs(telemetry): note that a failed project_id lookup also yields empty Addresses CodeRabbit on #7056. The docstring and inline comment described the empty string only as an undeclared project, but the except branch sets project_id to None and so lands on the same empty value. Both are deliberately indistinguishable - neither yields an id - and saying so matters to anyone debugging an empty value, since an unreadable pyproject.toml looks identical to a project that simply declares nothing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RfV2uMqWRcdfufMvtdCVoN --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-20 01:10:23 -03:00
João Moura	d74e647502	fix(core): record the running release on every emitted span (#6989) ... * fix(core): record the running release on every emitted span Nine of twenty-four span kinds never recorded crewai_version, including the two highest-volume ones - Task Created and Task Execution - plus Human Feedback, Flow Plotting, and the whole deployment family. add_crew_attributes writes crew_key, crew_id and crew_fingerprint but never the release, so any question filtered by version silently returned nothing for those spans and per-release comparison was blind to them. Add it at the fourteen sites that were missing it across both emitters, matching each module's existing convention: version("crewai") in crewai, get_crewai_version() with the file's local-import pattern in crewai_core. Guarded by a test that parses both modules and fails when any method creates a span without recording the release, so a span added later cannot reintroduce the gap. Verified non-vacuous: removing the attribute from one span makes it fail and names that method. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * test(core): count spans against release attributes, and cover both emitters Two review findings, both real. The guard only asked whether a method mentioned crewai_version anywhere, so a method opening two spans while recording the release on one of them passed. task_started is exactly that shape. It now counts start_span calls against _add_attribute(..., "crewai_version", ...) calls and fails when the second is smaller, naming the method and both counts. Verified non-vacuous: removing the attribute from Task Execution alone - which the previous version accepted - now fails with "task_started (2 span(s), 1 version attribute(s))". The behavioural cases only ever ran against crewai's emitter, because _emit builds that singleton, so the five changed crewai_core methods had no behavioural coverage at all. Added a parametrized case over all eight spans crewai_core emits, using the fixture already in that file - covering the three that already recorded the release as well, so a regression there is caught too. Also removed the function-local `import crewai`: the paths now come from inspect.getfile() on the two classes, which is both consistent with the file's existing import style and more direct than guessing the module layout. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-13 23:57:37 +00:00
João Moura	4b9b8bcbb9	feat(events): record what kind of exception ended a flow (#6982) ... * feat(telemetry): record what kind of exception ended a flow Flow failures are visible but undiagnosable. Live data shows roughly 17% of flows ending in outcome=failed, and 72% of AgentExecutor failures completing in under 200ms - far too fast to be an LLM call - but nothing records what the failure actually is, so there is no way to tell a real defect from a user pressing Ctrl-C. Record the exception's class name as error_type on Flow Completed and Flow Method Failed. The class name only: str(error) is never read, because it routinely carries prompts, model output, file paths and credentials. The isidentifier() check is the allowlist that enforces it - any message text reaching that argument carries a space or punctuation and is dropped - and it lives inside Telemetry rather than at the call site so a future caller cannot bypass it. Method names and flow state remain unrecorded. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * docs(frontend): point the frontend guides at their edge paths The frontend guides added in #6686 exist only under edge - they are not in any frozen version snapshot - but their 80 internal links use the bare /en/guides/frontend/... form, which resolves against the released versions where those pages do not exist. mint broken-links fails on every one of them, which blocks every open PR, not only the one that added them. Use the /edge/en/... form the repo already uses for other edge-only pages (concepts/streaming, learn/execution-boundary-hooks). Anchors are preserved. No page content changes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-13 16:12:19 +00:00
João Moura	8b646620be	fix(events): stop a failed turn from marking the next one failed (#6965) ... * fix(telemetry): stop a failed turn from marking the next one failed A conversational session that opts out of deferred finalization ends each turn with its own FlowFailedEvent, emitted inside kickoff() before handle_turn() emits ConversationTurnFailedEvent. The flag was therefore set after the run that owned it had already cleared it, survived on the instance, and reported the next healthy turn as failed. Gate the flag on the run still having its start stamp: a deferring session keeps it (no per-turn terminal event), so it still reports a failed turn at session end. Also aligns the Flow Lifecycle Signals privacy row with the rest of the telemetry table, which qualifies every user-authored field it records with "should not include personal info", and fixes a telemetry test that built InputResponse with an unsupported `value` keyword - ask() swallowed the TypeError, so the test asserted the signals while exercising the provider-error path. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * test(telemetry): cover the streamed turn emitter of the failure flag stream_turn() is the second emitter of ConversationTurnFailedEvent and leaks the same flag as handle_turn(). Both regression tests fail on 77c68bd with ['failed', 'failed']. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-12 20:59:19 +05:30
João Moura	7642e615a3	feat(flow): report flow outcome, duration and human-in-the-loop signals (#6961) ... * feat(flow): report flow outcome and human-in-the-loop signals A flow reported only that it started. FlowFinishedEvent, FlowFailedEvent, MethodExecutionFailedEvent, MethodExecutionPausedEvent and FlowPausedEvent all reached the console formatter and stopped there, and FlowInputRequestedEvent, FlowInputReceivedEvent and ConversationTurnFailedEvent had no listener at all - so success rate, failure rate and every HITL pause were unmeasurable. Adds flow:completed, flow:failed, flow:method_failed, flow:paused, flow:hitl_paused, flow:input_requested, flow:input_received and flow:conversation_turn_failed as feature-usage spans, which the existing feature-usage aggregation already reads. Deliberately does not hold the Flow Execution span open to measure duration: flow_executions_daily_target counts those spans at start, so a run that never finishes would disappear from the count entirely. Duration needs its own span. Counts only - flow names, method names, error text and flow state are never recorded. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * feat(flow): record how long a flow ran Adds a Flow Completed span carrying flow_name, duration_ms and outcome, emitted when a flow finishes or fails. Elapsed time comes from a monotonic stamp taken at flow start and cleared on use. Kept separate from the Flow Execution span rather than holding that one open: it is emitted and closed at start and the daily aggregate counts it, so holding it would drop every run that is killed or crashes from the execution count. A killed run now simply has no Flow Completed row, and the count is unaffected. Elapsed time is an explicit duration_ms attribute rather than the span's own duration, which the ingestion pipeline stores as a suffixed string ("0.0000184s") that downstream aggregation parses to zero. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * feat(flow): tag flow origin and report resumed runs Two gaps found while testing the pause/resume path end to end. Resumed runs were invisible. There is no resume event: a restored run re-enters through kickoff(), so it looked identical to a fresh start. flow:resumed is derived from _is_execution_resuming at flow start, which makes flow:paused - flow:resumed the abandonment rate. Flow counts are dominated by CrewAI's own AgentExecutor, which is itself a Flow and runs once per agent execution - it is the top flow in the warehouse by a wide margin. Nothing distinguished it from a user's flows except guessing at the name. Both Flow Execution and Flow Completed now carry origin: "internal" when the flow class is defined under crewai.*, "user" otherwise. Tagging only the new span would have left the existing daily count unsplittable. Both span methods take origin with a default, so their signatures stay backward compatible. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * fix(flow): scope outcome and resume signals to user flows Two findings from review, both confirmed against the code. Outcome features counted CrewAI's own flows. The agent executor, memory encoding and memory recall are all Flows and all set suppress_flow_events; they run far more often than anything a user wrote, so flow:completed, flow:failed and flow:method_failed were mostly bookkeeping. Those three are now emitted only for flows the caller wrote. Internal outcomes are still recorded on the Flow Completed span, which carries origin. flow:resumed counted checkpoint restores. _is_execution_resuming is set both by from_pending (a human pause) and by a checkpoint restore that never paused for anyone, so resumes could exceed pauses and the abandonment rate was unusable. Keyed off _pending_feedback_context instead, which only from_pending sets. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * fix(flow): declare internal flows instead of inferring them Three findings from review, all confirmed against the code. Gating on suppress_flow_events was wrong. That flag asks for console quiet and is a public field, so a caller who set it on their own flow silently lost flow:completed, flow:failed and flow:method_failed. Deciding origin from the defining module was also wrong. Flow.from_declaration() returns a Flow typed in crewai.flow.flow, so a caller's declarative flow was reported as one of CrewAI's own - the inversion this split exists to prevent. Both had the same root cause: the discriminator was inferred. Flow now declares is_crewai_internal, set on the agent executor and the memory encoding/recall flows, and one helper serves both origin and the outcome gate. A failed conversational session was reported as completed. Its session closes with FlowFinishedEvent whatever happened, so a failed turn produced flow:conversation_turn_failed and flow:completed together. The turn failure is now recorded on the flow and read back when the session finishes. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * refactor(flow): report flow lifecycle as spans, not feature usage Flow start, completion, pause and method failure are lifecycle facts, and the lifecycle is reported as spans everywhere else. Reporting them through feature usage put them in a table that aggregates on the feature string alone - it cannot carry origin, duration or outcome, so those signals could never be split between a user's flows and the ones CrewAI runs for itself. Adds Flow Paused and Flow Method Failed spans, and a resumed marker on Flow Execution so a run restored from a pause is not counted as a second fresh start. Removes the duplicate feature rows for completed, failed, method_failed, paused and resumed - every one of those facts is now on a span, with more attached to it than the feature row ever carried. Feature usage keeps only genuine adoption signals: flow:hitl_paused, flow:input_requested, flow:input_received and flow:conversation_turn_failed. Also clears the conversational turn-failure flag on every terminal path. A turn that failed without deferred finalization ends via FlowFailedEvent, and the flag left set there marked the next run on that instance as failed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * test(flow): update the flow_execution_span caller for the resumed argument Adding the resumed marker changed a signature that tests/utilities/test_events.py asserts on exactly, and that assertion was not re-run before pushing. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * test(flow): make the checkpoint-restore guard actually guard The test asserted that flow:resumed was absent from feature usage, but that signal moved onto the Flow Execution span. The assertion could no longer fail, so a regression that mis-tagged checkpoint restores as resumes would have gone unnoticed. Now asserts the resumed attribute, and waits for the handlers: the manual emit dispatches asynchronously, so the previous shape also read its result before the listener had run. Confirmed it discriminates - keying resumed off _is_execution_resuming again fails it with [('RestoredFlow', True)] == [('RestoredFlow', False)]. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * fix(telemetry): record the resumed marker as a string Verified end to end against the live collector and ClickHouse: the pipeline encodes a boolean attribute as the presence of a vBool key, so false arrives as the key simply being absent. That is invisible in the schema and easy to read wrongly - crew_memory is extracted as "the attribute exists" and consequently reports 1 for 99.8% of crews against a field that defaults to False. A string leaves nothing to infer. Confirmed in the warehouse: the emitted span reads resumed = "false". Adds direct coverage for the attributes each flow span records, including both resumed values, and resets the Telemetry singleton in the helper so more than one span method can be exercised per session. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-12 00:42:09 +00:00
João Moura	094b94e8d0	fix(core): scope span export to our own tracer provider (#6954) ... * fix(telemetry): stop exporting third-party spans to the collector set_tracer() installed CrewAI's TracerProvider as the global one, so every OTel-instrumented library in the host process - HTTP servers, Redis clients, ORMs - resolved trace.get_tracer() to our provider and exported to CrewAI's endpoint. A 20M-row sample of the telemetry table found 18,866 distinct operation names under our serviceName; CrewAI emits 21. The same wiring lost data in the other direction: when an application had already installed its own provider, our spans were created by theirs and went to their collector, so CrewAI received nothing from instrumented processes. Spans are now created from the private provider in both packages. Deletes _attach_common_attributes and its WeakSet/lock, whose multi-provider dedupe guarded a state that can no longer occur. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * fix(telemetry): keep process context on crewai-core spans Isolating each package to its own TracerProvider removed an accident the CLI spans depended on: crewai_core.telemetry had no CommonAttributesSpanProcessor, so its spans only ever carried coding_agent/runtime_context/project_id by riding the global provider that crewai installed at import. A differential capture of every span reaching the exporter showed 8 of 54 spans losing those attributes - Feature Usage (cli_usage:*), Start Deployment, Template Installed, Create Crew Deployment, Get Crew Logs, Remove Crew, Deploy Signup Error and Flow Creation. Moves the marker tables and the detect_* helpers to crewai_core.runtime_env and the processor plus common_span_attributes() to crewai_core.telemetry, so both implementations share one source of truth. crewai.telemetry.utils and crewai.utilities.constants re-export the moved names, so their import paths are unchanged. Also fixes a gap that predates the isolation change: a CLI-only process never imports crewai, so it never reported either attribute. It does now. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * fix(core): type test helpers and stop tests reaching the collector mypy runs over lib/crewai-core/tests (the one test tree not excluded), so the new test file needed full annotations and a narrowed span.attributes. Also patches SafeOTLPSpanExporter before Telemetry is constructed and shuts the provider down afterwards: __init__ wires a BatchSpanProcessor around the real OTLP exporter, so each test was attempting a live export and leaving its batch worker thread running. Corrects the marker-precedence docstring, which named Cursor third when the table checks it last so that assistants running inside its terminal are not masked. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * style(core): use one import form per module in telemetry tests Both test modules imported their telemetry module twice - once aliased for the monkeypatch target and once via from-import for the names. Dropping the alias in favour of monkeypatch's dotted-string target leaves a single import form and removes the need to qualify every reference. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH * docs(core): drop comments that restate the code The TRACER_NAME constants were annotated with what their name and set_tracer()'s docstring already say, and the test fixtures narrated provider.shutdown() and the exporter patch at more length than either needed. Keeps the ones carrying something the code cannot: the resource-attribute ingestion quirk, why the marker tables moved packages, and the two ordering traps the fixtures exist to avoid. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-11 00:16:27 +00:00
João Moura	92012aec55	feat: split runtime context from coding agent, add project id (#6867) ... * feat(telemetry): split runtime context from coding agent, add project id The coding-agent field answered two questions at once. A run with no TTY reported "non_interactive" and an editor's integrated terminal reported "vscode_terminal", both in the same field as the assistant name, so a run that never had an assistant to detect was indistinguishable from one whose assistant we failed to recognize. Together those two values were the majority of what the field reported. detect_coding_agent now answers only which assistant, returning "unknown" when no marker matches. detect_runtime_context answers where the process runs: ci, serverless, hosted_ide, notebook, container, the editor terminals, and the interactive/non_interactive fallback. Both ride on every span, so an assistant running inside CI reports both rather than one masking the other. The runtime markers are published platform contracts - CI providers, container and serverless runtimes, hosted IDEs - so unlike the assistant table they need no per-tool verification step. Presence is checked; no value is read. The assistant table is unchanged: its entries still require a confirmed, session-scoped variable, and the existing guard test still enforces that. Spans also carry project_id when the project declares one. It is read through the read-only accessor, since minting an id belongs to the CLI commands a user invoked rather than to a library call during execution, and it is omitted entirely for projects without one. The attributes are computed once per process and memoized, so the project file is not re-read for each provider. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * docs: document execution environment telemetry attributes Adds the execution-environment row to the data table in en, ar, ko and pt-BR. Covers the assistant and runtime fields this branch splits apart and the project id, and states that detection reads only whether known environment variables are set. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(telemetry): detect runtime markers by presence, split paas from serverless Three findings from the CodeRabbit, code-quality and Cursor reviews. The runtime loop tested truthiness while constants.py documented presence, so a platform exporting a bare CI= fell through to the TTY fallback and was mislabelled as an ordinary local run. Presence is now what it says. The assistant markers keep truthiness deliberately: there an empty value means the tool set a placeholder rather than claiming the session. DYNO and WEBSITE_INSTANCE_ID marked Heroku dynos and Azure App Service instances as serverless, and since serverless is checked first they could never reach the container label. They move to a paas context, which is what they are: long-lived containers rather than per-invocation functions. AWS_EXECUTION_ENV is dropped entirely - it is set on ECS and EC2 as well as Lambda, and AWS_LAMBDA_FUNCTION_NAME already covers Lambda without the collision. The container probe no longer wraps os.path.exists in a try/except. os.path.exists handles OSError internally and returns False, so the handler guarded a condition that cannot occur and only hid the intent. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * feat(telemetry): widen assistant detection from published marker sets The table previously covered three assistants because the rest were unverified. They are documented after all: vercel/detect-agent publishes a machine-readable detection matrix (agents.json), corroborated by the proposal in agentsmd/agents.md#136 and by microsoft/vscode#311734. Adds cline, gemini_cli, augment, opencode, antigravity and junie, plus CLAUDE_CODE alongside CLAUDECODE. Gemini's marker is confirmed by its own docs, which state that run_shell_command sets GEMINI_CLI=1 in the subprocess environment. Rule 2 excluded several entries those sources list. Goose's GOOSE_PROVIDER and Copilot's COPILOT_MODEL and COPILOT_GITHUB_TOKEN are user configuration, and a committed .env carrying one would relabel every ordinary run - the AIDER_MODEL trap the guard test already pins, now parametrized over all four. Replit's REPL_ID names a hosted environment rather than an assistant, so it stays a runtime context. Copilot sets no session marker at all today; that is an open request upstream. The new assistants are ordered ahead of Cursor, since CURSOR_* is set for every integrated terminal and would otherwise mask anything spawned inside it - the same ordering Codex already needed. Also adds the proposed cross-vendor AI_AGENT marker as a last resort, reported as "other". It establishes that an assistant is present without naming one, and its value is an arbitrary vendor string, so the value is never read. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(deps): raise gitpython and pypdf floors for new advisories gitpython 3.1.57 carries GHSA-9rj7-rf2p-w77r, GHSA-4gmw-gg2m-w46p, GHSA-hh9p-6wh2-4mfc, GHSA-wvpp-8hx9-p66j and GHSA-jm78-9fvv-mhgr: further unguarded git option forwarding in Repo.init, read-tree and git-config, plus arbitrary file read via --pathspec-from-file. Fixed in 3.1.58. pypdf 6.14.2 carries GHSA-fwg2-594c-jp42 and GHSA-fp3f-mc75-235c, unbounded runtime and memory on large content and /ToUnicode streams. Fixed in 6.15.0. Both floors were already pinned, so only the versions move. Their exclude-newer-package cutoffs had to move with them - 3.1.58 landed 2026-08-04 and 6.15.0 on 2026-08-06, both past the existing dates, so the resolver could not have seen either release. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(telemetry): share assistant precedence with the env-context path Four findings from the Cursor and CodeRabbit reviews, three of them the same root cause. get_env_context restated the precedence the shared table already defines, so every marker added for telemetry was invisible to it: a session exposing only CLAUDE_CODE reported claude_code on spans while emitting DefaultEnvEvent, and an assistant running inside a Cursor terminal reported that assistant on spans while emitting CursorEnvEvent. It now walks CODING_AGENT_ENV_MARKERS and maps the three assistants that have an event class of their own, defaulting the rest to DefaultEnvEvent. A test now asserts the two paths agree for every marker in the table, so they cannot drift again. The generic AI_AGENT marker was documented as presence-only but ran through the truthiness loop with everything else, so an empty value fell through to unknown. It moves out of the table and is checked by presence after it, which also keeps the named markers' truthiness intact. Azure Functions run on the App Service host and inherit WEBSITE_INSTANCE_ID, so moving that marker to paas would have relabelled them. The FUNCTIONS_* markers are checked first to keep them serverless. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(telemetry): stop export assertions depending on test order test_all_common_attributes_land_on_exported_spans failed in CI with an IndexError on an empty span list, and only in one shard: the suite runs with OTEL_SDK_DISABLED set, so TracerProvider hands out no-op tracers and an export-based assertion sees zero spans rather than a wrong attribute. It passed only when it happened to run after a test whose fixture flips the variable, which random ordering decides. Adds an otel_enabled fixture that sets the variable for the four tests asserting on exported spans. Three of them predate this branch and had the same latent dependency - they are fixed here because the new test made the ordering hit reachable, and leaving them would keep the required check red. Verified by running every test in the file individually, all of which previously exposed the dependency, and the telemetry suite three times under random ordering. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-07 17:28:52 -07:00
Lucas Gomide	9d659a644b	feat: track interception-hook dispatches in telemetry (#6805) ... `HookDispatchedEvent` was already emitted from the dispatcher but never landed in Feature Usage. Wire it through `hook_dispatched_span` so hook adoption and abort outcomes (e.g. policy checks) show up in the same ClickHouse aggregation as other features.	2026-08-04 13:13:24 -04:00
João Moura	b10c4ffcdc	feat: add project_id to link OSS usage to an enterprise account (#6791) ... * feat: add project_id to link OSS usage to an enterprise account Adds a stable per-project identifier so a project's OSS traces and runs can be attributed to an account after signup. There was no such identifier before: [tool.crewai] held only `type`, the deploy UUID was printed to the console but never persisted, Settings.org_uuid is global rather than per-project, and trace batches carried only crew_fingerprint/crew_name. The id lives in the project's pyproject.toml, so it is committed with the repository and stays stable across machines, teammates, CI, and containers - unlike a machine- or user-derived identifier, which is unstable in exactly the containerized production environments that matter most. crewai-core: - get_project_id(): read-only lookup of [tool.crewai].project_id. Safe for library code; never creates or modifies anything. - get_or_create_project_id(): mints a uuid4 and persists it, returning (id, created) so callers can tell the user. Best-effort - returns (None, False) for a missing, malformed, or read-only pyproject.toml rather than raising. - Insertion edits the raw TOML text instead of round-tripping through a writer, so comments, key order, and formatting elsewhere survive. The key is placed at the end of the [tool.crewai] table, before the next table header, so it cannot land in a neighbouring section. - LoginPayload and TraceExecutionContext gain optional project_id. Sent on two paths: - Traces: project_id is added to execution_context, which is sent on both the ephemeral and authenticated paths, so a project's traces remain attributable before and after the user creates an account. - Login: `crewai login` already sends the pseudonymous user_identifier on an authenticated request; adding project_id means one request carries account + user + project, which is the link itself. Minting is restricted to CLI commands the user explicitly invoked - `crewai create` for new projects and `crewai run` to backfill existing ones - and is announced when it happens. Library code only ever reads. Silently rewriting a user's pyproject.toml during Crew.kickoff() would be surprising. Privacy: project_id is a random uuid4 in a file the user commits. It is visible in a diff, contains nothing personal, and identifies a project rather than a person - so this needs none of the notice changes that attaching a user identifier to all telemetry would require. Tests: 18 new tests covering minting, stability, table placement, comment and formatting preservation, five pyproject layouts, the neighbouring-table regression, and graceful handling of missing/malformed/read-only files. Verified end-to-end that both create paths mint distinct ids, that the trace payload carries project_id on both the ephemeral and authenticated paths, and that the login payload carries user_identifier and project_id together. Follow-ups, deliberately not included: adding project_id to telemetry spans, and backend persistence of the (account, user_identifier, project_id) triple. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UNumDnNbiyw3pv1WakAe6t * refactor: drop the console announcement when minting project_id Minting now happens silently. With no message to print, the (id, created) tuple had no consumer, so simplify the API rather than keep the flag around for a hypothetical caller: - get_or_create_project_id() returns `str | None` instead of `tuple[str | None, bool]`. - Remove crewai_cli.utils.ensure_project_id, which existed only to print the message and discard the flag. The four call sites (crewai create crew, crewai create flow, crewai run, and tool-repository login) now call get_or_create_project_id directly. - Update tests for the simplified signature; still 18 tests covering minting, stability, table placement, formatting preservation, five pyproject layouts, and missing/malformed/read-only handling. Behaviour is otherwise unchanged: minting stays restricted to CLI commands the user invoked, library code still only reads via get_project_id, and a missing or read-only pyproject.toml still returns None rather than raising. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UNumDnNbiyw3pv1WakAe6t * fix: harden project_id minting against TOML corruption; address review Several reviewers found ways the raw-text edit could produce invalid TOML. Each is now fixed and covered by a test that fails without the fix. Duplicate project_id key (Cursor bugbot, Copilot x2): - get_project_id() reports a blank or non-string value as "absent", so a file containing `project_id = ""` took the insert path and gained a second project_id line - a duplicate key, and therefore invalid TOML that no tomli-based tool could read afterwards. - _insert_project_id is now _set_project_id: it replaces an existing assignment inside [tool.crewai] instead of appending unconditionally. Table header with a trailing comment (CodeRabbit major, Cursor bugbot): - `[tool.crewai] # config` is valid TOML but failed exact string equality, so the fallback appended a second [tool.crewai] header - a redefined table, also invalid TOML, and silent because get_project_id swallows the resulting decode error. - Added _is_table_header(), which tolerates a trailing comment and does not match similar names such as [tool.crewai-extra]. Writing into malformed TOML (Cursor bugbot, Copilot): - get_or_create_project_id relied on get_project_id, which cannot distinguish "no id" from "unparsable file", so it appended to files it could not parse. - The locked path now parses explicitly and bails on a decode error, and re-parses the updated content before writing, so this feature can never be the reason a project's pyproject.toml stops parsing. Concurrency and atomicity (CodeRabbit major): - Two CLI processes could both see no id, mint different uuids, and clobber each other, leaving a caller holding an id that is not on disk. Minting now takes the existing crewai_core cross-process lock, re-reads under it, and returns the id that persists. - Writes go through a temp file in the same directory plus os.replace, so an interruption cannot truncate pyproject.toml. File mode is copied across, and the temp file is removed on failure. - os.replace only needs a writable directory, which would have let an atomic write silently overwrite a file the user marked read-only; writability is now checked explicitly so that case still returns None. Line endings (CodeRabbit): - Path.read_text/write_text normalized CRLF to LF, so minting would rewrite a CRLF-committed file entirely. Read and write now use newline="" and the inserted line ending is derived from the existing content. Default create path skipped minting (Cursor bugbot): - `crewai create crew` defaults to create_json_crew; only the --classic and flow paths minted, so most new projects had no id until a later command. Wired into create_json_crew as well. Verified all three paths now mint distinct ids. Do not mint during login (CodeRabbit major): - ToolCommand.login ran get_or_create_project_id, which is outside the sanctioned minting commands and is invoked by `crewai tools create` from a freshly scaffolded directory before the project is persisted. It now uses the read-only get_project_id. Verified login leaves pyproject.toml untouched. Not applied: Copilot asked for a console message when an id is written, in create_crew and create_flow. Minting was made deliberately silent in the previous commit, so the (id, created) tuple and the announcement are both gone by design. Tests: 32 in test_project_id.py, up from 18. New cases cover blank and non-string existing ids, three commented-header forms, similar table names, malformed input, CRLF and LF preservation, concurrent minting convergence, file-mode preservation, and temp-file cleanup. Confirmed the header and duplicate-key tests fail when the fixes are reverted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UNumDnNbiyw3pv1WakAe6t * fix: never create [tool.crewai], treat whitespace ids as absent, harden test `crewai run` could rewrite unrelated projects (Cursor bugbot, high): - get_or_create_project_id ran before the cwd was established as a CrewAI project, and _set_project_id appended a [tool.crewai] table when none existed. Any directory with a pyproject.toml could therefore gain one - including on `crewai run --definition`, which may otherwise succeed. - _set_project_id no longer creates the table; it returns None when [tool.crewai] is absent, so a key is only ever added to a table the project already declares. The templates all ship the table, so no create path needs the old fallback. - The minting call in run_crew moved after the --definition early return, so an explicit-flow run does not touch the cwd at all. - Presence is checked, not truthiness: an empty [tool.crewai] is still a CrewAI marker, and get_crewai_project_config returns {} both for that and for an absent table. - Verified an unrelated project's pyproject.toml is byte-identical after a mint attempt. Whitespace-only project_id accepted as valid (CodeRabbit): - `project_id = " "` is truthy, so it was returned as an identity and would have propagated into login payloads and tracing context. It also meant the '" "' parameter of the replacement test asserted nothing. - Added _usable_project_id, which strips before deciding, used by both get_project_id and the locked mint path. Concurrency test could hang CI (CodeRabbit, major): - Neither the barrier nor the joins had timeouts, so a thread dying early or blocking on the lock would hang the job rather than fail it. The result count was also unchecked, so a dead thread still passed. - Added timeouts, an explicit liveness assertion, a result-count assertion, a lock around the shared result list, and corrected the docstring: this covers the read-modify-write race with threads, not the cross-process backend. Tests: 35, up from 32. New coverage for the absent-table refusal and three whitespace forms; the blank-id replacement case now asserts a real uuid replaced the blank value. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UNumDnNbiyw3pv1WakAe6t * chore(deps): force gitpython 3.1.57+ for GHSA-p538-c434-8v24 and GHSA-3f7w-8rr8-f37f Unrelated to project_id; bundled here only because it blocks this PR's vulnerability scan. Two advisories were published for gitpython 3.1.55 after main last passed the scan: - GHSA-p538-c434-8v24: arbitrary file truncation via `git rev-list --output` argument injection. Fixed in 3.1.56. - GHSA-3f7w-8rr8-f37f: unguarded git option forwarding in IndexFile.checkout() and TagReference. Fixed in 3.1.57. - Bump the override floor to gitpython>=3.1.57 and declare the same floor in crewai-tools, so consumers installing the published package are covered and not only this repo's lock. - 3.1.57 was published 2026-07-26, past gitpython's exclude-newer-package cutoff of 2026-07-24, so that cutoff moves to 2026-07-27. Without it the floor is unresolvable. pip-audit against the updated lock reports no known vulnerabilities. Verified gitpython 3.1.57 resolves and that crewai_tools and crewai_cli.git still import. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UNumDnNbiyw3pv1WakAe6t --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-08-03 15:25:06 -07:00
João Moura	766d71aefb	feat: surface AMP in AGENTS.md and detect coding agents in telemetry (#6779)	2026-08-03 12:13:05 -07:00
Lorenze Jay	799ab0f548	ensure we are writing version for flows (#6467)	2026-07-06 18:47:27 -07:00
Lorenze Jay	a5cc6f6d0e	Add crewai_version to flow execution telemetry (#6167)	2026-06-15 09:34:01 -07:00
João Moura	bb477f8a91	JSON first crews (#6131) ... * feat(cli): introduce JSON crew project support and TUI enhancements - Added support for creating and running JSON-defined crew projects, allowing users to scaffold projects with a new `create_json_crew.py` file. - Implemented a full-screen Textual TUI for crew execution in `crew_run_tui.py`, enhancing user interaction with a two-column layout. - Updated `run_crew.py` to prioritize JSON crew projects and added daemon mode for running without TUI. - Introduced interactive pickers in `tui_picker.py` for improved CLI prompts. - Enhanced validation for JSON crew files in `validate.py` to ensure proper structure and agent definitions. - Updated `.gitignore` to exclude demo and crewai directories. * feat: update LLM model references to gpt-5.4-mini - Changed default LLM model from gpt-4o-mini to gpt-5.4-mini across various files, including CLI options, JSON crew configurations, and agent definitions. - Enhanced benchmark and human feedback functionalities to utilize the new model. - Improved user interface elements in the TUI for better interaction and feedback during execution. - Added support for new skills directory in JSON crew project creation. * feat(benchmark): add crew-level benchmarking functionality - Introduced a new `benchmark` command in the CLI for crew-level benchmarking, allowing users to specify agents, models, and timeout settings. - Implemented `CrewBenchmarkCase` to handle crew-level benchmark cases with inputs and criteria. - Enhanced the benchmark runner to support progress tracking and detailed reporting of results for multiple models. - Added tests for loading crew benchmark cases and validating their structure. - Updated existing benchmark functions to accommodate the new crew-level execution model. * feat(cli): enhance JSON crew project functionality and TUI improvements - Added optional agent-level guardrails and advanced options in JSON crew configurations to improve output validation and flexibility. - Updated the TUI to better handle plan step statuses, including visual indicators for task completion and failure. - Introduced methods for parsing and managing step observation events, ensuring accurate updates to task statuses during execution. - Enhanced validation for JSON crew projects, ensuring proper structure and error handling for agent and task definitions. - Added comprehensive tests for new features and validation logic, ensuring robustness in JSON crew project handling. * refactor(cli): streamline JSON crew project handling and improve validation - Refactored JSON crew project loading and validation logic to enhance clarity and maintainability. - Introduced utility functions for finding JSON crew files, improving code reuse across modules. - Removed deprecated benchmark functionality and associated tests to simplify the codebase. - Updated CLI commands to utilize the new JSON project structure, ensuring compatibility with recent changes. - Enhanced test coverage for JSON crew project features, ensuring robust validation and error handling. * feat(cli): enhance activity log navigation and focus management - Added functionality to focus on the activity log when navigating through log entries. - Implemented refresh logic for the log panel to ensure updates are displayed correctly during navigation. - Improved keyboard navigation for log entries, allowing users to expand and scroll through logs seamlessly. - Added tests to verify the correct behavior of log navigation and focus management in the TUI. * feat(cli): enhance JSON crew project interaction and input handling - Introduced a new function to enable prompt line editing for better user experience during input prompts. - Updated the JSON crew project wizards to show interpolation hints for dynamic values, improving user guidance. - Enhanced the handling of missing input placeholders by prompting users for required values during crew setup. - Refactored the crew run logic to ensure proper loading and preparation of JSON-defined crews, including runtime input management. - Added tests to verify the correct behavior of new input handling features and JSON crew project interactions. * feat(cli): improve crew project input prompts and event handling - Enhanced the `_prompt_text` function to allow for configurable spacing before prompts, improving user experience during input collection. - Updated the wizards for agent and task creation to utilize the new prompt configuration, ensuring a more compact and streamlined interaction. - Introduced new plan step lifecycle events (`PlanStepStartedEvent`, `PlanStepCompletedEvent`) to better track the execution status of plan steps. - Refactored the step executor to emit these events during the execution of tasks, improving observability and debugging capabilities. - Added tests to verify the correct behavior of new prompt handling and event emissions during crew project execution. * fix: refine json-first crew interactions * fix: prioritize common json crew tools * fix: make json crew more tools expandable * fix: show json crew tools by category * feat(memory): update default embedder to OpenAI text-embedding-3-large and enhance memory compatibility - Changed the default embedding model for Memory to OpenAI text-embedding-3-large, which uses 3072-dimensional vectors. - Added warnings regarding compatibility issues with existing local memory stores created with 1536-dimensional embeddings. - Updated documentation to reflect the new default embedder and its configuration options. - Enhanced the CLI and codebase to support the new embedding model across various components, ensuring a seamless transition for users. * fix: address PR review feedback for JSON-first crews Review blockers: - Forward trained_agents_file to JSON crews: crewai run -f now exports CREWAI_TRAINED_AGENTS_FILE for the in-process JSON crew path - Wizard agent picker: Esc/cancel now reprompts instead of silently assigning the first agent - JSON tool resolution hard-fails: unknown tool names, missing custom tool files, and invalid custom tool modules raise JSONProjectError with actionable messages instead of warn-and-continue - Embedding dimension mismatch: LanceDB and Qdrant Edge storages raise EmbeddingDimensionMismatchError with reset/pin guidance instead of silently zero-filling vectors or returning empty search results - Custom tool code execution documented in loader docstring and the scaffolded project README CI fixes: - ruff format across lib/ - All 133 PR-introduced mypy errors fixed (llm.py lazy-litellm and cli.py lazy command shims now use TYPE_CHECKING imports; textual is_mounted misuse fixed; pick_many overloads; misc annotations) Bot review comments: - Empty except blocks now have explanatory comments or debug logging - Removed unused _C_BG/_C_PANEL/_C_BORDER globals and redundant import re; tests use a single import style for create_json_crew Tests: trained-agents propagation, wizard cancel, tool resolution failures, and dimension mismatch guidance. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address second round of PR review comments Cursor Bugbot: - Wizard agent slugs: strip to [a-z0-9_] and fall back to agent_<n> so symbol-only roles can't produce an empty agents/.jsonc filename - Wizard task names: dedupe against prior task names and fall back to task_<n> for symbol-only descriptions CodeRabbit: - Agent.message(): import Task explicitly at runtime instead of relying on the namespace injection done by crewai/__init__ - Async executor: move the native-tools-unsupported fallback from _ainvoke_loop_react (self-recursion) to _ainvoke_loop_native_tools, mirroring the sync implementation - StepExecutor downgrade: keep the in-step conversation and append the text-tooling instructions instead of rebuilding messages, so completed native tool calls are not re-executed - crewai-files: extension-based MIME lookup now runs before byte sniffing so csv/xml types are not degraded to text/plain - Memory storages: validate every record in a save() batch against a consistent embedding dimension (LanceDB previously checked only the first record); added mixed-batch tests - _print_post_tui_summary now typed against CrewRunApp - Docs: Azure OpenAI default embedder change called out in the memory migration warning and provider table Code quality bots: - Removed unused _C_YELLOW/_C_CYAN (crew_run_tui) and _GREEN (tui_picker) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(cli): accordion tool picker in JSON crew wizard The flat tool list had grown to ~90 rows. The picker now shows: - Common tools always visible at the top - Every other category as a single expandable row with tool and selection counts (e.g. "Search & Research (27 tools, 2 selected)") - Expanding a category collapses the previously expanded one - Selections persist across expand/collapse via new preselected support in pick_many; cursor follows the toggled category row tui_picker gains preselected + initial_cursor options on pick_many, and Esc in multi-select now confirms the current selection instead of discarding it (required so collapsing can't silently drop choices). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(cli): remove --daemon flag from crewai run The flag only affected JSON crew projects — classic and flow projects ignored it entirely, which made the behavior inconsistent. Removed the option, the daemon code path (_run_json_crew_daemon), and its helper (_load_json_crew_with_inputs). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: update run command tests after --daemon removal lib/crewai/tests/cli/test_run_crew.py still asserted the old run_crew(trained_agents_file=..., daemon=False) call signature. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): exit codes, mid-run quit, async statuses, hyphen placeholders Addresses the latest Bugbot review round: - Failed JSON crew runs now exit non-zero (SystemExit(1)) so scripts and CI don't treat failures as success, mirroring the classic path - Quitting the TUI mid-run now ends the process (os._exit(130)); kickoff runs in a thread worker that cannot be force-cancelled, so letting the CLI return would leave LLM/tool work burning tokens in the background - Sidebar task statuses are now async-safe: completion/failure events resolve the task's own row via identity instead of assuming the most recently started task, and starting a task no longer blanket-marks earlier active rows as done - The runtime-input prompt regex now accepts hyphenated placeholder names ({my-topic}), matching kickoff's interpolation pattern Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: validation safety, custom tool sandboxing, TUI log integrity, memory error surfacing - Deploy validation no longer executes project code: validation mode checks tool declarations structurally (well-formed entries, custom tool file exists) without importing or instantiating anything. custom:<name> resolution only happens on the actual run path. - custom:<name> is constrained to [A-Za-z_][A-Za-z0-9_]* and the resolved path must stay inside the project's tools/ directory, so custom:../foo or absolute-path names cannot execute code outside it. Tool paths resolve relative to the crew project root, not cwd. - TUI task logs are built from per-task state captured at task start (idx, description, agent, start time); an out-of-order completion takes its output from the event and no longer steals or resets the current task's streamed steps/output. - EmbeddingDimensionMismatchError now inherits ValueError instead of RuntimeError so background saves surface it through MemorySaveFailedEvent instead of silently dropping the save; the shutdown catch in _background_encode_batch is narrowed to the "cannot schedule new futures" case. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): declared project type wins over crew.json presence A flow project that also contains a crew.json(c) file now runs and validates as the flow it declares in pyproject.toml instead of being hijacked by the JSON crew path. Both crewai run (_has_json_crew) and deploy validation (_is_json_crew) check tool.crewai.type; a missing or unreadable pyproject still means a bare JSON crew project. Also documents why StepObservationFailedEvent intentionally marks the plan step "done": the event signals an observer failure, not a step failure, and the executor continues past it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): type the declared_type locals so mypy stays clean Comparing an Any-typed .get() chain returns Any, which tripped no-any-return on the previous commit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-06-14 04:19:48 -03:00
Lorenze Jay	f88ae54f96	fix telemetry setup on crewai-login (#6106) ... * fix telemetry setup on crewai-login * type check fix	2026-06-10 17:03:25 -07:00
Greyson LaLonde	fd10c64148	chore(crewai): drop self-explanatory comments	2026-05-26 10:23:33 -07:00
iris-clawd	3322634625	feat: deprecate CrewAgentExecutor, default Crew agents to AgentExecutor (#5745) ... * feat: deprecate CrewAgentExecutor, default Crew agents to AgentExecutor * regen cassettes * fix tests * addressing pr comments --------- Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com> Co-authored-by: lorenzejay <lorenzejaytech@gmail.com> Co-authored-by: Greyson LaLonde <greyson.r.lalonde@gmail.com>	2026-05-12 11:22:13 -07:00
Giulio Leone	b76022c1e7	fix(telemetry): skip signal handler registration in non-main threads ... * fix(telemetry): skip signal handler registration in non-main threads When CrewAI is initialized from a non-main thread (e.g. Streamlit, Flask, Django, Jupyter), the telemetry module attempted to register signal handlers which only work in the main thread. This caused multiple noisy ValueError tracebacks to be printed to stderr, confusing users even though the errors were caught and non-fatal. Check `threading.current_thread() is not threading.main_thread()` before attempting signal registration, and skip silently with a debug-level log message instead of printing full tracebacks. Fixes crewAIInc/crewAI#4289 * fix(test): move Telemetry() inside signal.signal mock context Refs: #4649 * fix(telemetry): move signal.signal mock inside thread to wrap Telemetry() construction The patch context now activates inside init_in_thread so the mock is guaranteed to be active before and during Telemetry.__init__, addressing the Copilot review feedback. Refs: #4289 * fix(test): mock logger.debug instead of capsys for deterministic assertion Replace signal.signal-only mock with combined logger + signal mock. Assert logger.debug was called with the skip message and signal.signal was never invoked from the non-main thread. Refs: #4289	2026-03-02 07:42:55 -05:00
Greyson LaLonde	f2f994612c	fix: ensure otel span is closed	2025-12-05 13:23:26 -05:00
Greyson LaLonde	c925d2d519	chore: restructure test env, cassettes, and conftest; fix flaky tests ... Consolidates pytest config, standardizes env handling, reorganizes cassette layout, removes outdated VCR configs, improves sync with threading.Condition, updates event-waiting logic, ensures cleanup, regenerates Gemini cassettes, and reverts unintended test changes.	2025-11-29 16:55:24 -05:00
Lorenze Jay	d1343b96ed	Release/v1.0.0 (#3618) ... * feat: add `apps` & `actions` attributes to Agent (#3504) * feat: add app attributes to Agent * feat: add actions attribute to Agent * chore: resolve linter issues * refactor: merge the apps and actions parameters into a single one * fix: remove unnecessary print * feat: logging error when CrewaiPlatformTools fails * chore: export CrewaiPlatformTools directly from crewai_tools * style: resolver linter issues * test: fix broken tests * style: solve linter issues * fix: fix broken test * feat: monorepo restructure and test/ci updates - Add crewai workspace member - Fix vcr cassette paths and restore test dirs - Resolve ci failures and update linter/pytest rules * chore: update python version to 3.13 and package metadata * feat: add crewai-tools workspace and fix tests/dependencies * feat: add crewai-tools workspace structure * Squashed 'temp-crewai-tools/' content from commit 9bae5633 git-subtree-dir: temp-crewai-tools git-subtree-split: 9bae56339096cb70f03873e600192bd2cd207ac9 * feat: configure crewai-tools workspace package with dependencies * fix: apply ruff auto-formatting to crewai-tools code * chore: update lockfile * fix: don't allow tool tests yet * fix: comment out extra pytest flags for now * fix: remove conflicting conftest.py from crewai-tools tests * fix: resolve dependency conflicts and test issues - Pin vcrpy to 7.0.0 to fix pytest-recording compatibility - Comment out types-requests to resolve urllib3 conflict - Update requests requirement in crewai-tools to >=2.32.0 * chore: update CI workflows and docs for monorepo structure * chore: update CI workflows and docs for monorepo structure * fix: actions syntax * chore: ci publish and pin versions * fix: add permission to action * chore: bump version to 1.0.0a1 across all packages - Updated version to 1.0.0a1 in pyproject.toml for crewai and crewai-tools - Adjusted version in __init__.py files for consistency * WIP: v1 docs (#3626) (cherry picked from commit d46e20fa09bcd2f5916282f5553ddeb7183bd92c) * docs: parity for all translations * docs: full name of acronym AMP * docs: fix lingering unused code * docs: expand contextual options in docs.json * docs: add contextual action to request feature on GitHub (#3635) * chore: apply linting fixes to crewai-tools * feat: add required env var validation for brightdata Co-authored-by: Greyson Lalonde <greyson.r.lalonde@gmail.com> * fix: handle properly anyOf oneOf allOf schema's props Co-authored-by: Greyson Lalonde <greyson.r.lalonde@gmail.com> * feat: bump version to 1.0.0a2 * Lorenze/native inference sdks (#3619) * ruff linted * using native sdks with litellm fallback * drop exa * drop print on completion * Refactor LLM and utility functions for type consistency - Updated `max_tokens` parameter in `LLM` class to accept `float` in addition to `int`. - Modified `create_llm` function to ensure consistent type hints and return types, now returning `LLM | BaseLLM | None`. - Adjusted type hints for various parameters in `create_llm` and `_llm_via_environment_or_fallback` functions for improved clarity and type safety. - Enhanced test cases to reflect changes in type handling and ensure proper instantiation of LLM instances. * fix agent_tests * fix litellm tests and usagemetrics fix * drop print * Refactor LLM event handling and improve test coverage - Removed commented-out event emission for LLM call failures in `llm.py`. - Added `from_agent` parameter to `CrewAgentExecutor` for better context in LLM responses. - Enhanced test for LLM call failure to simulate OpenAI API failure and updated assertions for clarity. - Updated agent and task ID assertions in tests to ensure they are consistently treated as strings. * fix test_converter * fixed tests/agents/test_agent.py * Refactor LLM context length exception handling and improve provider integration - Renamed `LLMContextLengthExceededException` to `LLMContextLengthExceededExceptionError` for clarity and consistency. - Updated LLM class to pass the provider parameter correctly during initialization. - Enhanced error handling in various LLM provider implementations to raise the new exception type. - Adjusted tests to reflect the updated exception name and ensure proper error handling in context length scenarios. * Enhance LLM context window handling across providers - Introduced CONTEXT_WINDOW_USAGE_RATIO to adjust context window sizes dynamically for Anthropic, Azure, Gemini, and OpenAI LLMs. - Added validation for context window sizes in Azure and Gemini providers to ensure they fall within acceptable limits. - Updated context window size calculations to use the new ratio, improving consistency and adaptability across different models. - Removed hardcoded context window sizes in favor of ratio-based calculations for better flexibility. * fix test agent again * fix test agent * feat: add native LLM providers for Anthropic, Azure, and Gemini - Introduced new completion implementations for Anthropic, Azure, and Gemini, integrating their respective SDKs. - Added utility functions for tool validation and extraction to support function calling across LLM providers. - Enhanced context window management and token usage extraction for each provider. - Created a common utility module for shared functionality among LLM providers. * chore: update dependencies and improve context management - Removed direct dependency on `litellm` from the main dependencies and added it under extras for better modularity. - Updated the `litellm` dependency specification to allow for greater flexibility in versioning. - Refactored context length exception handling across various LLM providers to use a consistent error class. - Enhanced platform-specific dependency markers for NVIDIA packages to ensure compatibility across different systems. * refactor(tests): update LLM instantiation to include is_litellm flag in test cases - Modified multiple test cases in test_llm.py to set the is_litellm parameter to True when instantiating the LLM class. - This change ensures that the tests are aligned with the latest LLM configuration requirements and improves consistency across test scenarios. - Adjusted relevant assertions and comments to reflect the updated LLM behavior. * linter * linted * revert constants * fix(tests): correct type hint in expected model description - Updated the expected description in the test_generate_model_description_dict_field function to use 'Dict' instead of 'dict' for consistency with type hinting conventions. - This change ensures that the test accurately reflects the expected output format for model descriptions. * refactor(llm): enhance LLM instantiation and error handling - Updated the LLM class to include validation for the model parameter, ensuring it is a non-empty string. - Improved error handling by logging warnings when the native SDK fails, allowing for a fallback to LiteLLM. - Adjusted the instantiation of LLM in test cases to consistently include the is_litellm flag, aligning with recent changes in LLM configuration. - Modified relevant tests to reflect these updates, ensuring better coverage and accuracy in testing scenarios. * fixed test * refactor(llm): enhance token usage tracking and add copy methods - Updated the LLM class to track token usage and log callbacks in streaming mode, improving monitoring capabilities. - Introduced shallow and deep copy methods for the LLM instance, allowing for better management of LLM configurations and parameters. - Adjusted test cases to instantiate LLM with the is_litellm flag, ensuring alignment with recent changes in LLM configuration. * refactor(tests): reorganize imports and enhance error messages in test cases - Cleaned up import statements in test_crew.py for better organization and readability. - Enhanced error messages in test cases to use `re.escape` for improved regex matching, ensuring more robust error handling. - Adjusted comments for clarity and consistency across test scenarios. - Ensured that all necessary modules are imported correctly to avoid potential runtime issues. * feat: add base devtooling * fix: ensure dep refs are updated for devtools * fix: allow pre-release * feat: allow release after tag * feat: bump versions to 1.0.0a3 Co-authored-by: Greyson LaLonde <greyson.r.lalonde@gmail.com> * fix: match tag and release title, ignore devtools build for pypi * fix: allow failed pypi publish * feat: introduce trigger listing and execution commands for local development (#3643) * chore: exclude tests from ruff linting * chore: exclude tests from GitHub Actions linter * fix: replace print statements with logger in agent and memory handling * chore: add noqa for intentional print in printer utility * fix: resolve linting errors across codebase * feat: update docs with new approach to consume Platform Actions (#3675) * fix: remove duplicate line and add explicit env var * feat: bump versions to 1.0.0a4 (#3686) * Update triggers docs (#3678) * docs: introduce triggers list & triggers run command * docs: add KO triggers docs * docs: ensure CREWAI_PLATFORM_INTEGRATION_TOKEN is mentioned on docs (#3687) * Lorenze/bedrock llm (#3693) * feat: add AWS Bedrock support and update dependencies - Introduced BedrockCompletion class for AWS Bedrock integration in LLM. - Added boto3 as a new dependency in both pyproject.toml and uv.lock. - Updated LLM class to support Bedrock provider. - Created new files for Bedrock provider implementation. * using converse api * converse * linted * refactor: update BedrockCompletion class to improve parameter handling - Changed max_tokens from a fixed integer to an optional integer. - Simplified model ID assignment by removing the inference profile mapping method. - Cleaned up comments and unnecessary code related to tool specifications and model-specific parameters. * feat: improve event bus thread safety and async support Add thread-safe, async-compatible event bus with read–write locking and handler dependency ordering. Remove blinker dependency and implement direct dispatch. Improve type safety, error handling, and deterministic event synchronization. Refactor tests to auto-wait for async handlers, ensure clean teardown, and add comprehensive concurrency coverage. Replace thread-local state in AgentEvaluator with instance-based locking for correct cross-thread access. Enhance tracing reliability and event finalization. * feat: enhance OpenAICompletion class with additional client parameters (#3701) * feat: enhance OpenAICompletion class with additional client parameters - Added support for default_headers, default_query, and client_params in the OpenAICompletion class. - Refactored client initialization to use a dedicated method for client parameter retrieval. - Introduced new test cases to validate the correct usage of OpenAICompletion with various parameters. * fix: correct test case for unsupported OpenAI model - Updated the test_openai.py to ensure that the LLM instance is created before calling the method, maintaining proper error handling for unsupported models. - This change ensures that the test accurately checks for the NotFoundError when an invalid model is specified. * fix: enhance error handling in OpenAICompletion class - Added specific exception handling for NotFoundError and APIConnectionError in the OpenAICompletion class to provide clearer error messages and improve logging. - Updated the test case for unsupported models to ensure it raises a ValueError with the appropriate message when a non-existent model is specified. - This change improves the robustness of the OpenAI API integration and enhances the clarity of error reporting. * fix: improve test for unsupported OpenAI model handling - Refactored the test case in test_openai.py to create the LLM instance after mocking the OpenAI client, ensuring proper error handling for unsupported models. - This change enhances the clarity of the test by accurately checking for ValueError when a non-existent model is specified, aligning with recent improvements in error handling for the OpenAICompletion class. * feat: bump versions to 1.0.0b1 (#3706) * Lorenze/tools drop litellm (#3710) * completely drop litellm and correctly pass config for qdrant * feat: add support for additional embedding models in EmbeddingService - Expanded the list of supported embedding models to include Google Vertex, Hugging Face, Jina, Ollama, OpenAI, Roboflow, Watson X, custom embeddings, Sentence Transformers, Text2Vec, OpenClip, and Instructor. - This enhancement improves the versatility of the EmbeddingService by allowing integration with a wider range of embedding providers. * fix: update collection parameter handling in CrewAIRagAdapter - Changed the condition for setting vectors_config in the CrewAIRagAdapter to check for QdrantConfig instance instead of using hasattr. This improves type safety and ensures proper configuration handling for Qdrant integration. * moved stagehand as optional dep (#3712) * feat: bump versions to 1.0.0b2 (#3713) * feat: enhance AnthropicCompletion class with additional client parame… (#3707) * feat: enhance AnthropicCompletion class with additional client parameters and tool handling - Added support for client_params in the AnthropicCompletion class to allow for additional client configuration. - Refactored client initialization to use a dedicated method for retrieving client parameters. - Implemented a new method to handle tool use conversation flow, ensuring proper execution and response handling. - Introduced comprehensive test cases to validate the functionality of the AnthropicCompletion class, including tool use scenarios and parameter handling. * drop print statements * test: add fixture to mock ANTHROPIC_API_KEY for tests - Introduced a pytest fixture to automatically mock the ANTHROPIC_API_KEY environment variable for all tests in the test_anthropic.py module. - This change ensures that tests can run without requiring a real API key, improving test isolation and reliability. * refactor: streamline streaming message handling in AnthropicCompletion class - Removed the 'stream' parameter from the API call as it is set internally by the SDK. - Simplified the handling of tool use events and response construction by extracting token usage from the final message. - Enhanced the flow for managing tool use conversation, ensuring proper integration with the streaming API response. * fix streaming here too * fix: improve error handling in tool conversion for AnthropicCompletion class - Enhanced exception handling during tool conversion by catching KeyError and ValueError. - Added logging for conversion errors to aid in debugging and maintain robustness in tool integration. * feat: enhance GeminiCompletion class with client parameter support (#3717) * feat: enhance GeminiCompletion class with client parameter support - Added support for client_params in the GeminiCompletion class to allow for additional client configuration. - Refactored client initialization into a dedicated method for improved parameter handling. - Introduced a new method to retrieve client parameters, ensuring compatibility with the base class. - Enhanced error handling during client initialization to provide clearer messages for missing configuration. - Updated documentation to reflect the changes in client parameter usage. * add optional dependancies * refactor: update test fixture to mock GOOGLE_API_KEY - Renamed the fixture from `mock_anthropic_api_key` to `mock_google_api_key` to reflect the change in the environment variable being mocked. - This update ensures that all tests in the module can run with a mocked GOOGLE_API_KEY, improving test isolation and reliability. * fix tests * feat: enhance BedrockCompletion class with advanced features * feat: enhance BedrockCompletion class with advanced features and error handling - Added support for guardrail configuration, additional model request fields, and custom response field paths in the BedrockCompletion class. - Improved error handling for AWS exceptions and added token usage tracking with stop reason logging. - Enhanced streaming response handling with comprehensive event management, including tool use and content block processing. - Updated documentation to reflect new features and initialization parameters. - Introduced a new test suite for BedrockCompletion to validate functionality and ensure robust integration with AWS Bedrock APIs. * chore: add boto typing * fix: use typing_extensions.Required for Python 3.10 compatibility --------- Co-authored-by: Greyson Lalonde <greyson.r.lalonde@gmail.com> * feat: azure native tests * feat: add Azure AI Inference support and related tests - Introduced the `azure-ai-inference` package with version `1.0.0b9` and its dependencies in `uv.lock` and `pyproject.toml`. - Added new test files for Azure LLM functionality, including tests for Azure completion and tool handling. - Implemented comprehensive test cases to validate Azure-specific behavior and integration with the CrewAI framework. - Enhanced the testing framework to mock Azure credentials and ensure proper isolation during tests. * feat: enhance AzureCompletion class with Azure OpenAI support - Added support for the Azure OpenAI endpoint in the AzureCompletion class, allowing for flexible endpoint configurations. - Implemented endpoint validation and correction to ensure proper URL formats for Azure OpenAI deployments. - Enhanced error handling to provide clearer messages for common HTTP errors, including authentication and rate limit issues. - Updated tests to validate the new endpoint handling and error messaging, ensuring robust integration with Azure AI Inference. - Refactored parameter preparation to conditionally include the model parameter based on the endpoint type. * refactor: convert project module to metaclass with full typing * Lorenze/OpenAI base url backwards support (#3723) * fix: enhance OpenAICompletion class base URL handling - Updated the base URL assignment in the OpenAICompletion class to prioritize the new `api_base` attribute and fallback to the environment variable `OPENAI_BASE_URL` if both are not set. - Added `api_base` to the list of parameters in the OpenAICompletion class to ensure proper configuration and flexibility in API endpoint management. * feat: enhance OpenAICompletion class with api_base support - Added the `api_base` parameter to the OpenAICompletion class to allow for flexible API endpoint configuration. - Updated the `_get_client_params` method to prioritize `base_url` over `api_base`, ensuring correct URL handling. - Introduced comprehensive tests to validate the behavior of `api_base` and `base_url` in various scenarios, including environment variable fallback. - Enhanced test coverage for client parameter retrieval, ensuring robust integration with the OpenAI API. * fix: improve OpenAICompletion class configuration handling - Added a debug print statement to log the client configuration parameters during initialization for better traceability. - Updated the base URL assignment logic to ensure it defaults to None if no valid base URL is provided, enhancing robustness in API endpoint configuration. - Refined the retrieval of the `api_base` environment variable to streamline the configuration process. * drop print * feat: improvements on import native sdk support (#3725) * feat: add support for Anthropic provider and enhance logging - Introduced the `anthropic` package with version `0.69.0` in `pyproject.toml` and `uv.lock`, allowing for integration with the Anthropic API. - Updated logging in the LLM class to provide clearer error messages when importing native providers, enhancing debugging capabilities. - Improved error handling in the AnthropicCompletion class to guide users on installation via the updated error message format. - Refactored import error handling in other provider classes to maintain consistency in error messaging and installation instructions. * feat: enhance LLM support with Bedrock provider and update dependencies - Added support for the `bedrock` provider in the LLM class, allowing integration with AWS Bedrock APIs. - Updated `uv.lock` to replace `boto3` with `bedrock` in the dependencies, reflecting the new provider structure. - Introduced `SUPPORTED_NATIVE_PROVIDERS` to include `bedrock` and ensure proper error handling when instantiating native providers. - Enhanced error handling in the LLM class to raise informative errors when native provider instantiation fails. - Added tests to validate the behavior of the new Bedrock provider and ensure fallback mechanisms work correctly for unsupported providers. * test: update native provider fallback tests to expect ImportError * adjust the test with the expected bevaior - raising ImportError * this is exoecting the litellm format, all gemini native tests are in test_google.py --------- Co-authored-by: Greyson LaLonde <greyson.r.lalonde@gmail.com> * fix: remove stdout prints, improve test determinism, and update trace handling Removed `print` statements from the `LLMStreamChunkEvent` handler to prevent LLM response chunks from being written directly to stdout. The listener now only tracks chunks internally. Fixes #3715 Added explicit return statements for trace-related tests. Updated cassette for `test_failed_evaluation` to reflect new behavior where an empty trace dict is used instead of returning early. Ensured deterministic cleanup order in test fixtures by making `clear_event_bus_handlers` depend on `setup_test_environment`. This guarantees event bus shutdown and file handle cleanup occur before temporary directory deletion, resolving intermittent “Directory not empty” errors in CI. * chore: remove lib/crewai exclusion from pre-commit hooks * feat: enhance task guardrail functionality and validation * feat: enhance task guardrail functionality and validation - Introduced support for multiple guardrails in the Task class, allowing for sequential processing of guardrails. - Added a new `guardrails` field to the Task model to accept a list of callable guardrails or string descriptions. - Implemented validation to ensure guardrails are processed correctly, including handling of retries and error messages. - Enhanced the `_invoke_guardrail_function` method to manage guardrail execution and integrate with existing task output processing. - Updated tests to cover various scenarios involving multiple guardrails, including success, failure, and retry mechanisms. This update improves the flexibility and robustness of task execution by allowing for more complex validation scenarios. * refactor: enhance guardrail type handling in Task model - Updated the Task class to improve guardrail type definitions, introducing GuardrailType and GuardrailsType for better clarity and type safety. - Simplified the validation logic for guardrails, ensuring that both single and multiple guardrails are processed correctly. - Enhanced error messages for guardrail validation to provide clearer feedback when incorrect types are provided. - This refactor improves the maintainability and robustness of task execution by standardizing guardrail handling. * feat: implement per-guardrail retry tracking in Task model - Introduced a new private attribute `_guardrail_retry_counts` to the Task class for tracking retry attempts on a per-guardrail basis. - Updated the guardrail processing logic to utilize the new retry tracking, allowing for independent retry counts for each guardrail. - Enhanced error handling to provide clearer feedback when guardrails fail validation after exceeding retry limits. - Modified existing tests to validate the new retry tracking behavior, ensuring accurate assertions on guardrail retries. This update improves the robustness and flexibility of task execution by allowing for more granular control over guardrail validation and retry mechanisms. * chore: 1.0.0b3 bump (#3734) * chore: full ruff and mypy improved linting, pre-commit setup, and internal architecture. Configured Ruff to respect .gitignore, added stricter rules, and introduced a lock pre-commit hook with virtualenv activation. Fixed type shadowing in EXASearchTool using a type_ alias to avoid PEP 563 conflicts and resolved circular imports in agent executor and guardrail modules. Removed agent-ops attributes, deprecated watson alias, and dropped crewai-enterprise tools with corresponding test updates. Refactored cache and memoization for thread safety and cleaned up structured output adapters and related logic. * New MCL DSL (#3738) * Adding MCP implementation * New tests for MCP implementation * fix tests * update docs * Revert "New tests for MCP implementation" This reverts commit 0bbe6dee90. * linter * linter * fix * verify mcp pacakge exists * adjust docs to be clear only remote servers are supported * reverted * ensure args schema generated properly * properly close out --------- Co-authored-by: lorenzejay <lorenzejaytech@gmail.com> Co-authored-by: Greyson Lalonde <greyson.r.lalonde@gmail.com> * feat: a2a experimental experimental a2a support --------- Co-authored-by: Lucas Gomide <lucaslg200@gmail.com> Co-authored-by: Greyson LaLonde <greyson.r.lalonde@gmail.com> Co-authored-by: Tony Kipkemboi <iamtonykipkemboi@gmail.com> Co-authored-by: Mike Plachta <mplachta@users.noreply.github.com> Co-authored-by: João Moura <joaomdmoura@gmail.com>	2025-10-20 14:10:19 -07:00