13 Commits

Author	SHA1	Message	Date
João Moura	453676c61a	feat(tools): surface tool failures instead of reporting them as success (#6712) ... * feat(tools): surface tool failures instead of reporting them as success A tool can finish without raising and still fail to do what it was asked. Slack answers HTTP 200 with `{"ok": false, "error": "channel_not_found"}`; an MCP server sets `isError`; a CrewAI AMP action returns `API request failed: ...`. In every case the call "worked", so the error text reached the agent as an ordinary result, the agent narrated the problem in prose, and the run was recorded as a success. Concretely: five failed `slackbot_send_message` calls each rendered as "Tool Execution Completed", the task passed, and the crew passed -- with the only evidence being a sentence in the final answer. Nothing downstream could tell the difference, and an agent that keeps going on a step that silently did nothing builds the rest of its work on it. Give that outcome a type and a reaction: - `ToolFailure` -- what a tool returns instead of an error string. The agent still reads prose via `as_agent_message()`, so model behavior is unchanged; the framework now knows the call failed. - `ToolFailurePolicy` -- `ignore` (previous behavior), `warn` (default: record + emit, keep going), `raise` (abort with `ToolExecutionFailedError`). Resolved most-specific-first: tool, task, agent, crew. - `ToolFailureDetectedEvent` -- emitted before a `raise` aborts, so subscribers always observe the failure. `ToolUsageFinishedEvent` also carries a `failure` field so a trace UI can mark the call failed without correlating two events. - `tool_failures` on `TaskOutput`, `CrewOutput` and `LiteAgentOutput`, plus `has_tool_failures`, so consumers never parse a string. Detection is strictly declarative -- no string sniffing, so a tool that legitimately returns text about an error is never misread as failing. Failures come from a returned `ToolFailure`, a raised exception, MCP `isError`, a spent `max_usage_count`, or an unknown tool. Wired into all four tool-execution paths (the ReAct path and the three native function-calling implementations). Sources updated to report structurally: `MCPClient.call_tool_result()` preserves `isError` that `call_tool()` dropped, and `CrewAIPlatformActionTool` returns a `ToolFailure` for non-2xx and for caught exceptions. Two latent bugs fixed along the way: `ToolUsage` assumed every agent has a `fingerprint` (LiteAgent does not), and policy resolution now tolerates malformed values rather than letting telemetry take down a tool call. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ETacm2dMASfpMAYUiDu5YG * fix(tools): address review round 1 on tool-failure signalling Five real defects from Bugbot, none of them cosmetic. Tool-scoped policy never applied (high). `resolve_tool_failure_policy` read `tool_failure_policy` off the object handed to it, but every execution path passes the `CrewStructuredTool` wrapper, which never carried the attribute -- and `BaseTool` never declared it in the first place. A tool-scoped `raise`/`ignore` was silently ignored while the docs and a unit test claimed otherwise; the test passed only because it called the resolver directly with an authored tool. Declared the field on `BaseTool`, propagated it through `to_structured_tool()` and `CrewStructuredTool`, and made resolution fall back through `_original_tool` so either shape works. A failed call still printed the green "Completed" panel, then the red one. That is the terminal version of the exact bug this PR is about. Suppressed the success panel when the call reported failure. A raised tool printed twice: `ToolUsageErrorEvent` already renders a red panel, and the new failure panel repeated it. The event is still emitted -- policy and traces need it -- but the duplicate console output is gone. Both decisions now live in named predicates on `ConsoleFormatter` rather than inline in the listener closure, so they are directly testable. Unknown tools were reported on the ReAct path but silently ignored on all three native paths, so the same miss was loud or silent depending on executor style. Native paths now record `UNKNOWN_TOOL` too. This also surfaced a live `NameError`: ruff had pruned `ToolFailureReason` from `agent_utils` as unused, so the new branch would have crashed at runtime. `LiteAgentOutput` had `tool_failures` but not `has_tool_failures`, which the PR promised on all three output types -- an `AttributeError` for any caller sharing one check across result types. Testing: 16 further tests, 45 total. Two console tests were passing vacuously because `emit()` dispatches sync handlers on a thread pool, so the assertions raced the handler; they now assert on the predicates directly, and the native-path test drains the bus with `flush()` and checks the synchronously-written record. Full suite still matches baseline exactly at 377 pre-existing failures. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ETacm2dMASfpMAYUiDu5YG * chore: update tool specifications * fix(tools): address review round 2 and fix CI type failure CI caught a type error I should have: widening `agent` to accept a `LiteAgent` (so a standalone LiteAgent resolves its own policy) left the declared signatures behind. Widened `execute_tool_and_check_finality`, its async twin, and `ToolCallHookContext` to `Agent | BaseAgent | LiteAgent | None`, which is what those actually receive now. Seven CodeRabbit findings, all verified against the code first: `raise` was being downgraded by three enclosing handlers. With `max_execution_time` set, `_execute_with_timeout` wrapped every exception in `RuntimeError`, so `_check_execution_error` no longer recognized the passthrough and sent the task through the retry loop instead of aborting. `StepExecutor.execute` turned it into `StepResult(success=False)` and let the plan continue. `LiteAgent.kickoff` ran it through `handle_unknown_error` and printed "This is likely a bug - please report it" for what is a deliberate, configured stop. Failure records were dropped on two paths. `reset_tool_failures()` only ran in `_prepare_task_execution`, so `Agent.kickoff()` / `kickoff_async()` — which enter through `_prepare_kickoff` — accumulated records across runs. And a guardrail retry calls `execute_task` again, which resets the agent, so a tool that failed on a blocked attempt vanished from the final output entirely: a run could report zero failures having demonstrably failed one. Failures now accumulate across guardrail attempts. Writing the tests for that surfaced a further miss of my own: `Agent.kickoff()` builds its `LiteAgentOutput` in `agent/core.py` via `AgentExecutor`, not through `LiteAgent`, so `tool_failures` was always empty there regardless of the recording fix. Wired up, and the LiteAgent path now reads from whichever agent the executor was handed (`original_agent` under kickoff, `self` standalone) rather than assuming. `last_tool_failures` returns a copy, so a caller cannot mutate the agent's record or watch it shift mid-run. Testing: 7 further tests, 52 total, covering the timeout wrapper, the retry limit, kickoff reset, the kickoff output path, copy semantics and guardrail accumulation. Full suite matches baseline exactly at 377 pre-existing failures; mypy clean on every changed file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ETacm2dMASfpMAYUiDu5YG * fix(tools): make crew-scoped policy real and close the last raise leak Two findings, and the first was a documented feature that never worked. `resolve_tool_failure_policy` consulted a crew, and the docs advertised crew as a scope, but `Crew` had no `tool_failure_policy` field at all -- and even with one it was unreachable, because `BaseAgent` defaulted the policy to `WARN` rather than `None`, so resolution always stopped at the agent. Crew-level configuration was silently ignored. Fixed by making "inherit" the default everywhere instead of baking `warn` into one layer: `Crew` gains the field, and `BaseAgent`/`LiteAgent` default to `None` like `Task` and `BaseTool` already did. The resolver owns the single fallback, so the chain is genuinely tool > task > agent > crew > warn and the effective default with nothing configured is still `warn`. Reading `agent.tool_failure_policy` now returns `None` (meaning "inherit") rather than `WARN`. The other: `StepExecutor` re-raised `ToolExecutionFailedError` from its outer handler, but the nested handler around the native-to-text tooling fallback still caught it and returned `StepResult(success=False)`. An agent whose LLM lacked native tool calling would therefore not abort under `raise`. That is the third distinct place this exception was being downgraded; it now re-raises there too. Testing: 8 further tests, 60 total, including the full precedence chain walked one level at a time and crew-scoped `raise`/`ignore` driven end-to-end through `kickoff()` rather than only through the resolver -- the gap that let the original crew bug pass review. Full suite matches baseline exactly at 377 pre-existing failures; mypy clean on every changed file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ETacm2dMASfpMAYUiDu5YG * docs: trim comments and docstrings on tool-failure signalling Prose only -- no behavior change. Cut the module docstring, the longer class and method docstrings, the multi-line inline comments, and the verbose Field descriptions down to what actually earns its place. Net 87 lines lighter. Kept the "why" in every case where the reason is non-obvious (why the event fires before a raise, why the policy reads through the tool wrapper, why the bus needs draining in tests) and dropped the restatements of what the code already says. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ETacm2dMASfpMAYUiDu5YG * fix(tools): make ignore truly silent, stop caching failures, close 4 gaps Six findings from the latest review round, all verified against the code before touching it. `ignore` was not silent. `ToolUsageFinishedEvent.failure` was set before the policy ran, so traces still saw a failed call under a policy documented as surfacing nothing. Worse, the console then showed *no* panel at all: green was suppressed because `failure` was present, red was skipped because `ignore` never emits `ToolFailureDetectedEvent`. New `reportable_failure()` resolves the policy before the finished event and drops the flag under `ignore`; wired into all four execution paths. Failures were being cached. `CacheHandler.add` stored a `ToolFailure` like any other result, so a transient error became permanent for the rest of the run and every later hit re-reported a call that never re-ran. The cache now refuses to store declared failures -- fixed at the single choke point rather than at each of the four call sites. A spent `max_usage_count` was invisible on the shared native path. `BaseTool._claim_usage` returned a bare string that only the executors recognising that exact message treated as a failure. It now returns a `ToolFailure` with `USAGE_LIMIT`, so every path records it. A guardrail returning a whole `TaskOutput` replaced the output without carrying accumulated failures over, so earlier attempts vanished from `CrewOutput.tool_failures`. New `merge_tool_failures()` combines and deduplicates, and the retry-rebuild path uses it too. A hook-blocked call inherited a cached failure and attributed it to a call that never ran. Now cleared. Not reachable through the built-in cache once failures stop being cached, so the test injects a custom cache handler that does retain them -- verified to fail without the guard. Also removed a `datetime` import left unused by the earlier console-test rewrite. Testing: 13 further tests, 73 total. Full suite matches baseline exactly at 377 pre-existing failures; the usage-limit suites that `_claim_usage` touches pass unchanged; mypy clean on every changed file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ETacm2dMASfpMAYUiDu5YG * fix(tools): let raise through the parallel native path, guard all handlers Chasing down CodeRabbit's note about callers of execute_single_native_tool_call turned up a fifth place this exception was being downgraded: the experimental executor's parallel branch wrapped future.result() in a broad except and folded the abort into a fake tool result, so the remaining parallel calls carried on. The sequential path and crew_agent_executor's parallel branch were already fine. Five separate handlers have swallowed this during review, so added a guard test asserting the passthrough at every site rather than trusting the next one gets spotted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ETacm2dMASfpMAYUiDu5YG * fix(tools): keep a failed tool out of the final answer, finish crew scope Three more findings, all confirmed against the code. A failed `result_as_answer` tool still became the task's output. The native paths already excluded raised errors and hook blocks from short-circuiting, but not declared failures -- so an error message silently became the answer, which is the exact shape of bug this PR exists to prevent. Fixed on all paths, and there were three independent override points, not one: `ToolResult.result_as_answer` in tool_utils, the `execution_result` finality checks in both executors, and `process_tool_results()`, which reads `agent.tools_results` back separately. The first two fixes alone left the behavior unchanged; only the third made the test pass. `ToolUsage` never received a crew, so a crew-level `ignore` half-applied: recording and `ToolFailureDetectedEvent` stayed quiet, but the flag was still attached to `ToolUsageFinishedEvent`. It now takes and stores `crew`. `CrewAgentExecutor.invoke`/`ainvoke` routed a deliberate stop through `handle_unknown_error`, printing "An unknown error occurred" on verbose runs. LiteAgent already special-cased this; both now do. Testing: 5 further tests, 79 total, including that a *successful* `result_as_answer` tool still short-circuits. Full suite matches baseline exactly at 377 pre-existing failures; mypy clean on every changed file. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ETacm2dMASfpMAYUiDu5YG * fix(tools): report malformed tool args, correlate the failure event Two findings from the latest round. Malformed native tool arguments returned early with a plain error dict and never reported a failure, so `ToolFailureReason.INVALID_INPUT` was declared but unreferenced -- a bad tool call was absent from records, events and `raise` aborts. `parse_tool_call_args` now carries an INVALID_INPUT failure on the error dict and both executors report it before returning. `ToolFailureDetectedEvent` never set `agent_id`, so a trace could not tie it to a specific agent instance. Fixing that exposed the same gap running the other way: `ToolUsage`'s own started/finished/error events never set `agent_id` either, so on the ReAct path the paired finished event had nothing to correlate against. Both now set it. Set explicitly rather than via `from_agent`, which would also overwrite `agent_role` and lose the `_original_role` preference those events already apply -- a behavior change that has nothing to do with correlation. Testing: 5 further tests, 84 total, asserting the ids match between the failure event and its paired finished event. One existing test pinned the exact key set of the parse-error dict and was updated for the new key. Full suite matches baseline at 377 pre-existing failures; the one apparent addition was the known `test_trace_enable_disable` order-flake, confirmed by re-running rather than assumed. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ETacm2dMASfpMAYUiDu5YG * fix(tools): scope failure accumulation per execution, drop deprecated executor Two review requests from @lorenzejay. Accumulation no longer lives as mutable state on the shared agent. A ContextVar collector is opened around each execution -- task, kickoff, and each guardrail retry -- and the output reads that collector directly instead of copying the agent's list. ContextVars are copied per asyncio task and per thread, so concurrent executions cannot see each other's records, and nesting is safe for retries. `last_tool_failures` prefers the active collector and falls back to the last completed execution, so the accessor is correct during a run too. The per-execution reset that caused the erasure is gone. Reproducing this took some digging and the finding is worth recording: crew tasks *cannot* hit it, because `AgentExecutor` refuses concurrent reuse of one instance and raises. `agent.kickoff()` has no such guard, and there the bug reproduces exactly as reported -- two concurrent kickoffs each returned two records. The regression test forces the overlap with a barrier so it is deterministic rather than timing-dependent, and I verified it reports [2, 2] against the old behavior and [1, 1] now. Removed the tool-failure integration from `CrewAgentExecutor` entirely; that file is back to its state on main. Note the shared ReAct helper it calls still records failures, since that is common code rather than new behavior in the deprecated file -- so a `raise` policy will be swallowed by that executor's generic handler. Flagged on the PR rather than papered over. Testing: 89 total. Two tests I wrote for this were vacuous on the first attempt -- they passed against the simulated pre-fix code -- so each concurrency test was checked against the old behavior before being kept. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ETacm2dMASfpMAYUiDu5YG * fix(tools): report malformed calls everywhere, drop the unused block reason Four findings. `execute_single_native_tool_call` swallowed a JSON decode error into an empty args dict and ran the tool with no input at all -- worse than not reporting it. It now routes through `parse_tool_call_args` like the executors do, so the StepExecutor/planning path reports INVALID_INPUT and returns instead of executing. That also removes a duplicated inline parse. The ReAct path returned a `ToolUsageError` message as an ordinary result without reporting it, so a malformed call there was invisible while the equivalent native failure was recorded. Now reported as INVALID_INPUT too. `Agent.kickoff` opened a collector but no longer reset the agent-level list, so `last_tool_failures` grew across kickoffs. Reset restored, matching task execution. `ToolFailureReason.BLOCKED_BY_HOOK` was declared and never produced. Rather than start reporting hook blocks as failures, the member is removed: a block is a deliberate decision by the hook author, and treating it as a failure would make `raise` abort on an intentional veto. Added a guard test that every remaining reason is actually produced somewhere, so a dead member cannot reappear -- the same smell that flagged INVALID_INPUT last round. Also switched the deprecation guard test to a single import style. Testing: 6 further tests, 95 total, including that the tool does not run when its args fail to parse. Full suite matches baseline at 377 pre-existing failures; mypy clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ETacm2dMASfpMAYUiDu5YG * fix(tools): merge failures across kickoff guardrail retries, cancel siblings Kickoff guardrail retries discarded the blocked attempt's failures. Each retry calls `_execute_and_build_output`, which opens a fresh collector and builds a new output, so a run could report zero failures having demonstrably failed one -- the same bug already fixed on the task guardrail path, which merges. Now merged there too. Verified the test fails without the fix. Under `raise`, one parallel native tool aborting left its siblings running: the pool waited for them and pending ones still started. It now shuts the pool down with `cancel_futures=True` so a not-yet-started sibling never runs. Threads already in flight cannot be interrupted in Python, so a concurrent tool may still complete before the abort surfaces; that is noted at the call site rather than left implied. Also satisfied CodeQL by materialising the enum in the guard test's loop. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01ETacm2dMASfpMAYUiDu5YG --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com>	2026-07-29 19:30:14 -07:00
João Moura	cc0759854d	fix(skills): resolve registry skills through the runtime's CrewAI+ client (#6658) ... * fix(skills): resolve registry skills through the installed AMP client Skill downloads built their own `PlusAPI` and authenticated it from `CREWAI_USER_PAT`, the platform integration token, or the saved CLI login. Managed runtimes have no user credential to offer: they install a client of their own, which `load_agent_from_repository` already resolves through, so Agent Repository lookups worked while the skill downloads beside them failed with 401. Skills now resolve their client the same way, via `resolve_plus_client()` next to the hook it reads. A client that can't fetch skills falls back to environment credentials and warns, so older runtimes behave as they do today. `resolve_plus_response()` shares the sync/async bridging both lookups need, since `PlusAPI` is synchronous while managed clients are not. Version pinning, which the same bug was hiding: - Registry refs accept `@org/name@version`, and `@org/name@v1.2.0` since people write it both ways. `parse_skill_ref()` returns a `SkillRef(org, name, version)`; `parse_registry_ref()` keeps its `(org, name)` shape and drops the pin, so existing callers are unaffected - Agent Repository agents record a version per skill, which was parsed off the response and dropped. Those pins now travel with the refs, so publishing a new version of a skill no longer changes every agent that uses it - A pinned ref only accepts a project-local copy declaring that version in its `metadata.version` frontmatter, and the cache reports a miss when the version it recorded differs — so a pin re-resolves rather than loading another version. Unpinned refs keep hitting the cache as before - An unknown pin fails instead of quietly falling back to the newest version Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(skills): reject a blank version pin instead of floating to latest A blank `version` passed to `download_skill` read as "unpinned" and quietly resolved the latest version, which is not what a caller supplying one asked for — and it disagreed with `parse_skill_ref`, which already rejects empty pins. Not reachable through `resolve_registry_ref` or the Agent Repository auto-pinning, both of which only ever pass a non-empty version. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(skills): carry the caller's context into the worker thread When resolve_plus_response bridges an async client from inside a running loop it runs the coroutine on a worker thread, which starts with empty ContextVars. A client reading runtime state there — the platform integration token, flow context — would see defaults rather than the caller's values, which is hard to diagnose from the resulting auth or routing failure. Copy the context across, matching how the parallel-summarization bridge in this module already does it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>	2026-07-26 13:27:29 -03:00
Vinicius Brasil	9db2d44766	Add typed output schemas for CrewAI tools (#6236) ... Currently, tools have a strong input contract through `args_schema`, but no output contract. This means that anything a tool outputs is converted to string. Not only the contract is weak, but the "invisible" conversion to string can have unexpected effects when the tool returns complex objects like dicts and arrays. With this PR, a tool can _optionally_ define an output contract with `output_schema`. CrewAI validates the raw result and sends the agent JSON. ```python class ProductResult(BaseModel): sku: str name: str in_stock: bool class ProductLookupTool(BaseTool): name: str = "Product Lookup" description: str = "Look up product availability by SKU." def _run(self, sku: str) -> ProductResult: return ProductResult(sku=sku, name="USB-C dock", in_stock=True) ``` If the result does not match the schema, CrewAI warns and falls back to `str(raw_result)` instead of failing the run: ```python @tool("Product Lookup", output_schema=ProductResult) def product_lookup(sku: str) -> dict[str, object]: return {"sku": sku, "name": "USB-C dock", "in_stock": True} #=> RuntimeWarning: Failed to validate or serialize output from tool 'Bad Product Lookup' using output_schema 'ProductResult'... Falling back to str(raw_result). ``` This is additive and non-breaking. Existing tools do not need to change. Tools without `output_schema` keep the old string behavior. Invalid typed outputs warn and fall back to the old formatting path.	2026-06-19 14:33:51 -07:00
Greyson LaLonde	fd10c64148	chore(crewai): drop self-explanatory comments	2026-05-26 10:23:33 -07:00
Greyson LaLonde	095f796922	fix: prevent result_as_answer from returning hook-block message as final answer	2026-05-04 19:42:07 +08:00
Zamuldinov Nikita	bfbdba426f	fix: prevent result_as_answer from returning error as final answer ... When a tool with result_as_answer=True raises an exception, the agent was receiving result_as_answer=True and returning the error string as the final answer. Now we set result_as_answer=False when an error event is emitted, allowing the agent to reflect and retry. Fixes crewAIInc/crewAI#5156 --------- Co-authored-by: NIK-TIGER-BILL <nik.tiger.bill@github.com> Co-authored-by: Greyson LaLonde <greyson.r.lalonde@gmail.com>	2026-05-04 19:28:21 +08:00
Greyson LaLonde	0e590ff669	refactor: use shared I18N_DEFAULT singleton	2026-04-09 04:29:53 +08:00
Greyson LaLonde	1ac5801578	fix: inject tool errors as observations and resolve name collisions	2026-03-01 00:46:04 -05:00
Lucas Gomide	09e3b81ca3	fix: preserve null types in tool parameter schemas for LLM (#4579) ... * fix: preserve null types in tool parameter schemas for LLM Tool parameter schemas were stripping null from optional fields via generate_model_description, forcing the LLM to provide non-null values for fields. Adds strip_null_types parameter to generate_model_description and passes False when generating tool schemas, so optional fields keep anyOf: [{type: T}, {type: null}] * Update lib/crewai/src/crewai/utilities/pydantic_schema_utils.py Co-authored-by: Gabe Milani <gabriel@crewai.com> --------- Co-authored-by: Greyson LaLonde <greyson.r.lalonde@gmail.com> Co-authored-by: Gabe Milani <gabriel@crewai.com> Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com>	2026-02-26 11:51:34 -05:00
João Moura	8102d0a6ca	feat: enhance JSON argument parsing and validation in CrewAgentExecutor and BaseTool ... * feat: enhance JSON argument parsing and validation in CrewAgentExecutor and BaseTool - Added error handling for malformed JSON tool arguments in CrewAgentExecutor, providing descriptive error messages. - Implemented schema validation for tool arguments in BaseTool, ensuring that invalid arguments raise appropriate exceptions. - Introduced tests to verify correct behavior for both valid and invalid JSON inputs, enhancing robustness of tool execution. * refactor: improve argument validation in BaseTool - Introduced a new private method to handle argument validation for tools, enhancing code clarity and reusability. - Updated the method to utilize the new validation method, ensuring consistent error handling for invalid arguments. - Enhanced exception handling to specifically catch , providing clearer error messages for tool argument validation failures. * feat: introduce parse_tool_call_args for improved argument parsing - Added a new utility function, parse_tool_call_args, to handle parsing of tool call arguments from JSON strings or dictionaries, enhancing error handling for malformed JSON inputs. - Updated CrewAgentExecutor and AgentExecutor to utilize the new parsing function, streamlining argument validation and improving clarity in error reporting. - Introduced unit tests for parse_tool_call_args to ensure robust functionality and correct handling of various input scenarios. * feat: add keyword argument validation in BaseTool and Tool classes - Introduced a new method `_validate_kwargs` in BaseTool to validate keyword arguments against the defined schema, ensuring proper argument handling. - Updated the `run` and `arun` methods in both BaseTool and Tool classes to utilize the new validation method, improving error handling and robustness. - Added comprehensive tests for asynchronous execution in `TestBaseToolArunValidation` to verify correct behavior for valid and invalid keyword arguments. * Potential fix for pull request finding 'Syntax error' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> --------- Co-authored-by: lorenzejay <lorenzejaytech@gmail.com> Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com> Co-authored-by: Greyson LaLonde <greyson.r.lalonde@gmail.com> Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>	2026-02-25 13:13:31 -05:00
Lorenze Jay	2ed0c2c043	imp compaction (#4399) ... * imp compaction * fix lint * cassette gen * cassette gen * improve assert * adding azure * fix global docstring	2026-02-11 15:52:03 -08:00
Greyson LaLonde	c4c9208229	feat: native multimodal file handling; openai responses api ... - add input_files parameter to Crew.kickoff(), Flow.kickoff(), Task, and Agent.kickoff() - add provider-specific file uploaders for OpenAI, Anthropic, Gemini, and Bedrock - add file type detection, constraint validation, and automatic format conversion - add URL file source support for multimodal content - add streaming uploads for large files - add prompt caching support for Anthropic - add OpenAI Responses API support	2026-01-23 15:13:25 -05:00
Lorenze Jay	bd4d039f63	Lorenze/imp/native tool calling (#4258) ... * wip restrcuturing agent executor and liteagent * fix: handle None task in AgentExecutor to prevent errors Added a check to ensure that if the task is None, the method returns early without attempting to access task properties. This change improves the robustness of the AgentExecutor by preventing potential errors when the task is not set. * refactor: streamline AgentExecutor initialization by removing redundant parameters Updated the Agent class to simplify the initialization of the AgentExecutor by removing unnecessary task and crew parameters in standalone mode. This change enhances code clarity and maintains backward compatibility by ensuring that the executor is correctly configured without redundant assignments. * wip: clean * ensure executors work inside a flow due to flow in flow async structure * refactor: enhance agent kickoff preparation by separating common logic Updated the Agent class to introduce a new private method that consolidates the common setup logic for both synchronous and asynchronous kickoff executions. This change improves code clarity and maintainability by reducing redundancy in the kickoff process, while ensuring that the agent can still execute effectively within both standalone and flow contexts. * linting and tests * fix test * refactor: improve test for Agent kickoff parameters Updated the test for the Agent class to ensure that the kickoff method correctly preserves parameters. The test now verifies the configuration of the agent after kickoff, enhancing clarity and maintainability. Additionally, the test for asynchronous kickoff within a flow context has been updated to reflect the Agent class instead of LiteAgent. * refactor: update test task guardrail process output for improved validation Refactored the test for task guardrail process output to enhance the validation of the output against the OpenAPI schema. The changes include a more structured request body and updated response handling to ensure compliance with the guardrail requirements. This update aims to improve the clarity and reliability of the test cases, ensuring that task outputs are correctly validated and feedback is appropriately provided. * test fix cassette * test fix cassette * working * working cassette * refactor: streamline agent execution and enhance flow compatibility Refactored the Agent class to simplify the execution method by removing the event loop check and clarifying the behavior when called from synchronous and asynchronous contexts. The changes ensure that the method operates seamlessly within flow methods, improving clarity in the documentation. Additionally, updated the AgentExecutor to set the response model to None, enhancing flexibility. New test cassettes were added to validate the functionality of agents within flow contexts, ensuring robust testing for both synchronous and asynchronous operations. * fixed cassette * Enhance Flow Execution Logic - Introduced conditional execution for start methods in the Flow class. - Unconditional start methods are prioritized during kickoff, while conditional starts are executed only if no unconditional starts are present. - Improved handling of cyclic flows by allowing re-execution of conditional start methods triggered by routers. - Added checks to continue execution chains for completed conditional starts. These changes improve the flexibility and control of flow execution, ensuring that the correct methods are triggered based on the defined conditions. * Enhance Agent and Flow Execution Logic - Updated the Agent class to automatically detect the event loop and return a coroutine when called within a Flow, simplifying async handling for users. - Modified Flow class to execute listeners sequentially, preventing race conditions on shared state during listener execution. - Improved handling of coroutine results from synchronous methods, ensuring proper execution flow and state management. These changes enhance the overall execution logic and user experience when working with agents and flows in CrewAI. * Enhance Flow Listener Logic and Agent Imports - Updated the Flow class to track fired OR listeners, ensuring that multi-source OR listeners only trigger once during execution. This prevents redundant executions and improves flow efficiency. - Cleared fired OR listeners during cyclic flow resets to allow re-execution in new cycles. - Modified the Agent class imports to include Coroutine from collections.abc, enhancing type handling for asynchronous operations. These changes improve the control and performance of flow execution in CrewAI, ensuring more predictable behavior in complex scenarios. * adjusted test due to new cassette * ensure native tool calling works with liteagent * ensure response model is respected * Enhance Tool Name Handling for LLM Compatibility - Added a new function to replace invalid characters in function names with underscores, ensuring compatibility with LLM providers. - Updated the function to sanitize tool names before validation. - Modified the function to use sanitized names for tool registration. These changes improve the robustness of tool name handling, preventing potential issues with invalid characters in function names. * ensure we dont finalize batch on just a liteagent finishing * max tools per turn wip and ensure we drop print times * fix sync main issues * fix llm_call_completed event serialization issue * drop max_tools_iterations * for fixing model dump with state * Add extract_tool_call_info function to handle various tool call formats - Introduced a new utility function to extract tool call ID, name, and arguments from different provider formats (OpenAI, Gemini, Anthropic, and dictionary). - This enhancement improves the flexibility and compatibility of tool calls across multiple LLM providers, ensuring consistent handling of tool call information. - The function returns a tuple containing the call ID, function name, and function arguments, or None if the format is unrecognized. * Refactor AgentExecutor to support batch execution of native tool calls - Updated the method to process all tools from in a single batch, enhancing efficiency and reducing the number of interactions with the LLM. - Introduced a new utility function to streamline the extraction of tool call details, improving compatibility with various tool formats. - Removed the parameter, simplifying the initialization of the . - Enhanced logging and message handling to provide clearer insights during tool execution. - This refactor improves the overall performance and usability of the agent execution flow. * Update English translations for tool usage and reasoning instructions - Revised the `post_tool_reasoning` message to clarify the analysis process after tool usage, emphasizing the need to provide only the final answer if requirements are met. - Updated the `format` message to simplify the instructions for deciding between using a tool or providing a final answer, enhancing clarity for users. - These changes improve the overall user experience by providing clearer guidance on task execution and response formatting. * fix * fixing azure tests * organizae imports * dropped unused * Remove debug print statements from AgentExecutor to clean up the code and improve readability. This change enhances the overall performance of the agent execution flow by eliminating unnecessary console output during LLM calls and iterations. * linted * updated cassette * regen cassette * revert crew agent executor * adjust cassettes and dropped tests due to native tool implementation * adjust * ensure we properly fail tools and emit their events * Enhance tool handling and delegation tracking in agent executors - Implemented immediate return for tools with result_as_answer=True in crew_agent_executor.py. - Added delegation tracking functionality in agent_utils.py to increment delegations when specific tools are used. - Updated tool usage logic to handle caching more effectively in tool_usage.py. - Enhanced test cases to validate new delegation features and tool caching behavior. This update improves the efficiency of tool execution and enhances the delegation capabilities of agents. * Enhance tool handling and delegation tracking in agent executors - Implemented immediate return for tools with result_as_answer=True in crew_agent_executor.py. - Added delegation tracking functionality in agent_utils.py to increment delegations when specific tools are used. - Updated tool usage logic to handle caching more effectively in tool_usage.py. - Enhanced test cases to validate new delegation features and tool caching behavior. This update improves the efficiency of tool execution and enhances the delegation capabilities of agents. * fix cassettes * fix * regen cassettes * regen gemini * ensure we support bedrock * supporting bedrock * regen azure cassettes * Implement max usage count tracking for tools in agent executors - Added functionality to check if a tool has reached its maximum usage count before execution in both crew_agent_executor.py and agent_executor.py. - Enhanced error handling to return a message when a tool's usage limit is reached. - Updated tool usage logic in tool_usage.py to increment usage counts and print current usage status. - Introduced tests to validate max usage count behavior for native tool calling, ensuring proper enforcement and tracking. This update improves tool management by preventing overuse and providing clear feedback when limits are reached. * fix other test * fix test * drop logs * better tests * regen * regen all azure cassettes * regen again placeholder for cassette matching * fix: unify tool name sanitization across codebase * fix: include tool role messages in save_last_messages * fix: update sanitize_tool_name test expectations Align test expectations with unified sanitize_tool_name behavior that lowercases and splits camelCase for LLM provider compatibility. * fix: apply sanitize_tool_name consistently across codebase Unify tool name sanitization to ensure consistency between tool names shown to LLMs and tool name matching/lookup logic. * regen * fix: sanitize tool names in native tool call processing - Update extract_tool_call_info to return sanitized tool names - Fix delegation tool name matching to use sanitized names - Add sanitization in crew_agent_executor tool call extraction - Add sanitization in experimental agent_executor - Add sanitization in LLM.call function lookup - Update streaming utility to use sanitized names - Update base_agent_executor_mixin delegation check * Extract text content from parts directly to avoid warning about non-text parts * Add test case for Gemini token usage tracking - Introduced a new YAML cassette for tracking token usage in Gemini API responses. - Updated the test for Gemini to validate token usage metrics and response content. - Ensured proper integration with the Gemini model and API key handling. --------- Co-authored-by: Greyson LaLonde <greyson.r.lalonde@gmail.com>	2026-01-22 17:44:03 -08:00