feat(tools): surface tool failures instead of reporting them as success

A tool can finish without raising and still fail to do what it was asked.
Slack answers HTTP 200 with `{"ok": false, "error": "channel_not_found"}`;
an MCP server sets `isError`; a CrewAI AMP action returns
`API request failed: ...`. In every case the call "worked", so the error
text reached the agent as an ordinary result, the agent narrated the
problem in prose, and the run was recorded as a success.

Concretely: five failed `slackbot_send_message` calls each rendered as
"Tool Execution Completed", the task passed, and the crew passed -- with
the only evidence being a sentence in the final answer. Nothing
downstream could tell the difference, and an agent that keeps going on a
step that silently did nothing builds the rest of its work on it.

Give that outcome a type and a reaction:

- `ToolFailure` -- what a tool returns instead of an error string. The
  agent still reads prose via `as_agent_message()`, so model behavior is
  unchanged; the framework now knows the call failed.
- `ToolFailurePolicy` -- `ignore` (previous behavior), `warn` (default:
  record + emit, keep going), `raise` (abort with
  `ToolExecutionFailedError`). Resolved most-specific-first: tool, task,
  agent, crew.
- `ToolFailureDetectedEvent` -- emitted before a `raise` aborts, so
  subscribers always observe the failure. `ToolUsageFinishedEvent` also
  carries a `failure` field so a trace UI can mark the call failed
  without correlating two events.
- `tool_failures` on `TaskOutput`, `CrewOutput` and `LiteAgentOutput`,
  plus `has_tool_failures`, so consumers never parse a string.

Detection is strictly declarative -- no string sniffing, so a tool that
legitimately returns text about an error is never misread as failing.
Failures come from a returned `ToolFailure`, a raised exception, MCP
`isError`, a spent `max_usage_count`, or an unknown tool.

Wired into all four tool-execution paths (the ReAct path and the three
native function-calling implementations). Sources updated to report
structurally: `MCPClient.call_tool_result()` preserves `isError` that
`call_tool()` dropped, and `CrewAIPlatformActionTool` returns a
`ToolFailure` for non-2xx and for caught exceptions.

Two latent bugs fixed along the way: `ToolUsage` assumed every agent has
a `fingerprint` (LiteAgent does not), and policy resolution now tolerates
malformed values rather than letting telemetry take down a tool call.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ETacm2dMASfpMAYUiDu5YG
This commit is contained in:
Joao Moura
2026-07-28 23:02:59 -07:00
parent f15844b219
commit 279c57fba3
28 changed files with 1346 additions and 21 deletions

View File

@@ -334,6 +334,118 @@ writer1 = Agent(
#...
```
## Reporting Tool Failures
A tool can finish without raising and still fail to do what it was asked. Slack
answers `HTTP 200` with `{"ok": false, "error": "channel_not_found"}`; an MCP
server sets `isError`; a platform action returns an error payload. The tool call
"worked", so the error text reaches the agent as an ordinary result — the agent
narrates the problem in its final answer and the run is recorded as a success.
Return a `ToolFailure` instead of an error string and the framework can tell the
difference:
```python Code
from typing import Any
from crewai.tools import BaseTool
from crewai.tools.tool_failure import ToolFailure
class SendSlackMessage(BaseTool):
name: str = "send_slack_message"
description: str = "Post a message to a Slack channel."
def _run(self, channel: str, text: str) -> Any:
payload = slack.post(channel=channel, text=text)
if not payload["ok"]:
return ToolFailure(
message=f"Slack rejected the message: {payload['error']}",
code=payload["error"],
retryable=payload["error"] == "rate_limited",
)
return payload
```
The agent still reads plain prose — `ToolFailure.as_agent_message()` renders the
message — so model behavior is unchanged. What changes is that the failure is now
visible to everything downstream.
Detection is strictly declarative. CrewAI never guesses whether a string "looks
like" an error, so a tool that legitimately returns text about an error is never
misread as having failed. Failures are recorded when a tool returns a
`ToolFailure`, when a tool raises, when an MCP server sets `isError`, when a
tool's `max_usage_count` is spent, or when the agent calls a tool that does not exist.
### Choosing a Failure Policy
`tool_failure_policy` controls what happens next:
| Policy | Behavior |
| :-- | :-- |
| `ignore` | Nothing is recorded, emitted, or acted on. |
| `warn` *(default)* | Records the failure, emits `ToolFailureDetectedEvent`, and continues. |
| `raise` | Records and emits, then aborts with `ToolExecutionFailedError`. |
```python Code
from crewai import Agent, Task
from crewai.tools.tool_failure import ToolFailurePolicy
agent = Agent(
role="Slack Messenger",
goal="Post the report to Slack",
backstory="...",
tools=[SendSlackMessage()],
tool_failure_policy=ToolFailurePolicy.WARN,
)
# Tighten a single high-stakes task without changing the agent.
task = Task(
description="Post the final report to #engineering",
expected_output="Confirmation the message was posted",
agent=agent,
tool_failure_policy=ToolFailurePolicy.RAISE,
)
```
The most specific setting wins: tool, then task, then agent, then crew, then the
`warn` default.
### Inspecting Failures
Recorded failures are structured, so nothing downstream has to parse a string:
```python Code
result = crew.kickoff()
if result.has_tool_failures:
for record in result.tool_failures:
print(record.tool_name) # "send_slack_message"
print(record.failure.code) # "channel_not_found"
print(record.failure.reason) # ToolFailureReason.TOOL_REPORTED
print(record.summary())
```
`tool_failures` is available on `TaskOutput`, `CrewOutput`, and
`LiteAgentOutput`. A crew can finish successfully with a non-empty list — check
it before treating `raw` as complete.
To react as failures happen, subscribe to the event:
```python Code
from crewai.events import ToolFailureDetectedEvent
from crewai.events.event_bus import crewai_event_bus
@crewai_event_bus.on(ToolFailureDetectedEvent)
def on_tool_failure(source, event):
print(f"{event.tool_name} failed: {event.failure.message} ({event.policy})")
```
The event is emitted before the `raise` policy aborts, so subscribers always
observe the failure. `ToolUsageFinishedEvent` also carries a `failure` field, letting
a trace UI mark the call as failed without correlating two events.
## Conclusion
Tools are pivotal in extending the capabilities of CrewAI agents, enabling them to undertake a broad spectrum of tasks and collaborate effectively.