Compare commits

..

1 Commits

Author SHA1 Message Date
Devin AI
02f9a36acb Fix Anthropic max_tokens issue causing slow execution (issue #3807)
Make max_tokens optional and compute dynamically when not set by user.

Previously, max_tokens defaulted to 4096 and was always passed to the
Anthropic API, causing the model to generate up to 4096 tokens even for
simple queries that should only need a few tokens. This resulted in
extremely slow execution times.

Changes:
- Changed max_tokens parameter from int (default 4096) to int | None (default None)
- Added dynamic computation in _prepare_completion_params():
  - Default: 1024 tokens (much more reasonable for most queries)
  - Large context models (200k+): 2048 tokens
  - User-specified values are always respected
- Updated docstring to reflect that max_tokens is now optional
- Added comprehensive tests covering:
  - Explicit max_tokens values are passed through unchanged
  - Default behavior computes reasonable max_tokens dynamically
  - max_tokens=None uses dynamic computation
  - Dynamic values are appropriate for model context window size
  - User-provided values are always respected

This fix aligns with the v0.203.1 behavior where max_tokens was optional
and only passed when explicitly set, while maintaining compatibility with
the Anthropic SDK requirement that max_tokens must be provided.

Fixes #3807

Co-Authored-By: João <joao@crewai.com>
2025-10-29 00:26:09 +00:00
205 changed files with 18045 additions and 38663 deletions

View File

@@ -1,11 +0,0 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
version: 2
updates:
- package-ecosystem: uv # See documentation for possible values
directory: "/" # Location of package manifests
schedule:
interval: "weekly"

View File

@@ -19,7 +19,6 @@ repos:
language: system
pass_filenames: true
types: [python]
exclude: ^(lib/crewai/src/crewai/cli/templates/|lib/crewai/tests/|lib/crewai-tools/tests/)
- repo: https://github.com/astral-sh/uv-pre-commit
rev: 0.9.3
hooks:

View File

@@ -1200,52 +1200,6 @@ Learn how to get the most out of your LLM configuration:
)
```
</Accordion>
<Accordion title="Transport Interceptors">
CrewAI provides message interceptors for several providers, allowing you to hook into request/response cycles at the transport layer.
**Supported Providers:**
- ✅ OpenAI
- ✅ Anthropic
**Basic Usage:**
```python
import httpx
from crewai import LLM
from crewai.llms.hooks import BaseInterceptor
class CustomInterceptor(BaseInterceptor[httpx.Request, httpx.Response]):
"""Custom interceptor to modify requests and responses."""
def on_outbound(self, request: httpx.Request) -> httpx.Request:
"""Print request before sending to the LLM provider."""
print(request)
return request
def on_inbound(self, response: httpx.Response) -> httpx.Response:
"""Process response after receiving from the LLM provider."""
print(f"Status: {response.status_code}")
print(f"Response time: {response.elapsed}")
return response
# Use the interceptor with an LLM
llm = LLM(
model="openai/gpt-4o",
interceptor=CustomInterceptor()
)
```
**Important Notes:**
- Both methods must return the received object or type of object.
- Modifying received objects may result in unexpected behavior or application crashes.
- Not all providers support interceptors - check the supported providers list above
<Info>
Interceptors operate at the transport layer. This is particularly useful for:
- Message transformation and filtering
- Debugging API interactions
</Info>
</Accordion>
</AccordionGroup>
## Common Issues and Solutions

View File

@@ -60,7 +60,6 @@ crew = Crew(
| **Output Pydantic** _(optional)_ | `output_pydantic` | `Optional[Type[BaseModel]]` | A Pydantic model for task output. |
| **Callback** _(optional)_ | `callback` | `Optional[Any]` | Function/object to be executed after task completion. |
| **Guardrail** _(optional)_ | `guardrail` | `Optional[Callable]` | Function to validate task output before proceeding to next task. |
| **Guardrails** _(optional)_ | `guardrails` | `Optional[List[Callable] | List[str]]` | List of guardrails to validate task output before proceeding to next task. |
| **Guardrail Max Retries** _(optional)_ | `guardrail_max_retries` | `Optional[int]` | Maximum number of retries when guardrail validation fails. Defaults to 3. |
<Note type="warning" title="Deprecated: max_retries">
@@ -224,7 +223,6 @@ By default, the `TaskOutput` will only include the `raw` output. A `TaskOutput`
| **JSON Dict** | `json_dict` | `Optional[Dict[str, Any]]` | A dictionary representing the JSON output of the task. |
| **Agent** | `agent` | `str` | The agent that executed the task. |
| **Output Format** | `output_format` | `OutputFormat` | The format of the task output, with options including RAW, JSON, and Pydantic. The default is RAW. |
| **Messages** | `messages` | `list[LLMMessage]` | The messages from the last task execution. |
### Task Methods and Properties
@@ -343,11 +341,7 @@ Task guardrails provide a way to validate and transform task outputs before they
are passed to the next task. This feature helps ensure data quality and provides
feedback to agents when their output doesn't meet specific criteria.
CrewAI supports two types of guardrails:
1. **Function-based guardrails**: Python functions with custom validation logic, giving you complete control over the validation process and ensuring reliable, deterministic results.
2. **LLM-based guardrails**: String descriptions that use the agent's LLM to validate outputs based on natural language criteria. These are ideal for complex or subjective validation requirements.
Guardrails are implemented as Python functions that contain custom validation logic, giving you complete control over the validation process and ensuring reliable, deterministic results.
### Function-Based Guardrails
@@ -361,12 +355,12 @@ def validate_blog_content(result: TaskOutput) -> Tuple[bool, Any]:
"""Validate blog content meets requirements."""
try:
# Check word count
word_count = len(result.raw.split())
word_count = len(result.split())
if word_count > 200:
return (False, "Blog content exceeds 200 words")
# Additional validation logic here
return (True, result.raw.strip())
return (True, result.strip())
except Exception as e:
return (False, "Unexpected error during validation")
@@ -378,147 +372,6 @@ blog_task = Task(
)
```
### LLM-Based Guardrails (String Descriptions)
Instead of writing custom validation functions, you can use string descriptions that leverage LLM-based validation. When you provide a string to the `guardrail` or `guardrails` parameter, CrewAI automatically creates an `LLMGuardrail` that uses the agent's LLM to validate the output based on your description.
**Requirements**:
- The task must have an `agent` assigned (the guardrail uses the agent's LLM)
- Provide a clear, descriptive string explaining the validation criteria
```python Code
from crewai import Task
# Single LLM-based guardrail
blog_task = Task(
description="Write a blog post about AI",
expected_output="A blog post under 200 words",
agent=blog_agent,
guardrail="The blog post must be under 200 words and contain no technical jargon"
)
```
LLM-based guardrails are particularly useful for:
- **Complex validation logic** that's difficult to express programmatically
- **Subjective criteria** like tone, style, or quality assessments
- **Natural language requirements** that are easier to describe than code
The LLM guardrail will:
1. Analyze the task output against your description
2. Return `(True, output)` if the output complies with the criteria
3. Return `(False, feedback)` with specific feedback if validation fails
**Example with detailed validation criteria**:
```python Code
research_task = Task(
description="Research the latest developments in quantum computing",
expected_output="A comprehensive research report",
agent=researcher_agent,
guardrail="""
The research report must:
- Be at least 1000 words long
- Include at least 5 credible sources
- Cover both technical and practical applications
- Be written in a professional, academic tone
- Avoid speculation or unverified claims
"""
)
```
### Multiple Guardrails
You can apply multiple guardrails to a task using the `guardrails` parameter. Multiple guardrails are executed sequentially, with each guardrail receiving the output from the previous one. This allows you to chain validation and transformation steps.
The `guardrails` parameter accepts:
- A list of guardrail functions or string descriptions
- A single guardrail function or string (same as `guardrail`)
**Note**: If `guardrails` is provided, it takes precedence over `guardrail`. The `guardrail` parameter will be ignored when `guardrails` is set.
```python Code
from typing import Tuple, Any
from crewai import TaskOutput, Task
def validate_word_count(result: TaskOutput) -> Tuple[bool, Any]:
"""Validate word count is within limits."""
word_count = len(result.raw.split())
if word_count < 100:
return (False, f"Content too short: {word_count} words. Need at least 100 words.")
if word_count > 500:
return (False, f"Content too long: {word_count} words. Maximum is 500 words.")
return (True, result.raw)
def validate_no_profanity(result: TaskOutput) -> Tuple[bool, Any]:
"""Check for inappropriate language."""
profanity_words = ["badword1", "badword2"] # Example list
content_lower = result.raw.lower()
for word in profanity_words:
if word in content_lower:
return (False, f"Inappropriate language detected: {word}")
return (True, result.raw)
def format_output(result: TaskOutput) -> Tuple[bool, Any]:
"""Format and clean the output."""
formatted = result.raw.strip()
# Capitalize first letter
formatted = formatted[0].upper() + formatted[1:] if formatted else formatted
return (True, formatted)
# Apply multiple guardrails sequentially
blog_task = Task(
description="Write a blog post about AI",
expected_output="A well-formatted blog post between 100-500 words",
agent=blog_agent,
guardrails=[
validate_word_count, # First: validate length
validate_no_profanity, # Second: check content
format_output # Third: format the result
],
guardrail_max_retries=3
)
```
In this example, the guardrails execute in order:
1. `validate_word_count` checks the word count
2. `validate_no_profanity` checks for inappropriate language (using the output from step 1)
3. `format_output` formats the final result (using the output from step 2)
If any guardrail fails, the error is sent back to the agent, and the task is retried up to `guardrail_max_retries` times.
**Mixing function-based and LLM-based guardrails**:
You can combine both function-based and string-based guardrails in the same list:
```python Code
from typing import Tuple, Any
from crewai import TaskOutput, Task
def validate_word_count(result: TaskOutput) -> Tuple[bool, Any]:
"""Validate word count is within limits."""
word_count = len(result.raw.split())
if word_count < 100:
return (False, f"Content too short: {word_count} words. Need at least 100 words.")
if word_count > 500:
return (False, f"Content too long: {word_count} words. Maximum is 500 words.")
return (True, result.raw)
# Mix function-based and LLM-based guardrails
blog_task = Task(
description="Write a blog post about AI",
expected_output="A well-formatted blog post between 100-500 words",
agent=blog_agent,
guardrails=[
validate_word_count, # Function-based: precise word count check
"The content must be engaging and suitable for a general audience", # LLM-based: subjective quality check
"The writing style should be clear, concise, and free of technical jargon" # LLM-based: style validation
],
guardrail_max_retries=3
)
```
This approach combines the precision of programmatic validation with the flexibility of LLM-based assessment for subjective criteria.
### Guardrail Function Requirements
1. **Function Signature**:

View File

@@ -1,291 +0,0 @@
---
title: Agent-to-Agent (A2A) Protocol
description: Enable CrewAI agents to delegate tasks to remote A2A-compliant agents for specialized handling
icon: network-wired
mode: "wide"
---
## A2A Agent Delegation
CrewAI supports the Agent-to-Agent (A2A) protocol, allowing agents to delegate tasks to remote specialized agents. The agent's LLM automatically decides whether to handle a task directly or delegate to an A2A agent based on the task requirements.
<Note>
A2A delegation requires the `a2a-sdk` package. Install with: `uv add 'crewai[a2a]'` or `pip install 'crewai[a2a]'`
</Note>
## How It Works
When an agent is configured with A2A capabilities:
1. The LLM analyzes each task
2. It decides to either:
- Handle the task directly using its own capabilities
- Delegate to a remote A2A agent for specialized handling
3. If delegating, the agent communicates with the remote A2A agent through the protocol
4. Results are returned to the CrewAI workflow
## Basic Configuration
Configure an agent for A2A delegation by setting the `a2a` parameter:
```python Code
from crewai import Agent, Crew, Task
from crewai.a2a import A2AConfig
agent = Agent(
role="Research Coordinator",
goal="Coordinate research tasks efficiently",
backstory="Expert at delegating to specialized research agents",
llm="gpt-4o",
a2a=A2AConfig(
endpoint="https://example.com/.well-known/agent-card.json",
timeout=120,
max_turns=10
)
)
task = Task(
description="Research the latest developments in quantum computing",
expected_output="A comprehensive research report",
agent=agent
)
crew = Crew(agents=[agent], tasks=[task], verbose=True)
result = crew.kickoff()
```
## Configuration Options
The `A2AConfig` class accepts the following parameters:
<ParamField path="endpoint" type="str" required>
The A2A agent endpoint URL (typically points to `.well-known/agent-card.json`)
</ParamField>
<ParamField path="auth" type="AuthScheme" default="None">
Authentication scheme for the A2A agent. Supports Bearer tokens, OAuth2, API keys, and HTTP authentication.
</ParamField>
<ParamField path="timeout" type="int" default="120">
Request timeout in seconds
</ParamField>
<ParamField path="max_turns" type="int" default="10">
Maximum number of conversation turns with the A2A agent
</ParamField>
<ParamField path="response_model" type="type[BaseModel]" default="None">
Optional Pydantic model for requesting structured output from an A2A agent. A2A protocol does not
enforce this, so an A2A agent does not need to honor this request.
</ParamField>
<ParamField path="fail_fast" type="bool" default="True">
Whether to raise an error immediately if agent connection fails. When `False`, the agent continues with available agents and informs the LLM about unavailable ones.
</ParamField>
## Authentication
For A2A agents that require authentication, use one of the provided auth schemes:
<Tabs>
<Tab title="Bearer Token">
```python Code
from crewai.a2a import A2AConfig
from crewai.a2a.auth import BearerTokenAuth
agent = Agent(
role="Secure Coordinator",
goal="Coordinate tasks with secured agents",
backstory="Manages secure agent communications",
llm="gpt-4o",
a2a=A2AConfig(
endpoint="https://secure-agent.example.com/.well-known/agent-card.json",
auth=BearerTokenAuth(token="your-bearer-token"),
timeout=120
)
)
```
</Tab>
<Tab title="API Key">
```python Code
from crewai.a2a import A2AConfig
from crewai.a2a.auth import APIKeyAuth
agent = Agent(
role="API Coordinator",
goal="Coordinate with API-based agents",
backstory="Manages API-authenticated communications",
llm="gpt-4o",
a2a=A2AConfig(
endpoint="https://api-agent.example.com/.well-known/agent-card.json",
auth=APIKeyAuth(
api_key="your-api-key",
location="header", # or "query" or "cookie"
name="X-API-Key"
),
timeout=120
)
)
```
</Tab>
<Tab title="OAuth2">
```python Code
from crewai.a2a import A2AConfig
from crewai.a2a.auth import OAuth2ClientCredentials
agent = Agent(
role="OAuth Coordinator",
goal="Coordinate with OAuth-secured agents",
backstory="Manages OAuth-authenticated communications",
llm="gpt-4o",
a2a=A2AConfig(
endpoint="https://oauth-agent.example.com/.well-known/agent-card.json",
auth=OAuth2ClientCredentials(
token_url="https://auth.example.com/oauth/token",
client_id="your-client-id",
client_secret="your-client-secret",
scopes=["read", "write"]
),
timeout=120
)
)
```
</Tab>
<Tab title="HTTP Basic">
```python Code
from crewai.a2a import A2AConfig
from crewai.a2a.auth import HTTPBasicAuth
agent = Agent(
role="Basic Auth Coordinator",
goal="Coordinate with basic auth agents",
backstory="Manages basic authentication communications",
llm="gpt-4o",
a2a=A2AConfig(
endpoint="https://basic-agent.example.com/.well-known/agent-card.json",
auth=HTTPBasicAuth(
username="your-username",
password="your-password"
),
timeout=120
)
)
```
</Tab>
</Tabs>
## Multiple A2A Agents
Configure multiple A2A agents for delegation by passing a list:
```python Code
from crewai.a2a import A2AConfig
from crewai.a2a.auth import BearerTokenAuth
agent = Agent(
role="Multi-Agent Coordinator",
goal="Coordinate with multiple specialized agents",
backstory="Expert at delegating to the right specialist",
llm="gpt-4o",
a2a=[
A2AConfig(
endpoint="https://research.example.com/.well-known/agent-card.json",
timeout=120
),
A2AConfig(
endpoint="https://data.example.com/.well-known/agent-card.json",
auth=BearerTokenAuth(token="data-token"),
timeout=90
)
]
)
```
The LLM will automatically choose which A2A agent to delegate to based on the task requirements.
## Error Handling
Control how agent connection failures are handled using the `fail_fast` parameter:
```python Code
from crewai.a2a import A2AConfig
# Fail immediately on connection errors (default)
agent = Agent(
role="Research Coordinator",
goal="Coordinate research tasks",
backstory="Expert at delegation",
llm="gpt-4o",
a2a=A2AConfig(
endpoint="https://research.example.com/.well-known/agent-card.json",
fail_fast=True
)
)
# Continue with available agents
agent = Agent(
role="Multi-Agent Coordinator",
goal="Coordinate with multiple agents",
backstory="Expert at working with available resources",
llm="gpt-4o",
a2a=[
A2AConfig(
endpoint="https://primary.example.com/.well-known/agent-card.json",
fail_fast=False
),
A2AConfig(
endpoint="https://backup.example.com/.well-known/agent-card.json",
fail_fast=False
)
]
)
```
When `fail_fast=False`:
- If some agents fail, the LLM is informed which agents are unavailable and can delegate to working agents
- If all agents fail, the LLM receives a notice about unavailable agents and handles the task directly
- Connection errors are captured and included in the context for better decision-making
## Best Practices
<CardGroup cols={2}>
<Card title="Set Appropriate Timeouts" icon="clock">
Configure timeouts based on expected A2A agent response times. Longer-running tasks may need higher timeout values.
</Card>
<Card title="Limit Conversation Turns" icon="comments">
Use `max_turns` to prevent excessive back-and-forth. The agent will automatically conclude conversations before hitting the limit.
</Card>
<Card title="Use Resilient Error Handling" icon="shield-check">
Set `fail_fast=False` for production environments with multiple agents to gracefully handle connection failures and maintain workflow continuity.
</Card>
<Card title="Secure Your Credentials" icon="lock">
Store authentication tokens and credentials as environment variables, not in code.
</Card>
<Card title="Monitor Delegation Decisions" icon="eye">
Use verbose mode to observe when the LLM chooses to delegate versus handle tasks directly.
</Card>
</CardGroup>
## Supported Authentication Methods
- **Bearer Token** - Simple token-based authentication
- **OAuth2 Client Credentials** - OAuth2 flow for machine-to-machine communication
- **OAuth2 Authorization Code** - OAuth2 flow requiring user authorization
- **API Key** - Key-based authentication (header, query param, or cookie)
- **HTTP Basic** - Username/password authentication
- **HTTP Digest** - Digest authentication (requires `httpx-auth` package)
## Learn More
For more information about the A2A protocol and reference implementations:
- [A2A Protocol Documentation](https://a2a-protocol.org)
- [A2A Sample Implementations](https://github.com/a2aproject/a2a-samples)
- [A2A Python SDK](https://github.com/a2aproject/a2a-python)

View File

@@ -11,13 +11,9 @@ The [Model Context Protocol](https://modelcontextprotocol.io/introduction) (MCP)
CrewAI offers **two approaches** for MCP integration:
### 🚀 **Simple DSL Integration** (Recommended)
### Simple DSL Integration** (Recommended)
Use the `mcps` field directly on agents for seamless MCP tool integration. The DSL supports both **string references** (for quick setup) and **structured configurations** (for full control).
#### String-Based References (Quick Setup)
Perfect for remote HTTPS servers and CrewAI AMP marketplace:
Use the `mcps` field directly on agents for seamless MCP tool integration:
```python
from crewai import Agent
@@ -36,46 +32,6 @@ agent = Agent(
# MCP tools are now automatically available to your agent!
```
#### Structured Configurations (Full Control)
For complete control over connection settings, tool filtering, and all transport types:
```python
from crewai import Agent
from crewai.mcp import MCPServerStdio, MCPServerHTTP, MCPServerSSE
from crewai.mcp.filters import create_static_tool_filter
agent = Agent(
role="Advanced Research Analyst",
goal="Research with full control over MCP connections",
backstory="Expert researcher with advanced tool access",
mcps=[
# Stdio transport for local servers
MCPServerStdio(
command="npx",
args=["-y", "@modelcontextprotocol/server-filesystem"],
env={"API_KEY": "your_key"},
tool_filter=create_static_tool_filter(
allowed_tool_names=["read_file", "list_directory"]
),
cache_tools_list=True,
),
# HTTP/Streamable HTTP transport for remote servers
MCPServerHTTP(
url="https://api.example.com/mcp",
headers={"Authorization": "Bearer your_token"},
streamable=True,
cache_tools_list=True,
),
# SSE transport for real-time streaming
MCPServerSSE(
url="https://stream.example.com/mcp/sse",
headers={"Authorization": "Bearer your_token"},
),
]
)
```
### 🔧 **Advanced: MCPServerAdapter** (For Complex Scenarios)
For advanced use cases requiring manual connection management, the `crewai-tools` library provides the `MCPServerAdapter` class.
@@ -112,14 +68,12 @@ uv pip install 'crewai-tools[mcp]'
## Quick Start: Simple DSL Integration
The easiest way to integrate MCP servers is using the `mcps` field on your agents. You can use either string references or structured configurations.
### Quick Start with String References
The easiest way to integrate MCP servers is using the `mcps` field on your agents:
```python
from crewai import Agent, Task, Crew
# Create agent with MCP tools using string references
# Create agent with MCP tools
research_agent = Agent(
role="Research Analyst",
goal="Find and analyze information using advanced search tools",
@@ -142,53 +96,13 @@ crew = Crew(agents=[research_agent], tasks=[research_task])
result = crew.kickoff()
```
### Quick Start with Structured Configurations
```python
from crewai import Agent, Task, Crew
from crewai.mcp import MCPServerStdio, MCPServerHTTP, MCPServerSSE
# Create agent with structured MCP configurations
research_agent = Agent(
role="Research Analyst",
goal="Find and analyze information using advanced search tools",
backstory="Expert researcher with access to multiple data sources",
mcps=[
# Local stdio server
MCPServerStdio(
command="python",
args=["local_server.py"],
env={"API_KEY": "your_key"},
),
# Remote HTTP server
MCPServerHTTP(
url="https://api.research.com/mcp",
headers={"Authorization": "Bearer your_token"},
),
]
)
# Create task
research_task = Task(
description="Research the latest developments in AI agent frameworks",
expected_output="Comprehensive research report with citations",
agent=research_agent
)
# Create and run crew
crew = Crew(agents=[research_agent], tasks=[research_task])
result = crew.kickoff()
```
That's it! The MCP tools are automatically discovered and available to your agent.
## MCP Reference Formats
The `mcps` field supports both **string references** (for quick setup) and **structured configurations** (for full control). You can mix both formats in the same list.
The `mcps` field supports various reference formats for maximum flexibility:
### String-Based References
#### External MCP Servers
### External MCP Servers
```python
mcps=[
@@ -203,7 +117,7 @@ mcps=[
]
```
#### CrewAI AMP Marketplace
### CrewAI AMP Marketplace
```python
mcps=[
@@ -219,166 +133,17 @@ mcps=[
]
```
### Structured Configurations
#### Stdio Transport (Local Servers)
Perfect for local MCP servers that run as processes:
```python
from crewai.mcp import MCPServerStdio
from crewai.mcp.filters import create_static_tool_filter
mcps=[
MCPServerStdio(
command="npx",
args=["-y", "@modelcontextprotocol/server-filesystem"],
env={"API_KEY": "your_key"},
tool_filter=create_static_tool_filter(
allowed_tool_names=["read_file", "write_file"]
),
cache_tools_list=True,
),
# Python-based server
MCPServerStdio(
command="python",
args=["path/to/server.py"],
env={"UV_PYTHON": "3.12", "API_KEY": "your_key"},
),
]
```
#### HTTP/Streamable HTTP Transport (Remote Servers)
For remote MCP servers over HTTP/HTTPS:
```python
from crewai.mcp import MCPServerHTTP
mcps=[
# Streamable HTTP (default)
MCPServerHTTP(
url="https://api.example.com/mcp",
headers={"Authorization": "Bearer your_token"},
streamable=True,
cache_tools_list=True,
),
# Standard HTTP
MCPServerHTTP(
url="https://api.example.com/mcp",
headers={"Authorization": "Bearer your_token"},
streamable=False,
),
]
```
#### SSE Transport (Real-Time Streaming)
For remote servers using Server-Sent Events:
```python
from crewai.mcp import MCPServerSSE
mcps=[
MCPServerSSE(
url="https://stream.example.com/mcp/sse",
headers={"Authorization": "Bearer your_token"},
cache_tools_list=True,
),
]
```
### Mixed References
You can combine string references and structured configurations:
```python
from crewai.mcp import MCPServerStdio, MCPServerHTTP
mcps=[
# String references
"https://external-api.com/mcp", # External server
"https://weather.service.com/mcp#forecast", # Specific external tool
"crewai-amp:financial-insights", # AMP service
# Structured configurations
MCPServerStdio(
command="npx",
args=["-y", "@modelcontextprotocol/server-filesystem"],
),
MCPServerHTTP(
url="https://api.example.com/mcp",
headers={"Authorization": "Bearer token"},
),
"crewai-amp:data-analysis#sentiment_tool" # Specific AMP tool
]
```
### Tool Filtering
Structured configurations support advanced tool filtering:
```python
from crewai.mcp import MCPServerStdio
from crewai.mcp.filters import create_static_tool_filter, create_dynamic_tool_filter, ToolFilterContext
# Static filtering (allow/block lists)
static_filter = create_static_tool_filter(
allowed_tool_names=["read_file", "write_file"],
blocked_tool_names=["delete_file"],
)
# Dynamic filtering (context-aware)
def dynamic_filter(context: ToolFilterContext, tool: dict) -> bool:
# Block dangerous tools for certain agent roles
if context.agent.role == "Code Reviewer":
if "delete" in tool.get("name", "").lower():
return False
return True
mcps=[
MCPServerStdio(
command="npx",
args=["-y", "@modelcontextprotocol/server-filesystem"],
tool_filter=static_filter, # or dynamic_filter
),
]
```
## Configuration Parameters
Each transport type supports specific configuration options:
### MCPServerStdio Parameters
- **`command`** (required): Command to execute (e.g., `"python"`, `"node"`, `"npx"`, `"uvx"`)
- **`args`** (optional): List of command arguments (e.g., `["server.py"]` or `["-y", "@mcp/server"]`)
- **`env`** (optional): Dictionary of environment variables to pass to the process
- **`tool_filter`** (optional): Tool filter function for filtering available tools
- **`cache_tools_list`** (optional): Whether to cache the tool list for faster subsequent access (default: `False`)
### MCPServerHTTP Parameters
- **`url`** (required): Server URL (e.g., `"https://api.example.com/mcp"`)
- **`headers`** (optional): Dictionary of HTTP headers for authentication or other purposes
- **`streamable`** (optional): Whether to use streamable HTTP transport (default: `True`)
- **`tool_filter`** (optional): Tool filter function for filtering available tools
- **`cache_tools_list`** (optional): Whether to cache the tool list for faster subsequent access (default: `False`)
### MCPServerSSE Parameters
- **`url`** (required): Server URL (e.g., `"https://api.example.com/mcp/sse"`)
- **`headers`** (optional): Dictionary of HTTP headers for authentication or other purposes
- **`tool_filter`** (optional): Tool filter function for filtering available tools
- **`cache_tools_list`** (optional): Whether to cache the tool list for faster subsequent access (default: `False`)
### Common Parameters
All transport types support:
- **`tool_filter`**: Filter function to control which tools are available. Can be:
- `None` (default): All tools are available
- Static filter: Created with `create_static_tool_filter()` for allow/block lists
- Dynamic filter: Created with `create_dynamic_tool_filter()` for context-aware filtering
- **`cache_tools_list`**: When `True`, caches the tool list after first discovery to improve performance on subsequent connections
## Key Features
- 🔄 **Automatic Tool Discovery**: Tools are automatically discovered and integrated
@@ -387,47 +152,26 @@ All transport types support:
- 🛡️ **Error Resilience**: Graceful handling of unavailable servers
- ⏱️ **Timeout Protection**: Built-in timeouts prevent hanging connections
- 📊 **Transparent Integration**: Works seamlessly with existing CrewAI features
- 🔧 **Full Transport Support**: Stdio, HTTP/Streamable HTTP, and SSE transports
- 🎯 **Advanced Filtering**: Static and dynamic tool filtering capabilities
- 🔐 **Flexible Authentication**: Support for headers, environment variables, and query parameters
## Error Handling
The MCP DSL integration is designed to be resilient and handles failures gracefully:
The MCP DSL integration is designed to be resilient:
```python
from crewai import Agent
from crewai.mcp import MCPServerStdio, MCPServerHTTP
agent = Agent(
role="Resilient Agent",
goal="Continue working despite server issues",
backstory="Agent that handles failures gracefully",
mcps=[
# String references
"https://reliable-server.com/mcp", # Will work
"https://unreachable-server.com/mcp", # Will be skipped gracefully
"crewai-amp:working-service", # Will work
# Structured configs
MCPServerStdio(
command="python",
args=["reliable_server.py"], # Will work
),
MCPServerHTTP(
url="https://slow-server.com/mcp", # Will timeout gracefully
),
"https://slow-server.com/mcp", # Will timeout gracefully
"crewai-amp:working-service" # Will work
]
)
# Agent will use tools from working servers and log warnings for failing ones
```
All connection errors are handled gracefully:
- **Connection failures**: Logged as warnings, agent continues with available tools
- **Timeout errors**: Connections timeout after 30 seconds (configurable)
- **Authentication errors**: Logged clearly for debugging
- **Invalid configurations**: Validation errors are raised at agent creation time
## Advanced: MCPServerAdapter
For complex scenarios requiring manual connection management, use the `MCPServerAdapter` class from `crewai-tools`. Using a Python context manager (`with` statement) is the recommended approach as it automatically handles starting and stopping the connection to the MCP server.

View File

@@ -12,7 +12,7 @@ dependencies = [
"pytube>=15.0.0",
"requests>=2.32.5",
"docker>=7.1.0",
"crewai==1.4.1",
"crewai==1.2.1",
"lancedb>=0.5.4",
"tiktoken>=0.8.0",
"beautifulsoup4>=4.13.4",

View File

@@ -287,4 +287,4 @@ __all__ = [
"ZapierActionTools",
]
__version__ = "1.4.1"
__version__ = "1.2.1"

View File

@@ -229,7 +229,6 @@ class CrewAIRagAdapter(Adapter):
continue
else:
metadata: dict[str, Any] = base_metadata.copy()
source_content = SourceContent(source_ref)
if data_type in [
DataType.PDF_FILE,
@@ -240,12 +239,13 @@ class CrewAIRagAdapter(Adapter):
DataType.XML,
DataType.MDX,
]:
if not source_content.is_url() and not source_content.path_exists():
if not os.path.isfile(source_ref):
raise FileNotFoundError(f"File does not exist: {source_ref}")
loader = data_type.get_loader()
chunker = data_type.get_chunker()
source_content = SourceContent(source_ref)
loader_result: LoaderResult = loader.load(source_content)
chunks = chunker.chunk(loader_result.content)

View File

@@ -22,23 +22,22 @@ class FirecrawlCrawlWebsiteToolSchema(BaseModel):
class FirecrawlCrawlWebsiteTool(BaseTool):
"""Tool for crawling websites using Firecrawl v2 API. To run this tool, you need to have a Firecrawl API key.
"""Tool for crawling websites using Firecrawl. To run this tool, you need to have a Firecrawl API key.
Args:
api_key (str): Your Firecrawl API key.
config (dict): Optional. It contains Firecrawl v2 API parameters.
config (dict): Optional. It contains Firecrawl API parameters.
Default configuration options (Firecrawl v2 API):
max_discovery_depth (int): Maximum depth for discovering pages. Default: 2
Default configuration options:
max_depth (int): Maximum depth to crawl. Default: 2
ignore_sitemap (bool): Whether to ignore sitemap. Default: True
limit (int): Maximum number of pages to crawl. Default: 10
limit (int): Maximum number of pages to crawl. Default: 100
allow_backward_links (bool): Allow crawling backward links. Default: False
allow_external_links (bool): Allow crawling external links. Default: False
allow_subdomains (bool): Allow crawling subdomains. Default: False
delay (int): Delay between requests in milliseconds. Default: None
scrape_options (dict): Options for scraping content
- formats (list[str]): Content formats to return. Default: ["markdown"]
scrape_options (ScrapeOptions): Options for scraping content
- formats (list[str]): Content formats to return. Default: ["markdown", "screenshot", "links"]
- only_main_content (bool): Only return main content. Default: True
- timeout (int): Timeout in milliseconds. Default: 10000
- timeout (int): Timeout in milliseconds. Default: 30000
"""
model_config = ConfigDict(
@@ -50,15 +49,14 @@ class FirecrawlCrawlWebsiteTool(BaseTool):
api_key: str | None = None
config: dict[str, Any] | None = Field(
default_factory=lambda: {
"max_discovery_depth": 2,
"ignore_sitemap": True,
"maxDepth": 2,
"ignoreSitemap": True,
"limit": 10,
"allow_external_links": False,
"allow_subdomains": False,
"delay": None,
"scrape_options": {
"formats": ["markdown"],
"only_main_content": True,
"allowBackwardLinks": False,
"allowExternalLinks": False,
"scrapeOptions": {
"formats": ["markdown", "screenshot", "links"],
"onlyMainContent": True,
"timeout": 10000,
},
}
@@ -109,7 +107,7 @@ class FirecrawlCrawlWebsiteTool(BaseTool):
if not self._firecrawl:
raise RuntimeError("FirecrawlApp not properly initialized")
return self._firecrawl.crawl(url=url, poll_interval=2, **self.config)
return self._firecrawl.crawl_url(url, poll_interval=2, params=self.config)
try:

View File

@@ -22,27 +22,20 @@ class FirecrawlScrapeWebsiteToolSchema(BaseModel):
class FirecrawlScrapeWebsiteTool(BaseTool):
"""Tool for scraping webpages using Firecrawl v2 API. To run this tool, you need to have a Firecrawl API key.
"""Tool for scraping webpages using Firecrawl. To run this tool, you need to have a Firecrawl API key.
Args:
api_key (str): Your Firecrawl API key.
config (dict): Optional. It contains Firecrawl v2 API parameters.
config (dict): Optional. It contains Firecrawl API parameters.
Default configuration options (Firecrawl v2 API):
Default configuration options:
formats (list[str]): Content formats to return. Default: ["markdown"]
only_main_content (bool): Only return main content excluding headers, navs, footers, etc. Default: True
include_tags (list[str]): Tags to include in the output. Default: []
exclude_tags (list[str]): Tags to exclude from the output. Default: []
max_age (int): Returns cached version if younger than this age in milliseconds. Default: 172800000 (2 days)
headers (dict): Headers to send with the request (e.g., cookies, user-agent). Default: {}
wait_for (int): Delay in milliseconds before fetching content. Default: 0
mobile (bool): Emulate scraping from a mobile device. Default: False
skip_tls_verification (bool): Skip TLS certificate verification. Default: True
timeout (int): Request timeout in milliseconds. Default: None
remove_base64_images (bool): Remove base64 images from output. Default: True
block_ads (bool): Enable ad-blocking and cookie popup blocking. Default: True
proxy (str): Proxy type ("basic", "stealth", "auto"). Default: "auto"
store_in_cache (bool): Store page in Firecrawl index and cache. Default: True
onlyMainContent (bool): Only return main content. Default: True
includeTags (list[str]): Tags to include. Default: []
excludeTags (list[str]): Tags to exclude. Default: []
headers (dict): Headers to include. Default: {}
waitFor (int): Time to wait for page to load in ms. Default: 0
json_options (dict): Options for JSON extraction. Default: None
"""
model_config = ConfigDict(
@@ -55,18 +48,11 @@ class FirecrawlScrapeWebsiteTool(BaseTool):
config: dict[str, Any] = Field(
default_factory=lambda: {
"formats": ["markdown"],
"only_main_content": True,
"include_tags": [],
"exclude_tags": [],
"max_age": 172800000, # 2 days cache
"onlyMainContent": True,
"includeTags": [],
"excludeTags": [],
"headers": {},
"wait_for": 0,
"mobile": False,
"skip_tls_verification": True,
"remove_base64_images": True,
"block_ads": True,
"proxy": "auto",
"store_in_cache": True,
"waitFor": 0,
}
)
@@ -109,7 +95,7 @@ class FirecrawlScrapeWebsiteTool(BaseTool):
if not self._firecrawl:
raise RuntimeError("FirecrawlApp not properly initialized")
return self._firecrawl.scrape(url=url, **self.config)
return self._firecrawl.scrape_url(url, params=self.config)
try:

View File

@@ -23,24 +23,19 @@ class FirecrawlSearchToolSchema(BaseModel):
class FirecrawlSearchTool(BaseTool):
"""Tool for searching webpages using Firecrawl v2 API. To run this tool, you need to have a Firecrawl API key.
"""Tool for searching webpages using Firecrawl. To run this tool, you need to have a Firecrawl API key.
Args:
api_key (str): Your Firecrawl API key.
config (dict): Optional. It contains Firecrawl v2 API parameters.
config (dict): Optional. It contains Firecrawl API parameters.
Default configuration options (Firecrawl v2 API):
limit (int): Maximum number of search results to return. Default: 5
tbs (str): Time-based search filter (e.g., "qdr:d" for past day). Default: None
location (str): Location for search results. Default: None
timeout (int): Request timeout in milliseconds. Default: None
scrape_options (dict): Options for scraping the search results. Default: {"formats": ["markdown"]}
- formats (list[str]): Content formats to return. Default: ["markdown"]
- only_main_content (bool): Only return main content. Default: True
- include_tags (list[str]): Tags to include. Default: []
- exclude_tags (list[str]): Tags to exclude. Default: []
- wait_for (int): Delay before fetching content in ms. Default: 0
- timeout (int): Request timeout in milliseconds. Default: None
Default configuration options:
limit (int): Maximum number of pages to crawl. Default: 5
tbs (str): Time before search. Default: None
lang (str): Language. Default: "en"
country (str): Country. Default: "us"
location (str): Location. Default: None
timeout (int): Timeout in milliseconds. Default: 60000
"""
model_config = ConfigDict(
@@ -54,15 +49,10 @@ class FirecrawlSearchTool(BaseTool):
default_factory=lambda: {
"limit": 5,
"tbs": None,
"lang": "en",
"country": "us",
"location": None,
"timeout": None,
"scrape_options": {
"formats": ["markdown"],
"only_main_content": True,
"include_tags": [],
"exclude_tags": [],
"wait_for": 0,
},
"timeout": 60000,
}
)
_firecrawl: FirecrawlApp | None = PrivateAttr(None)
@@ -116,7 +106,7 @@ class FirecrawlSearchTool(BaseTool):
return self._firecrawl.search(
query=query,
**self.config,
params=self.config,
)

View File

@@ -12,16 +12,12 @@ from pydantic.types import ImportString
class QdrantToolSchema(BaseModel):
query: str = Field(
..., description="Query to search in Qdrant DB - always required."
)
query: str = Field(..., description="Query to search in Qdrant DB")
filter_by: str | None = Field(
default=None,
description="Parameter to filter the search by. When filtering, needs to be used in conjunction with filter_value.",
default=None, description="Parameter to filter the search by."
)
filter_value: Any | None = Field(
default=None,
description="Value to filter the search by. When filtering, needs to be used in conjunction with filter_by.",
default=None, description="Value to filter the search by."
)

View File

@@ -1,289 +0,0 @@
interactions:
- request:
body: '{"url": "https://firecrawl.dev", "includeTags": [], "excludeTags": [],
"onlyMainContent": true, "waitFor": 0, "skipTlsVerification": true, "removeBase64Images":
true, "fastMode": false, "blockAds": true, "storeInCache": true, "maxAge": 172800000,
"formats": ["markdown"], "headers": {}, "mobile": false, "proxy": "auto", "origin":
"python-sdk@4.5.0"}'
headers:
Accept:
- '*/*'
Accept-Encoding:
- gzip, deflate, zstd
Connection:
- keep-alive
Content-Length:
- '350'
Content-Type:
- application/json
User-Agent:
- python-requests/2.32.5
method: POST
uri: https://api.firecrawl.dev/v2/scrape
response:
body:
string: "{\"success\":true,\"data\":{\"markdown\":\"We just raised our Series
A and shipped Firecrawl /v2 \U0001F389. [Read the blog.](https://www.firecrawl.dev/blog/firecrawl-v2-series-a-announcement)\\n\\n[2
Months Free \u2014 Annually](https://www.firecrawl.dev/pricing)\\n\\n# Turn
websites into LLM-ready data\\n\\nPower your AI apps with clean web data\\n\\nfrom
any website. [It's also open source.](https://github.com/firecrawl/firecrawl)\\n\\nScrape\\n\\nSearch\\nNew\\n\\nMap\\n\\nCrawl\\n\\nScrape\\n\\nLogo\\n\\nNavigation\\n\\nButton\\n\\nH1
Title\\n\\nDescription\\n\\nCTA Button\\n\\n\\\\[ .JSON \\\\]\\n\\n```json\\n1[\\\\\\n2
\ {\\\\\\n3 \\\"url\\\": \\\"https://example.com\\\",\\\\\\n4 \\\"markdown\\\":
\\\"# Getting Started...\\\",\\\\\\n5 \\\"json\\\": { \\\"title\\\": \\\"Guide\\\",
\\\"docs\\\": \\\"...\\\" },\\\\\\n6 \\\"screenshot\\\": \\\"https://example.com/hero.png\\\"\\\\\\n7
\ }\\\\\\n8]\\n```\\n\\nScrape Completed\\n\\nTrusted by5000+\\n\\ncompaniesof
all sizes\\n\\n![Logo 17](https://www.firecrawl.dev/assets-original/logocloud/17.png)\\n\\n![Logo
18](https://www.firecrawl.dev/assets-original/logocloud/18.png)\\n\\n![Logo
1](https://www.firecrawl.dev/assets-original/logocloud/1.png)\\n\\n![Logo
2](https://www.firecrawl.dev/assets-original/logocloud/2.png)\\n\\n![Logo
3](https://www.firecrawl.dev/assets-original/logocloud/3.png)\\n\\n![Logo
4](https://www.firecrawl.dev/assets-original/logocloud/4.png)\\n\\n![Logo
5](https://www.firecrawl.dev/assets-original/logocloud/5.png)\\n\\n![Logo
6](https://www.firecrawl.dev/assets-original/logocloud/6.png)\\n\\n![Logo
7](https://www.firecrawl.dev/assets-original/logocloud/7.png)\\n\\n![Logo
8](https://www.firecrawl.dev/assets-original/logocloud/8.png)\\n\\n![Logo
9](https://www.firecrawl.dev/assets-original/logocloud/9.png)\\n\\n![Logo
10](https://www.firecrawl.dev/assets-original/logocloud/10.png)\\n\\n![Logo
11](https://www.firecrawl.dev/assets-original/logocloud/11.png)\\n\\n![Logo
12](https://www.firecrawl.dev/assets-original/logocloud/12.png)\\n\\n![Logo
13](https://www.firecrawl.dev/assets-original/logocloud/13.png)\\n\\n![Logo
14](https://www.firecrawl.dev/assets-original/logocloud/14.png)\\n\\n![Logo
15](https://www.firecrawl.dev/assets-original/logocloud/15.png)\\n\\n![Logo
16](https://www.firecrawl.dev/assets-original/logocloud/16.png)\\n\\n![Logo
17](https://www.firecrawl.dev/assets-original/logocloud/17.png)\\n\\n![Logo
18](https://www.firecrawl.dev/assets-original/logocloud/18.png)\\n\\n![Logo
19](https://www.firecrawl.dev/assets-original/logocloud/19.png)\\n\\n![Logo
20](https://www.firecrawl.dev/assets-original/logocloud/20.png)\\n\\n![Logo
21](https://www.firecrawl.dev/assets-original/logocloud/21.png)\\n\\n![Logo
17](https://www.firecrawl.dev/assets-original/logocloud/17.png)\\n\\n![Logo
18](https://www.firecrawl.dev/assets-original/logocloud/18.png)\\n\\n![Logo
1](https://www.firecrawl.dev/assets-original/logocloud/1.png)\\n\\n![Logo
2](https://www.firecrawl.dev/assets-original/logocloud/2.png)\\n\\n![Logo
3](https://www.firecrawl.dev/assets-original/logocloud/3.png)\\n\\n![Logo
4](https://www.firecrawl.dev/assets-original/logocloud/4.png)\\n\\n![Logo
5](https://www.firecrawl.dev/assets-original/logocloud/5.png)\\n\\n![Logo
6](https://www.firecrawl.dev/assets-original/logocloud/6.png)\\n\\n![Logo
7](https://www.firecrawl.dev/assets-original/logocloud/7.png)\\n\\n![Logo
8](https://www.firecrawl.dev/assets-original/logocloud/8.png)\\n\\n![Logo
9](https://www.firecrawl.dev/assets-original/logocloud/9.png)\\n\\n![Logo
10](https://www.firecrawl.dev/assets-original/logocloud/10.png)\\n\\n![Logo
11](https://www.firecrawl.dev/assets-original/logocloud/11.png)\\n\\n![Logo
12](https://www.firecrawl.dev/assets-original/logocloud/12.png)\\n\\n![Logo
13](https://www.firecrawl.dev/assets-original/logocloud/13.png)\\n\\n![Logo
14](https://www.firecrawl.dev/assets-original/logocloud/14.png)\\n\\n![Logo
15](https://www.firecrawl.dev/assets-original/logocloud/15.png)\\n\\n![Logo
16](https://www.firecrawl.dev/assets-original/logocloud/16.png)\\n\\n![Logo
17](https://www.firecrawl.dev/assets-original/logocloud/17.png)\\n\\n![Logo
18](https://www.firecrawl.dev/assets-original/logocloud/18.png)\\n\\n![Logo
19](https://www.firecrawl.dev/assets-original/logocloud/19.png)\\n\\n![Logo
20](https://www.firecrawl.dev/assets-original/logocloud/20.png)\\n\\n![Logo
21](https://www.firecrawl.dev/assets-original/logocloud/21.png)\\n\\n\\\\[01/
07 \\\\]\\n\\n\xB7\\n\\nMain Features\\n\\n//\\n\\nDeveloper First\\n\\n//\\n\\n##
Startscraping today\\n\\nEnhance your apps with industry leading web scraping
and crawling capabilities.\\n\\nScrape\\n\\nGet llm-ready data from websites.
Markdown, JSON, screenshot, etc.\\n\\nSearch\\n\\nNew\\n\\nSearch the web
and get full content from results.\\n\\nCrawl\\n\\nCrawl all the pages on
a website and get data for each page.\\n\\nPython\\n\\nNode.js\\n\\nCurl\\n\\nCopy
code\\n\\n```python\\n1# pip install firecrawl-py\\n2from firecrawl import
Firecrawl\\n3\\n4app = Firecrawl(api_key=\\\"fc-YOUR_API_KEY\\\")\\n5\\n6#
Scrape a website:\\n7app.scrape('firecrawl.dev')\\n8\\n9\\n10\\n```\\n\\n\\\\[
.MD \\\\]\\n\\n```markdown\\n1# Firecrawl\\n2\\n3Firecrawl is a powerful web
scraping\\n4library that makes it easy to extract\\n5data from websites.\\n6\\n7##
Installation\\n8\\n9To install Firecrawl, run:\\n10\\n11\\n```\\n\\n![developer-1](https://www.firecrawl.dev/assets/developer/1.png)\\n\\n![developer-2](https://www.firecrawl.dev/assets/developer/2.png)\\n\\n![developer-3](https://www.firecrawl.dev/assets/developer/3.png)\\n\\n![developer-4](https://www.firecrawl.dev/assets/developer/4.png)\\n\\n![developer-5](https://www.firecrawl.dev/assets/developer/5.png)\\n\\n![developer-6](https://www.firecrawl.dev/assets/developer/6.png)\\n\\n![developer-7](https://www.firecrawl.dev/assets/developer/7.png)\\n\\n![developer-8](https://www.firecrawl.dev/assets/developer/8.png)\\n\\n![developer-9](https://www.firecrawl.dev/assets/developer/1.png)\\n\\n![developer-10](https://www.firecrawl.dev/assets/developer/2.png)\\n\\n![developer-11](https://www.firecrawl.dev/assets/developer/3.png)\\n\\n![developer-12](https://www.firecrawl.dev/assets/developer/4.png)\\n\\n![developer-13](https://www.firecrawl.dev/assets/developer/5.png)\\n\\n![developer-14](https://www.firecrawl.dev/assets/developer/6.png)\\n\\n![developer-15](https://www.firecrawl.dev/assets/developer/7.png)\\n\\n![developer-16](https://www.firecrawl.dev/assets/developer/8.png)\\n\\n![developer-17](https://www.firecrawl.dev/assets/developer/1.png)\\n\\n![developer-18](https://www.firecrawl.dev/assets/developer/2.png)\\n\\n![developer-19](https://www.firecrawl.dev/assets/developer/3.png)\\n\\n![developer-20](https://www.firecrawl.dev/assets/developer/4.png)\\n\\n![developer-21](https://www.firecrawl.dev/assets/developer/5.png)\\n\\n![developer-22](https://www.firecrawl.dev/assets/developer/6.png)\\n\\n![developer-23](https://www.firecrawl.dev/assets/developer/7.png)\\n\\n![developer-24](https://www.firecrawl.dev/assets/developer/8.png)\\n\\nIntegrations\\n\\n###
Use well-known tools\\n\\nAlready fully integrated with the greatest existing
tools and workflows.\\n\\n[See all integrations](https://www.firecrawl.dev/app)\\n\\n![Firecrawl
icon (blueprint)](https://www.firecrawl.dev/assets-original/developer-os-icon.png)\\n\\nmendableai/firecrawl\\n\\nPublic\\n\\nStar\\n\\n65.3K\\n\\n\\\\[python-SDK\\\\]
improvs/async\\n\\n#1337\\n\\n\xB7\\n\\nApr 18, 2025\\n\\n\xB7\\n\\n![rafaelsideguide](https://www.firecrawl.dev/_next/image?url=https%3A%2F%2Favatars.githubusercontent.com%2Fu%2F150964962%3Fv%3D4&w=48&q=75&dpl=dpl_7RqvseQXNVYetFdhTKj6RntohhL1)\\n\\nrafaelsideguide\\n\\nfeat(extract):
cost limit\\n\\n#1473\\n\\n\xB7\\n\\nApr 17, 2025\\n\\n\xB7\\n\\n![mogery](https://www.firecrawl.dev/_next/image?url=https%3A%2F%2Favatars.githubusercontent.com%2Fu%2F66118807%3Fv%3D4&w=48&q=75&dpl=dpl_7RqvseQXNVYetFdhTKj6RntohhL1)\\n\\nmogery\\n\\nfeat(scrape):
get job result from GCS, avoid Redis\\n\\n#1461\\n\\n\xB7\\n\\nApr 15, 2025\\n\\n\xB7\\n\\n![mogery](https://www.firecrawl.dev/_next/image?url=https%3A%2F%2Favatars.githubusercontent.com%2Fu%2F66118807%3Fv%3D4&w=48&q=75&dpl=dpl_7RqvseQXNVYetFdhTKj6RntohhL1)\\n\\nmogery\\n\\nExtract
v2/rerank improvs\\n\\n#1437\\n\\n\xB7\\n\\nApr 11, 2025\\n\\n\xB7\\n\\n![rafaelsideguide](https://www.firecrawl.dev/_next/image?url=https%3A%2F%2Favatars.githubusercontent.com%2Fu%2F150964962%3Fv%3D4&w=48&q=75&dpl=dpl_7RqvseQXNVYetFdhTKj6RntohhL1)\\n\\nrafaelsideguide\\n\\n![https://avatars.githubusercontent.com/u/150964962?v=4](https://www.firecrawl.dev/_next/image?url=https%3A%2F%2Favatars.githubusercontent.com%2Fu%2F150964962%3Fv%3D4&w=96&q=75&dpl=dpl_7RqvseQXNVYetFdhTKj6RntohhL1)\\n\\n![https://avatars.githubusercontent.com/u/66118807?v=4](https://www.firecrawl.dev/_next/image?url=https%3A%2F%2Favatars.githubusercontent.com%2Fu%2F66118807%3Fv%3D4&w=96&q=75&dpl=dpl_7RqvseQXNVYetFdhTKj6RntohhL1)\\n\\n+90\\n\\nOpen
Source\\n\\n### Code you can trust\\n\\nDeveloped transparently and collaboratively.
Join our community of contributors.\\n\\n[Check out our repo](https://github.com/firecrawl/firecrawl)\\n\\n\\\\[02/
07 \\\\]\\n\\n\xB7\\n\\nCore\\n\\n//\\n\\nBuilt to outperform\\n\\n//\\n\\n##
Core principles, provenperformance\\n\\nBuilt from the ground up to outperform
traditional scrapers.\\n\\nNo proxy headaches\\n\\nReliable.Covers 96% of
the web,\\n\\nincluding JS-heavy and protected pages. No proxies, no puppets,
just clean data.\\n\\nFirecrawl\\n\\n96%\\n\\n![Puppeteer icon](https://www.firecrawl.dev/assets/puppeteer.png)\\n\\nPuppeteer\\n\\n79%\\n\\ncURL\\n\\n75%\\n\\nSpeed
that feels invisible\\n\\nBlazingly fast.Delivers results in less than 1 second,
fast for real-time agents\\n\\nand dynamic apps.\\n\\nURL\\n\\nCrawl\\n\\nScrape\\n\\nfirecrawl.dev/docs\\n\\n50ms\\n\\n51ms\\n\\nfirecrawl.dev/templates\\n\\n52ms\\n\\n50ms\\n\\nfirecrawl.dev/changelog\\n\\n49ms\\n\\n52ms\\n\\nfirecrawl.dev/about\\n\\n52ms\\n\\n50ms\\n\\nfirecrawl.dev/changelog\\n\\n50ms\\n\\n52ms\\n\\nfirecrawl.dev/playground\\n\\n51ms\\n\\n49ms\\n\\n\\\\[
CTA \\\\]\\n\\n\\\\[ CRAWL \\\\]\\n\\n\\\\[ SCRAPE \\\\]\\n\\n\\\\[ CTA \\\\]\\n\\n//\\n\\nGet
started\\n\\n//\\n\\nReady to build?\\n\\nStart getting Web Data for free
and scale seamlessly as your project expands. No credit card needed.\\n\\n[Start
for free](https://www.firecrawl.dev/signin) [See our plans](https://www.firecrawl.dev/pricing)\\n\\n\\\\[03/
07 \\\\]\\n\\n\xB7\\n\\nFeatures\\n\\n//\\n\\nZero configuration\\n\\n//\\n\\n##
We handle the hard stuff\\n\\nRotating proxies, orchestration, rate limits,
js-blocked content and more.\\n\\nDocs to data\\n\\nMedia parsing.Firecrawl
can parse and output content from web hosted pdfs, docx, and more.\\n\\nhttps://example.com/docs/report.pdf\\n\\nhttps://example.com/files/brief.docx\\n\\nhttps://example.com/docs/guide.html\\n\\ndocx\\n\\nParsing...\\n\\nKnows
the moment\\n\\nSmart wait.Firecrawl intelligently waits for content to load,
making scraping faster and more reliable.\\n\\nhttps://example-spa.com\\n\\nRequest
Sent\\n\\nScrapes the real thing\\n\\nCached, when you need it.Selective caching,
you choose your caching patterns, growing web index.\\n\\n![User](https://www.firecrawl.dev/_next/image?url=%2Fassets-original%2Ffeatures%2Fcached-user.png&w=256&q=75&dpl=dpl_7RqvseQXNVYetFdhTKj6RntohhL1)\\n\\nUser\\n\\nFirecrawl\\n\\nCache\\n\\nInvisible
access\\n\\nStealth mode.Crawls the web without\\n\\nbeing blocked, mimics
real users to access protected or dynamic content.\\n\\nInteractive scraping\\n\\nActions.Click,
scroll, write, wait, press and more before extracting content.\\n\\nhttps://example.com\\n\\nNavigate\\n\\nClick\\n\\nType\\n\\nWait\\n\\nScroll\\n\\nPress\\n\\nScreenshot\\n\\nScrape\\n\\n\\\\[04/
07 \\\\]\\n\\n\xB7\\n\\nPricing\\n\\n//\\n\\nTransparent\\n\\n//\\n\\n## Flexible
pricing\\n\\nExplore transparent pricing built for real-world scraping. Start
for free, then scale as you grow.\\n\\n\U0001F1FA\U0001F1F8USD\\n\\nFree Plan\\n\\nA
lightweight way to try scraping.\\n\\nNo cost, no card, no hassle.\\n\\n500
credits\\n\\n$0123456789\\n\\none-time\\n\\nGet started\\n\\nScrape 500 pages\\n\\n2
concurrent requests\\n\\nLow rate limits\\n\\nHobby\\n\\nGreat for side projects
and small tools.\\n\\nFast, simple, no overkill.\\n\\n3,000 credits\\n\\n$01234567890123456789\\n\\n/monthly\\n\\nBilled
yearly\\n\\n2 months free\\n\\nSubscribe\\n\\nScrape 3,000 pages\\n\\n5 concurrent
requests\\n\\nBasic support\\n\\n$9 per extra 1k credits\\n\\nStandard\\n\\nMost
popular\\n\\nPerfect for scaling with less effort.\\n\\nSimple, solid, dependable.\\n\\n100,000
credits\\n\\n$01234567890123456789\\n\\n/monthly\\n\\nBilled yearly\\n\\n2
months free\\n\\nSubscribe\\n\\nScrape 100,000 pages\\n\\n50 concurrent requests\\n\\nStandard
support\\n\\n$47 per extra 35k credits\\n\\nGrowth\\n\\nBuilt for high volume
and speed.\\n\\nFirecrawl at full force.\\n\\n500,000 credits\\n\\n$012345678901234567890123456789\\n\\n/monthly\\n\\nBilled
yearly\\n\\n2 months free\\n\\nSubscribe\\n\\nScrape 500,000 pages\\n\\n100
concurrent requests\\n\\nPriority support\\n\\n$177 per extra 175k credits\\n\\nExtra
credits are available via auto-recharge packs. [Enable](https://www.firecrawl.dev/signin/signup)\\n\\nEnterprise\\n\\nPower
at your pace\\n\\nUnlimited credits. Custom RPMs.\\n\\n[Contact sales](https://fk4bvu0n5qp.typeform.com/to/Ej6oydlg)
[More details](https://www.firecrawl.dev/enterprise)\\n\\nBulk discounts\\n\\nTop
priority support\\n\\nCustom concurrency limits\\n\\nImproved stealth proxies\\n\\nSLAs\\n\\nAdvanced
security & controls\\n\\n\\\\[05/ 07 \\\\]\\n\\n\xB7\\n\\nTestimonials\\n\\n//\\n\\nCommunity\\n\\n//\\n\\n##
People love building withFirecrawl\\n\\nDiscover why developers choose
Firecrawl every day.\\n\\n[![Morgan Linton](https://www.firecrawl.dev/assets/testimonials/morgan-linton.png)Morgan
Linton@morganlinton\\\"If you're coding with AI, and haven't discovered @firecrawl\\\\_dev
yet, prepare to have your mind blown \U0001F92F\\\"](https://x.com/morganlinton/status/1839454165703204955)
[![Chris DeWeese](https://www.firecrawl.dev/assets/testimonials/chris-deweese.png)Chris
DeWeese@chrisdeweese\\\\_\\\"Started using @firecrawl\\\\_dev for a project,
I wish I used this sooner.\\\"](https://x.com/chrisdeweese_/status/1853587120406876601)
[![Alex Reibman](https://www.firecrawl.dev/assets/testimonials/alex-reibman.png)Alex
Reibman@AlexReibman\\\"Moved our internal agent's web scraping tool from Apify
to Firecrawl because it benchmarked 50x faster with AgentOps.\\\"](https://x.com/AlexReibman/status/1780299595484131836)
[![Tom - Morpho](https://www.firecrawl.dev/assets/testimonials/tom-morpho.png)Tom
- Morpho@TomReppelin\\\"I found gold today. Thank you @firecrawl\\\\_dev\\\"](https://x.com/TomReppelin/status/1844382491014201613)\\n\\n[![Morgan
Linton](https://www.firecrawl.dev/assets/testimonials/morgan-linton.png)Morgan
Linton@morganlinton\\\"If you're coding with AI, and haven't discovered @firecrawl\\\\_dev
yet, prepare to have your mind blown \U0001F92F\\\"](https://x.com/morganlinton/status/1839454165703204955)
[![Chris DeWeese](https://www.firecrawl.dev/assets/testimonials/chris-deweese.png)Chris
DeWeese@chrisdeweese\\\\_\\\"Started using @firecrawl\\\\_dev for a project,
I wish I used this sooner.\\\"](https://x.com/chrisdeweese_/status/1853587120406876601)
[![Alex Reibman](https://www.firecrawl.dev/assets/testimonials/alex-reibman.png)Alex
Reibman@AlexReibman\\\"Moved our internal agent's web scraping tool from Apify
to Firecrawl because it benchmarked 50x faster with AgentOps.\\\"](https://x.com/AlexReibman/status/1780299595484131836)
[![Tom - Morpho](https://www.firecrawl.dev/assets/testimonials/tom-morpho.png)Tom
- Morpho@TomReppelin\\\"I found gold today. Thank you @firecrawl\\\\_dev\\\"](https://x.com/TomReppelin/status/1844382491014201613)\\n\\n[![Bardia](https://www.firecrawl.dev/assets/testimonials/bardia.png)Bardia@thepericulum\\\"The
Firecrawl team ships. I wanted types for their node SDK, and less than an
hour later, I got them.\\\"](https://x.com/thepericulum/status/1781397799487078874)
[![Matt Busigin](https://www.firecrawl.dev/assets/testimonials/matt-busigin.png)Matt
Busigin@mbusigin\\\"Firecrawl is dope. Congrats guys \U0001F44F\\\"](https://x.com/mbusigin/status/1836065372010656069)
[![Sumanth](https://www.firecrawl.dev/assets/testimonials/sumanth.png)Sumanth@Sumanth\\\\_077\\\"Web
scraping will never be the same!\\\\\\\\\\n\\\\\\\\\\nFirecrawl is an open-source
framework that takes a URL, crawls it, and conver...\\\"](https://x.com/Sumanth_077/status/1940049003074478511)
[![Steven Tey](https://www.firecrawl.dev/assets/testimonials/steven-tey.png)Steven
Tey@steventey\\\"Open-source Clay alternative just dropped\\\\\\\\\\n\\\\\\\\\\nUpload
a CSV of emails and...\\\"](https://x.com/steventey/status/1932945651761098889)\\n\\n[![Bardia](https://www.firecrawl.dev/assets/testimonials/bardia.png)Bardia@thepericulum\\\"The
Firecrawl team ships. I wanted types for their node SDK, and less than an
hour later, I got them.\\\"](https://x.com/thepericulum/status/1781397799487078874)
[![Matt Busigin](https://www.firecrawl.dev/assets/testimonials/matt-busigin.png)Matt
Busigin@mbusigin\\\"Firecrawl is dope. Congrats guys \U0001F44F\\\"](https://x.com/mbusigin/status/1836065372010656069)
[![Sumanth](https://www.firecrawl.dev/assets/testimonials/sumanth.png)Sumanth@Sumanth\\\\_077\\\"Web
scraping will never be the same!\\\\\\\\\\n\\\\\\\\\\nFirecrawl is an open-source
framework that takes a URL, crawls it, and conver...\\\"](https://x.com/Sumanth_077/status/1940049003074478511)
[![Steven Tey](https://www.firecrawl.dev/assets/testimonials/steven-tey.png)Steven
Tey@steventey\\\"Open-source Clay alternative just dropped\\\\\\\\\\n\\\\\\\\\\nUpload
a CSV of emails and...\\\"](https://x.com/steventey/status/1932945651761098889)\\n\\n\\\\[06/
07 \\\\]\\n\\n\xB7\\n\\nUse Cases\\n\\n//\\n\\nUse cases\\n\\n//\\n\\n## Transform
\ web data into AI-powered solutions\\n\\nDiscover how Firecrawl customers
are getting the most out of our API.\\n\\n[View all use cases](https://docs.firecrawl.dev/use-cases/overview)\\n\\nChat
with context\\n\\nSmarter AI chats\\n\\nPower your AI assistants with real-time,
accurate web content.\\n\\n[View docs](https://docs.firecrawl.dev/introduction)\\n\\n![AI
Assistant](https://www.firecrawl.dev/assets/ai/bot.png)\\n\\nAI Assistant\\n\\nwithFirecrawl\\n\\nReal-time\xB7Updated
2 min ago\\n\\nAsk anything...\\n\\nKnow your leads\\n\\nLead enrichment\\n\\nEnhance
your sales data with\\n\\nweb information.\\n\\n[Check out Extract](https://www.firecrawl.dev/extract)\\n\\nExtracting
leads from directory...\\n\\nTech startups\\n\\nWith contact info\\n\\nDecision
makers\\n\\nFunding stage\\n\\nReady to engage\\n\\n![Emily Tran](https://www.firecrawl.dev/assets/ai/leads-1.png)\\n\\n![James
Carter](https://www.firecrawl.dev/assets/ai/leads-2.png)\\n\\n![Sophia Kim](https://www.firecrawl.dev/assets/ai/leads-3.png)\\n\\n![Michael
Rivera](https://www.firecrawl.dev/assets/ai/leads-4.png)\\n\\nKnow your leads\\n\\nMCPs\\n\\nAdd
powerful scraping to your\\n\\ncode editors.\\n\\n[Get started](https://docs.firecrawl.dev/mcp-server)\\n\\n![Claude
Code](https://www.firecrawl.dev/assets/ai/mcps-claude.png)\\n\\nClaude Code\\n\\n![Cursor](https://www.firecrawl.dev/assets/ai/mcps-cursor.png)\\n\\nCursor\\n\\n![Windsurf](https://www.firecrawl.dev/assets/ai/mcps-windsurf.png)\\n\\nWindsurf\\n\\n\u273B\\n\\nWelcome
to Claude Code!\\n\\n/help for help, /status for your current setup\\n\\n>Try
\\\"how do I log an error?\\\"\\n\\nBuild with context\\n\\nAI platforms\\n\\nLet
your customers build AI apps\\n\\nwith web data.\\n\\n[Check out Map](https://docs.firecrawl.dev/features/map)\\n\\n![Logo
1](https://www.firecrawl.dev/assets/ai/platforms-1.png)\\n\\n![Logo 2](https://www.firecrawl.dev/assets/ai/platforms-2.png)\\n\\n![Logo
4](https://www.firecrawl.dev/assets/ai/platforms-4.png)\\n\\n![Logo 3](https://www.firecrawl.dev/assets/ai/platforms-3.png)\\n\\nExtracting
text...\\n\\nNo insight missed\\n\\nDeep research\\n\\nExtract comprehensive
information for\\n\\nin-depth research.\\n\\n[Build your own with Search](https://docs.firecrawl.dev/features/search)\\n\\nDeep
research in progress...\\n\\nAcademic papers\\n\\n0 found\\n\\nNews articles\\n\\n0
found\\n\\nExpert opinions\\n\\n0 found\\n\\nResearch reports\\n\\n0 found\\n\\nIndustry
data\\n\\n0 found\\n\\nAsk anything...\\n\\n\\\\[ CTA \\\\]\\n\\n\\\\[ CRAWL
\\\\]\\n\\n\\\\[ SCRAPE \\\\]\\n\\n\\\\[ CTA \\\\]\\n\\n//\\n\\nGet started\\n\\n//\\n\\nReady
to build?\\n\\nStart getting Web Data for free and scale seamlessly as your
project expands. No credit card needed.\\n\\n[Start for free](https://www.firecrawl.dev/signin)
[See our plans](https://www.firecrawl.dev/pricing)\\n\\n\\\\[07/ 07 \\\\]\\n\\n\xB7\\n\\nFAQ\\n\\n//\\n\\nFAQ\\n\\n//\\n\\n##
Frequently askedquestions\\n\\nEverything you need to know about Firecrawl.\\n\\nGeneral\\n\\nWhat
is Firecrawl?\\n\\nWhat sites work?\\n\\nWho can benefit from using Firecrawl?\\n\\nIs
Firecrawl open-source?\\n\\nWhat is the difference between Firecrawl and other
web scrapers?\\n\\nWhat is the difference between the open-source version
and the hosted version?\\n\\nScraping & Crawling\\n\\nHow does Firecrawl handle
dynamic content on websites?\\n\\nWhy is it not crawling all the pages?\\n\\nCan
Firecrawl crawl websites without a sitemap?\\n\\nWhat formats can Firecrawl
convert web data into?\\n\\nHow does Firecrawl ensure the cleanliness of the
data?\\n\\nIs Firecrawl suitable for large-scale data scraping projects?\\n\\nDoes
it respect robots.txt?\\n\\nWhat measures does Firecrawl take to handle web
scraping challenges like rate limits and caching?\\n\\nDoes Firecrawl handle
captcha or authentication?\\n\\nAPI Related\\n\\nWhere can I find my API key?\\n\\nBilling\\n\\nIs
Firecrawl free?\\n\\nIs there a pay-per-use plan instead of monthly?\\n\\nDo
credits roll over to the next month?\\n\\nHow many credits do scraping and
crawling cost?\\n\\nDo you charge for failed requests?\\n\\nWhat payment methods
do you accept?\\n\\nFOOTER\\n\\nThe easiest way to extract\\n\\ndata from
the web\\n\\nBacked by\\n\\nY Combinator\\n\\n[Linkedin](https://www.linkedin.com/company/firecrawl)
[Github](https://github.com/firecrawl/firecrawl)\\n\\nSOC II \xB7 Type 2\\n\\nAICPA\\n\\nSOC
2\\n\\n[X (Twitter)](https://x.com/firecrawl_dev) [Discord](https://discord.gg/gSmWdAkdwd)\\n\\nProducts\\n\\n[Playground](https://www.firecrawl.dev/playground)
[Extract](https://www.firecrawl.dev/extract) [Pricing](https://www.firecrawl.dev/pricing)
[Templates](https://www.firecrawl.dev/templates) [Changelog](https://www.firecrawl.dev/changelog)\\n\\nUse
Cases\\n\\n[AI Platforms](https://docs.firecrawl.dev/use-cases/ai-platforms)
[Lead Enrichment](https://docs.firecrawl.dev/use-cases/lead-enrichment) [SEO
Platforms](https://docs.firecrawl.dev/use-cases/seo-platforms) [Deep Research](https://docs.firecrawl.dev/use-cases/deep-research)\\n\\nDocumentation\\n\\n[Getting
started](https://docs.firecrawl.dev/introduction) [API Reference](https://docs.firecrawl.dev/api-reference/introduction)
[Integrations](https://www.firecrawl.dev/app) [Examples](https://docs.firecrawl.dev/use-cases/overview)
[SDKs](https://docs.firecrawl.dev/sdks/overview)\\n\\nCompany\\n\\n[Blog](https://www.firecrawl.dev/blog)
[Careers](https://www.firecrawl.dev/careers) [Creator & OSS program](https://www.firecrawl.dev/creator-oss-program)
[Student program](https://www.firecrawl.dev/student-program)\\n\\n\xA9 2025
Firecrawl\\n\\n[Terms of Service](https://www.firecrawl.dev/terms-of-service)
[Privacy Policy](https://www.firecrawl.dev/privacy-policy) [Report Abuse](mailto:help@firecrawl.com?subject=Issue:)\\n\\n[All
systems normal](https://status.firecrawl.dev/)\\n\\nStripeM-Inner\",\"metadata\":{\"twitter:title\":\"Firecrawl
- The Web Data API for AI\",\"publisher\":\"Firecrawl\",\"ogUrl\":\"https://www.firecrawl.dev\",\"robots\":\"follow,
index\",\"title\":\"Firecrawl - The Web Data API for AI\",\"ogDescription\":\"The
web crawling, scraping, and search API for AI. Built for scale. Firecrawl
delivers the entire internet to AI agents and builders. Clean, structured,
and ready to reason with.\",\"ogImage\":\"https://www.firecrawl.dev/og.png\",\"viewport\":\"width=device-width,
initial-scale=1, maximum-scale=1, user-scalable=no\",\"og:url\":\"https://www.firecrawl.dev\",\"og:site_name\":\"Firecrawl
- The Web Data API for AI\",\"og:type\":\"website\",\"twitter:image\":\"https://www.firecrawl.dev/og.png\",\"author\":\"Firecrawl\",\"og:title\":\"Firecrawl
- The Web Data API for AI\",\"favicon\":\"https://www.firecrawl.dev/favicon.png\",\"description\":\"The
web crawling, scraping, and search API for AI. Built for scale. Firecrawl
delivers the entire internet to AI agents and builders. Clean, structured,
and ready to reason with.\",\"referrer\":\"origin-when-cross-origin\",\"twitter:site\":\"@Vercel\",\"ogSiteName\":\"Firecrawl
- The Web Data API for AI\",\"og:image\":\"https://www.firecrawl.dev/og.png\",\"twitter:card\":\"summary_large_image\",\"twitter:creator\":\"@Vercel\",\"twitter:description\":\"The
web crawling, scraping, and search API for AI. Built for scale. Firecrawl
delivers the entire internet to AI agents and builders. Clean, structured,
and ready to reason with.\",\"language\":\"en\",\"keywords\":\"Firecrawl,Markdown,Data,Mendable,Langchain\",\"creator\":\"Firecrawl\",\"ogTitle\":\"Firecrawl
- The Web Data API for AI\",\"og:description\":\"The web crawling, scraping,
and search API for AI. Built for scale. Firecrawl delivers the entire internet
to AI agents and builders. Clean, structured, and ready to reason with.\",\"scrapeId\":\"e78d8060-d581-4e5e-b25a-90cfdad48530\",\"sourceURL\":\"https://firecrawl.dev\",\"url\":\"https://www.firecrawl.dev/\",\"statusCode\":200,\"contentType\":\"text/html;
charset=utf-8\",\"proxyUsed\":\"basic\",\"cacheState\":\"hit\",\"cachedAt\":\"2025-10-29T13:09:07.713Z\",\"creditsUsed\":1}}}"
headers:
Access-Control-Allow-Origin:
- '*'
Alt-Svc:
- h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Content-Length:
- '24693'
Content-Type:
- application/json; charset=utf-8
Date:
- Wed, 29 Oct 2025 14:34:03 GMT
ETag:
- W/"6075-Q1W6uMv95JKEZARbtaiPYYMojlU"
Via:
- 1.1 google
X-Powered-By:
- Express
X-Response-Time:
- 4719.998ms
status:
code: 200
message: OK
version: 1

View File

@@ -1,937 +0,0 @@
interactions:
- request:
body: '{"query": "firecrawl", "limit": 5, "scrapeOptions": {"includeTags": [],
"excludeTags": [], "onlyMainContent": true, "waitFor": 0, "skipTlsVerification":
true, "removeBase64Images": true, "fastMode": false, "blockAds": true, "storeInCache":
true, "maxAge": 14400000, "formats": ["markdown"], "mobile": false}, "origin":
"python-sdk@4.5.0"}'
headers:
Accept:
- '*/*'
Accept-Encoding:
- gzip, deflate, zstd
Connection:
- keep-alive
Content-Length:
- '338'
Content-Type:
- application/json
User-Agent:
- python-requests/2.32.5
method: POST
uri: https://api.firecrawl.dev/v2/search
response:
body:
string: "{\"success\":true,\"data\":{\"web\":[{\"url\":\"https://www.firecrawl.dev/\",\"title\":\"Firecrawl
- The Web Data API for AI\",\"description\":\"The web crawling, scraping,
and search API for AI. Built for scale. Firecrawl delivers the entire internet
to AI agents and builders.\",\"position\":1,\"markdown\":\"We just raised
our Series A and shipped Firecrawl /v2 \U0001F389. [Read the blog.](https://www.firecrawl.dev/blog/firecrawl-v2-series-a-announcement)\\n\\n[2
Months Free \u2014 Annually](https://www.firecrawl.dev/pricing)\\n\\n# Turn
websites into LLM-ready data\\n\\nPower your AI apps with clean web data\\n\\nfrom
any website. [It's also open source.](https://github.com/firecrawl/firecrawl)\\n\\nScrape\\n\\nSearch\\nNew\\n\\nMap\\n\\nCrawl\\n\\nScrape\\n\\nLogo\\n\\nNavigation\\n\\nButton\\n\\nH1
Title\\n\\nDescription\\n\\nCTA Button\\n\\n\\\\[ .JSON \\\\]\\n\\n```json\\n1[\\\\\\n2
\ {\\\\\\n3 \\\"url\\\": \\\"https://example.com\\\",\\\\\\n4 \\\"markdown\\\":
\\\"# Getting Started...\\\",\\\\\\n5 \\\"json\\\": { \\\"title\\\": \\\"Guide\\\",
\\\"docs\\\": \\\"...\\\" },\\\\\\n6 \\\"screenshot\\\": \\\"https://example.com/hero.png\\\"\\\\\\n7
\ }\\\\\\n8]\\n```\\n\\nScrape Completed\\n\\nTrusted by5000+\\n\\ncompaniesof
all sizes\\n\\n![Logo 17](https://www.firecrawl.dev/assets-original/logocloud/17.png)\\n\\n![Logo
18](https://www.firecrawl.dev/assets-original/logocloud/18.png)\\n\\n![Logo
1](https://www.firecrawl.dev/assets-original/logocloud/1.png)\\n\\n![Logo
2](https://www.firecrawl.dev/assets-original/logocloud/2.png)\\n\\n![Logo
3](https://www.firecrawl.dev/assets-original/logocloud/3.png)\\n\\n![Logo
4](https://www.firecrawl.dev/assets-original/logocloud/4.png)\\n\\n![Logo
5](https://www.firecrawl.dev/assets-original/logocloud/5.png)\\n\\n![Logo
6](https://www.firecrawl.dev/assets-original/logocloud/6.png)\\n\\n![Logo
7](https://www.firecrawl.dev/assets-original/logocloud/7.png)\\n\\n![Logo
8](https://www.firecrawl.dev/assets-original/logocloud/8.png)\\n\\n![Logo
9](https://www.firecrawl.dev/assets-original/logocloud/9.png)\\n\\n![Logo
10](https://www.firecrawl.dev/assets-original/logocloud/10.png)\\n\\n![Logo
11](https://www.firecrawl.dev/assets-original/logocloud/11.png)\\n\\n![Logo
12](https://www.firecrawl.dev/assets-original/logocloud/12.png)\\n\\n![Logo
13](https://www.firecrawl.dev/assets-original/logocloud/13.png)\\n\\n![Logo
14](https://www.firecrawl.dev/assets-original/logocloud/14.png)\\n\\n![Logo
15](https://www.firecrawl.dev/assets-original/logocloud/15.png)\\n\\n![Logo
16](https://www.firecrawl.dev/assets-original/logocloud/16.png)\\n\\n![Logo
17](https://www.firecrawl.dev/assets-original/logocloud/17.png)\\n\\n![Logo
18](https://www.firecrawl.dev/assets-original/logocloud/18.png)\\n\\n![Logo
19](https://www.firecrawl.dev/assets-original/logocloud/19.png)\\n\\n![Logo
20](https://www.firecrawl.dev/assets-original/logocloud/20.png)\\n\\n![Logo
21](https://www.firecrawl.dev/assets-original/logocloud/21.png)\\n\\n![Logo
17](https://www.firecrawl.dev/assets-original/logocloud/17.png)\\n\\n![Logo
18](https://www.firecrawl.dev/assets-original/logocloud/18.png)\\n\\n![Logo
1](https://www.firecrawl.dev/assets-original/logocloud/1.png)\\n\\n![Logo
2](https://www.firecrawl.dev/assets-original/logocloud/2.png)\\n\\n![Logo
3](https://www.firecrawl.dev/assets-original/logocloud/3.png)\\n\\n![Logo
4](https://www.firecrawl.dev/assets-original/logocloud/4.png)\\n\\n![Logo
5](https://www.firecrawl.dev/assets-original/logocloud/5.png)\\n\\n![Logo
6](https://www.firecrawl.dev/assets-original/logocloud/6.png)\\n\\n![Logo
7](https://www.firecrawl.dev/assets-original/logocloud/7.png)\\n\\n![Logo
8](https://www.firecrawl.dev/assets-original/logocloud/8.png)\\n\\n![Logo
9](https://www.firecrawl.dev/assets-original/logocloud/9.png)\\n\\n![Logo
10](https://www.firecrawl.dev/assets-original/logocloud/10.png)\\n\\n![Logo
11](https://www.firecrawl.dev/assets-original/logocloud/11.png)\\n\\n![Logo
12](https://www.firecrawl.dev/assets-original/logocloud/12.png)\\n\\n![Logo
13](https://www.firecrawl.dev/assets-original/logocloud/13.png)\\n\\n![Logo
14](https://www.firecrawl.dev/assets-original/logocloud/14.png)\\n\\n![Logo
15](https://www.firecrawl.dev/assets-original/logocloud/15.png)\\n\\n![Logo
16](https://www.firecrawl.dev/assets-original/logocloud/16.png)\\n\\n![Logo
17](https://www.firecrawl.dev/assets-original/logocloud/17.png)\\n\\n![Logo
18](https://www.firecrawl.dev/assets-original/logocloud/18.png)\\n\\n![Logo
19](https://www.firecrawl.dev/assets-original/logocloud/19.png)\\n\\n![Logo
20](https://www.firecrawl.dev/assets-original/logocloud/20.png)\\n\\n![Logo
21](https://www.firecrawl.dev/assets-original/logocloud/21.png)\\n\\n\\\\[01/
07 \\\\]\\n\\n\xB7\\n\\nMain Features\\n\\n//\\n\\nDeveloper First\\n\\n//\\n\\n##
Startscraping today\\n\\nEnhance your apps with industry leading web scraping
and crawling capabilities.\\n\\nScrape\\n\\nGet llm-ready data from websites.
Markdown, JSON, screenshot, etc.\\n\\nSearch\\n\\nNew\\n\\nSearch the web
and get full content from results.\\n\\nCrawl\\n\\nCrawl all the pages on
a website and get data for each page.\\n\\nPython\\n\\nNode.js\\n\\nCurl\\n\\nCopy
code\\n\\n```python\\n1# pip install firecrawl-py\\n2from firecrawl import
Firecrawl\\n3\\n4app = Firecrawl(api_key=\\\"fc-YOUR_API_KEY\\\")\\n5\\n6#
Scrape a website:\\n7app.scrape('firecrawl.dev')\\n8\\n9\\n10\\n```\\n\\n\\\\[
.MD \\\\]\\n\\n```markdown\\n1# Firecrawl\\n2\\n3Firecrawl is a powerful web
scraping\\n4library that makes it easy to extract\\n5data from websites.\\n6\\n7##
Installation\\n8\\n9To install Firecrawl, run:\\n10\\n11\\n```\\n\\n![developer-1](https://www.firecrawl.dev/assets/developer/1.png)\\n\\n![developer-2](https://www.firecrawl.dev/assets/developer/2.png)\\n\\n![developer-3](https://www.firecrawl.dev/assets/developer/3.png)\\n\\n![developer-4](https://www.firecrawl.dev/assets/developer/4.png)\\n\\n![developer-5](https://www.firecrawl.dev/assets/developer/5.png)\\n\\n![developer-6](https://www.firecrawl.dev/assets/developer/6.png)\\n\\n![developer-7](https://www.firecrawl.dev/assets/developer/7.png)\\n\\n![developer-8](https://www.firecrawl.dev/assets/developer/8.png)\\n\\n![developer-9](https://www.firecrawl.dev/assets/developer/1.png)\\n\\n![developer-10](https://www.firecrawl.dev/assets/developer/2.png)\\n\\n![developer-11](https://www.firecrawl.dev/assets/developer/3.png)\\n\\n![developer-12](https://www.firecrawl.dev/assets/developer/4.png)\\n\\n![developer-13](https://www.firecrawl.dev/assets/developer/5.png)\\n\\n![developer-14](https://www.firecrawl.dev/assets/developer/6.png)\\n\\n![developer-15](https://www.firecrawl.dev/assets/developer/7.png)\\n\\n![developer-16](https://www.firecrawl.dev/assets/developer/8.png)\\n\\n![developer-17](https://www.firecrawl.dev/assets/developer/1.png)\\n\\n![developer-18](https://www.firecrawl.dev/assets/developer/2.png)\\n\\n![developer-19](https://www.firecrawl.dev/assets/developer/3.png)\\n\\n![developer-20](https://www.firecrawl.dev/assets/developer/4.png)\\n\\n![developer-21](https://www.firecrawl.dev/assets/developer/5.png)\\n\\n![developer-22](https://www.firecrawl.dev/assets/developer/6.png)\\n\\n![developer-23](https://www.firecrawl.dev/assets/developer/7.png)\\n\\n![developer-24](https://www.firecrawl.dev/assets/developer/8.png)\\n\\nIntegrations\\n\\n###
Use well-known tools\\n\\nAlready fully integrated with the greatest existing
tools and workflows.\\n\\n[See all integrations](https://www.firecrawl.dev/app)\\n\\n![Firecrawl
icon (blueprint)](https://www.firecrawl.dev/assets-original/developer-os-icon.png)\\n\\nmendableai/firecrawl\\n\\nPublic\\n\\nStar\\n\\n65.3K\\n\\n\\\\[python-SDK\\\\]
improvs/async\\n\\n#1337\\n\\n\xB7\\n\\nApr 18, 2025\\n\\n\xB7\\n\\n![rafaelsideguide](https://www.firecrawl.dev/_next/image?url=https%3A%2F%2Favatars.githubusercontent.com%2Fu%2F150964962%3Fv%3D4&w=48&q=75&dpl=dpl_7RqvseQXNVYetFdhTKj6RntohhL1)\\n\\nrafaelsideguide\\n\\nfeat(extract):
cost limit\\n\\n#1473\\n\\n\xB7\\n\\nApr 17, 2025\\n\\n\xB7\\n\\n![mogery](https://www.firecrawl.dev/_next/image?url=https%3A%2F%2Favatars.githubusercontent.com%2Fu%2F66118807%3Fv%3D4&w=48&q=75&dpl=dpl_7RqvseQXNVYetFdhTKj6RntohhL1)\\n\\nmogery\\n\\nfeat(scrape):
get job result from GCS, avoid Redis\\n\\n#1461\\n\\n\xB7\\n\\nApr 15, 2025\\n\\n\xB7\\n\\n![mogery](https://www.firecrawl.dev/_next/image?url=https%3A%2F%2Favatars.githubusercontent.com%2Fu%2F66118807%3Fv%3D4&w=48&q=75&dpl=dpl_7RqvseQXNVYetFdhTKj6RntohhL1)\\n\\nmogery\\n\\nExtract
v2/rerank improvs\\n\\n#1437\\n\\n\xB7\\n\\nApr 11, 2025\\n\\n\xB7\\n\\n![rafaelsideguide](https://www.firecrawl.dev/_next/image?url=https%3A%2F%2Favatars.githubusercontent.com%2Fu%2F150964962%3Fv%3D4&w=48&q=75&dpl=dpl_7RqvseQXNVYetFdhTKj6RntohhL1)\\n\\nrafaelsideguide\\n\\n![https://avatars.githubusercontent.com/u/150964962?v=4](https://www.firecrawl.dev/_next/image?url=https%3A%2F%2Favatars.githubusercontent.com%2Fu%2F150964962%3Fv%3D4&w=96&q=75&dpl=dpl_7RqvseQXNVYetFdhTKj6RntohhL1)\\n\\n![https://avatars.githubusercontent.com/u/66118807?v=4](https://www.firecrawl.dev/_next/image?url=https%3A%2F%2Favatars.githubusercontent.com%2Fu%2F66118807%3Fv%3D4&w=96&q=75&dpl=dpl_7RqvseQXNVYetFdhTKj6RntohhL1)\\n\\n+90\\n\\nOpen
Source\\n\\n### Code you can trust\\n\\nDeveloped transparently and collaboratively.
Join our community of contributors.\\n\\n[Check out our repo](https://github.com/firecrawl/firecrawl)\\n\\n\\\\[02/
07 \\\\]\\n\\n\xB7\\n\\nCore\\n\\n//\\n\\nBuilt to outperform\\n\\n//\\n\\n##
Core principles, provenperformance\\n\\nBuilt from the ground up to outperform
traditional scrapers.\\n\\nNo proxy headaches\\n\\nReliable.Covers 96% of
the web,\\n\\nincluding JS-heavy and protected pages. No proxies, no puppets,
just clean data.\\n\\nFirecrawl\\n\\n96%\\n\\n![Puppeteer icon](https://www.firecrawl.dev/assets/puppeteer.png)\\n\\nPuppeteer\\n\\n79%\\n\\ncURL\\n\\n75%\\n\\nSpeed
that feels invisible\\n\\nBlazingly fast.Delivers results in less than 1 second,
fast for real-time agents\\n\\nand dynamic apps.\\n\\nURL\\n\\nCrawl\\n\\nScrape\\n\\nfirecrawl.dev/docs\\n\\n50ms\\n\\n51ms\\n\\nfirecrawl.dev/templates\\n\\n52ms\\n\\n50ms\\n\\nfirecrawl.dev/changelog\\n\\n49ms\\n\\n52ms\\n\\nfirecrawl.dev/about\\n\\n52ms\\n\\n50ms\\n\\nfirecrawl.dev/changelog\\n\\n50ms\\n\\n52ms\\n\\nfirecrawl.dev/playground\\n\\n51ms\\n\\n49ms\\n\\n\\\\[
CTA \\\\]\\n\\n\\\\[ CRAWL \\\\]\\n\\n\\\\[ SCRAPE \\\\]\\n\\n\\\\[ CTA \\\\]\\n\\n//\\n\\nGet
started\\n\\n//\\n\\nReady to build?\\n\\nStart getting Web Data for free
and scale seamlessly as your project expands. No credit card needed.\\n\\n[Start
for free](https://www.firecrawl.dev/signin) [See our plans](https://www.firecrawl.dev/pricing)\\n\\n\\\\[03/
07 \\\\]\\n\\n\xB7\\n\\nFeatures\\n\\n//\\n\\nZero configuration\\n\\n//\\n\\n##
We handle the hard stuff\\n\\nRotating proxies, orchestration, rate limits,
js-blocked content and more.\\n\\nDocs to data\\n\\nMedia parsing.Firecrawl
can parse and output content from web hosted pdfs, docx, and more.\\n\\nhttps://example.com/docs/report.pdf\\n\\nhttps://example.com/files/brief.docx\\n\\nhttps://example.com/docs/guide.html\\n\\ndocx\\n\\nParsing...\\n\\nKnows
the moment\\n\\nSmart wait.Firecrawl intelligently waits for content to load,
making scraping faster and more reliable.\\n\\nhttps://example-spa.com\\n\\nRequest
Sent\\n\\nScrapes the real thing\\n\\nCached, when you need it.Selective caching,
you choose your caching patterns, growing web index.\\n\\n![User](https://www.firecrawl.dev/_next/image?url=%2Fassets-original%2Ffeatures%2Fcached-user.png&w=256&q=75&dpl=dpl_7RqvseQXNVYetFdhTKj6RntohhL1)\\n\\nUser\\n\\nFirecrawl\\n\\nCache\\n\\nInvisible
access\\n\\nStealth mode.Crawls the web without\\n\\nbeing blocked, mimics
real users to access protected or dynamic content.\\n\\nInteractive scraping\\n\\nActions.Click,
scroll, write, wait, press and more before extracting content.\\n\\nhttps://example.com\\n\\nNavigate\\n\\nClick\\n\\nType\\n\\nWait\\n\\nScroll\\n\\nPress\\n\\nScreenshot\\n\\nScrape\\n\\n\\\\[04/
07 \\\\]\\n\\n\xB7\\n\\nPricing\\n\\n//\\n\\nTransparent\\n\\n//\\n\\n## Flexible
pricing\\n\\nExplore transparent pricing built for real-world scraping. Start
for free, then scale as you grow.\\n\\n\U0001F1FA\U0001F1F8USD\\n\\nFree Plan\\n\\nA
lightweight way to try scraping.\\n\\nNo cost, no card, no hassle.\\n\\n500
credits\\n\\n$0123456789\\n\\none-time\\n\\nGet started\\n\\nScrape 500 pages\\n\\n2
concurrent requests\\n\\nLow rate limits\\n\\nHobby\\n\\nGreat for side projects
and small tools.\\n\\nFast, simple, no overkill.\\n\\n3,000 credits\\n\\n$01234567890123456789\\n\\n/monthly\\n\\nBilled
yearly\\n\\n2 months free\\n\\nSubscribe\\n\\nScrape 3,000 pages\\n\\n5 concurrent
requests\\n\\nBasic support\\n\\n$9 per extra 1k credits\\n\\nStandard\\n\\nMost
popular\\n\\nPerfect for scaling with less effort.\\n\\nSimple, solid, dependable.\\n\\n100,000
credits\\n\\n$01234567890123456789\\n\\n/monthly\\n\\nBilled yearly\\n\\n2
months free\\n\\nSubscribe\\n\\nScrape 100,000 pages\\n\\n50 concurrent requests\\n\\nStandard
support\\n\\n$47 per extra 35k credits\\n\\nGrowth\\n\\nBuilt for high volume
and speed.\\n\\nFirecrawl at full force.\\n\\n500,000 credits\\n\\n$012345678901234567890123456789\\n\\n/monthly\\n\\nBilled
yearly\\n\\n2 months free\\n\\nSubscribe\\n\\nScrape 500,000 pages\\n\\n100
concurrent requests\\n\\nPriority support\\n\\n$177 per extra 175k credits\\n\\nExtra
credits are available via auto-recharge packs. [Enable](https://www.firecrawl.dev/signin/signup)\\n\\nEnterprise\\n\\nPower
at your pace\\n\\nUnlimited credits. Custom RPMs.\\n\\n[Contact sales](https://fk4bvu0n5qp.typeform.com/to/Ej6oydlg)
[More details](https://www.firecrawl.dev/enterprise)\\n\\nBulk discounts\\n\\nTop
priority support\\n\\nCustom concurrency limits\\n\\nImproved stealth proxies\\n\\nSLAs\\n\\nAdvanced
security & controls\\n\\n\\\\[05/ 07 \\\\]\\n\\n\xB7\\n\\nTestimonials\\n\\n//\\n\\nCommunity\\n\\n//\\n\\n##
People love building withFirecrawl\\n\\nDiscover why developers choose
Firecrawl every day.\\n\\n[![Morgan Linton](https://www.firecrawl.dev/assets/testimonials/morgan-linton.png)Morgan
Linton@morganlinton\\\"If you're coding with AI, and haven't discovered @firecrawl\\\\_dev
yet, prepare to have your mind blown \U0001F92F\\\"](https://x.com/morganlinton/status/1839454165703204955)
[![Chris DeWeese](https://www.firecrawl.dev/assets/testimonials/chris-deweese.png)Chris
DeWeese@chrisdeweese\\\\_\\\"Started using @firecrawl\\\\_dev for a project,
I wish I used this sooner.\\\"](https://x.com/chrisdeweese_/status/1853587120406876601)
[![Alex Reibman](https://www.firecrawl.dev/assets/testimonials/alex-reibman.png)Alex
Reibman@AlexReibman\\\"Moved our internal agent's web scraping tool from Apify
to Firecrawl because it benchmarked 50x faster with AgentOps.\\\"](https://x.com/AlexReibman/status/1780299595484131836)
[![Tom - Morpho](https://www.firecrawl.dev/assets/testimonials/tom-morpho.png)Tom
- Morpho@TomReppelin\\\"I found gold today. Thank you @firecrawl\\\\_dev\\\"](https://x.com/TomReppelin/status/1844382491014201613)\\n\\n[![Morgan
Linton](https://www.firecrawl.dev/assets/testimonials/morgan-linton.png)Morgan
Linton@morganlinton\\\"If you're coding with AI, and haven't discovered @firecrawl\\\\_dev
yet, prepare to have your mind blown \U0001F92F\\\"](https://x.com/morganlinton/status/1839454165703204955)
[![Chris DeWeese](https://www.firecrawl.dev/assets/testimonials/chris-deweese.png)Chris
DeWeese@chrisdeweese\\\\_\\\"Started using @firecrawl\\\\_dev for a project,
I wish I used this sooner.\\\"](https://x.com/chrisdeweese_/status/1853587120406876601)
[![Alex Reibman](https://www.firecrawl.dev/assets/testimonials/alex-reibman.png)Alex
Reibman@AlexReibman\\\"Moved our internal agent's web scraping tool from Apify
to Firecrawl because it benchmarked 50x faster with AgentOps.\\\"](https://x.com/AlexReibman/status/1780299595484131836)
[![Tom - Morpho](https://www.firecrawl.dev/assets/testimonials/tom-morpho.png)Tom
- Morpho@TomReppelin\\\"I found gold today. Thank you @firecrawl\\\\_dev\\\"](https://x.com/TomReppelin/status/1844382491014201613)\\n\\n[![Bardia](https://www.firecrawl.dev/assets/testimonials/bardia.png)Bardia@thepericulum\\\"The
Firecrawl team ships. I wanted types for their node SDK, and less than an
hour later, I got them.\\\"](https://x.com/thepericulum/status/1781397799487078874)
[![Matt Busigin](https://www.firecrawl.dev/assets/testimonials/matt-busigin.png)Matt
Busigin@mbusigin\\\"Firecrawl is dope. Congrats guys \U0001F44F\\\"](https://x.com/mbusigin/status/1836065372010656069)
[![Sumanth](https://www.firecrawl.dev/assets/testimonials/sumanth.png)Sumanth@Sumanth\\\\_077\\\"Web
scraping will never be the same!\\\\\\\\\\n\\\\\\\\\\nFirecrawl is an open-source
framework that takes a URL, crawls it, and conver...\\\"](https://x.com/Sumanth_077/status/1940049003074478511)
[![Steven Tey](https://www.firecrawl.dev/assets/testimonials/steven-tey.png)Steven
Tey@steventey\\\"Open-source Clay alternative just dropped\\\\\\\\\\n\\\\\\\\\\nUpload
a CSV of emails and...\\\"](https://x.com/steventey/status/1932945651761098889)\\n\\n[![Bardia](https://www.firecrawl.dev/assets/testimonials/bardia.png)Bardia@thepericulum\\\"The
Firecrawl team ships. I wanted types for their node SDK, and less than an
hour later, I got them.\\\"](https://x.com/thepericulum/status/1781397799487078874)
[![Matt Busigin](https://www.firecrawl.dev/assets/testimonials/matt-busigin.png)Matt
Busigin@mbusigin\\\"Firecrawl is dope. Congrats guys \U0001F44F\\\"](https://x.com/mbusigin/status/1836065372010656069)
[![Sumanth](https://www.firecrawl.dev/assets/testimonials/sumanth.png)Sumanth@Sumanth\\\\_077\\\"Web
scraping will never be the same!\\\\\\\\\\n\\\\\\\\\\nFirecrawl is an open-source
framework that takes a URL, crawls it, and conver...\\\"](https://x.com/Sumanth_077/status/1940049003074478511)
[![Steven Tey](https://www.firecrawl.dev/assets/testimonials/steven-tey.png)Steven
Tey@steventey\\\"Open-source Clay alternative just dropped\\\\\\\\\\n\\\\\\\\\\nUpload
a CSV of emails and...\\\"](https://x.com/steventey/status/1932945651761098889)\\n\\n\\\\[06/
07 \\\\]\\n\\n\xB7\\n\\nUse Cases\\n\\n//\\n\\nUse cases\\n\\n//\\n\\n## Transform
\ web data into AI-powered solutions\\n\\nDiscover how Firecrawl customers
are getting the most out of our API.\\n\\n[View all use cases](https://docs.firecrawl.dev/use-cases/overview)\\n\\nChat
with context\\n\\nSmarter AI chats\\n\\nPower your AI assistants with real-time,
accurate web content.\\n\\n[View docs](https://docs.firecrawl.dev/introduction)\\n\\n![AI
Assistant](https://www.firecrawl.dev/assets/ai/bot.png)\\n\\nAI Assistant\\n\\nwithFirecrawl\\n\\nReal-time\xB7Updated
2 min ago\\n\\nAsk anything...\\n\\nKnow your leads\\n\\nLead enrichment\\n\\nEnhance
your sales data with\\n\\nweb information.\\n\\n[Check out Extract](https://www.firecrawl.dev/extract)\\n\\nExtracting
leads from directory...\\n\\nTech startups\\n\\nWith contact info\\n\\nDecision
makers\\n\\nFunding stage\\n\\nReady to engage\\n\\n![Emily Tran](https://www.firecrawl.dev/assets/ai/leads-1.png)\\n\\n![James
Carter](https://www.firecrawl.dev/assets/ai/leads-2.png)\\n\\n![Sophia Kim](https://www.firecrawl.dev/assets/ai/leads-3.png)\\n\\n![Michael
Rivera](https://www.firecrawl.dev/assets/ai/leads-4.png)\\n\\nKnow your leads\\n\\nMCPs\\n\\nAdd
powerful scraping to your\\n\\ncode editors.\\n\\n[Get started](https://docs.firecrawl.dev/mcp-server)\\n\\n![Claude
Code](https://www.firecrawl.dev/assets/ai/mcps-claude.png)\\n\\nClaude Code\\n\\n![Cursor](https://www.firecrawl.dev/assets/ai/mcps-cursor.png)\\n\\nCursor\\n\\n![Windsurf](https://www.firecrawl.dev/assets/ai/mcps-windsurf.png)\\n\\nWindsurf\\n\\n\u273B\\n\\nWelcome
to Claude Code!\\n\\n/help for help, /status for your current setup\\n\\n>Try
\\\"how do I log an error?\\\"\\n\\nBuild with context\\n\\nAI platforms\\n\\nLet
your customers build AI apps\\n\\nwith web data.\\n\\n[Check out Map](https://docs.firecrawl.dev/features/map)\\n\\n![Logo
1](https://www.firecrawl.dev/assets/ai/platforms-1.png)\\n\\n![Logo 2](https://www.firecrawl.dev/assets/ai/platforms-2.png)\\n\\n![Logo
4](https://www.firecrawl.dev/assets/ai/platforms-4.png)\\n\\n![Logo 3](https://www.firecrawl.dev/assets/ai/platforms-3.png)\\n\\nExtracting
text...\\n\\nNo insight missed\\n\\nDeep research\\n\\nExtract comprehensive
information for\\n\\nin-depth research.\\n\\n[Build your own with Search](https://docs.firecrawl.dev/features/search)\\n\\nDeep
research in progress...\\n\\nAcademic papers\\n\\n0 found\\n\\nNews articles\\n\\n0
found\\n\\nExpert opinions\\n\\n0 found\\n\\nResearch reports\\n\\n0 found\\n\\nIndustry
data\\n\\n0 found\\n\\nAsk anything...\\n\\n\\\\[ CTA \\\\]\\n\\n\\\\[ CRAWL
\\\\]\\n\\n\\\\[ SCRAPE \\\\]\\n\\n\\\\[ CTA \\\\]\\n\\n//\\n\\nGet started\\n\\n//\\n\\nReady
to build?\\n\\nStart getting Web Data for free and scale seamlessly as your
project expands. No credit card needed.\\n\\n[Start for free](https://www.firecrawl.dev/signin)
[See our plans](https://www.firecrawl.dev/pricing)\\n\\n\\\\[07/ 07 \\\\]\\n\\n\xB7\\n\\nFAQ\\n\\n//\\n\\nFAQ\\n\\n//\\n\\n##
Frequently askedquestions\\n\\nEverything you need to know about Firecrawl.\\n\\nGeneral\\n\\nWhat
is Firecrawl?\\n\\nWhat sites work?\\n\\nWho can benefit from using Firecrawl?\\n\\nIs
Firecrawl open-source?\\n\\nWhat is the difference between Firecrawl and other
web scrapers?\\n\\nWhat is the difference between the open-source version
and the hosted version?\\n\\nScraping & Crawling\\n\\nHow does Firecrawl handle
dynamic content on websites?\\n\\nWhy is it not crawling all the pages?\\n\\nCan
Firecrawl crawl websites without a sitemap?\\n\\nWhat formats can Firecrawl
convert web data into?\\n\\nHow does Firecrawl ensure the cleanliness of the
data?\\n\\nIs Firecrawl suitable for large-scale data scraping projects?\\n\\nDoes
it respect robots.txt?\\n\\nWhat measures does Firecrawl take to handle web
scraping challenges like rate limits and caching?\\n\\nDoes Firecrawl handle
captcha or authentication?\\n\\nAPI Related\\n\\nWhere can I find my API key?\\n\\nBilling\\n\\nIs
Firecrawl free?\\n\\nIs there a pay-per-use plan instead of monthly?\\n\\nDo
credits roll over to the next month?\\n\\nHow many credits do scraping and
crawling cost?\\n\\nDo you charge for failed requests?\\n\\nWhat payment methods
do you accept?\\n\\nFOOTER\\n\\nThe easiest way to extract\\n\\ndata from
the web\\n\\nBacked by\\n\\nY Combinator\\n\\n[Linkedin](https://www.linkedin.com/company/firecrawl)
[Github](https://github.com/firecrawl/firecrawl)\\n\\nSOC II \xB7 Type 2\\n\\nAICPA\\n\\nSOC
2\\n\\n[X (Twitter)](https://x.com/firecrawl_dev) [Discord](https://discord.gg/gSmWdAkdwd)\\n\\nProducts\\n\\n[Playground](https://www.firecrawl.dev/playground)
[Extract](https://www.firecrawl.dev/extract) [Pricing](https://www.firecrawl.dev/pricing)
[Templates](https://www.firecrawl.dev/templates) [Changelog](https://www.firecrawl.dev/changelog)\\n\\nUse
Cases\\n\\n[AI Platforms](https://docs.firecrawl.dev/use-cases/ai-platforms)
[Lead Enrichment](https://docs.firecrawl.dev/use-cases/lead-enrichment) [SEO
Platforms](https://docs.firecrawl.dev/use-cases/seo-platforms) [Deep Research](https://docs.firecrawl.dev/use-cases/deep-research)\\n\\nDocumentation\\n\\n[Getting
started](https://docs.firecrawl.dev/introduction) [API Reference](https://docs.firecrawl.dev/api-reference/introduction)
[Integrations](https://www.firecrawl.dev/app) [Examples](https://docs.firecrawl.dev/use-cases/overview)
[SDKs](https://docs.firecrawl.dev/sdks/overview)\\n\\nCompany\\n\\n[Blog](https://www.firecrawl.dev/blog)
[Careers](https://www.firecrawl.dev/careers) [Creator & OSS program](https://www.firecrawl.dev/creator-oss-program)
[Student program](https://www.firecrawl.dev/student-program)\\n\\n\xA9 2025
Firecrawl\\n\\n[Terms of Service](https://www.firecrawl.dev/terms-of-service)
[Privacy Policy](https://www.firecrawl.dev/privacy-policy) [Report Abuse](mailto:help@firecrawl.com?subject=Issue:)\\n\\n[All
systems normal](https://status.firecrawl.dev/)\\n\\nStripeM-Inner\",\"metadata\":{\"favicon\":\"https://www.firecrawl.dev/favicon.png\",\"ogUrl\":\"https://www.firecrawl.dev\",\"ogImage\":\"https://www.firecrawl.dev/og.png\",\"referrer\":\"origin-when-cross-origin\",\"ogDescription\":\"The
web crawling, scraping, and search API for AI. Built for scale. Firecrawl
delivers the entire internet to AI agents and builders. Clean, structured,
and ready to reason with.\",\"robots\":\"follow, index\",\"twitter:card\":\"summary_large_image\",\"og:site_name\":\"Firecrawl
- The Web Data API for AI\",\"twitter:title\":\"Firecrawl - The Web Data API
for AI\",\"og:image\":\"https://www.firecrawl.dev/og.png\",\"title\":\"Firecrawl
- The Web Data API for AI\",\"og:description\":\"The web crawling, scraping,
and search API for AI. Built for scale. Firecrawl delivers the entire internet
to AI agents and builders. Clean, structured, and ready to reason with.\",\"twitter:image\":\"https://www.firecrawl.dev/og.png\",\"viewport\":\"width=device-width,
initial-scale=1, maximum-scale=1, user-scalable=no\",\"ogSiteName\":\"Firecrawl
- The Web Data API for AI\",\"keywords\":\"Firecrawl,Markdown,Data,Mendable,Langchain\",\"author\":\"Firecrawl\",\"og:title\":\"Firecrawl
- The Web Data API for AI\",\"twitter:description\":\"The web crawling, scraping,
and search API for AI. Built for scale. Firecrawl delivers the entire internet
to AI agents and builders. Clean, structured, and ready to reason with.\",\"description\":\"The
web crawling, scraping, and search API for AI. Built for scale. Firecrawl
delivers the entire internet to AI agents and builders. Clean, structured,
and ready to reason with.\",\"twitter:site\":\"@Vercel\",\"og:url\":\"https://www.firecrawl.dev\",\"og:type\":\"website\",\"ogTitle\":\"Firecrawl
- The Web Data API for AI\",\"language\":\"en\",\"creator\":\"Firecrawl\",\"publisher\":\"Firecrawl\",\"twitter:creator\":\"@Vercel\",\"scrapeId\":\"57b0586f-36e8-4923-aaa2-88ff58c03999\",\"sourceURL\":\"https://www.firecrawl.dev/\",\"url\":\"https://www.firecrawl.dev/\",\"statusCode\":200,\"contentType\":\"text/html;
charset=utf-8\",\"proxyUsed\":\"basic\",\"cacheState\":\"hit\",\"cachedAt\":\"2025-10-29T13:09:07.713Z\"}},{\"url\":\"https://github.com/firecrawl/firecrawl\",\"title\":\"firecrawl/firecrawl:
The Web Data API for AI - Turn entire ... - GitHub\",\"description\":\"Firecrawl
is an API service that takes a URL, crawls it, and converts it into clean
markdown or structured data. We crawl all accessible subpages and give you
...\",\"position\":2,\"category\":\"github\",\"markdown\":\"[Skip to content](https://github.com/firecrawl/firecrawl#start-of-content)\\n\\nYou
signed in with another tab or window. [Reload](https://github.com/firecrawl/firecrawl)
to refresh your session.You signed out in another tab or window. [Reload](https://github.com/firecrawl/firecrawl)
to refresh your session.You switched accounts on another tab or window. [Reload](https://github.com/firecrawl/firecrawl)
to refresh your session.Dismiss alert\\n\\n{{ message }}\\n\\n[firecrawl](https://github.com/firecrawl)/
**[firecrawl](https://github.com/firecrawl/firecrawl)** Public\\n\\n- Couldn't
load subscription status.\\nRetry\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n### Uh
oh!\\n\\n\\n\\n\\n\\n\\n\\nThere was an error while loading. [Please reload
this page](https://github.com/firecrawl/firecrawl).\\n\\n- [Fork\\\\\\\\\\n5.1k](https://github.com/login?return_to=%2Ffirecrawl%2Ffirecrawl)\\n-
[Star\\\\\\\\\\n65.2k](https://github.com/login?return_to=%2Ffirecrawl%2Ffirecrawl)\\n\\n\\n\U0001F525
The Web Data API for AI - Turn entire websites into LLM-ready markdown or
structured data\\n\\n\\n[firecrawl.dev](https://firecrawl.dev/ \\\"https://firecrawl.dev\\\")\\n\\n###
License\\n\\n[AGPL-3.0 license](https://github.com/firecrawl/firecrawl/blob/main/LICENSE)\\n\\n[65.2k\\\\\\\\\\nstars](https://github.com/firecrawl/firecrawl/stargazers)
[5.1k\\\\\\\\\\nforks](https://github.com/firecrawl/firecrawl/forks) [Branches](https://github.com/firecrawl/firecrawl/branches)
[Tags](https://github.com/firecrawl/firecrawl/tags) [Activity](https://github.com/firecrawl/firecrawl/activity)\\n\\n[Star](https://github.com/login?return_to=%2Ffirecrawl%2Ffirecrawl)\\n\\nCouldn't
load subscription status.\\nRetry\\n\\n### Uh oh!\\n\\nThere was an error
while loading. [Please reload this page](https://github.com/firecrawl/firecrawl).\\n\\n#
firecrawl/firecrawl\\n\\nmain\\n\\n[**887** Branches](https://github.com/firecrawl/firecrawl/branches)
[**28** Tags](https://github.com/firecrawl/firecrawl/tags)\\n\\n[Go to Branches
page](https://github.com/firecrawl/firecrawl/branches)[Go to Tags page](https://github.com/firecrawl/firecrawl/tags)\\n\\nGo
to file\\n\\nCode\\n\\nOpen more actions menu\\n\\n## Folders and files\\n\\n|
Name | Name | Last commit message | Last commit date |\\n| --- | --- | ---
| --- |\\n| ## Latest commit<br>[![amplitudesxd](https://avatars.githubusercontent.com/u/62763456?v=4&size=40)](https://github.com/amplitudesxd)[amplitudesxd](https://github.com/firecrawl/firecrawl/commits?author=amplitudesxd)<br>[chore:
update last scrape rpc (](https://github.com/firecrawl/firecrawl/commit/37de2877fab4bae2de297e37bad3c9bcd49a64bc)
[#2339](https://github.com/firecrawl/firecrawl/pull/2339) [)](https://github.com/firecrawl/firecrawl/commit/37de2877fab4bae2de297e37bad3c9bcd49a64bc)<br>success<br>20
hours agoOct 27, 2025<br>[37de287](https://github.com/firecrawl/firecrawl/commit/37de2877fab4bae2de297e37bad3c9bcd49a64bc)\_\xB7\_20
hours agoOct 27, 2025<br>## History<br>[4,487 Commits](https://github.com/firecrawl/firecrawl/commits/main/)
<br>Open commit details<br>[View commit history for this file.](https://github.com/firecrawl/firecrawl/commits/main/)
|\\n| [.github](https://github.com/firecrawl/firecrawl/tree/main/.github \\\".github\\\")
| [.github](https://github.com/firecrawl/firecrawl/tree/main/.github \\\".github\\\")
| [fix(ci): temp disabled prod env tests](https://github.com/firecrawl/firecrawl/commit/42fc149c1ab738da0e15e772817774aa35273f8e
\\\"fix(ci): temp disabled prod env tests\\\") | 5 days agoOct 23, 2025 |\\n|
[apps](https://github.com/firecrawl/firecrawl/tree/main/apps \\\"apps\\\")
| [apps](https://github.com/firecrawl/firecrawl/tree/main/apps \\\"apps\\\")
| [chore: update last scrape rpc (](https://github.com/firecrawl/firecrawl/commit/37de2877fab4bae2de297e37bad3c9bcd49a64bc
\\\"chore: update last scrape rpc (#2339)\\\") [#2339](https://github.com/firecrawl/firecrawl/pull/2339)
[)](https://github.com/firecrawl/firecrawl/commit/37de2877fab4bae2de297e37bad3c9bcd49a64bc
\\\"chore: update last scrape rpc (#2339)\\\") | 20 hours agoOct 27, 2025
|\\n| [examples](https://github.com/firecrawl/firecrawl/tree/main/examples
\\\"examples\\\") | [examples](https://github.com/firecrawl/firecrawl/tree/main/examples
\\\"examples\\\") | [Merge pull request](https://github.com/firecrawl/firecrawl/commit/7ad57003b4ad8b230ba8252129e52bafa62dfae9
\\\"Merge pull request #2172 from MAVRICK-1/firecrawl-gemini-screenshot-editor
\ feat: Add Firecrawl + Gemini 2.5 Flash Image CLI Editor\\\") [#2172](https://github.com/firecrawl/firecrawl/pull/2172)
[from MAVRICK-1/firecrawl-gemini-screenshot-e\u2026](https://github.com/firecrawl/firecrawl/commit/7ad57003b4ad8b230ba8252129e52bafa62dfae9
\\\"Merge pull request #2172 from MAVRICK-1/firecrawl-gemini-screenshot-editor
\ feat: Add Firecrawl + Gemini 2.5 Flash Image CLI Editor\\\") | last monthSep
23, 2025 |\\n| [img](https://github.com/firecrawl/firecrawl/tree/main/img
\\\"img\\\") | [img](https://github.com/firecrawl/firecrawl/tree/main/img
\\\"img\\\") | [updated readme](https://github.com/firecrawl/firecrawl/commit/4f904e774831dc598681d3e998d0e5e15abcec27
\\\"updated readme\\\") | 2 months agoAug 18, 2025 |\\n| [.gitattributes](https://github.com/firecrawl/firecrawl/blob/main/.gitattributes
\\\".gitattributes\\\") | [.gitattributes](https://github.com/firecrawl/firecrawl/blob/main/.gitattributes
\\\".gitattributes\\\") | [Initial commit](https://github.com/firecrawl/firecrawl/commit/a6c2a878119321a196f720cce4195e086f1c6b46
\\\"Initial commit\\\") | last yearApr 15, 2024 |\\n| [.gitignore](https://github.com/firecrawl/firecrawl/blob/main/.gitignore
\\\".gitignore\\\") | [.gitignore](https://github.com/firecrawl/firecrawl/blob/main/.gitignore
\\\".gitignore\\\") | [Nick: init](https://github.com/firecrawl/firecrawl/commit/ab3fa4838458c8303a67dd30fdd75a16b89cc20b
\\\"Nick: init\\\") | 3 weeks agoOct 10, 2025 |\\n| [.gitmodules](https://github.com/firecrawl/firecrawl/blob/main/.gitmodules
\\\".gitmodules\\\") | [.gitmodules](https://github.com/firecrawl/firecrawl/blob/main/.gitmodules
\\\".gitmodules\\\") | [mendableai -> firecrawl](https://github.com/firecrawl/firecrawl/commit/2f3bc4e7a7b1a67a29c06df629f79402ee1aad1b
\\\"mendableai -> firecrawl\\\") | 2 months agoAug 18, 2025 |\\n| [CLAUDE.md](https://github.com/firecrawl/firecrawl/blob/main/CLAUDE.md
\\\"CLAUDE.md\\\") | [CLAUDE.md](https://github.com/firecrawl/firecrawl/blob/main/CLAUDE.md
\\\"CLAUDE.md\\\") | [add claude file](https://github.com/firecrawl/firecrawl/commit/3f0873c788823258a7d9f55d1c8772aed4e1a8de
\\\"add claude file\\\") | 2 months agoAug 6, 2025 |\\n| [CONTRIBUTING.md](https://github.com/firecrawl/firecrawl/blob/main/CONTRIBUTING.md
\\\"CONTRIBUTING.md\\\") | [CONTRIBUTING.md](https://github.com/firecrawl/firecrawl/blob/main/CONTRIBUTING.md
\\\"CONTRIBUTING.md\\\") | [Add Rust to CONTRIBUTING (](https://github.com/firecrawl/firecrawl/commit/f396cb20b54c3c2d7e64882642c5df6310a01002
\\\"Add Rust to CONTRIBUTING (#2180)\\\") [#2180](https://github.com/firecrawl/firecrawl/pull/2180)
[)](https://github.com/firecrawl/firecrawl/commit/f396cb20b54c3c2d7e64882642c5df6310a01002
\\\"Add Rust to CONTRIBUTING (#2180)\\\") | last monthSep 18, 2025 |\\n| [LICENSE](https://github.com/firecrawl/firecrawl/blob/main/LICENSE
\\\"LICENSE\\\") | [LICENSE](https://github.com/firecrawl/firecrawl/blob/main/LICENSE
\\\"LICENSE\\\") | [Update SDKs to MIT license](https://github.com/firecrawl/firecrawl/commit/afb49e21e7cff595ebad9ce0b7aba13b88f39cf8
\\\"Update SDKs to MIT license\\\") | last yearJul 8, 2024 |\\n| [README.md](https://github.com/firecrawl/firecrawl/blob/main/README.md
\\\"README.md\\\") | [README.md](https://github.com/firecrawl/firecrawl/blob/main/README.md
\\\"README.md\\\") | [Update README.md](https://github.com/firecrawl/firecrawl/commit/a21430e97818d95099bb365be711d9227bd75590
\\\"Update README.md\\\") | 3 weeks agoOct 6, 2025 |\\n| [SELF\\\\_HOST.md](https://github.com/firecrawl/firecrawl/blob/main/SELF_HOST.md
\\\"SELF_HOST.md\\\") | [SELF\\\\_HOST.md](https://github.com/firecrawl/firecrawl/blob/main/SELF_HOST.md
\\\"SELF_HOST.md\\\") | [Allow self-hosted webhook delivery to private IP
addresses (](https://github.com/firecrawl/firecrawl/commit/5756b834884d481382ce1f5674836a56b7fee33d
\\\"Allow self-hosted webhook delivery to private IP addresses (#2232)\\\")
[#2232](https://github.com/firecrawl/firecrawl/pull/2232) [)](https://github.com/firecrawl/firecrawl/commit/5756b834884d481382ce1f5674836a56b7fee33d
\\\"Allow self-hosted webhook delivery to private IP addresses (#2232)\\\")
| 27 days agoOct 1, 2025 |\\n| [docker-compose.yaml](https://github.com/firecrawl/firecrawl/blob/main/docker-compose.yaml
\\\"docker-compose.yaml\\\") | [docker-compose.yaml](https://github.com/firecrawl/firecrawl/blob/main/docker-compose.yaml
\\\"docker-compose.yaml\\\") | [Fix a self-hosted docker-compose.yaml bug
caused by a recent firecraw\u2026](https://github.com/firecrawl/firecrawl/commit/7d4100b274889977fa1ba26344532d9d8747494c
\\\"Fix a self-hosted docker-compose.yaml bug caused by a recent firecrawl
change (#2252) Add EXTRACT_WORKER_PORT to docker-compose environment\\\")
| 3 weeks agoOct 4, 2025 |\\n| View all files |\\n\\n## Repository files navigation\\n\\n###
[![](https://raw.githubusercontent.com/firecrawl/firecrawl/main/img/firecrawl_logo.png)](https://raw.githubusercontent.com/firecrawl/firecrawl/main/img/firecrawl_logo.png)\\n\\n[Permalink:
](https://github.com/firecrawl/firecrawl#----)\\n\\n[![License](https://camo.githubusercontent.com/d8ec6c81115d21c81bc26f2c80f8987a4d2a72e538b88afaa738fad5cd6289ff/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f6c6963656e73652f66697265637261776c2f66697265637261776c)](https://github.com/firecrawl/firecrawl/blob/main/LICENSE)[![Downloads](https://camo.githubusercontent.com/9d76afe428b4085c8b7103f2f4e31da110ee154ad7320bace4348d92ac0c2450/68747470733a2f2f7374617469632e706570792e746563682f62616467652f66697265637261776c2d7079)](https://pepy.tech/project/firecrawl-py)[![GitHub
Contributors](https://camo.githubusercontent.com/a9eabcb95ba00300afa51ce546660540c1f65764492cb2ba8fb67fe541c7e97f/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f636f6e7472696275746f72732f66697265637261776c2f66697265637261776c2e737667)](https://github.com/firecrawl/firecrawl/graphs/contributors)[![Visit
firecrawl.dev](https://camo.githubusercontent.com/3576b8cb0e77344c001cc8456d28c830691cb96480d4b65be90f8a4c99dead56/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f56697369742d66697265637261776c2e6465762d6f72616e6765)](https://firecrawl.dev/)\\n\\n[![Follow
on X](https://camo.githubusercontent.com/610127222e603752676f0275682f12398f8e434706861d577c1f6688d999191c/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f466f6c6c6f772532306f6e253230582d3030303030303f7374796c653d666f722d7468652d6261646765266c6f676f3d78266c6f676f436f6c6f723d7768697465)](https://twitter.com/firecrawl_dev)[![Follow
on LinkedIn](https://camo.githubusercontent.com/8741d51bb8e1c8ae576ac05e875f826bcf80e8711dcf9225935bb78d5bb03802/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f466f6c6c6f772532306f6e2532304c696e6b6564496e2d3030373742353f7374796c653d666f722d7468652d6261646765266c6f676f3d6c696e6b6564696e266c6f676f436f6c6f723d7768697465)](https://www.linkedin.com/company/104100957)[![Join
our Discord](https://camo.githubusercontent.com/886138c89a84dc2ad74d06900f364d736ccf753b2732d59fbd4106f6310f3616/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f4a6f696e2532306f7572253230446973636f72642d3538363546323f7374796c653d666f722d7468652d6261646765266c6f676f3d646973636f7264266c6f676f436f6c6f723d7768697465)](https://discord.com/invite/gSmWdAkdwd)\\n\\n#
\U0001F525 Firecrawl\\n\\n[Permalink: \U0001F525 Firecrawl](https://github.com/firecrawl/firecrawl#-firecrawl)\\n\\nEmpower
your AI apps with clean data from any website. Featuring advanced scraping,
crawling, and data extraction capabilities.\\n\\n_This repository is in development,
and we\u2019re still integrating custom modules into the mono repo. It's not
fully ready for self-hosted deployment yet, but you can run it locally._\\n\\n##
What is Firecrawl?\\n\\n[Permalink: What is Firecrawl?](https://github.com/firecrawl/firecrawl#what-is-firecrawl)\\n\\n[Firecrawl](https://firecrawl.dev/?ref=github)
is an API service that takes a URL, crawls it, and converts it into clean
markdown or structured data. We crawl all accessible subpages and give you
clean data for each. No sitemap required. Check out our [documentation](https://docs.firecrawl.dev/).\\n\\nLooking
for our MCP? Check out the [repo here](https://github.com/firecrawl/firecrawl-mcp-server).\\n\\n_Pst.
hey, you, join our stargazers :)_\\n\\n[![GitHub stars](https://camo.githubusercontent.com/11f7ce76e9f1608b3470b3a23a1db3a7d9ec083ee18f2d826862f420bac800dc/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f73746172732f66697265637261776c2f66697265637261776c2e7376673f7374796c653d736f6369616c266c6162656c3d53746172266d61784167653d32353932303030)](https://github.com/firecrawl/firecrawl)\\n\\n##
How to use it?\\n\\n[Permalink: How to use it?](https://github.com/firecrawl/firecrawl#how-to-use-it)\\n\\nWe
provide an easy to use API with our hosted version. You can find the playground
and documentation [here](https://firecrawl.dev/playground). You can also self
host the backend if you'd like.\\n\\nCheck out the following resources to
get started:\\n\\n- [x] **API**: [Documentation](https://docs.firecrawl.dev/api-reference/introduction)\\n-
[x] **SDKs**: [Python](https://docs.firecrawl.dev/sdks/python), [Node](https://docs.firecrawl.dev/sdks/node)\\n-
[x] **LLM Frameworks**: [Langchain (python)](https://python.langchain.com/docs/integrations/document_loaders/firecrawl/),
[Langchain (js)](https://js.langchain.com/docs/integrations/document_loaders/web_loaders/firecrawl),
[Llama Index](https://docs.llamaindex.ai/en/latest/examples/data_connectors/WebPageDemo/#using-firecrawl-reader),
[Crew.ai](https://docs.crewai.com/), [Composio](https://composio.dev/tools/firecrawl/all),
[PraisonAI](https://docs.praison.ai/firecrawl/), [Superinterface](https://superinterface.ai/docs/assistants/functions/firecrawl),
[Vectorize](https://docs.vectorize.io/integrations/source-connectors/firecrawl)\\n-
[x] **Low-code Frameworks**: [Dify](https://dify.ai/blog/dify-ai-blog-integrated-with-firecrawl),
[Langflow](https://docs.langflow.org/), [Flowise AI](https://docs.flowiseai.com/integrations/langchain/document-loaders/firecrawl),
[Cargo](https://docs.getcargo.io/integration/firecrawl), [Pipedream](https://pipedream.com/apps/firecrawl/)\\n-
[x] **Community SDKs**: [Go](https://docs.firecrawl.dev/sdks/go), [Rust](https://docs.firecrawl.dev/sdks/rust)\\n-
[x] **Others**: [Zapier](https://zapier.com/apps/firecrawl/integrations),
[Pabbly Connect](https://www.pabbly.com/connect/integrations/firecrawl/)\\n-
[ ] Want an SDK or Integration? Let us know by opening an issue.\\n\\nTo
run locally, refer to guide [here](https://github.com/firecrawl/firecrawl/blob/main/CONTRIBUTING.md).\\n\\n###
API Key\\n\\n[Permalink: API Key](https://github.com/firecrawl/firecrawl#api-key)\\n\\nTo
use the API, you need to sign up on [Firecrawl](https://firecrawl.dev/) and
get an API key.\\n\\n### Features\\n\\n[Permalink: Features](https://github.com/firecrawl/firecrawl#features)\\n\\n-
[**Scrape**](https://github.com/firecrawl/firecrawl#scraping): scrapes a URL
and get its content in LLM-ready format (markdown, structured data via [LLM
Extract](https://github.com/firecrawl/firecrawl#llm-extraction-beta), screenshot,
html)\\n- [**Crawl**](https://github.com/firecrawl/firecrawl#crawling): scrapes
all the URLs of a web page and return content in LLM-ready format\\n- [**Map**](https://github.com/firecrawl/firecrawl#map):
input a website and get all the website urls - extremely fast\\n- [**Search**](https://github.com/firecrawl/firecrawl#search):
search the web and get full content from results\\n- [**Extract**](https://github.com/firecrawl/firecrawl#extract):
get structured data from single page, multiple pages or entire websites with
AI.\\n\\n### Powerful Capabilities\\n\\n[Permalink: Powerful Capabilities](https://github.com/firecrawl/firecrawl#powerful-capabilities)\\n\\n-
**LLM-ready formats**: markdown, structured data, screenshot, HTML, links,
metadata\\n- **The hard stuff**: proxies, anti-bot mechanisms, dynamic content
(js-rendered), output parsing, orchestration\\n- **Customizability**: exclude
tags, crawl behind auth walls with custom headers, max crawl depth, etc...\\n-
**Media parsing**: pdfs, docx, images\\n- **Reliability first**: designed
to get the data you need - no matter how hard it is\\n- **Actions**: click,
scroll, input, wait and more before extracting data\\n- **Batching**: scrape
thousands of URLs at the same time with a new async endpoint\\n- **Change
Tracking**: monitor and detect changes in website content over time\\n\\nYou
can find all of Firecrawl's capabilities and how to use them in our [documentation](https://docs.firecrawl.dev/)\\n\\n###
Crawling\\n\\n[Permalink: Crawling](https://github.com/firecrawl/firecrawl#crawling)\\n\\nUsed
to crawl a URL and all accessible subpages. This submits a crawl job and returns
a job ID to check the status of the crawl.\\n\\n```\\ncurl -X POST https://api.firecrawl.dev/v2/crawl
\\\\\\n -H 'Content-Type: application/json' \\\\\\n -H 'Authorization:
Bearer fc-YOUR_API_KEY' \\\\\\n -d '{\\n \\\"url\\\": \\\"https://docs.firecrawl.dev\\\",\\n
\ \\\"limit\\\": 10,\\n \\\"scrapeOptions\\\": {\\n \\\"formats\\\":
[\\\"markdown\\\", \\\"html\\\"]\\n }\\n }'\\n```\\n\\nReturns a crawl
job id and the url to check the status of the crawl.\\n\\n```\\n{\\n \\\"success\\\":
true,\\n \\\"id\\\": \\\"123-456-789\\\",\\n \\\"url\\\": \\\"https://api.firecrawl.dev/v2/crawl/123-456-789\\\"\\n}\\n```\\n\\n###
Check Crawl Job\\n\\n[Permalink: Check Crawl Job](https://github.com/firecrawl/firecrawl#check-crawl-job)\\n\\nUsed
to check the status of a crawl job and get its result.\\n\\n```\\ncurl -X
GET https://api.firecrawl.dev/v2/crawl/123-456-789 \\\\\\n -H 'Content-Type:
application/json' \\\\\\n -H 'Authorization: Bearer YOUR_API_KEY'\\n```\\n\\n```\\n{\\n
\ \\\"status\\\": \\\"completed\\\",\\n \\\"total\\\": 36,\\n \\\"creditsUsed\\\":
36,\\n \\\"expiresAt\\\": \\\"2024-00-00T00:00:00.000Z\\\",\\n \\\"data\\\":
[\\\\\\n {\\\\\\n \\\"markdown\\\": \\\"[Firecrawl Docs home page![light
logo](https://mintlify.s3-us-west-1.amazonaws.com/firecrawl/logo/light.svg)!...\\\",\\\\\\n
\ \\\"html\\\": \\\"<!DOCTYPE html><html lang=\\\\\\\"en\\\\\\\" class=\\\\\\\"js-focus-visible
lg:[--scroll-mt:9.5rem]\\\\\\\" data-js-focus-visible=\\\\\\\"\\\\\\\">...\\\",\\\\\\n
\ \\\"metadata\\\": {\\\\\\n \\\"title\\\": \\\"Build a 'Chat with
website' using Groq Llama 3 | Firecrawl\\\",\\\\\\n \\\"language\\\":
\\\"en\\\",\\\\\\n \\\"sourceURL\\\": \\\"https://docs.firecrawl.dev/learn/rag-llama3\\\",\\\\\\n
\ \\\"description\\\": \\\"Learn how to use Firecrawl, Groq Llama 3,
and Langchain to build a 'Chat with your website' bot.\\\",\\\\\\n \\\"ogLocaleAlternate\\\":
[],\\\\\\n \\\"statusCode\\\": 200\\\\\\n }\\\\\\n }\\\\\\n
\ ]\\\\\\n}\\\\\\n```\\\\\\n\\\\\\n### Scraping\\\\\\n\\\\\\n[Permalink: Scraping](https://github.com/firecrawl/firecrawl#scraping)\\\\\\n\\\\\\nUsed
to scrape a URL and get its content in the specified formats.\\\\\\n\\\\\\n```\\\\\\ncurl
-X POST https://api.firecrawl.dev/v2/scrape \\\\\\\\\\n -H 'Content-Type:
application/json' \\\\\\\\\\n -H 'Authorization: Bearer YOUR_API_KEY' \\\\\\\\\\n
\ -d '{\\\\\\n \\\"url\\\": \\\"https://docs.firecrawl.dev\\\",\\\\\\n
\ \\\"formats\\\" : [\\\"markdown\\\", \\\"html\\\"]\\\\\\n }'\\\\\\n```\\\\\\n\\\\\\nResponse:\\\\\\n\\\\\\n```\\\\\\n{\\\\\\n
\ \\\"success\\\": true,\\\\\\n \\\"data\\\": {\\\\\\n \\\"markdown\\\":
\\\"Launch Week I is here! [See our Day 2 Release \U0001F680](https://www.firecrawl.dev/blog/launch-week-i-day-2-doubled-rate-limits)[\U0001F4A5
Get 2 months free...\\\",\\\\\\n \\\"html\\\": \\\"<!DOCTYPE html><html
lang=\\\\\\\"en\\\\\\\" class=\\\\\\\"light\\\\\\\" style=\\\\\\\"color-scheme:
light;\\\\\\\"><body class=\\\\\\\"__variable_36bd41 __variable_d7dc5d font-inter
...\\\",\\\\\\n \\\"metadata\\\": {\\\\\\n \\\"title\\\": \\\"Home
- Firecrawl\\\",\\\\\\n \\\"description\\\": \\\"Firecrawl crawls and
converts any website into clean markdown.\\\",\\\\\\n \\\"language\\\":
\\\"en\\\",\\\\\\n \\\"keywords\\\": \\\"Firecrawl,Markdown,Data,Mendable,Langchain\\\",\\\\\\n
\ \\\"robots\\\": \\\"follow, index\\\",\\\\\\n \\\"ogTitle\\\":
\\\"Firecrawl\\\",\\\\\\n \\\"ogDescription\\\": \\\"Turn any website
into LLM-ready data.\\\",\\\\\\n \\\"ogUrl\\\": \\\"https://www.firecrawl.dev/\\\",\\\\\\n
\ \\\"ogImage\\\": \\\"https://www.firecrawl.dev/og.png?123\\\",\\\\\\n
\ \\\"ogLocaleAlternate\\\": [],\\\\\\n \\\"ogSiteName\\\": \\\"Firecrawl\\\",\\\\\\n
\ \\\"sourceURL\\\": \\\"https://firecrawl.dev\\\",\\\\\\n \\\"statusCode\\\":
200\\\\\\n }\\\\\\n }\\\\\\n}\\\\\\n```\\\\\\n\\\\\\n### Map\\\\\\n\\\\\\n[Permalink:
Map](https://github.com/firecrawl/firecrawl#map)\\\\\\n\\\\\\nUsed to map
a URL and get urls of the website. This returns most links present on the
website.\\\\\\n\\\\\\n```\\\\\\ncurl -X POST https://api.firecrawl.dev/v2/map
\\\\\\\\\\n -H 'Content-Type: application/json' \\\\\\\\\\n -H 'Authorization:
Bearer YOUR_API_KEY' \\\\\\\\\\n -d '{\\\\\\n \\\"url\\\": \\\"https://firecrawl.dev\\\"\\\\\\n
\ }'\\\\\\n```\\\\\\n\\\\\\nResponse:\\\\\\n\\\\\\n```\\\\\\n{\\\\\\n \\\"success\\\":
true,\\\\\\n \\\"links\\\": [\\\\\\n { \\\"url\\\": \\\"https://firecrawl.dev\\\",
\\\"title\\\": \\\"Firecrawl\\\", \\\"description\\\": \\\"Firecrawl is a
tool that allows you to crawl a website and get the data you need.\\\" },\\\\\\n
\ { \\\"url\\\": \\\"https://www.firecrawl.dev/pricing\\\", \\\"title\\\":
\\\"Firecrawl Pricing\\\", \\\"description\\\": \\\"Firecrawl Pricing\\\"
},\\\\\\n { \\\"url\\\": \\\"https://www.firecrawl.dev/blog\\\", \\\"title\\\":
\\\"Firecrawl Blog\\\", \\\"description\\\": \\\"Firecrawl Blog\\\" },\\\\\\n
\ { \\\"url\\\": \\\"https://www.firecrawl.dev/playground\\\", \\\"title\\\":
\\\"Firecrawl Playground\\\", \\\"description\\\": \\\"Firecrawl Playground\\\"
},\\\\\\n { \\\"url\\\": \\\"https://www.firecrawl.dev/smart-crawl\\\",
\\\"title\\\": \\\"Firecrawl Smart Crawl\\\", \\\"description\\\": \\\"Firecrawl
Smart Crawl\\\" }\\\\\\n ]\\\\\\n}\\\\\\n```\\\\\\n\\\\\\n#### Map with search\\\\\\n\\\\\\n[Permalink:
Map with search](https://github.com/firecrawl/firecrawl#map-with-search)\\\\\\n\\\\\\nMap
with `search` param allows you to search for specific urls inside a website.\\\\\\n\\\\\\n```\\\\\\ncurl
-X POST https://api.firecrawl.dev/v2/map \\\\\\\\\\n -H 'Content-Type:
application/json' \\\\\\\\\\n -H 'Authorization: Bearer YOUR_API_KEY' \\\\\\\\\\n
\ -d '{\\\\\\n \\\"url\\\": \\\"https://firecrawl.dev\\\",\\\\\\n \\\"search\\\":
\\\"docs\\\"\\\\\\n }'\\\\\\n```\\\\\\n\\\\\\nResponse will be an ordered
list from the most relevant to the least relevant.\\\\\\n\\\\\\n```\\\\\\n{\\\\\\n
\ \\\"success\\\": true,\\\\\\n \\\"links\\\": [\\\\\\n { \\\"url\\\":
\\\"https://docs.firecrawl.dev\\\", \\\"title\\\": \\\"Firecrawl Docs\\\",
\\\"description\\\": \\\"Firecrawl Docs\\\" },\\\\\\n { \\\"url\\\": \\\"https://docs.firecrawl.dev/sdks/python\\\",
\\\"title\\\": \\\"Firecrawl Python SDK\\\", \\\"description\\\": \\\"Firecrawl
Python SDK\\\" },\\\\\\n { \\\"url\\\": \\\"https://docs.firecrawl.dev/learn/rag-llama3\\\",
\\\"title\\\": \\\"Firecrawl RAG Llama 3\\\", \\\"description\\\": \\\"Firecrawl
RAG Llama 3\\\" }\\\\\\n ]\\\\\\n}\\\\\\n```\\\\\\n\\\\\\n### Search\\\\\\n\\\\\\n[Permalink:
Search](https://github.com/firecrawl/firecrawl#search)\\\\\\n\\\\\\nSearch
the web and get full content from results\\\\\\n\\\\\\nFirecrawl\u2019s search
API allows you to perform web searches and optionally scrape the search results
in one operation.\\\\\\n\\\\\\n- Choose specific output formats (markdown,
HTML, links, screenshots)\\\\\\n- Search the web with customizable parameters
(language, country, etc.)\\\\\\n- Optionally retrieve content from search
results in various formats\\\\\\n- Control the number of results and set timeouts\\\\\\n\\\\\\n```\\\\\\ncurl
-X POST https://api.firecrawl.dev/v2/search \\\\\\\\\\n -H \\\"Content-Type:
application/json\\\" \\\\\\\\\\n -H \\\"Authorization: Bearer fc-YOUR_API_KEY\\\"
\\\\\\\\\\n -d '{\\\\\\n \\\"query\\\": \\\"what is firecrawl?\\\",\\\\\\n
\ \\\"limit\\\": 5\\\\\\n }'\\\\\\n```\\\\\\n\\\\\\n#### Response\\\\\\n\\\\\\n[Permalink:
Response](https://github.com/firecrawl/firecrawl#response)\\\\\\n\\\\\\n```\\\\\\n{\\\\\\n
\ \\\"success\\\": true,\\\\\\n \\\"data\\\": [\\\\\\n {\\\\\\n \\\"url\\\":
\\\"https://firecrawl.dev\\\",\\\\\\n \\\"title\\\": \\\"Firecrawl |
Home Page\\\",\\\\\\n \\\"description\\\": \\\"Turn websites into LLM-ready
data with Firecrawl\\\"\\\\\\n },\\\\\\n {\\\\\\n \\\"url\\\":
\\\"https://docs.firecrawl.dev\\\",\\\\\\n \\\"title\\\": \\\"Documentation
| Firecrawl\\\",\\\\\\n \\\"description\\\": \\\"Learn how to use Firecrawl
in your own applications\\\"\\\\\\n }\\\\\\n ]\\\\\\n}\\\\\\n```\\\\\\n\\\\\\n####
With content scraping\\\\\\n\\\\\\n[Permalink: With content scraping](https://github.com/firecrawl/firecrawl#with-content-scraping)\\\\\\n\\\\\\n```\\\\\\ncurl
-X POST https://api.firecrawl.dev/v2/search \\\\\\\\\\n -H \\\"Content-Type:
application/json\\\" \\\\\\\\\\n -H \\\"Authorization: Bearer fc-YOUR_API_KEY\\\"
\\\\\\\\\\n -d '{\\\\\\n \\\"query\\\": \\\"what is firecrawl?\\\",\\\\\\n
\ \\\"limit\\\": 5,\\\\\\n \\\"scrapeOptions\\\": {\\\\\\n \\\"formats\\\":
[\\\"markdown\\\", \\\"links\\\"]\\\\\\n }\\\\\\n }'\\\\\\n```\\\\\\n\\\\\\n###
Extract (Beta)\\\\\\n\\\\\\n[Permalink: Extract (Beta)](https://github.com/firecrawl/firecrawl#extract-beta)\\\\\\n\\\\\\nGet
structured data from entire websites with a prompt and/or a schema.\\\\\\n\\\\\\nYou
can extract structured data from one or multiple URLs, including wildcards:\\\\\\n\\\\\\nSingle
Page:\\\\\\nExample: [https://firecrawl.dev/some-page](https://firecrawl.dev/some-page)\\\\\\n\\\\\\nMultiple
Pages / Full Domain\\\\\\nExample: [https://firecrawl.dev/](https://firecrawl.dev/)\\\\*\\\\\\n\\\\\\nWhen
you use /\\\\*, Firecrawl will automatically crawl and parse all URLs it can
discover in that domain, then extract the requested data.\\\\\\n\\\\\\n```\\\\\\ncurl
-X POST https://api.firecrawl.dev/v2/extract \\\\\\\\\\n -H 'Content-Type:
application/json' \\\\\\\\\\n -H 'Authorization: Bearer YOUR_API_KEY' \\\\\\\\\\n
\ -d '{\\\\\\n \\\"urls\\\": [\\\\\\n \\\"https://firecrawl.dev/*\\\",\\\\\\n
\ \\\"https://docs.firecrawl.dev/\\\",\\\\\\n \\\"https://www.ycombinator.com/companies\\\"\\\\\\n
\ ],\\\\\\n \\\"prompt\\\": \\\"Extract the company mission, whether
it is open source, and whether it is in Y Combinator from the page.\\\",\\\\\\n
\ \\\"schema\\\": {\\\\\\n \\\"type\\\": \\\"object\\\",\\\\\\n
\ \\\"properties\\\": {\\\\\\n \\\"company_mission\\\": {\\\\\\n
\ \\\"type\\\": \\\"string\\\"\\\\\\n },\\\\\\n \\\"is_open_source\\\":
{\\\\\\n \\\"type\\\": \\\"boolean\\\"\\\\\\n },\\\\\\n
\ \\\"is_in_yc\\\": {\\\\\\n \\\"type\\\": \\\"boolean\\\"\\\\\\n
\ }\\\\\\n },\\\\\\n \\\"required\\\": [\\\\\\n \\\"company_mission\\\",\\\\\\n
\ \\\"is_open_source\\\",\\\\\\n \\\"is_in_yc\\\"\\\\\\n
\ ]\\\\\\n }\\\\\\n }'\\\\\\n```\\\\\\n\\\\\\n```\\\\\\n{\\\\\\n
\ \\\"success\\\": true,\\\\\\n \\\"id\\\": \\\"44aa536d-f1cb-4706-ab87-ed0386685740\\\",\\\\\\n
\ \\\"urlTrace\\\": []\\\\\\n}\\\\\\n```\\\\\\n\\\\\\nIf you are using the
sdks, it will auto pull the response for you:\\\\\\n\\\\\\n```\\\\\\n{\\\\\\n
\ \\\"success\\\": true,\\\\\\n \\\"data\\\": {\\\\\\n \\\"company_mission\\\":
\\\"Firecrawl is the easiest way to extract data from the web. Developers
use us to reliably convert URLs into LLM-ready markdown or structured data
with a single API call.\\\",\\\\\\n \\\"supports_sso\\\": false,\\\\\\n
\ \\\"is_open_source\\\": true,\\\\\\n \\\"is_in_yc\\\": true\\\\\\n
\ }\\\\\\n}\\\\\\n```\\\\\\n\\\\\\n### LLM Extraction (Beta)\\\\\\n\\\\\\n[Permalink:
LLM Extraction (Beta)](https://github.com/firecrawl/firecrawl#llm-extraction-beta)\\\\\\n\\\\\\nUsed
to extract structured data from scraped pages.\\\\\\n\\\\\\n```\\\\\\ncurl
-X POST https://api.firecrawl.dev/v2/scrape \\\\\\\\\\n -H 'Content-Type:
application/json' \\\\\\\\\\n -H 'Authorization: Bearer YOUR_API_KEY' \\\\\\\\\\n
\ -d '{\\\\\\n \\\"url\\\": \\\"https://www.mendable.ai/\\\",\\\\\\n \\\"formats\\\":
[\\\\\\n {\\\\\\n \\\"type\\\": \\\"json\\\",\\\\\\n \\\"schema\\\":
{\\\\\\n \\\"type\\\": \\\"object\\\",\\\\\\n \\\"properties\\\":
{\\\\\\n \\\"company_mission\\\": { \\\"type\\\": \\\"string\\\"
},\\\\\\n \\\"supports_sso\\\": { \\\"type\\\": \\\"boolean\\\"
},\\\\\\n \\\"is_open_source\\\": { \\\"type\\\": \\\"boolean\\\"
},\\\\\\n \\\"is_in_yc\\\": { \\\"type\\\": \\\"boolean\\\" }\\\\\\n
\ }\\\\\\n }\\\\\\n }\\\\\\n ]\\\\\\n }'\\\\\\n```\\\\\\n\\\\\\n```\\\\\\n{\\\\\\n
\ \\\"success\\\": true,\\\\\\n \\\"data\\\": {\\\\\\n \\\"content\\\":
\\\"Raw Content\\\",\\\\\\n \\\"metadata\\\": {\\\\\\n \\\"title\\\":
\\\"Mendable\\\",\\\\\\n \\\"description\\\": \\\"Mendable allows you
to easily build AI chat applications. Ingest, customize, then deploy with
one line of code anywhere you want. Brought to you by SideGuide\\\",\\\\\\n
\ \\\"robots\\\": \\\"follow, index\\\",\\\\\\n \\\"ogTitle\\\":
\\\"Mendable\\\",\\\\\\n \\\"ogDescription\\\": \\\"Mendable allows you
to easily build AI chat applications. Ingest, customize, then deploy with
one line of code anywhere you want. Brought to you by SideGuide\\\",\\\\\\n
\ \\\"ogUrl\\\": \\\"https://mendable.ai/\\\",\\\\\\n \\\"ogImage\\\":
\\\"https://mendable.ai/mendable_new_og1.png\\\",\\\\\\n \\\"ogLocaleAlternate\\\":
[],\\\\\\n \\\"ogSiteName\\\": \\\"Mendable\\\",\\\\\\n \\\"sourceURL\\\":
\\\"https://mendable.ai/\\\"\\\\\\n },\\\\\\n \\\"json\\\": {\\\\\\n
\ \\\"company_mission\\\": \\\"Train a secure AI on your technical resources
that answers customer and employee questions so your team doesn't have to\\\",\\\\\\n
\ \\\"supports_sso\\\": true,\\\\\\n \\\"is_open_source\\\": false,\\\\\\n
\ \\\"is_in_yc\\\": true\\\\\\n }\\\\\\n }\\\\\\n}\\\\\\n```\\\\\\n\\\\\\n###
Extracting without a schema (New)\\\\\\n\\\\\\n[Permalink: Extracting without
a schema (New)](https://github.com/firecrawl/firecrawl#extracting-without-a-schema-new)\\\\\\n\\\\\\nYou
can now extract without a schema by just passing a `prompt` to the endpoint.
The llm chooses the structure of the data.\\\\\\n\\\\\\n```\\\\\\ncurl -X
POST https://api.firecrawl.dev/v2/scrape \\\\\\\\\\n -H 'Content-Type:
application/json' \\\\\\\\\\n -H 'Authorization: Bearer YOUR_API_KEY' \\\\\\\\\\n
\ -d '{\\\\\\n \\\"url\\\": \\\"https://docs.firecrawl.dev/\\\",\\\\\\n
\ \\\"formats\\\": [\\\\\\n {\\\\\\n \\\"type\\\": \\\"json\\\",\\\\\\n
\ \\\"prompt\\\": \\\"Extract the company mission from the page.\\\"\\\\\\n
\ }\\\\\\n ]\\\\\\n }'\\\\\\n```\\\\\\n\\\\\\n### Interacting
with the page with Actions (Cloud-only)\\\\\\n\\\\\\n[Permalink: Interacting
with the page with Actions (Cloud-only)](https://github.com/firecrawl/firecrawl#interacting-with-the-page-with-actions-cloud-only)\\\\\\n\\\\\\nFirecrawl
allows you to perform various actions on a web page before scraping its content.
This is particularly useful for interacting with dynamic content, navigating
through pages, or accessing content that requires user interaction.\\\\\\n\\\\\\nHere
is an example of how to use actions to navigate to google.com, search for
Firecrawl, click on the first result, and take a screenshot.\\\\\\n\\\\\\n```\\\\\\ncurl
-X POST https://api.firecrawl.dev/v2/scrape \\\\\\\\\\n -H 'Content-Type:
application/json' \\\\\\\\\\n -H 'Authorization: Bearer YOUR_API_KEY' \\\\\\\\\\n
\ -d '{\\\\\\n \\\"url\\\": \\\"google.com\\\",\\\\\\n \\\"formats\\\":
[\\\"markdown\\\"],\\\\\\n \\\"actions\\\": [\\\\\\n {\\\"type\\\":
\\\"wait\\\", \\\"milliseconds\\\": 2000},\\\\\\n {\\\"type\\\":
\\\"click\\\", \\\"selector\\\": \\\"textarea[title=\\\\\\\"Search\\\\\\\"]\\\"},\\\\\\n
\ {\\\"type\\\": \\\"wait\\\", \\\"milliseconds\\\": 2000},\\\\\\n
\ {\\\"type\\\": \\\"write\\\", \\\"text\\\": \\\"firecrawl\\\"},\\\\\\n
\ {\\\"type\\\": \\\"wait\\\", \\\"milliseconds\\\": 2000},\\\\\\n
\ {\\\"type\\\": \\\"press\\\", \\\"key\\\": \\\"ENTER\\\"},\\\\\\n
\ {\\\"type\\\": \\\"wait\\\", \\\"milliseconds\\\": 3000},\\\\\\n
\ {\\\"type\\\": \\\"click\\\", \\\"selector\\\": \\\"h3\\\"},\\\\\\n
\ {\\\"type\\\": \\\"wait\\\", \\\"milliseconds\\\": 3000},\\\\\\n
\ {\\\"type\\\": \\\"screenshot\\\"}\\\\\\n ]\\\\\\n }'\\\\\\n```\\\\\\n\\\\\\n###
Batch Scraping Multiple URLs (New)\\\\\\n\\\\\\n[Permalink: Batch Scraping
Multiple URLs (New)](https://github.com/firecrawl/firecrawl#batch-scraping-multiple-urls-new)\\\\\\n\\\\\\nYou
can now batch scrape multiple URLs at the same time. It is very similar to
how the /crawl endpoint works. It submits a batch scrape job and returns a
job ID to check the status of the batch scrape.\\\\\\n\\\\\\n```\\\\\\ncurl
-X POST https://api.firecrawl.dev/v2/batch/scrape \\\\\\\\\\n -H 'Content-Type:
application/json' \\\\\\\\\\n -H 'Authorization: Bearer YOUR_API_KEY' \\\\\\\\\\n
\ -d '{\\\\\\n \\\"urls\\\": [\\\"https://docs.firecrawl.dev\\\", \\\"https://docs.firecrawl.dev/sdks/overview\\\"],\\\\\\n
\ \\\"formats\\\" : [\\\"markdown\\\", \\\"html\\\"]\\\\\\n }'\\\\\\n```\\\\\\n\\\\\\n##
Using Python SDK\\\\\\n\\\\\\n[Permalink: Using Python SDK](https://github.com/firecrawl/firecrawl#using-python-sdk)\\\\\\n\\\\\\n###
Installing Python SDK\\\\\\n\\\\\\n[Permalink: Installing Python SDK](https://github.com/firecrawl/firecrawl#installing-python-sdk)\\\\\\n\\\\\\n```\\\\\\npip
install firecrawl-py\\\\\\n```\\\\\\n\\\\\\n### Crawl a website\\\\\\n\\\\\\n[Permalink:
Crawl a website](https://github.com/firecrawl/firecrawl#crawl-a-website)\\\\\\n\\\\\\n```\\\\\\nfrom
firecrawl import Firecrawl\\\\\\n\\\\\\nfirecrawl = Firecrawl(api_key=\\\"fc-YOUR_API_KEY\\\")\\\\\\n\\\\\\n#
Scrape a website (returns a Document)\\\\\\ndoc = firecrawl.scrape(\\\\\\n
\ \\\"https://firecrawl.dev\\\",\\\\\\n formats=[\\\"markdown\\\", \\\"html\\\"],\\\\\\n)\\\\\\nprint(doc.markdown)\\\\\\n\\\\\\n#
Crawl a website\\\\\\nresponse = firecrawl.crawl(\\\\\\n \\\"https://firecrawl.dev\\\",\\\\\\n
\ limit=100,\\\\\\n scrape_options={\\\"formats\\\": [\\\"markdown\\\",
\\\"html\\\"]},\\\\\\n poll_interval=30,\\\\\\n)\\\\\\nprint(response)\\\\\\n```\\\\\\n\\\\\\n###
Extracting structured data from a URL\\\\\\n\\\\\\n[Permalink: Extracting
structured data from a URL](https://github.com/firecrawl/firecrawl#extracting-structured-data-from-a-url)\\\\\\n\\\\\\nWith
LLM extraction, you can easily extract structured data from any URL. We support
pydantic schemas to make it easier for you too. Here is how you to use it:\\\\\\n\\\\\\n```\\\\\\nfrom
pydantic import BaseModel, Field\\\\\\nfrom typing import List\\\\\\n\\\\\\nclass
Article(BaseModel):\\\\\\n title: str\\\\\\n points: int\\\\\\n by:
str\\\\\\n commentsURL: str\\\\\\n\\\\\\nclass TopArticles(BaseModel):\\\\\\n
\ top: List[Article] = Field(..., description=\\\"Top 5 stories\\\")\\\\\\n\\\\\\n#
Use JSON format with a Pydantic schema\\\\\\ndoc = firecrawl.scrape(\\\\\\n
\ \\\"https://news.ycombinator.com\\\",\\\\\\n formats=[{\\\"type\\\":
\\\"json\\\", \\\"schema\\\": TopArticles}],\\\\\\n)\\\\\\nprint(doc.json)\\\\\\n```\\\\\\n\\\\\\n##
Using the Node SDK\\\\\\n\\\\\\n[Permalink: Using the Node SDK](https://github.com/firecrawl/firecrawl#using-the-node-sdk)\\\\\\n\\\\\\n###
Installation\\\\\\n\\\\\\n[Permalink: Installation](https://github.com/firecrawl/firecrawl#installation)\\\\\\n\\\\\\nTo
install the Firecrawl Node SDK, you can use npm:\\\\\\n\\\\\\n```\\\\\\nnpm
install @mendable/firecrawl-js\\\\\\n```\\\\\\n\\\\\\n### Usage\\\\\\n\\\\\\n[Permalink:
Usage](https://github.com/firecrawl/firecrawl#usage)\\\\\\n\\\\\\n1. Get an
API key from [firecrawl.dev](https://firecrawl.dev/)\\\\\\n2. Set the API
key as an environment variable named `FIRECRAWL_API_KEY` or pass it as a parameter
to the `Firecrawl` class.\\\\\\n\\\\\\n```\\\\\\nimport Firecrawl from '@mendable/firecrawl-js';\\\\\\n\\\\\\nconst
firecrawl = new Firecrawl({ apiKey: 'fc-YOUR_API_KEY' });\\\\\\n\\\\\\n//
Scrape a website\\\\\\nconst doc = await firecrawl.scrape('https://firecrawl.dev',
{\\\\\\n formats: ['markdown', 'html'],\\\\\\n});\\\\\\nconsole.log(doc);\\\\\\n\\\\\\n//
Crawl a website\\\\\\nconst response = await firecrawl.crawl('https://firecrawl.dev',
{\\\\\\n limit: 100,\\\\\\n scrapeOptions: { formats: ['markdown', 'html']
},\\\\\\n});\\\\\\nconsole.log(response);\\\\\\n```\\\\\\n\\\\\\n### Extracting
structured data from a URL\\\\\\n\\\\\\n[Permalink: Extracting structured
data from a URL](https://github.com/firecrawl/firecrawl#extracting-structured-data-from-a-url-1)\\\\\\n\\\\\\nWith
LLM extraction, you can easily extract structured data from any URL. We support
zod schema to make it easier for you too. Here is how to use it:\\\\\\n\\\\\\n```\\\\\\nimport
Firecrawl from '@mendable/firecrawl-js';\\\\\\nimport { z } from 'zod';\\\\\\n\\\\\\nconst
firecrawl = new Firecrawl({ apiKey: 'fc-YOUR_API_KEY' });\\\\\\n\\\\\\n//
Define schema to extract contents into\\\\\\nconst schema = z.object({\\\\\\n
\ top: z\\\\\\n .array(\\\\\\n z.object({\\\\\\n title: z.string(),\\\\\\n
\ points: z.number(),\\\\\\n by: z.string(),\\\\\\n commentsURL:
z.string(),\\\\\\n })\\\\\\n )\\\\\\n .length(5)\\\\\\n .describe('Top
5 stories on Hacker News'),\\\\\\n});\\\\\\n\\\\\\n// Use the v2 extract API
with direct Zod schema support\\\\\\nconst extractRes = await firecrawl.extract({\\\\\\n
\ urls: ['https://news.ycombinator.com'],\\\\\\n schema,\\\\\\n prompt:
'Extract the top 5 stories',\\\\\\n});\\\\\\n\\\\\\nconsole.log(extractRes);\\\\\\n```\\\\\\n\\\\\\n##
Open Source vs Cloud Offering\\\\\\n\\\\\\n[Permalink: Open Source vs Cloud
Offering](https://github.com/firecrawl/firecrawl#open-source-vs-cloud-offering)\\\\\\n\\\\\\nFirecrawl
is open source available under the AGPL-3.0 license.\\\\\\n\\\\\\nTo deliver
the best possible product, we offer a hosted version of Firecrawl alongside
our open-source offering. The cloud solution allows us to continuously innovate
and maintain a high-quality, sustainable service for all users.\\\\\\n\\\\\\nFirecrawl
Cloud is available at [firecrawl.dev](https://firecrawl.dev/) and offers a
range of features that are not available in the open source version:\\\\\\n\\\\\\n[![Open
Source vs Cloud Offering](https://raw.githubusercontent.com/firecrawl/firecrawl/main/img/open-source-cloud.png)](https://raw.githubusercontent.com/firecrawl/firecrawl/main/img/open-source-cloud.png)\\\\\\n\\\\\\n##
Contributing\\\\\\n\\\\\\n[Permalink: Contributing](https://github.com/firecrawl/firecrawl#contributing)\\\\\\n\\\\\\nWe
love contributions! Please read our [contributing guide](https://github.com/firecrawl/firecrawl/blob/main/CONTRIBUTING.md)
before submitting a pull request. If you'd like to self-host, refer to the
[self-hosting guide](https://github.com/firecrawl/firecrawl/blob/main/SELF_HOST.md).\\\\\\n\\\\\\n_It
is the sole responsibility of the end users to respect websites' policies
when scraping, searching and crawling with Firecrawl. Users are advised to
adhere to the applicable privacy policies and terms of use of the websites
prior to initiating any scraping activities. By default, Firecrawl respects
the directives specified in the websites' robots.txt files when crawling.
By utilizing Firecrawl, you expressly agree to comply with these conditions._\\\\\\n\\\\\\n##
Contributors\\\\\\n\\\\\\n[Permalink: Contributors](https://github.com/firecrawl/firecrawl#contributors)\\\\\\n\\\\\\n[![contributors](https://camo.githubusercontent.com/e3b2e7ad4c1f76e68fc11ede158c87a0f039052f649002a1ff855d13fb9294fb/68747470733a2f2f636f6e747269622e726f636b732f696d6167653f7265706f3d66697265637261776c2f66697265637261776c)](https://github.com/firecrawl/firecrawl/graphs/contributors)\\\\\\n\\\\\\n##
License Disclaimer\\\\\\n\\\\\\n[Permalink: License Disclaimer](https://github.com/firecrawl/firecrawl#license-disclaimer)\\\\\\n\\\\\\nThis
project is primarily licensed under the GNU Affero General Public License
v3.0 (AGPL-3.0), as specified in the LICENSE file in the root directory of
this repository. However, certain components of this project are licensed
under the MIT License. Refer to the LICENSE files in these specific directories
for details.\\\\\\n\\\\\\nPlease note:\\\\\\n\\\\\\n- The AGPL-3.0 license
applies to all parts of the project unless otherwise specified.\\\\\\n- The
SDKs and some UI components are licensed under the MIT License. Refer to the
LICENSE files in these specific directories for details.\\\\\\n- When using
or contributing to this project, ensure you comply with the appropriate license
terms for the specific component you are working with.\\\\\\n\\\\\\nFor more
details on the licensing of specific components, please refer to the LICENSE
files in the respective directories or contact the project maintainers.\\\\\\n\\\\\\n[\u2191
Back to Top \u2191](https://github.com/firecrawl/firecrawl#readme-top)\\\\\\n\\\\\\n##
About\\\\\\n\\\\\\n\U0001F525 The Web Data API for AI - Turn entire websites
into LLM-ready markdown or structured data\\\\\\n\\\\\\n\\\\\\n[firecrawl.dev](https://firecrawl.dev/
\\\"https://firecrawl.dev\\\")\\\\\\n\\\\\\n### Topics\\\\\\n\\\\\\n[markdown](https://github.com/topics/markdown
\\\"Topic: markdown\\\") [crawler](https://github.com/topics/crawler \\\"Topic:
crawler\\\") [scraper](https://github.com/topics/scraper \\\"Topic: scraper\\\")
[ai](https://github.com/topics/ai \\\"Topic: ai\\\") [html-to-markdown](https://github.com/topics/html-to-markdown
\\\"Topic: html-to-markdown\\\") [web-crawler](https://github.com/topics/web-crawler
\\\"Topic: web-crawler\\\") [scraping](https://github.com/topics/scraping
\\\"Topic: scraping\\\") [web-scraper](https://github.com/topics/web-scraper
\\\"Topic: web-scraper\\\") [web-scraping](https://github.com/topics/web-scraping
\\\"Topic: web-scraping\\\") [data-extraction](https://github.com/topics/data-extraction
\\\"Topic: data-extraction\\\") [webscraping](https://github.com/topics/webscraping
\\\"Topic: webscraping\\\") [web-data-extraction](https://github.com/topics/web-data-extraction
\\\"Topic: web-data-extraction\\\") [ai-agents](https://github.com/topics/ai-agents
\\\"Topic: ai-agents\\\") [web-search](https://github.com/topics/web-search
\\\"Topic: web-search\\\") [ai-search](https://github.com/topics/ai-search
\\\"Topic: ai-search\\\") [web-data](https://github.com/topics/web-data \\\"Topic:
web-data\\\") [llm](https://github.com/topics/llm \\\"Topic: llm\\\") [ai-crawler](https://github.com/topics/ai-crawler
\\\"Topic: ai-crawler\\\") [ai-scraping](https://github.com/topics/ai-scraping
\\\"Topic: ai-scraping\\\")\\\\\\n\\\\\\n### Resources\\\\\\n\\\\\\n[Readme](https://github.com/firecrawl/firecrawl#readme-ov-file)\\\\\\n\\\\\\n###
License\\\\\\n\\\\\\n[AGPL-3.0 license](https://github.com/firecrawl/firecrawl#AGPL-3.0-1-ov-file)\\\\\\n\\\\\\n###
Contributing\\\\\\n\\\\\\n[Contributing](https://github.com/firecrawl/firecrawl#contributing-ov-file)\\\\\\n\\\\\\n###
Uh oh!\\\\\\n\\\\\\nThere was an error while loading. [Please reload this
page](https://github.com/firecrawl/firecrawl).\\\\\\n\\\\\\n[Activity](https://github.com/firecrawl/firecrawl/activity)\\\\\\n\\\\\\n[Custom
properties](https://github.com/firecrawl/firecrawl/custom-properties)\\\\\\n\\\\\\n###
Stars\\\\\\n\\\\\\n[**65.2k**\\\\\\\\\\nstars](https://github.com/firecrawl/firecrawl/stargazers)\\\\\\n\\\\\\n###
Watchers\\\\\\n\\\\\\n[**256**\\\\\\\\\\nwatching](https://github.com/firecrawl/firecrawl/watchers)\\\\\\n\\\\\\n###
Forks\\\\\\n\\\\\\n[**5.1k**\\\\\\\\\\nforks](https://github.com/firecrawl/firecrawl/forks)\\\\\\n\\\\\\n[Report
repository](https://github.com/contact/report-content?content_url=https%3A%2F%2Fgithub.com%2Ffirecrawl%2Ffirecrawl&report=firecrawl+%28user%29)\\\\\\n\\\\\\n##
[Releases\\\\ 28](https://github.com/firecrawl/firecrawl/releases)\\\\\\n\\\\\\n[v2.4.0\\\\\\\\\\nLatest\\\\\\\\\\n\\\\\\\\\\n2
weeks agoOct 13, 2025](https://github.com/firecrawl/firecrawl/releases/tag/v2.4.0)\\\\\\n\\\\\\n[\\\\+
27 releases](https://github.com/firecrawl/firecrawl/releases)\\\\\\n\\\\\\n##
[Packages\\\\ 3](https://github.com/orgs/firecrawl/packages?repo_name=firecrawl)\\\\\\n\\\\\\n-
[firecrawl](https://github.com/orgs/firecrawl/packages/container/package/firecrawl)\\\\\\n-
[playwright-service](https://github.com/orgs/firecrawl/packages/container/package/playwright-service)\\\\\\n-
[nuq-postgres](https://github.com/orgs/firecrawl/packages/container/package/nuq-postgres)\\\\\\n\\\\\\n##
[Contributors\\\\ 121](https://github.com/firecrawl/firecrawl/graphs/contributors)\\\\\\n\\\\\\n[\\\\+
107 contributors](https://github.com/firecrawl/firecrawl/graphs/contributors)\\\\\\n\\\\\\n##
Languages\\\\\\n\\\\\\n- [TypeScript73.5%](https://github.com/firecrawl/firecrawl/search?l=typescript)\\\\\\n-
[Python18.9%](https://github.com/firecrawl/firecrawl/search?l=python)\\\\\\n-
[Rust6.0%](https://github.com/firecrawl/firecrawl/search?l=rust)\\\\\\n- [Astro0.6%](https://github.com/firecrawl/firecrawl/search?l=astro)\\\\\\n-
[JavaScript0.3%](https://github.com/firecrawl/firecrawl/search?l=javascript)\\\\\\n-
[Jupyter Notebook0.2%](https://github.com/firecrawl/firecrawl/search?l=jupyter-notebook)\\\\\\n-
Other0.5%\",\"metadata\":{\"octolytics-dimension-repository_network_root_id\":\"787076358\",\"visitor-hmac\":\"163b2538b2335f7d4000a770785477e15881e1101708ca5ff1e8c021095f6ca1\",\"og:type\":\"object\",\"language\":\"en\",\"route-action\":\"disambiguate\",\"og:title\":\"GitHub
- firecrawl/firecrawl: \U0001F525 The Web Data API for AI - Turn entire websites
into LLM-ready markdown or structured data\",\"octolytics-dimension-repository_public\":\"true\",\"octolytics-dimension-repository_network_root_nwo\":\"firecrawl/firecrawl\",\"browser-errors-url\":\"https://api.github.com/_private/browser/errors\",\"browser-stats-url\":\"https://api.github.com/_private/browser/stats\",\"twitter:title\":\"GitHub
- firecrawl/firecrawl: \U0001F525 The Web Data API for AI - Turn entire websites
into LLM-ready markdown or structured data\",\"ui-target\":\"full\",\"og:image\":\"https://repository-images.githubusercontent.com/787076358/f9616c09-3701-41ef-b5a6-fdf912ffb15b\",\"google-site-verification\":\"Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I\",\"ogSiteName\":\"GitHub\",\"route-pattern\":\"/:user_id/:repository\",\"visitor-payload\":\"eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiJBMTVGOjE3OUI0RDo2MzdFQUREOjg3MzEwOTM6NjkwMTE4MjUiLCJ2aXNpdG9yX2lkIjoiNDkyNzk2MzExNjg5OTIxMTMwMSIsInJlZ2lvbl9lZGdlIjoiaWFkIiwicmVnaW9uX3JlbmRlciI6ImlhZCJ9\",\"og:description\":\"\U0001F525
The Web Data API for AI - Turn entire websites into LLM-ready markdown or
structured data - firecrawl/firecrawl\",\"expected-hostname\":\"github.com\",\"release\":\"66136a30a16cc69206f1249b6ba072daa2174535\",\"title\":\"GitHub
- firecrawl/firecrawl: \U0001F525 The Web Data API for AI - Turn entire websites
into LLM-ready markdown or structured data\",\"ogDescription\":\"\U0001F525
The Web Data API for AI - Turn entire websites into LLM-ready markdown or
structured data - firecrawl/firecrawl\",\"twitter:card\":\"summary_large_image\",\"fb:app_id\":\"1401488693436528\",\"color-scheme\":\"light
dark\",\"twitter:description\":\"\U0001F525 The Web Data API for AI - Turn
entire websites into LLM-ready markdown or structured data - firecrawl/firecrawl\",\"favicon\":\"https://github.githubassets.com/favicons/favicon.svg\",\"viewport\":\"width=device-width\",\"twitter:image\":\"https://repository-images.githubusercontent.com/787076358/f9616c09-3701-41ef-b5a6-fdf912ffb15b\",\"user-login\":\"\",\"description\":\"\U0001F525
The Web Data API for AI - Turn entire websites into LLM-ready markdown or
structured data - firecrawl/firecrawl\",\"octolytics-dimension-repository_nwo\":\"firecrawl/firecrawl\",\"octolytics-dimension-user_id\":\"135057108\",\"twitter:site\":\"@github\",\"og:url\":\"https://github.com/firecrawl/firecrawl\",\"octolytics-dimension-user_login\":\"firecrawl\",\"hostname\":\"github.com\",\"current-catalog-service-hash\":\"f3abb0cc802f3d7b95fc8762b94bdcb13bf39634c40c357301c4aa1d67a256fb\",\"html-safe-nonce\":\"e5653da3800db7de2c8b4a64ff6367242043a9e452608e9a6941a1c9e8346cfc\",\"apple-itunes-app\":\"app-id=1477376905,
app-argument=https://github.com/firecrawl/firecrawl\",\"turbo-cache-control\":\"no-preview\",\"og:site_name\":\"GitHub\",\"request-id\":\"A15F:179B4D:637EADD:8731093:69011825\",\"octolytics-url\":\"https://collector.github.com/github/collect\",\"octolytics-dimension-repository_is_fork\":\"false\",\"fetch-nonce\":\"v2:bc80e85b-edaf-e746-9f2c-ffdc75854b07\",\"og:image:alt\":\"\U0001F525
The Web Data API for AI - Turn entire websites into LLM-ready markdown or
structured data - firecrawl/firecrawl\",\"github-keyboard-shortcuts\":\"repository,copilot\",\"ogTitle\":\"GitHub
- firecrawl/firecrawl: \U0001F525 The Web Data API for AI - Turn entire websites
into LLM-ready markdown or structured data\",\"ogImage\":\"https://repository-images.githubusercontent.com/787076358/f9616c09-3701-41ef-b5a6-fdf912ffb15b\",\"analytics-location\":\"/<user-name>/<repo-name>\",\"route-controller\":\"files\",\"octolytics-dimension-repository_id\":\"787076358\",\"ogUrl\":\"https://github.com/firecrawl/firecrawl\",\"go-import\":\"github.com/firecrawl/firecrawl
git https://github.com/firecrawl/firecrawl.git\",\"hovercard-subject-tag\":\"repository:787076358\",\"theme-color\":\"#1e2327\",\"turbo-body-classes\":\"logged-out
env-production page-responsive\",\"scrapeId\":\"ec4d99a0-4c4f-4d1a-9fd2-08b8f891f883\",\"sourceURL\":\"https://github.com/firecrawl/firecrawl\",\"url\":\"https://github.com/firecrawl/firecrawl\",\"statusCode\":200,\"contentType\":\"text/html;
charset=utf-8\",\"proxyUsed\":\"basic\",\"cacheState\":\"hit\",\"cachedAt\":\"2025-10-28T19:23:20.106Z\"}},{\"url\":\"https://x.com/firecrawl_dev?lang=en\",\"title\":\"Firecrawl
(@firecrawl_dev) / Posts / X\",\"description\":\"Firecrawl (@firecrawl_dev)
- Posts - Turn websites into LLM-ready data. Built by @mendableai team Open
source: | X (formerly Twitter)\",\"position\":3},{\"url\":\"https://github.com/firecrawl\",\"title\":\"Firecrawl
- GitHub\",\"description\":\"Building AI applications? You need clean, structured
data from the web. Firecrawl handles the complexity of modern web scraping
so you can focus on building ...\",\"position\":4,\"category\":\"github\",\"markdown\":\"[Skip
to content](https://github.com/firecrawl#start-of-content)\\n\\nYou signed
in with another tab or window. [Reload](https://github.com/firecrawl) to refresh
your session.You signed out in another tab or window. [Reload](https://github.com/firecrawl)
to refresh your session.You switched accounts on another tab or window. [Reload](https://github.com/firecrawl)
to refresh your session.Dismiss alert\\n\\n{{ message }}\\n\\n[README.md](https://github.com/firecrawl/.github/tree/main/profile/README.md)\\n\\n#
\U0001F525 Firecrawl\\n\\n[Permalink: \U0001F525 Firecrawl](https://github.com/firecrawl#-firecrawl)\\n\\n[![Firecrawl
Logo](https://raw.githubusercontent.com/mendableai/firecrawl/main/img/firecrawl_logo.png)](https://raw.githubusercontent.com/mendableai/firecrawl/main/img/firecrawl_logo.png)\\n\\n###
Transform any website into LLM-ready data\\n\\n[Permalink: Transform any website
into LLM-ready data](https://github.com/firecrawl#transform-any-website-into-llm-ready-data)\\n\\nAdvanced
web scraping, crawling, and data extraction infrastructure for AI applications\\n\\n[![Get
Started](https://camo.githubusercontent.com/85b729c7fb201b60a98279ddc4e70268281fc6df999e110276f798cf5c050126/68747470733a2f2f696d672e736869656c64732e696f2f62616467652ff09f9a805f4765745f537461727465642d4646364233353f7374796c653d666f722d7468652d6261646765)](https://firecrawl.dev/)
[![Documentation](https://camo.githubusercontent.com/f7531cb91d3d3dcac76ac7c2ba9d36c1a74728016c64d942cfaf954f5ae4a238/68747470733a2f2f696d672e736869656c64732e696f2f62616467652ff09f939a5f446f63756d656e746174696f6e2d3441393045323f7374796c653d666f722d7468652d6261646765)](https://docs.firecrawl.dev/)
[![Discord](https://camo.githubusercontent.com/8c2d9f948c1d79b69e26d25add89a17a734b48cc6fd6fc0040f34f31c6a11774/68747470733a2f2f696d672e736869656c64732e696f2f62616467652ff09f92ac5f4a6f696e5f446973636f72642d3538363546323f7374796c653d666f722d7468652d6261646765)](https://discord.com/invite/gSmWdAkdwd)\\n\\n[![License](https://camo.githubusercontent.com/a6f4431b80529dbeaa43c3c5fbcf4649f6b4ebbeb82d5a58abeb39ca3eeca8be/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f6c6963656e73652f6d656e6461626c6561692f66697265637261776c)](https://github.com/mendableai/firecrawl/blob/main/LICENSE)[![GitHub
Stars](https://camo.githubusercontent.com/f42ce9a4d46d07baa67b74b49277e72ed33877743358deebfa774e17532eb4ff/68747470733a2f2f696d672e736869656c64732e696f2f6769746875622f73746172732f6d656e6461626c6561692f66697265637261776c3f7374796c653d736f6369616c)](https://github.com/mendableai/firecrawl/stargazers)[![Python
Downloads](https://camo.githubusercontent.com/9d76afe428b4085c8b7103f2f4e31da110ee154ad7320bace4348d92ac0c2450/68747470733a2f2f7374617469632e706570792e746563682f62616467652f66697265637261776c2d7079)](https://pepy.tech/project/firecrawl-py)[![Follow
on X](https://camo.githubusercontent.com/e32f3aece18eaab32ee100cadb592843d985aa1171a8da08ae047626363d65ae/68747470733a2f2f696d672e736869656c64732e696f2f747769747465722f666f6c6c6f772f66697265637261776c5f6465763f7374796c653d736f6369616c)](https://x.com/firecrawl_dev)\\n\\n*
* *\\n\\n## Why Firecrawl?\\n\\n[Permalink: Why Firecrawl?](https://github.com/firecrawl#why-firecrawl)\\n\\n**Building
AI applications?** You need clean, structured data from the web. Firecrawl
handles the complexity of modern web scraping so you can focus on building
great products.\\n\\n## Our Core Ecosystem\\n\\n[Permalink: Our Core Ecosystem](https://github.com/firecrawl#our-core-ecosystem)\\n\\n###
Main Repository\\n\\n[Permalink: Main Repository](https://github.com/firecrawl#main-repository)\\n\\n[![](https://camo.githubusercontent.com/97aa9741f2773cb2d192c516c7689f4e2bbab89403aa868d842487551743626a/68747470733a2f2f6769746875622d726561646d652d73746174732e76657263656c2e6170702f6170692f70696e2f3f757365726e616d653d6d656e6461626c656169267265706f3d66697265637261776c267468656d653d6c69676874)](https://github.com/mendableai/firecrawl)\\n\\n**[firecrawl](https://github.com/mendableai/firecrawl)**
\\\\- Core API & SDK\\n\\nTurn entire websites into LLM-ready markdown or
structured data. Our flagship product with 40k+ stars.\\n\\n### Cloud API\\n\\n[Permalink:
Cloud API](https://github.com/firecrawl#cloud-api)\\n\\n[![Cloud API](https://camo.githubusercontent.com/6ad9773ed98c84d54b1546ffbf8b0fbb085be0c60580ae5a1ead2d23ddf1b121/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f436c6f75645f4150492d4646364233353f7374796c653d666f722d7468652d6261646765266c6f676f3d636c6f7564266c6f676f436f6c6f723d7768697465)](https://firecrawl.dev/)\\n\\n**[Firecrawl](https://firecrawl.dev/)**
\\\\- Hosted API Service\\n\\nProduction-ready web scraping without infrastructure
management. Get your API key and start scraping in minutes with our reliable,
scalable cloud service.\\n\\n### MCP Integration\\n\\n[Permalink: MCP Integration](https://github.com/firecrawl#mcp-integration)\\n\\n[![MCP
Server](https://camo.githubusercontent.com/ea9cbd6a754e0932d17ae393ff53566bfb681492abddd08704dbe04b122dfaa1/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f4d43505f5365727665722d3441393045323f7374796c653d666f722d7468652d6261646765266c6f676f3d736572766572266c6f676f436f6c6f723d7768697465)](https://github.com/mendableai/firecrawl-mcp-server)\\n\\n**[firecrawl-mcp-server](https://github.com/mendableai/firecrawl-mcp-server)**
\\\\- Model Context Protocol Server\\n\\nAdd powerful web scraping capabilities
to Claude, Cursor, and any MCP-compatible LLM client.\\n\\n## Community &
Support\\n\\n[Permalink: Community & Support](https://github.com/firecrawl#community--support)\\n\\n[![Discord](https://camo.githubusercontent.com/62d3d35241760cf174631c4e6b5f4503c0a6b34640fd306e36a829ab5ec47b14/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f446973636f72642d3538363546323f7374796c653d666f722d7468652d6261646765266c6f676f3d646973636f7264266c6f676f436f6c6f723d7768697465)](https://discord.com/invite/gSmWdAkdwd)[![X](https://camo.githubusercontent.com/8c709aaebc7feee6050eba44984b294d9da3ace3353bd5eed8b499dd04af3c06/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f582d3030303030303f7374796c653d666f722d7468652d6261646765266c6f676f3d78266c6f676f436f6c6f723d7768697465)](https://x.com/firecrawl_dev)[![LinkedIn](https://camo.githubusercontent.com/8c0692475a5bfc1d9e7361074bdb648e567cae7b5b40ffd32adae31180b0d7b6/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f4c696e6b6564496e2d3030373742353f7374796c653d666f722d7468652d6261646765266c6f676f3d6c696e6b6564696e266c6f676f436f6c6f723d7768697465)](https://www.linkedin.com/company/104100957/)[![Discussions](https://camo.githubusercontent.com/9403fd9d6d54f5a23a79f9a8a6a256ae82159fb626710fd56c2495fff1257d62/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f4769744875625f44697363757373696f6e732d3138313731373f7374796c653d666f722d7468652d6261646765266c6f676f3d676974687562266c6f676f436f6c6f723d7768697465)](https://github.com/mendableai/firecrawl/discussions)[![Documentation](https://camo.githubusercontent.com/9d518c9da8018ae3524a2580522bd1ef591f343cc4df7983b4476e117fa70bba/68747470733a2f2f696d672e736869656c64732e696f2f62616467652f446f63756d656e746174696f6e2d3441393045323f7374796c653d666f722d7468652d6261646765266c6f676f3d626f6f6b266c6f676f436f6c6f723d7768697465)](https://docs.firecrawl.dev/)\\n\\n##
Built By Mendable\\n\\n[Permalink: Built By Mendable](https://github.com/firecrawl#built-by-mendable)\\n\\nWe're
the team behind [Mendable.ai](https://mendable.ai/), passionate about making
web data accessible for AI applications. Firecrawl powers thousands of AI
products worldwide.\\n\\n* * *\\n\\n**Ready to build something amazing?**\\n\\n[Get
your API key](https://firecrawl.dev/) and start scraping in minutes\\n\\n\\n[Star
our main repo](https://github.com/mendableai/firecrawl) \u2022\\n[Try the
playground](https://firecrawl.dev/playground) \u2022\\n[Read the docs](https://docs.firecrawl.dev/)\\n\\n##
Pinned Loading\\n\\n1. [firecrawl](https://github.com/firecrawl/firecrawl)
firecrawlPublic\\n\\n\\n\\n\\n\\n\\n\U0001F525 The Web Data API for AI - Turn
entire websites into LLM-ready markdown or structured data\\n\\n\\n\\n\\nTypeScript[64.9k](https://github.com/firecrawl/firecrawl/stargazers)
[5.1k](https://github.com/firecrawl/firecrawl/forks)\\n\\n2. [mendable-nextjs-chatbot](https://github.com/firecrawl/mendable-nextjs-chatbot)
mendable-nextjs-chatbotPublic template\\n\\n\\n\\n\\n\\n\\nNext.js Starter
Template for building chatbots with Mendable\\n\\n\\n\\n\\nTypeScript[256](https://github.com/firecrawl/mendable-nextjs-chatbot/stargazers)
[52](https://github.com/firecrawl/mendable-nextjs-chatbot/forks)\\n\\n3. [rag-arena](https://github.com/firecrawl/rag-arena)
rag-arenaPublic\\n\\n\\n\\n\\n\\n\\nOpen-source RAG evaluation through users'
feedback\\n\\n\\n\\n\\nTypeScript[206](https://github.com/firecrawl/rag-arena/stargazers)
[32](https://github.com/firecrawl/rag-arena/forks)\\n\\n4. [QA\\\\_clustering](https://github.com/firecrawl/QA_clustering)
QA\\\\_clusteringPublic\\n\\n\\n\\n\\n\\n\\nAnalyzing chat interactions w/
LLMs to improve \U0001F99C\U0001F517 Langchain docs\\n\\n\\n\\n\\nJupyter
Notebook[80](https://github.com/firecrawl/QA_clustering/stargazers) [12](https://github.com/firecrawl/QA_clustering/forks)\\n\\n5.
[data-connectors](https://github.com/firecrawl/data-connectors) data-connectorsPublic\\n\\n\\n\\n\\n\\n\\nLLM-ready
data connectors\\n\\n\\n\\n\\nTypeScript[95](https://github.com/firecrawl/data-connectors/stargazers)
[23](https://github.com/firecrawl/data-connectors/forks)\\n\\n6. [mendable-py](https://github.com/firecrawl/mendable-py)
mendable-pyPublic\\n\\n\\n\\n\\n\\n\\nBuild Production Ready LLM Chat Apps
in Minutes\\n\\n\\n\\n\\nPython[33](https://github.com/firecrawl/mendable-py/stargazers)
[7](https://github.com/firecrawl/mendable-py/forks)\\n\\n\\n### Repositories\\n\\nLoading\\n\\nType\\n\\nAllPublicSourcesForksArchivedMirrorsTemplates\\n\\nLanguage\\n\\nAllCSSGoJavaJavaScriptJupyter
NotebookMDXPythonRustTypeScript\\n\\nSort\\n\\nLast updatedNameStars\\n\\nShowing
10 of 61 repositories\\n\\n- [firecrawl](https://github.com/firecrawl/firecrawl)\\nPublic\\n\\n\\n\\n\U0001F525
The Web Data API for AI - Turn entire websites into LLM-ready markdown or
structured data\\n\\n\\n\\n\\n\\n\\nfirecrawl/firecrawl\u2019s past year of
commit activity\\n\\n\\n\\nTypeScript[64,949](https://github.com/firecrawl/firecrawl/stargazers)AGPL-3.0\\n[5,132](https://github.com/firecrawl/firecrawl/forks)
[27](https://github.com/firecrawl/firecrawl/issues) [(2 issues need help)](https://github.com/firecrawl/firecrawl/issues?q=label%3A%22good+first+issue%22+is%3Aissue+is%3Aopen)
[85](https://github.com/firecrawl/firecrawl/pulls)\\nUpdated 2 hours agoOct
27, 2025\\n\\n- [firecrawl-docs](https://github.com/firecrawl/firecrawl-docs)\\nPublic\\n\\n\\n\\nDocumentation
for Firecrawl.\\n\\n\\n\\n\\n\\n\\nfirecrawl/firecrawl-docs\u2019s past year
of commit activity\\n\\n\\n\\nMDX[17](https://github.com/firecrawl/firecrawl-docs/stargazers)
[35](https://github.com/firecrawl/firecrawl-docs/forks) [10](https://github.com/firecrawl/firecrawl-docs/issues)
[5](https://github.com/firecrawl/firecrawl-docs/pulls)\\nUpdated 20 hours
agoOct 26, 2025\\n\\n- [open-agent-builder](https://github.com/firecrawl/open-agent-builder)\\nPublic\\n\\n\\n\\n\U0001F525
Visual workflow builder for AI agents powered by Firecrawl - drag-and-drop
web scraping pipelines with real-time execution\\n\\n\\n\\n\\n\\n\\nfirecrawl/open-agent-builder\u2019s
past year of commit activity\\n\\n\\n\\nTypeScript[1,673](https://github.com/firecrawl/open-agent-builder/stargazers)
[274](https://github.com/firecrawl/open-agent-builder/forks) [4](https://github.com/firecrawl/open-agent-builder/issues)
[2](https://github.com/firecrawl/open-agent-builder/pulls)\\nUpdated last
weekOct 20, 2025\\n\\n- [firecrawl-mcp-server](https://github.com/firecrawl/firecrawl-mcp-server)\\nPublic\\n\\n\\n\\n\U0001F525
Official Firecrawl MCP Server - Adds powerful web scraping and search to Cursor,
Claude and any other LLM clients.\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n[**Uh
oh!**](https://github.com/firecrawl/firecrawl-mcp-server/graphs/commit-activity)\\n\\n[There
was an error while loading.](https://github.com/firecrawl/firecrawl-mcp-server/graphs/commit-activity)
[Please reload this page](https://github.com/firecrawl).\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nfirecrawl/firecrawl-mcp-server\u2019s
past year of commit activity\\n\\n\\n\\nJavaScript[4,794](https://github.com/firecrawl/firecrawl-mcp-server/stargazers)MIT\\n[519](https://github.com/firecrawl/firecrawl-mcp-server/forks)
[44](https://github.com/firecrawl/firecrawl-mcp-server/issues) [17](https://github.com/firecrawl/firecrawl-mcp-server/pulls)\\nUpdated
last weekOct 19, 2025\\n\\n- [n8n-nodes-firecrawl](https://github.com/firecrawl/n8n-nodes-firecrawl)\\nPublic\\n\\n\\n\\nn8n
node to interact with Firecrawl\\n\\n\\n\\n\\n\\n\\nfirecrawl/n8n-nodes-firecrawl\u2019s
past year of commit activity\\n\\n\\n\\nTypeScript[21](https://github.com/firecrawl/n8n-nodes-firecrawl/stargazers)MIT\\n[13](https://github.com/firecrawl/n8n-nodes-firecrawl/forks)
[3](https://github.com/firecrawl/n8n-nodes-firecrawl/issues) [0](https://github.com/firecrawl/n8n-nodes-firecrawl/pulls)\\nUpdated
2 weeks agoOct 17, 2025\\n\\n- [.github](https://github.com/firecrawl/.github)\\nPublic\\n\\n\\n\\n\\nfirecrawl/.github\u2019s
past year of commit activity\\n\\n\\n\\n0\\n[1](https://github.com/firecrawl/.github/forks)
[0](https://github.com/firecrawl/.github/issues) [0](https://github.com/firecrawl/.github/pulls)\\nUpdated
2 weeks agoOct 12, 2025\\n\\n- [fire-enrich](https://github.com/firecrawl/fire-enrich)\\nPublic\\n\\n\\n\\n\U0001F525
AI-powered data enrichment tool that transforms emails into rich datasets
with company profiles, funding data, tech stacks, and more using Firecrawl
and multi-agent AI\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n[**Uh oh!**](https://github.com/firecrawl/fire-enrich/graphs/commit-activity)\\n\\n[There
was an error while loading.](https://github.com/firecrawl/fire-enrich/graphs/commit-activity)
[Please reload this page](https://github.com/firecrawl).\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\n\\nfirecrawl/fire-enrich\u2019s
past year of commit activity\\n\\n\\n\\nTypeScript[953](https://github.com/firecrawl/fire-enrich/stargazers)MIT\\n[239](https://github.com/firecrawl/fire-enrich/forks)
[12](https://github.com/firecrawl/fire-enrich/issues) [3](https://github.com/firecrawl/fire-enrich/pulls)\\nUpdated
3 weeks agoOct 8, 2025\\n\\n- [firecrawl-java-sdk](https://github.com/firecrawl/firecrawl-java-sdk)\\nPublic\\n\\n\\n\\n\\nfirecrawl/firecrawl-java-sdk\u2019s
past year of commit activity\\n\\n\\n\\nJava[11](https://github.com/firecrawl/firecrawl-java-sdk/stargazers)MIT\\n[4](https://github.com/firecrawl/firecrawl-java-sdk/forks)
[0](https://github.com/firecrawl/firecrawl-java-sdk/issues) [0](https://github.com/firecrawl/firecrawl-java-sdk/pulls)\\nUpdated
last monthSep 28, 2025\\n\\n- [open-lovable](https://github.com/firecrawl/open-lovable)\\nPublic\\n\\n\\n\\n\U0001F525
Clone and recreate any website as a modern React app in seconds\\n\\n\\n\\n\\n\\n\\nfirecrawl/open-lovable\u2019s
past year of commit activity\\n\\n\\n\\nTypeScript[21,320](https://github.com/firecrawl/open-lovable/stargazers)MIT\\n[3,986](https://github.com/firecrawl/open-lovable/forks)
[70](https://github.com/firecrawl/open-lovable/issues) [33](https://github.com/firecrawl/open-lovable/pulls)\\nUpdated
last monthSep 27, 2025\\n\\n- [mineru-api](https://github.com/firecrawl/mineru-api)\\nPublic\\n\\n\\n\\n\\nfirecrawl/mineru-api\u2019s
past year of commit activity\\n\\n\\n\\nPython[12](https://github.com/firecrawl/mineru-api/stargazers)AGPL-3.0\\n[2](https://github.com/firecrawl/mineru-api/forks)
[1](https://github.com/firecrawl/mineru-api/issues) [1](https://github.com/firecrawl/mineru-api/pulls)\\nUpdated
on Sep 26Sep 26, 2025\\n\\n\\n[View all repositories](https://github.com/orgs/firecrawl/repositories?type=all)\\n\\n[**People**](https://github.com/orgs/firecrawl/people)\\n\\n[![@alexnucci](https://avatars.githubusercontent.com/u/1919849?s=70&v=4)](https://github.com/alexnucci)[![@micahstairs](https://avatars.githubusercontent.com/u/7231485?s=70&v=4)](https://github.com/micahstairs)[![@nickscamara](https://avatars.githubusercontent.com/u/20311743?s=70&v=4)](https://github.com/nickscamara)[![@mogery](https://avatars.githubusercontent.com/u/66118807?s=70&v=4)](https://github.com/mogery)[![@developersdigest](https://avatars.githubusercontent.com/u/124798203?s=70&v=4)](https://github.com/developersdigest)\\n\\n####
Top languages\\n\\n[TypeScript](https://github.com/orgs/firecrawl/repositories?language=typescript&type=all)
[Python](https://github.com/orgs/firecrawl/repositories?language=python&type=all)
[JavaScript](https://github.com/orgs/firecrawl/repositories?language=javascript&type=all)
[Go](https://github.com/orgs/firecrawl/repositories?language=go&type=all)
[MDX](https://github.com/orgs/firecrawl/repositories?language=mdx&type=all)\\n\\n####
Most used topics\\n\\n[ai](https://github.com/search?q=topic%3Aai+org%3Afirecrawl+fork%3Atrue&type=repositories
\\\"Topic: ai\\\") [firecrawl](https://github.com/search?q=topic%3Afirecrawl+org%3Afirecrawl+fork%3Atrue&type=repositories
\\\"Topic: firecrawl\\\") [llm](https://github.com/search?q=topic%3Allm+org%3Afirecrawl+fork%3Atrue&type=repositories
\\\"Topic: llm\\\") [web-crawler](https://github.com/search?q=topic%3Aweb-crawler+org%3Afirecrawl+fork%3Atrue&type=repositories
\\\"Topic: web-crawler\\\") [web-scraping](https://github.com/search?q=topic%3Aweb-scraping+org%3Afirecrawl+fork%3Atrue&type=repositories
\\\"Topic: web-scraping\\\")\\n\\nYou can\u2019t perform that action at this
time.\",\"metadata\":{\"analytics-location\":\"/<org-login>\",\"apple-itunes-app\":\"app-id=1477376905,
app-argument=https://github.com/firecrawl\",\"twitter:card\":\"summary_large_image\",\"google-site-verification\":\"Apib7-x98H0j5cPqHWwSMm6dNU4GmODRoqxLiDzdx9I\",\"description\":\"Web
data API for AI. Firecrawl has 61 repositories available. Follow their code
on GitHub.\",\"og:image\":\"https://avatars.githubusercontent.com/u/135057108?s=280&v=4\",\"og:type\":\"profile\",\"visitor-payload\":\"eyJyZWZlcnJlciI6IiIsInJlcXVlc3RfaWQiOiI5REFEOjIzM0ExMzo4RDgzNEI6QzQ5OUNCOjY4RkYzODlGIiwidmlzaXRvcl9pZCI6IjQwMDQ0MTI5MTkxMDA5NDY1OTEiLCJyZWdpb25fZWRnZSI6ImlhZCIsInJlZ2lvbl9yZW5kZXIiOiJpYWQifQ==\",\"github-keyboard-shortcuts\":\"copilot\",\"user-login\":\"\",\"viewport\":\"width=device-width\",\"og:description\":\"Web
data API for AI. Firecrawl has 61 repositories available. Follow their code
on GitHub.\",\"turbo-cache-control\":\"no-preview\",\"fetch-nonce\":\"v2:35a2e032-0081-3f6b-595e-967e32025c6f\",\"og:url\":\"https://github.com/firecrawl\",\"title\":\"Firecrawl
\xB7 GitHub\",\"route-pattern\":\"/:user_id(.:format)\",\"route-action\":\"show\",\"octolytics-url\":\"https://collector.github.com/github/collect\",\"og:site_name\":\"GitHub\",\"twitter:title\":\"Firecrawl\",\"request-id\":\"9DAD:233A13:8D834B:C499CB:68FF389F\",\"ogSiteName\":\"GitHub\",\"fb:app_id\":\"1401488693436528\",\"language\":\"en\",\"twitter:image\":\"https://avatars.githubusercontent.com/u/135057108?s=280&v=4\",\"ogImage\":\"https://avatars.githubusercontent.com/u/135057108?s=280&v=4\",\"release\":\"c44b7f7aa5c70f3296484971978c9f4b1b473352\",\"theme-color\":\"#1e2327\",\"color-scheme\":\"light
dark\",\"html-safe-nonce\":\"2d932295da6aa360d861f16279839ab109dc4e977a51bc99204477969d7d12c6\",\"ogTitle\":\"Firecrawl\",\"hovercard-subject-tag\":\"organization:135057108\",\"twitter:description\":\"Web
data API for AI. Firecrawl has 61 repositories available. Follow their code
on GitHub.\",\"hostname\":\"github.com\",\"ogUrl\":\"https://github.com/firecrawl\",\"ogDescription\":\"Web
data API for AI. Firecrawl has 61 repositories available. Follow their code
on GitHub.\",\"route-controller\":\"profiles\",\"favicon\":\"https://github.githubassets.com/favicons/favicon.svg\",\"visitor-hmac\":\"43cf3bbb9c57a3bcf0b1857fbcabcc78c274e7082e0d28c76cd6d4651bc2a920\",\"twitter:site\":\"@github\",\"ui-target\":\"full\",\"og:title\":\"Firecrawl\",\"expected-hostname\":\"github.com\",\"og:image:alt\":\"Web
data API for AI. Firecrawl has 61 repositories available. Follow their code
on GitHub.\",\"profile:username\":\"firecrawl\",\"current-catalog-service-hash\":\"4a1c50a83cf6cc4b55b6b9c53e553e3f847c876b87fb333f71f5d05db8f1a7db\",\"turbo-body-classes\":\"logged-out
env-production page-responsive\",\"browser-stats-url\":\"https://api.github.com/_private/browser/stats\",\"browser-errors-url\":\"https://api.github.com/_private/browser/errors\",\"scrapeId\":\"65cbc300-be11-4a1a-9d20-c114fb8473a7\",\"sourceURL\":\"https://github.com/firecrawl\",\"url\":\"https://github.com/firecrawl\",\"statusCode\":200,\"contentType\":\"text/html;
charset=utf-8\",\"proxyUsed\":\"basic\",\"cacheState\":\"hit\",\"cachedAt\":\"2025-10-27T09:17:20.756Z\"}},{\"url\":\"https://www.linkedin.com/company/firecrawl\",\"title\":\"Firecrawl
| LinkedIn\",\"description\":\"Our Dify integration now uses Firecrawl /v2
endpoints Scraping is 10x faster thanks to intelligent caching, plus we've
added semantic ...\",\"position\":5}]},\"creditsUsed\":3}"
headers:
Access-Control-Allow-Origin:
- '*'
Alt-Svc:
- h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
Content-Length:
- '93428'
Content-Type:
- application/json; charset=utf-8
Date:
- Wed, 29 Oct 2025 14:37:39 GMT
ETag:
- W/"16cf4-kHwVbMu4CCVG2UIt6p1g/gz5M4M"
Via:
- 1.1 google
X-Powered-By:
- Express
X-Response-Time:
- 13172.495ms
status:
code: 200
message: OK
version: 1

View File

@@ -1,18 +0,0 @@
import pytest
from crewai_tools.tools.firecrawl_crawl_website_tool.firecrawl_crawl_website_tool import (
FirecrawlCrawlWebsiteTool,
)
@pytest.mark.vcr(filter_headers=["authorization"])
def test_firecrawl_crawl_tool_integration():
tool = FirecrawlCrawlWebsiteTool(config={
"limit": 2,
"max_discovery_depth": 1,
"scrape_options": {"formats": ["markdown"]}
})
result = tool.run(url="https://firecrawl.dev")
assert result is not None
assert hasattr(result, 'status')
assert result.status in ["completed", "scraping"]

View File

@@ -1,15 +0,0 @@
import pytest
from crewai_tools.tools.firecrawl_scrape_website_tool.firecrawl_scrape_website_tool import (
FirecrawlScrapeWebsiteTool,
)
@pytest.mark.vcr(filter_headers=["authorization"])
def test_firecrawl_scrape_tool_integration():
tool = FirecrawlScrapeWebsiteTool()
result = tool.run(url="https://firecrawl.dev")
assert result is not None
assert hasattr(result, 'markdown')
assert len(result.markdown) > 0
assert "Firecrawl" in result.markdown or "firecrawl" in result.markdown.lower()

View File

@@ -1,12 +0,0 @@
import pytest
from crewai_tools.tools.firecrawl_search_tool.firecrawl_search_tool import FirecrawlSearchTool
@pytest.mark.vcr(filter_headers=["authorization"])
def test_firecrawl_search_tool_integration():
tool = FirecrawlSearchTool()
result = tool.run(query="firecrawl")
assert result is not None
assert hasattr(result, 'web') or hasattr(result, 'news') or hasattr(result, 'images')

View File

@@ -23,6 +23,7 @@ dependencies = [
"chromadb~=1.1.0",
"tokenizers>=0.20.3",
"openpyxl>=3.1.5",
"pyvis>=0.3.2",
# Authentication and Security
"python-dotenv>=1.1.1",
"pyjwt>=2.9.0",
@@ -48,7 +49,7 @@ Repository = "https://github.com/crewAIInc/crewAI"
[project.optional-dependencies]
tools = [
"crewai-tools==1.4.1",
"crewai-tools==1.2.1",
]
embeddings = [
"tiktoken~=0.8.0"
@@ -93,11 +94,10 @@ azure-ai-inference = [
anthropic = [
"anthropic>=0.69.0",
]
a2a = [
"a2a-sdk~=0.3.10",
"httpx-auth>=0.23.1",
"httpx-sse>=0.4.0",
]
# a2a = [
# "a2a-sdk~=0.3.9",
# "httpx-sse>=0.4.0",
# ]
[project.scripts]

View File

@@ -3,7 +3,7 @@ from typing import Any
import urllib.request
import warnings
from crewai.agent.core import Agent
from crewai.agent import Agent
from crewai.crew import Crew
from crewai.crews.crew_output import CrewOutput
from crewai.flow.flow import Flow
@@ -40,7 +40,7 @@ def _suppress_pydantic_deprecation_warnings() -> None:
_suppress_pydantic_deprecation_warnings()
__version__ = "1.4.1"
__version__ = "1.2.1"
_telemetry_submitted = False

View File

@@ -1,6 +0,0 @@
"""Agent-to-Agent (A2A) protocol communication module for CrewAI."""
from crewai.a2a.config import A2AConfig
__all__ = ["A2AConfig"]

View File

@@ -1,20 +0,0 @@
"""A2A authentication schemas."""
from crewai.a2a.auth.schemas import (
APIKeyAuth,
BearerTokenAuth,
HTTPBasicAuth,
HTTPDigestAuth,
OAuth2AuthorizationCode,
OAuth2ClientCredentials,
)
__all__ = [
"APIKeyAuth",
"BearerTokenAuth",
"HTTPBasicAuth",
"HTTPDigestAuth",
"OAuth2AuthorizationCode",
"OAuth2ClientCredentials",
]

View File

@@ -1,392 +0,0 @@
"""Authentication schemes for A2A protocol agents.
Supported authentication methods:
- Bearer tokens
- OAuth2 (Client Credentials, Authorization Code)
- API Keys (header, query, cookie)
- HTTP Basic authentication
- HTTP Digest authentication
"""
from __future__ import annotations
from abc import ABC, abstractmethod
import base64
from collections.abc import Awaitable, Callable, MutableMapping
import time
from typing import Literal
import urllib.parse
import httpx
from httpx import DigestAuth
from pydantic import BaseModel, Field, PrivateAttr
class AuthScheme(ABC, BaseModel):
"""Base class for authentication schemes."""
@abstractmethod
async def apply_auth(
self, client: httpx.AsyncClient, headers: MutableMapping[str, str]
) -> MutableMapping[str, str]:
"""Apply authentication to request headers.
Args:
client: HTTP client for making auth requests.
headers: Current request headers.
Returns:
Updated headers with authentication applied.
"""
...
class BearerTokenAuth(AuthScheme):
"""Bearer token authentication (Authorization: Bearer <token>).
Attributes:
token: Bearer token for authentication.
"""
token: str = Field(description="Bearer token")
async def apply_auth(
self, client: httpx.AsyncClient, headers: MutableMapping[str, str]
) -> MutableMapping[str, str]:
"""Apply Bearer token to Authorization header.
Args:
client: HTTP client for making auth requests.
headers: Current request headers.
Returns:
Updated headers with Bearer token in Authorization header.
"""
headers["Authorization"] = f"Bearer {self.token}"
return headers
class HTTPBasicAuth(AuthScheme):
"""HTTP Basic authentication.
Attributes:
username: Username for Basic authentication.
password: Password for Basic authentication.
"""
username: str = Field(description="Username")
password: str = Field(description="Password")
async def apply_auth(
self, client: httpx.AsyncClient, headers: MutableMapping[str, str]
) -> MutableMapping[str, str]:
"""Apply HTTP Basic authentication.
Args:
client: HTTP client for making auth requests.
headers: Current request headers.
Returns:
Updated headers with Basic auth in Authorization header.
"""
credentials = f"{self.username}:{self.password}"
encoded = base64.b64encode(credentials.encode()).decode()
headers["Authorization"] = f"Basic {encoded}"
return headers
class HTTPDigestAuth(AuthScheme):
"""HTTP Digest authentication.
Note: Uses httpx-auth library for digest implementation.
Attributes:
username: Username for Digest authentication.
password: Password for Digest authentication.
"""
username: str = Field(description="Username")
password: str = Field(description="Password")
async def apply_auth(
self, client: httpx.AsyncClient, headers: MutableMapping[str, str]
) -> MutableMapping[str, str]:
"""Digest auth is handled by httpx auth flow, not headers.
Args:
client: HTTP client for making auth requests.
headers: Current request headers.
Returns:
Unchanged headers (Digest auth handled by httpx auth flow).
"""
return headers
def configure_client(self, client: httpx.AsyncClient) -> None:
"""Configure client with Digest auth.
Args:
client: HTTP client to configure with Digest authentication.
"""
client.auth = DigestAuth(self.username, self.password)
class APIKeyAuth(AuthScheme):
"""API Key authentication (header, query, or cookie).
Attributes:
api_key: API key value for authentication.
location: Where to send the API key (header, query, or cookie).
name: Parameter name for the API key (default: X-API-Key).
"""
api_key: str = Field(description="API key value")
location: Literal["header", "query", "cookie"] = Field(
default="header", description="Where to send the API key"
)
name: str = Field(default="X-API-Key", description="Parameter name for the API key")
async def apply_auth(
self, client: httpx.AsyncClient, headers: MutableMapping[str, str]
) -> MutableMapping[str, str]:
"""Apply API key authentication.
Args:
client: HTTP client for making auth requests.
headers: Current request headers.
Returns:
Updated headers with API key (for header/cookie locations).
"""
if self.location == "header":
headers[self.name] = self.api_key
elif self.location == "cookie":
headers["Cookie"] = f"{self.name}={self.api_key}"
return headers
def configure_client(self, client: httpx.AsyncClient) -> None:
"""Configure client for query param API keys.
Args:
client: HTTP client to configure with query param API key hook.
"""
if self.location == "query":
async def _add_api_key_param(request: httpx.Request) -> None:
url = httpx.URL(request.url)
request.url = url.copy_add_param(self.name, self.api_key)
client.event_hooks["request"].append(_add_api_key_param)
class OAuth2ClientCredentials(AuthScheme):
"""OAuth2 Client Credentials flow authentication.
Attributes:
token_url: OAuth2 token endpoint URL.
client_id: OAuth2 client identifier.
client_secret: OAuth2 client secret.
scopes: List of required OAuth2 scopes.
"""
token_url: str = Field(description="OAuth2 token endpoint")
client_id: str = Field(description="OAuth2 client ID")
client_secret: str = Field(description="OAuth2 client secret")
scopes: list[str] = Field(
default_factory=list, description="Required OAuth2 scopes"
)
_access_token: str | None = PrivateAttr(default=None)
_token_expires_at: float | None = PrivateAttr(default=None)
async def apply_auth(
self, client: httpx.AsyncClient, headers: MutableMapping[str, str]
) -> MutableMapping[str, str]:
"""Apply OAuth2 access token to Authorization header.
Args:
client: HTTP client for making token requests.
headers: Current request headers.
Returns:
Updated headers with OAuth2 access token in Authorization header.
"""
if (
self._access_token is None
or self._token_expires_at is None
or time.time() >= self._token_expires_at
):
await self._fetch_token(client)
if self._access_token:
headers["Authorization"] = f"Bearer {self._access_token}"
return headers
async def _fetch_token(self, client: httpx.AsyncClient) -> None:
"""Fetch OAuth2 access token using client credentials flow.
Args:
client: HTTP client for making token request.
Raises:
httpx.HTTPStatusError: If token request fails.
"""
data = {
"grant_type": "client_credentials",
"client_id": self.client_id,
"client_secret": self.client_secret,
}
if self.scopes:
data["scope"] = " ".join(self.scopes)
response = await client.post(self.token_url, data=data)
response.raise_for_status()
token_data = response.json()
self._access_token = token_data["access_token"]
expires_in = token_data.get("expires_in", 3600)
self._token_expires_at = time.time() + expires_in - 60
class OAuth2AuthorizationCode(AuthScheme):
"""OAuth2 Authorization Code flow authentication.
Note: Requires interactive authorization.
Attributes:
authorization_url: OAuth2 authorization endpoint URL.
token_url: OAuth2 token endpoint URL.
client_id: OAuth2 client identifier.
client_secret: OAuth2 client secret.
redirect_uri: OAuth2 redirect URI for callback.
scopes: List of required OAuth2 scopes.
"""
authorization_url: str = Field(description="OAuth2 authorization endpoint")
token_url: str = Field(description="OAuth2 token endpoint")
client_id: str = Field(description="OAuth2 client ID")
client_secret: str = Field(description="OAuth2 client secret")
redirect_uri: str = Field(description="OAuth2 redirect URI")
scopes: list[str] = Field(
default_factory=list, description="Required OAuth2 scopes"
)
_access_token: str | None = PrivateAttr(default=None)
_refresh_token: str | None = PrivateAttr(default=None)
_token_expires_at: float | None = PrivateAttr(default=None)
_authorization_callback: Callable[[str], Awaitable[str]] | None = PrivateAttr(
default=None
)
def set_authorization_callback(
self, callback: Callable[[str], Awaitable[str]] | None
) -> None:
"""Set callback to handle authorization URL.
Args:
callback: Async function that receives authorization URL and returns auth code.
"""
self._authorization_callback = callback
async def apply_auth(
self, client: httpx.AsyncClient, headers: MutableMapping[str, str]
) -> MutableMapping[str, str]:
"""Apply OAuth2 access token to Authorization header.
Args:
client: HTTP client for making token requests.
headers: Current request headers.
Returns:
Updated headers with OAuth2 access token in Authorization header.
Raises:
ValueError: If authorization callback is not set.
"""
if self._access_token is None:
if self._authorization_callback is None:
msg = "Authorization callback not set. Use set_authorization_callback()"
raise ValueError(msg)
await self._fetch_initial_token(client)
elif self._token_expires_at and time.time() >= self._token_expires_at:
await self._refresh_access_token(client)
if self._access_token:
headers["Authorization"] = f"Bearer {self._access_token}"
return headers
async def _fetch_initial_token(self, client: httpx.AsyncClient) -> None:
"""Fetch initial access token using authorization code flow.
Args:
client: HTTP client for making token request.
Raises:
ValueError: If authorization callback is not set.
httpx.HTTPStatusError: If token request fails.
"""
params = {
"response_type": "code",
"client_id": self.client_id,
"redirect_uri": self.redirect_uri,
"scope": " ".join(self.scopes),
}
auth_url = f"{self.authorization_url}?{urllib.parse.urlencode(params)}"
if self._authorization_callback is None:
msg = "Authorization callback not set"
raise ValueError(msg)
auth_code = await self._authorization_callback(auth_url)
data = {
"grant_type": "authorization_code",
"code": auth_code,
"client_id": self.client_id,
"client_secret": self.client_secret,
"redirect_uri": self.redirect_uri,
}
response = await client.post(self.token_url, data=data)
response.raise_for_status()
token_data = response.json()
self._access_token = token_data["access_token"]
self._refresh_token = token_data.get("refresh_token")
expires_in = token_data.get("expires_in", 3600)
self._token_expires_at = time.time() + expires_in - 60
async def _refresh_access_token(self, client: httpx.AsyncClient) -> None:
"""Refresh the access token using refresh token.
Args:
client: HTTP client for making token request.
Raises:
httpx.HTTPStatusError: If token refresh request fails.
"""
if not self._refresh_token:
await self._fetch_initial_token(client)
return
data = {
"grant_type": "refresh_token",
"refresh_token": self._refresh_token,
"client_id": self.client_id,
"client_secret": self.client_secret,
}
response = await client.post(self.token_url, data=data)
response.raise_for_status()
token_data = response.json()
self._access_token = token_data["access_token"]
if "refresh_token" in token_data:
self._refresh_token = token_data["refresh_token"]
expires_in = token_data.get("expires_in", 3600)
self._token_expires_at = time.time() + expires_in - 60

View File

@@ -1,236 +0,0 @@
"""Authentication utilities for A2A protocol agent communication.
Provides validation and retry logic for various authentication schemes including
OAuth2, API keys, and HTTP authentication methods.
"""
import asyncio
from collections.abc import Awaitable, Callable, MutableMapping
import re
from typing import Final
from a2a.client.errors import A2AClientHTTPError
from a2a.types import (
APIKeySecurityScheme,
AgentCard,
HTTPAuthSecurityScheme,
OAuth2SecurityScheme,
)
from httpx import AsyncClient, Response
from crewai.a2a.auth.schemas import (
APIKeyAuth,
AuthScheme,
BearerTokenAuth,
HTTPBasicAuth,
HTTPDigestAuth,
OAuth2AuthorizationCode,
OAuth2ClientCredentials,
)
_auth_store: dict[int, AuthScheme | None] = {}
_SCHEME_PATTERN: Final[re.Pattern[str]] = re.compile(r"(\w+)\s+(.+?)(?=,\s*\w+\s+|$)")
_PARAM_PATTERN: Final[re.Pattern[str]] = re.compile(r'(\w+)=(?:"([^"]*)"|([^\s,]+))')
_SCHEME_AUTH_MAPPING: Final[dict[type, tuple[type[AuthScheme], ...]]] = {
OAuth2SecurityScheme: (
OAuth2ClientCredentials,
OAuth2AuthorizationCode,
BearerTokenAuth,
),
APIKeySecurityScheme: (APIKeyAuth,),
}
_HTTP_SCHEME_MAPPING: Final[dict[str, type[AuthScheme]]] = {
"basic": HTTPBasicAuth,
"digest": HTTPDigestAuth,
"bearer": BearerTokenAuth,
}
def _raise_auth_mismatch(
expected_classes: type[AuthScheme] | tuple[type[AuthScheme], ...],
provided_auth: AuthScheme,
) -> None:
"""Raise authentication mismatch error.
Args:
expected_classes: Expected authentication class or tuple of classes.
provided_auth: Actually provided authentication instance.
Raises:
A2AClientHTTPError: Always raises with 401 status code.
"""
if isinstance(expected_classes, tuple):
if len(expected_classes) == 1:
required = expected_classes[0].__name__
else:
names = [cls.__name__ for cls in expected_classes]
required = f"one of ({', '.join(names)})"
else:
required = expected_classes.__name__
msg = (
f"AgentCard requires {required} authentication, "
f"but {type(provided_auth).__name__} was provided"
)
raise A2AClientHTTPError(401, msg)
def parse_www_authenticate(header_value: str) -> dict[str, dict[str, str]]:
"""Parse WWW-Authenticate header into auth challenges.
Args:
header_value: The WWW-Authenticate header value.
Returns:
Dictionary mapping auth scheme to its parameters.
Example: {"Bearer": {"realm": "api", "scope": "read write"}}
"""
if not header_value:
return {}
challenges: dict[str, dict[str, str]] = {}
for match in _SCHEME_PATTERN.finditer(header_value):
scheme = match.group(1)
params_str = match.group(2)
params: dict[str, str] = {}
for param_match in _PARAM_PATTERN.finditer(params_str):
key = param_match.group(1)
value = param_match.group(2) or param_match.group(3)
params[key] = value
challenges[scheme] = params
return challenges
def validate_auth_against_agent_card(
agent_card: AgentCard, auth: AuthScheme | None
) -> None:
"""Validate that provided auth matches AgentCard security requirements.
Args:
agent_card: The A2A AgentCard containing security requirements.
auth: User-provided authentication scheme (or None).
Raises:
A2AClientHTTPError: If auth doesn't match AgentCard requirements (status_code=401).
"""
if not agent_card.security or not agent_card.security_schemes:
return
if not auth:
msg = "AgentCard requires authentication but no auth scheme provided"
raise A2AClientHTTPError(401, msg)
first_security_req = agent_card.security[0] if agent_card.security else {}
for scheme_name in first_security_req.keys():
security_scheme_wrapper = agent_card.security_schemes.get(scheme_name)
if not security_scheme_wrapper:
continue
scheme = security_scheme_wrapper.root
if allowed_classes := _SCHEME_AUTH_MAPPING.get(type(scheme)):
if not isinstance(auth, allowed_classes):
_raise_auth_mismatch(allowed_classes, auth)
return
if isinstance(scheme, HTTPAuthSecurityScheme):
if required_class := _HTTP_SCHEME_MAPPING.get(scheme.scheme.lower()):
if not isinstance(auth, required_class):
_raise_auth_mismatch(required_class, auth)
return
msg = "Could not validate auth against AgentCard security requirements"
raise A2AClientHTTPError(401, msg)
async def retry_on_401(
request_func: Callable[[], Awaitable[Response]],
auth_scheme: AuthScheme | None,
client: AsyncClient,
headers: MutableMapping[str, str],
max_retries: int = 3,
) -> Response:
"""Retry a request on 401 authentication error.
Handles 401 errors by:
1. Parsing WWW-Authenticate header
2. Re-acquiring credentials
3. Retrying the request
Args:
request_func: Async function that makes the HTTP request.
auth_scheme: Authentication scheme to refresh credentials with.
client: HTTP client for making requests.
headers: Request headers to update with new auth.
max_retries: Maximum number of retry attempts (default: 3).
Returns:
HTTP response from the request.
Raises:
httpx.HTTPStatusError: If retries are exhausted or auth scheme is None.
"""
last_response: Response | None = None
last_challenges: dict[str, dict[str, str]] = {}
for attempt in range(max_retries):
response = await request_func()
if response.status_code != 401:
return response
last_response = response
if auth_scheme is None:
response.raise_for_status()
return response
www_authenticate = response.headers.get("WWW-Authenticate", "")
challenges = parse_www_authenticate(www_authenticate)
last_challenges = challenges
if attempt >= max_retries - 1:
break
backoff_time = 2**attempt
await asyncio.sleep(backoff_time)
await auth_scheme.apply_auth(client, headers)
if last_response:
last_response.raise_for_status()
return last_response
msg = "retry_on_401 failed without making any requests"
if last_challenges:
challenge_info = ", ".join(
f"{scheme} (realm={params.get('realm', 'N/A')})"
for scheme, params in last_challenges.items()
)
msg = f"{msg}. Server challenges: {challenge_info}"
raise RuntimeError(msg)
def configure_auth_client(
auth: HTTPDigestAuth | APIKeyAuth, client: AsyncClient
) -> None:
"""Configure HTTP client with auth-specific settings.
Only HTTPDigestAuth and APIKeyAuth need client configuration.
Args:
auth: Authentication scheme that requires client configuration.
client: HTTP client to configure.
"""
auth.configure_client(client)

View File

@@ -1,59 +0,0 @@
"""A2A configuration types.
This module is separate from experimental.a2a to avoid circular imports.
"""
from __future__ import annotations
from typing import Annotated
from pydantic import (
BaseModel,
BeforeValidator,
Field,
HttpUrl,
TypeAdapter,
)
from crewai.a2a.auth.schemas import AuthScheme
http_url_adapter = TypeAdapter(HttpUrl)
Url = Annotated[
str,
BeforeValidator(
lambda value: str(http_url_adapter.validate_python(value, strict=True))
),
]
class A2AConfig(BaseModel):
"""Configuration for A2A protocol integration.
Attributes:
endpoint: A2A agent endpoint URL.
auth: Authentication scheme (Bearer, OAuth2, API Key, HTTP Basic/Digest).
timeout: Request timeout in seconds (default: 120).
max_turns: Maximum conversation turns with A2A agent (default: 10).
response_model: Optional Pydantic model for structured A2A agent responses.
fail_fast: If True, raise error when agent unreachable; if False, skip and continue (default: True).
"""
endpoint: Url = Field(description="A2A agent endpoint URL")
auth: AuthScheme | None = Field(
default=None,
description="Authentication scheme (Bearer, OAuth2, API Key, HTTP Basic/Digest)",
)
timeout: int = Field(default=120, description="Request timeout in seconds")
max_turns: int = Field(
default=10, description="Maximum conversation turns with A2A agent"
)
response_model: type[BaseModel] | None = Field(
default=None,
description="Optional Pydantic model for structured A2A agent responses. When specified, the A2A agent is expected to return JSON matching this schema.",
)
fail_fast: bool = Field(
default=True,
description="If True, raise an error immediately when the A2A agent is unreachable. If False, skip the A2A agent and continue execution.",
)

View File

@@ -1,29 +0,0 @@
"""String templates for A2A (Agent-to-Agent) protocol messaging and status."""
from string import Template
from typing import Final
AVAILABLE_AGENTS_TEMPLATE: Final[Template] = Template(
"\n<AVAILABLE_A2A_AGENTS>\n $available_a2a_agents\n</AVAILABLE_A2A_AGENTS>\n"
)
PREVIOUS_A2A_CONVERSATION_TEMPLATE: Final[Template] = Template(
"\n<PREVIOUS_A2A_CONVERSATION>\n"
" $previous_a2a_conversation"
"\n</PREVIOUS_A2A_CONVERSATION>\n"
)
CONVERSATION_TURN_INFO_TEMPLATE: Final[Template] = Template(
"\n<CONVERSATION_PROGRESS>\n"
' turn="$turn_count"\n'
' max_turns="$max_turns"\n'
" $warning"
"\n</CONVERSATION_PROGRESS>\n"
)
UNAVAILABLE_AGENTS_NOTICE_TEMPLATE: Final[Template] = Template(
"\n<A2A_AGENTS_STATUS>\n"
" NOTE: A2A agents were configured but are currently unavailable.\n"
" You cannot delegate to remote agents for this task.\n\n"
" Unavailable Agents:\n"
" $unavailable_agents"
"\n</A2A_AGENTS_STATUS>\n"
)

View File

@@ -1,38 +0,0 @@
"""Type definitions for A2A protocol message parts."""
from typing import Any, Literal, Protocol, TypedDict, runtime_checkable
from typing_extensions import NotRequired
@runtime_checkable
class AgentResponseProtocol(Protocol):
"""Protocol for the dynamically created AgentResponse model."""
a2a_ids: tuple[str, ...]
message: str
is_a2a: bool
class PartsMetadataDict(TypedDict, total=False):
"""Metadata for A2A message parts.
Attributes:
mimeType: MIME type for the part content.
schema: JSON schema for the part content.
"""
mimeType: Literal["application/json"]
schema: dict[str, Any]
class PartsDict(TypedDict):
"""A2A message part containing text and optional metadata.
Attributes:
text: The text content of the message part.
metadata: Optional metadata describing the part content.
"""
text: str
metadata: NotRequired[PartsMetadataDict]

View File

@@ -1,851 +0,0 @@
"""Utility functions for A2A (Agent-to-Agent) protocol delegation."""
from __future__ import annotations
import asyncio
from collections.abc import AsyncIterator, MutableMapping
from contextlib import asynccontextmanager
from functools import lru_cache
import time
from typing import TYPE_CHECKING, Any
import uuid
from a2a.client import Client, ClientConfig, ClientFactory
from a2a.client.errors import A2AClientHTTPError
from a2a.types import (
AgentCard,
Message,
Part,
Role,
TaskArtifactUpdateEvent,
TaskState,
TaskStatusUpdateEvent,
TextPart,
TransportProtocol,
)
import httpx
from pydantic import BaseModel, Field, create_model
from crewai.a2a.auth.schemas import APIKeyAuth, HTTPDigestAuth
from crewai.a2a.auth.utils import (
_auth_store,
configure_auth_client,
retry_on_401,
validate_auth_against_agent_card,
)
from crewai.a2a.config import A2AConfig
from crewai.a2a.types import PartsDict, PartsMetadataDict
from crewai.events.event_bus import crewai_event_bus
from crewai.events.types.a2a_events import (
A2AConversationStartedEvent,
A2ADelegationCompletedEvent,
A2ADelegationStartedEvent,
A2AMessageSentEvent,
A2AResponseReceivedEvent,
)
from crewai.types.utils import create_literals_from_strings
if TYPE_CHECKING:
from a2a.types import Message, Task as A2ATask
from crewai.a2a.auth.schemas import AuthScheme
@lru_cache()
def _fetch_agent_card_cached(
endpoint: str,
auth_hash: int,
timeout: int,
_ttl_hash: int,
) -> AgentCard:
"""Cached version of fetch_agent_card with auth support.
Args:
endpoint: A2A agent endpoint URL
auth_hash: Hash of the auth object
timeout: Request timeout
_ttl_hash: Time-based hash for cache invalidation (unused in body)
Returns:
Cached AgentCard
"""
auth = _auth_store.get(auth_hash)
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
try:
return loop.run_until_complete(
_fetch_agent_card_async(endpoint=endpoint, auth=auth, timeout=timeout)
)
finally:
loop.close()
def fetch_agent_card(
endpoint: str,
auth: AuthScheme | None = None,
timeout: int = 30,
use_cache: bool = True,
cache_ttl: int = 300,
) -> AgentCard:
"""Fetch AgentCard from an A2A endpoint with optional caching.
Args:
endpoint: A2A agent endpoint URL (AgentCard URL)
auth: Optional AuthScheme for authentication
timeout: Request timeout in seconds
use_cache: Whether to use caching (default True)
cache_ttl: Cache TTL in seconds (default 300 = 5 minutes)
Returns:
AgentCard object with agent capabilities and skills
Raises:
httpx.HTTPStatusError: If the request fails
A2AClientHTTPError: If authentication fails
"""
if use_cache:
auth_hash = hash((type(auth).__name__, id(auth))) if auth else 0
_auth_store[auth_hash] = auth
ttl_hash = int(time.time() // cache_ttl)
return _fetch_agent_card_cached(endpoint, auth_hash, timeout, ttl_hash)
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
try:
return loop.run_until_complete(
_fetch_agent_card_async(endpoint=endpoint, auth=auth, timeout=timeout)
)
finally:
loop.close()
async def _fetch_agent_card_async(
endpoint: str,
auth: AuthScheme | None,
timeout: int,
) -> AgentCard:
"""Async implementation of AgentCard fetching.
Args:
endpoint: A2A agent endpoint URL
auth: Optional AuthScheme for authentication
timeout: Request timeout in seconds
Returns:
AgentCard object
"""
if "/.well-known/agent-card.json" in endpoint:
base_url = endpoint.replace("/.well-known/agent-card.json", "")
agent_card_path = "/.well-known/agent-card.json"
else:
url_parts = endpoint.split("/", 3)
base_url = f"{url_parts[0]}//{url_parts[2]}"
agent_card_path = f"/{url_parts[3]}" if len(url_parts) > 3 else "/"
headers: MutableMapping[str, str] = {}
if auth:
async with httpx.AsyncClient(timeout=timeout) as temp_auth_client:
if isinstance(auth, (HTTPDigestAuth, APIKeyAuth)):
configure_auth_client(auth, temp_auth_client)
headers = await auth.apply_auth(temp_auth_client, {})
async with httpx.AsyncClient(timeout=timeout, headers=headers) as temp_client:
if auth and isinstance(auth, (HTTPDigestAuth, APIKeyAuth)):
configure_auth_client(auth, temp_client)
agent_card_url = f"{base_url}{agent_card_path}"
async def _fetch_agent_card_request() -> httpx.Response:
return await temp_client.get(agent_card_url)
try:
response = await retry_on_401(
request_func=_fetch_agent_card_request,
auth_scheme=auth,
client=temp_client,
headers=temp_client.headers,
max_retries=2,
)
response.raise_for_status()
return AgentCard.model_validate(response.json())
except httpx.HTTPStatusError as e:
if e.response.status_code == 401:
error_details = ["Authentication failed"]
www_auth = e.response.headers.get("WWW-Authenticate")
if www_auth:
error_details.append(f"WWW-Authenticate: {www_auth}")
if not auth:
error_details.append("No auth scheme provided")
msg = " | ".join(error_details)
raise A2AClientHTTPError(401, msg) from e
raise
def execute_a2a_delegation(
endpoint: str,
auth: AuthScheme | None,
timeout: int,
task_description: str,
context: str | None = None,
context_id: str | None = None,
task_id: str | None = None,
reference_task_ids: list[str] | None = None,
metadata: dict[str, Any] | None = None,
extensions: dict[str, Any] | None = None,
conversation_history: list[Message] | None = None,
agent_id: str | None = None,
agent_role: Role | None = None,
agent_branch: Any | None = None,
response_model: type[BaseModel] | None = None,
turn_number: int | None = None,
) -> dict[str, Any]:
"""Execute a task delegation to a remote A2A agent with multi-turn support.
Handles:
- AgentCard discovery
- Authentication setup
- Message creation and sending
- Response parsing
- Multi-turn conversations
Args:
endpoint: A2A agent endpoint URL (AgentCard URL)
auth: Optional AuthScheme for authentication (Bearer, OAuth2, API Key, HTTP Basic/Digest)
timeout: Request timeout in seconds
task_description: The task to delegate
context: Optional context information
context_id: Context ID for correlating messages/tasks
task_id: Specific task identifier
reference_task_ids: List of related task IDs
metadata: Additional metadata (external_id, request_id, etc.)
extensions: Protocol extensions for custom fields
conversation_history: Previous Message objects from conversation
agent_id: Agent identifier for logging
agent_role: Role of the CrewAI agent delegating the task
agent_branch: Optional agent tree branch for logging
response_model: Optional Pydantic model for structured outputs
turn_number: Optional turn number for multi-turn conversations
Returns:
Dictionary with:
- status: "completed", "input_required", "failed", etc.
- result: Result string (if completed)
- error: Error message (if failed)
- history: List of new Message objects from this exchange
Raises:
ImportError: If a2a-sdk is not installed
"""
is_multiturn = bool(conversation_history and len(conversation_history) > 0)
if turn_number is None:
turn_number = (
len([m for m in (conversation_history or []) if m.role == Role.user]) + 1
)
crewai_event_bus.emit(
agent_branch,
A2ADelegationStartedEvent(
endpoint=endpoint,
task_description=task_description,
agent_id=agent_id,
is_multiturn=is_multiturn,
turn_number=turn_number,
),
)
loop = asyncio.new_event_loop()
asyncio.set_event_loop(loop)
try:
result = loop.run_until_complete(
_execute_a2a_delegation_async(
endpoint=endpoint,
auth=auth,
timeout=timeout,
task_description=task_description,
context=context,
context_id=context_id,
task_id=task_id,
reference_task_ids=reference_task_ids,
metadata=metadata,
extensions=extensions,
conversation_history=conversation_history or [],
is_multiturn=is_multiturn,
turn_number=turn_number,
agent_branch=agent_branch,
agent_id=agent_id,
agent_role=agent_role,
response_model=response_model,
)
)
crewai_event_bus.emit(
agent_branch,
A2ADelegationCompletedEvent(
status=result["status"],
result=result.get("result"),
error=result.get("error"),
is_multiturn=is_multiturn,
),
)
return result
finally:
loop.close()
async def _execute_a2a_delegation_async(
endpoint: str,
auth: AuthScheme | None,
timeout: int,
task_description: str,
context: str | None,
context_id: str | None,
task_id: str | None,
reference_task_ids: list[str] | None,
metadata: dict[str, Any] | None,
extensions: dict[str, Any] | None,
conversation_history: list[Message],
is_multiturn: bool = False,
turn_number: int = 1,
agent_branch: Any | None = None,
agent_id: str | None = None,
agent_role: str | None = None,
response_model: type[BaseModel] | None = None,
) -> dict[str, Any]:
"""Async implementation of A2A delegation with multi-turn support.
Args:
endpoint: A2A agent endpoint URL
auth: Optional AuthScheme for authentication
timeout: Request timeout in seconds
task_description: Task to delegate
context: Optional context
context_id: Context ID for correlation
task_id: Specific task identifier
reference_task_ids: Related task IDs
metadata: Additional metadata
extensions: Protocol extensions
conversation_history: Previous Message objects
is_multiturn: Whether this is a multi-turn conversation
turn_number: Current turn number
agent_branch: Agent tree branch for logging
agent_id: Agent identifier for logging
agent_role: Agent role for logging
response_model: Optional Pydantic model for structured outputs
Returns:
Dictionary with status, result/error, and new history
"""
agent_card = await _fetch_agent_card_async(endpoint, auth, timeout)
validate_auth_against_agent_card(agent_card, auth)
headers: MutableMapping[str, str] = {}
if auth:
async with httpx.AsyncClient(timeout=timeout) as temp_auth_client:
if isinstance(auth, (HTTPDigestAuth, APIKeyAuth)):
configure_auth_client(auth, temp_auth_client)
headers = await auth.apply_auth(temp_auth_client, {})
a2a_agent_name = None
if agent_card.name:
a2a_agent_name = agent_card.name
if turn_number == 1:
agent_id_for_event = agent_id or endpoint
crewai_event_bus.emit(
agent_branch,
A2AConversationStartedEvent(
agent_id=agent_id_for_event,
endpoint=endpoint,
a2a_agent_name=a2a_agent_name,
),
)
message_parts = []
if context:
message_parts.append(f"Context:\n{context}\n\n")
message_parts.append(f"{task_description}")
message_text = "".join(message_parts)
if is_multiturn and conversation_history and not task_id:
if first_task_id := conversation_history[0].task_id:
task_id = first_task_id
parts: PartsDict = {"text": message_text}
if response_model:
parts.update(
{
"metadata": PartsMetadataDict(
mimeType="application/json",
schema=response_model.model_json_schema(),
)
}
)
message = Message(
role=Role.user,
message_id=str(uuid.uuid4()),
parts=[Part(root=TextPart(**parts))],
context_id=context_id,
task_id=task_id,
reference_task_ids=reference_task_ids,
metadata=metadata,
extensions=extensions,
)
transport_protocol = TransportProtocol("JSONRPC")
new_messages: list[Message] = [*conversation_history, message]
crewai_event_bus.emit(
None,
A2AMessageSentEvent(
message=message_text,
turn_number=turn_number,
is_multiturn=is_multiturn,
agent_role=agent_role,
),
)
async with _create_a2a_client(
agent_card=agent_card,
transport_protocol=transport_protocol,
timeout=timeout,
headers=headers,
streaming=True,
auth=auth,
) as client:
result_parts: list[str] = []
final_result: dict[str, Any] | None = None
event_stream = client.send_message(message)
try:
async for event in event_stream:
if isinstance(event, Message):
new_messages.append(event)
for part in event.parts:
if part.root.kind == "text":
text = part.root.text
result_parts.append(text)
elif isinstance(event, tuple):
a2a_task, update = event
if isinstance(update, TaskArtifactUpdateEvent):
artifact = update.artifact
result_parts.extend(
part.root.text
for part in artifact.parts
if part.root.kind == "text"
)
is_final_update = False
if isinstance(update, TaskStatusUpdateEvent):
is_final_update = update.final
if not is_final_update and a2a_task.status.state not in [
TaskState.completed,
TaskState.input_required,
TaskState.failed,
TaskState.rejected,
TaskState.auth_required,
TaskState.canceled,
]:
continue
if a2a_task.status.state == TaskState.completed:
extracted_parts = _extract_task_result_parts(a2a_task)
result_parts.extend(extracted_parts)
if a2a_task.history:
new_messages.extend(a2a_task.history)
response_text = " ".join(result_parts) if result_parts else ""
crewai_event_bus.emit(
None,
A2AResponseReceivedEvent(
response=response_text,
turn_number=turn_number,
is_multiturn=is_multiturn,
status="completed",
agent_role=agent_role,
),
)
final_result = {
"status": "completed",
"result": response_text,
"history": new_messages,
"agent_card": agent_card,
}
break
if a2a_task.status.state == TaskState.input_required:
if a2a_task.history:
new_messages.extend(a2a_task.history)
response_text = _extract_error_message(
a2a_task, "Additional input required"
)
if response_text and not a2a_task.history:
agent_message = Message(
role=Role.agent,
message_id=str(uuid.uuid4()),
parts=[Part(root=TextPart(text=response_text))],
context_id=a2a_task.context_id
if hasattr(a2a_task, "context_id")
else None,
task_id=a2a_task.task_id
if hasattr(a2a_task, "task_id")
else None,
)
new_messages.append(agent_message)
crewai_event_bus.emit(
None,
A2AResponseReceivedEvent(
response=response_text,
turn_number=turn_number,
is_multiturn=is_multiturn,
status="input_required",
agent_role=agent_role,
),
)
final_result = {
"status": "input_required",
"error": response_text,
"history": new_messages,
"agent_card": agent_card,
}
break
if a2a_task.status.state in [TaskState.failed, TaskState.rejected]:
error_msg = _extract_error_message(
a2a_task, "Task failed without error message"
)
if a2a_task.history:
new_messages.extend(a2a_task.history)
final_result = {
"status": "failed",
"error": error_msg,
"history": new_messages,
}
break
if a2a_task.status.state == TaskState.auth_required:
error_msg = _extract_error_message(
a2a_task, "Authentication required"
)
final_result = {
"status": "auth_required",
"error": error_msg,
"history": new_messages,
}
break
if a2a_task.status.state == TaskState.canceled:
error_msg = _extract_error_message(
a2a_task, "Task was canceled"
)
final_result = {
"status": "canceled",
"error": error_msg,
"history": new_messages,
}
break
except Exception as e:
current_exception: Exception | BaseException | None = e
while current_exception:
if hasattr(current_exception, "response"):
response = current_exception.response
if hasattr(response, "text"):
break
if current_exception and hasattr(current_exception, "__cause__"):
current_exception = current_exception.__cause__
raise
finally:
if hasattr(event_stream, "aclose"):
await event_stream.aclose()
if final_result:
return final_result
return {
"status": "completed",
"result": " ".join(result_parts) if result_parts else "",
"history": new_messages,
}
@asynccontextmanager
async def _create_a2a_client(
agent_card: AgentCard,
transport_protocol: TransportProtocol,
timeout: int,
headers: MutableMapping[str, str],
streaming: bool,
auth: AuthScheme | None = None,
) -> AsyncIterator[Client]:
"""Create and configure an A2A client.
Args:
agent_card: The A2A agent card
transport_protocol: Transport protocol to use
timeout: Request timeout in seconds
headers: HTTP headers (already with auth applied)
streaming: Enable streaming responses
auth: Optional AuthScheme for client configuration
Yields:
Configured A2A client instance
"""
async with httpx.AsyncClient(
timeout=timeout,
headers=headers,
) as httpx_client:
if auth and isinstance(auth, (HTTPDigestAuth, APIKeyAuth)):
configure_auth_client(auth, httpx_client)
config = ClientConfig(
httpx_client=httpx_client,
supported_transports=[str(transport_protocol.value)],
streaming=streaming,
accepted_output_modes=["application/json"],
)
factory = ClientFactory(config)
client = factory.create(agent_card)
yield client
def _extract_task_result_parts(a2a_task: A2ATask) -> list[str]:
"""Extract result parts from A2A task history and artifacts.
Args:
a2a_task: A2A Task object with history and artifacts
Returns:
List of result text parts
"""
result_parts: list[str] = []
if a2a_task.history:
for history_msg in reversed(a2a_task.history):
if history_msg.role == Role.agent:
result_parts.extend(
part.root.text
for part in history_msg.parts
if part.root.kind == "text"
)
break
if a2a_task.artifacts:
result_parts.extend(
part.root.text
for artifact in a2a_task.artifacts
for part in artifact.parts
if part.root.kind == "text"
)
return result_parts
def _extract_error_message(a2a_task: A2ATask, default: str) -> str:
"""Extract error message from A2A task.
Args:
a2a_task: A2A Task object
default: Default message if no error found
Returns:
Error message string
"""
if a2a_task.status and a2a_task.status.message:
msg = a2a_task.status.message
if msg:
for part in msg.parts:
if part.root.kind == "text":
return str(part.root.text)
return str(msg)
if a2a_task.history:
for history_msg in reversed(a2a_task.history):
for part in history_msg.parts:
if part.root.kind == "text":
return str(part.root.text)
return default
def create_agent_response_model(agent_ids: tuple[str, ...]) -> type[BaseModel]:
"""Create a dynamic AgentResponse model with Literal types for agent IDs.
Args:
agent_ids: List of available A2A agent IDs
Returns:
Dynamically created Pydantic model with Literal-constrained a2a_ids field
"""
DynamicLiteral = create_literals_from_strings(agent_ids) # noqa: N806
return create_model(
"AgentResponse",
a2a_ids=(
tuple[DynamicLiteral, ...], # type: ignore[valid-type]
Field(
default_factory=tuple,
max_length=len(agent_ids),
description="A2A agent IDs to delegate to.",
),
),
message=(
str,
Field(
description="The message content. If is_a2a=true, this is sent to the A2A agent. If is_a2a=false, this is your final answer ending the conversation."
),
),
is_a2a=(
bool,
Field(
description="Set to true to continue the conversation by sending this message to the A2A agent and awaiting their response. Set to false ONLY when you are completely done and providing your final answer (not when asking questions)."
),
),
__base__=BaseModel,
)
def extract_a2a_agent_ids_from_config(
a2a_config: list[A2AConfig] | A2AConfig | None,
) -> tuple[list[A2AConfig], tuple[str, ...]]:
"""Extract A2A agent IDs from A2A configuration.
Args:
a2a_config: A2A configuration
Returns:
List of A2A agent IDs
"""
if a2a_config is None:
return [], ()
if isinstance(a2a_config, A2AConfig):
a2a_agents = [a2a_config]
else:
a2a_agents = a2a_config
return a2a_agents, tuple(config.endpoint for config in a2a_agents)
def get_a2a_agents_and_response_model(
a2a_config: list[A2AConfig] | A2AConfig | None,
) -> tuple[list[A2AConfig], type[BaseModel]]:
"""Get A2A agent IDs and response model.
Args:
a2a_config: A2A configuration
Returns:
Tuple of A2A agent IDs and response model
"""
a2a_agents, agent_ids = extract_a2a_agent_ids_from_config(a2a_config=a2a_config)
return a2a_agents, create_agent_response_model(agent_ids)
def extract_agent_identifiers_from_cards(
a2a_agents: list[A2AConfig],
agent_cards: dict[str, AgentCard],
) -> tuple[str, ...]:
"""Extract all valid agent identifiers (endpoints and skill IDs) from agent cards.
Args:
a2a_agents: List of A2A agent configurations
agent_cards: Dictionary mapping endpoints to AgentCards
Returns:
Tuple of all valid identifiers (endpoints + skill IDs)
"""
identifiers = set()
for config in a2a_agents:
identifiers.add(config.endpoint)
for card in agent_cards.values():
if card.skills:
for skill in card.skills:
identifiers.add(skill.id)
return tuple(sorted(identifiers))
def resolve_agent_identifier(
identifier: str,
a2a_agents: list[A2AConfig],
agent_cards: dict[str, AgentCard],
) -> str:
"""Resolve an agent identifier (endpoint or skill ID) to a canonical endpoint.
This function allows both endpoint URLs and skill IDs to be used as agent identifiers.
If the identifier is already an endpoint, it's returned as-is. If it's a skill ID,
it's resolved to the endpoint of the agent card that contains that skill.
Args:
identifier: Either an endpoint URL or a skill ID
a2a_agents: List of A2A agent configurations
agent_cards: Dictionary mapping endpoints to AgentCards
Returns:
The canonical endpoint URL
Raises:
ValueError: If the identifier is unknown or ambiguous (matches multiple agents)
Examples:
>>> # Endpoint passthrough
>>> resolve_agent_identifier(
... "http://localhost:10001/.well-known/agent-card.json",
... a2a_agents,
... agent_cards
... )
'http://localhost:10001/.well-known/agent-card.json'
>>> # Skill ID resolution
>>> resolve_agent_identifier("Research", a2a_agents, agent_cards)
'http://localhost:10001/.well-known/agent-card.json'
"""
endpoints = {config.endpoint for config in a2a_agents}
if identifier in endpoints:
return identifier
matching_endpoints: list[str] = []
for endpoint, card in agent_cards.items():
if card.skills:
for skill in card.skills:
if skill.id == identifier:
matching_endpoints.append(endpoint)
break
if len(matching_endpoints) == 0:
available_endpoints = ", ".join(sorted(endpoints))
available_skill_ids = []
for card in agent_cards.values():
if card.skills:
available_skill_ids.extend([skill.id for skill in card.skills])
available_skills = ", ".join(sorted(set(available_skill_ids))) if available_skill_ids else "none"
raise ValueError(
f"Unknown A2A agent identifier '{identifier}'. "
f"Available endpoints: {available_endpoints}. "
f"Available skill IDs: {available_skills}."
)
if len(matching_endpoints) > 1:
endpoints_list = ", ".join(sorted(matching_endpoints))
raise ValueError(
f"Ambiguous skill ID '{identifier}' found in multiple agents: {endpoints_list}. "
f"Please use the specific endpoint URL to disambiguate."
)
return matching_endpoints[0]

View File

@@ -1,590 +0,0 @@
"""A2A agent wrapping logic for metaclass integration.
Wraps agent classes with A2A delegation capabilities.
"""
from __future__ import annotations
from collections.abc import Callable
from concurrent.futures import ThreadPoolExecutor, as_completed
from functools import wraps
from types import MethodType
from typing import TYPE_CHECKING, Any, cast
from a2a.types import Role
from pydantic import BaseModel, ValidationError
from crewai.a2a.config import A2AConfig
from crewai.a2a.templates import (
AVAILABLE_AGENTS_TEMPLATE,
CONVERSATION_TURN_INFO_TEMPLATE,
PREVIOUS_A2A_CONVERSATION_TEMPLATE,
UNAVAILABLE_AGENTS_NOTICE_TEMPLATE,
)
from crewai.a2a.types import AgentResponseProtocol
from crewai.a2a.utils import (
create_agent_response_model,
execute_a2a_delegation,
extract_agent_identifiers_from_cards,
fetch_agent_card,
get_a2a_agents_and_response_model,
resolve_agent_identifier,
)
from crewai.events.event_bus import crewai_event_bus
from crewai.events.types.a2a_events import (
A2AConversationCompletedEvent,
A2AMessageSentEvent,
)
if TYPE_CHECKING:
from a2a.types import AgentCard, Message
from crewai.agent.core import Agent
from crewai.task import Task
from crewai.tools.base_tool import BaseTool
def wrap_agent_with_a2a_instance(agent: Agent) -> None:
"""Wrap an agent instance's execute_task method with A2A support.
This function modifies the agent instance by wrapping its execute_task
method to add A2A delegation capabilities. Should only be called when
the agent has a2a configuration set.
Args:
agent: The agent instance to wrap
"""
original_execute_task = agent.execute_task.__func__
@wraps(original_execute_task)
def execute_task_with_a2a(
self: Agent,
task: Task,
context: str | None = None,
tools: list[BaseTool] | None = None,
) -> str:
"""Execute task with A2A delegation support.
Args:
self: The agent instance
task: The task to execute
context: Optional context for task execution
tools: Optional tools available to the agent
Returns:
Task execution result
"""
if not self.a2a:
return original_execute_task(self, task, context, tools)
a2a_agents, agent_response_model = get_a2a_agents_and_response_model(self.a2a)
return _execute_task_with_a2a(
self=self,
a2a_agents=a2a_agents,
original_fn=original_execute_task,
task=task,
agent_response_model=agent_response_model,
context=context,
tools=tools,
)
object.__setattr__(agent, "execute_task", MethodType(execute_task_with_a2a, agent))
def _fetch_card_from_config(
config: A2AConfig,
) -> tuple[A2AConfig, AgentCard | Exception]:
"""Fetch agent card from A2A config.
Args:
config: A2A configuration
Returns:
Tuple of (config, card or exception)
"""
try:
card = fetch_agent_card(
endpoint=config.endpoint,
auth=config.auth,
timeout=config.timeout,
)
return config, card
except Exception as e:
return config, e
def _fetch_agent_cards_concurrently(
a2a_agents: list[A2AConfig],
) -> tuple[dict[str, AgentCard], dict[str, str]]:
"""Fetch agent cards concurrently for multiple A2A agents.
Args:
a2a_agents: List of A2A agent configurations
Returns:
Tuple of (agent_cards dict, failed_agents dict mapping endpoint to error message)
"""
agent_cards: dict[str, AgentCard] = {}
failed_agents: dict[str, str] = {}
with ThreadPoolExecutor(max_workers=len(a2a_agents)) as executor:
futures = {
executor.submit(_fetch_card_from_config, config): config
for config in a2a_agents
}
for future in as_completed(futures):
config, result = future.result()
if isinstance(result, Exception):
if config.fail_fast:
raise RuntimeError(
f"Failed to fetch agent card from {config.endpoint}. "
f"Ensure the A2A agent is running and accessible. Error: {result}"
) from result
failed_agents[config.endpoint] = str(result)
else:
agent_cards[config.endpoint] = result
return agent_cards, failed_agents
def _execute_task_with_a2a(
self: Agent,
a2a_agents: list[A2AConfig],
original_fn: Callable[..., str],
task: Task,
agent_response_model: type[BaseModel],
context: str | None,
tools: list[BaseTool] | None,
) -> str:
"""Wrap execute_task with A2A delegation logic.
Args:
self: The agent instance
a2a_agents: Dictionary of A2A agent configurations
original_fn: The original execute_task method
task: The task to execute
context: Optional context for task execution
tools: Optional tools available to the agent
agent_response_model: Optional agent response model
Returns:
Task execution result (either from LLM or A2A agent)
"""
original_description: str = task.description
original_output_pydantic = task.output_pydantic
original_response_model = task.response_model
agent_cards, failed_agents = _fetch_agent_cards_concurrently(a2a_agents)
if not agent_cards and a2a_agents and failed_agents:
unavailable_agents_text = ""
for endpoint, error in failed_agents.items():
unavailable_agents_text += f" - {endpoint}: {error}\n"
notice = UNAVAILABLE_AGENTS_NOTICE_TEMPLATE.substitute(
unavailable_agents=unavailable_agents_text
)
task.description = f"{original_description}{notice}"
try:
return original_fn(self, task, context, tools)
finally:
task.description = original_description
agent_identifiers = extract_agent_identifiers_from_cards(a2a_agents, agent_cards)
agent_response_model = create_agent_response_model(agent_identifiers)
task.description = _augment_prompt_with_a2a(
a2a_agents=a2a_agents,
task_description=original_description,
agent_cards=agent_cards,
failed_agents=failed_agents,
)
task.response_model = agent_response_model
try:
raw_result = original_fn(self, task, context, tools)
agent_response = _parse_agent_response(
raw_result=raw_result, agent_response_model=agent_response_model
)
if isinstance(agent_response, BaseModel) and isinstance(
agent_response, AgentResponseProtocol
):
if agent_response.is_a2a:
return _delegate_to_a2a(
self,
agent_response=agent_response,
task=task,
original_fn=original_fn,
context=context,
tools=tools,
agent_cards=agent_cards,
original_task_description=original_description,
)
return str(agent_response.message)
return raw_result
finally:
task.description = original_description
task.output_pydantic = original_output_pydantic
task.response_model = original_response_model
def _augment_prompt_with_a2a(
a2a_agents: list[A2AConfig],
task_description: str,
agent_cards: dict[str, AgentCard],
conversation_history: list[Message] | None = None,
turn_num: int = 0,
max_turns: int | None = None,
failed_agents: dict[str, str] | None = None,
) -> str:
"""Add A2A delegation instructions to prompt.
Args:
a2a_agents: Dictionary of A2A agent configurations
task_description: Original task description
agent_cards: dictionary mapping agent IDs to AgentCards
conversation_history: Previous A2A Messages from conversation
turn_num: Current turn number (0-indexed)
max_turns: Maximum allowed turns (from config)
failed_agents: Dictionary mapping failed agent endpoints to error messages
Returns:
Augmented task description with A2A instructions
"""
if not agent_cards:
return task_description
agents_text = ""
for config in a2a_agents:
if config.endpoint in agent_cards:
card = agent_cards[config.endpoint]
agents_text += f"\n{card.model_dump_json(indent=2, exclude_none=True, include={'description', 'url', 'skills'})}\n"
failed_agents = failed_agents or {}
if failed_agents:
agents_text += "\n<!-- Unavailable Agents -->\n"
for endpoint, error in failed_agents.items():
agents_text += f"\n<!-- Agent: {endpoint}\n Status: Unavailable\n Error: {error} -->\n"
agents_text = AVAILABLE_AGENTS_TEMPLATE.substitute(available_a2a_agents=agents_text)
history_text = ""
if conversation_history:
for msg in conversation_history:
history_text += f"\n{msg.model_dump_json(indent=2, exclude_none=True, exclude={'message_id'})}\n"
history_text = PREVIOUS_A2A_CONVERSATION_TEMPLATE.substitute(
previous_a2a_conversation=history_text
)
turn_info = ""
if max_turns is not None and conversation_history:
turn_count = turn_num + 1
warning = ""
if turn_count >= max_turns:
warning = (
"CRITICAL: This is the FINAL turn. You MUST conclude the conversation now.\n"
"Set is_a2a=false and provide your final response to complete the task."
)
elif turn_count == max_turns - 1:
warning = "WARNING: Next turn will be the last. Consider wrapping up the conversation."
turn_info = CONVERSATION_TURN_INFO_TEMPLATE.substitute(
turn_count=turn_count,
max_turns=max_turns,
warning=warning,
)
return f"""{task_description}
IMPORTANT: You have the ability to delegate this task to remote A2A agents.
{agents_text}
AGENT IDENTIFICATION: When setting a2a_ids, you may use either:
1. The agent's endpoint URL (e.g., "http://localhost:10001/.well-known/agent-card.json")
2. The exact skill.id from the agent's skills list (e.g., "Research")
Prefer using endpoint URLs when possible to avoid ambiguity. If a skill.id appears on multiple agents, you MUST use the endpoint URL to specify which agent you want.
{history_text}{turn_info}
"""
def _parse_agent_response(
raw_result: str | dict[str, Any], agent_response_model: type[BaseModel]
) -> BaseModel | str:
"""Parse LLM output as AgentResponse or return raw agent response.
Args:
raw_result: Raw output from LLM
agent_response_model: The agent response model
Returns:
Parsed AgentResponse or string
"""
if agent_response_model:
try:
if isinstance(raw_result, str):
return agent_response_model.model_validate_json(raw_result)
if isinstance(raw_result, dict):
return agent_response_model.model_validate(raw_result)
except ValidationError:
return cast(str, raw_result)
return cast(str, raw_result)
def _handle_agent_response_and_continue(
self: Agent,
a2a_result: dict[str, Any],
agent_id: str,
agent_cards: dict[str, AgentCard] | None,
a2a_agents: list[A2AConfig],
original_task_description: str,
conversation_history: list[Message],
turn_num: int,
max_turns: int,
task: Task,
original_fn: Callable[..., str],
context: str | None,
tools: list[BaseTool] | None,
agent_response_model: type[BaseModel],
) -> tuple[str | None, str | None]:
"""Handle A2A result and get CrewAI agent's response.
Args:
self: The agent instance
a2a_result: Result from A2A delegation
agent_id: ID of the A2A agent
agent_cards: Pre-fetched agent cards
a2a_agents: List of A2A configurations
original_task_description: Original task description
conversation_history: Conversation history
turn_num: Current turn number
max_turns: Maximum turns allowed
task: The task being executed
original_fn: Original execute_task method
context: Optional context
tools: Optional tools
agent_response_model: Response model for parsing
Returns:
Tuple of (final_result, current_request) where:
- final_result is not None if conversation should end
- current_request is the next message to send if continuing
"""
agent_cards_dict = agent_cards or {}
if "agent_card" in a2a_result and agent_id not in agent_cards_dict:
agent_cards_dict[agent_id] = a2a_result["agent_card"]
agent_identifiers = extract_agent_identifiers_from_cards(a2a_agents, agent_cards_dict)
agent_response_model = create_agent_response_model(agent_identifiers)
task.description = _augment_prompt_with_a2a(
a2a_agents=a2a_agents,
task_description=original_task_description,
conversation_history=conversation_history,
turn_num=turn_num,
max_turns=max_turns,
agent_cards=agent_cards_dict,
)
raw_result = original_fn(self, task, context, tools)
llm_response = _parse_agent_response(
raw_result=raw_result, agent_response_model=agent_response_model
)
if isinstance(llm_response, BaseModel) and isinstance(
llm_response, AgentResponseProtocol
):
if not llm_response.is_a2a:
final_turn_number = turn_num + 1
crewai_event_bus.emit(
None,
A2AMessageSentEvent(
message=str(llm_response.message),
turn_number=final_turn_number,
is_multiturn=True,
agent_role=self.role,
),
)
crewai_event_bus.emit(
None,
A2AConversationCompletedEvent(
status="completed",
final_result=str(llm_response.message),
error=None,
total_turns=final_turn_number,
),
)
return str(llm_response.message), None
return None, str(llm_response.message)
return str(raw_result), None
def _delegate_to_a2a(
self: Agent,
agent_response: AgentResponseProtocol,
task: Task,
original_fn: Callable[..., str],
context: str | None,
tools: list[BaseTool] | None,
agent_cards: dict[str, AgentCard] | None = None,
original_task_description: str | None = None,
) -> str:
"""Delegate to A2A agent with multi-turn conversation support.
Args:
self: The agent instance
agent_response: The AgentResponse indicating delegation
task: The task being executed (for extracting A2A fields)
original_fn: The original execute_task method for follow-ups
context: Optional context for task execution
tools: Optional tools available to the agent
agent_cards: Pre-fetched agent cards from _execute_task_with_a2a
original_task_description: The original task description before A2A augmentation
Returns:
Result from A2A agent
Raises:
ImportError: If a2a-sdk is not installed
"""
a2a_agents, agent_response_model = get_a2a_agents_and_response_model(self.a2a)
current_request = str(agent_response.message)
agent_identifier = agent_response.a2a_ids[0]
agent_cards_dict = agent_cards or {}
try:
agent_endpoint = resolve_agent_identifier(
agent_identifier, a2a_agents, agent_cards_dict
)
except ValueError as e:
raise ValueError(
f"Failed to resolve A2A agent identifier '{agent_identifier}': {e}"
) from e
agent_config = next(filter(lambda x: x.endpoint == agent_endpoint, a2a_agents))
task_config = task.config or {}
context_id = task_config.get("context_id")
task_id_config = task_config.get("task_id")
reference_task_ids = task_config.get("reference_task_ids")
metadata = task_config.get("metadata")
extensions = task_config.get("extensions")
if original_task_description is None:
original_task_description = task.description
conversation_history: list[Message] = []
max_turns = agent_config.max_turns
try:
for turn_num in range(max_turns):
console_formatter = getattr(crewai_event_bus, "_console", None)
agent_branch = None
if console_formatter:
agent_branch = getattr(
console_formatter, "current_agent_branch", None
) or getattr(console_formatter, "current_task_branch", None)
a2a_result = execute_a2a_delegation(
endpoint=agent_config.endpoint,
auth=agent_config.auth,
timeout=agent_config.timeout,
task_description=current_request,
context_id=context_id,
task_id=task_id_config,
reference_task_ids=reference_task_ids,
metadata=metadata,
extensions=extensions,
conversation_history=conversation_history,
agent_id=agent_endpoint,
agent_role=Role.user,
agent_branch=agent_branch,
response_model=agent_config.response_model,
turn_number=turn_num + 1,
)
conversation_history = a2a_result.get("history", [])
if a2a_result["status"] in ["completed", "input_required"]:
final_result, next_request = _handle_agent_response_and_continue(
self=self,
a2a_result=a2a_result,
agent_id=agent_endpoint,
agent_cards=agent_cards,
a2a_agents=a2a_agents,
original_task_description=original_task_description,
conversation_history=conversation_history,
turn_num=turn_num,
max_turns=max_turns,
task=task,
original_fn=original_fn,
context=context,
tools=tools,
agent_response_model=agent_response_model,
)
if final_result is not None:
return final_result
if next_request is not None:
current_request = next_request
continue
error_msg = a2a_result.get("error", "Unknown error")
crewai_event_bus.emit(
None,
A2AConversationCompletedEvent(
status="failed",
final_result=None,
error=error_msg,
total_turns=turn_num + 1,
),
)
raise Exception(f"A2A delegation failed: {error_msg}")
if conversation_history:
for msg in reversed(conversation_history):
if msg.role == Role.agent:
text_parts = [
part.root.text for part in msg.parts if part.root.kind == "text"
]
final_message = (
" ".join(text_parts) if text_parts else "Conversation completed"
)
crewai_event_bus.emit(
None,
A2AConversationCompletedEvent(
status="completed",
final_result=final_message,
error=None,
total_turns=max_turns,
),
)
return final_message
crewai_event_bus.emit(
None,
A2AConversationCompletedEvent(
status="failed",
final_result=None,
error=f"Conversation exceeded maximum turns ({max_turns})",
total_turns=max_turns,
),
)
raise Exception(f"A2A conversation exceeded maximum turns ({max_turns})")
finally:
task.description = original_task_description

View File

@@ -2,27 +2,27 @@ from __future__ import annotations
import asyncio
from collections.abc import Sequence
import json
import shutil
import subprocess
import time
from typing import (
TYPE_CHECKING,
Any,
Final,
Literal,
cast,
)
from urllib.parse import urlparse
from pydantic import BaseModel, Field, InstanceOf, PrivateAttr, model_validator
from typing_extensions import Self
from crewai.a2a.config import A2AConfig
from crewai.agents.agent_builder.base_agent import BaseAgent
from crewai.agents.cache.cache_handler import CacheHandler
from crewai.agents.crew_agent_executor import CrewAgentExecutor
from crewai.events.event_bus import crewai_event_bus
from crewai.events.types.agent_events import (
AgentExecutionCompletedEvent,
AgentExecutionErrorEvent,
AgentExecutionStartedEvent,
)
from crewai.events.types.knowledge_events import (
KnowledgeQueryCompletedEvent,
KnowledgeQueryFailedEvent,
@@ -40,16 +40,6 @@ from crewai.knowledge.source.base_knowledge_source import BaseKnowledgeSource
from crewai.knowledge.utils.knowledge_utils import extract_knowledge_context
from crewai.lite_agent import LiteAgent
from crewai.llms.base_llm import BaseLLM
from crewai.mcp import (
MCPClient,
MCPServerConfig,
MCPServerHTTP,
MCPServerSSE,
MCPServerStdio,
)
from crewai.mcp.transports.http import HTTPTransport
from crewai.mcp.transports.sse import SSETransport
from crewai.mcp.transports.stdio import StdioTransport
from crewai.memory.contextual.contextual_memory import ContextualMemory
from crewai.rag.embeddings.types import EmbedderConfig
from crewai.security.fingerprint import Fingerprint
@@ -80,14 +70,14 @@ if TYPE_CHECKING:
# MCP Connection timeout constants (in seconds)
MCP_CONNECTION_TIMEOUT: Final[int] = 10
MCP_TOOL_EXECUTION_TIMEOUT: Final[int] = 30
MCP_DISCOVERY_TIMEOUT: Final[int] = 15
MCP_MAX_RETRIES: Final[int] = 3
MCP_CONNECTION_TIMEOUT = 10
MCP_TOOL_EXECUTION_TIMEOUT = 30
MCP_DISCOVERY_TIMEOUT = 15
MCP_MAX_RETRIES = 3
# Simple in-memory cache for MCP tool schemas (duration: 5 minutes)
_mcp_schema_cache: dict[str, Any] = {}
_cache_ttl: Final[int] = 300 # 5 minutes
_mcp_schema_cache = {}
_cache_ttl = 300 # 5 minutes
class Agent(BaseAgent):
@@ -118,8 +108,6 @@ class Agent(BaseAgent):
"""
_times_executed: int = PrivateAttr(default=0)
_mcp_clients: list[Any] = PrivateAttr(default_factory=list)
_last_messages: list[LLMMessage] = PrivateAttr(default_factory=list)
max_execution_time: int | None = Field(
default=None,
description="Maximum execution time for an agent to execute a task",
@@ -209,10 +197,6 @@ class Agent(BaseAgent):
guardrail_max_retries: int = Field(
default=3, description="Maximum number of retries when guardrail fails"
)
a2a: list[A2AConfig] | A2AConfig | None = Field(
default=None,
description="A2A (Agent-to-Agent) configuration for delegating tasks to remote agents. Can be a single A2AConfig or a dict mapping agent IDs to configs.",
)
@model_validator(mode="before")
def validate_from_repository(cls, v: Any) -> dict[str, Any] | None | Any: # noqa: N805
@@ -321,19 +305,17 @@ class Agent(BaseAgent):
# If the task requires output in JSON or Pydantic format,
# append specific instructions to the task prompt to ensure
# that the final answer does not include any code block markers
# Skip this if task.response_model is set, as native structured outputs handle schema automatically
if (task.output_json or task.output_pydantic) and not task.response_model:
if task.output_json or task.output_pydantic:
# Generate the schema based on the output format
if task.output_json:
schema_dict = generate_model_description(task.output_json)
schema = json.dumps(schema_dict["json_schema"]["schema"], indent=2)
# schema = json.dumps(task.output_json, indent=2)
schema = generate_model_description(task.output_json)
task_prompt += "\n" + self.i18n.slice(
"formatted_task_instructions"
).format(output_format=schema)
elif task.output_pydantic:
schema_dict = generate_model_description(task.output_pydantic)
schema = json.dumps(schema_dict["json_schema"]["schema"], indent=2)
schema = generate_model_description(task.output_pydantic)
task_prompt += "\n" + self.i18n.slice(
"formatted_task_instructions"
).format(output_format=schema)
@@ -456,13 +438,6 @@ class Agent(BaseAgent):
else:
task_prompt = self._use_trained_data(task_prompt=task_prompt)
# Import agent events locally to avoid circular imports
from crewai.events.types.agent_events import (
AgentExecutionCompletedEvent,
AgentExecutionErrorEvent,
AgentExecutionStartedEvent,
)
try:
crewai_event_bus.emit(
self,
@@ -538,15 +513,6 @@ class Agent(BaseAgent):
self,
event=AgentExecutionCompletedEvent(agent=self, task=task, output=result),
)
self._last_messages = (
self.agent_executor.messages.copy()
if self.agent_executor and hasattr(self.agent_executor, "messages")
else []
)
self._cleanup_mcp_clients()
return result
def _execute_with_timeout(self, task_prompt: str, task: Task, timeout: int) -> Any:
@@ -652,7 +618,6 @@ class Agent(BaseAgent):
self._rpm_controller.check_or_wait if self._rpm_controller else None
),
callbacks=[TokenCalcHandler(self._token_process)],
response_model=task.response_model if task else None,
)
def get_delegation_tools(self, agents: list[BaseAgent]) -> list[BaseTool]:
@@ -670,70 +635,30 @@ class Agent(BaseAgent):
self._logger.log("error", f"Error getting platform tools: {e!s}")
return []
def get_mcp_tools(self, mcps: list[str | MCPServerConfig]) -> list[BaseTool]:
"""Convert MCP server references/configs to CrewAI tools.
Supports both string references (backwards compatible) and structured
configuration objects (MCPServerStdio, MCPServerHTTP, MCPServerSSE).
Args:
mcps: List of MCP server references (strings) or configurations.
Returns:
List of BaseTool instances from MCP servers.
"""
def get_mcp_tools(self, mcps: list[str]) -> list[BaseTool]:
"""Convert MCP server references to CrewAI tools."""
all_tools = []
clients = []
for mcp_config in mcps:
if isinstance(mcp_config, str):
tools = self._get_mcp_tools_from_string(mcp_config)
else:
tools, client = self._get_native_mcp_tools(mcp_config)
if client:
clients.append(client)
for mcp_ref in mcps:
try:
if mcp_ref.startswith("crewai-amp:"):
tools = self._get_amp_mcp_tools(mcp_ref)
elif mcp_ref.startswith("https://"):
tools = self._get_external_mcp_tools(mcp_ref)
else:
continue
all_tools.extend(tools)
all_tools.extend(tools)
self._logger.log(
"info", f"Successfully loaded {len(tools)} tools from {mcp_ref}"
)
except Exception as e:
self._logger.log("warning", f"Skipping MCP {mcp_ref} due to error: {e}")
continue
# Store clients for cleanup
self._mcp_clients.extend(clients)
return all_tools
def _cleanup_mcp_clients(self) -> None:
"""Cleanup MCP client connections after task execution."""
if not self._mcp_clients:
return
async def _disconnect_all() -> None:
for client in self._mcp_clients:
if client and hasattr(client, "connected") and client.connected:
await client.disconnect()
try:
asyncio.run(_disconnect_all())
except Exception as e:
self._logger.log("error", f"Error during MCP client cleanup: {e}")
finally:
self._mcp_clients.clear()
def _get_mcp_tools_from_string(self, mcp_ref: str) -> list[BaseTool]:
"""Get tools from legacy string-based MCP references.
This method maintains backwards compatibility with string-based
MCP references (https://... and crewai-amp:...).
Args:
mcp_ref: String reference to MCP server.
Returns:
List of BaseTool instances.
"""
if mcp_ref.startswith("crewai-amp:"):
return self._get_amp_mcp_tools(mcp_ref)
if mcp_ref.startswith("https://"):
return self._get_external_mcp_tools(mcp_ref)
return []
def _get_external_mcp_tools(self, mcp_ref: str) -> list[BaseTool]:
"""Get tools from external HTTPS MCP server with graceful error handling."""
from crewai.tools.mcp_tool_wrapper import MCPToolWrapper
@@ -784,7 +709,7 @@ class Agent(BaseAgent):
f"Specific tool '{specific_tool}' not found on MCP server: {server_url}",
)
return cast(list[BaseTool], tools)
return tools
except Exception as e:
self._logger.log(
@@ -792,164 +717,6 @@ class Agent(BaseAgent):
)
return []
def _get_native_mcp_tools(
self, mcp_config: MCPServerConfig
) -> tuple[list[BaseTool], Any | None]:
"""Get tools from MCP server using structured configuration.
This method creates an MCP client based on the configuration type,
connects to the server, discovers tools, applies filtering, and
returns wrapped tools along with the client instance for cleanup.
Args:
mcp_config: MCP server configuration (MCPServerStdio, MCPServerHTTP, or MCPServerSSE).
Returns:
Tuple of (list of BaseTool instances, MCPClient instance for cleanup).
"""
from crewai.tools.base_tool import BaseTool
from crewai.tools.mcp_native_tool import MCPNativeTool
if isinstance(mcp_config, MCPServerStdio):
transport = StdioTransport(
command=mcp_config.command,
args=mcp_config.args,
env=mcp_config.env,
)
server_name = f"{mcp_config.command}_{'_'.join(mcp_config.args)}"
elif isinstance(mcp_config, MCPServerHTTP):
transport = HTTPTransport(
url=mcp_config.url,
headers=mcp_config.headers,
streamable=mcp_config.streamable,
)
server_name = self._extract_server_name(mcp_config.url)
elif isinstance(mcp_config, MCPServerSSE):
transport = SSETransport(
url=mcp_config.url,
headers=mcp_config.headers,
)
server_name = self._extract_server_name(mcp_config.url)
else:
raise ValueError(f"Unsupported MCP server config type: {type(mcp_config)}")
client = MCPClient(
transport=transport,
cache_tools_list=mcp_config.cache_tools_list,
)
async def _setup_client_and_list_tools() -> list[dict[str, Any]]:
"""Async helper to connect and list tools in same event loop."""
try:
if not client.connected:
await client.connect()
tools_list = await client.list_tools()
try:
await client.disconnect()
# Small delay to allow background tasks to finish cleanup
# This helps prevent "cancel scope in different task" errors
# when asyncio.run() closes the event loop
await asyncio.sleep(0.1)
except Exception as e:
self._logger.log("error", f"Error during disconnect: {e}")
return tools_list
except Exception as e:
if client.connected:
await client.disconnect()
await asyncio.sleep(0.1)
raise RuntimeError(
f"Error during setup client and list tools: {e}"
) from e
try:
try:
asyncio.get_running_loop()
import concurrent.futures
with concurrent.futures.ThreadPoolExecutor() as executor:
future = executor.submit(
asyncio.run, _setup_client_and_list_tools()
)
tools_list = future.result()
except RuntimeError:
try:
tools_list = asyncio.run(_setup_client_and_list_tools())
except RuntimeError as e:
error_msg = str(e).lower()
if "cancel scope" in error_msg or "task" in error_msg:
raise ConnectionError(
"MCP connection failed due to event loop cleanup issues. "
"This may be due to authentication errors or server unavailability."
) from e
except asyncio.CancelledError as e:
raise ConnectionError(
"MCP connection was cancelled. This may indicate an authentication "
"error or server unavailability."
) from e
if mcp_config.tool_filter:
filtered_tools = []
for tool in tools_list:
if callable(mcp_config.tool_filter):
try:
from crewai.mcp.filters import ToolFilterContext
context = ToolFilterContext(
agent=self,
server_name=server_name,
run_context=None,
)
if mcp_config.tool_filter(context, tool):
filtered_tools.append(tool)
except (TypeError, AttributeError):
if mcp_config.tool_filter(tool):
filtered_tools.append(tool)
else:
# Not callable - include tool
filtered_tools.append(tool)
tools_list = filtered_tools
tools = []
for tool_def in tools_list:
tool_name = tool_def.get("name", "")
if not tool_name:
continue
# Convert inputSchema to Pydantic model if present
args_schema = None
if tool_def.get("inputSchema"):
args_schema = self._json_schema_to_pydantic(
tool_name, tool_def["inputSchema"]
)
tool_schema = {
"description": tool_def.get("description", ""),
"args_schema": args_schema,
}
try:
native_tool = MCPNativeTool(
mcp_client=client,
tool_name=tool_name,
tool_schema=tool_schema,
server_name=server_name,
)
tools.append(native_tool)
except Exception as e:
self._logger.log("error", f"Failed to create native MCP tool: {e}")
continue
return cast(list[BaseTool], tools), client
except Exception as e:
if client.connected:
asyncio.run(client.disconnect())
raise RuntimeError(f"Failed to get native MCP tools: {e}") from e
def _get_amp_mcp_tools(self, amp_ref: str) -> list[BaseTool]:
"""Get tools from CrewAI AMP MCP marketplace."""
# Parse: "crewai-amp:mcp-name" or "crewai-amp:mcp-name#tool_name"
@@ -972,9 +739,9 @@ class Agent(BaseAgent):
return tools
@staticmethod
def _extract_server_name(server_url: str) -> str:
def _extract_server_name(self, server_url: str) -> str:
"""Extract clean server name from URL for tool prefixing."""
from urllib.parse import urlparse
parsed = urlparse(server_url)
domain = parsed.netloc.replace(".", "_")
@@ -1011,9 +778,7 @@ class Agent(BaseAgent):
)
return {}
async def _get_mcp_tool_schemas_async(
self, server_params: dict[str, Any]
) -> dict[str, dict]:
async def _get_mcp_tool_schemas_async(self, server_params: dict) -> dict[str, dict]:
"""Async implementation of MCP tool schema retrieval with timeouts and retries."""
server_url = server_params["url"]
return await self._retry_mcp_discovery(
@@ -1022,7 +787,7 @@ class Agent(BaseAgent):
async def _retry_mcp_discovery(
self, operation_func, server_url: str
) -> dict[str, dict[str, Any]]:
) -> dict[str, dict]:
"""Retry MCP discovery operation with exponential backoff, avoiding try-except in loop."""
last_error = None
@@ -1050,10 +815,9 @@ class Agent(BaseAgent):
f"Failed to discover MCP tools after {MCP_MAX_RETRIES} attempts: {last_error}"
)
@staticmethod
async def _attempt_mcp_discovery(
operation_func, server_url: str
) -> tuple[dict[str, dict[str, Any]] | None, str, bool]:
self, operation_func, server_url: str
) -> tuple[dict[str, dict] | None, str, bool]:
"""Attempt single MCP discovery operation and return (result, error_message, should_retry)."""
try:
result = await operation_func(server_url)
@@ -1087,13 +851,13 @@ class Agent(BaseAgent):
async def _discover_mcp_tools_with_timeout(
self, server_url: str
) -> dict[str, dict[str, Any]]:
) -> dict[str, dict]:
"""Discover MCP tools with timeout wrapper."""
return await asyncio.wait_for(
self._discover_mcp_tools(server_url), timeout=MCP_DISCOVERY_TIMEOUT
)
async def _discover_mcp_tools(self, server_url: str) -> dict[str, dict[str, Any]]:
async def _discover_mcp_tools(self, server_url: str) -> dict[str, dict]:
"""Discover tools from MCP server with proper timeout handling."""
from mcp import ClientSession
from mcp.client.streamable_http import streamablehttp_client
@@ -1125,9 +889,7 @@ class Agent(BaseAgent):
}
return schemas
def _json_schema_to_pydantic(
self, tool_name: str, json_schema: dict[str, Any]
) -> type:
def _json_schema_to_pydantic(self, tool_name: str, json_schema: dict) -> type:
"""Convert JSON Schema to Pydantic model for tool arguments.
Args:
@@ -1164,7 +926,7 @@ class Agent(BaseAgent):
model_name = f"{tool_name.replace('-', '_').replace(' ', '_')}Schema"
return create_model(model_name, **field_definitions)
def _json_type_to_python(self, field_schema: dict[str, Any]) -> type:
def _json_type_to_python(self, field_schema: dict) -> type:
"""Convert JSON Schema type to Python type.
Args:
@@ -1173,6 +935,7 @@ class Agent(BaseAgent):
Returns:
Python type
"""
from typing import Any
json_type = field_schema.get("type")
@@ -1202,15 +965,13 @@ class Agent(BaseAgent):
return type_mapping.get(json_type, Any)
@staticmethod
def _fetch_amp_mcp_servers(mcp_name: str) -> list[dict]:
def _fetch_amp_mcp_servers(self, mcp_name: str) -> list[dict]:
"""Fetch MCP server configurations from CrewAI AMP API."""
# TODO: Implement AMP API call to "integrations/mcps" endpoint
# Should return list of server configs with URLs
return []
@staticmethod
def get_multimodal_tools() -> Sequence[BaseTool]:
def get_multimodal_tools(self) -> Sequence[BaseTool]:
from crewai.tools.agent_tools.add_image_tool import AddImageTool
return [AddImageTool()]
@@ -1230,9 +991,8 @@ class Agent(BaseAgent):
)
return []
@staticmethod
def get_output_converter(
llm: BaseLLM, text: str, model: type[BaseModel], instructions: str
self, llm: BaseLLM, text: str, model: type[BaseModel], instructions: str
) -> Converter:
return Converter(llm=llm, text=text, model=model, instructions=instructions)
@@ -1262,8 +1022,7 @@ class Agent(BaseAgent):
)
return task_prompt
@staticmethod
def _render_text_description(tools: list[Any]) -> str:
def _render_text_description(self, tools: list[Any]) -> str:
"""Render the tool name and description in plain text.
Output will be in the format of:
@@ -1348,15 +1107,6 @@ class Agent(BaseAgent):
def set_fingerprint(self, fingerprint: Fingerprint) -> None:
self.security_config.fingerprint = fingerprint
@property
def last_messages(self) -> list[LLMMessage]:
"""Get messages from the last task execution.
Returns:
List of LLM messages from the most recent task execution.
"""
return self._last_messages
def _get_knowledge_search_query(self, task_prompt: str, task: Task) -> str | None:
"""Generate a search query for the knowledge base based on the task description."""
crewai_event_bus.emit(

View File

@@ -1,5 +0,0 @@
from crewai.agent.core import Agent
from crewai.utilities.training_handler import CrewTrainingHandler
__all__ = ["Agent", "CrewTrainingHandler"]

View File

@@ -1,76 +0,0 @@
"""Generic metaclass for agent extensions.
This metaclass enables extension capabilities for agents by detecting
extension fields in class annotations and applying appropriate wrappers.
"""
import warnings
from functools import wraps
from typing import Any
from pydantic import model_validator
from pydantic._internal._model_construction import ModelMetaclass
class AgentMeta(ModelMetaclass):
"""Generic metaclass for agent extensions.
Detects extension fields (like 'a2a') in class annotations and applies
the appropriate wrapper logic to enable extension functionality.
"""
def __new__(
mcs,
name: str,
bases: tuple[type, ...],
namespace: dict[str, Any],
**kwargs: Any,
) -> type:
"""Create a new class with extension support.
Args:
name: The name of the class being created
bases: Base classes
namespace: Class namespace dictionary
**kwargs: Additional keyword arguments
Returns:
The newly created class with extension support if applicable
"""
orig_post_init_setup = namespace.get("post_init_setup")
if orig_post_init_setup is not None:
original_func = (
orig_post_init_setup.wrapped
if hasattr(orig_post_init_setup, "wrapped")
else orig_post_init_setup
)
def post_init_setup_with_extensions(self: Any) -> Any:
"""Wrap post_init_setup to apply extensions after initialization.
Args:
self: The agent instance
Returns:
The agent instance
"""
result = original_func(self)
a2a_value = getattr(self, "a2a", None)
if a2a_value is not None:
from crewai.a2a.wrapper import wrap_agent_with_a2a_instance
wrap_agent_with_a2a_instance(self)
return result
with warnings.catch_warnings():
warnings.filterwarnings(
"ignore", message=".*overrides an existing Pydantic.*"
)
namespace["post_init_setup"] = model_validator(mode="after")(
post_init_setup_with_extensions
)
return super().__new__(mcs, name, bases, namespace, **kwargs)

View File

@@ -7,7 +7,7 @@ output conversion for OpenAI agents, supporting JSON and Pydantic model formats.
from typing import Any
from crewai.agents.agent_adapters.base_converter_adapter import BaseConverterAdapter
from crewai.utilities.i18n import get_i18n
from crewai.utilities.i18n import I18N
class OpenAIConverterAdapter(BaseConverterAdapter):
@@ -59,7 +59,7 @@ class OpenAIConverterAdapter(BaseConverterAdapter):
return base_prompt
output_schema: str = (
get_i18n()
I18N()
.slice("formatted_task_instructions")
.format(output_format=self._schema)
)

View File

@@ -18,19 +18,17 @@ from pydantic import (
from pydantic_core import PydanticCustomError
from typing_extensions import Self
from crewai.agent.internal.meta import AgentMeta
from crewai.agents.agent_builder.utilities.base_token_process import TokenProcess
from crewai.agents.cache.cache_handler import CacheHandler
from crewai.agents.tools_handler import ToolsHandler
from crewai.knowledge.knowledge import Knowledge
from crewai.knowledge.knowledge_config import KnowledgeConfig
from crewai.knowledge.source.base_knowledge_source import BaseKnowledgeSource
from crewai.mcp.config import MCPServerConfig
from crewai.rag.embeddings.types import EmbedderConfig
from crewai.security.security_config import SecurityConfig
from crewai.tools.base_tool import BaseTool, Tool
from crewai.utilities.config import process_config
from crewai.utilities.i18n import I18N, get_i18n
from crewai.utilities.i18n import I18N
from crewai.utilities.logger import Logger
from crewai.utilities.rpm_controller import RPMController
from crewai.utilities.string_utils import interpolate_only
@@ -58,7 +56,7 @@ PlatformApp = Literal[
PlatformAppOrAction = PlatformApp | str
class BaseAgent(BaseModel, ABC, metaclass=AgentMeta):
class BaseAgent(BaseModel, ABC):
"""Abstract Base Class for all third party agents compatible with CrewAI.
Attributes:
@@ -108,7 +106,7 @@ class BaseAgent(BaseModel, ABC, metaclass=AgentMeta):
Set private attributes.
"""
__hash__ = object.__hash__
__hash__ = object.__hash__ # type: ignore
_logger: Logger = PrivateAttr(default_factory=lambda: Logger(verbose=False))
_rpm_controller: RPMController | None = PrivateAttr(default=None)
_request_within_rpm_limit: Any = PrivateAttr(default=None)
@@ -151,7 +149,7 @@ class BaseAgent(BaseModel, ABC, metaclass=AgentMeta):
)
crew: Any = Field(default=None, description="Crew to which the agent belongs.")
i18n: I18N = Field(
default_factory=get_i18n, description="Internationalization settings."
default_factory=I18N, description="Internationalization settings."
)
cache_handler: CacheHandler | None = Field(
default=None, description="An instance of the CacheHandler class."
@@ -181,8 +179,8 @@ class BaseAgent(BaseModel, ABC, metaclass=AgentMeta):
default_factory=SecurityConfig,
description="Security configuration for the agent, including fingerprinting.",
)
callbacks: list[Callable[[Any], Any]] = Field(
default_factory=list, description="Callbacks to be used for the agent"
callbacks: list[Callable] = Field(
default=[], description="Callbacks to be used for the agent"
)
adapted_agent: bool = Field(
default=False, description="Whether the agent is adapted"
@@ -195,14 +193,14 @@ class BaseAgent(BaseModel, ABC, metaclass=AgentMeta):
default=None,
description="List of applications or application/action combinations that the agent can access through CrewAI Platform. Can contain app names (e.g., 'gmail') or specific actions (e.g., 'gmail/send_email')",
)
mcps: list[str | MCPServerConfig] | None = Field(
mcps: list[str] | None = Field(
default=None,
description="List of MCP server references. Supports 'https://server.com/path' for external servers and 'crewai-amp:mcp-name' for AMP marketplace. Use '#tool_name' suffix for specific tools.",
)
@model_validator(mode="before")
@classmethod
def process_model_config(cls, values: Any) -> dict[str, Any]:
def process_model_config(cls, values):
return process_config(values, cls)
@field_validator("tools")
@@ -254,39 +252,23 @@ class BaseAgent(BaseModel, ABC, metaclass=AgentMeta):
@field_validator("mcps")
@classmethod
def validate_mcps(
cls, mcps: list[str | MCPServerConfig] | None
) -> list[str | MCPServerConfig] | None:
"""Validate MCP server references and configurations.
Supports both string references (for backwards compatibility) and
structured configuration objects (MCPServerStdio, MCPServerHTTP, MCPServerSSE).
"""
def validate_mcps(cls, mcps: list[str] | None) -> list[str] | None:
if not mcps:
return mcps
validated_mcps = []
for mcp in mcps:
if isinstance(mcp, str):
if mcp.startswith(("https://", "crewai-amp:")):
validated_mcps.append(mcp)
else:
raise ValueError(
f"Invalid MCP reference: {mcp}. "
"String references must start with 'https://' or 'crewai-amp:'"
)
elif isinstance(mcp, (MCPServerConfig)):
if mcp.startswith(("https://", "crewai-amp:")):
validated_mcps.append(mcp)
else:
raise ValueError(
f"Invalid MCP configuration: {type(mcp)}. "
"Must be a string reference or MCPServerConfig instance."
f"Invalid MCP reference: {mcp}. Must start with 'https://' or 'crewai-amp:'"
)
return validated_mcps
return list(set(validated_mcps))
@model_validator(mode="after")
def validate_and_set_attributes(self) -> Self:
def validate_and_set_attributes(self):
# Validate required fields
for field in ["role", "goal", "backstory"]:
if getattr(self, field) is None:
@@ -318,7 +300,7 @@ class BaseAgent(BaseModel, ABC, metaclass=AgentMeta):
)
@model_validator(mode="after")
def set_private_attrs(self) -> Self:
def set_private_attrs(self):
"""Set private attributes."""
self._logger = Logger(verbose=self.verbose)
if self.max_rpm and not self._rpm_controller:
@@ -330,7 +312,7 @@ class BaseAgent(BaseModel, ABC, metaclass=AgentMeta):
return self
@property
def key(self) -> str:
def key(self):
source = [
self._original_role or self.role,
self._original_goal or self.goal,
@@ -348,7 +330,7 @@ class BaseAgent(BaseModel, ABC, metaclass=AgentMeta):
pass
@abstractmethod
def create_agent_executor(self, tools: list[BaseTool] | None = None) -> None:
def create_agent_executor(self, tools=None) -> None:
pass
@abstractmethod
@@ -360,7 +342,7 @@ class BaseAgent(BaseModel, ABC, metaclass=AgentMeta):
"""Get platform tools for the specified list of applications and/or application/action combinations."""
@abstractmethod
def get_mcp_tools(self, mcps: list[str | MCPServerConfig]) -> list[BaseTool]:
def get_mcp_tools(self, mcps: list[str]) -> list[BaseTool]:
"""Get MCP tools for the specified list of MCP server references."""
def copy(self) -> Self: # type: ignore # Signature of "copy" incompatible with supertype "BaseModel"
@@ -460,5 +442,5 @@ class BaseAgent(BaseModel, ABC, metaclass=AgentMeta):
self._rpm_controller = rpm_controller
self.create_agent_executor()
def set_knowledge(self, crew_embedder: EmbedderConfig | None = None) -> None:
def set_knowledge(self, crew_embedder: EmbedderConfig | None = None):
pass

View File

@@ -9,7 +9,7 @@ from __future__ import annotations
from collections.abc import Callable
from typing import TYPE_CHECKING, Any, Literal, cast
from pydantic import BaseModel, GetCoreSchemaHandler
from pydantic import GetCoreSchemaHandler
from pydantic_core import CoreSchema, core_schema
from crewai.agents.agent_builder.base_agent_executor_mixin import CrewAgentExecutorMixin
@@ -37,11 +37,7 @@ from crewai.utilities.agent_utils import (
process_llm_response,
)
from crewai.utilities.constants import TRAINING_DATA_FILE
from crewai.utilities.i18n import I18N, get_i18n
from crewai.utilities.llm_call_hooks import (
get_after_llm_call_hooks,
get_before_llm_call_hooks,
)
from crewai.utilities.i18n import I18N
from crewai.utilities.printer import Printer
from crewai.utilities.tool_utils import execute_tool_and_check_finality
from crewai.utilities.training_handler import CrewTrainingHandler
@@ -69,7 +65,7 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
def __init__(
self,
llm: BaseLLM,
llm: BaseLLM | Any,
task: Task,
crew: Crew,
agent: Agent,
@@ -86,7 +82,6 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
respect_context_window: bool = False,
request_within_rpm_limit: Callable[[], bool] | None = None,
callbacks: list[Any] | None = None,
response_model: type[BaseModel] | None = None,
) -> None:
"""Initialize executor.
@@ -108,9 +103,8 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
respect_context_window: Respect context limits.
request_within_rpm_limit: RPM limit check function.
callbacks: Optional callbacks list.
response_model: Optional Pydantic model for structured outputs.
"""
self._i18n: I18N = get_i18n()
self._i18n: I18N = I18N()
self.llm = llm
self.task = task
self.agent = agent
@@ -125,38 +119,23 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
self.tools_handler = tools_handler
self.original_tools = original_tools or []
self.step_callback = step_callback
self.use_stop_words = self.llm.supports_stop_words()
self.tools_description = tools_description
self.function_calling_llm = function_calling_llm
self.respect_context_window = respect_context_window
self.request_within_rpm_limit = request_within_rpm_limit
self.response_model = response_model
self.ask_for_human_input = False
self.messages: list[LLMMessage] = []
self.iterations = 0
self.log_error_after = 3
self.before_llm_call_hooks: list[Callable] = []
self.after_llm_call_hooks: list[Callable] = []
self.before_llm_call_hooks.extend(get_before_llm_call_hooks())
self.after_llm_call_hooks.extend(get_after_llm_call_hooks())
if self.llm:
# This may be mutating the shared llm object and needs further evaluation
existing_stop = getattr(self.llm, "stop", [])
self.llm.stop = list(
set(
existing_stop + self.stop
if isinstance(existing_stop, list)
else self.stop
)
existing_stop = getattr(self.llm, "stop", [])
self.llm.stop = list(
set(
existing_stop + self.stop
if isinstance(existing_stop, list)
else self.stop
)
@property
def use_stop_words(self) -> bool:
"""Check to determine if stop words are being used.
Returns:
bool: True if tool should be used or not.
"""
return self.llm.supports_stop_words() if self.llm else False
)
def invoke(self, inputs: dict[str, Any]) -> dict[str, Any]:
"""Execute the agent with given inputs.
@@ -222,7 +201,6 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
llm=self.llm,
callbacks=self.callbacks,
)
break
enforce_rpm_limit(self.request_within_rpm_limit)
@@ -233,10 +211,8 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
printer=self._printer,
from_task=self.task,
from_agent=self.agent,
response_model=self.response_model,
executor_context=self,
)
formatted_answer = process_llm_response(answer, self.use_stop_words) # type: ignore[assignment]
formatted_answer = process_llm_response(answer, self.use_stop_words)
if isinstance(formatted_answer, AgentAction):
# Extract agent fingerprint if available
@@ -268,11 +244,11 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
formatted_answer, tool_result
)
self._invoke_step_callback(formatted_answer) # type: ignore[arg-type]
self._append_message(formatted_answer.text) # type: ignore[union-attr,attr-defined]
self._invoke_step_callback(formatted_answer)
self._append_message(formatted_answer.text)
except OutputParserError as e:
formatted_answer = handle_output_parser_exception( # type: ignore[assignment]
except OutputParserError as e: # noqa: PERF203
formatted_answer = handle_output_parser_exception(
e=e,
messages=self.messages,
iterations=self.iterations,

View File

@@ -18,10 +18,10 @@ from crewai.agents.constants import (
MISSING_ACTION_INPUT_AFTER_ACTION_ERROR_MESSAGE,
UNABLE_TO_REPAIR_JSON_RESULTS,
)
from crewai.utilities.i18n import get_i18n
from crewai.utilities.i18n import I18N
_I18N = get_i18n()
_I18N = I18N()
@dataclass

View File

@@ -3,17 +3,10 @@ import json
import os
from pathlib import Path
import sys
from typing import BinaryIO, cast
from cryptography.fernet import Fernet
if sys.platform == "win32":
import msvcrt
else:
import fcntl
class TokenManager:
def __init__(self, file_path: str = "tokens.enc") -> None:
"""
@@ -25,74 +18,21 @@ class TokenManager:
self.key = self._get_or_create_key()
self.fernet = Fernet(self.key)
@staticmethod
def _acquire_lock(file_handle: BinaryIO) -> None:
"""
Acquire an exclusive lock on a file handle.
Args:
file_handle: Open file handle to lock.
"""
if sys.platform == "win32":
msvcrt.locking(file_handle.fileno(), msvcrt.LK_LOCK, 1)
else:
fcntl.flock(file_handle.fileno(), fcntl.LOCK_EX)
@staticmethod
def _release_lock(file_handle: BinaryIO) -> None:
"""
Release the lock on a file handle.
Args:
file_handle: Open file handle to unlock.
"""
if sys.platform == "win32":
msvcrt.locking(file_handle.fileno(), msvcrt.LK_UNLCK, 1)
else:
fcntl.flock(file_handle.fileno(), fcntl.LOCK_UN)
def _get_or_create_key(self) -> bytes:
"""
Get or create the encryption key with file locking to prevent race conditions.
Get or create the encryption key.
Returns:
The encryption key.
:return: The encryption key.
"""
key_filename = "secret.key"
storage_path = self.get_secure_storage_path()
key = self.read_secure_file(key_filename)
if key is not None and len(key) == 44:
if key is not None:
return key
lock_file_path = storage_path / f"{key_filename}.lock"
try:
lock_file_path.touch()
with open(lock_file_path, "r+b") as lock_file:
self._acquire_lock(lock_file)
try:
key = self.read_secure_file(key_filename)
if key is not None and len(key) == 44:
return key
new_key = Fernet.generate_key()
self.save_secure_file(key_filename, new_key)
return new_key
finally:
try:
self._release_lock(lock_file)
except OSError:
pass
except OSError:
key = self.read_secure_file(key_filename)
if key is not None and len(key) == 44:
return key
new_key = Fernet.generate_key()
self.save_secure_file(key_filename, new_key)
return new_key
new_key = Fernet.generate_key()
self.save_secure_file(key_filename, new_key)
return new_key
def save_tokens(self, access_token: str, expires_at: int) -> None:
"""
@@ -119,14 +59,14 @@ class TokenManager:
if encrypted_data is None:
return None
decrypted_data = self.fernet.decrypt(encrypted_data)
decrypted_data = self.fernet.decrypt(encrypted_data) # type: ignore
data = json.loads(decrypted_data)
expiration = datetime.fromisoformat(data["expiration"])
if expiration <= datetime.now():
return None
return cast(str | None, data["access_token"])
return data["access_token"]
def clear_tokens(self) -> None:
"""
@@ -134,18 +74,20 @@ class TokenManager:
"""
self.delete_secure_file(self.file_path)
@staticmethod
def get_secure_storage_path() -> Path:
def get_secure_storage_path(self) -> Path:
"""
Get the secure storage path based on the operating system.
:return: The secure storage path.
"""
if sys.platform == "win32":
# Windows: Use %LOCALAPPDATA%
base_path = os.environ.get("LOCALAPPDATA")
elif sys.platform == "darwin":
# macOS: Use ~/Library/Application Support
base_path = os.path.expanduser("~/Library/Application Support")
else:
# Linux and other Unix-like: Use ~/.local/share
base_path = os.path.expanduser("~/.local/share")
app_name = "crewai/credentials"
@@ -168,6 +110,7 @@ class TokenManager:
with open(file_path, "wb") as f:
f.write(content)
# Set appropriate permissions (read/write for owner only)
os.chmod(file_path, 0o600)
def read_secure_file(self, filename: str) -> bytes | None:

View File

@@ -5,7 +5,7 @@ description = "{{name}} using crewAI"
authors = [{ name = "Your Name", email = "you@example.com" }]
requires-python = ">=3.10,<3.14"
dependencies = [
"crewai[tools]==1.4.1"
"crewai[tools]==1.2.1"
]
[project.scripts]

View File

@@ -5,7 +5,7 @@ description = "{{name}} using crewAI"
authors = [{ name = "Your Name", email = "you@example.com" }]
requires-python = ">=3.10,<3.14"
dependencies = [
"crewai[tools]==1.4.1"
"crewai[tools]==1.2.1"
]
[project.scripts]

View File

@@ -27,7 +27,6 @@ from pydantic import (
model_validator,
)
from pydantic_core import PydanticCustomError
from typing_extensions import Self
from crewai.agent import Agent
from crewai.agents.agent_builder.base_agent import BaseAgent
@@ -71,7 +70,7 @@ from crewai.task import Task
from crewai.tasks.conditional_task import ConditionalTask
from crewai.tasks.task_output import TaskOutput
from crewai.tools.agent_tools.agent_tools import AgentTools
from crewai.tools.base_tool import BaseTool
from crewai.tools.base_tool import BaseTool, Tool
from crewai.types.usage_metrics import UsageMetrics
from crewai.utilities.constants import NOT_SPECIFIED, TRAINING_DATA_FILE
from crewai.utilities.crew.models import CrewContext
@@ -82,7 +81,7 @@ from crewai.utilities.formatter import (
aggregate_raw_outputs_from_task_outputs,
aggregate_raw_outputs_from_tasks,
)
from crewai.utilities.i18n import get_i18n
from crewai.utilities.i18n import I18N
from crewai.utilities.llm_utils import create_llm
from crewai.utilities.logger import Logger
from crewai.utilities.planning_handler import CrewPlanner
@@ -196,7 +195,7 @@ class Crew(FlowTrackable, BaseModel):
function_calling_llm: str | InstanceOf[LLM] | Any | None = Field(
description="Language model that will run the agent.", default=None
)
config: Json[dict[str, Any]] | dict[str, Any] | None = Field(default=None)
config: Json | dict[str, Any] | None = Field(default=None)
id: UUID4 = Field(default_factory=uuid.uuid4, frozen=True)
share_crew: bool | None = Field(default=False)
step_callback: Any | None = Field(
@@ -295,9 +294,7 @@ class Crew(FlowTrackable, BaseModel):
@field_validator("config", mode="before")
@classmethod
def check_config_type(
cls, v: Json[dict[str, Any]] | dict[str, Any]
) -> dict[str, Any]:
def check_config_type(cls, v: Json | dict[str, Any]) -> Json | dict[str, Any]:
"""Validates that the config is a valid type.
Args:
v: The config to be validated.
@@ -313,7 +310,7 @@ class Crew(FlowTrackable, BaseModel):
"""set private attributes."""
self._cache_handler = CacheHandler()
event_listener = EventListener() # type: ignore[no-untyped-call]
event_listener = EventListener()
if (
is_tracing_enabled()
@@ -333,13 +330,13 @@ class Crew(FlowTrackable, BaseModel):
return self
def _initialize_default_memories(self) -> None:
self._long_term_memory = self._long_term_memory or LongTermMemory() # type: ignore[no-untyped-call]
self._short_term_memory = self._short_term_memory or ShortTermMemory( # type: ignore[no-untyped-call]
def _initialize_default_memories(self):
self._long_term_memory = self._long_term_memory or LongTermMemory()
self._short_term_memory = self._short_term_memory or ShortTermMemory(
crew=self,
embedder_config=self.embedder,
)
self._entity_memory = self.entity_memory or EntityMemory( # type: ignore[no-untyped-call]
self._entity_memory = self.entity_memory or EntityMemory(
crew=self, embedder_config=self.embedder
)
@@ -383,7 +380,7 @@ class Crew(FlowTrackable, BaseModel):
return self
@model_validator(mode="after")
def check_manager_llm(self) -> Self:
def check_manager_llm(self):
"""Validates that the language model is set when using hierarchical process."""
if self.process == Process.hierarchical:
if not self.manager_llm and not self.manager_agent:
@@ -408,7 +405,7 @@ class Crew(FlowTrackable, BaseModel):
return self
@model_validator(mode="after")
def check_config(self) -> Self:
def check_config(self):
"""Validates that the crew is properly configured with agents and tasks."""
if not self.config and not self.tasks and not self.agents:
raise PydanticCustomError(
@@ -429,20 +426,23 @@ class Crew(FlowTrackable, BaseModel):
return self
@model_validator(mode="after")
def validate_tasks(self) -> Self:
def validate_tasks(self):
if self.process == Process.sequential:
for task in self.tasks:
if task.agent is None:
raise PydanticCustomError(
"missing_agent_in_task",
"Sequential process error: Agent is missing in the task with the following description: {description}",
{"description": task.description},
(
f"Sequential process error: Agent is missing in the task "
f"with the following description: {task.description}"
), # type: ignore # Dynamic string in error message
{},
)
return self
@model_validator(mode="after")
def validate_end_with_at_most_one_async_task(self) -> Self:
def validate_end_with_at_most_one_async_task(self):
"""Validates that the crew ends with at most one asynchronous task."""
final_async_task_count = 0
@@ -505,9 +505,7 @@ class Crew(FlowTrackable, BaseModel):
return self
@model_validator(mode="after")
def validate_async_task_cannot_include_sequential_async_tasks_in_context(
self,
) -> Self:
def validate_async_task_cannot_include_sequential_async_tasks_in_context(self):
"""
Validates that if a task is set to be executed asynchronously,
it cannot include other asynchronous tasks in its context unless
@@ -529,7 +527,7 @@ class Crew(FlowTrackable, BaseModel):
return self
@model_validator(mode="after")
def validate_context_no_future_tasks(self) -> Self:
def validate_context_no_future_tasks(self):
"""Validates that a task's context does not include future tasks."""
task_indices = {id(task): i for i, task in enumerate(self.tasks)}
@@ -563,7 +561,7 @@ class Crew(FlowTrackable, BaseModel):
"""
return self.security_config.fingerprint
def _setup_from_config(self) -> None:
def _setup_from_config(self):
"""Initializes agents and tasks from the provided config."""
if self.config is None:
raise ValueError("Config should not be None.")
@@ -630,12 +628,12 @@ class Crew(FlowTrackable, BaseModel):
for agent in train_crew.agents:
if training_data.get(str(agent.id)):
result = TaskEvaluator(agent).evaluate_training_data( # type: ignore[arg-type]
result = TaskEvaluator(agent).evaluate_training_data(
training_data=training_data, agent_id=str(agent.id)
)
CrewTrainingHandler(filename).save_trained_data(
agent_id=str(agent.role),
trained_data=result.model_dump(),
trained_data=result.model_dump(), # type: ignore[arg-type]
)
crewai_event_bus.emit(
@@ -686,8 +684,12 @@ class Crew(FlowTrackable, BaseModel):
self._set_tasks_callbacks()
self._set_allow_crewai_trigger_context_for_first_task()
i18n = I18N(prompt_file=self.prompt_file)
for agent in self.agents:
agent.crew = self
agent.i18n = i18n
# type: ignore[attr-defined] # Argument 1 to "_interpolate_inputs" of "Crew" has incompatible type "dict[str, Any] | None"; expected "dict[str, Any]"
agent.crew = self # type: ignore[attr-defined]
agent.set_knowledge(crew_embedder=self.embedder)
# TODO: Create an AgentFunctionCalling protocol for future refactoring
if not agent.function_calling_llm: # type: ignore # "BaseAgent" has no attribute "function_calling_llm"
@@ -751,12 +753,10 @@ class Crew(FlowTrackable, BaseModel):
inputs = inputs or {}
return await asyncio.to_thread(self.kickoff, inputs)
async def kickoff_for_each_async(
self, inputs: list[dict[str, Any]]
) -> list[CrewOutput]:
async def kickoff_for_each_async(self, inputs: list[dict]) -> list[CrewOutput]:
crew_copies = [self.copy() for _ in inputs]
async def run_crew(crew: Self, input_data: Any) -> CrewOutput:
async def run_crew(crew, input_data):
return await crew.kickoff_async(inputs=input_data)
tasks = [
@@ -775,7 +775,7 @@ class Crew(FlowTrackable, BaseModel):
self._task_output_handler.reset()
return results
def _handle_crew_planning(self) -> None:
def _handle_crew_planning(self):
"""Handles the Crew planning."""
self._logger.log("info", "Planning the crew execution")
result = CrewPlanner(
@@ -793,7 +793,7 @@ class Crew(FlowTrackable, BaseModel):
output: TaskOutput,
task_index: int,
was_replayed: bool = False,
) -> None:
):
if self._inputs:
inputs = self._inputs
else:
@@ -809,7 +809,6 @@ class Crew(FlowTrackable, BaseModel):
"json_dict": output.json_dict,
"output_format": output.output_format,
"agent": output.agent,
"messages": output.messages,
},
"task_index": task_index,
"inputs": inputs,
@@ -826,21 +825,19 @@ class Crew(FlowTrackable, BaseModel):
self._create_manager_agent()
return self._execute_tasks(self.tasks)
def _create_manager_agent(self) -> None:
def _create_manager_agent(self):
i18n = I18N(prompt_file=self.prompt_file)
if self.manager_agent is not None:
self.manager_agent.allow_delegation = True
manager = self.manager_agent
if manager.tools is not None and len(manager.tools) > 0:
self._logger.log(
"warning",
"Manager agent should not have tools",
color="bold_yellow",
"warning", "Manager agent should not have tools", color="orange"
)
manager.tools = []
raise Exception("Manager agent should not have tools")
else:
self.manager_llm = create_llm(self.manager_llm)
i18n = get_i18n(prompt_file=self.prompt_file)
manager = Agent(
role=i18n.retrieve("hierarchical_manager_agent", "role"),
goal=i18n.retrieve("hierarchical_manager_agent", "goal"),
@@ -898,7 +895,7 @@ class Crew(FlowTrackable, BaseModel):
tools_for_task = self._prepare_tools(
agent_to_use,
task,
tools_for_task,
cast(list[Tool] | list[BaseTool], tools_for_task),
)
self._log_task_start(task, agent_to_use.role)
@@ -918,7 +915,7 @@ class Crew(FlowTrackable, BaseModel):
future = task.execute_async(
agent=agent_to_use,
context=context,
tools=tools_for_task,
tools=cast(list[BaseTool], tools_for_task),
)
futures.append((task, future, task_index))
else:
@@ -930,7 +927,7 @@ class Crew(FlowTrackable, BaseModel):
task_output = task.execute_sync(
agent=agent_to_use,
context=context,
tools=tools_for_task,
tools=cast(list[BaseTool], tools_for_task),
)
task_outputs.append(task_output)
self._process_task_result(task, task_output)
@@ -968,7 +965,7 @@ class Crew(FlowTrackable, BaseModel):
return None
def _prepare_tools(
self, agent: BaseAgent, task: Task, tools: list[BaseTool]
self, agent: BaseAgent, task: Task, tools: list[Tool] | list[BaseTool]
) -> list[BaseTool]:
# Add delegation tools if agent allows delegation
if hasattr(agent, "allow_delegation") and getattr(
@@ -1005,21 +1002,21 @@ class Crew(FlowTrackable, BaseModel):
tools = self._add_mcp_tools(task, tools)
# Return a list[BaseTool] compatible with Task.execute_sync and execute_async
return tools
return cast(list[BaseTool], tools)
def _get_agent_to_use(self, task: Task) -> BaseAgent | None:
if self.process == Process.hierarchical:
return self.manager_agent
return task.agent
@staticmethod
def _merge_tools(
existing_tools: list[BaseTool],
new_tools: list[BaseTool],
self,
existing_tools: list[Tool] | list[BaseTool],
new_tools: list[Tool] | list[BaseTool],
) -> list[BaseTool]:
"""Merge new tools into existing tools list, avoiding duplicates."""
if not new_tools:
return existing_tools
return cast(list[BaseTool], existing_tools)
# Create mapping of tool names to new tools
new_tool_map = {tool.name: tool for tool in new_tools}
@@ -1030,62 +1027,63 @@ class Crew(FlowTrackable, BaseModel):
# Add all new tools
tools.extend(new_tools)
return tools
return cast(list[BaseTool], tools)
def _inject_delegation_tools(
self,
tools: list[BaseTool],
tools: list[Tool] | list[BaseTool],
task_agent: BaseAgent,
agents: list[BaseAgent],
) -> list[BaseTool]:
if hasattr(task_agent, "get_delegation_tools"):
delegation_tools = task_agent.get_delegation_tools(agents)
# Cast delegation_tools to the expected type for _merge_tools
return self._merge_tools(tools, delegation_tools)
return tools
return self._merge_tools(tools, cast(list[BaseTool], delegation_tools))
return cast(list[BaseTool], tools)
def _inject_platform_tools(
self,
tools: list[BaseTool],
tools: list[Tool] | list[BaseTool],
task_agent: BaseAgent,
) -> list[BaseTool]:
apps = getattr(task_agent, "apps", None) or []
if hasattr(task_agent, "get_platform_tools") and apps:
platform_tools = task_agent.get_platform_tools(apps=apps)
return self._merge_tools(tools, platform_tools)
return tools
return self._merge_tools(tools, cast(list[BaseTool], platform_tools))
return cast(list[BaseTool], tools)
def _inject_mcp_tools(
self,
tools: list[BaseTool],
tools: list[Tool] | list[BaseTool],
task_agent: BaseAgent,
) -> list[BaseTool]:
mcps = getattr(task_agent, "mcps", None) or []
if hasattr(task_agent, "get_mcp_tools") and mcps:
mcp_tools = task_agent.get_mcp_tools(mcps=mcps)
return self._merge_tools(tools, mcp_tools)
return tools
return self._merge_tools(tools, cast(list[BaseTool], mcp_tools))
return cast(list[BaseTool], tools)
def _add_multimodal_tools(
self, agent: BaseAgent, tools: list[BaseTool]
self, agent: BaseAgent, tools: list[Tool] | list[BaseTool]
) -> list[BaseTool]:
if hasattr(agent, "get_multimodal_tools"):
multimodal_tools = agent.get_multimodal_tools()
# Cast multimodal_tools to the expected type for _merge_tools
return self._merge_tools(tools, cast(list[BaseTool], multimodal_tools))
return tools
return cast(list[BaseTool], tools)
def _add_code_execution_tools(
self, agent: BaseAgent, tools: list[BaseTool]
self, agent: BaseAgent, tools: list[Tool] | list[BaseTool]
) -> list[BaseTool]:
if hasattr(agent, "get_code_execution_tools"):
code_tools = agent.get_code_execution_tools()
# Cast code_tools to the expected type for _merge_tools
return self._merge_tools(tools, cast(list[BaseTool], code_tools))
return tools
return cast(list[BaseTool], tools)
def _add_delegation_tools(
self, task: Task, tools: list[BaseTool]
self, task: Task, tools: list[Tool] | list[BaseTool]
) -> list[BaseTool]:
agents_for_delegation = [agent for agent in self.agents if agent != task.agent]
if len(self.agents) > 1 and len(agents_for_delegation) > 0 and task.agent:
@@ -1094,21 +1092,25 @@ class Crew(FlowTrackable, BaseModel):
tools = self._inject_delegation_tools(
tools, task.agent, agents_for_delegation
)
return tools
return cast(list[BaseTool], tools)
def _add_platform_tools(self, task: Task, tools: list[BaseTool]) -> list[BaseTool]:
def _add_platform_tools(
self, task: Task, tools: list[Tool] | list[BaseTool]
) -> list[BaseTool]:
if task.agent:
tools = self._inject_platform_tools(tools, task.agent)
return tools or []
return cast(list[BaseTool], tools or [])
def _add_mcp_tools(self, task: Task, tools: list[BaseTool]) -> list[BaseTool]:
def _add_mcp_tools(
self, task: Task, tools: list[Tool] | list[BaseTool]
) -> list[BaseTool]:
if task.agent:
tools = self._inject_mcp_tools(tools, task.agent)
return tools or []
return cast(list[BaseTool], tools or [])
def _log_task_start(self, task: Task, role: str = "None") -> None:
def _log_task_start(self, task: Task, role: str = "None"):
if self.output_log_file:
self._file_handler.log(
task_name=task.name, # type: ignore[arg-type]
@@ -1118,7 +1120,7 @@ class Crew(FlowTrackable, BaseModel):
)
def _update_manager_tools(
self, task: Task, tools: list[BaseTool]
self, task: Task, tools: list[Tool] | list[BaseTool]
) -> list[BaseTool]:
if self.manager_agent:
if task.agent:
@@ -1127,7 +1129,7 @@ class Crew(FlowTrackable, BaseModel):
tools = self._inject_delegation_tools(
tools, self.manager_agent, self.agents
)
return tools
return cast(list[BaseTool], tools)
def _get_context(self, task: Task, task_outputs: list[TaskOutput]) -> str:
if not task.context:
@@ -1237,7 +1239,6 @@ class Crew(FlowTrackable, BaseModel):
pydantic=stored_output["pydantic"],
json_dict=stored_output["json_dict"],
output_format=stored_output["output_format"],
messages=stored_output.get("messages", []),
)
self.tasks[i].output = task_output
@@ -1279,7 +1280,7 @@ class Crew(FlowTrackable, BaseModel):
return required_inputs
def copy(self) -> Crew: # type: ignore[override]
def copy(self):
"""
Creates a deep copy of the Crew instance.
@@ -1310,7 +1311,7 @@ class Crew(FlowTrackable, BaseModel):
manager_agent = self.manager_agent.copy() if self.manager_agent else None
manager_llm = shallow_copy(self.manager_llm) if self.manager_llm else None
task_mapping: dict[str, Any] = {}
task_mapping = {}
cloned_tasks = []
existing_knowledge_sources = shallow_copy(self.knowledge_sources)
@@ -1372,6 +1373,7 @@ class Crew(FlowTrackable, BaseModel):
)
for task in self.tasks
]
# type: ignore # "interpolate_inputs" of "Agent" does not return a value (it only ever returns None)
for agent in self.agents:
agent.interpolate_inputs(inputs)
@@ -1461,7 +1463,7 @@ class Crew(FlowTrackable, BaseModel):
)
raise
def __repr__(self) -> str:
def __repr__(self):
return (
f"Crew(id={self.id}, process={self.process}, "
f"number_of_agents={len(self.agents)}, "
@@ -1518,9 +1520,7 @@ class Crew(FlowTrackable, BaseModel):
if (system := config.get("system")) is not None:
name = config.get("name")
try:
reset_fn: Callable[[Any], Any] = cast(
Callable[[Any], Any], config.get("reset")
)
reset_fn: Callable = cast(Callable, config.get("reset"))
reset_fn(system)
self._logger.log(
"info",
@@ -1551,9 +1551,7 @@ class Crew(FlowTrackable, BaseModel):
raise RuntimeError(f"{name} memory system is not initialized")
try:
reset_fn: Callable[[Any], Any] = cast(
Callable[[Any], Any], config.get("reset")
)
reset_fn: Callable = cast(Callable, config.get("reset"))
reset_fn(system)
self._logger.log(
"info",
@@ -1566,7 +1564,7 @@ class Crew(FlowTrackable, BaseModel):
f"Failed to reset {name} memory: {e!s}"
) from e
def _get_memory_systems(self) -> dict[str, Any]:
def _get_memory_systems(self):
"""Get all available memory systems with their configuration.
Returns:
@@ -1574,10 +1572,10 @@ class Crew(FlowTrackable, BaseModel):
display names.
"""
def default_reset(memory: Any) -> Any:
def default_reset(memory):
return memory.reset()
def knowledge_reset(memory: Any) -> Any:
def knowledge_reset(memory):
return self.reset_knowledge(memory)
# Get knowledge for agents
@@ -1637,7 +1635,7 @@ class Crew(FlowTrackable, BaseModel):
for ks in knowledges:
ks.reset()
def _set_allow_crewai_trigger_context_for_first_task(self) -> None:
def _set_allow_crewai_trigger_context_for_first_task(self):
crewai_trigger_payload = self._inputs and self._inputs.get(
"crewai_trigger_payload"
)

View File

@@ -8,14 +8,21 @@ This module provides the event infrastructure that allows users to:
- Declare handler dependencies for ordered execution
"""
from __future__ import annotations
from typing import TYPE_CHECKING
from crewai.events.base_event_listener import BaseEventListener
from crewai.events.depends import Depends
from crewai.events.event_bus import crewai_event_bus
from crewai.events.handler_graph import CircularDependencyError
from crewai.events.types.agent_events import (
AgentEvaluationCompletedEvent,
AgentEvaluationFailedEvent,
AgentEvaluationStartedEvent,
AgentExecutionCompletedEvent,
AgentExecutionErrorEvent,
AgentExecutionStartedEvent,
LiteAgentExecutionCompletedEvent,
LiteAgentExecutionErrorEvent,
LiteAgentExecutionStartedEvent,
)
from crewai.events.types.crew_events import (
CrewKickoffCompletedEvent,
CrewKickoffFailedEvent,
@@ -60,14 +67,6 @@ from crewai.events.types.logging_events import (
AgentLogsExecutionEvent,
AgentLogsStartedEvent,
)
from crewai.events.types.mcp_events import (
MCPConnectionCompletedEvent,
MCPConnectionFailedEvent,
MCPConnectionStartedEvent,
MCPToolExecutionCompletedEvent,
MCPToolExecutionFailedEvent,
MCPToolExecutionStartedEvent,
)
from crewai.events.types.memory_events import (
MemoryQueryCompletedEvent,
MemoryQueryFailedEvent,
@@ -101,20 +100,6 @@ from crewai.events.types.tool_usage_events import (
)
if TYPE_CHECKING:
from crewai.events.types.agent_events import (
AgentEvaluationCompletedEvent,
AgentEvaluationFailedEvent,
AgentEvaluationStartedEvent,
AgentExecutionCompletedEvent,
AgentExecutionErrorEvent,
AgentExecutionStartedEvent,
LiteAgentExecutionCompletedEvent,
LiteAgentExecutionErrorEvent,
LiteAgentExecutionStartedEvent,
)
__all__ = [
"AgentEvaluationCompletedEvent",
"AgentEvaluationFailedEvent",
@@ -160,12 +145,6 @@ __all__ = [
"LiteAgentExecutionCompletedEvent",
"LiteAgentExecutionErrorEvent",
"LiteAgentExecutionStartedEvent",
"MCPConnectionCompletedEvent",
"MCPConnectionFailedEvent",
"MCPConnectionStartedEvent",
"MCPToolExecutionCompletedEvent",
"MCPToolExecutionFailedEvent",
"MCPToolExecutionStartedEvent",
"MemoryQueryCompletedEvent",
"MemoryQueryFailedEvent",
"MemoryQueryStartedEvent",
@@ -191,27 +170,3 @@ __all__ = [
"ToolValidateInputErrorEvent",
"crewai_event_bus",
]
_AGENT_EVENT_MAPPING = {
"AgentEvaluationCompletedEvent": "crewai.events.types.agent_events",
"AgentEvaluationFailedEvent": "crewai.events.types.agent_events",
"AgentEvaluationStartedEvent": "crewai.events.types.agent_events",
"AgentExecutionCompletedEvent": "crewai.events.types.agent_events",
"AgentExecutionErrorEvent": "crewai.events.types.agent_events",
"AgentExecutionStartedEvent": "crewai.events.types.agent_events",
"LiteAgentExecutionCompletedEvent": "crewai.events.types.agent_events",
"LiteAgentExecutionErrorEvent": "crewai.events.types.agent_events",
"LiteAgentExecutionStartedEvent": "crewai.events.types.agent_events",
}
def __getattr__(name: str):
"""Lazy import for agent events to avoid circular imports."""
if name in _AGENT_EVENT_MAPPING:
import importlib
module_path = _AGENT_EVENT_MAPPING[name]
module = importlib.import_module(module_path)
return getattr(module, name)
msg = f"module {__name__!r} has no attribute {name!r}"
raise AttributeError(msg)

View File

@@ -1,26 +1,16 @@
"""Base event listener for CrewAI event system."""
from abc import ABC, abstractmethod
from crewai.events.event_bus import CrewAIEventsBus, crewai_event_bus
class BaseEventListener(ABC):
"""Abstract base class for event listeners."""
verbose: bool = False
def __init__(self) -> None:
"""Initialize the event listener and register handlers."""
def __init__(self):
super().__init__()
self.setup_listeners(crewai_event_bus)
crewai_event_bus.validate_dependencies()
@abstractmethod
def setup_listeners(self, crewai_event_bus: CrewAIEventsBus) -> None:
"""Setup event listeners on the event bus.
Args:
crewai_event_bus: The event bus to register listeners on.
"""
def setup_listeners(self, crewai_event_bus: CrewAIEventsBus):
pass

View File

@@ -1,21 +1,12 @@
from __future__ import annotations
from io import StringIO
import threading
from typing import TYPE_CHECKING, Any
from typing import Any
from pydantic import Field, PrivateAttr
from crewai.events.base_event_listener import BaseEventListener
from crewai.events.listeners.tracing.trace_listener import TraceCollectionListener
from crewai.events.types.a2a_events import (
A2AConversationCompletedEvent,
A2AConversationStartedEvent,
A2ADelegationCompletedEvent,
A2ADelegationStartedEvent,
A2AMessageSentEvent,
A2AResponseReceivedEvent,
)
from crewai.events.listeners.memory_listener import MemoryListener
from crewai.events.types.agent_events import (
AgentExecutionCompletedEvent,
AgentExecutionStartedEvent,
@@ -65,14 +56,6 @@ from crewai.events.types.logging_events import (
AgentLogsExecutionEvent,
AgentLogsStartedEvent,
)
from crewai.events.types.mcp_events import (
MCPConnectionCompletedEvent,
MCPConnectionFailedEvent,
MCPConnectionStartedEvent,
MCPToolExecutionCompletedEvent,
MCPToolExecutionFailedEvent,
MCPToolExecutionStartedEvent,
)
from crewai.events.types.reasoning_events import (
AgentReasoningCompletedEvent,
AgentReasoningFailedEvent,
@@ -96,10 +79,6 @@ from crewai.utilities import Logger
from crewai.utilities.constants import EMITTER_COLOR
if TYPE_CHECKING:
from crewai.events.event_bus import CrewAIEventsBus
class EventListener(BaseEventListener):
_instance = None
_telemetry: Telemetry = PrivateAttr(default_factory=lambda: Telemetry())
@@ -109,7 +88,6 @@ class EventListener(BaseEventListener):
text_stream = StringIO()
knowledge_retrieval_in_progress = False
knowledge_query_in_progress = False
method_branches: dict[str, Any] = Field(default_factory=dict)
def __new__(cls):
if cls._instance is None:
@@ -123,27 +101,21 @@ class EventListener(BaseEventListener):
self._telemetry = Telemetry()
self._telemetry.set_tracer()
self.execution_spans = {}
self.method_branches = {}
self._initialized = True
self.formatter = ConsoleFormatter(verbose=True)
self._crew_tree_lock = threading.Condition()
# Initialize trace listener with formatter for memory event handling
trace_listener = TraceCollectionListener()
trace_listener.formatter = self.formatter
MemoryListener(formatter=self.formatter)
# ----------- CREW EVENTS -----------
def setup_listeners(self, crewai_event_bus: CrewAIEventsBus) -> None:
def setup_listeners(self, crewai_event_bus):
@crewai_event_bus.on(CrewKickoffStartedEvent)
def on_crew_started(source, event: CrewKickoffStartedEvent) -> None:
with self._crew_tree_lock:
self.formatter.create_crew_tree(event.crew_name or "Crew", source.id)
self._telemetry.crew_execution_span(source, event.inputs)
self._crew_tree_lock.notify_all()
def on_crew_started(source, event: CrewKickoffStartedEvent):
self.formatter.create_crew_tree(event.crew_name or "Crew", source.id)
self._telemetry.crew_execution_span(source, event.inputs)
@crewai_event_bus.on(CrewKickoffCompletedEvent)
def on_crew_completed(source, event: CrewKickoffCompletedEvent) -> None:
def on_crew_completed(source, event: CrewKickoffCompletedEvent):
# Handle telemetry
final_string_output = event.output.raw
self._telemetry.end_crew(source, final_string_output)
@@ -157,7 +129,7 @@ class EventListener(BaseEventListener):
)
@crewai_event_bus.on(CrewKickoffFailedEvent)
def on_crew_failed(source, event: CrewKickoffFailedEvent) -> None:
def on_crew_failed(source, event: CrewKickoffFailedEvent):
self.formatter.update_crew_tree(
self.formatter.current_crew_tree,
event.crew_name or "Crew",
@@ -166,23 +138,23 @@ class EventListener(BaseEventListener):
)
@crewai_event_bus.on(CrewTrainStartedEvent)
def on_crew_train_started(source, event: CrewTrainStartedEvent) -> None:
def on_crew_train_started(source, event: CrewTrainStartedEvent):
self.formatter.handle_crew_train_started(
event.crew_name or "Crew", str(event.timestamp)
)
@crewai_event_bus.on(CrewTrainCompletedEvent)
def on_crew_train_completed(source, event: CrewTrainCompletedEvent) -> None:
def on_crew_train_completed(source, event: CrewTrainCompletedEvent):
self.formatter.handle_crew_train_completed(
event.crew_name or "Crew", str(event.timestamp)
)
@crewai_event_bus.on(CrewTrainFailedEvent)
def on_crew_train_failed(source, event: CrewTrainFailedEvent) -> None:
def on_crew_train_failed(source, event: CrewTrainFailedEvent):
self.formatter.handle_crew_train_failed(event.crew_name or "Crew")
@crewai_event_bus.on(CrewTestResultEvent)
def on_crew_test_result(source, event: CrewTestResultEvent) -> None:
def on_crew_test_result(source, event: CrewTestResultEvent):
self._telemetry.individual_test_result_span(
source.crew,
event.quality,
@@ -193,22 +165,14 @@ class EventListener(BaseEventListener):
# ----------- TASK EVENTS -----------
@crewai_event_bus.on(TaskStartedEvent)
def on_task_started(source, event: TaskStartedEvent) -> None:
def on_task_started(source, event: TaskStartedEvent):
span = self._telemetry.task_started(crew=source.agent.crew, task=source)
self.execution_spans[source] = span
with self._crew_tree_lock:
self._crew_tree_lock.wait_for(
lambda: self.formatter.current_crew_tree is not None, timeout=5.0
)
if self.formatter.current_crew_tree is not None:
task_name = (
source.name if hasattr(source, "name") and source.name else None
)
self.formatter.create_task_branch(
self.formatter.current_crew_tree, source.id, task_name
)
# Pass both task ID and task name (if set)
task_name = source.name if hasattr(source, "name") and source.name else None
self.formatter.create_task_branch(
self.formatter.current_crew_tree, source.id, task_name
)
@crewai_event_bus.on(TaskCompletedEvent)
def on_task_completed(source, event: TaskCompletedEvent):
@@ -299,8 +263,7 @@ class EventListener(BaseEventListener):
@crewai_event_bus.on(FlowCreatedEvent)
def on_flow_created(source, event: FlowCreatedEvent):
self._telemetry.flow_creation_span(event.flow_name)
tree = self.formatter.create_flow_tree(event.flow_name, str(source.flow_id))
self.formatter.current_flow_tree = tree
self.formatter.create_flow_tree(event.flow_name, str(source.flow_id))
@crewai_event_bus.on(FlowStartedEvent)
def on_flow_started(source, event: FlowStartedEvent):
@@ -317,36 +280,30 @@ class EventListener(BaseEventListener):
@crewai_event_bus.on(MethodExecutionStartedEvent)
def on_method_execution_started(source, event: MethodExecutionStartedEvent):
method_branch = self.method_branches.get(event.method_name)
updated_branch = self.formatter.update_method_status(
method_branch,
self.formatter.update_method_status(
self.formatter.current_method_branch,
self.formatter.current_flow_tree,
event.method_name,
"running",
)
self.method_branches[event.method_name] = updated_branch
@crewai_event_bus.on(MethodExecutionFinishedEvent)
def on_method_execution_finished(source, event: MethodExecutionFinishedEvent):
method_branch = self.method_branches.get(event.method_name)
updated_branch = self.formatter.update_method_status(
method_branch,
self.formatter.update_method_status(
self.formatter.current_method_branch,
self.formatter.current_flow_tree,
event.method_name,
"completed",
)
self.method_branches[event.method_name] = updated_branch
@crewai_event_bus.on(MethodExecutionFailedEvent)
def on_method_execution_failed(source, event: MethodExecutionFailedEvent):
method_branch = self.method_branches.get(event.method_name)
updated_branch = self.formatter.update_method_status(
method_branch,
self.formatter.update_method_status(
self.formatter.current_method_branch,
self.formatter.current_flow_tree,
event.method_name,
"failed",
)
self.method_branches[event.method_name] = updated_branch
# ----------- TOOL USAGE EVENTS -----------
@@ -567,123 +524,5 @@ class EventListener(BaseEventListener):
event.verbose,
)
@crewai_event_bus.on(A2ADelegationStartedEvent)
def on_a2a_delegation_started(source, event: A2ADelegationStartedEvent):
self.formatter.handle_a2a_delegation_started(
event.endpoint,
event.task_description,
event.agent_id,
event.is_multiturn,
event.turn_number,
)
@crewai_event_bus.on(A2ADelegationCompletedEvent)
def on_a2a_delegation_completed(source, event: A2ADelegationCompletedEvent):
self.formatter.handle_a2a_delegation_completed(
event.status,
event.result,
event.error,
event.is_multiturn,
)
@crewai_event_bus.on(A2AConversationStartedEvent)
def on_a2a_conversation_started(source, event: A2AConversationStartedEvent):
# Store A2A agent name for display in conversation tree
if event.a2a_agent_name:
self.formatter._current_a2a_agent_name = event.a2a_agent_name
self.formatter.handle_a2a_conversation_started(
event.agent_id,
event.endpoint,
)
@crewai_event_bus.on(A2AMessageSentEvent)
def on_a2a_message_sent(source, event: A2AMessageSentEvent):
self.formatter.handle_a2a_message_sent(
event.message,
event.turn_number,
event.agent_role,
)
@crewai_event_bus.on(A2AResponseReceivedEvent)
def on_a2a_response_received(source, event: A2AResponseReceivedEvent):
self.formatter.handle_a2a_response_received(
event.response,
event.turn_number,
event.status,
event.agent_role,
)
@crewai_event_bus.on(A2AConversationCompletedEvent)
def on_a2a_conversation_completed(source, event: A2AConversationCompletedEvent):
self.formatter.handle_a2a_conversation_completed(
event.status,
event.final_result,
event.error,
event.total_turns,
)
# ----------- MCP EVENTS -----------
@crewai_event_bus.on(MCPConnectionStartedEvent)
def on_mcp_connection_started(source, event: MCPConnectionStartedEvent):
self.formatter.handle_mcp_connection_started(
event.server_name,
event.server_url,
event.transport_type,
event.is_reconnect,
event.connect_timeout,
)
@crewai_event_bus.on(MCPConnectionCompletedEvent)
def on_mcp_connection_completed(source, event: MCPConnectionCompletedEvent):
self.formatter.handle_mcp_connection_completed(
event.server_name,
event.server_url,
event.transport_type,
event.connection_duration_ms,
event.is_reconnect,
)
@crewai_event_bus.on(MCPConnectionFailedEvent)
def on_mcp_connection_failed(source, event: MCPConnectionFailedEvent):
self.formatter.handle_mcp_connection_failed(
event.server_name,
event.server_url,
event.transport_type,
event.error,
event.error_type,
)
@crewai_event_bus.on(MCPToolExecutionStartedEvent)
def on_mcp_tool_execution_started(source, event: MCPToolExecutionStartedEvent):
self.formatter.handle_mcp_tool_execution_started(
event.server_name,
event.tool_name,
event.tool_args,
)
@crewai_event_bus.on(MCPToolExecutionCompletedEvent)
def on_mcp_tool_execution_completed(
source, event: MCPToolExecutionCompletedEvent
):
self.formatter.handle_mcp_tool_execution_completed(
event.server_name,
event.tool_name,
event.tool_args,
event.result,
event.execution_duration_ms,
)
@crewai_event_bus.on(MCPToolExecutionFailedEvent)
def on_mcp_tool_execution_failed(source, event: MCPToolExecutionFailedEvent):
self.formatter.handle_mcp_tool_execution_failed(
event.server_name,
event.tool_name,
event.tool_args,
event.error,
event.error_type,
)
event_listener = EventListener()

View File

@@ -40,14 +40,6 @@ from crewai.events.types.llm_guardrail_events import (
LLMGuardrailCompletedEvent,
LLMGuardrailStartedEvent,
)
from crewai.events.types.mcp_events import (
MCPConnectionCompletedEvent,
MCPConnectionFailedEvent,
MCPConnectionStartedEvent,
MCPToolExecutionCompletedEvent,
MCPToolExecutionFailedEvent,
MCPToolExecutionStartedEvent,
)
from crewai.events.types.memory_events import (
MemoryQueryCompletedEvent,
MemoryQueryFailedEvent,
@@ -123,10 +115,4 @@ EventTypes = (
| MemoryQueryFailedEvent
| MemoryRetrievalStartedEvent
| MemoryRetrievalCompletedEvent
| MCPConnectionStartedEvent
| MCPConnectionCompletedEvent
| MCPConnectionFailedEvent
| MCPToolExecutionStartedEvent
| MCPToolExecutionCompletedEvent
| MCPToolExecutionFailedEvent
)

View File

@@ -0,0 +1,106 @@
from crewai.events.base_event_listener import BaseEventListener
from crewai.events.types.memory_events import (
MemoryQueryCompletedEvent,
MemoryQueryFailedEvent,
MemoryRetrievalCompletedEvent,
MemoryRetrievalStartedEvent,
MemorySaveCompletedEvent,
MemorySaveFailedEvent,
MemorySaveStartedEvent,
)
class MemoryListener(BaseEventListener):
def __init__(self, formatter):
super().__init__()
self.formatter = formatter
self.memory_retrieval_in_progress = False
self.memory_save_in_progress = False
def setup_listeners(self, crewai_event_bus):
@crewai_event_bus.on(MemoryRetrievalStartedEvent)
def on_memory_retrieval_started(source, event: MemoryRetrievalStartedEvent):
if self.memory_retrieval_in_progress:
return
self.memory_retrieval_in_progress = True
self.formatter.handle_memory_retrieval_started(
self.formatter.current_agent_branch,
self.formatter.current_crew_tree,
)
@crewai_event_bus.on(MemoryRetrievalCompletedEvent)
def on_memory_retrieval_completed(source, event: MemoryRetrievalCompletedEvent):
if not self.memory_retrieval_in_progress:
return
self.memory_retrieval_in_progress = False
self.formatter.handle_memory_retrieval_completed(
self.formatter.current_agent_branch,
self.formatter.current_crew_tree,
event.memory_content,
event.retrieval_time_ms,
)
@crewai_event_bus.on(MemoryQueryCompletedEvent)
def on_memory_query_completed(source, event: MemoryQueryCompletedEvent):
if not self.memory_retrieval_in_progress:
return
self.formatter.handle_memory_query_completed(
self.formatter.current_agent_branch,
event.source_type,
event.query_time_ms,
self.formatter.current_crew_tree,
)
@crewai_event_bus.on(MemoryQueryFailedEvent)
def on_memory_query_failed(source, event: MemoryQueryFailedEvent):
if not self.memory_retrieval_in_progress:
return
self.formatter.handle_memory_query_failed(
self.formatter.current_agent_branch,
self.formatter.current_crew_tree,
event.error,
event.source_type,
)
@crewai_event_bus.on(MemorySaveStartedEvent)
def on_memory_save_started(source, event: MemorySaveStartedEvent):
if self.memory_save_in_progress:
return
self.memory_save_in_progress = True
self.formatter.handle_memory_save_started(
self.formatter.current_agent_branch,
self.formatter.current_crew_tree,
)
@crewai_event_bus.on(MemorySaveCompletedEvent)
def on_memory_save_completed(source, event: MemorySaveCompletedEvent):
if not self.memory_save_in_progress:
return
self.memory_save_in_progress = False
self.formatter.handle_memory_save_completed(
self.formatter.current_agent_branch,
self.formatter.current_crew_tree,
event.save_time_ms,
event.source_type,
)
@crewai_event_bus.on(MemorySaveFailedEvent)
def on_memory_save_failed(source, event: MemorySaveFailedEvent):
if not self.memory_save_in_progress:
return
self.formatter.handle_memory_save_failed(
self.formatter.current_agent_branch,
event.error,
event.source_type,
self.formatter.current_crew_tree,
)

View File

@@ -73,19 +73,15 @@ class FirstTimeTraceHandler:
self.is_first_time = should_auto_collect_first_time_traces()
return self.is_first_time
def set_batch_manager(self, batch_manager: TraceBatchManager) -> None:
"""Set reference to batch manager for sending events.
Args:
batch_manager: The trace batch manager instance.
"""
def set_batch_manager(self, batch_manager: TraceBatchManager):
"""Set reference to batch manager for sending events."""
self.batch_manager = batch_manager
def mark_events_collected(self) -> None:
def mark_events_collected(self):
"""Mark that events have been collected during execution."""
self.collected_events = True
def handle_execution_completion(self) -> None:
def handle_execution_completion(self):
"""Handle the completion flow as shown in your diagram."""
if not self.is_first_time or not self.collected_events:
return

View File

@@ -44,7 +44,6 @@ class TraceBatchManager:
def __init__(self) -> None:
self._init_lock = Lock()
self._batch_ready_cv = Condition(self._init_lock)
self._pending_events_lock = Lock()
self._pending_events_cv = Condition(self._pending_events_lock)
self._pending_events_count = 0
@@ -95,8 +94,6 @@ class TraceBatchManager:
)
self.backend_initialized = True
self._batch_ready_cv.notify_all()
return self.current_batch
def _initialize_backend_batch(
@@ -164,13 +161,13 @@ class TraceBatchManager:
f"Error initializing trace batch: {e}. Continuing without tracing."
)
def begin_event_processing(self) -> None:
"""Mark that an event handler started processing (for synchronization)."""
def begin_event_processing(self):
"""Mark that an event handler started processing (for synchronization)"""
with self._pending_events_lock:
self._pending_events_count += 1
def end_event_processing(self) -> None:
"""Mark that an event handler finished processing (for synchronization)."""
def end_event_processing(self):
"""Mark that an event handler finished processing (for synchronization)"""
with self._pending_events_cv:
self._pending_events_count -= 1
if self._pending_events_count == 0:
@@ -388,22 +385,6 @@ class TraceBatchManager:
"""Check if batch is initialized"""
return self.current_batch is not None
def wait_for_batch_initialization(self, timeout: float = 2.0) -> bool:
"""Wait for batch to be initialized.
Args:
timeout: Maximum time to wait in seconds (default: 2.0)
Returns:
True if batch was initialized, False if timeout occurred
"""
with self._batch_ready_cv:
if self.current_batch is not None:
return True
return self._batch_ready_cv.wait_for(
lambda: self.current_batch is not None, timeout=timeout
)
def record_start_time(self, key: str):
"""Record start time for duration calculation"""
self.execution_start_times[key] = datetime.now(timezone.utc)

View File

@@ -1,16 +1,10 @@
"""Trace collection listener for orchestrating trace collection."""
import os
from typing import Any, ClassVar
import uuid
from typing_extensions import Self
from crewai.cli.authentication.token import AuthError, get_auth_token
from crewai.cli.version import get_crewai_version
from crewai.events.base_event_listener import BaseEventListener
from crewai.events.event_bus import CrewAIEventsBus
from crewai.events.utils.console_formatter import ConsoleFormatter
from crewai.events.listeners.tracing.first_time_trace_handler import (
FirstTimeTraceHandler,
)
@@ -59,8 +53,6 @@ from crewai.events.types.memory_events import (
MemoryQueryCompletedEvent,
MemoryQueryFailedEvent,
MemoryQueryStartedEvent,
MemoryRetrievalCompletedEvent,
MemoryRetrievalStartedEvent,
MemorySaveCompletedEvent,
MemorySaveFailedEvent,
MemorySaveStartedEvent,
@@ -83,7 +75,9 @@ from crewai.events.types.tool_usage_events import (
class TraceCollectionListener(BaseEventListener):
"""Trace collection listener that orchestrates trace collection."""
"""
Trace collection listener that orchestrates trace collection
"""
complex_events: ClassVar[list[str]] = [
"task_started",
@@ -94,12 +88,11 @@ class TraceCollectionListener(BaseEventListener):
"agent_execution_completed",
]
_instance: Self | None = None
_initialized: bool = False
_listeners_setup: bool = False
_instance = None
_initialized = False
_listeners_setup = False
def __new__(cls, batch_manager: TraceBatchManager | None = None) -> Self:
"""Create or return singleton instance."""
def __new__(cls, batch_manager: TraceBatchManager | None = None):
if cls._instance is None:
cls._instance = super().__new__(cls)
return cls._instance
@@ -107,14 +100,7 @@ class TraceCollectionListener(BaseEventListener):
def __init__(
self,
batch_manager: TraceBatchManager | None = None,
formatter: ConsoleFormatter | None = None,
) -> None:
"""Initialize trace collection listener.
Args:
batch_manager: Optional trace batch manager instance.
formatter: Optional console formatter for output.
"""
):
if self._initialized:
return
@@ -122,22 +108,19 @@ class TraceCollectionListener(BaseEventListener):
self.batch_manager = batch_manager or TraceBatchManager()
self._initialized = True
self.first_time_handler = FirstTimeTraceHandler()
self.formatter = formatter
self.memory_retrieval_in_progress = False
self.memory_save_in_progress = False
if self.first_time_handler.initialize_for_first_time_user():
self.first_time_handler.set_batch_manager(self.batch_manager)
def _check_authenticated(self) -> bool:
"""Check if tracing should be enabled."""
"""Check if tracing should be enabled"""
try:
return bool(get_auth_token())
except AuthError:
return False
def _get_user_context(self) -> dict[str, str]:
"""Extract user context for tracing."""
"""Extract user context for tracing"""
return {
"user_id": os.getenv("CREWAI_USER_ID", "anonymous"),
"organization_id": os.getenv("CREWAI_ORG_ID", ""),
@@ -145,12 +128,9 @@ class TraceCollectionListener(BaseEventListener):
"trace_id": str(uuid.uuid4()),
}
def setup_listeners(self, crewai_event_bus: CrewAIEventsBus) -> None:
"""Setup event listeners - delegates to specific handlers.
def setup_listeners(self, crewai_event_bus):
"""Setup event listeners - delegates to specific handlers"""
Args:
crewai_event_bus: The event bus to register listeners on.
"""
if self._listeners_setup:
return
@@ -160,52 +140,50 @@ class TraceCollectionListener(BaseEventListener):
self._listeners_setup = True
def _register_flow_event_handlers(self, event_bus: CrewAIEventsBus) -> None:
"""Register handlers for flow events."""
def _register_flow_event_handlers(self, event_bus):
"""Register handlers for flow events"""
@event_bus.on(FlowCreatedEvent)
def on_flow_created(source: Any, event: FlowCreatedEvent) -> None:
def on_flow_created(source, event):
pass
@event_bus.on(FlowStartedEvent)
def on_flow_started(source: Any, event: FlowStartedEvent) -> None:
def on_flow_started(source, event):
if not self.batch_manager.is_batch_initialized():
self._initialize_flow_batch(source, event)
self._handle_trace_event("flow_started", source, event)
@event_bus.on(MethodExecutionStartedEvent)
def on_method_started(source: Any, event: MethodExecutionStartedEvent) -> None:
def on_method_started(source, event):
self._handle_trace_event("method_execution_started", source, event)
@event_bus.on(MethodExecutionFinishedEvent)
def on_method_finished(
source: Any, event: MethodExecutionFinishedEvent
) -> None:
def on_method_finished(source, event):
self._handle_trace_event("method_execution_finished", source, event)
@event_bus.on(MethodExecutionFailedEvent)
def on_method_failed(source: Any, event: MethodExecutionFailedEvent) -> None:
def on_method_failed(source, event):
self._handle_trace_event("method_execution_failed", source, event)
@event_bus.on(FlowFinishedEvent)
def on_flow_finished(source: Any, event: FlowFinishedEvent) -> None:
def on_flow_finished(source, event):
self._handle_trace_event("flow_finished", source, event)
@event_bus.on(FlowPlotEvent)
def on_flow_plot(source: Any, event: FlowPlotEvent) -> None:
def on_flow_plot(source, event):
self._handle_action_event("flow_plot", source, event)
def _register_context_event_handlers(self, event_bus: CrewAIEventsBus) -> None:
"""Register handlers for context events (start/end)."""
def _register_context_event_handlers(self, event_bus):
"""Register handlers for context events (start/end)"""
@event_bus.on(CrewKickoffStartedEvent)
def on_crew_started(source: Any, event: CrewKickoffStartedEvent) -> None:
def on_crew_started(source, event):
if not self.batch_manager.is_batch_initialized():
self._initialize_crew_batch(source, event)
self._handle_trace_event("crew_kickoff_started", source, event)
@event_bus.on(CrewKickoffCompletedEvent)
def on_crew_completed(source: Any, event: CrewKickoffCompletedEvent) -> None:
def on_crew_completed(source, event):
self._handle_trace_event("crew_kickoff_completed", source, event)
if self.batch_manager.batch_owner_type == "crew":
if self.first_time_handler.is_first_time:
@@ -215,7 +193,7 @@ class TraceCollectionListener(BaseEventListener):
self.batch_manager.finalize_batch()
@event_bus.on(CrewKickoffFailedEvent)
def on_crew_failed(source: Any, event: CrewKickoffFailedEvent) -> None:
def on_crew_failed(source, event):
self._handle_trace_event("crew_kickoff_failed", source, event)
if self.first_time_handler.is_first_time:
self.first_time_handler.mark_events_collected()
@@ -224,245 +202,134 @@ class TraceCollectionListener(BaseEventListener):
self.batch_manager.finalize_batch()
@event_bus.on(TaskStartedEvent)
def on_task_started(source: Any, event: TaskStartedEvent) -> None:
def on_task_started(source, event):
self._handle_trace_event("task_started", source, event)
@event_bus.on(TaskCompletedEvent)
def on_task_completed(source: Any, event: TaskCompletedEvent) -> None:
def on_task_completed(source, event):
self._handle_trace_event("task_completed", source, event)
@event_bus.on(TaskFailedEvent)
def on_task_failed(source: Any, event: TaskFailedEvent) -> None:
def on_task_failed(source, event):
self._handle_trace_event("task_failed", source, event)
@event_bus.on(AgentExecutionStartedEvent)
def on_agent_started(source: Any, event: AgentExecutionStartedEvent) -> None:
def on_agent_started(source, event):
self._handle_trace_event("agent_execution_started", source, event)
@event_bus.on(AgentExecutionCompletedEvent)
def on_agent_completed(
source: Any, event: AgentExecutionCompletedEvent
) -> None:
def on_agent_completed(source, event):
self._handle_trace_event("agent_execution_completed", source, event)
@event_bus.on(LiteAgentExecutionStartedEvent)
def on_lite_agent_started(
source: Any, event: LiteAgentExecutionStartedEvent
) -> None:
def on_lite_agent_started(source, event):
self._handle_trace_event("lite_agent_execution_started", source, event)
@event_bus.on(LiteAgentExecutionCompletedEvent)
def on_lite_agent_completed(
source: Any, event: LiteAgentExecutionCompletedEvent
) -> None:
def on_lite_agent_completed(source, event):
self._handle_trace_event("lite_agent_execution_completed", source, event)
@event_bus.on(LiteAgentExecutionErrorEvent)
def on_lite_agent_error(
source: Any, event: LiteAgentExecutionErrorEvent
) -> None:
def on_lite_agent_error(source, event):
self._handle_trace_event("lite_agent_execution_error", source, event)
@event_bus.on(AgentExecutionErrorEvent)
def on_agent_error(source: Any, event: AgentExecutionErrorEvent) -> None:
def on_agent_error(source, event):
self._handle_trace_event("agent_execution_error", source, event)
@event_bus.on(LLMGuardrailStartedEvent)
def on_guardrail_started(source: Any, event: LLMGuardrailStartedEvent) -> None:
def on_guardrail_started(source, event):
self._handle_trace_event("llm_guardrail_started", source, event)
@event_bus.on(LLMGuardrailCompletedEvent)
def on_guardrail_completed(
source: Any, event: LLMGuardrailCompletedEvent
) -> None:
def on_guardrail_completed(source, event):
self._handle_trace_event("llm_guardrail_completed", source, event)
def _register_action_event_handlers(self, event_bus: CrewAIEventsBus) -> None:
"""Register handlers for action events (LLM calls, tool usage)."""
def _register_action_event_handlers(self, event_bus):
"""Register handlers for action events (LLM calls, tool usage)"""
@event_bus.on(LLMCallStartedEvent)
def on_llm_call_started(source: Any, event: LLMCallStartedEvent) -> None:
def on_llm_call_started(source, event):
self._handle_action_event("llm_call_started", source, event)
@event_bus.on(LLMCallCompletedEvent)
def on_llm_call_completed(source: Any, event: LLMCallCompletedEvent) -> None:
def on_llm_call_completed(source, event):
self._handle_action_event("llm_call_completed", source, event)
@event_bus.on(LLMCallFailedEvent)
def on_llm_call_failed(source: Any, event: LLMCallFailedEvent) -> None:
def on_llm_call_failed(source, event):
self._handle_action_event("llm_call_failed", source, event)
@event_bus.on(ToolUsageStartedEvent)
def on_tool_started(source: Any, event: ToolUsageStartedEvent) -> None:
def on_tool_started(source, event):
self._handle_action_event("tool_usage_started", source, event)
@event_bus.on(ToolUsageFinishedEvent)
def on_tool_finished(source: Any, event: ToolUsageFinishedEvent) -> None:
def on_tool_finished(source, event):
self._handle_action_event("tool_usage_finished", source, event)
@event_bus.on(ToolUsageErrorEvent)
def on_tool_error(source: Any, event: ToolUsageErrorEvent) -> None:
def on_tool_error(source, event):
self._handle_action_event("tool_usage_error", source, event)
@event_bus.on(MemoryQueryStartedEvent)
def on_memory_query_started(
source: Any, event: MemoryQueryStartedEvent
) -> None:
def on_memory_query_started(source, event):
self._handle_action_event("memory_query_started", source, event)
@event_bus.on(MemoryQueryCompletedEvent)
def on_memory_query_completed(
source: Any, event: MemoryQueryCompletedEvent
) -> None:
def on_memory_query_completed(source, event):
self._handle_action_event("memory_query_completed", source, event)
if self.formatter and self.memory_retrieval_in_progress:
self.formatter.handle_memory_query_completed(
self.formatter.current_agent_branch,
event.source_type or "memory",
event.query_time_ms,
self.formatter.current_crew_tree,
)
@event_bus.on(MemoryQueryFailedEvent)
def on_memory_query_failed(source: Any, event: MemoryQueryFailedEvent) -> None:
def on_memory_query_failed(source, event):
self._handle_action_event("memory_query_failed", source, event)
if self.formatter and self.memory_retrieval_in_progress:
self.formatter.handle_memory_query_failed(
self.formatter.current_agent_branch,
self.formatter.current_crew_tree,
event.error,
event.source_type or "memory",
)
@event_bus.on(MemorySaveStartedEvent)
def on_memory_save_started(source: Any, event: MemorySaveStartedEvent) -> None:
def on_memory_save_started(source, event):
self._handle_action_event("memory_save_started", source, event)
if self.formatter:
if self.memory_save_in_progress:
return
self.memory_save_in_progress = True
self.formatter.handle_memory_save_started(
self.formatter.current_agent_branch,
self.formatter.current_crew_tree,
)
@event_bus.on(MemorySaveCompletedEvent)
def on_memory_save_completed(
source: Any, event: MemorySaveCompletedEvent
) -> None:
def on_memory_save_completed(source, event):
self._handle_action_event("memory_save_completed", source, event)
if self.formatter:
if not self.memory_save_in_progress:
return
self.memory_save_in_progress = False
self.formatter.handle_memory_save_completed(
self.formatter.current_agent_branch,
self.formatter.current_crew_tree,
event.save_time_ms,
event.source_type or "memory",
)
@event_bus.on(MemorySaveFailedEvent)
def on_memory_save_failed(source: Any, event: MemorySaveFailedEvent) -> None:
def on_memory_save_failed(source, event):
self._handle_action_event("memory_save_failed", source, event)
if self.formatter and self.memory_save_in_progress:
self.formatter.handle_memory_save_failed(
self.formatter.current_agent_branch,
event.error,
event.source_type or "memory",
self.formatter.current_crew_tree,
)
@event_bus.on(MemoryRetrievalStartedEvent)
def on_memory_retrieval_started(
source: Any, event: MemoryRetrievalStartedEvent
) -> None:
if self.formatter:
if self.memory_retrieval_in_progress:
return
self.memory_retrieval_in_progress = True
self.formatter.handle_memory_retrieval_started(
self.formatter.current_agent_branch,
self.formatter.current_crew_tree,
)
@event_bus.on(MemoryRetrievalCompletedEvent)
def on_memory_retrieval_completed(
source: Any, event: MemoryRetrievalCompletedEvent
) -> None:
if self.formatter:
if not self.memory_retrieval_in_progress:
return
self.memory_retrieval_in_progress = False
self.formatter.handle_memory_retrieval_completed(
self.formatter.current_agent_branch,
self.formatter.current_crew_tree,
event.memory_content,
event.retrieval_time_ms,
)
@event_bus.on(AgentReasoningStartedEvent)
def on_agent_reasoning_started(
source: Any, event: AgentReasoningStartedEvent
) -> None:
def on_agent_reasoning_started(source, event):
self._handle_action_event("agent_reasoning_started", source, event)
@event_bus.on(AgentReasoningCompletedEvent)
def on_agent_reasoning_completed(
source: Any, event: AgentReasoningCompletedEvent
) -> None:
def on_agent_reasoning_completed(source, event):
self._handle_action_event("agent_reasoning_completed", source, event)
@event_bus.on(AgentReasoningFailedEvent)
def on_agent_reasoning_failed(
source: Any, event: AgentReasoningFailedEvent
) -> None:
def on_agent_reasoning_failed(source, event):
self._handle_action_event("agent_reasoning_failed", source, event)
@event_bus.on(KnowledgeRetrievalStartedEvent)
def on_knowledge_retrieval_started(
source: Any, event: KnowledgeRetrievalStartedEvent
) -> None:
def on_knowledge_retrieval_started(source, event):
self._handle_action_event("knowledge_retrieval_started", source, event)
@event_bus.on(KnowledgeRetrievalCompletedEvent)
def on_knowledge_retrieval_completed(
source: Any, event: KnowledgeRetrievalCompletedEvent
) -> None:
def on_knowledge_retrieval_completed(source, event):
self._handle_action_event("knowledge_retrieval_completed", source, event)
@event_bus.on(KnowledgeQueryStartedEvent)
def on_knowledge_query_started(
source: Any, event: KnowledgeQueryStartedEvent
) -> None:
def on_knowledge_query_started(source, event):
self._handle_action_event("knowledge_query_started", source, event)
@event_bus.on(KnowledgeQueryCompletedEvent)
def on_knowledge_query_completed(
source: Any, event: KnowledgeQueryCompletedEvent
) -> None:
def on_knowledge_query_completed(source, event):
self._handle_action_event("knowledge_query_completed", source, event)
@event_bus.on(KnowledgeQueryFailedEvent)
def on_knowledge_query_failed(
source: Any, event: KnowledgeQueryFailedEvent
) -> None:
def on_knowledge_query_failed(source, event):
self._handle_action_event("knowledge_query_failed", source, event)
def _initialize_crew_batch(self, source: Any, event: Any) -> None:
"""Initialize trace batch.
Args:
source: Source object that triggered the event.
event: Event object containing crew information.
"""
def _initialize_crew_batch(self, source: Any, event: Any):
"""Initialize trace batch"""
user_context = self._get_user_context()
execution_metadata = {
"crew_name": getattr(event, "crew_name", "Unknown Crew"),
@@ -475,13 +342,8 @@ class TraceCollectionListener(BaseEventListener):
self._initialize_batch(user_context, execution_metadata)
def _initialize_flow_batch(self, source: Any, event: Any) -> None:
"""Initialize trace batch for Flow execution.
Args:
source: Source object that triggered the event.
event: Event object containing flow information.
"""
def _initialize_flow_batch(self, source: Any, event: Any):
"""Initialize trace batch for Flow execution"""
user_context = self._get_user_context()
execution_metadata = {
"flow_name": getattr(event, "flow_name", "Unknown Flow"),
@@ -497,32 +359,21 @@ class TraceCollectionListener(BaseEventListener):
def _initialize_batch(
self, user_context: dict[str, str], execution_metadata: dict[str, Any]
) -> None:
"""Initialize trace batch - auto-enable ephemeral for first-time users.
):
"""Initialize trace batch - auto-enable ephemeral for first-time users."""
Args:
user_context: User context information.
execution_metadata: Metadata about the execution.
"""
if self.first_time_handler.is_first_time:
self.batch_manager.initialize_batch(
return self.batch_manager.initialize_batch(
user_context, execution_metadata, use_ephemeral=True
)
return
use_ephemeral = not self._check_authenticated()
self.batch_manager.initialize_batch(
return self.batch_manager.initialize_batch(
user_context, execution_metadata, use_ephemeral=use_ephemeral
)
def _handle_trace_event(self, event_type: str, source: Any, event: Any) -> None:
"""Generic handler for context end events.
Args:
event_type: Type of the event.
source: Source object that triggered the event.
event: Event object.
"""
def _handle_trace_event(self, event_type: str, source: Any, event: Any):
"""Generic handler for context end events"""
self.batch_manager.begin_event_processing()
try:
trace_event = self._create_trace_event(event_type, source, event)
@@ -530,14 +381,9 @@ class TraceCollectionListener(BaseEventListener):
finally:
self.batch_manager.end_event_processing()
def _handle_action_event(self, event_type: str, source: Any, event: Any) -> None:
"""Generic handler for action events (LLM calls, tool usage).
def _handle_action_event(self, event_type: str, source: Any, event: Any):
"""Generic handler for action events (LLM calls, tool usage)"""
Args:
event_type: Type of the event.
source: Source object that triggered the event.
event: Event object.
"""
if not self.batch_manager.is_batch_initialized():
user_context = self._get_user_context()
execution_metadata = {

View File

@@ -1,141 +0,0 @@
"""Events for A2A (Agent-to-Agent) delegation.
This module defines events emitted during A2A protocol delegation,
including both single-turn and multiturn conversation flows.
"""
from typing import Any, Literal
from crewai.events.base_events import BaseEvent
class A2AEventBase(BaseEvent):
"""Base class for A2A events with task/agent context."""
from_task: Any | None = None
from_agent: Any | None = None
def __init__(self, **data):
"""Initialize A2A event, extracting task and agent metadata."""
if data.get("from_task"):
task = data["from_task"]
data["task_id"] = str(task.id)
data["task_name"] = task.name or task.description
data["from_task"] = None
if data.get("from_agent"):
agent = data["from_agent"]
data["agent_id"] = str(agent.id)
data["agent_role"] = agent.role
data["from_agent"] = None
super().__init__(**data)
class A2ADelegationStartedEvent(A2AEventBase):
"""Event emitted when A2A delegation starts.
Attributes:
endpoint: A2A agent endpoint URL (AgentCard URL)
task_description: Task being delegated to the A2A agent
agent_id: A2A agent identifier
is_multiturn: Whether this is part of a multiturn conversation
turn_number: Current turn number (1-indexed, 1 for single-turn)
"""
type: str = "a2a_delegation_started"
endpoint: str
task_description: str
agent_id: str
is_multiturn: bool = False
turn_number: int = 1
class A2ADelegationCompletedEvent(A2AEventBase):
"""Event emitted when A2A delegation completes.
Attributes:
status: Completion status (completed, input_required, failed, etc.)
result: Result message if status is completed
error: Error/response message (error for failed, response for input_required)
is_multiturn: Whether this is part of a multiturn conversation
"""
type: str = "a2a_delegation_completed"
status: str
result: str | None = None
error: str | None = None
is_multiturn: bool = False
class A2AConversationStartedEvent(A2AEventBase):
"""Event emitted when a multiturn A2A conversation starts.
This is emitted once at the beginning of a multiturn conversation,
before the first message exchange.
Attributes:
agent_id: A2A agent identifier
endpoint: A2A agent endpoint URL
a2a_agent_name: Name of the A2A agent from agent card
"""
type: str = "a2a_conversation_started"
agent_id: str
endpoint: str
a2a_agent_name: str | None = None
class A2AMessageSentEvent(A2AEventBase):
"""Event emitted when a message is sent to the A2A agent.
Attributes:
message: Message content sent to the A2A agent
turn_number: Current turn number (1-indexed)
is_multiturn: Whether this is part of a multiturn conversation
agent_role: Role of the CrewAI agent sending the message
"""
type: str = "a2a_message_sent"
message: str
turn_number: int
is_multiturn: bool = False
agent_role: str | None = None
class A2AResponseReceivedEvent(A2AEventBase):
"""Event emitted when a response is received from the A2A agent.
Attributes:
response: Response content from the A2A agent
turn_number: Current turn number (1-indexed)
is_multiturn: Whether this is part of a multiturn conversation
status: Response status (input_required, completed, etc.)
agent_role: Role of the CrewAI agent (for display)
"""
type: str = "a2a_response_received"
response: str
turn_number: int
is_multiturn: bool = False
status: str
agent_role: str | None = None
class A2AConversationCompletedEvent(A2AEventBase):
"""Event emitted when a multiturn A2A conversation completes.
This is emitted once at the end of a multiturn conversation.
Attributes:
status: Final status (completed, failed, etc.)
final_result: Final result if completed successfully
error: Error message if failed
total_turns: Total number of turns in the conversation
"""
type: str = "a2a_conversation_completed"
status: Literal["completed", "failed"]
final_result: str | None = None
error: str | None = None
total_turns: int

View File

@@ -1,85 +0,0 @@
from datetime import datetime
from typing import Any
from crewai.events.base_events import BaseEvent
class MCPEvent(BaseEvent):
"""Base event for MCP operations."""
server_name: str
server_url: str | None = None
transport_type: str | None = None # "stdio", "http", "sse"
agent_id: str | None = None
agent_role: str | None = None
from_agent: Any | None = None
from_task: Any | None = None
def __init__(self, **data):
super().__init__(**data)
self._set_agent_params(data)
self._set_task_params(data)
class MCPConnectionStartedEvent(MCPEvent):
"""Event emitted when starting to connect to an MCP server."""
type: str = "mcp_connection_started"
connect_timeout: int | None = None
is_reconnect: bool = (
False # True if this is a reconnection, False for first connection
)
class MCPConnectionCompletedEvent(MCPEvent):
"""Event emitted when successfully connected to an MCP server."""
type: str = "mcp_connection_completed"
started_at: datetime | None = None
completed_at: datetime | None = None
connection_duration_ms: float | None = None
is_reconnect: bool = (
False # True if this was a reconnection, False for first connection
)
class MCPConnectionFailedEvent(MCPEvent):
"""Event emitted when connection to an MCP server fails."""
type: str = "mcp_connection_failed"
error: str
error_type: str | None = None # "timeout", "authentication", "network", etc.
started_at: datetime | None = None
failed_at: datetime | None = None
class MCPToolExecutionStartedEvent(MCPEvent):
"""Event emitted when starting to execute an MCP tool."""
type: str = "mcp_tool_execution_started"
tool_name: str
tool_args: dict[str, Any] | None = None
class MCPToolExecutionCompletedEvent(MCPEvent):
"""Event emitted when MCP tool execution completes."""
type: str = "mcp_tool_execution_completed"
tool_name: str
tool_args: dict[str, Any] | None = None
result: Any | None = None
started_at: datetime | None = None
completed_at: datetime | None = None
execution_duration_ms: float | None = None
class MCPToolExecutionFailedEvent(MCPEvent):
"""Event emitted when MCP tool execution fails."""
type: str = "mcp_tool_execution_failed"
tool_name: str
tool_args: dict[str, Any] | None = None
error: str
error_type: str | None = None # "timeout", "validation", "server_error", etc.
started_at: datetime | None = None
failed_at: datetime | None = None

View File

@@ -17,16 +17,9 @@ class ConsoleFormatter:
current_method_branch: Tree | None = None
current_lite_agent_branch: Tree | None = None
tool_usage_counts: ClassVar[dict[str, int]] = {}
current_reasoning_branch: Tree | None = None
current_reasoning_branch: Tree | None = None # Track reasoning status
_live_paused: bool = False
current_llm_tool_tree: Tree | None = None
current_a2a_conversation_branch: Tree | None = None
current_a2a_turn_count: int = 0
_pending_a2a_message: str | None = None
_pending_a2a_agent_role: str | None = None
_pending_a2a_turn_number: int | None = None
_a2a_turn_branches: ClassVar[dict[int, Tree]] = {}
_current_a2a_agent_name: str | None = None
def __init__(self, verbose: bool = False):
self.console = Console(width=None)
@@ -199,12 +192,7 @@ class ConsoleFormatter:
style,
ID=source_id,
)
if status == "failed" and final_string_output:
content.append("Error:\n", style="white bold")
content.append(f"{final_string_output}\n", style="red")
else:
content.append(f"Final Output: {final_string_output}\n", style="white")
content.append(f"Final Output: {final_string_output}\n", style="white")
self.print_panel(content, title, style)
@@ -369,14 +357,7 @@ class ConsoleFormatter:
return flow_tree
def start_flow(self, flow_name: str, flow_id: str) -> Tree | None:
"""Initialize or update a flow execution tree."""
if self.current_flow_tree is not None:
for child in self.current_flow_tree.children:
if "Starting Flow" in str(child.label):
child.label = Text("🚀 Flow Started", style="green")
break
return self.current_flow_tree
"""Initialize a flow execution tree."""
flow_tree = Tree("")
flow_label = Text()
flow_label.append("🌊 Flow: ", style="blue bold")
@@ -455,38 +436,27 @@ class ConsoleFormatter:
prefix, style = "🔄 Running:", "yellow"
elif status == "completed":
prefix, style = "✅ Completed:", "green"
# Update initialization node when a method completes successfully
for child in flow_tree.children:
if "Starting Flow" in str(child.label):
child.label = Text("Flow Method Step", style="white")
break
else:
prefix, style = "❌ Failed:", "red"
# Update initialization node on failure
for child in flow_tree.children:
if "Starting Flow" in str(child.label):
child.label = Text("❌ Flow Step Failed", style="red")
break
if method_branch is not None:
if method_branch in flow_tree.children:
method_branch.label = Text(prefix, style=f"{style} bold") + Text(
f" {method_name}", style=style
)
self.print(flow_tree)
self.print()
return method_branch
for branch in flow_tree.children:
label_str = str(branch.label)
if f" {method_name}" in label_str and (
"Running:" in label_str
or "Completed:" in label_str
or "Failed:" in label_str
):
method_branch = branch
break
if method_branch is None:
method_branch = flow_tree.add("")
if not method_branch:
# Find or create method branch
for branch in flow_tree.children:
if method_name in str(branch.label):
method_branch = branch
break
if not method_branch:
method_branch = flow_tree.add("")
method_branch.label = Text(prefix, style=f"{style} bold") + Text(
f" {method_name}", style=style
@@ -494,7 +464,6 @@ class ConsoleFormatter:
self.print(flow_tree)
self.print()
return method_branch
def get_llm_tree(self, tool_name: str):
@@ -1486,37 +1455,22 @@ class ConsoleFormatter:
self.print()
elif isinstance(formatted_answer, AgentFinish):
is_a2a_delegation = False
try:
output_data = json.loads(formatted_answer.output)
if isinstance(output_data, dict):
if output_data.get("is_a2a") is True:
is_a2a_delegation = True
elif "output" in output_data:
nested_output = output_data["output"]
if (
isinstance(nested_output, dict)
and nested_output.get("is_a2a") is True
):
is_a2a_delegation = True
except (json.JSONDecodeError, TypeError, ValueError):
pass
# Create content for the finish panel
content = Text()
content.append("Agent: ", style="white")
content.append(f"{agent_role}\n\n", style="bright_green bold")
content.append("Final Answer:\n", style="white")
content.append(f"{formatted_answer.output}", style="bright_green")
if not is_a2a_delegation:
content = Text()
content.append("Agent: ", style="white")
content.append(f"{agent_role}\n\n", style="bright_green bold")
content.append("Final Answer:\n", style="white")
content.append(f"{formatted_answer.output}", style="bright_green")
finish_panel = Panel(
content,
title="✅ Agent Final Answer",
border_style="green",
padding=(1, 2),
)
self.print(finish_panel)
self.print()
# Create and display the finish panel
finish_panel = Panel(
content,
title="✅ Agent Final Answer",
border_style="green",
padding=(1, 2),
)
self.print(finish_panel)
self.print()
def handle_memory_retrieval_started(
self,
@@ -1816,635 +1770,3 @@ class ConsoleFormatter:
Attempts=f"{retry_count + 1}",
)
self.print_panel(content, "🛡️ Guardrail Failed", "red")
def handle_a2a_delegation_started(
self,
endpoint: str,
task_description: str,
agent_id: str,
is_multiturn: bool = False,
turn_number: int = 1,
) -> None:
"""Handle A2A delegation started event.
Args:
endpoint: A2A agent endpoint URL
task_description: Task being delegated
agent_id: A2A agent identifier
is_multiturn: Whether this is part of a multiturn conversation
turn_number: Current turn number in conversation (1-indexed)
"""
branch_to_use = self.current_lite_agent_branch or self.current_task_branch
tree_to_use = self.current_crew_tree or branch_to_use
a2a_branch: Tree | None = None
if is_multiturn:
if self.current_a2a_turn_count == 0 and not isinstance(
self.current_a2a_conversation_branch, Tree
):
if branch_to_use is not None and tree_to_use is not None:
self.current_a2a_conversation_branch = branch_to_use.add("")
self.update_tree_label(
self.current_a2a_conversation_branch,
"💬",
f"Multiturn A2A Conversation ({agent_id})",
"cyan",
)
self.print(tree_to_use)
self.print()
else:
self.current_a2a_conversation_branch = "MULTITURN_NO_TREE"
content = Text()
content.append(
"Multiturn A2A Conversation Started\n\n", style="cyan bold"
)
content.append("Agent ID: ", style="white")
content.append(f"{agent_id}\n", style="cyan")
content.append("Note: ", style="white dim")
content.append(
"Conversation will be tracked in tree view", style="cyan dim"
)
panel = self.create_panel(
content, "💬 Multiturn Conversation", "cyan"
)
self.print(panel)
self.print()
self.current_a2a_turn_count = turn_number
return (
self.current_a2a_conversation_branch
if isinstance(self.current_a2a_conversation_branch, Tree)
else None
)
if branch_to_use is not None and tree_to_use is not None:
a2a_branch = branch_to_use.add("")
self.update_tree_label(
a2a_branch,
"🔗",
f"Delegating to A2A Agent ({agent_id})",
"cyan",
)
self.print(tree_to_use)
self.print()
content = Text()
content.append("A2A Delegation Started\n\n", style="cyan bold")
content.append("Agent ID: ", style="white")
content.append(f"{agent_id}\n", style="cyan")
content.append("Endpoint: ", style="white")
content.append(f"{endpoint}\n\n", style="cyan dim")
content.append("Task Description:\n", style="white")
task_preview = (
task_description
if len(task_description) <= 200
else task_description[:197] + "..."
)
content.append(task_preview, style="cyan")
panel = self.create_panel(content, "🔗 A2A Delegation", "cyan")
self.print(panel)
self.print()
return a2a_branch
def handle_a2a_delegation_completed(
self,
status: str,
result: str | None = None,
error: str | None = None,
is_multiturn: bool = False,
) -> None:
"""Handle A2A delegation completed event.
Args:
status: Completion status
result: Optional result message
error: Optional error message (or response for input_required)
is_multiturn: Whether this is part of a multiturn conversation
"""
tree_to_use = self.current_crew_tree or self.current_task_branch
a2a_branch = None
if is_multiturn and self.current_a2a_conversation_branch:
has_tree = isinstance(self.current_a2a_conversation_branch, Tree)
if status == "input_required" and error:
pass
elif status == "completed":
if has_tree:
final_turn = self.current_a2a_conversation_branch.add("")
self.update_tree_label(
final_turn,
"",
"Conversation Completed",
"green",
)
if tree_to_use:
self.print(tree_to_use)
self.print()
self.current_a2a_conversation_branch = None
self.current_a2a_turn_count = 0
elif status == "failed":
if has_tree:
error_turn = self.current_a2a_conversation_branch.add("")
error_msg = (
error[:150] + "..." if error and len(error) > 150 else error
)
self.update_tree_label(
error_turn,
"",
f"Failed: {error_msg}" if error else "Conversation Failed",
"red",
)
if tree_to_use:
self.print(tree_to_use)
self.print()
self.current_a2a_conversation_branch = None
self.current_a2a_turn_count = 0
return
if a2a_branch and tree_to_use:
if status == "completed":
self.update_tree_label(
a2a_branch,
"",
"A2A Delegation Completed",
"green",
)
elif status == "failed":
self.update_tree_label(
a2a_branch,
"",
"A2A Delegation Failed",
"red",
)
else:
self.update_tree_label(
a2a_branch,
"⚠️",
f"A2A Delegation {status.replace('_', ' ').title()}",
"yellow",
)
self.print(tree_to_use)
self.print()
if status == "completed" and result:
content = Text()
content.append("A2A Delegation Completed\n\n", style="green bold")
content.append("Result:\n", style="white")
result_preview = result if len(result) <= 500 else result[:497] + "..."
content.append(result_preview, style="green")
panel = self.create_panel(content, "✅ A2A Success", "green")
self.print(panel)
self.print()
elif status == "input_required" and error:
content = Text()
content.append("A2A Response\n\n", style="cyan bold")
content.append("Message:\n", style="white")
response_preview = error if len(error) <= 500 else error[:497] + "..."
content.append(response_preview, style="cyan")
panel = self.create_panel(content, "💬 A2A Response", "cyan")
self.print(panel)
self.print()
elif error:
content = Text()
content.append(
"A2A Delegation Issue\n\n",
style="red bold" if status == "failed" else "yellow bold",
)
content.append("Status: ", style="white")
content.append(
f"{status}\n\n", style="red" if status == "failed" else "yellow"
)
content.append("Message:\n", style="white")
content.append(error, style="red" if status == "failed" else "yellow")
panel_style = "red" if status == "failed" else "yellow"
panel_title = "❌ A2A Failed" if status == "failed" else "⚠️ A2A Status"
panel = self.create_panel(content, panel_title, panel_style)
self.print(panel)
self.print()
def handle_a2a_conversation_started(
self,
agent_id: str,
endpoint: str,
) -> None:
"""Handle A2A conversation started event.
Args:
agent_id: A2A agent identifier
endpoint: A2A agent endpoint URL
"""
branch_to_use = self.current_lite_agent_branch or self.current_task_branch
tree_to_use = self.current_crew_tree or branch_to_use
if not isinstance(self.current_a2a_conversation_branch, Tree):
if branch_to_use is not None and tree_to_use is not None:
self.current_a2a_conversation_branch = branch_to_use.add("")
self.update_tree_label(
self.current_a2a_conversation_branch,
"💬",
f"Multiturn A2A Conversation ({agent_id})",
"cyan",
)
self.print(tree_to_use)
self.print()
else:
self.current_a2a_conversation_branch = "MULTITURN_NO_TREE"
def handle_a2a_message_sent(
self,
message: str,
turn_number: int,
agent_role: str | None = None,
) -> None:
"""Handle A2A message sent event.
Args:
message: Message content sent to the A2A agent
turn_number: Current turn number
agent_role: Role of the CrewAI agent sending the message
"""
self._pending_a2a_message = message
self._pending_a2a_agent_role = agent_role
self._pending_a2a_turn_number = turn_number
def handle_a2a_response_received(
self,
response: str,
turn_number: int,
status: str,
agent_role: str | None = None,
) -> None:
"""Handle A2A response received event.
Args:
response: Response content from the A2A agent
turn_number: Current turn number
status: Response status (input_required, completed, etc.)
agent_role: Role of the CrewAI agent (for display)
"""
if self.current_a2a_conversation_branch and isinstance(
self.current_a2a_conversation_branch, Tree
):
if turn_number in self._a2a_turn_branches:
turn_branch = self._a2a_turn_branches[turn_number]
else:
turn_branch = self.current_a2a_conversation_branch.add("")
self.update_tree_label(
turn_branch,
"💬",
f"Turn {turn_number}",
"cyan",
)
self._a2a_turn_branches[turn_number] = turn_branch
crewai_agent_role = self._pending_a2a_agent_role or agent_role or "User"
message_content = self._pending_a2a_message or "sent message"
message_preview = (
message_content[:100] + "..."
if len(message_content) > 100
else message_content
)
user_node = turn_branch.add("")
self.update_tree_label(
user_node,
f"{crewai_agent_role} 👤 : ",
f'"{message_preview}"',
"blue",
)
agent_node = turn_branch.add("")
response_preview = (
response[:100] + "..." if len(response) > 100 else response
)
a2a_agent_display = f"{self._current_a2a_agent_name} \U0001f916: "
if status == "completed":
response_color = "green"
status_indicator = ""
elif status == "input_required":
response_color = "yellow"
status_indicator = ""
elif status == "failed":
response_color = "red"
status_indicator = ""
elif status == "auth_required":
response_color = "magenta"
status_indicator = "🔒"
elif status == "canceled":
response_color = "dim"
status_indicator = ""
else:
response_color = "cyan"
status_indicator = ""
label = f'"{response_preview}"'
if status_indicator:
label = f"{status_indicator} {label}"
self.update_tree_label(
agent_node,
a2a_agent_display,
label,
response_color,
)
self._pending_a2a_message = None
self._pending_a2a_agent_role = None
self._pending_a2a_turn_number = None
tree_to_use = self.current_crew_tree or self.current_task_branch
if tree_to_use:
self.print(tree_to_use)
self.print()
def handle_a2a_conversation_completed(
self,
status: str,
final_result: str | None,
error: str | None,
total_turns: int,
) -> None:
"""Handle A2A conversation completed event.
Args:
status: Final status (completed, failed, etc.)
final_result: Final result if completed successfully
error: Error message if failed
total_turns: Total number of turns in the conversation
"""
if self.current_a2a_conversation_branch and isinstance(
self.current_a2a_conversation_branch, Tree
):
if status == "completed":
if self._pending_a2a_message and self._pending_a2a_agent_role:
if total_turns in self._a2a_turn_branches:
turn_branch = self._a2a_turn_branches[total_turns]
else:
turn_branch = self.current_a2a_conversation_branch.add("")
self.update_tree_label(
turn_branch,
"💬",
f"Turn {total_turns}",
"cyan",
)
self._a2a_turn_branches[total_turns] = turn_branch
crewai_agent_role = self._pending_a2a_agent_role
message_content = self._pending_a2a_message
message_preview = (
message_content[:100] + "..."
if len(message_content) > 100
else message_content
)
user_node = turn_branch.add("")
self.update_tree_label(
user_node,
f"{crewai_agent_role} 👤 : ",
f'"{message_preview}"',
"green",
)
self._pending_a2a_message = None
self._pending_a2a_agent_role = None
self._pending_a2a_turn_number = None
elif status == "failed":
error_turn = self.current_a2a_conversation_branch.add("")
error_msg = error[:150] + "..." if error and len(error) > 150 else error
self.update_tree_label(
error_turn,
"",
f"Failed: {error_msg}" if error else "Conversation Failed",
"red",
)
tree_to_use = self.current_crew_tree or self.current_task_branch
if tree_to_use:
self.print(tree_to_use)
self.print()
self.current_a2a_conversation_branch = None
self.current_a2a_turn_count = 0
# ----------- MCP EVENTS -----------
def handle_mcp_connection_started(
self,
server_name: str,
server_url: str | None = None,
transport_type: str | None = None,
is_reconnect: bool = False,
connect_timeout: int | None = None,
) -> None:
"""Handle MCP connection started event."""
if not self.verbose:
return
content = Text()
reconnect_text = " (Reconnecting)" if is_reconnect else ""
content.append(f"MCP Connection Started{reconnect_text}\n\n", style="cyan bold")
content.append("Server: ", style="white")
content.append(f"{server_name}\n", style="cyan")
if server_url:
content.append("URL: ", style="white")
content.append(f"{server_url}\n", style="cyan dim")
if transport_type:
content.append("Transport: ", style="white")
content.append(f"{transport_type}\n", style="cyan")
if connect_timeout:
content.append("Timeout: ", style="white")
content.append(f"{connect_timeout}s\n", style="cyan")
panel = self.create_panel(content, "🔌 MCP Connection", "cyan")
self.print(panel)
self.print()
def handle_mcp_connection_completed(
self,
server_name: str,
server_url: str | None = None,
transport_type: str | None = None,
connection_duration_ms: float | None = None,
is_reconnect: bool = False,
) -> None:
"""Handle MCP connection completed event."""
if not self.verbose:
return
content = Text()
reconnect_text = " (Reconnected)" if is_reconnect else ""
content.append(
f"MCP Connection Completed{reconnect_text}\n\n", style="green bold"
)
content.append("Server: ", style="white")
content.append(f"{server_name}\n", style="green")
if server_url:
content.append("URL: ", style="white")
content.append(f"{server_url}\n", style="green dim")
if transport_type:
content.append("Transport: ", style="white")
content.append(f"{transport_type}\n", style="green")
if connection_duration_ms is not None:
content.append("Duration: ", style="white")
content.append(f"{connection_duration_ms:.2f}ms\n", style="green")
panel = self.create_panel(content, "✅ MCP Connected", "green")
self.print(panel)
self.print()
def handle_mcp_connection_failed(
self,
server_name: str,
server_url: str | None = None,
transport_type: str | None = None,
error: str = "",
error_type: str | None = None,
) -> None:
"""Handle MCP connection failed event."""
if not self.verbose:
return
content = Text()
content.append("MCP Connection Failed\n\n", style="red bold")
content.append("Server: ", style="white")
content.append(f"{server_name}\n", style="red")
if server_url:
content.append("URL: ", style="white")
content.append(f"{server_url}\n", style="red dim")
if transport_type:
content.append("Transport: ", style="white")
content.append(f"{transport_type}\n", style="red")
if error_type:
content.append("Error Type: ", style="white")
content.append(f"{error_type}\n", style="red")
if error:
content.append("\nError: ", style="white bold")
error_preview = error[:500] + "..." if len(error) > 500 else error
content.append(f"{error_preview}\n", style="red")
panel = self.create_panel(content, "❌ MCP Connection Failed", "red")
self.print(panel)
self.print()
def handle_mcp_tool_execution_started(
self,
server_name: str,
tool_name: str,
tool_args: dict[str, Any] | None = None,
) -> None:
"""Handle MCP tool execution started event."""
if not self.verbose:
return
content = self.create_status_content(
"MCP Tool Execution Started",
tool_name,
"yellow",
tool_args=tool_args or {},
Server=server_name,
)
panel = self.create_panel(content, "🔧 MCP Tool", "yellow")
self.print(panel)
self.print()
def handle_mcp_tool_execution_completed(
self,
server_name: str,
tool_name: str,
tool_args: dict[str, Any] | None = None,
result: Any | None = None,
execution_duration_ms: float | None = None,
) -> None:
"""Handle MCP tool execution completed event."""
if not self.verbose:
return
content = self.create_status_content(
"MCP Tool Execution Completed",
tool_name,
"green",
tool_args=tool_args or {},
Server=server_name,
)
if execution_duration_ms is not None:
content.append("Duration: ", style="white")
content.append(f"{execution_duration_ms:.2f}ms\n", style="green")
if result is not None:
result_str = str(result)
if len(result_str) > 500:
result_str = result_str[:497] + "..."
content.append("\nResult: ", style="white bold")
content.append(f"{result_str}\n", style="green")
panel = self.create_panel(content, "✅ MCP Tool Completed", "green")
self.print(panel)
self.print()
def handle_mcp_tool_execution_failed(
self,
server_name: str,
tool_name: str,
tool_args: dict[str, Any] | None = None,
error: str = "",
error_type: str | None = None,
) -> None:
"""Handle MCP tool execution failed event."""
if not self.verbose:
return
content = self.create_status_content(
"MCP Tool Execution Failed",
tool_name,
"red",
tool_args=tool_args or {},
Server=server_name,
)
if error_type:
content.append("Error Type: ", style="white")
content.append(f"{error_type}\n", style="red")
if error:
content.append("\nError: ", style="white bold")
error_preview = error[:500] + "..." if len(error) > 500 else error
content.append(f"{error_preview}\n", style="red")
panel = self.create_panel(content, "❌ MCP Tool Failed", "red")
self.print(panel)
self.print()

View File

@@ -0,0 +1,65 @@
"""A2A (Agent-to-Agent) Protocol adapter for CrewAI.
This module provides integration with A2A protocol-compliant agents,
enabling CrewAI to orchestrate external agents like ServiceNow, Bedrock Agents,
Glean, and other A2A-compliant systems.
Example:
```python
from crewai.experimental.a2a import A2AAgentAdapter
# Create A2A agent
servicenow_agent = A2AAgentAdapter(
agent_card_url="https://servicenow.example.com/.well-known/agent-card.json",
auth_token="your-token",
role="ServiceNow Incident Manager",
goal="Create and manage IT incidents",
backstory="Expert at incident management",
)
# Use in crew
crew = Crew(agents=[servicenow_agent], tasks=[task])
```
"""
from crewai.experimental.a2a.a2a_adapter import A2AAgentAdapter
from crewai.experimental.a2a.auth import (
APIKeyAuth,
AuthScheme,
BearerTokenAuth,
HTTPBasicAuth,
HTTPDigestAuth,
OAuth2AuthorizationCode,
OAuth2ClientCredentials,
create_auth_from_agent_card,
)
from crewai.experimental.a2a.exceptions import (
A2AAuthenticationError,
A2AConfigurationError,
A2AConnectionError,
A2AError,
A2AInputRequiredError,
A2ATaskCanceledError,
A2ATaskFailedError,
)
__all__ = [
"A2AAgentAdapter",
"A2AAuthenticationError",
"A2AConfigurationError",
"A2AConnectionError",
"A2AError",
"A2AInputRequiredError",
"A2ATaskCanceledError",
"A2ATaskFailedError",
"APIKeyAuth",
# Authentication
"AuthScheme",
"BearerTokenAuth",
"HTTPBasicAuth",
"HTTPDigestAuth",
"OAuth2AuthorizationCode",
"OAuth2ClientCredentials",
"create_auth_from_agent_card",
]

File diff suppressed because it is too large Load Diff

View File

@@ -0,0 +1,424 @@
"""Authentication schemes for A2A protocol agents.
This module provides support for various authentication methods:
- Bearer tokens (existing)
- OAuth2 (Client Credentials, Authorization Code)
- API Keys (header, query, cookie)
- HTTP Basic authentication
- HTTP Digest authentication
"""
from __future__ import annotations
from abc import ABC, abstractmethod
import base64
from collections.abc import Awaitable, Callable
from typing import TYPE_CHECKING, Any, Literal
import httpx
from pydantic import BaseModel, Field
if TYPE_CHECKING:
from a2a.types import AgentCard
class AuthScheme(ABC, BaseModel):
"""Base class for authentication schemes."""
@abstractmethod
async def apply_auth(
self, client: httpx.AsyncClient, headers: dict[str, str]
) -> dict[str, str]:
"""Apply authentication to request headers.
Args:
client: HTTP client for making auth requests.
headers: Current request headers.
Returns:
Updated headers with authentication applied.
"""
...
@abstractmethod
def configure_client(self, client: httpx.AsyncClient) -> None:
"""Configure the HTTP client for this auth scheme.
Args:
client: HTTP client to configure.
"""
...
class BearerTokenAuth(AuthScheme):
"""Bearer token authentication (Authorization: Bearer <token>)."""
token: str = Field(description="Bearer token")
async def apply_auth(
self, client: httpx.AsyncClient, headers: dict[str, str]
) -> dict[str, str]:
"""Apply Bearer token to Authorization header."""
headers["Authorization"] = f"Bearer {self.token}"
return headers
def configure_client(self, client: httpx.AsyncClient) -> None:
"""No client configuration needed for Bearer tokens."""
class HTTPBasicAuth(AuthScheme):
"""HTTP Basic authentication."""
username: str = Field(description="Username")
password: str = Field(description="Password")
async def apply_auth(
self, client: httpx.AsyncClient, headers: dict[str, str]
) -> dict[str, str]:
"""Apply HTTP Basic authentication."""
credentials = f"{self.username}:{self.password}"
encoded = base64.b64encode(credentials.encode()).decode()
headers["Authorization"] = f"Basic {encoded}"
return headers
def configure_client(self, client: httpx.AsyncClient) -> None:
"""No client configuration needed for Basic auth."""
class HTTPDigestAuth(AuthScheme):
"""HTTP Digest authentication.
Note: Uses httpx-auth library for proper digest implementation.
"""
username: str = Field(description="Username")
password: str = Field(description="Password")
async def apply_auth(
self, client: httpx.AsyncClient, headers: dict[str, str]
) -> dict[str, str]:
"""Digest auth is handled by httpx auth flow, not headers."""
return headers
def configure_client(self, client: httpx.AsyncClient) -> None:
"""Configure client with Digest auth."""
try:
from httpx_auth import DigestAuth # type: ignore[import-not-found]
client.auth = DigestAuth(self.username, self.password) # type: ignore[import-not-found]
except ImportError as e:
msg = "httpx-auth required for Digest authentication. Install with: pip install httpx-auth"
raise ImportError(msg) from e
class APIKeyAuth(AuthScheme):
"""API Key authentication (header, query, or cookie)."""
api_key: str = Field(description="API key value")
location: Literal["header", "query", "cookie"] = Field(
default="header", description="Where to send the API key"
)
name: str = Field(default="X-API-Key", description="Parameter name for the API key")
async def apply_auth(
self, client: httpx.AsyncClient, headers: dict[str, str]
) -> dict[str, str]:
"""Apply API key authentication."""
if self.location == "header":
headers[self.name] = self.api_key
elif self.location == "cookie":
headers["Cookie"] = f"{self.name}={self.api_key}"
# Query params are handled in configure_client via event hooks
return headers
def configure_client(self, client: httpx.AsyncClient) -> None:
"""Configure client for query param API keys."""
if self.location == "query":
# Add API key to all requests via event hook
async def add_api_key_param(request: httpx.Request) -> None:
url = httpx.URL(request.url)
request.url = url.copy_add_param(self.name, self.api_key)
client.event_hooks["request"].append(add_api_key_param)
class OAuth2ClientCredentials(AuthScheme):
"""OAuth2 Client Credentials flow authentication."""
token_url: str = Field(description="OAuth2 token endpoint")
client_id: str = Field(description="OAuth2 client ID")
client_secret: str = Field(description="OAuth2 client secret")
scopes: list[str] = Field(
default_factory=list, description="Required OAuth2 scopes"
)
_access_token: str | None = None
_token_expires_at: float | None = None
async def apply_auth(
self, client: httpx.AsyncClient, headers: dict[str, str]
) -> dict[str, str]:
"""Apply OAuth2 access token to Authorization header."""
# Get or refresh token if needed
import time
if (
self._access_token is None
or self._token_expires_at is None
or time.time() >= self._token_expires_at
):
await self._fetch_token(client)
if self._access_token:
headers["Authorization"] = f"Bearer {self._access_token}"
return headers
async def _fetch_token(self, client: httpx.AsyncClient) -> None:
"""Fetch OAuth2 access token using client credentials flow."""
import time
data = {
"grant_type": "client_credentials",
"client_id": self.client_id,
"client_secret": self.client_secret,
}
if self.scopes:
data["scope"] = " ".join(self.scopes)
response = await client.post(self.token_url, data=data)
response.raise_for_status()
token_data = response.json()
self._access_token = token_data["access_token"]
# Calculate expiration time (default to 3600 seconds if not provided)
expires_in = token_data.get("expires_in", 3600)
self._token_expires_at = time.time() + expires_in - 60 # 60s buffer
def configure_client(self, client: httpx.AsyncClient) -> None:
"""No client configuration needed for OAuth2."""
class OAuth2AuthorizationCode(AuthScheme):
"""OAuth2 Authorization Code flow authentication.
Note: This requires interactive authorization and is typically used
for user-facing applications. For server-to-server, use ClientCredentials.
"""
authorization_url: str = Field(description="OAuth2 authorization endpoint")
token_url: str = Field(description="OAuth2 token endpoint")
client_id: str = Field(description="OAuth2 client ID")
client_secret: str = Field(description="OAuth2 client secret")
redirect_uri: str = Field(description="OAuth2 redirect URI")
scopes: list[str] = Field(
default_factory=list, description="Required OAuth2 scopes"
)
_access_token: str | None = None
_refresh_token: str | None = None
_token_expires_at: float | None = None
_authorization_callback: Callable[[str], Awaitable[str]] | None = None
def set_authorization_callback(
self, callback: Callable[[str], Awaitable[str]] | None
) -> None:
"""Set callback to handle authorization URL.
The callback receives the authorization URL and should return
the authorization code after user completes the flow.
"""
self._authorization_callback = callback
async def apply_auth(
self, client: httpx.AsyncClient, headers: dict[str, str]
) -> dict[str, str]:
"""Apply OAuth2 access token to Authorization header."""
import time
# Get or refresh token if needed
if self._access_token is None:
if self._authorization_callback is None:
msg = "Authorization callback not set. Use set_authorization_callback()"
raise ValueError(msg)
await self._fetch_initial_token(client)
elif self._token_expires_at and time.time() >= self._token_expires_at:
await self._refresh_access_token(client)
if self._access_token:
headers["Authorization"] = f"Bearer {self._access_token}"
return headers
async def _fetch_initial_token(self, client: httpx.AsyncClient) -> None:
"""Fetch initial access token using authorization code flow."""
import time
import urllib.parse
# Build authorization URL
params = {
"response_type": "code",
"client_id": self.client_id,
"redirect_uri": self.redirect_uri,
"scope": " ".join(self.scopes),
}
auth_url = f"{self.authorization_url}?{urllib.parse.urlencode(params)}"
# Get authorization code from callback
if self._authorization_callback is None:
msg = "Authorization callback not set"
raise ValueError(msg)
auth_code = await self._authorization_callback(auth_url)
# Exchange code for token
data = {
"grant_type": "authorization_code",
"code": auth_code,
"client_id": self.client_id,
"client_secret": self.client_secret,
"redirect_uri": self.redirect_uri,
}
response = await client.post(self.token_url, data=data)
response.raise_for_status()
token_data = response.json()
self._access_token = token_data["access_token"]
self._refresh_token = token_data.get("refresh_token")
expires_in = token_data.get("expires_in", 3600)
self._token_expires_at = time.time() + expires_in - 60
async def _refresh_access_token(self, client: httpx.AsyncClient) -> None:
"""Refresh the access token using refresh token."""
import time
if not self._refresh_token:
# Re-authorize if no refresh token
await self._fetch_initial_token(client)
return
data = {
"grant_type": "refresh_token",
"refresh_token": self._refresh_token,
"client_id": self.client_id,
"client_secret": self.client_secret,
}
response = await client.post(self.token_url, data=data)
response.raise_for_status()
token_data = response.json()
self._access_token = token_data["access_token"]
if "refresh_token" in token_data:
self._refresh_token = token_data["refresh_token"]
expires_in = token_data.get("expires_in", 3600)
self._token_expires_at = time.time() + expires_in - 60
def configure_client(self, client: httpx.AsyncClient) -> None:
"""No client configuration needed for OAuth2."""
def create_auth_from_agent_card(
agent_card: AgentCard, credentials: dict[str, Any]
) -> AuthScheme | None:
"""Create an appropriate authentication scheme from AgentCard security config.
Args:
agent_card: The A2A AgentCard containing security requirements.
credentials: User-provided credentials (passwords, tokens, keys, etc.).
Returns:
Configured AuthScheme, or None if no authentication required.
Example:
```python
# For OAuth2
credentials = {
"client_id": "my-app",
"client_secret": "secret123",
}
auth = create_auth_from_agent_card(agent_card, credentials)
# For API Key
credentials = {"api_key": "key-12345"}
auth = create_auth_from_agent_card(agent_card, credentials)
# For HTTP Basic
credentials = {"username": "user", "password": "pass"}
auth = create_auth_from_agent_card(agent_card, credentials)
```
"""
if not agent_card.security or not agent_card.security_schemes:
return None
# Get the first required security scheme
first_security_req = agent_card.security[0] if agent_card.security else {}
for scheme_name, _scopes in first_security_req.items():
security_scheme_obj = agent_card.security_schemes.get(scheme_name)
if not security_scheme_obj:
continue
# SecurityScheme is a dict-like object
security_scheme = dict(security_scheme_obj) # type: ignore[arg-type]
scheme_type = str(security_scheme.get("type", "")).lower()
# OAuth2
if scheme_type == "oauth2":
flows = security_scheme.get("flows", {})
if "clientCredentials" in flows:
flow = flows["clientCredentials"]
return OAuth2ClientCredentials(
token_url=str(flow["tokenUrl"]),
client_id=str(credentials.get("client_id", "")),
client_secret=str(credentials.get("client_secret", "")),
scopes=list(flow.get("scopes", {}).keys()),
)
if "authorizationCode" in flows:
flow = flows["authorizationCode"]
return OAuth2AuthorizationCode(
authorization_url=str(flow["authorizationUrl"]),
token_url=str(flow["tokenUrl"]),
client_id=str(credentials.get("client_id", "")),
client_secret=str(credentials.get("client_secret", "")),
redirect_uri=str(credentials.get("redirect_uri", "")),
scopes=list(flow.get("scopes", {}).keys()),
)
# API Key
elif scheme_type == "apikey":
location = str(security_scheme.get("in", "header"))
name = str(security_scheme.get("name", "X-API-Key"))
return APIKeyAuth(
api_key=str(credentials.get("api_key", "")),
location=location, # type: ignore[arg-type]
name=name,
)
# HTTP Auth
elif scheme_type == "http":
http_scheme = str(security_scheme.get("scheme", "")).lower()
if http_scheme == "basic":
return HTTPBasicAuth(
username=str(credentials.get("username", "")),
password=str(credentials.get("password", "")),
)
if http_scheme == "digest":
return HTTPDigestAuth(
username=str(credentials.get("username", "")),
password=str(credentials.get("password", "")),
)
if http_scheme == "bearer":
return BearerTokenAuth(token=str(credentials.get("token", "")))
return None

View File

@@ -0,0 +1,56 @@
"""Custom exceptions for A2A Agent Adapter."""
class A2AError(Exception):
"""Base exception for A2A adapter errors."""
class A2ATaskFailedError(A2AError):
"""Raised when A2A agent task fails or is rejected.
This exception is raised when the A2A agent reports a task
in the 'failed' or 'rejected' state.
"""
class A2AInputRequiredError(A2AError):
"""Raised when A2A agent requires additional input.
This exception is raised when the A2A agent reports a task
in the 'input_required' state, indicating that it needs more
information to complete the task.
"""
class A2AConfigurationError(A2AError):
"""Raised when A2A adapter configuration is invalid.
This exception is raised during initialization or setup when
the adapter configuration is invalid or incompatible.
"""
class A2AConnectionError(A2AError):
"""Raised when connection to A2A agent fails.
This exception is raised when the adapter cannot establish
a connection to the A2A agent or when network errors occur.
"""
class A2AAuthenticationError(A2AError):
"""Raised when A2A agent requires authentication.
This exception is raised when the A2A agent reports a task
in the 'auth_required' state, indicating that authentication
is needed before the task can continue.
"""
class A2ATaskCanceledError(A2AError):
"""Raised when A2A task is canceled.
This exception is raised when the A2A agent reports a task
in the 'canceled' state, indicating the task was canceled
either by the user or the system.
"""

View File

@@ -0,0 +1,56 @@
"""Type protocols for A2A SDK components.
These protocols define the expected interfaces for A2A SDK types,
allowing for type checking without requiring the SDK to be installed.
"""
from collections.abc import AsyncIterator
from typing import Any, Protocol, runtime_checkable
@runtime_checkable
class AgentCardProtocol(Protocol):
"""Protocol for A2A AgentCard."""
name: str
version: str
description: str
skills: list[Any]
capabilities: Any
@runtime_checkable
class ClientProtocol(Protocol):
"""Protocol for A2A Client."""
async def send_message(self, message: Any) -> AsyncIterator[Any]:
"""Send message to A2A agent."""
...
async def get_card(self) -> AgentCardProtocol:
"""Get agent card."""
...
async def close(self) -> None:
"""Close client connection."""
...
@runtime_checkable
class MessageProtocol(Protocol):
"""Protocol for A2A Message."""
role: Any
message_id: str
parts: list[Any]
@runtime_checkable
class TaskProtocol(Protocol):
"""Protocol for A2A Task."""
id: str
context_id: str
status: Any
history: list[Any] | None
artifacts: list[Any] | None

View File

@@ -2,7 +2,7 @@ from collections.abc import Sequence
import threading
from typing import Any
from crewai.agent.core import Agent
from crewai.agent import Agent
from crewai.agents.agent_builder.base_agent import BaseAgent
from crewai.events.event_bus import crewai_event_bus
from crewai.events.types.agent_events import (

View File

@@ -1,21 +1,5 @@
from crewai.flow.flow import Flow, and_, listen, or_, router, start
from crewai.flow.persistence import persist
from crewai.flow.visualization import (
FlowStructure,
build_flow_structure,
visualize_flow_structure,
)
__all__ = [
"Flow",
"FlowStructure",
"and_",
"build_flow_structure",
"listen",
"or_",
"persist",
"router",
"start",
"visualize_flow_structure",
]
__all__ = ["Flow", "and_", "listen", "or_", "persist", "router", "start"]

View File

@@ -0,0 +1,93 @@
<!DOCTYPE html>
<html>
<head>
<meta charset="utf-8" />
<title>{{ title }}</title>
<script
src="https://cdnjs.cloudflare.com/ajax/libs/vis-network/9.1.2/dist/vis-network.min.js"
integrity="sha512-LnvoEWDFrqGHlHmDD2101OrLcbsfkrzoSpvtSQtxK3RMnRV0eOkhhBN2dXHKRrUU8p2DGRTk35n4O8nWSVe1mQ=="
crossorigin="anonymous"
referrerpolicy="no-referrer"
></script>
<link
rel="stylesheet"
href="https://cdnjs.cloudflare.com/ajax/libs/vis-network/9.1.2/dist/dist/vis-network.min.css"
integrity="sha512-WgxfT5LWjfszlPHXRmBWHkV2eceiWTOBvrKCNbdgDYTHrT2AeLCGbF4sZlZw3UMN3WtL0tGUoIAKsu8mllg/XA=="
crossorigin="anonymous"
referrerpolicy="no-referrer"
/>
<style type="text/css">
body {
font-family: verdana;
margin: 0;
padding: 0;
}
.container {
display: flex;
flex-direction: column;
height: 100vh;
}
#mynetwork {
flex-grow: 1;
width: 100%;
height: 750px;
background-color: #ffffff;
}
.card {
border: none;
}
.legend-container {
display: flex;
align-items: center;
justify-content: center;
padding: 10px;
background-color: #f8f9fa;
position: fixed; /* Make the legend fixed */
bottom: 0; /* Position it at the bottom */
width: 100%; /* Make it span the full width */
}
.legend-item {
display: flex;
align-items: center;
margin-right: 20px;
}
.legend-color-box {
width: 20px;
height: 20px;
margin-right: 5px;
}
.logo {
height: 50px;
margin-right: 20px;
}
.legend-dashed {
border-bottom: 2px dashed #666666;
width: 20px;
height: 0;
margin-right: 5px;
}
.legend-solid {
border-bottom: 2px solid #666666;
width: 20px;
height: 0;
margin-right: 5px;
}
</style>
</head>
<body>
<div class="container">
<div class="card" style="width: 100%">
<div id="mynetwork" class="card-body"></div>
</div>
<div class="legend-container">
<img
src="data:image/svg+xml;base64,{{ logo_svg_base64 }}"
alt="CrewAI logo"
class="logo"
/>
<!-- LEGEND_ITEMS_PLACEHOLDER -->
</div>
</div>
{{ network_content }}
</body>
</html>

File diff suppressed because one or more lines are too long

After

Width:  |  Height:  |  Size: 27 KiB

View File

@@ -1,4 +0,0 @@
from typing import Final, Literal
AND_CONDITION: Final[Literal["AND"]] = "AND"
OR_CONDITION: Final[Literal["OR"]] = "OR"

View File

@@ -1,9 +1,3 @@
"""Core flow execution framework with decorators and state management.
This module provides the Flow class and decorators (@start, @listen, @router)
for building event-driven workflows with conditional execution and routing.
"""
from __future__ import annotations
import asyncio
@@ -44,7 +38,7 @@ from crewai.events.types.flow_events import (
MethodExecutionFinishedEvent,
MethodExecutionStartedEvent,
)
from crewai.flow.constants import AND_CONDITION, OR_CONDITION
from crewai.flow.flow_visualizer import plot_flow
from crewai.flow.flow_wrappers import (
FlowCondition,
FlowConditions,
@@ -57,16 +51,14 @@ from crewai.flow.flow_wrappers import (
from crewai.flow.persistence.base import FlowPersistence
from crewai.flow.types import FlowExecutionData, FlowMethodName, PendingListenerKey
from crewai.flow.utils import (
_extract_all_methods,
_normalize_condition,
get_possible_return_constants,
is_flow_condition_dict,
is_flow_condition_list,
is_flow_method,
is_flow_method_callable,
is_flow_method_name,
is_simple_flow_condition,
)
from crewai.flow.visualization import build_flow_structure, render_interactive
from crewai.utilities.printer import Printer, PrinterColor
@@ -82,63 +74,95 @@ class FlowState(BaseModel):
)
T = TypeVar("T", bound=dict[str, Any] | BaseModel)
P = ParamSpec("P")
R = TypeVar("R")
F = TypeVar("F", bound=Callable[..., Any])
# type variables with explicit bounds
T = TypeVar("T", bound=dict[str, Any] | BaseModel) # Generic flow state type parameter
StateT = TypeVar(
"StateT", bound=dict[str, Any] | BaseModel
) # State validation type parameter
P = ParamSpec("P") # ParamSpec for preserving function signatures in decorators
R = TypeVar("R") # Generic return type for decorated methods
F = TypeVar("F", bound=Callable[..., Any]) # Function type for decorator preservation
def ensure_state_type(state: Any, expected_type: type[StateT]) -> StateT:
"""Ensure state matches expected type with proper validation.
Args:
state: State instance to validate
expected_type: Expected type for the state
Returns:
Validated state instance
Raises:
TypeError: If state doesn't match expected type
ValueError: If state validation fails
"""
if expected_type is dict:
if not isinstance(state, dict):
raise TypeError(f"Expected dict, got {type(state).__name__}")
return cast(StateT, state)
if isinstance(expected_type, type) and issubclass(expected_type, BaseModel):
if not isinstance(state, expected_type):
raise TypeError(
f"Expected {expected_type.__name__}, got {type(state).__name__}"
)
return state
raise TypeError(f"Invalid expected_type: {expected_type}")
def start(
condition: str | FlowCondition | Callable[..., Any] | None = None,
) -> Callable[[Callable[P, R]], StartMethod[P, R]]:
"""Marks a method as a flow's starting point.
"""
Marks a method as a flow's starting point.
This decorator designates a method as an entry point for the flow execution.
It can optionally specify conditions that trigger the start based on other
method executions.
Args:
condition: Defines when the start method should execute. Can be:
- str: Name of a method that triggers this start
- FlowCondition: Result from or_() or and_(), including nested conditions
- Callable[..., Any]: A method reference that triggers this start
Default is None, meaning unconditional start.
Parameters
----------
condition : Optional[Union[str, FlowCondition, Callable[..., Any]]], optional
Defines when the start method should execute. Can be:
- str: Name of a method that triggers this start
- FlowCondition: Result from or_() or and_(), including nested conditions
- Callable[..., Any]: A method reference that triggers this start
Default is None, meaning unconditional start.
Returns:
A decorator function that wraps the method as a flow start point and preserves its signature.
Returns
-------
Callable[[Callable[P, R]], StartMethod[P, R]]
A decorator function that wraps the method as a flow start point
and preserves its signature.
Raises:
ValueError: If the condition format is invalid.
Raises
------
ValueError
If the condition format is invalid.
Examples:
>>> @start() # Unconditional start
>>> def begin_flow(self):
... pass
Examples
--------
>>> @start() # Unconditional start
>>> def begin_flow(self):
... pass
>>> @start("method_name") # Start after specific method
>>> def conditional_start(self):
... pass
>>> @start("method_name") # Start after specific method
>>> def conditional_start(self):
... pass
>>> @start(and_("method1", "method2")) # Start after multiple methods
>>> def complex_start(self):
... pass
>>> @start(and_("method1", "method2")) # Start after multiple methods
>>> def complex_start(self):
... pass
"""
def decorator(func: Callable[P, R]) -> StartMethod[P, R]:
"""Decorator that wraps a function as a start method.
Args:
func: The function to wrap as a start method.
Returns:
A StartMethod wrapper around the function.
"""
wrapper = StartMethod(func)
if condition is not None:
if is_flow_method_name(condition):
wrapper.__trigger_methods__ = [condition]
wrapper.__condition_type__ = OR_CONDITION
wrapper.__condition_type__ = "OR"
elif is_flow_condition_dict(condition):
if "conditions" in condition:
wrapper.__trigger_condition__ = condition
@@ -153,7 +177,7 @@ def start(
)
elif is_flow_method_callable(condition):
wrapper.__trigger_methods__ = [condition.__name__]
wrapper.__condition_type__ = OR_CONDITION
wrapper.__condition_type__ = "OR"
else:
raise ValueError(
"Condition must be a method, string, or a result of or_() or and_()"
@@ -166,45 +190,49 @@ def start(
def listen(
condition: str | FlowCondition | Callable[..., Any],
) -> Callable[[Callable[P, R]], ListenMethod[P, R]]:
"""Creates a listener that executes when specified conditions are met.
"""
Creates a listener that executes when specified conditions are met.
This decorator sets up a method to execute in response to other method
executions in the flow. It supports both simple and complex triggering
conditions.
Args:
condition: Specifies when the listener should execute.
Parameters
----------
condition : Union[str, FlowCondition, Callable[..., Any]]
Specifies when the listener should execute. Can be:
- str: Name of a method that triggers this listener
- FlowCondition: Result from or_() or and_(), including nested conditions
- Callable[..., Any]: A method reference that triggers this listener
Returns:
A decorator function that wraps the method as a flow listener and preserves its signature.
Returns
-------
Callable[[Callable[P, R]], ListenMethod[P, R]]
A decorator function that wraps the method as a listener
and preserves its signature.
Raises:
ValueError: If the condition format is invalid.
Raises
------
ValueError
If the condition format is invalid.
Examples:
>>> @listen("process_data")
>>> def handle_processed_data(self):
... pass
Examples
--------
>>> @listen("process_data") # Listen to single method
>>> def handle_processed_data(self):
... pass
>>> @listen("method_name")
>>> def handle_completion(self):
... pass
>>> @listen(or_("success", "failure")) # Listen to multiple methods
>>> def handle_completion(self):
... pass
"""
def decorator(func: Callable[P, R]) -> ListenMethod[P, R]:
"""Decorator that wraps a function as a listener method.
Args:
func: The function to wrap as a listener method.
Returns:
A ListenMethod wrapper around the function.
"""
wrapper = ListenMethod(func)
if is_flow_method_name(condition):
wrapper.__trigger_methods__ = [condition]
wrapper.__condition_type__ = OR_CONDITION
wrapper.__condition_type__ = "OR"
elif is_flow_condition_dict(condition):
if "conditions" in condition:
wrapper.__trigger_condition__ = condition
@@ -219,7 +247,7 @@ def listen(
)
elif is_flow_method_callable(condition):
wrapper.__trigger_methods__ = [condition.__name__]
wrapper.__condition_type__ = OR_CONDITION
wrapper.__condition_type__ = "OR"
else:
raise ValueError(
"Condition must be a method, string, or a result of or_() or and_()"
@@ -232,53 +260,54 @@ def listen(
def router(
condition: str | FlowCondition | Callable[..., Any],
) -> Callable[[Callable[P, R]], RouterMethod[P, R]]:
"""Creates a routing method that directs flow execution based on conditions.
"""
Creates a routing method that directs flow execution based on conditions.
This decorator marks a method as a router, which can dynamically determine
the next steps in the flow based on its return value. Routers are triggered
by specified conditions and can return constants that determine which path
the flow should take.
Args:
condition: Specifies when the router should execute. Can be:
- str: Name of a method that triggers this router
- FlowCondition: Result from or_() or and_(), including nested conditions
- Callable[..., Any]: A method reference that triggers this router
Parameters
----------
condition : Union[str, FlowCondition, Callable[..., Any]]
Specifies when the router should execute. Can be:
- str: Name of a method that triggers this router
- FlowCondition: Result from or_() or and_(), including nested conditions
- Callable[..., Any]: A method reference that triggers this router
Returns:
A decorator function that wraps the method as a router and preserves its signature.
Returns
-------
Callable[[Callable[P, R]], RouterMethod[P, R]]
A decorator function that wraps the method as a router
and preserves its signature.
Raises:
ValueError: If the condition format is invalid.
Raises
------
ValueError
If the condition format is invalid.
Examples:
>>> @router("check_status")
>>> def route_based_on_status(self):
... if self.state.status == "success":
... return "SUCCESS"
... return "FAILURE"
Examples
--------
>>> @router("check_status")
>>> def route_based_on_status(self):
... if self.state.status == "success":
... return SUCCESS
... return FAILURE
>>> @router(and_("validate", "process"))
>>> def complex_routing(self):
... if all([self.state.valid, self.state.processed]):
... return "CONTINUE"
... return "STOP"
>>> @router(and_("validate", "process"))
>>> def complex_routing(self):
... if all([self.state.valid, self.state.processed]):
... return CONTINUE
... return STOP
"""
def decorator(func: Callable[P, R]) -> RouterMethod[P, R]:
"""Decorator that wraps a function as a router method.
Args:
func: The function to wrap as a router method.
Returns:
A RouterMethod wrapper around the function.
"""
wrapper = RouterMethod(func)
if is_flow_method_name(condition):
wrapper.__trigger_methods__ = [condition]
wrapper.__condition_type__ = OR_CONDITION
wrapper.__condition_type__ = "OR"
elif is_flow_condition_dict(condition):
if "conditions" in condition:
wrapper.__trigger_condition__ = condition
@@ -293,7 +322,7 @@ def router(
)
elif is_flow_method_callable(condition):
wrapper.__trigger_methods__ = [condition.__name__]
wrapper.__condition_type__ = OR_CONDITION
wrapper.__condition_type__ = "OR"
else:
raise ValueError(
"Condition must be a method, string, or a result of or_() or and_()"
@@ -304,29 +333,42 @@ def router(
def or_(*conditions: str | FlowCondition | Callable[..., Any]) -> FlowCondition:
"""Combines multiple conditions with OR logic for flow control.
"""
Combines multiple conditions with OR logic for flow control.
Creates a condition that is satisfied when any of the specified conditions
are met. This is used with @start, @listen, or @router decorators to create
complex triggering conditions.
Args:
conditions: Variable number of conditions that can be method names, existing condition dictionaries, or method references.
Parameters
----------
*conditions : Union[str, dict[str, Any], Callable[..., Any]]
Variable number of conditions that can be:
- str: Method names
- dict[str, Any]: Existing condition dictionaries (nested conditions)
- Callable[..., Any]: Method references
Returns:
A condition dictionary with format {"type": "OR", "conditions": list_of_conditions} where each condition can be a string (method name) or a nested dict
Returns
-------
dict[str, Any]
A condition dictionary with format:
{"type": "OR", "conditions": list_of_conditions}
where each condition can be a string (method name) or a nested dict
Raises:
ValueError: If condition format is invalid.
Raises
------
ValueError
If any condition is invalid.
Examples:
>>> @listen(or_("success", "timeout"))
>>> def handle_completion(self):
... pass
Examples
--------
>>> @listen(or_("success", "timeout"))
>>> def handle_completion(self):
... pass
>>> @listen(or_(and_("step1", "step2"), "step3"))
>>> def handle_nested(self):
... pass
>>> @listen(or_(and_("step1", "step2"), "step3"))
>>> def handle_nested(self):
... pass
"""
processed_conditions: FlowConditions = []
for condition in conditions:
@@ -336,34 +378,46 @@ def or_(*conditions: str | FlowCondition | Callable[..., Any]) -> FlowCondition:
processed_conditions.append(condition.__name__)
else:
raise ValueError("Invalid condition in or_()")
return {"type": OR_CONDITION, "conditions": processed_conditions}
return {"type": "OR", "conditions": processed_conditions}
def and_(*conditions: str | FlowCondition | Callable[..., Any]) -> FlowCondition:
"""Combines multiple conditions with AND logic for flow control.
"""
Combines multiple conditions with AND logic for flow control.
Creates a condition that is satisfied only when all specified conditions
are met. This is used with @start, @listen, or @router decorators to create
complex triggering conditions.
Args:
*conditions: Variable number of conditions that can be method names, existing condition dictionaries, or method references.
Parameters
----------
*conditions : Union[str, dict[str, Any], Callable[..., Any]]
Variable number of conditions that can be:
- str: Method names
- dict[str, Any]: Existing condition dictionaries (nested conditions)
- Callable[..., Any]: Method references
Returns:
A condition dictionary with format {"type": "AND", "conditions": list_of_conditions}
Returns
-------
dict[str, Any]
A condition dictionary with format:
{"type": "AND", "conditions": list_of_conditions}
where each condition can be a string (method name) or a nested dict
Raises:
ValueError: If any condition is invalid.
Raises
------
ValueError
If any condition is invalid.
Examples:
>>> @listen(and_("validated", "processed"))
>>> def handle_complete_data(self):
... pass
Examples
--------
>>> @listen(and_("validated", "processed"))
>>> def handle_complete_data(self):
... pass
>>> @listen(and_(or_("step1", "step2"), "step3"))
>>> def handle_nested(self):
... pass
>>> @listen(and_(or_("step1", "step2"), "step3"))
>>> def handle_nested(self):
... pass
"""
processed_conditions: FlowConditions = []
for condition in conditions:
@@ -373,7 +427,59 @@ def and_(*conditions: str | FlowCondition | Callable[..., Any]) -> FlowCondition
processed_conditions.append(condition.__name__)
else:
raise ValueError("Invalid condition in and_()")
return {"type": AND_CONDITION, "conditions": processed_conditions}
return {"type": "AND", "conditions": processed_conditions}
def _normalize_condition(
condition: FlowConditions | FlowCondition | FlowMethodName,
) -> FlowCondition:
"""Normalize a condition to standard format with 'conditions' key.
Args:
condition: Can be a string (method name), dict (condition), or list
Returns:
Normalized dict with 'type' and 'conditions' keys
"""
if is_flow_method_name(condition):
return {"type": "OR", "conditions": [condition]}
if is_flow_condition_dict(condition):
if "conditions" in condition:
return condition
if "methods" in condition:
return {"type": condition["type"], "conditions": condition["methods"]}
return condition
if is_flow_condition_list(condition):
return {"type": "OR", "conditions": condition}
raise ValueError(f"Cannot normalize condition: {condition}")
def _extract_all_methods(
condition: str | FlowCondition | dict[str, Any] | list[Any],
) -> list[FlowMethodName]:
"""Extract all method names from a condition (including nested).
Args:
condition: Can be a string, dict, or list
Returns:
List of all method names in the condition tree
"""
if is_flow_method_name(condition):
return [condition]
if is_flow_condition_dict(condition):
normalized = _normalize_condition(condition)
methods = []
for sub_cond in normalized.get("conditions", []):
methods.extend(_extract_all_methods(sub_cond))
return methods
if isinstance(condition, list):
methods = []
for item in condition:
methods.extend(_extract_all_methods(item))
return methods
return []
class FlowMeta(type):
@@ -409,9 +515,7 @@ class FlowMeta(type):
and attr_value.__trigger_methods__ is not None
):
methods = attr_value.__trigger_methods__
condition_type = getattr(
attr_value, "__condition_type__", OR_CONDITION
)
condition_type = getattr(attr_value, "__condition_type__", "OR")
if (
hasattr(attr_value, "__trigger_condition__")
and attr_value.__trigger_condition__ is not None
@@ -428,8 +532,6 @@ class FlowMeta(type):
possible_returns = get_possible_return_constants(attr_value)
if possible_returns:
router_paths[attr_name] = possible_returns
else:
router_paths[attr_name] = []
cls._start_methods = start_methods # type: ignore[attr-defined]
cls._listeners = listeners # type: ignore[attr-defined]
@@ -454,7 +556,7 @@ class Flow(Generic[T], metaclass=FlowMeta):
name: str | None = None
tracing: bool | None = False
def __class_getitem__(cls: type[Flow[T]], item: type[T]) -> type[Flow[T]]:
def __class_getitem__(cls: type[Flow[StateT]], item: type[T]) -> type[Flow[StateT]]:
class _FlowGeneric(cls): # type: ignore
_initial_state_t = item
@@ -494,7 +596,7 @@ class Flow(Generic[T], metaclass=FlowMeta):
or should_auto_collect_first_time_traces()
):
trace_listener = TraceCollectionListener()
trace_listener.setup_listeners(crewai_event_bus)
trace_listener.setup_listeners(crewai_event_bus) # type: ignore[no-untyped-call]
# Apply any additional kwargs
if kwargs:
self._initialize_state(kwargs)
@@ -600,26 +702,7 @@ class Flow(Generic[T], metaclass=FlowMeta):
)
def _copy_state(self) -> T:
"""Create a copy of the current state.
Returns:
A copy of the current state
"""
if isinstance(self._state, BaseModel):
try:
return self._state.model_copy(deep=True)
except (TypeError, AttributeError):
try:
state_dict = self._state.model_dump()
model_class = type(self._state)
return model_class(**state_dict)
except Exception:
return self._state.model_copy(deep=False)
else:
try:
return copy.deepcopy(self._state)
except (TypeError, AttributeError):
return cast(T, self._state.copy())
return copy.deepcopy(self._state)
@property
def state(self) -> T:
@@ -944,8 +1027,8 @@ class Flow(Generic[T], metaclass=FlowMeta):
trace_listener = TraceCollectionListener()
if trace_listener.batch_manager.batch_owner_type == "flow":
if trace_listener.first_time_handler.is_first_time:
trace_listener.first_time_handler.mark_events_collected()
trace_listener.first_time_handler.handle_execution_completion()
trace_listener.first_time_handler.mark_events_collected() # type: ignore[no-untyped-call]
trace_listener.first_time_handler.handle_execution_completion() # type: ignore[no-untyped-call]
else:
trace_listener.batch_manager.finalize_batch()
@@ -954,20 +1037,24 @@ class Flow(Generic[T], metaclass=FlowMeta):
detach(flow_token)
async def _execute_start_method(self, start_method_name: FlowMethodName) -> None:
"""Executes a flow's start method and its triggered listeners.
"""
Executes a flow's start method and its triggered listeners.
This internal method handles the execution of methods marked with @start
decorator and manages the subsequent chain of listener executions.
Args:
start_method_name: The name of the start method to execute.
Parameters
----------
start_method_name : str
The name of the start method to execute.
Note:
- Executes the start method and captures its result
- Triggers execution of any listeners waiting on this start method
- Part of the flow's initialization sequence
- Skips execution if method was already completed (e.g., after reload)
- Automatically injects crewai_trigger_payload if available in flow inputs
Notes
-----
- Executes the start method and captures its result
- Triggers execution of any listeners waiting on this start method
- Part of the flow's initialization sequence
- Skips execution if method was already completed (e.g., after reload)
- Automatically injects crewai_trigger_payload if available in flow inputs
"""
if start_method_name in self._completed_methods:
if self._is_execution_resuming:
@@ -1087,21 +1174,27 @@ class Flow(Generic[T], metaclass=FlowMeta):
async def _execute_listeners(
self, trigger_method: FlowMethodName, result: Any
) -> None:
"""Executes all listeners and routers triggered by a method completion.
"""
Executes all listeners and routers triggered by a method completion.
This internal method manages the execution flow by:
1. First executing all triggered routers sequentially
2. Then executing all triggered listeners in parallel
Args:
trigger_method: The name of the method that triggered these listeners.
result: The result from the triggering method, passed to listeners that accept parameters.
Parameters
----------
trigger_method : str
The name of the method that triggered these listeners.
result : Any
The result from the triggering method, passed to listeners
that accept parameters.
Note:
- Routers are executed sequentially to maintain flow control
- Each router's result becomes a new trigger_method
- Normal listeners are executed in parallel for efficiency
- Listeners can receive the trigger method's result as a parameter
Notes
-----
- Routers are executed sequentially to maintain flow control
- Each router's result becomes a new trigger_method
- Normal listeners are executed in parallel for efficiency
- Listeners can receive the trigger method's result as a parameter
"""
# First, handle routers repeatedly until no router triggers anymore
router_results = []
@@ -1188,16 +1281,16 @@ class Flow(Generic[T], metaclass=FlowMeta):
if is_flow_condition_dict(condition):
normalized = _normalize_condition(condition)
cond_type = normalized.get("type", OR_CONDITION)
cond_type = normalized.get("type", "OR")
sub_conditions = normalized.get("conditions", [])
if cond_type == OR_CONDITION:
if cond_type == "OR":
return any(
self._evaluate_condition(sub_cond, trigger_method, listener_name)
for sub_cond in sub_conditions
)
if cond_type == AND_CONDITION:
if cond_type == "AND":
pending_key = PendingListenerKey(f"{listener_name}:{id(condition)}")
if pending_key not in self._pending_and_listeners:
@@ -1207,20 +1300,7 @@ class Flow(Generic[T], metaclass=FlowMeta):
if trigger_method in self._pending_and_listeners[pending_key]:
self._pending_and_listeners[pending_key].discard(trigger_method)
direct_methods_satisfied = not self._pending_and_listeners[pending_key]
nested_conditions_satisfied = all(
(
self._evaluate_condition(
sub_cond, trigger_method, listener_name
)
if is_flow_condition_dict(sub_cond)
else True
)
for sub_cond in sub_conditions
)
if direct_methods_satisfied and nested_conditions_satisfied:
if not self._pending_and_listeners[pending_key]:
self._pending_and_listeners.pop(pending_key, None)
return True
@@ -1231,22 +1311,30 @@ class Flow(Generic[T], metaclass=FlowMeta):
def _find_triggered_methods(
self, trigger_method: FlowMethodName, router_only: bool
) -> list[FlowMethodName]:
"""Finds all methods that should be triggered based on conditions.
"""
Finds all methods that should be triggered based on conditions.
This internal method evaluates both OR and AND conditions to determine
which methods should be executed next in the flow. Supports nested conditions.
Args:
trigger_method: The name of the method that just completed execution.
router_only: If True, only consider router methods. If False, only consider non-router methods.
Parameters
----------
trigger_method : str
The name of the method that just completed execution.
router_only : bool
If True, only consider router methods.
If False, only consider non-router methods.
Returns:
Returns
-------
list[str]
Names of methods that should be triggered.
Note:
- Handles both OR and AND conditions, including nested combinations
- Maintains state for AND conditions using _pending_and_listeners
- Separates router and normal listener evaluation
Notes
-----
- Handles both OR and AND conditions, including nested combinations
- Maintains state for AND conditions using _pending_and_listeners
- Separates router and normal listener evaluation
"""
triggered: list[FlowMethodName] = []
@@ -1262,10 +1350,10 @@ class Flow(Generic[T], metaclass=FlowMeta):
if is_simple_flow_condition(condition_data):
condition_type, methods = condition_data
if condition_type == OR_CONDITION:
if condition_type == "OR":
if trigger_method in methods:
triggered.append(listener_name)
elif condition_type == AND_CONDITION:
elif condition_type == "AND":
pending_key = PendingListenerKey(listener_name)
if pending_key not in self._pending_and_listeners:
self._pending_and_listeners[pending_key] = set(methods)
@@ -1287,23 +1375,33 @@ class Flow(Generic[T], metaclass=FlowMeta):
async def _execute_single_listener(
self, listener_name: FlowMethodName, result: Any
) -> None:
"""Executes a single listener method with proper event handling.
"""
Executes a single listener method with proper event handling.
This internal method manages the execution of an individual listener,
including parameter inspection, event emission, and error handling.
Args:
listener_name: The name of the listener method to execute.
result: The result from the triggering method, which may be passed to the listener if it accepts parameters.
Parameters
----------
listener_name : str
The name of the listener method to execute.
result : Any
The result from the triggering method, which may be passed
to the listener if it accepts parameters.
Note:
- Inspects method signature to determine if it accepts the trigger result
- Emits events for method execution start and finish
- Handles errors gracefully with detailed logging
- Recursively triggers listeners of this listener
- Supports both parameterized and parameter-less listeners
- Skips execution if method was already completed (e.g., after reload)
- Catches and logs any exceptions during execution, preventing individual listener failures from breaking the entire flow
Notes
-----
- Inspects method signature to determine if it accepts the trigger result
- Emits events for method execution start and finish
- Handles errors gracefully with detailed logging
- Recursively triggers listeners of this listener
- Supports both parameterized and parameter-less listeners
- Skips execution if method was already completed (e.g., after reload)
Error Handling
-------------
Catches and logs any exceptions during execution, preventing
individual listener failures from breaking the entire flow.
"""
if listener_name in self._completed_methods:
if self._is_execution_resuming:
@@ -1362,16 +1460,7 @@ class Flow(Generic[T], metaclass=FlowMeta):
logger.info(message)
logger.warning(message)
def plot(self, filename: str = "crewai_flow.html", show: bool = True) -> str:
"""Create interactive HTML visualization of Flow structure.
Args:
filename: Output HTML filename (default: "crewai_flow.html").
show: Whether to open in browser (default: True).
Returns:
Absolute path to generated HTML file.
"""
def plot(self, filename: str = "crewai_flow") -> None:
crewai_event_bus.emit(
self,
FlowPlotEvent(
@@ -1379,5 +1468,4 @@ class Flow(Generic[T], metaclass=FlowMeta):
flow_name=self.name or self.__class__.__name__,
),
)
structure = build_flow_structure(self)
return render_interactive(structure, filename=filename, show=show)
plot_flow(self, filename)

View File

@@ -0,0 +1,234 @@
# flow_visualizer.py
from __future__ import annotations
import os
from typing import TYPE_CHECKING, Any
from pyvis.network import Network # type: ignore[import-untyped]
from crewai.flow.config import COLORS, NODE_STYLES, NodeStyles
from crewai.flow.html_template_handler import HTMLTemplateHandler
from crewai.flow.legend_generator import generate_legend_items_html, get_legend_items
from crewai.flow.path_utils import safe_path_join
from crewai.flow.utils import calculate_node_levels
from crewai.flow.visualization_utils import (
add_edges,
add_nodes_to_network,
compute_positions,
)
from crewai.utilities.printer import Printer
if TYPE_CHECKING:
from crewai.flow.flow import Flow
_printer = Printer()
class FlowPlot:
"""Handles the creation and rendering of flow visualization diagrams."""
def __init__(self, flow: Flow[Any]) -> None:
"""
Initialize FlowPlot with a flow object.
Parameters
----------
flow : Flow
A Flow instance to visualize.
Raises
------
ValueError
If flow object is invalid or missing required attributes.
"""
self.flow = flow
self.colors = COLORS
self.node_styles: NodeStyles = NODE_STYLES
def plot(self, filename: str) -> None:
"""
Generate and save an HTML visualization of the flow.
Parameters
----------
filename : str
Name of the output file (without extension).
Raises
------
ValueError
If filename is invalid or network generation fails.
IOError
If file operations fail or visualization cannot be generated.
RuntimeError
If network visualization generation fails.
"""
try:
# Initialize network
net = Network(directed=True, height="750px", bgcolor=self.colors["bg"])
# Set options to disable physics
net.set_options(
"""
var options = {
"nodes": {
"font": {
"multi": "html"
}
},
"physics": {
"enabled": false
}
}
"""
)
# Calculate levels for nodes
try:
node_levels = calculate_node_levels(self.flow)
except Exception as e:
raise ValueError(f"Failed to calculate node levels: {e!s}") from e
# Compute positions
try:
node_positions = compute_positions(self.flow, node_levels)
except Exception as e:
raise ValueError(f"Failed to compute node positions: {e!s}") from e
# Add nodes to the network
try:
add_nodes_to_network(net, self.flow, node_positions, self.node_styles)
except Exception as e:
raise RuntimeError(f"Failed to add nodes to network: {e!s}") from e
# Add edges to the network
try:
add_edges(net, self.flow, node_positions, self.colors)
except Exception as e:
raise RuntimeError(f"Failed to add edges to network: {e!s}") from e
# Generate HTML
try:
network_html = net.generate_html()
final_html_content = self._generate_final_html(network_html)
except Exception as e:
raise RuntimeError(
f"Failed to generate network visualization: {e!s}"
) from e
# Save the final HTML content to the file
try:
with open(f"{filename}.html", "w", encoding="utf-8") as f:
f.write(final_html_content)
_printer.print(f"Plot saved as {filename}.html", color="green")
except IOError as e:
raise IOError(
f"Failed to save flow visualization to {filename}.html: {e!s}"
) from e
except (ValueError, RuntimeError, IOError) as e:
raise e
except Exception as e:
raise RuntimeError(
f"Unexpected error during flow visualization: {e!s}"
) from e
finally:
self._cleanup_pyvis_lib(filename)
def _generate_final_html(self, network_html: str) -> str:
"""
Generate the final HTML content with network visualization and legend.
Parameters
----------
network_html : str
HTML content generated by pyvis Network.
Returns
-------
str
Complete HTML content with styling and legend.
Raises
------
IOError
If template or logo files cannot be accessed.
ValueError
If network_html is invalid.
"""
if not network_html:
raise ValueError("Invalid network HTML content")
try:
# Extract just the body content from the generated HTML
current_dir = os.path.dirname(__file__)
template_path = safe_path_join(
"assets", "crewai_flow_visual_template.html", root=current_dir
)
logo_path = safe_path_join("assets", "crewai_logo.svg", root=current_dir)
if not os.path.exists(template_path):
raise IOError(f"Template file not found: {template_path}")
if not os.path.exists(logo_path):
raise IOError(f"Logo file not found: {logo_path}")
html_handler = HTMLTemplateHandler(template_path, logo_path)
network_body = html_handler.extract_body_content(network_html)
# Generate the legend items HTML
legend_items = get_legend_items(self.colors)
legend_items_html = generate_legend_items_html(legend_items)
return html_handler.generate_final_html(network_body, legend_items_html)
except Exception as e:
raise IOError(f"Failed to generate visualization HTML: {e!s}") from e
@staticmethod
def _cleanup_pyvis_lib(filename: str) -> None:
"""
Clean up the generated lib folder from pyvis.
This method safely removes the temporary lib directory created by pyvis
during network visualization generation. The lib folder is created in the
same directory as the output HTML file.
Parameters
----------
filename : str
The output filename (without .html extension) used for the visualization.
"""
try:
import shutil
output_dir = os.path.dirname(os.path.abspath(filename)) or os.getcwd()
lib_folder = os.path.join(output_dir, "lib")
if os.path.exists(lib_folder) and os.path.isdir(lib_folder):
vis_js = os.path.join(lib_folder, "vis-network.min.js")
if os.path.exists(vis_js):
shutil.rmtree(lib_folder)
except Exception as e:
_printer.print(f"Error cleaning up lib folder: {e}", color="red")
def plot_flow(flow: Flow[Any], filename: str = "flow_plot") -> None:
"""
Convenience function to create and save a flow visualization.
Parameters
----------
flow : Flow
Flow instance to visualize.
filename : str, optional
Output filename without extension, by default "flow_plot".
Raises
------
ValueError
If flow object or filename is invalid.
IOError
If file operations fail.
"""
visualizer = FlowPlot(flow)
visualizer.plot(filename)

View File

@@ -5,6 +5,7 @@ from __future__ import annotations
from collections.abc import Callable, Sequence
import functools
import inspect
import types
from typing import Any, Generic, Literal, ParamSpec, TypeAlias, TypeVar, TypedDict
from typing_extensions import Required, Self
@@ -16,6 +17,8 @@ P = ParamSpec("P")
R = TypeVar("R")
FlowConditionType: TypeAlias = Literal["OR", "AND"]
# Simple flow condition stored as tuple (condition_type, method_list)
SimpleFlowCondition: TypeAlias = tuple[FlowConditionType, list[FlowMethodName]]
@@ -23,11 +26,6 @@ class FlowCondition(TypedDict, total=False):
"""Type definition for flow trigger conditions.
This is a recursive structure where conditions can contain nested FlowConditions.
Attributes:
type: The type of the condition.
conditions: A list of conditions types.
methods: A list of methods.
"""
type: Required[FlowConditionType]
@@ -81,7 +79,8 @@ class FlowMethod(Generic[P, R]):
The result of calling the wrapped method.
"""
if self._instance is not None:
return self._meth(self._instance, *args, **kwargs)
bound = types.MethodType(self._meth, self._instance)
return bound(*args, **kwargs)
return self._meth(*args, **kwargs)
def unwrap(self) -> Callable[P, R]:

View File

@@ -0,0 +1,91 @@
"""HTML template processing and generation for flow visualization diagrams."""
import base64
import re
from typing import Any
from crewai.flow.path_utils import validate_path_exists
class HTMLTemplateHandler:
"""Handles HTML template processing and generation for flow visualization diagrams."""
def __init__(self, template_path: str, logo_path: str) -> None:
"""
Initialize HTMLTemplateHandler with validated template and logo paths.
Parameters
----------
template_path : str
Path to the HTML template file.
logo_path : str
Path to the logo image file.
Raises
------
ValueError
If template or logo paths are invalid or files don't exist.
"""
try:
self.template_path = validate_path_exists(template_path, "file")
self.logo_path = validate_path_exists(logo_path, "file")
except ValueError as e:
raise ValueError(f"Invalid template or logo path: {e}") from e
def read_template(self) -> str:
"""Read and return the HTML template file contents."""
with open(self.template_path, "r", encoding="utf-8") as f:
return f.read()
def encode_logo(self) -> str:
"""Convert the logo SVG file to base64 encoded string."""
with open(self.logo_path, "rb") as logo_file:
logo_svg_data = logo_file.read()
return base64.b64encode(logo_svg_data).decode("utf-8")
def extract_body_content(self, html: str) -> str:
"""Extract and return content between body tags from HTML string."""
match = re.search("<body.*?>(.*?)</body>", html, re.DOTALL)
return match.group(1) if match else ""
def generate_legend_items_html(self, legend_items: list[dict[str, Any]]) -> str:
"""Generate HTML markup for the legend items."""
legend_items_html = ""
for item in legend_items:
if "border" in item:
legend_items_html += f"""
<div class="legend-item">
<div class="legend-color-box" style="background-color: {item["color"]}; border: 2px dashed {item["border"]};"></div>
<div>{item["label"]}</div>
</div>
"""
elif item.get("dashed") is not None:
style = "dashed" if item["dashed"] else "solid"
legend_items_html += f"""
<div class="legend-item">
<div class="legend-{style}" style="border-bottom: 2px {style} {item["color"]};"></div>
<div>{item["label"]}</div>
</div>
"""
else:
legend_items_html += f"""
<div class="legend-item">
<div class="legend-color-box" style="background-color: {item["color"]};"></div>
<div>{item["label"]}</div>
</div>
"""
return legend_items_html
def generate_final_html(
self, network_body: str, legend_items_html: str, title: str = "Flow Plot"
) -> str:
"""Combine all components into final HTML document with network visualization."""
html_template = self.read_template()
logo_svg_base64 = self.encode_logo()
return (
html_template.replace("{{ title }}", title)
.replace("{{ network_content }}", network_body)
.replace("{{ logo_svg_base64 }}", logo_svg_base64)
.replace("<!-- LEGEND_ITEMS_PLACEHOLDER -->", legend_items_html)
)

View File

@@ -0,0 +1,84 @@
"""Legend generation for flow visualization diagrams."""
from typing import Any
from crewai.flow.config import FlowColors
def get_legend_items(colors: FlowColors) -> list[dict[str, Any]]:
"""Generate legend items based on flow colors.
Parameters
----------
colors : FlowColors
Dictionary containing color definitions for flow elements.
Returns
-------
list[dict[str, Any]]
List of legend item dictionaries with labels and styling.
"""
return [
{"label": "Start Method", "color": colors["start"]},
{"label": "Method", "color": colors["method"]},
{
"label": "Crew Method",
"color": colors["bg"],
"border": colors["start"],
"dashed": False,
},
{
"label": "Router",
"color": colors["router"],
"border": colors["router_border"],
"dashed": True,
},
{"label": "Trigger", "color": colors["edge"], "dashed": False},
{"label": "AND Trigger", "color": colors["edge"], "dashed": True},
{
"label": "Router Trigger",
"color": colors["router_edge"],
"dashed": True,
},
]
def generate_legend_items_html(legend_items: list[dict[str, Any]]) -> str:
"""Generate HTML markup for legend items.
Parameters
----------
legend_items : list[dict[str, Any]]
List of legend item dictionaries containing labels and styling.
Returns
-------
str
HTML string containing formatted legend items.
"""
legend_items_html = ""
for item in legend_items:
if "border" in item:
style = "dashed" if item["dashed"] else "solid"
legend_items_html += f"""
<div class="legend-item">
<div class="legend-color-box" style="background-color: {item["color"]}; border: 2px {style} {item["border"]}; border-radius: 5px;"></div>
<div>{item["label"]}</div>
</div>
"""
elif item.get("dashed") is not None:
style = "dashed" if item["dashed"] else "solid"
legend_items_html += f"""
<div class="legend-item">
<div class="legend-{style}" style="border-bottom: 2px {style} {item["color"]}; border-radius: 5px;"></div>
<div>{item["label"]}</div>
</div>
"""
else:
legend_items_html += f"""
<div class="legend-item">
<div class="legend-color-box" style="background-color: {item["color"]}; border-radius: 5px;"></div>
<div>{item["label"]}</div>
</div>
"""
return legend_items_html

View File

@@ -0,0 +1,133 @@
"""
Path utilities for secure file operations in CrewAI flow module.
This module provides utilities for secure path handling to prevent directory
traversal attacks and ensure paths remain within allowed boundaries.
"""
from pathlib import Path
def safe_path_join(*parts: str, root: str | Path | None = None) -> str:
"""
Safely join path components and ensure the result is within allowed boundaries.
Parameters
----------
*parts : str
Variable number of path components to join.
root : Union[str, Path, None], optional
Root directory to use as base. If None, uses current working directory.
Returns
-------
str
String representation of the resolved path.
Raises
------
ValueError
If the resulting path would be outside the root directory
or if any path component is invalid.
"""
if not parts:
raise ValueError("No path components provided")
try:
# Convert all parts to strings and clean them
clean_parts = [str(part).strip() for part in parts if part]
if not clean_parts:
raise ValueError("No valid path components provided")
# Establish root directory
root_path = Path(root).resolve() if root else Path.cwd()
# Join and resolve the full path
full_path = Path(root_path, *clean_parts).resolve()
# Check if the resolved path is within root
if not str(full_path).startswith(str(root_path)):
raise ValueError(
f"Invalid path: Potential directory traversal. Path must be within {root_path}"
)
return str(full_path)
except Exception as e:
if isinstance(e, ValueError):
raise
raise ValueError(f"Invalid path components: {e!s}") from e
def validate_path_exists(path: str | Path, file_type: str = "file") -> str:
"""
Validate that a path exists and is of the expected type.
Parameters
----------
path : Union[str, Path]
Path to validate.
file_type : str, optional
Expected type ('file' or 'directory'), by default 'file'.
Returns
-------
str
Validated path as string.
Raises
------
ValueError
If path doesn't exist or is not of expected type.
"""
try:
path_obj = Path(path).resolve()
if not path_obj.exists():
raise ValueError(f"Path does not exist: {path}")
if file_type == "file" and not path_obj.is_file():
raise ValueError(f"Path is not a file: {path}")
if file_type == "directory" and not path_obj.is_dir():
raise ValueError(f"Path is not a directory: {path}")
return str(path_obj)
except Exception as e:
if isinstance(e, ValueError):
raise
raise ValueError(f"Invalid path: {e!s}") from e
def list_files(directory: str | Path, pattern: str = "*") -> list[str]:
"""
Safely list files in a directory matching a pattern.
Parameters
----------
directory : Union[str, Path]
Directory to search in.
pattern : str, optional
Glob pattern to match files against, by default "*".
Returns
-------
List[str]
List of matching file paths.
Raises
------
ValueError
If directory is invalid or inaccessible.
"""
try:
dir_path = Path(directory).resolve()
if not dir_path.is_dir():
raise ValueError(f"Not a directory: {directory}")
return [str(p) for p in dir_path.glob(pattern) if p.is_file()]
except Exception as e:
if isinstance(e, ValueError):
raise
raise ValueError(f"Error listing files: {e!s}") from e

View File

@@ -21,7 +21,6 @@ P = ParamSpec("P")
R = TypeVar("R", covariant=True)
FlowMethodName = NewType("FlowMethodName", str)
FlowRouteName = NewType("FlowRouteName", str)
PendingListenerKey = NewType(
"PendingListenerKey",
Annotated[str, "nested flow conditions use 'listener_name:object_id'"],

View File

@@ -13,17 +13,14 @@ Example
>>> ancestors = build_ancestor_dict(flow)
"""
from __future__ import annotations
import ast
from collections import defaultdict, deque
import inspect
import textwrap
from typing import TYPE_CHECKING, Any
from typing import Any
from typing_extensions import TypeIs
from crewai.flow.constants import AND_CONDITION, OR_CONDITION
from crewai.flow.flow_wrappers import (
FlowCondition,
FlowConditions,
@@ -34,29 +31,10 @@ from crewai.flow.types import FlowMethodCallable, FlowMethodName
from crewai.utilities.printer import Printer
if TYPE_CHECKING:
from crewai.flow.flow import Flow
_printer = Printer()
def get_possible_return_constants(function: Any) -> list[str] | None:
"""Extract possible string return values from a function using AST parsing.
This function analyzes the source code of a router method to identify
all possible string values it might return. It handles:
- Direct string literals: return "value"
- Variable assignments: x = "value"; return x
- Dictionary lookups: d = {"k": "v"}; return d[key]
- Conditional returns: return "a" if cond else "b"
- State attributes: return self.state.attr (infers from class context)
Args:
function: The function to analyze.
Returns:
List of possible string return values, or None if analysis fails.
"""
try:
source = inspect.getsource(function)
except OSError:
@@ -96,34 +74,11 @@ def get_possible_return_constants(function: Any) -> list[str] | None:
_printer.print(f"Source code:\n{source}", color="yellow")
return None
return_values: set[str] = set()
dict_definitions: dict[str, list[str]] = {}
variable_values: dict[str, list[str]] = {}
state_attribute_values: dict[str, list[str]] = {}
return_values = set()
dict_definitions = {}
def extract_string_constants(node: ast.expr) -> list[str]:
"""Recursively extract all string constants from an AST node."""
strings: list[str] = []
if isinstance(node, ast.Constant) and isinstance(node.value, str):
strings.append(node.value)
elif isinstance(node, ast.IfExp):
strings.extend(extract_string_constants(node.body))
strings.extend(extract_string_constants(node.orelse))
elif isinstance(node, ast.Call):
if (
isinstance(node.func, ast.Attribute)
and node.func.attr == "get"
and len(node.args) >= 2
):
default_arg = node.args[1]
if isinstance(default_arg, ast.Constant) and isinstance(
default_arg.value, str
):
strings.append(default_arg.value)
return strings
class VariableAssignmentVisitor(ast.NodeVisitor):
def visit_Assign(self, node: ast.Assign) -> None:
class DictionaryAssignmentVisitor(ast.NodeVisitor):
def visit_Assign(self, node):
# Check if this assignment is assigning a dictionary literal to a variable
if isinstance(node.value, ast.Dict) and len(node.targets) == 1:
target = node.targets[0]
@@ -137,142 +92,29 @@ def get_possible_return_constants(function: Any) -> list[str] | None:
]
if dict_values:
dict_definitions[var_name] = dict_values
if len(node.targets) == 1:
target = node.targets[0]
var_name_alt: str | None = None
if isinstance(target, ast.Name):
var_name_alt = target.id
elif isinstance(target, ast.Attribute):
var_name_alt = f"{target.value.id if isinstance(target.value, ast.Name) else '_'}.{target.attr}"
if var_name_alt:
strings = extract_string_constants(node.value)
if strings:
variable_values[var_name_alt] = strings
self.generic_visit(node)
def get_attribute_chain(node: ast.expr) -> str | None:
"""Extract the full attribute chain from an AST node.
Examples:
self.state.run_type -> "self.state.run_type"
x.y.z -> "x.y.z"
simple_var -> "simple_var"
"""
if isinstance(node, ast.Name):
return node.id
if isinstance(node, ast.Attribute):
base = get_attribute_chain(node.value)
if base:
return f"{base}.{node.attr}"
return None
class ReturnVisitor(ast.NodeVisitor):
def visit_Return(self, node: ast.Return) -> None:
if (
node.value
and isinstance(node.value, ast.Constant)
and isinstance(node.value.value, str)
def visit_Return(self, node):
# Direct string return
if isinstance(node.value, ast.Constant) and isinstance(
node.value.value, str
):
return_values.add(node.value.value)
elif node.value and isinstance(node.value, ast.Subscript):
# Dictionary-based return, like return paths[result]
elif isinstance(node.value, ast.Subscript):
# Check if we're subscripting a known dictionary variable
if isinstance(node.value.value, ast.Name):
var_name_dict = node.value.value.id
if var_name_dict in dict_definitions:
for v in dict_definitions[var_name_dict]:
var_name = node.value.value.id
if var_name in dict_definitions:
# Add all possible dictionary values
for v in dict_definitions[var_name]:
return_values.add(v)
elif node.value:
var_name_ret = get_attribute_chain(node.value)
if var_name_ret and var_name_ret in variable_values:
for v in variable_values[var_name_ret]:
return_values.add(v)
elif var_name_ret and var_name_ret in state_attribute_values:
for v in state_attribute_values[var_name_ret]:
return_values.add(v)
self.generic_visit(node)
def visit_If(self, node: ast.If) -> None:
self.generic_visit(node)
# Try to get the class context to infer state attribute values
try:
if hasattr(function, "__self__"):
# Method is bound, get the class
class_obj = function.__self__.__class__
elif hasattr(function, "__qualname__") and "." in function.__qualname__:
# Method is unbound but we can try to get class from module
class_name = function.__qualname__.rsplit(".", 1)[0]
if hasattr(function, "__globals__"):
class_obj = function.__globals__.get(class_name)
else:
class_obj = None
else:
class_obj = None
if class_obj is not None:
try:
class_source = inspect.getsource(class_obj)
class_source = textwrap.dedent(class_source)
class_ast = ast.parse(class_source)
# Look for comparisons and assignments involving state attributes
class StateAttributeVisitor(ast.NodeVisitor):
def visit_Compare(self, node: ast.Compare) -> None:
"""Find comparisons like: self.state.attr == "value" """
left_attr = get_attribute_chain(node.left)
if left_attr:
for comparator in node.comparators:
if isinstance(comparator, ast.Constant) and isinstance(
comparator.value, str
):
if left_attr not in state_attribute_values:
state_attribute_values[left_attr] = []
if (
comparator.value
not in state_attribute_values[left_attr]
):
state_attribute_values[left_attr].append(
comparator.value
)
# Also check right side
for comparator in node.comparators:
right_attr = get_attribute_chain(comparator)
if (
right_attr
and isinstance(node.left, ast.Constant)
and isinstance(node.left.value, str)
):
if right_attr not in state_attribute_values:
state_attribute_values[right_attr] = []
if (
node.left.value
not in state_attribute_values[right_attr]
):
state_attribute_values[right_attr].append(
node.left.value
)
self.generic_visit(node)
StateAttributeVisitor().visit(class_ast)
except Exception as e:
_printer.print(
f"Could not analyze class context for {function.__name__}: {e}",
color="yellow",
)
except Exception as e:
_printer.print(
f"Could not introspect class for {function.__name__}: {e}",
color="yellow",
)
VariableAssignmentVisitor().visit(code_ast)
# First pass: identify dictionary assignments
DictionaryAssignmentVisitor().visit(code_ast)
# Second pass: identify returns
ReturnVisitor().visit(code_ast)
return list(return_values) if return_values else None
@@ -316,15 +158,7 @@ def calculate_node_levels(flow: Any) -> dict[str, int]:
# Precompute listener dependencies
or_listeners = defaultdict(list)
and_listeners = defaultdict(set)
for listener_name, condition_data in flow._listeners.items():
if isinstance(condition_data, tuple):
condition_type, trigger_methods = condition_data
elif isinstance(condition_data, dict):
trigger_methods = _extract_all_methods_recursive(condition_data, flow)
condition_type = condition_data.get("type", "OR")
else:
continue
for listener_name, (condition_type, trigger_methods) in flow._listeners.items():
if condition_type == "OR":
for method in trigger_methods:
or_listeners[method].append(listener_name)
@@ -358,13 +192,9 @@ def calculate_node_levels(flow: Any) -> dict[str, int]:
if listener_name not in visited:
queue.append(listener_name)
# Handle router connections
process_router_paths(flow, current, current_level, levels, queue)
max_level = max(levels.values()) if levels else 0
for method_name in flow._methods:
if method_name not in levels:
levels[method_name] = max_level + 1
return levels
@@ -385,14 +215,8 @@ def count_outgoing_edges(flow: Any) -> dict[str, int]:
counts = {}
for method_name in flow._methods:
counts[method_name] = 0
for condition_data in flow._listeners.values():
if isinstance(condition_data, tuple):
_, trigger_methods = condition_data
elif isinstance(condition_data, dict):
trigger_methods = _extract_all_methods_recursive(condition_data, flow)
else:
continue
for method_name in flow._listeners:
_, trigger_methods = flow._listeners[method_name]
for trigger in trigger_methods:
if trigger in flow._methods:
counts[trigger] += 1
@@ -447,34 +271,21 @@ def dfs_ancestors(
return
visited.add(node)
for listener_name, condition_data in flow._listeners.items():
if isinstance(condition_data, tuple):
_, trigger_methods = condition_data
elif isinstance(condition_data, dict):
trigger_methods = _extract_all_methods_recursive(condition_data, flow)
else:
continue
# Handle regular listeners
for listener_name, (_, trigger_methods) in flow._listeners.items():
if node in trigger_methods:
ancestors[listener_name].add(node)
ancestors[listener_name].update(ancestors[node])
dfs_ancestors(listener_name, ancestors, visited, flow)
# Handle router methods separately
if node in flow._routers:
router_method_name = node
paths = flow._router_paths.get(router_method_name, [])
for path in paths:
for listener_name, condition_data in flow._listeners.items():
if isinstance(condition_data, tuple):
_, trigger_methods = condition_data
elif isinstance(condition_data, dict):
trigger_methods = _extract_all_methods_recursive(
condition_data, flow
)
else:
continue
for listener_name, (_, trigger_methods) in flow._listeners.items():
if path in trigger_methods:
# Only propagate the ancestors of the router method, not the router method itself
ancestors[listener_name].update(ancestors[node])
dfs_ancestors(listener_name, ancestors, visited, flow)
@@ -524,32 +335,19 @@ def build_parent_children_dict(flow: Any) -> dict[str, list[str]]:
"""
parent_children: dict[str, list[str]] = {}
for listener_name, condition_data in flow._listeners.items():
if isinstance(condition_data, tuple):
_, trigger_methods = condition_data
elif isinstance(condition_data, dict):
trigger_methods = _extract_all_methods_recursive(condition_data, flow)
else:
continue
# Map listeners to their trigger methods
for listener_name, (_, trigger_methods) in flow._listeners.items():
for trigger in trigger_methods:
if trigger not in parent_children:
parent_children[trigger] = []
if listener_name not in parent_children[trigger]:
parent_children[trigger].append(listener_name)
# Map router methods to their paths and to listeners
for router_method_name, paths in flow._router_paths.items():
for path in paths:
for listener_name, condition_data in flow._listeners.items():
if isinstance(condition_data, tuple):
_, trigger_methods = condition_data
elif isinstance(condition_data, dict):
trigger_methods = _extract_all_methods_recursive(
condition_data, flow
)
else:
continue
# Map router method to listeners of each path
for listener_name, (_, trigger_methods) in flow._listeners.items():
if path in trigger_methods:
if router_method_name not in parent_children:
parent_children[router_method_name] = []
@@ -584,27 +382,17 @@ def get_child_index(
return children.index(child)
def process_router_paths(
flow: Any,
current: str,
current_level: int,
levels: dict[str, int],
queue: deque[str],
) -> None:
"""Handle the router connections for the current node."""
def process_router_paths(flow, current, current_level, levels, queue):
"""
Handle the router connections for the current node.
"""
if current in flow._routers:
paths = flow._router_paths.get(current, [])
for path in paths:
for listener_name, condition_data in flow._listeners.items():
if isinstance(condition_data, tuple):
_condition_type, trigger_methods = condition_data
elif isinstance(condition_data, dict):
trigger_methods = _extract_all_methods_recursive(
condition_data, flow
)
else:
continue
for listener_name, (
_condition_type,
trigger_methods,
) in flow._listeners.items():
if path in trigger_methods:
if (
listener_name not in levels
@@ -625,7 +413,7 @@ def is_flow_method_name(obj: Any) -> TypeIs[FlowMethodName]:
return isinstance(obj, str)
def is_flow_method_callable(obj: Any) -> TypeIs[FlowMethodCallable[..., Any]]:
def is_flow_method_callable(obj: Any) -> TypeIs[FlowMethodCallable]:
"""Check if the object is a callable flow method.
Args:
@@ -729,107 +517,3 @@ def is_flow_condition_dict(obj: Any) -> TypeIs[FlowCondition]:
return False
return True
def _extract_all_methods_recursive(
condition: str | FlowCondition | dict[str, Any] | list[Any],
flow: Flow[Any] | None = None,
) -> list[FlowMethodName]:
"""Extract ALL method names from a condition tree recursively.
This function recursively extracts every method name from the entire
condition tree, regardless of nesting. Used for visualization and debugging.
Note: Only extracts actual method names, not router output strings.
If flow is provided, it will filter out strings that are not in flow._methods.
Args:
condition: Can be a string, dict, or list
flow: Optional flow instance to filter out non-method strings
Returns:
List of all method names found in the condition tree
"""
if is_flow_method_name(condition):
if flow is not None:
if condition in flow._methods:
return [condition]
return []
return [condition]
if is_flow_condition_dict(condition):
normalized = _normalize_condition(condition)
methods = []
for sub_cond in normalized.get("conditions", []):
methods.extend(_extract_all_methods_recursive(sub_cond, flow))
return methods
if isinstance(condition, list):
methods = []
for item in condition:
methods.extend(_extract_all_methods_recursive(item, flow))
return methods
return []
def _normalize_condition(
condition: FlowConditions | FlowCondition | FlowMethodName,
) -> FlowCondition:
"""Normalize a condition to standard format with 'conditions' key.
Args:
condition: Can be a string (method name), dict (condition), or list
Returns:
Normalized dict with 'type' and 'conditions' keys
"""
if is_flow_method_name(condition):
return {"type": OR_CONDITION, "conditions": [condition]}
if is_flow_condition_dict(condition):
if "conditions" in condition:
return condition
if "methods" in condition:
return {"type": condition["type"], "conditions": condition["methods"]}
return condition
if is_flow_condition_list(condition):
return {"type": OR_CONDITION, "conditions": condition}
raise ValueError(f"Cannot normalize condition: {condition}")
def _extract_all_methods(
condition: str | FlowCondition | dict[str, Any] | list[Any],
) -> list[FlowMethodName]:
"""Extract all method names from a condition (including nested).
For AND conditions, this extracts methods that must ALL complete.
For OR conditions nested inside AND, we don't extract their methods
since only one branch of the OR needs to trigger, not all methods.
This function is used for runtime execution logic, where we need to know
which methods must complete for AND conditions. For visualization purposes,
use _extract_all_methods_recursive() instead.
Args:
condition: Can be a string, dict, or list
Returns:
List of all method names in the condition tree that must complete
"""
if is_flow_method_name(condition):
return [condition]
if is_flow_condition_dict(condition):
normalized = _normalize_condition(condition)
cond_type = normalized.get("type", OR_CONDITION)
if cond_type == AND_CONDITION:
return [
sub_cond
for sub_cond in normalized.get("conditions", [])
if is_flow_method_name(sub_cond)
]
return []
if isinstance(condition, list):
methods = []
for item in condition:
methods.extend(_extract_all_methods(item))
return methods
return []

View File

@@ -1,21 +0,0 @@
"""Flow structure visualization utilities."""
from crewai.flow.visualization.builder import (
build_flow_structure,
calculate_execution_paths,
)
from crewai.flow.visualization.renderers import render_interactive
from crewai.flow.visualization.types import FlowStructure, NodeMetadata, StructureEdge
visualize_flow_structure = render_interactive
__all__ = [
"FlowStructure",
"NodeMetadata",
"StructureEdge",
"build_flow_structure",
"calculate_execution_paths",
"render_interactive",
"visualize_flow_structure",
]

File diff suppressed because it is too large Load Diff

View File

@@ -1,152 +0,0 @@
<!DOCTYPE html>
<html lang="EN">
<head>
<title>CrewAI Flow Visualization</title>
<link rel="preconnect" href="https://fonts.googleapis.com">
<link rel="preconnect" href="https://fonts.gstatic.com" crossorigin>
<link href="https://fonts.googleapis.com/css2?family=Inter:wght@400;500;600;700&display=swap" rel="stylesheet">
<link rel="stylesheet" href="'{{ css_path }}'" />
<script src="https://unpkg.com/lucide@latest"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/prism/1.29.0/prism.min.js"></script>
<script src="https://cdnjs.cloudflare.com/ajax/libs/prism/1.29.0/components/prism-python.min.js"></script>
<script src="'{{ js_path }}'"></script>
</head>
<body>
<!-- Drawer overlay -->
<div id="drawer-overlay"></div>
<!-- Highlight canvas for active nodes/edges above overlay -->
<canvas id="highlight-canvas"></canvas>
<!-- Side drawer -->
<div id="drawer" style="visibility: hidden;">
<div class="drawer-header">
<div class="drawer-title" id="drawer-node-name">Node Details</div>
<div style="display: flex; align-items: center;">
<button class="drawer-open-ide" id="drawer-open-ide" style="display: none;">
<i data-lucide="file-code" style="width: 16px; height: 16px;"></i>
Open in IDE
</button>
<button class="drawer-close" id="drawer-close">
<i data-lucide="x" style="width: 20px; height: 20px;"></i>
</button>
</div>
</div>
<div class="drawer-content" id="drawer-content"></div>
</div>
<div id="info">
<div style="text-align: center;">
<img src="https://cdn.prod.website-files.com/68de1ee6d7c127849807d7a6/68de1ee6d7c127849807d7ef_Logo.svg"
alt="CrewAI Logo"
style="width: 144px; height: auto;">
</div>
</div>
<!-- Custom navigation controls -->
<div class="nav-controls">
<div class="nav-button" id="theme-toggle" title="Toggle Dark Mode">
<i data-lucide="moon" style="width: 18px; height: 18px;"></i>
</div>
<div class="nav-button" id="zoom-in" title="Zoom In">
<i data-lucide="zoom-in" style="width: 18px; height: 18px;"></i>
</div>
<div class="nav-button" id="zoom-out" title="Zoom Out">
<i data-lucide="zoom-out" style="width: 18px; height: 18px;"></i>
</div>
<div class="nav-button" id="fit" title="Fit to Screen">
<i data-lucide="maximize-2" style="width: 18px; height: 18px;"></i>
</div>
<div class="nav-button" id="export-png" title="Export to PNG">
<i data-lucide="image" style="width: 18px; height: 18px;"></i>
</div>
<div class="nav-button" id="export-pdf" title="Export to PDF">
<i data-lucide="file-text" style="width: 18px; height: 18px;"></i>
</div>
<!-- <div class="nav-button" id="export-json" title="Export to JSON">
<i data-lucide="braces" style="width: 18px; height: 18px;"></i>
</div> -->
</div>
<div id="network-container">
<div id="network"></div>
</div>
<!-- Info panel at bottom -->
<div id="legend-panel">
<!-- Stats Section -->
<div class="legend-section">
<div class="legend-stats-row">
<div class="legend-stat-item">
<span class="stat-value">'{{ dag_nodes_count }}'</span>
<span class="stat-label">Nodes</span>
</div>
<div class="legend-stat-item">
<span class="stat-value">'{{ dag_edges_count }}'</span>
<span class="stat-label">Edges</span>
</div>
<div class="legend-stat-item">
<span class="stat-value">'{{ execution_paths }}'</span>
<span class="stat-label">Paths</span>
</div>
</div>
</div>
<!-- Node Types Section -->
<div class="legend-section">
<div class="legend-group">
<div class="legend-item-compact">
<div class="legend-color-small" style="background: var(--node-bg-start);"></div>
<span>Start</span>
</div>
<div class="legend-item-compact">
<div class="legend-color-small" style="background: var(--node-bg-router); border: 2px solid var(--node-border-start);"></div>
<span>Router</span>
</div>
<div class="legend-item-compact">
<div class="legend-color-small" style="background: var(--node-bg-listen); border: 2px solid var(--node-border-listen);"></div>
<span>Listen</span>
</div>
</div>
</div>
<!-- Edge Types Section -->
<div class="legend-section">
<div class="legend-group">
<div class="legend-item-compact">
<svg>
<line x1="0" y1="7" x2="29" y2="7" stroke="var(--edge-router-color)" stroke-width="2" stroke-dasharray="4,4"/>
</svg>
<span>Router</span>
</div>
<div class="legend-item-compact">
<svg class="legend-or-line">
<line x1="0" y1="7" x2="29" y2="7" stroke="var(--edge-or-color)" stroke-width="2"/>
</svg>
<span>OR</span>
</div>
<div class="legend-item-compact">
<svg>
<line x1="0" y1="7" x2="29" y2="7" stroke="var(--edge-router-color)" stroke-width="2"/>
</svg>
<span>AND</span>
</div>
</div>
</div>
<!-- IDE Selector Section -->
<div class="legend-section">
<div class="legend-ide-column">
<label class="legend-ide-label">IDE</label>
<select id="ide-selector" class="legend-ide-select">
<option value="auto">Auto-detect</option>
<option value="pycharm">PyCharm</option>
<option value="vscode">VS Code</option>
<option value="jetbrains">JetBrains</option>
</select>
</div>
</div>
</div>
</body>
</html>

File diff suppressed because it is too large Load Diff

View File

@@ -1,496 +0,0 @@
"""Flow structure builder for analyzing Flow execution."""
from __future__ import annotations
from collections import defaultdict
from collections.abc import Iterable
import inspect
from typing import TYPE_CHECKING, Any
from crewai.flow.constants import AND_CONDITION, OR_CONDITION
from crewai.flow.flow_wrappers import FlowCondition
from crewai.flow.types import FlowMethodName, FlowRouteName
from crewai.flow.utils import (
is_flow_condition_dict,
is_simple_flow_condition,
)
from crewai.flow.visualization.schema import extract_method_signature
from crewai.flow.visualization.types import FlowStructure, NodeMetadata, StructureEdge
if TYPE_CHECKING:
from crewai.flow.flow import Flow
def _extract_direct_or_triggers(
condition: str | dict[str, Any] | list[Any] | FlowCondition,
) -> list[str]:
"""Extract direct OR-level trigger strings from a condition.
This function extracts strings that would directly trigger a listener,
meaning they appear at the top level of an OR condition. Strings nested
inside AND conditions are NOT considered direct triggers for router paths.
For example:
- or_("a", "b") -> ["a", "b"] (both are direct triggers)
- and_("a", "b") -> [] (neither are direct triggers, both required)
- or_(and_("a", "b"), "c") -> ["c"] (only "c" is a direct trigger)
Args:
condition: Can be a string, dict, or list.
Returns:
List of direct OR-level trigger strings.
"""
if isinstance(condition, str):
return [condition]
if isinstance(condition, dict):
cond_type = condition.get("type", OR_CONDITION)
conditions_list = condition.get("conditions", [])
if cond_type == OR_CONDITION:
strings = []
for sub_cond in conditions_list:
strings.extend(_extract_direct_or_triggers(sub_cond))
return strings
return []
if isinstance(condition, list):
strings = []
for item in condition:
strings.extend(_extract_direct_or_triggers(item))
return strings
if callable(condition) and hasattr(condition, "__name__"):
return [condition.__name__]
return []
def _extract_all_trigger_names(
condition: str | dict[str, Any] | list[Any] | FlowCondition,
) -> list[str]:
"""Extract ALL trigger names from a condition for display purposes.
Unlike _extract_direct_or_triggers, this extracts ALL strings and method
names from the entire condition tree, including those nested in AND conditions.
This is used for displaying trigger information in the UI.
For example:
- or_("a", "b") -> ["a", "b"]
- and_("a", "b") -> ["a", "b"]
- or_(and_("a", method_6), method_4) -> ["a", "method_6", "method_4"]
Args:
condition: Can be a string, dict, or list.
Returns:
List of all trigger names found in the condition.
"""
if isinstance(condition, str):
return [condition]
if isinstance(condition, dict):
conditions_list = condition.get("conditions", [])
strings = []
for sub_cond in conditions_list:
strings.extend(_extract_all_trigger_names(sub_cond))
return strings
if isinstance(condition, list):
strings = []
for item in condition:
strings.extend(_extract_all_trigger_names(item))
return strings
if callable(condition) and hasattr(condition, "__name__"):
return [condition.__name__]
return []
def _create_edges_from_condition(
condition: str | dict[str, Any] | list[Any] | FlowCondition,
target: str,
nodes: dict[str, NodeMetadata],
) -> list[StructureEdge]:
"""Create edges from a condition tree, preserving AND/OR semantics.
This function recursively processes the condition tree and creates edges
with the appropriate condition_type for each trigger.
For AND conditions, all triggers get edges with condition_type="AND".
For OR conditions, triggers get edges with condition_type="OR".
Args:
condition: The condition tree (string, dict, or list).
target: The target node name.
nodes: Dictionary of all nodes for validation.
Returns:
List of StructureEdge objects representing the condition.
"""
edges: list[StructureEdge] = []
if isinstance(condition, str):
if condition in nodes:
edges.append(
StructureEdge(
source=condition,
target=target,
condition_type=OR_CONDITION,
is_router_path=False,
)
)
elif callable(condition) and hasattr(condition, "__name__"):
method_name = condition.__name__
if method_name in nodes:
edges.append(
StructureEdge(
source=method_name,
target=target,
condition_type=OR_CONDITION,
is_router_path=False,
)
)
elif isinstance(condition, dict):
cond_type = condition.get("type", OR_CONDITION)
conditions_list = condition.get("conditions", [])
if cond_type == AND_CONDITION:
triggers = _extract_all_trigger_names(condition)
edges.extend(
StructureEdge(
source=trigger,
target=target,
condition_type=AND_CONDITION,
is_router_path=False,
)
for trigger in triggers
if trigger in nodes
)
else:
for sub_cond in conditions_list:
edges.extend(_create_edges_from_condition(sub_cond, target, nodes))
elif isinstance(condition, list):
for item in condition:
edges.extend(_create_edges_from_condition(item, target, nodes))
return edges
def build_flow_structure(flow: Flow[Any]) -> FlowStructure:
"""Build a structure representation of a Flow's execution.
Args:
flow: Flow instance to analyze.
Returns:
Dictionary with nodes, edges, start_methods, and router_methods.
"""
nodes: dict[str, NodeMetadata] = {}
edges: list[StructureEdge] = []
start_methods: list[str] = []
router_methods: list[str] = []
for method_name, method in flow._methods.items():
node_metadata: NodeMetadata = {"type": "listen"}
if hasattr(method, "__is_start_method__") and method.__is_start_method__:
node_metadata["type"] = "start"
start_methods.append(method_name)
if hasattr(method, "__is_router__") and method.__is_router__:
node_metadata["is_router"] = True
node_metadata["type"] = "router"
router_methods.append(method_name)
if method_name in flow._router_paths:
node_metadata["router_paths"] = [
str(p) for p in flow._router_paths[method_name]
]
if hasattr(method, "__trigger_methods__") and method.__trigger_methods__:
node_metadata["trigger_methods"] = [
str(m) for m in method.__trigger_methods__
]
if hasattr(method, "__condition_type__") and method.__condition_type__:
node_metadata["trigger_condition_type"] = method.__condition_type__
if "condition_type" not in node_metadata:
node_metadata["condition_type"] = method.__condition_type__
if node_metadata.get("is_router") and "condition_type" not in node_metadata:
node_metadata["condition_type"] = "IF"
if (
hasattr(method, "__trigger_condition__")
and method.__trigger_condition__ is not None
):
node_metadata["trigger_condition"] = method.__trigger_condition__
if "trigger_methods" not in node_metadata:
extracted = _extract_all_trigger_names(method.__trigger_condition__)
if extracted:
node_metadata["trigger_methods"] = extracted
node_metadata["method_signature"] = extract_method_signature(
method, method_name
)
try:
source_code = inspect.getsource(method)
node_metadata["source_code"] = source_code
try:
source_lines, start_line = inspect.getsourcelines(method)
node_metadata["source_lines"] = source_lines
node_metadata["source_start_line"] = start_line
except (OSError, TypeError):
pass
try:
source_file = inspect.getsourcefile(method)
if source_file:
node_metadata["source_file"] = source_file
except (OSError, TypeError):
try:
class_file = inspect.getsourcefile(flow.__class__)
if class_file:
node_metadata["source_file"] = class_file
except (OSError, TypeError):
pass
except (OSError, TypeError):
pass
try:
class_obj = flow.__class__
if class_obj:
class_name = class_obj.__name__
bases = class_obj.__bases__
if bases:
base_strs = []
for base in bases:
if hasattr(base, "__name__"):
if hasattr(base, "__origin__"):
base_strs.append(str(base))
else:
base_strs.append(base.__name__)
else:
base_strs.append(str(base))
try:
source_lines = inspect.getsource(class_obj).split("\n")
_, class_start_line = inspect.getsourcelines(class_obj)
for idx, line in enumerate(source_lines):
stripped = line.strip()
if stripped.startswith("class ") and class_name in stripped:
class_signature = stripped.rstrip(":")
node_metadata["class_signature"] = class_signature
node_metadata["class_line_number"] = (
class_start_line + idx
)
break
except (OSError, TypeError):
class_signature = f"class {class_name}({', '.join(base_strs)})"
node_metadata["class_signature"] = class_signature
else:
class_signature = f"class {class_name}"
node_metadata["class_signature"] = class_signature
node_metadata["class_name"] = class_name
except (OSError, TypeError, AttributeError):
pass
nodes[method_name] = node_metadata
for listener_name, condition_data in flow._listeners.items():
if listener_name in router_methods:
continue
if is_simple_flow_condition(condition_data):
cond_type, methods = condition_data
edges.extend(
StructureEdge(
source=str(trigger_method),
target=str(listener_name),
condition_type=cond_type,
is_router_path=False,
)
for trigger_method in methods
if str(trigger_method) in nodes
)
elif is_flow_condition_dict(condition_data):
edges.extend(
_create_edges_from_condition(condition_data, str(listener_name), nodes)
)
for method_name, node_metadata in nodes.items(): # type: ignore[assignment]
if node_metadata.get("is_router") and "trigger_methods" in node_metadata:
trigger_methods = node_metadata["trigger_methods"]
condition_type = node_metadata.get("trigger_condition_type", OR_CONDITION)
if "trigger_condition" in node_metadata:
edges.extend(
_create_edges_from_condition(
node_metadata["trigger_condition"], # type: ignore[arg-type]
method_name,
nodes,
)
)
else:
edges.extend(
StructureEdge(
source=trigger_method,
target=method_name,
condition_type=condition_type,
is_router_path=False,
)
for trigger_method in trigger_methods
if trigger_method in nodes
)
for router_method_name in router_methods:
if router_method_name not in flow._router_paths:
flow._router_paths[FlowMethodName(router_method_name)] = []
inferred_paths: Iterable[FlowMethodName | FlowRouteName] = set(
flow._router_paths.get(FlowMethodName(router_method_name), [])
)
for condition_data in flow._listeners.values():
trigger_strings: list[str] = []
if is_simple_flow_condition(condition_data):
_, methods = condition_data
trigger_strings = [str(m) for m in methods]
elif is_flow_condition_dict(condition_data):
trigger_strings = _extract_direct_or_triggers(condition_data)
for trigger_str in trigger_strings:
if trigger_str not in nodes:
# This is likely a router path output
inferred_paths.add(trigger_str) # type: ignore[attr-defined]
if inferred_paths:
flow._router_paths[FlowMethodName(router_method_name)] = list(
inferred_paths # type: ignore[arg-type]
)
if router_method_name in nodes:
nodes[router_method_name]["router_paths"] = list(inferred_paths)
for router_method_name in router_methods:
if router_method_name not in flow._router_paths:
continue
router_paths = flow._router_paths[FlowMethodName(router_method_name)]
for path in router_paths:
for listener_name, condition_data in flow._listeners.items():
trigger_strings_from_cond: list[str] = []
if is_simple_flow_condition(condition_data):
_, methods = condition_data
trigger_strings_from_cond = [str(m) for m in methods]
elif is_flow_condition_dict(condition_data):
trigger_strings_from_cond = _extract_direct_or_triggers(
condition_data
)
if str(path) in trigger_strings_from_cond:
edges.append(
StructureEdge(
source=router_method_name,
target=str(listener_name),
condition_type=None,
is_router_path=True,
router_path_label=str(path),
)
)
for start_method in flow._start_methods:
if start_method not in nodes and start_method in flow._methods:
method = flow._methods[start_method]
nodes[str(start_method)] = NodeMetadata(type="start")
if hasattr(method, "__trigger_methods__") and method.__trigger_methods__:
nodes[str(start_method)]["trigger_methods"] = [
str(m) for m in method.__trigger_methods__
]
if hasattr(method, "__condition_type__") and method.__condition_type__:
nodes[str(start_method)]["condition_type"] = method.__condition_type__
return FlowStructure(
nodes=nodes,
edges=edges,
start_methods=start_methods,
router_methods=router_methods,
)
def calculate_execution_paths(structure: FlowStructure) -> int:
"""Calculate number of possible execution paths through the flow.
Args:
structure: FlowStructure to analyze.
Returns:
Number of possible execution paths.
"""
graph = defaultdict(list)
for edge in structure["edges"]:
graph[edge["source"]].append(
{
"target": edge["target"],
"is_router": edge["is_router_path"],
"condition": edge["condition_type"],
}
)
all_nodes = set(structure["nodes"].keys())
nodes_with_outgoing = set(edge["source"] for edge in structure["edges"])
terminal_nodes = all_nodes - nodes_with_outgoing
if not structure["start_methods"] or not terminal_nodes:
return 0
def count_paths_from(node: str, visited: set[str]) -> int:
"""Recursively count execution paths from a given node.
Args:
node: Node name to start counting from.
visited: Set of already visited nodes to prevent cycles.
Returns:
Number of execution paths from this node to terminal nodes.
"""
if node in terminal_nodes:
return 1
if node in visited:
return 0
visited.add(node)
outgoing = graph[node]
if not outgoing:
visited.remove(node)
return 1
if node in structure["router_methods"]:
total = 0
for edge_info in outgoing:
target = str(edge_info["target"])
total += count_paths_from(target, visited.copy())
visited.remove(node)
return total
total = 0
for edge_info in outgoing:
target = str(edge_info["target"])
total += count_paths_from(target, visited.copy())
visited.remove(node)
return total if total > 0 else 1
total_paths = 0
for start in structure["start_methods"]:
total_paths += count_paths_from(start, set())
return max(total_paths, 1)

View File

@@ -1,8 +0,0 @@
"""Flow structure visualization renderers."""
from crewai.flow.visualization.renderers.interactive import render_interactive
__all__ = [
"render_interactive",
]

View File

@@ -1,466 +0,0 @@
"""Interactive HTML renderer for Flow structure visualization."""
import json
from pathlib import Path
import tempfile
from typing import Any, ClassVar
import webbrowser
from jinja2 import Environment, FileSystemLoader, nodes, select_autoescape
from jinja2.ext import Extension
from jinja2.parser import Parser
from crewai.flow.visualization.builder import calculate_execution_paths
from crewai.flow.visualization.types import FlowStructure
class CSSExtension(Extension):
"""Jinja2 extension for rendering CSS link tags.
Provides {% css 'path/to/file.css' %} tag syntax.
"""
tags: ClassVar[set[str]] = {"css"} # type: ignore[misc]
def parse(self, parser: Parser) -> nodes.Node:
"""Parse {% css 'styles.css' %} tag.
Args:
parser: Jinja2 parser instance.
Returns:
Output node with rendered CSS link tag.
"""
lineno: int = next(parser.stream).lineno
args: list[nodes.Expr] = [parser.parse_expression()]
return nodes.Output([self.call_method("_render_css", args)]).set_lineno(lineno)
def _render_css(self, href: str) -> str:
"""Render CSS link tag.
Args:
href: Path to CSS file.
Returns:
HTML link tag string.
"""
return f'<link rel="stylesheet" href="{href}">'
class JSExtension(Extension):
"""Jinja2 extension for rendering script tags.
Provides {% js 'path/to/file.js' %} tag syntax.
"""
tags: ClassVar[set[str]] = {"js"} # type: ignore[misc]
def parse(self, parser: Parser) -> nodes.Node:
"""Parse {% js 'script.js' %} tag.
Args:
parser: Jinja2 parser instance.
Returns:
Output node with rendered script tag.
"""
lineno: int = next(parser.stream).lineno
args: list[nodes.Expr] = [parser.parse_expression()]
return nodes.Output([self.call_method("_render_js", args)]).set_lineno(lineno)
def _render_js(self, src: str) -> str:
"""Render script tag.
Args:
src: Path to JavaScript file.
Returns:
HTML script tag string.
"""
return f'<script src="{src}"></script>'
CREWAI_ORANGE = "#FF5A50"
DARK_GRAY = "#333333"
WHITE = "#FFFFFF"
GRAY = "#666666"
BG_DARK = "#0d1117"
BG_CARD = "#161b22"
BORDER_SUBTLE = "#30363d"
TEXT_PRIMARY = "#e6edf3"
TEXT_SECONDARY = "#7d8590"
def calculate_node_positions(
dag: FlowStructure,
) -> dict[str, dict[str, int | float]]:
"""Calculate hierarchical positions (level, x, y) for each node.
Args:
dag: FlowStructure containing nodes and edges.
Returns:
Dictionary mapping node names to their position data (level, x, y).
"""
children: dict[str, list[str]] = {name: [] for name in dag["nodes"]}
parents: dict[str, list[str]] = {name: [] for name in dag["nodes"]}
for edge in dag["edges"]:
source = edge["source"]
target = edge["target"]
if source in children and target in children:
children[source].append(target)
parents[target].append(source)
levels: dict[str, int] = {}
queue: list[tuple[str, int]] = []
for start_method in dag["start_methods"]:
if start_method in dag["nodes"]:
levels[start_method] = 0
queue.append((start_method, 0))
visited: set[str] = set()
while queue:
node, level = queue.pop(0)
if node in visited:
continue
visited.add(node)
if node not in levels or levels[node] < level:
levels[node] = level
for child in children.get(node, []):
if child not in visited:
child_level = level + 1
if child not in levels or levels[child] < child_level:
levels[child] = child_level
queue.append((child, child_level))
for name in dag["nodes"]:
if name not in levels:
levels[name] = 0
nodes_by_level: dict[int, list[str]] = {}
for node, level in levels.items():
if level not in nodes_by_level:
nodes_by_level[level] = []
nodes_by_level[level].append(node)
positions: dict[str, dict[str, int | float]] = {}
level_separation = 300 # Vertical spacing between levels
node_spacing = 400 # Horizontal spacing between nodes
parent_count: dict[str, int] = {}
for node, parent_list in parents.items():
parent_count[node] = len(parent_list)
for level, nodes_at_level in sorted(nodes_by_level.items()):
y = level * level_separation
if level == 0:
num_nodes = len(nodes_at_level)
for i, node in enumerate(nodes_at_level):
x = (i - (num_nodes - 1) / 2) * node_spacing
positions[node] = {"level": level, "x": x, "y": y}
else:
for i, node in enumerate(nodes_at_level):
parent_list = parents.get(node, [])
parent_positions: list[float] = [
positions[parent]["x"]
for parent in parent_list
if parent in positions
]
if parent_positions:
if len(parent_positions) > 1 and len(set(parent_positions)) == 1:
base_x = parent_positions[0]
avg_x = base_x + node_spacing * 0.4
else:
avg_x = sum(parent_positions) / len(parent_positions)
else:
avg_x = i * node_spacing * 0.5
positions[node] = {"level": level, "x": avg_x, "y": y}
nodes_at_level_sorted = sorted(
nodes_at_level, key=lambda n: positions[n]["x"]
)
min_spacing = node_spacing * 0.6 # Minimum horizontal distance
for i in range(len(nodes_at_level_sorted) - 1):
current_node = nodes_at_level_sorted[i]
next_node = nodes_at_level_sorted[i + 1]
current_x = positions[current_node]["x"]
next_x = positions[next_node]["x"]
if next_x - current_x < min_spacing:
positions[next_node]["x"] = current_x + min_spacing
return positions
def render_interactive(
dag: FlowStructure,
filename: str = "flow_dag.html",
show: bool = True,
) -> str:
"""Create interactive HTML visualization of Flow structure.
Generates three output files in a temporary directory: HTML template,
CSS stylesheet, and JavaScript. Optionally opens the visualization in
default browser.
Args:
dag: FlowStructure to visualize.
filename: Output HTML filename (basename only, no path).
show: Whether to open in browser.
Returns:
Absolute path to generated HTML file in temporary directory.
"""
node_positions = calculate_node_positions(dag)
nodes_list: list[dict[str, Any]] = []
for name, metadata in dag["nodes"].items():
node_type: str = metadata.get("type", "listen")
color_config: dict[str, Any]
font_color: str
border_width: int
if node_type == "start":
color_config = {
"background": "var(--node-bg-start)",
"border": "var(--node-border-start)",
"highlight": {
"background": "var(--node-bg-start)",
"border": "var(--node-border-start)",
},
}
font_color = "var(--node-text-color)"
border_width = 3
elif node_type == "router":
color_config = {
"background": "var(--node-bg-router)",
"border": CREWAI_ORANGE,
"highlight": {
"background": "var(--node-bg-router)",
"border": CREWAI_ORANGE,
},
}
font_color = "var(--node-text-color)"
border_width = 3
else:
color_config = {
"background": "var(--node-bg-listen)",
"border": "var(--node-border-listen)",
"highlight": {
"background": "var(--node-bg-listen)",
"border": "var(--node-border-listen)",
},
}
font_color = "var(--node-text-color)"
border_width = 3
title_parts: list[str] = []
type_badge_bg: str = (
CREWAI_ORANGE if node_type in ["start", "router"] else DARK_GRAY
)
title_parts.append(f"""
<div style="border-bottom: 1px solid rgba(102,102,102,0.15); padding-bottom: 8px; margin-bottom: 10px;">
<div style="font-size: 13px; font-weight: 700; color: {DARK_GRAY}; margin-bottom: 6px;">{name}</div>
<span style="display: inline-block; background: {type_badge_bg}; color: white; padding: 2px 8px; border-radius: 4px; font-size: 10px; font-weight: 600; text-transform: uppercase; letter-spacing: 0.5px;">{node_type}</span>
</div>
""")
if metadata.get("condition_type"):
condition = metadata["condition_type"]
if condition == "AND":
condition_badge_bg = "rgba(255,90,80,0.12)"
condition_color = CREWAI_ORANGE
elif condition == "IF":
condition_badge_bg = "rgba(255,90,80,0.18)"
condition_color = CREWAI_ORANGE
else:
condition_badge_bg = "rgba(102,102,102,0.12)"
condition_color = GRAY
title_parts.append(f"""
<div style="margin-bottom: 8px;">
<div style="font-size: 10px; text-transform: uppercase; color: {GRAY}; letter-spacing: 0.5px; margin-bottom: 3px; font-weight: 600;">Condition</div>
<span style="display: inline-block; background: {condition_badge_bg}; color: {condition_color}; padding: 3px 8px; border-radius: 4px; font-size: 11px; font-weight: 700;">{condition}</span>
</div>
""")
if metadata.get("trigger_methods"):
triggers = metadata["trigger_methods"]
triggers_items = "".join(
[
f'<li style="margin: 3px 0;"><code style="background: rgba(102,102,102,0.08); padding: 2px 6px; border-radius: 3px; font-size: 10px; color: {DARK_GRAY}; border: 1px solid rgba(102,102,102,0.12);">{t}</code></li>'
for t in triggers
]
)
title_parts.append(f"""
<div style="margin-bottom: 8px;">
<div style="font-size: 10px; text-transform: uppercase; color: {GRAY}; letter-spacing: 0.5px; margin-bottom: 4px; font-weight: 600;">Triggers</div>
<ul style="list-style: none; padding: 0; margin: 0;">{triggers_items}</ul>
</div>
""")
if metadata.get("router_paths"):
paths = metadata["router_paths"]
paths_items = "".join(
[
f'<li style="margin: 3px 0;"><code style="background: rgba(255,90,80,0.08); padding: 2px 6px; border-radius: 3px; font-size: 10px; color: {CREWAI_ORANGE}; border: 1px solid rgba(255,90,80,0.2); font-weight: 600;">{p}</code></li>'
for p in paths
]
)
title_parts.append(f"""
<div>
<div style="font-size: 10px; text-transform: uppercase; color: {GRAY}; letter-spacing: 0.5px; margin-bottom: 4px; font-weight: 600;">Router Paths</div>
<ul style="list-style: none; padding: 0; margin: 0;">{paths_items}</ul>
</div>
""")
bg_color = color_config["background"]
border_color = color_config["border"]
position_data = node_positions.get(name, {"level": 0, "x": 0, "y": 0})
node_data: dict[str, Any] = {
"id": name,
"label": name,
"title": "".join(title_parts),
"shape": "custom",
"size": 30,
"level": position_data["level"],
"nodeStyle": {
"name": name,
"bgColor": bg_color,
"borderColor": border_color,
"borderWidth": border_width,
"fontColor": font_color,
},
"opacity": 1.0,
"glowSize": 0,
"glowColor": None,
}
# Add x,y only for graphs with 3-4 nodes
total_nodes = len(dag["nodes"])
if 3 <= total_nodes <= 4:
node_data["x"] = position_data["x"]
node_data["y"] = position_data["y"]
nodes_list.append(node_data)
execution_paths: int = calculate_execution_paths(dag)
edges_list: list[dict[str, Any]] = []
for edge in dag["edges"]:
edge_label: str = ""
edge_color: str = GRAY
edge_dashes: bool | list[int] = False
if edge["is_router_path"]:
edge_color = CREWAI_ORANGE
edge_dashes = [15, 10]
if "router_path_label" in edge:
edge_label = edge["router_path_label"]
elif edge["condition_type"] == "AND":
edge_label = "AND"
edge_color = CREWAI_ORANGE
elif edge["condition_type"] == "OR":
edge_label = "OR"
edge_color = GRAY
edge_data: dict[str, Any] = {
"from": edge["source"],
"to": edge["target"],
"label": edge_label,
"arrows": "to",
"width": 2,
"selectionWidth": 0,
"color": {
"color": edge_color,
"highlight": edge_color,
},
}
if edge_dashes is not False:
edge_data["dashes"] = edge_dashes
edges_list.append(edge_data)
template_dir = Path(__file__).parent.parent / "assets"
env = Environment(
loader=FileSystemLoader(template_dir),
autoescape=select_autoescape(["html", "xml", "css", "js"]),
variable_start_string="'{{",
variable_end_string="}}'",
extensions=[CSSExtension, JSExtension],
)
temp_dir = Path(tempfile.mkdtemp(prefix="crewai_flow_"))
output_path = temp_dir / Path(filename).name
css_filename = output_path.stem + "_style.css"
css_output_path = temp_dir / css_filename
js_filename = output_path.stem + "_script.js"
js_output_path = temp_dir / js_filename
css_file = template_dir / "style.css"
css_content = css_file.read_text(encoding="utf-8")
css_content = css_content.replace("'{{ WHITE }}'", WHITE)
css_content = css_content.replace("'{{ DARK_GRAY }}'", DARK_GRAY)
css_content = css_content.replace("'{{ GRAY }}'", GRAY)
css_content = css_content.replace("'{{ CREWAI_ORANGE }}'", CREWAI_ORANGE)
css_output_path.write_text(css_content, encoding="utf-8")
js_file = template_dir / "interactive.js"
js_content = js_file.read_text(encoding="utf-8")
dag_nodes_json = json.dumps(dag["nodes"])
dag_full_json = json.dumps(dag)
js_content = js_content.replace("{{ WHITE }}", WHITE)
js_content = js_content.replace("{{ DARK_GRAY }}", DARK_GRAY)
js_content = js_content.replace("{{ GRAY }}", GRAY)
js_content = js_content.replace("{{ CREWAI_ORANGE }}", CREWAI_ORANGE)
js_content = js_content.replace("'{{ nodeData }}'", dag_nodes_json)
js_content = js_content.replace("'{{ dagData }}'", dag_full_json)
js_content = js_content.replace("'{{ nodes_list_json }}'", json.dumps(nodes_list))
js_content = js_content.replace("'{{ edges_list_json }}'", json.dumps(edges_list))
js_output_path.write_text(js_content, encoding="utf-8")
template = env.get_template("interactive_flow.html.j2")
html_content = template.render(
CREWAI_ORANGE=CREWAI_ORANGE,
DARK_GRAY=DARK_GRAY,
WHITE=WHITE,
GRAY=GRAY,
BG_DARK=BG_DARK,
BG_CARD=BG_CARD,
BORDER_SUBTLE=BORDER_SUBTLE,
TEXT_PRIMARY=TEXT_PRIMARY,
TEXT_SECONDARY=TEXT_SECONDARY,
nodes_list_json=json.dumps(nodes_list),
edges_list_json=json.dumps(edges_list),
dag_nodes_count=len(dag["nodes"]),
dag_edges_count=len(dag["edges"]),
execution_paths=execution_paths,
css_path=css_filename,
js_path=js_filename,
)
output_path.write_text(html_content, encoding="utf-8")
if show:
webbrowser.open(f"file://{output_path.absolute()}")
return str(output_path.absolute())

View File

@@ -1,104 +0,0 @@
"""OpenAPI schema conversion utilities for Flow methods."""
import inspect
from typing import Any, get_args, get_origin
def type_to_openapi_schema(type_hint: Any) -> dict[str, Any]:
"""Convert Python type hint to OpenAPI schema.
Args:
type_hint: Python type hint to convert.
Returns:
OpenAPI schema dictionary.
"""
if type_hint is inspect.Parameter.empty:
return {}
if type_hint is None or type_hint is type(None):
return {"type": "null"}
if hasattr(type_hint, "__module__") and hasattr(type_hint, "__name__"):
if type_hint.__module__ == "typing" and type_hint.__name__ == "Any":
return {}
type_str = str(type_hint)
if type_str == "typing.Any" or type_str == "<class 'typing.Any'>":
return {}
if isinstance(type_hint, str):
return {"type": type_hint}
origin = get_origin(type_hint)
args = get_args(type_hint)
if type_hint is str:
return {"type": "string"}
if type_hint is int:
return {"type": "integer"}
if type_hint is float:
return {"type": "number"}
if type_hint is bool:
return {"type": "boolean"}
if type_hint is dict or origin is dict:
if args and len(args) > 1:
return {
"type": "object",
"additionalProperties": type_to_openapi_schema(args[1]),
}
return {"type": "object"}
if type_hint is list or origin is list:
if args:
return {"type": "array", "items": type_to_openapi_schema(args[0])}
return {"type": "array"}
if hasattr(type_hint, "__name__"):
return {"type": "object", "className": type_hint.__name__}
return {}
def extract_method_signature(method: Any, method_name: str) -> dict[str, Any]:
"""Extract method signature as OpenAPI schema with documentation.
Args:
method: Method to analyze.
method_name: Method name.
Returns:
Dictionary with operationId, parameters, returns, summary, and description.
"""
try:
sig = inspect.signature(method)
parameters = {}
for param_name, param in sig.parameters.items():
if param_name == "self":
continue
parameters[param_name] = type_to_openapi_schema(param.annotation)
return_type = type_to_openapi_schema(sig.return_annotation)
docstring = inspect.getdoc(method)
result: dict[str, Any] = {
"operationId": method_name,
"parameters": parameters,
"returns": return_type,
}
if docstring:
lines = docstring.strip().split("\n")
summary = lines[0].strip()
if summary:
result["summary"] = summary
if len(lines) > 1:
description = "\n".join(line.strip() for line in lines[1:]).strip()
if description:
result["description"] = description
return result
except Exception:
return {"operationId": method_name, "parameters": {}, "returns": {}}

View File

@@ -1,42 +0,0 @@
"""Type definitions for Flow structure visualization."""
from typing import Any, TypedDict
class NodeMetadata(TypedDict, total=False):
"""Metadata for a single node in the flow structure."""
type: str
is_router: bool
router_paths: list[str]
condition_type: str | None
trigger_condition_type: str | None
trigger_methods: list[str]
trigger_condition: dict[str, Any] | None
method_signature: dict[str, Any]
source_code: str
source_lines: list[str]
source_start_line: int
source_file: str
class_signature: str
class_name: str
class_line_number: int
class StructureEdge(TypedDict, total=False):
"""Represents a connection in the flow structure."""
source: str
target: str
condition_type: str | None
is_router_path: bool
router_path_label: str
class FlowStructure(TypedDict):
"""Complete structure representation of a Flow."""
nodes: dict[str, NodeMetadata]
edges: list[StructureEdge]
start_methods: list[str]
router_methods: list[str]

View File

@@ -0,0 +1,342 @@
"""
Utilities for creating visual representations of flow structures.
This module provides functions for generating network visualizations of flows,
including node placement, edge creation, and visual styling. It handles the
conversion of flow structures into visual network graphs with appropriate
styling and layout.
Example
-------
>>> flow = Flow()
>>> net = Network(directed=True)
>>> node_positions = compute_positions(flow, node_levels)
>>> add_nodes_to_network(net, flow, node_positions, node_styles)
>>> add_edges(net, flow, node_positions, colors)
"""
import ast
import inspect
from typing import Any
from crewai.flow.config import (
CrewNodeStyle,
FlowColors,
MethodNodeStyle,
NodeStyles,
RouterNodeStyle,
StartNodeStyle,
)
from crewai.flow.utils import (
build_ancestor_dict,
build_parent_children_dict,
get_child_index,
is_ancestor,
)
from crewai.utilities.printer import Printer
_printer = Printer()
def method_calls_crew(method: Any) -> bool:
"""
Check if the method contains a call to `.crew()`, `.kickoff()`, or `.kickoff_async()`.
Parameters
----------
method : Any
The method to analyze for crew or agent execution calls.
Returns
-------
bool
True if the method calls .crew(), .kickoff(), or .kickoff_async(), False otherwise.
Notes
-----
Uses AST analysis to detect method calls, specifically looking for
attribute access of 'crew', 'kickoff', or 'kickoff_async'.
This includes both traditional Crew execution (.crew()) and Agent/LiteAgent
execution (.kickoff() or .kickoff_async()).
"""
try:
source = inspect.getsource(method)
source = inspect.cleandoc(source)
tree = ast.parse(source)
except Exception as e:
_printer.print(f"Could not parse method {method.__name__}: {e}", color="red")
return False
class CrewCallVisitor(ast.NodeVisitor):
"""AST visitor to detect .crew(), .kickoff(), or .kickoff_async() method calls."""
def __init__(self) -> None:
self.found = False
def visit_Call(self, node: ast.Call) -> None:
if isinstance(node.func, ast.Attribute):
if node.func.attr in ("crew", "kickoff", "kickoff_async"):
self.found = True
self.generic_visit(node)
visitor = CrewCallVisitor()
visitor.visit(tree)
return visitor.found
def add_nodes_to_network(
net: Any,
flow: Any,
node_positions: dict[str, tuple[float, float]],
node_styles: NodeStyles,
) -> None:
"""
Add nodes to the network visualization with appropriate styling.
Parameters
----------
net : Any
The pyvis Network instance to add nodes to.
flow : Any
The flow instance containing method information.
node_positions : Dict[str, Tuple[float, float]]
Dictionary mapping node names to their (x, y) positions.
node_styles : Dict[str, Dict[str, Any]]
Dictionary containing style configurations for different node types.
Notes
-----
Node types include:
- Start methods
- Router methods
- Crew methods
- Regular methods
"""
def human_friendly_label(method_name: str) -> str:
return method_name.replace("_", " ").title()
node_style: (
StartNodeStyle | RouterNodeStyle | CrewNodeStyle | MethodNodeStyle | None
)
for method_name, (x, y) in node_positions.items():
method = flow._methods.get(method_name)
if hasattr(method, "__is_start_method__"):
node_style = node_styles["start"]
elif hasattr(method, "__is_router__"):
node_style = node_styles["router"]
elif method_calls_crew(method):
node_style = node_styles["crew"]
else:
node_style = node_styles["method"]
node_style = node_style.copy()
label = human_friendly_label(method_name)
node_style.update(
{
"label": label,
"shape": "box",
"font": {
"multi": "html",
"color": node_style.get("font", {}).get("color", "#FFFFFF"),
},
}
)
net.add_node(
method_name,
x=x,
y=y,
fixed=True,
physics=False,
**node_style,
)
def compute_positions(
flow: Any,
node_levels: dict[str, int],
y_spacing: float = 150,
x_spacing: float = 300,
) -> dict[str, tuple[float, float]]:
"""
Compute the (x, y) positions for each node in the flow graph.
Parameters
----------
flow : Any
The flow instance to compute positions for.
node_levels : Dict[str, int]
Dictionary mapping node names to their hierarchical levels.
y_spacing : float, optional
Vertical spacing between levels, by default 150.
x_spacing : float, optional
Horizontal spacing between nodes, by default 300.
Returns
-------
Dict[str, Tuple[float, float]]
Dictionary mapping node names to their (x, y) coordinates.
"""
level_nodes: dict[int, list[str]] = {}
node_positions: dict[str, tuple[float, float]] = {}
for method_name, level in node_levels.items():
level_nodes.setdefault(level, []).append(method_name)
for level, nodes in level_nodes.items():
x_offset = -(len(nodes) - 1) * x_spacing / 2 # Center nodes horizontally
for i, method_name in enumerate(nodes):
x = x_offset + i * x_spacing
y = level * y_spacing
node_positions[method_name] = (x, y)
return node_positions
def add_edges(
net: Any,
flow: Any,
node_positions: dict[str, tuple[float, float]],
colors: FlowColors,
) -> None:
edge_smooth: dict[str, str | float] = {"type": "continuous"} # Default value
"""
Add edges to the network visualization with appropriate styling.
Parameters
----------
net : Any
The pyvis Network instance to add edges to.
flow : Any
The flow instance containing edge information.
node_positions : Dict[str, Tuple[float, float]]
Dictionary mapping node names to their positions.
colors : Dict[str, str]
Dictionary mapping edge types to their colors.
Notes
-----
- Handles both normal listener edges and router edges
- Applies appropriate styling (color, dashes) based on edge type
- Adds curvature to edges when needed (cycles or multiple children)
"""
ancestors = build_ancestor_dict(flow)
parent_children = build_parent_children_dict(flow)
# Edges for normal listeners
for method_name in flow._listeners:
condition_type, trigger_methods = flow._listeners[method_name]
is_and_condition = condition_type == "AND"
for trigger in trigger_methods:
# Check if nodes exist before adding edges
if trigger in node_positions and method_name in node_positions:
is_router_edge = any(
trigger in paths for paths in flow._router_paths.values()
)
edge_color = colors["router_edge"] if is_router_edge else colors["edge"]
is_cycle_edge = is_ancestor(trigger, method_name, ancestors)
parent_has_multiple_children = len(parent_children.get(trigger, [])) > 1
needs_curvature = is_cycle_edge or parent_has_multiple_children
if needs_curvature:
source_pos = node_positions.get(trigger)
target_pos = node_positions.get(method_name)
if source_pos and target_pos:
dx = target_pos[0] - source_pos[0]
smooth_type = "curvedCCW" if dx <= 0 else "curvedCW"
index = get_child_index(trigger, method_name, parent_children)
edge_smooth = {
"type": smooth_type,
"roundness": 0.2 + (0.1 * index),
}
else:
edge_smooth = {"type": "cubicBezier"}
else:
edge_smooth.update({"type": "continuous"})
edge_style = {
"color": edge_color,
"width": 2,
"arrows": "to",
"dashes": True if is_router_edge or is_and_condition else False,
"smooth": edge_smooth,
}
net.add_edge(trigger, method_name, **edge_style)
else:
# Nodes not found in node_positions. Check if it's a known router outcome and a known method.
is_router_edge = any(
trigger in paths for paths in flow._router_paths.values()
)
# Check if method_name is a known method
method_known = method_name in flow._methods
# If it's a known router edge and the method is known, don't warn.
# This means the path is legitimate, just not reflected as nodes here.
if not (is_router_edge and method_known):
_printer.print(
f"Warning: No node found for '{trigger}' or '{method_name}'. Skipping edge.",
color="yellow",
)
# Edges for router return paths
for router_method_name, paths in flow._router_paths.items():
for path in paths:
for listener_name, (
_condition_type,
trigger_methods,
) in flow._listeners.items():
if path in trigger_methods:
if (
router_method_name in node_positions
and listener_name in node_positions
):
is_cycle_edge = is_ancestor(
router_method_name, listener_name, ancestors
)
parent_has_multiple_children = (
len(parent_children.get(router_method_name, [])) > 1
)
needs_curvature = is_cycle_edge or parent_has_multiple_children
if needs_curvature:
source_pos = node_positions.get(router_method_name)
target_pos = node_positions.get(listener_name)
if source_pos and target_pos:
dx = target_pos[0] - source_pos[0]
smooth_type = "curvedCCW" if dx <= 0 else "curvedCW"
index = get_child_index(
router_method_name, listener_name, parent_children
)
edge_smooth = {
"type": smooth_type,
"roundness": 0.2 + (0.1 * index),
}
else:
edge_smooth = {"type": "cubicBezier"}
else:
edge_smooth.update({"type": "continuous"})
edge_style = {
"color": colors["router_edge"],
"width": 2,
"arrows": "to",
"dashes": True,
"smooth": edge_smooth,
}
net.add_edge(router_method_name, listener_name, **edge_style)
else:
# Same check here: known router edge and known method?
method_known = listener_name in flow._methods
if not method_known:
_printer.print(
f"Warning: No node found for '{router_method_name}' or '{listener_name}'. Skipping edge.",
color="yellow",
)

View File

@@ -1,7 +1,6 @@
import asyncio
from collections.abc import Callable
import inspect
import json
from typing import (
Any,
Literal,
@@ -59,14 +58,10 @@ from crewai.utilities.agent_utils import (
process_llm_response,
render_text_description_and_args,
)
from crewai.utilities.converter import (
Converter,
ConverterError,
generate_model_description,
)
from crewai.utilities.converter import generate_model_description
from crewai.utilities.guardrail import process_guardrail
from crewai.utilities.guardrail_types import GuardrailCallable, GuardrailType
from crewai.utilities.i18n import I18N, get_i18n
from crewai.utilities.i18n import I18N
from crewai.utilities.llm_utils import create_llm
from crewai.utilities.printer import Printer
from crewai.utilities.token_counter_callback import TokenCalcHandler
@@ -95,6 +90,8 @@ class LiteAgent(FlowTrackable, BaseModel):
"""
model_config = {"arbitrary_types_allowed": True}
# Core Agent Properties
id: UUID4 = Field(default_factory=uuid.uuid4, frozen=True)
role: str = Field(description="Role of the agent")
goal: str = Field(description="Goal of the agent")
@@ -105,6 +102,8 @@ class LiteAgent(FlowTrackable, BaseModel):
tools: list[BaseTool] = Field(
default_factory=list, description="Tools at agent's disposal"
)
# Execution Control Properties
max_iterations: int = Field(
default=15, description="Maximum number of iterations for tool usage"
)
@@ -121,17 +120,24 @@ class LiteAgent(FlowTrackable, BaseModel):
)
request_within_rpm_limit: Callable[[], bool] | None = Field(
default=None,
description="Callback to check if the request is within the RPM8 limit",
description="Callback to check if the request is within the RPM limit",
)
i18n: I18N = Field(
default_factory=get_i18n, description="Internationalization settings."
default_factory=I18N, description="Internationalization settings."
)
# Output and Formatting Properties
response_format: type[BaseModel] | None = Field(
default=None, description="Pydantic model for structured output"
)
verbose: bool = Field(
default=False, description="Whether to print execution details"
)
callbacks: list[Callable] = Field(
default_factory=list, description="Callbacks to be used for the agent"
)
# Guardrail Properties
guardrail: GuardrailType | None = Field(
default=None,
description="Function or string description of a guardrail to validate agent output",
@@ -139,12 +145,17 @@ class LiteAgent(FlowTrackable, BaseModel):
guardrail_max_retries: int = Field(
default=3, description="Maximum number of retries when guardrail fails"
)
# State and Results
tools_results: list[dict[str, Any]] = Field(
default_factory=list, description="Results of the tools used by the agent."
)
# Reference of Agent
original_agent: BaseAgent | None = Field(
default=None, description="Reference to the agent that created this LiteAgent"
)
# Private Attributes
_parsed_tools: list[CrewStructuredTool] = PrivateAttr(default_factory=list)
_token_process: TokenProcess = PrivateAttr(default_factory=TokenProcess)
_cache_handler: CacheHandler = PrivateAttr(default_factory=CacheHandler)
@@ -154,7 +165,6 @@ class LiteAgent(FlowTrackable, BaseModel):
_printer: Printer = PrivateAttr(default_factory=Printer)
_guardrail: GuardrailCallable | None = PrivateAttr(default=None)
_guardrail_retry_count: int = PrivateAttr(default=0)
_callbacks: list[TokenCalcHandler] = PrivateAttr(default_factory=list)
@model_validator(mode="after")
def setup_llm(self) -> Self:
@@ -164,13 +174,15 @@ class LiteAgent(FlowTrackable, BaseModel):
raise ValueError(
f"Expected LLM instance of type BaseLLM, got {type(self.llm).__name__}"
)
# Initialize callbacks
token_callback = TokenCalcHandler(token_cost_process=self._token_process)
self._callbacks = [token_callback]
return self
@model_validator(mode="after")
def parse_tools(self) -> Self:
def parse_tools(self):
"""Parse the tools and convert them to CrewStructuredTool instances."""
self._parsed_tools = parse_tools(self.tools)
@@ -189,7 +201,7 @@ class LiteAgent(FlowTrackable, BaseModel):
)
self._guardrail = cast(
GuardrailCallable,
cast(object, LLMGuardrail(description=self.guardrail, llm=self.llm)),
LLMGuardrail(description=self.guardrail, llm=self.llm),
)
return self
@@ -197,8 +209,8 @@ class LiteAgent(FlowTrackable, BaseModel):
@field_validator("guardrail", mode="before")
@classmethod
def validate_guardrail_function(
cls, v: GuardrailCallable | str | None
) -> GuardrailCallable | str | None:
cls, v: Callable | str | None
) -> Callable | str | None:
"""Validate that the guardrail function has the correct signature.
If v is a callable, validate that it has the correct signature.
@@ -246,11 +258,7 @@ class LiteAgent(FlowTrackable, BaseModel):
"""Return the original role for compatibility with tool interfaces."""
return self.role
def kickoff(
self,
messages: str | list[LLMMessage],
response_format: type[BaseModel] | None = None,
) -> LiteAgentOutput:
def kickoff(self, messages: str | list[LLMMessage]) -> LiteAgentOutput:
"""
Execute the agent with the given messages.
@@ -258,8 +266,6 @@ class LiteAgent(FlowTrackable, BaseModel):
messages: Either a string query or a list of message dictionaries.
If a string is provided, it will be converted to a user message.
If a list is provided, each dict should have 'role' and 'content' keys.
response_format: Optional Pydantic model for structured output. If provided,
overrides self.response_format for this execution.
Returns:
LiteAgentOutput: The result of the agent execution.
@@ -280,13 +286,9 @@ class LiteAgent(FlowTrackable, BaseModel):
self.tools_results = []
# Format messages for the LLM
self._messages = self._format_messages(
messages, response_format=response_format
)
self._messages = self._format_messages(messages)
return self._execute_core(
agent_info=agent_info, response_format=response_format
)
return self._execute_core(agent_info=agent_info)
except Exception as e:
self._printer.print(
@@ -304,9 +306,7 @@ class LiteAgent(FlowTrackable, BaseModel):
)
raise e
def _execute_core(
self, agent_info: dict[str, Any], response_format: type[BaseModel] | None = None
) -> LiteAgentOutput:
def _execute_core(self, agent_info: dict[str, Any]) -> LiteAgentOutput:
# Emit event for agent execution start
crewai_event_bus.emit(
self,
@@ -320,29 +320,15 @@ class LiteAgent(FlowTrackable, BaseModel):
# Execute the agent using invoke loop
agent_finish = self._invoke_loop()
formatted_result: BaseModel | None = None
active_response_format = response_format or self.response_format
if active_response_format:
if self.response_format:
try:
model_schema = generate_model_description(active_response_format)
schema = json.dumps(model_schema, indent=2)
instructions = self.i18n.slice("formatted_task_instructions").format(
output_format=schema
)
converter = Converter(
llm=self.llm,
text=agent_finish.output,
model=active_response_format,
instructions=instructions,
)
result = converter.to_pydantic()
# Cast to BaseModel to ensure type safety
result = self.response_format.model_validate_json(agent_finish.output)
if isinstance(result, BaseModel):
formatted_result = result
except ConverterError as e:
except Exception as e:
self._printer.print(
content=f"Failed to parse output into response format after retries: {e.message}",
content=f"Failed to parse output into response format: {e!s}",
color="yellow",
)
@@ -358,7 +344,6 @@ class LiteAgent(FlowTrackable, BaseModel):
pydantic=formatted_result,
agent_role=self.role,
usage_metrics=usage_metrics.model_dump() if usage_metrics else None,
messages=self._messages,
)
# Process guardrail if set
@@ -432,14 +417,8 @@ class LiteAgent(FlowTrackable, BaseModel):
"""
return await asyncio.to_thread(self.kickoff, messages)
def _get_default_system_prompt(
self, response_format: type[BaseModel] | None = None
) -> str:
"""Get the default system prompt for the agent.
Args:
response_format: Optional response format to use instead of self.response_format
"""
def _get_default_system_prompt(self) -> str:
"""Get the default system prompt for the agent."""
base_prompt = ""
if self._parsed_tools:
# Use the prompt template for agents with tools
@@ -460,31 +439,21 @@ class LiteAgent(FlowTrackable, BaseModel):
goal=self.goal,
)
active_response_format = response_format or self.response_format
if active_response_format:
model_description = generate_model_description(active_response_format)
schema_json = json.dumps(model_description, indent=2)
# Add response format instructions if specified
if self.response_format:
schema = generate_model_description(self.response_format)
base_prompt += self.i18n.slice("lite_agent_response_format").format(
response_format=schema_json
response_format=schema
)
return base_prompt
def _format_messages(
self,
messages: str | list[LLMMessage],
response_format: type[BaseModel] | None = None,
) -> list[LLMMessage]:
"""Format messages for the LLM.
Args:
messages: Input messages to format
response_format: Optional response format to use instead of self.response_format
"""
def _format_messages(self, messages: str | list[LLMMessage]) -> list[LLMMessage]:
"""Format messages for the LLM."""
if isinstance(messages, str):
messages = [{"role": "user", "content": messages}]
system_prompt = self._get_default_system_prompt(response_format=response_format)
system_prompt = self._get_default_system_prompt()
# Add system message at the beginning
formatted_messages: list[LLMMessage] = [
@@ -554,10 +523,6 @@ class LiteAgent(FlowTrackable, BaseModel):
self._append_message(formatted_answer.text, role="assistant")
except OutputParserError as e: # noqa: PERF203
self._printer.print(
content="Failed to parse LLM output. Retrying...",
color="yellow",
)
formatted_answer = handle_output_parser_exception(
e=e,
messages=self._messages,
@@ -594,7 +559,7 @@ class LiteAgent(FlowTrackable, BaseModel):
self._show_logs(formatted_answer)
return formatted_answer
def _show_logs(self, formatted_answer: AgentAction | AgentFinish) -> None:
def _show_logs(self, formatted_answer: AgentAction | AgentFinish):
"""Show logs for the agent's execution."""
crewai_event_bus.emit(
self,
@@ -609,4 +574,4 @@ class LiteAgent(FlowTrackable, BaseModel):
self, text: str, role: Literal["user", "assistant", "system"] = "assistant"
) -> None:
"""Append a message to the message list with the given role."""
self._messages.append(format_message_for_llm(text, role=role))
self._messages.append(cast(LLMMessage, format_message_for_llm(text, role=role)))

View File

@@ -6,8 +6,6 @@ from typing import Any
from pydantic import BaseModel, Field
from crewai.utilities.types import LLMMessage
class LiteAgentOutput(BaseModel):
"""Class that represents the result of a LiteAgent execution."""
@@ -22,7 +20,6 @@ class LiteAgentOutput(BaseModel):
usage_metrics: dict[str, Any] | None = Field(
description="Token usage metrics for this execution", default=None
)
messages: list[LLMMessage] = Field(description="Messages of the agent", default=[])
def to_dict(self) -> dict[str, Any]:
"""Convert pydantic_output to a dictionary."""

View File

@@ -20,9 +20,7 @@ from typing import (
)
from dotenv import load_dotenv
import httpx
from pydantic import BaseModel, Field
from typing_extensions import Self
from crewai.events.event_bus import crewai_event_bus
from crewai.events.types.llm_events import (
@@ -38,42 +36,30 @@ from crewai.events.types.tool_usage_events import (
ToolUsageStartedEvent,
)
from crewai.llms.base_llm import BaseLLM
from crewai.llms.constants import (
ANTHROPIC_MODELS,
AZURE_MODELS,
BEDROCK_MODELS,
GEMINI_MODELS,
OPENAI_MODELS,
)
from crewai.utilities import InternalInstructor
from crewai.utilities.exceptions.context_window_exceeding_exception import (
LLMContextLengthExceededError,
)
from crewai.utilities.logger_utils import suppress_warnings
from crewai.utilities.types import LLMMessage
if TYPE_CHECKING:
from litellm import Choices
from litellm.exceptions import ContextWindowExceededError
from litellm.litellm_core_utils.get_supported_openai_params import (
get_supported_openai_params,
)
from litellm.types.utils import ChatCompletionDeltaToolCall, Choices, ModelResponse
from litellm.types.utils import ChatCompletionDeltaToolCall, ModelResponse
from litellm.utils import supports_response_schema
from crewai.agent.core import Agent
from crewai.llms.hooks.base import BaseInterceptor
from crewai.task import Task
from crewai.tools.base_tool import BaseTool
from crewai.utilities.types import LLMMessage
try:
import litellm
from litellm import Choices, CustomLogger
from litellm.exceptions import ContextWindowExceededError
from litellm.integrations.custom_logger import CustomLogger
from litellm.litellm_core_utils.get_supported_openai_params import (
get_supported_openai_params,
)
from litellm.types.utils import ChatCompletionDeltaToolCall, Choices, ModelResponse
from litellm.types.utils import ChatCompletionDeltaToolCall, ModelResponse
from litellm.utils import supports_response_schema
LITELLM_AVAILABLE = True
@@ -86,7 +72,6 @@ except ImportError:
ChatCompletionDeltaToolCall = None # type: ignore
ModelResponse = None # type: ignore
supports_response_schema = None # type: ignore
CustomLogger = None # type: ignore
load_dotenv()
@@ -119,13 +104,11 @@ class FilteredStream(io.TextIOBase):
return self._original_stream.write(s)
def flush(self) -> None:
if self._lock:
with self._lock:
return self._original_stream.flush()
return None
def flush(self):
with self._lock:
return self._original_stream.flush()
def __getattr__(self, name: str) -> Any:
def __getattr__(self, name):
"""Delegate attribute access to the wrapped original stream.
This ensures compatibility with libraries (e.g., Rich) that rely on
@@ -139,16 +122,16 @@ class FilteredStream(io.TextIOBase):
# confuses Rich). These explicit pass-throughs ensure the wrapped Console
# still sees a fully-featured stream.
@property
def encoding(self) -> str | Any: # type: ignore[override]
def encoding(self):
return getattr(self._original_stream, "encoding", "utf-8")
def isatty(self) -> bool:
def isatty(self):
return self._original_stream.isatty()
def fileno(self) -> int:
def fileno(self):
return self._original_stream.fileno()
def writable(self) -> bool:
def writable(self):
return True
@@ -329,68 +312,18 @@ class AccumulatedToolArgs(BaseModel):
class LLM(BaseLLM):
completion_cost: float | None = None
def __new__(cls, model: str, is_litellm: bool = False, **kwargs: Any) -> LLM:
"""Factory method that routes to native SDK or falls back to LiteLLM.
Routing priority:
1. If 'provider' kwarg is present, use that provider with constants
2. If only 'model' kwarg, use constants to infer provider
3. If "/" in model name:
- Check if prefix is a native provider (openai/anthropic/azure/bedrock/gemini)
- If yes, validate model against constants
- If valid, route to native SDK; otherwise route to LiteLLM
"""
def __new__(cls, model: str, is_litellm: bool = False, **kwargs) -> LLM:
"""Factory method that routes to native SDK or falls back to LiteLLM."""
if not model or not isinstance(model, str):
raise ValueError("Model must be a non-empty string")
explicit_provider = kwargs.get("provider")
provider = model.partition("/")[0] if "/" in model else "openai"
if explicit_provider:
provider = explicit_provider
use_native = True
model_string = model
elif "/" in model:
prefix, _, model_part = model.partition("/")
provider_mapping = {
"openai": "openai",
"anthropic": "anthropic",
"claude": "anthropic",
"azure": "azure",
"azure_openai": "azure",
"google": "gemini",
"gemini": "gemini",
"bedrock": "bedrock",
"aws": "bedrock",
}
canonical_provider = provider_mapping.get(prefix.lower())
if canonical_provider and cls._validate_model_in_constants(
model_part, canonical_provider
):
provider = canonical_provider
use_native = True
model_string = model_part
else:
provider = prefix
use_native = False
model_string = model_part
else:
provider = cls._infer_provider_from_model(model)
use_native = True
model_string = model
native_class = cls._get_native_provider(provider) if use_native else None
native_class = cls._get_native_provider(provider)
if native_class and not is_litellm and provider in SUPPORTED_NATIVE_PROVIDERS:
try:
# Remove 'provider' from kwargs if it exists to avoid duplicate keyword argument
kwargs_copy = {k: v for k, v in kwargs.items() if k != 'provider'}
return cast(
Self, native_class(model=model_string, provider=provider, **kwargs_copy)
)
except NotImplementedError:
raise
model_string = model.partition("/")[2] if "/" in model else model
return native_class(model=model_string, provider=provider, **kwargs)
except Exception as e:
raise ImportError(f"Error importing native provider: {e}") from e
@@ -404,63 +337,6 @@ class LLM(BaseLLM):
instance.is_litellm = True
return instance
@classmethod
def _validate_model_in_constants(cls, model: str, provider: str) -> bool:
"""Validate if a model name exists in the provider's constants.
Args:
model: The model name to validate
provider: The provider to check against (canonical name)
Returns:
True if the model exists in the provider's constants, False otherwise
"""
if provider == "openai":
return model in OPENAI_MODELS
if provider == "anthropic" or provider == "claude":
return model in ANTHROPIC_MODELS
if provider == "gemini":
return model in GEMINI_MODELS
if provider == "bedrock":
return model in BEDROCK_MODELS
if provider == "azure":
# azure does not provide a list of available models, determine a better way to handle this
return True
return False
@classmethod
def _infer_provider_from_model(cls, model: str) -> str:
"""Infer the provider from the model name.
Args:
model: The model name without provider prefix
Returns:
The inferred provider name, defaults to "openai"
"""
if model in OPENAI_MODELS:
return "openai"
if model in ANTHROPIC_MODELS:
return "anthropic"
if model in GEMINI_MODELS:
return "gemini"
if model in BEDROCK_MODELS:
return "bedrock"
if model in AZURE_MODELS:
return "azure"
return "openai"
@classmethod
def _get_native_provider(cls, provider: str) -> type | None:
"""Get native provider class if available."""
@@ -517,22 +393,13 @@ class LLM(BaseLLM):
callbacks: list[Any] | None = None,
reasoning_effort: Literal["none", "low", "medium", "high"] | None = None,
stream: bool = False,
interceptor: BaseInterceptor[httpx.Request, httpx.Response] | None = None,
**kwargs: Any,
) -> None:
**kwargs,
):
"""Initialize LLM instance.
Note: This __init__ method is only called for fallback instances.
Native provider instances handle their own initialization in their respective classes.
"""
super().__init__(
model=model,
temperature=temperature,
api_key=api_key,
base_url=base_url,
timeout=timeout,
**kwargs,
)
self.model = model
self.timeout = timeout
self.temperature = temperature
@@ -557,7 +424,6 @@ class LLM(BaseLLM):
self.additional_params = kwargs
self.is_anthropic = self._is_anthropic_model(model)
self.stream = stream
self.interceptor = interceptor
litellm.drop_params = True
@@ -588,7 +454,7 @@ class LLM(BaseLLM):
def _prepare_completion_params(
self,
messages: str | list[LLMMessage],
tools: list[dict[str, BaseTool]] | None = None,
tools: list[dict] | None = None,
) -> dict[str, Any]:
"""Prepare parameters for the completion call.
@@ -639,10 +505,9 @@ class LLM(BaseLLM):
params: dict[str, Any],
callbacks: list[Any] | None = None,
available_functions: dict[str, Any] | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
response_model: type[BaseModel] | None = None,
) -> Any:
from_task: Any | None = None,
from_agent: Any | None = None,
) -> str:
"""Handle a streaming response from the LLM.
Args:
@@ -651,7 +516,6 @@ class LLM(BaseLLM):
available_functions: Dict of available functions
from_task: Optional task object
from_agent: Optional agent object
response_model: Optional response model
Returns:
str: The complete response text
@@ -852,30 +716,14 @@ class LLM(BaseLLM):
tool_calls = message.tool_calls
except Exception as e:
logging.debug(f"Error checking for tool calls: {e}")
# --- 8) If no tool calls or no available functions, return the text response directly
if not tool_calls or not available_functions:
# Track token usage and log callbacks if available in streaming mode
if usage_info:
self._track_token_usage_internal(usage_info)
self._handle_streaming_callbacks(callbacks, usage_info, last_chunk)
if response_model and self.is_litellm:
instructor_instance = InternalInstructor(
content=full_response,
model=response_model,
llm=self,
)
result = instructor_instance.to_pydantic()
structured_response = result.model_dump_json()
self._handle_emit_call_events(
response=structured_response,
call_type=LLMCallType.LLM_CALL,
from_task=from_task,
from_agent=from_agent,
messages=params["messages"],
)
return structured_response
# Emit completion event and return response
self._handle_emit_call_events(
response=full_response,
call_type=LLMCallType.LLM_CALL,
@@ -936,9 +784,9 @@ class LLM(BaseLLM):
tool_calls: list[ChatCompletionDeltaToolCall],
accumulated_tool_args: defaultdict[int, AccumulatedToolArgs],
available_functions: dict[str, Any] | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
) -> Any:
from_task: Any | None = None,
from_agent: Any | None = None,
) -> None | str:
for tool_call in tool_calls:
current_tool_accumulator = accumulated_tool_args[tool_call.index]
@@ -1021,9 +869,8 @@ class LLM(BaseLLM):
params: dict[str, Any],
callbacks: list[Any] | None = None,
available_functions: dict[str, Any] | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
response_model: type[BaseModel] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
) -> str | Any:
"""Handle a non-streaming response from the LLM.
@@ -1033,69 +880,23 @@ class LLM(BaseLLM):
available_functions: Dict of available functions
from_task: Optional Task that invoked the LLM
from_agent: Optional Agent that invoked the LLM
response_model: Optional Response model
Returns:
str: The response text
"""
# --- 1) Handle response_model with InternalInstructor for LiteLLM
if response_model and self.is_litellm:
from crewai.utilities.internal_instructor import InternalInstructor
messages = params.get("messages", [])
if not messages:
raise ValueError("Messages are required when using response_model")
# Combine all message content for InternalInstructor
combined_content = "\n\n".join(
f"{msg['role'].upper()}: {msg['content']}" for msg in messages
)
instructor_instance = InternalInstructor(
content=combined_content,
model=response_model,
llm=self,
)
result = instructor_instance.to_pydantic()
structured_response = result.model_dump_json()
self._handle_emit_call_events(
response=structured_response,
call_type=LLMCallType.LLM_CALL,
from_task=from_task,
from_agent=from_agent,
messages=params["messages"],
)
return structured_response
# --- 1) Make the completion call
try:
# Attempt to make the completion call, but catch context window errors
# and convert them to our own exception type for consistent handling
# across the codebase. This allows CrewAgentExecutor to handle context
# length issues appropriately.
if response_model:
params["response_model"] = response_model
response = litellm.completion(**params)
except ContextWindowExceededError as e:
# Convert litellm's context window error to our own exception type
# for consistent handling in the rest of the codebase
raise LLMContextLengthExceededError(str(e)) from e
# --- 2) Handle structured output response (when response_model is provided)
if response_model is not None:
# When using instructor/response_model, litellm returns a Pydantic model instance
if isinstance(response, BaseModel):
structured_response = response.model_dump_json()
self._handle_emit_call_events(
response=structured_response,
call_type=LLMCallType.LLM_CALL,
from_task=from_task,
from_agent=from_agent,
messages=params["messages"],
)
return structured_response
# --- 3) Extract response message and content (standard response)
# --- 2) Extract response message and content
response_message = cast(Choices, cast(ModelResponse, response).choices)[
0
].message
@@ -1150,9 +951,9 @@ class LLM(BaseLLM):
self,
tool_calls: list[Any],
available_functions: dict[str, Any] | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
) -> Any:
from_task: Any | None = None,
from_agent: Any | None = None,
) -> str | None:
"""Handle a tool call from the LLM.
Args:
@@ -1238,12 +1039,11 @@ class LLM(BaseLLM):
def call(
self,
messages: str | list[LLMMessage],
tools: list[dict[str, BaseTool]] | None = None,
tools: list[dict] | None = None,
callbacks: list[Any] | None = None,
available_functions: dict[str, Any] | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
response_model: type[BaseModel] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
) -> str | Any:
"""High-level LLM call method.
@@ -1260,7 +1060,6 @@ class LLM(BaseLLM):
that can be invoked by the LLM.
from_task: Optional Task that invoked the LLM
from_agent: Optional Agent that invoked the LLM
response_model: Optional Model that contains a pydantic response model.
Returns:
Union[str, Any]: Either a text response from the LLM (str) or
@@ -1306,21 +1105,11 @@ class LLM(BaseLLM):
# --- 7) Make the completion call and handle response
if self.stream:
return self._handle_streaming_response(
params=params,
callbacks=callbacks,
available_functions=available_functions,
from_task=from_task,
from_agent=from_agent,
response_model=response_model,
params, callbacks, available_functions, from_task, from_agent
)
return self._handle_non_streaming_response(
params=params,
callbacks=callbacks,
available_functions=available_functions,
from_task=from_task,
from_agent=from_agent,
response_model=response_model,
params, callbacks, available_functions, from_task, from_agent
)
except LLMContextLengthExceededError:
# Re-raise LLMContextLengthExceededError as it should be handled
@@ -1352,7 +1141,6 @@ class LLM(BaseLLM):
available_functions=available_functions,
from_task=from_task,
from_agent=from_agent,
response_model=response_model,
)
crewai_event_bus.emit(
@@ -1367,10 +1155,10 @@ class LLM(BaseLLM):
self,
response: Any,
call_type: LLMCallType,
from_task: Task | None = None,
from_agent: Agent | None = None,
messages: str | list[LLMMessage] | None = None,
) -> None:
from_task: Any | None = None,
from_agent: Any | None = None,
messages: str | list[dict[str, Any]] | None = None,
):
"""Handle the events for the LLM call.
Args:
@@ -1536,7 +1324,7 @@ class LLM(BaseLLM):
return self.context_window_size
@staticmethod
def set_callbacks(callbacks: list[Any]) -> None:
def set_callbacks(callbacks: list[Any]):
"""
Attempt to keep a single set of callbacks in litellm by removing old
duplicates and adding new ones.
@@ -1589,7 +1377,7 @@ class LLM(BaseLLM):
litellm.success_callback = success_callbacks
litellm.failure_callback = failure_callbacks
def __copy__(self) -> LLM:
def __copy__(self):
"""Create a shallow copy of the LLM instance."""
# Filter out parameters that are already explicitly passed to avoid conflicts
filtered_params = {
@@ -1649,7 +1437,7 @@ class LLM(BaseLLM):
**filtered_params,
)
def __deepcopy__(self, memo: dict[int, Any] | None) -> LLM:
def __deepcopy__(self, memo):
"""Create a deep copy of the LLM instance."""
import copy

View File

@@ -10,7 +10,6 @@ from abc import ABC, abstractmethod
from datetime import datetime
import json
import logging
import re
from typing import TYPE_CHECKING, Any, Final
from pydantic import BaseModel
@@ -32,15 +31,11 @@ from crewai.types.usage_metrics import UsageMetrics
if TYPE_CHECKING:
from crewai.agent.core import Agent
from crewai.task import Task
from crewai.tools.base_tool import BaseTool
from crewai.utilities.types import LLMMessage
DEFAULT_CONTEXT_WINDOW_SIZE: Final[int] = 4096
DEFAULT_SUPPORTS_STOP_WORDS: Final[bool] = True
_JSON_EXTRACTION_PATTERN: Final[re.Pattern[str]] = re.compile(r"\{.*}", re.DOTALL)
class BaseLLM(ABC):
@@ -70,8 +65,9 @@ class BaseLLM(ABC):
temperature: float | None = None,
api_key: str | None = None,
base_url: str | None = None,
timeout: float | None = None,
provider: str | None = None,
**kwargs: Any,
**kwargs,
) -> None:
"""Initialize the BaseLLM with default attributes.
@@ -97,10 +93,8 @@ class BaseLLM(ABC):
self.stop: list[str] = []
elif isinstance(stop, str):
self.stop = [stop]
elif isinstance(stop, list):
self.stop = stop
else:
self.stop = []
self.stop = stop
self._token_usage = {
"total_tokens": 0,
@@ -124,12 +118,11 @@ class BaseLLM(ABC):
def call(
self,
messages: str | list[LLMMessage],
tools: list[dict[str, BaseTool]] | None = None,
tools: list[dict] | None = None,
callbacks: list[Any] | None = None,
available_functions: dict[str, Any] | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
response_model: type[BaseModel] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
) -> str | Any:
"""Call the LLM with the given messages.
@@ -146,7 +139,6 @@ class BaseLLM(ABC):
that can be invoked by the LLM.
from_task: Optional task caller to be used for the LLM call.
from_agent: Optional agent caller to be used for the LLM call.
response_model: Optional response model to be used for the LLM call.
Returns:
Either a text response from the LLM (str) or
@@ -158,9 +150,7 @@ class BaseLLM(ABC):
RuntimeError: If the LLM request fails for other reasons.
"""
def _convert_tools_for_interference(
self, tools: list[dict[str, BaseTool]]
) -> list[dict[str, BaseTool]]:
def _convert_tools_for_interference(self, tools: list[dict]) -> list[dict]:
"""Convert tools to a format that can be used for interference.
Args:
@@ -247,11 +237,11 @@ class BaseLLM(ABC):
def _emit_call_started_event(
self,
messages: str | list[LLMMessage],
tools: list[dict[str, BaseTool]] | None = None,
tools: list[dict] | None = None,
callbacks: list[Any] | None = None,
available_functions: dict[str, Any] | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
) -> None:
"""Emit LLM call started event."""
if not hasattr(crewai_event_bus, "emit"):
@@ -274,8 +264,8 @@ class BaseLLM(ABC):
self,
response: Any,
call_type: LLMCallType,
from_task: Task | None = None,
from_agent: Agent | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
messages: str | list[dict[str, Any]] | None = None,
) -> None:
"""Emit LLM call completed event."""
@@ -294,8 +284,8 @@ class BaseLLM(ABC):
def _emit_call_failed_event(
self,
error: str,
from_task: Task | None = None,
from_agent: Agent | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
) -> None:
"""Emit LLM call failed event."""
if not hasattr(crewai_event_bus, "emit"):
@@ -313,8 +303,8 @@ class BaseLLM(ABC):
def _emit_stream_chunk_event(
self,
chunk: str,
from_task: Task | None = None,
from_agent: Agent | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
tool_call: dict[str, Any] | None = None,
) -> None:
"""Emit stream chunk event."""
@@ -336,8 +326,8 @@ class BaseLLM(ABC):
function_name: str,
function_args: dict[str, Any],
available_functions: dict[str, Any],
from_task: Task | None = None,
from_agent: Agent | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
) -> str | None:
"""Handle tool execution with proper event emission.
@@ -453,10 +443,10 @@ class BaseLLM(ABC):
f"Message at index {i} must have 'role' and 'content' keys"
)
return messages
return messages # type: ignore[return-value]
@staticmethod
def _validate_structured_output(
self,
response: str,
response_format: type[BaseModel] | None,
) -> str | BaseModel:
@@ -481,7 +471,10 @@ class BaseLLM(ABC):
data = json.loads(response)
return response_format.model_validate(data)
json_match = _JSON_EXTRACTION_PATTERN.search(response)
# Try to extract JSON from response
import re
json_match = re.search(r"\{.*\}", response, re.DOTALL)
if json_match:
data = json.loads(json_match.group())
return response_format.model_validate(data)
@@ -494,8 +487,7 @@ class BaseLLM(ABC):
f"Failed to parse response into {response_format.__name__}: {e}"
) from e
@staticmethod
def _extract_provider(model: str) -> str:
def _extract_provider(self, model: str) -> str:
"""Extract provider from model string.
Args:

View File

@@ -1,558 +0,0 @@
from typing import Literal, TypeAlias
OpenAIModels: TypeAlias = Literal[
"gpt-3.5-turbo",
"gpt-3.5-turbo-0125",
"gpt-3.5-turbo-0301",
"gpt-3.5-turbo-0613",
"gpt-3.5-turbo-1106",
"gpt-3.5-turbo-16k",
"gpt-3.5-turbo-16k-0613",
"gpt-3.5-turbo-instruct",
"gpt-3.5-turbo-instruct-0914",
"gpt-4",
"gpt-4-0125-preview",
"gpt-4-0314",
"gpt-4-0613",
"gpt-4-1106-preview",
"gpt-4-32k",
"gpt-4-32k-0314",
"gpt-4-32k-0613",
"gpt-4-turbo",
"gpt-4-turbo-2024-04-09",
"gpt-4-turbo-preview",
"gpt-4-vision-preview",
"gpt-4.1",
"gpt-4.1-2025-04-14",
"gpt-4.1-mini",
"gpt-4.1-mini-2025-04-14",
"gpt-4.1-nano",
"gpt-4.1-nano-2025-04-14",
"gpt-4o",
"gpt-4o-2024-05-13",
"gpt-4o-2024-08-06",
"gpt-4o-2024-11-20",
"gpt-4o-audio-preview",
"gpt-4o-audio-preview-2024-10-01",
"gpt-4o-audio-preview-2024-12-17",
"gpt-4o-audio-preview-2025-06-03",
"gpt-4o-mini",
"gpt-4o-mini-2024-07-18",
"gpt-4o-mini-audio-preview",
"gpt-4o-mini-audio-preview-2024-12-17",
"gpt-4o-mini-realtime-preview",
"gpt-4o-mini-realtime-preview-2024-12-17",
"gpt-4o-mini-search-preview",
"gpt-4o-mini-search-preview-2025-03-11",
"gpt-4o-mini-transcribe",
"gpt-4o-mini-tts",
"gpt-4o-realtime-preview",
"gpt-4o-realtime-preview-2024-10-01",
"gpt-4o-realtime-preview-2024-12-17",
"gpt-4o-realtime-preview-2025-06-03",
"gpt-4o-search-preview",
"gpt-4o-search-preview-2025-03-11",
"gpt-4o-transcribe",
"gpt-4o-transcribe-diarize",
"gpt-5",
"gpt-5-2025-08-07",
"gpt-5-chat",
"gpt-5-chat-latest",
"gpt-5-codex",
"gpt-5-mini",
"gpt-5-mini-2025-08-07",
"gpt-5-nano",
"gpt-5-nano-2025-08-07",
"gpt-5-pro",
"gpt-5-pro-2025-10-06",
"gpt-5-search-api",
"gpt-5-search-api-2025-10-14",
"gpt-audio",
"gpt-audio-2025-08-28",
"gpt-audio-mini",
"gpt-audio-mini-2025-10-06",
"gpt-image-1",
"gpt-image-1-mini",
"gpt-realtime",
"gpt-realtime-2025-08-28",
"gpt-realtime-mini",
"gpt-realtime-mini-2025-10-06",
"o1",
"o1-preview",
"o1-2024-12-17",
"o1-mini",
"o1-mini-2024-09-12",
"o1-pro",
"o1-pro-2025-03-19",
"o3-mini",
"o3",
"o4-mini",
"whisper-1",
]
OPENAI_MODELS: list[OpenAIModels] = [
"gpt-3.5-turbo",
"gpt-3.5-turbo-0125",
"gpt-3.5-turbo-0301",
"gpt-3.5-turbo-0613",
"gpt-3.5-turbo-1106",
"gpt-3.5-turbo-16k",
"gpt-3.5-turbo-16k-0613",
"gpt-3.5-turbo-instruct",
"gpt-3.5-turbo-instruct-0914",
"gpt-4",
"gpt-4-0125-preview",
"gpt-4-0314",
"gpt-4-0613",
"gpt-4-1106-preview",
"gpt-4-32k",
"gpt-4-32k-0314",
"gpt-4-32k-0613",
"gpt-4-turbo",
"gpt-4-turbo-2024-04-09",
"gpt-4-turbo-preview",
"gpt-4-vision-preview",
"gpt-4.1",
"gpt-4.1-2025-04-14",
"gpt-4.1-mini",
"gpt-4.1-mini-2025-04-14",
"gpt-4.1-nano",
"gpt-4.1-nano-2025-04-14",
"gpt-4o",
"gpt-4o-2024-05-13",
"gpt-4o-2024-08-06",
"gpt-4o-2024-11-20",
"gpt-4o-audio-preview",
"gpt-4o-audio-preview-2024-10-01",
"gpt-4o-audio-preview-2024-12-17",
"gpt-4o-audio-preview-2025-06-03",
"gpt-4o-mini",
"gpt-4o-mini-2024-07-18",
"gpt-4o-mini-audio-preview",
"gpt-4o-mini-audio-preview-2024-12-17",
"gpt-4o-mini-realtime-preview",
"gpt-4o-mini-realtime-preview-2024-12-17",
"gpt-4o-mini-search-preview",
"gpt-4o-mini-search-preview-2025-03-11",
"gpt-4o-mini-transcribe",
"gpt-4o-mini-tts",
"gpt-4o-realtime-preview",
"gpt-4o-realtime-preview-2024-10-01",
"gpt-4o-realtime-preview-2024-12-17",
"gpt-4o-realtime-preview-2025-06-03",
"gpt-4o-search-preview",
"gpt-4o-search-preview-2025-03-11",
"gpt-4o-transcribe",
"gpt-4o-transcribe-diarize",
"gpt-5",
"gpt-5-2025-08-07",
"gpt-5-chat",
"gpt-5-chat-latest",
"gpt-5-codex",
"gpt-5-mini",
"gpt-5-mini-2025-08-07",
"gpt-5-nano",
"gpt-5-nano-2025-08-07",
"gpt-5-pro",
"gpt-5-pro-2025-10-06",
"gpt-5-search-api",
"gpt-5-search-api-2025-10-14",
"gpt-audio",
"gpt-audio-2025-08-28",
"gpt-audio-mini",
"gpt-audio-mini-2025-10-06",
"gpt-image-1",
"gpt-image-1-mini",
"gpt-realtime",
"gpt-realtime-2025-08-28",
"gpt-realtime-mini",
"gpt-realtime-mini-2025-10-06",
"o1",
"o1-preview",
"o1-2024-12-17",
"o1-mini",
"o1-mini-2024-09-12",
"o1-pro",
"o1-pro-2025-03-19",
"o3-mini",
"o3",
"o4-mini",
"whisper-1",
]
AnthropicModels: TypeAlias = Literal[
"claude-3-7-sonnet-latest",
"claude-3-7-sonnet-20250219",
"claude-3-5-haiku-latest",
"claude-3-5-haiku-20241022",
"claude-haiku-4-5",
"claude-haiku-4-5-20251001",
"claude-sonnet-4-20250514",
"claude-sonnet-4-0",
"claude-4-sonnet-20250514",
"claude-sonnet-4-5",
"claude-sonnet-4-5-20250929",
"claude-3-5-sonnet-latest",
"claude-3-5-sonnet-20241022",
"claude-3-5-sonnet-20240620",
"claude-opus-4-0",
"claude-opus-4-20250514",
"claude-4-opus-20250514",
"claude-opus-4-1",
"claude-opus-4-1-20250805",
"claude-3-opus-latest",
"claude-3-opus-20240229",
"claude-3-sonnet-20240229",
"claude-3-haiku-latest",
"claude-3-haiku-20240307",
]
ANTHROPIC_MODELS: list[AnthropicModels] = [
"claude-3-7-sonnet-latest",
"claude-3-7-sonnet-20250219",
"claude-3-5-haiku-latest",
"claude-3-5-haiku-20241022",
"claude-haiku-4-5",
"claude-haiku-4-5-20251001",
"claude-sonnet-4-20250514",
"claude-sonnet-4-0",
"claude-4-sonnet-20250514",
"claude-sonnet-4-5",
"claude-sonnet-4-5-20250929",
"claude-3-5-sonnet-latest",
"claude-3-5-sonnet-20241022",
"claude-3-5-sonnet-20240620",
"claude-opus-4-0",
"claude-opus-4-20250514",
"claude-4-opus-20250514",
"claude-opus-4-1",
"claude-opus-4-1-20250805",
"claude-3-opus-latest",
"claude-3-opus-20240229",
"claude-3-sonnet-20240229",
"claude-3-haiku-latest",
"claude-3-haiku-20240307",
]
GeminiModels: TypeAlias = Literal[
"gemini-2.5-pro",
"gemini-2.5-pro-preview-03-25",
"gemini-2.5-pro-preview-05-06",
"gemini-2.5-pro-preview-06-05",
"gemini-2.5-flash",
"gemini-2.5-flash-preview-05-20",
"gemini-2.5-flash-preview-04-17",
"gemini-2.5-flash-image",
"gemini-2.5-flash-image-preview",
"gemini-2.5-flash-lite",
"gemini-2.5-flash-lite-preview-06-17",
"gemini-2.5-flash-preview-09-2025",
"gemini-2.5-flash-lite-preview-09-2025",
"gemini-2.5-flash-preview-tts",
"gemini-2.5-pro-preview-tts",
"gemini-2.5-computer-use-preview-10-2025",
"gemini-2.0-flash",
"gemini-2.0-flash-001",
"gemini-2.0-flash-exp",
"gemini-2.0-flash-exp-image-generation",
"gemini-2.0-flash-lite",
"gemini-2.0-flash-lite-001",
"gemini-2.0-flash-lite-preview",
"gemini-2.0-flash-lite-preview-02-05",
"gemini-2.0-flash-preview-image-generation",
"gemini-2.0-flash-thinking-exp",
"gemini-2.0-flash-thinking-exp-01-21",
"gemini-2.0-flash-thinking-exp-1219",
"gemini-2.0-pro-exp",
"gemini-2.0-pro-exp-02-05",
"gemini-exp-1206",
"gemini-1.5-pro",
"gemini-1.5-flash",
"gemini-1.5-flash-8b",
"gemini-flash-latest",
"gemini-flash-lite-latest",
"gemini-pro-latest",
"gemini-2.0-flash-live-001",
"gemini-live-2.5-flash-preview",
"gemini-2.5-flash-live-preview",
"gemini-robotics-er-1.5-preview",
"gemini-gemma-2-27b-it",
"gemini-gemma-2-9b-it",
"gemma-3-1b-it",
"gemma-3-4b-it",
"gemma-3-12b-it",
"gemma-3-27b-it",
"gemma-3n-e2b-it",
"gemma-3n-e4b-it",
"learnlm-2.0-flash-experimental",
]
GEMINI_MODELS: list[GeminiModels] = [
"gemini-2.5-pro",
"gemini-2.5-pro-preview-03-25",
"gemini-2.5-pro-preview-05-06",
"gemini-2.5-pro-preview-06-05",
"gemini-2.5-flash",
"gemini-2.5-flash-preview-05-20",
"gemini-2.5-flash-preview-04-17",
"gemini-2.5-flash-image",
"gemini-2.5-flash-image-preview",
"gemini-2.5-flash-lite",
"gemini-2.5-flash-lite-preview-06-17",
"gemini-2.5-flash-preview-09-2025",
"gemini-2.5-flash-lite-preview-09-2025",
"gemini-2.5-flash-preview-tts",
"gemini-2.5-pro-preview-tts",
"gemini-2.5-computer-use-preview-10-2025",
"gemini-2.0-flash",
"gemini-2.0-flash-001",
"gemini-2.0-flash-exp",
"gemini-2.0-flash-exp-image-generation",
"gemini-2.0-flash-lite",
"gemini-2.0-flash-lite-001",
"gemini-2.0-flash-lite-preview",
"gemini-2.0-flash-lite-preview-02-05",
"gemini-2.0-flash-preview-image-generation",
"gemini-2.0-flash-thinking-exp",
"gemini-2.0-flash-thinking-exp-01-21",
"gemini-2.0-flash-thinking-exp-1219",
"gemini-2.0-pro-exp",
"gemini-2.0-pro-exp-02-05",
"gemini-exp-1206",
"gemini-1.5-pro",
"gemini-1.5-flash",
"gemini-1.5-flash-8b",
"gemini-flash-latest",
"gemini-flash-lite-latest",
"gemini-pro-latest",
"gemini-2.0-flash-live-001",
"gemini-live-2.5-flash-preview",
"gemini-2.5-flash-live-preview",
"gemini-robotics-er-1.5-preview",
"gemini-gemma-2-27b-it",
"gemini-gemma-2-9b-it",
"gemma-3-1b-it",
"gemma-3-4b-it",
"gemma-3-12b-it",
"gemma-3-27b-it",
"gemma-3n-e2b-it",
"gemma-3n-e4b-it",
"learnlm-2.0-flash-experimental",
]
AzureModels: TypeAlias = Literal[
"gpt-3.5-turbo",
"gpt-3.5-turbo-0301",
"gpt-3.5-turbo-0613",
"gpt-3.5-turbo-16k",
"gpt-3.5-turbo-16k-0613",
"gpt-35-turbo",
"gpt-35-turbo-0125",
"gpt-35-turbo-1106",
"gpt-35-turbo-16k-0613",
"gpt-35-turbo-instruct-0914",
"gpt-4",
"gpt-4-0314",
"gpt-4-0613",
"gpt-4-1106-preview",
"gpt-4-0125-preview",
"gpt-4-32k",
"gpt-4-32k-0314",
"gpt-4-32k-0613",
"gpt-4-turbo",
"gpt-4-turbo-2024-04-09",
"gpt-4-vision",
"gpt-4o",
"gpt-4o-2024-05-13",
"gpt-4o-2024-08-06",
"gpt-4o-2024-11-20",
"gpt-4o-mini",
"gpt-5",
"o1",
"o1-mini",
"o1-preview",
"o3-mini",
"o3",
"o4-mini",
]
AZURE_MODELS: list[AzureModels] = [
"gpt-3.5-turbo",
"gpt-3.5-turbo-0301",
"gpt-3.5-turbo-0613",
"gpt-3.5-turbo-16k",
"gpt-3.5-turbo-16k-0613",
"gpt-35-turbo",
"gpt-35-turbo-0125",
"gpt-35-turbo-1106",
"gpt-35-turbo-16k-0613",
"gpt-35-turbo-instruct-0914",
"gpt-4",
"gpt-4-0314",
"gpt-4-0613",
"gpt-4-1106-preview",
"gpt-4-0125-preview",
"gpt-4-32k",
"gpt-4-32k-0314",
"gpt-4-32k-0613",
"gpt-4-turbo",
"gpt-4-turbo-2024-04-09",
"gpt-4-vision",
"gpt-4o",
"gpt-4o-2024-05-13",
"gpt-4o-2024-08-06",
"gpt-4o-2024-11-20",
"gpt-4o-mini",
"gpt-5",
"o1",
"o1-mini",
"o1-preview",
"o3-mini",
"o3",
"o4-mini",
]
BedrockModels: TypeAlias = Literal[
"ai21.jamba-1-5-large-v1:0",
"ai21.jamba-1-5-mini-v1:0",
"amazon.nova-lite-v1:0",
"amazon.nova-lite-v1:0:24k",
"amazon.nova-lite-v1:0:300k",
"amazon.nova-micro-v1:0",
"amazon.nova-micro-v1:0:128k",
"amazon.nova-micro-v1:0:24k",
"amazon.nova-premier-v1:0",
"amazon.nova-premier-v1:0:1000k",
"amazon.nova-premier-v1:0:20k",
"amazon.nova-premier-v1:0:8k",
"amazon.nova-premier-v1:0:mm",
"amazon.nova-pro-v1:0",
"amazon.nova-pro-v1:0:24k",
"amazon.nova-pro-v1:0:300k",
"amazon.titan-text-express-v1",
"amazon.titan-text-express-v1:0:8k",
"amazon.titan-text-lite-v1",
"amazon.titan-text-lite-v1:0:4k",
"amazon.titan-tg1-large",
"anthropic.claude-3-5-haiku-20241022-v1:0",
"anthropic.claude-3-5-sonnet-20240620-v1:0",
"anthropic.claude-3-5-sonnet-20241022-v2:0",
"anthropic.claude-3-7-sonnet-20250219-v1:0",
"anthropic.claude-3-haiku-20240307-v1:0",
"anthropic.claude-3-haiku-20240307-v1:0:200k",
"anthropic.claude-3-haiku-20240307-v1:0:48k",
"anthropic.claude-3-opus-20240229-v1:0",
"anthropic.claude-3-opus-20240229-v1:0:12k",
"anthropic.claude-3-opus-20240229-v1:0:200k",
"anthropic.claude-3-opus-20240229-v1:0:28k",
"anthropic.claude-3-sonnet-20240229-v1:0",
"anthropic.claude-3-sonnet-20240229-v1:0:200k",
"anthropic.claude-3-sonnet-20240229-v1:0:28k",
"anthropic.claude-haiku-4-5-20251001-v1:0",
"anthropic.claude-instant-v1:2:100k",
"anthropic.claude-opus-4-1-20250805-v1:0",
"anthropic.claude-opus-4-20250514-v1:0",
"anthropic.claude-sonnet-4-20250514-v1:0",
"anthropic.claude-sonnet-4-5-20250929-v1:0",
"anthropic.claude-v2:0:100k",
"anthropic.claude-v2:0:18k",
"anthropic.claude-v2:1:18k",
"anthropic.claude-v2:1:200k",
"cohere.command-r-plus-v1:0",
"cohere.command-r-v1:0",
"cohere.rerank-v3-5:0",
"deepseek.r1-v1:0",
"meta.llama3-1-70b-instruct-v1:0",
"meta.llama3-1-8b-instruct-v1:0",
"meta.llama3-2-11b-instruct-v1:0",
"meta.llama3-2-1b-instruct-v1:0",
"meta.llama3-2-3b-instruct-v1:0",
"meta.llama3-2-90b-instruct-v1:0",
"meta.llama3-3-70b-instruct-v1:0",
"meta.llama3-70b-instruct-v1:0",
"meta.llama3-8b-instruct-v1:0",
"meta.llama4-maverick-17b-instruct-v1:0",
"meta.llama4-scout-17b-instruct-v1:0",
"mistral.mistral-7b-instruct-v0:2",
"mistral.mistral-large-2402-v1:0",
"mistral.mistral-small-2402-v1:0",
"mistral.mixtral-8x7b-instruct-v0:1",
"mistral.pixtral-large-2502-v1:0",
"openai.gpt-oss-120b-1:0",
"openai.gpt-oss-20b-1:0",
"qwen.qwen3-32b-v1:0",
"qwen.qwen3-coder-30b-a3b-v1:0",
"twelvelabs.pegasus-1-2-v1:0",
]
BEDROCK_MODELS: list[BedrockModels] = [
"ai21.jamba-1-5-large-v1:0",
"ai21.jamba-1-5-mini-v1:0",
"amazon.nova-lite-v1:0",
"amazon.nova-lite-v1:0:24k",
"amazon.nova-lite-v1:0:300k",
"amazon.nova-micro-v1:0",
"amazon.nova-micro-v1:0:128k",
"amazon.nova-micro-v1:0:24k",
"amazon.nova-premier-v1:0",
"amazon.nova-premier-v1:0:1000k",
"amazon.nova-premier-v1:0:20k",
"amazon.nova-premier-v1:0:8k",
"amazon.nova-premier-v1:0:mm",
"amazon.nova-pro-v1:0",
"amazon.nova-pro-v1:0:24k",
"amazon.nova-pro-v1:0:300k",
"amazon.titan-text-express-v1",
"amazon.titan-text-express-v1:0:8k",
"amazon.titan-text-lite-v1",
"amazon.titan-text-lite-v1:0:4k",
"amazon.titan-tg1-large",
"anthropic.claude-3-5-haiku-20241022-v1:0",
"anthropic.claude-3-5-sonnet-20240620-v1:0",
"anthropic.claude-3-5-sonnet-20241022-v2:0",
"anthropic.claude-3-7-sonnet-20250219-v1:0",
"anthropic.claude-3-haiku-20240307-v1:0",
"anthropic.claude-3-haiku-20240307-v1:0:200k",
"anthropic.claude-3-haiku-20240307-v1:0:48k",
"anthropic.claude-3-opus-20240229-v1:0",
"anthropic.claude-3-opus-20240229-v1:0:12k",
"anthropic.claude-3-opus-20240229-v1:0:200k",
"anthropic.claude-3-opus-20240229-v1:0:28k",
"anthropic.claude-3-sonnet-20240229-v1:0",
"anthropic.claude-3-sonnet-20240229-v1:0:200k",
"anthropic.claude-3-sonnet-20240229-v1:0:28k",
"anthropic.claude-haiku-4-5-20251001-v1:0",
"anthropic.claude-instant-v1:2:100k",
"anthropic.claude-opus-4-1-20250805-v1:0",
"anthropic.claude-opus-4-20250514-v1:0",
"anthropic.claude-sonnet-4-20250514-v1:0",
"anthropic.claude-sonnet-4-5-20250929-v1:0",
"anthropic.claude-v2:0:100k",
"anthropic.claude-v2:0:18k",
"anthropic.claude-v2:1:18k",
"anthropic.claude-v2:1:200k",
"cohere.command-r-plus-v1:0",
"cohere.command-r-v1:0",
"cohere.rerank-v3-5:0",
"deepseek.r1-v1:0",
"meta.llama3-1-70b-instruct-v1:0",
"meta.llama3-1-8b-instruct-v1:0",
"meta.llama3-2-11b-instruct-v1:0",
"meta.llama3-2-1b-instruct-v1:0",
"meta.llama3-2-3b-instruct-v1:0",
"meta.llama3-2-90b-instruct-v1:0",
"meta.llama3-3-70b-instruct-v1:0",
"meta.llama3-70b-instruct-v1:0",
"meta.llama3-8b-instruct-v1:0",
"meta.llama4-maverick-17b-instruct-v1:0",
"meta.llama4-scout-17b-instruct-v1:0",
"mistral.mistral-7b-instruct-v0:2",
"mistral.mistral-large-2402-v1:0",
"mistral.mistral-small-2402-v1:0",
"mistral.mixtral-8x7b-instruct-v0:1",
"mistral.pixtral-large-2502-v1:0",
"openai.gpt-oss-120b-1:0",
"openai.gpt-oss-20b-1:0",
"qwen.qwen3-32b-v1:0",
"qwen.qwen3-coder-30b-a3b-v1:0",
"twelvelabs.pegasus-1-2-v1:0",
]

View File

@@ -1,6 +0,0 @@
"""Interceptor contracts for crewai"""
from crewai.llms.hooks.base import BaseInterceptor
__all__ = ["BaseInterceptor"]

View File

@@ -1,133 +0,0 @@
"""Base classes for LLM transport interceptors.
This module provides abstract base classes for intercepting and modifying
outbound and inbound messages at the transport level.
"""
from __future__ import annotations
from abc import ABC, abstractmethod
from typing import TYPE_CHECKING, Any, Generic, TypeVar
from pydantic_core import core_schema
if TYPE_CHECKING:
from pydantic import GetCoreSchemaHandler
from pydantic_core import CoreSchema
T = TypeVar("T")
U = TypeVar("U")
class BaseInterceptor(ABC, Generic[T, U]):
"""Abstract base class for intercepting transport-level messages.
Provides hooks to intercept and modify outbound and inbound messages
at the transport layer.
Type parameters:
T: Outbound message type (e.g., httpx.Request)
U: Inbound message type (e.g., httpx.Response)
Example:
>>> import httpx
>>> class CustomInterceptor(BaseInterceptor[httpx.Request, httpx.Response]):
... def on_outbound(self, message: httpx.Request) -> httpx.Request:
... message.headers["X-Custom-Header"] = "value"
... return message
...
... def on_inbound(self, message: httpx.Response) -> httpx.Response:
... print(f"Status: {message.status_code}")
... return message
"""
@abstractmethod
def on_outbound(self, message: T) -> T:
"""Intercept outbound message before sending.
Args:
message: Outbound message object.
Returns:
Modified message object.
"""
...
@abstractmethod
def on_inbound(self, message: U) -> U:
"""Intercept inbound message after receiving.
Args:
message: Inbound message object.
Returns:
Modified message object.
"""
...
async def aon_outbound(self, message: T) -> T:
"""Async version of on_outbound.
Args:
message: Outbound message object.
Returns:
Modified message object.
"""
raise NotImplementedError
async def aon_inbound(self, message: U) -> U:
"""Async version of on_inbound.
Args:
message: Inbound message object.
Returns:
Modified message object.
"""
raise NotImplementedError
@classmethod
def __get_pydantic_core_schema__(
cls, _source_type: Any, _handler: GetCoreSchemaHandler
) -> CoreSchema:
"""Generate Pydantic core schema for BaseInterceptor.
This allows the generic BaseInterceptor to be used in Pydantic models
without requiring arbitrary_types_allowed=True. The schema validates
that the value is an instance of BaseInterceptor.
Args:
_source_type: The source type being validated (unused).
_handler: Handler for generating schemas (unused).
Returns:
A Pydantic core schema that validates BaseInterceptor instances.
"""
return core_schema.no_info_plain_validator_function(
_validate_interceptor,
serialization=core_schema.plain_serializer_function_ser_schema(
lambda x: x, return_schema=core_schema.any_schema()
),
)
def _validate_interceptor(value: Any) -> BaseInterceptor[T, U]:
"""Validate that the value is a BaseInterceptor instance.
Args:
value: The value to validate.
Returns:
The validated BaseInterceptor instance.
Raises:
ValueError: If the value is not a BaseInterceptor instance.
"""
if not isinstance(value, BaseInterceptor):
raise ValueError(
f"Expected BaseInterceptor instance, got {type(value).__name__}"
)
return value

View File

@@ -1,123 +0,0 @@
"""HTTP transport implementations for LLM request/response interception.
This module provides internal transport classes that integrate with BaseInterceptor
to enable request/response modification at the transport level.
"""
from __future__ import annotations
from collections.abc import Iterable
from typing import TYPE_CHECKING, TypedDict
from httpx import (
AsyncHTTPTransport as _AsyncHTTPTransport,
HTTPTransport as _HTTPTransport,
)
from typing_extensions import NotRequired, Unpack
if TYPE_CHECKING:
from ssl import SSLContext
from httpx import Limits, Request, Response
from httpx._types import CertTypes, ProxyTypes
from crewai.llms.hooks.base import BaseInterceptor
class HTTPTransportKwargs(TypedDict, total=False):
"""Typed dictionary for httpx.HTTPTransport initialization parameters.
These parameters configure the underlying HTTP transport behavior including
SSL verification, proxies, connection limits, and low-level socket options.
"""
verify: bool | str | SSLContext
cert: NotRequired[CertTypes]
trust_env: bool
http1: bool
http2: bool
limits: Limits
proxy: NotRequired[ProxyTypes]
uds: NotRequired[str]
local_address: NotRequired[str]
retries: int
socket_options: NotRequired[
Iterable[
tuple[int, int, int]
| tuple[int, int, bytes | bytearray]
| tuple[int, int, None, int]
]
]
class HTTPTransport(_HTTPTransport):
"""HTTP transport that uses an interceptor for request/response modification.
This transport is used internally when a user provides a BaseInterceptor.
Users should not instantiate this class directly - instead, pass an interceptor
to the LLM client and this transport will be created automatically.
"""
def __init__(
self,
interceptor: BaseInterceptor[Request, Response],
**kwargs: Unpack[HTTPTransportKwargs],
) -> None:
"""Initialize transport with interceptor.
Args:
interceptor: HTTP interceptor for modifying raw request/response objects.
**kwargs: HTTPTransport configuration parameters (verify, cert, proxy, etc.).
"""
super().__init__(**kwargs)
self.interceptor = interceptor
def handle_request(self, request: Request) -> Response:
"""Handle request with interception.
Args:
request: The HTTP request to handle.
Returns:
The HTTP response.
"""
request = self.interceptor.on_outbound(request)
response = super().handle_request(request)
return self.interceptor.on_inbound(response)
class AsyncHTTPTransport(_AsyncHTTPTransport):
"""Async HTTP transport that uses an interceptor for request/response modification.
This transport is used internally when a user provides a BaseInterceptor.
Users should not instantiate this class directly - instead, pass an interceptor
to the LLM client and this transport will be created automatically.
"""
def __init__(
self,
interceptor: BaseInterceptor[Request, Response],
**kwargs: Unpack[HTTPTransportKwargs],
) -> None:
"""Initialize async transport with interceptor.
Args:
interceptor: HTTP interceptor for modifying raw request/response objects.
**kwargs: HTTPTransport configuration parameters (verify, cert, proxy, etc.).
"""
super().__init__(**kwargs)
self.interceptor = interceptor
async def handle_async_request(self, request: Request) -> Response:
"""Handle async request with interception.
Args:
request: The HTTP request to handle.
Returns:
The HTTP response.
"""
request = await self.interceptor.aon_outbound(request)
response = await super().handle_async_request(request)
return await self.interceptor.aon_inbound(response)

View File

@@ -1,15 +1,9 @@
from __future__ import annotations
import json
import logging
import os
from typing import TYPE_CHECKING, Any, cast
from pydantic import BaseModel
from typing import Any, cast
from crewai.events.types.llm_events import LLMCallType
from crewai.llms.base_llm import BaseLLM
from crewai.llms.hooks.transport import HTTPTransport
from crewai.utilities.agent_utils import is_context_length_exceeded
from crewai.utilities.exceptions.context_window_exceeding_exception import (
LLMContextLengthExceededError,
@@ -17,14 +11,10 @@ from crewai.utilities.exceptions.context_window_exceeding_exception import (
from crewai.utilities.types import LLMMessage
if TYPE_CHECKING:
from crewai.llms.hooks.base import BaseInterceptor
try:
from anthropic import Anthropic
from anthropic.types import Message
from anthropic.types.tool_use_block import ToolUseBlock
import httpx
except ImportError:
raise ImportError(
'Anthropic native provider not available, to install: uv add "crewai[anthropic]"'
@@ -46,13 +36,12 @@ class AnthropicCompletion(BaseLLM):
timeout: float | None = None,
max_retries: int = 2,
temperature: float | None = None,
max_tokens: int = 4096, # Required for Anthropic
max_tokens: int | None = None, # Optional, computed dynamically if not set
top_p: float | None = None,
stop_sequences: list[str] | None = None,
stream: bool = False,
client_params: dict[str, Any] | None = None,
interceptor: BaseInterceptor[httpx.Request, httpx.Response] | None = None,
**kwargs: Any,
**kwargs,
):
"""Initialize Anthropic chat completion client.
@@ -63,12 +52,12 @@ class AnthropicCompletion(BaseLLM):
timeout: Request timeout in seconds
max_retries: Maximum number of retries
temperature: Sampling temperature (0-1)
max_tokens: Maximum tokens in response (required for Anthropic)
max_tokens: Maximum tokens in response. If not set, will be computed
dynamically based on context window size (recommended for most use cases)
top_p: Nucleus sampling parameter
stop_sequences: Stop sequences (Anthropic uses stop_sequences, not stop)
stream: Enable streaming responses
client_params: Additional parameters for the Anthropic client
interceptor: HTTP interceptor for modifying requests/responses at transport level.
**kwargs: Additional parameters
"""
super().__init__(
@@ -76,7 +65,6 @@ class AnthropicCompletion(BaseLLM):
)
# Client params
self.interceptor = interceptor
self.client_params = client_params
self.base_url = base_url
self.timeout = timeout
@@ -85,7 +73,7 @@ class AnthropicCompletion(BaseLLM):
self.client = Anthropic(**self._get_client_params())
# Store completion parameters
self.max_tokens = max_tokens
self.max_tokens = max_tokens # Can be None, will be computed dynamically
self.top_p = top_p
self.stream = stream
self.stop_sequences = stop_sequences or []
@@ -94,30 +82,6 @@ class AnthropicCompletion(BaseLLM):
self.is_claude_3 = "claude-3" in model.lower()
self.supports_tools = self.is_claude_3 # Claude 3+ supports tool use
@property
def stop(self) -> list[str]:
"""Get stop sequences sent to the API."""
return self.stop_sequences
@stop.setter
def stop(self, value: list[str] | str | None) -> None:
"""Set stop sequences.
Synchronizes stop_sequences to ensure values set by CrewAgentExecutor
are properly sent to the Anthropic API.
Args:
value: Stop sequences as a list, single string, or None
"""
if value is None:
self.stop_sequences = []
elif isinstance(value, str):
self.stop_sequences = [value]
elif isinstance(value, list):
self.stop_sequences = value
else:
self.stop_sequences = []
def _get_client_params(self) -> dict[str, Any]:
"""Get client parameters."""
@@ -133,11 +97,6 @@ class AnthropicCompletion(BaseLLM):
"max_retries": self.max_retries,
}
if self.interceptor:
transport = HTTPTransport(interceptor=self.interceptor)
http_client = httpx.Client(transport=transport)
client_params["http_client"] = http_client # type: ignore[assignment]
if self.client_params:
client_params.update(self.client_params)
@@ -146,12 +105,11 @@ class AnthropicCompletion(BaseLLM):
def call(
self,
messages: str | list[LLMMessage],
tools: list[dict[str, Any]] | None = None,
tools: list[dict] | None = None,
callbacks: list[Any] | None = None,
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
response_model: type[BaseModel] | None = None,
) -> str | Any:
"""Call Anthropic messages API.
@@ -169,7 +127,7 @@ class AnthropicCompletion(BaseLLM):
try:
# Emit call started event
self._emit_call_started_event(
messages=messages,
messages=messages, # type: ignore[arg-type]
tools=tools,
callbacks=callbacks,
available_functions=available_functions,
@@ -179,7 +137,7 @@ class AnthropicCompletion(BaseLLM):
# Format messages for Anthropic
formatted_messages, system_message = self._format_messages_for_anthropic(
messages
messages # type: ignore[arg-type]
)
# Prepare completion parameters
@@ -190,19 +148,11 @@ class AnthropicCompletion(BaseLLM):
# Handle streaming vs non-streaming
if self.stream:
return self._handle_streaming_completion(
completion_params,
available_functions,
from_task,
from_agent,
response_model,
completion_params, available_functions, from_task, from_agent
)
return self._handle_completion(
completion_params,
available_functions,
from_task,
from_agent,
response_model,
completion_params, available_functions, from_task, from_agent
)
except Exception as e:
@@ -217,7 +167,7 @@ class AnthropicCompletion(BaseLLM):
self,
messages: list[LLMMessage],
system_message: str | None = None,
tools: list[dict[str, Any]] | None = None,
tools: list[dict] | None = None,
) -> dict[str, Any]:
"""Prepare parameters for Anthropic messages API.
@@ -229,10 +179,19 @@ class AnthropicCompletion(BaseLLM):
Returns:
Parameters dictionary for Anthropic API
"""
max_tokens = self.max_tokens
if max_tokens is None:
# while still allowing enough tokens for most responses
max_tokens = 1024
context_window = self.get_context_window_size()
if context_window > 100000: # For Claude models with 200k+ context
max_tokens = 2048
params = {
"model": self.model,
"messages": messages,
"max_tokens": self.max_tokens,
"max_tokens": max_tokens,
"stream": self.stream,
}
@@ -254,9 +213,7 @@ class AnthropicCompletion(BaseLLM):
return params
def _convert_tools_for_interference(
self, tools: list[dict[str, Any]]
) -> list[dict[str, Any]]:
def _convert_tools_for_interference(self, tools: list[dict]) -> list[dict]:
"""Convert CrewAI tool format to Anthropic tool use format."""
anthropic_tools = []
@@ -343,19 +300,8 @@ class AnthropicCompletion(BaseLLM):
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
response_model: type[BaseModel] | None = None,
) -> str | Any:
"""Handle non-streaming message completion."""
if response_model:
structured_tool = {
"name": "structured_output",
"description": "Returns structured data according to the schema",
"input_schema": response_model.model_json_schema(),
}
params["tools"] = [structured_tool]
params["tool_choice"] = {"type": "tool", "name": "structured_output"}
try:
response: Message = self.client.messages.create(**params)
@@ -368,24 +314,6 @@ class AnthropicCompletion(BaseLLM):
usage = self._extract_anthropic_token_usage(response)
self._track_token_usage_internal(usage)
if response_model and response.content:
tool_uses = [
block for block in response.content if isinstance(block, ToolUseBlock)
]
if tool_uses and tool_uses[0].name == "structured_output":
structured_data = tool_uses[0].input
structured_json = json.dumps(structured_data)
self._emit_call_completed_event(
response=structured_json,
call_type=LLMCallType.LLM_CALL,
from_task=from_task,
from_agent=from_agent,
messages=params["messages"],
)
return structured_json
# Check if Claude wants to use tools
if response.content and available_functions:
tool_uses = [
@@ -431,19 +359,8 @@ class AnthropicCompletion(BaseLLM):
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
response_model: type[BaseModel] | None = None,
) -> str:
"""Handle streaming message completion."""
if response_model:
structured_tool = {
"name": "structured_output",
"description": "Returns structured data according to the schema",
"input_schema": response_model.model_json_schema(),
}
params["tools"] = [structured_tool]
params["tool_choice"] = {"type": "tool", "name": "structured_output"}
full_response = ""
# Remove 'stream' parameter as messages.stream() doesn't accept it
@@ -467,26 +384,6 @@ class AnthropicCompletion(BaseLLM):
usage = self._extract_anthropic_token_usage(final_message)
self._track_token_usage_internal(usage)
if response_model and final_message.content:
tool_uses = [
block
for block in final_message.content
if isinstance(block, ToolUseBlock)
]
if tool_uses and tool_uses[0].name == "structured_output":
structured_data = tool_uses[0].input
structured_json = json.dumps(structured_data)
self._emit_call_completed_event(
response=structured_json,
call_type=LLMCallType.LLM_CALL,
from_task=from_task,
from_agent=from_agent,
messages=params["messages"],
)
return structured_json
if final_message.content and available_functions:
tool_uses = [
block

View File

@@ -1,11 +1,7 @@
from __future__ import annotations
import json
import logging
import os
from typing import TYPE_CHECKING, Any
from pydantic import BaseModel
from typing import Any
from crewai.utilities.agent_utils import is_context_length_exceeded
from crewai.utilities.exceptions.context_window_exceeding_exception import (
@@ -14,24 +10,19 @@ from crewai.utilities.exceptions.context_window_exceeding_exception import (
from crewai.utilities.types import LLMMessage
if TYPE_CHECKING:
from crewai.llms.hooks.base import BaseInterceptor
from crewai.tools.base_tool import BaseTool
try:
from azure.ai.inference import (
from azure.ai.inference import ( # type: ignore[import-not-found]
ChatCompletionsClient,
)
from azure.ai.inference.models import (
from azure.ai.inference.models import ( # type: ignore[import-not-found]
ChatCompletions,
ChatCompletionsToolCall,
StreamingChatCompletionsUpdate,
)
from azure.core.credentials import (
from azure.core.credentials import ( # type: ignore[import-not-found]
AzureKeyCredential,
)
from azure.core.exceptions import (
from azure.core.exceptions import ( # type: ignore[import-not-found]
HttpResponseError,
)
@@ -66,8 +57,7 @@ class AzureCompletion(BaseLLM):
max_tokens: int | None = None,
stop: list[str] | None = None,
stream: bool = False,
interceptor: BaseInterceptor[Any, Any] | None = None,
**kwargs: Any,
**kwargs,
):
"""Initialize Azure AI Inference chat completion client.
@@ -85,15 +75,8 @@ class AzureCompletion(BaseLLM):
max_tokens: Maximum tokens in response
stop: Stop sequences
stream: Enable streaming responses
interceptor: HTTP interceptor (not yet supported for Azure).
**kwargs: Additional parameters
"""
if interceptor is not None:
raise NotImplementedError(
"HTTP interceptors are not yet supported for Azure AI Inference provider. "
"Interceptors are currently supported for OpenAI and Anthropic providers only."
)
super().__init__(
model=model, temperature=temperature, stop=stop or [], **kwargs
)
@@ -131,7 +114,7 @@ class AzureCompletion(BaseLLM):
if self.api_version:
client_kwargs["api_version"] = self.api_version
self.client = ChatCompletionsClient(**client_kwargs) # type: ignore[arg-type]
self.client = ChatCompletionsClient(**client_kwargs)
self.top_p = top_p
self.frequency_penalty = frequency_penalty
@@ -174,12 +157,11 @@ class AzureCompletion(BaseLLM):
def call(
self,
messages: str | list[LLMMessage],
tools: list[dict[str, BaseTool]] | None = None,
tools: list[dict] | None = None,
callbacks: list[Any] | None = None,
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
response_model: type[BaseModel] | None = None,
) -> str | Any:
"""Call Azure AI Inference chat completions API.
@@ -210,25 +192,17 @@ class AzureCompletion(BaseLLM):
# Prepare completion parameters
completion_params = self._prepare_completion_params(
formatted_messages, tools, response_model
formatted_messages, tools
)
# Handle streaming vs non-streaming
if self.stream:
return self._handle_streaming_completion(
completion_params,
available_functions,
from_task,
from_agent,
response_model,
completion_params, available_functions, from_task, from_agent
)
return self._handle_completion(
completion_params,
available_functions,
from_task,
from_agent,
response_model,
completion_params, available_functions, from_task, from_agent
)
except HttpResponseError as e:
@@ -259,15 +233,13 @@ class AzureCompletion(BaseLLM):
def _prepare_completion_params(
self,
messages: list[LLMMessage],
tools: list[dict[str, Any]] | None = None,
response_model: type[BaseModel] | None = None,
tools: list[dict] | None = None,
) -> dict[str, Any]:
"""Prepare parameters for Azure AI Inference chat completion.
Args:
messages: Formatted messages for Azure
tools: Tool definitions
response_model: Pydantic model for structured output
Returns:
Parameters dictionary for Azure API
@@ -277,15 +249,6 @@ class AzureCompletion(BaseLLM):
"stream": self.stream,
}
if response_model and self.is_openai_model:
params["response_format"] = {
"type": "json_schema",
"json_schema": {
"name": response_model.__name__,
"schema": response_model.model_json_schema(),
},
}
# Only include model parameter for non-Azure OpenAI endpoints
# Azure OpenAI endpoints have the deployment name in the URL
if not self.is_azure_openai_endpoint:
@@ -312,9 +275,7 @@ class AzureCompletion(BaseLLM):
return params
def _convert_tools_for_interference(
self, tools: list[dict[str, Any]]
) -> list[dict[str, Any]]:
def _convert_tools_for_interference(self, tools: list[dict]) -> list[dict]:
"""Convert CrewAI tool format to Azure OpenAI function calling format."""
from crewai.llms.providers.utils.common import safe_tool_conversion
@@ -373,7 +334,6 @@ class AzureCompletion(BaseLLM):
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
response_model: type[BaseModel] | None = None,
) -> str | Any:
"""Handle non-streaming chat completion."""
# Make API call
@@ -390,26 +350,6 @@ class AzureCompletion(BaseLLM):
usage = self._extract_azure_token_usage(response)
self._track_token_usage_internal(usage)
if response_model and self.is_openai_model:
content = message.content or ""
try:
structured_data = response_model.model_validate_json(content)
structured_json = structured_data.model_dump_json()
self._emit_call_completed_event(
response=structured_json,
call_type=LLMCallType.LLM_CALL,
from_task=from_task,
from_agent=from_agent,
messages=params["messages"],
)
return structured_json
except Exception as e:
error_msg = f"Failed to validate structured output with model {response_model.__name__}: {e}"
logging.error(error_msg)
raise ValueError(error_msg) from e
# Handle tool calls
if message.tool_calls and available_functions:
tool_call = message.tool_calls[0] # Handle first tool call
@@ -469,7 +409,6 @@ class AzureCompletion(BaseLLM):
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
response_model: type[BaseModel] | None = None,
) -> str:
"""Handle streaming chat completion."""
full_response = ""

View File

@@ -5,7 +5,6 @@ import logging
import os
from typing import TYPE_CHECKING, Any, TypedDict, cast
from pydantic import BaseModel
from typing_extensions import Required
from crewai.events.types.llm_events import LLMCallType
@@ -30,8 +29,6 @@ if TYPE_CHECKING:
ToolTypeDef,
)
from crewai.llms.hooks.base import BaseInterceptor
try:
from boto3.session import Session
@@ -159,9 +156,8 @@ class BedrockCompletion(BaseLLM):
guardrail_config: dict[str, Any] | None = None,
additional_model_request_fields: dict[str, Any] | None = None,
additional_model_response_field_paths: list[str] | None = None,
interceptor: BaseInterceptor[Any, Any] | None = None,
**kwargs: Any,
) -> None:
**kwargs,
):
"""Initialize AWS Bedrock completion client.
Args:
@@ -179,15 +175,8 @@ class BedrockCompletion(BaseLLM):
guardrail_config: Guardrail configuration for content filtering
additional_model_request_fields: Model-specific request parameters
additional_model_response_field_paths: Custom response field paths
interceptor: HTTP interceptor (not yet supported for Bedrock).
**kwargs: Additional parameters
"""
if interceptor is not None:
raise NotImplementedError(
"HTTP interceptors are not yet supported for AWS Bedrock provider. "
"Interceptors are currently supported for OpenAI and Anthropic providers only."
)
# Extract provider from kwargs to avoid duplicate argument
kwargs.pop("provider", None)
@@ -243,30 +232,6 @@ class BedrockCompletion(BaseLLM):
# Handle inference profiles for newer models
self.model_id = model
@property
def stop(self) -> list[str]:
"""Get stop sequences sent to the API."""
return list(self.stop_sequences)
@stop.setter
def stop(self, value: Sequence[str] | str | None) -> None:
"""Set stop sequences.
Synchronizes stop_sequences to ensure values set by CrewAgentExecutor
are properly sent to the Bedrock API.
Args:
value: Stop sequences as a Sequence, single string, or None
"""
if value is None:
self.stop_sequences = []
elif isinstance(value, str):
self.stop_sequences = [value]
elif isinstance(value, Sequence):
self.stop_sequences = list(value)
else:
self.stop_sequences = []
def call(
self,
messages: str | list[LLMMessage],
@@ -275,13 +240,12 @@ class BedrockCompletion(BaseLLM):
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
response_model: type[BaseModel] | None = None,
) -> str | Any:
"""Call AWS Bedrock Converse API."""
try:
# Emit call started event
self._emit_call_started_event(
messages=messages,
messages=messages, # type: ignore[arg-type]
tools=tools,
callbacks=callbacks,
available_functions=available_functions,
@@ -774,9 +738,7 @@ class BedrockCompletion(BaseLLM):
return converse_messages, system_message
@staticmethod
def _format_tools_for_converse(
tools: list[dict[str, Any]],
) -> list[ConverseToolTypeDef]:
def _format_tools_for_converse(tools: list[dict]) -> list[ConverseToolTypeDef]:
"""Convert CrewAI tools to Converse API format following AWS specification."""
from crewai.llms.providers.utils.common import safe_tool_conversion

View File

@@ -2,11 +2,8 @@ import logging
import os
from typing import Any, cast
from pydantic import BaseModel
from crewai.events.types.llm_events import LLMCallType
from crewai.llms.base_llm import BaseLLM
from crewai.llms.hooks.base import BaseInterceptor
from crewai.utilities.agent_utils import is_context_length_exceeded
from crewai.utilities.exceptions.context_window_exceeding_exception import (
LLMContextLengthExceededError,
@@ -45,8 +42,7 @@ class GeminiCompletion(BaseLLM):
stream: bool = False,
safety_settings: dict[str, Any] | None = None,
client_params: dict[str, Any] | None = None,
interceptor: BaseInterceptor[Any, Any] | None = None,
**kwargs: Any,
**kwargs,
):
"""Initialize Google Gemini chat completion client.
@@ -64,15 +60,8 @@ class GeminiCompletion(BaseLLM):
safety_settings: Safety filter settings
client_params: Additional parameters to pass to the Google Gen AI Client constructor.
Supports parameters like http_options, credentials, debug_config, etc.
interceptor: HTTP interceptor (not yet supported for Gemini).
**kwargs: Additional parameters
"""
if interceptor is not None:
raise NotImplementedError(
"HTTP interceptors are not yet supported for Google Gemini provider. "
"Interceptors are currently supported for OpenAI and Anthropic providers only."
)
super().__init__(
model=model, temperature=temperature, stop=stop_sequences or [], **kwargs
)
@@ -104,31 +93,7 @@ class GeminiCompletion(BaseLLM):
self.is_gemini_1_5 = "gemini-1.5" in model.lower()
self.supports_tools = self.is_gemini_1_5 or self.is_gemini_2
@property
def stop(self) -> list[str]:
"""Get stop sequences sent to the API."""
return self.stop_sequences
@stop.setter
def stop(self, value: list[str] | str | None) -> None:
"""Set stop sequences.
Synchronizes stop_sequences to ensure values set by CrewAgentExecutor
are properly sent to the Gemini API.
Args:
value: Stop sequences as a list, single string, or None
"""
if value is None:
self.stop_sequences = []
elif isinstance(value, str):
self.stop_sequences = [value]
elif isinstance(value, list):
self.stop_sequences = value
else:
self.stop_sequences = []
def _initialize_client(self, use_vertexai: bool = False) -> genai.Client: # type: ignore[no-any-unimported]
def _initialize_client(self, use_vertexai: bool = False) -> genai.Client:
"""Initialize the Google Gen AI client with proper parameter handling.
Args:
@@ -203,12 +168,11 @@ class GeminiCompletion(BaseLLM):
def call(
self,
messages: str | list[LLMMessage],
tools: list[dict[str, Any]] | None = None,
tools: list[dict] | None = None,
callbacks: list[Any] | None = None,
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
response_model: type[BaseModel] | None = None,
) -> str | Any:
"""Call Google Gemini generate content API.
@@ -225,7 +189,7 @@ class GeminiCompletion(BaseLLM):
"""
try:
self._emit_call_started_event(
messages=messages,
messages=messages, # type: ignore[arg-type]
tools=tools,
callbacks=callbacks,
available_functions=available_functions,
@@ -235,12 +199,10 @@ class GeminiCompletion(BaseLLM):
self.tools = tools
formatted_content, system_instruction = self._format_messages_for_gemini(
messages
messages # type: ignore[arg-type]
)
config = self._prepare_generation_config(
system_instruction, tools, response_model
)
config = self._prepare_generation_config(system_instruction, tools)
if self.stream:
return self._handle_streaming_completion(
@@ -249,7 +211,6 @@ class GeminiCompletion(BaseLLM):
available_functions,
from_task,
from_agent,
response_model,
)
return self._handle_completion(
@@ -259,7 +220,6 @@ class GeminiCompletion(BaseLLM):
available_functions,
from_task,
from_agent,
response_model,
)
except APIError as e:
@@ -277,18 +237,16 @@ class GeminiCompletion(BaseLLM):
)
raise
def _prepare_generation_config( # type: ignore[no-any-unimported]
def _prepare_generation_config(
self,
system_instruction: str | None = None,
tools: list[dict[str, Any]] | None = None,
response_model: type[BaseModel] | None = None,
tools: list[dict] | None = None,
) -> types.GenerateContentConfig:
"""Prepare generation config for Google Gemini API.
Args:
system_instruction: System instruction for the model
tools: Tool definitions
response_model: Pydantic model for structured output
Returns:
GenerateContentConfig object for Gemini API
@@ -316,10 +274,6 @@ class GeminiCompletion(BaseLLM):
if self.stop_sequences:
config_params["stop_sequences"] = self.stop_sequences
if response_model:
config_params["response_mime_type"] = "application/json"
config_params["response_schema"] = response_model.model_json_schema()
# Handle tools for supported models
if tools and self.supports_tools:
config_params["tools"] = self._convert_tools_for_interference(tools)
@@ -329,9 +283,7 @@ class GeminiCompletion(BaseLLM):
return types.GenerateContentConfig(**config_params)
def _convert_tools_for_interference( # type: ignore[no-any-unimported]
self, tools: list[dict[str, Any]]
) -> list[types.Tool]:
def _convert_tools_for_interference(self, tools: list[dict]) -> list[types.Tool]:
"""Convert CrewAI tool format to Gemini function declaration format."""
gemini_tools = []
@@ -354,7 +306,7 @@ class GeminiCompletion(BaseLLM):
return gemini_tools
def _format_messages_for_gemini( # type: ignore[no-any-unimported]
def _format_messages_for_gemini(
self, messages: str | list[LLMMessage]
) -> tuple[list[types.Content], str | None]:
"""Format messages for Gemini API.
@@ -398,7 +350,7 @@ class GeminiCompletion(BaseLLM):
return contents, system_instruction
def _handle_completion( # type: ignore[no-any-unimported]
def _handle_completion(
self,
contents: list[types.Content],
system_instruction: str | None,
@@ -406,7 +358,6 @@ class GeminiCompletion(BaseLLM):
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
response_model: type[BaseModel] | None = None,
) -> str | Any:
"""Handle non-streaming content generation."""
api_params = {
@@ -465,14 +416,13 @@ class GeminiCompletion(BaseLLM):
return content
def _handle_streaming_completion( # type: ignore[no-any-unimported]
def _handle_streaming_completion(
self,
contents: list[types.Content],
config: types.GenerateContentConfig,
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
response_model: type[BaseModel] | None = None,
) -> str:
"""Handle streaming content generation."""
full_response = ""
@@ -594,9 +544,8 @@ class GeminiCompletion(BaseLLM):
}
return {"total_tokens": 0}
def _convert_contents_to_dict( # type: ignore[no-any-unimported]
self,
contents: list[types.Content],
def _convert_contents_to_dict(
self, contents: list[types.Content]
) -> list[dict[str, str]]:
"""Convert contents to dict format."""
return [

View File

@@ -1,12 +1,9 @@
from __future__ import annotations
from collections.abc import Iterator
import json
import logging
import os
from typing import TYPE_CHECKING, Any
from typing import Any
import httpx
from openai import APIConnectionError, NotFoundError, OpenAI
from openai.types.chat import ChatCompletion, ChatCompletionChunk
from openai.types.chat.chat_completion import Choice
@@ -15,7 +12,6 @@ from pydantic import BaseModel
from crewai.events.types.llm_events import LLMCallType
from crewai.llms.base_llm import BaseLLM
from crewai.llms.hooks.transport import HTTPTransport
from crewai.utilities.agent_utils import is_context_length_exceeded
from crewai.utilities.exceptions.context_window_exceeding_exception import (
LLMContextLengthExceededError,
@@ -23,13 +19,6 @@ from crewai.utilities.exceptions.context_window_exceeding_exception import (
from crewai.utilities.types import LLMMessage
if TYPE_CHECKING:
from crewai.agent.core import Agent
from crewai.llms.hooks.base import BaseInterceptor
from crewai.task import Task
from crewai.tools.base_tool import BaseTool
class OpenAICompletion(BaseLLM):
"""OpenAI native completion implementation.
@@ -62,15 +51,13 @@ class OpenAICompletion(BaseLLM):
top_logprobs: int | None = None,
reasoning_effort: str | None = None,
provider: str | None = None,
interceptor: BaseInterceptor[httpx.Request, httpx.Response] | None = None,
**kwargs: Any,
) -> None:
**kwargs,
):
"""Initialize OpenAI chat completion client."""
if provider is None:
provider = kwargs.pop("provider", "openai")
self.interceptor = interceptor
# Client configuration attributes
self.organization = organization
self.project = project
@@ -93,11 +80,6 @@ class OpenAICompletion(BaseLLM):
)
client_config = self._get_client_params()
if self.interceptor:
transport = HTTPTransport(interceptor=self.interceptor)
http_client = httpx.Client(transport=transport)
client_config["http_client"] = http_client
self.client = OpenAI(**client_config)
# Completion parameters
@@ -147,12 +129,11 @@ class OpenAICompletion(BaseLLM):
def call(
self,
messages: str | list[LLMMessage],
tools: list[dict[str, BaseTool]] | None = None,
tools: list[dict] | None = None,
callbacks: list[Any] | None = None,
available_functions: dict[str, Any] | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
response_model: type[BaseModel] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
) -> str | Any:
"""Call OpenAI chat completion API.
@@ -163,14 +144,13 @@ class OpenAICompletion(BaseLLM):
available_functions: Available functions for tool calling
from_task: Task that initiated the call
from_agent: Agent that initiated the call
response_model: Response model for structured output.
Returns:
Chat completion response or tool call result
"""
try:
self._emit_call_started_event(
messages=messages,
messages=messages, # type: ignore[arg-type]
tools=tools,
callbacks=callbacks,
available_functions=available_functions,
@@ -178,27 +158,19 @@ class OpenAICompletion(BaseLLM):
from_agent=from_agent,
)
formatted_messages = self._format_messages(messages)
formatted_messages = self._format_messages(messages) # type: ignore[arg-type]
completion_params = self._prepare_completion_params(
messages=formatted_messages, tools=tools
formatted_messages, tools
)
if self.stream:
return self._handle_streaming_completion(
params=completion_params,
available_functions=available_functions,
from_task=from_task,
from_agent=from_agent,
response_model=response_model,
completion_params, available_functions, from_task, from_agent
)
return self._handle_completion(
params=completion_params,
available_functions=available_functions,
from_task=from_task,
from_agent=from_agent,
response_model=response_model,
completion_params, available_functions, from_task, from_agent
)
except Exception as e:
@@ -210,15 +182,14 @@ class OpenAICompletion(BaseLLM):
raise
def _prepare_completion_params(
self, messages: list[LLMMessage], tools: list[dict[str, BaseTool]] | None = None
self, messages: list[LLMMessage], tools: list[dict] | None = None
) -> dict[str, Any]:
"""Prepare parameters for OpenAI chat completion."""
params: dict[str, Any] = {
params = {
"model": self.model,
"messages": messages,
"stream": self.stream,
}
if self.stream:
params["stream"] = self.stream
params.update(self.additional_params)
@@ -245,6 +216,22 @@ class OpenAICompletion(BaseLLM):
if self.is_o1_model and self.reasoning_effort:
params["reasoning_effort"] = self.reasoning_effort
# Handle response format for structured outputs
if self.response_format:
if isinstance(self.response_format, type) and issubclass(
self.response_format, BaseModel
):
# Convert Pydantic model to OpenAI response format
params["response_format"] = {
"type": "json_schema",
"json_schema": {
"name": self.response_format.__name__,
"schema": self.response_format.model_json_schema(),
},
}
else:
params["response_format"] = self.response_format
if tools:
params["tools"] = self._convert_tools_for_interference(tools)
params["tool_choice"] = "auto"
@@ -264,9 +251,7 @@ class OpenAICompletion(BaseLLM):
return {k: v for k, v in params.items() if k not in crewai_specific_params}
def _convert_tools_for_interference(
self, tools: list[dict[str, BaseTool]]
) -> list[dict[str, Any]]:
def _convert_tools_for_interference(self, tools: list[dict]) -> list[dict]:
"""Convert CrewAI tool format to OpenAI function calling format."""
from crewai.llms.providers.utils.common import safe_tool_conversion
@@ -298,35 +283,9 @@ class OpenAICompletion(BaseLLM):
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
response_model: type[BaseModel] | None = None,
) -> str | Any:
"""Handle non-streaming chat completion."""
try:
if response_model:
parsed_response = self.client.beta.chat.completions.parse(
**params,
response_format=response_model,
)
math_reasoning = parsed_response.choices[0].message
if math_reasoning.refusal:
pass
usage = self._extract_openai_token_usage(parsed_response)
self._track_token_usage_internal(usage)
parsed_object = parsed_response.choices[0].message.parsed
if parsed_object:
structured_json = parsed_object.model_dump_json()
self._emit_call_completed_event(
response=structured_json,
call_type=LLMCallType.LLM_CALL,
from_task=from_task,
from_agent=from_agent,
messages=params["messages"],
)
return structured_json
response: ChatCompletion = self.client.chat.completions.create(**params)
usage = self._extract_openai_token_usage(response)
@@ -421,57 +380,12 @@ class OpenAICompletion(BaseLLM):
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
response_model: type[BaseModel] | None = None,
) -> str:
"""Handle streaming chat completion."""
full_response = ""
tool_calls = {}
if response_model:
completion_stream: Iterator[ChatCompletionChunk] = (
self.client.chat.completions.create(**params)
)
accumulated_content = ""
for chunk in completion_stream:
if not chunk.choices:
continue
choice = chunk.choices[0]
delta: ChoiceDelta = choice.delta
if delta.content:
accumulated_content += delta.content
self._emit_stream_chunk_event(
chunk=delta.content,
from_task=from_task,
from_agent=from_agent,
)
try:
parsed_object = response_model.model_validate_json(accumulated_content)
structured_json = parsed_object.model_dump_json()
self._emit_call_completed_event(
response=structured_json,
call_type=LLMCallType.LLM_CALL,
from_task=from_task,
from_agent=from_agent,
messages=params["messages"],
)
return structured_json
except Exception as e:
logging.error(f"Failed to parse structured output from stream: {e}")
self._emit_call_completed_event(
response=accumulated_content,
call_type=LLMCallType.LLM_CALL,
from_task=from_task,
from_agent=from_agent,
messages=params["messages"],
)
return accumulated_content
# Make streaming API call
stream: Iterator[ChatCompletionChunk] = self.client.chat.completions.create(
**params
)
@@ -481,18 +395,20 @@ class OpenAICompletion(BaseLLM):
continue
choice = chunk.choices[0]
chunk_delta: ChoiceDelta = choice.delta
delta: ChoiceDelta = choice.delta
if chunk_delta.content:
full_response += chunk_delta.content
# Handle content streaming
if delta.content:
full_response += delta.content
self._emit_stream_chunk_event(
chunk=chunk_delta.content,
chunk=delta.content,
from_task=from_task,
from_agent=from_agent,
)
if chunk_delta.tool_calls:
for tool_call in chunk_delta.tool_calls:
# Handle tool call streaming
if delta.tool_calls:
for tool_call in delta.tool_calls:
call_id = tool_call.id or "default"
if call_id not in tool_calls:
tool_calls[call_id] = {
@@ -538,8 +454,10 @@ class OpenAICompletion(BaseLLM):
if result is not None:
return result
# Apply stop words to full response
full_response = self._apply_stop_words(full_response)
# Emit completion event and return full response
self._emit_call_completed_event(
response=full_response,
call_type=LLMCallType.LLM_CALL,
@@ -605,9 +523,12 @@ class OpenAICompletion(BaseLLM):
}
return {"total_tokens": 0}
def _format_messages(self, messages: str | list[LLMMessage]) -> list[LLMMessage]:
def _format_messages( # type: ignore[override]
self, messages: str | list[LLMMessage]
) -> list[LLMMessage]:
"""Format messages for OpenAI API."""
base_formatted = super()._format_messages(messages)
# Use base class formatting first
base_formatted = super()._format_messages(messages) # type: ignore[arg-type]
# Apply OpenAI-specific formatting
formatted_messages: list[LLMMessage] = []

View File

@@ -1,37 +0,0 @@
"""MCP (Model Context Protocol) client support for CrewAI agents.
This module provides native MCP client functionality, allowing CrewAI agents
to connect to any MCP-compliant server using various transport types.
"""
from crewai.mcp.client import MCPClient
from crewai.mcp.config import (
MCPServerConfig,
MCPServerHTTP,
MCPServerSSE,
MCPServerStdio,
)
from crewai.mcp.filters import (
StaticToolFilter,
ToolFilter,
ToolFilterContext,
create_dynamic_tool_filter,
create_static_tool_filter,
)
from crewai.mcp.transports.base import BaseTransport, TransportType
__all__ = [
"BaseTransport",
"MCPClient",
"MCPServerConfig",
"MCPServerHTTP",
"MCPServerSSE",
"MCPServerStdio",
"StaticToolFilter",
"ToolFilter",
"ToolFilterContext",
"TransportType",
"create_dynamic_tool_filter",
"create_static_tool_filter",
]

View File

@@ -1,742 +0,0 @@
"""MCP client with session management for CrewAI agents."""
import asyncio
from collections.abc import Callable
from contextlib import AsyncExitStack
from datetime import datetime
import logging
import time
from typing import Any
from typing_extensions import Self
# BaseExceptionGroup is available in Python 3.11+
try:
from builtins import BaseExceptionGroup
except ImportError:
# Fallback for Python < 3.11 (shouldn't happen in practice)
BaseExceptionGroup = Exception
from crewai.events.event_bus import crewai_event_bus
from crewai.events.types.mcp_events import (
MCPConnectionCompletedEvent,
MCPConnectionFailedEvent,
MCPConnectionStartedEvent,
MCPToolExecutionCompletedEvent,
MCPToolExecutionFailedEvent,
MCPToolExecutionStartedEvent,
)
from crewai.mcp.transports.base import BaseTransport
from crewai.mcp.transports.http import HTTPTransport
from crewai.mcp.transports.sse import SSETransport
from crewai.mcp.transports.stdio import StdioTransport
# MCP Connection timeout constants (in seconds)
MCP_CONNECTION_TIMEOUT = 30 # Increased for slow servers
MCP_TOOL_EXECUTION_TIMEOUT = 30
MCP_DISCOVERY_TIMEOUT = 30 # Increased for slow servers
MCP_MAX_RETRIES = 3
# Simple in-memory cache for MCP tool schemas (duration: 5 minutes)
_mcp_schema_cache: dict[str, tuple[dict[str, Any], float]] = {}
_cache_ttl = 300 # 5 minutes
class MCPClient:
"""MCP client with session management.
This client manages connections to MCP servers and provides a high-level
interface for interacting with MCP tools, prompts, and resources.
Example:
```python
transport = StdioTransport(command="python", args=["server.py"])
client = MCPClient(transport)
async with client:
tools = await client.list_tools()
result = await client.call_tool("tool_name", {"arg": "value"})
```
"""
def __init__(
self,
transport: BaseTransport,
connect_timeout: int = MCP_CONNECTION_TIMEOUT,
execution_timeout: int = MCP_TOOL_EXECUTION_TIMEOUT,
discovery_timeout: int = MCP_DISCOVERY_TIMEOUT,
max_retries: int = MCP_MAX_RETRIES,
cache_tools_list: bool = False,
logger: logging.Logger | None = None,
) -> None:
"""Initialize MCP client.
Args:
transport: Transport instance for MCP server connection.
connect_timeout: Connection timeout in seconds.
execution_timeout: Tool execution timeout in seconds.
discovery_timeout: Tool discovery timeout in seconds.
max_retries: Maximum retry attempts for operations.
cache_tools_list: Whether to cache tool list results.
logger: Optional logger instance.
"""
self.transport = transport
self.connect_timeout = connect_timeout
self.execution_timeout = execution_timeout
self.discovery_timeout = discovery_timeout
self.max_retries = max_retries
self.cache_tools_list = cache_tools_list
# self._logger = logger or logging.getLogger(__name__)
self._session: Any = None
self._initialized = False
self._exit_stack = AsyncExitStack()
self._was_connected = False
@property
def connected(self) -> bool:
"""Check if client is connected to server."""
return self.transport.connected and self._initialized
@property
def session(self) -> Any:
"""Get the MCP session."""
if self._session is None:
raise RuntimeError("Client not connected. Call connect() first.")
return self._session
def _get_server_info(self) -> tuple[str, str | None, str | None]:
"""Get server information for events.
Returns:
Tuple of (server_name, server_url, transport_type).
"""
if isinstance(self.transport, StdioTransport):
server_name = f"{self.transport.command} {' '.join(self.transport.args)}"
server_url = None
transport_type = self.transport.transport_type.value
elif isinstance(self.transport, HTTPTransport):
server_name = self.transport.url
server_url = self.transport.url
transport_type = self.transport.transport_type.value
elif isinstance(self.transport, SSETransport):
server_name = self.transport.url
server_url = self.transport.url
transport_type = self.transport.transport_type.value
else:
server_name = "Unknown MCP Server"
server_url = None
transport_type = (
self.transport.transport_type.value
if hasattr(self.transport, "transport_type")
else None
)
return server_name, server_url, transport_type
async def connect(self) -> Self:
"""Connect to MCP server and initialize session.
Returns:
Self for method chaining.
Raises:
ConnectionError: If connection fails.
ImportError: If MCP SDK not available.
"""
if self.connected:
return self
# Get server info for events
server_name, server_url, transport_type = self._get_server_info()
is_reconnect = self._was_connected
# Emit connection started event
started_at = datetime.now()
crewai_event_bus.emit(
self,
MCPConnectionStartedEvent(
server_name=server_name,
server_url=server_url,
transport_type=transport_type,
is_reconnect=is_reconnect,
connect_timeout=self.connect_timeout,
),
)
try:
from mcp import ClientSession
# Use AsyncExitStack to manage transport and session contexts together
# This ensures they're in the same async scope and prevents cancel scope errors
# Always enter transport context via exit stack (it handles already-connected state)
await self._exit_stack.enter_async_context(self.transport)
# Create ClientSession with transport streams
self._session = ClientSession(
self.transport.read_stream,
self.transport.write_stream,
)
# Enter the session's async context manager via exit stack
await self._exit_stack.enter_async_context(self._session)
# Initialize the session (required by MCP protocol)
try:
await asyncio.wait_for(
self._session.initialize(),
timeout=self.connect_timeout,
)
except asyncio.CancelledError:
# If initialization was cancelled (e.g., event loop closing),
# cleanup and re-raise - don't suppress cancellation
await self._cleanup_on_error()
raise
except BaseExceptionGroup as eg:
# Handle exception groups from anyio task groups
# Extract the actual meaningful error (not GeneratorExit)
actual_error = None
for exc in eg.exceptions:
if isinstance(exc, Exception) and not isinstance(
exc, GeneratorExit
):
# Check if it's an HTTP error (like 401)
error_msg = str(exc).lower()
if "401" in error_msg or "unauthorized" in error_msg:
actual_error = exc
break
if "cancel scope" not in error_msg and "task" not in error_msg:
actual_error = exc
break
await self._cleanup_on_error()
if actual_error:
raise ConnectionError(
f"Failed to connect to MCP server: {actual_error}"
) from actual_error
raise ConnectionError(f"Failed to connect to MCP server: {eg}") from eg
self._initialized = True
self._was_connected = True
completed_at = datetime.now()
connection_duration_ms = (completed_at - started_at).total_seconds() * 1000
crewai_event_bus.emit(
self,
MCPConnectionCompletedEvent(
server_name=server_name,
server_url=server_url,
transport_type=transport_type,
started_at=started_at,
completed_at=completed_at,
connection_duration_ms=connection_duration_ms,
is_reconnect=is_reconnect,
),
)
return self
except ImportError as e:
await self._cleanup_on_error()
error_msg = (
"MCP library not available. Please install with: pip install mcp"
)
self._emit_connection_failed(
server_name,
server_url,
transport_type,
error_msg,
"import_error",
started_at,
)
raise ImportError(error_msg) from e
except asyncio.TimeoutError as e:
await self._cleanup_on_error()
error_msg = f"MCP connection timed out after {self.connect_timeout} seconds. The server may be slow or unreachable."
self._emit_connection_failed(
server_name,
server_url,
transport_type,
error_msg,
"timeout",
started_at,
)
raise ConnectionError(error_msg) from e
except asyncio.CancelledError:
# Re-raise cancellation - don't suppress it
await self._cleanup_on_error()
self._emit_connection_failed(
server_name,
server_url,
transport_type,
"Connection cancelled",
"cancelled",
started_at,
)
raise
except BaseExceptionGroup as eg:
# Handle exception groups from anyio task groups at outer level
actual_error = None
for exc in eg.exceptions:
if isinstance(exc, Exception) and not isinstance(exc, GeneratorExit):
error_msg = str(exc).lower()
if "401" in error_msg or "unauthorized" in error_msg:
actual_error = exc
break
if "cancel scope" not in error_msg and "task" not in error_msg:
actual_error = exc
break
await self._cleanup_on_error()
error_type = (
"authentication"
if actual_error
and (
"401" in str(actual_error).lower()
or "unauthorized" in str(actual_error).lower()
)
else "network"
)
error_msg = str(actual_error) if actual_error else str(eg)
self._emit_connection_failed(
server_name,
server_url,
transport_type,
error_msg,
error_type,
started_at,
)
if actual_error:
raise ConnectionError(
f"Failed to connect to MCP server: {actual_error}"
) from actual_error
raise ConnectionError(f"Failed to connect to MCP server: {eg}") from eg
except Exception as e:
await self._cleanup_on_error()
error_type = (
"authentication"
if "401" in str(e).lower() or "unauthorized" in str(e).lower()
else "network"
)
self._emit_connection_failed(
server_name, server_url, transport_type, str(e), error_type, started_at
)
raise ConnectionError(f"Failed to connect to MCP server: {e}") from e
def _emit_connection_failed(
self,
server_name: str,
server_url: str | None,
transport_type: str | None,
error: str,
error_type: str,
started_at: datetime,
) -> None:
"""Emit connection failed event."""
failed_at = datetime.now()
crewai_event_bus.emit(
self,
MCPConnectionFailedEvent(
server_name=server_name,
server_url=server_url,
transport_type=transport_type,
error=error,
error_type=error_type,
started_at=started_at,
failed_at=failed_at,
),
)
async def _cleanup_on_error(self) -> None:
"""Cleanup resources when an error occurs during connection."""
try:
await self._exit_stack.aclose()
except Exception as e:
# Best effort cleanup - ignore all other errors
raise RuntimeError(f"Error during MCP client cleanup: {e}") from e
finally:
self._session = None
self._initialized = False
self._exit_stack = AsyncExitStack()
async def disconnect(self) -> None:
"""Disconnect from MCP server and cleanup resources."""
if not self.connected:
return
try:
await self._exit_stack.aclose()
except Exception as e:
raise RuntimeError(f"Error during MCP client disconnect: {e}") from e
finally:
self._session = None
self._initialized = False
self._exit_stack = AsyncExitStack()
async def list_tools(self, use_cache: bool | None = None) -> list[dict[str, Any]]:
"""List available tools from MCP server.
Args:
use_cache: Whether to use cached results. If None, uses
client's cache_tools_list setting.
Returns:
List of tool definitions with name, description, and inputSchema.
"""
if not self.connected:
await self.connect()
# Check cache if enabled
use_cache = use_cache if use_cache is not None else self.cache_tools_list
if use_cache:
cache_key = self._get_cache_key("tools")
if cache_key in _mcp_schema_cache:
cached_data, cache_time = _mcp_schema_cache[cache_key]
if time.time() - cache_time < _cache_ttl:
# Logger removed - return cached data
return cached_data
# List tools with timeout and retries
tools = await self._retry_operation(
self._list_tools_impl,
timeout=self.discovery_timeout,
)
# Cache results if enabled
if use_cache:
cache_key = self._get_cache_key("tools")
_mcp_schema_cache[cache_key] = (tools, time.time())
return tools
async def _list_tools_impl(self) -> list[dict[str, Any]]:
"""Internal implementation of list_tools."""
tools_result = await asyncio.wait_for(
self.session.list_tools(),
timeout=self.discovery_timeout,
)
return [
{
"name": tool.name,
"description": getattr(tool, "description", ""),
"inputSchema": getattr(tool, "inputSchema", {}),
}
for tool in tools_result.tools
]
async def call_tool(
self, tool_name: str, arguments: dict[str, Any] | None = None
) -> Any:
"""Call a tool on the MCP server.
Args:
tool_name: Name of the tool to call.
arguments: Tool arguments.
Returns:
Tool execution result.
"""
if not self.connected:
await self.connect()
arguments = arguments or {}
cleaned_arguments = self._clean_tool_arguments(arguments)
# Get server info for events
server_name, server_url, transport_type = self._get_server_info()
# Emit tool execution started event
started_at = datetime.now()
crewai_event_bus.emit(
self,
MCPToolExecutionStartedEvent(
server_name=server_name,
server_url=server_url,
transport_type=transport_type,
tool_name=tool_name,
tool_args=cleaned_arguments,
),
)
try:
result = await self._retry_operation(
lambda: self._call_tool_impl(tool_name, cleaned_arguments),
timeout=self.execution_timeout,
)
completed_at = datetime.now()
execution_duration_ms = (completed_at - started_at).total_seconds() * 1000
crewai_event_bus.emit(
self,
MCPToolExecutionCompletedEvent(
server_name=server_name,
server_url=server_url,
transport_type=transport_type,
tool_name=tool_name,
tool_args=cleaned_arguments,
result=result,
started_at=started_at,
completed_at=completed_at,
execution_duration_ms=execution_duration_ms,
),
)
return result
except Exception as e:
failed_at = datetime.now()
error_type = (
"timeout"
if isinstance(e, (asyncio.TimeoutError, ConnectionError))
and "timeout" in str(e).lower()
else "server_error"
)
crewai_event_bus.emit(
self,
MCPToolExecutionFailedEvent(
server_name=server_name,
server_url=server_url,
transport_type=transport_type,
tool_name=tool_name,
tool_args=cleaned_arguments,
error=str(e),
error_type=error_type,
started_at=started_at,
failed_at=failed_at,
),
)
raise
def _clean_tool_arguments(self, arguments: dict[str, Any]) -> dict[str, Any]:
"""Clean tool arguments by removing None values and fixing formats.
Args:
arguments: Raw tool arguments.
Returns:
Cleaned arguments ready for MCP server.
"""
cleaned = {}
for key, value in arguments.items():
# Skip None values
if value is None:
continue
# Fix sources array format: convert ["web"] to [{"type": "web"}]
if key == "sources" and isinstance(value, list):
fixed_sources = []
for item in value:
if isinstance(item, str):
# Convert string to object format
fixed_sources.append({"type": item})
elif isinstance(item, dict):
# Already in correct format
fixed_sources.append(item)
else:
# Keep as is if unknown format
fixed_sources.append(item)
if fixed_sources:
cleaned[key] = fixed_sources
continue
# Recursively clean nested dictionaries
if isinstance(value, dict):
nested_cleaned = self._clean_tool_arguments(value)
if nested_cleaned: # Only add if not empty
cleaned[key] = nested_cleaned
elif isinstance(value, list):
# Clean list items
cleaned_list = []
for item in value:
if isinstance(item, dict):
cleaned_item = self._clean_tool_arguments(item)
if cleaned_item:
cleaned_list.append(cleaned_item)
elif item is not None:
cleaned_list.append(item)
if cleaned_list:
cleaned[key] = cleaned_list
else:
# Keep primitive values
cleaned[key] = value
return cleaned
async def _call_tool_impl(self, tool_name: str, arguments: dict[str, Any]) -> Any:
"""Internal implementation of call_tool."""
result = await asyncio.wait_for(
self.session.call_tool(tool_name, arguments),
timeout=self.execution_timeout,
)
# Extract result content
if hasattr(result, "content") and result.content:
if isinstance(result.content, list) and len(result.content) > 0:
content_item = result.content[0]
if hasattr(content_item, "text"):
return str(content_item.text)
return str(content_item)
return str(result.content)
return str(result)
async def list_prompts(self) -> list[dict[str, Any]]:
"""List available prompts from MCP server.
Returns:
List of prompt definitions.
"""
if not self.connected:
await self.connect()
return await self._retry_operation(
self._list_prompts_impl,
timeout=self.discovery_timeout,
)
async def _list_prompts_impl(self) -> list[dict[str, Any]]:
"""Internal implementation of list_prompts."""
prompts_result = await asyncio.wait_for(
self.session.list_prompts(),
timeout=self.discovery_timeout,
)
return [
{
"name": prompt.name,
"description": getattr(prompt, "description", ""),
"arguments": getattr(prompt, "arguments", []),
}
for prompt in prompts_result.prompts
]
async def get_prompt(
self, prompt_name: str, arguments: dict[str, Any] | None = None
) -> dict[str, Any]:
"""Get a prompt from the MCP server.
Args:
prompt_name: Name of the prompt to get.
arguments: Optional prompt arguments.
Returns:
Prompt content and metadata.
"""
if not self.connected:
await self.connect()
arguments = arguments or {}
return await self._retry_operation(
lambda: self._get_prompt_impl(prompt_name, arguments),
timeout=self.execution_timeout,
)
async def _get_prompt_impl(
self, prompt_name: str, arguments: dict[str, Any]
) -> dict[str, Any]:
"""Internal implementation of get_prompt."""
result = await asyncio.wait_for(
self.session.get_prompt(prompt_name, arguments),
timeout=self.execution_timeout,
)
return {
"name": prompt_name,
"messages": [
{
"role": msg.role,
"content": msg.content,
}
for msg in result.messages
],
"arguments": arguments,
}
async def _retry_operation(
self,
operation: Callable[[], Any],
timeout: int | None = None,
) -> Any:
"""Retry an operation with exponential backoff.
Args:
operation: Async operation to retry.
timeout: Operation timeout in seconds.
Returns:
Operation result.
"""
last_error = None
timeout = timeout or self.execution_timeout
for attempt in range(self.max_retries):
try:
if timeout:
return await asyncio.wait_for(operation(), timeout=timeout)
return await operation()
except asyncio.TimeoutError as e: # noqa: PERF203
last_error = f"Operation timed out after {timeout} seconds"
if attempt < self.max_retries - 1:
wait_time = 2**attempt
await asyncio.sleep(wait_time)
else:
raise ConnectionError(last_error) from e
except Exception as e:
error_str = str(e).lower()
# Classify errors as retryable or non-retryable
if "authentication" in error_str or "unauthorized" in error_str:
raise ConnectionError(f"Authentication failed: {e}") from e
if "not found" in error_str:
raise ValueError(f"Resource not found: {e}") from e
# Retryable errors
last_error = str(e)
if attempt < self.max_retries - 1:
wait_time = 2**attempt
await asyncio.sleep(wait_time)
else:
raise ConnectionError(
f"Operation failed after {self.max_retries} attempts: {last_error}"
) from e
raise ConnectionError(f"Operation failed: {last_error}")
def _get_cache_key(self, resource_type: str) -> str:
"""Generate cache key for resource.
Args:
resource_type: Type of resource (e.g., "tools", "prompts").
Returns:
Cache key string.
"""
# Use transport type and URL/command as cache key
if isinstance(self.transport, StdioTransport):
key = f"stdio:{self.transport.command}:{':'.join(self.transport.args)}"
elif isinstance(self.transport, HTTPTransport):
key = f"http:{self.transport.url}"
elif isinstance(self.transport, SSETransport):
key = f"sse:{self.transport.url}"
else:
key = f"{self.transport.transport_type}:unknown"
return f"mcp:{key}:{resource_type}"
async def __aenter__(self) -> Self:
"""Async context manager entry."""
return await self.connect()
async def __aexit__(
self,
exc_type: type[BaseException] | None,
exc_val: BaseException | None,
exc_tb: Any,
) -> None:
"""Async context manager exit."""
await self.disconnect()

View File

@@ -1,124 +0,0 @@
"""MCP server configuration models for CrewAI agents.
This module provides Pydantic models for configuring MCP servers with
various transport types, similar to OpenAI's Agents SDK.
"""
from pydantic import BaseModel, Field
from crewai.mcp.filters import ToolFilter
class MCPServerStdio(BaseModel):
"""Stdio MCP server configuration.
This configuration is used for connecting to local MCP servers
that run as processes and communicate via standard input/output.
Example:
```python
mcp_server = MCPServerStdio(
command="python",
args=["path/to/server.py"],
env={"API_KEY": "..."},
tool_filter=create_static_tool_filter(
allowed_tool_names=["read_file", "write_file"]
),
)
```
"""
command: str = Field(
...,
description="Command to execute (e.g., 'python', 'node', 'npx', 'uvx').",
)
args: list[str] = Field(
default_factory=list,
description="Command arguments (e.g., ['server.py'] or ['-y', '@mcp/server']).",
)
env: dict[str, str] | None = Field(
default=None,
description="Environment variables to pass to the process.",
)
tool_filter: ToolFilter | None = Field(
default=None,
description="Optional tool filter for filtering available tools.",
)
cache_tools_list: bool = Field(
default=False,
description="Whether to cache the tool list for faster subsequent access.",
)
class MCPServerHTTP(BaseModel):
"""HTTP/Streamable HTTP MCP server configuration.
This configuration is used for connecting to remote MCP servers
over HTTP/HTTPS using streamable HTTP transport.
Example:
```python
mcp_server = MCPServerHTTP(
url="https://api.example.com/mcp",
headers={"Authorization": "Bearer ..."},
cache_tools_list=True,
)
```
"""
url: str = Field(
..., description="Server URL (e.g., 'https://api.example.com/mcp')."
)
headers: dict[str, str] | None = Field(
default=None,
description="Optional HTTP headers for authentication or other purposes.",
)
streamable: bool = Field(
default=True,
description="Whether to use streamable HTTP transport (default: True).",
)
tool_filter: ToolFilter | None = Field(
default=None,
description="Optional tool filter for filtering available tools.",
)
cache_tools_list: bool = Field(
default=False,
description="Whether to cache the tool list for faster subsequent access.",
)
class MCPServerSSE(BaseModel):
"""Server-Sent Events (SSE) MCP server configuration.
This configuration is used for connecting to remote MCP servers
using Server-Sent Events for real-time streaming communication.
Example:
```python
mcp_server = MCPServerSSE(
url="https://api.example.com/mcp/sse",
headers={"Authorization": "Bearer ..."},
)
```
"""
url: str = Field(
...,
description="Server URL (e.g., 'https://api.example.com/mcp/sse').",
)
headers: dict[str, str] | None = Field(
default=None,
description="Optional HTTP headers for authentication or other purposes.",
)
tool_filter: ToolFilter | None = Field(
default=None,
description="Optional tool filter for filtering available tools.",
)
cache_tools_list: bool = Field(
default=False,
description="Whether to cache the tool list for faster subsequent access.",
)
# Type alias for all MCP server configurations
MCPServerConfig = MCPServerStdio | MCPServerHTTP | MCPServerSSE

View File

@@ -1,166 +0,0 @@
"""Tool filtering support for MCP servers.
This module provides utilities for filtering tools from MCP servers,
including static allow/block lists and dynamic context-aware filtering.
"""
from collections.abc import Callable
from typing import TYPE_CHECKING, Any
from pydantic import BaseModel, Field
if TYPE_CHECKING:
pass
class ToolFilterContext(BaseModel):
"""Context for dynamic tool filtering.
This context is passed to dynamic tool filters to provide
information about the agent, run context, and server.
"""
agent: Any = Field(..., description="The agent requesting tools.")
server_name: str = Field(..., description="Name of the MCP server.")
run_context: dict[str, Any] | None = Field(
default=None,
description="Optional run context for additional filtering logic.",
)
# Type alias for tool filter functions
ToolFilter = (
Callable[[ToolFilterContext, dict[str, Any]], bool]
| Callable[[dict[str, Any]], bool]
)
class StaticToolFilter:
"""Static tool filter with allow/block lists.
This filter provides simple allow/block list filtering based on
tool names. Useful for restricting which tools are available
from an MCP server.
Example:
```python
filter = StaticToolFilter(
allowed_tool_names=["read_file", "write_file"],
blocked_tool_names=["delete_file"],
)
```
"""
def __init__(
self,
allowed_tool_names: list[str] | None = None,
blocked_tool_names: list[str] | None = None,
) -> None:
"""Initialize static tool filter.
Args:
allowed_tool_names: List of tool names to allow. If None,
all tools are allowed (unless blocked).
blocked_tool_names: List of tool names to block. Blocked tools
take precedence over allowed tools.
"""
self.allowed_tool_names = set(allowed_tool_names or [])
self.blocked_tool_names = set(blocked_tool_names or [])
def __call__(self, tool: dict[str, Any]) -> bool:
"""Filter tool based on allow/block lists.
Args:
tool: Tool definition dictionary with at least 'name' key.
Returns:
True if tool should be included, False otherwise.
"""
tool_name = tool.get("name", "")
# Blocked tools take precedence
if self.blocked_tool_names and tool_name in self.blocked_tool_names:
return False
# If allow list exists, tool must be in it
if self.allowed_tool_names:
return tool_name in self.allowed_tool_names
# No restrictions - allow all
return True
def create_static_tool_filter(
allowed_tool_names: list[str] | None = None,
blocked_tool_names: list[str] | None = None,
) -> Callable[[dict[str, Any]], bool]:
"""Create a static tool filter function.
This is a convenience function for creating static tool filters
with allow/block lists.
Args:
allowed_tool_names: List of tool names to allow. If None,
all tools are allowed (unless blocked).
blocked_tool_names: List of tool names to block. Blocked tools
take precedence over allowed tools.
Returns:
Tool filter function that returns True for allowed tools.
Example:
```python
filter_fn = create_static_tool_filter(
allowed_tool_names=["read_file", "write_file"],
blocked_tool_names=["delete_file"],
)
# Use in MCPServerStdio
mcp_server = MCPServerStdio(
command="npx",
args=["-y", "@modelcontextprotocol/server-filesystem"],
tool_filter=filter_fn,
)
```
"""
return StaticToolFilter(
allowed_tool_names=allowed_tool_names,
blocked_tool_names=blocked_tool_names,
)
def create_dynamic_tool_filter(
filter_func: Callable[[ToolFilterContext, dict[str, Any]], bool],
) -> Callable[[ToolFilterContext, dict[str, Any]], bool]:
"""Create a dynamic tool filter function.
This function wraps a dynamic filter function that has access
to the tool filter context (agent, server, run context).
Args:
filter_func: Function that takes (context, tool) and returns bool.
Returns:
Tool filter function that can be used with MCP server configs.
Example:
```python
async def context_aware_filter(
context: ToolFilterContext, tool: dict[str, Any]
) -> bool:
# Block dangerous tools for code reviewers
if context.agent.role == "Code Reviewer":
if tool["name"].startswith("danger_"):
return False
return True
filter_fn = create_dynamic_tool_filter(context_aware_filter)
mcp_server = MCPServerStdio(
command="python", args=["server.py"], tool_filter=filter_fn
)
```
"""
return filter_func

View File

@@ -1,15 +0,0 @@
"""MCP transport implementations for various connection types."""
from crewai.mcp.transports.base import BaseTransport, TransportType
from crewai.mcp.transports.http import HTTPTransport
from crewai.mcp.transports.sse import SSETransport
from crewai.mcp.transports.stdio import StdioTransport
__all__ = [
"BaseTransport",
"HTTPTransport",
"SSETransport",
"StdioTransport",
"TransportType",
]

View File

@@ -1,125 +0,0 @@
"""Base transport interface for MCP connections."""
from abc import ABC, abstractmethod
from enum import Enum
from typing import Any, Protocol
from typing_extensions import Self
class TransportType(str, Enum):
"""MCP transport types."""
STDIO = "stdio"
HTTP = "http"
STREAMABLE_HTTP = "streamable-http"
SSE = "sse"
class ReadStream(Protocol):
"""Protocol for read streams."""
async def read(self, n: int = -1) -> bytes:
"""Read bytes from stream."""
...
class WriteStream(Protocol):
"""Protocol for write streams."""
async def write(self, data: bytes) -> None:
"""Write bytes to stream."""
...
class BaseTransport(ABC):
"""Base class for MCP transport implementations.
This abstract base class defines the interface that all transport
implementations must follow. Transports handle the low-level communication
with MCP servers.
"""
def __init__(self, **kwargs: Any) -> None:
"""Initialize the transport.
Args:
**kwargs: Transport-specific configuration options.
"""
self._read_stream: ReadStream | None = None
self._write_stream: WriteStream | None = None
self._connected = False
@property
@abstractmethod
def transport_type(self) -> TransportType:
"""Return the transport type."""
...
@property
def connected(self) -> bool:
"""Check if transport is connected."""
return self._connected
@property
def read_stream(self) -> ReadStream:
"""Get the read stream."""
if self._read_stream is None:
raise RuntimeError("Transport not connected. Call connect() first.")
return self._read_stream
@property
def write_stream(self) -> WriteStream:
"""Get the write stream."""
if self._write_stream is None:
raise RuntimeError("Transport not connected. Call connect() first.")
return self._write_stream
@abstractmethod
async def connect(self) -> Self:
"""Establish connection to MCP server.
Returns:
Self for method chaining.
Raises:
ConnectionError: If connection fails.
"""
...
@abstractmethod
async def disconnect(self) -> None:
"""Close connection to MCP server."""
...
@abstractmethod
async def __aenter__(self) -> Self:
"""Async context manager entry."""
...
@abstractmethod
async def __aexit__(
self,
exc_type: type[BaseException] | None,
exc_val: BaseException | None,
exc_tb: Any,
) -> None:
"""Async context manager exit."""
...
def _set_streams(self, read: ReadStream, write: WriteStream) -> None:
"""Set the read and write streams.
Args:
read: Read stream.
write: Write stream.
"""
self._read_stream = read
self._write_stream = write
self._connected = True
def _clear_streams(self) -> None:
"""Clear the read and write streams."""
self._read_stream = None
self._write_stream = None
self._connected = False

Some files were not shown because too many files have changed in this diff Show More