Compare commits

..

1 Commits

Author SHA1 Message Date
Devin AI
d8cd5e5474 feat: add Confident AI observability integration documentation
- Add comprehensive Confident AI integration guide
- Include setup instructions and code examples
- Update observability overview with new integration
- Add navigation entry for new documentation

Closes #3285

Co-Authored-By: João <joao@crewai.com>
2025-08-07 13:11:06 +00:00
40 changed files with 292 additions and 1532 deletions

View File

@@ -128,9 +128,7 @@
"en/tools/file-document/jsonsearchtool",
"en/tools/file-document/csvsearchtool",
"en/tools/file-document/directorysearchtool",
"en/tools/file-document/directoryreadtool",
"en/tools/file-document/ocrtool",
"en/tools/file-document/pdf-text-writing-tool"
"en/tools/file-document/directoryreadtool"
]
},
{
@@ -148,8 +146,7 @@
"en/tools/web-scraping/stagehandtool",
"en/tools/web-scraping/firecrawlcrawlwebsitetool",
"en/tools/web-scraping/firecrawlscrapewebsitetool",
"en/tools/web-scraping/oxylabsscraperstool",
"en/tools/web-scraping/brightdata-tools"
"en/tools/web-scraping/oxylabsscraperstool"
]
},
{
@@ -166,11 +163,7 @@
"en/tools/search-research/youtubechannelsearchtool",
"en/tools/search-research/youtubevideosearchtool",
"en/tools/search-research/tavilysearchtool",
"en/tools/search-research/tavilyextractortool",
"en/tools/search-research/arxivpapertool",
"en/tools/search-research/serpapi-googlesearchtool",
"en/tools/search-research/serpapi-googleshoppingtool",
"en/tools/search-research/databricks-query-tool"
"en/tools/search-research/tavilyextractortool"
]
},
{
@@ -182,9 +175,7 @@
"en/tools/database-data/snowflakesearchtool",
"en/tools/database-data/nl2sqltool",
"en/tools/database-data/qdrantvectorsearchtool",
"en/tools/database-data/weaviatevectorsearchtool",
"en/tools/database-data/mongodbvectorsearchtool",
"en/tools/database-data/singlestoresearchtool"
"en/tools/database-data/weaviatevectorsearchtool"
]
},
{
@@ -216,8 +207,7 @@
"en/tools/automation/overview",
"en/tools/automation/apifyactorstool",
"en/tools/automation/composiotool",
"en/tools/automation/multiontool",
"en/tools/automation/zapieractionstool"
"en/tools/automation/multiontool"
]
}
]
@@ -228,6 +218,7 @@
"en/observability/overview",
"en/observability/agentops",
"en/observability/arize-phoenix",
"en/observability/confident-ai",
"en/observability/langdb",
"en/observability/langfuse",
"en/observability/langtrace",

View File

@@ -0,0 +1,170 @@
---
title: Confident AI Integration
description: Monitor and evaluate your CrewAI agents with Confident AI's comprehensive observability platform.
icon: shield-check
---
# Introduction
Confident AI provides a comprehensive observability and evaluation platform for LLM applications, including CrewAI agents. It offers detailed tracing, monitoring, and evaluation capabilities to help you understand agent performance, identify issues, and ensure reliable operation in production environments.
## Confident AI
[Confident AI](https://confident-ai.com) is built on top of DeepEval, an open-source evaluation framework, and provides advanced monitoring, tracing, and evaluation features for AI applications.
At a high level, Confident AI gives you the ability to monitor agent execution, evaluate output quality, track performance metrics, and gain insights into your CrewAI workflows. For more information, check out the [Confident AI Documentation](https://documentation.confident-ai.com).
### Overview
Confident AI provides end-to-end observability for CrewAI agents in both development and production environments. It offers detailed tracing of agent interactions, LLM calls, and task execution, along with comprehensive evaluation metrics and analytics dashboards.
The platform enables you to monitor agent performance, identify bottlenecks, evaluate output quality, and optimize your CrewAI workflows for better results and cost efficiency.
### Features
- **End-to-End Tracing**: Complete visibility into agent execution flows and LLM interactions
- **Performance Monitoring**: Track execution times, token usage, and resource consumption
- **Quality Evaluation**: Automated evaluation of agent outputs using various metrics
- **Cost Tracking**: Monitor LLM API usage and associated costs
- **Real-time Analytics**: Live dashboards for monitoring agent performance
- **Custom Metrics**: Define and track domain-specific evaluation criteria
- **Anomaly Detection**: Identify unusual patterns in agent behavior
- **Compliance Monitoring**: Ensure outputs meet safety and quality standards
- **A/B Testing**: Compare different agent configurations and prompts
- **Historical Analysis**: Track performance trends over time
### Using Confident AI
<Steps>
<Step title="Create an Account">
Sign up for a Confident AI account at: [Confident AI Platform](https://app.confident-ai.com)
</Step>
<Step title="Get Your API Key">
Obtain your API key from the Confident AI dashboard under Settings > API Keys
</Step>
<Step title="Configure Your Environment">
Add your API key to your environment variables:
```bash
CONFIDENT_API_KEY=<YOUR_CONFIDENT_API_KEY>
```
</Step>
<Step title="Install DeepEval">
Install DeepEval with Confident AI integration:
```bash
pip install deepeval
```
</Step>
<Step title="Initialize Confident AI Tracing">
Before using `Crew` in your script, include these lines:
```python
from deepeval.integrations.crewai import instrument_crewai
# Initialize Confident AI tracing for CrewAI
instrument_crewai(api_key="<your-confident-api-key>")
```
This will automatically trace all CrewAI agent interactions and send the data to your Confident AI dashboard.
</Step>
<Step title="Run Your CrewAI Application">
Execute your CrewAI workflows as usual. All agent interactions, LLM calls, and task executions will be automatically traced and sent to Confident AI:
```python
from crewai import Agent, Task, Crew
from deepeval.integrations.crewai import instrument_crewai
# Initialize tracing
instrument_crewai(api_key="<your-confident-api-key>")
# Define your agents and tasks
researcher = Agent(
role='Researcher',
goal='Research and analyze market trends',
backstory='You are an expert market researcher.',
verbose=True
)
task = Task(
description='Research the latest AI trends in 2024',
agent=researcher,
expected_output='A comprehensive report on AI trends'
)
crew = Crew(
agents=[researcher],
tasks=[task],
verbose=True
)
# Execute the crew - all interactions will be traced
result = crew.kickoff()
```
</Step>
</Steps>
### Advanced Configuration
You can customize the tracing behavior with additional configuration options:
```python
from deepeval.integrations.crewai import instrument_crewai
# Advanced configuration
instrument_crewai(
api_key="<your-confident-api-key>",
project_name="my-crewai-project",
environment="production",
custom_metadata={
"version": "1.0.0",
"team": "ai-research"
}
)
```
### Evaluation and Monitoring
Confident AI automatically evaluates your agent outputs using various metrics. You can also define custom evaluation criteria:
```python
from deepeval import evaluate
from deepeval.metrics import AnswerRelevancyMetric, FaithfulnessMetric
# Define evaluation metrics
metrics = [
AnswerRelevancyMetric(threshold=0.8),
FaithfulnessMetric(threshold=0.7)
]
# Your CrewAI execution will be automatically evaluated
# Results will appear in your Confident AI dashboard
```
### Key Benefits
- **Comprehensive Visibility**: Complete tracing of agent workflows and interactions
- **Quality Assurance**: Automated evaluation ensures consistent output quality
- **Performance Optimization**: Identify bottlenecks and optimize agent performance
- **Cost Management**: Track and optimize LLM usage costs
- **Production Monitoring**: Real-time monitoring for production deployments
- **Continuous Improvement**: Historical data enables iterative optimization
### Further Information
To get started with Confident AI:
- [Sign up for Confident AI](https://app.confident-ai.com)
- [Read the Documentation](https://documentation.confident-ai.com)
- [Explore DeepEval](https://github.com/confident-ai/deepeval)
- [CrewAI Integration Guide](https://documentation.confident-ai.com/docs/llm-tracing/integrations/crewai)
For support or questions, reach out to the Confident AI team through their documentation or support channels.
#### Extra links
<a href="https://confident-ai.com">🌐 Confident AI Website</a>
<span>&nbsp;&nbsp;•&nbsp;&nbsp;</span>
<a href="https://app.confident-ai.com">📊 Confident AI Dashboard</a>
<span>&nbsp;&nbsp;•&nbsp;&nbsp;</span>
<a href="https://documentation.confident-ai.com">📙 Documentation</a>
<span>&nbsp;&nbsp;•&nbsp;&nbsp;</span>
<a href="https://github.com/confident-ai/deepeval">🔧 DeepEval GitHub</a>

View File

@@ -49,6 +49,10 @@ Observability is crucial for understanding how your CrewAI agents perform, ident
AI observability platform for monitoring and troubleshooting.
</Card>
<Card title="Confident AI" icon="shield-check" href="/en/observability/confident-ai">
Comprehensive observability and evaluation platform with end-to-end tracing and quality monitoring.
</Card>
<Card title="Portkey" icon="key" href="/en/observability/portkey">
AI gateway with comprehensive monitoring and reliability features.
</Card>

View File

@@ -35,7 +35,7 @@ os.environ['OTEL_SDK_DISABLED'] = 'true'
### Data Explanation:
| Defaulted | Data | Reason and Specifics |
|:----------|:------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------|
|-----------|-------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------|
| Yes | CrewAI and Python Version | Tracks software versions. Example: CrewAI v1.2.3, Python 3.8.10. No personal data. |
| Yes | Crew Metadata | Includes: randomly generated key and ID, process type (e.g., 'sequential', 'parallel'), boolean flag for memory usage (true/false), count of tasks, count of agents. All non-personal. |
| Yes | Agent Data | Includes: randomly generated key and ID, role name (should not include personal info), boolean settings (verbose, delegation enabled, code execution allowed), max iterations, max RPM, max retry limit, LLM info (see LLM Attributes), list of tool names (should not include personal info). No personal data. |

View File

@@ -20,10 +20,6 @@ These tools enable your agents to automate workflows, integrate with external pl
<Card title="Multion Tool" icon="window-restore" href="/en/tools/automation/multiontool">
Automate browser interactions and web-based workflows.
</Card>
<Card title="Zapier Actions Adapter" icon="bolt" href="/en/tools/automation/zapieractionstool">
Expose Zapier Actions as CrewAI tools for automation across thousands of apps.
</Card>
</CardGroup>
## **Common Use Cases**

View File

@@ -1,58 +0,0 @@
---
title: Zapier Actions Tool
description: The `ZapierActionsAdapter` exposes Zapier actions as CrewAI tools for automation.
icon: bolt
---
# `ZapierActionsAdapter`
## Description
Use the Zapier adapter to list and call Zapier actions as CrewAI tools. This enables agents to trigger automations across thousands of apps.
## Installation
This adapter is included with `crewai-tools`. No extra install required.
## Environment Variables
- `ZAPIER_API_KEY` (required): Zapier API key. Get one from the Zapier Actions dashboard at https://actions.zapier.com/ (create an account, then generate an API key). You can also pass `zapier_api_key` directly when constructing the adapter.
## Example
```python Code
from crewai import Agent, Task, Crew
from crewai_tools.adapters.zapier_adapter import ZapierActionsAdapter
adapter = ZapierActionsAdapter(api_key="your_zapier_api_key")
tools = adapter.tools()
agent = Agent(
role="Automator",
goal="Execute Zapier actions",
backstory="Automation specialist",
tools=tools,
verbose=True,
)
task = Task(
description="Create a new Google Sheet and add a row using Zapier actions",
expected_output="Confirmation with created resource IDs",
agent=agent,
)
crew = Crew(agents=[agent], tasks=[task])
result = crew.kickoff()
```
## Notes & limits
- The adapter lists available actions for your key and creates `BaseTool` wrappers dynamically.
- Handle actionspecific required fields in your task instructions or tool call.
- Rate limits depend on your Zapier plan; see the Zapier Actions docs.
## Notes
- The adapter fetches available actions and generates `BaseTool` wrappers dynamically.

View File

@@ -1,168 +0,0 @@
---
title: MongoDB Vector Search Tool
description: The `MongoDBVectorSearchTool` performs vector search on MongoDB Atlas with optional indexing helpers.
icon: "leaf"
---
# `MongoDBVectorSearchTool`
## Description
Perform vector similarity queries on MongoDB Atlas collections. Supports index creation helpers and bulk insert of embedded texts.
MongoDB Atlas supports native vector search. Learn more:
https://www.mongodb.com/docs/atlas/atlas-vector-search/vector-search-overview/
## Installation
Install with the MongoDB extra:
```shell
pip install crewai-tools[mongodb]
```
or
```shell
uv add crewai-tools --extra mongodb
```
## Parameters
### Initialization
- `connection_string` (str, required)
- `database_name` (str, required)
- `collection_name` (str, required)
- `vector_index_name` (str, default `vector_index`)
- `text_key` (str, default `text`)
- `embedding_key` (str, default `embedding`)
- `dimensions` (int, default `1536`)
### Run Parameters
- `query` (str, required): Natural language query to embed and search.
## Quick start
```python Code
from crewai_tools import MongoDBVectorSearchTool
tool = MongoDBVectorSearchTool(
connection_string="mongodb+srv://...",
database_name="mydb",
collection_name="docs",
)
print(tool.run(query="how to create vector index"))
```
## Index creation helpers
Use `create_vector_search_index(...)` to provision an Atlas Vector Search index with the correct dimensions and similarity.
## Common issues
- Authentication failures: ensure your Atlas IP Access List allows your runner and the connection string includes credentials.
- Index not found: create the vector index first; name must match `vector_index_name`.
- Dimensions mismatch: align embedding model dimensions with `dimensions`.
## More examples
### Basic initialization
```python Code
from crewai_tools import MongoDBVectorSearchTool
tool = MongoDBVectorSearchTool(
database_name="example_database",
collection_name="example_collection",
connection_string="<your_mongodb_connection_string>",
)
```
### Custom query configuration
```python Code
from crewai_tools import MongoDBVectorSearchConfig, MongoDBVectorSearchTool
query_config = MongoDBVectorSearchConfig(limit=10, oversampling_factor=2)
tool = MongoDBVectorSearchTool(
database_name="example_database",
collection_name="example_collection",
connection_string="<your_mongodb_connection_string>",
query_config=query_config,
vector_index_name="my_vector_index",
)
rag_agent = Agent(
name="rag_agent",
role="You are a helpful assistant that can answer questions with the help of the MongoDBVectorSearchTool.",
goal="...",
backstory="...",
tools=[tool],
)
```
### Preloading the database and creating the index
```python Code
import os
from crewai_tools import MongoDBVectorSearchTool
tool = MongoDBVectorSearchTool(
database_name="example_database",
collection_name="example_collection",
connection_string="<your_mongodb_connection_string>",
)
# Load text content from a local folder and add to MongoDB
texts = []
for fname in os.listdir("knowledge"):
path = os.path.join("knowledge", fname)
if os.path.isfile(path):
with open(path, "r", encoding="utf-8") as f:
texts.append(f.read())
tool.add_texts(texts)
# Create the Atlas Vector Search index (e.g., 3072 dims for text-embedding-3-large)
tool.create_vector_search_index(dimensions=3072)
```
## Example
```python Code
from crewai import Agent, Task, Crew
from crewai_tools import MongoDBVectorSearchTool
tool = MongoDBVectorSearchTool(
connection_string="mongodb+srv://...",
database_name="mydb",
collection_name="docs",
)
agent = Agent(
role="RAG Agent",
goal="Answer using MongoDB vector search",
backstory="Knowledge retrieval specialist",
tools=[tool],
verbose=True,
)
task = Task(
description="Find relevant content for 'indexing guidance'",
expected_output="A concise answer citing the most relevant matches",
agent=agent,
)
crew = Crew(
agents=[agent],
tasks=[task],
verbose=True,
)
result = crew.kickoff()
```

View File

@@ -32,14 +32,6 @@ These tools enable your agents to interact with various database systems, from t
<Card title="Weaviate Vector Search" icon="network-wired" href="/en/tools/database-data/weaviatevectorsearchtool">
Perform semantic search with Weaviate vector database.
</Card>
<Card title="MongoDB Vector Search" icon="leaf" href="/en/tools/database-data/mongodbvectorsearchtool">
Vector similarity search on MongoDB Atlas with indexing helpers.
</Card>
<Card title="SingleStore Search" icon="database" href="/en/tools/database-data/singlestoresearchtool">
Safe SELECT/SHOW queries on SingleStore with pooling and validation.
</Card>
</CardGroup>
## **Common Use Cases**

View File

@@ -1,61 +0,0 @@
---
title: SingleStore Search Tool
description: The `SingleStoreSearchTool` safely executes SELECT/SHOW queries on SingleStore with pooling.
icon: circle
---
# `SingleStoreSearchTool`
## Description
Execute readonly queries (`SELECT`/`SHOW`) against SingleStore with connection pooling and input validation.
## Installation
```shell
uv add crewai-tools[singlestore]
```
## Environment Variables
Variables like `SINGLESTOREDB_HOST`, `SINGLESTOREDB_USER`, `SINGLESTOREDB_PASSWORD`, etc., can be used, or `SINGLESTOREDB_URL` as a single DSN.
Generate the API key from the SingleStore dashboard, [docs here](https://docs.singlestore.com/cloud/reference/management-api/#generate-an-api-key).
## Example
```python Code
from crewai import Agent, Task, Crew
from crewai_tools import SingleStoreSearchTool
tool = SingleStoreSearchTool(
tables=["products"],
host="host",
user="user",
password="pass",
database="db",
)
agent = Agent(
role="Analyst",
goal="Query SingleStore",
tools=[tool],
verbose=True,
)
task = Task(
description="List 5 products",
expected_output="5 rows as JSON/text",
agent=agent,
)
crew = Crew(
agents=[agent],
tasks=[task],
verbose=True,
)
result = crew.kickoff()
```

View File

@@ -1,89 +0,0 @@
---
title: OCR Tool
description: The `OCRTool` extracts text from local images or image URLs using an LLM with vision.
icon: image
---
# `OCRTool`
## Description
Extract text from images (local path or URL). Uses a visioncapable LLM via CrewAIs LLM interface.
## Installation
No extra install beyond `crewai-tools`. Ensure your selected LLM supports vision.
## Parameters
### Run Parameters
- `image_path_url` (str, required): Local image path or HTTP(S) URL.
## Examples
### Direct usage
```python Code
from crewai_tools import OCRTool
print(OCRTool().run(image_path_url="/tmp/receipt.png"))
```
### With an agent
```python Code
from crewai import Agent, Task, Crew
from crewai_tools import OCRTool
ocr = OCRTool()
agent = Agent(
role="OCR",
goal="Extract text",
tools=[ocr],
)
task = Task(
description="Extract text from https://example.com/invoice.jpg",
expected_output="All detected text in plain text",
agent=agent,
)
crew = Crew(agents=[agent], tasks=[task])
result = crew.kickoff()
```
## Notes
- Ensure the selected LLM supports image inputs.
- For large images, consider downscaling to reduce token usage.
- You can pass a specific LLM instance to the tool (e.g., `LLM(model="gpt-4o")`) if needed, matching the README guidance.
## Example
```python Code
from crewai import Agent, Task, Crew
from crewai_tools import OCRTool
tool = OCRTool()
agent = Agent(
role="OCR Specialist",
goal="Extract text from images",
backstory="Visionenabled analyst",
tools=[tool],
verbose=True,
)
task = Task(
description="Extract text from https://example.com/receipt.png",
expected_output="All detected text in plain text",
agent=agent,
)
crew = Crew(agents=[agent], tasks=[task])
result = crew.kickoff()
```

View File

@@ -52,14 +52,6 @@ These tools enable your agents to work with various file formats and document ty
<Card title="Directory Read Tool" icon="folder" href="/en/tools/file-document/directoryreadtool">
Read and list directory contents, file structures, and metadata.
</Card>
<Card title="OCR Tool" icon="image" href="/en/tools/file-document/ocrtool">
Extract text from images (local files or URLs) using a visioncapable LLM.
</Card>
<Card title="PDF Text Writing Tool" icon="file-pdf" href="/en/tools/file-document/pdf-text-writing-tool">
Write text at specific coordinates in PDFs, with optional custom fonts.
</Card>
</CardGroup>
## **Common Use Cases**

View File

@@ -1,76 +0,0 @@
---
title: PDF Text Writing Tool
description: The `PDFTextWritingTool` writes text to specific positions in a PDF, supporting custom fonts.
icon: file-pdf
---
# `PDFTextWritingTool`
## Description
Write text at precise coordinates on a PDF page, optionally embedding a custom TrueType font.
## Parameters
### Run Parameters
- `pdf_path` (str, required): Path to the input PDF.
- `text` (str, required): Text to add.
- `position` (tuple[int, int], required): `(x, y)` coordinates.
- `font_size` (int, default `12`)
- `font_color` (str, default `"0 0 0 rg"`)
- `font_name` (str, default `"F1"`)
- `font_file` (str, optional): Path to `.ttf` file.
- `page_number` (int, default `0`)
## Example
```python Code
from crewai import Agent, Task, Crew
from crewai_tools import PDFTextWritingTool
tool = PDFTextWritingTool()
agent = Agent(
role="PDF Editor",
goal="Annotate PDFs",
backstory="Documentation specialist",
tools=[tool],
verbose=True,
)
task = Task(
description="Write 'CONFIDENTIAL' at (72, 720) on page 1 of ./sample.pdf",
expected_output="Confirmation message",
agent=agent,
)
crew = Crew(
agents=[agent],
tasks=[task],
verbose=True,
)
result = crew.kickoff()
```
### Direct usage
```python Code
from crewai_tools import PDFTextWritingTool
PDFTextWritingTool().run(
pdf_path="./input.pdf",
text="CONFIDENTIAL",
position=(72, 720),
font_size=18,
page_number=0,
)
```
## Tips
- Coordinate origin is the bottomleft corner.
- If using a custom font (`font_file`), ensure it is a valid `.ttf`.

View File

@@ -1,112 +0,0 @@
---
title: Arxiv Paper Tool
description: The `ArxivPaperTool` searches arXiv for papers matching a query and optionally downloads PDFs.
icon: box-archive
---
# `ArxivPaperTool`
## Description
The `ArxivPaperTool` queries the arXiv API for academic papers and returns compact, readable results. It can also optionally download PDFs to disk.
## Installation
This tool has no special installation beyond `crewai-tools`.
```shell
uv add crewai-tools
```
No API key is required. This tool uses the public arXiv Atom API.
## Steps to Get Started
1. Initialize the tool.
2. Provide a `search_query` (e.g., "transformer neural network").
3. Optionally set `max_results` (1100) and enable PDF downloads in the constructor.
## Example
```python Code
from crewai import Agent, Task, Crew
from crewai_tools import ArxivPaperTool
tool = ArxivPaperTool(
download_pdfs=False,
save_dir="./arxiv_pdfs",
use_title_as_filename=True,
)
agent = Agent(
role="Researcher",
goal="Find relevant arXiv papers",
backstory="Expert at literature discovery",
tools=[tool],
verbose=True,
)
task = Task(
description="Search arXiv for 'transformer neural network' and list top 5 results.",
expected_output="A concise list of 5 relevant papers with titles, links, and summaries.",
agent=agent,
)
crew = Crew(agents=[agent], tasks=[task])
result = crew.kickoff()
```
### Direct usage (without Agent)
```python Code
from crewai_tools import ArxivPaperTool
tool = ArxivPaperTool(
download_pdfs=True,
save_dir="./arxiv_pdfs",
)
print(tool.run(search_query="mixture of experts", max_results=3))
```
## Parameters
### Initialization Parameters
- `download_pdfs` (bool, default `False`): Whether to download PDFs.
- `save_dir` (str, default `./arxiv_pdfs`): Directory to save PDFs.
- `use_title_as_filename` (bool, default `False`): Use paper titles for filenames.
### Run Parameters
- `search_query` (str, required): The arXiv search query.
- `max_results` (int, default `5`, range 1100): Number of results.
## Output format
The tool returns a humanreadable list of papers with:
- Title
- Link (abs page)
- Snippet/summary (truncated)
When `download_pdfs=True`, PDFs are saved to disk and the summary mentions saved files.
## Usage Notes
- The tool returns formatted text with key metadata and links.
- When `download_pdfs=True`, PDFs will be stored in `save_dir`.
## Troubleshooting
- If you receive a network timeout, retry or reduce `max_results`.
- Invalid XML errors indicate an arXiv response parse issue; try a simpler query.
- File system errors (e.g., permission denied) may occur when saving PDFs; ensure `save_dir` is writable.
## Related links
- arXiv API docs: https://info.arxiv.org/help/api/index.html
## Error Handling
- Network issues, invalid XML, and OS errors are handled with informative messages.

View File

@@ -23,7 +23,7 @@ pip install 'crewai[tools]'
To effectively use the `BraveSearchTool`, follow these steps:
1. **Package Installation**: Confirm that the `crewai[tools]` package is installed in your Python environment.
2. **API Key Acquisition**: Acquire a Brave Search API key at https://api.search.brave.com/app/keys (sign in to generate a key).
2. **API Key Acquisition**: Acquire a Brave Search API key by registering at [Brave Search API](https://api.search.brave.com/app/keys).
3. **Environment Configuration**: Store your obtained API key in an environment variable named `BRAVE_API_KEY` to facilitate its use by the tool.
## Example

View File

@@ -1,80 +0,0 @@
---
title: Databricks SQL Query Tool
description: The `DatabricksQueryTool` executes SQL queries against Databricks workspace tables.
icon: trowel-bricks
---
# `DatabricksQueryTool`
## Description
Run SQL against Databricks workspace tables with either CLI profile or direct host/token authentication.
## Installation
```shell
uv add crewai-tools[databricks-sdk]
```
## Environment Variables
- `DATABRICKS_CONFIG_PROFILE` or (`DATABRICKS_HOST` + `DATABRICKS_TOKEN`)
Create a personal access token and find host details in the Databricks workspace under User Settings → Developer.
Docs: https://docs.databricks.com/en/dev-tools/auth/pat.html
## Example
```python Code
from crewai import Agent, Task, Crew
from crewai_tools import DatabricksQueryTool
tool = DatabricksQueryTool(
default_catalog="main",
default_schema="default",
)
agent = Agent(
role="Data Analyst",
goal="Query Databricks",
tools=[tool],
verbose=True,
)
task = Task(
description="SELECT * FROM my_table LIMIT 10",
expected_output="10 rows",
agent=agent,
)
crew = Crew(
agents=[agent],
tasks=[task],
verbose=True,
)
result = crew.kickoff()
print(result)
```
## Parameters
- `query` (required): SQL query to execute
- `catalog` (optional): Override default catalog
- `db_schema` (optional): Override default schema
- `warehouse_id` (optional): Override default SQL warehouse
- `row_limit` (optional): Maximum rows to return (default: 1000)
## Defaults on initialization
- `default_catalog`
- `default_schema`
- `default_warehouse_id`
### Error handling & tips
- Authentication errors: verify `DATABRICKS_HOST` begins with `https://` and token is valid.
- Permissions: ensure your SQL warehouse and schema are accessible by your token.
- Limits: longrunning queries should be avoided in agent loops; add filters/limits.

View File

@@ -24,8 +24,6 @@ pip install 'crewai[tools]'
This command installs the necessary package to run the GithubSearchTool along with any other tools included in the crewai_tools package.
Get a GitHub Personal Access Token at https://github.com/settings/tokens (Developer settings → Finegrained tokens or classic tokens).
## Example
Heres how you can use the GithubSearchTool to perform semantic searches within a GitHub repository:

View File

@@ -52,18 +52,6 @@ These tools enable your agents to search the web, research topics, and find info
<Card title="Tavily Extractor Tool" icon="file-text" href="/en/tools/search-research/tavilyextractortool">
Extract structured content from web pages using the Tavily API.
</Card>
<Card title="Arxiv Paper Tool" icon="box-archive" href="/en/tools/search-research/arxivpapertool">
Search arXiv and optionally download PDFs.
</Card>
<Card title="SerpApi Google Search" icon="search" href="/en/tools/search-research/serpapi-googlesearchtool">
Google search via SerpApi with structured results.
</Card>
<Card title="SerpApi Google Shopping" icon="cart-shopping" href="/en/tools/search-research/serpapi-googleshoppingtool">
Google Shopping queries via SerpApi.
</Card>
</CardGroup>
## **Common Use Cases**

View File

@@ -1,65 +0,0 @@
---
title: SerpApi Google Search Tool
description: The `SerpApiGoogleSearchTool` performs Google searches using the SerpApi service.
icon: google
---
# `SerpApiGoogleSearchTool`
## Description
Use the `SerpApiGoogleSearchTool` to run Google searches with SerpApi and retrieve structured results. Requires a SerpApi API key.
## Installation
```shell
uv add crewai-tools[serpapi]
```
## Environment Variables
- `SERPAPI_API_KEY` (required): API key for SerpApi. Create one at https://serpapi.com/ (free tier available).
## Example
```python Code
from crewai import Agent, Task, Crew
from crewai_tools import SerpApiGoogleSearchTool
tool = SerpApiGoogleSearchTool()
agent = Agent(
role="Researcher",
goal="Answer questions using Google search",
backstory="Search specialist",
tools=[tool],
verbose=True,
)
task = Task(
description="Search for the latest CrewAI releases",
expected_output="A concise list of relevant results with titles and links",
agent=agent,
)
crew = Crew(agents=[agent], tasks=[task])
result = crew.kickoff()
```
## Notes
- Set `SERPAPI_API_KEY` in the environment. Create a key at https://serpapi.com/
- See also Google Shopping via SerpApi: `/en/tools/search-research/serpapi-googleshoppingtool`
## Parameters
### Run Parameters
- `search_query` (str, required): The Google query.
- `location` (str, optional): Geographic location parameter.
## Notes
- This tool wraps SerpApi and returns structured search results.

View File

@@ -1,61 +0,0 @@
---
title: SerpApi Google Shopping Tool
description: The `SerpApiGoogleShoppingTool` searches Google Shopping results using SerpApi.
icon: cart-shopping
---
# `SerpApiGoogleShoppingTool`
## Description
Leverage `SerpApiGoogleShoppingTool` to query Google Shopping via SerpApi and retrieve product-oriented results.
## Installation
```shell
uv add crewai-tools[serpapi]
```
## Environment Variables
- `SERPAPI_API_KEY` (required): API key for SerpApi. Create one at https://serpapi.com/ (free tier available).
## Example
```python Code
from crewai import Agent, Task, Crew
from crewai_tools import SerpApiGoogleShoppingTool
tool = SerpApiGoogleShoppingTool()
agent = Agent(
role="Shopping Researcher",
goal="Find relevant products",
backstory="Expert in product search",
tools=[tool],
verbose=True,
)
task = Task(
description="Search Google Shopping for 'wireless noise-canceling headphones'",
expected_output="Top relevant products with titles and links",
agent=agent,
)
crew = Crew(agents=[agent], tasks=[task])
result = crew.kickoff()
```
## Notes
- Set `SERPAPI_API_KEY` in the environment. Create a key at https://serpapi.com/
- See also Google Web Search via SerpApi: `/en/tools/search-research/serpapi-googlesearchtool`
## Parameters
### Run Parameters
- `search_query` (str, required): Product search query.
- `location` (str, optional): Geographic location parameter.

View File

@@ -16,7 +16,7 @@ to fetch and display the most relevant search results based on the query provide
To effectively use the `SerperDevTool`, follow these steps:
1. **Package Installation**: Confirm that the `crewai[tools]` package is installed in your Python environment.
2. **API Key Acquisition**: Acquire a `serper.dev` API key at https://serper.dev/ (free tier available).
2. **API Key Acquisition**: Acquire a `serper.dev` API key by registering for a free account at `serper.dev`.
3. **Environment Configuration**: Store your obtained API key in an environment variable named `SERPER_API_KEY` to facilitate its use by the tool.
To incorporate this tool into your project, follow the installation instructions below:

View File

@@ -1,7 +1,7 @@
---
title: "Tavily Extractor Tool"
description: "Extract structured content from web pages using the Tavily API"
icon: square-poll-horizontal
icon: "file-text"
---
The `TavilyExtractorTool` allows CrewAI agents to extract structured content from web pages using the Tavily API. It can process single URLs or lists of URLs and provides options for controlling the extraction depth and including images.

View File

@@ -22,8 +22,6 @@ Ensure your Tavily API key is set as an environment variable:
export TAVILY_API_KEY='your_tavily_api_key'
```
Get an API key at https://app.tavily.com/ (sign up, then create a key).
## Example Usage
Here's how to initialize and use the `TavilySearchTool` within a CrewAI agent:

View File

@@ -1,111 +0,0 @@
---
title: Bright Data Tools
description: Bright Data integrations for SERP search, Web Unlocker scraping, and Dataset API.
icon: spider
---
# Bright Data Tools
This set of tools integrates Bright Data services for web extraction.
## Installation
```shell
uv add crewai-tools requests aiohttp
```
## Environment Variables
- `BRIGHT_DATA_API_KEY` (required)
- `BRIGHT_DATA_ZONE` (for SERP/Web Unlocker)
Create credentials at https://brightdata.com/ (sign up, then create an API token and zone).
See their docs: https://developers.brightdata.com/
## Included Tools
- `BrightDataSearchTool`: SERP search (Google/Bing/Yandex) with geo/language/device options.
- `BrightDataWebUnlockerTool`: Scrape pages with anti-bot bypass and rendering.
- `BrightDataDatasetTool`: Run Dataset API jobs and fetch results.
## Examples
### SERP Search
```python Code
from crewai_tools import BrightDataSearchTool
tool = BrightDataSearchTool(
query="CrewAI",
country="us",
)
print(tool.run())
```
### Web Unlocker
```python Code
from crewai_tools import BrightDataWebUnlockerTool
tool = BrightDataWebUnlockerTool(
url="https://example.com",
format="markdown",
)
print(tool.run(url="https://example.com"))
```
### Dataset API
```python Code
from crewai_tools import BrightDataDatasetTool
tool = BrightDataDatasetTool(
dataset_type="ecommerce",
url="https://example.com/product",
)
print(tool.run())
```
## Troubleshooting
- 401/403: verify `BRIGHT_DATA_API_KEY` and `BRIGHT_DATA_ZONE`.
- Empty/blocked content: enable rendering or try a different zone.
## Example
```python Code
from crewai import Agent, Task, Crew
from crewai_tools import BrightDataSearchTool
tool = BrightDataSearchTool(
query="CrewAI",
country="us",
)
agent = Agent(
role="Web Researcher",
goal="Search with Bright Data",
backstory="Finds reliable results",
tools=[tool],
verbose=True,
)
task = Task(
description="Search for CrewAI and summarize top results",
expected_output="Short summary with links",
agent=agent,
)
crew = Crew(
agents=[agent],
tasks=[task],
verbose=True,
)
result = crew.kickoff()
```

View File

@@ -60,10 +60,6 @@ These tools enable your agents to interact with the web, extract data from websi
<Card title="Oxylabs Scraper Tool" icon="globe" href="/en/tools/web-scraping/oxylabsscraperstool">
Access web data at scale with Oxylabs.
</Card>
<Card title="Bright Data Tools" icon="spider" href="/en/tools/web-scraping/brightdata-tools">
SERP search, Web Unlocker, and Dataset API integrations.
</Card>
</CardGroup>
## **Common Use Cases**

View File

@@ -14,7 +14,6 @@ from crewai.memory.storage.kickoff_task_outputs_storage import (
from .authentication.main import AuthenticationCommand
from .deploy.main import DeployCommand
from .enterprise.main import EnterpriseConfigureCommand
from .evaluate_crew import evaluate_crew
from .install_crew import install_crew
from .kickoff_flow import kickoff_flow
@@ -393,20 +392,6 @@ def current():
org_command.current()
@crewai.group()
def enterprise():
"""Enterprise Configuration commands."""
pass
@enterprise.command("configure")
@click.argument("enterprise_url")
def enterprise_configure(enterprise_url: str):
"""Configure CrewAI Enterprise OAuth2 settings from the provided Enterprise URL."""
enterprise_command = EnterpriseConfigureCommand()
enterprise_command.configure(enterprise_url)
@crewai.group()
def config():
"""CLI Configuration commands."""

View File

@@ -1,84 +0,0 @@
import requests
from typing import Dict, Any
from rich.console import Console
from requests.exceptions import RequestException, JSONDecodeError
from crewai.cli.command import BaseCommand
from crewai.cli.settings.main import SettingsCommand
from crewai.cli.version import get_crewai_version
console = Console()
class EnterpriseConfigureCommand(BaseCommand):
def __init__(self):
super().__init__()
self.settings_command = SettingsCommand()
def configure(self, enterprise_url: str) -> None:
try:
enterprise_url = enterprise_url.rstrip('/')
oauth_config = self._fetch_oauth_config(enterprise_url)
self._update_oauth_settings(enterprise_url, oauth_config)
console.print(
f"✅ Successfully configured CrewAI Enterprise with OAuth2 settings from {enterprise_url}",
style="bold green"
)
except Exception as e:
console.print(f"❌ Failed to configure Enterprise settings: {str(e)}", style="bold red")
raise SystemExit(1)
def _fetch_oauth_config(self, enterprise_url: str) -> Dict[str, Any]:
oauth_endpoint = f"{enterprise_url}/auth/parameters"
try:
console.print(f"🔄 Fetching OAuth2 configuration from {oauth_endpoint}...")
headers = {
"Content-Type": "application/json",
"User-Agent": f"CrewAI-CLI/{get_crewai_version()}",
"X-Crewai-Version": get_crewai_version(),
}
response = requests.get(oauth_endpoint, timeout=30, headers=headers)
response.raise_for_status()
try:
oauth_config = response.json()
except JSONDecodeError:
raise ValueError(f"Invalid JSON response from {oauth_endpoint}")
required_fields = ['audience', 'domain', 'device_authorization_client_id', 'provider']
missing_fields = [field for field in required_fields if field not in oauth_config]
if missing_fields:
raise ValueError(f"Missing required fields in OAuth2 configuration: {', '.join(missing_fields)}")
console.print("✅ Successfully retrieved OAuth2 configuration", style="green")
return oauth_config
except RequestException as e:
raise ValueError(f"Failed to connect to enterprise URL: {str(e)}")
except Exception as e:
raise ValueError(f"Error fetching OAuth2 configuration: {str(e)}")
def _update_oauth_settings(self, enterprise_url: str, oauth_config: Dict[str, Any]) -> None:
try:
config_mapping = {
'enterprise_base_url': enterprise_url,
'oauth2_provider': oauth_config['provider'],
'oauth2_audience': oauth_config['audience'],
'oauth2_client_id': oauth_config['device_authorization_client_id'],
'oauth2_domain': oauth_config['domain']
}
console.print("🔄 Updating local OAuth2 configuration...")
for key, value in config_mapping.items():
self.settings_command.set(key, value)
console.print(f" ✓ Set {key}: {value}", style="dim")
except Exception as e:
raise ValueError(f"Failed to update OAuth2 settings: {str(e)}")

View File

@@ -1,3 +1,4 @@
import os
import asyncio
import json
import re
@@ -77,7 +78,6 @@ from crewai.utilities.events.listeners.tracing.trace_listener import (
)
from crewai.utilities.events.listeners.tracing.utils import is_tracing_enabled
from crewai.utilities.formatter import (
aggregate_raw_outputs_from_task_outputs,
aggregate_raw_outputs_from_tasks,
@@ -248,10 +248,6 @@ class Crew(FlowTrackable, BaseModel):
default=None,
description="Metrics for the LLM usage during all tasks execution.",
)
tracing: Optional[bool] = Field(
default=False,
description="Whether to enable tracing for the crew.",
)
@field_validator("id", mode="before")
@classmethod
@@ -283,8 +279,8 @@ class Crew(FlowTrackable, BaseModel):
self._cache_handler = CacheHandler()
event_listener = EventListener()
if is_tracing_enabled() or self.tracing:
trace_listener = TraceCollectionListener(tracing=self.tracing)
if os.getenv("CREWAI_TRACING_ENABLED", "false").lower() == "true":
trace_listener = TraceCollectionListener()
trace_listener.setup_listeners(crewai_event_bus)
event_listener.verbose = self.verbose
event_listener.formatter.verbose = self.verbose

View File

@@ -2,6 +2,7 @@ import asyncio
import copy
import inspect
import logging
import os
from typing import (
Any,
Callable,
@@ -35,7 +36,6 @@ from crewai.utilities.events.flow_events import (
from crewai.utilities.events.listeners.tracing.trace_listener import (
TraceCollectionListener,
)
from crewai.utilities.events.listeners.tracing.utils import is_tracing_enabled
from crewai.utilities.printer import Printer
logger = logging.getLogger(__name__)
@@ -441,7 +441,6 @@ class Flow(Generic[T], metaclass=FlowMeta):
_router_paths: Dict[str, List[str]] = {}
initial_state: Union[Type[T], T, None] = None
name: Optional[str] = None
tracing: Optional[bool] = False
def __class_getitem__(cls: Type["Flow"], item: Type[T]) -> Type["Flow"]:
class _FlowGeneric(cls): # type: ignore
@@ -453,7 +452,6 @@ class Flow(Generic[T], metaclass=FlowMeta):
def __init__(
self,
persistence: Optional[FlowPersistence] = None,
tracing: Optional[bool] = False,
**kwargs: Any,
) -> None:
"""Initialize a new Flow instance.
@@ -471,9 +469,8 @@ class Flow(Generic[T], metaclass=FlowMeta):
# Initialize state with initial values
self._state = self._create_initial_state()
self.tracing = tracing
if is_tracing_enabled() or tracing:
trace_listener = TraceCollectionListener(tracing=tracing)
if os.getenv("CREWAI_TRACING_ENABLED", "false").lower() == "true":
trace_listener = TraceCollectionListener()
trace_listener.setup_listeners(crewai_event_bus)
# Apply any additional kwargs
if kwargs:

View File

@@ -525,16 +525,7 @@ class LiteAgent(FlowTrackable, BaseModel):
self,
event=LLMCallStartedEvent(
messages=self._messages,
tools=[
{
"name": tool.name,
"description": tool.description,
"args": tool.args,
}
for tool in self._parsed_tools
]
if self._parsed_tools
else None,
tools=None,
callbacks=self._callbacks,
from_agent=self,
model=model,

View File

@@ -0,0 +1,33 @@
import json
from datetime import datetime
from crewai.cli.plus_api import PlusAPI
from crewai.cli.authentication.token import get_auth_token
from pydantic import BaseModel
from .trace_batch_manager import TraceBatch
from logging import getLogger
logger = getLogger(__name__)
class TraceSender(BaseModel):
"""Trace sender for sending trace batches to the backend"""
def send_batch(self, batch: TraceBatch) -> bool:
"""Print trace batch to console"""
try:
payload = batch.to_dict()
def datetime_handler(obj):
if isinstance(obj, datetime):
return obj.isoformat()
serialized_payload = json.loads(
json.dumps(payload, default=datetime_handler)
)
PlusAPI(api_key=get_auth_token()).send_trace_batch(serialized_payload)
return True
except Exception as e:
logger.error(f"Error sending trace batch: {e}")
return False

View File

@@ -66,7 +66,6 @@ class TraceBatchManager:
self, user_context: Dict[str, str], execution_metadata: Dict[str, Any]
):
"""Send batch initialization to backend"""
if not self.plus_api or not self.current_batch:
return
@@ -76,8 +75,8 @@ class TraceBatchManager:
"execution_type": execution_metadata.get("execution_type", "crew"),
"execution_context": {
"crew_fingerprint": execution_metadata.get("crew_fingerprint"),
"crew_name": execution_metadata.get("crew_name", None),
"flow_name": execution_metadata.get("flow_name", None),
"crew_name": execution_metadata.get("crew_name", "Unknown Crew"),
"flow_name": execution_metadata.get("flow_name", "Unknown Flow"),
"crewai_version": self.current_batch.version,
"privacy_level": user_context.get("privacy_level", "standard"),
},

View File

@@ -13,7 +13,6 @@ from crewai.utilities.events.agent_events import (
AgentExecutionErrorEvent,
)
from crewai.utilities.events.listeners.tracing.types import TraceEvent
from crewai.utilities.events.listeners.tracing.utils import is_tracing_enabled
from crewai.utilities.events.reasoning_events import (
AgentReasoningStartedEvent,
AgentReasoningCompletedEvent,
@@ -66,7 +65,7 @@ from crewai.utilities.events.memory_events import (
MemorySaveCompletedEvent,
MemorySaveFailedEvent,
)
from .interfaces import TraceSender
from crewai.cli.authentication.token import get_auth_token
from crewai.cli.version import get_crewai_version
@@ -76,13 +75,13 @@ class TraceCollectionListener(BaseEventListener):
Trace collection listener that orchestrates trace collection
"""
trace_enabled: Optional[bool] = False
trace_enabled: bool = False
complex_events = ["task_started", "llm_call_started", "llm_call_completed"]
_instance = None
_initialized = False
def __new__(cls, batch_manager=None, tracing: Optional[bool] = False):
def __new__(cls, batch_manager=None, trace_sender=None):
if cls._instance is None:
cls._instance = super().__new__(cls)
return cls._instance
@@ -90,14 +89,14 @@ class TraceCollectionListener(BaseEventListener):
def __init__(
self,
batch_manager: Optional[TraceBatchManager] = None,
tracing: Optional[bool] = False,
trace_sender: Optional[TraceSender] = None,
):
if self._initialized:
return
super().__init__()
self.batch_manager = batch_manager or TraceBatchManager()
self.tracing = tracing or False
self.trace_sender = trace_sender or TraceSender()
self.trace_enabled = self._check_trace_enabled()
self._initialized = True
@@ -107,7 +106,9 @@ class TraceCollectionListener(BaseEventListener):
if not auth_token:
return False
return is_tracing_enabled() or self.tracing
return os.getenv("CREWAI_TRACING_ENABLED", "false").lower() == "true" or bool(
os.getenv("CREWAI_USER_TOKEN")
)
def _get_user_context(self) -> Dict[str, str]:
"""Extract user context for tracing"""
@@ -155,7 +156,7 @@ class TraceCollectionListener(BaseEventListener):
@event_bus.on(FlowFinishedEvent)
def on_flow_finished(source, event):
self._handle_trace_event("flow_finished", source, event)
self.batch_manager.finalize_batch()
self._send_batch()
@event_bus.on(FlowPlotEvent)
def on_flow_plot(source, event):
@@ -173,12 +174,12 @@ class TraceCollectionListener(BaseEventListener):
@event_bus.on(CrewKickoffCompletedEvent)
def on_crew_completed(source, event):
self._handle_trace_event("crew_kickoff_completed", source, event)
self.batch_manager.finalize_batch()
self._send_batch()
@event_bus.on(CrewKickoffFailedEvent)
def on_crew_failed(source, event):
self._handle_trace_event("crew_kickoff_failed", source, event)
self.batch_manager.finalize_batch()
self._send_batch()
@event_bus.on(TaskStartedEvent)
def on_task_started(source, event):
@@ -302,7 +303,7 @@ class TraceCollectionListener(BaseEventListener):
"""Initialize trace batch for Flow execution"""
user_context = self._get_user_context()
execution_metadata = {
"flow_name": getattr(event, "flow_name", "Unknown Flow"),
"flow_name": getattr(source, "__class__.__name__", "Unknown Flow"),
"execution_start": event.timestamp if hasattr(event, "timestamp") else None,
"crewai_version": get_crewai_version(),
"execution_type": "flow",
@@ -331,6 +332,14 @@ class TraceCollectionListener(BaseEventListener):
trace_event = self._create_trace_event(event_type, source, event)
self.batch_manager.add_event(trace_event)
def _send_batch(self):
"""Send finalized batch using the configured sender"""
batch = self.batch_manager.finalize_batch()
if batch:
success = self.trace_sender.send_batch(batch)
if not success:
print("⚠️ Failed to send trace batch")
def _create_trace_event(
self, event_type: str, source: Any, event: Any
) -> TraceEvent:
@@ -351,15 +360,20 @@ class TraceCollectionListener(BaseEventListener):
elif event_type == "task_started":
return {
"task_description": event.task.description,
"expected_output": event.task.expected_output,
"task_name": event.task.name,
"context": event.context,
"agent": source.agent.role,
}
elif event_type == "llm_call_started":
return self._safe_serialize_to_dict(event)
return {
**self._safe_serialize_to_dict(event),
"messages": self._truncate_messages(event.messages),
}
elif event_type == "llm_call_completed":
return self._safe_serialize_to_dict(event)
return {
**self._safe_serialize_to_dict(event),
"messages": self._truncate_messages(event.messages),
}
else:
return {
"event_type": event_type,
@@ -382,7 +396,7 @@ class TraceCollectionListener(BaseEventListener):
return {"serialization_error": str(e), "object_type": type(obj).__name__}
# TODO: move to utils
def _truncate_messages(self, messages, max_content_length=500, max_messages=5):
def _truncate_messages(self, messages, max_content_length=200, max_messages=5):
"""Truncate message content and limit number of messages"""
if not messages or not isinstance(messages, list):
return messages

View File

@@ -1,5 +0,0 @@
import os
def is_tracing_enabled() -> bool:
return os.getenv("CREWAI_TRACING_ENABLED", "false").lower() == "true"

View File

@@ -1,125 +0,0 @@
interactions:
- request:
body: '{"messages": [{"role": "system", "content": "You are Test Agent. Test backstory\nYour
personal goal is: Test goal\nTo give my best complete final answer to the task
respond using the exact following format:\n\nThought: I now can give a great
answer\nFinal Answer: Your final answer must be the great and the most complete
as possible, it must be outcome described.\n\nI MUST use these formats, my job
depends on it!"}, {"role": "user", "content": "\nCurrent Task: Say hello to
the world\n\nThis is the expected criteria for your final answer: hello world\nyou
MUST return the actual complete content as the final answer, not a summary.\n\nBegin!
This is VERY important to you, use the tools available and give your best Final
Answer, your job depends on it!\n\nThought:"}], "model": "gpt-4o", "stop": ["\nObservation:"]}'
headers:
accept:
- application/json
accept-encoding:
- gzip, deflate, zstd
connection:
- keep-alive
content-length:
- '820'
content-type:
- application/json
cookie:
- _cfuvid=NaXWifUGChHp6Ap1mvfMrNzmO4HdzddrqXkSR9T.hYo-1754508545647-0.0.1.1-604800000
host:
- api.openai.com
user-agent:
- OpenAI/Python 1.93.0
x-stainless-arch:
- arm64
x-stainless-async:
- 'false'
x-stainless-lang:
- python
x-stainless-os:
- MacOS
x-stainless-package-version:
- 1.93.0
x-stainless-raw-response:
- 'true'
x-stainless-read-timeout:
- '600.0'
x-stainless-retry-count:
- '0'
x-stainless-runtime:
- CPython
x-stainless-runtime-version:
- 3.12.9
method: POST
uri: https://api.openai.com/v1/chat/completions
response:
body:
string: !!binary |
H4sIAAAAAAAAAwAAAP//jFJdj9MwEHzPr1j5uUFp6DUhbycE4g6eKEic4BS59ibx4Xgt27mCTv3v
yEmvSfmQeImUnZ3xzO4+JQBMSVYBEx0Porc6fZ3nH2725cc3O1Pevnv/+e5OPFx/2mW7wd5+YavI
oP0DivDMeiGotxqDIjPBwiEPGFXXxdVmm623r7Yj0JNEHWmtDemG0jzLN2lWptn2ROxICfSsgq8J
AMDT+I0WjcQfrIJs9Vzp0XveIqvOTQDMkY4Vxr1XPnAT2GoGBZmAZnR9A4YOILiBVj0icGijY+DG
H9ABfDNvleEarsf/CjrUmuBATsuloMNm8DzmMYPWC4AbQ4HHeYxR7k/I8WxeU2sd7f1vVNYoo3xX
O+SeTDTqA1k2oscE4H4c0nCRm1lHvQ11oO84Pre+KiY9Nq9lgb48gYEC14t6cRrtpV4tMXCl/WLM
THDRoZyp8074IBUtgGSR+k83f9OekivT/o/8DAiBNqCsrUOpxGXiuc1hvNp/tZ2nPBpmHt2jElgH
hS5uQmLDBz0dFPM/fcC+bpRp0VmnpqtqbJ0VZbHGnMuSJcfkFwAAAP//AwBXeOIeXgMAAA==
headers:
CF-RAY:
- 96b9d31b89dc1684-SJC
Connection:
- keep-alive
Content-Encoding:
- gzip
Content-Type:
- application/json
Date:
- Thu, 07 Aug 2025 21:21:37 GMT
Server:
- cloudflare
Set-Cookie:
- __cf_bm=nQuY9yOahy.xg.aHkxwJgC5gyX5c9Xjbhp3Y7GMX4Ek-1754601697-1.0.1.1-_K22zHDSq5PrNEgK7qwpgcjPitPpgoT54GksNiq6j.aSPasbC7UakO3AYT59smUo5j14NY_OrHkDhm.eGIdpUTpnoJZK7MfR7X8Z96FITGs;
path=/; expires=Thu, 07-Aug-25 21:51:37 GMT; domain=.api.openai.com; HttpOnly;
Secure; SameSite=None
Strict-Transport-Security:
- max-age=31536000; includeSubDomains; preload
Transfer-Encoding:
- chunked
X-Content-Type-Options:
- nosniff
access-control-expose-headers:
- X-Request-ID
alt-svc:
- h3=":443"; ma=86400
cf-cache-status:
- DYNAMIC
openai-organization:
- crewai-iuxna1
openai-processing-ms:
- '424'
openai-project:
- proj_xitITlrFeen7zjNSzML82h9x
openai-version:
- '2020-10-01'
x-envoy-upstream-service-time:
- '438'
x-ratelimit-limit-project-tokens:
- '30000000'
x-ratelimit-limit-requests:
- '10000'
x-ratelimit-limit-tokens:
- '30000000'
x-ratelimit-remaining-project-tokens:
- '29999828'
x-ratelimit-remaining-requests:
- '9999'
x-ratelimit-remaining-tokens:
- '29999828'
x-ratelimit-reset-project-tokens:
- 0s
x-ratelimit-reset-requests:
- 6ms
x-ratelimit-reset-tokens:
- 0s
x-request-id:
- req_6dbb2a6c9a864480b8d1584ae0f51fad
status:
code: 200
message: OK
version: 1

View File

@@ -1,151 +0,0 @@
import tempfile
import unittest
from pathlib import Path
from unittest.mock import Mock, patch
import requests
from requests.exceptions import JSONDecodeError
from crewai.cli.enterprise.main import EnterpriseConfigureCommand
from crewai.cli.settings.main import SettingsCommand
import shutil
class TestEnterpriseConfigureCommand(unittest.TestCase):
def setUp(self):
self.test_dir = Path(tempfile.mkdtemp())
self.config_path = self.test_dir / "settings.json"
with patch('crewai.cli.enterprise.main.SettingsCommand') as mock_settings_command_class:
self.mock_settings_command = Mock(spec=SettingsCommand)
mock_settings_command_class.return_value = self.mock_settings_command
self.enterprise_command = EnterpriseConfigureCommand()
def tearDown(self):
shutil.rmtree(self.test_dir)
@patch('crewai.cli.enterprise.main.requests.get')
@patch('crewai.cli.enterprise.main.get_crewai_version')
def test_successful_configuration(self, mock_get_version, mock_requests_get):
mock_get_version.return_value = "1.0.0"
mock_response = Mock()
mock_response.status_code = 200
mock_response.raise_for_status.return_value = None
mock_response.json.return_value = {
'audience': 'test_audience',
'domain': 'test.domain.com',
'device_authorization_client_id': 'test_client_id',
'provider': 'workos'
}
mock_requests_get.return_value = mock_response
enterprise_url = "https://enterprise.example.com"
self.enterprise_command.configure(enterprise_url)
expected_headers = {
"Content-Type": "application/json",
"User-Agent": "CrewAI-CLI/1.0.0",
"X-Crewai-Version": "1.0.0",
}
mock_requests_get.assert_called_once_with(
"https://enterprise.example.com/auth/parameters",
timeout=30,
headers=expected_headers
)
expected_calls = [
('enterprise_base_url', 'https://enterprise.example.com'),
('oauth2_provider', 'workos'),
('oauth2_audience', 'test_audience'),
('oauth2_client_id', 'test_client_id'),
('oauth2_domain', 'test.domain.com')
]
actual_calls = self.mock_settings_command.set.call_args_list
self.assertEqual(len(actual_calls), 5)
for i, (key, value) in enumerate(expected_calls):
call_args = actual_calls[i][0]
self.assertEqual(call_args[0], key)
self.assertEqual(call_args[1], value)
@patch('crewai.cli.enterprise.main.requests.get')
@patch('crewai.cli.enterprise.main.get_crewai_version')
def test_http_error_handling(self, mock_get_version, mock_requests_get):
mock_get_version.return_value = "1.0.0"
mock_response = Mock()
mock_response.raise_for_status.side_effect = requests.HTTPError("404 Not Found")
mock_requests_get.return_value = mock_response
with self.assertRaises(SystemExit):
self.enterprise_command.configure("https://enterprise.example.com")
@patch('crewai.cli.enterprise.main.requests.get')
@patch('crewai.cli.enterprise.main.get_crewai_version')
def test_invalid_json_response(self, mock_get_version, mock_requests_get):
mock_get_version.return_value = "1.0.0"
mock_response = Mock()
mock_response.status_code = 200
mock_response.raise_for_status.return_value = None
mock_response.json.side_effect = JSONDecodeError("Invalid JSON", "", 0)
mock_requests_get.return_value = mock_response
with self.assertRaises(SystemExit):
self.enterprise_command.configure("https://enterprise.example.com")
@patch('crewai.cli.enterprise.main.requests.get')
@patch('crewai.cli.enterprise.main.get_crewai_version')
def test_missing_required_fields(self, mock_get_version, mock_requests_get):
mock_get_version.return_value = "1.0.0"
mock_response = Mock()
mock_response.status_code = 200
mock_response.raise_for_status.return_value = None
mock_response.json.return_value = {
'audience': 'test_audience',
}
mock_requests_get.return_value = mock_response
with self.assertRaises(SystemExit):
self.enterprise_command.configure("https://enterprise.example.com")
@patch('crewai.cli.enterprise.main.requests.get')
@patch('crewai.cli.enterprise.main.get_crewai_version')
def test_settings_update_error(self, mock_get_version, mock_requests_get):
mock_get_version.return_value = "1.0.0"
mock_response = Mock()
mock_response.status_code = 200
mock_response.raise_for_status.return_value = None
mock_response.json.return_value = {
'audience': 'test_audience',
'domain': 'test.domain.com',
'device_authorization_client_id': 'test_client_id',
'provider': 'workos'
}
mock_requests_get.return_value = mock_response
self.mock_settings_command.set.side_effect = Exception("Settings update failed")
with self.assertRaises(SystemExit):
self.enterprise_command.configure("https://enterprise.example.com")
def test_url_trailing_slash_removal(self):
with patch.object(self.enterprise_command, '_fetch_oauth_config') as mock_fetch, \
patch.object(self.enterprise_command, '_update_oauth_settings') as mock_update:
mock_fetch.return_value = {
'audience': 'test_audience',
'domain': 'test.domain.com',
'device_authorization_client_id': 'test_client_id',
'provider': 'workos'
}
self.enterprise_command.configure("https://enterprise.example.com/")
mock_fetch.assert_called_once_with("https://enterprise.example.com")
mock_update.assert_called_once()

View File

@@ -10,7 +10,6 @@ from crewai.flow import Flow, start
from crewai.lite_agent import LiteAgent, LiteAgentOutput
from crewai.tools import BaseTool
from crewai.utilities.events import crewai_event_bus
from crewai.utilities.events.llm_events import LLMCallStartedEvent
from crewai.utilities.events.agent_events import LiteAgentExecutionStartedEvent
from crewai.utilities.events.tool_usage_events import ToolUsageStartedEvent
from crewai.llms.base_llm import BaseLLM
@@ -149,12 +148,12 @@ def test_lite_agent_with_tools():
"What is the population of Tokyo and how many people would that be per square kilometer if Tokyo's area is 2,194 square kilometers?"
)
assert "21 million" in result.raw or "37 million" in result.raw, (
"Agent should find Tokyo's population"
)
assert "per square kilometer" in result.raw, (
"Agent should calculate population density"
)
assert (
"21 million" in result.raw or "37 million" in result.raw
), "Agent should find Tokyo's population"
assert (
"per square kilometer" in result.raw
), "Agent should calculate population density"
received_events = []
@@ -319,17 +318,11 @@ def test_sets_parent_flow_when_inside_flow():
flow.kickoff()
assert captured_agent.parent_flow is flow
@pytest.mark.vcr(filter_headers=["authorization"])
def test_guardrail_is_called_using_string():
guardrail_events = defaultdict(list)
from crewai.utilities.events import (
LLMGuardrailCompletedEvent,
LLMGuardrailStartedEvent,
)
from crewai.utilities.events import LLMGuardrailCompletedEvent, LLMGuardrailStartedEvent
with crewai_event_bus.scoped_handlers():
@crewai_event_bus.on(LLMGuardrailStartedEvent)
def capture_guardrail_started(source, event):
guardrail_events["started"].append(event)
@@ -347,26 +340,17 @@ def test_guardrail_is_called_using_string():
result = agent.kickoff(messages="Top 10 best players in the world?")
assert len(guardrail_events["started"]) == 2
assert len(guardrail_events["completed"]) == 2
assert not guardrail_events["completed"][0].success
assert guardrail_events["completed"][1].success
assert (
"Here are the top 10 best soccer players in the world, focusing exclusively on Brazilian players"
in result.raw
)
assert len(guardrail_events['started']) == 2
assert len(guardrail_events['completed']) == 2
assert not guardrail_events['completed'][0].success
assert guardrail_events['completed'][1].success
assert "Here are the top 10 best soccer players in the world, focusing exclusively on Brazilian players" in result.raw
@pytest.mark.vcr(filter_headers=["authorization"])
def test_guardrail_is_called_using_callable():
guardrail_events = defaultdict(list)
from crewai.utilities.events import (
LLMGuardrailCompletedEvent,
LLMGuardrailStartedEvent,
)
from crewai.utilities.events import LLMGuardrailCompletedEvent, LLMGuardrailStartedEvent
with crewai_event_bus.scoped_handlers():
@crewai_event_bus.on(LLMGuardrailStartedEvent)
def capture_guardrail_started(source, event):
guardrail_events["started"].append(event)
@@ -384,22 +368,16 @@ def test_guardrail_is_called_using_callable():
result = agent.kickoff(messages="Top 1 best players in the world?")
assert len(guardrail_events["started"]) == 1
assert len(guardrail_events["completed"]) == 1
assert guardrail_events["completed"][0].success
assert len(guardrail_events['started']) == 1
assert len(guardrail_events['completed']) == 1
assert guardrail_events['completed'][0].success
assert "Pelé - Santos, 1958" in result.raw
@pytest.mark.vcr(filter_headers=["authorization"])
def test_guardrail_reached_attempt_limit():
guardrail_events = defaultdict(list)
from crewai.utilities.events import (
LLMGuardrailCompletedEvent,
LLMGuardrailStartedEvent,
)
from crewai.utilities.events import LLMGuardrailCompletedEvent, LLMGuardrailStartedEvent
with crewai_event_bus.scoped_handlers():
@crewai_event_bus.on(LLMGuardrailStartedEvent)
def capture_guardrail_started(source, event):
guardrail_events["started"].append(event)
@@ -412,23 +390,18 @@ def test_guardrail_reached_attempt_limit():
role="Sports Analyst",
goal="Gather information about the best soccer players",
backstory="""You are an expert at gathering and organizing information. You carefully collect details and present them in a structured way.""",
guardrail=lambda output: (
False,
"You are not allowed to include Brazilian players",
),
guardrail=lambda output: (False, "You are not allowed to include Brazilian players"),
guardrail_max_retries=2,
)
with pytest.raises(
Exception, match="Agent's guardrail failed validation after 2 retries"
):
with pytest.raises(Exception, match="Agent's guardrail failed validation after 2 retries"):
agent.kickoff(messages="Top 10 best players in the world?")
assert len(guardrail_events["started"]) == 3 # 2 retries + 1 initial call
assert len(guardrail_events["completed"]) == 3 # 2 retries + 1 initial call
assert not guardrail_events["completed"][0].success
assert not guardrail_events["completed"][1].success
assert not guardrail_events["completed"][2].success
assert len(guardrail_events['started']) == 3 # 2 retries + 1 initial call
assert len(guardrail_events['completed']) == 3 # 2 retries + 1 initial call
assert not guardrail_events['completed'][0].success
assert not guardrail_events['completed'][1].success
assert not guardrail_events['completed'][2].success
@pytest.mark.vcr(filter_headers=["authorization"])
@@ -441,35 +414,22 @@ def test_agent_output_when_guardrail_returns_base_model():
role="Sports Analyst",
goal="Gather information about the best soccer players",
backstory="""You are an expert at gathering and organizing information. You carefully collect details and present them in a structured way.""",
guardrail=lambda output: (
True,
Player(name="Lionel Messi", country="Argentina"),
),
guardrail=lambda output: (True, Player(name="Lionel Messi", country="Argentina")),
)
result = agent.kickoff(messages="Top 10 best players in the world?")
assert result.pydantic == Player(name="Lionel Messi", country="Argentina")
def test_lite_agent_with_custom_llm_and_guardrails():
"""Test that CustomLLM (inheriting from BaseLLM) works with guardrails."""
class CustomLLM(BaseLLM):
def __init__(self, response: str = "Custom response"):
super().__init__(model="custom-model")
self.response = response
self.call_count = 0
def call(
self,
messages,
tools=None,
callbacks=None,
available_functions=None,
from_task=None,
from_agent=None,
) -> str:
def call(self, messages, tools=None, callbacks=None, available_functions=None, from_task=None, from_agent=None) -> str:
self.call_count += 1
if "valid" in str(messages) and "feedback" in str(messages):
@@ -496,7 +456,7 @@ def test_lite_agent_with_custom_llm_and_guardrails():
goal="Analyze soccer players",
backstory="You analyze soccer players and their performance.",
llm=custom_llm,
guardrail="Only include Brazilian players",
guardrail="Only include Brazilian players"
)
result = agent.kickoff("Tell me about the best soccer players")
@@ -514,7 +474,7 @@ def test_lite_agent_with_custom_llm_and_guardrails():
goal="Test goal",
backstory="Test backstory",
llm=custom_llm2,
guardrail=test_guardrail,
guardrail=test_guardrail
)
result2 = agent2.kickoff("Test message")
@@ -524,96 +484,12 @@ def test_lite_agent_with_custom_llm_and_guardrails():
@pytest.mark.vcr(filter_headers=["authorization"])
def test_lite_agent_with_invalid_llm():
"""Test that LiteAgent raises proper error when create_llm returns None."""
with patch("crewai.lite_agent.create_llm", return_value=None):
with patch('crewai.lite_agent.create_llm', return_value=None):
with pytest.raises(ValueError) as exc_info:
LiteAgent(
role="Test Agent",
goal="Test goal",
goal="Test goal",
backstory="Test backstory",
llm="invalid-model",
llm="invalid-model"
)
assert "Expected LLM instance of type BaseLLM" in str(exc_info.value)
def test_lite_agent_emits_llm_call_started_event_with_tools():
"""Test that LiteAgent emits LLMCallStartedEvent with correct tools information."""
captured_events = []
@crewai_event_bus.on(LLMCallStartedEvent)
def capture_llm_event(source, event):
captured_events.append(event)
agent = LiteAgent(
role="Test Agent",
goal="Test Goal",
backstory="Test Backstory",
tools=[WebSearchTool(), CalculatorTool()],
llm=Mock(spec=LLM),
)
agent.llm.call.return_value = "Final Answer: Test response"
agent.kickoff("Test query")
# Verify event was emitted with tools
assert len(captured_events) > 0
event = captured_events[0]
assert event.tools is not None
assert len(event.tools) == 2
# Verify tool structure
tool_names = [tool["name"] for tool in event.tools]
assert "search_web" in tool_names
assert "calculate" in tool_names
for tool in event.tools:
assert "name" in tool
assert "description" in tool
assert "args" in tool
def test_lite_agent_emits_llm_call_started_event_without_tools():
"""Test that LiteAgent emits LLMCallStartedEvent with tools=None when no tools provided."""
captured_events = []
@crewai_event_bus.on(LLMCallStartedEvent)
def capture_llm_event(source, event):
captured_events.append(event)
agent = LiteAgent(
role="Test Agent",
goal="Test Goal",
backstory="Test Backstory",
tools=[], # No tools
llm=Mock(spec=LLM),
)
agent.llm.call.return_value = "Final Answer: Test response"
agent.kickoff("Test query")
assert len(captured_events) > 0
event = captured_events[0]
assert event.tools is None
@pytest.mark.vcr(filter_headers=["authorization"])
def test_lite_agent_tool_calling_reproduction():
"""Test reproduction of GitHub issue #3302 - LiteAgent tool calling."""
agent = LiteAgent(
role="Research Assistant",
goal="Help with research tasks",
backstory="You are a helpful research assistant.",
tools=[WebSearchTool()],
llm=LLM(model="gpt-4o-mini"),
)
result = agent.kickoff("Search for information about Python programming")
# Verify the agent produced a meaningful result
assert result.raw is not None
assert len(result.raw) > 0
# Verify tools were actually available during execution
assert len(agent._parsed_tools) > 0
assert "Expected LLM instance of type BaseLLM" in str(exc_info.value)

View File

@@ -33,6 +33,10 @@ class TestTraceListenerSetup:
"crewai.utilities.events.listeners.tracing.trace_batch_manager.get_auth_token",
return_value="mock_token_12345",
),
patch(
"crewai.utilities.events.listeners.tracing.interfaces.get_auth_token",
return_value="mock_token_12345",
),
):
yield
@@ -292,21 +296,7 @@ class TestTraceListenerSetup:
assert trace_listener.trace_enabled is True
assert trace_listener.batch_manager is not None
@pytest.mark.vcr(filter_headers=["authorization"])
def test_trace_listener_setup_correctly_with_tracing_flag(self):
"""Test that trace listener is set up correctly when enabled"""
agent = Agent(role="Test Agent", goal="Test goal", backstory="Test backstory")
task = Task(
description="Say hello to the world",
expected_output="hello world",
agent=agent,
)
crew = Crew(agents=[agent], tasks=[task], verbose=True, tracing=True)
crew.kickoff()
trace_listener = TraceCollectionListener(tracing=True)
assert trace_listener.trace_enabled is True
assert trace_listener.batch_manager is not None
assert trace_listener.trace_sender is not None
# Helper method to ensure cleanup
def teardown_method(self):