Compare commits
9 Commits
devin/1750
...
lg-mcp-cre
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
4a7c21f0e7 | ||
|
|
060c486948 | ||
|
|
8b176d0598 | ||
|
|
c96d4a6823 | ||
|
|
59032817c7 | ||
|
|
e9d8a853ea | ||
|
|
463ea2b97f | ||
|
|
ec2903e5ee | ||
|
|
4364585ebc |
45
.github/workflows/mkdocs.yml
vendored
@@ -1,45 +0,0 @@
|
||||
name: Deploy MkDocs
|
||||
|
||||
on:
|
||||
release:
|
||||
types: [published]
|
||||
|
||||
permissions:
|
||||
contents: write
|
||||
|
||||
jobs:
|
||||
deploy:
|
||||
runs-on: ubuntu-latest
|
||||
|
||||
steps:
|
||||
- name: Checkout code
|
||||
uses: actions/checkout@v4
|
||||
|
||||
- name: Setup Python
|
||||
uses: actions/setup-python@v5
|
||||
with:
|
||||
python-version: '3.10'
|
||||
|
||||
- name: Calculate requirements hash
|
||||
id: req-hash
|
||||
run: echo "::set-output name=hash::$(sha256sum requirements-doc.txt | awk '{print $1}')"
|
||||
|
||||
- name: Setup cache
|
||||
uses: actions/cache@v4
|
||||
with:
|
||||
key: mkdocs-material-${{ steps.req-hash.outputs.hash }}
|
||||
path: .cache
|
||||
restore-keys: |
|
||||
mkdocs-material-
|
||||
|
||||
- name: Install Requirements
|
||||
run: |
|
||||
sudo apt-get update &&
|
||||
sudo apt-get install pngquant &&
|
||||
pip install mkdocs-material mkdocs-material-extensions pillow cairosvg
|
||||
|
||||
env:
|
||||
GH_TOKEN: ${{ secrets.GH_TOKEN }}
|
||||
|
||||
- name: Build and deploy MkDocs
|
||||
run: mkdocs gh-deploy --force
|
||||
@@ -134,7 +134,7 @@
|
||||
"tools/web-scraping/stagehandtool",
|
||||
"tools/web-scraping/firecrawlcrawlwebsitetool",
|
||||
"tools/web-scraping/firecrawlscrapewebsitetool",
|
||||
"tools/web-scraping/firecrawlsearchtool"
|
||||
"tools/web-scraping/oxylabsscraperstool"
|
||||
]
|
||||
},
|
||||
{
|
||||
|
||||
@@ -124,7 +124,7 @@ from crewai_tools import CrewaiEnterpriseTools
|
||||
enterprise_tools = CrewaiEnterpriseTools(
|
||||
actions_list=["gmail_find_email"] # only gmail_find_email tool will be available
|
||||
)
|
||||
gmail_tool = enterprise_tools[0]
|
||||
gmail_tool = enterprise_tools["gmail_find_email"]
|
||||
|
||||
gmail_agent = Agent(
|
||||
role="Gmail Manager",
|
||||
|
||||
BIN
docs/images/crewai_traces.gif
Normal file
|
After Width: | Height: | Size: 10 MiB |
BIN
docs/images/maxim_agent_tracking.png
Normal file
|
After Width: | Height: | Size: 1.3 MiB |
BIN
docs/images/maxim_alerts_1.png
Normal file
|
After Width: | Height: | Size: 1.1 MiB |
BIN
docs/images/maxim_dashboard_1.png
Normal file
|
After Width: | Height: | Size: 617 KiB |
BIN
docs/images/maxim_playground.png
Normal file
|
After Width: | Height: | Size: 1.2 MiB |
BIN
docs/images/maxim_trace_eval.png
Normal file
|
After Width: | Height: | Size: 845 KiB |
BIN
docs/images/maxim_versions.png
Normal file
|
After Width: | Height: | Size: 1.3 MiB |
@@ -6,11 +6,11 @@ icon: plug
|
||||
|
||||
## Overview
|
||||
|
||||
The [Model Context Protocol](https://modelcontextprotocol.io/introduction) (MCP) provides a standardized way for AI agents to provide context to LLMs by communicating with external services, known as MCP Servers.
|
||||
The `crewai-tools` library extends CrewAI's capabilities by allowing you to seamlessly integrate tools from these MCP servers into your agents.
|
||||
This gives your crews access to a vast ecosystem of functionalities.
|
||||
The [Model Context Protocol](https://modelcontextprotocol.io/introduction) (MCP) provides a standardized way for AI agents to provide context to LLMs by communicating with external services, known as MCP Servers.
|
||||
The `crewai-tools` library extends CrewAI's capabilities by allowing you to seamlessly integrate tools from these MCP servers into your agents.
|
||||
This gives your crews access to a vast ecosystem of functionalities.
|
||||
|
||||
We currently support the following transport mechanisms:
|
||||
We currently support the following transport mechanisms:
|
||||
|
||||
- **Stdio**: for local servers (communication via standard input/output between processes on the same machine)
|
||||
- **Server-Sent Events (SSE)**: for remote servers (unidirectional, real-time data streaming from server to client over HTTP)
|
||||
@@ -52,27 +52,27 @@ from mcp import StdioServerParameters # For Stdio Server
|
||||
# Example server_params (choose one based on your server type):
|
||||
# 1. Stdio Server:
|
||||
server_params=StdioServerParameters(
|
||||
command="python3",
|
||||
command="python3",
|
||||
args=["servers/your_server.py"],
|
||||
env={"UV_PYTHON": "3.12", **os.environ},
|
||||
)
|
||||
|
||||
# 2. SSE Server:
|
||||
server_params = {
|
||||
"url": "http://localhost:8000/sse",
|
||||
"url": "http://localhost:8000/sse",
|
||||
"transport": "sse"
|
||||
}
|
||||
|
||||
# 3. Streamable HTTP Server:
|
||||
server_params = {
|
||||
"url": "http://localhost:8001/mcp",
|
||||
"url": "http://localhost:8001/mcp",
|
||||
"transport": "streamable-http"
|
||||
}
|
||||
|
||||
# Example usage (uncomment and adapt once server_params is set):
|
||||
with MCPServerAdapter(server_params) as mcp_tools:
|
||||
print(f"Available tools: {[tool.name for tool in mcp_tools]}")
|
||||
|
||||
|
||||
my_agent = Agent(
|
||||
role="MCP Tool User",
|
||||
goal="Utilize tools from an MCP server.",
|
||||
@@ -85,44 +85,95 @@ with MCPServerAdapter(server_params) as mcp_tools:
|
||||
```
|
||||
This general pattern shows how to integrate tools. For specific examples tailored to each transport, refer to the detailed guides below.
|
||||
|
||||
## Filtering Tools
|
||||
|
||||
```python
|
||||
with MCPServerAdapter(server_params) as mcp_tools:
|
||||
print(f"Available tools: {[tool.name for tool in mcp_tools]}")
|
||||
|
||||
my_agent = Agent(
|
||||
role="MCP Tool User",
|
||||
goal="Utilize tools from an MCP server.",
|
||||
backstory="I can connect to MCP servers and use their tools.",
|
||||
tools=mcp_tools["tool_name"], # Pass the loaded tools to your agent
|
||||
reasoning=True,
|
||||
verbose=True
|
||||
)
|
||||
# ... rest of your crew setup ...
|
||||
```
|
||||
|
||||
## Using with CrewBase
|
||||
|
||||
To use MCPServer tools within a CrewBase class, use the `mcp_tools` method. Server configurations should be provided via the mcp_server_params attribute. You can pass either a single configuration or a list of multiple server configurations.
|
||||
|
||||
```python
|
||||
@CrewBase
|
||||
class CrewWithMCP:
|
||||
# ... define your agents and tasks config file ...
|
||||
|
||||
mcp_server_params = [
|
||||
# Streamable HTTP Server
|
||||
{
|
||||
"url": "http://localhost:8001/mcp",
|
||||
"transport": "streamable-http"
|
||||
},
|
||||
# SSE Server
|
||||
{
|
||||
"url": "http://localhost:8000/sse",
|
||||
"transport": "sse"
|
||||
},
|
||||
# StdIO Server
|
||||
StdioServerParameters(
|
||||
command="python3",
|
||||
args=["servers/your_stdio_server.py"],
|
||||
env={"UV_PYTHON": "3.12", **os.environ},
|
||||
)
|
||||
]
|
||||
|
||||
@agent
|
||||
def your_agent(self):
|
||||
return Agent(config=self.agents_config["your_agent"], tools=self.get_mcp_tools()) # you can filter which tool are available also
|
||||
|
||||
# ... rest of your crew setup ...
|
||||
```
|
||||
## Explore MCP Integrations
|
||||
|
||||
<CardGroup cols={2}>
|
||||
<Card
|
||||
title="Stdio Transport"
|
||||
icon="server"
|
||||
<Card
|
||||
title="Stdio Transport"
|
||||
icon="server"
|
||||
href="/mcp/stdio"
|
||||
color="#3B82F6"
|
||||
>
|
||||
Connect to local MCP servers via standard input/output. Ideal for scripts and local executables.
|
||||
</Card>
|
||||
<Card
|
||||
title="SSE Transport"
|
||||
icon="wifi"
|
||||
<Card
|
||||
title="SSE Transport"
|
||||
icon="wifi"
|
||||
href="/mcp/sse"
|
||||
color="#10B981"
|
||||
>
|
||||
Integrate with remote MCP servers using Server-Sent Events for real-time data streaming.
|
||||
</Card>
|
||||
<Card
|
||||
title="Streamable HTTP Transport"
|
||||
icon="globe"
|
||||
<Card
|
||||
title="Streamable HTTP Transport"
|
||||
icon="globe"
|
||||
href="/mcp/streamable-http"
|
||||
color="#F59E0B"
|
||||
>
|
||||
Utilize flexible Streamable HTTP for robust communication with remote MCP servers.
|
||||
</Card>
|
||||
<Card
|
||||
title="Connecting to Multiple Servers"
|
||||
icon="layer-group"
|
||||
<Card
|
||||
title="Connecting to Multiple Servers"
|
||||
icon="layer-group"
|
||||
href="/mcp/multiple-servers"
|
||||
color="#8B5CF6"
|
||||
>
|
||||
Aggregate tools from several MCP servers simultaneously using a single adapter.
|
||||
</Card>
|
||||
<Card
|
||||
title="Security Considerations"
|
||||
icon="lock"
|
||||
<Card
|
||||
title="Security Considerations"
|
||||
icon="lock"
|
||||
href="/mcp/security"
|
||||
color="#EF4444"
|
||||
>
|
||||
@@ -132,7 +183,7 @@ This general pattern shows how to integrate tools. For specific examples tailore
|
||||
|
||||
Checkout this repository for full demos and examples of MCP integration with CrewAI! 👇
|
||||
|
||||
<Card
|
||||
<Card
|
||||
title="GitHub Repository"
|
||||
icon="github"
|
||||
href="https://github.com/tonykipkemboi/crewai-mcp-demo"
|
||||
@@ -147,7 +198,7 @@ Always ensure that you trust an MCP Server before using it.
|
||||
</Warning>
|
||||
|
||||
#### Security Warning: DNS Rebinding Attacks
|
||||
SSE transports can be vulnerable to DNS rebinding attacks if not properly secured.
|
||||
SSE transports can be vulnerable to DNS rebinding attacks if not properly secured.
|
||||
To prevent this:
|
||||
|
||||
1. **Always validate Origin headers** on incoming SSE connections to ensure they come from expected sources
|
||||
@@ -159,6 +210,6 @@ Without these protections, attackers could use DNS rebinding to interact with lo
|
||||
For more details, see the [Anthropic's MCP Transport Security docs](https://modelcontextprotocol.io/docs/concepts/transports#security-considerations).
|
||||
|
||||
### Limitations
|
||||
* **Supported Primitives**: Currently, `MCPServerAdapter` primarily supports adapting MCP `tools`.
|
||||
* **Supported Primitives**: Currently, `MCPServerAdapter` primarily supports adapting MCP `tools`.
|
||||
Other MCP primitives like `prompts` or `resources` are not directly integrated as CrewAI components through this adapter at this time.
|
||||
* **Output Handling**: The adapter typically processes the primary text output from an MCP tool (e.g., `.content[0].text`). Complex or multi-modal outputs might require custom handling if not fitting this pattern.
|
||||
|
||||
@@ -30,18 +30,29 @@ Set your Langfuse API keys and configure OpenTelemetry export settings to send t
|
||||
|
||||
```python
|
||||
import os
|
||||
import base64
|
||||
|
||||
# Get keys for your project from the project settings page: https://cloud.langfuse.com
|
||||
os.environ["LANGFUSE_PUBLIC_KEY"] = "pk-lf-..."
|
||||
os.environ["LANGFUSE_SECRET_KEY"] = "sk-lf-..."
|
||||
os.environ["LANGFUSE_HOST"] = "https://cloud.langfuse.com" # 🇪🇺 EU region
|
||||
# os.environ["LANGFUSE_HOST"] = "https://us.cloud.langfuse.com" # 🇺🇸 US region
|
||||
|
||||
|
||||
# Your OpenAI key
|
||||
os.environ["OPENAI_API_KEY"] = "sk-proj-..."
|
||||
```
|
||||
With the environment variables set, we can now initialize the Langfuse client. get_client() initializes the Langfuse client using the credentials provided in the environment variables.
|
||||
|
||||
LANGFUSE_PUBLIC_KEY="pk-lf-..."
|
||||
LANGFUSE_SECRET_KEY="sk-lf-..."
|
||||
LANGFUSE_AUTH=base64.b64encode(f"{LANGFUSE_PUBLIC_KEY}:{LANGFUSE_SECRET_KEY}".encode()).decode()
|
||||
|
||||
os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://cloud.langfuse.com/api/public/otel" # EU data region
|
||||
# os.environ["OTEL_EXPORTER_OTLP_ENDPOINT"] = "https://us.cloud.langfuse.com/api/public/otel" # US data region
|
||||
os.environ["OTEL_EXPORTER_OTLP_HEADERS"] = f"Authorization=Basic {LANGFUSE_AUTH}"
|
||||
|
||||
# your openai key
|
||||
os.environ["OPENAI_API_KEY"] = "sk-..."
|
||||
```python
|
||||
from langfuse import get_client
|
||||
|
||||
langfuse = get_client()
|
||||
|
||||
# Verify connection
|
||||
if langfuse.auth_check():
|
||||
print("Langfuse client is authenticated and ready!")
|
||||
else:
|
||||
print("Authentication failed. Please check your credentials and host.")
|
||||
```
|
||||
|
||||
### Step 3: Initialize OpenLit
|
||||
|
||||
@@ -1,28 +1,107 @@
|
||||
---
|
||||
title: Maxim Integration
|
||||
description: Start Agent monitoring, evaluation, and observability
|
||||
icon: bars-staggered
|
||||
title: "Maxim Integration"
|
||||
description: "Start Agent monitoring, evaluation, and observability"
|
||||
icon: "infinity"
|
||||
---
|
||||
|
||||
# Maxim Integration
|
||||
# Maxim Overview
|
||||
|
||||
Maxim AI provides comprehensive agent monitoring, evaluation, and observability for your CrewAI applications. With Maxim's one-line integration, you can easily trace and analyse agent interactions, performance metrics, and more.
|
||||
|
||||
## Features
|
||||
|
||||
## Features: One Line Integration
|
||||
### Prompt Management
|
||||
|
||||
- **End-to-End Agent Tracing**: Monitor the complete lifecycle of your agents
|
||||
- **Performance Analytics**: Track latency, tokens consumed, and costs
|
||||
- **Hyperparameter Monitoring**: View the configuration details of your agent runs
|
||||
- **Tool Call Tracking**: Observe when and how agents use their tools
|
||||
- **Advanced Visualisation**: Understand agent trajectories through intuitive dashboards
|
||||
Maxim's Prompt Management capabilities enable you to create, organize, and optimize prompts for your CrewAI agents. Rather than hardcoding instructions, leverage Maxim’s SDK to dynamically retrieve and apply version-controlled prompts.
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Prompt Playground">
|
||||
Create, refine, experiment and deploy your prompts via the playground. Organize of your prompts using folders and versions, experimenting with the real world cases by linking tools and context, and deploying based on custom logic.
|
||||
|
||||
Easily experiment across models by [**configuring models**](https://www.getmaxim.ai/docs/introduction/quickstart/setting-up-workspace#add-model-api-keys) and selecting the relevant model from the dropdown at the top of the prompt playground.
|
||||
|
||||
<img src='https://raw.githubusercontent.com/akmadan/crewAI/docs_maxim_observability/docs/images/maxim_playground.png'> </img>
|
||||
</Tab>
|
||||
<Tab title="Prompt Versions">
|
||||
As teams build their AI applications, a big part of experimentation is iterating on the prompt structure. In order to collaborate effectively and organize your changes clearly, Maxim allows prompt versioning and comparison runs across versions.
|
||||
|
||||
<img src='https://raw.githubusercontent.com/akmadan/crewAI/docs_maxim_observability/docs/images/maxim_versions.png'> </img>
|
||||
</Tab>
|
||||
<Tab title="Prompt Comparisons">
|
||||
Iterating on Prompts as you evolve your AI application would need experiments across models, prompt structures, etc. In order to compare versions and make informed decisions about changes, the comparison playground allows a side by side view of results.
|
||||
|
||||
## **Why use Prompt comparison?**
|
||||
|
||||
Prompt comparison combines multiple single Prompts into one view, enabling a streamlined approach for various workflows:
|
||||
|
||||
1. **Model comparison**: Evaluate the performance of different models on the same Prompt.
|
||||
2. **Prompt optimization**: Compare different versions of a Prompt to identify the most effective formulation.
|
||||
3. **Cross-Model consistency**: Ensure consistent outputs across various models for the same Prompt.
|
||||
4. **Performance benchmarking**: Analyze metrics like latency, cost, and token count across different models and Prompts.
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
### Observability & Evals
|
||||
|
||||
Maxim AI provides comprehensive observability & evaluation for your CrewAI agents, helping you understand exactly what's happening during each execution.
|
||||
|
||||
<Tabs>
|
||||
<Tab title="Agent Tracing">
|
||||
Track your agent’s complete lifecycle, including tool calls, agent trajectories, and decision flows effortlessly.
|
||||
|
||||
<img src='https://raw.githubusercontent.com/akmadan/crewAI/docs_maxim_observability/docs/images/maxim_agent_tracking.png'> </img>
|
||||
</Tab>
|
||||
<Tab title="Analytics + Evals">
|
||||
Run detailed evaluations on full traces or individual nodes with support for:
|
||||
|
||||
- Multi-step interactions and granular trace analysis
|
||||
- Session Level Evaluations
|
||||
- Simulations for real-world testing
|
||||
|
||||
<img src='https://raw.githubusercontent.com/akmadan/crewAI/docs_maxim_observability/docs/images/maxim_trace_eval.png'> </img>
|
||||
|
||||
<CardGroup cols={3}>
|
||||
<Card title="Auto Evals on Logs" icon="e" href="https://www.getmaxim.ai/docs/observe/how-to/evaluate-logs/auto-evaluation">
|
||||
<p>
|
||||
Evaluate captured logs automatically from the UI based on filters and sampling
|
||||
|
||||
</p>
|
||||
</Card>
|
||||
<Card title="Human Evals on Logs" icon="hand" href="https://www.getmaxim.ai/docs/observe/how-to/evaluate-logs/human-evaluation">
|
||||
<p>
|
||||
Use human evaluation or rating to assess the quality of your logs and evaluate them.
|
||||
|
||||
</p>
|
||||
</Card>
|
||||
<Card title="Node Level Evals" icon="road" href="https://www.getmaxim.ai/docs/observe/how-to/evaluate-logs/node-level-evaluation">
|
||||
<p>
|
||||
Evaluate any component of your trace or log to gain insights into your agent’s behavior.
|
||||
|
||||
</p>
|
||||
</Card>
|
||||
</CardGroup>
|
||||
---
|
||||
</Tab>
|
||||
<Tab title="Alerting">
|
||||
Set thresholds on **error**, **cost, token usage, user feedback, latency** and get real-time alerts via Slack or PagerDuty.
|
||||
|
||||
<img src='https://raw.githubusercontent.com/akmadan/crewAI/docs_maxim_observability/docs/images/maxim_alerts_1.png'> </img>
|
||||
</Tab>
|
||||
<Tab title="Dashboards">
|
||||
Visualize Traces over time, usage metrics, latency & error rates with ease.
|
||||
|
||||
<img src='https://raw.githubusercontent.com/akmadan/crewAI/docs_maxim_observability/docs/images/maxim_dashboard_1.png'> </img>
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
## Getting Started
|
||||
|
||||
### Prerequisites
|
||||
|
||||
- Python version >=3.10
|
||||
|
||||
- Python version \>=3.10
|
||||
- A Maxim account ([sign up here](https://getmaxim.ai/))
|
||||
- Generate Maxim API Key
|
||||
- A CrewAI project
|
||||
|
||||
### Installation
|
||||
@@ -30,16 +109,14 @@ Maxim AI provides comprehensive agent monitoring, evaluation, and observability
|
||||
Install the Maxim SDK via pip:
|
||||
|
||||
```python
|
||||
pip install maxim-py>=3.6.2
|
||||
pip install maxim-py
|
||||
```
|
||||
|
||||
Or add it to your `requirements.txt`:
|
||||
|
||||
```
|
||||
maxim-py>=3.6.2
|
||||
maxim-py
|
||||
```
|
||||
|
||||
|
||||
### Basic Setup
|
||||
|
||||
### 1. Set up environment variables
|
||||
@@ -64,18 +141,15 @@ from maxim.logger.crewai import instrument_crewai
|
||||
|
||||
### 3. Initialise Maxim with your API key
|
||||
|
||||
```python
|
||||
# Initialize Maxim logger
|
||||
logger = Maxim().logger()
|
||||
|
||||
```python {8}
|
||||
# Instrument CrewAI with just one line
|
||||
instrument_crewai(logger)
|
||||
instrument_crewai(Maxim().logger())
|
||||
```
|
||||
|
||||
### 4. Create and run your CrewAI application as usual
|
||||
|
||||
```python
|
||||
|
||||
# Create your agent
|
||||
researcher = Agent(
|
||||
role='Senior Research Analyst',
|
||||
@@ -105,7 +179,8 @@ finally:
|
||||
maxim.cleanup() # Ensure cleanup happens even if errors occur
|
||||
```
|
||||
|
||||
That's it! All your CrewAI agent interactions will now be logged and available in your Maxim dashboard.
|
||||
|
||||
That's it\! All your CrewAI agent interactions will now be logged and available in your Maxim dashboard.
|
||||
|
||||
Check this Google Colab Notebook for a quick reference - [Notebook](https://colab.research.google.com/drive/1ZKIZWsmgQQ46n8TH9zLsT1negKkJA6K8?usp=sharing)
|
||||
|
||||
@@ -113,40 +188,44 @@ Check this Google Colab Notebook for a quick reference - [Notebook](https://cola
|
||||
|
||||
After running your CrewAI application:
|
||||
|
||||

|
||||
|
||||
1. Log in to your [Maxim Dashboard](https://getmaxim.ai/dashboard)
|
||||
1. Log in to your [Maxim Dashboard](https://app.getmaxim.ai/login)
|
||||
2. Navigate to your repository
|
||||
3. View detailed agent traces, including:
|
||||
- Agent conversations
|
||||
- Tool usage patterns
|
||||
- Performance metrics
|
||||
- Cost analytics
|
||||
- Agent conversations
|
||||
- Tool usage patterns
|
||||
- Performance metrics
|
||||
- Cost analytics
|
||||
|
||||
<img src='https://raw.githubusercontent.com/akmadan/crewAI/docs_maxim_observability/docs/images/crewai_traces.gif'> </img>
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Common Issues
|
||||
|
||||
- **No traces appearing**: Ensure your API key and repository ID are correc
|
||||
- Ensure you've **called `instrument_crewai()`** ***before*** running your crew. This initializes logging hooks correctly.
|
||||
- **No traces appearing**: Ensure your API key and repository ID are correct
|
||||
- Ensure you've **`called instrument_crewai()`** **_before_** running your crew. This initializes logging hooks correctly.
|
||||
- Set `debug=True` in your `instrument_crewai()` call to surface any internal errors:
|
||||
|
||||
```python
|
||||
instrument_crewai(logger, debug=True)
|
||||
```
|
||||
|
||||
|
||||
```python
|
||||
instrument_crewai(logger, debug=True)
|
||||
```
|
||||
- Configure your agents with `verbose=True` to capture detailed logs:
|
||||
|
||||
```python
|
||||
|
||||
agent = CrewAgent(..., verbose=True)
|
||||
```
|
||||
|
||||
|
||||
```python
|
||||
agent = CrewAgent(..., verbose=True)
|
||||
```
|
||||
- Double-check that `instrument_crewai()` is called **before** creating or executing agents. This might be obvious, but it's a common oversight.
|
||||
|
||||
### Support
|
||||
## Resources
|
||||
|
||||
If you encounter any issues:
|
||||
|
||||
- Check the [Maxim Documentation](https://getmaxim.ai/docs)
|
||||
- Maxim Github [Link](https://github.com/maximhq)
|
||||
<CardGroup cols="3">
|
||||
<Card title="CrewAI Docs" icon="book" href="https://docs.crewai.com/">
|
||||
Official CrewAI documentation
|
||||
</Card>
|
||||
<Card title="Maxim Docs" icon="book" href="https://getmaxim.ai/docs">
|
||||
Official Maxim documentation
|
||||
</Card>
|
||||
<Card title="Maxim Github" icon="github" href="https://github.com/maximhq">
|
||||
Maxim Github
|
||||
</Card>
|
||||
</CardGroup>
|
||||
@@ -56,6 +56,10 @@ These tools enable your agents to interact with the web, extract data from websi
|
||||
<Card title="Stagehand Tool" icon="hand" href="/tools/web-scraping/stagehandtool">
|
||||
Intelligent browser automation with natural language commands.
|
||||
</Card>
|
||||
|
||||
<Card title="Oxylabs Scraper Tool" icon="globe" href="/tools/web-scraping/oxylabsscraperstool">
|
||||
Access web data at scale with Oxylabs.
|
||||
</Card>
|
||||
</CardGroup>
|
||||
|
||||
## **Common Use Cases**
|
||||
@@ -100,4 +104,4 @@ agent = Agent(
|
||||
- **JavaScript-Heavy Sites**: Use `SeleniumScrapingTool` for dynamic content
|
||||
- **Scale & Performance**: Use `FirecrawlScrapeWebsiteTool` for high-volume scraping
|
||||
- **Cloud Infrastructure**: Use `BrowserBaseLoadTool` for scalable browser automation
|
||||
- **Complex Workflows**: Use `StagehandTool` for intelligent browser interactions
|
||||
- **Complex Workflows**: Use `StagehandTool` for intelligent browser interactions
|
||||
|
||||
236
docs/tools/web-scraping/oxylabsscraperstool.mdx
Normal file
@@ -0,0 +1,236 @@
|
||||
---
|
||||
title: Oxylabs Scrapers
|
||||
description: >
|
||||
Oxylabs Scrapers allow to easily access the information from the respective sources. Please see the list of available sources below:
|
||||
- `Amazon Product`
|
||||
- `Amazon Search`
|
||||
- `Google Seach`
|
||||
- `Universal`
|
||||
icon: globe
|
||||
---
|
||||
|
||||
## Installation
|
||||
|
||||
Get the credentials by creating an Oxylabs Account [here](https://oxylabs.io).
|
||||
```shell
|
||||
pip install 'crewai[tools]' oxylabs
|
||||
```
|
||||
Check [Oxylabs Documentation](https://developers.oxylabs.io/scraping-solutions/web-scraper-api/targets) to get more information about API parameters.
|
||||
|
||||
# `OxylabsAmazonProductScraperTool`
|
||||
|
||||
### Example
|
||||
|
||||
```python
|
||||
from crewai_tools import OxylabsAmazonProductScraperTool
|
||||
|
||||
# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
|
||||
tool = OxylabsAmazonProductScraperTool()
|
||||
|
||||
result = tool.run(query="AAAAABBBBCC")
|
||||
|
||||
print(result)
|
||||
```
|
||||
|
||||
### Parameters
|
||||
|
||||
- `query` - 10-symbol ASIN code.
|
||||
- `domain` - domain localization for Amazon.
|
||||
- `geo_location` - the _Deliver to_ location.
|
||||
- `user_agent_type` - device type and browser.
|
||||
- `render` - enables JavaScript rendering when set to `html`.
|
||||
- `callback_url` - URL to your callback endpoint.
|
||||
- `context` - Additional advanced settings and controls for specialized requirements.
|
||||
- `parse` - returns parsed data when set to true.
|
||||
- `parsing_instructions` - define your own parsing and data transformation logic that will be executed on an HTML scraping result.
|
||||
|
||||
### Advanced example
|
||||
|
||||
```python
|
||||
from crewai_tools import OxylabsAmazonProductScraperTool
|
||||
|
||||
# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
|
||||
tool = OxylabsAmazonProductScraperTool(
|
||||
config={
|
||||
"domain": "com",
|
||||
"parse": True,
|
||||
"context": [
|
||||
{
|
||||
"key": "autoselect_variant",
|
||||
"value": True
|
||||
}
|
||||
]
|
||||
}
|
||||
)
|
||||
|
||||
result = tool.run(query="AAAAABBBBCC")
|
||||
|
||||
print(result)
|
||||
```
|
||||
|
||||
# `OxylabsAmazonSearchScraperTool`
|
||||
|
||||
### Example
|
||||
|
||||
```python
|
||||
from crewai_tools import OxylabsAmazonSearchScraperTool
|
||||
|
||||
# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
|
||||
tool = OxylabsAmazonSearchScraperTool()
|
||||
|
||||
result = tool.run(query="headsets")
|
||||
|
||||
print(result)
|
||||
```
|
||||
|
||||
### Parameters
|
||||
|
||||
- `query` - Amazon search term.
|
||||
- `domain` - Domain localization for Bestbuy.
|
||||
- `start_page` - starting page number.
|
||||
- `pages` - number of pages to retrieve.
|
||||
- `geo_location` - the _Deliver to_ location.
|
||||
- `user_agent_type` - device type and browser.
|
||||
- `render` - enables JavaScript rendering when set to `html`.
|
||||
- `callback_url` - URL to your callback endpoint.
|
||||
- `context` - Additional advanced settings and controls for specialized requirements.
|
||||
- `parse` - returns parsed data when set to true.
|
||||
- `parsing_instructions` - define your own parsing and data transformation logic that will be executed on an HTML scraping result.
|
||||
|
||||
### Advanced example
|
||||
|
||||
```python
|
||||
from crewai_tools import OxylabsAmazonSearchScraperTool
|
||||
|
||||
# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
|
||||
tool = OxylabsAmazonSearchScraperTool(
|
||||
config={
|
||||
"domain": 'nl',
|
||||
"start_page": 2,
|
||||
"pages": 2,
|
||||
"parse": True,
|
||||
"context": [
|
||||
{'key': 'category_id', 'value': 16391693031}
|
||||
],
|
||||
}
|
||||
)
|
||||
|
||||
result = tool.run(query='nirvana tshirt')
|
||||
|
||||
print(result)
|
||||
```
|
||||
|
||||
# `OxylabsGoogleSearchScraperTool`
|
||||
|
||||
### Example
|
||||
|
||||
```python
|
||||
from crewai_tools import OxylabsGoogleSearchScraperTool
|
||||
|
||||
# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
|
||||
tool = OxylabsGoogleSearchScraperTool()
|
||||
|
||||
result = tool.run(query="iPhone 16")
|
||||
|
||||
print(result)
|
||||
```
|
||||
|
||||
### Parameters
|
||||
|
||||
- `query` - search keyword.
|
||||
- `domain` - domain localization for Google.
|
||||
- `start_page` - starting page number.
|
||||
- `pages` - number of pages to retrieve.
|
||||
- `limit` - number of results to retrieve in each page.
|
||||
- `locale` - `Accept-Language` header value which changes your Google search page web interface language.
|
||||
- `geo_location` - the geographical location that the result should be adapted for. Using this parameter correctly is extremely important to get the right data.
|
||||
- `user_agent_type` - device type and browser.
|
||||
- `render` - enables JavaScript rendering when set to `html`.
|
||||
- `callback_url` - URL to your callback endpoint.
|
||||
- `context` - Additional advanced settings and controls for specialized requirements.
|
||||
- `parse` - returns parsed data when set to true.
|
||||
- `parsing_instructions` - define your own parsing and data transformation logic that will be executed on an HTML scraping result.
|
||||
|
||||
### Advanced example
|
||||
|
||||
```python
|
||||
from crewai_tools import OxylabsGoogleSearchScraperTool
|
||||
|
||||
# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
|
||||
tool = OxylabsGoogleSearchScraperTool(
|
||||
config={
|
||||
"parse": True,
|
||||
"geo_location": "Paris, France",
|
||||
"user_agent_type": "tablet",
|
||||
}
|
||||
)
|
||||
|
||||
result = tool.run(query="iPhone 16")
|
||||
|
||||
print(result)
|
||||
```
|
||||
|
||||
# `OxylabsUniversalScraperTool`
|
||||
|
||||
### Example
|
||||
|
||||
```python
|
||||
from crewai_tools import OxylabsUniversalScraperTool
|
||||
|
||||
# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
|
||||
tool = OxylabsUniversalScraperTool()
|
||||
|
||||
result = tool.run(url="https://ip.oxylabs.io")
|
||||
|
||||
print(result)
|
||||
```
|
||||
|
||||
### Parameters
|
||||
|
||||
- `url` - website url to scrape.
|
||||
- `user_agent_type` - device type and browser.
|
||||
- `geo_location` - sets the proxy's geolocation to retrieve data.
|
||||
- `render` - enables JavaScript rendering when set to `html`.
|
||||
- `callback_url` - URL to your callback endpoint.
|
||||
- `context` - Additional advanced settings and controls for specialized requirements.
|
||||
- `parse` - returns parsed data when set to `true`, as long as a dedicated parser exists for the submitted URL's page type.
|
||||
- `parsing_instructions` - define your own parsing and data transformation logic that will be executed on an HTML scraping result.
|
||||
|
||||
|
||||
### Advanced example
|
||||
|
||||
```python
|
||||
from crewai_tools import OxylabsUniversalScraperTool
|
||||
|
||||
# make sure OXYLABS_USERNAME and OXYLABS_PASSWORD variables are set
|
||||
tool = OxylabsUniversalScraperTool(
|
||||
config={
|
||||
"render": "html",
|
||||
"user_agent_type": "mobile",
|
||||
"context": [
|
||||
{"key": "force_headers", "value": True},
|
||||
{"key": "force_cookies", "value": True},
|
||||
{
|
||||
"key": "headers",
|
||||
"value": {
|
||||
"Custom-Header-Name": "custom header content",
|
||||
},
|
||||
},
|
||||
{
|
||||
"key": "cookies",
|
||||
"value": [
|
||||
{"key": "NID", "value": "1234567890"},
|
||||
{"key": "1P JAR", "value": "0987654321"},
|
||||
],
|
||||
},
|
||||
{"key": "http_method", "value": "get"},
|
||||
{"key": "follow_redirects", "value": True},
|
||||
{"key": "successful_status_codes", "value": [808, 909]},
|
||||
],
|
||||
}
|
||||
)
|
||||
|
||||
result = tool.run(url="https://ip.oxylabs.io")
|
||||
|
||||
print(result)
|
||||
```
|
||||
216
mkdocs.yml
@@ -1,216 +0,0 @@
|
||||
site_name: crewAI
|
||||
site_author: crewAI, Inc
|
||||
site_description: Cutting-edge framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.
|
||||
repo_name: crewAI
|
||||
repo_url: https://github.com/crewAIInc/crewAI
|
||||
site_url: https://docs.crewai.com
|
||||
edit_uri: edit/main/docs/
|
||||
copyright: Copyright © 2024 crewAI, Inc
|
||||
|
||||
markdown_extensions:
|
||||
- abbr
|
||||
- admonition
|
||||
- pymdownx.details
|
||||
- attr_list
|
||||
- def_list
|
||||
- footnotes
|
||||
- md_in_html
|
||||
- toc:
|
||||
permalink: true
|
||||
- pymdownx.arithmatex:
|
||||
generic: true
|
||||
- pymdownx.betterem:
|
||||
smart_enable: all
|
||||
- pymdownx.caret
|
||||
- pymdownx.emoji:
|
||||
emoji_generator: !!python/name:material.extensions.emoji.to_svg
|
||||
emoji_index: !!python/name:material.extensions.emoji.twemoji
|
||||
- pymdownx.highlight:
|
||||
anchor_linenums: true
|
||||
line_spans: __span
|
||||
pygments_lang_class: true
|
||||
- pymdownx.inlinehilite
|
||||
- pymdownx.keys
|
||||
- pymdownx.magiclink:
|
||||
normalize_issue_symbols: true
|
||||
repo_url_shorthand: true
|
||||
user: joaomdmoura
|
||||
repo: crewAI
|
||||
- pymdownx.mark
|
||||
- pymdownx.smartsymbols
|
||||
- pymdownx.snippets:
|
||||
auto_append:
|
||||
- includes/mkdocs.md
|
||||
- pymdownx.superfences:
|
||||
custom_fences:
|
||||
- name: mermaid
|
||||
class: mermaid
|
||||
format: !!python/name:pymdownx.superfences.fence_code_format
|
||||
- pymdownx.tabbed:
|
||||
alternate_style: true
|
||||
combine_header_slug: true
|
||||
slugify: !!python/object/apply:pymdownx.slugs.slugify
|
||||
kwds:
|
||||
case: lower
|
||||
- pymdownx.tasklist:
|
||||
custom_checkbox: true
|
||||
- pymdownx.tilde
|
||||
theme:
|
||||
name: material
|
||||
language: en
|
||||
icon:
|
||||
repo: fontawesome/brands/github
|
||||
edit: material/pencil
|
||||
view: material/eye
|
||||
admonition:
|
||||
note: octicons/light-bulb-16
|
||||
abstract: octicons/checklist-16
|
||||
info: octicons/info-16
|
||||
tip: octicons/squirrel-16
|
||||
success: octicons/check-16
|
||||
question: octicons/question-16
|
||||
warning: octicons/alert-16
|
||||
failure: octicons/x-circle-16
|
||||
danger: octicons/zap-16
|
||||
bug: octicons/bug-16
|
||||
example: octicons/beaker-16
|
||||
quote: octicons/quote-16
|
||||
|
||||
palette:
|
||||
- scheme: default
|
||||
primary: deep orange
|
||||
accent: deep orange
|
||||
toggle:
|
||||
icon: material/brightness-7
|
||||
name: Switch to dark mode
|
||||
- scheme: slate
|
||||
primary: deep orange
|
||||
accent: deep orange
|
||||
toggle:
|
||||
icon: material/brightness-4
|
||||
name: Switch to light mode
|
||||
features:
|
||||
- announce.dismiss
|
||||
- content.action.edit
|
||||
- content.action.view
|
||||
- content.code.annotate
|
||||
- content.code.copy
|
||||
- content.code.select
|
||||
- content.tabs.link
|
||||
- content.tooltips
|
||||
- header.autohide
|
||||
- navigation.footer
|
||||
- navigation.indexes
|
||||
# - navigation.prune
|
||||
# - navigation.sections
|
||||
# - navigation.tabs
|
||||
- search.suggest
|
||||
- navigation.instant
|
||||
- navigation.instant.progress
|
||||
- navigation.instant.prefetch
|
||||
- navigation.tracking
|
||||
# - navigation.expand
|
||||
- navigation.path
|
||||
- navigation.top
|
||||
- toc.follow
|
||||
- toc.integrate
|
||||
- search.highlight
|
||||
- search.share
|
||||
|
||||
nav:
|
||||
- Home: '/'
|
||||
- Getting Started:
|
||||
- Installing CrewAI: 'getting-started/Installing-CrewAI.md'
|
||||
- Starting a new CrewAI project: 'getting-started/Start-a-New-CrewAI-Project-Template-Method.md'
|
||||
- Core Concepts:
|
||||
- Agents: 'core-concepts/Agents.md'
|
||||
- Tasks: 'core-concepts/Tasks.md'
|
||||
- Tools: 'core-concepts/Tools.md'
|
||||
- Processes: 'core-concepts/Processes.md'
|
||||
- Crews: 'core-concepts/Crews.md'
|
||||
- Collaboration: 'core-concepts/Collaboration.md'
|
||||
- Training: 'core-concepts/Training-Crew.md'
|
||||
- Memory: 'core-concepts/Memory.md'
|
||||
- Planning: 'core-concepts/Planning.md'
|
||||
- Testing: 'core-concepts/Testing.md'
|
||||
- Using LangChain Tools: 'core-concepts/Using-LangChain-Tools.md'
|
||||
- Using LlamaIndex Tools: 'core-concepts/Using-LlamaIndex-Tools.md'
|
||||
- How to Guides:
|
||||
- Create Custom Tools: 'how-to/Create-Custom-Tools.md'
|
||||
- Using Sequential Process: 'how-to/Sequential.md'
|
||||
- Using Hierarchical Process: 'how-to/Hierarchical.md'
|
||||
- Create your own Manager Agent: 'how-to/Your-Own-Manager-Agent.md'
|
||||
- Connecting to any LLM: 'how-to/LLM-Connections.md'
|
||||
- Customizing Agents: 'how-to/Customizing-Agents.md'
|
||||
- Coding Agents: 'how-to/Coding-Agents.md'
|
||||
- Forcing Tool Output as Result: 'how-to/Force-Tool-Ouput-as-Result.md'
|
||||
- Human Input on Execution: 'how-to/Human-Input-on-Execution.md'
|
||||
- Kickoff a Crew Asynchronously: 'how-to/Kickoff-async.md'
|
||||
- Kickoff a Crew for a List: 'how-to/Kickoff-for-each.md'
|
||||
- Replay from a specific task from a kickoff: 'how-to/Replay-tasks-from-latest-Crew-Kickoff.md'
|
||||
- Conditional Tasks: 'how-to/Conditional-Tasks.md'
|
||||
- Agent Monitoring with AgentOps: 'how-to/AgentOps-Observability.md'
|
||||
- Agent Monitoring with LangTrace: 'how-to/Langtrace-Observability.md'
|
||||
- Agent Monitoring with OpenLIT: 'how-to/openlit-Observability.md'
|
||||
- Agent Monitoring with MLflow: 'how-to/mlflow-Observability.md'
|
||||
- Tools Docs:
|
||||
- Browserbase Web Loader: 'tools/BrowserbaseLoadTool.md'
|
||||
- Code Docs RAG Search: 'tools/CodeDocsSearchTool.md'
|
||||
- Code Interpreter: 'tools/CodeInterpreterTool.md'
|
||||
- Composio Tools: 'tools/ComposioTool.md'
|
||||
- CSV RAG Search: 'tools/CSVSearchTool.md'
|
||||
- DALL-E Tool: 'tools/DALL-ETool.md'
|
||||
- Directory RAG Search: 'tools/DirectorySearchTool.md'
|
||||
- Directory Read: 'tools/DirectoryReadTool.md'
|
||||
- Docx Rag Search: 'tools/DOCXSearchTool.md'
|
||||
- EXA Search Web Loader: 'tools/EXASearchTool.md'
|
||||
- File Read: 'tools/FileReadTool.md'
|
||||
- File Write: 'tools/FileWriteTool.md'
|
||||
- Firecrawl Crawl Website Tool: 'tools/FirecrawlCrawlWebsiteTool.md'
|
||||
- Firecrawl Scrape Website Tool: 'tools/FirecrawlScrapeWebsiteTool.md'
|
||||
- Firecrawl Search Tool: 'tools/FirecrgstawlSearchTool.md'
|
||||
- Github RAG Search: 'tools/GitHubSearchTool.md'
|
||||
- Google Serper Search: 'tools/SerperDevTool.md'
|
||||
- JSON RAG Search: 'tools/JSONSearchTool.md'
|
||||
- MDX RAG Search: 'tools/MDXSearchTool.md'
|
||||
- MySQL Tool: 'tools/MySQLTool.md'
|
||||
- NL2SQL Tool: 'tools/NL2SQLTool.md'
|
||||
- PDF RAG Search: 'tools/PDFSearchTool.md'
|
||||
- PG RAG Search: 'tools/PGSearchTool.md'
|
||||
- Scrape Website: 'tools/ScrapeWebsiteTool.md'
|
||||
- Selenium Scraper: 'tools/SeleniumScrapingTool.md'
|
||||
- Spider Scraper: 'tools/SpiderTool.md'
|
||||
- TXT RAG Search: 'tools/TXTSearchTool.md'
|
||||
- Vision Tool: 'tools/VisionTool.md'
|
||||
- Website RAG Search: 'tools/WebsiteSearchTool.md'
|
||||
- XML RAG Search: 'tools/XMLSearchTool.md'
|
||||
- Youtube Channel RAG Search: 'tools/YoutubeChannelSearchTool.md'
|
||||
- Youtube Video RAG Search: 'tools/YoutubeVideoSearchTool.md'
|
||||
- Examples:
|
||||
- Trip Planner Crew: https://github.com/joaomdmoura/crewAI-examples/tree/main/trip_planner"
|
||||
- Create Instagram Post: https://github.com/joaomdmoura/crewAI-examples/tree/main/instagram_post"
|
||||
- Stock Analysis: https://github.com/joaomdmoura/crewAI-examples/tree/main/stock_analysis"
|
||||
- Game Generator: https://github.com/joaomdmoura/crewAI-examples/tree/main/game-builder-crew"
|
||||
- Drafting emails with LangGraph: https://github.com/joaomdmoura/crewAI-examples/tree/main/CrewAI-LangGraph"
|
||||
- Landing Page Generator: https://github.com/joaomdmoura/crewAI-examples/tree/main/landing_page_generator"
|
||||
- Prepare for meetings: https://github.com/joaomdmoura/crewAI-examples/tree/main/prep-for-a-meeting"
|
||||
- Telemetry: 'telemetry/Telemetry.md'
|
||||
- Change Log: 'https://github.com/crewAIInc/crewAI/releases'
|
||||
|
||||
extra_css:
|
||||
- stylesheets/output.css
|
||||
- stylesheets/extra.css
|
||||
|
||||
plugins:
|
||||
- social
|
||||
- search
|
||||
|
||||
extra:
|
||||
analytics:
|
||||
provider: google
|
||||
property: G-N3Q505TMQ6
|
||||
social:
|
||||
- icon: fontawesome/brands/x-twitter
|
||||
link: https://x.com/crewAIInc
|
||||
- icon: fontawesome/brands/github
|
||||
link: https://github.com/crewAIInc/crewAI
|
||||
@@ -74,11 +74,6 @@ dev-dependencies = [
|
||||
"ruff>=0.8.2",
|
||||
"mypy>=1.10.0",
|
||||
"pre-commit>=3.6.0",
|
||||
"mkdocs>=1.4.3",
|
||||
"mkdocstrings>=0.22.0",
|
||||
"mkdocstrings-python>=1.1.2",
|
||||
"mkdocs-material>=9.5.7",
|
||||
"mkdocs-material-extensions>=1.3.1",
|
||||
"pillow>=10.2.0",
|
||||
"cairosvg>=2.7.1",
|
||||
"pytest>=8.0.0",
|
||||
|
||||
@@ -91,9 +91,6 @@ class Agent(BaseAgent):
|
||||
function_calling_llm: Optional[Union[str, InstanceOf[BaseLLM], Any]] = Field(
|
||||
description="Language model that will run the agent.", default=None
|
||||
)
|
||||
fallback_llms: Optional[List[Union[str, InstanceOf[BaseLLM], Any]]] = Field(
|
||||
default=None, description="List of fallback language models to try if the primary LLM fails."
|
||||
)
|
||||
system_template: Optional[str] = Field(
|
||||
default=None, description="System format for the agent."
|
||||
)
|
||||
@@ -177,8 +174,6 @@ class Agent(BaseAgent):
|
||||
self.agent_ops_agent_name = self.role
|
||||
|
||||
self.llm = create_llm(self.llm)
|
||||
if self.fallback_llms:
|
||||
self.fallback_llms = [create_llm(fallback_llm) for fallback_llm in self.fallback_llms]
|
||||
if self.function_calling_llm and not isinstance(
|
||||
self.function_calling_llm, BaseLLM
|
||||
):
|
||||
|
||||
@@ -159,7 +159,6 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
|
||||
messages=self.messages,
|
||||
callbacks=self.callbacks,
|
||||
printer=self._printer,
|
||||
fallback_llms=getattr(self.agent, 'fallback_llms', None),
|
||||
)
|
||||
formatted_answer = process_llm_response(answer, self.use_stop_words)
|
||||
|
||||
|
||||
@@ -94,17 +94,18 @@ def _get_project_attribute(
|
||||
|
||||
attribute = _get_nested_value(pyproject_content, keys)
|
||||
except FileNotFoundError:
|
||||
print(f"Error: {pyproject_path} not found.")
|
||||
console.print(f"Error: {pyproject_path} not found.", style="bold red")
|
||||
except KeyError:
|
||||
print(f"Error: {pyproject_path} is not a valid pyproject.toml file.")
|
||||
console.print(f"Error: {pyproject_path} is not a valid pyproject.toml file.", style="bold red")
|
||||
except tomllib.TOMLDecodeError if sys.version_info >= (3, 11) else Exception as e: # type: ignore
|
||||
print(
|
||||
console.print(
|
||||
f"Error: {pyproject_path} is not a valid TOML file."
|
||||
if sys.version_info >= (3, 11)
|
||||
else f"Error reading the pyproject.toml file: {e}"
|
||||
else f"Error reading the pyproject.toml file: {e}",
|
||||
style="bold red",
|
||||
)
|
||||
except Exception as e:
|
||||
print(f"Error reading the pyproject.toml file: {e}")
|
||||
console.print(f"Error reading the pyproject.toml file: {e}", style="bold red")
|
||||
|
||||
if require and not attribute:
|
||||
console.print(
|
||||
@@ -137,9 +138,9 @@ def fetch_and_json_env_file(env_file_path: str = ".env") -> dict:
|
||||
return env_dict
|
||||
|
||||
except FileNotFoundError:
|
||||
print(f"Error: {env_file_path} not found.")
|
||||
console.print(f"Error: {env_file_path} not found.", style="bold red")
|
||||
except Exception as e:
|
||||
print(f"Error reading the .env file: {e}")
|
||||
console.print(f"Error reading the .env file: {e}", style="bold red")
|
||||
|
||||
return {}
|
||||
|
||||
@@ -255,50 +256,69 @@ def write_env_file(folder_path, env_vars):
|
||||
|
||||
|
||||
def get_crews(crew_path: str = "crew.py", require: bool = False) -> list[Crew]:
|
||||
"""Get the crew instances from the a file."""
|
||||
"""Get the crew instances from a file."""
|
||||
crew_instances = []
|
||||
try:
|
||||
import importlib.util
|
||||
|
||||
for root, _, files in os.walk("."):
|
||||
if crew_path in files:
|
||||
crew_os_path = os.path.join(root, crew_path)
|
||||
try:
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
"crew_module", crew_os_path
|
||||
)
|
||||
if not spec or not spec.loader:
|
||||
continue
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
# Add the current directory to sys.path to ensure imports resolve correctly
|
||||
current_dir = os.getcwd()
|
||||
if current_dir not in sys.path:
|
||||
sys.path.insert(0, current_dir)
|
||||
|
||||
# If we're not in src directory but there's a src directory, add it to path
|
||||
src_dir = os.path.join(current_dir, "src")
|
||||
if os.path.isdir(src_dir) and src_dir not in sys.path:
|
||||
sys.path.insert(0, src_dir)
|
||||
|
||||
# Search in both current directory and src directory if it exists
|
||||
search_paths = [".", "src"] if os.path.isdir("src") else ["."]
|
||||
|
||||
for search_path in search_paths:
|
||||
for root, _, files in os.walk(search_path):
|
||||
if crew_path in files and "cli/templates" not in root:
|
||||
crew_os_path = os.path.join(root, crew_path)
|
||||
try:
|
||||
sys.modules[spec.name] = module
|
||||
spec.loader.exec_module(module)
|
||||
|
||||
for attr_name in dir(module):
|
||||
module_attr = getattr(module, attr_name)
|
||||
|
||||
try:
|
||||
crew_instances.extend(fetch_crews(module_attr))
|
||||
except Exception as e:
|
||||
print(f"Error processing attribute {attr_name}: {e}")
|
||||
continue
|
||||
|
||||
except Exception as exec_error:
|
||||
print(f"Error executing module: {exec_error}")
|
||||
import traceback
|
||||
|
||||
print(f"Traceback: {traceback.format_exc()}")
|
||||
except (ImportError, AttributeError) as e:
|
||||
if require:
|
||||
console.print(
|
||||
f"Error importing crew from {crew_path}: {str(e)}",
|
||||
style="bold red",
|
||||
spec = importlib.util.spec_from_file_location(
|
||||
"crew_module", crew_os_path
|
||||
)
|
||||
if not spec or not spec.loader:
|
||||
continue
|
||||
|
||||
module = importlib.util.module_from_spec(spec)
|
||||
sys.modules[spec.name] = module
|
||||
|
||||
try:
|
||||
spec.loader.exec_module(module)
|
||||
|
||||
for attr_name in dir(module):
|
||||
module_attr = getattr(module, attr_name)
|
||||
try:
|
||||
crew_instances.extend(fetch_crews(module_attr))
|
||||
except Exception as e:
|
||||
console.print(f"Error processing attribute {attr_name}: {e}", style="bold red")
|
||||
continue
|
||||
|
||||
# If we found crew instances, break out of the loop
|
||||
if crew_instances:
|
||||
break
|
||||
|
||||
except Exception as exec_error:
|
||||
console.print(f"Error executing module: {exec_error}", style="bold red")
|
||||
|
||||
except (ImportError, AttributeError) as e:
|
||||
if require:
|
||||
console.print(
|
||||
f"Error importing crew from {crew_path}: {str(e)}",
|
||||
style="bold red",
|
||||
)
|
||||
continue
|
||||
|
||||
# If we found crew instances in this search path, break out of the search paths loop
|
||||
if crew_instances:
|
||||
break
|
||||
|
||||
if require:
|
||||
if require and not crew_instances:
|
||||
console.print("No valid Crew instance found in crew.py", style="bold red")
|
||||
raise SystemExit
|
||||
|
||||
@@ -318,11 +338,15 @@ def get_crew_instance(module_attr) -> Crew | None:
|
||||
and module_attr.is_crew_class
|
||||
):
|
||||
return module_attr().crew()
|
||||
if (ismethod(module_attr) or isfunction(module_attr)) and get_type_hints(
|
||||
module_attr
|
||||
).get("return") is Crew:
|
||||
return module_attr()
|
||||
elif isinstance(module_attr, Crew):
|
||||
try:
|
||||
if (ismethod(module_attr) or isfunction(module_attr)) and get_type_hints(
|
||||
module_attr
|
||||
).get("return") is Crew:
|
||||
return module_attr()
|
||||
except Exception:
|
||||
return None
|
||||
|
||||
if isinstance(module_attr, Crew):
|
||||
return module_attr
|
||||
else:
|
||||
return None
|
||||
@@ -402,7 +426,8 @@ def _load_tools_from_init(init_file: Path) -> list[dict[str, Any]]:
|
||||
|
||||
if not hasattr(module, "__all__"):
|
||||
console.print(
|
||||
f"[bold yellow]Warning: No __all__ defined in {init_file}[/bold yellow]"
|
||||
f"Warning: No __all__ defined in {init_file}",
|
||||
style="bold yellow",
|
||||
)
|
||||
raise SystemExit(1)
|
||||
|
||||
|
||||
@@ -526,7 +526,6 @@ class LiteAgent(FlowTrackable, BaseModel):
|
||||
messages=self._messages,
|
||||
callbacks=self._callbacks,
|
||||
printer=self._printer,
|
||||
fallback_llms=getattr(self, 'fallback_llms', None),
|
||||
)
|
||||
|
||||
# Emit LLM call completed event
|
||||
|
||||
@@ -1,7 +1,8 @@
|
||||
import inspect
|
||||
import logging
|
||||
from pathlib import Path
|
||||
from typing import Any, Callable, Dict, TypeVar, cast
|
||||
from typing import Any, Callable, Dict, TypeVar, cast, List
|
||||
from crewai.tools import BaseTool
|
||||
|
||||
import yaml
|
||||
from dotenv import load_dotenv
|
||||
@@ -27,6 +28,8 @@ def CrewBase(cls: T) -> T:
|
||||
)
|
||||
original_tasks_config_path = getattr(cls, "tasks_config", "config/tasks.yaml")
|
||||
|
||||
mcp_server_params: Any = getattr(cls, "mcp_server_params", None)
|
||||
|
||||
def __init__(self, *args, **kwargs):
|
||||
super().__init__(*args, **kwargs)
|
||||
self.load_configurations()
|
||||
@@ -64,6 +67,39 @@ def CrewBase(cls: T) -> T:
|
||||
self._original_functions, "is_kickoff"
|
||||
)
|
||||
|
||||
# Add close mcp server method to after kickoff
|
||||
bound_method = self._create_close_mcp_server_method()
|
||||
self._after_kickoff['_close_mcp_server'] = bound_method
|
||||
|
||||
def _create_close_mcp_server_method(self):
|
||||
def _close_mcp_server(self, instance, outputs):
|
||||
adapter = getattr(self, '_mcp_server_adapter', None)
|
||||
if adapter is not None:
|
||||
try:
|
||||
adapter.stop()
|
||||
except Exception as e:
|
||||
logging.warning(f"Error stopping MCP server: {e}")
|
||||
return outputs
|
||||
|
||||
_close_mcp_server.is_after_kickoff = True
|
||||
|
||||
import types
|
||||
return types.MethodType(_close_mcp_server, self)
|
||||
|
||||
def get_mcp_tools(self) -> List[BaseTool]:
|
||||
if not self.mcp_server_params:
|
||||
return []
|
||||
|
||||
from crewai_tools import MCPServerAdapter
|
||||
|
||||
adapter = getattr(self, '_mcp_server_adapter', None)
|
||||
if adapter and isinstance(adapter, MCPServerAdapter):
|
||||
return adapter.tools
|
||||
|
||||
self._mcp_server_adapter = MCPServerAdapter(self.mcp_server_params)
|
||||
return self._mcp_server_adapter.tools
|
||||
|
||||
|
||||
def load_configurations(self):
|
||||
"""Load agent and task configurations from YAML files."""
|
||||
if isinstance(self.original_agents_config_path, str):
|
||||
|
||||
@@ -39,7 +39,7 @@ from crewai.tasks.output_format import OutputFormat
|
||||
from crewai.tasks.task_output import TaskOutput
|
||||
from crewai.tools.base_tool import BaseTool
|
||||
from crewai.utilities.config import process_config
|
||||
from crewai.utilities.constants import NOT_SPECIFIED
|
||||
from crewai.utilities.constants import NOT_SPECIFIED, _NotSpecified
|
||||
from crewai.utilities.guardrail import process_guardrail, GuardrailResult
|
||||
from crewai.utilities.converter import Converter, convert_to_model
|
||||
from crewai.utilities.events import (
|
||||
@@ -95,9 +95,9 @@ class Task(BaseModel):
|
||||
agent: Optional[BaseAgent] = Field(
|
||||
description="Agent responsible for execution the task.", default=None
|
||||
)
|
||||
context: Optional[List["Task"]] = Field(
|
||||
context: Union[List["Task"], None, _NotSpecified] = Field(
|
||||
description="Other tasks that will have their output used as context for this task.",
|
||||
default=NOT_SPECIFIED,
|
||||
default=NOT_SPECIFIED
|
||||
)
|
||||
async_execution: Optional[bool] = Field(
|
||||
description="Whether the task should be executed asynchronously or not.",
|
||||
@@ -158,6 +158,9 @@ class Task(BaseModel):
|
||||
end_time: Optional[datetime.datetime] = Field(
|
||||
default=None, description="End time of the task execution"
|
||||
)
|
||||
model_config = {
|
||||
"arbitrary_types_allowed": True
|
||||
}
|
||||
|
||||
@field_validator("guardrail")
|
||||
@classmethod
|
||||
|
||||
@@ -145,52 +145,27 @@ def get_llm_response(
|
||||
messages: List[Dict[str, str]],
|
||||
callbacks: List[Any],
|
||||
printer: Printer,
|
||||
fallback_llms: Optional[List[Union[LLM, BaseLLM]]] = None,
|
||||
) -> str:
|
||||
"""Call the LLM and return the response, handling any invalid responses and trying fallbacks if available."""
|
||||
llms_to_try = [llm]
|
||||
if fallback_llms:
|
||||
llms_to_try.extend(fallback_llms)
|
||||
|
||||
last_exception = None
|
||||
|
||||
for i, current_llm in enumerate(llms_to_try):
|
||||
try:
|
||||
answer = current_llm.call(
|
||||
messages,
|
||||
callbacks=callbacks,
|
||||
)
|
||||
if not answer:
|
||||
error_msg = "Received None or empty response from LLM call."
|
||||
printer.print(content=error_msg, color="red")
|
||||
if i < len(llms_to_try) - 1:
|
||||
printer.print(content=f"Trying fallback LLM {i+1}...", color="yellow")
|
||||
continue
|
||||
else:
|
||||
raise ValueError("Invalid response from LLM call - None or empty.")
|
||||
return answer
|
||||
except Exception as e:
|
||||
last_exception = e
|
||||
if i == 0:
|
||||
printer.print(content=f"Primary LLM failed: {e}", color="red")
|
||||
else:
|
||||
printer.print(content=f"Fallback LLM {i} failed: {e}", color="red")
|
||||
|
||||
if e.__class__.__module__.startswith("litellm"):
|
||||
error_str = str(e).lower()
|
||||
if any(term in error_str for term in ["authentication", "api key", "unauthorized", "forbidden"]):
|
||||
printer.print(content="Authentication error detected, skipping remaining fallbacks", color="red")
|
||||
raise e
|
||||
|
||||
if i < len(llms_to_try) - 1:
|
||||
printer.print(content=f"Trying fallback LLM {i+1}...", color="yellow")
|
||||
continue
|
||||
|
||||
printer.print(content="All LLMs failed, raising last exception", color="red")
|
||||
if last_exception is not None:
|
||||
raise last_exception
|
||||
else:
|
||||
raise RuntimeError("All LLMs failed but no exception was captured")
|
||||
"""Call the LLM and return the response, handling any invalid responses."""
|
||||
try:
|
||||
answer = llm.call(
|
||||
messages,
|
||||
callbacks=callbacks,
|
||||
)
|
||||
except Exception as e:
|
||||
printer.print(
|
||||
content=f"Error during LLM call: {e}",
|
||||
color="red",
|
||||
)
|
||||
raise e
|
||||
if not answer:
|
||||
printer.print(
|
||||
content="Received None or empty response from LLM call.",
|
||||
color="red",
|
||||
)
|
||||
raise ValueError("Invalid response from LLM call - None or empty.")
|
||||
|
||||
return answer
|
||||
|
||||
|
||||
def process_llm_response(
|
||||
@@ -501,7 +476,14 @@ def load_agent_from_repository(from_repository: str) -> Dict[str, Any]:
|
||||
try:
|
||||
module = importlib.import_module(tool["module"])
|
||||
tool_class = getattr(module, tool["name"])
|
||||
attributes[key].append(tool_class())
|
||||
|
||||
tool_value = tool_class(**tool["init_params"])
|
||||
|
||||
if isinstance(tool_value, list):
|
||||
attributes[key].extend(tool_value)
|
||||
else:
|
||||
attributes[key].append(tool_value)
|
||||
|
||||
except Exception as e:
|
||||
raise AgentRepositoryError(
|
||||
f"Tool {tool['name']} could not be loaded: {e}"
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
from typing import TYPE_CHECKING, List
|
||||
|
||||
from typing import TYPE_CHECKING, List, Union
|
||||
from crewai.utilities.constants import _NotSpecified
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from crewai.task import Task
|
||||
@@ -15,7 +15,7 @@ def aggregate_raw_outputs_from_task_outputs(task_outputs: List["TaskOutput"]) ->
|
||||
return context
|
||||
|
||||
|
||||
def aggregate_raw_outputs_from_tasks(tasks: List["Task"]) -> str:
|
||||
def aggregate_raw_outputs_from_tasks(tasks: Union[List["Task"],_NotSpecified]) -> str:
|
||||
"""Generate string context from the tasks."""
|
||||
|
||||
task_outputs = (
|
||||
|
||||
@@ -1984,7 +1984,7 @@ def test_crew_agent_executor_litellm_auth_error():
|
||||
)
|
||||
|
||||
# Verify error handling messages
|
||||
error_message = f"Primary LLM failed: {str(mock_llm_call.side_effect)}"
|
||||
error_message = f"Error during LLM call: {str(mock_llm_call.side_effect)}"
|
||||
mock_printer.assert_any_call(
|
||||
content=error_message,
|
||||
color="red",
|
||||
@@ -2099,7 +2099,7 @@ def mock_get_auth_token():
|
||||
|
||||
@patch("crewai.cli.plus_api.PlusAPI.get_agent")
|
||||
def test_agent_from_repository(mock_get_agent, mock_get_auth_token):
|
||||
from crewai_tools import SerperDevTool, XMLSearchTool
|
||||
from crewai_tools import SerperDevTool, XMLSearchTool, CSVSearchTool, EnterpriseActionTool
|
||||
|
||||
mock_get_response = MagicMock()
|
||||
mock_get_response.status_code = 200
|
||||
@@ -2108,19 +2108,42 @@ def test_agent_from_repository(mock_get_agent, mock_get_auth_token):
|
||||
"goal": "test goal",
|
||||
"backstory": "test backstory",
|
||||
"tools": [
|
||||
{"module": "crewai_tools", "name": "SerperDevTool"},
|
||||
{"module": "crewai_tools", "name": "XMLSearchTool"},
|
||||
{"module": "crewai_tools", "name": "SerperDevTool", "init_params": {"n_results": 30}},
|
||||
{"module": "crewai_tools", "name": "XMLSearchTool", "init_params": {"summarize": True}},
|
||||
{"module": "crewai_tools", "name": "CSVSearchTool", "init_params": {}},
|
||||
|
||||
# using a tools that returns a list of BaseTools
|
||||
{"module": "crewai_tools", "name": "CrewaiEnterpriseTools", "init_params": {"actions_list": [], "enterprise_token": "test_key"}},
|
||||
],
|
||||
}
|
||||
mock_get_agent.return_value = mock_get_response
|
||||
agent = Agent(from_repository="test_agent")
|
||||
|
||||
tool_action = EnterpriseActionTool(
|
||||
name="test_name",
|
||||
description="test_description",
|
||||
enterprise_action_token="test_token",
|
||||
action_name="test_action_name",
|
||||
action_schema={"test": "test"},
|
||||
)
|
||||
|
||||
with patch("crewai_tools.CrewaiEnterpriseTools", return_value=[tool_action]):
|
||||
agent = Agent(from_repository="test_agent")
|
||||
|
||||
assert agent.role == "test role"
|
||||
assert agent.goal == "test goal"
|
||||
assert agent.backstory == "test backstory"
|
||||
assert len(agent.tools) == 2
|
||||
assert len(agent.tools) == 4
|
||||
|
||||
assert isinstance(agent.tools[0], SerperDevTool)
|
||||
assert agent.tools[0].n_results == 30
|
||||
assert isinstance(agent.tools[1], XMLSearchTool)
|
||||
assert agent.tools[1].summarize
|
||||
|
||||
assert isinstance(agent.tools[2], CSVSearchTool)
|
||||
assert not agent.tools[2].summarize
|
||||
|
||||
assert isinstance(agent.tools[3], EnterpriseActionTool)
|
||||
assert agent.tools[3].name == "test_name"
|
||||
|
||||
|
||||
@patch("crewai.cli.plus_api.PlusAPI.get_agent")
|
||||
@@ -2133,7 +2156,7 @@ def test_agent_from_repository_override_attributes(mock_get_agent, mock_get_auth
|
||||
"role": "test role",
|
||||
"goal": "test goal",
|
||||
"backstory": "test backstory",
|
||||
"tools": [{"name": "SerperDevTool", "module": "crewai_tools"}],
|
||||
"tools": [{"name": "SerperDevTool", "module": "crewai_tools", "init_params": {}}],
|
||||
}
|
||||
mock_get_agent.return_value = mock_get_response
|
||||
agent = Agent(from_repository="test_agent", role="Custom Role")
|
||||
|
||||
@@ -261,3 +261,104 @@ __all__ = ['MyTool']
|
||||
captured = capsys.readouterr()
|
||||
|
||||
assert "was never closed" in captured.out
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_crew():
|
||||
from crewai.crew import Crew
|
||||
|
||||
class MockCrew(Crew):
|
||||
def __init__(self):
|
||||
pass
|
||||
|
||||
return MockCrew()
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def temp_crew_project():
|
||||
with tempfile.TemporaryDirectory() as temp_dir:
|
||||
old_cwd = os.getcwd()
|
||||
os.chdir(temp_dir)
|
||||
|
||||
crew_content = """
|
||||
from crewai.crew import Crew
|
||||
from crewai.agent import Agent
|
||||
|
||||
def create_crew() -> Crew:
|
||||
agent = Agent(role="test", goal="test", backstory="test")
|
||||
return Crew(agents=[agent], tasks=[])
|
||||
|
||||
# Direct crew instance
|
||||
direct_crew = Crew(agents=[], tasks=[])
|
||||
"""
|
||||
|
||||
with open("crew.py", "w") as f:
|
||||
f.write(crew_content)
|
||||
|
||||
os.makedirs("src", exist_ok=True)
|
||||
with open(os.path.join("src", "crew.py"), "w") as f:
|
||||
f.write(crew_content)
|
||||
|
||||
# Create a src/templates directory that should be ignored
|
||||
os.makedirs(os.path.join("src", "templates"), exist_ok=True)
|
||||
with open(os.path.join("src", "templates", "crew.py"), "w") as f:
|
||||
f.write("# This should be ignored")
|
||||
|
||||
yield temp_dir
|
||||
|
||||
os.chdir(old_cwd)
|
||||
|
||||
|
||||
def test_get_crews_finds_valid_crews(temp_crew_project, monkeypatch, mock_crew):
|
||||
def mock_fetch_crews(module_attr):
|
||||
return [mock_crew]
|
||||
|
||||
monkeypatch.setattr(utils, "fetch_crews", mock_fetch_crews)
|
||||
|
||||
crews = utils.get_crews()
|
||||
|
||||
assert len(crews) > 0
|
||||
assert mock_crew in crews
|
||||
|
||||
|
||||
def test_get_crews_with_nonexistent_file(temp_crew_project):
|
||||
crews = utils.get_crews(crew_path="nonexistent.py", require=False)
|
||||
assert len(crews) == 0
|
||||
|
||||
|
||||
def test_get_crews_with_required_nonexistent_file(temp_crew_project, capsys):
|
||||
with pytest.raises(SystemExit):
|
||||
utils.get_crews(crew_path="nonexistent.py", require=True)
|
||||
|
||||
captured = capsys.readouterr()
|
||||
assert "No valid Crew instance found" in captured.out
|
||||
|
||||
|
||||
def test_get_crews_with_invalid_module(temp_crew_project, capsys):
|
||||
with open("crew.py", "w") as f:
|
||||
f.write("import nonexistent_module\n")
|
||||
|
||||
crews = utils.get_crews(crew_path="crew.py", require=False)
|
||||
assert len(crews) == 0
|
||||
|
||||
with pytest.raises(SystemExit):
|
||||
utils.get_crews(crew_path="crew.py", require=True)
|
||||
|
||||
captured = capsys.readouterr()
|
||||
assert "Error" in captured.out
|
||||
|
||||
|
||||
def test_get_crews_ignores_template_directories(temp_crew_project, monkeypatch, mock_crew):
|
||||
template_crew_detected = False
|
||||
|
||||
def mock_fetch_crews(module_attr):
|
||||
nonlocal template_crew_detected
|
||||
if hasattr(module_attr, "__file__") and "templates" in module_attr.__file__:
|
||||
template_crew_detected = True
|
||||
return [mock_crew]
|
||||
|
||||
monkeypatch.setattr(utils, "fetch_crews", mock_fetch_crews)
|
||||
|
||||
utils.get_crews()
|
||||
|
||||
assert not template_crew_detected
|
||||
|
||||
@@ -1,5 +1,5 @@
|
||||
from typing import List
|
||||
|
||||
from unittest.mock import Mock, patch
|
||||
import pytest
|
||||
|
||||
from crewai.agent import Agent
|
||||
@@ -16,7 +16,7 @@ from crewai.project import (
|
||||
task,
|
||||
)
|
||||
from crewai.task import Task
|
||||
|
||||
from crewai.tools import tool
|
||||
|
||||
class SimpleCrew:
|
||||
@agent
|
||||
@@ -85,6 +85,14 @@ class InternalCrew:
|
||||
def crew(self):
|
||||
return Crew(agents=self.agents, tasks=self.tasks, verbose=True)
|
||||
|
||||
@CrewBase
|
||||
class InternalCrewWithMCP(InternalCrew):
|
||||
mcp_server_params = {"host": "localhost", "port": 8000}
|
||||
|
||||
@agent
|
||||
def reporting_analyst(self):
|
||||
return Agent(config=self.agents_config["reporting_analyst"], tools=self.get_mcp_tools()) # type: ignore[index]
|
||||
|
||||
|
||||
def test_agent_memoization():
|
||||
crew = SimpleCrew()
|
||||
@@ -237,3 +245,17 @@ def test_multiple_before_after_kickoff():
|
||||
def test_crew_name():
|
||||
crew = InternalCrew()
|
||||
assert crew._crew_name == "InternalCrew"
|
||||
|
||||
@tool
|
||||
def simple_tool():
|
||||
"""Return 'Hi!'"""
|
||||
return "Hi!"
|
||||
|
||||
def test_internal_crew_with_mcp():
|
||||
mock = Mock()
|
||||
mock.tools = [simple_tool]
|
||||
with patch("crewai_tools.MCPServerAdapter", return_value=mock) as adapter_mock:
|
||||
crew = InternalCrewWithMCP()
|
||||
assert crew.reporting_analyst().tools == [simple_tool]
|
||||
|
||||
adapter_mock.assert_called_once_with({"host": "localhost", "port": 8000})
|
||||
@@ -1,320 +0,0 @@
|
||||
"""Tests for Agent fallback LLM functionality."""
|
||||
|
||||
import pytest
|
||||
from unittest.mock import patch, MagicMock
|
||||
|
||||
from crewai import Agent, Task
|
||||
from crewai.llm import LLM
|
||||
from crewai.utilities.agent_utils import get_llm_response
|
||||
from crewai.utilities import Printer
|
||||
from litellm.exceptions import AuthenticationError, ContextWindowExceededError
|
||||
|
||||
|
||||
def test_agent_with_fallback_llms_basic():
|
||||
"""Test agent with fallback LLMs when primary fails."""
|
||||
primary_llm = LLM("gpt-4")
|
||||
fallback_llm = LLM("gpt-3.5-turbo")
|
||||
|
||||
agent = Agent(
|
||||
role="Test Agent",
|
||||
goal="Test fallback functionality",
|
||||
backstory="I test fallback LLMs",
|
||||
llm=primary_llm,
|
||||
fallback_llms=[fallback_llm]
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Simple test task",
|
||||
expected_output="Test output",
|
||||
agent=agent
|
||||
)
|
||||
|
||||
with patch.object(primary_llm, 'call') as mock_primary, \
|
||||
patch.object(fallback_llm, 'call') as mock_fallback:
|
||||
|
||||
mock_primary.side_effect = Exception("Primary LLM failed")
|
||||
mock_fallback.return_value = "Fallback response"
|
||||
|
||||
result = agent.execute_task(task)
|
||||
|
||||
assert result == "Fallback response"
|
||||
mock_primary.assert_called_once()
|
||||
mock_fallback.assert_called_once()
|
||||
|
||||
|
||||
def test_agent_fallback_llms_multiple():
|
||||
"""Test agent with multiple fallback LLMs."""
|
||||
primary_llm = LLM("gpt-4")
|
||||
fallback1 = LLM("gpt-3.5-turbo")
|
||||
fallback2 = LLM("claude-3-sonnet-20240229")
|
||||
|
||||
agent = Agent(
|
||||
role="Test Agent",
|
||||
goal="Test multiple fallbacks",
|
||||
backstory="I test multiple fallback LLMs",
|
||||
llm=primary_llm,
|
||||
fallback_llms=[fallback1, fallback2]
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Test task",
|
||||
expected_output="Test output",
|
||||
agent=agent
|
||||
)
|
||||
|
||||
with patch.object(primary_llm, 'call') as mock_primary, \
|
||||
patch.object(fallback1, 'call') as mock_fallback1, \
|
||||
patch.object(fallback2, 'call') as mock_fallback2:
|
||||
|
||||
mock_primary.side_effect = Exception("Primary failed")
|
||||
mock_fallback1.side_effect = Exception("Fallback 1 failed")
|
||||
mock_fallback2.return_value = "Fallback 2 response"
|
||||
|
||||
result = agent.execute_task(task)
|
||||
|
||||
assert result == "Fallback 2 response"
|
||||
mock_primary.assert_called_once()
|
||||
mock_fallback1.assert_called_once()
|
||||
mock_fallback2.assert_called_once()
|
||||
|
||||
|
||||
def test_agent_fallback_auth_error_skips_fallbacks():
|
||||
"""Test that authentication errors skip fallback attempts."""
|
||||
primary_llm = LLM("gpt-4")
|
||||
fallback_llm = LLM("gpt-3.5-turbo")
|
||||
|
||||
agent = Agent(
|
||||
role="Test Agent",
|
||||
goal="Test auth error handling",
|
||||
backstory="I test auth error handling",
|
||||
llm=primary_llm,
|
||||
fallback_llms=[fallback_llm]
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Test task",
|
||||
expected_output="Test output",
|
||||
agent=agent
|
||||
)
|
||||
|
||||
with patch.object(primary_llm, 'call') as mock_primary, \
|
||||
patch.object(fallback_llm, 'call') as mock_fallback, \
|
||||
pytest.raises(AuthenticationError):
|
||||
|
||||
mock_primary.side_effect = AuthenticationError(
|
||||
message="Invalid API key", llm_provider="openai", model="gpt-4"
|
||||
)
|
||||
|
||||
agent.execute_task(task)
|
||||
|
||||
mock_primary.assert_called_once()
|
||||
mock_fallback.assert_not_called()
|
||||
|
||||
|
||||
def test_agent_fallback_context_window_error():
|
||||
"""Test that context window errors try fallbacks."""
|
||||
primary_llm = LLM("gpt-4")
|
||||
fallback_llm = LLM("gpt-3.5-turbo")
|
||||
|
||||
agent = Agent(
|
||||
role="Test Agent",
|
||||
goal="Test context window error handling",
|
||||
backstory="I test context window error handling",
|
||||
llm=primary_llm,
|
||||
fallback_llms=[fallback_llm]
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Test task",
|
||||
expected_output="Test output",
|
||||
agent=agent
|
||||
)
|
||||
|
||||
with patch.object(primary_llm, 'call') as mock_primary, \
|
||||
patch.object(fallback_llm, 'call') as mock_fallback:
|
||||
|
||||
mock_primary.side_effect = ContextWindowExceededError(
|
||||
message="Context window exceeded", model="gpt-4", llm_provider="openai"
|
||||
)
|
||||
mock_fallback.return_value = "Fallback response"
|
||||
|
||||
result = agent.execute_task(task)
|
||||
|
||||
assert result == "Fallback response"
|
||||
mock_primary.assert_called_once()
|
||||
mock_fallback.assert_called_once()
|
||||
|
||||
|
||||
def test_agent_all_llms_fail():
|
||||
"""Test behavior when all LLMs fail."""
|
||||
primary_llm = LLM("gpt-4")
|
||||
fallback_llm = LLM("gpt-3.5-turbo")
|
||||
|
||||
agent = Agent(
|
||||
role="Test Agent",
|
||||
goal="Test all LLMs failing",
|
||||
backstory="I test all LLMs failing",
|
||||
llm=primary_llm,
|
||||
fallback_llms=[fallback_llm]
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Test task",
|
||||
expected_output="Test output",
|
||||
agent=agent
|
||||
)
|
||||
|
||||
with patch.object(primary_llm, 'call') as mock_primary, \
|
||||
patch.object(fallback_llm, 'call') as mock_fallback, \
|
||||
pytest.raises(Exception, match="Fallback failed"):
|
||||
|
||||
mock_primary.side_effect = Exception("Primary failed")
|
||||
mock_fallback.side_effect = Exception("Fallback failed")
|
||||
|
||||
agent.execute_task(task)
|
||||
|
||||
mock_primary.assert_called_once()
|
||||
mock_fallback.assert_called_once()
|
||||
|
||||
|
||||
def test_agent_backward_compatibility():
|
||||
"""Test that agents without fallback LLMs work as before."""
|
||||
agent = Agent(
|
||||
role="Test Agent",
|
||||
goal="Test backward compatibility",
|
||||
backstory="I test backward compatibility",
|
||||
llm=LLM("gpt-4")
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Test task",
|
||||
expected_output="Test output",
|
||||
agent=agent
|
||||
)
|
||||
|
||||
with patch.object(agent.llm, 'call') as mock_llm:
|
||||
mock_llm.return_value = "Primary response"
|
||||
|
||||
result = agent.execute_task(task)
|
||||
|
||||
assert result == "Primary response"
|
||||
mock_llm.assert_called_once()
|
||||
|
||||
|
||||
def test_get_llm_response_with_fallbacks():
|
||||
"""Test get_llm_response function directly with fallbacks."""
|
||||
primary_llm = MagicMock()
|
||||
fallback_llm = MagicMock()
|
||||
printer = Printer()
|
||||
|
||||
primary_llm.call.side_effect = Exception("Primary failed")
|
||||
fallback_llm.call.return_value = "Fallback success"
|
||||
|
||||
result = get_llm_response(
|
||||
llm=primary_llm,
|
||||
messages=[{"role": "user", "content": "test"}],
|
||||
callbacks=[],
|
||||
printer=printer,
|
||||
fallback_llms=[fallback_llm]
|
||||
)
|
||||
|
||||
assert result == "Fallback success"
|
||||
primary_llm.call.assert_called_once()
|
||||
fallback_llm.call.assert_called_once()
|
||||
|
||||
|
||||
def test_get_llm_response_no_fallbacks():
|
||||
"""Test get_llm_response function without fallbacks (backward compatibility)."""
|
||||
primary_llm = MagicMock()
|
||||
printer = Printer()
|
||||
|
||||
primary_llm.call.return_value = "Primary success"
|
||||
|
||||
result = get_llm_response(
|
||||
llm=primary_llm,
|
||||
messages=[{"role": "user", "content": "test"}],
|
||||
callbacks=[],
|
||||
printer=printer
|
||||
)
|
||||
|
||||
assert result == "Primary success"
|
||||
primary_llm.call.assert_called_once()
|
||||
|
||||
|
||||
def test_agent_fallback_llms_string_initialization():
|
||||
"""Test that fallback LLMs can be initialized with string model names."""
|
||||
agent = Agent(
|
||||
role="Test Agent",
|
||||
goal="Test string initialization",
|
||||
backstory="I test string initialization",
|
||||
llm="gpt-4",
|
||||
fallback_llms=["gpt-3.5-turbo", "claude-3-sonnet-20240229"]
|
||||
)
|
||||
|
||||
assert agent.fallback_llms is not None
|
||||
assert len(agent.fallback_llms) == 2
|
||||
assert hasattr(agent.fallback_llms[0], 'call')
|
||||
assert hasattr(agent.fallback_llms[1], 'call')
|
||||
|
||||
|
||||
def test_agent_primary_success_no_fallback():
|
||||
"""Test that fallback LLMs are not called when primary succeeds."""
|
||||
primary_llm = LLM("gpt-4")
|
||||
fallback_llm = LLM("gpt-3.5-turbo")
|
||||
|
||||
agent = Agent(
|
||||
role="Test Agent",
|
||||
goal="Test primary success",
|
||||
backstory="I test primary success",
|
||||
llm=primary_llm,
|
||||
fallback_llms=[fallback_llm]
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Test task",
|
||||
expected_output="Test output",
|
||||
agent=agent
|
||||
)
|
||||
|
||||
with patch.object(primary_llm, 'call') as mock_primary, \
|
||||
patch.object(fallback_llm, 'call') as mock_fallback:
|
||||
|
||||
mock_primary.return_value = "Primary success"
|
||||
|
||||
result = agent.execute_task(task)
|
||||
|
||||
assert result == "Primary success"
|
||||
mock_primary.assert_called_once()
|
||||
mock_fallback.assert_not_called()
|
||||
|
||||
|
||||
def test_agent_empty_response_triggers_fallback():
|
||||
"""Test that empty responses from primary LLM trigger fallback."""
|
||||
primary_llm = LLM("gpt-4")
|
||||
fallback_llm = LLM("gpt-3.5-turbo")
|
||||
|
||||
agent = Agent(
|
||||
role="Test Agent",
|
||||
goal="Test empty response handling",
|
||||
backstory="I test empty response handling",
|
||||
llm=primary_llm,
|
||||
fallback_llms=[fallback_llm]
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Test task",
|
||||
expected_output="Test output",
|
||||
agent=agent
|
||||
)
|
||||
|
||||
with patch.object(primary_llm, 'call') as mock_primary, \
|
||||
patch.object(fallback_llm, 'call') as mock_fallback:
|
||||
|
||||
mock_primary.return_value = ""
|
||||
mock_fallback.return_value = "Fallback response"
|
||||
|
||||
result = agent.execute_task(task)
|
||||
|
||||
assert result == "Fallback response"
|
||||
mock_primary.assert_called_once()
|
||||
mock_fallback.assert_called_once()
|
||||
252
uv.lock
generated
@@ -337,15 +337,6 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/a2/ee/3fd29bf416eb4f1c5579cf12bf393ae954099258abd7bde03c4f9716ef6b/autoflake-2.3.1-py3-none-any.whl", hash = "sha256:3ae7495db9084b7b32818b4140e6dc4fc280b712fb414f5b8fe57b0a8e85a840", size = 32483 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "babel"
|
||||
version = "2.16.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/2a/74/f1bc80f23eeba13393b7222b11d95ca3af2c1e28edca18af487137eefed9/babel-2.16.0.tar.gz", hash = "sha256:d1f3554ca26605fe173f3de0c65f750f5a42f924499bf134de6423582298e316", size = 9348104 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/ed/20/bc79bc575ba2e2a7f70e8a1155618bb1301eaa5132a8271373a6903f73f8/babel-2.16.0-py3-none-any.whl", hash = "sha256:368b5b98b37c06b7daf6696391c3240c938b37767d4584413e8438c5c435fa8b", size = 9587599 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "backoff"
|
||||
version = "2.2.1"
|
||||
@@ -787,11 +778,6 @@ tools = [
|
||||
[package.dev-dependencies]
|
||||
dev = [
|
||||
{ name = "cairosvg" },
|
||||
{ name = "mkdocs" },
|
||||
{ name = "mkdocs-material" },
|
||||
{ name = "mkdocs-material-extensions" },
|
||||
{ name = "mkdocstrings" },
|
||||
{ name = "mkdocstrings-python" },
|
||||
{ name = "mypy" },
|
||||
{ name = "pillow" },
|
||||
{ name = "pre-commit" },
|
||||
@@ -846,11 +832,6 @@ requires-dist = [
|
||||
[package.metadata.requires-dev]
|
||||
dev = [
|
||||
{ name = "cairosvg", specifier = ">=2.7.1" },
|
||||
{ name = "mkdocs", specifier = ">=1.4.3" },
|
||||
{ name = "mkdocs-material", specifier = ">=9.5.7" },
|
||||
{ name = "mkdocs-material-extensions", specifier = ">=1.3.1" },
|
||||
{ name = "mkdocstrings", specifier = ">=0.22.0" },
|
||||
{ name = "mkdocstrings-python", specifier = ">=1.1.2" },
|
||||
{ name = "mypy", specifier = ">=1.10.0" },
|
||||
{ name = "pillow", specifier = ">=10.2.0" },
|
||||
{ name = "pre-commit", specifier = ">=3.6.0" },
|
||||
@@ -1428,18 +1409,6 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/c6/b2/454d6e7f0158951d8a78c2e1eb4f69ae81beb8dca5fee9809c6c99e9d0d0/fsspec-2024.10.0-py3-none-any.whl", hash = "sha256:03b9a6785766a4de40368b88906366755e2819e758b83705c88cd7cb5fe81871", size = 179641 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "ghp-import"
|
||||
version = "2.1.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "python-dateutil" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/d9/29/d40217cbe2f6b1359e00c6c307bb3fc876ba74068cbab3dde77f03ca0dc4/ghp-import-2.1.0.tar.gz", hash = "sha256:9c535c4c61193c2df8871222567d7fd7e5014d835f97dc7b7439069e2413d343", size = 10943 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/f7/ec/67fbef5d497f86283db54c22eec6f6140243aae73265799baaaa19cd17fb/ghp_import-2.1.0-py3-none-any.whl", hash = "sha256:8337dd7b50877f163d4c0289bc1f1c7f127550241988d568c1db512c4324a619", size = 11034 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "google-auth"
|
||||
version = "2.35.0"
|
||||
@@ -1531,18 +1500,6 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/ac/38/08cc303ddddc4b3d7c628c3039a61a3aae36c241ed01393d00c2fd663473/greenlet-3.1.1-cp313-cp313t-musllinux_1_1_x86_64.whl", hash = "sha256:411f015496fec93c1c8cd4e5238da364e1da7a124bcb293f085bf2860c32c6f6", size = 1142112 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "griffe"
|
||||
version = "1.5.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "colorama" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/d4/c9/8167810358ca129839156dc002526e7398b5fad4a9d7b6e88b875e802d0d/griffe-1.5.1.tar.gz", hash = "sha256:72964f93e08c553257706d6cd2c42d1c172213feb48b2be386f243380b405d4b", size = 384113 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/ab/00/e693a155da0a2a72fd2df75b8fe338146cae59d590ad6f56800adde90cb5/griffe-1.5.1-py3-none-any.whl", hash = "sha256:ad6a7980f8c424c9102160aafa3bcdf799df0e75f7829d75af9ee5aef656f860", size = 127132 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "grpcio"
|
||||
version = "1.67.0"
|
||||
@@ -2360,15 +2317,6 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/48/22/bc14c6f02e6dccaafb3eba95764c8f096714260c2aa5f76f654fd16a23dd/Mako-1.3.6-py3-none-any.whl", hash = "sha256:a91198468092a2f1a0de86ca92690fb0cfc43ca90ee17e15d93662b4c04b241a", size = 78557 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "markdown"
|
||||
version = "3.7"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/54/28/3af612670f82f4c056911fbbbb42760255801b3068c48de792d354ff4472/markdown-3.7.tar.gz", hash = "sha256:2ae2471477cfd02dbbf038d5d9bc226d40def84b4fe2986e49b59b6b472bbed2", size = 357086 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/3f/08/83871f3c50fc983b88547c196d11cf8c3340e37c32d2e9d6152abe2c61f7/Markdown-3.7-py3-none-any.whl", hash = "sha256:7eb6df5690b81a1d7942992c97fad2938e956e79df20cbc6186e9c3a77b1c803", size = 106349 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "markdown-it-py"
|
||||
version = "3.0.0"
|
||||
@@ -2499,131 +2447,6 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/0c/fa/a2043c5a228077423678d9e7e0fe3aab89ab9a625a4f171a30967de1bbb3/mem0ai-0.1.94-py3-none-any.whl", hash = "sha256:862aaccf4ec41d65a5c935dc49c7b8175c96f6b60b29abeda338d8a335027e2c", size = 143077 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "mergedeep"
|
||||
version = "1.3.4"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/3a/41/580bb4006e3ed0361b8151a01d324fb03f420815446c7def45d02f74c270/mergedeep-1.3.4.tar.gz", hash = "sha256:0096d52e9dad9939c3d975a774666af186eda617e6ca84df4c94dec30004f2a8", size = 4661 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/2c/19/04f9b178c2d8a15b076c8b5140708fa6ffc5601fb6f1e975537072df5b2a/mergedeep-1.3.4-py3-none-any.whl", hash = "sha256:70775750742b25c0d8f36c55aed03d24c3384d17c951b3175d898bd778ef0307", size = 6354 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "mkdocs"
|
||||
version = "1.6.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "click" },
|
||||
{ name = "colorama", marker = "platform_system == 'Windows'" },
|
||||
{ name = "ghp-import" },
|
||||
{ name = "jinja2" },
|
||||
{ name = "markdown" },
|
||||
{ name = "markupsafe" },
|
||||
{ name = "mergedeep" },
|
||||
{ name = "mkdocs-get-deps" },
|
||||
{ name = "packaging" },
|
||||
{ name = "pathspec" },
|
||||
{ name = "pyyaml" },
|
||||
{ name = "pyyaml-env-tag" },
|
||||
{ name = "watchdog" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/bc/c6/bbd4f061bd16b378247f12953ffcb04786a618ce5e904b8c5a01a0309061/mkdocs-1.6.1.tar.gz", hash = "sha256:7b432f01d928c084353ab39c57282f29f92136665bdd6abf7c1ec8d822ef86f2", size = 3889159 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/22/5b/dbc6a8cddc9cfa9c4971d59fb12bb8d42e161b7e7f8cc89e49137c5b279c/mkdocs-1.6.1-py3-none-any.whl", hash = "sha256:db91759624d1647f3f34aa0c3f327dd2601beae39a366d6e064c03468d35c20e", size = 3864451 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "mkdocs-autorefs"
|
||||
version = "1.2.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "markdown" },
|
||||
{ name = "markupsafe" },
|
||||
{ name = "mkdocs" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/fb/ae/0f1154c614d6a8b8a36fff084e5b82af3a15f7d2060cf0dcdb1c53297a71/mkdocs_autorefs-1.2.0.tar.gz", hash = "sha256:a86b93abff653521bda71cf3fc5596342b7a23982093915cb74273f67522190f", size = 40262 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/71/26/4d39d52ea2219604053a4d05b98e90d6a335511cc01806436ec4886b1028/mkdocs_autorefs-1.2.0-py3-none-any.whl", hash = "sha256:d588754ae89bd0ced0c70c06f58566a4ee43471eeeee5202427da7de9ef85a2f", size = 16522 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "mkdocs-get-deps"
|
||||
version = "0.2.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "mergedeep" },
|
||||
{ name = "platformdirs" },
|
||||
{ name = "pyyaml" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/98/f5/ed29cd50067784976f25ed0ed6fcd3c2ce9eb90650aa3b2796ddf7b6870b/mkdocs_get_deps-0.2.0.tar.gz", hash = "sha256:162b3d129c7fad9b19abfdcb9c1458a651628e4b1dea628ac68790fb3061c60c", size = 10239 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/9f/d4/029f984e8d3f3b6b726bd33cafc473b75e9e44c0f7e80a5b29abc466bdea/mkdocs_get_deps-0.2.0-py3-none-any.whl", hash = "sha256:2bf11d0b133e77a0dd036abeeb06dec8775e46efa526dc70667d8863eefc6134", size = 9521 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "mkdocs-material"
|
||||
version = "9.5.42"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "babel" },
|
||||
{ name = "colorama" },
|
||||
{ name = "jinja2" },
|
||||
{ name = "markdown" },
|
||||
{ name = "mkdocs" },
|
||||
{ name = "mkdocs-material-extensions" },
|
||||
{ name = "paginate" },
|
||||
{ name = "pygments" },
|
||||
{ name = "pymdown-extensions" },
|
||||
{ name = "regex" },
|
||||
{ name = "requests" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/f9/33/b3343ed975fbbd6798b8d8a7c4f1bf8489cc321fc8fd426eba3d87b0242e/mkdocs_material-9.5.42.tar.gz", hash = "sha256:92779b5e9b5934540c574c11647131d217dc540dce72b05feeda088c8eb1b8f2", size = 3963891 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/55/55/ad3e6a60ac1e8e76025543c49c1f24ecd80fb38e8a57000403bf2f0a4293/mkdocs_material-9.5.42-py3-none-any.whl", hash = "sha256:452a7c5d21284b373f36b981a2cbebfff59263feebeede1bc28652e9c5bbe316", size = 8672619 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "mkdocs-material-extensions"
|
||||
version = "1.3.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/79/9b/9b4c96d6593b2a541e1cb8b34899a6d021d208bb357042823d4d2cabdbe7/mkdocs_material_extensions-1.3.1.tar.gz", hash = "sha256:10c9511cea88f568257f960358a467d12b970e1f7b2c0e5fb2bb48cab1928443", size = 11847 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/5b/54/662a4743aa81d9582ee9339d4ffa3c8fd40a4965e033d77b9da9774d3960/mkdocs_material_extensions-1.3.1-py3-none-any.whl", hash = "sha256:adff8b62700b25cb77b53358dad940f3ef973dd6db797907c49e3c2ef3ab4e31", size = 8728 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "mkdocstrings"
|
||||
version = "0.26.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "click" },
|
||||
{ name = "jinja2" },
|
||||
{ name = "markdown" },
|
||||
{ name = "markupsafe" },
|
||||
{ name = "mkdocs" },
|
||||
{ name = "mkdocs-autorefs" },
|
||||
{ name = "platformdirs" },
|
||||
{ name = "pymdown-extensions" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/c0/76/0475d10d27f3384df3a6ddfdf4a4fdfef83766f77cd4e327d905dc956c15/mkdocstrings-0.26.2.tar.gz", hash = "sha256:34a8b50f1e6cfd29546c6c09fbe02154adfb0b361bb758834bf56aa284ba876e", size = 92512 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/80/b6/4ee320d7c313da3774eff225875eb278f7e6bb26a9cd8e680b8dbc38fdea/mkdocstrings-0.26.2-py3-none-any.whl", hash = "sha256:1248f3228464f3b8d1a15bd91249ce1701fe3104ac517a5f167a0e01ca850ba5", size = 29716 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "mkdocstrings-python"
|
||||
version = "1.12.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "griffe" },
|
||||
{ name = "mkdocs-autorefs" },
|
||||
{ name = "mkdocstrings" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/23/ec/cb6debe2db77f1ef42b25b21d93b5021474de3037cd82385e586aee72545/mkdocstrings_python-1.12.2.tar.gz", hash = "sha256:7a1760941c0b52a2cd87b960a9e21112ffe52e7df9d0b9583d04d47ed2e186f3", size = 168207 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/5b/c1/ac524e1026d9580cbc654b5d19f5843c8b364a66d30f956372cd09fd2f92/mkdocstrings_python-1.12.2-py3-none-any.whl", hash = "sha256:7f7d40d6db3cb1f5d19dbcd80e3efe4d0ba32b073272c0c0de9de2e604eda62a", size = 111759 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "mmh3"
|
||||
version = "4.1.0"
|
||||
@@ -3389,15 +3212,6 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/ec/1a/610693ac4ee14fcdf2d9bf3c493370e4f2ef7ae2e19217d7a237ff42367d/packaging-23.2-py3-none-any.whl", hash = "sha256:8c491190033a9af7e1d931d0b5dacc2ef47509b34dd0de67ed209b5203fc88c7", size = 53011 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "paginate"
|
||||
version = "0.5.7"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/ec/46/68dde5b6bc00c1296ec6466ab27dddede6aec9af1b99090e1107091b3b84/paginate-0.5.7.tar.gz", hash = "sha256:22bd083ab41e1a8b4f3690544afb2c60c25e5c9a63a30fa2f483f6c60c8e5945", size = 19252 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/90/96/04b8e52da071d28f5e21a805b19cb9390aa17a47462ac87f5e2696b9566d/paginate-0.5.7-py2.py3-none-any.whl", hash = "sha256:b885e2af73abcf01d9559fd5216b57ef722f8c42affbb63942377668e35c7591", size = 13746 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pandas"
|
||||
version = "2.2.3"
|
||||
@@ -3464,15 +3278,6 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/c6/ac/dac4a63f978e4dcb3c6d3a78c4d8e0192a113d288502a1216950c41b1027/parso-0.8.4-py2.py3-none-any.whl", hash = "sha256:a418670a20291dacd2dddc80c377c5c3791378ee1e8d12bffc35420643d43f18", size = 103650 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pathspec"
|
||||
version = "0.12.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/ca/bc/f35b8446f4531a7cb215605d100cd88b7ac6f44ab3fc94870c120ab3adbf/pathspec-0.12.1.tar.gz", hash = "sha256:a482d51503a1ab33b1c67a6c3813a26953dbdc71c31dacaef9a838c4e29f5712", size = 51043 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/cc/20/ff623b09d963f88bfde16306a54e12ee5ea43e9b597108672ff3a408aad6/pathspec-0.12.1-py3-none-any.whl", hash = "sha256:a0d503e138a4c123b27490a4f7beda6a01c6f288df0e4a8b79c7eb0dc7b4cc08", size = 31191 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pdfminer-six"
|
||||
version = "20231228"
|
||||
@@ -4055,19 +3860,6 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/79/84/0fdf9b18ba31d69877bd39c9cd6052b47f3761e9910c15de788e519f079f/PyJWT-2.9.0-py3-none-any.whl", hash = "sha256:3b02fb0f44517787776cf48f2ae25d8e14f300e6d7545a4315cee571a415e850", size = 22344 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pymdown-extensions"
|
||||
version = "10.11.2"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "markdown" },
|
||||
{ name = "pyyaml" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/f4/71/2730a20e9e3752393d78998347f8b1085ef9c417646ea9befbeef221e3c4/pymdown_extensions-10.11.2.tar.gz", hash = "sha256:bc8847ecc9e784a098efd35e20cba772bc5a1b529dfcef9dc1972db9021a1049", size = 830241 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/c2/35/c0edf199257ef0a7d407d29cd51c4e70d1dad4370a5f44deb65a7a5475e2/pymdown_extensions-10.11.2-py3-none-any.whl", hash = "sha256:41cdde0a77290e480cf53892f5c5e50921a7ee3e5cd60ba91bf19837b33badcf", size = 259044 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pypdf"
|
||||
version = "5.0.1"
|
||||
@@ -4444,18 +4236,6 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/fa/de/02b54f42487e3d3c6efb3f89428677074ca7bf43aae402517bc7cca949f3/PyYAML-6.0.2-cp313-cp313-win_amd64.whl", hash = "sha256:8388ee1976c416731879ac16da0aff3f63b286ffdd57cdeb95f3f2e085687563", size = 156446 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "pyyaml-env-tag"
|
||||
version = "0.1"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "pyyaml" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/fb/8e/da1c6c58f751b70f8ceb1eb25bc25d524e8f14fe16edcce3f4e3ba08629c/pyyaml_env_tag-0.1.tar.gz", hash = "sha256:70092675bda14fdec33b31ba77e7543de9ddc88f2e5b99160396572d11525bdb", size = 5631 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/5a/66/bbb1dd374f5c870f59c5bb1db0e18cbe7fa739415a24cbd95b2d1f5ae0c4/pyyaml_env_tag-0.1-py3-none-any.whl", hash = "sha256:af31106dec8a4d68c60207c1886031cbf839b68aa7abccdb19868200532c2069", size = 3911 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "qdrant-client"
|
||||
version = "1.12.0"
|
||||
@@ -5666,38 +5446,6 @@ wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/c8/15/828ec11907aee2349a9342fa71fba4ba7f3af938162a382dd7da339dea16/virtualenv-20.27.0-py3-none-any.whl", hash = "sha256:44a72c29cceb0ee08f300b314848c86e57bf8d1f13107a5e671fb9274138d655", size = 3110969 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "watchdog"
|
||||
version = "5.0.3"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/a2/48/a86139aaeab2db0a2482676f64798d8ac4d2dbb457523f50ab37bf02ce2c/watchdog-5.0.3.tar.gz", hash = "sha256:108f42a7f0345042a854d4d0ad0834b741d421330d5f575b81cb27b883500176", size = 129556 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/05/2b/dd2081aab6fc9e785c2eee7146d3c6de58e607f4e70049d715cd170cbf77/watchdog-5.0.3-cp310-cp310-macosx_10_9_universal2.whl", hash = "sha256:85527b882f3facda0579bce9d743ff7f10c3e1e0db0a0d0e28170a7d0e5ce2ea", size = 96652 },
|
||||
{ url = "https://files.pythonhosted.org/packages/9e/4f/f643c0a720d16ef7316aea06a79b96e229e59df4e0d83bec5e12713c1f29/watchdog-5.0.3-cp310-cp310-macosx_10_9_x86_64.whl", hash = "sha256:53adf73dcdc0ef04f7735066b4a57a4cd3e49ef135daae41d77395f0b5b692cb", size = 88651 },
|
||||
{ url = "https://files.pythonhosted.org/packages/2b/72/acb22067d1f18161914c9b1087c703d63638131a9fde78090da290663407/watchdog-5.0.3-cp310-cp310-macosx_11_0_arm64.whl", hash = "sha256:e25adddab85f674acac303cf1f5835951345a56c5f7f582987d266679979c75b", size = 89289 },
|
||||
{ url = "https://files.pythonhosted.org/packages/70/34/946f08602f8b8e6af45bc725e4a8013975a34883ab5570bd0d827a4c9829/watchdog-5.0.3-cp311-cp311-macosx_10_9_universal2.whl", hash = "sha256:f01f4a3565a387080dc49bdd1fefe4ecc77f894991b88ef927edbfa45eb10818", size = 96650 },
|
||||
{ url = "https://files.pythonhosted.org/packages/96/2b/b84e35d49e8b0bad77e5d086fc1e2c6c833bbfe74d53144cfe8b26117eff/watchdog-5.0.3-cp311-cp311-macosx_10_9_x86_64.whl", hash = "sha256:91b522adc25614cdeaf91f7897800b82c13b4b8ac68a42ca959f992f6990c490", size = 88653 },
|
||||
{ url = "https://files.pythonhosted.org/packages/d5/3f/41b5d77c10f450b79921c17b7d0b416616048867bfe63acaa072a619a0cb/watchdog-5.0.3-cp311-cp311-macosx_11_0_arm64.whl", hash = "sha256:d52db5beb5e476e6853da2e2d24dbbbed6797b449c8bf7ea118a4ee0d2c9040e", size = 89286 },
|
||||
{ url = "https://files.pythonhosted.org/packages/1c/9b/8b206a928c188fdeb7b12e1c795199534cd44bdef223b8470129016009dd/watchdog-5.0.3-cp312-cp312-macosx_10_13_universal2.whl", hash = "sha256:94d11b07c64f63f49876e0ab8042ae034674c8653bfcdaa8c4b32e71cfff87e8", size = 96739 },
|
||||
{ url = "https://files.pythonhosted.org/packages/e1/26/129ca9cd0f8016672f37000010c2fedc0b86816e894ebdc0af9bb04a6439/watchdog-5.0.3-cp312-cp312-macosx_10_13_x86_64.whl", hash = "sha256:349c9488e1d85d0a58e8cb14222d2c51cbc801ce11ac3936ab4c3af986536926", size = 88708 },
|
||||
{ url = "https://files.pythonhosted.org/packages/8f/b3/5e10ec32f0c429cdb55b1369066d6e83faf9985b3a53a4e37bb5c5e29aa0/watchdog-5.0.3-cp312-cp312-macosx_11_0_arm64.whl", hash = "sha256:53a3f10b62c2d569e260f96e8d966463dec1a50fa4f1b22aec69e3f91025060e", size = 89309 },
|
||||
{ url = "https://files.pythonhosted.org/packages/54/c4/49af4ab00bcfb688e9962eace2edda07a2cf89b9699ea536da48e8585cff/watchdog-5.0.3-cp313-cp313-macosx_10_13_universal2.whl", hash = "sha256:950f531ec6e03696a2414b6308f5c6ff9dab7821a768c9d5788b1314e9a46ca7", size = 96740 },
|
||||
{ url = "https://files.pythonhosted.org/packages/96/a4/b24de77cc9ae424c1687c9d4fb15aa560d7d7b28ba559aca72f781d0202b/watchdog-5.0.3-cp313-cp313-macosx_10_13_x86_64.whl", hash = "sha256:ae6deb336cba5d71476caa029ceb6e88047fc1dc74b62b7c4012639c0b563906", size = 88711 },
|
||||
{ url = "https://files.pythonhosted.org/packages/a4/71/3f2e9fe8403386b99d788868955b3a790f7a09721501a7e1eb58f514ffaa/watchdog-5.0.3-cp313-cp313-macosx_11_0_arm64.whl", hash = "sha256:1021223c08ba8d2d38d71ec1704496471ffd7be42cfb26b87cd5059323a389a1", size = 89319 },
|
||||
{ url = "https://files.pythonhosted.org/packages/a2/d6/1d1ca81c75d903eca3fdb7061d93845485b58a5ba182d146843b88fc51c2/watchdog-5.0.3-pp310-pypy310_pp73-macosx_10_15_x86_64.whl", hash = "sha256:90a67d7857adb1d985aca232cc9905dd5bc4803ed85cfcdcfcf707e52049eda7", size = 88172 },
|
||||
{ url = "https://files.pythonhosted.org/packages/47/bb/d5e0abcfd6d729029a24766682e062526db8b59e9ae0c94aff509e0fd2b9/watchdog-5.0.3-pp310-pypy310_pp73-macosx_11_0_arm64.whl", hash = "sha256:720ef9d3a4f9ca575a780af283c8fd3a0674b307651c1976714745090da5a9e8", size = 88644 },
|
||||
{ url = "https://files.pythonhosted.org/packages/60/33/7cb71c9df9a77b6927ee5f48d25e1de5562ce0fa7e0c56dcf2b0472e64a2/watchdog-5.0.3-py3-none-manylinux2014_aarch64.whl", hash = "sha256:dd021efa85970bd4824acacbb922066159d0f9e546389a4743d56919b6758b91", size = 79335 },
|
||||
{ url = "https://files.pythonhosted.org/packages/f6/91/320bc1496cf951a3cf93a7ffd18a581f0792c304be963d943e0e608c2919/watchdog-5.0.3-py3-none-manylinux2014_armv7l.whl", hash = "sha256:78864cc8f23dbee55be34cc1494632a7ba30263951b5b2e8fc8286b95845f82c", size = 79334 },
|
||||
{ url = "https://files.pythonhosted.org/packages/8b/2c/567c5e042ed667d3544c43d48a65cf853450a2d2a9089d9523a65f195e94/watchdog-5.0.3-py3-none-manylinux2014_i686.whl", hash = "sha256:1e9679245e3ea6498494b3028b90c7b25dbb2abe65c7d07423ecfc2d6218ff7c", size = 79333 },
|
||||
{ url = "https://files.pythonhosted.org/packages/c3/f0/64059fe162ef3274662e67bbdea6c45b3cd53e846d5bd1365fcdc3dc1d15/watchdog-5.0.3-py3-none-manylinux2014_ppc64.whl", hash = "sha256:9413384f26b5d050b6978e6fcd0c1e7f0539be7a4f1a885061473c5deaa57221", size = 79334 },
|
||||
{ url = "https://files.pythonhosted.org/packages/f6/d9/19b7d02965be2801e2d0f6f4bde23e4ae172620071b65430fa0c2f8441ac/watchdog-5.0.3-py3-none-manylinux2014_ppc64le.whl", hash = "sha256:294b7a598974b8e2c6123d19ef15de9abcd282b0fbbdbc4d23dfa812959a9e05", size = 79333 },
|
||||
{ url = "https://files.pythonhosted.org/packages/cb/a1/5393ac6d0b095d3a44946b09258e9b5f22cb2fb67bcfa419dd868478826c/watchdog-5.0.3-py3-none-manylinux2014_s390x.whl", hash = "sha256:26dd201857d702bdf9d78c273cafcab5871dd29343748524695cecffa44a8d97", size = 79332 },
|
||||
{ url = "https://files.pythonhosted.org/packages/a0/58/edec25190b6403caf4426dd418234f2358a106634b7d6aa4aec6939b104f/watchdog-5.0.3-py3-none-manylinux2014_x86_64.whl", hash = "sha256:0f9332243355643d567697c3e3fa07330a1d1abf981611654a1f2bf2175612b7", size = 79334 },
|
||||
{ url = "https://files.pythonhosted.org/packages/97/69/cfb2d17ba8aabc73be2e2d03c8c319b1f32053a02c4b571852983aa24ff2/watchdog-5.0.3-py3-none-win32.whl", hash = "sha256:c66f80ee5b602a9c7ab66e3c9f36026590a0902db3aea414d59a2f55188c1f49", size = 79320 },
|
||||
{ url = "https://files.pythonhosted.org/packages/91/b4/2b5b59358dadfa2c8676322f955b6c22cde4937602f40490e2f7403e548e/watchdog-5.0.3-py3-none-win_amd64.whl", hash = "sha256:f00b4cf737f568be9665563347a910f8bdc76f88c2970121c86243c8cfdf90e9", size = 79325 },
|
||||
{ url = "https://files.pythonhosted.org/packages/38/b8/0aa69337651b3005f161f7f494e59188a1d8d94171666900d26d29d10f69/watchdog-5.0.3-py3-none-win_ia64.whl", hash = "sha256:49f4d36cb315c25ea0d946e018c01bb028048023b9e103d3d3943f58e109dd45", size = 79324 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
name = "watchfiles"
|
||||
version = "0.24.0"
|
||||
|
||||