Compare commits

..

2 Commits

Author SHA1 Message Date
Brandon Hancock (bhancock_ai)
1d8584fe33 Merge branch 'main' into bugfix/plot_cli 2024-10-24 12:15:21 -04:00
Brandon Hancock
5af7df8031 update plot command 2024-10-24 12:12:33 -04:00
298 changed files with 23883 additions and 27566 deletions

View File

@@ -65,6 +65,7 @@ body:
- '3.10'
- '3.11'
- '3.12'
- '3.13'
validations:
required: true
- type: input
@@ -112,4 +113,4 @@ body:
label: Additional context
description: Add any other context about the problem here.
validations:
required: true
required: true

19
.github/security.md vendored
View File

@@ -1,19 +0,0 @@
CrewAI takes the security of our software products and services seriously, which includes all source code repositories managed through our GitHub organization.
If you believe you have found a security vulnerability in any CrewAI product or service, please report it to us as described below.
## Reporting a Vulnerability
Please do not report security vulnerabilities through public GitHub issues.
To report a vulnerability, please email us at security@crewai.com.
Please include the requested information listed below so that we can triage your report more quickly
- Type of issue (e.g. SQL injection, cross-site scripting, etc.)
- Full paths of source file(s) related to the manifestation of the issue
- The location of the affected source code (tag/branch/commit or direct URL)
- Any special configuration required to reproduce the issue
- Step-by-step instructions to reproduce the issue (please include screenshots if needed)
- Proof-of-concept or exploit code (if possible)
- Impact of the issue, including how an attacker might exploit the issue
Once we have received your report, we will respond to you at the email address you provide. If the issue is confirmed, we will release a patch as soon as possible depending on the complexity of the issue.
At this time, we are not offering a bug bounty program. Any rewards will be at our discretion.

View File

@@ -6,11 +6,11 @@ jobs:
lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- uses: actions/checkout@v3
- name: Install Requirements
run: |
pip install ruff
- name: Run Ruff Linter
run: ruff check
run: ruff check --exclude "templates","__init__.py"

View File

@@ -13,10 +13,10 @@ jobs:
steps:
- name: Checkout code
uses: actions/checkout@v4
uses: actions/checkout@v2
- name: Setup Python
uses: actions/setup-python@v5
uses: actions/setup-python@v4
with:
python-version: '3.10'
@@ -25,7 +25,7 @@ jobs:
run: echo "::set-output name=hash::$(sha256sum requirements-doc.txt | awk '{print $1}')"
- name: Setup cache
uses: actions/cache@v4
uses: actions/cache@v3
with:
key: mkdocs-material-${{ steps.req-hash.outputs.hash }}
path: .cache
@@ -42,4 +42,4 @@ jobs:
GH_TOKEN: ${{ secrets.GH_TOKEN }}
- name: Build and deploy MkDocs
run: mkdocs gh-deploy --force
run: mkdocs gh-deploy --force

View File

@@ -11,7 +11,7 @@ jobs:
uses: actions/checkout@v4
- name: Set up Python
uses: actions/setup-python@v5
uses: actions/setup-python@v4
with:
python-version: "3.11.9"
@@ -19,5 +19,5 @@ jobs:
run: pip install bandit
- name: Run Bandit
run: bandit -c pyproject.toml -r src/ -ll
run: bandit -c pyproject.toml -r src/ -lll

View File

@@ -1,10 +1,5 @@
name: Mark stale issues and pull requests
permissions:
contents: write
issues: write
pull-requests: write
on:
schedule:
- cron: '10 12 * * *'
@@ -13,6 +8,9 @@ on:
jobs:
stale:
runs-on: ubuntu-latest
permissions:
issues: write
pull-requests: write
steps:
- uses: actions/stale@v9
with:

View File

@@ -23,10 +23,10 @@ jobs:
- name: Set up Python
run: uv python install 3.12.8
run: uv python install 3.11.9
- name: Install the project
run: uv sync --dev --all-extras
run: uv sync --dev
- name: Run tests
run: uv run pytest tests -vv
run: uv run pytest tests

View File

@@ -14,7 +14,7 @@ jobs:
uses: actions/checkout@v4
- name: Setup Python
uses: actions/setup-python@v5
uses: actions/setup-python@v4
with:
python-version: "3.11.9"

4
.gitignore vendored
View File

@@ -17,7 +17,3 @@ rc-tests/*
temp/*
.vscode/*
crew_tasks_output.json
.codesight
.mypy_cache
.ruff_cache
.venv

View File

@@ -1,7 +1,9 @@
repos:
- repo: https://github.com/astral-sh/ruff-pre-commit
rev: v0.8.2
rev: v0.4.4
hooks:
- id: ruff
args: ["--fix"]
exclude: "templates"
- id: ruff-format
exclude: "templates"

View File

@@ -1,9 +0,0 @@
exclude = [
"templates",
"__init__.py",
]
[lint]
select = [
"I", # isort rules
]

189
README.md
View File

@@ -4,7 +4,7 @@
# **CrewAI**
🤖 **CrewAI**: Production-grade framework for orchestrating sophisticated AI agent systems. From simple automations to complex real-world applications, CrewAI provides precise control and deep customization. By fostering collaborative intelligence through flexible, production-ready architecture, CrewAI empowers agents to work together seamlessly, tackling complex business challenges with predictable, consistent results.
🤖 **CrewAI**: Cutting-edge framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.
<h3>
@@ -22,17 +22,13 @@
- [Why CrewAI?](#why-crewai)
- [Getting Started](#getting-started)
- [Key Features](#key-features)
- [Understanding Flows and Crews](#understanding-flows-and-crews)
- [CrewAI vs LangGraph](#how-crewai-compares)
- [Examples](#examples)
- [Quick Tutorial](#quick-tutorial)
- [Write Job Descriptions](#write-job-descriptions)
- [Trip Planner](#trip-planner)
- [Stock Analysis](#stock-analysis)
- [Using Crews and Flows Together](#using-crews-and-flows-together)
- [Connecting Your Crew to a Model](#connecting-your-crew-to-a-model)
- [How CrewAI Compares](#how-crewai-compares)
- [Frequently Asked Questions (FAQ)](#frequently-asked-questions-faq)
- [Contribution](#contribution)
- [Telemetry](#telemetry)
- [License](#license)
@@ -40,51 +36,22 @@
## Why CrewAI?
The power of AI collaboration has too much to offer.
CrewAI is a standalone framework, built from the ground up without dependencies on Langchain or other agent frameworks. It's designed to enable AI agents to assume roles, share goals, and operate in a cohesive unit - much like a well-oiled crew. Whether you're building a smart assistant platform, an automated customer service ensemble, or a multi-agent research team, CrewAI provides the backbone for sophisticated multi-agent interactions.
CrewAI is designed to enable AI agents to assume roles, share goals, and operate in a cohesive unit - much like a well-oiled crew. Whether you're building a smart assistant platform, an automated customer service ensemble, or a multi-agent research team, CrewAI provides the backbone for sophisticated multi-agent interactions.
## Getting Started
### Learning Resources
Learn CrewAI through our comprehensive courses:
- [Multi AI Agent Systems with CrewAI](https://www.deeplearning.ai/short-courses/multi-ai-agent-systems-with-crewai/) - Master the fundamentals of multi-agent systems
- [Practical Multi AI Agents and Advanced Use Cases](https://www.deeplearning.ai/short-courses/practical-multi-ai-agents-and-advanced-use-cases-with-crewai/) - Deep dive into advanced implementations
### Understanding Flows and Crews
CrewAI offers two powerful, complementary approaches that work seamlessly together to build sophisticated AI applications:
1. **Crews**: Teams of AI agents with true autonomy and agency, working together to accomplish complex tasks through role-based collaboration. Crews enable:
- Natural, autonomous decision-making between agents
- Dynamic task delegation and collaboration
- Specialized roles with defined goals and expertise
- Flexible problem-solving approaches
2. **Flows**: Production-ready, event-driven workflows that deliver precise control over complex automations. Flows provide:
- Fine-grained control over execution paths for real-world scenarios
- Secure, consistent state management between tasks
- Clean integration of AI agents with production Python code
- Conditional branching for complex business logic
The true power of CrewAI emerges when combining Crews and Flows. This synergy allows you to:
- Build complex, production-grade applications
- Balance autonomy with precise control
- Handle sophisticated real-world scenarios
- Maintain clean, maintainable code structure
### Getting Started with Installation
To get started with CrewAI, follow these simple steps:
### 1. Installation
Ensure you have Python >=3.10 <3.13 installed on your system. CrewAI uses [UV](https://docs.astral.sh/uv/) for dependency management and package handling, offering a seamless setup and execution experience.
Ensure you have Python >=3.10 <=3.13 installed on your system. CrewAI uses [UV](https://docs.astral.sh/uv/) for dependency management and package handling, offering a seamless setup and execution experience.
First, install CrewAI:
```shell
pip install crewai
```
If you want to install the 'crewai' package along with its optional features that include additional tools for agents, you can do so by using the following command:
```shell
@@ -92,22 +59,6 @@ pip install 'crewai[tools]'
```
The command above installs the basic package and also adds extra components which require more dependencies to function.
### Troubleshooting Dependencies
If you encounter issues during installation or usage, here are some common solutions:
#### Common Issues
1. **ModuleNotFoundError: No module named 'tiktoken'**
- Install tiktoken explicitly: `pip install 'crewai[embeddings]'`
- If using embedchain or other tools: `pip install 'crewai[tools]'`
2. **Failed building wheel for tiktoken**
- Ensure Rust compiler is installed (see installation steps above)
- For Windows: Verify Visual C++ Build Tools are installed
- Try upgrading pip: `pip install --upgrade pip`
- If issues persist, use a pre-built wheel: `pip install tiktoken --prefer-binary`
### 2. Setting Up Your Crew with the YAML Configuration
To create a new CrewAI project, run the following CLI (Command Line Interface) command:
@@ -149,7 +100,7 @@ You can now start developing your crew by editing the files in the `src/my_proje
#### Example of a simple crew with a sequential process:
Instantiate your crew:
Instatiate your crew:
```shell
crewai create crew latest-ai-development
@@ -170,7 +121,7 @@ researcher:
You're a seasoned researcher with a knack for uncovering the latest
developments in {topic}. Known for your ability to find the most relevant
information and present it in a clear and concise manner.
reporting_analyst:
role: >
{topic} Reporting Analyst
@@ -254,7 +205,7 @@ class LatestAiDevelopmentCrew():
tasks=self.tasks, # Automatically created by the @task decorator
process=Process.sequential,
verbose=True,
)
)
```
**main.py**
@@ -313,16 +264,13 @@ In addition to the sequential process, you can use the hierarchical process, whi
## Key Features
**Note**: CrewAI is a standalone framework built from the ground up, without dependencies on Langchain or other agent frameworks.
- **Deep Customization**: Build sophisticated agents with full control over the system - from overriding inner prompts to accessing low-level APIs. Customize roles, goals, tools, and behaviors while maintaining clean abstractions.
- **Autonomous Inter-Agent Delegation**: Agents can autonomously delegate tasks and inquire amongst themselves, enabling complex problem-solving in real-world scenarios.
- **Flexible Task Management**: Define and customize tasks with granular control, from simple operations to complex multi-step processes.
- **Production-Grade Architecture**: Support for both high-level abstractions and low-level customization, with robust error handling and state management.
- **Predictable Results**: Ensure consistent, accurate outputs through programmatic guardrails, agent training capabilities, and flow-based execution control. See our [documentation on guardrails](https://docs.crewai.com/how-to/guardrails/) for implementation details.
- **Model Flexibility**: Run your crew using OpenAI or open source models with production-ready integrations. See [Connect CrewAI to LLMs](https://docs.crewai.com/how-to/LLM-Connections/) for detailed configuration options.
- **Event-Driven Flows**: Build complex, real-world workflows with precise control over execution paths, state management, and conditional logic.
- **Process Orchestration**: Achieve any workflow pattern through flows - from simple sequential and hierarchical processes to complex, custom orchestration patterns with conditional branching and parallel execution.
- **Role-Based Agent Design**: Customize agents with specific roles, goals, and tools.
- **Autonomous Inter-Agent Delegation**: Agents can autonomously delegate tasks and inquire amongst themselves, enhancing problem-solving efficiency.
- **Flexible Task Management**: Define tasks with customizable tools and assign them to agents dynamically.
- **Processes Driven**: Currently only supports `sequential` task execution and `hierarchical` processes, but more complex processes like consensual and autonomous are being worked on.
- **Save output as file**: Save the output of individual tasks as a file, so you can use it later.
- **Parse output as Pydantic or Json**: Parse the output of individual tasks as a Pydantic model or as a Json if you want to.
- **Works with Open Source Models**: Run your crew using Open AI or open source models refer to the [Connect CrewAI to LLMs](https://docs.crewai.com/how-to/LLM-Connections/) page for details on configuring your agents' connections to models, even ones running locally!
![CrewAI Mind Map](./docs/crewAI-mindmap.png "CrewAI Mind Map")
@@ -357,98 +305,6 @@ You can test different real life examples of AI crews in the [CrewAI-examples re
[![Stock Analysis](https://img.youtube.com/vi/e0Uj4yWdaAg/maxresdefault.jpg)](https://www.youtube.com/watch?v=e0Uj4yWdaAg "Stock Analysis")
### Using Crews and Flows Together
CrewAI's power truly shines when combining Crews with Flows to create sophisticated automation pipelines. Here's how you can orchestrate multiple Crews within a Flow:
```python
from crewai.flow.flow import Flow, listen, start, router
from crewai import Crew, Agent, Task
from pydantic import BaseModel
# Define structured state for precise control
class MarketState(BaseModel):
sentiment: str = "neutral"
confidence: float = 0.0
recommendations: list = []
class AdvancedAnalysisFlow(Flow[MarketState]):
@start()
def fetch_market_data(self):
# Demonstrate low-level control with structured state
self.state.sentiment = "analyzing"
return {"sector": "tech", "timeframe": "1W"} # These parameters match the task description template
@listen(fetch_market_data)
def analyze_with_crew(self, market_data):
# Show crew agency through specialized roles
analyst = Agent(
role="Senior Market Analyst",
goal="Conduct deep market analysis with expert insight",
backstory="You're a veteran analyst known for identifying subtle market patterns"
)
researcher = Agent(
role="Data Researcher",
goal="Gather and validate supporting market data",
backstory="You excel at finding and correlating multiple data sources"
)
analysis_task = Task(
description="Analyze {sector} sector data for the past {timeframe}",
expected_output="Detailed market analysis with confidence score",
agent=analyst
)
research_task = Task(
description="Find supporting data to validate the analysis",
expected_output="Corroborating evidence and potential contradictions",
agent=researcher
)
# Demonstrate crew autonomy
analysis_crew = Crew(
agents=[analyst, researcher],
tasks=[analysis_task, research_task],
process=Process.sequential,
verbose=True
)
return analysis_crew.kickoff(inputs=market_data) # Pass market_data as named inputs
@router(analyze_with_crew)
def determine_next_steps(self):
# Show flow control with conditional routing
if self.state.confidence > 0.8:
return "high_confidence"
elif self.state.confidence > 0.5:
return "medium_confidence"
return "low_confidence"
@listen("high_confidence")
def execute_strategy(self):
# Demonstrate complex decision making
strategy_crew = Crew(
agents=[
Agent(role="Strategy Expert",
goal="Develop optimal market strategy")
],
tasks=[
Task(description="Create detailed strategy based on analysis",
expected_output="Step-by-step action plan")
]
)
return strategy_crew.kickoff()
@listen("medium_confidence", "low_confidence")
def request_additional_analysis(self):
self.state.recommendations.append("Gather more data")
return "Additional analysis required"
```
This example demonstrates how to:
1. Use Python code for basic data operations
2. Create and execute Crews as steps in your workflow
3. Use Flow decorators to manage the sequence of operations
4. Implement conditional branching based on Crew results
## Connecting Your Crew to a Model
CrewAI supports using various LLMs through a variety of connection options. By default your agents will use the OpenAI API when querying the model. However, there are several other ways to allow your agents to connect to models. For example, you can configure your agents to use a local model via the Ollama tool.
@@ -457,13 +313,9 @@ Please refer to the [Connect CrewAI to LLMs](https://docs.crewai.com/how-to/LLM-
## How CrewAI Compares
**CrewAI's Advantage**: CrewAI combines autonomous agent intelligence with precise workflow control through its unique Crews and Flows architecture. The framework excels at both high-level orchestration and low-level customization, enabling complex, production-grade systems with granular control.
**CrewAI's Advantage**: CrewAI is built with production in mind. It offers the flexibility of Autogen's conversational agents and the structured process approach of ChatDev, but without the rigidity. CrewAI's processes are designed to be dynamic and adaptable, fitting seamlessly into both development and production workflows.
- **LangGraph**: While LangGraph provides a foundation for building agent workflows, its approach requires significant boilerplate code and complex state management patterns. The framework's tight coupling with LangChain can limit flexibility when implementing custom agent behaviors or integrating with external systems.
*P.S. CrewAI demonstrates significant performance advantages over LangGraph, executing 5.76x faster in certain cases like this QA task example ([see comparison](https://github.com/crewAIInc/crewAI-examples/tree/main/Notebooks/CrewAI%20Flows%20%26%20Langgraph/QA%20Agent)) while achieving higher evaluation scores with faster completion times in certain coding tasks, like in this example ([detailed analysis](https://github.com/crewAIInc/crewAI-examples/blob/main/Notebooks/CrewAI%20Flows%20%26%20Langgraph/Coding%20Assistant/coding_assistant_eval.ipynb)).*
- **Autogen**: While Autogen excels at creating conversational agents capable of working together, it lacks an inherent concept of process. In Autogen, orchestrating agents' interactions requires additional programming, which can become complex and cumbersome as the scale of tasks grows.
- **Autogen**: While Autogen does good in creating conversational agents capable of working together, it lacks an inherent concept of process. In Autogen, orchestrating agents' interactions requires additional programming, which can become complex and cumbersome as the scale of tasks grows.
- **ChatDev**: ChatDev introduced the idea of processes into the realm of AI agents, but its implementation is quite rigid. Customizations in ChatDev are limited and not geared towards production environments, which can hinder scalability and flexibility in real-world applications.
@@ -505,7 +357,7 @@ uv run pytest .
### Running static type checks
```bash
uvx mypy src
uvx mypy
```
### Packaging
@@ -524,7 +376,7 @@ pip install dist/*.tar.gz
CrewAI uses anonymous telemetry to collect usage data with the main purpose of helping us improve the library by focusing our efforts on the most used features, integrations and tools.
It's pivotal to understand that **NO data is collected** concerning prompts, task descriptions, agents' backstories or goals, usage of tools, API calls, responses, any data processed by the agents, or secrets and environment variables, with the exception of the conditions mentioned. When the `share_crew` feature is enabled, detailed data including task descriptions, agents' backstories or goals, and other specific attributes are collected to provide deeper insights while respecting user privacy. Users can disable telemetry by setting the environment variable OTEL_SDK_DISABLED to true.
It's pivotal to understand that **NO data is collected** concerning prompts, task descriptions, agents' backstories or goals, usage of tools, API calls, responses, any data processed by the agents, or secrets and environment variables, with the exception of the conditions mentioned. When the `share_crew` feature is enabled, detailed data including task descriptions, agents' backstories or goals, and other specific attributes are collected to provide deeper insights while respecting user privacy. We don't offer a way to disable it now, but we will in the future.
Data collected includes:
@@ -547,7 +399,7 @@ Data collected includes:
- Roles of agents in a crew
- Understand high level use cases so we can build better tools, integrations and examples about it
- Tools names available
- Understand out of the publicly available tools, which ones are being used the most so we can improve them
- Understand out of the publically available tools, which ones are being used the most so we can improve them
Users can opt-in to Further Telemetry, sharing the complete telemetry data by setting the `share_crew` attribute to `True` on their Crews. Enabling `share_crew` results in the collection of detailed crew and task execution data, including `goal`, `backstory`, `context`, and `output` of tasks. This enables a deeper insight into usage patterns while respecting the user's choice to share.
@@ -588,8 +440,5 @@ A: CrewAI uses anonymous telemetry to collect usage data for improvement purpose
### Q: Where can I find examples of CrewAI in action?
A: You can find various real-life examples in the [CrewAI-examples repository](https://github.com/crewAIInc/crewAI-examples), including trip planners, stock analysis tools, and more.
### Q: What is the difference between Crews and Flows?
A: Crews and Flows serve different but complementary purposes in CrewAI. Crews are teams of AI agents working together to accomplish specific tasks through role-based collaboration, delivering accurate and predictable results. Flows, on the other hand, are event-driven workflows that can orchestrate both Crews and regular Python code, allowing you to build complex automation pipelines with secure state management and conditional execution paths.
### Q: How can I contribute to CrewAI?
A: Contributions are welcome! You can fork the repository, create a new branch for your feature, add your improvement, and send a pull request. Check the Contribution section in the README for more details.

View File

@@ -1,343 +1,161 @@
---
title: Agents
description: Detailed guide on creating and managing agents within the CrewAI framework.
description: What are CrewAI Agents and how to use them.
icon: robot
---
## Overview of an Agent
## What is an agent?
In the CrewAI framework, an `Agent` is an autonomous unit that can:
- Perform specific tasks
- Make decisions based on its role and goal
- Use tools to accomplish objectives
- Communicate and collaborate with other agents
- Maintain memory of interactions
- Delegate tasks when allowed
An agent is an **autonomous unit** programmed to:
<ul>
<li class='leading-3'>Perform tasks</li>
<li class='leading-3'>Make decisions</li>
<li class='leading-3'>Communicate with other agents</li>
</ul>
<Tip>
Think of an agent as a specialized team member with specific skills, expertise, and responsibilities. For example, a `Researcher` agent might excel at gathering and analyzing information, while a `Writer` agent might be better at creating content.
Think of an agent as a member of a team, with specific skills and a particular job to do. Agents can have different roles like `Researcher`, `Writer`, or `Customer Support`, each contributing to the overall goal of the crew.
</Tip>
## Agent Attributes
## Agent attributes
| Attribute | Parameter | Type | Description |
| :-------------------------------------- | :----------------------- | :---------------------------- | :------------------------------------------------------------------------------------------------------------------- |
| **Role** | `role` | `str` | Defines the agent's function and expertise within the crew. |
| **Goal** | `goal` | `str` | The individual objective that guides the agent's decision-making. |
| **Backstory** | `backstory` | `str` | Provides context and personality to the agent, enriching interactions. |
| **LLM** _(optional)_ | `llm` | `Union[str, LLM, Any]` | Language model that powers the agent. Defaults to the model specified in `OPENAI_MODEL_NAME` or "gpt-4". |
| **Tools** _(optional)_ | `tools` | `List[BaseTool]` | Capabilities or functions available to the agent. Defaults to an empty list. |
| **Function Calling LLM** _(optional)_ | `function_calling_llm` | `Optional[Any]` | Language model for tool calling, overrides crew's LLM if specified. |
| **Max Iterations** _(optional)_ | `max_iter` | `int` | Maximum iterations before the agent must provide its best answer. Default is 20. |
| **Max RPM** _(optional)_ | `max_rpm` | `Optional[int]` | Maximum requests per minute to avoid rate limits. |
| **Max Execution Time** _(optional)_ | `max_execution_time` | `Optional[int]` | Maximum time (in seconds) for task execution. |
| **Memory** _(optional)_ | `memory` | `bool` | Whether the agent should maintain memory of interactions. Default is True. |
| **Verbose** _(optional)_ | `verbose` | `bool` | Enable detailed execution logs for debugging. Default is False. |
| **Allow Delegation** _(optional)_ | `allow_delegation` | `bool` | Allow the agent to delegate tasks to other agents. Default is False. |
| **Step Callback** _(optional)_ | `step_callback` | `Optional[Any]` | Function called after each agent step, overrides crew callback. |
| **Cache** _(optional)_ | `cache` | `bool` | Enable caching for tool usage. Default is True. |
| **System Template** _(optional)_ | `system_template` | `Optional[str]` | Custom system prompt template for the agent. |
| **Prompt Template** _(optional)_ | `prompt_template` | `Optional[str]` | Custom prompt template for the agent. |
| **Response Template** _(optional)_ | `response_template` | `Optional[str]` | Custom response template for the agent. |
| **Allow Code Execution** _(optional)_ | `allow_code_execution` | `Optional[bool]` | Enable code execution for the agent. Default is False. |
| **Max Retry Limit** _(optional)_ | `max_retry_limit` | `int` | Maximum number of retries when an error occurs. Default is 2. |
| **Respect Context Window** _(optional)_ | `respect_context_window` | `bool` | Keep messages under context window size by summarizing. Default is True. |
| **Code Execution Mode** _(optional)_ | `code_execution_mode` | `Literal["safe", "unsafe"]` | Mode for code execution: 'safe' (using Docker) or 'unsafe' (direct). Default is 'safe'. |
| **Embedder Config** _(optional)_ | `embedder_config` | `Optional[Dict[str, Any]]` | Configuration for the embedder used by the agent. |
| **Knowledge Sources** _(optional)_ | `knowledge_sources` | `Optional[List[BaseKnowledgeSource]]` | Knowledge sources available to the agent. |
| **Use System Prompt** _(optional)_ | `use_system_prompt` | `Optional[bool]` | Whether to use system prompt (for o1 model support). Default is True. |
| Attribute | Parameter | Description |
| :------------------------- | :--------- | :--------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------------- |
| **Role** | `role` | Defines the agent's function within the crew. It determines the kind of tasks the agent is best suited for. |
| **Goal** | `goal` | The individual objective that the agent aims to achieve. It guides the agent's decision-making process. |
| **Backstory** | `backstory`| Provides context to the agent's role and goal, enriching the interaction and collaboration dynamics. |
| **LLM** *(optional)* | `llm` | Represents the language model that will run the agent. It dynamically fetches the model name from the `OPENAI_MODEL_NAME` environment variable, defaulting to "gpt-4" if not specified. |
| **Tools** *(optional)* | `tools` | Set of capabilities or functions that the agent can use to perform tasks. Expected to be instances of custom classes compatible with the agent's execution environment. Tools are initialized with a default value of an empty list. |
| **Function Calling LLM** *(optional)* | `function_calling_llm` | Specifies the language model that will handle the tool calling for this agent, overriding the crew function calling LLM if passed. Default is `None`. |
| **Max Iter** *(optional)* | `max_iter` | Max Iter is the maximum number of iterations the agent can perform before being forced to give its best answer. Default is `25`. |
| **Max RPM** *(optional)* | `max_rpm` | Max RPM is the maximum number of requests per minute the agent can perform to avoid rate limits. It's optional and can be left unspecified, with a default value of `None`. |
| **Max Execution Time** *(optional)* | `max_execution_time` | Max Execution Time is the maximum execution time for an agent to execute a task. It's optional and can be left unspecified, with a default value of `None`, meaning no max execution time. |
| **Verbose** *(optional)* | `verbose` | Setting this to `True` configures the internal logger to provide detailed execution logs, aiding in debugging and monitoring. Default is `False`. |
| **Allow Delegation** *(optional)* | `allow_delegation` | Agents can delegate tasks or questions to one another, ensuring that each task is handled by the most suitable agent. Default is `False`. |
| **Step Callback** *(optional)* | `step_callback` | A function that is called after each step of the agent. This can be used to log the agent's actions or to perform other operations. It will overwrite the crew `step_callback`. |
| **Cache** *(optional)* | `cache` | Indicates if the agent should use a cache for tool usage. Default is `True`. |
| **System Template** *(optional)* | `system_template` | Specifies the system format for the agent. Default is `None`. |
| **Prompt Template** *(optional)* | `prompt_template` | Specifies the prompt format for the agent. Default is `None`. |
| **Response Template** *(optional)* | `response_template` | Specifies the response format for the agent. Default is `None`. |
| **Allow Code Execution** *(optional)* | `allow_code_execution` | Enable code execution for the agent. Default is `False`. |
| **Max Retry Limit** *(optional)* | `max_retry_limit` | Maximum number of retries for an agent to execute a task when an error occurs. Default is `2`. |
| **Use System Prompt** *(optional)* | `use_system_prompt` | Adds the ability to not use system prompt (to support o1 models). Default is `True`. |
| **Respect Context Window** *(optional)* | `respect_context_window` | Summary strategy to avoid overflowing the context window. Default is `True`. |
| **Code Execution Mode** *(optional)* | `code_execution_mode` | Determines the mode for code execution: 'safe' (using Docker) or 'unsafe' (direct execution on the host machine). Default is `safe`. |
## Creating Agents
There are two ways to create agents in CrewAI: using **YAML configuration (recommended)** or defining them **directly in code**.
### YAML Configuration (Recommended)
Using YAML configuration provides a cleaner, more maintainable way to define agents. We strongly recommend using this approach in your CrewAI projects.
After creating your CrewAI project as outlined in the [Installation](/installation) section, navigate to the `src/latest_ai_development/config/agents.yaml` file and modify the template to match your requirements.
## Creating an agent
<Note>
Variables in your YAML files (like `{topic}`) will be replaced with values from your inputs when running the crew:
```python Code
crew.kickoff(inputs={'topic': 'AI Agents'})
```
**Agent interaction**: Agents can interact with each other using CrewAI's built-in delegation and communication mechanisms. This allows for dynamic task management and problem-solving within the crew.
</Note>
Here's an example of how to configure agents using YAML:
To create an agent, you would typically initialize an instance of the `Agent` class with the desired properties. Here's a conceptual example including all attributes:
```yaml agents.yaml
# src/latest_ai_development/config/agents.yaml
researcher:
role: >
{topic} Senior Data Researcher
goal: >
Uncover cutting-edge developments in {topic}
backstory: >
You're a seasoned researcher with a knack for uncovering the latest
developments in {topic}. Known for your ability to find the most relevant
information and present it in a clear and concise manner.
reporting_analyst:
role: >
{topic} Reporting Analyst
goal: >
Create detailed reports based on {topic} data analysis and research findings
backstory: >
You're a meticulous analyst with a keen eye for detail. You're known for
your ability to turn complex data into clear and concise reports, making
it easy for others to understand and act on the information you provide.
```
To use this YAML configuration in your code, create a crew class that inherits from `CrewBase`:
```python Code
# src/latest_ai_development/crew.py
from crewai import Agent, Crew, Process
from crewai.project import CrewBase, agent, crew
from crewai_tools import SerperDevTool
@CrewBase
class LatestAiDevelopmentCrew():
"""LatestAiDevelopment crew"""
@agent
def researcher(self) -> Agent:
return Agent(
config=self.agents_config['researcher'],
verbose=True,
tools=[SerperDevTool()]
)
@agent
def reporting_analyst(self) -> Agent:
return Agent(
config=self.agents_config['reporting_analyst'],
verbose=True
)
```
<Note>
The names you use in your YAML files (`agents.yaml`) should match the method names in your Python code.
</Note>
### Direct Code Definition
You can create agents directly in code by instantiating the `Agent` class. Here's a comprehensive example showing all available parameters:
```python Code
```python Code example
from crewai import Agent
from crewai_tools import SerperDevTool
# Create an agent with all available parameters
agent = Agent(
role="Senior Data Scientist",
goal="Analyze and interpret complex datasets to provide actionable insights",
backstory="With over 10 years of experience in data science and machine learning, "
"you excel at finding patterns in complex datasets.",
llm="gpt-4", # Default: OPENAI_MODEL_NAME or "gpt-4"
function_calling_llm=None, # Optional: Separate LLM for tool calling
memory=True, # Default: True
verbose=False, # Default: False
allow_delegation=False, # Default: False
max_iter=20, # Default: 20 iterations
max_rpm=None, # Optional: Rate limit for API calls
max_execution_time=None, # Optional: Maximum execution time in seconds
max_retry_limit=2, # Default: 2 retries on error
allow_code_execution=False, # Default: False
code_execution_mode="safe", # Default: "safe" (options: "safe", "unsafe")
respect_context_window=True, # Default: True
use_system_prompt=True, # Default: True
tools=[SerperDevTool()], # Optional: List of tools
knowledge_sources=None, # Optional: List of knowledge sources
embedder_config=None, # Optional: Custom embedder configuration
system_template=None, # Optional: Custom system prompt template
prompt_template=None, # Optional: Custom prompt template
response_template=None, # Optional: Custom response template
step_callback=None, # Optional: Callback function for monitoring
role='Data Analyst',
goal='Extract actionable insights',
backstory="""You're a data analyst at a large company.
You're responsible for analyzing data and providing insights
to the business.
You're currently working on a project to analyze the
performance of our marketing campaigns.""",
tools=[my_tool1, my_tool2], # Optional, defaults to an empty list
llm=my_llm, # Optional
function_calling_llm=my_llm, # Optional
max_iter=15, # Optional
max_rpm=None, # Optional
max_execution_time=None, # Optional
verbose=True, # Optional
allow_delegation=False, # Optional
step_callback=my_intermediate_step_callback, # Optional
cache=True, # Optional
system_template=my_system_template, # Optional
prompt_template=my_prompt_template, # Optional
response_template=my_response_template, # Optional
config=my_config, # Optional
crew=my_crew, # Optional
tools_handler=my_tools_handler, # Optional
cache_handler=my_cache_handler, # Optional
callbacks=[callback1, callback2], # Optional
allow_code_execution=True, # Optional
max_retry_limit=2, # Optional
use_system_prompt=True, # Optional
respect_context_window=True, # Optional
code_execution_mode='safe', # Optional, defaults to 'safe'
)
```
Let's break down some key parameter combinations for common use cases:
## Setting prompt templates
#### Basic Research Agent
```python Code
research_agent = Agent(
role="Research Analyst",
goal="Find and summarize information about specific topics",
backstory="You are an experienced researcher with attention to detail",
tools=[SerperDevTool()],
verbose=True # Enable logging for debugging
)
```
Prompt templates are used to format the prompt for the agent. You can use to update the system, regular and response templates for the agent. Here's an example of how to set prompt templates:
#### Code Development Agent
```python Code
dev_agent = Agent(
role="Senior Python Developer",
goal="Write and debug Python code",
backstory="Expert Python developer with 10 years of experience",
allow_code_execution=True,
code_execution_mode="safe", # Uses Docker for safety
max_execution_time=300, # 5-minute timeout
max_retry_limit=3 # More retries for complex code tasks
)
```
#### Long-Running Analysis Agent
```python Code
analysis_agent = Agent(
role="Data Analyst",
goal="Perform deep analysis of large datasets",
backstory="Specialized in big data analysis and pattern recognition",
memory=True,
respect_context_window=True,
max_rpm=10, # Limit API calls
function_calling_llm="gpt-4o-mini" # Cheaper model for tool calls
)
```
#### Custom Template Agent
```python Code
custom_agent = Agent(
role="Customer Service Representative",
goal="Assist customers with their inquiries",
backstory="Experienced in customer support with a focus on satisfaction",
system_template="""<|start_header_id|>system<|end_header_id|>
```python Code example
agent = Agent(
role="{topic} specialist",
goal="Figure {goal} out",
backstory="I am the master of {role}",
system_template="""<|start_header_id|>system<|end_header_id|>
{{ .System }}<|eot_id|>""",
prompt_template="""<|start_header_id|>user<|end_header_id|>
prompt_template="""<|start_header_id|>user<|end_header_id|>
{{ .Prompt }}<|eot_id|>""",
response_template="""<|start_header_id|>assistant<|end_header_id|>
response_template="""<|start_header_id|>assistant<|end_header_id|>
{{ .Response }}<|eot_id|>""",
)
```
### Parameter Details
## Bring your third-party agents
#### Critical Parameters
- `role`, `goal`, and `backstory` are required and shape the agent's behavior
- `llm` determines the language model used (default: OpenAI's GPT-4)
Extend your third-party agents like LlamaIndex, Langchain, Autogen or fully custom agents using the the CrewAI's `BaseAgent` class.
#### Memory and Context
- `memory`: Enable to maintain conversation history
- `respect_context_window`: Prevents token limit issues
- `knowledge_sources`: Add domain-specific knowledge bases
#### Execution Control
- `max_iter`: Maximum attempts before giving best answer
- `max_execution_time`: Timeout in seconds
- `max_rpm`: Rate limiting for API calls
- `max_retry_limit`: Retries on error
#### Code Execution
- `allow_code_execution`: Must be True to run code
- `code_execution_mode`:
- `"safe"`: Uses Docker (recommended for production)
- `"unsafe"`: Direct execution (use only in trusted environments)
#### Templates
- `system_template`: Defines agent's core behavior
- `prompt_template`: Structures input format
- `response_template`: Formats agent responses
<Note>
When using custom templates, you can use variables like `{role}`, `{goal}`, and `{input}` in your templates. These will be automatically populated during execution.
<Note>
**BaseAgent** includes attributes and methods required to integrate with your crews to run and delegate tasks to other agents within your own crew.
</Note>
## Agent Tools
CrewAI is a universal multi-agent framework that allows for all agents to work together to automate tasks and solve problems.
Agents can be equipped with various tools to enhance their capabilities. CrewAI supports tools from:
- [CrewAI Toolkit](https://github.com/joaomdmoura/crewai-tools)
- [LangChain Tools](https://python.langchain.com/docs/integrations/tools)
```python Code example
from crewai import Agent, Task, Crew
from custom_agent import CustomAgent # You need to build and extend your own agent logic with the CrewAI BaseAgent class then import it here.
Here's how to add tools to an agent:
from langchain.agents import load_tools
```python Code
from crewai import Agent
from crewai_tools import SerperDevTool, WikipediaTools
langchain_tools = load_tools(["google-serper"], llm=llm)
# Create tools
search_tool = SerperDevTool()
wiki_tool = WikipediaTools()
# Add tools to agent
researcher = Agent(
role="AI Technology Researcher",
goal="Research the latest AI developments",
tools=[search_tool, wiki_tool],
verbose=True
agent1 = CustomAgent(
role="agent role",
goal="who is {input}?",
backstory="agent backstory",
verbose=True,
)
task1 = Task(
expected_output="a short biography of {input}",
description="a short biography of {input}",
agent=agent1,
)
agent2 = Agent(
role="agent role",
goal="summarize the short bio for {input} and if needed do more research",
backstory="agent backstory",
verbose=True,
)
task2 = Task(
description="a tldr summary of the short biography",
expected_output="5 bullet point summary of the biography",
agent=agent2,
context=[task1],
)
my_crew = Crew(agents=[agent1, agent2], tasks=[task1, task2])
crew = my_crew.kickoff(inputs={"input": "Mark Twain"})
```
## Agent Memory and Context
## Conclusion
Agents can maintain memory of their interactions and use context from previous tasks. This is particularly useful for complex workflows where information needs to be retained across multiple tasks.
```python Code
from crewai import Agent
analyst = Agent(
role="Data Analyst",
goal="Analyze and remember complex data patterns",
memory=True, # Enable memory
verbose=True
)
```
<Note>
When `memory` is enabled, the agent will maintain context across multiple interactions, improving its ability to handle complex, multi-step tasks.
</Note>
## Important Considerations and Best Practices
### Security and Code Execution
- When using `allow_code_execution`, be cautious with user input and always validate it
- Use `code_execution_mode: "safe"` (Docker) in production environments
- Consider setting appropriate `max_execution_time` limits to prevent infinite loops
### Performance Optimization
- Use `respect_context_window: true` to prevent token limit issues
- Set appropriate `max_rpm` to avoid rate limiting
- Enable `cache: true` to improve performance for repetitive tasks
- Adjust `max_iter` and `max_retry_limit` based on task complexity
### Memory and Context Management
- Use `memory: true` for tasks requiring historical context
- Leverage `knowledge_sources` for domain-specific information
- Configure `embedder_config` when using custom embedding models
- Use custom templates (`system_template`, `prompt_template`, `response_template`) for fine-grained control over agent behavior
### Agent Collaboration
- Enable `allow_delegation: true` when agents need to work together
- Use `step_callback` to monitor and log agent interactions
- Consider using different LLMs for different purposes:
- Main `llm` for complex reasoning
- `function_calling_llm` for efficient tool usage
### Model Compatibility
- Set `use_system_prompt: false` for older models that don't support system messages
- Ensure your chosen `llm` supports the features you need (like function calling)
## Troubleshooting Common Issues
1. **Rate Limiting**: If you're hitting API rate limits:
- Implement appropriate `max_rpm`
- Use caching for repetitive operations
- Consider batching requests
2. **Context Window Errors**: If you're exceeding context limits:
- Enable `respect_context_window`
- Use more efficient prompts
- Clear agent memory periodically
3. **Code Execution Issues**: If code execution fails:
- Verify Docker is installed for safe mode
- Check execution permissions
- Review code sandbox settings
4. **Memory Issues**: If agent responses seem inconsistent:
- Verify memory is enabled
- Check knowledge source configuration
- Review conversation history management
Remember that agents are most effective when configured according to their specific use case. Take time to understand your requirements and adjust these parameters accordingly.
Agents are the building blocks of the CrewAI framework. By understanding how to define and interact with agents,
you can create sophisticated AI systems that leverage the power of collaborative intelligence. The `code_execution_mode` attribute provides flexibility in how agents execute code, allowing for both secure and direct execution options.

View File

@@ -28,19 +28,20 @@ crewai [COMMAND] [OPTIONS] [ARGUMENTS]
### 1. Create
Create a new crew or flow.
Create a new crew or pipeline.
```shell
crewai create [OPTIONS] TYPE NAME
```
- `TYPE`: Choose between "crew" or "flow"
- `NAME`: Name of the crew or flow
- `TYPE`: Choose between "crew" or "pipeline"
- `NAME`: Name of the crew or pipeline
- `--router`: (Optional) Create a pipeline with router functionality
Example:
```shell
crewai create crew my_new_crew
crewai create flow my_new_flow
crewai create pipeline my_new_pipeline --router
```
### 2. Version

View File

@@ -22,8 +22,7 @@ A crew in crewAI represents a collaborative group of agents working together to
| **Max RPM** _(optional)_ | `max_rpm` | Maximum requests per minute the crew adheres to during execution. Defaults to `None`. |
| **Language** _(optional)_ | `language` | Language used for the crew, defaults to English. |
| **Language File** _(optional)_ | `language_file` | Path to the language file to be used for the crew. |
| **Memory** _(optional)_ | `memory` | Utilized for storing execution memories (short-term, long-term, entity memory). |
| **Memory Config** _(optional)_ | `memory_config` | Configuration for the memory provider to be used by the crew. |
| **Memory** _(optional)_ | `memory` | Utilized for storing execution memories (short-term, long-term, entity memory). Defaults to `False`. |
| **Cache** _(optional)_ | `cache` | Specifies whether to use a cache for storing the results of tools' execution. Defaults to `True`. |
| **Embedder** _(optional)_ | `embedder` | Configuration for the embedder to be used by the crew. Mostly used by memory for now. Default is `{"provider": "openai"}`. |
| **Full Output** _(optional)_ | `full_output` | Whether the crew should return the full output with all tasks outputs or just the final output. Defaults to `False`. |
@@ -32,6 +31,7 @@ A crew in crewAI represents a collaborative group of agents working together to
| **Share Crew** _(optional)_ | `share_crew` | Whether you want to share the complete crew information and execution with the crewAI team to make the library better, and allow us to train models. |
| **Output Log File** _(optional)_ | `output_log_file` | Whether you want to have a file with the complete crew output and execution. You can set it using True and it will default to the folder you are currently in and it will be called logs.txt or passing a string with the full path and name of the file. |
| **Manager Agent** _(optional)_ | `manager_agent` | `manager` sets a custom agent that will be used as a manager. |
| **Manager Callbacks** _(optional)_ | `manager_callbacks` | `manager_callbacks` takes a list of callback handlers to be executed by the manager agent when a hierarchical process is used. |
| **Prompt File** _(optional)_ | `prompt_file` | Path to the prompt JSON file to be used for the crew. |
| **Planning** *(optional)* | `planning` | Adds planning ability to the Crew. When activated before each Crew iteration, all Crew data is sent to an AgentPlanner that will plan the tasks and this plan will be added to each task description. |
| **Planning LLM** *(optional)* | `planning_llm` | The language model used by the AgentPlanner in a planning process. |
@@ -40,155 +40,6 @@ A crew in crewAI represents a collaborative group of agents working together to
**Crew Max RPM**: The `max_rpm` attribute sets the maximum number of requests per minute the crew can perform to avoid rate limits and will override individual agents' `max_rpm` settings if you set it.
</Tip>
## Creating Crews
There are two ways to create crews in CrewAI: using **YAML configuration (recommended)** or defining them **directly in code**.
### YAML Configuration (Recommended)
Using YAML configuration provides a cleaner, more maintainable way to define crews and is consistent with how agents and tasks are defined in CrewAI projects.
After creating your CrewAI project as outlined in the [Installation](/installation) section, you can define your crew in a class that inherits from `CrewBase` and uses decorators to define agents, tasks, and the crew itself.
#### Example Crew Class with Decorators
```python code
from crewai import Agent, Crew, Task, Process
from crewai.project import CrewBase, agent, task, crew, before_kickoff, after_kickoff
@CrewBase
class YourCrewName:
"""Description of your crew"""
# Paths to your YAML configuration files
# To see an example agent and task defined in YAML, checkout the following:
# - Task: https://docs.crewai.com/concepts/tasks#yaml-configuration-recommended
# - Agents: https://docs.crewai.com/concepts/agents#yaml-configuration-recommended
agents_config = 'config/agents.yaml'
tasks_config = 'config/tasks.yaml'
@before_kickoff
def prepare_inputs(self, inputs):
# Modify inputs before the crew starts
inputs['additional_data'] = "Some extra information"
return inputs
@after_kickoff
def process_output(self, output):
# Modify output after the crew finishes
output.raw += "\nProcessed after kickoff."
return output
@agent
def agent_one(self) -> Agent:
return Agent(
config=self.agents_config['agent_one'],
verbose=True
)
@agent
def agent_two(self) -> Agent:
return Agent(
config=self.agents_config['agent_two'],
verbose=True
)
@task
def task_one(self) -> Task:
return Task(
config=self.tasks_config['task_one']
)
@task
def task_two(self) -> Task:
return Task(
config=self.tasks_config['task_two']
)
@crew
def crew(self) -> Crew:
return Crew(
agents=self.agents, # Automatically collected by the @agent decorator
tasks=self.tasks, # Automatically collected by the @task decorator.
process=Process.sequential,
verbose=True,
)
```
<Note>
Tasks will be executed in the order they are defined.
</Note>
The `CrewBase` class, along with these decorators, automates the collection of agents and tasks, reducing the need for manual management.
#### Decorators overview from `annotations.py`
CrewAI provides several decorators in the `annotations.py` file that are used to mark methods within your crew class for special handling:
- `@CrewBase`: Marks the class as a crew base class.
- `@agent`: Denotes a method that returns an `Agent` object.
- `@task`: Denotes a method that returns a `Task` object.
- `@crew`: Denotes the method that returns the `Crew` object.
- `@before_kickoff`: (Optional) Marks a method to be executed before the crew starts.
- `@after_kickoff`: (Optional) Marks a method to be executed after the crew finishes.
These decorators help in organizing your crew's structure and automatically collecting agents and tasks without manually listing them.
### Direct Code Definition (Alternative)
Alternatively, you can define the crew directly in code without using YAML configuration files.
```python code
from crewai import Agent, Crew, Task, Process
from crewai_tools import YourCustomTool
class YourCrewName:
def agent_one(self) -> Agent:
return Agent(
role="Data Analyst",
goal="Analyze data trends in the market",
backstory="An experienced data analyst with a background in economics",
verbose=True,
tools=[YourCustomTool()]
)
def agent_two(self) -> Agent:
return Agent(
role="Market Researcher",
goal="Gather information on market dynamics",
backstory="A diligent researcher with a keen eye for detail",
verbose=True
)
def task_one(self) -> Task:
return Task(
description="Collect recent market data and identify trends.",
expected_output="A report summarizing key trends in the market.",
agent=self.agent_one()
)
def task_two(self) -> Task:
return Task(
description="Research factors affecting market dynamics.",
expected_output="An analysis of factors influencing the market.",
agent=self.agent_two()
)
def crew(self) -> Crew:
return Crew(
agents=[self.agent_one(), self.agent_two()],
tasks=[self.task_one(), self.task_two()],
process=Process.sequential,
verbose=True
)
```
In this example:
- Agents and tasks are defined directly within the class without decorators.
- We manually create and manage the list of agents and tasks.
- This approach provides more control but can be less maintainable for larger projects.
## Crew Output
@@ -336,4 +187,4 @@ Then, to replay from a specific task, use:
crewai replay -t <task_id>
```
These commands let you replay from your latest kickoff tasks, still retaining context from previously executed tasks.
These commands let you replay from your latest kickoff tasks, still retaining context from previously executed tasks.

View File

@@ -628,4 +628,4 @@ Also, check out our YouTube video on how to use flows in CrewAI below!
allow="accelerometer; autoplay; clipboard-write; encrypted-media; gyroscope; picture-in-picture; web-share"
referrerpolicy="strict-origin-when-cross-origin"
allowfullscreen
></iframe>
></iframe>

View File

@@ -1,420 +0,0 @@
---
title: Knowledge
description: What is knowledge in CrewAI and how to use it.
icon: book
---
# Using Knowledge in CrewAI
## What is Knowledge?
Knowledge in CrewAI is a powerful system that allows AI agents to access and utilize external information sources during their tasks.
Think of it as giving your agents a reference library they can consult while working.
<Info>
Key benefits of using Knowledge:
- Enhance agents with domain-specific information
- Support decisions with real-world data
- Maintain context across conversations
- Ground responses in factual information
</Info>
## Supported Knowledge Sources
CrewAI supports various types of knowledge sources out of the box:
<CardGroup cols={2}>
<Card title="Text Sources" icon="text">
- Raw strings
- Text files (.txt)
- PDF documents
</Card>
<Card title="Structured Data" icon="table">
- CSV files
- Excel spreadsheets
- JSON documents
</Card>
</CardGroup>
## Quick Start
Here's an example using string-based knowledge:
```python Code
from crewai import Agent, Task, Crew, Process, LLM
from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource
# Create a knowledge source
content = "Users name is John. He is 30 years old and lives in San Francisco."
string_source = StringKnowledgeSource(
content=content,
)
# Create an LLM with a temperature of 0 to ensure deterministic outputs
llm = LLM(model="gpt-4o-mini", temperature=0)
# Create an agent with the knowledge store
agent = Agent(
role="About User",
goal="You know everything about the user.",
backstory="""You are a master at understanding people and their preferences.""",
verbose=True,
allow_delegation=False,
llm=llm,
)
task = Task(
description="Answer the following questions about the user: {question}",
expected_output="An answer to the question.",
agent=agent,
)
crew = Crew(
agents=[agent],
tasks=[task],
verbose=True,
process=Process.sequential,
knowledge_sources=[string_source], # Enable knowledge by adding the sources here. You can also add more sources to the sources list.
)
result = crew.kickoff(inputs={"question": "What city does John live in and how old is he?"})
```
Here's another example with the `CrewDoclingSource`
```python Code
from crewai import LLM, Agent, Crew, Process, Task
from crewai.knowledge.source.crew_docling_source import CrewDoclingSource
# Create a knowledge source
content_source = CrewDoclingSource(
file_paths=[
"https://lilianweng.github.io/posts/2024-11-28-reward-hacking",
"https://lilianweng.github.io/posts/2024-07-07-hallucination",
],
)
# Create an LLM with a temperature of 0 to ensure deterministic outputs
llm = LLM(model="gpt-4o-mini", temperature=0)
# Create an agent with the knowledge store
agent = Agent(
role="About papers",
goal="You know everything about the papers.",
backstory="""You are a master at understanding papers and their content.""",
verbose=True,
allow_delegation=False,
llm=llm,
)
task = Task(
description="Answer the following questions about the papers: {question}",
expected_output="An answer to the question.",
agent=agent,
)
crew = Crew(
agents=[agent],
tasks=[task],
verbose=True,
process=Process.sequential,
knowledge_sources=[
content_source
], # Enable knowledge by adding the sources here. You can also add more sources to the sources list.
)
result = crew.kickoff(
inputs={
"question": "What is the reward hacking paper about? Be sure to provide sources."
}
)
```
## Knowledge Configuration
### Chunking Configuration
Control how content is split for processing by setting the chunk size and overlap.
```python Code
knowledge_source = StringKnowledgeSource(
content="Long content...",
chunk_size=4000, # Characters per chunk (default)
chunk_overlap=200 # Overlap between chunks (default)
)
```
## Embedder Configuration
You can also configure the embedder for the knowledge store. This is useful if you want to use a different embedder for the knowledge store than the one used for the agents.
```python Code
...
string_source = StringKnowledgeSource(
content="Users name is John. He is 30 years old and lives in San Francisco.",
)
crew = Crew(
...
knowledge_sources=[string_source],
embedder={
"provider": "openai",
"config": {"model": "text-embedding-3-small"},
},
)
```
## Clearing Knowledge
If you need to clear the knowledge stored in CrewAI, you can use the `crewai reset-memories` command with the `--knowledge` option.
```bash Command
crewai reset-memories --knowledge
```
This is useful when you've updated your knowledge sources and want to ensure that the agents are using the most recent information.
## Agent-Specific Knowledge
While knowledge can be provided at the crew level using `crew.knowledge_sources`, individual agents can also have their own knowledge sources using the `knowledge_sources` parameter:
```python Code
from crewai import Agent, Task, Crew
from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource
# Create agent-specific knowledge about a product
product_specs = StringKnowledgeSource(
content="""The XPS 13 laptop features:
- 13.4-inch 4K display
- Intel Core i7 processor
- 16GB RAM
- 512GB SSD storage
- 12-hour battery life""",
metadata={"category": "product_specs"}
)
# Create a support agent with product knowledge
support_agent = Agent(
role="Technical Support Specialist",
goal="Provide accurate product information and support.",
backstory="You are an expert on our laptop products and specifications.",
knowledge_sources=[product_specs] # Agent-specific knowledge
)
# Create a task that requires product knowledge
support_task = Task(
description="Answer this customer question: {question}",
agent=support_agent
)
# Create and run the crew
crew = Crew(
agents=[support_agent],
tasks=[support_task]
)
# Get answer about the laptop's specifications
result = crew.kickoff(
inputs={"question": "What is the storage capacity of the XPS 13?"}
)
```
<Info>
Benefits of agent-specific knowledge:
- Give agents specialized information for their roles
- Maintain separation of concerns between agents
- Combine with crew-level knowledge for layered information access
</Info>
## Custom Knowledge Sources
CrewAI allows you to create custom knowledge sources for any type of data by extending the `BaseKnowledgeSource` class. Let's create a practical example that fetches and processes space news articles.
#### Space News Knowledge Source Example
<CodeGroup>
```python Code
from crewai import Agent, Task, Crew, Process, LLM
from crewai.knowledge.source.base_knowledge_source import BaseKnowledgeSource
import requests
from datetime import datetime
from typing import Dict, Any
from pydantic import BaseModel, Field
class SpaceNewsKnowledgeSource(BaseKnowledgeSource):
"""Knowledge source that fetches data from Space News API."""
api_endpoint: str = Field(description="API endpoint URL")
limit: int = Field(default=10, description="Number of articles to fetch")
def load_content(self) -> Dict[Any, str]:
"""Fetch and format space news articles."""
try:
response = requests.get(
f"{self.api_endpoint}?limit={self.limit}"
)
response.raise_for_status()
data = response.json()
articles = data.get('results', [])
formatted_data = self._format_articles(articles)
return {self.api_endpoint: formatted_data}
except Exception as e:
raise ValueError(f"Failed to fetch space news: {str(e)}")
def _format_articles(self, articles: list) -> str:
"""Format articles into readable text."""
formatted = "Space News Articles:\n\n"
for article in articles:
formatted += f"""
Title: {article['title']}
Published: {article['published_at']}
Summary: {article['summary']}
News Site: {article['news_site']}
URL: {article['url']}
-------------------"""
return formatted
def add(self) -> None:
"""Process and store the articles."""
content = self.load_content()
for _, text in content.items():
chunks = self._chunk_text(text)
self.chunks.extend(chunks)
self._save_documents()
# Create knowledge source
recent_news = SpaceNewsKnowledgeSource(
api_endpoint="https://api.spaceflightnewsapi.net/v4/articles",
limit=10,
)
# Create specialized agent
space_analyst = Agent(
role="Space News Analyst",
goal="Answer questions about space news accurately and comprehensively",
backstory="""You are a space industry analyst with expertise in space exploration,
satellite technology, and space industry trends. You excel at answering questions
about space news and providing detailed, accurate information.""",
knowledge_sources=[recent_news],
llm=LLM(model="gpt-4", temperature=0.0)
)
# Create task that handles user questions
analysis_task = Task(
description="Answer this question about space news: {user_question}",
expected_output="A detailed answer based on the recent space news articles",
agent=space_analyst
)
# Create and run the crew
crew = Crew(
agents=[space_analyst],
tasks=[analysis_task],
verbose=True,
process=Process.sequential
)
# Example usage
result = crew.kickoff(
inputs={"user_question": "What are the latest developments in space exploration?"}
)
```
```output Output
# Agent: Space News Analyst
## Task: Answer this question about space news: What are the latest developments in space exploration?
# Agent: Space News Analyst
## Final Answer:
The latest developments in space exploration, based on recent space news articles, include the following:
1. SpaceX has received the final regulatory approvals to proceed with the second integrated Starship/Super Heavy launch, scheduled for as soon as the morning of Nov. 17, 2023. This is a significant step in SpaceX's ambitious plans for space exploration and colonization. [Source: SpaceNews](https://spacenews.com/starship-cleared-for-nov-17-launch/)
2. SpaceX has also informed the US Federal Communications Commission (FCC) that it plans to begin launching its first next-generation Starlink Gen2 satellites. This represents a major upgrade to the Starlink satellite internet service, which aims to provide high-speed internet access worldwide. [Source: Teslarati](https://www.teslarati.com/spacex-first-starlink-gen2-satellite-launch-2022/)
3. AI startup Synthetaic has raised $15 million in Series B funding. The company uses artificial intelligence to analyze data from space and air sensors, which could have significant applications in space exploration and satellite technology. [Source: SpaceNews](https://spacenews.com/ai-startup-synthetaic-raises-15-million-in-series-b-funding/)
4. The Space Force has formally established a unit within the U.S. Indo-Pacific Command, marking a permanent presence in the Indo-Pacific region. This could have significant implications for space security and geopolitics. [Source: SpaceNews](https://spacenews.com/space-force-establishes-permanent-presence-in-indo-pacific-region/)
5. Slingshot Aerospace, a space tracking and data analytics company, is expanding its network of ground-based optical telescopes to increase coverage of low Earth orbit. This could improve our ability to track and analyze objects in low Earth orbit, including satellites and space debris. [Source: SpaceNews](https://spacenews.com/slingshots-space-tracking-network-to-extend-coverage-of-low-earth-orbit/)
6. The National Natural Science Foundation of China has outlined a five-year project for researchers to study the assembly of ultra-large spacecraft. This could lead to significant advancements in spacecraft technology and space exploration capabilities. [Source: SpaceNews](https://spacenews.com/china-researching-challenges-of-kilometer-scale-ultra-large-spacecraft/)
7. The Center for AEroSpace Autonomy Research (CAESAR) at Stanford University is focusing on spacecraft autonomy. The center held a kickoff event on May 22, 2024, to highlight the industry, academia, and government collaboration it seeks to foster. This could lead to significant advancements in autonomous spacecraft technology. [Source: SpaceNews](https://spacenews.com/stanford-center-focuses-on-spacecraft-autonomy/)
```
</CodeGroup>
#### Key Components Explained
1. **Custom Knowledge Source (`SpaceNewsKnowledgeSource`)**:
- Extends `BaseKnowledgeSource` for integration with CrewAI
- Configurable API endpoint and article limit
- Implements three key methods:
- `load_content()`: Fetches articles from the API
- `_format_articles()`: Structures the articles into readable text
- `add()`: Processes and stores the content
2. **Agent Configuration**:
- Specialized role as a Space News Analyst
- Uses the knowledge source to access space news
3. **Task Setup**:
- Takes a user question as input through `{user_question}`
- Designed to provide detailed answers based on the knowledge source
4. **Crew Orchestration**:
- Manages the workflow between agent and task
- Handles input/output through the kickoff method
This example demonstrates how to:
- Create a custom knowledge source that fetches real-time data
- Process and format external data for AI consumption
- Use the knowledge source to answer specific user questions
- Integrate everything seamlessly with CrewAI's agent system
#### About the Spaceflight News API
The example uses the [Spaceflight News API](https://api.spaceflightnewsapi.net/v4/docs/), which:
- Provides free access to space-related news articles
- Requires no authentication
- Returns structured data about space news
- Supports pagination and filtering
You can customize the API query by modifying the endpoint URL:
```python
# Fetch more articles
recent_news = SpaceNewsKnowledgeSource(
api_endpoint="https://api.spaceflightnewsapi.net/v4/articles",
limit=20, # Increase the number of articles
)
# Add search parameters
recent_news = SpaceNewsKnowledgeSource(
api_endpoint="https://api.spaceflightnewsapi.net/v4/articles?search=NASA", # Search for NASA news
limit=10,
)
```
## Best Practices
<AccordionGroup>
<Accordion title="Content Organization">
- Keep chunk sizes appropriate for your content type
- Consider content overlap for context preservation
- Organize related information into separate knowledge sources
</Accordion>
<Accordion title="Performance Tips">
- Adjust chunk sizes based on content complexity
- Configure appropriate embedding models
- Consider using local embedding providers for faster processing
</Accordion>
</AccordionGroup>

View File

@@ -7,45 +7,32 @@ icon: link
## Using LangChain Tools
<Info>
CrewAI seamlessly integrates with LangChain's comprehensive [list of tools](https://python.langchain.com/docs/integrations/tools/), all of which can be used with CrewAI.
CrewAI seamlessly integrates with LangChains comprehensive [list of tools](https://python.langchain.com/docs/integrations/tools/), all of which can be used with CrewAI.
</Info>
```python Code
import os
from dotenv import load_dotenv
from crewai import Agent, Task, Crew
from crewai.tools import BaseTool
from pydantic import Field
from langchain_community.utilities import GoogleSerperAPIWrapper
from crewai import Agent
from langchain.agents import Tool
from langchain.utilities import GoogleSerperAPIWrapper
# Set up your SERPER_API_KEY key in an .env file, eg:
# SERPER_API_KEY=<your api key>
load_dotenv()
# Setup API keys
os.environ["SERPER_API_KEY"] = "Your Key"
search = GoogleSerperAPIWrapper()
class SearchTool(BaseTool):
name: str = "Search"
description: str = "Useful for search-based queries. Use this to find current information about markets, companies, and trends."
search: GoogleSerperAPIWrapper = Field(default_factory=GoogleSerperAPIWrapper)
# Create and assign the search tool to an agent
serper_tool = Tool(
name="Intermediate Answer",
func=search.run,
description="Useful for search-based queries",
)
def _run(self, query: str) -> str:
"""Execute the search query and return results"""
try:
return self.search.run(query)
except Exception as e:
return f"Error performing search: {str(e)}"
# Create Agents
researcher = Agent(
role='Research Analyst',
goal='Gather current market data and trends',
backstory="""You are an expert research analyst with years of experience in
gathering market intelligence. You're known for your ability to find
relevant and up-to-date market information and present it in a clear,
actionable format.""",
tools=[SearchTool()],
verbose=True
agent = Agent(
role='Research Analyst',
goal='Provide up-to-date market analysis',
backstory='An expert analyst with a keen eye for market trends.',
tools=[serper_tool]
)
# rest of the code ...
@@ -53,6 +40,6 @@ researcher = Agent(
## Conclusion
Tools are pivotal in extending the capabilities of CrewAI agents, enabling them to undertake a broad spectrum of tasks and collaborate effectively.
When building solutions with CrewAI, leverage both custom and existing tools to empower your agents and enhance the AI ecosystem. Consider utilizing error handling, caching mechanisms,
and the flexibility of tool arguments to optimize your agents' performance and capabilities.
Tools are pivotal in extending the capabilities of CrewAI agents, enabling them to undertake a broad spectrum of tasks and collaborate effectively.
When building solutions with CrewAI, leverage both custom and existing tools to empower your agents and enhance the AI ecosystem. Consider utilizing error handling, caching mechanisms,
and the flexibility of tool arguments to optimize your agents' performance and capabilities.

View File

@@ -1,730 +1,180 @@
---
title: 'LLMs'
description: 'A comprehensive guide to configuring and using Large Language Models (LLMs) in your CrewAI projects'
icon: 'microchip-ai'
title: LLMs
description: Learn how to configure and optimize LLMs for your CrewAI projects.
icon: microchip-ai
---
<Note>
CrewAI integrates with multiple LLM providers through LiteLLM, giving you the flexibility to choose the right model for your specific use case. This guide will help you understand how to configure and use different LLM providers in your CrewAI projects.
</Note>
## What are LLMs?
Large Language Models (LLMs) are the core intelligence behind CrewAI agents. They enable agents to understand context, make decisions, and generate human-like responses. Here's what you need to know:
<CardGroup cols={2}>
<Card title="LLM Basics" icon="brain">
Large Language Models are AI systems trained on vast amounts of text data. They power the intelligence of your CrewAI agents, enabling them to understand and generate human-like text.
</Card>
<Card title="Context Window" icon="window">
The context window determines how much text an LLM can process at once. Larger windows (e.g., 128K tokens) allow for more context but may be more expensive and slower.
</Card>
<Card title="Temperature" icon="temperature-three-quarters">
Temperature (0.0 to 1.0) controls response randomness. Lower values (e.g., 0.2) produce more focused, deterministic outputs, while higher values (e.g., 0.8) increase creativity and variability.
</Card>
<Card title="Provider Selection" icon="server">
Each LLM provider (e.g., OpenAI, Anthropic, Google) offers different models with varying capabilities, pricing, and features. Choose based on your needs for accuracy, speed, and cost.
</Card>
</CardGroup>
## Available Models and Their Capabilities
Here's a detailed breakdown of supported models and their capabilities, you can compare performance at [lmarena.ai](https://lmarena.ai/?leaderboard) and [artificialanalysis.ai](https://artificialanalysis.ai/):
<Tabs>
<Tab title="OpenAI">
| Model | Context Window | Best For |
|-------|---------------|-----------|
| GPT-4 | 8,192 tokens | High-accuracy tasks, complex reasoning |
| GPT-4 Turbo | 128,000 tokens | Long-form content, document analysis |
| GPT-4o & GPT-4o-mini | 128,000 tokens | Cost-effective large context processing |
<Note>
1 token ≈ 4 characters in English. For example, 8,192 tokens ≈ 32,768 characters or about 6,000 words.
</Note>
</Tab>
<Tab title="Nvidia NIM">
| Model | Context Window | Best For |
|-------|---------------|-----------|
| nvidia/mistral-nemo-minitron-8b-8k-instruct | 8,192 tokens | State-of-the-art small language model delivering superior accuracy for chatbot, virtual assistants, and content generation. |
| nvidia/nemotron-4-mini-hindi-4b-instruct| 4,096 tokens | A bilingual Hindi-English SLM for on-device inference, tailored specifically for Hindi Language. |
| "nvidia/llama-3.1-nemotron-70b-instruct | 128k tokens | Llama-3.1-Nemotron-70B-Instruct is a large language model customized by NVIDIA in order to improve the helpfulness of LLM generated responses. |
| nvidia/llama3-chatqa-1.5-8b | 128k tokens | Advanced LLM to generate high-quality, context-aware responses for chatbots and search engines. |
| nvidia/llama3-chatqa-1.5-70b | 128k tokens | Advanced LLM to generate high-quality, context-aware responses for chatbots and search engines. |
| nvidia/vila | 128k tokens | Multi-modal vision-language model that understands text/img/video and creates informative responses |
| nvidia/neva-22| 4,096 tokens | Multi-modal vision-language model that understands text/images and generates informative responses |
| nvidia/nemotron-mini-4b-instruct | 8,192 tokens | General-purpose tasks |
| nvidia/usdcode-llama3-70b-instruct | 128k tokens | State-of-the-art LLM that answers OpenUSD knowledge queries and generates USD-Python code. |
| nvidia/nemotron-4-340b-instruct | 4,096 tokens | Creates diverse synthetic data that mimics the characteristics of real-world data. |
| meta/codellama-70b | 100k tokens | LLM capable of generating code from natural language and vice versa. |
| meta/llama2-70b | 4,096 tokens | Cutting-edge large language AI model capable of generating text and code in response to prompts. |
| meta/llama3-8b-instruct | 8,192 tokens | Advanced state-of-the-art LLM with language understanding, superior reasoning, and text generation. |
| meta/llama3-70b-instruct | 8,192 tokens | Powers complex conversations with superior contextual understanding, reasoning and text generation. |
| meta/llama-3.1-8b-instruct | 128k tokens | Advanced state-of-the-art model with language understanding, superior reasoning, and text generation. |
| meta/llama-3.1-70b-instruct | 128k tokens | Powers complex conversations with superior contextual understanding, reasoning and text generation. |
| meta/llama-3.1-405b-instruct | 128k tokens | Advanced LLM for synthetic data generation, distillation, and inference for chatbots, coding, and domain-specific tasks. |
| meta/llama-3.2-1b-instruct | 128k tokens | Advanced state-of-the-art small language model with language understanding, superior reasoning, and text generation. |
| meta/llama-3.2-3b-instruct | 128k tokens | Advanced state-of-the-art small language model with language understanding, superior reasoning, and text generation. |
| meta/llama-3.2-11b-vision-instruct | 128k tokens | Advanced state-of-the-art small language model with language understanding, superior reasoning, and text generation. |
| meta/llama-3.2-90b-vision-instruct | 128k tokens | Advanced state-of-the-art small language model with language understanding, superior reasoning, and text generation. |
| meta/llama-3.1-70b-instruct | 128k tokens | Powers complex conversations with superior contextual understanding, reasoning and text generation. |
| google/gemma-7b | 8,192 tokens | Cutting-edge text generation model text understanding, transformation, and code generation. |
| google/gemma-2b | 8,192 tokens | Cutting-edge text generation model text understanding, transformation, and code generation. |
| google/codegemma-7b | 8,192 tokens | Cutting-edge model built on Google's Gemma-7B specialized for code generation and code completion. |
| google/codegemma-1.1-7b | 8,192 tokens | Advanced programming model for code generation, completion, reasoning, and instruction following. |
| google/recurrentgemma-2b | 8,192 tokens | Novel recurrent architecture based language model for faster inference when generating long sequences. |
| google/gemma-2-9b-it | 8,192 tokens | Cutting-edge text generation model text understanding, transformation, and code generation. |
| google/gemma-2-27b-it | 8,192 tokens | Cutting-edge text generation model text understanding, transformation, and code generation. |
| google/gemma-2-2b-it | 8,192 tokens | Cutting-edge text generation model text understanding, transformation, and code generation. |
| google/deplot | 512 tokens | One-shot visual language understanding model that translates images of plots into tables. |
| google/paligemma | 8,192 tokens | Vision language model adept at comprehending text and visual inputs to produce informative responses. |
| mistralai/mistral-7b-instruct-v0.2 | 32k tokens | This LLM follows instructions, completes requests, and generates creative text. |
| mistralai/mixtral-8x7b-instruct-v0.1 | 8,192 tokens | An MOE LLM that follows instructions, completes requests, and generates creative text. |
| mistralai/mistral-large | 4,096 tokens | Creates diverse synthetic data that mimics the characteristics of real-world data. |
| mistralai/mixtral-8x22b-instruct-v0.1 | 8,192 tokens | Creates diverse synthetic data that mimics the characteristics of real-world data. |
| mistralai/mistral-7b-instruct-v0.3 | 32k tokens | This LLM follows instructions, completes requests, and generates creative text. |
| nv-mistralai/mistral-nemo-12b-instruct | 128k tokens | Most advanced language model for reasoning, code, multilingual tasks; runs on a single GPU. |
| mistralai/mamba-codestral-7b-v0.1 | 256k tokens | Model for writing and interacting with code across a wide range of programming languages and tasks. |
| microsoft/phi-3-mini-128k-instruct | 128K tokens | Lightweight, state-of-the-art open LLM with strong math and logical reasoning skills. |
| microsoft/phi-3-mini-4k-instruct | 4,096 tokens | Lightweight, state-of-the-art open LLM with strong math and logical reasoning skills. |
| microsoft/phi-3-small-8k-instruct | 8,192 tokens | Lightweight, state-of-the-art open LLM with strong math and logical reasoning skills. |
| microsoft/phi-3-small-128k-instruct | 128K tokens | Lightweight, state-of-the-art open LLM with strong math and logical reasoning skills. |
| microsoft/phi-3-medium-4k-instruct | 4,096 tokens | Lightweight, state-of-the-art open LLM with strong math and logical reasoning skills. |
| microsoft/phi-3-medium-128k-instruct | 128K tokens | Lightweight, state-of-the-art open LLM with strong math and logical reasoning skills. |
| microsoft/phi-3.5-mini-instruct | 128K tokens | Lightweight multilingual LLM powering AI applications in latency bound, memory/compute constrained environments |
| microsoft/phi-3.5-moe-instruct | 128K tokens | Advanced LLM based on Mixture of Experts architecure to deliver compute efficient content generation |
| microsoft/kosmos-2 | 1,024 tokens | Groundbreaking multimodal model designed to understand and reason about visual elements in images. |
| microsoft/phi-3-vision-128k-instruct | 128k tokens | Cutting-edge open multimodal model exceling in high-quality reasoning from images. |
| microsoft/phi-3.5-vision-instruct | 128k tokens | Cutting-edge open multimodal model exceling in high-quality reasoning from images. |
| databricks/dbrx-instruct | 12k tokens | A general-purpose LLM with state-of-the-art performance in language understanding, coding, and RAG. |
| snowflake/arctic | 1,024 tokens | Delivers high efficiency inference for enterprise applications focused on SQL generation and coding. |
| aisingapore/sea-lion-7b-instruct | 4,096 tokens | LLM to represent and serve the linguistic and cultural diversity of Southeast Asia |
| ibm/granite-8b-code-instruct | 4,096 tokens | Software programming LLM for code generation, completion, explanation, and multi-turn conversion. |
| ibm/granite-34b-code-instruct | 8,192 tokens | Software programming LLM for code generation, completion, explanation, and multi-turn conversion. |
| ibm/granite-3.0-8b-instruct | 4,096 tokens | Advanced Small Language Model supporting RAG, summarization, classification, code, and agentic AI |
| ibm/granite-3.0-3b-a800m-instruct | 4,096 tokens | Highly efficient Mixture of Experts model for RAG, summarization, entity extraction, and classification |
| mediatek/breeze-7b-instruct | 4,096 tokens | Creates diverse synthetic data that mimics the characteristics of real-world data. |
| upstage/solar-10.7b-instruct | 4,096 tokens | Excels in NLP tasks, particularly in instruction-following, reasoning, and mathematics. |
| writer/palmyra-med-70b-32k | 32k tokens | Leading LLM for accurate, contextually relevant responses in the medical domain. |
| writer/palmyra-med-70b | 32k tokens | Leading LLM for accurate, contextually relevant responses in the medical domain. |
| writer/palmyra-fin-70b-32k | 32k tokens | Specialized LLM for financial analysis, reporting, and data processing |
| 01-ai/yi-large | 32k tokens | Powerful model trained on English and Chinese for diverse tasks including chatbot and creative writing. |
| deepseek-ai/deepseek-coder-6.7b-instruct | 2k tokens | Powerful coding model offering advanced capabilities in code generation, completion, and infilling |
| rakuten/rakutenai-7b-instruct | 1,024 tokens | Advanced state-of-the-art LLM with language understanding, superior reasoning, and text generation. |
| rakuten/rakutenai-7b-chat | 1,024 tokens | Advanced state-of-the-art LLM with language understanding, superior reasoning, and text generation. |
| baichuan-inc/baichuan2-13b-chat | 4,096 tokens | Support Chinese and English chat, coding, math, instruction following, solving quizzes |
<Note>
NVIDIA's NIM support for models is expanding continuously! For the most up-to-date list of available models, please visit build.nvidia.com.
</Note>
</Tab>
<Tab title="Gemini">
| Model | Context Window | Best For |
|-------|---------------|-----------|
| gemini-2.0-flash-exp | 1M tokens | Higher quality at faster speed, multimodal model, good for most tasks |
| gemini-1.5-flash | 1M tokens | Balanced multimodal model, good for most tasks |
| gemini-1.5-flash-8B | 1M tokens | Fastest, most cost-efficient, good for high-frequency tasks |
| gemini-1.5-pro | 2M tokens | Best performing, wide variety of reasoning tasks including logical reasoning, coding, and creative collaboration |
<Tip>
Google's Gemini models are all multimodal, supporting audio, images, video and text, supporting context caching, json schema, function calling, etc.
These models are available via API_KEY from
[The Gemini API](https://ai.google.dev/gemini-api/docs) and also from
[Google Cloud Vertex](https://cloud.google.com/vertex-ai/generative-ai/docs/migrate/migrate-google-ai) as part of the
[Model Garden](https://cloud.google.com/vertex-ai/generative-ai/docs/model-garden/explore-models).
</Tip>
</Tab>
<Tab title="Groq">
| Model | Context Window | Best For |
|-------|---------------|-----------|
| Llama 3.1 70B/8B | 131,072 tokens | High-performance, large context tasks |
| Llama 3.2 Series | 8,192 tokens | General-purpose tasks |
| Mixtral 8x7B | 32,768 tokens | Balanced performance and context |
<Tip>
Groq is known for its fast inference speeds, making it suitable for real-time applications.
</Tip>
</Tab>
<Tab title="Others">
| Provider | Context Window | Key Features |
|----------|---------------|--------------|
| Deepseek Chat | 128,000 tokens | Specialized in technical discussions |
| Claude 3 | Up to 200K tokens | Strong reasoning, code understanding |
| Gemma Series | 8,192 tokens | Efficient, smaller-scale tasks |
<Info>
Provider selection should consider factors like:
- API availability in your region
- Pricing structure
- Required features (e.g., streaming, function calling)
- Performance requirements
</Info>
</Tab>
</Tabs>
## Setting Up Your LLM
There are three ways to configure LLMs in CrewAI. Choose the method that best fits your workflow:
<Tabs>
<Tab title="1. Environment Variables">
The simplest way to get started. Set these variables in your environment:
```bash
# Required: Your API key for authentication
OPENAI_API_KEY=<your-api-key>
# Optional: Default model selection
OPENAI_MODEL_NAME=gpt-4o-mini # Default if not set
# Optional: Organization ID (if applicable)
OPENAI_ORGANIZATION_ID=<your-org-id>
```
<Warning>
Never commit API keys to version control. Use environment files (.env) or your system's secret management.
</Warning>
</Tab>
<Tab title="2. YAML Configuration">
Create a YAML file to define your agent configurations. This method is great for version control and team collaboration:
```yaml
researcher:
# Agent Definition
role: Research Specialist
goal: Conduct comprehensive research and analysis
backstory: A dedicated research professional with years of experience
verbose: true
# Model Selection (uncomment your choice)
# OpenAI Models - Known for reliability and performance
llm: openai/gpt-4o-mini
# llm: openai/gpt-4 # More accurate but expensive
# llm: openai/gpt-4-turbo # Fast with large context
# llm: openai/gpt-4o # Optimized for longer texts
# llm: openai/o1-preview # Latest features
# llm: openai/o1-mini # Cost-effective
# Azure Models - For enterprise deployments
# llm: azure/gpt-4o-mini
# llm: azure/gpt-4
# llm: azure/gpt-35-turbo
# Anthropic Models - Strong reasoning capabilities
# llm: anthropic/claude-3-opus-20240229-v1:0
# llm: anthropic/claude-3-sonnet-20240229-v1:0
# llm: anthropic/claude-3-haiku-20240307-v1:0
# llm: anthropic/claude-2.1
# llm: anthropic/claude-2.0
# Google Models - Strong reasoning, large cachable context window, multimodal
# llm: gemini/gemini-1.5-pro-latest
# llm: gemini/gemini-1.5-flash-latest
# llm: gemini/gemini-1.5-flash-8b-latest
# AWS Bedrock Models - Enterprise-grade
# llm: bedrock/anthropic.claude-3-sonnet-20240229-v1:0
# llm: bedrock/anthropic.claude-v2:1
# llm: bedrock/amazon.titan-text-express-v1
# llm: bedrock/meta.llama2-70b-chat-v1
# Mistral Models - Open source alternative
# llm: mistral/mistral-large-latest
# llm: mistral/mistral-medium-latest
# llm: mistral/mistral-small-latest
# Groq Models - Fast inference
# llm: groq/mixtral-8x7b-32768
# llm: groq/llama-3.1-70b-versatile
# llm: groq/llama-3.2-90b-text-preview
# llm: groq/gemma2-9b-it
# llm: groq/gemma-7b-it
# IBM watsonx.ai Models - Enterprise features
# llm: watsonx/ibm/granite-13b-chat-v2
# llm: watsonx/meta-llama/llama-3-1-70b-instruct
# llm: watsonx/bigcode/starcoder2-15b
# Ollama Models - Local deployment
# llm: ollama/llama3:70b
# llm: ollama/codellama
# llm: ollama/mistral
# llm: ollama/mixtral
# llm: ollama/phi
# Fireworks AI Models - Specialized tasks
# llm: fireworks_ai/accounts/fireworks/models/llama-v3-70b-instruct
# llm: fireworks_ai/accounts/fireworks/models/mixtral-8x7b
# llm: fireworks_ai/accounts/fireworks/models/zephyr-7b-beta
# Perplexity AI Models - Research focused
# llm: pplx/llama-3.1-sonar-large-128k-online
# llm: pplx/mistral-7b-instruct
# llm: pplx/codellama-34b-instruct
# llm: pplx/mixtral-8x7b-instruct
# Hugging Face Models - Community models
# llm: huggingface/meta-llama/Meta-Llama-3.1-8B-Instruct
# llm: huggingface/mistralai/Mixtral-8x7B-Instruct-v0.1
# llm: huggingface/tiiuae/falcon-180B-chat
# llm: huggingface/google/gemma-7b-it
# Nvidia NIM Models - GPU-optimized
# llm: nvidia_nim/meta/llama3-70b-instruct
# llm: nvidia_nim/mistral/mixtral-8x7b
# llm: nvidia_nim/google/gemma-7b
# SambaNova Models - Enterprise AI
# llm: sambanova/Meta-Llama-3.1-8B-Instruct
# llm: sambanova/BioMistral-7B
# llm: sambanova/Falcon-180B
```
<Info>
The YAML configuration allows you to:
- Version control your agent settings
- Easily switch between different models
- Share configurations across team members
- Document model choices and their purposes
</Info>
</Tab>
<Tab title="3. Direct Code">
For maximum flexibility, configure LLMs directly in your Python code:
```python
from crewai import LLM
# Basic configuration
llm = LLM(model="gpt-4")
# Advanced configuration with detailed parameters
llm = LLM(
model="gpt-4o-mini",
temperature=0.7, # Higher for more creative outputs
timeout=120, # Seconds to wait for response
max_tokens=4000, # Maximum length of response
top_p=0.9, # Nucleus sampling parameter
frequency_penalty=0.1, # Reduce repetition
presence_penalty=0.1, # Encourage topic diversity
response_format={"type": "json"}, # For structured outputs
seed=42 # For reproducible results
)
```
<Info>
Parameter explanations:
- `temperature`: Controls randomness (0.0-1.0)
- `timeout`: Maximum wait time for response
- `max_tokens`: Limits response length
- `top_p`: Alternative to temperature for sampling
- `frequency_penalty`: Reduces word repetition
- `presence_penalty`: Encourages new topics
- `response_format`: Specifies output structure
- `seed`: Ensures consistent outputs
</Info>
</Tab>
</Tabs>
## Advanced Features and Optimization
Learn how to get the most out of your LLM configuration:
<AccordionGroup>
<Accordion title="Context Window Management">
CrewAI includes smart context management features:
```python
from crewai import LLM
# CrewAI automatically handles:
# 1. Token counting and tracking
# 2. Content summarization when needed
# 3. Task splitting for large contexts
llm = LLM(
model="gpt-4",
max_tokens=4000, # Limit response length
)
```
<Info>
Best practices for context management:
1. Choose models with appropriate context windows
2. Pre-process long inputs when possible
3. Use chunking for large documents
4. Monitor token usage to optimize costs
</Info>
</Accordion>
<Accordion title="Performance Optimization">
<Steps>
<Step title="Token Usage Optimization">
Choose the right context window for your task:
- Small tasks (up to 4K tokens): Standard models
- Medium tasks (between 4K-32K): Enhanced models
- Large tasks (over 32K): Large context models
```python
# Configure model with appropriate settings
llm = LLM(
model="openai/gpt-4-turbo-preview",
temperature=0.7, # Adjust based on task
max_tokens=4096, # Set based on output needs
timeout=300 # Longer timeout for complex tasks
)
```
<Tip>
- Lower temperature (0.1 to 0.3) for factual responses
- Higher temperature (0.7 to 0.9) for creative tasks
</Tip>
</Step>
<Step title="Best Practices">
1. Monitor token usage
2. Implement rate limiting
3. Use caching when possible
4. Set appropriate max_tokens limits
</Step>
</Steps>
<Info>
Remember to regularly monitor your token usage and adjust your configuration as needed to optimize costs and performance.
</Info>
</Accordion>
</AccordionGroup>
## Provider Configuration Examples
<AccordionGroup>
<Accordion title="OpenAI">
```python Code
# Required
OPENAI_API_KEY=sk-...
# Optional
OPENAI_API_BASE=<custom-base-url>
OPENAI_ORGANIZATION=<your-org-id>
```
Example usage:
```python Code
from crewai import LLM
llm = LLM(
model="gpt-4",
temperature=0.8,
max_tokens=150,
top_p=0.9,
frequency_penalty=0.1,
presence_penalty=0.1,
stop=["END"],
seed=42
)
```
</Accordion>
<Accordion title="Anthropic">
```python Code
ANTHROPIC_API_KEY=sk-ant-...
```
Example usage:
```python Code
llm = LLM(
model="anthropic/claude-3-sonnet-20240229-v1:0",
temperature=0.7
)
```
</Accordion>
<Accordion title="Google">
```python Code
# Option 1. Gemini accessed with an API key.
# https://ai.google.dev/gemini-api/docs/api-key
GEMINI_API_KEY=<your-api-key>
# Option 2. Vertex AI IAM credentials for Gemini, Anthropic, and anything in the Model Garden.
# https://cloud.google.com/vertex-ai/generative-ai/docs/overview
```
Example usage:
```python Code
llm = LLM(
model="gemini/gemini-1.5-pro-latest",
temperature=0.7
)
```
</Accordion>
<Accordion title="Azure">
```python Code
# Required
AZURE_API_KEY=<your-api-key>
AZURE_API_BASE=<your-resource-url>
AZURE_API_VERSION=<api-version>
# Optional
AZURE_AD_TOKEN=<your-azure-ad-token>
AZURE_API_TYPE=<your-azure-api-type>
```
Example usage:
```python Code
llm = LLM(
model="azure/gpt-4",
api_version="2023-05-15"
)
```
</Accordion>
<Accordion title="AWS Bedrock">
```python Code
AWS_ACCESS_KEY_ID=<your-access-key>
AWS_SECRET_ACCESS_KEY=<your-secret-key>
AWS_DEFAULT_REGION=<your-region>
```
Example usage:
```python Code
llm = LLM(
model="bedrock/anthropic.claude-3-sonnet-20240229-v1:0"
)
```
</Accordion>
<Accordion title="Mistral">
```python Code
MISTRAL_API_KEY=<your-api-key>
```
Example usage:
```python Code
llm = LLM(
model="mistral/mistral-large-latest",
temperature=0.7
)
```
</Accordion>
<Accordion title="Nvidia NIM">
```python Code
NVIDIA_API_KEY=<your-api-key>
```
Example usage:
```python Code
llm = LLM(
model="nvidia_nim/meta/llama3-70b-instruct",
temperature=0.7
)
```
</Accordion>
<Accordion title="Groq">
```python Code
GROQ_API_KEY=<your-api-key>
```
Example usage:
```python Code
llm = LLM(
model="groq/llama-3.2-90b-text-preview",
temperature=0.7
)
```
</Accordion>
<Accordion title="IBM watsonx.ai">
```python Code
# Required
WATSONX_URL=<your-url>
WATSONX_APIKEY=<your-apikey>
WATSONX_PROJECT_ID=<your-project-id>
# Optional
WATSONX_TOKEN=<your-token>
WATSONX_DEPLOYMENT_SPACE_ID=<your-space-id>
```
Example usage:
```python Code
llm = LLM(
model="watsonx/meta-llama/llama-3-1-70b-instruct",
base_url="https://api.watsonx.ai/v1"
)
```
</Accordion>
<Accordion title="Ollama (Local LLMs)">
1. Install Ollama: [ollama.ai](https://ollama.ai/)
2. Run a model: `ollama run llama2`
3. Configure:
```python Code
llm = LLM(
model="ollama/llama3:70b",
base_url="http://localhost:11434"
)
```
</Accordion>
<Accordion title="Fireworks AI">
```python Code
FIREWORKS_API_KEY=<your-api-key>
```
Example usage:
```python Code
llm = LLM(
model="fireworks_ai/accounts/fireworks/models/llama-v3-70b-instruct",
temperature=0.7
)
```
</Accordion>
<Accordion title="Perplexity AI">
```python Code
PERPLEXITY_API_KEY=<your-api-key>
```
Example usage:
```python Code
llm = LLM(
model="llama-3.1-sonar-large-128k-online",
base_url="https://api.perplexity.ai/"
)
```
</Accordion>
<Accordion title="Hugging Face">
```python Code
HUGGINGFACE_API_KEY=<your-api-key>
```
Example usage:
```python Code
llm = LLM(
model="huggingface/meta-llama/Meta-Llama-3.1-8B-Instruct",
base_url="your_api_endpoint"
)
```
</Accordion>
<Accordion title="SambaNova">
```python Code
SAMBANOVA_API_KEY=<your-api-key>
```
Example usage:
```python Code
llm = LLM(
model="sambanova/Meta-Llama-3.1-8B-Instruct",
temperature=0.7
)
```
</Accordion>
<Accordion title="Cerebras">
```python Code
# Required
CEREBRAS_API_KEY=<your-api-key>
```
Example usage:
```python Code
llm = LLM(
model="cerebras/llama3.1-70b",
temperature=0.7,
max_tokens=8192
)
```
<Info>
Cerebras features:
- Fast inference speeds
- Competitive pricing
- Good balance of speed and quality
- Support for long context windows
</Info>
</Accordion>
</AccordionGroup>
## Common Issues and Solutions
<Tabs>
<Tab title="Authentication">
<Warning>
Most authentication issues can be resolved by checking API key format and environment variable names.
</Warning>
```bash
# OpenAI
OPENAI_API_KEY=sk-...
# Anthropic
ANTHROPIC_API_KEY=sk-ant-...
```
</Tab>
<Tab title="Model Names">
<Check>
Always include the provider prefix in model names
</Check>
```python
# Correct
llm = LLM(model="openai/gpt-4")
# Incorrect
llm = LLM(model="gpt-4")
```
</Tab>
<Tab title="Context Length">
<Tip>
Use larger context models for extensive tasks
</Tip>
```python
# Large context model
llm = LLM(model="openai/gpt-4o") # 128K tokens
```
</Tab>
</Tabs>
## Getting Help
If you need assistance, these resources are available:
<CardGroup cols={3}>
<Card
title="LiteLLM Documentation"
href="https://docs.litellm.ai/docs/"
icon="book"
>
Comprehensive documentation for LiteLLM integration and troubleshooting common issues.
</Card>
<Card
title="GitHub Issues"
href="https://github.com/joaomdmoura/crewAI/issues"
icon="bug"
>
Report bugs, request features, or browse existing issues for solutions.
</Card>
<Card
title="Community Forum"
href="https://community.crewai.com"
icon="comment-question"
>
Connect with other CrewAI users, share experiences, and get help from the community.
</Card>
</CardGroup>
<Note>
Best Practices for API Key Security:
- Use environment variables or secure vaults
- Never commit keys to version control
- Rotate keys regularly
- Use separate keys for development and production
- Monitor key usage for unusual patterns
</Note>
# Large Language Models (LLMs) in CrewAI
Large Language Models (LLMs) are the backbone of intelligent agents in the CrewAI framework. This guide will help you understand, configure, and optimize LLM usage for your CrewAI projects.
## Key Concepts
- **LLM**: Large Language Model, the AI powering agent intelligence
- **Agent**: A CrewAI entity that uses an LLM to perform tasks
- **Provider**: A service that offers LLM capabilities (e.g., OpenAI, Anthropic, Ollama, [more providers](https://docs.litellm.ai/docs/providers))
## Configuring LLMs for Agents
CrewAI offers flexible options for setting up LLMs:
### 1. Default Configuration
By default, CrewAI uses the `gpt-4o-mini` model. It uses environment variables if no LLM is specified:
- `OPENAI_MODEL_NAME` (defaults to "gpt-4o-mini" if not set)
- `OPENAI_API_BASE`
- `OPENAI_API_KEY`
### 2. String Identifier
```python Code
agent = Agent(llm="gpt-4o", ...)
```
### 3. LLM Instance
List of [more providers](https://docs.litellm.ai/docs/providers).
```python Code
from crewai import LLM
llm = LLM(model="gpt-4", temperature=0.7)
agent = Agent(llm=llm, ...)
```
### 4. Custom LLM Objects
Pass a custom LLM implementation or object from another library.
## Connecting to OpenAI-Compatible LLMs
You can connect to OpenAI-compatible LLMs using either environment variables or by setting specific attributes on the LLM class:
1. Using environment variables:
```python Code
import os
os.environ["OPENAI_API_KEY"] = "your-api-key"
os.environ["OPENAI_API_BASE"] = "https://api.your-provider.com/v1"
```
2. Using LLM class attributes:
```python Code
from crewai import LLM
llm = LLM(
model="custom-model-name",
api_key="your-api-key",
base_url="https://api.your-provider.com/v1"
)
agent = Agent(llm=llm, ...)
```
## LLM Configuration Options
When configuring an LLM for your agent, you have access to a wide range of parameters:
| Parameter | Type | Description |
|:------------------|:---------------:|:-------------------------------------------------------------------------------------------------|
| **model** | `str` | Name of the model to use (e.g., "gpt-4", "gpt-3.5-turbo", "ollama/llama3.1"). For more options, visit the providers documentation. |
| **timeout** | `float, int` | Maximum time (in seconds) to wait for a response. |
| **temperature** | `float` | Controls randomness in output (0.0 to 1.0). |
| **top_p** | `float` | Controls diversity of output (0.0 to 1.0). |
| **n** | `int` | Number of completions to generate. |
| **stop** | `str, List[str]` | Sequence(s) where generation should stop. |
| **max_tokens** | `int` | Maximum number of tokens to generate. |
| **presence_penalty** | `float` | Penalizes new tokens based on their presence in prior text. |
| **frequency_penalty**| `float` | Penalizes new tokens based on their frequency in prior text. |
| **logit_bias** | `Dict[int, float]`| Modifies likelihood of specified tokens appearing. |
| **response_format** | `Dict[str, Any]` | Specifies the format of the response (e.g., JSON object). |
| **seed** | `int` | Sets a random seed for deterministic results. |
| **logprobs** | `bool` | Returns log probabilities of output tokens if enabled. |
| **top_logprobs** | `int` | Number of most likely tokens for which to return log probabilities. |
| **base_url** | `str` | The base URL for the API endpoint. |
| **api_version** | `str` | Version of the API to use. |
| **api_key** | `str` | Your API key for authentication. |
## OpenAI Example Configuration
```python Code
from crewai import LLM
llm = LLM(
model="gpt-4",
temperature=0.8,
max_tokens=150,
top_p=0.9,
frequency_penalty=0.1,
presence_penalty=0.1,
stop=["END"],
seed=42,
base_url="https://api.openai.com/v1",
api_key="your-api-key-here"
)
agent = Agent(llm=llm, ...)
```
## Cerebras Example Configuration
```python Code
from crewai import LLM
llm = LLM(
model="cerebras/llama-3.1-70b",
base_url="https://api.cerebras.ai/v1",
api_key="your-api-key-here"
)
agent = Agent(llm=llm, ...)
```
## Using Ollama (Local LLMs)
CrewAI supports using Ollama for running open-source models locally:
1. Install Ollama: [ollama.ai](https://ollama.ai/)
2. Run a model: `ollama run llama2`
3. Configure agent:
```python Code
from crewai import LLM
agent = Agent(
llm=LLM(model="ollama/llama3.1", base_url="http://localhost:11434"),
...
)
```
## Changing the Base API URL
You can change the base API URL for any LLM provider by setting the `base_url` parameter:
```python Code
from crewai import LLM
llm = LLM(
model="custom-model-name",
base_url="https://api.your-provider.com/v1",
api_key="your-api-key"
)
agent = Agent(llm=llm, ...)
```
This is particularly useful when working with OpenAI-compatible APIs or when you need to specify a different endpoint for your chosen provider.
## Best Practices
1. **Choose the right model**: Balance capability and cost.
2. **Optimize prompts**: Clear, concise instructions improve output.
3. **Manage tokens**: Monitor and limit token usage for efficiency.
4. **Use appropriate temperature**: Lower for factual tasks, higher for creative ones.
5. **Implement error handling**: Gracefully manage API errors and rate limits.
## Troubleshooting
- **API Errors**: Check your API key, network connection, and rate limits.
- **Unexpected Outputs**: Refine your prompts and adjust temperature or top_p.
- **Performance Issues**: Consider using a more powerful model or optimizing your queries.
- **Timeout Errors**: Increase the `timeout` parameter or optimize your input.

View File

@@ -18,7 +18,6 @@ reason, and learn from past interactions.
| **Long-Term Memory** | Preserves valuable insights and learnings from past executions, allowing agents to build and refine their knowledge over time. |
| **Entity Memory** | Captures and organizes information about entities (people, places, concepts) encountered during tasks, facilitating deeper understanding and relationship mapping. Uses `RAG` for storing entity information. |
| **Contextual Memory**| Maintains the context of interactions by combining `ShortTermMemory`, `LongTermMemory`, and `EntityMemory`, aiding in the coherence and relevance of agent responses over a sequence of tasks or a conversation. |
| **User Memory** | Stores user-specific information and preferences, enhancing personalization and user experience. |
## How Memory Systems Empower Agents
@@ -93,47 +92,6 @@ my_crew = Crew(
)
```
## Integrating Mem0 for Enhanced User Memory
[Mem0](https://mem0.ai/) is a self-improving memory layer for LLM applications, enabling personalized AI experiences.
To include user-specific memory you can get your API key [here](https://app.mem0.ai/dashboard/api-keys) and refer the [docs](https://docs.mem0.ai/platform/quickstart#4-1-create-memories) for adding user preferences.
```python Code
import os
from crewai import Crew, Process
from mem0 import MemoryClient
# Set environment variables for Mem0
os.environ["MEM0_API_KEY"] = "m0-xx"
# Step 1: Record preferences based on past conversation or user input
client = MemoryClient()
messages = [
{"role": "user", "content": "Hi there! I'm planning a vacation and could use some advice."},
{"role": "assistant", "content": "Hello! I'd be happy to help with your vacation planning. What kind of destination do you prefer?"},
{"role": "user", "content": "I am more of a beach person than a mountain person."},
{"role": "assistant", "content": "That's interesting. Do you like hotels or Airbnb?"},
{"role": "user", "content": "I like Airbnb more."},
]
client.add(messages, user_id="john")
# Step 2: Create a Crew with User Memory
crew = Crew(
agents=[...],
tasks=[...],
verbose=True,
process=Process.sequential,
memory=True,
memory_config={
"provider": "mem0",
"config": {"user_id": "john"},
},
)
```
## Additional Embedding Providers
@@ -296,31 +254,6 @@ my_crew = Crew(
)
```
### Using Watson embeddings
```python Code
from crewai import Crew, Agent, Task, Process
# Note: Ensure you have installed and imported `ibm_watsonx_ai` for Watson embeddings to work.
my_crew = Crew(
agents=[...],
tasks=[...],
process=Process.sequential,
memory=True,
verbose=True,
embedder={
"provider": "watson",
"config": {
"model": "<model_name>",
"api_url": "<api_url>",
"api_key": "<YOUR_API_KEY>",
"project_id": "<YOUR_PROJECT_ID>",
}
}
)
```
### Resetting Memory
```shell

View File

@@ -1,178 +1,48 @@
---
title: Tasks
description: Detailed guide on managing and creating tasks within the CrewAI framework.
description: Detailed guide on managing and creating tasks within the CrewAI framework, reflecting the latest codebase updates.
icon: list-check
---
## Overview of a Task
In the CrewAI framework, a `Task` is a specific assignment completed by an `Agent`.
In the CrewAI framework, a `Task` is a specific assignment completed by an `Agent`.
They provide all necessary details for execution, such as a description, the agent responsible, required tools, and more, facilitating a wide range of action complexities.
Tasks provide all necessary details for execution, such as a description, the agent responsible, required tools, and more, facilitating a wide range of action complexities.
Tasks within CrewAI can be collaborative, requiring multiple agents to work together. This is managed through the task properties and orchestrated by the Crew's process, enhancing teamwork and efficiency.
### Task Execution Flow
Tasks can be executed in two ways:
- **Sequential**: Tasks are executed in the order they are defined
- **Hierarchical**: Tasks are assigned to agents based on their roles and expertise
The execution flow is defined when creating the crew:
```python Code
crew = Crew(
agents=[agent1, agent2],
tasks=[task1, task2],
process=Process.sequential # or Process.hierarchical
)
```
## Task Attributes
| Attribute | Parameters | Type | Description |
| :------------------------------- | :---------------- | :---------------------------- | :------------------------------------------------------------------------------------------------------------------- |
| **Description** | `description` | `str` | A clear, concise statement of what the task entails. |
| **Agent** | `agent` | `Optional[BaseAgent]` | The agent responsible for the task, assigned either directly or by the crew's process. |
| **Expected Output** | `expected_output` | `str` | A detailed description of what the task's completion looks like. |
| **Name** _(optional)_ | `name` | `Optional[str]` | A name identifier for the task. |
| **Agent** _(optional)_ | `agent` | `Optional[BaseAgent]` | The agent responsible for executing the task. |
| **Tools** _(optional)_ | `tools` | `List[BaseTool]` | The tools/resources the agent is limited to use for this task. |
| **Context** _(optional)_ | `context` | `Optional[List["Task"]]` | Other tasks whose outputs will be used as context for this task. |
| **Async Execution** _(optional)_ | `async_execution` | `Optional[bool]` | Whether the task should be executed asynchronously. Defaults to False. |
| **Config** _(optional)_ | `config` | `Optional[Dict[str, Any]]` | Task-specific configuration parameters. |
| **Output File** _(optional)_ | `output_file` | `Optional[str]` | File path for storing the task output. |
| **Output JSON** _(optional)_ | `output_json` | `Optional[Type[BaseModel]]` | A Pydantic model to structure the JSON output. |
| **Output Pydantic** _(optional)_ | `output_pydantic` | `Optional[Type[BaseModel]]` | A Pydantic model for task output. |
| **Callback** _(optional)_ | `callback` | `Optional[Any]` | Function/object to be executed after task completion. |
| **Tools** _(optional)_ | `tools` | `Optional[List[Any]]` | The functions or capabilities the agent can utilize to perform the task. Defaults to an empty list. |
| **Async Execution** _(optional)_ | `async_execution` | `Optional[bool]` | If set, the task executes asynchronously, allowing progression without waiting for completion. Defaults to False. |
| **Context** _(optional)_ | `context` | `Optional[List["Task"]]` | Specifies tasks whose outputs are used as context for this task. |
| **Config** _(optional)_ | `config` | `Optional[Dict[str, Any]]` | Additional configuration details for the agent executing the task, allowing further customization. Defaults to None. |
| **Output JSON** _(optional)_ | `output_json` | `Optional[Type[BaseModel]]` | Outputs a JSON object, requiring an OpenAI client. Only one output format can be set. |
| **Output Pydantic** _(optional)_ | `output_pydantic` | `Optional[Type[BaseModel]]` | Outputs a Pydantic model object, requiring an OpenAI client. Only one output format can be set. |
| **Output File** _(optional)_ | `output_file` | `Optional[str]` | Saves the task output to a file. If used with `Output JSON` or `Output Pydantic`, specifies how the output is saved. |
| **Output** _(optional)_ | `output` | `Optional[TaskOutput]` | An instance of `TaskOutput`, containing the raw, JSON, and Pydantic output plus additional details. |
| **Callback** _(optional)_ | `callback` | `Optional[Any]` | A callable that is executed with the task's output upon completion. |
| **Human Input** _(optional)_ | `human_input` | `Optional[bool]` | Indicates if the task should involve human review at the end, useful for tasks needing human oversight. Defaults to False.|
| **Converter Class** _(optional)_ | `converter_cls` | `Optional[Type[Converter]]` | A converter class used to export structured output. Defaults to None. |
## Creating Tasks
## Creating a Task
There are two ways to create tasks in CrewAI: using **YAML configuration (recommended)** or defining them **directly in code**.
Creating a task involves defining its scope, responsible agent, and any additional attributes for flexibility:
### YAML Configuration (Recommended)
Using YAML configuration provides a cleaner, more maintainable way to define tasks. We strongly recommend using this approach to define tasks in your CrewAI projects.
After creating your CrewAI project as outlined in the [Installation](/installation) section, navigate to the `src/latest_ai_development/config/tasks.yaml` file and modify the template to match your specific task requirements.
<Note>
Variables in your YAML files (like `{topic}`) will be replaced with values from your inputs when running the crew:
```python Code
crew.kickoff(inputs={'topic': 'AI Agents'})
```
</Note>
Here's an example of how to configure tasks using YAML:
```yaml tasks.yaml
research_task:
description: >
Conduct a thorough research about {topic}
Make sure you find any interesting and relevant information given
the current year is 2024.
expected_output: >
A list with 10 bullet points of the most relevant information about {topic}
agent: researcher
reporting_task:
description: >
Review the context you got and expand each topic into a full section for a report.
Make sure the report is detailed and contains any and all relevant information.
expected_output: >
A fully fledge reports with the mains topics, each with a full section of information.
Formatted as markdown without '```'
agent: reporting_analyst
output_file: report.md
```
To use this YAML configuration in your code, create a crew class that inherits from `CrewBase`:
```python crew.py
# src/latest_ai_development/crew.py
from crewai import Agent, Crew, Process, Task
from crewai.project import CrewBase, agent, crew, task
from crewai_tools import SerperDevTool
@CrewBase
class LatestAiDevelopmentCrew():
"""LatestAiDevelopment crew"""
@agent
def researcher(self) -> Agent:
return Agent(
config=self.agents_config['researcher'],
verbose=True,
tools=[SerperDevTool()]
)
@agent
def reporting_analyst(self) -> Agent:
return Agent(
config=self.agents_config['reporting_analyst'],
verbose=True
)
@task
def research_task(self) -> Task:
return Task(
config=self.tasks_config['research_task']
)
@task
def reporting_task(self) -> Task:
return Task(
config=self.tasks_config['reporting_task']
)
@crew
def crew(self) -> Crew:
return Crew(
agents=[
self.researcher(),
self.reporting_analyst()
],
tasks=[
self.research_task(),
self.reporting_task()
],
process=Process.sequential
)
```
<Note>
The names you use in your YAML files (`agents.yaml` and `tasks.yaml`) should match the method names in your Python code.
</Note>
### Direct Code Definition (Alternative)
Alternatively, you can define tasks directly in your code without using YAML configuration:
```python task.py
from crewai import Task
research_task = Task(
description="""
Conduct a thorough research about AI Agents.
Make sure you find any interesting and relevant information given
the current year is 2024.
""",
expected_output="""
A list with 10 bullet points of the most relevant information about AI Agents
""",
agent=researcher
)
reporting_task = Task(
description="""
Review the context you got and expand each topic into a full section for a report.
Make sure the report is detailed and contains any and all relevant information.
""",
expected_output="""
A fully fledge reports with the mains topics, each with a full section of information.
Formatted as markdown without '```'
""",
agent=reporting_analyst,
output_file="report.md"
task = Task(
description='Find and summarize the latest and most relevant news on AI',
agent=sales_agent,
expected_output='A bullet list summary of the top 5 most important AI news',
)
```
@@ -182,8 +52,6 @@ reporting_task = Task(
## Task Output
Understanding task outputs is crucial for building effective AI workflows. CrewAI provides a structured way to handle task results through the `TaskOutput` class, which supports multiple output formats and can be easily passed between tasks.
The output of a task in CrewAI framework is encapsulated within the `TaskOutput` class. This class provides a structured way to access results of a task, including various formats such as raw output, JSON, and Pydantic models.
By default, the `TaskOutput` will only include the `raw` output. A `TaskOutput` will only include the `pydantic` or `json_dict` output if the original `Task` object was configured with `output_pydantic` or `output_json`, respectively.
@@ -244,326 +112,6 @@ if task_output.pydantic:
print(f"Pydantic Output: {task_output.pydantic}")
```
## Task Dependencies and Context
Tasks can depend on the output of other tasks using the `context` attribute. For example:
```python Code
research_task = Task(
description="Research the latest developments in AI",
expected_output="A list of recent AI developments",
agent=researcher
)
analysis_task = Task(
description="Analyze the research findings and identify key trends",
expected_output="Analysis report of AI trends",
agent=analyst,
context=[research_task] # This task will wait for research_task to complete
)
```
## Task Guardrails
Task guardrails provide a way to validate and transform task outputs before they
are passed to the next task. This feature helps ensure data quality and provides
efeedback to agents when their output doesn't meet specific criteria.
### Using Task Guardrails
To add a guardrail to a task, provide a validation function through the `guardrail` parameter:
```python Code
from typing import Tuple, Union, Dict, Any
def validate_blog_content(result: str) -> Tuple[bool, Union[Dict[str, Any], str]]:
"""Validate blog content meets requirements."""
try:
# Check word count
word_count = len(result.split())
if word_count > 200:
return (False, {
"error": "Blog content exceeds 200 words",
"code": "WORD_COUNT_ERROR",
"context": {"word_count": word_count}
})
# Additional validation logic here
return (True, result.strip())
except Exception as e:
return (False, {
"error": "Unexpected error during validation",
"code": "SYSTEM_ERROR"
})
blog_task = Task(
description="Write a blog post about AI",
expected_output="A blog post under 200 words",
agent=blog_agent,
guardrail=validate_blog_content # Add the guardrail function
)
```
### Guardrail Function Requirements
1. **Function Signature**:
- Must accept exactly one parameter (the task output)
- Should return a tuple of `(bool, Any)`
- Type hints are recommended but optional
2. **Return Values**:
- Success: Return `(True, validated_result)`
- Failure: Return `(False, error_details)`
### Error Handling Best Practices
1. **Structured Error Responses**:
```python Code
def validate_with_context(result: str) -> Tuple[bool, Union[Dict[str, Any], str]]:
try:
# Main validation logic
validated_data = perform_validation(result)
return (True, validated_data)
except ValidationError as e:
return (False, {
"error": str(e),
"code": "VALIDATION_ERROR",
"context": {"input": result}
})
except Exception as e:
return (False, {
"error": "Unexpected error",
"code": "SYSTEM_ERROR"
})
```
2. **Error Categories**:
- Use specific error codes
- Include relevant context
- Provide actionable feedback
3. **Validation Chain**:
```python Code
from typing import Any, Dict, List, Tuple, Union
def complex_validation(result: str) -> Tuple[bool, Union[str, Dict[str, Any]]]:
"""Chain multiple validation steps."""
# Step 1: Basic validation
if not result:
return (False, {"error": "Empty result", "code": "EMPTY_INPUT"})
# Step 2: Content validation
try:
validated = validate_content(result)
if not validated:
return (False, {"error": "Invalid content", "code": "CONTENT_ERROR"})
# Step 3: Format validation
formatted = format_output(validated)
return (True, formatted)
except Exception as e:
return (False, {
"error": str(e),
"code": "VALIDATION_ERROR",
"context": {"step": "content_validation"}
})
```
### Handling Guardrail Results
When a guardrail returns `(False, error)`:
1. The error is sent back to the agent
2. The agent attempts to fix the issue
3. The process repeats until:
- The guardrail returns `(True, result)`
- Maximum retries are reached
Example with retry handling:
```python Code
from typing import Optional, Tuple, Union
def validate_json_output(result: str) -> Tuple[bool, Union[Dict[str, Any], str]]:
"""Validate and parse JSON output."""
try:
# Try to parse as JSON
data = json.loads(result)
return (True, data)
except json.JSONDecodeError as e:
return (False, {
"error": "Invalid JSON format",
"code": "JSON_ERROR",
"context": {"line": e.lineno, "column": e.colno}
})
task = Task(
description="Generate a JSON report",
expected_output="A valid JSON object",
agent=analyst,
guardrail=validate_json_output,
max_retries=3 # Limit retry attempts
)
```
## Getting Structured Consistent Outputs from Tasks
<Note>
It's also important to note that the output of the final task of a crew becomes the final output of the actual crew itself.
</Note>
### Using `output_pydantic`
The `output_pydantic` property allows you to define a Pydantic model that the task output should conform to. This ensures that the output is not only structured but also validated according to the Pydantic model.
Heres an example demonstrating how to use output_pydantic:
```python Code
import json
from crewai import Agent, Crew, Process, Task
from pydantic import BaseModel
class Blog(BaseModel):
title: str
content: str
blog_agent = Agent(
role="Blog Content Generator Agent",
goal="Generate a blog title and content",
backstory="""You are an expert content creator, skilled in crafting engaging and informative blog posts.""",
verbose=False,
allow_delegation=False,
llm="gpt-4o",
)
task1 = Task(
description="""Create a blog title and content on a given topic. Make sure the content is under 200 words.""",
expected_output="A compelling blog title and well-written content.",
agent=blog_agent,
output_pydantic=Blog,
)
# Instantiate your crew with a sequential process
crew = Crew(
agents=[blog_agent],
tasks=[task1],
verbose=True,
process=Process.sequential,
)
result = crew.kickoff()
# Option 1: Accessing Properties Using Dictionary-Style Indexing
print("Accessing Properties - Option 1")
title = result["title"]
content = result["content"]
print("Title:", title)
print("Content:", content)
# Option 2: Accessing Properties Directly from the Pydantic Model
print("Accessing Properties - Option 2")
title = result.pydantic.title
content = result.pydantic.content
print("Title:", title)
print("Content:", content)
# Option 3: Accessing Properties Using the to_dict() Method
print("Accessing Properties - Option 3")
output_dict = result.to_dict()
title = output_dict["title"]
content = output_dict["content"]
print("Title:", title)
print("Content:", content)
# Option 4: Printing the Entire Blog Object
print("Accessing Properties - Option 5")
print("Blog:", result)
```
In this example:
* A Pydantic model Blog is defined with title and content fields.
* The task task1 uses the output_pydantic property to specify that its output should conform to the Blog model.
* After executing the crew, you can access the structured output in multiple ways as shown.
#### Explanation of Accessing the Output
1. Dictionary-Style Indexing: You can directly access the fields using result["field_name"]. This works because the CrewOutput class implements the __getitem__ method.
2. Directly from Pydantic Model: Access the attributes directly from the result.pydantic object.
3. Using to_dict() Method: Convert the output to a dictionary and access the fields.
4. Printing the Entire Object: Simply print the result object to see the structured output.
### Using `output_json`
The `output_json` property allows you to define the expected output in JSON format. This ensures that the task's output is a valid JSON structure that can be easily parsed and used in your application.
Heres an example demonstrating how to use `output_json`:
```python Code
import json
from crewai import Agent, Crew, Process, Task
from pydantic import BaseModel
# Define the Pydantic model for the blog
class Blog(BaseModel):
title: str
content: str
# Define the agent
blog_agent = Agent(
role="Blog Content Generator Agent",
goal="Generate a blog title and content",
backstory="""You are an expert content creator, skilled in crafting engaging and informative blog posts.""",
verbose=False,
allow_delegation=False,
llm="gpt-4o",
)
# Define the task with output_json set to the Blog model
task1 = Task(
description="""Create a blog title and content on a given topic. Make sure the content is under 200 words.""",
expected_output="A JSON object with 'title' and 'content' fields.",
agent=blog_agent,
output_json=Blog,
)
# Instantiate the crew with a sequential process
crew = Crew(
agents=[blog_agent],
tasks=[task1],
verbose=True,
process=Process.sequential,
)
# Kickoff the crew to execute the task
result = crew.kickoff()
# Option 1: Accessing Properties Using Dictionary-Style Indexing
print("Accessing Properties - Option 1")
title = result["title"]
content = result["content"]
print("Title:", title)
print("Content:", content)
# Option 2: Printing the Entire Blog Object
print("Accessing Properties - Option 2")
print("Blog:", result)
```
In this example:
* A Pydantic model Blog is defined with title and content fields, which is used to specify the structure of the JSON output.
* The task task1 uses the output_json property to indicate that it expects a JSON output conforming to the Blog model.
* After executing the crew, you can access the structured JSON output in two ways as shown.
#### Explanation of Accessing the Output
1. Accessing Properties Using Dictionary-Style Indexing: You can access the fields directly using result["field_name"]. This is possible because the CrewOutput class implements the __getitem__ method, allowing you to treat the output like a dictionary. In this option, we're retrieving the title and content from the result.
2. Printing the Entire Blog Object: By printing result, you get the string representation of the CrewOutput object. Since the __str__ method is implemented to return the JSON output, this will display the entire output as a formatted string representing the Blog object.
---
By using output_pydantic or output_json, you ensure that your tasks produce outputs in a consistent and structured format, making it easier to process and utilize the data within your application or across multiple tasks.
## Integrating Tools with Tasks
Leverage tools from the [CrewAI Toolkit](https://github.com/joaomdmoura/crewai-tools) and [LangChain Tools](https://python.langchain.com/docs/integrations/tools) for enhanced task performance and agent interaction.
@@ -619,16 +167,16 @@ This is useful when you have a task that depends on the output of another task t
# ...
research_ai_task = Task(
description="Research the latest developments in AI",
expected_output="A list of recent AI developments",
description='Find and summarize the latest AI news',
expected_output='A bullet list summary of the top 5 most important AI news',
async_execution=True,
agent=research_agent,
tools=[search_tool]
)
research_ops_task = Task(
description="Research the latest developments in AI Ops",
expected_output="A list of recent AI Ops developments",
description='Find and summarize the latest AI Ops news',
expected_output='A bullet list summary of the top 5 most important AI Ops news',
async_execution=True,
agent=research_agent,
tools=[search_tool]
@@ -636,7 +184,7 @@ research_ops_task = Task(
write_blog_task = Task(
description="Write a full blog post about the importance of AI and its latest news",
expected_output="Full blog post that is 4 paragraphs long",
expected_output='Full blog post that is 4 paragraphs long',
agent=writer_agent,
context=[research_ai_task, research_ops_task]
)
@@ -748,114 +296,6 @@ While creating and executing tasks, certain validation mechanisms are in place t
These validations help in maintaining the consistency and reliability of task executions within the crewAI framework.
## Task Guardrails
Task guardrails provide a powerful way to validate, transform, or filter task outputs before they are passed to the next task. Guardrails are optional functions that execute before the next task starts, allowing you to ensure that task outputs meet specific requirements or formats.
### Basic Usage
```python Code
from typing import Tuple, Union
from crewai import Task
def validate_json_output(result: str) -> Tuple[bool, Union[dict, str]]:
"""Validate that the output is valid JSON."""
try:
json_data = json.loads(result)
return (True, json_data)
except json.JSONDecodeError:
return (False, "Output must be valid JSON")
task = Task(
description="Generate JSON data",
expected_output="Valid JSON object",
guardrail=validate_json_output
)
```
### How Guardrails Work
1. **Optional Attribute**: Guardrails are an optional attribute at the task level, allowing you to add validation only where needed.
2. **Execution Timing**: The guardrail function is executed before the next task starts, ensuring valid data flow between tasks.
3. **Return Format**: Guardrails must return a tuple of `(success, data)`:
- If `success` is `True`, `data` is the validated/transformed result
- If `success` is `False`, `data` is the error message
4. **Result Routing**:
- On success (`True`), the result is automatically passed to the next task
- On failure (`False`), the error is sent back to the agent to generate a new answer
### Common Use Cases
#### Data Format Validation
```python Code
def validate_email_format(result: str) -> Tuple[bool, Union[str, str]]:
"""Ensure the output contains a valid email address."""
import re
email_pattern = r'^[\w\.-]+@[\w\.-]+\.\w+$'
if re.match(email_pattern, result.strip()):
return (True, result.strip())
return (False, "Output must be a valid email address")
```
#### Content Filtering
```python Code
def filter_sensitive_info(result: str) -> Tuple[bool, Union[str, str]]:
"""Remove or validate sensitive information."""
sensitive_patterns = ['SSN:', 'password:', 'secret:']
for pattern in sensitive_patterns:
if pattern.lower() in result.lower():
return (False, f"Output contains sensitive information ({pattern})")
return (True, result)
```
#### Data Transformation
```python Code
def normalize_phone_number(result: str) -> Tuple[bool, Union[str, str]]:
"""Ensure phone numbers are in a consistent format."""
import re
digits = re.sub(r'\D', '', result)
if len(digits) == 10:
formatted = f"({digits[:3]}) {digits[3:6]}-{digits[6:]}"
return (True, formatted)
return (False, "Output must be a 10-digit phone number")
```
### Advanced Features
#### Chaining Multiple Validations
```python Code
def chain_validations(*validators):
"""Chain multiple validators together."""
def combined_validator(result):
for validator in validators:
success, data = validator(result)
if not success:
return (False, data)
result = data
return (True, result)
return combined_validator
# Usage
task = Task(
description="Get user contact info",
expected_output="Email and phone",
guardrail=chain_validations(
validate_email_format,
filter_sensitive_info
)
)
```
#### Custom Retry Logic
```python Code
task = Task(
description="Generate data",
expected_output="Valid data",
guardrail=validate_data,
max_retries=5 # Override default retry limit
)
```
## Creating Directories when Saving Files
You can now specify if a task should create directories when saving its output to a file. This is particularly useful for organizing outputs and ensuring that file paths are correctly structured.
@@ -877,7 +317,7 @@ save_output_task = Task(
## Conclusion
Tasks are the driving force behind the actions of agents in CrewAI.
By properly defining tasks and their outcomes, you set the stage for your AI agents to work effectively, either independently or as a collaborative unit.
Equipping tasks with appropriate tools, understanding the execution process, and following robust validation practices are crucial for maximizing CrewAI's potential,
ensuring agents are effectively prepared for their assignments and that tasks are executed as intended.
Tasks are the driving force behind the actions of agents in CrewAI.
By properly defining tasks and their outcomes, you set the stage for your AI agents to work effectively, either independently or as a collaborative unit.
Equipping tasks with appropriate tools, understanding the execution process, and following robust validation practices are crucial for maximizing CrewAI's potential,
ensuring agents are effectively prepared for their assignments and that tasks are executed as intended.

View File

@@ -5,14 +5,13 @@ icon: screwdriver-wrench
---
## Introduction
CrewAI tools empower agents with capabilities ranging from web searching and data analysis to collaboration and delegating tasks among coworkers.
CrewAI tools empower agents with capabilities ranging from web searching and data analysis to collaboration and delegating tasks among coworkers.
This documentation outlines how to create, integrate, and leverage these tools within the CrewAI framework, including a new focus on collaboration tools.
## What is a Tool?
A tool in CrewAI is a skill or function that agents can utilize to perform various actions.
This includes tools from the [CrewAI Toolkit](https://github.com/joaomdmoura/crewai-tools) and [LangChain Tools](https://python.langchain.com/docs/integrations/tools),
A tool in CrewAI is a skill or function that agents can utilize to perform various actions.
This includes tools from the [CrewAI Toolkit](https://github.com/joaomdmoura/crewai-tools) and [LangChain Tools](https://python.langchain.com/docs/integrations/tools),
enabling everything from simple searches to complex interactions and effective teamwork among agents.
## Key Characteristics of Tools
@@ -104,53 +103,57 @@ crew.kickoff()
Here is a list of the available tools and their descriptions:
| Tool | Description |
| :------------------------------- | :--------------------------------------------------------------------------------------------- |
| **BrowserbaseLoadTool** | A tool for interacting with and extracting data from web browsers. |
| **CodeDocsSearchTool** | A RAG tool optimized for searching through code documentation and related technical documents. |
| **CodeInterpreterTool** | A tool for interpreting python code. |
| **ComposioTool** | Enables use of Composio tools. |
| **CSVSearchTool** | A RAG tool designed for searching within CSV files, tailored to handle structured data. |
| **DALL-E Tool** | A tool for generating images using the DALL-E API. |
| **DirectorySearchTool** | A RAG tool for searching within directories, useful for navigating through file systems. |
| **DOCXSearchTool** | A RAG tool aimed at searching within DOCX documents, ideal for processing Word files. |
| **DirectoryReadTool** | Facilitates reading and processing of directory structures and their contents. |
| **EXASearchTool** | A tool designed for performing exhaustive searches across various data sources. |
| **FileReadTool** | Enables reading and extracting data from files, supporting various file formats. |
| **FirecrawlSearchTool** | A tool to search webpages using Firecrawl and return the results. |
| **FirecrawlCrawlWebsiteTool** | A tool for crawling webpages using Firecrawl. |
| **FirecrawlScrapeWebsiteTool** | A tool for scraping webpages URL using Firecrawl and returning its contents. |
| **GithubSearchTool** | A RAG tool for searching within GitHub repositories, useful for code and documentation search. |
| **SerperDevTool** | A specialized tool for development purposes, with specific functionalities under development. |
| **TXTSearchTool** | A RAG tool focused on searching within text (.txt) files, suitable for unstructured data. |
| **JSONSearchTool** | A RAG tool designed for searching within JSON files, catering to structured data handling. |
| **LlamaIndexTool** | Enables the use of LlamaIndex tools. |
| **MDXSearchTool** | A RAG tool tailored for searching within Markdown (MDX) files, useful for documentation. |
| **PDFSearchTool** | A RAG tool aimed at searching within PDF documents, ideal for processing scanned documents. |
| **PGSearchTool** | A RAG tool optimized for searching within PostgreSQL databases, suitable for database queries. |
| **Vision Tool** | A tool for generating images using the DALL-E API. |
| **RagTool** | A general-purpose RAG tool capable of handling various data sources and types. |
| **ScrapeElementFromWebsiteTool** | Enables scraping specific elements from websites, useful for targeted data extraction. |
| **ScrapeWebsiteTool** | Facilitates scraping entire websites, ideal for comprehensive data collection. |
| **WebsiteSearchTool** | A RAG tool for searching website content, optimized for web data extraction. |
| **XMLSearchTool** | A RAG tool designed for searching within XML files, suitable for structured data formats. |
| **YoutubeChannelSearchTool** | A RAG tool for searching within YouTube channels, useful for video content analysis. |
| **YoutubeVideoSearchTool** | A RAG tool aimed at searching within YouTube videos, ideal for video data extraction. |
| Tool | Description |
| :-------------------------- | :-------------------------------------------------------------------------------------------- |
| **BrowserbaseLoadTool** | A tool for interacting with and extracting data from web browsers. |
| **CodeDocsSearchTool** | A RAG tool optimized for searching through code documentation and related technical documents. |
| **CodeInterpreterTool** | A tool for interpreting python code. |
| **ComposioTool** | Enables use of Composio tools. |
| **CSVSearchTool** | A RAG tool designed for searching within CSV files, tailored to handle structured data. |
| **DALL-E Tool** | A tool for generating images using the DALL-E API. |
| **DirectorySearchTool** | A RAG tool for searching within directories, useful for navigating through file systems. |
| **DOCXSearchTool** | A RAG tool aimed at searching within DOCX documents, ideal for processing Word files. |
| **DirectoryReadTool** | Facilitates reading and processing of directory structures and their contents. |
| **EXASearchTool** | A tool designed for performing exhaustive searches across various data sources. |
| **FileReadTool** | Enables reading and extracting data from files, supporting various file formats. |
| **FirecrawlSearchTool** | A tool to search webpages using Firecrawl and return the results. |
| **FirecrawlCrawlWebsiteTool** | A tool for crawling webpages using Firecrawl. |
| **FirecrawlScrapeWebsiteTool** | A tool for scraping webpages URL using Firecrawl and returning its contents. |
| **GithubSearchTool** | A RAG tool for searching within GitHub repositories, useful for code and documentation search.|
| **SerperDevTool** | A specialized tool for development purposes, with specific functionalities under development. |
| **TXTSearchTool** | A RAG tool focused on searching within text (.txt) files, suitable for unstructured data. |
| **JSONSearchTool** | A RAG tool designed for searching within JSON files, catering to structured data handling. |
| **LlamaIndexTool** | Enables the use of LlamaIndex tools. |
| **MDXSearchTool** | A RAG tool tailored for searching within Markdown (MDX) files, useful for documentation. |
| **PDFSearchTool** | A RAG tool aimed at searching within PDF documents, ideal for processing scanned documents. |
| **PGSearchTool** | A RAG tool optimized for searching within PostgreSQL databases, suitable for database queries. |
| **Vision Tool** | A tool for generating images using the DALL-E API. |
| **RagTool** | A general-purpose RAG tool capable of handling various data sources and types. |
| **ScrapeElementFromWebsiteTool** | Enables scraping specific elements from websites, useful for targeted data extraction. |
| **ScrapeWebsiteTool** | Facilitates scraping entire websites, ideal for comprehensive data collection. |
| **WebsiteSearchTool** | A RAG tool for searching website content, optimized for web data extraction. |
| **XMLSearchTool** | A RAG tool designed for searching within XML files, suitable for structured data formats. |
| **YoutubeChannelSearchTool**| A RAG tool for searching within YouTube channels, useful for video content analysis. |
| **YoutubeVideoSearchTool** | A RAG tool aimed at searching within YouTube videos, ideal for video data extraction. |
## Creating your own Tools
<Tip>
Developers can craft `custom tools` tailored for their agents needs or
utilize pre-built options.
Developers can craft `custom tools` tailored for their agents needs or utilize pre-built options.
</Tip>
There are two main ways for one to create a CrewAI tool:
To create your own CrewAI tools you will need to install our extra tools package:
```bash
pip install 'crewai[tools]'
```
Once you do that there are two main ways for one to create a CrewAI tool:
### Subclassing `BaseTool`
```python Code
from crewai.tools import BaseTool
from crewai_tools import BaseTool
class MyCustomTool(BaseTool):
name: str = "Name of my tool"
@@ -164,7 +167,7 @@ class MyCustomTool(BaseTool):
### Utilizing the `tool` Decorator
```python Code
from crewai.tools import tool
from crewai_tools import tool
@tool("Name of my tool")
def my_tool(question: str) -> str:
"""Clear description for what this tool is useful for, your agent will need this information to use it."""
@@ -172,58 +175,14 @@ def my_tool(question: str) -> str:
return "Result from your custom tool"
```
### Structured Tools
The `StructuredTool` class wraps functions as tools, providing flexibility and validation while reducing boilerplate. It supports custom schemas and dynamic logic for seamless integration of complex functionalities.
#### Example:
Using `StructuredTool.from_function`, you can wrap a function that interacts with an external API or system, providing a structured interface. This enables robust validation and consistent execution, making it easier to integrate complex functionalities into your applications as demonstrated in the following example:
```python
from crewai.tools.structured_tool import CrewStructuredTool
from pydantic import BaseModel
# Define the schema for the tool's input using Pydantic
class APICallInput(BaseModel):
endpoint: str
parameters: dict
# Wrapper function to execute the API call
def tool_wrapper(*args, **kwargs):
# Here, you would typically call the API using the parameters
# For demonstration, we'll return a placeholder string
return f"Call the API at {kwargs['endpoint']} with parameters {kwargs['parameters']}"
# Create and return the structured tool
def create_structured_tool():
return CrewStructuredTool.from_function(
name='Wrapper API',
description="A tool to wrap API calls with structured input.",
args_schema=APICallInput,
func=tool_wrapper,
)
# Example usage
structured_tool = create_structured_tool()
# Execute the tool with structured input
result = structured_tool._run(**{
"endpoint": "https://example.com/api",
"parameters": {"key1": "value1", "key2": "value2"}
})
print(result) # Output: Call the API at https://example.com/api with parameters {'key1': 'value1', 'key2': 'value2'}
```
### Custom Caching Mechanism
<Tip>
Tools can optionally implement a `cache_function` to fine-tune caching
behavior. This function determines when to cache results based on specific
conditions, offering granular control over caching logic.
Tools can optionally implement a `cache_function` to fine-tune caching behavior. This function determines when to cache results based on specific conditions, offering granular control over caching logic.
</Tip>
```python Code
from crewai.tools import tool
from crewai_tools import tool
@tool
def multiplication_tool(first_number: int, second_number: int) -> str:
@@ -249,6 +208,6 @@ writer1 = Agent(
## Conclusion
Tools are pivotal in extending the capabilities of CrewAI agents, enabling them to undertake a broad spectrum of tasks and collaborate effectively.
When building solutions with CrewAI, leverage both custom and existing tools to empower your agents and enhance the AI ecosystem. Consider utilizing error handling,
caching mechanisms, and the flexibility of tool arguments to optimize your agents' performance and capabilities.
Tools are pivotal in extending the capabilities of CrewAI agents, enabling them to undertake a broad spectrum of tasks and collaborate effectively.
When building solutions with CrewAI, leverage both custom and existing tools to empower your agents and enhance the AI ecosystem. Consider utilizing error handling,
caching mechanisms, and the flexibility of tool arguments to optimize your agents' performance and capabilities.

View File

@@ -1,211 +0,0 @@
# Portkey Integration with CrewAI
<img src="https://raw.githubusercontent.com/siddharthsambharia-portkey/Portkey-Product-Images/main/Portkey-CrewAI.png" alt="Portkey CrewAI Header Image" width="70%" />
[Portkey](https://portkey.ai/?utm_source=crewai&utm_medium=crewai&utm_campaign=crewai) is a 2-line upgrade to make your CrewAI agents reliable, cost-efficient, and fast.
Portkey adds 4 core production capabilities to any CrewAI agent:
1. Routing to **200+ LLMs**
2. Making each LLM call more robust
3. Full-stack tracing & cost, performance analytics
4. Real-time guardrails to enforce behavior
## Getting Started
1. **Install Required Packages:**
```bash
pip install -qU crewai portkey-ai
```
2. **Configure the LLM Client:**
To build CrewAI Agents with Portkey, you'll need two keys:
- **Portkey API Key**: Sign up on the [Portkey app](https://app.portkey.ai/?utm_source=crewai&utm_medium=crewai&utm_campaign=crewai) and copy your API key
- **Virtual Key**: Virtual Keys securely manage your LLM API keys in one place. Store your LLM provider API keys securely in Portkey's vault
```python
from crewai import LLM
from portkey_ai import createHeaders, PORTKEY_GATEWAY_URL
gpt_llm = LLM(
model="gpt-4",
base_url=PORTKEY_GATEWAY_URL,
api_key="dummy", # We are using Virtual key
extra_headers=createHeaders(
api_key="YOUR_PORTKEY_API_KEY",
virtual_key="YOUR_VIRTUAL_KEY", # Enter your Virtual key from Portkey
)
)
```
3. **Create and Run Your First Agent:**
```python
from crewai import Agent, Task, Crew
# Define your agents with roles and goals
coder = Agent(
role='Software developer',
goal='Write clear, concise code on demand',
backstory='An expert coder with a keen eye for software trends.',
llm=gpt_llm
)
# Create tasks for your agents
task1 = Task(
description="Define the HTML for making a simple website with heading- Hello World! Portkey is working!",
expected_output="A clear and concise HTML code",
agent=coder
)
# Instantiate your crew
crew = Crew(
agents=[coder],
tasks=[task1],
)
result = crew.kickoff()
print(result)
```
## Key Features
| Feature | Description |
|---------|-------------|
| 🌐 Multi-LLM Support | Access OpenAI, Anthropic, Gemini, Azure, and 250+ providers through a unified interface |
| 🛡️ Production Reliability | Implement retries, timeouts, load balancing, and fallbacks |
| 📊 Advanced Observability | Track 40+ metrics including costs, tokens, latency, and custom metadata |
| 🔍 Comprehensive Logging | Debug with detailed execution traces and function call logs |
| 🚧 Security Controls | Set budget limits and implement role-based access control |
| 🔄 Performance Analytics | Capture and analyze feedback for continuous improvement |
| 💾 Intelligent Caching | Reduce costs and latency with semantic or simple caching |
## Production Features with Portkey Configs
All features mentioned below are through Portkey's Config system. Portkey's Config system allows you to define routing strategies using simple JSON objects in your LLM API calls. You can create and manage Configs directly in your code or through the Portkey Dashboard. Each Config has a unique ID for easy reference.
<Frame>
<img src="https://raw.githubusercontent.com/Portkey-AI/docs-core/refs/heads/main/images/libraries/libraries-3.avif"/>
</Frame>
### 1. Use 250+ LLMs
Access various LLMs like Anthropic, Gemini, Mistral, Azure OpenAI, and more with minimal code changes. Switch between providers or use them together seamlessly. [Learn more about Universal API](https://portkey.ai/docs/product/ai-gateway/universal-api)
Easily switch between different LLM providers:
```python
# Anthropic Configuration
anthropic_llm = LLM(
model="claude-3-5-sonnet-latest",
base_url=PORTKEY_GATEWAY_URL,
api_key="dummy",
extra_headers=createHeaders(
api_key="YOUR_PORTKEY_API_KEY",
virtual_key="YOUR_ANTHROPIC_VIRTUAL_KEY", #You don't need provider when using Virtual keys
trace_id="anthropic_agent"
)
)
# Azure OpenAI Configuration
azure_llm = LLM(
model="gpt-4",
base_url=PORTKEY_GATEWAY_URL,
api_key="dummy",
extra_headers=createHeaders(
api_key="YOUR_PORTKEY_API_KEY",
virtual_key="YOUR_AZURE_VIRTUAL_KEY", #You don't need provider when using Virtual keys
trace_id="azure_agent"
)
)
```
### 2. Caching
Improve response times and reduce costs with two powerful caching modes:
- **Simple Cache**: Perfect for exact matches
- **Semantic Cache**: Matches responses for requests that are semantically similar
[Learn more about Caching](https://portkey.ai/docs/product/ai-gateway/cache-simple-and-semantic)
```py
config = {
"cache": {
"mode": "semantic", # or "simple" for exact matching
}
}
```
### 3. Production Reliability
Portkey provides comprehensive reliability features:
- **Automatic Retries**: Handle temporary failures gracefully
- **Request Timeouts**: Prevent hanging operations
- **Conditional Routing**: Route requests based on specific conditions
- **Fallbacks**: Set up automatic provider failovers
- **Load Balancing**: Distribute requests efficiently
[Learn more about Reliability Features](https://portkey.ai/docs/product/ai-gateway/)
### 4. Metrics
Agent runs are complex. Portkey automatically logs **40+ comprehensive metrics** for your AI agents, including cost, tokens used, latency, etc. Whether you need a broad overview or granular insights into your agent runs, Portkey's customizable filters provide the metrics you need.
- Cost per agent interaction
- Response times and latency
- Token usage and efficiency
- Success/failure rates
- Cache hit rates
<img src="https://github.com/siddharthsambharia-portkey/Portkey-Product-Images/blob/main/Portkey-Dashboard.png?raw=true" width="70%" alt="Portkey Dashboard" />
### 5. Detailed Logging
Logs are essential for understanding agent behavior, diagnosing issues, and improving performance. They provide a detailed record of agent activities and tool use, which is crucial for debugging and optimizing processes.
Access a dedicated section to view records of agent executions, including parameters, outcomes, function calls, and errors. Filter logs based on multiple parameters such as trace ID, model, tokens used, and metadata.
<details>
<summary><b>Traces</b></summary>
<img src="https://raw.githubusercontent.com/siddharthsambharia-portkey/Portkey-Product-Images/main/Portkey-Traces.png" alt="Portkey Traces" width="70%" />
</details>
<details>
<summary><b>Logs</b></summary>
<img src="https://raw.githubusercontent.com/siddharthsambharia-portkey/Portkey-Product-Images/main/Portkey-Logs.png" alt="Portkey Logs" width="70%" />
</details>
### 6. Enterprise Security Features
- Set budget limit and rate limts per Virtual Key (disposable API keys)
- Implement role-based access control
- Track system changes with audit logs
- Configure data retention policies
For detailed information on creating and managing Configs, visit the [Portkey documentation](https://docs.portkey.ai/product/ai-gateway/configs).
## Resources
- [📘 Portkey Documentation](https://docs.portkey.ai)
- [📊 Portkey Dashboard](https://app.portkey.ai/?utm_source=crewai&utm_medium=crewai&utm_campaign=crewai)
- [🐦 Twitter](https://twitter.com/portkeyai)
- [💬 Discord Community](https://discord.gg/DD7vgKK299)

View File

@@ -57,7 +57,7 @@ This feature is useful for debugging and understanding how agents interact with
<Step title="Install AgentOps">
Install AgentOps with:
```bash
pip install 'crewai[agentops]'
pip install crewai[agentops]
```
or
```bash

View File

@@ -1,59 +0,0 @@
---
title: Before and After Kickoff Hooks
description: Learn how to use before and after kickoff hooks in CrewAI
---
CrewAI provides hooks that allow you to execute code before and after a crew's kickoff. These hooks are useful for preprocessing inputs or post-processing results.
## Before Kickoff Hook
The before kickoff hook is executed before the crew starts its tasks. It receives the input dictionary and can modify it before passing it to the crew. You can use this hook to set up your environment, load necessary data, or preprocess your inputs. This is useful in scenarios where the input data might need enrichment or validation before being processed by the crew.
Here's an example of defining a before kickoff function in your `crew.py`:
```python
from crewai import CrewBase, before_kickoff
@CrewBase
class MyCrew:
@before_kickoff
def prepare_data(self, inputs):
# Preprocess or modify inputs
inputs['processed'] = True
return inputs
#...
```
In this example, the prepare_data function modifies the inputs by adding a new key-value pair indicating that the inputs have been processed.
## After Kickoff Hook
The after kickoff hook is executed after the crew has completed its tasks. It receives the result object, which contains the outputs of the crew's execution. This hook is ideal for post-processing results, such as logging, data transformation, or further analysis.
Here's how you can define an after kickoff function in your `crew.py`:
```python
from crewai import CrewBase, after_kickoff
@CrewBase
class MyCrew:
@after_kickoff
def log_results(self, result):
# Log or modify the results
print("Crew execution completed with result:", result)
return result
# ...
```
In the `log_results` function, the results of the crew execution are simply printed out. You can extend this to perform more complex operations such as sending notifications or integrating with other services.
## Utilizing Both Hooks
Both hooks can be used together to provide a comprehensive setup and teardown process for your crew's execution. They are particularly useful in maintaining clean code architecture by separating concerns and enhancing the modularity of your CrewAI implementations.
## Conclusion
Before and after kickoff hooks in CrewAI offer powerful ways to interact with the lifecycle of a crew's execution. By understanding and utilizing these hooks, you can greatly enhance the robustness and flexibility of your AI agents.

View File

@@ -6,27 +6,28 @@ icon: hammer
## Creating and Utilizing Tools in CrewAI
This guide provides detailed instructions on creating custom tools for the CrewAI framework and how to efficiently manage and utilize these tools,
incorporating the latest functionalities such as tool delegation, error handling, and dynamic tool calling. It also highlights the importance of collaboration tools,
This guide provides detailed instructions on creating custom tools for the CrewAI framework and how to efficiently manage and utilize these tools,
incorporating the latest functionalities such as tool delegation, error handling, and dynamic tool calling. It also highlights the importance of collaboration tools,
enabling agents to perform a wide range of actions.
### Prerequisites
Before creating your own tools, ensure you have the crewAI extra tools package installed:
```bash
pip install 'crewai[tools]'
```
### Subclassing `BaseTool`
To create a personalized tool, inherit from `BaseTool` and define the necessary attributes, including the `args_schema` for input validation, and the `_run` method.
To create a personalized tool, inherit from `BaseTool` and define the necessary attributes and the `_run` method.
```python Code
from typing import Type
from crewai.tools import BaseTool
from pydantic import BaseModel, Field
class MyToolInput(BaseModel):
"""Input schema for MyCustomTool."""
argument: str = Field(..., description="Description of the argument.")
from crewai_tools import BaseTool
class MyCustomTool(BaseTool):
name: str = "Name of my tool"
description: str = "What this tool does. It's vital for effective utilization."
args_schema: Type[BaseModel] = MyToolInput
def _run(self, argument: str) -> str:
# Your tool's logic here
@@ -39,7 +40,7 @@ Alternatively, you can use the tool decorator `@tool`. This approach allows you
offering a concise and efficient way to create specialized tools tailored to your needs.
```python Code
from crewai.tools import tool
from crewai_tools import tool
@tool("Tool Name")
def my_simple_tool(question: str) -> str:
@@ -65,5 +66,5 @@ def my_cache_strategy(arguments: dict, result: str) -> bool:
cached_tool.cache_function = my_cache_strategy
```
By adhering to these guidelines and incorporating new functionalities and collaboration tools into your tool creation and management processes,
By adhering to these guidelines and incorporating new functionalities and collaboration tools into your tool creation and management processes,
you can leverage the full capabilities of the CrewAI framework, enhancing both the development experience and the efficiency of your AI agents.

View File

@@ -32,7 +32,6 @@ LiteLLM supports a wide range of providers, including but not limited to:
- Cloudflare Workers AI
- DeepInfra
- Groq
- [NVIDIA NIMs](https://docs.api.nvidia.com/nim/reference/models-1)
- And many more!
For a complete and up-to-date list of supported providers, please refer to the [LiteLLM Providers documentation](https://docs.litellm.ai/docs/providers).
@@ -126,10 +125,10 @@ You can connect to OpenAI-compatible LLMs using either environment variables or
</Tab>
<Tab title="Using LLM Class Attributes">
<CodeGroup>
```python Code
llm = LLM(
model="custom-model-name",
api_key="your-api-key",
```python Code
llm = LLM(
model="custom-model-name",
api_key="your-api-key",
base_url="https://api.your-provider.com/v1"
)
agent = Agent(llm=llm, ...)
@@ -180,4 +179,4 @@ This is particularly useful when working with OpenAI-compatible APIs or when you
## Conclusion
By leveraging LiteLLM, CrewAI offers seamless integration with a vast array of LLMs. This flexibility allows you to choose the most suitable model for your specific needs, whether you prioritize performance, cost-efficiency, or local deployment. Remember to consult the [LiteLLM documentation](https://docs.litellm.ai/docs/) for the most up-to-date information on supported models and configuration options.
By leveraging LiteLLM, CrewAI offers seamless integration with a vast array of LLMs. This flexibility allows you to choose the most suitable model for your specific needs, whether you prioritize performance, cost-efficiency, or local deployment. Remember to consult the [LiteLLM documentation](https://docs.litellm.ai/docs/) for the most up-to-date information on supported models and configuration options.

View File

@@ -1,138 +0,0 @@
---
title: Using Multimodal Agents
description: Learn how to enable and use multimodal capabilities in your agents for processing images and other non-text content within the CrewAI framework.
icon: image
---
# Using Multimodal Agents
CrewAI supports multimodal agents that can process both text and non-text content like images. This guide will show you how to enable and use multimodal capabilities in your agents.
## Enabling Multimodal Capabilities
To create a multimodal agent, simply set the `multimodal` parameter to `True` when initializing your agent:
```python
from crewai import Agent
agent = Agent(
role="Image Analyst",
goal="Analyze and extract insights from images",
backstory="An expert in visual content interpretation with years of experience in image analysis",
multimodal=True # This enables multimodal capabilities
)
```
When you set `multimodal=True`, the agent is automatically configured with the necessary tools for handling non-text content, including the `AddImageTool`.
## Working with Images
The multimodal agent comes pre-configured with the `AddImageTool`, which allows it to process images. You don't need to manually add this tool - it's automatically included when you enable multimodal capabilities.
Here's a complete example showing how to use a multimodal agent to analyze an image:
```python
from crewai import Agent, Task, Crew
# Create a multimodal agent
image_analyst = Agent(
role="Product Analyst",
goal="Analyze product images and provide detailed descriptions",
backstory="Expert in visual product analysis with deep knowledge of design and features",
multimodal=True
)
# Create a task for image analysis
task = Task(
description="Analyze the product image at https://example.com/product.jpg and provide a detailed description",
agent=image_analyst
)
# Create and run the crew
crew = Crew(
agents=[image_analyst],
tasks=[task]
)
result = crew.kickoff()
```
### Advanced Usage with Context
You can provide additional context or specific questions about the image when creating tasks for multimodal agents. The task description can include specific aspects you want the agent to focus on:
```python
from crewai import Agent, Task, Crew
# Create a multimodal agent for detailed analysis
expert_analyst = Agent(
role="Visual Quality Inspector",
goal="Perform detailed quality analysis of product images",
backstory="Senior quality control expert with expertise in visual inspection",
multimodal=True # AddImageTool is automatically included
)
# Create a task with specific analysis requirements
inspection_task = Task(
description="""
Analyze the product image at https://example.com/product.jpg with focus on:
1. Quality of materials
2. Manufacturing defects
3. Compliance with standards
Provide a detailed report highlighting any issues found.
""",
agent=expert_analyst
)
# Create and run the crew
crew = Crew(
agents=[expert_analyst],
tasks=[inspection_task]
)
result = crew.kickoff()
```
### Tool Details
When working with multimodal agents, the `AddImageTool` is automatically configured with the following schema:
```python
class AddImageToolSchema:
image_url: str # Required: The URL or path of the image to process
action: Optional[str] = None # Optional: Additional context or specific questions about the image
```
The multimodal agent will automatically handle the image processing through its built-in tools, allowing it to:
- Access images via URLs or local file paths
- Process image content with optional context or specific questions
- Provide analysis and insights based on the visual information and task requirements
## Best Practices
When working with multimodal agents, keep these best practices in mind:
1. **Image Access**
- Ensure your images are accessible via URLs that the agent can reach
- For local images, consider hosting them temporarily or using absolute file paths
- Verify that image URLs are valid and accessible before running tasks
2. **Task Description**
- Be specific about what aspects of the image you want the agent to analyze
- Include clear questions or requirements in the task description
- Consider using the optional `action` parameter for focused analysis
3. **Resource Management**
- Image processing may require more computational resources than text-only tasks
- Some language models may require base64 encoding for image data
- Consider batch processing for multiple images to optimize performance
4. **Environment Setup**
- Verify that your environment has the necessary dependencies for image processing
- Ensure your language model supports multimodal capabilities
- Test with small images first to validate your setup
5. **Error Handling**
- Implement proper error handling for image loading failures
- Have fallback strategies for when image processing fails
- Monitor and log image processing operations for debugging

View File

@@ -1,181 +0,0 @@
---
title: Agent Monitoring with OpenLIT
description: Quickly start monitoring your Agents in just a single line of code with OpenTelemetry.
icon: magnifying-glass-chart
---
# OpenLIT Overview
[OpenLIT](https://github.com/openlit/openlit?src=crewai-docs) is an open-source tool that makes it simple to monitor the performance of AI agents, LLMs, VectorDBs, and GPUs with just **one** line of code.
It provides OpenTelemetry-native tracing and metrics to track important parameters like cost, latency, interactions and task sequences.
This setup enables you to track hyperparameters and monitor for performance issues, helping you find ways to enhance and fine-tune your agents over time.
<Frame caption="OpenLIT Dashboard">
<img src="/images/openlit1.png" alt="Overview Agent usage including cost and tokens" />
<img src="/images/openlit2.png" alt="Overview of agent otel traces and metrics" />
<img src="/images/openlit3.png" alt="Overview of agent traces in details" />
</Frame>
### Features
- **Analytics Dashboard**: Monitor your Agents health and performance with detailed dashboards that track metrics, costs, and user interactions.
- **OpenTelemetry-native Observability SDK**: Vendor-neutral SDKs to send traces and metrics to your existing observability tools like Grafana, DataDog and more.
- **Cost Tracking for Custom and Fine-Tuned Models**: Tailor cost estimations for specific models using custom pricing files for precise budgeting.
- **Exceptions Monitoring Dashboard**: Quickly spot and resolve issues by tracking common exceptions and errors with a monitoring dashboard.
- **Compliance and Security**: Detect potential threats such as profanity and PII leaks.
- **Prompt Injection Detection**: Identify potential code injection and secret leaks.
- **API Keys and Secrets Management**: Securely handle your LLM API keys and secrets centrally, avoiding insecure practices.
- **Prompt Management**: Manage and version Agent prompts using PromptHub for consistent and easy access across Agents.
- **Model Playground** Test and compare different models for your CrewAI agents before deployment.
## Setup Instructions
<Steps>
<Step title="Deploy OpenLIT">
<Steps>
<Step title="Git Clone OpenLIT Repository">
```shell
git clone git@github.com:openlit/openlit.git
```
</Step>
<Step title="Start Docker Compose">
From the root directory of the [OpenLIT Repo](https://github.com/openlit/openlit), Run the below command:
```shell
docker compose up -d
```
</Step>
</Steps>
</Step>
<Step title="Install OpenLIT SDK">
```shell
pip install openlit
```
</Step>
<Step title="Initialize OpenLIT in Your Application">
Add the following two lines to your application code:
<Tabs>
<Tab title="Setup using function arguments">
```python
import openlit
openlit.init(otlp_endpoint="http://127.0.0.1:4318")
```
Example Usage for monitoring a CrewAI Agent:
```python
from crewai import Agent, Task, Crew, Process
import openlit
openlit.init(disable_metrics=True)
# Define your agents
researcher = Agent(
role="Researcher",
goal="Conduct thorough research and analysis on AI and AI agents",
backstory="You're an expert researcher, specialized in technology, software engineering, AI, and startups. You work as a freelancer and are currently researching for a new client.",
allow_delegation=False,
llm='command-r'
)
# Define your task
task = Task(
description="Generate a list of 5 interesting ideas for an article, then write one captivating paragraph for each idea that showcases the potential of a full article on this topic. Return the list of ideas with their paragraphs and your notes.",
expected_output="5 bullet points, each with a paragraph and accompanying notes.",
)
# Define the manager agent
manager = Agent(
role="Project Manager",
goal="Efficiently manage the crew and ensure high-quality task completion",
backstory="You're an experienced project manager, skilled in overseeing complex projects and guiding teams to success. Your role is to coordinate the efforts of the crew members, ensuring that each task is completed on time and to the highest standard.",
allow_delegation=True,
llm='command-r'
)
# Instantiate your crew with a custom manager
crew = Crew(
agents=[researcher],
tasks=[task],
manager_agent=manager,
process=Process.hierarchical,
)
# Start the crew's work
result = crew.kickoff()
print(result)
```
</Tab>
<Tab title="Setup using Environment Variables">
Add the following two lines to your application code:
```python
import openlit
openlit.init()
```
Run the following command to configure the OTEL export endpoint:
```shell
export OTEL_EXPORTER_OTLP_ENDPOINT = "http://127.0.0.1:4318"
```
Example Usage for monitoring a CrewAI Async Agent:
```python
import asyncio
from crewai import Crew, Agent, Task
import openlit
openlit.init(otlp_endpoint="http://127.0.0.1:4318")
# Create an agent with code execution enabled
coding_agent = Agent(
role="Python Data Analyst",
goal="Analyze data and provide insights using Python",
backstory="You are an experienced data analyst with strong Python skills.",
allow_code_execution=True,
llm="command-r"
)
# Create a task that requires code execution
data_analysis_task = Task(
description="Analyze the given dataset and calculate the average age of participants. Ages: {ages}",
agent=coding_agent,
expected_output="5 bullet points, each with a paragraph and accompanying notes.",
)
# Create a crew and add the task
analysis_crew = Crew(
agents=[coding_agent],
tasks=[data_analysis_task]
)
# Async function to kickoff the crew asynchronously
async def async_crew_execution():
result = await analysis_crew.kickoff_async(inputs={"ages": [25, 30, 35, 40, 45]})
print("Crew Result:", result)
# Run the async function
asyncio.run(async_crew_execution())
```
</Tab>
</Tabs>
Refer to OpenLIT [Python SDK repository](https://github.com/openlit/openlit/tree/main/sdk/python) for more advanced configurations and use cases.
</Step>
<Step title="Visualize and Analyze">
With the Agent Observability data now being collected and sent to OpenLIT, the next step is to visualize and analyze this data to get insights into your Agent's performance, behavior, and identify areas of improvement.
Just head over to OpenLIT at `127.0.0.1:3000` on your browser to start exploring. You can login using the default credentials
- **Email**: `user@openlit.io`
- **Password**: `openlituser`
<Frame caption="OpenLIT Dashboard">
<img src="/images/openlit1.png" alt="Overview Agent usage including cost and tokens" />
<img src="/images/openlit2.png" alt="Overview of agent otel traces and metrics" />
</Frame>
</Step>
</Steps>

Binary file not shown.

Before

Width:  |  Height:  |  Size: 390 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 422 KiB

Binary file not shown.

Before

Width:  |  Height:  |  Size: 799 KiB

View File

@@ -1,145 +1,128 @@
---
title: Installation
description: Get started with CrewAI - Install, configure, and build your first AI crew
description:
icon: wrench
---
<Note>
**Python Version Requirements**
CrewAI requires `Python >=3.10 and <3.13`. Here's how to check your version:
```bash
python3 --version
```
If you need to update Python, visit [python.org/downloads](https://python.org/downloads)
</Note>
# Installing CrewAI
This guide will walk you through the installation process for CrewAI and its dependencies.
CrewAI is a flexible and powerful AI framework that enables you to create and manage AI agents, tools, and tasks efficiently.
Let's get you set up! 🚀
Let's get started! 🚀
<Tip>
Make sure you have `Python >=3.10 <=3.13` installed on your system before you proceed.
</Tip>
<Steps>
<Step title="Install CrewAI">
Install CrewAI with all recommended tools using either method:
```shell Terminal
pip install 'crewai[tools]'
```
or
```shell Terminal
pip install crewai crewai-tools
```
<Note>
Both methods install the core package and additional tools needed for most use cases.
</Note>
Install the main CrewAI package with the following command:
<CodeGroup>
```shell Terminal
pip install crewai
```
</CodeGroup>
You can also install the main CrewAI package and the tools package that include a series of helpful tools for your agents:
<CodeGroup>
```shell Terminal
pip install 'crewai[tools]'
```
</CodeGroup>
Alternatively, you can also use:
<CodeGroup>
```shell Terminal
pip install crewai crewai-tools
```
</CodeGroup>
</Step>
<Step title="Upgrade CrewAI">
To upgrade CrewAI and CrewAI Tools to the latest version, run the following command
<CodeGroup>
```shell Terminal
pip install --upgrade crewai crewai-tools
```
</CodeGroup>
<Note>
1. If you're using an older version of CrewAI, you may receive a warning about using `Poetry` for dependency management.
![Error from older versions](./images/crewai-run-poetry-error.png)
<Step title="Upgrade CrewAI (Existing Installations Only)">
If you have an older version of CrewAI installed, you can upgrade it:
2. In this case, you'll need to run the command below to update your project.
This command will migrate your project to use [UV](https://github.com/astral-sh/uv) and update the necessary files.
```shell Terminal
pip install --upgrade crewai crewai-tools
crewai update
```
3. After running the command above, you should see the following output:
![Successfully migrated to UV](./images/crewai-update.png)
<Warning>
If you see a Poetry-related warning, you'll need to migrate to our new dependency manager:
```shell Terminal
crewai update
```
This will update your project to use [UV](https://github.com/astral-sh/uv), our new faster dependency manager.
</Warning>
<Note>
Skip this step if you're doing a fresh installation.
</Note>
4. You're all set! You can now proceed to the next step! 🎉
</Note>
</Step>
<Step title="Verify Installation">
Check your installed versions:
```shell Terminal
pip freeze | grep crewai
```
You should see something like:
```markdown Output
crewai==X.X.X
crewai-tools==X.X.X
```
<Check>Installation successful! You're ready to create your first crew.</Check>
<Step title="Verify the installation">
To verify that `crewai` and `crewai-tools` are installed correctly, run the following command
<CodeGroup>
```shell Terminal
pip freeze | grep crewai
```
</CodeGroup>
You should see the version number of `crewai` and `crewai-tools`.
<CodeGroup>
```markdown Version
crewai==X.X.X
crewai-tools==X.X.X
```
</CodeGroup>
If you see the version number, then the installation was successful! 🎉
</Step>
</Steps>
# Creating a New Project
## Create a new CrewAI project
<Info>
We recommend using the YAML Template scaffolding for a structured approach to defining agents and tasks.
</Info>
The next step is to create a new CrewAI project.
We recommend using the YAML Template scaffolding to get started as it provides a structured approach to defining agents and tasks.
<Steps>
<Step title="Generate Project Structure">
Run the CrewAI CLI command:
```shell Terminal
crewai create crew <project_name>
```
<Step title="Create a new CrewAI project using the YAML Template Configuration">
To create a new CrewAI project, run the following CLI (Command Line Interface) command:
<CodeGroup>
```shell Terminal
crewai create crew <project_name>
```
</CodeGroup>
This command creates a new project folder with the following structure:
| File/Directory | Description |
|:------------------------|:-------------------------------------------------|
| `my_project/` | Root directory of the project |
| ├── `.gitignore` | Specifies files and directories to ignore in Git |
| ├── `pyproject.toml` | Project configuration and dependencies |
| ├── `README.md` | Project documentation |
| ├── `.env` | Environment variables |
| └── `src/` | Source code directory |
| &nbsp;&nbsp;&nbsp;&nbsp;└── `my_project/` | Main application package |
| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;├── `__init__.py` | Marks the directory as a Python package |
| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;├── `main.py` | Main application script |
| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;├── `crew.py` | Crew-related functionalities |
| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;├── `tools/` | Custom tools directory |
| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;│ ├── `custom_tool.py` | Custom tool implementation |
| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;│ └── `__init__.py` | Marks tools directory as a package |
| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;└── `config/` | Configuration files directory |
| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;├── `agents.yaml` | Agent configurations |
| &nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;&nbsp;└── `tasks.yaml` | Task configurations |
This creates a new project with the following structure:
<Frame>
```
my_project/
├── .gitignore
├── pyproject.toml
├── README.md
├── .env
└── src/
└── my_project/
├── __init__.py
├── main.py
├── crew.py
├── tools/
│ ├── custom_tool.py
│ └── __init__.py
└── config/
├── agents.yaml
└── tasks.yaml
```
</Frame>
You can now start developing your crew by editing the files in the `src/my_project` folder.
The `main.py` file is the entry point of the project, the `crew.py` file is where you define your crew, the `agents.yaml` file is where you define your agents,
and the `tasks.yaml` file is where you define your tasks.
</Step>
<Step title="Customize Your Project">
Your project will contain these essential files:
| File | Purpose |
| --- | --- |
| `agents.yaml` | Define your AI agents and their roles |
| `tasks.yaml` | Set up agent tasks and workflows |
| `.env` | Store API keys and environment variables |
| `main.py` | Project entry point and execution flow |
| `crew.py` | Crew orchestration and coordination |
| `tools/` | Directory for custom agent tools |
<Tip>
Start by editing `agents.yaml` and `tasks.yaml` to define your crew's behavior.
Keep sensitive information like API keys in `.env`.
</Tip>
<Step title="Customize your project">
To customize your project, you can:
- Modify `src/my_project/config/agents.yaml` to define your agents.
- Modify `src/my_project/config/tasks.yaml` to define your tasks.
- Modify `src/my_project/crew.py` to add your own logic, tools, and specific arguments.
- Modify `src/my_project/main.py` to add custom inputs for your agents and tasks.
- Add your environment variables into the `.env` file.
</Step>
</Steps>
## Next Steps
## Next steps
<CardGroup cols={2}>
<Card
title="Build Your First Agent"
icon="code"
href="/quickstart"
>
Follow our quickstart guide to create your first CrewAI agent and get hands-on experience.
</Card>
<Card
title="Join the Community"
icon="comments"
href="https://community.crewai.com"
>
Connect with other developers, get help, and share your CrewAI experiences.
</Card>
</CardGroup>
Now that you have installed `crewai` and `crewai-tools`, you're ready to spin up your first crew!
- 👨‍💻 Build your first agent with CrewAI by following the [Quickstart](/quickstart) guide.
- 💬 Join the [Community](https://community.crewai.com) to get help and share your feedback.

View File

@@ -1,85 +1,49 @@
---
title: Introduction
description: Build AI agent teams that work together to tackle complex tasks
description: Welcome to CrewAI docs!
icon: handshake
---
# What is CrewAI?
**CrewAI is a cutting-edge framework for orchestrating autonomous AI agents.**
**CrewAI is a cutting-edge Python framework for orchestrating role-playing, autonomous AI agents.**
CrewAI enables you to create AI teams where each agent has specific roles, tools, and goals, working together to accomplish complex tasks.
By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.
Think of it as assembling your dream team - each member (agent) brings unique skills and expertise, collaborating seamlessly to achieve your objectives.
<Frame caption="CrewAI Mindmap">
<img src="crewAI-mindmap.png" alt="CrewAI Mindmap" />
</Frame>
## How CrewAI Works
## Why CrewAI?
- 🤼‍♀️ **Role-Playing Agents**: Agents can take on different roles and personas to better understand and interact with complex systems.
- 🤖 **Autonomous Decision Making**: Agents can make decisions autonomously based on the given context and available tools.
- 🤝 **Seamless Collaboration**: Agents can work together seamlessly, sharing information and resources to achieve common goals.
- 🧠 **Complex Task Tackling**: CrewAI is designed to tackle complex tasks, such as multi-step workflows, decision making, and problem solving.
<Note>
Just like a company has departments (Sales, Engineering, Marketing) working together under leadership to achieve business goals, CrewAI helps you create an organization of AI agents with specialized roles collaborating to accomplish complex tasks.
</Note>
<Frame caption="CrewAI Framework Overview">
<img src="crewAI-mindmap.png" alt="CrewAI Framework Overview" />
</Frame>
| Component | Description | Key Features |
|:----------|:-----------:|:------------|
| **Crew** | The top-level organization | • Manages AI agent teams<br/>• Oversees workflows<br/>• Ensures collaboration<br/>• Delivers outcomes |
| **AI Agents** | Specialized team members | • Have specific roles (researcher, writer)<br/>• Use designated tools<br/>• Can delegate tasks<br/>• Make autonomous decisions |
| **Process** | Workflow management system | • Defines collaboration patterns<br/>• Controls task assignments<br/>• Manages interactions<br/>• Ensures efficient execution |
| **Tasks** | Individual assignments | • Have clear objectives<br/>• Use specific tools<br/>• Feed into larger process<br/>• Produce actionable results |
### How It All Works Together
1. The **Crew** organizes the overall operation
2. **AI Agents** work on their specialized tasks
3. The **Process** ensures smooth collaboration
4. **Tasks** get completed to achieve the goal
## Key Features
# Get Started with CrewAI
<CardGroup cols={2}>
<Card title="Role-Based Agents" icon="users">
Create specialized agents with defined roles, expertise, and goals - from researchers to analysts to writers
</Card>
<Card title="Flexible Tools" icon="screwdriver-wrench">
Equip agents with custom tools and APIs to interact with external services and data sources
</Card>
<Card title="Intelligent Collaboration" icon="people-arrows">
Agents work together, sharing insights and coordinating tasks to achieve complex objectives
</Card>
<Card title="Task Management" icon="list-check">
Define sequential or parallel workflows, with agents automatically handling task dependencies
</Card>
</CardGroup>
## Why Choose CrewAI?
- 🧠 **Autonomous Operation**: Agents make intelligent decisions based on their roles and available tools
- 📝 **Natural Interaction**: Agents communicate and collaborate like human team members
- 🛠️ **Extensible Design**: Easy to add new tools, roles, and capabilities
- 🚀 **Production Ready**: Built for reliability and scalability in real-world applications
<CardGroup cols={3}>
<Card
title="Install CrewAI"
icon="wrench"
href="/installation"
title="Quickstart"
color="#F3A78B"
href="quickstart"
icon="terminal"
iconType="solid"
>
Get started with CrewAI in your development environment.
</Card>
<Card
title="Quick Start"
icon="bolt"
href="/quickstart"
>
Follow our quickstart guide to create your first CrewAI agent and get hands-on experience.
Getting started with CrewAI
</Card>
<Card
title="Join the Community"
icon="comments"
color="#F3A78B"
href="https://community.crewai.com"
>
Connect with other developers, get help, and share your CrewAI experiences.
</Card>
</CardGroup>
icon="comment-question"
iconType="duotone"
>
Join the CrewAI community and get help with your project!
</Card>
</CardGroup>
## Next Step
- [Install CrewAI](/installation) to get started with your first agent.

View File

@@ -68,7 +68,6 @@
"concepts/tasks",
"concepts/crews",
"concepts/flows",
"concepts/knowledge",
"concepts/llms",
"concepts/processes",
"concepts/collaboration",
@@ -99,8 +98,7 @@
"how-to/replay-tasks-from-latest-crew-kickoff",
"how-to/conditional-tasks",
"how-to/agentops-observability",
"how-to/langtrace-observability",
"how-to/openlit-observability"
"how-to/langtrace-observability"
]
},
{

View File

@@ -8,7 +8,7 @@ icon: rocket
Let's create a simple crew that will help us `research` and `report` on the `latest AI developments` for a given topic or subject.
Before we proceed, make sure you have `crewai` and `crewai-tools` installed.
Before we proceed, make sure you have `crewai` and `crewai-tools` installed.
If you haven't installed them yet, you can do so by following the [installation guide](/installation).
Follow the steps below to get crewing! 🚣‍♂️
@@ -23,7 +23,7 @@ Follow the steps below to get crewing! 🚣‍♂️
```
</CodeGroup>
</Step>
<Step title="Modify your `agents.yaml` file">
<Step title="Modify your `agents.yaml` file">
<Tip>
You can also modify the agents as needed to fit your use case or copy and paste as is to your project.
Any variable interpolated in your `agents.yaml` and `tasks.yaml` files like `{topic}` will be replaced by the value of the variable in the `main.py` file.
@@ -39,7 +39,7 @@ Follow the steps below to get crewing! 🚣‍♂️
You're a seasoned researcher with a knack for uncovering the latest
developments in {topic}. Known for your ability to find the most relevant
information and present it in a clear and concise manner.
reporting_analyst:
role: >
{topic} Reporting Analyst
@@ -51,7 +51,7 @@ Follow the steps below to get crewing! 🚣‍♂️
it easy for others to understand and act on the information you provide.
```
</Step>
<Step title="Modify your `tasks.yaml` file">
<Step title="Modify your `tasks.yaml` file">
```yaml tasks.yaml
# src/latest_ai_development/config/tasks.yaml
research_task:
@@ -73,8 +73,8 @@ Follow the steps below to get crewing! 🚣‍♂️
agent: reporting_analyst
output_file: report.md
```
</Step>
<Step title="Modify your `crew.py` file">
</Step>
<Step title="Modify your `crew.py` file">
```python crew.py
# src/latest_ai_development/crew.py
from crewai import Agent, Crew, Process, Task
@@ -121,34 +121,10 @@ Follow the steps below to get crewing! 🚣‍♂️
tasks=self.tasks, # Automatically created by the @task decorator
process=Process.sequential,
verbose=True,
)
)
```
</Step>
<Step title="[Optional] Add before and after crew functions">
```python crew.py
# src/latest_ai_development/crew.py
from crewai import Agent, Crew, Process, Task
from crewai.project import CrewBase, agent, crew, task, before_kickoff, after_kickoff
from crewai_tools import SerperDevTool
@CrewBase
class LatestAiDevelopmentCrew():
"""LatestAiDevelopment crew"""
@before_kickoff
def before_kickoff_function(self, inputs):
print(f"Before kickoff function with inputs: {inputs}")
return inputs # You can return the inputs or modify them as needed
@after_kickoff
def after_kickoff_function(self, result):
print(f"After kickoff function with result: {result}")
return result # You can return the result or modify it as needed
# ... remaining code
```
</Step>
<Step title="Feel free to pass custom inputs to your crew">
<Step title="Feel free to pass custom inputs to your crew">
For example, you can pass the `topic` input to your crew to customize the research and reporting.
```python main.py
#!/usr/bin/env python
@@ -261,14 +237,14 @@ Follow the steps below to get crewing! 🚣‍♂️
### Note on Consistency in Naming
The names you use in your YAML files (`agents.yaml` and `tasks.yaml`) should match the method names in your Python code.
For example, you can reference the agent for specific tasks from `tasks.yaml` file.
For example, you can reference the agent for specific tasks from `tasks.yaml` file.
This naming consistency allows CrewAI to automatically link your configurations with your code; otherwise, your task won't recognize the reference properly.
#### Example References
<Tip>
Note how we use the same name for the agent in the `agents.yaml` (`email_summarizer`) file as the method name in the `crew.py` (`email_summarizer`) file.
</Tip>
</Tip>
```yaml agents.yaml
email_summarizer:
@@ -305,8 +281,6 @@ Use the annotations to properly reference the agent and task in the `crew.py` fi
* `@task`
* `@crew`
* `@tool`
* `@before_kickoff`
* `@after_kickoff`
* `@callback`
* `@output_json`
* `@output_pydantic`
@@ -330,7 +304,7 @@ def email_summarizer_task(self) -> Task:
<Tip>
In addition to the [sequential process](../how-to/sequential-process), you can use the [hierarchical process](../how-to/hierarchical-process),
which automatically assigns a manager to the defined crew to properly coordinate the planning and execution of tasks through delegation and validation of results.
which automatically assigns a manager to the defined crew to properly coordinate the planning and execution of tasks through delegation and validation of results.
You can learn more about the core concepts [here](/concepts).
</Tip>
@@ -349,28 +323,11 @@ Replace `<task_id>` with the ID of the task you want to replay.
If you need to reset the memory of your crew before running it again, you can do so by calling the reset memory feature:
```shell
crewai reset-memories --all
crewai reset-memory
```
This will clear the crew's memory, allowing for a fresh start.
## Deploying Your Project
The easiest way to deploy your crew is through CrewAI Enterprise, where you can deploy your crew in a few clicks.
<CardGroup cols={2}>
<Card
title="Deploy on Enterprise"
icon="rocket"
href="http://app.crewai.com"
>
Get started with CrewAI Enterprise and deploy your crew in a production environment with just a few clicks.
</Card>
<Card
title="Join the Community"
icon="comments"
href="https://community.crewai.com"
>
Join our open source community to discuss ideas, share your projects, and connect with other CrewAI developers.
</Card>
</CardGroup>
The easiest way to deploy your crew is through [CrewAI Enterprise](https://www.crewai.com/crewaiplus), where you can deploy your crew in a few clicks.

View File

@@ -1,222 +0,0 @@
---
title: BraveSearchTool
description: A tool for performing web searches using the Brave Search API
icon: search
---
## BraveSearchTool
The BraveSearchTool enables web searches using the Brave Search API, providing customizable result counts, country-specific searches, and rate-limited operations. It formats search results with titles, URLs, and snippets for easy consumption.
## Installation
```bash
pip install 'crewai[tools]'
```
## Authentication
Set up your Brave Search API key:
```bash
export BRAVE_API_KEY='your-brave-api-key'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import BraveSearchTool
# Basic initialization
search_tool = BraveSearchTool()
# Advanced initialization with custom parameters
search_tool = BraveSearchTool(
country="US", # Country-specific search
n_results=5, # Number of results to return
save_file=True # Save results to file
)
# Create an agent with the tool
researcher = Agent(
role='Web Researcher',
goal='Search and analyze web content',
backstory='Expert at finding relevant information online.',
tools=[search_tool],
verbose=True
)
```
## Input Schema
```python
class BraveSearchToolSchema(BaseModel):
search_query: str = Field(
description="Mandatory search query you want to use to search the internet"
)
```
## Function Signature
```python
def __init__(
self,
country: Optional[str] = "",
n_results: int = 10,
save_file: bool = False,
*args,
**kwargs
):
"""
Initialize the Brave search tool.
Args:
country (Optional[str]): Country code for region-specific search
n_results (int): Number of results to return (default: 10)
save_file (bool): Whether to save results to file (default: False)
"""
def _run(
self,
**kwargs: Any
) -> str:
"""
Execute web search using Brave Search API.
Args:
search_query (str): Query to search
save_file (bool, optional): Override save_file setting
n_results (int, optional): Override n_results setting
Returns:
str: Formatted search results with titles, URLs, and snippets
"""
```
## Best Practices
1. API Authentication:
- Securely store BRAVE_API_KEY
- Keep API key confidential
- Handle authentication errors
2. Rate Limiting:
- Tool automatically handles rate limiting
- Minimum 1-second interval between requests
- Consider implementing additional rate limits
3. Search Optimization:
- Use specific search queries
- Adjust result count based on needs
- Consider regional search requirements
4. Error Handling:
- Handle API request failures
- Manage parsing errors
- Monitor rate limit errors
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import BraveSearchTool
# Initialize tool with custom configuration
search_tool = BraveSearchTool(
country="GB", # UK-specific search
n_results=3, # Limit to 3 results
save_file=True # Save results to file
)
# Create agent
researcher = Agent(
role='Web Researcher',
goal='Research latest AI developments',
backstory='Expert at finding and analyzing tech news.',
tools=[search_tool]
)
# Define task
research_task = Task(
description="""Find the latest news about artificial
intelligence developments in quantum computing.""",
agent=researcher
)
# The tool will use:
# {
# "search_query": "latest quantum computing AI developments"
# }
# Create crew
crew = Crew(
agents=[researcher],
tasks=[research_task]
)
# Execute
result = crew.kickoff()
```
## Advanced Usage
### Country-Specific Search
```python
# Initialize tools for different regions
us_search = BraveSearchTool(country="US")
uk_search = BraveSearchTool(country="GB")
jp_search = BraveSearchTool(country="JP")
# Compare results across regions
us_results = us_search.run(
search_query="local news"
)
uk_results = uk_search.run(
search_query="local news"
)
jp_results = jp_search.run(
search_query="local news"
)
```
### Result Management
```python
# Save results to file
archival_search = BraveSearchTool(
save_file=True,
n_results=20
)
# Search and save
results = archival_search.run(
search_query="historical events 2023"
)
# Results saved to search_results_YYYY-MM-DD_HH-MM-SS.txt
```
### Error Handling Example
```python
try:
search_tool = BraveSearchTool()
results = search_tool.run(
search_query="important topic"
)
print(results)
except ValueError as e: # API key missing
print(f"Authentication error: {str(e)}")
except Exception as e:
print(f"Search error: {str(e)}")
```
## Notes
- Requires Brave Search API key
- Implements automatic rate limiting
- Supports country-specific searches
- Customizable result count
- Optional file saving feature
- Thread-safe operations
- Efficient result formatting
- Handles API errors gracefully
- Supports parallel searches
- Maintains search context

View File

@@ -1,164 +0,0 @@
---
title: CodeDocsSearchTool
description: A semantic search tool for code documentation websites using RAG capabilities
icon: book-open
---
## CodeDocsSearchTool
The CodeDocsSearchTool is a specialized Retrieval-Augmented Generation (RAG) tool that enables semantic search within code documentation websites. It inherits from the base RagTool class and provides both fixed and dynamic documentation URL searching capabilities.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import CodeDocsSearchTool
# Method 1: Dynamic documentation URL
docs_search = CodeDocsSearchTool()
# Method 2: Fixed documentation URL
fixed_docs_search = CodeDocsSearchTool(
docs_url="https://docs.example.com"
)
# Create an agent with the tool
researcher = Agent(
role='Documentation Researcher',
goal='Search through code documentation semantically',
backstory='Expert at finding relevant information in technical documentation.',
tools=[docs_search],
verbose=True
)
```
## Input Schema
The tool supports two input schemas depending on initialization:
### Dynamic URL Schema
```python
class CodeDocsSearchToolSchema(BaseModel):
search_query: str # The semantic search query
docs_url: str # URL of the documentation site to search
```
### Fixed URL Schema
```python
class FixedCodeDocsSearchToolSchema(BaseModel):
search_query: str # The semantic search query
```
## Function Signature
```python
def __init__(self, docs_url: Optional[str] = None, **kwargs):
"""
Initialize the documentation search tool.
Args:
docs_url (Optional[str]): Fixed URL to a documentation site. If provided,
the tool will only search this documentation.
**kwargs: Additional arguments passed to the parent RagTool
"""
def _run(self, search_query: str, **kwargs: Any) -> Any:
"""
Perform semantic search on the documentation site.
Args:
search_query (str): The semantic search query
**kwargs: Additional arguments (including 'docs_url' for dynamic mode)
Returns:
str: Relevant documentation passages based on semantic search
"""
```
## Best Practices
1. Choose initialization method based on use case:
- Use fixed URL when repeatedly searching the same documentation
- Use dynamic URL when searching different documentation sites
2. Write clear, semantic search queries
3. Ensure documentation sites are accessible
4. Consider documentation structure and size
5. Handle potential URL access errors in agent prompts
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import CodeDocsSearchTool
# Example 1: Fixed documentation search
api_docs_search = CodeDocsSearchTool(
docs_url="https://api.example.com/docs"
)
# Example 2: Dynamic documentation search
flexible_docs_search = CodeDocsSearchTool()
# Create agents
api_analyst = Agent(
role='API Documentation Analyst',
goal='Find relevant API endpoints and usage examples',
backstory='Expert at analyzing API documentation.',
tools=[api_docs_search]
)
docs_researcher = Agent(
role='Documentation Researcher',
goal='Search through various documentation sites',
backstory='Specialist in finding information across multiple docs.',
tools=[flexible_docs_search]
)
# Define tasks
fixed_search_task = Task(
description="""Find all authentication-related endpoints
in the API documentation.""",
agent=api_analyst
)
# The agent will use:
# {
# "search_query": "authentication endpoints and methods"
# }
dynamic_search_task = Task(
description="""Search through the Python documentation at
docs.python.org for information about async/await.""",
agent=docs_researcher
)
# The agent will use:
# {
# "search_query": "async await syntax and usage",
# "docs_url": "https://docs.python.org"
# }
# Create crew
crew = Crew(
agents=[api_analyst, docs_researcher],
tasks=[fixed_search_task, dynamic_search_task]
)
# Execute
result = crew.kickoff()
```
## Notes
- Inherits from RagTool for semantic search capabilities
- Supports both fixed and dynamic documentation URLs
- Uses embeddings for semantic search
- Thread-safe operations
- Automatically handles documentation loading and embedding
- Optimized for technical documentation search

View File

@@ -1,224 +0,0 @@
---
title: CodeInterpreterTool
description: A tool for secure Python code execution in isolated Docker environments
icon: code
---
## CodeInterpreterTool
The CodeInterpreterTool provides secure Python code execution capabilities using Docker containers. It supports dynamic library installation and offers both safe (Docker-based) and unsafe (direct) execution modes.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import CodeInterpreterTool
# Initialize the tool
code_tool = CodeInterpreterTool()
# Create an agent with the tool
programmer = Agent(
role='Code Executor',
goal='Execute and analyze Python code',
backstory='Expert at writing and executing Python code.',
tools=[code_tool],
verbose=True
)
```
## Input Schema
```python
class CodeInterpreterSchema(BaseModel):
code: str = Field(
description="Python3 code used to be interpreted in the Docker container. ALWAYS PRINT the final result and the output of the code"
)
libraries_used: List[str] = Field(
description="List of libraries used in the code with proper installing names separated by commas. Example: numpy,pandas,beautifulsoup4"
)
```
## Function Signature
```python
def __init__(
self,
code: Optional[str] = None,
user_dockerfile_path: Optional[str] = None,
user_docker_base_url: Optional[str] = None,
unsafe_mode: bool = False,
**kwargs
):
"""
Initialize the code interpreter tool.
Args:
code (Optional[str]): Default code to execute
user_dockerfile_path (Optional[str]): Custom Dockerfile path
user_docker_base_url (Optional[str]): Custom Docker daemon URL
unsafe_mode (bool): Enable direct code execution
**kwargs: Additional arguments for base tool
"""
def _run(
self,
code: str,
libraries_used: List[str],
**kwargs: Any
) -> str:
"""
Execute Python code in Docker container or directly.
Args:
code (str): Python code to execute
libraries_used (List[str]): Required libraries
**kwargs: Additional arguments
Returns:
str: Execution output or error message
"""
```
## Best Practices
1. Security Considerations:
- Use Docker mode by default
- Validate input code
- Control library access
- Monitor execution time
2. Docker Configuration:
- Use custom Dockerfile when needed
- Handle container lifecycle
- Manage resource limits
- Clean up after execution
3. Library Management:
- Specify exact versions
- Use trusted packages
- Handle dependencies
- Verify installations
4. Error Handling:
- Catch execution errors
- Handle timeouts
- Manage Docker errors
- Provide clear messages
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import CodeInterpreterTool
# Initialize tool
code_tool = CodeInterpreterTool()
# Create agent
programmer = Agent(
role='Code Executor',
goal='Execute data analysis code',
backstory='Expert Python programmer specializing in data analysis.',
tools=[code_tool]
)
# Define task
analysis_task = Task(
description="""Analyze the dataset using pandas and
create a summary visualization with matplotlib.""",
agent=programmer
)
# The tool will use:
# {
# "code": """
# import pandas as pd
# import matplotlib.pyplot as plt
#
# # Load and analyze data
# df = pd.read_csv('data.csv')
# summary = df.describe()
#
# # Create visualization
# plt.figure(figsize=(10, 6))
# df['column'].hist()
# plt.savefig('output.png')
#
# print(summary)
# """,
# "libraries_used": "pandas,matplotlib"
# }
# Create crew
crew = Crew(
agents=[programmer],
tasks=[analysis_task]
)
# Execute
result = crew.kickoff()
```
## Advanced Usage
### Custom Docker Configuration
```python
# Use custom Dockerfile
tool = CodeInterpreterTool(
user_dockerfile_path="/path/to/Dockerfile"
)
# Use custom Docker daemon
tool = CodeInterpreterTool(
user_docker_base_url="tcp://remote-docker:2375"
)
```
### Direct Execution Mode
```python
# Enable unsafe mode (not recommended)
tool = CodeInterpreterTool(unsafe_mode=True)
# Execute code directly
result = tool.run(
code="print('Hello, World!')",
libraries_used=[]
)
```
### Error Handling Example
```python
try:
code_tool = CodeInterpreterTool()
result = code_tool.run(
code="""
import numpy as np
arr = np.array([1, 2, 3])
print(f"Array mean: {arr.mean()}")
""",
libraries_used=["numpy"]
)
print(result)
except Exception as e:
print(f"Error executing code: {str(e)}")
```
## Notes
- Inherits from BaseTool
- Docker-based isolation
- Dynamic library installation
- Secure code execution
- Custom Docker support
- Comprehensive error handling
- Resource management
- Container cleanup
- Library dependency handling
- Execution output capture

View File

@@ -1,207 +0,0 @@
---
title: CSVSearchTool
description: A tool for semantic search within CSV files using RAG capabilities
icon: table
---
## CSVSearchTool
The CSVSearchTool enables semantic search capabilities for CSV files using Retrieval-Augmented Generation (RAG). It can process CSV files either specified during initialization or at runtime, making it flexible for various use cases.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import CSVSearchTool
# Method 1: Initialize with specific CSV file
csv_tool = CSVSearchTool(csv="path/to/data.csv")
# Method 2: Initialize without CSV (specify at runtime)
flexible_csv_tool = CSVSearchTool()
# Create an agent with the tool
data_analyst = Agent(
role='Data Analyst',
goal='Search and analyze CSV data semantically',
backstory='Expert at analyzing and extracting insights from CSV data.',
tools=[csv_tool],
verbose=True
)
```
## Input Schema
### Fixed CSV Schema (when CSV path provided during initialization)
```python
class FixedCSVSearchToolSchema(BaseModel):
search_query: str = Field(
description="Mandatory search query you want to use to search the CSV's content"
)
```
### Flexible CSV Schema (when CSV path provided at runtime)
```python
class CSVSearchToolSchema(FixedCSVSearchToolSchema):
csv: str = Field(
description="Mandatory csv path you want to search"
)
```
## Function Signature
```python
def __init__(
self,
csv: Optional[str] = None,
**kwargs
):
"""
Initialize the CSV search tool.
Args:
csv (Optional[str]): Path to CSV file (optional)
**kwargs: Additional arguments for RAG tool configuration
"""
def _run(
self,
search_query: str,
**kwargs: Any
) -> str:
"""
Execute semantic search on CSV content.
Args:
search_query (str): Query to search in the CSV
**kwargs: Additional arguments including csv path if not initialized
Returns:
str: Relevant content from the CSV matching the query
"""
```
## Best Practices
1. CSV File Handling:
- Ensure CSV files are properly formatted
- Use absolute paths for reliability
- Verify file permissions before processing
2. Search Optimization:
- Use specific, focused search queries
- Consider column names and data structure
- Test with sample queries first
3. Performance Considerations:
- Pre-initialize with CSV for repeated searches
- Handle large CSV files appropriately
- Monitor memory usage with big datasets
4. Error Handling:
- Verify CSV file existence
- Handle malformed CSV data
- Manage file access permissions
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import CSVSearchTool
# Initialize tool with specific CSV
csv_tool = CSVSearchTool(csv="/path/to/sales_data.csv")
# Create agent
analyst = Agent(
role='Data Analyst',
goal='Extract insights from sales data',
backstory='Expert at analyzing sales data and trends.',
tools=[csv_tool]
)
# Define task
analysis_task = Task(
description="""Find all sales records from the CSV
that relate to product returns in Q4 2023.""",
agent=analyst
)
# The tool will use:
# {
# "search_query": "product returns Q4 2023"
# }
# Create crew
crew = Crew(
agents=[analyst],
tasks=[analysis_task]
)
# Execute
result = crew.kickoff()
```
## Advanced Usage
### Dynamic CSV Selection
```python
# Initialize without CSV
flexible_tool = CSVSearchTool()
# Search different CSVs
result1 = flexible_tool.run(
search_query="revenue 2023",
csv="/path/to/finance.csv"
)
result2 = flexible_tool.run(
search_query="customer feedback",
csv="/path/to/surveys.csv"
)
```
### Multiple CSV Analysis
```python
# Create tools for different CSVs
sales_tool = CSVSearchTool(csv="/path/to/sales.csv")
inventory_tool = CSVSearchTool(csv="/path/to/inventory.csv")
# Create agent with multiple tools
analyst = Agent(
role='Business Analyst',
goal='Cross-reference sales and inventory data',
tools=[sales_tool, inventory_tool]
)
```
### Error Handling Example
```python
try:
csv_tool = CSVSearchTool(csv="/path/to/data.csv")
result = csv_tool.run(
search_query="important metrics"
)
print(result)
except Exception as e:
print(f"Error processing CSV: {str(e)}")
```
## Notes
- Inherits from RagTool for semantic search
- Supports dynamic CSV file specification
- Uses embedchain for data processing
- Maintains search context across queries
- Thread-safe operations
- Efficient semantic search capabilities
- Supports various CSV formats
- Handles large datasets effectively
- Preserves CSV structure in search
- Enables natural language queries

View File

@@ -1,217 +0,0 @@
---
title: Directory Read Tool
description: A tool for recursively listing directory contents
---
# Directory Read Tool
The Directory Read Tool provides functionality to recursively list all files within a directory. It supports both fixed and dynamic directory path modes, allowing you to specify the directory at initialization or runtime.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage
You can use the Directory Read Tool in two ways:
### 1. Fixed Directory Path
Initialize the tool with a specific directory path:
```python
from crewai import Agent
from crewai_tools import DirectoryReadTool
# Initialize with a fixed directory
tool = DirectoryReadTool(directory="/path/to/your/directory")
# Create an agent with the tool
agent = Agent(
role='File System Analyst',
goal='Analyze directory contents',
backstory='I help analyze and organize file systems',
tools=[tool]
)
# Use in a task
task = Task(
description="List all files in the project directory",
agent=agent
)
```
### 2. Dynamic Directory Path
Initialize the tool without a specific directory path to provide it at runtime:
```python
from crewai import Agent
from crewai_tools import DirectoryReadTool
# Initialize without a fixed directory
tool = DirectoryReadTool()
# Create an agent with the tool
agent = Agent(
role='File System Explorer',
goal='Explore different directories',
backstory='I analyze various directory structures',
tools=[tool]
)
# Use in a task with dynamic directory path
task = Task(
description="List all files in the specified directory",
agent=agent,
context={
"directory": "/path/to/explore"
}
)
```
## Input Schema
### Fixed Directory Mode
```python
class FixedDirectoryReadToolSchema(BaseModel):
pass # No additional parameters needed when directory is fixed
```
### Dynamic Directory Mode
```python
class DirectoryReadToolSchema(BaseModel):
directory: str # The path to the directory to list contents
```
## Function Signatures
```python
def __init__(self, directory: Optional[str] = None, **kwargs):
"""
Initialize the Directory Read Tool.
Args:
directory (Optional[str]): Path to the directory (optional)
**kwargs: Additional arguments passed to BaseTool
"""
def _run(
self,
**kwargs: Any,
) -> str:
"""
Execute the directory listing.
Args:
**kwargs: Arguments including 'directory' for dynamic mode
Returns:
str: A formatted string containing all file paths in the directory
"""
```
## Best Practices
1. **Path Handling**:
- Use absolute paths to avoid path resolution issues
- Handle trailing slashes appropriately
- Verify directory existence before listing
2. **Performance Considerations**:
- Be mindful of directory size when listing large directories
- Consider implementing pagination for large directories
- Handle symlinks appropriately
3. **Error Handling**:
- Handle directory not found errors gracefully
- Manage permission issues appropriately
- Validate input parameters before processing
## Example Integration
Here's a complete example showing how to integrate the Directory Read Tool with CrewAI:
```python
from crewai import Agent, Task, Crew
from crewai_tools import DirectoryReadTool
# Initialize the tool
dir_tool = DirectoryReadTool()
# Create an agent with the tool
file_analyst = Agent(
role='File System Analyst',
goal='Analyze and report on directory structures',
backstory='I am an expert at analyzing file system organization',
tools=[dir_tool]
)
# Create tasks
analysis_task = Task(
description="""
Analyze the project directory structure:
1. List all files recursively
2. Identify key file types
3. Report on directory organization
Provide a comprehensive analysis of the findings.
""",
agent=file_analyst,
context={
"directory": "/path/to/project"
}
)
# Create and run the crew
crew = Crew(
agents=[file_analyst],
tasks=[analysis_task]
)
result = crew.kickoff()
```
## Error Handling
The tool handles various error scenarios:
1. **Directory Not Found**:
```python
try:
tool = DirectoryReadTool(directory="/nonexistent/path")
except FileNotFoundError:
print("Directory not found. Please verify the path.")
```
2. **Permission Issues**:
```python
try:
tool = DirectoryReadTool(directory="/restricted/path")
except PermissionError:
print("Insufficient permissions to access the directory.")
```
3. **Invalid Path**:
```python
try:
result = tool._run(directory="invalid/path")
except ValueError:
print("Invalid directory path provided.")
```
## Output Format
The tool returns a formatted string containing all file paths in the directory:
```
File paths:
- /path/to/directory/file1.txt
- /path/to/directory/subdirectory/file2.txt
- /path/to/directory/subdirectory/file3.py
```
Each file path is listed on a new line with a hyphen prefix, making it easy to parse and read the output.

View File

@@ -1,214 +0,0 @@
---
title: DirectorySearchTool
description: A tool for semantic search within directory contents using RAG capabilities
icon: folder-search
---
## DirectorySearchTool
The DirectorySearchTool enables semantic search capabilities for directory contents using Retrieval-Augmented Generation (RAG). It processes files recursively within a directory and allows searching through their contents using natural language queries.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import DirectorySearchTool
# Method 1: Initialize with specific directory
dir_tool = DirectorySearchTool(directory="/path/to/documents")
# Method 2: Initialize without directory (specify at runtime)
flexible_dir_tool = DirectorySearchTool()
# Create an agent with the tool
researcher = Agent(
role='Directory Researcher',
goal='Search and analyze directory contents',
backstory='Expert at finding relevant information in document collections.',
tools=[dir_tool],
verbose=True
)
```
## Input Schema
### Fixed Directory Schema (when path provided during initialization)
```python
class FixedDirectorySearchToolSchema(BaseModel):
search_query: str = Field(
description="Mandatory search query you want to use to search the directory's content"
)
```
### Flexible Directory Schema (when path provided at runtime)
```python
class DirectorySearchToolSchema(FixedDirectorySearchToolSchema):
directory: str = Field(
description="Mandatory directory you want to search"
)
```
## Function Signature
```python
def __init__(
self,
directory: Optional[str] = None,
**kwargs
):
"""
Initialize the directory search tool.
Args:
directory (Optional[str]): Path to directory (optional)
**kwargs: Additional arguments for RAG tool configuration
"""
def _run(
self,
search_query: str,
**kwargs: Any
) -> str:
"""
Execute semantic search on directory contents.
Args:
search_query (str): Query to search in the directory
**kwargs: Additional arguments including directory if not initialized
Returns:
str: Relevant content from the directory matching the query
"""
```
## Best Practices
1. Directory Management:
- Use absolute paths
- Verify directory existence
- Handle permissions properly
2. Search Optimization:
- Use specific queries
- Consider file types
- Test with sample queries
3. Performance Considerations:
- Pre-initialize for repeated searches
- Handle large directories
- Monitor processing time
4. Error Handling:
- Verify directory access
- Handle missing files
- Manage permissions
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import DirectorySearchTool
# Initialize tool with specific directory
dir_tool = DirectorySearchTool(
directory="/path/to/documents"
)
# Create agent
researcher = Agent(
role='Directory Researcher',
goal='Extract insights from document collections',
backstory='Expert at analyzing document collections.',
tools=[dir_tool]
)
# Define task
research_task = Task(
description="""Find all mentions of machine learning
applications from the directory contents.""",
agent=researcher
)
# The tool will use:
# {
# "search_query": "machine learning applications"
# }
# Create crew
crew = Crew(
agents=[researcher],
tasks=[research_task]
)
# Execute
result = crew.kickoff()
```
## Advanced Usage
### Dynamic Directory Selection
```python
# Initialize without directory path
flexible_tool = DirectorySearchTool()
# Search different directories
docs_results = flexible_tool.run(
search_query="technical specifications",
directory="/path/to/docs"
)
reports_results = flexible_tool.run(
search_query="financial metrics",
directory="/path/to/reports"
)
```
### Multiple Directory Analysis
```python
# Create tools for different directories
docs_tool = DirectorySearchTool(
directory="/path/to/docs"
)
reports_tool = DirectorySearchTool(
directory="/path/to/reports"
)
# Create agent with multiple tools
analyst = Agent(
role='Content Analyst',
goal='Cross-reference multiple document collections',
tools=[docs_tool, reports_tool]
)
```
### Error Handling Example
```python
try:
dir_tool = DirectorySearchTool()
results = dir_tool.run(
search_query="key concepts",
directory="/path/to/documents"
)
print(results)
except Exception as e:
print(f"Error processing directory: {str(e)}")
```
## Notes
- Inherits from RagTool
- Uses DirectoryLoader
- Supports recursive search
- Dynamic directory specification
- Efficient content retrieval
- Thread-safe operations
- Maintains search context
- Processes multiple file types
- Handles nested directories
- Memory-efficient processing

View File

@@ -1,224 +0,0 @@
---
title: DOCXSearchTool
description: A tool for semantic search within DOCX documents using RAG capabilities
icon: file-text
---
## DOCXSearchTool
The DOCXSearchTool enables semantic search capabilities for Microsoft Word (DOCX) documents using Retrieval-Augmented Generation (RAG). It supports both fixed and dynamic document selection modes.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import DOCXSearchTool
# Method 1: Fixed document (specified at initialization)
fixed_tool = DOCXSearchTool(
docx="path/to/document.docx"
)
# Method 2: Dynamic document (specified at runtime)
dynamic_tool = DOCXSearchTool()
# Create an agent with the tool
researcher = Agent(
role='Document Researcher',
goal='Search and analyze document contents',
backstory='Expert at finding relevant information in documents.',
tools=[fixed_tool], # or [dynamic_tool]
verbose=True
)
```
## Input Schema
### Fixed Document Mode
```python
class FixedDOCXSearchToolSchema(BaseModel):
search_query: str = Field(
description="Mandatory search query you want to use to search the DOCX's content"
)
```
### Dynamic Document Mode
```python
class DOCXSearchToolSchema(BaseModel):
docx: str = Field(
description="Mandatory docx path you want to search"
)
search_query: str = Field(
description="Mandatory search query you want to use to search the DOCX's content"
)
```
## Function Signature
```python
def __init__(
self,
docx: Optional[str] = None,
**kwargs
):
"""
Initialize the DOCX search tool.
Args:
docx (Optional[str]): Path to DOCX file (optional for dynamic mode)
**kwargs: Additional arguments for RAG tool configuration
"""
def _run(
self,
search_query: str,
docx: Optional[str] = None,
**kwargs: Any
) -> str:
"""
Execute semantic search on document contents.
Args:
search_query (str): Query to search in the document
docx (Optional[str]): Document path (required for dynamic mode)
**kwargs: Additional arguments
Returns:
str: Relevant content from the document matching the query
"""
```
## Best Practices
1. Document Handling:
- Use absolute file paths
- Verify file existence
- Handle large documents
- Monitor memory usage
2. Query Optimization:
- Structure queries clearly
- Consider document size
- Handle formatting
- Monitor performance
3. Error Handling:
- Check file access
- Validate file format
- Handle corrupted files
- Log issues
4. Mode Selection:
- Choose fixed mode for static documents
- Use dynamic mode for runtime selection
- Consider memory implications
- Manage document lifecycle
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import DOCXSearchTool
# Initialize tool
docx_tool = DOCXSearchTool(
docx="reports/annual_report_2023.docx"
)
# Create agent
researcher = Agent(
role='Document Analyst',
goal='Extract insights from annual report',
backstory='Expert at analyzing business documents.',
tools=[docx_tool]
)
# Define task
analysis_task = Task(
description="""Find all mentions of revenue growth
and market expansion.""",
agent=researcher
)
# Create crew
crew = Crew(
agents=[researcher],
tasks=[analysis_task]
)
# Execute
result = crew.kickoff()
```
## Advanced Usage
### Multiple Document Analysis
```python
# Create tools for different documents
report_tool = DOCXSearchTool(
docx="reports/annual_report.docx"
)
policy_tool = DOCXSearchTool(
docx="policies/compliance.docx"
)
# Create agent with multiple tools
analyst = Agent(
role='Document Analyst',
goal='Cross-reference reports and policies',
tools=[report_tool, policy_tool]
)
```
### Dynamic Document Loading
```python
# Initialize dynamic tool
dynamic_tool = DOCXSearchTool()
# Use with different documents
result1 = dynamic_tool.run(
docx="document1.docx",
search_query="project timeline"
)
result2 = dynamic_tool.run(
docx="document2.docx",
search_query="budget allocation"
)
```
### Error Handling Example
```python
try:
docx_tool = DOCXSearchTool(
docx="reports/quarterly_report.docx"
)
results = docx_tool.run(
search_query="Q3 performance metrics"
)
print(results)
except FileNotFoundError as e:
print(f"Document not found: {str(e)}")
except Exception as e:
print(f"Error processing document: {str(e)}")
```
## Notes
- Inherits from RagTool
- Supports fixed/dynamic modes
- Document path validation
- Memory management
- Performance optimization
- Error handling
- Search capabilities
- Content extraction
- Format handling
- Security features

View File

@@ -1,193 +0,0 @@
---
title: FileReadTool
description: A tool for reading file contents with flexible path specification
icon: file-text
---
## FileReadTool
The FileReadTool provides functionality to read file contents with support for both fixed and dynamic file path specification. It includes comprehensive error handling for common file operations and maintains clear descriptions of its configured state.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import FileReadTool
# Method 1: Initialize with specific file
reader = FileReadTool(file_path="/path/to/data.txt")
# Method 2: Initialize without file (specify at runtime)
flexible_reader = FileReadTool()
# Create an agent with the tool
file_processor = Agent(
role='File Processor',
goal='Read and process file contents',
backstory='Expert at handling file operations and content processing.',
tools=[reader],
verbose=True
)
```
## Input Schema
```python
class FileReadToolSchema(BaseModel):
file_path: str = Field(
description="Mandatory file full path to read the file"
)
```
## Function Signature
```python
def __init__(
self,
file_path: Optional[str] = None,
**kwargs: Any
) -> None:
"""
Initialize the file read tool.
Args:
file_path (Optional[str]): Path to file to read (optional)
**kwargs: Additional arguments passed to BaseTool
"""
def _run(
self,
**kwargs: Any
) -> str:
"""
Read and return file contents.
Args:
file_path (str, optional): Override default file path
**kwargs: Additional arguments
Returns:
str: File contents or error message
"""
```
## Best Practices
1. File Path Management:
- Use absolute paths for reliability
- Verify file existence before operations
- Handle path resolution properly
2. Error Handling:
- Check for file existence
- Handle permission issues
- Manage encoding errors
- Process file access failures
3. Performance Considerations:
- Close files after reading
- Handle large files appropriately
- Consider memory constraints
4. Security Practices:
- Validate file paths
- Check file permissions
- Avoid path traversal issues
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import FileReadTool
# Initialize tool with specific file
reader = FileReadTool(file_path="/path/to/config.txt")
# Create agent
processor = Agent(
role='File Processor',
goal='Process configuration files',
backstory='Expert at reading and analyzing configuration files.',
tools=[reader]
)
# Define task
read_task = Task(
description="""Read and analyze the contents of
the configuration file.""",
agent=processor
)
# The tool will use the default file path
# Create crew
crew = Crew(
agents=[processor],
tasks=[read_task]
)
# Execute
result = crew.kickoff()
```
## Advanced Usage
### Dynamic File Selection
```python
# Initialize without file path
flexible_reader = FileReadTool()
# Read different files
config_content = flexible_reader.run(
file_path="/path/to/config.txt"
)
log_content = flexible_reader.run(
file_path="/path/to/logs.txt"
)
```
### Multiple File Processing
```python
# Create tools for different files
config_reader = FileReadTool(file_path="/path/to/config.txt")
log_reader = FileReadTool(file_path="/path/to/logs.txt")
# Create agent with multiple tools
processor = Agent(
role='File Analyst',
goal='Analyze multiple file types',
tools=[config_reader, log_reader]
)
```
### Error Handling Example
```python
try:
reader = FileReadTool()
content = reader.run(
file_path="/path/to/file.txt"
)
print(content)
except Exception as e:
print(f"Error reading file: {str(e)}")
```
## Notes
- Inherits from BaseTool
- Supports fixed or dynamic file paths
- Comprehensive error handling
- Thread-safe operations
- Clear error messages
- Flexible path specification
- Maintains tool description
- Handles common file errors
- Supports various file types
- Memory-efficient operations

View File

@@ -1,141 +0,0 @@
---
title: FileWriterTool
description: A tool for writing content to files with support for various file formats.
icon: file-pen
---
## FileWriterTool
The FileWriterTool provides agents with the capability to write content to files, supporting various file formats and ensuring proper file handling.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import FileWriterTool
# Initialize the tool
file_writer = FileWriterTool()
# Create an agent with the tool
writer_agent = Agent(
role='Content Writer',
goal='Write and save content to files',
backstory='Expert at creating and managing file content.',
tools=[file_writer],
verbose=True
)
# Use in a task
task = Task(
description='Write a report and save it to report.txt',
agent=writer_agent
)
```
## Tool Attributes
| Attribute | Type | Description |
| :-------- | :--- | :---------- |
| name | str | "File Writer Tool" |
| description | str | "A tool that writes content to a file." |
## Input Schema
```python
class FileWriterToolInput(BaseModel):
filename: str # Name of the file to write
directory: str = "./" # Optional directory path, defaults to current directory
overwrite: str = "False" # Whether to overwrite existing file ("True"/"False")
content: str # Content to write to the file
```
## Function Signature
```python
def _run(self, **kwargs: Any) -> str:
"""
Write content to a file with specified parameters.
Args:
filename (str): Name of the file to write
content (str): Content to write to the file
directory (str, optional): Directory path. Defaults to "./".
overwrite (str, optional): Whether to overwrite existing file. Defaults to "False".
Returns:
str: Success message with filepath or error message
"""
```
## Error Handling
The tool includes error handling for common file operations:
- FileExistsError: When file exists and overwrite is not allowed
- KeyError: When required parameters are missing
- Directory Creation: Automatically creates directories if they don't exist
- General Exceptions: Catches and reports any other file operation errors
## Best Practices
1. Always provide absolute file paths
2. Ensure proper file permissions
3. Handle potential errors in your agent prompts
4. Verify file contents after writing
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import FileWriterTool
# Initialize tool
file_writer = FileWriterTool()
# Create agent
writer = Agent(
role='Technical Writer',
goal='Create and save technical documentation',
backstory='Expert technical writer with experience in documentation.',
tools=[file_writer]
)
# Define task
writing_task = Task(
description="""Write a technical guide about Python best practices and save it
to the docs directory. The file should be named 'python_guide.md'.
Include sections on code style, documentation, and testing.
If a file already exists, overwrite it.""",
agent=writer
)
# The agent can use the tool with these parameters:
# {
# "filename": "python_guide.md",
# "directory": "docs",
# "overwrite": "True",
# "content": "# Python Best Practices\n\n## Code Style\n..."
# }
# Create crew
crew = Crew(
agents=[writer],
tasks=[writing_task]
)
# Execute
result = crew.kickoff()
```
## Notes
- The tool automatically creates directories in the file path if they don't exist
- Supports various file formats (txt, md, json, etc.)
- Returns descriptive error messages for better debugging
- Thread-safe file operations

View File

@@ -1,181 +0,0 @@
---
title: FirecrawlCrawlWebsiteTool
description: A web crawling tool powered by Firecrawl API for comprehensive website content extraction
icon: spider-web
---
## FirecrawlCrawlWebsiteTool
The FirecrawlCrawlWebsiteTool provides website crawling capabilities using the Firecrawl API. It allows for customizable crawling with options for polling intervals, idempotency, and URL parameters.
## Installation
```bash
pip install 'crewai[tools]'
pip install firecrawl-py # Required dependency
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import FirecrawlCrawlWebsiteTool
# Method 1: Using environment variable
# export FIRECRAWL_API_KEY='your-api-key'
crawler = FirecrawlCrawlWebsiteTool()
# Method 2: Providing API key directly
crawler = FirecrawlCrawlWebsiteTool(
api_key="your-firecrawl-api-key"
)
# Method 3: With custom configuration
crawler = FirecrawlCrawlWebsiteTool(
api_key="your-firecrawl-api-key",
url="https://example.com", # Base URL
poll_interval=5, # Custom polling interval
idempotency_key="unique-key"
)
# Create an agent with the tool
researcher = Agent(
role='Web Crawler',
goal='Extract and analyze website content',
backstory='Expert at crawling and analyzing web content.',
tools=[crawler],
verbose=True
)
```
## Input Schema
```python
class FirecrawlCrawlWebsiteToolSchema(BaseModel):
url: str = Field(description="Website URL")
```
## Function Signature
```python
def __init__(
self,
api_key: Optional[str] = None,
url: Optional[str] = None,
params: Optional[Dict[str, Any]] = None,
poll_interval: Optional[int] = 2,
idempotency_key: Optional[str] = None,
**kwargs
):
"""
Initialize the website crawling tool.
Args:
api_key (Optional[str]): Firecrawl API key. If not provided, checks FIRECRAWL_API_KEY env var
url (Optional[str]): Base URL to crawl. Can be overridden in _run
params (Optional[Dict[str, Any]]): Additional parameters for FirecrawlApp
poll_interval (Optional[int]): Poll interval for FirecrawlApp
idempotency_key (Optional[str]): Idempotency key for FirecrawlApp
**kwargs: Additional arguments for tool creation
"""
def _run(self, url: str) -> Any:
"""
Crawl a website using Firecrawl.
Args:
url (str): Website URL to crawl (overrides constructor URL if provided)
Returns:
Any: Crawled website content from Firecrawl API
"""
```
## Best Practices
1. Set up API authentication:
- Use environment variable: `export FIRECRAWL_API_KEY='your-api-key'`
- Or provide directly in constructor
2. Configure crawling parameters:
- Set appropriate poll intervals
- Use idempotency keys for retry safety
- Customize URL parameters as needed
3. Handle rate limits and quotas
4. Consider website robots.txt policies
5. Handle potential crawling errors in agent prompts
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import FirecrawlCrawlWebsiteTool
# Initialize crawler with configuration
crawler = FirecrawlCrawlWebsiteTool(
api_key="your-firecrawl-api-key",
poll_interval=5,
params={
"max_depth": 3,
"follow_links": True
}
)
# Create agent
web_analyst = Agent(
role='Web Content Analyst',
goal='Extract and analyze website content comprehensively',
backstory='Expert at web crawling and content analysis.',
tools=[crawler]
)
# Define task
crawl_task = Task(
description="""Crawl the documentation website at docs.example.com
and extract all API-related content.""",
agent=web_analyst
)
# The agent will use:
# {
# "url": "https://docs.example.com"
# }
# Create crew
crew = Crew(
agents=[web_analyst],
tasks=[crawl_task]
)
# Execute
result = crew.kickoff()
```
## Configuration Options
### URL Parameters
```python
params = {
"max_depth": 3, # Maximum crawl depth
"follow_links": True, # Follow internal links
"exclude_patterns": [], # URL patterns to exclude
"include_patterns": [] # URL patterns to include
}
```
### Polling Configuration
```python
crawler = FirecrawlCrawlWebsiteTool(
poll_interval=5, # Poll every 5 seconds
idempotency_key="unique-key-123" # For retry safety
)
```
## Notes
- Requires valid Firecrawl API key
- Supports both environment variable and direct API key configuration
- Configurable polling intervals for crawl status
- Idempotency support for safe retries
- Thread-safe operations
- Customizable crawling parameters
- Respects robots.txt by default

View File

@@ -1,154 +0,0 @@
---
title: FirecrawlSearchTool
description: A web search tool powered by Firecrawl API for comprehensive web search capabilities
icon: magnifying-glass-chart
---
## FirecrawlSearchTool
The FirecrawlSearchTool provides web search capabilities using the Firecrawl API. It allows for customizable search queries with options for result formatting and search parameters.
## Installation
```bash
pip install 'crewai[tools]'
pip install firecrawl-py # Required dependency
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import FirecrawlSearchTool
# Initialize the tool with your API key
search_tool = FirecrawlSearchTool(api_key="your-firecrawl-api-key")
# Create an agent with the tool
researcher = Agent(
role='Web Researcher',
goal='Find relevant information across the web',
backstory='Expert at web research and information gathering.',
tools=[search_tool],
verbose=True
)
```
## Input Schema
```python
class FirecrawlSearchToolSchema(BaseModel):
query: str = Field(description="Search query")
page_options: Optional[Dict[str, Any]] = Field(
default=None,
description="Options for result formatting"
)
search_options: Optional[Dict[str, Any]] = Field(
default=None,
description="Options for searching"
)
```
## Function Signature
```python
def __init__(self, api_key: Optional[str] = None, **kwargs):
"""
Initialize the Firecrawl search tool.
Args:
api_key (Optional[str]): Firecrawl API key
**kwargs: Additional arguments for tool creation
"""
def _run(
self,
query: str,
page_options: Optional[Dict[str, Any]] = None,
result_options: Optional[Dict[str, Any]] = None,
) -> Any:
"""
Perform a web search using Firecrawl.
Args:
query (str): Search query string
page_options (Optional[Dict[str, Any]]): Options for result formatting
result_options (Optional[Dict[str, Any]]): Options for search results
Returns:
Any: Search results from Firecrawl API
"""
```
## Best Practices
1. Always provide a valid API key
2. Use specific, focused search queries
3. Customize page and result options for better results
4. Handle potential API errors in agent prompts
5. Consider rate limits and usage quotas
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import FirecrawlSearchTool
# Initialize tool with API key
search_tool = FirecrawlSearchTool(api_key="your-firecrawl-api-key")
# Create agent
researcher = Agent(
role='Market Researcher',
goal='Research market trends and competitor analysis',
backstory='Expert market analyst with deep research skills.',
tools=[search_tool]
)
# Define task
research_task = Task(
description="""Research the latest developments in electric vehicles,
focusing on market leaders and emerging technologies. Format the results
in a structured way.""",
agent=researcher
)
# The agent will use:
# {
# "query": "electric vehicle market leaders emerging technologies",
# "page_options": {
# "format": "structured",
# "maxLength": 1000
# },
# "result_options": {
# "limit": 5,
# "sortBy": "relevance"
# }
# }
# Create crew
crew = Crew(
agents=[researcher],
tasks=[research_task]
)
# Execute
result = crew.kickoff()
```
## Error Handling
The tool includes error handling for:
- Missing API key
- Missing firecrawl-py package
- API request failures
- Invalid options parameters
## Notes
- Requires valid Firecrawl API key
- Supports customizable search parameters
- Provides structured web search results
- Thread-safe operations
- Efficient for large-scale web searches
- Handles rate limiting automatically

View File

@@ -1,233 +0,0 @@
---
title: GithubSearchTool
description: A tool for semantic search within GitHub repositories using RAG capabilities
icon: github
---
## GithubSearchTool
The GithubSearchTool enables semantic search capabilities for GitHub repositories using Retrieval-Augmented Generation (RAG). It processes various content types including code, repository information, pull requests, and issues, allowing natural language queries across repository content.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import GithubSearchTool
# Method 1: Initialize with specific repository
github_tool = GithubSearchTool(
github_repo="owner/repo",
gh_token="your_github_token",
content_types=["code", "pr", "issue"]
)
# Method 2: Initialize without repository (specify at runtime)
flexible_github_tool = GithubSearchTool(
gh_token="your_github_token",
content_types=["code", "repo"]
)
# Create an agent with the tool
researcher = Agent(
role='GitHub Researcher',
goal='Search and analyze repository contents',
backstory='Expert at finding relevant information in GitHub repositories.',
tools=[github_tool],
verbose=True
)
```
## Input Schema
### Fixed Repository Schema (when repo provided during initialization)
```python
class FixedGithubSearchToolSchema(BaseModel):
search_query: str = Field(
description="Mandatory search query you want to use to search the github repo's content"
)
```
### Flexible Repository Schema (when repo provided at runtime)
```python
class GithubSearchToolSchema(FixedGithubSearchToolSchema):
github_repo: str = Field(
description="Mandatory github you want to search"
)
content_types: List[str] = Field(
description="Mandatory content types you want to be included search, options: [code, repo, pr, issue]"
)
```
## Function Signature
```python
def __init__(
self,
github_repo: Optional[str] = None,
gh_token: str,
content_types: List[str],
**kwargs
):
"""
Initialize the GitHub search tool.
Args:
github_repo (Optional[str]): Repository to search (optional)
gh_token (str): GitHub authentication token
content_types (List[str]): Content types to search
**kwargs: Additional arguments for RAG tool configuration
"""
def _run(
self,
search_query: str,
**kwargs: Any
) -> str:
"""
Execute semantic search on repository contents.
Args:
search_query (str): Query to search in the repository
**kwargs: Additional arguments including github_repo and content_types if not initialized
Returns:
str: Relevant content from the repository matching the query
"""
```
## Best Practices
1. Authentication:
- Secure token management
- Use environment variables
- Handle token expiration
2. Search Optimization:
- Target specific content types
- Use focused queries
- Consider rate limits
3. Performance Considerations:
- Pre-initialize for repeated searches
- Handle large repositories
- Monitor API usage
4. Error Handling:
- Verify repository access
- Handle API limits
- Manage authentication errors
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import GithubSearchTool
# Initialize tool with specific repository
github_tool = GithubSearchTool(
github_repo="owner/repo",
gh_token="your_github_token",
content_types=["code", "pr", "issue"]
)
# Create agent
researcher = Agent(
role='GitHub Researcher',
goal='Extract insights from repository content',
backstory='Expert at analyzing GitHub repositories.',
tools=[github_tool]
)
# Define task
research_task = Task(
description="""Find all implementations of
machine learning algorithms in the codebase.""",
agent=researcher
)
# The tool will use:
# {
# "search_query": "machine learning implementation"
# }
# Create crew
crew = Crew(
agents=[researcher],
tasks=[research_task]
)
# Execute
result = crew.kickoff()
```
## Advanced Usage
### Dynamic Repository Selection
```python
# Initialize without repository
flexible_tool = GithubSearchTool(
gh_token="your_github_token",
content_types=["code", "repo"]
)
# Search different repositories
backend_results = flexible_tool.run(
search_query="authentication implementation",
github_repo="owner/backend-repo"
)
frontend_results = flexible_tool.run(
search_query="component architecture",
github_repo="owner/frontend-repo"
)
```
### Multiple Content Type Analysis
```python
# Create tool with multiple content types
multi_tool = GithubSearchTool(
github_repo="owner/repo",
gh_token="your_github_token",
content_types=["code", "pr", "issue", "repo"]
)
# Search across all content types
results = multi_tool.run(
search_query="feature implementation status"
)
```
### Error Handling Example
```python
try:
github_tool = GithubSearchTool(
gh_token="your_github_token",
content_types=["code"]
)
results = github_tool.run(
search_query="api endpoints",
github_repo="owner/repo"
)
print(results)
except Exception as e:
print(f"Error searching repository: {str(e)}")
```
## Notes
- Inherits from RagTool
- Uses GithubLoader
- Requires authentication
- Supports multiple content types
- Dynamic repository specification
- Efficient content retrieval
- Thread-safe operations
- Maintains search context
- Handles API rate limits
- Memory-efficient processing

View File

@@ -34,7 +34,6 @@ from crewai_tools import GithubSearchTool
# Initialize the tool for semantic searches within a specific GitHub repository
tool = GithubSearchTool(
github_repo='https://github.com/example/repo',
gh_token='your_github_personal_access_token',
content_types=['code', 'issue'] # Options: code, repo, pr, issue
)
@@ -42,7 +41,6 @@ tool = GithubSearchTool(
# Initialize the tool for semantic searches within a specific GitHub repository, so the agent can search any repository if it learns about during its execution
tool = GithubSearchTool(
gh_token='your_github_personal_access_token',
content_types=['code', 'issue'] # Options: code, repo, pr, issue
)
```
@@ -50,7 +48,6 @@ tool = GithubSearchTool(
## Arguments
- `github_repo` : The URL of the GitHub repository where the search will be conducted. This is a mandatory field and specifies the target repository for your search.
- `gh_token` : Your GitHub Personal Access Token (PAT) required for authentication. You can create one in your GitHub account settings under Developer Settings > Personal Access Tokens.
- `content_types` : Specifies the types of content to include in your search. You must provide a list of content types from the following options: `code` for searching within the code,
`repo` for searching within the repository's general information, `pr` for searching within pull requests, and `issue` for searching within issues.
This field is mandatory and allows tailoring the search to specific content types within the GitHub repository.
@@ -80,4 +77,5 @@ tool = GithubSearchTool(
),
),
)
)
)
```

View File

@@ -1,220 +0,0 @@
---
title: JinaScrapeWebsiteTool
description: A tool for scraping website content using Jina.ai's reader service with markdown output
icon: globe
---
## JinaScrapeWebsiteTool
The JinaScrapeWebsiteTool provides website content scraping capabilities using Jina.ai's reader service. It converts web content into clean markdown format and supports both fixed and dynamic URL modes with optional authentication.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import JinaScrapeWebsiteTool
# Method 1: Fixed URL (specified at initialization)
fixed_tool = JinaScrapeWebsiteTool(
website_url="https://example.com",
api_key="your-jina-api-key" # Optional
)
# Method 2: Dynamic URL (specified at runtime)
dynamic_tool = JinaScrapeWebsiteTool(
api_key="your-jina-api-key" # Optional
)
# Create an agent with the tool
researcher = Agent(
role='Web Content Researcher',
goal='Extract and analyze website content',
backstory='Expert at gathering and processing web information.',
tools=[fixed_tool], # or [dynamic_tool]
verbose=True
)
```
## Input Schema
```python
class JinaScrapeWebsiteToolInput(BaseModel):
website_url: str = Field(
description="Mandatory website url to read the file"
)
```
## Function Signature
```python
def __init__(
self,
website_url: Optional[str] = None,
api_key: Optional[str] = None,
custom_headers: Optional[dict] = None,
**kwargs
):
"""
Initialize the website scraping tool.
Args:
website_url (Optional[str]): URL to scrape (optional for dynamic mode)
api_key (Optional[str]): Jina.ai API key for authentication
custom_headers (Optional[dict]): Custom HTTP headers
**kwargs: Additional arguments for base tool
"""
def _run(
self,
website_url: Optional[str] = None
) -> str:
"""
Execute website scraping.
Args:
website_url (Optional[str]): URL to scrape (required for dynamic mode)
Returns:
str: Markdown-formatted website content
"""
```
## Best Practices
1. URL Handling:
- Use complete URLs
- Validate URL format
- Handle redirects
- Monitor timeouts
2. Authentication:
- Secure API key storage
- Use environment variables
- Manage headers properly
- Handle auth errors
3. Content Processing:
- Handle large pages
- Process markdown output
- Manage encoding
- Handle errors
4. Mode Selection:
- Choose fixed mode for static sites
- Use dynamic mode for variable URLs
- Consider caching
- Manage timeouts
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import JinaScrapeWebsiteTool
import os
# Initialize tool with API key
scraper_tool = JinaScrapeWebsiteTool(
api_key=os.getenv('JINA_API_KEY'),
custom_headers={
'User-Agent': 'CrewAI Bot 1.0'
}
)
# Create agent
researcher = Agent(
role='Web Content Analyst',
goal='Extract and analyze website content',
backstory='Expert at processing web information.',
tools=[scraper_tool]
)
# Define task
analysis_task = Task(
description="""Analyze the content of
https://example.com/blog for key insights.""",
agent=researcher
)
# Create crew
crew = Crew(
agents=[researcher],
tasks=[analysis_task]
)
# Execute
result = crew.kickoff()
```
## Advanced Usage
### Multiple Site Analysis
```python
# Initialize tool
scraper = JinaScrapeWebsiteTool(
api_key=os.getenv('JINA_API_KEY')
)
# Analyze multiple sites
results = []
sites = [
"https://site1.com",
"https://site2.com",
"https://site3.com"
]
for site in sites:
content = scraper.run(
website_url=site
)
results.append(content)
```
### Custom Headers Configuration
```python
# Initialize with custom headers
tool = JinaScrapeWebsiteTool(
custom_headers={
'User-Agent': 'Custom Bot 1.0',
'Accept-Language': 'en-US,en;q=0.9',
'Accept': 'text/html,application/xhtml+xml'
}
)
# Use the tool
content = tool.run(
website_url="https://example.com"
)
```
### Error Handling Example
```python
try:
scraper = JinaScrapeWebsiteTool()
content = scraper.run(
website_url="https://example.com"
)
print(content)
except requests.exceptions.RequestException as e:
print(f"Error accessing website: {str(e)}")
except Exception as e:
print(f"Error processing content: {str(e)}")
```
## Notes
- Uses Jina.ai reader service
- Markdown output format
- API key authentication
- Custom headers support
- Error handling
- Timeout management
- Content processing
- URL validation
- Redirect handling
- Response formatting

View File

@@ -1,224 +0,0 @@
---
title: JSONSearchTool
description: A tool for semantic search within JSON files using RAG capabilities
icon: braces
---
## JSONSearchTool
The JSONSearchTool enables semantic search capabilities for JSON files using Retrieval-Augmented Generation (RAG). It supports both fixed and dynamic file path modes, allowing flexible usage patterns.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import JSONSearchTool
# Method 1: Fixed path (specified at initialization)
fixed_tool = JSONSearchTool(
json_path="path/to/data.json"
)
# Method 2: Dynamic path (specified at runtime)
dynamic_tool = JSONSearchTool()
# Create an agent with the tool
researcher = Agent(
role='JSON Data Researcher',
goal='Search and analyze JSON data',
backstory='Expert at finding relevant information in JSON files.',
tools=[fixed_tool], # or [dynamic_tool]
verbose=True
)
```
## Input Schema
### Fixed Path Mode
```python
class FixedJSONSearchToolSchema(BaseModel):
search_query: str = Field(
description="Mandatory search query you want to use to search the JSON's content"
)
```
### Dynamic Path Mode
```python
class JSONSearchToolSchema(BaseModel):
json_path: str = Field(
description="Mandatory json path you want to search"
)
search_query: str = Field(
description="Mandatory search query you want to use to search the JSON's content"
)
```
## Function Signature
```python
def __init__(
self,
json_path: Optional[str] = None,
**kwargs
):
"""
Initialize the JSON search tool.
Args:
json_path (Optional[str]): Path to JSON file (optional for dynamic mode)
**kwargs: Additional arguments for RAG tool configuration
"""
def _run(
self,
search_query: str,
**kwargs: Any
) -> str:
"""
Execute semantic search on JSON contents.
Args:
search_query (str): Query to search in the JSON
**kwargs: Additional arguments
Returns:
str: Relevant content from the JSON matching the query
"""
```
## Best Practices
1. File Handling:
- Use absolute file paths
- Verify file existence
- Handle large JSON files
- Monitor memory usage
2. Query Optimization:
- Structure queries clearly
- Consider JSON structure
- Handle nested data
- Monitor performance
3. Error Handling:
- Check file access
- Validate JSON format
- Handle malformed JSON
- Log issues
4. Mode Selection:
- Choose fixed mode for static files
- Use dynamic mode for runtime selection
- Consider caching
- Manage file lifecycle
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import JSONSearchTool
# Initialize tool
json_tool = JSONSearchTool(
json_path="data/config.json"
)
# Create agent
researcher = Agent(
role='JSON Data Analyst',
goal='Extract insights from JSON configuration',
backstory='Expert at analyzing JSON data structures.',
tools=[json_tool]
)
# Define task
analysis_task = Task(
description="""Find all configuration settings
related to security.""",
agent=researcher
)
# Create crew
crew = Crew(
agents=[researcher],
tasks=[analysis_task]
)
# Execute
result = crew.kickoff()
```
## Advanced Usage
### Multiple File Analysis
```python
# Create tools for different JSON files
config_tool = JSONSearchTool(
json_path="config/settings.json"
)
data_tool = JSONSearchTool(
json_path="data/records.json"
)
# Create agent with multiple tools
analyst = Agent(
role='JSON Data Analyst',
goal='Cross-reference configuration and data',
tools=[config_tool, data_tool]
)
```
### Dynamic File Loading
```python
# Initialize dynamic tool
dynamic_tool = JSONSearchTool()
# Use with different JSON files
result1 = dynamic_tool.run(
json_path="file1.json",
search_query="security settings"
)
result2 = dynamic_tool.run(
json_path="file2.json",
search_query="user preferences"
)
```
### Error Handling Example
```python
try:
json_tool = JSONSearchTool(
json_path="config/settings.json"
)
results = json_tool.run(
search_query="encryption settings"
)
print(results)
except FileNotFoundError as e:
print(f"JSON file not found: {str(e)}")
except ValueError as e:
print(f"Invalid JSON format: {str(e)}")
except Exception as e:
print(f"Error processing JSON: {str(e)}")
```
## Notes
- Inherits from RagTool
- Supports fixed/dynamic modes
- JSON path validation
- Memory management
- Performance optimization
- Error handling
- Search capabilities
- Content extraction
- Format validation
- Security features

View File

@@ -1,184 +0,0 @@
---
title: LinkupSearchTool
description: A search tool powered by Linkup API for retrieving contextual information
icon: search
---
## LinkupSearchTool
The LinkupSearchTool provides search capabilities using the Linkup API. It allows for customizable search depth and output formatting, returning structured results with contextual information.
## Installation
```bash
pip install 'crewai[tools]'
pip install linkup # Required dependency
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import LinkupSearchTool
# Initialize the tool with your API key
search_tool = LinkupSearchTool(api_key="your-linkup-api-key")
# Create an agent with the tool
researcher = Agent(
role='Information Researcher',
goal='Find relevant contextual information',
backstory='Expert at retrieving and analyzing contextual data.',
tools=[search_tool],
verbose=True
)
```
## Function Signature
```python
def __init__(self, api_key: str):
"""
Initialize the Linkup search tool.
Args:
api_key (str): Linkup API key for authentication
"""
def _run(
self,
query: str,
depth: str = "standard",
output_type: str = "searchResults"
) -> dict:
"""
Perform a search using the Linkup API.
Args:
query (str): The search query
depth (str): Search depth ("standard" by default)
output_type (str): Desired result type ("searchResults" by default)
Returns:
dict: {
"success": bool,
"results": List[Dict] | None,
"error": str | None
}
On success, results contains list of:
{
"name": str,
"url": str,
"content": str
}
"""
```
## Best Practices
1. Always provide a valid API key
2. Use specific, focused search queries
3. Choose appropriate search depth based on needs
4. Handle potential API errors in agent prompts
5. Process structured results effectively
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import LinkupSearchTool
# Initialize tool with API key
search_tool = LinkupSearchTool(api_key="your-linkup-api-key")
# Create agent
researcher = Agent(
role='Context Researcher',
goal='Find detailed contextual information about topics',
backstory='Expert at discovering and analyzing contextual data.',
tools=[search_tool]
)
# Define task
research_task = Task(
description="""Research the latest developments in quantum computing,
focusing on recent breakthroughs and applications. Use standard depth
for comprehensive results.""",
agent=researcher
)
# The tool will use:
# query: "quantum computing recent breakthroughs applications"
# depth: "standard"
# output_type: "searchResults"
# Create crew
crew = Crew(
agents=[researcher],
tasks=[research_task]
)
# Execute
result = crew.kickoff()
```
## Advanced Usage
### Search Depth Options
```python
# Quick surface-level search
results = search_tool._run(
query="quantum computing",
depth="basic"
)
# Standard comprehensive search
results = search_tool._run(
query="quantum computing",
depth="standard"
)
# Deep detailed search
results = search_tool._run(
query="quantum computing",
depth="deep"
)
```
### Output Type Options
```python
# Default search results
results = search_tool._run(
query="quantum computing",
output_type="searchResults"
)
# Custom output format
results = search_tool._run(
query="quantum computing",
output_type="customFormat"
)
```
### Error Handling
```python
results = search_tool._run(query="quantum computing")
if results["success"]:
for result in results["results"]:
print(f"Name: {result['name']}")
print(f"URL: {result['url']}")
print(f"Content: {result['content']}")
else:
print(f"Error: {results['error']}")
```
## Notes
- Requires valid Linkup API key
- Returns structured search results
- Supports multiple search depths
- Configurable output formats
- Built-in error handling
- Thread-safe operations
- Efficient for contextual searches

View File

@@ -1,192 +0,0 @@
---
title: LlamaIndexTool
description: A wrapper tool for integrating LlamaIndex tools and query engines with CrewAI
icon: link
---
## LlamaIndexTool
The LlamaIndexTool serves as a bridge between CrewAI and LlamaIndex, allowing you to use LlamaIndex tools and query engines within your CrewAI agents. It supports both direct tool wrapping and query engine integration.
## Installation
```bash
pip install 'crewai[tools]'
pip install llama-index # Required for LlamaIndex integration
```
## Usage Examples
### Using with LlamaIndex Tools
```python
from crewai import Agent
from crewai_tools import LlamaIndexTool
from llama_index.core.tools import BaseTool as LlamaBaseTool
from pydantic import BaseModel, Field
# Create a LlamaIndex tool
class CustomLlamaSchema(BaseModel):
query: str = Field(..., description="Query to process")
class CustomLlamaTool(LlamaBaseTool):
name = "Custom Llama Tool"
description = "A custom LlamaIndex tool"
def __call__(self, query: str) -> str:
return f"Processed: {query}"
# Wrap the LlamaIndex tool
llama_tool = CustomLlamaTool()
wrapped_tool = LlamaIndexTool.from_tool(llama_tool)
# Create an agent with the tool
agent = Agent(
role='LlamaIndex Integration Agent',
goal='Process queries using LlamaIndex tools',
backstory='Specialist in integrating LlamaIndex capabilities.',
tools=[wrapped_tool]
)
```
### Using with Query Engines
```python
from crewai import Agent
from crewai_tools import LlamaIndexTool
from llama_index.core import VectorStoreIndex, Document
# Create a query engine
documents = [Document(text="Sample document content")]
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
# Create the tool
query_tool = LlamaIndexTool.from_query_engine(
query_engine,
name="Document Search",
description="Search through indexed documents"
)
# Create an agent with the tool
agent = Agent(
role='Document Researcher',
goal='Find relevant information in documents',
backstory='Expert at searching through document collections.',
tools=[query_tool]
)
```
## Tool Creation Methods
### From LlamaIndex Tool
```python
@classmethod
def from_tool(cls, tool: Any, **kwargs: Any) -> "LlamaIndexTool":
"""
Create a CrewAI tool from a LlamaIndex tool.
Args:
tool (LlamaBaseTool): A LlamaIndex tool to wrap
**kwargs: Additional arguments for tool creation
Returns:
LlamaIndexTool: A CrewAI-compatible tool wrapper
Raises:
ValueError: If tool is not a LlamaBaseTool or lacks fn_schema
"""
```
### From Query Engine
```python
@classmethod
def from_query_engine(
cls,
query_engine: Any,
name: Optional[str] = None,
description: Optional[str] = None,
return_direct: bool = False,
**kwargs: Any
) -> "LlamaIndexTool":
"""
Create a CrewAI tool from a LlamaIndex query engine.
Args:
query_engine (BaseQueryEngine): The query engine to wrap
name (Optional[str]): Custom name for the tool
description (Optional[str]): Custom description
return_direct (bool): Whether to return query engine response directly
**kwargs: Additional arguments for tool creation
Returns:
LlamaIndexTool: A CrewAI-compatible tool wrapper
Raises:
ValueError: If query_engine is not a BaseQueryEngine
"""
```
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import LlamaIndexTool
from llama_index.core import VectorStoreIndex, Document
from llama_index.core.tools import QueryEngineTool
# Create documents and index
documents = [
Document(text="AI is a technology that simulates human intelligence."),
Document(text="Machine learning is a subset of AI.")
]
index = VectorStoreIndex.from_documents(documents)
query_engine = index.as_query_engine()
# Create the tool
search_tool = LlamaIndexTool.from_query_engine(
query_engine,
name="AI Knowledge Base",
description="Search through AI-related documents"
)
# Create agent
researcher = Agent(
role='AI Researcher',
goal='Research AI concepts',
backstory='Expert at finding and explaining AI concepts.',
tools=[search_tool]
)
# Define task
research_task = Task(
description="""Find and explain what AI is and its relationship
with machine learning.""",
agent=researcher
)
# The agent will use:
# {
# "query": "What is AI and how does it relate to machine learning?"
# }
# Create crew
crew = Crew(
agents=[researcher],
tasks=[research_task]
)
# Execute
result = crew.kickoff()
```
## Notes
- Automatically adapts LlamaIndex tool schemas for CrewAI compatibility
- Renames 'input' parameter to 'query' for better integration
- Supports both direct tool wrapping and query engine integration
- Handles schema validation and error resolution
- Thread-safe operations
- Compatible with all LlamaIndex tool types and query engines

View File

@@ -1,209 +0,0 @@
---
title: MDX Search Tool
description: A tool for semantic searching within MDX files using RAG capabilities
---
# MDX Search Tool
The MDX Search Tool enables semantic searching within MDX (Markdown with JSX) files using Retrieval-Augmented Generation (RAG) capabilities. It supports both fixed and dynamic file path modes, allowing you to specify the MDX file at initialization or runtime.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage
You can use the MDX Search Tool in two ways:
### 1. Fixed MDX File Path
Initialize the tool with a specific MDX file path:
```python
from crewai import Agent
from crewai_tools import MDXSearchTool
# Initialize with a fixed MDX file
tool = MDXSearchTool(mdx="/path/to/your/document.mdx")
# Create an agent with the tool
agent = Agent(
role='Technical Writer',
goal='Search through MDX documentation',
backstory='I help find relevant information in MDX documentation',
tools=[tool]
)
# Use in a task
task = Task(
description="Find information about API endpoints in the documentation",
agent=agent
)
```
### 2. Dynamic MDX File Path
Initialize the tool without a specific file path to provide it at runtime:
```python
from crewai import Agent
from crewai_tools import MDXSearchTool
# Initialize without a fixed MDX file
tool = MDXSearchTool()
# Create an agent with the tool
agent = Agent(
role='Documentation Analyst',
goal='Search through various MDX files',
backstory='I analyze different MDX documentation files',
tools=[tool]
)
# Use in a task with dynamic file path
task = Task(
description="Search for 'authentication' in the API documentation",
agent=agent,
context={
"mdx": "/path/to/api-docs.mdx",
"search_query": "authentication"
}
)
```
## Input Schema
### Fixed MDX File Mode
```python
class FixedMDXSearchToolSchema(BaseModel):
search_query: str # The search query to find content in the MDX file
```
### Dynamic MDX File Mode
```python
class MDXSearchToolSchema(BaseModel):
search_query: str # The search query to find content in the MDX file
mdx: str # The path to the MDX file to search
```
## Function Signatures
```python
def __init__(self, mdx: Optional[str] = None, **kwargs):
"""
Initialize the MDX Search Tool.
Args:
mdx (Optional[str]): Path to the MDX file (optional)
**kwargs: Additional arguments passed to RagTool
"""
def _run(
self,
search_query: str,
**kwargs: Any,
) -> str:
"""
Execute the search on the MDX file.
Args:
search_query (str): The query to search for
**kwargs: Additional arguments including 'mdx' for dynamic mode
Returns:
str: The search results from the MDX content
"""
```
## Best Practices
1. **File Path Handling**:
- Use absolute paths to avoid path resolution issues
- Verify file existence before searching
- Handle file permissions appropriately
2. **Query Optimization**:
- Use specific, focused search queries
- Consider context when formulating queries
- Break down complex searches into smaller queries
3. **Error Handling**:
- Handle file not found errors gracefully
- Manage permission issues appropriately
- Validate input parameters before processing
## Example Integration
Here's a complete example showing how to integrate the MDX Search Tool with CrewAI:
```python
from crewai import Agent, Task, Crew
from crewai_tools import MDXSearchTool
# Initialize the tool
mdx_tool = MDXSearchTool()
# Create an agent with the tool
researcher = Agent(
role='Documentation Researcher',
goal='Find and analyze information in MDX documentation',
backstory='I am an expert at finding relevant information in documentation',
tools=[mdx_tool]
)
# Create tasks
search_task = Task(
description="""
Search through the API documentation for information about authentication methods.
Look for:
1. Authentication endpoints
2. Security best practices
3. Token handling
Provide a comprehensive summary of the findings.
""",
agent=researcher,
context={
"mdx": "/path/to/api-docs.mdx",
"search_query": "authentication security tokens"
}
)
# Create and run the crew
crew = Crew(
agents=[researcher],
tasks=[search_task]
)
result = crew.kickoff()
```
## Error Handling
The tool handles various error scenarios:
1. **File Not Found**:
```python
try:
tool = MDXSearchTool(mdx="/path/to/nonexistent.mdx")
except FileNotFoundError:
print("MDX file not found. Please verify the file path.")
```
2. **Permission Issues**:
```python
try:
tool = MDXSearchTool(mdx="/restricted/docs.mdx")
except PermissionError:
print("Insufficient permissions to access the MDX file.")
```
3. **Invalid Content**:
```python
try:
result = tool._run(search_query="query", mdx="/path/to/invalid.mdx")
except ValueError:
print("Invalid MDX content or format.")
```

View File

@@ -1,217 +0,0 @@
---
title: MySQLSearchTool
description: A tool for semantic search within MySQL database tables using RAG capabilities
icon: database
---
## MySQLSearchTool
The MySQLSearchTool enables semantic search capabilities for MySQL database tables using Retrieval-Augmented Generation (RAG). It processes table contents and allows natural language queries to search through the data.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import MySQLSearchTool
# Initialize the tool
mysql_tool = MySQLSearchTool(
table_name="users",
db_uri="mysql://user:pass@localhost:3306/database"
)
# Create an agent with the tool
researcher = Agent(
role='Database Researcher',
goal='Search and analyze database contents',
backstory='Expert at finding relevant information in databases.',
tools=[mysql_tool],
verbose=True
)
```
## Input Schema
```python
class MySQLSearchToolSchema(BaseModel):
search_query: str = Field(
description="Mandatory semantic search query you want to use to search the database's content"
)
```
## Function Signature
```python
def __init__(
self,
table_name: str,
db_uri: str,
**kwargs
):
"""
Initialize the MySQL search tool.
Args:
table_name (str): Name of the table to search
db_uri (str): Database connection URI
**kwargs: Additional arguments for RAG tool configuration
"""
def _run(
self,
search_query: str,
**kwargs: Any
) -> str:
"""
Execute semantic search on table contents.
Args:
search_query (str): Query to search in the table
**kwargs: Additional arguments
Returns:
str: Relevant content from the table matching the query
"""
```
## Best Practices
1. Database Connection:
- Use secure connection URIs
- Handle authentication properly
- Manage connection lifecycle
- Monitor timeouts
2. Query Optimization:
- Structure queries clearly
- Consider table size
- Handle large datasets
- Monitor performance
3. Security Considerations:
- Protect credentials
- Use environment variables
- Limit table access
- Validate inputs
4. Error Handling:
- Handle connection errors
- Manage query timeouts
- Provide clear messages
- Log issues
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import MySQLSearchTool
# Initialize tool
mysql_tool = MySQLSearchTool(
table_name="customers",
db_uri="mysql://user:pass@localhost:3306/crm"
)
# Create agent
researcher = Agent(
role='Database Analyst',
goal='Extract customer insights from database',
backstory='Expert at analyzing customer data.',
tools=[mysql_tool]
)
# Define task
analysis_task = Task(
description="""Find all premium customers
with recent purchases.""",
agent=researcher
)
# The tool will use:
# {
# "search_query": "premium customers recent purchases"
# }
# Create crew
crew = Crew(
agents=[researcher],
tasks=[analysis_task]
)
# Execute
result = crew.kickoff()
```
## Advanced Usage
### Multiple Table Analysis
```python
# Create tools for different tables
customers_tool = MySQLSearchTool(
table_name="customers",
db_uri="mysql://user:pass@localhost:3306/crm"
)
orders_tool = MySQLSearchTool(
table_name="orders",
db_uri="mysql://user:pass@localhost:3306/crm"
)
# Create agent with multiple tools
analyst = Agent(
role='Data Analyst',
goal='Cross-reference customer and order data',
tools=[customers_tool, orders_tool]
)
```
### Secure Connection Configuration
```python
import os
# Use environment variables for credentials
db_uri = (
f"mysql://{os.getenv('DB_USER')}:{os.getenv('DB_PASS')}"
f"@{os.getenv('DB_HOST')}:{os.getenv('DB_PORT')}"
f"/{os.getenv('DB_NAME')}"
)
tool = MySQLSearchTool(
table_name="sensitive_data",
db_uri=db_uri
)
```
### Error Handling Example
```python
try:
mysql_tool = MySQLSearchTool(
table_name="users",
db_uri="mysql://user:pass@localhost:3306/app"
)
results = mysql_tool.run(
search_query="active users in California"
)
print(results)
except Exception as e:
print(f"Error querying database: {str(e)}")
```
## Notes
- Inherits from RagTool
- Uses MySQLLoader
- Requires database URI
- Table-specific search
- Semantic query support
- Connection management
- Error handling
- Performance optimization
- Security features
- Memory efficiency

View File

@@ -1,208 +0,0 @@
---
title: PDFSearchTool
description: A tool for semantic search within PDF documents using RAG capabilities
icon: file-search
---
## PDFSearchTool
The PDFSearchTool enables semantic search capabilities for PDF documents using Retrieval-Augmented Generation (RAG). It leverages embedchain's PDFEmbedchainAdapter for efficient PDF processing and supports both fixed and dynamic PDF path specification.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import PDFSearchTool
# Method 1: Initialize with specific PDF
pdf_tool = PDFSearchTool(pdf="/path/to/document.pdf")
# Method 2: Initialize without PDF (specify at runtime)
flexible_pdf_tool = PDFSearchTool()
# Create an agent with the tool
researcher = Agent(
role='PDF Researcher',
goal='Search and analyze PDF documents',
backstory='Expert at finding relevant information in PDFs.',
tools=[pdf_tool],
verbose=True
)
```
## Input Schema
### Fixed PDF Schema (when PDF path provided during initialization)
```python
class FixedPDFSearchToolSchema(BaseModel):
query: str = Field(
description="Mandatory query you want to use to search the PDF's content"
)
```
### Flexible PDF Schema (when PDF path provided at runtime)
```python
class PDFSearchToolSchema(FixedPDFSearchToolSchema):
pdf: str = Field(
description="Mandatory pdf path you want to search"
)
```
## Function Signature
```python
def __init__(
self,
pdf: Optional[str] = None,
**kwargs
):
"""
Initialize the PDF search tool.
Args:
pdf (Optional[str]): Path to PDF file (optional)
**kwargs: Additional arguments for RAG tool configuration
"""
def _run(
self,
query: str,
**kwargs: Any
) -> str:
"""
Execute semantic search on PDF content.
Args:
query (str): Search query for the PDF
**kwargs: Additional arguments including pdf path if not initialized
Returns:
str: Relevant content from the PDF matching the query
"""
```
## Best Practices
1. PDF File Handling:
- Use absolute paths for reliability
- Verify PDF file existence
- Handle large PDFs appropriately
2. Search Optimization:
- Use specific, focused queries
- Consider document structure
- Test with sample queries first
3. Performance Considerations:
- Pre-initialize with PDF for repeated searches
- Handle large documents efficiently
- Monitor memory usage
4. Error Handling:
- Verify PDF file existence
- Handle malformed PDFs
- Manage file access permissions
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import PDFSearchTool
# Initialize tool with specific PDF
pdf_tool = PDFSearchTool(pdf="/path/to/research.pdf")
# Create agent
researcher = Agent(
role='PDF Researcher',
goal='Extract insights from research papers',
backstory='Expert at analyzing research documents.',
tools=[pdf_tool]
)
# Define task
research_task = Task(
description="""Find all mentions of machine learning
applications in healthcare from the PDF.""",
agent=researcher
)
# The tool will use:
# {
# "query": "machine learning applications healthcare"
# }
# Create crew
crew = Crew(
agents=[researcher],
tasks=[research_task]
)
# Execute
result = crew.kickoff()
```
## Advanced Usage
### Dynamic PDF Selection
```python
# Initialize without PDF
flexible_tool = PDFSearchTool()
# Search different PDFs
research_results = flexible_tool.run(
query="quantum computing",
pdf="/path/to/research.pdf"
)
report_results = flexible_tool.run(
query="financial metrics",
pdf="/path/to/report.pdf"
)
```
### Multiple PDF Analysis
```python
# Create tools for different PDFs
research_tool = PDFSearchTool(pdf="/path/to/research.pdf")
report_tool = PDFSearchTool(pdf="/path/to/report.pdf")
# Create agent with multiple tools
analyst = Agent(
role='Document Analyst',
goal='Cross-reference multiple documents',
tools=[research_tool, report_tool]
)
```
### Error Handling Example
```python
try:
pdf_tool = PDFSearchTool()
results = pdf_tool.run(
query="important findings",
pdf="/path/to/document.pdf"
)
print(results)
except Exception as e:
print(f"Error processing PDF: {str(e)}")
```
## Notes
- Inherits from RagTool
- Uses PDFEmbedchainAdapter
- Supports semantic search
- Dynamic PDF specification
- Efficient content retrieval
- Thread-safe operations
- Maintains search context
- Handles large documents
- Supports various PDF formats
- Memory-efficient processing

View File

@@ -1,234 +0,0 @@
---
title: PDFTextWritingTool
description: A tool for adding text to specific positions in PDF documents with custom font support
icon: file-pdf
---
## PDFTextWritingTool
The PDFTextWritingTool allows you to add text to specific positions in PDF documents with support for custom fonts, colors, and positioning. It's particularly useful for adding annotations, watermarks, or any text overlay to existing PDFs.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import PDFTextWritingTool
# Basic initialization
pdf_tool = PDFTextWritingTool()
# Create an agent with the tool
document_processor = Agent(
role='Document Processor',
goal='Add text annotations to PDF documents',
backstory='Expert at PDF document processing and text manipulation.',
tools=[pdf_tool],
verbose=True
)
```
## Input Schema
```python
class PDFTextWritingToolSchema(BaseModel):
pdf_path: str = Field(
description="Path to the PDF file to modify"
)
text: str = Field(
description="Text to add to the PDF"
)
position: tuple = Field(
description="Tuple of (x, y) coordinates for text placement"
)
font_size: int = Field(
default=12,
description="Font size of the text"
)
font_color: str = Field(
default="0 0 0 rg",
description="RGB color code for the text"
)
font_name: Optional[str] = Field(
default="F1",
description="Font name for standard fonts"
)
font_file: Optional[str] = Field(
default=None,
description="Path to a .ttf font file for custom font usage"
)
page_number: int = Field(
default=0,
description="Page number to add text to"
)
```
## Function Signature
```python
def run(
self,
pdf_path: str,
text: str,
position: tuple,
font_size: int,
font_color: str,
font_name: str = "F1",
font_file: Optional[str] = None,
page_number: int = 0,
**kwargs
) -> str:
"""
Add text to a specific position in a PDF document.
Args:
pdf_path (str): Path to the PDF file to modify
text (str): Text to add to the PDF
position (tuple): (x, y) coordinates for text placement
font_size (int): Font size of the text
font_color (str): RGB color code for the text (e.g., "0 0 0 rg" for black)
font_name (str, optional): Font name for standard fonts (default: "F1")
font_file (str, optional): Path to a .ttf font file for custom font
page_number (int, optional): Page number to add text to (default: 0)
Returns:
str: Success message with output file path
"""
```
## Best Practices
1. File Handling:
- Ensure PDF files exist before processing
- Use absolute paths for reliability
- Handle file permissions appropriately
2. Text Positioning:
- Use appropriate coordinates based on PDF dimensions
- Consider page orientation and margins
- Test positioning with small changes first
3. Font Usage:
- Verify custom font files exist
- Use standard fonts when possible
- Test font rendering before production use
4. Error Handling:
- Check page numbers are valid
- Verify font file accessibility
- Handle file writing permissions
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import PDFTextWritingTool
# Initialize tool
pdf_tool = PDFTextWritingTool()
# Create agent
document_processor = Agent(
role='Document Processor',
goal='Process and annotate PDF documents',
backstory='Expert at PDF manipulation and text placement.',
tools=[pdf_tool]
)
# Define task
annotation_task = Task(
description="""Add a watermark saying 'CONFIDENTIAL' to
the center of the first page of the document at
'/path/to/document.pdf'.""",
agent=document_processor
)
# The tool will use:
# {
# "pdf_path": "/path/to/document.pdf",
# "text": "CONFIDENTIAL",
# "position": (300, 400),
# "font_size": 24,
# "font_color": "1 0 0 rg", # Red color
# "page_number": 0
# }
# Create crew
crew = Crew(
agents=[document_processor],
tasks=[annotation_task]
)
# Execute
result = crew.kickoff()
```
## Advanced Usage
### Custom Font Example
```python
# Using a custom font
result = pdf_tool.run(
pdf_path="/path/to/input.pdf",
text="Custom Font Text",
position=(100, 500),
font_size=16,
font_color="0 0 1 rg", # Blue color
font_file="/path/to/custom_font.ttf",
page_number=0
)
```
### Multiple Text Elements
```python
# Add multiple text elements
positions = [(100, 700), (100, 650), (100, 600)]
texts = ["Header", "Subheader", "Body Text"]
font_sizes = [18, 14, 12]
for text, position, size in zip(texts, positions, font_sizes):
pdf_tool.run(
pdf_path="/path/to/input.pdf",
text=text,
position=position,
font_size=size,
font_color="0 0 0 rg" # Black color
)
```
### Color Text Example
```python
# Add colored text
colors = {
"red": "1 0 0 rg",
"green": "0 1 0 rg",
"blue": "0 0 1 rg"
}
for y_pos, (color_name, color_code) in enumerate(colors.items()):
pdf_tool.run(
pdf_path="/path/to/input.pdf",
text=f"This text is {color_name}",
position=(100, 700 - y_pos * 50),
font_size=14,
font_color=color_code
)
```
## Notes
- Supports custom TrueType fonts (.ttf)
- Allows RGB color specifications
- Handles multi-page PDFs
- Preserves original PDF content
- Supports text positioning with x,y coordinates
- Maintains PDF structure and metadata
- Creates new output file for safety
- Thread-safe operations
- Efficient PDF manipulation
- Supports various text attributes

View File

@@ -1,181 +0,0 @@
---
title: PGSearchTool
description: A RAG-based semantic search tool for PostgreSQL database content
icon: database-search
---
## PGSearchTool
The PGSearchTool provides semantic search capabilities for PostgreSQL database content using RAG (Retrieval-Augmented Generation). It allows for natural language queries over database table content by leveraging embeddings and semantic search.
## Installation
```bash
pip install 'crewai[tools]'
pip install embedchain # Required dependency
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import PGSearchTool
# Initialize the tool with database configuration
search_tool = PGSearchTool(
db_uri="postgresql://user:password@localhost:5432/dbname",
table_name="your_table"
)
# Create an agent with the tool
researcher = Agent(
role='Database Researcher',
goal='Find relevant information in database content',
backstory='Expert at searching and analyzing database content.',
tools=[search_tool],
verbose=True
)
```
## Input Schema
```python
class PGSearchToolSchema(BaseModel):
search_query: str = Field(
description="Mandatory semantic search query for searching the database's content"
)
```
## Function Signature
```python
def __init__(self, table_name: str, **kwargs):
"""
Initialize the PostgreSQL search tool.
Args:
table_name (str): Name of the table to search
db_uri (str): PostgreSQL database URI (required in kwargs)
**kwargs: Additional arguments for RagTool initialization
"""
def _run(
self,
search_query: str,
**kwargs: Any
) -> Any:
"""
Perform semantic search on database content.
Args:
search_query (str): Semantic search query
**kwargs: Additional search parameters
Returns:
Any: Relevant database content based on semantic search
"""
```
## Best Practices
1. Secure database credentials:
```python
# Use environment variables for sensitive data
import os
db_uri = (
f"postgresql://{os.getenv('DB_USER')}:{os.getenv('DB_PASS')}"
f"@{os.getenv('DB_HOST')}:{os.getenv('DB_PORT')}/{os.getenv('DB_NAME')}"
)
```
2. Optimize table selection
3. Use specific semantic queries
4. Handle database connection errors
5. Consider table size and query performance
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import PGSearchTool
# Initialize tool with database configuration
db_search = PGSearchTool(
db_uri="postgresql://user:password@localhost:5432/dbname",
table_name="customer_feedback"
)
# Create agent
analyst = Agent(
role='Database Analyst',
goal='Analyze customer feedback data',
backstory='Expert at finding insights in customer feedback.',
tools=[db_search]
)
# Define task
analysis_task = Task(
description="""Find all customer feedback related to product usability
and ease of use. Focus on common patterns and issues.""",
agent=analyst
)
# The tool will use:
# {
# "search_query": "product usability feedback ease of use issues"
# }
# Create crew
crew = Crew(
agents=[analyst],
tasks=[analysis_task]
)
# Execute
result = crew.kickoff()
```
## Advanced Usage
### Multiple Table Search
```python
# Create tools for different tables
customer_search = PGSearchTool(
db_uri="postgresql://user:password@localhost:5432/dbname",
table_name="customers"
)
orders_search = PGSearchTool(
db_uri="postgresql://user:password@localhost:5432/dbname",
table_name="orders"
)
# Use both tools in an agent
analyst = Agent(
role='Multi-table Analyst',
goal='Analyze customer and order data',
tools=[customer_search, orders_search]
)
```
### Error Handling
```python
try:
results = search_tool._run(
search_query="customer satisfaction ratings"
)
# Process results
except Exception as e:
print(f"Database search error: {str(e)}")
```
## Notes
- Inherits from RagTool for semantic search
- Uses embedchain's PostgresLoader
- Requires valid PostgreSQL connection
- Supports semantic natural language queries
- Thread-safe operations
- Efficient for large tables
- Handles connection pooling automatically

View File

@@ -1,282 +0,0 @@
---
title: RagTool
description: Base class for Retrieval-Augmented Generation (RAG) tools with flexible adapter support
icon: database
---
## RagTool
The RagTool serves as the base class for all Retrieval-Augmented Generation (RAG) tools in the CrewAI ecosystem. It provides a flexible adapter-based architecture for implementing knowledge base functionality with semantic search capabilities.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import RagTool
from crewai_tools.adapters import EmbedchainAdapter
from embedchain import App
# Create custom adapter
class CustomAdapter(RagTool.Adapter):
def query(self, question: str) -> str:
# Implement custom query logic
return "Answer based on knowledge base"
def add(self, *args, **kwargs) -> None:
# Implement custom add logic
pass
# Method 1: Use default EmbedchainAdapter
rag_tool = RagTool(
name="Custom Knowledge Base",
description="Specialized knowledge base for domain data",
summarize=True
)
# Method 2: Use custom adapter
custom_tool = RagTool(
name="Custom Knowledge Base",
adapter=CustomAdapter(),
summarize=False
)
# Create an agent with the tool
researcher = Agent(
role='Knowledge Base Researcher',
goal='Search and analyze knowledge base content',
backstory='Expert at finding relevant information in specialized datasets.',
tools=[rag_tool],
verbose=True
)
```
## Adapter Interface
```python
class Adapter(BaseModel, ABC):
@abstractmethod
def query(self, question: str) -> str:
"""
Query the knowledge base with a question.
Args:
question (str): Query to search in knowledge base
Returns:
str: Answer based on knowledge base content
"""
@abstractmethod
def add(self, *args: Any, **kwargs: Any) -> None:
"""
Add content to the knowledge base.
Args:
*args: Variable length argument list
**kwargs: Arbitrary keyword arguments
"""
```
## Function Signature
```python
def __init__(
self,
name: str = "Knowledge base",
description: str = "A knowledge base that can be used to answer questions.",
summarize: bool = False,
adapter: Optional[Adapter] = None,
config: Optional[dict[str, Any]] = None,
**kwargs
):
"""
Initialize the RAG tool.
Args:
name (str): Tool name
description (str): Tool description
summarize (bool): Enable answer summarization
adapter (Optional[Adapter]): Custom adapter implementation
config (Optional[dict]): Configuration for default adapter
**kwargs: Additional arguments for base tool
"""
def _run(
self,
query: str,
**kwargs: Any
) -> str:
"""
Execute query against knowledge base.
Args:
query (str): Question to ask
**kwargs: Additional arguments
Returns:
str: Answer from knowledge base
"""
```
## Best Practices
1. Adapter Implementation:
- Define clear interfaces
- Handle edge cases
- Implement error handling
- Document behavior
2. Knowledge Base Management:
- Organize content logically
- Update content regularly
- Monitor performance
- Handle large datasets
3. Query Optimization:
- Structure queries clearly
- Consider context
- Handle ambiguity
- Validate inputs
4. Error Handling:
- Handle missing data
- Manage timeouts
- Provide clear messages
- Log issues
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import RagTool
from embedchain import App
# Initialize tool with custom configuration
rag_tool = RagTool(
name="Technical Documentation KB",
description="Knowledge base for technical documentation",
summarize=True,
config={
"collection_name": "tech_docs",
"chunking": {
"chunk_size": 500,
"chunk_overlap": 50
}
}
)
# Add content to knowledge base
rag_tool.add(
"Technical documentation content here...",
data_type="text"
)
# Create agent
researcher = Agent(
role='Documentation Expert',
goal='Extract technical information from documentation',
backstory='Expert at analyzing technical documentation.',
tools=[rag_tool]
)
# Define task
research_task = Task(
description="""Find all mentions of API endpoints
and their authentication requirements.""",
agent=researcher
)
# Create crew
crew = Crew(
agents=[researcher],
tasks=[research_task]
)
# Execute
result = crew.kickoff()
```
## Advanced Usage
### Custom Adapter Implementation
```python
from typing import Any
from pydantic import BaseModel
from abc import ABC, abstractmethod
class SpecializedAdapter(RagTool.Adapter):
def __init__(self, config: dict):
self.config = config
self.knowledge_base = {}
def query(self, question: str) -> str:
# Implement specialized query logic
return self._process_query(question)
def add(self, content: str, **kwargs: Any) -> None:
# Implement specialized content addition
self._process_content(content, **kwargs)
# Use custom adapter
specialized_tool = RagTool(
name="Specialized KB",
adapter=SpecializedAdapter(config={"mode": "advanced"})
)
```
### Configuration Management
```python
# Configure default EmbedchainAdapter
config = {
"collection_name": "custom_collection",
"embedding": {
"model": "sentence-transformers/all-mpnet-base-v2",
"dimensions": 768
},
"chunking": {
"chunk_size": 1000,
"chunk_overlap": 100
}
}
tool = RagTool(config=config)
```
### Error Handling Example
```python
try:
rag_tool = RagTool()
# Add content
rag_tool.add(
"Documentation content...",
data_type="text"
)
# Query content
result = rag_tool.run(
query="What are the system requirements?"
)
print(result)
except Exception as e:
print(f"Error using knowledge base: {str(e)}")
```
## Notes
- Base class for RAG tools
- Flexible adapter pattern
- Default EmbedchainAdapter
- Custom adapter support
- Content management
- Query processing
- Error handling
- Configuration options
- Performance optimization
- Memory management

View File

@@ -1,229 +0,0 @@
---
title: SerpApi Google Search Tool
description: A tool for performing Google searches using the SerpApi service
---
# SerpApi Google Search Tool
The SerpApi Google Search Tool enables performing Google searches using the SerpApi service. It provides location-aware search capabilities with comprehensive result filtering.
## Installation
```bash
pip install 'crewai[tools]'
pip install serpapi
```
## Prerequisites
You need a SerpApi API key to use this tool. You can get one from [SerpApi's website](https://serpapi.com/manage-api-key).
Set your API key as an environment variable:
```bash
export SERPAPI_API_KEY="your_api_key_here"
```
## Usage
Here's how to use the SerpApi Google Search Tool:
```python
from crewai import Agent
from crewai_tools import SerpApiGoogleSearchTool
# Initialize the tool
search_tool = SerpApiGoogleSearchTool()
# Create an agent with the tool
search_agent = Agent(
role='Web Researcher',
goal='Find accurate information online',
backstory='I help research and analyze online information',
tools=[search_tool]
)
# Use in a task
task = Task(
description="Research recent AI developments",
agent=search_agent,
context={
"search_query": "latest artificial intelligence breakthroughs 2024",
"location": "United States" # Optional
}
)
```
## Input Schema
```python
class SerpApiGoogleSearchToolSchema(BaseModel):
search_query: str # The search query for Google Search
location: Optional[str] = None # Optional location for localized results
```
## Function Signatures
### Base Tool Initialization
```python
def __init__(self, **kwargs):
"""
Initialize the SerpApi tool with API credentials.
Raises:
ImportError: If serpapi package is not installed
ValueError: If SERPAPI_API_KEY environment variable is not set
"""
```
### Search Execution
```python
def _run(
self,
**kwargs: Any,
) -> dict:
"""
Execute the Google search.
Args:
search_query (str): The search query
location (Optional[str]): Optional location for results
Returns:
dict: Filtered search results from Google
Raises:
HTTPError: If the API request fails
"""
```
## Best Practices
1. **API Key Management**:
- Store the API key securely in environment variables
- Never hardcode the API key in your code
- Verify API key validity before making requests
2. **Search Optimization**:
- Use specific, targeted search queries
- Include relevant keywords and time frames
- Leverage location parameter for regional results
3. **Error Handling**:
- Handle API rate limits gracefully
- Implement retry logic for failed requests
- Validate input parameters before making requests
## Example Integration
Here's a complete example showing how to integrate the SerpApi Google Search Tool with CrewAI:
```python
from crewai import Agent, Task, Crew
from crewai_tools import SerpApiGoogleSearchTool
# Initialize the tool
search_tool = SerpApiGoogleSearchTool()
# Create an agent with the tool
researcher = Agent(
role='Research Analyst',
goal='Find and analyze current information',
backstory="""I am an expert at finding and analyzing
information from various online sources.""",
tools=[search_tool]
)
# Create tasks
research_task = Task(
description="""
Research the following topic:
1. Latest developments in quantum computing
2. Focus on practical applications
3. Include major company announcements
Provide a comprehensive analysis of the findings.
""",
agent=researcher,
context={
"search_query": "quantum computing breakthroughs applications companies",
"location": "United States"
}
)
# Create and run the crew
crew = Crew(
agents=[researcher],
tasks=[research_task]
)
result = crew.kickoff()
```
## Error Handling
The tool handles various error scenarios:
1. **Missing API Key**:
```python
try:
tool = SerpApiGoogleSearchTool()
except ValueError as e:
print("API key not found. Set SERPAPI_API_KEY environment variable.")
```
2. **API Request Errors**:
```python
try:
results = tool._run(
search_query="quantum computing",
location="United States"
)
except HTTPError as e:
print(f"API request failed: {str(e)}")
```
3. **Invalid Parameters**:
```python
try:
results = tool._run(
search_query="", # Empty query
location="Invalid Location"
)
except ValueError as e:
print("Invalid search parameters provided.")
```
## Response Format
The tool returns a filtered dictionary containing Google search results. Example response structure:
```python
{
"organic_results": [
{
"title": "Page Title",
"link": "https://...",
"snippet": "Page description or excerpt...",
"position": 1
}
# Additional results...
],
"knowledge_graph": {
"title": "Topic Title",
"description": "Topic description...",
"source": {
"name": "Source Name",
"link": "https://..."
}
},
"related_questions": [
{
"question": "Related question?",
"answer": "Answer to related question..."
}
# Additional related questions...
]
}
```
The response is automatically filtered to remove metadata and unnecessary fields, focusing on the most relevant search information. Fields like search metadata, parameters, and pagination are omitted for clarity.

View File

@@ -1,225 +0,0 @@
---
title: SerpApi Google Shopping Tool
description: A tool for searching Google Shopping using the SerpApi service
---
# SerpApi Google Shopping Tool
The SerpApi Google Shopping Tool enables searching Google Shopping results using the SerpApi service. It provides location-aware shopping search capabilities with comprehensive result filtering.
## Installation
```bash
pip install 'crewai[tools]'
pip install serpapi
```
## Prerequisites
You need a SerpApi API key to use this tool. You can get one from [SerpApi's website](https://serpapi.com/manage-api-key).
Set your API key as an environment variable:
```bash
export SERPAPI_API_KEY="your_api_key_here"
```
## Usage
Here's how to use the SerpApi Google Shopping Tool:
```python
from crewai import Agent
from crewai_tools import SerpApiGoogleShoppingTool
# Initialize the tool
shopping_tool = SerpApiGoogleShoppingTool()
# Create an agent with the tool
shopping_agent = Agent(
role='Shopping Researcher',
goal='Find the best shopping deals',
backstory='I help find and analyze shopping options',
tools=[shopping_tool]
)
# Use in a task
task = Task(
description="Find best deals for gaming laptops",
agent=shopping_agent,
context={
"search_query": "gaming laptop deals",
"location": "United States" # Optional
}
)
```
## Input Schema
```python
class SerpApiGoogleShoppingToolSchema(BaseModel):
search_query: str # The search query for Google Shopping
location: Optional[str] = None # Optional location for localized results
```
## Function Signatures
### Base Tool Initialization
```python
def __init__(self, **kwargs):
"""
Initialize the SerpApi tool with API credentials.
Raises:
ImportError: If serpapi package is not installed
ValueError: If SERPAPI_API_KEY environment variable is not set
"""
```
### Search Execution
```python
def _run(
self,
**kwargs: Any,
) -> dict:
"""
Execute the Google Shopping search.
Args:
search_query (str): The search query for Google Shopping
location (Optional[str]): Optional location for results
Returns:
dict: Filtered search results from Google Shopping
Raises:
HTTPError: If the API request fails
"""
```
## Best Practices
1. **API Key Management**:
- Store the API key securely in environment variables
- Never hardcode the API key in your code
- Verify API key validity before making requests
2. **Search Optimization**:
- Use specific, targeted search queries
- Include relevant product details in queries
- Leverage location parameter for regional pricing
3. **Error Handling**:
- Handle API rate limits gracefully
- Implement retry logic for failed requests
- Validate input parameters before making requests
## Example Integration
Here's a complete example showing how to integrate the SerpApi Google Shopping Tool with CrewAI:
```python
from crewai import Agent, Task, Crew
from crewai_tools import SerpApiGoogleShoppingTool
# Initialize the tool
shopping_tool = SerpApiGoogleShoppingTool()
# Create an agent with the tool
researcher = Agent(
role='Shopping Analyst',
goal='Find and analyze the best shopping deals',
backstory="""I am an expert at finding the best shopping deals
and analyzing product offerings across different regions.""",
tools=[shopping_tool]
)
# Create tasks
search_task = Task(
description="""
Research gaming laptops with the following criteria:
1. Price range: $800-$1500
2. Released in the last year
3. Compare prices across different retailers
Provide a comprehensive analysis of the findings.
""",
agent=researcher,
context={
"search_query": "gaming laptop RTX 4060 2023",
"location": "United States"
}
)
# Create and run the crew
crew = Crew(
agents=[researcher],
tasks=[search_task]
)
result = crew.kickoff()
```
## Error Handling
The tool handles various error scenarios:
1. **Missing API Key**:
```python
try:
tool = SerpApiGoogleShoppingTool()
except ValueError as e:
print("API key not found. Set SERPAPI_API_KEY environment variable.")
```
2. **API Request Errors**:
```python
try:
results = tool._run(
search_query="gaming laptop",
location="United States"
)
except HTTPError as e:
print(f"API request failed: {str(e)}")
```
3. **Invalid Parameters**:
```python
try:
results = tool._run(
search_query="", # Empty query
location="Invalid Location"
)
except ValueError as e:
print("Invalid search parameters provided.")
```
## Response Format
The tool returns a filtered dictionary containing Google Shopping results. Example response structure:
```python
{
"shopping_results": [
{
"title": "Product Title",
"price": "$999.99",
"link": "https://...",
"source": "Retailer Name",
"rating": 4.5,
"reviews": 123,
"thumbnail": "https://..."
}
# Additional results...
],
"organic_results": [
{
"title": "Related Product",
"link": "https://...",
"snippet": "Product description..."
}
# Additional organic results...
]
}
```
The response is automatically filtered to remove metadata and unnecessary fields, focusing on the most relevant shopping information.

View File

@@ -1,184 +0,0 @@
---
title: SerplyJobSearchTool
description: A tool for searching US job postings using the Serply API
icon: briefcase
---
## SerplyJobSearchTool
The SerplyJobSearchTool provides job search capabilities using the Serply API. It allows for searching job postings in the US market, returning structured information about positions, employers, locations, and remote work status.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import SerplyJobSearchTool
# Set environment variable
# export SERPLY_API_KEY='your-api-key'
# Initialize the tool
search_tool = SerplyJobSearchTool()
# Create an agent with the tool
job_researcher = Agent(
role='Job Market Researcher',
goal='Find relevant job opportunities',
backstory='Expert at analyzing job market trends and opportunities.',
tools=[search_tool],
verbose=True
)
```
## Input Schema
```python
class SerplyJobSearchToolSchema(BaseModel):
search_query: str = Field(
description="Mandatory search query for fetching job postings"
)
```
## Function Signature
```python
def __init__(self, **kwargs):
"""
Initialize the job search tool.
Args:
**kwargs: Additional arguments for RagTool initialization
Note:
Requires SERPLY_API_KEY environment variable
"""
def _run(
self,
**kwargs: Any
) -> str:
"""
Perform job search using Serply API.
Args:
search_query (str): Job search query
**kwargs: Additional search parameters
Returns:
str: Formatted string containing job listings with details:
- Position
- Employer
- Location
- Link
- Highlights
- Remote/Hybrid status
"""
```
## Best Practices
1. Set up API authentication:
```bash
export SERPLY_API_KEY='your-serply-api-key'
```
2. Use specific search queries
3. Handle potential API errors
4. Process structured results effectively
5. Consider rate limits and quotas
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import SerplyJobSearchTool
# Initialize tool
job_search = SerplyJobSearchTool()
# Create agent
recruiter = Agent(
role='Technical Recruiter',
goal='Find relevant job opportunities in tech',
backstory='Expert at identifying promising tech positions.',
tools=[job_search]
)
# Define task
search_task = Task(
description="""Search for senior software engineer positions
with remote work options in the US. Focus on positions
requiring Python expertise.""",
agent=recruiter
)
# The tool will use:
# {
# "search_query": "senior software engineer python remote"
# }
# Create crew
crew = Crew(
agents=[recruiter],
tasks=[search_task]
)
# Execute
result = crew.kickoff()
```
## Advanced Usage
### Handling Search Results
```python
# Example of processing structured results
results = search_tool._run(
search_query="machine learning engineer"
)
# Results format:
"""
Search results:
Position: Senior Machine Learning Engineer
Employer: TechCorp Inc
Location: San Francisco, CA
Link: https://example.com/job/123
Highlights: Python, TensorFlow, 5+ years experience
Is Remote: True
Is Hybrid: False
---
Position: ML Engineer
...
"""
```
### Error Handling
```python
try:
results = search_tool._run(
search_query="data scientist"
)
if not results:
print("No jobs found")
else:
print(results)
except Exception as e:
print(f"Job search error: {str(e)}")
```
## Notes
- Requires valid Serply API key
- Currently supports US job market only
- Returns structured job information
- Includes remote/hybrid status
- Thread-safe operations
- Efficient job search capabilities
- Handles API rate limiting automatically
- Provides detailed job highlights

View File

@@ -1,209 +0,0 @@
---
title: SerplyNewsSearchTool
description: A news article search tool powered by Serply API with configurable search parameters
icon: newspaper
---
## SerplyNewsSearchTool
The SerplyNewsSearchTool provides news article search capabilities using the Serply API. It allows for customizable search parameters including result limits and proxy location for region-specific news results.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import SerplyNewsSearchTool
# Set environment variable
# export SERPLY_API_KEY='your-api-key'
# Basic initialization
news_tool = SerplyNewsSearchTool()
# Advanced initialization with custom parameters
news_tool = SerplyNewsSearchTool(
limit=20, # Return 20 results
proxy_location="FR" # Search from France
)
# Create an agent with the tool
news_researcher = Agent(
role='News Researcher',
goal='Find relevant news articles',
backstory='Expert at news research and information gathering.',
tools=[news_tool],
verbose=True
)
```
## Input Schema
```python
class SerplyNewsSearchToolSchema(BaseModel):
search_query: str = Field(
description="Mandatory search query for fetching news articles"
)
```
## Function Signature
```python
def __init__(
self,
limit: Optional[int] = 10,
proxy_location: Optional[str] = "US",
**kwargs
):
"""
Initialize the news search tool.
Args:
limit (int): Maximum number of results [10-100] (default: 10)
proxy_location (str): Region for local news results (default: "US")
Options: US, CA, IE, GB, FR, DE, SE, IN, JP, KR, SG, AU, BR
**kwargs: Additional arguments for tool creation
"""
def _run(
self,
**kwargs: Any
) -> str:
"""
Perform news search using Serply API.
Args:
search_query (str): News search query
Returns:
str: Formatted string containing news results:
- Title
- Link
- Source
- Published Date
"""
```
## Best Practices
1. Set up API authentication:
```bash
export SERPLY_API_KEY='your-serply-api-key'
```
2. Configure search parameters appropriately:
- Set reasonable result limits
- Select relevant proxy location for regional news
- Consider time sensitivity of news content
3. Handle potential API errors
4. Process structured results effectively
5. Consider rate limits and quotas
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import SerplyNewsSearchTool
# Initialize tool with custom configuration
news_tool = SerplyNewsSearchTool(
limit=15, # 15 results
proxy_location="US" # US news sources
)
# Create agent
news_analyst = Agent(
role='News Analyst',
goal='Research breaking news and developments',
backstory='Expert at analyzing news trends and developments.',
tools=[news_tool]
)
# Define task
news_task = Task(
description="""Research the latest developments in renewable
energy technology and investments, focusing on major
announcements and industry trends.""",
agent=news_analyst
)
# The tool will use:
# {
# "search_query": "renewable energy technology investments news"
# }
# Create crew
crew = Crew(
agents=[news_analyst],
tasks=[news_task]
)
# Execute
result = crew.kickoff()
```
## Advanced Usage
### Regional News Configuration
```python
# French news sources
fr_news = SerplyNewsSearchTool(
proxy_location="FR",
limit=20
)
# Japanese news sources
jp_news = SerplyNewsSearchTool(
proxy_location="JP",
limit=20
)
```
### Result Processing
```python
# Get news results
try:
results = news_tool._run(
search_query="renewable energy investments"
)
print(results)
except Exception as e:
print(f"News search error: {str(e)}")
```
### Multiple Region Search
```python
# Search across multiple regions
regions = ["US", "GB", "DE"]
all_results = []
for region in regions:
regional_tool = SerplyNewsSearchTool(
proxy_location=region,
limit=5
)
results = regional_tool._run(
search_query="global tech innovations"
)
all_results.append(f"Results from {region}:\n{results}")
combined_results = "\n\n".join(all_results)
```
## Notes
- Requires valid Serply API key
- Supports multiple regions for news sources
- Configurable result limits (10-100)
- Returns structured news article data
- Thread-safe operations
- Efficient news search capabilities
- Handles API rate limiting automatically
- Includes source attribution and publication dates
- Follows redirects for final article URLs

View File

@@ -1,209 +0,0 @@
---
title: SerplyScholarSearchTool
description: A scholarly literature search tool powered by Serply API with configurable search parameters
icon: book
---
## SerplyScholarSearchTool
The SerplyScholarSearchTool provides scholarly literature search capabilities using the Serply API. It allows for customizable search parameters including language and proxy location for region-specific academic results.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import SerplyScholarSearchTool
# Set environment variable
# export SERPLY_API_KEY='your-api-key'
# Basic initialization
scholar_tool = SerplyScholarSearchTool()
# Advanced initialization with custom parameters
scholar_tool = SerplyScholarSearchTool(
hl="fr", # French language results
proxy_location="FR" # Search from France
)
# Create an agent with the tool
academic_researcher = Agent(
role='Academic Researcher',
goal='Find relevant scholarly literature',
backstory='Expert at academic research and literature review.',
tools=[scholar_tool],
verbose=True
)
```
## Input Schema
```python
class SerplyScholarSearchToolSchema(BaseModel):
search_query: str = Field(
description="Mandatory search query for fetching scholarly literature"
)
```
## Function Signature
```python
def __init__(
self,
hl: str = "us",
proxy_location: Optional[str] = "US",
**kwargs
):
"""
Initialize the scholar search tool.
Args:
hl (str): Host language code for results (default: "us")
Reference: https://developers.google.com/custom-search/docs/xml_results?hl=en#wsInterfaceLanguages
proxy_location (str): Region for local results (default: "US")
Options: US, CA, IE, GB, FR, DE, SE, IN, JP, KR, SG, AU, BR
**kwargs: Additional arguments for tool creation
"""
def _run(
self,
**kwargs: Any
) -> str:
"""
Perform scholarly literature search using Serply API.
Args:
search_query (str): Academic search query
Returns:
str: Formatted string containing scholarly results:
- Title
- Link
- Description
- Citation
- Authors
"""
```
## Best Practices
1. Set up API authentication:
```bash
export SERPLY_API_KEY='your-serply-api-key'
```
2. Configure search parameters appropriately:
- Use relevant language codes
- Select appropriate proxy location
- Provide specific academic search terms
3. Handle potential API errors
4. Process structured results effectively
5. Consider rate limits and quotas
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import SerplyScholarSearchTool
# Initialize tool with custom configuration
scholar_tool = SerplyScholarSearchTool(
hl="en", # English results
proxy_location="US" # US academic sources
)
# Create agent
researcher = Agent(
role='Academic Researcher',
goal='Research recent academic publications',
backstory='Expert at analyzing academic literature and research trends.',
tools=[scholar_tool]
)
# Define task
research_task = Task(
description="""Research recent academic publications on
machine learning applications in healthcare, focusing on
peer-reviewed articles from the last two years.""",
agent=researcher
)
# The tool will use:
# {
# "search_query": "machine learning healthcare applications"
# }
# Create crew
crew = Crew(
agents=[researcher],
tasks=[research_task]
)
# Execute
result = crew.kickoff()
```
## Advanced Usage
### Language and Region Configuration
```python
# French academic sources
fr_scholar = SerplyScholarSearchTool(
hl="fr",
proxy_location="FR"
)
# German academic sources
de_scholar = SerplyScholarSearchTool(
hl="de",
proxy_location="DE"
)
```
### Result Processing
```python
try:
results = scholar_tool._run(
search_query="machine learning healthcare applications"
)
print(results)
except Exception as e:
print(f"Scholar search error: {str(e)}")
```
### Citation Analysis
```python
# Extract and analyze citations
def analyze_citations(results):
citations = []
for result in results.split("---"):
if "Cite:" in result:
citation = result.split("Cite:")[1].split("\n")[0].strip()
citations.append(citation)
return citations
results = scholar_tool._run(
search_query="artificial intelligence ethics"
)
citations = analyze_citations(results)
```
## Notes
- Requires valid Serply API key
- Supports multiple languages and regions
- Returns structured academic article data
- Includes citation information
- Lists all authors of publications
- Thread-safe operations
- Efficient scholarly search capabilities
- Handles API rate limiting automatically
- Supports both direct and document links
- Provides comprehensive article metadata

View File

@@ -1,213 +0,0 @@
---
title: SerplyWebSearchTool
description: A Google search tool powered by Serply API with configurable search parameters
icon: search
---
## SerplyWebSearchTool
The SerplyWebSearchTool provides Google search capabilities using the Serply API. It allows for customizable search parameters including language, result limits, device type, and proxy location for region-specific results.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import SerplyWebSearchTool
# Set environment variable
# export SERPLY_API_KEY='your-api-key'
# Basic initialization
search_tool = SerplyWebSearchTool()
# Advanced initialization with custom parameters
search_tool = SerplyWebSearchTool(
hl="fr", # French language results
limit=20, # Return 20 results
device_type="mobile", # Mobile search results
proxy_location="FR" # Search from France
)
# Create an agent with the tool
researcher = Agent(
role='Web Researcher',
goal='Find relevant information online',
backstory='Expert at web research and information gathering.',
tools=[search_tool],
verbose=True
)
```
## Input Schema
```python
class SerplyWebSearchToolSchema(BaseModel):
search_query: str = Field(
description="Mandatory search query for Google search"
)
```
## Function Signature
```python
def __init__(
self,
hl: str = "us",
limit: int = 10,
device_type: str = "desktop",
proxy_location: str = "US",
**kwargs
):
"""
Initialize the Google search tool.
Args:
hl (str): Host language code for results (default: "us")
Reference: https://developers.google.com/custom-search/docs/xml_results?hl=en#wsInterfaceLanguages
limit (int): Maximum number of results [10-100] (default: 10)
device_type (str): "desktop" or "mobile" results (default: "desktop")
proxy_location (str): Region for local results (default: "US")
Options: US, CA, IE, GB, FR, DE, SE, IN, JP, KR, SG, AU, BR
**kwargs: Additional arguments for tool creation
"""
def _run(
self,
**kwargs: Any
) -> str:
"""
Perform Google search using Serply API.
Args:
search_query (str): Search query
Returns:
str: Formatted string containing search results:
- Title
- Link
- Description
"""
```
## Best Practices
1. Set up API authentication:
```bash
export SERPLY_API_KEY='your-serply-api-key'
```
2. Configure search parameters appropriately:
- Use relevant language codes
- Set reasonable result limits
- Choose appropriate device type
- Select relevant proxy location
3. Handle potential API errors
4. Process structured results effectively
5. Consider rate limits and quotas
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import SerplyWebSearchTool
# Initialize tool with custom configuration
search_tool = SerplyWebSearchTool(
hl="en", # English results
limit=15, # 15 results
device_type="desktop",
proxy_location="US"
)
# Create agent
researcher = Agent(
role='Web Researcher',
goal='Research emerging technology trends',
backstory='Expert at finding and analyzing tech trends.',
tools=[search_tool]
)
# Define task
research_task = Task(
description="""Research the latest developments in artificial
intelligence and machine learning, focusing on practical
applications in business.""",
agent=researcher
)
# The tool will use:
# {
# "search_query": "latest AI ML developments business applications"
# }
# Create crew
crew = Crew(
agents=[researcher],
tasks=[research_task]
)
# Execute
result = crew.kickoff()
```
## Advanced Usage
### Language and Region Configuration
```python
# French search from France
fr_search = SerplyWebSearchTool(
hl="fr",
proxy_location="FR"
)
# Japanese search from Japan
jp_search = SerplyWebSearchTool(
hl="ja",
proxy_location="JP"
)
```
### Device-Specific Results
```python
# Mobile results
mobile_search = SerplyWebSearchTool(
device_type="mobile",
limit=20
)
# Desktop results
desktop_search = SerplyWebSearchTool(
device_type="desktop",
limit=20
)
```
### Error Handling
```python
try:
results = search_tool._run(
search_query="artificial intelligence trends"
)
print(results)
except Exception as e:
print(f"Search error: {str(e)}")
```
## Notes
- Requires valid Serply API key
- Supports multiple languages and regions
- Configurable result limits (10-100)
- Device-specific search results
- Thread-safe operations
- Efficient search capabilities
- Handles API rate limiting automatically
- Returns structured search results

View File

@@ -1,201 +0,0 @@
---
title: SerplyWebpageToMarkdownTool
description: A tool for converting web pages to markdown format using Serply API
icon: markdown
---
## SerplyWebpageToMarkdownTool
The SerplyWebpageToMarkdownTool converts web pages to markdown format using the Serply API, making it easier for LLMs to process and understand web content. It supports configurable proxy locations for region-specific access.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import SerplyWebpageToMarkdownTool
# Set environment variable
# export SERPLY_API_KEY='your-api-key'
# Basic initialization
markdown_tool = SerplyWebpageToMarkdownTool()
# Advanced initialization with custom parameters
markdown_tool = SerplyWebpageToMarkdownTool(
proxy_location="FR" # Access from France
)
# Create an agent with the tool
web_processor = Agent(
role='Web Content Processor',
goal='Convert web content to markdown format',
backstory='Expert at processing and formatting web content.',
tools=[markdown_tool],
verbose=True
)
```
## Input Schema
```python
class SerplyWebpageToMarkdownToolSchema(BaseModel):
url: str = Field(
description="Mandatory URL of the webpage to convert to markdown"
)
```
## Function Signature
```python
def __init__(
self,
proxy_location: Optional[str] = "US",
**kwargs
):
"""
Initialize the webpage to markdown conversion tool.
Args:
proxy_location (str): Region for accessing the webpage (default: "US")
Options: US, CA, IE, GB, FR, DE, SE, IN, JP, KR, SG, AU, BR
**kwargs: Additional arguments for tool creation
"""
def _run(
self,
**kwargs: Any
) -> str:
"""
Convert webpage to markdown using Serply API.
Args:
url (str): URL of the webpage to convert
Returns:
str: Markdown formatted content of the webpage
"""
```
## Best Practices
1. Set up API authentication:
```bash
export SERPLY_API_KEY='your-serply-api-key'
```
2. Configure proxy location appropriately:
- Select relevant region for access
- Consider content accessibility
- Handle region-specific content
3. Handle potential API errors
4. Process markdown output effectively
5. Consider rate limits and quotas
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import SerplyWebpageToMarkdownTool
# Initialize tool with custom configuration
markdown_tool = SerplyWebpageToMarkdownTool(
proxy_location="US" # US access point
)
# Create agent
processor = Agent(
role='Content Processor',
goal='Convert web content to structured markdown',
backstory='Expert at processing web content into structured formats.',
tools=[markdown_tool]
)
# Define task
conversion_task = Task(
description="""Convert the documentation page at
https://example.com/docs into markdown format for
further processing.""",
agent=processor
)
# The tool will use:
# {
# "url": "https://example.com/docs"
# }
# Create crew
crew = Crew(
agents=[processor],
tasks=[conversion_task]
)
# Execute
result = crew.kickoff()
```
## Advanced Usage
### Regional Access Configuration
```python
# European access points
fr_processor = SerplyWebpageToMarkdownTool(
proxy_location="FR"
)
de_processor = SerplyWebpageToMarkdownTool(
proxy_location="DE"
)
```
### Error Handling
```python
try:
markdown_content = markdown_tool._run(
url="https://example.com/page"
)
print(markdown_content)
except Exception as e:
print(f"Conversion error: {str(e)}")
```
### Content Processing
```python
# Process multiple pages
urls = [
"https://example.com/page1",
"https://example.com/page2",
"https://example.com/page3"
]
markdown_contents = []
for url in urls:
try:
content = markdown_tool._run(url=url)
markdown_contents.append(content)
except Exception as e:
print(f"Error processing {url}: {str(e)}")
continue
# Combine contents
combined_markdown = "\n\n---\n\n".join(markdown_contents)
```
## Notes
- Requires valid Serply API key
- Supports multiple proxy locations
- Returns markdown-formatted content
- Simplifies web content for LLM processing
- Thread-safe operations
- Efficient content conversion
- Handles API rate limiting automatically
- Preserves content structure in markdown
- Supports various webpage formats
- Makes web content more accessible to AI agents

View File

@@ -1,158 +0,0 @@
---
title: TXTSearchTool
description: A semantic search tool for text files using RAG capabilities
icon: magnifying-glass-document
---
## TXTSearchTool
The TXTSearchTool is a specialized Retrieval-Augmented Generation (RAG) tool that enables semantic search within text files. It inherits from the base RagTool class and provides both fixed and dynamic text file searching capabilities.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import TXTSearchTool
# Method 1: Dynamic file path
txt_search = TXTSearchTool()
# Method 2: Fixed file path
fixed_txt_search = TXTSearchTool(txt="path/to/fixed/document.txt")
# Create an agent with the tool
researcher = Agent(
role='Research Assistant',
goal='Search through text documents semantically',
backstory='Expert at finding relevant information in documents using semantic search.',
tools=[txt_search],
verbose=True
)
```
## Input Schema
The tool supports two input schemas depending on initialization:
### Dynamic File Path Schema
```python
class TXTSearchToolSchema(BaseModel):
search_query: str # The semantic search query
txt: str # Path to the text file to search
```
### Fixed File Path Schema
```python
class FixedTXTSearchToolSchema(BaseModel):
search_query: str # The semantic search query
```
## Function Signature
```python
def __init__(self, txt: Optional[str] = None, **kwargs):
"""
Initialize the TXT search tool.
Args:
txt (Optional[str]): Fixed path to a text file. If provided, the tool will only search this file.
**kwargs: Additional arguments passed to the parent RagTool
"""
def _run(self, search_query: str, **kwargs: Any) -> Any:
"""
Perform semantic search on the text file.
Args:
search_query (str): The semantic search query
**kwargs: Additional arguments (including 'txt' for dynamic file path)
Returns:
str: Relevant text passages based on semantic search
"""
```
## Best Practices
1. Choose initialization method based on use case:
- Use fixed file path when repeatedly searching the same document
- Use dynamic file path when searching different documents
2. Write clear, semantic search queries
3. Handle potential file access errors in agent prompts
4. Consider memory usage for large text files
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import TXTSearchTool
# Example 1: Fixed document search
documentation_search = TXTSearchTool(txt="api_documentation.txt")
# Example 2: Dynamic document search
flexible_search = TXTSearchTool()
# Create agents
doc_analyst = Agent(
role='Documentation Analyst',
goal='Find relevant API documentation sections',
backstory='Expert at analyzing technical documentation.',
tools=[documentation_search]
)
file_analyst = Agent(
role='File Analyst',
goal='Search through various text files',
backstory='Specialist in finding information across multiple documents.',
tools=[flexible_search]
)
# Define tasks
fixed_search_task = Task(
description="""Find all API endpoints related to user authentication
in the documentation.""",
agent=doc_analyst
)
# The agent will use:
# {
# "search_query": "user authentication API endpoints"
# }
dynamic_search_task = Task(
description="""Search through the logs.txt file for any database
connection errors.""",
agent=file_analyst
)
# The agent will use:
# {
# "search_query": "database connection errors",
# "txt": "logs.txt"
# }
# Create crew
crew = Crew(
agents=[doc_analyst, file_analyst],
tasks=[fixed_search_task, dynamic_search_task]
)
# Execute
result = crew.kickoff()
```
## Notes
- Inherits from RagTool for semantic search capabilities
- Supports both fixed and dynamic text file paths
- Uses embeddings for semantic search
- Optimized for text file analysis
- Thread-safe operations
- Automatically handles file loading and embedding

View File

@@ -1,159 +0,0 @@
---
title: YoutubeChannelSearchTool
description: A semantic search tool for YouTube channel content using RAG capabilities
icon: youtube
---
## YoutubeChannelSearchTool
The YoutubeChannelSearchTool is a specialized Retrieval-Augmented Generation (RAG) tool that enables semantic search within YouTube channel content. It inherits from the base RagTool class and provides both fixed and dynamic YouTube channel searching capabilities.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import YoutubeChannelSearchTool
# Method 1: Dynamic channel handle
youtube_search = YoutubeChannelSearchTool()
# Method 2: Fixed channel handle
fixed_channel_search = YoutubeChannelSearchTool(youtube_channel_handle="@example_channel")
# Create an agent with the tool
researcher = Agent(
role='Content Researcher',
goal='Search through YouTube channel content semantically',
backstory='Expert at finding relevant information in YouTube content.',
tools=[youtube_search],
verbose=True
)
```
## Input Schema
The tool supports two input schemas depending on initialization:
### Dynamic Channel Schema
```python
class YoutubeChannelSearchToolSchema(BaseModel):
search_query: str # The semantic search query
youtube_channel_handle: str # YouTube channel handle (with or without @)
```
### Fixed Channel Schema
```python
class FixedYoutubeChannelSearchToolSchema(BaseModel):
search_query: str # The semantic search query
```
## Function Signature
```python
def __init__(self, youtube_channel_handle: Optional[str] = None, **kwargs):
"""
Initialize the YouTube channel search tool.
Args:
youtube_channel_handle (Optional[str]): Fixed channel handle. If provided,
the tool will only search this channel.
**kwargs: Additional arguments passed to the parent RagTool
"""
def _run(self, search_query: str, **kwargs: Any) -> Any:
"""
Perform semantic search on the YouTube channel content.
Args:
search_query (str): The semantic search query
**kwargs: Additional arguments (including 'youtube_channel_handle' for dynamic mode)
Returns:
str: Relevant content from the YouTube channel based on semantic search
"""
```
## Best Practices
1. Choose initialization method based on use case:
- Use fixed channel handle when repeatedly searching the same channel
- Use dynamic handle when searching different channels
2. Write clear, semantic search queries
3. Channel handles can be provided with or without '@' prefix
4. Consider content availability and channel size
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import YoutubeChannelSearchTool
# Example 1: Fixed channel search
tech_channel_search = YoutubeChannelSearchTool(youtube_channel_handle="@TechChannel")
# Example 2: Dynamic channel search
flexible_search = YoutubeChannelSearchTool()
# Create agents
tech_analyst = Agent(
role='Tech Content Analyst',
goal='Find relevant tech tutorials and explanations',
backstory='Expert at analyzing technical YouTube content.',
tools=[tech_channel_search]
)
content_researcher = Agent(
role='Content Researcher',
goal='Search across multiple YouTube channels',
backstory='Specialist in finding information across various channels.',
tools=[flexible_search]
)
# Define tasks
fixed_search_task = Task(
description="""Find all tutorials related to machine learning
basics in the channel.""",
agent=tech_analyst
)
# The agent will use:
# {
# "search_query": "machine learning basics tutorial"
# }
dynamic_search_task = Task(
description="""Search through the @AIResearch channel for
content about neural networks.""",
agent=content_researcher
)
# The agent will use:
# {
# "search_query": "neural networks explanation",
# "youtube_channel_handle": "@AIResearch"
# }
# Create crew
crew = Crew(
agents=[tech_analyst, content_researcher],
tasks=[fixed_search_task, dynamic_search_task]
)
# Execute
result = crew.kickoff()
```
## Notes
- Inherits from RagTool for semantic search capabilities
- Supports both fixed and dynamic YouTube channel handles
- Automatically adds '@' prefix to channel handles if missing
- Uses embeddings for semantic search
- Thread-safe operations
- Automatically handles YouTube content loading and embedding

View File

@@ -1,216 +0,0 @@
---
title: YoutubeVideoSearchTool
description: A tool for semantic search within YouTube video content using RAG capabilities
icon: video
---
## YoutubeVideoSearchTool
The YoutubeVideoSearchTool enables semantic search capabilities for YouTube video content using Retrieval-Augmented Generation (RAG). It processes video content and allows searching through transcripts and metadata using natural language queries.
## Installation
```bash
pip install 'crewai[tools]'
```
## Usage Example
```python
from crewai import Agent
from crewai_tools import YoutubeVideoSearchTool
# Method 1: Initialize with specific video
video_tool = YoutubeVideoSearchTool(
youtube_video_url="https://www.youtube.com/watch?v=example"
)
# Method 2: Initialize without video (specify at runtime)
flexible_video_tool = YoutubeVideoSearchTool()
# Create an agent with the tool
researcher = Agent(
role='Video Researcher',
goal='Search and analyze video content',
backstory='Expert at finding relevant information in videos.',
tools=[video_tool],
verbose=True
)
```
## Input Schema
### Fixed Video Schema (when URL provided during initialization)
```python
class FixedYoutubeVideoSearchToolSchema(BaseModel):
search_query: str = Field(
description="Mandatory search query you want to use to search the Youtube Video content"
)
```
### Flexible Video Schema (when URL provided at runtime)
```python
class YoutubeVideoSearchToolSchema(FixedYoutubeVideoSearchToolSchema):
youtube_video_url: str = Field(
description="Mandatory youtube_video_url path you want to search"
)
```
## Function Signature
```python
def __init__(
self,
youtube_video_url: Optional[str] = None,
**kwargs
):
"""
Initialize the YouTube video search tool.
Args:
youtube_video_url (Optional[str]): URL of YouTube video (optional)
**kwargs: Additional arguments for RAG tool configuration
"""
def _run(
self,
search_query: str,
**kwargs: Any
) -> str:
"""
Execute semantic search on video content.
Args:
search_query (str): Query to search in the video
**kwargs: Additional arguments including youtube_video_url if not initialized
Returns:
str: Relevant content from the video matching the query
"""
```
## Best Practices
1. Video URL Management:
- Use complete YouTube URLs
- Verify video accessibility
- Handle region restrictions
2. Search Optimization:
- Use specific, focused queries
- Consider video context
- Test with sample queries first
3. Performance Considerations:
- Pre-initialize for repeated searches
- Handle long videos appropriately
- Monitor processing time
4. Error Handling:
- Verify video availability
- Handle unavailable videos
- Manage API limitations
## Integration Example
```python
from crewai import Agent, Task, Crew
from crewai_tools import YoutubeVideoSearchTool
# Initialize tool with specific video
video_tool = YoutubeVideoSearchTool(
youtube_video_url="https://www.youtube.com/watch?v=example"
)
# Create agent
researcher = Agent(
role='Video Researcher',
goal='Extract insights from video content',
backstory='Expert at analyzing video content.',
tools=[video_tool]
)
# Define task
research_task = Task(
description="""Find all mentions of machine learning
applications from the video content.""",
agent=researcher
)
# The tool will use:
# {
# "search_query": "machine learning applications"
# }
# Create crew
crew = Crew(
agents=[researcher],
tasks=[research_task]
)
# Execute
result = crew.kickoff()
```
## Advanced Usage
### Dynamic Video Selection
```python
# Initialize without video URL
flexible_tool = YoutubeVideoSearchTool()
# Search different videos
tech_results = flexible_tool.run(
search_query="quantum computing",
youtube_video_url="https://youtube.com/watch?v=tech123"
)
science_results = flexible_tool.run(
search_query="particle physics",
youtube_video_url="https://youtube.com/watch?v=science456"
)
```
### Multiple Video Analysis
```python
# Create tools for different videos
tech_tool = YoutubeVideoSearchTool(
youtube_video_url="https://youtube.com/watch?v=tech123"
)
science_tool = YoutubeVideoSearchTool(
youtube_video_url="https://youtube.com/watch?v=science456"
)
# Create agent with multiple tools
analyst = Agent(
role='Content Analyst',
goal='Cross-reference multiple videos',
tools=[tech_tool, science_tool]
)
```
### Error Handling Example
```python
try:
video_tool = YoutubeVideoSearchTool()
results = video_tool.run(
search_query="key concepts",
youtube_video_url="https://youtube.com/watch?v=example"
)
print(results)
except Exception as e:
print(f"Error processing video: {str(e)}")
```
## Notes
- Inherits from RagTool
- Uses embedchain for processing
- Supports semantic search
- Dynamic video specification
- Efficient content retrieval
- Thread-safe operations
- Maintains search context
- Handles video transcripts
- Processes video metadata
- Memory-efficient processing

View File

@@ -129,6 +129,7 @@ nav:
- Processes: 'core-concepts/Processes.md'
- Crews: 'core-concepts/Crews.md'
- Collaboration: 'core-concepts/Collaboration.md'
- Pipeline: 'core-concepts/Pipeline.md'
- Training: 'core-concepts/Training-Crew.md'
- Memory: 'core-concepts/Memory.md'
- Planning: 'core-concepts/Planning.md'
@@ -151,7 +152,6 @@ nav:
- Conditional Tasks: 'how-to/Conditional-Tasks.md'
- Agent Monitoring with AgentOps: 'how-to/AgentOps-Observability.md'
- Agent Monitoring with LangTrace: 'how-to/Langtrace-Observability.md'
- Agent Monitoring with OpenLIT: 'how-to/openlit-Observability.md'
- Tools Docs:
- Browserbase Web Loader: 'tools/BrowserbaseLoadTool.md'
- Code Docs RAG Search: 'tools/CodeDocsSearchTool.md'

7507
poetry.lock generated Normal file

File diff suppressed because it is too large Load Diff

View File

@@ -1,46 +1,33 @@
[project]
name = "crewai"
version = "0.86.0"
version = "0.76.2"
description = "Cutting-edge framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks."
readme = "README.md"
requires-python = ">=3.10,<3.13"
requires-python = ">=3.10,<=3.13"
authors = [
{ name = "Joao Moura", email = "joao@crewai.com" }
]
dependencies = [
# Core Dependencies
"pydantic>=2.4.2",
"langchain>=0.2.16",
"openai>=1.13.3",
"litellm>=1.44.22",
"instructor>=1.3.3",
# Text Processing
"pdfplumber>=0.11.4",
"regex>=2024.9.11",
# Telemetry and Monitoring
"opentelemetry-api>=1.22.0",
"opentelemetry-sdk>=1.22.0",
"opentelemetry-exporter-otlp-proto-http>=1.22.0",
# Data Handling
"chromadb>=0.5.23",
"openpyxl>=3.1.5",
"pyvis>=0.3.2",
# Authentication and Security
"auth0-python>=4.7.1",
"python-dotenv>=1.0.0",
# Configuration and Utils
"instructor>=1.3.3",
"regex>=2024.9.11",
"crewai-tools>=0.13.2",
"click>=8.1.7",
"python-dotenv>=1.0.0",
"appdirs>=1.4.4",
"jsonref>=1.1.0",
"json-repair>=0.25.2",
"auth0-python>=4.7.1",
"litellm>=1.44.22",
"pyvis>=0.3.2",
"uv>=0.4.25",
"tomli-w>=1.1.0",
"tomli>=2.0.2",
"blinker>=1.9.0",
"chromadb>=0.4.24",
]
[project.urls]
@@ -49,29 +36,12 @@ Documentation = "https://docs.crewai.com"
Repository = "https://github.com/crewAIInc/crewAI"
[project.optional-dependencies]
tools = ["crewai-tools>=0.17.0"]
embeddings = [
"tiktoken~=0.7.0"
]
tools = ["crewai-tools>=0.13.2"]
agentops = ["agentops>=0.3.0"]
fastembed = ["fastembed>=0.4.1"]
pdfplumber = [
"pdfplumber>=0.11.4",
]
pandas = [
"pandas>=2.2.3",
]
openpyxl = [
"openpyxl>=3.1.5",
]
mem0 = ["mem0ai>=0.1.29"]
docling = [
"docling>=2.12.0",
]
[tool.uv]
dev-dependencies = [
"ruff>=0.8.2",
"ruff>=0.4.10",
"mypy>=1.10.0",
"pre-commit>=3.6.0",
"mkdocs>=1.4.3",
@@ -81,6 +51,7 @@ dev-dependencies = [
"mkdocs-material-extensions>=1.3.1",
"pillow>=10.2.0",
"cairosvg>=2.7.1",
"crewai-tools>=0.13.2",
"pytest>=8.0.0",
"pytest-vcr>=1.0.2",
"python-dotenv>=1.0.0",

View File

@@ -1,11 +1,11 @@
import warnings
from crewai.agent import Agent
from crewai.crew import Crew
from crewai.flow.flow import Flow
from crewai.knowledge.knowledge import Knowledge
from crewai.llm import LLM
from crewai.pipeline import Pipeline
from crewai.process import Process
from crewai.routers import Router
from crewai.task import Task
warnings.filterwarnings(
@@ -14,13 +14,5 @@ warnings.filterwarnings(
category=UserWarning,
module="pydantic.main",
)
__version__ = "0.86.0"
__all__ = [
"Agent",
"Crew",
"Process",
"Task",
"LLM",
"Flow",
"Knowledge",
]
__version__ = "0.76.2"
__all__ = ["Agent", "Crew", "Process", "Task", "Pipeline", "Router", "LLM", "Flow"]

View File

@@ -1,42 +1,43 @@
import os
import shutil
import subprocess
from typing import Any, Dict, List, Literal, Optional, Union
from inspect import signature
from typing import Any, List, Literal, Optional, Union
from pydantic import Field, InstanceOf, PrivateAttr, model_validator
from crewai.agents import CacheHandler
from crewai.agents.agent_builder.base_agent import BaseAgent
from crewai.agents.crew_agent_executor import CrewAgentExecutor
from crewai.cli.constants import ENV_VARS, LITELLM_PARAMS
from crewai.knowledge.knowledge import Knowledge
from crewai.knowledge.source.base_knowledge_source import BaseKnowledgeSource
from crewai.knowledge.utils.knowledge_utils import extract_knowledge_context
from crewai.llm import LLM
from crewai.memory.contextual.contextual_memory import ContextualMemory
from crewai.task import Task
from crewai.tools import BaseTool
from crewai.tools.agent_tools.agent_tools import AgentTools
from crewai.tools.base_tool import Tool
from crewai.tools.agent_tools import AgentTools
from crewai.utilities import Converter, Prompts
from crewai.utilities.constants import TRAINED_AGENTS_DATA_FILE, TRAINING_DATA_FILE
from crewai.utilities.converter import generate_model_description
from crewai.utilities.token_counter_callback import TokenCalcHandler
from crewai.utilities.training_handler import CrewTrainingHandler
agentops = None
try:
import agentops # type: ignore # Name "agentops" is already defined
from agentops import track_agent # type: ignore
except ImportError:
def track_agent():
def mock_agent_ops_provider():
def track_agent(*args, **kwargs):
def noop(f):
return f
return noop
return track_agent
agentops = None
if os.environ.get("AGENTOPS_API_KEY"):
try:
from agentops import track_agent
except ImportError:
track_agent = mock_agent_ops_provider()
else:
track_agent = mock_agent_ops_provider()
@track_agent()
class Agent(BaseAgent):
@@ -50,7 +51,6 @@ class Agent(BaseAgent):
role: The role of the agent.
goal: The objective of the agent.
backstory: The backstory of the agent.
knowledge: The knowledge base of the agent.
config: Dict representation of agent configuration.
llm: The language model that will run the agent.
function_calling_llm: The language model that will handle the tool calling for this agent, it overrides the crew function_calling_llm.
@@ -61,7 +61,6 @@ class Agent(BaseAgent):
allow_delegation: Whether the agent is allowed to delegate tasks to other agents.
tools: Tools at agents disposal
step_callback: Callback to be executed after each step of the agent execution.
knowledge_sources: Knowledge sources for the agent.
"""
_times_executed: int = PrivateAttr(default=0)
@@ -115,35 +114,14 @@ class Agent(BaseAgent):
default=2,
description="Maximum number of retries for an agent to execute a task when an error occurs.",
)
multimodal: bool = Field(
default=False,
description="Whether the agent is multimodal.",
)
code_execution_mode: Literal["safe", "unsafe"] = Field(
default="safe",
description="Mode for code execution: 'safe' (using Docker) or 'unsafe' (direct execution).",
)
embedder_config: Optional[Dict[str, Any]] = Field(
default=None,
description="Embedder configuration for the agent.",
)
knowledge_sources: Optional[List[BaseKnowledgeSource]] = Field(
default=None,
description="Knowledge sources for the agent.",
)
_knowledge: Optional[Knowledge] = PrivateAttr(
default=None,
)
@model_validator(mode="after")
def post_init_setup(self):
self._set_knowledge()
self.agent_ops_agent_name = self.role
unaccepted_attributes = [
"AWS_ACCESS_KEY_ID",
"AWS_SECRET_ACCESS_KEY",
"AWS_REGION_NAME",
]
# Handle different cases for self.llm
if isinstance(self.llm, str):
@@ -153,12 +131,8 @@ class Agent(BaseAgent):
# If it's already an LLM instance, keep it as is
pass
elif self.llm is None:
# Determine the model name from environment variables or use default
model_name = (
os.environ.get("OPENAI_MODEL_NAME")
or os.environ.get("MODEL")
or "gpt-4o-mini"
)
# If it's None, use environment variables or default
model_name = os.environ.get("OPENAI_MODEL_NAME", "gpt-4o-mini")
llm_params = {"model": model_name}
api_base = os.environ.get("OPENAI_API_BASE") or os.environ.get(
@@ -167,30 +141,9 @@ class Agent(BaseAgent):
if api_base:
llm_params["base_url"] = api_base
set_provider = model_name.split("/")[0] if "/" in model_name else "openai"
# Iterate over all environment variables to find matching API keys or use defaults
for provider, env_vars in ENV_VARS.items():
if provider == set_provider:
for env_var in env_vars:
# Check if the environment variable is set
key_name = env_var.get("key_name")
if key_name and key_name not in unaccepted_attributes:
env_value = os.environ.get(key_name)
if env_value:
key_name = key_name.lower()
for pattern in LITELLM_PARAMS:
if pattern in key_name:
key_name = pattern
break
llm_params[key_name] = env_value
# Check for default values if the environment variable is not set
elif env_var.get("default", False):
for key, value in env_var.items():
if key not in ["prompt", "key_name", "default"]:
# Only add default if the key is already set in os.environ
if key in os.environ:
llm_params[key] = value
api_key = os.environ.get("OPENAI_API_KEY")
if api_key:
llm_params["api_key"] = api_key
self.llm = LLM(**llm_params)
else:
@@ -236,26 +189,11 @@ class Agent(BaseAgent):
self.cache_handler = CacheHandler()
self.set_cache_handler(self.cache_handler)
def _set_knowledge(self):
try:
if self.knowledge_sources:
knowledge_agent_name = f"{self.role.replace(' ', '_')}"
if isinstance(self.knowledge_sources, list) and all(
isinstance(k, BaseKnowledgeSource) for k in self.knowledge_sources
):
self._knowledge = Knowledge(
sources=self.knowledge_sources,
embedder_config=self.embedder_config,
collection_name=knowledge_agent_name,
)
except (TypeError, ValueError) as e:
raise ValueError(f"Invalid Knowledge Configuration: {str(e)}")
def execute_task(
self,
task: Task,
task: Any,
context: Optional[str] = None,
tools: Optional[List[BaseTool]] = None,
tools: Optional[List[Any]] = None,
) -> str:
"""Execute a task with the agent.
@@ -272,22 +210,6 @@ class Agent(BaseAgent):
task_prompt = task.prompt()
# If the task requires output in JSON or Pydantic format,
# append specific instructions to the task prompt to ensure
# that the final answer does not include any code block markers
if task.output_json or task.output_pydantic:
# Generate the schema based on the output format
if task.output_json:
# schema = json.dumps(task.output_json, indent=2)
schema = generate_model_description(task.output_json)
elif task.output_pydantic:
schema = generate_model_description(task.output_pydantic)
task_prompt += "\n" + self.i18n.slice("formatted_task_instructions").format(
output_format=schema
)
if context:
task_prompt = self.i18n.slice("task_with_context").format(
task=task_prompt, context=context
@@ -295,32 +217,14 @@ class Agent(BaseAgent):
if self.crew and self.crew.memory:
contextual_memory = ContextualMemory(
self.crew.memory_config,
self.crew._short_term_memory,
self.crew._long_term_memory,
self.crew._entity_memory,
self.crew._user_memory,
)
memory = contextual_memory.build_context_for_task(task, context)
if memory.strip() != "":
task_prompt += self.i18n.slice("memory").format(memory=memory)
if self._knowledge:
agent_knowledge_snippets = self._knowledge.query([task.prompt()])
if agent_knowledge_snippets:
agent_knowledge_context = extract_knowledge_context(
agent_knowledge_snippets
)
if agent_knowledge_context:
task_prompt += agent_knowledge_context
if self.crew:
knowledge_snippets = self.crew.query_knowledge([task.prompt()])
if knowledge_snippets:
crew_knowledge_context = extract_knowledge_context(knowledge_snippets)
if crew_knowledge_context:
task_prompt += crew_knowledge_context
tools = tools or self.tools or []
self.create_agent_executor(tools=tools, task=task)
@@ -356,9 +260,7 @@ class Agent(BaseAgent):
return result
def create_agent_executor(
self, tools: Optional[List[BaseTool]] = None, task=None
) -> None:
def create_agent_executor(self, tools=None, task=None) -> None:
"""Create an agent executor for the agent.
Returns:
@@ -411,10 +313,6 @@ class Agent(BaseAgent):
tools = agent_tools.tools()
return tools
def get_multimodal_tools(self) -> List[Tool]:
from crewai.tools.agent_tools.add_image_tool import AddImageTool
return [AddImageTool()]
def get_code_execution_tools(self):
try:
from crewai_tools import CodeInterpreterTool
@@ -435,11 +333,11 @@ class Agent(BaseAgent):
tools_list = []
try:
# tentatively try to import from crewai_tools import BaseTool as CrewAITool
from crewai.tools import BaseTool as CrewAITool
from crewai_tools import BaseTool as CrewAITool
for tool in tools:
if isinstance(tool, CrewAITool):
tools_list.append(tool.to_structured_tool())
tools_list.append(tool.to_langchain())
else:
tools_list.append(tool)
except ModuleNotFoundError:
@@ -494,20 +392,30 @@ class Agent(BaseAgent):
return description
def _render_text_description_and_args(self, tools: List[BaseTool]) -> str:
def _render_text_description_and_args(self, tools: List[Any]) -> str:
"""Render the tool name, description, and args in plain text.
Output will be in the format of:
Output will be in the format of:
.. code-block:: markdown
.. code-block:: markdown
search: This tool is used for search, args: {"query": {"type": "string"}}
calculator: This tool is used for math, \
args: {"expression": {"type": "string"}}
args: {"expression": {"type": "string"}}
"""
tool_strings = []
for tool in tools:
tool_strings.append(tool.description)
args_schema = str(tool.model_fields)
if hasattr(tool, "func") and tool.func:
sig = signature(tool.func)
description = (
f"Tool Name: {tool.name}{sig}\nTool Description: {tool.description}"
)
else:
description = (
f"Tool Name: {tool.name}\nTool Description: {tool.description}"
)
tool_strings.append(f"{description}\nTool Arguments: {args_schema}")
return "\n".join(tool_strings)

View File

@@ -18,8 +18,6 @@ from pydantic_core import PydanticCustomError
from crewai.agents.agent_builder.utilities.base_token_process import TokenProcess
from crewai.agents.cache.cache_handler import CacheHandler
from crewai.agents.tools_handler import ToolsHandler
from crewai.tools import BaseTool
from crewai.tools.base_tool import Tool
from crewai.utilities import I18N, Logger, RPMController
from crewai.utilities.config import process_config
@@ -51,11 +49,11 @@ class BaseAgent(ABC, BaseModel):
Methods:
execute_task(task: Any, context: Optional[str] = None, tools: Optional[List[BaseTool]] = None) -> str:
execute_task(task: Any, context: Optional[str] = None, tools: Optional[List[Any]] = None) -> str:
Abstract method to execute a task.
create_agent_executor(tools=None) -> None:
Abstract method to create an agent executor.
_parse_tools(tools: List[BaseTool]) -> List[Any]:
_parse_tools(tools: List[Any]) -> List[Any]:
Abstract method to parse tools.
get_delegation_tools(agents: List["BaseAgent"]):
Abstract method to set the agents task tools for handling delegation and question asking to other agents in crew.
@@ -136,35 +134,6 @@ class BaseAgent(ABC, BaseModel):
def process_model_config(cls, values):
return process_config(values, cls)
@field_validator("tools")
@classmethod
def validate_tools(cls, tools: List[Any]) -> List[BaseTool]:
"""Validate and process the tools provided to the agent.
This method ensures that each tool is either an instance of BaseTool
or an object with 'name', 'func', and 'description' attributes. If the
tool meets these criteria, it is processed and added to the list of
tools. Otherwise, a ValueError is raised.
"""
processed_tools = []
for tool in tools:
if isinstance(tool, BaseTool):
processed_tools.append(tool)
elif (
hasattr(tool, "name")
and hasattr(tool, "func")
and hasattr(tool, "description")
):
# Tool has the required attributes, create a Tool instance
processed_tools.append(Tool.from_langchain(tool))
else:
raise ValueError(
f"Invalid tool type: {type(tool)}. "
"Tool must be an instance of BaseTool or "
"an object with 'name', 'func', and 'description' attributes."
)
return processed_tools
@model_validator(mode="after")
def validate_and_set_attributes(self):
# Validate required fields
@@ -219,7 +188,7 @@ class BaseAgent(ABC, BaseModel):
self,
task: Any,
context: Optional[str] = None,
tools: Optional[List[BaseTool]] = None,
tools: Optional[List[Any]] = None,
) -> str:
pass
@@ -228,11 +197,11 @@ class BaseAgent(ABC, BaseModel):
pass
@abstractmethod
def _parse_tools(self, tools: List[BaseTool]) -> List[BaseTool]:
def _parse_tools(self, tools: List[Any]) -> List[Any]:
pass
@abstractmethod
def get_delegation_tools(self, agents: List["BaseAgent"]) -> List[BaseTool]:
def get_delegation_tools(self, agents: List["BaseAgent"]) -> List[Any]:
"""Set the task tools that init BaseAgenTools class."""
pass

View File

@@ -3,15 +3,16 @@ from typing import TYPE_CHECKING, Optional
from crewai.memory.entity.entity_memory_item import EntityMemoryItem
from crewai.memory.long_term.long_term_memory_item import LongTermMemoryItem
from crewai.utilities import I18N
from crewai.utilities.converter import ConverterError
from crewai.utilities.evaluators.task_evaluator import TaskEvaluator
from crewai.utilities import I18N
from crewai.utilities.printer import Printer
if TYPE_CHECKING:
from crewai.agents.agent_builder.base_agent import BaseAgent
from crewai.crew import Crew
from crewai.task import Task
from crewai.agents.agent_builder.base_agent import BaseAgent
class CrewAgentExecutorMixin:
@@ -99,19 +100,14 @@ class CrewAgentExecutorMixin:
print(f"Failed to add to long term memory: {e}")
pass
def _ask_human_input(self, final_answer: str) -> str:
def _ask_human_input(self, final_answer: dict) -> str:
"""Prompt human input for final decision making."""
self._printer.print(
content=f"\033[1m\033[95m ## Final Result:\033[00m \033[92m{final_answer}\033[00m"
)
self._printer.print(
content=(
"\n\n=====\n"
"## Please provide feedback on the Final Result and the Agent's actions. "
"Respond with 'looks good' or a similar phrase when you're satisfied.\n"
"=====\n"
),
content="\n\n=====\n## Please provide feedback on the Final Result and the Agent's actions:",
color="bold_yellow",
)
return input()

View File

@@ -0,0 +1,87 @@
from abc import ABC, abstractmethod
from typing import List, Optional, Union
from pydantic import BaseModel, Field
from crewai.agents.agent_builder.base_agent import BaseAgent
from crewai.task import Task
from crewai.utilities import I18N
class BaseAgentTools(BaseModel, ABC):
"""Default tools around agent delegation"""
agents: List[BaseAgent] = Field(description="List of agents in this crew.")
i18n: I18N = Field(default=I18N(), description="Internationalization settings.")
@abstractmethod
def tools(self):
pass
def _get_coworker(self, coworker: Optional[str], **kwargs) -> Optional[str]:
coworker = coworker or kwargs.get("co_worker") or kwargs.get("coworker")
if coworker:
is_list = coworker.startswith("[") and coworker.endswith("]")
if is_list:
coworker = coworker[1:-1].split(",")[0]
return coworker
def delegate_work(
self, task: str, context: str, coworker: Optional[str] = None, **kwargs
):
"""Useful to delegate a specific task to a coworker passing all necessary context and names."""
coworker = self._get_coworker(coworker, **kwargs)
return self._execute(coworker, task, context)
def ask_question(
self, question: str, context: str, coworker: Optional[str] = None, **kwargs
):
"""Useful to ask a question, opinion or take from a coworker passing all necessary context and names."""
coworker = self._get_coworker(coworker, **kwargs)
return self._execute(coworker, question, context)
def _execute(
self, agent_name: Union[str, None], task: str, context: Union[str, None]
):
"""Execute the command."""
try:
if agent_name is None:
agent_name = ""
# It is important to remove the quotes from the agent name.
# The reason we have to do this is because less-powerful LLM's
# have difficulty producing valid JSON.
# As a result, we end up with invalid JSON that is truncated like this:
# {"task": "....", "coworker": "....
# when it should look like this:
# {"task": "....", "coworker": "...."}
agent_name = agent_name.casefold().replace('"', "").replace("\n", "")
agent = [ # type: ignore # Incompatible types in assignment (expression has type "list[BaseAgent]", variable has type "str | None")
available_agent
for available_agent in self.agents
if available_agent.role.casefold().replace("\n", "") == agent_name
]
except Exception as _:
return self.i18n.errors("agent_tool_unexsiting_coworker").format(
coworkers="\n".join(
[f"- {agent.role.casefold()}" for agent in self.agents]
)
)
if not agent:
return self.i18n.errors("agent_tool_unexsiting_coworker").format(
coworkers="\n".join(
[f"- {agent.role.casefold()}" for agent in self.agents]
)
)
agent = agent[0]
task_with_assigned_agent = Task( # type: ignore # Incompatible types in assignment (expression has type "Task", variable has type "str")
description=task,
agent=agent,
expected_output=agent.i18n.slice("manager_request"),
i18n=agent.i18n,
)
return agent.execute_task(task_with_assigned_agent, context)

View File

@@ -4,7 +4,6 @@ from crewai.types.usage_metrics import UsageMetrics
class TokenProcess:
total_tokens: int = 0
prompt_tokens: int = 0
cached_prompt_tokens: int = 0
completion_tokens: int = 0
successful_requests: int = 0
@@ -16,9 +15,6 @@ class TokenProcess:
self.completion_tokens = self.completion_tokens + tokens
self.total_tokens = self.total_tokens + tokens
def sum_cached_prompt_tokens(self, tokens: int):
self.cached_prompt_tokens = self.cached_prompt_tokens + tokens
def sum_successful_requests(self, requests: int):
self.successful_requests = self.successful_requests + requests
@@ -26,7 +22,6 @@ class TokenProcess:
return UsageMetrics(
total_tokens=self.total_tokens,
prompt_tokens=self.prompt_tokens,
cached_prompt_tokens=self.cached_prompt_tokens,
completion_tokens=self.completion_tokens,
successful_requests=self.successful_requests,
)

View File

@@ -1,6 +1,5 @@
import json
import re
from dataclasses import dataclass
from typing import Any, Dict, List, Union
from crewai.agents.agent_builder.base_agent import BaseAgent
@@ -13,10 +12,9 @@ from crewai.agents.parser import (
OutputParserException,
)
from crewai.agents.tools_handler import ToolsHandler
from crewai.tools.base_tool import BaseTool
from crewai.tools.tool_usage import ToolUsage, ToolUsageErrorException
from crewai.utilities import I18N, Printer
from crewai.utilities.constants import MAX_LLM_RETRY, TRAINING_DATA_FILE
from crewai.utilities.constants import TRAINING_DATA_FILE
from crewai.utilities.exceptions.context_window_exceeding_exception import (
LLMContextLengthExceededException,
)
@@ -24,12 +22,6 @@ from crewai.utilities.logger import Logger
from crewai.utilities.training_handler import CrewTrainingHandler
@dataclass
class ToolResult:
result: Any
result_as_answer: bool
class CrewAgentExecutor(CrewAgentExecutorMixin):
_logger: Logger = Logger()
@@ -41,7 +33,7 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
agent: BaseAgent,
prompt: dict[str, str],
max_iter: int,
tools: List[BaseTool],
tools: List[Any],
tools_names: str,
stop_words: List[str],
tools_description: str,
@@ -78,9 +70,7 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
self.iterations = 0
self.log_error_after = 3
self.have_forced_answer = False
self.tool_name_to_tool_map: Dict[str, BaseTool] = {
tool.name: tool for tool in self.tools
}
self.name_to_tool_map = {tool.name: tool for tool in self.tools}
if self.llm.stop:
self.llm.stop = list(set(self.llm.stop + self.stop))
else:
@@ -90,6 +80,7 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
if "system" in self.prompt:
system_prompt = self._format_prompt(self.prompt.get("system", ""), inputs)
user_prompt = self._format_prompt(self.prompt.get("user", ""), inputs)
self.messages.append(self._format_msg(system_prompt, role="system"))
self.messages.append(self._format_msg(user_prompt))
else:
@@ -102,8 +93,17 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
formatted_answer = self._invoke_loop()
if self.ask_for_human_input:
formatted_answer = self._handle_human_feedback(formatted_answer)
human_feedback = self._ask_human_input(formatted_answer.output)
if self.crew and self.crew._train:
self._handle_crew_training_output(formatted_answer, human_feedback)
# Making sure we only ask for it once, so disabling for the next thought loop
self.ask_for_human_input = False
self.messages.append(self._format_msg(f"Feedback: {human_feedback}"))
formatted_answer = self._invoke_loop()
if self.crew and self.crew._train:
self._handle_crew_training_output(formatted_answer)
self._create_short_term_memory(formatted_answer)
self._create_long_term_memory(formatted_answer)
return {"output": formatted_answer.output}
@@ -117,15 +117,6 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
callbacks=self.callbacks,
)
if answer is None or answer == "":
self._printer.print(
content="Received None or empty response from LLM call.",
color="red",
)
raise ValueError(
"Invalid response from LLM call - None or empty."
)
if not self.use_stop_words:
try:
self._format_answer(answer)
@@ -140,52 +131,30 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
formatted_answer = self._format_answer(answer)
if isinstance(formatted_answer, AgentAction):
tool_result = self._execute_tool_and_check_finality(
formatted_answer
)
# Directly append the result to the messages if the
# tool is "Add image to content" in case of multimodal
# agents
if formatted_answer.tool == self._i18n.tools("add_image")["name"]:
self.messages.append(tool_result.result)
continue
else:
if self.step_callback:
self.step_callback(tool_result)
formatted_answer.text += f"\nObservation: {tool_result.result}"
formatted_answer.result = tool_result.result
if tool_result.result_as_answer:
return AgentFinish(
thought="",
output=tool_result.result,
text=formatted_answer.text,
)
action_result = self._use_tool(formatted_answer)
formatted_answer.text += f"\nObservation: {action_result}"
formatted_answer.result = action_result
self._show_logs(formatted_answer)
if self.step_callback:
self.step_callback(formatted_answer)
if self.step_callback:
self.step_callback(formatted_answer)
if self._should_force_answer():
if self.have_forced_answer:
return AgentFinish(
thought="",
output=self._i18n.errors(
"force_final_answer_error"
).format(formatted_answer.text),
text=formatted_answer.text,
)
else:
formatted_answer.text += (
f'\n{self._i18n.errors("force_final_answer")}'
)
self.have_forced_answer = True
self.messages.append(
self._format_msg(formatted_answer.text, role="assistant")
)
if self._should_force_answer():
if self.have_forced_answer:
return AgentFinish(
output=self._i18n.errors(
"force_final_answer_error"
).format(formatted_answer.text),
text=formatted_answer.text,
)
else:
formatted_answer.text += (
f'\n{self._i18n.errors("force_final_answer")}'
)
self.have_forced_answer = True
self.messages.append(
self._format_msg(formatted_answer.text, role="assistant")
)
except OutputParserException as e:
self.messages.append({"role": "user", "content": e.error})
@@ -260,7 +229,7 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
content=f"\033[95m## Final Answer:\033[00m \033[92m\n{formatted_answer.output}\033[00m\n\n"
)
def _execute_tool_and_check_finality(self, agent_action: AgentAction) -> ToolResult:
def _use_tool(self, agent_action: AgentAction) -> Any:
tool_usage = ToolUsage(
tools_handler=self.tools_handler,
tools=self.tools,
@@ -276,25 +245,19 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
if isinstance(tool_calling, ToolUsageErrorException):
tool_result = tool_calling.message
return ToolResult(result=tool_result, result_as_answer=False)
else:
if tool_calling.tool_name.casefold().strip() in [
name.casefold().strip() for name in self.tool_name_to_tool_map
name.casefold().strip() for name in self.name_to_tool_map
] or tool_calling.tool_name.casefold().replace("_", " ") in [
name.casefold().strip() for name in self.tool_name_to_tool_map
name.casefold().strip() for name in self.name_to_tool_map
]:
tool_result = tool_usage.use(tool_calling, agent_action.text)
tool = self.tool_name_to_tool_map.get(tool_calling.tool_name)
if tool:
return ToolResult(
result=tool_result, result_as_answer=tool.result_as_answer
)
else:
tool_result = self._i18n.errors("wrong_tool_name").format(
tool=tool_calling.tool_name,
tools=", ".join([tool.name.casefold() for tool in self.tools]),
)
return ToolResult(result=tool_result, result_as_answer=False)
return tool_result
def _summarize_messages(self) -> None:
messages_groups = []
@@ -312,7 +275,7 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
self._i18n.slice("summarizer_system_message"), role="system"
),
self._format_msg(
self._i18n.slice("summarize_instruction").format(group=group),
self._i18n.slice("sumamrize_instruction").format(group=group),
),
],
callbacks=self.callbacks,
@@ -329,14 +292,16 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
def _handle_context_length(self) -> None:
if self.respect_context_window:
self._printer.print(
content="Context length exceeded. Summarizing content to fit the model context window.",
self._logger.log(
"debug",
"Context length exceeded. Summarizing content to fit the model context window.",
color="yellow",
)
self._summarize_messages()
else:
self._printer.print(
content="Context length exceeded. Consider using smaller text or RAG tools from crewai_tools.",
self._logger.log(
"debug",
"Context length exceeded. Consider using smaller text or RAG tools from crewai_tools.",
color="red",
)
raise SystemExit(
@@ -363,13 +328,15 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
] = result.output
training_handler.save(training_data)
else:
self._printer.print(
content="Invalid train iteration type or agent_id not in training data.",
self._logger.log(
"error",
"Invalid train iteration type or agent_id not in training data.",
color="red",
)
else:
self._printer.print(
content="Crew is None or does not have _train_iteration attribute.",
self._logger.log(
"error",
"Crew is None or does not have _train_iteration attribute.",
color="red",
)
@@ -387,13 +354,15 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
train_iteration, agent_id, training_data
)
else:
self._printer.print(
content="Invalid train iteration type. Expected int.",
self._logger.log(
"error",
"Invalid train iteration type. Expected int.",
color="red",
)
else:
self._printer.print(
content="Crew is None or does not have _train_iteration attribute.",
self._logger.log(
"error",
"Crew is None or does not have _train_iteration attribute.",
color="red",
)
@@ -407,83 +376,4 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
return CrewAgentParser(agent=self.agent).parse(answer)
def _format_msg(self, prompt: str, role: str = "user") -> Dict[str, str]:
prompt = prompt.rstrip()
return {"role": role, "content": prompt}
def _handle_human_feedback(self, formatted_answer: AgentFinish) -> AgentFinish:
"""
Handles the human feedback loop, allowing the user to provide feedback
on the agent's output and determining if additional iterations are needed.
Parameters:
formatted_answer (AgentFinish): The initial output from the agent.
Returns:
AgentFinish: The final output after incorporating human feedback.
"""
while self.ask_for_human_input:
human_feedback = self._ask_human_input(formatted_answer.output)
if self.crew and self.crew._train:
self._handle_crew_training_output(formatted_answer, human_feedback)
# Make an LLM call to verify if additional changes are requested based on human feedback
additional_changes_prompt = self._i18n.slice(
"human_feedback_classification"
).format(feedback=human_feedback)
retry_count = 0
llm_call_successful = False
additional_changes_response = None
while retry_count < MAX_LLM_RETRY and not llm_call_successful:
try:
additional_changes_response = (
self.llm.call(
[
self._format_msg(
additional_changes_prompt, role="system"
)
],
callbacks=self.callbacks,
)
.strip()
.lower()
)
llm_call_successful = True
except Exception as e:
retry_count += 1
self._printer.print(
content=f"Error during LLM call to classify human feedback: {e}. Retrying... ({retry_count}/{MAX_LLM_RETRY})",
color="red",
)
if not llm_call_successful:
self._printer.print(
content="Error processing feedback after multiple attempts.",
color="red",
)
self.ask_for_human_input = False
break
if additional_changes_response == "false":
self.ask_for_human_input = False
elif additional_changes_response == "true":
self.ask_for_human_input = True
# Add human feedback to messages
self.messages.append(self._format_msg(f"Feedback: {human_feedback}"))
# Invoke the loop again with updated messages
formatted_answer = self._invoke_loop()
if self.crew and self.crew._train:
self._handle_crew_training_output(formatted_answer)
else:
# Unexpected response
self._printer.print(
content=f"Unexpected response from LLM: '{additional_changes_response}'. Assuming no additional changes requested.",
color="red",
)
self.ask_for_human_input = False
return formatted_answer

View File

@@ -1,6 +1,5 @@
import re
from typing import Any, Union
from json_repair import repair_json
from crewai.utilities import I18N

View File

@@ -1,6 +1,6 @@
from typing import Any, Optional, Union
from ..tools.cache_tools.cache_tools import CacheTools
from ..tools.cache_tools import CacheTools
from ..tools.tool_calling import InstructorToolCalling, ToolCalling
from .cache.cache_handler import CacheHandler

View File

@@ -1,70 +0,0 @@
from pathlib import Path
import click
from crewai.cli.utils import copy_template
def add_crew_to_flow(crew_name: str) -> None:
"""Add a new crew to the current flow."""
# Check if pyproject.toml exists in the current directory
if not Path("pyproject.toml").exists():
print("This command must be run from the root of a flow project.")
raise click.ClickException(
"This command must be run from the root of a flow project."
)
# Determine the flow folder based on the current directory
flow_folder = Path.cwd()
crews_folder = flow_folder / "src" / flow_folder.name / "crews"
if not crews_folder.exists():
print("Crews folder does not exist in the current flow.")
raise click.ClickException("Crews folder does not exist in the current flow.")
# Create the crew within the flow's crews directory
create_embedded_crew(crew_name, parent_folder=crews_folder)
click.echo(
f"Crew {crew_name} added to the current flow successfully!",
)
def create_embedded_crew(crew_name: str, parent_folder: Path) -> None:
"""Create a new crew within an existing flow project."""
folder_name = crew_name.replace(" ", "_").replace("-", "_").lower()
class_name = crew_name.replace("_", " ").replace("-", " ").title().replace(" ", "")
crew_folder = parent_folder / folder_name
if crew_folder.exists():
if not click.confirm(
f"Crew {folder_name} already exists. Do you want to override it?"
):
click.secho("Operation cancelled.", fg="yellow")
return
click.secho(f"Overriding crew {folder_name}...", fg="green", bold=True)
else:
click.secho(f"Creating crew {folder_name}...", fg="green", bold=True)
crew_folder.mkdir(parents=True)
# Create config and crew.py files
config_folder = crew_folder / "config"
config_folder.mkdir(exist_ok=True)
templates_dir = Path(__file__).parent / "templates" / "crew"
config_template_files = ["agents.yaml", "tasks.yaml"]
crew_template_file = f"{folder_name}.py" # Updated file name
for file_name in config_template_files:
src_file = templates_dir / "config" / file_name
dst_file = config_folder / file_name
copy_template(src_file, dst_file, crew_name, class_name, folder_name)
src_file = templates_dir / "crew.py"
dst_file = crew_folder / crew_template_file
copy_template(src_file, dst_file, crew_name, class_name, folder_name)
click.secho(
f"Crew {crew_name} added to the flow successfully!", fg="green", bold=True
)

View File

@@ -5,8 +5,6 @@ from typing import Any, Dict
import requests
from rich.console import Console
from crewai.cli.tools.main import ToolCommand
from .constants import AUTH0_AUDIENCE, AUTH0_CLIENT_ID, AUTH0_DOMAIN
from .utils import TokenManager, validate_token
@@ -36,9 +34,7 @@ class AuthenticationCommand:
"scope": "openid",
"audience": AUTH0_AUDIENCE,
}
response = requests.post(
url=self.DEVICE_CODE_URL, data=device_code_payload, timeout=20
)
response = requests.post(url=self.DEVICE_CODE_URL, data=device_code_payload)
response.raise_for_status()
return response.json()
@@ -58,31 +54,14 @@ class AuthenticationCommand:
attempts = 0
while True and attempts < 5:
response = requests.post(self.TOKEN_URL, data=token_payload, timeout=30)
response = requests.post(self.TOKEN_URL, data=token_payload)
token_data = response.json()
if response.status_code == 200:
validate_token(token_data["id_token"])
expires_in = 360000 # Token expiration time in seconds
self.token_manager.save_tokens(token_data["access_token"], expires_in)
try:
ToolCommand().login()
except Exception:
console.print(
"\n[bold yellow]Warning:[/bold yellow] Authentication with the Tool Repository failed.",
style="yellow",
)
console.print(
"Other features will work normally, but you may experience limitations "
"with downloading and publishing tools."
"\nRun [bold]crewai login[/bold] to try logging in again.\n",
style="yellow",
)
console.print(
"\n[bold green]Welcome to CrewAI Enterprise![/bold green]\n"
)
console.print("\nWelcome to CrewAI+ !!", style="green")
return
if token_data["error"] not in ("authorization_pending", "slow_down"):

View File

@@ -1,9 +0,0 @@
from .utils import TokenManager
def get_auth_token() -> str:
"""Get the authentication token."""
access_token = TokenManager().get_token()
if not access_token:
raise Exception()
return access_token

View File

@@ -1,11 +1,11 @@
from importlib.metadata import version as get_version
from typing import Optional
import click
import pkg_resources
from crewai.cli.add_crew_to_flow import add_crew_to_flow
from crewai.cli.create_crew import create_crew
from crewai.cli.create_flow import create_flow
from crewai.cli.create_pipeline import create_pipeline
from crewai.memory.storage.kickoff_task_outputs_storage import (
KickoffTaskOutputsSQLiteStorage,
)
@@ -25,24 +25,27 @@ from .update_crew import update_crew
@click.group()
@click.version_option(get_version("crewai"))
def crewai():
"""Top-level command group for crewai."""
@crewai.command()
@click.argument("type", type=click.Choice(["crew", "flow"]))
@click.argument("type", type=click.Choice(["crew", "pipeline", "flow"]))
@click.argument("name")
@click.option("--provider", type=str, help="The provider to use for the crew")
@click.option("--skip_provider", is_flag=True, help="Skip provider validation")
def create(type, name, provider, skip_provider=False):
"""Create a new crew, or flow."""
"""Create a new crew, pipeline, or flow."""
if type == "crew":
create_crew(name, provider, skip_provider)
elif type == "pipeline":
create_pipeline(name)
elif type == "flow":
create_flow(name)
else:
click.secho("Error: Invalid type. Must be 'crew' or 'flow'.", fg="red")
click.secho(
"Error: Invalid type. Must be 'crew', 'pipeline', or 'flow'.", fg="red"
)
@crewai.command()
@@ -51,17 +54,14 @@ def create(type, name, provider, skip_provider=False):
)
def version(tools):
"""Show the installed version of crewai."""
try:
crewai_version = get_version("crewai")
except Exception:
crewai_version = "unknown version"
crewai_version = pkg_resources.get_distribution("crewai").version
click.echo(f"crewai version: {crewai_version}")
if tools:
try:
tools_version = get_version("crewai")
tools_version = pkg_resources.get_distribution("crewai-tools").version
click.echo(f"crewai tools version: {tools_version}")
except Exception:
except pkg_resources.DistributionNotFound:
click.echo("crewai tools not installed")
@@ -135,7 +135,6 @@ def log_tasks_outputs() -> None:
@click.option("-l", "--long", is_flag=True, help="Reset LONG TERM memory")
@click.option("-s", "--short", is_flag=True, help="Reset SHORT TERM memory")
@click.option("-e", "--entities", is_flag=True, help="Reset ENTITIES memory")
@click.option("-kn", "--knowledge", is_flag=True, help="Reset KNOWLEDGE storage")
@click.option(
"-k",
"--kickoff-outputs",
@@ -143,24 +142,17 @@ def log_tasks_outputs() -> None:
help="Reset LATEST KICKOFF TASK OUTPUTS",
)
@click.option("-a", "--all", is_flag=True, help="Reset ALL memories")
def reset_memories(
long: bool,
short: bool,
entities: bool,
knowledge: bool,
kickoff_outputs: bool,
all: bool,
) -> None:
def reset_memories(long, short, entities, kickoff_outputs, all):
"""
Reset the crew memories (long, short, entity, latest_crew_kickoff_ouputs). This will delete all the data saved.
"""
try:
if not all and not (long or short or entities or knowledge or kickoff_outputs):
if not all and not (long or short or entities or kickoff_outputs):
click.echo(
"Please specify at least one memory type to reset using the appropriate flags."
)
return
reset_memories_command(long, short, entities, knowledge, kickoff_outputs, all)
reset_memories_command(long, short, entities, kickoff_outputs, all)
except Exception as e:
click.echo(f"An error occurred while resetting memories: {e}", err=True)
@@ -186,16 +178,10 @@ def test(n_iterations: int, model: str):
evaluate_crew(n_iterations, model)
@crewai.command(
context_settings=dict(
ignore_unknown_options=True,
allow_extra_args=True,
)
)
@click.pass_context
def install(context):
@crewai.command()
def install():
"""Install the Crew."""
install_crew(context.args)
install_crew()
@crewai.command()
@@ -334,13 +320,5 @@ def flow_plot():
plot_flow()
@flow.command(name="add-crew")
@click.argument("crew_name")
def flow_add_crew(crew_name):
"""Add a crew to an existing flow."""
click.echo(f"Adding crew {crew_name} to the flow")
add_crew_to_flow(crew_name)
if __name__ == "__main__":
crewai()

View File

@@ -1,9 +1,8 @@
import requests
from requests.exceptions import JSONDecodeError
from rich.console import Console
from crewai.cli.authentication.token import get_auth_token
from crewai.cli.plus_api import PlusAPI
from crewai.cli.utils import get_auth_token
from crewai.telemetry.telemetry import Telemetry
console = Console()

View File

@@ -1,44 +0,0 @@
import json
from pathlib import Path
from typing import Optional
from pydantic import BaseModel, Field
DEFAULT_CONFIG_PATH = Path.home() / ".config" / "crewai" / "settings.json"
class Settings(BaseModel):
tool_repository_username: Optional[str] = Field(
None, description="Username for interacting with the Tool Repository"
)
tool_repository_password: Optional[str] = Field(
None, description="Password for interacting with the Tool Repository"
)
config_path: Path = Field(default=DEFAULT_CONFIG_PATH, exclude=True)
def __init__(self, config_path: Path = DEFAULT_CONFIG_PATH, **data):
"""Load Settings from config path"""
config_path.parent.mkdir(parents=True, exist_ok=True)
file_data = {}
if config_path.is_file():
try:
with config_path.open("r") as f:
file_data = json.load(f)
except json.JSONDecodeError:
file_data = {}
merged_data = {**file_data, **data}
super().__init__(config_path=config_path, **merged_data)
def dump(self) -> None:
"""Save current settings to settings.json"""
if self.config_path.is_file():
with self.config_path.open("r") as f:
existing_data = json.load(f)
else:
existing_data = {}
updated_data = {**existing_data, **self.model_dump(exclude_unset=True)}
with self.config_path.open("w") as f:
json.dump(updated_data, f, indent=4)

View File

@@ -1,164 +1,19 @@
ENV_VARS = {
"openai": [
{
"prompt": "Enter your OPENAI API key (press Enter to skip)",
"key_name": "OPENAI_API_KEY",
}
],
"anthropic": [
{
"prompt": "Enter your ANTHROPIC API key (press Enter to skip)",
"key_name": "ANTHROPIC_API_KEY",
}
],
"gemini": [
{
"prompt": "Enter your GEMINI API key (press Enter to skip)",
"key_name": "GEMINI_API_KEY",
}
],
"groq": [
{
"prompt": "Enter your GROQ API key (press Enter to skip)",
"key_name": "GROQ_API_KEY",
}
],
"watson": [
{
"prompt": "Enter your WATSONX URL (press Enter to skip)",
"key_name": "WATSONX_URL",
},
{
"prompt": "Enter your WATSONX API Key (press Enter to skip)",
"key_name": "WATSONX_APIKEY",
},
{
"prompt": "Enter your WATSONX Project Id (press Enter to skip)",
"key_name": "WATSONX_PROJECT_ID",
},
],
"ollama": [
{
"default": True,
"API_BASE": "http://localhost:11434",
}
],
"bedrock": [
{
"prompt": "Enter your AWS Access Key ID (press Enter to skip)",
"key_name": "AWS_ACCESS_KEY_ID",
},
{
"prompt": "Enter your AWS Secret Access Key (press Enter to skip)",
"key_name": "AWS_SECRET_ACCESS_KEY",
},
{
"prompt": "Enter your AWS Region Name (press Enter to skip)",
"key_name": "AWS_REGION_NAME",
},
],
"azure": [
{
"prompt": "Enter your Azure deployment name (must start with 'azure/')",
"key_name": "model",
},
{
"prompt": "Enter your AZURE API key (press Enter to skip)",
"key_name": "AZURE_API_KEY",
},
{
"prompt": "Enter your AZURE API base URL (press Enter to skip)",
"key_name": "AZURE_API_BASE",
},
{
"prompt": "Enter your AZURE API version (press Enter to skip)",
"key_name": "AZURE_API_VERSION",
},
],
"cerebras": [
{
"prompt": "Enter your Cerebras model name (must start with 'cerebras/')",
"key_name": "model",
},
{
"prompt": "Enter your Cerebras API version (press Enter to skip)",
"key_name": "CEREBRAS_API_KEY",
},
],
'openai': ['OPENAI_API_KEY'],
'anthropic': ['ANTHROPIC_API_KEY'],
'gemini': ['GEMINI_API_KEY'],
'groq': ['GROQ_API_KEY'],
'ollama': ['FAKE_KEY'],
}
PROVIDERS = [
"openai",
"anthropic",
"gemini",
"groq",
"ollama",
"watson",
"bedrock",
"azure",
"cerebras",
]
PROVIDERS = ['openai', 'anthropic', 'gemini', 'groq', 'ollama']
MODELS = {
"openai": ["gpt-4", "gpt-4o", "gpt-4o-mini", "o1-mini", "o1-preview"],
"anthropic": [
"claude-3-5-sonnet-20240620",
"claude-3-sonnet-20240229",
"claude-3-opus-20240229",
"claude-3-haiku-20240307",
],
"gemini": [
"gemini/gemini-1.5-flash",
"gemini/gemini-1.5-pro",
"gemini/gemini-gemma-2-9b-it",
"gemini/gemini-gemma-2-27b-it",
],
"groq": [
"groq/llama-3.1-8b-instant",
"groq/llama-3.1-70b-versatile",
"groq/llama-3.1-405b-reasoning",
"groq/gemma2-9b-it",
"groq/gemma-7b-it",
],
"ollama": ["ollama/llama3.1", "ollama/mixtral"],
"watson": [
"watsonx/meta-llama/llama-3-1-70b-instruct",
"watsonx/meta-llama/llama-3-1-8b-instruct",
"watsonx/meta-llama/llama-3-2-11b-vision-instruct",
"watsonx/meta-llama/llama-3-2-1b-instruct",
"watsonx/meta-llama/llama-3-2-90b-vision-instruct",
"watsonx/meta-llama/llama-3-405b-instruct",
"watsonx/mistral/mistral-large",
"watsonx/ibm/granite-3-8b-instruct",
],
"bedrock": [
"bedrock/anthropic.claude-3-5-sonnet-20240620-v1:0",
"bedrock/anthropic.claude-3-sonnet-20240229-v1:0",
"bedrock/anthropic.claude-3-haiku-20240307-v1:0",
"bedrock/anthropic.claude-3-opus-20240229-v1:0",
"bedrock/anthropic.claude-v2:1",
"bedrock/anthropic.claude-v2",
"bedrock/anthropic.claude-instant-v1",
"bedrock/meta.llama3-1-405b-instruct-v1:0",
"bedrock/meta.llama3-1-70b-instruct-v1:0",
"bedrock/meta.llama3-1-8b-instruct-v1:0",
"bedrock/meta.llama3-70b-instruct-v1:0",
"bedrock/meta.llama3-8b-instruct-v1:0",
"bedrock/amazon.titan-text-lite-v1",
"bedrock/amazon.titan-text-express-v1",
"bedrock/cohere.command-text-v14",
"bedrock/ai21.j2-mid-v1",
"bedrock/ai21.j2-ultra-v1",
"bedrock/ai21.jamba-instruct-v1:0",
"bedrock/meta.llama2-13b-chat-v1",
"bedrock/meta.llama2-70b-chat-v1",
"bedrock/mistral.mistral-7b-instruct-v0:2",
"bedrock/mistral.mixtral-8x7b-instruct-v0:1",
],
'openai': ['gpt-4', 'gpt-4o', 'gpt-4o-mini', 'o1-mini', 'o1-preview'],
'anthropic': ['claude-3-5-sonnet-20240620', 'claude-3-sonnet-20240229', 'claude-3-opus-20240229', 'claude-3-haiku-20240307'],
'gemini': ['gemini-1.5-flash', 'gemini-1.5-pro', 'gemini-gemma-2-9b-it', 'gemini-gemma-2-27b-it'],
'groq': ['llama-3.1-8b-instant', 'llama-3.1-70b-versatile', 'llama-3.1-405b-reasoning', 'gemma2-9b-it', 'gemma-7b-it'],
'ollama': ['llama3.1', 'mixtral'],
}
JSON_URL = "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"
LITELLM_PARAMS = ["api_key", "api_base", "api_version"]
JSON_URL = "https://raw.githubusercontent.com/BerriAI/litellm/main/model_prices_and_context_window.json"

View File

@@ -1,11 +1,11 @@
import shutil
import sys
from pathlib import Path
import click
from crewai.cli.constants import ENV_VARS, MODELS
from crewai.cli.constants import ENV_VARS
from crewai.cli.provider import (
PROVIDERS,
get_provider_data,
select_model,
select_provider,
@@ -29,21 +29,20 @@ def create_folder_structure(name, parent_folder=None):
click.secho("Operation cancelled.", fg="yellow")
sys.exit(0)
click.secho(f"Overriding folder {folder_name}...", fg="green", bold=True)
shutil.rmtree(folder_path) # Delete the existing folder and its contents
else:
click.secho(
f"Creating {'crew' if parent_folder else 'folder'} {folder_name}...",
fg="green",
bold=True,
)
click.secho(
f"Creating {'crew' if parent_folder else 'folder'} {folder_name}...",
fg="green",
bold=True,
)
folder_path.mkdir(parents=True)
(folder_path / "tests").mkdir(exist_ok=True)
(folder_path / "knowledge").mkdir(exist_ok=True)
if not parent_folder:
(folder_path / "src" / folder_name).mkdir(parents=True)
(folder_path / "src" / folder_name / "tools").mkdir(parents=True)
(folder_path / "src" / folder_name / "config").mkdir(parents=True)
if not folder_path.exists():
folder_path.mkdir(parents=True)
(folder_path / "tests").mkdir(exist_ok=True)
if not parent_folder:
(folder_path / "src" / folder_name).mkdir(parents=True)
(folder_path / "src" / folder_name / "tools").mkdir(parents=True)
(folder_path / "src" / folder_name / "config").mkdir(parents=True)
return folder_path, folder_name, class_name
@@ -53,14 +52,7 @@ def copy_template_files(folder_path, name, class_name, parent_folder):
templates_dir = package_dir / "templates" / "crew"
root_template_files = (
[
".gitignore",
"pyproject.toml",
"README.md",
"knowledge/user_preference.txt",
]
if not parent_folder
else []
[".gitignore", "pyproject.toml", "README.md"] if not parent_folder else []
)
tools_template_files = ["tools/custom_tool.py", "tools/__init__.py"]
config_template_files = ["config/agents.yaml", "config/tasks.yaml"]
@@ -100,10 +92,7 @@ def create_crew(name, provider=None, skip_provider=False, parent_folder=None):
existing_provider = None
for provider, env_keys in ENV_VARS.items():
if any(
"key_name" in details and details["key_name"] in env_vars
for details in env_keys
):
if any(key in env_vars for key in env_keys):
existing_provider = provider
break
@@ -129,56 +118,53 @@ def create_crew(name, provider=None, skip_provider=False, parent_folder=None):
"No provider selected. Please try again or press 'q' to exit.", fg="red"
)
# Check if the selected provider has predefined models
if selected_provider in MODELS and MODELS[selected_provider]:
while True:
selected_model = select_model(selected_provider, provider_models)
if selected_model is None: # User typed 'q'
click.secho("Exiting...", fg="yellow")
sys.exit(0)
if selected_model: # Valid selection
break
click.secho(
"No model selected. Please try again or press 'q' to exit.",
fg="red",
)
env_vars["MODEL"] = selected_model
# Check if the selected provider requires API keys
if selected_provider in ENV_VARS:
provider_env_vars = ENV_VARS[selected_provider]
for details in provider_env_vars:
if details.get("default", False):
# Automatically add default key-value pairs
for key, value in details.items():
if key not in ["prompt", "key_name", "default"]:
env_vars[key] = value
elif "key_name" in details:
# Prompt for non-default key-value pairs
prompt = details["prompt"]
key_name = details["key_name"]
api_key_value = click.prompt(prompt, default="", show_default=False)
if api_key_value.strip():
env_vars[key_name] = api_key_value
if env_vars:
write_env_file(folder_path, env_vars)
click.secho("API keys and model saved to .env file", fg="green")
else:
while True:
selected_model = select_model(selected_provider, provider_models)
if selected_model is None: # User typed 'q'
click.secho("Exiting...", fg="yellow")
sys.exit(0)
if selected_model: # Valid selection
break
click.secho(
"No API keys provided. Skipping .env file creation.", fg="yellow"
"No model selected. Please try again or press 'q' to exit.", fg="red"
)
click.secho(f"Selected model: {env_vars.get('MODEL', 'N/A')}", fg="green")
if selected_provider in PROVIDERS:
api_key_var = ENV_VARS[selected_provider][0]
else:
api_key_var = click.prompt(
f"Enter the environment variable name for your {selected_provider.capitalize()} API key",
type=str,
default="",
)
api_key_value = ""
click.echo(
f"Enter your {selected_provider.capitalize()} API key (press Enter to skip): ",
nl=False,
)
try:
api_key_value = input()
except (KeyboardInterrupt, EOFError):
api_key_value = ""
if api_key_value.strip():
env_vars = {api_key_var: api_key_value}
write_env_file(folder_path, env_vars)
click.secho("API key saved to .env file", fg="green")
else:
click.secho(
"No API key provided. Skipping .env file creation.", fg="yellow"
)
env_vars["MODEL"] = selected_model
click.secho(f"Selected model: {selected_model}", fg="green")
package_dir = Path(__file__).parent
templates_dir = package_dir / "templates" / "crew"
root_template_files = (
[".gitignore", "pyproject.toml", "README.md", "knowledge/user_preference.txt"]
if not parent_folder
else []
[".gitignore", "pyproject.toml", "README.md"] if not parent_folder else []
)
tools_template_files = ["tools/custom_tool.py", "tools/__init__.py"]
config_template_files = ["config/agents.yaml", "config/tasks.yaml"]

View File

@@ -0,0 +1,107 @@
import shutil
from pathlib import Path
import click
def create_pipeline(name, router=False):
"""Create a new pipeline project."""
folder_name = name.replace(" ", "_").replace("-", "_").lower()
class_name = name.replace("_", " ").replace("-", " ").title().replace(" ", "")
click.secho(f"Creating pipeline {folder_name}...", fg="green", bold=True)
project_root = Path(folder_name)
if project_root.exists():
click.secho(f"Error: Folder {folder_name} already exists.", fg="red")
return
# Create directory structure
(project_root / "src" / folder_name).mkdir(parents=True)
(project_root / "src" / folder_name / "pipelines").mkdir(parents=True)
(project_root / "src" / folder_name / "crews").mkdir(parents=True)
(project_root / "src" / folder_name / "tools").mkdir(parents=True)
(project_root / "tests").mkdir(exist_ok=True)
# Create .env file
with open(project_root / ".env", "w") as file:
file.write("OPENAI_API_KEY=YOUR_API_KEY")
package_dir = Path(__file__).parent
template_folder = "pipeline_router" if router else "pipeline"
templates_dir = package_dir / "templates" / template_folder
# List of template files to copy
root_template_files = [".gitignore", "pyproject.toml", "README.md"]
src_template_files = ["__init__.py", "main.py"]
tools_template_files = ["tools/__init__.py", "tools/custom_tool.py"]
if router:
crew_folders = [
"classifier_crew",
"normal_crew",
"urgent_crew",
]
pipelines_folders = [
"pipelines/__init__.py",
"pipelines/pipeline_classifier.py",
"pipelines/pipeline_normal.py",
"pipelines/pipeline_urgent.py",
]
else:
crew_folders = [
"research_crew",
"write_linkedin_crew",
"write_x_crew",
]
pipelines_folders = ["pipelines/__init__.py", "pipelines/pipeline.py"]
def process_file(src_file, dst_file):
with open(src_file, "r") as file:
content = file.read()
content = content.replace("{{name}}", name)
content = content.replace("{{crew_name}}", class_name)
content = content.replace("{{folder_name}}", folder_name)
content = content.replace("{{pipeline_name}}", class_name)
with open(dst_file, "w") as file:
file.write(content)
# Copy and process root template files
for file_name in root_template_files:
src_file = templates_dir / file_name
dst_file = project_root / file_name
process_file(src_file, dst_file)
# Copy and process src template files
for file_name in src_template_files:
src_file = templates_dir / file_name
dst_file = project_root / "src" / folder_name / file_name
process_file(src_file, dst_file)
# Copy tools files
for file_name in tools_template_files:
src_file = templates_dir / file_name
dst_file = project_root / "src" / folder_name / file_name
shutil.copy(src_file, dst_file)
# Copy pipelines folders
for file_name in pipelines_folders:
src_file = templates_dir / file_name
dst_file = project_root / "src" / folder_name / file_name
process_file(src_file, dst_file)
# Copy crew folders
for crew_folder in crew_folders:
src_crew_folder = templates_dir / "crews" / crew_folder
dst_crew_folder = project_root / "src" / folder_name / "crews" / crew_folder
if src_crew_folder.exists():
shutil.copytree(src_crew_folder, dst_crew_folder)
else:
click.secho(
f"Warning: Crew folder {crew_folder} not found in template.",
fg="yellow",
)
click.secho(f"Pipeline {name} created successfully!", fg="green", bold=True)

View File

@@ -3,13 +3,12 @@ import subprocess
import click
def install_crew(proxy_options: list[str]) -> None:
def install_crew() -> None:
"""
Install the crew by running the UV command to lock and install.
"""
try:
command = ["uv", "sync"] + proxy_options
subprocess.run(command, check=True, capture_output=False, text=True)
subprocess.run(["uv", "sync"], check=True, capture_output=False, text=True)
except subprocess.CalledProcessError as e:
click.echo(f"An error occurred while running the crew: {e}", err=True)

View File

@@ -1,10 +1,8 @@
from os import getenv
from typing import Optional
from urllib.parse import urljoin
import requests
from crewai.cli.version import get_crewai_version
from os import getenv
from crewai.cli.utils import get_crewai_version
from urllib.parse import urljoin
class PlusAPI:

View File

@@ -164,7 +164,7 @@ def fetch_provider_data(cache_file):
- dict or None: The fetched provider data or None if the operation fails.
"""
try:
response = requests.get(JSON_URL, stream=True, timeout=60)
response = requests.get(JSON_URL, stream=True, timeout=10)
response.raise_for_status()
data = download_data(response)
with open(cache_file, "w") as f:

View File

@@ -1,22 +1,13 @@
import subprocess
import click
from crewai.knowledge.storage.knowledge_storage import KnowledgeStorage
from crewai.memory.entity.entity_memory import EntityMemory
from crewai.memory.long_term.long_term_memory import LongTermMemory
from crewai.memory.short_term.short_term_memory import ShortTermMemory
from crewai.utilities.task_output_storage_handler import TaskOutputStorageHandler
def reset_memories_command(
long,
short,
entity,
knowledge,
kickoff_outputs,
all,
) -> None:
def reset_memories_command(long, short, entity, kickoff_outputs, all) -> None:
"""
Reset the crew memories.
@@ -26,7 +17,6 @@ def reset_memories_command(
entity (bool): Whether to reset the entity memory.
kickoff_outputs (bool): Whether to reset the latest kickoff task outputs.
all (bool): Whether to reset all memories.
knowledge (bool): Whether to reset the knowledge.
"""
try:
@@ -35,7 +25,6 @@ def reset_memories_command(
EntityMemory().reset()
LongTermMemory().reset()
TaskOutputStorageHandler().reset()
KnowledgeStorage().reset()
click.echo("All memories have been reset.")
else:
if long:
@@ -51,9 +40,6 @@ def reset_memories_command(
if kickoff_outputs:
TaskOutputStorageHandler().reset()
click.echo("Latest Kickoff outputs stored has been reset.")
if knowledge:
KnowledgeStorage().reset()
click.echo("Knowledge has been reset.")
except subprocess.CalledProcessError as e:
click.echo(f"An error occurred while resetting the memories: {e}", err=True)

View File

@@ -1,10 +1,10 @@
import subprocess
import click
import tomllib
from packaging import version
from crewai.cli.utils import read_toml
from crewai.cli.version import get_crewai_version
from crewai.cli.utils import get_crewai_version
def run_crew() -> None:
@@ -15,9 +15,10 @@ def run_crew() -> None:
crewai_version = get_crewai_version()
min_required_version = "0.71.0"
pyproject_data = read_toml()
with open("pyproject.toml", "rb") as f:
data = tomllib.load(f)
if pyproject_data.get("tool", {}).get("poetry") and (
if data.get("tool", {}).get("poetry") and (
version.parse(crewai_version) < version.parse(min_required_version)
):
click.secho(
@@ -25,6 +26,7 @@ def run_crew() -> None:
f"Please run `crewai update` to update your pyproject.toml to use uv.",
fg="red",
)
print()
try:
subprocess.run(command, capture_output=False, text=True, check=True)
@@ -33,7 +35,10 @@ def run_crew() -> None:
click.echo(f"An error occurred while running the crew: {e}", err=True)
click.echo(e.output, err=True, nl=True)
if pyproject_data.get("tool", {}).get("poetry"):
with open("pyproject.toml", "rb") as f:
data = tomllib.load(f)
if data.get("tool", {}).get("poetry"):
click.secho(
"It's possible that you are using an old version of crewAI that uses poetry, please run `crewai update` to update your pyproject.toml to use uv.",
fg="yellow",

View File

@@ -4,7 +4,7 @@ Welcome to the {{crew_name}} Crew project, powered by [crewAI](https://crewai.co
## Installation
Ensure you have Python >=3.10 <3.13 installed on your system. This project uses [UV](https://docs.astral.sh/uv/) for dependency management and package handling, offering a seamless setup and execution experience.
Ensure you have Python >=3.10 <=3.13 installed on your system. This project uses [UV](https://docs.astral.sh/uv/) for dependency management and package handling, offering a seamless setup and execution experience.
First, if you haven't already, install uv:

View File

@@ -12,6 +12,6 @@ reporting_task:
Review the context you got and expand each topic into a full section for a report.
Make sure the report is detailed and contains any and all relevant information.
expected_output: >
A fully fledged report with the main topics, each with a full section of information.
A fully fledge reports with the mains topics, each with a full section of information.
Formatted as markdown without '```'
agent: reporting_analyst

View File

@@ -1,26 +1,21 @@
from crewai import Agent, Crew, Process, Task
from crewai.project import CrewBase, agent, crew, task
# If you want to run a snippet of code before or after the crew starts,
# you can use the @before_kickoff and @after_kickoff decorators
# https://docs.crewai.com/concepts/crews#example-crew-class-with-decorators
# Uncomment the following line to use an example of a custom tool
# from {{folder_name}}.tools.custom_tool import MyCustomTool
# Check our tools documentations for more information on how to use them
# from crewai_tools import SerperDevTool
@CrewBase
class {{crew_name}}():
class {{crew_name}}Crew():
"""{{crew_name}} crew"""
# Learn more about YAML configuration files here:
# Agents: https://docs.crewai.com/concepts/agents#yaml-configuration-recommended
# Tasks: https://docs.crewai.com/concepts/tasks#yaml-configuration-recommended
agents_config = 'config/agents.yaml'
tasks_config = 'config/tasks.yaml'
# If you would like to add tools to your agents, you can learn more about it here:
# https://docs.crewai.com/concepts/agents#agent-tools
@agent
def researcher(self) -> Agent:
return Agent(
config=self.agents_config['researcher'],
# tools=[MyCustomTool()], # Example of custom tool, loaded on the beginning of file
verbose=True
)
@@ -31,9 +26,6 @@ class {{crew_name}}():
verbose=True
)
# To learn more about structured task outputs,
# task dependencies, and task callbacks, check out the documentation:
# https://docs.crewai.com/concepts/tasks#overview-of-a-task
@task
def research_task(self) -> Task:
return Task(
@@ -50,13 +42,10 @@ class {{crew_name}}():
@crew
def crew(self) -> Crew:
"""Creates the {{crew_name}} crew"""
# To learn how to add knowledge sources to your crew, check out the documentation:
# https://docs.crewai.com/concepts/knowledge#what-is-knowledge
return Crew(
agents=self.agents, # Automatically created by the @agent decorator
tasks=self.tasks, # Automatically created by the @task decorator
process=Process.sequential,
verbose=True,
# process=Process.hierarchical, # In case you wanna use that instead https://docs.crewai.com/how-to/Hierarchical/
)
)

View File

@@ -1,4 +0,0 @@
User name is John Doe.
User is an AI Engineer.
User is interested in AI Agents.
User is based in San Francisco, California.

View File

@@ -1,10 +1,6 @@
#!/usr/bin/env python
import sys
import warnings
from {{folder_name}}.crew import {{crew_name}}
warnings.filterwarnings("ignore", category=SyntaxWarning, module="pysbd")
from {{folder_name}}.crew import {{crew_name}}Crew
# This main file is intended to be a way for you to run your
# crew locally, so refrain from adding unnecessary logic into this file.
@@ -18,7 +14,7 @@ def run():
inputs = {
'topic': 'AI LLMs'
}
{{crew_name}}().crew().kickoff(inputs=inputs)
{{crew_name}}Crew().crew().kickoff(inputs=inputs)
def train():
@@ -29,7 +25,7 @@ def train():
"topic": "AI LLMs"
}
try:
{{crew_name}}().crew().train(n_iterations=int(sys.argv[1]), filename=sys.argv[2], inputs=inputs)
{{crew_name}}Crew().crew().train(n_iterations=int(sys.argv[1]), filename=sys.argv[2], inputs=inputs)
except Exception as e:
raise Exception(f"An error occurred while training the crew: {e}")
@@ -39,7 +35,7 @@ def replay():
Replay the crew execution from a specific task.
"""
try:
{{crew_name}}().crew().replay(task_id=sys.argv[1])
{{crew_name}}Crew().crew().replay(task_id=sys.argv[1])
except Exception as e:
raise Exception(f"An error occurred while replaying the crew: {e}")
@@ -52,7 +48,7 @@ def test():
"topic": "AI LLMs"
}
try:
{{crew_name}}().crew().test(n_iterations=int(sys.argv[1]), openai_model_name=sys.argv[2], inputs=inputs)
{{crew_name}}Crew().crew().test(n_iterations=int(sys.argv[1]), openai_model_name=sys.argv[2], inputs=inputs)
except Exception as e:
raise Exception(f"An error occurred while replaying the crew: {e}")

Some files were not shown because too many files have changed in this diff Show More