Compare commits

..

8 Commits

Author SHA1 Message Date
Devin AI
d1ca108e26 Update test docstring to trigger CI
Co-Authored-By: Joe Moura <joao@crewai.com>
2025-04-29 12:13:07 +00:00
Devin AI
4b116638ff Fix import sorting in test_encoding.py with ruff auto-fix
Co-Authored-By: Joe Moura <joao@crewai.com>
2025-04-29 12:09:28 +00:00
Devin AI
cd7deed7d8 Fix import order in test_encoding.py
Co-Authored-By: Joe Moura <joao@crewai.com>
2025-04-29 12:07:08 +00:00
Devin AI
85a408577c Fix import formatting in test_encoding.py
Co-Authored-By: Joe Moura <joao@crewai.com>
2025-04-29 12:05:18 +00:00
Devin AI
288668f1d9 Fix import sorting in test_encoding.py
Co-Authored-By: Joe Moura <joao@crewai.com>
2025-04-29 12:00:31 +00:00
Devin AI
f095f3e6c8 Implement code review suggestions: add newline parameter and UnicodeDecodeError handling
Co-Authored-By: Joe Moura <joao@crewai.com>
2025-04-29 11:58:28 +00:00
Devin AI
2995478a69 Fix import sorting in test files
Co-Authored-By: Joe Moura <joao@crewai.com>
2025-04-29 11:53:27 +00:00
Devin AI
200ec42613 Fix UnicodeDecodeError when running crewai create crew on Windows (issue #2715)
Co-Authored-By: Joe Moura <joao@crewai.com>
2025-04-29 11:48:50 +00:00
12 changed files with 247 additions and 276 deletions

158
README.md
View File

@@ -4,7 +4,7 @@
# **CrewAI** # **CrewAI**
🤖 **CrewAI**: Production-grade framework for orchestrating sophisticated AI agent systems. From simple automations to complex real-world applications, CrewAI provides precise control and deep customization. By fostering collaborative intelligence through flexible, production-ready architecture, CrewAI empowers agents to work together seamlessly, tackling complex business challenges with predictable, consistent results. 🤖 **CrewAI**: Cutting-edge framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks.
<h3> <h3>
@@ -22,17 +22,13 @@
- [Why CrewAI?](#why-crewai) - [Why CrewAI?](#why-crewai)
- [Getting Started](#getting-started) - [Getting Started](#getting-started)
- [Key Features](#key-features) - [Key Features](#key-features)
- [Understanding Flows and Crews](#understanding-flows-and-crews)
- [CrewAI vs LangGraph](#how-crewai-compares)
- [Examples](#examples) - [Examples](#examples)
- [Quick Tutorial](#quick-tutorial) - [Quick Tutorial](#quick-tutorial)
- [Write Job Descriptions](#write-job-descriptions) - [Write Job Descriptions](#write-job-descriptions)
- [Trip Planner](#trip-planner) - [Trip Planner](#trip-planner)
- [Stock Analysis](#stock-analysis) - [Stock Analysis](#stock-analysis)
- [Using Crews and Flows Together](#using-crews-and-flows-together)
- [Connecting Your Crew to a Model](#connecting-your-crew-to-a-model) - [Connecting Your Crew to a Model](#connecting-your-crew-to-a-model)
- [How CrewAI Compares](#how-crewai-compares) - [How CrewAI Compares](#how-crewai-compares)
- [Frequently Asked Questions (FAQ)](#frequently-asked-questions-faq)
- [Contribution](#contribution) - [Contribution](#contribution)
- [Telemetry](#telemetry) - [Telemetry](#telemetry)
- [License](#license) - [License](#license)
@@ -40,40 +36,10 @@
## Why CrewAI? ## Why CrewAI?
The power of AI collaboration has too much to offer. The power of AI collaboration has too much to offer.
CrewAI is a standalone framework, built from the ground up without dependencies on Langchain or other agent frameworks. It's designed to enable AI agents to assume roles, share goals, and operate in a cohesive unit - much like a well-oiled crew. Whether you're building a smart assistant platform, an automated customer service ensemble, or a multi-agent research team, CrewAI provides the backbone for sophisticated multi-agent interactions. CrewAI is designed to enable AI agents to assume roles, share goals, and operate in a cohesive unit - much like a well-oiled crew. Whether you're building a smart assistant platform, an automated customer service ensemble, or a multi-agent research team, CrewAI provides the backbone for sophisticated multi-agent interactions.
## Getting Started ## Getting Started
### Learning Resources
Learn CrewAI through our comprehensive courses:
- [Multi AI Agent Systems with CrewAI](https://www.deeplearning.ai/short-courses/multi-ai-agent-systems-with-crewai/) - Master the fundamentals of multi-agent systems
- [Practical Multi AI Agents and Advanced Use Cases](https://www.deeplearning.ai/short-courses/practical-multi-ai-agents-and-advanced-use-cases-with-crewai/) - Deep dive into advanced implementations
### Understanding Flows and Crews
CrewAI offers two powerful, complementary approaches that work seamlessly together to build sophisticated AI applications:
1. **Crews**: Teams of AI agents with true autonomy and agency, working together to accomplish complex tasks through role-based collaboration. Crews enable:
- Natural, autonomous decision-making between agents
- Dynamic task delegation and collaboration
- Specialized roles with defined goals and expertise
- Flexible problem-solving approaches
2. **Flows**: Production-ready, event-driven workflows that deliver precise control over complex automations. Flows provide:
- Fine-grained control over execution paths for real-world scenarios
- Secure, consistent state management between tasks
- Clean integration of AI agents with production Python code
- Conditional branching for complex business logic
The true power of CrewAI emerges when combining Crews and Flows. This synergy allows you to:
- Build complex, production-grade applications
- Balance autonomy with precise control
- Handle sophisticated real-world scenarios
- Maintain clean, maintainable code structure
### Getting Started with Installation
To get started with CrewAI, follow these simple steps: To get started with CrewAI, follow these simple steps:
### 1. Installation ### 1. Installation
@@ -298,16 +264,13 @@ In addition to the sequential process, you can use the hierarchical process, whi
## Key Features ## Key Features
**Note**: CrewAI is a standalone framework built from the ground up, without dependencies on Langchain or other agent frameworks. - **Role-Based Agent Design**: Customize agents with specific roles, goals, and tools.
- **Autonomous Inter-Agent Delegation**: Agents can autonomously delegate tasks and inquire amongst themselves, enhancing problem-solving efficiency.
- **Deep Customization**: Build sophisticated agents with full control over the system - from overriding inner prompts to accessing low-level APIs. Customize roles, goals, tools, and behaviors while maintaining clean abstractions. - **Flexible Task Management**: Define tasks with customizable tools and assign them to agents dynamically.
- **Autonomous Inter-Agent Delegation**: Agents can autonomously delegate tasks and inquire amongst themselves, enabling complex problem-solving in real-world scenarios. - **Processes Driven**: Currently only supports `sequential` task execution and `hierarchical` processes, but more complex processes like consensual and autonomous are being worked on.
- **Flexible Task Management**: Define and customize tasks with granular control, from simple operations to complex multi-step processes. - **Save output as file**: Save the output of individual tasks as a file, so you can use it later.
- **Production-Grade Architecture**: Support for both high-level abstractions and low-level customization, with robust error handling and state management. - **Parse output as Pydantic or Json**: Parse the output of individual tasks as a Pydantic model or as a Json if you want to.
- **Predictable Results**: Ensure consistent, accurate outputs through programmatic guardrails, agent training capabilities, and flow-based execution control. See our [documentation on guardrails](https://docs.crewai.com/how-to/guardrails/) for implementation details. - **Works with Open Source Models**: Run your crew using Open AI or open source models refer to the [Connect CrewAI to LLMs](https://docs.crewai.com/how-to/LLM-Connections/) page for details on configuring your agents' connections to models, even ones running locally!
- **Model Flexibility**: Run your crew using OpenAI or open source models with production-ready integrations. See [Connect CrewAI to LLMs](https://docs.crewai.com/how-to/LLM-Connections/) for detailed configuration options.
- **Event-Driven Flows**: Build complex, real-world workflows with precise control over execution paths, state management, and conditional logic.
- **Process Orchestration**: Achieve any workflow pattern through flows - from simple sequential and hierarchical processes to complex, custom orchestration patterns with conditional branching and parallel execution.
![CrewAI Mind Map](./docs/crewAI-mindmap.png "CrewAI Mind Map") ![CrewAI Mind Map](./docs/crewAI-mindmap.png "CrewAI Mind Map")
@@ -342,98 +305,6 @@ You can test different real life examples of AI crews in the [CrewAI-examples re
[![Stock Analysis](https://img.youtube.com/vi/e0Uj4yWdaAg/maxresdefault.jpg)](https://www.youtube.com/watch?v=e0Uj4yWdaAg "Stock Analysis") [![Stock Analysis](https://img.youtube.com/vi/e0Uj4yWdaAg/maxresdefault.jpg)](https://www.youtube.com/watch?v=e0Uj4yWdaAg "Stock Analysis")
### Using Crews and Flows Together
CrewAI's power truly shines when combining Crews with Flows to create sophisticated automation pipelines. Here's how you can orchestrate multiple Crews within a Flow:
```python
from crewai.flow.flow import Flow, listen, start, router
from crewai import Crew, Agent, Task
from pydantic import BaseModel
# Define structured state for precise control
class MarketState(BaseModel):
sentiment: str = "neutral"
confidence: float = 0.0
recommendations: list = []
class AdvancedAnalysisFlow(Flow[MarketState]):
@start()
def fetch_market_data(self):
# Demonstrate low-level control with structured state
self.state.sentiment = "analyzing"
return {"sector": "tech", "timeframe": "1W"} # These parameters match the task description template
@listen(fetch_market_data)
def analyze_with_crew(self, market_data):
# Show crew agency through specialized roles
analyst = Agent(
role="Senior Market Analyst",
goal="Conduct deep market analysis with expert insight",
backstory="You're a veteran analyst known for identifying subtle market patterns"
)
researcher = Agent(
role="Data Researcher",
goal="Gather and validate supporting market data",
backstory="You excel at finding and correlating multiple data sources"
)
analysis_task = Task(
description="Analyze {sector} sector data for the past {timeframe}",
expected_output="Detailed market analysis with confidence score",
agent=analyst
)
research_task = Task(
description="Find supporting data to validate the analysis",
expected_output="Corroborating evidence and potential contradictions",
agent=researcher
)
# Demonstrate crew autonomy
analysis_crew = Crew(
agents=[analyst, researcher],
tasks=[analysis_task, research_task],
process=Process.sequential,
verbose=True
)
return analysis_crew.kickoff(inputs=market_data) # Pass market_data as named inputs
@router(analyze_with_crew)
def determine_next_steps(self):
# Show flow control with conditional routing
if self.state.confidence > 0.8:
return "high_confidence"
elif self.state.confidence > 0.5:
return "medium_confidence"
return "low_confidence"
@listen("high_confidence")
def execute_strategy(self):
# Demonstrate complex decision making
strategy_crew = Crew(
agents=[
Agent(role="Strategy Expert",
goal="Develop optimal market strategy")
],
tasks=[
Task(description="Create detailed strategy based on analysis",
expected_output="Step-by-step action plan")
]
)
return strategy_crew.kickoff()
@listen("medium_confidence", "low_confidence")
def request_additional_analysis(self):
self.state.recommendations.append("Gather more data")
return "Additional analysis required"
```
This example demonstrates how to:
1. Use Python code for basic data operations
2. Create and execute Crews as steps in your workflow
3. Use Flow decorators to manage the sequence of operations
4. Implement conditional branching based on Crew results
## Connecting Your Crew to a Model ## Connecting Your Crew to a Model
CrewAI supports using various LLMs through a variety of connection options. By default your agents will use the OpenAI API when querying the model. However, there are several other ways to allow your agents to connect to models. For example, you can configure your agents to use a local model via the Ollama tool. CrewAI supports using various LLMs through a variety of connection options. By default your agents will use the OpenAI API when querying the model. However, there are several other ways to allow your agents to connect to models. For example, you can configure your agents to use a local model via the Ollama tool.
@@ -442,13 +313,9 @@ Please refer to the [Connect CrewAI to LLMs](https://docs.crewai.com/how-to/LLM-
## How CrewAI Compares ## How CrewAI Compares
**CrewAI's Advantage**: CrewAI combines autonomous agent intelligence with precise workflow control through its unique Crews and Flows architecture. The framework excels at both high-level orchestration and low-level customization, enabling complex, production-grade systems with granular control. **CrewAI's Advantage**: CrewAI is built with production in mind. It offers the flexibility of Autogen's conversational agents and the structured process approach of ChatDev, but without the rigidity. CrewAI's processes are designed to be dynamic and adaptable, fitting seamlessly into both development and production workflows.
- **LangGraph**: While LangGraph provides a foundation for building agent workflows, its approach requires significant boilerplate code and complex state management patterns. The framework's tight coupling with LangChain can limit flexibility when implementing custom agent behaviors or integrating with external systems. - **Autogen**: While Autogen does good in creating conversational agents capable of working together, it lacks an inherent concept of process. In Autogen, orchestrating agents' interactions requires additional programming, which can become complex and cumbersome as the scale of tasks grows.
*P.S. CrewAI demonstrates significant performance advantages over LangGraph, executing 5.76x faster in certain cases like this QA task example ([see comparison](https://github.com/crewAIInc/crewAI-examples/tree/main/Notebooks/CrewAI%20Flows%20%26%20Langgraph/QA%20Agent)) while achieving higher evaluation scores with faster completion times in certain coding tasks, like in this example ([detailed analysis](https://github.com/crewAIInc/crewAI-examples/blob/main/Notebooks/CrewAI%20Flows%20%26%20Langgraph/Coding%20Assistant/coding_assistant_eval.ipynb)).*
- **Autogen**: While Autogen excels at creating conversational agents capable of working together, it lacks an inherent concept of process. In Autogen, orchestrating agents' interactions requires additional programming, which can become complex and cumbersome as the scale of tasks grows.
- **ChatDev**: ChatDev introduced the idea of processes into the realm of AI agents, but its implementation is quite rigid. Customizations in ChatDev are limited and not geared towards production environments, which can hinder scalability and flexibility in real-world applications. - **ChatDev**: ChatDev introduced the idea of processes into the realm of AI agents, but its implementation is quite rigid. Customizations in ChatDev are limited and not geared towards production environments, which can hinder scalability and flexibility in real-world applications.
@@ -573,8 +440,5 @@ A: CrewAI uses anonymous telemetry to collect usage data for improvement purpose
### Q: Where can I find examples of CrewAI in action? ### Q: Where can I find examples of CrewAI in action?
A: You can find various real-life examples in the [CrewAI-examples repository](https://github.com/crewAIInc/crewAI-examples), including trip planners, stock analysis tools, and more. A: You can find various real-life examples in the [CrewAI-examples repository](https://github.com/crewAIInc/crewAI-examples), including trip planners, stock analysis tools, and more.
### Q: What is the difference between Crews and Flows?
A: Crews and Flows serve different but complementary purposes in CrewAI. Crews are teams of AI agents working together to accomplish specific tasks through role-based collaboration, delivering accurate and predictable results. Flows, on the other hand, are event-driven workflows that can orchestrate both Crews and regular Python code, allowing you to build complex automation pipelines with secure state management and conditional execution paths.
### Q: How can I contribute to CrewAI? ### Q: How can I contribute to CrewAI?
A: Contributions are welcome! You can fork the repository, create a new branch for your feature, add your improvement, and send a pull request. Check the Contribution section in the README for more details. A: Contributions are welcome! You can fork the repository, create a new branch for your feature, add your improvement, and send a pull request. Check the Contribution section in the README for more details.

View File

@@ -171,58 +171,6 @@ crewai reset-memories --knowledge
This is useful when you've updated your knowledge sources and want to ensure that the agents are using the most recent information. This is useful when you've updated your knowledge sources and want to ensure that the agents are using the most recent information.
## Agent-Specific Knowledge
While knowledge can be provided at the crew level using `crew.knowledge_sources`, individual agents can also have their own knowledge sources using the `knowledge_sources` parameter:
```python Code
from crewai import Agent, Task, Crew
from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource
# Create agent-specific knowledge about a product
product_specs = StringKnowledgeSource(
content="""The XPS 13 laptop features:
- 13.4-inch 4K display
- Intel Core i7 processor
- 16GB RAM
- 512GB SSD storage
- 12-hour battery life""",
metadata={"category": "product_specs"}
)
# Create a support agent with product knowledge
support_agent = Agent(
role="Technical Support Specialist",
goal="Provide accurate product information and support.",
backstory="You are an expert on our laptop products and specifications.",
knowledge_sources=[product_specs] # Agent-specific knowledge
)
# Create a task that requires product knowledge
support_task = Task(
description="Answer this customer question: {question}",
agent=support_agent
)
# Create and run the crew
crew = Crew(
agents=[support_agent],
tasks=[support_task]
)
# Get answer about the laptop's specifications
result = crew.kickoff(
inputs={"question": "What is the storage capacity of the XPS 13?"}
)
```
<Info>
Benefits of agent-specific knowledge:
- Give agents specialized information for their roles
- Maintain separation of concerns between agents
- Combine with crew-level knowledge for layered information access
</Info>
## Custom Knowledge Sources ## Custom Knowledge Sources
CrewAI allows you to create custom knowledge sources for any type of data by extending the `BaseKnowledgeSource` class. Let's create a practical example that fetches and processes space news articles. CrewAI allows you to create custom knowledge sources for any type of data by extending the `BaseKnowledgeSource` class. Let's create a practical example that fetches and processes space news articles.

View File

@@ -26,7 +26,7 @@ class CrewAgentExecutorMixin:
def _should_force_answer(self) -> bool: def _should_force_answer(self) -> bool:
"""Determine if a forced answer is required based on iteration count.""" """Determine if a forced answer is required based on iteration count."""
return self.iterations >= self.max_iter return (self.iterations >= self.max_iter) and not self.have_forced_answer
def _create_short_term_memory(self, output) -> None: def _create_short_term_memory(self, output) -> None:
"""Create and save a short-term memory item if conditions are met.""" """Create and save a short-term memory item if conditions are met."""

View File

@@ -28,7 +28,7 @@ def create_flow(name):
(project_root / "tests").mkdir(exist_ok=True) (project_root / "tests").mkdir(exist_ok=True)
# Create .env file # Create .env file
with open(project_root / ".env", "w") as file: with open(project_root / ".env", "w", encoding="utf-8", newline="\n") as file:
file.write("OPENAI_API_KEY=YOUR_API_KEY") file.write("OPENAI_API_KEY=YOUR_API_KEY")
package_dir = Path(__file__).parent package_dir = Path(__file__).parent
@@ -58,7 +58,7 @@ def create_flow(name):
content = content.replace("{{flow_name}}", class_name) content = content.replace("{{flow_name}}", class_name)
content = content.replace("{{folder_name}}", folder_name) content = content.replace("{{folder_name}}", folder_name)
with open(dst_file, "w") as file: with open(dst_file, "w", encoding="utf-8", newline="\n") as file:
file.write(content) file.write(content)
# Copy and process root template files # Copy and process root template files

View File

@@ -138,17 +138,22 @@ def load_provider_data(cache_file, cache_expiry):
def read_cache_file(cache_file): def read_cache_file(cache_file):
""" """
Reads and returns the JSON content from a cache file. Returns None if the file contains invalid JSON. Reads and returns the JSON content from a cache file. Returns None if the file contains invalid JSON
or if there's an encoding error.
Args: Args:
- cache_file (Path): The path to the cache file. - cache_file (Path): The path to the cache file.
Returns: Returns:
- dict or None: The JSON content of the cache file or None if the JSON is invalid. - dict or None: The JSON content of the cache file or None if the JSON is invalid or there's an encoding error.
""" """
try: try:
with open(cache_file, "r") as f: with open(cache_file, "r", encoding="utf-8") as f:
return json.load(f) return json.load(f)
except UnicodeDecodeError as e:
click.secho(f"Error reading cache file: Unicode decode error - {e}", fg="red")
click.secho("This may be due to file encoding issues. Try deleting the cache file and trying again.", fg="yellow")
return None
except json.JSONDecodeError: except json.JSONDecodeError:
return None return None
@@ -167,13 +172,16 @@ def fetch_provider_data(cache_file):
response = requests.get(JSON_URL, stream=True, timeout=60) response = requests.get(JSON_URL, stream=True, timeout=60)
response.raise_for_status() response.raise_for_status()
data = download_data(response) data = download_data(response)
with open(cache_file, "w") as f: with open(cache_file, "w", encoding="utf-8", newline="\n") as f:
json.dump(data, f) json.dump(data, f)
return data return data
except requests.RequestException as e: except requests.RequestException as e:
click.secho(f"Error fetching provider data: {e}", fg="red") click.secho(f"Error fetching provider data: {e}", fg="red")
except json.JSONDecodeError: except json.JSONDecodeError:
click.secho("Error parsing provider data. Invalid JSON format.", fg="red") click.secho("Error parsing provider data. Invalid JSON format.", fg="red")
except UnicodeDecodeError as e:
click.secho(f"Unicode decode error when processing provider data: {e}", fg="red")
click.secho("This may be due to encoding issues with the downloaded data.", fg="yellow")
return None return None

View File

@@ -18,19 +18,24 @@ console = Console()
def copy_template(src, dst, name, class_name, folder_name): def copy_template(src, dst, name, class_name, folder_name):
"""Copy a file from src to dst.""" """Copy a file from src to dst."""
with open(src, "r") as file: try:
content = file.read() with open(src, "r", encoding="utf-8") as file:
content = file.read()
# Interpolate the content # Interpolate the content
content = content.replace("{{name}}", name) content = content.replace("{{name}}", name)
content = content.replace("{{crew_name}}", class_name) content = content.replace("{{crew_name}}", class_name)
content = content.replace("{{folder_name}}", folder_name) content = content.replace("{{folder_name}}", folder_name)
# Write the interpolated content to the new file # Write the interpolated content to the new file
with open(dst, "w") as file: with open(dst, "w", encoding="utf-8", newline="\n") as file:
file.write(content) file.write(content)
click.secho(f" - Created {dst}", fg="green") click.secho(f" - Created {dst}", fg="green")
except UnicodeDecodeError as e:
click.secho(f"Error reading template file {src}: Unicode decode error - {e}", fg="red")
click.secho("This may be due to file encoding issues. Please ensure all template files use UTF-8 encoding.", fg="yellow")
raise
def read_toml(file_path: str = "pyproject.toml"): def read_toml(file_path: str = "pyproject.toml"):
@@ -78,7 +83,7 @@ def _get_project_attribute(
attribute = None attribute = None
try: try:
with open(pyproject_path, "r") as f: with open(pyproject_path, "r", encoding="utf-8") as f:
pyproject_content = parse_toml(f.read()) pyproject_content = parse_toml(f.read())
dependencies = ( dependencies = (
@@ -119,7 +124,7 @@ def fetch_and_json_env_file(env_file_path: str = ".env") -> dict:
"""Fetch the environment variables from a .env file and return them as a dictionary.""" """Fetch the environment variables from a .env file and return them as a dictionary."""
try: try:
# Read the .env file # Read the .env file
with open(env_file_path, "r") as f: with open(env_file_path, "r", encoding="utf-8") as f:
env_content = f.read() env_content = f.read()
# Parse the .env file content to a dictionary # Parse the .env file content to a dictionary
@@ -133,6 +138,9 @@ def fetch_and_json_env_file(env_file_path: str = ".env") -> dict:
except FileNotFoundError: except FileNotFoundError:
print(f"Error: {env_file_path} not found.") print(f"Error: {env_file_path} not found.")
except UnicodeDecodeError as e:
click.secho(f"Error reading .env file: Unicode decode error - {e}", fg="red")
click.secho("This may be due to file encoding issues. Please ensure the .env file uses UTF-8 encoding.", fg="yellow")
except Exception as e: except Exception as e:
print(f"Error reading the .env file: {e}") print(f"Error reading the .env file: {e}")
@@ -158,10 +166,15 @@ def tree_find_and_replace(directory, find, replace):
for filename in files: for filename in files:
filepath = os.path.join(path, filename) filepath = os.path.join(path, filename)
with open(filepath, "r") as file: try:
contents = file.read() with open(filepath, "r", encoding="utf-8") as file:
with open(filepath, "w") as file: contents = file.read()
file.write(contents.replace(find, replace)) with open(filepath, "w", encoding="utf-8", newline="\n") as file:
file.write(contents.replace(find, replace))
except UnicodeDecodeError as e:
click.secho(f"Error processing file {filepath}: Unicode decode error - {e}", fg="red")
click.secho("This may be due to file encoding issues. Skipping this file.", fg="yellow")
continue
if find in filename: if find in filename:
new_filename = filename.replace(find, replace) new_filename = filename.replace(find, replace)
@@ -189,11 +202,15 @@ def load_env_vars(folder_path):
env_file_path = folder_path / ".env" env_file_path = folder_path / ".env"
env_vars = {} env_vars = {}
if env_file_path.exists(): if env_file_path.exists():
with open(env_file_path, "r") as file: try:
for line in file: with open(env_file_path, "r", encoding="utf-8") as file:
key, _, value = line.strip().partition("=") for line in file:
if key and value: key, _, value = line.strip().partition("=")
env_vars[key] = value if key and value:
env_vars[key] = value
except UnicodeDecodeError as e:
click.secho(f"Error reading .env file: Unicode decode error - {e}", fg="red")
click.secho("This may be due to file encoding issues. Please ensure the .env file uses UTF-8 encoding.", fg="yellow")
return env_vars return env_vars
@@ -244,6 +261,11 @@ def write_env_file(folder_path, env_vars):
- env_vars (dict): A dictionary of environment variables to write. - env_vars (dict): A dictionary of environment variables to write.
""" """
env_file_path = folder_path / ".env" env_file_path = folder_path / ".env"
with open(env_file_path, "w") as file: try:
for key, value in env_vars.items(): with open(env_file_path, "w", encoding="utf-8", newline="\n") as file:
file.write(f"{key}={value}\n") for key, value in env_vars.items():
file.write(f"{key}={value}\n")
except Exception as e:
click.secho(f"Error writing .env file: {e}", fg="red")
click.secho("This may be due to file system permissions or other issues.", fg="yellow")
raise

View File

@@ -14,13 +14,13 @@ class Knowledge(BaseModel):
Knowledge is a collection of sources and setup for the vector store to save and query relevant context. Knowledge is a collection of sources and setup for the vector store to save and query relevant context.
Args: Args:
sources: List[BaseKnowledgeSource] = Field(default_factory=list) sources: List[BaseKnowledgeSource] = Field(default_factory=list)
storage: Optional[KnowledgeStorage] = Field(default=None) storage: KnowledgeStorage = Field(default_factory=KnowledgeStorage)
embedder_config: Optional[Dict[str, Any]] = None embedder_config: Optional[Dict[str, Any]] = None
""" """
sources: List[BaseKnowledgeSource] = Field(default_factory=list) sources: List[BaseKnowledgeSource] = Field(default_factory=list)
model_config = ConfigDict(arbitrary_types_allowed=True) model_config = ConfigDict(arbitrary_types_allowed=True)
storage: Optional[KnowledgeStorage] = Field(default=None) storage: KnowledgeStorage = Field(default_factory=KnowledgeStorage)
embedder_config: Optional[Dict[str, Any]] = None embedder_config: Optional[Dict[str, Any]] = None
collection_name: Optional[str] = None collection_name: Optional[str] = None
@@ -49,13 +49,8 @@ class Knowledge(BaseModel):
""" """
Query across all knowledge sources to find the most relevant information. Query across all knowledge sources to find the most relevant information.
Returns the top_k most relevant chunks. Returns the top_k most relevant chunks.
Raises:
ValueError: If storage is not initialized.
""" """
if self.storage is None:
raise ValueError("Storage is not initialized.")
results = self.storage.search( results = self.storage.search(
query, query,
limit, limit,

View File

@@ -22,14 +22,13 @@ class BaseFileKnowledgeSource(BaseKnowledgeSource, ABC):
default_factory=list, description="The path to the file" default_factory=list, description="The path to the file"
) )
content: Dict[Path, str] = Field(init=False, default_factory=dict) content: Dict[Path, str] = Field(init=False, default_factory=dict)
storage: Optional[KnowledgeStorage] = Field(default=None) storage: KnowledgeStorage = Field(default_factory=KnowledgeStorage)
safe_file_paths: List[Path] = Field(default_factory=list) safe_file_paths: List[Path] = Field(default_factory=list)
@field_validator("file_path", "file_paths", mode="before") @field_validator("file_path", "file_paths", mode="before")
def validate_file_path(cls, v, info): def validate_file_path(cls, v, values):
"""Validate that at least one of file_path or file_paths is provided.""" """Validate that at least one of file_path or file_paths is provided."""
# Single check if both are None, O(1) instead of nested conditions if v is None and ("file_path" not in values or values.get("file_path") is None):
if v is None and info.data.get("file_path" if info.field_name == "file_paths" else "file_paths") is None:
raise ValueError("Either file_path or file_paths must be provided") raise ValueError("Either file_path or file_paths must be provided")
return v return v
@@ -63,10 +62,7 @@ class BaseFileKnowledgeSource(BaseKnowledgeSource, ABC):
def _save_documents(self): def _save_documents(self):
"""Save the documents to the storage.""" """Save the documents to the storage."""
if self.storage: self.storage.save(self.chunks)
self.storage.save(self.chunks)
else:
raise ValueError("No storage found to save documents.")
def convert_to_path(self, path: Union[Path, str]) -> Path: def convert_to_path(self, path: Union[Path, str]) -> Path:
"""Convert a path to a Path object.""" """Convert a path to a Path object."""

View File

@@ -16,7 +16,7 @@ class BaseKnowledgeSource(BaseModel, ABC):
chunk_embeddings: List[np.ndarray] = Field(default_factory=list) chunk_embeddings: List[np.ndarray] = Field(default_factory=list)
model_config = ConfigDict(arbitrary_types_allowed=True) model_config = ConfigDict(arbitrary_types_allowed=True)
storage: Optional[KnowledgeStorage] = Field(default=None) storage: KnowledgeStorage = Field(default_factory=KnowledgeStorage)
metadata: Dict[str, Any] = Field(default_factory=dict) # Currently unused metadata: Dict[str, Any] = Field(default_factory=dict) # Currently unused
collection_name: Optional[str] = Field(default=None) collection_name: Optional[str] = Field(default=None)
@@ -46,7 +46,4 @@ class BaseKnowledgeSource(BaseModel, ABC):
Save the documents to the storage. Save the documents to the storage.
This method should be called after the chunks and embeddings are generated. This method should be called after the chunks and embeddings are generated.
""" """
if self.storage: self.storage.save(self.chunks)
self.storage.save(self.chunks)
else:
raise ValueError("No storage found to save documents.")

View File

@@ -0,0 +1,77 @@
import os
import tempfile
import unittest
from pathlib import Path
from unittest.mock import MagicMock, patch
import click
from click.testing import CliRunner
from crewai.cli.cli import create
from crewai.cli.create_crew import create_crew
class TestCreateCrew(unittest.TestCase):
def setUp(self):
self.runner = CliRunner()
self.temp_dir = tempfile.TemporaryDirectory()
self.test_dir = Path(self.temp_dir.name)
def tearDown(self):
self.temp_dir.cleanup()
@patch("crewai.cli.create_crew.get_provider_data")
@patch("crewai.cli.create_crew.select_provider")
@patch("crewai.cli.create_crew.select_model")
@patch("crewai.cli.create_crew.write_env_file")
@patch("crewai.cli.create_crew.load_env_vars")
@patch("click.confirm")
def test_create_crew_handles_unicode(self, mock_confirm, mock_load_env,
mock_write_env, mock_select_model,
mock_select_provider, mock_get_provider_data):
"""Test that create_crew command handles Unicode properly."""
mock_confirm.return_value = True
mock_load_env.return_value = {}
mock_get_provider_data.return_value = {"openai": ["gpt-4"]}
mock_select_provider.return_value = "openai"
mock_select_model.return_value = "gpt-4"
templates_dir = Path("src/crewai/cli/templates/crew")
templates_dir.mkdir(parents=True, exist_ok=True)
template_content = """
Hello {{name}}! Unicode test: 你好, こんにちは, Привет 🚀
Class: {{crew_name}}
Folder: {{folder_name}}
"""
(templates_dir / "tools").mkdir(exist_ok=True)
(templates_dir / "config").mkdir(exist_ok=True)
for file_name in [".gitignore", "pyproject.toml", "README.md", "__init__.py", "main.py", "crew.py"]:
with open(templates_dir / file_name, "w", encoding="utf-8") as f:
f.write(template_content)
(templates_dir / "knowledge").mkdir(exist_ok=True)
with open(templates_dir / "knowledge" / "user_preference.txt", "w", encoding="utf-8") as f:
f.write(template_content)
for file_path in ["tools/custom_tool.py", "tools/__init__.py", "config/agents.yaml", "config/tasks.yaml"]:
(templates_dir / file_path).parent.mkdir(exist_ok=True, parents=True)
with open(templates_dir / file_path, "w", encoding="utf-8") as f:
f.write(template_content)
with patch("crewai.cli.create_crew.Path") as mock_path:
mock_path.return_value = self.test_dir
mock_path.side_effect = lambda x: self.test_dir / x if isinstance(x, str) else x
create_crew("test_crew", skip_provider=True)
crew_dir = self.test_dir / "test_crew"
for root, _, files in os.walk(crew_dir):
for file in files:
file_path = os.path.join(root, file)
with open(file_path, "r", encoding="utf-8") as f:
content = f.read()
self.assertIn("你好", content, f"Unicode characters not preserved in {file_path}")
self.assertIn("🚀", content, f"Emoji not preserved in {file_path}")

View File

@@ -0,0 +1,89 @@
import os
import tempfile
import unittest
from pathlib import Path
from unittest.mock import patch
from crewai.cli.provider import fetch_provider_data, read_cache_file
from crewai.cli.utils import (
copy_template,
load_env_vars,
tree_find_and_replace,
write_env_file,
)
class TestEncoding(unittest.TestCase):
def setUp(self):
self.temp_dir = tempfile.TemporaryDirectory()
self.test_dir = Path(self.temp_dir.name)
self.unicode_content = "Hello Unicode: 你好, こんにちは, Привет, مرحبا, 안녕하세요 🚀"
self.src_file = self.test_dir / "src_file.txt"
self.dst_file = self.test_dir / "dst_file.txt"
with open(self.src_file, "w", encoding="utf-8") as f:
f.write(self.unicode_content)
def tearDown(self):
self.temp_dir.cleanup()
def test_copy_template_handles_unicode(self):
"""Test that copy_template handles Unicode characters properly in all environments."""
copy_template(
self.src_file,
self.dst_file,
"test_name",
"TestClass",
"test_folder"
)
with open(self.dst_file, "r", encoding="utf-8") as f:
content = f.read()
self.assertIn("你好", content)
self.assertIn("こんにちは", content)
self.assertIn("🚀", content)
def test_env_vars_handle_unicode(self):
"""Test that environment variable functions handle Unicode characters properly."""
test_env_path = self.test_dir / ".env"
test_env_vars = {
"KEY1": "Value with Unicode: 你好",
"KEY2": "More Unicode: こんにちは 🚀"
}
write_env_file(self.test_dir, test_env_vars)
loaded_vars = load_env_vars(self.test_dir)
self.assertEqual(loaded_vars["KEY1"], "Value with Unicode: 你好")
self.assertEqual(loaded_vars["KEY2"], "More Unicode: こんにちは 🚀")
def test_tree_find_and_replace_handles_unicode(self):
"""Test that tree_find_and_replace handles Unicode characters properly."""
test_file = self.test_dir / "replace_test.txt"
with open(test_file, "w", encoding="utf-8") as f:
f.write("Replace this: PLACEHOLDER with Unicode: 你好")
tree_find_and_replace(self.test_dir, "PLACEHOLDER", "🚀")
with open(test_file, "r", encoding="utf-8") as f:
content = f.read()
self.assertIn("Replace this: 🚀 with Unicode: 你好", content)
@patch("crewai.cli.provider.requests.get")
def test_provider_functions_handle_unicode(self, mock_get):
"""Test that provider data functions handle Unicode properly."""
mock_response = unittest.mock.Mock()
mock_response.iter_content.return_value = [self.unicode_content.encode("utf-8")]
mock_response.headers.get.return_value = str(len(self.unicode_content))
mock_get.return_value = mock_response
cache_file = self.test_dir / "cache.json"
with open(cache_file, "w", encoding="utf-8") as f:
f.write('{"model": "Unicode test: 你好 🚀"}')
cache_data = read_cache_file(cache_file)
self.assertEqual(cache_data["model"], "Unicode test: 你好 🚀")

View File

@@ -584,28 +584,3 @@ def test_docling_source_with_local_file():
docling_source = CrewDoclingSource(file_paths=[pdf_path]) docling_source = CrewDoclingSource(file_paths=[pdf_path])
assert docling_source.file_paths == [pdf_path] assert docling_source.file_paths == [pdf_path]
assert docling_source.content is not None assert docling_source.content is not None
def test_file_path_validation():
"""Test file path validation for knowledge sources."""
current_dir = Path(__file__).parent
pdf_path = current_dir / "crewai_quickstart.pdf"
# Test valid single file_path
source = PDFKnowledgeSource(file_path=pdf_path)
assert source.safe_file_paths == [pdf_path]
# Test valid file_paths list
source = PDFKnowledgeSource(file_paths=[pdf_path])
assert source.safe_file_paths == [pdf_path]
# Test both file_path and file_paths provided (should use file_paths)
source = PDFKnowledgeSource(file_path=pdf_path, file_paths=[pdf_path])
assert source.safe_file_paths == [pdf_path]
# Test neither file_path nor file_paths provided
with pytest.raises(
ValueError,
match="file_path/file_paths must be a Path, str, or a list of these types"
):
PDFKnowledgeSource()