Compare commits

..

7 Commits

Author SHA1 Message Date
João Moura
10b5082c0a new tests 2025-05-21 04:12:55 -07:00
João Moura
bddfe1c780 unnecesary 2025-05-21 03:13:14 -07:00
Devin AI
c3dc839b12 docs: Update documentation for inject_date feature
Co-Authored-By: Joe Moura <joao@crewai.com>
2025-05-21 04:03:52 +00:00
Devin AI
270a473d5d fix: Add date format validation to prevent invalid formats
Co-Authored-By: Joe Moura <joao@crewai.com>
2025-05-21 03:44:51 +00:00
Devin AI
98df434eb9 fix: Update test implementation for inject_date feature
Co-Authored-By: Joe Moura <joao@crewai.com>
2025-05-21 03:40:31 +00:00
Devin AI
9973011be5 feat: Add date_format parameter and error handling to inject_date feature
Co-Authored-By: Joe Moura <joao@crewai.com>
2025-05-21 03:30:20 +00:00
Devin AI
547e46b8cf feat: Add inject_date flag to Agent for automatic date injection
Co-Authored-By: Joe Moura <joao@crewai.com>
2025-05-21 03:24:06 +00:00
40 changed files with 235 additions and 1060 deletions

View File

@@ -21,7 +21,7 @@ Think of an agent as a specialized team member with specific skills, expertise,
<Note type="info" title="Enterprise Enhancement: Visual Agent Builder">
CrewAI Enterprise includes a Visual Agent Builder that simplifies agent creation and configuration without writing code. Design your agents visually and test them in real-time.
![Visual Agent Builder Screenshot](/images/enterprise/crew-studio-interface.png)
![Visual Agent Builder Screenshot](../images/enterprise/crew-studio-quickstart)
The Visual Agent Builder enables:
- Intuitive agent configuration with form-based interfaces

View File

@@ -4,7 +4,7 @@ description: Learn how to use the CrewAI CLI to interact with CrewAI.
icon: terminal
---
## Overview
# CrewAI CLI Documentation
The CrewAI CLI provides a set of commands to interact with CrewAI, allowing you to create, train, run, and manage crews & flows.

View File

@@ -4,7 +4,7 @@ description: Exploring the dynamics of agent collaboration within the CrewAI fra
icon: screen-users
---
## Overview
## Collaboration Fundamentals
Collaboration in CrewAI is fundamental, enabling agents to combine their skills, share information, and assist each other in task execution, embodying a truly cooperative ecosystem.

View File

@@ -4,7 +4,7 @@ description: Understanding and utilizing crews in the crewAI framework with comp
icon: people-group
---
## Overview
## What is a Crew?
A crew in crewAI represents a collaborative group of agents working together to achieve a set of tasks. Each crew defines the strategy for task execution, agent collaboration, and the overall workflow.

View File

@@ -4,7 +4,7 @@ description: 'Tap into CrewAI events to build custom integrations and monitoring
icon: spinner
---
## Overview
# Event Listeners
CrewAI provides a powerful event system that allows you to listen for and react to various events that occur during the execution of your Crew. This feature enables you to build custom integrations, monitoring solutions, logging systems, or any other functionality that needs to be triggered based on CrewAI's internal events.
@@ -21,7 +21,7 @@ When specific actions occur in CrewAI (like a Crew starting execution, an Agent
<Note type="info" title="Enterprise Enhancement: Prompt Tracing">
CrewAI Enterprise provides a built-in Prompt Tracing feature that leverages the event system to track, store, and visualize all prompts, completions, and associated metadata. This provides powerful debugging capabilities and transparency into your agent operations.
![Prompt Tracing Dashboard](/images/enterprise/traces-overview.png)
![Prompt Tracing Dashboard](../images/enterprise/prompt-tracing.png)
With Prompt Tracing you can:
- View the complete history of all prompts sent to your LLM
@@ -224,15 +224,6 @@ CrewAI provides a wide range of events that you can listen for:
- **ToolExecutionErrorEvent**: Emitted when a tool execution encounters an error
- **ToolSelectionErrorEvent**: Emitted when there's an error selecting a tool
### Knowledge Events
- **KnowledgeRetrievalStartedEvent**: Emitted when a knowledge retrieval is started
- **KnowledgeRetrievalCompletedEvent**: Emitted when a knowledge retrieval is completed
- **KnowledgeQueryStartedEvent**: Emitted when a knowledge query is started
- **KnowledgeQueryCompletedEvent**: Emitted when a knowledge query is completed
- **KnowledgeQueryFailedEvent**: Emitted when a knowledge query fails
- **KnowledgeSearchQueryFailedEvent**: Emitted when a knowledge search query fails
### Flow Events
- **FlowCreatedEvent**: Emitted when a Flow is created

View File

@@ -4,7 +4,7 @@ description: Learn how to create and manage AI workflows using CrewAI Flows.
icon: arrow-progress
---
## Overview
## Introduction
CrewAI Flows is a powerful feature designed to streamline the creation and management of AI workflows. Flows allow developers to combine and coordinate coding tasks and Crews efficiently, providing a robust framework for building sophisticated AI automations.

View File

@@ -4,7 +4,7 @@ description: What is knowledge in CrewAI and how to use it.
icon: book
---
## Overview
## What is Knowledge?
Knowledge in CrewAI is a powerful system that allows AI agents to access and utilize external information sources during their tasks.
Think of it as giving your agents a reference library they can consult while working.
@@ -36,15 +36,12 @@ CrewAI supports various types of knowledge sources out of the box:
## Supported Knowledge Parameters
<ParamField body="sources" type="List[BaseKnowledgeSource]" required="Yes">
List of knowledge sources that provide content to be stored and queried. Can include PDF, CSV, Excel, JSON, text files, or string content.
</ParamField>
<ParamField body="collection_name" type="str">
Name of the collection where the knowledge will be stored. Used to identify different sets of knowledge. Defaults to \"knowledge\" if not provided.
</ParamField>
<ParamField body="storage" type="Optional[KnowledgeStorage]">
Custom storage configuration for managing how the knowledge is stored and retrieved. If not provided, a default storage will be created.
</ParamField>
| Parameter | Type | Required | Description |
| :--------------------------- | :---------------------------------- | :------- | :---------------------------------------------------------------------------------------------------------------------------------------------------- |
| `sources` | **List[BaseKnowledgeSource]** | Yes | List of knowledge sources that provide content to be stored and queried. Can include PDF, CSV, Excel, JSON, text files, or string content. |
| `collection_name` | **str** | No | Name of the collection where the knowledge will be stored. Used to identify different sets of knowledge. Defaults to "knowledge" if not provided. |
| `storage` | **Optional[KnowledgeStorage]** | No | Custom storage configuration for managing how the knowledge is stored and retrieved. If not provided, a default storage will be created. |
<Tip>
Unlike retrieval from a vector database using a tool, agents preloaded with knowledge will not need a retrieval persona or task.
@@ -435,46 +432,6 @@ Query rewriting happens transparently using a system prompt that instructs the L
This mechanism is fully automatic and requires no configuration from users. The agent's LLM is used to perform the query rewriting, so using a more capable LLM can improve the quality of rewritten queries.
</Tip>
## Knowledge Events
CrewAI emits events during the knowledge retrieval process that you can listen for using the event system. These events allow you to monitor, debug, and analyze how knowledge is being retrieved and used by your agents.
### Available Knowledge Events
- **KnowledgeRetrievalStartedEvent**: Emitted when an agent starts retrieving knowledge from sources
- **KnowledgeRetrievalCompletedEvent**: Emitted when knowledge retrieval is completed, including the query used and the retrieved content
- **KnowledgeQueryStartedEvent**: Emitted when a query to knowledge sources begins
- **KnowledgeQueryCompletedEvent**: Emitted when a query completes successfully
- **KnowledgeQueryFailedEvent**: Emitted when a query to knowledge sources fails
- **KnowledgeSearchQueryFailedEvent**: Emitted when a search query fails
### Example: Monitoring Knowledge Retrieval
```python
from crewai.utilities.events import (
KnowledgeRetrievalStartedEvent,
KnowledgeRetrievalCompletedEvent,
)
from crewai.utilities.events.base_event_listener import BaseEventListener
class KnowledgeMonitorListener(BaseEventListener):
def setup_listeners(self, crewai_event_bus):
@crewai_event_bus.on(KnowledgeRetrievalStartedEvent)
def on_knowledge_retrieval_started(source, event):
print(f"Agent '{event.agent.role}' started retrieving knowledge")
@crewai_event_bus.on(KnowledgeRetrievalCompletedEvent)
def on_knowledge_retrieval_completed(source, event):
print(f"Agent '{event.agent.role}' completed knowledge retrieval")
print(f"Query: {event.query}")
print(f"Retrieved {len(event.retrieved_knowledge)} knowledge chunks")
# Create an instance of your listener
knowledge_monitor = KnowledgeMonitorListener()
```
For more information on using events, see the [Event Listeners](https://docs.crewai.com/concepts/event-listener) documentation.
### Example
```python

View File

@@ -4,10 +4,9 @@ description: 'A comprehensive guide to configuring and using Large Language Mode
icon: 'microchip-ai'
---
## Overview
CrewAI integrates with multiple LLM providers through LiteLLM, giving you the flexibility to choose the right model for your specific use case. This guide will help you understand how to configure and use different LLM providers in your CrewAI projects.
<Note>
CrewAI integrates with multiple LLM providers through LiteLLM, giving you the flexibility to choose the right model for your specific use case. This guide will help you understand how to configure and use different LLM providers in your CrewAI projects.
</Note>
## What are LLMs?

View File

@@ -4,7 +4,7 @@ description: Leveraging memory systems in the CrewAI framework to enhance agent
icon: database
---
## Overview
## Introduction to Memory Systems in CrewAI
The crewAI framework introduces a sophisticated memory system designed to significantly enhance the capabilities of AI agents.
This system comprises `short-term memory`, `long-term memory`, `entity memory`, and `contextual memory`, each serving a unique purpose in aiding agents to remember,

View File

@@ -1,10 +1,10 @@
---
title: Planning
description: Learn how to add planning to your CrewAI Crew and improve their performance.
icon: ruler-combined
icon: brain
---
## Overview
## Introduction
The planning feature in CrewAI allows you to add planning capability to your crew. When enabled, before each Crew iteration,
all Crew information is sent to an AgentPlanner that will plan the tasks step by step, and this plan will be added to each task description.

View File

@@ -4,8 +4,7 @@ description: Detailed guide on workflow management through processes in CrewAI,
icon: bars-staggered
---
## Overview
## Understanding Processes
<Tip>
Processes orchestrate the execution of tasks by agents, akin to project management in human teams.
These processes ensure tasks are distributed and executed efficiently, in alignment with a predefined strategy.

View File

@@ -1,14 +1,12 @@
---
title: Reasoning
description: "Learn how to enable and use agent reasoning to improve task execution."
icon: brain
title: "Agent Reasoning"
---
## Overview
# Agent Reasoning
Agent reasoning is a feature that allows agents to reflect on a task and create a plan before execution. This helps agents approach tasks more methodically and ensures they're ready to perform the assigned work.
## Usage
## How to Use Agent Reasoning
To enable reasoning for an agent, simply set `reasoning=True` when creating the agent:
@@ -37,13 +35,8 @@ This process helps the agent break down complex tasks into manageable steps and
## Configuration Options
<ParamField body="reasoning" type="bool" default="False">
Enable or disable reasoning
</ParamField>
<ParamField body="max_reasoning_attempts" type="int" default="None">
Maximum number of attempts to refine the plan before proceeding with execution. If None (default), the agent will continue refining until it's ready.
</ParamField>
- `reasoning` (bool): Enable or disable reasoning (default: False)
- `max_reasoning_attempts` (int, optional): Maximum number of attempts to refine the plan before proceeding with execution. If None (default), the agent will continue refining until it's ready.
## Example
@@ -116,7 +109,7 @@ Here's an example of what a reasoning plan might look like for a data analysis t
Task: Analyze the provided sales data and identify key trends.
Reasoning Plan:
I'll analyze the sales data to identify the top 3 trends.
I'll analyze the sales data to identify the top 3 trends.
1. Understanding of the task:
I need to analyze sales data to identify key trends that would be valuable for business decision-making.

View File

@@ -4,7 +4,7 @@ description: Detailed guide on managing and creating tasks within the CrewAI fra
icon: list-check
---
## Overview
## Overview of a Task
In the CrewAI framework, a `Task` is a specific assignment completed by an `Agent`.
@@ -15,7 +15,7 @@ Tasks within CrewAI can be collaborative, requiring multiple agents to work toge
<Note type="info" title="Enterprise Enhancement: Visual Task Builder">
CrewAI Enterprise includes a Visual Task Builder in Crew Studio that simplifies complex task creation and chaining. Design your task flows visually and test them in real-time without writing code.
![Task Builder Screenshot](/images/enterprise/crew-studio-interface.png)
![Task Builder Screenshot](../images/enterprise/crew-studio-quickstart.png)
The Visual Task Builder enables:
- Drag-and-drop task creation

View File

@@ -4,7 +4,7 @@ description: Learn how to test your CrewAI Crew and evaluate their performance.
icon: vial
---
## Overview
## Introduction
Testing is a crucial part of the development process, and it is essential to ensure that your crew is performing as expected. With crewAI, you can easily test your crew and evaluate its performance using the built-in testing capabilities.

View File

@@ -4,7 +4,7 @@ description: Understanding and leveraging tools within the CrewAI framework for
icon: screwdriver-wrench
---
## Overview
## Introduction
CrewAI tools empower agents with capabilities ranging from web searching and data analysis to collaboration and delegating tasks among coworkers.
This documentation outlines how to create, integrate, and leverage these tools within the CrewAI framework, including a new focus on collaboration tools.
@@ -18,6 +18,8 @@ enabling everything from simple searches to complex interactions and effective t
<Note type="info" title="Enterprise Enhancement: Tools Repository">
CrewAI Enterprise provides a comprehensive Tools Repository with pre-built integrations for common business systems and APIs. Deploy agents with enterprise tools in minutes instead of days.
![Tools Repository Screenshot](../images/enterprise/tools-repository.png)
The Enterprise Tools Repository includes:
- Pre-built connectors for popular enterprise systems
- Custom tool creation interface

View File

@@ -4,7 +4,7 @@ description: Learn how to train your CrewAI agents by giving them feedback early
icon: dumbbell
---
## Overview
## Introduction
The training feature in CrewAI allows you to train your AI agents using the command-line interface (CLI).
By running the command `crewai train -n <n_iterations>`, you can specify the number of iterations for the training process.

View File

@@ -74,7 +74,6 @@
"concepts/collaboration",
"concepts/training",
"concepts/memory",
"concepts/reasoning",
"concepts/planning",
"concepts/testing",
"concepts/cli",
@@ -144,7 +143,7 @@
"group": "MCP Integration",
"pages": [
"mcp/crewai-mcp-integration"
]
]
},
{
"group": "Agent Monitoring & Observability",
@@ -213,8 +212,7 @@
"pages": [
"enterprise/features/tool-repository",
"enterprise/features/webhook-streaming",
"enterprise/features/traces",
"enterprise/features/hallucination-guardrail"
"enterprise/features/traces"
]
},
{
@@ -309,4 +307,4 @@
"reddit": "https://www.reddit.com/r/crewAIInc/"
}
}
}
}

View File

@@ -1,245 +0,0 @@
---
title: Hallucination Guardrail
description: "Prevent and detect AI hallucinations in your CrewAI tasks"
icon: "shield-check"
---
## Overview
The Hallucination Guardrail is an enterprise feature that validates AI-generated content to ensure it's grounded in facts and doesn't contain hallucinations. It analyzes task outputs against reference context and provides detailed feedback when potentially hallucinated content is detected.
## What are Hallucinations?
AI hallucinations occur when language models generate content that appears plausible but is factually incorrect or not supported by the provided context. The Hallucination Guardrail helps prevent these issues by:
- Comparing outputs against reference context
- Evaluating faithfulness to source material
- Providing detailed feedback on problematic content
- Supporting custom thresholds for validation strictness
## Basic Usage
### Setting Up the Guardrail
```python
from crewai.tasks.hallucination_guardrail import HallucinationGuardrail
from crewai import LLM
# Initialize the guardrail with reference context
guardrail = HallucinationGuardrail(
context="AI helps with various tasks including analysis and generation.",
llm=LLM(model="gpt-4o-mini")
)
```
### Adding to Tasks
```python
from crewai import Task
# Create your task with the guardrail
task = Task(
description="Write a summary about AI capabilities",
expected_output="A factual summary based on the provided context",
agent=my_agent,
guardrail=guardrail # Add the guardrail to validate output
)
```
## Advanced Configuration
### Custom Threshold Validation
For stricter validation, you can set a custom faithfulness threshold (0-10 scale):
```python
# Strict guardrail requiring high faithfulness score
strict_guardrail = HallucinationGuardrail(
context="Quantum computing uses qubits that exist in superposition states.",
llm=LLM(model="gpt-4o-mini"),
threshold=8.0 # Requires score >= 8 to pass validation
)
```
### Including Tool Response Context
When your task uses tools, you can include tool responses for more accurate validation:
```python
# Guardrail with tool response context
weather_guardrail = HallucinationGuardrail(
context="Current weather information for the requested location",
llm=LLM(model="gpt-4o-mini"),
tool_response="Weather API returned: Temperature 22°C, Humidity 65%, Clear skies"
)
```
## How It Works
### Validation Process
1. **Context Analysis**: The guardrail compares task output against the provided reference context
2. **Faithfulness Scoring**: Uses an internal evaluator to assign a faithfulness score (0-10)
3. **Verdict Determination**: Determines if content is faithful or contains hallucinations
4. **Threshold Checking**: If a custom threshold is set, validates against that score
5. **Feedback Generation**: Provides detailed reasons when validation fails
### Validation Logic
- **Default Mode**: Uses verdict-based validation (FAITHFUL vs HALLUCINATED)
- **Threshold Mode**: Requires faithfulness score to meet or exceed the specified threshold
- **Error Handling**: Gracefully handles evaluation errors and provides informative feedback
## Guardrail Results
The guardrail returns structured results indicating validation status:
```python
# Example of guardrail result structure
{
"valid": False,
"feedback": "Content appears to be hallucinated (score: 4.2/10, verdict: HALLUCINATED). The output contains information not supported by the provided context."
}
```
### Result Properties
- **valid**: Boolean indicating whether the output passed validation
- **feedback**: Detailed explanation when validation fails, including:
- Faithfulness score
- Verdict classification
- Specific reasons for failure
## Integration with Task System
### Automatic Validation
When a guardrail is added to a task, it automatically validates the output before the task is marked as complete:
```python
# Task output validation flow
task_output = agent.execute_task(task)
validation_result = guardrail(task_output)
if validation_result.valid:
# Task completes successfully
return task_output
else:
# Task fails with validation feedback
raise ValidationError(validation_result.feedback)
```
### Event Tracking
The guardrail integrates with CrewAI's event system to provide observability:
- **Validation Started**: When guardrail evaluation begins
- **Validation Completed**: When evaluation finishes with results
- **Validation Failed**: When technical errors occur during evaluation
## Best Practices
### Context Guidelines
<Steps>
<Step title="Provide Comprehensive Context">
Include all relevant factual information that the AI should base its output on:
```python
context = """
Company XYZ was founded in 2020 and specializes in renewable energy solutions.
They have 150 employees and generated $50M revenue in 2023.
Their main products include solar panels and wind turbines.
"""
```
</Step>
<Step title="Keep Context Relevant">
Only include information directly related to the task to avoid confusion:
```python
# Good: Focused context
context = "The current weather in New York is 18°C with light rain."
# Avoid: Unrelated information
context = "The weather is 18°C. The city has 8 million people. Traffic is heavy."
```
</Step>
<Step title="Update Context Regularly">
Ensure your reference context reflects current, accurate information.
</Step>
</Steps>
### Threshold Selection
<Steps>
<Step title="Start with Default Validation">
Begin without custom thresholds to understand baseline performance.
</Step>
<Step title="Adjust Based on Requirements">
- **High-stakes content**: Use threshold 8-10 for maximum accuracy
- **General content**: Use threshold 6-7 for balanced validation
- **Creative content**: Use threshold 4-5 or default verdict-based validation
</Step>
<Step title="Monitor and Iterate">
Track validation results and adjust thresholds based on false positives/negatives.
</Step>
</Steps>
## Performance Considerations
### Impact on Execution Time
- **Validation Overhead**: Each guardrail adds ~1-3 seconds per task
- **LLM Efficiency**: Choose efficient models for evaluation (e.g., gpt-4o-mini)
### Cost Optimization
- **Model Selection**: Use smaller, efficient models for guardrail evaluation
- **Context Size**: Keep reference context concise but comprehensive
- **Caching**: Consider caching validation results for repeated content
## Troubleshooting
<Accordion title="Validation Always Fails">
**Possible Causes:**
- Context is too restrictive or unrelated to task output
- Threshold is set too high for the content type
- Reference context contains outdated information
**Solutions:**
- Review and update context to match task requirements
- Lower threshold or use default verdict-based validation
- Ensure context is current and accurate
</Accordion>
<Accordion title="False Positives (Valid Content Marked Invalid)">
**Possible Causes:**
- Threshold too high for creative or interpretive tasks
- Context doesn't cover all valid aspects of the output
- Evaluation model being overly conservative
**Solutions:**
- Lower threshold or use default validation
- Expand context to include broader acceptable content
- Test with different evaluation models
</Accordion>
<Accordion title="Evaluation Errors">
**Possible Causes:**
- Network connectivity issues
- LLM model unavailable or rate limited
- Malformed task output or context
**Solutions:**
- Check network connectivity and LLM service status
- Implement retry logic for transient failures
- Validate task output format before guardrail evaluation
</Accordion>
<Card title="Need Help?" icon="headset" href="mailto:support@crewai.com">
Contact our support team for assistance with hallucination guardrail configuration or troubleshooting.
</Card>

View File

@@ -12,8 +12,7 @@ The before kickoff hook is executed before the crew starts its tasks. It receive
Here's an example of defining a before kickoff function in your `crew.py`:
```python
from crewai import CrewBase
from crewai.project import before_kickoff
from crewai import CrewBase, before_kickoff
@CrewBase
class MyCrew:
@@ -35,8 +34,7 @@ The after kickoff hook is executed after the crew has completed its tasks. It re
Here's how you can define an after kickoff function in your `crew.py`:
```python
from crewai import CrewBase
from crewai.project import after_kickoff
from crewai import CrewBase, after_kickoff
@CrewBase
class MyCrew:

View File

@@ -1,6 +1,6 @@
[project]
name = "crewai"
version = "0.121.0"
version = "0.120.1"
description = "Cutting-edge framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks."
readme = "README.md"
requires-python = ">=3.10,<3.13"

View File

@@ -17,7 +17,7 @@ warnings.filterwarnings(
category=UserWarning,
module="pydantic.main",
)
__version__ = "0.121.0"
__version__ = "0.120.1"
__all__ = [
"Agent",
"Crew",

View File

@@ -5,7 +5,7 @@ description = "{{name}} using crewAI"
authors = [{ name = "Your Name", email = "you@example.com" }]
requires-python = ">=3.10,<3.13"
dependencies = [
"crewai[tools]>=0.121.0,<1.0.0"
"crewai[tools]>=0.120.1,<1.0.0"
]
[project.scripts]

View File

@@ -5,7 +5,7 @@ description = "{{name}} using crewAI"
authors = [{ name = "Your Name", email = "you@example.com" }]
requires-python = ">=3.10,<3.13"
dependencies = [
"crewai[tools]>=0.121.0,<1.0.0",
"crewai[tools]>=0.120.1,<1.0.0",
]
[project.scripts]

View File

@@ -5,7 +5,7 @@ description = "Power up your crews with {{folder_name}}"
readme = "README.md"
requires-python = ">=3.10,<3.13"
dependencies = [
"crewai[tools]>=0.121.0"
"crewai[tools]>=0.120.1"
]
[tool.crewai]

View File

@@ -161,7 +161,7 @@ def tree_find_and_replace(directory, find, replace):
for filename in files:
filepath = os.path.join(path, filename)
with open(filepath, "r", encoding="utf-8", errors="ignore") as file:
with open(filepath, "r") as file:
contents = file.read()
with open(filepath, "w") as file:
file.write(contents.replace(find, replace))

View File

@@ -315,7 +315,9 @@ class Crew(FlowTrackable, BaseModel):
"""Initialize private memory attributes."""
self._external_memory = (
# External memory doesnt support a default value since it was designed to be managed entirely externally
self.external_memory.set_crew(self) if self.external_memory else None
self.external_memory.set_crew(self)
if self.external_memory
else None
)
self._long_term_memory = self.long_term_memory
@@ -1202,6 +1204,7 @@ class Crew(FlowTrackable, BaseModel):
"_long_term_memory",
"_entity_memory",
"_external_memory",
"_telemetry",
"agents",
"tasks",
"knowledge_sources",
@@ -1394,10 +1397,10 @@ class Crew(FlowTrackable, BaseModel):
memory_systems = self._get_memory_systems()
for memory_type, config in memory_systems.items():
if (system := config.get("system")) is not None:
name = config.get("name")
if (system := config.get('system')) is not None:
name = config.get('name')
try:
reset_fn: Callable = cast(Callable, config.get("reset"))
reset_fn: Callable = cast(Callable, config.get('reset'))
reset_fn(system)
self._logger.log(
"info",
@@ -1419,14 +1422,14 @@ class Crew(FlowTrackable, BaseModel):
"""
memory_systems = self._get_memory_systems()
config = memory_systems[memory_type]
system = config.get("system")
name = config.get("name")
system = config.get('system')
name = config.get('name')
if system is None:
raise RuntimeError(f"{name} memory system is not initialized")
try:
reset_fn: Callable = cast(Callable, config.get("reset"))
reset_fn: Callable = cast(Callable, config.get('reset'))
reset_fn(system)
self._logger.log(
"info",
@@ -1439,67 +1442,58 @@ class Crew(FlowTrackable, BaseModel):
def _get_memory_systems(self):
"""Get all available memory systems with their configuration.
Returns:
Dict containing all memory systems with their reset functions and display names.
"""
def default_reset(memory):
return memory.reset()
def knowledge_reset(memory):
return self.reset_knowledge(memory)
# Get knowledge for agents
agent_knowledges = [
getattr(agent, "knowledge", None)
for agent in self.agents
if getattr(agent, "knowledge", None) is not None
]
# Get knowledge for agents
agent_knowledges = [getattr(agent, "knowledge", None) for agent in self.agents
if getattr(agent, "knowledge", None) is not None]
# Get knowledge for crew and agents
crew_knowledge = getattr(self, "knowledge", None)
crew_and_agent_knowledges = (
[crew_knowledge] if crew_knowledge is not None else []
) + agent_knowledges
crew_and_agent_knowledges = ([crew_knowledge] if crew_knowledge is not None else []) + agent_knowledges
return {
"short": {
"system": getattr(self, "_short_term_memory", None),
"reset": default_reset,
"name": "Short Term",
'short': {
'system': getattr(self, "_short_term_memory", None),
'reset': default_reset,
'name': 'Short Term'
},
"entity": {
"system": getattr(self, "_entity_memory", None),
"reset": default_reset,
"name": "Entity",
'entity': {
'system': getattr(self, "_entity_memory", None),
'reset': default_reset,
'name': 'Entity'
},
"external": {
"system": getattr(self, "_external_memory", None),
"reset": default_reset,
"name": "External",
'external': {
'system': getattr(self, "_external_memory", None),
'reset': default_reset,
'name': 'External'
},
"long": {
"system": getattr(self, "_long_term_memory", None),
"reset": default_reset,
"name": "Long Term",
'long': {
'system': getattr(self, "_long_term_memory", None),
'reset': default_reset,
'name': 'Long Term'
},
"kickoff_outputs": {
"system": getattr(self, "_task_output_handler", None),
"reset": default_reset,
"name": "Task Output",
'kickoff_outputs': {
'system': getattr(self, "_task_output_handler", None),
'reset': default_reset,
'name': 'Task Output'
},
"knowledge": {
"system": crew_and_agent_knowledges
if crew_and_agent_knowledges
else None,
"reset": knowledge_reset,
"name": "Crew Knowledge and Agent Knowledge",
},
"agent_knowledge": {
"system": agent_knowledges if agent_knowledges else None,
"reset": knowledge_reset,
"name": "Agent Knowledge",
'knowledge': {
'system': crew_and_agent_knowledges if crew_and_agent_knowledges else None,
'reset': knowledge_reset,
'name': 'Crew Knowledge and Agent Knowledge'
},
'agent_knowledge': {
'system': agent_knowledges if agent_knowledges else None,
'reset': knowledge_reset,
'name': 'Agent Knowledge'
}
}
def reset_knowledge(self, knowledges: List[Knowledge]) -> None:

View File

@@ -1,96 +0,0 @@
"""Hallucination Guardrail Placeholder for CrewAI.
This is a no-op version of the HallucinationGuardrail for the open-source repository.
Classes:
HallucinationGuardrail: Placeholder guardrail that validates task outputs.
"""
from typing import Any, Optional, Tuple
from crewai.llm import LLM
from crewai.tasks.task_output import TaskOutput
from crewai.utilities.logger import Logger
class HallucinationGuardrail:
"""Placeholder for the HallucinationGuardrail feature.
Attributes:
context: The reference context that outputs would be checked against.
llm: The language model that would be used for evaluation.
threshold: Optional minimum faithfulness score that would be required to pass.
tool_response: Optional tool response information that would be used in evaluation.
Examples:
>>> # Basic usage with default verdict logic
>>> guardrail = HallucinationGuardrail(
... context="AI helps with various tasks including analysis and generation.",
... llm=agent.llm
... )
>>> # With custom threshold for stricter validation
>>> strict_guardrail = HallucinationGuardrail(
... context="Quantum computing uses qubits in superposition.",
... llm=agent.llm,
... threshold=8.0 # Would require score >= 8 to pass in enterprise version
... )
>>> # With tool response for additional context
>>> guardrail_with_tools = HallucinationGuardrail(
... context="The current weather data",
... llm=agent.llm,
... tool_response="Weather API returned: Temperature 22°C, Humidity 65%"
... )
"""
def __init__(
self,
context: str,
llm: LLM,
threshold: Optional[float] = None,
tool_response: str = "",
):
"""Initialize the HallucinationGuardrail placeholder.
Args:
context: The reference context that outputs would be checked against.
llm: The language model that would be used for evaluation.
threshold: Optional minimum faithfulness score that would be required to pass.
tool_response: Optional tool response information that would be used in evaluation.
"""
self.context = context
self.llm: LLM = llm
self.threshold = threshold
self.tool_response = tool_response
self._logger = Logger(verbose=True)
self._logger.log(
"warning",
"""Hallucination detection is a no-op in open source, use it for free at https://app.crewai.com\n""",
color="red",
)
@property
def description(self) -> str:
"""Generate a description of this guardrail for event logging."""
return "HallucinationGuardrail (no-op)"
def __call__(self, task_output: TaskOutput) -> Tuple[bool, Any]:
"""Validate a task output against hallucination criteria.
In the open source, this method always returns that the output is valid.
Args:
task_output: The output to be validated.
Returns:
A tuple containing:
- True
- The raw task output
"""
self._logger.log(
"warning",
"Premium hallucination detection skipped (use for free at https://app.crewai.com)\n",
color="red",
)
return True, task_output.raw

View File

@@ -9,7 +9,6 @@ import warnings
from contextlib import contextmanager
from importlib.metadata import version
from typing import TYPE_CHECKING, Any, Optional
import threading
from opentelemetry import trace
from opentelemetry.exporter.otlp.proto.http.trace_exporter import (
@@ -65,17 +64,7 @@ class Telemetry:
attribute in the Crew class.
"""
_instance = None
_lock = threading.Lock()
def __new__(cls):
if cls._instance is None:
with cls._lock:
if cls._instance is None:
cls._instance = super(Telemetry, cls).__new__(cls)
return cls._instance
def __init__(self) -> None:
def __init__(self):
self.ready: bool = False
self.trace_set: bool = False

View File

@@ -9,8 +9,7 @@ from crewai.agent import Agent
from crewai.llm import BaseLLM
from crewai.task import Task
from crewai.tasks.task_output import TaskOutput
from crewai.utilities.events import crewai_event_bus
from crewai.utilities.events.crew_events import CrewTestResultEvent
from crewai.telemetry import Telemetry
class TaskEvaluationPydanticOutput(BaseModel):
@@ -37,6 +36,7 @@ class CrewEvaluator:
def __init__(self, crew, eval_llm: InstanceOf[BaseLLM]):
self.crew = crew
self.llm = eval_llm
self._telemetry = Telemetry()
self._setup_for_evaluating()
def _setup_for_evaluating(self) -> None:
@@ -178,15 +178,11 @@ class CrewEvaluator:
evaluation_result = evaluation_task.execute_sync()
if isinstance(evaluation_result.pydantic, TaskEvaluationPydanticOutput):
crewai_event_bus.emit(
self._test_result_span = self._telemetry.individual_test_result_span(
self.crew,
CrewTestResultEvent(
quality=evaluation_result.pydantic.quality,
execution_duration=current_task.execution_duration,
model=self.llm.model,
crew_name=self.crew.name,
crew=self.crew,
),
evaluation_result.pydantic.quality,
current_task.execution_duration,
self.llm.model,
)
self.tasks_scores[self.iteration].append(evaluation_result.pydantic.quality)
self.run_execution_times[self.iteration].append(

View File

@@ -100,12 +100,3 @@ class CrewTestFailedEvent(CrewBaseEvent):
error: str
type: str = "crew_test_failed"
class CrewTestResultEvent(CrewBaseEvent):
"""Event emitted when a crew test result is available"""
quality: float
execution_duration: float
model: str
type: str = "crew_test_result"

View File

@@ -37,7 +37,6 @@ from .crew_events import (
CrewKickoffStartedEvent,
CrewTestCompletedEvent,
CrewTestFailedEvent,
CrewTestResultEvent,
CrewTestStartedEvent,
CrewTrainCompletedEvent,
CrewTrainFailedEvent,
@@ -135,15 +134,6 @@ class EventListener(BaseEventListener):
def on_crew_train_failed(source, event: CrewTrainFailedEvent):
self.formatter.handle_crew_train_failed(event.crew_name or "Crew")
@crewai_event_bus.on(CrewTestResultEvent)
def on_crew_test_result(source, event: CrewTestResultEvent):
self._telemetry.individual_test_result_span(
source.crew,
event.quality,
int(event.execution_duration),
event.model,
)
# ----------- TASK EVENTS -----------
@crewai_event_bus.on(TaskStartedEvent)

View File

@@ -48,14 +48,6 @@ from .reasoning_events import (
AgentReasoningCompletedEvent,
AgentReasoningFailedEvent,
)
from .knowledge_events import (
KnowledgeRetrievalStartedEvent,
KnowledgeRetrievalCompletedEvent,
KnowledgeQueryStartedEvent,
KnowledgeQueryCompletedEvent,
KnowledgeQueryFailedEvent,
KnowledgeSearchQueryFailedEvent,
)
EventTypes = Union[
CrewKickoffStartedEvent,
@@ -90,10 +82,4 @@ EventTypes = Union[
AgentReasoningStartedEvent,
AgentReasoningCompletedEvent,
AgentReasoningFailedEvent,
KnowledgeRetrievalStartedEvent,
KnowledgeRetrievalCompletedEvent,
KnowledgeQueryStartedEvent,
KnowledgeQueryCompletedEvent,
KnowledgeQueryFailedEvent,
KnowledgeSearchQueryFailedEvent,
]

View File

@@ -19,13 +19,10 @@ class LLMGuardrailStartedEvent(BaseEvent):
from inspect import getsource
from crewai.tasks.llm_guardrail import LLMGuardrail
from crewai.tasks.hallucination_guardrail import HallucinationGuardrail
super().__init__(**data)
if isinstance(self.guardrail, LLMGuardrail) or isinstance(
self.guardrail, HallucinationGuardrail
):
if isinstance(self.guardrail, LLMGuardrail):
self.guardrail = self.guardrail.description.strip()
elif isinstance(self.guardrail, Callable):
self.guardrail = getsource(self.guardrail).strip()

View File

@@ -4,7 +4,6 @@ from rich.console import Console
from rich.panel import Panel
from rich.text import Text
from rich.tree import Tree
from rich.live import Live
class ConsoleFormatter:
@@ -21,12 +20,6 @@ class ConsoleFormatter:
def __init__(self, verbose: bool = False):
self.console = Console(width=None)
self.verbose = verbose
# Live instance to dynamically update a Tree renderable (e.g. the Crew tree)
# When multiple Tree objects are printed sequentially we reuse this Live
# instance so the previous render is replaced instead of writing a new one.
# Once any non-Tree renderable is printed we stop the Live session so the
# final Tree persists on the terminal.
self._live: Optional[Live] = None
def create_panel(self, content: Text, title: str, style: str = "blue") -> Panel:
"""Create a standardized panel with consistent styling."""
@@ -67,7 +60,7 @@ class ConsoleFormatter:
label.append(f"{prefix} ", style=f"{style} bold")
label.append(name, style=style)
if status:
label.append("\nStatus: ", style="white")
label.append("\n Status: ", style="white")
label.append(status, style=f"{style} bold")
tree.label = label
@@ -76,46 +69,7 @@ class ConsoleFormatter:
return parent.add(Text(text, style=style))
def print(self, *args, **kwargs) -> None:
"""Custom print that replaces consecutive Tree renders.
* If the argument is a single ``Tree`` instance, we either start a
``Live`` session (first tree) or update the existing one (subsequent
trees). This results in the tree being rendered in-place instead of
being appended repeatedly to the log.
* A blank call (no positional arguments) is ignored while a Live
session is active so it does not prematurely terminate the tree
rendering.
* Any other renderable will terminate the Live session (if one is
active) so the last tree stays on screen and the new content is
printed normally.
"""
# Case 1: updating / starting live Tree rendering
if len(args) == 1 and isinstance(args[0], Tree):
tree = args[0]
if not self._live:
# Start a new Live session for the first tree
self._live = Live(tree, console=self.console, refresh_per_second=4)
self._live.start()
else:
# Update existing Live session
self._live.update(tree, refresh=True)
return # Nothing else to do
# Case 2: blank line while a live session is running ignore so we
# don't break the in-place rendering behaviour
if len(args) == 0 and self._live:
return
# Case 3: printing something other than a Tree → terminate live session
if self._live:
self._live.stop()
self._live = None
# Finally, pass through to the regular Console.print implementation
"""Print to console with consistent formatting if verbose is enabled."""
self.console.print(*args, **kwargs)
def print_panel(
@@ -203,7 +157,7 @@ class ConsoleFormatter:
task_content = Text()
task_content.append(f"📋 Task: {task_id}", style="yellow bold")
task_content.append("\nStatus: ", style="white")
task_content.append("\n Status: ", style="white")
task_content.append("Executing Task...", style="yellow dim")
task_branch = None
@@ -243,17 +197,11 @@ class ConsoleFormatter:
# Update tree label
for branch in crew_tree.children:
if str(task_id) in str(branch.label):
# Build label without introducing stray blank lines
task_content = Text()
# First line: Task ID
task_content.append(f"📋 Task: {task_id}", style=f"{style} bold")
# Second line: Assigned to
task_content.append("\nAssigned to: ", style="white")
task_content.append("\n Assigned to: ", style="white")
task_content.append(agent_role, style=style)
# Third line: Status
task_content.append("Status: ", style="white")
task_content.append("\n Status: ", style="white")
task_content.append(status_text, style=f"{style} bold")
branch.label = task_content
self.print(crew_tree)
@@ -272,16 +220,18 @@ class ConsoleFormatter:
if not self.verbose or not task_branch or not crew_tree:
return None
# Instead of creating a separate Agent node, we treat the task branch
# itself as the logical agent branch so that Reasoning/Tool nodes are
# nested under the task without an extra visual level.
agent_branch = task_branch.add("")
self.update_tree_label(
agent_branch, "🤖 Agent:", agent_role, "green", "In Progress"
)
# Store the task branch as the current_agent_branch for future nesting.
self.current_agent_branch = task_branch
self.print(crew_tree)
self.print()
# No additional tree modification needed; return the task branch so
# caller logic remains unchanged.
return task_branch
# Set the current_agent_branch attribute directly
self.current_agent_branch = agent_branch
return agent_branch
def update_agent_status(
self,
@@ -291,10 +241,19 @@ class ConsoleFormatter:
status: str = "completed",
) -> None:
"""Update agent status in the tree."""
# We no longer render a separate agent branch, so this method simply
# updates the stored branch reference (already the task branch) without
# altering the tree. Keeping it a no-op avoids duplicate status lines.
return
if not self.verbose or agent_branch is None or crew_tree is None:
return
self.update_tree_label(
agent_branch,
"🤖 Agent:",
agent_role,
"green",
"✅ Completed" if status == "completed" else "❌ Failed",
)
self.print(crew_tree)
self.print()
def create_flow_tree(self, flow_name: str, flow_id: str) -> Optional[Tree]:
"""Create and initialize a flow tree."""
@@ -307,7 +266,7 @@ class ConsoleFormatter:
flow_label = Text()
flow_label.append("🌊 Flow: ", style="blue bold")
flow_label.append(flow_name, style="blue")
flow_label.append("\nID: ", style="white")
flow_label.append("\n ID: ", style="white")
flow_label.append(flow_id, style="blue")
flow_tree = Tree(flow_label)
@@ -322,7 +281,7 @@ class ConsoleFormatter:
flow_label = Text()
flow_label.append("🌊 Flow: ", style="blue bold")
flow_label.append(flow_name, style="blue")
flow_label.append("\nID: ", style="white")
flow_label.append("\n ID: ", style="white")
flow_label.append(flow_id, style="blue")
flow_tree.label = flow_label
@@ -436,15 +395,9 @@ class ConsoleFormatter:
if not self.verbose:
return None
# Parent for tool usage: LiteAgent > Agent > Task
branch_to_use = (
self.current_lite_agent_branch
or agent_branch
or self.current_task_branch
)
# Render full crew tree when available for consistent live updates
tree_to_use = self.current_crew_tree or crew_tree or branch_to_use
# Use LiteAgent branch if available, otherwise use regular agent branch
branch_to_use = self.current_lite_agent_branch or agent_branch
tree_to_use = branch_to_use or crew_tree
if branch_to_use is None or tree_to_use is None:
# If we don't have a valid branch, default to crew_tree if provided
@@ -470,9 +423,10 @@ class ConsoleFormatter:
"yellow",
)
# Print updated tree immediately
self.print(tree_to_use)
self.print()
# Only print if this is a new tool usage
if tool_branch not in branch_to_use.children:
self.print(tree_to_use)
self.print()
return tool_branch
@@ -486,8 +440,8 @@ class ConsoleFormatter:
if not self.verbose or tool_branch is None:
return
# Decide which tree to render: prefer full crew tree, else parent branch
tree_to_use = self.current_crew_tree or crew_tree or self.current_task_branch
# Use LiteAgent branch if available, otherwise use crew tree
tree_to_use = self.current_lite_agent_branch or crew_tree
if tree_to_use is None:
return
@@ -518,8 +472,8 @@ class ConsoleFormatter:
if not self.verbose:
return
# Decide which tree to render: prefer full crew tree, else parent branch
tree_to_use = self.current_crew_tree or crew_tree or self.current_task_branch
# Use LiteAgent branch if available, otherwise use crew tree
tree_to_use = self.current_lite_agent_branch or crew_tree
if tool_branch:
self.update_tree_label(
@@ -547,15 +501,9 @@ class ConsoleFormatter:
if not self.verbose:
return None
# Parent for tool usage: LiteAgent > Agent > Task
branch_to_use = (
self.current_lite_agent_branch
or agent_branch
or self.current_task_branch
)
# Render full crew tree when available for consistent live updates
tree_to_use = self.current_crew_tree or crew_tree or branch_to_use
# Use LiteAgent branch if available, otherwise use regular agent branch
branch_to_use = self.current_lite_agent_branch or agent_branch
tree_to_use = branch_to_use or crew_tree
if branch_to_use is None or tree_to_use is None:
# If we don't have a valid branch, default to crew_tree if provided
@@ -584,33 +532,24 @@ class ConsoleFormatter:
if not self.verbose or tool_branch is None:
return
# Decide which tree to render: prefer full crew tree, else parent branch
tree_to_use = self.current_crew_tree or crew_tree or self.current_task_branch
if tree_to_use is None:
# Use LiteAgent branch if available, otherwise use regular agent branch
branch_to_use = self.current_lite_agent_branch or agent_branch
tree_to_use = branch_to_use or crew_tree
if branch_to_use is None or tree_to_use is None:
return
# Remove the thinking status node when complete
# Remove the thinking status node when complete, but only if it exists
if "Thinking" in str(tool_branch.label):
parents = [
self.current_lite_agent_branch,
self.current_agent_branch,
self.current_task_branch,
tree_to_use,
]
removed = False
for parent in parents:
if isinstance(parent, Tree) and tool_branch in parent.children:
parent.children.remove(tool_branch)
removed = True
break
# Clear pointer if we just removed the current_tool_branch
if self.current_tool_branch is tool_branch:
self.current_tool_branch = None
if removed:
self.print(tree_to_use)
self.print()
try:
# Check if the node is actually in the children list
if tool_branch in branch_to_use.children:
branch_to_use.children.remove(tool_branch)
self.print(tree_to_use)
self.print()
except Exception:
# If any error occurs during removal, just continue without removing
pass
def handle_llm_call_failed(
self, tool_branch: Optional[Tree], error: str, crew_tree: Optional[Tree]
@@ -619,8 +558,8 @@ class ConsoleFormatter:
if not self.verbose:
return
# Decide which tree to render: prefer full crew tree, else parent branch
tree_to_use = self.current_crew_tree or crew_tree or self.current_task_branch
# Use LiteAgent branch if available, otherwise use crew tree
tree_to_use = self.current_lite_agent_branch or crew_tree
# Update tool branch if it exists
if tool_branch:
@@ -662,7 +601,7 @@ class ConsoleFormatter:
test_label = Text()
test_label.append("🧪 Test: ", style="blue bold")
test_label.append(crew_name or "Crew", style="blue")
test_label.append("\nStatus: ", style="white")
test_label.append("\n Status: ", style="white")
test_label.append("In Progress", style="yellow")
test_tree = Tree(test_label)
@@ -684,7 +623,7 @@ class ConsoleFormatter:
test_label = Text()
test_label.append("✅ Test: ", style="green bold")
test_label.append(crew_name or "Crew", style="green")
test_label.append("\nStatus: ", style="white")
test_label.append("\n Status: ", style="white")
test_label.append("Completed", style="green bold")
flow_tree.label = test_label
@@ -773,7 +712,7 @@ class ConsoleFormatter:
lite_agent_label = Text()
lite_agent_label.append("🤖 LiteAgent: ", style="cyan bold")
lite_agent_label.append(lite_agent_role, style="cyan")
lite_agent_label.append("\nStatus: ", style="white")
lite_agent_label.append("\n Status: ", style="white")
lite_agent_label.append("In Progress", style="yellow")
lite_agent_tree = Tree(lite_agent_label)
@@ -812,7 +751,7 @@ class ConsoleFormatter:
lite_agent_label = Text()
lite_agent_label.append(f"{prefix} ", style=f"{style} bold")
lite_agent_label.append(lite_agent_role, style=style)
lite_agent_label.append("Status: ", style="white")
lite_agent_label.append("\n Status: ", style="white")
lite_agent_label.append(status_text, style=f"{style} bold")
lite_agent_branch.label = lite_agent_label
@@ -1065,20 +1004,16 @@ class ConsoleFormatter:
if not self.verbose:
return None
# Prefer LiteAgent > Agent > Task branch as the parent for reasoning
branch_to_use = (
self.current_lite_agent_branch
or agent_branch
or self.current_task_branch
)
# Prefer LiteAgent branch if we are within a LiteAgent context
branch_to_use = self.current_lite_agent_branch or agent_branch
tree_to_use = branch_to_use or crew_tree
# We always want to render the full crew tree when possible so the
# Live view updates coherently. Fallbacks: crew tree → branch itself.
tree_to_use = self.current_crew_tree or crew_tree or branch_to_use
if branch_to_use is None:
# Nothing to attach to, abort
return None
if branch_to_use is None or tree_to_use is None:
# If we don't have a valid branch, default to crew_tree if provided
if crew_tree is not None:
branch_to_use = tree_to_use = crew_tree
else:
return None
# Reuse existing reasoning branch if present
reasoning_branch = self.current_reasoning_branch
@@ -1109,9 +1044,8 @@ class ConsoleFormatter:
reasoning_branch = self.current_reasoning_branch
tree_to_use = (
self.current_crew_tree
or self.current_lite_agent_branch
or self.current_task_branch
self.current_lite_agent_branch
or self.current_agent_branch
or crew_tree
)
@@ -1150,9 +1084,8 @@ class ConsoleFormatter:
reasoning_branch = self.current_reasoning_branch
tree_to_use = (
self.current_crew_tree
or self.current_lite_agent_branch
or self.current_task_branch
self.current_lite_agent_branch
or self.current_agent_branch
or crew_tree
)

View File

@@ -6,8 +6,6 @@ import pytest
from crewai import Agent, Crew, Task
from crewai.telemetry import Telemetry
from opentelemetry import trace
@pytest.mark.parametrize(
"env_var,value,expected_ready",
@@ -36,6 +34,9 @@ def test_telemetry_enabled_by_default():
assert telemetry.ready is True
from opentelemetry import trace
@patch("crewai.telemetry.telemetry.logger.error")
@patch(
"opentelemetry.exporter.otlp.proto.http.trace_exporter.OTLPSpanExporter.export",
@@ -66,32 +67,3 @@ def test_telemetry_fails_due_connect_timeout(export_mock, logger_mock):
export_mock.assert_called_once()
logger_mock.assert_called_once_with(error)
def test_telemetry_singleton_pattern():
"""Test that Telemetry uses the singleton pattern correctly."""
Telemetry._instance = None
telemetry1 = Telemetry()
telemetry2 = Telemetry()
assert telemetry1 is telemetry2
setattr(telemetry1, "test_attribute", "test_value")
assert hasattr(telemetry2, "test_attribute")
assert getattr(telemetry2, "test_attribute") == "test_value"
import threading
instances = []
def create_instance():
instances.append(Telemetry())
threads = [threading.Thread(target=create_instance) for _ in range(5)]
for thread in threads:
thread.start()
for thread in threads:
thread.join()
assert all(instance is telemetry1 for instance in instances)

View File

@@ -1,108 +0,0 @@
from unittest.mock import Mock
import pytest
from crewai.llm import LLM
from crewai.tasks.hallucination_guardrail import HallucinationGuardrail
from crewai.tasks.task_output import TaskOutput
def test_hallucination_guardrail_initialization():
"""Test that the hallucination guardrail initializes correctly with all parameters."""
mock_llm = Mock(spec=LLM)
guardrail = HallucinationGuardrail(context="Test reference context", llm=mock_llm)
assert guardrail.context == "Test reference context"
assert guardrail.llm == mock_llm
assert guardrail.threshold is None
assert guardrail.tool_response == ""
guardrail = HallucinationGuardrail(
context="Test reference context",
llm=mock_llm,
threshold=8.5,
tool_response="Sample tool response",
)
assert guardrail.context == "Test reference context"
assert guardrail.llm == mock_llm
assert guardrail.threshold == 8.5
assert guardrail.tool_response == "Sample tool response"
def test_hallucination_guardrail_no_op_behavior():
"""Test that the guardrail always returns True in the open-source version."""
mock_llm = Mock(spec=LLM)
guardrail = HallucinationGuardrail(
context="Test reference context",
llm=mock_llm,
threshold=9.0,
)
task_output = TaskOutput(
raw="Sample task output",
description="Test task",
expected_output="Expected output",
agent="Test Agent",
)
result, output = guardrail(task_output)
assert result is True
assert output == "Sample task output"
def test_hallucination_guardrail_description():
"""Test that the guardrail provides the correct description for event logging."""
guardrail = HallucinationGuardrail(
context="Test reference context", llm=Mock(spec=LLM)
)
assert guardrail.description == "HallucinationGuardrail (no-op)"
@pytest.mark.parametrize(
"context,task_output_text,threshold,tool_response",
[
(
"Earth orbits the Sun once every 365.25 days.",
"It takes Earth approximately one year to go around the Sun.",
None,
"",
),
(
"Python was created by Guido van Rossum in 1991.",
"Python is a programming language developed by Guido van Rossum.",
7.5,
"",
),
(
"The capital of France is Paris.",
"Paris is the largest city and capital of France.",
9.0,
"Geographic API returned: France capital is Paris",
),
],
)
def test_hallucination_guardrail_always_passes(
context, task_output_text, threshold, tool_response
):
"""Test that the guardrail always passes regardless of configuration in open-source version."""
mock_llm = Mock(spec=LLM)
guardrail = HallucinationGuardrail(
context=context, llm=mock_llm, threshold=threshold, tool_response=tool_response
)
task_output = TaskOutput(
raw=task_output_text,
description="Test task",
expected_output="Expected output",
agent="Test Agent",
)
result, output = guardrail(task_output)
assert result is True
assert output == task_output_text

View File

@@ -1,10 +1,9 @@
from unittest.mock import Mock, patch
from unittest.mock import ANY, Mock, patch
import pytest
from crewai import Agent, Task
from crewai.llm import LLM
from crewai.tasks.hallucination_guardrail import HallucinationGuardrail
from crewai.tasks.llm_guardrail import LLMGuardrail
from crewai.tasks.task_output import TaskOutput
from crewai.utilities.events import (
@@ -268,37 +267,3 @@ def test_guardrail_when_an_error_occurs(sample_agent, task_output):
max_retries=0,
)
task.execute_sync(agent=sample_agent)
def test_hallucination_guardrail_integration():
"""Test that HallucinationGuardrail integrates properly with the task system."""
agent = Mock()
agent.role = "test_agent"
agent.execute_task.return_value = "test result"
agent.crew = None
mock_llm = Mock(spec=LLM)
guardrail = HallucinationGuardrail(
context="Test reference context for validation", llm=mock_llm, threshold=8.0
)
task = Task(
description="Test task with hallucination guardrail",
expected_output="Valid output",
guardrail=guardrail,
)
result = task.execute_sync(agent=agent)
assert isinstance(result, TaskOutput)
assert result.raw == "test result"
def test_hallucination_guardrail_description_in_events():
"""Test that HallucinationGuardrail description appears correctly in events."""
mock_llm = Mock(spec=LLM)
guardrail = HallucinationGuardrail(context="Test context", llm=mock_llm)
assert guardrail.description == "HallucinationGuardrail (no-op)"
event = LLMGuardrailStartedEvent(guardrail=guardrail, retry_count=0)
assert event.guardrail == "HallucinationGuardrail (no-op)"

File diff suppressed because one or more lines are too long

View File

@@ -1,3 +1,4 @@
import os
from datetime import datetime
from unittest.mock import Mock, patch
@@ -21,7 +22,6 @@ from crewai.utilities.events.crew_events import (
CrewKickoffFailedEvent,
CrewKickoffStartedEvent,
CrewTestCompletedEvent,
CrewTestResultEvent,
CrewTestStartedEvent,
)
from crewai.utilities.events.crewai_event_bus import crewai_event_bus
@@ -38,6 +38,7 @@ from crewai.utilities.events.llm_events import (
LLMCallCompletedEvent,
LLMCallFailedEvent,
LLMCallStartedEvent,
LLMCallType,
LLMStreamChunkEvent,
)
from crewai.utilities.events.task_events import (
@@ -131,10 +132,6 @@ def test_crew_emits_test_kickoff_type_event():
def handle_crew_test_end(source, event):
received_events.append(event)
@crewai_event_bus.on(CrewTestResultEvent)
def handle_crew_test_result(source, event):
received_events.append(event)
eval_llm = LLM(model="gpt-4o-mini")
with (
patch.object(
@@ -152,16 +149,13 @@ def test_crew_emits_test_kickoff_type_event():
assert args[2] is None
assert args[3] == eval_llm
assert len(received_events) == 3
assert len(received_events) == 2
assert received_events[0].crew_name == "TestCrew"
assert isinstance(received_events[0].timestamp, datetime)
assert received_events[0].type == "crew_test_started"
assert received_events[1].crew_name == "TestCrew"
assert isinstance(received_events[1].timestamp, datetime)
assert received_events[1].type == "crew_test_result"
assert received_events[2].crew_name == "TestCrew"
assert isinstance(received_events[2].timestamp, datetime)
assert received_events[2].type == "crew_test_completed"
assert received_events[1].type == "crew_test_completed"
@pytest.mark.vcr(filter_headers=["authorization"])
@@ -315,7 +309,7 @@ def test_agent_emits_execution_error_event():
) as invoke_mock:
invoke_mock.side_effect = Exception(error_message)
with pytest.raises(Exception):
with pytest.raises(Exception) as e:
base_agent.execute_task(
task=base_task,
)

2
uv.lock generated
View File

@@ -738,7 +738,7 @@ wheels = [
[[package]]
name = "crewai"
version = "0.121.0"
version = "0.120.1"
source = { editable = "." }
dependencies = [
{ name = "appdirs" },