Compare commits
25 Commits
lg-support
...
0.121.0
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
e59627adf2 | ||
|
|
d0855987f8 | ||
|
|
c1672613bc | ||
|
|
9945da7dbe | ||
|
|
31ffa90075 | ||
|
|
169d3233e8 | ||
|
|
b3484c1d0e | ||
|
|
910ed716d9 | ||
|
|
eb6364284f | ||
|
|
e21d54654c | ||
|
|
50b8f83428 | ||
|
|
8d2928e49a | ||
|
|
1ef22131e6 | ||
|
|
227b521f9e | ||
|
|
bef5971598 | ||
|
|
aa6e5b703e | ||
|
|
0b35e40a24 | ||
|
|
49bbf3f234 | ||
|
|
c566747d4a | ||
|
|
3a114463f9 | ||
|
|
b4dfb19a3a | ||
|
|
30ef8ed70b | ||
|
|
e1541b2619 | ||
|
|
7c4889f5c9 | ||
|
|
c403497cf4 |
@@ -21,7 +21,7 @@ Think of an agent as a specialized team member with specific skills, expertise,
|
||||
<Note type="info" title="Enterprise Enhancement: Visual Agent Builder">
|
||||
CrewAI Enterprise includes a Visual Agent Builder that simplifies agent creation and configuration without writing code. Design your agents visually and test them in real-time.
|
||||
|
||||

|
||||

|
||||
|
||||
The Visual Agent Builder enables:
|
||||
- Intuitive agent configuration with form-based interfaces
|
||||
@@ -58,6 +58,8 @@ The Visual Agent Builder enables:
|
||||
| **Embedder** _(optional)_ | `embedder` | `Optional[Dict[str, Any]]` | Configuration for the embedder used by the agent. |
|
||||
| **Knowledge Sources** _(optional)_ | `knowledge_sources` | `Optional[List[BaseKnowledgeSource]]` | Knowledge sources available to the agent. |
|
||||
| **Use System Prompt** _(optional)_ | `use_system_prompt` | `Optional[bool]` | Whether to use system prompt (for o1 model support). Default is True. |
|
||||
| **Inject Date** _(optional)_ | `inject_date` | `bool` | Whether to automatically inject the current date into tasks. Default is False. |
|
||||
| **Date Format** _(optional)_ | `date_format` | `str` | Format string for date when inject_date is enabled. Default is "%Y-%m-%d" (ISO format). |
|
||||
|
||||
## Creating Agents
|
||||
|
||||
@@ -226,6 +228,18 @@ custom_agent = Agent(
|
||||
)
|
||||
```
|
||||
|
||||
#### Date-Aware Agent
|
||||
```python Code
|
||||
date_aware_agent = Agent(
|
||||
role="Market Analyst",
|
||||
goal="Track market movements with precise date references",
|
||||
backstory="Expert in time-sensitive financial analysis and reporting",
|
||||
inject_date=True, # Automatically inject current date into tasks
|
||||
date_format="%B %d, %Y", # Format as "May 21, 2025"
|
||||
verbose=True
|
||||
)
|
||||
```
|
||||
|
||||
### Parameter Details
|
||||
|
||||
#### Critical Parameters
|
||||
@@ -332,6 +346,12 @@ When `memory` is enabled, the agent will maintain context across multiple intera
|
||||
- Main `llm` for complex reasoning
|
||||
- `function_calling_llm` for efficient tool usage
|
||||
|
||||
### Date Awareness
|
||||
- Use `inject_date: true` to provide agents with current date awareness
|
||||
- Customize the date format with `date_format` using standard Python datetime format codes
|
||||
- Valid format codes include: %Y (year), %m (month), %d (day), %B (full month name), etc.
|
||||
- Invalid date formats will be logged as warnings and will not modify the task description
|
||||
|
||||
### Model Compatibility
|
||||
- Set `use_system_prompt: false` for older models that don't support system messages
|
||||
- Ensure your chosen `llm` supports the features you need (like function calling)
|
||||
|
||||
@@ -4,7 +4,7 @@ description: Learn how to use the CrewAI CLI to interact with CrewAI.
|
||||
icon: terminal
|
||||
---
|
||||
|
||||
# CrewAI CLI Documentation
|
||||
## Overview
|
||||
|
||||
The CrewAI CLI provides a set of commands to interact with CrewAI, allowing you to create, train, run, and manage crews & flows.
|
||||
|
||||
@@ -110,6 +110,8 @@ crewai reset-memories [OPTIONS]
|
||||
- `-s, --short`: Reset SHORT TERM memory
|
||||
- `-e, --entities`: Reset ENTITIES memory
|
||||
- `-k, --kickoff-outputs`: Reset LATEST KICKOFF TASK OUTPUTS
|
||||
- `-kn, --knowledge`: Reset KNOWLEDGE storage
|
||||
- `-akn, --agent-knowledge`: Reset AGENT KNOWLEDGE storage
|
||||
- `-a, --all`: Reset ALL memories
|
||||
|
||||
Example:
|
||||
|
||||
@@ -4,7 +4,7 @@ description: Exploring the dynamics of agent collaboration within the CrewAI fra
|
||||
icon: screen-users
|
||||
---
|
||||
|
||||
## Collaboration Fundamentals
|
||||
## Overview
|
||||
|
||||
Collaboration in CrewAI is fundamental, enabling agents to combine their skills, share information, and assist each other in task execution, embodying a truly cooperative ecosystem.
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ description: Understanding and utilizing crews in the crewAI framework with comp
|
||||
icon: people-group
|
||||
---
|
||||
|
||||
## What is a Crew?
|
||||
## Overview
|
||||
|
||||
A crew in crewAI represents a collaborative group of agents working together to achieve a set of tasks. Each crew defines the strategy for task execution, agent collaboration, and the overall workflow.
|
||||
|
||||
@@ -117,6 +117,12 @@ class YourCrewName:
|
||||
)
|
||||
```
|
||||
|
||||
How to run the above code:
|
||||
|
||||
```python code
|
||||
YourCrewName().crew().kickoff(inputs={"any": "input here"})
|
||||
```
|
||||
|
||||
<Note>
|
||||
Tasks will be executed in the order they are defined.
|
||||
</Note>
|
||||
@@ -184,6 +190,11 @@ class YourCrewName:
|
||||
verbose=True
|
||||
)
|
||||
```
|
||||
How to run the above code:
|
||||
|
||||
```python code
|
||||
YourCrewName().crew().kickoff(inputs={})
|
||||
```
|
||||
|
||||
In this example:
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ description: 'Tap into CrewAI events to build custom integrations and monitoring
|
||||
icon: spinner
|
||||
---
|
||||
|
||||
# Event Listeners
|
||||
## Overview
|
||||
|
||||
CrewAI provides a powerful event system that allows you to listen for and react to various events that occur during the execution of your Crew. This feature enables you to build custom integrations, monitoring solutions, logging systems, or any other functionality that needs to be triggered based on CrewAI's internal events.
|
||||
|
||||
@@ -21,7 +21,7 @@ When specific actions occur in CrewAI (like a Crew starting execution, an Agent
|
||||
<Note type="info" title="Enterprise Enhancement: Prompt Tracing">
|
||||
CrewAI Enterprise provides a built-in Prompt Tracing feature that leverages the event system to track, store, and visualize all prompts, completions, and associated metadata. This provides powerful debugging capabilities and transparency into your agent operations.
|
||||
|
||||

|
||||

|
||||
|
||||
With Prompt Tracing you can:
|
||||
- View the complete history of all prompts sent to your LLM
|
||||
@@ -224,6 +224,15 @@ CrewAI provides a wide range of events that you can listen for:
|
||||
- **ToolExecutionErrorEvent**: Emitted when a tool execution encounters an error
|
||||
- **ToolSelectionErrorEvent**: Emitted when there's an error selecting a tool
|
||||
|
||||
### Knowledge Events
|
||||
|
||||
- **KnowledgeRetrievalStartedEvent**: Emitted when a knowledge retrieval is started
|
||||
- **KnowledgeRetrievalCompletedEvent**: Emitted when a knowledge retrieval is completed
|
||||
- **KnowledgeQueryStartedEvent**: Emitted when a knowledge query is started
|
||||
- **KnowledgeQueryCompletedEvent**: Emitted when a knowledge query is completed
|
||||
- **KnowledgeQueryFailedEvent**: Emitted when a knowledge query fails
|
||||
- **KnowledgeSearchQueryFailedEvent**: Emitted when a knowledge search query fails
|
||||
|
||||
### Flow Events
|
||||
|
||||
- **FlowCreatedEvent**: Emitted when a Flow is created
|
||||
|
||||
@@ -4,7 +4,7 @@ description: Learn how to create and manage AI workflows using CrewAI Flows.
|
||||
icon: arrow-progress
|
||||
---
|
||||
|
||||
## Introduction
|
||||
## Overview
|
||||
|
||||
CrewAI Flows is a powerful feature designed to streamline the creation and management of AI workflows. Flows allow developers to combine and coordinate coding tasks and Crews efficiently, providing a robust framework for building sophisticated AI automations.
|
||||
|
||||
@@ -75,11 +75,12 @@ class ExampleFlow(Flow):
|
||||
|
||||
|
||||
flow = ExampleFlow()
|
||||
flow.plot()
|
||||
result = flow.kickoff()
|
||||
|
||||
print(f"Generated fun fact: {result}")
|
||||
```
|
||||
|
||||

|
||||
In the above example, we have created a simple Flow that generates a random city using OpenAI and then generates a fun fact about that city. The Flow consists of two tasks: `generate_city` and `generate_fun_fact`. The `generate_city` task is the starting point of the Flow, and the `generate_fun_fact` task listens for the output of the `generate_city` task.
|
||||
|
||||
Each Flow instance automatically receives a unique identifier (UUID) in its state, which helps track and manage flow executions. The state can also store additional data (like the generated city and fun fact) that persists throughout the flow's execution.
|
||||
@@ -146,6 +147,7 @@ class OutputExampleFlow(Flow):
|
||||
|
||||
|
||||
flow = OutputExampleFlow()
|
||||
flow.plot("my_flow_plot")
|
||||
final_output = flow.kickoff()
|
||||
|
||||
print("---- Final Output ----")
|
||||
@@ -158,9 +160,10 @@ Second method received: Output from first_method
|
||||
```
|
||||
|
||||
</CodeGroup>
|
||||

|
||||
|
||||
In this example, the `second_method` is the last method to complete, so its output will be the final output of the Flow.
|
||||
The `kickoff()` method will return the final output, which is then printed to the console.
|
||||
The `kickoff()` method will return the final output, which is then printed to the console. The `plot()` method will generate the HTML file, which will help you understand the flow.
|
||||
|
||||
#### Accessing and Updating State
|
||||
|
||||
@@ -192,6 +195,7 @@ class StateExampleFlow(Flow[ExampleState]):
|
||||
return self.state.message
|
||||
|
||||
flow = StateExampleFlow()
|
||||
flow.plot("my_flow_plot")
|
||||
final_output = flow.kickoff()
|
||||
print(f"Final Output: {final_output}")
|
||||
print("Final State:")
|
||||
@@ -206,6 +210,8 @@ counter=2 message='Hello from first_method - updated by second_method'
|
||||
|
||||
</CodeGroup>
|
||||
|
||||

|
||||
|
||||
In this example, the state is updated by both `first_method` and `second_method`.
|
||||
After the Flow has run, you can access the final state to see the updates made by these methods.
|
||||
|
||||
@@ -249,9 +255,12 @@ class UnstructuredExampleFlow(Flow):
|
||||
|
||||
|
||||
flow = UnstructuredExampleFlow()
|
||||
flow.plot("my_flow_plot")
|
||||
flow.kickoff()
|
||||
```
|
||||
|
||||

|
||||
|
||||
**Note:** The `id` field is automatically generated and preserved throughout the flow's execution. You don't need to manage or set it manually, and it will be maintained even when updating the state with new data.
|
||||
|
||||
**Key Points:**
|
||||
@@ -302,6 +311,8 @@ flow = StructuredExampleFlow()
|
||||
flow.kickoff()
|
||||
```
|
||||
|
||||

|
||||
|
||||
**Key Points:**
|
||||
|
||||
- **Defined Schema:** `ExampleState` clearly outlines the state structure, enhancing code readability and maintainability.
|
||||
@@ -436,6 +447,7 @@ class OrExampleFlow(Flow):
|
||||
|
||||
|
||||
flow = OrExampleFlow()
|
||||
flow.plot("my_flow_plot")
|
||||
flow.kickoff()
|
||||
```
|
||||
|
||||
@@ -446,6 +458,8 @@ Logger: Hello from the second method
|
||||
|
||||
</CodeGroup>
|
||||
|
||||

|
||||
|
||||
When you run this Flow, the `logger` method will be triggered by the output of either the `start_method` or the `second_method`.
|
||||
The `or_` function is used to listen to multiple methods and trigger the listener method when any of the specified methods emit an output.
|
||||
|
||||
@@ -474,6 +488,7 @@ class AndExampleFlow(Flow):
|
||||
print(self.state)
|
||||
|
||||
flow = AndExampleFlow()
|
||||
flow.plot()
|
||||
flow.kickoff()
|
||||
```
|
||||
|
||||
@@ -484,6 +499,8 @@ flow.kickoff()
|
||||
|
||||
</CodeGroup>
|
||||
|
||||

|
||||
|
||||
When you run this Flow, the `logger` method will be triggered only when both the `start_method` and the `second_method` emit an output.
|
||||
The `and_` function is used to listen to multiple methods and trigger the listener method only when all the specified methods emit an output.
|
||||
|
||||
@@ -527,6 +544,7 @@ class RouterFlow(Flow[ExampleState]):
|
||||
|
||||
|
||||
flow = RouterFlow()
|
||||
flow.plot("my_flow_plot")
|
||||
flow.kickoff()
|
||||
```
|
||||
|
||||
@@ -538,6 +556,8 @@ Fourth method running
|
||||
|
||||
</CodeGroup>
|
||||
|
||||

|
||||
|
||||
In the above example, the `start_method` generates a random boolean value and sets it in the state.
|
||||
The `second_method` uses the `@router()` decorator to define conditional routing logic based on the value of the boolean.
|
||||
If the boolean is `True`, the method returns `"success"`, and if it is `False`, the method returns `"failed"`.
|
||||
@@ -641,6 +661,7 @@ class MarketResearchFlow(Flow[MarketResearchState]):
|
||||
# Usage example
|
||||
async def run_flow():
|
||||
flow = MarketResearchFlow()
|
||||
flow.plot("MarketResearchFlowPlot")
|
||||
result = await flow.kickoff_async(inputs={"product": "AI-powered chatbots"})
|
||||
return result
|
||||
|
||||
@@ -650,6 +671,8 @@ if __name__ == "__main__":
|
||||
asyncio.run(run_flow())
|
||||
```
|
||||
|
||||

|
||||
|
||||
This example demonstrates several key features of using Agents in flows:
|
||||
|
||||
1. **Structured Output**: Using Pydantic models to define the expected output format (`MarketAnalysis`) ensures type safety and structured data throughout the flow.
|
||||
@@ -746,13 +769,16 @@ def kickoff():
|
||||
|
||||
def plot():
|
||||
poem_flow = PoemFlow()
|
||||
poem_flow.plot()
|
||||
poem_flow.plot("PoemFlowPlot")
|
||||
|
||||
if __name__ == "__main__":
|
||||
kickoff()
|
||||
plot()
|
||||
```
|
||||
|
||||
In this example, the `PoemFlow` class defines a flow that generates a sentence count, uses the `PoemCrew` to generate a poem, and then saves the poem to a file. The flow is kicked off by calling the `kickoff()` method.
|
||||
In this example, the `PoemFlow` class defines a flow that generates a sentence count, uses the `PoemCrew` to generate a poem, and then saves the poem to a file. The flow is kicked off by calling the `kickoff()` method. The PoemFlowPlot will be generated by `plot()` method.
|
||||
|
||||

|
||||
|
||||
### Running the Flow
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ description: What is knowledge in CrewAI and how to use it.
|
||||
icon: book
|
||||
---
|
||||
|
||||
## What is Knowledge?
|
||||
## Overview
|
||||
|
||||
Knowledge in CrewAI is a powerful system that allows AI agents to access and utilize external information sources during their tasks.
|
||||
Think of it as giving your agents a reference library they can consult while working.
|
||||
@@ -36,12 +36,15 @@ CrewAI supports various types of knowledge sources out of the box:
|
||||
|
||||
## Supported Knowledge Parameters
|
||||
|
||||
| Parameter | Type | Required | Description |
|
||||
| :--------------------------- | :---------------------------------- | :------- | :---------------------------------------------------------------------------------------------------------------------------------------------------- |
|
||||
| `sources` | **List[BaseKnowledgeSource]** | Yes | List of knowledge sources that provide content to be stored and queried. Can include PDF, CSV, Excel, JSON, text files, or string content. |
|
||||
| `collection_name` | **str** | No | Name of the collection where the knowledge will be stored. Used to identify different sets of knowledge. Defaults to "knowledge" if not provided. |
|
||||
| `storage` | **Optional[KnowledgeStorage]** | No | Custom storage configuration for managing how the knowledge is stored and retrieved. If not provided, a default storage will be created. |
|
||||
|
||||
<ParamField body="sources" type="List[BaseKnowledgeSource]" required="Yes">
|
||||
List of knowledge sources that provide content to be stored and queried. Can include PDF, CSV, Excel, JSON, text files, or string content.
|
||||
</ParamField>
|
||||
<ParamField body="collection_name" type="str">
|
||||
Name of the collection where the knowledge will be stored. Used to identify different sets of knowledge. Defaults to \"knowledge\" if not provided.
|
||||
</ParamField>
|
||||
<ParamField body="storage" type="Optional[KnowledgeStorage]">
|
||||
Custom storage configuration for managing how the knowledge is stored and retrieved. If not provided, a default storage will be created.
|
||||
</ParamField>
|
||||
|
||||
<Tip>
|
||||
Unlike retrieval from a vector database using a tool, agents preloaded with knowledge will not need a retrieval persona or task.
|
||||
@@ -432,6 +435,46 @@ Query rewriting happens transparently using a system prompt that instructs the L
|
||||
This mechanism is fully automatic and requires no configuration from users. The agent's LLM is used to perform the query rewriting, so using a more capable LLM can improve the quality of rewritten queries.
|
||||
</Tip>
|
||||
|
||||
## Knowledge Events
|
||||
|
||||
CrewAI emits events during the knowledge retrieval process that you can listen for using the event system. These events allow you to monitor, debug, and analyze how knowledge is being retrieved and used by your agents.
|
||||
|
||||
### Available Knowledge Events
|
||||
|
||||
- **KnowledgeRetrievalStartedEvent**: Emitted when an agent starts retrieving knowledge from sources
|
||||
- **KnowledgeRetrievalCompletedEvent**: Emitted when knowledge retrieval is completed, including the query used and the retrieved content
|
||||
- **KnowledgeQueryStartedEvent**: Emitted when a query to knowledge sources begins
|
||||
- **KnowledgeQueryCompletedEvent**: Emitted when a query completes successfully
|
||||
- **KnowledgeQueryFailedEvent**: Emitted when a query to knowledge sources fails
|
||||
- **KnowledgeSearchQueryFailedEvent**: Emitted when a search query fails
|
||||
|
||||
### Example: Monitoring Knowledge Retrieval
|
||||
|
||||
```python
|
||||
from crewai.utilities.events import (
|
||||
KnowledgeRetrievalStartedEvent,
|
||||
KnowledgeRetrievalCompletedEvent,
|
||||
)
|
||||
from crewai.utilities.events.base_event_listener import BaseEventListener
|
||||
|
||||
class KnowledgeMonitorListener(BaseEventListener):
|
||||
def setup_listeners(self, crewai_event_bus):
|
||||
@crewai_event_bus.on(KnowledgeRetrievalStartedEvent)
|
||||
def on_knowledge_retrieval_started(source, event):
|
||||
print(f"Agent '{event.agent.role}' started retrieving knowledge")
|
||||
|
||||
@crewai_event_bus.on(KnowledgeRetrievalCompletedEvent)
|
||||
def on_knowledge_retrieval_completed(source, event):
|
||||
print(f"Agent '{event.agent.role}' completed knowledge retrieval")
|
||||
print(f"Query: {event.query}")
|
||||
print(f"Retrieved {len(event.retrieved_knowledge)} knowledge chunks")
|
||||
|
||||
# Create an instance of your listener
|
||||
knowledge_monitor = KnowledgeMonitorListener()
|
||||
```
|
||||
|
||||
For more information on using events, see the [Event Listeners](https://docs.crewai.com/concepts/event-listener) documentation.
|
||||
|
||||
### Example
|
||||
|
||||
```python
|
||||
@@ -497,6 +540,13 @@ crew = Crew(
|
||||
result = crew.kickoff(
|
||||
inputs={"question": "What is the storage capacity of the XPS 13?"}
|
||||
)
|
||||
|
||||
# Resetting the agent specific knowledge via crew object
|
||||
crew.reset_memories(command_type = 'agent_knowledge')
|
||||
|
||||
# Resetting the agent specific knowledge via CLI
|
||||
crewai reset-memories --agent-knowledge
|
||||
crewai reset-memories -akn
|
||||
```
|
||||
|
||||
<Info>
|
||||
|
||||
@@ -4,9 +4,10 @@ description: 'A comprehensive guide to configuring and using Large Language Mode
|
||||
icon: 'microchip-ai'
|
||||
---
|
||||
|
||||
<Note>
|
||||
CrewAI integrates with multiple LLM providers through LiteLLM, giving you the flexibility to choose the right model for your specific use case. This guide will help you understand how to configure and use different LLM providers in your CrewAI projects.
|
||||
</Note>
|
||||
## Overview
|
||||
|
||||
CrewAI integrates with multiple LLM providers through LiteLLM, giving you the flexibility to choose the right model for your specific use case. This guide will help you understand how to configure and use different LLM providers in your CrewAI projects.
|
||||
|
||||
|
||||
## What are LLMs?
|
||||
|
||||
@@ -677,18 +678,24 @@ CrewAI supports streaming responses from LLMs, allowing your application to rece
|
||||
CrewAI emits events for each chunk received during streaming:
|
||||
|
||||
```python
|
||||
from crewai import LLM
|
||||
from crewai.utilities.events import EventHandler, LLMStreamChunkEvent
|
||||
from crewai.utilities.events import (
|
||||
LLMStreamChunkEvent
|
||||
)
|
||||
from crewai.utilities.events.base_event_listener import BaseEventListener
|
||||
|
||||
class MyEventHandler(EventHandler):
|
||||
def on_llm_stream_chunk(self, event: LLMStreamChunkEvent):
|
||||
# Process each chunk as it arrives
|
||||
print(f"Received chunk: {event.chunk}")
|
||||
class MyCustomListener(BaseEventListener):
|
||||
def setup_listeners(self, crewai_event_bus):
|
||||
@crewai_event_bus.on(LLMStreamChunkEvent)
|
||||
def on_llm_stream_chunk(self, event: LLMStreamChunkEvent):
|
||||
# Process each chunk as it arrives
|
||||
print(f"Received chunk: {event.chunk}")
|
||||
|
||||
# Register the event handler
|
||||
from crewai.utilities.events import crewai_event_bus
|
||||
crewai_event_bus.register_handler(MyEventHandler())
|
||||
my_listener = MyCustomListener()
|
||||
```
|
||||
|
||||
<Tip>
|
||||
[Click here](https://docs.crewai.com/concepts/event-listener#event-listeners) for more details
|
||||
</Tip>
|
||||
</Tab>
|
||||
</Tabs>
|
||||
|
||||
@@ -785,6 +792,24 @@ Learn how to get the most out of your LLM configuration:
|
||||
Remember to regularly monitor your token usage and adjust your configuration as needed to optimize costs and performance.
|
||||
</Info>
|
||||
</Accordion>
|
||||
|
||||
<Accordion title="Drop Additional Parameters">
|
||||
CrewAI internally uses Litellm for LLM calls, which allows you to drop additional parameters that are not needed for your specific use case. This can help simplify your code and reduce the complexity of your LLM configuration.
|
||||
For example, if you don't need to send the <code>stop</code> parameter, you can simply omit it from your LLM call:
|
||||
|
||||
```python
|
||||
from crewai import LLM
|
||||
import os
|
||||
|
||||
os.environ["OPENAI_API_KEY"] = "<api-key>"
|
||||
|
||||
o3_llm = LLM(
|
||||
model="o3",
|
||||
drop_params=True,
|
||||
additional_drop_params=["stop"]
|
||||
)
|
||||
```
|
||||
</Accordion>
|
||||
</AccordionGroup>
|
||||
|
||||
## Common Issues and Solutions
|
||||
|
||||
@@ -4,7 +4,7 @@ description: Leveraging memory systems in the CrewAI framework to enhance agent
|
||||
icon: database
|
||||
---
|
||||
|
||||
## Introduction to Memory Systems in CrewAI
|
||||
## Overview
|
||||
|
||||
The crewAI framework introduces a sophisticated memory system designed to significantly enhance the capabilities of AI agents.
|
||||
This system comprises `short-term memory`, `long-term memory`, `entity memory`, and `contextual memory`, each serving a unique purpose in aiding agents to remember,
|
||||
@@ -679,6 +679,7 @@ crewai reset-memories [OPTIONS]
|
||||
| `-e`, `--entities` | Reset ENTITIES memory. | Flag (boolean) | False |
|
||||
| `-k`, `--kickoff-outputs` | Reset LATEST KICKOFF TASK OUTPUTS. | Flag (boolean) | False |
|
||||
| `-kn`, `--knowledge` | Reset KNOWLEDEGE storage | Flag (boolean) | False |
|
||||
| `-akn`, `--agent-knowledge` | Reset AGENT KNOWLEDGE storage | Flag (boolean) | False |
|
||||
| `-a`, `--all` | Reset ALL memories. | Flag (boolean) | False |
|
||||
|
||||
Note: To use the cli command you need to have your crew in a file called crew.py in the same directory.
|
||||
@@ -716,9 +717,11 @@ my_crew.reset_memories(command_type = 'all') # Resets all the memory
|
||||
| `entities` | Reset ENTITIES memory. |
|
||||
| `kickoff_outputs` | Reset LATEST KICKOFF TASK OUTPUTS. |
|
||||
| `knowledge` | Reset KNOWLEDGE memory. |
|
||||
| `agent_knowledge` | Reset AGENT KNOWLEDGE memory. |
|
||||
| `all` | Reset ALL memories. |
|
||||
|
||||
|
||||
|
||||
## Benefits of Using CrewAI's Memory System
|
||||
|
||||
- 🦾 **Adaptive Learning:** Crews become more efficient over time, adapting to new information and refining their approach to tasks.
|
||||
|
||||
@@ -1,10 +1,10 @@
|
||||
---
|
||||
title: Planning
|
||||
description: Learn how to add planning to your CrewAI Crew and improve their performance.
|
||||
icon: brain
|
||||
icon: ruler-combined
|
||||
---
|
||||
|
||||
## Introduction
|
||||
## Overview
|
||||
|
||||
The planning feature in CrewAI allows you to add planning capability to your crew. When enabled, before each Crew iteration,
|
||||
all Crew information is sent to an AgentPlanner that will plan the tasks step by step, and this plan will be added to each task description.
|
||||
|
||||
@@ -4,7 +4,8 @@ description: Detailed guide on workflow management through processes in CrewAI,
|
||||
icon: bars-staggered
|
||||
---
|
||||
|
||||
## Understanding Processes
|
||||
## Overview
|
||||
|
||||
<Tip>
|
||||
Processes orchestrate the execution of tasks by agents, akin to project management in human teams.
|
||||
These processes ensure tasks are distributed and executed efficiently, in alignment with a predefined strategy.
|
||||
|
||||
147
docs/concepts/reasoning.mdx
Normal file
@@ -0,0 +1,147 @@
|
||||
---
|
||||
title: Reasoning
|
||||
description: "Learn how to enable and use agent reasoning to improve task execution."
|
||||
icon: brain
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
Agent reasoning is a feature that allows agents to reflect on a task and create a plan before execution. This helps agents approach tasks more methodically and ensures they're ready to perform the assigned work.
|
||||
|
||||
## Usage
|
||||
|
||||
To enable reasoning for an agent, simply set `reasoning=True` when creating the agent:
|
||||
|
||||
```python
|
||||
from crewai import Agent
|
||||
|
||||
agent = Agent(
|
||||
role="Data Analyst",
|
||||
goal="Analyze complex datasets and provide insights",
|
||||
backstory="You are an experienced data analyst with expertise in finding patterns in complex data.",
|
||||
reasoning=True, # Enable reasoning
|
||||
max_reasoning_attempts=3 # Optional: Set a maximum number of reasoning attempts
|
||||
)
|
||||
```
|
||||
|
||||
## How It Works
|
||||
|
||||
When reasoning is enabled, before executing a task, the agent will:
|
||||
|
||||
1. Reflect on the task and create a detailed plan
|
||||
2. Evaluate whether it's ready to execute the task
|
||||
3. Refine the plan as necessary until it's ready or max_reasoning_attempts is reached
|
||||
4. Inject the reasoning plan into the task description before execution
|
||||
|
||||
This process helps the agent break down complex tasks into manageable steps and identify potential challenges before starting.
|
||||
|
||||
## Configuration Options
|
||||
|
||||
<ParamField body="reasoning" type="bool" default="False">
|
||||
Enable or disable reasoning
|
||||
</ParamField>
|
||||
|
||||
<ParamField body="max_reasoning_attempts" type="int" default="None">
|
||||
Maximum number of attempts to refine the plan before proceeding with execution. If None (default), the agent will continue refining until it's ready.
|
||||
</ParamField>
|
||||
|
||||
## Example
|
||||
|
||||
Here's a complete example:
|
||||
|
||||
```python
|
||||
from crewai import Agent, Task, Crew
|
||||
|
||||
# Create an agent with reasoning enabled
|
||||
analyst = Agent(
|
||||
role="Data Analyst",
|
||||
goal="Analyze data and provide insights",
|
||||
backstory="You are an expert data analyst.",
|
||||
reasoning=True,
|
||||
max_reasoning_attempts=3 # Optional: Set a limit on reasoning attempts
|
||||
)
|
||||
|
||||
# Create a task
|
||||
analysis_task = Task(
|
||||
description="Analyze the provided sales data and identify key trends.",
|
||||
expected_output="A report highlighting the top 3 sales trends.",
|
||||
agent=analyst
|
||||
)
|
||||
|
||||
# Create a crew and run the task
|
||||
crew = Crew(agents=[analyst], tasks=[analysis_task])
|
||||
result = crew.kickoff()
|
||||
|
||||
print(result)
|
||||
```
|
||||
|
||||
## Error Handling
|
||||
|
||||
The reasoning process is designed to be robust, with error handling built in. If an error occurs during reasoning, the agent will proceed with executing the task without the reasoning plan. This ensures that tasks can still be executed even if the reasoning process fails.
|
||||
|
||||
Here's how to handle potential errors in your code:
|
||||
|
||||
```python
|
||||
from crewai import Agent, Task
|
||||
import logging
|
||||
|
||||
# Set up logging to capture any reasoning errors
|
||||
logging.basicConfig(level=logging.INFO)
|
||||
|
||||
# Create an agent with reasoning enabled
|
||||
agent = Agent(
|
||||
role="Data Analyst",
|
||||
goal="Analyze data and provide insights",
|
||||
reasoning=True,
|
||||
max_reasoning_attempts=3
|
||||
)
|
||||
|
||||
# Create a task
|
||||
task = Task(
|
||||
description="Analyze the provided sales data and identify key trends.",
|
||||
expected_output="A report highlighting the top 3 sales trends.",
|
||||
agent=agent
|
||||
)
|
||||
|
||||
# Execute the task
|
||||
# If an error occurs during reasoning, it will be logged and execution will continue
|
||||
result = agent.execute_task(task)
|
||||
```
|
||||
|
||||
## Example Reasoning Output
|
||||
|
||||
Here's an example of what a reasoning plan might look like for a data analysis task:
|
||||
|
||||
```
|
||||
Task: Analyze the provided sales data and identify key trends.
|
||||
|
||||
Reasoning Plan:
|
||||
I'll analyze the sales data to identify the top 3 trends.
|
||||
|
||||
1. Understanding of the task:
|
||||
I need to analyze sales data to identify key trends that would be valuable for business decision-making.
|
||||
|
||||
2. Key steps I'll take:
|
||||
- First, I'll examine the data structure to understand what fields are available
|
||||
- Then I'll perform exploratory data analysis to identify patterns
|
||||
- Next, I'll analyze sales by time periods to identify temporal trends
|
||||
- I'll also analyze sales by product categories and customer segments
|
||||
- Finally, I'll identify the top 3 most significant trends
|
||||
|
||||
3. Approach to challenges:
|
||||
- If the data has missing values, I'll decide whether to fill or filter them
|
||||
- If the data has outliers, I'll investigate whether they're valid data points or errors
|
||||
- If trends aren't immediately obvious, I'll apply statistical methods to uncover patterns
|
||||
|
||||
4. Use of available tools:
|
||||
- I'll use data analysis tools to explore and visualize the data
|
||||
- I'll use statistical tools to identify significant patterns
|
||||
- I'll use knowledge retrieval to access relevant information about sales analysis
|
||||
|
||||
5. Expected outcome:
|
||||
A concise report highlighting the top 3 sales trends with supporting evidence from the data.
|
||||
|
||||
READY: I am ready to execute the task.
|
||||
```
|
||||
|
||||
This reasoning plan helps the agent organize its approach to the task, consider potential challenges, and ensure it delivers the expected output.
|
||||
@@ -4,7 +4,7 @@ description: Detailed guide on managing and creating tasks within the CrewAI fra
|
||||
icon: list-check
|
||||
---
|
||||
|
||||
## Overview of a Task
|
||||
## Overview
|
||||
|
||||
In the CrewAI framework, a `Task` is a specific assignment completed by an `Agent`.
|
||||
|
||||
@@ -15,7 +15,7 @@ Tasks within CrewAI can be collaborative, requiring multiple agents to work toge
|
||||
<Note type="info" title="Enterprise Enhancement: Visual Task Builder">
|
||||
CrewAI Enterprise includes a Visual Task Builder in Crew Studio that simplifies complex task creation and chaining. Design your task flows visually and test them in real-time without writing code.
|
||||
|
||||

|
||||

|
||||
|
||||
The Visual Task Builder enables:
|
||||
- Drag-and-drop task creation
|
||||
|
||||
@@ -4,7 +4,7 @@ description: Learn how to test your CrewAI Crew and evaluate their performance.
|
||||
icon: vial
|
||||
---
|
||||
|
||||
## Introduction
|
||||
## Overview
|
||||
|
||||
Testing is a crucial part of the development process, and it is essential to ensure that your crew is performing as expected. With crewAI, you can easily test your crew and evaluate its performance using the built-in testing capabilities.
|
||||
|
||||
|
||||
@@ -4,7 +4,7 @@ description: Understanding and leveraging tools within the CrewAI framework for
|
||||
icon: screwdriver-wrench
|
||||
---
|
||||
|
||||
## Introduction
|
||||
## Overview
|
||||
|
||||
CrewAI tools empower agents with capabilities ranging from web searching and data analysis to collaboration and delegating tasks among coworkers.
|
||||
This documentation outlines how to create, integrate, and leverage these tools within the CrewAI framework, including a new focus on collaboration tools.
|
||||
@@ -18,8 +18,6 @@ enabling everything from simple searches to complex interactions and effective t
|
||||
<Note type="info" title="Enterprise Enhancement: Tools Repository">
|
||||
CrewAI Enterprise provides a comprehensive Tools Repository with pre-built integrations for common business systems and APIs. Deploy agents with enterprise tools in minutes instead of days.
|
||||
|
||||

|
||||
|
||||
The Enterprise Tools Repository includes:
|
||||
- Pre-built connectors for popular enterprise systems
|
||||
- Custom tool creation interface
|
||||
|
||||
@@ -4,7 +4,7 @@ description: Learn how to train your CrewAI agents by giving them feedback early
|
||||
icon: dumbbell
|
||||
---
|
||||
|
||||
## Introduction
|
||||
## Overview
|
||||
|
||||
The training feature in CrewAI allows you to train your AI agents using the command-line interface (CLI).
|
||||
By running the command `crewai train -n <n_iterations>`, you can specify the number of iterations for the training process.
|
||||
|
||||
@@ -74,6 +74,7 @@
|
||||
"concepts/collaboration",
|
||||
"concepts/training",
|
||||
"concepts/memory",
|
||||
"concepts/reasoning",
|
||||
"concepts/planning",
|
||||
"concepts/testing",
|
||||
"concepts/cli",
|
||||
@@ -129,6 +130,7 @@
|
||||
"tools/seleniumscrapingtool",
|
||||
"tools/snowflakesearchtool",
|
||||
"tools/spidertool",
|
||||
"tools/stagehandtool",
|
||||
"tools/txtsearchtool",
|
||||
"tools/visiontool",
|
||||
"tools/weaviatevectorsearchtool",
|
||||
@@ -138,6 +140,12 @@
|
||||
"tools/youtubevideosearchtool"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "MCP Integration",
|
||||
"pages": [
|
||||
"mcp/crewai-mcp-integration"
|
||||
]
|
||||
},
|
||||
{
|
||||
"group": "Agent Monitoring & Observability",
|
||||
"pages": [
|
||||
|
||||
@@ -64,14 +64,14 @@ CrewAI Enterprise extends the power of the open-source framework with features d
|
||||
Sign Up
|
||||
</Card>
|
||||
</Step>
|
||||
<Step title="Create your first crew">
|
||||
Use code or Crew Studio to create your crew
|
||||
<Step title="Build your first crew">
|
||||
Use code or Crew Studio to build your crew
|
||||
<Card
|
||||
title="Create Crew"
|
||||
title="Build Crew"
|
||||
icon="paintbrush"
|
||||
href="/enterprise/guides/create-crew"
|
||||
href="/enterprise/guides/build-crew"
|
||||
>
|
||||
Create Crew
|
||||
Build Crew
|
||||
</Card>
|
||||
</Step>
|
||||
<Step title="Deploy your crew">
|
||||
|
||||
@@ -4,8 +4,6 @@ description: Dive deeper into low-level prompt customization for CrewAI, enablin
|
||||
icon: message-pen
|
||||
---
|
||||
|
||||
# Customizing Prompts at a Low Level
|
||||
|
||||
## Why Customize Prompts?
|
||||
|
||||
Although CrewAI's default prompts work well for many scenarios, low-level customization opens the door to significantly more flexible and powerful agent behavior. Here’s why you might want to take advantage of this deeper control:
|
||||
|
||||
@@ -4,8 +4,6 @@ description: Learn how to use CrewAI's fingerprinting system to uniquely identif
|
||||
icon: fingerprint
|
||||
---
|
||||
|
||||
# Fingerprinting in CrewAI
|
||||
|
||||
## Overview
|
||||
|
||||
Fingerprints in CrewAI provide a way to uniquely identify and track components throughout their lifecycle. Each `Agent`, `Crew`, and `Task` automatically receives a unique fingerprint when created, which cannot be manually overridden.
|
||||
|
||||
@@ -4,8 +4,6 @@ description: Learn best practices for designing powerful, specialized AI agents
|
||||
icon: robot
|
||||
---
|
||||
|
||||
# Crafting Effective Agents
|
||||
|
||||
## The Art and Science of Agent Design
|
||||
|
||||
At the heart of CrewAI lies the agent - a specialized AI entity designed to perform specific roles within a collaborative framework. While creating basic agents is simple, crafting truly effective agents that produce exceptional results requires understanding key design principles and best practices.
|
||||
|
||||
@@ -4,8 +4,6 @@ description: Learn how to assess your AI application needs and choose the right
|
||||
icon: scale-balanced
|
||||
---
|
||||
|
||||
# Evaluating Use Cases for CrewAI
|
||||
|
||||
## Understanding the Decision Framework
|
||||
|
||||
When building AI applications with CrewAI, one of the most important decisions you'll make is choosing the right approach for your specific use case. Should you use a Crew? A Flow? A combination of both? This guide will help you evaluate your requirements and make informed architectural decisions.
|
||||
|
||||
@@ -4,8 +4,6 @@ description: Step-by-step tutorial to create a collaborative AI team that works
|
||||
icon: users-gear
|
||||
---
|
||||
|
||||
# Build Your First Crew
|
||||
|
||||
## Unleashing the Power of Collaborative AI
|
||||
|
||||
Imagine having a team of specialized AI agents working together seamlessly to solve complex problems, each contributing their unique skills to achieve a common goal. This is the power of CrewAI - a framework that enables you to create collaborative AI systems that can accomplish tasks far beyond what a single AI could achieve alone.
|
||||
|
||||
@@ -4,8 +4,6 @@ description: Learn how to create structured, event-driven workflows with precise
|
||||
icon: diagram-project
|
||||
---
|
||||
|
||||
# Build Your First Flow
|
||||
|
||||
## Taking Control of AI Workflows with Flows
|
||||
|
||||
CrewAI Flows represent the next level in AI orchestration - combining the collaborative power of AI agent crews with the precision and flexibility of procedural programming. While crews excel at agent collaboration, flows give you fine-grained control over exactly how and when different components of your AI system interact.
|
||||
|
||||
@@ -4,8 +4,6 @@ description: A comprehensive guide to managing, persisting, and leveraging state
|
||||
icon: diagram-project
|
||||
---
|
||||
|
||||
# Mastering Flow State Management
|
||||
|
||||
## Understanding the Power of State in Flows
|
||||
|
||||
State management is the backbone of any sophisticated AI workflow. In CrewAI Flows, the state system allows you to maintain context, share data between steps, and build complex application logic. Mastering state management is essential for creating reliable, maintainable, and powerful AI applications.
|
||||
|
||||
@@ -12,7 +12,8 @@ The before kickoff hook is executed before the crew starts its tasks. It receive
|
||||
Here's an example of defining a before kickoff function in your `crew.py`:
|
||||
|
||||
```python
|
||||
from crewai import CrewBase, before_kickoff
|
||||
from crewai import CrewBase
|
||||
from crewai.project import before_kickoff
|
||||
|
||||
@CrewBase
|
||||
class MyCrew:
|
||||
@@ -34,7 +35,8 @@ The after kickoff hook is executed after the crew has completed its tasks. It re
|
||||
Here's how you can define an after kickoff function in your `crew.py`:
|
||||
|
||||
```python
|
||||
from crewai import CrewBase, after_kickoff
|
||||
from crewai import CrewBase
|
||||
from crewai.project import after_kickoff
|
||||
|
||||
@CrewBase
|
||||
class MyCrew:
|
||||
|
||||
BIN
docs/images/crewai-flow-1.png
Normal file
|
After Width: | Height: | Size: 44 KiB |
BIN
docs/images/crewai-flow-2.png
Normal file
|
After Width: | Height: | Size: 43 KiB |
BIN
docs/images/crewai-flow-3.png
Normal file
|
After Width: | Height: | Size: 45 KiB |
BIN
docs/images/crewai-flow-4.png
Normal file
|
After Width: | Height: | Size: 57 KiB |
BIN
docs/images/crewai-flow-5.png
Normal file
|
After Width: | Height: | Size: 48 KiB |
BIN
docs/images/crewai-flow-6.png
Normal file
|
After Width: | Height: | Size: 57 KiB |
BIN
docs/images/crewai-flow-7.png
Normal file
|
After Width: | Height: | Size: 60 KiB |
BIN
docs/images/crewai-flow-8.png
Normal file
|
After Width: | Height: | Size: 48 KiB |
BIN
docs/images/enterprise/crew-secrets.png
Normal file
|
After Width: | Height: | Size: 362 KiB |
BIN
docs/images/enterprise/mcp-enterprise-download.png
Normal file
|
After Width: | Height: | Size: 241 KiB |
229
docs/mcp/crewai-mcp-integration.mdx
Normal file
@@ -0,0 +1,229 @@
|
||||
---
|
||||
title: 'MCP Servers as Tools in CrewAI'
|
||||
description: 'Learn how to integrate MCP servers as tools in your CrewAI agents using the `crewai-tools` library.'
|
||||
icon: 'plug'
|
||||
---
|
||||
|
||||
## Overview
|
||||
|
||||
The [Model Context Protocol](https://modelcontextprotocol.io/introduction) (MCP) provides a standardized way for AI agents to provide context to LLMs by communicating with external services, known as MCP Servers.
|
||||
The `crewai-tools` library extends CrewAI's capabilities by allowing you to seamlessly integrate tools from these MCP servers into your agents.
|
||||
This gives your crews access to a vast ecosystem of functionalities. For now, we support **Standard Input/Output** (Stdio) and **Server-Sent Events** (SSE) transport mechanisms.
|
||||
|
||||
<Info>
|
||||
We will also be integrating **Streamable HTTP** transport in the near future.
|
||||
Streamable HTTP is designed for efficient, bi-directional communication over a single HTTP connection.
|
||||
</Info>
|
||||
|
||||
## Installation
|
||||
|
||||
Before you start using MCP with `crewai-tools`, you need to install the `mcp` extra `crewai-tools` dependency with the following command:
|
||||
|
||||
```shell
|
||||
uv pip install 'crewai-tools[mcp]'
|
||||
```
|
||||
|
||||
### Integrating MCP Tools with `MCPServerAdapter`
|
||||
|
||||
The `MCPServerAdapter` class from `crewai-tools` is the primary way to connect to an MCP server and make its tools available to your CrewAI agents.
|
||||
It supports different transport mechanisms, primarily **Stdio** (for local servers) and **SSE** (Server-Sent Events).You have two main options for managing the connection lifecycle:
|
||||
|
||||
### Option 1: Fully Managed Connection (Recommended)
|
||||
|
||||
Using a Python context manager (`with` statement) is the recommended approach. It automatically handles starting and stopping the connection to the MCP server.
|
||||
|
||||
**For a local Stdio-based MCP server:**
|
||||
|
||||
```python
|
||||
from crewai import Agent, Task, Crew
|
||||
from crewai_tools import MCPServerAdapter
|
||||
from mcp import StdioServerParameters
|
||||
import os
|
||||
|
||||
server_params=StdioServerParameters(
|
||||
command="uxv", # Or your python3 executable i.e. "python3"
|
||||
args=["mock_server.py"],
|
||||
env={"UV_PYTHON": "3.12", **os.environ},
|
||||
)
|
||||
|
||||
with MCPServerAdapter(server_params) as tools:
|
||||
print(f"Available tools from Stdio MCP server: {[tool.name for tool in tools]}")
|
||||
|
||||
# Example: Using the tools from the Stdio MCP server in a CrewAI Agent
|
||||
agent = Agent(
|
||||
role="Web Information Retriever",
|
||||
goal="Scrape content from a specified URL.",
|
||||
backstory="An AI that can fetch and process web page data via an MCP tool.",
|
||||
tools=tools,
|
||||
verbose=True,
|
||||
)
|
||||
task = Task(
|
||||
description="Scrape content from a specified URL.",
|
||||
expected_output="Scraped content from the specified URL.",
|
||||
agent=agent,
|
||||
)
|
||||
crew = Crew(
|
||||
agents=[agent],
|
||||
tasks=[task],
|
||||
verbose=True,
|
||||
)
|
||||
result = crew.kickoff()
|
||||
print(result)
|
||||
```
|
||||
|
||||
**For a remote SSE-based MCP server:**
|
||||
|
||||
```python
|
||||
from crewai_tools import MCPServerAdapter
|
||||
from crewai import Agent, Task, Crew
|
||||
|
||||
server_params = {"url": "http://localhost:8000/sse"}
|
||||
|
||||
with MCPServerAdapter(server_params) as tools:
|
||||
print(f"Available tools from SSE MCP server: {[tool.name for tool in tools]}")
|
||||
|
||||
# Example: Using the tools from the SSE MCP server in a CrewAI Agent
|
||||
agent = Agent(
|
||||
role="Web Information Retriever",
|
||||
goal="Scrape content from a specified URL.",
|
||||
backstory="An AI that can fetch and process web page data via an MCP tool.",
|
||||
tools=tools,
|
||||
verbose=True,
|
||||
)
|
||||
task = Task(
|
||||
description="Scrape content from a specified URL.",
|
||||
expected_output="Scraped content from the specified URL.",
|
||||
agent=agent,
|
||||
)
|
||||
crew = Crew(
|
||||
agents=[agent],
|
||||
tasks=[task],
|
||||
verbose=True,
|
||||
)
|
||||
result = crew.kickoff()
|
||||
print(result)
|
||||
```
|
||||
|
||||
### Option 2: More control over the MCP server connection lifecycle
|
||||
|
||||
If you need finer-grained control over the MCP server connection lifecycle, you can instantiate `MCPServerAdapter` directly and manage its `start()` and `stop()` methods.
|
||||
|
||||
<Info>
|
||||
You **MUST** call `mcp_server_adapter.stop()` to ensure the connection is closed and resources are released. Using a `try...finally` block is highly recommended.
|
||||
</Info>
|
||||
|
||||
#### Stdio Transport Example (Manual)
|
||||
|
||||
```python
|
||||
from mcp import StdioServerParameters
|
||||
from crewai_tools import MCPServerAdapter
|
||||
from crewai import Agent, Task, Crew
|
||||
import os
|
||||
|
||||
stdio_params = StdioServerParameters(
|
||||
command="uvx", # Or your python3 executable i.e. "python3"
|
||||
args=["--quiet", "your-mcp-server@0.1.3"],
|
||||
env={"UV_PYTHON": "3.12", **os.environ},
|
||||
)
|
||||
|
||||
mcp_server_adapter = MCPServerAdapter(server_params=stdio_params)
|
||||
try:
|
||||
mcp_server_adapter.start() # Manually start the connection
|
||||
tools = mcp_server_adapter.tools
|
||||
print(f"Available tools (manual Stdio): {[tool.name for tool in tools]}")
|
||||
|
||||
# Use 'tools' with your Agent, Task, Crew setup as in Option 1
|
||||
agent = Agent(
|
||||
role="Medical Researcher",
|
||||
goal="Find recent studies on a given topic using PubMed.",
|
||||
backstory="An AI assistant specialized in biomedical literature research.",
|
||||
tools=tools,
|
||||
verbose=True
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Search for recent articles on 'crispr gene editing'.",
|
||||
expected_output="A summary of the top 3 recent articles.",
|
||||
agent=agent
|
||||
)
|
||||
|
||||
crew = Crew(
|
||||
agents=[agent],
|
||||
tasks=[task],
|
||||
verbose=True,
|
||||
process=Process.sequential
|
||||
)
|
||||
|
||||
result = crew.kickoff()
|
||||
print(result)
|
||||
finally:
|
||||
print("Stopping Stdio MCP server connection (manual)...")
|
||||
mcp_server_adapter.stop() # **Crucial: Ensure stop is called**
|
||||
```
|
||||
|
||||
|
||||
#### SSE Transport Example (Manual)
|
||||
|
||||
```python
|
||||
from crewai_tools import MCPServerAdapter
|
||||
from crewai import Agent, Task, Crew, Process
|
||||
from mcp import StdioServerParameters
|
||||
|
||||
|
||||
server_params = {"url": "http://localhost:8000/sse"}
|
||||
|
||||
try:
|
||||
mcp_server_adapter = MCPServerAdapter(server_params)
|
||||
mcp_server_adapter.start()
|
||||
tools = mcp_server_adapter.tools
|
||||
print(f"Available tools (manual SSE): {[tool.name for tool in tools]}")
|
||||
|
||||
agent = Agent(
|
||||
role="Medical Researcher",
|
||||
goal="Find recent studies on a given topic using PubMed.",
|
||||
backstory="An AI assistant specialized in biomedical literature research.",
|
||||
tools=tools,
|
||||
verbose=True
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Search for recent articles on 'crispr gene editing'.",
|
||||
expected_output="A summary of the top 3 recent articles.",
|
||||
agent=agent
|
||||
)
|
||||
|
||||
crew = Crew(
|
||||
agents=[agent],
|
||||
tasks=[task],
|
||||
verbose=True,
|
||||
process=Process.sequential
|
||||
)
|
||||
|
||||
result = crew.kickoff()
|
||||
print(result)
|
||||
finally:
|
||||
print("Stopping SSE MCP server connection (manual)...")
|
||||
mcp_server_adapter.stop() # **Crucial: Ensure stop is called**
|
||||
```
|
||||
|
||||
## Staying Safe with MCP
|
||||
<Warning>
|
||||
Always ensure that you trust an MCP Server before using it.
|
||||
</Warning>
|
||||
|
||||
#### Security Warning: DNS Rebinding Attacks
|
||||
SSE transports can be vulnerable to DNS rebinding attacks if not properly secured.
|
||||
To prevent this:
|
||||
|
||||
1. **Always validate Origin headers** on incoming SSE connections to ensure they come from expected sources
|
||||
2. **Avoid binding servers to all network interfaces** (0.0.0.0) when running locally - bind only to localhost (127.0.0.1) instead
|
||||
3. **Implement proper authentication** for all SSE connections
|
||||
|
||||
Without these protections, attackers could use DNS rebinding to interact with local MCP servers from remote websites.
|
||||
|
||||
For more details, see the [MCP Transport Security](https://modelcontextprotocol.io/docs/concepts/transports#security-considerations) documentation.
|
||||
|
||||
### Limitations
|
||||
* **Supported Primitives**: Currently, `MCPServerAdapter` primarily supports adapting MCP `tools`.
|
||||
Other MCP primitives like `prompts` or `resources` are not directly integrated as CrewAI components through this adapter at this time.
|
||||
* **Output Handling**: The adapter typically processes the primary text output from an MCP tool (e.g., `.content[0].text`). Complex or multi-modal outputs might require custom handling if not fitting this pattern.
|
||||
244
docs/tools/stagehandtool.mdx
Normal file
@@ -0,0 +1,244 @@
|
||||
---
|
||||
title: Stagehand Tool
|
||||
description: Web automation tool that integrates Stagehand with CrewAI for browser interaction and automation
|
||||
icon: hand
|
||||
---
|
||||
|
||||
|
||||
# Overview
|
||||
|
||||
The `StagehandTool` integrates the [Stagehand](https://docs.stagehand.dev/get_started/introduction) framework with CrewAI, enabling agents to interact with websites and automate browser tasks using natural language instructions.
|
||||
|
||||
## Overview
|
||||
|
||||
Stagehand is a powerful browser automation framework built by Browserbase that allows AI agents to:
|
||||
|
||||
- Navigate to websites
|
||||
- Click buttons, links, and other elements
|
||||
- Fill in forms
|
||||
- Extract data from web pages
|
||||
- Observe and identify elements
|
||||
- Perform complex workflows
|
||||
|
||||
The StagehandTool wraps the Stagehand Python SDK to provide CrewAI agents with browser control capabilities through three core primitives:
|
||||
|
||||
1. **Act**: Perform actions like clicking, typing, or navigating
|
||||
2. **Extract**: Extract structured data from web pages
|
||||
3. **Observe**: Identify and analyze elements on the page
|
||||
|
||||
## Prerequisites
|
||||
|
||||
Before using this tool, ensure you have:
|
||||
|
||||
1. A [Browserbase](https://www.browserbase.com/) account with API key and project ID
|
||||
2. An API key for an LLM (OpenAI or Anthropic Claude)
|
||||
3. The Stagehand Python SDK installed
|
||||
|
||||
Install the required dependency:
|
||||
|
||||
```bash
|
||||
pip install stagehand-py
|
||||
```
|
||||
|
||||
## Usage
|
||||
|
||||
### Basic Implementation
|
||||
|
||||
The StagehandTool can be implemented in two ways:
|
||||
|
||||
#### 1. Using Context Manager (Recommended)
|
||||
<Tip>
|
||||
The context manager approach is recommended as it ensures proper cleanup of resources even if exceptions occur.
|
||||
</Tip>
|
||||
|
||||
```python
|
||||
from crewai import Agent, Task, Crew
|
||||
from crewai_tools import StagehandTool
|
||||
from stagehand.schemas import AvailableModel
|
||||
|
||||
# Initialize the tool with your API keys using a context manager
|
||||
with StagehandTool(
|
||||
api_key="your-browserbase-api-key",
|
||||
project_id="your-browserbase-project-id",
|
||||
model_api_key="your-llm-api-key", # OpenAI or Anthropic API key
|
||||
model_name=AvailableModel.CLAUDE_3_7_SONNET_LATEST, # Optional: specify which model to use
|
||||
) as stagehand_tool:
|
||||
# Create an agent with the tool
|
||||
researcher = Agent(
|
||||
role="Web Researcher",
|
||||
goal="Find and summarize information from websites",
|
||||
backstory="I'm an expert at finding information online.",
|
||||
verbose=True,
|
||||
tools=[stagehand_tool],
|
||||
)
|
||||
|
||||
# Create a task that uses the tool
|
||||
research_task = Task(
|
||||
description="Go to https://www.example.com and tell me what you see on the homepage.",
|
||||
agent=researcher,
|
||||
)
|
||||
|
||||
# Run the crew
|
||||
crew = Crew(
|
||||
agents=[researcher],
|
||||
tasks=[research_task],
|
||||
verbose=True,
|
||||
)
|
||||
|
||||
result = crew.kickoff()
|
||||
print(result)
|
||||
```
|
||||
|
||||
#### 2. Manual Resource Management
|
||||
|
||||
```python
|
||||
from crewai import Agent, Task, Crew
|
||||
from crewai_tools import StagehandTool
|
||||
from stagehand.schemas import AvailableModel
|
||||
|
||||
# Initialize the tool with your API keys
|
||||
stagehand_tool = StagehandTool(
|
||||
api_key="your-browserbase-api-key",
|
||||
project_id="your-browserbase-project-id",
|
||||
model_api_key="your-llm-api-key",
|
||||
model_name=AvailableModel.CLAUDE_3_7_SONNET_LATEST,
|
||||
)
|
||||
|
||||
try:
|
||||
# Create an agent with the tool
|
||||
researcher = Agent(
|
||||
role="Web Researcher",
|
||||
goal="Find and summarize information from websites",
|
||||
backstory="I'm an expert at finding information online.",
|
||||
verbose=True,
|
||||
tools=[stagehand_tool],
|
||||
)
|
||||
|
||||
# Create a task that uses the tool
|
||||
research_task = Task(
|
||||
description="Go to https://www.example.com and tell me what you see on the homepage.",
|
||||
agent=researcher,
|
||||
)
|
||||
|
||||
# Run the crew
|
||||
crew = Crew(
|
||||
agents=[researcher],
|
||||
tasks=[research_task],
|
||||
verbose=True,
|
||||
)
|
||||
|
||||
result = crew.kickoff()
|
||||
print(result)
|
||||
finally:
|
||||
# Explicitly clean up resources
|
||||
stagehand_tool.close()
|
||||
```
|
||||
|
||||
## Command Types
|
||||
|
||||
The StagehandTool supports three different command types for specific web automation tasks:
|
||||
|
||||
### 1. Act Command
|
||||
|
||||
The `act` command type (default) enables webpage interactions like clicking buttons, filling forms, and navigation.
|
||||
|
||||
```python
|
||||
# Perform an action (default behavior)
|
||||
result = stagehand_tool.run(
|
||||
instruction="Click the login button",
|
||||
url="https://example.com",
|
||||
command_type="act" # Default, so can be omitted
|
||||
)
|
||||
|
||||
# Fill out a form
|
||||
result = stagehand_tool.run(
|
||||
instruction="Fill the contact form with name 'John Doe', email 'john@example.com', and message 'Hello world'",
|
||||
url="https://example.com/contact"
|
||||
)
|
||||
```
|
||||
|
||||
### 2. Extract Command
|
||||
|
||||
The `extract` command type retrieves structured data from webpages.
|
||||
|
||||
```python
|
||||
# Extract all product information
|
||||
result = stagehand_tool.run(
|
||||
instruction="Extract all product names, prices, and descriptions",
|
||||
url="https://example.com/products",
|
||||
command_type="extract"
|
||||
)
|
||||
|
||||
# Extract specific information with a selector
|
||||
result = stagehand_tool.run(
|
||||
instruction="Extract the main article title and content",
|
||||
url="https://example.com/blog/article",
|
||||
command_type="extract",
|
||||
selector=".article-container" # Optional CSS selector
|
||||
)
|
||||
```
|
||||
|
||||
### 3. Observe Command
|
||||
|
||||
The `observe` command type identifies and analyzes webpage elements.
|
||||
|
||||
```python
|
||||
# Find interactive elements
|
||||
result = stagehand_tool.run(
|
||||
instruction="Find all interactive elements in the navigation menu",
|
||||
url="https://example.com",
|
||||
command_type="observe"
|
||||
)
|
||||
|
||||
# Identify form fields
|
||||
result = stagehand_tool.run(
|
||||
instruction="Identify all the input fields in the registration form",
|
||||
url="https://example.com/register",
|
||||
command_type="observe",
|
||||
selector="#registration-form"
|
||||
)
|
||||
```
|
||||
|
||||
## Configuration Options
|
||||
|
||||
Customize the StagehandTool behavior with these parameters:
|
||||
|
||||
```python
|
||||
stagehand_tool = StagehandTool(
|
||||
api_key="your-browserbase-api-key",
|
||||
project_id="your-browserbase-project-id",
|
||||
model_api_key="your-llm-api-key",
|
||||
model_name=AvailableModel.CLAUDE_3_7_SONNET_LATEST,
|
||||
dom_settle_timeout_ms=5000, # Wait longer for DOM to settle
|
||||
headless=True, # Run browser in headless mode
|
||||
self_heal=True, # Attempt to recover from errors
|
||||
wait_for_captcha_solves=True, # Wait for CAPTCHA solving
|
||||
verbose=1, # Control logging verbosity (0-3)
|
||||
)
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Be Specific**: Provide detailed instructions for better results
|
||||
2. **Choose Appropriate Command Type**: Select the right command type for your task
|
||||
3. **Use Selectors**: Leverage CSS selectors to improve accuracy
|
||||
4. **Break Down Complex Tasks**: Split complex workflows into multiple tool calls
|
||||
5. **Implement Error Handling**: Add error handling for potential issues
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
|
||||
Common issues and solutions:
|
||||
|
||||
- **Session Issues**: Verify API keys for both Browserbase and LLM provider
|
||||
- **Element Not Found**: Increase `dom_settle_timeout_ms` for slower pages
|
||||
- **Action Failures**: Use `observe` to identify correct elements first
|
||||
- **Incomplete Data**: Refine instructions or provide specific selectors
|
||||
|
||||
|
||||
## Additional Resources
|
||||
|
||||
For questions about the CrewAI integration:
|
||||
- Join Stagehand's [Slack community](https://stagehand.dev/slack)
|
||||
- Open an issue in the [Stagehand repository](https://github.com/browserbase/stagehand)
|
||||
- Visit [Stagehand documentation](https://docs.stagehand.dev/)
|
||||
@@ -1,6 +1,6 @@
|
||||
[project]
|
||||
name = "crewai"
|
||||
version = "0.119.0"
|
||||
version = "0.121.0"
|
||||
description = "Cutting-edge framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks."
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10,<3.13"
|
||||
@@ -45,7 +45,7 @@ Documentation = "https://docs.crewai.com"
|
||||
Repository = "https://github.com/crewAIInc/crewAI"
|
||||
|
||||
[project.optional-dependencies]
|
||||
tools = ["crewai-tools~=0.44.0"]
|
||||
tools = ["crewai-tools~=0.45.0"]
|
||||
embeddings = [
|
||||
"tiktoken~=0.7.0"
|
||||
]
|
||||
|
||||
@@ -17,7 +17,7 @@ warnings.filterwarnings(
|
||||
category=UserWarning,
|
||||
module="pydantic.main",
|
||||
)
|
||||
__version__ = "0.119.0"
|
||||
__version__ = "0.121.0"
|
||||
__all__ = [
|
||||
"Agent",
|
||||
"Crew",
|
||||
|
||||
@@ -115,10 +115,26 @@ class Agent(BaseAgent):
|
||||
default=False,
|
||||
description="Whether the agent is multimodal.",
|
||||
)
|
||||
inject_date: bool = Field(
|
||||
default=False,
|
||||
description="Whether to automatically inject the current date into tasks.",
|
||||
)
|
||||
date_format: str = Field(
|
||||
default="%Y-%m-%d",
|
||||
description="Format string for date when inject_date is enabled.",
|
||||
)
|
||||
code_execution_mode: Literal["safe", "unsafe"] = Field(
|
||||
default="safe",
|
||||
description="Mode for code execution: 'safe' (using Docker) or 'unsafe' (direct execution).",
|
||||
)
|
||||
reasoning: bool = Field(
|
||||
default=False,
|
||||
description="Whether the agent should reflect and create a plan before executing a task.",
|
||||
)
|
||||
max_reasoning_attempts: Optional[int] = Field(
|
||||
default=None,
|
||||
description="Maximum number of reasoning attempts before executing the task. If None, will try until ready.",
|
||||
)
|
||||
embedder: Optional[Dict[str, Any]] = Field(
|
||||
default=None,
|
||||
description="Embedder configuration for the agent.",
|
||||
@@ -225,6 +241,23 @@ class Agent(BaseAgent):
|
||||
ValueError: If the max execution time is not a positive integer.
|
||||
RuntimeError: If the agent execution fails for other reasons.
|
||||
"""
|
||||
if self.reasoning:
|
||||
try:
|
||||
from crewai.utilities.reasoning_handler import AgentReasoning, AgentReasoningOutput
|
||||
|
||||
reasoning_handler = AgentReasoning(task=task, agent=self)
|
||||
reasoning_output: AgentReasoningOutput = reasoning_handler.handle_agent_reasoning()
|
||||
|
||||
# Add the reasoning plan to the task description
|
||||
task.description += f"\n\nReasoning Plan:\n{reasoning_output.plan.plan}"
|
||||
except Exception as e:
|
||||
if hasattr(self, '_logger'):
|
||||
self._logger.log("error", f"Error during reasoning process: {str(e)}")
|
||||
else:
|
||||
print(f"Error during reasoning process: {str(e)}")
|
||||
|
||||
self._inject_date_to_task(task)
|
||||
|
||||
if self.tools_handler:
|
||||
self.tools_handler.last_used_tool = {} # type: ignore # Incompatible types in assignment (expression has type "dict[Never, Never]", variable has type "ToolCalling")
|
||||
|
||||
@@ -585,6 +618,26 @@ class Agent(BaseAgent):
|
||||
|
||||
return description
|
||||
|
||||
def _inject_date_to_task(self, task):
|
||||
"""Inject the current date into the task description if inject_date is enabled."""
|
||||
if self.inject_date:
|
||||
from datetime import datetime
|
||||
try:
|
||||
valid_format_codes = ['%Y', '%m', '%d', '%H', '%M', '%S', '%B', '%b', '%A', '%a']
|
||||
is_valid = any(code in self.date_format for code in valid_format_codes)
|
||||
|
||||
if not is_valid:
|
||||
raise ValueError(f"Invalid date format: {self.date_format}")
|
||||
|
||||
current_date: str = datetime.now().strftime(self.date_format)
|
||||
task.description += f"\n\nCurrent Date: {current_date}"
|
||||
except Exception as e:
|
||||
if hasattr(self, '_logger'):
|
||||
self._logger.log("warning", f"Failed to inject date: {str(e)}")
|
||||
else:
|
||||
print(f"Warning: Failed to inject date: {str(e)}")
|
||||
|
||||
|
||||
def _validate_docker_installation(self) -> None:
|
||||
"""Check if Docker is installed and running."""
|
||||
if not shutil.which("docker"):
|
||||
|
||||
@@ -1,6 +1,5 @@
|
||||
import os
|
||||
from importlib.metadata import version as get_version
|
||||
from typing import Optional, Tuple
|
||||
from typing import Optional
|
||||
|
||||
import click
|
||||
|
||||
@@ -138,12 +137,8 @@ def log_tasks_outputs() -> None:
|
||||
@click.option("-s", "--short", is_flag=True, help="Reset SHORT TERM memory")
|
||||
@click.option("-e", "--entities", is_flag=True, help="Reset ENTITIES memory")
|
||||
@click.option("-kn", "--knowledge", is_flag=True, help="Reset KNOWLEDGE storage")
|
||||
@click.option(
|
||||
"-k",
|
||||
"--kickoff-outputs",
|
||||
is_flag=True,
|
||||
help="Reset LATEST KICKOFF TASK OUTPUTS",
|
||||
)
|
||||
@click.option("-akn", "--agent-knowledge", is_flag=True, help="Reset AGENT KNOWLEDGE storage")
|
||||
@click.option("-k","--kickoff-outputs",is_flag=True,help="Reset LATEST KICKOFF TASK OUTPUTS")
|
||||
@click.option("-a", "--all", is_flag=True, help="Reset ALL memories")
|
||||
def reset_memories(
|
||||
long: bool,
|
||||
@@ -151,18 +146,20 @@ def reset_memories(
|
||||
entities: bool,
|
||||
knowledge: bool,
|
||||
kickoff_outputs: bool,
|
||||
agent_knowledge: bool,
|
||||
all: bool,
|
||||
) -> None:
|
||||
"""
|
||||
Reset the crew memories (long, short, entity, latest_crew_kickoff_ouputs). This will delete all the data saved.
|
||||
Reset the crew memories (long, short, entity, latest_crew_kickoff_ouputs, knowledge, agent_knowledge). This will delete all the data saved.
|
||||
"""
|
||||
try:
|
||||
if not all and not (long or short or entities or knowledge or kickoff_outputs):
|
||||
memory_types = [long, short, entities, knowledge, agent_knowledge, kickoff_outputs, all]
|
||||
if not any(memory_types):
|
||||
click.echo(
|
||||
"Please specify at least one memory type to reset using the appropriate flags."
|
||||
)
|
||||
return
|
||||
reset_memories_command(long, short, entities, knowledge, kickoff_outputs, all)
|
||||
reset_memories_command(long, short, entities, knowledge, agent_knowledge, kickoff_outputs, all)
|
||||
except Exception as e:
|
||||
click.echo(f"An error occurred while resetting memories: {e}", err=True)
|
||||
|
||||
|
||||
@@ -10,6 +10,7 @@ def reset_memories_command(
|
||||
short,
|
||||
entity,
|
||||
knowledge,
|
||||
agent_knowledge,
|
||||
kickoff_outputs,
|
||||
all,
|
||||
) -> None:
|
||||
@@ -23,10 +24,11 @@ def reset_memories_command(
|
||||
kickoff_outputs (bool): Whether to reset the latest kickoff task outputs.
|
||||
all (bool): Whether to reset all memories.
|
||||
knowledge (bool): Whether to reset the knowledge.
|
||||
agent_knowledge (bool): Whether to reset the agents knowledge.
|
||||
"""
|
||||
|
||||
try:
|
||||
if not any([long, short, entity, kickoff_outputs, knowledge, all]):
|
||||
if not any([long, short, entity, kickoff_outputs, knowledge, agent_knowledge, all]):
|
||||
click.echo(
|
||||
"No memory type specified. Please specify at least one type to reset."
|
||||
)
|
||||
@@ -67,6 +69,11 @@ def reset_memories_command(
|
||||
click.echo(
|
||||
f"[Crew ({crew.name if crew.name else crew.id})] Knowledge has been reset."
|
||||
)
|
||||
if agent_knowledge:
|
||||
crew.reset_memories(command_type="agent_knowledge")
|
||||
click.echo(
|
||||
f"[Crew ({crew.name if crew.name else crew.id})] Agents knowledge has been reset."
|
||||
)
|
||||
|
||||
except subprocess.CalledProcessError as e:
|
||||
click.echo(f"An error occurred while resetting the memories: {e}", err=True)
|
||||
|
||||
@@ -5,7 +5,7 @@ description = "{{name}} using crewAI"
|
||||
authors = [{ name = "Your Name", email = "you@example.com" }]
|
||||
requires-python = ">=3.10,<3.13"
|
||||
dependencies = [
|
||||
"crewai[tools]>=0.119.0,<1.0.0"
|
||||
"crewai[tools]>=0.121.0,<1.0.0"
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
|
||||
@@ -5,7 +5,7 @@ description = "{{name}} using crewAI"
|
||||
authors = [{ name = "Your Name", email = "you@example.com" }]
|
||||
requires-python = ">=3.10,<3.13"
|
||||
dependencies = [
|
||||
"crewai[tools]>=0.119.0,<1.0.0",
|
||||
"crewai[tools]>=0.121.0,<1.0.0",
|
||||
]
|
||||
|
||||
[project.scripts]
|
||||
|
||||
@@ -5,7 +5,7 @@ description = "Power up your crews with {{folder_name}}"
|
||||
readme = "README.md"
|
||||
requires-python = ">=3.10,<3.13"
|
||||
dependencies = [
|
||||
"crewai[tools]>=0.119.0"
|
||||
"crewai[tools]>=0.121.0"
|
||||
]
|
||||
|
||||
[tool.crewai]
|
||||
|
||||
@@ -161,7 +161,7 @@ def tree_find_and_replace(directory, find, replace):
|
||||
for filename in files:
|
||||
filepath = os.path.join(path, filename)
|
||||
|
||||
with open(filepath, "r") as file:
|
||||
with open(filepath, "r", encoding="utf-8", errors="ignore") as file:
|
||||
contents = file.read()
|
||||
with open(filepath, "w") as file:
|
||||
file.write(contents.replace(find, replace))
|
||||
|
||||
@@ -52,7 +52,7 @@ from crewai.tools.agent_tools.agent_tools import AgentTools
|
||||
from crewai.tools.base_tool import BaseTool, Tool
|
||||
from crewai.types.usage_metrics import UsageMetrics
|
||||
from crewai.utilities import I18N, FileHandler, Logger, RPMController
|
||||
from crewai.utilities.constants import TRAINING_DATA_FILE
|
||||
from crewai.utilities.constants import NOT_SPECIFIED, TRAINING_DATA_FILE
|
||||
from crewai.utilities.evaluators.crew_evaluator_handler import CrewEvaluator
|
||||
from crewai.utilities.evaluators.task_evaluator import TaskEvaluator
|
||||
from crewai.utilities.events.crew_events import (
|
||||
@@ -315,9 +315,7 @@ class Crew(FlowTrackable, BaseModel):
|
||||
"""Initialize private memory attributes."""
|
||||
self._external_memory = (
|
||||
# External memory doesn’t support a default value since it was designed to be managed entirely externally
|
||||
self.external_memory.set_crew(self)
|
||||
if self.external_memory
|
||||
else None
|
||||
self.external_memory.set_crew(self) if self.external_memory else None
|
||||
)
|
||||
|
||||
self._long_term_memory = self.long_term_memory
|
||||
@@ -478,7 +476,7 @@ class Crew(FlowTrackable, BaseModel):
|
||||
separated by a synchronous task.
|
||||
"""
|
||||
for i, task in enumerate(self.tasks):
|
||||
if task.async_execution and task.context:
|
||||
if task.async_execution and isinstance(task.context, list):
|
||||
for context_task in task.context:
|
||||
if context_task.async_execution:
|
||||
for j in range(i - 1, -1, -1):
|
||||
@@ -496,7 +494,7 @@ class Crew(FlowTrackable, BaseModel):
|
||||
task_indices = {id(task): i for i, task in enumerate(self.tasks)}
|
||||
|
||||
for task in self.tasks:
|
||||
if task.context:
|
||||
if isinstance(task.context, list):
|
||||
for context_task in task.context:
|
||||
if id(context_task) not in task_indices:
|
||||
continue # Skip context tasks not in the main tasks list
|
||||
@@ -1034,11 +1032,14 @@ class Crew(FlowTrackable, BaseModel):
|
||||
)
|
||||
return cast(List[BaseTool], tools)
|
||||
|
||||
def _get_context(self, task: Task, task_outputs: List[TaskOutput]):
|
||||
def _get_context(self, task: Task, task_outputs: List[TaskOutput]) -> str:
|
||||
if not task.context:
|
||||
return ""
|
||||
|
||||
context = (
|
||||
aggregate_raw_outputs_from_tasks(task.context)
|
||||
if task.context
|
||||
else aggregate_raw_outputs_from_task_outputs(task_outputs)
|
||||
aggregate_raw_outputs_from_task_outputs(task_outputs)
|
||||
if task.context is NOT_SPECIFIED
|
||||
else aggregate_raw_outputs_from_tasks(task.context)
|
||||
)
|
||||
return context
|
||||
|
||||
@@ -1201,7 +1202,6 @@ class Crew(FlowTrackable, BaseModel):
|
||||
"_long_term_memory",
|
||||
"_entity_memory",
|
||||
"_external_memory",
|
||||
"_telemetry",
|
||||
"agents",
|
||||
"tasks",
|
||||
"knowledge_sources",
|
||||
@@ -1226,7 +1226,7 @@ class Crew(FlowTrackable, BaseModel):
|
||||
task_mapping[task.key] = cloned_task
|
||||
|
||||
for cloned_task, original_task in zip(cloned_tasks, self.tasks):
|
||||
if original_task.context:
|
||||
if isinstance(original_task.context, list):
|
||||
cloned_context = [
|
||||
task_mapping[context_task.key]
|
||||
for context_task in original_task.context
|
||||
@@ -1353,7 +1353,7 @@ class Crew(FlowTrackable, BaseModel):
|
||||
|
||||
Args:
|
||||
command_type: Type of memory to reset.
|
||||
Valid options: 'long', 'short', 'entity', 'knowledge',
|
||||
Valid options: 'long', 'short', 'entity', 'knowledge', 'agent_knowledge'
|
||||
'kickoff_outputs', or 'all'
|
||||
|
||||
Raises:
|
||||
@@ -1366,6 +1366,7 @@ class Crew(FlowTrackable, BaseModel):
|
||||
"short",
|
||||
"entity",
|
||||
"knowledge",
|
||||
"agent_knowledge",
|
||||
"kickoff_outputs",
|
||||
"all",
|
||||
"external",
|
||||
@@ -1390,19 +1391,14 @@ class Crew(FlowTrackable, BaseModel):
|
||||
|
||||
def _reset_all_memories(self) -> None:
|
||||
"""Reset all available memory systems."""
|
||||
memory_systems = [
|
||||
("short term", getattr(self, "_short_term_memory", None)),
|
||||
("entity", getattr(self, "_entity_memory", None)),
|
||||
("external", getattr(self, "_external_memory", None)),
|
||||
("long term", getattr(self, "_long_term_memory", None)),
|
||||
("task output", getattr(self, "_task_output_handler", None)),
|
||||
("knowledge", getattr(self, "knowledge", None)),
|
||||
]
|
||||
memory_systems = self._get_memory_systems()
|
||||
|
||||
for name, system in memory_systems:
|
||||
if system is not None:
|
||||
for memory_type, config in memory_systems.items():
|
||||
if (system := config.get("system")) is not None:
|
||||
name = config.get("name")
|
||||
try:
|
||||
system.reset()
|
||||
reset_fn: Callable = cast(Callable, config.get("reset"))
|
||||
reset_fn(system)
|
||||
self._logger.log(
|
||||
"info",
|
||||
f"[Crew ({self.name if self.name else self.id})] {name} memory has been reset",
|
||||
@@ -1421,24 +1417,17 @@ class Crew(FlowTrackable, BaseModel):
|
||||
Raises:
|
||||
RuntimeError: If the specified memory system fails to reset
|
||||
"""
|
||||
reset_functions = {
|
||||
"long": (getattr(self, "_long_term_memory", None), "long term"),
|
||||
"short": (getattr(self, "_short_term_memory", None), "short term"),
|
||||
"entity": (getattr(self, "_entity_memory", None), "entity"),
|
||||
"knowledge": (getattr(self, "knowledge", None), "knowledge"),
|
||||
"kickoff_outputs": (
|
||||
getattr(self, "_task_output_handler", None),
|
||||
"task output",
|
||||
),
|
||||
"external": (getattr(self, "_external_memory", None), "external"),
|
||||
}
|
||||
memory_systems = self._get_memory_systems()
|
||||
config = memory_systems[memory_type]
|
||||
system = config.get("system")
|
||||
name = config.get("name")
|
||||
|
||||
memory_system, name = reset_functions[memory_type]
|
||||
if memory_system is None:
|
||||
if system is None:
|
||||
raise RuntimeError(f"{name} memory system is not initialized")
|
||||
|
||||
try:
|
||||
memory_system.reset()
|
||||
reset_fn: Callable = cast(Callable, config.get("reset"))
|
||||
reset_fn(system)
|
||||
self._logger.log(
|
||||
"info",
|
||||
f"[Crew ({self.name if self.name else self.id})] {name} memory has been reset",
|
||||
@@ -1447,3 +1436,73 @@ class Crew(FlowTrackable, BaseModel):
|
||||
raise RuntimeError(
|
||||
f"[Crew ({self.name if self.name else self.id})] Failed to reset {name} memory: {str(e)}"
|
||||
) from e
|
||||
|
||||
def _get_memory_systems(self):
|
||||
"""Get all available memory systems with their configuration.
|
||||
|
||||
Returns:
|
||||
Dict containing all memory systems with their reset functions and display names.
|
||||
"""
|
||||
|
||||
def default_reset(memory):
|
||||
return memory.reset()
|
||||
|
||||
def knowledge_reset(memory):
|
||||
return self.reset_knowledge(memory)
|
||||
|
||||
# Get knowledge for agents
|
||||
agent_knowledges = [
|
||||
getattr(agent, "knowledge", None)
|
||||
for agent in self.agents
|
||||
if getattr(agent, "knowledge", None) is not None
|
||||
]
|
||||
# Get knowledge for crew and agents
|
||||
crew_knowledge = getattr(self, "knowledge", None)
|
||||
crew_and_agent_knowledges = (
|
||||
[crew_knowledge] if crew_knowledge is not None else []
|
||||
) + agent_knowledges
|
||||
|
||||
return {
|
||||
"short": {
|
||||
"system": getattr(self, "_short_term_memory", None),
|
||||
"reset": default_reset,
|
||||
"name": "Short Term",
|
||||
},
|
||||
"entity": {
|
||||
"system": getattr(self, "_entity_memory", None),
|
||||
"reset": default_reset,
|
||||
"name": "Entity",
|
||||
},
|
||||
"external": {
|
||||
"system": getattr(self, "_external_memory", None),
|
||||
"reset": default_reset,
|
||||
"name": "External",
|
||||
},
|
||||
"long": {
|
||||
"system": getattr(self, "_long_term_memory", None),
|
||||
"reset": default_reset,
|
||||
"name": "Long Term",
|
||||
},
|
||||
"kickoff_outputs": {
|
||||
"system": getattr(self, "_task_output_handler", None),
|
||||
"reset": default_reset,
|
||||
"name": "Task Output",
|
||||
},
|
||||
"knowledge": {
|
||||
"system": crew_and_agent_knowledges
|
||||
if crew_and_agent_knowledges
|
||||
else None,
|
||||
"reset": knowledge_reset,
|
||||
"name": "Crew Knowledge and Agent Knowledge",
|
||||
},
|
||||
"agent_knowledge": {
|
||||
"system": agent_knowledges if agent_knowledges else None,
|
||||
"reset": knowledge_reset,
|
||||
"name": "Agent Knowledge",
|
||||
},
|
||||
}
|
||||
|
||||
def reset_knowledge(self, knowledges: List[Knowledge]) -> None:
|
||||
"""Reset crew and agent knowledge storage."""
|
||||
for ks in knowledges:
|
||||
ks.reset()
|
||||
|
||||
@@ -2,7 +2,6 @@ import datetime
|
||||
import inspect
|
||||
import json
|
||||
import logging
|
||||
import re
|
||||
import threading
|
||||
import uuid
|
||||
from concurrent.futures import Future
|
||||
@@ -41,6 +40,7 @@ from crewai.tasks.output_format import OutputFormat
|
||||
from crewai.tasks.task_output import TaskOutput
|
||||
from crewai.tools.base_tool import BaseTool
|
||||
from crewai.utilities.config import process_config
|
||||
from crewai.utilities.constants import NOT_SPECIFIED
|
||||
from crewai.utilities.converter import Converter, convert_to_model
|
||||
from crewai.utilities.events import (
|
||||
TaskCompletedEvent,
|
||||
@@ -97,7 +97,7 @@ class Task(BaseModel):
|
||||
)
|
||||
context: Optional[List["Task"]] = Field(
|
||||
description="Other tasks that will have their output used as context for this task.",
|
||||
default=None,
|
||||
default=NOT_SPECIFIED,
|
||||
)
|
||||
async_execution: Optional[bool] = Field(
|
||||
description="Whether the task should be executed asynchronously or not.",
|
||||
@@ -135,6 +135,10 @@ class Task(BaseModel):
|
||||
description="Whether the task should have a human review the final answer of the agent",
|
||||
default=False,
|
||||
)
|
||||
markdown: Optional[bool] = Field(
|
||||
description="Whether the task should instruct the agent to return the final answer formatted in Markdown",
|
||||
default=False,
|
||||
)
|
||||
converter_cls: Optional[Type[Converter]] = Field(
|
||||
description="A converter class used to export structured output",
|
||||
default=None,
|
||||
@@ -522,10 +526,14 @@ class Task(BaseModel):
|
||||
return guardrail_result
|
||||
|
||||
def prompt(self) -> str:
|
||||
"""Prompt the task.
|
||||
|
||||
"""Generates the task prompt with optional markdown formatting.
|
||||
|
||||
When the markdown attribute is True, instructions for formatting the
|
||||
response in Markdown syntax will be added to the prompt.
|
||||
|
||||
Returns:
|
||||
Prompt of the task.
|
||||
str: The formatted prompt string containing the task description,
|
||||
expected output, and optional markdown formatting instructions.
|
||||
"""
|
||||
tasks_slices = [self.description]
|
||||
|
||||
@@ -533,6 +541,17 @@ class Task(BaseModel):
|
||||
expected_output=self.expected_output
|
||||
)
|
||||
tasks_slices = [self.description, output]
|
||||
|
||||
if self.markdown:
|
||||
markdown_instruction = """Your final answer MUST be formatted in Markdown syntax.
|
||||
Follow these guidelines:
|
||||
- Use # for headers
|
||||
- Use ** for bold text
|
||||
- Use * for italic text
|
||||
- Use - or * for bullet points
|
||||
- Use `code` for inline code
|
||||
- Use ```language for code blocks"""
|
||||
tasks_slices.append(markdown_instruction)
|
||||
return "\n".join(tasks_slices)
|
||||
|
||||
def interpolate_inputs_and_add_conversation_history(
|
||||
@@ -643,7 +662,7 @@ class Task(BaseModel):
|
||||
|
||||
cloned_context = (
|
||||
[task_mapping[context_task.key] for context_task in self.context]
|
||||
if self.context
|
||||
if isinstance(self.context, list)
|
||||
else None
|
||||
)
|
||||
|
||||
|
||||
96
src/crewai/tasks/hallucination_guardrail.py
Normal file
@@ -0,0 +1,96 @@
|
||||
"""Hallucination Guardrail Placeholder for CrewAI.
|
||||
|
||||
This is a no-op version of the HallucinationGuardrail for the open-source repository.
|
||||
|
||||
Classes:
|
||||
HallucinationGuardrail: Placeholder guardrail that validates task outputs.
|
||||
"""
|
||||
|
||||
from typing import Any, Optional, Tuple
|
||||
|
||||
from crewai.llm import LLM
|
||||
from crewai.tasks.task_output import TaskOutput
|
||||
from crewai.utilities.logger import Logger
|
||||
|
||||
|
||||
class HallucinationGuardrail:
|
||||
"""Placeholder for the HallucinationGuardrail feature.
|
||||
|
||||
Attributes:
|
||||
context: The reference context that outputs would be checked against.
|
||||
llm: The language model that would be used for evaluation.
|
||||
threshold: Optional minimum faithfulness score that would be required to pass.
|
||||
tool_response: Optional tool response information that would be used in evaluation.
|
||||
|
||||
Examples:
|
||||
>>> # Basic usage with default verdict logic
|
||||
>>> guardrail = HallucinationGuardrail(
|
||||
... context="AI helps with various tasks including analysis and generation.",
|
||||
... llm=agent.llm
|
||||
... )
|
||||
|
||||
>>> # With custom threshold for stricter validation
|
||||
>>> strict_guardrail = HallucinationGuardrail(
|
||||
... context="Quantum computing uses qubits in superposition.",
|
||||
... llm=agent.llm,
|
||||
... threshold=8.0 # Would require score >= 8 to pass in enterprise version
|
||||
... )
|
||||
|
||||
>>> # With tool response for additional context
|
||||
>>> guardrail_with_tools = HallucinationGuardrail(
|
||||
... context="The current weather data",
|
||||
... llm=agent.llm,
|
||||
... tool_response="Weather API returned: Temperature 22°C, Humidity 65%"
|
||||
... )
|
||||
"""
|
||||
|
||||
def __init__(
|
||||
self,
|
||||
context: str,
|
||||
llm: LLM,
|
||||
threshold: Optional[float] = None,
|
||||
tool_response: str = "",
|
||||
):
|
||||
"""Initialize the HallucinationGuardrail placeholder.
|
||||
|
||||
Args:
|
||||
context: The reference context that outputs would be checked against.
|
||||
llm: The language model that would be used for evaluation.
|
||||
threshold: Optional minimum faithfulness score that would be required to pass.
|
||||
tool_response: Optional tool response information that would be used in evaluation.
|
||||
"""
|
||||
self.context = context
|
||||
self.llm: LLM = llm
|
||||
self.threshold = threshold
|
||||
self.tool_response = tool_response
|
||||
self._logger = Logger(verbose=True)
|
||||
self._logger.log(
|
||||
"warning",
|
||||
"""Hallucination detection is a no-op in open source, use it for free at https://app.crewai.com\n""",
|
||||
color="red",
|
||||
)
|
||||
|
||||
@property
|
||||
def description(self) -> str:
|
||||
"""Generate a description of this guardrail for event logging."""
|
||||
return "HallucinationGuardrail (no-op)"
|
||||
|
||||
def __call__(self, task_output: TaskOutput) -> Tuple[bool, Any]:
|
||||
"""Validate a task output against hallucination criteria.
|
||||
|
||||
In the open source, this method always returns that the output is valid.
|
||||
|
||||
Args:
|
||||
task_output: The output to be validated.
|
||||
|
||||
Returns:
|
||||
A tuple containing:
|
||||
- True
|
||||
- The raw task output
|
||||
"""
|
||||
self._logger.log(
|
||||
"warning",
|
||||
"Premium hallucination detection skipped (use for free at https://app.crewai.com)\n",
|
||||
color="red",
|
||||
)
|
||||
return True, task_output.raw
|
||||
@@ -9,6 +9,19 @@ import warnings
|
||||
from contextlib import contextmanager
|
||||
from importlib.metadata import version
|
||||
from typing import TYPE_CHECKING, Any, Optional
|
||||
import threading
|
||||
|
||||
from opentelemetry import trace
|
||||
from opentelemetry.exporter.otlp.proto.http.trace_exporter import (
|
||||
OTLPSpanExporter,
|
||||
)
|
||||
from opentelemetry.sdk.resources import SERVICE_NAME, Resource
|
||||
from opentelemetry.sdk.trace import TracerProvider
|
||||
from opentelemetry.sdk.trace.export import (
|
||||
BatchSpanProcessor,
|
||||
SpanExportResult,
|
||||
)
|
||||
from opentelemetry.trace import Span, Status, StatusCode
|
||||
|
||||
from crewai.telemetry.constants import (
|
||||
CREWAI_TELEMETRY_BASE_URL,
|
||||
@@ -25,18 +38,6 @@ def suppress_warnings():
|
||||
yield
|
||||
|
||||
|
||||
from opentelemetry import trace # noqa: E402
|
||||
from opentelemetry.exporter.otlp.proto.http.trace_exporter import (
|
||||
OTLPSpanExporter, # noqa: E402
|
||||
)
|
||||
from opentelemetry.sdk.resources import SERVICE_NAME, Resource # noqa: E402
|
||||
from opentelemetry.sdk.trace import TracerProvider # noqa: E402
|
||||
from opentelemetry.sdk.trace.export import ( # noqa: E402
|
||||
BatchSpanProcessor,
|
||||
SpanExportResult,
|
||||
)
|
||||
from opentelemetry.trace import Span, Status, StatusCode # noqa: E402
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from crewai.crew import Crew
|
||||
from crewai.task import Task
|
||||
@@ -64,7 +65,17 @@ class Telemetry:
|
||||
attribute in the Crew class.
|
||||
"""
|
||||
|
||||
def __init__(self):
|
||||
_instance = None
|
||||
_lock = threading.Lock()
|
||||
|
||||
def __new__(cls):
|
||||
if cls._instance is None:
|
||||
with cls._lock:
|
||||
if cls._instance is None:
|
||||
cls._instance = super(Telemetry, cls).__new__(cls)
|
||||
return cls._instance
|
||||
|
||||
def __init__(self) -> None:
|
||||
self.ready: bool = False
|
||||
self.trace_set: bool = False
|
||||
|
||||
@@ -232,7 +243,7 @@ class Telemetry:
|
||||
"agent_key": task.agent.key if task.agent else None,
|
||||
"context": (
|
||||
[task.description for task in task.context]
|
||||
if task.context
|
||||
if isinstance(task.context, list)
|
||||
else None
|
||||
),
|
||||
"tools_names": [
|
||||
@@ -748,7 +759,7 @@ class Telemetry:
|
||||
"agent_key": task.agent.key if task.agent else None,
|
||||
"context": (
|
||||
[task.description for task in task.context]
|
||||
if task.context
|
||||
if isinstance(task.context, list)
|
||||
else None
|
||||
),
|
||||
"tools_names": [
|
||||
|
||||
@@ -51,5 +51,11 @@
|
||||
"description": "See image to understand its content, you can optionally ask a question about the image",
|
||||
"default_action": "Please provide a detailed description of this image, including all visual elements, context, and any notable details you can observe."
|
||||
}
|
||||
},
|
||||
"reasoning": {
|
||||
"initial_plan": "You are {role}, a professional with the following background: {backstory}\n\nYour primary goal is: {goal}\n\nAs {role}, you are creating a strategic plan for a task that requires your expertise and unique perspective.",
|
||||
"refine_plan": "You are {role}, a professional with the following background: {backstory}\n\nYour primary goal is: {goal}\n\nAs {role}, you are refining a strategic plan for a task that requires your expertise and unique perspective.",
|
||||
"create_plan_prompt": "You are {role} with this background: {backstory}\n\nYour primary goal is: {goal}\n\nYou have been assigned the following task:\n{description}\n\nExpected output:\n{expected_output}\n\nAvailable tools: {tools}\n\nBefore executing this task, create a detailed plan that leverages your expertise as {role} and outlines:\n1. Your understanding of the task from your professional perspective\n2. The key steps you'll take to complete it, drawing on your background and skills\n3. How you'll approach any challenges that might arise, considering your expertise\n4. How you'll strategically use the available tools based on your experience\n5. The expected outcome and how it aligns with your goal\n\nAfter creating your plan, assess whether you feel ready to execute the task.\nConclude with one of these statements:\n- \"READY: I am ready to execute the task.\"\n- \"NOT READY: I need to refine my plan because [specific reason].\"",
|
||||
"refine_plan_prompt": "You are {role} with this background: {backstory}\n\nYour primary goal is: {goal}\n\nYou created the following plan for this task:\n{current_plan}\n\nHowever, you indicated that you're not ready to execute the task yet.\n\nPlease refine your plan further, drawing on your expertise as {role} to address any gaps or uncertainties.\n\nAfter refining your plan, assess whether you feel ready to execute the task.\nConclude with one of these statements:\n- \"READY: I am ready to execute the task.\"\n- \"NOT READY: I need to refine my plan further because [specific reason].\""
|
||||
}
|
||||
}
|
||||
|
||||
@@ -441,6 +441,11 @@ def load_agent_from_repository(from_repository: str) -> Dict[str, Any]:
|
||||
|
||||
client = PlusAPI(api_key=get_auth_token())
|
||||
response = client.get_agent(from_repository)
|
||||
if response.status_code == 404:
|
||||
raise AgentRepositoryError(
|
||||
f"Agent {from_repository} does not exist, make sure the name is correct or the agent is available on your organization"
|
||||
)
|
||||
|
||||
if response.status_code != 200:
|
||||
raise AgentRepositoryError(
|
||||
f"Agent {from_repository} could not be loaded: {response.text}"
|
||||
@@ -450,14 +455,14 @@ def load_agent_from_repository(from_repository: str) -> Dict[str, Any]:
|
||||
for key, value in agent.items():
|
||||
if key == "tools":
|
||||
attributes[key] = []
|
||||
for tool_name in value:
|
||||
for tool in value:
|
||||
try:
|
||||
module = importlib.import_module("crewai_tools")
|
||||
tool_class = getattr(module, tool_name)
|
||||
tool_class = getattr(module, tool["name"])
|
||||
attributes[key].append(tool_class())
|
||||
except Exception as e:
|
||||
raise AgentRepositoryError(
|
||||
f"Tool {tool_name} could not be loaded: {e}"
|
||||
f"Tool {tool['name']} could not be loaded: {e}"
|
||||
) from e
|
||||
else:
|
||||
attributes[key] = value
|
||||
|
||||
@@ -5,3 +5,14 @@ KNOWLEDGE_DIRECTORY = "knowledge"
|
||||
MAX_LLM_RETRY = 3
|
||||
MAX_FILE_NAME_LENGTH = 255
|
||||
EMITTER_COLOR = "bold_blue"
|
||||
|
||||
|
||||
class _NotSpecified:
|
||||
def __repr__(self):
|
||||
return "NOT_SPECIFIED"
|
||||
|
||||
|
||||
# Sentinel value used to detect when no value has been explicitly provided.
|
||||
# Unlike `None`, which might be a valid value from the user, `NOT_SPECIFIED` allows
|
||||
# us to distinguish between "not passed at all" and "explicitly passed None" or "[]".
|
||||
NOT_SPECIFIED = _NotSpecified()
|
||||
|
||||
@@ -9,7 +9,8 @@ from crewai.agent import Agent
|
||||
from crewai.llm import BaseLLM
|
||||
from crewai.task import Task
|
||||
from crewai.tasks.task_output import TaskOutput
|
||||
from crewai.telemetry import Telemetry
|
||||
from crewai.utilities.events import crewai_event_bus
|
||||
from crewai.utilities.events.crew_events import CrewTestResultEvent
|
||||
|
||||
|
||||
class TaskEvaluationPydanticOutput(BaseModel):
|
||||
@@ -36,7 +37,6 @@ class CrewEvaluator:
|
||||
def __init__(self, crew, eval_llm: InstanceOf[BaseLLM]):
|
||||
self.crew = crew
|
||||
self.llm = eval_llm
|
||||
self._telemetry = Telemetry()
|
||||
self._setup_for_evaluating()
|
||||
|
||||
def _setup_for_evaluating(self) -> None:
|
||||
@@ -178,11 +178,15 @@ class CrewEvaluator:
|
||||
evaluation_result = evaluation_task.execute_sync()
|
||||
|
||||
if isinstance(evaluation_result.pydantic, TaskEvaluationPydanticOutput):
|
||||
self._test_result_span = self._telemetry.individual_test_result_span(
|
||||
crewai_event_bus.emit(
|
||||
self.crew,
|
||||
evaluation_result.pydantic.quality,
|
||||
current_task.execution_duration,
|
||||
self.llm.model,
|
||||
CrewTestResultEvent(
|
||||
quality=evaluation_result.pydantic.quality,
|
||||
execution_duration=current_task.execution_duration,
|
||||
model=self.llm.model,
|
||||
crew_name=self.crew.name,
|
||||
crew=self.crew,
|
||||
),
|
||||
)
|
||||
self.tasks_scores[self.iteration].append(evaluation_result.pydantic.quality)
|
||||
self.run_execution_times[self.iteration].append(
|
||||
|
||||
@@ -100,3 +100,12 @@ class CrewTestFailedEvent(CrewBaseEvent):
|
||||
|
||||
error: str
|
||||
type: str = "crew_test_failed"
|
||||
|
||||
|
||||
class CrewTestResultEvent(CrewBaseEvent):
|
||||
"""Event emitted when a crew test result is available"""
|
||||
|
||||
quality: float
|
||||
execution_duration: float
|
||||
model: str
|
||||
type: str = "crew_test_result"
|
||||
|
||||
@@ -37,6 +37,7 @@ from .crew_events import (
|
||||
CrewKickoffStartedEvent,
|
||||
CrewTestCompletedEvent,
|
||||
CrewTestFailedEvent,
|
||||
CrewTestResultEvent,
|
||||
CrewTestStartedEvent,
|
||||
CrewTrainCompletedEvent,
|
||||
CrewTrainFailedEvent,
|
||||
@@ -56,6 +57,11 @@ from .tool_usage_events import (
|
||||
ToolUsageFinishedEvent,
|
||||
ToolUsageStartedEvent,
|
||||
)
|
||||
from .reasoning_events import (
|
||||
AgentReasoningStartedEvent,
|
||||
AgentReasoningCompletedEvent,
|
||||
AgentReasoningFailedEvent,
|
||||
)
|
||||
|
||||
|
||||
class EventListener(BaseEventListener):
|
||||
@@ -129,6 +135,15 @@ class EventListener(BaseEventListener):
|
||||
def on_crew_train_failed(source, event: CrewTrainFailedEvent):
|
||||
self.formatter.handle_crew_train_failed(event.crew_name or "Crew")
|
||||
|
||||
@crewai_event_bus.on(CrewTestResultEvent)
|
||||
def on_crew_test_result(source, event: CrewTestResultEvent):
|
||||
self._telemetry.individual_test_result_span(
|
||||
source.crew,
|
||||
event.quality,
|
||||
int(event.execution_duration),
|
||||
event.model,
|
||||
)
|
||||
|
||||
# ----------- TASK EVENTS -----------
|
||||
|
||||
@crewai_event_bus.on(TaskStartedEvent)
|
||||
@@ -406,5 +421,30 @@ class EventListener(BaseEventListener):
|
||||
self.formatter.current_crew_tree,
|
||||
)
|
||||
|
||||
# ----------- REASONING EVENTS -----------
|
||||
|
||||
@crewai_event_bus.on(AgentReasoningStartedEvent)
|
||||
def on_agent_reasoning_started(source, event: AgentReasoningStartedEvent):
|
||||
self.formatter.handle_reasoning_started(
|
||||
self.formatter.current_agent_branch,
|
||||
event.attempt,
|
||||
self.formatter.current_crew_tree,
|
||||
)
|
||||
|
||||
@crewai_event_bus.on(AgentReasoningCompletedEvent)
|
||||
def on_agent_reasoning_completed(source, event: AgentReasoningCompletedEvent):
|
||||
self.formatter.handle_reasoning_completed(
|
||||
event.plan,
|
||||
event.ready,
|
||||
self.formatter.current_crew_tree,
|
||||
)
|
||||
|
||||
@crewai_event_bus.on(AgentReasoningFailedEvent)
|
||||
def on_agent_reasoning_failed(source, event: AgentReasoningFailedEvent):
|
||||
self.formatter.handle_reasoning_failed(
|
||||
event.error,
|
||||
self.formatter.current_crew_tree,
|
||||
)
|
||||
|
||||
|
||||
event_listener = EventListener()
|
||||
|
||||
@@ -43,6 +43,19 @@ from .tool_usage_events import (
|
||||
ToolUsageFinishedEvent,
|
||||
ToolUsageStartedEvent,
|
||||
)
|
||||
from .reasoning_events import (
|
||||
AgentReasoningStartedEvent,
|
||||
AgentReasoningCompletedEvent,
|
||||
AgentReasoningFailedEvent,
|
||||
)
|
||||
from .knowledge_events import (
|
||||
KnowledgeRetrievalStartedEvent,
|
||||
KnowledgeRetrievalCompletedEvent,
|
||||
KnowledgeQueryStartedEvent,
|
||||
KnowledgeQueryCompletedEvent,
|
||||
KnowledgeQueryFailedEvent,
|
||||
KnowledgeSearchQueryFailedEvent,
|
||||
)
|
||||
|
||||
EventTypes = Union[
|
||||
CrewKickoffStartedEvent,
|
||||
@@ -74,4 +87,13 @@ EventTypes = Union[
|
||||
LLMStreamChunkEvent,
|
||||
LLMGuardrailStartedEvent,
|
||||
LLMGuardrailCompletedEvent,
|
||||
AgentReasoningStartedEvent,
|
||||
AgentReasoningCompletedEvent,
|
||||
AgentReasoningFailedEvent,
|
||||
KnowledgeRetrievalStartedEvent,
|
||||
KnowledgeRetrievalCompletedEvent,
|
||||
KnowledgeQueryStartedEvent,
|
||||
KnowledgeQueryCompletedEvent,
|
||||
KnowledgeQueryFailedEvent,
|
||||
KnowledgeSearchQueryFailedEvent,
|
||||
]
|
||||
|
||||
@@ -19,10 +19,13 @@ class LLMGuardrailStartedEvent(BaseEvent):
|
||||
from inspect import getsource
|
||||
|
||||
from crewai.tasks.llm_guardrail import LLMGuardrail
|
||||
from crewai.tasks.hallucination_guardrail import HallucinationGuardrail
|
||||
|
||||
super().__init__(**data)
|
||||
|
||||
if isinstance(self.guardrail, LLMGuardrail):
|
||||
if isinstance(self.guardrail, LLMGuardrail) or isinstance(
|
||||
self.guardrail, HallucinationGuardrail
|
||||
):
|
||||
self.guardrail = self.guardrail.description.strip()
|
||||
elif isinstance(self.guardrail, Callable):
|
||||
self.guardrail = getsource(self.guardrail).strip()
|
||||
|
||||
31
src/crewai/utilities/events/reasoning_events.py
Normal file
@@ -0,0 +1,31 @@
|
||||
from crewai.utilities.events.base_events import BaseEvent
|
||||
|
||||
|
||||
class AgentReasoningStartedEvent(BaseEvent):
|
||||
"""Event emitted when an agent starts reasoning about a task."""
|
||||
|
||||
type: str = "agent_reasoning_started"
|
||||
agent_role: str
|
||||
task_id: str
|
||||
attempt: int = 1 # The current reasoning/refinement attempt
|
||||
|
||||
|
||||
class AgentReasoningCompletedEvent(BaseEvent):
|
||||
"""Event emitted when an agent finishes its reasoning process."""
|
||||
|
||||
type: str = "agent_reasoning_completed"
|
||||
agent_role: str
|
||||
task_id: str
|
||||
plan: str
|
||||
ready: bool
|
||||
attempt: int = 1
|
||||
|
||||
|
||||
class AgentReasoningFailedEvent(BaseEvent):
|
||||
"""Event emitted when the reasoning process fails."""
|
||||
|
||||
type: str = "agent_reasoning_failed"
|
||||
agent_role: str
|
||||
task_id: str
|
||||
error: str
|
||||
attempt: int = 1
|
||||
@@ -4,6 +4,7 @@ from rich.console import Console
|
||||
from rich.panel import Panel
|
||||
from rich.text import Text
|
||||
from rich.tree import Tree
|
||||
from rich.live import Live
|
||||
|
||||
|
||||
class ConsoleFormatter:
|
||||
@@ -15,10 +16,17 @@ class ConsoleFormatter:
|
||||
current_method_branch: Optional[Tree] = None
|
||||
current_lite_agent_branch: Optional[Tree] = None
|
||||
tool_usage_counts: Dict[str, int] = {}
|
||||
current_reasoning_branch: Optional[Tree] = None # Track reasoning status
|
||||
|
||||
def __init__(self, verbose: bool = False):
|
||||
self.console = Console(width=None)
|
||||
self.verbose = verbose
|
||||
# Live instance to dynamically update a Tree renderable (e.g. the Crew tree)
|
||||
# When multiple Tree objects are printed sequentially we reuse this Live
|
||||
# instance so the previous render is replaced instead of writing a new one.
|
||||
# Once any non-Tree renderable is printed we stop the Live session so the
|
||||
# final Tree persists on the terminal.
|
||||
self._live: Optional[Live] = None
|
||||
|
||||
def create_panel(self, content: Text, title: str, style: str = "blue") -> Panel:
|
||||
"""Create a standardized panel with consistent styling."""
|
||||
@@ -59,7 +67,7 @@ class ConsoleFormatter:
|
||||
label.append(f"{prefix} ", style=f"{style} bold")
|
||||
label.append(name, style=style)
|
||||
if status:
|
||||
label.append("\n Status: ", style="white")
|
||||
label.append("\nStatus: ", style="white")
|
||||
label.append(status, style=f"{style} bold")
|
||||
tree.label = label
|
||||
|
||||
@@ -68,7 +76,46 @@ class ConsoleFormatter:
|
||||
return parent.add(Text(text, style=style))
|
||||
|
||||
def print(self, *args, **kwargs) -> None:
|
||||
"""Print to console with consistent formatting if verbose is enabled."""
|
||||
"""Custom print that replaces consecutive Tree renders.
|
||||
|
||||
* If the argument is a single ``Tree`` instance, we either start a
|
||||
``Live`` session (first tree) or update the existing one (subsequent
|
||||
trees). This results in the tree being rendered in-place instead of
|
||||
being appended repeatedly to the log.
|
||||
|
||||
* A blank call (no positional arguments) is ignored while a Live
|
||||
session is active so it does not prematurely terminate the tree
|
||||
rendering.
|
||||
|
||||
* Any other renderable will terminate the Live session (if one is
|
||||
active) so the last tree stays on screen and the new content is
|
||||
printed normally.
|
||||
"""
|
||||
|
||||
# Case 1: updating / starting live Tree rendering
|
||||
if len(args) == 1 and isinstance(args[0], Tree):
|
||||
tree = args[0]
|
||||
|
||||
if not self._live:
|
||||
# Start a new Live session for the first tree
|
||||
self._live = Live(tree, console=self.console, refresh_per_second=4)
|
||||
self._live.start()
|
||||
else:
|
||||
# Update existing Live session
|
||||
self._live.update(tree, refresh=True)
|
||||
return # Nothing else to do
|
||||
|
||||
# Case 2: blank line while a live session is running – ignore so we
|
||||
# don't break the in-place rendering behaviour
|
||||
if len(args) == 0 and self._live:
|
||||
return
|
||||
|
||||
# Case 3: printing something other than a Tree → terminate live session
|
||||
if self._live:
|
||||
self._live.stop()
|
||||
self._live = None
|
||||
|
||||
# Finally, pass through to the regular Console.print implementation
|
||||
self.console.print(*args, **kwargs)
|
||||
|
||||
def print_panel(
|
||||
@@ -156,7 +203,7 @@ class ConsoleFormatter:
|
||||
|
||||
task_content = Text()
|
||||
task_content.append(f"📋 Task: {task_id}", style="yellow bold")
|
||||
task_content.append("\n Status: ", style="white")
|
||||
task_content.append("\nStatus: ", style="white")
|
||||
task_content.append("Executing Task...", style="yellow dim")
|
||||
|
||||
task_branch = None
|
||||
@@ -196,11 +243,17 @@ class ConsoleFormatter:
|
||||
# Update tree label
|
||||
for branch in crew_tree.children:
|
||||
if str(task_id) in str(branch.label):
|
||||
# Build label without introducing stray blank lines
|
||||
task_content = Text()
|
||||
# First line: Task ID
|
||||
task_content.append(f"📋 Task: {task_id}", style=f"{style} bold")
|
||||
task_content.append("\n Assigned to: ", style="white")
|
||||
|
||||
# Second line: Assigned to
|
||||
task_content.append("\nAssigned to: ", style="white")
|
||||
task_content.append(agent_role, style=style)
|
||||
task_content.append("\n Status: ", style="white")
|
||||
|
||||
# Third line: Status
|
||||
task_content.append("Status: ", style="white")
|
||||
task_content.append(status_text, style=f"{style} bold")
|
||||
branch.label = task_content
|
||||
self.print(crew_tree)
|
||||
@@ -219,18 +272,16 @@ class ConsoleFormatter:
|
||||
if not self.verbose or not task_branch or not crew_tree:
|
||||
return None
|
||||
|
||||
agent_branch = task_branch.add("")
|
||||
self.update_tree_label(
|
||||
agent_branch, "🤖 Agent:", agent_role, "green", "In Progress"
|
||||
)
|
||||
# Instead of creating a separate Agent node, we treat the task branch
|
||||
# itself as the logical agent branch so that Reasoning/Tool nodes are
|
||||
# nested under the task without an extra visual level.
|
||||
|
||||
self.print(crew_tree)
|
||||
self.print()
|
||||
# Store the task branch as the current_agent_branch for future nesting.
|
||||
self.current_agent_branch = task_branch
|
||||
|
||||
# Set the current_agent_branch attribute directly
|
||||
self.current_agent_branch = agent_branch
|
||||
|
||||
return agent_branch
|
||||
# No additional tree modification needed; return the task branch so
|
||||
# caller logic remains unchanged.
|
||||
return task_branch
|
||||
|
||||
def update_agent_status(
|
||||
self,
|
||||
@@ -240,19 +291,10 @@ class ConsoleFormatter:
|
||||
status: str = "completed",
|
||||
) -> None:
|
||||
"""Update agent status in the tree."""
|
||||
if not self.verbose or agent_branch is None or crew_tree is None:
|
||||
return
|
||||
|
||||
self.update_tree_label(
|
||||
agent_branch,
|
||||
"🤖 Agent:",
|
||||
agent_role,
|
||||
"green",
|
||||
"✅ Completed" if status == "completed" else "❌ Failed",
|
||||
)
|
||||
|
||||
self.print(crew_tree)
|
||||
self.print()
|
||||
# We no longer render a separate agent branch, so this method simply
|
||||
# updates the stored branch reference (already the task branch) without
|
||||
# altering the tree. Keeping it a no-op avoids duplicate status lines.
|
||||
return
|
||||
|
||||
def create_flow_tree(self, flow_name: str, flow_id: str) -> Optional[Tree]:
|
||||
"""Create and initialize a flow tree."""
|
||||
@@ -265,7 +307,7 @@ class ConsoleFormatter:
|
||||
flow_label = Text()
|
||||
flow_label.append("🌊 Flow: ", style="blue bold")
|
||||
flow_label.append(flow_name, style="blue")
|
||||
flow_label.append("\n ID: ", style="white")
|
||||
flow_label.append("\nID: ", style="white")
|
||||
flow_label.append(flow_id, style="blue")
|
||||
|
||||
flow_tree = Tree(flow_label)
|
||||
@@ -280,7 +322,7 @@ class ConsoleFormatter:
|
||||
flow_label = Text()
|
||||
flow_label.append("🌊 Flow: ", style="blue bold")
|
||||
flow_label.append(flow_name, style="blue")
|
||||
flow_label.append("\n ID: ", style="white")
|
||||
flow_label.append("\nID: ", style="white")
|
||||
flow_label.append(flow_id, style="blue")
|
||||
flow_tree.label = flow_label
|
||||
|
||||
@@ -394,12 +436,22 @@ class ConsoleFormatter:
|
||||
if not self.verbose:
|
||||
return None
|
||||
|
||||
# Use LiteAgent branch if available, otherwise use regular agent branch
|
||||
branch_to_use = self.current_lite_agent_branch or agent_branch
|
||||
tree_to_use = branch_to_use or crew_tree
|
||||
# Parent for tool usage: LiteAgent > Agent > Task
|
||||
branch_to_use = (
|
||||
self.current_lite_agent_branch
|
||||
or agent_branch
|
||||
or self.current_task_branch
|
||||
)
|
||||
|
||||
# Render full crew tree when available for consistent live updates
|
||||
tree_to_use = self.current_crew_tree or crew_tree or branch_to_use
|
||||
|
||||
if branch_to_use is None or tree_to_use is None:
|
||||
return None
|
||||
# If we don't have a valid branch, default to crew_tree if provided
|
||||
if crew_tree is not None:
|
||||
branch_to_use = tree_to_use = crew_tree
|
||||
else:
|
||||
return None
|
||||
|
||||
# Update tool usage count
|
||||
self.tool_usage_counts[tool_name] = self.tool_usage_counts.get(tool_name, 0) + 1
|
||||
@@ -418,10 +470,9 @@ class ConsoleFormatter:
|
||||
"yellow",
|
||||
)
|
||||
|
||||
# Only print if this is a new tool usage
|
||||
if tool_branch not in branch_to_use.children:
|
||||
self.print(tree_to_use)
|
||||
self.print()
|
||||
# Print updated tree immediately
|
||||
self.print(tree_to_use)
|
||||
self.print()
|
||||
|
||||
return tool_branch
|
||||
|
||||
@@ -435,8 +486,8 @@ class ConsoleFormatter:
|
||||
if not self.verbose or tool_branch is None:
|
||||
return
|
||||
|
||||
# Use LiteAgent branch if available, otherwise use crew tree
|
||||
tree_to_use = self.current_lite_agent_branch or crew_tree
|
||||
# Decide which tree to render: prefer full crew tree, else parent branch
|
||||
tree_to_use = self.current_crew_tree or crew_tree or self.current_task_branch
|
||||
if tree_to_use is None:
|
||||
return
|
||||
|
||||
@@ -467,8 +518,8 @@ class ConsoleFormatter:
|
||||
if not self.verbose:
|
||||
return
|
||||
|
||||
# Use LiteAgent branch if available, otherwise use crew tree
|
||||
tree_to_use = self.current_lite_agent_branch or crew_tree
|
||||
# Decide which tree to render: prefer full crew tree, else parent branch
|
||||
tree_to_use = self.current_crew_tree or crew_tree or self.current_task_branch
|
||||
|
||||
if tool_branch:
|
||||
self.update_tree_label(
|
||||
@@ -496,12 +547,22 @@ class ConsoleFormatter:
|
||||
if not self.verbose:
|
||||
return None
|
||||
|
||||
# Use LiteAgent branch if available, otherwise use regular agent branch
|
||||
branch_to_use = self.current_lite_agent_branch or agent_branch
|
||||
tree_to_use = branch_to_use or crew_tree
|
||||
# Parent for tool usage: LiteAgent > Agent > Task
|
||||
branch_to_use = (
|
||||
self.current_lite_agent_branch
|
||||
or agent_branch
|
||||
or self.current_task_branch
|
||||
)
|
||||
|
||||
# Render full crew tree when available for consistent live updates
|
||||
tree_to_use = self.current_crew_tree or crew_tree or branch_to_use
|
||||
|
||||
if branch_to_use is None or tree_to_use is None:
|
||||
return None
|
||||
# If we don't have a valid branch, default to crew_tree if provided
|
||||
if crew_tree is not None:
|
||||
branch_to_use = tree_to_use = crew_tree
|
||||
else:
|
||||
return None
|
||||
|
||||
# Only add thinking status if we don't have a current tool branch
|
||||
if self.current_tool_branch is None:
|
||||
@@ -523,24 +584,33 @@ class ConsoleFormatter:
|
||||
if not self.verbose or tool_branch is None:
|
||||
return
|
||||
|
||||
# Use LiteAgent branch if available, otherwise use regular agent branch
|
||||
branch_to_use = self.current_lite_agent_branch or agent_branch
|
||||
tree_to_use = branch_to_use or crew_tree
|
||||
|
||||
if branch_to_use is None or tree_to_use is None:
|
||||
# Decide which tree to render: prefer full crew tree, else parent branch
|
||||
tree_to_use = self.current_crew_tree or crew_tree or self.current_task_branch
|
||||
if tree_to_use is None:
|
||||
return
|
||||
|
||||
# Remove the thinking status node when complete, but only if it exists
|
||||
# Remove the thinking status node when complete
|
||||
if "Thinking" in str(tool_branch.label):
|
||||
try:
|
||||
# Check if the node is actually in the children list
|
||||
if tool_branch in branch_to_use.children:
|
||||
branch_to_use.children.remove(tool_branch)
|
||||
self.print(tree_to_use)
|
||||
self.print()
|
||||
except Exception:
|
||||
# If any error occurs during removal, just continue without removing
|
||||
pass
|
||||
parents = [
|
||||
self.current_lite_agent_branch,
|
||||
self.current_agent_branch,
|
||||
self.current_task_branch,
|
||||
tree_to_use,
|
||||
]
|
||||
removed = False
|
||||
for parent in parents:
|
||||
if isinstance(parent, Tree) and tool_branch in parent.children:
|
||||
parent.children.remove(tool_branch)
|
||||
removed = True
|
||||
break
|
||||
|
||||
# Clear pointer if we just removed the current_tool_branch
|
||||
if self.current_tool_branch is tool_branch:
|
||||
self.current_tool_branch = None
|
||||
|
||||
if removed:
|
||||
self.print(tree_to_use)
|
||||
self.print()
|
||||
|
||||
def handle_llm_call_failed(
|
||||
self, tool_branch: Optional[Tree], error: str, crew_tree: Optional[Tree]
|
||||
@@ -549,8 +619,8 @@ class ConsoleFormatter:
|
||||
if not self.verbose:
|
||||
return
|
||||
|
||||
# Use LiteAgent branch if available, otherwise use crew tree
|
||||
tree_to_use = self.current_lite_agent_branch or crew_tree
|
||||
# Decide which tree to render: prefer full crew tree, else parent branch
|
||||
tree_to_use = self.current_crew_tree or crew_tree or self.current_task_branch
|
||||
|
||||
# Update tool branch if it exists
|
||||
if tool_branch:
|
||||
@@ -592,7 +662,7 @@ class ConsoleFormatter:
|
||||
test_label = Text()
|
||||
test_label.append("🧪 Test: ", style="blue bold")
|
||||
test_label.append(crew_name or "Crew", style="blue")
|
||||
test_label.append("\n Status: ", style="white")
|
||||
test_label.append("\nStatus: ", style="white")
|
||||
test_label.append("In Progress", style="yellow")
|
||||
|
||||
test_tree = Tree(test_label)
|
||||
@@ -614,7 +684,7 @@ class ConsoleFormatter:
|
||||
test_label = Text()
|
||||
test_label.append("✅ Test: ", style="green bold")
|
||||
test_label.append(crew_name or "Crew", style="green")
|
||||
test_label.append("\n Status: ", style="white")
|
||||
test_label.append("\nStatus: ", style="white")
|
||||
test_label.append("Completed", style="green bold")
|
||||
flow_tree.label = test_label
|
||||
|
||||
@@ -703,7 +773,7 @@ class ConsoleFormatter:
|
||||
lite_agent_label = Text()
|
||||
lite_agent_label.append("🤖 LiteAgent: ", style="cyan bold")
|
||||
lite_agent_label.append(lite_agent_role, style="cyan")
|
||||
lite_agent_label.append("\n Status: ", style="white")
|
||||
lite_agent_label.append("\nStatus: ", style="white")
|
||||
lite_agent_label.append("In Progress", style="yellow")
|
||||
|
||||
lite_agent_tree = Tree(lite_agent_label)
|
||||
@@ -742,7 +812,7 @@ class ConsoleFormatter:
|
||||
lite_agent_label = Text()
|
||||
lite_agent_label.append(f"{prefix} ", style=f"{style} bold")
|
||||
lite_agent_label.append(lite_agent_role, style=style)
|
||||
lite_agent_label.append("\n Status: ", style="white")
|
||||
lite_agent_label.append("Status: ", style="white")
|
||||
lite_agent_label.append(status_text, style=f"{style} bold")
|
||||
lite_agent_branch.label = lite_agent_label
|
||||
|
||||
@@ -797,7 +867,7 @@ class ConsoleFormatter:
|
||||
tree_to_use = branch_to_use or crew_tree
|
||||
|
||||
if branch_to_use is None or tree_to_use is None:
|
||||
# If we don't have a valid branch, use crew_tree as the branch if available
|
||||
# If we don't have a valid branch, default to crew_tree if provided
|
||||
if crew_tree is not None:
|
||||
branch_to_use = tree_to_use = crew_tree
|
||||
else:
|
||||
@@ -982,3 +1052,124 @@ class ConsoleFormatter:
|
||||
"Knowledge Search Failed", "Search Error", "red", Error=error
|
||||
)
|
||||
self.print_panel(error_content, "Search Error", "red")
|
||||
|
||||
# ----------- AGENT REASONING EVENTS -----------
|
||||
|
||||
def handle_reasoning_started(
|
||||
self,
|
||||
agent_branch: Optional[Tree],
|
||||
attempt: int,
|
||||
crew_tree: Optional[Tree],
|
||||
) -> Optional[Tree]:
|
||||
"""Handle agent reasoning started (or refinement) event."""
|
||||
if not self.verbose:
|
||||
return None
|
||||
|
||||
# Prefer LiteAgent > Agent > Task branch as the parent for reasoning
|
||||
branch_to_use = (
|
||||
self.current_lite_agent_branch
|
||||
or agent_branch
|
||||
or self.current_task_branch
|
||||
)
|
||||
|
||||
# We always want to render the full crew tree when possible so the
|
||||
# Live view updates coherently. Fallbacks: crew tree → branch itself.
|
||||
tree_to_use = self.current_crew_tree or crew_tree or branch_to_use
|
||||
|
||||
if branch_to_use is None:
|
||||
# Nothing to attach to, abort
|
||||
return None
|
||||
|
||||
# Reuse existing reasoning branch if present
|
||||
reasoning_branch = self.current_reasoning_branch
|
||||
if reasoning_branch is None:
|
||||
reasoning_branch = branch_to_use.add("")
|
||||
self.current_reasoning_branch = reasoning_branch
|
||||
|
||||
# Build label text depending on attempt
|
||||
status_text = (
|
||||
f"Reasoning (Attempt {attempt})" if attempt > 1 else "Reasoning..."
|
||||
)
|
||||
self.update_tree_label(reasoning_branch, "🧠", status_text, "blue")
|
||||
|
||||
self.print(tree_to_use)
|
||||
self.print()
|
||||
|
||||
return reasoning_branch
|
||||
|
||||
def handle_reasoning_completed(
|
||||
self,
|
||||
plan: str,
|
||||
ready: bool,
|
||||
crew_tree: Optional[Tree],
|
||||
) -> None:
|
||||
"""Handle agent reasoning completed event."""
|
||||
if not self.verbose:
|
||||
return
|
||||
|
||||
reasoning_branch = self.current_reasoning_branch
|
||||
tree_to_use = (
|
||||
self.current_crew_tree
|
||||
or self.current_lite_agent_branch
|
||||
or self.current_task_branch
|
||||
or crew_tree
|
||||
)
|
||||
|
||||
style = "green" if ready else "yellow"
|
||||
status_text = "Reasoning Completed" if ready else "Reasoning Completed (Not Ready)"
|
||||
|
||||
if reasoning_branch is not None:
|
||||
self.update_tree_label(reasoning_branch, "✅", status_text, style)
|
||||
|
||||
if tree_to_use is not None:
|
||||
self.print(tree_to_use)
|
||||
|
||||
# Show plan in a panel (trim very long plans)
|
||||
if plan:
|
||||
plan_panel = Panel(
|
||||
Text(plan, style="white"),
|
||||
title="🧠 Reasoning Plan",
|
||||
border_style=style,
|
||||
padding=(1, 2),
|
||||
)
|
||||
self.print(plan_panel)
|
||||
|
||||
self.print()
|
||||
|
||||
# Clear stored branch after completion
|
||||
self.current_reasoning_branch = None
|
||||
|
||||
def handle_reasoning_failed(
|
||||
self,
|
||||
error: str,
|
||||
crew_tree: Optional[Tree],
|
||||
) -> None:
|
||||
"""Handle agent reasoning failure event."""
|
||||
if not self.verbose:
|
||||
return
|
||||
|
||||
reasoning_branch = self.current_reasoning_branch
|
||||
tree_to_use = (
|
||||
self.current_crew_tree
|
||||
or self.current_lite_agent_branch
|
||||
or self.current_task_branch
|
||||
or crew_tree
|
||||
)
|
||||
|
||||
if reasoning_branch is not None:
|
||||
self.update_tree_label(reasoning_branch, "❌", "Reasoning Failed", "red")
|
||||
|
||||
if tree_to_use is not None:
|
||||
self.print(tree_to_use)
|
||||
|
||||
# Error panel
|
||||
error_content = self.create_status_content(
|
||||
"Reasoning Failed",
|
||||
"Error",
|
||||
"red",
|
||||
Error=error,
|
||||
)
|
||||
self.print_panel(error_content, "Reasoning Error", "red")
|
||||
|
||||
# Clear stored branch after failure
|
||||
self.current_reasoning_branch = None
|
||||
|
||||
@@ -1,6 +1,6 @@
|
||||
import re
|
||||
from typing import TYPE_CHECKING, List
|
||||
|
||||
|
||||
if TYPE_CHECKING:
|
||||
from crewai.task import Task
|
||||
from crewai.tasks.task_output import TaskOutput
|
||||
@@ -17,6 +17,11 @@ def aggregate_raw_outputs_from_task_outputs(task_outputs: List["TaskOutput"]) ->
|
||||
|
||||
def aggregate_raw_outputs_from_tasks(tasks: List["Task"]) -> str:
|
||||
"""Generate string context from the tasks."""
|
||||
task_outputs = [task.output for task in tasks if task.output is not None]
|
||||
|
||||
task_outputs = (
|
||||
[task.output for task in tasks if task.output is not None]
|
||||
if isinstance(tasks, list)
|
||||
else []
|
||||
)
|
||||
|
||||
return aggregate_raw_outputs_from_task_outputs(task_outputs)
|
||||
|
||||
387
src/crewai/utilities/reasoning_handler.py
Normal file
@@ -0,0 +1,387 @@
|
||||
import logging
|
||||
import json
|
||||
from typing import Tuple, cast
|
||||
|
||||
from pydantic import BaseModel, Field
|
||||
|
||||
from crewai.agent import Agent
|
||||
from crewai.task import Task
|
||||
from crewai.utilities import I18N
|
||||
from crewai.llm import LLM
|
||||
from crewai.utilities.events.crewai_event_bus import crewai_event_bus
|
||||
from crewai.utilities.events.reasoning_events import (
|
||||
AgentReasoningStartedEvent,
|
||||
AgentReasoningCompletedEvent,
|
||||
AgentReasoningFailedEvent,
|
||||
)
|
||||
|
||||
|
||||
class ReasoningPlan(BaseModel):
|
||||
"""Model representing a reasoning plan for a task."""
|
||||
plan: str = Field(description="The detailed reasoning plan for the task.")
|
||||
ready: bool = Field(description="Whether the agent is ready to execute the task.")
|
||||
|
||||
|
||||
class AgentReasoningOutput(BaseModel):
|
||||
"""Model representing the output of the agent reasoning process."""
|
||||
plan: ReasoningPlan = Field(description="The reasoning plan for the task.")
|
||||
|
||||
|
||||
class ReasoningFunction(BaseModel):
|
||||
"""Model for function calling with reasoning."""
|
||||
plan: str = Field(description="The detailed reasoning plan for the task.")
|
||||
ready: bool = Field(description="Whether the agent is ready to execute the task.")
|
||||
|
||||
|
||||
class AgentReasoning:
|
||||
"""
|
||||
Handles the agent reasoning process, enabling an agent to reflect and create a plan
|
||||
before executing a task.
|
||||
"""
|
||||
def __init__(self, task: Task, agent: Agent):
|
||||
if not task or not agent:
|
||||
raise ValueError("Both task and agent must be provided.")
|
||||
self.task = task
|
||||
self.agent = agent
|
||||
self.llm = cast(LLM, agent.llm)
|
||||
self.logger = logging.getLogger(__name__)
|
||||
self.i18n = I18N()
|
||||
|
||||
def handle_agent_reasoning(self) -> AgentReasoningOutput:
|
||||
"""
|
||||
Public method for the reasoning process that creates and refines a plan
|
||||
for the task until the agent is ready to execute it.
|
||||
|
||||
Returns:
|
||||
AgentReasoningOutput: The output of the agent reasoning process.
|
||||
"""
|
||||
# Emit a reasoning started event (attempt 1)
|
||||
try:
|
||||
crewai_event_bus.emit(
|
||||
self.agent,
|
||||
AgentReasoningStartedEvent(
|
||||
agent_role=self.agent.role,
|
||||
task_id=str(self.task.id),
|
||||
attempt=1,
|
||||
),
|
||||
)
|
||||
except Exception:
|
||||
# Ignore event bus errors to avoid breaking execution
|
||||
pass
|
||||
|
||||
try:
|
||||
output = self.__handle_agent_reasoning()
|
||||
|
||||
# Emit reasoning completed event
|
||||
try:
|
||||
crewai_event_bus.emit(
|
||||
self.agent,
|
||||
AgentReasoningCompletedEvent(
|
||||
agent_role=self.agent.role,
|
||||
task_id=str(self.task.id),
|
||||
plan=output.plan.plan,
|
||||
ready=output.plan.ready,
|
||||
attempt=1,
|
||||
),
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
return output
|
||||
except Exception as e:
|
||||
# Emit reasoning failed event
|
||||
try:
|
||||
crewai_event_bus.emit(
|
||||
self.agent,
|
||||
AgentReasoningFailedEvent(
|
||||
agent_role=self.agent.role,
|
||||
task_id=str(self.task.id),
|
||||
error=str(e),
|
||||
attempt=1,
|
||||
),
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
raise
|
||||
|
||||
def __handle_agent_reasoning(self) -> AgentReasoningOutput:
|
||||
"""
|
||||
Private method that handles the agent reasoning process.
|
||||
|
||||
Returns:
|
||||
AgentReasoningOutput: The output of the agent reasoning process.
|
||||
"""
|
||||
plan, ready = self.__create_initial_plan()
|
||||
|
||||
plan, ready = self.__refine_plan_if_needed(plan, ready)
|
||||
|
||||
reasoning_plan = ReasoningPlan(plan=plan, ready=ready)
|
||||
return AgentReasoningOutput(plan=reasoning_plan)
|
||||
|
||||
def __create_initial_plan(self) -> Tuple[str, bool]:
|
||||
"""
|
||||
Creates the initial reasoning plan for the task.
|
||||
|
||||
Returns:
|
||||
Tuple[str, bool]: The initial plan and whether the agent is ready to execute the task.
|
||||
"""
|
||||
reasoning_prompt = self.__create_reasoning_prompt()
|
||||
|
||||
if self.llm.supports_function_calling():
|
||||
plan, ready = self.__call_with_function(reasoning_prompt, "initial_plan")
|
||||
return plan, ready
|
||||
else:
|
||||
system_prompt = self.i18n.retrieve("reasoning", "initial_plan").format(
|
||||
role=self.agent.role,
|
||||
goal=self.agent.goal,
|
||||
backstory=self.__get_agent_backstory()
|
||||
)
|
||||
|
||||
response = self.llm.call(
|
||||
[
|
||||
{"role": "system", "content": system_prompt},
|
||||
{"role": "user", "content": reasoning_prompt}
|
||||
]
|
||||
)
|
||||
|
||||
return self.__parse_reasoning_response(str(response))
|
||||
|
||||
def __refine_plan_if_needed(self, plan: str, ready: bool) -> Tuple[str, bool]:
|
||||
"""
|
||||
Refines the reasoning plan if the agent is not ready to execute the task.
|
||||
|
||||
Args:
|
||||
plan: The current reasoning plan.
|
||||
ready: Whether the agent is ready to execute the task.
|
||||
|
||||
Returns:
|
||||
Tuple[str, bool]: The refined plan and whether the agent is ready to execute the task.
|
||||
"""
|
||||
attempt = 1
|
||||
max_attempts = self.agent.max_reasoning_attempts
|
||||
|
||||
while not ready and (max_attempts is None or attempt < max_attempts):
|
||||
# Emit event for each refinement attempt
|
||||
try:
|
||||
crewai_event_bus.emit(
|
||||
self.agent,
|
||||
AgentReasoningStartedEvent(
|
||||
agent_role=self.agent.role,
|
||||
task_id=str(self.task.id),
|
||||
attempt=attempt + 1,
|
||||
),
|
||||
)
|
||||
except Exception:
|
||||
pass
|
||||
|
||||
refine_prompt = self.__create_refine_prompt(plan)
|
||||
|
||||
if self.llm.supports_function_calling():
|
||||
plan, ready = self.__call_with_function(refine_prompt, "refine_plan")
|
||||
else:
|
||||
system_prompt = self.i18n.retrieve("reasoning", "refine_plan").format(
|
||||
role=self.agent.role,
|
||||
goal=self.agent.goal,
|
||||
backstory=self.__get_agent_backstory()
|
||||
)
|
||||
|
||||
response = self.llm.call(
|
||||
[
|
||||
{"role": "system", "content": system_prompt},
|
||||
{"role": "user", "content": refine_prompt}
|
||||
]
|
||||
)
|
||||
plan, ready = self.__parse_reasoning_response(str(response))
|
||||
|
||||
attempt += 1
|
||||
|
||||
if max_attempts is not None and attempt >= max_attempts:
|
||||
self.logger.warning(
|
||||
f"Agent reasoning reached maximum attempts ({max_attempts}) without being ready. Proceeding with current plan."
|
||||
)
|
||||
break
|
||||
|
||||
return plan, ready
|
||||
|
||||
def __call_with_function(self, prompt: str, prompt_type: str) -> Tuple[str, bool]:
|
||||
"""
|
||||
Calls the LLM with function calling to get a reasoning plan.
|
||||
|
||||
Args:
|
||||
prompt: The prompt to send to the LLM.
|
||||
prompt_type: The type of prompt (initial_plan or refine_plan).
|
||||
|
||||
Returns:
|
||||
Tuple[str, bool]: A tuple containing the plan and whether the agent is ready.
|
||||
"""
|
||||
self.logger.debug(f"Using function calling for {prompt_type} reasoning")
|
||||
|
||||
function_schema = {
|
||||
"type": "function",
|
||||
"function": {
|
||||
"name": "create_reasoning_plan",
|
||||
"description": "Create or refine a reasoning plan for a task",
|
||||
"parameters": {
|
||||
"type": "object",
|
||||
"properties": {
|
||||
"plan": {
|
||||
"type": "string",
|
||||
"description": "The detailed reasoning plan for the task."
|
||||
},
|
||||
"ready": {
|
||||
"type": "boolean",
|
||||
"description": "Whether the agent is ready to execute the task."
|
||||
}
|
||||
},
|
||||
"required": ["plan", "ready"]
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
try:
|
||||
system_prompt = self.i18n.retrieve("reasoning", prompt_type).format(
|
||||
role=self.agent.role,
|
||||
goal=self.agent.goal,
|
||||
backstory=self.__get_agent_backstory()
|
||||
)
|
||||
|
||||
# Prepare a simple callable that just returns the tool arguments as JSON
|
||||
def _create_reasoning_plan(plan: str, ready: bool): # noqa: N802
|
||||
"""Return the reasoning plan result in JSON string form."""
|
||||
return json.dumps({"plan": plan, "ready": ready})
|
||||
|
||||
response = self.llm.call(
|
||||
[
|
||||
{"role": "system", "content": system_prompt},
|
||||
{"role": "user", "content": prompt}
|
||||
],
|
||||
tools=[function_schema],
|
||||
available_functions={"create_reasoning_plan": _create_reasoning_plan},
|
||||
)
|
||||
|
||||
self.logger.debug(f"Function calling response: {response[:100]}...")
|
||||
|
||||
try:
|
||||
result = json.loads(response)
|
||||
if "plan" in result and "ready" in result:
|
||||
return result["plan"], result["ready"]
|
||||
except (json.JSONDecodeError, KeyError):
|
||||
pass
|
||||
|
||||
response_str = str(response)
|
||||
return response_str, "READY: I am ready to execute the task." in response_str
|
||||
|
||||
except Exception as e:
|
||||
self.logger.warning(f"Error during function calling: {str(e)}. Falling back to text parsing.")
|
||||
|
||||
try:
|
||||
system_prompt = self.i18n.retrieve("reasoning", prompt_type).format(
|
||||
role=self.agent.role,
|
||||
goal=self.agent.goal,
|
||||
backstory=self.__get_agent_backstory()
|
||||
)
|
||||
|
||||
fallback_response = self.llm.call(
|
||||
[
|
||||
{"role": "system", "content": system_prompt},
|
||||
{"role": "user", "content": prompt}
|
||||
]
|
||||
)
|
||||
|
||||
fallback_str = str(fallback_response)
|
||||
return fallback_str, "READY: I am ready to execute the task." in fallback_str
|
||||
except Exception as inner_e:
|
||||
self.logger.error(f"Error during fallback text parsing: {str(inner_e)}")
|
||||
return "Failed to generate a plan due to an error.", True # Default to ready to avoid getting stuck
|
||||
|
||||
def __get_agent_backstory(self) -> str:
|
||||
"""
|
||||
Safely gets the agent's backstory, providing a default if not available.
|
||||
|
||||
Returns:
|
||||
str: The agent's backstory or a default value.
|
||||
"""
|
||||
return getattr(self.agent, "backstory", "No backstory provided")
|
||||
|
||||
def __create_reasoning_prompt(self) -> str:
|
||||
"""
|
||||
Creates a prompt for the agent to reason about the task.
|
||||
|
||||
Returns:
|
||||
str: The reasoning prompt.
|
||||
"""
|
||||
available_tools = self.__format_available_tools()
|
||||
|
||||
return self.i18n.retrieve("reasoning", "create_plan_prompt").format(
|
||||
role=self.agent.role,
|
||||
goal=self.agent.goal,
|
||||
backstory=self.__get_agent_backstory(),
|
||||
description=self.task.description,
|
||||
expected_output=self.task.expected_output,
|
||||
tools=available_tools
|
||||
)
|
||||
|
||||
def __format_available_tools(self) -> str:
|
||||
"""
|
||||
Formats the available tools for inclusion in the prompt.
|
||||
|
||||
Returns:
|
||||
str: Comma-separated list of tool names.
|
||||
"""
|
||||
try:
|
||||
return ', '.join([tool.name for tool in (self.task.tools or [])])
|
||||
except (AttributeError, TypeError):
|
||||
return "No tools available"
|
||||
|
||||
def __create_refine_prompt(self, current_plan: str) -> str:
|
||||
"""
|
||||
Creates a prompt for the agent to refine its reasoning plan.
|
||||
|
||||
Args:
|
||||
current_plan: The current reasoning plan.
|
||||
|
||||
Returns:
|
||||
str: The refine prompt.
|
||||
"""
|
||||
return self.i18n.retrieve("reasoning", "refine_plan_prompt").format(
|
||||
role=self.agent.role,
|
||||
goal=self.agent.goal,
|
||||
backstory=self.__get_agent_backstory(),
|
||||
current_plan=current_plan
|
||||
)
|
||||
|
||||
def __parse_reasoning_response(self, response: str) -> Tuple[str, bool]:
|
||||
"""
|
||||
Parses the reasoning response to extract the plan and whether
|
||||
the agent is ready to execute the task.
|
||||
|
||||
Args:
|
||||
response: The LLM response.
|
||||
|
||||
Returns:
|
||||
Tuple[str, bool]: The plan and whether the agent is ready to execute the task.
|
||||
"""
|
||||
if not response:
|
||||
return "No plan was generated.", False
|
||||
|
||||
plan = response
|
||||
ready = False
|
||||
|
||||
if "READY: I am ready to execute the task." in response:
|
||||
ready = True
|
||||
|
||||
return plan, ready
|
||||
|
||||
def _handle_agent_reasoning(self) -> AgentReasoningOutput:
|
||||
"""
|
||||
Deprecated method for backward compatibility.
|
||||
Use handle_agent_reasoning() instead.
|
||||
|
||||
Returns:
|
||||
AgentReasoningOutput: The output of the agent reasoning process.
|
||||
"""
|
||||
self.logger.warning(
|
||||
"The _handle_agent_reasoning method is deprecated. Use handle_agent_reasoning instead."
|
||||
)
|
||||
return self.handle_agent_reasoning()
|
||||
@@ -59,7 +59,7 @@ def interpolate_only(
|
||||
# The regex pattern to find valid variable placeholders
|
||||
# Matches {variable_name} where variable_name starts with a letter/underscore
|
||||
# and contains only letters, numbers, and underscores
|
||||
pattern = r"\{([A-Za-z_][A-Za-z0-9_]*)\}"
|
||||
pattern = r"\{([A-Za-z_][A-Za-z0-9_\-]*)\}"
|
||||
|
||||
# Find all matching variables in the input string
|
||||
variables = re.findall(pattern, input_string)
|
||||
|
||||
261
tests/agent_reasoning_test.py
Normal file
@@ -0,0 +1,261 @@
|
||||
"""Tests for reasoning in agents."""
|
||||
|
||||
import json
|
||||
import pytest
|
||||
|
||||
from crewai import Agent, Task
|
||||
from crewai.llm import LLM
|
||||
from crewai.utilities.reasoning_handler import AgentReasoning
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_llm_responses():
|
||||
"""Fixture for mock LLM responses."""
|
||||
return {
|
||||
"ready": "I'll solve this simple math problem.\n\nREADY: I am ready to execute the task.\n\n",
|
||||
"not_ready": "I need to think about derivatives.\n\nNOT READY: I need to refine my plan because I'm not sure about the derivative rules.",
|
||||
"ready_after_refine": "I'll use the power rule for derivatives where d/dx(x^n) = n*x^(n-1).\n\nREADY: I am ready to execute the task.",
|
||||
"execution": "4"
|
||||
}
|
||||
|
||||
|
||||
def test_agent_with_reasoning(mock_llm_responses):
|
||||
"""Test agent with reasoning."""
|
||||
llm = LLM("gpt-3.5-turbo")
|
||||
|
||||
agent = Agent(
|
||||
role="Test Agent",
|
||||
goal="To test the reasoning feature",
|
||||
backstory="I am a test agent created to verify the reasoning feature works correctly.",
|
||||
llm=llm,
|
||||
reasoning=True,
|
||||
verbose=True
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Simple math task: What's 2+2?",
|
||||
expected_output="The answer should be a number.",
|
||||
agent=agent
|
||||
)
|
||||
|
||||
agent.llm.call = lambda messages, *args, **kwargs: (
|
||||
mock_llm_responses["ready"]
|
||||
if any("create a detailed plan" in msg.get("content", "") for msg in messages)
|
||||
else mock_llm_responses["execution"]
|
||||
)
|
||||
|
||||
result = agent.execute_task(task)
|
||||
|
||||
assert result == mock_llm_responses["execution"]
|
||||
assert "Reasoning Plan:" in task.description
|
||||
|
||||
|
||||
def test_agent_with_reasoning_not_ready_initially(mock_llm_responses):
|
||||
"""Test agent with reasoning that requires refinement."""
|
||||
llm = LLM("gpt-3.5-turbo")
|
||||
|
||||
agent = Agent(
|
||||
role="Test Agent",
|
||||
goal="To test the reasoning feature",
|
||||
backstory="I am a test agent created to verify the reasoning feature works correctly.",
|
||||
llm=llm,
|
||||
reasoning=True,
|
||||
max_reasoning_attempts=2,
|
||||
verbose=True
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Complex math task: What's the derivative of x²?",
|
||||
expected_output="The answer should be a mathematical expression.",
|
||||
agent=agent
|
||||
)
|
||||
|
||||
call_count = [0]
|
||||
|
||||
def mock_llm_call(messages, *args, **kwargs):
|
||||
if any("create a detailed plan" in msg.get("content", "") for msg in messages) or any("refine your plan" in msg.get("content", "") for msg in messages):
|
||||
call_count[0] += 1
|
||||
if call_count[0] == 1:
|
||||
return mock_llm_responses["not_ready"]
|
||||
else:
|
||||
return mock_llm_responses["ready_after_refine"]
|
||||
else:
|
||||
return "2x"
|
||||
|
||||
agent.llm.call = mock_llm_call
|
||||
|
||||
result = agent.execute_task(task)
|
||||
|
||||
assert result == "2x"
|
||||
assert call_count[0] == 2 # Should have made 2 reasoning calls
|
||||
assert "Reasoning Plan:" in task.description
|
||||
|
||||
|
||||
def test_agent_with_reasoning_max_attempts_reached():
|
||||
"""Test agent with reasoning that reaches max attempts without being ready."""
|
||||
llm = LLM("gpt-3.5-turbo")
|
||||
|
||||
agent = Agent(
|
||||
role="Test Agent",
|
||||
goal="To test the reasoning feature",
|
||||
backstory="I am a test agent created to verify the reasoning feature works correctly.",
|
||||
llm=llm,
|
||||
reasoning=True,
|
||||
max_reasoning_attempts=2,
|
||||
verbose=True
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Complex math task: Solve the Riemann hypothesis.",
|
||||
expected_output="A proof or disproof of the hypothesis.",
|
||||
agent=agent
|
||||
)
|
||||
|
||||
call_count = [0]
|
||||
|
||||
def mock_llm_call(messages, *args, **kwargs):
|
||||
if any("create a detailed plan" in msg.get("content", "") for msg in messages) or any("refine your plan" in msg.get("content", "") for msg in messages):
|
||||
call_count[0] += 1
|
||||
return f"Attempt {call_count[0]}: I need more time to think.\n\nNOT READY: I need to refine my plan further."
|
||||
else:
|
||||
return "This is an unsolved problem in mathematics."
|
||||
|
||||
agent.llm.call = mock_llm_call
|
||||
|
||||
result = agent.execute_task(task)
|
||||
|
||||
assert result == "This is an unsolved problem in mathematics."
|
||||
assert call_count[0] == 2 # Should have made exactly 2 reasoning calls (max_attempts)
|
||||
assert "Reasoning Plan:" in task.description
|
||||
|
||||
|
||||
def test_agent_reasoning_input_validation():
|
||||
"""Test input validation in AgentReasoning."""
|
||||
llm = LLM("gpt-3.5-turbo")
|
||||
|
||||
agent = Agent(
|
||||
role="Test Agent",
|
||||
goal="To test the reasoning feature",
|
||||
backstory="I am a test agent created to verify the reasoning feature works correctly.",
|
||||
llm=llm,
|
||||
reasoning=True
|
||||
)
|
||||
|
||||
with pytest.raises(ValueError, match="Both task and agent must be provided"):
|
||||
AgentReasoning(task=None, agent=agent)
|
||||
|
||||
task = Task(
|
||||
description="Simple task",
|
||||
expected_output="Simple output"
|
||||
)
|
||||
with pytest.raises(ValueError, match="Both task and agent must be provided"):
|
||||
AgentReasoning(task=task, agent=None)
|
||||
|
||||
|
||||
def test_agent_reasoning_error_handling():
|
||||
"""Test error handling during the reasoning process."""
|
||||
llm = LLM("gpt-3.5-turbo")
|
||||
|
||||
agent = Agent(
|
||||
role="Test Agent",
|
||||
goal="To test the reasoning feature",
|
||||
backstory="I am a test agent created to verify the reasoning feature works correctly.",
|
||||
llm=llm,
|
||||
reasoning=True
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Task that will cause an error",
|
||||
expected_output="Output that will never be generated",
|
||||
agent=agent
|
||||
)
|
||||
|
||||
call_count = [0]
|
||||
|
||||
def mock_llm_call_error(*args, **kwargs):
|
||||
call_count[0] += 1
|
||||
if call_count[0] <= 2: # First calls are for reasoning
|
||||
raise Exception("LLM error during reasoning")
|
||||
return "Fallback execution result" # Return a value for task execution
|
||||
|
||||
agent.llm.call = mock_llm_call_error
|
||||
|
||||
result = agent.execute_task(task)
|
||||
|
||||
assert result == "Fallback execution result"
|
||||
assert call_count[0] > 2 # Ensure we called the mock multiple times
|
||||
|
||||
|
||||
def test_agent_with_function_calling():
|
||||
"""Test agent with reasoning using function calling."""
|
||||
llm = LLM("gpt-3.5-turbo")
|
||||
|
||||
agent = Agent(
|
||||
role="Test Agent",
|
||||
goal="To test the reasoning feature",
|
||||
backstory="I am a test agent created to verify the reasoning feature works correctly.",
|
||||
llm=llm,
|
||||
reasoning=True,
|
||||
verbose=True
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Simple math task: What's 2+2?",
|
||||
expected_output="The answer should be a number.",
|
||||
agent=agent
|
||||
)
|
||||
|
||||
agent.llm.supports_function_calling = lambda: True
|
||||
|
||||
def mock_function_call(messages, *args, **kwargs):
|
||||
if "tools" in kwargs:
|
||||
return json.dumps({
|
||||
"plan": "I'll solve this simple math problem: 2+2=4.",
|
||||
"ready": True
|
||||
})
|
||||
else:
|
||||
return "4"
|
||||
|
||||
agent.llm.call = mock_function_call
|
||||
|
||||
result = agent.execute_task(task)
|
||||
|
||||
assert result == "4"
|
||||
assert "Reasoning Plan:" in task.description
|
||||
assert "I'll solve this simple math problem: 2+2=4." in task.description
|
||||
|
||||
|
||||
def test_agent_with_function_calling_fallback():
|
||||
"""Test agent with reasoning using function calling that falls back to text parsing."""
|
||||
llm = LLM("gpt-3.5-turbo")
|
||||
|
||||
agent = Agent(
|
||||
role="Test Agent",
|
||||
goal="To test the reasoning feature",
|
||||
backstory="I am a test agent created to verify the reasoning feature works correctly.",
|
||||
llm=llm,
|
||||
reasoning=True,
|
||||
verbose=True
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Simple math task: What's 2+2?",
|
||||
expected_output="The answer should be a number.",
|
||||
agent=agent
|
||||
)
|
||||
|
||||
agent.llm.supports_function_calling = lambda: True
|
||||
|
||||
def mock_function_call(messages, *args, **kwargs):
|
||||
if "tools" in kwargs:
|
||||
return "Invalid JSON that will trigger fallback. READY: I am ready to execute the task."
|
||||
else:
|
||||
return "4"
|
||||
|
||||
agent.llm.call = mock_function_call
|
||||
|
||||
result = agent.execute_task(task)
|
||||
|
||||
assert result == "4"
|
||||
assert "Reasoning Plan:" in task.description
|
||||
assert "Invalid JSON that will trigger fallback" in task.description
|
||||
@@ -2044,7 +2044,7 @@ def test_agent_from_repository(mock_get_agent, mock_get_auth_token):
|
||||
"role": "test role",
|
||||
"goal": "test goal",
|
||||
"backstory": "test backstory",
|
||||
"tools": ["SerperDevTool"],
|
||||
"tools": [{"name": "SerperDevTool"}],
|
||||
}
|
||||
mock_get_agent.return_value = mock_get_response
|
||||
agent = Agent(from_repository="test_agent")
|
||||
@@ -2066,7 +2066,7 @@ def test_agent_from_repository_override_attributes(mock_get_agent, mock_get_auth
|
||||
"role": "test role",
|
||||
"goal": "test goal",
|
||||
"backstory": "test backstory",
|
||||
"tools": ["SerperDevTool"],
|
||||
"tools": [{"name": "SerperDevTool"}],
|
||||
}
|
||||
mock_get_agent.return_value = mock_get_response
|
||||
agent = Agent(from_repository="test_agent", role="Custom Role")
|
||||
@@ -2086,7 +2086,7 @@ def test_agent_from_repository_with_invalid_tools(mock_get_agent, mock_get_auth_
|
||||
"role": "test role",
|
||||
"goal": "test goal",
|
||||
"backstory": "test backstory",
|
||||
"tools": ["DoesNotExist"],
|
||||
"tools": [{"name": "DoesNotExist"}],
|
||||
}
|
||||
mock_get_agent.return_value = mock_get_response
|
||||
with pytest.raises(
|
||||
@@ -2096,6 +2096,19 @@ def test_agent_from_repository_with_invalid_tools(mock_get_agent, mock_get_auth_
|
||||
Agent(from_repository="test_agent")
|
||||
|
||||
|
||||
@patch("crewai.cli.plus_api.PlusAPI.get_agent")
|
||||
def test_agent_from_repository_internal_error(mock_get_agent, mock_get_auth_token):
|
||||
mock_get_response = MagicMock()
|
||||
mock_get_response.status_code = 500
|
||||
mock_get_response.text = "Internal server error"
|
||||
mock_get_agent.return_value = mock_get_response
|
||||
with pytest.raises(
|
||||
AgentRepositoryError,
|
||||
match="Agent test_agent could not be loaded: Internal server error",
|
||||
):
|
||||
Agent(from_repository="test_agent")
|
||||
|
||||
|
||||
@patch("crewai.cli.plus_api.PlusAPI.get_agent")
|
||||
def test_agent_from_repository_agent_not_found(mock_get_agent, mock_get_auth_token):
|
||||
mock_get_response = MagicMock()
|
||||
@@ -2104,6 +2117,6 @@ def test_agent_from_repository_agent_not_found(mock_get_agent, mock_get_auth_tok
|
||||
mock_get_agent.return_value = mock_get_response
|
||||
with pytest.raises(
|
||||
AgentRepositoryError,
|
||||
match="Agent NOT_FOUND could not be loaded: Agent not found",
|
||||
match="Agent test_agent does not exist, make sure the name is correct or the agent is available on your organization",
|
||||
):
|
||||
Agent(from_repository="NOT_FOUND")
|
||||
Agent(from_repository="test_agent")
|
||||
|
||||
227
tests/cassettes/test_inject_date.yaml
Normal file
232
tests/cassettes/test_inject_date_custom_format.yaml
Normal file
226
tests/cassettes/test_no_inject_date.yaml
Normal file
121
tests/cassettes/test_task_interpolation_with_hyphens.yaml
Normal file
@@ -0,0 +1,121 @@
|
||||
interactions:
|
||||
- request:
|
||||
body: '{"messages": [{"role": "system", "content": "You are Researcher. You''re
|
||||
an expert researcher, specialized in technology, software engineering, AI and
|
||||
startups. You work as a freelancer and is now working on doing research and
|
||||
analysis for a new customer.\nYour personal goal is: be an assistant that responds
|
||||
with say hello world\nTo give my best complete final answer to the task respond
|
||||
using the exact following format:\n\nThought: I now can give a great answer\nFinal
|
||||
Answer: Your final answer must be the great and the most complete as possible,
|
||||
it must be outcome described.\n\nI MUST use these formats, my job depends on
|
||||
it!"}, {"role": "user", "content": "\nCurrent Task: be an assistant that responds
|
||||
with say hello world\n\nThis is the expected criteria for your final answer:
|
||||
The response should be addressing: say hello world\nyou MUST return the actual
|
||||
complete content as the final answer, not a summary.\n\nBegin! This is VERY
|
||||
important to you, use the tools available and give your best Final Answer, your
|
||||
job depends on it!\n\nThought:"}], "model": "gpt-4o-mini", "stop": ["\nObservation:"]}'
|
||||
headers:
|
||||
accept:
|
||||
- application/json
|
||||
accept-encoding:
|
||||
- gzip, deflate, zstd
|
||||
connection:
|
||||
- keep-alive
|
||||
content-length:
|
||||
- '1108'
|
||||
content-type:
|
||||
- application/json
|
||||
host:
|
||||
- api.openai.com
|
||||
user-agent:
|
||||
- OpenAI/Python 1.68.2
|
||||
x-stainless-arch:
|
||||
- arm64
|
||||
x-stainless-async:
|
||||
- 'false'
|
||||
x-stainless-lang:
|
||||
- python
|
||||
x-stainless-os:
|
||||
- MacOS
|
||||
x-stainless-package-version:
|
||||
- 1.68.2
|
||||
x-stainless-raw-response:
|
||||
- 'true'
|
||||
x-stainless-read-timeout:
|
||||
- '600.0'
|
||||
x-stainless-retry-count:
|
||||
- '0'
|
||||
x-stainless-runtime:
|
||||
- CPython
|
||||
x-stainless-runtime-version:
|
||||
- 3.12.9
|
||||
method: POST
|
||||
uri: https://api.openai.com/v1/chat/completions
|
||||
response:
|
||||
body:
|
||||
string: !!binary |
|
||||
H4sIAAAAAAAAA4xSTW/UMBC951cMPicoScMu3RuIooUDcOOrVeS1J4mp4zG2sy2q9r9XTrqbtBSJ
|
||||
iyX7zXt+b2buEgCmJNsAEx0Porc6e/vt4nf3xVxweVZ+3v/Q17fF9+pjs92+O//0iqWRQbtfKMKR
|
||||
9VJQbzUGRWaChUMeMKoW62pdropVfjYCPUnUkdbakFWU9cqorMzLKsvXWfH6gd2REujZBn4mAAB3
|
||||
4xl9Gom3bAN5enzp0XveItucigCYIx1fGPde+cBNYOkMCjIBzWj9Axi6AcENtGqPwKGNtoEbf4MO
|
||||
4NK8V4ZreDPeN7BFrSmFr+S0fLGUdNgMnsdYZtB6AXBjKPDYljHM1QNyONnX1FpHO/+EyhpllO9q
|
||||
h9yTiVZ9IMtG9JAAXI1tGh4lZ9ZRb0Md6BrH78p8NemxeTozWhzBQIHrBass02f0aomBK+0XjWaC
|
||||
iw7lTJ2nwgepaAEki9R/u3lOe0quTPs/8jMgBNqAsrYOpRKPE89lDuPy/qvs1OXRMPPo9kpgHRS6
|
||||
OAmJDR/0tFLM//EB+7pRpkVnnZr2qrH1utjl5bo6bzhLDsk9AAAA//8DAAxaM/dlAwAA
|
||||
headers:
|
||||
CF-RAY:
|
||||
- 93fdd19cdbfb6428-SJC
|
||||
Connection:
|
||||
- keep-alive
|
||||
Content-Encoding:
|
||||
- gzip
|
||||
Content-Type:
|
||||
- application/json
|
||||
Date:
|
||||
- Wed, 14 May 2025 22:26:43 GMT
|
||||
Server:
|
||||
- cloudflare
|
||||
Set-Cookie:
|
||||
- __cf_bm=eCtOgOCsKt_ybdNPdtFAocCmuQbNltR52chaHVe7Y_Q-1747261603-1.0.1.1-827eoA7wHS5SOkTsTqoMq6OSioi0VznQBVjvmabNSVX1bf5PpWZvblw58iggZ_wyKDB0EuVoeLKFspgBJa0kuQYR17hu43Y2C14sgdvOXIE;
|
||||
path=/; expires=Wed, 14-May-25 22:56:43 GMT; domain=.api.openai.com; HttpOnly;
|
||||
Secure; SameSite=None
|
||||
- _cfuvid=QUa5MnypdaVxO826bwdQaN4G6CBEV8HYVV.7OLF.qvQ-1747261603742-0.0.1.1-604800000;
|
||||
path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None
|
||||
Transfer-Encoding:
|
||||
- chunked
|
||||
X-Content-Type-Options:
|
||||
- nosniff
|
||||
access-control-expose-headers:
|
||||
- X-Request-ID
|
||||
alt-svc:
|
||||
- h3=":443"; ma=86400
|
||||
cf-cache-status:
|
||||
- DYNAMIC
|
||||
openai-organization:
|
||||
- crewai-iuxna1
|
||||
openai-processing-ms:
|
||||
- '307'
|
||||
openai-version:
|
||||
- '2020-10-01'
|
||||
strict-transport-security:
|
||||
- max-age=31536000; includeSubDomains; preload
|
||||
x-envoy-upstream-service-time:
|
||||
- '309'
|
||||
x-ratelimit-limit-requests:
|
||||
- '30000'
|
||||
x-ratelimit-limit-tokens:
|
||||
- '150000000'
|
||||
x-ratelimit-remaining-requests:
|
||||
- '29999'
|
||||
x-ratelimit-remaining-tokens:
|
||||
- '149999757'
|
||||
x-ratelimit-reset-requests:
|
||||
- 2ms
|
||||
x-ratelimit-reset-tokens:
|
||||
- 0s
|
||||
x-request-id:
|
||||
- req_61d9066e0258b7095517f9f9c01d38e9
|
||||
status:
|
||||
code: 200
|
||||
message: OK
|
||||
version: 1
|
||||
@@ -78,4 +78,134 @@ interactions:
|
||||
status:
|
||||
code: 200
|
||||
message: OK
|
||||
- request:
|
||||
body: '{"messages": [{"role": "system", "content": "You are Researcher. You''re
|
||||
an expert researcher, specialized in technology, software engineering, AI and
|
||||
startups. You work as a freelancer and are now working on doing research and
|
||||
analysis for a new customer.\nYour personal goal is: Make the best research
|
||||
and analysis on content about AI and AI agents. Use the tool provided to you.\nYou
|
||||
ONLY have access to the following tools, and should NEVER make up tools that
|
||||
are not listed here:\n\nTool Name: what amazing tool\nTool Arguments: {}\nTool
|
||||
Description: My tool\n\nIMPORTANT: Use the following format in your response:\n\n```\nThought:
|
||||
you should always think about what to do\nAction: the action to take, only one
|
||||
name of [what amazing tool], just the name, exactly as it''s written.\nAction
|
||||
Input: the input to the action, just a simple JSON object, enclosed in curly
|
||||
braces, using \" to wrap keys and values.\nObservation: the result of the action\n```\n\nOnce
|
||||
all necessary information is gathered, return the following format:\n\n```\nThought:
|
||||
I now know the final answer\nFinal Answer: the final answer to the original
|
||||
input question\n```"}, {"role": "user", "content": "\nCurrent Task: Give me
|
||||
a list of 5 interesting ideas to explore for an article, what makes them unique
|
||||
and interesting.\n\nThis is the expected criteria for your final answer: Bullet
|
||||
point list of 5 interesting ideas.\nyou MUST return the actual complete content
|
||||
as the final answer, not a summary.\n\nBegin! This is VERY important to you,
|
||||
use the tools available and give your best Final Answer, your job depends on
|
||||
it!\n\nThought:"}], "model": "gpt-4o-mini", "stop": ["\nObservation:"]}'
|
||||
headers:
|
||||
accept:
|
||||
- application/json
|
||||
accept-encoding:
|
||||
- gzip, deflate
|
||||
connection:
|
||||
- keep-alive
|
||||
content-length:
|
||||
- '1666'
|
||||
content-type:
|
||||
- application/json
|
||||
cookie:
|
||||
- _cfuvid=CW_cKQGYWY3cL.S6Xo5z0cmkmWHy5Q50OA_KjPEijNk-1735926034530-0.0.1.1-604800000
|
||||
host:
|
||||
- api.openai.com
|
||||
user-agent:
|
||||
- OpenAI/Python 1.68.2
|
||||
x-stainless-arch:
|
||||
- arm64
|
||||
x-stainless-async:
|
||||
- 'false'
|
||||
x-stainless-lang:
|
||||
- python
|
||||
x-stainless-os:
|
||||
- MacOS
|
||||
x-stainless-package-version:
|
||||
- 1.68.2
|
||||
x-stainless-raw-response:
|
||||
- 'true'
|
||||
x-stainless-read-timeout:
|
||||
- '600.0'
|
||||
x-stainless-retry-count:
|
||||
- '0'
|
||||
x-stainless-runtime:
|
||||
- CPython
|
||||
x-stainless-runtime-version:
|
||||
- 3.11.7
|
||||
method: POST
|
||||
uri: https://api.openai.com/v1/chat/completions
|
||||
response:
|
||||
body:
|
||||
string: !!binary |
|
||||
H4sIAAAAAAAAAwAAAP//jJNPb9swDMXv+RSELrskRZukSZNbBxRbUWwYdhmwtTAYiba5ypQn0f2z
|
||||
It+9kNPW6dYBu9iAfnzP5BP9MAIw7MwajK1RbdP6yfvvG7r79PnDxYK/LPni5uO35uxstfp6fTvz
|
||||
1oyzImx+ktVn1YENTetJOcgO20iolF2PlvPlyfR4NZ/1oAmOfJZVrU7mYdKw8GR6OJ1PDpeTo5Mn
|
||||
dR3YUjJr+DECAHjon7lPcXRn1nA4fj5pKCWsyKxfigBMDD6fGEyJk6KoGQ/QBlGSvvVzECIHGqAi
|
||||
oYhKgOA5KYQSWJQiJWWpAKOy9QTsCBNE8nm4rDs9BxTXvyoSTWMog+1S1gQBrYkjdMK/OhJKqa+N
|
||||
5OkGxRKwgAaH9+8SKNlagg8VW/TgUVyy2NLBpVzKqc25ruG2RgVs8Hd21xA8wDOEc2k7XcPDFmB/
|
||||
1khllzDnLZ33ewBFgmKW9ilfPZHtS64+VG0Mm/SH1JQsnOoiEqYgOcOkoTU93Y4Arvr7615diWlj
|
||||
aFotNFxT/7nZdLHzM8PaDHR+9AQ1KPo91WI5fsOvcKTIPu1tgLFoa3KDdFgX7ByHPTDam/rvbt7y
|
||||
3k3OUv2P/QCspVbJFW0kx/b1xENZpPxX/avsJeW+YZMo3rClQplivglHJXZ+t+sm3SelpihZKopt
|
||||
5N3Cl21xPKfNfOMWq5kZbUePAAAA//8DALOFcXD+AwAA
|
||||
headers:
|
||||
CF-RAY:
|
||||
- 9433a372ec1069e6-LAS
|
||||
Connection:
|
||||
- keep-alive
|
||||
Content-Encoding:
|
||||
- gzip
|
||||
Content-Type:
|
||||
- application/json
|
||||
Date:
|
||||
- Wed, 21 May 2025 11:12:24 GMT
|
||||
Server:
|
||||
- cloudflare
|
||||
Set-Cookie:
|
||||
- __cf_bm=SC.7rKr584CqggyyZVMEQ5_zFD.g4Q5djrKS1Kg2ifs-1747825944-1.0.1.1-M3vY0AX_JtRWZBGWsq8v4VWUTYLoQRB5_X2WbagS7emC73L80mIv3OUlMwPOwh7ag8LdkVtbkQ_hpAdM9kVJ_wyV7mhTNCoCPLE._sZWMeI;
|
||||
path=/; expires=Wed, 21-May-25 11:42:24 GMT; domain=.api.openai.com; HttpOnly;
|
||||
Secure; SameSite=None
|
||||
- _cfuvid=LMbhtXYRu2foKMlmDSxZF0LlpAWtafPdjq_4PWulGz0-1747825944424-0.0.1.1-604800000;
|
||||
path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None
|
||||
Transfer-Encoding:
|
||||
- chunked
|
||||
X-Content-Type-Options:
|
||||
- nosniff
|
||||
access-control-expose-headers:
|
||||
- X-Request-ID
|
||||
alt-svc:
|
||||
- h3=":443"; ma=86400
|
||||
cf-cache-status:
|
||||
- DYNAMIC
|
||||
openai-organization:
|
||||
- crewai-iuxna1
|
||||
openai-processing-ms:
|
||||
- '819'
|
||||
openai-version:
|
||||
- '2020-10-01'
|
||||
strict-transport-security:
|
||||
- max-age=31536000; includeSubDomains; preload
|
||||
x-envoy-upstream-service-time:
|
||||
- '827'
|
||||
x-ratelimit-limit-requests:
|
||||
- '30000'
|
||||
x-ratelimit-limit-tokens:
|
||||
- '150000000'
|
||||
x-ratelimit-remaining-requests:
|
||||
- '29999'
|
||||
x-ratelimit-remaining-tokens:
|
||||
- '149999620'
|
||||
x-ratelimit-reset-requests:
|
||||
- 2ms
|
||||
x-ratelimit-reset-tokens:
|
||||
- 0s
|
||||
x-request-id:
|
||||
- req_93e3b862779b855ab166d171911fa9e0
|
||||
status:
|
||||
code: 200
|
||||
message: OK
|
||||
version: 1
|
||||
|
||||
@@ -162,8 +162,18 @@ def test_reset_knowledge(mock_get_crews, runner):
|
||||
assert call_count == 1, "reset_memories should have been called once"
|
||||
|
||||
|
||||
def test_reset_memory_from_many_crews(mock_get_crews, runner):
|
||||
def test_reset_agent_knowledge(mock_get_crews, runner):
|
||||
result = runner.invoke(reset_memories, ["--agent-knowledge"])
|
||||
call_count = 0
|
||||
for crew in mock_get_crews.return_value:
|
||||
crew.reset_memories.assert_called_once_with(command_type="agent_knowledge")
|
||||
assert f"[Crew ({crew.name})] Agents knowledge has been reset." in result.output
|
||||
call_count += 1
|
||||
|
||||
assert call_count == 1, "reset_memories should have been called once"
|
||||
|
||||
|
||||
def test_reset_memory_from_many_crews(mock_get_crews, runner):
|
||||
crews = []
|
||||
for crew_id in ["id-1234", "id-5678"]:
|
||||
mock_crew = mock.Mock(spec=Crew)
|
||||
|
||||
@@ -46,4 +46,6 @@ def setup_test_environment():
|
||||
def vcr_config(request) -> dict:
|
||||
return {
|
||||
"cassette_library_dir": "tests/cassettes",
|
||||
"record_mode": "new_episodes",
|
||||
"filter_headers": [("authorization", "AUTHORIZATION-XXX")],
|
||||
}
|
||||
|
||||
@@ -2,22 +2,19 @@
|
||||
|
||||
import hashlib
|
||||
import json
|
||||
import os
|
||||
import tempfile
|
||||
from concurrent.futures import Future
|
||||
from unittest import mock
|
||||
from unittest.mock import MagicMock, patch
|
||||
from unittest.mock import ANY, MagicMock, patch
|
||||
|
||||
import pydantic_core
|
||||
import pytest
|
||||
|
||||
from crewai.agent import Agent
|
||||
from crewai.agents import CacheHandler
|
||||
from crewai.agents.cache import CacheHandler
|
||||
from crewai.agents.crew_agent_executor import CrewAgentExecutor
|
||||
from crewai.crew import Crew
|
||||
from crewai.crews.crew_output import CrewOutput
|
||||
from crewai.flow import Flow, listen, start
|
||||
from crewai.flow import Flow, start
|
||||
from crewai.knowledge.knowledge import Knowledge
|
||||
from crewai.knowledge.source.string_knowledge_source import StringKnowledgeSource
|
||||
from crewai.llm import LLM
|
||||
from crewai.memory.contextual.contextual_memory import ContextualMemory
|
||||
@@ -3141,6 +3138,30 @@ def test_replay_with_context():
|
||||
assert crew.tasks[1].context[0].output.raw == "context raw output"
|
||||
|
||||
|
||||
def test_replay_with_context_set_to_nullable():
|
||||
agent = Agent(role="test_agent", backstory="Test Description", goal="Test Goal")
|
||||
task1 = Task(
|
||||
description="Context Task", expected_output="Say Task Output", agent=agent
|
||||
)
|
||||
task2 = Task(
|
||||
description="Test Task", expected_output="Say Hi", agent=agent, context=[]
|
||||
)
|
||||
task3 = Task(
|
||||
description="Test Task 3", expected_output="Say Hi", agent=agent, context=None
|
||||
)
|
||||
|
||||
crew = Crew(agents=[agent], tasks=[task1, task2, task3], process=Process.sequential)
|
||||
with patch("crewai.task.Task.execute_sync") as mock_execute_task:
|
||||
mock_execute_task.return_value = TaskOutput(
|
||||
description="Test Task Output",
|
||||
raw="test raw output",
|
||||
agent="test_agent",
|
||||
)
|
||||
crew.kickoff()
|
||||
|
||||
mock_execute_task.assert_called_with(agent=ANY, context="", tools=ANY)
|
||||
|
||||
|
||||
@pytest.mark.vcr(filter_headers=["authorization"])
|
||||
def test_replay_with_invalid_task_id():
|
||||
agent = Agent(role="test_agent", backstory="Test Description", goal="Test Goal")
|
||||
@@ -4383,3 +4404,165 @@ def test_sets_parent_flow_when_inside_flow(researcher, writer):
|
||||
flow = MyFlow()
|
||||
result = flow.kickoff()
|
||||
assert result.parent_flow is flow
|
||||
|
||||
|
||||
def test_reset_knowledge_with_no_crew_knowledge(researcher,writer):
|
||||
crew = Crew(
|
||||
agents=[researcher, writer],
|
||||
process=Process.sequential,
|
||||
tasks=[
|
||||
Task(description="Task 1", expected_output="output", agent=researcher),
|
||||
Task(description="Task 2", expected_output="output", agent=writer),
|
||||
]
|
||||
)
|
||||
|
||||
with pytest.raises(RuntimeError) as excinfo:
|
||||
crew.reset_memories(command_type='knowledge')
|
||||
|
||||
# Optionally, you can also check the error message
|
||||
assert "Crew Knowledge and Agent Knowledge memory system is not initialized" in str(excinfo.value) # Replace with the expected message
|
||||
|
||||
|
||||
def test_reset_knowledge_with_only_crew_knowledge(researcher,writer):
|
||||
mock_ks = MagicMock(spec=Knowledge)
|
||||
|
||||
with patch.object(Crew,'reset_knowledge') as mock_reset_agent_knowledge:
|
||||
crew = Crew(
|
||||
agents=[researcher, writer],
|
||||
process=Process.sequential,
|
||||
tasks=[
|
||||
Task(description="Task 1", expected_output="output", agent=researcher),
|
||||
Task(description="Task 2", expected_output="output", agent=writer),
|
||||
],
|
||||
knowledge=mock_ks
|
||||
)
|
||||
|
||||
crew.reset_memories(command_type='knowledge')
|
||||
mock_reset_agent_knowledge.assert_called_once_with([mock_ks])
|
||||
|
||||
|
||||
def test_reset_knowledge_with_crew_and_agent_knowledge(researcher,writer):
|
||||
mock_ks_crew = MagicMock(spec=Knowledge)
|
||||
mock_ks_research = MagicMock(spec=Knowledge)
|
||||
mock_ks_writer = MagicMock(spec=Knowledge)
|
||||
|
||||
researcher.knowledge = mock_ks_research
|
||||
writer.knowledge = mock_ks_writer
|
||||
|
||||
with patch.object(Crew,'reset_knowledge') as mock_reset_agent_knowledge:
|
||||
crew = Crew(
|
||||
agents=[researcher, writer],
|
||||
process=Process.sequential,
|
||||
tasks=[
|
||||
Task(description="Task 1", expected_output="output", agent=researcher),
|
||||
Task(description="Task 2", expected_output="output", agent=writer),
|
||||
],
|
||||
knowledge=mock_ks_crew
|
||||
)
|
||||
|
||||
crew.reset_memories(command_type='knowledge')
|
||||
mock_reset_agent_knowledge.assert_called_once_with([mock_ks_crew,mock_ks_research,mock_ks_writer])
|
||||
|
||||
|
||||
def test_reset_knowledge_with_only_agent_knowledge(researcher,writer):
|
||||
mock_ks_research = MagicMock(spec=Knowledge)
|
||||
mock_ks_writer = MagicMock(spec=Knowledge)
|
||||
|
||||
researcher.knowledge = mock_ks_research
|
||||
writer.knowledge = mock_ks_writer
|
||||
|
||||
with patch.object(Crew,'reset_knowledge') as mock_reset_agent_knowledge:
|
||||
crew = Crew(
|
||||
agents=[researcher, writer],
|
||||
process=Process.sequential,
|
||||
tasks=[
|
||||
Task(description="Task 1", expected_output="output", agent=researcher),
|
||||
Task(description="Task 2", expected_output="output", agent=writer),
|
||||
],
|
||||
)
|
||||
|
||||
crew.reset_memories(command_type='knowledge')
|
||||
mock_reset_agent_knowledge.assert_called_once_with([mock_ks_research,mock_ks_writer])
|
||||
|
||||
|
||||
def test_reset_agent_knowledge_with_no_agent_knowledge(researcher,writer):
|
||||
crew = Crew(
|
||||
agents=[researcher, writer],
|
||||
process=Process.sequential,
|
||||
tasks=[
|
||||
Task(description="Task 1", expected_output="output", agent=researcher),
|
||||
Task(description="Task 2", expected_output="output", agent=writer),
|
||||
],
|
||||
)
|
||||
|
||||
with pytest.raises(RuntimeError) as excinfo:
|
||||
crew.reset_memories(command_type='agent_knowledge')
|
||||
|
||||
# Optionally, you can also check the error message
|
||||
assert "Agent Knowledge memory system is not initialized" in str(excinfo.value) # Replace with the expected message
|
||||
|
||||
|
||||
def test_reset_agent_knowledge_with_only_crew_knowledge(researcher,writer):
|
||||
mock_ks = MagicMock(spec=Knowledge)
|
||||
|
||||
crew = Crew(
|
||||
agents=[researcher, writer],
|
||||
process=Process.sequential,
|
||||
tasks=[
|
||||
Task(description="Task 1", expected_output="output", agent=researcher),
|
||||
Task(description="Task 2", expected_output="output", agent=writer),
|
||||
],
|
||||
knowledge=mock_ks
|
||||
)
|
||||
|
||||
with pytest.raises(RuntimeError) as excinfo:
|
||||
crew.reset_memories(command_type='agent_knowledge')
|
||||
|
||||
# Optionally, you can also check the error message
|
||||
assert "Agent Knowledge memory system is not initialized" in str(excinfo.value) # Replace with the expected message
|
||||
|
||||
|
||||
def test_reset_agent_knowledge_with_crew_and_agent_knowledge(researcher,writer):
|
||||
mock_ks_crew = MagicMock(spec=Knowledge)
|
||||
mock_ks_research = MagicMock(spec=Knowledge)
|
||||
mock_ks_writer = MagicMock(spec=Knowledge)
|
||||
|
||||
researcher.knowledge = mock_ks_research
|
||||
writer.knowledge = mock_ks_writer
|
||||
|
||||
with patch.object(Crew,'reset_knowledge') as mock_reset_agent_knowledge:
|
||||
crew = Crew(
|
||||
agents=[researcher, writer],
|
||||
process=Process.sequential,
|
||||
tasks=[
|
||||
Task(description="Task 1", expected_output="output", agent=researcher),
|
||||
Task(description="Task 2", expected_output="output", agent=writer),
|
||||
],
|
||||
knowledge=mock_ks_crew
|
||||
)
|
||||
|
||||
crew.reset_memories(command_type='agent_knowledge')
|
||||
mock_reset_agent_knowledge.assert_called_once_with([mock_ks_research,mock_ks_writer])
|
||||
|
||||
|
||||
def test_reset_agent_knowledge_with_only_agent_knowledge(researcher,writer):
|
||||
mock_ks_research = MagicMock(spec=Knowledge)
|
||||
mock_ks_writer = MagicMock(spec=Knowledge)
|
||||
|
||||
researcher.knowledge = mock_ks_research
|
||||
writer.knowledge = mock_ks_writer
|
||||
|
||||
with patch.object(Crew,'reset_knowledge') as mock_reset_agent_knowledge:
|
||||
crew = Crew(
|
||||
agents=[researcher, writer],
|
||||
process=Process.sequential,
|
||||
tasks=[
|
||||
Task(description="Task 1", expected_output="output", agent=researcher),
|
||||
Task(description="Task 2", expected_output="output", agent=writer),
|
||||
],
|
||||
)
|
||||
|
||||
crew.reset_memories(command_type='agent_knowledge')
|
||||
mock_reset_agent_knowledge.assert_called_once_with([mock_ks_research,mock_ks_writer])
|
||||
|
||||
|
||||
|
||||
@@ -398,6 +398,79 @@ def test_output_json_hierarchical():
|
||||
assert result.json == '{"score": 4}'
|
||||
assert result.to_dict() == {"score": 4}
|
||||
|
||||
@pytest.mark.vcr(filter_headers=["authorization"])
|
||||
def test_inject_date():
|
||||
reporter = Agent(
|
||||
role="Reporter",
|
||||
goal="Report the date",
|
||||
backstory="You're an expert reporter, specialized in reporting the date.",
|
||||
allow_delegation=False,
|
||||
inject_date=True,
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="What is the date today?",
|
||||
expected_output="The date today as you were told, same format as the date you were told.",
|
||||
agent=reporter,
|
||||
)
|
||||
|
||||
crew = Crew(
|
||||
agents=[reporter],
|
||||
tasks=[task],
|
||||
process=Process.sequential,
|
||||
)
|
||||
result = crew.kickoff()
|
||||
assert "2025-05-21" in result.raw
|
||||
|
||||
@pytest.mark.vcr(filter_headers=["authorization"])
|
||||
def test_inject_date_custom_format():
|
||||
reporter = Agent(
|
||||
role="Reporter",
|
||||
goal="Report the date",
|
||||
backstory="You're an expert reporter, specialized in reporting the date.",
|
||||
allow_delegation=False,
|
||||
inject_date=True,
|
||||
date_format="%B %d, %Y",
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="What is the date today?",
|
||||
expected_output="The date today.",
|
||||
agent=reporter,
|
||||
)
|
||||
|
||||
crew = Crew(
|
||||
agents=[reporter],
|
||||
tasks=[task],
|
||||
process=Process.sequential,
|
||||
)
|
||||
result = crew.kickoff()
|
||||
assert "May 21, 2025" in result.raw
|
||||
|
||||
@pytest.mark.vcr(filter_headers=["authorization"])
|
||||
def test_no_inject_date():
|
||||
reporter = Agent(
|
||||
role="Reporter",
|
||||
goal="Report the date",
|
||||
backstory="You're an expert reporter, specialized in reporting the date.",
|
||||
allow_delegation=False,
|
||||
inject_date=False,
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="What is the date today?",
|
||||
expected_output="The date today.",
|
||||
agent=reporter,
|
||||
)
|
||||
|
||||
crew = Crew(
|
||||
agents=[reporter],
|
||||
tasks=[task],
|
||||
process=Process.sequential,
|
||||
)
|
||||
result = crew.kickoff()
|
||||
assert "2025-05-21" not in result.raw
|
||||
|
||||
|
||||
@pytest.mark.vcr(filter_headers=["authorization"])
|
||||
def test_json_property_without_output_json():
|
||||
@@ -837,9 +910,6 @@ def test_interpolate_inputs():
|
||||
|
||||
def test_interpolate_only():
|
||||
"""Test the interpolate_only method for various scenarios including JSON structure preservation."""
|
||||
task = Task(
|
||||
description="Unused in this test", expected_output="Unused in this test"
|
||||
)
|
||||
|
||||
# Test JSON structure preservation
|
||||
json_string = '{"info": "Look at {placeholder}", "nested": {"val": "{nestedVal}"}}'
|
||||
@@ -871,10 +941,6 @@ def test_interpolate_only():
|
||||
|
||||
def test_interpolate_only_with_dict_inside_expected_output():
|
||||
"""Test the interpolate_only method for various scenarios including JSON structure preservation."""
|
||||
task = Task(
|
||||
description="Unused in this test",
|
||||
expected_output="Unused in this test: {questions}",
|
||||
)
|
||||
|
||||
json_string = '{"questions": {"main_question": "What is the user\'s name?", "secondary_question": "What is the user\'s age?"}}'
|
||||
result = interpolate_only(
|
||||
@@ -1094,11 +1160,6 @@ def test_task_execution_times():
|
||||
|
||||
|
||||
def test_interpolate_with_list_of_strings():
|
||||
task = Task(
|
||||
description="Test list interpolation",
|
||||
expected_output="List: {items}",
|
||||
)
|
||||
|
||||
# Test simple list of strings
|
||||
input_str = "Available items: {items}"
|
||||
inputs = {"items": ["apple", "banana", "cherry"]}
|
||||
@@ -1112,11 +1173,6 @@ def test_interpolate_with_list_of_strings():
|
||||
|
||||
|
||||
def test_interpolate_with_list_of_dicts():
|
||||
task = Task(
|
||||
description="Test list of dicts interpolation",
|
||||
expected_output="People: {people}",
|
||||
)
|
||||
|
||||
input_data = {
|
||||
"people": [
|
||||
{"name": "Alice", "age": 30, "skills": ["Python", "AI"]},
|
||||
@@ -1137,11 +1193,6 @@ def test_interpolate_with_list_of_dicts():
|
||||
|
||||
|
||||
def test_interpolate_with_nested_structures():
|
||||
task = Task(
|
||||
description="Test nested structures",
|
||||
expected_output="Company: {company}",
|
||||
)
|
||||
|
||||
input_data = {
|
||||
"company": {
|
||||
"name": "TechCorp",
|
||||
@@ -1165,11 +1216,6 @@ def test_interpolate_with_nested_structures():
|
||||
|
||||
|
||||
def test_interpolate_with_special_characters():
|
||||
task = Task(
|
||||
description="Test special characters in dicts",
|
||||
expected_output="Data: {special_data}",
|
||||
)
|
||||
|
||||
input_data = {
|
||||
"special_data": {
|
||||
"quotes": """This has "double" and 'single' quotes""",
|
||||
@@ -1188,11 +1234,6 @@ def test_interpolate_with_special_characters():
|
||||
|
||||
|
||||
def test_interpolate_mixed_types():
|
||||
task = Task(
|
||||
description="Test mixed type interpolation",
|
||||
expected_output="Mixed: {data}",
|
||||
)
|
||||
|
||||
input_data = {
|
||||
"data": {
|
||||
"name": "Test Dataset",
|
||||
@@ -1214,11 +1255,6 @@ def test_interpolate_mixed_types():
|
||||
|
||||
|
||||
def test_interpolate_complex_combination():
|
||||
task = Task(
|
||||
description="Test complex combination",
|
||||
expected_output="Report: {report}",
|
||||
)
|
||||
|
||||
input_data = {
|
||||
"report": [
|
||||
{
|
||||
@@ -1243,11 +1279,6 @@ def test_interpolate_complex_combination():
|
||||
|
||||
|
||||
def test_interpolate_invalid_type_validation():
|
||||
task = Task(
|
||||
description="Test invalid type validation",
|
||||
expected_output="Should never reach here",
|
||||
)
|
||||
|
||||
# Test with invalid top-level type
|
||||
with pytest.raises(ValueError) as excinfo:
|
||||
interpolate_only("{data}", {"data": set()}) # type: ignore we are purposely testing this failure
|
||||
@@ -1268,11 +1299,6 @@ def test_interpolate_invalid_type_validation():
|
||||
|
||||
|
||||
def test_interpolate_custom_object_validation():
|
||||
task = Task(
|
||||
description="Test custom object rejection",
|
||||
expected_output="Should never reach here",
|
||||
)
|
||||
|
||||
class CustomObject:
|
||||
def __init__(self, value):
|
||||
self.value = value
|
||||
@@ -1304,11 +1330,6 @@ def test_interpolate_custom_object_validation():
|
||||
|
||||
|
||||
def test_interpolate_valid_complex_types():
|
||||
task = Task(
|
||||
description="Test valid complex types",
|
||||
expected_output="Validation should pass",
|
||||
)
|
||||
|
||||
# Valid complex structure
|
||||
valid_data = {
|
||||
"name": "Valid Dataset",
|
||||
@@ -1328,11 +1349,6 @@ def test_interpolate_valid_complex_types():
|
||||
|
||||
|
||||
def test_interpolate_edge_cases():
|
||||
task = Task(
|
||||
description="Test edge cases",
|
||||
expected_output="Edge case handling",
|
||||
)
|
||||
|
||||
# Test empty dict and list
|
||||
assert interpolate_only("{}", {"data": {}}) == "{}"
|
||||
assert interpolate_only("[]", {"data": []}) == "[]"
|
||||
@@ -1347,11 +1363,6 @@ def test_interpolate_edge_cases():
|
||||
|
||||
|
||||
def test_interpolate_valid_types():
|
||||
task = Task(
|
||||
description="Test valid types including null and boolean",
|
||||
expected_output="Should pass validation",
|
||||
)
|
||||
|
||||
# Test with boolean and null values (valid JSON types)
|
||||
valid_data = {
|
||||
"name": "Test",
|
||||
@@ -1373,11 +1384,11 @@ def test_interpolate_valid_types():
|
||||
|
||||
def test_task_with_no_max_execution_time():
|
||||
researcher = Agent(
|
||||
role="Researcher",
|
||||
goal="Make the best research and analysis on content about AI and AI agents",
|
||||
backstory="You're an expert researcher, specialized in technology, software engineering, AI and startups. You work as a freelancer and is now working on doing research and analysis for a new customer.",
|
||||
allow_delegation=False,
|
||||
max_execution_time=None
|
||||
role="Researcher",
|
||||
goal="Make the best research and analysis on content about AI and AI agents",
|
||||
backstory="You're an expert researcher, specialized in technology, software engineering, AI and startups. You work as a freelancer and is now working on doing research and analysis for a new customer.",
|
||||
allow_delegation=False,
|
||||
max_execution_time=None,
|
||||
)
|
||||
|
||||
task = Task(
|
||||
@@ -1386,7 +1397,7 @@ def test_task_with_no_max_execution_time():
|
||||
agent=researcher,
|
||||
)
|
||||
|
||||
with patch.object(Agent, "_execute_without_timeout", return_value = "ok") as execute:
|
||||
with patch.object(Agent, "_execute_without_timeout", return_value="ok") as execute:
|
||||
result = task.execute_sync(agent=researcher)
|
||||
assert result.raw == "ok"
|
||||
execute.assert_called_once()
|
||||
@@ -1395,6 +1406,7 @@ def test_task_with_no_max_execution_time():
|
||||
@pytest.mark.vcr(filter_headers=["authorization"])
|
||||
def test_task_with_max_execution_time():
|
||||
from crewai.tools import tool
|
||||
|
||||
"""Test that execution raises TimeoutError when max_execution_time is exceeded."""
|
||||
|
||||
@tool("what amazing tool", result_as_answer=True)
|
||||
@@ -1412,7 +1424,7 @@ def test_task_with_max_execution_time():
|
||||
),
|
||||
allow_delegation=False,
|
||||
tools=[my_tool],
|
||||
max_execution_time=4
|
||||
max_execution_time=4,
|
||||
)
|
||||
|
||||
task = Task(
|
||||
@@ -1428,6 +1440,7 @@ def test_task_with_max_execution_time():
|
||||
@pytest.mark.vcr(filter_headers=["authorization"])
|
||||
def test_task_with_max_execution_time_exceeded():
|
||||
from crewai.tools import tool
|
||||
|
||||
"""Test that execution raises TimeoutError when max_execution_time is exceeded."""
|
||||
|
||||
@tool("what amazing tool", result_as_answer=True)
|
||||
@@ -1445,7 +1458,7 @@ def test_task_with_max_execution_time_exceeded():
|
||||
),
|
||||
allow_delegation=False,
|
||||
tools=[my_tool],
|
||||
max_execution_time=1
|
||||
max_execution_time=1,
|
||||
)
|
||||
|
||||
task = Task(
|
||||
@@ -1455,4 +1468,28 @@ def test_task_with_max_execution_time_exceeded():
|
||||
)
|
||||
|
||||
with pytest.raises(TimeoutError):
|
||||
task.execute_sync(agent=researcher)
|
||||
task.execute_sync(agent=researcher)
|
||||
|
||||
|
||||
@pytest.mark.vcr(filter_headers=["authorization"])
|
||||
def test_task_interpolation_with_hyphens():
|
||||
agent = Agent(
|
||||
role="Researcher",
|
||||
goal="be an assistant that responds with {interpolation-with-hyphens}",
|
||||
backstory="You're an expert researcher, specialized in technology, software engineering, AI and startups. You work as a freelancer and is now working on doing research and analysis for a new customer.",
|
||||
allow_delegation=False,
|
||||
)
|
||||
task = Task(
|
||||
description="be an assistant that responds with {interpolation-with-hyphens}",
|
||||
expected_output="The response should be addressing: {interpolation-with-hyphens}",
|
||||
agent=agent,
|
||||
)
|
||||
crew = Crew(
|
||||
agents=[agent],
|
||||
tasks=[task],
|
||||
verbose=True,
|
||||
)
|
||||
result = crew.kickoff(inputs={"interpolation-with-hyphens": "say hello world"})
|
||||
assert "say hello world" in task.prompt()
|
||||
|
||||
assert result.raw == "Hello, World!"
|
||||
|
||||
@@ -6,6 +6,8 @@ import pytest
|
||||
from crewai import Agent, Crew, Task
|
||||
from crewai.telemetry import Telemetry
|
||||
|
||||
from opentelemetry import trace
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"env_var,value,expected_ready",
|
||||
@@ -34,9 +36,6 @@ def test_telemetry_enabled_by_default():
|
||||
assert telemetry.ready is True
|
||||
|
||||
|
||||
from opentelemetry import trace
|
||||
|
||||
|
||||
@patch("crewai.telemetry.telemetry.logger.error")
|
||||
@patch(
|
||||
"opentelemetry.exporter.otlp.proto.http.trace_exporter.OTLPSpanExporter.export",
|
||||
@@ -67,3 +66,32 @@ def test_telemetry_fails_due_connect_timeout(export_mock, logger_mock):
|
||||
|
||||
export_mock.assert_called_once()
|
||||
logger_mock.assert_called_once_with(error)
|
||||
|
||||
|
||||
def test_telemetry_singleton_pattern():
|
||||
"""Test that Telemetry uses the singleton pattern correctly."""
|
||||
Telemetry._instance = None
|
||||
|
||||
telemetry1 = Telemetry()
|
||||
telemetry2 = Telemetry()
|
||||
|
||||
assert telemetry1 is telemetry2
|
||||
|
||||
setattr(telemetry1, "test_attribute", "test_value")
|
||||
assert hasattr(telemetry2, "test_attribute")
|
||||
assert getattr(telemetry2, "test_attribute") == "test_value"
|
||||
|
||||
import threading
|
||||
|
||||
instances = []
|
||||
|
||||
def create_instance():
|
||||
instances.append(Telemetry())
|
||||
|
||||
threads = [threading.Thread(target=create_instance) for _ in range(5)]
|
||||
for thread in threads:
|
||||
thread.start()
|
||||
for thread in threads:
|
||||
thread.join()
|
||||
|
||||
assert all(instance is telemetry1 for instance in instances)
|
||||
|
||||
117
tests/test_agent_inject_date.py
Normal file
@@ -0,0 +1,117 @@
|
||||
from datetime import datetime
|
||||
from unittest.mock import patch
|
||||
|
||||
from crewai.agent import Agent
|
||||
from crewai.task import Task
|
||||
|
||||
|
||||
def test_agent_inject_date():
|
||||
"""Test that the inject_date flag injects the current date into the task.
|
||||
|
||||
Tests that when inject_date=True, the current date is added to the task description.
|
||||
"""
|
||||
with patch('datetime.datetime') as mock_datetime:
|
||||
mock_datetime.now.return_value = datetime(2025, 1, 1)
|
||||
|
||||
agent = Agent(
|
||||
role="test_agent",
|
||||
goal="test_goal",
|
||||
backstory="test_backstory",
|
||||
inject_date=True,
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Test task",
|
||||
expected_output="Test output",
|
||||
agent=agent,
|
||||
)
|
||||
|
||||
# Store original description
|
||||
original_description = task.description
|
||||
|
||||
agent._inject_date_to_task(task)
|
||||
|
||||
assert "Current Date: 2025-01-01" in task.description
|
||||
assert task.description != original_description
|
||||
|
||||
|
||||
def test_agent_without_inject_date():
|
||||
"""Test that without inject_date flag, no date is injected.
|
||||
|
||||
Tests that when inject_date=False (default), no date is added to the task description.
|
||||
"""
|
||||
agent = Agent(
|
||||
role="test_agent",
|
||||
goal="test_goal",
|
||||
backstory="test_backstory",
|
||||
# inject_date is False by default
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Test task",
|
||||
expected_output="Test output",
|
||||
agent=agent,
|
||||
)
|
||||
|
||||
original_description = task.description
|
||||
|
||||
agent._inject_date_to_task(task)
|
||||
|
||||
assert task.description == original_description
|
||||
|
||||
|
||||
def test_agent_inject_date_custom_format():
|
||||
"""Test that the inject_date flag with custom date_format works correctly.
|
||||
|
||||
Tests that when inject_date=True with a custom date_format, the date is formatted correctly.
|
||||
"""
|
||||
with patch('datetime.datetime') as mock_datetime:
|
||||
mock_datetime.now.return_value = datetime(2025, 1, 1)
|
||||
|
||||
agent = Agent(
|
||||
role="test_agent",
|
||||
goal="test_goal",
|
||||
backstory="test_backstory",
|
||||
inject_date=True,
|
||||
date_format="%d/%m/%Y",
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Test task",
|
||||
expected_output="Test output",
|
||||
agent=agent,
|
||||
)
|
||||
|
||||
# Store original description
|
||||
original_description = task.description
|
||||
|
||||
agent._inject_date_to_task(task)
|
||||
|
||||
assert "Current Date: 01/01/2025" in task.description
|
||||
assert task.description != original_description
|
||||
|
||||
|
||||
def test_agent_inject_date_invalid_format():
|
||||
"""Test error handling with invalid date format.
|
||||
|
||||
Tests that when an invalid date_format is provided, the task description remains unchanged.
|
||||
"""
|
||||
agent = Agent(
|
||||
role="test_agent",
|
||||
goal="test_goal",
|
||||
backstory="test_backstory",
|
||||
inject_date=True,
|
||||
date_format="invalid",
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Test task",
|
||||
expected_output="Test output",
|
||||
agent=agent,
|
||||
)
|
||||
|
||||
original_description = task.description
|
||||
|
||||
agent._inject_date_to_task(task)
|
||||
|
||||
assert task.description == original_description
|
||||
108
tests/test_hallucination_guardrail.py
Normal file
@@ -0,0 +1,108 @@
|
||||
from unittest.mock import Mock
|
||||
|
||||
import pytest
|
||||
|
||||
from crewai.llm import LLM
|
||||
from crewai.tasks.hallucination_guardrail import HallucinationGuardrail
|
||||
from crewai.tasks.task_output import TaskOutput
|
||||
|
||||
|
||||
def test_hallucination_guardrail_initialization():
|
||||
"""Test that the hallucination guardrail initializes correctly with all parameters."""
|
||||
mock_llm = Mock(spec=LLM)
|
||||
|
||||
guardrail = HallucinationGuardrail(context="Test reference context", llm=mock_llm)
|
||||
|
||||
assert guardrail.context == "Test reference context"
|
||||
assert guardrail.llm == mock_llm
|
||||
assert guardrail.threshold is None
|
||||
assert guardrail.tool_response == ""
|
||||
|
||||
guardrail = HallucinationGuardrail(
|
||||
context="Test reference context",
|
||||
llm=mock_llm,
|
||||
threshold=8.5,
|
||||
tool_response="Sample tool response",
|
||||
)
|
||||
|
||||
assert guardrail.context == "Test reference context"
|
||||
assert guardrail.llm == mock_llm
|
||||
assert guardrail.threshold == 8.5
|
||||
assert guardrail.tool_response == "Sample tool response"
|
||||
|
||||
|
||||
def test_hallucination_guardrail_no_op_behavior():
|
||||
"""Test that the guardrail always returns True in the open-source version."""
|
||||
mock_llm = Mock(spec=LLM)
|
||||
guardrail = HallucinationGuardrail(
|
||||
context="Test reference context",
|
||||
llm=mock_llm,
|
||||
threshold=9.0,
|
||||
)
|
||||
|
||||
task_output = TaskOutput(
|
||||
raw="Sample task output",
|
||||
description="Test task",
|
||||
expected_output="Expected output",
|
||||
agent="Test Agent",
|
||||
)
|
||||
|
||||
result, output = guardrail(task_output)
|
||||
|
||||
assert result is True
|
||||
assert output == "Sample task output"
|
||||
|
||||
|
||||
def test_hallucination_guardrail_description():
|
||||
"""Test that the guardrail provides the correct description for event logging."""
|
||||
guardrail = HallucinationGuardrail(
|
||||
context="Test reference context", llm=Mock(spec=LLM)
|
||||
)
|
||||
|
||||
assert guardrail.description == "HallucinationGuardrail (no-op)"
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"context,task_output_text,threshold,tool_response",
|
||||
[
|
||||
(
|
||||
"Earth orbits the Sun once every 365.25 days.",
|
||||
"It takes Earth approximately one year to go around the Sun.",
|
||||
None,
|
||||
"",
|
||||
),
|
||||
(
|
||||
"Python was created by Guido van Rossum in 1991.",
|
||||
"Python is a programming language developed by Guido van Rossum.",
|
||||
7.5,
|
||||
"",
|
||||
),
|
||||
(
|
||||
"The capital of France is Paris.",
|
||||
"Paris is the largest city and capital of France.",
|
||||
9.0,
|
||||
"Geographic API returned: France capital is Paris",
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_hallucination_guardrail_always_passes(
|
||||
context, task_output_text, threshold, tool_response
|
||||
):
|
||||
"""Test that the guardrail always passes regardless of configuration in open-source version."""
|
||||
mock_llm = Mock(spec=LLM)
|
||||
|
||||
guardrail = HallucinationGuardrail(
|
||||
context=context, llm=mock_llm, threshold=threshold, tool_response=tool_response
|
||||
)
|
||||
|
||||
task_output = TaskOutput(
|
||||
raw=task_output_text,
|
||||
description="Test task",
|
||||
expected_output="Expected output",
|
||||
agent="Test Agent",
|
||||
)
|
||||
|
||||
result, output = guardrail(task_output)
|
||||
|
||||
assert result is True
|
||||
assert output == task_output_text
|
||||
96
tests/test_markdown_task.py
Normal file
@@ -0,0 +1,96 @@
|
||||
"""Test the markdown attribute in Task class."""
|
||||
|
||||
import pytest
|
||||
from pydantic import BaseModel
|
||||
|
||||
from crewai import Agent, Task
|
||||
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"markdown_enabled,should_contain_instructions",
|
||||
[
|
||||
(True, True),
|
||||
(False, False),
|
||||
],
|
||||
)
|
||||
def test_markdown_option_in_task_prompt(markdown_enabled, should_contain_instructions):
|
||||
"""Test that markdown flag correctly controls the inclusion of markdown formatting instructions."""
|
||||
|
||||
researcher = Agent(
|
||||
role="Researcher",
|
||||
goal="Research a topic",
|
||||
backstory="You're a researcher specialized in providing well-formatted content.",
|
||||
allow_delegation=False,
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Research advances in AI in 2023",
|
||||
expected_output="A summary of key AI advances in 2023",
|
||||
markdown=markdown_enabled,
|
||||
agent=researcher,
|
||||
)
|
||||
|
||||
prompt = task.prompt()
|
||||
|
||||
assert "Research advances in AI in 2023" in prompt
|
||||
assert "A summary of key AI advances in 2023" in prompt
|
||||
|
||||
if should_contain_instructions:
|
||||
assert "Your final answer MUST be formatted in Markdown syntax." in prompt
|
||||
assert "Use # for headers" in prompt
|
||||
assert "Use ** for bold text" in prompt
|
||||
else:
|
||||
assert "Your final answer MUST be formatted in Markdown syntax." not in prompt
|
||||
|
||||
|
||||
def test_markdown_with_empty_description():
|
||||
"""Test markdown formatting with empty description."""
|
||||
|
||||
researcher = Agent(
|
||||
role="Researcher",
|
||||
goal="Research a topic",
|
||||
backstory="You're a researcher.",
|
||||
allow_delegation=False,
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="",
|
||||
expected_output="A summary",
|
||||
markdown=True,
|
||||
agent=researcher,
|
||||
)
|
||||
|
||||
prompt = task.prompt()
|
||||
|
||||
assert prompt.strip() != ""
|
||||
assert "A summary" in prompt
|
||||
assert "Your final answer MUST be formatted in Markdown syntax." in prompt
|
||||
|
||||
|
||||
def test_markdown_with_complex_output_format():
|
||||
"""Test markdown with JSON output format to ensure compatibility."""
|
||||
|
||||
class ResearchOutput(BaseModel):
|
||||
title: str
|
||||
findings: list[str]
|
||||
|
||||
researcher = Agent(
|
||||
role="Researcher",
|
||||
goal="Research a topic",
|
||||
backstory="You're a researcher.",
|
||||
allow_delegation=False,
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Research topic",
|
||||
expected_output="Research results",
|
||||
markdown=True,
|
||||
output_json=ResearchOutput,
|
||||
agent=researcher,
|
||||
)
|
||||
|
||||
prompt = task.prompt()
|
||||
|
||||
assert "Your final answer MUST be formatted in Markdown syntax." in prompt
|
||||
assert "Research topic" in prompt
|
||||
assert "Research results" in prompt
|
||||
@@ -1,9 +1,10 @@
|
||||
from unittest.mock import ANY, Mock, patch
|
||||
from unittest.mock import Mock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from crewai import Agent, Task
|
||||
from crewai.llm import LLM
|
||||
from crewai.tasks.hallucination_guardrail import HallucinationGuardrail
|
||||
from crewai.tasks.llm_guardrail import LLMGuardrail
|
||||
from crewai.tasks.task_output import TaskOutput
|
||||
from crewai.utilities.events import (
|
||||
@@ -267,3 +268,37 @@ def test_guardrail_when_an_error_occurs(sample_agent, task_output):
|
||||
max_retries=0,
|
||||
)
|
||||
task.execute_sync(agent=sample_agent)
|
||||
|
||||
|
||||
def test_hallucination_guardrail_integration():
|
||||
"""Test that HallucinationGuardrail integrates properly with the task system."""
|
||||
agent = Mock()
|
||||
agent.role = "test_agent"
|
||||
agent.execute_task.return_value = "test result"
|
||||
agent.crew = None
|
||||
|
||||
mock_llm = Mock(spec=LLM)
|
||||
guardrail = HallucinationGuardrail(
|
||||
context="Test reference context for validation", llm=mock_llm, threshold=8.0
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Test task with hallucination guardrail",
|
||||
expected_output="Valid output",
|
||||
guardrail=guardrail,
|
||||
)
|
||||
|
||||
result = task.execute_sync(agent=agent)
|
||||
assert isinstance(result, TaskOutput)
|
||||
assert result.raw == "test result"
|
||||
|
||||
|
||||
def test_hallucination_guardrail_description_in_events():
|
||||
"""Test that HallucinationGuardrail description appears correctly in events."""
|
||||
mock_llm = Mock(spec=LLM)
|
||||
guardrail = HallucinationGuardrail(context="Test context", llm=mock_llm)
|
||||
|
||||
assert guardrail.description == "HallucinationGuardrail (no-op)"
|
||||
|
||||
event = LLMGuardrailStartedEvent(guardrail=guardrail, retry_count=0)
|
||||
assert event.guardrail == "HallucinationGuardrail (no-op)"
|
||||
|
||||
@@ -1,5 +1,4 @@
|
||||
import json
|
||||
import os
|
||||
from typing import Dict, List, Optional
|
||||
from unittest.mock import MagicMock, Mock, patch
|
||||
|
||||
@@ -19,6 +18,8 @@ from crewai.utilities.converter import (
|
||||
validate_model,
|
||||
)
|
||||
from crewai.utilities.pydantic_schema_parser import PydanticSchemaParser
|
||||
# Tests for enums
|
||||
from enum import Enum
|
||||
|
||||
|
||||
@pytest.fixture(scope="module")
|
||||
@@ -359,7 +360,7 @@ def test_convert_with_instructions():
|
||||
|
||||
@pytest.mark.vcr(filter_headers=["authorization"])
|
||||
def test_converter_with_llama3_2_model():
|
||||
llm = LLM(model="ollama/llama3.2:3b", base_url="http://localhost:11434")
|
||||
llm = LLM(model="openrouter/meta-llama/llama-3.2-3b-instruct")
|
||||
sample_text = "Name: Alice Llama, Age: 30"
|
||||
instructions = get_conversion_instructions(SimpleModel, llm)
|
||||
converter = Converter(
|
||||
@@ -431,7 +432,7 @@ def test_converter_error_handling():
|
||||
)
|
||||
|
||||
with pytest.raises(ConverterError) as exc_info:
|
||||
output = converter.to_pydantic()
|
||||
converter.to_pydantic()
|
||||
|
||||
assert "Failed to convert text into a Pydantic model" in str(exc_info.value)
|
||||
|
||||
@@ -515,10 +516,6 @@ def test_converter_with_list_field():
|
||||
assert output.items == [1, 2, 3]
|
||||
|
||||
|
||||
# Tests for enums
|
||||
from enum import Enum
|
||||
|
||||
|
||||
def test_converter_with_enum():
|
||||
class Color(Enum):
|
||||
RED = "red"
|
||||
@@ -565,7 +562,7 @@ def test_converter_with_ambiguous_input():
|
||||
)
|
||||
|
||||
with pytest.raises(ConverterError) as exc_info:
|
||||
output = converter.to_pydantic()
|
||||
converter.to_pydantic()
|
||||
|
||||
assert "failed to convert text into a pydantic model" in str(exc_info.value).lower()
|
||||
|
||||
|
||||
@@ -1,4 +1,3 @@
|
||||
import os
|
||||
from datetime import datetime
|
||||
from unittest.mock import Mock, patch
|
||||
|
||||
@@ -22,6 +21,7 @@ from crewai.utilities.events.crew_events import (
|
||||
CrewKickoffFailedEvent,
|
||||
CrewKickoffStartedEvent,
|
||||
CrewTestCompletedEvent,
|
||||
CrewTestResultEvent,
|
||||
CrewTestStartedEvent,
|
||||
)
|
||||
from crewai.utilities.events.crewai_event_bus import crewai_event_bus
|
||||
@@ -38,7 +38,6 @@ from crewai.utilities.events.llm_events import (
|
||||
LLMCallCompletedEvent,
|
||||
LLMCallFailedEvent,
|
||||
LLMCallStartedEvent,
|
||||
LLMCallType,
|
||||
LLMStreamChunkEvent,
|
||||
)
|
||||
from crewai.utilities.events.task_events import (
|
||||
@@ -132,6 +131,10 @@ def test_crew_emits_test_kickoff_type_event():
|
||||
def handle_crew_test_end(source, event):
|
||||
received_events.append(event)
|
||||
|
||||
@crewai_event_bus.on(CrewTestResultEvent)
|
||||
def handle_crew_test_result(source, event):
|
||||
received_events.append(event)
|
||||
|
||||
eval_llm = LLM(model="gpt-4o-mini")
|
||||
with (
|
||||
patch.object(
|
||||
@@ -149,13 +152,16 @@ def test_crew_emits_test_kickoff_type_event():
|
||||
assert args[2] is None
|
||||
assert args[3] == eval_llm
|
||||
|
||||
assert len(received_events) == 2
|
||||
assert len(received_events) == 3
|
||||
assert received_events[0].crew_name == "TestCrew"
|
||||
assert isinstance(received_events[0].timestamp, datetime)
|
||||
assert received_events[0].type == "crew_test_started"
|
||||
assert received_events[1].crew_name == "TestCrew"
|
||||
assert isinstance(received_events[1].timestamp, datetime)
|
||||
assert received_events[1].type == "crew_test_completed"
|
||||
assert received_events[1].type == "crew_test_result"
|
||||
assert received_events[2].crew_name == "TestCrew"
|
||||
assert isinstance(received_events[2].timestamp, datetime)
|
||||
assert received_events[2].type == "crew_test_completed"
|
||||
|
||||
|
||||
@pytest.mark.vcr(filter_headers=["authorization"])
|
||||
@@ -309,7 +315,7 @@ def test_agent_emits_execution_error_event():
|
||||
) as invoke_mock:
|
||||
invoke_mock.side_effect = Exception(error_message)
|
||||
|
||||
with pytest.raises(Exception) as e:
|
||||
with pytest.raises(Exception):
|
||||
base_agent.execute_task(
|
||||
task=base_task,
|
||||
)
|
||||
|
||||
10
uv.lock
generated
@@ -738,7 +738,7 @@ wheels = [
|
||||
|
||||
[[package]]
|
||||
name = "crewai"
|
||||
version = "0.119.0"
|
||||
version = "0.121.0"
|
||||
source = { editable = "." }
|
||||
dependencies = [
|
||||
{ name = "appdirs" },
|
||||
@@ -828,7 +828,7 @@ requires-dist = [
|
||||
{ name = "blinker", specifier = ">=1.9.0" },
|
||||
{ name = "chromadb", specifier = ">=0.5.23" },
|
||||
{ name = "click", specifier = ">=8.1.7" },
|
||||
{ name = "crewai-tools", marker = "extra == 'tools'", specifier = "~=0.44.0" },
|
||||
{ name = "crewai-tools", marker = "extra == 'tools'", specifier = "~=0.45.0" },
|
||||
{ name = "docling", marker = "extra == 'docling'", specifier = ">=2.12.0" },
|
||||
{ name = "fastembed", marker = "extra == 'fastembed'", specifier = ">=0.4.1" },
|
||||
{ name = "instructor", specifier = ">=1.3.3" },
|
||||
@@ -879,7 +879,7 @@ dev = [
|
||||
|
||||
[[package]]
|
||||
name = "crewai-tools"
|
||||
version = "0.44.0"
|
||||
version = "0.45.0"
|
||||
source = { registry = "https://pypi.org/simple" }
|
||||
dependencies = [
|
||||
{ name = "chromadb" },
|
||||
@@ -894,9 +894,9 @@ dependencies = [
|
||||
{ name = "pytube" },
|
||||
{ name = "requests" },
|
||||
]
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/b8/1f/2977dc72628c1225bf5788ae22a65e5a53df384d19b197646d2c4760684e/crewai_tools-0.44.0.tar.gz", hash = "sha256:44e0c26079396503a326efdd9ff34bf369d410cbf95c362cc523db65b18f3c3a", size = 892004 }
|
||||
sdist = { url = "https://files.pythonhosted.org/packages/e9/3a/7070dcacef56702c5d83ad1a87021b1666ff1850ff80b3aa7540892406e7/crewai_tools-0.45.0.tar.gz", hash = "sha256:1b2e4eff3f928ce5fac308d6e648719a0e4718a1228ae98980aa0d74fc16bfc7", size = 909723 }
|
||||
wheels = [
|
||||
{ url = "https://files.pythonhosted.org/packages/ba/80/b91aa837d06edbb472445ea3c92d7619518894fd3049d480e5fffbf0c21b/crewai_tools-0.44.0-py3-none-any.whl", hash = "sha256:119e2365fe66ee16e18a5e8e222994b19f76bafcc8c1bb87f61609c1e39b2463", size = 583462 },
|
||||
{ url = "https://files.pythonhosted.org/packages/6e/72/db45626973027c992df75cbc7ef391f18393d631be3bceb6388c1b9f01e1/crewai_tools-0.45.0-py3-none-any.whl", hash = "sha256:9dd34e4792c075ee7a72134aedaab268e78d0e350114fd7fe2426e691c5f52a3", size = 602659 },
|
||||
]
|
||||
|
||||
[[package]]
|
||||
|
||||