Compare commits

..

6 Commits

Author SHA1 Message Date
Devin AI
9297655edc Fix lint error: remove unused pytest import
Co-Authored-By: Jo\u00E3o <joao@crewai.com>
2025-07-13 14:13:24 +00:00
Devin AI
3cd55459b9 Fix Flow initial_state BaseModel dict coercion issue #3147
- Fix Flow constructor to accept initial_state parameter
- Replace dict conversion with model.model_copy() in _create_initial_state
- Replace dict conversion with model.copy(update=...) in _initialize_state
- Add comprehensive tests covering dict method name collisions
- Preserve BaseModel structure to prevent attribute collision

Fixes issue where Pydantic BaseModel instances were coerced into dicts,
causing field names like 'items', 'keys', 'values' to be overridden by
built-in dict methods. Now BaseModel structure is preserved using
Pydantic's built-in copying methods.

Co-Authored-By: Jo\u00E3o <joao@crewai.com>
2025-07-13 14:09:32 +00:00
Vidit Ostwal
e7a5747c6b Comparing BaseLLM class instead of LLM (#3120)
Some checks failed
Notify Downstream / notify-downstream (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
* Compaing BaseLLM class instead of LLM

* Fixed test cases

* Fixed Linting Issues

* removed last line

---------

Co-authored-by: Lucas Gomide <lucaslg200@gmail.com>
2025-07-11 20:50:36 -04:00
Vidit Ostwal
eec1262d4f Fix agent knowledge (#2831)
Some checks failed
Notify Downstream / notify-downstream (push) Has been cancelled
* Added add_sources()

* Fixed the agent knowledge querying

* Added test cases

* Fixed linting issue

* Fixed logic

* Seems like a falky test case

* Minor changes

* Added knowledge attriute to the crew documentation

* Flaky test

* fixed spaces

* Flaky Test Case

* Seems like a flaky test case

---------

Co-authored-by: Lucas Gomide <lucaslg200@gmail.com>
2025-07-11 13:52:26 -04:00
Tony Kipkemboi
c6caa763d7 docs: Add guardrail attribute documentation and examples (#3139)
- Document string-based guardrails in tasks
- Add guardrail examples to YAML configuration
- Fix Python code formatting in PT-BR CLI docs
2025-07-11 13:32:59 -04:00
Lucas Gomide
08fa3797ca Introducing Agent evaluation (#3130)
* feat: add exchanged messages in LLMCallCompletedEvent

* feat: add GoalAlignment metric for Agent evaluation

* feat: add SemanticQuality metric for Agent evaluation

* feat: add Tool Metrics for Agent evaluation

* feat: add Reasoning Metrics for Agent evaluation, still in progress

* feat: add AgentEvaluator class

This class will evaluate Agent' results and report to user

* fix: do not evaluate Agent by default

This is a experimental feature we still need refine it further

* test: add Agent eval tests

* fix: render all feedback per iteration

* style: resolve linter issues

* style: fix mypy issues

* fix: allow messages be empty on LLMCallCompletedEvent
2025-07-11 13:18:03 -04:00
14 changed files with 1013 additions and 230 deletions

View File

@@ -32,6 +32,7 @@ A crew in crewAI represents a collaborative group of agents working together to
| **Prompt File** _(optional)_ | `prompt_file` | Path to the prompt JSON file to be used for the crew. |
| **Planning** *(optional)* | `planning` | Adds planning ability to the Crew. When activated before each Crew iteration, all Crew data is sent to an AgentPlanner that will plan the tasks and this plan will be added to each task description. |
| **Planning LLM** *(optional)* | `planning_llm` | The language model used by the AgentPlanner in a planning process. |
| **Knowledge Sources** _(optional)_ | `knowledge_sources` | Knowledge sources available at the crew level, accessible to all the agents. |
<Tip>
**Crew Max RPM**: The `max_rpm` attribute sets the maximum number of requests per minute the crew can perform to avoid rate limits and will override individual agents' `max_rpm` settings if you set it.

View File

@@ -57,6 +57,7 @@ crew = Crew(
| **Output JSON** _(optional)_ | `output_json` | `Optional[Type[BaseModel]]` | A Pydantic model to structure the JSON output. |
| **Output Pydantic** _(optional)_ | `output_pydantic` | `Optional[Type[BaseModel]]` | A Pydantic model for task output. |
| **Callback** _(optional)_ | `callback` | `Optional[Any]` | Function/object to be executed after task completion. |
| **Guardrail** _(optional)_ | `guardrail` | `Optional[Union[Callable, str]]` | Function or string description to validate task output before proceeding to next task. |
## Creating Tasks
@@ -86,6 +87,7 @@ research_task:
expected_output: >
A list with 10 bullet points of the most relevant information about {topic}
agent: researcher
guardrail: ensure each bullet contains a minimum of 100 words
reporting_task:
description: >
@@ -332,9 +334,13 @@ Task guardrails provide a way to validate and transform task outputs before they
are passed to the next task. This feature helps ensure data quality and provides
feedback to agents when their output doesn't meet specific criteria.
### Using Task Guardrails
**Guardrails can be defined in two ways:**
1. **Function-based guardrails**: Python functions that implement custom validation logic
2. **String-based guardrails**: Natural language descriptions that are automatically converted to LLM-powered validation
To add a guardrail to a task, provide a validation function through the `guardrail` parameter:
### Function-Based Guardrails
To add a function-based guardrail to a task, provide a validation function through the `guardrail` parameter:
```python Code
from typing import Tuple, Union, Dict, Any
@@ -372,9 +378,82 @@ blog_task = Task(
- On success: it returns a tuple of `(bool, Any)`. For example: `(True, validated_result)`
- On Failure: it returns a tuple of `(bool, str)`. For example: `(False, "Error message explain the failure")`
### LLMGuardrail
### String-Based Guardrails
The `LLMGuardrail` class offers a robust mechanism for validating task outputs.
String-based guardrails allow you to describe validation criteria in natural language. When you provide a string instead of a function, CrewAI automatically converts it to an `LLMGuardrail` that uses an AI agent to validate the task output.
#### Using String Guardrails in Python
```python Code
from crewai import Task
# Simple string-based guardrail
blog_task = Task(
description="Write a blog post about AI",
expected_output="A blog post under 200 words",
agent=blog_agent,
guardrail="Ensure the blog post is under 200 words and includes practical examples"
)
# More complex validation criteria
research_task = Task(
description="Research AI trends for 2025",
expected_output="A comprehensive research report",
agent=research_agent,
guardrail="Ensure each finding includes a credible source and is backed by recent data from 2024-2025"
)
```
#### Using String Guardrails in YAML
```yaml
research_task:
description: Research the latest AI developments
expected_output: A list of 10 bullet points about AI
agent: researcher
guardrail: ensure each bullet contains a minimum of 100 words
validation_task:
description: Validate the research findings
expected_output: A validation report
agent: validator
guardrail: confirm all sources are from reputable publications and published within the last 2 years
```
#### How String Guardrails Work
When you provide a string guardrail, CrewAI automatically:
1. Creates an `LLMGuardrail` instance using the string as validation criteria
2. Uses the task's agent LLM to power the validation
3. Creates a temporary validation agent that checks the output against your criteria
4. Returns detailed feedback if validation fails
This approach is ideal when you want to use natural language to describe validation rules without writing custom validation functions.
### LLMGuardrail Class
The `LLMGuardrail` class is the underlying mechanism that powers string-based guardrails. You can also use it directly for more advanced control:
```python Code
from crewai import Task
from crewai.tasks.llm_guardrail import LLMGuardrail
from crewai.llm import LLM
# Create a custom LLMGuardrail with specific LLM
custom_guardrail = LLMGuardrail(
description="Ensure the response contains exactly 5 bullet points with proper citations",
llm=LLM(model="gpt-4o-mini")
)
task = Task(
description="Research AI safety measures",
expected_output="A detailed analysis with bullet points",
agent=research_agent,
guardrail=custom_guardrail
)
```
**Note**: When you use a string guardrail, CrewAI automatically creates an `LLMGuardrail` instance using your task's agent LLM. Using `LLMGuardrail` directly gives you more control over the validation process and LLM selection.
### Error Handling Best Practices
@@ -798,166 +877,7 @@ While creating and executing tasks, certain validation mechanisms are in place t
These validations help in maintaining the consistency and reliability of task executions within the crewAI framework.
## Task Guardrails
Task guardrails provide a powerful way to validate, transform, or filter task outputs before they are passed to the next task. Guardrails are optional functions that execute before the next task starts, allowing you to ensure that task outputs meet specific requirements or formats.
### Basic Usage
#### Define your own logic to validate
```python Code
from typing import Tuple, Union
from crewai import Task
def validate_json_output(result: str) -> Tuple[bool, Union[dict, str]]:
"""Validate that the output is valid JSON."""
try:
json_data = json.loads(result)
return (True, json_data)
except json.JSONDecodeError:
return (False, "Output must be valid JSON")
task = Task(
description="Generate JSON data",
expected_output="Valid JSON object",
guardrail=validate_json_output
)
```
#### Leverage a no-code approach for validation
```python Code
from crewai import Task
task = Task(
description="Generate JSON data",
expected_output="Valid JSON object",
guardrail="Ensure the response is a valid JSON object"
)
```
#### Using YAML
```yaml
research_task:
...
guardrail: make sure each bullet contains a minimum of 100 words
...
```
```python Code
@CrewBase
class InternalCrew:
agents_config = "config/agents.yaml"
tasks_config = "config/tasks.yaml"
...
@task
def research_task(self):
return Task(config=self.tasks_config["research_task"]) # type: ignore[index]
...
```
#### Use custom models for code generation
```python Code
from crewai import Task
from crewai.llm import LLM
task = Task(
description="Generate JSON data",
expected_output="Valid JSON object",
guardrail=LLMGuardrail(
description="Ensure the response is a valid JSON object",
llm=LLM(model="gpt-4o-mini"),
)
)
```
### How Guardrails Work
1. **Optional Attribute**: Guardrails are an optional attribute at the task level, allowing you to add validation only where needed.
2. **Execution Timing**: The guardrail function is executed before the next task starts, ensuring valid data flow between tasks.
3. **Return Format**: Guardrails must return a tuple of `(success, data)`:
- If `success` is `True`, `data` is the validated/transformed result
- If `success` is `False`, `data` is the error message
4. **Result Routing**:
- On success (`True`), the result is automatically passed to the next task
- On failure (`False`), the error is sent back to the agent to generate a new answer
### Common Use Cases
#### Data Format Validation
```python Code
def validate_email_format(result: str) -> Tuple[bool, Union[str, str]]:
"""Ensure the output contains a valid email address."""
import re
email_pattern = r'^[\w\.-]+@[\w\.-]+\.\w+$'
if re.match(email_pattern, result.strip()):
return (True, result.strip())
return (False, "Output must be a valid email address")
```
#### Content Filtering
```python Code
def filter_sensitive_info(result: str) -> Tuple[bool, Union[str, str]]:
"""Remove or validate sensitive information."""
sensitive_patterns = ['SSN:', 'password:', 'secret:']
for pattern in sensitive_patterns:
if pattern.lower() in result.lower():
return (False, f"Output contains sensitive information ({pattern})")
return (True, result)
```
#### Data Transformation
```python Code
def normalize_phone_number(result: str) -> Tuple[bool, Union[str, str]]:
"""Ensure phone numbers are in a consistent format."""
import re
digits = re.sub(r'\D', '', result)
if len(digits) == 10:
formatted = f"({digits[:3]}) {digits[3:6]}-{digits[6:]}"
return (True, formatted)
return (False, "Output must be a 10-digit phone number")
```
### Advanced Features
#### Chaining Multiple Validations
```python Code
def chain_validations(*validators):
"""Chain multiple validators together."""
def combined_validator(result):
for validator in validators:
success, data = validator(result)
if not success:
return (False, data)
result = data
return (True, result)
return combined_validator
# Usage
task = Task(
description="Get user contact info",
expected_output="Email and phone",
guardrail=chain_validations(
validate_email_format,
filter_sensitive_info
)
)
```
#### Custom Retry Logic
```python Code
task = Task(
description="Generate data",
expected_output="Valid data",
guardrail=validate_data,
max_retries=5 # Override default retry limit
)
```
## Creating Directories when Saving Files

View File

@@ -76,6 +76,7 @@ Exemplo:
crewai train -n 10 -f my_training_data.pkl
```
```python
# Exemplo de uso programático do comando train
n_iterations = 2
inputs = {"topic": "Treinamento CrewAI"}
@@ -89,6 +90,7 @@ try:
)
except Exception as e:
raise Exception(f"Ocorreu um erro ao treinar a crew: {e}")
```
### 4. Replay

View File

@@ -57,6 +57,7 @@ crew = Crew(
| **Saída JSON** _(opcional)_ | `output_json` | `Optional[Type[BaseModel]]` | Um modelo Pydantic para estruturar a saída em JSON. |
| **Output Pydantic** _(opcional)_ | `output_pydantic` | `Optional[Type[BaseModel]]` | Um modelo Pydantic para a saída da tarefa. |
| **Callback** _(opcional)_ | `callback` | `Optional[Any]` | Função/objeto a ser executado após a conclusão da tarefa. |
| **Guardrail** _(opcional)_ | `guardrail` | `Optional[Union[Callable, str]]` | Função ou descrição em string para validar a saída da tarefa antes de prosseguir para a próxima tarefa. |
## Criando Tarefas
@@ -86,6 +87,7 @@ research_task:
expected_output: >
Uma lista com 10 tópicos em bullet points das informações mais relevantes sobre {topic}
agent: researcher
guardrail: garanta que cada bullet point contenha no mínimo 100 palavras
reporting_task:
description: >
@@ -330,9 +332,13 @@ analysis_task = Task(
Guardrails (trilhas de proteção) de tarefas fornecem uma maneira de validar e transformar as saídas das tarefas antes que elas sejam passadas para a próxima tarefa. Esse recurso assegura a qualidade dos dados e oferece feedback aos agentes quando sua saída não atende a critérios específicos.
### Usando Guardrails em Tarefas
**Guardrails podem ser definidos de duas maneiras:**
1. **Guardrails baseados em função**: Funções Python que implementam lógica de validação customizada
2. **Guardrails baseados em string**: Descrições em linguagem natural que são automaticamente convertidas em validação baseada em LLM
Para adicionar um guardrail a uma tarefa, forneça uma função de validação por meio do parâmetro `guardrail`:
### Guardrails Baseados em Função
Para adicionar um guardrail baseado em função a uma tarefa, forneça uma função de validação por meio do parâmetro `guardrail`:
```python Code
from typing import Tuple, Union, Dict, Any
@@ -370,9 +376,82 @@ blog_task = Task(
- Em caso de sucesso: retorna uma tupla `(True, resultado_validado)`
- Em caso de falha: retorna uma tupla `(False, "mensagem de erro explicando a falha")`
### LLMGuardrail
### Guardrails Baseados em String
A classe `LLMGuardrail` oferece um mecanismo robusto para validação das saídas das tarefas.
Guardrails baseados em string permitem que você descreva critérios de validação em linguagem natural. Quando você fornece uma string em vez de uma função, o CrewAI automaticamente a converte em um `LLMGuardrail` que usa um agente de IA para validar a saída da tarefa.
#### Usando Guardrails de String em Python
```python Code
from crewai import Task
# Guardrail simples baseado em string
blog_task = Task(
description="Escreva um post de blog sobre IA",
expected_output="Um post de blog com menos de 200 palavras",
agent=blog_agent,
guardrail="Garanta que o post do blog tenha menos de 200 palavras e inclua exemplos práticos"
)
# Critérios de validação mais complexos
research_task = Task(
description="Pesquise tendências de IA para 2025",
expected_output="Um relatório abrangente de pesquisa",
agent=research_agent,
guardrail="Garanta que cada descoberta inclua uma fonte confiável e seja respaldada por dados recentes de 2024-2025"
)
```
#### Usando Guardrails de String em YAML
```yaml
research_task:
description: Pesquise os últimos desenvolvimentos em IA
expected_output: Uma lista de 10 bullet points sobre IA
agent: researcher
guardrail: garanta que cada bullet point contenha no mínimo 100 palavras
validation_task:
description: Valide os achados da pesquisa
expected_output: Um relatório de validação
agent: validator
guardrail: confirme que todas as fontes são de publicações respeitáveis e publicadas nos últimos 2 anos
```
#### Como Funcionam os Guardrails de String
Quando você fornece um guardrail de string, o CrewAI automaticamente:
1. Cria uma instância `LLMGuardrail` usando a string como critério de validação
2. Usa o LLM do agente da tarefa para alimentar a validação
3. Cria um agente temporário de validação que verifica a saída contra seus critérios
4. Retorna feedback detalhado se a validação falhar
Esta abordagem é ideal quando você quer usar linguagem natural para descrever regras de validação sem escrever funções de validação customizadas.
### Classe LLMGuardrail
A classe `LLMGuardrail` é o mecanismo subjacente que alimenta os guardrails baseados em string. Você também pode usá-la diretamente para maior controle avançado:
```python Code
from crewai import Task
from crewai.tasks.llm_guardrail import LLMGuardrail
from crewai.llm import LLM
# Crie um LLMGuardrail customizado com LLM específico
custom_guardrail = LLMGuardrail(
description="Garanta que a resposta contenha exatamente 5 bullet points com citações adequadas",
llm=LLM(model="gpt-4o-mini")
)
task = Task(
description="Pesquise medidas de segurança em IA",
expected_output="Uma análise detalhada com bullet points",
agent=research_agent,
guardrail=custom_guardrail
)
```
**Nota**: Quando você usa um guardrail de string, o CrewAI automaticamente cria uma instância `LLMGuardrail` usando o LLM do agente da sua tarefa. Usar `LLMGuardrail` diretamente lhe dá mais controle sobre o processo de validação e seleção de LLM.
### Melhores Práticas de Tratamento de Erros

View File

@@ -210,7 +210,6 @@ class Agent(BaseAgent):
sources=self.knowledge_sources,
embedder=self.embedder,
collection_name=self.role,
storage=self.knowledge_storage or None,
)
self.knowledge.add_sources()
except (TypeError, ValueError) as e:
@@ -341,7 +340,8 @@ class Agent(BaseAgent):
self.knowledge_config.model_dump() if self.knowledge_config else {}
)
if self.knowledge:
if self.knowledge or (self.crew and self.crew.knowledge):
crewai_event_bus.emit(
self,
event=KnowledgeRetrievalStartedEvent(
@@ -353,25 +353,28 @@ class Agent(BaseAgent):
task_prompt
)
if self.knowledge_search_query:
agent_knowledge_snippets = self.knowledge.query(
[self.knowledge_search_query], **knowledge_config
)
if agent_knowledge_snippets:
self.agent_knowledge_context = extract_knowledge_context(
agent_knowledge_snippets
)
if self.agent_knowledge_context:
task_prompt += self.agent_knowledge_context
if self.crew:
knowledge_snippets = self.crew.query_knowledge(
# Quering agent specific knowledge
if self.knowledge:
agent_knowledge_snippets = self.knowledge.query(
[self.knowledge_search_query], **knowledge_config
)
if knowledge_snippets:
self.crew_knowledge_context = extract_knowledge_context(
knowledge_snippets
if agent_knowledge_snippets:
self.agent_knowledge_context = extract_knowledge_context(
agent_knowledge_snippets
)
if self.crew_knowledge_context:
task_prompt += self.crew_knowledge_context
if self.agent_knowledge_context:
task_prompt += self.agent_knowledge_context
# Quering crew specific knowledge
knowledge_snippets = self.crew.query_knowledge(
[self.knowledge_search_query], **knowledge_config
)
if knowledge_snippets:
self.crew_knowledge_context = extract_knowledge_context(
knowledge_snippets
)
if self.crew_knowledge_context:
task_prompt += self.crew_knowledge_context
crewai_event_bus.emit(
self,

View File

@@ -446,15 +446,20 @@ class Flow(Generic[T], metaclass=FlowMeta):
def __init__(
self,
initial_state: Union[Type[T], T, None] = None,
persistence: Optional[FlowPersistence] = None,
**kwargs: Any,
) -> None:
"""Initialize a new Flow instance.
Args:
initial_state: Initial state for the flow (BaseModel instance or dict)
persistence: Optional persistence backend for storing flow states
**kwargs: Additional state values to initialize or override
"""
# Set the initial_state for this instance
if initial_state is not None:
self.initial_state = initial_state
# Initialize basic instance attributes
self._methods: Dict[str, Callable] = {}
self._method_execution_counts: Dict[str, int] = {}
@@ -552,25 +557,21 @@ class Flow(Generic[T], metaclass=FlowMeta):
# Handle BaseModel instance case
if isinstance(self.initial_state, BaseModel):
model = cast(BaseModel, self.initial_state)
if not hasattr(model, "id"):
raise ValueError("Flow state model must have an 'id' field")
# Create new instance with same values to avoid mutations
if hasattr(model, "model_dump"):
# Create copy of the BaseModel to avoid mutations
if hasattr(model, "model_copy"):
# Pydantic v2
state_dict = model.model_dump()
elif hasattr(model, "dict"):
return cast(T, model.model_copy())
elif hasattr(model, "copy"):
# Pydantic v1
state_dict = model.dict()
return cast(T, model.copy())
else:
# Fallback for other BaseModel implementations
# Fallback for other BaseModel implementations - preserve original logic
state_dict = {
k: v for k, v in model.__dict__.items() if not k.startswith("_")
}
# Create new instance of the same class
model_class = type(model)
return cast(T, model_class(**state_dict))
model_class = type(model)
return cast(T, model_class(**state_dict))
raise TypeError(
f"Initial state must be dict or BaseModel, got {type(self.initial_state)}"
)
@@ -645,30 +646,26 @@ class Flow(Generic[T], metaclass=FlowMeta):
# For BaseModel states, preserve existing fields unless overridden
try:
model = cast(BaseModel, self._state)
# Get current state as dict
if hasattr(model, "model_dump"):
current_state = model.model_dump()
elif hasattr(model, "dict"):
current_state = model.dict()
if hasattr(model, "model_copy"):
# Pydantic v2
self._state = cast(T, model.model_copy(update=inputs))
elif hasattr(model, "copy"):
# Pydantic v1
self._state = cast(T, model.copy(update=inputs))
else:
# Fallback for other BaseModel implementations - preserve original logic
current_state = {
k: v for k, v in model.__dict__.items() if not k.startswith("_")
}
# Create new state with preserved fields and updates
new_state = {**current_state, **inputs}
# Create new instance with merged state
model_class = type(model)
if hasattr(model_class, "model_validate"):
# Pydantic v2
self._state = cast(T, model_class.model_validate(new_state))
elif hasattr(model_class, "parse_obj"):
# Pydantic v1
self._state = cast(T, model_class.parse_obj(new_state))
else:
# Fallback for other BaseModel implementations
self._state = cast(T, model_class(**new_state))
new_state = {**current_state, **inputs}
model_class = type(model)
if hasattr(model_class, "model_validate"):
self._state = cast(T, model_class.model_validate(new_state))
elif hasattr(model_class, "parse_obj"):
self._state = cast(T, model_class.parse_obj(new_state))
else:
self._state = cast(T, model_class(**new_state))
except ValidationError as e:
raise ValueError(f"Invalid inputs for structured state: {e}") from e
else:

View File

@@ -28,7 +28,7 @@ from pydantic import (
InstanceOf,
PrivateAttr,
model_validator,
field_validator,
field_validator
)
from crewai.agents.agent_builder.base_agent import BaseAgent
@@ -40,7 +40,7 @@ from crewai.agents.parser import (
OutputParserException,
)
from crewai.flow.flow_trackable import FlowTrackable
from crewai.llm import LLM
from crewai.llm import LLM, BaseLLM
from crewai.tools.base_tool import BaseTool
from crewai.tools.structured_tool import CrewStructuredTool
from crewai.utilities import I18N
@@ -135,7 +135,7 @@ class LiteAgent(FlowTrackable, BaseModel):
role: str = Field(description="Role of the agent")
goal: str = Field(description="Goal of the agent")
backstory: str = Field(description="Backstory of the agent")
llm: Optional[Union[str, InstanceOf[LLM], Any]] = Field(
llm: Optional[Union[str, InstanceOf[BaseLLM], Any]] = Field(
default=None, description="Language model that will run the agent"
)
tools: List[BaseTool] = Field(
@@ -209,8 +209,8 @@ class LiteAgent(FlowTrackable, BaseModel):
def setup_llm(self):
"""Set up the LLM and other components after initialization."""
self.llm = create_llm(self.llm)
if not isinstance(self.llm, LLM):
raise ValueError("Unable to create LLM instance")
if not isinstance(self.llm, BaseLLM):
raise ValueError(f"Expected LLM instance of type BaseLLM, got {type(self.llm).__name__}")
# Initialize callbacks
token_callback = TokenCalcHandler(token_cost_process=self._token_process)
@@ -232,7 +232,8 @@ class LiteAgent(FlowTrackable, BaseModel):
elif isinstance(self.guardrail, str):
from crewai.tasks.llm_guardrail import LLMGuardrail
assert isinstance(self.llm, LLM)
if not isinstance(self.llm, BaseLLM):
raise TypeError(f"Guardrail requires LLM instance of type BaseLLM, got {type(self.llm).__name__}")
self._guardrail = LLMGuardrail(description=self.guardrail, llm=self.llm)
@@ -620,4 +621,4 @@ class LiteAgent(FlowTrackable, BaseModel):
def _append_message(self, text: str, role: str = "assistant") -> None:
"""Append a message to the message list with the given role."""
self._messages.append(format_message_for_llm(text, role=role))
self._messages.append(format_message_for_llm(text, role=role))

View File

@@ -1,10 +1,9 @@
from typing import Any, Optional, Tuple
from typing import Any, Tuple
from pydantic import BaseModel, Field
from crewai.agent import Agent, LiteAgentOutput
from crewai.llm import LLM
from crewai.task import Task
from crewai.llm import BaseLLM
from crewai.tasks.task_output import TaskOutput
@@ -32,11 +31,11 @@ class LLMGuardrail:
def __init__(
self,
description: str,
llm: LLM,
llm: BaseLLM,
):
self.description = description
self.llm: LLM = llm
self.llm: BaseLLM = llm
def _validate_output(self, task_output: TaskOutput) -> LiteAgentOutput:
agent = Agent(

View File

@@ -1896,6 +1896,80 @@ def test_agent_with_knowledge_sources_generate_search_query():
assert "red" in result.raw.lower()
@pytest.mark.vcr(record_mode='none', filter_headers=["authorization"])
def test_agent_with_knowledge_with_no_crewai_knowledge():
mock_knowledge = MagicMock(spec=Knowledge)
agent = Agent(
role="Information Agent",
goal="Provide information based on knowledge sources",
backstory="You have access to specific knowledge sources.",
llm=LLM(model="openrouter/openai/gpt-4o-mini",api_key=os.getenv('OPENROUTER_API_KEY')),
knowledge=mock_knowledge
)
# Create a task that requires the agent to use the knowledge
task = Task(
description="What is Vidit's favorite color?",
expected_output="Vidit's favorclearite color.",
agent=agent,
)
crew = Crew(agents=[agent], tasks=[task])
crew.kickoff()
mock_knowledge.query.assert_called_once()
@pytest.mark.vcr(record_mode='none', filter_headers=["authorization"])
def test_agent_with_only_crewai_knowledge():
mock_knowledge = MagicMock(spec=Knowledge)
agent = Agent(
role="Information Agent",
goal="Provide information based on knowledge sources",
backstory="You have access to specific knowledge sources.",
llm=LLM(model="openrouter/openai/gpt-4o-mini",api_key=os.getenv('OPENROUTER_API_KEY'))
)
# Create a task that requires the agent to use the knowledge
task = Task(
description="What is Vidit's favorite color?",
expected_output="Vidit's favorclearite color.",
agent=agent
)
crew = Crew(agents=[agent], tasks=[task],knowledge=mock_knowledge)
crew.kickoff()
mock_knowledge.query.assert_called_once()
@pytest.mark.vcr(record_mode='none', filter_headers=["authorization"])
def test_agent_knowledege_with_crewai_knowledge():
crew_knowledge = MagicMock(spec=Knowledge)
agent_knowledge = MagicMock(spec=Knowledge)
agent = Agent(
role="Information Agent",
goal="Provide information based on knowledge sources",
backstory="You have access to specific knowledge sources.",
llm=LLM(model="openrouter/openai/gpt-4o-mini",api_key=os.getenv('OPENROUTER_API_KEY')),
knowledge=agent_knowledge
)
# Create a task that requires the agent to use the knowledge
task = Task(
description="What is Vidit's favorite color?",
expected_output="Vidit's favorclearite color.",
agent=agent,
)
crew = Crew(agents=[agent],tasks=[task],knowledge=crew_knowledge)
crew.kickoff()
agent_knowledge.query.assert_called_once()
crew_knowledge.query.assert_called_once()
@pytest.mark.vcr(filter_headers=["authorization"])
def test_litellm_auth_error_handling():
"""Test that LiteLLM authentication errors are handled correctly and not retried."""

View File

@@ -0,0 +1,150 @@
interactions:
- request:
body: '{"model": "openai/gpt-4o-mini", "messages": [{"role": "system", "content":
"Your goal is to rewrite the user query so that it is optimized for retrieval
from a vector database. Consider how the query will be used to find relevant
documents, and aim to make it more specific and context-aware. \n\n Do not include
any other text than the rewritten query, especially any preamble or postamble
and only add expected output format if its relevant to the rewritten query.
\n\n Focus on the key words of the intended task and to retrieve the most relevant
information. \n\n There will be some extra context provided that might need
to be removed such as expected_output formats structured_outputs and other instructions."},
{"role": "user", "content": "The original query is: What is Vidit''s favorite
color?\n\nThis is the expected criteria for your final answer: Vidit''s favorclearite
color.\nyou MUST return the actual complete content as the final answer, not
a summary.."}], "stream": false, "stop": ["\nObservation:"]}'
headers:
accept:
- '*/*'
accept-encoding:
- gzip, deflate
connection:
- keep-alive
content-length:
- '1017'
content-type:
- application/json
host:
- openrouter.ai
http-referer:
- https://litellm.ai
user-agent:
- litellm/1.68.0
x-title:
- liteLLM
method: POST
uri: https://openrouter.ai/api/v1/chat/completions
response:
body:
string: !!binary |
H4sIAAAAAAAAAwAAAP//4lKAAS4AAAAA//90kE1vE0EMhv9K9V64TMrmgyadG8ceECAhhIrQarrj
3bidHY/GTgSK9r+jpUpaJLja78djn8ARHgPlxXK72a6X6+12szhq7Id72d2V8b58/nbzQb98gkOp
cuRIFR4fC+X3d3AYJVKChxTKgd8OxRYbWYycGQ7y8EidwaPbB7vuZCyJjCXDoasUjCL8S61Dtxfu
SOG/n5BkKFUeFD4fUnLoObPu20pBJcNDTQoccjA+UvufLedIP+Ebh5FUw0DwJ1RJBI+gymoh20wj
2SjPpF85sr3Rqz4cpbLRVSdJ6jUcKvUHDenM81zFeXgeTNMPB/2lRuMMM1Atlf8k9qVt1rer3WrV
3DZwOJw5SpWxWGvyRFnnR7ybQc4/usxvHEwspBfhbun+NreRLHDSObUL3Z7iRdxM/wh9rb/c8coy
Tb8BAAD//wMAqVt3JyMCAAA=
headers:
Access-Control-Allow-Origin:
- '*'
CF-RAY:
- 9402cb503aec46c0-BOM
Connection:
- keep-alive
Content-Encoding:
- gzip
Content-Type:
- application/json
Date:
- Thu, 15 May 2025 12:56:14 GMT
Server:
- cloudflare
Transfer-Encoding:
- chunked
Vary:
- Accept-Encoding
x-clerk-auth-message:
- Invalid JWT form. A JWT consists of three parts separated by dots. (reason=token-invalid,
token-carrier=header)
x-clerk-auth-reason:
- token-invalid
x-clerk-auth-status:
- signed-out
status:
code: 200
message: OK
- request:
body: '{"model": "openai/gpt-4o-mini", "messages": [{"role": "system", "content":
"You are Information Agent. You have access to specific knowledge sources.\nYour
personal goal is: Provide information based on knowledge sources\nTo give my
best complete final answer to the task respond using the exact following format:\n\nThought:
I now can give a great answer\nFinal Answer: Your final answer must be the great
and the most complete as possible, it must be outcome described.\n\nI MUST use
these formats, my job depends on it!"}, {"role": "user", "content": "\nCurrent
Task: What is Vidit''s favorite color?\n\nThis is the expected criteria for
your final answer: Vidit''s favorclearite color.\nyou MUST return the actual
complete content as the final answer, not a summary.\n\nBegin! This is VERY
important to you, use the tools available and give your best Final Answer, your
job depends on it!\n\nThought:"}], "stream": false, "stop": ["\nObservation:"]}'
headers:
accept:
- '*/*'
accept-encoding:
- gzip, deflate
connection:
- keep-alive
content-length:
- '951'
content-type:
- application/json
host:
- openrouter.ai
http-referer:
- https://litellm.ai
user-agent:
- litellm/1.68.0
x-title:
- liteLLM
method: POST
uri: https://openrouter.ai/api/v1/chat/completions
response:
body:
string: !!binary |
H4sIAAAAAAAAAwAAAP//4lKAAS4AAAAA///iQjABAAAA//90kE9rG0EMxb/K8C69jNON7WJ7boFS
CD2ENm2g/1jGs/Ja7aw0zIydBuPvXjbBcQrtUU9P0u/pAO7g0JNMLhfzxexytli8mdy8r7c6/3Lb
v13eff00088fPj7AImXdc0cZDjeJ5OoaFoN2FOGgicTz6z7VyVwnAwvDQtc/KVQ4hK2vF0GHFKmy
CixCJl+pgzuftQhb5UAF7tsBUfuUdV3gZBejxYaFy7bN5IsKHErVBAvxlffU/qfL0tFvuMZioFJ8
T3AHZI0EB18Kl+qljjQqlWQkvTai9yZ4MT3vyXjTj6DGS7mnbMx3ecfio7l6rJ25447rq2I2fq+Z
K5mgUbPhYtZxRxewyLTZFR9PMZ4IWfon4Xj8YVEeSqVhzNBTTpkfQTapbWar6XI6bVYNLHYn/JR1
SLWt+oukjP9rRv7Ta8/6yqJq9fGsLFf27+m2o+o5lnFt8GFL3bO5Of5j60v/c5AXI8fjHwAAAP//
AwDEkP8dZgIAAA==
headers:
Access-Control-Allow-Origin:
- '*'
CF-RAY:
- 9402cb55c9fe46c0-BOM
Connection:
- keep-alive
Content-Encoding:
- gzip
Content-Type:
- application/json
Date:
- Thu, 15 May 2025 12:56:15 GMT
Server:
- cloudflare
Transfer-Encoding:
- chunked
Vary:
- Accept-Encoding
x-clerk-auth-message:
- Invalid JWT form. A JWT consists of three parts separated by dots. (reason=token-invalid,
token-carrier=header)
x-clerk-auth-reason:
- token-invalid
x-clerk-auth-status:
- signed-out
status:
code: 200
message: OK
version: 1

View File

@@ -0,0 +1,151 @@
interactions:
- request:
body: '{"model": "openai/gpt-4o-mini", "messages": [{"role": "system", "content":
"Your goal is to rewrite the user query so that it is optimized for retrieval
from a vector database. Consider how the query will be used to find relevant
documents, and aim to make it more specific and context-aware. \n\n Do not include
any other text than the rewritten query, especially any preamble or postamble
and only add expected output format if its relevant to the rewritten query.
\n\n Focus on the key words of the intended task and to retrieve the most relevant
information. \n\n There will be some extra context provided that might need
to be removed such as expected_output formats structured_outputs and other instructions."},
{"role": "user", "content": "The original query is: What is Vidit''s favorite
color?\n\nThis is the expected criteria for your final answer: Vidit''s favorclearite
color.\nyou MUST return the actual complete content as the final answer, not
a summary.."}], "stream": false, "stop": ["\nObservation:"]}'
headers:
accept:
- '*/*'
accept-encoding:
- gzip, deflate
connection:
- keep-alive
content-length:
- '1017'
content-type:
- application/json
host:
- openrouter.ai
http-referer:
- https://litellm.ai
user-agent:
- litellm/1.68.0
x-title:
- liteLLM
method: POST
uri: https://openrouter.ai/api/v1/chat/completions
response:
body:
string: !!binary |
H4sIAAAAAAAAAwAAAP//4lKAAS4AAAAA//90kE1vE0EMhv9K9V64TGCbNGQ7N46gIg6IXhBaTWed
Xbez49HYiaii/e9oqRKKBFf7/XjsE7iHx0B5db272W2uN++b3ep585k+jcmo/XqnYXvX5m/3cChV
jtxThceXQvnDRzhM0lOChxTKgd8NxVY3spo4Mxzk4ZGiwSOOwd5GmUoiY8lwiJWCUQ9/qW0d4igc
SeG/n5BkKFUeFD4fUnLYc2Ydu0pBJcNDTQoccjA+UvefLeeefsI3DhOphoHgT6iSCB5BldVCtoVG
slFeSO+5Z3ujV/twlMpGV1GSVDhU2h80pDPOSxPn4WUwzz8c9FmNpoVloFoq/w7cl67Z3K7b9bq5
beBwOGOUKlOxzuSJsi5/2C4c5xdd5lsHEwvpj7Bt3N/mricLnHRJjSGO1F/EzfyP0Nf6yx2vLPP8
CwAA//8DAOHu/cIiAgAA
headers:
Access-Control-Allow-Origin:
- '*'
CF-RAY:
- 9402c73df9d8859c-BOM
Connection:
- keep-alive
Content-Encoding:
- gzip
Content-Type:
- application/json
Date:
- Thu, 15 May 2025 12:53:27 GMT
Server:
- cloudflare
Transfer-Encoding:
- chunked
Vary:
- Accept-Encoding
x-clerk-auth-message:
- Invalid JWT form. A JWT consists of three parts separated by dots. (reason=token-invalid,
token-carrier=header)
x-clerk-auth-reason:
- token-invalid
x-clerk-auth-status:
- signed-out
status:
code: 200
message: OK
- request:
body: '{"model": "openai/gpt-4o-mini", "messages": [{"role": "system", "content":
"You are Information Agent. You have access to specific knowledge sources.\nYour
personal goal is: Provide information based on knowledge sources\nTo give my
best complete final answer to the task respond using the exact following format:\n\nThought:
I now can give a great answer\nFinal Answer: Your final answer must be the great
and the most complete as possible, it must be outcome described.\n\nI MUST use
these formats, my job depends on it!"}, {"role": "user", "content": "\nCurrent
Task: What is Vidit''s favorite color?\n\nThis is the expected criteria for
your final answer: Vidit''s favorclearite color.\nyou MUST return the actual
complete content as the final answer, not a summary.\n\nBegin! This is VERY
important to you, use the tools available and give your best Final Answer, your
job depends on it!\n\nThought:"}], "stream": false, "stop": ["\nObservation:"]}'
headers:
accept:
- '*/*'
accept-encoding:
- gzip, deflate
connection:
- keep-alive
content-length:
- '951'
content-type:
- application/json
host:
- openrouter.ai
http-referer:
- https://litellm.ai
user-agent:
- litellm/1.68.0
x-title:
- liteLLM
method: POST
uri: https://openrouter.ai/api/v1/chat/completions
response:
body:
string: !!binary |
H4sIAAAAAAAAAwAAAP//4lKAAS4AAAAA///iQjABAAAA//90kUGPEzEMhf+K5QuXdJmlpbvkthIg
emFXQoIDoMpNPFNDJo6STLul6n9H09KyIDjmxc9+/rxH8Wix4zi5vpndTK+n8+Z2wo9vXj28fHff
vW4+PNT5j1l6/wkNpqwb8ZzR4n3ieLdAg716DmhRE0eS512qk5lOeomCBnX1jV1Fi25N9cppnwJX
0YgGXWaq7NH+HmvQrVUcF7Sf9xi0S1lXBW0cQjDYSpSyXmamohEtlqoJDUaqsuHlf34len5E2xjs
uRTqGO0eswZGi1SKlEqxjmk0Vo5j0gVE3YKjCJ1sGAi6MShQLFvOAF/iW4kU4O74tvBRvNRnBVra
aJbK4DRoBikQtcJWPIcdeHVDz7GyB4mQhlUQF3ZAG5JAq8BQdMiOi4GisBiHj+ZftIHA87hePeY5
5cjcUfYSO1hLgZLYSSvurxRXaDBzOxQKZ4gnPhK7k3A4fDVYdqVyPxLsOKcsRwxtWvoVOZo3vm3Q
4HCGl7L2qS6rfudYxus1I73zYS/69NZg1UrhorwYD/yHe+m5koQytnXk1uwvxc3hH12f1l8WeWI5
HH4CAAD//wMAhZKqO+QCAAA=
headers:
Access-Control-Allow-Origin:
- '*'
CF-RAY:
- 9402c7459f3f859c-BOM
Connection:
- keep-alive
Content-Encoding:
- gzip
Content-Type:
- application/json
Date:
- Thu, 15 May 2025 12:53:28 GMT
Server:
- cloudflare
Transfer-Encoding:
- chunked
Vary:
- Accept-Encoding
x-clerk-auth-message:
- Invalid JWT form. A JWT consists of three parts separated by dots. (reason=token-invalid,
token-carrier=header)
x-clerk-auth-reason:
- token-invalid
x-clerk-auth-status:
- signed-out
status:
code: 200
message: OK
version: 1

View File

@@ -0,0 +1,150 @@
interactions:
- request:
body: '{"model": "openai/gpt-4o-mini", "messages": [{"role": "system", "content":
"Your goal is to rewrite the user query so that it is optimized for retrieval
from a vector database. Consider how the query will be used to find relevant
documents, and aim to make it more specific and context-aware. \n\n Do not include
any other text than the rewritten query, especially any preamble or postamble
and only add expected output format if its relevant to the rewritten query.
\n\n Focus on the key words of the intended task and to retrieve the most relevant
information. \n\n There will be some extra context provided that might need
to be removed such as expected_output formats structured_outputs and other instructions."},
{"role": "user", "content": "The original query is: What is Vidit''s favorite
color?\n\nThis is the expected criteria for your final answer: Vidit''s favorclearite
color.\nyou MUST return the actual complete content as the final answer, not
a summary.."}], "stream": false, "stop": ["\nObservation:"]}'
headers:
accept:
- '*/*'
accept-encoding:
- gzip, deflate
connection:
- keep-alive
content-length:
- '1017'
content-type:
- application/json
host:
- openrouter.ai
http-referer:
- https://litellm.ai
user-agent:
- litellm/1.68.0
x-title:
- liteLLM
method: POST
uri: https://openrouter.ai/api/v1/chat/completions
response:
body:
string: !!binary |
H4sIAAAAAAAAAwAAAP//4lKAAS4AAAAA//90kE1PIzEMhv8Kei97Sdnplwq5gTgAF8ShcFitRmnG
nTFk4ihxq11V899Xs6gFJLja78djH8ANLFqKk+lqsZpP56vpYqJhublfP1eP65v1i79Lt9fdMwxS
lj03lGHxkChe3cGgl4YCLCRRdPyzTTpZyKTnyDCQzQt5hYXvnJ576VMgZYkw8JmcUgP7XmvgO2FP
BfbXAUHalGVTYOMuBIMtRy5dnckVibAoKgkG0Snvqf5my7GhP7CVQU+luJZgD8gSCBauFC7qoo40
EpXiSPrEDeuPcrZ1e8msdOYlSIZBpu2uuHDEeWvi2L4NhuG3QflblPqRpaWcMv8P3Ka6ml/OLmaz
6rKCwe6IkbL0SWuVV4pl/MNy5Di+6DRfGqioC+/Ci8p8NtcNqeNQxlTvfEfNSVwNX4R+1J/u+GAZ
hn8AAAD//wMAIwJ79CICAAA=
headers:
Access-Control-Allow-Origin:
- '*'
CF-RAY:
- 9402c9db99ec4722-BOM
Connection:
- keep-alive
Content-Encoding:
- gzip
Content-Type:
- application/json
Date:
- Thu, 15 May 2025 12:55:14 GMT
Server:
- cloudflare
Transfer-Encoding:
- chunked
Vary:
- Accept-Encoding
x-clerk-auth-message:
- Invalid JWT form. A JWT consists of three parts separated by dots. (reason=token-invalid,
token-carrier=header)
x-clerk-auth-reason:
- token-invalid
x-clerk-auth-status:
- signed-out
status:
code: 200
message: OK
- request:
body: '{"model": "openai/gpt-4o-mini", "messages": [{"role": "system", "content":
"You are Information Agent. You have access to specific knowledge sources.\nYour
personal goal is: Provide information based on knowledge sources\nTo give my
best complete final answer to the task respond using the exact following format:\n\nThought:
I now can give a great answer\nFinal Answer: Your final answer must be the great
and the most complete as possible, it must be outcome described.\n\nI MUST use
these formats, my job depends on it!"}, {"role": "user", "content": "\nCurrent
Task: What is Vidit''s favorite color?\n\nThis is the expected criteria for
your final answer: Vidit''s favorclearite color.\nyou MUST return the actual
complete content as the final answer, not a summary.\n\nBegin! This is VERY
important to you, use the tools available and give your best Final Answer, your
job depends on it!\n\nThought:"}], "stream": false, "stop": ["\nObservation:"]}'
headers:
accept:
- '*/*'
accept-encoding:
- gzip, deflate
connection:
- keep-alive
content-length:
- '951'
content-type:
- application/json
host:
- openrouter.ai
http-referer:
- https://litellm.ai
user-agent:
- litellm/1.68.0
x-title:
- liteLLM
method: POST
uri: https://openrouter.ai/api/v1/chat/completions
response:
body:
string: !!binary |
H4sIAAAAAAAAAwAAAP//4lKAAS4AAAAA///iQjABAAAA//90kN1qGzEQRl9FfNdyul4nday73ARy
VUpLE2jLIu+O15NoZ4QkOy1moa/R1+uTlE1wnEB7qU/zc84cwB0cepLZfHm+XMwXy/nF7II/3d7V
H+tOPvsS3le3d+keFjHpnjtKcPgQSa5uYDFoRwEOGkk8v+tjmZ3rbGBhWOj6ntoCh3bry1mrQwxU
WAUWbSJfqIM7rbVot8otZbivBwTtY9J1hpNdCBYbFs7bJpHPKnDIRSMsxBfeU/OfX5aOfsBVFgPl
7HuCOyBpIDj4nDkXL2WiUSkkE+mNEX00rRfT856MN/0EarzkR0rGfJNrFh/M1dPbmS/ccfnz63c2
G7/XxIVMq0GT4WzWYUdnsEi02WUfjiLPjCz9czCO3y3yz1xomCx6SjHxE8omNtViVV/WdbWqYLE7
CsSkQyxN0QeSPF2wmgyOxz3lK4uixYdTcrmyb7ubjornkKexrW+31L0UV+M/pr6ufxF51TKOfwEA
AP//AwBybekMaAIAAA==
headers:
Access-Control-Allow-Origin:
- '*'
CF-RAY:
- 9402c9e1b94a4722-BOM
Connection:
- keep-alive
Content-Encoding:
- gzip
Content-Type:
- application/json
Date:
- Thu, 15 May 2025 12:55:15 GMT
Server:
- cloudflare
Transfer-Encoding:
- chunked
Vary:
- Accept-Encoding
x-clerk-auth-message:
- Invalid JWT form. A JWT consists of three parts separated by dots. (reason=token-invalid,
token-carrier=header)
x-clerk-auth-reason:
- token-invalid
x-clerk-auth-status:
- signed-out
status:
code: 200
message: OK
version: 1

View File

@@ -0,0 +1,181 @@
"""Test Flow initial_state BaseModel dict coercion fix for issue #3147"""
from pydantic import BaseModel
from crewai.flow.flow import Flow
class StateWithItems(BaseModel):
items: list = [1, 2, 3]
metadata: dict = {"x": 1}
class StateWithKeys(BaseModel):
keys: list = ["a", "b", "c"]
data: str = "test"
class StateWithValues(BaseModel):
values: list = [10, 20, 30]
name: str = "example"
class StateWithGet(BaseModel):
get: str = "method_name"
config: dict = {"enabled": True}
class StateWithPop(BaseModel):
pop: int = 42
settings: list = ["option1", "option2"]
class StateWithUpdate(BaseModel):
update: bool = True
version: str = "1.0.0"
class StateWithClear(BaseModel):
clear: str = "action"
status: str = "active"
def test_flow_initial_state_items_field():
"""Test that BaseModel with 'items' field preserves structure and doesn't get dict coercion."""
flow = Flow(initial_state=StateWithItems())
flow.kickoff()
assert isinstance(flow.state, StateWithItems)
assert not isinstance(flow.state, dict)
assert isinstance(flow.state.items, list)
assert flow.state.items == [1, 2, 3]
assert len(flow.state.items) == 3
assert flow.state.metadata == {"x": 1}
def test_flow_initial_state_keys_field():
"""Test that BaseModel with 'keys' field preserves structure."""
flow = Flow(initial_state=StateWithKeys())
flow.kickoff()
assert isinstance(flow.state, StateWithKeys)
assert isinstance(flow.state.keys, list)
assert flow.state.keys == ["a", "b", "c"]
assert len(flow.state.keys) == 3
assert flow.state.data == "test"
def test_flow_initial_state_values_field():
"""Test that BaseModel with 'values' field preserves structure."""
flow = Flow(initial_state=StateWithValues())
flow.kickoff()
assert isinstance(flow.state, StateWithValues)
assert isinstance(flow.state.values, list)
assert flow.state.values == [10, 20, 30]
assert len(flow.state.values) == 3
assert flow.state.name == "example"
def test_flow_initial_state_get_field():
"""Test that BaseModel with 'get' field preserves structure."""
flow = Flow(initial_state=StateWithGet())
flow.kickoff()
assert isinstance(flow.state, StateWithGet)
assert isinstance(flow.state.get, str)
assert flow.state.get == "method_name"
assert flow.state.config == {"enabled": True}
def test_flow_initial_state_pop_field():
"""Test that BaseModel with 'pop' field preserves structure."""
flow = Flow(initial_state=StateWithPop())
flow.kickoff()
assert isinstance(flow.state, StateWithPop)
assert isinstance(flow.state.pop, int)
assert flow.state.pop == 42
assert flow.state.settings == ["option1", "option2"]
def test_flow_initial_state_update_field():
"""Test that BaseModel with 'update' field preserves structure."""
flow = Flow(initial_state=StateWithUpdate())
flow.kickoff()
assert isinstance(flow.state, StateWithUpdate)
assert isinstance(flow.state.update, bool)
assert flow.state.update is True
assert flow.state.version == "1.0.0"
def test_flow_initial_state_clear_field():
"""Test that BaseModel with 'clear' field preserves structure."""
flow = Flow(initial_state=StateWithClear())
flow.kickoff()
assert isinstance(flow.state, StateWithClear)
assert isinstance(flow.state.clear, str)
assert flow.state.clear == "action"
assert flow.state.status == "active"
def test_flow_state_modification_preserves_basemodel():
"""Test that modifying flow state preserves BaseModel structure."""
class ModifiableState(BaseModel):
items: list = [1, 2, 3]
counter: int = 0
class TestFlow(Flow[ModifiableState]):
@Flow.start()
def modify_state(self):
self.state.counter += 1
self.state.items.append(4)
flow = TestFlow(initial_state=ModifiableState())
flow.kickoff()
assert isinstance(flow.state, ModifiableState)
assert not isinstance(flow.state, dict)
assert flow.state.counter == 1
assert flow.state.items == [1, 2, 3, 4]
def test_flow_with_inputs_preserves_basemodel():
"""Test that providing inputs to flow preserves BaseModel structure."""
class InputState(BaseModel):
items: list = []
name: str = ""
flow = Flow(initial_state=InputState())
flow.kickoff(inputs={"name": "test_flow", "items": [5, 6, 7]})
assert isinstance(flow.state, InputState)
assert not isinstance(flow.state, dict)
assert flow.state.name == "test_flow"
assert flow.state.items == [5, 6, 7]
def test_reproduction_case_from_issue_3147():
"""Test the exact reproduction case from GitHub issue #3147."""
class MyState(BaseModel):
items: list = [1, 2, 3]
metadata: dict = {"x": 1}
flow = Flow(initial_state=MyState())
flow.kickoff()
assert isinstance(flow.state.items, list)
assert len(flow.state.items) == 3
assert flow.state.items == [1, 2, 3]
assert not callable(flow.state.items)
assert str(type(flow.state.items)) != "<class 'builtin_function_or_method'>"

View File

@@ -12,6 +12,8 @@ from crewai.tools import BaseTool
from crewai.utilities.events import crewai_event_bus
from crewai.utilities.events.agent_events import LiteAgentExecutionStartedEvent
from crewai.utilities.events.tool_usage_events import ToolUsageStartedEvent
from crewai.llms.base_llm import BaseLLM
from unittest.mock import patch
# A simple test tool
@@ -418,3 +420,76 @@ def test_agent_output_when_guardrail_returns_base_model():
result = agent.kickoff(messages="Top 10 best players in the world?")
assert result.pydantic == Player(name="Lionel Messi", country="Argentina")
def test_lite_agent_with_custom_llm_and_guardrails():
"""Test that CustomLLM (inheriting from BaseLLM) works with guardrails."""
class CustomLLM(BaseLLM):
def __init__(self, response: str = "Custom response"):
super().__init__(model="custom-model")
self.response = response
self.call_count = 0
def call(self, messages, tools=None, callbacks=None, available_functions=None, from_task=None, from_agent=None) -> str:
self.call_count += 1
if "valid" in str(messages) and "feedback" in str(messages):
return '{"valid": true, "feedback": null}'
if "Thought:" in str(messages):
return f"Thought: I will analyze soccer players\nFinal Answer: {self.response}"
return self.response
def supports_function_calling(self) -> bool:
return False
def supports_stop_words(self) -> bool:
return False
def get_context_window_size(self) -> int:
return 4096
custom_llm = CustomLLM(response="Brazilian soccer players are the best!")
agent = LiteAgent(
role="Sports Analyst",
goal="Analyze soccer players",
backstory="You analyze soccer players and their performance.",
llm=custom_llm,
guardrail="Only include Brazilian players"
)
result = agent.kickoff("Tell me about the best soccer players")
assert custom_llm.call_count > 0
assert "Brazilian" in result.raw
custom_llm2 = CustomLLM(response="Original response")
def test_guardrail(output):
return (True, "Modified by guardrail")
agent2 = LiteAgent(
role="Test Agent",
goal="Test goal",
backstory="Test backstory",
llm=custom_llm2,
guardrail=test_guardrail
)
result2 = agent2.kickoff("Test message")
assert result2.raw == "Modified by guardrail"
@pytest.mark.vcr(filter_headers=["authorization"])
def test_lite_agent_with_invalid_llm():
"""Test that LiteAgent raises proper error when create_llm returns None."""
with patch('crewai.lite_agent.create_llm', return_value=None):
with pytest.raises(ValueError) as exc_info:
LiteAgent(
role="Test Agent",
goal="Test goal",
backstory="Test backstory",
llm="invalid-model"
)
assert "Expected LLM instance of type BaseLLM" in str(exc_info.value)