Compare commits

..

8 Commits

Author SHA1 Message Date
Greyson LaLonde
3a8889ce61 Merge branch 'main' into lorenze/trace-improvements-3 2025-09-12 10:45:11 -04:00
Greyson LaLonde
d865a49f5a Merge branch 'main' into lorenze/trace-improvements-3 2025-09-11 17:41:47 -04:00
lorenzejay
677fe9032c Merge branch 'lorenze/trace-improvements-3' of github.com:crewAIInc/crewAI into lorenze/trace-improvements-3 2025-09-11 12:27:33 -07:00
lorenzejay
6e8c1f332f linted 2025-09-11 12:26:01 -07:00
Lorenze Jay
abe170cdc2 Merge branch 'main' into lorenze/trace-improvements-3 2025-09-11 12:20:49 -07:00
lorenzejay
51767f2e15 Merge branch 'main' of github.com:crewAIInc/crewAI into lorenze/trace-improvements-3 2025-09-10 10:43:38 -07:00
lorenzejay
dc41a0d13b fixed types 2025-09-10 10:32:59 -07:00
lorenzejay
6d02b64674 refactor: update type hints and enhance trace batch display logic
- Changed type hints from `Dict` and `List` to built-in `dict` and `list` for consistency with modern Python standards.
- Refactored the trace batch finalization display logic to improve user experience by introducing a new method `_display_traces_events_link`.
- Enhanced error handling in the display method to fallback to a simpler display in case of exceptions.
- Cleaned up unused imports and organized the import statements for better readability.
2025-09-10 10:28:48 -07:00
307 changed files with 8973 additions and 15592 deletions

View File

@@ -1,102 +0,0 @@
# For most projects, this workflow file will not need changing; you simply need
# to commit it to your repository.
#
# You may wish to alter this file to override the set of languages analyzed,
# or to provide custom queries or build logic.
#
# ******** NOTE ********
# We have attempted to detect the languages in your repository. Please check
# the `language` matrix defined below to confirm you have the correct set of
# supported CodeQL languages.
#
name: "CodeQL Advanced"
on:
push:
branches: [ "main" ]
paths-ignore:
- "src/crewai/cli/templates/**"
pull_request:
branches: [ "main" ]
paths-ignore:
- "src/crewai/cli/templates/**"
jobs:
analyze:
name: Analyze (${{ matrix.language }})
# Runner size impacts CodeQL analysis time. To learn more, please see:
# - https://gh.io/recommended-hardware-resources-for-running-codeql
# - https://gh.io/supported-runners-and-hardware-resources
# - https://gh.io/using-larger-runners (GitHub.com only)
# Consider using larger runners or machines with greater resources for possible analysis time improvements.
runs-on: ${{ (matrix.language == 'swift' && 'macos-latest') || 'ubuntu-latest' }}
permissions:
# required for all workflows
security-events: write
# required to fetch internal or private CodeQL packs
packages: read
# only required for workflows in private repositories
actions: read
contents: read
strategy:
fail-fast: false
matrix:
include:
- language: actions
build-mode: none
- language: python
build-mode: none
# CodeQL supports the following values keywords for 'language': 'actions', 'c-cpp', 'csharp', 'go', 'java-kotlin', 'javascript-typescript', 'python', 'ruby', 'rust', 'swift'
# Use `c-cpp` to analyze code written in C, C++ or both
# Use 'java-kotlin' to analyze code written in Java, Kotlin or both
# Use 'javascript-typescript' to analyze code written in JavaScript, TypeScript or both
# To learn more about changing the languages that are analyzed or customizing the build mode for your analysis,
# see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/customizing-your-advanced-setup-for-code-scanning.
# If you are analyzing a compiled language, you can modify the 'build-mode' for that language to customize how
# your codebase is analyzed, see https://docs.github.com/en/code-security/code-scanning/creating-an-advanced-setup-for-code-scanning/codeql-code-scanning-for-compiled-languages
steps:
- name: Checkout repository
uses: actions/checkout@v4
# Add any setup steps before running the `github/codeql-action/init` action.
# This includes steps like installing compilers or runtimes (`actions/setup-node`
# or others). This is typically only required for manual builds.
# - name: Setup runtime (example)
# uses: actions/setup-example@v1
# Initializes the CodeQL tools for scanning.
- name: Initialize CodeQL
uses: github/codeql-action/init@v3
with:
languages: ${{ matrix.language }}
build-mode: ${{ matrix.build-mode }}
# If you wish to specify custom queries, you can do so here or in a config file.
# By default, queries listed here will override any specified in a config file.
# Prefix the list here with "+" to use these queries and those in the config file.
# For more details on CodeQL's query packs, refer to: https://docs.github.com/en/code-security/code-scanning/automatically-scanning-your-code-for-vulnerabilities-and-errors/configuring-code-scanning#using-queries-in-ql-packs
# queries: security-extended,security-and-quality
# If the analyze step fails for one of the languages you are analyzing with
# "We were unable to automatically build your code", modify the matrix above
# to set the build mode to "manual" for that language. Then modify this step
# to build your code.
# Command-line programs to run using the OS shell.
# 📚 See https://docs.github.com/en/actions/using-workflows/workflow-syntax-for-github-actions#jobsjob_idstepsrun
- if: matrix.build-mode == 'manual'
shell: bash
run: |
echo 'If you are using a "manual" build mode for one or more of the' \
'languages you are analyzing, replace this with the commands to build' \
'your code, for example:'
echo ' make bootstrap'
echo ' make release'
exit 1
- name: Perform CodeQL Analysis
uses: github/codeql-action/analyze@v3
with:
category: "/language:${{matrix.language}}"

View File

@@ -5,82 +5,6 @@ icon: "clock"
mode: "wide"
---
<Update label="Sep 20, 2025">
## v0.193.2
[View release on GitHub](https://github.com/crewAIInc/crewAI/releases/tag/0.193.2)
## What's Changed
- Updated pyproject templates to use the right version
</Update>
<Update label="Sep 20, 2025">
## v0.193.1
[View release on GitHub](https://github.com/crewAIInc/crewAI/releases/tag/0.193.1)
## What's Changed
- Series of minor fixes and linter improvements
</Update>
<Update label="Sep 19, 2025">
## v0.193.0
[View release on GitHub](https://github.com/crewAIInc/crewAI/releases/tag/0.193.0)
## Core Improvements & Fixes
- Fixed handling of the `model` parameter during OpenAI adapter initialization
- Resolved test duration cache issues in CI workflows
- Fixed flaky test related to repeated tool usage by agents
- Added missing event exports to `__init__.py` for consistent module behavior
- Dropped message storage from metadata in Mem0 to reduce bloat
- Fixed L2 distance metric support for backward compatibility in vector search
## New Features & Enhancements
- Introduced thread-safe platform context management
- Added test duration caching for optimized `pytest-split` runs
- Added ephemeral trace improvements for better trace control
- Made search parameters for RAG, knowledge, and memory fully configurable
- Enabled ChromaDB to use OpenAI API for embedding functions
- Added deeper observability tools for user-level insights
- Unified RAG storage system with instance-specific client support
## Documentation & Guides
- Updated `RagTool` references to reflect CrewAI native RAG implementation
- Improved internal docs for `langgraph` and `openai` agent adapters with type annotations and docstrings
</Update>
<Update label="Sep 11, 2025">
## v0.186.1
[View release on GitHub](https://github.com/crewAIInc/crewAI/releases/tag/0.186.1)
## What's Changed
- Fixed version not being found and silently failing reversion
- Bumped CrewAI version to 0.186.1 and updated dependencies in the CLI
</Update>
<Update label="Sep 10, 2025">
## v0.186.0
[View release on GitHub](https://github.com/crewAIInc/crewAI/releases/tag/0.186.0)
## What's Changed
- Refer to the GitHub release notes for detailed changes
</Update>
<Update label="Sep 04, 2025">
## v0.177.0

View File

@@ -404,10 +404,6 @@ crewai config reset
After resetting configuration, re-run `crewai login` to authenticate again.
</Tip>
<Tip>
CrewAI CLI handles authentication to the Tool Repository automatically when adding packages to your project. Just append `crewai` before any `uv` command to use it. E.g. `crewai uv add requests`. For more information, see [Tool Repository](https://docs.crewai.com/enterprise/features/tool-repository) docs.
</Tip>
<Note>
Configuration settings are stored in `~/.config/crewai/settings.json`. Some settings like organization name and UUID are read-only and managed through authentication and organization commands. Tool repository related settings are hidden and cannot be set directly by users.
</Note>

View File

@@ -52,36 +52,6 @@ researcher = Agent(
)
```
## Adding other packages after installing a tool
After installing a tool from the CrewAI Enterprise Tool Repository, you need to use the `crewai uv` command to add other packages to your project.
Using pure `uv` commands will fail due to authentication to tool repository being handled by the CLI. By using the `crewai uv` command, you can add other packages to your project without having to worry about authentication.
Any `uv` command can be used with the `crewai uv` command, making it a powerful tool for managing your project's dependencies without the hassle of managing authentication through environment variables or other methods.
Say that you have installed a custom tool from the CrewAI Enterprise Tool Repository called "my-tool":
```bash
crewai tool install my-tool
```
And now you want to add another package to your project, you can use the following command:
```bash
crewai uv add requests
```
Other commands like `uv sync` or `uv remove` can also be used with the `crewai uv` command:
```bash
crewai uv sync
```
```bash
crewai uv remove requests
```
This will add the package to your project and update `pyproject.toml` accordingly.
## Creating and Publishing Tools
To create a new tool project:

View File

@@ -22,45 +22,6 @@ Human-in-the-Loop (HITL) is a powerful approach that combines artificial intelli
<Frame>
<img src="/images/enterprise/crew-webhook-url.png" alt="Crew Webhook URL" />
</Frame>
Example with Bearer authentication:
```bash
curl -X POST {BASE_URL}/kickoff \
-H "Authorization: Bearer YOUR_API_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"inputs": {
"topic": "AI Research"
},
"humanInputWebhook": {
"url": "https://your-webhook.com/hitl",
"authentication": {
"strategy": "bearer",
"token": "your-webhook-secret-token"
}
}
}'
```
Or with Basic authentication:
```bash
curl -X POST {BASE_URL}/kickoff \
-H "Authorization: Bearer YOUR_API_TOKEN" \
-H "Content-Type: application/json" \
-d '{
"inputs": {
"topic": "AI Research"
},
"humanInputWebhook": {
"url": "https://your-webhook.com/hitl",
"authentication": {
"strategy": "basic",
"username": "your-username",
"password": "your-password"
}
}
}'
```
</Step>
<Step title="Receive Webhook Notification">
@@ -115,4 +76,4 @@ HITL workflows are particularly valuable for:
- Complex decision-making scenarios
- Sensitive or high-stakes operations
- Creative tasks requiring human judgment
- Compliance and regulatory reviews
- Compliance and regulatory reviews

View File

@@ -27,7 +27,7 @@ Follow the steps below to get Crewing! 🚣‍♂️
<Step title="Navigate to your new crew project">
<CodeGroup>
```shell Terminal
cd latest_ai_development
cd latest-ai-development
```
</CodeGroup>
</Step>

View File

@@ -9,7 +9,7 @@ mode: "wide"
## Description
The `RagTool` is designed to answer questions by leveraging the power of Retrieval-Augmented Generation (RAG) through CrewAI's native RAG system.
The `RagTool` is designed to answer questions by leveraging the power of Retrieval-Augmented Generation (RAG) through EmbedChain.
It provides a dynamic knowledge base that can be queried to retrieve relevant information from various data sources.
This tool is particularly useful for applications that require access to a vast array of information and need to provide contextually relevant answers.
@@ -76,8 +76,8 @@ The `RagTool` can be used with a wide variety of data sources, including:
The `RagTool` accepts the following parameters:
- **summarize**: Optional. Whether to summarize the retrieved content. Default is `False`.
- **adapter**: Optional. A custom adapter for the knowledge base. If not provided, a CrewAIRagAdapter will be used.
- **config**: Optional. Configuration for the underlying CrewAI RAG system.
- **adapter**: Optional. A custom adapter for the knowledge base. If not provided, an EmbedchainAdapter will be used.
- **config**: Optional. Configuration for the underlying EmbedChain App.
## Adding Content
@@ -130,23 +130,44 @@ from crewai_tools import RagTool
# Create a RAG tool with custom configuration
config = {
"vectordb": {
"provider": "qdrant",
"app": {
"name": "custom_app",
},
"llm": {
"provider": "openai",
"config": {
"collection_name": "my-collection"
"model": "gpt-4",
}
},
"embedding_model": {
"provider": "openai",
"config": {
"model": "text-embedding-3-small"
"model": "text-embedding-ada-002"
}
},
"vectordb": {
"provider": "elasticsearch",
"config": {
"collection_name": "my-collection",
"cloud_id": "deployment-name:xxxx",
"api_key": "your-key",
"verify_certs": False
}
},
"chunker": {
"chunk_size": 400,
"chunk_overlap": 100,
"length_function": "len",
"min_chunk_size": 0
}
}
rag_tool = RagTool(config=config, summarize=True)
```
The internal RAG tool utilizes the Embedchain adapter, allowing you to pass any configuration options that are supported by Embedchain.
You can refer to the [Embedchain documentation](https://docs.embedchain.ai/components/introduction) for details.
Make sure to review the configuration options available in the .yaml file.
## Conclusion
The `RagTool` provides a powerful way to create and query knowledge bases from various data sources. By leveraging Retrieval-Augmented Generation, it enables agents to access and retrieve relevant information efficiently, enhancing their ability to provide accurate and contextually appropriate responses.

View File

@@ -5,82 +5,6 @@ icon: "clock"
mode: "wide"
---
<Update label="2025년 9월 20일">
## v0.193.2
[GitHub 릴리스 보기](https://github.com/crewAIInc/crewAI/releases/tag/0.193.2)
## 변경 사항
- 올바른 버전을 사용하도록 pyproject 템플릿 업데이트
</Update>
<Update label="2025년 9월 20일">
## v0.193.1
[GitHub 릴리스 보기](https://github.com/crewAIInc/crewAI/releases/tag/0.193.1)
## 변경 사항
- 일련의 사소한 수정 및 린터 개선
</Update>
<Update label="2025년 9월 19일">
## v0.193.0
[GitHub 릴리스 보기](https://github.com/crewAIInc/crewAI/releases/tag/0.193.0)
## 핵심 개선 사항 및 수정 사항
- OpenAI 어댑터 초기화 중 `model` 매개변수 처리 수정
- CI 워크플로에서 테스트 소요 시간 캐시 문제 해결
- 에이전트의 반복 도구 사용과 관련된 불안정한 테스트 수정
- 일관된 모듈 동작을 위해 누락된 이벤트 내보내기를 `__init__.py`에 추가
- 메타데이터 부하를 줄이기 위해 Mem0에서 메시지 저장 제거
- 벡터 검색의 하위 호환성을 위해 L2 거리 메트릭 지원 수정
## 새로운 기능 및 향상 사항
- 스레드 안전한 플랫폼 컨텍스트 관리 도입
- `pytest-split` 실행 최적화를 위한 테스트 소요 시간 캐싱 추가
- 더 나은 추적 제어를 위한 일시적(trace) 개선
- RAG, 지식, 메모리 검색 매개변수를 완전 구성 가능하게 변경
- ChromaDB가 임베딩 함수에 OpenAI API를 사용할 수 있도록 지원
- 사용자 수준 인사이트를 위한 심화된 관찰 가능성 도구 추가
- 인스턴스별 클라이언트를 지원하는 통합 RAG 스토리지 시스템
## 문서 및 가이드
- CrewAI 네이티브 RAG 구현을 반영하도록 `RagTool` 참조 업데이트
- 타입 주석과 도크스트링을 포함해 `langgraph` 및 `openai` 에이전트 어댑터 내부 문서 개선
</Update>
<Update label="2025년 9월 11일">
## v0.186.1
[GitHub 릴리스 보기](https://github.com/crewAIInc/crewAI/releases/tag/0.186.1)
## 변경 사항
- 버전을 찾지 못해 조용히 되돌리는(reversion) 문제 수정
- CLI에서 CrewAI 버전을 0.186.1로 올리고 의존성 업데이트
</Update>
<Update label="2025년 9월 10일">
## v0.186.0
[GitHub 릴리스 보기](https://github.com/crewAIInc/crewAI/releases/tag/0.186.0)
## 변경 사항
- 자세한 변경 사항은 GitHub 릴리스 노트를 참조하세요
</Update>
<Update label="2025년 9월 4일">
## v0.177.0

View File

@@ -27,7 +27,7 @@ mode: "wide"
<Step title="새로운 crew 프로젝트로 이동하기">
<CodeGroup>
```shell Terminal
cd latest_ai_development
cd latest-ai-development
```
</CodeGroup>
</Step>

View File

@@ -5,82 +5,6 @@ icon: "clock"
mode: "wide"
---
<Update label="20 set 2025">
## v0.193.2
[Ver release no GitHub](https://github.com/crewAIInc/crewAI/releases/tag/0.193.2)
## O que Mudou
- Atualizados templates do pyproject para usar a versão correta
</Update>
<Update label="20 set 2025">
## v0.193.1
[Ver release no GitHub](https://github.com/crewAIInc/crewAI/releases/tag/0.193.1)
## O que Mudou
- Série de pequenas correções e melhorias de linter
</Update>
<Update label="19 set 2025">
## v0.193.0
[Ver release no GitHub](https://github.com/crewAIInc/crewAI/releases/tag/0.193.0)
## Melhorias e Correções Principais
- Corrigido manuseio do parâmetro `model` durante a inicialização do adaptador OpenAI
- Resolvidos problemas de cache da duração de testes nos fluxos de CI
- Corrigido teste instável relacionado ao uso repetido de ferramentas pelos agentes
- Adicionadas exportações de eventos ausentes no `__init__.py` para comportamento consistente do módulo
- Removido armazenamento de mensagem dos metadados no Mem0 para reduzir inchaço
- Corrigido suporte à métrica de distância L2 para compatibilidade retroativa na busca vetorial
## Novos Recursos e Melhorias
- Introduzida gestão de contexto de plataforma com segurança de threads
- Adicionado cache da duração de testes para execuções otimizadas do `pytest-split`
- Melhorias de traces efêmeros para melhor controle de rastreamento
- Parâmetros de busca para RAG, conhecimento e memória totalmente configuráveis
- Habilitado ChromaDB para usar a OpenAI API para funções de embedding
- Adicionadas ferramentas de observabilidade mais profundas para insights ao nível do usuário
- Sistema de armazenamento RAG unificado com suporte a cliente específico por instância
## Documentação e Guias
- Atualizadas referências do `RagTool` para refletir a implementação nativa de RAG do CrewAI
- Melhorada documentação interna para adaptadores de agente `langgraph` e `openai` com anotações de tipo e docstrings
</Update>
<Update label="11 set 2025">
## v0.186.1
[Ver release no GitHub](https://github.com/crewAIInc/crewAI/releases/tag/0.186.1)
## O que Mudou
- Corrigida falha silenciosa de reversão quando a versão não era encontrada
- Versão do CrewAI atualizada para 0.186.1 e dependências do CLI atualizadas
</Update>
<Update label="10 set 2025">
## v0.186.0
[Ver release no GitHub](https://github.com/crewAIInc/crewAI/releases/tag/0.186.0)
## O que Mudou
- Consulte as notas de lançamento no GitHub para detalhes completos
</Update>
<Update label="04 set 2025">
## v0.177.0

View File

@@ -27,7 +27,7 @@ Siga os passos abaixo para começar a tripular! 🚣‍♂️
<Step title="Navegue até o novo projeto da sua tripulação">
<CodeGroup>
```shell Terminal
cd latest_ai_development
cd latest-ai-development
```
</CodeGroup>
</Step>

View File

@@ -9,7 +9,7 @@ authors = [
]
dependencies = [
# Core Dependencies
"pydantic>=2.11.9",
"pydantic>=2.4.2",
"openai>=1.13.3",
"litellm==1.74.9",
"instructor>=1.3.3",
@@ -21,12 +21,13 @@ dependencies = [
"opentelemetry-sdk>=1.30.0",
"opentelemetry-exporter-otlp-proto-http>=1.30.0",
# Data Handling
"chromadb~=1.1.0",
"chromadb>=0.5.23",
"tokenizers>=0.20.3",
"onnxruntime==1.22.0",
"openpyxl>=3.1.5",
"pyvis>=0.3.2",
# Authentication and Security
"python-dotenv>=1.1.1",
"python-dotenv>=1.0.0",
"pyjwt>=2.9.0",
# Configuration and Utils
"click>=8.1.7",
@@ -39,7 +40,6 @@ dependencies = [
"blinker>=1.9.0",
"json5>=0.10.0",
"portalocker==2.7.0",
"pydantic-settings>=2.10.1",
]
[project.urls]
@@ -48,9 +48,7 @@ Documentation = "https://docs.crewai.com"
Repository = "https://github.com/crewAIInc/crewAI"
[project.optional-dependencies]
tools = [
"crewai-tools>=0.74.0",
]
tools = ["crewai-tools~=0.71.0"]
embeddings = [
"tiktoken~=0.8.0"
]
@@ -73,30 +71,24 @@ aisuite = [
qdrant = [
"qdrant-client[fastembed]>=1.14.3",
]
aws = [
"boto3>=1.40.38",
]
watson = [
"ibm-watsonx-ai>=1.3.39",
]
voyageai = [
"voyageai>=0.3.5",
]
[dependency-groups]
dev = [
"ruff>=0.13.1",
"mypy>=1.18.2",
[tool.uv]
dev-dependencies = [
"ruff>=0.12.11",
"mypy>=1.17.1",
"pre-commit>=4.3.0",
"bandit>=1.8.6",
"pytest>=8.4.2",
"pytest-asyncio>=1.2.0",
"pytest-subprocess>=1.5.3",
"pytest-recording>=0.13.4",
"pytest-randomly>=4.0.1",
"pytest-timeout>=2.4.0",
"pytest-xdist>=3.8.0",
"pytest-split>=0.10.0",
"pillow>=10.2.0",
"cairosvg>=2.7.1",
"pytest>=8.0.0",
"python-dotenv>=1.0.0",
"pytest-asyncio>=0.23.7",
"pytest-subprocess>=1.5.2",
"pytest-recording>=0.13.2",
"pytest-randomly>=3.16.0",
"pytest-timeout>=2.3.1",
"pytest-xdist>=3.6.1",
"pytest-split>=0.9.0",
"types-requests==2.32.*",
"types-pyyaml==6.0.*",
"types-regex==2024.11.6.*",
@@ -139,15 +131,13 @@ select = [
"I001", # sort imports
"I002", # remove unused imports
]
ignore = ["E501"] # ignore line too long globally
ignore = ["E501"] # ignore line too long
[tool.ruff.lint.per-file-ignores]
"tests/**/*.py" = ["S101", "RET504"] # Allow assert statements and unnecessary assignments before return in tests
"tests/**/*.py" = ["S101"] # Allow assert statements in tests
[tool.mypy]
exclude = ["src/crewai/cli/templates", "tests/"]
plugins = ["pydantic.mypy"]
exclude = ["src/crewai/cli/templates", "tests"]
[tool.bandit]
exclude_dirs = ["src/crewai/cli/templates"]

View File

@@ -40,7 +40,7 @@ def _suppress_pydantic_deprecation_warnings() -> None:
_suppress_pydantic_deprecation_warnings()
__version__ = "0.201.1"
__version__ = "0.186.1"
_telemetry_submitted = False

View File

@@ -1,10 +1,17 @@
import shutil
import subprocess
import time
from collections.abc import Callable, Sequence
from typing import (
Any,
Callable,
Dict,
List,
Literal,
Optional,
Sequence,
Tuple,
Type,
Union,
)
from pydantic import Field, InstanceOf, PrivateAttr, model_validator
@@ -12,31 +19,12 @@ from pydantic import Field, InstanceOf, PrivateAttr, model_validator
from crewai.agents import CacheHandler
from crewai.agents.agent_builder.base_agent import BaseAgent
from crewai.agents.crew_agent_executor import CrewAgentExecutor
from crewai.events.event_bus import crewai_event_bus
from crewai.events.types.agent_events import (
AgentExecutionCompletedEvent,
AgentExecutionErrorEvent,
AgentExecutionStartedEvent,
)
from crewai.events.types.knowledge_events import (
KnowledgeQueryCompletedEvent,
KnowledgeQueryFailedEvent,
KnowledgeQueryStartedEvent,
KnowledgeRetrievalCompletedEvent,
KnowledgeRetrievalStartedEvent,
KnowledgeSearchQueryFailedEvent,
)
from crewai.events.types.memory_events import (
MemoryRetrievalCompletedEvent,
MemoryRetrievalStartedEvent,
)
from crewai.knowledge.knowledge import Knowledge
from crewai.knowledge.source.base_knowledge_source import BaseKnowledgeSource
from crewai.knowledge.utils.knowledge_utils import extract_knowledge_context
from crewai.lite_agent import LiteAgent, LiteAgentOutput
from crewai.llm import BaseLLM
from crewai.memory.contextual.contextual_memory import ContextualMemory
from crewai.rag.embeddings.types import EmbedderConfig
from crewai.security import Fingerprint
from crewai.task import Task
from crewai.tools import BaseTool
@@ -50,6 +38,24 @@ from crewai.utilities.agent_utils import (
)
from crewai.utilities.constants import TRAINED_AGENTS_DATA_FILE, TRAINING_DATA_FILE
from crewai.utilities.converter import generate_model_description
from crewai.events.types.agent_events import (
AgentExecutionCompletedEvent,
AgentExecutionErrorEvent,
AgentExecutionStartedEvent,
)
from crewai.events.event_bus import crewai_event_bus
from crewai.events.types.memory_events import (
MemoryRetrievalStartedEvent,
MemoryRetrievalCompletedEvent,
)
from crewai.events.types.knowledge_events import (
KnowledgeQueryCompletedEvent,
KnowledgeQueryFailedEvent,
KnowledgeQueryStartedEvent,
KnowledgeRetrievalCompletedEvent,
KnowledgeRetrievalStartedEvent,
KnowledgeSearchQueryFailedEvent,
)
from crewai.utilities.llm_utils import create_llm
from crewai.utilities.token_counter_callback import TokenCalcHandler
from crewai.utilities.training_handler import CrewTrainingHandler
@@ -81,36 +87,36 @@ class Agent(BaseAgent):
"""
_times_executed: int = PrivateAttr(default=0)
max_execution_time: int | None = Field(
max_execution_time: Optional[int] = Field(
default=None,
description="Maximum execution time for an agent to execute a task",
)
agent_ops_agent_name: str = None # type: ignore # Incompatible types in assignment (expression has type "None", variable has type "str")
agent_ops_agent_id: str = None # type: ignore # Incompatible types in assignment (expression has type "None", variable has type "str")
step_callback: Any | None = Field(
step_callback: Optional[Any] = Field(
default=None,
description="Callback to be executed after each step of the agent execution.",
)
use_system_prompt: bool | None = Field(
use_system_prompt: Optional[bool] = Field(
default=True,
description="Use system prompt for the agent.",
)
llm: str | InstanceOf[BaseLLM] | Any = Field(
llm: Union[str, InstanceOf[BaseLLM], Any] = Field(
description="Language model that will run the agent.", default=None
)
function_calling_llm: str | InstanceOf[BaseLLM] | Any | None = Field(
function_calling_llm: Optional[Union[str, InstanceOf[BaseLLM], Any]] = Field(
description="Language model that will run the agent.", default=None
)
system_template: str | None = Field(
system_template: Optional[str] = Field(
default=None, description="System format for the agent."
)
prompt_template: str | None = Field(
prompt_template: Optional[str] = Field(
default=None, description="Prompt format for the agent."
)
response_template: str | None = Field(
response_template: Optional[str] = Field(
default=None, description="Response format for the agent."
)
allow_code_execution: bool | None = Field(
allow_code_execution: Optional[bool] = Field(
default=False, description="Enable code execution for the agent."
)
respect_context_window: bool = Field(
@@ -141,31 +147,31 @@ class Agent(BaseAgent):
default=False,
description="Whether the agent should reflect and create a plan before executing a task.",
)
max_reasoning_attempts: int | None = Field(
max_reasoning_attempts: Optional[int] = Field(
default=None,
description="Maximum number of reasoning attempts before executing the task. If None, will try until ready.",
)
embedder: EmbedderConfig | None = Field(
embedder: Optional[Dict[str, Any]] = Field(
default=None,
description="Embedder configuration for the agent.",
)
agent_knowledge_context: str | None = Field(
agent_knowledge_context: Optional[str] = Field(
default=None,
description="Knowledge context for the agent.",
)
crew_knowledge_context: str | None = Field(
crew_knowledge_context: Optional[str] = Field(
default=None,
description="Knowledge context for the crew.",
)
knowledge_search_query: str | None = Field(
knowledge_search_query: Optional[str] = Field(
default=None,
description="Knowledge search query for the agent dynamically generated by the agent.",
)
from_repository: str | None = Field(
from_repository: Optional[str] = Field(
default=None,
description="The Agent's role to be used from your repository.",
)
guardrail: Callable[[Any], tuple[bool, Any]] | str | None = Field(
guardrail: Optional[Union[Callable[[Any], Tuple[bool, Any]], str]] = Field(
default=None,
description="Function or string description of a guardrail to validate agent output",
)
@@ -174,7 +180,7 @@ class Agent(BaseAgent):
)
@model_validator(mode="before")
def validate_from_repository(cls, v): # noqa: N805
def validate_from_repository(cls, v):
if v is not None and (from_repository := v.get("from_repository")):
return load_agent_from_repository(from_repository) | v
return v
@@ -202,7 +208,7 @@ class Agent(BaseAgent):
self.cache_handler = CacheHandler()
self.set_cache_handler(self.cache_handler)
def set_knowledge(self, crew_embedder: EmbedderConfig | None = None):
def set_knowledge(self, crew_embedder: Optional[Dict[str, Any]] = None):
try:
if self.embedder is None and crew_embedder:
self.embedder = crew_embedder
@@ -218,7 +224,7 @@ class Agent(BaseAgent):
)
self.knowledge.add_sources()
except (TypeError, ValueError) as e:
raise ValueError(f"Invalid Knowledge Configuration: {e!s}") from e
raise ValueError(f"Invalid Knowledge Configuration: {str(e)}")
def _is_any_available_memory(self) -> bool:
"""Check if any memory is available."""
@@ -238,8 +244,8 @@ class Agent(BaseAgent):
def execute_task(
self,
task: Task,
context: str | None = None,
tools: list[BaseTool] | None = None,
context: Optional[str] = None,
tools: Optional[List[BaseTool]] = None,
) -> str:
"""Execute a task with the agent.
@@ -272,9 +278,11 @@ class Agent(BaseAgent):
task.description += f"\n\nReasoning Plan:\n{reasoning_output.plan.plan}"
except Exception as e:
if hasattr(self, "_logger"):
self._logger.log("error", f"Error during reasoning process: {e!s}")
self._logger.log(
"error", f"Error during reasoning process: {str(e)}"
)
else:
print(f"Error during reasoning process: {e!s}")
print(f"Error during reasoning process: {str(e)}")
self._inject_date_to_task(task)
@@ -327,7 +335,7 @@ class Agent(BaseAgent):
agent=self,
task=task,
)
memory = contextual_memory.build_context_for_task(task, context) # type: ignore[arg-type]
memory = contextual_memory.build_context_for_task(task, context)
if memory.strip() != "":
task_prompt += self.i18n.slice("memory").format(memory=memory)
@@ -517,14 +525,14 @@ class Agent(BaseAgent):
try:
return future.result(timeout=timeout)
except concurrent.futures.TimeoutError as e:
except concurrent.futures.TimeoutError:
future.cancel()
raise TimeoutError(
f"Task '{task.description}' execution timed out after {timeout} seconds. Consider increasing max_execution_time or optimizing the task."
) from e
)
except Exception as e:
future.cancel()
raise RuntimeError(f"Task execution failed: {e!s}") from e
raise RuntimeError(f"Task execution failed: {str(e)}")
def _execute_without_timeout(self, task_prompt: str, task: Task) -> str:
"""Execute a task without a timeout.
@@ -546,14 +554,14 @@ class Agent(BaseAgent):
)["output"]
def create_agent_executor(
self, tools: list[BaseTool] | None = None, task=None
self, tools: Optional[List[BaseTool]] = None, task=None
) -> None:
"""Create an agent executor for the agent.
Returns:
An instance of the CrewAgentExecutor class.
"""
raw_tools: list[BaseTool] = tools or self.tools or []
raw_tools: List[BaseTool] = tools or self.tools or []
parsed_tools = parse_tools(raw_tools)
prompt = Prompts(
@@ -579,7 +587,7 @@ class Agent(BaseAgent):
agent=self,
crew=self.crew,
tools=parsed_tools,
prompt=prompt, # type: ignore[arg-type]
prompt=prompt,
original_tools=raw_tools,
stop_words=stop_words,
max_iter=self.max_iter,
@@ -595,9 +603,10 @@ class Agent(BaseAgent):
callbacks=[TokenCalcHandler(self._token_process)],
)
def get_delegation_tools(self, agents: list[BaseAgent]):
def get_delegation_tools(self, agents: List[BaseAgent]):
agent_tools = AgentTools(agents=agents)
return agent_tools.tools()
tools = agent_tools.tools()
return tools
def get_multimodal_tools(self) -> Sequence[BaseTool]:
from crewai.tools.agent_tools.add_image_tool import AddImageTool
@@ -645,7 +654,7 @@ class Agent(BaseAgent):
)
return task_prompt
def _render_text_description(self, tools: list[Any]) -> str:
def _render_text_description(self, tools: List[Any]) -> str:
"""Render the tool name and description in plain text.
Output will be in the format of:
@@ -655,13 +664,15 @@ class Agent(BaseAgent):
search: This tool is used for search
calculator: This tool is used for math
"""
return "\n".join(
description = "\n".join(
[
f"Tool name: {tool.name}\nTool description:\n{tool.description}"
for tool in tools
]
)
return description
def _inject_date_to_task(self, task):
"""Inject the current date into the task description if inject_date is enabled."""
if self.inject_date:
@@ -685,13 +696,13 @@ class Agent(BaseAgent):
if not is_valid:
raise ValueError(f"Invalid date format: {self.date_format}")
current_date = datetime.now().strftime(self.date_format)
current_date: str = datetime.now().strftime(self.date_format)
task.description += f"\n\nCurrent Date: {current_date}"
except Exception as e:
if hasattr(self, "_logger"):
self._logger.log("warning", f"Failed to inject date: {e!s}")
self._logger.log("warning", f"Failed to inject date: {str(e)}")
else:
print(f"Warning: Failed to inject date: {e!s}")
print(f"Warning: Failed to inject date: {str(e)}")
def _validate_docker_installation(self) -> None:
"""Check if Docker is installed and running."""
@@ -702,15 +713,15 @@ class Agent(BaseAgent):
try:
subprocess.run(
["/usr/bin/docker", "info"],
["docker", "info"],
check=True,
stdout=subprocess.PIPE,
stderr=subprocess.PIPE,
)
except subprocess.CalledProcessError as e:
except subprocess.CalledProcessError:
raise RuntimeError(
f"Docker is not running. Please start Docker to use code execution with agent: {self.role}"
) from e
)
def __repr__(self):
return f"Agent(role={self.role}, goal={self.goal}, backstory={self.backstory})"
@@ -785,8 +796,8 @@ class Agent(BaseAgent):
def kickoff(
self,
messages: str | list[dict[str, str]],
response_format: type[Any] | None = None,
messages: Union[str, List[Dict[str, str]]],
response_format: Optional[Type[Any]] = None,
) -> LiteAgentOutput:
"""
Execute the agent with the given messages using a LiteAgent instance.
@@ -825,8 +836,8 @@ class Agent(BaseAgent):
async def kickoff_async(
self,
messages: str | list[dict[str, str]],
response_format: type[Any] | None = None,
messages: Union[str, List[Dict[str, str]]],
response_format: Optional[Type[Any]] = None,
) -> LiteAgentOutput:
"""
Execute the agent asynchronously with the given messages using a LiteAgent instance.

View File

@@ -1,12 +1,5 @@
from crewai.agents.cache.cache_handler import CacheHandler
from crewai.agents.parser import AgentAction, AgentFinish, OutputParserError, parse
from crewai.agents.parser import parse, AgentAction, AgentFinish, OutputParserException
from crewai.agents.tools_handler import ToolsHandler
__all__ = [
"AgentAction",
"AgentFinish",
"CacheHandler",
"OutputParserError",
"ToolsHandler",
"parse",
]
__all__ = ["CacheHandler", "parse", "AgentAction", "AgentFinish", "OutputParserException", "ToolsHandler"]

View File

@@ -1,7 +1,7 @@
from abc import ABC, abstractmethod
from typing import Any
from typing import Any, Dict, List, Optional
from pydantic import ConfigDict, PrivateAttr
from pydantic import PrivateAttr
from crewai.agent import BaseAgent
from crewai.tools import BaseTool
@@ -16,21 +16,22 @@ class BaseAgentAdapter(BaseAgent, ABC):
"""
adapted_structured_output: bool = False
_agent_config: dict[str, Any] | None = PrivateAttr(default=None)
_agent_config: Optional[Dict[str, Any]] = PrivateAttr(default=None)
model_config = ConfigDict(arbitrary_types_allowed=True)
model_config = {"arbitrary_types_allowed": True}
def __init__(self, agent_config: dict[str, Any] | None = None, **kwargs: Any):
def __init__(self, agent_config: Optional[Dict[str, Any]] = None, **kwargs: Any):
super().__init__(adapted_agent=True, **kwargs)
self._agent_config = agent_config
@abstractmethod
def configure_tools(self, tools: list[BaseTool] | None = None) -> None:
def configure_tools(self, tools: Optional[List[BaseTool]] = None) -> None:
"""Configure and adapt tools for the specific agent implementation.
Args:
tools: Optional list of BaseTool instances to be configured
"""
pass
def configure_structured_output(self, structured_output: Any) -> None:
"""Configure the structured output for the specific agent implementation.
@@ -38,3 +39,4 @@ class BaseAgentAdapter(BaseAgent, ABC):
Args:
structured_output: The structured output to be configured
"""
pass

View File

@@ -1,5 +1,5 @@
from abc import ABC, abstractmethod
from typing import Any
from typing import Any, List, Optional
from crewai.tools.base_tool import BaseTool
@@ -12,22 +12,23 @@ class BaseToolAdapter(ABC):
different frameworks and platforms.
"""
original_tools: list[BaseTool]
converted_tools: list[Any]
original_tools: List[BaseTool]
converted_tools: List[Any]
def __init__(self, tools: list[BaseTool] | None = None):
def __init__(self, tools: Optional[List[BaseTool]] = None):
self.original_tools = tools or []
self.converted_tools = []
@abstractmethod
def configure_tools(self, tools: list[BaseTool]) -> None:
def configure_tools(self, tools: List[BaseTool]) -> None:
"""Configure and convert tools for the specific implementation.
Args:
tools: List of BaseTool instances to be configured and converted
"""
pass
def tools(self) -> list[Any]:
def tools(self) -> List[Any]:
"""Return all converted tools."""
return self.converted_tools

View File

@@ -77,9 +77,9 @@ class OpenAIAgentAdapter(BaseAgentAdapter):
Raises:
ImportError: If OpenAI agent dependencies are not installed.
"""
self.llm = kwargs.pop("model", "gpt-4o-mini")
super().__init__(**kwargs)
self._tool_adapter = OpenAIAgentToolAdapter(tools=kwargs.get("tools"))
self.llm = kwargs.get("model", "gpt-4o-mini")
self._converter_adapter = OpenAIConverterAdapter(agent_adapter=self)
def _build_system_prompt(self) -> str:

View File

@@ -1,9 +1,8 @@
import uuid
from abc import ABC, abstractmethod
from collections.abc import Callable
from copy import copy as shallow_copy
from hashlib import md5
from typing import Any, TypeVar
from typing import Any, Callable, Dict, List, Optional, TypeVar
from pydantic import (
UUID4,
@@ -22,11 +21,11 @@ from crewai.agents.tools_handler import ToolsHandler
from crewai.knowledge.knowledge import Knowledge
from crewai.knowledge.knowledge_config import KnowledgeConfig
from crewai.knowledge.source.base_knowledge_source import BaseKnowledgeSource
from crewai.rag.embeddings.types import EmbedderConfig
from crewai.security.security_config import SecurityConfig
from crewai.tools.base_tool import BaseTool, Tool
from crewai.utilities import I18N, Logger, RPMController
from crewai.utilities.config import process_config
from crewai.utilities.converter import Converter
from crewai.utilities.string_utils import interpolate_only
T = TypeVar("T", bound="BaseAgent")
@@ -82,17 +81,17 @@ class BaseAgent(ABC, BaseModel):
__hash__ = object.__hash__ # type: ignore
_logger: Logger = PrivateAttr(default_factory=lambda: Logger(verbose=False))
_rpm_controller: RPMController | None = PrivateAttr(default=None)
_rpm_controller: Optional[RPMController] = PrivateAttr(default=None)
_request_within_rpm_limit: Any = PrivateAttr(default=None)
_original_role: str | None = PrivateAttr(default=None)
_original_goal: str | None = PrivateAttr(default=None)
_original_backstory: str | None = PrivateAttr(default=None)
_original_role: Optional[str] = PrivateAttr(default=None)
_original_goal: Optional[str] = PrivateAttr(default=None)
_original_backstory: Optional[str] = PrivateAttr(default=None)
_token_process: TokenProcess = PrivateAttr(default_factory=TokenProcess)
id: UUID4 = Field(default_factory=uuid.uuid4, frozen=True)
role: str = Field(description="Role of the agent")
goal: str = Field(description="Objective of the agent")
backstory: str = Field(description="Backstory of the agent")
config: dict[str, Any] | None = Field(
config: Optional[Dict[str, Any]] = Field(
description="Configuration for the agent", default=None, exclude=True
)
cache: bool = Field(
@@ -101,7 +100,7 @@ class BaseAgent(ABC, BaseModel):
verbose: bool = Field(
default=False, description="Verbose mode for the Agent Execution"
)
max_rpm: int | None = Field(
max_rpm: Optional[int] = Field(
default=None,
description="Maximum number of requests per minute for the agent execution to be respected.",
)
@@ -109,7 +108,7 @@ class BaseAgent(ABC, BaseModel):
default=False,
description="Enable agent to delegate and ask questions among each other.",
)
tools: list[BaseTool] | None = Field(
tools: Optional[List[BaseTool]] = Field(
default_factory=list, description="Tools at agents' disposal"
)
max_iter: int = Field(
@@ -123,27 +122,27 @@ class BaseAgent(ABC, BaseModel):
)
crew: Any = Field(default=None, description="Crew to which the agent belongs.")
i18n: I18N = Field(default=I18N(), description="Internationalization settings.")
cache_handler: InstanceOf[CacheHandler] | None = Field(
cache_handler: Optional[InstanceOf[CacheHandler]] = Field(
default=None, description="An instance of the CacheHandler class."
)
tools_handler: InstanceOf[ToolsHandler] = Field(
default_factory=ToolsHandler,
description="An instance of the ToolsHandler class.",
)
tools_results: list[dict[str, Any]] = Field(
tools_results: List[Dict[str, Any]] = Field(
default=[], description="Results of the tools used by the agent."
)
max_tokens: int | None = Field(
max_tokens: Optional[int] = Field(
default=None, description="Maximum number of tokens for the agent's execution."
)
knowledge: Knowledge | None = Field(
knowledge: Optional[Knowledge] = Field(
default=None, description="Knowledge for the agent."
)
knowledge_sources: list[BaseKnowledgeSource] | None = Field(
knowledge_sources: Optional[List[BaseKnowledgeSource]] = Field(
default=None,
description="Knowledge sources for the agent.",
)
knowledge_storage: Any | None = Field(
knowledge_storage: Optional[Any] = Field(
default=None,
description="Custom knowledge storage for the agent.",
)
@@ -151,13 +150,13 @@ class BaseAgent(ABC, BaseModel):
default_factory=SecurityConfig,
description="Security configuration for the agent, including fingerprinting.",
)
callbacks: list[Callable] = Field(
callbacks: List[Callable] = Field(
default=[], description="Callbacks to be used for the agent"
)
adapted_agent: bool = Field(
default=False, description="Whether the agent is adapted"
)
knowledge_config: KnowledgeConfig | None = Field(
knowledge_config: Optional[KnowledgeConfig] = Field(
default=None,
description="Knowledge configuration for the agent such as limits and threshold",
)
@@ -169,7 +168,7 @@ class BaseAgent(ABC, BaseModel):
@field_validator("tools")
@classmethod
def validate_tools(cls, tools: list[Any]) -> list[BaseTool]:
def validate_tools(cls, tools: List[Any]) -> List[BaseTool]:
"""Validate and process the tools provided to the agent.
This method ensures that each tool is either an instance of BaseTool
@@ -222,7 +221,7 @@ class BaseAgent(ABC, BaseModel):
@field_validator("id", mode="before")
@classmethod
def _deny_user_set_id(cls, v: UUID4 | None) -> None:
def _deny_user_set_id(cls, v: Optional[UUID4]) -> None:
if v:
raise PydanticCustomError(
"may_not_set_field", "This field is not to be set by the user.", {}
@@ -253,8 +252,8 @@ class BaseAgent(ABC, BaseModel):
def execute_task(
self,
task: Any,
context: str | None = None,
tools: list[BaseTool] | None = None,
context: Optional[str] = None,
tools: Optional[List[BaseTool]] = None,
) -> str:
pass
@@ -263,8 +262,9 @@ class BaseAgent(ABC, BaseModel):
pass
@abstractmethod
def get_delegation_tools(self, agents: list["BaseAgent"]) -> list[BaseTool]:
def get_delegation_tools(self, agents: List["BaseAgent"]) -> List[BaseTool]:
"""Set the task tools that init BaseAgenTools class."""
pass
def copy(self: T) -> T: # type: ignore # Signature of "copy" incompatible with supertype "BaseModel"
"""Create a deep copy of the Agent."""
@@ -309,7 +309,7 @@ class BaseAgent(ABC, BaseModel):
copied_data = self.model_dump(exclude=exclude)
copied_data = {k: v for k, v in copied_data.items() if v is not None}
return type(self)(
copied_agent = type(self)(
**copied_data,
llm=existing_llm,
tools=self.tools,
@@ -318,7 +318,9 @@ class BaseAgent(ABC, BaseModel):
knowledge_storage=copied_knowledge_storage,
)
def interpolate_inputs(self, inputs: dict[str, Any]) -> None:
return copied_agent
def interpolate_inputs(self, inputs: Dict[str, Any]) -> None:
"""Interpolate inputs into the agent description and backstory."""
if self._original_role is None:
self._original_role = self.role
@@ -360,5 +362,5 @@ class BaseAgent(ABC, BaseModel):
self._rpm_controller = rpm_controller
self.create_agent_executor()
def set_knowledge(self, crew_embedder: EmbedderConfig | None = None):
def set_knowledge(self, crew_embedder: Optional[Dict[str, Any]] = None):
pass

View File

@@ -1,13 +1,13 @@
import time
from typing import TYPE_CHECKING
from typing import TYPE_CHECKING, Dict, List
from crewai.events.event_listener import event_listener
from crewai.memory.entity.entity_memory_item import EntityMemoryItem
from crewai.memory.long_term.long_term_memory_item import LongTermMemoryItem
from crewai.utilities import I18N
from crewai.utilities.converter import ConverterError
from crewai.utilities.evaluators.task_evaluator import TaskEvaluator
from crewai.utilities.printer import Printer
from crewai.events.event_listener import event_listener
if TYPE_CHECKING:
from crewai.agents.agent_builder.base_agent import BaseAgent
@@ -21,7 +21,7 @@ class CrewAgentExecutorMixin:
task: "Task"
iterations: int
max_iter: int
messages: list[dict[str, str]]
messages: List[Dict[str, str]]
_i18n: I18N
_printer: Printer = Printer()
@@ -46,6 +46,7 @@ class CrewAgentExecutorMixin:
)
except Exception as e:
print(f"Failed to add to short term memory: {e}")
pass
def _create_external_memory(self, output) -> None:
"""Create and save a external-term memory item if conditions are met."""
@@ -66,6 +67,7 @@ class CrewAgentExecutorMixin:
)
except Exception as e:
print(f"Failed to add to external memory: {e}")
pass
def _create_long_term_memory(self, output) -> None:
"""Create and save long-term and entity memory items based on evaluation."""
@@ -111,8 +113,10 @@ class CrewAgentExecutorMixin:
self.crew._entity_memory.save(entity_memories)
except AttributeError as e:
print(f"Missing attributes for long term memory: {e}")
pass
except Exception as e:
print(f"Failed to add to long term memory: {e}")
pass
elif (
self.crew
and self.crew._long_term_memory

View File

@@ -12,7 +12,7 @@ from crewai.agents.agent_builder.base_agent_executor_mixin import CrewAgentExecu
from crewai.agents.parser import (
AgentAction,
AgentFinish,
OutputParserError,
OutputParserException,
)
from crewai.agents.tools_handler import ToolsHandler
from crewai.events.event_bus import crewai_event_bus
@@ -228,7 +228,7 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
self._invoke_step_callback(formatted_answer)
self._append_message(formatted_answer.text)
except OutputParserError as e: # noqa: PERF203
except OutputParserException as e:
formatted_answer = handle_output_parser_exception(
e=e,
messages=self.messages,
@@ -251,20 +251,17 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
i18n=self._i18n,
)
continue
handle_unknown_error(self._printer, e)
raise e
else:
handle_unknown_error(self._printer, e)
raise e
finally:
self.iterations += 1
# During the invoke loop, formatted_answer alternates between AgentAction
# (when the agent is using tools) and eventually becomes AgentFinish
# (when the agent reaches a final answer). This check confirms we've
# (when the agent reaches a final answer). This assertion confirms we've
# reached a final answer and helps type checking understand this transition.
if not isinstance(formatted_answer, AgentFinish):
raise RuntimeError(
"Agent execution ended without reaching a final answer. "
f"Got {type(formatted_answer).__name__} instead of AgentFinish."
)
assert isinstance(formatted_answer, AgentFinish)
self._show_logs(formatted_answer)
return formatted_answer
@@ -327,7 +324,9 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
self.agent,
AgentLogsStartedEvent(
agent_role=self.agent.role,
task_description=(self.task.description if self.task else "Not Found"),
task_description=(
getattr(self.task, "description") if self.task else "Not Found"
),
verbose=self.agent.verbose
or (hasattr(self, "crew") and getattr(self.crew, "verbose", False)),
),
@@ -416,7 +415,8 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
"""
prompt = prompt.replace("{input}", inputs["input"])
prompt = prompt.replace("{tool_names}", inputs["tool_names"])
return prompt.replace("{tools}", inputs["tools"])
prompt = prompt.replace("{tools}", inputs["tools"])
return prompt
def _handle_human_feedback(self, formatted_answer: AgentFinish) -> AgentFinish:
"""Process human feedback.

View File

@@ -7,18 +7,18 @@ AgentAction or AgentFinish objects.
from dataclasses import dataclass
from json_repair import repair_json # type: ignore[import-untyped]
from json_repair import repair_json
from crewai.agents.constants import (
ACTION_INPUT_ONLY_REGEX,
ACTION_INPUT_REGEX,
ACTION_REGEX,
ACTION_INPUT_ONLY_REGEX,
FINAL_ANSWER_ACTION,
MISSING_ACTION_AFTER_THOUGHT_ERROR_MESSAGE,
MISSING_ACTION_INPUT_AFTER_ACTION_ERROR_MESSAGE,
UNABLE_TO_REPAIR_JSON_RESULTS,
)
from crewai.utilities.i18n import I18N
from crewai.utilities import I18N
_I18N = I18N()
@@ -43,7 +43,7 @@ class AgentFinish:
text: str
class OutputParserError(Exception):
class OutputParserException(Exception):
"""Exception raised when output parsing fails.
Attributes:
@@ -51,7 +51,7 @@ class OutputParserError(Exception):
"""
def __init__(self, error: str) -> None:
"""Initialize OutputParserError.
"""Initialize OutputParserException.
Args:
error: The error message.
@@ -87,7 +87,7 @@ def parse(text: str) -> AgentAction | AgentFinish:
AgentAction or AgentFinish based on the content.
Raises:
OutputParserError: If the text format is invalid.
OutputParserException: If the text format is invalid.
"""
thought = _extract_thought(text)
includes_answer = FINAL_ANSWER_ACTION in text
@@ -104,7 +104,7 @@ def parse(text: str) -> AgentAction | AgentFinish:
final_answer = final_answer[:-3].rstrip()
return AgentFinish(thought=thought, output=final_answer, text=text)
if action_match:
elif action_match:
action = action_match.group(1)
clean_action = _clean_action(action)
@@ -118,18 +118,19 @@ def parse(text: str) -> AgentAction | AgentFinish:
)
if not ACTION_REGEX.search(text):
raise OutputParserError(
raise OutputParserException(
f"{MISSING_ACTION_AFTER_THOUGHT_ERROR_MESSAGE}\n{_I18N.slice('final_answer_format')}",
)
if not ACTION_INPUT_ONLY_REGEX.search(text):
raise OutputParserError(
elif not ACTION_INPUT_ONLY_REGEX.search(text):
raise OutputParserException(
MISSING_ACTION_INPUT_AFTER_ACTION_ERROR_MESSAGE,
)
err_format = _I18N.slice("format_without_tools")
error = f"{err_format}"
raise OutputParserError(
error,
)
else:
err_format = _I18N.slice("format_without_tools")
error = f"{err_format}"
raise OutputParserException(
error,
)
def _extract_thought(text: str) -> str:
@@ -148,7 +149,8 @@ def _extract_thought(text: str) -> str:
return ""
thought = text[:thought_index].strip()
# Remove any triple backticks from the thought string
return thought.replace("```", "").strip()
thought = thought.replace("```", "").strip()
return thought
def _clean_action(text: str) -> str:

View File

@@ -1,10 +1,8 @@
"""Tools handler for managing tool execution and caching."""
import json
from crewai.agents.cache.cache_handler import CacheHandler
from crewai.tools.cache_tools.cache_tools import CacheTools
from crewai.tools.tool_calling import InstructorToolCalling, ToolCalling
from crewai.agents.cache.cache_handler import CacheHandler
class ToolsHandler:
@@ -39,16 +37,8 @@ class ToolsHandler:
"""
self.last_used_tool = calling
if self.cache and should_cache and calling.tool_name != CacheTools().name:
# Convert arguments to string for cache
input_str = ""
if calling.arguments:
if isinstance(calling.arguments, dict):
input_str = json.dumps(calling.arguments)
else:
input_str = str(calling.arguments)
self.cache.add(
tool=calling.tool_name,
input=input_str,
input=calling.arguments,
output=output,
)

View File

@@ -1,6 +1,5 @@
from crewai.cli.authentication.providers.base_provider import BaseProvider
class Auth0Provider(BaseProvider):
def get_authorize_url(self) -> str:
return f"https://{self._get_domain()}/oauth/device/code"
@@ -15,20 +14,13 @@ class Auth0Provider(BaseProvider):
return f"https://{self._get_domain()}/"
def get_audience(self) -> str:
if self.settings.audience is None:
raise ValueError(
"Audience is required. Please set it in the configuration."
)
assert self.settings.audience is not None, "Audience is required"
return self.settings.audience
def get_client_id(self) -> str:
if self.settings.client_id is None:
raise ValueError(
"Client ID is required. Please set it in the configuration."
)
assert self.settings.client_id is not None, "Client ID is required"
return self.settings.client_id
def _get_domain(self) -> str:
if self.settings.domain is None:
raise ValueError("Domain is required. Please set it in the configuration.")
assert self.settings.domain is not None, "Domain is required"
return self.settings.domain

View File

@@ -1,26 +1,30 @@
from abc import ABC, abstractmethod
from crewai.cli.authentication.main import Oauth2Settings
class BaseProvider(ABC):
def __init__(self, settings: Oauth2Settings):
self.settings = settings
@abstractmethod
def get_authorize_url(self) -> str: ...
def get_authorize_url(self) -> str:
...
@abstractmethod
def get_token_url(self) -> str: ...
def get_token_url(self) -> str:
...
@abstractmethod
def get_jwks_url(self) -> str: ...
def get_jwks_url(self) -> str:
...
@abstractmethod
def get_issuer(self) -> str: ...
def get_issuer(self) -> str:
...
@abstractmethod
def get_audience(self) -> str: ...
def get_audience(self) -> str:
...
@abstractmethod
def get_client_id(self) -> str: ...
def get_client_id(self) -> str:
...

View File

@@ -1,6 +1,5 @@
from crewai.cli.authentication.providers.base_provider import BaseProvider
class OktaProvider(BaseProvider):
def get_authorize_url(self) -> str:
return f"https://{self.settings.domain}/oauth2/default/v1/device/authorize"
@@ -15,15 +14,9 @@ class OktaProvider(BaseProvider):
return f"https://{self.settings.domain}/oauth2/default"
def get_audience(self) -> str:
if self.settings.audience is None:
raise ValueError(
"Audience is required. Please set it in the configuration."
)
assert self.settings.audience is not None
return self.settings.audience
def get_client_id(self) -> str:
if self.settings.client_id is None:
raise ValueError(
"Client ID is required. Please set it in the configuration."
)
assert self.settings.client_id is not None
return self.settings.client_id

View File

@@ -1,6 +1,5 @@
from crewai.cli.authentication.providers.base_provider import BaseProvider
class WorkosProvider(BaseProvider):
def get_authorize_url(self) -> str:
return f"https://{self._get_domain()}/oauth2/device_authorization"
@@ -18,13 +17,9 @@ class WorkosProvider(BaseProvider):
return self.settings.audience or ""
def get_client_id(self) -> str:
if self.settings.client_id is None:
raise ValueError(
"Client ID is required. Please set it in the configuration."
)
assert self.settings.client_id is not None, "Client ID is required"
return self.settings.client_id
def _get_domain(self) -> str:
if self.settings.domain is None:
raise ValueError("Domain is required. Please set it in the configuration.")
assert self.settings.domain is not None, "Domain is required"
return self.settings.domain

View File

@@ -17,6 +17,8 @@ def validate_jwt_token(
missing required claims).
"""
decoded_token = None
try:
jwk_client = PyJWKClient(jwks_url)
signing_key = jwk_client.get_signing_key_from_jwt(jwt_token)
@@ -24,7 +26,7 @@ def validate_jwt_token(
_unverified_decoded_token = jwt.decode(
jwt_token, options={"verify_signature": False}
)
return jwt.decode(
decoded_token = jwt.decode(
jwt_token,
signing_key.key,
algorithms=["RS256"],
@@ -38,22 +40,23 @@ def validate_jwt_token(
"require": ["exp", "iat", "iss", "aud", "sub"],
},
)
return decoded_token
except jwt.ExpiredSignatureError as e:
raise Exception("Token has expired.") from e
except jwt.InvalidAudienceError as e:
except jwt.ExpiredSignatureError:
raise Exception("Token has expired.")
except jwt.InvalidAudienceError:
actual_audience = _unverified_decoded_token.get("aud", "[no audience found]")
raise Exception(
f"Invalid token audience. Got: '{actual_audience}'. Expected: '{audience}'"
) from e
except jwt.InvalidIssuerError as e:
)
except jwt.InvalidIssuerError:
actual_issuer = _unverified_decoded_token.get("iss", "[no issuer found]")
raise Exception(
f"Invalid token issuer. Got: '{actual_issuer}'. Expected: '{issuer}'"
) from e
)
except jwt.MissingRequiredClaimError as e:
raise Exception(f"Token is missing required claims: {e!s}") from e
raise Exception(f"Token is missing required claims: {str(e)}")
except jwt.exceptions.PyJWKClientError as e:
raise Exception(f"JWKS or key processing error: {e!s}") from e
raise Exception(f"JWKS or key processing error: {str(e)}")
except jwt.InvalidTokenError as e:
raise Exception(f"Invalid token: {e!s}") from e
raise Exception(f"Invalid token: {str(e)}")

View File

@@ -1,16 +1,13 @@
import os
import subprocess
from importlib.metadata import version as get_version
from typing import Optional
import click
from crewai.cli.add_crew_to_flow import add_crew_to_flow
from crewai.cli.config import Settings
from crewai.cli.settings.main import SettingsCommand
from crewai.cli.add_crew_to_flow import add_crew_to_flow
from crewai.cli.create_crew import create_crew
from crewai.cli.create_flow import create_flow
from crewai.cli.crew_chat import run_chat
from crewai.cli.settings.main import SettingsCommand
from crewai.cli.utils import build_env_with_tool_repository_credentials, read_toml
from crewai.memory.storage.kickoff_task_outputs_storage import (
KickoffTaskOutputsSQLiteStorage,
)
@@ -37,46 +34,6 @@ def crewai():
"""Top-level command group for crewai."""
@crewai.command(
name="uv",
context_settings=dict(
ignore_unknown_options=True,
),
)
@click.argument("uv_args", nargs=-1, type=click.UNPROCESSED)
def uv(uv_args):
"""A wrapper around uv commands that adds custom tool authentication through env vars."""
env = os.environ.copy()
try:
pyproject_data = read_toml()
sources = pyproject_data.get("tool", {}).get("uv", {}).get("sources", {})
for source_config in sources.values():
if isinstance(source_config, dict):
index = source_config.get("index")
if index:
index_env = build_env_with_tool_repository_credentials(index)
env.update(index_env)
except (FileNotFoundError, KeyError) as e:
raise SystemExit(
"Error. A valid pyproject.toml file is required. Check that a valid pyproject.toml file exists in the current directory."
) from e
except Exception as e:
raise SystemExit(f"Error: {e}") from e
try:
subprocess.run( # noqa: S603
["uv", *uv_args], # noqa: S607
capture_output=False,
env=env,
text=True,
check=True,
)
except subprocess.CalledProcessError as e:
click.secho(f"uv command failed with exit code {e.returncode}", fg="red")
raise SystemExit(e.returncode) from e
@crewai.command()
@click.argument("type", type=click.Choice(["crew", "flow"]))
@click.argument("name")
@@ -280,6 +237,13 @@ def login():
@crewai.group()
def deploy():
"""Deploy the Crew CLI group."""
pass
@crewai.group()
def tool():
"""Tool Repository related commands."""
pass
@deploy.command(name="create")
@@ -299,7 +263,7 @@ def deploy_list():
@deploy.command(name="push")
@click.option("-u", "--uuid", type=str, help="Crew UUID parameter")
def deploy_push(uuid: str | None):
def deploy_push(uuid: Optional[str]):
"""Deploy the Crew."""
deploy_cmd = DeployCommand()
deploy_cmd.deploy(uuid=uuid)
@@ -307,7 +271,7 @@ def deploy_push(uuid: str | None):
@deploy.command(name="status")
@click.option("-u", "--uuid", type=str, help="Crew UUID parameter")
def deply_status(uuid: str | None):
def deply_status(uuid: Optional[str]):
"""Get the status of a deployment."""
deploy_cmd = DeployCommand()
deploy_cmd.get_crew_status(uuid=uuid)
@@ -315,7 +279,7 @@ def deply_status(uuid: str | None):
@deploy.command(name="logs")
@click.option("-u", "--uuid", type=str, help="Crew UUID parameter")
def deploy_logs(uuid: str | None):
def deploy_logs(uuid: Optional[str]):
"""Get the logs of a deployment."""
deploy_cmd = DeployCommand()
deploy_cmd.get_crew_logs(uuid=uuid)
@@ -323,17 +287,12 @@ def deploy_logs(uuid: str | None):
@deploy.command(name="remove")
@click.option("-u", "--uuid", type=str, help="Crew UUID parameter")
def deploy_remove(uuid: str | None):
def deploy_remove(uuid: Optional[str]):
"""Remove a deployment."""
deploy_cmd = DeployCommand()
deploy_cmd.remove_crew(uuid=uuid)
@crewai.group()
def tool():
"""Tool Repository related commands."""
@tool.command(name="create")
@click.argument("handle")
def tool_create(handle: str):
@@ -368,6 +327,7 @@ def tool_publish(is_public: bool, force: bool):
@crewai.group()
def flow():
"""Flow related commands."""
pass
@flow.command(name="kickoff")
@@ -399,7 +359,7 @@ def chat():
and using the Chat LLM to generate responses.
"""
click.secho(
"\nStarting a conversation with the Crew\nType 'exit' or Ctrl+C to quit.\n",
"\nStarting a conversation with the Crew\n" "Type 'exit' or Ctrl+C to quit.\n",
)
run_chat()
@@ -408,6 +368,7 @@ def chat():
@crewai.group(invoke_without_command=True)
def org():
"""Organization management commands."""
pass
@org.command("list")
@@ -435,6 +396,7 @@ def current():
@crewai.group()
def enterprise():
"""Enterprise Configuration commands."""
pass
@enterprise.command("configure")
@@ -448,6 +410,7 @@ def enterprise_configure(enterprise_url: str):
@crewai.group()
def config():
"""CLI Configuration commands."""
pass
@config.command("list")

View File

@@ -1,61 +1,20 @@
import json
import tempfile
from logging import getLogger
from pathlib import Path
from typing import Optional
from pydantic import BaseModel, Field
from crewai.cli.constants import (
DEFAULT_CREWAI_ENTERPRISE_URL,
CREWAI_ENTERPRISE_DEFAULT_OAUTH2_PROVIDER,
CREWAI_ENTERPRISE_DEFAULT_OAUTH2_AUDIENCE,
CREWAI_ENTERPRISE_DEFAULT_OAUTH2_CLIENT_ID,
CREWAI_ENTERPRISE_DEFAULT_OAUTH2_DOMAIN,
CREWAI_ENTERPRISE_DEFAULT_OAUTH2_PROVIDER,
DEFAULT_CREWAI_ENTERPRISE_URL,
)
from crewai.cli.shared.token_manager import TokenManager
logger = getLogger(__name__)
DEFAULT_CONFIG_PATH = Path.home() / ".config" / "crewai" / "settings.json"
def get_writable_config_path() -> Path | None:
"""
Find a writable location for the config file with fallback options.
Tries in order:
1. Default: ~/.config/crewai/settings.json
2. Temp directory: /tmp/crewai_settings.json (or OS equivalent)
3. Current directory: ./crewai_settings.json
4. In-memory only (returns None)
Returns:
Path object for writable config location, or None if no writable location found
"""
fallback_paths = [
DEFAULT_CONFIG_PATH, # Default location
Path(tempfile.gettempdir()) / "crewai_settings.json", # Temporary directory
Path.cwd() / "crewai_settings.json", # Current working directory
]
for config_path in fallback_paths:
try:
config_path.parent.mkdir(parents=True, exist_ok=True)
test_file = config_path.parent / ".crewai_write_test"
try:
test_file.write_text("test")
test_file.unlink() # Clean up test file
logger.info(f"Using config path: {config_path}")
return config_path
except Exception: # noqa: S112
continue
except Exception: # noqa: S112
continue
return None
# Settings that are related to the user's account
USER_SETTINGS_KEYS = [
"tool_repository_username",
@@ -97,20 +56,20 @@ HIDDEN_SETTINGS_KEYS = [
class Settings(BaseModel):
enterprise_base_url: str | None = Field(
enterprise_base_url: Optional[str] = Field(
default=DEFAULT_CLI_SETTINGS["enterprise_base_url"],
description="Base URL of the CrewAI Enterprise instance",
)
tool_repository_username: str | None = Field(
tool_repository_username: Optional[str] = Field(
None, description="Username for interacting with the Tool Repository"
)
tool_repository_password: str | None = Field(
tool_repository_password: Optional[str] = Field(
None, description="Password for interacting with the Tool Repository"
)
org_name: str | None = Field(
org_name: Optional[str] = Field(
None, description="Name of the currently active organization"
)
org_uuid: str | None = Field(
org_uuid: Optional[str] = Field(
None, description="UUID of the currently active organization"
)
config_path: Path = Field(default=DEFAULT_CONFIG_PATH, frozen=True, exclude=True)
@@ -120,7 +79,7 @@ class Settings(BaseModel):
default=DEFAULT_CLI_SETTINGS["oauth2_provider"],
)
oauth2_audience: str | None = Field(
oauth2_audience: Optional[str] = Field(
description="OAuth2 audience value, typically used to identify the target API or resource.",
default=DEFAULT_CLI_SETTINGS["oauth2_audience"],
)
@@ -135,32 +94,16 @@ class Settings(BaseModel):
default=DEFAULT_CLI_SETTINGS["oauth2_domain"],
)
def __init__(self, config_path: Path | None = None, **data):
"""Load Settings from config path with fallback support"""
if config_path is None:
config_path = get_writable_config_path()
# If config_path is None, we're in memory-only mode
if config_path is None:
merged_data = {**data}
# Dummy path for memory-only mode
super().__init__(config_path=Path("/dev/null"), **merged_data)
return
try:
config_path.parent.mkdir(parents=True, exist_ok=True)
except Exception:
merged_data = {**data}
# Dummy path for memory-only mode
super().__init__(config_path=Path("/dev/null"), **merged_data)
return
def __init__(self, config_path: Path = DEFAULT_CONFIG_PATH, **data):
"""Load Settings from config path"""
config_path.parent.mkdir(parents=True, exist_ok=True)
file_data = {}
if config_path.is_file():
try:
with config_path.open("r") as f:
file_data = json.load(f)
except Exception:
except json.JSONDecodeError:
file_data = {}
merged_data = {**file_data, **data}
@@ -180,22 +123,15 @@ class Settings(BaseModel):
def dump(self) -> None:
"""Save current settings to settings.json"""
if str(self.config_path) == "/dev/null":
return
if self.config_path.is_file():
with self.config_path.open("r") as f:
existing_data = json.load(f)
else:
existing_data = {}
try:
if self.config_path.is_file():
with self.config_path.open("r") as f:
existing_data = json.load(f)
else:
existing_data = {}
updated_data = {**existing_data, **self.model_dump(exclude_unset=True)}
with self.config_path.open("w") as f:
json.dump(updated_data, f, indent=4)
except Exception: # noqa: S110
pass
updated_data = {**existing_data, **self.model_dump(exclude_unset=True)}
with self.config_path.open("w") as f:
json.dump(updated_data, f, indent=4)
def _reset_user_settings(self) -> None:
"""Reset all user settings to default values"""

View File

@@ -16,72 +16,48 @@ from crewai.cli.utils import copy_template, load_env_vars, write_env_file
def create_folder_structure(name, parent_folder=None):
import keyword
import re
name = name.rstrip("/")
name = name.rstrip('/')
if not name.strip():
raise ValueError("Project name cannot be empty or contain only whitespace")
folder_name = name.replace(" ", "_").replace("-", "_").lower()
folder_name = re.sub(r"[^a-zA-Z0-9_]", "", folder_name)
folder_name = re.sub(r'[^a-zA-Z0-9_]', '', folder_name)
# Check if the name starts with invalid characters or is primarily invalid
if re.match(r"^[^a-zA-Z0-9_-]+", name):
raise ValueError(
f"Project name '{name}' contains no valid characters for a Python module name"
)
if re.match(r'^[^a-zA-Z0-9_-]+', name):
raise ValueError(f"Project name '{name}' contains no valid characters for a Python module name")
if not folder_name:
raise ValueError(
f"Project name '{name}' contains no valid characters for a Python module name"
)
raise ValueError(f"Project name '{name}' contains no valid characters for a Python module name")
if folder_name[0].isdigit():
raise ValueError(
f"Project name '{name}' would generate folder name '{folder_name}' which cannot start with a digit (invalid Python module name)"
)
raise ValueError(f"Project name '{name}' would generate folder name '{folder_name}' which cannot start with a digit (invalid Python module name)")
if keyword.iskeyword(folder_name):
raise ValueError(
f"Project name '{name}' would generate folder name '{folder_name}' which is a reserved Python keyword"
)
raise ValueError(f"Project name '{name}' would generate folder name '{folder_name}' which is a reserved Python keyword")
if not folder_name.isidentifier():
raise ValueError(
f"Project name '{name}' would generate invalid Python module name '{folder_name}'"
)
raise ValueError(f"Project name '{name}' would generate invalid Python module name '{folder_name}'")
class_name = name.replace("_", " ").replace("-", " ").title().replace(" ", "")
class_name = re.sub(r"[^a-zA-Z0-9_]", "", class_name)
class_name = re.sub(r'[^a-zA-Z0-9_]', '', class_name)
if not class_name:
raise ValueError(
f"Project name '{name}' contains no valid characters for a Python class name"
)
raise ValueError(f"Project name '{name}' contains no valid characters for a Python class name")
if class_name[0].isdigit():
raise ValueError(
f"Project name '{name}' would generate class name '{class_name}' which cannot start with a digit"
)
raise ValueError(f"Project name '{name}' would generate class name '{class_name}' which cannot start with a digit")
# Check if the original name (before title casing) is a keyword
original_name_clean = re.sub(
r"[^a-zA-Z0-9_]", "", name.replace("_", "").replace("-", "").lower()
)
if (
keyword.iskeyword(original_name_clean)
or keyword.iskeyword(class_name)
or class_name in ("True", "False", "None")
):
raise ValueError(
f"Project name '{name}' would generate class name '{class_name}' which is a reserved Python keyword"
)
original_name_clean = re.sub(r'[^a-zA-Z0-9_]', '', name.replace("_", "").replace("-", "").lower())
if keyword.iskeyword(original_name_clean) or keyword.iskeyword(class_name) or class_name in ('True', 'False', 'None'):
raise ValueError(f"Project name '{name}' would generate class name '{class_name}' which is a reserved Python keyword")
if not class_name.isidentifier():
raise ValueError(
f"Project name '{name}' would generate invalid Python class name '{class_name}'"
)
raise ValueError(f"Project name '{name}' would generate invalid Python class name '{class_name}'")
if parent_folder:
folder_path = Path(parent_folder) / folder_name
@@ -196,7 +172,7 @@ def create_crew(name, provider=None, skip_provider=False, parent_folder=None):
)
# Check if the selected provider has predefined models
if MODELS.get(selected_provider):
if selected_provider in MODELS and MODELS[selected_provider]:
while True:
selected_model = select_model(selected_provider, provider_models)
if selected_model is None: # User typed 'q'

View File

@@ -5,7 +5,7 @@ import sys
import threading
import time
from pathlib import Path
from typing import Any
from typing import Any, Dict, List, Optional, Set, Tuple
import click
import tomli
@@ -116,7 +116,7 @@ def show_loading(event: threading.Event):
print()
def initialize_chat_llm(crew: Crew) -> LLM | BaseLLM | None:
def initialize_chat_llm(crew: Crew) -> Optional[LLM | BaseLLM]:
"""Initializes the chat LLM and handles exceptions."""
try:
return create_llm(crew.chat_llm)
@@ -157,7 +157,7 @@ def build_system_message(crew_chat_inputs: ChatInputs) -> str:
)
def create_tool_function(crew: Crew, messages: list[dict[str, str]]) -> Any:
def create_tool_function(crew: Crew, messages: List[Dict[str, str]]) -> Any:
"""Creates a wrapper function for running the crew tool with messages."""
def run_crew_tool_with_messages(**kwargs):
@@ -193,7 +193,7 @@ def chat_loop(chat_llm, messages, crew_tool_schema, available_functions):
user_input, chat_llm, messages, crew_tool_schema, available_functions
)
except KeyboardInterrupt: # noqa: PERF203
except KeyboardInterrupt:
click.echo("\nExiting chat. Goodbye!")
break
except Exception as e:
@@ -221,9 +221,9 @@ def get_user_input() -> str:
def handle_user_input(
user_input: str,
chat_llm: LLM,
messages: list[dict[str, str]],
crew_tool_schema: dict[str, Any],
available_functions: dict[str, Any],
messages: List[Dict[str, str]],
crew_tool_schema: Dict[str, Any],
available_functions: Dict[str, Any],
) -> None:
if user_input.strip().lower() == "exit":
click.echo("Exiting chat. Goodbye!")
@@ -281,7 +281,7 @@ def generate_crew_tool_schema(crew_inputs: ChatInputs) -> dict:
}
def run_crew_tool(crew: Crew, messages: list[dict[str, str]], **kwargs):
def run_crew_tool(crew: Crew, messages: List[Dict[str, str]], **kwargs):
"""
Runs the crew using crew.kickoff(inputs=kwargs) and returns the output.
@@ -304,8 +304,9 @@ def run_crew_tool(crew: Crew, messages: list[dict[str, str]], **kwargs):
crew_output = crew.kickoff(inputs=kwargs)
# Convert CrewOutput to a string to send back to the user
return str(crew_output)
result = str(crew_output)
return result
except Exception as e:
# Exit the chat and show the error message
click.secho("An error occurred while running the crew:", fg="red")
@@ -313,7 +314,7 @@ def run_crew_tool(crew: Crew, messages: list[dict[str, str]], **kwargs):
sys.exit(1)
def load_crew_and_name() -> tuple[Crew, str]:
def load_crew_and_name() -> Tuple[Crew, str]:
"""
Loads the crew by importing the crew class from the user's project.
@@ -350,17 +351,15 @@ def load_crew_and_name() -> tuple[Crew, str]:
try:
crew_module = __import__(crew_module_name, fromlist=[crew_class_name])
except ImportError as e:
raise ImportError(
f"Failed to import crew module {crew_module_name}: {e}"
) from e
raise ImportError(f"Failed to import crew module {crew_module_name}: {e}")
# Get the crew class from the module
try:
crew_class = getattr(crew_module, crew_class_name)
except AttributeError as e:
except AttributeError:
raise AttributeError(
f"Crew class {crew_class_name} not found in module {crew_module_name}"
) from e
)
# Instantiate the crew
crew_instance = crew_class().crew()
@@ -396,7 +395,7 @@ def generate_crew_chat_inputs(crew: Crew, crew_name: str, chat_llm) -> ChatInput
)
def fetch_required_inputs(crew: Crew) -> set[str]:
def fetch_required_inputs(crew: Crew) -> Set[str]:
"""
Extracts placeholders from the crew's tasks and agents.
@@ -406,8 +405,8 @@ def fetch_required_inputs(crew: Crew) -> set[str]:
Returns:
Set[str]: A set of placeholder names.
"""
placeholder_pattern = re.compile(r"\{(.+?)}")
required_inputs: set[str] = set()
placeholder_pattern = re.compile(r"\{(.+?)\}")
required_inputs: Set[str] = set()
# Scan tasks
for task in crew.tasks:
@@ -436,7 +435,7 @@ def generate_input_description_with_ai(input_name: str, crew: Crew, chat_llm) ->
"""
# Gather context from tasks and agents where the input is used
context_texts = []
placeholder_pattern = re.compile(r"\{(.+?)}")
placeholder_pattern = re.compile(r"\{(.+?)\}")
for task in crew.tasks:
if (
@@ -480,7 +479,9 @@ def generate_input_description_with_ai(input_name: str, crew: Crew, chat_llm) ->
f"{context}"
)
response = chat_llm.call(messages=[{"role": "user", "content": prompt}])
return response.strip()
description = response.strip()
return description
def generate_crew_description_with_ai(crew: Crew, chat_llm) -> str:
@@ -496,7 +497,7 @@ def generate_crew_description_with_ai(crew: Crew, chat_llm) -> str:
"""
# Gather context from tasks and agents
context_texts = []
placeholder_pattern = re.compile(r"\{(.+?)}")
placeholder_pattern = re.compile(r"\{(.+?)\}")
for task in crew.tasks:
# Replace placeholders with input names
@@ -530,4 +531,6 @@ def generate_crew_description_with_ai(crew: Crew, chat_llm) -> str:
f"{context}"
)
response = chat_llm.call(messages=[{"role": "user", "content": prompt}])
return response.strip()
crew_description = response.strip()
return crew_description

View File

@@ -14,15 +14,11 @@ class Repository:
self.fetch()
@staticmethod
def is_git_installed() -> bool:
def is_git_installed(self) -> bool:
"""Check if Git is installed and available in the system."""
try:
subprocess.run(
["git", "--version"], # noqa: S607
capture_output=True,
check=True,
text=True,
["git", "--version"], capture_output=True, check=True, text=True
)
return True
except (subprocess.CalledProcessError, FileNotFoundError):
@@ -30,26 +26,22 @@ class Repository:
def fetch(self) -> None:
"""Fetch latest updates from the remote."""
subprocess.run(["git", "fetch"], cwd=self.path, check=True) # noqa: S607
subprocess.run(["git", "fetch"], cwd=self.path, check=True)
def status(self) -> str:
"""Get the git status in porcelain format."""
return subprocess.check_output(
["git", "status", "--branch", "--porcelain"], # noqa: S607
["git", "status", "--branch", "--porcelain"],
cwd=self.path,
encoding="utf-8",
).strip()
@lru_cache(maxsize=None) # noqa: B019
@lru_cache(maxsize=None)
def is_git_repo(self) -> bool:
"""Check if the current directory is a git repository.
Notes:
- TODO: This method is cached to avoid redundant checks, but using lru_cache on methods can lead to memory leaks
"""
"""Check if the current directory is a git repository."""
try:
subprocess.check_output(
["git", "rev-parse", "--is-inside-work-tree"], # noqa: S607
["git", "rev-parse", "--is-inside-work-tree"],
cwd=self.path,
encoding="utf-8",
)
@@ -72,13 +64,14 @@ class Repository:
"""Return True if the Git repository is fully synced with the remote, False otherwise."""
if self.has_uncommitted_changes() or self.is_ahead_or_behind():
return False
return True
else:
return True
def origin_url(self) -> str | None:
"""Get the Git repository's remote URL."""
try:
result = subprocess.run(
["git", "remote", "get-url", "origin"], # noqa: S607
["git", "remote", "get-url", "origin"],
cwd=self.path,
capture_output=True,
text=True,

View File

@@ -12,8 +12,8 @@ def install_crew(proxy_options: list[str]) -> None:
Install the crew by running the UV command to lock and install.
"""
try:
command = ["uv", "sync", *proxy_options]
subprocess.run(command, check=True, capture_output=False, text=True) # noqa: S603
command = ["uv", "sync"] + proxy_options
subprocess.run(command, check=True, capture_output=False, text=True)
except subprocess.CalledProcessError as e:
click.echo(f"An error occurred while running the crew: {e}", err=True)

View File

@@ -1,10 +1,11 @@
from typing import List, Optional
from urllib.parse import urljoin
import requests
from crewai.cli.config import Settings
from crewai.cli.constants import DEFAULT_CREWAI_ENTERPRISE_URL
from crewai.cli.version import get_crewai_version
from crewai.cli.constants import DEFAULT_CREWAI_ENTERPRISE_URL
class PlusAPI:
@@ -55,9 +56,9 @@ class PlusAPI:
handle: str,
is_public: bool,
version: str,
description: str | None,
description: Optional[str],
encoded_file: str,
available_exports: list[str] | None = None,
available_exports: Optional[List[str]] = None,
):
params = {
"handle": handle,
@@ -166,13 +167,3 @@ class PlusAPI:
json=payload,
timeout=30,
)
def mark_trace_batch_as_failed(
self, trace_batch_id: str, error_message: str
) -> requests.Response:
return self._make_request(
"PATCH",
f"{self.TRACING_RESOURCE}/batches/{trace_batch_id}",
json={"status": "failed", "failure_reason": error_message},
timeout=30,
)

View File

@@ -1,10 +1,10 @@
import json
import os
import certifi
import json
import time
from collections import defaultdict
from pathlib import Path
import certifi
import click
import requests
@@ -25,7 +25,7 @@ def select_choice(prompt_message, choices):
provider_models = get_provider_data()
if not provider_models:
return None
return
click.secho(prompt_message, fg="cyan")
for idx, choice in enumerate(choices, start=1):
click.secho(f"{idx}. {choice}", fg="cyan")
@@ -67,7 +67,7 @@ def select_provider(provider_models):
all_providers = sorted(set(predefined_providers + list(provider_models.keys())))
provider = select_choice(
"Select a provider to set up:", [*predefined_providers, "other"]
"Select a provider to set up:", predefined_providers + ["other"]
)
if provider is None: # User typed 'q'
return None
@@ -102,9 +102,10 @@ def select_model(provider, provider_models):
click.secho(f"No models available for provider '{provider}'.", fg="red")
return None
return select_choice(
selected_model = select_choice(
f"Select a model to use for {provider.capitalize()}:", available_models
)
return selected_model
def load_provider_data(cache_file, cache_expiry):
@@ -164,7 +165,7 @@ def fetch_provider_data(cache_file):
Returns:
- dict or None: The fetched provider data or None if the operation fails.
"""
ssl_config = os.environ["SSL_CERT_FILE"] = certifi.where()
ssl_config = os.environ['SSL_CERT_FILE'] = certifi.where()
try:
response = requests.get(JSON_URL, stream=True, timeout=60, verify=ssl_config)

View File

@@ -1,5 +1,6 @@
import subprocess
from enum import Enum
from typing import List, Optional
import click
from packaging import version
@@ -56,7 +57,7 @@ def execute_command(crew_type: CrewType) -> None:
command = ["uv", "run", "kickoff" if crew_type == CrewType.FLOW else "run_crew"]
try:
subprocess.run(command, capture_output=False, text=True, check=True) # noqa: S603
subprocess.run(command, capture_output=False, text=True, check=True)
except subprocess.CalledProcessError as e:
handle_error(e, crew_type)

View File

@@ -3,7 +3,7 @@ import os
import sys
from datetime import datetime
from pathlib import Path
from typing import Optional
from cryptography.fernet import Fernet
@@ -49,7 +49,7 @@ class TokenManager:
encrypted_data = self.fernet.encrypt(json.dumps(data).encode())
self.save_secure_file(self.file_path, encrypted_data)
def get_token(self) -> str | None:
def get_token(self) -> Optional[str]:
"""
Get the access token if it is valid and not expired.
@@ -113,7 +113,7 @@ class TokenManager:
# Set appropriate permissions (read/write for owner only)
os.chmod(file_path, 0o600)
def read_secure_file(self, filename: str) -> bytes | None:
def read_secure_file(self, filename: str) -> Optional[bytes]:
"""
Read the content of a secure file.

View File

@@ -5,7 +5,7 @@ description = "{{name}} using crewAI"
authors = [{ name = "Your Name", email = "you@example.com" }]
requires-python = ">=3.10,<3.14"
dependencies = [
"crewai[tools]>=0.201.1,<1.0.0"
"crewai[tools]>=0.186.1,<1.0.0"
]
[project.scripts]

View File

@@ -5,7 +5,7 @@ description = "{{name}} using crewAI"
authors = [{ name = "Your Name", email = "you@example.com" }]
requires-python = ">=3.10,<3.14"
dependencies = [
"crewai[tools]>=0.201.1,<1.0.0",
"crewai[tools]>=0.186.1,<1.0.0",
]
[project.scripts]

View File

@@ -5,7 +5,7 @@ description = "Power up your crews with {{folder_name}}"
readme = "README.md"
requires-python = ">=3.10,<3.14"
dependencies = [
"crewai[tools]>=0.201.1"
"crewai[tools]>=0.186.1"
]
[tool.crewai]

View File

@@ -12,7 +12,6 @@ from crewai.cli import git
from crewai.cli.command import BaseCommand, PlusAPIMixin
from crewai.cli.config import Settings
from crewai.cli.utils import (
build_env_with_tool_repository_credentials,
extract_available_exports,
get_project_description,
get_project_name,
@@ -43,7 +42,8 @@ class ToolCommand(BaseCommand, PlusAPIMixin):
if project_root.exists():
click.secho(f"Folder {folder_name} already exists.", fg="red")
raise SystemExit
os.makedirs(project_root)
else:
os.makedirs(project_root)
click.secho(f"Creating custom tool {folder_name}...", fg="green", bold=True)
@@ -56,7 +56,7 @@ class ToolCommand(BaseCommand, PlusAPIMixin):
os.chdir(project_root)
try:
self.login()
subprocess.run(["git", "init"], check=True) # noqa: S607
subprocess.run(["git", "init"], check=True)
console.print(
f"[green]Created custom tool [bold]{folder_name}[/bold]. Run [bold]cd {project_root}[/bold] to start working.[/green]"
)
@@ -76,10 +76,10 @@ class ToolCommand(BaseCommand, PlusAPIMixin):
raise SystemExit()
project_name = get_project_name(require=True)
assert isinstance(project_name, str) # noqa: S101
assert isinstance(project_name, str)
project_version = get_project_version(require=True)
assert isinstance(project_version, str) # noqa: S101
assert isinstance(project_version, str)
project_description = get_project_description(require=False)
encoded_tarball = None
@@ -94,8 +94,8 @@ class ToolCommand(BaseCommand, PlusAPIMixin):
self._print_current_organization()
with tempfile.TemporaryDirectory() as temp_build_dir:
subprocess.run( # noqa: S603
["uv", "build", "--sdist", "--out-dir", temp_build_dir], # noqa: S607
subprocess.run(
["uv", "build", "--sdist", "--out-dir", temp_build_dir],
check=True,
capture_output=False,
)
@@ -146,7 +146,7 @@ class ToolCommand(BaseCommand, PlusAPIMixin):
style="bold red",
)
raise SystemExit
if get_response.status_code != 200:
elif get_response.status_code != 200:
console.print(
"Failed to get tool details. Please try again later.", style="bold red"
)
@@ -196,10 +196,10 @@ class ToolCommand(BaseCommand, PlusAPIMixin):
else:
add_package_command.extend(["--index", index, tool_handle])
add_package_result = subprocess.run( # noqa: S603
add_package_result = subprocess.run(
add_package_command,
capture_output=False,
env=build_env_with_tool_repository_credentials(repository_handle),
env=self._build_env_with_credentials(repository_handle),
text=True,
check=True,
)
@@ -221,6 +221,20 @@ class ToolCommand(BaseCommand, PlusAPIMixin):
)
raise SystemExit
def _build_env_with_credentials(self, repository_handle: str):
repository_handle = repository_handle.upper().replace("-", "_")
settings = Settings()
env = os.environ.copy()
env[f"UV_INDEX_{repository_handle}_USERNAME"] = str(
settings.tool_repository_username or ""
)
env[f"UV_INDEX_{repository_handle}_PASSWORD"] = str(
settings.tool_repository_password or ""
)
return env
def _print_current_organization(self) -> None:
settings = Settings()
if settings.org_uuid:

View File

@@ -5,13 +5,12 @@ import sys
from functools import reduce
from inspect import getmro, isclass, isfunction, ismethod
from pathlib import Path
from typing import Any, get_type_hints
from typing import Any, Dict, List, get_type_hints
import click
import tomli
from rich.console import Console
from crewai.cli.config import Settings
from crewai.cli.constants import ENV_VARS
from crewai.crew import Crew
from crewai.flow import Flow
@@ -42,7 +41,8 @@ def copy_template(src, dst, name, class_name, folder_name):
def read_toml(file_path: str = "pyproject.toml"):
"""Read the content of a TOML file and return it as a dictionary."""
with open(file_path, "rb") as f:
return tomli.load(f)
toml_dict = tomli.load(f)
return toml_dict
def parse_toml(content):
@@ -77,7 +77,7 @@ def get_project_description(
def _get_project_attribute(
pyproject_path: str, keys: list[str], require: bool
pyproject_path: str, keys: List[str], require: bool
) -> Any | None:
"""Get an attribute from the pyproject.toml file."""
attribute = None
@@ -96,20 +96,16 @@ def _get_project_attribute(
except FileNotFoundError:
console.print(f"Error: {pyproject_path} not found.", style="bold red")
except KeyError:
console.print(f"Error: {pyproject_path} is not a valid pyproject.toml file.", style="bold red")
except tomllib.TOMLDecodeError if sys.version_info >= (3, 11) else Exception as e: # type: ignore
console.print(
f"Error: {pyproject_path} is not a valid pyproject.toml file.",
f"Error: {pyproject_path} is not a valid TOML file."
if sys.version_info >= (3, 11)
else f"Error reading the pyproject.toml file: {e}",
style="bold red",
)
except Exception as e:
# Handle TOML decode errors for Python 3.11+
if sys.version_info >= (3, 11) and isinstance(e, tomllib.TOMLDecodeError): # type: ignore
console.print(
f"Error: {pyproject_path} is not a valid TOML file.", style="bold red"
)
else:
console.print(
f"Error reading the pyproject.toml file: {e}", style="bold red"
)
console.print(f"Error reading the pyproject.toml file: {e}", style="bold red")
if require and not attribute:
console.print(
@@ -121,7 +117,7 @@ def _get_project_attribute(
return attribute
def _get_nested_value(data: dict[str, Any], keys: list[str]) -> Any:
def _get_nested_value(data: Dict[str, Any], keys: List[str]) -> Any:
return reduce(dict.__getitem__, keys, data)
@@ -300,10 +296,7 @@ def get_crews(crew_path: str = "crew.py", require: bool = False) -> list[Crew]:
try:
crew_instances.extend(fetch_crews(module_attr))
except Exception as e:
console.print(
f"Error processing attribute {attr_name}: {e}",
style="bold red",
)
console.print(f"Error processing attribute {attr_name}: {e}", style="bold red")
continue
# If we found crew instances, break out of the loop
@@ -311,15 +304,12 @@ def get_crews(crew_path: str = "crew.py", require: bool = False) -> list[Crew]:
break
except Exception as exec_error:
console.print(
f"Error executing module: {exec_error}",
style="bold red",
)
console.print(f"Error executing module: {exec_error}", style="bold red")
except (ImportError, AttributeError) as e:
if require:
console.print(
f"Error importing crew from {crew_path}: {e!s}",
f"Error importing crew from {crew_path}: {str(e)}",
style="bold red",
)
continue
@@ -335,9 +325,9 @@ def get_crews(crew_path: str = "crew.py", require: bool = False) -> list[Crew]:
except Exception as e:
if require:
console.print(
f"Unexpected error while loading crew: {e!s}", style="bold red"
f"Unexpected error while loading crew: {str(e)}", style="bold red"
)
raise SystemExit from e
raise SystemExit
return crew_instances
@@ -358,7 +348,8 @@ def get_crew_instance(module_attr) -> Crew | None:
if isinstance(module_attr, Crew):
return module_attr
return None
else:
return None
def fetch_crews(module_attr) -> list[Crew]:
@@ -411,26 +402,11 @@ def extract_available_exports(dir_path: str = "src"):
return available_exports
except Exception as e:
console.print(f"[red]Error: Could not extract tool classes: {e!s}[/red]")
console.print(f"[red]Error: Could not extract tool classes: {str(e)}[/red]")
console.print(
"Please ensure your project contains valid tools (classes inheriting from BaseTool or functions with @tool decorator)."
)
raise SystemExit(1) from e
def build_env_with_tool_repository_credentials(repository_handle: str):
repository_handle = repository_handle.upper().replace("-", "_")
settings = Settings()
env = os.environ.copy()
env[f"UV_INDEX_{repository_handle}_USERNAME"] = str(
settings.tool_repository_username or ""
)
env[f"UV_INDEX_{repository_handle}_PASSWORD"] = str(
settings.tool_repository_password or ""
)
return env
raise SystemExit(1)
def _load_tools_from_init(init_file: Path) -> list[dict[str, Any]]:
@@ -464,8 +440,8 @@ def _load_tools_from_init(init_file: Path) -> list[dict[str, Any]]:
]
except Exception as e:
console.print(f"[red]Warning: Could not load {init_file}: {e!s}[/red]")
raise SystemExit(1) from e
console.print(f"[red]Warning: Could not load {init_file}: {str(e)}[/red]")
raise SystemExit(1)
finally:
sys.modules.pop("temp_module", None)

View File

@@ -3,17 +3,26 @@ import json
import re
import uuid
import warnings
from collections.abc import Callable
from concurrent.futures import Future
from copy import copy as shallow_copy
from hashlib import md5
from typing import (
Any,
Callable,
Dict,
List,
Optional,
Set,
Tuple,
Union,
cast,
)
from opentelemetry import baggage
from opentelemetry.context import attach, detach
from crewai.utilities.crew.models import CrewContext
from pydantic import (
UUID4,
BaseModel,
@@ -30,15 +39,26 @@ from crewai.agent import Agent
from crewai.agents.agent_builder.base_agent import BaseAgent
from crewai.agents.cache import CacheHandler
from crewai.crews.crew_output import CrewOutput
from crewai.events.event_bus import crewai_event_bus
from crewai.events.event_listener import EventListener
from crewai.events.listeners.tracing.trace_listener import (
TraceCollectionListener,
)
from crewai.events.listeners.tracing.utils import (
is_tracing_enabled,
should_auto_collect_first_time_traces,
)
from crewai.flow.flow_trackable import FlowTrackable
from crewai.knowledge.knowledge import Knowledge
from crewai.knowledge.source.base_knowledge_source import BaseKnowledgeSource
from crewai.llm import LLM, BaseLLM
from crewai.memory.entity.entity_memory import EntityMemory
from crewai.memory.external.external_memory import ExternalMemory
from crewai.memory.long_term.long_term_memory import LongTermMemory
from crewai.memory.short_term.short_term_memory import ShortTermMemory
from crewai.process import Process
from crewai.security import Fingerprint, SecurityConfig
from crewai.task import Task
from crewai.tasks.conditional_task import ConditionalTask
from crewai.tasks.task_output import TaskOutput
from crewai.tools.agent_tools.agent_tools import AgentTools
from crewai.tools.base_tool import BaseTool, Tool
from crewai.types.usage_metrics import UsageMetrics
from crewai.utilities import I18N, FileHandler, Logger, RPMController
from crewai.utilities.constants import NOT_SPECIFIED, TRAINING_DATA_FILE
from crewai.utilities.evaluators.crew_evaluator_handler import CrewEvaluator
from crewai.utilities.evaluators.task_evaluator import TaskEvaluator
from crewai.events.types.crew_events import (
CrewKickoffCompletedEvent,
CrewKickoffFailedEvent,
@@ -50,29 +70,16 @@ from crewai.events.types.crew_events import (
CrewTrainFailedEvent,
CrewTrainStartedEvent,
)
from crewai.flow.flow_trackable import FlowTrackable
from crewai.knowledge.knowledge import Knowledge
from crewai.knowledge.source.base_knowledge_source import BaseKnowledgeSource
from crewai.llm import LLM, BaseLLM
from crewai.memory.entity.entity_memory import EntityMemory
from crewai.memory.external.external_memory import ExternalMemory
from crewai.memory.long_term.long_term_memory import LongTermMemory
from crewai.memory.short_term.short_term_memory import ShortTermMemory
from crewai.process import Process
from crewai.rag.embeddings.types import EmbedderConfig
from crewai.rag.types import SearchResult
from crewai.security import Fingerprint, SecurityConfig
from crewai.task import Task
from crewai.tasks.conditional_task import ConditionalTask
from crewai.tasks.task_output import TaskOutput
from crewai.tools.agent_tools.agent_tools import AgentTools
from crewai.tools.base_tool import BaseTool, Tool
from crewai.types.usage_metrics import UsageMetrics
from crewai.utilities import I18N, FileHandler, Logger, RPMController
from crewai.utilities.constants import NOT_SPECIFIED, TRAINING_DATA_FILE
from crewai.utilities.crew.models import CrewContext
from crewai.utilities.evaluators.crew_evaluator_handler import CrewEvaluator
from crewai.utilities.evaluators.task_evaluator import TaskEvaluator
from crewai.events.event_bus import crewai_event_bus
from crewai.events.event_listener import EventListener
from crewai.events.listeners.tracing.trace_listener import (
TraceCollectionListener,
)
from crewai.events.listeners.tracing.utils import (
is_tracing_enabled,
)
from crewai.utilities.formatter import (
aggregate_raw_outputs_from_task_outputs,
aggregate_raw_outputs_from_tasks,
@@ -87,40 +94,28 @@ warnings.filterwarnings("ignore", category=SyntaxWarning, module="pysbd")
class Crew(FlowTrackable, BaseModel):
"""
Represents a group of agents, defining how they should collaborate and the
tasks they should perform.
Represents a group of agents, defining how they should collaborate and the tasks they should perform.
Attributes:
tasks: list of tasks assigned to the crew.
agents: list of agents part of this crew.
tasks: List of tasks assigned to the crew.
agents: List of agents part of this crew.
manager_llm: The language model that will run manager agent.
manager_agent: Custom agent that will be used as manager.
memory: Whether the crew should use memory to store memories of it's
execution.
cache: Whether the crew should use a cache to store the results of the
tools execution.
function_calling_llm: The language model that will run the tool calling
for all the agents.
process: The process flow that the crew will follow (e.g., sequential,
hierarchical).
memory: Whether the crew should use memory to store memories of it's execution.
cache: Whether the crew should use a cache to store the results of the tools execution.
function_calling_llm: The language model that will run the tool calling for all the agents.
process: The process flow that the crew will follow (e.g., sequential, hierarchical).
verbose: Indicates the verbosity level for logging during execution.
config: Configuration settings for the crew.
max_rpm: Maximum number of requests per minute for the crew execution to
be respected.
max_rpm: Maximum number of requests per minute for the crew execution to be respected.
prompt_file: Path to the prompt json file to be used for the crew.
id: A unique identifier for the crew instance.
task_callback: Callback to be executed after each task for every agents
execution.
step_callback: Callback to be executed after each step for every agents
execution.
share_crew: Whether you want to share the complete crew information and
execution with crewAI to make the library better, and allow us to
train models.
task_callback: Callback to be executed after each task for every agents execution.
step_callback: Callback to be executed after each step for every agents execution.
share_crew: Whether you want to share the complete crew information and execution with crewAI to make the library better, and allow us to train models.
planning: Plan the crew execution and add the plan to the crew.
chat_llm: The language model used for orchestrating chat interactions
with the crew.
security_config: Security configuration for the crew, including
fingerprinting.
chat_llm: The language model used for orchestrating chat interactions with the crew.
security_config: Security configuration for the crew, including fingerprinting.
"""
__hash__ = object.__hash__ # type: ignore
@@ -129,13 +124,13 @@ class Crew(FlowTrackable, BaseModel):
_logger: Logger = PrivateAttr()
_file_handler: FileHandler = PrivateAttr()
_cache_handler: InstanceOf[CacheHandler] = PrivateAttr(default=CacheHandler())
_short_term_memory: InstanceOf[ShortTermMemory] | None = PrivateAttr()
_long_term_memory: InstanceOf[LongTermMemory] | None = PrivateAttr()
_entity_memory: InstanceOf[EntityMemory] | None = PrivateAttr()
_external_memory: InstanceOf[ExternalMemory] | None = PrivateAttr()
_train: bool | None = PrivateAttr(default=False)
_train_iteration: int | None = PrivateAttr()
_inputs: dict[str, Any] | None = PrivateAttr(default=None)
_short_term_memory: Optional[InstanceOf[ShortTermMemory]] = PrivateAttr()
_long_term_memory: Optional[InstanceOf[LongTermMemory]] = PrivateAttr()
_entity_memory: Optional[InstanceOf[EntityMemory]] = PrivateAttr()
_external_memory: Optional[InstanceOf[ExternalMemory]] = PrivateAttr()
_train: Optional[bool] = PrivateAttr(default=False)
_train_iteration: Optional[int] = PrivateAttr()
_inputs: Optional[Dict[str, Any]] = PrivateAttr(default=None)
_logging_color: str = PrivateAttr(
default="bold_purple",
)
@@ -143,121 +138,107 @@ class Crew(FlowTrackable, BaseModel):
default_factory=TaskOutputStorageHandler
)
name: str | None = Field(default="crew")
name: Optional[str] = Field(default="crew")
cache: bool = Field(default=True)
tasks: list[Task] = Field(default_factory=list)
agents: list[BaseAgent] = Field(default_factory=list)
tasks: List[Task] = Field(default_factory=list)
agents: List[BaseAgent] = Field(default_factory=list)
process: Process = Field(default=Process.sequential)
verbose: bool = Field(default=False)
memory: bool = Field(
default=False,
description="If crew should use memory to store memories of it's execution",
description="Whether the crew should use memory to store memories of it's execution",
)
short_term_memory: InstanceOf[ShortTermMemory] | None = Field(
short_term_memory: Optional[InstanceOf[ShortTermMemory]] = Field(
default=None,
description="An Instance of the ShortTermMemory to be used by the Crew",
)
long_term_memory: InstanceOf[LongTermMemory] | None = Field(
long_term_memory: Optional[InstanceOf[LongTermMemory]] = Field(
default=None,
description="An Instance of the LongTermMemory to be used by the Crew",
)
entity_memory: InstanceOf[EntityMemory] | None = Field(
entity_memory: Optional[InstanceOf[EntityMemory]] = Field(
default=None,
description="An Instance of the EntityMemory to be used by the Crew",
)
external_memory: InstanceOf[ExternalMemory] | None = Field(
external_memory: Optional[InstanceOf[ExternalMemory]] = Field(
default=None,
description="An Instance of the ExternalMemory to be used by the Crew",
)
embedder: EmbedderConfig | None = Field(
embedder: Optional[dict] = Field(
default=None,
description="Configuration for the embedder to be used for the crew.",
)
usage_metrics: UsageMetrics | None = Field(
usage_metrics: Optional[UsageMetrics] = Field(
default=None,
description="Metrics for the LLM usage during all tasks execution.",
)
manager_llm: str | InstanceOf[BaseLLM] | Any | None = Field(
manager_llm: Optional[Union[str, InstanceOf[BaseLLM], Any]] = Field(
description="Language model that will run the agent.", default=None
)
manager_agent: BaseAgent | None = Field(
manager_agent: Optional[BaseAgent] = Field(
description="Custom agent that will be used as manager.", default=None
)
function_calling_llm: str | InstanceOf[LLM] | Any | None = Field(
function_calling_llm: Optional[Union[str, InstanceOf[LLM], Any]] = Field(
description="Language model that will run the agent.", default=None
)
config: Json | dict[str, Any] | None = Field(default=None)
config: Optional[Union[Json, Dict[str, Any]]] = Field(default=None)
id: UUID4 = Field(default_factory=uuid.uuid4, frozen=True)
share_crew: bool | None = Field(default=False)
step_callback: Any | None = Field(
share_crew: Optional[bool] = Field(default=False)
step_callback: Optional[Any] = Field(
default=None,
description="Callback to be executed after each step for all agents execution.",
)
task_callback: Any | None = Field(
task_callback: Optional[Any] = Field(
default=None,
description="Callback to be executed after each task for all agents execution.",
)
before_kickoff_callbacks: list[
Callable[[dict[str, Any] | None], dict[str, Any] | None]
before_kickoff_callbacks: List[
Callable[[Optional[Dict[str, Any]]], Optional[Dict[str, Any]]]
] = Field(
default_factory=list,
description=(
"List of callbacks to be executed before crew kickoff. "
"It may be used to adjust inputs before the crew is executed."
),
description="List of callbacks to be executed before crew kickoff. It may be used to adjust inputs before the crew is executed.",
)
after_kickoff_callbacks: list[Callable[[CrewOutput], CrewOutput]] = Field(
after_kickoff_callbacks: List[Callable[[CrewOutput], CrewOutput]] = Field(
default_factory=list,
description=(
"List of callbacks to be executed after crew kickoff. "
"It may be used to adjust the output of the crew."
),
description="List of callbacks to be executed after crew kickoff. It may be used to adjust the output of the crew.",
)
max_rpm: int | None = Field(
max_rpm: Optional[int] = Field(
default=None,
description=(
"Maximum number of requests per minute for the crew execution "
"to be respected."
),
description="Maximum number of requests per minute for the crew execution to be respected.",
)
prompt_file: str | None = Field(
prompt_file: Optional[str] = Field(
default=None,
description="Path to the prompt json file to be used for the crew.",
)
output_log_file: bool | str | None = Field(
output_log_file: Optional[Union[bool, str]] = Field(
default=None,
description="Path to the log file to be saved",
)
planning: bool | None = Field(
planning: Optional[bool] = Field(
default=False,
description="Plan the crew execution and add the plan to the crew.",
)
planning_llm: str | InstanceOf[BaseLLM] | Any | None = Field(
planning_llm: Optional[Union[str, InstanceOf[BaseLLM], Any]] = Field(
default=None,
description=(
"Language model that will run the AgentPlanner if planning is True."
),
description="Language model that will run the AgentPlanner if planning is True.",
)
task_execution_output_json_files: list[str] | None = Field(
task_execution_output_json_files: Optional[List[str]] = Field(
default=None,
description="list of file paths for task execution JSON files.",
description="List of file paths for task execution JSON files.",
)
execution_logs: list[dict[str, Any]] = Field(
execution_logs: List[Dict[str, Any]] = Field(
default=[],
description="list of execution logs for tasks",
description="List of execution logs for tasks",
)
knowledge_sources: list[BaseKnowledgeSource] | None = Field(
knowledge_sources: Optional[List[BaseKnowledgeSource]] = Field(
default=None,
description=(
"Knowledge sources for the crew. Add knowledge sources to the "
"knowledge object."
),
description="Knowledge sources for the crew. Add knowledge sources to the knowledge object.",
)
chat_llm: str | InstanceOf[BaseLLM] | Any | None = Field(
chat_llm: Optional[Union[str, InstanceOf[BaseLLM], Any]] = Field(
default=None,
description="LLM used to handle chatting with the crew.",
)
knowledge: Knowledge | None = Field(
knowledge: Optional[Knowledge] = Field(
default=None,
description="Knowledge for the crew.",
)
@@ -265,18 +246,18 @@ class Crew(FlowTrackable, BaseModel):
default_factory=SecurityConfig,
description="Security configuration for the crew, including fingerprinting.",
)
token_usage: UsageMetrics | None = Field(
token_usage: Optional[UsageMetrics] = Field(
default=None,
description="Metrics for the LLM usage during all tasks execution.",
)
tracing: bool | None = Field(
tracing: Optional[bool] = Field(
default=False,
description="Whether to enable tracing for the crew.",
)
@field_validator("id", mode="before")
@classmethod
def _deny_user_set_id(cls, v: UUID4 | None) -> None:
def _deny_user_set_id(cls, v: Optional[UUID4]) -> None:
"""Prevent manual setting of the 'id' field by users."""
if v:
raise PydanticCustomError(
@@ -285,7 +266,9 @@ class Crew(FlowTrackable, BaseModel):
@field_validator("config", mode="before")
@classmethod
def check_config_type(cls, v: Json | dict[str, Any]) -> Json | dict[str, Any]:
def check_config_type(
cls, v: Union[Json, Dict[str, Any]]
) -> Union[Json, Dict[str, Any]]:
"""Validates that the config is a valid type.
Args:
v: The config to be validated.
@@ -298,16 +281,12 @@ class Crew(FlowTrackable, BaseModel):
@model_validator(mode="after")
def set_private_attrs(self) -> "Crew":
"""set private attributes."""
"""Set private attributes."""
self._cache_handler = CacheHandler()
event_listener = EventListener()
if (
is_tracing_enabled()
or self.tracing
or should_auto_collect_first_time_traces()
):
if is_tracing_enabled() or self.tracing:
trace_listener = TraceCollectionListener()
trace_listener.setup_listeners(crewai_event_bus)
event_listener.verbose = self.verbose
@@ -335,8 +314,7 @@ class Crew(FlowTrackable, BaseModel):
def create_crew_memory(self) -> "Crew":
"""Initialize private memory attributes."""
self._external_memory = (
# External memory does not support a default value since it was
# designed to be managed entirely externally
# External memory doesnt support a default value since it was designed to be managed entirely externally
self.external_memory.set_crew(self) if self.external_memory else None
)
@@ -377,10 +355,7 @@ class Crew(FlowTrackable, BaseModel):
if not self.manager_llm and not self.manager_agent:
raise PydanticCustomError(
"missing_manager_llm_or_manager_agent",
(
"Attribute `manager_llm` or `manager_agent` is required "
"when using hierarchical process."
),
"Attribute `manager_llm` or `manager_agent` is required when using hierarchical process.",
{},
)
@@ -423,10 +398,7 @@ class Crew(FlowTrackable, BaseModel):
if task.agent is None:
raise PydanticCustomError(
"missing_agent_in_task",
(
f"Sequential process error: Agent is missing in the task "
f"with the following description: {task.description}"
), # type: ignore # Dynamic string in error message
f"Sequential process error: Agent is missing in the task with the following description: {task.description}", # type: ignore # Argument of type "str" cannot be assigned to parameter "message_template" of type "LiteralString"
{},
)
@@ -487,10 +459,7 @@ class Crew(FlowTrackable, BaseModel):
if task.async_execution and isinstance(task, ConditionalTask):
raise PydanticCustomError(
"invalid_async_conditional_task",
(
f"Conditional Task: {task.description}, "
f"cannot be executed asynchronously."
),
f"Conditional Task: {task.description} , cannot be executed asynchronously.", # type: ignore # Argument of type "str" cannot be assigned to parameter "message_template" of type "LiteralString"
{},
)
return self
@@ -509,9 +478,7 @@ class Crew(FlowTrackable, BaseModel):
for j in range(i - 1, -1, -1):
if self.tasks[j] == context_task:
raise ValueError(
f"Task '{task.description}' is asynchronous and "
f"cannot include other sequential asynchronous "
f"tasks in its context."
f"Task '{task.description}' is asynchronous and cannot include other sequential asynchronous tasks in its context."
)
if not self.tasks[j].async_execution:
break
@@ -529,15 +496,13 @@ class Crew(FlowTrackable, BaseModel):
continue # Skip context tasks not in the main tasks list
if task_indices[id(context_task)] > task_indices[id(task)]:
raise ValueError(
f"Task '{task.description}' has a context dependency "
f"on a future task '{context_task.description}', "
f"which is not allowed."
f"Task '{task.description}' has a context dependency on a future task '{context_task.description}', which is not allowed."
)
return self
@property
def key(self) -> str:
source: list[str] = [agent.key for agent in self.agents] + [
source: List[str] = [agent.key for agent in self.agents] + [
task.key for task in self.tasks
]
return md5("|".join(source).encode(), usedforsecurity=False).hexdigest()
@@ -553,9 +518,9 @@ class Crew(FlowTrackable, BaseModel):
return self.security_config.fingerprint
def _setup_from_config(self):
assert self.config is not None, "Config should not be None."
"""Initializes agents and tasks from the provided config."""
if self.config is None:
raise ValueError("Config should not be None.")
if not self.config.get("agents") or not self.config.get("tasks"):
raise PydanticCustomError(
"missing_keys_in_config", "Config should have 'agents' and 'tasks'.", {}
@@ -565,7 +530,7 @@ class Crew(FlowTrackable, BaseModel):
self.agents = [Agent(**agent) for agent in self.config["agents"]]
self.tasks = [self._create_task(task) for task in self.config["tasks"]]
def _create_task(self, task_config: dict[str, Any]) -> Task:
def _create_task(self, task_config: Dict[str, Any]) -> Task:
"""Creates a task instance from its configuration.
Args:
@@ -594,7 +559,7 @@ class Crew(FlowTrackable, BaseModel):
CrewTrainingHandler(filename).initialize_file()
def train(
self, n_iterations: int, filename: str, inputs: dict[str, Any] | None = None
self, n_iterations: int, filename: str, inputs: Optional[Dict[str, Any]] = None
) -> None:
"""Trains the crew for a given number of iterations."""
inputs = inputs or {}
@@ -623,8 +588,7 @@ class Crew(FlowTrackable, BaseModel):
training_data=training_data, agent_id=str(agent.id)
)
CrewTrainingHandler(filename).save_trained_data(
agent_id=str(agent.role),
trained_data=result.model_dump(), # type: ignore[arg-type]
agent_id=str(agent.role), trained_data=result.model_dump()
)
crewai_event_bus.emit(
@@ -647,7 +611,7 @@ class Crew(FlowTrackable, BaseModel):
def kickoff(
self,
inputs: dict[str, Any] | None = None,
inputs: Optional[Dict[str, Any]] = None,
) -> CrewOutput:
ctx = baggage.set_baggage(
"crew_context", CrewContext(id=str(self.id), key=self.key)
@@ -718,9 +682,9 @@ class Crew(FlowTrackable, BaseModel):
finally:
detach(token)
def kickoff_for_each(self, inputs: list[dict[str, Any]]) -> list[CrewOutput]:
"""Executes the Crew's workflow for each input and aggregates results."""
results: list[CrewOutput] = []
def kickoff_for_each(self, inputs: List[Dict[str, Any]]) -> List[CrewOutput]:
"""Executes the Crew's workflow for each input in the list and aggregates results."""
results: List[CrewOutput] = []
# Initialize the parent crew's usage metrics
total_usage_metrics = UsageMetrics()
@@ -739,12 +703,14 @@ class Crew(FlowTrackable, BaseModel):
self._task_output_handler.reset()
return results
async def kickoff_async(self, inputs: dict[str, Any] | None = None) -> CrewOutput:
async def kickoff_async(
self, inputs: Optional[Dict[str, Any]] = None
) -> CrewOutput:
"""Asynchronous kickoff method to start the crew execution."""
inputs = inputs or {}
return await asyncio.to_thread(self.kickoff, inputs)
async def kickoff_for_each_async(self, inputs: list[dict]) -> list[CrewOutput]:
async def kickoff_for_each_async(self, inputs: List[Dict]) -> List[CrewOutput]:
crew_copies = [self.copy() for _ in inputs]
async def run_crew(crew, input_data):
@@ -773,9 +739,7 @@ class Crew(FlowTrackable, BaseModel):
tasks=self.tasks, planning_agent_llm=self.planning_llm
)._handle_crew_planning()
for task, step_plan in zip(
self.tasks, result.list_of_plans_per_task, strict=False
):
for task, step_plan in zip(self.tasks, result.list_of_plans_per_task):
task.description += step_plan.plan
def _store_execution_log(
@@ -812,7 +776,7 @@ class Crew(FlowTrackable, BaseModel):
return self._execute_tasks(self.tasks)
def _run_hierarchical_process(self) -> CrewOutput:
"""Creates and assigns a manager agent to complete the tasks."""
"""Creates and assigns a manager agent to make sure the crew completes the tasks."""
self._create_manager_agent()
return self._execute_tasks(self.tasks)
@@ -843,24 +807,23 @@ class Crew(FlowTrackable, BaseModel):
def _execute_tasks(
self,
tasks: list[Task],
start_index: int | None = 0,
tasks: List[Task],
start_index: Optional[int] = 0,
was_replayed: bool = False,
) -> CrewOutput:
"""Executes tasks sequentially and returns the final output.
Args:
tasks (List[Task]): List of tasks to execute
manager (Optional[BaseAgent], optional): Manager agent to use for
delegation. Defaults to None.
manager (Optional[BaseAgent], optional): Manager agent to use for delegation. Defaults to None.
Returns:
CrewOutput: Final output of the crew
"""
task_outputs: list[TaskOutput] = []
futures: list[tuple[Task, Future[TaskOutput], int]] = []
last_sync_output: TaskOutput | None = None
task_outputs: List[TaskOutput] = []
futures: List[Tuple[Task, Future[TaskOutput], int]] = []
last_sync_output: Optional[TaskOutput] = None
for task_index, task in enumerate(tasks):
if start_index is not None and task_index < start_index:
@@ -875,9 +838,7 @@ class Crew(FlowTrackable, BaseModel):
agent_to_use = self._get_agent_to_use(task)
if agent_to_use is None:
raise ValueError(
f"No agent available for task: {task.description}. "
f"Ensure that either the task has an assigned agent "
f"or a manager agent is provided."
f"No agent available for task: {task.description}. Ensure that either the task has an assigned agent or a manager agent is provided."
)
# Determine which tools to use - task tools take precedence over agent tools
@@ -886,7 +847,7 @@ class Crew(FlowTrackable, BaseModel):
tools_for_task = self._prepare_tools(
agent_to_use,
task,
cast(list[Tool] | list[BaseTool], tools_for_task),
cast(Union[List[Tool], List[BaseTool]], tools_for_task),
)
self._log_task_start(task, agent_to_use.role)
@@ -906,7 +867,7 @@ class Crew(FlowTrackable, BaseModel):
future = task.execute_async(
agent=agent_to_use,
context=context,
tools=cast(list[BaseTool], tools_for_task),
tools=cast(List[BaseTool], tools_for_task),
)
futures.append((task, future, task_index))
else:
@@ -918,7 +879,7 @@ class Crew(FlowTrackable, BaseModel):
task_output = task.execute_sync(
agent=agent_to_use,
context=context,
tools=cast(list[BaseTool], tools_for_task),
tools=cast(List[BaseTool], tools_for_task),
)
task_outputs.append(task_output)
self._process_task_result(task, task_output)
@@ -932,11 +893,11 @@ class Crew(FlowTrackable, BaseModel):
def _handle_conditional_task(
self,
task: ConditionalTask,
task_outputs: list[TaskOutput],
futures: list[tuple[Task, Future[TaskOutput], int]],
task_outputs: List[TaskOutput],
futures: List[Tuple[Task, Future[TaskOutput], int]],
task_index: int,
was_replayed: bool,
) -> TaskOutput | None:
) -> Optional[TaskOutput]:
if futures:
task_outputs = self._process_async_tasks(futures, was_replayed)
futures.clear()
@@ -956,8 +917,8 @@ class Crew(FlowTrackable, BaseModel):
return None
def _prepare_tools(
self, agent: BaseAgent, task: Task, tools: list[Tool] | list[BaseTool]
) -> list[BaseTool]:
self, agent: BaseAgent, task: Task, tools: Union[List[Tool], List[BaseTool]]
) -> List[BaseTool]:
# Add delegation tools if agent allows delegation
if hasattr(agent, "allow_delegation") and getattr(
agent, "allow_delegation", False
@@ -986,22 +947,22 @@ class Crew(FlowTrackable, BaseModel):
):
tools = self._add_multimodal_tools(agent, tools)
# Return a List[BaseTool] compatible with Task.execute_sync and execute_async
return cast(list[BaseTool], tools)
# Return a List[BaseTool] which is compatible with both Task.execute_sync and Task.execute_async
return cast(List[BaseTool], tools)
def _get_agent_to_use(self, task: Task) -> BaseAgent | None:
def _get_agent_to_use(self, task: Task) -> Optional[BaseAgent]:
if self.process == Process.hierarchical:
return self.manager_agent
return task.agent
def _merge_tools(
self,
existing_tools: list[Tool] | list[BaseTool],
new_tools: list[Tool] | list[BaseTool],
) -> list[BaseTool]:
"""Merge new tools into existing tools list, avoiding duplicates."""
existing_tools: Union[List[Tool], List[BaseTool]],
new_tools: Union[List[Tool], List[BaseTool]],
) -> List[BaseTool]:
"""Merge new tools into existing tools list, avoiding duplicates by tool name."""
if not new_tools:
return cast(list[BaseTool], existing_tools)
return cast(List[BaseTool], existing_tools)
# Create mapping of tool names to new tools
new_tool_map = {tool.name: tool for tool in new_tools}
@@ -1012,41 +973,41 @@ class Crew(FlowTrackable, BaseModel):
# Add all new tools
tools.extend(new_tools)
return cast(list[BaseTool], tools)
return cast(List[BaseTool], tools)
def _inject_delegation_tools(
self,
tools: list[Tool] | list[BaseTool],
tools: Union[List[Tool], List[BaseTool]],
task_agent: BaseAgent,
agents: list[BaseAgent],
) -> list[BaseTool]:
agents: List[BaseAgent],
) -> List[BaseTool]:
if hasattr(task_agent, "get_delegation_tools"):
delegation_tools = task_agent.get_delegation_tools(agents)
# Cast delegation_tools to the expected type for _merge_tools
return self._merge_tools(tools, cast(list[BaseTool], delegation_tools))
return cast(list[BaseTool], tools)
return self._merge_tools(tools, cast(List[BaseTool], delegation_tools))
return cast(List[BaseTool], tools)
def _add_multimodal_tools(
self, agent: BaseAgent, tools: list[Tool] | list[BaseTool]
) -> list[BaseTool]:
self, agent: BaseAgent, tools: Union[List[Tool], List[BaseTool]]
) -> List[BaseTool]:
if hasattr(agent, "get_multimodal_tools"):
multimodal_tools = agent.get_multimodal_tools()
# Cast multimodal_tools to the expected type for _merge_tools
return self._merge_tools(tools, cast(list[BaseTool], multimodal_tools))
return cast(list[BaseTool], tools)
return self._merge_tools(tools, cast(List[BaseTool], multimodal_tools))
return cast(List[BaseTool], tools)
def _add_code_execution_tools(
self, agent: BaseAgent, tools: list[Tool] | list[BaseTool]
) -> list[BaseTool]:
self, agent: BaseAgent, tools: Union[List[Tool], List[BaseTool]]
) -> List[BaseTool]:
if hasattr(agent, "get_code_execution_tools"):
code_tools = agent.get_code_execution_tools()
# Cast code_tools to the expected type for _merge_tools
return self._merge_tools(tools, cast(list[BaseTool], code_tools))
return cast(list[BaseTool], tools)
return self._merge_tools(tools, cast(List[BaseTool], code_tools))
return cast(List[BaseTool], tools)
def _add_delegation_tools(
self, task: Task, tools: list[Tool] | list[BaseTool]
) -> list[BaseTool]:
self, task: Task, tools: Union[List[Tool], List[BaseTool]]
) -> List[BaseTool]:
agents_for_delegation = [agent for agent in self.agents if agent != task.agent]
if len(self.agents) > 1 and len(agents_for_delegation) > 0 and task.agent:
if not tools:
@@ -1054,20 +1015,17 @@ class Crew(FlowTrackable, BaseModel):
tools = self._inject_delegation_tools(
tools, task.agent, agents_for_delegation
)
return cast(list[BaseTool], tools)
return cast(List[BaseTool], tools)
def _log_task_start(self, task: Task, role: str = "None"):
if self.output_log_file:
self._file_handler.log(
task_name=task.name, # type: ignore[arg-type]
task=task.description,
agent=role,
status="started",
task_name=task.name, task=task.description, agent=role, status="started"
)
def _update_manager_tools(
self, task: Task, tools: list[Tool] | list[BaseTool]
) -> list[BaseTool]:
self, task: Task, tools: Union[List[Tool], List[BaseTool]]
) -> List[BaseTool]:
if self.manager_agent:
if task.agent:
tools = self._inject_delegation_tools(tools, task.agent, [task.agent])
@@ -1075,30 +1033,31 @@ class Crew(FlowTrackable, BaseModel):
tools = self._inject_delegation_tools(
tools, self.manager_agent, self.agents
)
return cast(list[BaseTool], tools)
return cast(List[BaseTool], tools)
def _get_context(self, task: Task, task_outputs: list[TaskOutput]) -> str:
def _get_context(self, task: Task, task_outputs: List[TaskOutput]) -> str:
if not task.context:
return ""
return (
context = (
aggregate_raw_outputs_from_task_outputs(task_outputs)
if task.context is NOT_SPECIFIED
else aggregate_raw_outputs_from_tasks(task.context)
)
return context
def _process_task_result(self, task: Task, output: TaskOutput) -> None:
role = task.agent.role if task.agent is not None else "None"
if self.output_log_file:
self._file_handler.log(
task_name=task.name, # type: ignore[arg-type]
task_name=task.name,
task=task.description,
agent=role,
status="completed",
output=output.raw,
)
def _create_crew_output(self, task_outputs: list[TaskOutput]) -> CrewOutput:
def _create_crew_output(self, task_outputs: List[TaskOutput]) -> CrewOutput:
if not task_outputs:
raise ValueError("No task outputs available to create crew output.")
@@ -1129,10 +1088,10 @@ class Crew(FlowTrackable, BaseModel):
def _process_async_tasks(
self,
futures: list[tuple[Task, Future[TaskOutput], int]],
futures: List[Tuple[Task, Future[TaskOutput], int]],
was_replayed: bool = False,
) -> list[TaskOutput]:
task_outputs: list[TaskOutput] = []
) -> List[TaskOutput]:
task_outputs: List[TaskOutput] = []
for future_task, future, task_index in futures:
task_output = future.result()
task_outputs.append(task_output)
@@ -1142,7 +1101,9 @@ class Crew(FlowTrackable, BaseModel):
)
return task_outputs
def _find_task_index(self, task_id: str, stored_outputs: list[Any]) -> int | None:
def _find_task_index(
self, task_id: str, stored_outputs: List[Any]
) -> Optional[int]:
return next(
(
index
@@ -1152,8 +1113,9 @@ class Crew(FlowTrackable, BaseModel):
None,
)
def replay(self, task_id: str, inputs: dict[str, Any] | None = None) -> CrewOutput:
"""Replay the crew execution from a specific task."""
def replay(
self, task_id: str, inputs: Optional[Dict[str, Any]] = None
) -> CrewOutput:
stored_outputs = self._task_output_handler.load()
if not stored_outputs:
raise ValueError(f"Task with id {task_id} not found in the crew's tasks.")
@@ -1189,19 +1151,19 @@ class Crew(FlowTrackable, BaseModel):
self.tasks[i].output = task_output
self._logging_color = "bold_blue"
return self._execute_tasks(self.tasks, start_index, True)
result = self._execute_tasks(self.tasks, start_index, True)
return result
def query_knowledge(
self, query: list[str], results_limit: int = 3, score_threshold: float = 0.35
) -> list[SearchResult] | None:
"""Query the crew's knowledge base for relevant information."""
self, query: List[str], results_limit: int = 3, score_threshold: float = 0.35
) -> Union[List[Dict[str, Any]], None]:
if self.knowledge:
return self.knowledge.query(
query, results_limit=results_limit, score_threshold=score_threshold
)
return None
def fetch_inputs(self) -> set[str]:
def fetch_inputs(self) -> Set[str]:
"""
Gathers placeholders (e.g., {something}) referenced in tasks or agents.
Scans each task's 'description' + 'expected_output', and each agent's
@@ -1210,11 +1172,11 @@ class Crew(FlowTrackable, BaseModel):
Returns a set of all discovered placeholder names.
"""
placeholder_pattern = re.compile(r"\{(.+?)\}")
required_inputs: set[str] = set()
required_inputs: Set[str] = set()
# Scan tasks for inputs
for task in self.tasks:
# description and expected_output might contain e.g. {topic}, {user_name}
# description and expected_output might contain e.g. {topic}, {user_name}, etc.
text = f"{task.description or ''} {task.expected_output or ''}"
required_inputs.update(placeholder_pattern.findall(text))
@@ -1268,7 +1230,7 @@ class Crew(FlowTrackable, BaseModel):
cloned_tasks.append(cloned_task)
task_mapping[task.key] = cloned_task
for cloned_task, original_task in zip(cloned_tasks, self.tasks, strict=False):
for cloned_task, original_task in zip(cloned_tasks, self.tasks):
if isinstance(original_task.context, list):
cloned_context = [
task_mapping[context_task.key]
@@ -1294,7 +1256,7 @@ class Crew(FlowTrackable, BaseModel):
copied_data.pop("agents", None)
copied_data.pop("tasks", None)
return Crew(
copied_crew = Crew(
**copied_data,
agents=cloned_agents,
tasks=cloned_tasks,
@@ -1304,13 +1266,15 @@ class Crew(FlowTrackable, BaseModel):
manager_llm=manager_llm,
)
return copied_crew
def _set_tasks_callbacks(self) -> None:
"""Sets callback for every task suing task_callback"""
for task in self.tasks:
if not task.callback:
task.callback = self.task_callback
def _interpolate_inputs(self, inputs: dict[str, Any]) -> None:
def _interpolate_inputs(self, inputs: Dict[str, Any]) -> None:
"""Interpolates the inputs in the tasks and agents."""
[
task.interpolate_inputs_and_add_conversation_history(
@@ -1343,13 +1307,10 @@ class Crew(FlowTrackable, BaseModel):
def test(
self,
n_iterations: int,
eval_llm: str | InstanceOf[BaseLLM],
inputs: dict[str, Any] | None = None,
eval_llm: Union[str, InstanceOf[BaseLLM]],
inputs: Optional[Dict[str, Any]] = None,
) -> None:
"""Test and evaluate the Crew with the given inputs for n iterations.
Uses concurrent.futures for concurrent execution.
"""
"""Test and evaluate the Crew with the given inputs for n iterations concurrently using concurrent.futures."""
try:
# Create LLM instance and ensure it's of type LLM for CrewEvaluator
llm_instance = create_llm(eval_llm)
@@ -1389,11 +1350,7 @@ class Crew(FlowTrackable, BaseModel):
raise
def __repr__(self):
return (
f"Crew(id={self.id}, process={self.process}, "
f"number_of_agents={len(self.agents)}, "
f"number_of_tasks={len(self.tasks)})"
)
return f"Crew(id={self.id}, process={self.process}, number_of_agents={len(self.agents)}, number_of_tasks={len(self.tasks)})"
def reset_memories(self, command_type: str) -> None:
"""Reset specific or all memories for the crew.
@@ -1407,7 +1364,7 @@ class Crew(FlowTrackable, BaseModel):
ValueError: If an invalid command type is provided.
RuntimeError: If memory reset operation fails.
"""
valid_types = frozenset(
VALID_TYPES = frozenset(
[
"long",
"short",
@@ -1420,10 +1377,9 @@ class Crew(FlowTrackable, BaseModel):
]
)
if command_type not in valid_types:
if command_type not in VALID_TYPES:
raise ValueError(
f"Invalid command type. Must be one of: "
f"{', '.join(sorted(valid_types))}"
f"Invalid command type. Must be one of: {', '.join(sorted(VALID_TYPES))}"
)
try:
@@ -1433,7 +1389,7 @@ class Crew(FlowTrackable, BaseModel):
self._reset_specific_memory(command_type)
except Exception as e:
error_msg = f"Failed to reset {command_type} memory: {e!s}"
error_msg = f"Failed to reset {command_type} memory: {str(e)}"
self._logger.log("error", error_msg)
raise RuntimeError(error_msg) from e
@@ -1441,7 +1397,7 @@ class Crew(FlowTrackable, BaseModel):
"""Reset all available memory systems."""
memory_systems = self._get_memory_systems()
for config in memory_systems.values():
for memory_type, config in memory_systems.items():
if (system := config.get("system")) is not None:
name = config.get("name")
try:
@@ -1449,13 +1405,11 @@ class Crew(FlowTrackable, BaseModel):
reset_fn(system)
self._logger.log(
"info",
f"[Crew ({self.name if self.name else self.id})] "
f"{name} memory has been reset",
f"[Crew ({self.name if self.name else self.id})] {name} memory has been reset",
)
except Exception as e:
raise RuntimeError(
f"[Crew ({self.name if self.name else self.id})] "
f"Failed to reset {name} memory: {e!s}"
f"[Crew ({self.name if self.name else self.id})] Failed to reset {name} memory: {str(e)}"
) from e
def _reset_specific_memory(self, memory_type: str) -> None:
@@ -1480,21 +1434,18 @@ class Crew(FlowTrackable, BaseModel):
reset_fn(system)
self._logger.log(
"info",
f"[Crew ({self.name if self.name else self.id})] "
f"{name} memory has been reset",
f"[Crew ({self.name if self.name else self.id})] {name} memory has been reset",
)
except Exception as e:
raise RuntimeError(
f"[Crew ({self.name if self.name else self.id})] "
f"Failed to reset {name} memory: {e!s}"
f"[Crew ({self.name if self.name else self.id})] Failed to reset {name} memory: {str(e)}"
) from e
def _get_memory_systems(self):
"""Get all available memory systems with their configuration.
Returns:
Dict containing all memory systems with their reset functions and
display names.
Dict containing all memory systems with their reset functions and display names.
"""
def default_reset(memory):
@@ -1555,7 +1506,7 @@ class Crew(FlowTrackable, BaseModel):
},
}
def reset_knowledge(self, knowledges: list[Knowledge]) -> None:
def reset_knowledge(self, knowledges: List[Knowledge]) -> None:
"""Reset crew and agent knowledge storage."""
for ks in knowledges:
ks.reset()

View File

@@ -1,5 +1,5 @@
import json
from typing import Any
from typing import Any, Dict, Optional
from pydantic import BaseModel, Field
@@ -12,21 +12,19 @@ class CrewOutput(BaseModel):
"""Class that represents the result of a crew."""
raw: str = Field(description="Raw output of crew", default="")
pydantic: BaseModel | None = Field(
pydantic: Optional[BaseModel] = Field(
description="Pydantic output of Crew", default=None
)
json_dict: dict[str, Any] | None = Field(
json_dict: Optional[Dict[str, Any]] = Field(
description="JSON dict output of Crew", default=None
)
tasks_output: list[TaskOutput] = Field(
description="Output of each task", default=[]
)
token_usage: UsageMetrics = Field(
description="Processed token summary", default_factory=UsageMetrics
)
token_usage: UsageMetrics = Field(description="Processed token summary", default={})
@property
def json(self) -> str | None: # type: ignore[override]
def json(self) -> Optional[str]:
if self.tasks_output[-1].output_format != OutputFormat.JSON:
raise ValueError(
"No JSON output found in the final task. Please make sure to set the output_json property in the final task in your crew."
@@ -34,7 +32,7 @@ class CrewOutput(BaseModel):
return json.dumps(self.json_dict)
def to_dict(self) -> dict[str, Any]:
def to_dict(self) -> Dict[str, Any]:
"""Convert json_output and pydantic_output to a dictionary."""
output_dict = {}
if self.json_dict:
@@ -46,9 +44,10 @@ class CrewOutput(BaseModel):
def __getitem__(self, key):
if self.pydantic and hasattr(self.pydantic, key):
return getattr(self.pydantic, key)
if self.json_dict and key in self.json_dict:
elif self.json_dict and key in self.json_dict:
return self.json_dict[key]
raise KeyError(f"Key '{key}' not found in CrewOutput.")
else:
raise KeyError(f"Key '{key}' not found in CrewOutput.")
def __str__(self):
if self.pydantic:

View File

@@ -9,158 +9,48 @@ This module provides the event infrastructure that allows users to:
from crewai.events.base_event_listener import BaseEventListener
from crewai.events.event_bus import crewai_event_bus
from crewai.events.types.agent_events import (
AgentEvaluationCompletedEvent,
AgentEvaluationFailedEvent,
AgentEvaluationStartedEvent,
AgentExecutionCompletedEvent,
AgentExecutionErrorEvent,
AgentExecutionStartedEvent,
LiteAgentExecutionCompletedEvent,
LiteAgentExecutionErrorEvent,
LiteAgentExecutionStartedEvent,
)
from crewai.events.types.crew_events import (
CrewKickoffCompletedEvent,
CrewKickoffFailedEvent,
CrewKickoffStartedEvent,
CrewTestCompletedEvent,
CrewTestFailedEvent,
CrewTestResultEvent,
CrewTestStartedEvent,
CrewTrainCompletedEvent,
CrewTrainFailedEvent,
CrewTrainStartedEvent,
)
from crewai.events.types.flow_events import (
FlowCreatedEvent,
FlowEvent,
FlowFinishedEvent,
FlowPlotEvent,
FlowStartedEvent,
MethodExecutionFailedEvent,
MethodExecutionFinishedEvent,
MethodExecutionStartedEvent,
)
from crewai.events.types.knowledge_events import (
KnowledgeQueryCompletedEvent,
KnowledgeQueryFailedEvent,
KnowledgeQueryStartedEvent,
KnowledgeRetrievalCompletedEvent,
KnowledgeRetrievalStartedEvent,
KnowledgeSearchQueryFailedEvent,
)
from crewai.events.types.llm_events import (
LLMCallCompletedEvent,
LLMCallFailedEvent,
LLMCallStartedEvent,
LLMStreamChunkEvent,
)
from crewai.events.types.llm_guardrail_events import (
LLMGuardrailCompletedEvent,
LLMGuardrailStartedEvent,
)
from crewai.events.types.logging_events import (
AgentLogsExecutionEvent,
AgentLogsStartedEvent,
)
from crewai.events.types.memory_events import (
MemoryQueryCompletedEvent,
MemoryQueryFailedEvent,
MemorySaveCompletedEvent,
MemorySaveStartedEvent,
MemoryQueryStartedEvent,
MemoryRetrievalCompletedEvent,
MemoryRetrievalStartedEvent,
MemorySaveCompletedEvent,
MemorySaveFailedEvent,
MemorySaveStartedEvent,
MemoryQueryFailedEvent,
)
from crewai.events.types.reasoning_events import (
AgentReasoningCompletedEvent,
AgentReasoningFailedEvent,
AgentReasoningStartedEvent,
ReasoningEvent,
from crewai.events.types.knowledge_events import (
KnowledgeRetrievalStartedEvent,
KnowledgeRetrievalCompletedEvent,
)
from crewai.events.types.task_events import (
TaskCompletedEvent,
TaskEvaluationEvent,
TaskFailedEvent,
TaskStartedEvent,
from crewai.events.types.crew_events import (
CrewKickoffStartedEvent,
CrewKickoffCompletedEvent,
)
from crewai.events.types.tool_usage_events import (
ToolExecutionErrorEvent,
ToolSelectionErrorEvent,
ToolUsageErrorEvent,
ToolUsageEvent,
ToolUsageFinishedEvent,
ToolUsageStartedEvent,
ToolValidateInputErrorEvent,
from crewai.events.types.agent_events import (
AgentExecutionCompletedEvent,
)
from crewai.events.types.llm_events import (
LLMStreamChunkEvent,
)
__all__ = [
"AgentEvaluationCompletedEvent",
"AgentEvaluationFailedEvent",
"AgentEvaluationStartedEvent",
"AgentExecutionCompletedEvent",
"AgentExecutionErrorEvent",
"AgentExecutionStartedEvent",
"AgentLogsExecutionEvent",
"AgentLogsStartedEvent",
"AgentReasoningCompletedEvent",
"AgentReasoningFailedEvent",
"AgentReasoningStartedEvent",
"BaseEventListener",
"CrewKickoffCompletedEvent",
"CrewKickoffFailedEvent",
"CrewKickoffStartedEvent",
"CrewTestCompletedEvent",
"CrewTestFailedEvent",
"CrewTestResultEvent",
"CrewTestStartedEvent",
"CrewTrainCompletedEvent",
"CrewTrainFailedEvent",
"CrewTrainStartedEvent",
"FlowCreatedEvent",
"FlowEvent",
"FlowFinishedEvent",
"FlowPlotEvent",
"FlowStartedEvent",
"KnowledgeQueryCompletedEvent",
"KnowledgeQueryFailedEvent",
"KnowledgeQueryStartedEvent",
"KnowledgeRetrievalCompletedEvent",
"KnowledgeRetrievalStartedEvent",
"KnowledgeSearchQueryFailedEvent",
"LLMCallCompletedEvent",
"LLMCallFailedEvent",
"LLMCallStartedEvent",
"LLMGuardrailCompletedEvent",
"LLMGuardrailStartedEvent",
"LLMStreamChunkEvent",
"LiteAgentExecutionCompletedEvent",
"LiteAgentExecutionErrorEvent",
"LiteAgentExecutionStartedEvent",
"crewai_event_bus",
"MemoryQueryCompletedEvent",
"MemoryQueryFailedEvent",
"MemorySaveCompletedEvent",
"MemorySaveStartedEvent",
"MemoryQueryStartedEvent",
"MemoryRetrievalCompletedEvent",
"MemoryRetrievalStartedEvent",
"MemorySaveCompletedEvent",
"MemorySaveFailedEvent",
"MemorySaveStartedEvent",
"MethodExecutionFailedEvent",
"MethodExecutionFinishedEvent",
"MethodExecutionStartedEvent",
"ReasoningEvent",
"TaskCompletedEvent",
"TaskEvaluationEvent",
"TaskFailedEvent",
"TaskStartedEvent",
"ToolExecutionErrorEvent",
"ToolSelectionErrorEvent",
"ToolUsageErrorEvent",
"ToolUsageEvent",
"ToolUsageFinishedEvent",
"ToolUsageStartedEvent",
"ToolValidateInputErrorEvent",
"crewai_event_bus",
]
"MemoryQueryFailedEvent",
"KnowledgeRetrievalStartedEvent",
"KnowledgeRetrievalCompletedEvent",
"CrewKickoffStartedEvent",
"CrewKickoffCompletedEvent",
"AgentExecutionCompletedEvent",
"LLMStreamChunkEvent",
]

View File

@@ -1,6 +1,5 @@
from datetime import datetime, timezone
from typing import Any
from typing import Any, Dict, Optional
from pydantic import BaseModel, Field
from crewai.utilities.serialization import to_serializable
@@ -11,11 +10,11 @@ class BaseEvent(BaseModel):
timestamp: datetime = Field(default_factory=lambda: datetime.now(timezone.utc))
type: str
source_fingerprint: str | None = None # UUID string of the source entity
source_type: str | None = (
source_fingerprint: Optional[str] = None # UUID string of the source entity
source_type: Optional[str] = (
None # "agent", "task", "crew", "memory", "entity_memory", "short_term_memory", "long_term_memory", "external_memory"
)
fingerprint_metadata: dict[str, Any] | None = None # Any relevant metadata
fingerprint_metadata: Optional[Dict[str, Any]] = None # Any relevant metadata
def to_json(self, exclude: set[str] | None = None):
"""
@@ -29,13 +28,13 @@ class BaseEvent(BaseModel):
"""
return to_serializable(self, exclude=exclude)
def _set_task_params(self, data: dict[str, Any]):
def _set_task_params(self, data: Dict[str, Any]):
if "from_task" in data and (task := data["from_task"]):
self.task_id = task.id
self.task_name = task.name or task.description
self.from_task = None
def _set_agent_params(self, data: dict[str, Any]):
def _set_agent_params(self, data: Dict[str, Any]):
task = data.get("from_task", None)
agent = task.agent if task else data.get("from_agent", None)

View File

@@ -1,9 +1,8 @@
from __future__ import annotations
import threading
from collections.abc import Callable
from contextlib import contextmanager
from typing import Any, TypeVar, cast
from typing import Any, Callable, Dict, List, Type, TypeVar, cast
from blinker import Signal
@@ -26,17 +25,17 @@ class CrewAIEventsBus:
if cls._instance is None:
with cls._lock:
if cls._instance is None: # prevent race condition
cls._instance = super().__new__(cls)
cls._instance = super(CrewAIEventsBus, cls).__new__(cls)
cls._instance._initialize()
return cls._instance
def _initialize(self) -> None:
"""Initialize the event bus internal state"""
self._signal = Signal("crewai_event_bus")
self._handlers: dict[type[BaseEvent], list[Callable]] = {}
self._handlers: Dict[Type[BaseEvent], List[Callable]] = {}
def on(
self, event_type: type[EventT]
self, event_type: Type[EventT]
) -> Callable[[Callable[[Any, EventT], None]], Callable[[Any, EventT], None]]:
"""
Decorator to register an event handler for a specific event type.
@@ -62,18 +61,6 @@ class CrewAIEventsBus:
return decorator
@staticmethod
def _call_handler(
handler: Callable, source: Any, event: BaseEvent, event_type: type
) -> None:
"""Call a single handler with error handling."""
try:
handler(source, event)
except Exception as e:
print(
f"[EventBus Error] Handler '{handler.__name__}' failed for event '{event_type.__name__}': {e}"
)
def emit(self, source: Any, event: BaseEvent) -> None:
"""
Emit an event to all registered handlers
@@ -85,12 +72,17 @@ class CrewAIEventsBus:
for event_type, handlers in self._handlers.items():
if isinstance(event, event_type):
for handler in handlers:
self._call_handler(handler, source, event, event_type)
try:
handler(source, event)
except Exception as e:
print(
f"[EventBus Error] Handler '{handler.__name__}' failed for event '{event_type.__name__}': {e}"
)
self._signal.send(source, event=event)
def register_handler(
self, event_type: type[EventTypes], handler: Callable[[Any, EventTypes], None]
self, event_type: Type[EventTypes], handler: Callable[[Any, EventTypes], None]
) -> None:
"""Register an event handler for a specific event type"""
if event_type not in self._handlers:

View File

@@ -1,30 +1,15 @@
from __future__ import annotations
from io import StringIO
from typing import Any
from typing import Any, Dict
from pydantic import Field, PrivateAttr
from crewai.llm import LLM
from crewai.task import Task
from crewai.telemetry.telemetry import Telemetry
from crewai.utilities import Logger
from crewai.utilities.constants import EMITTER_COLOR
from crewai.events.base_event_listener import BaseEventListener
from crewai.events.types.agent_events import (
AgentExecutionCompletedEvent,
AgentExecutionStartedEvent,
LiteAgentExecutionCompletedEvent,
LiteAgentExecutionErrorEvent,
LiteAgentExecutionStartedEvent,
)
from crewai.events.types.crew_events import (
CrewKickoffCompletedEvent,
CrewKickoffFailedEvent,
CrewKickoffStartedEvent,
CrewTestCompletedEvent,
CrewTestFailedEvent,
CrewTestResultEvent,
CrewTestStartedEvent,
CrewTrainCompletedEvent,
CrewTrainFailedEvent,
CrewTrainStartedEvent,
)
from crewai.events.types.knowledge_events import (
KnowledgeQueryCompletedEvent,
KnowledgeQueryFailedEvent,
@@ -40,21 +25,34 @@ from crewai.events.types.llm_events import (
LLMStreamChunkEvent,
)
from crewai.events.types.llm_guardrail_events import (
LLMGuardrailCompletedEvent,
LLMGuardrailStartedEvent,
)
from crewai.events.types.logging_events import (
AgentLogsExecutionEvent,
AgentLogsStartedEvent,
LLMGuardrailCompletedEvent,
)
from crewai.events.utils.console_formatter import ConsoleFormatter
from crewai.llm import LLM
from crewai.task import Task
from crewai.telemetry.telemetry import Telemetry
from crewai.utilities import Logger
from crewai.utilities.constants import EMITTER_COLOR
from .listeners.memory_listener import MemoryListener
from crewai.events.types.agent_events import (
AgentExecutionCompletedEvent,
AgentExecutionStartedEvent,
LiteAgentExecutionCompletedEvent,
LiteAgentExecutionErrorEvent,
LiteAgentExecutionStartedEvent,
)
from crewai.events.types.logging_events import (
AgentLogsStartedEvent,
AgentLogsExecutionEvent,
)
from crewai.events.types.crew_events import (
CrewKickoffCompletedEvent,
CrewKickoffFailedEvent,
CrewKickoffStartedEvent,
CrewTestCompletedEvent,
CrewTestFailedEvent,
CrewTestResultEvent,
CrewTestStartedEvent,
CrewTrainCompletedEvent,
CrewTrainFailedEvent,
CrewTrainStartedEvent,
)
from .types.flow_events import (
FlowCreatedEvent,
FlowFinishedEvent,
@@ -63,24 +61,26 @@ from .types.flow_events import (
MethodExecutionFinishedEvent,
MethodExecutionStartedEvent,
)
from .types.reasoning_events import (
AgentReasoningCompletedEvent,
AgentReasoningFailedEvent,
AgentReasoningStartedEvent,
)
from .types.task_events import TaskCompletedEvent, TaskFailedEvent, TaskStartedEvent
from .types.tool_usage_events import (
ToolUsageErrorEvent,
ToolUsageFinishedEvent,
ToolUsageStartedEvent,
)
from .types.reasoning_events import (
AgentReasoningStartedEvent,
AgentReasoningCompletedEvent,
AgentReasoningFailedEvent,
)
from .listeners.memory_listener import MemoryListener
class EventListener(BaseEventListener):
_instance = None
_telemetry: Telemetry = PrivateAttr(default_factory=lambda: Telemetry())
logger = Logger(verbose=True, default_color=EMITTER_COLOR)
execution_spans: dict[Task, Any] = Field(default_factory=dict)
execution_spans: Dict[Task, Any] = Field(default_factory=dict)
next_chunk = 0
text_stream = StringIO()
knowledge_retrieval_in_progress = False

View File

@@ -1,10 +1,11 @@
from typing import Union
from crewai.events.types.agent_events import (
AgentExecutionCompletedEvent,
AgentExecutionErrorEvent,
AgentExecutionStartedEvent,
LiteAgentExecutionCompletedEvent,
)
from .types.crew_events import (
CrewKickoffCompletedEvent,
CrewKickoffFailedEvent,
@@ -23,14 +24,6 @@ from .types.flow_events import (
MethodExecutionFinishedEvent,
MethodExecutionStartedEvent,
)
from .types.knowledge_events import (
KnowledgeQueryCompletedEvent,
KnowledgeQueryFailedEvent,
KnowledgeQueryStartedEvent,
KnowledgeRetrievalCompletedEvent,
KnowledgeRetrievalStartedEvent,
KnowledgeSearchQueryFailedEvent,
)
from .types.llm_events import (
LLMCallCompletedEvent,
LLMCallFailedEvent,
@@ -41,21 +34,6 @@ from .types.llm_guardrail_events import (
LLMGuardrailCompletedEvent,
LLMGuardrailStartedEvent,
)
from .types.memory_events import (
MemoryQueryCompletedEvent,
MemoryQueryFailedEvent,
MemoryQueryStartedEvent,
MemoryRetrievalCompletedEvent,
MemoryRetrievalStartedEvent,
MemorySaveCompletedEvent,
MemorySaveFailedEvent,
MemorySaveStartedEvent,
)
from .types.reasoning_events import (
AgentReasoningCompletedEvent,
AgentReasoningFailedEvent,
AgentReasoningStartedEvent,
)
from .types.task_events import (
TaskCompletedEvent,
TaskFailedEvent,
@@ -66,53 +44,77 @@ from .types.tool_usage_events import (
ToolUsageFinishedEvent,
ToolUsageStartedEvent,
)
EventTypes = (
CrewKickoffStartedEvent
| CrewKickoffCompletedEvent
| CrewKickoffFailedEvent
| CrewTestStartedEvent
| CrewTestCompletedEvent
| CrewTestFailedEvent
| CrewTrainStartedEvent
| CrewTrainCompletedEvent
| CrewTrainFailedEvent
| AgentExecutionStartedEvent
| AgentExecutionCompletedEvent
| LiteAgentExecutionCompletedEvent
| TaskStartedEvent
| TaskCompletedEvent
| TaskFailedEvent
| FlowStartedEvent
| FlowFinishedEvent
| MethodExecutionStartedEvent
| MethodExecutionFinishedEvent
| MethodExecutionFailedEvent
| AgentExecutionErrorEvent
| ToolUsageFinishedEvent
| ToolUsageErrorEvent
| ToolUsageStartedEvent
| LLMCallStartedEvent
| LLMCallCompletedEvent
| LLMCallFailedEvent
| LLMStreamChunkEvent
| LLMGuardrailStartedEvent
| LLMGuardrailCompletedEvent
| AgentReasoningStartedEvent
| AgentReasoningCompletedEvent
| AgentReasoningFailedEvent
| KnowledgeRetrievalStartedEvent
| KnowledgeRetrievalCompletedEvent
| KnowledgeQueryStartedEvent
| KnowledgeQueryCompletedEvent
| KnowledgeQueryFailedEvent
| KnowledgeSearchQueryFailedEvent
| MemorySaveStartedEvent
| MemorySaveCompletedEvent
| MemorySaveFailedEvent
| MemoryQueryStartedEvent
| MemoryQueryCompletedEvent
| MemoryQueryFailedEvent
| MemoryRetrievalStartedEvent
| MemoryRetrievalCompletedEvent
from .types.reasoning_events import (
AgentReasoningStartedEvent,
AgentReasoningCompletedEvent,
AgentReasoningFailedEvent,
)
from .types.knowledge_events import (
KnowledgeRetrievalStartedEvent,
KnowledgeRetrievalCompletedEvent,
KnowledgeQueryStartedEvent,
KnowledgeQueryCompletedEvent,
KnowledgeQueryFailedEvent,
KnowledgeSearchQueryFailedEvent,
)
from .types.memory_events import (
MemorySaveStartedEvent,
MemorySaveCompletedEvent,
MemorySaveFailedEvent,
MemoryQueryStartedEvent,
MemoryQueryCompletedEvent,
MemoryQueryFailedEvent,
MemoryRetrievalStartedEvent,
MemoryRetrievalCompletedEvent,
)
EventTypes = Union[
CrewKickoffStartedEvent,
CrewKickoffCompletedEvent,
CrewKickoffFailedEvent,
CrewTestStartedEvent,
CrewTestCompletedEvent,
CrewTestFailedEvent,
CrewTrainStartedEvent,
CrewTrainCompletedEvent,
CrewTrainFailedEvent,
AgentExecutionStartedEvent,
AgentExecutionCompletedEvent,
LiteAgentExecutionCompletedEvent,
TaskStartedEvent,
TaskCompletedEvent,
TaskFailedEvent,
FlowStartedEvent,
FlowFinishedEvent,
MethodExecutionStartedEvent,
MethodExecutionFinishedEvent,
MethodExecutionFailedEvent,
AgentExecutionErrorEvent,
ToolUsageFinishedEvent,
ToolUsageErrorEvent,
ToolUsageStartedEvent,
LLMCallStartedEvent,
LLMCallCompletedEvent,
LLMCallFailedEvent,
LLMStreamChunkEvent,
LLMGuardrailStartedEvent,
LLMGuardrailCompletedEvent,
AgentReasoningStartedEvent,
AgentReasoningCompletedEvent,
AgentReasoningFailedEvent,
KnowledgeRetrievalStartedEvent,
KnowledgeRetrievalCompletedEvent,
KnowledgeQueryStartedEvent,
KnowledgeQueryCompletedEvent,
KnowledgeQueryFailedEvent,
KnowledgeSearchQueryFailedEvent,
MemorySaveStartedEvent,
MemorySaveCompletedEvent,
MemorySaveFailedEvent,
MemoryQueryStartedEvent,
MemoryQueryCompletedEvent,
MemoryQueryFailedEvent,
MemoryRetrievalStartedEvent,
MemoryRetrievalCompletedEvent,
]

View File

@@ -2,4 +2,4 @@
This module contains various event listener implementations
for handling memory, tracing, and other event-driven functionality.
"""
"""

View File

@@ -1,12 +1,12 @@
from crewai.events.base_event_listener import BaseEventListener
from crewai.events.types.memory_events import (
MemoryQueryCompletedEvent,
MemoryQueryFailedEvent,
MemoryRetrievalCompletedEvent,
MemoryRetrievalStartedEvent,
MemoryQueryFailedEvent,
MemoryQueryCompletedEvent,
MemorySaveStartedEvent,
MemorySaveCompletedEvent,
MemorySaveFailedEvent,
MemorySaveStartedEvent,
)

View File

@@ -1,229 +0,0 @@
import logging
import uuid
import webbrowser
from pathlib import Path
from rich.console import Console
from rich.panel import Panel
from crewai.events.listeners.tracing.trace_batch_manager import TraceBatchManager
from crewai.events.listeners.tracing.utils import (
mark_first_execution_completed,
prompt_user_for_trace_viewing,
should_auto_collect_first_time_traces,
)
logger = logging.getLogger(__name__)
def _update_or_create_env_file():
"""Update or create .env file with CREWAI_TRACING_ENABLED=true."""
env_path = Path(".env")
env_content = ""
variable_name = "CREWAI_TRACING_ENABLED"
variable_value = "true"
# Read existing content if file exists
if env_path.exists():
with open(env_path, "r") as f:
env_content = f.read()
# Check if CREWAI_TRACING_ENABLED is already set
lines = env_content.splitlines()
variable_exists = False
updated_lines = []
for line in lines:
if line.strip().startswith(f"{variable_name}="):
# Update existing variable
updated_lines.append(f"{variable_name}={variable_value}")
variable_exists = True
else:
updated_lines.append(line)
# Add variable if it doesn't exist
if not variable_exists:
if updated_lines and not updated_lines[-1].strip():
# If last line is empty, replace it
updated_lines[-1] = f"{variable_name}={variable_value}"
else:
# Add new line and then the variable
updated_lines.append(f"{variable_name}={variable_value}")
# Write updated content
with open(env_path, "w") as f:
f.write("\n".join(updated_lines))
if updated_lines: # Add final newline if there's content
f.write("\n")
class FirstTimeTraceHandler:
"""Handles the first-time user trace collection and display flow."""
def __init__(self):
self.is_first_time: bool = False
self.collected_events: bool = False
self.trace_batch_id: str | None = None
self.ephemeral_url: str | None = None
self.batch_manager: TraceBatchManager | None = None
def initialize_for_first_time_user(self) -> bool:
"""Check if this is first time and initialize collection."""
self.is_first_time = should_auto_collect_first_time_traces()
return self.is_first_time
def set_batch_manager(self, batch_manager: TraceBatchManager):
"""Set reference to batch manager for sending events."""
self.batch_manager = batch_manager
def mark_events_collected(self):
"""Mark that events have been collected during execution."""
self.collected_events = True
def handle_execution_completion(self):
"""Handle the completion flow as shown in your diagram."""
if not self.is_first_time or not self.collected_events:
return
try:
user_wants_traces = prompt_user_for_trace_viewing(timeout_seconds=20)
if user_wants_traces:
self._initialize_backend_and_send_events()
# Enable tracing for future runs by updating .env file
try:
_update_or_create_env_file()
except Exception: # noqa: S110
pass
if self.ephemeral_url:
self._display_ephemeral_trace_link()
mark_first_execution_completed()
except Exception as e:
self._gracefully_fail(f"Error in trace handling: {e}")
mark_first_execution_completed()
def _initialize_backend_and_send_events(self):
"""Initialize backend batch and send collected events."""
if not self.batch_manager:
return
try:
if not self.batch_manager.backend_initialized:
original_metadata = (
self.batch_manager.current_batch.execution_metadata
if self.batch_manager.current_batch
else {}
)
user_context = {
"privacy_level": "standard",
"user_id": "first_time_user",
"session_id": str(uuid.uuid4()),
"trace_id": self.batch_manager.trace_batch_id,
}
execution_metadata = {
"execution_type": original_metadata.get("execution_type", "crew"),
"crew_name": original_metadata.get(
"crew_name", "First Time Execution"
),
"flow_name": original_metadata.get("flow_name"),
"agent_count": original_metadata.get("agent_count", 1),
"task_count": original_metadata.get("task_count", 1),
"crewai_version": original_metadata.get("crewai_version"),
}
self.batch_manager._initialize_backend_batch(
user_context=user_context,
execution_metadata=execution_metadata,
use_ephemeral=True,
)
self.batch_manager.backend_initialized = True
if self.batch_manager.event_buffer:
self.batch_manager._send_events_to_backend()
self.batch_manager.finalize_batch()
self.ephemeral_url = self.batch_manager.ephemeral_trace_url
if not self.ephemeral_url:
self._show_local_trace_message()
except Exception as e:
self._gracefully_fail(f"Backend initialization failed: {e}")
def _display_ephemeral_trace_link(self):
"""Display the ephemeral trace link to the user and automatically open browser."""
console = Console()
try:
webbrowser.open(self.ephemeral_url)
except Exception: # noqa: S110
pass
panel_content = f"""
🎉 Your First CrewAI Execution Trace is Ready!
View your execution details here:
{self.ephemeral_url}
This trace shows:
• Agent decisions and interactions
• Task execution timeline
• Tool usage and results
• LLM calls and responses
✅ Tracing has been enabled for future runs! (CREWAI_TRACING_ENABLED=true added to .env)
You can also add tracing=True to your Crew(tracing=True) / Flow(tracing=True) for more control.
📝 Note: This link will expire in 24 hours.
""".strip()
panel = Panel(
panel_content,
title="🔍 Execution Trace Generated",
border_style="bright_green",
padding=(1, 2),
)
console.print("\n")
console.print(panel)
console.print()
def _gracefully_fail(self, error_message: str):
"""Handle errors gracefully without disrupting user experience."""
console = Console()
console.print(f"[yellow]Note: {error_message}[/yellow]")
logger.debug(f"First-time trace error: {error_message}")
def _show_local_trace_message(self):
"""Show message when traces were collected locally but couldn't be uploaded."""
console = Console()
panel_content = f"""
📊 Your execution traces were collected locally!
Unfortunately, we couldn't upload them to the server right now, but here's what we captured:
{len(self.batch_manager.event_buffer)} trace events
• Execution duration: {self.batch_manager.calculate_duration("execution")}ms
• Batch ID: {self.batch_manager.trace_batch_id}
Tracing has been enabled for future runs! (CREWAI_TRACING_ENABLED=true added to .env)
The traces include agent decisions, task execution, and tool usage.
""".strip()
panel = Panel(
panel_content,
title="🔍 Local Traces Collected",
border_style="yellow",
padding=(1, 2),
)
console.print("\n")
console.print(panel)
console.print()

View File

@@ -2,16 +2,17 @@ import uuid
from dataclasses import dataclass, field
from datetime import datetime, timezone
from logging import getLogger
from typing import Any
from typing import Any, ClassVar
from rich.align import Align
from rich.console import Console
from rich.panel import Panel
from rich.text import Text
from crewai.cli.authentication.token import AuthError, get_auth_token
from crewai.cli.plus_api import PlusAPI
from crewai.cli.version import get_crewai_version
from crewai.events.listeners.tracing.types import TraceEvent
from crewai.events.listeners.tracing.utils import should_auto_collect_first_time_traces
from crewai.utilities.constants import CREWAI_BASE_URL
logger = getLogger(__name__)
@@ -40,23 +41,21 @@ class TraceBatch:
class TraceBatchManager:
"""Single responsibility: Manage batches and event buffering"""
is_current_batch_ephemeral: bool = False
trace_batch_id: str | None = None
current_batch: TraceBatch | None = None
event_buffer: ClassVar[list[TraceEvent]] = []
execution_start_times: ClassVar[dict[str, datetime]] = {}
batch_owner_type: str | None = None
batch_owner_id: str | None = None
def __init__(self):
self.is_current_batch_ephemeral: bool = False
self.trace_batch_id: str | None = None
self.current_batch: TraceBatch | None = None
self.event_buffer: list[TraceEvent] = []
self.execution_start_times: dict[str, datetime] = {}
self.batch_owner_type: str | None = None
self.batch_owner_id: str | None = None
self.backend_initialized: bool = False
self.ephemeral_trace_url: str | None = None
try:
self.plus_api = PlusAPI(
api_key=get_auth_token(),
)
except AuthError:
self.plus_api = PlusAPI(api_key="")
self.ephemeral_trace_url = None
def initialize_batch(
self,
@@ -72,14 +71,7 @@ class TraceBatchManager:
self.is_current_batch_ephemeral = use_ephemeral
self.record_start_time("execution")
if should_auto_collect_first_time_traces():
self.trace_batch_id = self.current_batch.batch_id
else:
self._initialize_backend_batch(
user_context, execution_metadata, use_ephemeral
)
self.backend_initialized = True
self._initialize_backend_batch(user_context, execution_metadata, use_ephemeral)
return self.current_batch
@@ -138,6 +130,13 @@ class TraceBatchManager:
if not use_ephemeral
else response_data["ephemeral_trace_id"]
)
console = Console()
panel = Panel(
f"✅ Trace batch initialized with session ID: {self.trace_batch_id}",
title="Trace Batch Initialization",
border_style="green",
)
console.print(panel)
else:
logger.warning(
f"Trace batch initialization returned status {response.status_code}. Continuing without tracing."
@@ -145,7 +144,7 @@ class TraceBatchManager:
except Exception as e:
logger.warning(
f"Error initializing trace batch: {e}. Continuing without tracing."
f"Error initializing trace batch: {e!s}. Continuing without tracing."
)
def add_event(self, trace_event: TraceEvent):
@@ -156,6 +155,7 @@ class TraceBatchManager:
"""Send buffered events to backend with graceful failure handling"""
if not self.plus_api or not self.trace_batch_id or not self.event_buffer:
return 500
try:
payload = {
"events": [event.to_dict() for event in self.event_buffer],
@@ -179,7 +179,6 @@ class TraceBatchManager:
if response.status_code in [200, 201]:
self.event_buffer.clear()
return 200
logger.warning(
f"Failed to send events: {response.status_code}. Events will be lost."
)
@@ -187,7 +186,7 @@ class TraceBatchManager:
except Exception as e:
logger.warning(
f"Error sending events to backend: {e}. Events will be lost."
f"Error sending events to backend: {e!s}. Events will be lost."
)
return 500
@@ -200,9 +199,6 @@ class TraceBatchManager:
if self.event_buffer:
events_sent_to_backend_status = self._send_events_to_backend()
if events_sent_to_backend_status == 500:
self.plus_api.mark_trace_batch_as_failed(
self.trace_batch_id, "Error sending events to backend"
)
return None
self._finalize_backend_batch()
@@ -250,39 +246,67 @@ class TraceBatchManager:
if not self.is_current_batch_ephemeral and access_code is None
else f"{CREWAI_BASE_URL}/crewai_plus/ephemeral_trace_batches/{self.trace_batch_id}?access_code={access_code}"
)
if self.is_current_batch_ephemeral:
self.ephemeral_trace_url = return_link
# Create a properly formatted message with URL on its own line
message_parts = [
f"✅ Trace batch finalized with session ID: {self.trace_batch_id}",
"",
f"🔗 View here: {return_link}",
]
if access_code:
message_parts.append(f"🔑 Access Code: {access_code}")
panel = Panel(
"\n".join(message_parts),
title="Trace Batch Finalization",
border_style="green",
)
if not should_auto_collect_first_time_traces():
console.print(panel)
self._display_traces_events_link(console, return_link, access_code)
else:
logger.error(
f"❌ Failed to finalize trace batch: {response.status_code} - {response.text}"
)
self.plus_api.mark_trace_batch_as_failed(
self.trace_batch_id, response.text
)
except Exception as e:
logger.error(f"❌ Error finalizing trace batch: {e}")
self.plus_api.mark_trace_batch_as_failed(self.trace_batch_id, str(e))
logger.error(f"❌ Error finalizing trace batch: {e!s}")
# TODO: send error to app
def _display_traces_events_link(
self, console: Console, return_link: str, access_code: str | None = None
):
"""Display trace batch finalization information"""
try:
final_text = Text()
final_text.append("🎊", style="bold bright_yellow")
final_text.append(" TRACES READY FOR VIEWING! ", style="bold bright_green")
final_text.append("🎊", style="bold bright_yellow")
final_text.append("\n\n")
final_text.append("Trace ID: ", style="bold bright_cyan")
final_text.append(
f"{self.trace_batch_id}",
style="bright_blue",
)
final_text.append("\n\n")
final_text.append("View Your Traces: ", style="bold bright_cyan")
final_text.append(f"{return_link}", style="bright_white on red")
if access_code:
final_text.append("\n\n")
final_text.append("Access Code: ", style="bold bright_cyan")
final_text.append(f"{access_code}", style="bright_blue")
final_text.append("\n\n")
final_text.append("💡 ", style="bright_yellow")
final_text.append(
"Click the link above to dive into your agentic automation traces!",
style="italic bright_white",
)
final_panel = Panel(
Align.center(final_text),
title="🎊 Your Traces Are Ready! 🎊",
style="bright_green",
expand=True,
padding=(2, 4),
)
console.print(final_panel)
except Exception as e:
logger.warning(f"Display failed, falling back to simple display: {e!s}")
fallback_panel = Panel(
f"✅ Trace batch finalized with session ID: {self.trace_batch_id}. View here: {return_link} {f', Access Code: {access_code}' if access_code else ''}",
title="Trace Batch Finalization",
border_style="green",
)
console.print(fallback_panel)
def _cleanup_batch_data(self):
"""Clean up batch data after successful finalization to free memory"""
@@ -299,7 +323,7 @@ class TraceBatchManager:
self.batch_sequence = 0
except Exception as e:
logger.error(f"Warning: Error during cleanup: {e}")
logger.error(f"Warning: Error during cleanup: {e!s}")
def has_events(self) -> bool:
"""Check if there are events in the buffer"""

View File

@@ -5,11 +5,7 @@ from typing import Any, ClassVar
from crewai.cli.authentication.token import AuthError, get_auth_token
from crewai.cli.version import get_crewai_version
from crewai.events.base_event_listener import BaseEventListener
from crewai.events.listeners.tracing.first_time_trace_handler import (
FirstTimeTraceHandler,
)
from crewai.events.listeners.tracing.types import TraceEvent
from crewai.events.listeners.tracing.utils import safe_serialize_to_dict
from crewai.events.types.agent_events import (
AgentExecutionCompletedEvent,
AgentExecutionErrorEvent,
@@ -64,6 +60,7 @@ from crewai.events.types.tool_usage_events import (
ToolUsageFinishedEvent,
ToolUsageStartedEvent,
)
from crewai.utilities.serialization import to_serializable
from .trace_batch_manager import TraceBatchManager
@@ -86,7 +83,7 @@ class TraceCollectionListener(BaseEventListener):
_initialized = False
_listeners_setup = False
def __new__(cls, batch_manager: TraceBatchManager | None = None):
def __new__(cls, batch_manager=None):
if cls._instance is None:
cls._instance = super().__new__(cls)
return cls._instance
@@ -101,10 +98,6 @@ class TraceCollectionListener(BaseEventListener):
super().__init__()
self.batch_manager = batch_manager or TraceBatchManager()
self._initialized = True
self.first_time_handler = FirstTimeTraceHandler()
if self.first_time_handler.initialize_for_first_time_user():
self.first_time_handler.set_batch_manager(self.batch_manager)
def _check_authenticated(self) -> bool:
"""Check if tracing should be enabled"""
@@ -162,14 +155,8 @@ class TraceCollectionListener(BaseEventListener):
@event_bus.on(FlowFinishedEvent)
def on_flow_finished(source, event):
self._handle_trace_event("flow_finished", source, event)
if self.batch_manager.batch_owner_type == "flow":
if self.first_time_handler.is_first_time:
self.first_time_handler.mark_events_collected()
self.first_time_handler.handle_execution_completion()
else:
# Normal flow finalization
self.batch_manager.finalize_batch()
self.batch_manager.finalize_batch()
@event_bus.on(FlowPlotEvent)
def on_flow_plot(source, event):
@@ -188,20 +175,12 @@ class TraceCollectionListener(BaseEventListener):
def on_crew_completed(source, event):
self._handle_trace_event("crew_kickoff_completed", source, event)
if self.batch_manager.batch_owner_type == "crew":
if self.first_time_handler.is_first_time:
self.first_time_handler.mark_events_collected()
self.first_time_handler.handle_execution_completion()
else:
self.batch_manager.finalize_batch()
self.batch_manager.finalize_batch()
@event_bus.on(CrewKickoffFailedEvent)
def on_crew_failed(source, event):
self._handle_trace_event("crew_kickoff_failed", source, event)
if self.first_time_handler.is_first_time:
self.first_time_handler.mark_events_collected()
self.first_time_handler.handle_execution_completion()
else:
self.batch_manager.finalize_batch()
self.batch_manager.finalize_batch()
@event_bus.on(TaskStartedEvent)
def on_task_started(source, event):
@@ -342,17 +321,15 @@ class TraceCollectionListener(BaseEventListener):
def _initialize_batch(
self, user_context: dict[str, str], execution_metadata: dict[str, Any]
):
"""Initialize trace batch - auto-enable ephemeral for first-time users."""
if self.first_time_handler.is_first_time:
return self.batch_manager.initialize_batch(
"""Initialize trace batch if ephemeral"""
if not self._check_authenticated():
self.batch_manager.initialize_batch(
user_context, execution_metadata, use_ephemeral=True
)
use_ephemeral = not self._check_authenticated()
return self.batch_manager.initialize_batch(
user_context, execution_metadata, use_ephemeral=use_ephemeral
)
else:
self.batch_manager.initialize_batch(
user_context, execution_metadata, use_ephemeral=False
)
def _handle_trace_event(self, event_type: str, source: Any, event: Any):
"""Generic handler for context end events"""
@@ -391,7 +368,7 @@ class TraceCollectionListener(BaseEventListener):
) -> dict[str, Any]:
"""Build event data"""
if event_type not in self.complex_events:
return safe_serialize_to_dict(event)
return self._safe_serialize_to_dict(event)
if event_type == "task_started":
return {
"task_description": event.task.description,
@@ -427,7 +404,7 @@ class TraceCollectionListener(BaseEventListener):
"agent_backstory": event.agent.backstory,
}
if event_type == "llm_call_started":
event_data = safe_serialize_to_dict(event)
event_data = self._safe_serialize_to_dict(event)
event_data["task_name"] = (
event.task_name or event.task_description
if hasattr(event, "task_name") and event.task_name
@@ -435,10 +412,22 @@ class TraceCollectionListener(BaseEventListener):
)
return event_data
if event_type == "llm_call_completed":
return safe_serialize_to_dict(event)
return self._safe_serialize_to_dict(event)
return {
"event_type": event_type,
"event": safe_serialize_to_dict(event),
"event": self._safe_serialize_to_dict(event),
"source": source,
}
# TODO: move to utils
def _safe_serialize_to_dict(
self, obj, exclude: set[str] | None = None
) -> dict[str, Any]:
"""Safely serialize an object to a dictionary for event data."""
try:
serialized = to_serializable(obj, exclude)
if isinstance(serialized, dict):
return serialized
return {"serialized_data": serialized}
except Exception as e:
return {"serialization_error": str(e), "object_type": type(obj).__name__}

View File

@@ -1,7 +1,7 @@
import uuid
from dataclasses import asdict, dataclass, field
from dataclasses import dataclass, field, asdict
from datetime import datetime, timezone
from typing import Any
from typing import Dict, Any
import uuid
@dataclass
@@ -13,7 +13,7 @@ class TraceEvent:
default_factory=lambda: datetime.now(timezone.utc).isoformat()
)
type: str = ""
event_data: dict[str, Any] = field(default_factory=dict)
event_data: Dict[str, Any] = field(default_factory=dict)
def to_dict(self) -> dict[str, Any]:
def to_dict(self) -> Dict[str, Any]:
return asdict(self)

View File

@@ -1,25 +1,17 @@
import getpass
import hashlib
import json
import logging
import os
import platform
import re
import subprocess
import uuid
from datetime import datetime
import hashlib
import subprocess
import getpass
from pathlib import Path
from typing import Any
from datetime import datetime
import re
import json
import click
from rich.console import Console
from rich.panel import Panel
from rich.text import Text
from crewai.utilities.paths import db_storage_path
from crewai.utilities.serialization import to_serializable
logger = logging.getLogger(__name__)
def is_tracing_enabled() -> bool:
@@ -51,167 +43,49 @@ def _get_machine_id() -> str:
try:
mac = ":".join(
[f"{(uuid.getnode() >> b) & 0xFF:02x}" for b in range(0, 12, 2)][::-1]
["{:02x}".format((uuid.getnode() >> b) & 0xFF) for b in range(0, 12, 2)][
::-1
]
)
parts.append(mac)
except Exception: # noqa: S110
except Exception:
pass
try:
sysname = platform.system()
parts.append(sysname)
except Exception:
sysname = "unknown"
parts.append(sysname)
sysname = platform.system()
parts.append(sysname)
try:
if sysname == "Darwin":
try:
res = subprocess.run(
["/usr/sbin/system_profiler", "SPHardwareDataType"],
capture_output=True,
text=True,
timeout=2,
)
m = re.search(r"Hardware UUID:\s*([A-Fa-f0-9\-]+)", res.stdout)
if m:
parts.append(m.group(1))
except Exception: # noqa: S110
pass
res = subprocess.run(
["system_profiler", "SPHardwareDataType"],
capture_output=True,
text=True,
timeout=2,
)
m = re.search(r"Hardware UUID:\s*([A-Fa-f0-9\-]+)", res.stdout)
if m:
parts.append(m.group(1))
elif sysname == "Linux":
linux_id = _get_linux_machine_id()
if linux_id:
parts.append(linux_id)
elif sysname == "Windows":
try:
res = subprocess.run(
[
"C:\\Windows\\System32\\wbem\\wmic.exe",
"csproduct",
"get",
"UUID",
],
capture_output=True,
text=True,
timeout=2,
)
lines = [
line.strip() for line in res.stdout.splitlines() if line.strip()
]
if len(lines) >= 2:
parts.append(lines[1])
except Exception: # noqa: S110
pass
else:
generic_id = _get_generic_system_id()
if generic_id:
parts.append(generic_id)
except Exception: # noqa: S110
parts.append(Path("/etc/machine-id").read_text().strip())
except Exception:
parts.append(Path("/sys/class/dmi/id/product_uuid").read_text().strip())
elif sysname == "Windows":
res = subprocess.run(
["wmic", "csproduct", "get", "UUID"],
capture_output=True,
text=True,
timeout=2,
)
lines = [line.strip() for line in res.stdout.splitlines() if line.strip()]
if len(lines) >= 2:
parts.append(lines[1])
except Exception:
pass
if len(parts) <= 1:
try:
import socket
parts.append(socket.gethostname())
except Exception: # noqa: S110
pass
try:
parts.append(getpass.getuser())
except Exception: # noqa: S110
pass
try:
parts.append(platform.machine())
parts.append(platform.processor())
except Exception: # noqa: S110
pass
if not parts:
parts.append("unknown-system")
parts.append(str(uuid.uuid4()))
return hashlib.sha256("".join(parts).encode()).hexdigest()
def _get_linux_machine_id() -> str | None:
linux_id_sources = [
"/etc/machine-id",
"/sys/class/dmi/id/product_uuid",
"/proc/sys/kernel/random/boot_id",
"/sys/class/dmi/id/board_serial",
"/sys/class/dmi/id/chassis_serial",
]
for source in linux_id_sources:
try:
path = Path(source)
if path.exists() and path.is_file():
content = path.read_text().strip()
if content and content.lower() not in [
"unknown",
"to be filled by o.e.m.",
"",
]:
return content
except Exception: # noqa: S112, PERF203
continue
try:
import socket
hostname = socket.gethostname()
arch = platform.machine()
if hostname and arch:
return f"{hostname}-{arch}"
except Exception: # noqa: S110
pass
return None
def _get_generic_system_id() -> str | None:
try:
parts = []
try:
import socket
hostname = socket.gethostname()
if hostname:
parts.append(hostname)
except Exception: # noqa: S110
pass
try:
parts.append(platform.machine())
parts.append(platform.processor())
parts.append(platform.architecture()[0])
except Exception: # noqa: S110
pass
try:
container_id = os.environ.get(
"HOSTNAME", os.environ.get("CONTAINER_ID", "")
)
if container_id:
parts.append(container_id)
except Exception: # noqa: S110
pass
if parts:
return "-".join(filter(None, parts))
except Exception: # noqa: S110
pass
return None
def _user_data_file() -> Path:
base = Path(db_storage_path())
base.mkdir(parents=True, exist_ok=True)
@@ -223,8 +97,8 @@ def _load_user_data() -> dict:
if p.exists():
try:
return json.loads(p.read_text())
except (json.JSONDecodeError, OSError, PermissionError) as e:
logger.warning(f"Failed to load user data: {e}")
except Exception:
pass
return {}
@@ -232,8 +106,8 @@ def _save_user_data(data: dict) -> None:
try:
p = _user_data_file()
p.write_text(json.dumps(data, indent=2))
except (OSError, PermissionError) as e:
logger.warning(f"Failed to save user data: {e}")
except Exception:
pass
def get_user_id() -> str:
@@ -277,103 +151,3 @@ def mark_first_execution_done() -> None:
}
)
_save_user_data(data)
def safe_serialize_to_dict(obj, exclude: set[str] | None = None) -> dict[str, Any]:
"""Safely serialize an object to a dictionary for event data."""
try:
serialized = to_serializable(obj, exclude)
if isinstance(serialized, dict):
return serialized
return {"serialized_data": serialized}
except Exception as e:
return {"serialization_error": str(e), "object_type": type(obj).__name__}
def truncate_messages(messages, max_content_length=500, max_messages=5):
"""Truncate message content and limit number of messages"""
if not messages or not isinstance(messages, list):
return messages
limited_messages = messages[:max_messages]
for msg in limited_messages:
if isinstance(msg, dict) and "content" in msg:
content = msg["content"]
if len(content) > max_content_length:
msg["content"] = content[:max_content_length] + "..."
return limited_messages
def should_auto_collect_first_time_traces() -> bool:
"""True if we should auto-collect traces for first-time user."""
if _is_test_environment():
return False
return is_first_execution()
def prompt_user_for_trace_viewing(timeout_seconds: int = 20) -> bool:
"""
Prompt user if they want to see their traces with timeout.
Returns True if user wants to see traces, False otherwise.
"""
if _is_test_environment():
return False
try:
import threading
console = Console()
content = Text()
content.append("🔍 ", style="cyan bold")
content.append(
"Detailed execution traces are available!\n\n", style="cyan bold"
)
content.append("View insights including:\n", style="white")
content.append(" • Agent decision-making process\n", style="bright_blue")
content.append(" • Task execution flow and timing\n", style="bright_blue")
content.append(" • Tool usage details", style="bright_blue")
panel = Panel(
content,
title="[bold cyan]Execution Traces[/bold cyan]",
border_style="cyan",
padding=(1, 2),
)
console.print("\n")
console.print(panel)
prompt_text = click.style(
f"Would you like to view your execution traces? [y/N] ({timeout_seconds}s timeout): ",
fg="white",
bold=True,
)
click.echo(prompt_text, nl=False)
result = [False]
def get_input():
try:
response = input().strip().lower()
result[0] = response in ["y", "yes"]
except (EOFError, KeyboardInterrupt):
result[0] = False
input_thread = threading.Thread(target=get_input, daemon=True)
input_thread.start()
input_thread.join(timeout=timeout_seconds)
if input_thread.is_alive():
return False
return result[0]
except Exception:
return False
def mark_first_execution_completed() -> None:
"""Mark first execution as completed (called after trace prompt)."""
mark_first_execution_done()

View File

@@ -2,4 +2,4 @@
This module contains all event types used throughout the CrewAI system
for monitoring and extending agent, crew, task, and tool execution.
"""
"""

View File

@@ -2,15 +2,14 @@
from __future__ import annotations
from collections.abc import Sequence
from typing import Any
from typing import Any, Dict, List, Optional, Sequence, Union
from pydantic import ConfigDict, model_validator
from pydantic import model_validator
from crewai.agents.agent_builder.base_agent import BaseAgent
from crewai.events.base_events import BaseEvent
from crewai.tools.base_tool import BaseTool
from crewai.tools.structured_tool import CrewStructuredTool
from crewai.events.base_events import BaseEvent
class AgentExecutionStartedEvent(BaseEvent):
@@ -18,11 +17,11 @@ class AgentExecutionStartedEvent(BaseEvent):
agent: BaseAgent
task: Any
tools: Sequence[BaseTool | CrewStructuredTool] | None
tools: Optional[Sequence[Union[BaseTool, CrewStructuredTool]]]
task_prompt: str
type: str = "agent_execution_started"
model_config = ConfigDict(arbitrary_types_allowed=True)
model_config = {"arbitrary_types_allowed": True}
@model_validator(mode="after")
def set_fingerprint_data(self):
@@ -46,7 +45,7 @@ class AgentExecutionCompletedEvent(BaseEvent):
output: str
type: str = "agent_execution_completed"
model_config = ConfigDict(arbitrary_types_allowed=True)
model_config = {"arbitrary_types_allowed": True}
@model_validator(mode="after")
def set_fingerprint_data(self):
@@ -70,7 +69,7 @@ class AgentExecutionErrorEvent(BaseEvent):
error: str
type: str = "agent_execution_error"
model_config = ConfigDict(arbitrary_types_allowed=True)
model_config = {"arbitrary_types_allowed": True}
@model_validator(mode="after")
def set_fingerprint_data(self):
@@ -90,18 +89,18 @@ class AgentExecutionErrorEvent(BaseEvent):
class LiteAgentExecutionStartedEvent(BaseEvent):
"""Event emitted when a LiteAgent starts executing"""
agent_info: dict[str, Any]
tools: Sequence[BaseTool | CrewStructuredTool] | None
messages: str | list[dict[str, str]]
agent_info: Dict[str, Any]
tools: Optional[Sequence[Union[BaseTool, CrewStructuredTool]]]
messages: Union[str, List[Dict[str, str]]]
type: str = "lite_agent_execution_started"
model_config = ConfigDict(arbitrary_types_allowed=True)
model_config = {"arbitrary_types_allowed": True}
class LiteAgentExecutionCompletedEvent(BaseEvent):
"""Event emitted when a LiteAgent completes execution"""
agent_info: dict[str, Any]
agent_info: Dict[str, Any]
output: str
type: str = "lite_agent_execution_completed"
@@ -109,7 +108,7 @@ class LiteAgentExecutionCompletedEvent(BaseEvent):
class LiteAgentExecutionErrorEvent(BaseEvent):
"""Event emitted when a LiteAgent encounters an error during execution"""
agent_info: dict[str, Any]
agent_info: Dict[str, Any]
error: str
type: str = "lite_agent_execution_error"

View File

@@ -1,4 +1,4 @@
from typing import TYPE_CHECKING, Any
from typing import TYPE_CHECKING, Any, Dict, Optional, Union
from crewai.events.base_events import BaseEvent
@@ -11,8 +11,8 @@ else:
class CrewBaseEvent(BaseEvent):
"""Base class for crew events with fingerprint handling"""
crew_name: str | None
crew: Crew | None = None
crew_name: Optional[str]
crew: Optional[Crew] = None
def __init__(self, **data):
super().__init__(**data)
@@ -38,7 +38,7 @@ class CrewBaseEvent(BaseEvent):
class CrewKickoffStartedEvent(CrewBaseEvent):
"""Event emitted when a crew starts execution"""
inputs: dict[str, Any] | None
inputs: Optional[Dict[str, Any]]
type: str = "crew_kickoff_started"
@@ -62,7 +62,7 @@ class CrewTrainStartedEvent(CrewBaseEvent):
n_iterations: int
filename: str
inputs: dict[str, Any] | None
inputs: Optional[Dict[str, Any]]
type: str = "crew_train_started"
@@ -85,8 +85,8 @@ class CrewTestStartedEvent(CrewBaseEvent):
"""Event emitted when a crew starts testing"""
n_iterations: int
eval_llm: str | Any | None
inputs: dict[str, Any] | None
eval_llm: Optional[Union[str, Any]]
inputs: Optional[Dict[str, Any]]
type: str = "crew_test_started"

View File

@@ -1,4 +1,4 @@
from typing import Any
from typing import Any, Dict, Optional, Union
from pydantic import BaseModel, ConfigDict
@@ -16,7 +16,7 @@ class FlowStartedEvent(FlowEvent):
"""Event emitted when a flow starts execution"""
flow_name: str
inputs: dict[str, Any] | None = None
inputs: Optional[Dict[str, Any]] = None
type: str = "flow_started"
@@ -32,8 +32,8 @@ class MethodExecutionStartedEvent(FlowEvent):
flow_name: str
method_name: str
state: dict[str, Any] | BaseModel
params: dict[str, Any] | None = None
state: Union[Dict[str, Any], BaseModel]
params: Optional[Dict[str, Any]] = None
type: str = "method_execution_started"
@@ -43,7 +43,7 @@ class MethodExecutionFinishedEvent(FlowEvent):
flow_name: str
method_name: str
result: Any = None
state: dict[str, Any] | BaseModel
state: Union[Dict[str, Any], BaseModel]
type: str = "method_execution_finished"
@@ -62,7 +62,7 @@ class FlowFinishedEvent(FlowEvent):
"""Event emitted when a flow completes execution"""
flow_name: str
result: Any | None = None
result: Optional[Any] = None
type: str = "flow_finished"

View File

@@ -1,6 +1,7 @@
from crewai.agents.agent_builder.base_agent import BaseAgent
from crewai.events.base_events import BaseEvent
from crewai.agents.agent_builder.base_agent import BaseAgent
class KnowledgeRetrievalStartedEvent(BaseEvent):
"""Event emitted when a knowledge retrieval is started."""

View File

@@ -1,5 +1,5 @@
from enum import Enum
from typing import Any
from typing import Any, Dict, List, Optional, Union
from pydantic import BaseModel
@@ -7,14 +7,14 @@ from crewai.events.base_events import BaseEvent
class LLMEventBase(BaseEvent):
task_name: str | None = None
task_id: str | None = None
task_name: Optional[str] = None
task_id: Optional[str] = None
agent_id: str | None = None
agent_role: str | None = None
agent_id: Optional[str] = None
agent_role: Optional[str] = None
from_task: Any | None = None
from_agent: Any | None = None
from_task: Optional[Any] = None
from_agent: Optional[Any] = None
def __init__(self, **data):
super().__init__(**data)
@@ -38,11 +38,11 @@ class LLMCallStartedEvent(LLMEventBase):
"""
type: str = "llm_call_started"
model: str | None = None
messages: str | list[dict[str, Any]] | None = None
tools: list[dict[str, Any]] | None = None
callbacks: list[Any] | None = None
available_functions: dict[str, Any] | None = None
model: Optional[str] = None
messages: Optional[Union[str, List[Dict[str, Any]]]] = None
tools: Optional[List[dict[str, Any]]] = None
callbacks: Optional[List[Any]] = None
available_functions: Optional[Dict[str, Any]] = None
class LLMCallCompletedEvent(LLMEventBase):
@@ -52,7 +52,7 @@ class LLMCallCompletedEvent(LLMEventBase):
messages: str | list[dict[str, Any]] | None = None
response: Any
call_type: LLMCallType
model: str | None = None
model: Optional[str] = None
class LLMCallFailedEvent(LLMEventBase):
@@ -64,13 +64,13 @@ class LLMCallFailedEvent(LLMEventBase):
class FunctionCall(BaseModel):
arguments: str
name: str | None = None
name: Optional[str] = None
class ToolCall(BaseModel):
id: str | None = None
id: Optional[str] = None
function: FunctionCall
type: str | None = None
type: Optional[str] = None
index: int
@@ -79,4 +79,4 @@ class LLMStreamChunkEvent(LLMEventBase):
type: str = "llm_stream_chunk"
chunk: str
tool_call: ToolCall | None = None
tool_call: Optional[ToolCall] = None

View File

@@ -1,6 +1,5 @@
from collections.abc import Callable
from inspect import getsource
from typing import Any
from typing import Any, Callable, Optional, Union
from crewai.events.base_events import BaseEvent
@@ -14,12 +13,12 @@ class LLMGuardrailStartedEvent(BaseEvent):
"""
type: str = "llm_guardrail_started"
guardrail: str | Callable
guardrail: Union[str, Callable]
retry_count: int
def __init__(self, **data):
from crewai.tasks.hallucination_guardrail import HallucinationGuardrail
from crewai.tasks.llm_guardrail import LLMGuardrail
from crewai.tasks.hallucination_guardrail import HallucinationGuardrail
super().__init__(**data)
@@ -42,5 +41,5 @@ class LLMGuardrailCompletedEvent(BaseEvent):
type: str = "llm_guardrail_completed"
success: bool
result: Any
error: str | None = None
error: Optional[str] = None
retry_count: int

View File

@@ -1,8 +1,6 @@
"""Agent logging events that don't reference BaseAgent to avoid circular imports."""
from typing import Any
from pydantic import ConfigDict
from typing import Any, Optional
from crewai.events.base_events import BaseEvent
@@ -11,7 +9,7 @@ class AgentLogsStartedEvent(BaseEvent):
"""Event emitted when agent logs should be shown at start"""
agent_role: str
task_description: str | None = None
task_description: Optional[str] = None
verbose: bool = False
type: str = "agent_logs_started"
@@ -24,4 +22,4 @@ class AgentLogsExecutionEvent(BaseEvent):
verbose: bool = False
type: str = "agent_logs_execution"
model_config = ConfigDict(arbitrary_types_allowed=True)
model_config = {"arbitrary_types_allowed": True}

View File

@@ -1,4 +1,4 @@
from typing import Any
from typing import Any, Dict, Optional
from crewai.events.base_events import BaseEvent
@@ -7,12 +7,12 @@ class MemoryBaseEvent(BaseEvent):
"""Base event for memory operations"""
type: str
task_id: str | None = None
task_name: str | None = None
from_task: Any | None = None
from_agent: Any | None = None
agent_role: str | None = None
agent_id: str | None = None
task_id: Optional[str] = None
task_name: Optional[str] = None
from_task: Optional[Any] = None
from_agent: Optional[Any] = None
agent_role: Optional[str] = None
agent_id: Optional[str] = None
def __init__(self, **data):
super().__init__(**data)
@@ -26,7 +26,7 @@ class MemoryQueryStartedEvent(MemoryBaseEvent):
type: str = "memory_query_started"
query: str
limit: int
score_threshold: float | None = None
score_threshold: Optional[float] = None
class MemoryQueryCompletedEvent(MemoryBaseEvent):
@@ -36,7 +36,7 @@ class MemoryQueryCompletedEvent(MemoryBaseEvent):
query: str
results: Any
limit: int
score_threshold: float | None = None
score_threshold: Optional[float] = None
query_time_ms: float
@@ -46,7 +46,7 @@ class MemoryQueryFailedEvent(MemoryBaseEvent):
type: str = "memory_query_failed"
query: str
limit: int
score_threshold: float | None = None
score_threshold: Optional[float] = None
error: str
@@ -54,9 +54,9 @@ class MemorySaveStartedEvent(MemoryBaseEvent):
"""Event emitted when a memory save operation is started"""
type: str = "memory_save_started"
value: str | None = None
metadata: dict[str, Any] | None = None
agent_role: str | None = None
value: Optional[str] = None
metadata: Optional[Dict[str, Any]] = None
agent_role: Optional[str] = None
class MemorySaveCompletedEvent(MemoryBaseEvent):
@@ -64,8 +64,8 @@ class MemorySaveCompletedEvent(MemoryBaseEvent):
type: str = "memory_save_completed"
value: str
metadata: dict[str, Any] | None = None
agent_role: str | None = None
metadata: Optional[Dict[str, Any]] = None
agent_role: Optional[str] = None
save_time_ms: float
@@ -73,9 +73,9 @@ class MemorySaveFailedEvent(MemoryBaseEvent):
"""Event emitted when a memory save operation fails"""
type: str = "memory_save_failed"
value: str | None = None
metadata: dict[str, Any] | None = None
agent_role: str | None = None
value: Optional[str] = None
metadata: Optional[Dict[str, Any]] = None
agent_role: Optional[str] = None
error: str
@@ -83,13 +83,13 @@ class MemoryRetrievalStartedEvent(MemoryBaseEvent):
"""Event emitted when memory retrieval for a task prompt starts"""
type: str = "memory_retrieval_started"
task_id: str | None = None
task_id: Optional[str] = None
class MemoryRetrievalCompletedEvent(MemoryBaseEvent):
"""Event emitted when memory retrieval for a task prompt completes successfully"""
type: str = "memory_retrieval_completed"
task_id: str | None = None
task_id: Optional[str] = None
memory_content: str
retrieval_time_ms: float

View File

@@ -1,6 +1,5 @@
from typing import Any
from crewai.events.base_events import BaseEvent
from typing import Any, Optional
class ReasoningEvent(BaseEvent):
@@ -10,10 +9,10 @@ class ReasoningEvent(BaseEvent):
attempt: int = 1
agent_role: str
task_id: str
task_name: str | None = None
from_task: Any | None = None
agent_id: str | None = None
from_agent: Any | None = None
task_name: Optional[str] = None
from_task: Optional[Any] = None
agent_id: Optional[str] = None
from_agent: Optional[Any] = None
def __init__(self, **data):
super().__init__(**data)

View File

@@ -1,15 +1,15 @@
from typing import Any
from typing import Any, Optional
from crewai.events.base_events import BaseEvent
from crewai.tasks.task_output import TaskOutput
from crewai.events.base_events import BaseEvent
class TaskStartedEvent(BaseEvent):
"""Event emitted when a task starts"""
type: str = "task_started"
context: str | None
task: Any | None = None
context: Optional[str]
task: Optional[Any] = None
def __init__(self, **data):
super().__init__(**data)
@@ -29,7 +29,7 @@ class TaskCompletedEvent(BaseEvent):
output: TaskOutput
type: str = "task_completed"
task: Any | None = None
task: Optional[Any] = None
def __init__(self, **data):
super().__init__(**data)
@@ -49,7 +49,7 @@ class TaskFailedEvent(BaseEvent):
error: str
type: str = "task_failed"
task: Any | None = None
task: Optional[Any] = None
def __init__(self, **data):
super().__init__(**data)
@@ -69,7 +69,7 @@ class TaskEvaluationEvent(BaseEvent):
type: str = "task_evaluation"
evaluation_type: str
task: Any | None = None
task: Optional[Any] = None
def __init__(self, **data):
super().__init__(**data)

View File

@@ -1,8 +1,5 @@
from collections.abc import Callable
from datetime import datetime
from typing import Any
from pydantic import ConfigDict
from typing import Any, Callable, Dict, Optional
from crewai.events.base_events import BaseEvent
@@ -10,21 +7,21 @@ from crewai.events.base_events import BaseEvent
class ToolUsageEvent(BaseEvent):
"""Base event for tool usage tracking"""
agent_key: str | None = None
agent_role: str | None = None
agent_id: str | None = None
agent_key: Optional[str] = None
agent_role: Optional[str] = None
agent_id: Optional[str] = None
tool_name: str
tool_args: dict[str, Any] | str
tool_class: str | None = None
tool_args: Dict[str, Any] | str
tool_class: Optional[str] = None
run_attempts: int | None = None
delegations: int | None = None
agent: Any | None = None
task_name: str | None = None
task_id: str | None = None
from_task: Any | None = None
from_agent: Any | None = None
agent: Optional[Any] = None
task_name: Optional[str] = None
task_id: Optional[str] = None
from_task: Optional[Any] = None
from_agent: Optional[Any] = None
model_config = ConfigDict(arbitrary_types_allowed=True)
model_config = {"arbitrary_types_allowed": True}
def __init__(self, **data):
super().__init__(**data)
@@ -84,9 +81,9 @@ class ToolExecutionErrorEvent(BaseEvent):
error: Any
type: str = "tool_execution_error"
tool_name: str
tool_args: dict[str, Any]
tool_args: Dict[str, Any]
tool_class: Callable
agent: Any | None = None
agent: Optional[Any] = None
def __init__(self, **data):
super().__init__(**data)

View File

@@ -1,25 +1,25 @@
from typing import Any, ClassVar
from typing import Any, Dict, Optional
from rich.console import Console
from rich.live import Live
from rich.panel import Panel
from rich.syntax import Syntax
from rich.text import Text
from rich.tree import Tree
from rich.live import Live
from rich.syntax import Syntax
class ConsoleFormatter:
current_crew_tree: Tree | None = None
current_task_branch: Tree | None = None
current_agent_branch: Tree | None = None
current_tool_branch: Tree | None = None
current_flow_tree: Tree | None = None
current_method_branch: Tree | None = None
current_lite_agent_branch: Tree | None = None
tool_usage_counts: ClassVar[dict[str, int]] = {}
current_reasoning_branch: Tree | None = None # Track reasoning status
current_crew_tree: Optional[Tree] = None
current_task_branch: Optional[Tree] = None
current_agent_branch: Optional[Tree] = None
current_tool_branch: Optional[Tree] = None
current_flow_tree: Optional[Tree] = None
current_method_branch: Optional[Tree] = None
current_lite_agent_branch: Optional[Tree] = None
tool_usage_counts: Dict[str, int] = {}
current_reasoning_branch: Optional[Tree] = None # Track reasoning status
_live_paused: bool = False
current_llm_tool_tree: Tree | None = None
current_llm_tool_tree: Optional[Tree] = None
def __init__(self, verbose: bool = False):
self.console = Console(width=None)
@@ -29,7 +29,7 @@ class ConsoleFormatter:
# instance so the previous render is replaced instead of writing a new one.
# Once any non-Tree renderable is printed we stop the Live session so the
# final Tree persists on the terminal.
self._live: Live | None = None
self._live: Optional[Live] = None
def create_panel(self, content: Text, title: str, style: str = "blue") -> Panel:
"""Create a standardized panel with consistent styling."""
@@ -45,7 +45,7 @@ class ConsoleFormatter:
title: str,
name: str,
status_style: str = "blue",
tool_args: dict[str, Any] | str = "",
tool_args: Dict[str, Any] | str = "",
**fields,
) -> Text:
"""Create standardized status content with consistent formatting."""
@@ -70,7 +70,7 @@ class ConsoleFormatter:
prefix: str,
name: str,
style: str = "blue",
status: str | None = None,
status: Optional[str] = None,
) -> None:
"""Update tree label with consistent formatting."""
label = Text()
@@ -115,7 +115,7 @@ class ConsoleFormatter:
self._live.update(tree, refresh=True)
return # Nothing else to do
# Case 2: blank line while a live session is running - ignore so we
# Case 2: blank line while a live session is running ignore so we
# don't break the in-place rendering behaviour
if len(args) == 0 and self._live:
return
@@ -156,7 +156,7 @@ class ConsoleFormatter:
def update_crew_tree(
self,
tree: Tree | None,
tree: Optional[Tree],
crew_name: str,
source_id: str,
status: str = "completed",
@@ -196,7 +196,7 @@ class ConsoleFormatter:
self.print_panel(content, title, style)
def create_crew_tree(self, crew_name: str, source_id: str) -> Tree | None:
def create_crew_tree(self, crew_name: str, source_id: str) -> Optional[Tree]:
"""Create and initialize a new crew tree with initial status."""
if not self.verbose:
return None
@@ -220,8 +220,8 @@ class ConsoleFormatter:
return tree
def create_task_branch(
self, crew_tree: Tree | None, task_id: str, task_name: str | None = None
) -> Tree | None:
self, crew_tree: Optional[Tree], task_id: str, task_name: Optional[str] = None
) -> Optional[Tree]:
"""Create and initialize a task branch."""
if not self.verbose:
return None
@@ -255,11 +255,11 @@ class ConsoleFormatter:
def update_task_status(
self,
crew_tree: Tree | None,
crew_tree: Optional[Tree],
task_id: str,
agent_role: str,
status: str = "completed",
task_name: str | None = None,
task_name: Optional[str] = None,
) -> None:
"""Update task status in the tree."""
if not self.verbose or crew_tree is None:
@@ -306,8 +306,8 @@ class ConsoleFormatter:
self.print_panel(content, panel_title, style)
def create_agent_branch(
self, task_branch: Tree | None, agent_role: str, crew_tree: Tree | None
) -> Tree | None:
self, task_branch: Optional[Tree], agent_role: str, crew_tree: Optional[Tree]
) -> Optional[Tree]:
"""Create and initialize an agent branch."""
if not self.verbose or not task_branch or not crew_tree:
return None
@@ -325,9 +325,9 @@ class ConsoleFormatter:
def update_agent_status(
self,
agent_branch: Tree | None,
agent_branch: Optional[Tree],
agent_role: str,
crew_tree: Tree | None,
crew_tree: Optional[Tree],
status: str = "completed",
) -> None:
"""Update agent status in the tree."""
@@ -336,7 +336,7 @@ class ConsoleFormatter:
# altering the tree. Keeping it a no-op avoids duplicate status lines.
return
def create_flow_tree(self, flow_name: str, flow_id: str) -> Tree | None:
def create_flow_tree(self, flow_name: str, flow_id: str) -> Optional[Tree]:
"""Create and initialize a flow tree."""
content = self.create_status_content(
"Starting Flow Execution", flow_name, "blue", ID=flow_id
@@ -356,7 +356,7 @@ class ConsoleFormatter:
return flow_tree
def start_flow(self, flow_name: str, flow_id: str) -> Tree | None:
def start_flow(self, flow_name: str, flow_id: str) -> Optional[Tree]:
"""Initialize a flow execution tree."""
flow_tree = Tree("")
flow_label = Text()
@@ -376,7 +376,7 @@ class ConsoleFormatter:
def update_flow_status(
self,
flow_tree: Tree | None,
flow_tree: Optional[Tree],
flow_name: str,
flow_id: str,
status: str = "completed",
@@ -423,11 +423,11 @@ class ConsoleFormatter:
def update_method_status(
self,
method_branch: Tree | None,
flow_tree: Tree | None,
method_branch: Optional[Tree],
flow_tree: Optional[Tree],
method_name: str,
status: str = "running",
) -> Tree | None:
) -> Optional[Tree]:
"""Update method status in the flow tree."""
if not flow_tree:
return None
@@ -480,7 +480,7 @@ class ConsoleFormatter:
def handle_llm_tool_usage_started(
self,
tool_name: str,
tool_args: dict[str, Any] | str,
tool_args: Dict[str, Any] | str,
):
# Create status content for the tool usage
content = self.create_status_content(
@@ -520,11 +520,11 @@ class ConsoleFormatter:
def handle_tool_usage_started(
self,
agent_branch: Tree | None,
agent_branch: Optional[Tree],
tool_name: str,
crew_tree: Tree | None,
tool_args: dict[str, Any] | str = "",
) -> Tree | None:
crew_tree: Optional[Tree],
tool_args: Dict[str, Any] | str = "",
) -> Optional[Tree]:
"""Handle tool usage started event."""
if not self.verbose:
return None
@@ -569,9 +569,9 @@ class ConsoleFormatter:
def handle_tool_usage_finished(
self,
tool_branch: Tree | None,
tool_branch: Optional[Tree],
tool_name: str,
crew_tree: Tree | None,
crew_tree: Optional[Tree],
) -> None:
"""Handle tool usage finished event."""
if not self.verbose or tool_branch is None:
@@ -600,10 +600,10 @@ class ConsoleFormatter:
def handle_tool_usage_error(
self,
tool_branch: Tree | None,
tool_branch: Optional[Tree],
tool_name: str,
error: str,
crew_tree: Tree | None,
crew_tree: Optional[Tree],
) -> None:
"""Handle tool usage error event."""
if not self.verbose:
@@ -631,9 +631,9 @@ class ConsoleFormatter:
def handle_llm_call_started(
self,
agent_branch: Tree | None,
crew_tree: Tree | None,
) -> Tree | None:
agent_branch: Optional[Tree],
crew_tree: Optional[Tree],
) -> Optional[Tree]:
"""Handle LLM call started event."""
if not self.verbose:
return None
@@ -672,9 +672,9 @@ class ConsoleFormatter:
def handle_llm_call_completed(
self,
tool_branch: Tree | None,
agent_branch: Tree | None,
crew_tree: Tree | None,
tool_branch: Optional[Tree],
agent_branch: Optional[Tree],
crew_tree: Optional[Tree],
) -> None:
"""Handle LLM call completed event."""
if not self.verbose:
@@ -736,7 +736,7 @@ class ConsoleFormatter:
self.print()
def handle_llm_call_failed(
self, tool_branch: Tree | None, error: str, crew_tree: Tree | None
self, tool_branch: Optional[Tree], error: str, crew_tree: Optional[Tree]
) -> None:
"""Handle LLM call failed event."""
if not self.verbose:
@@ -789,7 +789,7 @@ class ConsoleFormatter:
def handle_crew_test_started(
self, crew_name: str, source_id: str, n_iterations: int
) -> Tree | None:
) -> Optional[Tree]:
"""Handle crew test started event."""
if not self.verbose:
return None
@@ -823,7 +823,7 @@ class ConsoleFormatter:
return test_tree
def handle_crew_test_completed(
self, flow_tree: Tree | None, crew_name: str
self, flow_tree: Optional[Tree], crew_name: str
) -> None:
"""Handle crew test completed event."""
if not self.verbose:
@@ -913,7 +913,7 @@ class ConsoleFormatter:
self.print_panel(failure_content, "Test Failure", "red")
self.print()
def create_lite_agent_branch(self, lite_agent_role: str) -> Tree | None:
def create_lite_agent_branch(self, lite_agent_role: str) -> Optional[Tree]:
"""Create and initialize a lite agent branch."""
if not self.verbose:
return None
@@ -935,10 +935,10 @@ class ConsoleFormatter:
def update_lite_agent_status(
self,
lite_agent_branch: Tree | None,
lite_agent_branch: Optional[Tree],
lite_agent_role: str,
status: str = "completed",
**fields: dict[str, Any],
**fields: Dict[str, Any],
) -> None:
"""Update lite agent status in the tree."""
if not self.verbose or lite_agent_branch is None:
@@ -981,7 +981,7 @@ class ConsoleFormatter:
lite_agent_role: str,
status: str = "started",
error: Any = None,
**fields: dict[str, Any],
**fields: Dict[str, Any],
) -> None:
"""Handle lite agent execution events with consistent formatting."""
if not self.verbose:
@@ -1006,9 +1006,9 @@ class ConsoleFormatter:
def handle_knowledge_retrieval_started(
self,
agent_branch: Tree | None,
crew_tree: Tree | None,
) -> Tree | None:
agent_branch: Optional[Tree],
crew_tree: Optional[Tree],
) -> Optional[Tree]:
"""Handle knowledge retrieval started event."""
if not self.verbose:
return None
@@ -1034,13 +1034,13 @@ class ConsoleFormatter:
def handle_knowledge_retrieval_completed(
self,
agent_branch: Tree | None,
crew_tree: Tree | None,
agent_branch: Optional[Tree],
crew_tree: Optional[Tree],
retrieved_knowledge: Any,
) -> None:
"""Handle knowledge retrieval completed event."""
if not self.verbose:
return
return None
branch_to_use = self.current_lite_agent_branch or agent_branch
tree_to_use = branch_to_use or crew_tree
@@ -1062,7 +1062,7 @@ class ConsoleFormatter:
)
self.print(knowledge_panel)
self.print()
return
return None
knowledge_branch_found = False
for child in branch_to_use.children:
@@ -1111,18 +1111,18 @@ class ConsoleFormatter:
def handle_knowledge_query_started(
self,
agent_branch: Tree | None,
agent_branch: Optional[Tree],
task_prompt: str,
crew_tree: Tree | None,
crew_tree: Optional[Tree],
) -> None:
"""Handle knowledge query generated event."""
if not self.verbose:
return
return None
branch_to_use = self.current_lite_agent_branch or agent_branch
tree_to_use = branch_to_use or crew_tree
if branch_to_use is None or tree_to_use is None:
return
return None
query_branch = branch_to_use.add("")
self.update_tree_label(
@@ -1134,9 +1134,9 @@ class ConsoleFormatter:
def handle_knowledge_query_failed(
self,
agent_branch: Tree | None,
agent_branch: Optional[Tree],
error: str,
crew_tree: Tree | None,
crew_tree: Optional[Tree],
) -> None:
"""Handle knowledge query failed event."""
if not self.verbose:
@@ -1159,18 +1159,18 @@ class ConsoleFormatter:
def handle_knowledge_query_completed(
self,
agent_branch: Tree | None,
crew_tree: Tree | None,
agent_branch: Optional[Tree],
crew_tree: Optional[Tree],
) -> None:
"""Handle knowledge query completed event."""
if not self.verbose:
return
return None
branch_to_use = self.current_lite_agent_branch or agent_branch
tree_to_use = branch_to_use or crew_tree
if branch_to_use is None or tree_to_use is None:
return
return None
query_branch = branch_to_use.add("")
self.update_tree_label(query_branch, "", "Knowledge Query Completed", "green")
@@ -1180,9 +1180,9 @@ class ConsoleFormatter:
def handle_knowledge_search_query_failed(
self,
agent_branch: Tree | None,
agent_branch: Optional[Tree],
error: str,
crew_tree: Tree | None,
crew_tree: Optional[Tree],
) -> None:
"""Handle knowledge search query failed event."""
if not self.verbose:
@@ -1207,10 +1207,10 @@ class ConsoleFormatter:
def handle_reasoning_started(
self,
agent_branch: Tree | None,
agent_branch: Optional[Tree],
attempt: int,
crew_tree: Tree | None,
) -> Tree | None:
crew_tree: Optional[Tree],
) -> Optional[Tree]:
"""Handle agent reasoning started (or refinement) event."""
if not self.verbose:
return None
@@ -1249,7 +1249,7 @@ class ConsoleFormatter:
self,
plan: str,
ready: bool,
crew_tree: Tree | None,
crew_tree: Optional[Tree],
) -> None:
"""Handle agent reasoning completed event."""
if not self.verbose:
@@ -1292,7 +1292,7 @@ class ConsoleFormatter:
def handle_reasoning_failed(
self,
error: str,
crew_tree: Tree | None,
crew_tree: Optional[Tree],
) -> None:
"""Handle agent reasoning failure event."""
if not self.verbose:
@@ -1329,7 +1329,7 @@ class ConsoleFormatter:
def handle_agent_logs_started(
self,
agent_role: str,
task_description: str | None = None,
task_description: Optional[str] = None,
verbose: bool = False,
) -> None:
"""Handle agent logs started event."""
@@ -1367,11 +1367,10 @@ class ConsoleFormatter:
if not verbose:
return
from crewai.agents.parser import AgentAction, AgentFinish
import json
import re
from crewai.agents.parser import AgentAction, AgentFinish
agent_role = agent_role.partition("\n")[0]
if isinstance(formatted_answer, AgentAction):
@@ -1474,9 +1473,9 @@ class ConsoleFormatter:
def handle_memory_retrieval_started(
self,
agent_branch: Tree | None,
crew_tree: Tree | None,
) -> Tree | None:
agent_branch: Optional[Tree],
crew_tree: Optional[Tree],
) -> Optional[Tree]:
if not self.verbose:
return None
@@ -1498,13 +1497,13 @@ class ConsoleFormatter:
def handle_memory_retrieval_completed(
self,
agent_branch: Tree | None,
crew_tree: Tree | None,
agent_branch: Optional[Tree],
crew_tree: Optional[Tree],
memory_content: str,
retrieval_time_ms: float,
) -> None:
if not self.verbose:
return
return None
branch_to_use = self.current_lite_agent_branch or agent_branch
tree_to_use = branch_to_use or crew_tree
@@ -1529,7 +1528,7 @@ class ConsoleFormatter:
if branch_to_use is None or tree_to_use is None:
add_panel()
return
return None
memory_branch_found = False
for child in branch_to_use.children:
@@ -1566,13 +1565,13 @@ class ConsoleFormatter:
def handle_memory_query_completed(
self,
agent_branch: Tree | None,
agent_branch: Optional[Tree],
source_type: str,
query_time_ms: float,
crew_tree: Tree | None,
crew_tree: Optional[Tree],
) -> None:
if not self.verbose:
return
return None
branch_to_use = self.current_lite_agent_branch or agent_branch
tree_to_use = branch_to_use or crew_tree
@@ -1581,15 +1580,15 @@ class ConsoleFormatter:
branch_to_use = tree_to_use
if branch_to_use is None:
return
return None
memory_type = source_type.replace("_", " ").title()
for child in branch_to_use.children:
if "Memory Retrieval" in str(child.label):
for inner_child in child.children:
sources_branch = inner_child
if "Sources Used" in str(inner_child.label):
for child in child.children:
sources_branch = child
if "Sources Used" in str(child.label):
sources_branch.add(f"{memory_type} ({query_time_ms:.2f}ms)")
break
else:
@@ -1599,13 +1598,13 @@ class ConsoleFormatter:
def handle_memory_query_failed(
self,
agent_branch: Tree | None,
crew_tree: Tree | None,
agent_branch: Optional[Tree],
crew_tree: Optional[Tree],
error: str,
source_type: str,
) -> None:
if not self.verbose:
return
return None
branch_to_use = self.current_lite_agent_branch or agent_branch
tree_to_use = branch_to_use or crew_tree
@@ -1614,15 +1613,15 @@ class ConsoleFormatter:
branch_to_use = tree_to_use
if branch_to_use is None:
return
return None
memory_type = source_type.replace("_", " ").title()
for child in branch_to_use.children:
if "Memory Retrieval" in str(child.label):
for inner_child in child.children:
sources_branch = inner_child
if "Sources Used" in str(inner_child.label):
for child in child.children:
sources_branch = child
if "Sources Used" in str(child.label):
sources_branch.add(f"{memory_type} - Error: {error}")
break
else:
@@ -1631,16 +1630,16 @@ class ConsoleFormatter:
break
def handle_memory_save_started(
self, agent_branch: Tree | None, crew_tree: Tree | None
self, agent_branch: Optional[Tree], crew_tree: Optional[Tree]
) -> None:
if not self.verbose:
return
return None
branch_to_use = agent_branch or self.current_lite_agent_branch
tree_to_use = branch_to_use or crew_tree
if tree_to_use is None:
return
return None
for child in tree_to_use.children:
if "Memory Update" in str(child.label):
@@ -1656,19 +1655,19 @@ class ConsoleFormatter:
def handle_memory_save_completed(
self,
agent_branch: Tree | None,
crew_tree: Tree | None,
agent_branch: Optional[Tree],
crew_tree: Optional[Tree],
save_time_ms: float,
source_type: str,
) -> None:
if not self.verbose:
return
return None
branch_to_use = agent_branch or self.current_lite_agent_branch
tree_to_use = branch_to_use or crew_tree
if tree_to_use is None:
return
return None
memory_type = source_type.replace("_", " ").title()
content = f"{memory_type} Memory Saved ({save_time_ms:.2f}ms)"
@@ -1686,19 +1685,19 @@ class ConsoleFormatter:
def handle_memory_save_failed(
self,
agent_branch: Tree | None,
agent_branch: Optional[Tree],
error: str,
source_type: str,
crew_tree: Tree | None,
crew_tree: Optional[Tree],
) -> None:
if not self.verbose:
return
return None
branch_to_use = agent_branch or self.current_lite_agent_branch
tree_to_use = branch_to_use or crew_tree
if branch_to_use is None or tree_to_use is None:
return
return None
memory_type = source_type.replace("_", " ").title()
content = f"{memory_type} Memory Save Failed"
@@ -1739,7 +1738,7 @@ class ConsoleFormatter:
def handle_guardrail_completed(
self,
success: bool,
error: str | None,
error: Optional[str],
retry_count: int,
) -> None:
"""Display guardrail evaluation result.

View File

@@ -1,39 +1,40 @@
from crewai.experimental.evaluation import (
AgentEvaluationResult,
AgentEvaluator,
BaseEvaluator,
EvaluationScore,
EvaluationTraceCallback,
ExperimentResult,
ExperimentResults,
ExperimentRunner,
GoalAlignmentEvaluator,
MetricCategory,
ParameterExtractionEvaluator,
ReasoningEfficiencyEvaluator,
AgentEvaluationResult,
SemanticQualityEvaluator,
ToolInvocationEvaluator,
GoalAlignmentEvaluator,
ReasoningEfficiencyEvaluator,
ToolSelectionEvaluator,
create_default_evaluator,
ParameterExtractionEvaluator,
ToolInvocationEvaluator,
EvaluationTraceCallback,
create_evaluation_callbacks,
AgentEvaluator,
create_default_evaluator,
ExperimentRunner,
ExperimentResults,
ExperimentResult,
)
__all__ = [
"AgentEvaluationResult",
"AgentEvaluator",
"BaseEvaluator",
"EvaluationScore",
"EvaluationTraceCallback",
"ExperimentResult",
"ExperimentResults",
"ExperimentRunner",
"GoalAlignmentEvaluator",
"MetricCategory",
"ParameterExtractionEvaluator",
"ReasoningEfficiencyEvaluator",
"AgentEvaluationResult",
"SemanticQualityEvaluator",
"ToolInvocationEvaluator",
"GoalAlignmentEvaluator",
"ReasoningEfficiencyEvaluator",
"ToolSelectionEvaluator",
"create_default_evaluator",
"ParameterExtractionEvaluator",
"ToolInvocationEvaluator",
"EvaluationTraceCallback",
"create_evaluation_callbacks",
]
"AgentEvaluator",
"create_default_evaluator",
"ExperimentRunner",
"ExperimentResults",
"ExperimentResult"
]

View File

@@ -1,47 +1,51 @@
from crewai.experimental.evaluation.agent_evaluator import (
AgentEvaluator,
create_default_evaluator,
)
from crewai.experimental.evaluation.base_evaluator import (
AgentEvaluationResult,
BaseEvaluator,
EvaluationScore,
MetricCategory,
AgentEvaluationResult
)
from crewai.experimental.evaluation.metrics import (
SemanticQualityEvaluator,
GoalAlignmentEvaluator,
ReasoningEfficiencyEvaluator,
ToolSelectionEvaluator,
ParameterExtractionEvaluator,
ToolInvocationEvaluator
)
from crewai.experimental.evaluation.evaluation_listener import (
EvaluationTraceCallback,
create_evaluation_callbacks,
create_evaluation_callbacks
)
from crewai.experimental.evaluation.agent_evaluator import (
AgentEvaluator,
create_default_evaluator
)
from crewai.experimental.evaluation.experiment import (
ExperimentResult,
ExperimentResults,
ExperimentRunner,
)
from crewai.experimental.evaluation.metrics import (
GoalAlignmentEvaluator,
ParameterExtractionEvaluator,
ReasoningEfficiencyEvaluator,
SemanticQualityEvaluator,
ToolInvocationEvaluator,
ToolSelectionEvaluator,
ExperimentResults,
ExperimentResult
)
__all__ = [
"AgentEvaluationResult",
"AgentEvaluator",
"BaseEvaluator",
"EvaluationScore",
"EvaluationTraceCallback",
"ExperimentResult",
"ExperimentResults",
"ExperimentRunner",
"GoalAlignmentEvaluator",
"MetricCategory",
"ParameterExtractionEvaluator",
"ReasoningEfficiencyEvaluator",
"AgentEvaluationResult",
"SemanticQualityEvaluator",
"ToolInvocationEvaluator",
"GoalAlignmentEvaluator",
"ReasoningEfficiencyEvaluator",
"ToolSelectionEvaluator",
"create_default_evaluator",
"ParameterExtractionEvaluator",
"ToolInvocationEvaluator",
"EvaluationTraceCallback",
"create_evaluation_callbacks",
"AgentEvaluator",
"create_default_evaluator",
"ExperimentRunner",
"ExperimentResults",
"ExperimentResult"
]

View File

@@ -1,36 +1,34 @@
import threading
from collections.abc import Sequence
from typing import Any
from typing import Any, Optional
from crewai.agent import Agent
from crewai.agents.agent_builder.base_agent import BaseAgent
from crewai.events.event_bus import crewai_event_bus
from crewai.events.types.agent_events import (
AgentEvaluationCompletedEvent,
AgentEvaluationFailedEvent,
AgentEvaluationStartedEvent,
LiteAgentExecutionCompletedEvent,
)
from crewai.events.types.task_events import TaskCompletedEvent
from crewai.events.utils.console_formatter import ConsoleFormatter
from crewai.experimental.evaluation.base_evaluator import (
AgentAggregatedEvaluationResult,
AgentEvaluationResult,
AggregationStrategy,
BaseEvaluator,
)
from crewai.agent import Agent
from crewai.task import Task
from crewai.experimental.evaluation.evaluation_display import EvaluationDisplayFormatter
from crewai.events.types.agent_events import (
AgentEvaluationStartedEvent,
AgentEvaluationCompletedEvent,
AgentEvaluationFailedEvent,
)
from crewai.experimental.evaluation import BaseEvaluator, create_evaluation_callbacks
from collections.abc import Sequence
from crewai.events.event_bus import crewai_event_bus
from crewai.events.utils.console_formatter import ConsoleFormatter
from crewai.events.types.task_events import TaskCompletedEvent
from crewai.events.types.agent_events import LiteAgentExecutionCompletedEvent
from crewai.experimental.evaluation.base_evaluator import (
AgentAggregatedEvaluationResult,
EvaluationScore,
MetricCategory,
)
from crewai.experimental.evaluation.evaluation_display import EvaluationDisplayFormatter
from crewai.experimental.evaluation.evaluation_listener import (
create_evaluation_callbacks,
)
from crewai.task import Task
class ExecutionState:
current_agent_id: str | None = None
current_task_id: str | None = None
current_agent_id: Optional[str] = None
current_task_id: Optional[str] = None
def __init__(self):
self.traces = {}
@@ -42,10 +40,10 @@ class ExecutionState:
class AgentEvaluator:
def __init__(
self,
agents: list[Agent] | list[BaseAgent],
agents: list[Agent],
evaluators: Sequence[BaseEvaluator] | None = None,
):
self.agents: list[Agent] | list[BaseAgent] = agents
self.agents: list[Agent] = agents
self.evaluators: Sequence[BaseEvaluator] | None = evaluators
self.callback = create_evaluation_callbacks()
@@ -77,8 +75,7 @@ class AgentEvaluator:
)
def _handle_task_completed(self, source: Any, event: TaskCompletedEvent) -> None:
if event.task is None:
raise ValueError("TaskCompletedEvent must have a task")
assert event.task is not None
agent = event.task.agent
if (
agent
@@ -95,8 +92,9 @@ class AgentEvaluator:
state.current_agent_id = str(agent.id)
state.current_task_id = str(event.task.id)
if state.current_agent_id is None or state.current_task_id is None:
raise ValueError("Agent ID and Task ID must not be None")
assert (
state.current_agent_id is not None and state.current_task_id is not None
)
trace = self.callback.get_trace(
state.current_agent_id, state.current_task_id
)
@@ -148,8 +146,9 @@ class AgentEvaluator:
if not target_agent:
return
if state.current_agent_id is None or state.current_task_id is None:
raise ValueError("Agent ID and Task ID must not be None")
assert (
state.current_agent_id is not None and state.current_task_id is not None
)
trace = self.callback.get_trace(
state.current_agent_id, state.current_task_id
)
@@ -245,7 +244,7 @@ class AgentEvaluator:
def evaluate(
self,
agent: Agent | BaseAgent,
agent: Agent,
execution_trace: dict[str, Any],
final_output: Any,
state: ExecutionState,
@@ -256,8 +255,7 @@ class AgentEvaluator:
task_id=state.current_task_id or (str(task.id) if task else "unknown_task"),
)
if self.evaluators is None:
raise ValueError("Evaluators must be initialized")
assert self.evaluators is not None
task_id = str(task.id) if task else None
for evaluator in self.evaluators:
try:
@@ -278,7 +276,7 @@ class AgentEvaluator:
metric_category=evaluator.metric_category,
score=score,
)
except Exception as e: # noqa: PERF203
except Exception as e:
self.emit_evaluation_failed_event(
agent_role=agent.role,
agent_id=str(agent.id),
@@ -286,7 +284,7 @@ class AgentEvaluator:
error=str(e),
)
self.console_formatter.print(
f"Error in {evaluator.metric_category.value} evaluator: {e!s}"
f"Error in {evaluator.metric_category.value} evaluator: {str(e)}"
)
return result
@@ -339,14 +337,14 @@ class AgentEvaluator:
)
def create_default_evaluator(agents: list[Agent] | list[BaseAgent], llm: None = None):
def create_default_evaluator(agents: list[Agent], llm: None = None):
from crewai.experimental.evaluation import (
GoalAlignmentEvaluator,
ParameterExtractionEvaluator,
ReasoningEfficiencyEvaluator,
SemanticQualityEvaluator,
ToolInvocationEvaluator,
ToolSelectionEvaluator,
ParameterExtractionEvaluator,
ToolInvocationEvaluator,
ReasoningEfficiencyEvaluator,
)
evaluators = [

View File

@@ -1,17 +1,15 @@
import abc
import enum
from enum import Enum
from typing import Any
from typing import Any, Dict, List, Optional
from pydantic import BaseModel, Field
from crewai.agent import Agent
from crewai.agents.agent_builder.base_agent import BaseAgent
from crewai.llm import BaseLLM
from crewai.task import Task
from crewai.llm import BaseLLM
from crewai.utilities.llm_utils import create_llm
class MetricCategory(enum.Enum):
GOAL_ALIGNMENT = "goal_alignment"
SEMANTIC_QUALITY = "semantic_quality"
@@ -21,7 +19,7 @@ class MetricCategory(enum.Enum):
TOOL_INVOCATION = "tool_invocation"
def title(self):
return self.value.replace("_", " ").title()
return self.value.replace('_', ' ').title()
class EvaluationScore(BaseModel):
@@ -29,13 +27,15 @@ class EvaluationScore(BaseModel):
default=5.0,
description="Numeric score from 0-10 where 0 is worst and 10 is best, None if not applicable",
ge=0.0,
le=10.0,
le=10.0
)
feedback: str = Field(
default="", description="Detailed feedback explaining the evaluation score"
default="",
description="Detailed feedback explaining the evaluation score"
)
raw_response: str | None = Field(
default=None, description="Raw response from the evaluator (e.g., LLM)"
default=None,
description="Raw response from the evaluator (e.g., LLM)"
)
def __str__(self) -> str:
@@ -56,8 +56,8 @@ class BaseEvaluator(abc.ABC):
@abc.abstractmethod
def evaluate(
self,
agent: Agent | BaseAgent,
execution_trace: dict[str, Any],
agent: Agent,
execution_trace: Dict[str, Any],
final_output: Any,
task: Task | None = None,
) -> EvaluationScore:
@@ -67,8 +67,9 @@ class BaseEvaluator(abc.ABC):
class AgentEvaluationResult(BaseModel):
agent_id: str = Field(description="ID of the evaluated agent")
task_id: str = Field(description="ID of the task that was executed")
metrics: dict[MetricCategory, EvaluationScore] = Field(
default_factory=dict, description="Evaluation scores for each metric category"
metrics: Dict[MetricCategory, EvaluationScore] = Field(
default_factory=dict,
description="Evaluation scores for each metric category"
)
@@ -80,23 +81,33 @@ class AggregationStrategy(Enum):
class AgentAggregatedEvaluationResult(BaseModel):
agent_id: str = Field(default="", description="ID of the agent")
agent_role: str = Field(default="", description="Role of the agent")
agent_id: str = Field(
default="",
description="ID of the agent"
)
agent_role: str = Field(
default="",
description="Role of the agent"
)
task_count: int = Field(
default=0, description="Number of tasks included in this aggregation"
default=0,
description="Number of tasks included in this aggregation"
)
aggregation_strategy: AggregationStrategy = Field(
default=AggregationStrategy.SIMPLE_AVERAGE,
description="Strategy used for aggregation",
description="Strategy used for aggregation"
)
metrics: dict[MetricCategory, EvaluationScore] = Field(
default_factory=dict, description="Aggregated metrics across all tasks"
metrics: Dict[MetricCategory, EvaluationScore] = Field(
default_factory=dict,
description="Aggregated metrics across all tasks"
)
task_results: list[str] = Field(
default_factory=list, description="IDs of tasks included in this aggregation"
task_results: List[str] = Field(
default_factory=list,
description="IDs of tasks included in this aggregation"
)
overall_score: float | None = Field(
default=None, description="Overall score for this agent"
overall_score: Optional[float] = Field(
default=None,
description="Overall score for this agent"
)
def __str__(self) -> str:
@@ -108,7 +119,7 @@ class AgentAggregatedEvaluationResult(BaseModel):
result += f"\n\n- {category.value.upper()}: {score.score}/10\n"
if score.feedback:
detailed_feedback = "\n ".join(score.feedback.split("\n"))
detailed_feedback = "\n ".join(score.feedback.split('\n'))
result += f" {detailed_feedback}\n"
return result
return result

View File

@@ -1,18 +1,16 @@
from collections import defaultdict
from collections.abc import Sequence
from typing import Any
from rich.box import HEAVY_EDGE, ROUNDED
from typing import Dict, Any, List
from rich.table import Table
from crewai.events.utils.console_formatter import ConsoleFormatter
from rich.box import HEAVY_EDGE, ROUNDED
from collections.abc import Sequence
from crewai.experimental.evaluation.base_evaluator import (
AgentAggregatedEvaluationResult,
AgentEvaluationResult,
AggregationStrategy,
EvaluationScore,
AgentEvaluationResult,
MetricCategory,
)
from crewai.experimental.evaluation import EvaluationScore
from crewai.events.utils.console_formatter import ConsoleFormatter
from crewai.utilities.llm_utils import create_llm
@@ -21,7 +19,7 @@ class EvaluationDisplayFormatter:
self.console_formatter = ConsoleFormatter()
def display_evaluation_with_feedback(
self, iterations_results: dict[int, dict[str, list[Any]]]
self, iterations_results: Dict[int, Dict[str, List[Any]]]
):
if not iterations_results:
self.console_formatter.print(
@@ -101,7 +99,7 @@ class EvaluationDisplayFormatter:
def display_summary_results(
self,
iterations_results: dict[int, dict[str, list[AgentEvaluationResult]]],
iterations_results: Dict[int, Dict[str, List[AgentAggregatedEvaluationResult]]],
):
if not iterations_results:
self.console_formatter.print(
@@ -282,7 +280,7 @@ class EvaluationDisplayFormatter:
feedback_summary = feedbacks[0]
aggregated_metrics[category] = EvaluationScore(
score=avg_score, feedback=feedback_summary or ""
score=avg_score, feedback=feedback_summary
)
overall_score = None
@@ -306,25 +304,25 @@ class EvaluationDisplayFormatter:
self,
agent_role: str,
metric: str,
feedbacks: list[str],
scores: list[float | None],
feedbacks: List[str],
scores: List[float | None],
strategy: AggregationStrategy,
) -> str:
if len(feedbacks) <= 2 and all(len(fb) < 200 for fb in feedbacks):
return "\n\n".join(
[f"Feedback {i + 1}: {fb}" for i, fb in enumerate(feedbacks)]
[f"Feedback {i+1}: {fb}" for i, fb in enumerate(feedbacks)]
)
try:
llm = create_llm()
formatted_feedbacks = []
for i, (feedback, score) in enumerate(zip(feedbacks, scores, strict=False)):
for i, (feedback, score) in enumerate(zip(feedbacks, scores)):
if len(feedback) > 500:
feedback = feedback[:500] + "..."
score_text = f"{score:.1f}" if score is not None else "N/A"
formatted_feedbacks.append(
f"Feedback #{i + 1} (Score: {score_text}):\n{feedback}"
f"Feedback #{i+1} (Score: {score_text}):\n{feedback}"
)
all_feedbacks = "\n\n" + "\n\n---\n\n".join(formatted_feedbacks)
@@ -367,9 +365,10 @@ class EvaluationDisplayFormatter:
""",
},
]
if llm is None:
raise ValueError("LLM must be initialized")
return llm.call(prompt)
assert llm is not None
response = llm.call(prompt)
return response
except Exception:
return "Synthesized from multiple tasks: " + "\n\n".join(

View File

@@ -1,25 +1,26 @@
from collections.abc import Sequence
from datetime import datetime
from typing import Any
from typing import Any, Dict, Optional
from crewai.agents.agent_builder.base_agent import BaseAgent
from collections.abc import Sequence
from crewai.agent import Agent
from crewai.task import Task
from crewai.events.base_event_listener import BaseEventListener
from crewai.events.event_bus import CrewAIEventsBus
from crewai.events.types.agent_events import (
AgentExecutionCompletedEvent,
AgentExecutionStartedEvent,
LiteAgentExecutionCompletedEvent,
AgentExecutionCompletedEvent,
LiteAgentExecutionStartedEvent,
LiteAgentExecutionCompletedEvent,
)
from crewai.events.types.llm_events import LLMCallCompletedEvent, LLMCallStartedEvent
from crewai.events.types.tool_usage_events import (
ToolUsageFinishedEvent,
ToolUsageErrorEvent,
ToolExecutionErrorEvent,
ToolSelectionErrorEvent,
ToolUsageErrorEvent,
ToolUsageFinishedEvent,
ToolValidateInputErrorEvent,
)
from crewai.task import Task
from crewai.events.types.llm_events import LLMCallStartedEvent, LLMCallCompletedEvent
class EvaluationTraceCallback(BaseEventListener):
@@ -135,7 +136,7 @@ class EvaluationTraceCallback(BaseEventListener):
def _init_trace(self, trace_key: str, **kwargs: Any):
self.traces[trace_key] = kwargs
def on_agent_start(self, agent: BaseAgent, task: Task):
def on_agent_start(self, agent: Agent, task: Task):
self.current_agent_id = agent.id
self.current_task_id = task.id
@@ -150,7 +151,7 @@ class EvaluationTraceCallback(BaseEventListener):
final_output=None,
)
def on_agent_finish(self, agent: BaseAgent, task: Task, output: Any):
def on_agent_finish(self, agent: Agent, task: Task, output: Any):
trace_key = f"{agent.id}_{task.id}"
if trace_key in self.traces:
self.traces[trace_key]["final_output"] = output
@@ -252,7 +253,7 @@ class EvaluationTraceCallback(BaseEventListener):
if hasattr(self, "current_llm_call"):
self.current_llm_call = {}
def get_trace(self, agent_id: str, task_id: str) -> dict[str, Any] | None:
def get_trace(self, agent_id: str, task_id: str) -> Optional[Dict[str, Any]]:
trace_key = f"{agent_id}_{task_id}"
return self.traces.get(trace_key)

View File

@@ -1,7 +1,8 @@
from crewai.experimental.evaluation.experiment.result import (
ExperimentResult,
ExperimentResults,
)
from crewai.experimental.evaluation.experiment.runner import ExperimentRunner
from crewai.experimental.evaluation.experiment.result import ExperimentResults, ExperimentResult
__all__ = ["ExperimentResult", "ExperimentResults", "ExperimentRunner"]
__all__ = [
"ExperimentRunner",
"ExperimentResults",
"ExperimentResult"
]

View File

@@ -2,60 +2,45 @@ import json
import os
from datetime import datetime, timezone
from typing import Any
from pydantic import BaseModel
class ExperimentResult(BaseModel):
identifier: str
inputs: dict[str, Any]
score: float | dict[str, float]
expected_score: float | dict[str, float]
score: int | dict[str, int | float]
expected_score: int | dict[str, int | float]
passed: bool
agent_evaluations: dict[str, Any] | None = None
class ExperimentResults:
def __init__(
self, results: list[ExperimentResult], metadata: dict[str, Any] | None = None
):
def __init__(self, results: list[ExperimentResult], metadata: dict[str, Any] | None = None):
self.results = results
self.metadata = metadata or {}
self.timestamp = datetime.now(timezone.utc)
from crewai.experimental.evaluation.experiment.result_display import (
ExperimentResultsDisplay,
)
from crewai.experimental.evaluation.experiment.result_display import ExperimentResultsDisplay
self.display = ExperimentResultsDisplay()
def to_json(self, filepath: str | None = None) -> dict[str, Any]:
data = {
"timestamp": self.timestamp.isoformat(),
"metadata": self.metadata,
"results": [
r.model_dump(exclude={"agent_evaluations"}) for r in self.results
],
"results": [r.model_dump(exclude={"agent_evaluations"}) for r in self.results]
}
if filepath:
with open(filepath, "w") as f:
with open(filepath, 'w') as f:
json.dump(data, f, indent=2)
self.display.console.print(f"[green]Results saved to {filepath}[/green]")
return data
def compare_with_baseline(
self,
baseline_filepath: str,
save_current: bool = True,
print_summary: bool = False,
) -> dict[str, Any]:
def compare_with_baseline(self, baseline_filepath: str, save_current: bool = True, print_summary: bool = False) -> dict[str, Any]:
baseline_runs = []
if os.path.exists(baseline_filepath) and os.path.getsize(baseline_filepath) > 0:
try:
with open(baseline_filepath, "r") as f:
with open(baseline_filepath, 'r') as f:
baseline_data = json.load(f)
if isinstance(baseline_data, dict) and "timestamp" in baseline_data:
@@ -63,18 +48,14 @@ class ExperimentResults:
elif isinstance(baseline_data, list):
baseline_runs = baseline_data
except (json.JSONDecodeError, FileNotFoundError) as e:
self.display.console.print(
f"[yellow]Warning: Could not load baseline file: {e!s}[/yellow]"
)
self.display.console.print(f"[yellow]Warning: Could not load baseline file: {str(e)}[/yellow]")
if not baseline_runs:
if save_current:
current_data = self.to_json()
with open(baseline_filepath, "w") as f:
with open(baseline_filepath, 'w') as f:
json.dump([current_data], f, indent=2)
self.display.console.print(
f"[green]Saved current results as new baseline to {baseline_filepath}[/green]"
)
self.display.console.print(f"[green]Saved current results as new baseline to {baseline_filepath}[/green]")
return {"is_baseline": True, "changes": {}}
baseline_runs.sort(key=lambda x: x.get("timestamp", ""), reverse=True)
@@ -88,11 +69,9 @@ class ExperimentResults:
if save_current:
current_data = self.to_json()
baseline_runs.append(current_data)
with open(baseline_filepath, "w") as f:
with open(baseline_filepath, 'w') as f:
json.dump(baseline_runs, f, indent=2)
self.display.console.print(
f"[green]Added current results to baseline file {baseline_filepath}[/green]"
)
self.display.console.print(f"[green]Added current results to baseline file {baseline_filepath}[/green]")
return comparison
@@ -139,5 +118,5 @@ class ExperimentResults:
"new_tests": new_tests,
"missing_tests": missing_tests,
"total_compared": len(improved) + len(regressed) + len(unchanged),
"baseline_timestamp": baseline_run.get("timestamp", "unknown"),
"baseline_timestamp": baseline_run.get("timestamp", "unknown")
}

View File

@@ -1,12 +1,9 @@
from typing import Any
from typing import Dict, Any
from rich.console import Console
from rich.panel import Panel
from rich.table import Table
from rich.panel import Panel
from crewai.experimental.evaluation.experiment.result import ExperimentResults
class ExperimentResultsDisplay:
def __init__(self):
self.console = Console()
@@ -22,19 +19,13 @@ class ExperimentResultsDisplay:
table.add_row("Total Test Cases", str(total))
table.add_row("Passed", str(passed))
table.add_row("Failed", str(total - passed))
table.add_row(
"Success Rate", f"{(passed / total * 100):.1f}%" if total > 0 else "N/A"
)
table.add_row("Success Rate", f"{(passed / total * 100):.1f}%" if total > 0 else "N/A")
self.console.print(table)
def comparison_summary(self, comparison: dict[str, Any], baseline_timestamp: str):
self.console.print(
Panel(
f"[bold]Comparison with baseline run from {baseline_timestamp}[/bold]",
expand=False,
)
)
def comparison_summary(self, comparison: Dict[str, Any], baseline_timestamp: str):
self.console.print(Panel(f"[bold]Comparison with baseline run from {baseline_timestamp}[/bold]",
expand=False))
table = Table(title="Results Comparison")
table.add_column("Metric", style="cyan")
@@ -43,9 +34,7 @@ class ExperimentResultsDisplay:
improved = comparison.get("improved", [])
if improved:
details = ", ".join(
[f"{test_identifier}" for test_identifier in improved[:3]]
)
details = ", ".join([f"{test_identifier}" for test_identifier in improved[:3]])
if len(improved) > 3:
details += f" and {len(improved) - 3} more"
table.add_row("✅ Improved", str(len(improved)), details)
@@ -54,9 +43,7 @@ class ExperimentResultsDisplay:
regressed = comparison.get("regressed", [])
if regressed:
details = ", ".join(
[f"{test_identifier}" for test_identifier in regressed[:3]]
)
details = ", ".join([f"{test_identifier}" for test_identifier in regressed[:3]])
if len(regressed) > 3:
details += f" and {len(regressed) - 3} more"
table.add_row("❌ Regressed", str(len(regressed)), details, style="red")
@@ -71,13 +58,13 @@ class ExperimentResultsDisplay:
details = ", ".join(new_tests[:3])
if len(new_tests) > 3:
details += f" and {len(new_tests) - 3} more"
table.add_row("+ New Tests", str(len(new_tests)), details)
table.add_row(" New Tests", str(len(new_tests)), details)
missing_tests = comparison.get("missing_tests", [])
if missing_tests:
details = ", ".join(missing_tests[:3])
if len(missing_tests) > 3:
details += f" and {len(missing_tests) - 3} more"
table.add_row("- Missing Tests", str(len(missing_tests)), details)
table.add_row(" Missing Tests", str(len(missing_tests)), details)
self.console.print(table)

View File

@@ -2,20 +2,11 @@ from collections import defaultdict
from hashlib import md5
from typing import Any
from crewai import Agent, Crew
from crewai.agents.agent_builder.base_agent import BaseAgent
from crewai import Crew, Agent
from crewai.experimental.evaluation import AgentEvaluator, create_default_evaluator
from crewai.experimental.evaluation.evaluation_display import (
AgentAggregatedEvaluationResult,
)
from crewai.experimental.evaluation.experiment.result import (
ExperimentResult,
ExperimentResults,
)
from crewai.experimental.evaluation.experiment.result_display import (
ExperimentResultsDisplay,
)
from crewai.experimental.evaluation.experiment.result_display import ExperimentResultsDisplay
from crewai.experimental.evaluation.experiment.result import ExperimentResults, ExperimentResult
from crewai.experimental.evaluation.evaluation_display import AgentAggregatedEvaluationResult
class ExperimentRunner:
def __init__(self, dataset: list[dict[str, Any]]):
@@ -23,17 +14,11 @@ class ExperimentRunner:
self.evaluator: AgentEvaluator | None = None
self.display = ExperimentResultsDisplay()
def run(
self,
crew: Crew | None = None,
agents: list[Agent] | list[BaseAgent] | None = None,
print_summary: bool = False,
) -> ExperimentResults:
def run(self, crew: Crew | None = None, agents: list[Agent] | None = None, print_summary: bool = False) -> ExperimentResults:
if crew and not agents:
agents = crew.agents
if agents is None:
raise ValueError("Agents must be provided either directly or via a crew")
assert agents is not None
self.evaluator = create_default_evaluator(agents=agents)
results = []
@@ -50,37 +35,21 @@ class ExperimentRunner:
return experiment_results
def _run_test_case(
self,
test_case: dict[str, Any],
agents: list[Agent] | list[BaseAgent],
crew: Crew | None = None,
) -> ExperimentResult:
def _run_test_case(self, test_case: dict[str, Any], agents: list[Agent], crew: Crew | None = None) -> ExperimentResult:
inputs = test_case["inputs"]
expected_score = test_case["expected_score"]
identifier = (
test_case.get("identifier")
or md5(str(test_case).encode(), usedforsecurity=False).hexdigest()
)
identifier = test_case.get("identifier") or md5(str(test_case).encode(), usedforsecurity=False).hexdigest()
try:
self.display.console.print(
f"[dim]Running crew with input: {str(inputs)[:50]}...[/dim]"
)
self.display.console.print(f"[dim]Running crew with input: {str(inputs)[:50]}...[/dim]")
self.display.console.print("\n")
if crew:
crew.kickoff(inputs=inputs)
else:
for agent in agents:
if isinstance(agent, Agent):
agent.kickoff(**inputs)
else:
raise TypeError(
f"Agent {agent} is not an instance of Agent and cannot be kicked off directly"
)
agent.kickoff(**inputs)
if self.evaluator is None:
raise ValueError("Evaluator must be initialized")
assert self.evaluator is not None
agent_evaluations = self.evaluator.get_agent_evaluation()
actual_score = self._extract_scores(agent_evaluations)
@@ -92,38 +61,35 @@ class ExperimentRunner:
score=actual_score,
expected_score=expected_score,
passed=passed,
agent_evaluations=agent_evaluations,
agent_evaluations=agent_evaluations
)
except Exception as e:
self.display.console.print(f"[red]Error running test case: {e!s}[/red]")
self.display.console.print(f"[red]Error running test case: {str(e)}[/red]")
return ExperimentResult(
identifier=identifier,
inputs=inputs,
score=0.0,
score=0,
expected_score=expected_score,
passed=False,
passed=False
)
def _extract_scores(
self, agent_evaluations: dict[str, AgentAggregatedEvaluationResult]
) -> float | dict[str, float]:
def _extract_scores(self, agent_evaluations: dict[str, AgentAggregatedEvaluationResult]) -> float | dict[str, float]:
all_scores: dict[str, list[float]] = defaultdict(list)
for evaluation in agent_evaluations.values():
for metric_name, score in evaluation.metrics.items():
if score.score is not None:
all_scores[metric_name.value].append(score.score)
avg_scores = {m: sum(s) / len(s) for m, s in all_scores.items()}
avg_scores = {m: sum(s)/len(s) for m, s in all_scores.items()}
if len(avg_scores) == 1:
return next(iter(avg_scores.values()))
return list(avg_scores.values())[0]
return avg_scores
def _assert_scores(
self, expected: float | dict[str, float], actual: float | dict[str, float]
) -> bool:
def _assert_scores(self, expected: float | dict[str, float],
actual: float | dict[str, float]) -> bool:
"""
Compare expected and actual scores, and return whether the test case passed.
@@ -156,4 +122,4 @@ class ExperimentRunner:
# All matching keys must have actual >= expected
return all(actual[key] >= expected[key] for key in matching_keys)
return False
return False

View File

@@ -13,11 +13,11 @@ def extract_json_from_llm_response(text: str) -> dict[str, Any]:
json_patterns = [
# Standard markdown code blocks with json
r"```json\s*([\s\S]*?)\s*```",
r'```json\s*([\s\S]*?)\s*```',
# Code blocks without language specifier
r"```\s*([\s\S]*?)\s*```",
r'```\s*([\s\S]*?)\s*```',
# Inline code with JSON
r"`([{\\[].*[}\]])`",
r'`([{\\[].*[}\]])`',
]
for pattern in json_patterns:
@@ -25,6 +25,6 @@ def extract_json_from_llm_response(text: str) -> dict[str, Any]:
for match in matches:
try:
return json.loads(match.strip())
except json.JSONDecodeError: # noqa: PERF203
except json.JSONDecodeError:
continue
raise ValueError("No valid JSON found in the response")

View File

@@ -1,21 +1,26 @@
from crewai.experimental.evaluation.metrics.goal_metrics import GoalAlignmentEvaluator
from crewai.experimental.evaluation.metrics.reasoning_metrics import (
ReasoningEfficiencyEvaluator,
)
from crewai.experimental.evaluation.metrics.semantic_quality_metrics import (
SemanticQualityEvaluator,
ReasoningEfficiencyEvaluator
)
from crewai.experimental.evaluation.metrics.tools_metrics import (
ParameterExtractionEvaluator,
ToolInvocationEvaluator,
ToolSelectionEvaluator,
ParameterExtractionEvaluator,
ToolInvocationEvaluator
)
from crewai.experimental.evaluation.metrics.goal_metrics import (
GoalAlignmentEvaluator
)
from crewai.experimental.evaluation.metrics.semantic_quality_metrics import (
SemanticQualityEvaluator
)
__all__ = [
"GoalAlignmentEvaluator",
"ParameterExtractionEvaluator",
"ReasoningEfficiencyEvaluator",
"SemanticQualityEvaluator",
"ToolInvocationEvaluator",
"ToolSelectionEvaluator",
]
"ParameterExtractionEvaluator",
"ToolInvocationEvaluator",
"GoalAlignmentEvaluator",
"SemanticQualityEvaluator"
]

View File

@@ -1,15 +1,10 @@
from typing import Any
from typing import Any, Dict
from crewai.agent import Agent
from crewai.agents.agent_builder.base_agent import BaseAgent
from crewai.experimental.evaluation.base_evaluator import (
BaseEvaluator,
EvaluationScore,
MetricCategory,
)
from crewai.experimental.evaluation.json_parser import extract_json_from_llm_response
from crewai.task import Task
from crewai.experimental.evaluation.base_evaluator import BaseEvaluator, EvaluationScore, MetricCategory
from crewai.experimental.evaluation.json_parser import extract_json_from_llm_response
class GoalAlignmentEvaluator(BaseEvaluator):
@property
@@ -18,8 +13,8 @@ class GoalAlignmentEvaluator(BaseEvaluator):
def evaluate(
self,
agent: Agent | BaseAgent,
execution_trace: dict[str, Any],
agent: Agent,
execution_trace: Dict[str, Any],
final_output: Any,
task: Task | None = None,
) -> EvaluationScore:
@@ -28,9 +23,7 @@ class GoalAlignmentEvaluator(BaseEvaluator):
task_context = f"Task description: {task.description}\nExpected output: {task.expected_output}\n"
prompt = [
{
"role": "system",
"content": """You are an expert evaluator assessing how well an AI agent's output aligns with its assigned task goal.
{"role": "system", "content": """You are an expert evaluator assessing how well an AI agent's output aligns with its assigned task goal.
Score the agent's goal alignment on a scale from 0-10 where:
- 0: Complete misalignment, agent did not understand or attempt the task goal
@@ -44,11 +37,8 @@ Consider:
4. Did the agent provide all requested information or deliverables?
Return your evaluation as JSON with fields 'score' (number) and 'feedback' (string).
""",
},
{
"role": "user",
"content": f"""
"""},
{"role": "user", "content": f"""
Agent role: {agent.role}
Agent goal: {agent.goal}
{task_context}
@@ -57,26 +47,23 @@ Agent's final output:
{final_output}
Evaluate how well the agent's output aligns with the assigned task goal.
""",
},
"""}
]
if self.llm is None:
raise ValueError("LLM must be initialized")
assert self.llm is not None
response = self.llm.call(prompt)
try:
evaluation_data: dict[str, Any] = extract_json_from_llm_response(response)
if evaluation_data is None:
raise ValueError("Failed to extract evaluation data from LLM response")
assert evaluation_data is not None
return EvaluationScore(
score=evaluation_data.get("score", 0),
feedback=evaluation_data.get("feedback", response),
raw_response=response,
raw_response=response
)
except Exception:
return EvaluationScore(
score=None,
feedback=f"Failed to parse evaluation. Raw response: {response}",
raw_response=response,
raw_response=response
)

View File

@@ -8,23 +8,17 @@ This module provides evaluator implementations for:
import logging
import re
from collections.abc import Sequence
from enum import Enum
from typing import Any
from typing import Any, Dict, List, Tuple
import numpy as np
from collections.abc import Sequence
from crewai.agent import Agent
from crewai.agents.agent_builder.base_agent import BaseAgent
from crewai.experimental.evaluation.base_evaluator import (
BaseEvaluator,
EvaluationScore,
MetricCategory,
)
from crewai.experimental.evaluation.json_parser import extract_json_from_llm_response
from crewai.task import Task
from crewai.tasks.task_output import TaskOutput
from crewai.experimental.evaluation.base_evaluator import BaseEvaluator, EvaluationScore, MetricCategory
from crewai.experimental.evaluation.json_parser import extract_json_from_llm_response
from crewai.tasks.task_output import TaskOutput
class ReasoningPatternType(Enum):
EFFICIENT = "efficient" # Good reasoning flow
@@ -41,8 +35,8 @@ class ReasoningEfficiencyEvaluator(BaseEvaluator):
def evaluate(
self,
agent: Agent | BaseAgent,
execution_trace: dict[str, Any],
agent: Agent,
execution_trace: Dict[str, Any],
final_output: TaskOutput | str,
task: Task | None = None,
) -> EvaluationScore:
@@ -55,7 +49,7 @@ class ReasoningEfficiencyEvaluator(BaseEvaluator):
if not llm_calls or len(llm_calls) < 2:
return EvaluationScore(
score=None,
feedback="Insufficient LLM calls to evaluate reasoning efficiency.",
feedback="Insufficient LLM calls to evaluate reasoning efficiency."
)
total_calls = len(llm_calls)
@@ -64,16 +58,12 @@ class ReasoningEfficiencyEvaluator(BaseEvaluator):
time_intervals = []
has_reliable_timing = True
for i in range(1, len(llm_calls)):
start_time = llm_calls[i - 1].get("end_time")
start_time = llm_calls[i-1].get("end_time")
end_time = llm_calls[i].get("start_time")
if start_time and end_time and start_time != end_time:
try:
interval = end_time - start_time
time_intervals.append(
interval.total_seconds()
if hasattr(interval, "total_seconds")
else 0
)
time_intervals.append(interval.total_seconds() if hasattr(interval, 'total_seconds') else 0)
except Exception:
has_reliable_timing = False
else:
@@ -93,22 +83,14 @@ class ReasoningEfficiencyEvaluator(BaseEvaluator):
if has_reliable_timing and time_intervals:
efficiency_metrics["avg_time_between_calls"] = np.mean(time_intervals)
loop_info = (
f"Detected {len(loop_details)} potential reasoning loops."
if loop_detected
else "No significant reasoning loops detected."
)
loop_info = f"Detected {len(loop_details)} potential reasoning loops." if loop_detected else "No significant reasoning loops detected."
call_samples = self._get_call_samples(llm_calls)
final_output = (
final_output.raw if isinstance(final_output, TaskOutput) else final_output
)
final_output = final_output.raw if isinstance(final_output, TaskOutput) else final_output
prompt = [
{
"role": "system",
"content": """You are an expert evaluator assessing the reasoning efficiency of an AI agent's thought process.
{"role": "system", "content": """You are an expert evaluator assessing the reasoning efficiency of an AI agent's thought process.
Evaluate the agent's reasoning efficiency across these five key subcategories:
@@ -138,11 +120,8 @@ Return your evaluation as JSON with the following structure:
"feedback": string (general feedback about overall reasoning efficiency),
"optimization_suggestions": string (concrete suggestions for improving reasoning efficiency),
"detected_patterns": string (describe any inefficient reasoning patterns you observe)
}""",
},
{
"role": "user",
"content": f"""
}"""},
{"role": "user", "content": f"""
Agent role: {agent.role}
{task_context}
@@ -161,12 +140,10 @@ Agent's final output:
Evaluate the reasoning efficiency of this agent based on these interaction patterns.
Identify any inefficient reasoning patterns and provide specific suggestions for optimization.
""",
},
"""}
]
if self.llm is None:
raise ValueError("LLM must be initialized")
assert self.llm is not None
response = self.llm.call(prompt)
try:
@@ -179,46 +156,34 @@ Identify any inefficient reasoning patterns and provide specific suggestions for
conciseness = scores.get("conciseness", 5.0)
loop_avoidance = scores.get("loop_avoidance", 5.0)
overall_score = evaluation_data.get(
"overall_score", evaluation_data.get("score", 5.0)
)
overall_score = evaluation_data.get("overall_score", evaluation_data.get("score", 5.0))
feedback = evaluation_data.get("feedback", "No detailed feedback provided.")
optimization_suggestions = evaluation_data.get(
"optimization_suggestions", "No specific suggestions provided."
)
optimization_suggestions = evaluation_data.get("optimization_suggestions", "No specific suggestions provided.")
detailed_feedback = "Reasoning Efficiency Evaluation:\n"
detailed_feedback += (
f"• Focus: {focus}/10 - Staying on topic without tangents\n"
)
detailed_feedback += (
f"• Progression: {progression}/10 - Building on previous thinking\n"
)
detailed_feedback += f"• Focus: {focus}/10 - Staying on topic without tangents\n"
detailed_feedback += f"• Progression: {progression}/10 - Building on previous thinking\n"
detailed_feedback += f"• Decision Quality: {decision_quality}/10 - Making appropriate decisions\n"
detailed_feedback += (
f"• Conciseness: {conciseness}/10 - Communicating efficiently\n"
)
detailed_feedback += f"• Conciseness: {conciseness}/10 - Communicating efficiently\n"
detailed_feedback += f"• Loop Avoidance: {loop_avoidance}/10 - Avoiding repetitive patterns\n\n"
detailed_feedback += f"Feedback:\n{feedback}\n\n"
detailed_feedback += (
f"Optimization Suggestions:\n{optimization_suggestions}"
)
detailed_feedback += f"Optimization Suggestions:\n{optimization_suggestions}"
return EvaluationScore(
score=float(overall_score),
feedback=detailed_feedback,
raw_response=response,
raw_response=response
)
except Exception as e:
logging.warning(f"Failed to parse reasoning efficiency evaluation: {e}")
return EvaluationScore(
score=None,
feedback=f"Failed to parse reasoning efficiency evaluation. Raw response: {response[:200]}...",
raw_response=response,
raw_response=response
)
def _detect_loops(self, llm_calls: list[dict]) -> tuple[bool, list[dict]]:
def _detect_loops(self, llm_calls: List[Dict]) -> Tuple[bool, List[Dict]]:
loop_details = []
messages = []
@@ -228,11 +193,9 @@ Identify any inefficient reasoning patterns and provide specific suggestions for
messages.append(content)
elif isinstance(content, list) and len(content) > 0:
# Handle message list format
messages.extend(
msg["content"]
for msg in content
if isinstance(msg, dict) and "content" in msg
)
for msg in content:
if isinstance(msg, dict) and "content" in msg:
messages.append(msg["content"])
# Simple n-gram based similarity detection
# For a more robust implementation, consider using embedding-based similarity
@@ -242,20 +205,18 @@ Identify any inefficient reasoning patterns and provide specific suggestions for
# A more sophisticated approach would use semantic similarity
similarity = self._calculate_text_similarity(messages[i], messages[j])
if similarity > 0.7: # Arbitrary threshold
loop_details.append(
{
"first_occurrence": i,
"second_occurrence": j,
"similarity": similarity,
"snippet": messages[i][:100] + "...",
}
)
loop_details.append({
"first_occurrence": i,
"second_occurrence": j,
"similarity": similarity,
"snippet": messages[i][:100] + "..."
})
return len(loop_details) > 0, loop_details
def _calculate_text_similarity(self, text1: str, text2: str) -> float:
text1 = re.sub(r"\s+", " ", text1.lower()).strip()
text2 = re.sub(r"\s+", " ", text2.lower()).strip()
text1 = re.sub(r'\s+', ' ', text1.lower()).strip()
text2 = re.sub(r'\s+', ' ', text2.lower()).strip()
# Simple Jaccard similarity on word sets
words1 = set(text1.split())
@@ -266,7 +227,7 @@ Identify any inefficient reasoning patterns and provide specific suggestions for
return intersection / union if union > 0 else 0.0
def _analyze_reasoning_patterns(self, llm_calls: list[dict]) -> dict[str, Any]:
def _analyze_reasoning_patterns(self, llm_calls: List[Dict]) -> Dict[str, Any]:
call_lengths = []
response_times = []
@@ -287,8 +248,8 @@ Identify any inefficient reasoning patterns and provide specific suggestions for
if start_time and end_time:
try:
response_times.append(end_time - start_time)
except Exception as e:
logging.debug(f"Failed to calculate response time: {e}")
except Exception:
pass
avg_length = np.mean(call_lengths) if call_lengths else 0
std_length = np.std(call_lengths) if call_lengths else 0
@@ -306,9 +267,7 @@ Identify any inefficient reasoning patterns and provide specific suggestions for
details = "Agent is consistently verbose across interactions."
elif len(llm_calls) > 10 and length_trend > 0.5:
primary_pattern = ReasoningPatternType.INDECISIVE
details = (
"Agent shows signs of indecisiveness with increasing message lengths."
)
details = "Agent shows signs of indecisiveness with increasing message lengths."
elif std_length / avg_length > 0.8:
primary_pattern = ReasoningPatternType.SCATTERED
details = "Agent shows inconsistent reasoning flow with highly variable responses."
@@ -320,8 +279,8 @@ Identify any inefficient reasoning patterns and provide specific suggestions for
"avg_length": avg_length,
"std_length": std_length,
"length_trend": length_trend,
"loop_score": loop_score,
},
"loop_score": loop_score
}
}
def _calculate_trend(self, values: Sequence[float | int]) -> float:
@@ -344,9 +303,7 @@ Identify any inefficient reasoning patterns and provide specific suggestions for
except Exception:
return 0.0
def _calculate_loop_likelihood(
self, call_lengths: Sequence[float], response_times: Sequence[float]
) -> float:
def _calculate_loop_likelihood(self, call_lengths: Sequence[float], response_times: Sequence[float]) -> float:
if not call_lengths or len(call_lengths) < 3:
return 0.0
@@ -355,11 +312,7 @@ Identify any inefficient reasoning patterns and provide specific suggestions for
if len(call_lengths) >= 4:
repeated_lengths = 0
for i in range(len(call_lengths) - 2):
ratio = (
call_lengths[i] / call_lengths[i + 2]
if call_lengths[i + 2] > 0
else 0
)
ratio = call_lengths[i] / call_lengths[i + 2] if call_lengths[i + 2] > 0 else 0
if 0.85 <= ratio <= 1.15:
repeated_lengths += 1
@@ -371,27 +324,21 @@ Identify any inefficient reasoning patterns and provide specific suggestions for
std_time = np.std(response_times)
mean_time = np.mean(response_times)
if mean_time > 0:
time_consistency = 1.0 - (float(std_time) / float(mean_time))
indicators.append(max(0.0, float(time_consistency - 0.3)) * 1.5)
except Exception as e:
logging.debug(f"Time consistency calculation failed: {e}")
time_consistency = 1.0 - (std_time / mean_time)
indicators.append(max(0, time_consistency - 0.3) * 1.5)
except Exception:
pass
return float(np.mean(indicators)) if indicators else 0.0
return np.mean(indicators) if indicators else 0.0
def _get_call_samples(self, llm_calls: list[dict]) -> str:
def _get_call_samples(self, llm_calls: List[Dict]) -> str:
samples = []
if len(llm_calls) <= 6:
sample_indices = list(range(len(llm_calls)))
else:
sample_indices = [
0,
1,
len(llm_calls) // 2 - 1,
len(llm_calls) // 2,
len(llm_calls) - 2,
len(llm_calls) - 1,
]
sample_indices = [0, 1, len(llm_calls) // 2 - 1, len(llm_calls) // 2,
len(llm_calls) - 2, len(llm_calls) - 1]
for idx in sample_indices:
call = llm_calls[idx]
@@ -400,11 +347,10 @@ Identify any inefficient reasoning patterns and provide specific suggestions for
if isinstance(content, str):
sample = content
elif isinstance(content, list) and len(content) > 0:
sample_parts = [
msg["content"]
for msg in content
if isinstance(msg, dict) and "content" in msg
]
sample_parts = []
for msg in content:
if isinstance(msg, dict) and "content" in msg:
sample_parts.append(msg["content"])
sample = "\n".join(sample_parts)
else:
sample = str(content)

View File

@@ -1,15 +1,10 @@
from typing import Any
from typing import Any, Dict
from crewai.agent import Agent
from crewai.agents.agent_builder.base_agent import BaseAgent
from crewai.experimental.evaluation.base_evaluator import (
BaseEvaluator,
EvaluationScore,
MetricCategory,
)
from crewai.experimental.evaluation.json_parser import extract_json_from_llm_response
from crewai.task import Task
from crewai.experimental.evaluation.base_evaluator import BaseEvaluator, EvaluationScore, MetricCategory
from crewai.experimental.evaluation.json_parser import extract_json_from_llm_response
class SemanticQualityEvaluator(BaseEvaluator):
@property
@@ -18,8 +13,8 @@ class SemanticQualityEvaluator(BaseEvaluator):
def evaluate(
self,
agent: Agent | BaseAgent,
execution_trace: dict[str, Any],
agent: Agent,
execution_trace: Dict[str, Any],
final_output: Any,
task: Task | None = None,
) -> EvaluationScore:
@@ -27,9 +22,7 @@ class SemanticQualityEvaluator(BaseEvaluator):
if task is not None:
task_context = f"Task description: {task.description}"
prompt = [
{
"role": "system",
"content": """You are an expert evaluator assessing the semantic quality of an AI agent's output.
{"role": "system", "content": """You are an expert evaluator assessing the semantic quality of an AI agent's output.
Score the semantic quality on a scale from 0-10 where:
- 0: Completely incoherent, confusing, or logically flawed output
@@ -44,11 +37,8 @@ Consider:
5. Is the output free from contradictions and logical fallacies?
Return your evaluation as JSON with fields 'score' (number) and 'feedback' (string).
""",
},
{
"role": "user",
"content": f"""
"""},
{"role": "user", "content": f"""
Agent role: {agent.role}
{task_context}
@@ -56,28 +46,23 @@ Agent's final output:
{final_output}
Evaluate the semantic quality and reasoning of this output.
""",
},
"""}
]
if self.llm is None:
raise ValueError("LLM must be initialized")
assert self.llm is not None
response = self.llm.call(prompt)
try:
evaluation_data: dict[str, Any] = extract_json_from_llm_response(response)
if evaluation_data is None:
raise ValueError("Failed to extract evaluation data from LLM response")
assert evaluation_data is not None
return EvaluationScore(
score=float(evaluation_data["score"])
if evaluation_data.get("score") is not None
else None,
score=float(evaluation_data["score"]) if evaluation_data.get("score") is not None else None,
feedback=evaluation_data.get("feedback", response),
raw_response=response,
raw_response=response
)
except Exception:
return EvaluationScore(
score=None,
feedback=f"Failed to parse evaluation. Raw response: {response}",
raw_response=response,
)
raw_response=response
)

View File

@@ -1,26 +1,22 @@
import json
from typing import Any
from typing import Dict, Any
from crewai.agent import Agent
from crewai.agents.agent_builder.base_agent import BaseAgent
from crewai.experimental.evaluation.base_evaluator import (
BaseEvaluator,
EvaluationScore,
MetricCategory,
)
from crewai.experimental.evaluation.base_evaluator import BaseEvaluator, EvaluationScore, MetricCategory
from crewai.experimental.evaluation.json_parser import extract_json_from_llm_response
from crewai.agent import Agent
from crewai.task import Task
class ToolSelectionEvaluator(BaseEvaluator):
@property
def metric_category(self) -> MetricCategory:
return MetricCategory.TOOL_SELECTION
def evaluate(
self,
agent: Agent | BaseAgent,
execution_trace: dict[str, Any],
agent: Agent,
execution_trace: Dict[str, Any],
final_output: str,
task: Task | None = None,
) -> EvaluationScore:
@@ -30,18 +26,19 @@ class ToolSelectionEvaluator(BaseEvaluator):
tool_uses = execution_trace.get("tool_uses", [])
tool_count = len(tool_uses)
unique_tool_types = set(
[tool.get("tool", "Unknown tool") for tool in tool_uses]
)
unique_tool_types = set([tool.get("tool", "Unknown tool") for tool in tool_uses])
if tool_count == 0:
if not agent.tools:
return EvaluationScore(
score=None, feedback="Agent had no tools available to use."
score=None,
feedback="Agent had no tools available to use."
)
else:
return EvaluationScore(
score=None,
feedback="Agent had tools available but didn't use any."
)
return EvaluationScore(
score=None, feedback="Agent had tools available but didn't use any."
)
available_tools_info = ""
if agent.tools:
@@ -55,9 +52,7 @@ class ToolSelectionEvaluator(BaseEvaluator):
tool_types_summary += f"- {tool_type}\n"
prompt = [
{
"role": "system",
"content": """You are an expert evaluator assessing if an AI agent selected the most appropriate tools for a given task.
{"role": "system", "content": """You are an expert evaluator assessing if an AI agent selected the most appropriate tools for a given task.
You must evaluate based on these 2 criteria:
1. Relevance (0-10): Were the tools chosen directly aligned with the task's goals?
@@ -78,11 +73,8 @@ Return your evaluation as JSON with these fields:
- overall_score: number (average of all scores, 0-10)
- feedback: string (focused ONLY on tool selection decisions from available tools)
- improvement_suggestions: string (ONLY suggest better selection from the AVAILABLE tools list, NOT new tools)
""",
},
{
"role": "user",
"content": f"""
"""},
{"role": "user", "content": f"""
Agent role: {agent.role}
{task_context}
@@ -97,17 +89,14 @@ IMPORTANT:
- ONLY evaluate selection from tools listed as available
- DO NOT suggest new tools that aren't in the available tools list
- DO NOT evaluate tool usage or results
""",
},
"""}
]
if self.llm is None:
raise ValueError("LLM must be initialized")
assert self.llm is not None
response = self.llm.call(prompt)
try:
evaluation_data = extract_json_from_llm_response(response)
if evaluation_data is None:
raise ValueError("Failed to extract evaluation data from LLM response")
assert evaluation_data is not None
scores = evaluation_data.get("scores", {})
relevance = scores.get("relevance", 5.0)
@@ -116,24 +105,22 @@ IMPORTANT:
feedback = "Tool Selection Evaluation:\n"
feedback += f"• Relevance: {relevance}/10 - Selection of appropriate tool types for the task\n"
feedback += (
f"• Coverage: {coverage}/10 - Selection of all necessary tool types\n"
)
feedback += f"• Coverage: {coverage}/10 - Selection of all necessary tool types\n"
if "improvement_suggestions" in evaluation_data:
feedback += f"Improvement Suggestions:\n{evaluation_data['improvement_suggestions']}"
else:
feedback += evaluation_data.get(
"feedback", "No detailed feedback available."
)
feedback += evaluation_data.get("feedback", "No detailed feedback available.")
return EvaluationScore(
score=overall_score, feedback=feedback, raw_response=response
score=overall_score,
feedback=feedback,
raw_response=response
)
except Exception as e:
return EvaluationScore(
score=None,
feedback=f"Error evaluating tool selection: {e}",
raw_response=response,
raw_response=response
)
@@ -144,8 +131,8 @@ class ParameterExtractionEvaluator(BaseEvaluator):
def evaluate(
self,
agent: Agent | BaseAgent,
execution_trace: dict[str, Any],
agent: Agent,
execution_trace: Dict[str, Any],
final_output: str,
task: Task | None = None,
) -> EvaluationScore:
@@ -158,23 +145,19 @@ class ParameterExtractionEvaluator(BaseEvaluator):
if tool_count == 0:
return EvaluationScore(
score=None,
feedback="No tool usage detected. Cannot evaluate parameter extraction.",
feedback="No tool usage detected. Cannot evaluate parameter extraction."
)
validation_errors = [
{
"tool": tool_use.get("tool", "Unknown tool"),
"error": tool_use.get("result"),
"args": tool_use.get("args", {}),
}
for tool_use in tool_uses
if not tool_use.get("success", True)
and tool_use.get("error_type") == "validation_error"
]
validation_errors = []
for tool_use in tool_uses:
if not tool_use.get("success", True) and tool_use.get("error_type") == "validation_error":
validation_errors.append({
"tool": tool_use.get("tool", "Unknown tool"),
"error": tool_use.get("result"),
"args": tool_use.get("args", {})
})
validation_error_rate = (
len(validation_errors) / tool_count if tool_count > 0 else 0
)
validation_error_rate = len(validation_errors) / tool_count if tool_count > 0 else 0
param_samples = []
for i, tool_use in enumerate(tool_uses[:5]):
@@ -185,7 +168,7 @@ class ParameterExtractionEvaluator(BaseEvaluator):
is_validation_error = error_type == "validation_error"
sample = f"Tool use #{i + 1} - {tool_name}:\n"
sample = f"Tool use #{i+1} - {tool_name}:\n"
sample += f"- Parameters: {json.dumps(tool_args, indent=2)}\n"
sample += f"- Success: {'No' if not success else 'Yes'}"
@@ -204,17 +187,13 @@ class ParameterExtractionEvaluator(BaseEvaluator):
tool_name = err.get("tool", "Unknown tool")
error_msg = err.get("error", "Unknown error")
args = err.get("args", {})
validation_errors_info += f"\nValidation Error #{i + 1}:\n- Tool: {tool_name}\n- Args: {json.dumps(args, indent=2)}\n- Error: {error_msg}"
validation_errors_info += f"\nValidation Error #{i+1}:\n- Tool: {tool_name}\n- Args: {json.dumps(args, indent=2)}\n- Error: {error_msg}"
if len(validation_errors) > 3:
validation_errors_info += (
f"\n...and {len(validation_errors) - 3} more validation errors."
)
validation_errors_info += f"\n...and {len(validation_errors) - 3} more validation errors."
param_samples_text = "\n\n".join(param_samples)
prompt = [
{
"role": "system",
"content": """You are an expert evaluator assessing how well an AI agent extracts and formats PARAMETER VALUES for tool calls.
{"role": "system", "content": """You are an expert evaluator assessing how well an AI agent extracts and formats PARAMETER VALUES for tool calls.
Your job is to evaluate ONLY whether the agent used the correct parameter VALUES, not whether the right tools were selected or how the tools were invoked.
@@ -237,11 +216,8 @@ Return your evaluation as JSON with these fields:
- overall_score: number (average of all scores, 0-10)
- feedback: string (focused ONLY on parameter value extraction quality)
- improvement_suggestions: string (concrete suggestions for better parameter VALUE extraction)
""",
},
{
"role": "user",
"content": f"""
"""},
{"role": "user", "content": f"""
Agent role: {agent.role}
{task_context}
@@ -250,18 +226,15 @@ Parameter extraction examples:
{validation_errors_info}
Evaluate the quality of the agent's parameter extraction for this task.
""",
},
"""}
]
if self.llm is None:
raise ValueError("LLM must be initialized")
assert self.llm is not None
response = self.llm.call(prompt)
try:
evaluation_data = extract_json_from_llm_response(response)
if evaluation_data is None:
raise ValueError("Failed to extract evaluation data from LLM response")
assert evaluation_data is not None
scores = evaluation_data.get("scores", {})
accuracy = scores.get("accuracy", 5.0)
@@ -278,18 +251,18 @@ Evaluate the quality of the agent's parameter extraction for this task.
if "improvement_suggestions" in evaluation_data:
feedback += f"Improvement Suggestions:\n{evaluation_data['improvement_suggestions']}"
else:
feedback += evaluation_data.get(
"feedback", "No detailed feedback available."
)
feedback += evaluation_data.get("feedback", "No detailed feedback available.")
return EvaluationScore(
score=overall_score, feedback=feedback, raw_response=response
score=overall_score,
feedback=feedback,
raw_response=response
)
except Exception as e:
return EvaluationScore(
score=None,
feedback=f"Error evaluating parameter extraction: {e}",
raw_response=response,
raw_response=response
)
@@ -300,8 +273,8 @@ class ToolInvocationEvaluator(BaseEvaluator):
def evaluate(
self,
agent: Agent | BaseAgent,
execution_trace: dict[str, Any],
agent: Agent,
execution_trace: Dict[str, Any],
final_output: str,
task: Task | None = None,
) -> EvaluationScore:
@@ -315,7 +288,7 @@ class ToolInvocationEvaluator(BaseEvaluator):
if tool_count == 0:
return EvaluationScore(
score=None,
feedback="No tool usage detected. Cannot evaluate tool invocation.",
feedback="No tool usage detected. Cannot evaluate tool invocation."
)
for tool_use in tool_uses:
@@ -323,7 +296,7 @@ class ToolInvocationEvaluator(BaseEvaluator):
error_info = {
"tool": tool_use.get("tool", "Unknown tool"),
"error": tool_use.get("result"),
"error_type": tool_use.get("error_type", "unknown_error"),
"error_type": tool_use.get("error_type", "unknown_error")
}
tool_errors.append(error_info)
@@ -342,11 +315,9 @@ class ToolInvocationEvaluator(BaseEvaluator):
tool_args = tool_use.get("args", {})
success = tool_use.get("success", True) and not tool_use.get("error", False)
error_type = tool_use.get("error_type", "") if not success else ""
error_msg = (
tool_use.get("result", "No error") if not success else "No error"
)
error_msg = tool_use.get("result", "No error") if not success else "No error"
sample = f"Tool invocation #{i + 1}:\n"
sample = f"Tool invocation #{i+1}:\n"
sample += f"- Tool: {tool_name}\n"
sample += f"- Parameters: {json.dumps(tool_args, indent=2)}\n"
sample += f"- Success: {'No' if not success else 'Yes'}\n"
@@ -359,13 +330,11 @@ class ToolInvocationEvaluator(BaseEvaluator):
if error_types:
error_type_summary = "Error type breakdown:\n"
for error_type, count in error_types.items():
error_type_summary += f"- {error_type}: {count} occurrences ({(count / tool_count):.1%})\n"
error_type_summary += f"- {error_type}: {count} occurrences ({(count/tool_count):.1%})\n"
invocation_samples_text = "\n\n".join(invocation_samples)
prompt = [
{
"role": "system",
"content": """You are an expert evaluator assessing how correctly an AI agent's tool invocations are STRUCTURED.
{"role": "system", "content": """You are an expert evaluator assessing how correctly an AI agent's tool invocations are STRUCTURED.
Your job is to evaluate ONLY the structural and syntactical aspects of how the agent called tools, NOT which tools were selected or what parameter values were used.
@@ -390,11 +359,8 @@ Return your evaluation as JSON with these fields:
- overall_score: number (average of all scores, 0-10)
- feedback: string (focused ONLY on structural aspects of tool invocation)
- improvement_suggestions: string (concrete suggestions for better structuring of tool calls)
""",
},
{
"role": "user",
"content": f"""
"""},
{"role": "user", "content": f"""
Agent role: {agent.role}
{task_context}
@@ -405,18 +371,15 @@ Tool error rate: {error_rate:.2%} ({len(tool_errors)} errors out of {tool_count}
{error_type_summary}
Evaluate the quality of the agent's tool invocation structure during this task.
""",
},
"""}
]
if self.llm is None:
raise ValueError("LLM must be initialized")
assert self.llm is not None
response = self.llm.call(prompt)
try:
evaluation_data = extract_json_from_llm_response(response)
if evaluation_data is None:
raise ValueError("Failed to extract evaluation data from LLM response")
assert evaluation_data is not None
scores = evaluation_data.get("scores", {})
structure = scores.get("structure", 5.0)
error_handling = scores.get("error_handling", 5.0)
@@ -425,25 +388,23 @@ Evaluate the quality of the agent's tool invocation structure during this task.
overall_score = float(evaluation_data.get("overall_score", 5.0))
feedback = "Tool Invocation Evaluation:\n"
feedback += (
f"• Structure: {structure}/10 - Following proper syntax and format\n"
)
feedback += f"• Structure: {structure}/10 - Following proper syntax and format\n"
feedback += f"• Error Handling: {error_handling}/10 - Appropriately handling tool errors\n"
feedback += f"• Invocation Patterns: {invocation_patterns}/10 - Proper sequencing and management of calls\n\n"
if "improvement_suggestions" in evaluation_data:
feedback += f"Improvement Suggestions:\n{evaluation_data['improvement_suggestions']}"
else:
feedback += evaluation_data.get(
"feedback", "No detailed feedback available."
)
feedback += evaluation_data.get("feedback", "No detailed feedback available.")
return EvaluationScore(
score=overall_score, feedback=feedback, raw_response=response
score=overall_score,
feedback=feedback,
raw_response=response
)
except Exception as e:
return EvaluationScore(
score=None,
feedback=f"Error evaluating tool invocation: {e}",
raw_response=response,
raw_response=response
)

View File

@@ -1,21 +1,12 @@
import inspect
import warnings
from typing_extensions import Any
import warnings
from crewai.experimental.evaluation.experiment import ExperimentResults, ExperimentRunner
from crewai import Crew, Agent
from crewai import Agent, Crew
from crewai.experimental.evaluation.experiment import (
ExperimentResults,
ExperimentRunner,
)
def assert_experiment_successfully(
experiment_results: ExperimentResults, baseline_filepath: str | None = None
) -> None:
failed_tests = [
result for result in experiment_results.results if not result.passed
]
def assert_experiment_successfully(experiment_results: ExperimentResults, baseline_filepath: str | None = None) -> None:
failed_tests = [result for result in experiment_results.results if not result.passed]
if failed_tests:
detailed_failures: list[str] = []
@@ -23,54 +14,39 @@ def assert_experiment_successfully(
for result in failed_tests:
expected = result.expected_score
actual = result.score
detailed_failures.append(
f"- {result.identifier}: expected {expected}, got {actual}"
)
detailed_failures.append(f"- {result.identifier}: expected {expected}, got {actual}")
failure_details = "\n".join(detailed_failures)
raise AssertionError(f"The following test cases failed:\n{failure_details}")
baseline_filepath = baseline_filepath or _get_baseline_filepath_fallback()
comparison = experiment_results.compare_with_baseline(
baseline_filepath=baseline_filepath
)
comparison = experiment_results.compare_with_baseline(baseline_filepath=baseline_filepath)
assert_experiment_no_regression(comparison)
def assert_experiment_no_regression(comparison_result: dict[str, list[str]]) -> None:
regressed = comparison_result.get("regressed", [])
if regressed:
raise AssertionError(
f"Regression detected! The following tests that previously passed now fail: {regressed}"
)
raise AssertionError(f"Regression detected! The following tests that previously passed now fail: {regressed}")
missing_tests = comparison_result.get("missing_tests", [])
if missing_tests:
warnings.warn(
f"Warning: {len(missing_tests)} tests from the baseline are missing in the current run: {missing_tests}",
UserWarning,
stacklevel=2,
UserWarning
)
def run_experiment(
dataset: list[dict[str, Any]],
crew: Crew | None = None,
agents: list[Agent] | None = None,
verbose: bool = False,
) -> ExperimentResults:
def run_experiment(dataset: list[dict[str, Any]], crew: Crew | None = None, agents: list[Agent] | None = None, verbose: bool = False) -> ExperimentResults:
runner = ExperimentRunner(dataset=dataset)
return runner.run(agents=agents, crew=crew, print_summary=verbose)
def _get_baseline_filepath_fallback() -> str:
test_func_name = "experiment_fallback"
try:
current_frame = inspect.currentframe()
if current_frame is not None:
test_func_name = current_frame.f_back.f_back.f_code.co_name # type: ignore[union-attr]
test_func_name = current_frame.f_back.f_back.f_code.co_name # type: ignore[union-attr]
except Exception:
...
return f"{test_func_name}_results.json"
return f"{test_func_name}_results.json"

View File

@@ -1,4 +1,5 @@
from crewai.flow.flow import Flow, and_, listen, or_, router, start
from crewai.flow.flow import Flow, start, listen, or_, and_, router
from crewai.flow.persistence import persist
__all__ = ["Flow", "and_", "listen", "or_", "persist", "router", "start"]
__all__ = ["Flow", "start", "listen", "or_", "and_", "router", "persist"]

View File

@@ -2,22 +2,30 @@ import asyncio
import copy
import inspect
import logging
from collections.abc import Callable
from typing import Any, ClassVar, Generic, TypeVar, cast
from typing import (
Any,
Callable,
Dict,
Generic,
List,
Optional,
Set,
Type,
TypeVar,
Union,
cast,
)
from uuid import uuid4
from opentelemetry import baggage
from opentelemetry.context import attach, detach
from pydantic import BaseModel, Field, ValidationError
from crewai.flow.flow_visualizer import plot_flow
from crewai.flow.persistence.base import FlowPersistence
from crewai.flow.types import FlowExecutionData
from crewai.flow.utils import get_possible_return_constants
from crewai.events.event_bus import crewai_event_bus
from crewai.events.listeners.tracing.trace_listener import (
TraceCollectionListener,
)
from crewai.events.listeners.tracing.utils import (
is_tracing_enabled,
should_auto_collect_first_time_traces,
)
from crewai.events.types.flow_events import (
FlowCreatedEvent,
FlowFinishedEvent,
@@ -27,11 +35,13 @@ from crewai.events.types.flow_events import (
MethodExecutionFinishedEvent,
MethodExecutionStartedEvent,
)
from crewai.flow.flow_visualizer import plot_flow
from crewai.flow.persistence.base import FlowPersistence
from crewai.flow.types import FlowExecutionData
from crewai.flow.utils import get_possible_return_constants
from crewai.utilities.printer import Printer, PrinterColor
from crewai.events.listeners.tracing.trace_listener import (
TraceCollectionListener,
)
from crewai.events.listeners.tracing.utils import (
is_tracing_enabled,
)
from crewai.utilities.printer import Printer
logger = logging.getLogger(__name__)
@@ -45,14 +55,16 @@ class FlowState(BaseModel):
)
# type variables with explicit bounds
T = TypeVar("T", bound=dict[str, Any] | BaseModel) # Generic flow state type parameter
# Type variables with explicit bounds
T = TypeVar(
"T", bound=Union[Dict[str, Any], BaseModel]
) # Generic flow state type parameter
StateT = TypeVar(
"StateT", bound=dict[str, Any] | BaseModel
"StateT", bound=Union[Dict[str, Any], BaseModel]
) # State validation type parameter
def ensure_state_type(state: Any, expected_type: type[StateT]) -> StateT:
def ensure_state_type(state: Any, expected_type: Type[StateT]) -> StateT:
"""Ensure state matches expected type with proper validation.
Args:
@@ -92,7 +104,7 @@ def ensure_state_type(state: Any, expected_type: type[StateT]) -> StateT:
raise TypeError(f"Invalid expected_type: {expected_type}")
def start(condition: str | dict | Callable | None = None) -> Callable:
def start(condition: Optional[Union[str, dict, Callable]] = None) -> Callable:
"""
Marks a method as a flow's starting point.
@@ -159,7 +171,7 @@ def start(condition: str | dict | Callable | None = None) -> Callable:
return decorator
def listen(condition: str | dict | Callable) -> Callable:
def listen(condition: Union[str, dict, Callable]) -> Callable:
"""
Creates a listener that executes when specified conditions are met.
@@ -219,7 +231,7 @@ def listen(condition: str | dict | Callable) -> Callable:
return decorator
def router(condition: str | dict | Callable) -> Callable:
def router(condition: Union[str, dict, Callable]) -> Callable:
"""
Creates a routing method that directs flow execution based on conditions.
@@ -285,7 +297,7 @@ def router(condition: str | dict | Callable) -> Callable:
return decorator
def or_(*conditions: str | dict | Callable) -> dict:
def or_(*conditions: Union[str, dict, Callable]) -> dict:
"""
Combines multiple conditions with OR logic for flow control.
@@ -331,7 +343,7 @@ def or_(*conditions: str | dict | Callable) -> dict:
return {"type": "OR", "methods": methods}
def and_(*conditions: str | dict | Callable) -> dict:
def and_(*conditions: Union[str, dict, Callable]) -> dict:
"""
Combines multiple conditions with AND logic for flow control.
@@ -413,10 +425,10 @@ class FlowMeta(type):
if possible_returns:
router_paths[attr_name] = possible_returns
cls._start_methods = start_methods
cls._listeners = listeners
cls._routers = routers
cls._router_paths = router_paths
setattr(cls, "_start_methods", start_methods)
setattr(cls, "_listeners", listeners)
setattr(cls, "_routers", routers)
setattr(cls, "_router_paths", router_paths)
return cls
@@ -424,29 +436,29 @@ class FlowMeta(type):
class Flow(Generic[T], metaclass=FlowMeta):
"""Base class for all flows.
type parameter T must be either dict[str, Any] or a subclass of BaseModel."""
Type parameter T must be either Dict[str, Any] or a subclass of BaseModel."""
_printer = Printer()
_start_methods: ClassVar[list[str]] = []
_listeners: ClassVar[dict[str, tuple[str, list[str]]]] = {}
_routers: ClassVar[set[str]] = set()
_router_paths: ClassVar[dict[str, list[str]]] = {}
initial_state: type[T] | T | None = None
name: str | None = None
tracing: bool | None = False
_start_methods: List[str] = []
_listeners: Dict[str, tuple[str, List[str]]] = {}
_routers: Set[str] = set()
_router_paths: Dict[str, List[str]] = {}
initial_state: Union[Type[T], T, None] = None
name: Optional[str] = None
tracing: Optional[bool] = False
def __class_getitem__(cls: type["Flow"], item: type[T]) -> type["Flow"]:
def __class_getitem__(cls: Type["Flow"], item: Type[T]) -> Type["Flow"]:
class _FlowGeneric(cls): # type: ignore
_initial_state_t = item # type: ignore
_initial_state_T = item # type: ignore
_FlowGeneric.__name__ = f"{cls.__name__}[{item.__name__}]"
return _FlowGeneric
def __init__(
self,
persistence: FlowPersistence | None = None,
tracing: bool | None = False,
persistence: Optional[FlowPersistence] = None,
tracing: Optional[bool] = False,
**kwargs: Any,
) -> None:
"""Initialize a new Flow instance.
@@ -456,24 +468,23 @@ class Flow(Generic[T], metaclass=FlowMeta):
**kwargs: Additional state values to initialize or override
"""
# Initialize basic instance attributes
self._methods: dict[str, Callable] = {}
self._method_execution_counts: dict[str, int] = {}
self._pending_and_listeners: dict[str, set[str]] = {}
self._method_outputs: list[Any] = [] # list to store all method outputs
self._completed_methods: set[str] = set() # Track completed methods for reload
self._persistence: FlowPersistence | None = persistence
self._methods: Dict[str, Callable] = {}
self._method_execution_counts: Dict[str, int] = {}
self._pending_and_listeners: Dict[str, Set[str]] = {}
self._method_outputs: List[Any] = [] # List to store all method outputs
self._completed_methods: Set[str] = set() # Track completed methods for reload
self._persistence: Optional[FlowPersistence] = persistence
self._is_execution_resuming: bool = False
# Initialize state with initial values
self._state = self._create_initial_state(kwargs)
self._state = self._create_initial_state()
self.tracing = tracing
if (
is_tracing_enabled()
or self.tracing
or should_auto_collect_first_time_traces()
):
if is_tracing_enabled() or self.tracing:
trace_listener = TraceCollectionListener()
trace_listener.setup_listeners(crewai_event_bus)
# Apply any additional kwargs
if kwargs:
self._initialize_state(kwargs)
crewai_event_bus.emit(
self,
@@ -499,12 +510,9 @@ class Flow(Generic[T], metaclass=FlowMeta):
method = method.__get__(self, self.__class__)
self._methods[method_name] = method
def _create_initial_state(self, kwargs: dict[str, Any] | None = None) -> T:
def _create_initial_state(self) -> T:
"""Create and initialize flow state with UUID and default values.
Args:
kwargs: Optional initial values for state fields
Returns:
New state instance with UUID and default values initialized
@@ -513,27 +521,25 @@ class Flow(Generic[T], metaclass=FlowMeta):
TypeError: If state is neither BaseModel nor dictionary
"""
# Handle case where initial_state is None but we have a type parameter
if self.initial_state is None and hasattr(self, "_initial_state_t"):
state_type = self._initial_state_t
if self.initial_state is None and hasattr(self, "_initial_state_T"):
state_type = getattr(self, "_initial_state_T")
if isinstance(state_type, type):
if issubclass(state_type, FlowState):
# Create instance without id, then set it
init_kwargs = kwargs or {}
instance = state_type(**init_kwargs)
instance = state_type()
if not hasattr(instance, "id"):
instance.id = str(uuid4())
setattr(instance, "id", str(uuid4()))
return cast(T, instance)
if issubclass(state_type, BaseModel):
elif issubclass(state_type, BaseModel):
# Create a new type that includes the ID field
class StateWithId(state_type, FlowState): # type: ignore
pass
init_kwargs = kwargs or {}
instance = StateWithId(**init_kwargs)
instance = StateWithId()
if not hasattr(instance, "id"):
instance.id = str(uuid4())
setattr(instance, "id", str(uuid4()))
return cast(T, instance)
if state_type is dict:
elif state_type is dict:
return cast(T, {"id": str(uuid4())})
# Handle case where no initial state is provided
@@ -543,14 +549,14 @@ class Flow(Generic[T], metaclass=FlowMeta):
# Handle case where initial_state is a type (class)
if isinstance(self.initial_state, type):
if issubclass(self.initial_state, FlowState):
return cast(T, self.initial_state(**(kwargs or {})))
if issubclass(self.initial_state, BaseModel):
return cast(T, self.initial_state()) # Uses model defaults
elif issubclass(self.initial_state, BaseModel):
# Validate that the model has an id field
model_fields = getattr(self.initial_state, "model_fields", None)
if not model_fields or "id" not in model_fields:
raise ValueError("Flow state model must have an 'id' field")
return cast(T, self.initial_state(**(kwargs or {})))
if self.initial_state is dict:
return cast(T, self.initial_state()) # Uses model defaults
elif self.initial_state is dict:
return cast(T, {"id": str(uuid4())})
# Handle dictionary instance case
@@ -594,7 +600,7 @@ class Flow(Generic[T], metaclass=FlowMeta):
return self._state
@property
def method_outputs(self) -> list[Any]:
def method_outputs(self) -> List[Any]:
"""Returns the list of all outputs from executed methods."""
return self._method_outputs
@@ -625,13 +631,13 @@ class Flow(Generic[T], metaclass=FlowMeta):
if isinstance(self._state, dict):
return str(self._state.get("id", ""))
if isinstance(self._state, BaseModel):
elif isinstance(self._state, BaseModel):
return str(getattr(self._state, "id", ""))
return ""
except (AttributeError, TypeError):
return "" # Safely handle any unexpected attribute access issues
def _initialize_state(self, inputs: dict[str, Any]) -> None:
def _initialize_state(self, inputs: Dict[str, Any]) -> None:
"""Initialize or update flow state with new inputs.
Args:
@@ -685,7 +691,7 @@ class Flow(Generic[T], metaclass=FlowMeta):
else:
raise TypeError("State must be a BaseModel instance or a dictionary.")
def _restore_state(self, stored_state: dict[str, Any]) -> None:
def _restore_state(self, stored_state: Dict[str, Any]) -> None:
"""Restore flow state from persistence.
Args:
@@ -729,7 +735,7 @@ class Flow(Generic[T], metaclass=FlowMeta):
execution_data: Flow execution data containing:
- id: Flow execution ID
- flow: Flow structure
- completed_methods: list of successfully completed methods
- completed_methods: List of successfully completed methods
- execution_methods: All execution methods with their status
"""
flow_id = execution_data.get("id")
@@ -765,7 +771,7 @@ class Flow(Generic[T], metaclass=FlowMeta):
if state_to_apply:
self._apply_state_updates(state_to_apply)
for method in sorted_methods[:-1]:
for i, method in enumerate(sorted_methods[:-1]):
method_name = method.get("flow_method", {}).get("name")
if method_name:
self._completed_methods.add(method_name)
@@ -777,7 +783,7 @@ class Flow(Generic[T], metaclass=FlowMeta):
elif hasattr(self._state, field_name):
object.__setattr__(self._state, field_name, value)
def _apply_state_updates(self, updates: dict[str, Any]) -> None:
def _apply_state_updates(self, updates: Dict[str, Any]) -> None:
"""Apply multiple state updates efficiently."""
if isinstance(self._state, dict):
self._state.update(updates)
@@ -786,7 +792,7 @@ class Flow(Generic[T], metaclass=FlowMeta):
if hasattr(self._state, key):
object.__setattr__(self._state, key, value)
def kickoff(self, inputs: dict[str, Any] | None = None) -> Any:
def kickoff(self, inputs: Optional[Dict[str, Any]] = None) -> Any:
"""
Start the flow execution in a synchronous context.
@@ -799,7 +805,7 @@ class Flow(Generic[T], metaclass=FlowMeta):
return asyncio.run(run_flow())
async def kickoff_async(self, inputs: dict[str, Any] | None = None) -> Any:
async def kickoff_async(self, inputs: Optional[Dict[str, Any]] = None) -> Any:
"""
Start the flow execution asynchronously.
@@ -834,7 +840,7 @@ class Flow(Generic[T], metaclass=FlowMeta):
if isinstance(self._state, dict):
self._state["id"] = inputs["id"]
elif isinstance(self._state, BaseModel):
setattr(self._state, "id", inputs["id"]) # noqa: B010
setattr(self._state, "id", inputs["id"])
# If persistence is enabled, attempt to restore the stored state using the provided id.
if "id" in inputs and self._persistence is not None:
@@ -1069,7 +1075,7 @@ class Flow(Generic[T], metaclass=FlowMeta):
)
# Now execute normal listeners for all router results and the original trigger
all_triggers = [trigger_method, *router_results]
all_triggers = [trigger_method] + router_results
for current_trigger in all_triggers:
if current_trigger: # Skip None results
@@ -1088,7 +1094,7 @@ class Flow(Generic[T], metaclass=FlowMeta):
for method_name in self._start_methods:
# Check if this start method is triggered by the current trigger
if method_name in self._listeners:
_, trigger_methods = self._listeners[
condition_type, trigger_methods = self._listeners[
method_name
]
if current_trigger in trigger_methods:
@@ -1103,7 +1109,7 @@ class Flow(Generic[T], metaclass=FlowMeta):
def _find_triggered_methods(
self, trigger_method: str, router_only: bool
) -> list[str]:
) -> List[str]:
"""
Finds all methods that should be triggered based on conditions.
@@ -1120,7 +1126,7 @@ class Flow(Generic[T], metaclass=FlowMeta):
Returns
-------
list[str]
List[str]
Names of methods that should be triggered.
Notes
@@ -1220,7 +1226,7 @@ class Flow(Generic[T], metaclass=FlowMeta):
raise
def _log_flow_event(
self, message: str, color: PrinterColor = "yellow", level: str = "info"
self, message: str, color: str = "yellow", level: str = "info"
) -> None:
"""Centralized logging method for flow events.

View File

@@ -1,4 +1,5 @@
import inspect
from typing import Optional
from pydantic import BaseModel, Field, InstanceOf, model_validator
@@ -13,7 +14,7 @@ class FlowTrackable(BaseModel):
inspecting the call stack.
"""
parent_flow: InstanceOf[Flow] | None = Field(
parent_flow: Optional[InstanceOf[Flow]] = Field(
default=None,
description="The parent flow of the instance, if it was created inside a flow.",
)

View File

@@ -1,13 +1,14 @@
# flow_visualizer.py
import os
from pathlib import Path
from pyvis.network import Network # type: ignore[import-untyped]
from pyvis.network import Network
from crewai.flow.config import COLORS, NODE_STYLES
from crewai.flow.html_template_handler import HTMLTemplateHandler
from crewai.flow.legend_generator import generate_legend_items_html, get_legend_items
from crewai.flow.path_utils import safe_path_join
from crewai.flow.path_utils import safe_path_join, validate_path_exists
from crewai.flow.utils import calculate_node_levels
from crewai.flow.visualization_utils import (
add_edges,
@@ -33,13 +34,13 @@ class FlowPlot:
ValueError
If flow object is invalid or missing required attributes.
"""
if not hasattr(flow, "_methods"):
if not hasattr(flow, '_methods'):
raise ValueError("Invalid flow object: missing '_methods' attribute")
if not hasattr(flow, "_listeners"):
if not hasattr(flow, '_listeners'):
raise ValueError("Invalid flow object: missing '_listeners' attribute")
if not hasattr(flow, "_start_methods"):
if not hasattr(flow, '_start_methods'):
raise ValueError("Invalid flow object: missing '_start_methods' attribute")
self.flow = flow
self.colors = COLORS
self.node_styles = NODE_STYLES
@@ -64,7 +65,7 @@ class FlowPlot:
"""
if not filename or not isinstance(filename, str):
raise ValueError("Filename must be a non-empty string")
try:
# Initialize network
net = Network(
@@ -95,34 +96,32 @@ class FlowPlot:
try:
node_levels = calculate_node_levels(self.flow)
except Exception as e:
raise ValueError(f"Failed to calculate node levels: {e!s}") from e
raise ValueError(f"Failed to calculate node levels: {str(e)}")
# Compute positions
try:
node_positions = compute_positions(self.flow, node_levels)
except Exception as e:
raise ValueError(f"Failed to compute node positions: {e!s}") from e
raise ValueError(f"Failed to compute node positions: {str(e)}")
# Add nodes to the network
try:
add_nodes_to_network(net, self.flow, node_positions, self.node_styles)
except Exception as e:
raise RuntimeError(f"Failed to add nodes to network: {e!s}") from e
raise RuntimeError(f"Failed to add nodes to network: {str(e)}")
# Add edges to the network
try:
add_edges(net, self.flow, node_positions, self.colors)
except Exception as e:
raise RuntimeError(f"Failed to add edges to network: {e!s}") from e
raise RuntimeError(f"Failed to add edges to network: {str(e)}")
# Generate HTML
try:
network_html = net.generate_html()
final_html_content = self._generate_final_html(network_html)
except Exception as e:
raise RuntimeError(
f"Failed to generate network visualization: {e!s}"
) from e
raise RuntimeError(f"Failed to generate network visualization: {str(e)}")
# Save the final HTML content to the file
try:
@@ -130,16 +129,12 @@ class FlowPlot:
f.write(final_html_content)
print(f"Plot saved as {filename}.html")
except IOError as e:
raise IOError(
f"Failed to save flow visualization to {filename}.html: {e!s}"
) from e
raise IOError(f"Failed to save flow visualization to {filename}.html: {str(e)}")
except (ValueError, RuntimeError, IOError) as e:
raise e
except Exception as e:
raise RuntimeError(
f"Unexpected error during flow visualization: {e!s}"
) from e
raise RuntimeError(f"Unexpected error during flow visualization: {str(e)}")
finally:
self._cleanup_pyvis_lib()
@@ -170,9 +165,7 @@ class FlowPlot:
try:
# Extract just the body content from the generated HTML
current_dir = os.path.dirname(__file__)
template_path = safe_path_join(
"assets", "crewai_flow_visual_template.html", root=current_dir
)
template_path = safe_path_join("assets", "crewai_flow_visual_template.html", root=current_dir)
logo_path = safe_path_join("assets", "crewai_logo.svg", root=current_dir)
if not os.path.exists(template_path):
@@ -186,9 +179,12 @@ class FlowPlot:
# Generate the legend items HTML
legend_items = get_legend_items(self.colors)
legend_items_html = generate_legend_items_html(legend_items)
return html_handler.generate_final_html(network_body, legend_items_html)
final_html_content = html_handler.generate_final_html(
network_body, legend_items_html
)
return final_html_content
except Exception as e:
raise IOError(f"Failed to generate visualization HTML: {e!s}") from e
raise IOError(f"Failed to generate visualization HTML: {str(e)}")
def _cleanup_pyvis_lib(self):
"""
@@ -201,7 +197,6 @@ class FlowPlot:
lib_folder = safe_path_join("lib", root=os.getcwd())
if os.path.exists(lib_folder) and os.path.isdir(lib_folder):
import shutil
shutil.rmtree(lib_folder)
except ValueError as e:
print(f"Error validating lib folder path: {e}")

View File

@@ -1,7 +1,8 @@
import base64
import re
from pathlib import Path
from crewai.flow.path_utils import validate_path_exists
from crewai.flow.path_utils import safe_path_join, validate_path_exists
class HTMLTemplateHandler:
@@ -27,7 +28,7 @@ class HTMLTemplateHandler:
self.template_path = validate_path_exists(template_path, "file")
self.logo_path = validate_path_exists(logo_path, "file")
except ValueError as e:
raise ValueError(f"Invalid template or logo path: {e}") from e
raise ValueError(f"Invalid template or logo path: {e}")
def read_template(self):
"""Read and return the HTML template file contents."""
@@ -52,23 +53,23 @@ class HTMLTemplateHandler:
if "border" in item:
legend_items_html += f"""
<div class="legend-item">
<div class="legend-color-box" style="background-color: {item["color"]}; border: 2px dashed {item["border"]};"></div>
<div>{item["label"]}</div>
<div class="legend-color-box" style="background-color: {item['color']}; border: 2px dashed {item['border']};"></div>
<div>{item['label']}</div>
</div>
"""
elif item.get("dashed") is not None:
style = "dashed" if item["dashed"] else "solid"
legend_items_html += f"""
<div class="legend-item">
<div class="legend-{style}" style="border-bottom: 2px {style} {item["color"]};"></div>
<div>{item["label"]}</div>
<div class="legend-{style}" style="border-bottom: 2px {style} {item['color']};"></div>
<div>{item['label']}</div>
</div>
"""
else:
legend_items_html += f"""
<div class="legend-item">
<div class="legend-color-box" style="background-color: {item["color"]};"></div>
<div>{item["label"]}</div>
<div class="legend-color-box" style="background-color: {item['color']};"></div>
<div>{item['label']}</div>
</div>
"""
return legend_items_html
@@ -78,9 +79,15 @@ class HTMLTemplateHandler:
html_template = self.read_template()
logo_svg_base64 = self.encode_logo()
return (
html_template.replace("{{ title }}", title)
.replace("{{ network_content }}", network_body)
.replace("{{ logo_svg_base64 }}", logo_svg_base64)
.replace("<!-- LEGEND_ITEMS_PLACEHOLDER -->", legend_items_html)
final_html_content = html_template.replace("{{ title }}", title)
final_html_content = final_html_content.replace(
"{{ network_content }}", network_body
)
final_html_content = final_html_content.replace(
"{{ logo_svg_base64 }}", logo_svg_base64
)
final_html_content = final_html_content.replace(
"<!-- LEGEND_ITEMS_PLACEHOLDER -->", legend_items_html
)
return final_html_content

View File

@@ -5,10 +5,12 @@ This module provides utilities for secure path handling to prevent directory
traversal attacks and ensure paths remain within allowed boundaries.
"""
import os
from pathlib import Path
from typing import List, Union
def safe_path_join(*parts: str, root: str | Path | None = None) -> str:
def safe_path_join(*parts: str, root: Union[str, Path, None] = None) -> str:
"""
Safely join path components and ensure the result is within allowed boundaries.
@@ -41,25 +43,25 @@ def safe_path_join(*parts: str, root: str | Path | None = None) -> str:
# Establish root directory
root_path = Path(root).resolve() if root else Path.cwd()
# Join and resolve the full path
full_path = Path(root_path, *clean_parts).resolve()
# Check if the resolved path is within root
if not str(full_path).startswith(str(root_path)):
raise ValueError(
f"Invalid path: Potential directory traversal. Path must be within {root_path}"
)
return str(full_path)
except Exception as e:
if isinstance(e, ValueError):
raise
raise ValueError(f"Invalid path components: {e!s}") from e
raise ValueError(f"Invalid path components: {str(e)}")
def validate_path_exists(path: str | Path, file_type: str = "file") -> str:
def validate_path_exists(path: Union[str, Path], file_type: str = "file") -> str:
"""
Validate that a path exists and is of the expected type.
@@ -82,24 +84,24 @@ def validate_path_exists(path: str | Path, file_type: str = "file") -> str:
"""
try:
path_obj = Path(path).resolve()
if not path_obj.exists():
raise ValueError(f"Path does not exist: {path}")
if file_type == "file" and not path_obj.is_file():
raise ValueError(f"Path is not a file: {path}")
if file_type == "directory" and not path_obj.is_dir():
elif file_type == "directory" and not path_obj.is_dir():
raise ValueError(f"Path is not a directory: {path}")
return str(path_obj)
except Exception as e:
if isinstance(e, ValueError):
raise
raise ValueError(f"Invalid path: {e!s}") from e
raise ValueError(f"Invalid path: {str(e)}")
def list_files(directory: str | Path, pattern: str = "*") -> list[str]:
def list_files(directory: Union[str, Path], pattern: str = "*") -> List[str]:
"""
Safely list files in a directory matching a pattern.
@@ -124,10 +126,10 @@ def list_files(directory: str | Path, pattern: str = "*") -> list[str]:
dir_path = Path(directory).resolve()
if not dir_path.is_dir():
raise ValueError(f"Not a directory: {directory}")
return [str(p) for p in dir_path.glob(pattern) if p.is_file()]
except Exception as e:
if isinstance(e, ValueError):
raise
raise ValueError(f"Error listing files: {e!s}") from e
raise ValueError(f"Error listing files: {str(e)}")

View File

@@ -4,7 +4,7 @@ CrewAI Flow Persistence.
This module provides interfaces and implementations for persisting flow states.
"""
from typing import Any, TypeVar
from typing import Any, Dict, TypeVar, Union
from pydantic import BaseModel
@@ -12,7 +12,7 @@ from crewai.flow.persistence.base import FlowPersistence
from crewai.flow.persistence.decorators import persist
from crewai.flow.persistence.sqlite import SQLiteFlowPersistence
__all__ = ["FlowPersistence", "SQLiteFlowPersistence", "persist"]
__all__ = ["FlowPersistence", "persist", "SQLiteFlowPersistence"]
StateType = TypeVar("StateType", bound=dict[str, Any] | BaseModel)
DictStateType = dict[str, Any]
StateType = TypeVar('StateType', bound=Union[Dict[str, Any], BaseModel])
DictStateType = Dict[str, Any]

View File

@@ -1,47 +1,53 @@
"""Base class for flow state persistence."""
import abc
from typing import Any
from typing import Any, Dict, Optional, Union
from pydantic import BaseModel
class FlowPersistence(abc.ABC):
"""Abstract base class for flow state persistence.
This class defines the interface that all persistence implementations must follow.
It supports both structured (Pydantic BaseModel) and unstructured (dict) states.
"""
@abc.abstractmethod
def init_db(self) -> None:
"""Initialize the persistence backend.
This method should handle any necessary setup, such as:
- Creating tables
- Establishing connections
- Setting up indexes
"""
pass
@abc.abstractmethod
def save_state(
self, flow_uuid: str, method_name: str, state_data: dict[str, Any] | BaseModel
self,
flow_uuid: str,
method_name: str,
state_data: Union[Dict[str, Any], BaseModel]
) -> None:
"""Persist the flow state after method completion.
Args:
flow_uuid: Unique identifier for the flow instance
method_name: Name of the method that just completed
state_data: Current state data (either dict or Pydantic model)
"""
pass
@abc.abstractmethod
def load_state(self, flow_uuid: str) -> dict[str, Any] | None:
def load_state(self, flow_uuid: str) -> Optional[Dict[str, Any]]:
"""Load the most recent state for a given flow UUID.
Args:
flow_uuid: Unique identifier for the flow instance
Returns:
The most recent state as a dictionary, or None if no state exists
"""
pass

View File

@@ -24,10 +24,13 @@ Example:
import asyncio
import functools
import logging
from collections.abc import Callable
from typing import (
Any,
Callable,
Optional,
Type,
TypeVar,
Union,
cast,
)
@@ -45,7 +48,7 @@ LOG_MESSAGES = {
"save_state": "Saving flow state to memory for ID: {}",
"save_error": "Failed to persist state for method {}: {}",
"state_missing": "Flow instance has no state",
"id_missing": "Flow state must have an 'id' field for persistence",
"id_missing": "Flow state must have an 'id' field for persistence"
}
@@ -55,13 +58,7 @@ class PersistenceDecorator:
_printer = Printer() # Class-level printer instance
@classmethod
def persist_state(
cls,
flow_instance: Any,
method_name: str,
persistence_instance: FlowPersistence,
verbose: bool = False,
) -> None:
def persist_state(cls, flow_instance: Any, method_name: str, persistence_instance: FlowPersistence, verbose: bool = False) -> None:
"""Persist flow state with proper error handling and logging.
This method handles the persistence of flow state data, including proper
@@ -79,24 +76,22 @@ class PersistenceDecorator:
AttributeError: If flow instance lacks required state attributes
"""
try:
state = getattr(flow_instance, "state", None)
state = getattr(flow_instance, 'state', None)
if state is None:
raise ValueError("Flow instance has no state")
flow_uuid: str | None = None
flow_uuid: Optional[str] = None
if isinstance(state, dict):
flow_uuid = state.get("id")
flow_uuid = state.get('id')
elif isinstance(state, BaseModel):
flow_uuid = getattr(state, "id", None)
flow_uuid = getattr(state, 'id', None)
if not flow_uuid:
raise ValueError("Flow state must have an 'id' field for persistence")
# Log state saving only if verbose is True
if verbose:
cls._printer.print(
LOG_MESSAGES["save_state"].format(flow_uuid), color="cyan"
)
cls._printer.print(LOG_MESSAGES["save_state"].format(flow_uuid), color="cyan")
logger.info(LOG_MESSAGES["save_state"].format(flow_uuid))
try:
@@ -109,12 +104,12 @@ class PersistenceDecorator:
error_msg = LOG_MESSAGES["save_error"].format(method_name, str(e))
cls._printer.print(error_msg, color="red")
logger.error(error_msg)
raise RuntimeError(f"State persistence failed: {e!s}") from e
except AttributeError as e:
raise RuntimeError(f"State persistence failed: {str(e)}") from e
except AttributeError:
error_msg = LOG_MESSAGES["state_missing"]
cls._printer.print(error_msg, color="red")
logger.error(error_msg)
raise ValueError(error_msg) from e
raise ValueError(error_msg)
except (TypeError, ValueError) as e:
error_msg = LOG_MESSAGES["id_missing"]
cls._printer.print(error_msg, color="red")
@@ -122,7 +117,7 @@ class PersistenceDecorator:
raise ValueError(error_msg) from e
def persist(persistence: FlowPersistence | None = None, verbose: bool = False):
def persist(persistence: Optional[FlowPersistence] = None, verbose: bool = False):
"""Decorator to persist flow state.
This decorator can be applied at either the class level or method level.
@@ -149,151 +144,111 @@ def persist(persistence: FlowPersistence | None = None, verbose: bool = False):
def begin(self):
pass
"""
def decorator(target: type | Callable[..., T]) -> type | Callable[..., T]:
def decorator(target: Union[Type, Callable[..., T]]) -> Union[Type, Callable[..., T]]:
"""Decorator that handles both class and method decoration."""
actual_persistence = persistence or SQLiteFlowPersistence()
if isinstance(target, type):
# Class decoration
original_init = target.__init__ # type: ignore[misc]
original_init = getattr(target, "__init__")
@functools.wraps(original_init)
def new_init(self: Any, *args: Any, **kwargs: Any) -> None:
if "persistence" not in kwargs:
kwargs["persistence"] = actual_persistence
if 'persistence' not in kwargs:
kwargs['persistence'] = actual_persistence
original_init(self, *args, **kwargs)
target.__init__ = new_init # type: ignore[misc]
setattr(target, "__init__", new_init)
# Store original methods to preserve their decorators
original_methods = {
name: method
for name, method in target.__dict__.items()
if callable(method)
and (
hasattr(method, "__is_start_method__")
or hasattr(method, "__trigger_methods__")
or hasattr(method, "__condition_type__")
or hasattr(method, "__is_flow_method__")
or hasattr(method, "__is_router__")
)
}
original_methods = {}
for name, method in target.__dict__.items():
if callable(method) and (
hasattr(method, "__is_start_method__") or
hasattr(method, "__trigger_methods__") or
hasattr(method, "__condition_type__") or
hasattr(method, "__is_flow_method__") or
hasattr(method, "__is_router__")
):
original_methods[name] = method
# Create wrapped versions of the methods that include persistence
for name, method in original_methods.items():
if asyncio.iscoroutinefunction(method):
# Create a closure to capture the current name and method
def create_async_wrapper(
method_name: str, original_method: Callable
):
def create_async_wrapper(method_name: str, original_method: Callable):
@functools.wraps(original_method)
async def method_wrapper(
self: Any, *args: Any, **kwargs: Any
) -> Any:
async def method_wrapper(self: Any, *args: Any, **kwargs: Any) -> Any:
result = await original_method(self, *args, **kwargs)
PersistenceDecorator.persist_state(
self, method_name, actual_persistence, verbose
)
PersistenceDecorator.persist_state(self, method_name, actual_persistence, verbose)
return result
return method_wrapper
wrapped = create_async_wrapper(name, method)
# Preserve all original decorators and attributes
for attr in [
"__is_start_method__",
"__trigger_methods__",
"__condition_type__",
"__is_router__",
]:
for attr in ["__is_start_method__", "__trigger_methods__", "__condition_type__", "__is_router__"]:
if hasattr(method, attr):
setattr(wrapped, attr, getattr(method, attr))
wrapped.__is_flow_method__ = True # type: ignore[attr-defined]
setattr(wrapped, "__is_flow_method__", True)
# Update the class with the wrapped method
setattr(target, name, wrapped)
else:
# Create a closure to capture the current name and method
def create_sync_wrapper(
method_name: str, original_method: Callable
):
def create_sync_wrapper(method_name: str, original_method: Callable):
@functools.wraps(original_method)
def method_wrapper(self: Any, *args: Any, **kwargs: Any) -> Any:
result = original_method(self, *args, **kwargs)
PersistenceDecorator.persist_state(
self, method_name, actual_persistence, verbose
)
PersistenceDecorator.persist_state(self, method_name, actual_persistence, verbose)
return result
return method_wrapper
wrapped = create_sync_wrapper(name, method)
# Preserve all original decorators and attributes
for attr in [
"__is_start_method__",
"__trigger_methods__",
"__condition_type__",
"__is_router__",
]:
for attr in ["__is_start_method__", "__trigger_methods__", "__condition_type__", "__is_router__"]:
if hasattr(method, attr):
setattr(wrapped, attr, getattr(method, attr))
wrapped.__is_flow_method__ = True # type: ignore[attr-defined]
setattr(wrapped, "__is_flow_method__", True)
# Update the class with the wrapped method
setattr(target, name, wrapped)
return target
# Method decoration
method = target
method.__is_flow_method__ = True # type: ignore[attr-defined]
else:
# Method decoration
method = target
setattr(method, "__is_flow_method__", True)
if asyncio.iscoroutinefunction(method):
if asyncio.iscoroutinefunction(method):
@functools.wraps(method)
async def method_async_wrapper(flow_instance: Any, *args: Any, **kwargs: Any) -> T:
method_coro = method(flow_instance, *args, **kwargs)
if asyncio.iscoroutine(method_coro):
result = await method_coro
else:
result = method_coro
PersistenceDecorator.persist_state(flow_instance, method.__name__, actual_persistence, verbose)
return result
@functools.wraps(method)
async def method_async_wrapper(
flow_instance: Any, *args: Any, **kwargs: Any
) -> T:
method_coro = method(flow_instance, *args, **kwargs)
if asyncio.iscoroutine(method_coro):
result = await method_coro
else:
result = method_coro
PersistenceDecorator.persist_state(
flow_instance, method.__name__, actual_persistence, verbose
)
return result
for attr in ["__is_start_method__", "__trigger_methods__", "__condition_type__", "__is_router__"]:
if hasattr(method, attr):
setattr(method_async_wrapper, attr, getattr(method, attr))
setattr(method_async_wrapper, "__is_flow_method__", True)
return cast(Callable[..., T], method_async_wrapper)
else:
@functools.wraps(method)
def method_sync_wrapper(flow_instance: Any, *args: Any, **kwargs: Any) -> T:
result = method(flow_instance, *args, **kwargs)
PersistenceDecorator.persist_state(flow_instance, method.__name__, actual_persistence, verbose)
return result
for attr in [
"__is_start_method__",
"__trigger_methods__",
"__condition_type__",
"__is_router__",
]:
if hasattr(method, attr):
setattr(method_async_wrapper, attr, getattr(method, attr))
method_async_wrapper.__is_flow_method__ = True # type: ignore[attr-defined]
return cast(Callable[..., T], method_async_wrapper)
@functools.wraps(method)
def method_sync_wrapper(flow_instance: Any, *args: Any, **kwargs: Any) -> T:
result = method(flow_instance, *args, **kwargs)
PersistenceDecorator.persist_state(
flow_instance, method.__name__, actual_persistence, verbose
)
return result
for attr in [
"__is_start_method__",
"__trigger_methods__",
"__condition_type__",
"__is_router__",
]:
if hasattr(method, attr):
setattr(method_sync_wrapper, attr, getattr(method, attr))
method_sync_wrapper.__is_flow_method__ = True # type: ignore[attr-defined]
return cast(Callable[..., T], method_sync_wrapper)
for attr in ["__is_start_method__", "__trigger_methods__", "__condition_type__", "__is_router__"]:
if hasattr(method, attr):
setattr(method_sync_wrapper, attr, getattr(method, attr))
setattr(method_sync_wrapper, "__is_flow_method__", True)
return cast(Callable[..., T], method_sync_wrapper)
return decorator

View File

@@ -6,7 +6,7 @@ import json
import sqlite3
from datetime import datetime, timezone
from pathlib import Path
from typing import Any
from typing import Any, Dict, Optional, Union
from pydantic import BaseModel
@@ -23,7 +23,7 @@ class SQLiteFlowPersistence(FlowPersistence):
db_path: str
def __init__(self, db_path: str | None = None):
def __init__(self, db_path: Optional[str] = None):
"""Initialize SQLite persistence.
Args:
@@ -70,7 +70,7 @@ class SQLiteFlowPersistence(FlowPersistence):
self,
flow_uuid: str,
method_name: str,
state_data: dict[str, Any] | BaseModel,
state_data: Union[Dict[str, Any], BaseModel],
) -> None:
"""Save the current flow state to SQLite.
@@ -107,7 +107,7 @@ class SQLiteFlowPersistence(FlowPersistence):
),
)
def load_state(self, flow_uuid: str) -> dict[str, Any] | None:
def load_state(self, flow_uuid: str) -> Optional[Dict[str, Any]]:
"""Load the most recent state for a given flow UUID.
Args:

View File

@@ -5,7 +5,6 @@ the Flow system.
"""
from typing import Any, TypedDict
from typing_extensions import NotRequired, Required

View File

@@ -17,10 +17,10 @@ import ast
import inspect
import textwrap
from collections import defaultdict, deque
from typing import Any
from typing import Any, Deque, Dict, List, Optional, Set, Union
def get_possible_return_constants(function: Any) -> list[str] | None:
def get_possible_return_constants(function: Any) -> Optional[List[str]]:
try:
source = inspect.getsource(function)
except OSError:
@@ -58,12 +58,12 @@ def get_possible_return_constants(function: Any) -> list[str] | None:
target = node.targets[0]
if isinstance(target, ast.Name):
var_name = target.id
dict_values = []
# Extract string values from the dictionary
dict_values = [
val.value
for val in node.value.values
if isinstance(val, ast.Constant) and isinstance(val.value, str)
]
for val in node.value.values:
if isinstance(val, ast.Constant) and isinstance(val.value, str):
dict_values.append(val.value)
# If non-string, skip or just ignore
if dict_values:
dict_definitions[var_name] = dict_values
self.generic_visit(node)
@@ -94,7 +94,7 @@ def get_possible_return_constants(function: Any) -> list[str] | None:
return list(return_values) if return_values else None
def calculate_node_levels(flow: Any) -> dict[str, int]:
def calculate_node_levels(flow: Any) -> Dict[str, int]:
"""
Calculate the hierarchical level of each node in the flow.
@@ -118,10 +118,10 @@ def calculate_node_levels(flow: Any) -> dict[str, int]:
- Handles both OR and AND conditions for listeners
- Processes router paths separately
"""
levels: dict[str, int] = {}
queue: deque[str] = deque()
visited: set[str] = set()
pending_and_listeners: dict[str, set[str]] = {}
levels: Dict[str, int] = {}
queue: Deque[str] = deque()
visited: Set[str] = set()
pending_and_listeners: Dict[str, Set[str]] = {}
# Make all start methods at level 0
for method_name, method in flow._methods.items():
@@ -172,7 +172,7 @@ def calculate_node_levels(flow: Any) -> dict[str, int]:
return levels
def count_outgoing_edges(flow: Any) -> dict[str, int]:
def count_outgoing_edges(flow: Any) -> Dict[str, int]:
"""
Count the number of outgoing edges for each method in the flow.
@@ -197,7 +197,7 @@ def count_outgoing_edges(flow: Any) -> dict[str, int]:
return counts
def build_ancestor_dict(flow: Any) -> dict[str, set[str]]:
def build_ancestor_dict(flow: Any) -> Dict[str, Set[str]]:
"""
Build a dictionary mapping each node to its ancestor nodes.
@@ -211,8 +211,8 @@ def build_ancestor_dict(flow: Any) -> dict[str, set[str]]:
Dict[str, Set[str]]
Dictionary mapping each node to a set of its ancestor nodes.
"""
ancestors: dict[str, set[str]] = {node: set() for node in flow._methods}
visited: set[str] = set()
ancestors: Dict[str, Set[str]] = {node: set() for node in flow._methods}
visited: Set[str] = set()
for node in flow._methods:
if node not in visited:
dfs_ancestors(node, ancestors, visited, flow)
@@ -220,7 +220,7 @@ def build_ancestor_dict(flow: Any) -> dict[str, set[str]]:
def dfs_ancestors(
node: str, ancestors: dict[str, set[str]], visited: set[str], flow: Any
node: str, ancestors: Dict[str, Set[str]], visited: Set[str], flow: Any
) -> None:
"""
Perform depth-first search to build ancestor relationships.
@@ -265,7 +265,7 @@ def dfs_ancestors(
def is_ancestor(
node: str, ancestor_candidate: str, ancestors: dict[str, set[str]]
node: str, ancestor_candidate: str, ancestors: Dict[str, Set[str]]
) -> bool:
"""
Check if one node is an ancestor of another.
@@ -287,7 +287,7 @@ def is_ancestor(
return ancestor_candidate in ancestors.get(node, set())
def build_parent_children_dict(flow: Any) -> dict[str, list[str]]:
def build_parent_children_dict(flow: Any) -> Dict[str, List[str]]:
"""
Build a dictionary mapping parent nodes to their children.
@@ -307,7 +307,7 @@ def build_parent_children_dict(flow: Any) -> dict[str, list[str]]:
- Maps router methods to their paths and listeners
- Children lists are sorted for consistent ordering
"""
parent_children: dict[str, list[str]] = {}
parent_children: Dict[str, List[str]] = {}
# Map listeners to their trigger methods
for listener_name, (_, trigger_methods) in flow._listeners.items():
@@ -332,7 +332,7 @@ def build_parent_children_dict(flow: Any) -> dict[str, list[str]]:
def get_child_index(
parent: str, child: str, parent_children: dict[str, list[str]]
parent: str, child: str, parent_children: Dict[str, List[str]]
) -> int:
"""
Get the index of a child node in its parent's sorted children list.
@@ -364,7 +364,7 @@ def process_router_paths(flow, current, current_level, levels, queue):
paths = flow._router_paths.get(current, [])
for path in paths:
for listener_name, (
_condition_type,
condition_type,
trigger_methods,
) in flow._listeners.items():
if path in trigger_methods:

Some files were not shown because too many files have changed in this diff Show More