Compare commits

...

9 Commits

Author SHA1 Message Date
Devin AI
b568bcee87 chore: trigger CI re-run
Co-Authored-By: João <joao@crewai.com>
2026-01-14 22:59:02 +00:00
Devin AI
7fb8999545 fix: only pass tools parameter when tools are available
This fixes the CI test failures by only passing the tools parameter to
llm.call() and llm.acall() when tools are actually available. This
maintains backward compatibility with existing code that checks
'tools' in kwargs to determine if tools were provided.

The previous commit always passed tools=None, which caused tests that
check 'tools' in kwargs to fail because the key was always present.

Co-Authored-By: João <joao@crewai.com>
2026-01-14 22:54:18 +00:00
Devin AI
d97d58d2c3 fix: pass tools to llm.call() for native function calling support
This fixes issue #4238 where Gemini models fail with UNEXPECTED_TOOL_CALL
errors because tools were not being passed to the LLM call.

Changes:
- Add _extract_tools_from_context() helper function to extract tools from
  executor context (CrewAgentExecutor or LiteAgent) and convert them to
  dict format compatible with LLM providers
- Update get_llm_response() to extract and pass tools to llm.call()
- Update aget_llm_response() to extract and pass tools to llm.acall()
- Add comprehensive tests for the new functionality

Co-Authored-By: João <joao@crewai.com>
2026-01-14 22:48:48 +00:00
Lorenze Jay
9edbf89b68 fix: enhance Azure model stop word support detection (#4227)
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Notify Downstream / notify-downstream (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
- Updated the `supports_stop_words` method to accurately reflect support for stop sequences based on model type, specifically excluding GPT-5 and O-series models.
- Added comprehensive tests to verify that GPT-5 family and O-series models do not support stop words, ensuring correct behavior in completion parameter preparation.
- Ensured that stop words are not included in parameters for unsupported models while maintaining expected behavior for supported models.
2026-01-13 10:23:59 -08:00
Vini Brasil
685f7b9af1 Increase frame inspection depth to detect parent_flow (#4231)
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Notify Downstream / notify-downstream (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
This commit fixes a bug where `parent_flow` was not being set because
the maximum depth was not sufficient to search for an instance of `Flow`
in the current call stack frame during Flow instantiation.
2026-01-13 18:40:22 +01:00
Anaisdg
595fdfb6e7 feat: add galileo to integrations page (#4130)
* feat: add galileo to integrations page

* fix: linting issues

* fix: clarification on hanlder

* fix: uv install, load_dotenv redundancy, spelling error

* add: translations fix uv install and typo

* fix: broken links

---------

Co-authored-by: Anais <anais@Anaiss-MacBook-Pro.local>
Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com>
Co-authored-by: Anais <anais@Mac.lan>
2026-01-13 08:49:17 -08:00
Koushiv
8f99fa76ed feat: additional a2a transports
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
Notify Downstream / notify-downstream (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
Co-authored-by: Koushiv Sadhukhan <koushiv.777@gmail.com>
Co-authored-by: Greyson LaLonde <greyson.r.lalonde@gmail.com>
2026-01-12 12:03:06 -05:00
GininDenis
17e3fcbe1f fix: unlink task in execution spans
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Notify Downstream / notify-downstream (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
Co-authored-by: Greyson LaLonde <greyson.r.lalonde@gmail.com>
2026-01-12 02:58:42 -05:00
Joao Moura
b858d705a8 updating docs
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Notify Downstream / notify-downstream (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
2026-01-11 16:02:55 -08:00
25 changed files with 1185 additions and 38 deletions

View File

@@ -291,6 +291,7 @@
"en/observability/arize-phoenix",
"en/observability/braintrust",
"en/observability/datadog",
"en/observability/galileo",
"en/observability/langdb",
"en/observability/langfuse",
"en/observability/langtrace",
@@ -742,6 +743,7 @@
"pt-BR/observability/arize-phoenix",
"pt-BR/observability/braintrust",
"pt-BR/observability/datadog",
"pt-BR/observability/galileo",
"pt-BR/observability/langdb",
"pt-BR/observability/langfuse",
"pt-BR/observability/langtrace",
@@ -1203,6 +1205,7 @@
"ko/observability/arize-phoenix",
"ko/observability/braintrust",
"ko/observability/datadog",
"ko/observability/galileo",
"ko/observability/langdb",
"ko/observability/langfuse",
"ko/observability/langtrace",

View File

@@ -574,6 +574,10 @@ When you run this Flow, the output will change based on the random boolean value
### Human in the Loop (human feedback)
<Note>
The `@human_feedback` decorator requires **CrewAI version 1.8.0 or higher**.
</Note>
The `@human_feedback` decorator enables human-in-the-loop workflows by pausing flow execution to collect feedback from a human. This is useful for approval gates, quality review, and decision points that require human judgment.
```python Code

View File

@@ -91,6 +91,10 @@ The `A2AConfig` class accepts the following parameters:
Update mechanism for receiving task status. Options: `StreamingConfig`, `PollingConfig`, or `PushNotificationConfig`.
</ParamField>
<ParamField path="transport_protocol" type="Literal['JSONRPC', 'GRPC', 'HTTP+JSON']" default="JSONRPC">
Transport protocol for A2A communication. Options: `JSONRPC` (default), `GRPC`, or `HTTP+JSON`.
</ParamField>
## Authentication
For A2A agents that require authentication, use one of the provided auth schemes:

View File

@@ -7,6 +7,10 @@ mode: "wide"
## Overview
<Note>
The `@human_feedback` decorator requires **CrewAI version 1.8.0 or higher**. Make sure to update your installation before using this feature.
</Note>
The `@human_feedback` decorator enables human-in-the-loop (HITL) workflows directly within CrewAI Flows. It allows you to pause flow execution, present output to a human for review, collect their feedback, and optionally route to different listeners based on the feedback outcome.
This is particularly valuable for:

View File

@@ -11,10 +11,10 @@ Human-in-the-Loop (HITL) is a powerful approach that combines artificial intelli
CrewAI offers two main approaches for implementing human-in-the-loop workflows:
| Approach | Best For | Integration |
|----------|----------|-------------|
| **Flow-based** (`@human_feedback` decorator) | Local development, console-based review, synchronous workflows | [Human Feedback in Flows](/en/learn/human-feedback-in-flows) |
| **Webhook-based** (Enterprise) | Production deployments, async workflows, external integrations (Slack, Teams, etc.) | This guide |
| Approach | Best For | Integration | Version |
|----------|----------|-------------|---------|
| **Flow-based** (`@human_feedback` decorator) | Local development, console-based review, synchronous workflows | [Human Feedback in Flows](/en/learn/human-feedback-in-flows) | **1.8.0+** |
| **Webhook-based** (Enterprise) | Production deployments, async workflows, external integrations (Slack, Teams, etc.) | This guide | - |
<Tip>
If you're building flows and want to add human review steps with routing based on feedback, check out the [Human Feedback in Flows](/en/learn/human-feedback-in-flows) guide for the `@human_feedback` decorator.

View File

@@ -0,0 +1,115 @@
---
title: Galileo
description: Galileo integration for CrewAI tracing and evaluation
icon: telescope
mode: "wide"
---
## Overview
This guide demonstrates how to integrate **Galileo** with **CrewAI**
for comprehensive tracing and Evaluation Engineering.
By the end of this guide, you will be able to trace your CrewAI agents,
monitor their performance, and evaluate their behaviour with
Galileo's powerful observability platform.
> **What is Galileo?** [Galileo](https://galileo.ai) is AI evaluation and observability
platform that delivers end-to-end tracing, evaluation,
and monitoring for AI applications. It enables teams to capture ground truth,
create robust guardrails, and run systematic experiments with
built-in experiment tracking and performance analytics—ensuring reliability,
transparency, and continuous improvement across the AI lifecycle.
## Getting started
This tutorial follows the [CrewAI quickstart](/en/quickstart) and shows how to add
Galileo's [CrewAIEventListener](https://v2docs.galileo.ai/sdk-api/python/reference/handlers/crewai/handler),
an event handler.
For more information, see Galileos
[Add Galileo to a CrewAI Application](https://v2docs.galileo.ai/how-to-guides/third-party-integrations/add-galileo-to-crewai/add-galileo-to-crewai)
how-to guide.
> **Note** This tutorial assumes you have completed the [CrewAI quickstart](/en/quickstart).
If you want a completed comprehensive example, see the Galileo
[CrewAI sdk-example repo](https://github.com/rungalileo/sdk-examples/tree/main/python/agent/crew-ai).
### Step 1: Install dependencies
Install the required dependencies for your app.
Create a virtual environment using your preferred method,
then install dependencies inside that environment using your
preferred tool:
```bash
uv add galileo
```
### Step 2: Add to the .env file from the [CrewAI quickstart](/en/quickstart)
```bash
# Your Galileo API key
GALILEO_API_KEY="your-galileo-api-key"
# Your Galileo project name
GALILEO_PROJECT="your-galileo-project-name"
# The name of the Log stream you want to use for logging
GALILEO_LOG_STREAM="your-galileo-log-stream "
```
### Step 3: Add the Galileo event listener
To enable logging with Galileo, you need to create an instance of the `CrewAIEventListener`.
Import the Galileo CrewAI handler package by
adding the following code at the top of your main.py file:
```python
from galileo.handlers.crewai.handler import CrewAIEventListener
```
At the start of your run function, create the event listener:
```python
def run():
# Create the event listener
CrewAIEventListener()
# The rest of your existing code goes here
```
When you create the listener instance, it is automatically
registered with CrewAI.
### Step 4: Run your crew
Run your crew with the CrewAI CLI:
```bash
crewai run
```
### Step 5: View the traces in Galileo
Once your crew has finished, the traces will be flushed and appear in Galileo.
![Galileo trace view](/images/galileo-trace-veiw.png)
## Understanding the Galileo Integration
Galileo integrates with CrewAI by registering an event listener
that captures Crew execution events (e.g., agent actions, tool calls, model responses)
and forwards them to Galileo for observability and evaluation.
### Understanding the event listener
Creating a `CrewAIEventListener()` instance is all thats
required to enable Galileo for a CrewAI run. When instantiated, the listener:
- Automatically registers itself with CrewAI
- Reads Galileo configuration from environment variables
- Logs all run data to the Galileo project and log stream specified by
`GALILEO_PROJECT` and `GALILEO_LOG_STREAM`
No additional configuration or code changes are required.
All data from this run is logged to the Galileo project and
log stream specified by your environment configuration
(for example, GALILEO_PROJECT and GALILEO_LOG_STREAM).

Binary file not shown.

After

Width:  |  Height:  |  Size: 239 KiB

View File

@@ -567,6 +567,10 @@ Fourth method running
### Human in the Loop (인간 피드백)
<Note>
`@human_feedback` 데코레이터는 **CrewAI 버전 1.8.0 이상**이 필요합니다.
</Note>
`@human_feedback` 데코레이터는 인간의 피드백을 수집하기 위해 플로우 실행을 일시 중지하는 human-in-the-loop 워크플로우를 가능하게 합니다. 이는 승인 게이트, 품질 검토, 인간의 판단이 필요한 결정 지점에 유용합니다.
```python Code

View File

@@ -7,6 +7,10 @@ mode: "wide"
## 개요
<Note>
`@human_feedback` 데코레이터는 **CrewAI 버전 1.8.0 이상**이 필요합니다. 이 기능을 사용하기 전에 설치를 업데이트하세요.
</Note>
`@human_feedback` 데코레이터는 CrewAI Flow 내에서 직접 human-in-the-loop(HITL) 워크플로우를 가능하게 합니다. Flow 실행을 일시 중지하고, 인간에게 검토를 위해 출력을 제시하고, 피드백을 수집하고, 선택적으로 피드백 결과에 따라 다른 리스너로 라우팅할 수 있습니다.
이는 특히 다음과 같은 경우에 유용합니다:

View File

@@ -5,9 +5,22 @@ icon: "user-check"
mode: "wide"
---
휴먼 인 더 루프(HITL, Human-in-the-Loop)는 인공지능과 인간의 전문 지식을 결합하여 의사결정을 강화하고 작업 결과를 향상시키는 강력한 접근 방식입니다. 이 가이드에서는 CrewAI 내에서 HITL을 구현하는 방법을 안내합니다.
휴먼 인 더 루프(HITL, Human-in-the-Loop)는 인공지능과 인간의 전문 지식을 결합하여 의사결정을 강화하고 작업 결과를 향상시키는 강력한 접근 방식입니다. CrewAI는 필요에 따라 HITL을 구현하는 여러 가지 방법을 제공합니다.
## HITL 워크플로우 설정
## HITL 접근 방식 선택
CrewAI는 human-in-the-loop 워크플로우를 구현하기 위한 두 가지 주요 접근 방식을 제공합니다:
| 접근 방식 | 적합한 용도 | 통합 | 버전 |
|----------|----------|-------------|---------|
| **Flow 기반** (`@human_feedback` 데코레이터) | 로컬 개발, 콘솔 기반 검토, 동기식 워크플로우 | [Flow에서 인간 피드백](/ko/learn/human-feedback-in-flows) | **1.8.0+** |
| **Webhook 기반** (Enterprise) | 프로덕션 배포, 비동기 워크플로우, 외부 통합 (Slack, Teams 등) | 이 가이드 | - |
<Tip>
Flow를 구축하면서 피드백을 기반으로 라우팅하는 인간 검토 단계를 추가하려면 `@human_feedback` 데코레이터에 대한 [Flow에서 인간 피드백](/ko/learn/human-feedback-in-flows) 가이드를 참조하세요.
</Tip>
## Webhook 기반 HITL 워크플로우 설정
<Steps>
<Step title="작업 구성">

View File

@@ -0,0 +1,115 @@
---
title: Galileo 갈릴레오
description: CrewAI 추적 및 평가를 위한 Galileo 통합
icon: telescope
mode: "wide"
---
## 개요
이 가이드는 **Galileo**를 **CrewAI**와 통합하는 방법을 보여줍니다.
포괄적인 추적 및 평가 엔지니어링을 위한 것입니다.
이 가이드가 끝나면 CrewAI 에이전트를 추적할 수 있게 됩니다.
성과를 모니터링하고 행동을 평가합니다.
Galileo의 강력한 관측 플랫폼.
> **갈릴레오(Galileo)란 무엇인가요?**[Galileo](https://galileo.ai/)는 AI 평가 및 관찰 가능성입니다.
엔드투엔드 추적, 평가,
AI 애플리케이션 모니터링. 이를 통해 팀은 실제 사실을 포착할 수 있습니다.
견고한 가드레일을 만들고 체계적인 실험을 실행하세요.
내장된 실험 추적 및 성능 분석으로 신뢰성 보장
AI 수명주기 전반에 걸쳐 투명성과 지속적인 개선을 제공합니다.
## 시작하기
이 튜토리얼은 [CrewAI 빠른 시작](/ko/quickstart.mdx)을 따르며 추가하는 방법을 보여줍니다.
갈릴레오의 [CrewAIEventListener](https://v2docs.galileo.ai/sdk-api/python/reference/handlers/crewai/handler),
이벤트 핸들러.
자세한 내용은 갈릴레오 문서를 참고하세요.
[CrewAI 애플리케이션에 Galileo 추가](https://v2docs.galileo.ai/how-to-guides/third-party-integrations/add-galileo-to-crewai/add-galileo-to-crewai)
방법 안내.
> **참고**이 튜토리얼에서는 [CrewAI 빠른 시작](/ko/quickstart.mdx)을 완료했다고 가정합니다.
완전한 포괄적인 예제를 원한다면 Galileo
[CrewAI SDK 예제 저장소](https://github.com/rungalileo/sdk-examples/tree/main/python/agent/crew-ai).
### 1단계: 종속성 설치
앱에 필요한 종속성을 설치합니다.
원하는 방법으로 가상 환경을 생성하고,
그런 다음 다음을 사용하여 해당 환경 내에 종속성을 설치하십시오.
선호하는 도구:
```bash
uv add galileo
```
### 2단계: [CrewAI 빠른 시작](/ko/quickstart.mdx)에서 .env 파일에 추가
```bash
# Your Galileo API key
GALILEO_API_KEY="your-galileo-api-key"
# Your Galileo project name
GALILEO_PROJECT="your-galileo-project-name"
# The name of the Log stream you want to use for logging
GALILEO_LOG_STREAM="your-galileo-log-stream "
```
### 3단계: Galileo 이벤트 리스너 추가
Galileo로 로깅을 활성화하려면 `CrewAIEventListener`의 인스턴스를 생성해야 합니다.
다음을 통해 Galileo CrewAI 핸들러 패키지를 가져옵니다.
main.py 파일 상단에 다음 코드를 추가하세요.
```python
from galileo.handlers.crewai.handler import CrewAIEventListener
```
실행 함수 시작 시 이벤트 리스너를 생성합니다.
```python
def run():
# Create the event listener
CrewAIEventListener()
# The rest of your existing code goes here
```
리스너 인스턴스를 생성하면 자동으로
CrewAI에 등록되었습니다.
### 4단계: Crew Agent 실행
CrewAI CLI를 사용하여 Crew Agent를 실행하세요.
```bash
crewai run
```
### 5단계: Galileo에서 추적 보기
승무원 에이전트가 완료되면 흔적이 플러시되어 Galileo에 나타납니다.
![Galileo trace view](/images/galileo-trace-veiw.png)
## 갈릴레오 통합 이해
Galileo는 이벤트 리스너를 등록하여 CrewAI와 통합됩니다.
승무원 실행 이벤트(예: 에이전트 작업, 도구 호출, 모델 응답)를 캡처합니다.
관찰 가능성과 평가를 위해 이를 갈릴레오에 전달합니다.
### 이벤트 리스너 이해
`CrewAIEventListener()` 인스턴스를 생성하는 것이 전부입니다.
CrewAI 실행을 위해 Galileo를 활성화하는 데 필요합니다. 인스턴스화되면 리스너는 다음을 수행합니다.
-CrewAI에 자동으로 등록됩니다.
-환경 변수에서 Galileo 구성을 읽습니다.
-모든 실행 데이터를 Galileo 프로젝트 및 다음에서 지정한 로그 스트림에 기록합니다.
`GALILEO_PROJECT` 및 `GALILEO_LOG_STREAM`
추가 구성이나 코드 변경이 필요하지 않습니다.
이 실행의 모든 데이터는 Galileo 프로젝트에 기록되며
환경 구성에 따라 지정된 로그 스트림
(예: GALILEO_PROJECT 및 GALILEO_LOG_STREAM)

View File

@@ -309,6 +309,10 @@ Ao executar esse Flow, a saída será diferente dependendo do valor booleano ale
### Human in the Loop (feedback humano)
<Note>
O decorador `@human_feedback` requer **CrewAI versão 1.8.0 ou superior**.
</Note>
O decorador `@human_feedback` permite fluxos de trabalho human-in-the-loop, pausando a execução do flow para coletar feedback de um humano. Isso é útil para portões de aprovação, revisão de qualidade e pontos de decisão que requerem julgamento humano.
```python Code

View File

@@ -7,6 +7,10 @@ mode: "wide"
## Visão Geral
<Note>
O decorador `@human_feedback` requer **CrewAI versão 1.8.0 ou superior**. Certifique-se de atualizar sua instalação antes de usar este recurso.
</Note>
O decorador `@human_feedback` permite fluxos de trabalho human-in-the-loop (HITL) diretamente nos CrewAI Flows. Ele permite pausar a execução do flow, apresentar a saída para um humano revisar, coletar seu feedback e, opcionalmente, rotear para diferentes listeners com base no resultado do feedback.
Isso é particularmente valioso para:

View File

@@ -5,9 +5,22 @@ icon: "user-check"
mode: "wide"
---
Human-in-the-Loop (HITL) é uma abordagem poderosa que combina a inteligência artificial com a experiência humana para aprimorar a tomada de decisões e melhorar os resultados das tarefas. Este guia mostra como implementar HITL dentro da CrewAI.
Human-in-the-Loop (HITL) é uma abordagem poderosa que combina a inteligência artificial com a experiência humana para aprimorar a tomada de decisões e melhorar os resultados das tarefas. CrewAI oferece várias maneiras de implementar HITL dependendo das suas necessidades.
## Configurando Workflows HITL
## Escolhendo Sua Abordagem HITL
CrewAI oferece duas abordagens principais para implementar workflows human-in-the-loop:
| Abordagem | Melhor Para | Integração | Versão |
|----------|----------|-------------|---------|
| **Baseada em Flow** (decorador `@human_feedback`) | Desenvolvimento local, revisão via console, workflows síncronos | [Feedback Humano em Flows](/pt-BR/learn/human-feedback-in-flows) | **1.8.0+** |
| **Baseada em Webhook** (Enterprise) | Deployments em produção, workflows assíncronos, integrações externas (Slack, Teams, etc.) | Este guia | - |
<Tip>
Se você está construindo flows e deseja adicionar etapas de revisão humana com roteamento baseado em feedback, confira o guia [Feedback Humano em Flows](/pt-BR/learn/human-feedback-in-flows) para o decorador `@human_feedback`.
</Tip>
## Configurando Workflows HITL Baseados em Webhook
<Steps>
<Step title="Configure sua Tarefa">

View File

@@ -0,0 +1,115 @@
---
title: Galileo Galileu
description: Integração Galileo para rastreamento e avaliação CrewAI
icon: telescope
mode: "wide"
---
## Visão geral
Este guia demonstra como integrar o **Galileo**com o **CrewAI**
para rastreamento abrangente e engenharia de avaliação.
Ao final deste guia, você será capaz de rastrear seus agentes CrewAI,
monitorar seu desempenho e avaliar seu comportamento com
A poderosa plataforma de observabilidade do Galileo.
> **O que é Galileo?**[Galileo](https://galileo.ai/) é avaliação e observabilidade de IA
plataforma que oferece rastreamento, avaliação e
e monitoramento de aplicações de IA. Ele permite que as equipes capturem a verdade,
criar grades de proteção robustas e realizar experimentos sistemáticos com
rastreamento de experimentos integrado e análise de desempenho -garantindo confiabilidade,
transparência e melhoria contínua em todo o ciclo de vida da IA.
## Primeiros passos
Este tutorial segue o [CrewAI Quickstart](pt-BR/quickstart) e mostra como adicionar
[CrewAIEventListener] do Galileo(https://v2docs.galileo.ai/sdk-api/python/reference/handlers/crewai/handler),
um manipulador de eventos.
Para mais informações, consulte Galileu
[Adicionar Galileo a um aplicativo CrewAI](https://v2docs.galileo.ai/how-to-guides/third-party-integrations/add-galileo-to-crewai/add-galileo-to-crewai)
guia prático.
> **Observação**Este tutorial pressupõe que você concluiu o [CrewAI Quickstart](pt-BR/quickstart).
Se você quiser um exemplo completo e abrangente, consulte o Galileo
[Repositório de exemplo SDK da CrewAI](https://github.com/rungalileo/sdk-examples/tree/main/python/agent/crew-ai).
### Etapa 1: instalar dependências
Instale as dependências necessárias para seu aplicativo.
Crie um ambiente virtual usando seu método preferido,
em seguida, instale dependências dentro desse ambiente usando seu
ferramenta preferida:
```bash
uv add galileo
```
### Etapa 2: adicione ao arquivo .env do [CrewAI Quickstart](/pt-BR/quickstart)
```bash
# Your Galileo API key
GALILEO_API_KEY="your-galileo-api-key"
# Your Galileo project name
GALILEO_PROJECT="your-galileo-project-name"
# The name of the Log stream you want to use for logging
GALILEO_LOG_STREAM="your-galileo-log-stream "
```
### Etapa 3: adicionar o ouvinte de eventos Galileo
Para habilitar o registro com Galileo, você precisa criar uma instância do `CrewAIEventListener`.
Importe o pacote manipulador Galileo CrewAI por
adicionando o seguinte código no topo do seu arquivo main.py:
```python
from galileo.handlers.crewai.handler import CrewAIEventListener
```
No início da sua função run, crie o ouvinte de evento:
```python
def run():
# Create the event listener
CrewAIEventListener()
# The rest of your existing code goes here
```
Quando você cria a instância do listener, ela é automaticamente
registrado na CrewAI.
### Etapa 4: administre sua Crew
Administre sua Crew com o CrewAI CLI:
```bash
crewai run
```
### Passo 5: Visualize os traços no Galileo
Assim que sua tripulação terminar, os rastros serão eliminados e aparecerão no Galileo.
![Galileo trace view](/images/galileo-trace-veiw.png)
## Compreendendo a integração do Galileo
Galileo se integra ao CrewAI registrando um ouvinte de evento
que captura eventos de execução da tripulação (por exemplo, ações do agente, chamadas de ferramentas, respostas do modelo)
e os encaminha ao Galileo para observabilidade e avaliação.
### Compreendendo o ouvinte de eventos
Criar uma instância `CrewAIEventListener()` é tudo o que você precisa
necessário para habilitar o Galileo para uma execução do CrewAI. Quando instanciado, o ouvinte:
-Registra-se automaticamente no CrewAI
-Lê a configuração do Galileo a partir de variáveis de ambiente
-Registra todos os dados de execução no projeto Galileo e fluxo de log especificado por
`GALILEO_PROJECT` e `GALILEO_LOG_STREAM`
Nenhuma configuração adicional ou alterações de código são necessárias.
Todos os dados desta execução são registados no projecto Galileo e
fluxo de log especificado pela configuração do seu ambiente
(por exemplo, GALILEO_PROJECT e GALILEO_LOG_STREAM).

View File

@@ -5,7 +5,7 @@ This module is separate from experimental.a2a to avoid circular imports.
from __future__ import annotations
from typing import Annotated, Any, ClassVar
from typing import Annotated, Any, ClassVar, Literal
from pydantic import (
BaseModel,
@@ -53,6 +53,7 @@ class A2AConfig(BaseModel):
fail_fast: If True, raise error when agent unreachable; if False, skip and continue.
trust_remote_completion_status: If True, return A2A agent's result directly when completed.
updates: Update mechanism config.
transport_protocol: A2A transport protocol (grpc, jsonrpc, http+json).
"""
model_config: ClassVar[ConfigDict] = ConfigDict(extra="forbid")
@@ -82,3 +83,7 @@ class A2AConfig(BaseModel):
default_factory=_get_default_update_config,
description="Update mechanism config",
)
transport_protocol: Literal["JSONRPC", "GRPC", "HTTP+JSON"] = Field(
default="JSONRPC",
description="Specified mode of A2A transport protocol",
)

View File

@@ -7,7 +7,7 @@ from collections.abc import AsyncIterator, MutableMapping
from contextlib import asynccontextmanager
from functools import lru_cache
import time
from typing import TYPE_CHECKING, Any
from typing import TYPE_CHECKING, Any, Literal
import uuid
from a2a.client import A2AClientHTTPError, Client, ClientConfig, ClientFactory
@@ -18,7 +18,6 @@ from a2a.types import (
PushNotificationConfig as A2APushNotificationConfig,
Role,
TextPart,
TransportProtocol,
)
from aiocache import cached # type: ignore[import-untyped]
from aiocache.serializers import PickleSerializer # type: ignore[import-untyped]
@@ -259,6 +258,7 @@ async def _afetch_agent_card_impl(
def execute_a2a_delegation(
endpoint: str,
transport_protocol: Literal["JSONRPC", "GRPC", "HTTP+JSON"],
auth: AuthScheme | None,
timeout: int,
task_description: str,
@@ -282,6 +282,23 @@ def execute_a2a_delegation(
use aexecute_a2a_delegation directly.
Args:
endpoint: A2A agent endpoint URL (AgentCard URL)
transport_protocol: Optional A2A transport protocol (grpc, jsonrpc, http+json)
auth: Optional AuthScheme for authentication (Bearer, OAuth2, API Key, HTTP Basic/Digest)
timeout: Request timeout in seconds
task_description: The task to delegate
context: Optional context information
context_id: Context ID for correlating messages/tasks
task_id: Specific task identifier
reference_task_ids: List of related task IDs
metadata: Additional metadata (external_id, request_id, etc.)
extensions: Protocol extensions for custom fields
conversation_history: Previous Message objects from conversation
agent_id: Agent identifier for logging
agent_role: Role of the CrewAI agent delegating the task
agent_branch: Optional agent tree branch for logging
response_model: Optional Pydantic model for structured outputs
turn_number: Optional turn number for multi-turn conversations
endpoint: A2A agent endpoint URL.
auth: Optional AuthScheme for authentication.
timeout: Request timeout in seconds.
@@ -323,6 +340,7 @@ def execute_a2a_delegation(
agent_role=agent_role,
agent_branch=agent_branch,
response_model=response_model,
transport_protocol=transport_protocol,
turn_number=turn_number,
updates=updates,
)
@@ -333,6 +351,7 @@ def execute_a2a_delegation(
async def aexecute_a2a_delegation(
endpoint: str,
transport_protocol: Literal["JSONRPC", "GRPC", "HTTP+JSON"],
auth: AuthScheme | None,
timeout: int,
task_description: str,
@@ -356,6 +375,23 @@ async def aexecute_a2a_delegation(
in an async context (e.g., with Crew.akickoff() or agent.aexecute_task()).
Args:
endpoint: A2A agent endpoint URL
transport_protocol: Optional A2A transport protocol (grpc, jsonrpc, http+json)
auth: Optional AuthScheme for authentication
timeout: Request timeout in seconds
task_description: Task to delegate
context: Optional context
context_id: Context ID for correlation
task_id: Specific task identifier
reference_task_ids: Related task IDs
metadata: Additional metadata
extensions: Protocol extensions
conversation_history: Previous Message objects
turn_number: Current turn number
agent_branch: Agent tree branch for logging
agent_id: Agent identifier for logging
agent_role: Agent role for logging
response_model: Optional Pydantic model for structured outputs
endpoint: A2A agent endpoint URL.
auth: Optional AuthScheme for authentication.
timeout: Request timeout in seconds.
@@ -414,6 +450,7 @@ async def aexecute_a2a_delegation(
agent_role=agent_role,
response_model=response_model,
updates=updates,
transport_protocol=transport_protocol,
)
crewai_event_bus.emit(
@@ -431,6 +468,7 @@ async def aexecute_a2a_delegation(
async def _aexecute_a2a_delegation_impl(
endpoint: str,
transport_protocol: Literal["JSONRPC", "GRPC", "HTTP+JSON"],
auth: AuthScheme | None,
timeout: int,
task_description: str,
@@ -524,7 +562,6 @@ async def _aexecute_a2a_delegation_impl(
extensions=extensions,
)
transport_protocol = TransportProtocol("JSONRPC")
new_messages: list[Message] = [*conversation_history, message]
crewai_event_bus.emit(
None,
@@ -596,7 +633,7 @@ async def _aexecute_a2a_delegation_impl(
@asynccontextmanager
async def _create_a2a_client(
agent_card: AgentCard,
transport_protocol: TransportProtocol,
transport_protocol: Literal["JSONRPC", "GRPC", "HTTP+JSON"],
timeout: int,
headers: MutableMapping[str, str],
streaming: bool,
@@ -640,7 +677,7 @@ async def _create_a2a_client(
config = ClientConfig(
httpx_client=httpx_client,
supported_transports=[str(transport_protocol.value)],
supported_transports=[transport_protocol],
streaming=streaming and not use_polling,
polling=use_polling,
accepted_output_modes=["application/json"],

View File

@@ -771,6 +771,7 @@ def _delegate_to_a2a(
response_model=agent_config.response_model,
turn_number=turn_num + 1,
updates=agent_config.updates,
transport_protocol=agent_config.transport_protocol,
)
conversation_history = a2a_result.get("history", [])
@@ -1085,6 +1086,7 @@ async def _adelegate_to_a2a(
agent_branch=agent_branch,
response_model=agent_config.response_model,
turn_number=turn_num + 1,
transport_protocol=agent_config.transport_protocol,
updates=agent_config.updates,
)

View File

@@ -209,10 +209,9 @@ class EventListener(BaseEventListener):
@crewai_event_bus.on(TaskCompletedEvent)
def on_task_completed(source: Any, event: TaskCompletedEvent) -> None:
# Handle telemetry
span = self.execution_spans.get(source)
span = self.execution_spans.pop(source, None)
if span:
self._telemetry.task_ended(span, source, source.agent.crew)
self.execution_spans[source] = None
# Pass task name if it exists
task_name = get_task_name(source)
@@ -222,11 +221,10 @@ class EventListener(BaseEventListener):
@crewai_event_bus.on(TaskFailedEvent)
def on_task_failed(source: Any, event: TaskFailedEvent) -> None:
span = self.execution_spans.get(source)
span = self.execution_spans.pop(source, None)
if span:
if source.agent and source.agent.crew:
self._telemetry.task_ended(span, source, source.agent.crew)
self.execution_spans[source] = None
# Pass task name if it exists
task_name = get_task_name(source)

View File

@@ -1,4 +1,5 @@
import inspect
from typing import Any
from pydantic import BaseModel, Field, InstanceOf, model_validator
from typing_extensions import Self
@@ -14,14 +15,14 @@ class FlowTrackable(BaseModel):
inspecting the call stack.
"""
parent_flow: InstanceOf[Flow] | None = Field(
parent_flow: InstanceOf[Flow[Any]] | None = Field(
default=None,
description="The parent flow of the instance, if it was created inside a flow.",
)
@model_validator(mode="after")
def _set_parent_flow(self) -> Self:
max_depth = 5
max_depth = 8
frame = inspect.currentframe()
try:

View File

@@ -443,7 +443,7 @@ class AzureCompletion(BaseLLM):
params["presence_penalty"] = self.presence_penalty
if self.max_tokens is not None:
params["max_tokens"] = self.max_tokens
if self.stop:
if self.stop and self.supports_stop_words():
params["stop"] = self.stop
# Handle tools/functions for Azure OpenAI models
@@ -931,8 +931,28 @@ class AzureCompletion(BaseLLM):
return self.is_openai_model
def supports_stop_words(self) -> bool:
"""Check if the model supports stop words."""
return True # Most Azure models support stop sequences
"""Check if the model supports stop words.
Models using the Responses API (GPT-5 family, o-series reasoning models,
computer-use-preview) do not support stop sequences.
See: https://learn.microsoft.com/en-us/azure/ai-foundry/foundry-models/concepts/models-sold-directly-by-azure
"""
model_lower = self.model.lower() if self.model else ""
if "gpt-5" in model_lower:
return False
o_series_models = ["o1", "o3", "o4", "o1-mini", "o3-mini", "o4-mini"]
responses_api_models = ["computer-use-preview"]
unsupported_stop_models = o_series_models + responses_api_models
for unsupported in unsupported_stop_models:
if unsupported in model_lower:
return False
return True
def get_context_window_size(self) -> int:
"""Get the context window size for the model."""

View File

@@ -229,6 +229,48 @@ def enforce_rpm_limit(
request_within_rpm_limit()
def _extract_tools_from_context(
executor_context: CrewAgentExecutor | LiteAgent | None,
) -> list[dict[str, Any]] | None:
"""Extract tools from executor context and convert to LLM-compatible format.
Args:
executor_context: The executor context containing tools.
Returns:
List of tool dictionaries in LLM-compatible format, or None if no tools.
"""
if executor_context is None:
return None
# Get tools from executor context
# CrewAgentExecutor has 'tools' attribute, LiteAgent has '_parsed_tools'
tools: list[CrewStructuredTool] | None = None
if hasattr(executor_context, "tools"):
context_tools = executor_context.tools
if isinstance(context_tools, list) and len(context_tools) > 0:
tools = context_tools
if tools is None and hasattr(executor_context, "_parsed_tools"):
parsed_tools = executor_context._parsed_tools
if isinstance(parsed_tools, list) and len(parsed_tools) > 0:
tools = parsed_tools
if not tools:
return None
# Convert CrewStructuredTool to dict format expected by LLM
tool_dicts: list[dict[str, Any]] = []
for tool in tools:
tool_dict: dict[str, Any] = {
"name": tool.name,
"description": tool.description,
"args_schema": tool.args_schema,
}
tool_dicts.append(tool_dict)
return tool_dicts if tool_dicts else None
def get_llm_response(
llm: LLM | BaseLLM,
messages: list[LLMMessage],
@@ -264,14 +306,29 @@ def get_llm_response(
raise ValueError("LLM call blocked by before_llm_call hook")
messages = executor_context.messages
# Extract tools from executor context for native function calling support
tools = _extract_tools_from_context(executor_context)
try:
answer = llm.call(
messages,
callbacks=callbacks,
from_task=from_task,
from_agent=from_agent, # type: ignore[arg-type]
response_model=response_model,
)
# Only pass tools parameter if tools are available to maintain backward compatibility
# with code that checks "tools" in kwargs
if tools is not None:
answer = llm.call(
messages,
tools=tools,
callbacks=callbacks,
from_task=from_task,
from_agent=from_agent, # type: ignore[arg-type]
response_model=response_model,
)
else:
answer = llm.call(
messages,
callbacks=callbacks,
from_task=from_task,
from_agent=from_agent, # type: ignore[arg-type]
response_model=response_model,
)
except Exception as e:
raise e
if not answer:
@@ -292,7 +349,7 @@ async def aget_llm_response(
from_task: Task | None = None,
from_agent: Agent | LiteAgent | None = None,
response_model: type[BaseModel] | None = None,
executor_context: CrewAgentExecutor | None = None,
executor_context: CrewAgentExecutor | LiteAgent | None = None,
) -> str:
"""Call the LLM asynchronously and return the response.
@@ -318,14 +375,29 @@ async def aget_llm_response(
raise ValueError("LLM call blocked by before_llm_call hook")
messages = executor_context.messages
# Extract tools from executor context for native function calling support
tools = _extract_tools_from_context(executor_context)
try:
answer = await llm.acall(
messages,
callbacks=callbacks,
from_task=from_task,
from_agent=from_agent, # type: ignore[arg-type]
response_model=response_model,
)
# Only pass tools parameter if tools are available to maintain backward compatibility
# with code that checks "tools" in kwargs
if tools is not None:
answer = await llm.acall(
messages,
tools=tools,
callbacks=callbacks,
from_task=from_task,
from_agent=from_agent, # type: ignore[arg-type]
response_model=response_model,
)
else:
answer = await llm.acall(
messages,
callbacks=callbacks,
from_task=from_task,
from_agent=from_agent, # type: ignore[arg-type]
response_model=response_model,
)
except Exception as e:
raise e
if not answer:

View File

@@ -515,6 +515,94 @@ def test_azure_supports_stop_words():
assert llm.supports_stop_words() == True
def test_azure_gpt5_models_do_not_support_stop_words():
"""
Test that GPT-5 family models do not support stop words.
GPT-5 models use the Responses API which doesn't support stop sequences.
See: https://learn.microsoft.com/en-us/azure/ai-foundry/foundry-models/concepts/models-sold-directly-by-azure
"""
# GPT-5 base models
gpt5_models = [
"azure/gpt-5",
"azure/gpt-5-mini",
"azure/gpt-5-nano",
"azure/gpt-5-chat",
# GPT-5.1 series
"azure/gpt-5.1",
"azure/gpt-5.1-chat",
"azure/gpt-5.1-codex",
"azure/gpt-5.1-codex-mini",
# GPT-5.2 series
"azure/gpt-5.2",
"azure/gpt-5.2-chat",
]
for model_name in gpt5_models:
llm = LLM(model=model_name)
assert llm.supports_stop_words() == False, f"Expected {model_name} to NOT support stop words"
def test_azure_o_series_models_do_not_support_stop_words():
"""
Test that o-series reasoning models do not support stop words.
"""
o_series_models = [
"azure/o1",
"azure/o1-mini",
"azure/o3",
"azure/o3-mini",
"azure/o4",
"azure/o4-mini",
]
for model_name in o_series_models:
llm = LLM(model=model_name)
assert llm.supports_stop_words() == False, f"Expected {model_name} to NOT support stop words"
def test_azure_responses_api_models_do_not_support_stop_words():
"""
Test that models using the Responses API do not support stop words.
"""
responses_api_models = [
"azure/computer-use-preview",
]
for model_name in responses_api_models:
llm = LLM(model=model_name)
assert llm.supports_stop_words() == False, f"Expected {model_name} to NOT support stop words"
def test_azure_stop_words_not_included_for_unsupported_models():
"""
Test that stop words are not included in completion params for models that don't support them.
"""
with patch.dict(os.environ, {
"AZURE_API_KEY": "test-key",
"AZURE_ENDPOINT": "https://models.inference.ai.azure.com"
}):
# Test GPT-5 model - stop should NOT be included even if set
llm_gpt5 = LLM(
model="azure/gpt-5-nano",
stop=["STOP", "END"]
)
params = llm_gpt5._prepare_completion_params(
messages=[{"role": "user", "content": "test"}]
)
assert "stop" not in params, "stop should not be included for GPT-5 models"
# Test regular model - stop SHOULD be included
llm_gpt4 = LLM(
model="azure/gpt-4",
stop=["STOP", "END"]
)
params = llm_gpt4._prepare_completion_params(
messages=[{"role": "user", "content": "test"}]
)
assert "stop" in params, "stop should be included for GPT-4 models"
assert params["stop"] == ["STOP", "END"]
def test_azure_context_window_size():
"""
Test that Azure models return correct context window sizes

View File

@@ -4500,6 +4500,71 @@ def test_crew_copy_with_memory():
pytest.fail(f"Copying crew raised an unexpected exception: {e}")
def test_sets_parent_flow_when_using_crewbase_pattern_inside_flow():
@CrewBase
class TestCrew:
agents_config = None
tasks_config = None
agents: list[BaseAgent]
tasks: list[Task]
@agent
def researcher(self) -> Agent:
return Agent(
role="Researcher",
goal="Research things",
backstory="Expert researcher",
)
@agent
def writer_agent(self) -> Agent:
return Agent(
role="Writer",
goal="Write things",
backstory="Expert writer",
)
@task
def research_task(self) -> Task:
return Task(
description="Test task for researcher",
expected_output="output",
agent=self.researcher(),
)
@task
def write_task(self) -> Task:
return Task(
description="Test task for writer",
expected_output="output",
agent=self.writer_agent(),
)
@crew
def crew(self) -> Crew:
return Crew(
agents=self.agents,
tasks=self.tasks,
process=Process.sequential,
)
captured_crew = None
class MyFlow(Flow):
@start()
def start_method(self):
nonlocal captured_crew
captured_crew = TestCrew().crew()
return captured_crew
flow = MyFlow()
flow.kickoff()
assert captured_crew is not None
assert captured_crew.parent_flow is flow
def test_sets_parent_flow_when_outside_flow(researcher, writer):
crew = Crew(
agents=[researcher, writer],

View File

@@ -0,0 +1,457 @@
"""Unit tests for agent_utils module.
Tests the utility functions for agent execution including tool extraction
and LLM response handling.
"""
from unittest.mock import AsyncMock, MagicMock, Mock, patch
import pytest
from pydantic import BaseModel, Field
from crewai.tools.structured_tool import CrewStructuredTool
from crewai.utilities.agent_utils import (
_extract_tools_from_context,
aget_llm_response,
get_llm_response,
)
from crewai.utilities.printer import Printer
class MockArgsSchema(BaseModel):
"""Mock args schema for testing."""
query: str = Field(description="The search query")
class TestExtractToolsFromContext:
"""Test _extract_tools_from_context function."""
def test_returns_none_when_context_is_none(self):
"""Test that None is returned when executor_context is None."""
result = _extract_tools_from_context(None)
assert result is None
def test_returns_none_when_no_tools_attribute(self):
"""Test that None is returned when context has no tools."""
mock_context = Mock(spec=[])
result = _extract_tools_from_context(mock_context)
assert result is None
def test_returns_none_when_tools_is_empty(self):
"""Test that None is returned when tools list is empty."""
mock_context = Mock()
mock_context.tools = []
result = _extract_tools_from_context(mock_context)
assert result is None
def test_extracts_tools_from_crew_agent_executor(self):
"""Test tool extraction from CrewAgentExecutor (has 'tools' attribute)."""
mock_tool = CrewStructuredTool(
name="search_tool",
description="A tool for searching",
args_schema=MockArgsSchema,
func=lambda query: f"Results for {query}",
)
mock_context = Mock()
mock_context.tools = [mock_tool]
result = _extract_tools_from_context(mock_context)
assert result is not None
assert len(result) == 1
assert result[0]["name"] == "search_tool"
assert result[0]["description"] == "A tool for searching"
assert result[0]["args_schema"] == MockArgsSchema
def test_extracts_tools_from_lite_agent(self):
"""Test tool extraction from LiteAgent (has '_parsed_tools' attribute)."""
mock_tool = CrewStructuredTool(
name="calculator_tool",
description="A tool for calculations",
args_schema=MockArgsSchema,
func=lambda query: f"Calculated {query}",
)
mock_context = Mock(spec=["_parsed_tools"])
mock_context._parsed_tools = [mock_tool]
result = _extract_tools_from_context(mock_context)
assert result is not None
assert len(result) == 1
assert result[0]["name"] == "calculator_tool"
assert result[0]["description"] == "A tool for calculations"
assert result[0]["args_schema"] == MockArgsSchema
def test_extracts_multiple_tools(self):
"""Test extraction of multiple tools."""
tool1 = CrewStructuredTool(
name="tool1",
description="First tool",
args_schema=MockArgsSchema,
func=lambda query: "result1",
)
tool2 = CrewStructuredTool(
name="tool2",
description="Second tool",
args_schema=MockArgsSchema,
func=lambda query: "result2",
)
mock_context = Mock()
mock_context.tools = [tool1, tool2]
result = _extract_tools_from_context(mock_context)
assert result is not None
assert len(result) == 2
assert result[0]["name"] == "tool1"
assert result[1]["name"] == "tool2"
def test_prefers_tools_over_parsed_tools(self):
"""Test that 'tools' attribute is preferred over '_parsed_tools'."""
tool_from_tools = CrewStructuredTool(
name="from_tools",
description="Tool from tools attribute",
args_schema=MockArgsSchema,
func=lambda query: "from_tools",
)
tool_from_parsed = CrewStructuredTool(
name="from_parsed",
description="Tool from _parsed_tools attribute",
args_schema=MockArgsSchema,
func=lambda query: "from_parsed",
)
mock_context = Mock()
mock_context.tools = [tool_from_tools]
mock_context._parsed_tools = [tool_from_parsed]
result = _extract_tools_from_context(mock_context)
assert result is not None
assert len(result) == 1
assert result[0]["name"] == "from_tools"
class TestGetLlmResponse:
"""Test get_llm_response function."""
@pytest.fixture
def mock_llm(self):
"""Create a mock LLM."""
llm = Mock()
llm.call = Mock(return_value="LLM response")
return llm
@pytest.fixture
def mock_printer(self):
"""Create a mock printer."""
return Mock(spec=Printer)
def test_passes_tools_to_llm_call(self, mock_llm, mock_printer):
"""Test that tools are extracted and passed to llm.call()."""
mock_tool = CrewStructuredTool(
name="test_tool",
description="A test tool",
args_schema=MockArgsSchema,
func=lambda query: "result",
)
mock_context = Mock()
mock_context.tools = [mock_tool]
mock_context.messages = [{"role": "user", "content": "test"}]
mock_context.before_llm_call_hooks = []
mock_context.after_llm_call_hooks = []
with patch(
"crewai.utilities.agent_utils._setup_before_llm_call_hooks",
return_value=True,
):
with patch(
"crewai.utilities.agent_utils._setup_after_llm_call_hooks",
return_value="LLM response",
):
result = get_llm_response(
llm=mock_llm,
messages=[{"role": "user", "content": "test"}],
callbacks=[],
printer=mock_printer,
executor_context=mock_context,
)
# Verify llm.call was called with tools parameter
mock_llm.call.assert_called_once()
call_kwargs = mock_llm.call.call_args[1]
assert "tools" in call_kwargs
assert call_kwargs["tools"] is not None
assert len(call_kwargs["tools"]) == 1
assert call_kwargs["tools"][0]["name"] == "test_tool"
def test_does_not_pass_tools_when_no_context(self, mock_llm, mock_printer):
"""Test that tools parameter is not passed when no executor_context."""
result = get_llm_response(
llm=mock_llm,
messages=[{"role": "user", "content": "test"}],
callbacks=[],
printer=mock_printer,
executor_context=None,
)
mock_llm.call.assert_called_once()
call_kwargs = mock_llm.call.call_args[1]
# tools should NOT be in kwargs when there are no tools
# This maintains backward compatibility with code that checks "tools" in kwargs
assert "tools" not in call_kwargs
def test_does_not_pass_tools_when_context_has_no_tools(
self, mock_llm, mock_printer
):
"""Test that tools parameter is not passed when context has no tools."""
mock_context = Mock()
mock_context.tools = []
mock_context.messages = [{"role": "user", "content": "test"}]
mock_context.before_llm_call_hooks = []
mock_context.after_llm_call_hooks = []
with patch(
"crewai.utilities.agent_utils._setup_before_llm_call_hooks",
return_value=True,
):
with patch(
"crewai.utilities.agent_utils._setup_after_llm_call_hooks",
return_value="LLM response",
):
result = get_llm_response(
llm=mock_llm,
messages=[{"role": "user", "content": "test"}],
callbacks=[],
printer=mock_printer,
executor_context=mock_context,
)
mock_llm.call.assert_called_once()
call_kwargs = mock_llm.call.call_args[1]
# tools should NOT be in kwargs when there are no tools
# This maintains backward compatibility with code that checks "tools" in kwargs
assert "tools" not in call_kwargs
class TestAgetLlmResponse:
"""Test aget_llm_response async function."""
@pytest.fixture
def mock_llm(self):
"""Create a mock LLM with async call."""
llm = Mock()
llm.acall = AsyncMock(return_value="Async LLM response")
return llm
@pytest.fixture
def mock_printer(self):
"""Create a mock printer."""
return Mock(spec=Printer)
@pytest.mark.asyncio
async def test_passes_tools_to_llm_acall(self, mock_llm, mock_printer):
"""Test that tools are extracted and passed to llm.acall()."""
mock_tool = CrewStructuredTool(
name="async_test_tool",
description="An async test tool",
args_schema=MockArgsSchema,
func=lambda query: "async result",
)
mock_context = Mock()
mock_context.tools = [mock_tool]
mock_context.messages = [{"role": "user", "content": "async test"}]
mock_context.before_llm_call_hooks = []
mock_context.after_llm_call_hooks = []
with patch(
"crewai.utilities.agent_utils._setup_before_llm_call_hooks",
return_value=True,
):
with patch(
"crewai.utilities.agent_utils._setup_after_llm_call_hooks",
return_value="Async LLM response",
):
result = await aget_llm_response(
llm=mock_llm,
messages=[{"role": "user", "content": "async test"}],
callbacks=[],
printer=mock_printer,
executor_context=mock_context,
)
# Verify llm.acall was called with tools parameter
mock_llm.acall.assert_called_once()
call_kwargs = mock_llm.acall.call_args[1]
assert "tools" in call_kwargs
assert call_kwargs["tools"] is not None
assert len(call_kwargs["tools"]) == 1
assert call_kwargs["tools"][0]["name"] == "async_test_tool"
@pytest.mark.asyncio
async def test_does_not_pass_tools_when_no_context(self, mock_llm, mock_printer):
"""Test that tools parameter is not passed when no executor_context."""
result = await aget_llm_response(
llm=mock_llm,
messages=[{"role": "user", "content": "test"}],
callbacks=[],
printer=mock_printer,
executor_context=None,
)
mock_llm.acall.assert_called_once()
call_kwargs = mock_llm.acall.call_args[1]
# tools should NOT be in kwargs when there are no tools
# This maintains backward compatibility with code that checks "tools" in kwargs
assert "tools" not in call_kwargs
class TestToolsPassedToGeminiModels:
"""Test that tools are properly passed for Gemini models.
This test class specifically addresses GitHub issue #4238 where
Gemini models fail with UNEXPECTED_TOOL_CALL errors because tools
were not being passed to llm.call().
"""
@pytest.fixture
def mock_gemini_llm(self):
"""Create a mock Gemini LLM."""
llm = Mock()
llm.model = "gemini/gemini-2.0-flash-exp"
llm.call = Mock(return_value="Gemini response with tool call")
return llm
@pytest.fixture
def mock_printer(self):
"""Create a mock printer."""
return Mock(spec=Printer)
@pytest.fixture
def delegation_tools(self):
"""Create mock delegation tools similar to hierarchical crew setup."""
class DelegateWorkArgsSchema(BaseModel):
task: str = Field(description="The task to delegate")
context: str = Field(description="Context for the task")
coworker: str = Field(description="The coworker to delegate to")
class AskQuestionArgsSchema(BaseModel):
question: str = Field(description="The question to ask")
context: str = Field(description="Context for the question")
coworker: str = Field(description="The coworker to ask")
delegate_tool = CrewStructuredTool(
name="Delegate work to coworker",
description="Delegate a specific task to one of your coworkers",
args_schema=DelegateWorkArgsSchema,
func=lambda task, context, coworker: f"Delegated {task} to {coworker}",
)
ask_question_tool = CrewStructuredTool(
name="Ask question to coworker",
description="Ask a specific question to one of your coworkers",
args_schema=AskQuestionArgsSchema,
func=lambda question, context, coworker: f"Asked {question} to {coworker}",
)
return [delegate_tool, ask_question_tool]
def test_gemini_receives_tools_for_hierarchical_crew(
self, mock_gemini_llm, mock_printer, delegation_tools
):
"""Test that Gemini models receive tools when used in hierarchical crew.
This test verifies the fix for issue #4238 where the manager agent
in a hierarchical crew would fail because tools weren't passed to
the Gemini model, causing UNEXPECTED_TOOL_CALL errors.
"""
mock_context = Mock()
mock_context.tools = delegation_tools
mock_context.messages = [
{"role": "system", "content": "You are a manager agent"},
{"role": "user", "content": "Coordinate the team to answer this question"},
]
mock_context.before_llm_call_hooks = []
mock_context.after_llm_call_hooks = []
with patch(
"crewai.utilities.agent_utils._setup_before_llm_call_hooks",
return_value=True,
):
with patch(
"crewai.utilities.agent_utils._setup_after_llm_call_hooks",
return_value="Gemini response with tool call",
):
result = get_llm_response(
llm=mock_gemini_llm,
messages=mock_context.messages,
callbacks=[],
printer=mock_printer,
executor_context=mock_context,
)
# Verify that tools were passed to the Gemini model
mock_gemini_llm.call.assert_called_once()
call_kwargs = mock_gemini_llm.call.call_args[1]
assert "tools" in call_kwargs
assert call_kwargs["tools"] is not None
assert len(call_kwargs["tools"]) == 2
# Verify the delegation tools are properly formatted
tool_names = [t["name"] for t in call_kwargs["tools"]]
assert "Delegate work to coworker" in tool_names
assert "Ask question to coworker" in tool_names
# Verify each tool has the required fields
for tool_dict in call_kwargs["tools"]:
assert "name" in tool_dict
assert "description" in tool_dict
assert "args_schema" in tool_dict
def test_tool_dict_format_compatible_with_llm_providers(
self, mock_gemini_llm, mock_printer, delegation_tools
):
"""Test that extracted tools are in a format compatible with LLM providers.
The tool dictionaries should have 'name', 'description', and 'args_schema'
fields that can be processed by the LLM's _prepare_completion_params method.
"""
mock_context = Mock()
mock_context.tools = delegation_tools
mock_context.messages = [{"role": "user", "content": "test"}]
mock_context.before_llm_call_hooks = []
mock_context.after_llm_call_hooks = []
with patch(
"crewai.utilities.agent_utils._setup_before_llm_call_hooks",
return_value=True,
):
with patch(
"crewai.utilities.agent_utils._setup_after_llm_call_hooks",
return_value="response",
):
get_llm_response(
llm=mock_gemini_llm,
messages=mock_context.messages,
callbacks=[],
printer=mock_printer,
executor_context=mock_context,
)
call_kwargs = mock_gemini_llm.call.call_args[1]
tools = call_kwargs["tools"]
for tool_dict in tools:
# Verify the format matches what extract_tool_info() in common.py expects
assert isinstance(tool_dict["name"], str)
assert isinstance(tool_dict["description"], str)
# args_schema should be a Pydantic model class
assert hasattr(tool_dict["args_schema"], "model_json_schema")