Compare commits

...

20 Commits

Author SHA1 Message Date
Greyson LaLonde
2cd5a30873 Merge branch 'main' into gl/chore/use-base-model-for-llms 2025-11-13 14:08:29 -05:00
Greyson LaLonde
d7bdac12a2 feat: a2a trust remote completion status flag
Some checks failed
Notify Downstream / notify-downstream (push) Has been cancelled
Update Test Durations / update-durations (3.10) (push) Has been cancelled
Update Test Durations / update-durations (3.11) (push) Has been cancelled
Update Test Durations / update-durations (3.12) (push) Has been cancelled
Update Test Durations / update-durations (3.13) (push) Has been cancelled
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
Build uv cache / build-cache (3.10) (push) Has been cancelled
Build uv cache / build-cache (3.11) (push) Has been cancelled
Build uv cache / build-cache (3.12) (push) Has been cancelled
Build uv cache / build-cache (3.13) (push) Has been cancelled
- add trust_remote_completion_status flag to A2AConfig, Adds configuration flag to control whether to trust A2A agent completion status. Resolves #3899
- update docs
2025-11-13 13:43:09 -05:00
Lorenze Jay
528d812263 Lorenze/feat hooks (#3902)
* feat: implement LLM call hooks and enhance agent execution context

- Introduced LLM call hooks to allow modification of messages and responses during LLM interactions.
- Added support for before and after hooks in the CrewAgentExecutor, enabling dynamic adjustments to the execution flow.
- Created LLMCallHookContext for comprehensive access to the executor state, facilitating in-place modifications.
- Added validation for hook callables to ensure proper functionality.
- Enhanced tests for LLM hooks and tool hooks to verify their behavior and error handling capabilities.
- Updated LiteAgent and CrewAgentExecutor to accommodate the new crew context in their execution processes.

* feat: implement LLM call hooks and enhance agent execution context

- Introduced LLM call hooks to allow modification of messages and responses during LLM interactions.
- Added support for before and after hooks in the CrewAgentExecutor, enabling dynamic adjustments to the execution flow.
- Created LLMCallHookContext for comprehensive access to the executor state, facilitating in-place modifications.
- Added validation for hook callables to ensure proper functionality.
- Enhanced tests for LLM hooks and tool hooks to verify their behavior and error handling capabilities.
- Updated LiteAgent and CrewAgentExecutor to accommodate the new crew context in their execution processes.

* fix verbose

* feat: introduce crew-scoped hook decorators and refactor hook registration

- Added decorators for before and after LLM and tool calls to enhance flexibility in modifying execution behavior.
- Implemented a centralized hook registration mechanism within CrewBase to automatically register crew-scoped hooks.
- Removed the obsolete base.py file as its functionality has been integrated into the new decorators and registration system.
- Enhanced tests for the new hook decorators to ensure proper registration and execution flow.
- Updated existing hook handling to accommodate the new decorator-based approach, improving code organization and maintainability.

* feat: enhance hook management with clear and unregister functions

- Introduced functions to unregister specific before and after hooks for both LLM and tool calls, improving flexibility in hook management.
- Added clear functions to remove all registered hooks of each type, facilitating easier state management and cleanup.
- Implemented a convenience function to clear all global hooks in one call, streamlining the process for testing and execution context resets.
- Enhanced tests to verify the functionality of unregistering and clearing hooks, ensuring robust behavior in various scenarios.

* refactor: enhance hook type management for LLM and tool hooks

- Updated hook type definitions to use generic protocols for better type safety and flexibility.
- Replaced Callable type annotations with specific BeforeLLMCallHookType and AfterLLMCallHookType for clarity.
- Improved the registration and retrieval functions for before and after hooks to align with the new type definitions.
- Enhanced the setup functions to handle hook execution results, allowing for blocking of LLM calls based on hook logic.
- Updated related tests to ensure proper functionality and type adherence across the hook management system.

* feat: add execution and tool hooks documentation

- Introduced new documentation for execution hooks, LLM call hooks, and tool call hooks to provide comprehensive guidance on their usage and implementation in CrewAI.
- Updated existing documentation to include references to the new hooks, enhancing the learning resources available for users.
- Ensured consistency across multiple languages (English, Portuguese, Korean) for the new documentation, improving accessibility for a wider audience.
- Added examples and troubleshooting sections to assist users in effectively utilizing hooks for agent operations.

---------

Co-authored-by: Greyson LaLonde <greyson.r.lalonde@gmail.com>
2025-11-13 10:11:50 -08:00
Greyson LaLonde
ffd717c51a fix: custom tool docs links, add mintlify broken links action (#3903)
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
Notify Downstream / notify-downstream (push) Has been cancelled
* fix: update docs links to point to correct endpoints

* fix: update all broken doc links
2025-11-12 22:55:10 -08:00
Greyson LaLonde
02580f58d1 Merge branch 'main' into gl/chore/use-base-model-for-llms 2025-11-12 21:49:40 -05:00
Heitor Carvalho
fbe4aa4bd1 feat: fetch and store more data about okta authorization server (#3894)
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Notify Downstream / notify-downstream (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
2025-11-12 15:28:00 -03:00
Lorenze Jay
c205d2e8de feat: implement before and after LLM call hooks in CrewAgentExecutor (#3893)
- Added support for before and after LLM call hooks to allow modification of messages and responses during LLM interactions.
- Introduced LLMCallHookContext to provide hooks with access to the executor state, enabling in-place modifications of messages.
- Updated get_llm_response function to utilize the new hooks, ensuring that modifications persist across iterations.
- Enhanced tests to verify the functionality of the hooks and their error handling capabilities, ensuring robust execution flow.
2025-11-12 08:38:13 -08:00
Greyson LaLonde
8b83bf3e54 chore: remove duplication in azure client 2025-11-12 00:48:01 -05:00
Greyson LaLonde
93f1fbd75e chore: move api key validation to base 2025-11-11 17:46:26 -05:00
Greyson LaLonde
0803318002 chore: improve typing 2025-11-11 17:37:08 -05:00
Daniel Barreto
fcb5b19b2e Enhance schema description of QdrantVectorSearchTool (#3891)
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Notify Downstream / notify-downstream (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
2025-11-11 14:33:33 -08:00
Greyson LaLonde
6fb13ee3e0 chore: fix attr ref 2025-11-11 17:18:12 -05:00
Greyson LaLonde
67e39073c7 Merge branch 'gl/chore/use-base-model-for-llms' of https://github.com/crewAIInc/crewAI into gl/chore/use-base-model-for-llms 2025-11-10 23:51:56 -05:00
Greyson LaLonde
722d316824 chore: continue refactoring llms to base models 2025-11-10 23:49:50 -05:00
Greyson LaLonde
a824d52e5e Merge branch 'main' into gl/chore/use-base-model-for-llms 2025-11-10 23:40:41 -05:00
Greyson LaLonde
d8fe83f76c chore: continue refactoring llms to base models 2025-11-10 23:38:03 -05:00
Rip&Tear
01f0111d52 dependabot.yml creation (#3868)
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Notify Downstream / notify-downstream (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
* dependabot.yml creation

* Configure dependabot for pip package updates

Co-authored-by: matt <matt@crewai.com>

* Fix Dependabot package ecosystem

* Refactor: Use uv package-ecosystem in dependabot

Co-authored-by: matt <matt@crewai.com>

* fix: ensure dependabot uses uv ecosystem

---------

Co-authored-by: Greyson LaLonde <greyson.r.lalonde@gmail.com>
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: matt <matt@crewai.com>
2025-11-11 12:14:16 +08:00
Lorenze Jay
6b52587c67 feat: expose messages to TaskOutput and LiteAgentOutputs (#3880)
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Notify Downstream / notify-downstream (push) Has been cancelled
* feat: add messages to task and agent outputs

- Introduced a new  field in  and  to capture messages from the last task execution.
- Updated the  class to store the last messages and provide a property for easy access.
- Enhanced the  and  classes to include messages in their outputs.
- Added tests to ensure that messages are correctly included in task outputs and agent outputs during execution.

* using typing_extensions for 3.10 compatability

* feat: add last_messages attribute to agent for improved task tracking

- Introduced a new `last_messages` attribute in the agent class to store messages from the last task execution.
- Updated the `Crew` class to handle the new messages attribute in task outputs.
- Enhanced existing tests to ensure that the `last_messages` attribute is correctly initialized and utilized across various guardrail scenarios.

* fix: add messages field to TaskOutput in tests for consistency

- Updated multiple test cases to include the new `messages` field in the `TaskOutput` instances.
- Ensured that all relevant tests reflect the latest changes in the TaskOutput structure, maintaining consistency across the test suite.
- This change aligns with the recent addition of the `last_messages` attribute in the agent class for improved task tracking.

* feat: preserve messages in task outputs during replay

- Added functionality to the Crew class to store and retrieve messages in task outputs.
- Enhanced the replay mechanism to ensure that messages from stored task outputs are preserved and accessible.
- Introduced a new test case to verify that messages are correctly stored and replayed, ensuring consistency in task execution and output handling.
- This change improves the overall tracking and context retention of task interactions within the CrewAI framework.

* fix original test, prev was debugging
2025-11-10 17:38:30 -08:00
Lorenze Jay
629f7f34ce docs: enhance task guardrail documentation with LLM-based validation support (#3879)
- Added section on LLM-based guardrails, explaining their usage and requirements.
- Updated examples to demonstrate the implementation of multiple guardrails, including both function-based and LLM-based approaches.
- Clarified the distinction between single and multiple guardrails in task configurations.
- Improved explanations of guardrail functionality to ensure better understanding of validation processes.
2025-11-10 15:35:42 -08:00
Greyson LaLonde
46785adf58 chore: refactor llms to base models 2025-11-10 14:22:09 -05:00
158 changed files with 12073 additions and 2480 deletions

11
.github/dependabot.yml vendored Normal file
View File

@@ -0,0 +1,11 @@
# To get started with Dependabot version updates, you'll need to specify which
# package ecosystems to update and where the package manifests are located.
# Please see the documentation for all configuration options:
# https://docs.github.com/code-security/dependabot/dependabot-version-updates/configuration-options-for-the-dependabot.yml-file
version: 2
updates:
- package-ecosystem: uv # See documentation for possible values
directory: "/" # Location of package manifests
schedule:
interval: "weekly"

35
.github/workflows/docs-broken-links.yml vendored Normal file
View File

@@ -0,0 +1,35 @@
name: Check Documentation Broken Links
on:
pull_request:
paths:
- "docs/**"
- "docs.json"
push:
branches:
- main
paths:
- "docs/**"
- "docs.json"
workflow_dispatch:
jobs:
check-links:
name: Check broken links
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v4
- name: Set up Node
uses: actions/setup-node@v4
with:
node-version: "latest"
- name: Install Mintlify CLI
run: npm i -g mintlify
- name: Run broken link checker
run: |
# Auto-answer the prompt with yes command
yes "" | mintlify broken-links || test $? -eq 141
working-directory: ./docs

View File

@@ -313,7 +313,10 @@
"en/learn/multimodal-agents",
"en/learn/replay-tasks-from-latest-crew-kickoff",
"en/learn/sequential-process",
"en/learn/using-annotations"
"en/learn/using-annotations",
"en/learn/execution-hooks",
"en/learn/llm-hooks",
"en/learn/tool-hooks"
]
},
{
@@ -737,7 +740,10 @@
"pt-BR/learn/multimodal-agents",
"pt-BR/learn/replay-tasks-from-latest-crew-kickoff",
"pt-BR/learn/sequential-process",
"pt-BR/learn/using-annotations"
"pt-BR/learn/using-annotations",
"pt-BR/learn/execution-hooks",
"pt-BR/learn/llm-hooks",
"pt-BR/learn/tool-hooks"
]
},
{
@@ -1170,7 +1176,10 @@
"ko/learn/multimodal-agents",
"ko/learn/replay-tasks-from-latest-crew-kickoff",
"ko/learn/sequential-process",
"ko/learn/using-annotations"
"ko/learn/using-annotations",
"ko/learn/execution-hooks",
"ko/learn/llm-hooks",
"ko/learn/tool-hooks"
]
},
{

View File

@@ -739,7 +739,7 @@ class KnowledgeMonitorListener(BaseEventListener):
knowledge_monitor = KnowledgeMonitorListener()
```
For more information on using events, see the [Event Listeners](https://docs.crewai.com/concepts/event-listener) documentation.
For more information on using events, see the [Event Listeners](/en/concepts/event-listener) documentation.
### Custom Knowledge Sources

View File

@@ -1035,7 +1035,7 @@ CrewAI supports streaming responses from LLMs, allowing your application to rece
```
<Tip>
[Click here](https://docs.crewai.com/concepts/event-listener#event-listeners) for more details
[Click here](/en/concepts/event-listener#event-listeners) for more details
</Tip>
</Tab>
@@ -1212,7 +1212,7 @@ Learn how to get the most out of your LLM configuration:
```python
import httpx
from crewai import LLM
from crewai.llms.hooks import BaseInterceptor
from crewai.llm.hooks import BaseInterceptor
class CustomInterceptor(BaseInterceptor[httpx.Request, httpx.Response]):
"""Custom interceptor to modify requests and responses."""

View File

@@ -60,6 +60,7 @@ crew = Crew(
| **Output Pydantic** _(optional)_ | `output_pydantic` | `Optional[Type[BaseModel]]` | A Pydantic model for task output. |
| **Callback** _(optional)_ | `callback` | `Optional[Any]` | Function/object to be executed after task completion. |
| **Guardrail** _(optional)_ | `guardrail` | `Optional[Callable]` | Function to validate task output before proceeding to next task. |
| **Guardrails** _(optional)_ | `guardrails` | `Optional[List[Callable] | List[str]]` | List of guardrails to validate task output before proceeding to next task. |
| **Guardrail Max Retries** _(optional)_ | `guardrail_max_retries` | `Optional[int]` | Maximum number of retries when guardrail validation fails. Defaults to 3. |
<Note type="warning" title="Deprecated: max_retries">
@@ -223,6 +224,7 @@ By default, the `TaskOutput` will only include the `raw` output. A `TaskOutput`
| **JSON Dict** | `json_dict` | `Optional[Dict[str, Any]]` | A dictionary representing the JSON output of the task. |
| **Agent** | `agent` | `str` | The agent that executed the task. |
| **Output Format** | `output_format` | `OutputFormat` | The format of the task output, with options including RAW, JSON, and Pydantic. The default is RAW. |
| **Messages** | `messages` | `list[LLMMessage]` | The messages from the last task execution. |
### Task Methods and Properties
@@ -341,7 +343,11 @@ Task guardrails provide a way to validate and transform task outputs before they
are passed to the next task. This feature helps ensure data quality and provides
feedback to agents when their output doesn't meet specific criteria.
Guardrails are implemented as Python functions that contain custom validation logic, giving you complete control over the validation process and ensuring reliable, deterministic results.
CrewAI supports two types of guardrails:
1. **Function-based guardrails**: Python functions with custom validation logic, giving you complete control over the validation process and ensuring reliable, deterministic results.
2. **LLM-based guardrails**: String descriptions that use the agent's LLM to validate outputs based on natural language criteria. These are ideal for complex or subjective validation requirements.
### Function-Based Guardrails
@@ -355,12 +361,12 @@ def validate_blog_content(result: TaskOutput) -> Tuple[bool, Any]:
"""Validate blog content meets requirements."""
try:
# Check word count
word_count = len(result.split())
word_count = len(result.raw.split())
if word_count > 200:
return (False, "Blog content exceeds 200 words")
# Additional validation logic here
return (True, result.strip())
return (True, result.raw.strip())
except Exception as e:
return (False, "Unexpected error during validation")
@@ -372,6 +378,147 @@ blog_task = Task(
)
```
### LLM-Based Guardrails (String Descriptions)
Instead of writing custom validation functions, you can use string descriptions that leverage LLM-based validation. When you provide a string to the `guardrail` or `guardrails` parameter, CrewAI automatically creates an `LLMGuardrail` that uses the agent's LLM to validate the output based on your description.
**Requirements**:
- The task must have an `agent` assigned (the guardrail uses the agent's LLM)
- Provide a clear, descriptive string explaining the validation criteria
```python Code
from crewai import Task
# Single LLM-based guardrail
blog_task = Task(
description="Write a blog post about AI",
expected_output="A blog post under 200 words",
agent=blog_agent,
guardrail="The blog post must be under 200 words and contain no technical jargon"
)
```
LLM-based guardrails are particularly useful for:
- **Complex validation logic** that's difficult to express programmatically
- **Subjective criteria** like tone, style, or quality assessments
- **Natural language requirements** that are easier to describe than code
The LLM guardrail will:
1. Analyze the task output against your description
2. Return `(True, output)` if the output complies with the criteria
3. Return `(False, feedback)` with specific feedback if validation fails
**Example with detailed validation criteria**:
```python Code
research_task = Task(
description="Research the latest developments in quantum computing",
expected_output="A comprehensive research report",
agent=researcher_agent,
guardrail="""
The research report must:
- Be at least 1000 words long
- Include at least 5 credible sources
- Cover both technical and practical applications
- Be written in a professional, academic tone
- Avoid speculation or unverified claims
"""
)
```
### Multiple Guardrails
You can apply multiple guardrails to a task using the `guardrails` parameter. Multiple guardrails are executed sequentially, with each guardrail receiving the output from the previous one. This allows you to chain validation and transformation steps.
The `guardrails` parameter accepts:
- A list of guardrail functions or string descriptions
- A single guardrail function or string (same as `guardrail`)
**Note**: If `guardrails` is provided, it takes precedence over `guardrail`. The `guardrail` parameter will be ignored when `guardrails` is set.
```python Code
from typing import Tuple, Any
from crewai import TaskOutput, Task
def validate_word_count(result: TaskOutput) -> Tuple[bool, Any]:
"""Validate word count is within limits."""
word_count = len(result.raw.split())
if word_count < 100:
return (False, f"Content too short: {word_count} words. Need at least 100 words.")
if word_count > 500:
return (False, f"Content too long: {word_count} words. Maximum is 500 words.")
return (True, result.raw)
def validate_no_profanity(result: TaskOutput) -> Tuple[bool, Any]:
"""Check for inappropriate language."""
profanity_words = ["badword1", "badword2"] # Example list
content_lower = result.raw.lower()
for word in profanity_words:
if word in content_lower:
return (False, f"Inappropriate language detected: {word}")
return (True, result.raw)
def format_output(result: TaskOutput) -> Tuple[bool, Any]:
"""Format and clean the output."""
formatted = result.raw.strip()
# Capitalize first letter
formatted = formatted[0].upper() + formatted[1:] if formatted else formatted
return (True, formatted)
# Apply multiple guardrails sequentially
blog_task = Task(
description="Write a blog post about AI",
expected_output="A well-formatted blog post between 100-500 words",
agent=blog_agent,
guardrails=[
validate_word_count, # First: validate length
validate_no_profanity, # Second: check content
format_output # Third: format the result
],
guardrail_max_retries=3
)
```
In this example, the guardrails execute in order:
1. `validate_word_count` checks the word count
2. `validate_no_profanity` checks for inappropriate language (using the output from step 1)
3. `format_output` formats the final result (using the output from step 2)
If any guardrail fails, the error is sent back to the agent, and the task is retried up to `guardrail_max_retries` times.
**Mixing function-based and LLM-based guardrails**:
You can combine both function-based and string-based guardrails in the same list:
```python Code
from typing import Tuple, Any
from crewai import TaskOutput, Task
def validate_word_count(result: TaskOutput) -> Tuple[bool, Any]:
"""Validate word count is within limits."""
word_count = len(result.raw.split())
if word_count < 100:
return (False, f"Content too short: {word_count} words. Need at least 100 words.")
if word_count > 500:
return (False, f"Content too long: {word_count} words. Maximum is 500 words.")
return (True, result.raw)
# Mix function-based and LLM-based guardrails
blog_task = Task(
description="Write a blog post about AI",
expected_output="A well-formatted blog post between 100-500 words",
agent=blog_agent,
guardrails=[
validate_word_count, # Function-based: precise word count check
"The content must be engaging and suitable for a general audience", # LLM-based: subjective quality check
"The writing style should be clear, concise, and free of technical jargon" # LLM-based: style validation
],
guardrail_max_retries=3
)
```
This approach combines the precision of programmatic validation with the flexibility of LLM-based assessment for subjective criteria.
### Guardrail Function Requirements
1. **Function Signature**:

View File

@@ -37,7 +37,7 @@ you can use them locally or refine them to your needs.
<Card title="Tools & Integrations" href="/en/enterprise/features/tools-and-integrations" icon="wrench">
Connect external apps and manage internal tools your agents can use.
</Card>
<Card title="Tool Repository" href="/en/enterprise/features/tool-repository" icon="toolbox">
<Card title="Tool Repository" href="/en/enterprise/guides/tool-repository#tool-repository" icon="toolbox">
Publish and install tools to enhance your crews' capabilities.
</Card>
<Card title="Agents Repository" href="/en/enterprise/features/agent-repositories" icon="people-group">

View File

@@ -241,7 +241,7 @@ Tools & Integrations is the central hub for connecting thirdparty apps and ma
## Related
<CardGroup cols={2}>
<Card title="Tool Repository" href="/en/enterprise/features/tool-repository" icon="toolbox">
<Card title="Tool Repository" href="/en/enterprise/guides/tool-repository#tool-repository" icon="toolbox">
Create, publish, and version custom tools for your organization.
</Card>
<Card title="Webhook Automation" href="/en/enterprise/guides/webhook-automation" icon="bolt">

View File

@@ -21,7 +21,7 @@ The repository is not a version control system. Use Git to track code changes an
Before using the Tool Repository, ensure you have:
- A [CrewAI AMP](https://app.crewai.com) account
- [CrewAI CLI](https://docs.crewai.com/concepts/cli#cli) installed
- [CrewAI CLI](/en/concepts/cli#cli) installed
- uv>=0.5.0 installed. Check out [how to upgrade](https://docs.astral.sh/uv/getting-started/installation/#upgrading-uv)
- [Git](https://git-scm.com) installed and configured
- Access permissions to publish or install tools in your CrewAI AMP organization
@@ -112,7 +112,7 @@ By default, tools are published as private. To make a tool public:
crewai tool publish --public
```
For more details on how to build tools, see [Creating your own tools](https://docs.crewai.com/concepts/tools#creating-your-own-tools).
For more details on how to build tools, see [Creating your own tools](/en/concepts/tools#creating-your-own-tools).
## Updating Tools

View File

@@ -49,7 +49,7 @@ mode: "wide"
To integrate human input into agent execution, set the `human_input` flag in the task definition. When enabled, the agent prompts the user for input before delivering its final answer. This input can provide extra context, clarify ambiguities, or validate the agent's output.
For detailed implementation guidance, see our [Human-in-the-Loop guide](/en/how-to/human-in-the-loop).
For detailed implementation guidance, see our [Human-in-the-Loop guide](/en/enterprise/guides/human-in-the-loop).
</Accordion>
<Accordion title="What advanced customization options are available for tailoring and enhancing agent behavior and capabilities in CrewAI?">
@@ -142,7 +142,7 @@ mode: "wide"
<Accordion title="How can I create custom tools for my CrewAI agents?">
You can create custom tools by subclassing the `BaseTool` class provided by CrewAI or by using the tool decorator. Subclassing involves defining a new class that inherits from `BaseTool`, specifying the name, description, and the `_run` method for operational logic. The tool decorator allows you to create a `Tool` object directly with the required attributes and a functional logic.
<Card href="https://docs.crewai.com/how-to/create-custom-tools" icon="code">CrewAI Tools Guide</Card>
<Card href="/en/learn/create-custom-tools" icon="code">CrewAI Tools Guide</Card>
</Accordion>
<Accordion title="How can you control the maximum number of requests per minute that the entire crew can perform?">

View File

@@ -83,6 +83,10 @@ The `A2AConfig` class accepts the following parameters:
Whether to raise an error immediately if agent connection fails. When `False`, the agent continues with available agents and informs the LLM about unavailable ones.
</ParamField>
<ParamField path="trust_remote_completion_status" type="bool" default="False">
When `True`, returns the A2A agent's result directly when it signals completion. When `False`, allows the server agent to review the result and potentially continue the conversation.
</ParamField>
## Authentication
For A2A agents that require authentication, use one of the provided auth schemes:

View File

@@ -0,0 +1,522 @@
---
title: Execution Hooks Overview
description: Understanding and using execution hooks in CrewAI for fine-grained control over agent operations
mode: "wide"
---
Execution Hooks provide fine-grained control over the runtime behavior of your CrewAI agents. Unlike kickoff hooks that run before and after crew execution, execution hooks intercept specific operations during agent execution, allowing you to modify behavior, implement safety checks, and add comprehensive monitoring.
## Types of Execution Hooks
CrewAI provides two main categories of execution hooks:
### 1. [LLM Call Hooks](/learn/llm-hooks)
Control and monitor language model interactions:
- **Before LLM Call**: Modify prompts, validate inputs, implement approval gates
- **After LLM Call**: Transform responses, sanitize outputs, update conversation history
**Use Cases:**
- Iteration limiting
- Cost tracking and token usage monitoring
- Response sanitization and content filtering
- Human-in-the-loop approval for LLM calls
- Adding safety guidelines or context
- Debug logging and request/response inspection
[View LLM Hooks Documentation →](/learn/llm-hooks)
### 2. [Tool Call Hooks](/learn/tool-hooks)
Control and monitor tool execution:
- **Before Tool Call**: Modify inputs, validate parameters, block dangerous operations
- **After Tool Call**: Transform results, sanitize outputs, log execution details
**Use Cases:**
- Safety guardrails for destructive operations
- Human approval for sensitive actions
- Input validation and sanitization
- Result caching and rate limiting
- Tool usage analytics
- Debug logging and monitoring
[View Tool Hooks Documentation →](/learn/tool-hooks)
## Hook Registration Methods
### 1. Decorator-Based Hooks (Recommended)
The cleanest and most Pythonic way to register hooks:
```python
from crewai.hooks import before_llm_call, after_llm_call, before_tool_call, after_tool_call
@before_llm_call
def limit_iterations(context):
"""Prevent infinite loops by limiting iterations."""
if context.iterations > 10:
return False # Block execution
return None
@after_llm_call
def sanitize_response(context):
"""Remove sensitive data from LLM responses."""
if "API_KEY" in context.response:
return context.response.replace("API_KEY", "[REDACTED]")
return None
@before_tool_call
def block_dangerous_tools(context):
"""Block destructive operations."""
if context.tool_name == "delete_database":
return False # Block execution
return None
@after_tool_call
def log_tool_result(context):
"""Log tool execution."""
print(f"Tool {context.tool_name} completed")
return None
```
### 2. Crew-Scoped Hooks
Apply hooks only to specific crew instances:
```python
from crewai import CrewBase
from crewai.project import crew
from crewai.hooks import before_llm_call_crew, after_tool_call_crew
@CrewBase
class MyProjCrew:
@before_llm_call_crew
def validate_inputs(self, context):
# Only applies to this crew
print(f"LLM call in {self.__class__.__name__}")
return None
@after_tool_call_crew
def log_results(self, context):
# Crew-specific logging
print(f"Tool result: {context.tool_result[:50]}...")
return None
@crew
def crew(self) -> Crew:
return Crew(
agents=self.agents,
tasks=self.tasks,
process=Process.sequential
)
```
## Hook Execution Flow
### LLM Call Flow
```
Agent needs to call LLM
[Before LLM Call Hooks Execute]
├→ Hook 1: Validate iteration count
├→ Hook 2: Add safety context
└→ Hook 3: Log request
If any hook returns False:
├→ Block LLM call
└→ Raise ValueError
If all hooks return True/None:
├→ LLM call proceeds
└→ Response generated
[After LLM Call Hooks Execute]
├→ Hook 1: Sanitize response
├→ Hook 2: Log response
└→ Hook 3: Update metrics
Final response returned
```
### Tool Call Flow
```
Agent needs to execute tool
[Before Tool Call Hooks Execute]
├→ Hook 1: Check if tool is allowed
├→ Hook 2: Validate inputs
└→ Hook 3: Request approval if needed
If any hook returns False:
├→ Block tool execution
└→ Return error message
If all hooks return True/None:
├→ Tool execution proceeds
└→ Result generated
[After Tool Call Hooks Execute]
├→ Hook 1: Sanitize result
├→ Hook 2: Cache result
└→ Hook 3: Log metrics
Final result returned
```
## Hook Context Objects
### LLMCallHookContext
Provides access to LLM execution state:
```python
class LLMCallHookContext:
executor: CrewAgentExecutor # Full executor access
messages: list # Mutable message list
agent: Agent # Current agent
task: Task # Current task
crew: Crew # Crew instance
llm: BaseLLM # LLM instance
iterations: int # Current iteration
response: str | None # LLM response (after hooks)
```
### ToolCallHookContext
Provides access to tool execution state:
```python
class ToolCallHookContext:
tool_name: str # Tool being called
tool_input: dict # Mutable input parameters
tool: CrewStructuredTool # Tool instance
agent: Agent | None # Agent executing
task: Task | None # Current task
crew: Crew | None # Crew instance
tool_result: str | None # Tool result (after hooks)
```
## Common Patterns
### Safety and Validation
```python
@before_tool_call
def safety_check(context):
"""Block destructive operations."""
dangerous = ['delete_file', 'drop_table', 'system_shutdown']
if context.tool_name in dangerous:
print(f"🛑 Blocked: {context.tool_name}")
return False
return None
@before_llm_call
def iteration_limit(context):
"""Prevent infinite loops."""
if context.iterations > 15:
print("⛔ Maximum iterations exceeded")
return False
return None
```
### Human-in-the-Loop
```python
@before_tool_call
def require_approval(context):
"""Require approval for sensitive operations."""
sensitive = ['send_email', 'make_payment', 'post_message']
if context.tool_name in sensitive:
response = context.request_human_input(
prompt=f"Approve {context.tool_name}?",
default_message="Type 'yes' to approve:"
)
if response.lower() != 'yes':
return False
return None
```
### Monitoring and Analytics
```python
from collections import defaultdict
import time
metrics = defaultdict(lambda: {'count': 0, 'total_time': 0})
@before_tool_call
def start_timer(context):
context.tool_input['_start'] = time.time()
return None
@after_tool_call
def track_metrics(context):
start = context.tool_input.get('_start', time.time())
duration = time.time() - start
metrics[context.tool_name]['count'] += 1
metrics[context.tool_name]['total_time'] += duration
return None
# View metrics
def print_metrics():
for tool, data in metrics.items():
avg = data['total_time'] / data['count']
print(f"{tool}: {data['count']} calls, {avg:.2f}s avg")
```
### Response Sanitization
```python
import re
@after_llm_call
def sanitize_llm_response(context):
"""Remove sensitive data from LLM responses."""
if not context.response:
return None
result = context.response
result = re.sub(r'(api[_-]?key)["\']?\s*[:=]\s*["\']?[\w-]+',
r'\1: [REDACTED]', result, flags=re.IGNORECASE)
return result
@after_tool_call
def sanitize_tool_result(context):
"""Remove sensitive data from tool results."""
if not context.tool_result:
return None
result = context.tool_result
result = re.sub(r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b',
'[EMAIL-REDACTED]', result)
return result
```
## Hook Management
### Clearing All Hooks
```python
from crewai.hooks import clear_all_global_hooks
# Clear all hooks at once
result = clear_all_global_hooks()
print(f"Cleared {result['total']} hooks")
# Output: {'llm_hooks': (2, 1), 'tool_hooks': (1, 2), 'total': (3, 3)}
```
### Clearing Specific Hook Types
```python
from crewai.hooks import (
clear_before_llm_call_hooks,
clear_after_llm_call_hooks,
clear_before_tool_call_hooks,
clear_after_tool_call_hooks
)
# Clear specific types
llm_before_count = clear_before_llm_call_hooks()
tool_after_count = clear_after_tool_call_hooks()
```
### Unregistering Individual Hooks
```python
from crewai.hooks import (
unregister_before_llm_call_hook,
unregister_after_tool_call_hook
)
def my_hook(context):
...
# Register
register_before_llm_call_hook(my_hook)
# Later, unregister
success = unregister_before_llm_call_hook(my_hook)
print(f"Unregistered: {success}")
```
## Best Practices
### 1. Keep Hooks Focused
Each hook should have a single, clear responsibility:
```python
# ✅ Good - focused responsibility
@before_tool_call
def validate_file_path(context):
if context.tool_name == 'read_file':
if '..' in context.tool_input.get('path', ''):
return False
return None
# ❌ Bad - too many responsibilities
@before_tool_call
def do_everything(context):
# Validation + logging + metrics + approval...
...
```
### 2. Handle Errors Gracefully
```python
@before_llm_call
def safe_hook(context):
try:
# Your logic
if some_condition:
return False
except Exception as e:
print(f"Hook error: {e}")
return None # Allow execution despite error
```
### 3. Modify Context In-Place
```python
# ✅ Correct - modify in-place
@before_llm_call
def add_context(context):
context.messages.append({"role": "system", "content": "Be concise"})
# ❌ Wrong - replaces reference
@before_llm_call
def wrong_approach(context):
context.messages = [{"role": "system", "content": "Be concise"}]
```
### 4. Use Type Hints
```python
from crewai.hooks import LLMCallHookContext, ToolCallHookContext
def my_llm_hook(context: LLMCallHookContext) -> bool | None:
# IDE autocomplete and type checking
return None
def my_tool_hook(context: ToolCallHookContext) -> str | None:
return None
```
### 5. Clean Up in Tests
```python
import pytest
from crewai.hooks import clear_all_global_hooks
@pytest.fixture(autouse=True)
def clean_hooks():
"""Reset hooks before each test."""
yield
clear_all_global_hooks()
```
## When to Use Which Hook
### Use LLM Hooks When:
- Implementing iteration limits
- Adding context or safety guidelines to prompts
- Tracking token usage and costs
- Sanitizing or transforming responses
- Implementing approval gates for LLM calls
- Debugging prompt/response interactions
### Use Tool Hooks When:
- Blocking dangerous or destructive operations
- Validating tool inputs before execution
- Implementing approval gates for sensitive actions
- Caching tool results
- Tracking tool usage and performance
- Sanitizing tool outputs
- Rate limiting tool calls
### Use Both When:
Building comprehensive observability, safety, or approval systems that need to monitor all agent operations.
## Alternative Registration Methods
### Programmatic Registration (Advanced)
For dynamic hook registration or when you need to register hooks programmatically:
```python
from crewai.hooks import (
register_before_llm_call_hook,
register_after_tool_call_hook
)
def my_hook(context):
return None
# Register programmatically
register_before_llm_call_hook(my_hook)
# Useful for:
# - Loading hooks from configuration
# - Conditional hook registration
# - Plugin systems
```
**Note:** For most use cases, decorators are cleaner and more maintainable.
## Performance Considerations
1. **Keep Hooks Fast**: Hooks execute on every call - avoid heavy computation
2. **Cache When Possible**: Store expensive validations or lookups
3. **Be Selective**: Use crew-scoped hooks when global hooks aren't needed
4. **Monitor Hook Overhead**: Profile hook execution time in production
5. **Lazy Import**: Import heavy dependencies only when needed
## Debugging Hooks
### Enable Debug Logging
```python
import logging
logging.basicConfig(level=logging.DEBUG)
logger = logging.getLogger(__name__)
@before_llm_call
def debug_hook(context):
logger.debug(f"LLM call: {context.agent.role}, iteration {context.iterations}")
return None
```
### Hook Execution Order
Hooks execute in registration order. If a before hook returns `False`, subsequent hooks don't execute:
```python
# Register order matters!
register_before_tool_call_hook(hook1) # Executes first
register_before_tool_call_hook(hook2) # Executes second
register_before_tool_call_hook(hook3) # Executes third
# If hook2 returns False:
# - hook1 executed
# - hook2 executed and returned False
# - hook3 NOT executed
# - Tool call blocked
```
## Related Documentation
- [LLM Call Hooks →](/learn/llm-hooks) - Detailed LLM hook documentation
- [Tool Call Hooks →](/learn/tool-hooks) - Detailed tool hook documentation
- [Before and After Kickoff Hooks →](/learn/before-and-after-kickoff-hooks) - Crew lifecycle hooks
- [Human-in-the-Loop →](/learn/human-in-the-loop) - Human input patterns
## Conclusion
Execution hooks provide powerful control over agent runtime behavior. Use them to implement safety guardrails, approval workflows, comprehensive monitoring, and custom business logic. Combined with proper error handling, type safety, and performance considerations, hooks enable production-ready, secure, and observable agent systems.

View File

@@ -97,7 +97,7 @@ project_crew = Crew(
```
<Tip>
For more details on creating and customizing a manager agent, check out the [Custom Manager Agent documentation](https://docs.crewai.com/how-to/custom-manager-agent#custom-manager-agent).
For more details on creating and customizing a manager agent, check out the [Custom Manager Agent documentation](/en/learn/custom-manager-agent).
</Tip>

427
docs/en/learn/llm-hooks.mdx Normal file
View File

@@ -0,0 +1,427 @@
---
title: LLM Call Hooks
description: Learn how to use LLM call hooks to intercept, modify, and control language model interactions in CrewAI
mode: "wide"
---
LLM Call Hooks provide fine-grained control over language model interactions during agent execution. These hooks allow you to intercept LLM calls, modify prompts, transform responses, implement approval gates, and add custom logging or monitoring.
## Overview
LLM hooks are executed at two critical points:
- **Before LLM Call**: Modify messages, validate inputs, or block execution
- **After LLM Call**: Transform responses, sanitize outputs, or modify conversation history
## Hook Types
### Before LLM Call Hooks
Executed before every LLM call, these hooks can:
- Inspect and modify messages sent to the LLM
- Block LLM execution based on conditions
- Implement rate limiting or approval gates
- Add context or system messages
- Log request details
**Signature:**
```python
def before_hook(context: LLMCallHookContext) -> bool | None:
# Return False to block execution
# Return True or None to allow execution
...
```
### After LLM Call Hooks
Executed after every LLM call, these hooks can:
- Modify or sanitize LLM responses
- Add metadata or formatting
- Log response details
- Update conversation history
- Implement content filtering
**Signature:**
```python
def after_hook(context: LLMCallHookContext) -> str | None:
# Return modified response string
# Return None to keep original response
...
```
## LLM Hook Context
The `LLMCallHookContext` object provides comprehensive access to execution state:
```python
class LLMCallHookContext:
executor: CrewAgentExecutor # Full executor reference
messages: list # Mutable message list
agent: Agent # Current agent
task: Task # Current task
crew: Crew # Crew instance
llm: BaseLLM # LLM instance
iterations: int # Current iteration count
response: str | None # LLM response (after hooks only)
```
### Modifying Messages
**Important:** Always modify messages in-place:
```python
# ✅ Correct - modify in-place
def add_context(context: LLMCallHookContext) -> None:
context.messages.append({"role": "system", "content": "Be concise"})
# ❌ Wrong - replaces list reference
def wrong_approach(context: LLMCallHookContext) -> None:
context.messages = [{"role": "system", "content": "Be concise"}]
```
## Registration Methods
### 1. Global Hook Registration
Register hooks that apply to all LLM calls across all crews:
```python
from crewai.hooks import register_before_llm_call_hook, register_after_llm_call_hook
def log_llm_call(context):
print(f"LLM call by {context.agent.role} at iteration {context.iterations}")
return None # Allow execution
register_before_llm_call_hook(log_llm_call)
```
### 2. Decorator-Based Registration
Use decorators for cleaner syntax:
```python
from crewai.hooks import before_llm_call, after_llm_call
@before_llm_call
def validate_iteration_count(context):
if context.iterations > 10:
print("⚠️ Exceeded maximum iterations")
return False # Block execution
return None
@after_llm_call
def sanitize_response(context):
if context.response and "API_KEY" in context.response:
return context.response.replace("API_KEY", "[REDACTED]")
return None
```
### 3. Crew-Scoped Hooks
Register hooks for a specific crew instance:
```python
@CrewBase
class MyProjCrew:
@before_llm_call_crew
def validate_inputs(self, context):
# Only applies to this crew
if context.iterations == 0:
print(f"Starting task: {context.task.description}")
return None
@after_llm_call_crew
def log_responses(self, context):
# Crew-specific response logging
print(f"Response length: {len(context.response)}")
return None
@crew
def crew(self) -> Crew:
return Crew(
agents=self.agents,
tasks=self.tasks,
process=Process.sequential,
verbose=True
)
```
## Common Use Cases
### 1. Iteration Limiting
```python
@before_llm_call
def limit_iterations(context: LLMCallHookContext) -> bool | None:
max_iterations = 15
if context.iterations > max_iterations:
print(f"⛔ Blocked: Exceeded {max_iterations} iterations")
return False # Block execution
return None
```
### 2. Human Approval Gate
```python
@before_llm_call
def require_approval(context: LLMCallHookContext) -> bool | None:
if context.iterations > 5:
response = context.request_human_input(
prompt=f"Iteration {context.iterations}: Approve LLM call?",
default_message="Press Enter to approve, or type 'no' to block:"
)
if response.lower() == "no":
print("🚫 LLM call blocked by user")
return False
return None
```
### 3. Adding System Context
```python
@before_llm_call
def add_guardrails(context: LLMCallHookContext) -> None:
# Add safety guidelines to every LLM call
context.messages.append({
"role": "system",
"content": "Ensure responses are factual and cite sources when possible."
})
return None
```
### 4. Response Sanitization
```python
@after_llm_call
def sanitize_sensitive_data(context: LLMCallHookContext) -> str | None:
if not context.response:
return None
# Remove sensitive patterns
import re
sanitized = context.response
sanitized = re.sub(r'\b\d{3}-\d{2}-\d{4}\b', '[SSN-REDACTED]', sanitized)
sanitized = re.sub(r'\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b', '[CARD-REDACTED]', sanitized)
return sanitized
```
### 5. Cost Tracking
```python
import tiktoken
@before_llm_call
def track_token_usage(context: LLMCallHookContext) -> None:
encoding = tiktoken.get_encoding("cl100k_base")
total_tokens = sum(
len(encoding.encode(msg.get("content", "")))
for msg in context.messages
)
print(f"📊 Input tokens: ~{total_tokens}")
return None
@after_llm_call
def track_response_tokens(context: LLMCallHookContext) -> None:
if context.response:
encoding = tiktoken.get_encoding("cl100k_base")
tokens = len(encoding.encode(context.response))
print(f"📊 Response tokens: ~{tokens}")
return None
```
### 6. Debug Logging
```python
@before_llm_call
def debug_request(context: LLMCallHookContext) -> None:
print(f"""
🔍 LLM Call Debug:
- Agent: {context.agent.role}
- Task: {context.task.description[:50]}...
- Iteration: {context.iterations}
- Message Count: {len(context.messages)}
- Last Message: {context.messages[-1] if context.messages else 'None'}
""")
return None
@after_llm_call
def debug_response(context: LLMCallHookContext) -> None:
if context.response:
print(f"✅ Response Preview: {context.response[:100]}...")
return None
```
## Hook Management
### Unregistering Hooks
```python
from crewai.hooks import (
unregister_before_llm_call_hook,
unregister_after_llm_call_hook
)
# Unregister specific hook
def my_hook(context):
...
register_before_llm_call_hook(my_hook)
# Later...
unregister_before_llm_call_hook(my_hook) # Returns True if found
```
### Clearing Hooks
```python
from crewai.hooks import (
clear_before_llm_call_hooks,
clear_after_llm_call_hooks,
clear_all_llm_call_hooks
)
# Clear specific hook type
count = clear_before_llm_call_hooks()
print(f"Cleared {count} before hooks")
# Clear all LLM hooks
before_count, after_count = clear_all_llm_call_hooks()
print(f"Cleared {before_count} before and {after_count} after hooks")
```
### Listing Registered Hooks
```python
from crewai.hooks import (
get_before_llm_call_hooks,
get_after_llm_call_hooks
)
# Get current hooks
before_hooks = get_before_llm_call_hooks()
after_hooks = get_after_llm_call_hooks()
print(f"Registered: {len(before_hooks)} before, {len(after_hooks)} after")
```
## Advanced Patterns
### Conditional Hook Execution
```python
@before_llm_call
def conditional_blocking(context: LLMCallHookContext) -> bool | None:
# Only block for specific agents
if context.agent.role == "researcher" and context.iterations > 10:
return False
# Only block for specific tasks
if "sensitive" in context.task.description.lower() and context.iterations > 5:
return False
return None
```
### Context-Aware Modifications
```python
@before_llm_call
def adaptive_prompting(context: LLMCallHookContext) -> None:
# Add different context based on iteration
if context.iterations == 0:
context.messages.append({
"role": "system",
"content": "Start with a high-level overview."
})
elif context.iterations > 3:
context.messages.append({
"role": "system",
"content": "Focus on specific details and provide examples."
})
return None
```
### Chaining Hooks
```python
# Multiple hooks execute in registration order
@before_llm_call
def first_hook(context):
print("1. First hook executed")
return None
@before_llm_call
def second_hook(context):
print("2. Second hook executed")
return None
@before_llm_call
def blocking_hook(context):
if context.iterations > 10:
print("3. Blocking hook - execution stopped")
return False # Subsequent hooks won't execute
print("3. Blocking hook - execution allowed")
return None
```
## Best Practices
1. **Keep Hooks Focused**: Each hook should have a single responsibility
2. **Avoid Heavy Computation**: Hooks execute on every LLM call
3. **Handle Errors Gracefully**: Use try-except to prevent hook failures from breaking execution
4. **Use Type Hints**: Leverage `LLMCallHookContext` for better IDE support
5. **Document Hook Behavior**: Especially for blocking conditions
6. **Test Hooks Independently**: Unit test hooks before using in production
7. **Clear Hooks in Tests**: Use `clear_all_llm_call_hooks()` between test runs
8. **Modify In-Place**: Always modify `context.messages` in-place, never replace
## Error Handling
```python
@before_llm_call
def safe_hook(context: LLMCallHookContext) -> bool | None:
try:
# Your hook logic
if some_condition:
return False
except Exception as e:
print(f"⚠️ Hook error: {e}")
# Decide: allow or block on error
return None # Allow execution despite error
```
## Type Safety
```python
from crewai.hooks import LLMCallHookContext, BeforeLLMCallHookType, AfterLLMCallHookType
# Explicit type annotations
def my_before_hook(context: LLMCallHookContext) -> bool | None:
return None
def my_after_hook(context: LLMCallHookContext) -> str | None:
return None
# Type-safe registration
register_before_llm_call_hook(my_before_hook)
register_after_llm_call_hook(my_after_hook)
```
## Troubleshooting
### Hook Not Executing
- Verify hook is registered before crew execution
- Check if previous hook returned `False` (blocks subsequent hooks)
- Ensure hook signature matches expected type
### Message Modifications Not Persisting
- Use in-place modifications: `context.messages.append()`
- Don't replace the list: `context.messages = []`
### Response Modifications Not Working
- Return the modified string from after hooks
- Returning `None` keeps the original response
## Conclusion
LLM Call Hooks provide powerful capabilities for controlling and monitoring language model interactions in CrewAI. Use them to implement safety guardrails, approval gates, logging, cost tracking, and response sanitization. Combined with proper error handling and type safety, hooks enable robust and production-ready agent systems.

View File

@@ -0,0 +1,600 @@
---
title: Tool Call Hooks
description: Learn how to use tool call hooks to intercept, modify, and control tool execution in CrewAI
mode: "wide"
---
Tool Call Hooks provide fine-grained control over tool execution during agent operations. These hooks allow you to intercept tool calls, modify inputs, transform outputs, implement safety checks, and add comprehensive logging or monitoring.
## Overview
Tool hooks are executed at two critical points:
- **Before Tool Call**: Modify inputs, validate parameters, or block execution
- **After Tool Call**: Transform results, sanitize outputs, or log execution details
## Hook Types
### Before Tool Call Hooks
Executed before every tool execution, these hooks can:
- Inspect and modify tool inputs
- Block tool execution based on conditions
- Implement approval gates for dangerous operations
- Validate parameters
- Log tool invocations
**Signature:**
```python
def before_hook(context: ToolCallHookContext) -> bool | None:
# Return False to block execution
# Return True or None to allow execution
...
```
### After Tool Call Hooks
Executed after every tool execution, these hooks can:
- Modify or sanitize tool results
- Add metadata or formatting
- Log execution results
- Implement result validation
- Transform output formats
**Signature:**
```python
def after_hook(context: ToolCallHookContext) -> str | None:
# Return modified result string
# Return None to keep original result
...
```
## Tool Hook Context
The `ToolCallHookContext` object provides comprehensive access to tool execution state:
```python
class ToolCallHookContext:
tool_name: str # Name of the tool being called
tool_input: dict[str, Any] # Mutable tool input parameters
tool: CrewStructuredTool # Tool instance reference
agent: Agent | BaseAgent | None # Agent executing the tool
task: Task | None # Current task
crew: Crew | None # Crew instance
tool_result: str | None # Tool result (after hooks only)
```
### Modifying Tool Inputs
**Important:** Always modify tool inputs in-place:
```python
# ✅ Correct - modify in-place
def sanitize_input(context: ToolCallHookContext) -> None:
context.tool_input['query'] = context.tool_input['query'].lower()
# ❌ Wrong - replaces dict reference
def wrong_approach(context: ToolCallHookContext) -> None:
context.tool_input = {'query': 'new query'}
```
## Registration Methods
### 1. Global Hook Registration
Register hooks that apply to all tool calls across all crews:
```python
from crewai.hooks import register_before_tool_call_hook, register_after_tool_call_hook
def log_tool_call(context):
print(f"Tool: {context.tool_name}")
print(f"Input: {context.tool_input}")
return None # Allow execution
register_before_tool_call_hook(log_tool_call)
```
### 2. Decorator-Based Registration
Use decorators for cleaner syntax:
```python
from crewai.hooks import before_tool_call, after_tool_call
@before_tool_call
def block_dangerous_tools(context):
dangerous_tools = ['delete_database', 'drop_table', 'rm_rf']
if context.tool_name in dangerous_tools:
print(f"⛔ Blocked dangerous tool: {context.tool_name}")
return False # Block execution
return None
@after_tool_call
def sanitize_results(context):
if context.tool_result and "password" in context.tool_result.lower():
return context.tool_result.replace("password", "[REDACTED]")
return None
```
### 3. Crew-Scoped Hooks
Register hooks for a specific crew instance:
```python
@CrewBase
class MyProjCrew:
@before_tool_call_crew
def validate_tool_inputs(self, context):
# Only applies to this crew
if context.tool_name == "web_search":
if not context.tool_input.get('query'):
print("❌ Invalid search query")
return False
return None
@after_tool_call_crew
def log_tool_results(self, context):
# Crew-specific tool logging
print(f"✅ {context.tool_name} completed")
return None
@crew
def crew(self) -> Crew:
return Crew(
agents=self.agents,
tasks=self.tasks,
process=Process.sequential,
verbose=True
)
```
## Common Use Cases
### 1. Safety Guardrails
```python
@before_tool_call
def safety_check(context: ToolCallHookContext) -> bool | None:
# Block tools that could cause harm
destructive_tools = [
'delete_file',
'drop_table',
'remove_user',
'system_shutdown'
]
if context.tool_name in destructive_tools:
print(f"🛑 Blocked destructive tool: {context.tool_name}")
return False
# Warn on sensitive operations
sensitive_tools = ['send_email', 'post_to_social_media', 'charge_payment']
if context.tool_name in sensitive_tools:
print(f"⚠️ Executing sensitive tool: {context.tool_name}")
return None
```
### 2. Human Approval Gate
```python
@before_tool_call
def require_approval_for_actions(context: ToolCallHookContext) -> bool | None:
approval_required = [
'send_email',
'make_purchase',
'delete_file',
'post_message'
]
if context.tool_name in approval_required:
response = context.request_human_input(
prompt=f"Approve {context.tool_name}?",
default_message=f"Input: {context.tool_input}\nType 'yes' to approve:"
)
if response.lower() != 'yes':
print(f"❌ Tool execution denied: {context.tool_name}")
return False
return None
```
### 3. Input Validation and Sanitization
```python
@before_tool_call
def validate_and_sanitize_inputs(context: ToolCallHookContext) -> bool | None:
# Validate search queries
if context.tool_name == 'web_search':
query = context.tool_input.get('query', '')
if len(query) < 3:
print("❌ Search query too short")
return False
# Sanitize query
context.tool_input['query'] = query.strip().lower()
# Validate file paths
if context.tool_name == 'read_file':
path = context.tool_input.get('path', '')
if '..' in path or path.startswith('/'):
print("❌ Invalid file path")
return False
return None
```
### 4. Result Sanitization
```python
@after_tool_call
def sanitize_sensitive_data(context: ToolCallHookContext) -> str | None:
if not context.tool_result:
return None
import re
result = context.tool_result
# Remove API keys
result = re.sub(
r'(api[_-]?key|token)["\']?\s*[:=]\s*["\']?[\w-]+',
r'\1: [REDACTED]',
result,
flags=re.IGNORECASE
)
# Remove email addresses
result = re.sub(
r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b',
'[EMAIL-REDACTED]',
result
)
# Remove credit card numbers
result = re.sub(
r'\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b',
'[CARD-REDACTED]',
result
)
return result
```
### 5. Tool Usage Analytics
```python
import time
from collections import defaultdict
tool_stats = defaultdict(lambda: {'count': 0, 'total_time': 0, 'failures': 0})
@before_tool_call
def start_timer(context: ToolCallHookContext) -> None:
context.tool_input['_start_time'] = time.time()
return None
@after_tool_call
def track_tool_usage(context: ToolCallHookContext) -> None:
start_time = context.tool_input.get('_start_time', time.time())
duration = time.time() - start_time
tool_stats[context.tool_name]['count'] += 1
tool_stats[context.tool_name]['total_time'] += duration
if not context.tool_result or 'error' in context.tool_result.lower():
tool_stats[context.tool_name]['failures'] += 1
print(f"""
📊 Tool Stats for {context.tool_name}:
- Executions: {tool_stats[context.tool_name]['count']}
- Avg Time: {tool_stats[context.tool_name]['total_time'] / tool_stats[context.tool_name]['count']:.2f}s
- Failures: {tool_stats[context.tool_name]['failures']}
""")
return None
```
### 6. Rate Limiting
```python
from collections import defaultdict
from datetime import datetime, timedelta
tool_call_history = defaultdict(list)
@before_tool_call
def rate_limit_tools(context: ToolCallHookContext) -> bool | None:
tool_name = context.tool_name
now = datetime.now()
# Clean old entries (older than 1 minute)
tool_call_history[tool_name] = [
call_time for call_time in tool_call_history[tool_name]
if now - call_time < timedelta(minutes=1)
]
# Check rate limit (max 10 calls per minute)
if len(tool_call_history[tool_name]) >= 10:
print(f"🚫 Rate limit exceeded for {tool_name}")
return False
# Record this call
tool_call_history[tool_name].append(now)
return None
```
### 7. Caching Tool Results
```python
import hashlib
import json
tool_cache = {}
def cache_key(tool_name: str, tool_input: dict) -> str:
"""Generate cache key from tool name and input."""
input_str = json.dumps(tool_input, sort_keys=True)
return hashlib.md5(f"{tool_name}:{input_str}".encode()).hexdigest()
@before_tool_call
def check_cache(context: ToolCallHookContext) -> bool | None:
key = cache_key(context.tool_name, context.tool_input)
if key in tool_cache:
print(f"💾 Cache hit for {context.tool_name}")
# Note: Can't return cached result from before hook
# Would need to implement this differently
return None
@after_tool_call
def cache_result(context: ToolCallHookContext) -> None:
if context.tool_result:
key = cache_key(context.tool_name, context.tool_input)
tool_cache[key] = context.tool_result
print(f"💾 Cached result for {context.tool_name}")
return None
```
### 8. Debug Logging
```python
@before_tool_call
def debug_tool_call(context: ToolCallHookContext) -> None:
print(f"""
🔍 Tool Call Debug:
- Tool: {context.tool_name}
- Agent: {context.agent.role if context.agent else 'Unknown'}
- Task: {context.task.description[:50] if context.task else 'Unknown'}...
- Input: {context.tool_input}
""")
return None
@after_tool_call
def debug_tool_result(context: ToolCallHookContext) -> None:
if context.tool_result:
result_preview = context.tool_result[:200]
print(f"✅ Result Preview: {result_preview}...")
else:
print("⚠️ No result returned")
return None
```
## Hook Management
### Unregistering Hooks
```python
from crewai.hooks import (
unregister_before_tool_call_hook,
unregister_after_tool_call_hook
)
# Unregister specific hook
def my_hook(context):
...
register_before_tool_call_hook(my_hook)
# Later...
success = unregister_before_tool_call_hook(my_hook)
print(f"Unregistered: {success}")
```
### Clearing Hooks
```python
from crewai.hooks import (
clear_before_tool_call_hooks,
clear_after_tool_call_hooks,
clear_all_tool_call_hooks
)
# Clear specific hook type
count = clear_before_tool_call_hooks()
print(f"Cleared {count} before hooks")
# Clear all tool hooks
before_count, after_count = clear_all_tool_call_hooks()
print(f"Cleared {before_count} before and {after_count} after hooks")
```
### Listing Registered Hooks
```python
from crewai.hooks import (
get_before_tool_call_hooks,
get_after_tool_call_hooks
)
# Get current hooks
before_hooks = get_before_tool_call_hooks()
after_hooks = get_after_tool_call_hooks()
print(f"Registered: {len(before_hooks)} before, {len(after_hooks)} after")
```
## Advanced Patterns
### Conditional Hook Execution
```python
@before_tool_call
def conditional_blocking(context: ToolCallHookContext) -> bool | None:
# Only block for specific agents
if context.agent and context.agent.role == "junior_agent":
if context.tool_name in ['delete_file', 'send_email']:
print(f"❌ Junior agents cannot use {context.tool_name}")
return False
# Only block during specific tasks
if context.task and "sensitive" in context.task.description.lower():
if context.tool_name == 'web_search':
print("❌ Web search blocked for sensitive tasks")
return False
return None
```
### Context-Aware Input Modification
```python
@before_tool_call
def enhance_tool_inputs(context: ToolCallHookContext) -> None:
# Add context based on agent role
if context.agent and context.agent.role == "researcher":
if context.tool_name == 'web_search':
# Add domain restrictions for researchers
context.tool_input['domains'] = ['edu', 'gov', 'org']
# Add context based on task
if context.task and "urgent" in context.task.description.lower():
if context.tool_name == 'send_email':
context.tool_input['priority'] = 'high'
return None
```
### Tool Chain Monitoring
```python
tool_call_chain = []
@before_tool_call
def track_tool_chain(context: ToolCallHookContext) -> None:
tool_call_chain.append({
'tool': context.tool_name,
'timestamp': time.time(),
'agent': context.agent.role if context.agent else 'Unknown'
})
# Detect potential infinite loops
recent_calls = tool_call_chain[-5:]
if len(recent_calls) == 5 and all(c['tool'] == context.tool_name for c in recent_calls):
print(f"⚠️ Warning: {context.tool_name} called 5 times in a row")
return None
```
## Best Practices
1. **Keep Hooks Focused**: Each hook should have a single responsibility
2. **Avoid Heavy Computation**: Hooks execute on every tool call
3. **Handle Errors Gracefully**: Use try-except to prevent hook failures
4. **Use Type Hints**: Leverage `ToolCallHookContext` for better IDE support
5. **Document Blocking Conditions**: Make it clear when/why tools are blocked
6. **Test Hooks Independently**: Unit test hooks before using in production
7. **Clear Hooks in Tests**: Use `clear_all_tool_call_hooks()` between test runs
8. **Modify In-Place**: Always modify `context.tool_input` in-place, never replace
9. **Log Important Decisions**: Especially when blocking tool execution
10. **Consider Performance**: Cache expensive validations when possible
## Error Handling
```python
@before_tool_call
def safe_validation(context: ToolCallHookContext) -> bool | None:
try:
# Your validation logic
if not validate_input(context.tool_input):
return False
except Exception as e:
print(f"⚠️ Hook error: {e}")
# Decide: allow or block on error
return None # Allow execution despite error
```
## Type Safety
```python
from crewai.hooks import ToolCallHookContext, BeforeToolCallHookType, AfterToolCallHookType
# Explicit type annotations
def my_before_hook(context: ToolCallHookContext) -> bool | None:
return None
def my_after_hook(context: ToolCallHookContext) -> str | None:
return None
# Type-safe registration
register_before_tool_call_hook(my_before_hook)
register_after_tool_call_hook(my_after_hook)
```
## Integration with Existing Tools
### Wrapping Existing Validation
```python
def existing_validator(tool_name: str, inputs: dict) -> bool:
"""Your existing validation function."""
# Your validation logic
return True
@before_tool_call
def integrate_validator(context: ToolCallHookContext) -> bool | None:
if not existing_validator(context.tool_name, context.tool_input):
print(f"❌ Validation failed for {context.tool_name}")
return False
return None
```
### Logging to External Systems
```python
import logging
logger = logging.getLogger(__name__)
@before_tool_call
def log_to_external_system(context: ToolCallHookContext) -> None:
logger.info(f"Tool call: {context.tool_name}", extra={
'tool_name': context.tool_name,
'tool_input': context.tool_input,
'agent': context.agent.role if context.agent else None
})
return None
```
## Troubleshooting
### Hook Not Executing
- Verify hook is registered before crew execution
- Check if previous hook returned `False` (blocks execution and subsequent hooks)
- Ensure hook signature matches expected type
### Input Modifications Not Working
- Use in-place modifications: `context.tool_input['key'] = value`
- Don't replace the dict: `context.tool_input = {}`
### Result Modifications Not Working
- Return the modified string from after hooks
- Returning `None` keeps the original result
- Ensure the tool actually returned a result
### Tool Blocked Unexpectedly
- Check all before hooks for blocking conditions
- Verify hook execution order
- Add debug logging to identify which hook is blocking
## Conclusion
Tool Call Hooks provide powerful capabilities for controlling and monitoring tool execution in CrewAI. Use them to implement safety guardrails, approval gates, input validation, result sanitization, logging, and analytics. Combined with proper error handling and type safety, hooks enable secure and production-ready agent systems with comprehensive observability.

View File

@@ -733,9 +733,7 @@ Here's a basic configuration to route requests to OpenAI, specifically using GPT
- Collect relevant metadata to filter logs
- Enforce access permissions
Create API keys through:
- [Portkey App](https://app.portkey.ai/)
- [API Key Management API](/en/api-reference/admin-api/control-plane/api-keys/create-api-key)
Create API keys through the [Portkey App](https://app.portkey.ai/)
Example using Python SDK:
```python
@@ -758,7 +756,7 @@ Here's a basic configuration to route requests to OpenAI, specifically using GPT
)
```
For detailed key management instructions, see our [API Keys documentation](/en/api-reference/admin-api/control-plane/api-keys/create-api-key).
For detailed key management instructions, see the [Portkey documentation](https://portkey.ai/docs).
</Accordion>
<Accordion title="Step 4: Deploy & Monitor">

View File

@@ -18,7 +18,7 @@ These tools enable your agents to interact with cloud services, access cloud sto
Write and upload files to Amazon S3 storage.
</Card>
<Card title="Bedrock Invoke Agent" icon="aws" href="/en/tools/cloud-storage/bedrockinvokeagenttool">
<Card title="Bedrock Invoke Agent" icon="aws" href="/en/tools/integration/bedrockinvokeagenttool">
Invoke Amazon Bedrock agents for AI-powered tasks.
</Card>

View File

@@ -632,11 +632,11 @@ mode: "wide"
## 기여
기여를 원하시면, [기여 가이드](CONTRIBUTING.md)를 참조하세요.
기여를 원하시면, [기여 가이드](https://github.com/crewAIInc/crewAI/blob/main/CONTRIBUTING.md)를 참조하세요.
## 라이센스
이 프로젝트는 MIT 라이센스 하에 배포됩니다. 자세한 내용은 [LICENSE](LICENSE) 파일을 확인하세요.
이 프로젝트는 MIT 라이센스 하에 배포됩니다. 자세한 내용은 [LICENSE](https://github.com/crewAIInc/crewAI/blob/main/LICENSE) 파일을 확인하세요.
</Update>
<Update label="2025년 5월 22일">

View File

@@ -706,7 +706,7 @@ class KnowledgeMonitorListener(BaseEventListener):
knowledge_monitor = KnowledgeMonitorListener()
```
이벤트 사용에 대한 자세한 내용은 [이벤트 리스너](https://docs.crewai.com/concepts/event-listener) 문서를 참고하세요.
이벤트 사용에 대한 자세한 내용은 [이벤트 리스너](/ko/concepts/event-listener) 문서를 참고하세요.
### 맞춤형 지식 소스

View File

@@ -748,7 +748,7 @@ CrewAI는 LLM의 스트리밍 응답을 지원하여, 애플리케이션이 출
```
<Tip>
[자세한 내용은 여기를 클릭하세요](https://docs.crewai.com/concepts/event-listener#event-listeners)
[자세한 내용은 여기를 클릭하세요](/ko/concepts/event-listener#event-listeners)
</Tip>
</Tab>

View File

@@ -36,7 +36,7 @@ mode: "wide"
<Card title="도구 & 통합" href="/ko/enterprise/features/tools-and-integrations" icon="wrench">
에이전트가 사용할 외부 앱 연결 및 내부 도구 관리.
</Card>
<Card title="도구 저장소" href="/ko/enterprise/features/tool-repository" icon="toolbox">
<Card title="도구 저장소" href="/ko/enterprise/guides/tool-repository" icon="toolbox">
크루 기능을 확장할 수 있도록 도구를 게시하고 설치.
</Card>
<Card title="에이전트 저장소" href="/ko/enterprise/features/agent-repositories" icon="people-group">

View File

@@ -231,7 +231,7 @@ mode: "wide"
## 관련 문서
<CardGroup cols={2}>
<Card title="도구 저장소" href="/ko/enterprise/features/tool-repository" icon="toolbox">
<Card title="도구 저장소" href="/ko/enterprise/guides/tool-repository" icon="toolbox">
크루 기능을 확장할 수 있도록 도구를 게시하고 설치하세요.
</Card>
<Card title="Webhook 자동화" href="/ko/enterprise/guides/webhook-automation" icon="bolt">

View File

@@ -21,7 +21,7 @@ Tool Repository는 CrewAI 도구를 위한 패키지 관리자입니다. 사용
Tool Repository를 사용하기 전에 다음이 준비되어 있어야 합니다:
- [CrewAI AMP](https://app.crewai.com) 계정
- [CrewAI CLI](https://docs.crewai.com/concepts/cli#cli) 설치됨
- [CrewAI CLI](/ko/concepts/cli#cli) 설치됨
- uv>=0.5.0 이 설치되어 있어야 합니다. [업그레이드 방법](https://docs.astral.sh/uv/getting-started/installation/#upgrading-uv)을 참고하세요.
- [Git](https://git-scm.com) 설치 및 구성 완료
- CrewAI AMP 조직에서 도구를 게시하거나 설치할 수 있는 액세스 권한
@@ -66,7 +66,7 @@ crewai tool publish
crewai tool publish --public
```
도구 빌드에 대한 자세한 내용은 [나만의 도구 만들기](https://docs.crewai.com/concepts/tools#creating-your-own-tools)를 참고하세요.
도구 빌드에 대한 자세한 내용은 [나만의 도구 만들기](/ko/concepts/tools#creating-your-own-tools)를 참고하세요.
## 도구 업데이트

View File

@@ -49,7 +49,7 @@ mode: "wide"
에이전트 실행에 인간 입력을 통합하려면 작업 정의에서 `human_input` 플래그를 설정하세요. 활성화하면, 에이전트가 최종 답변을 제공하기 전에 사용자에게 입력을 요청합니다. 이 입력은 추가 맥락을 제공하거나, 애매함을 해소하거나, 에이전트의 출력을 검증해야 할 때 활용될 수 있습니다.
자세한 구현 방법은 [Human-in-the-Loop 가이드](/ko/how-to/human-in-the-loop)를 참고해 주세요.
자세한 구현 방법은 [Human-in-the-Loop 가이드](/ko/enterprise/guides/human-in-the-loop)를 참고해 주세요.
</Accordion>
<Accordion title="CrewAI에서 에이전트의 행동과 역량을 맞춤화하고 향상시키기 위한 고급 커스터마이징 옵션에는 어떤 것이 있나요?">
@@ -142,7 +142,7 @@ mode: "wide"
<Accordion title="CrewAI 에이전트를 위한 커스텀 도구는 어떻게 만들 수 있습니까?">
CrewAI에서 제공하는 `BaseTool` 클래스를 상속받아 커스텀 도구를 직접 만들거나, tool 데코레이터를 활용할 수 있습니다. 상속 방식은 `BaseTool`을 상속하는 새로운 클래스를 정의해 이름, 설명, 그리고 실제 논리를 처리하는 `_run` 메서드를 작성합니다. tool 데코레이터를 사용하면 필수 속성과 운영 로직만 정의해 바로 `Tool` 객체를 만들 수 있습니다.
<Card href="https://docs.crewai.com/how-to/create-custom-tools" icon="code">CrewAI 도구 가이드</Card>
<Card href="/ko/learn/create-custom-tools" icon="code">CrewAI 도구 가이드</Card>
</Accordion>
<Accordion title="전체 crew가 수행할 수 있는 분당 최대 요청 수는 어떻게 제한할 수 있나요?">

View File

@@ -0,0 +1,379 @@
---
title: 실행 훅 개요
description: 에이전트 작업에 대한 세밀한 제어를 위한 CrewAI 실행 훅 이해 및 사용
mode: "wide"
---
실행 훅(Execution Hooks)은 CrewAI 에이전트의 런타임 동작을 세밀하게 제어할 수 있게 해줍니다. 크루 실행 전후에 실행되는 킥오프 훅과 달리, 실행 훅은 에이전트 실행 중 특정 작업을 가로채서 동작을 수정하고, 안전성 검사를 구현하며, 포괄적인 모니터링을 추가할 수 있습니다.
## 실행 훅의 유형
CrewAI는 두 가지 주요 범주의 실행 훅을 제공합니다:
### 1. [LLM 호출 훅](/learn/llm-hooks)
언어 모델 상호작용을 제어하고 모니터링합니다:
- **LLM 호출 전**: 프롬프트 수정, 입력 검증, 승인 게이트 구현
- **LLM 호출 후**: 응답 변환, 출력 정제, 대화 기록 업데이트
**사용 사례:**
- 반복 제한
- 비용 추적 및 토큰 사용량 모니터링
- 응답 정제 및 콘텐츠 필터링
- LLM 호출에 대한 사람의 승인
- 안전 가이드라인 또는 컨텍스트 추가
- 디버그 로깅 및 요청/응답 검사
[LLM 훅 문서 보기 →](/learn/llm-hooks)
### 2. [도구 호출 훅](/learn/tool-hooks)
도구 실행을 제어하고 모니터링합니다:
- **도구 호출 전**: 입력 수정, 매개변수 검증, 위험한 작업 차단
- **도구 호출 후**: 결과 변환, 출력 정제, 실행 세부사항 로깅
**사용 사례:**
- 파괴적인 작업에 대한 안전 가드레일
- 민감한 작업에 대한 사람의 승인
- 입력 검증 및 정제
- 결과 캐싱 및 속도 제한
- 도구 사용 분석
- 디버그 로깅 및 모니터링
[도구 훅 문서 보기 →](/learn/tool-hooks)
## 훅 등록 방법
### 1. 데코레이터 기반 훅 (권장)
훅을 등록하는 가장 깔끔하고 파이썬스러운 방법:
```python
from crewai.hooks import before_llm_call, after_llm_call, before_tool_call, after_tool_call
@before_llm_call
def limit_iterations(context):
"""반복 횟수를 제한하여 무한 루프를 방지합니다."""
if context.iterations > 10:
return False # 실행 차단
return None
@after_llm_call
def sanitize_response(context):
"""LLM 응답에서 민감한 데이터를 제거합니다."""
if "API_KEY" in context.response:
return context.response.replace("API_KEY", "[수정됨]")
return None
@before_tool_call
def block_dangerous_tools(context):
"""파괴적인 작업을 차단합니다."""
if context.tool_name == "delete_database":
return False # 실행 차단
return None
@after_tool_call
def log_tool_result(context):
"""도구 실행을 로깅합니다."""
print(f"도구 {context.tool_name} 완료")
return None
```
### 2. 크루 범위 훅
특정 크루 인스턴스에만 훅을 적용합니다:
```python
from crewai import CrewBase
from crewai.project import crew
from crewai.hooks import before_llm_call_crew, after_tool_call_crew
@CrewBase
class MyProjCrew:
@before_llm_call_crew
def validate_inputs(self, context):
# 이 크루에만 적용됩니다
print(f"{self.__class__.__name__}에서 LLM 호출")
return None
@after_tool_call_crew
def log_results(self, context):
# 크루별 로깅
print(f"도구 결과: {context.tool_result[:50]}...")
return None
@crew
def crew(self) -> Crew:
return Crew(
agents=self.agents,
tasks=self.tasks,
process=Process.sequential
)
```
## 훅 실행 흐름
### LLM 호출 흐름
```
에이전트가 LLM을 호출해야 함
[LLM 호출 전 훅 실행]
├→ 훅 1: 반복 횟수 검증
├→ 훅 2: 안전 컨텍스트 추가
└→ 훅 3: 요청 로깅
훅이 False를 반환하는 경우:
├→ LLM 호출 차단
└→ ValueError 발생
모든 훅이 True/None을 반환하는 경우:
├→ LLM 호출 진행
└→ 응답 생성
[LLM 호출 후 훅 실행]
├→ 훅 1: 응답 정제
├→ 훅 2: 응답 로깅
└→ 훅 3: 메트릭 업데이트
최종 응답 반환
```
### 도구 호출 흐름
```
에이전트가 도구를 실행해야 함
[도구 호출 전 훅 실행]
├→ 훅 1: 도구 허용 여부 확인
├→ 훅 2: 입력 검증
└→ 훅 3: 필요시 승인 요청
훅이 False를 반환하는 경우:
├→ 도구 실행 차단
└→ 오류 메시지 반환
모든 훅이 True/None을 반환하는 경우:
├→ 도구 실행 진행
└→ 결과 생성
[도구 호출 후 훅 실행]
├→ 훅 1: 결과 정제
├→ 훅 2: 결과 캐싱
└→ 훅 3: 메트릭 로깅
최종 결과 반환
```
## 훅 컨텍스트 객체
### LLMCallHookContext
LLM 실행 상태에 대한 액세스를 제공합니다:
```python
class LLMCallHookContext:
executor: CrewAgentExecutor # 전체 실행자 액세스
messages: list # 변경 가능한 메시지 목록
agent: Agent # 현재 에이전트
task: Task # 현재 작업
crew: Crew # 크루 인스턴스
llm: BaseLLM # LLM 인스턴스
iterations: int # 현재 반복 횟수
response: str | None # LLM 응답 (후 훅용)
```
### ToolCallHookContext
도구 실행 상태에 대한 액세스를 제공합니다:
```python
class ToolCallHookContext:
tool_name: str # 호출되는 도구
tool_input: dict # 변경 가능한 입력 매개변수
tool: CrewStructuredTool # 도구 인스턴스
agent: Agent | None # 실행 중인 에이전트
task: Task | None # 현재 작업
crew: Crew | None # 크루 인스턴스
tool_result: str | None # 도구 결과 (후 훅용)
```
## 일반적인 패턴
### 안전 및 검증
```python
@before_tool_call
def safety_check(context):
"""파괴적인 작업을 차단합니다."""
dangerous = ['delete_file', 'drop_table', 'system_shutdown']
if context.tool_name in dangerous:
print(f"🛑 차단됨: {context.tool_name}")
return False
return None
@before_llm_call
def iteration_limit(context):
"""무한 루프를 방지합니다."""
if context.iterations > 15:
print("⛔ 최대 반복 횟수 초과")
return False
return None
```
### 사람의 개입
```python
@before_tool_call
def require_approval(context):
"""민감한 작업에 대한 승인을 요구합니다."""
sensitive = ['send_email', 'make_payment', 'post_message']
if context.tool_name in sensitive:
response = context.request_human_input(
prompt=f"{context.tool_name} 승인하시겠습니까?",
default_message="승인하려면 'yes'를 입력하세요:"
)
if response.lower() != 'yes':
return False
return None
```
### 모니터링 및 분석
```python
from collections import defaultdict
import time
metrics = defaultdict(lambda: {'count': 0, 'total_time': 0})
@before_tool_call
def start_timer(context):
context.tool_input['_start'] = time.time()
return None
@after_tool_call
def track_metrics(context):
start = context.tool_input.get('_start', time.time())
duration = time.time() - start
metrics[context.tool_name]['count'] += 1
metrics[context.tool_name]['total_time'] += duration
return None
```
## 훅 관리
### 모든 훅 지우기
```python
from crewai.hooks import clear_all_global_hooks
# 모든 훅을 한 번에 지웁니다
result = clear_all_global_hooks()
print(f"{result['total']} 훅이 지워졌습니다")
```
### 특정 훅 유형 지우기
```python
from crewai.hooks import (
clear_before_llm_call_hooks,
clear_after_llm_call_hooks,
clear_before_tool_call_hooks,
clear_after_tool_call_hooks
)
# 특정 유형 지우기
llm_before_count = clear_before_llm_call_hooks()
tool_after_count = clear_after_tool_call_hooks()
```
## 모범 사례
### 1. 훅을 집중적으로 유지
각 훅은 단일하고 명확한 책임을 가져야 합니다.
### 2. 오류를 우아하게 처리
```python
@before_llm_call
def safe_hook(context):
try:
if some_condition:
return False
except Exception as e:
print(f"훅 오류: {e}")
return None # 오류에도 불구하고 실행 허용
```
### 3. 컨텍스트를 제자리에서 수정
```python
# ✅ 올바름 - 제자리에서 수정
@before_llm_call
def add_context(context):
context.messages.append({"role": "system", "content": "간결하게"})
# ❌ 잘못됨 - 참조를 교체
@before_llm_call
def wrong_approach(context):
context.messages = [{"role": "system", "content": "간결하게"}]
```
### 4. 타입 힌트 사용
```python
from crewai.hooks import LLMCallHookContext, ToolCallHookContext
def my_llm_hook(context: LLMCallHookContext) -> bool | None:
return None
def my_tool_hook(context: ToolCallHookContext) -> str | None:
return None
```
### 5. 테스트에서 정리
```python
import pytest
from crewai.hooks import clear_all_global_hooks
@pytest.fixture(autouse=True)
def clean_hooks():
"""각 테스트 전에 훅을 재설정합니다."""
yield
clear_all_global_hooks()
```
## 어떤 훅을 사용해야 할까요
### LLM 훅을 사용하는 경우:
- 반복 제한 구현
- 프롬프트에 컨텍스트 또는 안전 가이드라인 추가
- 토큰 사용량 및 비용 추적
- 응답 정제 또는 변환
- LLM 호출에 대한 승인 게이트 구현
- 프롬프트/응답 상호작용 디버깅
### 도구 훅을 사용하는 경우:
- 위험하거나 파괴적인 작업 차단
- 실행 전 도구 입력 검증
- 민감한 작업에 대한 승인 게이트 구현
- 도구 결과 캐싱
- 도구 사용 및 성능 추적
- 도구 출력 정제
- 도구 호출 속도 제한
### 둘 다 사용하는 경우:
모든 에이전트 작업을 모니터링해야 하는 포괄적인 관찰성, 안전 또는 승인 시스템을 구축하는 경우.
## 관련 문서
- [LLM 호출 훅 →](/learn/llm-hooks) - 상세한 LLM 훅 문서
- [도구 호출 훅 →](/learn/tool-hooks) - 상세한 도구 훅 문서
- [킥오프 전후 훅 →](/learn/before-and-after-kickoff-hooks) - 크루 생명주기 훅
- [사람의 개입 →](/learn/human-in-the-loop) - 사람 입력 패턴
## 결론
실행 훅은 에이전트 런타임 동작에 대한 강력한 제어를 제공합니다. 이를 사용하여 안전 가드레일, 승인 워크플로우, 포괄적인 모니터링 및 사용자 정의 비즈니스 로직을 구현하세요. 적절한 오류 처리, 타입 안전성 및 성능 고려사항과 결합하면, 훅을 통해 프로덕션 준비가 된 안전하고 관찰 가능한 에이전트 시스템을 구축할 수 있습니다.

View File

@@ -95,7 +95,7 @@ project_crew = Crew(
```
<Tip>
매니저 에이전트 생성 및 맞춤화에 대한 자세한 내용은 [커스텀 매니저 에이전트 문서](https://docs.crewai.com/how-to/custom-manager-agent#custom-manager-agent)를 참고하세요.
매니저 에이전트 생성 및 맞춤화에 대한 자세한 내용은 [커스텀 매니저 에이전트 문서](/ko/learn/custom-manager-agent)를 참고하세요.
</Tip>
### 워크플로우 실행

412
docs/ko/learn/llm-hooks.mdx Normal file
View File

@@ -0,0 +1,412 @@
---
title: LLM 호출 훅
description: CrewAI에서 언어 모델 상호작용을 가로채고, 수정하고, 제어하는 LLM 호출 훅 사용 방법 배우기
mode: "wide"
---
LLM 호출 훅(LLM Call Hooks)은 에이전트 실행 중 언어 모델 상호작용에 대한 세밀한 제어를 제공합니다. 이러한 훅을 사용하면 LLM 호출을 가로채고, 프롬프트를 수정하고, 응답을 변환하고, 승인 게이트를 구현하고, 사용자 정의 로깅 또는 모니터링을 추가할 수 있습니다.
## 개요
LLM 훅은 두 가지 중요한 시점에 실행됩니다:
- **LLM 호출 전**: 메시지 수정, 입력 검증 또는 실행 차단
- **LLM 호출 후**: 응답 변환, 출력 정제 또는 대화 기록 수정
## 훅 타입
### LLM 호출 전 훅
모든 LLM 호출 전에 실행되며, 다음을 수행할 수 있습니다:
- LLM에 전송되는 메시지 검사 및 수정
- 조건에 따라 LLM 실행 차단
- 속도 제한 또는 승인 게이트 구현
- 컨텍스트 또는 시스템 메시지 추가
- 요청 세부사항 로깅
**시그니처:**
```python
def before_hook(context: LLMCallHookContext) -> bool | None:
# 실행을 차단하려면 False 반환
# 실행을 허용하려면 True 또는 None 반환
...
```
### LLM 호출 후 훅
모든 LLM 호출 후에 실행되며, 다음을 수행할 수 있습니다:
- LLM 응답 수정 또는 정제
- 메타데이터 또는 서식 추가
- 응답 세부사항 로깅
- 대화 기록 업데이트
- 콘텐츠 필터링 구현
**시그니처:**
```python
def after_hook(context: LLMCallHookContext) -> str | None:
# 수정된 응답 문자열 반환
# 원본 응답을 유지하려면 None 반환
...
```
## LLM 훅 컨텍스트
`LLMCallHookContext` 객체는 실행 상태에 대한 포괄적인 액세스를 제공합니다:
```python
class LLMCallHookContext:
executor: CrewAgentExecutor # 전체 실행자 참조
messages: list # 변경 가능한 메시지 목록
agent: Agent # 현재 에이전트
task: Task # 현재 작업
crew: Crew # 크루 인스턴스
llm: BaseLLM # LLM 인스턴스
iterations: int # 현재 반복 횟수
response: str | None # LLM 응답 (후 훅용)
```
### 메시지 수정
**중요:** 항상 메시지를 제자리에서 수정하세요:
```python
# ✅ 올바름 - 제자리에서 수정
def add_context(context: LLMCallHookContext) -> None:
context.messages.append({"role": "system", "content": "간결하게 작성하세요"})
# ❌ 잘못됨 - 리스트 참조를 교체
def wrong_approach(context: LLMCallHookContext) -> None:
context.messages = [{"role": "system", "content": "간결하게 작성하세요"}]
```
## 등록 방법
### 1. 데코레이터 기반 등록 (권장)
더 깔끔한 구문을 위해 데코레이터를 사용합니다:
```python
from crewai.hooks import before_llm_call, after_llm_call
@before_llm_call
def validate_iteration_count(context):
"""반복 횟수를 검증합니다."""
if context.iterations > 10:
print("⚠️ 최대 반복 횟수 초과")
return False # 실행 차단
return None
@after_llm_call
def sanitize_response(context):
"""민감한 데이터를 제거합니다."""
if context.response and "API_KEY" in context.response:
return context.response.replace("API_KEY", "[수정됨]")
return None
```
### 2. 크루 범위 훅
특정 크루 인스턴스에 대한 훅을 등록합니다:
```python
from crewai import CrewBase
from crewai.project import crew
from crewai.hooks import before_llm_call_crew, after_llm_call_crew
@CrewBase
class MyProjCrew:
@before_llm_call_crew
def validate_inputs(self, context):
# 이 크루에만 적용됩니다
if context.iterations == 0:
print(f"작업 시작: {context.task.description}")
return None
@after_llm_call_crew
def log_responses(self, context):
# 크루별 응답 로깅
print(f"응답 길이: {len(context.response)}")
return None
@crew
def crew(self) -> Crew:
return Crew(
agents=self.agents,
tasks=self.tasks,
process=Process.sequential,
verbose=True
)
```
## 일반적인 사용 사례
### 1. 반복 제한
```python
@before_llm_call
def limit_iterations(context: LLMCallHookContext) -> bool | None:
"""무한 루프를 방지하기 위해 반복을 제한합니다."""
max_iterations = 15
if context.iterations > max_iterations:
print(f"⛔ 차단됨: {max_iterations}회 반복 초과")
return False # 실행 차단
return None
```
### 2. 사람의 승인 게이트
```python
@before_llm_call
def require_approval(context: LLMCallHookContext) -> bool | None:
"""특정 반복 후 승인을 요구합니다."""
if context.iterations > 5:
response = context.request_human_input(
prompt=f"반복 {context.iterations}: LLM 호출을 승인하시겠습니까?",
default_message="승인하려면 Enter를 누르고, 차단하려면 'no'를 입력하세요:"
)
if response.lower() == "no":
print("🚫 사용자에 의해 LLM 호출이 차단되었습니다")
return False
return None
```
### 3. 시스템 컨텍스트 추가
```python
@before_llm_call
def add_guardrails(context: LLMCallHookContext) -> None:
"""모든 LLM 호출에 안전 가이드라인을 추가합니다."""
context.messages.append({
"role": "system",
"content": "응답이 사실에 기반하고 가능한 경우 출처를 인용하도록 하세요."
})
return None
```
### 4. 응답 정제
```python
@after_llm_call
def sanitize_sensitive_data(context: LLMCallHookContext) -> str | None:
"""민감한 데이터 패턴을 제거합니다."""
if not context.response:
return None
import re
sanitized = context.response
sanitized = re.sub(r'\b\d{3}-\d{2}-\d{4}\b', '[주민번호-수정됨]', sanitized)
sanitized = re.sub(r'\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b', '[카드번호-수정됨]', sanitized)
return sanitized
```
### 5. 비용 추적
```python
import tiktoken
@before_llm_call
def track_token_usage(context: LLMCallHookContext) -> None:
"""입력 토큰을 추적합니다."""
encoding = tiktoken.get_encoding("cl100k_base")
total_tokens = sum(
len(encoding.encode(msg.get("content", "")))
for msg in context.messages
)
print(f"📊 입력 토큰: ~{total_tokens}")
return None
@after_llm_call
def track_response_tokens(context: LLMCallHookContext) -> None:
"""응답 토큰을 추적합니다."""
if context.response:
encoding = tiktoken.get_encoding("cl100k_base")
tokens = len(encoding.encode(context.response))
print(f"📊 응답 토큰: ~{tokens}")
return None
```
### 6. 디버그 로깅
```python
@before_llm_call
def debug_request(context: LLMCallHookContext) -> None:
"""LLM 요청을 디버그합니다."""
print(f"""
🔍 LLM 호출 디버그:
- 에이전트: {context.agent.role}
- 작업: {context.task.description[:50]}...
- 반복: {context.iterations}
- 메시지 수: {len(context.messages)}
- 마지막 메시지: {context.messages[-1] if context.messages else 'None'}
""")
return None
@after_llm_call
def debug_response(context: LLMCallHookContext) -> None:
"""LLM 응답을 디버그합니다."""
if context.response:
print(f"✅ 응답 미리보기: {context.response[:100]}...")
return None
```
## 훅 관리
### 훅 등록 해제
```python
from crewai.hooks import (
unregister_before_llm_call_hook,
unregister_after_llm_call_hook
)
# 특정 훅 등록 해제
def my_hook(context):
...
register_before_llm_call_hook(my_hook)
# 나중에...
unregister_before_llm_call_hook(my_hook) # 찾으면 True 반환
```
### 훅 지우기
```python
from crewai.hooks import (
clear_before_llm_call_hooks,
clear_after_llm_call_hooks,
clear_all_llm_call_hooks
)
# 특정 훅 타입 지우기
count = clear_before_llm_call_hooks()
print(f"{count}개의 전(before) 훅이 지워졌습니다")
# 모든 LLM 훅 지우기
before_count, after_count = clear_all_llm_call_hooks()
print(f"{before_count}개의 전(before) 훅과 {after_count}개의 후(after) 훅이 지워졌습니다")
```
## 고급 패턴
### 조건부 훅 실행
```python
@before_llm_call
def conditional_blocking(context: LLMCallHookContext) -> bool | None:
"""특정 조건에서만 차단합니다."""
# 특정 에이전트에 대해서만 차단
if context.agent.role == "researcher" and context.iterations > 10:
return False
# 특정 작업에 대해서만 차단
if "민감한" in context.task.description.lower() and context.iterations > 5:
return False
return None
```
### 컨텍스트 인식 수정
```python
@before_llm_call
def adaptive_prompting(context: LLMCallHookContext) -> None:
"""반복에 따라 다른 컨텍스트를 추가합니다."""
if context.iterations == 0:
context.messages.append({
"role": "system",
"content": "높은 수준의 개요부터 시작하세요."
})
elif context.iterations > 3:
context.messages.append({
"role": "system",
"content": "구체적인 세부사항에 집중하고 예제를 제공하세요."
})
return None
```
### 훅 체이닝
```python
# 여러 훅은 등록 순서대로 실행됩니다
@before_llm_call
def first_hook(context):
print("1. 첫 번째 훅 실행됨")
return None
@before_llm_call
def second_hook(context):
print("2. 두 번째 훅 실행됨")
return None
@before_llm_call
def blocking_hook(context):
if context.iterations > 10:
print("3. 차단 훅 - 실행 중지")
return False # 후속 훅은 실행되지 않습니다
print("3. 차단 훅 - 실행 허용")
return None
```
## 모범 사례
1. **훅을 집중적으로 유지**: 각 훅은 단일 책임을 가져야 합니다
2. **무거운 계산 피하기**: 훅은 모든 LLM 호출마다 실행됩니다
3. **오류를 우아하게 처리**: try-except를 사용하여 훅 실패로 인한 실행 중단 방지
4. **타입 힌트 사용**: 더 나은 IDE 지원을 위해 `LLMCallHookContext` 활용
5. **훅 동작 문서화**: 특히 차단 조건에 대해
6. **훅을 독립적으로 테스트**: 프로덕션에서 사용하기 전에 단위 테스트
7. **테스트에서 훅 지우기**: 테스트 실행 간 `clear_all_llm_call_hooks()` 사용
8. **제자리에서 수정**: 항상 `context.messages`를 제자리에서 수정하고 교체하지 마세요
## 오류 처리
```python
@before_llm_call
def safe_hook(context: LLMCallHookContext) -> bool | None:
try:
# 훅 로직
if some_condition:
return False
except Exception as e:
print(f"⚠️ 훅 오류: {e}")
# 결정: 오류 발생 시 허용 또는 차단
return None # 오류에도 불구하고 실행 허용
```
## 타입 안전성
```python
from crewai.hooks import LLMCallHookContext, BeforeLLMCallHookType, AfterLLMCallHookType
# 명시적 타입 주석
def my_before_hook(context: LLMCallHookContext) -> bool | None:
return None
def my_after_hook(context: LLMCallHookContext) -> str | None:
return None
# 타입 안전 등록
register_before_llm_call_hook(my_before_hook)
register_after_llm_call_hook(my_after_hook)
```
## 문제 해결
### 훅이 실행되지 않음
- 크루 실행 전에 훅이 등록되었는지 확인
- 이전 훅이 `False`를 반환했는지 확인 (후속 훅 차단)
- 훅 시그니처가 예상 타입과 일치하는지 확인
### 메시지 수정이 지속되지 않음
- 제자리 수정 사용: `context.messages.append()`
- 리스트를 교체하지 마세요: `context.messages = []`
### 응답 수정이 작동하지 않음
- 후 훅에서 수정된 문자열을 반환
- `None`을 반환하면 원본 응답이 유지됩니다
## 결론
LLM 호출 훅은 CrewAI에서 언어 모델 상호작용을 제어하고 모니터링하는 강력한 기능을 제공합니다. 이를 사용하여 안전 가드레일, 승인 게이트, 로깅, 비용 추적 및 응답 정제를 구현하세요. 적절한 오류 처리 및 타입 안전성과 결합하면, 훅을 통해 강력하고 프로덕션 준비가 된 에이전트 시스템을 구축할 수 있습니다.

View File

@@ -0,0 +1,498 @@
---
title: 도구 호출 훅
description: CrewAI에서 도구 실행을 가로채고, 수정하고, 제어하는 도구 호출 훅 사용 방법 배우기
mode: "wide"
---
도구 호출 훅(Tool Call Hooks)은 에이전트 작업 중 도구 실행에 대한 세밀한 제어를 제공합니다. 이러한 훅을 사용하면 도구 호출을 가로채고, 입력을 수정하고, 출력을 변환하고, 안전 검사를 구현하고, 포괄적인 로깅 또는 모니터링을 추가할 수 있습니다.
## 개요
도구 훅은 두 가지 중요한 시점에 실행됩니다:
- **도구 호출 전**: 입력 수정, 매개변수 검증 또는 실행 차단
- **도구 호출 후**: 결과 변환, 출력 정제 또는 실행 세부사항 로깅
## 훅 타입
### 도구 호출 전 훅
모든 도구 실행 전에 실행되며, 다음을 수행할 수 있습니다:
- 도구 입력 검사 및 수정
- 조건에 따라 도구 실행 차단
- 위험한 작업에 대한 승인 게이트 구현
- 매개변수 검증
- 도구 호출 로깅
**시그니처:**
```python
def before_hook(context: ToolCallHookContext) -> bool | None:
# 실행을 차단하려면 False 반환
# 실행을 허용하려면 True 또는 None 반환
...
```
### 도구 호출 후 훅
모든 도구 실행 후에 실행되며, 다음을 수행할 수 있습니다:
- 도구 결과 수정 또는 정제
- 메타데이터 또는 서식 추가
- 실행 결과 로깅
- 결과 검증 구현
- 출력 형식 변환
**시그니처:**
```python
def after_hook(context: ToolCallHookContext) -> str | None:
# 수정된 결과 문자열 반환
# 원본 결과를 유지하려면 None 반환
...
```
## 도구 훅 컨텍스트
`ToolCallHookContext` 객체는 도구 실행 상태에 대한 포괄적인 액세스를 제공합니다:
```python
class ToolCallHookContext:
tool_name: str # 호출되는 도구의 이름
tool_input: dict[str, Any] # 변경 가능한 도구 입력 매개변수
tool: CrewStructuredTool # 도구 인스턴스 참조
agent: Agent | BaseAgent | None # 도구를 실행하는 에이전트
task: Task | None # 현재 작업
crew: Crew | None # 크루 인스턴스
tool_result: str | None # 도구 결과 (후 훅용)
```
### 도구 입력 수정
**중요:** 항상 도구 입력을 제자리에서 수정하세요:
```python
# ✅ 올바름 - 제자리에서 수정
def sanitize_input(context: ToolCallHookContext) -> None:
context.tool_input['query'] = context.tool_input['query'].lower()
# ❌ 잘못됨 - 딕셔너리 참조를 교체
def wrong_approach(context: ToolCallHookContext) -> None:
context.tool_input = {'query': 'new query'}
```
## 등록 방법
### 1. 데코레이터 기반 등록 (권장)
더 깔끔한 구문을 위해 데코레이터를 사용합니다:
```python
from crewai.hooks import before_tool_call, after_tool_call
@before_tool_call
def block_dangerous_tools(context):
"""위험한 도구를 차단합니다."""
dangerous_tools = ['delete_database', 'drop_table', 'rm_rf']
if context.tool_name in dangerous_tools:
print(f"⛔ 위험한 도구 차단됨: {context.tool_name}")
return False # 실행 차단
return None
@after_tool_call
def sanitize_results(context):
"""결과를 정제합니다."""
if context.tool_result and "password" in context.tool_result.lower():
return context.tool_result.replace("password", "[수정됨]")
return None
```
### 2. 크루 범위 훅
특정 크루 인스턴스에 대한 훅을 등록합니다:
```python
from crewai import CrewBase
from crewai.project import crew
from crewai.hooks import before_tool_call_crew, after_tool_call_crew
@CrewBase
class MyProjCrew:
@before_tool_call_crew
def validate_tool_inputs(self, context):
# 이 크루에만 적용됩니다
if context.tool_name == "web_search":
if not context.tool_input.get('query'):
print("❌ 잘못된 검색 쿼리")
return False
return None
@after_tool_call_crew
def log_tool_results(self, context):
# 크루별 도구 로깅
print(f"✅ {context.tool_name} 완료됨")
return None
@crew
def crew(self) -> Crew:
return Crew(
agents=self.agents,
tasks=self.tasks,
process=Process.sequential,
verbose=True
)
```
## 일반적인 사용 사례
### 1. 안전 가드레일
```python
@before_tool_call
def safety_check(context: ToolCallHookContext) -> bool | None:
"""해를 끼칠 수 있는 도구를 차단합니다."""
destructive_tools = [
'delete_file',
'drop_table',
'remove_user',
'system_shutdown'
]
if context.tool_name in destructive_tools:
print(f"🛑 파괴적인 도구 차단됨: {context.tool_name}")
return False
# 민감한 작업에 대해 경고
sensitive_tools = ['send_email', 'post_to_social_media', 'charge_payment']
if context.tool_name in sensitive_tools:
print(f"⚠️ 민감한 도구 실행 중: {context.tool_name}")
return None
```
### 2. 사람의 승인 게이트
```python
@before_tool_call
def require_approval_for_actions(context: ToolCallHookContext) -> bool | None:
"""특정 작업에 대한 승인을 요구합니다."""
approval_required = [
'send_email',
'make_purchase',
'delete_file',
'post_message'
]
if context.tool_name in approval_required:
response = context.request_human_input(
prompt=f"{context.tool_name}을(를) 승인하시겠습니까?",
default_message=f"입력: {context.tool_input}\n승인하려면 'yes'를 입력하세요:"
)
if response.lower() != 'yes':
print(f"❌ 도구 실행 거부됨: {context.tool_name}")
return False
return None
```
### 3. 입력 검증 및 정제
```python
@before_tool_call
def validate_and_sanitize_inputs(context: ToolCallHookContext) -> bool | None:
"""입력을 검증하고 정제합니다."""
# 검색 쿼리 검증
if context.tool_name == 'web_search':
query = context.tool_input.get('query', '')
if len(query) < 3:
print("❌ 검색 쿼리가 너무 짧습니다")
return False
# 쿼리 정제
context.tool_input['query'] = query.strip().lower()
# 파일 경로 검증
if context.tool_name == 'read_file':
path = context.tool_input.get('path', '')
if '..' in path or path.startswith('/'):
print("❌ 잘못된 파일 경로")
return False
return None
```
### 4. 결과 정제
```python
@after_tool_call
def sanitize_sensitive_data(context: ToolCallHookContext) -> str | None:
"""민감한 데이터를 정제합니다."""
if not context.tool_result:
return None
import re
result = context.tool_result
# API 키 제거
result = re.sub(
r'(api[_-]?key|token)["\']?\s*[:=]\s*["\']?[\w-]+',
r'\1: [수정됨]',
result,
flags=re.IGNORECASE
)
# 이메일 주소 제거
result = re.sub(
r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b',
'[이메일-수정됨]',
result
)
# 신용카드 번호 제거
result = re.sub(
r'\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b',
'[카드-수정됨]',
result
)
return result
```
### 5. 도구 사용 분석
```python
import time
from collections import defaultdict
tool_stats = defaultdict(lambda: {'count': 0, 'total_time': 0, 'failures': 0})
@before_tool_call
def start_timer(context: ToolCallHookContext) -> None:
context.tool_input['_start_time'] = time.time()
return None
@after_tool_call
def track_tool_usage(context: ToolCallHookContext) -> None:
start_time = context.tool_input.get('_start_time', time.time())
duration = time.time() - start_time
tool_stats[context.tool_name]['count'] += 1
tool_stats[context.tool_name]['total_time'] += duration
if not context.tool_result or 'error' in context.tool_result.lower():
tool_stats[context.tool_name]['failures'] += 1
print(f"""
📊 {context.tool_name} 도구 통계:
- 실행 횟수: {tool_stats[context.tool_name]['count']}
- 평균 시간: {tool_stats[context.tool_name]['total_time'] / tool_stats[context.tool_name]['count']:.2f}초
- 실패: {tool_stats[context.tool_name]['failures']}
""")
return None
```
### 6. 속도 제한
```python
from collections import defaultdict
from datetime import datetime, timedelta
tool_call_history = defaultdict(list)
@before_tool_call
def rate_limit_tools(context: ToolCallHookContext) -> bool | None:
"""도구 호출 속도를 제한합니다."""
tool_name = context.tool_name
now = datetime.now()
# 오래된 항목 정리 (1분 이상 된 것)
tool_call_history[tool_name] = [
call_time for call_time in tool_call_history[tool_name]
if now - call_time < timedelta(minutes=1)
]
# 속도 제한 확인 (분당 최대 10회 호출)
if len(tool_call_history[tool_name]) >= 10:
print(f"🚫 {tool_name}에 대한 속도 제한 초과")
return False
# 이 호출 기록
tool_call_history[tool_name].append(now)
return None
```
### 7. 디버그 로깅
```python
@before_tool_call
def debug_tool_call(context: ToolCallHookContext) -> None:
"""도구 호출을 디버그합니다."""
print(f"""
🔍 도구 호출 디버그:
- 도구: {context.tool_name}
- 에이전트: {context.agent.role if context.agent else '알 수 없음'}
- 작업: {context.task.description[:50] if context.task else '알 수 없음'}...
- 입력: {context.tool_input}
""")
return None
@after_tool_call
def debug_tool_result(context: ToolCallHookContext) -> None:
"""도구 결과를 디버그합니다."""
if context.tool_result:
result_preview = context.tool_result[:200]
print(f"✅ 결과 미리보기: {result_preview}...")
else:
print("⚠️ 반환된 결과 없음")
return None
```
## 훅 관리
### 훅 등록 해제
```python
from crewai.hooks import (
unregister_before_tool_call_hook,
unregister_after_tool_call_hook
)
# 특정 훅 등록 해제
def my_hook(context):
...
register_before_tool_call_hook(my_hook)
# 나중에...
success = unregister_before_tool_call_hook(my_hook)
print(f"등록 해제됨: {success}")
```
### 훅 지우기
```python
from crewai.hooks import (
clear_before_tool_call_hooks,
clear_after_tool_call_hooks,
clear_all_tool_call_hooks
)
# 특정 훅 타입 지우기
count = clear_before_tool_call_hooks()
print(f"{count}개의 전(before) 훅이 지워졌습니다")
# 모든 도구 훅 지우기
before_count, after_count = clear_all_tool_call_hooks()
print(f"{before_count}개의 전(before) 훅과 {after_count}개의 후(after) 훅이 지워졌습니다")
```
## 고급 패턴
### 조건부 훅 실행
```python
@before_tool_call
def conditional_blocking(context: ToolCallHookContext) -> bool | None:
"""특정 조건에서만 차단합니다."""
# 특정 에이전트에 대해서만 차단
if context.agent and context.agent.role == "junior_agent":
if context.tool_name in ['delete_file', 'send_email']:
print(f"❌ 주니어 에이전트는 {context.tool_name}을(를) 사용할 수 없습니다")
return False
# 특정 작업 중에만 차단
if context.task and "민감한" in context.task.description.lower():
if context.tool_name == 'web_search':
print("❌ 민감한 작업에서는 웹 검색이 차단됩니다")
return False
return None
```
### 컨텍스트 인식 입력 수정
```python
@before_tool_call
def enhance_tool_inputs(context: ToolCallHookContext) -> None:
"""에이전트 역할에 따라 컨텍스트를 추가합니다."""
# 에이전트 역할에 따라 컨텍스트 추가
if context.agent and context.agent.role == "researcher":
if context.tool_name == 'web_search':
# 연구원에 대한 도메인 제한 추가
context.tool_input['domains'] = ['edu', 'gov', 'org']
# 작업에 따라 컨텍스트 추가
if context.task and "긴급" in context.task.description.lower():
if context.tool_name == 'send_email':
context.tool_input['priority'] = 'high'
return None
```
## 모범 사례
1. **훅을 집중적으로 유지**: 각 훅은 단일 책임을 가져야 합니다
2. **무거운 계산 피하기**: 훅은 모든 도구 호출마다 실행됩니다
3. **오류를 우아하게 처리**: try-except를 사용하여 훅 실패 방지
4. **타입 힌트 사용**: 더 나은 IDE 지원을 위해 `ToolCallHookContext` 활용
5. **차단 조건 문서화**: 도구가 차단되는 시기/이유를 명확히 하세요
6. **훅을 독립적으로 테스트**: 프로덕션에서 사용하기 전에 단위 테스트
7. **테스트에서 훅 지우기**: 테스트 실행 간 `clear_all_tool_call_hooks()` 사용
8. **제자리에서 수정**: 항상 `context.tool_input`을 제자리에서 수정하고 교체하지 마세요
9. **중요한 결정 로깅**: 특히 도구 실행을 차단할 때
10. **성능 고려**: 가능한 경우 비용이 많이 드는 검증을 캐시
## 오류 처리
```python
@before_tool_call
def safe_validation(context: ToolCallHookContext) -> bool | None:
try:
# 검증 로직
if not validate_input(context.tool_input):
return False
except Exception as e:
print(f"⚠️ 훅 오류: {e}")
# 결정: 오류 발생 시 허용 또는 차단
return None # 오류에도 불구하고 실행 허용
```
## 타입 안전성
```python
from crewai.hooks import ToolCallHookContext, BeforeToolCallHookType, AfterToolCallHookType
# 명시적 타입 주석
def my_before_hook(context: ToolCallHookContext) -> bool | None:
return None
def my_after_hook(context: ToolCallHookContext) -> str | None:
return None
# 타입 안전 등록
register_before_tool_call_hook(my_before_hook)
register_after_tool_call_hook(my_after_hook)
```
## 문제 해결
### 훅이 실행되지 않음
- 크루 실행 전에 훅이 등록되었는지 확인
- 이전 훅이 `False`를 반환했는지 확인 (실행 및 후속 훅 차단)
- 훅 시그니처가 예상 타입과 일치하는지 확인
### 입력 수정이 작동하지 않음
- 제자리 수정 사용: `context.tool_input['key'] = value`
- 딕셔너리를 교체하지 마세요: `context.tool_input = {}`
### 결과 수정이 작동하지 않음
- 후 훅에서 수정된 문자열을 반환
- `None`을 반환하면 원본 결과가 유지됩니다
- 도구가 실제로 결과를 반환했는지 확인
### 도구가 예기치 않게 차단됨
- 차단 조건에 대한 모든 전(before) 훅 확인
- 훅 실행 순서 확인
- 어떤 훅이 차단하는지 식별하기 위해 디버그 로깅 추가
## 결론
도구 호출 훅은 CrewAI에서 도구 실행을 제어하고 모니터링하는 강력한 기능을 제공합니다. 이를 사용하여 안전 가드레일, 승인 게이트, 입력 검증, 결과 정제, 로깅 및 분석을 구현하세요. 적절한 오류 처리 및 타입 안전성과 결합하면, 훅을 통해 포괄적인 관찰성을 갖춘 안전하고 프로덕션 준비가 된 에이전트 시스템을 구축할 수 있습니다.

View File

@@ -730,9 +730,7 @@ Portkey 대시보드에서 [구성 페이지](https://app.portkey.ai/configs)에
- 로그를 필터링하기 위한 관련 메타데이터 수집
- 액세스 권한 적용
API 키 생성 방법:
- [Portkey App](https://app.portkey.ai/)
- [API Key Management API](/ko/api-reference/admin-api/control-plane/api-keys/create-api-key)
[Portkey App](https://app.portkey.ai/)를 통해 API 키 생성하세요
Python SDK를 사용한 예시:
```python
@@ -755,7 +753,7 @@ api_key = portkey.api_keys.create(
)
```
자세한 키 관리 방법은 [API 키 문서](/ko/api-reference/admin-api/control-plane/api-keys/create-api-key)를 참조하세요.
자세한 키 관리 방법은 [Portkey 문서](https://portkey.ai/docs)를 참조하세요.
</Accordion>
<Accordion title="4단계: 배포 및 모니터링">

View File

@@ -18,7 +18,7 @@ mode: "wide"
파일을 Amazon S3 스토리지에 작성하고 업로드합니다.
</Card>
<Card title="Bedrock Invoke Agent" icon="aws" href="/ko/tools/cloud-storage/bedrockinvokeagenttool">
<Card title="Bedrock Invoke Agent" icon="aws" href="/ko/tools/integration/bedrockinvokeagenttool">
AI 기반 작업을 위해 Amazon Bedrock 에이전트를 호출합니다.
</Card>

View File

@@ -11,7 +11,7 @@ mode: "wide"
<Card
title="Bedrock Invoke Agent Tool"
icon="cloud"
href="/en/tools/tool-integrations/bedrockinvokeagenttool"
href="/ko/tools/integration/bedrockinvokeagenttool"
color="#0891B2"
>
Invoke Amazon Bedrock Agents from CrewAI to orchestrate actions across AWS services.
@@ -20,7 +20,7 @@ mode: "wide"
<Card
title="CrewAI Automation Tool"
icon="bolt"
href="/en/tools/tool-integrations/crewaiautomationtool"
href="/ko/tools/integration/crewaiautomationtool"
color="#7C3AED"
>
Automate deployment and operations by integrating CrewAI with external platforms and workflows.

View File

@@ -704,7 +704,7 @@ class KnowledgeMonitorListener(BaseEventListener):
knowledge_monitor = KnowledgeMonitorListener()
```
Para mais informações sobre como usar eventos, consulte a documentação [Event Listeners](https://docs.crewai.com/concepts/event-listener).
Para mais informações sobre como usar eventos, consulte a documentação [Event Listeners](/pt-BR/concepts/event-listener).
### Fontes de Knowledge Personalizadas

View File

@@ -725,7 +725,7 @@ O CrewAI suporta respostas em streaming de LLMs, permitindo que sua aplicação
```
<Tip>
[Clique aqui](https://docs.crewai.com/concepts/event-listener#event-listeners) para mais detalhes
[Clique aqui](/pt-BR/concepts/event-listener#event-listeners) para mais detalhes
</Tip>
</Tab>
</Tabs>

View File

@@ -36,7 +36,7 @@ Você também pode baixar templates diretamente do marketplace clicando em `Down
<Card title="Ferramentas & Integrações" href="/pt-BR/enterprise/features/tools-and-integrations" icon="wrench">
Conecte apps externos e gerencie ferramentas internas que seus agentes podem usar.
</Card>
<Card title="Repositório de Ferramentas" href="/pt-BR/enterprise/features/tool-repository" icon="toolbox">
<Card title="Repositório de Ferramentas" href="/pt-BR/enterprise/guides/tool-repository" icon="toolbox">
Publique e instale ferramentas para ampliar as capacidades dos seus crews.
</Card>
<Card title="Repositório de Agentes" href="/pt-BR/enterprise/features/agent-repositories" icon="people-group">

View File

@@ -231,7 +231,7 @@ Ferramentas & Integrações é o hub central para conectar aplicações de terce
## Relacionados
<CardGroup cols={2}>
<Card title="Repositório de Ferramentas" href="/pt-BR/enterprise/features/tool-repository" icon="toolbox">
<Card title="Repositório de Ferramentas" href="/pt-BR/enterprise/guides/tool-repository" icon="toolbox">
Publique e instale ferramentas para ampliar as capacidades dos seus crews.
</Card>
<Card title="Automação com Webhook" href="/pt-BR/enterprise/guides/webhook-automation" icon="bolt">

View File

@@ -21,7 +21,7 @@ O repositório não é um sistema de controle de versões. Use Git para rastrear
Antes de usar o Repositório de Ferramentas, certifique-se de que você possui:
- Uma conta [CrewAI AMP](https://app.crewai.com)
- [CrewAI CLI](https://docs.crewai.com/concepts/cli#cli) instalada
- [CrewAI CLI](/pt-BR/concepts/cli#cli) instalada
- uv>=0.5.0 instalado. Veja [como atualizar](https://docs.astral.sh/uv/getting-started/installation/#upgrading-uv)
- [Git](https://git-scm.com) instalado e configurado
- Permissões de acesso para publicar ou instalar ferramentas em sua organização CrewAI AMP
@@ -66,7 +66,7 @@ Por padrão, as ferramentas são publicadas como privadas. Para tornar uma ferra
crewai tool publish --public
```
Para mais detalhes sobre como construir ferramentas, acesse [Criando suas próprias ferramentas](https://docs.crewai.com/concepts/tools#creating-your-own-tools).
Para mais detalhes sobre como construir ferramentas, acesse [Criando suas próprias ferramentas](/pt-BR/concepts/tools#creating-your-own-tools).
## Atualizando ferramentas

View File

@@ -49,7 +49,7 @@ mode: "wide"
Para integrar a entrada humana na execução do agente, defina a flag `human_input` na definição da tarefa. Quando habilitada, o agente solicitará a entrada do usuário antes de entregar sua resposta final. Essa entrada pode fornecer contexto extra, esclarecer ambiguidades ou validar a saída do agente.
Para orientações detalhadas de implementação, veja nosso [guia Human-in-the-Loop](/pt-BR/how-to/human-in-the-loop).
Para orientações detalhadas de implementação, veja nosso [guia Human-in-the-Loop](/pt-BR/enterprise/guides/human-in-the-loop).
</Accordion>
<Accordion title="Quais opções avançadas de customização estão disponíveis para aprimorar e personalizar o comportamento e as capacidades dos agentes na CrewAI?">
@@ -142,7 +142,7 @@ mode: "wide"
<Accordion title="Como posso criar ferramentas personalizadas para meus agentes CrewAI?">
Você pode criar ferramentas personalizadas herdando da classe `BaseTool` fornecida pela CrewAI ou usando o decorador de ferramenta. Herdar envolve definir uma nova classe que herda de `BaseTool`, especificando o nome, a descrição e o método `_run` para a lógica operacional. O decorador de ferramenta permite criar um objeto `Tool` diretamente com os atributos necessários e uma lógica funcional.
<Card href="https://docs.crewai.com/how-to/create-custom-tools" icon="code">CrewAI Tools Guide</Card>
<Card href="/pt-BR/learn/create-custom-tools" icon="code">CrewAI Tools Guide</Card>
</Accordion>
<Accordion title="Como controlar o número máximo de solicitações por minuto que toda a crew pode realizar?">

View File

@@ -0,0 +1,379 @@
---
title: Visão Geral dos Hooks de Execução
description: Entendendo e usando hooks de execução no CrewAI para controle fino sobre operações de agentes
mode: "wide"
---
Os Hooks de Execução fornecem controle fino sobre o comportamento em tempo de execução dos seus agentes CrewAI. Diferentemente dos hooks de kickoff que são executados antes e depois da execução da crew, os hooks de execução interceptam operações específicas durante a execução do agente, permitindo que você modifique comportamentos, implemente verificações de segurança e adicione monitoramento abrangente.
## Tipos de Hooks de Execução
O CrewAI fornece duas categorias principais de hooks de execução:
### 1. [Hooks de Chamada LLM](/learn/llm-hooks)
Controle e monitore interações com o modelo de linguagem:
- **Antes da Chamada LLM**: Modifique prompts, valide entradas, implemente gates de aprovação
- **Depois da Chamada LLM**: Transforme respostas, sanitize saídas, atualize histórico de conversação
**Casos de Uso:**
- Limitação de iterações
- Rastreamento de custos e monitoramento de uso de tokens
- Sanitização de respostas e filtragem de conteúdo
- Aprovação humana para chamadas LLM
- Adição de diretrizes de segurança ou contexto
- Logging de debug e inspeção de requisição/resposta
[Ver Documentação de Hooks LLM →](/learn/llm-hooks)
### 2. [Hooks de Chamada de Ferramenta](/learn/tool-hooks)
Controle e monitore execução de ferramentas:
- **Antes da Chamada de Ferramenta**: Modifique entradas, valide parâmetros, bloqueie operações perigosas
- **Depois da Chamada de Ferramenta**: Transforme resultados, sanitize saídas, registre detalhes de execução
**Casos de Uso:**
- Guardrails de segurança para operações destrutivas
- Aprovação humana para ações sensíveis
- Validação e sanitização de entrada
- Cache de resultados e limitação de taxa
- Análise de uso de ferramentas
- Logging de debug e monitoramento
[Ver Documentação de Hooks de Ferramenta →](/learn/tool-hooks)
## Métodos de Registro
### 1. Hooks Baseados em Decoradores (Recomendado)
A maneira mais limpa e pythônica de registrar hooks:
```python
from crewai.hooks import before_llm_call, after_llm_call, before_tool_call, after_tool_call
@before_llm_call
def limit_iterations(context):
"""Previne loops infinitos limitando iterações."""
if context.iterations > 10:
return False # Bloquear execução
return None
@after_llm_call
def sanitize_response(context):
"""Remove dados sensíveis das respostas do LLM."""
if "API_KEY" in context.response:
return context.response.replace("API_KEY", "[CENSURADO]")
return None
@before_tool_call
def block_dangerous_tools(context):
"""Bloqueia operações destrutivas."""
if context.tool_name == "delete_database":
return False # Bloquear execução
return None
@after_tool_call
def log_tool_result(context):
"""Registra execução de ferramenta."""
print(f"Ferramenta {context.tool_name} concluída")
return None
```
### 2. Hooks com Escopo de Crew
Aplica hooks apenas a instâncias específicas de crew:
```python
from crewai import CrewBase
from crewai.project import crew
from crewai.hooks import before_llm_call_crew, after_tool_call_crew
@CrewBase
class MyProjCrew:
@before_llm_call_crew
def validate_inputs(self, context):
# Aplica-se apenas a esta crew
print(f"Chamada LLM em {self.__class__.__name__}")
return None
@after_tool_call_crew
def log_results(self, context):
# Logging específico da crew
print(f"Resultado da ferramenta: {context.tool_result[:50]}...")
return None
@crew
def crew(self) -> Crew:
return Crew(
agents=self.agents,
tasks=self.tasks,
process=Process.sequential
)
```
## Fluxo de Execução de Hooks
### Fluxo de Chamada LLM
```
Agente precisa chamar LLM
[Hooks Antes da Chamada LLM Executam]
├→ Hook 1: Validar contagem de iterações
├→ Hook 2: Adicionar contexto de segurança
└→ Hook 3: Registrar requisição
Se algum hook retornar False:
├→ Bloquear chamada LLM
└→ Lançar ValueError
Se todos os hooks retornarem True/None:
├→ Chamada LLM prossegue
└→ Resposta gerada
[Hooks Depois da Chamada LLM Executam]
├→ Hook 1: Sanitizar resposta
├→ Hook 2: Registrar resposta
└→ Hook 3: Atualizar métricas
Resposta final retornada
```
### Fluxo de Chamada de Ferramenta
```
Agente precisa executar ferramenta
[Hooks Antes da Chamada de Ferramenta Executam]
├→ Hook 1: Verificar se ferramenta é permitida
├→ Hook 2: Validar entradas
└→ Hook 3: Solicitar aprovação se necessário
Se algum hook retornar False:
├→ Bloquear execução da ferramenta
└→ Retornar mensagem de erro
Se todos os hooks retornarem True/None:
├→ Execução da ferramenta prossegue
└→ Resultado gerado
[Hooks Depois da Chamada de Ferramenta Executam]
├→ Hook 1: Sanitizar resultado
├→ Hook 2: Fazer cache do resultado
└→ Hook 3: Registrar métricas
Resultado final retornado
```
## Objetos de Contexto de Hook
### LLMCallHookContext
Fornece acesso ao estado de execução do LLM:
```python
class LLMCallHookContext:
executor: CrewAgentExecutor # Acesso completo ao executor
messages: list # Lista de mensagens mutável
agent: Agent # Agente atual
task: Task # Tarefa atual
crew: Crew # Instância da crew
llm: BaseLLM # Instância do LLM
iterations: int # Iteração atual
response: str | None # Resposta do LLM (hooks posteriores)
```
### ToolCallHookContext
Fornece acesso ao estado de execução da ferramenta:
```python
class ToolCallHookContext:
tool_name: str # Ferramenta sendo chamada
tool_input: dict # Parâmetros de entrada mutáveis
tool: CrewStructuredTool # Instância da ferramenta
agent: Agent | None # Agente executando
task: Task | None # Tarefa atual
crew: Crew | None # Instância da crew
tool_result: str | None # Resultado da ferramenta (hooks posteriores)
```
## Padrões Comuns
### Segurança e Validação
```python
@before_tool_call
def safety_check(context):
"""Bloqueia operações destrutivas."""
dangerous = ['delete_file', 'drop_table', 'system_shutdown']
if context.tool_name in dangerous:
print(f"🛑 Bloqueado: {context.tool_name}")
return False
return None
@before_llm_call
def iteration_limit(context):
"""Previne loops infinitos."""
if context.iterations > 15:
print("⛔ Máximo de iterações excedido")
return False
return None
```
### Humano no Loop
```python
@before_tool_call
def require_approval(context):
"""Requer aprovação para operações sensíveis."""
sensitive = ['send_email', 'make_payment', 'post_message']
if context.tool_name in sensitive:
response = context.request_human_input(
prompt=f"Aprovar {context.tool_name}?",
default_message="Digite 'sim' para aprovar:"
)
if response.lower() != 'sim':
return False
return None
```
### Monitoramento e Análise
```python
from collections import defaultdict
import time
metrics = defaultdict(lambda: {'count': 0, 'total_time': 0})
@before_tool_call
def start_timer(context):
context.tool_input['_start'] = time.time()
return None
@after_tool_call
def track_metrics(context):
start = context.tool_input.get('_start', time.time())
duration = time.time() - start
metrics[context.tool_name]['count'] += 1
metrics[context.tool_name]['total_time'] += duration
return None
```
## Gerenciamento de Hooks
### Limpar Todos os Hooks
```python
from crewai.hooks import clear_all_global_hooks
# Limpa todos os hooks de uma vez
result = clear_all_global_hooks()
print(f"Limpou {result['total']} hooks")
```
### Limpar Tipos Específicos de Hooks
```python
from crewai.hooks import (
clear_before_llm_call_hooks,
clear_after_llm_call_hooks,
clear_before_tool_call_hooks,
clear_after_tool_call_hooks
)
# Limpar tipos específicos
llm_before_count = clear_before_llm_call_hooks()
tool_after_count = clear_after_tool_call_hooks()
```
## Melhores Práticas
### 1. Mantenha os Hooks Focados
Cada hook deve ter uma responsabilidade única e clara.
### 2. Trate Erros Graciosamente
```python
@before_llm_call
def safe_hook(context):
try:
if some_condition:
return False
except Exception as e:
print(f"Erro no hook: {e}")
return None # Permitir execução apesar do erro
```
### 3. Modifique o Contexto In-Place
```python
# ✅ Correto - modificar in-place
@before_llm_call
def add_context(context):
context.messages.append({"role": "system", "content": "Seja conciso"})
# ❌ Errado - substitui referência
@before_llm_call
def wrong_approach(context):
context.messages = [{"role": "system", "content": "Seja conciso"}]
```
### 4. Use Type Hints
```python
from crewai.hooks import LLMCallHookContext, ToolCallHookContext
def my_llm_hook(context: LLMCallHookContext) -> bool | None:
return None
def my_tool_hook(context: ToolCallHookContext) -> str | None:
return None
```
### 5. Limpe em Testes
```python
import pytest
from crewai.hooks import clear_all_global_hooks
@pytest.fixture(autouse=True)
def clean_hooks():
"""Reseta hooks antes de cada teste."""
yield
clear_all_global_hooks()
```
## Quando Usar Qual Hook
### Use Hooks LLM Quando:
- Implementar limites de iteração
- Adicionar contexto ou diretrizes de segurança aos prompts
- Rastrear uso de tokens e custos
- Sanitizar ou transformar respostas
- Implementar gates de aprovação para chamadas LLM
- Fazer debug de interações de prompt/resposta
### Use Hooks de Ferramenta Quando:
- Bloquear operações perigosas ou destrutivas
- Validar entradas de ferramenta antes da execução
- Implementar gates de aprovação para ações sensíveis
- Fazer cache de resultados de ferramenta
- Rastrear uso e performance de ferramentas
- Sanitizar saídas de ferramenta
- Limitar taxa de chamadas de ferramenta
### Use Ambos Quando:
Construir sistemas abrangentes de observabilidade, segurança ou aprovação que precisam monitorar todas as operações do agente.
## Documentação Relacionada
- [Hooks de Chamada LLM →](/learn/llm-hooks) - Documentação detalhada de hooks LLM
- [Hooks de Chamada de Ferramenta →](/learn/tool-hooks) - Documentação detalhada de hooks de ferramenta
- [Hooks Antes e Depois do Kickoff →](/learn/before-and-after-kickoff-hooks) - Hooks do ciclo de vida da crew
- [Humano no Loop →](/learn/human-in-the-loop) - Padrões de entrada humana
## Conclusão
Os Hooks de Execução fornecem controle poderoso sobre o comportamento em tempo de execução do agente. Use-os para implementar guardrails de segurança, fluxos de trabalho de aprovação, monitoramento abrangente e lógica de negócio personalizada. Combinados com tratamento adequado de erros, segurança de tipos e considerações de performance, os hooks permitem sistemas de agentes seguros, prontos para produção e observáveis.

View File

@@ -96,7 +96,7 @@ project_crew = Crew(
```
<Tip>
Para mais detalhes sobre a criação e personalização de um agente gerente, confira a [documentação do Custom Manager Agent](https://docs.crewai.com/how-to/custom-manager-agent#custom-manager-agent).
Para mais detalhes sobre a criação e personalização de um agente gerente, confira a [documentação do Custom Manager Agent](/pt-BR/learn/custom-manager-agent).
</Tip>

View File

@@ -0,0 +1,388 @@
---
title: Hooks de Chamada LLM
description: Aprenda a usar hooks de chamada LLM para interceptar, modificar e controlar interações com modelos de linguagem no CrewAI
mode: "wide"
---
Os Hooks de Chamada LLM fornecem controle fino sobre interações com modelos de linguagem durante a execução do agente. Esses hooks permitem interceptar chamadas LLM, modificar prompts, transformar respostas, implementar gates de aprovação e adicionar logging ou monitoramento personalizado.
## Visão Geral
Os hooks LLM são executados em dois pontos críticos:
- **Antes da Chamada LLM**: Modificar mensagens, validar entradas ou bloquear execução
- **Depois da Chamada LLM**: Transformar respostas, sanitizar saídas ou modificar histórico de conversação
## Tipos de Hook
### Hooks Antes da Chamada LLM
Executados antes de cada chamada LLM, esses hooks podem:
- Inspecionar e modificar mensagens enviadas ao LLM
- Bloquear execução LLM com base em condições
- Implementar limitação de taxa ou gates de aprovação
- Adicionar contexto ou mensagens do sistema
- Registrar detalhes da requisição
**Assinatura:**
```python
def before_hook(context: LLMCallHookContext) -> bool | None:
# Retorne False para bloquear execução
# Retorne True ou None para permitir execução
...
```
### Hooks Depois da Chamada LLM
Executados depois de cada chamada LLM, esses hooks podem:
- Modificar ou sanitizar respostas do LLM
- Adicionar metadados ou formatação
- Registrar detalhes da resposta
- Atualizar histórico de conversação
- Implementar filtragem de conteúdo
**Assinatura:**
```python
def after_hook(context: LLMCallHookContext) -> str | None:
# Retorne string de resposta modificada
# Retorne None para manter resposta original
...
```
## Contexto do Hook LLM
O objeto `LLMCallHookContext` fornece acesso abrangente ao estado de execução:
```python
class LLMCallHookContext:
executor: CrewAgentExecutor # Referência completa ao executor
messages: list # Lista de mensagens mutável
agent: Agent # Agente atual
task: Task # Tarefa atual
crew: Crew # Instância da crew
llm: BaseLLM # Instância do LLM
iterations: int # Contagem de iteração atual
response: str | None # Resposta do LLM (apenas hooks posteriores)
```
### Modificando Mensagens
**Importante:** Sempre modifique mensagens in-place:
```python
# ✅ Correto - modificar in-place
def add_context(context: LLMCallHookContext) -> None:
context.messages.append({"role": "system", "content": "Seja conciso"})
# ❌ Errado - substitui referência da lista
def wrong_approach(context: LLMCallHookContext) -> None:
context.messages = [{"role": "system", "content": "Seja conciso"}]
```
## Métodos de Registro
### 1. Registro Baseado em Decoradores (Recomendado)
Use decoradores para sintaxe mais limpa:
```python
from crewai.hooks import before_llm_call, after_llm_call
@before_llm_call
def validate_iteration_count(context):
"""Valida a contagem de iterações."""
if context.iterations > 10:
print("⚠️ Máximo de iterações excedido")
return False # Bloquear execução
return None
@after_llm_call
def sanitize_response(context):
"""Remove dados sensíveis."""
if context.response and "API_KEY" in context.response:
return context.response.replace("API_KEY", "[CENSURADO]")
return None
```
### 2. Hooks com Escopo de Crew
Registre hooks para uma instância específica de crew:
```python
from crewai import CrewBase
from crewai.project import crew
from crewai.hooks import before_llm_call_crew, after_llm_call_crew
@CrewBase
class MyProjCrew:
@before_llm_call_crew
def validate_inputs(self, context):
# Aplica-se apenas a esta crew
if context.iterations == 0:
print(f"Iniciando tarefa: {context.task.description}")
return None
@after_llm_call_crew
def log_responses(self, context):
# Logging específico da crew
print(f"Comprimento da resposta: {len(context.response)}")
return None
@crew
def crew(self) -> Crew:
return Crew(
agents=self.agents,
tasks=self.tasks,
process=Process.sequential,
verbose=True
)
```
## Casos de Uso Comuns
### 1. Limitação de Iterações
```python
@before_llm_call
def limit_iterations(context: LLMCallHookContext) -> bool | None:
"""Previne loops infinitos limitando iterações."""
max_iterations = 15
if context.iterations > max_iterations:
print(f"⛔ Bloqueado: Excedeu {max_iterations} iterações")
return False # Bloquear execução
return None
```
### 2. Gate de Aprovação Humana
```python
@before_llm_call
def require_approval(context: LLMCallHookContext) -> bool | None:
"""Requer aprovação após certas iterações."""
if context.iterations > 5:
response = context.request_human_input(
prompt=f"Iteração {context.iterations}: Aprovar chamada LLM?",
default_message="Pressione Enter para aprovar, ou digite 'não' para bloquear:"
)
if response.lower() == "não":
print("🚫 Chamada LLM bloqueada pelo usuário")
return False
return None
```
### 3. Adicionando Contexto do Sistema
```python
@before_llm_call
def add_guardrails(context: LLMCallHookContext) -> None:
"""Adiciona diretrizes de segurança a cada chamada LLM."""
context.messages.append({
"role": "system",
"content": "Garanta que as respostas sejam factuais e cite fontes quando possível."
})
return None
```
### 4. Sanitização de Resposta
```python
@after_llm_call
def sanitize_sensitive_data(context: LLMCallHookContext) -> str | None:
"""Remove padrões sensíveis."""
if not context.response:
return None
import re
sanitized = context.response
sanitized = re.sub(r'\b\d{3}\.\d{3}\.\d{3}-\d{2}\b', '[CPF-CENSURADO]', sanitized)
sanitized = re.sub(r'\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b', '[CARTÃO-CENSURADO]', sanitized)
return sanitized
```
### 5. Rastreamento de Custos
```python
import tiktoken
@before_llm_call
def track_token_usage(context: LLMCallHookContext) -> None:
"""Rastreia tokens de entrada."""
encoding = tiktoken.get_encoding("cl100k_base")
total_tokens = sum(
len(encoding.encode(msg.get("content", "")))
for msg in context.messages
)
print(f"📊 Tokens de entrada: ~{total_tokens}")
return None
@after_llm_call
def track_response_tokens(context: LLMCallHookContext) -> None:
"""Rastreia tokens de resposta."""
if context.response:
encoding = tiktoken.get_encoding("cl100k_base")
tokens = len(encoding.encode(context.response))
print(f"📊 Tokens de resposta: ~{tokens}")
return None
```
### 6. Logging de Debug
```python
@before_llm_call
def debug_request(context: LLMCallHookContext) -> None:
"""Debug de requisição LLM."""
print(f"""
🔍 Debug de Chamada LLM:
- Agente: {context.agent.role}
- Tarefa: {context.task.description[:50]}...
- Iteração: {context.iterations}
- Contagem de Mensagens: {len(context.messages)}
- Última Mensagem: {context.messages[-1] if context.messages else 'Nenhuma'}
""")
return None
@after_llm_call
def debug_response(context: LLMCallHookContext) -> None:
"""Debug de resposta LLM."""
if context.response:
print(f"✅ Preview da Resposta: {context.response[:100]}...")
return None
```
## Gerenciamento de Hooks
### Desregistrando Hooks
```python
from crewai.hooks import (
unregister_before_llm_call_hook,
unregister_after_llm_call_hook
)
# Desregistrar hook específico
def my_hook(context):
...
register_before_llm_call_hook(my_hook)
# Mais tarde...
unregister_before_llm_call_hook(my_hook) # Retorna True se encontrado
```
### Limpando Hooks
```python
from crewai.hooks import (
clear_before_llm_call_hooks,
clear_after_llm_call_hooks,
clear_all_llm_call_hooks
)
# Limpar tipo específico de hook
count = clear_before_llm_call_hooks()
print(f"Limpou {count} hooks antes")
# Limpar todos os hooks LLM
before_count, after_count = clear_all_llm_call_hooks()
print(f"Limpou {before_count} hooks antes e {after_count} hooks depois")
```
## Padrões Avançados
### Execução Condicional de Hook
```python
@before_llm_call
def conditional_blocking(context: LLMCallHookContext) -> bool | None:
"""Bloqueia apenas em condições específicas."""
# Bloquear apenas para agentes específicos
if context.agent.role == "researcher" and context.iterations > 10:
return False
# Bloquear apenas para tarefas específicas
if "sensível" in context.task.description.lower() and context.iterations > 5:
return False
return None
```
### Modificações com Consciência de Contexto
```python
@before_llm_call
def adaptive_prompting(context: LLMCallHookContext) -> None:
"""Adiciona contexto diferente baseado na iteração."""
if context.iterations == 0:
context.messages.append({
"role": "system",
"content": "Comece com uma visão geral de alto nível."
})
elif context.iterations > 3:
context.messages.append({
"role": "system",
"content": "Foque em detalhes específicos e forneça exemplos."
})
return None
```
## Melhores Práticas
1. **Mantenha Hooks Focados**: Cada hook deve ter uma responsabilidade única
2. **Evite Computação Pesada**: Hooks executam em cada chamada LLM
3. **Trate Erros Graciosamente**: Use try-except para prevenir falhas de hooks
4. **Use Type Hints**: Aproveite `LLMCallHookContext` para melhor suporte IDE
5. **Documente Comportamento do Hook**: Especialmente para condições de bloqueio
6. **Teste Hooks Independentemente**: Teste unitário de hooks antes de usar em produção
7. **Limpe Hooks em Testes**: Use `clear_all_llm_call_hooks()` entre execuções de teste
8. **Modifique In-Place**: Sempre modifique `context.messages` in-place, nunca substitua
## Tratamento de Erros
```python
@before_llm_call
def safe_hook(context: LLMCallHookContext) -> bool | None:
try:
# Sua lógica de hook
if some_condition:
return False
except Exception as e:
print(f"⚠️ Erro no hook: {e}")
# Decida: permitir ou bloquear em erro
return None # Permitir execução apesar do erro
```
## Segurança de Tipos
```python
from crewai.hooks import LLMCallHookContext, BeforeLLMCallHookType, AfterLLMCallHookType
# Anotações de tipo explícitas
def my_before_hook(context: LLMCallHookContext) -> bool | None:
return None
def my_after_hook(context: LLMCallHookContext) -> str | None:
return None
# Registro type-safe
register_before_llm_call_hook(my_before_hook)
register_after_llm_call_hook(my_after_hook)
```
## Solução de Problemas
### Hook Não Está Executando
- Verifique se o hook está registrado antes da execução da crew
- Verifique se hook anterior retornou `False` (bloqueia hooks subsequentes)
- Garanta que assinatura do hook corresponda ao tipo esperado
### Modificações de Mensagem Não Persistem
- Use modificações in-place: `context.messages.append()`
- Não substitua a lista: `context.messages = []`
### Modificações de Resposta Não Funcionam
- Retorne a string modificada dos hooks posteriores
- Retornar `None` mantém a resposta original
## Conclusão
Os Hooks de Chamada LLM fornecem capacidades poderosas para controlar e monitorar interações com modelos de linguagem no CrewAI. Use-os para implementar guardrails de segurança, gates de aprovação, logging, rastreamento de custos e sanitização de respostas. Combinados com tratamento adequado de erros e segurança de tipos, os hooks permitem sistemas de agentes robustos e prontos para produção.

View File

@@ -0,0 +1,498 @@
---
title: Hooks de Chamada de Ferramenta
description: Aprenda a usar hooks de chamada de ferramenta para interceptar, modificar e controlar execução de ferramentas no CrewAI
mode: "wide"
---
Os Hooks de Chamada de Ferramenta fornecem controle fino sobre a execução de ferramentas durante operações do agente. Esses hooks permitem interceptar chamadas de ferramenta, modificar entradas, transformar saídas, implementar verificações de segurança e adicionar logging ou monitoramento abrangente.
## Visão Geral
Os hooks de ferramenta são executados em dois pontos críticos:
- **Antes da Chamada de Ferramenta**: Modificar entradas, validar parâmetros ou bloquear execução
- **Depois da Chamada de Ferramenta**: Transformar resultados, sanitizar saídas ou registrar detalhes de execução
## Tipos de Hook
### Hooks Antes da Chamada de Ferramenta
Executados antes de cada execução de ferramenta, esses hooks podem:
- Inspecionar e modificar entradas de ferramenta
- Bloquear execução de ferramenta com base em condições
- Implementar gates de aprovação para operações perigosas
- Validar parâmetros
- Registrar invocações de ferramenta
**Assinatura:**
```python
def before_hook(context: ToolCallHookContext) -> bool | None:
# Retorne False para bloquear execução
# Retorne True ou None para permitir execução
...
```
### Hooks Depois da Chamada de Ferramenta
Executados depois de cada execução de ferramenta, esses hooks podem:
- Modificar ou sanitizar resultados de ferramenta
- Adicionar metadados ou formatação
- Registrar resultados de execução
- Implementar validação de resultado
- Transformar formatos de saída
**Assinatura:**
```python
def after_hook(context: ToolCallHookContext) -> str | None:
# Retorne string de resultado modificado
# Retorne None para manter resultado original
...
```
## Contexto do Hook de Ferramenta
O objeto `ToolCallHookContext` fornece acesso abrangente ao estado de execução da ferramenta:
```python
class ToolCallHookContext:
tool_name: str # Nome da ferramenta sendo chamada
tool_input: dict[str, Any] # Parâmetros de entrada mutáveis da ferramenta
tool: CrewStructuredTool # Referência da instância da ferramenta
agent: Agent | BaseAgent | None # Agente executando a ferramenta
task: Task | None # Tarefa atual
crew: Crew | None # Instância da crew
tool_result: str | None # Resultado da ferramenta (apenas hooks posteriores)
```
### Modificando Entradas de Ferramenta
**Importante:** Sempre modifique entradas de ferramenta in-place:
```python
# ✅ Correto - modificar in-place
def sanitize_input(context: ToolCallHookContext) -> None:
context.tool_input['query'] = context.tool_input['query'].lower()
# ❌ Errado - substitui referência do dict
def wrong_approach(context: ToolCallHookContext) -> None:
context.tool_input = {'query': 'nova consulta'}
```
## Métodos de Registro
### 1. Registro Baseado em Decoradores (Recomendado)
Use decoradores para sintaxe mais limpa:
```python
from crewai.hooks import before_tool_call, after_tool_call
@before_tool_call
def block_dangerous_tools(context):
"""Bloqueia ferramentas perigosas."""
dangerous_tools = ['delete_database', 'drop_table', 'rm_rf']
if context.tool_name in dangerous_tools:
print(f"⛔ Ferramenta perigosa bloqueada: {context.tool_name}")
return False # Bloquear execução
return None
@after_tool_call
def sanitize_results(context):
"""Sanitiza resultados."""
if context.tool_result and "password" in context.tool_result.lower():
return context.tool_result.replace("password", "[CENSURADO]")
return None
```
### 2. Hooks com Escopo de Crew
Registre hooks para uma instância específica de crew:
```python
from crewai import CrewBase
from crewai.project import crew
from crewai.hooks import before_tool_call_crew, after_tool_call_crew
@CrewBase
class MyProjCrew:
@before_tool_call_crew
def validate_tool_inputs(self, context):
# Aplica-se apenas a esta crew
if context.tool_name == "web_search":
if not context.tool_input.get('query'):
print("❌ Consulta de busca inválida")
return False
return None
@after_tool_call_crew
def log_tool_results(self, context):
# Logging de ferramenta específico da crew
print(f"✅ {context.tool_name} concluída")
return None
@crew
def crew(self) -> Crew:
return Crew(
agents=self.agents,
tasks=self.tasks,
process=Process.sequential,
verbose=True
)
```
## Casos de Uso Comuns
### 1. Guardrails de Segurança
```python
@before_tool_call
def safety_check(context: ToolCallHookContext) -> bool | None:
"""Bloqueia ferramentas que podem causar danos."""
destructive_tools = [
'delete_file',
'drop_table',
'remove_user',
'system_shutdown'
]
if context.tool_name in destructive_tools:
print(f"🛑 Ferramenta destrutiva bloqueada: {context.tool_name}")
return False
# Avisar em operações sensíveis
sensitive_tools = ['send_email', 'post_to_social_media', 'charge_payment']
if context.tool_name in sensitive_tools:
print(f"⚠️ Executando ferramenta sensível: {context.tool_name}")
return None
```
### 2. Gate de Aprovação Humana
```python
@before_tool_call
def require_approval_for_actions(context: ToolCallHookContext) -> bool | None:
"""Requer aprovação para ações específicas."""
approval_required = [
'send_email',
'make_purchase',
'delete_file',
'post_message'
]
if context.tool_name in approval_required:
response = context.request_human_input(
prompt=f"Aprovar {context.tool_name}?",
default_message=f"Entrada: {context.tool_input}\nDigite 'sim' para aprovar:"
)
if response.lower() != 'sim':
print(f"❌ Execução de ferramenta negada: {context.tool_name}")
return False
return None
```
### 3. Validação e Sanitização de Entrada
```python
@before_tool_call
def validate_and_sanitize_inputs(context: ToolCallHookContext) -> bool | None:
"""Valida e sanitiza entradas."""
# Validar consultas de busca
if context.tool_name == 'web_search':
query = context.tool_input.get('query', '')
if len(query) < 3:
print("❌ Consulta de busca muito curta")
return False
# Sanitizar consulta
context.tool_input['query'] = query.strip().lower()
# Validar caminhos de arquivo
if context.tool_name == 'read_file':
path = context.tool_input.get('path', '')
if '..' in path or path.startswith('/'):
print("❌ Caminho de arquivo inválido")
return False
return None
```
### 4. Sanitização de Resultado
```python
@after_tool_call
def sanitize_sensitive_data(context: ToolCallHookContext) -> str | None:
"""Sanitiza dados sensíveis."""
if not context.tool_result:
return None
import re
result = context.tool_result
# Remover chaves de API
result = re.sub(
r'(api[_-]?key|token)["\']?\s*[:=]\s*["\']?[\w-]+',
r'\1: [CENSURADO]',
result,
flags=re.IGNORECASE
)
# Remover endereços de email
result = re.sub(
r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b',
'[EMAIL-CENSURADO]',
result
)
# Remover números de cartão de crédito
result = re.sub(
r'\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b',
'[CARTÃO-CENSURADO]',
result
)
return result
```
### 5. Análise de Uso de Ferramenta
```python
import time
from collections import defaultdict
tool_stats = defaultdict(lambda: {'count': 0, 'total_time': 0, 'failures': 0})
@before_tool_call
def start_timer(context: ToolCallHookContext) -> None:
context.tool_input['_start_time'] = time.time()
return None
@after_tool_call
def track_tool_usage(context: ToolCallHookContext) -> None:
start_time = context.tool_input.get('_start_time', time.time())
duration = time.time() - start_time
tool_stats[context.tool_name]['count'] += 1
tool_stats[context.tool_name]['total_time'] += duration
if not context.tool_result or 'error' in context.tool_result.lower():
tool_stats[context.tool_name]['failures'] += 1
print(f"""
📊 Estatísticas da Ferramenta {context.tool_name}:
- Execuções: {tool_stats[context.tool_name]['count']}
- Tempo Médio: {tool_stats[context.tool_name]['total_time'] / tool_stats[context.tool_name]['count']:.2f}s
- Falhas: {tool_stats[context.tool_name]['failures']}
""")
return None
```
### 6. Limitação de Taxa
```python
from collections import defaultdict
from datetime import datetime, timedelta
tool_call_history = defaultdict(list)
@before_tool_call
def rate_limit_tools(context: ToolCallHookContext) -> bool | None:
"""Limita taxa de chamadas de ferramenta."""
tool_name = context.tool_name
now = datetime.now()
# Limpar entradas antigas (mais antigas que 1 minuto)
tool_call_history[tool_name] = [
call_time for call_time in tool_call_history[tool_name]
if now - call_time < timedelta(minutes=1)
]
# Verificar limite de taxa (máximo 10 chamadas por minuto)
if len(tool_call_history[tool_name]) >= 10:
print(f"🚫 Limite de taxa excedido para {tool_name}")
return False
# Registrar esta chamada
tool_call_history[tool_name].append(now)
return None
```
### 7. Logging de Debug
```python
@before_tool_call
def debug_tool_call(context: ToolCallHookContext) -> None:
"""Debug de chamada de ferramenta."""
print(f"""
🔍 Debug de Chamada de Ferramenta:
- Ferramenta: {context.tool_name}
- Agente: {context.agent.role if context.agent else 'Desconhecido'}
- Tarefa: {context.task.description[:50] if context.task else 'Desconhecida'}...
- Entrada: {context.tool_input}
""")
return None
@after_tool_call
def debug_tool_result(context: ToolCallHookContext) -> None:
"""Debug de resultado de ferramenta."""
if context.tool_result:
result_preview = context.tool_result[:200]
print(f"✅ Preview do Resultado: {result_preview}...")
else:
print("⚠️ Nenhum resultado retornado")
return None
```
## Gerenciamento de Hooks
### Desregistrando Hooks
```python
from crewai.hooks import (
unregister_before_tool_call_hook,
unregister_after_tool_call_hook
)
# Desregistrar hook específico
def my_hook(context):
...
register_before_tool_call_hook(my_hook)
# Mais tarde...
success = unregister_before_tool_call_hook(my_hook)
print(f"Desregistrado: {success}")
```
### Limpando Hooks
```python
from crewai.hooks import (
clear_before_tool_call_hooks,
clear_after_tool_call_hooks,
clear_all_tool_call_hooks
)
# Limpar tipo específico de hook
count = clear_before_tool_call_hooks()
print(f"Limpou {count} hooks antes")
# Limpar todos os hooks de ferramenta
before_count, after_count = clear_all_tool_call_hooks()
print(f"Limpou {before_count} hooks antes e {after_count} hooks depois")
```
## Padrões Avançados
### Execução Condicional de Hook
```python
@before_tool_call
def conditional_blocking(context: ToolCallHookContext) -> bool | None:
"""Bloqueia apenas em condições específicas."""
# Bloquear apenas para agentes específicos
if context.agent and context.agent.role == "junior_agent":
if context.tool_name in ['delete_file', 'send_email']:
print(f"❌ Agentes júnior não podem usar {context.tool_name}")
return False
# Bloquear apenas durante tarefas específicas
if context.task and "sensível" in context.task.description.lower():
if context.tool_name == 'web_search':
print("❌ Busca na web bloqueada para tarefas sensíveis")
return False
return None
```
### Modificação de Entrada com Consciência de Contexto
```python
@before_tool_call
def enhance_tool_inputs(context: ToolCallHookContext) -> None:
"""Adiciona contexto baseado no papel do agente."""
# Adicionar contexto baseado no papel do agente
if context.agent and context.agent.role == "researcher":
if context.tool_name == 'web_search':
# Adicionar restrições de domínio para pesquisadores
context.tool_input['domains'] = ['edu', 'gov', 'org']
# Adicionar contexto baseado na tarefa
if context.task and "urgente" in context.task.description.lower():
if context.tool_name == 'send_email':
context.tool_input['priority'] = 'high'
return None
```
## Melhores Práticas
1. **Mantenha Hooks Focados**: Cada hook deve ter uma responsabilidade única
2. **Evite Computação Pesada**: Hooks executam em cada chamada de ferramenta
3. **Trate Erros Graciosamente**: Use try-except para prevenir falhas de hooks
4. **Use Type Hints**: Aproveite `ToolCallHookContext` para melhor suporte IDE
5. **Documente Condições de Bloqueio**: Deixe claro quando/por que ferramentas são bloqueadas
6. **Teste Hooks Independentemente**: Teste unitário de hooks antes de usar em produção
7. **Limpe Hooks em Testes**: Use `clear_all_tool_call_hooks()` entre execuções de teste
8. **Modifique In-Place**: Sempre modifique `context.tool_input` in-place, nunca substitua
9. **Registre Decisões Importantes**: Especialmente ao bloquear execução de ferramenta
10. **Considere Performance**: Cache validações caras quando possível
## Tratamento de Erros
```python
@before_tool_call
def safe_validation(context: ToolCallHookContext) -> bool | None:
try:
# Sua lógica de validação
if not validate_input(context.tool_input):
return False
except Exception as e:
print(f"⚠️ Erro no hook: {e}")
# Decida: permitir ou bloquear em erro
return None # Permitir execução apesar do erro
```
## Segurança de Tipos
```python
from crewai.hooks import ToolCallHookContext, BeforeToolCallHookType, AfterToolCallHookType
# Anotações de tipo explícitas
def my_before_hook(context: ToolCallHookContext) -> bool | None:
return None
def my_after_hook(context: ToolCallHookContext) -> str | None:
return None
# Registro type-safe
register_before_tool_call_hook(my_before_hook)
register_after_tool_call_hook(my_after_hook)
```
## Solução de Problemas
### Hook Não Está Executando
- Verifique se hook está registrado antes da execução da crew
- Verifique se hook anterior retornou `False` (bloqueia execução e hooks subsequentes)
- Garanta que assinatura do hook corresponda ao tipo esperado
### Modificações de Entrada Não Funcionam
- Use modificações in-place: `context.tool_input['key'] = value`
- Não substitua o dict: `context.tool_input = {}`
### Modificações de Resultado Não Funcionam
- Retorne a string modificada dos hooks posteriores
- Retornar `None` mantém o resultado original
- Garanta que a ferramenta realmente retornou um resultado
### Ferramenta Bloqueada Inesperadamente
- Verifique todos os hooks antes por condições de bloqueio
- Verifique ordem de execução do hook
- Adicione logging de debug para identificar qual hook está bloqueando
## Conclusão
Os Hooks de Chamada de Ferramenta fornecem capacidades poderosas para controlar e monitorar execução de ferramentas no CrewAI. Use-os para implementar guardrails de segurança, gates de aprovação, validação de entrada, sanitização de resultado, logging e análise. Combinados com tratamento adequado de erros e segurança de tipos, os hooks permitem sistemas de agentes seguros e prontos para produção com observabilidade abrangente.

View File

@@ -733,9 +733,7 @@ Aqui está um exemplo básico para rotear requisições ao OpenAI, usando especi
- Coletam metadados relevantes para filtragem de logs
- Impõem permissões de acesso
Crie chaves de API através de:
- [Portkey App](https://app.portkey.ai/)
- [API Key Management API](/pt-BR/api-reference/admin-api/control-plane/api-keys/create-api-key)
Crie chaves de API através do [Portkey App](https://app.portkey.ai/)
Exemplo usando Python SDK:
```python
@@ -758,7 +756,7 @@ Aqui está um exemplo básico para rotear requisições ao OpenAI, usando especi
)
```
Para instruções detalhadas de gerenciamento de chaves, veja nossa [documentação de API Keys](/pt-BR/api-reference/admin-api/control-plane/api-keys/create-api-key).
Para instruções detalhadas de gerenciamento de chaves, veja a [documentação Portkey](https://portkey.ai/docs).
</Accordion>
<Accordion title="Etapa 4: Implante & Monitore">

View File

@@ -18,7 +18,7 @@ Essas ferramentas permitem que seus agentes interajam com serviços em nuvem, ac
Escreva e faça upload de arquivos para o armazenamento Amazon S3.
</Card>
<Card title="Bedrock Invoke Agent" icon="aws" href="/pt-BR/tools/cloud-storage/bedrockinvokeagenttool">
<Card title="Bedrock Invoke Agent" icon="aws" href="/pt-BR/tools/integration/bedrockinvokeagenttool">
Acione agentes Amazon Bedrock para tarefas orientadas por IA.
</Card>

View File

@@ -11,7 +11,7 @@ mode: "wide"
<Card
title="Bedrock Invoke Agent Tool"
icon="cloud"
href="/en/tools/tool-integrations/bedrockinvokeagenttool"
href="/pt-BR/tools/integration/bedrockinvokeagenttool"
color="#0891B2"
>
Invoke Amazon Bedrock Agents from CrewAI to orchestrate actions across AWS services.
@@ -20,7 +20,7 @@ mode: "wide"
<Card
title="CrewAI Automation Tool"
icon="bolt"
href="/en/tools/tool-integrations/crewaiautomationtool"
href="/pt-BR/tools/integration/crewaiautomationtool"
color="#7C3AED"
>
Automate deployment and operations by integrating CrewAI with external platforms and workflows.

View File

@@ -12,12 +12,16 @@ from pydantic.types import ImportString
class QdrantToolSchema(BaseModel):
query: str = Field(..., description="Query to search in Qdrant DB")
query: str = Field(
..., description="Query to search in Qdrant DB - always required."
)
filter_by: str | None = Field(
default=None, description="Parameter to filter the search by."
default=None,
description="Parameter to filter the search by. When filtering, needs to be used in conjunction with filter_value.",
)
filter_value: Any | None = Field(
default=None, description="Value to filter the search by."
default=None,
description="Value to filter the search by. When filtering, needs to be used in conjunction with filter_by.",
)

View File

@@ -8,8 +8,8 @@ from crewai.crew import Crew
from crewai.crews.crew_output import CrewOutput
from crewai.flow.flow import Flow
from crewai.knowledge.knowledge import Knowledge
from crewai.llm import LLM
from crewai.llms.base_llm import BaseLLM
from crewai.llm.base_llm import BaseLLM
from crewai.llm.core import LLM
from crewai.process import Process
from crewai.task import Task
from crewai.tasks.llm_guardrail import LLMGuardrail

View File

@@ -38,6 +38,7 @@ class A2AConfig(BaseModel):
max_turns: Maximum conversation turns with A2A agent (default: 10).
response_model: Optional Pydantic model for structured A2A agent responses.
fail_fast: If True, raise error when agent unreachable; if False, skip and continue (default: True).
trust_remote_completion_status: If True, return A2A agent's result directly when status is "completed"; if False, always ask server agent to respond (default: False).
"""
endpoint: Url = Field(description="A2A agent endpoint URL")
@@ -57,3 +58,7 @@ class A2AConfig(BaseModel):
default=True,
description="If True, raise an error immediately when the A2A agent is unreachable. If False, skip the A2A agent and continue execution.",
)
trust_remote_completion_status: bool = Field(
default=False,
description='If True, return the A2A agent\'s result directly when status is "completed" without asking the server agent to respond. If False, always ask the server agent to respond, allowing it to potentially delegate again.',
)

View File

@@ -52,7 +52,7 @@ def wrap_agent_with_a2a_instance(agent: Agent) -> None:
Args:
agent: The agent instance to wrap
"""
original_execute_task = agent.execute_task.__func__
original_execute_task = agent.execute_task.__func__ # type: ignore[attr-defined]
@wraps(original_execute_task)
def execute_task_with_a2a(
@@ -73,7 +73,7 @@ def wrap_agent_with_a2a_instance(agent: Agent) -> None:
Task execution result
"""
if not self.a2a:
return original_execute_task(self, task, context, tools)
return original_execute_task(self, task, context, tools) # type: ignore[no-any-return]
a2a_agents, agent_response_model = get_a2a_agents_and_response_model(self.a2a)
@@ -498,6 +498,23 @@ def _delegate_to_a2a(
conversation_history = a2a_result.get("history", [])
if a2a_result["status"] in ["completed", "input_required"]:
if (
a2a_result["status"] == "completed"
and agent_config.trust_remote_completion_status
):
result_text = a2a_result.get("result", "")
final_turn_number = turn_num + 1
crewai_event_bus.emit(
None,
A2AConversationCompletedEvent(
status="completed",
final_result=result_text,
error=None,
total_turns=final_turn_number,
),
)
return result_text # type: ignore[no-any-return]
final_result, next_request = _handle_agent_response_and_continue(
self=self,
a2a_result=a2a_result,

View File

@@ -39,7 +39,7 @@ from crewai.knowledge.knowledge import Knowledge
from crewai.knowledge.source.base_knowledge_source import BaseKnowledgeSource
from crewai.knowledge.utils.knowledge_utils import extract_knowledge_context
from crewai.lite_agent import LiteAgent
from crewai.llms.base_llm import BaseLLM
from crewai.llm.base_llm import BaseLLM
from crewai.mcp import (
MCPClient,
MCPServerConfig,
@@ -119,6 +119,7 @@ class Agent(BaseAgent):
_times_executed: int = PrivateAttr(default=0)
_mcp_clients: list[Any] = PrivateAttr(default_factory=list)
_last_messages: list[LLMMessage] = PrivateAttr(default_factory=list)
max_execution_time: int | None = Field(
default=None,
description="Maximum execution time for an agent to execute a task",
@@ -538,6 +539,12 @@ class Agent(BaseAgent):
event=AgentExecutionCompletedEvent(agent=self, task=task, output=result),
)
self._last_messages = (
self.agent_executor.messages.copy()
if self.agent_executor and hasattr(self.agent_executor, "messages")
else []
)
self._cleanup_mcp_clients()
return result
@@ -626,7 +633,7 @@ class Agent(BaseAgent):
)
self.agent_executor = CrewAgentExecutor(
llm=self.llm,
llm=self.llm, # type: ignore[arg-type]
task=task, # type: ignore[arg-type]
agent=self,
crew=self.crew,
@@ -803,6 +810,7 @@ class Agent(BaseAgent):
from crewai.tools.base_tool import BaseTool
from crewai.tools.mcp_native_tool import MCPNativeTool
transport: StdioTransport | HTTPTransport | SSETransport
if isinstance(mcp_config, MCPServerStdio):
transport = StdioTransport(
command=mcp_config.command,
@@ -896,10 +904,12 @@ class Agent(BaseAgent):
server_name=server_name,
run_context=None,
)
if mcp_config.tool_filter(context, tool):
# Try new signature first
if mcp_config.tool_filter(context, tool): # type: ignore[arg-type,call-arg]
filtered_tools.append(tool)
except (TypeError, AttributeError):
if mcp_config.tool_filter(tool):
# Fallback to old signature
if mcp_config.tool_filter(tool): # type: ignore[arg-type,call-arg]
filtered_tools.append(tool)
else:
# Not callable - include tool
@@ -974,7 +984,9 @@ class Agent(BaseAgent):
path = parsed.path.replace("/", "_").strip("_")
return f"{domain}_{path}" if path else domain
def _get_mcp_tool_schemas(self, server_params: dict) -> dict[str, dict]:
def _get_mcp_tool_schemas(
self, server_params: dict[str, Any]
) -> dict[str, dict[str, Any]]:
"""Get tool schemas from MCP server for wrapper creation with caching."""
server_url = server_params["url"]
@@ -988,7 +1000,7 @@ class Agent(BaseAgent):
self._logger.log(
"debug", f"Using cached MCP tool schemas for {server_url}"
)
return cached_data
return cast(dict[str, dict[str, Any]], cached_data)
try:
schemas = asyncio.run(self._get_mcp_tool_schemas_async(server_params))
@@ -1006,7 +1018,7 @@ class Agent(BaseAgent):
async def _get_mcp_tool_schemas_async(
self, server_params: dict[str, Any]
) -> dict[str, dict]:
) -> dict[str, dict[str, Any]]:
"""Async implementation of MCP tool schema retrieval with timeouts and retries."""
server_url = server_params["url"]
return await self._retry_mcp_discovery(
@@ -1014,7 +1026,7 @@ class Agent(BaseAgent):
)
async def _retry_mcp_discovery(
self, operation_func, server_url: str
self, operation_func: Any, server_url: str
) -> dict[str, dict[str, Any]]:
"""Retry MCP discovery operation with exponential backoff, avoiding try-except in loop."""
last_error = None
@@ -1045,7 +1057,7 @@ class Agent(BaseAgent):
@staticmethod
async def _attempt_mcp_discovery(
operation_func, server_url: str
operation_func: Any, server_url: str
) -> tuple[dict[str, dict[str, Any]] | None, str, bool]:
"""Attempt single MCP discovery operation and return (result, error_message, should_retry)."""
try:
@@ -1149,13 +1161,13 @@ class Agent(BaseAgent):
Field(..., description=field_description),
)
else:
field_definitions[field_name] = (
field_definitions[field_name] = ( # type: ignore[assignment]
field_type | None,
Field(default=None, description=field_description),
)
model_name = f"{tool_name.replace('-', '_').replace(' ', '_')}Schema"
return create_model(model_name, **field_definitions)
return create_model(model_name, **field_definitions) # type: ignore[no-any-return,call-overload]
def _json_type_to_python(self, field_schema: dict[str, Any]) -> type:
"""Convert JSON Schema type to Python type.
@@ -1175,16 +1187,16 @@ class Agent(BaseAgent):
if "const" in option:
types.append(str)
else:
types.append(self._json_type_to_python(option))
types.append(self._json_type_to_python(option)) # type: ignore[arg-type]
unique_types = list(set(types))
if len(unique_types) > 1:
result = unique_types[0]
for t in unique_types[1:]:
result = result | t
result = result | t # type: ignore[assignment]
return result
return unique_types[0]
type_mapping = {
type_mapping: dict[str, type] = {
"string": str,
"number": float,
"integer": int,
@@ -1193,10 +1205,10 @@ class Agent(BaseAgent):
"object": dict,
}
return type_mapping.get(json_type, Any)
return type_mapping.get(json_type or "", Any)
@staticmethod
def _fetch_amp_mcp_servers(mcp_name: str) -> list[dict]:
def _fetch_amp_mcp_servers(mcp_name: str) -> list[dict[str, Any]]:
"""Fetch MCP server configurations from CrewAI AMP API."""
# TODO: Implement AMP API call to "integrations/mcps" endpoint
# Should return list of server configs with URLs
@@ -1341,6 +1353,15 @@ class Agent(BaseAgent):
def set_fingerprint(self, fingerprint: Fingerprint) -> None:
self.security_config.fingerprint = fingerprint
@property
def last_messages(self) -> list[LLMMessage]:
"""Get messages from the last task execution.
Returns:
List of LLM messages from the most recent task execution.
"""
return self._last_messages
def _get_knowledge_search_query(self, task_prompt: str, task: Task) -> str | None:
"""Generate a search query for the knowledge base based on the task description."""
crewai_event_bus.emit(

View File

@@ -137,7 +137,7 @@ class BaseAgent(BaseModel, ABC, metaclass=AgentMeta):
default=False,
description="Enable agent to delegate and ask questions among each other.",
)
tools: list[BaseTool] | None = Field(
tools: list[BaseTool] = Field(
default_factory=list, description="Tools at agents' disposal"
)
max_iter: int = Field(
@@ -161,7 +161,7 @@ class BaseAgent(BaseModel, ABC, metaclass=AgentMeta):
description="An instance of the ToolsHandler class.",
)
tools_results: list[dict[str, Any]] = Field(
default=[], description="Results of the tools used by the agent."
default_factory=list, description="Results of the tools used by the agent."
)
max_tokens: int | None = Field(
default=None, description="Maximum number of tokens for the agent's execution."
@@ -265,7 +265,7 @@ class BaseAgent(BaseModel, ABC, metaclass=AgentMeta):
if not mcps:
return mcps
validated_mcps = []
validated_mcps: list[str | MCPServerConfig] = []
for mcp in mcps:
if isinstance(mcp, str):
if mcp.startswith(("https://", "crewai-amp:")):

View File

@@ -23,6 +23,10 @@ from crewai.events.types.logging_events import (
AgentLogsExecutionEvent,
AgentLogsStartedEvent,
)
from crewai.hooks.llm_hooks import (
get_after_llm_call_hooks,
get_before_llm_call_hooks,
)
from crewai.utilities.agent_utils import (
enforce_rpm_limit,
format_message_for_llm,
@@ -47,7 +51,7 @@ if TYPE_CHECKING:
from crewai.agent import Agent
from crewai.agents.tools_handler import ToolsHandler
from crewai.crew import Crew
from crewai.llms.base_llm import BaseLLM
from crewai.llm.base_llm import BaseLLM
from crewai.task import Task
from crewai.tools.base_tool import BaseTool
from crewai.tools.structured_tool import CrewStructuredTool
@@ -130,6 +134,10 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
self.messages: list[LLMMessage] = []
self.iterations = 0
self.log_error_after = 3
self.before_llm_call_hooks: list[Callable] = []
self.after_llm_call_hooks: list[Callable] = []
self.before_llm_call_hooks.extend(get_before_llm_call_hooks())
self.after_llm_call_hooks.extend(get_after_llm_call_hooks())
if self.llm:
# This may be mutating the shared llm object and needs further evaluation
existing_stop = getattr(self.llm, "stop", [])
@@ -226,6 +234,7 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
from_task=self.task,
from_agent=self.agent,
response_model=self.response_model,
executor_context=self,
)
formatted_answer = process_llm_response(answer, self.use_stop_words) # type: ignore[assignment]
@@ -254,6 +263,7 @@ class CrewAgentExecutor(CrewAgentExecutorMixin):
task=self.task,
agent=self.agent,
function_calling_llm=self.function_calling_llm,
crew=self.crew,
)
formatted_answer = self._handle_agent_action(
formatted_answer, tool_result

View File

@@ -1,5 +1,5 @@
import time
from typing import Any
from typing import TYPE_CHECKING, Any, TypeVar, cast
import webbrowser
from pydantic import BaseModel, Field
@@ -13,6 +13,8 @@ from crewai.cli.shared.token_manager import TokenManager
console = Console()
TOauth2Settings = TypeVar("TOauth2Settings", bound="Oauth2Settings")
class Oauth2Settings(BaseModel):
provider: str = Field(
@@ -28,9 +30,15 @@ class Oauth2Settings(BaseModel):
description="OAuth2 audience value, typically used to identify the target API or resource.",
default=None,
)
extra: dict[str, Any] = Field(
description="Extra configuration for the OAuth2 provider.",
default={},
)
@classmethod
def from_settings(cls):
def from_settings(cls: type[TOauth2Settings]) -> TOauth2Settings:
"""Create an Oauth2Settings instance from the CLI settings."""
settings = Settings()
return cls(
@@ -38,12 +46,20 @@ class Oauth2Settings(BaseModel):
domain=settings.oauth2_domain,
client_id=settings.oauth2_client_id,
audience=settings.oauth2_audience,
extra=settings.oauth2_extra,
)
if TYPE_CHECKING:
from crewai.cli.authentication.providers.base_provider import BaseProvider
class ProviderFactory:
@classmethod
def from_settings(cls, settings: Oauth2Settings | None = None):
def from_settings(
cls: type["ProviderFactory"], # noqa: UP037
settings: Oauth2Settings | None = None,
) -> "BaseProvider": # noqa: UP037
settings = settings or Oauth2Settings.from_settings()
import importlib
@@ -53,11 +69,11 @@ class ProviderFactory:
)
provider = getattr(module, f"{settings.provider.capitalize()}Provider")
return provider(settings)
return cast("BaseProvider", provider(settings))
class AuthenticationCommand:
def __init__(self):
def __init__(self) -> None:
self.token_manager = TokenManager()
self.oauth2_provider = ProviderFactory.from_settings()
@@ -84,7 +100,7 @@ class AuthenticationCommand:
timeout=20,
)
response.raise_for_status()
return response.json()
return cast(dict[str, Any], response.json())
def _display_auth_instructions(self, device_code_data: dict[str, str]) -> None:
"""Display the authentication instructions to the user."""

View File

@@ -24,3 +24,7 @@ class BaseProvider(ABC):
@abstractmethod
def get_client_id(self) -> str: ...
def get_required_fields(self) -> list[str]:
"""Returns which provider-specific fields inside the "extra" dict will be required"""
return []

View File

@@ -3,16 +3,16 @@ from crewai.cli.authentication.providers.base_provider import BaseProvider
class OktaProvider(BaseProvider):
def get_authorize_url(self) -> str:
return f"https://{self.settings.domain}/oauth2/default/v1/device/authorize"
return f"{self._oauth2_base_url()}/v1/device/authorize"
def get_token_url(self) -> str:
return f"https://{self.settings.domain}/oauth2/default/v1/token"
return f"{self._oauth2_base_url()}/v1/token"
def get_jwks_url(self) -> str:
return f"https://{self.settings.domain}/oauth2/default/v1/keys"
return f"{self._oauth2_base_url()}/v1/keys"
def get_issuer(self) -> str:
return f"https://{self.settings.domain}/oauth2/default"
return self._oauth2_base_url().removesuffix("/oauth2")
def get_audience(self) -> str:
if self.settings.audience is None:
@@ -27,3 +27,16 @@ class OktaProvider(BaseProvider):
"Client ID is required. Please set it in the configuration."
)
return self.settings.client_id
def get_required_fields(self) -> list[str]:
return ["authorization_server_name", "using_org_auth_server"]
def _oauth2_base_url(self) -> str:
using_org_auth_server = self.settings.extra.get("using_org_auth_server", False)
if using_org_auth_server:
base_url = f"https://{self.settings.domain}/oauth2"
else:
base_url = f"https://{self.settings.domain}/oauth2/{self.settings.extra.get('authorization_server_name', 'default')}"
return f"{base_url}"

View File

@@ -11,18 +11,18 @@ console = Console()
class BaseCommand:
def __init__(self):
def __init__(self) -> None:
self._telemetry = Telemetry()
self._telemetry.set_tracer()
class PlusAPIMixin:
def __init__(self, telemetry):
def __init__(self, telemetry: Telemetry) -> None:
try:
telemetry.set_tracer()
self.plus_api_client = PlusAPI(api_key=get_auth_token())
except Exception:
self._deploy_signup_error_span = telemetry.deploy_signup_error_span()
telemetry.deploy_signup_error_span()
console.print(
"Please sign up/login to CrewAI+ before using the CLI.",
style="bold red",

View File

@@ -2,6 +2,7 @@ import json
from logging import getLogger
from pathlib import Path
import tempfile
from typing import Any
from pydantic import BaseModel, Field
@@ -136,7 +137,12 @@ class Settings(BaseModel):
default=DEFAULT_CLI_SETTINGS["oauth2_domain"],
)
def __init__(self, config_path: Path | None = None, **data):
oauth2_extra: dict[str, Any] = Field(
description="Extra configuration for the OAuth2 provider.",
default={},
)
def __init__(self, config_path: Path | None = None, **data: dict[str, Any]) -> None:
"""Load Settings from config path with fallback support"""
if config_path is None:
config_path = get_writable_config_path()

View File

@@ -14,7 +14,8 @@ import tomli
from crewai.cli.utils import read_toml
from crewai.cli.version import get_crewai_version
from crewai.crew import Crew
from crewai.llm import LLM, BaseLLM
from crewai.llm import LLM
from crewai.llm.base_llm import BaseLLM
from crewai.types.crew_chat import ChatInputField, ChatInputs
from crewai.utilities.llm_utils import create_llm
from crewai.utilities.printer import Printer

View File

@@ -1,9 +1,10 @@
from typing import Any
from typing import Any, cast
import requests
from requests.exceptions import JSONDecodeError, RequestException
from rich.console import Console
from crewai.cli.authentication.main import Oauth2Settings, ProviderFactory
from crewai.cli.command import BaseCommand
from crewai.cli.settings.main import SettingsCommand
from crewai.cli.version import get_crewai_version
@@ -13,7 +14,7 @@ console = Console()
class EnterpriseConfigureCommand(BaseCommand):
def __init__(self):
def __init__(self) -> None:
super().__init__()
self.settings_command = SettingsCommand()
@@ -54,25 +55,12 @@ class EnterpriseConfigureCommand(BaseCommand):
except JSONDecodeError as e:
raise ValueError(f"Invalid JSON response from {oauth_endpoint}") from e
required_fields = [
"audience",
"domain",
"device_authorization_client_id",
"provider",
]
missing_fields = [
field for field in required_fields if field not in oauth_config
]
if missing_fields:
raise ValueError(
f"Missing required fields in OAuth2 configuration: {', '.join(missing_fields)}"
)
self._validate_oauth_config(oauth_config)
console.print(
"✅ Successfully retrieved OAuth2 configuration", style="green"
)
return oauth_config
return cast(dict[str, Any], oauth_config)
except RequestException as e:
raise ValueError(f"Failed to connect to enterprise URL: {e!s}") from e
@@ -89,6 +77,7 @@ class EnterpriseConfigureCommand(BaseCommand):
"oauth2_audience": oauth_config["audience"],
"oauth2_client_id": oauth_config["device_authorization_client_id"],
"oauth2_domain": oauth_config["domain"],
"oauth2_extra": oauth_config["extra"],
}
console.print("🔄 Updating local OAuth2 configuration...")
@@ -99,3 +88,38 @@ class EnterpriseConfigureCommand(BaseCommand):
except Exception as e:
raise ValueError(f"Failed to update OAuth2 settings: {e!s}") from e
def _validate_oauth_config(self, oauth_config: dict[str, Any]) -> None:
required_fields = [
"audience",
"domain",
"device_authorization_client_id",
"provider",
"extra",
]
missing_basic_fields = [
field for field in required_fields if field not in oauth_config
]
missing_provider_specific_fields = [
field
for field in self._get_provider_specific_fields(oauth_config["provider"])
if field not in oauth_config.get("extra", {})
]
if missing_basic_fields:
raise ValueError(
f"Missing required fields in OAuth2 configuration: [{', '.join(missing_basic_fields)}]"
)
if missing_provider_specific_fields:
raise ValueError(
f"Missing authentication provider required fields in OAuth2 configuration: [{', '.join(missing_provider_specific_fields)}] (Configured provider: '{oauth_config['provider']}')"
)
def _get_provider_specific_fields(self, provider_name: str) -> list[str]:
provider = ProviderFactory.from_settings(
Oauth2Settings(provider=provider_name, client_id="dummy", domain="dummy")
)
return provider.get_required_fields()

View File

@@ -3,7 +3,7 @@ import subprocess
class Repository:
def __init__(self, path="."):
def __init__(self, path: str = ".") -> None:
self.path = path
if not self.is_git_installed():

View File

@@ -1,3 +1,4 @@
from typing import Any
from urllib.parse import urljoin
import requests
@@ -36,19 +37,21 @@ class PlusAPI:
str(settings.enterprise_base_url) or DEFAULT_CREWAI_ENTERPRISE_URL
)
def _make_request(self, method: str, endpoint: str, **kwargs) -> requests.Response:
def _make_request(
self, method: str, endpoint: str, **kwargs: Any
) -> requests.Response:
url = urljoin(self.base_url, endpoint)
session = requests.Session()
session.trust_env = False
return session.request(method, url, headers=self.headers, **kwargs)
def login_to_tool_repository(self):
def login_to_tool_repository(self) -> requests.Response:
return self._make_request("POST", f"{self.TOOLS_RESOURCE}/login")
def get_tool(self, handle: str):
def get_tool(self, handle: str) -> requests.Response:
return self._make_request("GET", f"{self.TOOLS_RESOURCE}/{handle}")
def get_agent(self, handle: str):
def get_agent(self, handle: str) -> requests.Response:
return self._make_request("GET", f"{self.AGENTS_RESOURCE}/{handle}")
def publish_tool(
@@ -58,8 +61,8 @@ class PlusAPI:
version: str,
description: str | None,
encoded_file: str,
available_exports: list[str] | None = None,
):
available_exports: list[dict[str, Any]] | None = None,
) -> requests.Response:
params = {
"handle": handle,
"public": is_public,
@@ -111,13 +114,13 @@ class PlusAPI:
def list_crews(self) -> requests.Response:
return self._make_request("GET", self.CREWS_RESOURCE)
def create_crew(self, payload) -> requests.Response:
def create_crew(self, payload: dict[str, Any]) -> requests.Response:
return self._make_request("POST", self.CREWS_RESOURCE, json=payload)
def get_organizations(self) -> requests.Response:
return self._make_request("GET", self.ORGANIZATIONS_RESOURCE)
def initialize_trace_batch(self, payload) -> requests.Response:
def initialize_trace_batch(self, payload: dict[str, Any]) -> requests.Response:
return self._make_request(
"POST",
f"{self.TRACING_RESOURCE}/batches",
@@ -125,14 +128,18 @@ class PlusAPI:
timeout=30,
)
def initialize_ephemeral_trace_batch(self, payload) -> requests.Response:
def initialize_ephemeral_trace_batch(
self, payload: dict[str, Any]
) -> requests.Response:
return self._make_request(
"POST",
f"{self.EPHEMERAL_TRACING_RESOURCE}/batches",
json=payload,
)
def send_trace_events(self, trace_batch_id: str, payload) -> requests.Response:
def send_trace_events(
self, trace_batch_id: str, payload: dict[str, Any]
) -> requests.Response:
return self._make_request(
"POST",
f"{self.TRACING_RESOURCE}/batches/{trace_batch_id}/events",
@@ -141,7 +148,7 @@ class PlusAPI:
)
def send_ephemeral_trace_events(
self, trace_batch_id: str, payload
self, trace_batch_id: str, payload: dict[str, Any]
) -> requests.Response:
return self._make_request(
"POST",
@@ -150,7 +157,9 @@ class PlusAPI:
timeout=30,
)
def finalize_trace_batch(self, trace_batch_id: str, payload) -> requests.Response:
def finalize_trace_batch(
self, trace_batch_id: str, payload: dict[str, Any]
) -> requests.Response:
return self._make_request(
"PATCH",
f"{self.TRACING_RESOURCE}/batches/{trace_batch_id}/finalize",
@@ -159,7 +168,7 @@ class PlusAPI:
)
def finalize_ephemeral_trace_batch(
self, trace_batch_id: str, payload
self, trace_batch_id: str, payload: dict[str, Any]
) -> requests.Response:
return self._make_request(
"PATCH",

View File

@@ -34,7 +34,7 @@ class SettingsCommand(BaseCommand):
current_value = getattr(self.settings, field_name)
description = field_info.description or "No description available"
display_value = (
str(current_value) if current_value is not None else "Not set"
str(current_value) if current_value not in [None, {}] else "Not set"
)
table.add_row(field_name, display_value, description)

View File

@@ -30,11 +30,11 @@ class ToolCommand(BaseCommand, PlusAPIMixin):
A class to handle tool repository related operations for CrewAI projects.
"""
def __init__(self):
def __init__(self) -> None:
BaseCommand.__init__(self)
PlusAPIMixin.__init__(self, telemetry=self._telemetry)
def create(self, handle: str):
def create(self, handle: str) -> None:
self._ensure_not_in_project()
folder_name = handle.replace(" ", "_").replace("-", "_").lower()
@@ -64,7 +64,7 @@ class ToolCommand(BaseCommand, PlusAPIMixin):
finally:
os.chdir(old_directory)
def publish(self, is_public: bool, force: bool = False):
def publish(self, is_public: bool, force: bool = False) -> None:
if not git.Repository().is_synced() and not force:
console.print(
"[bold red]Failed to publish tool.[/bold red]\n"
@@ -137,7 +137,7 @@ class ToolCommand(BaseCommand, PlusAPIMixin):
style="bold green",
)
def install(self, handle: str):
def install(self, handle: str) -> None:
self._print_current_organization()
get_response = self.plus_api_client.get_tool(handle)
@@ -180,7 +180,7 @@ class ToolCommand(BaseCommand, PlusAPIMixin):
settings.org_name = login_response_json["current_organization"]["name"]
settings.dump()
def _add_package(self, tool_details: dict[str, Any]):
def _add_package(self, tool_details: dict[str, Any]) -> None:
is_from_pypi = tool_details.get("source", None) == "pypi"
tool_handle = tool_details["handle"]
repository_handle = tool_details["repository"]["handle"]
@@ -209,7 +209,7 @@ class ToolCommand(BaseCommand, PlusAPIMixin):
click.echo(add_package_result.stderr, err=True)
raise SystemExit
def _ensure_not_in_project(self):
def _ensure_not_in_project(self) -> None:
if os.path.isfile("./pyproject.toml"):
console.print(
"[bold red]Oops! It looks like you're inside a project.[/bold red]"

View File

@@ -5,7 +5,7 @@ import os
from pathlib import Path
import shutil
import sys
from typing import Any, get_type_hints
from typing import Any, cast, get_type_hints
import click
from rich.console import Console
@@ -23,7 +23,9 @@ if sys.version_info >= (3, 11):
console = Console()
def copy_template(src, dst, name, class_name, folder_name):
def copy_template(
src: Path, dst: Path, name: str, class_name: str, folder_name: str
) -> None:
"""Copy a file from src to dst."""
with open(src, "r") as file:
content = file.read()
@@ -40,13 +42,13 @@ def copy_template(src, dst, name, class_name, folder_name):
click.secho(f" - Created {dst}", fg="green")
def read_toml(file_path: str = "pyproject.toml"):
def read_toml(file_path: str = "pyproject.toml") -> dict[str, Any]:
"""Read the content of a TOML file and return it as a dictionary."""
with open(file_path, "rb") as f:
return tomli.load(f)
def parse_toml(content):
def parse_toml(content: str) -> dict[str, Any]:
if sys.version_info >= (3, 11):
return tomllib.loads(content)
return tomli.loads(content)
@@ -103,7 +105,7 @@ def _get_project_attribute(
)
except Exception as e:
# Handle TOML decode errors for Python 3.11+
if sys.version_info >= (3, 11) and isinstance(e, tomllib.TOMLDecodeError): # type: ignore
if sys.version_info >= (3, 11) and isinstance(e, tomllib.TOMLDecodeError):
console.print(
f"Error: {pyproject_path} is not a valid TOML file.", style="bold red"
)
@@ -126,7 +128,7 @@ def _get_nested_value(data: dict[str, Any], keys: list[str]) -> Any:
return reduce(dict.__getitem__, keys, data)
def fetch_and_json_env_file(env_file_path: str = ".env") -> dict:
def fetch_and_json_env_file(env_file_path: str = ".env") -> dict[str, Any]:
"""Fetch the environment variables from a .env file and return them as a dictionary."""
try:
# Read the .env file
@@ -150,7 +152,7 @@ def fetch_and_json_env_file(env_file_path: str = ".env") -> dict:
return {}
def tree_copy(source, destination):
def tree_copy(source: Path, destination: Path) -> None:
"""Copies the entire directory structure from the source to the destination."""
for item in os.listdir(source):
source_item = os.path.join(source, item)
@@ -161,7 +163,7 @@ def tree_copy(source, destination):
shutil.copy2(source_item, destination_item)
def tree_find_and_replace(directory, find, replace):
def tree_find_and_replace(directory: Path, find: str, replace: str) -> None:
"""Recursively searches through a directory, replacing a target string in
both file contents and filenames with a specified replacement string.
"""
@@ -187,7 +189,7 @@ def tree_find_and_replace(directory, find, replace):
os.rename(old_dirpath, new_dirpath)
def load_env_vars(folder_path):
def load_env_vars(folder_path: Path) -> dict[str, Any]:
"""
Loads environment variables from a .env file in the specified folder path.
@@ -208,7 +210,9 @@ def load_env_vars(folder_path):
return env_vars
def update_env_vars(env_vars, provider, model):
def update_env_vars(
env_vars: dict[str, Any], provider: str, model: str
) -> dict[str, Any] | None:
"""
Updates environment variables with the API key for the selected provider and model.
@@ -220,15 +224,20 @@ def update_env_vars(env_vars, provider, model):
Returns:
- None
"""
api_key_var = ENV_VARS.get(
provider,
[
click.prompt(
f"Enter the environment variable name for your {provider.capitalize()} API key",
type=str,
)
],
)[0]
provider_config = cast(
list[str],
ENV_VARS.get(
provider,
[
click.prompt(
f"Enter the environment variable name for your {provider.capitalize()} API key",
type=str,
)
],
),
)
api_key_var = provider_config[0]
if api_key_var not in env_vars:
try:
@@ -246,7 +255,7 @@ def update_env_vars(env_vars, provider, model):
return env_vars
def write_env_file(folder_path, env_vars):
def write_env_file(folder_path: Path, env_vars: dict[str, Any]) -> None:
"""
Writes environment variables to a .env file in the specified folder.
@@ -342,18 +351,18 @@ def get_crews(crew_path: str = "crew.py", require: bool = False) -> list[Crew]:
return crew_instances
def get_crew_instance(module_attr) -> Crew | None:
def get_crew_instance(module_attr: Any) -> Crew | None:
if (
callable(module_attr)
and hasattr(module_attr, "is_crew_class")
and module_attr.is_crew_class
):
return module_attr().crew()
return cast(Crew, module_attr().crew())
try:
if (ismethod(module_attr) or isfunction(module_attr)) and get_type_hints(
module_attr
).get("return") is Crew:
return module_attr()
return cast(Crew, module_attr())
except Exception:
return None
@@ -362,7 +371,7 @@ def get_crew_instance(module_attr) -> Crew | None:
return None
def fetch_crews(module_attr) -> list[Crew]:
def fetch_crews(module_attr: Any) -> list[Crew]:
crew_instances: list[Crew] = []
if crew_instance := get_crew_instance(module_attr):
@@ -377,7 +386,7 @@ def fetch_crews(module_attr) -> list[Crew]:
return crew_instances
def is_valid_tool(obj):
def is_valid_tool(obj: Any) -> bool:
from crewai.tools.base_tool import Tool
if isclass(obj):
@@ -389,7 +398,7 @@ def is_valid_tool(obj):
return isinstance(obj, Tool)
def extract_available_exports(dir_path: str = "src"):
def extract_available_exports(dir_path: str = "src") -> list[dict[str, Any]]:
"""
Extract available tool classes from the project's __init__.py files.
Only includes classes that inherit from BaseTool or functions decorated with @tool.
@@ -419,7 +428,9 @@ def extract_available_exports(dir_path: str = "src"):
raise SystemExit(1) from e
def build_env_with_tool_repository_credentials(repository_handle: str):
def build_env_with_tool_repository_credentials(
repository_handle: str,
) -> dict[str, Any]:
repository_handle = repository_handle.upper().replace("-", "_")
settings = Settings()
@@ -472,7 +483,7 @@ def _load_tools_from_init(init_file: Path) -> list[dict[str, Any]]:
sys.modules.pop("temp_module", None)
def _print_no_tools_warning():
def _print_no_tools_warning() -> None:
"""
Display warning and usage instructions if no tools were found.
"""

View File

@@ -56,8 +56,8 @@ from crewai.events.types.crew_events import (
from crewai.flow.flow_trackable import FlowTrackable
from crewai.knowledge.knowledge import Knowledge
from crewai.knowledge.source.base_knowledge_source import BaseKnowledgeSource
from crewai.llm import LLM
from crewai.llms.base_llm import BaseLLM
from crewai.llm.base_llm import BaseLLM
from crewai.llm.core import LLM
from crewai.memory.entity.entity_memory import EntityMemory
from crewai.memory.external.external_memory import ExternalMemory
from crewai.memory.long_term.long_term_memory import LongTermMemory
@@ -809,6 +809,7 @@ class Crew(FlowTrackable, BaseModel):
"json_dict": output.json_dict,
"output_format": output.output_format,
"agent": output.agent,
"messages": output.messages,
},
"task_index": task_index,
"inputs": inputs,
@@ -1236,6 +1237,7 @@ class Crew(FlowTrackable, BaseModel):
pydantic=stored_output["pydantic"],
json_dict=stored_output["json_dict"],
output_format=stored_output["output_format"],
messages=stored_output.get("messages", []),
)
self.tasks[i].output = task_output

View File

@@ -89,7 +89,7 @@ from crewai.events.types.tool_usage_events import (
ToolUsageStartedEvent,
)
from crewai.events.utils.console_formatter import ConsoleFormatter
from crewai.llm import LLM
from crewai.llm.core import LLM
from crewai.task import Task
from crewai.telemetry.telemetry import Telemetry
from crewai.utilities import Logger

View File

@@ -7,7 +7,7 @@ from pydantic import BaseModel, Field
from crewai.agent import Agent
from crewai.agents.agent_builder.base_agent import BaseAgent
from crewai.llm import BaseLLM
from crewai.llm.base_llm import BaseLLM
from crewai.task import Task
from crewai.utilities.llm_utils import create_llm

View File

@@ -0,0 +1,108 @@
from __future__ import annotations
from crewai.hooks.decorators import (
after_llm_call,
after_tool_call,
before_llm_call,
before_tool_call,
)
from crewai.hooks.llm_hooks import (
LLMCallHookContext,
clear_after_llm_call_hooks,
clear_all_llm_call_hooks,
clear_before_llm_call_hooks,
get_after_llm_call_hooks,
get_before_llm_call_hooks,
register_after_llm_call_hook,
register_before_llm_call_hook,
unregister_after_llm_call_hook,
unregister_before_llm_call_hook,
)
from crewai.hooks.tool_hooks import (
ToolCallHookContext,
clear_after_tool_call_hooks,
clear_all_tool_call_hooks,
clear_before_tool_call_hooks,
get_after_tool_call_hooks,
get_before_tool_call_hooks,
register_after_tool_call_hook,
register_before_tool_call_hook,
unregister_after_tool_call_hook,
unregister_before_tool_call_hook,
)
def clear_all_global_hooks() -> dict[str, tuple[int, int]]:
"""Clear all global hooks across all hook types (LLM and Tool).
This is a convenience function that clears all registered hooks in one call.
Useful for testing, resetting state, or cleaning up between different
execution contexts.
Returns:
Dictionary with counts of cleared hooks:
{
"llm_hooks": (before_count, after_count),
"tool_hooks": (before_count, after_count),
"total": (total_before_count, total_after_count)
}
Example:
>>> # Register various hooks
>>> register_before_llm_call_hook(llm_hook1)
>>> register_after_llm_call_hook(llm_hook2)
>>> register_before_tool_call_hook(tool_hook1)
>>> register_after_tool_call_hook(tool_hook2)
>>>
>>> # Clear all hooks at once
>>> result = clear_all_global_hooks()
>>> print(result)
{
'llm_hooks': (1, 1),
'tool_hooks': (1, 1),
'total': (2, 2)
}
"""
llm_counts = clear_all_llm_call_hooks()
tool_counts = clear_all_tool_call_hooks()
return {
"llm_hooks": llm_counts,
"tool_hooks": tool_counts,
"total": (llm_counts[0] + tool_counts[0], llm_counts[1] + tool_counts[1]),
}
__all__ = [
# Context classes
"LLMCallHookContext",
"ToolCallHookContext",
# Decorators
"after_llm_call",
"after_tool_call",
"before_llm_call",
"before_tool_call",
"clear_after_llm_call_hooks",
"clear_after_tool_call_hooks",
"clear_all_global_hooks",
"clear_all_llm_call_hooks",
"clear_all_tool_call_hooks",
# Clear hooks
"clear_before_llm_call_hooks",
"clear_before_tool_call_hooks",
"get_after_llm_call_hooks",
"get_after_tool_call_hooks",
# Get hooks
"get_before_llm_call_hooks",
"get_before_tool_call_hooks",
"register_after_llm_call_hook",
"register_after_tool_call_hook",
# LLM Hook registration
"register_before_llm_call_hook",
# Tool Hook registration
"register_before_tool_call_hook",
"unregister_after_llm_call_hook",
"unregister_after_tool_call_hook",
"unregister_before_llm_call_hook",
"unregister_before_tool_call_hook",
]

View File

@@ -0,0 +1,300 @@
from __future__ import annotations
from collections.abc import Callable
from functools import wraps
import inspect
from typing import TYPE_CHECKING, Any, TypeVar, overload
if TYPE_CHECKING:
from crewai.hooks.llm_hooks import LLMCallHookContext
from crewai.hooks.tool_hooks import ToolCallHookContext
F = TypeVar("F", bound=Callable[..., Any])
def _create_hook_decorator(
hook_type: str,
register_function: Callable[..., Any],
marker_attribute: str,
) -> Callable[..., Any]:
"""Create a hook decorator with filtering support.
This factory function eliminates code duplication across the four hook decorators.
Args:
hook_type: Type of hook ("llm" or "tool")
register_function: Function to call for registration (e.g., register_before_llm_call_hook)
marker_attribute: Attribute name to mark functions (e.g., "is_before_llm_call_hook")
Returns:
A decorator function that supports filters and auto-registration
"""
def decorator_factory(
func: Callable[..., Any] | None = None,
*,
tools: list[str] | None = None,
agents: list[str] | None = None,
) -> Callable[..., Any]:
def decorator(f: Callable[..., Any]) -> Callable[..., Any]:
setattr(f, marker_attribute, True)
sig = inspect.signature(f)
params = list(sig.parameters.keys())
is_method = len(params) >= 2 and params[0] == "self"
if tools:
f._filter_tools = tools # type: ignore[attr-defined]
if agents:
f._filter_agents = agents # type: ignore[attr-defined]
if tools or agents:
@wraps(f)
def filtered_hook(context: Any) -> Any:
if tools and hasattr(context, "tool_name"):
if context.tool_name not in tools:
return None
if agents and hasattr(context, "agent"):
if context.agent and context.agent.role not in agents:
return None
return f(context)
if not is_method:
register_function(filtered_hook)
return f
if not is_method:
register_function(f)
return f
if func is None:
return decorator
return decorator(func)
return decorator_factory
@overload
def before_llm_call(
func: Callable[[LLMCallHookContext], None],
) -> Callable[[LLMCallHookContext], None]: ...
@overload
def before_llm_call(
*,
agents: list[str] | None = None,
) -> Callable[
[Callable[[LLMCallHookContext], None]], Callable[[LLMCallHookContext], None]
]: ...
def before_llm_call(
func: Callable[[LLMCallHookContext], None] | None = None,
*,
agents: list[str] | None = None,
) -> (
Callable[[LLMCallHookContext], None]
| Callable[
[Callable[[LLMCallHookContext], None]], Callable[[LLMCallHookContext], None]
]
):
"""Decorator to register a function as a before_llm_call hook.
Example:
Simple usage::
@before_llm_call
def log_calls(context):
print(f"LLM call by {context.agent.role}")
With agent filter::
@before_llm_call(agents=["Researcher", "Analyst"])
def log_specific_agents(context):
print(f"Filtered LLM call: {context.agent.role}")
"""
from crewai.hooks.llm_hooks import register_before_llm_call_hook
return _create_hook_decorator( # type: ignore[return-value]
hook_type="llm",
register_function=register_before_llm_call_hook,
marker_attribute="is_before_llm_call_hook",
)(func=func, agents=agents)
@overload
def after_llm_call(
func: Callable[[LLMCallHookContext], str | None],
) -> Callable[[LLMCallHookContext], str | None]: ...
@overload
def after_llm_call(
*,
agents: list[str] | None = None,
) -> Callable[
[Callable[[LLMCallHookContext], str | None]],
Callable[[LLMCallHookContext], str | None],
]: ...
def after_llm_call(
func: Callable[[LLMCallHookContext], str | None] | None = None,
*,
agents: list[str] | None = None,
) -> (
Callable[[LLMCallHookContext], str | None]
| Callable[
[Callable[[LLMCallHookContext], str | None]],
Callable[[LLMCallHookContext], str | None],
]
):
"""Decorator to register a function as an after_llm_call hook.
Example:
Simple usage::
@after_llm_call
def sanitize(context):
if "SECRET" in context.response:
return context.response.replace("SECRET", "[REDACTED]")
return None
With agent filter::
@after_llm_call(agents=["Researcher"])
def log_researcher_responses(context):
print(f"Response length: {len(context.response)}")
return None
"""
from crewai.hooks.llm_hooks import register_after_llm_call_hook
return _create_hook_decorator( # type: ignore[return-value]
hook_type="llm",
register_function=register_after_llm_call_hook,
marker_attribute="is_after_llm_call_hook",
)(func=func, agents=agents)
@overload
def before_tool_call(
func: Callable[[ToolCallHookContext], bool | None],
) -> Callable[[ToolCallHookContext], bool | None]: ...
@overload
def before_tool_call(
*,
tools: list[str] | None = None,
agents: list[str] | None = None,
) -> Callable[
[Callable[[ToolCallHookContext], bool | None]],
Callable[[ToolCallHookContext], bool | None],
]: ...
def before_tool_call(
func: Callable[[ToolCallHookContext], bool | None] | None = None,
*,
tools: list[str] | None = None,
agents: list[str] | None = None,
) -> (
Callable[[ToolCallHookContext], bool | None]
| Callable[
[Callable[[ToolCallHookContext], bool | None]],
Callable[[ToolCallHookContext], bool | None],
]
):
"""Decorator to register a function as a before_tool_call hook.
Example:
Simple usage::
@before_tool_call
def log_all_tools(context):
print(f"Tool: {context.tool_name}")
return None
With tool filter::
@before_tool_call(tools=["delete_file", "execute_code"])
def approve_dangerous(context):
response = context.request_human_input(prompt="Approve?")
return None if response == "yes" else False
With combined filters::
@before_tool_call(tools=["write_file"], agents=["Developer"])
def approve_dev_writes(context):
return None # Only for Developer writing files
"""
from crewai.hooks.tool_hooks import register_before_tool_call_hook
return _create_hook_decorator( # type: ignore[return-value]
hook_type="tool",
register_function=register_before_tool_call_hook,
marker_attribute="is_before_tool_call_hook",
)(func=func, tools=tools, agents=agents)
@overload
def after_tool_call(
func: Callable[[ToolCallHookContext], str | None],
) -> Callable[[ToolCallHookContext], str | None]: ...
@overload
def after_tool_call(
*,
tools: list[str] | None = None,
agents: list[str] | None = None,
) -> Callable[
[Callable[[ToolCallHookContext], str | None]],
Callable[[ToolCallHookContext], str | None],
]: ...
def after_tool_call(
func: Callable[[ToolCallHookContext], str | None] | None = None,
*,
tools: list[str] | None = None,
agents: list[str] | None = None,
) -> (
Callable[[ToolCallHookContext], str | None]
| Callable[
[Callable[[ToolCallHookContext], str | None]],
Callable[[ToolCallHookContext], str | None],
]
):
"""Decorator to register a function as an after_tool_call hook.
Example:
Simple usage::
@after_tool_call
def log_results(context):
print(f"Result: {len(context.tool_result)} chars")
return None
With tool filter::
@after_tool_call(tools=["web_search", "ExaSearchTool"])
def sanitize_search_results(context):
if "SECRET" in context.tool_result:
return context.tool_result.replace("SECRET", "[REDACTED]")
return None
"""
from crewai.hooks.tool_hooks import register_after_tool_call_hook
return _create_hook_decorator( # type: ignore[return-value]
hook_type="tool",
register_function=register_after_tool_call_hook,
marker_attribute="is_after_tool_call_hook",
)(func=func, tools=tools, agents=agents)

View File

@@ -0,0 +1,290 @@
from __future__ import annotations
from typing import TYPE_CHECKING
from crewai.events.event_listener import event_listener
from crewai.hooks.types import AfterLLMCallHookType, BeforeLLMCallHookType
from crewai.utilities.printer import Printer
if TYPE_CHECKING:
from crewai.agents.crew_agent_executor import CrewAgentExecutor
class LLMCallHookContext:
"""Context object passed to LLM call hooks with full executor access.
Provides hooks with complete access to the executor state, allowing
modification of messages, responses, and executor attributes.
Attributes:
executor: Full reference to the CrewAgentExecutor instance
messages: Direct reference to executor.messages (mutable list).
Can be modified in both before_llm_call and after_llm_call hooks.
Modifications in after_llm_call hooks persist to the next iteration,
allowing hooks to modify conversation history for subsequent LLM calls.
IMPORTANT: Modify messages in-place (e.g., append, extend, remove items).
Do NOT replace the list (e.g., context.messages = []), as this will break
the executor. Use context.messages.append() or context.messages.extend()
instead of assignment.
agent: Reference to the agent executing the task
task: Reference to the task being executed
crew: Reference to the crew instance
llm: Reference to the LLM instance
iterations: Current iteration count
response: LLM response string (only set for after_llm_call hooks).
Can be modified by returning a new string from after_llm_call hook.
"""
def __init__(
self,
executor: CrewAgentExecutor,
response: str | None = None,
) -> None:
"""Initialize hook context with executor reference.
Args:
executor: The CrewAgentExecutor instance
response: Optional response string (for after_llm_call hooks)
"""
self.executor = executor
self.messages = executor.messages
self.agent = executor.agent
self.task = executor.task
self.crew = executor.crew
self.llm = executor.llm
self.iterations = executor.iterations
self.response = response
def request_human_input(
self,
prompt: str,
default_message: str = "Press Enter to continue, or provide feedback:",
) -> str:
"""Request human input during LLM hook execution.
This method pauses live console updates, displays a prompt to the user,
waits for their input, and then resumes live updates. This is useful for
approval gates, debugging, or getting human feedback during execution.
Args:
prompt: Custom message to display to the user
default_message: Message shown after the prompt
Returns:
User's input as a string (empty string if just Enter pressed)
Example:
>>> def approval_hook(context: LLMCallHookContext) -> None:
... if context.iterations > 5:
... response = context.request_human_input(
... prompt="Allow this LLM call?",
... default_message="Type 'no' to skip, or press Enter:",
... )
... if response.lower() == "no":
... print("LLM call skipped by user")
"""
printer = Printer()
event_listener.formatter.pause_live_updates()
try:
printer.print(content=f"\n{prompt}", color="bold_yellow")
printer.print(content=default_message, color="cyan")
response = input().strip()
if response:
printer.print(content="\nProcessing your input...", color="cyan")
return response
finally:
event_listener.formatter.resume_live_updates()
_before_llm_call_hooks: list[BeforeLLMCallHookType] = []
_after_llm_call_hooks: list[AfterLLMCallHookType] = []
def register_before_llm_call_hook(
hook: BeforeLLMCallHookType,
) -> None:
"""Register a global before_llm_call hook.
Global hooks are added to all executors automatically.
This is a convenience function for registering hooks that should
apply to all LLM calls across all executors.
Args:
hook: Function that receives LLMCallHookContext and can:
- Modify context.messages directly (in-place)
- Return False to block LLM execution
- Return True or None to allow execution
IMPORTANT: Modify messages in-place (append, extend, remove items).
Do NOT replace the list (context.messages = []), as this will break execution.
Example:
>>> def log_llm_calls(context: LLMCallHookContext) -> None:
... print(f"LLM call by {context.agent.role}")
... print(f"Messages: {len(context.messages)}")
... return None # Allow execution
>>>
>>> register_before_llm_call_hook(log_llm_calls)
>>>
>>> def block_excessive_iterations(context: LLMCallHookContext) -> bool | None:
... if context.iterations > 10:
... print("Blocked: Too many iterations")
... return False # Block execution
... return None # Allow execution
>>>
>>> register_before_llm_call_hook(block_excessive_iterations)
"""
_before_llm_call_hooks.append(hook)
def register_after_llm_call_hook(
hook: AfterLLMCallHookType,
) -> None:
"""Register a global after_llm_call hook.
Global hooks are added to all executors automatically.
This is a convenience function for registering hooks that should
apply to all LLM calls across all executors.
Args:
hook: Function that receives LLMCallHookContext and can modify:
- The response: Return modified response string or None to keep original
- The messages: Modify context.messages directly (mutable reference)
Both modifications are supported and can be used together.
IMPORTANT: Modify messages in-place (append, extend, remove items).
Do NOT replace the list (context.messages = []), as this will break execution.
Example:
>>> def sanitize_response(context: LLMCallHookContext) -> str | None:
... if context.response and "SECRET" in context.response:
... return context.response.replace("SECRET", "[REDACTED]")
... return None
>>>
>>> register_after_llm_call_hook(sanitize_response)
"""
_after_llm_call_hooks.append(hook)
def get_before_llm_call_hooks() -> list[BeforeLLMCallHookType]:
"""Get all registered global before_llm_call hooks.
Returns:
List of registered before hooks
"""
return _before_llm_call_hooks.copy()
def get_after_llm_call_hooks() -> list[AfterLLMCallHookType]:
"""Get all registered global after_llm_call hooks.
Returns:
List of registered after hooks
"""
return _after_llm_call_hooks.copy()
def unregister_before_llm_call_hook(
hook: BeforeLLMCallHookType,
) -> bool:
"""Unregister a specific global before_llm_call hook.
Args:
hook: The hook function to remove
Returns:
True if the hook was found and removed, False otherwise
Example:
>>> def my_hook(context: LLMCallHookContext) -> None:
... print("Before LLM call")
>>>
>>> register_before_llm_call_hook(my_hook)
>>> unregister_before_llm_call_hook(my_hook)
True
"""
try:
_before_llm_call_hooks.remove(hook)
return True
except ValueError:
return False
def unregister_after_llm_call_hook(
hook: AfterLLMCallHookType,
) -> bool:
"""Unregister a specific global after_llm_call hook.
Args:
hook: The hook function to remove
Returns:
True if the hook was found and removed, False otherwise
Example:
>>> def my_hook(context: LLMCallHookContext) -> str | None:
... return None
>>>
>>> register_after_llm_call_hook(my_hook)
>>> unregister_after_llm_call_hook(my_hook)
True
"""
try:
_after_llm_call_hooks.remove(hook)
return True
except ValueError:
return False
def clear_before_llm_call_hooks() -> int:
"""Clear all registered global before_llm_call hooks.
Returns:
Number of hooks that were cleared
Example:
>>> register_before_llm_call_hook(hook1)
>>> register_before_llm_call_hook(hook2)
>>> clear_before_llm_call_hooks()
2
"""
count = len(_before_llm_call_hooks)
_before_llm_call_hooks.clear()
return count
def clear_after_llm_call_hooks() -> int:
"""Clear all registered global after_llm_call hooks.
Returns:
Number of hooks that were cleared
Example:
>>> register_after_llm_call_hook(hook1)
>>> register_after_llm_call_hook(hook2)
>>> clear_after_llm_call_hooks()
2
"""
count = len(_after_llm_call_hooks)
_after_llm_call_hooks.clear()
return count
def clear_all_llm_call_hooks() -> tuple[int, int]:
"""Clear all registered global LLM call hooks (both before and after).
Returns:
Tuple of (before_hooks_cleared, after_hooks_cleared)
Example:
>>> register_before_llm_call_hook(before_hook)
>>> register_after_llm_call_hook(after_hook)
>>> clear_all_llm_call_hooks()
(1, 1)
"""
before_count = clear_before_llm_call_hooks()
after_count = clear_after_llm_call_hooks()
return (before_count, after_count)

View File

@@ -0,0 +1,305 @@
from __future__ import annotations
from typing import TYPE_CHECKING, Any
from crewai.events.event_listener import event_listener
from crewai.hooks.types import AfterToolCallHookType, BeforeToolCallHookType
from crewai.utilities.printer import Printer
if TYPE_CHECKING:
from crewai.agent import Agent
from crewai.agents.agent_builder.base_agent import BaseAgent
from crewai.crew import Crew
from crewai.task import Task
from crewai.tools.structured_tool import CrewStructuredTool
class ToolCallHookContext:
"""Context object passed to tool call hooks.
Provides hooks with access to the tool being called, its input,
the agent/task/crew context, and the result (for after hooks).
Attributes:
tool_name: Name of the tool being called
tool_input: Tool input parameters (mutable dict).
Can be modified in-place by before_tool_call hooks.
IMPORTANT: Modify in-place (e.g., context.tool_input['key'] = value).
Do NOT replace the dict (e.g., context.tool_input = {}), as this
will not affect the actual tool execution.
tool: Reference to the CrewStructuredTool instance
agent: Agent executing the tool (may be None)
task: Current task being executed (may be None)
crew: Crew instance (may be None)
tool_result: Tool execution result (only set for after_tool_call hooks).
Can be modified by returning a new string from after_tool_call hook.
"""
def __init__(
self,
tool_name: str,
tool_input: dict[str, Any],
tool: CrewStructuredTool,
agent: Agent | BaseAgent | None = None,
task: Task | None = None,
crew: Crew | None = None,
tool_result: str | None = None,
) -> None:
"""Initialize tool call hook context.
Args:
tool_name: Name of the tool being called
tool_input: Tool input parameters (mutable)
tool: Tool instance reference
agent: Optional agent executing the tool
task: Optional current task
crew: Optional crew instance
tool_result: Optional tool result (for after hooks)
"""
self.tool_name = tool_name
self.tool_input = tool_input
self.tool = tool
self.agent = agent
self.task = task
self.crew = crew
self.tool_result = tool_result
def request_human_input(
self,
prompt: str,
default_message: str = "Press Enter to continue, or provide feedback:",
) -> str:
"""Request human input during tool hook execution.
This method pauses live console updates, displays a prompt to the user,
waits for their input, and then resumes live updates. This is useful for
approval gates, reviewing tool results, or getting human feedback during execution.
Args:
prompt: Custom message to display to the user
default_message: Message shown after the prompt
Returns:
User's input as a string (empty string if just Enter pressed)
Example:
>>> def approval_hook(context: ToolCallHookContext) -> bool | None:
... if context.tool_name == "delete_file":
... response = context.request_human_input(
... prompt="Allow file deletion?",
... default_message="Type 'approve' to continue:",
... )
... if response.lower() != "approve":
... return False # Block execution
... return None # Allow execution
"""
printer = Printer()
event_listener.formatter.pause_live_updates()
try:
printer.print(content=f"\n{prompt}", color="bold_yellow")
printer.print(content=default_message, color="cyan")
response = input().strip()
if response:
printer.print(content="\nProcessing your input...", color="cyan")
return response
finally:
event_listener.formatter.resume_live_updates()
# Global hook registries
_before_tool_call_hooks: list[BeforeToolCallHookType] = []
_after_tool_call_hooks: list[AfterToolCallHookType] = []
def register_before_tool_call_hook(
hook: BeforeToolCallHookType,
) -> None:
"""Register a global before_tool_call hook.
Global hooks are added to all tool executions automatically.
This is a convenience function for registering hooks that should
apply to all tool calls across all agents and crews.
Args:
hook: Function that receives ToolCallHookContext and can:
- Modify tool_input in-place
- Return False to block tool execution
- Return True or None to allow execution
IMPORTANT: Modify tool_input in-place (e.g., context.tool_input['key'] = value).
Do NOT replace the dict (context.tool_input = {}), as this will not affect
the actual tool execution.
Example:
>>> def log_tool_usage(context: ToolCallHookContext) -> None:
... print(f"Executing tool: {context.tool_name}")
... print(f"Input: {context.tool_input}")
... return None # Allow execution
>>>
>>> register_before_tool_call_hook(log_tool_usage)
>>> def block_dangerous_tools(context: ToolCallHookContext) -> bool | None:
... if context.tool_name == "delete_database":
... print("Blocked dangerous tool execution!")
... return False # Block execution
... return None # Allow execution
>>>
>>> register_before_tool_call_hook(block_dangerous_tools)
"""
_before_tool_call_hooks.append(hook)
def register_after_tool_call_hook(
hook: AfterToolCallHookType,
) -> None:
"""Register a global after_tool_call hook.
Global hooks are added to all tool executions automatically.
This is a convenience function for registering hooks that should
apply to all tool calls across all agents and crews.
Args:
hook: Function that receives ToolCallHookContext and can modify
the tool result. Return modified result string or None to keep
the original result. The tool_result is available in context.tool_result.
Example:
>>> def sanitize_output(context: ToolCallHookContext) -> str | None:
... if context.tool_result and "SECRET_KEY" in context.tool_result:
... return context.tool_result.replace("SECRET_KEY=...", "[REDACTED]")
... return None # Keep original result
>>>
>>> register_after_tool_call_hook(sanitize_output)
>>> def log_tool_results(context: ToolCallHookContext) -> None:
... print(f"Tool {context.tool_name} returned: {context.tool_result[:100]}")
... return None # Keep original result
>>>
>>> register_after_tool_call_hook(log_tool_results)
"""
_after_tool_call_hooks.append(hook)
def get_before_tool_call_hooks() -> list[BeforeToolCallHookType]:
"""Get all registered global before_tool_call hooks.
Returns:
List of registered before hooks
"""
return _before_tool_call_hooks.copy()
def get_after_tool_call_hooks() -> list[AfterToolCallHookType]:
"""Get all registered global after_tool_call hooks.
Returns:
List of registered after hooks
"""
return _after_tool_call_hooks.copy()
def unregister_before_tool_call_hook(
hook: BeforeToolCallHookType,
) -> bool:
"""Unregister a specific global before_tool_call hook.
Args:
hook: The hook function to remove
Returns:
True if the hook was found and removed, False otherwise
Example:
>>> def my_hook(context: ToolCallHookContext) -> None:
... print("Before tool call")
>>>
>>> register_before_tool_call_hook(my_hook)
>>> unregister_before_tool_call_hook(my_hook)
True
"""
try:
_before_tool_call_hooks.remove(hook)
return True
except ValueError:
return False
def unregister_after_tool_call_hook(
hook: AfterToolCallHookType,
) -> bool:
"""Unregister a specific global after_tool_call hook.
Args:
hook: The hook function to remove
Returns:
True if the hook was found and removed, False otherwise
Example:
>>> def my_hook(context: ToolCallHookContext) -> str | None:
... return None
>>>
>>> register_after_tool_call_hook(my_hook)
>>> unregister_after_tool_call_hook(my_hook)
True
"""
try:
_after_tool_call_hooks.remove(hook)
return True
except ValueError:
return False
def clear_before_tool_call_hooks() -> int:
"""Clear all registered global before_tool_call hooks.
Returns:
Number of hooks that were cleared
Example:
>>> register_before_tool_call_hook(hook1)
>>> register_before_tool_call_hook(hook2)
>>> clear_before_tool_call_hooks()
2
"""
count = len(_before_tool_call_hooks)
_before_tool_call_hooks.clear()
return count
def clear_after_tool_call_hooks() -> int:
"""Clear all registered global after_tool_call hooks.
Returns:
Number of hooks that were cleared
Example:
>>> register_after_tool_call_hook(hook1)
>>> register_after_tool_call_hook(hook2)
>>> clear_after_tool_call_hooks()
2
"""
count = len(_after_tool_call_hooks)
_after_tool_call_hooks.clear()
return count
def clear_all_tool_call_hooks() -> tuple[int, int]:
"""Clear all registered global tool call hooks (both before and after).
Returns:
Tuple of (before_hooks_cleared, after_hooks_cleared)
Example:
>>> register_before_tool_call_hook(before_hook)
>>> register_after_tool_call_hook(after_hook)
>>> clear_all_tool_call_hooks()
(1, 1)
"""
before_count = clear_before_tool_call_hooks()
after_count = clear_after_tool_call_hooks()
return (before_count, after_count)

View File

@@ -0,0 +1,137 @@
from __future__ import annotations
from collections.abc import Callable
from typing import TYPE_CHECKING, Generic, Protocol, TypeVar, runtime_checkable
if TYPE_CHECKING:
from crewai.hooks.llm_hooks import LLMCallHookContext
from crewai.hooks.tool_hooks import ToolCallHookContext
ContextT = TypeVar("ContextT", contravariant=True)
ReturnT = TypeVar("ReturnT", covariant=True)
@runtime_checkable
class Hook(Protocol, Generic[ContextT, ReturnT]):
"""Generic protocol for hook functions.
This protocol defines the common interface for all hook types in CrewAI.
Hooks receive a context object and optionally return a modified result.
Type Parameters:
ContextT: The context type (LLMCallHookContext or ToolCallHookContext)
ReturnT: The return type (None, str | None, or bool | None)
Example:
>>> # Before LLM call hook: receives LLMCallHookContext, returns None
>>> hook: Hook[LLMCallHookContext, None] = lambda ctx: print(ctx.iterations)
>>>
>>> # After LLM call hook: receives LLMCallHookContext, returns str | None
>>> hook: Hook[LLMCallHookContext, str | None] = lambda ctx: ctx.response
"""
def __call__(self, context: ContextT) -> ReturnT:
"""Execute the hook with the given context.
Args:
context: Context object with relevant execution state
Returns:
Hook-specific return value (None, str | None, or bool | None)
"""
...
class BeforeLLMCallHook(Hook["LLMCallHookContext", bool | None], Protocol):
"""Protocol for before_llm_call hooks.
These hooks are called before an LLM is invoked and can modify the messages
that will be sent to the LLM or block the execution entirely.
"""
def __call__(self, context: LLMCallHookContext) -> bool | None:
"""Execute the before LLM call hook.
Args:
context: Context object with executor, messages, agent, task, etc.
Messages can be modified in-place.
Returns:
False to block LLM execution, True or None to allow execution
"""
...
class AfterLLMCallHook(Hook["LLMCallHookContext", str | None], Protocol):
"""Protocol for after_llm_call hooks.
These hooks are called after an LLM returns a response and can modify
the response or the message history.
"""
def __call__(self, context: LLMCallHookContext) -> str | None:
"""Execute the after LLM call hook.
Args:
context: Context object with executor, messages, agent, task, response, etc.
Messages can be modified in-place. Response is available in context.response.
Returns:
Modified response string, or None to keep the original response
"""
...
class BeforeToolCallHook(Hook["ToolCallHookContext", bool | None], Protocol):
"""Protocol for before_tool_call hooks.
These hooks are called before a tool is executed and can modify the tool
input or block the execution entirely.
"""
def __call__(self, context: ToolCallHookContext) -> bool | None:
"""Execute the before tool call hook.
Args:
context: Context object with tool_name, tool_input, tool, agent, task, etc.
Tool input can be modified in-place.
Returns:
False to block tool execution, True or None to allow execution
"""
...
class AfterToolCallHook(Hook["ToolCallHookContext", str | None], Protocol):
"""Protocol for after_tool_call hooks.
These hooks are called after a tool executes and can modify the result.
"""
def __call__(self, context: ToolCallHookContext) -> str | None:
"""Execute the after tool call hook.
Args:
context: Context object with tool_name, tool_input, tool_result, etc.
Tool result is available in context.tool_result.
Returns:
Modified tool result string, or None to keep the original result
"""
...
# - All before hooks: bool | None (False = block execution, True/None = allow)
# - All after hooks: str | None (str = modified result, None = keep original)
BeforeLLMCallHookType = Hook["LLMCallHookContext", bool | None]
AfterLLMCallHookType = Hook["LLMCallHookContext", str | None]
BeforeToolCallHookType = Hook["ToolCallHookContext", bool | None]
AfterToolCallHookType = Hook["ToolCallHookContext", str | None]
# Alternative Callable-based type aliases for compatibility
BeforeLLMCallHookCallable = Callable[["LLMCallHookContext"], bool | None]
AfterLLMCallHookCallable = Callable[["LLMCallHookContext"], str | None]
BeforeToolCallHookCallable = Callable[["ToolCallHookContext"], bool | None]
AfterToolCallHookCallable = Callable[["ToolCallHookContext"], str | None]

View File

@@ -0,0 +1,157 @@
from __future__ import annotations
from collections.abc import Callable
from typing import TYPE_CHECKING, Any, TypeVar
if TYPE_CHECKING:
from crewai.hooks.llm_hooks import LLMCallHookContext
from crewai.hooks.tool_hooks import ToolCallHookContext
P = TypeVar("P")
R = TypeVar("R")
def _copy_method_metadata(wrapper: Any, original: Callable[..., Any]) -> None:
"""Copy metadata from original function to wrapper.
Args:
wrapper: The wrapper object to copy metadata to
original: The original function to copy from
"""
wrapper.__name__ = original.__name__
wrapper.__doc__ = original.__doc__
wrapper.__module__ = original.__module__
wrapper.__qualname__ = original.__qualname__
wrapper.__annotations__ = original.__annotations__
class BeforeLLMCallHookMethod:
"""Wrapper for methods marked as before_llm_call hooks within @CrewBase classes.
This wrapper marks a method so it can be detected and registered as a
crew-scoped hook during crew initialization.
"""
is_before_llm_call_hook: bool = True
def __init__(
self,
meth: Callable[[Any, LLMCallHookContext], None],
agents: list[str] | None = None,
) -> None:
"""Initialize the hook method wrapper.
Args:
meth: The method to wrap
agents: Optional list of agent roles to filter
"""
self._meth = meth
self.agents = agents
_copy_method_metadata(self, meth)
def __call__(self, *args: Any, **kwargs: Any) -> None:
"""Call the wrapped method.
Args:
*args: Positional arguments
**kwargs: Keyword arguments
"""
return self._meth(*args, **kwargs)
def __get__(self, obj: Any, objtype: type[Any] | None = None) -> Any:
"""Support instance methods by implementing descriptor protocol.
Args:
obj: The instance that the method is accessed through
objtype: The type of the instance
Returns:
Self when accessed through class, bound method when accessed through instance
"""
if obj is None:
return self
# Return bound method
return lambda context: self._meth(obj, context)
class AfterLLMCallHookMethod:
"""Wrapper for methods marked as after_llm_call hooks within @CrewBase classes."""
is_after_llm_call_hook: bool = True
def __init__(
self,
meth: Callable[[Any, LLMCallHookContext], str | None],
agents: list[str] | None = None,
) -> None:
"""Initialize the hook method wrapper."""
self._meth = meth
self.agents = agents
_copy_method_metadata(self, meth)
def __call__(self, *args: Any, **kwargs: Any) -> str | None:
"""Call the wrapped method."""
return self._meth(*args, **kwargs)
def __get__(self, obj: Any, objtype: type[Any] | None = None) -> Any:
"""Support instance methods."""
if obj is None:
return self
return lambda context: self._meth(obj, context)
class BeforeToolCallHookMethod:
"""Wrapper for methods marked as before_tool_call hooks within @CrewBase classes."""
is_before_tool_call_hook: bool = True
def __init__(
self,
meth: Callable[[Any, ToolCallHookContext], bool | None],
tools: list[str] | None = None,
agents: list[str] | None = None,
) -> None:
"""Initialize the hook method wrapper."""
self._meth = meth
self.tools = tools
self.agents = agents
_copy_method_metadata(self, meth)
def __call__(self, *args: Any, **kwargs: Any) -> bool | None:
"""Call the wrapped method."""
return self._meth(*args, **kwargs)
def __get__(self, obj: Any, objtype: type[Any] | None = None) -> Any:
"""Support instance methods."""
if obj is None:
return self
return lambda context: self._meth(obj, context)
class AfterToolCallHookMethod:
"""Wrapper for methods marked as after_tool_call hooks within @CrewBase classes."""
is_after_tool_call_hook: bool = True
def __init__(
self,
meth: Callable[[Any, ToolCallHookContext], str | None],
tools: list[str] | None = None,
agents: list[str] | None = None,
) -> None:
"""Initialize the hook method wrapper."""
self._meth = meth
self.tools = tools
self.agents = agents
_copy_method_metadata(self, meth)
def __call__(self, *args: Any, **kwargs: Any) -> str | None:
"""Call the wrapped method."""
return self._meth(*args, **kwargs)
def __get__(self, obj: Any, objtype: type[Any] | None = None) -> Any:
"""Support instance methods."""
if obj is None:
return self
return lambda context: self._meth(obj, context)

View File

@@ -39,8 +39,8 @@ from crewai.events.types.agent_events import (
from crewai.events.types.logging_events import AgentLogsExecutionEvent
from crewai.flow.flow_trackable import FlowTrackable
from crewai.lite_agent_output import LiteAgentOutput
from crewai.llm import LLM
from crewai.llms.base_llm import BaseLLM
from crewai.llm.base_llm import BaseLLM
from crewai.llm.core import LLM
from crewai.tools.base_tool import BaseTool
from crewai.tools.structured_tool import CrewStructuredTool
from crewai.utilities.agent_utils import (
@@ -358,6 +358,7 @@ class LiteAgent(FlowTrackable, BaseModel):
pydantic=formatted_result,
agent_role=self.role,
usage_metrics=usage_metrics.model_dump() if usage_metrics else None,
messages=self._messages,
)
# Process guardrail if set
@@ -503,7 +504,7 @@ class LiteAgent(FlowTrackable, BaseModel):
AgentFinish: The final result of the agent execution.
"""
# Execute the agent loop
formatted_answer = None
formatted_answer: AgentAction | AgentFinish | None = None
while not isinstance(formatted_answer, AgentFinish):
try:
if has_reached_max_iterations(self._iterations, self.max_iterations):
@@ -541,6 +542,7 @@ class LiteAgent(FlowTrackable, BaseModel):
agent_key=self.key,
agent_role=self.role,
agent=self.original_agent,
crew=None,
)
except Exception as e:
raise e
@@ -551,7 +553,8 @@ class LiteAgent(FlowTrackable, BaseModel):
show_logs=self._show_logs,
)
self._append_message(formatted_answer.text, role="assistant")
if formatted_answer is not None:
self._append_message(formatted_answer.text, role="assistant")
except OutputParserError as e: # noqa: PERF203
self._printer.print(
content="Failed to parse LLM output. Retrying...",

View File

@@ -6,6 +6,8 @@ from typing import Any
from pydantic import BaseModel, Field
from crewai.utilities.types import LLMMessage
class LiteAgentOutput(BaseModel):
"""Class that represents the result of a LiteAgent execution."""
@@ -20,6 +22,7 @@ class LiteAgentOutput(BaseModel):
usage_metrics: dict[str, Any] | None = Field(
description="Token usage metrics for this execution", default=None
)
messages: list[LLMMessage] = Field(description="Messages of the agent", default=[])
def to_dict(self) -> dict[str, Any]:
"""Convert pydantic_output to a dictionary."""

View File

@@ -0,0 +1,4 @@
from crewai.llm.core import LLM
__all__ = ["LLM"]

View File

@@ -0,0 +1,588 @@
"""Base LLM abstract class for CrewAI.
This module provides the abstract base class for all LLM implementations
in CrewAI, including common functionality for native SDK implementations.
"""
from __future__ import annotations
from abc import ABC, abstractmethod
from datetime import datetime
import json
import logging
import os
import re
from typing import TYPE_CHECKING, Any, Final
from dotenv import load_dotenv
import httpx
from pydantic import BaseModel, Field, field_validator
from crewai.events.event_bus import crewai_event_bus
from crewai.events.types.llm_events import (
LLMCallCompletedEvent,
LLMCallFailedEvent,
LLMCallStartedEvent,
LLMCallType,
LLMStreamChunkEvent,
)
from crewai.events.types.tool_usage_events import (
ToolUsageErrorEvent,
ToolUsageFinishedEvent,
ToolUsageStartedEvent,
)
from crewai.llm.hooks.base import BaseInterceptor
from crewai.llm.internal.meta import LLMMeta
from crewai.types.usage_metrics import UsageMetrics
if TYPE_CHECKING:
from crewai.agent.core import Agent
from crewai.task import Task
from crewai.tools.base_tool import BaseTool
from crewai.utilities.types import LLMMessage
load_dotenv()
DEFAULT_CONTEXT_WINDOW_SIZE: Final[int] = 4096
DEFAULT_SUPPORTS_STOP_WORDS: Final[bool] = True
_JSON_EXTRACTION_PATTERN: Final[re.Pattern[str]] = re.compile(r"\{.*}", re.DOTALL)
class BaseLLM(BaseModel, ABC, metaclass=LLMMeta):
"""Abstract base class for LLM implementations.
This class defines the interface that all LLM implementations must follow.
Users can extend this class to create custom LLM implementations that don't
rely on litellm's authentication mechanism.
Custom LLM implementations should handle error cases gracefully, including
timeouts, authentication failures, and malformed responses. They should also
implement proper validation for input parameters and provide clear error
messages when things go wrong.
Attributes:
model: The model identifier/name.
temperature: Optional temperature setting for response generation.
stop: A list of stop sequences that the LLM should use to stop generation.
"""
# Core fields
model: str = Field(..., description="The model identifier/name")
temperature: float | None = Field(
None, description="Temperature setting for response generation"
)
api_key: str | None = Field(None, description="API key for authentication")
base_url: str | None = Field(None, description="Base URL for API requests")
provider: str = Field(
default="openai", description="Provider name (openai, anthropic, etc.)"
)
stop: list[str] = Field(
default_factory=list,
description="Stop sequences for generation",
alias="stop_sequences",
)
# Internal fields
is_litellm: bool = Field(
default=False, description="Whether this instance uses LiteLLM"
)
interceptor: BaseInterceptor[httpx.Request, httpx.Response] | None = Field(
default=None, description="HTTP request/response interceptor"
)
_token_usage: dict[str, int] = {
"total_tokens": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"successful_requests": 0,
"cached_prompt_tokens": 0,
}
@field_validator("api_key", mode="after")
@classmethod
def _validate_api_key(cls, value: str | None) -> str | None:
"""Validate API key for authentication.
Args:
value: API key value or None
Returns:
API key from environment if not provided, or the original value
"""
if value is None:
cls_name = cls.__name__
provider_prefix = cls_name.replace("Completion", "").upper()
env_var = f"{provider_prefix}_API_KEY"
value = os.getenv(env_var)
return value
@field_validator("stop", mode="before")
@classmethod
def _normalize_stop(cls, value: Any) -> list[str]:
"""Normalize stop sequences to a list.
Args:
value: Stop sequences as string, list, or None
Returns:
Normalized list of stop sequences
"""
if value is None:
return []
if isinstance(value, str):
return [value]
if isinstance(value, list):
return value
return []
@property
def additional_params(self) -> dict[str, Any]:
"""Get additional parameters stored as extra fields.
Returns:
Dictionary of additional parameters
"""
return self.__pydantic_extra__ or {}
@additional_params.setter
def additional_params(self, value: dict[str, Any]) -> None:
"""Set additional parameters as extra fields.
Args:
value: Dictionary of additional parameters to set
"""
if not isinstance(value, dict):
raise ValueError("additional_params must be a dictionary")
if self.__pydantic_extra__ is None:
self.__pydantic_extra__ = {}
self.__pydantic_extra__.update(value)
@abstractmethod
def call(
self,
messages: str | list[LLMMessage],
tools: list[dict[str, BaseTool]] | None = None,
callbacks: list[Any] | None = None,
available_functions: dict[str, Any] | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
response_model: type[BaseModel] | None = None,
) -> str | Any:
"""Call the LLM with the given messages.
Args:
messages: Input messages for the LLM.
Can be a string or list of message dictionaries.
If string, it will be converted to a single user message.
If list, each dict must have 'role' and 'content' keys.
tools: Optional list of tool schemas for function calling.
Each tool should define its name, description, and parameters.
callbacks: Optional list of callback functions to be executed
during and after the LLM call.
available_functions: Optional dict mapping function names to callables
that can be invoked by the LLM.
from_task: Optional task caller to be used for the LLM call.
from_agent: Optional agent caller to be used for the LLM call.
response_model: Optional response model to be used for the LLM call.
Returns:
Either a text response from the LLM (str) or
the result of a tool function call (Any).
Raises:
ValueError: If the messages format is invalid.
TimeoutError: If the LLM request times out.
RuntimeError: If the LLM request fails for other reasons.
"""
def _convert_tools_for_interference(
self, tools: list[dict[str, BaseTool]]
) -> list[dict[str, BaseTool]]:
"""Convert tools to a format that can be used for interference.
Args:
tools: List of tools to convert.
Returns:
List of converted tools (default implementation returns as-is)
"""
return tools
def supports_stop_words(self) -> bool:
"""Check if the LLM supports stop words.
Returns:
True if the LLM supports stop words, False otherwise.
"""
return DEFAULT_SUPPORTS_STOP_WORDS
def _supports_stop_words_implementation(self) -> bool:
"""Check if stop words are configured for this LLM instance.
Native providers can override supports_stop_words() to return this value
to ensure consistent behavior based on whether stop words are actually configured.
Returns:
True if stop words are configured and can be applied
"""
return bool(self.stop)
def _apply_stop_words(self, content: str) -> str:
"""Apply stop words to truncate response content.
This method provides consistent stop word behavior across all native SDK providers.
Native providers should call this method to post-process their responses.
Args:
content: The raw response content from the LLM
Returns:
Content truncated at the first occurrence of any stop word
Example:
>>> llm = MyNativeLLM(stop=["Observation:", "Final Answer:"])
>>> response = (
... "I need to search.\\n\\nAction: search\\nObservation: Found results"
... )
>>> llm._apply_stop_words(response)
"I need to search.\\n\\nAction: search"
"""
if not self.stop or not content:
return content
# Find the earliest occurrence of any stop word
earliest_stop_pos = len(content)
found_stop_word = None
for stop_word in self.stop:
stop_pos = content.find(stop_word)
if stop_pos != -1 and stop_pos < earliest_stop_pos:
earliest_stop_pos = stop_pos
found_stop_word = stop_word
# Truncate at the stop word if found
if found_stop_word is not None:
truncated = content[:earliest_stop_pos].strip()
logging.debug(
f"Applied stop word '{found_stop_word}' at position {earliest_stop_pos}"
)
return truncated
return content
def get_context_window_size(self) -> int:
"""Get the context window size for the LLM.
Returns:
The number of tokens/characters the model can handle.
"""
# Default implementation - subclasses should override with model-specific values
return DEFAULT_CONTEXT_WINDOW_SIZE
# Common helper methods for native SDK implementations
def _emit_call_started_event(
self,
messages: str | list[LLMMessage],
tools: list[dict[str, BaseTool]] | None = None,
callbacks: list[Any] | None = None,
available_functions: dict[str, Any] | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
) -> None:
"""Emit LLM call started event."""
if not hasattr(crewai_event_bus, "emit"):
raise ValueError("crewai_event_bus does not have an emit method") from None
crewai_event_bus.emit(
self,
event=LLMCallStartedEvent(
messages=messages,
tools=tools,
callbacks=callbacks,
available_functions=available_functions,
from_task=from_task,
from_agent=from_agent,
model=self.model,
),
)
def _emit_call_completed_event(
self,
response: Any,
call_type: LLMCallType,
from_task: Task | None = None,
from_agent: Agent | None = None,
messages: str | list[dict[str, Any]] | None = None,
) -> None:
"""Emit LLM call completed event."""
crewai_event_bus.emit(
self,
event=LLMCallCompletedEvent(
messages=messages,
response=response,
call_type=call_type,
from_task=from_task,
from_agent=from_agent,
model=self.model,
),
)
def _emit_call_failed_event(
self,
error: str,
from_task: Task | None = None,
from_agent: Agent | None = None,
) -> None:
"""Emit LLM call failed event."""
if not hasattr(crewai_event_bus, "emit"):
raise ValueError("crewai_event_bus does not have an emit method") from None
crewai_event_bus.emit(
self,
event=LLMCallFailedEvent(
error=error,
from_task=from_task,
from_agent=from_agent,
),
)
def _emit_stream_chunk_event(
self,
chunk: str,
from_task: Task | None = None,
from_agent: Agent | None = None,
tool_call: dict[str, Any] | None = None,
) -> None:
"""Emit stream chunk event."""
if not hasattr(crewai_event_bus, "emit"):
raise ValueError("crewai_event_bus does not have an emit method") from None
crewai_event_bus.emit(
self,
event=LLMStreamChunkEvent(
chunk=chunk,
tool_call=tool_call,
from_task=from_task,
from_agent=from_agent,
),
)
def _handle_tool_execution(
self,
function_name: str,
function_args: dict[str, Any],
available_functions: dict[str, Any],
from_task: Task | None = None,
from_agent: Agent | None = None,
) -> str | None:
"""Handle tool execution with proper event emission.
Args:
function_name: Name of the function to execute
function_args: Arguments to pass to the function
available_functions: Dict of available functions
from_task: Optional task object
from_agent: Optional agent object
Returns:
Result of function execution or None if function not found
"""
if function_name not in available_functions:
logging.warning(
f"Function '{function_name}' not found in available functions"
)
return None
try:
# Emit tool usage started event
started_at = datetime.now()
crewai_event_bus.emit(
self,
event=ToolUsageStartedEvent(
tool_name=function_name,
tool_args=function_args,
from_agent=from_agent,
from_task=from_task,
),
)
# Execute the function
fn = available_functions[function_name]
result = fn(**function_args)
# Emit tool usage finished event
crewai_event_bus.emit(
self,
event=ToolUsageFinishedEvent(
output=result,
tool_name=function_name,
tool_args=function_args,
started_at=started_at,
finished_at=datetime.now(),
from_task=from_task,
from_agent=from_agent,
),
)
# Emit LLM call completed event for tool call
self._emit_call_completed_event(
response=result,
call_type=LLMCallType.TOOL_CALL,
from_task=from_task,
from_agent=from_agent,
)
return str(result)
except Exception as e:
error_msg = f"Error executing function '{function_name}': {e!s}"
logging.error(error_msg)
# Emit tool usage error event
if not hasattr(crewai_event_bus, "emit"):
raise ValueError(
"crewai_event_bus does not have an emit method"
) from None
crewai_event_bus.emit(
self,
event=ToolUsageErrorEvent(
tool_name=function_name,
tool_args=function_args,
error=error_msg,
from_task=from_task,
from_agent=from_agent,
),
)
# Emit LLM call failed event
self._emit_call_failed_event(
error=error_msg,
from_task=from_task,
from_agent=from_agent,
)
return None
def _format_messages(self, messages: str | list[LLMMessage]) -> list[LLMMessage]:
"""Convert messages to standard format.
Args:
messages: Input messages (string or list of message dicts)
Returns:
List of message dictionaries with 'role' and 'content' keys
Raises:
ValueError: If message format is invalid
"""
if isinstance(messages, str):
return [{"role": "user", "content": messages}]
# Validate message format
for i, msg in enumerate(messages):
if not isinstance(msg, dict):
raise ValueError(f"Message at index {i} must be a dictionary")
if "role" not in msg or "content" not in msg:
raise ValueError(
f"Message at index {i} must have 'role' and 'content' keys"
)
return messages
@staticmethod
def _validate_structured_output(
response: str,
response_format: type[BaseModel] | None,
) -> str | BaseModel:
"""Validate and parse structured output.
Args:
response: Raw response string
response_format: Optional Pydantic model for structured output
Returns:
Parsed response (BaseModel instance if response_format provided, otherwise string)
Raises:
ValueError: If structured output validation fails
"""
if response_format is None:
return response
try:
# Try to parse as JSON first
if response.strip().startswith("{") or response.strip().startswith("["):
data = json.loads(response)
return response_format.model_validate(data)
json_match = _JSON_EXTRACTION_PATTERN.search(response)
if json_match:
data = json.loads(json_match.group())
return response_format.model_validate(data)
raise ValueError("No JSON found in response")
except (json.JSONDecodeError, ValueError) as e:
logging.warning(f"Failed to parse structured output: {e}")
raise ValueError(
f"Failed to parse response into {response_format.__name__}: {e}"
) from e
@staticmethod
def _extract_provider(model: str) -> str:
"""Extract provider from model string.
Args:
model: Model string (e.g., 'openai/gpt-4' or 'gpt-4')
Returns:
Provider name (e.g., 'openai')
"""
if "/" in model:
return model.partition("/")[0]
return "openai" # Default provider
def _track_token_usage_internal(self, usage_data: dict[str, Any]) -> None:
"""Track token usage internally in the LLM instance.
Args:
usage_data: Token usage data from the API response
"""
# Extract tokens in a provider-agnostic way
prompt_tokens = (
usage_data.get("prompt_tokens")
or usage_data.get("prompt_token_count")
or usage_data.get("input_tokens")
or 0
)
completion_tokens = (
usage_data.get("completion_tokens")
or usage_data.get("candidates_token_count")
or usage_data.get("output_tokens")
or 0
)
cached_tokens = (
usage_data.get("cached_tokens")
or usage_data.get("cached_prompt_tokens")
or 0
)
self._token_usage["prompt_tokens"] += prompt_tokens
self._token_usage["completion_tokens"] += completion_tokens
self._token_usage["total_tokens"] += prompt_tokens + completion_tokens
self._token_usage["successful_requests"] += 1
self._token_usage["cached_prompt_tokens"] += cached_tokens
def get_token_usage_summary(self) -> UsageMetrics:
"""Get summary of token usage for this LLM instance.
Returns:
Dictionary with token usage totals
"""
return UsageMetrics(**self._token_usage)

View File

@@ -0,0 +1,587 @@
from typing import Literal, TypeAlias
SupportedNativeProviders: TypeAlias = Literal[
"openai",
"anthropic",
"claude",
"azure",
"azure_openai",
"google",
"gemini",
"bedrock",
"aws",
]
SUPPORTED_NATIVE_PROVIDERS: list[SupportedNativeProviders] = [
"openai",
"anthropic",
"claude",
"azure",
"azure_openai",
"google",
"gemini",
"bedrock",
"aws",
]
OpenAIModels: TypeAlias = Literal[
"gpt-3.5-turbo",
"gpt-3.5-turbo-0125",
"gpt-3.5-turbo-0301",
"gpt-3.5-turbo-0613",
"gpt-3.5-turbo-1106",
"gpt-3.5-turbo-16k",
"gpt-3.5-turbo-16k-0613",
"gpt-3.5-turbo-instruct",
"gpt-3.5-turbo-instruct-0914",
"gpt-4",
"gpt-4-0125-preview",
"gpt-4-0314",
"gpt-4-0613",
"gpt-4-1106-preview",
"gpt-4-32k",
"gpt-4-32k-0314",
"gpt-4-32k-0613",
"gpt-4-turbo",
"gpt-4-turbo-2024-04-09",
"gpt-4-turbo-preview",
"gpt-4-vision-preview",
"gpt-4.1",
"gpt-4.1-2025-04-14",
"gpt-4.1-mini",
"gpt-4.1-mini-2025-04-14",
"gpt-4.1-nano",
"gpt-4.1-nano-2025-04-14",
"gpt-4o",
"gpt-4o-2024-05-13",
"gpt-4o-2024-08-06",
"gpt-4o-2024-11-20",
"gpt-4o-audio-preview",
"gpt-4o-audio-preview-2024-10-01",
"gpt-4o-audio-preview-2024-12-17",
"gpt-4o-audio-preview-2025-06-03",
"gpt-4o-mini",
"gpt-4o-mini-2024-07-18",
"gpt-4o-mini-audio-preview",
"gpt-4o-mini-audio-preview-2024-12-17",
"gpt-4o-mini-realtime-preview",
"gpt-4o-mini-realtime-preview-2024-12-17",
"gpt-4o-mini-search-preview",
"gpt-4o-mini-search-preview-2025-03-11",
"gpt-4o-mini-transcribe",
"gpt-4o-mini-tts",
"gpt-4o-realtime-preview",
"gpt-4o-realtime-preview-2024-10-01",
"gpt-4o-realtime-preview-2024-12-17",
"gpt-4o-realtime-preview-2025-06-03",
"gpt-4o-search-preview",
"gpt-4o-search-preview-2025-03-11",
"gpt-4o-transcribe",
"gpt-4o-transcribe-diarize",
"gpt-5",
"gpt-5-2025-08-07",
"gpt-5-chat",
"gpt-5-chat-latest",
"gpt-5-codex",
"gpt-5-mini",
"gpt-5-mini-2025-08-07",
"gpt-5-nano",
"gpt-5-nano-2025-08-07",
"gpt-5-pro",
"gpt-5-pro-2025-10-06",
"gpt-5-search-api",
"gpt-5-search-api-2025-10-14",
"gpt-audio",
"gpt-audio-2025-08-28",
"gpt-audio-mini",
"gpt-audio-mini-2025-10-06",
"gpt-image-1",
"gpt-image-1-mini",
"gpt-realtime",
"gpt-realtime-2025-08-28",
"gpt-realtime-mini",
"gpt-realtime-mini-2025-10-06",
"o1",
"o1-preview",
"o1-2024-12-17",
"o1-mini",
"o1-mini-2024-09-12",
"o1-pro",
"o1-pro-2025-03-19",
"o3-mini",
"o3",
"o4-mini",
"whisper-1",
]
OPENAI_MODELS: list[OpenAIModels] = [
"gpt-3.5-turbo",
"gpt-3.5-turbo-0125",
"gpt-3.5-turbo-0301",
"gpt-3.5-turbo-0613",
"gpt-3.5-turbo-1106",
"gpt-3.5-turbo-16k",
"gpt-3.5-turbo-16k-0613",
"gpt-3.5-turbo-instruct",
"gpt-3.5-turbo-instruct-0914",
"gpt-4",
"gpt-4-0125-preview",
"gpt-4-0314",
"gpt-4-0613",
"gpt-4-1106-preview",
"gpt-4-32k",
"gpt-4-32k-0314",
"gpt-4-32k-0613",
"gpt-4-turbo",
"gpt-4-turbo-2024-04-09",
"gpt-4-turbo-preview",
"gpt-4-vision-preview",
"gpt-4.1",
"gpt-4.1-2025-04-14",
"gpt-4.1-mini",
"gpt-4.1-mini-2025-04-14",
"gpt-4.1-nano",
"gpt-4.1-nano-2025-04-14",
"gpt-4o",
"gpt-4o-2024-05-13",
"gpt-4o-2024-08-06",
"gpt-4o-2024-11-20",
"gpt-4o-audio-preview",
"gpt-4o-audio-preview-2024-10-01",
"gpt-4o-audio-preview-2024-12-17",
"gpt-4o-audio-preview-2025-06-03",
"gpt-4o-mini",
"gpt-4o-mini-2024-07-18",
"gpt-4o-mini-audio-preview",
"gpt-4o-mini-audio-preview-2024-12-17",
"gpt-4o-mini-realtime-preview",
"gpt-4o-mini-realtime-preview-2024-12-17",
"gpt-4o-mini-search-preview",
"gpt-4o-mini-search-preview-2025-03-11",
"gpt-4o-mini-transcribe",
"gpt-4o-mini-tts",
"gpt-4o-realtime-preview",
"gpt-4o-realtime-preview-2024-10-01",
"gpt-4o-realtime-preview-2024-12-17",
"gpt-4o-realtime-preview-2025-06-03",
"gpt-4o-search-preview",
"gpt-4o-search-preview-2025-03-11",
"gpt-4o-transcribe",
"gpt-4o-transcribe-diarize",
"gpt-5",
"gpt-5-2025-08-07",
"gpt-5-chat",
"gpt-5-chat-latest",
"gpt-5-codex",
"gpt-5-mini",
"gpt-5-mini-2025-08-07",
"gpt-5-nano",
"gpt-5-nano-2025-08-07",
"gpt-5-pro",
"gpt-5-pro-2025-10-06",
"gpt-5-search-api",
"gpt-5-search-api-2025-10-14",
"gpt-audio",
"gpt-audio-2025-08-28",
"gpt-audio-mini",
"gpt-audio-mini-2025-10-06",
"gpt-image-1",
"gpt-image-1-mini",
"gpt-realtime",
"gpt-realtime-2025-08-28",
"gpt-realtime-mini",
"gpt-realtime-mini-2025-10-06",
"o1",
"o1-preview",
"o1-2024-12-17",
"o1-mini",
"o1-mini-2024-09-12",
"o1-pro",
"o1-pro-2025-03-19",
"o3-mini",
"o3",
"o4-mini",
"whisper-1",
]
AnthropicModels: TypeAlias = Literal[
"claude-3-7-sonnet-latest",
"claude-3-7-sonnet-20250219",
"claude-3-5-haiku-latest",
"claude-3-5-haiku-20241022",
"claude-haiku-4-5",
"claude-haiku-4-5-20251001",
"claude-sonnet-4-20250514",
"claude-sonnet-4-0",
"claude-4-sonnet-20250514",
"claude-sonnet-4-5",
"claude-sonnet-4-5-20250929",
"claude-3-5-sonnet-latest",
"claude-3-5-sonnet-20241022",
"claude-3-5-sonnet-20240620",
"claude-opus-4-0",
"claude-opus-4-20250514",
"claude-4-opus-20250514",
"claude-opus-4-1",
"claude-opus-4-1-20250805",
"claude-3-opus-latest",
"claude-3-opus-20240229",
"claude-3-sonnet-20240229",
"claude-3-haiku-latest",
"claude-3-haiku-20240307",
]
ANTHROPIC_MODELS: list[AnthropicModels] = [
"claude-3-7-sonnet-latest",
"claude-3-7-sonnet-20250219",
"claude-3-5-haiku-latest",
"claude-3-5-haiku-20241022",
"claude-haiku-4-5",
"claude-haiku-4-5-20251001",
"claude-sonnet-4-20250514",
"claude-sonnet-4-0",
"claude-4-sonnet-20250514",
"claude-sonnet-4-5",
"claude-sonnet-4-5-20250929",
"claude-3-5-sonnet-latest",
"claude-3-5-sonnet-20241022",
"claude-3-5-sonnet-20240620",
"claude-opus-4-0",
"claude-opus-4-20250514",
"claude-4-opus-20250514",
"claude-opus-4-1",
"claude-opus-4-1-20250805",
"claude-3-opus-latest",
"claude-3-opus-20240229",
"claude-3-sonnet-20240229",
"claude-3-haiku-latest",
"claude-3-haiku-20240307",
]
GeminiModels: TypeAlias = Literal[
"gemini-2.5-pro",
"gemini-2.5-pro-preview-03-25",
"gemini-2.5-pro-preview-05-06",
"gemini-2.5-pro-preview-06-05",
"gemini-2.5-flash",
"gemini-2.5-flash-preview-05-20",
"gemini-2.5-flash-preview-04-17",
"gemini-2.5-flash-image",
"gemini-2.5-flash-image-preview",
"gemini-2.5-flash-lite",
"gemini-2.5-flash-lite-preview-06-17",
"gemini-2.5-flash-preview-09-2025",
"gemini-2.5-flash-lite-preview-09-2025",
"gemini-2.5-flash-preview-tts",
"gemini-2.5-pro-preview-tts",
"gemini-2.5-computer-use-preview-10-2025",
"gemini-2.0-flash",
"gemini-2.0-flash-001",
"gemini-2.0-flash-exp",
"gemini-2.0-flash-exp-image-generation",
"gemini-2.0-flash-lite",
"gemini-2.0-flash-lite-001",
"gemini-2.0-flash-lite-preview",
"gemini-2.0-flash-lite-preview-02-05",
"gemini-2.0-flash-preview-image-generation",
"gemini-2.0-flash-thinking-exp",
"gemini-2.0-flash-thinking-exp-01-21",
"gemini-2.0-flash-thinking-exp-1219",
"gemini-2.0-pro-exp",
"gemini-2.0-pro-exp-02-05",
"gemini-exp-1206",
"gemini-1.5-pro",
"gemini-1.5-flash",
"gemini-1.5-flash-8b",
"gemini-flash-latest",
"gemini-flash-lite-latest",
"gemini-pro-latest",
"gemini-2.0-flash-live-001",
"gemini-live-2.5-flash-preview",
"gemini-2.5-flash-live-preview",
"gemini-robotics-er-1.5-preview",
"gemini-gemma-2-27b-it",
"gemini-gemma-2-9b-it",
"gemma-3-1b-it",
"gemma-3-4b-it",
"gemma-3-12b-it",
"gemma-3-27b-it",
"gemma-3n-e2b-it",
"gemma-3n-e4b-it",
"learnlm-2.0-flash-experimental",
]
GEMINI_MODELS: list[GeminiModels] = [
"gemini-2.5-pro",
"gemini-2.5-pro-preview-03-25",
"gemini-2.5-pro-preview-05-06",
"gemini-2.5-pro-preview-06-05",
"gemini-2.5-flash",
"gemini-2.5-flash-preview-05-20",
"gemini-2.5-flash-preview-04-17",
"gemini-2.5-flash-image",
"gemini-2.5-flash-image-preview",
"gemini-2.5-flash-lite",
"gemini-2.5-flash-lite-preview-06-17",
"gemini-2.5-flash-preview-09-2025",
"gemini-2.5-flash-lite-preview-09-2025",
"gemini-2.5-flash-preview-tts",
"gemini-2.5-pro-preview-tts",
"gemini-2.5-computer-use-preview-10-2025",
"gemini-2.0-flash",
"gemini-2.0-flash-001",
"gemini-2.0-flash-exp",
"gemini-2.0-flash-exp-image-generation",
"gemini-2.0-flash-lite",
"gemini-2.0-flash-lite-001",
"gemini-2.0-flash-lite-preview",
"gemini-2.0-flash-lite-preview-02-05",
"gemini-2.0-flash-preview-image-generation",
"gemini-2.0-flash-thinking-exp",
"gemini-2.0-flash-thinking-exp-01-21",
"gemini-2.0-flash-thinking-exp-1219",
"gemini-2.0-pro-exp",
"gemini-2.0-pro-exp-02-05",
"gemini-exp-1206",
"gemini-1.5-pro",
"gemini-1.5-flash",
"gemini-1.5-flash-8b",
"gemini-flash-latest",
"gemini-flash-lite-latest",
"gemini-pro-latest",
"gemini-2.0-flash-live-001",
"gemini-live-2.5-flash-preview",
"gemini-2.5-flash-live-preview",
"gemini-robotics-er-1.5-preview",
"gemini-gemma-2-27b-it",
"gemini-gemma-2-9b-it",
"gemma-3-1b-it",
"gemma-3-4b-it",
"gemma-3-12b-it",
"gemma-3-27b-it",
"gemma-3n-e2b-it",
"gemma-3n-e4b-it",
"learnlm-2.0-flash-experimental",
]
AzureModels: TypeAlias = Literal[
"gpt-3.5-turbo",
"gpt-3.5-turbo-0301",
"gpt-3.5-turbo-0613",
"gpt-3.5-turbo-16k",
"gpt-3.5-turbo-16k-0613",
"gpt-35-turbo",
"gpt-35-turbo-0125",
"gpt-35-turbo-1106",
"gpt-35-turbo-16k-0613",
"gpt-35-turbo-instruct-0914",
"gpt-4",
"gpt-4-0314",
"gpt-4-0613",
"gpt-4-1106-preview",
"gpt-4-0125-preview",
"gpt-4-32k",
"gpt-4-32k-0314",
"gpt-4-32k-0613",
"gpt-4-turbo",
"gpt-4-turbo-2024-04-09",
"gpt-4-vision",
"gpt-4o",
"gpt-4o-2024-05-13",
"gpt-4o-2024-08-06",
"gpt-4o-2024-11-20",
"gpt-4o-mini",
"gpt-5",
"o1",
"o1-mini",
"o1-preview",
"o3-mini",
"o3",
"o4-mini",
]
AZURE_MODELS: list[AzureModels] = [
"gpt-3.5-turbo",
"gpt-3.5-turbo-0301",
"gpt-3.5-turbo-0613",
"gpt-3.5-turbo-16k",
"gpt-3.5-turbo-16k-0613",
"gpt-35-turbo",
"gpt-35-turbo-0125",
"gpt-35-turbo-1106",
"gpt-35-turbo-16k-0613",
"gpt-35-turbo-instruct-0914",
"gpt-4",
"gpt-4-0314",
"gpt-4-0613",
"gpt-4-1106-preview",
"gpt-4-0125-preview",
"gpt-4-32k",
"gpt-4-32k-0314",
"gpt-4-32k-0613",
"gpt-4-turbo",
"gpt-4-turbo-2024-04-09",
"gpt-4-vision",
"gpt-4o",
"gpt-4o-2024-05-13",
"gpt-4o-2024-08-06",
"gpt-4o-2024-11-20",
"gpt-4o-mini",
"gpt-5",
"o1",
"o1-mini",
"o1-preview",
"o3-mini",
"o3",
"o4-mini",
]
BedrockModels: TypeAlias = Literal[
"ai21.jamba-1-5-large-v1:0",
"ai21.jamba-1-5-mini-v1:0",
"amazon.nova-lite-v1:0",
"amazon.nova-lite-v1:0:24k",
"amazon.nova-lite-v1:0:300k",
"amazon.nova-micro-v1:0",
"amazon.nova-micro-v1:0:128k",
"amazon.nova-micro-v1:0:24k",
"amazon.nova-premier-v1:0",
"amazon.nova-premier-v1:0:1000k",
"amazon.nova-premier-v1:0:20k",
"amazon.nova-premier-v1:0:8k",
"amazon.nova-premier-v1:0:mm",
"amazon.nova-pro-v1:0",
"amazon.nova-pro-v1:0:24k",
"amazon.nova-pro-v1:0:300k",
"amazon.titan-text-express-v1",
"amazon.titan-text-express-v1:0:8k",
"amazon.titan-text-lite-v1",
"amazon.titan-text-lite-v1:0:4k",
"amazon.titan-tg1-large",
"anthropic.claude-3-5-haiku-20241022-v1:0",
"anthropic.claude-3-5-sonnet-20240620-v1:0",
"anthropic.claude-3-5-sonnet-20241022-v2:0",
"anthropic.claude-3-7-sonnet-20250219-v1:0",
"anthropic.claude-3-haiku-20240307-v1:0",
"anthropic.claude-3-haiku-20240307-v1:0:200k",
"anthropic.claude-3-haiku-20240307-v1:0:48k",
"anthropic.claude-3-opus-20240229-v1:0",
"anthropic.claude-3-opus-20240229-v1:0:12k",
"anthropic.claude-3-opus-20240229-v1:0:200k",
"anthropic.claude-3-opus-20240229-v1:0:28k",
"anthropic.claude-3-sonnet-20240229-v1:0",
"anthropic.claude-3-sonnet-20240229-v1:0:200k",
"anthropic.claude-3-sonnet-20240229-v1:0:28k",
"anthropic.claude-haiku-4-5-20251001-v1:0",
"anthropic.claude-instant-v1:2:100k",
"anthropic.claude-opus-4-1-20250805-v1:0",
"anthropic.claude-opus-4-20250514-v1:0",
"anthropic.claude-sonnet-4-20250514-v1:0",
"anthropic.claude-sonnet-4-5-20250929-v1:0",
"anthropic.claude-v2:0:100k",
"anthropic.claude-v2:0:18k",
"anthropic.claude-v2:1:18k",
"anthropic.claude-v2:1:200k",
"cohere.command-r-plus-v1:0",
"cohere.command-r-v1:0",
"cohere.rerank-v3-5:0",
"deepseek.r1-v1:0",
"meta.llama3-1-70b-instruct-v1:0",
"meta.llama3-1-8b-instruct-v1:0",
"meta.llama3-2-11b-instruct-v1:0",
"meta.llama3-2-1b-instruct-v1:0",
"meta.llama3-2-3b-instruct-v1:0",
"meta.llama3-2-90b-instruct-v1:0",
"meta.llama3-3-70b-instruct-v1:0",
"meta.llama3-70b-instruct-v1:0",
"meta.llama3-8b-instruct-v1:0",
"meta.llama4-maverick-17b-instruct-v1:0",
"meta.llama4-scout-17b-instruct-v1:0",
"mistral.mistral-7b-instruct-v0:2",
"mistral.mistral-large-2402-v1:0",
"mistral.mistral-small-2402-v1:0",
"mistral.mixtral-8x7b-instruct-v0:1",
"mistral.pixtral-large-2502-v1:0",
"openai.gpt-oss-120b-1:0",
"openai.gpt-oss-20b-1:0",
"qwen.qwen3-32b-v1:0",
"qwen.qwen3-coder-30b-a3b-v1:0",
"twelvelabs.pegasus-1-2-v1:0",
]
BEDROCK_MODELS: list[BedrockModels] = [
"ai21.jamba-1-5-large-v1:0",
"ai21.jamba-1-5-mini-v1:0",
"amazon.nova-lite-v1:0",
"amazon.nova-lite-v1:0:24k",
"amazon.nova-lite-v1:0:300k",
"amazon.nova-micro-v1:0",
"amazon.nova-micro-v1:0:128k",
"amazon.nova-micro-v1:0:24k",
"amazon.nova-premier-v1:0",
"amazon.nova-premier-v1:0:1000k",
"amazon.nova-premier-v1:0:20k",
"amazon.nova-premier-v1:0:8k",
"amazon.nova-premier-v1:0:mm",
"amazon.nova-pro-v1:0",
"amazon.nova-pro-v1:0:24k",
"amazon.nova-pro-v1:0:300k",
"amazon.titan-text-express-v1",
"amazon.titan-text-express-v1:0:8k",
"amazon.titan-text-lite-v1",
"amazon.titan-text-lite-v1:0:4k",
"amazon.titan-tg1-large",
"anthropic.claude-3-5-haiku-20241022-v1:0",
"anthropic.claude-3-5-sonnet-20240620-v1:0",
"anthropic.claude-3-5-sonnet-20241022-v2:0",
"anthropic.claude-3-7-sonnet-20250219-v1:0",
"anthropic.claude-3-haiku-20240307-v1:0",
"anthropic.claude-3-haiku-20240307-v1:0:200k",
"anthropic.claude-3-haiku-20240307-v1:0:48k",
"anthropic.claude-3-opus-20240229-v1:0",
"anthropic.claude-3-opus-20240229-v1:0:12k",
"anthropic.claude-3-opus-20240229-v1:0:200k",
"anthropic.claude-3-opus-20240229-v1:0:28k",
"anthropic.claude-3-sonnet-20240229-v1:0",
"anthropic.claude-3-sonnet-20240229-v1:0:200k",
"anthropic.claude-3-sonnet-20240229-v1:0:28k",
"anthropic.claude-haiku-4-5-20251001-v1:0",
"anthropic.claude-instant-v1:2:100k",
"anthropic.claude-opus-4-1-20250805-v1:0",
"anthropic.claude-opus-4-20250514-v1:0",
"anthropic.claude-sonnet-4-20250514-v1:0",
"anthropic.claude-sonnet-4-5-20250929-v1:0",
"anthropic.claude-v2:0:100k",
"anthropic.claude-v2:0:18k",
"anthropic.claude-v2:1:18k",
"anthropic.claude-v2:1:200k",
"cohere.command-r-plus-v1:0",
"cohere.command-r-v1:0",
"cohere.rerank-v3-5:0",
"deepseek.r1-v1:0",
"meta.llama3-1-70b-instruct-v1:0",
"meta.llama3-1-8b-instruct-v1:0",
"meta.llama3-2-11b-instruct-v1:0",
"meta.llama3-2-1b-instruct-v1:0",
"meta.llama3-2-3b-instruct-v1:0",
"meta.llama3-2-90b-instruct-v1:0",
"meta.llama3-3-70b-instruct-v1:0",
"meta.llama3-70b-instruct-v1:0",
"meta.llama3-8b-instruct-v1:0",
"meta.llama4-maverick-17b-instruct-v1:0",
"meta.llama4-scout-17b-instruct-v1:0",
"mistral.mistral-7b-instruct-v0:2",
"mistral.mistral-large-2402-v1:0",
"mistral.mistral-small-2402-v1:0",
"mistral.mixtral-8x7b-instruct-v0:1",
"mistral.pixtral-large-2502-v1:0",
"openai.gpt-oss-120b-1:0",
"openai.gpt-oss-20b-1:0",
"qwen.qwen3-32b-v1:0",
"qwen.qwen3-coder-30b-a3b-v1:0",
"twelvelabs.pegasus-1-2-v1:0",
]
SupportedModels: TypeAlias = (
OpenAIModels | AnthropicModels | GeminiModels | AzureModels | BedrockModels
)

View File

@@ -20,9 +20,7 @@ from typing import (
)
from dotenv import load_dotenv
import httpx
from pydantic import BaseModel, Field
from typing_extensions import Self
from crewai.events.event_bus import crewai_event_bus
from crewai.events.types.llm_events import (
@@ -37,14 +35,7 @@ from crewai.events.types.tool_usage_events import (
ToolUsageFinishedEvent,
ToolUsageStartedEvent,
)
from crewai.llms.base_llm import BaseLLM
from crewai.llms.constants import (
ANTHROPIC_MODELS,
AZURE_MODELS,
BEDROCK_MODELS,
GEMINI_MODELS,
OPENAI_MODELS,
)
from crewai.llm.base_llm import BaseLLM
from crewai.utilities import InternalInstructor
from crewai.utilities.exceptions.context_window_exceeding_exception import (
LLMContextLengthExceededError,
@@ -61,7 +52,6 @@ if TYPE_CHECKING:
from litellm.utils import supports_response_schema
from crewai.agent.core import Agent
from crewai.llms.hooks.base import BaseInterceptor
from crewai.task import Task
from crewai.tools.base_tool import BaseTool
from crewai.utilities.types import LLMMessage
@@ -327,249 +317,57 @@ class AccumulatedToolArgs(BaseModel):
class LLM(BaseLLM):
completion_cost: float | None = None
"""LiteLLM-based LLM implementation for CrewAI.
def __new__(cls, model: str, is_litellm: bool = False, **kwargs: Any) -> LLM:
"""Factory method that routes to native SDK or falls back to LiteLLM.
This class provides LiteLLM integration for models not covered by native providers.
The metaclass (LLMMeta) automatically routes to native providers when appropriate.
"""
Routing priority:
1. If 'provider' kwarg is present, use that provider with constants
2. If only 'model' kwarg, use constants to infer provider
3. If "/" in model name:
- Check if prefix is a native provider (openai/anthropic/azure/bedrock/gemini)
- If yes, validate model against constants
- If valid, route to native SDK; otherwise route to LiteLLM
"""
if not model or not isinstance(model, str):
raise ValueError("Model must be a non-empty string")
# LiteLLM-specific fields
completion_cost: float | None = Field(None, description="Cost of completion")
timeout: float | int | None = Field(None, description="Request timeout")
top_p: float | None = Field(None, description="Top-p sampling parameter")
n: int | None = Field(None, description="Number of completions to generate")
max_completion_tokens: int | None = Field(
None, description="Maximum completion tokens"
)
max_tokens: int | float | None = Field(None, description="Maximum total tokens")
presence_penalty: float | None = Field(None, description="Presence penalty")
frequency_penalty: float | None = Field(None, description="Frequency penalty")
logit_bias: dict[int, float] | None = Field(None, description="Logit bias")
response_format: type[BaseModel] | None = Field(
None, description="Response format model"
)
seed: int | None = Field(None, description="Random seed for reproducibility")
logprobs: int | None = Field(None, description="Log probabilities to return")
top_logprobs: int | None = Field(None, description="Top log probabilities")
api_base: str | None = Field(None, description="API base URL (alias for base_url)")
api_version: str | None = Field(None, description="API version")
callbacks: list[Any] | None = Field(None, description="Callback functions")
context_window_size: int = Field(0, description="Context window size in tokens")
reasoning_effort: Literal["none", "low", "medium", "high"] | None = Field(
None, description="Reasoning effort level"
)
is_anthropic: bool = Field(False, description="Whether model is from Anthropic")
stream: bool = Field(False, description="Whether to stream responses")
explicit_provider = kwargs.get("provider")
if explicit_provider:
provider = explicit_provider
use_native = True
model_string = model
elif "/" in model:
prefix, _, model_part = model.partition("/")
provider_mapping = {
"openai": "openai",
"anthropic": "anthropic",
"claude": "anthropic",
"azure": "azure",
"azure_openai": "azure",
"google": "gemini",
"gemini": "gemini",
"bedrock": "bedrock",
"aws": "bedrock",
}
canonical_provider = provider_mapping.get(prefix.lower())
if canonical_provider and cls._validate_model_in_constants(
model_part, canonical_provider
):
provider = canonical_provider
use_native = True
model_string = model_part
else:
provider = prefix
use_native = False
model_string = model_part
else:
provider = cls._infer_provider_from_model(model)
use_native = True
model_string = model
native_class = cls._get_native_provider(provider) if use_native else None
if native_class and not is_litellm and provider in SUPPORTED_NATIVE_PROVIDERS:
try:
# Remove 'provider' from kwargs if it exists to avoid duplicate keyword argument
kwargs_copy = {k: v for k, v in kwargs.items() if k != 'provider'}
return cast(
Self, native_class(model=model_string, provider=provider, **kwargs_copy)
)
except NotImplementedError:
raise
except Exception as e:
raise ImportError(f"Error importing native provider: {e}") from e
# FALLBACK to LiteLLM
if not LITELLM_AVAILABLE:
logger.error("LiteLLM is not available, falling back to LiteLLM")
raise ImportError("Fallback to LiteLLM is not available") from None
instance = object.__new__(cls)
super(LLM, instance).__init__(model=model, is_litellm=True, **kwargs)
instance.is_litellm = True
return instance
@classmethod
def _validate_model_in_constants(cls, model: str, provider: str) -> bool:
"""Validate if a model name exists in the provider's constants.
def model_post_init(self, __context: Any) -> None:
"""Initialize LiteLLM-specific settings after model initialization.
Args:
model: The model name to validate
provider: The provider to check against (canonical name)
Returns:
True if the model exists in the provider's constants, False otherwise
__context: Pydantic context
"""
if provider == "openai":
return model in OPENAI_MODELS
super().model_post_init(__context)
if provider == "anthropic" or provider == "claude":
return model in ANTHROPIC_MODELS
# Configure LiteLLM
if LITELLM_AVAILABLE:
litellm.drop_params = True
if provider == "gemini":
return model in GEMINI_MODELS
# Determine if this is an Anthropic model
self.is_anthropic = self._is_anthropic_model(self.model)
if provider == "bedrock":
return model in BEDROCK_MODELS
if provider == "azure":
# azure does not provide a list of available models, determine a better way to handle this
return True
return False
@classmethod
def _infer_provider_from_model(cls, model: str) -> str:
"""Infer the provider from the model name.
Args:
model: The model name without provider prefix
Returns:
The inferred provider name, defaults to "openai"
"""
if model in OPENAI_MODELS:
return "openai"
if model in ANTHROPIC_MODELS:
return "anthropic"
if model in GEMINI_MODELS:
return "gemini"
if model in BEDROCK_MODELS:
return "bedrock"
if model in AZURE_MODELS:
return "azure"
return "openai"
@classmethod
def _get_native_provider(cls, provider: str) -> type | None:
"""Get native provider class if available."""
if provider == "openai":
from crewai.llms.providers.openai.completion import OpenAICompletion
return OpenAICompletion
if provider == "anthropic" or provider == "claude":
from crewai.llms.providers.anthropic.completion import (
AnthropicCompletion,
)
return AnthropicCompletion
if provider == "azure" or provider == "azure_openai":
from crewai.llms.providers.azure.completion import AzureCompletion
return AzureCompletion
if provider == "google" or provider == "gemini":
from crewai.llms.providers.gemini.completion import GeminiCompletion
return GeminiCompletion
if provider == "bedrock":
from crewai.llms.providers.bedrock.completion import BedrockCompletion
return BedrockCompletion
return None
def __init__(
self,
model: str,
timeout: float | int | None = None,
temperature: float | None = None,
top_p: float | None = None,
n: int | None = None,
stop: str | list[str] | None = None,
max_completion_tokens: int | None = None,
max_tokens: int | float | None = None,
presence_penalty: float | None = None,
frequency_penalty: float | None = None,
logit_bias: dict[int, float] | None = None,
response_format: type[BaseModel] | None = None,
seed: int | None = None,
logprobs: int | None = None,
top_logprobs: int | None = None,
base_url: str | None = None,
api_base: str | None = None,
api_version: str | None = None,
api_key: str | None = None,
callbacks: list[Any] | None = None,
reasoning_effort: Literal["none", "low", "medium", "high"] | None = None,
stream: bool = False,
interceptor: BaseInterceptor[httpx.Request, httpx.Response] | None = None,
**kwargs: Any,
) -> None:
"""Initialize LLM instance.
Note: This __init__ method is only called for fallback instances.
Native provider instances handle their own initialization in their respective classes.
"""
super().__init__(
model=model,
temperature=temperature,
api_key=api_key,
base_url=base_url,
timeout=timeout,
**kwargs,
)
self.model = model
self.timeout = timeout
self.temperature = temperature
self.top_p = top_p
self.n = n
self.max_completion_tokens = max_completion_tokens
self.max_tokens = max_tokens
self.presence_penalty = presence_penalty
self.frequency_penalty = frequency_penalty
self.logit_bias = logit_bias
self.response_format = response_format
self.seed = seed
self.logprobs = logprobs
self.top_logprobs = top_logprobs
self.base_url = base_url
self.api_base = api_base
self.api_version = api_version
self.api_key = api_key
self.callbacks = callbacks
self.context_window_size = 0
self.reasoning_effort = reasoning_effort
self.additional_params = kwargs
self.is_anthropic = self._is_anthropic_model(model)
self.stream = stream
self.interceptor = interceptor
litellm.drop_params = True
# Normalize self.stop to always be a list[str]
if stop is None:
self.stop: list[str] = []
elif isinstance(stop, str):
self.stop = [stop]
else:
self.stop = stop
self.set_callbacks(callbacks or [])
# Set up callbacks
self.set_callbacks(self.callbacks or [])
self.set_env_callbacks()
@staticmethod
@@ -1649,7 +1447,7 @@ class LLM(BaseLLM):
**filtered_params,
)
def __deepcopy__(self, memo: dict[int, Any] | None) -> LLM:
def __deepcopy__(self, memo: dict[int, Any] | None) -> LLM: # type: ignore[override]
"""Create a deep copy of the LLM instance."""
import copy

View File

@@ -0,0 +1,6 @@
"""Interceptor contracts for crewai"""
from crewai.llm.hooks.base import BaseInterceptor
__all__ = ["BaseInterceptor"]

View File

@@ -0,0 +1,133 @@
"""Base classes for LLM transport interceptors.
This module provides abstract base classes for intercepting and modifying
outbound and inbound messages at the transport level.
"""
from __future__ import annotations
from abc import ABC, abstractmethod
from typing import TYPE_CHECKING, Any, Generic, TypeVar
from pydantic_core import core_schema
if TYPE_CHECKING:
from pydantic import GetCoreSchemaHandler
from pydantic_core import CoreSchema
T = TypeVar("T")
U = TypeVar("U")
class BaseInterceptor(ABC, Generic[T, U]):
"""Abstract base class for intercepting transport-level messages.
Provides hooks to intercept and modify outbound and inbound messages
at the transport layer.
Type parameters:
T: Outbound message type (e.g., httpx.Request)
U: Inbound message type (e.g., httpx.Response)
Example:
>>> import httpx
>>> class CustomInterceptor(BaseInterceptor[httpx.Request, httpx.Response]):
... def on_outbound(self, message: httpx.Request) -> httpx.Request:
... message.headers["X-Custom-Header"] = "value"
... return message
...
... def on_inbound(self, message: httpx.Response) -> httpx.Response:
... print(f"Status: {message.status_code}")
... return message
"""
@abstractmethod
def on_outbound(self, message: T) -> T:
"""Intercept outbound message before sending.
Args:
message: Outbound message object.
Returns:
Modified message object.
"""
...
@abstractmethod
def on_inbound(self, message: U) -> U:
"""Intercept inbound message after receiving.
Args:
message: Inbound message object.
Returns:
Modified message object.
"""
...
async def aon_outbound(self, message: T) -> T:
"""Async version of on_outbound.
Args:
message: Outbound message object.
Returns:
Modified message object.
"""
raise NotImplementedError
async def aon_inbound(self, message: U) -> U:
"""Async version of on_inbound.
Args:
message: Inbound message object.
Returns:
Modified message object.
"""
raise NotImplementedError
@classmethod
def __get_pydantic_core_schema__(
cls, _source_type: Any, _handler: GetCoreSchemaHandler
) -> CoreSchema:
"""Generate Pydantic core schema for BaseInterceptor.
This allows the generic BaseInterceptor to be used in Pydantic models
without requiring arbitrary_types_allowed=True. The schema validates
that the value is an instance of BaseInterceptor.
Args:
_source_type: The source type being validated (unused).
_handler: Handler for generating schemas (unused).
Returns:
A Pydantic core schema that validates BaseInterceptor instances.
"""
return core_schema.no_info_plain_validator_function(
_validate_interceptor,
serialization=core_schema.plain_serializer_function_ser_schema(
lambda x: x, return_schema=core_schema.any_schema()
),
)
def _validate_interceptor(value: Any) -> BaseInterceptor[T, U]:
"""Validate that the value is a BaseInterceptor instance.
Args:
value: The value to validate.
Returns:
The validated BaseInterceptor instance.
Raises:
ValueError: If the value is not a BaseInterceptor instance.
"""
if not isinstance(value, BaseInterceptor):
raise ValueError(
f"Expected BaseInterceptor instance, got {type(value).__name__}"
)
return value

View File

@@ -0,0 +1,123 @@
"""HTTP transport implementations for LLM request/response interception.
This module provides internal transport classes that integrate with BaseInterceptor
to enable request/response modification at the transport level.
"""
from __future__ import annotations
from collections.abc import Iterable
from typing import TYPE_CHECKING, TypedDict
from httpx import (
AsyncHTTPTransport as _AsyncHTTPTransport,
HTTPTransport as _HTTPTransport,
)
from typing_extensions import NotRequired, Unpack
if TYPE_CHECKING:
from ssl import SSLContext
from httpx import Limits, Request, Response
from httpx._types import CertTypes, ProxyTypes
from crewai.llm.hooks.base import BaseInterceptor
class HTTPTransportKwargs(TypedDict, total=False):
"""Typed dictionary for httpx.HTTPTransport initialization parameters.
These parameters configure the underlying HTTP transport behavior including
SSL verification, proxies, connection limits, and low-level socket options.
"""
verify: bool | str | SSLContext
cert: NotRequired[CertTypes]
trust_env: bool
http1: bool
http2: bool
limits: Limits
proxy: NotRequired[ProxyTypes]
uds: NotRequired[str]
local_address: NotRequired[str]
retries: int
socket_options: NotRequired[
Iterable[
tuple[int, int, int]
| tuple[int, int, bytes | bytearray]
| tuple[int, int, None, int]
]
]
class HTTPTransport(_HTTPTransport):
"""HTTP transport that uses an interceptor for request/response modification.
This transport is used internally when a user provides a BaseInterceptor.
Users should not instantiate this class directly - instead, pass an interceptor
to the LLM client and this transport will be created automatically.
"""
def __init__(
self,
interceptor: BaseInterceptor[Request, Response],
**kwargs: Unpack[HTTPTransportKwargs],
) -> None:
"""Initialize transport with interceptor.
Args:
interceptor: HTTP interceptor for modifying raw request/response objects.
**kwargs: HTTPTransport configuration parameters (verify, cert, proxy, etc.).
"""
super().__init__(**kwargs)
self.interceptor = interceptor
def handle_request(self, request: Request) -> Response:
"""Handle request with interception.
Args:
request: The HTTP request to handle.
Returns:
The HTTP response.
"""
request = self.interceptor.on_outbound(request)
response = super().handle_request(request)
return self.interceptor.on_inbound(response)
class AsyncHTTPTransport(_AsyncHTTPTransport):
"""Async HTTP transport that uses an interceptor for request/response modification.
This transport is used internally when a user provides a BaseInterceptor.
Users should not instantiate this class directly - instead, pass an interceptor
to the LLM client and this transport will be created automatically.
"""
def __init__(
self,
interceptor: BaseInterceptor[Request, Response],
**kwargs: Unpack[HTTPTransportKwargs],
) -> None:
"""Initialize async transport with interceptor.
Args:
interceptor: HTTP interceptor for modifying raw request/response objects.
**kwargs: HTTPTransport configuration parameters (verify, cert, proxy, etc.).
"""
super().__init__(**kwargs)
self.interceptor = interceptor
async def handle_async_request(self, request: Request) -> Response:
"""Handle async request with interception.
Args:
request: The HTTP request to handle.
Returns:
The HTTP response.
"""
request = await self.interceptor.aon_outbound(request)
response = await super().handle_async_request(request)
return await self.interceptor.aon_inbound(response)

View File

@@ -0,0 +1,14 @@
from crewai.llm.constants import SupportedNativeProviders
PROVIDER_MAPPING: dict[str, SupportedNativeProviders] = {
"openai": "openai",
"anthropic": "anthropic",
"claude": "anthropic",
"azure": "azure",
"azure_openai": "azure",
"google": "gemini",
"gemini": "gemini",
"bedrock": "bedrock",
"aws": "bedrock",
}

View File

@@ -0,0 +1,251 @@
"""Metaclass for LLM provider routing.
This metaclass enables automatic routing to native provider implementations
based on the model parameter at instantiation time.
"""
from __future__ import annotations
import logging
from typing import Any, cast
from pydantic import ConfigDict
from pydantic._internal._model_construction import ModelMetaclass
from crewai.llm.constants import (
ANTHROPIC_MODELS,
AZURE_MODELS,
BEDROCK_MODELS,
GEMINI_MODELS,
OPENAI_MODELS,
SUPPORTED_NATIVE_PROVIDERS,
SupportedModels,
SupportedNativeProviders,
)
from crewai.llm.internal.constants import PROVIDER_MAPPING
class LLMMeta(ModelMetaclass):
"""Metaclass for LLM that handles provider routing.
This metaclass intercepts LLM instantiation and routes to the appropriate
native provider implementation based on the model parameter.
"""
def __new__(
mcs,
name: str,
bases: tuple[type, ...],
namespace: dict[str, Any],
**kwargs: Any,
) -> type:
"""Create new LLM class with proper model_config for custom LLMs.
Args:
name: Class name
bases: Base classes
namespace: Class namespace
**kwargs: Additional arguments
Returns:
New class
"""
if name != "BaseLLM" and any(
base.__name__ in ("BaseLLM", "LLM") for base in bases
):
if "model_config" not in namespace:
namespace["model_config"] = ConfigDict(
extra="allow", populate_by_name=True
)
elif isinstance(namespace["model_config"], dict):
config_dict = cast(
ConfigDict, cast(object, dict(namespace["model_config"]))
)
config_dict.setdefault("extra", "allow")
config_dict.setdefault("populate_by_name", True)
namespace["model_config"] = ConfigDict(**config_dict)
return super().__new__(mcs, name, bases, namespace)
def __call__(cls, *args: Any, **kwargs: Any) -> Any: # noqa: N805
"""Route to appropriate provider implementation at instantiation time.
Args:
*args: Positional arguments (model should be first for LLM class)
**kwargs: Keyword arguments including model, is_litellm, etc.
Returns:
Instance of the appropriate provider class or LLM class
Raises:
ValueError: If model is not a valid string
"""
if cls.__name__ != "LLM":
return super().__call__(*args, **kwargs)
model = cast(
str | SupportedModels | None,
(kwargs.get("model") or (args[0] if args else None)),
)
is_litellm = kwargs.get("is_litellm", False)
if not model or not isinstance(model, str):
raise ValueError("Model must be a non-empty string")
if args and not kwargs.get("model"):
kwargs["model"] = cast(SupportedModels, args[0])
_ = args[1:]
explicit_provider = cast(SupportedNativeProviders, kwargs.get("provider"))
if explicit_provider:
provider = explicit_provider
use_native = True
model_string = model
elif "/" in model:
prefix, _, model_part = cast(
tuple[SupportedNativeProviders, Any, SupportedModels],
model.partition("/"),
)
canonical_provider = PROVIDER_MAPPING.get(prefix.lower())
if canonical_provider and cls._validate_model_in_constants(
model_part, canonical_provider
):
provider = canonical_provider
use_native = True
model_string = model_part
else:
provider = prefix
use_native = False
model_string = model_part
else:
provider = cls._infer_provider_from_model(model)
use_native = True
model_string = model
native_class = cls._get_native_provider(provider) if use_native else None
if native_class and not is_litellm and provider in SUPPORTED_NATIVE_PROVIDERS:
try:
kwargs_copy = {
k: v for k, v in kwargs.items() if k not in ("provider", "model")
}
return native_class(
model=model_string, provider=provider, **kwargs_copy
)
except NotImplementedError:
raise
except Exception as e:
raise ImportError(f"Error importing native provider: {e}") from e
try:
import litellm # noqa: F401
except ImportError:
logging.error("LiteLLM is not available, falling back to LiteLLM")
raise ImportError("Fallback to LiteLLM is not available") from None
kwargs_copy = {
k: v for k, v in kwargs.items() if k not in ("model", "is_litellm")
}
return super().__call__(model=model, is_litellm=True, **kwargs_copy)
@staticmethod
def _validate_model_in_constants(
model: SupportedModels, provider: SupportedNativeProviders | None
) -> bool:
"""Validate if a model name exists in the provider's constants.
Args:
model: The model name to validate
provider: The provider to check against (canonical name)
Returns:
True if the model exists in the provider's constants, False otherwise
"""
if provider == "openai":
return model in OPENAI_MODELS
if provider == "anthropic" or provider == "claude":
return model in ANTHROPIC_MODELS
if provider == "gemini":
return model in GEMINI_MODELS
if provider == "bedrock":
return model in BEDROCK_MODELS
if provider == "azure":
# azure does not provide a list of available models
return True
return False
@staticmethod
def _infer_provider_from_model(
model: SupportedModels | str,
) -> SupportedNativeProviders:
"""Infer the provider from the model name.
Args:
model: The model name without provider prefix
Returns:
The inferred provider name, defaults to "openai"
"""
if model in OPENAI_MODELS:
return "openai"
if model in ANTHROPIC_MODELS:
return "anthropic"
if model in GEMINI_MODELS:
return "gemini"
if model in BEDROCK_MODELS:
return "bedrock"
if model in AZURE_MODELS:
return "azure"
return "openai"
@staticmethod
def _get_native_provider(provider: SupportedNativeProviders | None) -> type | None:
"""Get native provider class if available.
Args:
provider: The provider name
Returns:
The provider class or None if not available
"""
if provider == "openai":
from crewai.llm.providers.openai.completion import OpenAICompletion
return OpenAICompletion
if provider == "anthropic" or provider == "claude":
from crewai.llm.providers.anthropic.completion import (
AnthropicCompletion,
)
return AnthropicCompletion
if provider == "azure" or provider == "azure_openai":
from crewai.llm.providers.azure.completion import AzureCompletion
return AzureCompletion
if provider == "google" or provider == "gemini":
from crewai.llm.providers.gemini.completion import GeminiCompletion
return GeminiCompletion
if provider == "bedrock":
from crewai.llm.providers.bedrock.completion import BedrockCompletion
return BedrockCompletion
return None

View File

@@ -2,14 +2,18 @@ from __future__ import annotations
import json
import logging
import os
from typing import TYPE_CHECKING, Any, cast
from pydantic import BaseModel
from dotenv import load_dotenv
import httpx
from pydantic import BaseModel, Field, PrivateAttr, model_validator
from typing_extensions import Self
from crewai.events.types.llm_events import LLMCallType
from crewai.llms.base_llm import BaseLLM
from crewai.llms.hooks.transport import HTTPTransport
from crewai.llm.base_llm import BaseLLM
from crewai.llm.core import CONTEXT_WINDOW_USAGE_RATIO
from crewai.llm.hooks.transport import HTTPTransport
from crewai.llm.providers.utils.common import safe_tool_conversion
from crewai.utilities.agent_utils import is_context_length_exceeded
from crewai.utilities.exceptions.context_window_exceeding_exception import (
LLMContextLengthExceededError,
@@ -18,114 +22,85 @@ from crewai.utilities.types import LLMMessage
if TYPE_CHECKING:
from crewai.llms.hooks.base import BaseInterceptor
from anthropic.types import Message
from crewai.agent.core import Agent
from crewai.task import Task
try:
from anthropic import Anthropic
from anthropic.types import Message
from anthropic.types.tool_use_block import ToolUseBlock
import httpx
except ImportError:
raise ImportError(
'Anthropic native provider not available, to install: uv add "crewai[anthropic]"'
) from None
load_dotenv()
class AnthropicCompletion(BaseLLM):
"""Anthropic native completion implementation.
This class provides direct integration with the Anthropic Python SDK,
offering native tool use, streaming support, and proper message formatting.
Attributes:
model: Anthropic model name (e.g., 'claude-3-5-sonnet-20241022')
base_url: Custom base URL for Anthropic API
timeout: Request timeout in seconds
max_retries: Maximum number of retries
max_tokens: Maximum tokens in response (required for Anthropic)
top_p: Nucleus sampling parameter
stream: Enable streaming responses
client_params: Additional parameters for the Anthropic client
interceptor: HTTP interceptor for modifying requests/responses at transport level
"""
def __init__(
self,
model: str = "claude-3-5-sonnet-20241022",
api_key: str | None = None,
base_url: str | None = None,
timeout: float | None = None,
max_retries: int = 2,
temperature: float | None = None,
max_tokens: int = 4096, # Required for Anthropic
top_p: float | None = None,
stop_sequences: list[str] | None = None,
stream: bool = False,
client_params: dict[str, Any] | None = None,
interceptor: BaseInterceptor[httpx.Request, httpx.Response] | None = None,
**kwargs: Any,
):
"""Initialize Anthropic chat completion client.
base_url: str | None = Field(
default=None, description="Custom base URL for Anthropic API"
)
timeout: float | None = Field(
default=None, description="Request timeout in seconds"
)
max_retries: int = Field(default=2, description="Maximum number of retries")
max_tokens: int = Field(
default=4096, description="Maximum tokens in response (required for Anthropic)"
)
top_p: float | None = Field(default=None, description="Nucleus sampling parameter")
stream: bool = Field(default=False, description="Enable streaming responses")
client_params: dict[str, Any] | None = Field(
default_factory=dict, description="Additional Anthropic client parameters"
)
_client: Anthropic = PrivateAttr(default=None) # type: ignore[assignment]
Args:
model: Anthropic model name (e.g., 'claude-3-5-sonnet-20241022')
api_key: Anthropic API key (defaults to ANTHROPIC_API_KEY env var)
base_url: Custom base URL for Anthropic API
timeout: Request timeout in seconds
max_retries: Maximum number of retries
temperature: Sampling temperature (0-1)
max_tokens: Maximum tokens in response (required for Anthropic)
top_p: Nucleus sampling parameter
stop_sequences: Stop sequences (Anthropic uses stop_sequences, not stop)
stream: Enable streaming responses
client_params: Additional parameters for the Anthropic client
interceptor: HTTP interceptor for modifying requests/responses at transport level.
**kwargs: Additional parameters
"""
super().__init__(
model=model, temperature=temperature, stop=stop_sequences or [], **kwargs
)
_is_claude_3: bool = PrivateAttr(default=False)
_supports_tools: bool = PrivateAttr(default=False)
# Client params
self.interceptor = interceptor
self.client_params = client_params
self.base_url = base_url
self.timeout = timeout
self.max_retries = max_retries
@model_validator(mode="after")
def setup_client(self) -> Self:
"""Initialize the Anthropic client and model-specific settings."""
self._client = Anthropic(**self._get_client_params())
self.client = Anthropic(**self._get_client_params())
self._is_claude_3 = "claude-3" in self.model.lower()
self._supports_tools = self._is_claude_3
# Store completion parameters
self.max_tokens = max_tokens
self.top_p = top_p
self.stream = stream
self.stop_sequences = stop_sequences or []
# Model-specific settings
self.is_claude_3 = "claude-3" in model.lower()
self.supports_tools = self.is_claude_3 # Claude 3+ supports tool use
return self
@property
def stop(self) -> list[str]:
"""Get stop sequences sent to the API."""
return self.stop_sequences
def is_claude_3(self) -> bool:
"""Check if model is Claude 3."""
return self._is_claude_3
@stop.setter
def stop(self, value: list[str] | str | None) -> None:
"""Set stop sequences.
Synchronizes stop_sequences to ensure values set by CrewAgentExecutor
are properly sent to the Anthropic API.
Args:
value: Stop sequences as a list, single string, or None
"""
if value is None:
self.stop_sequences = []
elif isinstance(value, str):
self.stop_sequences = [value]
elif isinstance(value, list):
self.stop_sequences = value
else:
self.stop_sequences = []
@property
def supports_tools(self) -> bool:
"""Check if model supports tools."""
return self._supports_tools
def _get_client_params(self) -> dict[str, Any]:
"""Get client parameters."""
if self.api_key is None:
self.api_key = os.getenv("ANTHROPIC_API_KEY")
if self.api_key is None:
raise ValueError("ANTHROPIC_API_KEY is required")
client_params = {
"api_key": self.api_key,
"base_url": self.base_url,
@@ -149,8 +124,8 @@ class AnthropicCompletion(BaseLLM):
tools: list[dict[str, Any]] | None = None,
callbacks: list[Any] | None = None,
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
response_model: type[BaseModel] | None = None,
) -> str | Any:
"""Call Anthropic messages API.
@@ -245,8 +220,8 @@ class AnthropicCompletion(BaseLLM):
params["temperature"] = self.temperature
if self.top_p is not None:
params["top_p"] = self.top_p
if self.stop_sequences:
params["stop_sequences"] = self.stop_sequences
if self.stop:
params["stop_sequences"] = self.stop
# Handle tools for Claude 3+
if tools and self.supports_tools:
@@ -266,8 +241,6 @@ class AnthropicCompletion(BaseLLM):
continue
try:
from crewai.llms.providers.utils.common import safe_tool_conversion
name, description, parameters = safe_tool_conversion(tool, "Anthropic")
except (ImportError, KeyError, ValueError) as e:
logging.error(f"Error converting tool to Anthropic format: {e}")
@@ -341,8 +314,8 @@ class AnthropicCompletion(BaseLLM):
self,
params: dict[str, Any],
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
response_model: type[BaseModel] | None = None,
) -> str | Any:
"""Handle non-streaming message completion."""
@@ -357,7 +330,7 @@ class AnthropicCompletion(BaseLLM):
params["tool_choice"] = {"type": "tool", "name": "structured_output"}
try:
response: Message = self.client.messages.create(**params)
response: Message = self._client.messages.create(**params)
except Exception as e:
if is_context_length_exceeded(e):
@@ -429,8 +402,8 @@ class AnthropicCompletion(BaseLLM):
self,
params: dict[str, Any],
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
response_model: type[BaseModel] | None = None,
) -> str:
"""Handle streaming message completion."""
@@ -451,7 +424,7 @@ class AnthropicCompletion(BaseLLM):
stream_params = {k: v for k, v in params.items() if k != "stream"}
# Make streaming API call
with self.client.messages.stream(**stream_params) as stream:
with self._client.messages.stream(**stream_params) as stream:
for event in stream:
if hasattr(event, "delta") and hasattr(event.delta, "text"):
text_delta = event.delta.text
@@ -525,8 +498,8 @@ class AnthropicCompletion(BaseLLM):
tool_uses: list[ToolUseBlock],
params: dict[str, Any],
available_functions: dict[str, Any],
from_task: Any | None = None,
from_agent: Any | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
) -> str:
"""Handle the complete tool use conversation flow.
@@ -579,7 +552,7 @@ class AnthropicCompletion(BaseLLM):
try:
# Send tool results back to Claude for final response
final_response: Message = self.client.messages.create(**follow_up_params)
final_response: Message = self._client.messages.create(**follow_up_params)
# Track token usage for follow-up call
follow_up_usage = self._extract_anthropic_token_usage(final_response)
@@ -636,7 +609,6 @@ class AnthropicCompletion(BaseLLM):
def get_context_window_size(self) -> int:
"""Get the context window size for the model."""
from crewai.llm import CONTEXT_WINDOW_USAGE_RATIO
# Context window sizes for Anthropic models
context_windows = {

View File

@@ -5,8 +5,12 @@ import logging
import os
from typing import TYPE_CHECKING, Any
from pydantic import BaseModel
from dotenv import load_dotenv
from pydantic import BaseModel, Field, PrivateAttr, model_validator
from typing_extensions import Self
from crewai.llm.core import CONTEXT_WINDOW_USAGE_RATIO, LLM_CONTEXT_WINDOW_SIZES
from crewai.llm.providers.utils.common import safe_tool_conversion
from crewai.utilities.agent_utils import is_context_length_exceeded
from crewai.utilities.exceptions.context_window_exceeding_exception import (
LLMContextLengthExceededError,
@@ -15,7 +19,8 @@ from crewai.utilities.types import LLMMessage
if TYPE_CHECKING:
from crewai.llms.hooks.base import BaseInterceptor
from crewai.agent.core import Agent
from crewai.task import Task
from crewai.tools.base_tool import BaseTool
@@ -36,7 +41,7 @@ try:
)
from crewai.events.types.llm_events import LLMCallType
from crewai.llms.base_llm import BaseLLM
from crewai.llm.base_llm import BaseLLM
except ImportError:
raise ImportError(
@@ -44,111 +49,109 @@ except ImportError:
) from None
load_dotenv()
class AzureCompletion(BaseLLM):
"""Azure AI Inference native completion implementation.
This class provides direct integration with the Azure AI Inference Python SDK,
offering native function calling, streaming support, and proper Azure authentication.
Attributes:
model: Azure deployment name or model name
endpoint: Azure endpoint URL
api_version: Azure API version
timeout: Request timeout in seconds
max_retries: Maximum number of retries
top_p: Nucleus sampling parameter
frequency_penalty: Frequency penalty (-2 to 2)
presence_penalty: Presence penalty (-2 to 2)
max_tokens: Maximum tokens in response
stream: Enable streaming responses
interceptor: HTTP interceptor (not yet supported for Azure)
"""
def __init__(
self,
model: str,
api_key: str | None = None,
endpoint: str | None = None,
api_version: str | None = None,
timeout: float | None = None,
max_retries: int = 2,
temperature: float | None = None,
top_p: float | None = None,
frequency_penalty: float | None = None,
presence_penalty: float | None = None,
max_tokens: int | None = None,
stop: list[str] | None = None,
stream: bool = False,
interceptor: BaseInterceptor[Any, Any] | None = None,
**kwargs: Any,
):
"""Initialize Azure AI Inference chat completion client.
endpoint: str = Field( # type: ignore[assignment]
default_factory=lambda: os.getenv("AZURE_ENDPOINT")
or os.getenv("AZURE_OPENAI_ENDPOINT")
or os.getenv("AZURE_API_BASE"),
description="Azure endpoint URL (defaults to AZURE_ENDPOINT env var)",
)
api_version: str = Field(
default_factory=lambda: os.getenv("AZURE_API_VERSION", "2024-06-01"),
description="Azure API version (defaults to AZURE_API_VERSION env var or 2024-06-01)",
)
timeout: float | None = Field(
default=None, description="Request timeout in seconds"
)
max_retries: int = Field(default=2, description="Maximum number of retries")
top_p: float | None = Field(default=None, description="Nucleus sampling parameter")
frequency_penalty: float | None = Field(
default=None, le=2.0, ge=-2.0, description="Frequency penalty (-2 to 2)"
)
presence_penalty: float | None = Field(
default=None, le=2.0, ge=-2.0, description="Presence penalty (-2 to 2)"
)
max_tokens: int | None = Field(
default=None, description="Maximum tokens in response"
)
stream: bool = Field(default=False, description="Enable streaming responses")
_client: ChatCompletionsClient = PrivateAttr(default=None) # type: ignore[assignment]
Args:
model: Azure deployment name or model name
api_key: Azure API key (defaults to AZURE_API_KEY env var)
endpoint: Azure endpoint URL (defaults to AZURE_ENDPOINT env var)
api_version: Azure API version (defaults to AZURE_API_VERSION env var)
timeout: Request timeout in seconds
max_retries: Maximum number of retries
temperature: Sampling temperature (0-2)
top_p: Nucleus sampling parameter
frequency_penalty: Frequency penalty (-2 to 2)
presence_penalty: Presence penalty (-2 to 2)
max_tokens: Maximum tokens in response
stop: Stop sequences
stream: Enable streaming responses
interceptor: HTTP interceptor (not yet supported for Azure).
**kwargs: Additional parameters
"""
if interceptor is not None:
_is_openai_model: bool = PrivateAttr(default=False)
_is_azure_openai_endpoint: bool = PrivateAttr(default=False)
@model_validator(mode="after")
def setup_client(self) -> Self:
"""Initialize the Azure client and validate configuration."""
if self.interceptor is not None:
raise NotImplementedError(
"HTTP interceptors are not yet supported for Azure AI Inference provider. "
"Interceptors are currently supported for OpenAI and Anthropic providers only."
)
super().__init__(
model=model, temperature=temperature, stop=stop or [], **kwargs
)
self.api_key = api_key or os.getenv("AZURE_API_KEY")
self.endpoint = (
endpoint
or os.getenv("AZURE_ENDPOINT")
or os.getenv("AZURE_OPENAI_ENDPOINT")
or os.getenv("AZURE_API_BASE")
)
self.api_version = api_version or os.getenv("AZURE_API_VERSION") or "2024-06-01"
self.timeout = timeout
self.max_retries = max_retries
if not self.api_key:
self.api_key = os.getenv("AZURE_API_KEY")
if not self.api_key:
raise ValueError(
"Azure API key is required. Set AZURE_API_KEY environment variable or pass api_key parameter."
)
if not self.endpoint:
raise ValueError(
"Azure endpoint is required. Set AZURE_ENDPOINT environment variable or pass endpoint parameter."
)
# Validate and potentially fix Azure OpenAI endpoint URL
self.endpoint = self._validate_and_fix_endpoint(self.endpoint, model)
self.endpoint = self._validate_and_fix_endpoint(self.endpoint, self.model)
# Build client kwargs
client_kwargs = {
client_kwargs: dict[str, Any] = {
"endpoint": self.endpoint,
"credential": AzureKeyCredential(self.api_key),
}
# Add api_version if specified (primarily for Azure OpenAI endpoints)
if self.api_version:
client_kwargs["api_version"] = self.api_version
self.client = ChatCompletionsClient(**client_kwargs) # type: ignore[arg-type]
self._client = ChatCompletionsClient(**client_kwargs)
self.top_p = top_p
self.frequency_penalty = frequency_penalty
self.presence_penalty = presence_penalty
self.max_tokens = max_tokens
self.stream = stream
self.is_openai_model = any(
prefix in model.lower() for prefix in ["gpt-", "o1-", "text-"]
self._is_openai_model = any(
prefix in self.model.lower() for prefix in ["gpt-", "o1-", "text-"]
)
self.is_azure_openai_endpoint = (
self._is_azure_openai_endpoint = (
"openai.azure.com" in self.endpoint
and "/openai/deployments/" in self.endpoint
)
def _validate_and_fix_endpoint(self, endpoint: str, model: str) -> str:
return self
@property
def is_openai_model(self) -> bool:
"""Check if model is an OpenAI model."""
return self._is_openai_model
@property
def is_azure_openai_endpoint(self) -> bool:
"""Check if endpoint is an Azure OpenAI endpoint."""
return self._is_azure_openai_endpoint
def _validate_and_fix_endpoint(self, endpoint: str | None, model: str) -> str:
"""Validate and fix Azure endpoint URL format.
Azure OpenAI endpoints should be in the format:
@@ -160,7 +163,15 @@ class AzureCompletion(BaseLLM):
Returns:
Validated and potentially corrected endpoint URL
Raises:
ValueError: If endpoint is None or empty
"""
if not endpoint:
raise ValueError(
"Azure endpoint is required. Set AZURE_ENDPOINT environment variable or pass endpoint parameter."
)
if "openai.azure.com" in endpoint and "/openai/deployments/" not in endpoint:
endpoint = endpoint.rstrip("/")
@@ -177,8 +188,8 @@ class AzureCompletion(BaseLLM):
tools: list[dict[str, BaseTool]] | None = None,
callbacks: list[Any] | None = None,
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
response_model: type[BaseModel] | None = None,
) -> str | Any:
"""Call Azure AI Inference chat completions API.
@@ -317,8 +328,6 @@ class AzureCompletion(BaseLLM):
) -> list[dict[str, Any]]:
"""Convert CrewAI tool format to Azure OpenAI function calling format."""
from crewai.llms.providers.utils.common import safe_tool_conversion
azure_tools = []
for tool in tools:
@@ -371,14 +380,14 @@ class AzureCompletion(BaseLLM):
self,
params: dict[str, Any],
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
response_model: type[BaseModel] | None = None,
) -> str | Any:
"""Handle non-streaming chat completion."""
# Make API call
try:
response: ChatCompletions = self.client.complete(**params)
response: ChatCompletions = self._client.complete(**params)
if not response.choices:
raise ValueError("No choices returned from Azure API")
@@ -467,8 +476,8 @@ class AzureCompletion(BaseLLM):
self,
params: dict[str, Any],
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
response_model: type[BaseModel] | None = None,
) -> str:
"""Handle streaming chat completion."""
@@ -476,7 +485,7 @@ class AzureCompletion(BaseLLM):
tool_calls = {}
# Make streaming API call
for update in self.client.complete(**params):
for update in self._client.complete(**params):
if isinstance(update, StreamingChatCompletionsUpdate):
if update.choices:
choice = update.choices[0]
@@ -554,7 +563,6 @@ class AzureCompletion(BaseLLM):
def get_context_window_size(self) -> int:
"""Get the context window size for the model."""
from crewai.llm import CONTEXT_WINDOW_USAGE_RATIO, LLM_CONTEXT_WINDOW_SIZES
min_context = 1024
max_context = 2097152

View File

@@ -5,11 +5,15 @@ import logging
import os
from typing import TYPE_CHECKING, Any, TypedDict, cast
from pydantic import BaseModel
from typing_extensions import Required
from dotenv import load_dotenv
from mypy_boto3_bedrock_runtime.client import BedrockRuntimeClient
from pydantic import BaseModel, Field, PrivateAttr, model_validator
from typing_extensions import Required, Self
from crewai.events.types.llm_events import LLMCallType
from crewai.llms.base_llm import BaseLLM
from crewai.llm.base_llm import BaseLLM
from crewai.llm.core import CONTEXT_WINDOW_USAGE_RATIO
from crewai.llm.providers.utils.common import safe_tool_conversion
from crewai.utilities.agent_utils import is_context_length_exceeded
from crewai.utilities.exceptions.context_window_exceeding_exception import (
LLMContextLengthExceededError,
@@ -30,7 +34,8 @@ if TYPE_CHECKING:
ToolTypeDef,
)
from crewai.llms.hooks.base import BaseInterceptor
from crewai.agent.core import Agent
from crewai.task import Task
try:
@@ -72,6 +77,9 @@ else:
topK: int
load_dotenv()
class ToolInputSchema(TypedDict):
"""Type definition for tool input schema in Converse API."""
@@ -141,74 +149,84 @@ class BedrockCompletion(BaseLLM):
- Complete streaming event handling (messageStart, contentBlockStart, etc.)
- Response metadata and trace information capture
- Model-specific conversation format handling (e.g., Cohere requirements)
Attributes:
model: The Bedrock model ID to use
aws_access_key_id: AWS access key (defaults to environment variable)
aws_secret_access_key: AWS secret key (defaults to environment variable)
aws_session_token: AWS session token for temporary credentials
region_name: AWS region name
max_tokens: Maximum tokens to generate
top_p: Nucleus sampling parameter
top_k: Top-k sampling parameter (Claude models only)
stream: Whether to use streaming responses
guardrail_config: Guardrail configuration for content filtering
additional_model_request_fields: Model-specific request parameters
additional_model_response_field_paths: Custom response field paths
interceptor: HTTP interceptor (not yet supported for Bedrock)
"""
def __init__(
self,
model: str = "anthropic.claude-3-5-sonnet-20241022-v2:0",
aws_access_key_id: str | None = None,
aws_secret_access_key: str | None = None,
aws_session_token: str | None = None,
region_name: str = "us-east-1",
temperature: float | None = None,
max_tokens: int | None = None,
top_p: float | None = None,
top_k: int | None = None,
stop_sequences: Sequence[str] | None = None,
stream: bool = False,
guardrail_config: dict[str, Any] | None = None,
additional_model_request_fields: dict[str, Any] | None = None,
additional_model_response_field_paths: list[str] | None = None,
interceptor: BaseInterceptor[Any, Any] | None = None,
**kwargs: Any,
) -> None:
"""Initialize AWS Bedrock completion client.
aws_access_key_id: str = Field( # type: ignore[assignment]
default_factory=lambda: os.getenv("AWS_ACCESS_KEY_ID"),
description="AWS access key (defaults to environment variable)",
)
aws_secret_access_key: str = Field( # type: ignore[assignment]
default_factory=lambda: os.getenv("AWS_SECRET_ACCESS_KEY"),
description="AWS secret key (defaults to environment variable)",
)
aws_session_token: str = Field( # type: ignore[assignment]
default_factory=lambda: os.getenv("AWS_SESSION_TOKEN"),
description="AWS session token for temporary credentials",
)
region_name: str = Field(
default_factory=lambda: os.getenv("AWS_REGION", "us-east-1"),
description="AWS region name",
)
max_tokens: int | None = Field(
default=None, description="Maximum tokens to generate"
)
top_p: float | None = Field(default=None, description="Nucleus sampling parameter")
top_k: int | None = Field(
default=None, description="Top-k sampling parameter (Claude models only)"
)
stream: bool = Field(
default=False, description="Whether to use streaming responses"
)
guardrail_config: dict[str, Any] = Field(
default_factory=dict,
description="Guardrail configuration for content filtering",
)
additional_model_request_fields: dict[str, Any] = Field(
default_factory=dict, description="Model-specific request parameters"
)
additional_model_response_field_paths: list[str] = Field(
default_factory=list, description="Custom response field paths"
)
_client: BedrockRuntimeClient = PrivateAttr( # type: ignore[assignment]
default_factory=lambda: Session().client,
)
Args:
model: The Bedrock model ID to use
aws_access_key_id: AWS access key (defaults to environment variable)
aws_secret_access_key: AWS secret key (defaults to environment variable)
aws_session_token: AWS session token for temporary credentials
region_name: AWS region name
temperature: Sampling temperature for response generation
max_tokens: Maximum tokens to generate
top_p: Nucleus sampling parameter
top_k: Top-k sampling parameter (Claude models only)
stop_sequences: List of sequences that stop generation
stream: Whether to use streaming responses
guardrail_config: Guardrail configuration for content filtering
additional_model_request_fields: Model-specific request parameters
additional_model_response_field_paths: Custom response field paths
interceptor: HTTP interceptor (not yet supported for Bedrock).
**kwargs: Additional parameters
"""
if interceptor is not None:
_is_claude_model: bool = PrivateAttr(default=False)
_supports_tools: bool = PrivateAttr(default=True)
_supports_streaming: bool = PrivateAttr(default=True)
_model_id: str = PrivateAttr()
@model_validator(mode="after")
def setup_client(self) -> Self:
"""Initialize the Bedrock client and validate configuration."""
if self.interceptor is not None:
raise NotImplementedError(
"HTTP interceptors are not yet supported for AWS Bedrock provider. "
"Interceptors are currently supported for OpenAI and Anthropic providers only."
)
# Extract provider from kwargs to avoid duplicate argument
kwargs.pop("provider", None)
super().__init__(
model=model,
temperature=temperature,
stop=stop_sequences or [],
provider="bedrock",
**kwargs,
)
# Initialize Bedrock client with proper configuration
session = Session(
aws_access_key_id=aws_access_key_id or os.getenv("AWS_ACCESS_KEY_ID"),
aws_secret_access_key=aws_secret_access_key
or os.getenv("AWS_SECRET_ACCESS_KEY"),
aws_session_token=aws_session_token or os.getenv("AWS_SESSION_TOKEN"),
region_name=region_name,
aws_access_key_id=self.aws_access_key_id,
aws_secret_access_key=self.aws_secret_access_key,
aws_session_token=self.aws_session_token,
region_name=self.region_name,
)
# Configure client with timeouts and retries following AWS best practices
config = Config(
read_timeout=300,
retries={
@@ -218,54 +236,34 @@ class BedrockCompletion(BaseLLM):
tcp_keepalive=True,
)
self.client = session.client("bedrock-runtime", config=config)
self.region_name = region_name
self._client = session.client("bedrock-runtime", config=config)
# Store completion parameters
self.max_tokens = max_tokens
self.top_p = top_p
self.top_k = top_k
self.stream = stream
self.stop_sequences = stop_sequences or []
self._is_claude_model = "claude" in self.model.lower()
self._supports_tools = True
self._supports_streaming = True
self._model_id = self.model
# Store advanced features (optional)
self.guardrail_config = guardrail_config
self.additional_model_request_fields = additional_model_request_fields
self.additional_model_response_field_paths = (
additional_model_response_field_paths
)
# Model-specific settings
self.is_claude_model = "claude" in model.lower()
self.supports_tools = True # Converse API supports tools for most models
self.supports_streaming = True
# Handle inference profiles for newer models
self.model_id = model
return self
@property
def stop(self) -> list[str]:
"""Get stop sequences sent to the API."""
return list(self.stop_sequences)
def is_claude_model(self) -> bool:
"""Check if model is a Claude model."""
return self._is_claude_model
@stop.setter
def stop(self, value: Sequence[str] | str | None) -> None:
"""Set stop sequences.
@property
def supports_tools(self) -> bool:
"""Check if model supports tools."""
return self._supports_tools
Synchronizes stop_sequences to ensure values set by CrewAgentExecutor
are properly sent to the Bedrock API.
@property
def supports_streaming(self) -> bool:
"""Check if model supports streaming."""
return self._supports_streaming
Args:
value: Stop sequences as a Sequence, single string, or None
"""
if value is None:
self.stop_sequences = []
elif isinstance(value, str):
self.stop_sequences = [value]
elif isinstance(value, Sequence):
self.stop_sequences = list(value)
else:
self.stop_sequences = []
@property
def model_id(self) -> str:
"""Get the model ID."""
return self._model_id
def call(
self,
@@ -273,8 +271,8 @@ class BedrockCompletion(BaseLLM):
tools: list[dict[Any, Any]] | None = None,
callbacks: list[Any] | None = None,
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
response_model: type[BaseModel] | None = None,
) -> str | Any:
"""Call AWS Bedrock Converse API."""
@@ -359,8 +357,8 @@ class BedrockCompletion(BaseLLM):
messages: list[dict[str, Any]],
body: BedrockConverseRequestBody,
available_functions: Mapping[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
) -> str:
"""Handle non-streaming converse API call following AWS best practices."""
try:
@@ -378,7 +376,7 @@ class BedrockCompletion(BaseLLM):
raise ValueError(f"Invalid message format at index {i}")
# Call Bedrock Converse API with proper error handling
response = self.client.converse(
response = self._client.converse(
modelId=self.model_id,
messages=cast(
"Sequence[MessageTypeDef | MessageOutputTypeDef]",
@@ -540,8 +538,8 @@ class BedrockCompletion(BaseLLM):
messages: list[dict[str, Any]],
body: BedrockConverseRequestBody,
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
) -> str:
"""Handle streaming converse API call with comprehensive event handling."""
full_response = ""
@@ -549,7 +547,7 @@ class BedrockCompletion(BaseLLM):
tool_use_id = None
try:
response = self.client.converse_stream(
response = self._client.converse_stream(
modelId=self.model_id,
messages=cast(
"Sequence[MessageTypeDef | MessageOutputTypeDef]",
@@ -778,7 +776,6 @@ class BedrockCompletion(BaseLLM):
tools: list[dict[str, Any]],
) -> list[ConverseToolTypeDef]:
"""Convert CrewAI tools to Converse API format following AWS specification."""
from crewai.llms.providers.utils.common import safe_tool_conversion
converse_tools: list[ConverseToolTypeDef] = []
@@ -818,8 +815,8 @@ class BedrockCompletion(BaseLLM):
config["temperature"] = float(self.temperature)
if self.top_p is not None:
config["topP"] = float(self.top_p)
if self.stop_sequences:
config["stopSequences"] = self.stop_sequences
if self.stop:
config["stopSequences"] = self.stop
if self.is_claude_model and self.top_k is not None:
# top_k is supported by Claude models
@@ -871,7 +868,6 @@ class BedrockCompletion(BaseLLM):
def get_context_window_size(self) -> int:
"""Get the context window size for the model."""
from crewai.llm import CONTEXT_WINDOW_USAGE_RATIO
# Context window sizes for common Bedrock models
context_windows = {

View File

@@ -1,12 +1,17 @@
from __future__ import annotations
import logging
import os
from typing import Any, cast
from typing import TYPE_CHECKING, Any, cast
from pydantic import BaseModel
from dotenv import load_dotenv
from pydantic import BaseModel, Field, PrivateAttr, model_validator
from typing_extensions import Self
from crewai.events.types.llm_events import LLMCallType
from crewai.llms.base_llm import BaseLLM
from crewai.llms.hooks.base import BaseInterceptor
from crewai.llm.base_llm import BaseLLM
from crewai.llm.core import CONTEXT_WINDOW_USAGE_RATIO, LLM_CONTEXT_WINDOW_SIZES
from crewai.llm.providers.utils.common import safe_tool_conversion
from crewai.utilities.agent_utils import is_context_length_exceeded
from crewai.utilities.exceptions.context_window_exceeding_exception import (
LLMContextLengthExceededError,
@@ -14,6 +19,11 @@ from crewai.utilities.exceptions.context_window_exceeding_exception import (
from crewai.utilities.types import LLMMessage
if TYPE_CHECKING:
from crewai.agent.core import Agent
from crewai.task import Task
try:
from google import genai # type: ignore[import-untyped]
from google.genai import types # type: ignore[import-untyped]
@@ -24,111 +34,93 @@ except ImportError:
) from None
load_dotenv()
class GeminiCompletion(BaseLLM):
"""Google Gemini native completion implementation.
This class provides direct integration with the Google Gen AI Python SDK,
offering native function calling, streaming support, and proper Gemini formatting.
Attributes:
model: Gemini model name (e.g., 'gemini-2.0-flash-001', 'gemini-1.5-pro')
project: Google Cloud project ID (for Vertex AI)
location: Google Cloud location (for Vertex AI, defaults to 'us-central1')
top_p: Nucleus sampling parameter
top_k: Top-k sampling parameter
max_output_tokens: Maximum tokens in response
stream: Enable streaming responses
safety_settings: Safety filter settings
client_params: Additional parameters for Google Gen AI Client constructor
interceptor: HTTP interceptor (not yet supported for Gemini)
"""
def __init__(
self,
model: str = "gemini-2.0-flash-001",
api_key: str | None = None,
project: str | None = None,
location: str | None = None,
temperature: float | None = None,
top_p: float | None = None,
top_k: int | None = None,
max_output_tokens: int | None = None,
stop_sequences: list[str] | None = None,
stream: bool = False,
safety_settings: dict[str, Any] | None = None,
client_params: dict[str, Any] | None = None,
interceptor: BaseInterceptor[Any, Any] | None = None,
**kwargs: Any,
):
"""Initialize Google Gemini chat completion client.
project: str | None = Field(
default_factory=lambda: os.getenv("GOOGLE_CLOUD_PROJECT"),
description="Google Cloud project ID (for Vertex AI)",
)
location: str = Field(
default_factory=lambda: os.getenv("GOOGLE_CLOUD_LOCATION", "us-central1"),
description="Google Cloud location (for Vertex AI, defaults to 'us-central1')",
)
top_p: float | None = Field(default=None, description="Nucleus sampling parameter")
top_k: int | None = Field(default=None, description="Top-k sampling parameter")
max_output_tokens: int | None = Field(
default=None, description="Maximum tokens in response"
)
stream: bool = Field(default=False, description="Enable streaming responses")
safety_settings: dict[str, Any] = Field(
default_factory=dict, description="Safety filter settings"
)
client_params: dict[str, Any] = Field(
default_factory=dict,
description="Additional parameters for Google Gen AI Client constructor",
)
_client: Any = PrivateAttr(default=None)
Args:
model: Gemini model name (e.g., 'gemini-2.0-flash-001', 'gemini-1.5-pro')
api_key: Google API key (defaults to GOOGLE_API_KEY or GEMINI_API_KEY env var)
project: Google Cloud project ID (for Vertex AI)
location: Google Cloud location (for Vertex AI, defaults to 'us-central1')
temperature: Sampling temperature (0-2)
top_p: Nucleus sampling parameter
top_k: Top-k sampling parameter
max_output_tokens: Maximum tokens in response
stop_sequences: Stop sequences
stream: Enable streaming responses
safety_settings: Safety filter settings
client_params: Additional parameters to pass to the Google Gen AI Client constructor.
Supports parameters like http_options, credentials, debug_config, etc.
interceptor: HTTP interceptor (not yet supported for Gemini).
**kwargs: Additional parameters
"""
if interceptor is not None:
_is_gemini_2: bool = PrivateAttr(default=False)
_is_gemini_1_5: bool = PrivateAttr(default=False)
_supports_tools: bool = PrivateAttr(default=False)
@model_validator(mode="after")
def setup_client(self) -> Self:
"""Initialize the Gemini client and validate configuration."""
if self.interceptor is not None:
raise NotImplementedError(
"HTTP interceptors are not yet supported for Google Gemini provider. "
"Interceptors are currently supported for OpenAI and Anthropic providers only."
)
super().__init__(
model=model, temperature=temperature, stop=stop_sequences or [], **kwargs
)
# Store client params for later use
self.client_params = client_params or {}
# Get API configuration with environment variable fallbacks
self.api_key = (
api_key or os.getenv("GOOGLE_API_KEY") or os.getenv("GEMINI_API_KEY")
)
self.project = project or os.getenv("GOOGLE_CLOUD_PROJECT")
self.location = location or os.getenv("GOOGLE_CLOUD_LOCATION") or "us-central1"
if self.api_key is None:
self.api_key = os.getenv("GOOGLE_API_KEY") or os.getenv("GEMINI_API_KEY")
use_vertexai = os.getenv("GOOGLE_GENAI_USE_VERTEXAI", "").lower() == "true"
self.client = self._initialize_client(use_vertexai)
self._client = self._initialize_client(use_vertexai)
# Store completion parameters
self.top_p = top_p
self.top_k = top_k
self.max_output_tokens = max_output_tokens
self.stream = stream
self.safety_settings = safety_settings or {}
self.stop_sequences = stop_sequences or []
self._is_gemini_2 = "gemini-2" in self.model.lower()
self._is_gemini_1_5 = "gemini-1.5" in self.model.lower()
self._supports_tools = self._is_gemini_1_5 or self._is_gemini_2
# Model-specific settings
self.is_gemini_2 = "gemini-2" in model.lower()
self.is_gemini_1_5 = "gemini-1.5" in model.lower()
self.supports_tools = self.is_gemini_1_5 or self.is_gemini_2
return self
@property
def stop(self) -> list[str]:
"""Get stop sequences sent to the API."""
return self.stop_sequences
def is_gemini_2(self) -> bool:
"""Check if model is Gemini 2."""
return self._is_gemini_2
@stop.setter
def stop(self, value: list[str] | str | None) -> None:
"""Set stop sequences.
@property
def is_gemini_1_5(self) -> bool:
"""Check if model is Gemini 1.5."""
return self._is_gemini_1_5
Synchronizes stop_sequences to ensure values set by CrewAgentExecutor
are properly sent to the Gemini API.
@property
def supports_tools(self) -> bool:
"""Check if model supports tools."""
return self._supports_tools
Args:
value: Stop sequences as a list, single string, or None
"""
if value is None:
self.stop_sequences = []
elif isinstance(value, str):
self.stop_sequences = [value]
elif isinstance(value, list):
self.stop_sequences = value
else:
self.stop_sequences = []
def _initialize_client(self, use_vertexai: bool = False) -> genai.Client: # type: ignore[no-any-unimported]
def _initialize_client(self, use_vertexai: bool = False) -> Any:
"""Initialize the Google Gen AI client with proper parameter handling.
Args:
@@ -150,12 +142,9 @@ class GeminiCompletion(BaseLLM):
"location": self.location,
}
)
client_params.pop("api_key", None)
elif self.api_key:
client_params["api_key"] = self.api_key
client_params.pop("vertexai", None)
client_params.pop("project", None)
client_params.pop("location", None)
@@ -180,11 +169,10 @@ class GeminiCompletion(BaseLLM):
params = {}
if (
hasattr(self, "client")
and hasattr(self.client, "vertexai")
and self.client.vertexai
hasattr(self, "_client")
and hasattr(self._client, "vertexai")
and self._client.vertexai
):
# Vertex AI configuration
params.update(
{
"vertexai": True,
@@ -206,8 +194,8 @@ class GeminiCompletion(BaseLLM):
tools: list[dict[str, Any]] | None = None,
callbacks: list[Any] | None = None,
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
response_model: type[BaseModel] | None = None,
) -> str | Any:
"""Call Google Gemini generate content API.
@@ -296,15 +284,12 @@ class GeminiCompletion(BaseLLM):
self.tools = tools
config_params = {}
# Add system instruction if present
if system_instruction:
# Convert system instruction to Content format
system_content = types.Content(
role="user", parts=[types.Part.from_text(text=system_instruction)]
)
config_params["system_instruction"] = system_content
# Add generation config parameters
if self.temperature is not None:
config_params["temperature"] = self.temperature
if self.top_p is not None:
@@ -313,14 +298,13 @@ class GeminiCompletion(BaseLLM):
config_params["top_k"] = self.top_k
if self.max_output_tokens is not None:
config_params["max_output_tokens"] = self.max_output_tokens
if self.stop_sequences:
config_params["stop_sequences"] = self.stop_sequences
if self.stop:
config_params["stop_sequences"] = self.stop
if response_model:
config_params["response_mime_type"] = "application/json"
config_params["response_schema"] = response_model.model_json_schema()
# Handle tools for supported models
if tools and self.supports_tools:
config_params["tools"] = self._convert_tools_for_interference(tools)
@@ -335,8 +319,6 @@ class GeminiCompletion(BaseLLM):
"""Convert CrewAI tool format to Gemini function declaration format."""
gemini_tools = []
from crewai.llms.providers.utils.common import safe_tool_conversion
for tool in tools:
name, description, parameters = safe_tool_conversion(tool, "Gemini")
@@ -345,7 +327,6 @@ class GeminiCompletion(BaseLLM):
description=description,
)
# Add parameters if present - ensure parameters is a dict
if parameters and isinstance(parameters, dict):
function_declaration.parameters = parameters
@@ -381,16 +362,12 @@ class GeminiCompletion(BaseLLM):
content = message.get("content", "")
if role == "system":
# Extract system instruction - Gemini handles it separately
if system_instruction:
system_instruction += f"\n\n{content}"
else:
system_instruction = cast(str, content)
else:
# Convert role for Gemini (assistant -> model)
gemini_role = "model" if role == "assistant" else "user"
# Create Content object
gemini_content = types.Content(
role=gemini_role, parts=[types.Part.from_text(text=content)]
)
@@ -404,8 +381,8 @@ class GeminiCompletion(BaseLLM):
system_instruction: str | None,
config: types.GenerateContentConfig,
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
response_model: type[BaseModel] | None = None,
) -> str | Any:
"""Handle non-streaming content generation."""
@@ -416,7 +393,7 @@ class GeminiCompletion(BaseLLM):
}
try:
response = self.client.models.generate_content(**api_params)
response = self._client.models.generate_content(**api_params)
usage = self._extract_token_usage(response)
except Exception as e:
@@ -470,8 +447,8 @@ class GeminiCompletion(BaseLLM):
contents: list[types.Content],
config: types.GenerateContentConfig,
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
response_model: type[BaseModel] | None = None,
) -> str:
"""Handle streaming content generation."""
@@ -484,7 +461,7 @@ class GeminiCompletion(BaseLLM):
"config": config,
}
for chunk in self.client.models.generate_content_stream(**api_params):
for chunk in self._client.models.generate_content_stream(**api_params):
if hasattr(chunk, "text") and chunk.text:
full_response += chunk.text
self._emit_stream_chunk_event(
@@ -507,13 +484,11 @@ class GeminiCompletion(BaseLLM):
else {},
}
# Handle completed function calls
if function_calls and available_functions:
for call_data in function_calls.values():
function_name = call_data["name"]
function_args = call_data["args"]
# Execute tool
result = self._handle_tool_execution(
function_name=function_name,
function_args=function_args,
@@ -547,7 +522,6 @@ class GeminiCompletion(BaseLLM):
def get_context_window_size(self) -> int:
"""Get the context window size for the model."""
from crewai.llm import CONTEXT_WINDOW_USAGE_RATIO, LLM_CONTEXT_WINDOW_SIZES
min_context = 1024
max_context = 2097152
@@ -574,13 +548,11 @@ class GeminiCompletion(BaseLLM):
"gemma-3-27b": 128000,
}
# Find the best match for the model name
for model_prefix, size in context_windows.items():
if self.model.startswith(model_prefix):
return int(size * CONTEXT_WINDOW_USAGE_RATIO)
# Default context window size for Gemini models
return int(1048576 * CONTEXT_WINDOW_USAGE_RATIO) # 1M tokens
return int(1048576 * CONTEXT_WINDOW_USAGE_RATIO)
def _extract_token_usage(self, response: dict[str, Any]) -> dict[str, Any]:
"""Extract token usage from Gemini response."""

View File

@@ -6,16 +6,20 @@ import logging
import os
from typing import TYPE_CHECKING, Any
from dotenv import load_dotenv
import httpx
from openai import APIConnectionError, NotFoundError, OpenAI
from openai.types.chat import ChatCompletion, ChatCompletionChunk
from openai.types.chat.chat_completion import Choice
from openai.types.chat.chat_completion_chunk import ChoiceDelta
from pydantic import BaseModel
from pydantic import BaseModel, Field, PrivateAttr, model_validator
from typing_extensions import Self
from crewai.events.types.llm_events import LLMCallType
from crewai.llms.base_llm import BaseLLM
from crewai.llms.hooks.transport import HTTPTransport
from crewai.llm.base_llm import BaseLLM
from crewai.llm.core import CONTEXT_WINDOW_USAGE_RATIO, LLM_CONTEXT_WINDOW_SIZES
from crewai.llm.hooks.transport import HTTPTransport
from crewai.llm.providers.utils.common import safe_tool_conversion
from crewai.utilities.agent_utils import is_context_length_exceeded
from crewai.utilities.exceptions.context_window_exceeding_exception import (
LLMContextLengthExceededError,
@@ -25,11 +29,13 @@ from crewai.utilities.types import LLMMessage
if TYPE_CHECKING:
from crewai.agent.core import Agent
from crewai.llms.hooks.base import BaseInterceptor
from crewai.task import Task
from crewai.tools.base_tool import BaseTool
load_dotenv()
class OpenAICompletion(BaseLLM):
"""OpenAI native completion implementation.
@@ -37,60 +43,56 @@ class OpenAICompletion(BaseLLM):
offering native structured outputs, function calling, and streaming support.
"""
def __init__(
self,
model: str = "gpt-4o",
api_key: str | None = None,
base_url: str | None = None,
organization: str | None = None,
project: str | None = None,
timeout: float | None = None,
max_retries: int = 2,
default_headers: dict[str, str] | None = None,
default_query: dict[str, Any] | None = None,
client_params: dict[str, Any] | None = None,
temperature: float | None = None,
top_p: float | None = None,
frequency_penalty: float | None = None,
presence_penalty: float | None = None,
max_tokens: int | None = None,
max_completion_tokens: int | None = None,
seed: int | None = None,
stream: bool = False,
response_format: dict[str, Any] | type[BaseModel] | None = None,
logprobs: bool | None = None,
top_logprobs: int | None = None,
reasoning_effort: str | None = None,
provider: str | None = None,
interceptor: BaseInterceptor[httpx.Request, httpx.Response] | None = None,
**kwargs: Any,
) -> None:
"""Initialize OpenAI chat completion client."""
# Client configuration fields
organization: str | None = Field(default=None, description="OpenAI organization ID")
project: str | None = Field(default=None, description="OpenAI project ID")
max_retries: int = Field(default=2, description="Maximum number of retries")
default_headers: dict[str, str] = Field(
default_factory=dict, description="Default headers for requests"
)
default_query: dict[str, Any] = Field(
default_factory=dict, description="Default query parameters"
)
client_params: dict[str, Any] = Field(
default_factory=dict, description="Additional client parameters"
)
timeout: float | None = Field(default=None, description="Request timeout")
api_base: str | None = Field(
default=None, description="API base URL", deprecated=True
)
if provider is None:
provider = kwargs.pop("provider", "openai")
# Completion parameters
top_p: float | None = Field(default=None, description="Top-p sampling parameter")
frequency_penalty: float | None = Field(
default=None, description="Frequency penalty"
)
presence_penalty: float | None = Field(default=None, description="Presence penalty")
max_tokens: int | None = Field(default=None, description="Maximum tokens")
max_completion_tokens: int | None = Field(
None, description="Maximum completion tokens"
)
seed: int | None = Field(default=None, description="Random seed")
stream: bool = Field(default=False, description="Enable streaming")
response_format: dict[str, Any] | type[BaseModel] | None = Field(
default=None, description="Response format"
)
logprobs: bool | None = Field(default=None, description="Return log probabilities")
top_logprobs: int | None = Field(
default=None, description="Number of top log probabilities"
)
reasoning_effort: str | None = Field(
default=None, description="Reasoning effort level"
)
self.interceptor = interceptor
# Client configuration attributes
self.organization = organization
self.project = project
self.max_retries = max_retries
self.default_headers = default_headers
self.default_query = default_query
self.client_params = client_params
self.timeout = timeout
self.base_url = base_url
self.api_base = kwargs.pop("api_base", None)
_client: OpenAI = PrivateAttr(default=None) # type: ignore[assignment]
is_o1_model: bool = Field(default=False, description="Whether this is an O1 model")
is_gpt4_model: bool = Field(
default=False, description="Whether this is a GPT-4 model"
)
super().__init__(
model=model,
temperature=temperature,
api_key=api_key or os.getenv("OPENAI_API_KEY"),
base_url=base_url,
timeout=timeout,
provider=provider,
**kwargs,
)
@model_validator(mode="after")
def setup_client(self) -> Self:
"""Initialize OpenAI client after model validation."""
client_config = self._get_client_params()
if self.interceptor:
@@ -98,31 +100,15 @@ class OpenAICompletion(BaseLLM):
http_client = httpx.Client(transport=transport)
client_config["http_client"] = http_client
self.client = OpenAI(**client_config)
self._client = OpenAI(**client_config)
# Completion parameters
self.top_p = top_p
self.frequency_penalty = frequency_penalty
self.presence_penalty = presence_penalty
self.max_tokens = max_tokens
self.max_completion_tokens = max_completion_tokens
self.seed = seed
self.stream = stream
self.response_format = response_format
self.logprobs = logprobs
self.top_logprobs = top_logprobs
self.reasoning_effort = reasoning_effort
self.is_o1_model = "o1" in model.lower()
self.is_gpt4_model = "gpt-4" in model.lower()
self.is_o1_model = "o1" in self.model.lower()
self.is_gpt4_model = "gpt-4" in self.model.lower()
return self
def _get_client_params(self) -> dict[str, Any]:
"""Get OpenAI client parameters."""
if self.api_key is None:
self.api_key = os.getenv("OPENAI_API_KEY")
if self.api_key is None:
raise ValueError("OPENAI_API_KEY is required")
base_params = {
"api_key": self.api_key,
"organization": self.organization,
@@ -268,7 +254,6 @@ class OpenAICompletion(BaseLLM):
self, tools: list[dict[str, BaseTool]]
) -> list[dict[str, Any]]:
"""Convert CrewAI tool format to OpenAI function calling format."""
from crewai.llms.providers.utils.common import safe_tool_conversion
openai_tools = []
@@ -296,14 +281,14 @@ class OpenAICompletion(BaseLLM):
self,
params: dict[str, Any],
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
response_model: type[BaseModel] | None = None,
) -> str | Any:
"""Handle non-streaming chat completion."""
try:
if response_model:
parsed_response = self.client.beta.chat.completions.parse(
parsed_response = self._client.beta.chat.completions.parse(
**params,
response_format=response_model,
)
@@ -327,7 +312,7 @@ class OpenAICompletion(BaseLLM):
)
return structured_json
response: ChatCompletion = self.client.chat.completions.create(**params)
response: ChatCompletion = self._client.chat.completions.create(**params)
usage = self._extract_openai_token_usage(response)
@@ -419,8 +404,8 @@ class OpenAICompletion(BaseLLM):
self,
params: dict[str, Any],
available_functions: dict[str, Any] | None = None,
from_task: Any | None = None,
from_agent: Any | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
response_model: type[BaseModel] | None = None,
) -> str:
"""Handle streaming chat completion."""
@@ -429,7 +414,7 @@ class OpenAICompletion(BaseLLM):
if response_model:
completion_stream: Iterator[ChatCompletionChunk] = (
self.client.chat.completions.create(**params)
self._client.chat.completions.create(**params)
)
accumulated_content = ""
@@ -472,7 +457,7 @@ class OpenAICompletion(BaseLLM):
)
return accumulated_content
stream: Iterator[ChatCompletionChunk] = self.client.chat.completions.create(
stream: Iterator[ChatCompletionChunk] = self._client.chat.completions.create(
**params
)
@@ -560,7 +545,6 @@ class OpenAICompletion(BaseLLM):
def get_context_window_size(self) -> int:
"""Get the context window size for the model."""
from crewai.llm import CONTEXT_WINDOW_USAGE_RATIO, LLM_CONTEXT_WINDOW_SIZES
min_context = 1024
max_context = 2097152

View File

@@ -1 +1,38 @@
"""LLM implementations for crewAI."""
"""LLM implementations for crewAI.
.. deprecated:: 1.4.0
The `crewai.llms` package is deprecated. Use `crewai.llm` instead.
This package was reorganized from `crewai.llms.*` to `crewai.llm.*`.
All submodules are redirected to their new locations in `crewai.llm.*`.
Migration guide:
Old: from crewai.llms.base_llm import BaseLLM
New: from crewai.llm.base_llm import BaseLLM
Old: from crewai.llms.hooks.base import BaseInterceptor
New: from crewai.llm.hooks.base import BaseInterceptor
Old: from crewai.llms.constants import OPENAI_MODELS
New: from crewai.llm.constants import OPENAI_MODELS
Or use top-level imports:
from crewai import LLM, BaseLLM
"""
import warnings
from crewai.llm import LLM
from crewai.llm.base_llm import BaseLLM
# Issue deprecation warning when this module is imported
warnings.warn(
"The 'crewai.llms' package is deprecated and will be removed in a future version. "
"Please use 'crewai.llm' (singular) instead. "
"All submodules have been reorganized from 'crewai.llms.*' to 'crewai.llm.*'.",
DeprecationWarning,
stacklevel=2,
)
__all__ = ["LLM", "BaseLLM"]

View File

@@ -1,550 +1,15 @@
"""Base LLM abstract class for CrewAI.
"""Deprecated: Use crewai.llm.base_llm instead.
This module provides the abstract base class for all LLM implementations
in CrewAI, including common functionality for native SDK implementations.
.. deprecated:: 1.4.0
"""
from __future__ import annotations
import warnings
from abc import ABC, abstractmethod
from datetime import datetime
import json
import logging
import re
from typing import TYPE_CHECKING, Any, Final
from pydantic import BaseModel
from crewai.events.event_bus import crewai_event_bus
from crewai.events.types.llm_events import (
LLMCallCompletedEvent,
LLMCallFailedEvent,
LLMCallStartedEvent,
LLMCallType,
LLMStreamChunkEvent,
warnings.warn(
"crewai.llms.base_llm is deprecated. Use crewai.llm.base_llm instead.",
DeprecationWarning,
stacklevel=2,
)
from crewai.events.types.tool_usage_events import (
ToolUsageErrorEvent,
ToolUsageFinishedEvent,
ToolUsageStartedEvent,
)
from crewai.types.usage_metrics import UsageMetrics
if TYPE_CHECKING:
from crewai.agent.core import Agent
from crewai.task import Task
from crewai.tools.base_tool import BaseTool
from crewai.utilities.types import LLMMessage
DEFAULT_CONTEXT_WINDOW_SIZE: Final[int] = 4096
DEFAULT_SUPPORTS_STOP_WORDS: Final[bool] = True
_JSON_EXTRACTION_PATTERN: Final[re.Pattern[str]] = re.compile(r"\{.*}", re.DOTALL)
class BaseLLM(ABC):
"""Abstract base class for LLM implementations.
This class defines the interface that all LLM implementations must follow.
Users can extend this class to create custom LLM implementations that don't
rely on litellm's authentication mechanism.
Custom LLM implementations should handle error cases gracefully, including
timeouts, authentication failures, and malformed responses. They should also
implement proper validation for input parameters and provide clear error
messages when things go wrong.
Attributes:
model: The model identifier/name.
temperature: Optional temperature setting for response generation.
stop: A list of stop sequences that the LLM should use to stop generation.
additional_params: Additional provider-specific parameters.
"""
is_litellm: bool = False
def __init__(
self,
model: str,
temperature: float | None = None,
api_key: str | None = None,
base_url: str | None = None,
provider: str | None = None,
**kwargs: Any,
) -> None:
"""Initialize the BaseLLM with default attributes.
Args:
model: The model identifier/name.
temperature: Optional temperature setting for response generation.
stop: Optional list of stop sequences for generation.
**kwargs: Additional provider-specific parameters.
"""
if not model:
raise ValueError("Model name is required and cannot be empty")
self.model = model
self.temperature = temperature
self.api_key = api_key
self.base_url = base_url
# Store additional parameters for provider-specific use
self.additional_params = kwargs
self._provider = provider or "openai"
stop = kwargs.pop("stop", None)
if stop is None:
self.stop: list[str] = []
elif isinstance(stop, str):
self.stop = [stop]
elif isinstance(stop, list):
self.stop = stop
else:
self.stop = []
self._token_usage = {
"total_tokens": 0,
"prompt_tokens": 0,
"completion_tokens": 0,
"successful_requests": 0,
"cached_prompt_tokens": 0,
}
@property
def provider(self) -> str:
"""Get the provider of the LLM."""
return self._provider
@provider.setter
def provider(self, value: str) -> None:
"""Set the provider of the LLM."""
self._provider = value
@abstractmethod
def call(
self,
messages: str | list[LLMMessage],
tools: list[dict[str, BaseTool]] | None = None,
callbacks: list[Any] | None = None,
available_functions: dict[str, Any] | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
response_model: type[BaseModel] | None = None,
) -> str | Any:
"""Call the LLM with the given messages.
Args:
messages: Input messages for the LLM.
Can be a string or list of message dictionaries.
If string, it will be converted to a single user message.
If list, each dict must have 'role' and 'content' keys.
tools: Optional list of tool schemas for function calling.
Each tool should define its name, description, and parameters.
callbacks: Optional list of callback functions to be executed
during and after the LLM call.
available_functions: Optional dict mapping function names to callables
that can be invoked by the LLM.
from_task: Optional task caller to be used for the LLM call.
from_agent: Optional agent caller to be used for the LLM call.
response_model: Optional response model to be used for the LLM call.
Returns:
Either a text response from the LLM (str) or
the result of a tool function call (Any).
Raises:
ValueError: If the messages format is invalid.
TimeoutError: If the LLM request times out.
RuntimeError: If the LLM request fails for other reasons.
"""
def _convert_tools_for_interference(
self, tools: list[dict[str, BaseTool]]
) -> list[dict[str, BaseTool]]:
"""Convert tools to a format that can be used for interference.
Args:
tools: List of tools to convert.
Returns:
List of converted tools (default implementation returns as-is)
"""
return tools
def supports_stop_words(self) -> bool:
"""Check if the LLM supports stop words.
Returns:
True if the LLM supports stop words, False otherwise.
"""
return DEFAULT_SUPPORTS_STOP_WORDS
def _supports_stop_words_implementation(self) -> bool:
"""Check if stop words are configured for this LLM instance.
Native providers can override supports_stop_words() to return this value
to ensure consistent behavior based on whether stop words are actually configured.
Returns:
True if stop words are configured and can be applied
"""
return bool(self.stop)
def _apply_stop_words(self, content: str) -> str:
"""Apply stop words to truncate response content.
This method provides consistent stop word behavior across all native SDK providers.
Native providers should call this method to post-process their responses.
Args:
content: The raw response content from the LLM
Returns:
Content truncated at the first occurrence of any stop word
Example:
>>> llm = MyNativeLLM(stop=["Observation:", "Final Answer:"])
>>> response = (
... "I need to search.\\n\\nAction: search\\nObservation: Found results"
... )
>>> llm._apply_stop_words(response)
"I need to search.\\n\\nAction: search"
"""
if not self.stop or not content:
return content
# Find the earliest occurrence of any stop word
earliest_stop_pos = len(content)
found_stop_word = None
for stop_word in self.stop:
stop_pos = content.find(stop_word)
if stop_pos != -1 and stop_pos < earliest_stop_pos:
earliest_stop_pos = stop_pos
found_stop_word = stop_word
# Truncate at the stop word if found
if found_stop_word is not None:
truncated = content[:earliest_stop_pos].strip()
logging.debug(
f"Applied stop word '{found_stop_word}' at position {earliest_stop_pos}"
)
return truncated
return content
def get_context_window_size(self) -> int:
"""Get the context window size for the LLM.
Returns:
The number of tokens/characters the model can handle.
"""
# Default implementation - subclasses should override with model-specific values
return DEFAULT_CONTEXT_WINDOW_SIZE
# Common helper methods for native SDK implementations
def _emit_call_started_event(
self,
messages: str | list[LLMMessage],
tools: list[dict[str, BaseTool]] | None = None,
callbacks: list[Any] | None = None,
available_functions: dict[str, Any] | None = None,
from_task: Task | None = None,
from_agent: Agent | None = None,
) -> None:
"""Emit LLM call started event."""
if not hasattr(crewai_event_bus, "emit"):
raise ValueError("crewai_event_bus does not have an emit method") from None
crewai_event_bus.emit(
self,
event=LLMCallStartedEvent(
messages=messages,
tools=tools,
callbacks=callbacks,
available_functions=available_functions,
from_task=from_task,
from_agent=from_agent,
model=self.model,
),
)
def _emit_call_completed_event(
self,
response: Any,
call_type: LLMCallType,
from_task: Task | None = None,
from_agent: Agent | None = None,
messages: str | list[dict[str, Any]] | None = None,
) -> None:
"""Emit LLM call completed event."""
crewai_event_bus.emit(
self,
event=LLMCallCompletedEvent(
messages=messages,
response=response,
call_type=call_type,
from_task=from_task,
from_agent=from_agent,
model=self.model,
),
)
def _emit_call_failed_event(
self,
error: str,
from_task: Task | None = None,
from_agent: Agent | None = None,
) -> None:
"""Emit LLM call failed event."""
if not hasattr(crewai_event_bus, "emit"):
raise ValueError("crewai_event_bus does not have an emit method") from None
crewai_event_bus.emit(
self,
event=LLMCallFailedEvent(
error=error,
from_task=from_task,
from_agent=from_agent,
),
)
def _emit_stream_chunk_event(
self,
chunk: str,
from_task: Task | None = None,
from_agent: Agent | None = None,
tool_call: dict[str, Any] | None = None,
) -> None:
"""Emit stream chunk event."""
if not hasattr(crewai_event_bus, "emit"):
raise ValueError("crewai_event_bus does not have an emit method") from None
crewai_event_bus.emit(
self,
event=LLMStreamChunkEvent(
chunk=chunk,
tool_call=tool_call,
from_task=from_task,
from_agent=from_agent,
),
)
def _handle_tool_execution(
self,
function_name: str,
function_args: dict[str, Any],
available_functions: dict[str, Any],
from_task: Task | None = None,
from_agent: Agent | None = None,
) -> str | None:
"""Handle tool execution with proper event emission.
Args:
function_name: Name of the function to execute
function_args: Arguments to pass to the function
available_functions: Dict of available functions
from_task: Optional task object
from_agent: Optional agent object
Returns:
Result of function execution or None if function not found
"""
if function_name not in available_functions:
logging.warning(
f"Function '{function_name}' not found in available functions"
)
return None
try:
# Emit tool usage started event
started_at = datetime.now()
crewai_event_bus.emit(
self,
event=ToolUsageStartedEvent(
tool_name=function_name,
tool_args=function_args,
from_agent=from_agent,
from_task=from_task,
),
)
# Execute the function
fn = available_functions[function_name]
result = fn(**function_args)
# Emit tool usage finished event
crewai_event_bus.emit(
self,
event=ToolUsageFinishedEvent(
output=result,
tool_name=function_name,
tool_args=function_args,
started_at=started_at,
finished_at=datetime.now(),
from_task=from_task,
from_agent=from_agent,
),
)
# Emit LLM call completed event for tool call
self._emit_call_completed_event(
response=result,
call_type=LLMCallType.TOOL_CALL,
from_task=from_task,
from_agent=from_agent,
)
return str(result)
except Exception as e:
error_msg = f"Error executing function '{function_name}': {e!s}"
logging.error(error_msg)
# Emit tool usage error event
if not hasattr(crewai_event_bus, "emit"):
raise ValueError(
"crewai_event_bus does not have an emit method"
) from None
crewai_event_bus.emit(
self,
event=ToolUsageErrorEvent(
tool_name=function_name,
tool_args=function_args,
error=error_msg,
from_task=from_task,
from_agent=from_agent,
),
)
# Emit LLM call failed event
self._emit_call_failed_event(
error=error_msg,
from_task=from_task,
from_agent=from_agent,
)
return None
def _format_messages(self, messages: str | list[LLMMessage]) -> list[LLMMessage]:
"""Convert messages to standard format.
Args:
messages: Input messages (string or list of message dicts)
Returns:
List of message dictionaries with 'role' and 'content' keys
Raises:
ValueError: If message format is invalid
"""
if isinstance(messages, str):
return [{"role": "user", "content": messages}]
# Validate message format
for i, msg in enumerate(messages):
if not isinstance(msg, dict):
raise ValueError(f"Message at index {i} must be a dictionary")
if "role" not in msg or "content" not in msg:
raise ValueError(
f"Message at index {i} must have 'role' and 'content' keys"
)
return messages
@staticmethod
def _validate_structured_output(
response: str,
response_format: type[BaseModel] | None,
) -> str | BaseModel:
"""Validate and parse structured output.
Args:
response: Raw response string
response_format: Optional Pydantic model for structured output
Returns:
Parsed response (BaseModel instance if response_format provided, otherwise string)
Raises:
ValueError: If structured output validation fails
"""
if response_format is None:
return response
try:
# Try to parse as JSON first
if response.strip().startswith("{") or response.strip().startswith("["):
data = json.loads(response)
return response_format.model_validate(data)
json_match = _JSON_EXTRACTION_PATTERN.search(response)
if json_match:
data = json.loads(json_match.group())
return response_format.model_validate(data)
raise ValueError("No JSON found in response")
except (json.JSONDecodeError, ValueError) as e:
logging.warning(f"Failed to parse structured output: {e}")
raise ValueError(
f"Failed to parse response into {response_format.__name__}: {e}"
) from e
@staticmethod
def _extract_provider(model: str) -> str:
"""Extract provider from model string.
Args:
model: Model string (e.g., 'openai/gpt-4' or 'gpt-4')
Returns:
Provider name (e.g., 'openai')
"""
if "/" in model:
return model.partition("/")[0]
return "openai" # Default provider
def _track_token_usage_internal(self, usage_data: dict[str, Any]) -> None:
"""Track token usage internally in the LLM instance.
Args:
usage_data: Token usage data from the API response
"""
# Extract tokens in a provider-agnostic way
prompt_tokens = (
usage_data.get("prompt_tokens")
or usage_data.get("prompt_token_count")
or usage_data.get("input_tokens")
or 0
)
completion_tokens = (
usage_data.get("completion_tokens")
or usage_data.get("candidates_token_count")
or usage_data.get("output_tokens")
or 0
)
cached_tokens = (
usage_data.get("cached_tokens")
or usage_data.get("cached_prompt_tokens")
or 0
)
self._token_usage["prompt_tokens"] += prompt_tokens
self._token_usage["completion_tokens"] += completion_tokens
self._token_usage["total_tokens"] += prompt_tokens + completion_tokens
self._token_usage["successful_requests"] += 1
self._token_usage["cached_prompt_tokens"] += cached_tokens
def get_token_usage_summary(self) -> UsageMetrics:
"""Get summary of token usage for this LLM instance.
Returns:
Dictionary with token usage totals
"""
return UsageMetrics(**self._token_usage)
from crewai.llm.base_llm import * # noqa: E402, F403

View File

@@ -1,558 +1,15 @@
from typing import Literal, TypeAlias
"""Deprecated: Use crewai.llm.constants instead.
.. deprecated:: 1.4.0
"""
import warnings
OpenAIModels: TypeAlias = Literal[
"gpt-3.5-turbo",
"gpt-3.5-turbo-0125",
"gpt-3.5-turbo-0301",
"gpt-3.5-turbo-0613",
"gpt-3.5-turbo-1106",
"gpt-3.5-turbo-16k",
"gpt-3.5-turbo-16k-0613",
"gpt-3.5-turbo-instruct",
"gpt-3.5-turbo-instruct-0914",
"gpt-4",
"gpt-4-0125-preview",
"gpt-4-0314",
"gpt-4-0613",
"gpt-4-1106-preview",
"gpt-4-32k",
"gpt-4-32k-0314",
"gpt-4-32k-0613",
"gpt-4-turbo",
"gpt-4-turbo-2024-04-09",
"gpt-4-turbo-preview",
"gpt-4-vision-preview",
"gpt-4.1",
"gpt-4.1-2025-04-14",
"gpt-4.1-mini",
"gpt-4.1-mini-2025-04-14",
"gpt-4.1-nano",
"gpt-4.1-nano-2025-04-14",
"gpt-4o",
"gpt-4o-2024-05-13",
"gpt-4o-2024-08-06",
"gpt-4o-2024-11-20",
"gpt-4o-audio-preview",
"gpt-4o-audio-preview-2024-10-01",
"gpt-4o-audio-preview-2024-12-17",
"gpt-4o-audio-preview-2025-06-03",
"gpt-4o-mini",
"gpt-4o-mini-2024-07-18",
"gpt-4o-mini-audio-preview",
"gpt-4o-mini-audio-preview-2024-12-17",
"gpt-4o-mini-realtime-preview",
"gpt-4o-mini-realtime-preview-2024-12-17",
"gpt-4o-mini-search-preview",
"gpt-4o-mini-search-preview-2025-03-11",
"gpt-4o-mini-transcribe",
"gpt-4o-mini-tts",
"gpt-4o-realtime-preview",
"gpt-4o-realtime-preview-2024-10-01",
"gpt-4o-realtime-preview-2024-12-17",
"gpt-4o-realtime-preview-2025-06-03",
"gpt-4o-search-preview",
"gpt-4o-search-preview-2025-03-11",
"gpt-4o-transcribe",
"gpt-4o-transcribe-diarize",
"gpt-5",
"gpt-5-2025-08-07",
"gpt-5-chat",
"gpt-5-chat-latest",
"gpt-5-codex",
"gpt-5-mini",
"gpt-5-mini-2025-08-07",
"gpt-5-nano",
"gpt-5-nano-2025-08-07",
"gpt-5-pro",
"gpt-5-pro-2025-10-06",
"gpt-5-search-api",
"gpt-5-search-api-2025-10-14",
"gpt-audio",
"gpt-audio-2025-08-28",
"gpt-audio-mini",
"gpt-audio-mini-2025-10-06",
"gpt-image-1",
"gpt-image-1-mini",
"gpt-realtime",
"gpt-realtime-2025-08-28",
"gpt-realtime-mini",
"gpt-realtime-mini-2025-10-06",
"o1",
"o1-preview",
"o1-2024-12-17",
"o1-mini",
"o1-mini-2024-09-12",
"o1-pro",
"o1-pro-2025-03-19",
"o3-mini",
"o3",
"o4-mini",
"whisper-1",
]
OPENAI_MODELS: list[OpenAIModels] = [
"gpt-3.5-turbo",
"gpt-3.5-turbo-0125",
"gpt-3.5-turbo-0301",
"gpt-3.5-turbo-0613",
"gpt-3.5-turbo-1106",
"gpt-3.5-turbo-16k",
"gpt-3.5-turbo-16k-0613",
"gpt-3.5-turbo-instruct",
"gpt-3.5-turbo-instruct-0914",
"gpt-4",
"gpt-4-0125-preview",
"gpt-4-0314",
"gpt-4-0613",
"gpt-4-1106-preview",
"gpt-4-32k",
"gpt-4-32k-0314",
"gpt-4-32k-0613",
"gpt-4-turbo",
"gpt-4-turbo-2024-04-09",
"gpt-4-turbo-preview",
"gpt-4-vision-preview",
"gpt-4.1",
"gpt-4.1-2025-04-14",
"gpt-4.1-mini",
"gpt-4.1-mini-2025-04-14",
"gpt-4.1-nano",
"gpt-4.1-nano-2025-04-14",
"gpt-4o",
"gpt-4o-2024-05-13",
"gpt-4o-2024-08-06",
"gpt-4o-2024-11-20",
"gpt-4o-audio-preview",
"gpt-4o-audio-preview-2024-10-01",
"gpt-4o-audio-preview-2024-12-17",
"gpt-4o-audio-preview-2025-06-03",
"gpt-4o-mini",
"gpt-4o-mini-2024-07-18",
"gpt-4o-mini-audio-preview",
"gpt-4o-mini-audio-preview-2024-12-17",
"gpt-4o-mini-realtime-preview",
"gpt-4o-mini-realtime-preview-2024-12-17",
"gpt-4o-mini-search-preview",
"gpt-4o-mini-search-preview-2025-03-11",
"gpt-4o-mini-transcribe",
"gpt-4o-mini-tts",
"gpt-4o-realtime-preview",
"gpt-4o-realtime-preview-2024-10-01",
"gpt-4o-realtime-preview-2024-12-17",
"gpt-4o-realtime-preview-2025-06-03",
"gpt-4o-search-preview",
"gpt-4o-search-preview-2025-03-11",
"gpt-4o-transcribe",
"gpt-4o-transcribe-diarize",
"gpt-5",
"gpt-5-2025-08-07",
"gpt-5-chat",
"gpt-5-chat-latest",
"gpt-5-codex",
"gpt-5-mini",
"gpt-5-mini-2025-08-07",
"gpt-5-nano",
"gpt-5-nano-2025-08-07",
"gpt-5-pro",
"gpt-5-pro-2025-10-06",
"gpt-5-search-api",
"gpt-5-search-api-2025-10-14",
"gpt-audio",
"gpt-audio-2025-08-28",
"gpt-audio-mini",
"gpt-audio-mini-2025-10-06",
"gpt-image-1",
"gpt-image-1-mini",
"gpt-realtime",
"gpt-realtime-2025-08-28",
"gpt-realtime-mini",
"gpt-realtime-mini-2025-10-06",
"o1",
"o1-preview",
"o1-2024-12-17",
"o1-mini",
"o1-mini-2024-09-12",
"o1-pro",
"o1-pro-2025-03-19",
"o3-mini",
"o3",
"o4-mini",
"whisper-1",
]
warnings.warn(
"crewai.llms.constants is deprecated. Use crewai.llm.constants instead.",
DeprecationWarning,
stacklevel=2,
)
AnthropicModels: TypeAlias = Literal[
"claude-3-7-sonnet-latest",
"claude-3-7-sonnet-20250219",
"claude-3-5-haiku-latest",
"claude-3-5-haiku-20241022",
"claude-haiku-4-5",
"claude-haiku-4-5-20251001",
"claude-sonnet-4-20250514",
"claude-sonnet-4-0",
"claude-4-sonnet-20250514",
"claude-sonnet-4-5",
"claude-sonnet-4-5-20250929",
"claude-3-5-sonnet-latest",
"claude-3-5-sonnet-20241022",
"claude-3-5-sonnet-20240620",
"claude-opus-4-0",
"claude-opus-4-20250514",
"claude-4-opus-20250514",
"claude-opus-4-1",
"claude-opus-4-1-20250805",
"claude-3-opus-latest",
"claude-3-opus-20240229",
"claude-3-sonnet-20240229",
"claude-3-haiku-latest",
"claude-3-haiku-20240307",
]
ANTHROPIC_MODELS: list[AnthropicModels] = [
"claude-3-7-sonnet-latest",
"claude-3-7-sonnet-20250219",
"claude-3-5-haiku-latest",
"claude-3-5-haiku-20241022",
"claude-haiku-4-5",
"claude-haiku-4-5-20251001",
"claude-sonnet-4-20250514",
"claude-sonnet-4-0",
"claude-4-sonnet-20250514",
"claude-sonnet-4-5",
"claude-sonnet-4-5-20250929",
"claude-3-5-sonnet-latest",
"claude-3-5-sonnet-20241022",
"claude-3-5-sonnet-20240620",
"claude-opus-4-0",
"claude-opus-4-20250514",
"claude-4-opus-20250514",
"claude-opus-4-1",
"claude-opus-4-1-20250805",
"claude-3-opus-latest",
"claude-3-opus-20240229",
"claude-3-sonnet-20240229",
"claude-3-haiku-latest",
"claude-3-haiku-20240307",
]
GeminiModels: TypeAlias = Literal[
"gemini-2.5-pro",
"gemini-2.5-pro-preview-03-25",
"gemini-2.5-pro-preview-05-06",
"gemini-2.5-pro-preview-06-05",
"gemini-2.5-flash",
"gemini-2.5-flash-preview-05-20",
"gemini-2.5-flash-preview-04-17",
"gemini-2.5-flash-image",
"gemini-2.5-flash-image-preview",
"gemini-2.5-flash-lite",
"gemini-2.5-flash-lite-preview-06-17",
"gemini-2.5-flash-preview-09-2025",
"gemini-2.5-flash-lite-preview-09-2025",
"gemini-2.5-flash-preview-tts",
"gemini-2.5-pro-preview-tts",
"gemini-2.5-computer-use-preview-10-2025",
"gemini-2.0-flash",
"gemini-2.0-flash-001",
"gemini-2.0-flash-exp",
"gemini-2.0-flash-exp-image-generation",
"gemini-2.0-flash-lite",
"gemini-2.0-flash-lite-001",
"gemini-2.0-flash-lite-preview",
"gemini-2.0-flash-lite-preview-02-05",
"gemini-2.0-flash-preview-image-generation",
"gemini-2.0-flash-thinking-exp",
"gemini-2.0-flash-thinking-exp-01-21",
"gemini-2.0-flash-thinking-exp-1219",
"gemini-2.0-pro-exp",
"gemini-2.0-pro-exp-02-05",
"gemini-exp-1206",
"gemini-1.5-pro",
"gemini-1.5-flash",
"gemini-1.5-flash-8b",
"gemini-flash-latest",
"gemini-flash-lite-latest",
"gemini-pro-latest",
"gemini-2.0-flash-live-001",
"gemini-live-2.5-flash-preview",
"gemini-2.5-flash-live-preview",
"gemini-robotics-er-1.5-preview",
"gemini-gemma-2-27b-it",
"gemini-gemma-2-9b-it",
"gemma-3-1b-it",
"gemma-3-4b-it",
"gemma-3-12b-it",
"gemma-3-27b-it",
"gemma-3n-e2b-it",
"gemma-3n-e4b-it",
"learnlm-2.0-flash-experimental",
]
GEMINI_MODELS: list[GeminiModels] = [
"gemini-2.5-pro",
"gemini-2.5-pro-preview-03-25",
"gemini-2.5-pro-preview-05-06",
"gemini-2.5-pro-preview-06-05",
"gemini-2.5-flash",
"gemini-2.5-flash-preview-05-20",
"gemini-2.5-flash-preview-04-17",
"gemini-2.5-flash-image",
"gemini-2.5-flash-image-preview",
"gemini-2.5-flash-lite",
"gemini-2.5-flash-lite-preview-06-17",
"gemini-2.5-flash-preview-09-2025",
"gemini-2.5-flash-lite-preview-09-2025",
"gemini-2.5-flash-preview-tts",
"gemini-2.5-pro-preview-tts",
"gemini-2.5-computer-use-preview-10-2025",
"gemini-2.0-flash",
"gemini-2.0-flash-001",
"gemini-2.0-flash-exp",
"gemini-2.0-flash-exp-image-generation",
"gemini-2.0-flash-lite",
"gemini-2.0-flash-lite-001",
"gemini-2.0-flash-lite-preview",
"gemini-2.0-flash-lite-preview-02-05",
"gemini-2.0-flash-preview-image-generation",
"gemini-2.0-flash-thinking-exp",
"gemini-2.0-flash-thinking-exp-01-21",
"gemini-2.0-flash-thinking-exp-1219",
"gemini-2.0-pro-exp",
"gemini-2.0-pro-exp-02-05",
"gemini-exp-1206",
"gemini-1.5-pro",
"gemini-1.5-flash",
"gemini-1.5-flash-8b",
"gemini-flash-latest",
"gemini-flash-lite-latest",
"gemini-pro-latest",
"gemini-2.0-flash-live-001",
"gemini-live-2.5-flash-preview",
"gemini-2.5-flash-live-preview",
"gemini-robotics-er-1.5-preview",
"gemini-gemma-2-27b-it",
"gemini-gemma-2-9b-it",
"gemma-3-1b-it",
"gemma-3-4b-it",
"gemma-3-12b-it",
"gemma-3-27b-it",
"gemma-3n-e2b-it",
"gemma-3n-e4b-it",
"learnlm-2.0-flash-experimental",
]
AzureModels: TypeAlias = Literal[
"gpt-3.5-turbo",
"gpt-3.5-turbo-0301",
"gpt-3.5-turbo-0613",
"gpt-3.5-turbo-16k",
"gpt-3.5-turbo-16k-0613",
"gpt-35-turbo",
"gpt-35-turbo-0125",
"gpt-35-turbo-1106",
"gpt-35-turbo-16k-0613",
"gpt-35-turbo-instruct-0914",
"gpt-4",
"gpt-4-0314",
"gpt-4-0613",
"gpt-4-1106-preview",
"gpt-4-0125-preview",
"gpt-4-32k",
"gpt-4-32k-0314",
"gpt-4-32k-0613",
"gpt-4-turbo",
"gpt-4-turbo-2024-04-09",
"gpt-4-vision",
"gpt-4o",
"gpt-4o-2024-05-13",
"gpt-4o-2024-08-06",
"gpt-4o-2024-11-20",
"gpt-4o-mini",
"gpt-5",
"o1",
"o1-mini",
"o1-preview",
"o3-mini",
"o3",
"o4-mini",
]
AZURE_MODELS: list[AzureModels] = [
"gpt-3.5-turbo",
"gpt-3.5-turbo-0301",
"gpt-3.5-turbo-0613",
"gpt-3.5-turbo-16k",
"gpt-3.5-turbo-16k-0613",
"gpt-35-turbo",
"gpt-35-turbo-0125",
"gpt-35-turbo-1106",
"gpt-35-turbo-16k-0613",
"gpt-35-turbo-instruct-0914",
"gpt-4",
"gpt-4-0314",
"gpt-4-0613",
"gpt-4-1106-preview",
"gpt-4-0125-preview",
"gpt-4-32k",
"gpt-4-32k-0314",
"gpt-4-32k-0613",
"gpt-4-turbo",
"gpt-4-turbo-2024-04-09",
"gpt-4-vision",
"gpt-4o",
"gpt-4o-2024-05-13",
"gpt-4o-2024-08-06",
"gpt-4o-2024-11-20",
"gpt-4o-mini",
"gpt-5",
"o1",
"o1-mini",
"o1-preview",
"o3-mini",
"o3",
"o4-mini",
]
BedrockModels: TypeAlias = Literal[
"ai21.jamba-1-5-large-v1:0",
"ai21.jamba-1-5-mini-v1:0",
"amazon.nova-lite-v1:0",
"amazon.nova-lite-v1:0:24k",
"amazon.nova-lite-v1:0:300k",
"amazon.nova-micro-v1:0",
"amazon.nova-micro-v1:0:128k",
"amazon.nova-micro-v1:0:24k",
"amazon.nova-premier-v1:0",
"amazon.nova-premier-v1:0:1000k",
"amazon.nova-premier-v1:0:20k",
"amazon.nova-premier-v1:0:8k",
"amazon.nova-premier-v1:0:mm",
"amazon.nova-pro-v1:0",
"amazon.nova-pro-v1:0:24k",
"amazon.nova-pro-v1:0:300k",
"amazon.titan-text-express-v1",
"amazon.titan-text-express-v1:0:8k",
"amazon.titan-text-lite-v1",
"amazon.titan-text-lite-v1:0:4k",
"amazon.titan-tg1-large",
"anthropic.claude-3-5-haiku-20241022-v1:0",
"anthropic.claude-3-5-sonnet-20240620-v1:0",
"anthropic.claude-3-5-sonnet-20241022-v2:0",
"anthropic.claude-3-7-sonnet-20250219-v1:0",
"anthropic.claude-3-haiku-20240307-v1:0",
"anthropic.claude-3-haiku-20240307-v1:0:200k",
"anthropic.claude-3-haiku-20240307-v1:0:48k",
"anthropic.claude-3-opus-20240229-v1:0",
"anthropic.claude-3-opus-20240229-v1:0:12k",
"anthropic.claude-3-opus-20240229-v1:0:200k",
"anthropic.claude-3-opus-20240229-v1:0:28k",
"anthropic.claude-3-sonnet-20240229-v1:0",
"anthropic.claude-3-sonnet-20240229-v1:0:200k",
"anthropic.claude-3-sonnet-20240229-v1:0:28k",
"anthropic.claude-haiku-4-5-20251001-v1:0",
"anthropic.claude-instant-v1:2:100k",
"anthropic.claude-opus-4-1-20250805-v1:0",
"anthropic.claude-opus-4-20250514-v1:0",
"anthropic.claude-sonnet-4-20250514-v1:0",
"anthropic.claude-sonnet-4-5-20250929-v1:0",
"anthropic.claude-v2:0:100k",
"anthropic.claude-v2:0:18k",
"anthropic.claude-v2:1:18k",
"anthropic.claude-v2:1:200k",
"cohere.command-r-plus-v1:0",
"cohere.command-r-v1:0",
"cohere.rerank-v3-5:0",
"deepseek.r1-v1:0",
"meta.llama3-1-70b-instruct-v1:0",
"meta.llama3-1-8b-instruct-v1:0",
"meta.llama3-2-11b-instruct-v1:0",
"meta.llama3-2-1b-instruct-v1:0",
"meta.llama3-2-3b-instruct-v1:0",
"meta.llama3-2-90b-instruct-v1:0",
"meta.llama3-3-70b-instruct-v1:0",
"meta.llama3-70b-instruct-v1:0",
"meta.llama3-8b-instruct-v1:0",
"meta.llama4-maverick-17b-instruct-v1:0",
"meta.llama4-scout-17b-instruct-v1:0",
"mistral.mistral-7b-instruct-v0:2",
"mistral.mistral-large-2402-v1:0",
"mistral.mistral-small-2402-v1:0",
"mistral.mixtral-8x7b-instruct-v0:1",
"mistral.pixtral-large-2502-v1:0",
"openai.gpt-oss-120b-1:0",
"openai.gpt-oss-20b-1:0",
"qwen.qwen3-32b-v1:0",
"qwen.qwen3-coder-30b-a3b-v1:0",
"twelvelabs.pegasus-1-2-v1:0",
]
BEDROCK_MODELS: list[BedrockModels] = [
"ai21.jamba-1-5-large-v1:0",
"ai21.jamba-1-5-mini-v1:0",
"amazon.nova-lite-v1:0",
"amazon.nova-lite-v1:0:24k",
"amazon.nova-lite-v1:0:300k",
"amazon.nova-micro-v1:0",
"amazon.nova-micro-v1:0:128k",
"amazon.nova-micro-v1:0:24k",
"amazon.nova-premier-v1:0",
"amazon.nova-premier-v1:0:1000k",
"amazon.nova-premier-v1:0:20k",
"amazon.nova-premier-v1:0:8k",
"amazon.nova-premier-v1:0:mm",
"amazon.nova-pro-v1:0",
"amazon.nova-pro-v1:0:24k",
"amazon.nova-pro-v1:0:300k",
"amazon.titan-text-express-v1",
"amazon.titan-text-express-v1:0:8k",
"amazon.titan-text-lite-v1",
"amazon.titan-text-lite-v1:0:4k",
"amazon.titan-tg1-large",
"anthropic.claude-3-5-haiku-20241022-v1:0",
"anthropic.claude-3-5-sonnet-20240620-v1:0",
"anthropic.claude-3-5-sonnet-20241022-v2:0",
"anthropic.claude-3-7-sonnet-20250219-v1:0",
"anthropic.claude-3-haiku-20240307-v1:0",
"anthropic.claude-3-haiku-20240307-v1:0:200k",
"anthropic.claude-3-haiku-20240307-v1:0:48k",
"anthropic.claude-3-opus-20240229-v1:0",
"anthropic.claude-3-opus-20240229-v1:0:12k",
"anthropic.claude-3-opus-20240229-v1:0:200k",
"anthropic.claude-3-opus-20240229-v1:0:28k",
"anthropic.claude-3-sonnet-20240229-v1:0",
"anthropic.claude-3-sonnet-20240229-v1:0:200k",
"anthropic.claude-3-sonnet-20240229-v1:0:28k",
"anthropic.claude-haiku-4-5-20251001-v1:0",
"anthropic.claude-instant-v1:2:100k",
"anthropic.claude-opus-4-1-20250805-v1:0",
"anthropic.claude-opus-4-20250514-v1:0",
"anthropic.claude-sonnet-4-20250514-v1:0",
"anthropic.claude-sonnet-4-5-20250929-v1:0",
"anthropic.claude-v2:0:100k",
"anthropic.claude-v2:0:18k",
"anthropic.claude-v2:1:18k",
"anthropic.claude-v2:1:200k",
"cohere.command-r-plus-v1:0",
"cohere.command-r-v1:0",
"cohere.rerank-v3-5:0",
"deepseek.r1-v1:0",
"meta.llama3-1-70b-instruct-v1:0",
"meta.llama3-1-8b-instruct-v1:0",
"meta.llama3-2-11b-instruct-v1:0",
"meta.llama3-2-1b-instruct-v1:0",
"meta.llama3-2-3b-instruct-v1:0",
"meta.llama3-2-90b-instruct-v1:0",
"meta.llama3-3-70b-instruct-v1:0",
"meta.llama3-70b-instruct-v1:0",
"meta.llama3-8b-instruct-v1:0",
"meta.llama4-maverick-17b-instruct-v1:0",
"meta.llama4-scout-17b-instruct-v1:0",
"mistral.mistral-7b-instruct-v0:2",
"mistral.mistral-large-2402-v1:0",
"mistral.mistral-small-2402-v1:0",
"mistral.mixtral-8x7b-instruct-v0:1",
"mistral.pixtral-large-2502-v1:0",
"openai.gpt-oss-120b-1:0",
"openai.gpt-oss-20b-1:0",
"qwen.qwen3-32b-v1:0",
"qwen.qwen3-coder-30b-a3b-v1:0",
"twelvelabs.pegasus-1-2-v1:0",
]
from crewai.llm.constants import * # noqa: E402, F403

Some files were not shown because too many files have changed in this diff Show More