mirror of
https://github.com/crewAIInc/crewAI.git
synced 2025-12-16 04:18:35 +00:00
Lorenze/feat hooks (#3902)
* feat: implement LLM call hooks and enhance agent execution context - Introduced LLM call hooks to allow modification of messages and responses during LLM interactions. - Added support for before and after hooks in the CrewAgentExecutor, enabling dynamic adjustments to the execution flow. - Created LLMCallHookContext for comprehensive access to the executor state, facilitating in-place modifications. - Added validation for hook callables to ensure proper functionality. - Enhanced tests for LLM hooks and tool hooks to verify their behavior and error handling capabilities. - Updated LiteAgent and CrewAgentExecutor to accommodate the new crew context in their execution processes. * feat: implement LLM call hooks and enhance agent execution context - Introduced LLM call hooks to allow modification of messages and responses during LLM interactions. - Added support for before and after hooks in the CrewAgentExecutor, enabling dynamic adjustments to the execution flow. - Created LLMCallHookContext for comprehensive access to the executor state, facilitating in-place modifications. - Added validation for hook callables to ensure proper functionality. - Enhanced tests for LLM hooks and tool hooks to verify their behavior and error handling capabilities. - Updated LiteAgent and CrewAgentExecutor to accommodate the new crew context in their execution processes. * fix verbose * feat: introduce crew-scoped hook decorators and refactor hook registration - Added decorators for before and after LLM and tool calls to enhance flexibility in modifying execution behavior. - Implemented a centralized hook registration mechanism within CrewBase to automatically register crew-scoped hooks. - Removed the obsolete base.py file as its functionality has been integrated into the new decorators and registration system. - Enhanced tests for the new hook decorators to ensure proper registration and execution flow. - Updated existing hook handling to accommodate the new decorator-based approach, improving code organization and maintainability. * feat: enhance hook management with clear and unregister functions - Introduced functions to unregister specific before and after hooks for both LLM and tool calls, improving flexibility in hook management. - Added clear functions to remove all registered hooks of each type, facilitating easier state management and cleanup. - Implemented a convenience function to clear all global hooks in one call, streamlining the process for testing and execution context resets. - Enhanced tests to verify the functionality of unregistering and clearing hooks, ensuring robust behavior in various scenarios. * refactor: enhance hook type management for LLM and tool hooks - Updated hook type definitions to use generic protocols for better type safety and flexibility. - Replaced Callable type annotations with specific BeforeLLMCallHookType and AfterLLMCallHookType for clarity. - Improved the registration and retrieval functions for before and after hooks to align with the new type definitions. - Enhanced the setup functions to handle hook execution results, allowing for blocking of LLM calls based on hook logic. - Updated related tests to ensure proper functionality and type adherence across the hook management system. * feat: add execution and tool hooks documentation - Introduced new documentation for execution hooks, LLM call hooks, and tool call hooks to provide comprehensive guidance on their usage and implementation in CrewAI. - Updated existing documentation to include references to the new hooks, enhancing the learning resources available for users. - Ensured consistency across multiple languages (English, Portuguese, Korean) for the new documentation, improving accessibility for a wider audience. - Added examples and troubleshooting sections to assist users in effectively utilizing hooks for agent operations. --------- Co-authored-by: Greyson LaLonde <greyson.r.lalonde@gmail.com>
This commit is contained in:
Lorenze Jay
522
docs/en/learn/execution-hooks.mdx
Normal file
522
docs/en/learn/execution-hooks.mdx
Normal file
@@ -0,0 +1,522 @@
|
||||
---
|
||||
title: Execution Hooks Overview
|
||||
description: Understanding and using execution hooks in CrewAI for fine-grained control over agent operations
|
||||
mode: "wide"
|
||||
---
|
||||
|
||||
Execution Hooks provide fine-grained control over the runtime behavior of your CrewAI agents. Unlike kickoff hooks that run before and after crew execution, execution hooks intercept specific operations during agent execution, allowing you to modify behavior, implement safety checks, and add comprehensive monitoring.
|
||||
|
||||
## Types of Execution Hooks
|
||||
|
||||
CrewAI provides two main categories of execution hooks:
|
||||
|
||||
### 1. [LLM Call Hooks](/learn/llm-hooks)
|
||||
|
||||
Control and monitor language model interactions:
|
||||
- **Before LLM Call**: Modify prompts, validate inputs, implement approval gates
|
||||
- **After LLM Call**: Transform responses, sanitize outputs, update conversation history
|
||||
|
||||
**Use Cases:**
|
||||
- Iteration limiting
|
||||
- Cost tracking and token usage monitoring
|
||||
- Response sanitization and content filtering
|
||||
- Human-in-the-loop approval for LLM calls
|
||||
- Adding safety guidelines or context
|
||||
- Debug logging and request/response inspection
|
||||
|
||||
[View LLM Hooks Documentation →](/learn/llm-hooks)
|
||||
|
||||
### 2. [Tool Call Hooks](/learn/tool-hooks)
|
||||
|
||||
Control and monitor tool execution:
|
||||
- **Before Tool Call**: Modify inputs, validate parameters, block dangerous operations
|
||||
- **After Tool Call**: Transform results, sanitize outputs, log execution details
|
||||
|
||||
**Use Cases:**
|
||||
- Safety guardrails for destructive operations
|
||||
- Human approval for sensitive actions
|
||||
- Input validation and sanitization
|
||||
- Result caching and rate limiting
|
||||
- Tool usage analytics
|
||||
- Debug logging and monitoring
|
||||
|
||||
[View Tool Hooks Documentation →](/learn/tool-hooks)
|
||||
|
||||
## Hook Registration Methods
|
||||
|
||||
### 1. Decorator-Based Hooks (Recommended)
|
||||
|
||||
The cleanest and most Pythonic way to register hooks:
|
||||
|
||||
```python
|
||||
from crewai.hooks import before_llm_call, after_llm_call, before_tool_call, after_tool_call
|
||||
|
||||
@before_llm_call
|
||||
def limit_iterations(context):
|
||||
"""Prevent infinite loops by limiting iterations."""
|
||||
if context.iterations > 10:
|
||||
return False # Block execution
|
||||
return None
|
||||
|
||||
@after_llm_call
|
||||
def sanitize_response(context):
|
||||
"""Remove sensitive data from LLM responses."""
|
||||
if "API_KEY" in context.response:
|
||||
return context.response.replace("API_KEY", "[REDACTED]")
|
||||
return None
|
||||
|
||||
@before_tool_call
|
||||
def block_dangerous_tools(context):
|
||||
"""Block destructive operations."""
|
||||
if context.tool_name == "delete_database":
|
||||
return False # Block execution
|
||||
return None
|
||||
|
||||
@after_tool_call
|
||||
def log_tool_result(context):
|
||||
"""Log tool execution."""
|
||||
print(f"Tool {context.tool_name} completed")
|
||||
return None
|
||||
```
|
||||
|
||||
### 2. Crew-Scoped Hooks
|
||||
|
||||
Apply hooks only to specific crew instances:
|
||||
|
||||
```python
|
||||
from crewai import CrewBase
|
||||
from crewai.project import crew
|
||||
from crewai.hooks import before_llm_call_crew, after_tool_call_crew
|
||||
|
||||
@CrewBase
|
||||
class MyProjCrew:
|
||||
@before_llm_call_crew
|
||||
def validate_inputs(self, context):
|
||||
# Only applies to this crew
|
||||
print(f"LLM call in {self.__class__.__name__}")
|
||||
return None
|
||||
|
||||
@after_tool_call_crew
|
||||
def log_results(self, context):
|
||||
# Crew-specific logging
|
||||
print(f"Tool result: {context.tool_result[:50]}...")
|
||||
return None
|
||||
|
||||
@crew
|
||||
def crew(self) -> Crew:
|
||||
return Crew(
|
||||
agents=self.agents,
|
||||
tasks=self.tasks,
|
||||
process=Process.sequential
|
||||
)
|
||||
```
|
||||
|
||||
## Hook Execution Flow
|
||||
|
||||
### LLM Call Flow
|
||||
|
||||
```
|
||||
Agent needs to call LLM
|
||||
↓
|
||||
[Before LLM Call Hooks Execute]
|
||||
├→ Hook 1: Validate iteration count
|
||||
├→ Hook 2: Add safety context
|
||||
└→ Hook 3: Log request
|
||||
↓
|
||||
If any hook returns False:
|
||||
├→ Block LLM call
|
||||
└→ Raise ValueError
|
||||
↓
|
||||
If all hooks return True/None:
|
||||
├→ LLM call proceeds
|
||||
└→ Response generated
|
||||
↓
|
||||
[After LLM Call Hooks Execute]
|
||||
├→ Hook 1: Sanitize response
|
||||
├→ Hook 2: Log response
|
||||
└→ Hook 3: Update metrics
|
||||
↓
|
||||
Final response returned
|
||||
```
|
||||
|
||||
### Tool Call Flow
|
||||
|
||||
```
|
||||
Agent needs to execute tool
|
||||
↓
|
||||
[Before Tool Call Hooks Execute]
|
||||
├→ Hook 1: Check if tool is allowed
|
||||
├→ Hook 2: Validate inputs
|
||||
└→ Hook 3: Request approval if needed
|
||||
↓
|
||||
If any hook returns False:
|
||||
├→ Block tool execution
|
||||
└→ Return error message
|
||||
↓
|
||||
If all hooks return True/None:
|
||||
├→ Tool execution proceeds
|
||||
└→ Result generated
|
||||
↓
|
||||
[After Tool Call Hooks Execute]
|
||||
├→ Hook 1: Sanitize result
|
||||
├→ Hook 2: Cache result
|
||||
└→ Hook 3: Log metrics
|
||||
↓
|
||||
Final result returned
|
||||
```
|
||||
|
||||
## Hook Context Objects
|
||||
|
||||
### LLMCallHookContext
|
||||
|
||||
Provides access to LLM execution state:
|
||||
|
||||
```python
|
||||
class LLMCallHookContext:
|
||||
executor: CrewAgentExecutor # Full executor access
|
||||
messages: list # Mutable message list
|
||||
agent: Agent # Current agent
|
||||
task: Task # Current task
|
||||
crew: Crew # Crew instance
|
||||
llm: BaseLLM # LLM instance
|
||||
iterations: int # Current iteration
|
||||
response: str | None # LLM response (after hooks)
|
||||
```
|
||||
|
||||
### ToolCallHookContext
|
||||
|
||||
Provides access to tool execution state:
|
||||
|
||||
```python
|
||||
class ToolCallHookContext:
|
||||
tool_name: str # Tool being called
|
||||
tool_input: dict # Mutable input parameters
|
||||
tool: CrewStructuredTool # Tool instance
|
||||
agent: Agent | None # Agent executing
|
||||
task: Task | None # Current task
|
||||
crew: Crew | None # Crew instance
|
||||
tool_result: str | None # Tool result (after hooks)
|
||||
```
|
||||
|
||||
## Common Patterns
|
||||
|
||||
### Safety and Validation
|
||||
|
||||
```python
|
||||
@before_tool_call
|
||||
def safety_check(context):
|
||||
"""Block destructive operations."""
|
||||
dangerous = ['delete_file', 'drop_table', 'system_shutdown']
|
||||
if context.tool_name in dangerous:
|
||||
print(f"🛑 Blocked: {context.tool_name}")
|
||||
return False
|
||||
return None
|
||||
|
||||
@before_llm_call
|
||||
def iteration_limit(context):
|
||||
"""Prevent infinite loops."""
|
||||
if context.iterations > 15:
|
||||
print("⛔ Maximum iterations exceeded")
|
||||
return False
|
||||
return None
|
||||
```
|
||||
|
||||
### Human-in-the-Loop
|
||||
|
||||
```python
|
||||
@before_tool_call
|
||||
def require_approval(context):
|
||||
"""Require approval for sensitive operations."""
|
||||
sensitive = ['send_email', 'make_payment', 'post_message']
|
||||
|
||||
if context.tool_name in sensitive:
|
||||
response = context.request_human_input(
|
||||
prompt=f"Approve {context.tool_name}?",
|
||||
default_message="Type 'yes' to approve:"
|
||||
)
|
||||
|
||||
if response.lower() != 'yes':
|
||||
return False
|
||||
|
||||
return None
|
||||
```
|
||||
|
||||
### Monitoring and Analytics
|
||||
|
||||
```python
|
||||
from collections import defaultdict
|
||||
import time
|
||||
|
||||
metrics = defaultdict(lambda: {'count': 0, 'total_time': 0})
|
||||
|
||||
@before_tool_call
|
||||
def start_timer(context):
|
||||
context.tool_input['_start'] = time.time()
|
||||
return None
|
||||
|
||||
@after_tool_call
|
||||
def track_metrics(context):
|
||||
start = context.tool_input.get('_start', time.time())
|
||||
duration = time.time() - start
|
||||
|
||||
metrics[context.tool_name]['count'] += 1
|
||||
metrics[context.tool_name]['total_time'] += duration
|
||||
|
||||
return None
|
||||
|
||||
# View metrics
|
||||
def print_metrics():
|
||||
for tool, data in metrics.items():
|
||||
avg = data['total_time'] / data['count']
|
||||
print(f"{tool}: {data['count']} calls, {avg:.2f}s avg")
|
||||
```
|
||||
|
||||
### Response Sanitization
|
||||
|
||||
```python
|
||||
import re
|
||||
|
||||
@after_llm_call
|
||||
def sanitize_llm_response(context):
|
||||
"""Remove sensitive data from LLM responses."""
|
||||
if not context.response:
|
||||
return None
|
||||
|
||||
result = context.response
|
||||
result = re.sub(r'(api[_-]?key)["\']?\s*[:=]\s*["\']?[\w-]+',
|
||||
r'\1: [REDACTED]', result, flags=re.IGNORECASE)
|
||||
return result
|
||||
|
||||
@after_tool_call
|
||||
def sanitize_tool_result(context):
|
||||
"""Remove sensitive data from tool results."""
|
||||
if not context.tool_result:
|
||||
return None
|
||||
|
||||
result = context.tool_result
|
||||
result = re.sub(r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b',
|
||||
'[EMAIL-REDACTED]', result)
|
||||
return result
|
||||
```
|
||||
|
||||
## Hook Management
|
||||
|
||||
### Clearing All Hooks
|
||||
|
||||
```python
|
||||
from crewai.hooks import clear_all_global_hooks
|
||||
|
||||
# Clear all hooks at once
|
||||
result = clear_all_global_hooks()
|
||||
print(f"Cleared {result['total']} hooks")
|
||||
# Output: {'llm_hooks': (2, 1), 'tool_hooks': (1, 2), 'total': (3, 3)}
|
||||
```
|
||||
|
||||
### Clearing Specific Hook Types
|
||||
|
||||
```python
|
||||
from crewai.hooks import (
|
||||
clear_before_llm_call_hooks,
|
||||
clear_after_llm_call_hooks,
|
||||
clear_before_tool_call_hooks,
|
||||
clear_after_tool_call_hooks
|
||||
)
|
||||
|
||||
# Clear specific types
|
||||
llm_before_count = clear_before_llm_call_hooks()
|
||||
tool_after_count = clear_after_tool_call_hooks()
|
||||
```
|
||||
|
||||
### Unregistering Individual Hooks
|
||||
|
||||
```python
|
||||
from crewai.hooks import (
|
||||
unregister_before_llm_call_hook,
|
||||
unregister_after_tool_call_hook
|
||||
)
|
||||
|
||||
def my_hook(context):
|
||||
...
|
||||
|
||||
# Register
|
||||
register_before_llm_call_hook(my_hook)
|
||||
|
||||
# Later, unregister
|
||||
success = unregister_before_llm_call_hook(my_hook)
|
||||
print(f"Unregistered: {success}")
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
### 1. Keep Hooks Focused
|
||||
Each hook should have a single, clear responsibility:
|
||||
|
||||
```python
|
||||
# ✅ Good - focused responsibility
|
||||
@before_tool_call
|
||||
def validate_file_path(context):
|
||||
if context.tool_name == 'read_file':
|
||||
if '..' in context.tool_input.get('path', ''):
|
||||
return False
|
||||
return None
|
||||
|
||||
# ❌ Bad - too many responsibilities
|
||||
@before_tool_call
|
||||
def do_everything(context):
|
||||
# Validation + logging + metrics + approval...
|
||||
...
|
||||
```
|
||||
|
||||
### 2. Handle Errors Gracefully
|
||||
|
||||
```python
|
||||
@before_llm_call
|
||||
def safe_hook(context):
|
||||
try:
|
||||
# Your logic
|
||||
if some_condition:
|
||||
return False
|
||||
except Exception as e:
|
||||
print(f"Hook error: {e}")
|
||||
return None # Allow execution despite error
|
||||
```
|
||||
|
||||
### 3. Modify Context In-Place
|
||||
|
||||
```python
|
||||
# ✅ Correct - modify in-place
|
||||
@before_llm_call
|
||||
def add_context(context):
|
||||
context.messages.append({"role": "system", "content": "Be concise"})
|
||||
|
||||
# ❌ Wrong - replaces reference
|
||||
@before_llm_call
|
||||
def wrong_approach(context):
|
||||
context.messages = [{"role": "system", "content": "Be concise"}]
|
||||
```
|
||||
|
||||
### 4. Use Type Hints
|
||||
|
||||
```python
|
||||
from crewai.hooks import LLMCallHookContext, ToolCallHookContext
|
||||
|
||||
def my_llm_hook(context: LLMCallHookContext) -> bool | None:
|
||||
# IDE autocomplete and type checking
|
||||
return None
|
||||
|
||||
def my_tool_hook(context: ToolCallHookContext) -> str | None:
|
||||
return None
|
||||
```
|
||||
|
||||
### 5. Clean Up in Tests
|
||||
|
||||
```python
|
||||
import pytest
|
||||
from crewai.hooks import clear_all_global_hooks
|
||||
|
||||
@pytest.fixture(autouse=True)
|
||||
def clean_hooks():
|
||||
"""Reset hooks before each test."""
|
||||
yield
|
||||
clear_all_global_hooks()
|
||||
```
|
||||
|
||||
## When to Use Which Hook
|
||||
|
||||
### Use LLM Hooks When:
|
||||
- Implementing iteration limits
|
||||
- Adding context or safety guidelines to prompts
|
||||
- Tracking token usage and costs
|
||||
- Sanitizing or transforming responses
|
||||
- Implementing approval gates for LLM calls
|
||||
- Debugging prompt/response interactions
|
||||
|
||||
### Use Tool Hooks When:
|
||||
- Blocking dangerous or destructive operations
|
||||
- Validating tool inputs before execution
|
||||
- Implementing approval gates for sensitive actions
|
||||
- Caching tool results
|
||||
- Tracking tool usage and performance
|
||||
- Sanitizing tool outputs
|
||||
- Rate limiting tool calls
|
||||
|
||||
### Use Both When:
|
||||
Building comprehensive observability, safety, or approval systems that need to monitor all agent operations.
|
||||
|
||||
## Alternative Registration Methods
|
||||
|
||||
### Programmatic Registration (Advanced)
|
||||
|
||||
For dynamic hook registration or when you need to register hooks programmatically:
|
||||
|
||||
```python
|
||||
from crewai.hooks import (
|
||||
register_before_llm_call_hook,
|
||||
register_after_tool_call_hook
|
||||
)
|
||||
|
||||
def my_hook(context):
|
||||
return None
|
||||
|
||||
# Register programmatically
|
||||
register_before_llm_call_hook(my_hook)
|
||||
|
||||
# Useful for:
|
||||
# - Loading hooks from configuration
|
||||
# - Conditional hook registration
|
||||
# - Plugin systems
|
||||
```
|
||||
|
||||
**Note:** For most use cases, decorators are cleaner and more maintainable.
|
||||
|
||||
## Performance Considerations
|
||||
|
||||
1. **Keep Hooks Fast**: Hooks execute on every call - avoid heavy computation
|
||||
2. **Cache When Possible**: Store expensive validations or lookups
|
||||
3. **Be Selective**: Use crew-scoped hooks when global hooks aren't needed
|
||||
4. **Monitor Hook Overhead**: Profile hook execution time in production
|
||||
5. **Lazy Import**: Import heavy dependencies only when needed
|
||||
|
||||
## Debugging Hooks
|
||||
|
||||
### Enable Debug Logging
|
||||
|
||||
```python
|
||||
import logging
|
||||
|
||||
logging.basicConfig(level=logging.DEBUG)
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@before_llm_call
|
||||
def debug_hook(context):
|
||||
logger.debug(f"LLM call: {context.agent.role}, iteration {context.iterations}")
|
||||
return None
|
||||
```
|
||||
|
||||
### Hook Execution Order
|
||||
|
||||
Hooks execute in registration order. If a before hook returns `False`, subsequent hooks don't execute:
|
||||
|
||||
```python
|
||||
# Register order matters!
|
||||
register_before_tool_call_hook(hook1) # Executes first
|
||||
register_before_tool_call_hook(hook2) # Executes second
|
||||
register_before_tool_call_hook(hook3) # Executes third
|
||||
|
||||
# If hook2 returns False:
|
||||
# - hook1 executed
|
||||
# - hook2 executed and returned False
|
||||
# - hook3 NOT executed
|
||||
# - Tool call blocked
|
||||
```
|
||||
|
||||
## Related Documentation
|
||||
|
||||
- [LLM Call Hooks →](/learn/llm-hooks) - Detailed LLM hook documentation
|
||||
- [Tool Call Hooks →](/learn/tool-hooks) - Detailed tool hook documentation
|
||||
- [Before and After Kickoff Hooks →](/learn/before-and-after-kickoff-hooks) - Crew lifecycle hooks
|
||||
- [Human-in-the-Loop →](/learn/human-in-the-loop) - Human input patterns
|
||||
|
||||
## Conclusion
|
||||
|
||||
Execution hooks provide powerful control over agent runtime behavior. Use them to implement safety guardrails, approval workflows, comprehensive monitoring, and custom business logic. Combined with proper error handling, type safety, and performance considerations, hooks enable production-ready, secure, and observable agent systems.
|
||||
427
docs/en/learn/llm-hooks.mdx
Normal file
427
docs/en/learn/llm-hooks.mdx
Normal file
@@ -0,0 +1,427 @@
|
||||
---
|
||||
title: LLM Call Hooks
|
||||
description: Learn how to use LLM call hooks to intercept, modify, and control language model interactions in CrewAI
|
||||
mode: "wide"
|
||||
---
|
||||
|
||||
LLM Call Hooks provide fine-grained control over language model interactions during agent execution. These hooks allow you to intercept LLM calls, modify prompts, transform responses, implement approval gates, and add custom logging or monitoring.
|
||||
|
||||
## Overview
|
||||
|
||||
LLM hooks are executed at two critical points:
|
||||
- **Before LLM Call**: Modify messages, validate inputs, or block execution
|
||||
- **After LLM Call**: Transform responses, sanitize outputs, or modify conversation history
|
||||
|
||||
## Hook Types
|
||||
|
||||
### Before LLM Call Hooks
|
||||
|
||||
Executed before every LLM call, these hooks can:
|
||||
- Inspect and modify messages sent to the LLM
|
||||
- Block LLM execution based on conditions
|
||||
- Implement rate limiting or approval gates
|
||||
- Add context or system messages
|
||||
- Log request details
|
||||
|
||||
**Signature:**
|
||||
```python
|
||||
def before_hook(context: LLMCallHookContext) -> bool | None:
|
||||
# Return False to block execution
|
||||
# Return True or None to allow execution
|
||||
...
|
||||
```
|
||||
|
||||
### After LLM Call Hooks
|
||||
|
||||
Executed after every LLM call, these hooks can:
|
||||
- Modify or sanitize LLM responses
|
||||
- Add metadata or formatting
|
||||
- Log response details
|
||||
- Update conversation history
|
||||
- Implement content filtering
|
||||
|
||||
**Signature:**
|
||||
```python
|
||||
def after_hook(context: LLMCallHookContext) -> str | None:
|
||||
# Return modified response string
|
||||
# Return None to keep original response
|
||||
...
|
||||
```
|
||||
|
||||
## LLM Hook Context
|
||||
|
||||
The `LLMCallHookContext` object provides comprehensive access to execution state:
|
||||
|
||||
```python
|
||||
class LLMCallHookContext:
|
||||
executor: CrewAgentExecutor # Full executor reference
|
||||
messages: list # Mutable message list
|
||||
agent: Agent # Current agent
|
||||
task: Task # Current task
|
||||
crew: Crew # Crew instance
|
||||
llm: BaseLLM # LLM instance
|
||||
iterations: int # Current iteration count
|
||||
response: str | None # LLM response (after hooks only)
|
||||
```
|
||||
|
||||
### Modifying Messages
|
||||
|
||||
**Important:** Always modify messages in-place:
|
||||
|
||||
```python
|
||||
# ✅ Correct - modify in-place
|
||||
def add_context(context: LLMCallHookContext) -> None:
|
||||
context.messages.append({"role": "system", "content": "Be concise"})
|
||||
|
||||
# ❌ Wrong - replaces list reference
|
||||
def wrong_approach(context: LLMCallHookContext) -> None:
|
||||
context.messages = [{"role": "system", "content": "Be concise"}]
|
||||
```
|
||||
|
||||
## Registration Methods
|
||||
|
||||
### 1. Global Hook Registration
|
||||
|
||||
Register hooks that apply to all LLM calls across all crews:
|
||||
|
||||
```python
|
||||
from crewai.hooks import register_before_llm_call_hook, register_after_llm_call_hook
|
||||
|
||||
def log_llm_call(context):
|
||||
print(f"LLM call by {context.agent.role} at iteration {context.iterations}")
|
||||
return None # Allow execution
|
||||
|
||||
register_before_llm_call_hook(log_llm_call)
|
||||
```
|
||||
|
||||
### 2. Decorator-Based Registration
|
||||
|
||||
Use decorators for cleaner syntax:
|
||||
|
||||
```python
|
||||
from crewai.hooks import before_llm_call, after_llm_call
|
||||
|
||||
@before_llm_call
|
||||
def validate_iteration_count(context):
|
||||
if context.iterations > 10:
|
||||
print("⚠️ Exceeded maximum iterations")
|
||||
return False # Block execution
|
||||
return None
|
||||
|
||||
@after_llm_call
|
||||
def sanitize_response(context):
|
||||
if context.response and "API_KEY" in context.response:
|
||||
return context.response.replace("API_KEY", "[REDACTED]")
|
||||
return None
|
||||
```
|
||||
|
||||
### 3. Crew-Scoped Hooks
|
||||
|
||||
Register hooks for a specific crew instance:
|
||||
|
||||
```python
|
||||
@CrewBase
|
||||
class MyProjCrew:
|
||||
@before_llm_call_crew
|
||||
def validate_inputs(self, context):
|
||||
# Only applies to this crew
|
||||
if context.iterations == 0:
|
||||
print(f"Starting task: {context.task.description}")
|
||||
return None
|
||||
|
||||
@after_llm_call_crew
|
||||
def log_responses(self, context):
|
||||
# Crew-specific response logging
|
||||
print(f"Response length: {len(context.response)}")
|
||||
return None
|
||||
|
||||
@crew
|
||||
def crew(self) -> Crew:
|
||||
return Crew(
|
||||
agents=self.agents,
|
||||
tasks=self.tasks,
|
||||
process=Process.sequential,
|
||||
verbose=True
|
||||
)
|
||||
```
|
||||
|
||||
## Common Use Cases
|
||||
|
||||
### 1. Iteration Limiting
|
||||
|
||||
```python
|
||||
@before_llm_call
|
||||
def limit_iterations(context: LLMCallHookContext) -> bool | None:
|
||||
max_iterations = 15
|
||||
if context.iterations > max_iterations:
|
||||
print(f"⛔ Blocked: Exceeded {max_iterations} iterations")
|
||||
return False # Block execution
|
||||
return None
|
||||
```
|
||||
|
||||
### 2. Human Approval Gate
|
||||
|
||||
```python
|
||||
@before_llm_call
|
||||
def require_approval(context: LLMCallHookContext) -> bool | None:
|
||||
if context.iterations > 5:
|
||||
response = context.request_human_input(
|
||||
prompt=f"Iteration {context.iterations}: Approve LLM call?",
|
||||
default_message="Press Enter to approve, or type 'no' to block:"
|
||||
)
|
||||
if response.lower() == "no":
|
||||
print("🚫 LLM call blocked by user")
|
||||
return False
|
||||
return None
|
||||
```
|
||||
|
||||
### 3. Adding System Context
|
||||
|
||||
```python
|
||||
@before_llm_call
|
||||
def add_guardrails(context: LLMCallHookContext) -> None:
|
||||
# Add safety guidelines to every LLM call
|
||||
context.messages.append({
|
||||
"role": "system",
|
||||
"content": "Ensure responses are factual and cite sources when possible."
|
||||
})
|
||||
return None
|
||||
```
|
||||
|
||||
### 4. Response Sanitization
|
||||
|
||||
```python
|
||||
@after_llm_call
|
||||
def sanitize_sensitive_data(context: LLMCallHookContext) -> str | None:
|
||||
if not context.response:
|
||||
return None
|
||||
|
||||
# Remove sensitive patterns
|
||||
import re
|
||||
sanitized = context.response
|
||||
sanitized = re.sub(r'\b\d{3}-\d{2}-\d{4}\b', '[SSN-REDACTED]', sanitized)
|
||||
sanitized = re.sub(r'\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b', '[CARD-REDACTED]', sanitized)
|
||||
|
||||
return sanitized
|
||||
```
|
||||
|
||||
### 5. Cost Tracking
|
||||
|
||||
```python
|
||||
import tiktoken
|
||||
|
||||
@before_llm_call
|
||||
def track_token_usage(context: LLMCallHookContext) -> None:
|
||||
encoding = tiktoken.get_encoding("cl100k_base")
|
||||
total_tokens = sum(
|
||||
len(encoding.encode(msg.get("content", "")))
|
||||
for msg in context.messages
|
||||
)
|
||||
print(f"📊 Input tokens: ~{total_tokens}")
|
||||
return None
|
||||
|
||||
@after_llm_call
|
||||
def track_response_tokens(context: LLMCallHookContext) -> None:
|
||||
if context.response:
|
||||
encoding = tiktoken.get_encoding("cl100k_base")
|
||||
tokens = len(encoding.encode(context.response))
|
||||
print(f"📊 Response tokens: ~{tokens}")
|
||||
return None
|
||||
```
|
||||
|
||||
### 6. Debug Logging
|
||||
|
||||
```python
|
||||
@before_llm_call
|
||||
def debug_request(context: LLMCallHookContext) -> None:
|
||||
print(f"""
|
||||
🔍 LLM Call Debug:
|
||||
- Agent: {context.agent.role}
|
||||
- Task: {context.task.description[:50]}...
|
||||
- Iteration: {context.iterations}
|
||||
- Message Count: {len(context.messages)}
|
||||
- Last Message: {context.messages[-1] if context.messages else 'None'}
|
||||
""")
|
||||
return None
|
||||
|
||||
@after_llm_call
|
||||
def debug_response(context: LLMCallHookContext) -> None:
|
||||
if context.response:
|
||||
print(f"✅ Response Preview: {context.response[:100]}...")
|
||||
return None
|
||||
```
|
||||
|
||||
## Hook Management
|
||||
|
||||
### Unregistering Hooks
|
||||
|
||||
```python
|
||||
from crewai.hooks import (
|
||||
unregister_before_llm_call_hook,
|
||||
unregister_after_llm_call_hook
|
||||
)
|
||||
|
||||
# Unregister specific hook
|
||||
def my_hook(context):
|
||||
...
|
||||
|
||||
register_before_llm_call_hook(my_hook)
|
||||
# Later...
|
||||
unregister_before_llm_call_hook(my_hook) # Returns True if found
|
||||
```
|
||||
|
||||
### Clearing Hooks
|
||||
|
||||
```python
|
||||
from crewai.hooks import (
|
||||
clear_before_llm_call_hooks,
|
||||
clear_after_llm_call_hooks,
|
||||
clear_all_llm_call_hooks
|
||||
)
|
||||
|
||||
# Clear specific hook type
|
||||
count = clear_before_llm_call_hooks()
|
||||
print(f"Cleared {count} before hooks")
|
||||
|
||||
# Clear all LLM hooks
|
||||
before_count, after_count = clear_all_llm_call_hooks()
|
||||
print(f"Cleared {before_count} before and {after_count} after hooks")
|
||||
```
|
||||
|
||||
### Listing Registered Hooks
|
||||
|
||||
```python
|
||||
from crewai.hooks import (
|
||||
get_before_llm_call_hooks,
|
||||
get_after_llm_call_hooks
|
||||
)
|
||||
|
||||
# Get current hooks
|
||||
before_hooks = get_before_llm_call_hooks()
|
||||
after_hooks = get_after_llm_call_hooks()
|
||||
|
||||
print(f"Registered: {len(before_hooks)} before, {len(after_hooks)} after")
|
||||
```
|
||||
|
||||
## Advanced Patterns
|
||||
|
||||
### Conditional Hook Execution
|
||||
|
||||
```python
|
||||
@before_llm_call
|
||||
def conditional_blocking(context: LLMCallHookContext) -> bool | None:
|
||||
# Only block for specific agents
|
||||
if context.agent.role == "researcher" and context.iterations > 10:
|
||||
return False
|
||||
|
||||
# Only block for specific tasks
|
||||
if "sensitive" in context.task.description.lower() and context.iterations > 5:
|
||||
return False
|
||||
|
||||
return None
|
||||
```
|
||||
|
||||
### Context-Aware Modifications
|
||||
|
||||
```python
|
||||
@before_llm_call
|
||||
def adaptive_prompting(context: LLMCallHookContext) -> None:
|
||||
# Add different context based on iteration
|
||||
if context.iterations == 0:
|
||||
context.messages.append({
|
||||
"role": "system",
|
||||
"content": "Start with a high-level overview."
|
||||
})
|
||||
elif context.iterations > 3:
|
||||
context.messages.append({
|
||||
"role": "system",
|
||||
"content": "Focus on specific details and provide examples."
|
||||
})
|
||||
return None
|
||||
```
|
||||
|
||||
### Chaining Hooks
|
||||
|
||||
```python
|
||||
# Multiple hooks execute in registration order
|
||||
|
||||
@before_llm_call
|
||||
def first_hook(context):
|
||||
print("1. First hook executed")
|
||||
return None
|
||||
|
||||
@before_llm_call
|
||||
def second_hook(context):
|
||||
print("2. Second hook executed")
|
||||
return None
|
||||
|
||||
@before_llm_call
|
||||
def blocking_hook(context):
|
||||
if context.iterations > 10:
|
||||
print("3. Blocking hook - execution stopped")
|
||||
return False # Subsequent hooks won't execute
|
||||
print("3. Blocking hook - execution allowed")
|
||||
return None
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Keep Hooks Focused**: Each hook should have a single responsibility
|
||||
2. **Avoid Heavy Computation**: Hooks execute on every LLM call
|
||||
3. **Handle Errors Gracefully**: Use try-except to prevent hook failures from breaking execution
|
||||
4. **Use Type Hints**: Leverage `LLMCallHookContext` for better IDE support
|
||||
5. **Document Hook Behavior**: Especially for blocking conditions
|
||||
6. **Test Hooks Independently**: Unit test hooks before using in production
|
||||
7. **Clear Hooks in Tests**: Use `clear_all_llm_call_hooks()` between test runs
|
||||
8. **Modify In-Place**: Always modify `context.messages` in-place, never replace
|
||||
|
||||
## Error Handling
|
||||
|
||||
```python
|
||||
@before_llm_call
|
||||
def safe_hook(context: LLMCallHookContext) -> bool | None:
|
||||
try:
|
||||
# Your hook logic
|
||||
if some_condition:
|
||||
return False
|
||||
except Exception as e:
|
||||
print(f"⚠️ Hook error: {e}")
|
||||
# Decide: allow or block on error
|
||||
return None # Allow execution despite error
|
||||
```
|
||||
|
||||
## Type Safety
|
||||
|
||||
```python
|
||||
from crewai.hooks import LLMCallHookContext, BeforeLLMCallHookType, AfterLLMCallHookType
|
||||
|
||||
# Explicit type annotations
|
||||
def my_before_hook(context: LLMCallHookContext) -> bool | None:
|
||||
return None
|
||||
|
||||
def my_after_hook(context: LLMCallHookContext) -> str | None:
|
||||
return None
|
||||
|
||||
# Type-safe registration
|
||||
register_before_llm_call_hook(my_before_hook)
|
||||
register_after_llm_call_hook(my_after_hook)
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Hook Not Executing
|
||||
- Verify hook is registered before crew execution
|
||||
- Check if previous hook returned `False` (blocks subsequent hooks)
|
||||
- Ensure hook signature matches expected type
|
||||
|
||||
### Message Modifications Not Persisting
|
||||
- Use in-place modifications: `context.messages.append()`
|
||||
- Don't replace the list: `context.messages = []`
|
||||
|
||||
### Response Modifications Not Working
|
||||
- Return the modified string from after hooks
|
||||
- Returning `None` keeps the original response
|
||||
|
||||
## Conclusion
|
||||
|
||||
LLM Call Hooks provide powerful capabilities for controlling and monitoring language model interactions in CrewAI. Use them to implement safety guardrails, approval gates, logging, cost tracking, and response sanitization. Combined with proper error handling and type safety, hooks enable robust and production-ready agent systems.
|
||||
600
docs/en/learn/tool-hooks.mdx
Normal file
600
docs/en/learn/tool-hooks.mdx
Normal file
@@ -0,0 +1,600 @@
|
||||
---
|
||||
title: Tool Call Hooks
|
||||
description: Learn how to use tool call hooks to intercept, modify, and control tool execution in CrewAI
|
||||
mode: "wide"
|
||||
---
|
||||
|
||||
Tool Call Hooks provide fine-grained control over tool execution during agent operations. These hooks allow you to intercept tool calls, modify inputs, transform outputs, implement safety checks, and add comprehensive logging or monitoring.
|
||||
|
||||
## Overview
|
||||
|
||||
Tool hooks are executed at two critical points:
|
||||
- **Before Tool Call**: Modify inputs, validate parameters, or block execution
|
||||
- **After Tool Call**: Transform results, sanitize outputs, or log execution details
|
||||
|
||||
## Hook Types
|
||||
|
||||
### Before Tool Call Hooks
|
||||
|
||||
Executed before every tool execution, these hooks can:
|
||||
- Inspect and modify tool inputs
|
||||
- Block tool execution based on conditions
|
||||
- Implement approval gates for dangerous operations
|
||||
- Validate parameters
|
||||
- Log tool invocations
|
||||
|
||||
**Signature:**
|
||||
```python
|
||||
def before_hook(context: ToolCallHookContext) -> bool | None:
|
||||
# Return False to block execution
|
||||
# Return True or None to allow execution
|
||||
...
|
||||
```
|
||||
|
||||
### After Tool Call Hooks
|
||||
|
||||
Executed after every tool execution, these hooks can:
|
||||
- Modify or sanitize tool results
|
||||
- Add metadata or formatting
|
||||
- Log execution results
|
||||
- Implement result validation
|
||||
- Transform output formats
|
||||
|
||||
**Signature:**
|
||||
```python
|
||||
def after_hook(context: ToolCallHookContext) -> str | None:
|
||||
# Return modified result string
|
||||
# Return None to keep original result
|
||||
...
|
||||
```
|
||||
|
||||
## Tool Hook Context
|
||||
|
||||
The `ToolCallHookContext` object provides comprehensive access to tool execution state:
|
||||
|
||||
```python
|
||||
class ToolCallHookContext:
|
||||
tool_name: str # Name of the tool being called
|
||||
tool_input: dict[str, Any] # Mutable tool input parameters
|
||||
tool: CrewStructuredTool # Tool instance reference
|
||||
agent: Agent | BaseAgent | None # Agent executing the tool
|
||||
task: Task | None # Current task
|
||||
crew: Crew | None # Crew instance
|
||||
tool_result: str | None # Tool result (after hooks only)
|
||||
```
|
||||
|
||||
### Modifying Tool Inputs
|
||||
|
||||
**Important:** Always modify tool inputs in-place:
|
||||
|
||||
```python
|
||||
# ✅ Correct - modify in-place
|
||||
def sanitize_input(context: ToolCallHookContext) -> None:
|
||||
context.tool_input['query'] = context.tool_input['query'].lower()
|
||||
|
||||
# ❌ Wrong - replaces dict reference
|
||||
def wrong_approach(context: ToolCallHookContext) -> None:
|
||||
context.tool_input = {'query': 'new query'}
|
||||
```
|
||||
|
||||
## Registration Methods
|
||||
|
||||
### 1. Global Hook Registration
|
||||
|
||||
Register hooks that apply to all tool calls across all crews:
|
||||
|
||||
```python
|
||||
from crewai.hooks import register_before_tool_call_hook, register_after_tool_call_hook
|
||||
|
||||
def log_tool_call(context):
|
||||
print(f"Tool: {context.tool_name}")
|
||||
print(f"Input: {context.tool_input}")
|
||||
return None # Allow execution
|
||||
|
||||
register_before_tool_call_hook(log_tool_call)
|
||||
```
|
||||
|
||||
### 2. Decorator-Based Registration
|
||||
|
||||
Use decorators for cleaner syntax:
|
||||
|
||||
```python
|
||||
from crewai.hooks import before_tool_call, after_tool_call
|
||||
|
||||
@before_tool_call
|
||||
def block_dangerous_tools(context):
|
||||
dangerous_tools = ['delete_database', 'drop_table', 'rm_rf']
|
||||
if context.tool_name in dangerous_tools:
|
||||
print(f"⛔ Blocked dangerous tool: {context.tool_name}")
|
||||
return False # Block execution
|
||||
return None
|
||||
|
||||
@after_tool_call
|
||||
def sanitize_results(context):
|
||||
if context.tool_result and "password" in context.tool_result.lower():
|
||||
return context.tool_result.replace("password", "[REDACTED]")
|
||||
return None
|
||||
```
|
||||
|
||||
### 3. Crew-Scoped Hooks
|
||||
|
||||
Register hooks for a specific crew instance:
|
||||
|
||||
```python
|
||||
@CrewBase
|
||||
class MyProjCrew:
|
||||
@before_tool_call_crew
|
||||
def validate_tool_inputs(self, context):
|
||||
# Only applies to this crew
|
||||
if context.tool_name == "web_search":
|
||||
if not context.tool_input.get('query'):
|
||||
print("❌ Invalid search query")
|
||||
return False
|
||||
return None
|
||||
|
||||
@after_tool_call_crew
|
||||
def log_tool_results(self, context):
|
||||
# Crew-specific tool logging
|
||||
print(f"✅ {context.tool_name} completed")
|
||||
return None
|
||||
|
||||
@crew
|
||||
def crew(self) -> Crew:
|
||||
return Crew(
|
||||
agents=self.agents,
|
||||
tasks=self.tasks,
|
||||
process=Process.sequential,
|
||||
verbose=True
|
||||
)
|
||||
```
|
||||
|
||||
## Common Use Cases
|
||||
|
||||
### 1. Safety Guardrails
|
||||
|
||||
```python
|
||||
@before_tool_call
|
||||
def safety_check(context: ToolCallHookContext) -> bool | None:
|
||||
# Block tools that could cause harm
|
||||
destructive_tools = [
|
||||
'delete_file',
|
||||
'drop_table',
|
||||
'remove_user',
|
||||
'system_shutdown'
|
||||
]
|
||||
|
||||
if context.tool_name in destructive_tools:
|
||||
print(f"🛑 Blocked destructive tool: {context.tool_name}")
|
||||
return False
|
||||
|
||||
# Warn on sensitive operations
|
||||
sensitive_tools = ['send_email', 'post_to_social_media', 'charge_payment']
|
||||
if context.tool_name in sensitive_tools:
|
||||
print(f"⚠️ Executing sensitive tool: {context.tool_name}")
|
||||
|
||||
return None
|
||||
```
|
||||
|
||||
### 2. Human Approval Gate
|
||||
|
||||
```python
|
||||
@before_tool_call
|
||||
def require_approval_for_actions(context: ToolCallHookContext) -> bool | None:
|
||||
approval_required = [
|
||||
'send_email',
|
||||
'make_purchase',
|
||||
'delete_file',
|
||||
'post_message'
|
||||
]
|
||||
|
||||
if context.tool_name in approval_required:
|
||||
response = context.request_human_input(
|
||||
prompt=f"Approve {context.tool_name}?",
|
||||
default_message=f"Input: {context.tool_input}\nType 'yes' to approve:"
|
||||
)
|
||||
|
||||
if response.lower() != 'yes':
|
||||
print(f"❌ Tool execution denied: {context.tool_name}")
|
||||
return False
|
||||
|
||||
return None
|
||||
```
|
||||
|
||||
### 3. Input Validation and Sanitization
|
||||
|
||||
```python
|
||||
@before_tool_call
|
||||
def validate_and_sanitize_inputs(context: ToolCallHookContext) -> bool | None:
|
||||
# Validate search queries
|
||||
if context.tool_name == 'web_search':
|
||||
query = context.tool_input.get('query', '')
|
||||
if len(query) < 3:
|
||||
print("❌ Search query too short")
|
||||
return False
|
||||
|
||||
# Sanitize query
|
||||
context.tool_input['query'] = query.strip().lower()
|
||||
|
||||
# Validate file paths
|
||||
if context.tool_name == 'read_file':
|
||||
path = context.tool_input.get('path', '')
|
||||
if '..' in path or path.startswith('/'):
|
||||
print("❌ Invalid file path")
|
||||
return False
|
||||
|
||||
return None
|
||||
```
|
||||
|
||||
### 4. Result Sanitization
|
||||
|
||||
```python
|
||||
@after_tool_call
|
||||
def sanitize_sensitive_data(context: ToolCallHookContext) -> str | None:
|
||||
if not context.tool_result:
|
||||
return None
|
||||
|
||||
import re
|
||||
result = context.tool_result
|
||||
|
||||
# Remove API keys
|
||||
result = re.sub(
|
||||
r'(api[_-]?key|token)["\']?\s*[:=]\s*["\']?[\w-]+',
|
||||
r'\1: [REDACTED]',
|
||||
result,
|
||||
flags=re.IGNORECASE
|
||||
)
|
||||
|
||||
# Remove email addresses
|
||||
result = re.sub(
|
||||
r'\b[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Z|a-z]{2,}\b',
|
||||
'[EMAIL-REDACTED]',
|
||||
result
|
||||
)
|
||||
|
||||
# Remove credit card numbers
|
||||
result = re.sub(
|
||||
r'\b\d{4}[- ]?\d{4}[- ]?\d{4}[- ]?\d{4}\b',
|
||||
'[CARD-REDACTED]',
|
||||
result
|
||||
)
|
||||
|
||||
return result
|
||||
```
|
||||
|
||||
### 5. Tool Usage Analytics
|
||||
|
||||
```python
|
||||
import time
|
||||
from collections import defaultdict
|
||||
|
||||
tool_stats = defaultdict(lambda: {'count': 0, 'total_time': 0, 'failures': 0})
|
||||
|
||||
@before_tool_call
|
||||
def start_timer(context: ToolCallHookContext) -> None:
|
||||
context.tool_input['_start_time'] = time.time()
|
||||
return None
|
||||
|
||||
@after_tool_call
|
||||
def track_tool_usage(context: ToolCallHookContext) -> None:
|
||||
start_time = context.tool_input.get('_start_time', time.time())
|
||||
duration = time.time() - start_time
|
||||
|
||||
tool_stats[context.tool_name]['count'] += 1
|
||||
tool_stats[context.tool_name]['total_time'] += duration
|
||||
|
||||
if not context.tool_result or 'error' in context.tool_result.lower():
|
||||
tool_stats[context.tool_name]['failures'] += 1
|
||||
|
||||
print(f"""
|
||||
📊 Tool Stats for {context.tool_name}:
|
||||
- Executions: {tool_stats[context.tool_name]['count']}
|
||||
- Avg Time: {tool_stats[context.tool_name]['total_time'] / tool_stats[context.tool_name]['count']:.2f}s
|
||||
- Failures: {tool_stats[context.tool_name]['failures']}
|
||||
""")
|
||||
|
||||
return None
|
||||
```
|
||||
|
||||
### 6. Rate Limiting
|
||||
|
||||
```python
|
||||
from collections import defaultdict
|
||||
from datetime import datetime, timedelta
|
||||
|
||||
tool_call_history = defaultdict(list)
|
||||
|
||||
@before_tool_call
|
||||
def rate_limit_tools(context: ToolCallHookContext) -> bool | None:
|
||||
tool_name = context.tool_name
|
||||
now = datetime.now()
|
||||
|
||||
# Clean old entries (older than 1 minute)
|
||||
tool_call_history[tool_name] = [
|
||||
call_time for call_time in tool_call_history[tool_name]
|
||||
if now - call_time < timedelta(minutes=1)
|
||||
]
|
||||
|
||||
# Check rate limit (max 10 calls per minute)
|
||||
if len(tool_call_history[tool_name]) >= 10:
|
||||
print(f"🚫 Rate limit exceeded for {tool_name}")
|
||||
return False
|
||||
|
||||
# Record this call
|
||||
tool_call_history[tool_name].append(now)
|
||||
return None
|
||||
```
|
||||
|
||||
### 7. Caching Tool Results
|
||||
|
||||
```python
|
||||
import hashlib
|
||||
import json
|
||||
|
||||
tool_cache = {}
|
||||
|
||||
def cache_key(tool_name: str, tool_input: dict) -> str:
|
||||
"""Generate cache key from tool name and input."""
|
||||
input_str = json.dumps(tool_input, sort_keys=True)
|
||||
return hashlib.md5(f"{tool_name}:{input_str}".encode()).hexdigest()
|
||||
|
||||
@before_tool_call
|
||||
def check_cache(context: ToolCallHookContext) -> bool | None:
|
||||
key = cache_key(context.tool_name, context.tool_input)
|
||||
if key in tool_cache:
|
||||
print(f"💾 Cache hit for {context.tool_name}")
|
||||
# Note: Can't return cached result from before hook
|
||||
# Would need to implement this differently
|
||||
return None
|
||||
|
||||
@after_tool_call
|
||||
def cache_result(context: ToolCallHookContext) -> None:
|
||||
if context.tool_result:
|
||||
key = cache_key(context.tool_name, context.tool_input)
|
||||
tool_cache[key] = context.tool_result
|
||||
print(f"💾 Cached result for {context.tool_name}")
|
||||
return None
|
||||
```
|
||||
|
||||
### 8. Debug Logging
|
||||
|
||||
```python
|
||||
@before_tool_call
|
||||
def debug_tool_call(context: ToolCallHookContext) -> None:
|
||||
print(f"""
|
||||
🔍 Tool Call Debug:
|
||||
- Tool: {context.tool_name}
|
||||
- Agent: {context.agent.role if context.agent else 'Unknown'}
|
||||
- Task: {context.task.description[:50] if context.task else 'Unknown'}...
|
||||
- Input: {context.tool_input}
|
||||
""")
|
||||
return None
|
||||
|
||||
@after_tool_call
|
||||
def debug_tool_result(context: ToolCallHookContext) -> None:
|
||||
if context.tool_result:
|
||||
result_preview = context.tool_result[:200]
|
||||
print(f"✅ Result Preview: {result_preview}...")
|
||||
else:
|
||||
print("⚠️ No result returned")
|
||||
return None
|
||||
```
|
||||
|
||||
## Hook Management
|
||||
|
||||
### Unregistering Hooks
|
||||
|
||||
```python
|
||||
from crewai.hooks import (
|
||||
unregister_before_tool_call_hook,
|
||||
unregister_after_tool_call_hook
|
||||
)
|
||||
|
||||
# Unregister specific hook
|
||||
def my_hook(context):
|
||||
...
|
||||
|
||||
register_before_tool_call_hook(my_hook)
|
||||
# Later...
|
||||
success = unregister_before_tool_call_hook(my_hook)
|
||||
print(f"Unregistered: {success}")
|
||||
```
|
||||
|
||||
### Clearing Hooks
|
||||
|
||||
```python
|
||||
from crewai.hooks import (
|
||||
clear_before_tool_call_hooks,
|
||||
clear_after_tool_call_hooks,
|
||||
clear_all_tool_call_hooks
|
||||
)
|
||||
|
||||
# Clear specific hook type
|
||||
count = clear_before_tool_call_hooks()
|
||||
print(f"Cleared {count} before hooks")
|
||||
|
||||
# Clear all tool hooks
|
||||
before_count, after_count = clear_all_tool_call_hooks()
|
||||
print(f"Cleared {before_count} before and {after_count} after hooks")
|
||||
```
|
||||
|
||||
### Listing Registered Hooks
|
||||
|
||||
```python
|
||||
from crewai.hooks import (
|
||||
get_before_tool_call_hooks,
|
||||
get_after_tool_call_hooks
|
||||
)
|
||||
|
||||
# Get current hooks
|
||||
before_hooks = get_before_tool_call_hooks()
|
||||
after_hooks = get_after_tool_call_hooks()
|
||||
|
||||
print(f"Registered: {len(before_hooks)} before, {len(after_hooks)} after")
|
||||
```
|
||||
|
||||
## Advanced Patterns
|
||||
|
||||
### Conditional Hook Execution
|
||||
|
||||
```python
|
||||
@before_tool_call
|
||||
def conditional_blocking(context: ToolCallHookContext) -> bool | None:
|
||||
# Only block for specific agents
|
||||
if context.agent and context.agent.role == "junior_agent":
|
||||
if context.tool_name in ['delete_file', 'send_email']:
|
||||
print(f"❌ Junior agents cannot use {context.tool_name}")
|
||||
return False
|
||||
|
||||
# Only block during specific tasks
|
||||
if context.task and "sensitive" in context.task.description.lower():
|
||||
if context.tool_name == 'web_search':
|
||||
print("❌ Web search blocked for sensitive tasks")
|
||||
return False
|
||||
|
||||
return None
|
||||
```
|
||||
|
||||
### Context-Aware Input Modification
|
||||
|
||||
```python
|
||||
@before_tool_call
|
||||
def enhance_tool_inputs(context: ToolCallHookContext) -> None:
|
||||
# Add context based on agent role
|
||||
if context.agent and context.agent.role == "researcher":
|
||||
if context.tool_name == 'web_search':
|
||||
# Add domain restrictions for researchers
|
||||
context.tool_input['domains'] = ['edu', 'gov', 'org']
|
||||
|
||||
# Add context based on task
|
||||
if context.task and "urgent" in context.task.description.lower():
|
||||
if context.tool_name == 'send_email':
|
||||
context.tool_input['priority'] = 'high'
|
||||
|
||||
return None
|
||||
```
|
||||
|
||||
### Tool Chain Monitoring
|
||||
|
||||
```python
|
||||
tool_call_chain = []
|
||||
|
||||
@before_tool_call
|
||||
def track_tool_chain(context: ToolCallHookContext) -> None:
|
||||
tool_call_chain.append({
|
||||
'tool': context.tool_name,
|
||||
'timestamp': time.time(),
|
||||
'agent': context.agent.role if context.agent else 'Unknown'
|
||||
})
|
||||
|
||||
# Detect potential infinite loops
|
||||
recent_calls = tool_call_chain[-5:]
|
||||
if len(recent_calls) == 5 and all(c['tool'] == context.tool_name for c in recent_calls):
|
||||
print(f"⚠️ Warning: {context.tool_name} called 5 times in a row")
|
||||
|
||||
return None
|
||||
```
|
||||
|
||||
## Best Practices
|
||||
|
||||
1. **Keep Hooks Focused**: Each hook should have a single responsibility
|
||||
2. **Avoid Heavy Computation**: Hooks execute on every tool call
|
||||
3. **Handle Errors Gracefully**: Use try-except to prevent hook failures
|
||||
4. **Use Type Hints**: Leverage `ToolCallHookContext` for better IDE support
|
||||
5. **Document Blocking Conditions**: Make it clear when/why tools are blocked
|
||||
6. **Test Hooks Independently**: Unit test hooks before using in production
|
||||
7. **Clear Hooks in Tests**: Use `clear_all_tool_call_hooks()` between test runs
|
||||
8. **Modify In-Place**: Always modify `context.tool_input` in-place, never replace
|
||||
9. **Log Important Decisions**: Especially when blocking tool execution
|
||||
10. **Consider Performance**: Cache expensive validations when possible
|
||||
|
||||
## Error Handling
|
||||
|
||||
```python
|
||||
@before_tool_call
|
||||
def safe_validation(context: ToolCallHookContext) -> bool | None:
|
||||
try:
|
||||
# Your validation logic
|
||||
if not validate_input(context.tool_input):
|
||||
return False
|
||||
except Exception as e:
|
||||
print(f"⚠️ Hook error: {e}")
|
||||
# Decide: allow or block on error
|
||||
return None # Allow execution despite error
|
||||
```
|
||||
|
||||
## Type Safety
|
||||
|
||||
```python
|
||||
from crewai.hooks import ToolCallHookContext, BeforeToolCallHookType, AfterToolCallHookType
|
||||
|
||||
# Explicit type annotations
|
||||
def my_before_hook(context: ToolCallHookContext) -> bool | None:
|
||||
return None
|
||||
|
||||
def my_after_hook(context: ToolCallHookContext) -> str | None:
|
||||
return None
|
||||
|
||||
# Type-safe registration
|
||||
register_before_tool_call_hook(my_before_hook)
|
||||
register_after_tool_call_hook(my_after_hook)
|
||||
```
|
||||
|
||||
## Integration with Existing Tools
|
||||
|
||||
### Wrapping Existing Validation
|
||||
|
||||
```python
|
||||
def existing_validator(tool_name: str, inputs: dict) -> bool:
|
||||
"""Your existing validation function."""
|
||||
# Your validation logic
|
||||
return True
|
||||
|
||||
@before_tool_call
|
||||
def integrate_validator(context: ToolCallHookContext) -> bool | None:
|
||||
if not existing_validator(context.tool_name, context.tool_input):
|
||||
print(f"❌ Validation failed for {context.tool_name}")
|
||||
return False
|
||||
return None
|
||||
```
|
||||
|
||||
### Logging to External Systems
|
||||
|
||||
```python
|
||||
import logging
|
||||
|
||||
logger = logging.getLogger(__name__)
|
||||
|
||||
@before_tool_call
|
||||
def log_to_external_system(context: ToolCallHookContext) -> None:
|
||||
logger.info(f"Tool call: {context.tool_name}", extra={
|
||||
'tool_name': context.tool_name,
|
||||
'tool_input': context.tool_input,
|
||||
'agent': context.agent.role if context.agent else None
|
||||
})
|
||||
return None
|
||||
```
|
||||
|
||||
## Troubleshooting
|
||||
|
||||
### Hook Not Executing
|
||||
- Verify hook is registered before crew execution
|
||||
- Check if previous hook returned `False` (blocks execution and subsequent hooks)
|
||||
- Ensure hook signature matches expected type
|
||||
|
||||
### Input Modifications Not Working
|
||||
- Use in-place modifications: `context.tool_input['key'] = value`
|
||||
- Don't replace the dict: `context.tool_input = {}`
|
||||
|
||||
### Result Modifications Not Working
|
||||
- Return the modified string from after hooks
|
||||
- Returning `None` keeps the original result
|
||||
- Ensure the tool actually returned a result
|
||||
|
||||
### Tool Blocked Unexpectedly
|
||||
- Check all before hooks for blocking conditions
|
||||
- Verify hook execution order
|
||||
- Add debug logging to identify which hook is blocking
|
||||
|
||||
## Conclusion
|
||||
|
||||
Tool Call Hooks provide powerful capabilities for controlling and monitoring tool execution in CrewAI. Use them to implement safety guardrails, approval gates, input validation, result sanitization, logging, and analytics. Combined with proper error handling and type safety, hooks enable secure and production-ready agent systems with comprehensive observability.
|
||||
Reference in New Issue
Block a user