Address PR review feedback: Add error handling, type validation, and edge case tests

Co-Authored-By: Joe Moura <joao@crewai.com>
This commit is contained in:
Devin AI
2025-05-11 01:49:08 +00:00
parent 8f3162b8e8
commit 8476fb2c64
4 changed files with 212 additions and 16 deletions

View File

@@ -112,4 +112,102 @@ model_list = [
]
```
## Error Handling and Troubleshooting
When working with multiple model configurations, you may encounter various issues. Here are some common problems and their solutions:
### Missing Required Parameters
**Problem**: Router initialization fails with an error about missing parameters.
**Solution**: Ensure each model configuration in `model_list` includes both `model_name` and `litellm_params` with the required `model` parameter:
```python
# Correct configuration
model_config = {
"model_name": "gpt-4o-mini", # Required
"litellm_params": {
"model": "gpt-4o-mini", # Required
"api_key": "your-api-key"
}
}
```
### Invalid Routing Strategy
**Problem**: Error when specifying an unsupported routing strategy.
**Solution**: Use only the supported routing strategies:
```python
# Valid routing strategies
valid_strategies = [
"simple-shuffle",
"least-busy",
"usage-based",
"latency-based",
"cost-based"
]
```
### API Key Authentication Errors
**Problem**: Authentication errors when making API calls.
**Solution**: Verify that all API keys are valid and have the necessary permissions:
```python
# Check environment variables first
import os
os.environ.get("OPENAI_API_KEY") # Should be set if using OpenAI models
# Or explicitly provide in the configuration
model_list = [{
"model_name": "gpt-4o-mini",
"litellm_params": {
"model": "gpt-4o-mini",
"api_key": "valid-api-key-here" # Ensure this is correct
}
}]
```
### Rate Limit Handling
**Problem**: Encountering rate limits with multiple models.
**Solution**: Configure rate limits and implement fallback mechanisms:
```python
model_list = [
{
"model_name": "primary-model",
"litellm_params": {"model": "primary-model", "api_key": "key1"},
"rpm": 100 # Requests per minute
},
{
"model_name": "fallback-model",
"litellm_params": {"model": "fallback-model", "api_key": "key2"}
}
]
# Configure with fallback
llm = LLM(
model="primary-model",
model_list=model_list,
routing_strategy="least-busy" # Will route to fallback when primary is busy
)
```
### Debugging Router Issues
If you're experiencing issues with the router, you can enable verbose logging to get more information:
```python
import litellm
litellm.set_verbose = True
# Then initialize your LLM
llm = LLM(model="gpt-4o-mini", model_list=model_list)
```
This feature leverages litellm's Router functionality under the hood, providing robust load balancing and fallback capabilities for your CrewAI agents. The implementation ensures predictability and consistency in model selection while maintaining security through proper API key management.