andre/crewAI
1
0
Fork 0
mirror of https://github.com/crewAIInc/crewAI.git synced 2026-01-09 08:08:32 +00:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity

updating docs

Browse Source
This commit is contained in:
João Moura
2024-09-23 03:59:16 -03:00
parent a19a4a5556
commit 854c58ded7
3 changed files with 248 additions and 38 deletions
Download Patch File Download Diff File Expand all files Collapse all files

126
docs/how-to/LLM-Connections.md
View File

@@ -5,10 +5,10 @@ description: Comprehensive guide on integrating CrewAI with various Large Langua
## Connect CrewAI to LLMs
CrewAI now uses LiteLLM to connect to a wide variety of Language Models (LLMs). This integration provides extensive versatility, allowing you to use models from numerous providers with a simple, unified interface.
CrewAI uses LiteLLM to connect to a wide variety of Language Models (LLMs). This integration provides extensive versatility, allowing you to use models from numerous providers with a simple, unified interface.
!!! note "Default LLM"
By default, CrewAI uses OpenAI's GPT-4 model (specifically, the model specified by the OPENAI_MODEL_NAME environment variable, defaulting to "gpt-4") for language processing. You can easily configure your agents to use a different model or provider as described in this guide.
By default, CrewAI uses the `gpt-4o-mini` model. This is determined by the `OPENAI_MODEL_NAME` environment variable, which defaults to "gpt-4o-mini" if not set. You can easily configure your agents to use a different model or provider as described in this guide.
## Supported Providers
@@ -35,7 +35,11 @@ For a complete and up-to-date list of supported providers, please refer to the [
## Changing the LLM
To use a different LLM with your CrewAI agents, you simply need to pass the model name as a string when initializing the agent. Here are some examples:
To use a different LLM with your CrewAI agents, you have several options:
### 1. Using a String Identifier
Pass the model name as a string when initializing the agent:
```python
from crewai import Agent
@@ -55,59 +59,105 @@ claude_agent = Agent(
backstory="An AI assistant leveraging Anthropic's language model.",
llm='claude-2'
)
```
# Using Ollama's local Llama 2 model
ollama_agent = Agent(
role='Local AI Expert',
goal='Process information using a local model',
backstory="An AI assistant running on local hardware.",
llm='ollama/llama2'
### 2. Using the LLM Class
For more detailed configuration, use the LLM class:
```python
from crewai import Agent, LLM
llm = LLM(
model="gpt-4",
temperature=0.7,
base_url="https://api.openai.com/v1",
api_key="your-api-key-here"
)
# Using Google's Gemini model
gemini_agent = Agent(
role='Google AI Expert',
goal='Generate creative content with Gemini',
backstory="An AI assistant powered by Google's advanced language model.",
llm='gemini-pro'
agent = Agent(
role='Customized LLM Expert',
goal='Provide tailored responses',
backstory="An AI assistant with custom LLM settings.",
llm=llm
)
```
## Configuration
## Configuration Options
For most providers, you'll need to set up your API keys as environment variables. Here's how you can do it for some common providers:
When configuring an LLM for your agent, you have access to a wide range of parameters:
| Parameter | Type | Description |
|-----------|------|-------------|
| `model` | str | The name of the model to use (e.g., "gpt-4", "claude-2") |
| `temperature` | float | Controls randomness in output (0.0 to 1.0) |
| `max_tokens` | int | Maximum number of tokens to generate |
| `top_p` | float | Controls diversity of output (0.0 to 1.0) |
| `frequency_penalty` | float | Penalizes new tokens based on their frequency in the text so far |
| `presence_penalty` | float | Penalizes new tokens based on their presence in the text so far |
| `stop` | str, List[str] | Sequence(s) to stop generation |
| `base_url` | str | The base URL for the API endpoint |
| `api_key` | str | Your API key for authentication |
For a complete list of parameters and their descriptions, refer to the LLM class documentation.
## Connecting to OpenAI-Compatible LLMs
You can connect to OpenAI-compatible LLMs using either environment variables or by setting specific attributes on the LLM class:
### Using Environment Variables
```python
import os
# OpenAI
os.environ["OPENAI_API_KEY"] = "your-openai-api-key"
# Anthropic
os.environ["ANTHROPIC_API_KEY"] = "your-anthropic-api-key"
# Google (Vertex AI)
os.environ["GOOGLE_APPLICATION_CREDENTIALS"] = "path/to/your/credentials.json"
# Azure OpenAI
os.environ["AZURE_API_KEY"] = "your-azure-api-key"
os.environ["AZURE_API_BASE"] = "your-azure-endpoint"
# AWS (Bedrock)
os.environ["AWS_ACCESS_KEY_ID"] = "your-aws-access-key-id"
os.environ["AWS_SECRET_ACCESS_KEY"] = "your-aws-secret-access-key"
os.environ["OPENAI_API_KEY"] = "your-api-key"
os.environ["OPENAI_API_BASE"] = "https://api.your-provider.com/v1"
os.environ["OPENAI_MODEL_NAME"] = "your-model-name"
```
For providers that require additional configuration or have specific setup requirements, please refer to the [LiteLLM documentation](https://docs.litellm.ai/docs/) for detailed instructions.
### Using LLM Class Attributes
## Using Local Models
```python
llm = LLM(
model="custom-model-name",
api_key="your-api-key",
base_url="https://api.your-provider.com/v1"
)
agent = Agent(llm=llm, ...)
```
For local models like those provided by Ollama, ensure you have the necessary software installed and running. For example, to use Ollama:
## Using Local Models with Ollama
For local models like those provided by Ollama:
1. [Download and install Ollama](https://ollama.com/download)
2. Pull the desired model (e.g., `ollama pull llama2`)
3. Use the model in your CrewAI agent by specifying `llm='ollama/llama2'`
3. Configure your agent:
```python
agent = Agent(
role='Local AI Expert',
goal='Process information using a local model',
backstory="An AI assistant running on local hardware.",
llm=LLM(model="ollama/llama2", base_url="http://localhost:11434")
)
```
## Changing the Base API URL
You can change the base API URL for any LLM provider by setting the `base_url` parameter:
```python
llm = LLM(
model="custom-model-name",
base_url="https://api.your-provider.com/v1",
api_key="your-api-key"
)
agent = Agent(llm=llm, ...)
```
This is particularly useful when working with OpenAI-compatible APIs or when you need to specify a different endpoint for your chosen provider.
## Conclusion
By leveraging LiteLLM, CrewAI now offers seamless integration with a vast array of LLMs. This flexibility allows you to choose the most suitable model for your specific needs, whether you prioritize performance, cost-efficiency, or local deployment. Remember to consult the [LiteLLM documentation](https://docs.litellm.ai/docs/) for the most up-to-date information on supported models and configuration options.
By leveraging LiteLLM, CrewAI offers seamless integration with a vast array of LLMs. This flexibility allows you to choose the most suitable model for your specific needs, whether you prioritize performance, cost-efficiency, or local deployment. Remember to consult the [LiteLLM documentation](https://docs.litellm.ai/docs/) for the most up-to-date information on supported models and configuration options.