andre/crewAI
1
0
Fork 0
mirror of https://github.com/crewAIInc/crewAI.git synced 2026-01-08 23:58:34 +00:00
Code Issues Actions 3 Packages Projects Releases Wiki Activity

ensure original embedding config works (#1476)

Browse Source 
* ensure original embedding config works

* some fixes

* raise error on unsupported provider

* WIP: brandons notes

* fixes

* rm prints

* fixed docs

* fixed run types

* updates to add more docs and correct imports with huggingface embedding server enabled

---------

Co-authored-by: Brandon Hancock <brandon@brandonhancock.io>
This commit is contained in:
Lorenze Jay
2024-10-22 12:30:30 -07:00
committed by GitHub
parent 8731915330
commit 4687779702
3 changed files with 166 additions and 18 deletions
Download Patch File Download Diff File Expand all files Collapse all files

86
docs/concepts/memory.mdx
View File

@@ -105,9 +105,48 @@ my_crew = Crew(
process=Process.sequential,
memory=True,
verbose=True,
embedder=embedding_functions.OpenAIEmbeddingFunction(
api_key=os.getenv("OPENAI_API_KEY"), model_name="text-embedding-3-small"
)
embedder={
"provider": "openai",
"config": {
"model": 'text-embedding-3-small'
}
}
)
```
Alternatively, you can directly pass the OpenAIEmbeddingFunction to the embedder parameter.
Example:
```python Code
from crewai import Crew, Agent, Task, Process
from chromadb.utils.embedding_functions.openai_embedding_function import OpenAIEmbeddingFunction
my_crew = Crew(
agents=[...],
tasks=[...],
process=Process.sequential,
memory=True,
verbose=True,
embedder=OpenAIEmbeddingFunction(api_key=os.getenv("OPENAI_API_KEY"), model_name="text-embedding-3-small"),
)
```
### Using Ollama embeddings
```python Code
from crewai import Crew, Agent, Task, Process
my_crew = Crew(
agents=[...],
tasks=[...],
process=Process.sequential,
memory=True,
verbose=True,
embedder={
"provider": "ollama",
"config": {
"model": "mxbai-embed-large"
}
}
)
```
@@ -122,10 +161,13 @@ my_crew = Crew(
process=Process.sequential,
memory=True,
verbose=True,
embedder=embedding_functions.OpenAIEmbeddingFunction(
api_key=os.getenv("OPENAI_API_KEY"),
model_name="text-embedding-ada-002"
)
embedder={
"provider": "google",
"config": {
"api_key": "<YOUR_API_KEY>",
"model_name": "<model_name>"
}
}
)
```
@@ -181,10 +223,32 @@ my_crew = Crew(
process=Process.sequential,
memory=True,
verbose=True,
embedder=embedding_functions.CohereEmbeddingFunction(
api_key=YOUR_API_KEY,
model_name="<model_name>"
)
embedder={
"provider": "cohere",
"config": {
"api_key": "YOUR_API_KEY",
"model_name": "<model_name>"
}
}
)
```
### Using HuggingFace embeddings
```python Code
from crewai import Crew, Agent, Task, Process
my_crew = Crew(
agents=[...],
tasks=[...],
process=Process.sequential,
memory=True,
verbose=True,
embedder={
"provider": "huggingface",
"config": {
"api_url": "<api_url>",
}
}
)
```