mirror of
https://github.com/crewAIInc/crewAI.git
synced 2026-07-01 21:28:10 +00:00
* feat: adopt directory-based docs versioning with Edge channel Switch docs.crewai.com from navigation-only versioning (every version selector entry rendered the same docs/<lang>/* source files) to Mintlify's directory-based versioning so each version selector entry renders its own snapshot. Add an "Edge" channel under docs/edge/<lang>/* that always reflects main HEAD for unreleased work, eliminating pre-release leakage onto frozen release labels. External links to canonical /<lang>/* URLs are preserved via wildcard redirects that always land on the current default version. Layout: - docs/edge/<lang>/* rolling source (you edit here) - docs/edge/enterprise-api.*.yaml - docs/v<X.Y.Z>/<lang>/* frozen, immutable snapshots - docs/v<X.Y.Z>/enterprise-api.*.yaml - docs/images/ shared, append-only - docs/docs.json nav + redirects URLs follow the Mintlify-idiomatic shape: /edge/<lang>/<page> for Edge, /v<X.Y.Z>/<lang>/<page> for every frozen snapshot. The wildcard redirects /<lang>/:slug* -> /<default>/<lang>/:slug* keep stale links working, and every freeze rewrites them (plus all per-section/per-page redirects) so destinations always resolve to the current default without depending on a second redirect hop. Release flow integration (devtools release): - New module crewai_devtools.docs_versioning.freeze() materialises docs/v<X.Y.Z>/ from docs/edge/, rewrites openapi: refs inside the snapshot, inserts the version into every language block in docs.json, and refreshes all redirect destinations. - _update_docs_and_create_pr() in cli.py now calls that freeze during Phase 2 of devtools release. Edge changelogs are updated first (so the snapshot freeze picks them up), then the snapshot is staged alongside docs.json, branched as docs/freeze-v<X.Y.Z>, and the PR is titled [docs-freeze] docs: snapshot and changelog for v<X.Y.Z> — the title prefix the new CI guard reads. - The PR still gates tag, GitHub release, PyPI publish, and the enterprise release as before; no new PRs are added. - Pre-releases (1.X.YaN, 1.X.YbN, ...) skip the snapshot — they ride Edge — and the docs PR title omits the [docs-freeze] prefix. - docs_check (AI-generated docs scaffolding) writes to docs/edge/<lang>/* so newly-generated unreleased docs land in Edge and never accidentally touch a frozen snapshot. Migration scripts (one-shot): - scripts/docs/freeze_historical_versions.py reconstructs all 16 historical snapshots (v1.10.0 .. v1.14.7) from git tags via git archive | tar, rewriting openapi: MDX refs so each snapshot reads its own enterprise-api YAML rather than the live one. - scripts/docs/prefix_version_paths.py one-shot-migrates docs.json: rewrites every page path in 16 versioned blocks to point under docs/v<X.Y.Z>/, inserts a new Edge entry per language, tags v1.14.7 as Latest (default), prunes pages whose target file doesn't exist in the snapshot (e.g. docs/ar/ didn't exist before v1.12.0), and writes the wildcard + per-section redirects. - scripts/docs/freeze_current_edge.py is now a thin CLI wrapper around docs_versioning.freeze for manual one-off freezes (e.g. retroactively snapshotting a forgotten release). CI guards (.github/workflows/docs-snapshots.yml): - Frozen snapshots under docs/v[0-9]*/ are immutable; only PRs whose title contains [docs-freeze] (i.e. release-cut PRs generated by devtools release or the manual wrapper) may modify them. - Images under docs/images/ are append-only since snapshots share a single image directory. Deleting or renaming an image breaks every historical snapshot that still references it. Restored docs/images/crewai-otel-export.png from PR #3673; it was deleted in PR #4908 but v1.10.0 / v1.10.1 snapshots still reference it. Restoring instead of editing the snapshots preserves historical rendering fidelity and validates the new append-only rule retroactively. Tests: - lib/devtools/tests/test_docs_versioning.py covers the freeze: file copy, openapi rewrite, version insertion, default demotion, redirect upserts, per-section redirect rewriting, idempotency, and invalid inputs. Verified locally with mintlify broken-links: 0 broken links across the full site (Edge + 16 frozen versions, 4 locales). AGENTS.md (repo root) is the contributor guide for the new model; RELEASING.md is the release-cut runbook; README's Contribution section links to both. Co-authored-by: Cursor <cursoragent@cursor.com> * style: resolve linter issues --------- Co-authored-by: Cursor <cursoragent@cursor.com>
345 lines
11 KiB
Plaintext
345 lines
11 KiB
Plaintext
---
|
|
title: 'Qdrant Vector Search Tool'
|
|
description: 'Capacidades de busca semântica para agentes CrewAI usando o banco de dados vetorial Qdrant'
|
|
icon: vector-square
|
|
mode: "wide"
|
|
---
|
|
|
|
## Visão Geral
|
|
|
|
A ferramenta Qdrant Vector Search permite adicionar capacidades de busca semântica aos seus agentes CrewAI utilizando o [Qdrant](https://qdrant.tech/), um mecanismo de busca por similaridade vetorial. Com essa ferramenta, seus agentes podem pesquisar em documentos armazenados em uma coleção Qdrant usando similaridade semântica.
|
|
|
|
## Instalação
|
|
|
|
Instale os pacotes necessários:
|
|
|
|
```bash
|
|
uv add qdrant-client
|
|
```
|
|
|
|
## Uso Básico
|
|
|
|
Veja um exemplo mínimo de como utilizar a ferramenta:
|
|
|
|
```python
|
|
from crewai import Agent
|
|
from crewai_tools import QdrantVectorSearchTool, QdrantConfig
|
|
|
|
# Inicialize a ferramenta com QdrantConfig
|
|
qdrant_tool = QdrantVectorSearchTool(
|
|
qdrant_config=QdrantConfig(
|
|
qdrant_url="your_qdrant_url",
|
|
qdrant_api_key="your_qdrant_api_key",
|
|
collection_name="your_collection"
|
|
)
|
|
)
|
|
|
|
# Crie um agente que utiliza a ferramenta
|
|
agent = Agent(
|
|
role="Research Assistant",
|
|
goal="Find relevant information in documents",
|
|
tools=[qdrant_tool]
|
|
)
|
|
|
|
# A ferramenta usará automaticamente embeddings da OpenAI
|
|
# e retornará os 3 resultados mais relevantes com pontuação > 0.35
|
|
```
|
|
|
|
## Exemplo Completo e Funcional
|
|
|
|
Veja um exemplo completo mostrando como:
|
|
1. Extrair texto de um PDF
|
|
2. Gerar embeddings usando OpenAI
|
|
3. Armazenar no Qdrant
|
|
4. Criar um fluxo de trabalho RAG agente CrewAI para busca semântica
|
|
|
|
```python
|
|
import os
|
|
import uuid
|
|
import pdfplumber
|
|
from openai import OpenAI
|
|
from dotenv import load_dotenv
|
|
from crewai import Agent, Task, Crew, Process, LLM
|
|
from crewai_tools import QdrantVectorSearchTool
|
|
from qdrant_client import QdrantClient
|
|
from qdrant_client.models import PointStruct, Distance, VectorParams
|
|
|
|
# Carregar variáveis de ambiente
|
|
load_dotenv()
|
|
|
|
# Inicializar cliente OpenAI
|
|
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))
|
|
|
|
# Extrair texto do PDF
|
|
def extract_text_from_pdf(pdf_path):
|
|
text = []
|
|
with pdfplumber.open(pdf_path) as pdf:
|
|
for page in pdf.pages:
|
|
page_text = page.extract_text()
|
|
if page_text:
|
|
text.append(page_text.strip())
|
|
return text
|
|
|
|
# Gerar embeddings da OpenAI
|
|
def get_openai_embedding(text):
|
|
response = client.embeddings.create(
|
|
input=text,
|
|
model="text-embedding-3-large"
|
|
)
|
|
return response.data[0].embedding
|
|
|
|
# Armazenar texto e embeddings no Qdrant
|
|
def load_pdf_to_qdrant(pdf_path, qdrant, collection_name):
|
|
# Extrair texto do PDF
|
|
text_chunks = extract_text_from_pdf(pdf_path)
|
|
|
|
# Criar coleção no Qdrant
|
|
if qdrant.collection_exists(collection_name):
|
|
qdrant.delete_collection(collection_name)
|
|
qdrant.create_collection(
|
|
collection_name=collection_name,
|
|
vectors_config=VectorParams(size=3072, distance=Distance.COSINE)
|
|
)
|
|
|
|
# Armazenar embeddings
|
|
points = []
|
|
for chunk in text_chunks:
|
|
embedding = get_openai_embedding(chunk)
|
|
points.append(PointStruct(
|
|
id=str(uuid.uuid4()),
|
|
vector=embedding,
|
|
payload={"text": chunk}
|
|
))
|
|
qdrant.upsert(collection_name=collection_name, points=points)
|
|
|
|
# Inicializar cliente Qdrant e carregar dados
|
|
qdrant = QdrantClient(
|
|
url=os.getenv("QDRANT_URL"),
|
|
api_key=os.getenv("QDRANT_API_KEY")
|
|
)
|
|
collection_name = "example_collection"
|
|
pdf_path = "path/to/your/document.pdf"
|
|
load_pdf_to_qdrant(pdf_path, qdrant, collection_name)
|
|
|
|
# Inicializar ferramenta de busca Qdrant
|
|
from crewai_tools import QdrantConfig
|
|
|
|
qdrant_tool = QdrantVectorSearchTool(
|
|
qdrant_config=QdrantConfig(
|
|
qdrant_url=os.getenv("QDRANT_URL"),
|
|
qdrant_api_key=os.getenv("QDRANT_API_KEY"),
|
|
collection_name=collection_name,
|
|
limit=3,
|
|
score_threshold=0.35
|
|
)
|
|
)
|
|
|
|
# Criar agentes CrewAI
|
|
search_agent = Agent(
|
|
role="Senior Semantic Search Agent",
|
|
goal="Find and analyze documents based on semantic search",
|
|
backstory="""You are an expert research assistant who can find relevant
|
|
information using semantic search in a Qdrant database.""",
|
|
tools=[qdrant_tool],
|
|
verbose=True
|
|
)
|
|
|
|
answer_agent = Agent(
|
|
role="Senior Answer Assistant",
|
|
goal="Generate answers to questions based on the context provided",
|
|
backstory="""You are an expert answer assistant who can generate
|
|
answers to questions based on the context provided.""",
|
|
tools=[qdrant_tool],
|
|
verbose=True
|
|
)
|
|
|
|
# Definir tarefas
|
|
search_task = Task(
|
|
description="""Search for relevant documents about the {query}.
|
|
Your final answer should include:
|
|
- The relevant information found
|
|
- The similarity scores of the results
|
|
- The metadata of the relevant documents""",
|
|
agent=search_agent
|
|
)
|
|
|
|
answer_task = Task(
|
|
description="""Given the context and metadata of relevant documents,
|
|
generate a final answer based on the context.""",
|
|
agent=answer_agent
|
|
)
|
|
|
|
# Executar fluxo CrewAI
|
|
crew = Crew(
|
|
agents=[search_agent, answer_agent],
|
|
tasks=[search_task, answer_task],
|
|
process=Process.sequential,
|
|
verbose=True
|
|
)
|
|
|
|
result = crew.kickoff(
|
|
inputs={"query": "What is the role of X in the document?"}
|
|
)
|
|
print(result)
|
|
```
|
|
|
|
## Parâmetros da Ferramenta
|
|
|
|
### Parâmetros Obrigatórios
|
|
- `qdrant_config` (QdrantConfig): Objeto de configuração contendo todas as configurações do Qdrant
|
|
|
|
### Parâmetros do QdrantConfig
|
|
- `qdrant_url` (str): URL do seu servidor Qdrant
|
|
- `qdrant_api_key` (str, opcional): Chave de API para autenticação com o Qdrant
|
|
- `collection_name` (str): Nome da coleção Qdrant a ser pesquisada
|
|
- `limit` (int): Número máximo de resultados a serem retornados (padrão: 3)
|
|
- `score_threshold` (float): Limite mínimo de similaridade (padrão: 0.35)
|
|
- `filter` (Any, opcional): Instância de Filter do Qdrant para filtragem avançada (padrão: None)
|
|
|
|
### Parâmetros Opcionais da Ferramenta
|
|
- `custom_embedding_fn` (Callable[[str], list[float]]): Função personalizada para vetorização de textos
|
|
- `qdrant_package` (str): Caminho base do pacote Qdrant (padrão: "qdrant_client")
|
|
- `client` (Any): Cliente Qdrant pré-inicializado (opcional)
|
|
|
|
## Filtragem Avançada
|
|
|
|
A ferramenta QdrantVectorSearchTool oferece recursos poderosos de filtragem para refinar os resultados da busca:
|
|
|
|
### Filtragem Dinâmica
|
|
Use os parâmetros `filter_by` e `filter_value` na sua busca para filtrar resultados dinamicamente:
|
|
|
|
```python
|
|
# O agente usará esses parâmetros ao chamar a ferramenta
|
|
# O schema da ferramenta aceita filter_by e filter_value
|
|
# Exemplo: busca com filtro de categoria
|
|
# Os resultados serão filtrados onde categoria == "tecnologia"
|
|
```
|
|
|
|
### Filtros Pré-definidos com QdrantConfig
|
|
Para filtragens complexas, use instâncias de Filter do Qdrant na sua configuração:
|
|
|
|
```python
|
|
from qdrant_client.http import models as qmodels
|
|
from crewai_tools import QdrantVectorSearchTool, QdrantConfig
|
|
|
|
# Criar um filtro para condições específicas
|
|
preset_filter = qmodels.Filter(
|
|
must=[
|
|
qmodels.FieldCondition(
|
|
key="categoria",
|
|
match=qmodels.MatchValue(value="pesquisa")
|
|
),
|
|
qmodels.FieldCondition(
|
|
key="ano",
|
|
match=qmodels.MatchValue(value=2024)
|
|
)
|
|
]
|
|
)
|
|
|
|
# Inicializar ferramenta com filtro pré-definido
|
|
qdrant_tool = QdrantVectorSearchTool(
|
|
qdrant_config=QdrantConfig(
|
|
qdrant_url="your_url",
|
|
qdrant_api_key="your_key",
|
|
collection_name="your_collection",
|
|
filter=preset_filter # Filtro pré-definido aplicado a todas as buscas
|
|
)
|
|
)
|
|
```
|
|
|
|
### Combinando Filtros
|
|
A ferramenta combina automaticamente os filtros pré-definidos do `QdrantConfig` com os filtros dinâmicos de `filter_by` e `filter_value`:
|
|
|
|
```python
|
|
# Se QdrantConfig tem um filtro pré-definido para categoria="pesquisa"
|
|
# E a busca usa filter_by="ano", filter_value=2024
|
|
# Ambos os filtros serão combinados (lógica AND)
|
|
```
|
|
|
|
## Parâmetros de Busca
|
|
|
|
A ferramenta aceita estes parâmetros em seu schema:
|
|
- `query` (str): Consulta de busca para encontrar documentos similares
|
|
- `filter_by` (str, opcional): Campo de metadado para filtrar
|
|
- `filter_value` (Any, opcional): Valor para filtrar
|
|
|
|
## Formato de Retorno
|
|
|
|
A ferramenta retorna resultados no formato JSON:
|
|
|
|
```json
|
|
[
|
|
{
|
|
"metadata": {
|
|
// Todos os metadados armazenados junto com o documento
|
|
},
|
|
"context": "O conteúdo textual real do documento",
|
|
"distance": 0.95 // Pontuação de similaridade
|
|
}
|
|
]
|
|
```
|
|
|
|
## Embedding Padrão
|
|
|
|
Por padrão, a ferramenta utiliza o modelo `text-embedding-3-large` da OpenAI para vetorização. Isso requer:
|
|
- Chave de API da OpenAI definida na variável de ambiente: `OPENAI_API_KEY`
|
|
|
|
## Embeddings Personalizados
|
|
|
|
Em vez de utilizar o modelo padrão de embeddings, você pode utilizar sua própria função de embeddings nos casos em que:
|
|
|
|
1. Deseja usar um modelo de embeddings diferente (ex: Cohere, HuggingFace, modelos Ollama)
|
|
2. Precisa reduzir custos utilizando modelos de código aberto
|
|
3. Tem requisitos específicos quanto à dimensão dos vetores ou à qualidade dos embeddings
|
|
4. Deseja utilizar embeddings específicos para determinado domínio (ex: textos médicos ou jurídicos)
|
|
|
|
Veja um exemplo utilizando um modelo HuggingFace:
|
|
|
|
```python
|
|
from transformers import AutoTokenizer, AutoModel
|
|
import torch
|
|
|
|
# Carregar modelo e tokenizer
|
|
tokenizer = AutoTokenizer.from_pretrained('sentence-transformers/all-MiniLM-L6-v2')
|
|
model = AutoModel.from_pretrained('sentence-transformers/all-MiniLM-L6-v2')
|
|
|
|
def custom_embeddings(text: str) -> list[float]:
|
|
# Tokenizar e obter saídas do modelo
|
|
inputs = tokenizer(text, return_tensors="pt", padding=True, truncation=True)
|
|
outputs = model(**inputs)
|
|
|
|
# Usar mean pooling para obter o embedding do texto
|
|
embeddings = outputs.last_hidden_state.mean(dim=1)
|
|
|
|
# Converter para lista de floats e retornar
|
|
return embeddings[0].tolist()
|
|
|
|
# Usar embeddings personalizados com a ferramenta
|
|
from crewai_tools import QdrantConfig
|
|
|
|
tool = QdrantVectorSearchTool(
|
|
qdrant_config=QdrantConfig(
|
|
qdrant_url="your_url",
|
|
qdrant_api_key="your_key",
|
|
collection_name="your_collection"
|
|
),
|
|
custom_embedding_fn=custom_embeddings # Passe sua função personalizada
|
|
)
|
|
```
|
|
|
|
## Tratamento de Erros
|
|
|
|
A ferramenta trata os seguintes erros específicos:
|
|
- Lança ImportError se `qdrant-client` não estiver instalado (com opção de instalar automaticamente)
|
|
- Lança ValueError se `QDRANT_URL` não estiver definido
|
|
- Solicita instalação de `qdrant-client` se estiver ausente utilizando `uv add qdrant-client`
|
|
|
|
## Variáveis de Ambiente
|
|
|
|
Variáveis de ambiente obrigatórias:
|
|
```bash
|
|
export QDRANT_URL="your_qdrant_url" # Se não for informado no construtor
|
|
export QDRANT_API_KEY="your_api_key" # Se não for informado no construtor
|
|
export OPENAI_API_KEY="your_openai_key" # Se estiver usando embeddings padrão
|
|
```
|