mirror of
https://github.com/crewAIInc/crewAI.git
synced 2026-01-11 09:08:31 +00:00
feat: implement crewAI lite version with optional dependencies
- Restructure pyproject.toml to move non-essential dependencies to optional extras - Add graceful handling for missing optional dependencies in core modules - Create memory, knowledge, telemetry, visualization, auth, and llm-integrations extras - Implement helpful ImportError messages directing users to install specific extras - Add comprehensive test suite for lite installation scenarios - Maintain backward compatibility with existing installations - Support minimal core installation with Agent/Crew/Task functionality Addresses GitHub issue #3026 for lightweight crewAI installation Co-Authored-By: João <joao@crewai.com>
This commit is contained in:
119
docs/LITE_VERSION.md
Normal file
119
docs/LITE_VERSION.md
Normal file
@@ -0,0 +1,119 @@
|
||||
# CrewAI Lite Version
|
||||
|
||||
CrewAI now supports a "lite" installation with minimal dependencies, allowing you to use the core functionality without installing heavy optional dependencies.
|
||||
|
||||
## Installation
|
||||
|
||||
### Lite Installation (Minimal Dependencies)
|
||||
```bash
|
||||
pip install crewai
|
||||
```
|
||||
|
||||
This installs only the core dependencies needed for basic Agent, Crew, and Task functionality.
|
||||
|
||||
### Full Installation (All Dependencies)
|
||||
```bash
|
||||
pip install crewai[all]
|
||||
```
|
||||
|
||||
### Selective Installation (Optional Extras)
|
||||
|
||||
Install only the features you need:
|
||||
|
||||
```bash
|
||||
# Memory and knowledge storage
|
||||
pip install crewai[memory]
|
||||
|
||||
# Knowledge sources (PDF, Excel, etc.)
|
||||
pip install crewai[knowledge]
|
||||
|
||||
# Telemetry and monitoring
|
||||
pip install crewai[telemetry]
|
||||
|
||||
# Flow visualization
|
||||
pip install crewai[visualization]
|
||||
|
||||
# Authentication features
|
||||
pip install crewai[auth]
|
||||
|
||||
# Additional LLM integrations
|
||||
pip install crewai[llm-integrations]
|
||||
|
||||
# AgentOps integration
|
||||
pip install crewai[agentops]
|
||||
|
||||
# FastEmbed embeddings
|
||||
pip install crewai[embeddings]
|
||||
```
|
||||
|
||||
You can also combine multiple extras:
|
||||
```bash
|
||||
pip install crewai[memory,knowledge,telemetry]
|
||||
```
|
||||
|
||||
## Core vs Optional Features
|
||||
|
||||
### Core Features (Always Available)
|
||||
- Basic Agent, Crew, and Task functionality
|
||||
- LiteAgent for simple interactions
|
||||
- Core LLM integrations (OpenAI, etc.)
|
||||
- Basic tools and utilities
|
||||
- Process management
|
||||
|
||||
### Optional Features (Require Extras)
|
||||
|
||||
#### Memory (`crewai[memory]`)
|
||||
- RAG storage with ChromaDB
|
||||
- Memory management
|
||||
- Embeddings configuration
|
||||
|
||||
#### Knowledge (`crewai[knowledge]`)
|
||||
- PDF knowledge sources
|
||||
- Excel/spreadsheet processing
|
||||
- Document processing with Docling
|
||||
- Knowledge storage and retrieval
|
||||
|
||||
#### Telemetry (`crewai[telemetry]`)
|
||||
- OpenTelemetry integration
|
||||
- Performance monitoring
|
||||
- Usage analytics
|
||||
|
||||
#### Visualization (`crewai[visualization]`)
|
||||
- Flow visualization with Pyvis
|
||||
- Network diagrams
|
||||
|
||||
#### Authentication (`crewai[auth]`)
|
||||
- Auth0 integration
|
||||
- Secure token management
|
||||
|
||||
#### LLM Integrations (`crewai[llm-integrations]`)
|
||||
- AISuite integration
|
||||
- Additional model providers
|
||||
|
||||
## Error Handling
|
||||
|
||||
When you try to use a feature that requires optional dependencies, you'll get a helpful error message:
|
||||
|
||||
```python
|
||||
from crewai.memory.storage.rag_storage import RAGStorage
|
||||
|
||||
# Without crewai[memory] installed:
|
||||
# ImportError: ChromaDB is required for RAG storage functionality.
|
||||
# Please install it with: pip install 'crewai[memory]'
|
||||
```
|
||||
|
||||
## Migration Guide
|
||||
|
||||
Existing installations will continue to work as before. If you want to switch to the lite version:
|
||||
|
||||
1. Uninstall current crewai: `pip uninstall crewai`
|
||||
2. Install lite version: `pip install crewai`
|
||||
3. Add extras as needed: `pip install crewai[memory,knowledge]`
|
||||
|
||||
## Benefits
|
||||
|
||||
- **Reduced installation size**: Core installation is much smaller
|
||||
- **Faster installation**: Fewer dependencies to download and compile
|
||||
- **Reduced security surface**: Fewer dependencies means fewer potential vulnerabilities
|
||||
- **Flexible**: Install only what you need
|
||||
- **Backward compatible**: Existing code continues to work with full installation
|
||||
@@ -14,20 +14,8 @@ dependencies = [
|
||||
"litellm==1.72.0",
|
||||
"instructor>=1.3.3",
|
||||
# Text Processing
|
||||
"pdfplumber>=0.11.4",
|
||||
"regex>=2024.9.11",
|
||||
# Telemetry and Monitoring
|
||||
"opentelemetry-api>=1.30.0",
|
||||
"opentelemetry-sdk>=1.30.0",
|
||||
"opentelemetry-exporter-otlp-proto-http>=1.30.0",
|
||||
# Data Handling
|
||||
"chromadb>=0.5.23",
|
||||
"tokenizers>=0.20.3",
|
||||
"onnxruntime==1.22.0",
|
||||
"openpyxl>=3.1.5",
|
||||
"pyvis>=0.3.2",
|
||||
# Authentication and Security
|
||||
"auth0-python>=4.7.1",
|
||||
# Security
|
||||
"python-dotenv>=1.0.0",
|
||||
# Configuration and Utils
|
||||
"click>=8.1.7",
|
||||
@@ -68,6 +56,53 @@ docling = [
|
||||
aisuite = [
|
||||
"aisuite>=0.1.10",
|
||||
]
|
||||
memory = [
|
||||
"chromadb>=0.5.23",
|
||||
"tokenizers>=0.20.3",
|
||||
"tiktoken~=0.8.0",
|
||||
"mem0ai>=0.1.94",
|
||||
]
|
||||
knowledge = [
|
||||
"chromadb>=0.5.23",
|
||||
"pdfplumber>=0.11.4",
|
||||
"openpyxl>=3.1.5",
|
||||
"docling>=2.12.0",
|
||||
"tiktoken~=0.8.0",
|
||||
]
|
||||
telemetry = [
|
||||
"opentelemetry-api>=1.30.0",
|
||||
"opentelemetry-sdk>=1.30.0",
|
||||
"opentelemetry-exporter-otlp-proto-http>=1.30.0",
|
||||
]
|
||||
visualization = [
|
||||
"pyvis>=0.3.2",
|
||||
]
|
||||
auth = [
|
||||
"auth0-python>=4.7.1",
|
||||
]
|
||||
llm-integrations = [
|
||||
"aisuite>=0.1.10",
|
||||
"onnxruntime==1.22.0",
|
||||
]
|
||||
all = [
|
||||
"chromadb>=0.5.23",
|
||||
"tokenizers>=0.20.3",
|
||||
"tiktoken~=0.8.0",
|
||||
"mem0ai>=0.1.94",
|
||||
"pdfplumber>=0.11.4",
|
||||
"openpyxl>=3.1.5",
|
||||
"docling>=2.12.0",
|
||||
"opentelemetry-api>=1.30.0",
|
||||
"opentelemetry-sdk>=1.30.0",
|
||||
"opentelemetry-exporter-otlp-proto-http>=1.30.0",
|
||||
"pyvis>=0.3.2",
|
||||
"auth0-python>=4.7.1",
|
||||
"aisuite>=0.1.10",
|
||||
"onnxruntime==1.22.0",
|
||||
"agentops>=0.3.0",
|
||||
"crewai-tools~=0.47.1",
|
||||
"pandas>=2.2.3",
|
||||
]
|
||||
|
||||
[tool.uv]
|
||||
dev-dependencies = [
|
||||
|
||||
@@ -5,16 +5,27 @@ from datetime import datetime, timedelta
|
||||
from pathlib import Path
|
||||
from typing import Optional
|
||||
|
||||
from auth0.authentication.token_verifier import (
|
||||
AsymmetricSignatureVerifier,
|
||||
TokenVerifier,
|
||||
)
|
||||
try:
|
||||
from auth0.authentication.token_verifier import (
|
||||
AsymmetricSignatureVerifier,
|
||||
TokenVerifier,
|
||||
)
|
||||
AUTH0_AVAILABLE = True
|
||||
except ImportError:
|
||||
AUTH0_AVAILABLE = False
|
||||
AsymmetricSignatureVerifier = None
|
||||
TokenVerifier = None
|
||||
from cryptography.fernet import Fernet
|
||||
|
||||
from .constants import AUTH0_CLIENT_ID, AUTH0_DOMAIN
|
||||
|
||||
|
||||
def validate_token(id_token: str) -> None:
|
||||
if not AUTH0_AVAILABLE:
|
||||
raise ImportError(
|
||||
"Auth0 is required for authentication functionality. "
|
||||
"Please install it with: pip install 'crewai[auth]'"
|
||||
)
|
||||
"""
|
||||
Verify the token and its precedence
|
||||
|
||||
|
||||
@@ -3,7 +3,12 @@
|
||||
import os
|
||||
from pathlib import Path
|
||||
|
||||
from pyvis.network import Network
|
||||
try:
|
||||
from pyvis.network import Network
|
||||
PYVIS_AVAILABLE = True
|
||||
except ImportError:
|
||||
PYVIS_AVAILABLE = False
|
||||
Network = None
|
||||
|
||||
from crewai.flow.config import COLORS, NODE_STYLES
|
||||
from crewai.flow.html_template_handler import HTMLTemplateHandler
|
||||
@@ -63,6 +68,12 @@ class FlowPlot:
|
||||
RuntimeError
|
||||
If network visualization generation fails.
|
||||
"""
|
||||
if not PYVIS_AVAILABLE:
|
||||
raise ImportError(
|
||||
"Pyvis is required for flow visualization. "
|
||||
"Please install it with: pip install 'crewai[visualization]'"
|
||||
)
|
||||
|
||||
if not filename or not isinstance(filename, str):
|
||||
raise ValueError("Filename must be a non-empty string")
|
||||
|
||||
@@ -222,5 +233,11 @@ def plot_flow(flow, filename="flow_plot"):
|
||||
IOError
|
||||
If file operations fail.
|
||||
"""
|
||||
if not PYVIS_AVAILABLE:
|
||||
raise ImportError(
|
||||
"Pyvis is required for flow visualization. "
|
||||
"Please install it with: pip install 'crewai[visualization]'"
|
||||
)
|
||||
|
||||
visualizer = FlowPlot(flow)
|
||||
visualizer.plot(filename)
|
||||
|
||||
@@ -1,6 +1,13 @@
|
||||
from pathlib import Path
|
||||
from typing import Dict, List
|
||||
|
||||
try:
|
||||
import pdfplumber
|
||||
PDFPLUMBER_AVAILABLE = True
|
||||
except ImportError:
|
||||
PDFPLUMBER_AVAILABLE = False
|
||||
pdfplumber = None
|
||||
|
||||
from crewai.knowledge.source.base_file_knowledge_source import BaseFileKnowledgeSource
|
||||
|
||||
|
||||
@@ -26,14 +33,12 @@ class PDFKnowledgeSource(BaseFileKnowledgeSource):
|
||||
|
||||
def _import_pdfplumber(self):
|
||||
"""Dynamically import pdfplumber."""
|
||||
try:
|
||||
import pdfplumber
|
||||
|
||||
return pdfplumber
|
||||
except ImportError:
|
||||
if not PDFPLUMBER_AVAILABLE:
|
||||
raise ImportError(
|
||||
"pdfplumber is not installed. Please install it with: pip install pdfplumber"
|
||||
"pdfplumber is required for PDF knowledge sources. "
|
||||
"Please install it with: pip install 'crewai[knowledge]'"
|
||||
)
|
||||
return pdfplumber
|
||||
|
||||
def add(self) -> None:
|
||||
"""
|
||||
|
||||
@@ -6,11 +6,19 @@ import os
|
||||
import shutil
|
||||
from typing import Any, Dict, List, Optional, Union
|
||||
|
||||
import chromadb
|
||||
import chromadb.errors
|
||||
from chromadb.api import ClientAPI
|
||||
from chromadb.api.types import OneOrMany
|
||||
from chromadb.config import Settings
|
||||
try:
|
||||
import chromadb
|
||||
import chromadb.errors
|
||||
from chromadb.api import ClientAPI
|
||||
from chromadb.api.types import OneOrMany
|
||||
from chromadb.config import Settings
|
||||
CHROMADB_AVAILABLE = True
|
||||
except ImportError:
|
||||
CHROMADB_AVAILABLE = False
|
||||
chromadb = None
|
||||
ClientAPI = None
|
||||
OneOrMany = None
|
||||
Settings = None
|
||||
|
||||
from crewai.knowledge.storage.base_knowledge_storage import BaseKnowledgeStorage
|
||||
from crewai.utilities import EmbeddingConfigurator
|
||||
@@ -43,7 +51,7 @@ class KnowledgeStorage(BaseKnowledgeStorage):
|
||||
search efficiency.
|
||||
"""
|
||||
|
||||
collection: Optional[chromadb.Collection] = None
|
||||
collection: Optional[Any] = None
|
||||
collection_name: Optional[str] = "knowledge"
|
||||
app: Optional[ClientAPI] = None
|
||||
|
||||
@@ -52,6 +60,12 @@ class KnowledgeStorage(BaseKnowledgeStorage):
|
||||
embedder: Optional[Dict[str, Any]] = None,
|
||||
collection_name: Optional[str] = None,
|
||||
):
|
||||
if not CHROMADB_AVAILABLE:
|
||||
raise ImportError(
|
||||
"ChromaDB is required for knowledge storage functionality. "
|
||||
"Please install it with: pip install 'crewai[knowledge]'"
|
||||
)
|
||||
|
||||
self.collection_name = collection_name
|
||||
self._set_embedder_config(embedder)
|
||||
|
||||
@@ -181,6 +195,12 @@ class KnowledgeStorage(BaseKnowledgeStorage):
|
||||
raise
|
||||
|
||||
def _create_default_embedding_function(self):
|
||||
if not CHROMADB_AVAILABLE:
|
||||
raise ImportError(
|
||||
"ChromaDB is required for embedding functionality. "
|
||||
"Please install it with: pip install 'crewai[knowledge]'"
|
||||
)
|
||||
|
||||
from chromadb.utils.embedding_functions.openai_embedding_function import (
|
||||
OpenAIEmbeddingFunction,
|
||||
)
|
||||
|
||||
12
src/crewai/llms/third_party/ai_suite.py
vendored
12
src/crewai/llms/third_party/ai_suite.py
vendored
@@ -1,12 +1,22 @@
|
||||
from typing import Any, Dict, List, Optional, Union
|
||||
|
||||
import aisuite as ai
|
||||
try:
|
||||
import aisuite as ai
|
||||
AISUITE_AVAILABLE = True
|
||||
except ImportError:
|
||||
AISUITE_AVAILABLE = False
|
||||
ai = None
|
||||
|
||||
from crewai.llms.base_llm import BaseLLM
|
||||
|
||||
|
||||
class AISuiteLLM(BaseLLM):
|
||||
def __init__(self, model: str, temperature: Optional[float] = None, **kwargs):
|
||||
if not AISUITE_AVAILABLE:
|
||||
raise ImportError(
|
||||
"AISuite is required for AISuiteLLM. "
|
||||
"Please install it with: pip install 'crewai[llm-integrations]'"
|
||||
)
|
||||
super().__init__(model, temperature, **kwargs)
|
||||
self.client = ai.Client()
|
||||
|
||||
|
||||
@@ -6,7 +6,12 @@ import shutil
|
||||
import uuid
|
||||
from typing import Any, Dict, List, Optional
|
||||
|
||||
from chromadb.api import ClientAPI
|
||||
try:
|
||||
from chromadb.api import ClientAPI
|
||||
CHROMADB_AVAILABLE = True
|
||||
except ImportError:
|
||||
CHROMADB_AVAILABLE = False
|
||||
ClientAPI = None
|
||||
|
||||
from crewai.memory.storage.base_rag_storage import BaseRAGStorage
|
||||
from crewai.utilities import EmbeddingConfigurator
|
||||
@@ -37,11 +42,17 @@ class RAGStorage(BaseRAGStorage):
|
||||
search efficiency.
|
||||
"""
|
||||
|
||||
app: ClientAPI | None = None
|
||||
app: Optional[Any] = None
|
||||
|
||||
def __init__(
|
||||
self, type, allow_reset=True, embedder_config=None, crew=None, path=None
|
||||
):
|
||||
if not CHROMADB_AVAILABLE:
|
||||
raise ImportError(
|
||||
"ChromaDB is required for RAG storage functionality. "
|
||||
"Please install it with: pip install 'crewai[memory]'"
|
||||
)
|
||||
|
||||
super().__init__(type, allow_reset, embedder_config, crew)
|
||||
agents = crew.agents if crew else []
|
||||
agents = [self._sanitize_role(agent.role) for agent in agents]
|
||||
@@ -60,6 +71,12 @@ class RAGStorage(BaseRAGStorage):
|
||||
self.embedder_config = configurator.configure_embedder(self.embedder_config)
|
||||
|
||||
def _initialize_app(self):
|
||||
if not CHROMADB_AVAILABLE:
|
||||
raise ImportError(
|
||||
"ChromaDB is required for RAG storage functionality. "
|
||||
"Please install it with: pip install 'crewai[memory]'"
|
||||
)
|
||||
|
||||
import chromadb
|
||||
from chromadb.config import Settings
|
||||
|
||||
@@ -165,6 +182,12 @@ class RAGStorage(BaseRAGStorage):
|
||||
)
|
||||
|
||||
def _create_default_embedding_function(self):
|
||||
if not CHROMADB_AVAILABLE:
|
||||
raise ImportError(
|
||||
"ChromaDB is required for embedding functionality. "
|
||||
"Please install it with: pip install 'crewai[memory]'"
|
||||
)
|
||||
|
||||
from chromadb.utils.embedding_functions.openai_embedding_function import (
|
||||
OpenAIEmbeddingFunction,
|
||||
)
|
||||
|
||||
@@ -11,17 +11,31 @@ from importlib.metadata import version
|
||||
from typing import TYPE_CHECKING, Any, Callable, Optional
|
||||
import threading
|
||||
|
||||
from opentelemetry import trace
|
||||
from opentelemetry.exporter.otlp.proto.http.trace_exporter import (
|
||||
OTLPSpanExporter,
|
||||
)
|
||||
from opentelemetry.sdk.resources import SERVICE_NAME, Resource
|
||||
from opentelemetry.sdk.trace import TracerProvider
|
||||
from opentelemetry.sdk.trace.export import (
|
||||
BatchSpanProcessor,
|
||||
SpanExportResult,
|
||||
)
|
||||
from opentelemetry.trace import Span, Status, StatusCode
|
||||
try:
|
||||
from opentelemetry import trace
|
||||
from opentelemetry.exporter.otlp.proto.http.trace_exporter import (
|
||||
OTLPSpanExporter,
|
||||
)
|
||||
from opentelemetry.sdk.resources import SERVICE_NAME, Resource
|
||||
from opentelemetry.sdk.trace import TracerProvider
|
||||
from opentelemetry.sdk.trace.export import (
|
||||
BatchSpanProcessor,
|
||||
SpanExportResult,
|
||||
)
|
||||
from opentelemetry.trace import Span, Status, StatusCode
|
||||
OPENTELEMETRY_AVAILABLE = True
|
||||
except ImportError:
|
||||
OPENTELEMETRY_AVAILABLE = False
|
||||
trace = None
|
||||
OTLPSpanExporter = None
|
||||
SERVICE_NAME = None
|
||||
Resource = None
|
||||
TracerProvider = None
|
||||
BatchSpanProcessor = None
|
||||
SpanExportResult = None
|
||||
Span = None
|
||||
Status = None
|
||||
StatusCode = None
|
||||
|
||||
from crewai.telemetry.constants import (
|
||||
CREWAI_TELEMETRY_BASE_URL,
|
||||
@@ -43,13 +57,21 @@ if TYPE_CHECKING:
|
||||
from crewai.task import Task
|
||||
|
||||
|
||||
class SafeOTLPSpanExporter(OTLPSpanExporter):
|
||||
def export(self, spans) -> SpanExportResult:
|
||||
class SafeOTLPSpanExporter:
|
||||
def __init__(self, *args, **kwargs):
|
||||
if OPENTELEMETRY_AVAILABLE:
|
||||
self._exporter = OTLPSpanExporter(*args, **kwargs)
|
||||
else:
|
||||
self._exporter = None
|
||||
|
||||
def export(self, spans):
|
||||
if not OPENTELEMETRY_AVAILABLE or not self._exporter:
|
||||
return None
|
||||
try:
|
||||
return super().export(spans)
|
||||
return self._exporter.export(spans)
|
||||
except Exception as e:
|
||||
logger.error(e)
|
||||
return SpanExportResult.FAILURE
|
||||
return SpanExportResult.FAILURE if SpanExportResult else None
|
||||
|
||||
|
||||
class Telemetry:
|
||||
@@ -84,7 +106,7 @@ class Telemetry:
|
||||
self.trace_set: bool = False
|
||||
self._initialized: bool = True
|
||||
|
||||
if self._is_telemetry_disabled():
|
||||
if self._is_telemetry_disabled() or not OPENTELEMETRY_AVAILABLE:
|
||||
return
|
||||
|
||||
try:
|
||||
@@ -116,11 +138,12 @@ class Telemetry:
|
||||
return (
|
||||
os.getenv("OTEL_SDK_DISABLED", "false").lower() == "true"
|
||||
or os.getenv("CREWAI_DISABLE_TELEMETRY", "false").lower() == "true"
|
||||
or not OPENTELEMETRY_AVAILABLE
|
||||
)
|
||||
|
||||
def _should_execute_telemetry(self) -> bool:
|
||||
"""Check if telemetry operations should be executed."""
|
||||
return self.ready and not self._is_telemetry_disabled()
|
||||
return self.ready and not self._is_telemetry_disabled() and OPENTELEMETRY_AVAILABLE
|
||||
|
||||
def set_tracer(self):
|
||||
if self.ready and not self.trace_set:
|
||||
|
||||
@@ -1,8 +1,15 @@
|
||||
import os
|
||||
from typing import Any, Dict, Optional, cast
|
||||
|
||||
from chromadb import Documents, EmbeddingFunction, Embeddings
|
||||
from chromadb.api.types import validate_embedding_function
|
||||
try:
|
||||
from chromadb import Documents, EmbeddingFunction, Embeddings
|
||||
from chromadb.api.types import validate_embedding_function
|
||||
CHROMADB_AVAILABLE = True
|
||||
except ImportError:
|
||||
CHROMADB_AVAILABLE = False
|
||||
Documents = None
|
||||
EmbeddingFunction = None
|
||||
Embeddings = None
|
||||
|
||||
|
||||
class EmbeddingConfigurator:
|
||||
@@ -25,6 +32,11 @@ class EmbeddingConfigurator:
|
||||
self,
|
||||
embedder_config: Optional[Dict[str, Any]] = None,
|
||||
) -> EmbeddingFunction:
|
||||
if not CHROMADB_AVAILABLE:
|
||||
raise ImportError(
|
||||
"ChromaDB is required for embedding configuration. "
|
||||
"Please install it with: pip install 'crewai[memory]' or 'crewai[knowledge]'"
|
||||
)
|
||||
"""Configures and returns an embedding function based on the provided config."""
|
||||
if embedder_config is None:
|
||||
return self._create_default_embedding_function()
|
||||
@@ -47,6 +59,12 @@ class EmbeddingConfigurator:
|
||||
|
||||
@staticmethod
|
||||
def _create_default_embedding_function():
|
||||
if not CHROMADB_AVAILABLE:
|
||||
raise ImportError(
|
||||
"ChromaDB is required for embedding functionality. "
|
||||
"Please install it with: pip install 'crewai[memory]' or 'crewai[knowledge]'"
|
||||
)
|
||||
|
||||
from chromadb.utils.embedding_functions.openai_embedding_function import (
|
||||
OpenAIEmbeddingFunction,
|
||||
)
|
||||
@@ -57,6 +75,12 @@ class EmbeddingConfigurator:
|
||||
|
||||
@staticmethod
|
||||
def _configure_openai(config, model_name):
|
||||
if not CHROMADB_AVAILABLE:
|
||||
raise ImportError(
|
||||
"ChromaDB is required for OpenAI embedding configuration. "
|
||||
"Please install it with: pip install 'crewai[memory]' or 'crewai[knowledge]'"
|
||||
)
|
||||
|
||||
from chromadb.utils.embedding_functions.openai_embedding_function import (
|
||||
OpenAIEmbeddingFunction,
|
||||
)
|
||||
@@ -75,6 +99,12 @@ class EmbeddingConfigurator:
|
||||
|
||||
@staticmethod
|
||||
def _configure_azure(config, model_name):
|
||||
if not CHROMADB_AVAILABLE:
|
||||
raise ImportError(
|
||||
"ChromaDB is required for Azure embedding configuration. "
|
||||
"Please install it with: pip install 'crewai[memory]' or 'crewai[knowledge]'"
|
||||
)
|
||||
|
||||
from chromadb.utils.embedding_functions.openai_embedding_function import (
|
||||
OpenAIEmbeddingFunction,
|
||||
)
|
||||
@@ -93,6 +123,12 @@ class EmbeddingConfigurator:
|
||||
|
||||
@staticmethod
|
||||
def _configure_ollama(config, model_name):
|
||||
if not CHROMADB_AVAILABLE:
|
||||
raise ImportError(
|
||||
"ChromaDB is required for Ollama embedding configuration. "
|
||||
"Please install it with: pip install 'crewai[memory]' or 'crewai[knowledge]'"
|
||||
)
|
||||
|
||||
from chromadb.utils.embedding_functions.ollama_embedding_function import (
|
||||
OllamaEmbeddingFunction,
|
||||
)
|
||||
@@ -104,6 +140,12 @@ class EmbeddingConfigurator:
|
||||
|
||||
@staticmethod
|
||||
def _configure_vertexai(config, model_name):
|
||||
if not CHROMADB_AVAILABLE:
|
||||
raise ImportError(
|
||||
"ChromaDB is required for VertexAI embedding configuration. "
|
||||
"Please install it with: pip install 'crewai[memory]' or 'crewai[knowledge]'"
|
||||
)
|
||||
|
||||
from chromadb.utils.embedding_functions.google_embedding_function import (
|
||||
GoogleVertexEmbeddingFunction,
|
||||
)
|
||||
@@ -117,6 +159,12 @@ class EmbeddingConfigurator:
|
||||
|
||||
@staticmethod
|
||||
def _configure_google(config, model_name):
|
||||
if not CHROMADB_AVAILABLE:
|
||||
raise ImportError(
|
||||
"ChromaDB is required for Google embedding configuration. "
|
||||
"Please install it with: pip install 'crewai[memory]' or 'crewai[knowledge]'"
|
||||
)
|
||||
|
||||
from chromadb.utils.embedding_functions.google_embedding_function import (
|
||||
GoogleGenerativeAiEmbeddingFunction,
|
||||
)
|
||||
@@ -129,6 +177,12 @@ class EmbeddingConfigurator:
|
||||
|
||||
@staticmethod
|
||||
def _configure_cohere(config, model_name):
|
||||
if not CHROMADB_AVAILABLE:
|
||||
raise ImportError(
|
||||
"ChromaDB is required for Cohere embedding configuration. "
|
||||
"Please install it with: pip install 'crewai[memory]' or 'crewai[knowledge]'"
|
||||
)
|
||||
|
||||
from chromadb.utils.embedding_functions.cohere_embedding_function import (
|
||||
CohereEmbeddingFunction,
|
||||
)
|
||||
@@ -234,3 +288,11 @@ class EmbeddingConfigurator:
|
||||
raise ValueError(
|
||||
"Custom embedder must be an instance of `EmbeddingFunction` or a callable that creates one"
|
||||
)
|
||||
|
||||
def validate_embedder_config(self, config: EmbeddingFunction) -> EmbeddingFunction:
|
||||
if not CHROMADB_AVAILABLE:
|
||||
raise ImportError(
|
||||
"ChromaDB is required for embedding validation. "
|
||||
"Please install it with: pip install 'crewai[memory]' or 'crewai[knowledge]'"
|
||||
)
|
||||
return cast(EmbeddingFunction, validate_embedding_function(config))
|
||||
|
||||
193
tests/test_lite_installation.py
Normal file
193
tests/test_lite_installation.py
Normal file
@@ -0,0 +1,193 @@
|
||||
"""Test that crewAI lite installation works with minimal dependencies."""
|
||||
|
||||
import pytest
|
||||
import subprocess
|
||||
import sys
|
||||
import tempfile
|
||||
import os
|
||||
from pathlib import Path
|
||||
from unittest.mock import patch, Mock
|
||||
|
||||
|
||||
def test_core_imports_work_without_optional_deps():
|
||||
"""Test that core crewAI functionality can be imported without optional dependencies."""
|
||||
|
||||
try:
|
||||
from crewai import Agent, Crew, Task, LLM
|
||||
from crewai.lite_agent import LiteAgent
|
||||
from crewai.process import Process
|
||||
assert Agent is not None
|
||||
assert Crew is not None
|
||||
assert Task is not None
|
||||
assert LLM is not None
|
||||
assert LiteAgent is not None
|
||||
assert Process is not None
|
||||
except ImportError as e:
|
||||
pytest.fail(f"Core imports should work without optional dependencies: {e}")
|
||||
|
||||
|
||||
def test_optional_memory_import_error():
|
||||
"""Test that memory functionality raises helpful error without chromadb."""
|
||||
with patch.dict('sys.modules', {'chromadb': None}):
|
||||
with patch('crewai.memory.storage.rag_storage.CHROMADB_AVAILABLE', False):
|
||||
from crewai.memory.storage.rag_storage import RAGStorage
|
||||
|
||||
with pytest.raises(ImportError) as exc_info:
|
||||
RAGStorage("test")
|
||||
|
||||
assert "ChromaDB is required" in str(exc_info.value)
|
||||
assert "crewai[memory]" in str(exc_info.value)
|
||||
|
||||
|
||||
def test_optional_knowledge_import_error():
|
||||
"""Test that knowledge functionality raises helpful error without dependencies."""
|
||||
with patch.dict('sys.modules', {'chromadb': None}):
|
||||
with patch('crewai.knowledge.storage.knowledge_storage.CHROMADB_AVAILABLE', False):
|
||||
from crewai.knowledge.storage.knowledge_storage import KnowledgeStorage
|
||||
|
||||
with pytest.raises(ImportError) as exc_info:
|
||||
KnowledgeStorage()
|
||||
|
||||
assert "ChromaDB is required" in str(exc_info.value)
|
||||
assert "crewai[knowledge]" in str(exc_info.value)
|
||||
|
||||
|
||||
def test_optional_pdf_import_error():
|
||||
"""Test that PDF knowledge source raises helpful error without pdfplumber."""
|
||||
with patch.dict('sys.modules', {'pdfplumber': None}):
|
||||
with patch('crewai.knowledge.source.pdf_knowledge_source.PDFPLUMBER_AVAILABLE', False):
|
||||
from crewai.knowledge.source.pdf_knowledge_source import PDFKnowledgeSource
|
||||
|
||||
knowledge_dir = Path("knowledge/tmp")
|
||||
knowledge_dir.mkdir(parents=True, exist_ok=True)
|
||||
|
||||
test_file = knowledge_dir / "test.pdf"
|
||||
test_file.touch()
|
||||
|
||||
try:
|
||||
with pytest.raises(ImportError) as exc_info:
|
||||
PDFKnowledgeSource(file_paths=["tmp/test.pdf"])
|
||||
|
||||
assert "pdfplumber is required" in str(exc_info.value)
|
||||
assert "crewai[knowledge]" in str(exc_info.value)
|
||||
finally:
|
||||
if test_file.exists():
|
||||
test_file.unlink()
|
||||
if knowledge_dir.exists() and not any(knowledge_dir.iterdir()):
|
||||
knowledge_dir.rmdir()
|
||||
|
||||
|
||||
def test_optional_visualization_import_error():
|
||||
"""Test that flow visualization raises helpful error without pyvis."""
|
||||
with patch.dict('sys.modules', {'pyvis': None}):
|
||||
with patch('crewai.flow.flow_visualizer.PYVIS_AVAILABLE', False):
|
||||
from crewai.flow.flow_visualizer import plot_flow
|
||||
|
||||
mock_flow = Mock()
|
||||
|
||||
with pytest.raises(ImportError) as exc_info:
|
||||
plot_flow(mock_flow)
|
||||
|
||||
assert "Pyvis is required" in str(exc_info.value)
|
||||
assert "crewai[visualization]" in str(exc_info.value)
|
||||
|
||||
|
||||
def test_telemetry_disabled_without_opentelemetry():
|
||||
"""Test that telemetry is disabled gracefully without opentelemetry."""
|
||||
from crewai.telemetry.telemetry import Telemetry
|
||||
|
||||
telemetry = Telemetry()
|
||||
|
||||
assert isinstance(telemetry.ready, bool)
|
||||
assert isinstance(telemetry._is_telemetry_disabled(), bool)
|
||||
|
||||
|
||||
def test_lite_agent_works_without_optional_deps():
|
||||
"""Test that LiteAgent can be created and used without optional dependencies."""
|
||||
from crewai.lite_agent import LiteAgent
|
||||
from crewai import LLM
|
||||
from unittest.mock import Mock
|
||||
|
||||
mock_llm = Mock(spec=LLM)
|
||||
mock_llm.call.return_value = "Test response"
|
||||
mock_llm.model = "test-model"
|
||||
|
||||
agent = LiteAgent(
|
||||
role="Test Agent",
|
||||
goal="Test Goal",
|
||||
backstory="Test Backstory",
|
||||
llm=mock_llm,
|
||||
verbose=False
|
||||
)
|
||||
|
||||
assert agent.role == "Test Agent"
|
||||
assert agent.goal == "Test Goal"
|
||||
assert agent.backstory == "Test Backstory"
|
||||
|
||||
|
||||
def test_basic_crew_creation_without_optional_deps():
|
||||
"""Test that basic Crew can be created without optional dependencies."""
|
||||
from crewai import Agent, Crew, Task, LLM
|
||||
from unittest.mock import Mock
|
||||
|
||||
mock_llm = Mock(spec=LLM)
|
||||
mock_llm.call.return_value = "Test response"
|
||||
mock_llm.model = "test-model"
|
||||
|
||||
agent = Agent(
|
||||
role="Test Agent",
|
||||
goal="Test Goal",
|
||||
backstory="Test Backstory",
|
||||
llm=mock_llm,
|
||||
verbose=False
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Test task",
|
||||
agent=agent,
|
||||
expected_output="Test output"
|
||||
)
|
||||
|
||||
crew = Crew(
|
||||
agents=[agent],
|
||||
tasks=[task],
|
||||
verbose=False
|
||||
)
|
||||
|
||||
assert crew.agents[0].role == "Test Agent"
|
||||
assert crew.tasks[0].description == "Test task"
|
||||
|
||||
|
||||
def test_core_functionality_without_optional_deps():
|
||||
"""Test that core crewAI functionality works without optional dependencies."""
|
||||
from crewai import Agent, Task, Crew, LLM
|
||||
from unittest.mock import Mock
|
||||
|
||||
mock_llm = Mock(spec=LLM)
|
||||
mock_llm.call.return_value = "Test response"
|
||||
mock_llm.model = "test-model"
|
||||
|
||||
agent = Agent(
|
||||
role="Test Agent",
|
||||
goal="Test Goal",
|
||||
backstory="Test Backstory",
|
||||
llm=mock_llm,
|
||||
verbose=False
|
||||
)
|
||||
|
||||
task = Task(
|
||||
description="Test task description",
|
||||
agent=agent,
|
||||
expected_output="Test expected output"
|
||||
)
|
||||
|
||||
crew = Crew(
|
||||
agents=[agent],
|
||||
tasks=[task],
|
||||
verbose=False
|
||||
)
|
||||
|
||||
assert agent.role == "Test Agent"
|
||||
assert task.description == "Test task description"
|
||||
assert len(crew.agents) == 1
|
||||
assert len(crew.tasks) == 1
|
||||
122
tests/test_optional_dependencies.py
Normal file
122
tests/test_optional_dependencies.py
Normal file
@@ -0,0 +1,122 @@
|
||||
"""Test optional dependency handling for crewAI lite version."""
|
||||
|
||||
import pytest
|
||||
from unittest.mock import patch, Mock
|
||||
import sys
|
||||
|
||||
|
||||
class TestOptionalDependencies:
|
||||
"""Test that optional dependencies are handled gracefully."""
|
||||
|
||||
def test_chromadb_import_error_memory(self):
|
||||
"""Test that memory functionality raises helpful error without chromadb."""
|
||||
with patch.dict('sys.modules', {'chromadb': None}):
|
||||
with patch('crewai.memory.storage.rag_storage.CHROMADB_AVAILABLE', False):
|
||||
from crewai.memory.storage.rag_storage import RAGStorage
|
||||
|
||||
with pytest.raises(ImportError) as exc_info:
|
||||
RAGStorage("test")
|
||||
|
||||
assert "ChromaDB is required" in str(exc_info.value)
|
||||
assert "crewai[memory]" in str(exc_info.value)
|
||||
|
||||
def test_chromadb_import_error_knowledge(self):
|
||||
"""Test that knowledge functionality raises helpful error without chromadb."""
|
||||
with patch.dict('sys.modules', {'chromadb': None}):
|
||||
with patch('crewai.knowledge.storage.knowledge_storage.CHROMADB_AVAILABLE', False):
|
||||
from crewai.knowledge.storage.knowledge_storage import KnowledgeStorage
|
||||
|
||||
with pytest.raises(ImportError) as exc_info:
|
||||
KnowledgeStorage()
|
||||
|
||||
assert "ChromaDB is required" in str(exc_info.value)
|
||||
assert "crewai[knowledge]" in str(exc_info.value)
|
||||
|
||||
def test_pdfplumber_import_error(self):
|
||||
"""Test that PDF knowledge source raises helpful error without pdfplumber."""
|
||||
with patch.dict('sys.modules', {'pdfplumber': None}):
|
||||
from crewai.knowledge.source.pdf_knowledge_source import PDFKnowledgeSource
|
||||
|
||||
pdf_source = PDFKnowledgeSource(file_paths=["test.pdf"])
|
||||
|
||||
with pytest.raises(ImportError) as exc_info:
|
||||
pdf_source._import_pdfplumber()
|
||||
|
||||
assert "pdfplumber is required" in str(exc_info.value)
|
||||
assert "crewai[knowledge]" in str(exc_info.value)
|
||||
|
||||
def test_pyvis_import_error(self):
|
||||
"""Test that flow visualization raises helpful error without pyvis."""
|
||||
with patch.dict('sys.modules', {'pyvis': None}):
|
||||
with patch('crewai.flow.flow_visualizer.PYVIS_AVAILABLE', False):
|
||||
from crewai.flow.flow_visualizer import plot_flow
|
||||
|
||||
mock_flow = Mock()
|
||||
|
||||
with pytest.raises(ImportError) as exc_info:
|
||||
plot_flow(mock_flow)
|
||||
|
||||
assert "Pyvis is required" in str(exc_info.value)
|
||||
assert "crewai[visualization]" in str(exc_info.value)
|
||||
|
||||
def test_auth0_import_error(self):
|
||||
"""Test that authentication raises helpful error without auth0."""
|
||||
with patch.dict('sys.modules', {'auth0': None}):
|
||||
with patch('crewai.cli.authentication.utils.AUTH0_AVAILABLE', False):
|
||||
from crewai.cli.authentication.utils import validate_token
|
||||
|
||||
with pytest.raises(ImportError) as exc_info:
|
||||
validate_token("fake_token")
|
||||
|
||||
assert "Auth0 is required" in str(exc_info.value)
|
||||
assert "crewai[auth]" in str(exc_info.value)
|
||||
|
||||
def test_aisuite_import_error(self):
|
||||
"""Test that AISuite LLM raises helpful error without aisuite."""
|
||||
with patch.dict('sys.modules', {'aisuite': None}):
|
||||
with patch('crewai.llms.third_party.ai_suite.AISUITE_AVAILABLE', False):
|
||||
from crewai.llms.third_party.ai_suite import AISuiteLLM
|
||||
|
||||
with pytest.raises(ImportError) as exc_info:
|
||||
AISuiteLLM("test-model")
|
||||
|
||||
assert "AISuite is required" in str(exc_info.value)
|
||||
assert "crewai[llm-integrations]" in str(exc_info.value)
|
||||
|
||||
def test_opentelemetry_graceful_degradation(self):
|
||||
"""Test that telemetry degrades gracefully without opentelemetry."""
|
||||
with patch.dict('sys.modules', {'opentelemetry': None}):
|
||||
with patch('crewai.telemetry.telemetry.OPENTELEMETRY_AVAILABLE', False):
|
||||
from crewai.telemetry.telemetry import Telemetry
|
||||
|
||||
telemetry = Telemetry()
|
||||
|
||||
assert not telemetry.ready
|
||||
assert telemetry._is_telemetry_disabled()
|
||||
assert not telemetry._should_execute_telemetry()
|
||||
|
||||
def test_embedding_configurator_import_error(self):
|
||||
"""Test that embedding configurator raises helpful error without chromadb."""
|
||||
with patch.dict('sys.modules', {'chromadb': None}):
|
||||
with patch('crewai.utilities.embedding_configurator.CHROMADB_AVAILABLE', False):
|
||||
from crewai.utilities.embedding_configurator import EmbeddingConfigurator
|
||||
|
||||
configurator = EmbeddingConfigurator()
|
||||
|
||||
with pytest.raises(ImportError) as exc_info:
|
||||
configurator.configure_embedder(None)
|
||||
|
||||
assert "ChromaDB is required" in str(exc_info.value)
|
||||
assert "crewai[memory]" in str(exc_info.value) or "crewai[knowledge]" in str(exc_info.value)
|
||||
|
||||
def test_docling_import_error(self):
|
||||
"""Test that docling knowledge source raises helpful error without docling."""
|
||||
with patch.dict('sys.modules', {'docling': None}):
|
||||
with patch('crewai.knowledge.source.crew_docling_source.DOCLING_AVAILABLE', False):
|
||||
from crewai.knowledge.source.crew_docling_source import CrewDoclingSource
|
||||
|
||||
with pytest.raises(ImportError) as exc_info:
|
||||
CrewDoclingSource()
|
||||
|
||||
assert "docling package is required" in str(exc_info.value)
|
||||
assert "uv add docling" in str(exc_info.value)
|
||||
Reference in New Issue
Block a user