Compare commits

..

3 Commits

Author SHA1 Message Date
Devin AI
575d1c729d Fix CI test failures: regex pattern, mock patches, method signature
- Update regex pattern to handle Docker registry URLs with port numbers
- Fix mock patches to target correct import path (crewai_tools.CodeInterpreterTool)
- Fix integration test method signature for Crew._prepare_tools()
- All Docker image formats now validate correctly including my-registry.com:5000/python:latest

Co-Authored-By: João <joao@crewai.com>
2025-06-02 18:29:16 +00:00
Devin AI
45404537fd Address PR feedback: Add validation, refactor code duplication, enhance tests
- Add regex pattern validation for execution_image parameter
- Refactor get_code_execution_tools() to eliminate code duplication using tool_kwargs
- Add comprehensive tests for invalid Docker image format validation
- Add tests for various valid Docker image formats
- Update documentation with best practices section for custom Docker images
- Fix lint issues: remove unused imports and variables

Addresses feedback from joaomdmoura in PR #2934

Co-Authored-By: João <joao@crewai.com>
2025-06-02 18:23:11 +00:00
Devin AI
229bbd9bbe Add execution_image parameter to Agent for custom Docker images
- Add execution_image field to Agent class for specifying custom Docker images
- Modify get_code_execution_tools() to pass custom image to CodeInterpreterTool
- Add comprehensive tests for the new functionality
- Update documentation with usage examples
- Fixes #2933: Allow customizing CrewAI code executor Docker image

Co-Authored-By: João <joao@crewai.com>
2025-06-02 18:18:14 +00:00
10 changed files with 1236 additions and 1849 deletions

View File

@@ -14,7 +14,7 @@ jobs:
timeout-minutes: 15
strategy:
matrix:
python-version: ['3.10', '3.11', '3.12', '3.13']
python-version: ['3.10', '3.11', '3.12']
steps:
- name: Checkout code
uses: actions/checkout@v4
@@ -28,9 +28,7 @@ jobs:
run: uv python install ${{ matrix.python-version }}
- name: Install the project
run: |
uv sync --dev --all-extras
uv pip uninstall pytest-vcr --quiet || true
run: uv sync --dev --all-extras
- name: Run tests
run: uv run pytest --block-network --timeout=60 -vv

View File

@@ -22,7 +22,7 @@ Watch this video tutorial for a step-by-step demonstration of the installation p
<Note>
**Python Version Requirements**
CrewAI requires `Python >=3.10 and <=3.13`. Here's how to check your version:
CrewAI requires `Python >=3.10 and <3.13`. Here's how to check your version:
```bash
python3 --version
```

File diff suppressed because it is too large Load Diff

View File

@@ -92,6 +92,46 @@ programmer_agent = Agent(
)
```
You can also specify a custom Docker image for code execution:
```python Code
from crewai import Agent
# Create an agent with code execution enabled and custom Docker image
programmer_agent = Agent(
role="Python Programmer",
goal="Write and execute Python code to solve problems",
backstory="An expert Python programmer who can write efficient code to solve complex problems.",
allow_code_execution=True, # This automatically adds the CodeInterpreterTool
execution_image="python:3.11-slim", # Custom Docker image with specific Python version
verbose=True,
)
```
### Best Practices for Custom Docker Images
When using custom Docker images for code execution, consider the following best practices:
- **Use specific version tags**: Instead of `latest`, use specific version tags like `python:3.11-slim` for reproducible builds
- **Security scanning**: Ensure your custom images are regularly scanned for security vulnerabilities
- **Image size optimization**: Consider using slim or alpine variants to reduce image size and improve performance
- **Pre-installed dependencies**: Include commonly used libraries in your custom image to avoid repeated installations
- **Registry accessibility**: Ensure the Docker registry hosting your custom image is accessible from your execution environment
Example with a custom image containing data science libraries:
```python Code
# Custom image with pre-installed data science packages
data_scientist_agent = Agent(
role="Data Scientist",
goal="Analyze datasets and create visualizations",
backstory="Expert in data analysis with access to specialized tools",
allow_code_execution=True,
execution_image="my-registry.com/datascience:python3.11-pandas-numpy",
verbose=True,
)
```
### Enabling `unsafe_mode`
```python Code

View File

@@ -3,7 +3,7 @@ name = "crewai"
version = "0.121.1"
description = "Cutting-edge framework for orchestrating role-playing, autonomous AI agents. By fostering collaborative intelligence, CrewAI empowers agents to work together seamlessly, tackling complex tasks."
readme = "README.md"
requires-python = ">=3.10,<3.14"
requires-python = ">=3.10,<3.13"
authors = [
{ name = "Joao Moura", email = "joao@crewai.com" }
]
@@ -22,8 +22,6 @@ dependencies = [
"opentelemetry-exporter-otlp-proto-http>=1.30.0",
# Data Handling
"chromadb>=0.5.23",
"tokenizers>=0.20.3",
"onnxruntime==1.22.0",
"openpyxl>=3.1.5",
"pyvis>=0.3.2",
# Authentication and Security
@@ -49,9 +47,10 @@ Repository = "https://github.com/crewAIInc/crewAI"
[project.optional-dependencies]
tools = ["crewai-tools~=0.45.0"]
embeddings = [
"tiktoken~=0.8.0"
"tiktoken~=0.7.0"
]
agentops = ["agentops>=0.3.0"]
fastembed = ["fastembed>=0.4.1"]
pdfplumber = [
"pdfplumber>=0.11.4",
]
@@ -89,9 +88,6 @@ dev-dependencies = [
"pytest-randomly>=3.16.0",
"pytest-timeout>=2.3.1",
]
constraint-dependencies = [
"pytest-vcr; python_version>='0'",
]
[project.scripts]
crewai = "crewai.cli.cli:crewai"
@@ -104,38 +100,6 @@ exclude = ["cli/templates"]
[tool.bandit]
exclude_dirs = ["src/crewai/cli/templates"]
# PyTorch index configuration, since torch 2.5.0 is not compatible with python 3.13
[[tool.uv.index]]
name = "pytorch-nightly"
url = "https://download.pytorch.org/whl/nightly/cpu"
explicit = true
[[tool.uv.index]]
name = "pytorch"
url = "https://download.pytorch.org/whl/cpu"
explicit = true
[tool.uv.sources]
torch = [
{ index = "pytorch-nightly", marker = "python_version >= '3.13'" },
{ index = "pytorch", marker = "python_version < '3.13'" },
]
torchvision = [
{ index = "pytorch-nightly", marker = "python_version >= '3.13'" },
{ index = "pytorch", marker = "python_version < '3.13'" },
]
[tool.pytest.ini_options]
testpaths = ["tests"]
python_files = "test_*.py"
python_classes = "Test*"
python_functions = "test_*"
addopts = "--strict-markers --disable-warnings --tb=short"
markers = [
"vcr: marks tests as using VCR.py for HTTP request recording",
]
[build-system]
requires = ["hatchling"]
build-backend = "hatchling.build"

View File

@@ -127,6 +127,11 @@ class Agent(BaseAgent):
default="safe",
description="Mode for code execution: 'safe' (using Docker) or 'unsafe' (direct execution).",
)
execution_image: Optional[str] = Field(
default=None,
description="Custom Docker image to use for code execution. If not specified, uses the default image.",
pattern=r"^[a-zA-Z0-9._/-]+(?:\:[0-9]+)?(?:/[a-zA-Z0-9._/-]+)*(?:\:[a-zA-Z0-9._-]+)?$",
)
reasoning: bool = Field(
default=False,
description="Whether the agent should reflect and create a plan before executing a task.",
@@ -564,7 +569,11 @@ class Agent(BaseAgent):
# Set the unsafe_mode based on the code_execution_mode attribute
unsafe_mode = self.code_execution_mode == "unsafe"
return [CodeInterpreterTool(unsafe_mode=unsafe_mode)]
tool_kwargs = {"unsafe_mode": unsafe_mode}
if self.execution_image:
tool_kwargs["default_image_tag"] = self.execution_image
return [CodeInterpreterTool(**tool_kwargs)]
except ModuleNotFoundError:
self._logger.log(
"info", "Coding tools not available. Install crewai_tools. "

View File

@@ -0,0 +1,93 @@
from pathlib import Path
from typing import List, Optional, Union
import numpy as np
from .base_embedder import BaseEmbedder
try:
from fastembed_gpu import TextEmbedding # type: ignore
FASTEMBED_AVAILABLE = True
except ImportError:
try:
from fastembed import TextEmbedding
FASTEMBED_AVAILABLE = True
except ImportError:
FASTEMBED_AVAILABLE = False
class FastEmbed(BaseEmbedder):
"""
A wrapper class for text embedding models using FastEmbed
"""
def __init__(
self,
model_name: str = "BAAI/bge-small-en-v1.5",
cache_dir: Optional[Union[str, Path]] = None,
):
"""
Initialize the embedding model
Args:
model_name: Name of the model to use
cache_dir: Directory to cache the model
gpu: Whether to use GPU acceleration
"""
if not FASTEMBED_AVAILABLE:
raise ImportError(
"FastEmbed is not installed. Please install it with: "
"uv pip install fastembed or uv pip install fastembed-gpu for GPU support"
)
self.model = TextEmbedding(
model_name=model_name,
cache_dir=str(cache_dir) if cache_dir else None,
)
def embed_chunks(self, chunks: List[str]) -> List[np.ndarray]:
"""
Generate embeddings for a list of text chunks
Args:
chunks: List of text chunks to embed
Returns:
List of embeddings
"""
embeddings = list(self.model.embed(chunks))
return embeddings
def embed_texts(self, texts: List[str]) -> List[np.ndarray]:
"""
Generate embeddings for a list of texts
Args:
texts: List of texts to embed
Returns:
List of embeddings
"""
embeddings = list(self.model.embed(texts))
return embeddings
def embed_text(self, text: str) -> np.ndarray:
"""
Generate embedding for a single text
Args:
text: Text to embed
Returns:
Embedding array
"""
return self.embed_texts([text])[0]
@property
def dimension(self) -> int:
"""Get the dimension of the embeddings"""
# Generate a test embedding to get dimensions
test_embed = self.embed_text("test")
return len(test_embed)

View File

@@ -0,0 +1,118 @@
import pytest
from unittest.mock import patch
from crewai import Agent
from pydantic import ValidationError
def test_agent_with_custom_execution_image():
"""Test that Agent can be created with custom execution image."""
agent = Agent(
role="Test Agent",
goal="Test goal",
backstory="Test backstory",
allow_code_execution=True,
execution_image="my-custom-image:latest"
)
assert agent.execution_image == "my-custom-image:latest"
def test_agent_without_custom_execution_image():
"""Test that Agent works without custom execution image (default behavior)."""
agent = Agent(
role="Test Agent",
goal="Test goal",
backstory="Test backstory",
allow_code_execution=True
)
assert agent.execution_image is None
@patch('crewai_tools.CodeInterpreterTool')
def test_get_code_execution_tools_with_custom_image(mock_code_interpreter):
"""Test that get_code_execution_tools passes custom image to CodeInterpreterTool."""
agent = Agent(
role="Test Agent",
goal="Test goal",
backstory="Test backstory",
allow_code_execution=True,
execution_image="my-custom-image:latest"
)
agent.get_code_execution_tools()
mock_code_interpreter.assert_called_once_with(
unsafe_mode=False,
default_image_tag="my-custom-image:latest"
)
@patch('crewai_tools.CodeInterpreterTool')
def test_get_code_execution_tools_without_custom_image(mock_code_interpreter):
"""Test that get_code_execution_tools works without custom image."""
agent = Agent(
role="Test Agent",
goal="Test goal",
backstory="Test backstory",
allow_code_execution=True
)
agent.get_code_execution_tools()
mock_code_interpreter.assert_called_once_with(unsafe_mode=False)
@patch('crewai_tools.CodeInterpreterTool')
def test_get_code_execution_tools_with_unsafe_mode_and_custom_image(mock_code_interpreter):
"""Test that both unsafe_mode and custom image work together."""
agent = Agent(
role="Test Agent",
goal="Test goal",
backstory="Test backstory",
allow_code_execution=True,
code_execution_mode="unsafe",
execution_image="my-custom-image:latest"
)
agent.get_code_execution_tools()
mock_code_interpreter.assert_called_once_with(
unsafe_mode=True,
default_image_tag="my-custom-image:latest"
)
def test_agent_with_invalid_execution_image():
"""Test that Agent validates execution image format."""
with pytest.raises(ValidationError, match="String should match pattern"):
Agent(
role="Test Agent",
goal="Test goal",
backstory="Test backstory",
allow_code_execution=True,
execution_image="invalid@@image"
)
def test_agent_with_valid_execution_image_formats():
"""Test that Agent accepts various valid Docker image formats."""
valid_images = [
"python:3.11",
"python:3.11-slim",
"registry.example.com/python:3.11",
"my-registry.com:5000/python:latest",
"python",
"ubuntu:20.04",
"gcr.io/project/image:tag"
]
for image in valid_images:
agent = Agent(
role="Test Agent",
goal="Test goal",
backstory="Test backstory",
allow_code_execution=True,
execution_image=image
)
assert agent.execution_image == image

View File

@@ -0,0 +1,35 @@
from unittest.mock import patch, MagicMock
from crewai import Agent, Task, Crew
@patch('crewai_tools.CodeInterpreterTool')
def test_crew_with_custom_execution_image_integration(mock_code_interpreter_class):
"""Integration test for custom execution image in a Crew workflow."""
mock_tool_instance = MagicMock()
mock_code_interpreter_class.return_value = mock_tool_instance
agent = Agent(
role="Python Developer",
goal="Execute Python code",
backstory="Expert in Python programming",
allow_code_execution=True,
execution_image="python:3.11-slim"
)
task = Task(
description="Calculate 2 + 2",
expected_output="The result of 2 + 2",
agent=agent
)
crew = Crew(
agents=[agent],
tasks=[task]
)
crew._prepare_tools(agent, task, [])
mock_code_interpreter_class.assert_called_with(
unsafe_mode=False,
default_image_tag="python:3.11-slim"
)

1766
uv.lock generated

File diff suppressed because it is too large Load Diff