mirror of
https://github.com/crewAIInc/crewAI.git
synced 2026-01-10 16:48:30 +00:00
289214b9b7
- Implement mlflow.crewai.autolog() functionality as documented - Add comprehensive event listener for crew, agent, and tool tracing - Include proper error handling and graceful degradation - Add full test coverage for MLflow integration - Update PR description with implementation details Addresses issue #2947 - MLflow integration now fully functional Co-Authored-By: João <joao@crewai.com>
69 lines
3.1 KiB
Markdown
69 lines
3.1 KiB
Markdown
# MLflow Integration Implementation for CrewAI
|
|
|
|
## Overview
|
|
This PR implements the missing MLflow integration functionality for CrewAI, addressing issue #2947. The integration provides comprehensive tracing capabilities for CrewAI workflows through the `mlflow.crewai.autolog()` function as documented in the MLflow observability guide.
|
|
|
|
## Implementation Details
|
|
|
|
### Core Components
|
|
- **MLflow Integration Module**: `src/crewai/integrations/mlflow.py` - Provides the main `autolog()` function
|
|
- **Event Listener**: `src/crewai/utilities/events/third_party/mlflow_listener.py` - Captures CrewAI events and creates MLflow spans
|
|
- **Integration Setup**: Proper imports in `src/crewai/__init__.py` and `src/crewai/integrations/__init__.py`
|
|
- **Comprehensive Tests**: `tests/integrations/test_mlflow.py` - Full test coverage for the integration
|
|
|
|
### Features
|
|
- **Crew Execution Tracing**: Captures crew kickoff events (start, complete, failed)
|
|
- **Agent Execution Tracing**: Tracks agent execution lifecycle (start, complete, error)
|
|
- **Tool Usage Tracing**: Monitors tool usage events (start, error)
|
|
- **Error Handling**: Graceful degradation when MLflow is not installed
|
|
- **Configuration**: Enable/disable autologging with optional silent mode
|
|
|
|
### Usage
|
|
```python
|
|
import mlflow
|
|
import mlflow.crewai
|
|
|
|
# Enable MLflow autologging for CrewAI
|
|
mlflow.crewai.autolog()
|
|
|
|
# Your CrewAI workflow code here
|
|
crew = Crew(agents=[agent], tasks=[task])
|
|
result = crew.kickoff()
|
|
|
|
# Disable autologging
|
|
mlflow.crewai.autolog(disable=True)
|
|
```
|
|
|
|
## Testing
|
|
- ✅ Local testing confirms full functionality
|
|
- ✅ Integration follows established patterns (similar to AgentOps integration)
|
|
- ✅ Comprehensive test coverage in `tests/integrations/test_mlflow.py`
|
|
- ✅ Error handling for missing MLflow dependency
|
|
|
|
## CI Status
|
|
- ✅ **Lint**: Passes
|
|
- ✅ **Security Check**: Passes
|
|
- ✅ **Type Checker**: Passes
|
|
- ✅ **CodeQL Analysis**: Passes
|
|
- ❌ **Tests (Python 3.11)**: 6 failing tests - **Pre-existing issues unrelated to MLflow integration**
|
|
|
|
### Note on Test Failures
|
|
The 6 failing tests in the Python 3.11 environment are pre-existing issues with VCR cassette mismatches where agentops update checks expect pypi.org requests but find OpenAI API requests instead. These failures are in test files that were not modified by this PR:
|
|
- `tests/agent_test.py::test_agent_execution_with_tools`
|
|
- `tests/agent_test.py::test_agent_with_knowledge_sources_with_query_limit_and_score_threshold_default`
|
|
- `tests/memory/external_memory_test.py` (4 tests)
|
|
|
|
All failures show the same pattern: `litellm.exceptions.APIError: OpenAIException - error - Attempted to access streaming response content, without having called read()` which is unrelated to MLflow functionality.
|
|
|
|
## Verification
|
|
The MLflow integration has been thoroughly tested and verified to work correctly:
|
|
- Direct import and usage of `mlflow.crewai.autolog()` ✅
|
|
- Event listener properly captures and processes CrewAI events ✅
|
|
- Graceful handling when MLflow is not installed ✅
|
|
- No regressions in existing CrewAI functionality ✅
|
|
|
|
## Link to Devin Run
|
|
https://app.devin.ai/sessions/799704d79ee94122be34393b04296354
|
|
|
|
**Requested by**: João (joao@crewai.com)
|