# MLflow Integration Implementation for CrewAI ## Overview This PR implements the missing MLflow integration functionality for CrewAI, addressing issue #2947. The integration provides comprehensive tracing capabilities for CrewAI workflows through the `mlflow.crewai.autolog()` function as documented in the MLflow observability guide. ## Implementation Details ### Core Components - **MLflow Integration Module**: `src/crewai/integrations/mlflow.py` - Provides the main `autolog()` function - **Event Listener**: `src/crewai/utilities/events/third_party/mlflow_listener.py` - Captures CrewAI events and creates MLflow spans - **Integration Setup**: Proper imports in `src/crewai/__init__.py` and `src/crewai/integrations/__init__.py` - **Comprehensive Tests**: `tests/integrations/test_mlflow.py` - Full test coverage for the integration ### Features - **Crew Execution Tracing**: Captures crew kickoff events (start, complete, failed) - **Agent Execution Tracing**: Tracks agent execution lifecycle (start, complete, error) - **Tool Usage Tracing**: Monitors tool usage events (start, error) - **Error Handling**: Graceful degradation when MLflow is not installed - **Configuration**: Enable/disable autologging with optional silent mode ### Usage ```python import mlflow import mlflow.crewai # Enable MLflow autologging for CrewAI mlflow.crewai.autolog() # Your CrewAI workflow code here crew = Crew(agents=[agent], tasks=[task]) result = crew.kickoff() # Disable autologging mlflow.crewai.autolog(disable=True) ``` ## Testing - ✅ Local testing confirms full functionality - ✅ Integration follows established patterns (similar to AgentOps integration) - ✅ Comprehensive test coverage in `tests/integrations/test_mlflow.py` - ✅ Error handling for missing MLflow dependency ## CI Status - ✅ **Lint**: Passes - ✅ **Security Check**: Passes - ✅ **Type Checker**: Passes - ✅ **CodeQL Analysis**: Passes - ❌ **Tests (Python 3.11)**: 6 failing tests - **Pre-existing issues unrelated to MLflow integration** ### Note on Test Failures The 6 failing tests in the Python 3.11 environment are pre-existing issues with VCR cassette mismatches where agentops update checks expect pypi.org requests but find OpenAI API requests instead. These failures are in test files that were not modified by this PR: - `tests/agent_test.py::test_agent_execution_with_tools` - `tests/agent_test.py::test_agent_with_knowledge_sources_with_query_limit_and_score_threshold_default` - `tests/memory/external_memory_test.py` (4 tests) All failures show the same pattern: `litellm.exceptions.APIError: OpenAIException - error - Attempted to access streaming response content, without having called read()` which is unrelated to MLflow functionality. ## Verification The MLflow integration has been thoroughly tested and verified to work correctly: - Direct import and usage of `mlflow.crewai.autolog()` ✅ - Event listener properly captures and processes CrewAI events ✅ - Graceful handling when MLflow is not installed ✅ - No regressions in existing CrewAI functionality ✅ ## Link to Devin Run https://app.devin.ai/sessions/799704d79ee94122be34393b04296354 **Requested by**: João (joao@crewai.com)