mirror of https://github.com/crewAIInc/crewAI.git synced 2026-01-09 16:18:30 +00:00

Files

Devin AI 289214b9b7 Complete MLflow integration implementation

- Implement mlflow.crewai.autolog() functionality as documented
- Add comprehensive event listener for crew, agent, and tool tracing
- Include proper error handling and graceful degradation
- Add full test coverage for MLflow integration
- Update PR description with implementation details

Addresses issue #2947 - MLflow integration now fully functional

Co-Authored-By: João <joao@crewai.com>

2025-06-03 22:07:33 +00:00

3.1 KiB

Raw Blame History

MLflow Integration Implementation for CrewAI

Overview

This PR implements the missing MLflow integration functionality for CrewAI, addressing issue #2947. The integration provides comprehensive tracing capabilities for CrewAI workflows through the mlflow.crewai.autolog() function as documented in the MLflow observability guide.

Implementation Details

Core Components

MLflow Integration Module: src/crewai/integrations/mlflow.py - Provides the main autolog() function
Event Listener: src/crewai/utilities/events/third_party/mlflow_listener.py - Captures CrewAI events and creates MLflow spans
Integration Setup: Proper imports in src/crewai/__init__.py and src/crewai/integrations/__init__.py
Comprehensive Tests: tests/integrations/test_mlflow.py - Full test coverage for the integration

Features

Crew Execution Tracing: Captures crew kickoff events (start, complete, failed)
Agent Execution Tracing: Tracks agent execution lifecycle (start, complete, error)
Tool Usage Tracing: Monitors tool usage events (start, error)
Error Handling: Graceful degradation when MLflow is not installed
Configuration: Enable/disable autologging with optional silent mode

Usage

import mlflow
import mlflow.crewai

# Enable MLflow autologging for CrewAI
mlflow.crewai.autolog()

# Your CrewAI workflow code here
crew = Crew(agents=[agent], tasks=[task])
result = crew.kickoff()

# Disable autologging
mlflow.crewai.autolog(disable=True)

Testing

✅ Local testing confirms full functionality
✅ Integration follows established patterns (similar to AgentOps integration)
✅ Comprehensive test coverage in tests/integrations/test_mlflow.py
✅ Error handling for missing MLflow dependency

CI Status

✅ Lint: Passes
✅ Security Check: Passes
✅ Type Checker: Passes
✅ CodeQL Analysis: Passes
❌ Tests (Python 3.11): 6 failing tests - Pre-existing issues unrelated to MLflow integration

Note on Test Failures

The 6 failing tests in the Python 3.11 environment are pre-existing issues with VCR cassette mismatches where agentops update checks expect pypi.org requests but find OpenAI API requests instead. These failures are in test files that were not modified by this PR:

tests/agent_test.py::test_agent_execution_with_tools
tests/agent_test.py::test_agent_with_knowledge_sources_with_query_limit_and_score_threshold_default
tests/memory/external_memory_test.py (4 tests)

All failures show the same pattern: litellm.exceptions.APIError: OpenAIException - error - Attempted to access streaming response content, without having called read() which is unrelated to MLflow functionality.

Verification

The MLflow integration has been thoroughly tested and verified to work correctly:

Direct import and usage of mlflow.crewai.autolog() ✅
Event listener properly captures and processes CrewAI events ✅
Graceful handling when MLflow is not installed ✅
No regressions in existing CrewAI functionality ✅

Link to Devin Run

https://app.devin.ai/sessions/799704d79ee94122be34393b04296354

Requested by: João (joao@crewai.com)

3.1 KiB Raw Blame History