telemetry initialization and enhance event handling (#2853)

* Refactor Crew class memory initialization and enhance event handling

- Simplified the initialization of the external memory attribute in the Crew class.
- Updated memory system retrieval logic for consistency in key usage.
- Introduced a singleton pattern for the Telemetry class to ensure a single instance.
- Replaced telemetry usage in CrewEvaluator with event bus emissions for test results.
- Added new CrewTestResultEvent to handle crew test results more effectively.
- Updated event listener to process CrewTestResultEvent and log telemetry data accordingly.
- Enhanced tests to validate the singleton pattern in Telemetry and the new event handling logic.

* linted

* Remove unused telemetry attribute from Crew class memory initialization

* fix ordering of test

* Implement thread-safe singleton pattern in Telemetry class

- Introduced a threading lock to ensure safe instantiation of the Telemetry singleton.
- Updated the __new__ method to utilize double-checked locking for instance creation.
This commit is contained in:
Lorenze Jay
2025-05-21 10:32:03 -07:00
committed by GitHub
parent 169d3233e8
commit 31ffa90075
8 changed files with 302 additions and 108 deletions

View File

@@ -100,3 +100,12 @@ class CrewTestFailedEvent(CrewBaseEvent):
error: str
type: str = "crew_test_failed"
class CrewTestResultEvent(CrewBaseEvent):
"""Event emitted when a crew test result is available"""
quality: float
execution_duration: float
model: str
type: str = "crew_test_result"

View File

@@ -37,6 +37,7 @@ from .crew_events import (
CrewKickoffStartedEvent,
CrewTestCompletedEvent,
CrewTestFailedEvent,
CrewTestResultEvent,
CrewTestStartedEvent,
CrewTrainCompletedEvent,
CrewTrainFailedEvent,
@@ -134,6 +135,15 @@ class EventListener(BaseEventListener):
def on_crew_train_failed(source, event: CrewTrainFailedEvent):
self.formatter.handle_crew_train_failed(event.crew_name or "Crew")
@crewai_event_bus.on(CrewTestResultEvent)
def on_crew_test_result(source, event: CrewTestResultEvent):
self._telemetry.individual_test_result_span(
source.crew,
event.quality,
int(event.execution_duration),
event.model,
)
# ----------- TASK EVENTS -----------
@crewai_event_bus.on(TaskStartedEvent)