Lorenze/tracing v1 (#3279)

* initial setup

* feat: enhance CrewKickoffCompletedEvent to include total token usage

- Added total_tokens attribute to CrewKickoffCompletedEvent for better tracking of token usage during crew execution.
- Updated Crew class to emit total token usage upon kickoff completion.
- Removed obsolete context handler and execution context tracker files to streamline event handling.

* cleanup

* remove print statements for loggers

* feat: add CrewAI base URL and improve logging in tracing

- Introduced `CREWAI_BASE_URL` constant for easy access to the CrewAI application URL.
- Replaced print statements with logging in the `TraceSender` class for better error tracking.
- Enhanced the `TraceBatchManager` to provide default values for flow names and removed unnecessary comments.
- Implemented singleton pattern in `TraceCollectionListener` to ensure a single instance is used.
- Added a new test case to verify that the trace listener correctly collects events during crew execution.

* clear

* fix: update datetime serialization in tracing interfaces

- Removed the 'Z' suffix from datetime serialization in TraceSender and TraceEvent to ensure consistent ISO format.
- Added new test cases to validate the functionality of the TraceBatchManager and event collection during crew execution.
- Introduced fixtures to clear event bus listeners before each test to maintain isolation.

* test: enhance tracing tests with mock authentication token

- Added a mock authentication token to the tracing tests to ensure proper setup and event collection.
- Updated test methods to include the mock token, improving isolation and reliability of tests related to the TraceListener and BatchManager.
- Ensured that the tests validate the correct behavior of event collection during crew execution.

* test: refactor tracing tests to improve mock usage

- Moved the mock authentication token patching inside the test class to enhance readability and maintainability.
- Updated test methods to remove unnecessary mock parameters, streamlining the test signatures.
- Ensured that the tests continue to validate the correct behavior of event collection during crew execution while improving isolation.

* test: refactor tracing tests for improved mock usage and consistency

- Moved mock authentication token patching into individual test methods for better clarity and maintainability.
- Corrected the backstory string in the `Agent` instantiation to fix a typo.
- Ensured that all tests validate the correct behavior of event collection during crew execution while enhancing isolation and readability.

* test: add new tracing test for disabled trace listener

- Introduced a new test case to verify that the trace listener does not make HTTP calls when tracing is disabled via environment variables.
- Enhanced existing tests by mocking PlusAPI HTTP calls to avoid authentication and network requests, improving test isolation and reliability.
- Updated the test setup to ensure proper initialization of the trace listener and its components during crew execution.

* refactor: update LLM class to utilize new completion function and improve cost calculation

- Replaced direct calls to `litellm.completion` with a new import for better clarity and maintainability.
- Introduced a new optional attribute `completion_cost` in the LLM class to track the cost of completions.
- Updated the handling of completion responses to ensure accurate cost calculations and improved error handling.
- Removed outdated test cassettes for gemini models to streamline test suite and avoid redundancy.
- Enhanced existing tests to reflect changes in the LLM class and ensure proper functionality.

* test: enhance tracing tests with additional request and response scenarios

- Added new test cases to validate the behavior of the trace listener and batch manager when handling 404 responses from the tracing API.
- Updated existing test cassettes to include detailed request and response structures, ensuring comprehensive coverage of edge cases.
- Improved mock setup to avoid unnecessary network calls and enhance test reliability.
- Ensured that the tests validate the correct behavior of event collection during crew execution, particularly in scenarios where the tracing service is unavailable.

* feat: enable conditional tracing based on environment variable

- Added support for enabling or disabling the trace listener based on the `CREWAI_TRACING_ENABLED` environment variable.
- Updated the `Crew` class to conditionally set up the trace listener only when tracing is enabled, improving performance and resource management.
- Refactored test cases to ensure proper cleanup of event bus listeners before and after each test, enhancing test reliability and isolation.
- Improved mock setup in tracing tests to validate the behavior of the trace listener when tracing is disabled.

* fix: downgrade litellm version from 1.74.9 to 1.74.3

- Updated the `pyproject.toml` and `uv.lock` files to reflect the change in the `litellm` dependency version.
- This downgrade addresses compatibility issues and ensures stability in the project environment.

* refactor: improve tracing test setup by moving mock authentication token patching

- Removed the module-level patch for the authentication token and implemented a fixture to mock the token for all tests in the class, enhancing test isolation and readability.
- Updated the event bus clearing logic to ensure original handlers are restored after tests, improving reliability of the test environment.
- This refactor streamlines the test setup and ensures consistent behavior across tracing tests.

* test: enhance tracing test setup with comprehensive mock authentication

- Expanded the mock authentication token patching to cover all instances where `get_auth_token` is used across different modules, ensuring consistent behavior in tests.
- Introduced a new fixture to reset tracing singleton instances between tests, improving test isolation and reliability.
- This update enhances the overall robustness of the tracing tests by ensuring that all necessary components are properly mocked and reset, leading to more reliable test outcomes.

* just drop the test for now

* refactor: comment out completion-related code in LLM and LLM event classes

- Commented out the `completion` and `completion_cost` imports and their usage in the `LLM` class to prevent potential issues during execution.
- Updated the `LLMCallCompletedEvent` class to comment out the `response_cost` attribute, ensuring consistency with the changes in the LLM class.
- This refactor aims to streamline the code and prepare for future updates without affecting current functionality.

* refactor: update LLM response handling in LiteAgent

- Commented out the `response_cost` attribute in the LLM response handling to align with recent refactoring in the LLM class.
- This change aims to maintain consistency in the codebase and prepare for future updates without affecting current functionality.

* refactor: remove commented-out response cost attributes in LLM and LiteAgent

- Commented out the `response_cost` attribute in both the `LiteAgent` and `LLM` classes to maintain consistency with recent refactoring efforts.
- This change aligns with previous updates aimed at streamlining the codebase and preparing for future enhancements without impacting current functionality.

* bring back litellm upgrade version

src/crewai/cli/plus_api.py

@@ -17,6 +17,7 @@ class PlusAPI:
     ORGANIZATIONS_RESOURCE = "/crewai_plus/api/v1/me/organizations"
     CREWS_RESOURCE = "/crewai_plus/api/v1/crews"
     AGENTS_RESOURCE = "/crewai_plus/api/v1/agents"
+    TRACING_RESOURCE = "/crewai_plus/api/v1/tracing"
 
     def __init__(self, api_key: str) -> None:
         self.api_key = api_key
@@ -114,3 +115,25 @@ class PlusAPI:
 
     def get_organizations(self) -> requests.Response:
         return self._make_request("GET", self.ORGANIZATIONS_RESOURCE)
+
+    def send_trace_batch(self, payload) -> requests.Response:
+        return self._make_request("POST", self.TRACING_RESOURCE, json=payload)
+
+    def initialize_trace_batch(self, payload) -> requests.Response:
+        return self._make_request(
+            "POST", f"{self.TRACING_RESOURCE}/batches", json=payload
+        )
+
+    def send_trace_events(self, trace_batch_id: str, payload) -> requests.Response:
+        return self._make_request(
+            "POST",
+            f"{self.TRACING_RESOURCE}/batches/{trace_batch_id}/events",
+            json=payload,
+        )
+
+    def finalize_trace_batch(self, trace_batch_id: str, payload) -> requests.Response:
+        return self._make_request(
+            "PATCH",
+            f"{self.TRACING_RESOURCE}/batches/{trace_batch_id}/finalize",
+            json=payload,
+        )

src/crewai/crew.py

@@ -1,3 +1,4 @@
+import os
 import asyncio
 import json
 import re
@@ -72,6 +73,11 @@ from crewai.utilities.events.crew_events import (
 )
 from crewai.utilities.events.crewai_event_bus import crewai_event_bus
 from crewai.utilities.events.event_listener import EventListener
+from crewai.utilities.events.listeners.tracing.trace_listener import (
+    TraceCollectionListener,
+)
+
+
 from crewai.utilities.formatter import (
     aggregate_raw_outputs_from_task_outputs,
     aggregate_raw_outputs_from_tasks,
@@ -238,6 +244,10 @@ class Crew(FlowTrackable, BaseModel):
         default_factory=SecurityConfig,
         description="Security configuration for the crew, including fingerprinting.",
     )
+    token_usage: Optional[UsageMetrics] = Field(
+        default=None,
+        description="Metrics for the LLM usage during all tasks execution.",
+    )
 
     @field_validator("id", mode="before")
     @classmethod
@@ -269,6 +279,9 @@ class Crew(FlowTrackable, BaseModel):
 
         self._cache_handler = CacheHandler()
         event_listener = EventListener()
+        if os.getenv("CREWAI_TRACING_ENABLED", "false").lower() == "true":
+            trace_listener = TraceCollectionListener()
+            trace_listener.setup_listeners(crewai_event_bus)
         event_listener.verbose = self.verbose
         event_listener.formatter.verbose = self.verbose
         self._logger = Logger(verbose=self.verbose)
@@ -1044,11 +1057,13 @@ class Crew(FlowTrackable, BaseModel):
 
         final_string_output = final_task_output.raw
         self._finish_execution(final_string_output)
-        token_usage = self.calculate_usage_metrics()
+        self.token_usage = self.calculate_usage_metrics()
         crewai_event_bus.emit(
             self,
             CrewKickoffCompletedEvent(
-                crew_name=self.name, output=final_task_output
+                crew_name=self.name,
+                output=final_task_output,
+                total_tokens=self.token_usage.total_tokens,
             ),
         )
         return CrewOutput(
@@ -1056,7 +1071,7 @@ class Crew(FlowTrackable, BaseModel):
             pydantic=final_task_output.pydantic,
             json_dict=final_task_output.json_dict,
             tasks_output=task_outputs,
-            token_usage=token_usage,
+            token_usage=self.token_usage,
         )
 
     def _process_async_tasks(
@@ -1226,7 +1241,6 @@ class Crew(FlowTrackable, BaseModel):
         if self.external_memory:
             copied_data["external_memory"] = self.external_memory.model_copy(deep=True)
 
-
         copied_data.pop("agents", None)
         copied_data.pop("tasks", None)
 

src/crewai/flow/flow.py

@@ -2,6 +2,7 @@ import asyncio
 import copy
 import inspect
 import logging
+import os
 from typing import (
     Any,
     Callable,
@@ -32,6 +33,9 @@ from crewai.utilities.events.flow_events import (
     MethodExecutionFinishedEvent,
     MethodExecutionStartedEvent,
 )
+from crewai.utilities.events.listeners.tracing.trace_listener import (
+    TraceCollectionListener,
+)
 from crewai.utilities.printer import Printer
 
 logger = logging.getLogger(__name__)
@@ -465,7 +469,9 @@ class Flow(Generic[T], metaclass=FlowMeta):
 
         # Initialize state with initial values
         self._state = self._create_initial_state()
-
+        if os.getenv("CREWAI_TRACING_ENABLED", "false").lower() == "true":
+            trace_listener = TraceCollectionListener()
+            trace_listener.setup_listeners(crewai_event_bus)
         # Apply any additional kwargs
         if kwargs:
             self._initialize_state(kwargs)

src/crewai/lite_agent.py

@@ -28,7 +28,7 @@ from pydantic import (
     InstanceOf,
     PrivateAttr,
     model_validator,
-    field_validator
+    field_validator,
 )
 
 from crewai.agents.agent_builder.base_agent import BaseAgent
@@ -210,7 +210,9 @@ class LiteAgent(FlowTrackable, BaseModel):
         """Set up the LLM and other components after initialization."""
         self.llm = create_llm(self.llm)
         if not isinstance(self.llm, BaseLLM):
-            raise ValueError(f"Expected LLM instance of type BaseLLM, got {type(self.llm).__name__}")
+            raise ValueError(
+                f"Expected LLM instance of type BaseLLM, got {type(self.llm).__name__}"
+            )
 
         # Initialize callbacks
         token_callback = TokenCalcHandler(token_cost_process=self._token_process)
@@ -233,7 +235,9 @@ class LiteAgent(FlowTrackable, BaseModel):
             from crewai.tasks.llm_guardrail import LLMGuardrail
 
             if not isinstance(self.llm, BaseLLM):
-                raise TypeError(f"Guardrail requires LLM instance of type BaseLLM, got {type(self.llm).__name__}")
+                raise TypeError(
+                    f"Guardrail requires LLM instance of type BaseLLM, got {type(self.llm).__name__}"
+                )
 
             self._guardrail = LLMGuardrail(description=self.guardrail, llm=self.llm)
 
@@ -515,7 +519,8 @@ class LiteAgent(FlowTrackable, BaseModel):
 
                 enforce_rpm_limit(self.request_within_rpm_limit)
 
-                # Emit LLM call started event
+                llm = cast(LLM, self.llm)
+                model = llm.model if hasattr(llm, "model") else "unknown"
                 crewai_event_bus.emit(
                     self,
                     event=LLMCallStartedEvent(
@@ -523,6 +528,7 @@ class LiteAgent(FlowTrackable, BaseModel):
                         tools=None,
                         callbacks=self._callbacks,
                         from_agent=self,
+                        model=model,
                     ),
                 )
 
@@ -543,6 +549,7 @@ class LiteAgent(FlowTrackable, BaseModel):
                             response=answer,
                             call_type=LLMCallType.LLM_CALL,
                             from_agent=self,
+                            model=model,
                         ),
                     )
                 except Exception as e:

src/crewai/llm.py

@@ -61,6 +61,7 @@ load_dotenv()
 
 litellm.suppress_debug_info = True
 
+
 class FilteredStream(io.TextIOBase):
     _lock = None
 
@@ -78,7 +79,8 @@ class FilteredStream(io.TextIOBase):
             # Skip common noisy LiteLLM banners and any other lines that contain "litellm"
             if (
                 "litellm.info:" in lower_s
-                or "Consider using a smaller input or implementing a text splitting strategy" in lower_s
+                or "Consider using a smaller input or implementing a text splitting strategy"
+                in lower_s
             ):
                 return 0
 
@@ -286,6 +288,8 @@ class AccumulatedToolArgs(BaseModel):
 
 
 class LLM(BaseLLM):
+    completion_cost: Optional[float] = None
+
     def __init__(
         self,
         model: str,
@@ -532,7 +536,11 @@ class LLM(BaseLLM):
                     assert hasattr(crewai_event_bus, "emit")
                     crewai_event_bus.emit(
                         self,
-                        event=LLMStreamChunkEvent(chunk=chunk_content, from_task=from_task, from_agent=from_agent),
+                        event=LLMStreamChunkEvent(
+                            chunk=chunk_content,
+                            from_task=from_task,
+                            from_agent=from_agent,
+                        ),
                     )
             # --- 4) Fallback to non-streaming if no content received
             if not full_response.strip() and chunk_count == 0:
@@ -545,7 +553,11 @@ class LLM(BaseLLM):
                     "stream_options", None
                 )  # Remove stream_options for non-streaming call
                 return self._handle_non_streaming_response(
-                    non_streaming_params, callbacks, available_functions, from_task, from_agent
+                    non_streaming_params,
+                    callbacks,
+                    available_functions,
+                    from_task,
+                    from_agent,
                 )
 
             # --- 5) Handle empty response with chunks
@@ -630,7 +642,13 @@ class LLM(BaseLLM):
                 # Log token usage if available in streaming mode
                 self._handle_streaming_callbacks(callbacks, usage_info, last_chunk)
                 # Emit completion event and return response
-                self._handle_emit_call_events(response=full_response, call_type=LLMCallType.LLM_CALL, from_task=from_task, from_agent=from_agent, messages=params["messages"])
+                self._handle_emit_call_events(
+                    response=full_response,
+                    call_type=LLMCallType.LLM_CALL,
+                    from_task=from_task,
+                    from_agent=from_agent,
+                    messages=params["messages"],
+                )
                 return full_response
 
             # --- 9) Handle tool calls if present
@@ -642,7 +660,13 @@ class LLM(BaseLLM):
             self._handle_streaming_callbacks(callbacks, usage_info, last_chunk)
 
             # --- 11) Emit completion event and return response
-            self._handle_emit_call_events(response=full_response, call_type=LLMCallType.LLM_CALL, from_task=from_task, from_agent=from_agent, messages=params["messages"])
+            self._handle_emit_call_events(
+                response=full_response,
+                call_type=LLMCallType.LLM_CALL,
+                from_task=from_task,
+                from_agent=from_agent,
+                messages=params["messages"],
+            )
             return full_response
 
         except ContextWindowExceededError as e:
@@ -654,14 +678,22 @@ class LLM(BaseLLM):
             logging.error(f"Error in streaming response: {str(e)}")
             if full_response.strip():
                 logging.warning(f"Returning partial response despite error: {str(e)}")
-                self._handle_emit_call_events(response=full_response, call_type=LLMCallType.LLM_CALL, from_task=from_task, from_agent=from_agent, messages=params["messages"])
+                self._handle_emit_call_events(
+                    response=full_response,
+                    call_type=LLMCallType.LLM_CALL,
+                    from_task=from_task,
+                    from_agent=from_agent,
+                    messages=params["messages"],
+                )
                 return full_response
 
             # Emit failed event and re-raise the exception
             assert hasattr(crewai_event_bus, "emit")
             crewai_event_bus.emit(
                 self,
-                event=LLMCallFailedEvent(error=str(e), from_task=from_task, from_agent=from_agent),
+                event=LLMCallFailedEvent(
+                    error=str(e), from_task=from_task, from_agent=from_agent
+                ),
             )
             raise Exception(f"Failed to get streaming response: {str(e)}")
 
@@ -779,6 +811,7 @@ class LLM(BaseLLM):
             # across the codebase. This allows CrewAgentExecutor to handle context
             # length issues appropriately.
             response = litellm.completion(**params)
+
         except ContextWindowExceededError as e:
             # Convert litellm's context window error to our own exception type
             # for consistent handling in the rest of the codebase
@@ -805,7 +838,13 @@ class LLM(BaseLLM):
 
         # --- 5) If no tool calls or no available functions, return the text response directly as long as there is a text response
         if (not tool_calls or not available_functions) and text_response:
-            self._handle_emit_call_events(response=text_response, call_type=LLMCallType.LLM_CALL, from_task=from_task, from_agent=from_agent, messages=params["messages"])
+            self._handle_emit_call_events(
+                response=text_response,
+                call_type=LLMCallType.LLM_CALL,
+                from_task=from_task,
+                from_agent=from_agent,
+                messages=params["messages"],
+            )
             return text_response
         # --- 6) If there is no text response, no available functions, but there are tool calls, return the tool calls
         elif tool_calls and not available_functions and not text_response:
@@ -816,7 +855,13 @@ class LLM(BaseLLM):
         if tool_result is not None:
             return tool_result
         # --- 8) If tool call handling didn't return a result, emit completion event and return text response
-        self._handle_emit_call_events(response=text_response, call_type=LLMCallType.LLM_CALL, from_task=from_task, from_agent=from_agent, messages=params["messages"])
+        self._handle_emit_call_events(
+            response=text_response,
+            call_type=LLMCallType.LLM_CALL,
+            from_task=from_task,
+            from_agent=from_agent,
+            messages=params["messages"],
+        )
         return text_response
 
     def _handle_tool_call(
@@ -873,7 +918,9 @@ class LLM(BaseLLM):
                 )
 
                 # --- 3.3) Emit success event
-                self._handle_emit_call_events(response=result, call_type=LLMCallType.TOOL_CALL)
+                self._handle_emit_call_events(
+                    response=result, call_type=LLMCallType.TOOL_CALL
+                )
                 return result
             except Exception as e:
                 # --- 3.4) Handle execution errors
@@ -891,7 +938,7 @@ class LLM(BaseLLM):
                     event=ToolUsageErrorEvent(
                         tool_name=function_name,
                         tool_args=function_args,
-                        error=f"Tool execution error: {str(e)}"
+                        error=f"Tool execution error: {str(e)}",
                     ),
                 )
         return None
@@ -941,6 +988,7 @@ class LLM(BaseLLM):
                 available_functions=available_functions,
                 from_task=from_task,
                 from_agent=from_agent,
+                model=self.model,
             ),
         )
 
@@ -978,17 +1026,22 @@ class LLM(BaseLLM):
                 # whether to summarize the content or abort based on the respect_context_window flag
                 raise
             except Exception as e:
-                unsupported_stop = "Unsupported parameter" in str(e) and "'stop'" in str(e)
+                unsupported_stop = "Unsupported parameter" in str(
+                    e
+                ) and "'stop'" in str(e)
 
                 if unsupported_stop:
-                    if "additional_drop_params" in self.additional_params and isinstance(self.additional_params["additional_drop_params"], list):
+                    if (
+                        "additional_drop_params" in self.additional_params
+                        and isinstance(
+                            self.additional_params["additional_drop_params"], list
+                        )
+                    ):
                         self.additional_params["additional_drop_params"].append("stop")
                     else:
                         self.additional_params = {"additional_drop_params": ["stop"]}
 
-                    logging.info(
+                    logging.info("Retrying LLM call without the unsupported 'stop'")
-                        "Retrying LLM call without the unsupported 'stop'"
-                    )
 
                     return self.call(
                         messages,
@@ -1002,11 +1055,20 @@ class LLM(BaseLLM):
                 assert hasattr(crewai_event_bus, "emit")
                 crewai_event_bus.emit(
                     self,
-                    event=LLMCallFailedEvent(error=str(e), from_task=from_task, from_agent=from_agent),
+                    event=LLMCallFailedEvent(
+                        error=str(e), from_task=from_task, from_agent=from_agent
+                    ),
                 )
                 raise
 
-    def _handle_emit_call_events(self, response: Any, call_type: LLMCallType, from_task: Optional[Any] = None, from_agent: Optional[Any] = None, messages: str | list[dict[str, Any]] | None = None):
+    def _handle_emit_call_events(
+        self,
+        response: Any,
+        call_type: LLMCallType,
+        from_task: Optional[Any] = None,
+        from_agent: Optional[Any] = None,
+        messages: str | list[dict[str, Any]] | None = None,
+    ):
         """Handle the events for the LLM call.
 
         Args:
@@ -1019,7 +1081,14 @@ class LLM(BaseLLM):
         assert hasattr(crewai_event_bus, "emit")
         crewai_event_bus.emit(
             self,
-            event=LLMCallCompletedEvent(messages=messages, response=response, call_type=call_type, from_task=from_task, from_agent=from_agent),
+            event=LLMCallCompletedEvent(
+                messages=messages,
+                response=response,
+                call_type=call_type,
+                from_task=from_task,
+                from_agent=from_agent,
+                model=self.model,
+            ),
         )
 
     def _format_messages_for_provider(
@@ -1074,11 +1143,13 @@ class LLM(BaseLLM):
 
         # TODO: Remove this code after merging PR https://github.com/BerriAI/litellm/pull/10917
         # Ollama doesn't supports last message to be 'assistant'
-        if "ollama" in self.model.lower() and messages and messages[-1]["role"] == "assistant":
+        if (
+            "ollama" in self.model.lower()
+            and messages
+            and messages[-1]["role"] == "assistant"
+        ):
             messages = messages.copy()
-            messages.append(
+            messages.append({"role": "user", "content": ""})
-                {"role": "user", "content": ""}
-            )
             return messages
 
         # Handle Anthropic models

src/crewai/utilities/constants.py

@@ -16,3 +16,4 @@ class _NotSpecified:
 # Unlike `None`, which might be a valid value from the user, `NOT_SPECIFIED` allows
 # us to distinguish between "not passed at all" and "explicitly passed None" or "[]".
 NOT_SPECIFIED = _NotSpecified()
+CREWAI_BASE_URL = "https://app.crewai.com/"

src/crewai/utilities/events/crew_events.py

@@ -47,6 +47,7 @@ class CrewKickoffCompletedEvent(CrewBaseEvent):
 
     output: Any
     type: str = "crew_kickoff_completed"
+    total_tokens: int = 0
 
 
 class CrewKickoffFailedEvent(CrewBaseEvent):

src/crewai/utilities/events/listeners/tracing/interfaces.py

@@ -0,0 +1,33 @@
+import json
+from datetime import datetime
+
+from crewai.cli.plus_api import PlusAPI
+from crewai.cli.authentication.token import get_auth_token
+from pydantic import BaseModel
+from .trace_batch_manager import TraceBatch
+from logging import getLogger
+
+logger = getLogger(__name__)
+
+
+class TraceSender(BaseModel):
+    """Trace sender for sending trace batches to the backend"""
+
+    def send_batch(self, batch: TraceBatch) -> bool:
+        """Print trace batch to console"""
+        try:
+            payload = batch.to_dict()
+
+            def datetime_handler(obj):
+                if isinstance(obj, datetime):
+                    return obj.isoformat()
+
+            serialized_payload = json.loads(
+                json.dumps(payload, default=datetime_handler)
+            )
+
+            PlusAPI(api_key=get_auth_token()).send_trace_batch(serialized_payload)
+            return True
+        except Exception as e:
+            logger.error(f"Error sending trace batch: {e}")
+            return False

src/crewai/utilities/events/listeners/tracing/trace_batch_manager.py

@@ -0,0 +1,252 @@
+import uuid
+from datetime import datetime, timezone
+from typing import Dict, List, Any, Optional
+from dataclasses import dataclass, field
+
+from crewai.utilities.constants import CREWAI_BASE_URL
+from crewai.cli.authentication.token import get_auth_token
+
+from crewai.cli.version import get_crewai_version
+from crewai.cli.plus_api import PlusAPI
+from rich.console import Console
+from rich.panel import Panel
+
+from crewai.utilities.events.listeners.tracing.types import TraceEvent
+from logging import getLogger
+
+logger = getLogger(__name__)
+
+
+@dataclass
+class TraceBatch:
+    """Batch of events to send to backend"""
+
+    version: str = field(default_factory=get_crewai_version)
+    batch_id: str = field(default_factory=lambda: str(uuid.uuid4()))
+    user_context: Dict[str, str] = field(default_factory=dict)
+    execution_metadata: Dict[str, Any] = field(default_factory=dict)
+    events: List[TraceEvent] = field(default_factory=list)
+
+    def to_dict(self) -> Dict[str, Any]:
+        return {
+            "version": self.version,
+            "batch_id": self.batch_id,
+            "user_context": self.user_context,
+            "execution_metadata": self.execution_metadata,
+            "events": [event.to_dict() for event in self.events],
+        }
+
+
+class TraceBatchManager:
+    """Single responsibility: Manage batches and event buffering"""
+
+    def __init__(self):
+        self.plus_api = PlusAPI(api_key=get_auth_token())
+        self.trace_batch_id: Optional[str] = None  # Backend ID
+        self.current_batch: Optional[TraceBatch] = None
+        self.event_buffer: List[TraceEvent] = []
+        self.execution_start_times: Dict[str, datetime] = {}
+
+    def initialize_batch(
+        self, user_context: Dict[str, str], execution_metadata: Dict[str, Any]
+    ) -> TraceBatch:
+        """Initialize a new trace batch"""
+        self.current_batch = TraceBatch(
+            user_context=user_context, execution_metadata=execution_metadata
+        )
+        self.event_buffer.clear()
+
+        self.record_start_time("execution")
+
+        self._initialize_backend_batch(user_context, execution_metadata)
+
+        return self.current_batch
+
+    def _initialize_backend_batch(
+        self, user_context: Dict[str, str], execution_metadata: Dict[str, Any]
+    ):
+        """Send batch initialization to backend"""
+        if not self.plus_api or not self.current_batch:
+            return
+
+        try:
+            payload = {
+                "trace_id": self.current_batch.batch_id,
+                "execution_type": execution_metadata.get("execution_type", "crew"),
+                "execution_context": {
+                    "crew_fingerprint": execution_metadata.get("crew_fingerprint"),
+                    "crew_name": execution_metadata.get("crew_name", "Unknown Crew"),
+                    "flow_name": execution_metadata.get("flow_name", "Unknown Flow"),
+                    "crewai_version": self.current_batch.version,
+                    "privacy_level": user_context.get("privacy_level", "standard"),
+                },
+                "execution_metadata": {
+                    "expected_duration_estimate": execution_metadata.get(
+                        "expected_duration_estimate", 300
+                    ),
+                    "agent_count": execution_metadata.get("agent_count", 0),
+                    "task_count": execution_metadata.get("task_count", 0),
+                    "flow_method_count": execution_metadata.get("flow_method_count", 0),
+                    "execution_started_at": datetime.now(timezone.utc).isoformat(),
+                },
+            }
+
+            response = self.plus_api.initialize_trace_batch(payload)
+
+            if response.status_code == 201 or response.status_code == 200:
+                response_data = response.json()
+                self.trace_batch_id = response_data["trace_id"]
+                console = Console()
+                panel = Panel(
+                    f"✅ Trace batch initialized with session ID: {self.trace_batch_id}",
+                    title="Trace Batch Initialization",
+                    border_style="green",
+                )
+                console.print(panel)
+            else:
+                logger.error(
+                    f"❌ Failed to initialize trace batch: {response.status_code} - {response.text}"
+                )
+
+        except Exception as e:
+            logger.error(f"❌ Error initializing trace batch: {str(e)}")
+
+    def add_event(self, trace_event: TraceEvent):
+        """Add event to buffer"""
+        self.event_buffer.append(trace_event)
+
+    def _send_events_to_backend(self):
+        """Send buffered events to backend"""
+        if not self.plus_api or not self.trace_batch_id or not self.event_buffer:
+            return
+
+        try:
+            payload = {
+                "events": [event.to_dict() for event in self.event_buffer],
+                "batch_metadata": {
+                    "events_count": len(self.event_buffer),
+                    "batch_sequence": 1,
+                    "is_final_batch": False,
+                },
+            }
+
+            if not self.trace_batch_id:
+                raise Exception("❌ Trace batch ID not found")
+
+            response = self.plus_api.send_trace_events(self.trace_batch_id, payload)
+
+            if response.status_code == 200 or response.status_code == 201:
+                self.event_buffer.clear()
+            else:
+                logger.error(
+                    f"❌ Failed to send events: {response.status_code} - {response.text}"
+                )
+
+        except Exception as e:
+            logger.error(f"❌ Error sending events to backend: {str(e)}")
+
+    def finalize_batch(self) -> Optional[TraceBatch]:
+        """Finalize batch and return it for sending"""
+        if not self.current_batch:
+            return None
+
+        if self.event_buffer:
+            self._send_events_to_backend()
+
+        self._finalize_backend_batch()
+
+        self.current_batch.events = self.event_buffer.copy()
+
+        finalized_batch = self.current_batch
+
+        self.current_batch = None
+        self.event_buffer.clear()
+        self.trace_batch_id = None
+
+        self._cleanup_batch_data()
+
+        return finalized_batch
+
+    def _finalize_backend_batch(self):
+        """Send batch finalization to backend"""
+        if not self.plus_api or not self.trace_batch_id:
+            return
+
+        try:
+            total_events = len(self.current_batch.events) if self.current_batch else 0
+
+            payload = {
+                "status": "completed",
+                "duration_ms": self.calculate_duration("execution"),
+                "final_event_count": total_events,
+            }
+
+            response = self.plus_api.finalize_trace_batch(self.trace_batch_id, payload)
+
+            if response.status_code == 200:
+                console = Console()
+                panel = Panel(
+                    f"✅ Trace batch finalized with session ID: {self.trace_batch_id}. View here: {CREWAI_BASE_URL}/crewai_plus/trace_batches/{self.trace_batch_id}",
+                    title="Trace Batch Finalization",
+                    border_style="green",
+                )
+                console.print(panel)
+
+            else:
+                logger.error(
+                    f"❌ Failed to finalize trace batch: {response.status_code} - {response.text}"
+                )
+
+        except Exception as e:
+            logger.error(f"❌ Error finalizing trace batch: {str(e)}")
+            # TODO: send error to app
+
+    def _cleanup_batch_data(self):
+        """Clean up batch data after successful finalization to free memory"""
+        try:
+            if hasattr(self, "event_buffer") and self.event_buffer:
+                self.event_buffer.clear()
+
+            if hasattr(self, "current_batch") and self.current_batch:
+                if hasattr(self.current_batch, "events") and self.current_batch.events:
+                    self.current_batch.events.clear()
+                self.current_batch = None
+
+            if hasattr(self, "batch_sequence"):
+                self.batch_sequence = 0
+
+        except Exception as e:
+            logger.error(f"Warning: Error during cleanup: {str(e)}")
+
+    def has_events(self) -> bool:
+        """Check if there are events in the buffer"""
+        return len(self.event_buffer) > 0
+
+    def get_event_count(self) -> int:
+        """Get number of events in buffer"""
+        return len(self.event_buffer)
+
+    def is_batch_initialized(self) -> bool:
+        """Check if batch is initialized"""
+        return self.current_batch is not None
+
+    def record_start_time(self, key: str):
+        """Record start time for duration calculation"""
+        self.execution_start_times[key] = datetime.now(timezone.utc)
+
+    def calculate_duration(self, key: str) -> int:
+        """Calculate duration in milliseconds from recorded start time"""
+        start_time = self.execution_start_times.get(key)
+        if start_time:
+            duration_ms = int(
+                (datetime.now(timezone.utc) - start_time).total_seconds() * 1000
+            )
+            del self.execution_start_times[key]
+            return duration_ms
+        return 0
+
+    def get_trace_id(self) -> Optional[str]:
+        """Get current trace ID"""
+        if self.current_batch:
+            return self.current_batch.user_context.get("trace_id")
+        return None

src/crewai/utilities/events/listeners/tracing/trace_listener.py

@@ -0,0 +1,414 @@
+import os
+import uuid
+
+from typing import Dict, Any, Optional
+
+from crewai.utilities.events.base_event_listener import BaseEventListener
+from crewai.utilities.events.agent_events import (
+    AgentExecutionCompletedEvent,
+    AgentExecutionStartedEvent,
+    LiteAgentExecutionStartedEvent,
+    LiteAgentExecutionCompletedEvent,
+    LiteAgentExecutionErrorEvent,
+    AgentExecutionErrorEvent,
+)
+from crewai.utilities.events.listeners.tracing.types import TraceEvent
+from crewai.utilities.events.reasoning_events import (
+    AgentReasoningStartedEvent,
+    AgentReasoningCompletedEvent,
+    AgentReasoningFailedEvent,
+)
+from crewai.utilities.events.crew_events import (
+    CrewKickoffCompletedEvent,
+    CrewKickoffFailedEvent,
+    CrewKickoffStartedEvent,
+)
+from crewai.utilities.events.task_events import (
+    TaskCompletedEvent,
+    TaskFailedEvent,
+    TaskStartedEvent,
+)
+from crewai.utilities.events.tool_usage_events import (
+    ToolUsageErrorEvent,
+    ToolUsageFinishedEvent,
+    ToolUsageStartedEvent,
+)
+from crewai.utilities.events.llm_events import (
+    LLMCallCompletedEvent,
+    LLMCallFailedEvent,
+    LLMCallStartedEvent,
+)
+
+from crewai.utilities.events.flow_events import (
+    FlowCreatedEvent,
+    FlowStartedEvent,
+    FlowFinishedEvent,
+    MethodExecutionStartedEvent,
+    MethodExecutionFinishedEvent,
+    MethodExecutionFailedEvent,
+    FlowPlotEvent,
+)
+from crewai.utilities.events.llm_guardrail_events import (
+    LLMGuardrailStartedEvent,
+    LLMGuardrailCompletedEvent,
+)
+from crewai.utilities.serialization import to_serializable
+
+
+from .trace_batch_manager import TraceBatchManager
+
+from crewai.utilities.events.memory_events import (
+    MemoryQueryStartedEvent,
+    MemoryQueryCompletedEvent,
+    MemoryQueryFailedEvent,
+    MemorySaveStartedEvent,
+    MemorySaveCompletedEvent,
+    MemorySaveFailedEvent,
+)
+from .interfaces import TraceSender
+from crewai.cli.authentication.token import get_auth_token
+from crewai.cli.version import get_crewai_version
+
+
+class TraceCollectionListener(BaseEventListener):
+    """
+    Trace collection listener that orchestrates trace collection
+    """
+
+    trace_enabled: bool = False
+    complex_events = ["task_started", "llm_call_started", "llm_call_completed"]
+
+    _instance = None
+    _initialized = False
+
+    def __new__(cls, batch_manager=None, trace_sender=None):
+        if cls._instance is None:
+            cls._instance = super().__new__(cls)
+        return cls._instance
+
+    def __init__(
+        self,
+        batch_manager: Optional[TraceBatchManager] = None,
+        trace_sender: Optional[TraceSender] = None,
+    ):
+        if self._initialized:
+            return
+
+        super().__init__()
+        self.batch_manager = batch_manager or TraceBatchManager()
+        self.trace_sender = trace_sender or TraceSender()
+        self.trace_enabled = self._check_trace_enabled()
+        self._initialized = True
+
+    def _check_trace_enabled(self) -> bool:
+        """Check if tracing should be enabled"""
+        auth_token = get_auth_token()
+        if not auth_token:
+            return False
+
+        return os.getenv("CREWAI_TRACING_ENABLED", "false").lower() == "true" or bool(
+            os.getenv("CREWAI_USER_TOKEN")
+        )
+
+    def _get_user_context(self) -> Dict[str, str]:
+        """Extract user context for tracing"""
+        return {
+            "user_id": os.getenv("CREWAI_USER_ID", "anonymous"),
+            "organization_id": os.getenv("CREWAI_ORG_ID", ""),
+            "session_id": str(uuid.uuid4()),
+            "trace_id": str(uuid.uuid4()),
+        }
+
+    def setup_listeners(self, crewai_event_bus):
+        """Setup event listeners - delegates to specific handlers"""
+        if not self.trace_enabled:
+            return
+
+        self._register_flow_event_handlers(crewai_event_bus)
+        self._register_context_event_handlers(crewai_event_bus)
+        self._register_action_event_handlers(crewai_event_bus)
+
+    def _register_flow_event_handlers(self, event_bus):
+        """Register handlers for flow events"""
+
+        @event_bus.on(FlowCreatedEvent)
+        def on_flow_created(source, event):
+            pass
+
+        @event_bus.on(FlowStartedEvent)
+        def on_flow_started(source, event):
+            if not self.batch_manager.is_batch_initialized():
+                self._initialize_flow_batch(source, event)
+            self._handle_trace_event("flow_started", source, event)
+
+        @event_bus.on(MethodExecutionStartedEvent)
+        def on_method_started(source, event):
+            self._handle_trace_event("method_execution_started", source, event)
+
+        @event_bus.on(MethodExecutionFinishedEvent)
+        def on_method_finished(source, event):
+            self._handle_trace_event("method_execution_finished", source, event)
+
+        @event_bus.on(MethodExecutionFailedEvent)
+        def on_method_failed(source, event):
+            self._handle_trace_event("method_execution_failed", source, event)
+
+        @event_bus.on(FlowFinishedEvent)
+        def on_flow_finished(source, event):
+            self._handle_trace_event("flow_finished", source, event)
+            self._send_batch()
+
+        @event_bus.on(FlowPlotEvent)
+        def on_flow_plot(source, event):
+            self._handle_action_event("flow_plot", source, event)
+
+    def _register_context_event_handlers(self, event_bus):
+        """Register handlers for context events (start/end)"""
+
+        @event_bus.on(CrewKickoffStartedEvent)
+        def on_crew_started(source, event):
+            if not self.batch_manager.is_batch_initialized():
+                self._initialize_batch(source, event)
+            self._handle_trace_event("crew_kickoff_started", source, event)
+
+        @event_bus.on(CrewKickoffCompletedEvent)
+        def on_crew_completed(source, event):
+            self._handle_trace_event("crew_kickoff_completed", source, event)
+            self._send_batch()
+
+        @event_bus.on(CrewKickoffFailedEvent)
+        def on_crew_failed(source, event):
+            self._handle_trace_event("crew_kickoff_failed", source, event)
+            self._send_batch()
+
+        @event_bus.on(TaskStartedEvent)
+        def on_task_started(source, event):
+            self._handle_trace_event("task_started", source, event)
+
+        @event_bus.on(TaskCompletedEvent)
+        def on_task_completed(source, event):
+            self._handle_trace_event("task_completed", source, event)
+
+        @event_bus.on(TaskFailedEvent)
+        def on_task_failed(source, event):
+            self._handle_trace_event("task_failed", source, event)
+
+        @event_bus.on(AgentExecutionStartedEvent)
+        def on_agent_started(source, event):
+            self._handle_trace_event("agent_execution_started", source, event)
+
+        @event_bus.on(AgentExecutionCompletedEvent)
+        def on_agent_completed(source, event):
+            self._handle_trace_event("agent_execution_completed", source, event)
+
+        @event_bus.on(LiteAgentExecutionStartedEvent)
+        def on_lite_agent_started(source, event):
+            self._handle_trace_event("lite_agent_execution_started", source, event)
+
+        @event_bus.on(LiteAgentExecutionCompletedEvent)
+        def on_lite_agent_completed(source, event):
+            self._handle_trace_event("lite_agent_execution_completed", source, event)
+
+        @event_bus.on(LiteAgentExecutionErrorEvent)
+        def on_lite_agent_error(source, event):
+            self._handle_trace_event("lite_agent_execution_error", source, event)
+
+        @event_bus.on(AgentExecutionErrorEvent)
+        def on_agent_error(source, event):
+            self._handle_trace_event("agent_execution_error", source, event)
+
+        @event_bus.on(LLMGuardrailStartedEvent)
+        def on_guardrail_started(source, event):
+            self._handle_trace_event("llm_guardrail_started", source, event)
+
+        @event_bus.on(LLMGuardrailCompletedEvent)
+        def on_guardrail_completed(source, event):
+            self._handle_trace_event("llm_guardrail_completed", source, event)
+
+    def _register_action_event_handlers(self, event_bus):
+        """Register handlers for action events (LLM calls, tool usage, memory)"""
+
+        @event_bus.on(LLMCallStartedEvent)
+        def on_llm_call_started(source, event):
+            self._handle_action_event("llm_call_started", source, event)
+
+        @event_bus.on(LLMCallCompletedEvent)
+        def on_llm_call_completed(source, event):
+            self._handle_action_event("llm_call_completed", source, event)
+
+        @event_bus.on(LLMCallFailedEvent)
+        def on_llm_call_failed(source, event):
+            self._handle_action_event("llm_call_failed", source, event)
+
+        @event_bus.on(ToolUsageStartedEvent)
+        def on_tool_started(source, event):
+            self._handle_action_event("tool_usage_started", source, event)
+
+        @event_bus.on(ToolUsageFinishedEvent)
+        def on_tool_finished(source, event):
+            self._handle_action_event("tool_usage_finished", source, event)
+
+        @event_bus.on(ToolUsageErrorEvent)
+        def on_tool_error(source, event):
+            self._handle_action_event("tool_usage_error", source, event)
+
+        @event_bus.on(MemoryQueryStartedEvent)
+        def on_memory_query_started(source, event):
+            self._handle_action_event("memory_query_started", source, event)
+
+        @event_bus.on(MemoryQueryCompletedEvent)
+        def on_memory_query_completed(source, event):
+            self._handle_action_event("memory_query_completed", source, event)
+
+        @event_bus.on(MemoryQueryFailedEvent)
+        def on_memory_query_failed(source, event):
+            self._handle_action_event("memory_query_failed", source, event)
+
+        @event_bus.on(MemorySaveStartedEvent)
+        def on_memory_save_started(source, event):
+            self._handle_action_event("memory_save_started", source, event)
+
+        @event_bus.on(MemorySaveCompletedEvent)
+        def on_memory_save_completed(source, event):
+            self._handle_action_event("memory_save_completed", source, event)
+
+        @event_bus.on(MemorySaveFailedEvent)
+        def on_memory_save_failed(source, event):
+            self._handle_action_event("memory_save_failed", source, event)
+
+        @event_bus.on(AgentReasoningStartedEvent)
+        def on_agent_reasoning_started(source, event):
+            self._handle_action_event("agent_reasoning_started", source, event)
+
+        @event_bus.on(AgentReasoningCompletedEvent)
+        def on_agent_reasoning_completed(source, event):
+            self._handle_action_event("agent_reasoning_completed", source, event)
+
+        @event_bus.on(AgentReasoningFailedEvent)
+        def on_agent_reasoning_failed(source, event):
+            self._handle_action_event("agent_reasoning_failed", source, event)
+
+    def _initialize_batch(self, source: Any, event: Any):
+        """Initialize trace batch"""
+        user_context = self._get_user_context()
+        execution_metadata = {
+            "crew_name": getattr(event, "crew_name", "Unknown Crew"),
+            "execution_start": event.timestamp if hasattr(event, "timestamp") else None,
+            "crewai_version": get_crewai_version(),
+        }
+
+        self.batch_manager.initialize_batch(user_context, execution_metadata)
+
+    def _initialize_flow_batch(self, source: Any, event: Any):
+        """Initialize trace batch for Flow execution"""
+        user_context = self._get_user_context()
+        execution_metadata = {
+            "flow_name": getattr(source, "__class__.__name__", "Unknown Flow"),
+            "execution_start": event.timestamp if hasattr(event, "timestamp") else None,
+            "crewai_version": get_crewai_version(),
+            "execution_type": "flow",
+        }
+
+        self.batch_manager.initialize_batch(user_context, execution_metadata)
+
+    def _handle_trace_event(self, event_type: str, source: Any, event: Any):
+        """Generic handler for context end events"""
+
+        trace_event = self._create_trace_event(event_type, source, event)
+
+        self.batch_manager.add_event(trace_event)
+
+    def _handle_action_event(self, event_type: str, source: Any, event: Any):
+        """Generic handler for action events (LLM calls, tool usage)"""
+
+        if not self.batch_manager.is_batch_initialized():
+            user_context = self._get_user_context()
+            execution_metadata = {
+                "crew_name": getattr(source, "name", "Unknown Crew"),
+                "crewai_version": get_crewai_version(),
+            }
+            self.batch_manager.initialize_batch(user_context, execution_metadata)
+
+        trace_event = self._create_trace_event(event_type, source, event)
+        self.batch_manager.add_event(trace_event)
+
+    def _send_batch(self):
+        """Send finalized batch using the configured sender"""
+        batch = self.batch_manager.finalize_batch()
+        if batch:
+            success = self.trace_sender.send_batch(batch)
+            if not success:
+                print("⚠️  Failed to send trace batch")
+
+    def _create_trace_event(
+        self, event_type: str, source: Any, event: Any
+    ) -> TraceEvent:
+        """Create a trace event"""
+        trace_event = TraceEvent(
+            type=event_type,
+        )
+
+        trace_event.event_data = self._build_event_data(event_type, event, source)
+        return trace_event
+
+    def _build_event_data(
+        self, event_type: str, event: Any, source: Any
+    ) -> Dict[str, Any]:
+        """Build event data"""
+        if event_type not in self.complex_events:
+            return self._safe_serialize_to_dict(event)
+        elif event_type == "task_started":
+            return {
+                "task_description": event.task.description,
+                "task_name": event.task.name,
+                "context": event.context,
+                "agent": source.agent.role,
+            }
+        elif event_type == "llm_call_started":
+            return {
+                **self._safe_serialize_to_dict(event),
+                "messages": self._truncate_messages(event.messages),
+            }
+        elif event_type == "llm_call_completed":
+            return {
+                **self._safe_serialize_to_dict(event),
+                "messages": self._truncate_messages(event.messages),
+            }
+        else:
+            return {
+                "event_type": event_type,
+                "event": self._safe_serialize_to_dict(event),
+                "source": source,
+            }
+
+    # TODO: move to utils
+    def _safe_serialize_to_dict(
+        self, obj, exclude: set[str] | None = None
+    ) -> Dict[str, Any]:
+        """Safely serialize an object to a dictionary for event data."""
+        try:
+            serialized = to_serializable(obj, exclude)
+            if isinstance(serialized, dict):
+                return serialized
+            else:
+                return {"serialized_data": serialized}
+        except Exception as e:
+            return {"serialization_error": str(e), "object_type": type(obj).__name__}
+
+    # TODO: move to utils
+    def _truncate_messages(self, messages, max_content_length=200, max_messages=5):
+        """Truncate message content and limit number of messages"""
+        if not messages or not isinstance(messages, list):
+            return messages
+
+        # Limit number of messages
+        limited_messages = messages[:max_messages]
+
+        # Truncate each message content
+        for msg in limited_messages:
+            if isinstance(msg, dict) and "content" in msg:
+                content = msg["content"]
+                if len(content) > max_content_length:
+                    msg["content"] = content[:max_content_length] + "..."
+
+        return limited_messages

src/crewai/utilities/events/listeners/tracing/types.py

@@ -0,0 +1,19 @@
+from dataclasses import dataclass, field, asdict
+from datetime import datetime, timezone
+from typing import Dict, Any
+import uuid
+
+
+@dataclass
+class TraceEvent:
+    """Individual trace event payload"""
+
+    event_id: str = field(default_factory=lambda: str(uuid.uuid4()))
+    timestamp: str = field(
+        default_factory=lambda: datetime.now(timezone.utc).isoformat()
+    )
+    type: str = ""
+    event_data: Dict[str, Any] = field(default_factory=dict)
+
+    def to_dict(self) -> Dict[str, Any]:
+        return asdict(self)

src/crewai/utilities/events/llm_events.py

@@ -5,6 +5,7 @@ from pydantic import BaseModel
 
 from crewai.utilities.events.base_events import BaseEvent
 
+
 class LLMEventBase(BaseEvent):
     task_name: Optional[str] = None
     task_id: Optional[str] = None
@@ -32,6 +33,7 @@ class LLMEventBase(BaseEvent):
             self.task_id = task.id
             self.task_name = task.name
 
+
 class LLMCallType(Enum):
     """Type of LLM call being made"""
 
@@ -48,6 +50,7 @@ class LLMCallStartedEvent(LLMEventBase):
     """
 
     type: str = "llm_call_started"
+    model: Optional[str] = None
     messages: Optional[Union[str, List[Dict[str, Any]]]] = None
     tools: Optional[List[dict[str, Any]]] = None
     callbacks: Optional[List[Any]] = None
@@ -61,6 +64,8 @@ class LLMCallCompletedEvent(LLMEventBase):
     messages: str | list[dict[str, Any]] | None = None
     response: Any
     call_type: LLMCallType
+    model: Optional[str] = None
+
 
 class LLMCallFailedEvent(LLMEventBase):
     """Event emitted when a LLM call fails"""

tests/cassettes/TestTraceListenerSetup.test_batch_manager_finalizes_batch_clears_buffer.yaml

@@ -0,0 +1,470 @@
+interactions:
+- request:
+    body: '{"messages": [{"role": "system", "content": "You are Test Agent. Test backstory\nYour
+      personal goal is: Test goal\nTo give my best complete final answer to the task
+      respond using the exact following format:\n\nThought: I now can give a great
+      answer\nFinal Answer: Your final answer must be the great and the most complete
+      as possible, it must be outcome described.\n\nI MUST use these formats, my job
+      depends on it!"}, {"role": "user", "content": "\nCurrent Task: Say hello to
+      the world\n\nThis is the expected criteria for your final answer: hello world\nyou
+      MUST return the actual complete content as the final answer, not a summary.\n\nBegin!
+      This is VERY important to you, use the tools available and give your best Final
+      Answer, your job depends on it!\n\nThought:"}], "model": "gpt-4o-mini", "stop":
+      ["\nObservation:"]}'
+    headers:
+      accept:
+      - application/json
+      accept-encoding:
+      - gzip, deflate, zstd
+      connection:
+      - keep-alive
+      content-length:
+      - '825'
+      content-type:
+      - application/json
+      cookie:
+      - __cf_bm=ePO5hy0kEoADCuKcboFy1iS1qckCE5KCpifQaXnlomM-1754508545-1.0.1.1-ieWfjcdIxQIXGfaMizvmgTvZPRFehqDXliegaOT7EO.kt7KSSFGmNDcC35_D9hOhE.fJ5K302uX0snQF3nLaapds2dqgGbNcsyFPOKNvAdI;
+        _cfuvid=NaXWifUGChHp6Ap1mvfMrNzmO4HdzddrqXkSR9T.hYo-1754508545647-0.0.1.1-604800000
+      host:
+      - api.openai.com
+      user-agent:
+      - OpenAI/Python 1.93.0
+      x-stainless-arch:
+      - arm64
+      x-stainless-async:
+      - 'false'
+      x-stainless-lang:
+      - python
+      x-stainless-os:
+      - MacOS
+      x-stainless-package-version:
+      - 1.93.0
+      x-stainless-raw-response:
+      - 'true'
+      x-stainless-read-timeout:
+      - '600.0'
+      x-stainless-retry-count:
+      - '0'
+      x-stainless-runtime:
+      - CPython
+      x-stainless-runtime-version:
+      - 3.12.9
+    method: POST
+    uri: https://api.openai.com/v1/chat/completions
+  response:
+    body:
+      string: !!binary |
+        H4sIAAAAAAAAAwAAAP//jFLBbpwwEL3zFSOflwqykE25RZWiRuq9hzZCEzOAE+NxbbPbJtp/rwyb
+        hU1bqRck5s17fm9mXhMAoRpRgZA9BjlYnX7K6dq/jNkXaV9uyx/9vfx617Ed9mP+uRObyODHJ5Lh
+        jfVB8mA1BcVmhqUjDBRV811ZlNlNWVxPwMAN6UjrbEgLTgdlVHqVXRVptkvzmxO7ZyXJiwq+JQAA
+        r9M3+jQN/RQVZJu3ykDeY0eiOjcBCMc6VgR6r3xAE8RmASWbQGayfg+GDyDRQKf2BAhdtA1o/IEc
+        wHdzpwxquJ3+K+hJa4YDO92sBR21o8cYyoxarwA0hgPGoUxRHk7I8Wxec2cdP/p3VNEqo3xfO0LP
+        Jhr1ga2Y0GMC8DANabzILazjwYY68DNNz+XlbtYTy25W6PYEBg6oV/XdabSXenVDAZX2qzELibKn
+        ZqEuO8GxUbwCklXqP938TXtOrkz3P/ILICXZQE1tHTVKXiZe2hzF0/1X23nKk2Hhye2VpDoocnET
+        DbU46vmghP/lAw11q0xHzjo1X1Vr622BZYH0cStFckx+AwAA//8DAMHQtj5jAwAA
+    headers:
+      CF-RAY:
+      - 96b0f0f0ac9e7ad9-SJC
+      Connection:
+      - keep-alive
+      Content-Encoding:
+      - gzip
+      Content-Type:
+      - application/json
+      Date:
+      - Wed, 06 Aug 2025 19:29:07 GMT
+      Server:
+      - cloudflare
+      Strict-Transport-Security:
+      - max-age=31536000; includeSubDomains; preload
+      Transfer-Encoding:
+      - chunked
+      X-Content-Type-Options:
+      - nosniff
+      access-control-expose-headers:
+      - X-Request-ID
+      alt-svc:
+      - h3=":443"; ma=86400
+      cf-cache-status:
+      - DYNAMIC
+      openai-organization:
+      - crewai-iuxna1
+      openai-processing-ms:
+      - '653'
+      openai-project:
+      - proj_xitITlrFeen7zjNSzML82h9x
+      openai-version:
+      - '2020-10-01'
+      x-envoy-upstream-service-time:
+      - '667'
+      x-ratelimit-limit-project-tokens:
+      - '150000000'
+      x-ratelimit-limit-requests:
+      - '30000'
+      x-ratelimit-limit-tokens:
+      - '150000000'
+      x-ratelimit-remaining-project-tokens:
+      - '149999830'
+      x-ratelimit-remaining-requests:
+      - '29999'
+      x-ratelimit-remaining-tokens:
+      - '149999827'
+      x-ratelimit-reset-project-tokens:
+      - 0s
+      x-ratelimit-reset-requests:
+      - 2ms
+      x-ratelimit-reset-tokens:
+      - 0s
+      x-request-id:
+      - req_3f500b79ab1a400ea9e26d0f12e890bb
+    status:
+      code: 200
+      message: OK
+- request:
+    body: '{"messages": [{"role": "system", "content": "You are Test Agent. Test backstory\nYour
+      personal goal is: Test goal\nTo give my best complete final answer to the task
+      respond using the exact following format:\n\nThought: I now can give a great
+      answer\nFinal Answer: Your final answer must be the great and the most complete
+      as possible, it must be outcome described.\n\nI MUST use these formats, my job
+      depends on it!"}, {"role": "user", "content": "\nCurrent Task: Say hello to
+      the world\n\nThis is the expected criteria for your final answer: hello world\nyou
+      MUST return the actual complete content as the final answer, not a summary.\n\nBegin!
+      This is VERY important to you, use the tools available and give your best Final
+      Answer, your job depends on it!\n\nThought:"}], "model": "gpt-4o-mini"}'
+    headers:
+      accept:
+      - application/json
+      accept-encoding:
+      - gzip, deflate, zstd
+      connection:
+      - keep-alive
+      content-length:
+      - '797'
+      content-type:
+      - application/json
+      cookie:
+      - __cf_bm=f59gEPi_nA3TTxtjbKaSQpvkTwezaAqOvqfxiGzRnVQ-1754508546-1.0.1.1-JrSaytxVIQSVE00I.vyGj7d4HJbbMV6R9fWPJbkDKu0Y8ueMRzTwTUnfz0YzP5nsZX5oxoE6WlmFxOuz0rRuq9YhZZsO_TbaFBOFk1jGK9U;
+        _cfuvid=3D66v3.J_RcVoYy9dlF.jHwq1zTIm842xynZxzSy1Wc-1754508546352-0.0.1.1-604800000
+      host:
+      - api.openai.com
+      user-agent:
+      - OpenAI/Python 1.93.0
+      x-stainless-arch:
+      - arm64
+      x-stainless-async:
+      - 'false'
+      x-stainless-lang:
+      - python
+      x-stainless-os:
+      - MacOS
+      x-stainless-package-version:
+      - 1.93.0
+      x-stainless-raw-response:
+      - 'true'
+      x-stainless-read-timeout:
+      - '200.0'
+      x-stainless-retry-count:
+      - '0'
+      x-stainless-runtime:
+      - CPython
+      x-stainless-runtime-version:
+      - 3.12.9
+    method: POST
+    uri: https://api.openai.com/v1/chat/completions
+  response:
+    body:
+      string: !!binary |
+        H4sIAAAAAAAAAwAAAP//jFLBatwwEL37K6Y622V3Y8eJb6FQ2kIPoS0EmmAm8tirRNYISc52Cfvv
+        RfZm7bQp9GLwvHlP783McwIgVCMqEHKLQfZWZx/WdB42Nw9Xtux3dPMNr89/fPm+f7zct9dfRRoZ
+        fP9AMryw3kvuraag2EywdISBouq6LPJidVHk5Qj03JCOtM6GLOesV0Zlm9Umz1Zltr44sresJHlR
+        wc8EAOB5/EafpqFfooJV+lLpyXvsSFSnJgDhWMeKQO+VD2iCSGdQsglkRuufwfAOJBro1BMBQhdt
+        Axq/Iwdwaz4qgxquxv8KPpHWnMKOnW7eLSUdtYPHGMsMWi8ANIYDxrGMYe6OyOFkX3NnHd/7P6ii
+        VUb5be0IPZto1Qe2YkQPCcDdOKbhVXJhHfc21IEfaXxuXZSTnpi3s0SPYOCAelEvN+kbenVDAZX2
+        i0ELiXJLzUydt4JDo3gBJIvUf7t5S3tKrkz3P/IzICXZQE1tHTVKvk48tzmKx/uvttOUR8PCk3tS
+        kuqgyMVNNNTioKeTEn7vA/V1q0xHzjo13VVr67Mcixzp8kyK5JD8BgAA//8DAB06pnJlAwAA
+    headers:
+      CF-RAY:
+      - 96b0f0f54d6aeb2c-SJC
+      Connection:
+      - keep-alive
+      Content-Encoding:
+      - gzip
+      Content-Type:
+      - application/json
+      Date:
+      - Wed, 06 Aug 2025 19:29:08 GMT
+      Server:
+      - cloudflare
+      Strict-Transport-Security:
+      - max-age=31536000; includeSubDomains; preload
+      Transfer-Encoding:
+      - chunked
+      X-Content-Type-Options:
+      - nosniff
+      access-control-expose-headers:
+      - X-Request-ID
+      alt-svc:
+      - h3=":443"; ma=86400
+      cf-cache-status:
+      - DYNAMIC
+      openai-organization:
+      - crewai-iuxna1
+      openai-processing-ms:
+      - '809'
+      openai-project:
+      - proj_xitITlrFeen7zjNSzML82h9x
+      openai-version:
+      - '2020-10-01'
+      x-envoy-upstream-service-time:
+      - '823'
+      x-ratelimit-limit-project-tokens:
+      - '150000000'
+      x-ratelimit-limit-requests:
+      - '30000'
+      x-ratelimit-limit-tokens:
+      - '150000000'
+      x-ratelimit-remaining-project-tokens:
+      - '149999827'
+      x-ratelimit-remaining-requests:
+      - '29999'
+      x-ratelimit-remaining-tokens:
+      - '149999827'
+      x-ratelimit-reset-project-tokens:
+      - 0s
+      x-ratelimit-reset-requests:
+      - 2ms
+      x-ratelimit-reset-tokens:
+      - 0s
+      x-request-id:
+      - req_806f7071fb664da48953f5b216b56d9a
+    status:
+      code: 200
+      message: OK
+- request:
+    body: '{"trace_id": "eb9e0ee1-15ed-4044-b84b-f17e493a1e28", "execution_type":
+      "crew", "execution_context": {"crew_fingerprint": null, "crew_name": "crew",
+      "flow_name": "Unknown Flow", "crewai_version": "0.152.0", "privacy_level": "standard"},
+      "execution_metadata": {"expected_duration_estimate": 300, "agent_count": 0,
+      "task_count": 0, "flow_method_count": 0, "execution_started_at": "2025-08-06T19:30:52.210701+00:00"}}'
+    headers:
+      Accept:
+      - '*/*'
+      Accept-Encoding:
+      - gzip, deflate, zstd
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '413'
+      Content-Type:
+      - application/json
+      User-Agent:
+      - CrewAI-CLI/0.152.0
+      X-Crewai-Organization-Id:
+      - d3a3d10c-35db-423f-a7a4-c026030ba64d
+      X-Crewai-Version:
+      - 0.152.0
+    method: POST
+    uri: https://app.crewai.com/crewai_plus/api/v1/tracing/batches
+  response:
+    body:
+      string: "<!DOCTYPE html>\n<html>\n<head>\n  <title>The page you were looking
+        for doesn't exist (404)</title>\n  <meta name=\"viewport\" content=\"width=device-width,initial-scale=1\">\n
+        \ <style>\n  .rails-default-error-page {\n    background-color: #EFEFEF;\n
+        \   color: #2E2F30;\n    text-align: center;\n    font-family: arial, sans-serif;\n
+        \   margin: 0;\n  }\n\n  .rails-default-error-page div.dialog {\n    width:
+        95%;\n    max-width: 33em;\n    margin: 4em auto 0;\n  }\n\n  .rails-default-error-page
+        div.dialog > div {\n    border: 1px solid #CCC;\n    border-right-color: #999;\n
+        \   border-left-color: #999;\n    border-bottom-color: #BBB;\n    border-top:
+        #B00100 solid 4px;\n    border-top-left-radius: 9px;\n    border-top-right-radius:
+        9px;\n    background-color: white;\n    padding: 7px 12% 0;\n    box-shadow:
+        0 3px 8px rgba(50, 50, 50, 0.17);\n  }\n\n  .rails-default-error-page h1 {\n
+        \   font-size: 100%;\n    color: #730E15;\n    line-height: 1.5em;\n  }\n\n
+        \ .rails-default-error-page div.dialog > p {\n    margin: 0 0 1em;\n    padding:
+        1em;\n    background-color: #F7F7F7;\n    border: 1px solid #CCC;\n    border-right-color:
+        #999;\n    border-left-color: #999;\n    border-bottom-color: #999;\n    border-bottom-left-radius:
+        4px;\n    border-bottom-right-radius: 4px;\n    border-top-color: #DADADA;\n
+        \   color: #666;\n    box-shadow: 0 3px 8px rgba(50, 50, 50, 0.17);\n  }\n
+        \ </style>\n</head>\n\n<body class=\"rails-default-error-page\">\n  <!-- This
+        file lives in public/404.html -->\n  <div class=\"dialog\">\n    <div>\n      <h1>The
+        page you were looking for doesn't exist.</h1>\n      <p>You may have mistyped
+        the address or the page may have moved.</p>\n    </div>\n    <p>If you are
+        the application owner check the logs for more information.</p>\n  </div>\n</body>\n</html>\n"
+    headers:
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '1722'
+      Content-Type:
+      - text/html; charset=UTF-8
+      Date:
+      - Wed, 06 Aug 2025 19:30:52 GMT
+      strict-transport-security:
+      - max-age=63072000; includeSubDomains
+      x-request-id:
+      - bec0cf39-af9c-4955-b600-607187a7b10b
+      x-runtime:
+      - '0.005352'
+    status:
+      code: 404
+      message: Not Found
+- request:
+    body: '{"version": "0.152.0", "batch_id": "eb9e0ee1-15ed-4044-b84b-f17e493a1e28",
+      "user_context": {"user_id": "anonymous", "organization_id": "", "session_id":
+      "e7e7a716-e64b-490b-96db-5c5367042114", "trace_id": "54e95e1f-cd41-4ece-9e5e-21984d635e6a"},
+      "execution_metadata": {"crew_name": "crew", "execution_start": "2025-08-06T19:30:52.209750+00:00",
+      "crewai_version": "0.152.0"}, "events": [{"event_id": "98b2a833-63fc-457c-a2e0-6ce228a8214c",
+      "timestamp": "2025-08-06T19:30:52.328066+00:00", "type": "crew_kickoff_started",
+      "event_data": {"timestamp": "2025-08-06T19:30:52.209750+00:00", "type": "crew_kickoff_started",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "crew_name": "crew", "crew": null, "inputs": null}}, {"event_id": "4abf563c-d35f-4a09-867d-75c1c54b3fed",
+      "timestamp": "2025-08-06T19:30:52.328113+00:00", "type": "crew_kickoff_started",
+      "event_data": {"timestamp": "2025-08-06T19:30:52.209750+00:00", "type": "crew_kickoff_started",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "crew_name": "crew", "crew": null, "inputs": null}}, {"event_id": "60bdc932-6b56-4f1d-bcc2-5b3b57c8dc94",
+      "timestamp": "2025-08-06T19:30:52.330079+00:00", "type": "task_started", "event_data":
+      {"task_description": "Say hello to the world", "task_name": null, "context":
+      "", "agent": "Test Agent"}}, {"event_id": "97761b9f-d132-47e7-8857-5fdda8c80b65",
+      "timestamp": "2025-08-06T19:30:52.330089+00:00", "type": "task_started", "event_data":
+      {"task_description": "Say hello to the world", "task_name": null, "context":
+      "", "agent": "Test Agent"}}, {"event_id": "cdaa47c1-448f-476e-9761-14a25f26c481",
+      "timestamp": "2025-08-06T19:30:52.330477+00:00", "type": "agent_execution_started",
+      "event_data": {"serialization_error": "Circular reference detected (id repeated)",
+      "object_type": "AgentExecutionStartedEvent"}}, {"event_id": "7aa43738-3903-44cf-8416-d47542469537",
+      "timestamp": "2025-08-06T19:30:52.330612+00:00", "type": "agent_execution_started",
+      "event_data": {"serialization_error": "Circular reference detected (id repeated)",
+      "object_type": "AgentExecutionStartedEvent"}}, {"event_id": "6eb42795-be95-4f1c-b70f-385c59483e43",
+      "timestamp": "2025-08-06T19:30:52.330751+00:00", "type": "llm_call_started",
+      "event_data": {"timestamp": "2025-08-06T19:30:52.330725+00:00", "type": "llm_call_started",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "task_name": null, "task_id": "1bfe4b49-ba6a-464d-9b6a-ca2eb8e965d8", "agent_id":
+      "5d6dbe70-71fc-42e2-ba0d-61b460542dad", "agent_role": "Test Agent", "model":
+      "gpt-4o-mini", "messages": [{"role": "system", "content": "You are Test Agent.
+      Test backstory\nYour personal goal is: Test goal\nTo give my best complete final
+      answer to the task respond using the exact following format:\n\nThought: I now
+      can give a great answer\n..."}, {"role": "user", "content": "\nCurrent Task:
+      Say hello to the world\n\nThis is the expected criteria for your final answer:
+      hello world\nyou MUST return the actual complete content as the final answer,
+      not a summary.\n\nBegin! This is ..."}], "tools": null, "callbacks": ["<crewai.utilities.token_counter_callback.TokenCalcHandler
+      object at 0x11bd10a10>"], "available_functions": null}}, {"event_id": "679e3211-ef91-45c0-9d4a-e5118e653dbd",
+      "timestamp": "2025-08-06T19:30:52.330798+00:00", "type": "llm_call_started",
+      "event_data": {"timestamp": "2025-08-06T19:30:52.330725+00:00", "type": "llm_call_started",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "task_name": null, "task_id": "1bfe4b49-ba6a-464d-9b6a-ca2eb8e965d8", "agent_id":
+      "5d6dbe70-71fc-42e2-ba0d-61b460542dad", "agent_role": "Test Agent", "model":
+      "gpt-4o-mini", "messages": [{"role": "system", "content": "You are Test Agent.
+      Test backstory\nYour personal goal is: Test goal\nTo give my best complete final
+      answer to the task respond using the exact following format:\n\nThought: I now
+      can give a great answer\n..."}, {"role": "user", "content": "\nCurrent Task:
+      Say hello to the world\n\nThis is the expected criteria for your final answer:
+      hello world\nyou MUST return the actual complete content as the final answer,
+      not a summary.\n\nBegin! This is ..."}], "tools": null, "callbacks": ["<crewai.utilities.token_counter_callback.TokenCalcHandler
+      object at 0x11bd10a10>"], "available_functions": null}}, {"event_id": "911c67ea-125b-4adf-87a5-4a9265575f93",
+      "timestamp": "2025-08-06T19:30:52.335757+00:00", "type": "llm_call_completed",
+      "event_data": {"timestamp": "2025-08-06T19:30:52.335728+00:00", "type": "llm_call_completed",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "task_name": null, "task_id": "1bfe4b49-ba6a-464d-9b6a-ca2eb8e965d8", "agent_id":
+      "5d6dbe70-71fc-42e2-ba0d-61b460542dad", "agent_role": "Test Agent", "messages":
+      [{"role": "system", "content": "You are Test Agent. Test backstory\nYour personal
+      goal is: Test goal\nTo give my best complete final answer to the task respond
+      using the exact following format:\n\nThought: I now can give a great answer\n..."},
+      {"role": "user", "content": "\nCurrent Task: Say hello to the world\n\nThis
+      is the expected criteria for your final answer: hello world\nyou MUST return
+      the actual complete content as the final answer, not a summary.\n\nBegin! This
+      is ..."}], "response": "I now can give a great answer  \nFinal Answer: hello
+      world", "call_type": "<LLMCallType.LLM_CALL: ''llm_call''>", "response_cost":
+      3.255e-05, "model": "gpt-4o-mini"}}, {"event_id": "1c93586f-82b9-4999-adda-78c8010b59f6",
+      "timestamp": "2025-08-06T19:30:52.335800+00:00", "type": "llm_call_completed",
+      "event_data": {"timestamp": "2025-08-06T19:30:52.335728+00:00", "type": "llm_call_completed",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "task_name": null, "task_id": "1bfe4b49-ba6a-464d-9b6a-ca2eb8e965d8", "agent_id":
+      "5d6dbe70-71fc-42e2-ba0d-61b460542dad", "agent_role": "Test Agent", "messages":
+      [{"role": "system", "content": "You are Test Agent. Test backstory\nYour personal
+      goal is: Test goal\nTo give my best complete final answer to the task respond
+      using the exact following format:\n\nThought: I now can give a great answer\n..."},
+      {"role": "user", "content": "\nCurrent Task: Say hello to the world\n\nThis
+      is the expected criteria for your final answer: hello world\nyou MUST return
+      the actual complete content as the final answer, not a summary.\n\nBegin! This
+      is ..."}], "response": "I now can give a great answer  \nFinal Answer: hello
+      world", "call_type": "<LLMCallType.LLM_CALL: ''llm_call''>", "response_cost":
+      3.255e-05, "model": "gpt-4o-mini"}}, {"event_id": "eb9d53af-ce39-4241-ab93-e40545a1ee78",
+      "timestamp": "2025-08-06T19:30:52.335904+00:00", "type": "agent_execution_completed",
+      "event_data": {"serialization_error": "Circular reference detected (id repeated)",
+      "object_type": "AgentExecutionCompletedEvent"}}, {"event_id": "dc71f5f8-5762-4e44-ac60-aa20b033c9f9",
+      "timestamp": "2025-08-06T19:30:52.335989+00:00", "type": "agent_execution_completed",
+      "event_data": {"serialization_error": "Circular reference detected (id repeated)",
+      "object_type": "AgentExecutionCompletedEvent"}}, {"event_id": "84da8fb8-9247-4718-bc85-a69033c9261f",
+      "timestamp": "2025-08-06T19:30:52.336082+00:00", "type": "task_completed", "event_data":
+      {"serialization_error": "Circular reference detected (id repeated)", "object_type":
+      "TaskCompletedEvent"}}, {"event_id": "c1a23877-2b87-40be-98a1-a3b2630c8657",
+      "timestamp": "2025-08-06T19:30:52.336107+00:00", "type": "task_completed", "event_data":
+      {"serialization_error": "Circular reference detected (id repeated)", "object_type":
+      "TaskCompletedEvent"}}, {"event_id": "c77587d7-68d6-4600-b98a-74fe58af41fc",
+      "timestamp": "2025-08-06T19:30:52.337164+00:00", "type": "crew_kickoff_completed",
+      "event_data": {"timestamp": "2025-08-06T19:30:52.337145+00:00", "type": "crew_kickoff_completed",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "crew_name": "crew", "crew": null, "output": {"description": "Say hello to the
+      world", "name": null, "expected_output": "hello world", "summary": "Say hello
+      to the world...", "raw": "hello world", "pydantic": null, "json_dict": null,
+      "agent": "Test Agent", "output_format": "raw"}, "total_tokens": 170}}]}'
+    headers:
+      Accept:
+      - '*/*'
+      Accept-Encoding:
+      - gzip, deflate, zstd
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '8300'
+      Content-Type:
+      - application/json
+      User-Agent:
+      - CrewAI-CLI/0.152.0
+      X-Crewai-Organization-Id:
+      - d3a3d10c-35db-423f-a7a4-c026030ba64d
+      X-Crewai-Version:
+      - 0.152.0
+    method: POST
+    uri: https://app.crewai.com/crewai_plus/api/v1/tracing
+  response:
+    body:
+      string: "<!DOCTYPE html>\n<html>\n<head>\n  <title>The page you were looking
+        for doesn't exist (404)</title>\n  <meta name=\"viewport\" content=\"width=device-width,initial-scale=1\">\n
+        \ <style>\n  .rails-default-error-page {\n    background-color: #EFEFEF;\n
+        \   color: #2E2F30;\n    text-align: center;\n    font-family: arial, sans-serif;\n
+        \   margin: 0;\n  }\n\n  .rails-default-error-page div.dialog {\n    width:
+        95%;\n    max-width: 33em;\n    margin: 4em auto 0;\n  }\n\n  .rails-default-error-page
+        div.dialog > div {\n    border: 1px solid #CCC;\n    border-right-color: #999;\n
+        \   border-left-color: #999;\n    border-bottom-color: #BBB;\n    border-top:
+        #B00100 solid 4px;\n    border-top-left-radius: 9px;\n    border-top-right-radius:
+        9px;\n    background-color: white;\n    padding: 7px 12% 0;\n    box-shadow:
+        0 3px 8px rgba(50, 50, 50, 0.17);\n  }\n\n  .rails-default-error-page h1 {\n
+        \   font-size: 100%;\n    color: #730E15;\n    line-height: 1.5em;\n  }\n\n
+        \ .rails-default-error-page div.dialog > p {\n    margin: 0 0 1em;\n    padding:
+        1em;\n    background-color: #F7F7F7;\n    border: 1px solid #CCC;\n    border-right-color:
+        #999;\n    border-left-color: #999;\n    border-bottom-color: #999;\n    border-bottom-left-radius:
+        4px;\n    border-bottom-right-radius: 4px;\n    border-top-color: #DADADA;\n
+        \   color: #666;\n    box-shadow: 0 3px 8px rgba(50, 50, 50, 0.17);\n  }\n
+        \ </style>\n</head>\n\n<body class=\"rails-default-error-page\">\n  <!-- This
+        file lives in public/404.html -->\n  <div class=\"dialog\">\n    <div>\n      <h1>The
+        page you were looking for doesn't exist.</h1>\n      <p>You may have mistyped
+        the address or the page may have moved.</p>\n    </div>\n    <p>If you are
+        the application owner check the logs for more information.</p>\n  </div>\n</body>\n</html>\n"
+    headers:
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '1722'
+      Content-Type:
+      - text/html; charset=UTF-8
+      Date:
+      - Wed, 06 Aug 2025 19:30:52 GMT
+      strict-transport-security:
+      - max-age=63072000; includeSubDomains
+      x-request-id:
+      - 78674bcb-6c8a-4eaf-8577-5cb27cac4089
+      x-runtime:
+      - '0.006009'
+    status:
+      code: 404
+      message: Not Found
+version: 1

tests/cassettes/TestTraceListenerSetup.test_events_collection_batch_manager.yaml

@@ -0,0 +1,470 @@
+interactions:
+- request:
+    body: '{"messages": [{"role": "system", "content": "You are Test Agent. Test backstory\nYour
+      personal goal is: Test goal\nTo give my best complete final answer to the task
+      respond using the exact following format:\n\nThought: I now can give a great
+      answer\nFinal Answer: Your final answer must be the great and the most complete
+      as possible, it must be outcome described.\n\nI MUST use these formats, my job
+      depends on it!"}, {"role": "user", "content": "\nCurrent Task: Say hello to
+      the world\n\nThis is the expected criteria for your final answer: hello world\nyou
+      MUST return the actual complete content as the final answer, not a summary.\n\nBegin!
+      This is VERY important to you, use the tools available and give your best Final
+      Answer, your job depends on it!\n\nThought:"}], "model": "gpt-4o-mini", "stop":
+      ["\nObservation:"]}'
+    headers:
+      accept:
+      - application/json
+      accept-encoding:
+      - gzip, deflate, zstd
+      connection:
+      - keep-alive
+      content-length:
+      - '825'
+      content-type:
+      - application/json
+      cookie:
+      - __cf_bm=ePO5hy0kEoADCuKcboFy1iS1qckCE5KCpifQaXnlomM-1754508545-1.0.1.1-ieWfjcdIxQIXGfaMizvmgTvZPRFehqDXliegaOT7EO.kt7KSSFGmNDcC35_D9hOhE.fJ5K302uX0snQF3nLaapds2dqgGbNcsyFPOKNvAdI;
+        _cfuvid=NaXWifUGChHp6Ap1mvfMrNzmO4HdzddrqXkSR9T.hYo-1754508545647-0.0.1.1-604800000
+      host:
+      - api.openai.com
+      user-agent:
+      - OpenAI/Python 1.93.0
+      x-stainless-arch:
+      - arm64
+      x-stainless-async:
+      - 'false'
+      x-stainless-lang:
+      - python
+      x-stainless-os:
+      - MacOS
+      x-stainless-package-version:
+      - 1.93.0
+      x-stainless-raw-response:
+      - 'true'
+      x-stainless-read-timeout:
+      - '600.0'
+      x-stainless-retry-count:
+      - '0'
+      x-stainless-runtime:
+      - CPython
+      x-stainless-runtime-version:
+      - 3.12.9
+    method: POST
+    uri: https://api.openai.com/v1/chat/completions
+  response:
+    body:
+      string: !!binary |
+        H4sIAAAAAAAAAwAAAP//jFJNa9wwEL37Vww6x8Wbtbu7voXQQC+lh0Jb2mAm0thWI0tCkuOWsP+9
+        SN6svf2AXAyeN+/pvZl5zgCYFKwGxnsMfLAqv93Q22mP+O7z9OlwUIW4LT7S+LX68nj3YWJXkWEe
+        fhAPL6w33AxWUZBGzzB3hIGi6mZXlVWxr6oiAYMRpCKtsyEvTT5ILfPr4rrMi12+2Z/YvZGcPKvh
+        WwYA8Jy+0acW9JPVkLRSZSDvsSNWn5sAmDMqVhh6L31AHdjVAnKjA+lk/T1oMwFHDZ18IkDoom1A
+        7SdyAN/1ndSo4Cb919CTUgYm45RYCzpqR48xlB6VWgGotQkYh5Ki3J+Q49m8Mp115sH/QWWt1NL3
+        jSP0RkejPhjLEnrMAO7TkMaL3Mw6M9jQBPNI6blNtZv12LKbFbo9gcEEVKv67jTaS71GUECp/GrM
+        jCPvSSzUZSc4CmlWQLZK/bebf2nPyaXuXiO/AJyTDSQa60hIfpl4aXMUT/d/becpJ8PMk3uSnJog
+        ycVNCGpxVPNBMf/LBxqaVuqOnHVyvqrWNtsSqxLpsOUsO2a/AQAA//8DAD59q5pjAwAA
+    headers:
+      CF-RAY:
+      - 96b0f1059ae17ad9-SJC
+      Connection:
+      - keep-alive
+      Content-Encoding:
+      - gzip
+      Content-Type:
+      - application/json
+      Date:
+      - Wed, 06 Aug 2025 19:29:10 GMT
+      Server:
+      - cloudflare
+      Strict-Transport-Security:
+      - max-age=31536000; includeSubDomains; preload
+      Transfer-Encoding:
+      - chunked
+      X-Content-Type-Options:
+      - nosniff
+      access-control-expose-headers:
+      - X-Request-ID
+      alt-svc:
+      - h3=":443"; ma=86400
+      cf-cache-status:
+      - DYNAMIC
+      openai-organization:
+      - crewai-iuxna1
+      openai-processing-ms:
+      - '521'
+      openai-project:
+      - proj_xitITlrFeen7zjNSzML82h9x
+      openai-version:
+      - '2020-10-01'
+      x-envoy-upstream-service-time:
+      - '537'
+      x-ratelimit-limit-project-tokens:
+      - '150000000'
+      x-ratelimit-limit-requests:
+      - '30000'
+      x-ratelimit-limit-tokens:
+      - '150000000'
+      x-ratelimit-remaining-project-tokens:
+      - '149999827'
+      x-ratelimit-remaining-requests:
+      - '29999'
+      x-ratelimit-remaining-tokens:
+      - '149999827'
+      x-ratelimit-reset-project-tokens:
+      - 0s
+      x-ratelimit-reset-requests:
+      - 2ms
+      x-ratelimit-reset-tokens:
+      - 0s
+      x-request-id:
+      - req_c94c2a416aee4c93bae1f801c8ae3e72
+    status:
+      code: 200
+      message: OK
+- request:
+    body: '{"messages": [{"role": "system", "content": "You are Test Agent. Test backstory\nYour
+      personal goal is: Test goal\nTo give my best complete final answer to the task
+      respond using the exact following format:\n\nThought: I now can give a great
+      answer\nFinal Answer: Your final answer must be the great and the most complete
+      as possible, it must be outcome described.\n\nI MUST use these formats, my job
+      depends on it!"}, {"role": "user", "content": "\nCurrent Task: Say hello to
+      the world\n\nThis is the expected criteria for your final answer: hello world\nyou
+      MUST return the actual complete content as the final answer, not a summary.\n\nBegin!
+      This is VERY important to you, use the tools available and give your best Final
+      Answer, your job depends on it!\n\nThought:"}], "model": "gpt-4o-mini"}'
+    headers:
+      accept:
+      - application/json
+      accept-encoding:
+      - gzip, deflate, zstd
+      connection:
+      - keep-alive
+      content-length:
+      - '797'
+      content-type:
+      - application/json
+      cookie:
+      - __cf_bm=f59gEPi_nA3TTxtjbKaSQpvkTwezaAqOvqfxiGzRnVQ-1754508546-1.0.1.1-JrSaytxVIQSVE00I.vyGj7d4HJbbMV6R9fWPJbkDKu0Y8ueMRzTwTUnfz0YzP5nsZX5oxoE6WlmFxOuz0rRuq9YhZZsO_TbaFBOFk1jGK9U;
+        _cfuvid=3D66v3.J_RcVoYy9dlF.jHwq1zTIm842xynZxzSy1Wc-1754508546352-0.0.1.1-604800000
+      host:
+      - api.openai.com
+      user-agent:
+      - OpenAI/Python 1.93.0
+      x-stainless-arch:
+      - arm64
+      x-stainless-async:
+      - 'false'
+      x-stainless-lang:
+      - python
+      x-stainless-os:
+      - MacOS
+      x-stainless-package-version:
+      - 1.93.0
+      x-stainless-raw-response:
+      - 'true'
+      x-stainless-read-timeout:
+      - '200.0'
+      x-stainless-retry-count:
+      - '0'
+      x-stainless-runtime:
+      - CPython
+      x-stainless-runtime-version:
+      - 3.12.9
+    method: POST
+    uri: https://api.openai.com/v1/chat/completions
+  response:
+    body:
+      string: !!binary |
+        H4sIAAAAAAAAAwAAAP//jFJNa9wwEL37Vww6x2U/7GzqW2j6BYVSaE5tMBN5bKuVNYok7zaE/e9F
+        2u3a2ybQi8Hz5j29NzNPGYBQjahAyB6DHKzO3yzpcnf74f3nh6+rx/bT9saa2+u343D55eGmFBeR
+        wfc/SIY/rFeSB6spKDYHWDrCQFF1uSmLcnFVlosEDNyQjrTOhrzgfFBG5avFqsgXm3x5dWT3rCR5
+        UcG3DADgKX2jT9PQL1FB0kqVgbzHjkR1agIQjnWsCPRe+YAmiIsJlGwCmWT9IxjegUQDndoSIHTR
+        NqDxO3IA3807ZVDDdfqvoCetGXbsdDMXdNSOHmMoM2o9A9AYDhiHkqLcHZH9ybzmzjq+939RRauM
+        8n3tCD2baNQHtiKh+wzgLg1pPMstrOPBhjrwT0rPLcvNQU9Mu5mh6yMYOKCe1TfH0Z7r1Q0FVNrP
+        xiwkyp6aiTrtBMdG8QzIZqn/dfOc9iG5Mt3/yE+AlGQDNbV11Ch5nnhqcxRP96W205STYeHJbZWk
+        OihycRMNtTjqw0EJ/+gDDXWrTEfOOnW4qtbW6wLLAun1Wopsn/0GAAD//wMASJr3q2MDAAA=
+    headers:
+      CF-RAY:
+      - 96b0f109ae7aeb2c-SJC
+      Connection:
+      - keep-alive
+      Content-Encoding:
+      - gzip
+      Content-Type:
+      - application/json
+      Date:
+      - Wed, 06 Aug 2025 19:29:11 GMT
+      Server:
+      - cloudflare
+      Strict-Transport-Security:
+      - max-age=31536000; includeSubDomains; preload
+      Transfer-Encoding:
+      - chunked
+      X-Content-Type-Options:
+      - nosniff
+      access-control-expose-headers:
+      - X-Request-ID
+      alt-svc:
+      - h3=":443"; ma=86400
+      cf-cache-status:
+      - DYNAMIC
+      openai-organization:
+      - crewai-iuxna1
+      openai-processing-ms:
+      - '499'
+      openai-project:
+      - proj_xitITlrFeen7zjNSzML82h9x
+      openai-version:
+      - '2020-10-01'
+      x-envoy-upstream-service-time:
+      - '511'
+      x-ratelimit-limit-project-tokens:
+      - '150000000'
+      x-ratelimit-limit-requests:
+      - '30000'
+      x-ratelimit-limit-tokens:
+      - '150000000'
+      x-ratelimit-remaining-project-tokens:
+      - '149999830'
+      x-ratelimit-remaining-requests:
+      - '29999'
+      x-ratelimit-remaining-tokens:
+      - '149999827'
+      x-ratelimit-reset-project-tokens:
+      - 0s
+      x-ratelimit-reset-requests:
+      - 2ms
+      x-ratelimit-reset-tokens:
+      - 0s
+      x-request-id:
+      - req_dece4be9f37c4d64b324ab36d1ed9cf4
+    status:
+      code: 200
+      message: OK
+- request:
+    body: '{"trace_id": "ff5ac8a9-dec2-4b73-8928-3dd06d12051f", "execution_type":
+      "crew", "execution_context": {"crew_fingerprint": null, "crew_name": "crew",
+      "flow_name": "Unknown Flow", "crewai_version": "0.152.0", "privacy_level": "standard"},
+      "execution_metadata": {"expected_duration_estimate": 300, "agent_count": 0,
+      "task_count": 0, "flow_method_count": 0, "execution_started_at": "2025-08-06T19:30:51.727534+00:00"}}'
+    headers:
+      Accept:
+      - '*/*'
+      Accept-Encoding:
+      - gzip, deflate, zstd
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '413'
+      Content-Type:
+      - application/json
+      User-Agent:
+      - CrewAI-CLI/0.152.0
+      X-Crewai-Organization-Id:
+      - d3a3d10c-35db-423f-a7a4-c026030ba64d
+      X-Crewai-Version:
+      - 0.152.0
+    method: POST
+    uri: https://app.crewai.com/crewai_plus/api/v1/tracing/batches
+  response:
+    body:
+      string: "<!DOCTYPE html>\n<html>\n<head>\n  <title>The page you were looking
+        for doesn't exist (404)</title>\n  <meta name=\"viewport\" content=\"width=device-width,initial-scale=1\">\n
+        \ <style>\n  .rails-default-error-page {\n    background-color: #EFEFEF;\n
+        \   color: #2E2F30;\n    text-align: center;\n    font-family: arial, sans-serif;\n
+        \   margin: 0;\n  }\n\n  .rails-default-error-page div.dialog {\n    width:
+        95%;\n    max-width: 33em;\n    margin: 4em auto 0;\n  }\n\n  .rails-default-error-page
+        div.dialog > div {\n    border: 1px solid #CCC;\n    border-right-color: #999;\n
+        \   border-left-color: #999;\n    border-bottom-color: #BBB;\n    border-top:
+        #B00100 solid 4px;\n    border-top-left-radius: 9px;\n    border-top-right-radius:
+        9px;\n    background-color: white;\n    padding: 7px 12% 0;\n    box-shadow:
+        0 3px 8px rgba(50, 50, 50, 0.17);\n  }\n\n  .rails-default-error-page h1 {\n
+        \   font-size: 100%;\n    color: #730E15;\n    line-height: 1.5em;\n  }\n\n
+        \ .rails-default-error-page div.dialog > p {\n    margin: 0 0 1em;\n    padding:
+        1em;\n    background-color: #F7F7F7;\n    border: 1px solid #CCC;\n    border-right-color:
+        #999;\n    border-left-color: #999;\n    border-bottom-color: #999;\n    border-bottom-left-radius:
+        4px;\n    border-bottom-right-radius: 4px;\n    border-top-color: #DADADA;\n
+        \   color: #666;\n    box-shadow: 0 3px 8px rgba(50, 50, 50, 0.17);\n  }\n
+        \ </style>\n</head>\n\n<body class=\"rails-default-error-page\">\n  <!-- This
+        file lives in public/404.html -->\n  <div class=\"dialog\">\n    <div>\n      <h1>The
+        page you were looking for doesn't exist.</h1>\n      <p>You may have mistyped
+        the address or the page may have moved.</p>\n    </div>\n    <p>If you are
+        the application owner check the logs for more information.</p>\n  </div>\n</body>\n</html>\n"
+    headers:
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '1722'
+      Content-Type:
+      - text/html; charset=UTF-8
+      Date:
+      - Wed, 06 Aug 2025 19:30:51 GMT
+      strict-transport-security:
+      - max-age=63072000; includeSubDomains
+      x-request-id:
+      - 0b6a5ff5-789e-4c0d-a10b-316fecc0e905
+      x-runtime:
+      - '0.005528'
+    status:
+      code: 404
+      message: Not Found
+- request:
+    body: '{"version": "0.152.0", "batch_id": "ff5ac8a9-dec2-4b73-8928-3dd06d12051f",
+      "user_context": {"user_id": "anonymous", "organization_id": "", "session_id":
+      "aabc00e7-d423-4385-8b83-0468c03ae47b", "trace_id": "0a0586da-135c-4080-a352-dbe47bb2ac86"},
+      "execution_metadata": {"crew_name": "crew", "execution_start": "2025-08-06T19:30:51.726805+00:00",
+      "crewai_version": "0.152.0"}, "events": [{"event_id": "211eb90d-fb76-4ee5-bee7-62cc2f1d9aa8",
+      "timestamp": "2025-08-06T19:30:51.842887+00:00", "type": "crew_kickoff_started",
+      "event_data": {"timestamp": "2025-08-06T19:30:51.726805+00:00", "type": "crew_kickoff_started",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "crew_name": "crew", "crew": null, "inputs": null}}, {"event_id": "713e4dbd-887f-4481-a6c8-554b637848e2",
+      "timestamp": "2025-08-06T19:30:51.842982+00:00", "type": "crew_kickoff_started",
+      "event_data": {"timestamp": "2025-08-06T19:30:51.726805+00:00", "type": "crew_kickoff_started",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "crew_name": "crew", "crew": null, "inputs": null}}, {"event_id": "b920108c-c6fe-40d7-baa3-29c23d76a8e1",
+      "timestamp": "2025-08-06T19:30:51.844489+00:00", "type": "task_started", "event_data":
+      {"task_description": "Say hello to the world", "task_name": null, "context":
+      "", "agent": "Test Agent"}}, {"event_id": "96180117-d060-49ab-8327-712f230653f2",
+      "timestamp": "2025-08-06T19:30:51.844512+00:00", "type": "task_started", "event_data":
+      {"task_description": "Say hello to the world", "task_name": null, "context":
+      "", "agent": "Test Agent"}}, {"event_id": "82baa39d-d1ae-44f8-8f35-40646fdec793",
+      "timestamp": "2025-08-06T19:30:51.845195+00:00", "type": "agent_execution_started",
+      "event_data": {"serialization_error": "Circular reference detected (id repeated)",
+      "object_type": "AgentExecutionStartedEvent"}}, {"event_id": "c34d2e12-6671-4593-a45d-8742704f6ace",
+      "timestamp": "2025-08-06T19:30:51.845868+00:00", "type": "agent_execution_started",
+      "event_data": {"serialization_error": "Circular reference detected (id repeated)",
+      "object_type": "AgentExecutionStartedEvent"}}, {"event_id": "87d12818-f0b4-46d0-8ecc-e46afaf8eddb",
+      "timestamp": "2025-08-06T19:30:51.846100+00:00", "type": "llm_call_started",
+      "event_data": {"timestamp": "2025-08-06T19:30:51.846006+00:00", "type": "llm_call_started",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "task_name": null, "task_id": "7f026c34-8c77-4710-8ecb-9d4c830b9eb4", "agent_id":
+      "bd02dc4e-982e-481c-9358-2b4a7ac73831", "agent_role": "Test Agent", "model":
+      "gpt-4o-mini", "messages": [{"role": "system", "content": "You are Test Agent.
+      Test backstory\nYour personal goal is: Test goal\nTo give my best complete final
+      answer to the task respond using the exact following format:\n\nThought: I now
+      can give a great answer\n..."}, {"role": "user", "content": "\nCurrent Task:
+      Say hello to the world\n\nThis is the expected criteria for your final answer:
+      hello world\nyou MUST return the actual complete content as the final answer,
+      not a summary.\n\nBegin! This is ..."}], "tools": null, "callbacks": ["<crewai.utilities.token_counter_callback.TokenCalcHandler
+      object at 0x11bd13470>"], "available_functions": null}}, {"event_id": "bbfd4480-87aa-4a56-b988-2dcc9e142c20",
+      "timestamp": "2025-08-06T19:30:51.846155+00:00", "type": "llm_call_started",
+      "event_data": {"timestamp": "2025-08-06T19:30:51.846006+00:00", "type": "llm_call_started",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "task_name": null, "task_id": "7f026c34-8c77-4710-8ecb-9d4c830b9eb4", "agent_id":
+      "bd02dc4e-982e-481c-9358-2b4a7ac73831", "agent_role": "Test Agent", "model":
+      "gpt-4o-mini", "messages": [{"role": "system", "content": "You are Test Agent.
+      Test backstory\nYour personal goal is: Test goal\nTo give my best complete final
+      answer to the task respond using the exact following format:\n\nThought: I now
+      can give a great answer\n..."}, {"role": "user", "content": "\nCurrent Task:
+      Say hello to the world\n\nThis is the expected criteria for your final answer:
+      hello world\nyou MUST return the actual complete content as the final answer,
+      not a summary.\n\nBegin! This is ..."}], "tools": null, "callbacks": ["<crewai.utilities.token_counter_callback.TokenCalcHandler
+      object at 0x11bd13470>"], "available_functions": null}}, {"event_id": "25a17ec7-b2ee-4eeb-bdf5-27efffed961c",
+      "timestamp": "2025-08-06T19:30:52.018207+00:00", "type": "llm_call_completed",
+      "event_data": {"timestamp": "2025-08-06T19:30:52.017914+00:00", "type": "llm_call_completed",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "task_name": null, "task_id": "7f026c34-8c77-4710-8ecb-9d4c830b9eb4", "agent_id":
+      "bd02dc4e-982e-481c-9358-2b4a7ac73831", "agent_role": "Test Agent", "messages":
+      [{"role": "system", "content": "You are Test Agent. Test backstory\nYour personal
+      goal is: Test goal\nTo give my best complete final answer to the task respond
+      using the exact following format:\n\nThought: I now can give a great answer\n..."},
+      {"role": "user", "content": "\nCurrent Task: Say hello to the world\n\nThis
+      is the expected criteria for your final answer: hello world\nyou MUST return
+      the actual complete content as the final answer, not a summary.\n\nBegin! This
+      is ..."}], "response": "I now can give a great answer  \nFinal Answer: hello
+      world", "call_type": "<LLMCallType.LLM_CALL: ''llm_call''>", "response_cost":
+      3.135e-05, "model": "gpt-4o-mini"}}, {"event_id": "0ccb9b70-c5ad-4f7f-b3ee-ecfd62c2d7cc",
+      "timestamp": "2025-08-06T19:30:52.018273+00:00", "type": "llm_call_completed",
+      "event_data": {"timestamp": "2025-08-06T19:30:52.017914+00:00", "type": "llm_call_completed",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "task_name": null, "task_id": "7f026c34-8c77-4710-8ecb-9d4c830b9eb4", "agent_id":
+      "bd02dc4e-982e-481c-9358-2b4a7ac73831", "agent_role": "Test Agent", "messages":
+      [{"role": "system", "content": "You are Test Agent. Test backstory\nYour personal
+      goal is: Test goal\nTo give my best complete final answer to the task respond
+      using the exact following format:\n\nThought: I now can give a great answer\n..."},
+      {"role": "user", "content": "\nCurrent Task: Say hello to the world\n\nThis
+      is the expected criteria for your final answer: hello world\nyou MUST return
+      the actual complete content as the final answer, not a summary.\n\nBegin! This
+      is ..."}], "response": "I now can give a great answer  \nFinal Answer: hello
+      world", "call_type": "<LLMCallType.LLM_CALL: ''llm_call''>", "response_cost":
+      3.135e-05, "model": "gpt-4o-mini"}}, {"event_id": "d7f4440b-8f9f-4e29-a946-6d10f4bdfc3c",
+      "timestamp": "2025-08-06T19:30:52.018559+00:00", "type": "agent_execution_completed",
+      "event_data": {"serialization_error": "Circular reference detected (id repeated)",
+      "object_type": "AgentExecutionCompletedEvent"}}, {"event_id": "072195c3-54df-4cba-9068-b9a25bbb8d7c",
+      "timestamp": "2025-08-06T19:30:52.018669+00:00", "type": "agent_execution_completed",
+      "event_data": {"serialization_error": "Circular reference detected (id repeated)",
+      "object_type": "AgentExecutionCompletedEvent"}}, {"event_id": "0b6f9e85-32c9-4c62-9049-6890953e2143",
+      "timestamp": "2025-08-06T19:30:52.018838+00:00", "type": "task_completed", "event_data":
+      {"serialization_error": "Circular reference detected (id repeated)", "object_type":
+      "TaskCompletedEvent"}}, {"event_id": "5ff20fcb-ec10-40ac-bb90-9568aa4eb1de",
+      "timestamp": "2025-08-06T19:30:52.018867+00:00", "type": "task_completed", "event_data":
+      {"serialization_error": "Circular reference detected (id repeated)", "object_type":
+      "TaskCompletedEvent"}}, {"event_id": "c5a36300-3911-4d75-a660-d133a7a4be94",
+      "timestamp": "2025-08-06T19:30:52.020135+00:00", "type": "crew_kickoff_completed",
+      "event_data": {"timestamp": "2025-08-06T19:30:52.020115+00:00", "type": "crew_kickoff_completed",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "crew_name": "crew", "crew": null, "output": {"description": "Say hello to the
+      world", "name": null, "expected_output": "hello world", "summary": "Say hello
+      to the world...", "raw": "hello world", "pydantic": null, "json_dict": null,
+      "agent": "Test Agent", "output_format": "raw"}, "total_tokens": 170}}]}'
+    headers:
+      Accept:
+      - '*/*'
+      Accept-Encoding:
+      - gzip, deflate, zstd
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '8300'
+      Content-Type:
+      - application/json
+      User-Agent:
+      - CrewAI-CLI/0.152.0
+      X-Crewai-Organization-Id:
+      - d3a3d10c-35db-423f-a7a4-c026030ba64d
+      X-Crewai-Version:
+      - 0.152.0
+    method: POST
+    uri: https://app.crewai.com/crewai_plus/api/v1/tracing
+  response:
+    body:
+      string: "<!DOCTYPE html>\n<html>\n<head>\n  <title>The page you were looking
+        for doesn't exist (404)</title>\n  <meta name=\"viewport\" content=\"width=device-width,initial-scale=1\">\n
+        \ <style>\n  .rails-default-error-page {\n    background-color: #EFEFEF;\n
+        \   color: #2E2F30;\n    text-align: center;\n    font-family: arial, sans-serif;\n
+        \   margin: 0;\n  }\n\n  .rails-default-error-page div.dialog {\n    width:
+        95%;\n    max-width: 33em;\n    margin: 4em auto 0;\n  }\n\n  .rails-default-error-page
+        div.dialog > div {\n    border: 1px solid #CCC;\n    border-right-color: #999;\n
+        \   border-left-color: #999;\n    border-bottom-color: #BBB;\n    border-top:
+        #B00100 solid 4px;\n    border-top-left-radius: 9px;\n    border-top-right-radius:
+        9px;\n    background-color: white;\n    padding: 7px 12% 0;\n    box-shadow:
+        0 3px 8px rgba(50, 50, 50, 0.17);\n  }\n\n  .rails-default-error-page h1 {\n
+        \   font-size: 100%;\n    color: #730E15;\n    line-height: 1.5em;\n  }\n\n
+        \ .rails-default-error-page div.dialog > p {\n    margin: 0 0 1em;\n    padding:
+        1em;\n    background-color: #F7F7F7;\n    border: 1px solid #CCC;\n    border-right-color:
+        #999;\n    border-left-color: #999;\n    border-bottom-color: #999;\n    border-bottom-left-radius:
+        4px;\n    border-bottom-right-radius: 4px;\n    border-top-color: #DADADA;\n
+        \   color: #666;\n    box-shadow: 0 3px 8px rgba(50, 50, 50, 0.17);\n  }\n
+        \ </style>\n</head>\n\n<body class=\"rails-default-error-page\">\n  <!-- This
+        file lives in public/404.html -->\n  <div class=\"dialog\">\n    <div>\n      <h1>The
+        page you were looking for doesn't exist.</h1>\n      <p>You may have mistyped
+        the address or the page may have moved.</p>\n    </div>\n    <p>If you are
+        the application owner check the logs for more information.</p>\n  </div>\n</body>\n</html>\n"
+    headers:
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '1722'
+      Content-Type:
+      - text/html; charset=UTF-8
+      Date:
+      - Wed, 06 Aug 2025 19:30:52 GMT
+      strict-transport-security:
+      - max-age=63072000; includeSubDomains
+      x-request-id:
+      - 9edcdee4-f720-431e-9d6d-2dbc1a7bb8fe
+      x-runtime:
+      - '0.005504'
+    status:
+      code: 404
+      message: Not Found
+version: 1

tests/cassettes/TestTraceListenerSetup.test_trace_listener_collects_crew_events.yaml

@@ -0,0 +1,450 @@
+interactions:
+- request:
+    body: '{"messages": [{"role": "system", "content": "You are Test Agent. Test backstory\nYour
+      personal goal is: Test goal\nTo give my best complete final answer to the task
+      respond using the exact following format:\n\nThought: I now can give a great
+      answer\nFinal Answer: Your final answer must be the great and the most complete
+      as possible, it must be outcome described.\n\nI MUST use these formats, my job
+      depends on it!"}, {"role": "user", "content": "\nCurrent Task: Say hello to
+      the world\n\nThis is the expected criteria for your final answer: hello world\nyou
+      MUST return the actual complete content as the final answer, not a summary.\n\nBegin!
+      This is VERY important to you, use the tools available and give your best Final
+      Answer, your job depends on it!\n\nThought:"}], "model": "gpt-4o-mini", "stop":
+      ["\nObservation:"]}'
+    headers:
+      accept:
+      - application/json
+      accept-encoding:
+      - gzip, deflate, zstd
+      connection:
+      - keep-alive
+      content-length:
+      - '825'
+      content-type:
+      - application/json
+      cookie:
+      - __cf_bm=ePO5hy0kEoADCuKcboFy1iS1qckCE5KCpifQaXnlomM-1754508545-1.0.1.1-ieWfjcdIxQIXGfaMizvmgTvZPRFehqDXliegaOT7EO.kt7KSSFGmNDcC35_D9hOhE.fJ5K302uX0snQF3nLaapds2dqgGbNcsyFPOKNvAdI;
+        _cfuvid=NaXWifUGChHp6Ap1mvfMrNzmO4HdzddrqXkSR9T.hYo-1754508545647-0.0.1.1-604800000
+      host:
+      - api.openai.com
+      user-agent:
+      - OpenAI/Python 1.93.0
+      x-stainless-arch:
+      - arm64
+      x-stainless-async:
+      - 'false'
+      x-stainless-lang:
+      - python
+      x-stainless-os:
+      - MacOS
+      x-stainless-package-version:
+      - 1.93.0
+      x-stainless-raw-response:
+      - 'true'
+      x-stainless-read-timeout:
+      - '600.0'
+      x-stainless-retry-count:
+      - '0'
+      x-stainless-runtime:
+      - CPython
+      x-stainless-runtime-version:
+      - 3.12.9
+    method: POST
+    uri: https://api.openai.com/v1/chat/completions
+  response:
+    body:
+      string: !!binary |
+        H4sIAAAAAAAAAwAAAP//jFLBbpwwEL3zFSOfl2oJ0N1ySypV6aG3ntpGaGIGcGI8lm2yjaL998qw
+        Wdi2kXJBYt685/dm5iUBEKoRFQjZY5CD1ennjD6ON/5Hrr4dysxu8/72++NNd/twXT4PYhMZfP9A
+        MryyPkgerKag2MywdISBomq2K4tyuy+L/QQM3JCOtM6GtOB0UEalV9urIt3u0mx/YvesJHlRwc8E
+        AOBl+kafpqHfooLt5rUykPfYkajOTQDCsY4Vgd4rH9AEsVlAySaQmax/BcMHkGigU08ECF20DWj8
+        gRzAL/NFGdRwPf1X0JPWDAd2ulkLOmpHjzGUGbVeAWgMB4xDmaLcnZDj2bzmzjq+939RRauM8n3t
+        CD2baNQHtmJCjwnA3TSk8SK3sI4HG+rAjzQ9l5W7WU8su1mh+QkMHFCv6rvTaC/16oYCKu1XYxYS
+        ZU/NQl12gmOjeAUkq9T/uvmf9pxcme498gsgJdlATW0dNUpeJl7aHMXTfavtPOXJsPDknpSkOihy
+        cRMNtTjq+aCEf/aBhrpVpiNnnZqvqrV1XmBZIH3KpUiOyR8AAAD//wMAErrW9WMDAAA=
+    headers:
+      CF-RAY:
+      - 96b0f0fb5c067ad9-SJC
+      Connection:
+      - keep-alive
+      Content-Encoding:
+      - gzip
+      Content-Type:
+      - application/json
+      Date:
+      - Wed, 06 Aug 2025 19:29:09 GMT
+      Server:
+      - cloudflare
+      Strict-Transport-Security:
+      - max-age=31536000; includeSubDomains; preload
+      Transfer-Encoding:
+      - chunked
+      X-Content-Type-Options:
+      - nosniff
+      access-control-expose-headers:
+      - X-Request-ID
+      alt-svc:
+      - h3=":443"; ma=86400
+      cf-cache-status:
+      - DYNAMIC
+      openai-organization:
+      - crewai-iuxna1
+      openai-processing-ms:
+      - '628'
+      openai-project:
+      - proj_xitITlrFeen7zjNSzML82h9x
+      openai-version:
+      - '2020-10-01'
+      x-envoy-upstream-service-time:
+      - '657'
+      x-ratelimit-limit-project-tokens:
+      - '150000000'
+      x-ratelimit-limit-requests:
+      - '30000'
+      x-ratelimit-limit-tokens:
+      - '150000000'
+      x-ratelimit-remaining-project-tokens:
+      - '149999827'
+      x-ratelimit-remaining-requests:
+      - '29999'
+      x-ratelimit-remaining-tokens:
+      - '149999827'
+      x-ratelimit-reset-project-tokens:
+      - 0s
+      x-ratelimit-reset-requests:
+      - 2ms
+      x-ratelimit-reset-tokens:
+      - 0s
+      x-request-id:
+      - req_a0daca00035c423daf0e9df208720180
+    status:
+      code: 200
+      message: OK
+- request:
+    body: '{"messages": [{"role": "system", "content": "You are Test Agent. Test backstory\nYour
+      personal goal is: Test goal\nTo give my best complete final answer to the task
+      respond using the exact following format:\n\nThought: I now can give a great
+      answer\nFinal Answer: Your final answer must be the great and the most complete
+      as possible, it must be outcome described.\n\nI MUST use these formats, my job
+      depends on it!"}, {"role": "user", "content": "\nCurrent Task: Say hello to
+      the world\n\nThis is the expected criteria for your final answer: hello world\nyou
+      MUST return the actual complete content as the final answer, not a summary.\n\nBegin!
+      This is VERY important to you, use the tools available and give your best Final
+      Answer, your job depends on it!\n\nThought:"}], "model": "gpt-4o-mini"}'
+    headers:
+      accept:
+      - application/json
+      accept-encoding:
+      - gzip, deflate, zstd
+      connection:
+      - keep-alive
+      content-length:
+      - '797'
+      content-type:
+      - application/json
+      cookie:
+      - __cf_bm=f59gEPi_nA3TTxtjbKaSQpvkTwezaAqOvqfxiGzRnVQ-1754508546-1.0.1.1-JrSaytxVIQSVE00I.vyGj7d4HJbbMV6R9fWPJbkDKu0Y8ueMRzTwTUnfz0YzP5nsZX5oxoE6WlmFxOuz0rRuq9YhZZsO_TbaFBOFk1jGK9U;
+        _cfuvid=3D66v3.J_RcVoYy9dlF.jHwq1zTIm842xynZxzSy1Wc-1754508546352-0.0.1.1-604800000
+      host:
+      - api.openai.com
+      user-agent:
+      - OpenAI/Python 1.93.0
+      x-stainless-arch:
+      - arm64
+      x-stainless-async:
+      - 'false'
+      x-stainless-lang:
+      - python
+      x-stainless-os:
+      - MacOS
+      x-stainless-package-version:
+      - 1.93.0
+      x-stainless-raw-response:
+      - 'true'
+      x-stainless-read-timeout:
+      - '200.0'
+      x-stainless-retry-count:
+      - '0'
+      x-stainless-runtime:
+      - CPython
+      x-stainless-runtime-version:
+      - 3.12.9
+    method: POST
+    uri: https://api.openai.com/v1/chat/completions
+  response:
+    body:
+      string: !!binary |
+        H4sIAAAAAAAAAwAAAP//jFLLjtswDLz7Kwid4yLZ2HXiW7FA0R567KXtwmAk2tZWlgSJTlos8u+F
+        nGzs9AH0YsAczmiG5EsGILQSNQjZI8vBm/xxQ2+Pnz89n8Lj7gvvP3BArNb94cClqsQqMdzhmSS/
+        st5IN3hDrJ29wDIQMiXVTVUW5XpXFvsJGJwik2id57xw+aCtzh/WD0W+rvLN7srunZYURQ1fMwCA
+        l+mbfFpFP0QN69VrZaAYsSNR35oARHAmVQTGqCOjZbGaQeksk52sfwTrTiDRQqePBAhdsg1o44kC
+        wDf7Xls08G76r6EnYxycXDBqKRioHSOmUHY0ZgGgtY4xDWWK8nRFzjfzxnU+uEP8jSpabXXsm0AY
+        nU1GIzsvJvScATxNQxrvcgsf3OC5Yfedpuc2ZXXRE/NuFuj2CrJjNIt6dR3tvV6jiFGbuBizkCh7
+        UjN13gmOSrsFkC1S/+nmb9qX5Np2/yM/A1KSZ1KND6S0vE88twVKp/uvttuUJ8MiUjhqSQ1rCmkT
+        iloczeWgRPwZmYam1baj4IO+XFXrm22BZYG030qRnbNfAAAA//8DAOX6h6tjAwAA
+    headers:
+      CF-RAY:
+      - 96b0f101793aeb2c-SJC
+      Connection:
+      - keep-alive
+      Content-Encoding:
+      - gzip
+      Content-Type:
+      - application/json
+      Date:
+      - Wed, 06 Aug 2025 19:29:09 GMT
+      Server:
+      - cloudflare
+      Strict-Transport-Security:
+      - max-age=31536000; includeSubDomains; preload
+      Transfer-Encoding:
+      - chunked
+      X-Content-Type-Options:
+      - nosniff
+      access-control-expose-headers:
+      - X-Request-ID
+      alt-svc:
+      - h3=":443"; ma=86400
+      cf-cache-status:
+      - DYNAMIC
+      openai-organization:
+      - crewai-iuxna1
+      openai-processing-ms:
+      - '541'
+      openai-project:
+      - proj_xitITlrFeen7zjNSzML82h9x
+      openai-version:
+      - '2020-10-01'
+      x-envoy-upstream-service-time:
+      - '557'
+      x-ratelimit-limit-project-tokens:
+      - '150000000'
+      x-ratelimit-limit-requests:
+      - '30000'
+      x-ratelimit-limit-tokens:
+      - '150000000'
+      x-ratelimit-remaining-project-tokens:
+      - '149999827'
+      x-ratelimit-remaining-requests:
+      - '29999'
+      x-ratelimit-remaining-tokens:
+      - '149999827'
+      x-ratelimit-reset-project-tokens:
+      - 0s
+      x-ratelimit-reset-requests:
+      - 2ms
+      x-ratelimit-reset-tokens:
+      - 0s
+      x-request-id:
+      - req_df70f95325b14817a692f23cf9cca880
+    status:
+      code: 200
+      message: OK
+- request:
+    body: '{"trace_id": "2487456d-e03a-4eae-92a1-e9779e8f06a1", "execution_type":
+      "crew", "execution_context": {"crew_fingerprint": null, "crew_name": "Unknown
+      Crew", "flow_name": "Unknown Flow", "crewai_version": "0.152.0", "privacy_level":
+      "standard"}, "execution_metadata": {"expected_duration_estimate": 300, "agent_count":
+      0, "task_count": 0, "flow_method_count": 0, "execution_started_at": "2025-08-06T19:30:52.475039+00:00"}}'
+    headers:
+      Accept:
+      - '*/*'
+      Accept-Encoding:
+      - gzip, deflate, zstd
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '421'
+      Content-Type:
+      - application/json
+      User-Agent:
+      - CrewAI-CLI/0.152.0
+      X-Crewai-Organization-Id:
+      - d3a3d10c-35db-423f-a7a4-c026030ba64d
+      X-Crewai-Version:
+      - 0.152.0
+    method: POST
+    uri: https://app.crewai.com/crewai_plus/api/v1/tracing/batches
+  response:
+    body:
+      string: "<!DOCTYPE html>\n<html>\n<head>\n  <title>The page you were looking
+        for doesn't exist (404)</title>\n  <meta name=\"viewport\" content=\"width=device-width,initial-scale=1\">\n
+        \ <style>\n  .rails-default-error-page {\n    background-color: #EFEFEF;\n
+        \   color: #2E2F30;\n    text-align: center;\n    font-family: arial, sans-serif;\n
+        \   margin: 0;\n  }\n\n  .rails-default-error-page div.dialog {\n    width:
+        95%;\n    max-width: 33em;\n    margin: 4em auto 0;\n  }\n\n  .rails-default-error-page
+        div.dialog > div {\n    border: 1px solid #CCC;\n    border-right-color: #999;\n
+        \   border-left-color: #999;\n    border-bottom-color: #BBB;\n    border-top:
+        #B00100 solid 4px;\n    border-top-left-radius: 9px;\n    border-top-right-radius:
+        9px;\n    background-color: white;\n    padding: 7px 12% 0;\n    box-shadow:
+        0 3px 8px rgba(50, 50, 50, 0.17);\n  }\n\n  .rails-default-error-page h1 {\n
+        \   font-size: 100%;\n    color: #730E15;\n    line-height: 1.5em;\n  }\n\n
+        \ .rails-default-error-page div.dialog > p {\n    margin: 0 0 1em;\n    padding:
+        1em;\n    background-color: #F7F7F7;\n    border: 1px solid #CCC;\n    border-right-color:
+        #999;\n    border-left-color: #999;\n    border-bottom-color: #999;\n    border-bottom-left-radius:
+        4px;\n    border-bottom-right-radius: 4px;\n    border-top-color: #DADADA;\n
+        \   color: #666;\n    box-shadow: 0 3px 8px rgba(50, 50, 50, 0.17);\n  }\n
+        \ </style>\n</head>\n\n<body class=\"rails-default-error-page\">\n  <!-- This
+        file lives in public/404.html -->\n  <div class=\"dialog\">\n    <div>\n      <h1>The
+        page you were looking for doesn't exist.</h1>\n      <p>You may have mistyped
+        the address or the page may have moved.</p>\n    </div>\n    <p>If you are
+        the application owner check the logs for more information.</p>\n  </div>\n</body>\n</html>\n"
+    headers:
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '1722'
+      Content-Type:
+      - text/html; charset=UTF-8
+      Date:
+      - Wed, 06 Aug 2025 19:30:52 GMT
+      strict-transport-security:
+      - max-age=63072000; includeSubDomains
+      x-request-id:
+      - 3640ddcd-56a3-48cc-9a5a-110ebe34ac80
+      x-runtime:
+      - '0.007004'
+    status:
+      code: 404
+      message: Not Found
+- request:
+    body: '{"version": "0.152.0", "batch_id": "2487456d-e03a-4eae-92a1-e9779e8f06a1",
+      "user_context": {"user_id": "anonymous", "organization_id": "", "session_id":
+      "57ab4cf7-915a-4d4e-b01d-3791f418dd39", "trace_id": "c780f111-40df-4c4b-ae88-b09c0f7e3276"},
+      "execution_metadata": {"crew_name": "Unknown Crew", "crewai_version": "0.152.0"},
+      "events": [{"event_id": "55b93497-f7f7-4c2e-baf9-eeec358cf90f", "timestamp":
+      "2025-08-06T19:30:52.588780+00:00", "type": "llm_call_started", "event_data":
+      {"timestamp": "2025-08-06T19:30:52.473897+00:00", "type": "llm_call_started",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "task_name": null, "task_id": "020f12f8-4bcf-4e49-9bf9-d8fee70eaf6a", "agent_id":
+      "126a2971-a630-405c-a0d9-72d2c46e8074", "agent_role": "Test Agent", "model":
+      "gpt-4o-mini", "messages": [{"role": "system", "content": "You are Test Agent.
+      Test backstory\nYour personal goal is: Test goal\nTo give my best complete final
+      answer to the task respond using the exact following format:\n\nThought: I now
+      can give a great answer\n..."}, {"role": "user", "content": "\nCurrent Task:
+      Say hello to the world\n\nThis is the expected criteria for your final answer:
+      hello world\nyou MUST return the actual complete content as the final answer,
+      not a summary.\n\nBegin! This is ..."}], "tools": null, "callbacks": ["<crewai.utilities.token_counter_callback.TokenCalcHandler
+      object at 0x11c5860f0>"], "available_functions": null}}, {"event_id": "967e03f5-fc65-477f-a1ba-4614acbd0527",
+      "timestamp": "2025-08-06T19:30:52.588932+00:00", "type": "llm_call_started",
+      "event_data": {"timestamp": "2025-08-06T19:30:52.473897+00:00", "type": "llm_call_started",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "task_name": null, "task_id": "020f12f8-4bcf-4e49-9bf9-d8fee70eaf6a", "agent_id":
+      "126a2971-a630-405c-a0d9-72d2c46e8074", "agent_role": "Test Agent", "model":
+      "gpt-4o-mini", "messages": [{"role": "system", "content": "You are Test Agent.
+      Test backstory\nYour personal goal is: Test goal\nTo give my best complete final
+      answer to the task respond using the exact following format:\n\nThought: I now
+      can give a great answer\n..."}, {"role": "user", "content": "\nCurrent Task:
+      Say hello to the world\n\nThis is the expected criteria for your final answer:
+      hello world\nyou MUST return the actual complete content as the final answer,
+      not a summary.\n\nBegin! This is ..."}], "tools": null, "callbacks": ["<crewai.utilities.token_counter_callback.TokenCalcHandler
+      object at 0x11c5860f0>"], "available_functions": null}}, {"event_id": "f67f33c0-f9ac-450e-987e-bb48880b9b69",
+      "timestamp": "2025-08-06T19:30:52.597813+00:00", "type": "llm_call_completed",
+      "event_data": {"timestamp": "2025-08-06T19:30:52.597748+00:00", "type": "llm_call_completed",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "task_name": null, "task_id": "020f12f8-4bcf-4e49-9bf9-d8fee70eaf6a", "agent_id":
+      "126a2971-a630-405c-a0d9-72d2c46e8074", "agent_role": "Test Agent", "messages":
+      [{"role": "system", "content": "You are Test Agent. Test backstory\nYour personal
+      goal is: Test goal\nTo give my best complete final answer to the task respond
+      using the exact following format:\n\nThought: I now can give a great answer\n..."},
+      {"role": "user", "content": "\nCurrent Task: Say hello to the world\n\nThis
+      is the expected criteria for your final answer: hello world\nyou MUST return
+      the actual complete content as the final answer, not a summary.\n\nBegin! This
+      is ..."}], "response": "I now can give a great answer  \nFinal Answer: hello
+      world", "call_type": "<LLMCallType.LLM_CALL: ''llm_call''>", "response_cost":
+      3.135e-05, "model": "gpt-4o-mini"}}, {"event_id": "0868106f-9491-4214-9674-4f9c5621875b",
+      "timestamp": "2025-08-06T19:30:52.597885+00:00", "type": "llm_call_completed",
+      "event_data": {"timestamp": "2025-08-06T19:30:52.597748+00:00", "type": "llm_call_completed",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "task_name": null, "task_id": "020f12f8-4bcf-4e49-9bf9-d8fee70eaf6a", "agent_id":
+      "126a2971-a630-405c-a0d9-72d2c46e8074", "agent_role": "Test Agent", "messages":
+      [{"role": "system", "content": "You are Test Agent. Test backstory\nYour personal
+      goal is: Test goal\nTo give my best complete final answer to the task respond
+      using the exact following format:\n\nThought: I now can give a great answer\n..."},
+      {"role": "user", "content": "\nCurrent Task: Say hello to the world\n\nThis
+      is the expected criteria for your final answer: hello world\nyou MUST return
+      the actual complete content as the final answer, not a summary.\n\nBegin! This
+      is ..."}], "response": "I now can give a great answer  \nFinal Answer: hello
+      world", "call_type": "<LLMCallType.LLM_CALL: ''llm_call''>", "response_cost":
+      3.135e-05, "model": "gpt-4o-mini"}}, {"event_id": "4b68fa53-1829-4201-bfa2-9364dbd8fc51",
+      "timestamp": "2025-08-06T19:30:52.598054+00:00", "type": "agent_execution_completed",
+      "event_data": {"serialization_error": "Circular reference detected (id repeated)",
+      "object_type": "AgentExecutionCompletedEvent"}}, {"event_id": "264a2a9a-c234-45f2-8021-5e4b1b7c4c98",
+      "timestamp": "2025-08-06T19:30:52.598224+00:00", "type": "agent_execution_completed",
+      "event_data": {"serialization_error": "Circular reference detected (id repeated)",
+      "object_type": "AgentExecutionCompletedEvent"}}, {"event_id": "b8b7f244-e2c0-4cac-9411-0efab7b9936a",
+      "timestamp": "2025-08-06T19:30:52.598372+00:00", "type": "task_completed", "event_data":
+      {"serialization_error": "Circular reference detected (id repeated)", "object_type":
+      "TaskCompletedEvent"}}, {"event_id": "c15489a3-31af-48bb-8f32-cb3c3f46f7b9",
+      "timestamp": "2025-08-06T19:30:52.598416+00:00", "type": "task_completed", "event_data":
+      {"serialization_error": "Circular reference detected (id repeated)", "object_type":
+      "TaskCompletedEvent"}}, {"event_id": "6806c6bd-1399-4261-aa23-5b6166c2ab31",
+      "timestamp": "2025-08-06T19:30:52.601018+00:00", "type": "crew_kickoff_completed",
+      "event_data": {"timestamp": "2025-08-06T19:30:52.600994+00:00", "type": "crew_kickoff_completed",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "crew_name": "crew", "crew": null, "output": {"description": "Say hello to the
+      world", "name": null, "expected_output": "hello world", "summary": "Say hello
+      to the world...", "raw": "hello world", "pydantic": null, "json_dict": null,
+      "agent": "Test Agent", "output_format": "raw"}, "total_tokens": 170}}]}'
+    headers:
+      Accept:
+      - '*/*'
+      Accept-Encoding:
+      - gzip, deflate, zstd
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '6503'
+      Content-Type:
+      - application/json
+      User-Agent:
+      - CrewAI-CLI/0.152.0
+      X-Crewai-Organization-Id:
+      - d3a3d10c-35db-423f-a7a4-c026030ba64d
+      X-Crewai-Version:
+      - 0.152.0
+    method: POST
+    uri: https://app.crewai.com/crewai_plus/api/v1/tracing
+  response:
+    body:
+      string: "<!DOCTYPE html>\n<html>\n<head>\n  <title>The page you were looking
+        for doesn't exist (404)</title>\n  <meta name=\"viewport\" content=\"width=device-width,initial-scale=1\">\n
+        \ <style>\n  .rails-default-error-page {\n    background-color: #EFEFEF;\n
+        \   color: #2E2F30;\n    text-align: center;\n    font-family: arial, sans-serif;\n
+        \   margin: 0;\n  }\n\n  .rails-default-error-page div.dialog {\n    width:
+        95%;\n    max-width: 33em;\n    margin: 4em auto 0;\n  }\n\n  .rails-default-error-page
+        div.dialog > div {\n    border: 1px solid #CCC;\n    border-right-color: #999;\n
+        \   border-left-color: #999;\n    border-bottom-color: #BBB;\n    border-top:
+        #B00100 solid 4px;\n    border-top-left-radius: 9px;\n    border-top-right-radius:
+        9px;\n    background-color: white;\n    padding: 7px 12% 0;\n    box-shadow:
+        0 3px 8px rgba(50, 50, 50, 0.17);\n  }\n\n  .rails-default-error-page h1 {\n
+        \   font-size: 100%;\n    color: #730E15;\n    line-height: 1.5em;\n  }\n\n
+        \ .rails-default-error-page div.dialog > p {\n    margin: 0 0 1em;\n    padding:
+        1em;\n    background-color: #F7F7F7;\n    border: 1px solid #CCC;\n    border-right-color:
+        #999;\n    border-left-color: #999;\n    border-bottom-color: #999;\n    border-bottom-left-radius:
+        4px;\n    border-bottom-right-radius: 4px;\n    border-top-color: #DADADA;\n
+        \   color: #666;\n    box-shadow: 0 3px 8px rgba(50, 50, 50, 0.17);\n  }\n
+        \ </style>\n</head>\n\n<body class=\"rails-default-error-page\">\n  <!-- This
+        file lives in public/404.html -->\n  <div class=\"dialog\">\n    <div>\n      <h1>The
+        page you were looking for doesn't exist.</h1>\n      <p>You may have mistyped
+        the address or the page may have moved.</p>\n    </div>\n    <p>If you are
+        the application owner check the logs for more information.</p>\n  </div>\n</body>\n</html>\n"
+    headers:
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '1722'
+      Content-Type:
+      - text/html; charset=UTF-8
+      Date:
+      - Wed, 06 Aug 2025 19:30:52 GMT
+      strict-transport-security:
+      - max-age=63072000; includeSubDomains
+      x-request-id:
+      - 50885aa8-a8e2-4a5a-87f7-6d0e2f2f80f9
+      x-runtime:
+      - '0.006464'
+    status:
+      code: 404
+      message: Not Found
+version: 1

tests/cassettes/TestTraceListenerSetup.test_trace_listener_disabled_when_env_false.yaml

@@ -0,0 +1,433 @@
+interactions:
+- request:
+    body: '{"messages": [{"role": "system", "content": "You are Test Agent. Test backstory\nYour
+      personal goal is: Test goal\nTo give my best complete final answer to the task
+      respond using the exact following format:\n\nThought: I now can give a great
+      answer\nFinal Answer: Your final answer must be the great and the most complete
+      as possible, it must be outcome described.\n\nI MUST use these formats, my job
+      depends on it!"}, {"role": "user", "content": "\nCurrent Task: Say hello to
+      the world\n\nThis is the expected criteria for your final answer: hello world\nyou
+      MUST return the actual complete content as the final answer, not a summary.\n\nBegin!
+      This is VERY important to you, use the tools available and give your best Final
+      Answer, your job depends on it!\n\nThought:"}], "model": "gpt-4o-mini", "stop":
+      ["\nObservation:"]}'
+    headers:
+      accept:
+      - application/json
+      accept-encoding:
+      - gzip, deflate, zstd
+      connection:
+      - keep-alive
+      content-length:
+      - '825'
+      content-type:
+      - application/json
+      host:
+      - api.openai.com
+      user-agent:
+      - OpenAI/Python 1.93.0
+      x-stainless-arch:
+      - arm64
+      x-stainless-async:
+      - 'false'
+      x-stainless-lang:
+      - python
+      x-stainless-os:
+      - MacOS
+      x-stainless-package-version:
+      - 1.93.0
+      x-stainless-raw-response:
+      - 'true'
+      x-stainless-read-timeout:
+      - '600.0'
+      x-stainless-retry-count:
+      - '0'
+      x-stainless-runtime:
+      - CPython
+      x-stainless-runtime-version:
+      - 3.12.9
+    method: POST
+    uri: https://api.openai.com/v1/chat/completions
+  response:
+    body:
+      string: !!binary |
+        H4sIAAAAAAAAAwAAAP//jFJdb9QwEHzPr1j8nKDkmvTavBUQnw8goQoJqKKts8kZHK9lOy1Q3X9H
+        Tq6XFIrES6Ts7IxndvcuARCqFTUIucMgB6uz5wWduvz8g3nx/tnny3d4MRabX6flW9O9qj6KNDL4
+        +hvJcM96KnmwmoJiM8PSEQaKqsW2Kqv8rCqrCRi4JR1pvQ1ZydmgjMo2+abM8m1WnB3YO1aSvKjh
+        SwIAcDd9o0/T0g9RQ57eVwbyHnsS9bEJQDjWsSLQe+UDmiDSBZRsApnJ+hswfAsSDfTqhgChj7YB
+        jb8lB/DVvFQGNVxM/zW8Jq05hU/sdPtkLemoGz3GWGbUegWgMRwwjmUKc3VA9kf7mnvr+Nr/QRWd
+        MsrvGkfo2USrPrAVE7pPAK6mMY0PkgvreLChCfydpueKajvriWU7a/QABg6oV/XtJn1Er2kpoNJ+
+        NWghUe6oXajLVnBsFa+AZJX6bzePac/Jlen/R34BpCQbqG2so1bJh4mXNkfxeP/VdpzyZFh4cjdK
+        UhMUubiJljoc9XxSwv/0gYamU6YnZ52a76qzzUmJVYl0fiJFsk9+AwAA//8DABLzfQllAwAA
+    headers:
+      CF-RAY:
+      - 96b0f0e62d177ad9-SJC
+      Connection:
+      - keep-alive
+      Content-Encoding:
+      - gzip
+      Content-Type:
+      - application/json
+      Date:
+      - Wed, 06 Aug 2025 19:29:05 GMT
+      Server:
+      - cloudflare
+      Set-Cookie:
+      - __cf_bm=ePO5hy0kEoADCuKcboFy1iS1qckCE5KCpifQaXnlomM-1754508545-1.0.1.1-ieWfjcdIxQIXGfaMizvmgTvZPRFehqDXliegaOT7EO.kt7KSSFGmNDcC35_D9hOhE.fJ5K302uX0snQF3nLaapds2dqgGbNcsyFPOKNvAdI;
+        path=/; expires=Wed, 06-Aug-25 19:59:05 GMT; domain=.api.openai.com; HttpOnly;
+        Secure; SameSite=None
+      - _cfuvid=NaXWifUGChHp6Ap1mvfMrNzmO4HdzddrqXkSR9T.hYo-1754508545647-0.0.1.1-604800000;
+        path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None
+      Strict-Transport-Security:
+      - max-age=31536000; includeSubDomains; preload
+      Transfer-Encoding:
+      - chunked
+      X-Content-Type-Options:
+      - nosniff
+      access-control-expose-headers:
+      - X-Request-ID
+      alt-svc:
+      - h3=":443"; ma=86400
+      cf-cache-status:
+      - DYNAMIC
+      openai-organization:
+      - crewai-iuxna1
+      openai-processing-ms:
+      - '526'
+      openai-project:
+      - proj_xitITlrFeen7zjNSzML82h9x
+      openai-version:
+      - '2020-10-01'
+      x-envoy-upstream-service-time:
+      - '568'
+      x-ratelimit-limit-project-tokens:
+      - '150000000'
+      x-ratelimit-limit-requests:
+      - '30000'
+      x-ratelimit-limit-tokens:
+      - '150000000'
+      x-ratelimit-remaining-project-tokens:
+      - '149999827'
+      x-ratelimit-remaining-requests:
+      - '29999'
+      x-ratelimit-remaining-tokens:
+      - '149999827'
+      x-ratelimit-reset-project-tokens:
+      - 0s
+      x-ratelimit-reset-requests:
+      - 2ms
+      x-ratelimit-reset-tokens:
+      - 0s
+      x-request-id:
+      - req_0e70b38c85e144d289fbdf89082cf16e
+    status:
+      code: 200
+      message: OK
+- request:
+    body: '{"messages": [{"role": "system", "content": "You are Test Agent. Test backstory\nYour
+      personal goal is: Test goal\nTo give my best complete final answer to the task
+      respond using the exact following format:\n\nThought: I now can give a great
+      answer\nFinal Answer: Your final answer must be the great and the most complete
+      as possible, it must be outcome described.\n\nI MUST use these formats, my job
+      depends on it!"}, {"role": "user", "content": "\nCurrent Task: Say hello to
+      the world\n\nThis is the expected criteria for your final answer: hello world\nyou
+      MUST return the actual complete content as the final answer, not a summary.\n\nBegin!
+      This is VERY important to you, use the tools available and give your best Final
+      Answer, your job depends on it!\n\nThought:"}], "model": "gpt-4o-mini"}'
+    headers:
+      accept:
+      - application/json
+      accept-encoding:
+      - gzip, deflate, zstd
+      connection:
+      - keep-alive
+      content-length:
+      - '797'
+      content-type:
+      - application/json
+      host:
+      - api.openai.com
+      user-agent:
+      - OpenAI/Python 1.93.0
+      x-stainless-arch:
+      - arm64
+      x-stainless-async:
+      - 'false'
+      x-stainless-lang:
+      - python
+      x-stainless-os:
+      - MacOS
+      x-stainless-package-version:
+      - 1.93.0
+      x-stainless-raw-response:
+      - 'true'
+      x-stainless-read-timeout:
+      - '200.0'
+      x-stainless-retry-count:
+      - '0'
+      x-stainless-runtime:
+      - CPython
+      x-stainless-runtime-version:
+      - 3.12.9
+    method: POST
+    uri: https://api.openai.com/v1/chat/completions
+  response:
+    body:
+      string: !!binary |
+        H4sIAAAAAAAAAwAAAP//jFJdj9MwEHzPr1j8nKC0TejRN4TE3YmTQIAEEpyiPWeTujheYztX0Kn/
+        HTnpNbkPJF4iZWdnPLO7dwmAULXYgJBbDLKzOnu7oFdu//nLr8v8Ij//1jdXq93Xjx8+7a7OzXuR
+        Rgbf7EiGe9ZLyZ3VFBSbEZaOMFBUXazLoszPyqIcgI5r0pHW2pAVnHXKqGyZL4ssX2eLsyN7y0qS
+        Fxv4ngAA3A3f6NPU9FtsIE/vKx15jy2JzakJQDjWsSLQe+UDmiDSCZRsApnB+iUY3oNEA626JUBo
+        o21A4/fkAH6Yd8qghjfD/wYuSGtOYc9O1y/mko6a3mOMZXqtZwAawwHjWIYw10fkcLKvubWOb/wj
+        qmiUUX5bOULPJlr1ga0Y0EMCcD2MqX+QXFjHnQ1V4J80PLco16OemLYzR49g4IB6Vl8v02f0qpoC
+        Ku1ngxYS5ZbqiTptBfta8QxIZqmfunlOe0yuTPs/8hMgJdlAdWUd1Uo+TDy1OYrH+6+205QHw8KT
+        u1WSqqDIxU3U1GCvx5MS/o8P1FWNMi0569R4V42tVgWWBdLrlRTJIfkLAAD//wMAE4F9LmUDAAA=
+    headers:
+      CF-RAY:
+      - 96b0f0eadf69eb2c-SJC
+      Connection:
+      - keep-alive
+      Content-Encoding:
+      - gzip
+      Content-Type:
+      - application/json
+      Date:
+      - Wed, 06 Aug 2025 19:29:06 GMT
+      Server:
+      - cloudflare
+      Set-Cookie:
+      - __cf_bm=f59gEPi_nA3TTxtjbKaSQpvkTwezaAqOvqfxiGzRnVQ-1754508546-1.0.1.1-JrSaytxVIQSVE00I.vyGj7d4HJbbMV6R9fWPJbkDKu0Y8ueMRzTwTUnfz0YzP5nsZX5oxoE6WlmFxOuz0rRuq9YhZZsO_TbaFBOFk1jGK9U;
+        path=/; expires=Wed, 06-Aug-25 19:59:06 GMT; domain=.api.openai.com; HttpOnly;
+        Secure; SameSite=None
+      - _cfuvid=3D66v3.J_RcVoYy9dlF.jHwq1zTIm842xynZxzSy1Wc-1754508546352-0.0.1.1-604800000;
+        path=/; domain=.api.openai.com; HttpOnly; Secure; SameSite=None
+      Strict-Transport-Security:
+      - max-age=31536000; includeSubDomains; preload
+      Transfer-Encoding:
+      - chunked
+      X-Content-Type-Options:
+      - nosniff
+      access-control-expose-headers:
+      - X-Request-ID
+      alt-svc:
+      - h3=":443"; ma=86400
+      cf-cache-status:
+      - DYNAMIC
+      openai-organization:
+      - crewai-iuxna1
+      openai-processing-ms:
+      - '504'
+      openai-project:
+      - proj_xitITlrFeen7zjNSzML82h9x
+      openai-version:
+      - '2020-10-01'
+      x-envoy-upstream-service-time:
+      - '527'
+      x-ratelimit-limit-project-tokens:
+      - '150000000'
+      x-ratelimit-limit-requests:
+      - '30000'
+      x-ratelimit-limit-tokens:
+      - '150000000'
+      x-ratelimit-remaining-project-tokens:
+      - '149999827'
+      x-ratelimit-remaining-requests:
+      - '29999'
+      x-ratelimit-remaining-tokens:
+      - '149999830'
+      x-ratelimit-reset-project-tokens:
+      - 0s
+      x-ratelimit-reset-requests:
+      - 2ms
+      x-ratelimit-reset-tokens:
+      - 0s
+      x-request-id:
+      - req_32abf5c6f27e42579bc84b0bfcc9c4b4
+    status:
+      code: 200
+      message: OK
+- request:
+    body: '{"trace_id": "e3677f76-4763-4f55-94b2-f38707f353c3", "execution_type":
+      "crew", "execution_context": {"crew_fingerprint": null, "crew_name": "crew",
+      "flow_name": "Unknown Flow", "crewai_version": "0.152.0", "privacy_level": "standard"},
+      "execution_metadata": {"expected_duration_estimate": 300, "agent_count": 0,
+      "task_count": 0, "flow_method_count": 0, "execution_started_at": "2025-08-06T19:30:52.778875+00:00"}}'
+    headers:
+      Accept:
+      - '*/*'
+      Accept-Encoding:
+      - gzip, deflate, zstd
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '413'
+      Content-Type:
+      - application/json
+      User-Agent:
+      - CrewAI-CLI/0.152.0
+      X-Crewai-Organization-Id:
+      - d3a3d10c-35db-423f-a7a4-c026030ba64d
+      X-Crewai-Version:
+      - 0.152.0
+    method: POST
+    uri: https://app.crewai.com/crewai_plus/api/v1/tracing/batches
+  response:
+    body:
+      string: "<!DOCTYPE html>\n<html>\n<head>\n  <title>The page you were looking
+        for doesn't exist (404)</title>\n  <meta name=\"viewport\" content=\"width=device-width,initial-scale=1\">\n
+        \ <style>\n  .rails-default-error-page {\n    background-color: #EFEFEF;\n
+        \   color: #2E2F30;\n    text-align: center;\n    font-family: arial, sans-serif;\n
+        \   margin: 0;\n  }\n\n  .rails-default-error-page div.dialog {\n    width:
+        95%;\n    max-width: 33em;\n    margin: 4em auto 0;\n  }\n\n  .rails-default-error-page
+        div.dialog > div {\n    border: 1px solid #CCC;\n    border-right-color: #999;\n
+        \   border-left-color: #999;\n    border-bottom-color: #BBB;\n    border-top:
+        #B00100 solid 4px;\n    border-top-left-radius: 9px;\n    border-top-right-radius:
+        9px;\n    background-color: white;\n    padding: 7px 12% 0;\n    box-shadow:
+        0 3px 8px rgba(50, 50, 50, 0.17);\n  }\n\n  .rails-default-error-page h1 {\n
+        \   font-size: 100%;\n    color: #730E15;\n    line-height: 1.5em;\n  }\n\n
+        \ .rails-default-error-page div.dialog > p {\n    margin: 0 0 1em;\n    padding:
+        1em;\n    background-color: #F7F7F7;\n    border: 1px solid #CCC;\n    border-right-color:
+        #999;\n    border-left-color: #999;\n    border-bottom-color: #999;\n    border-bottom-left-radius:
+        4px;\n    border-bottom-right-radius: 4px;\n    border-top-color: #DADADA;\n
+        \   color: #666;\n    box-shadow: 0 3px 8px rgba(50, 50, 50, 0.17);\n  }\n
+        \ </style>\n</head>\n\n<body class=\"rails-default-error-page\">\n  <!-- This
+        file lives in public/404.html -->\n  <div class=\"dialog\">\n    <div>\n      <h1>The
+        page you were looking for doesn't exist.</h1>\n      <p>You may have mistyped
+        the address or the page may have moved.</p>\n    </div>\n    <p>If you are
+        the application owner check the logs for more information.</p>\n  </div>\n</body>\n</html>\n"
+    headers:
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '1722'
+      Content-Type:
+      - text/html; charset=UTF-8
+      Date:
+      - Wed, 06 Aug 2025 19:30:52 GMT
+      strict-transport-security:
+      - max-age=63072000; includeSubDomains
+      x-request-id:
+      - 8c9c1556-d30f-4736-b62c-5a41150e859f
+      x-runtime:
+      - '0.005329'
+    status:
+      code: 404
+      message: Not Found
+- request:
+    body: '{"version": "0.152.0", "batch_id": "e3677f76-4763-4f55-94b2-f38707f353c3",
+      "user_context": {"user_id": "anonymous", "organization_id": "", "session_id":
+      "eb96086e-c3b3-4757-a118-328be61c9aad", "trace_id": "90245ff6-bd46-4e0e-83da-b12edd241b0e"},
+      "execution_metadata": {"crew_name": "crew", "execution_start": "2025-08-06T19:30:52.777333+00:00",
+      "crewai_version": "0.152.0"}, "events": [{"event_id": "d5c81b9a-b8a9-4638-ab50-aa91792b95c8",
+      "timestamp": "2025-08-06T19:30:52.909777+00:00", "type": "crew_kickoff_started",
+      "event_data": {"timestamp": "2025-08-06T19:30:52.777333+00:00", "type": "crew_kickoff_started",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "crew_name": "crew", "crew": null, "inputs": null}}, {"event_id": "a5bd314d-f9eb-471f-b3c4-e176e6ec62f9",
+      "timestamp": "2025-08-06T19:30:52.911914+00:00", "type": "task_started", "event_data":
+      {"task_description": "Say hello to the world", "task_name": null, "context":
+      "", "agent": "Test Agent"}}, {"event_id": "ce0e41d9-90a9-4585-8dc3-c04ee02232bc",
+      "timestamp": "2025-08-06T19:30:52.912403+00:00", "type": "agent_execution_started",
+      "event_data": {"serialization_error": "Circular reference detected (id repeated)",
+      "object_type": "AgentExecutionStartedEvent"}}, {"event_id": "d7c3546e-fe60-4c8a-9e4a-a510fa631a8b",
+      "timestamp": "2025-08-06T19:30:52.912693+00:00", "type": "llm_call_started",
+      "event_data": {"timestamp": "2025-08-06T19:30:52.912657+00:00", "type": "llm_call_started",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "task_name": null, "task_id": "e4abe414-b25d-44ea-8a0d-4998d7e55ed3", "agent_id":
+      "91a39492-d0c8-4994-b8b4-acdd256a2e96", "agent_role": "Test Agent", "model":
+      "gpt-4o-mini", "messages": [{"role": "system", "content": "You are Test Agent.
+      Test backstory\nYour personal goal is: Test goal\nTo give my best complete final
+      answer to the task respond using the exact following format:\n\nThought: I now
+      can give a great answer\n..."}, {"role": "user", "content": "\nCurrent Task:
+      Say hello to the world\n\nThis is the expected criteria for your final answer:
+      hello world\nyou MUST return the actual complete content as the final answer,
+      not a summary.\n\nBegin! This is ..."}], "tools": null, "callbacks": ["<crewai.utilities.token_counter_callback.TokenCalcHandler
+      object at 0x11c5b5e50>"], "available_functions": null}}, {"event_id": "6ed9e994-3e66-4653-97e0-a9e8e8c1d978",
+      "timestamp": "2025-08-06T19:30:52.919664+00:00", "type": "llm_call_completed",
+      "event_data": {"timestamp": "2025-08-06T19:30:52.919623+00:00", "type": "llm_call_completed",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "task_name": null, "task_id": "e4abe414-b25d-44ea-8a0d-4998d7e55ed3", "agent_id":
+      "91a39492-d0c8-4994-b8b4-acdd256a2e96", "agent_role": "Test Agent", "messages":
+      [{"role": "system", "content": "You are Test Agent. Test backstory\nYour personal
+      goal is: Test goal\nTo give my best complete final answer to the task respond
+      using the exact following format:\n\nThought: I now can give a great answer\n..."},
+      {"role": "user", "content": "\nCurrent Task: Say hello to the world\n\nThis
+      is the expected criteria for your final answer: hello world\nyou MUST return
+      the actual complete content as the final answer, not a summary.\n\nBegin! This
+      is ..."}], "response": "I now can give a great answer  \nFinal Answer: Hello,
+      World!", "call_type": "<LLMCallType.LLM_CALL: ''llm_call''>", "response_cost":
+      3.255e-05, "model": "gpt-4o-mini"}}, {"event_id": "2f03d7fe-2faf-4d6b-a9e1-d3cb9e87ef10",
+      "timestamp": "2025-08-06T19:30:52.919798+00:00", "type": "agent_execution_completed",
+      "event_data": {"serialization_error": "Circular reference detected (id repeated)",
+      "object_type": "AgentExecutionCompletedEvent"}}, {"event_id": "c2ed7fa8-0361-406f-8a0f-4cf0f580dbee",
+      "timestamp": "2025-08-06T19:30:52.919953+00:00", "type": "task_completed", "event_data":
+      {"serialization_error": "Circular reference detected (id repeated)", "object_type":
+      "TaskCompletedEvent"}}, {"event_id": "727d1ea2-4f7b-4d12-b491-42a27f3c3123",
+      "timestamp": "2025-08-06T19:30:52.921547+00:00", "type": "crew_kickoff_completed",
+      "event_data": {"timestamp": "2025-08-06T19:30:52.921522+00:00", "type": "crew_kickoff_completed",
+      "source_fingerprint": null, "source_type": null, "fingerprint_metadata": null,
+      "crew_name": "crew", "crew": null, "output": {"description": "Say hello to the
+      world", "name": null, "expected_output": "hello world", "summary": "Say hello
+      to the world...", "raw": "Hello, World!", "pydantic": null, "json_dict": null,
+      "agent": "Test Agent", "output_format": "raw"}, "total_tokens": 172}}]}'
+    headers:
+      Accept:
+      - '*/*'
+      Accept-Encoding:
+      - gzip, deflate, zstd
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '4656'
+      Content-Type:
+      - application/json
+      User-Agent:
+      - CrewAI-CLI/0.152.0
+      X-Crewai-Organization-Id:
+      - d3a3d10c-35db-423f-a7a4-c026030ba64d
+      X-Crewai-Version:
+      - 0.152.0
+    method: POST
+    uri: https://app.crewai.com/crewai_plus/api/v1/tracing
+  response:
+    body:
+      string: "<!DOCTYPE html>\n<html>\n<head>\n  <title>The page you were looking
+        for doesn't exist (404)</title>\n  <meta name=\"viewport\" content=\"width=device-width,initial-scale=1\">\n
+        \ <style>\n  .rails-default-error-page {\n    background-color: #EFEFEF;\n
+        \   color: #2E2F30;\n    text-align: center;\n    font-family: arial, sans-serif;\n
+        \   margin: 0;\n  }\n\n  .rails-default-error-page div.dialog {\n    width:
+        95%;\n    max-width: 33em;\n    margin: 4em auto 0;\n  }\n\n  .rails-default-error-page
+        div.dialog > div {\n    border: 1px solid #CCC;\n    border-right-color: #999;\n
+        \   border-left-color: #999;\n    border-bottom-color: #BBB;\n    border-top:
+        #B00100 solid 4px;\n    border-top-left-radius: 9px;\n    border-top-right-radius:
+        9px;\n    background-color: white;\n    padding: 7px 12% 0;\n    box-shadow:
+        0 3px 8px rgba(50, 50, 50, 0.17);\n  }\n\n  .rails-default-error-page h1 {\n
+        \   font-size: 100%;\n    color: #730E15;\n    line-height: 1.5em;\n  }\n\n
+        \ .rails-default-error-page div.dialog > p {\n    margin: 0 0 1em;\n    padding:
+        1em;\n    background-color: #F7F7F7;\n    border: 1px solid #CCC;\n    border-right-color:
+        #999;\n    border-left-color: #999;\n    border-bottom-color: #999;\n    border-bottom-left-radius:
+        4px;\n    border-bottom-right-radius: 4px;\n    border-top-color: #DADADA;\n
+        \   color: #666;\n    box-shadow: 0 3px 8px rgba(50, 50, 50, 0.17);\n  }\n
+        \ </style>\n</head>\n\n<body class=\"rails-default-error-page\">\n  <!-- This
+        file lives in public/404.html -->\n  <div class=\"dialog\">\n    <div>\n      <h1>The
+        page you were looking for doesn't exist.</h1>\n      <p>You may have mistyped
+        the address or the page may have moved.</p>\n    </div>\n    <p>If you are
+        the application owner check the logs for more information.</p>\n  </div>\n</body>\n</html>\n"
+    headers:
+      Connection:
+      - keep-alive
+      Content-Length:
+      - '1722'
+      Content-Type:
+      - text/html; charset=UTF-8
+      Date:
+      - Wed, 06 Aug 2025 19:30:53 GMT
+      strict-transport-security:
+      - max-age=63072000; includeSubDomains
+      x-request-id:
+      - 3b16d4bb-ba79-4a32-a776-26bbdf8d0a68
+      x-runtime:
+      - '0.005566'
+    status:
+      code: 404
+      message: Not Found
+version: 1

tests/cassettes/test_gemma3[gemini-gemma-3-12b-it].yaml

@@ -1,59 +0,0 @@
-interactions:
-- request:
-    body: '{"contents": [{"role": "user", "parts": [{"text": "What is the capital
-      of France?"}]}], "generationConfig": {"stop_sequences": []}}'
-    headers:
-      accept:
-      - '*/*'
-      accept-encoding:
-      - gzip, deflate
-      connection:
-      - keep-alive
-      content-length:
-      - '131'
-      content-type:
-      - application/json
-      host:
-      - generativelanguage.googleapis.com
-      user-agent:
-      - litellm/1.60.2
-    method: POST
-    uri: https://generativelanguage.googleapis.com/v1beta/models/gemma-3-12b-it:generateContent
-  response:
-    body:
-      string: !!binary |
-        H4sIAAAAAAAC/2WRTWvDMAyG7/kVwpdBSEvX7jB23QfsMFa2MAZbD2qipGaOFWwFWkr/+5ykaVPq
-        gGP0SvLrR/sIQGVoc52jkFcP8BMiAPtubzW2QlaCMIRCsEYn59x+7UfnkCK0bYtUuiHIsNaCBriA
-        F4c2I9Ae4niJTvs4nv7a/nuVGw9oPIOEIoOuJC+QadmBtkNlsAoIpeF1aJgFZ+SgYAfBUQIF+o1m
-        m0CJXhxbrnZJV5E1RhpHUzUyeTidV8n5aY4Ntb4rzskM6YchQRXaar/5IPRs27TP9H2pTqq2OW1D
-        eBYNF3StVeOxpDcSDJDxhFLVjqtaUv4j+8hNB/m+7zUayYW8mB914QD0QrqbJVdd/VO4U5vxqEZT
-        DE9EE+h2Y3r+TtUIg1yYGjB0/1V0BNIz+iLndQ+jpKrCyWJyO19PtKjoEP0DlZtdIF8CAAA=
-    headers:
-      Alt-Svc:
-      - h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
-      Content-Encoding:
-      - gzip
-      Content-Type:
-      - application/json; charset=UTF-8
-      Date:
-      - Tue, 22 Apr 2025 14:25:39 GMT
-      Server:
-      - scaffolding on HTTPServer2
-      Server-Timing:
-      - gfet4t7; dur=3835
-      Transfer-Encoding:
-      - chunked
-      Vary:
-      - Origin
-      - X-Origin
-      - Referer
-      X-Content-Type-Options:
-      - nosniff
-      X-Frame-Options:
-      - SAMEORIGIN
-      X-XSS-Protection:
-      - '0'
-    status:
-      code: 200
-      message: OK
-version: 1

tests/cassettes/test_gemma3[gemini-gemma-3-1b-it].yaml

@@ -1,60 +0,0 @@
-interactions:
-- request:
-    body: '{"contents": [{"role": "user", "parts": [{"text": "What is the capital
-      of France?"}]}], "generationConfig": {"stop_sequences": []}}'
-    headers:
-      accept:
-      - '*/*'
-      accept-encoding:
-      - gzip, deflate
-      connection:
-      - keep-alive
-      content-length:
-      - '131'
-      content-type:
-      - application/json
-      host:
-      - generativelanguage.googleapis.com
-      user-agent:
-      - litellm/1.60.2
-    method: POST
-    uri: https://generativelanguage.googleapis.com/v1beta/models/gemma-3-1b-it:generateContent
-  response:
-    body:
-      string: !!binary |
-        H4sIAAAAAAAC/2VRy07DQAy85yusPUZtBSoIxJWHxAFRQYSQKAc3cVqL7DrKuqKlqsRv8Ht8CZuk
-        aVOxh314xuP1eBMBmBRdxhkqeXMFbyECsGn2GhOn5DQAXSgES6z0wG3XpncPFKVVnWSSBUGKJSsW
-        IDncVehSAvYQxxOs2MfxCKZu6u719/vHA0Iq1ooDyz6UTqlUDi9doELDr1M1aMbiinXcSQ9gtlTg
-        VqOGJc855VAztAZWvMInZ1SsoaJU5o6/KANxNDK9X2/39/fBoddKCqobsRLyO/q2I5icHfvFE6EX
-        V9Oek8eJ2aPsMlqF8EnUFWikzdLjnB5IMbiOe29NWYktNZEPcteybFy/bLV6MzqCxxc7XCXYcASd
-        nQ/+qfqbUJOL/ux6Yw0tYsG6buZ2+5qYng169KnOhuZ8j3aGtB69UOW5NWNO1uJwPDydDVlNtI3+
-        AD6XWQdvAgAA
-    headers:
-      Alt-Svc:
-      - h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
-      Content-Encoding:
-      - gzip
-      Content-Type:
-      - application/json; charset=UTF-8
-      Date:
-      - Tue, 22 Apr 2025 14:25:32 GMT
-      Server:
-      - scaffolding on HTTPServer2
-      Server-Timing:
-      - gfet4t7; dur=1535
-      Transfer-Encoding:
-      - chunked
-      Vary:
-      - Origin
-      - X-Origin
-      - Referer
-      X-Content-Type-Options:
-      - nosniff
-      X-Frame-Options:
-      - SAMEORIGIN
-      X-XSS-Protection:
-      - '0'
-    status:
-      code: 200
-      message: OK
-version: 1

tests/cassettes/test_gemma3[gemini-gemma-3-27b-it].yaml

@@ -6,7 +6,7 @@ interactions:
       accept:
       - '*/*'
       accept-encoding:
-      - gzip, deflate
+      - gzip, deflate, zstd
       connection:
       - keep-alive
       content-length:
@@ -16,19 +16,19 @@ interactions:
       host:
       - generativelanguage.googleapis.com
       user-agent:
-      - litellm/1.60.2
+      - litellm/1.74.9
     method: POST
     uri: https://generativelanguage.googleapis.com/v1beta/models/gemma-3-27b-it:generateContent
   response:
     body:
       string: !!binary |
-        H4sIAAAAAAAC/2VRXUvDMBR976+45EUo3RDnUHwTnSA4HFpEcHuI7e16aZqU5NZNxv67abtuHTbQ
+        H4sIAAAAAAAC/21RwWrbQBC96yuGvRSMFYp7aOktxAnY1MQ0ahtIfJhII3vwalfZGdUpxv/elRQ5
-        hHPu5zm7AEAkUqeUSkYn7uDLIwC79t9wRjNq9kQPebCSlk+x3bcbvH0I47ZJEnGOkMiKWCowGTxZ
+        CkQL2uXNm7f75h0TAJOjK7hAJTHf4SEiAMfu39a8U3IaCwMUwRqDvnH77zg6R4rSS9tksh1BjjUr
-        qRMEchCGC2nJheEYlnqpn/nCQaHNRkNmLJDvSwkoP1kpbeFAUYHAvtiMsgwVxGaDNmqRF1P/WIR5
+        WvAl3AR0OQELTCZrDCyTyQU8uke30E8Ce+cPDkofgOO9nIONL6sw7AUs7wk0il1zWZKFzB8oTDvk
-        7bAuI/ApLXxvE0gRYkumrHL0hIMNKtXcxA4y6XIyOoKkJkcau8ykVlxbHDczNUcM1tof36voJIY1
+        h2/+BoJVI9RUU4gtHXwZcigIssC+qncUCwIHsrbdWQVKlB17N4W8YWFHfWfeWG0CXbRvapcZ2Tqd
-        CptNS5Oi6sP3fYDISJPL31A6o5uw9/h1IY4s6RS3Hr4M+gZtaVE7ucY5svS2yKP4orJ+F45NgfrB
+        z5vp2zCCt9Q6rXxBdqCfBoIp2bHsfhKKdy3tLrtdm3OVXUEvEf6cDBd00qYR3NKKFGMseB6+qUP0
-        1K0tt12tgYln9PX0wLPxFpxR00n0r6p79D1JDc0d+O5XlIr4tzV29hmLgQx8NlQvQ3uvgoMgnUYf
+        opnfk7vyTRfLt17LqI8j/rAyapJ5lGQ7zm4Ua3SAlvVfl9v1fWZGLvWd8uCy2zfJq99+BL8pCPde
-        aB11YqyxLOVoMrq6+R4Ri2Af/AEDrXcbkQIAAA==
+        t1RVmH5JZ1+fUtZOzgSS2juhRdEylvgnw+VTMU/T5bPKmos7nV3+Mskp+Q+x/LCbmwIAAA==
     headers:
       Alt-Svc:
       - h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
@@ -37,11 +37,11 @@ interactions:
       Content-Type:
       - application/json; charset=UTF-8
       Date:
-      - Tue, 22 Apr 2025 14:25:41 GMT
+      - Wed, 06 Aug 2025 18:55:33 GMT
       Server:
       - scaffolding on HTTPServer2
       Server-Timing:
-      - gfet4t7; dur=2447
+      - gfet4t7; dur=1529
       Transfer-Encoding:
       - chunked
       Vary:

tests/cassettes/test_gemma3[gemini-gemma-3-4b-it].yaml

@@ -1,60 +0,0 @@
-interactions:
-- request:
-    body: '{"contents": [{"role": "user", "parts": [{"text": "What is the capital
-      of France?"}]}], "generationConfig": {"stop_sequences": []}}'
-    headers:
-      accept:
-      - '*/*'
-      accept-encoding:
-      - gzip, deflate
-      connection:
-      - keep-alive
-      content-length:
-      - '131'
-      content-type:
-      - application/json
-      host:
-      - generativelanguage.googleapis.com
-      user-agent:
-      - litellm/1.60.2
-    method: POST
-    uri: https://generativelanguage.googleapis.com/v1beta/models/gemma-3-4b-it:generateContent
-  response:
-    body:
-      string: !!binary |
-        H4sIAAAAAAAC/2WRzUrDQBDH73mKYY+lLUKLiBcPfoAHsWhQwXqYJtNk6WYn7E6woRQ8+wR68t18
-        Ah/BbWraFPeQXeb/z3z8ZhUBqARtqlMU8uoUnkMEYNV8NxpbIStBaEMhWKKTvXd7Vp13sAgtNz+p
-        OCdIsNSCBngOVw5tQqA99HoTdNr3ekOY2qm9lu+3Tw8ImeFZ8CahKDmYs4NQrA9z9Llm24cMvTi2
-        XNQQ2oakMlI5GsLP18d7k+mRK5NCzRUYvSAQhoXl12CuJdc2g4IdAc64Emg6OFOdzte790t/P69j
-        Q5thCk7JtPZ1a1BzbbXP7wg9243tPr6dqJ2qbUrLED6K2gJNalV5zOiGBAN53PFVpeOilJgXZM+5
-        asifbHN19nQgj1pdOFA+kMbH/X9Z/UWoqU13f53VhhHRaKmb3V0+xaqDQQ6aajE090v0B2TL6IGc
-        11sYGRUFDkaD8WygRUXr6BcxmBLccwIAAA==
-    headers:
-      Alt-Svc:
-      - h3=":443"; ma=2592000,h3-29=":443"; ma=2592000
-      Content-Encoding:
-      - gzip
-      Content-Type:
-      - application/json; charset=UTF-8
-      Date:
-      - Tue, 22 Apr 2025 14:25:35 GMT
-      Server:
-      - scaffolding on HTTPServer2
-      Server-Timing:
-      - gfet4t7; dur=2349
-      Transfer-Encoding:
-      - chunked
-      Vary:
-      - Origin
-      - X-Origin
-      - Referer
-      X-Content-Type-Options:
-      - nosniff
-      X-Frame-Options:
-      - SAMEORIGIN
-      X-XSS-Protection:
-      - '0'
-    status:
-      code: 200
-      message: OK
-version: 1

tests/llm_test.py

@@ -282,9 +282,6 @@ def test_gemini_models(model):
 @pytest.mark.parametrize(
     "model",
     [
-        "gemini/gemma-3-1b-it",
-        "gemini/gemma-3-4b-it",
-        "gemini/gemma-3-12b-it",
         "gemini/gemma-3-27b-it",
     ],
 )
@@ -377,6 +374,7 @@ def get_weather_tool_schema():
         },
     }
 
+
 def test_context_window_exceeded_error_handling():
     """Test that litellm.ContextWindowExceededError is converted to LLMContextLengthExceededException."""
     from litellm.exceptions import ContextWindowExceededError
@@ -392,7 +390,7 @@ def test_context_window_exceeded_error_handling():
         mock_completion.side_effect = ContextWindowExceededError(
             "This model's maximum context length is 8192 tokens. However, your messages resulted in 10000 tokens.",
             model="gpt-4",
-            llm_provider="openai"
+            llm_provider="openai",
         )
 
         with pytest.raises(LLMContextLengthExceededException) as excinfo:
@@ -407,7 +405,7 @@ def test_context_window_exceeded_error_handling():
         mock_completion.side_effect = ContextWindowExceededError(
             "This model's maximum context length is 8192 tokens. However, your messages resulted in 10000 tokens.",
             model="gpt-4",
-            llm_provider="openai"
+            llm_provider="openai",
         )
 
         with pytest.raises(LLMContextLengthExceededException) as excinfo:
@@ -598,6 +596,7 @@ def test_handle_streaming_tool_calls(get_weather_tool_schema, mock_emit):
         expected_final_chunk_result=expected_final_chunk_result,
     )
 
+
 @pytest.mark.vcr(filter_headers=["authorization"])
 def test_handle_streaming_tool_calls_with_error(get_weather_tool_schema, mock_emit):
     def get_weather_error(location):
@@ -609,9 +608,7 @@ def test_handle_streaming_tool_calls_with_error(get_weather_tool_schema, mock_em
             {"role": "user", "content": "What is the weather in New York?"},
         ],
         tools=[get_weather_tool_schema],
-        available_functions={
+        available_functions={"get_weather": get_weather_error},
-            "get_weather": get_weather_error
-        },
     )
     assert response == ""
     expected_final_chunk_result = '{"location":"New York, NY"}'
@@ -676,8 +673,11 @@ def test_llm_call_when_stop_is_unsupported(caplog):
     assert isinstance(result, str)
     assert "Paris" in result
 
+
 @pytest.mark.vcr(filter_headers=["authorization"])
-def test_llm_call_when_stop_is_unsupported_when_additional_drop_params_is_provided(caplog):
+def test_llm_call_when_stop_is_unsupported_when_additional_drop_params_is_provided(
+    caplog,
+):
     llm = LLM(model="o1-mini", stop=["stop"], additional_drop_params=["another_param"])
     with caplog.at_level(logging.INFO):
         result = llm.call("What is the capital of France?")
@@ -690,6 +690,7 @@ def test_llm_call_when_stop_is_unsupported_when_additional_drop_params_is_provid
 def ollama_llm():
     return LLM(model="ollama/llama3.2:3b")
 
+
 def test_ollama_appends_dummy_user_message_when_last_is_assistant(ollama_llm):
     original_messages = [
         {"role": "user", "content": "Hi there"},

tests/tracing/test_tracing.py

@@ -0,0 +1,313 @@
+import os
+import pytest
+from unittest.mock import patch, MagicMock
+
+# Remove the module-level patch
+from crewai import Agent, Task, Crew
+from crewai.utilities.events.listeners.tracing.trace_listener import (
+    TraceCollectionListener,
+)
+from crewai.utilities.events.listeners.tracing.trace_batch_manager import (
+    TraceBatchManager,
+)
+from crewai.utilities.events.listeners.tracing.types import TraceEvent
+
+
+class TestTraceListenerSetup:
+    """Test TraceListener is properly setup and collecting events"""
+
+    @pytest.fixture(autouse=True)
+    def mock_auth_token(self):
+        """Mock authentication token for all tests in this class"""
+        # Need to patch all the places where get_auth_token is imported/used
+        with (
+            patch(
+                "crewai.cli.authentication.token.get_auth_token",
+                return_value="mock_token_12345",
+            ),
+            patch(
+                "crewai.utilities.events.listeners.tracing.trace_listener.get_auth_token",
+                return_value="mock_token_12345",
+            ),
+            patch(
+                "crewai.utilities.events.listeners.tracing.trace_batch_manager.get_auth_token",
+                return_value="mock_token_12345",
+            ),
+            patch(
+                "crewai.utilities.events.listeners.tracing.interfaces.get_auth_token",
+                return_value="mock_token_12345",
+            ),
+        ):
+            yield
+
+    @pytest.fixture(autouse=True)
+    def clear_event_bus(self):
+        """Clear event bus listeners before and after each test"""
+        from crewai.utilities.events import crewai_event_bus
+
+        # Store original handlers
+        original_handlers = crewai_event_bus._handlers.copy()
+
+        # Clear for test
+        crewai_event_bus._handlers.clear()
+
+        yield
+
+        # Restore original state
+        crewai_event_bus._handlers.clear()
+        crewai_event_bus._handlers.update(original_handlers)
+
+    @pytest.fixture(autouse=True)
+    def reset_tracing_singletons(self):
+        """Reset tracing singleton instances between tests"""
+        # Reset TraceCollectionListener singleton
+        if hasattr(TraceCollectionListener, "_instance"):
+            TraceCollectionListener._instance = None
+            TraceCollectionListener._initialized = False
+
+        yield
+
+        # Clean up after test
+        if hasattr(TraceCollectionListener, "_instance"):
+            TraceCollectionListener._instance = None
+            TraceCollectionListener._initialized = False
+
+    @pytest.fixture(autouse=True)
+    def mock_plus_api_calls(self):
+        """Mock all PlusAPI HTTP calls to avoid network requests"""
+        with (
+            patch("requests.post") as mock_post,
+            patch("requests.get") as mock_get,
+            patch("requests.put") as mock_put,
+            patch("requests.delete") as mock_delete,
+            patch.object(TraceBatchManager, "initialize_batch", return_value=None),
+            patch.object(
+                TraceBatchManager, "_finalize_backend_batch", return_value=True
+            ),
+            patch.object(TraceBatchManager, "_cleanup_batch_data", return_value=True),
+        ):
+            mock_response = MagicMock()
+            mock_response.status_code = 200
+            mock_response.json.return_value = {
+                "id": "mock_trace_batch_id",
+                "status": "success",
+                "message": "Batch created successfully",
+            }
+            mock_response.raise_for_status.return_value = None
+
+            mock_post.return_value = mock_response
+            mock_get.return_value = mock_response
+            mock_put.return_value = mock_response
+            mock_delete.return_value = mock_response
+
+            yield {
+                "post": mock_post,
+                "get": mock_get,
+                "put": mock_put,
+                "delete": mock_delete,
+            }
+
+    @pytest.mark.vcr(filter_headers=["authorization"])
+    def test_trace_listener_collects_crew_events(self):
+        """Test that trace listener properly collects events from crew execution"""
+
+        with patch.dict(os.environ, {"CREWAI_TRACING_ENABLED": "true"}):
+            agent = Agent(
+                role="Test Agent",
+                goal="Test goal",
+                backstory="Test backstory",
+                llm="gpt-4o-mini",
+            )
+            task = Task(
+                description="Say hello to the world",
+                expected_output="hello world",
+                agent=agent,
+            )
+            crew = Crew(agents=[agent], tasks=[task], verbose=True)
+
+            trace_listener = TraceCollectionListener()
+            from crewai.utilities.events import crewai_event_bus
+
+            trace_listener.setup_listeners(crewai_event_bus)
+
+            with patch.object(
+                trace_listener.batch_manager,
+                "initialize_batch",
+                return_value=None,
+            ) as initialize_mock:
+                crew.kickoff()
+
+                assert initialize_mock.call_count >= 1
+
+                call_args = initialize_mock.call_args_list[0]
+                assert len(call_args[0]) == 2  # user_context, execution_metadata
+                _, execution_metadata = call_args[0]
+                assert isinstance(execution_metadata, dict)
+                assert "crew_name" in execution_metadata
+
+    @pytest.mark.vcr(filter_headers=["authorization"])
+    def test_batch_manager_finalizes_batch_clears_buffer(self):
+        """Test that batch manager properly finalizes batch and clears buffer"""
+
+        with patch.dict(os.environ, {"CREWAI_TRACING_ENABLED": "true"}):
+            agent = Agent(
+                role="Test Agent",
+                goal="Test goal",
+                backstory="Test backstory",
+                llm="gpt-4o-mini",
+            )
+
+            task = Task(
+                description="Say hello to the world",
+                expected_output="hello world",
+                agent=agent,
+            )
+
+            crew = Crew(agents=[agent], tasks=[task], verbose=True)
+
+            from crewai.utilities.events import crewai_event_bus
+
+            trace_listener = None
+            for handler_list in crewai_event_bus._handlers.values():
+                for handler in handler_list:
+                    if hasattr(handler, "__self__") and isinstance(
+                        handler.__self__, TraceCollectionListener
+                    ):
+                        trace_listener = handler.__self__
+                        break
+                if trace_listener:
+                    break
+
+            if not trace_listener:
+                pytest.skip(
+                    "No trace listener found - tracing may not be properly enabled"
+                )
+
+            with patch.object(
+                trace_listener.batch_manager,
+                "finalize_batch",
+                wraps=trace_listener.batch_manager.finalize_batch,
+            ) as finalize_mock:
+                crew.kickoff()
+
+                assert finalize_mock.call_count >= 1
+
+    @pytest.mark.vcr(filter_headers=["authorization"])
+    def test_events_collection_batch_manager(self, mock_plus_api_calls):
+        """Test that trace listener properly collects events from crew execution"""
+
+        with patch.dict(os.environ, {"CREWAI_TRACING_ENABLED": "true"}):
+            agent = Agent(
+                role="Test Agent",
+                goal="Test goal",
+                backstory="Test backstory",
+                llm="gpt-4o-mini",
+            )
+            task = Task(
+                description="Say hello to the world",
+                expected_output="hello world",
+                agent=agent,
+            )
+            crew = Crew(agents=[agent], tasks=[task], verbose=True)
+
+            from crewai.utilities.events import crewai_event_bus
+
+            # Create and setup trace listener explicitly
+            trace_listener = TraceCollectionListener()
+            trace_listener.setup_listeners(crewai_event_bus)
+
+            with patch.object(
+                trace_listener.batch_manager,
+                "add_event",
+                wraps=trace_listener.batch_manager.add_event,
+            ) as add_event_mock:
+                crew.kickoff()
+
+                assert add_event_mock.call_count >= 2
+
+                completion_events = [
+                    call.args[0]
+                    for call in add_event_mock.call_args_list
+                    if call.args[0].type == "crew_kickoff_completed"
+                ]
+                assert len(completion_events) >= 1
+
+                # Verify the first completion event has proper structure
+                completion_event = completion_events[0]
+                assert "crew_name" in completion_event.event_data
+                assert completion_event.event_data["crew_name"] == "crew"
+
+                # Verify all events have proper structure
+                for call in add_event_mock.call_args_list:
+                    event = call.args[0]
+                    assert isinstance(event, TraceEvent)
+                    assert hasattr(event, "event_data")
+                    assert hasattr(event, "type")
+
+    @pytest.mark.vcr(filter_headers=["authorization"])
+    def test_trace_listener_disabled_when_env_false(self):
+        """Test that trace listener doesn't make HTTP calls when tracing is disabled"""
+
+        with patch.dict(os.environ, {"CREWAI_TRACING_ENABLED": "false"}):
+            agent = Agent(
+                role="Test Agent",
+                goal="Test goal",
+                backstory="Test backstory",
+                llm="gpt-4o-mini",
+            )
+            task = Task(
+                description="Say hello to the world",
+                expected_output="hello world",
+                agent=agent,
+            )
+
+            crew = Crew(agents=[agent], tasks=[task], verbose=True)
+            result = crew.kickoff()
+            assert result is not None
+
+            from crewai.utilities.events import crewai_event_bus
+
+            trace_handlers = []
+            for handlers in crewai_event_bus._handlers.values():
+                for handler in handlers:
+                    if hasattr(handler, "__self__") and isinstance(
+                        handler.__self__, TraceCollectionListener
+                    ):
+                        trace_handlers.append(handler)
+                    elif hasattr(handler, "__name__") and any(
+                        trace_name in handler.__name__
+                        for trace_name in [
+                            "on_crew_started",
+                            "on_crew_completed",
+                            "on_flow_started",
+                        ]
+                    ):
+                        trace_handlers.append(handler)
+
+            assert len(trace_handlers) == 0, (
+                f"Found {len(trace_handlers)} trace handlers when tracing should be disabled"
+            )
+
+    def test_trace_listener_setup_correctly(self):
+        """Test that trace listener is set up correctly when enabled"""
+
+        with patch.dict(os.environ, {"CREWAI_TRACING_ENABLED": "true"}):
+            trace_listener = TraceCollectionListener()
+
+            assert trace_listener.trace_enabled is True
+            assert trace_listener.batch_manager is not None
+            assert trace_listener.trace_sender is not None
+
+    # Helper method to ensure cleanup
+    def teardown_method(self):
+        """Cleanup after each test method"""
+        from crewai.utilities.events import crewai_event_bus
+
+        crewai_event_bus._handlers.clear()
+
+    @classmethod
+    def teardown_class(cls):
+        """Final cleanup after all tests in this class"""
+        from crewai.utilities.events import crewai_event_bus
+
+        crewai_event_bus._handlers.clear()