Lorenze/better tracing events (#3382)

* feat: implement tool usage limit exception handling

- Introduced `ToolUsageLimitExceeded` exception to manage maximum usage limits for tools.
- Enhanced `CrewStructuredTool` to check and raise this exception when the usage limit is reached.
- Updated `_run` and `_execute` methods to include usage limit checks and handle exceptions appropriately, improving reliability and user feedback.

* feat: enhance PlusAPI and ToolUsage with task metadata

- Removed the `send_trace_batch` method from PlusAPI to streamline the API.
- Added timeout parameters to trace event methods in PlusAPI for improved reliability.
- Updated ToolUsage to include task metadata (task name and ID) in event emissions, enhancing traceability and context during tool usage.
- Refactored event handling in LLM and ToolUsage events to ensure task information is consistently captured.

* feat: enhance memory and event handling with task and agent metadata

- Added task and agent metadata to various memory and event classes, improving traceability and context during memory operations.
- Updated the `ContextualMemory` and `Memory` classes to associate tasks and agents, allowing for better context management.
- Enhanced event emissions in `LLM`, `ToolUsage`, and memory events to include task and agent information, facilitating improved debugging and monitoring.
- Refactored event handling to ensure consistent capture of task and agent details across the system.

* drop

* refactor: clean up unused imports in memory and event modules

- Removed unused TYPE_CHECKING imports from long_term_memory.py to streamline the code.
- Eliminated unnecessary import from memory_events.py, enhancing clarity and maintainability.

* fix memory tests

* fix task_completed payload

* fix: remove unused test agent variable in external memory tests

* refactor: remove unused agent parameter from Memory class save method

- Eliminated the agent parameter from the save method in the Memory class to streamline the code and improve clarity.
- Updated the TraceBatchManager class by moving initialization of attributes into the constructor for better organization and readability.

* refactor: enhance ExecutionState and ReasoningEvent classes with optional task and agent identifiers

- Added optional `current_agent_id` and `current_task_id` attributes to the `ExecutionState` class for better tracking of agent and task states.
- Updated the `from_task` attribute in the `ReasoningEvent` class to use `Optional[Any]` instead of a specific type, improving flexibility in event handling.

* refactor: update ExecutionState class by removing unused agent and task identifiers

- Removed the `current_agent_id` and `current_task_id` attributes from the `ExecutionState` class to simplify the code and enhance clarity.
- Adjusted the import statements to include `Optional` for better type handling.

* refactor: streamline LLM event handling in LiteAgent

- Removed unused LLM event emissions (LLMCallStartedEvent, LLMCallCompletedEvent, LLMCallFailedEvent) from the LiteAgent class to simplify the code and improve performance.
- Adjusted the flow of LLM response handling by eliminating unnecessary event bus interactions, enhancing clarity and maintainability.

* flow ownership and not emitting events when a crew is done

* refactor: remove unused agent parameter from ShortTermMemory save method

- Eliminated the agent parameter from the save method in the ShortTermMemory class to streamline the code and improve clarity.
- This change enhances the maintainability of the memory management system by reducing unnecessary complexity.

* runtype check fix

* fixing tests

* fix lints

* fix: update event assertions in test_llm_emits_event_with_lite_agent

- Adjusted the expected counts for completed and started events in the test to reflect the correct behavior of the LiteAgent.
- Updated assertions for agent roles and IDs to match the expected values after recent changes in event handling.

* fix: update task name assertions in event tests

- Modified assertions in `test_stream_llm_emits_event_with_task_and_agent_info` and `test_llm_emits_event_with_task_and_agent_info` to use `task.description` as a fallback for `task.name`. This ensures that the tests correctly validate the task name even when it is not explicitly set.

* fix: update test assertions for output values and improve readability

- Updated assertions in `test_output_json_dict_hierarchical` to reflect the correct expected score value.
- Enhanced readability of assertions in `test_output_pydantic_to_another_task` and `test_key` by formatting the error messages for clarity.
- These changes ensure that the tests accurately validate the expected outputs and improve overall code quality.

* test fixes

* fix crew_test

* added another fixture

* fix: ensure agent and task assignments in contextual memory are conditional

- Updated the ContextualMemory class to check for the existence of short-term, long-term, external, and extended memory before assigning agent and task attributes. This prevents potential attribute errors when memory types are not initialized.

tests/memory/test_external_memory.py

@@ -222,13 +222,12 @@ def test_external_memory_custom_storage(custom_storage, crew_with_external_memor
 
     test_value = "test value"
     test_metadata = {"source": "test"}
-    test_agent = "test_agent"
-    external_memory.save(value=test_value, metadata=test_metadata, agent=test_agent)
+    external_memory.save(value=test_value, metadata=test_metadata)
 
     results = external_memory.search("test")
     assert len(results) == 1
     assert results[0]["value"] == test_value
-    assert results[0]["metadata"] == test_metadata | {"agent": test_agent}
+    assert results[0]["metadata"] == test_metadata
 
     external_memory.reset()
     results = external_memory.search("test")
@@ -259,7 +258,6 @@ def test_external_memory_search_events(
 
     assert len(events["MemoryQueryStartedEvent"]) == 1
     assert len(events["MemoryQueryCompletedEvent"]) == 1
-    assert len(events["MemoryQueryFailedEvent"]) == 0
 
     assert dict(events["MemoryQueryStartedEvent"][0]) == {
         "timestamp": ANY,
@@ -267,6 +265,12 @@ def test_external_memory_search_events(
         "source_fingerprint": None,
         "source_type": "external_memory",
         "fingerprint_metadata": None,
+        "task_id": None,
+        "task_name": None,
+        "from_task": None,
+        "from_agent": None,
+        "agent_role": None,
+        "agent_id": None,
         "query": "test value",
         "limit": 3,
         "score_threshold": 0.35,
@@ -278,6 +282,12 @@ def test_external_memory_search_events(
         "source_fingerprint": None,
         "source_type": "external_memory",
         "fingerprint_metadata": None,
+        "task_id": None,
+        "task_name": None,
+        "from_task": None,
+        "from_agent": None,
+        "agent_role": None,
+        "agent_id": None,
         "query": "test value",
         "results": [],
         "limit": 3,
@@ -306,12 +316,10 @@ def test_external_memory_save_events(
         external_memory_with_mocked_config.save(
             value="saving value",
             metadata={"task": "test_task"},
-            agent="test_agent",
         )
 
     assert len(events["MemorySaveStartedEvent"]) == 1
     assert len(events["MemorySaveCompletedEvent"]) == 1
-    assert len(events["MemorySaveFailedEvent"]) == 0
 
     assert dict(events["MemorySaveStartedEvent"][0]) == {
         "timestamp": ANY,
@@ -319,9 +327,14 @@ def test_external_memory_save_events(
         "source_fingerprint": None,
         "source_type": "external_memory",
         "fingerprint_metadata": None,
+        "task_id": None,
+        "task_name": None,
+        "from_task": None,
+        "from_agent": None,
+        "agent_role": None,
+        "agent_id": None,
         "value": "saving value",
         "metadata": {"task": "test_task"},
-        "agent_role": "test_agent",
     }
 
     assert dict(events["MemorySaveCompletedEvent"][0]) == {
@@ -330,8 +343,13 @@ def test_external_memory_save_events(
         "source_fingerprint": None,
         "source_type": "external_memory",
         "fingerprint_metadata": None,
+        "task_id": None,
+        "task_name": None,
+        "from_task": None,
+        "from_agent": None,
+        "agent_role": None,
+        "agent_id": None,
         "value": "saving value",
-        "metadata": {"task": "test_task", "agent": "test_agent"},
-        "agent_role": "test_agent",
+        "metadata": {"task": "test_task"},
         "save_time_ms": ANY,
     }

tests/memory/test_long_term_memory.py

@@ -51,9 +51,14 @@ def test_long_term_memory_save_events(long_term_memory):
         "source_fingerprint": None,
         "source_type": "long_term_memory",
         "fingerprint_metadata": None,
+        "task_id": None,
+        "task_name": None,
+        "from_task": None,
+        "from_agent": None,
+        "agent_role": "test_agent",
+        "agent_id": None,
         "value": "test_task",
         "metadata": {"task": "test_task", "quality": 0.5},
-        "agent_role": "test_agent",
     }
     assert dict(events["MemorySaveCompletedEvent"][0]) == {
         "timestamp": ANY,
@@ -61,6 +66,12 @@ def test_long_term_memory_save_events(long_term_memory):
         "source_fingerprint": None,
         "source_type": "long_term_memory",
         "fingerprint_metadata": None,
+        "task_id": None,
+        "task_name": None,
+        "from_task": None,
+        "from_agent": None,
+        "agent_role": "test_agent",
+        "agent_id": None,
         "value": "test_task",
         "metadata": {
             "task": "test_task",
@@ -68,7 +79,6 @@ def test_long_term_memory_save_events(long_term_memory):
             "agent": "test_agent",
             "expected_output": "test_output",
         },
-        "agent_role": "test_agent",
         "save_time_ms": ANY,
     }
 
@@ -100,6 +110,12 @@ def test_long_term_memory_search_events(long_term_memory):
         "source_fingerprint": None,
         "source_type": "long_term_memory",
         "fingerprint_metadata": None,
+        "task_id": None,
+        "task_name": None,
+        "from_task": None,
+        "from_agent": None,
+        "agent_role": None,
+        "agent_id": None,
         "query": "test query",
         "limit": 5,
         "score_threshold": None,
@@ -111,6 +127,12 @@ def test_long_term_memory_search_events(long_term_memory):
         "source_fingerprint": None,
         "source_type": "long_term_memory",
         "fingerprint_metadata": None,
+        "task_id": None,
+        "task_name": None,
+        "from_task": None,
+        "from_agent": None,
+        "agent_role": None,
+        "agent_id": None,
         "query": "test query",
         "results": None,
         "limit": 5,

tests/memory/test_short_term_memory.py

@@ -57,7 +57,6 @@ def test_short_term_memory_search_events(short_term_memory):
 
     assert len(events["MemoryQueryStartedEvent"]) == 1
     assert len(events["MemoryQueryCompletedEvent"]) == 1
-    assert len(events["MemoryQueryFailedEvent"]) == 0
 
     assert dict(events["MemoryQueryStartedEvent"][0]) == {
         "timestamp": ANY,
@@ -65,6 +64,12 @@ def test_short_term_memory_search_events(short_term_memory):
         "source_fingerprint": None,
         "source_type": "short_term_memory",
         "fingerprint_metadata": None,
+        "task_id": None,
+        "task_name": None,
+        "from_task": None,
+        "from_agent": None,
+        "agent_role": None,
+        "agent_id": None,
         "query": "test value",
         "limit": 3,
         "score_threshold": 0.35,
@@ -76,6 +81,12 @@ def test_short_term_memory_search_events(short_term_memory):
         "source_fingerprint": None,
         "source_type": "short_term_memory",
         "fingerprint_metadata": None,
+        "task_id": None,
+        "task_name": None,
+        "from_task": None,
+        "from_agent": None,
+        "agent_role": None,
+        "agent_id": None,
         "query": "test value",
         "results": [],
         "limit": 3,
@@ -99,12 +110,10 @@ def test_short_term_memory_save_events(short_term_memory):
         short_term_memory.save(
             value="test value",
             metadata={"task": "test_task"},
-            agent="test_agent",
         )
 
     assert len(events["MemorySaveStartedEvent"]) == 1
     assert len(events["MemorySaveCompletedEvent"]) == 1
-    assert len(events["MemorySaveFailedEvent"]) == 0
 
     assert dict(events["MemorySaveStartedEvent"][0]) == {
         "timestamp": ANY,
@@ -112,9 +121,14 @@ def test_short_term_memory_save_events(short_term_memory):
         "source_fingerprint": None,
         "source_type": "short_term_memory",
         "fingerprint_metadata": None,
+        "task_id": None,
+        "task_name": None,
+        "from_task": None,
+        "from_agent": None,
+        "agent_role": None,
+        "agent_id": None,
         "value": "test value",
         "metadata": {"task": "test_task"},
-        "agent_role": "test_agent",
     }
 
     assert dict(events["MemorySaveCompletedEvent"][0]) == {
@@ -123,9 +137,14 @@ def test_short_term_memory_save_events(short_term_memory):
         "source_fingerprint": None,
         "source_type": "short_term_memory",
         "fingerprint_metadata": None,
+        "task_id": None,
+        "task_name": None,
+        "from_task": None,
+        "from_agent": None,
+        "agent_role": None,
+        "agent_id": None,
         "value": "test value",
-        "metadata": {"task": "test_task", "agent": "test_agent"},
-        "agent_role": "test_agent",
+        "metadata": {"task": "test_task"},
         "save_time_ms": ANY,
     }