refactor(events): relocate events module & update imports
- Move events from utilities/ to top-level events/ with types/, listeners/, utils/ structure
- Update all source/tests/docs to new import paths
- Add backwards compatibility stubs in crewai.utilities.events with deprecation warnings
- Restore test mocks and fix related test imports
* feat: implement tool usage limit exception handling
- Introduced `ToolUsageLimitExceeded` exception to manage maximum usage limits for tools.
- Enhanced `CrewStructuredTool` to check and raise this exception when the usage limit is reached.
- Updated `_run` and `_execute` methods to include usage limit checks and handle exceptions appropriately, improving reliability and user feedback.
* feat: enhance PlusAPI and ToolUsage with task metadata
- Removed the `send_trace_batch` method from PlusAPI to streamline the API.
- Added timeout parameters to trace event methods in PlusAPI for improved reliability.
- Updated ToolUsage to include task metadata (task name and ID) in event emissions, enhancing traceability and context during tool usage.
- Refactored event handling in LLM and ToolUsage events to ensure task information is consistently captured.
* feat: enhance memory and event handling with task and agent metadata
- Added task and agent metadata to various memory and event classes, improving traceability and context during memory operations.
- Updated the `ContextualMemory` and `Memory` classes to associate tasks and agents, allowing for better context management.
- Enhanced event emissions in `LLM`, `ToolUsage`, and memory events to include task and agent information, facilitating improved debugging and monitoring.
- Refactored event handling to ensure consistent capture of task and agent details across the system.
* drop
* refactor: clean up unused imports in memory and event modules
- Removed unused TYPE_CHECKING imports from long_term_memory.py to streamline the code.
- Eliminated unnecessary import from memory_events.py, enhancing clarity and maintainability.
* fix memory tests
* fix task_completed payload
* fix: remove unused test agent variable in external memory tests
* refactor: remove unused agent parameter from Memory class save method
- Eliminated the agent parameter from the save method in the Memory class to streamline the code and improve clarity.
- Updated the TraceBatchManager class by moving initialization of attributes into the constructor for better organization and readability.
* refactor: enhance ExecutionState and ReasoningEvent classes with optional task and agent identifiers
- Added optional `current_agent_id` and `current_task_id` attributes to the `ExecutionState` class for better tracking of agent and task states.
- Updated the `from_task` attribute in the `ReasoningEvent` class to use `Optional[Any]` instead of a specific type, improving flexibility in event handling.
* refactor: update ExecutionState class by removing unused agent and task identifiers
- Removed the `current_agent_id` and `current_task_id` attributes from the `ExecutionState` class to simplify the code and enhance clarity.
- Adjusted the import statements to include `Optional` for better type handling.
* refactor: streamline LLM event handling in LiteAgent
- Removed unused LLM event emissions (LLMCallStartedEvent, LLMCallCompletedEvent, LLMCallFailedEvent) from the LiteAgent class to simplify the code and improve performance.
- Adjusted the flow of LLM response handling by eliminating unnecessary event bus interactions, enhancing clarity and maintainability.
* flow ownership and not emitting events when a crew is done
* refactor: remove unused agent parameter from ShortTermMemory save method
- Eliminated the agent parameter from the save method in the ShortTermMemory class to streamline the code and improve clarity.
- This change enhances the maintainability of the memory management system by reducing unnecessary complexity.
* runtype check fix
* fixing tests
* fix lints
* fix: update event assertions in test_llm_emits_event_with_lite_agent
- Adjusted the expected counts for completed and started events in the test to reflect the correct behavior of the LiteAgent.
- Updated assertions for agent roles and IDs to match the expected values after recent changes in event handling.
* fix: update task name assertions in event tests
- Modified assertions in `test_stream_llm_emits_event_with_task_and_agent_info` and `test_llm_emits_event_with_task_and_agent_info` to use `task.description` as a fallback for `task.name`. This ensures that the tests correctly validate the task name even when it is not explicitly set.
* fix: update test assertions for output values and improve readability
- Updated assertions in `test_output_json_dict_hierarchical` to reflect the correct expected score value.
- Enhanced readability of assertions in `test_output_pydantic_to_another_task` and `test_key` by formatting the error messages for clarity.
- These changes ensure that the tests accurately validate the expected outputs and improve overall code quality.
* test fixes
* fix crew_test
* added another fixture
* fix: ensure agent and task assignments in contextual memory are conditional
- Updated the ContextualMemory class to check for the existence of short-term, long-term, external, and extended memory before assigning agent and task attributes. This prevents potential attribute errors when memory types are not initialized.
* feat: add exchanged messages in LLMCallCompletedEvent
* feat: add GoalAlignment metric for Agent evaluation
* feat: add SemanticQuality metric for Agent evaluation
* feat: add Tool Metrics for Agent evaluation
* feat: add Reasoning Metrics for Agent evaluation, still in progress
* feat: add AgentEvaluator class
This class will evaluate Agent' results and report to user
* fix: do not evaluate Agent by default
This is a experimental feature we still need refine it further
* test: add Agent eval tests
* fix: render all feedback per iteration
* style: resolve linter issues
* style: fix mypy issues
* fix: allow messages be empty on LLMCallCompletedEvent
* feat: add Experiment evaluation framework with baseline comparison
* fix: reset evaluator for each experiement iteraction
* fix: fix track of new test cases
* chore: split Experimental evaluation classes
* refactor: remove unused method
* refactor: isolate Console print in a dedicated class
* fix: make crew required to run an experiment
* fix: use time-aware to define experiment result
* test: add tests for Evaluator Experiment
* style: fix linter issues
* fix: encode string before hashing
* style: resolve linter issues
* feat: add experimental folder for beta features (#3141)
* test: move tests to experimental folder
