* feat: implement tool usage limit exception handling
- Introduced `ToolUsageLimitExceeded` exception to manage maximum usage limits for tools.
- Enhanced `CrewStructuredTool` to check and raise this exception when the usage limit is reached.
- Updated `_run` and `_execute` methods to include usage limit checks and handle exceptions appropriately, improving reliability and user feedback.
* feat: enhance PlusAPI and ToolUsage with task metadata
- Removed the `send_trace_batch` method from PlusAPI to streamline the API.
- Added timeout parameters to trace event methods in PlusAPI for improved reliability.
- Updated ToolUsage to include task metadata (task name and ID) in event emissions, enhancing traceability and context during tool usage.
- Refactored event handling in LLM and ToolUsage events to ensure task information is consistently captured.
* feat: enhance memory and event handling with task and agent metadata
- Added task and agent metadata to various memory and event classes, improving traceability and context during memory operations.
- Updated the `ContextualMemory` and `Memory` classes to associate tasks and agents, allowing for better context management.
- Enhanced event emissions in `LLM`, `ToolUsage`, and memory events to include task and agent information, facilitating improved debugging and monitoring.
- Refactored event handling to ensure consistent capture of task and agent details across the system.
* drop
* refactor: clean up unused imports in memory and event modules
- Removed unused TYPE_CHECKING imports from long_term_memory.py to streamline the code.
- Eliminated unnecessary import from memory_events.py, enhancing clarity and maintainability.
* fix memory tests
* fix task_completed payload
* fix: remove unused test agent variable in external memory tests
* refactor: remove unused agent parameter from Memory class save method
- Eliminated the agent parameter from the save method in the Memory class to streamline the code and improve clarity.
- Updated the TraceBatchManager class by moving initialization of attributes into the constructor for better organization and readability.
* refactor: enhance ExecutionState and ReasoningEvent classes with optional task and agent identifiers
- Added optional `current_agent_id` and `current_task_id` attributes to the `ExecutionState` class for better tracking of agent and task states.
- Updated the `from_task` attribute in the `ReasoningEvent` class to use `Optional[Any]` instead of a specific type, improving flexibility in event handling.
* refactor: update ExecutionState class by removing unused agent and task identifiers
- Removed the `current_agent_id` and `current_task_id` attributes from the `ExecutionState` class to simplify the code and enhance clarity.
- Adjusted the import statements to include `Optional` for better type handling.
* refactor: streamline LLM event handling in LiteAgent
- Removed unused LLM event emissions (LLMCallStartedEvent, LLMCallCompletedEvent, LLMCallFailedEvent) from the LiteAgent class to simplify the code and improve performance.
- Adjusted the flow of LLM response handling by eliminating unnecessary event bus interactions, enhancing clarity and maintainability.
* flow ownership and not emitting events when a crew is done
* refactor: remove unused agent parameter from ShortTermMemory save method
- Eliminated the agent parameter from the save method in the ShortTermMemory class to streamline the code and improve clarity.
- This change enhances the maintainability of the memory management system by reducing unnecessary complexity.
* runtype check fix
* fixing tests
* fix lints
* fix: update event assertions in test_llm_emits_event_with_lite_agent
- Adjusted the expected counts for completed and started events in the test to reflect the correct behavior of the LiteAgent.
- Updated assertions for agent roles and IDs to match the expected values after recent changes in event handling.
* fix: update task name assertions in event tests
- Modified assertions in `test_stream_llm_emits_event_with_task_and_agent_info` and `test_llm_emits_event_with_task_and_agent_info` to use `task.description` as a fallback for `task.name`. This ensures that the tests correctly validate the task name even when it is not explicitly set.
* fix: update test assertions for output values and improve readability
- Updated assertions in `test_output_json_dict_hierarchical` to reflect the correct expected score value.
- Enhanced readability of assertions in `test_output_pydantic_to_another_task` and `test_key` by formatting the error messages for clarity.
- These changes ensure that the tests accurately validate the expected outputs and improve overall code quality.
* test fixes
* fix crew_test
* added another fixture
* fix: ensure agent and task assignments in contextual memory are conditional
- Updated the ContextualMemory class to check for the existence of short-term, long-term, external, and extended memory before assigning agent and task attributes. This prevents potential attribute errors when memory types are not initialized.
* feat: enhance BaseTool and CrewStructuredTool with usage tracking
This commit introduces a mechanism to track the usage count of tools within the CrewAI framework. The `BaseTool` class now includes a `_increment_usage_count` method that updates the current usage count, which is also reflected in the associated `CrewStructuredTool`. Additionally, a new test has been added to ensure that the maximum usage count is respected when invoking tools, enhancing the overall reliability and functionality of the tool system.
* feat: add max usage count feature to tools documentation
This commit introduces a new section in the tools overview documentation that explains the maximum usage count feature for tools within the CrewAI framework. Users can now set a limit on how many times a tool can be used, enhancing control over tool usage. An example of implementing the `FileReadTool` with a maximum usage count is also provided, improving the clarity and usability of the documentation.
* undo field string
* feat: add guardrail support for Agents when using direct kickoff calls
* refactor: expose guardrail func in a proper utils file
* fix: resolve Self import on python 3.10
* test: fix structured tool tests
No tests were being executed from this file
* feat: support to run async tool
Some Tool requires async execution. This commit allow us to collect tool result from coroutines
* docs: add docs about asynchronous tool support
* feat: add capability to see and expose public Tool classes
* feat: persist available Tools from repository on publish
* ci: ignore explictly templates from ruff check
Ruff only applies --exclude to files it discovers itself. So we have to skip manually the same files excluded from `ruff.toml`
* sytle: fix linter issues
* refactor: renaming available_tools_classes by available_exports
* feat: provide more context about exportable tools
* feat: allow to install a Tool from pypi
* test: fix tests
* feat: add env_vars attribute to BaseTool
* remove TODO: security check since we are handle that on enterprise side
* Add usage limit feature to BaseTool class
- Add max_usage_count and current_usage_count attributes to BaseTool
- Implement usage limit checking in ToolUsage._use method
- Add comprehensive tests for usage limit functionality
- Maintain backward compatibility with None default for unlimited usage
Co-Authored-By: Joe Moura <joao@crewai.com>
* Fix CI failures and address code review feedback
- Add max_usage_count/current_usage_count to CrewStructuredTool
- Add input validation for positive max_usage_count
- Add reset_usage_count method to BaseTool
- Extract usage limit check into separate method
- Add comprehensive edge case tests
- Add proper type hints throughout
- Fix linting issues
Co-Authored-By: Joe Moura <joao@crewai.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Joe Moura <joao@crewai.com>
Add `__init__.py` files to 20 directories to conform with Python package standards. This ensures directories are properly recognized as packages, enabling cleaner imports.
* WIP
* WIP
* wip
* wip
* WIP
* More WIP
* Its working but needs a massive clean up
* output type works now
* Usage metrics fixed
* more testing
* WIP
* cleaning up
* Update logger
* 99% done. Need to make docs match new example
* cleanup
* drop hard coded examples
* docs
* Clean up
* Fix errors
* Trying to fix CI issues
* more type checker fixes
* More type checking fixes
* Update LiteAgent documentation for clarity and consistency; replace WebsiteSearchTool with SerperDevTool, and improve formatting in examples.
* fix fingerprinting issues
* fix type-checker
* Fix type-checker issue by adding type ignore comment for cache read in ToolUsage class
* Add optional agent parameter to CrewAgentParser and enhance action handling logic
* Remove unused parameters from ToolUsage instantiation in tests and clean up debug print statement in CrewAgentParser.
* Remove deprecated test files and examples for LiteAgent; add comprehensive tests for LiteAgent functionality, including tool usage and structured output handling.
* Remove unused variable 'result' from ToolUsage class to clean up code.
* Add initialization for 'result' variable in ToolUsage class to resolve type-checker warnings
* Refactor agent_utils.py by removing unused event imports and adding missing commas in function definitions. Update test_events.py to reflect changes in expected event counts and adjust assertions accordingly. Modify test_tools_emits_error_events.yaml to include new headers and update response content for consistency with recent API changes.
* Enhance tests in crew_test.py by verifying cache behavior in test_tools_with_custom_caching and ensuring proper agent initialization with added commas in test_crew_kickoff_for_each_works_with_manager_agent_copy.
* Update agent tests to reflect changes in expected call counts and improve response formatting in YAML cassette. Adjusted mock call count from 2 to 3 and refined interaction formats for clarity and consistency.
* Refactor agent tests to update model versions and improve response formatting in YAML cassettes. Changed model references from 'o1-preview' to 'o3-mini' and adjusted interaction formats for consistency. Enhanced error handling in context length tests and refined mock setups for better clarity.
* Update tool usage logging to ensure tool arguments are consistently formatted as strings. Adjust agent test cases to reflect changes in maximum iterations and expected outputs, enhancing clarity in assertions. Update YAML cassettes to align with new response formats and improve overall consistency across tests.
* Update YAML cassette for LLM tests to reflect changes in response structure and model version. Adjusted request and response headers, including updated content length and user agent. Enhanced token limits and request counts for improved testing accuracy.
* Update tool usage logging to store tool arguments as native types instead of strings, enhancing data integrity and usability.
* Refactor agent tests by removing outdated test cases and updating YAML cassettes to reflect changes in tool usage and response formats. Adjusted request and response headers, including user agent and content length, for improved accuracy in testing. Enhanced interaction formats for consistency across tests.
* Add Excalidraw diagram file for visual representation of input-output flow
Created a new Excalidraw file that includes a diagram illustrating the input box, database, and output box with connecting arrows. This visual aid enhances understanding of the data flow within the application.
* Remove redundant error handling for action and final answer in CrewAgentParser. Update tests to reflect this change by deleting the corresponding test case.
---------
Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com>
Co-authored-by: Lorenze Jay <lorenzejaytech@gmail.com>
* Revert "feat: add prompt observability code (#2027)"
This reverts commit 90f1bee602.
* Fix issues with flows post merge
* Decoupling telemetry and ensure tests (#2212)
* feat: Enhance event listener and telemetry tracking
- Update event listener to improve telemetry span handling
- Add execution_span field to Task for better tracing
- Modify event handling in EventListener to use new span tracking
- Remove debug print statements
- Improve test coverage for crew and flow events
- Update cassettes to reflect new event tracking behavior
* Remove telemetry references from Crew class
- Remove Telemetry import and initialization from Crew class
- Delete _telemetry attribute from class configuration
- Clean up unused telemetry-related code
* test: Improve crew verbose output test with event log filtering
- Filter out event listener logs in verbose output test
- Ensure no output when verbose is set to False
- Enhance test coverage for crew logging behavior
* dropped comment
* refactor: Improve telemetry span tracking in EventListener
- Remove `execution_span` from Task class
- Add `execution_spans` dictionary to EventListener to track spans
- Update task event handlers to use new span tracking mechanism
- Simplify span management across task lifecycle events
* lint
* Fix failing test
---------
Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com>
* WIP crew events emitter
* Refactor event handling and introduce new event types
- Migrate from global `emit` function to `event_bus.emit`
- Add new event types for task failures, tool usage, and agent execution
- Update event listeners and event bus to support more granular event tracking
- Remove deprecated event emission methods
- Improve event type consistency and add more detailed event information
* Add event emission for agent execution lifecycle
- Emit AgentExecutionStarted and AgentExecutionError events
- Update CrewAgentExecutor to use event_bus for tracking agent execution
- Refactor error handling to include event emission
- Minor code formatting improvements in task.py and crew_agent_executor.py
- Fix a typo in test file
* Refactor event system and add third-party event listeners
- Move event_bus import to correct module paths
- Introduce BaseEventListener abstract base class
- Add AgentOpsListener for third-party event tracking
- Update event listener initialization and setup
- Clean up event-related imports and exports
* Enhance event system type safety and error handling
- Improve type annotations for event bus and event types
- Add null checks for agent and task in event emissions
- Update import paths for base tool and base agent
- Refactor event listener type hints
- Remove unnecessary print statements
- Update test configurations to match new event handling
* Refactor event classes to improve type safety and naming consistency
- Rename event classes to have explicit 'Event' suffix (e.g., TaskStartedEvent)
- Update import statements and references across multiple files
- Remove deprecated events.py module
- Enhance event type hints and configurations
- Clean up unnecessary event-related code
* Add default model for CrewEvaluator and fix event import order
- Set default model to "gpt-4o-mini" in CrewEvaluator when no model is specified
- Reorder event-related imports in task.py to follow standard import conventions
- Update event bus initialization method return type hint
- Export event_bus in events/__init__.py
* Fix tool usage and event import handling
- Update tool usage to use `.get()` method when checking tool name
- Remove unnecessary `__all__` export list in events/__init__.py
* Refactor Flow and Agent event handling to use event_bus
- Remove `event_emitter` from Flow class and replace with `event_bus.emit()`
- Update Flow and Agent tests to use event_bus event listeners
- Remove redundant event emissions in Flow methods
- Add debug print statements in Flow execution
- Simplify event tracking in test cases
* Enhance event handling for Crew, Task, and Event classes
- Add crew name to failed event types (CrewKickoffFailedEvent, CrewTrainFailedEvent, CrewTestFailedEvent)
- Update Task events to remove redundant task and context attributes
- Refactor EventListener to use Logger for consistent event logging
- Add new event types for Crew train and test events
- Improve event bus event tracking in test cases
* Remove telemetry and tracing dependencies from Task and Flow classes
- Remove telemetry-related imports and private attributes from Task class
- Remove `_telemetry` attribute from Flow class
- Update event handling to emit events without direct telemetry tracking
- Simplify task and flow execution by removing explicit telemetry spans
- Move telemetry-related event handling to EventListener
* Clean up unused imports and event-related code
- Remove unused imports from various event and flow-related files
- Reorder event imports to follow standard conventions
- Remove unnecessary event type references
- Simplify import statements in event and flow modules
* Update crew test to validate verbose output and kickoff_for_each method
- Enhance test_crew_verbose_output to check specific listener log messages
- Modify test_kickoff_for_each_invalid_input to use Pydantic validation error
- Improve test coverage for crew logging and input validation
* Update crew test verbose output with improved emoji icons
- Replace task and agent completion icons from 👍 to ✅
- Enhance readability of test output logging
- Maintain consistent test coverage for crew verbose output
* Add MethodExecutionFailedEvent to handle flow method execution failures
- Introduce new MethodExecutionFailedEvent in flow_events module
- Update Flow class to catch and emit method execution failures
- Add event listener for method execution failure events
- Update event-related imports to include new event type
- Enhance test coverage for method execution failure handling
* Propagate method execution failures in Flow class
- Modify Flow class to re-raise exceptions after emitting MethodExecutionFailedEvent
- Reorder MethodExecutionFailedEvent import to maintain consistent import style
* Enable test coverage for Flow method execution failure event
- Uncomment pytest.raises() in test_events to verify exception handling
- Ensure test validates MethodExecutionFailedEvent emission during flow kickoff
* Add event handling for tool usage events
- Introduce event listeners for ToolUsageFinishedEvent and ToolUsageErrorEvent
- Log tool usage events with descriptive emoji icons (✅ and ❌)
- Update event_listener to track and log tool usage lifecycle
* Reorder and clean up event imports in event_listener
- Reorganize imports for tool usage events and other event types
- Maintain consistent import ordering and remove unused imports
- Ensure clean and organized import structure in event_listener module
* moving to dedicated eventlistener
* dont forget crew level
* Refactor AgentOps event listener for crew-level tracking
- Modify AgentOpsListener to handle crew-level events
- Initialize and end AgentOps session at crew kickoff and completion
- Create agents for each crew member during session initialization
- Improve session management and event recording
- Clean up and simplify event handling logic
* Update test_events to validate tool usage error event handling
- Modify test to assert single error event with correct attributes
- Use pytest.raises() to verify error event generation
- Simplify error event validation in test case
* Improve AgentOps listener type hints and formatting
- Add string type hints for AgentOps classes to resolve potential import issues
- Clean up unnecessary whitespace and improve code indentation
- Simplify initialization and event handling logic
* Update test_events to validate multiple tool usage events
- Modify test to assert 75 events instead of a single error event
- Remove pytest.raises() check, allowing crew kickoff to complete
- Adjust event validation to support broader event tracking
* Rename event_bus to crewai_event_bus for improved clarity and specificity
- Replace all references to `event_bus` with `crewai_event_bus`
- Update import statements across multiple files
- Remove the old `event_bus.py` file
- Maintain existing event handling functionality
* Enhance EventListener with singleton pattern and color configuration
- Implement singleton pattern for EventListener to ensure single instance
- Add default color configuration using EMITTER_COLOR from constants
- Modify log method calls to use default color and remove redundant color parameters
- Improve initialization logic to prevent multiple initializations
* Add FlowPlotEvent and update event bus to support flow plotting
- Introduce FlowPlotEvent to track flow plotting events
- Replace Telemetry method with event bus emission in Flow.plot()
- Update event bus to support new FlowPlotEvent type
- Add test case to validate flow plotting event emission
* Remove RunType enum and clean up crew events module
- Delete unused RunType enum from crew_events.py
- Simplify crew_events.py by removing unnecessary enum definition
- Improve code clarity by removing unneeded imports
* Enhance event handling for tool usage and agent execution
- Add new events for tool usage: ToolSelectionErrorEvent, ToolValidateInputErrorEvent
- Improve error tracking and event emission in ToolUsage and LLM classes
- Update AgentExecutionStartedEvent to use task_prompt instead of inputs
- Add comprehensive test coverage for new event types and error scenarios
* Refactor event system and improve crew testing
- Extract base CrewEvent class to a new base_events.py module
- Update event imports across multiple event-related files
- Modify CrewTestStartedEvent to use eval_llm instead of openai_model_name
- Add LLM creation validation in crew testing method
- Improve type handling and event consistency
* Refactor task events to use base CrewEvent
- Move CrewEvent import from crew_events to base_events
- Remove unnecessary blank lines in task_events.py
- Simplify event class structure for task-related events
* Update AgentExecutionStartedEvent to use task_prompt
- Modify test_events.py to use task_prompt instead of inputs
- Simplify event input validation in test case
- Align with recent event system refactoring
* Improve type hinting for TaskCompletedEvent handler
- Add explicit type annotation for TaskCompletedEvent in event_listener.py
- Enhance type safety for event handling in EventListener
* Improve test_validate_tool_input_invalid_input with mock objects
- Add explicit mock objects for agent and action in test case
- Ensure proper string values for mock agent and action attributes
- Simplify test setup for ToolUsage validation method
* Remove ToolUsageStartedEvent emission in tool usage process
- Remove unnecessary event emission for tool usage start
- Simplify tool usage event handling
- Eliminate redundant event data preparation step
* refactor: clean up and organize imports in llm and flow modules
* test: Improve flow persistence test cases and logging
* clean up. fix type safety. address memory config docs
* improve manager
* Include fix for o1 models not supporting system messages
* more broad with o1
* address fix: Typo in expected_output string #2045
* drop prints
* drop prints
* wip
* wip
* fix failing memory tests
* Fix memory provider issue
* clean up short term memory
* revert ltm
* drop
* Improving tool calling to pass dictionaries instead of strings
* Fix issues with parsing none/null
* remove prints and unnecessary comments
* Fix crew_test issues with function calling
* improve prompting
* add back in support for add_image
* add tests for tool validation
* revert back to figure out why tests are timing out
* Update cassette
* trying to find what is timing out
* add back in guardrails
* add back in manager delegation tests
* Trying to fix tests
* Force test to pass
* Trying to fix tests
* add in more role tests
* add back old tool validation
* updating tests
* vcr
* Fix tests
* improve function llm logic
* vcr 2
* drop llm
* Failing test
* add more tests back in
* Revert tool validation
* fix(manager_llm): handle coworker role name case/whitespace properly
- Add .strip() to agent name and role comparisons in base_agent_tools.py
- Add test case for varied role name cases and whitespace
- Fix issue #1503 with manager LLM delegation
Co-Authored-By: Joe Moura <joao@crewai.com>
* fix(manager_llm): improve error handling and add debug logging
- Add debug logging for better observability
- Add sanitize_agent_name helper method
- Enhance error messages with more context
- Add parameterized tests for edge cases:
- Embedded quotes
- Trailing newlines
- Multiple whitespace
- Case variations
- None values
- Improve error handling with specific exceptions
Co-Authored-By: Joe Moura <joao@crewai.com>
* style: fix import sorting in base_agent_tools and test_manager_llm_delegation
Co-Authored-By: Joe Moura <joao@crewai.com>
* fix(manager_llm): improve whitespace normalization in role name matching
Co-Authored-By: Joe Moura <joao@crewai.com>
* style: fix import sorting in base_agent_tools and test_manager_llm_delegation
Co-Authored-By: Joe Moura <joao@crewai.com>
* fix(manager_llm): add error message template for agent tool execution errors
Co-Authored-By: Joe Moura <joao@crewai.com>
* style: fix import sorting in test_manager_llm_delegation.py
Co-Authored-By: Joe Moura <joao@crewai.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Joe Moura <joao@crewai.com>
* apply agent ops changes and resolve merge conflicts
* Trying to fix tests
* add back in vcr
* update tools
* remove pkg_resources which was causing issues
* Fix tests
* experimenting to see if unique content is an issue with knowledge
* experimenting to see if unique content is an issue with knowledge
* update chromadb which seems to have issues with upsert
* generate new yaml for failing test
* Investigating upsert
* Drop patch
* Update casettes
* Fix duplicate document issue
* more fixes
* add back in vcr
* new cassette for test
---------
Co-authored-by: Lorenze Jay <lorenzejaytech@gmail.com>
* rebuilding executor
* removing langchain
* Making all tests good
* fixing types and adding ability for nor using system prompts
* improving types
* pleasing the types gods
* pleasing the types gods
* fixing parser, tools and executor
* making sure all tests pass
* final pass
* fixing type
* Updating Docs
* preparing to cut new version
