- Replace agent.role with agent.id (UUID) in RAGStorage directory naming
- UUIDs are guaranteed short (36 chars) and filesystem-safe
- Remove unused _sanitize_role method since UUIDs don't need sanitization
- Add comprehensive tests for Windows path length scenarios
- Add test case for long agent roles in existing memory tests
- Maintains backward compatibility while fixing Windows 260-char limit
Fixes#3236
Co-Authored-By: João <joao@crewai.com>
* Changed v1.1 -> v2
* Fixed Test Cases:
* Fixed linting issues
* Changed docs
* Refractored the storage
* Fixed test cases
* Fixing run-time checks
* Fixed Test Case
* Updated docs and added test case for custom categories
* Add the TODO back
* Minor Changes
* Added output_format in search
* Minor changes
* Added output_format and version in both search and save
* Small change
* Minor bugs
* Fixed test cases
* Changed docs
---------
Co-authored-by: Lucas Gomide <lucaslg200@gmail.com>
Replaced the try-except block for collection retrieval with a single call to get_or_create_collection, simplifying the code and improving readability. Added logging to confirm whether the collection was found or created.
Add `__init__.py` files to 20 directories to conform with Python package standards. This ensures directories are properly recognized as packages, enabling cleaner imports.
* fix: use mem0_local_config instead of config in Memory.from_config (#2587)
Co-Authored-By: Joe Moura <joao@crewai.com>
* refactor: consolidate tests as per PR feedback
Co-Authored-By: Joe Moura <joao@crewai.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Joe Moura <joao@crewai.com>
* fix: surfacing properly supported types by Mem0Storage
* feat: prepare Mem0Storage to accept config paramenter
We're planning to remove `memory_config` soon. This commit kindly prepare this storage to accept the config provided directly
* feat: add external memory
* fix: cleanup Mem0 warning while adding messages to the memory
* feat: support set the current crew in memory
This can be useful when a memory is initialized before the crew, but the crew might still be a very relevant attribute
* fix: allow to reset only an external_memory from crew
* test: add external memory test
* test: ensure the config takes precedence over memory_config when setting mem0
* fix: support to provide a custom storage to External Memory
* docs: add docs about external memory
* chore: add warning messages about the deprecation of UserMemory
* fix: fix typing check
---------
Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com>
* Enhance embedding configuration with custom embedder support
- Add support for custom embedding functions in EmbeddingConfigurator
- Update type hints for embedder configuration
- Extend configuration options for various embedding providers
- Add optional embedder configuration to Memory class
* added docs
* Refine custom embedder configuration support
- Update custom embedder configuration method to handle custom embedding functions
- Modify type hints for embedder configuration
- Remove unused model_name parameter in custom embedder configuration
* fix: ensure persisted state overrides class defaults
- Remove early return in Flow.__init__ to allow proper state initialization
- Add test_flow_default_override.py to verify state override behavior
- Fix issue where default values weren't being overridden by persisted state
Fixes the issue where persisted state values weren't properly overriding
class defaults when restarting a flow with a previously saved state ID.
Co-Authored-By: Joe Moura <joao@crewai.com>
* test: improve state restoration verification with has_set_count flag
Co-Authored-By: Joe Moura <joao@crewai.com>
* test: add has_set_count field to PoemState
Co-Authored-By: Joe Moura <joao@crewai.com>
* refactoring test
* fix: ensure persisted state overrides class defaults
- Remove early return in Flow.__init__ to allow proper state initialization
- Add test_flow_default_override.py to verify state override behavior
- Fix issue where default values weren't being overridden by persisted state
Fixes the issue where persisted state values weren't properly overriding
class defaults when restarting a flow with a previously saved state ID.
Co-Authored-By: Joe Moura <joao@crewai.com>
* test: improve state restoration verification with has_set_count flag
Co-Authored-By: Joe Moura <joao@crewai.com>
* test: add has_set_count field to PoemState
Co-Authored-By: Joe Moura <joao@crewai.com>
* refactoring test
* Fixing flow state
* fixing peristed stateful flows
* linter
* type fix
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Joe Moura <joao@crewai.com>
* Fix SQLite log handling issue causing ValueError: Logs cannot be None in tests
- Add proper error handling in SQLite storage operations
- Set up isolated test environment with temporary storage directory
- Ensure consistent error messages across all database operations
Co-Authored-By: Joe Moura <joao@crewai.com>
* fix: Sort imports in conftest.py
Co-Authored-By: Joe Moura <joao@crewai.com>
* fix: Convert TokenProcess counters to instance variables to fix callback tracking
Co-Authored-By: Joe Moura <joao@crewai.com>
* refactor: Replace print statements with logging and improve error handling
- Add proper logging setup in kickoff_task_outputs_storage.py
- Replace self._printer.print() with logger calls
- Use appropriate log levels (error/warning)
- Add directory validation in test environment setup
- Maintain consistent error messages with DatabaseError format
Co-Authored-By: Joe Moura <joao@crewai.com>
* fix: Comprehensive improvements to database and token handling
- Fix SQLite database path handling in storage classes
- Add proper directory creation and error handling
- Improve token tracking with robust type checking
- Convert TokenProcess counters to instance variables
- Add standardized database error handling
- Set up isolated test environment with temporary storage
Resolves test failures in PR #1899
Co-Authored-By: Joe Moura <joao@crewai.com>
---------
Co-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>
Co-authored-by: Joe Moura <joao@crewai.com>
Co-authored-by: João Moura <joaomdmoura@gmail.com>
* docs: clarify how to specify org_id and project_id in Mem0 configuration
* Add org_id and project_id to mem0 config and fix mem0 entity '400 Bad Request'
* Remove ruff changes to docs
---------
Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com>
* Fix disk I/O error when resetting short-term memory.
Reset chromadb client and nullifies references before
removing directory.
* Nit for clarity
* did the same for knowledge_storage
* cleanup
* cleanup order
* Cleanup after the rm of the directories
---------
Co-authored-by: Lorenze Jay <lorenzejaytech@gmail.com>
Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com>
* incorporate #1683
* add in --version flag to cli. closes#1679.
* Fix env issue
* Add in suggestions from @caike to make sure ragstorage doesnt exceed os file limit. Also, included additional checks to support windows.
* remove poetry.lock as pointed out by @sanders41 in #1574.
* Incorporate feedback from crewai reviewer
* Incorporate @lorenzejay feedback
* added path to RAGStorage
* added path to short term and entity memory
* add path for long_term_storage for completeness
---------
Co-authored-by: Brandon Hancock (bhancock_ai) <109994880+bhancockio@users.noreply.github.com>
* initial knowledge
* WIP
* Adding core knowledge sources
* Improve types and better support for file paths
* added additional sources
* fix linting
* update yaml to include optional deps
* adding in lorenze feedback
* ensure embeddings are persisted
* improvements all around Knowledge class
* return this
* properly reset memory
* properly reset memory+knowledge
* consolodation and improvements
* linted
* cleanup rm unused embedder
* fix test
* fix duplicate
* generating cassettes for knowledge test
* updated default embedder
* None embedder to use default on pipeline cloning
* improvements
* fixed text_file_knowledge
* mypysrc fixes
* type check fixes
* added extra cassette
* just mocks
* linted
* mock knowledge query to not spin up db
* linted
* verbose run
* put a flag
* fix
* adding docs
* better docs
* improvements from review
* more docs
* linted
* rm print
* more fixes
* clearer docs
* added docstrings and type hints for cli
---------
Co-authored-by: João Moura <joaomdmoura@gmail.com>
Co-authored-by: Lorenze Jay <lorenzejaytech@gmail.com>
* ensure original embedding config works
* some fixes
* raise error on unsupported provider
* WIP: brandons notes
* fixes
* rm prints
* fixed docs
* fixed run types
* updates to add more docs and correct imports with huggingface embedding server enabled
---------
Co-authored-by: Brandon Hancock <brandon@brandonhancock.io>
* byom - short/entity memory
* better
* rm uneeded
* fix text
* use context
* rm dep and sync
* type check fix
* fixed test using new cassete
* fixing types
* fixed types
* fix types
* fixed types
* fixing types
* fix type
* cassette update
* just mock the return of short term mem
* remove print
* try catch block
* added docs
* dding error handling here
* WIP. Procedure appears to be working well. Working on mocking properly for tests
* All tests are passing now
* rshift working
* Add back in Gui's tool_usage fix
* WIP
* Going to start refactoring for pipeline_output
* Update terminology
* new pipeline flow with traces and usage metrics working. need to add more tests and make sure PipelineOutput behaves likew CrewOutput
* Fix pipelineoutput to look more like crewoutput and taskoutput
* Implemented additional tests for pipeline. One test is failing. Need team support
* Update docs for pipeline
* Update pipeline to properly process input and ouput dictionary
* Update Pipeline docs
* Add back in commentary at top of pipeline file
* Starting to work on router
* Drop router for now. will add in separately
* In the middle of fixing router. A ton of circular dependencies. Moving over to a new design.
* WIP.
* Fix circular dependencies and updated PipelineRouter
* Add in Eduardo feedback. Still need to add in more commentary describing the design decisions for pipeline
* Add developer notes to explain what is going on in pipelines.
* Add doc strings
* Fix missing rag datatype
* WIP. Converting usage metrics from a dict to an object
* Fix tests that were checking usage metrics
* Drop todo
* Fix 1 type error in pipeline
* Update pipeline to use UsageMetric
* Add missing doc string
* WIP.
* Change names
* Rename variables based on joaos feedback
* Fix critical circular dependency issues. Now needing to fix trace issue.
* Tests working now!
* Add more tests which showed underlying issue with traces
* Fix tests
* Remove overly complicated test
* Add router example to docs
* Clean up end of docs
* Clean up docs
* Working on creating Crew templates and pipeline templates
* WIP.
* WIP
* Fix poetry install from templates
* WIP
* Restructure
* changes for lorenze
* more todos
* WIP: create pipelines cli working
* wrapped up router
* ignore mypy src on templates
* ignored signature of copy
* fix all verbose
* rm print statements
* brought back correct folders
* fixes missing folders and then rm print statements
* fixed tests
* fixed broken test
* fixed type checker
* fixed type ignore
* ignore types for templates
* needed
* revert
* exclude only required
* rm type errors on templates
* rm excluding type checks for template files on github action
* fixed missing quotes
---------
Co-authored-by: Brandon Hancock <brandon@brandonhancock.io>
* Cleaned up task execution to now have separate paths for async and sync execution. Updating all kickoff functions to return CrewOutput. WIP. Waiting for Joao feedback on async task execution with task_output
* Consistently storing async and sync output for context
* outline tests I need to create going forward
* Major rehaul of TaskOutput and CrewOutput. Updated all tests to work with new change. Need to add in a few final tricky async tests and add a few more to verify output types on TaskOutput and CrewOutput.
* Encountering issues with callback. Need to test on main. WIP
* working on tests. WIP
* WIP. Figuring out disconnect issue.
* Cleaned up logs now that I've isolated the issue to the LLM
* more wip.
* WIP. It looks like usage metrics has always been broken for async
* Update parent crew who is managing for_each loop
* Merge in main to bugfix/kickoff-for-each-usage-metrics
* Clean up code for review
* Add new tests
* Final cleanup. Ready for review.
* Moving copy functionality from Agent to BaseAgent
* Fix renaming issue
* Fix linting errors
* use BaseAgent instead of Agent where applicable
* Fixing missing function. Working on tests.
* WIP. Needing team to review change
* Fixing issues brought about by merge
* WIP: need to fix json encoder
* WIP need to fix encoder
* WIP
* WIP: replay working with async. need to add tests
* Implement major fixes from yesterdays group conversation. Now working on tests.
* The majority of tasks are working now. Need to fix converter class
* Fix final failing test
* Fix linting and type-checker issues
* Add more tests to fully test CrewOutput and TaskOutput changes
* Add in validation for async cannot depend on other async tasks.
* WIP: working replay feat fixing inputs, need tests
* WIP: core logic of seq and heir for executing tasks added into one
* Update validators and tests
* better logic for seq and hier
* replay working for both seq and hier just need tests
* fixed context
* added cli command + code cleanup TODO: need better refactoring
* refactoring for cleaner code
* added better tests
* removed todo comments and fixed some tests
* fix logging now all tests should pass
* cleaner code
* ensure replay is delcared when replaying specific tasks
* ensure hierarchical works
* better typing for stored_outputs and separated task_output_handler
* added better tests
* added replay feature to crew docs
* easier cli command name
* fixing changes
* using sqllite instead of .json file for logging previous task_outputs
* tools fix
* added to docs and fixed tests
* fixed .db
* fixed docs and removed unneeded comments
* separating ltm and replay db
* fixed printing colors
* added how to doc
---------
Co-authored-by: Brandon Hancock <brandon@brandonhancock.io>
* fix: fix test actually running
* fix: fix test to not send request to openai
* fix: fix linting to remove cli files
* fix: exclude only files that breaks black
* fix: Fix all Ruff checkings on the code and Fix Test with repeated name
* fix: Change linter name on yml file
* feat: update pre-commit
* feat: remove need for isort on the code
* feat: add mypy as type checker, update code and add comment to reference
* feat: remove black linter
* feat: remove poetry to run the command
* feat: change logic to test mypy
* feat: update tests yml to try to fix the tests gh action
* feat: try to add just mypy to run on gh action
* feat: fix yml file
* feat: add comment to avoid issue on gh action
* feat: decouple pytest from the necessity of poetry install
* feat: change tests.yml to test different approach
* feat: change to poetry run
* fix: parameter field on yml file
* fix: update parameters to be on the pyproject
* fix: update pyproject to remove import untyped errors
