- update imports and include handling for chromadb v1.1.0
- fix mypy and typing_compat issues (required, typeddict, voyageai)
- refine embedderconfig typing and allow base provider instances
- handle mem0 as special case for external memory storage
- bump tools and clean up redundant deps
- add batch_size field to baseragconfig (default=100)
- update chromadb/qdrant clients and factories to use batch_size
- extract and filter batch_size from embedder config in knowledgestorage
- fix large csv files exceeding embedder token limits (#3574)
- remove unneeded conditional for type
Co-authored-by: Vini Brasil <vini@hey.com>
- Sanitize ChromaDB collection names and use original dir naming
- Add persistent client with file locking to the ChromaDB factory
- Add upsert support to the ChromaDB client
- Suppress ChromaDB deprecation warnings for `model_fields`
- Extract `suppress_logging` into shared `logger_utils`
- Update tests to reflect upsert behavior
- Docs: add additional note
### RAG Config System
* Added ChromaDB client creation via config with sensible defaults
* Introduced optional imports and shared RAG config utilities/schema
* Enabled embedding function support with ChromaDB provider integration
* Refactored configs for immutability and stronger type safety
* Removed unused code and expanded test coverage
Add ChromaDB client implementation with async support
- Implement core collection operations (create, get_or_create, delete)
- Add search functionality with cosine similarity scoring
- Include both sync and async method variants
- Add type safety with NamedTuples and TypeGuards
- Extract utility functions to separate modules
- Default to cosine distance metric for text similarity
- Add comprehensive test coverage
TODO:
- l2, ip score calculations are not settled on
