Fixes#4145
The HuggingFace embedder was failing with 'could not convert string to float: error'
because chromadb's HuggingFaceEmbeddingFunction uses the deprecated
api-inference.huggingface.co endpoint which returns error messages instead of embeddings.
This fix creates a custom HuggingFaceEmbeddingFunction that uses huggingface_hub's
InferenceClient with provider='hf-inference' instead of the deprecated endpoint.
Changes:
- Add custom embedding_callable.py using huggingface_hub.InferenceClient
- Update HuggingFaceProvider to use the new embedding callable
- Handle different embedding response formats (1D, 2D, 3D arrays)
- Add comprehensive error handling with actionable error messages
- Add 16 test cases covering initialization, embedding generation, error handling,
and ChromaDB integration
Co-Authored-By: João <joao@crewai.com>
9460e5e182