feat: async llm support
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
Notify Downstream / notify-downstream (push) Has been cancelled
Build uv cache / build-cache (3.10) (push) Has been cancelled
Build uv cache / build-cache (3.11) (push) Has been cancelled
Build uv cache / build-cache (3.12) (push) Has been cancelled
Build uv cache / build-cache (3.13) (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled

feat: introduce async contract to BaseLLM

feat: add async call support for:

Azure provider

Anthropic provider

OpenAI provider

Gemini provider

Bedrock provider

LiteLLM provider

chore: expand scrubbed header fields (conftest, anthropic, bedrock)

chore: update docs to cover async functionality

chore: update and harden tests to support acall; re-add uri for cassette compatibility

chore: generate missing cassette

fix: ensure acall is non-abstract and set supports_tools = true for supported Anthropic models

chore: improve Bedrock async docstring and general test robustness
This commit is contained in:
Greyson LaLonde
2025-12-01 18:56:56 -05:00
committed by GitHub
parent 59180e9c9f
commit 20704742e2
70 changed files with 8586 additions and 151 deletions

View File

@@ -77,6 +77,9 @@ class CustomLLM(BaseLLM):
"""
return 4096
async def acall(self, messages, tools=None, callbacks=None, available_functions=None, from_task=None, from_agent=None, response_model=None):
raise NotImplementedError
@pytest.mark.vcr()
def test_custom_llm_implementation():
@@ -192,6 +195,9 @@ class JWTAuthLLM(BaseLLM):
"""Return a default context window size."""
return 8192
async def acall(self, messages, tools=None, callbacks=None, available_functions=None, from_task=None, from_agent=None, response_model=None):
raise NotImplementedError
def test_custom_llm_with_jwt_auth():
"""Test a custom LLM implementation with JWT authentication."""
@@ -339,6 +345,9 @@ class TimeoutHandlingLLM(BaseLLM):
"""
return 8192
async def acall(self, messages, tools=None, callbacks=None, available_functions=None, from_task=None, from_agent=None, response_model=None):
raise NotImplementedError
def test_timeout_handling_llm():
"""Test a custom LLM implementation with timeout handling and retry logic."""