andre/crewAI
1
0
Fork 0
mirror of https://github.com/crewAIInc/crewAI.git synced 2026-05-04 00:32:36 +00:00
Code Issues Actions 5 Packages Projects Releases Wiki Activity

feat: async llm support
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
Details
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Details
Check Documentation Broken Links / Check broken links (push) Has been cancelled
Details
Notify Downstream / notify-downstream (push) Has been cancelled
Details
Build uv cache / build-cache (3.10) (push) Has been cancelled
Details
Build uv cache / build-cache (3.11) (push) Has been cancelled
Details
Build uv cache / build-cache (3.12) (push) Has been cancelled
Details
Build uv cache / build-cache (3.13) (push) Has been cancelled
Details
Mark stale issues and pull requests / stale (push) Has been cancelled
Details

Browse Source 
feat: introduce async contract to BaseLLM

feat: add async call support for:

Azure provider

Anthropic provider

OpenAI provider

Gemini provider

Bedrock provider

LiteLLM provider

chore: expand scrubbed header fields (conftest, anthropic, bedrock)

chore: update docs to cover async functionality

chore: update and harden tests to support acall; re-add uri for cassette compatibility

chore: generate missing cassette

fix: ensure acall is non-abstract and set supports_tools = true for supported Anthropic models

chore: improve Bedrock async docstring and general test robustness
This commit is contained in:
Greyson LaLonde
2025-12-01 18:56:56 -05:00
committed by GitHub
parent 59180e9c9f
commit 20704742e2
70 changed files with 8586 additions and 151 deletions
Download Patch File Download Diff File Expand all files Collapse all files

42
lib/crewai/tests/cassettes/llms/bedrock/test_bedrock_async_basic_call.yaml Normal file
View File

@@ -0,0 +1,42 @@
interactions:
- request:
body: '{"messages": [{"role": "user", "content": [{"text": "Say hello"}]}], "inferenceConfig":
{}}'
headers:
Content-Length:
- '91'
Content-Type:
- application/json
User-Agent:
- X-USER-AGENT-XXX
accept-encoding:
- ACCEPT-ENCODING-XXX
amz-sdk-invocation-id:
- AMZ-SDK-INVOCATION-ID-XXX
amz-sdk-request:
- attempt=1
authorization:
- AUTHORIZATION-XXX
x-amz-date:
- X-AMZ-DATE-XXX
method: POST
uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.anthropic.claude-3-5-sonnet-20241022-v2%3A0/converse
response:
body:
string: '{"metrics":{"latencyMs":776},"output":{"message":{"content":[{"text":"Hello!
How are you today?"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":9,"outputTokens":10,"serverToolUsage":{},"totalTokens":19}}'
headers:
Connection:
- keep-alive
Content-Length:
- '226'
Content-Type:
- application/json
Date:
- Mon, 01 Dec 2025 08:50:59 GMT
x-amzn-RequestId:
- X-AMZN-REQUESTID-XXX
status:
code: 200
message: OK
version: 1

44
lib/crewai/tests/cassettes/llms/bedrock/test_bedrock_async_conversation.yaml Normal file
View File

@@ -0,0 +1,44 @@
interactions:
- request:
body: '{"messages": [{"role": "user", "content": [{"text": "My name is Alice."}]},
{"role": "assistant", "content": [{"text": "Hello Alice! Nice to meet you."}]},
{"role": "user", "content": [{"text": "What is my name?"}]}], "inferenceConfig":
{}}'
headers:
Content-Length:
- '240'
Content-Type:
- application/json
User-Agent:
- X-USER-AGENT-XXX
accept-encoding:
- ACCEPT-ENCODING-XXX
amz-sdk-invocation-id:
- AMZ-SDK-INVOCATION-ID-XXX
amz-sdk-request:
- attempt=1
authorization:
- AUTHORIZATION-XXX
x-amz-date:
- X-AMZ-DATE-XXX
method: POST
uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.anthropic.claude-3-5-sonnet-20241022-v2%3A0/converse
response:
body:
string: '{"metrics":{"latencyMs":622},"output":{"message":{"content":[{"text":"Your
name is Alice."}],"role":"assistant"}},"stopReason":"end_turn","usage":{"cacheReadInputTokenCount":0,"cacheReadInputTokens":0,"cacheWriteInputTokenCount":0,"cacheWriteInputTokens":0,"inputTokens":31,"outputTokens":8,"serverToolUsage":{},"totalTokens":39}}'
headers:
Connection:
- keep-alive
Content-Length:
- '330'
Content-Type:
- application/json
Date:
- Mon, 01 Dec 2025 08:51:04 GMT
x-amzn-RequestId:
- X-AMZN-REQUESTID-XXX
status:
code: 200
message: OK
version: 1

80
lib/crewai/tests/cassettes/llms/bedrock/test_bedrock_async_multiple_calls.yaml Normal file
View File

@@ -0,0 +1,80 @@
interactions:
- request:
body: '{"messages": [{"role": "user", "content": [{"text": "What is 1+1?"}]}],
"inferenceConfig": {}}'
headers:
Content-Length:
- '94'
Content-Type:
- application/json
User-Agent:
- X-USER-AGENT-XXX
accept-encoding:
- ACCEPT-ENCODING-XXX
amz-sdk-invocation-id:
- AMZ-SDK-INVOCATION-ID-XXX
amz-sdk-request:
- attempt=1
authorization:
- AUTHORIZATION-XXX
x-amz-date:
- X-AMZ-DATE-XXX
method: POST
uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.anthropic.claude-3-5-sonnet-20241022-v2%3A0/converse
response:
body:
string: '{"metrics":{"latencyMs":583},"output":{"message":{"content":[{"text":"1+1=2"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":14,"outputTokens":9,"serverToolUsage":{},"totalTokens":23}}'
headers:
Connection:
- keep-alive
Content-Length:
- '206'
Content-Type:
- application/json
Date:
- Mon, 01 Dec 2025 08:51:00 GMT
x-amzn-RequestId:
- X-AMZN-REQUESTID-XXX
status:
code: 200
message: OK
- request:
body: '{"messages": [{"role": "user", "content": [{"text": "What is 2+2?"}]}],
"inferenceConfig": {}}'
headers:
Content-Length:
- '94'
Content-Type:
- application/json
User-Agent:
- X-USER-AGENT-XXX
accept-encoding:
- ACCEPT-ENCODING-XXX
amz-sdk-invocation-id:
- AMZ-SDK-INVOCATION-ID-XXX
amz-sdk-request:
- attempt=1
authorization:
- AUTHORIZATION-XXX
x-amz-date:
- X-AMZ-DATE-XXX
method: POST
uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.anthropic.claude-3-5-sonnet-20241022-v2%3A0/converse
response:
body:
string: '{"metrics":{"latencyMs":869},"output":{"message":{"content":[{"text":"2+2=4"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":14,"outputTokens":9,"serverToolUsage":{},"totalTokens":23}}'
headers:
Connection:
- keep-alive
Content-Length:
- '206'
Content-Type:
- application/json
Date:
- Mon, 01 Dec 2025 08:51:01 GMT
x-amzn-RequestId:
- X-AMZN-REQUESTID-XXX
status:
code: 200
message: OK
version: 1

42
lib/crewai/tests/cassettes/llms/bedrock/test_bedrock_async_with_max_tokens.yaml Normal file
View File

@@ -0,0 +1,42 @@
interactions:
- request:
body: '{"messages": [{"role": "user", "content": [{"text": "Write a very long
story about a dragon."}]}], "inferenceConfig": {"maxTokens": 10}}'
headers:
Content-Length:
- '136'
Content-Type:
- application/json
User-Agent:
- X-USER-AGENT-XXX
accept-encoding:
- ACCEPT-ENCODING-XXX
amz-sdk-invocation-id:
- AMZ-SDK-INVOCATION-ID-XXX
amz-sdk-request:
- attempt=1
authorization:
- AUTHORIZATION-XXX
x-amz-date:
- X-AMZ-DATE-XXX
method: POST
uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.anthropic.claude-3-5-sonnet-20241022-v2%3A0/converse
response:
body:
string: '{"metrics":{"latencyMs":966},"output":{"message":{"content":[{"text":"Here''s
a long story about a dragon:"}],"role":"assistant"}},"stopReason":"max_tokens","usage":{"inputTokens":16,"outputTokens":10,"serverToolUsage":{},"totalTokens":26}}'
headers:
Connection:
- keep-alive
Content-Length:
- '239'
Content-Type:
- application/json
Date:
- Mon, 01 Dec 2025 08:50:58 GMT
x-amzn-RequestId:
- X-AMZN-REQUESTID-XXX
status:
code: 200
message: OK
version: 1

43
lib/crewai/tests/cassettes/llms/bedrock/test_bedrock_async_with_parameters.yaml Normal file
View File

@@ -0,0 +1,43 @@
interactions:
- request:
body: '{"messages": [{"role": "user", "content": [{"text": "Tell me a short fact"}]}],
"inferenceConfig": {"maxTokens": 100, "temperature": 0.7, "topP": 0.9}}'
headers:
Content-Length:
- '151'
Content-Type:
- application/json
User-Agent:
- X-USER-AGENT-XXX
accept-encoding:
- ACCEPT-ENCODING-XXX
amz-sdk-invocation-id:
- AMZ-SDK-INVOCATION-ID-XXX
amz-sdk-request:
- attempt=1
authorization:
- AUTHORIZATION-XXX
x-amz-date:
- X-AMZ-DATE-XXX
method: POST
uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.anthropic.claude-3-5-sonnet-20241022-v2%3A0/converse
response:
body:
string: '{"metrics":{"latencyMs":1360},"output":{"message":{"content":[{"text":"Here''s
a short fact: Honeybees can recognize human faces by learning and remembering
facial features, similar to how we do."}],"role":"assistant"}},"stopReason":"end_turn","usage":{"cacheReadInputTokenCount":0,"cacheReadInputTokens":0,"cacheWriteInputTokenCount":0,"cacheWriteInputTokens":0,"inputTokens":12,"outputTokens":31,"serverToolUsage":{},"totalTokens":43}}'
headers:
Connection:
- keep-alive
Content-Length:
- '436'
Content-Type:
- application/json
Date:
- Mon, 01 Dec 2025 08:50:57 GMT
x-amzn-RequestId:
- X-AMZN-REQUESTID-XXX
status:
code: 200
message: OK
version: 1

42
lib/crewai/tests/cassettes/llms/bedrock/test_bedrock_async_with_system_message.yaml Normal file
View File

@@ -0,0 +1,42 @@
interactions:
- request:
body: '{"messages": [{"role": "user", "content": [{"text": "What is 2+2?"}]}],
"inferenceConfig": {}, "system": [{"text": "You are a helpful assistant."}]}'
headers:
Content-Length:
- '148'
Content-Type:
- application/json
User-Agent:
- X-USER-AGENT-XXX
accept-encoding:
- ACCEPT-ENCODING-XXX
amz-sdk-invocation-id:
- AMZ-SDK-INVOCATION-ID-XXX
amz-sdk-request:
- attempt=1
authorization:
- AUTHORIZATION-XXX
x-amz-date:
- X-AMZ-DATE-XXX
method: POST
uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.anthropic.claude-3-5-sonnet-20241022-v2%3A0/converse
response:
body:
string: '{"metrics":{"latencyMs":677},"output":{"message":{"content":[{"text":"2
+ 2 = 4"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":20,"outputTokens":13,"serverToolUsage":{},"totalTokens":33}}'
headers:
Connection:
- keep-alive
Content-Length:
- '211'
Content-Type:
- application/json
Date:
- Mon, 01 Dec 2025 08:51:03 GMT
x-amzn-RequestId:
- X-AMZN-REQUESTID-XXX
status:
code: 200
message: OK
version: 1

41
lib/crewai/tests/cassettes/llms/bedrock/test_bedrock_async_with_temperature.yaml Normal file
View File

@@ -0,0 +1,41 @@
interactions:
- request:
body: '{"messages": [{"role": "user", "content": [{"text": "Say the word ''test''
once"}]}], "inferenceConfig": {"temperature": 0.1}}'
headers:
Content-Length:
- '124'
Content-Type:
- application/json
User-Agent:
- X-USER-AGENT-XXX
accept-encoding:
- ACCEPT-ENCODING-XXX
amz-sdk-invocation-id:
- AMZ-SDK-INVOCATION-ID-XXX
amz-sdk-request:
- attempt=1
authorization:
- AUTHORIZATION-XXX
x-amz-date:
- X-AMZ-DATE-XXX
method: POST
uri: https://bedrock-runtime.us-east-1.amazonaws.com/model/us.anthropic.claude-3-5-sonnet-20241022-v2%3A0/converse
response:
body:
string: '{"metrics":{"latencyMs":654},"output":{"message":{"content":[{"text":"test"}],"role":"assistant"}},"stopReason":"end_turn","usage":{"inputTokens":15,"outputTokens":4,"serverToolUsage":{},"totalTokens":19}}'
headers:
Connection:
- keep-alive
Content-Length:
- '205'
Content-Type:
- application/json
Date:
- Mon, 01 Dec 2025 08:51:02 GMT
x-amzn-RequestId:
- X-AMZN-REQUESTID-XXX
status:
code: 200
message: OK
version: 1