mirror of
https://github.com/crewAIInc/crewAI.git
synced 2026-07-09 00:45:16 +00:00
feat(otel): surface real finish_reason + sampling params + response.id on LLM events (#5945)
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Vulnerability Scan / pip-audit (push) Has been cancelled
Build uv cache / build-cache (3.10) (push) Has been cancelled
Build uv cache / build-cache (3.11) (push) Has been cancelled
Build uv cache / build-cache (3.12) (push) Has been cancelled
Build uv cache / build-cache (3.13) (push) Has been cancelled
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Vulnerability Scan / pip-audit (push) Has been cancelled
Build uv cache / build-cache (3.10) (push) Has been cancelled
Build uv cache / build-cache (3.11) (push) Has been cancelled
Build uv cache / build-cache (3.12) (push) Has been cancelled
Build uv cache / build-cache (3.13) (push) Has been cancelled
* feat(otel): surface real finish_reason + sampling params + response.id on LLM events
Companion to the OTel GenAI emitter compliance work in crewai-enterprise
(CON-172). Today the enterprise emitter reads these fields off the OSS
LLM events via `getattr(..., None)`, so it produces valid (but partial)
spans against the existing OSS surface. This change makes those fields
first-class on the events so spans can carry the real provider data.
What this adds:
- `LLMCallStartedEvent` gains the sampling-param fields the emitter needs
for `gen_ai.request.*`: `temperature`, `top_p`, `max_tokens`, `stream`,
`seed`, `stop_sequences`, `frequency_penalty`, `presence_penalty`, `n`.
All optional; existing call sites keep working.
- `BaseLLM._emit_call_started_event` introspects those values off `self`
(the LLM instance) via `getattr(..., None)` so every provider gets the
fields propagated for free without per-provider plumbing.
- `LLMCallCompletedEvent` gains `finish_reason: str | None` and
`response_id: str | None`. A field validator coerces any non-string
value (MagicMock, unexpected provider object) to None so the event
never raises on construction.
- `LLM._emit_call_completed_event` accepts both as kwargs.
- `LLM` (LiteLLM path) gets a defensive `_extract_finish_reason_and_response_id`
helper that handles both streaming (`StreamingChoices`) and non-streaming
(`Choices`) shapes and is wired into every completion-event emission site.
- Provider completions extract native values from their SDK responses and
pass them through:
- OpenAI: `_extract_responses_finish_reason_and_id` for Responses-API,
`_extract_finish_reason_and_id` for Chat-Completions.
- Anthropic: `_extract_finish_reason_and_id` (Messages API + streaming).
- Bedrock: `_extract_finish_reason_and_id` (`stopReason` from converse).
- Gemini: `_extract_finish_reason_and_id` (`finish_reason` from candidates).
- Azure: inherits via OpenAI sub-class; adds the helper for Azure-specific
response shapes.
- openai_compatible: inherits from OpenAICompletion, no edits needed.
Compatibility:
- All new fields are optional with sensible defaults. No existing call
sites need to change.
- The validator on `LLMCallCompletedEvent` swallows non-string values for
the new fields so legacy mocks / exotic provider types don't blow up
event construction.
- Enterprise side already reads these fields defensively, so OSS and
enterprise can merge independently and cut on the same synchronized
release.
Tested against the full LLM + events + provider test suite — all green;
the 14 pre-existing multimodal failures on main are unrelated and
reproduce without this diff.
* fix(bedrock): propagate finish_reason + response_id on async paths
The original commit covered every provider's sync path and Bedrock's
sync streaming path, but two Bedrock async paths still emitted
LLMCallCompletedEvent without finish_reason/response_id:
- _ahandle_converse: the final fallback emit_call_completed_event call
was missing both fields. Added stop_reason + response_id matching the
other emission sites in the same function.
- _ahandle_streaming_converse: response_id was never seeded from the
initial response object, and stream_finish_reason wasn't propagated
to the structured-output and final-text emissions. Now extracts
response_id up front and threads stream_finish_reason through every
completion event.
Adds a dedicated test file covering the new event fields end-to-end:
- LLMCallCompletedEvent.finish_reason / response_id Pydantic validation
(string accepted, None default, non-string coerced to None).
- LLMCallStartedEvent sampling params (all nine fields accepted, default
to None).
- BaseLLM._emit_call_started_event introspecting sampling params off
self, with explicit kwargs overriding.
- BaseLLM._emit_call_completed_event passing finish_reason/response_id
through to the event.
- LLM._extract_finish_reason_and_response_id across the LiteLLM shapes
(non-streaming response, streaming chunk, dict, missing fields,
non-string values, unexpected input).
* fix(otel): correct streaming finish_reason + bedrock response_id semantics
Two correctness fixes uncovered while landing the OTel finish_reason +
response_id plumbing:
- LiteLLM streaming (sync + async): `stream_options={"include_usage": True}`
causes LiteLLM to emit a final usage-only chunk with `choices=[]`. The
post-loop `_extract_finish_reason_and_response_id(last_chunk)` silently
returned `(None, None)` because the last chunk has no choices, even though
earlier chunks carried `finish_reason="stop"`. Track both fields
incrementally inside the loop (mirroring how OpenAI/Gemini/Azure already
handle their native streams) and use the tracked values for the
LLMCallCompletedEvent emission and the partial-response error path.
- Bedrock Converse: `ResponseMetadata.RequestId` is an AWS infra trace id,
not a model-level response id (semantically different from OpenAI's
`chatcmpl-XXX`). Return None for `response_id` rather than mislead
downstream telemetry consumers. The audit-fix's async propagation chain
still works — None propagates through unchanged.
Adds `test_llm_streaming_finish_reason.py` pinning both the sync and async
LiteLLM streaming paths against the include_usage chunk shape.
* refactor(otel): unify LLM event introspection + drop redundant defensive code
Three cohesion cleanups uncovered during PR review, all behavior-preserving:
- LLM.call / LLM.acall in llm.py now delegate to BaseLLM._emit_call_started_event
instead of constructing LLMCallStartedEvent inline. The base helper already
introspects sampling params off self via getattr; the inline duplication was
accidental, not justified, and a duplication risk if anyone adds a tenth
OTel sampling param later.
- Extracted lib/crewai/llms/_finish_reason_utils.py:extract_choices_finish_reason_and_id
as the shared extractor for the choices-based response shape. OpenAI Chat,
Azure, and LiteLLM all read the same shape (response.id + choices[0].finish_reason)
as both object attrs and dict keys. Providers with genuinely different shapes
- Anthropic (stop_reason), Bedrock (stopReason), Gemini (protobuf enum),
OpenAI Responses (status) - keep their own provider-specific helpers.
- Dropped redundant try/except (AttributeError, TypeError) wrappers around
bare getattr(obj, "field", None) calls across the new extraction helpers.
getattr with a default already suppresses AttributeError, and the inner
isinstance / dict.get / int-coercion ops can't raise TypeError in practice.
Kept the catches that legitimately guard against IndexError (e.g. choices[0]
on an empty list).
Tests: 600 passed, 23 skipped, 14 pre-existing multimodal failures unchanged.
Added 12 parametrized tests for the shared helper covering object + dict
shapes, missing fields, non-string coercion, and never-raises invariants.
* chore(otel): drop dead last_chunk variable from async streaming
The streaming-fix commit (49e5581b5) replaced the post-loop
`_extract_finish_reason_and_response_id(last_chunk)` call with the
incrementally-tracked `stream_finish_reason` / `stream_response_id`,
which removed the only reader of `last_chunk` in
`_ahandle_streaming_response`. The declaration and per-iteration
assignment were left behind — harmless but confusing for future
readers because the sync sibling still legitimately uses `last_chunk`
(for usage and content fallbacks via `_handle_streaming_callbacks`).
The async path inlines its usage extraction directly inside the loop
(`chunk.model_extra.get("usage")`), so there's no fallback consumer.
Drop both lines.
Sync path untouched — `last_chunk` there is still load-bearing.
* fix(otel): coerce non-list stop_sequences to list[str] on LLMCallStartedEvent
Observed in Datadog: gen_ai.request.stop_sequences on a Gemini/Vertex
span surfaced the textproto repr of a google.protobuf.struct_pb2.ListValue
(values { string_value: "\nObservation:" }) instead of a real Sequence[str].
Root cause is upstream - a Vertex AI / Gemini code path stores the stop
list in a protobuf container (RepeatedScalarContainer or ListValue) rather
than a plain Python list. When that container reaches LLMCallStartedEvent
and then BaseLLM._emit_call_started_event hands it to the OTel SDK as a
span attribute, the SDK falls back to str(value) because the type isn't a
recognised Sequence[str] - producing the protobuf textproto string instead
of an array attribute.
* chore: fix ruff lint findings
* refactor(otel): declare sampling params on BaseLLM + honor stop overrides + dict chunk id
* fix: widen max_tokens to int | float | None + apply ruff format
* fix(otel): coerce unknown finish_reason / response_id to None instead of stringifying
* fix(otel): extract Azure stream finish_reason/id before usage-continue
Match the LiteLLM ordering so a finish_reason or response id riding on a
usage-carrying chunk isn't dropped by the early `continue`.
* fix(otel): report effective max_tokens cap + bedrock structured finish_reason
This commit is contained in:
@@ -1,7 +1,7 @@
|
||||
from enum import Enum
|
||||
from typing import Any, Literal
|
||||
|
||||
from pydantic import BaseModel
|
||||
from pydantic import BaseModel, field_validator
|
||||
|
||||
from crewai.events.base_events import BaseEvent
|
||||
|
||||
@@ -48,6 +48,43 @@ class LLMCallStartedEvent(LLMEventBase):
|
||||
tools: list[dict[str, Any]] | None = None
|
||||
callbacks: list[Any] | None = None
|
||||
available_functions: dict[str, Any] | None = None
|
||||
# Sampling/request parameters forwarded for OTel GenAI compliance.
|
||||
# All optional so legacy emitters keep working unchanged.
|
||||
temperature: float | None = None
|
||||
top_p: float | None = None
|
||||
max_tokens: int | float | None = None
|
||||
stream: bool | None = None
|
||||
seed: int | None = None
|
||||
stop_sequences: list[str] | None = None
|
||||
frequency_penalty: float | None = None
|
||||
presence_penalty: float | None = None
|
||||
n: int | None = None
|
||||
|
||||
@field_validator("stop_sequences", mode="before")
|
||||
@classmethod
|
||||
def _coerce_stop_sequences_to_str_list(cls, value: Any) -> list[str] | None:
|
||||
"""Normalize stop_sequences to ``list[str] | None``.
|
||||
|
||||
Some providers store stop sequences in non-Python-list containers —
|
||||
e.g. a Vertex AI / Gemini code path can hand back a
|
||||
``google.protobuf.struct_pb2.ListValue`` or a ``RepeatedScalarContainer``.
|
||||
Without coercion the OTel SDK falls back to ``str(value)`` when
|
||||
``gen_ai.request.stop_sequences`` is set, producing the protobuf
|
||||
textproto repr (``values { string_value: \"...\" }``) instead of a
|
||||
proper ``Sequence[str]``.
|
||||
|
||||
A bare string is treated as a single stop sequence. Anything that
|
||||
can't be iterated cleanly falls back to ``None`` rather than crashing
|
||||
event construction.
|
||||
"""
|
||||
if value is None:
|
||||
return None
|
||||
if isinstance(value, str):
|
||||
return [value]
|
||||
try:
|
||||
return [item if isinstance(item, str) else str(item) for item in value]
|
||||
except TypeError:
|
||||
return None
|
||||
|
||||
|
||||
class LLMCallCompletedEvent(LLMEventBase):
|
||||
@@ -58,6 +95,23 @@ class LLMCallCompletedEvent(LLMEventBase):
|
||||
response: Any
|
||||
call_type: LLMCallType
|
||||
usage: dict[str, Any] | None = None
|
||||
finish_reason: str | None = None
|
||||
response_id: str | None = None
|
||||
|
||||
@field_validator("finish_reason", "response_id", mode="before")
|
||||
@classmethod
|
||||
def _coerce_non_string_to_none(cls, value: Any) -> str | None:
|
||||
"""Drop non-string values so test mocks and exotic provider types
|
||||
(MagicMock, protobuf enums, etc.) never crash event construction.
|
||||
|
||||
Provider helpers are best-effort: when extraction returns something
|
||||
non-string (e.g. a ``MagicMock`` in unit tests), we treat it as
|
||||
"no value" rather than raising. Downstream telemetry already
|
||||
handles the missing-attribute case.
|
||||
"""
|
||||
if value is None or isinstance(value, str):
|
||||
return value
|
||||
return None
|
||||
|
||||
|
||||
class LLMCallFailedEvent(LLMEventBase):
|
||||
|
||||
@@ -23,7 +23,6 @@ from crewai.events.event_bus import crewai_event_bus
|
||||
from crewai.events.types.llm_events import (
|
||||
LLMCallCompletedEvent,
|
||||
LLMCallFailedEvent,
|
||||
LLMCallStartedEvent,
|
||||
LLMCallType,
|
||||
LLMStreamChunkEvent,
|
||||
)
|
||||
@@ -32,6 +31,7 @@ from crewai.events.types.tool_usage_events import (
|
||||
ToolUsageFinishedEvent,
|
||||
ToolUsageStartedEvent,
|
||||
)
|
||||
from crewai.llms._finish_reason_utils import extract_choices_finish_reason_and_id
|
||||
from crewai.llms.base_llm import (
|
||||
BaseLLM,
|
||||
JsonResponseFormat,
|
||||
@@ -732,6 +732,11 @@ class LLM(BaseLLM):
|
||||
last_chunk = None
|
||||
chunk_count = 0
|
||||
usage_info = None
|
||||
# Tracked across the loop: LiteLLM with include_usage emits a final
|
||||
# usage-only chunk with empty choices, so the post-loop last_chunk has
|
||||
# no finish_reason. Capture both incrementally instead.
|
||||
stream_finish_reason: str | None = None
|
||||
stream_response_id: str | None = None
|
||||
|
||||
accumulated_tool_args: defaultdict[int, AccumulatedToolArgs] = defaultdict(
|
||||
AccumulatedToolArgs
|
||||
@@ -750,6 +755,16 @@ class LLM(BaseLLM):
|
||||
|
||||
if isinstance(chunk, ModelResponseBase):
|
||||
response_id = chunk.id
|
||||
elif isinstance(chunk, dict):
|
||||
response_id = chunk.get("id")
|
||||
|
||||
chunk_finish, chunk_id = self._extract_finish_reason_and_response_id(
|
||||
chunk
|
||||
)
|
||||
if chunk_finish:
|
||||
stream_finish_reason = chunk_finish
|
||||
if chunk_id and not stream_response_id:
|
||||
stream_response_id = chunk_id
|
||||
|
||||
try:
|
||||
choices = None
|
||||
@@ -922,6 +937,11 @@ class LLM(BaseLLM):
|
||||
if tool_calls_list:
|
||||
return tool_calls_list
|
||||
|
||||
finish_reason, response_id_last = (
|
||||
stream_finish_reason,
|
||||
stream_response_id,
|
||||
)
|
||||
|
||||
if not tool_calls or not available_functions:
|
||||
if response_model and self.is_litellm:
|
||||
instructor_instance = InternalInstructor(
|
||||
@@ -939,6 +959,8 @@ class LLM(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage_dict,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id_last,
|
||||
)
|
||||
return structured_response
|
||||
|
||||
@@ -950,6 +972,8 @@ class LLM(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage_dict,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id_last,
|
||||
)
|
||||
return full_response
|
||||
|
||||
@@ -965,6 +989,8 @@ class LLM(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage_dict,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id_last,
|
||||
)
|
||||
return full_response
|
||||
|
||||
@@ -978,6 +1004,10 @@ class LLM(BaseLLM):
|
||||
logging.error(f"Error in streaming response: {e!s}")
|
||||
if full_response.strip():
|
||||
logging.warning(f"Returning partial response despite error: {e!s}")
|
||||
finish_reason, response_id_last = (
|
||||
stream_finish_reason,
|
||||
stream_response_id,
|
||||
)
|
||||
self._handle_emit_call_events(
|
||||
response=full_response,
|
||||
call_type=LLMCallType.LLM_CALL,
|
||||
@@ -985,6 +1015,8 @@ class LLM(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=self._usage_to_dict(usage_info),
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id_last,
|
||||
)
|
||||
return full_response
|
||||
|
||||
@@ -1169,6 +1201,10 @@ class LLM(BaseLLM):
|
||||
else None
|
||||
)
|
||||
|
||||
finish_reason, response_id = self._extract_finish_reason_and_response_id(
|
||||
response
|
||||
)
|
||||
|
||||
if response_model is not None:
|
||||
# When using instructor/response_model, litellm returns a Pydantic model instance
|
||||
if isinstance(response, BaseModel):
|
||||
@@ -1180,6 +1216,8 @@ class LLM(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=response_usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return structured_response
|
||||
|
||||
@@ -1216,6 +1254,8 @@ class LLM(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=response_usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return text_response
|
||||
|
||||
@@ -1233,6 +1273,8 @@ class LLM(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=response_usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return text_response
|
||||
|
||||
@@ -1310,6 +1352,10 @@ class LLM(BaseLLM):
|
||||
else None
|
||||
)
|
||||
|
||||
finish_reason, response_id = self._extract_finish_reason_and_response_id(
|
||||
response
|
||||
)
|
||||
|
||||
if response_model is not None:
|
||||
if isinstance(response, BaseModel):
|
||||
structured_response = response.model_dump_json()
|
||||
@@ -1320,6 +1366,8 @@ class LLM(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=response_usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return structured_response
|
||||
|
||||
@@ -1358,6 +1406,8 @@ class LLM(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=response_usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return text_response
|
||||
|
||||
@@ -1375,6 +1425,8 @@ class LLM(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=response_usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return text_response
|
||||
|
||||
@@ -1412,12 +1464,29 @@ class LLM(BaseLLM):
|
||||
params["stream"] = True
|
||||
params["stream_options"] = {"include_usage": True}
|
||||
response_id = None
|
||||
# See sync sibling: incrementally track finish_reason/response_id so the
|
||||
# usage-only final chunk doesn't wipe them.
|
||||
stream_finish_reason: str | None = None
|
||||
stream_response_id: str | None = None
|
||||
|
||||
try:
|
||||
async for chunk in await litellm.acompletion(**params):
|
||||
chunk_count += 1
|
||||
chunk_content = None
|
||||
response_id = chunk.id if isinstance(chunk, ModelResponseBase) else None
|
||||
if isinstance(chunk, ModelResponseBase):
|
||||
response_id = chunk.id
|
||||
elif isinstance(chunk, dict):
|
||||
response_id = chunk.get("id")
|
||||
else:
|
||||
response_id = None
|
||||
|
||||
chunk_finish, chunk_id = self._extract_finish_reason_and_response_id(
|
||||
chunk
|
||||
)
|
||||
if chunk_finish:
|
||||
stream_finish_reason = chunk_finish
|
||||
if chunk_id and not stream_response_id:
|
||||
stream_response_id = chunk_id
|
||||
|
||||
try:
|
||||
choices = None
|
||||
@@ -1525,6 +1594,10 @@ class LLM(BaseLLM):
|
||||
return tool_calls_list
|
||||
|
||||
usage_dict = self._usage_to_dict(usage_info)
|
||||
finish_reason, response_id_last = (
|
||||
stream_finish_reason,
|
||||
stream_response_id,
|
||||
)
|
||||
self._handle_emit_call_events(
|
||||
response=full_response,
|
||||
call_type=LLMCallType.LLM_CALL,
|
||||
@@ -1532,6 +1605,8 @@ class LLM(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params.get("messages"),
|
||||
usage=usage_dict,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id_last,
|
||||
)
|
||||
return full_response
|
||||
|
||||
@@ -1545,6 +1620,10 @@ class LLM(BaseLLM):
|
||||
if chunk_count == 0:
|
||||
raise
|
||||
if full_response:
|
||||
finish_reason, response_id_last = (
|
||||
stream_finish_reason,
|
||||
stream_response_id,
|
||||
)
|
||||
self._handle_emit_call_events(
|
||||
response=full_response,
|
||||
call_type=LLMCallType.LLM_CALL,
|
||||
@@ -1552,6 +1631,8 @@ class LLM(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params.get("messages"),
|
||||
usage=self._usage_to_dict(usage_info),
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id_last,
|
||||
)
|
||||
return full_response
|
||||
raise
|
||||
@@ -1678,19 +1759,14 @@ class LLM(BaseLLM):
|
||||
ValueError: If response format is not supported
|
||||
LLMContextLengthExceededError: If input exceeds model's context limit
|
||||
"""
|
||||
with llm_call_context() as call_id:
|
||||
crewai_event_bus.emit(
|
||||
self,
|
||||
event=LLMCallStartedEvent(
|
||||
messages=messages,
|
||||
tools=tools,
|
||||
callbacks=callbacks,
|
||||
available_functions=available_functions,
|
||||
from_task=from_task,
|
||||
from_agent=from_agent,
|
||||
model=self.model,
|
||||
call_id=call_id,
|
||||
),
|
||||
with llm_call_context():
|
||||
self._emit_call_started_event(
|
||||
messages=messages,
|
||||
tools=tools,
|
||||
callbacks=callbacks,
|
||||
available_functions=available_functions,
|
||||
from_task=from_task,
|
||||
from_agent=from_agent,
|
||||
)
|
||||
|
||||
self._validate_call_params()
|
||||
@@ -1822,19 +1898,14 @@ class LLM(BaseLLM):
|
||||
ValueError: If response format is not supported
|
||||
LLMContextLengthExceededError: If input exceeds model's context limit
|
||||
"""
|
||||
with llm_call_context() as call_id:
|
||||
crewai_event_bus.emit(
|
||||
self,
|
||||
event=LLMCallStartedEvent(
|
||||
messages=messages,
|
||||
tools=tools,
|
||||
callbacks=callbacks,
|
||||
available_functions=available_functions,
|
||||
from_task=from_task,
|
||||
from_agent=from_agent,
|
||||
model=self.model,
|
||||
call_id=call_id,
|
||||
),
|
||||
with llm_call_context():
|
||||
self._emit_call_started_event(
|
||||
messages=messages,
|
||||
tools=tools,
|
||||
callbacks=callbacks,
|
||||
available_functions=available_functions,
|
||||
from_task=from_task,
|
||||
from_agent=from_agent,
|
||||
)
|
||||
|
||||
self._validate_call_params()
|
||||
@@ -1990,6 +2061,8 @@ class LLM(BaseLLM):
|
||||
from_agent: BaseAgent | None = None,
|
||||
messages: str | list[LLMMessage] | None = None,
|
||||
usage: dict[str, Any] | None = None,
|
||||
finish_reason: str | None = None,
|
||||
response_id: str | None = None,
|
||||
) -> None:
|
||||
"""Handle the events for the LLM call.
|
||||
|
||||
@@ -2000,6 +2073,10 @@ class LLM(BaseLLM):
|
||||
from_agent: Optional agent object
|
||||
messages: Optional messages object
|
||||
usage: Optional token usage data
|
||||
finish_reason: Raw provider finish reason (e.g. "stop", "length",
|
||||
"tool_calls"). Optional; downstream telemetry coerces to the
|
||||
OTel GenAI enum.
|
||||
response_id: Raw provider response identifier. Optional.
|
||||
"""
|
||||
crewai_event_bus.emit(
|
||||
self,
|
||||
@@ -2012,9 +2089,24 @@ class LLM(BaseLLM):
|
||||
model=self.model,
|
||||
call_id=get_current_call_id(),
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
),
|
||||
)
|
||||
|
||||
def _effective_max_tokens(self) -> int | float | None:
|
||||
"""LiteLLM sends ``max_tokens or max_completion_tokens`` as the cap."""
|
||||
return self.max_tokens or self.max_completion_tokens
|
||||
|
||||
@staticmethod
|
||||
def _extract_finish_reason_and_response_id(
|
||||
response_or_chunk: Any,
|
||||
) -> tuple[str | None, str | None]:
|
||||
"""LiteLLM responses/chunks share the choices-shape with OpenAI/Azure;
|
||||
delegate to the shared extractor.
|
||||
"""
|
||||
return extract_choices_finish_reason_and_id(response_or_chunk)
|
||||
|
||||
def _process_message_files(self, messages: list[LLMMessage]) -> list[LLMMessage]:
|
||||
"""Process files attached to messages and format for provider.
|
||||
|
||||
|
||||
55
lib/crewai/src/crewai/llms/_finish_reason_utils.py
Normal file
55
lib/crewai/src/crewai/llms/_finish_reason_utils.py
Normal file
@@ -0,0 +1,55 @@
|
||||
"""Shared extractors for ``finish_reason`` + ``response_id`` across LLM providers.
|
||||
|
||||
OpenAI Chat Completions, Azure AI Inference, and LiteLLM all expose the same
|
||||
choices-based response shape (``response.id`` + ``response.choices[0].finish_reason``),
|
||||
both as object attributes and (for LiteLLM stream chunks) as dict keys. This
|
||||
module centralises that introspection so every provider doesn't reinvent the
|
||||
defensive walk. Providers with genuinely different shapes — Anthropic
|
||||
(``stop_reason``), Bedrock (``stopReason``), Gemini (protobuf enum), OpenAI
|
||||
Responses (``status``) — keep their own helpers.
|
||||
"""
|
||||
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
|
||||
|
||||
def _as_str(value: Any) -> str | None:
|
||||
return value if isinstance(value, str) else None
|
||||
|
||||
|
||||
def extract_choices_finish_reason_and_id(
|
||||
response_or_chunk: Any,
|
||||
) -> tuple[str | None, str | None]:
|
||||
"""Extract ``(finish_reason, response_id)`` from a choices-shaped response.
|
||||
|
||||
Handles both object-style (``response.id``, ``response.choices[0].finish_reason``)
|
||||
and dict-style (``response["id"]``, ``response["choices"][0]["finish_reason"]``)
|
||||
inputs. Returns ``(None, None)`` on any failure; never raises. Non-string
|
||||
raw values are coerced to ``None`` so test mocks and exotic provider types
|
||||
(MagicMock, protobuf enums, etc.) don't propagate downstream.
|
||||
"""
|
||||
raw_id = getattr(response_or_chunk, "id", None)
|
||||
if raw_id is None and isinstance(response_or_chunk, dict):
|
||||
raw_id = response_or_chunk.get("id")
|
||||
response_id = _as_str(raw_id)
|
||||
|
||||
if isinstance(response_or_chunk, dict):
|
||||
choices = response_or_chunk.get("choices")
|
||||
else:
|
||||
choices = getattr(response_or_chunk, "choices", None)
|
||||
|
||||
finish_reason: str | None = None
|
||||
if choices:
|
||||
try:
|
||||
first = choices[0]
|
||||
except (IndexError, TypeError, KeyError):
|
||||
first = None
|
||||
if first is not None:
|
||||
if isinstance(first, dict):
|
||||
raw_finish = first.get("finish_reason")
|
||||
else:
|
||||
raw_finish = getattr(first, "finish_reason", None)
|
||||
finish_reason = _as_str(raw_finish)
|
||||
|
||||
return finish_reason, response_id
|
||||
@@ -150,6 +150,13 @@ class BaseLLM(BaseModel, ABC):
|
||||
llm_type: str = "base"
|
||||
model: str
|
||||
temperature: float | None = None
|
||||
top_p: float | None = None
|
||||
max_tokens: int | float | None = None
|
||||
stream: bool | None = None
|
||||
seed: int | None = None
|
||||
frequency_penalty: float | None = None
|
||||
presence_penalty: float | None = None
|
||||
n: int | None = None
|
||||
api_key: str | None = None
|
||||
base_url: str | None = None
|
||||
provider: str = Field(default="openai")
|
||||
@@ -464,6 +471,16 @@ class BaseLLM(BaseModel, ABC):
|
||||
"""
|
||||
return None
|
||||
|
||||
def _effective_max_tokens(self) -> int | float | None:
|
||||
"""Token cap actually sent to the provider, for start-event telemetry.
|
||||
|
||||
Defaults to ``self.max_tokens``. Providers that cap generation through a
|
||||
differently named field (e.g. ``max_completion_tokens`` on OpenAI/Azure,
|
||||
``max_output_tokens`` on Gemini) override this so ``LLMCallStartedEvent``
|
||||
reports the real limit instead of ``None``.
|
||||
"""
|
||||
return self.max_tokens
|
||||
|
||||
def _emit_call_started_event(
|
||||
self,
|
||||
messages: str | list[LLMMessage],
|
||||
@@ -472,10 +489,38 @@ class BaseLLM(BaseModel, ABC):
|
||||
available_functions: dict[str, Any] | None = None,
|
||||
from_task: Task | None = None,
|
||||
from_agent: BaseAgent | None = None,
|
||||
temperature: float | None = None,
|
||||
top_p: float | None = None,
|
||||
max_tokens: int | float | None = None,
|
||||
stream: bool | None = None,
|
||||
seed: int | None = None,
|
||||
stop_sequences: list[str] | None = None,
|
||||
frequency_penalty: float | None = None,
|
||||
presence_penalty: float | None = None,
|
||||
n: int | None = None,
|
||||
) -> None:
|
||||
"""Emit LLM call started event."""
|
||||
from crewai.utilities.serialization import to_serializable
|
||||
|
||||
if temperature is None:
|
||||
temperature = self.temperature
|
||||
if top_p is None:
|
||||
top_p = self.top_p
|
||||
if max_tokens is None:
|
||||
max_tokens = self._effective_max_tokens()
|
||||
if stream is None:
|
||||
stream = self.stream
|
||||
if seed is None:
|
||||
seed = self.seed
|
||||
if stop_sequences is None:
|
||||
stop_sequences = self.stop_sequences or None
|
||||
if frequency_penalty is None:
|
||||
frequency_penalty = self.frequency_penalty
|
||||
if presence_penalty is None:
|
||||
presence_penalty = self.presence_penalty
|
||||
if n is None:
|
||||
n = self.n
|
||||
|
||||
crewai_event_bus.emit(
|
||||
self,
|
||||
event=LLMCallStartedEvent(
|
||||
@@ -487,6 +532,15 @@ class BaseLLM(BaseModel, ABC):
|
||||
from_agent=from_agent,
|
||||
model=self.model,
|
||||
call_id=get_current_call_id(),
|
||||
temperature=temperature,
|
||||
top_p=top_p,
|
||||
max_tokens=max_tokens,
|
||||
stream=stream,
|
||||
seed=seed,
|
||||
stop_sequences=stop_sequences,
|
||||
frequency_penalty=frequency_penalty,
|
||||
presence_penalty=presence_penalty,
|
||||
n=n,
|
||||
),
|
||||
)
|
||||
|
||||
@@ -498,6 +552,8 @@ class BaseLLM(BaseModel, ABC):
|
||||
from_agent: BaseAgent | None = None,
|
||||
messages: str | list[LLMMessage] | None = None,
|
||||
usage: dict[str, Any] | None = None,
|
||||
finish_reason: str | None = None,
|
||||
response_id: str | None = None,
|
||||
) -> None:
|
||||
"""Emit LLM call completed event."""
|
||||
from crewai.utilities.serialization import to_serializable
|
||||
@@ -513,6 +569,8 @@ class BaseLLM(BaseModel, ABC):
|
||||
model=self.model,
|
||||
call_id=get_current_call_id(),
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
),
|
||||
)
|
||||
|
||||
|
||||
@@ -923,6 +923,8 @@ class AnthropicCompletion(BaseLLM):
|
||||
usage = self._extract_anthropic_token_usage(response)
|
||||
self._track_token_usage_internal(usage)
|
||||
|
||||
finish_reason, response_id = self._extract_finish_reason_and_id(response)
|
||||
|
||||
if _is_pydantic_model_class(response_model) and response.content:
|
||||
if use_native_structured_output:
|
||||
for block in response.content:
|
||||
@@ -935,6 +937,8 @@ class AnthropicCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return structured_data
|
||||
else:
|
||||
@@ -951,6 +955,8 @@ class AnthropicCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return structured_data
|
||||
|
||||
@@ -973,6 +979,8 @@ class AnthropicCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return list(tool_uses)
|
||||
|
||||
@@ -1005,6 +1013,8 @@ class AnthropicCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
if usage.get("total_tokens", 0) > 0:
|
||||
@@ -1147,6 +1157,10 @@ class AnthropicCompletion(BaseLLM):
|
||||
usage = self._extract_anthropic_token_usage(final_message)
|
||||
self._track_token_usage_internal(usage)
|
||||
|
||||
finish_reason, final_response_id = self._extract_finish_reason_and_id(
|
||||
final_message
|
||||
)
|
||||
|
||||
if _is_pydantic_model_class(response_model):
|
||||
if use_native_structured_output:
|
||||
structured_data = response_model.model_validate_json(full_response)
|
||||
@@ -1157,6 +1171,8 @@ class AnthropicCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=final_response_id,
|
||||
)
|
||||
return structured_data
|
||||
for block in final_message.content:
|
||||
@@ -1172,6 +1188,8 @@ class AnthropicCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=final_response_id,
|
||||
)
|
||||
return structured_data
|
||||
|
||||
@@ -1201,6 +1219,8 @@ class AnthropicCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=final_response_id,
|
||||
)
|
||||
|
||||
return self._invoke_after_llm_call_hooks(
|
||||
@@ -1361,6 +1381,10 @@ class AnthropicCompletion(BaseLLM):
|
||||
|
||||
final_content = self._apply_stop_words(final_content)
|
||||
|
||||
finish_reason, final_response_id = self._extract_finish_reason_and_id(
|
||||
final_response
|
||||
)
|
||||
|
||||
self._emit_call_completed_event(
|
||||
response=final_content,
|
||||
call_type=LLMCallType.LLM_CALL,
|
||||
@@ -1368,6 +1392,8 @@ class AnthropicCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=follow_up_params["messages"],
|
||||
usage=follow_up_usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=final_response_id,
|
||||
)
|
||||
|
||||
total_usage = {
|
||||
@@ -1447,6 +1473,8 @@ class AnthropicCompletion(BaseLLM):
|
||||
usage = self._extract_anthropic_token_usage(response)
|
||||
self._track_token_usage_internal(usage)
|
||||
|
||||
finish_reason, response_id = self._extract_finish_reason_and_id(response)
|
||||
|
||||
if _is_pydantic_model_class(response_model) and response.content:
|
||||
if use_native_structured_output:
|
||||
for block in response.content:
|
||||
@@ -1459,6 +1487,8 @@ class AnthropicCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return structured_data
|
||||
else:
|
||||
@@ -1475,6 +1505,8 @@ class AnthropicCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return structured_data
|
||||
|
||||
@@ -1495,6 +1527,8 @@ class AnthropicCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return list(tool_uses)
|
||||
|
||||
@@ -1519,6 +1553,8 @@ class AnthropicCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
if usage.get("total_tokens", 0) > 0:
|
||||
@@ -1647,6 +1683,10 @@ class AnthropicCompletion(BaseLLM):
|
||||
usage = self._extract_anthropic_token_usage(final_message)
|
||||
self._track_token_usage_internal(usage)
|
||||
|
||||
finish_reason, final_response_id = self._extract_finish_reason_and_id(
|
||||
final_message
|
||||
)
|
||||
|
||||
if _is_pydantic_model_class(response_model):
|
||||
if use_native_structured_output:
|
||||
structured_data = response_model.model_validate_json(full_response)
|
||||
@@ -1657,6 +1697,8 @@ class AnthropicCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=final_response_id,
|
||||
)
|
||||
return structured_data
|
||||
for block in final_message.content:
|
||||
@@ -1672,6 +1714,8 @@ class AnthropicCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=final_response_id,
|
||||
)
|
||||
return structured_data
|
||||
|
||||
@@ -1701,6 +1745,8 @@ class AnthropicCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=final_response_id,
|
||||
)
|
||||
|
||||
return full_response
|
||||
@@ -1753,6 +1799,10 @@ class AnthropicCompletion(BaseLLM):
|
||||
|
||||
final_content = self._apply_stop_words(final_content)
|
||||
|
||||
finish_reason, final_response_id = self._extract_finish_reason_and_id(
|
||||
final_response
|
||||
)
|
||||
|
||||
self._emit_call_completed_event(
|
||||
response=final_content,
|
||||
call_type=LLMCallType.LLM_CALL,
|
||||
@@ -1760,6 +1810,8 @@ class AnthropicCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=follow_up_params["messages"],
|
||||
usage=follow_up_usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=final_response_id,
|
||||
)
|
||||
|
||||
total_usage = {
|
||||
@@ -1813,6 +1865,20 @@ class AnthropicCompletion(BaseLLM):
|
||||
|
||||
return int(200000 * CONTEXT_WINDOW_USAGE_RATIO)
|
||||
|
||||
@staticmethod
|
||||
def _extract_finish_reason_and_id(
|
||||
message: Any,
|
||||
) -> tuple[str | None, str | None]:
|
||||
"""Extract raw finish_reason and response_id from an Anthropic
|
||||
``Message`` / ``BetaMessage``. Anthropic exposes ``stop_reason`` (e.g.
|
||||
``"end_turn"``, ``"max_tokens"``, ``"tool_use"``); we forward it raw
|
||||
and let downstream telemetry map to the OTel GenAI enum.
|
||||
"""
|
||||
return (
|
||||
getattr(message, "stop_reason", None),
|
||||
getattr(message, "id", None),
|
||||
)
|
||||
|
||||
@staticmethod
|
||||
def _extract_anthropic_token_usage(
|
||||
response: Message | BetaMessage,
|
||||
|
||||
@@ -9,6 +9,7 @@ from urllib.parse import urlparse
|
||||
from pydantic import BaseModel, PrivateAttr, model_validator
|
||||
from typing_extensions import Self
|
||||
|
||||
from crewai.llms._finish_reason_utils import extract_choices_finish_reason_and_id
|
||||
from crewai.llms.hooks.base import BaseInterceptor
|
||||
from crewai.utilities.agent_utils import is_context_length_exceeded
|
||||
from crewai.utilities.exceptions.context_window_exceeding_exception import (
|
||||
@@ -783,6 +784,8 @@ class AzureCompletion(BaseLLM):
|
||||
from_task: Any | None = None,
|
||||
from_agent: Any | None = None,
|
||||
usage: dict[str, Any] | None = None,
|
||||
finish_reason: str | None = None,
|
||||
response_id: str | None = None,
|
||||
) -> BaseModel:
|
||||
"""Validate content against response model and emit completion event.
|
||||
|
||||
@@ -792,6 +795,8 @@ class AzureCompletion(BaseLLM):
|
||||
params: Completion parameters containing messages
|
||||
from_task: Task that initiated the call
|
||||
from_agent: Agent that initiated the call
|
||||
finish_reason: Raw provider finish reason.
|
||||
response_id: Raw provider response id.
|
||||
|
||||
Returns:
|
||||
Validated Pydantic model instance
|
||||
@@ -809,6 +814,8 @@ class AzureCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
return structured_data
|
||||
@@ -848,6 +855,8 @@ class AzureCompletion(BaseLLM):
|
||||
usage = self._extract_azure_token_usage(response)
|
||||
self._track_token_usage_internal(usage)
|
||||
|
||||
finish_reason, response_id = self._extract_finish_reason_and_id(response)
|
||||
|
||||
# Without available_functions, return tool_calls so the caller (executor) handles execution
|
||||
if message.tool_calls and not available_functions:
|
||||
self._emit_call_completed_event(
|
||||
@@ -857,6 +866,8 @@ class AzureCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return list(message.tool_calls)
|
||||
|
||||
@@ -892,6 +903,8 @@ class AzureCompletion(BaseLLM):
|
||||
from_task=from_task,
|
||||
from_agent=from_agent,
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
content = self._apply_stop_words(content)
|
||||
@@ -903,6 +916,8 @@ class AzureCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
return self._invoke_after_llm_call_hooks(
|
||||
@@ -1011,6 +1026,8 @@ class AzureCompletion(BaseLLM):
|
||||
from_task: Any | None = None,
|
||||
from_agent: Any | None = None,
|
||||
response_model: type[BaseModel] | None = None,
|
||||
finish_reason: str | None = None,
|
||||
response_id: str | None = None,
|
||||
) -> str | Any:
|
||||
"""Finalize streaming response with usage tracking, tool execution, and events.
|
||||
|
||||
@@ -1039,6 +1056,8 @@ class AzureCompletion(BaseLLM):
|
||||
from_task=from_task,
|
||||
from_agent=from_agent,
|
||||
usage=usage_data,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
# Without available_functions, return tool calls in OpenAI-compatible format for the executor
|
||||
@@ -1061,6 +1080,8 @@ class AzureCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage_data,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return formatted_tool_calls
|
||||
|
||||
@@ -1094,6 +1115,8 @@ class AzureCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage_data,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
return self._invoke_after_llm_call_hooks(
|
||||
@@ -1113,8 +1136,16 @@ class AzureCompletion(BaseLLM):
|
||||
tool_calls: dict[int, dict[str, Any]] = {}
|
||||
|
||||
usage_data: dict[str, Any] | None = None
|
||||
stream_finish_reason: str | None = None
|
||||
stream_response_id: str | None = None
|
||||
for update in self._get_sync_client().complete(**params):
|
||||
if isinstance(update, StreamingChatCompletionsUpdate):
|
||||
chunk_finish, chunk_id = self._extract_finish_reason_and_id(update)
|
||||
if chunk_finish:
|
||||
stream_finish_reason = chunk_finish
|
||||
if chunk_id:
|
||||
stream_response_id = chunk_id
|
||||
|
||||
if update.usage:
|
||||
usage = update.usage
|
||||
usage_data = {
|
||||
@@ -1141,6 +1172,8 @@ class AzureCompletion(BaseLLM):
|
||||
from_task=from_task,
|
||||
from_agent=from_agent,
|
||||
response_model=response_model,
|
||||
finish_reason=stream_finish_reason,
|
||||
response_id=stream_response_id,
|
||||
)
|
||||
|
||||
async def _ahandle_completion(
|
||||
@@ -1180,10 +1213,18 @@ class AzureCompletion(BaseLLM):
|
||||
tool_calls: dict[int, dict[str, Any]] = {}
|
||||
|
||||
usage_data: dict[str, Any] | None = None
|
||||
stream_finish_reason: str | None = None
|
||||
stream_response_id: str | None = None
|
||||
|
||||
stream = await self._get_async_client().complete(**params)
|
||||
async for update in stream:
|
||||
if isinstance(update, StreamingChatCompletionsUpdate):
|
||||
chunk_finish, chunk_id = self._extract_finish_reason_and_id(update)
|
||||
if chunk_finish:
|
||||
stream_finish_reason = chunk_finish
|
||||
if chunk_id:
|
||||
stream_response_id = chunk_id
|
||||
|
||||
if hasattr(update, "usage") and update.usage:
|
||||
usage = update.usage
|
||||
usage_data = {
|
||||
@@ -1210,6 +1251,8 @@ class AzureCompletion(BaseLLM):
|
||||
from_task=from_task,
|
||||
from_agent=from_agent,
|
||||
response_model=response_model,
|
||||
finish_reason=stream_finish_reason,
|
||||
response_id=stream_response_id,
|
||||
)
|
||||
|
||||
def supports_function_calling(self) -> bool:
|
||||
@@ -1271,6 +1314,19 @@ class AzureCompletion(BaseLLM):
|
||||
|
||||
return int(8192 * CONTEXT_WINDOW_USAGE_RATIO)
|
||||
|
||||
def _effective_max_tokens(self) -> int | float | None:
|
||||
"""Azure reasoning/newer chat models cap via ``max_completion_tokens``."""
|
||||
return self.max_tokens or self.max_completion_tokens
|
||||
|
||||
@staticmethod
|
||||
def _extract_finish_reason_and_id(
|
||||
response_or_update: Any,
|
||||
) -> tuple[str | None, str | None]:
|
||||
"""Azure ``ChatCompletions`` / ``StreamingChatCompletionsUpdate``
|
||||
share the choices-shape; delegate to the shared extractor.
|
||||
"""
|
||||
return extract_choices_finish_reason_and_id(response_or_update)
|
||||
|
||||
@staticmethod
|
||||
def _extract_azure_token_usage(response: ChatCompletions) -> dict[str, Any]:
|
||||
"""Extract token usage and response metadata from Azure response."""
|
||||
|
||||
@@ -677,7 +677,7 @@ class BedrockCompletion(BaseLLM):
|
||||
if usage:
|
||||
self._track_token_usage_internal(usage)
|
||||
|
||||
stop_reason = response.get("stopReason")
|
||||
stop_reason, response_id = self._extract_finish_reason_and_id(response)
|
||||
if stop_reason:
|
||||
logging.debug(f"Response stop reason: {stop_reason}")
|
||||
if stop_reason == "max_tokens":
|
||||
@@ -716,6 +716,8 @@ class BedrockCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=messages,
|
||||
usage=usage,
|
||||
finish_reason=stop_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return result
|
||||
except Exception as e:
|
||||
@@ -738,6 +740,8 @@ class BedrockCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=messages,
|
||||
usage=usage,
|
||||
finish_reason=stop_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return non_structured_output_tool_uses
|
||||
|
||||
@@ -812,6 +816,8 @@ class BedrockCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=messages,
|
||||
usage=usage,
|
||||
finish_reason=stop_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
return self._invoke_after_llm_call_hooks(
|
||||
@@ -951,7 +957,9 @@ class BedrockCompletion(BaseLLM):
|
||||
)
|
||||
|
||||
stream = response.get("stream")
|
||||
response_id = None
|
||||
_, stream_response_id = self._extract_finish_reason_and_id(response)
|
||||
response_id = stream_response_id
|
||||
stream_finish_reason: str | None = None
|
||||
if stream:
|
||||
for event in stream:
|
||||
if "messageStart" in event:
|
||||
@@ -1042,6 +1050,9 @@ class BedrockCompletion(BaseLLM):
|
||||
result = response_model.model_validate(
|
||||
function_args
|
||||
)
|
||||
# contentBlockStop fires before messageStop sets
|
||||
# stream_finish_reason; structured output always
|
||||
# completes via the tool-call path.
|
||||
self._emit_call_completed_event(
|
||||
response=result.model_dump_json(),
|
||||
call_type=LLMCallType.LLM_CALL,
|
||||
@@ -1049,6 +1060,9 @@ class BedrockCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=messages,
|
||||
usage=usage_data,
|
||||
finish_reason=stream_finish_reason
|
||||
or "tool_use",
|
||||
response_id=response_id,
|
||||
)
|
||||
return result # type: ignore[return-value]
|
||||
except Exception as e:
|
||||
@@ -1102,6 +1116,7 @@ class BedrockCompletion(BaseLLM):
|
||||
tool_use_id = None
|
||||
elif "messageStop" in event:
|
||||
stop_reason = event["messageStop"].get("stopReason")
|
||||
stream_finish_reason = stop_reason
|
||||
logging.debug(f"Streaming message stopped: {stop_reason}")
|
||||
if stop_reason == "max_tokens":
|
||||
logging.warning(
|
||||
@@ -1147,6 +1162,8 @@ class BedrockCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=messages,
|
||||
usage=usage_data,
|
||||
finish_reason=stream_finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
return full_response
|
||||
@@ -1262,7 +1279,7 @@ class BedrockCompletion(BaseLLM):
|
||||
if usage:
|
||||
self._track_token_usage_internal(usage)
|
||||
|
||||
stop_reason = response.get("stopReason")
|
||||
stop_reason, response_id = self._extract_finish_reason_and_id(response)
|
||||
if stop_reason:
|
||||
logging.debug(f"Response stop reason: {stop_reason}")
|
||||
if stop_reason == "max_tokens":
|
||||
@@ -1300,6 +1317,8 @@ class BedrockCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=messages,
|
||||
usage=usage,
|
||||
finish_reason=stop_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return result
|
||||
except Exception as e:
|
||||
@@ -1322,6 +1341,8 @@ class BedrockCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=messages,
|
||||
usage=usage,
|
||||
finish_reason=stop_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return non_structured_output_tool_uses
|
||||
|
||||
@@ -1397,6 +1418,8 @@ class BedrockCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=messages,
|
||||
usage=usage,
|
||||
finish_reason=stop_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
return text_content
|
||||
@@ -1531,7 +1554,9 @@ class BedrockCompletion(BaseLLM):
|
||||
)
|
||||
|
||||
stream = response.get("stream")
|
||||
response_id = None
|
||||
_, stream_response_id = self._extract_finish_reason_and_id(response)
|
||||
response_id = stream_response_id
|
||||
stream_finish_reason: str | None = None
|
||||
if stream:
|
||||
async for event in stream:
|
||||
if "messageStart" in event:
|
||||
@@ -1623,6 +1648,9 @@ class BedrockCompletion(BaseLLM):
|
||||
result = response_model.model_validate(
|
||||
function_args
|
||||
)
|
||||
# contentBlockStop fires before messageStop sets
|
||||
# stream_finish_reason; structured output always
|
||||
# completes via the tool-call path.
|
||||
self._emit_call_completed_event(
|
||||
response=result.model_dump_json(),
|
||||
call_type=LLMCallType.LLM_CALL,
|
||||
@@ -1630,6 +1658,9 @@ class BedrockCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=messages,
|
||||
usage=usage_data,
|
||||
finish_reason=stream_finish_reason
|
||||
or "tool_use",
|
||||
response_id=response_id,
|
||||
)
|
||||
return result # type: ignore[return-value]
|
||||
except Exception as e:
|
||||
@@ -1687,6 +1718,7 @@ class BedrockCompletion(BaseLLM):
|
||||
|
||||
elif "messageStop" in event:
|
||||
stop_reason = event["messageStop"].get("stopReason")
|
||||
stream_finish_reason = stop_reason
|
||||
logging.debug(f"Streaming message stopped: {stop_reason}")
|
||||
if stop_reason == "max_tokens":
|
||||
logging.warning(
|
||||
@@ -1733,6 +1765,8 @@ class BedrockCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=messages,
|
||||
usage=usage_data,
|
||||
finish_reason=stream_finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
return self._invoke_after_llm_call_hooks(
|
||||
@@ -1988,6 +2022,25 @@ class BedrockCompletion(BaseLLM):
|
||||
|
||||
return config
|
||||
|
||||
@staticmethod
|
||||
def _extract_finish_reason_and_id(
|
||||
response: Any,
|
||||
) -> tuple[str | None, str | None]:
|
||||
"""Extract raw finish_reason (``stopReason``) from a Bedrock Converse
|
||||
response dict. Defensive — returns (None, None) on any failure.
|
||||
|
||||
Bedrock Converse has no model-level response id; ResponseMetadata.RequestId
|
||||
is an AWS infra trace id (semantically different from OpenAI's chatcmpl-XXX),
|
||||
so we omit response_id rather than mislead downstream telemetry consumers.
|
||||
"""
|
||||
finish_reason: str | None = None
|
||||
try:
|
||||
if isinstance(response, dict):
|
||||
finish_reason = response.get("stopReason")
|
||||
except (AttributeError, KeyError, TypeError, IndexError):
|
||||
finish_reason = None
|
||||
return finish_reason, None
|
||||
|
||||
def _handle_client_error(self, e: ClientError) -> str:
|
||||
"""Handle AWS ClientError with specific error codes and return error message."""
|
||||
error_code = e.response.get("Error", {}).get("Code", "Unknown")
|
||||
|
||||
@@ -682,6 +682,8 @@ class GeminiCompletion(BaseLLM):
|
||||
from_task: Any | None = None,
|
||||
from_agent: Any | None = None,
|
||||
usage: dict[str, Any] | None = None,
|
||||
finish_reason: str | None = None,
|
||||
response_id: str | None = None,
|
||||
) -> BaseModel:
|
||||
"""Validate content against response model and emit completion event.
|
||||
|
||||
@@ -691,6 +693,8 @@ class GeminiCompletion(BaseLLM):
|
||||
messages_for_event: Messages to include in event
|
||||
from_task: Task that initiated the call
|
||||
from_agent: Agent that initiated the call
|
||||
finish_reason: Raw provider finish reason.
|
||||
response_id: Raw provider response id.
|
||||
|
||||
Returns:
|
||||
Validated Pydantic model instance
|
||||
@@ -708,6 +712,8 @@ class GeminiCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=messages_for_event,
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
return structured_data
|
||||
@@ -724,6 +730,8 @@ class GeminiCompletion(BaseLLM):
|
||||
from_task: Any | None = None,
|
||||
from_agent: Any | None = None,
|
||||
usage: dict[str, Any] | None = None,
|
||||
finish_reason: str | None = None,
|
||||
response_id: str | None = None,
|
||||
) -> str | BaseModel:
|
||||
"""Finalize completion response with validation and event emission.
|
||||
|
||||
@@ -747,6 +755,8 @@ class GeminiCompletion(BaseLLM):
|
||||
from_task=from_task,
|
||||
from_agent=from_agent,
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
self._emit_call_completed_event(
|
||||
@@ -756,6 +766,8 @@ class GeminiCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=messages_for_event,
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
return self._invoke_after_llm_call_hooks(
|
||||
@@ -770,6 +782,8 @@ class GeminiCompletion(BaseLLM):
|
||||
from_task: Any | None = None,
|
||||
from_agent: Any | None = None,
|
||||
usage: dict[str, Any] | None = None,
|
||||
finish_reason: str | None = None,
|
||||
response_id: str | None = None,
|
||||
) -> BaseModel:
|
||||
"""Validate and emit event for structured_output tool call.
|
||||
|
||||
@@ -795,6 +809,8 @@ class GeminiCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=self._convert_contents_to_dict(contents),
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return validated_data
|
||||
except Exception as e:
|
||||
@@ -828,6 +844,8 @@ class GeminiCompletion(BaseLLM):
|
||||
Returns:
|
||||
Final response content or function call result
|
||||
"""
|
||||
finish_reason, response_id = self._extract_finish_reason_and_id(response)
|
||||
|
||||
if response.candidates and (self.tools or available_functions):
|
||||
candidate = response.candidates[0]
|
||||
if candidate.content and candidate.content.parts:
|
||||
@@ -854,6 +872,8 @@ class GeminiCompletion(BaseLLM):
|
||||
from_task=from_task,
|
||||
from_agent=from_agent,
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
non_structured_output_parts = [
|
||||
@@ -875,6 +895,8 @@ class GeminiCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=self._convert_contents_to_dict(contents),
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return non_structured_output_parts
|
||||
|
||||
@@ -915,6 +937,8 @@ class GeminiCompletion(BaseLLM):
|
||||
from_task=from_task,
|
||||
from_agent=from_agent,
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
def _process_stream_chunk(
|
||||
@@ -925,7 +949,13 @@ class GeminiCompletion(BaseLLM):
|
||||
usage_data: dict[str, int] | None,
|
||||
from_task: Any | None = None,
|
||||
from_agent: Any | None = None,
|
||||
) -> tuple[str, dict[int, dict[str, Any]], dict[str, int] | None]:
|
||||
) -> tuple[
|
||||
str,
|
||||
dict[int, dict[str, Any]],
|
||||
dict[str, int] | None,
|
||||
str | None,
|
||||
str | None,
|
||||
]:
|
||||
"""Process a single streaming chunk.
|
||||
|
||||
Args:
|
||||
@@ -937,9 +967,13 @@ class GeminiCompletion(BaseLLM):
|
||||
from_agent: Agent that initiated the call
|
||||
|
||||
Returns:
|
||||
Tuple of (updated full_response, updated function_calls, updated usage_data)
|
||||
Tuple of (updated full_response, updated function_calls, updated
|
||||
usage_data, chunk finish_reason, chunk response_id).
|
||||
"""
|
||||
response_id = chunk.response_id if hasattr(chunk, "response_id") else None
|
||||
chunk_finish_reason, chunk_response_id = self._extract_finish_reason_and_id(
|
||||
chunk
|
||||
)
|
||||
if chunk.usage_metadata:
|
||||
usage_data = self._extract_token_usage(chunk)
|
||||
|
||||
@@ -996,7 +1030,13 @@ class GeminiCompletion(BaseLLM):
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
return full_response, function_calls, usage_data
|
||||
return (
|
||||
full_response,
|
||||
function_calls,
|
||||
usage_data,
|
||||
chunk_finish_reason,
|
||||
chunk_response_id,
|
||||
)
|
||||
|
||||
def _finalize_streaming_response(
|
||||
self,
|
||||
@@ -1008,6 +1048,8 @@ class GeminiCompletion(BaseLLM):
|
||||
from_task: Any | None = None,
|
||||
from_agent: Any | None = None,
|
||||
response_model: type[BaseModel] | None = None,
|
||||
finish_reason: str | None = None,
|
||||
response_id: str | None = None,
|
||||
) -> str | BaseModel | list[dict[str, Any]]:
|
||||
"""Finalize streaming response with usage tracking, function execution, and events.
|
||||
|
||||
@@ -1038,6 +1080,8 @@ class GeminiCompletion(BaseLLM):
|
||||
from_task=from_task,
|
||||
from_agent=from_agent,
|
||||
usage=usage_data,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
non_structured_output_calls = {
|
||||
@@ -1058,6 +1102,8 @@ class GeminiCompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=self._convert_contents_to_dict(contents),
|
||||
usage=usage_data,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return raw_parts
|
||||
|
||||
@@ -1095,6 +1141,8 @@ class GeminiCompletion(BaseLLM):
|
||||
from_task=from_task,
|
||||
from_agent=from_agent,
|
||||
usage=usage_data,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
def _handle_completion(
|
||||
@@ -1148,6 +1196,8 @@ class GeminiCompletion(BaseLLM):
|
||||
full_response = ""
|
||||
function_calls: dict[int, dict[str, Any]] = {}
|
||||
usage_data: dict[str, int] | None = None
|
||||
stream_finish_reason: str | None = None
|
||||
stream_response_id: str | None = None
|
||||
|
||||
# The API accepts list[Content] but mypy is overly strict about variance
|
||||
contents_for_api: Any = contents
|
||||
@@ -1156,7 +1206,13 @@ class GeminiCompletion(BaseLLM):
|
||||
contents=contents_for_api,
|
||||
config=config,
|
||||
):
|
||||
full_response, function_calls, usage_data = self._process_stream_chunk(
|
||||
(
|
||||
full_response,
|
||||
function_calls,
|
||||
usage_data,
|
||||
chunk_finish_reason,
|
||||
chunk_response_id,
|
||||
) = self._process_stream_chunk(
|
||||
chunk=chunk,
|
||||
full_response=full_response,
|
||||
function_calls=function_calls,
|
||||
@@ -1164,6 +1220,10 @@ class GeminiCompletion(BaseLLM):
|
||||
from_task=from_task,
|
||||
from_agent=from_agent,
|
||||
)
|
||||
if chunk_finish_reason:
|
||||
stream_finish_reason = chunk_finish_reason
|
||||
if chunk_response_id:
|
||||
stream_response_id = chunk_response_id
|
||||
|
||||
return self._finalize_streaming_response(
|
||||
full_response=full_response,
|
||||
@@ -1174,6 +1234,8 @@ class GeminiCompletion(BaseLLM):
|
||||
from_task=from_task,
|
||||
from_agent=from_agent,
|
||||
response_model=response_model,
|
||||
finish_reason=stream_finish_reason,
|
||||
response_id=stream_response_id,
|
||||
)
|
||||
|
||||
async def _ahandle_completion(
|
||||
@@ -1227,6 +1289,8 @@ class GeminiCompletion(BaseLLM):
|
||||
full_response = ""
|
||||
function_calls: dict[int, dict[str, Any]] = {}
|
||||
usage_data: dict[str, int] | None = None
|
||||
stream_finish_reason: str | None = None
|
||||
stream_response_id: str | None = None
|
||||
|
||||
# The API accepts list[Content] but mypy is overly strict about variance
|
||||
contents_for_api: Any = contents
|
||||
@@ -1236,7 +1300,13 @@ class GeminiCompletion(BaseLLM):
|
||||
config=config,
|
||||
)
|
||||
async for chunk in stream:
|
||||
full_response, function_calls, usage_data = self._process_stream_chunk(
|
||||
(
|
||||
full_response,
|
||||
function_calls,
|
||||
usage_data,
|
||||
chunk_finish_reason,
|
||||
chunk_response_id,
|
||||
) = self._process_stream_chunk(
|
||||
chunk=chunk,
|
||||
full_response=full_response,
|
||||
function_calls=function_calls,
|
||||
@@ -1244,6 +1314,10 @@ class GeminiCompletion(BaseLLM):
|
||||
from_task=from_task,
|
||||
from_agent=from_agent,
|
||||
)
|
||||
if chunk_finish_reason:
|
||||
stream_finish_reason = chunk_finish_reason
|
||||
if chunk_response_id:
|
||||
stream_response_id = chunk_response_id
|
||||
|
||||
return self._finalize_streaming_response(
|
||||
full_response=full_response,
|
||||
@@ -1254,6 +1328,8 @@ class GeminiCompletion(BaseLLM):
|
||||
from_task=from_task,
|
||||
from_agent=from_agent,
|
||||
response_model=response_model,
|
||||
finish_reason=stream_finish_reason,
|
||||
response_id=stream_response_id,
|
||||
)
|
||||
|
||||
def supports_function_calling(self) -> bool:
|
||||
@@ -1300,6 +1376,34 @@ class GeminiCompletion(BaseLLM):
|
||||
|
||||
return int(1048576 * CONTEXT_WINDOW_USAGE_RATIO) # 1M tokens default
|
||||
|
||||
def _effective_max_tokens(self) -> int | float | None:
|
||||
"""Gemini caps generation via ``max_output_tokens``."""
|
||||
return self.max_output_tokens or self.max_tokens
|
||||
|
||||
@staticmethod
|
||||
def _extract_finish_reason_and_id(
|
||||
response: Any,
|
||||
) -> tuple[str | None, str | None]:
|
||||
"""Extract raw finish_reason and response_id from a Gemini
|
||||
``GenerateContentResponse``. ``finish_reason`` is the protobuf enum's
|
||||
``.name`` attribute (e.g. ``"STOP"``, ``"MAX_TOKENS"``); we forward
|
||||
it raw and let downstream telemetry map to the OTel GenAI enum.
|
||||
"""
|
||||
raw_response_id = getattr(response, "response_id", None)
|
||||
response_id = raw_response_id if isinstance(raw_response_id, str) else None
|
||||
|
||||
finish_reason: str | None = None
|
||||
candidates = getattr(response, "candidates", None)
|
||||
if candidates:
|
||||
try:
|
||||
candidate_finish = getattr(candidates[0], "finish_reason", None)
|
||||
except (IndexError, TypeError, KeyError):
|
||||
candidate_finish = None
|
||||
if candidate_finish is not None:
|
||||
name = getattr(candidate_finish, "name", None)
|
||||
finish_reason = name if isinstance(name, str) else None
|
||||
return finish_reason, response_id
|
||||
|
||||
@staticmethod
|
||||
def _extract_token_usage(response: GenerateContentResponse) -> dict[str, Any]:
|
||||
"""Extract token usage and response metadata from Gemini response."""
|
||||
|
||||
@@ -29,6 +29,7 @@ from openai.types.responses import (
|
||||
from pydantic import BaseModel, PrivateAttr, model_validator
|
||||
|
||||
from crewai.events.types.llm_events import LLMCallType
|
||||
from crewai.llms._finish_reason_utils import extract_choices_finish_reason_and_id
|
||||
from crewai.llms.base_llm import BaseLLM, JsonResponseFormat, llm_call_context
|
||||
from crewai.llms.hooks.base import BaseInterceptor
|
||||
from crewai.llms.hooks.transport import AsyncHTTPTransport, HTTPTransport
|
||||
@@ -825,6 +826,10 @@ class OpenAICompletion(BaseLLM):
|
||||
usage = self._extract_responses_token_usage(response)
|
||||
self._track_token_usage_internal(usage)
|
||||
|
||||
finish_reason, response_id = self._extract_responses_finish_reason_and_id(
|
||||
response
|
||||
)
|
||||
|
||||
if self.parse_tool_outputs:
|
||||
parsed_result = self._extract_builtin_tool_outputs(response)
|
||||
parsed_result.text = self._apply_stop_words(parsed_result.text)
|
||||
@@ -836,6 +841,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params.get("input", []),
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
return parsed_result
|
||||
@@ -849,6 +856,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params.get("input", []),
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return function_calls
|
||||
|
||||
@@ -887,6 +896,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params.get("input", []),
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return structured_result
|
||||
except ValueError as e:
|
||||
@@ -901,6 +912,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params.get("input", []),
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
content = self._invoke_after_llm_call_hooks(
|
||||
@@ -960,6 +973,10 @@ class OpenAICompletion(BaseLLM):
|
||||
usage = self._extract_responses_token_usage(response)
|
||||
self._track_token_usage_internal(usage)
|
||||
|
||||
finish_reason, response_id = self._extract_responses_finish_reason_and_id(
|
||||
response
|
||||
)
|
||||
|
||||
if self.parse_tool_outputs:
|
||||
parsed_result = self._extract_builtin_tool_outputs(response)
|
||||
parsed_result.text = self._apply_stop_words(parsed_result.text)
|
||||
@@ -971,6 +988,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params.get("input", []),
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
return parsed_result
|
||||
@@ -984,6 +1003,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params.get("input", []),
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return function_calls
|
||||
|
||||
@@ -1022,6 +1043,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params.get("input", []),
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return structured_result
|
||||
except ValueError as e:
|
||||
@@ -1036,6 +1059,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params.get("input", []),
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
except NotFoundError as e:
|
||||
@@ -1123,6 +1148,12 @@ class OpenAICompletion(BaseLLM):
|
||||
usage = self._extract_responses_token_usage(event.response)
|
||||
self._track_token_usage_internal(usage)
|
||||
|
||||
finish_reason, response_id = (
|
||||
self._extract_responses_finish_reason_and_id(final_response)
|
||||
if final_response is not None
|
||||
else (None, response_id_stream)
|
||||
)
|
||||
|
||||
if self.parse_tool_outputs and final_response:
|
||||
parsed_result = self._extract_builtin_tool_outputs(final_response)
|
||||
parsed_result.text = self._apply_stop_words(parsed_result.text)
|
||||
@@ -1134,6 +1165,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params.get("input", []),
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
return parsed_result
|
||||
@@ -1171,6 +1204,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params.get("input", []),
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return structured_result
|
||||
except ValueError as e:
|
||||
@@ -1185,6 +1220,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params.get("input", []),
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
return self._invoke_after_llm_call_hooks(
|
||||
@@ -1248,6 +1285,12 @@ class OpenAICompletion(BaseLLM):
|
||||
usage = self._extract_responses_token_usage(event.response)
|
||||
self._track_token_usage_internal(usage)
|
||||
|
||||
finish_reason, response_id = (
|
||||
self._extract_responses_finish_reason_and_id(final_response)
|
||||
if final_response is not None
|
||||
else (None, response_id_stream)
|
||||
)
|
||||
|
||||
if self.parse_tool_outputs and final_response:
|
||||
parsed_result = self._extract_builtin_tool_outputs(final_response)
|
||||
parsed_result.text = self._apply_stop_words(parsed_result.text)
|
||||
@@ -1259,6 +1302,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params.get("input", []),
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
return parsed_result
|
||||
@@ -1296,6 +1341,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params.get("input", []),
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return structured_result
|
||||
except ValueError as e:
|
||||
@@ -1310,6 +1357,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params.get("input", []),
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
return full_response
|
||||
@@ -1603,6 +1652,9 @@ class OpenAICompletion(BaseLLM):
|
||||
usage = self._extract_openai_token_usage(parsed_response)
|
||||
self._track_token_usage_internal(usage)
|
||||
|
||||
parsed_finish_reason, parsed_response_id = (
|
||||
self._extract_chat_finish_reason_and_id(parsed_response)
|
||||
)
|
||||
parsed_object = parsed_response.choices[0].message.parsed
|
||||
if parsed_object:
|
||||
self._emit_call_completed_event(
|
||||
@@ -1612,6 +1664,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=parsed_finish_reason,
|
||||
response_id=parsed_response_id,
|
||||
)
|
||||
return parsed_object
|
||||
|
||||
@@ -1625,6 +1679,9 @@ class OpenAICompletion(BaseLLM):
|
||||
|
||||
choice: Choice = response.choices[0]
|
||||
message = choice.message
|
||||
finish_reason, response_id = self._extract_chat_finish_reason_and_id(
|
||||
response
|
||||
)
|
||||
|
||||
# Without available_functions, return tool_calls so the caller (executor) handles execution
|
||||
if message.tool_calls and not available_functions:
|
||||
@@ -1635,6 +1692,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return list(message.tool_calls)
|
||||
|
||||
@@ -1675,6 +1734,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return structured_result
|
||||
except ValueError as e:
|
||||
@@ -1689,6 +1750,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
if usage.get("total_tokens", 0) > 0:
|
||||
@@ -1734,6 +1797,8 @@ class OpenAICompletion(BaseLLM):
|
||||
available_functions: dict[str, Any] | None = None,
|
||||
from_task: Any | None = None,
|
||||
from_agent: Any | None = None,
|
||||
finish_reason: str | None = None,
|
||||
response_id: str | None = None,
|
||||
) -> str | list[dict[str, Any]]:
|
||||
"""Finalize a streaming response with usage tracking, tool call handling, and events.
|
||||
|
||||
@@ -1745,6 +1810,9 @@ class OpenAICompletion(BaseLLM):
|
||||
available_functions: Available functions for tool calling.
|
||||
from_task: Task that initiated the call.
|
||||
from_agent: Agent that initiated the call.
|
||||
finish_reason: Raw provider finish reason (e.g. "stop", "length",
|
||||
"tool_calls") extracted from the last streaming chunk.
|
||||
response_id: Raw provider response id from any chunk.
|
||||
|
||||
Returns:
|
||||
Tool calls list when tools were invoked without available_functions,
|
||||
@@ -1774,6 +1842,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage_data,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return tool_calls_list
|
||||
|
||||
@@ -1817,6 +1887,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage_data,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
return full_response
|
||||
@@ -1861,6 +1933,9 @@ class OpenAICompletion(BaseLLM):
|
||||
if final_completion:
|
||||
usage = self._extract_openai_token_usage(final_completion)
|
||||
self._track_token_usage_internal(usage)
|
||||
parsed_finish_reason, parsed_response_id = (
|
||||
self._extract_chat_finish_reason_and_id(final_completion)
|
||||
)
|
||||
if final_completion.choices:
|
||||
parsed_result = final_completion.choices[0].message.parsed
|
||||
if parsed_result:
|
||||
@@ -1871,6 +1946,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=parsed_finish_reason,
|
||||
response_id=parsed_response_id,
|
||||
)
|
||||
return parsed_result
|
||||
|
||||
@@ -1882,11 +1959,15 @@ class OpenAICompletion(BaseLLM):
|
||||
)
|
||||
|
||||
usage_data: dict[str, Any] | None = None
|
||||
stream_finish_reason: str | None = None
|
||||
stream_response_id: str | None = None
|
||||
|
||||
for completion_chunk in completion_stream:
|
||||
response_id_stream = (
|
||||
completion_chunk.id if hasattr(completion_chunk, "id") else None
|
||||
)
|
||||
if response_id_stream:
|
||||
stream_response_id = response_id_stream
|
||||
|
||||
if hasattr(completion_chunk, "usage") and completion_chunk.usage:
|
||||
usage_data = self._extract_openai_token_usage(completion_chunk)
|
||||
@@ -1897,6 +1978,9 @@ class OpenAICompletion(BaseLLM):
|
||||
|
||||
choice = completion_chunk.choices[0]
|
||||
chunk_delta: ChoiceDelta = choice.delta
|
||||
chunk_finish = getattr(choice, "finish_reason", None)
|
||||
if chunk_finish:
|
||||
stream_finish_reason = chunk_finish
|
||||
|
||||
if chunk_delta.content:
|
||||
full_response += chunk_delta.content
|
||||
@@ -1954,6 +2038,8 @@ class OpenAICompletion(BaseLLM):
|
||||
available_functions=available_functions,
|
||||
from_task=from_task,
|
||||
from_agent=from_agent,
|
||||
finish_reason=stream_finish_reason,
|
||||
response_id=stream_response_id,
|
||||
)
|
||||
if isinstance(result, str):
|
||||
return self._invoke_after_llm_call_hooks(
|
||||
@@ -1989,6 +2075,9 @@ class OpenAICompletion(BaseLLM):
|
||||
usage = self._extract_openai_token_usage(parsed_response)
|
||||
self._track_token_usage_internal(usage)
|
||||
|
||||
parsed_finish_reason, parsed_response_id = (
|
||||
self._extract_chat_finish_reason_and_id(parsed_response)
|
||||
)
|
||||
parsed_object = parsed_response.choices[0].message.parsed
|
||||
if parsed_object:
|
||||
self._emit_call_completed_event(
|
||||
@@ -1998,6 +2087,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=parsed_finish_reason,
|
||||
response_id=parsed_response_id,
|
||||
)
|
||||
return parsed_object
|
||||
|
||||
@@ -2011,6 +2102,9 @@ class OpenAICompletion(BaseLLM):
|
||||
|
||||
choice: Choice = response.choices[0]
|
||||
message = choice.message
|
||||
finish_reason, response_id = self._extract_chat_finish_reason_and_id(
|
||||
response
|
||||
)
|
||||
|
||||
# Without available_functions, return tool_calls so the caller (executor) handles execution
|
||||
if message.tool_calls and not available_functions:
|
||||
@@ -2021,6 +2115,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return list(message.tool_calls)
|
||||
|
||||
@@ -2065,6 +2161,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
return structured_result
|
||||
except ValueError as e:
|
||||
@@ -2079,6 +2177,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage,
|
||||
finish_reason=finish_reason,
|
||||
response_id=response_id,
|
||||
)
|
||||
|
||||
if usage.get("total_tokens", 0) > 0:
|
||||
@@ -2130,8 +2230,12 @@ class OpenAICompletion(BaseLLM):
|
||||
|
||||
accumulated_content = ""
|
||||
usage_data: dict[str, Any] | None = None
|
||||
parsed_stream_finish_reason: str | None = None
|
||||
parsed_stream_response_id: str | None = None
|
||||
async for chunk in completion_stream:
|
||||
response_id_stream = chunk.id if hasattr(chunk, "id") else None
|
||||
if response_id_stream:
|
||||
parsed_stream_response_id = response_id_stream
|
||||
|
||||
if hasattr(chunk, "usage") and chunk.usage:
|
||||
usage_data = self._extract_openai_token_usage(chunk)
|
||||
@@ -2142,6 +2246,9 @@ class OpenAICompletion(BaseLLM):
|
||||
|
||||
choice = chunk.choices[0]
|
||||
delta: ChoiceDelta = choice.delta
|
||||
chunk_finish = getattr(choice, "finish_reason", None)
|
||||
if chunk_finish:
|
||||
parsed_stream_finish_reason = chunk_finish
|
||||
|
||||
if delta.content:
|
||||
accumulated_content += delta.content
|
||||
@@ -2165,6 +2272,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage_data,
|
||||
finish_reason=parsed_stream_finish_reason,
|
||||
response_id=parsed_stream_response_id,
|
||||
)
|
||||
|
||||
return parsed_object
|
||||
@@ -2177,6 +2286,8 @@ class OpenAICompletion(BaseLLM):
|
||||
from_agent=from_agent,
|
||||
messages=params["messages"],
|
||||
usage=usage_data,
|
||||
finish_reason=parsed_stream_finish_reason,
|
||||
response_id=parsed_stream_response_id,
|
||||
)
|
||||
return accumulated_content
|
||||
|
||||
@@ -2185,9 +2296,13 @@ class OpenAICompletion(BaseLLM):
|
||||
] = await self._get_async_client().chat.completions.create(**params)
|
||||
|
||||
usage_data = None
|
||||
stream_finish_reason: str | None = None
|
||||
stream_response_id: str | None = None
|
||||
|
||||
async for chunk in stream:
|
||||
response_id_stream = chunk.id if hasattr(chunk, "id") else None
|
||||
if response_id_stream:
|
||||
stream_response_id = response_id_stream
|
||||
|
||||
if hasattr(chunk, "usage") and chunk.usage:
|
||||
usage_data = self._extract_openai_token_usage(chunk)
|
||||
@@ -2198,6 +2313,9 @@ class OpenAICompletion(BaseLLM):
|
||||
|
||||
choice = chunk.choices[0]
|
||||
chunk_delta: ChoiceDelta = choice.delta
|
||||
chunk_finish = getattr(choice, "finish_reason", None)
|
||||
if chunk_finish:
|
||||
stream_finish_reason = chunk_finish
|
||||
|
||||
if chunk_delta.content:
|
||||
full_response += chunk_delta.content
|
||||
@@ -2255,6 +2373,8 @@ class OpenAICompletion(BaseLLM):
|
||||
available_functions=available_functions,
|
||||
from_task=from_task,
|
||||
from_agent=from_agent,
|
||||
finish_reason=stream_finish_reason,
|
||||
response_id=stream_response_id,
|
||||
)
|
||||
|
||||
def supports_function_calling(self) -> bool:
|
||||
@@ -2305,6 +2425,32 @@ class OpenAICompletion(BaseLLM):
|
||||
|
||||
return int(8192 * CONTEXT_WINDOW_USAGE_RATIO)
|
||||
|
||||
def _effective_max_tokens(self) -> int | float | None:
|
||||
"""Newer OpenAI chat models cap via ``max_completion_tokens``."""
|
||||
return self.max_tokens or self.max_completion_tokens
|
||||
|
||||
@staticmethod
|
||||
def _extract_chat_finish_reason_and_id(
|
||||
response: Any,
|
||||
) -> tuple[str | None, str | None]:
|
||||
"""ChatCompletion / ChatCompletionChunk share the choices-shape;
|
||||
delegate to the shared extractor.
|
||||
"""
|
||||
return extract_choices_finish_reason_and_id(response)
|
||||
|
||||
@staticmethod
|
||||
def _extract_responses_finish_reason_and_id(
|
||||
response: Any,
|
||||
) -> tuple[str | None, str | None]:
|
||||
"""Extract finish_reason and response_id from an OpenAI Responses
|
||||
API ``Response`` object. The Responses API exposes ``status`` rather
|
||||
than ``finish_reason``; we forward the raw status value.
|
||||
"""
|
||||
return (
|
||||
getattr(response, "status", None),
|
||||
getattr(response, "id", None),
|
||||
)
|
||||
|
||||
def _extract_openai_token_usage(
|
||||
self, response: ChatCompletion | ChatCompletionChunk
|
||||
) -> dict[str, Any]:
|
||||
|
||||
526
lib/crewai/tests/events/test_llm_finish_reason_response_id.py
Normal file
526
lib/crewai/tests/events/test_llm_finish_reason_response_id.py
Normal file
@@ -0,0 +1,526 @@
|
||||
from types import SimpleNamespace
|
||||
from typing import Any
|
||||
from unittest.mock import MagicMock, patch
|
||||
|
||||
import pytest
|
||||
|
||||
from crewai.events.event_bus import CrewAIEventsBus
|
||||
from crewai.events.types.llm_events import (
|
||||
LLMCallCompletedEvent,
|
||||
LLMCallStartedEvent,
|
||||
LLMCallType,
|
||||
LLMStreamChunkEvent,
|
||||
)
|
||||
from crewai.llm import LLM
|
||||
from crewai.llms._finish_reason_utils import extract_choices_finish_reason_and_id
|
||||
from crewai.llms.base_llm import BaseLLM
|
||||
|
||||
|
||||
class _StubLLM(BaseLLM):
|
||||
model: str = "test-model"
|
||||
|
||||
def call(self, *args: Any, **kwargs: Any) -> str:
|
||||
return ""
|
||||
|
||||
async def acall(self, *args: Any, **kwargs: Any) -> str:
|
||||
return ""
|
||||
|
||||
def supports_function_calling(self) -> bool:
|
||||
return False
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_emit():
|
||||
with patch.object(CrewAIEventsBus, "emit") as mock:
|
||||
yield mock
|
||||
|
||||
|
||||
class TestLLMCallCompletedEventFinishReasonAndResponseId:
|
||||
def test_accepts_string_values(self):
|
||||
event = LLMCallCompletedEvent(
|
||||
response="hi",
|
||||
call_type=LLMCallType.LLM_CALL,
|
||||
call_id="call-1",
|
||||
finish_reason="stop",
|
||||
response_id="resp_123",
|
||||
)
|
||||
assert event.finish_reason == "stop"
|
||||
assert event.response_id == "resp_123"
|
||||
|
||||
def test_defaults_to_none(self):
|
||||
event = LLMCallCompletedEvent(
|
||||
response="hi",
|
||||
call_type=LLMCallType.LLM_CALL,
|
||||
call_id="call-1",
|
||||
)
|
||||
assert event.finish_reason is None
|
||||
assert event.response_id is None
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"value",
|
||||
[MagicMock(), 42, 1.5, ["stop"], {"reason": "stop"}, object()],
|
||||
)
|
||||
def test_coerces_non_string_to_none(self, value):
|
||||
event = LLMCallCompletedEvent(
|
||||
response="hi",
|
||||
call_type=LLMCallType.LLM_CALL,
|
||||
call_id="call-1",
|
||||
finish_reason=value,
|
||||
response_id=value,
|
||||
)
|
||||
assert event.finish_reason is None
|
||||
assert event.response_id is None
|
||||
|
||||
|
||||
class TestLLMCallStartedEventSamplingParams:
|
||||
def test_accepts_all_sampling_params(self):
|
||||
event = LLMCallStartedEvent(
|
||||
call_id="call-1",
|
||||
temperature=0.7,
|
||||
top_p=0.9,
|
||||
max_tokens=512,
|
||||
stream=True,
|
||||
seed=42,
|
||||
stop_sequences=["END"],
|
||||
frequency_penalty=0.1,
|
||||
presence_penalty=0.2,
|
||||
n=3,
|
||||
)
|
||||
assert event.temperature == 0.7
|
||||
assert event.top_p == 0.9
|
||||
assert event.max_tokens == 512
|
||||
assert event.stream is True
|
||||
assert event.seed == 42
|
||||
assert event.stop_sequences == ["END"]
|
||||
assert event.frequency_penalty == 0.1
|
||||
assert event.presence_penalty == 0.2
|
||||
assert event.n == 3
|
||||
|
||||
def test_all_sampling_params_default_to_none(self):
|
||||
event = LLMCallStartedEvent(call_id="call-1")
|
||||
assert event.temperature is None
|
||||
assert event.top_p is None
|
||||
assert event.max_tokens is None
|
||||
assert event.stream is None
|
||||
assert event.seed is None
|
||||
assert event.stop_sequences is None
|
||||
assert event.frequency_penalty is None
|
||||
assert event.presence_penalty is None
|
||||
assert event.n is None
|
||||
|
||||
|
||||
class TestStopSequencesCoercion:
|
||||
# The OTel SDK falls back to str(value) when a span attribute isn't a
|
||||
# recognised Sequence[str], producing the protobuf textproto repr
|
||||
# ("values { string_value: ... }") in downstream telemetry. The
|
||||
# field_validator coerces exotic iterables (Vertex/Gemini protobuf
|
||||
# containers, tuples, generators) to a clean list[str] up front so the
|
||||
# OTel attribute is always shaped correctly.
|
||||
def test_bare_string_is_wrapped_in_list(self):
|
||||
event = LLMCallStartedEvent(call_id="call-1", stop_sequences="\nObservation:")
|
||||
assert event.stop_sequences == ["\nObservation:"]
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"raw, expected",
|
||||
[
|
||||
(["\nObservation:", "Final Answer:"], ["\nObservation:", "Final Answer:"]),
|
||||
(("\nObservation:",), ["\nObservation:"]),
|
||||
((s for s in ["a", "b"]), ["a", "b"]),
|
||||
([], []),
|
||||
],
|
||||
)
|
||||
def test_python_iterables_pass_through(
|
||||
self, raw: Any, expected: list[str]
|
||||
) -> None:
|
||||
event = LLMCallStartedEvent(call_id="call-1", stop_sequences=raw)
|
||||
assert event.stop_sequences == expected
|
||||
|
||||
def test_protobuf_like_repeated_container_is_coerced(self):
|
||||
# Mirrors google.protobuf RepeatedScalarContainer: iterable yielding
|
||||
# actual Python str objects. Should pass through cleanly.
|
||||
class _RepeatedScalar:
|
||||
def __init__(self, items: list[str]) -> None:
|
||||
self._items = items
|
||||
|
||||
def __iter__(self):
|
||||
return iter(self._items)
|
||||
|
||||
event = LLMCallStartedEvent(
|
||||
call_id="call-1",
|
||||
stop_sequences=_RepeatedScalar(["\nObservation:"]),
|
||||
)
|
||||
assert event.stop_sequences == ["\nObservation:"]
|
||||
|
||||
def test_protobuf_listvalue_with_nested_values_coerces_to_textproto_strings(self):
|
||||
# Mirrors google.protobuf.struct_pb2.ListValue: iterable yielding
|
||||
# `Value` messages whose str() is "string_value: \"...\"". The
|
||||
# coercion will str() each element, which is still wrong-shaped but
|
||||
# at least lands as a real list[str] for the OTel attribute instead
|
||||
# of a single textproto-blob string. Documents observed behaviour;
|
||||
# the upstream fix is to pass list[str] to LLM.stop, not ListValue.
|
||||
class _PbValue:
|
||||
def __init__(self, string_value: str) -> None:
|
||||
self.string_value = string_value
|
||||
|
||||
def __str__(self) -> str:
|
||||
return f'string_value: "{self.string_value}"'
|
||||
|
||||
class _PbListValue:
|
||||
def __init__(self, values: list[_PbValue]) -> None:
|
||||
self.values = values
|
||||
|
||||
def __iter__(self):
|
||||
return iter(self.values)
|
||||
|
||||
event = LLMCallStartedEvent(
|
||||
call_id="call-1",
|
||||
stop_sequences=_PbListValue([_PbValue("\\nObservation:")]),
|
||||
)
|
||||
assert event.stop_sequences == ['string_value: "\\nObservation:"']
|
||||
|
||||
@pytest.mark.parametrize("bad_input", [123, 12.5, object()])
|
||||
def test_non_iterable_falls_back_to_none(self, bad_input: Any) -> None:
|
||||
event = LLMCallStartedEvent(call_id="call-1", stop_sequences=bad_input)
|
||||
assert event.stop_sequences is None
|
||||
|
||||
def test_none_stays_none(self):
|
||||
event = LLMCallStartedEvent(call_id="call-1", stop_sequences=None)
|
||||
assert event.stop_sequences is None
|
||||
|
||||
|
||||
class TestEmitCallStartedEventIntrospectsSamplingParams:
|
||||
def test_reads_sampling_params_off_self(self, mock_emit):
|
||||
llm = _StubLLM(model="test-model", temperature=0.4)
|
||||
llm.top_p = 0.8
|
||||
llm.max_tokens = 256
|
||||
llm.stream = False
|
||||
llm.seed = 7
|
||||
llm.frequency_penalty = 0.5
|
||||
llm.presence_penalty = 0.6
|
||||
llm.n = 2
|
||||
llm.stop = ["STOP"]
|
||||
|
||||
llm._emit_call_started_event(messages="hi")
|
||||
|
||||
event = mock_emit.call_args[1]["event"]
|
||||
assert isinstance(event, LLMCallStartedEvent)
|
||||
assert event.temperature == 0.4
|
||||
assert event.top_p == 0.8
|
||||
assert event.max_tokens == 256
|
||||
assert event.stream is False
|
||||
assert event.seed == 7
|
||||
assert event.stop_sequences == ["STOP"]
|
||||
assert event.frequency_penalty == 0.5
|
||||
assert event.presence_penalty == 0.6
|
||||
assert event.n == 2
|
||||
|
||||
def test_explicit_kwargs_override_introspection(self, mock_emit):
|
||||
llm = _StubLLM(model="test-model", temperature=0.4)
|
||||
|
||||
llm._emit_call_started_event(messages="hi", temperature=0.9)
|
||||
|
||||
event = mock_emit.call_args[1]["event"]
|
||||
assert event.temperature == 0.9
|
||||
|
||||
|
||||
class TestBaseLLMSamplingParamFields:
|
||||
# Regression: PR #5945 review feedback. Sampling params are declared as
|
||||
# typed fields on BaseLLM so ``_emit_call_started_event`` reads them via
|
||||
# plain attribute access instead of getattr/hasattr fallbacks. Kwargs
|
||||
# like ``n=1`` bind directly to the typed field via Pydantic; there is
|
||||
# no promotion from ``additional_params``.
|
||||
def test_sampling_kwargs_bind_to_typed_fields(self, mock_emit):
|
||||
from crewai.llms.providers.openai.completion import OpenAICompletion
|
||||
|
||||
llm = LLM(model="gpt-4", n=1, temperature=0.5, seed=42)
|
||||
|
||||
assert isinstance(llm, OpenAICompletion)
|
||||
assert llm.n == 1
|
||||
assert llm.temperature == 0.5
|
||||
assert llm.seed == 42
|
||||
assert "n" not in llm.additional_params
|
||||
assert "temperature" not in llm.additional_params
|
||||
assert "seed" not in llm.additional_params
|
||||
|
||||
llm._emit_call_started_event(messages="hi")
|
||||
|
||||
event = mock_emit.call_args[1]["event"]
|
||||
assert isinstance(event, LLMCallStartedEvent)
|
||||
assert event.n == 1
|
||||
assert event.temperature == 0.5
|
||||
assert event.seed == 42
|
||||
|
||||
def test_additional_params_are_not_promoted_to_typed_fields(self, mock_emit):
|
||||
# Callers who pass sampling params through ``additional_params``
|
||||
# opt out of typed-field semantics. We intentionally do NOT promote
|
||||
# those values back into ``self.n`` / ``self.temperature``, so the
|
||||
# emitter sees ``None`` for those attributes. If a caller wants the
|
||||
# value surfaced in telemetry, they pass it as a kwarg.
|
||||
llm = LLM(
|
||||
model="gpt-4",
|
||||
additional_params={"n": 1, "temperature": 0.5, "seed": 42},
|
||||
)
|
||||
|
||||
assert llm.n is None
|
||||
assert llm.temperature is None
|
||||
assert llm.seed is None
|
||||
assert llm.additional_params == {"n": 1, "temperature": 0.5, "seed": 42}
|
||||
|
||||
llm._emit_call_started_event(messages="hi")
|
||||
|
||||
event = mock_emit.call_args[1]["event"]
|
||||
assert isinstance(event, LLMCallStartedEvent)
|
||||
assert event.n is None
|
||||
assert event.temperature is None
|
||||
assert event.seed is None
|
||||
|
||||
def test_emit_uses_call_scoped_stop_override(self, mock_emit):
|
||||
from crewai.llms.base_llm import call_stop_override
|
||||
|
||||
llm = _StubLLM(model="test-model", stop=["A"])
|
||||
|
||||
with call_stop_override(llm, ["X"]):
|
||||
llm._emit_call_started_event(messages="hi")
|
||||
|
||||
event = mock_emit.call_args[1]["event"]
|
||||
assert isinstance(event, LLMCallStartedEvent)
|
||||
assert event.stop_sequences == ["X"]
|
||||
# Instance-level stop is never mutated by the override.
|
||||
assert llm.stop == ["A"]
|
||||
|
||||
|
||||
class TestEffectiveMaxTokensTelemetry:
|
||||
def test_base_defaults_to_max_tokens(self, mock_emit):
|
||||
llm = _StubLLM(model="test-model", max_tokens=256)
|
||||
|
||||
llm._emit_call_started_event(messages="hi")
|
||||
|
||||
event = mock_emit.call_args[1]["event"]
|
||||
assert event.max_tokens == 256
|
||||
|
||||
def test_openai_surfaces_max_completion_tokens(self, mock_emit):
|
||||
from crewai.llms.providers.openai.completion import OpenAICompletion
|
||||
|
||||
llm = LLM(model="gpt-4o", max_completion_tokens=512)
|
||||
assert isinstance(llm, OpenAICompletion)
|
||||
assert llm.max_tokens is None
|
||||
|
||||
llm._emit_call_started_event(messages="hi")
|
||||
|
||||
event = mock_emit.call_args[1]["event"]
|
||||
assert event.max_tokens == 512
|
||||
|
||||
def test_explicit_max_tokens_takes_precedence(self, mock_emit):
|
||||
llm = LLM(model="gpt-4o", max_tokens=128, max_completion_tokens=512)
|
||||
|
||||
llm._emit_call_started_event(messages="hi")
|
||||
|
||||
event = mock_emit.call_args[1]["event"]
|
||||
assert event.max_tokens == 128
|
||||
|
||||
|
||||
class TestStreamingDictChunkResponseIdPropagation:
|
||||
# Regression: PR #5945 coderabbitai feedback. The streaming loop only
|
||||
# extracted ``chunk.id`` for ``ModelResponseBase`` instances; dict-shaped
|
||||
# chunks (LiteLLM emits these in some configs) silently dropped the id
|
||||
# and ``LLMStreamChunkEvent.response_id`` came through as ``None``.
|
||||
def _dict_chunks(self) -> list[dict[str, Any]]:
|
||||
return [
|
||||
{
|
||||
"id": "test-chunk-id",
|
||||
"choices": [{"delta": {"content": "hi"}, "finish_reason": None}],
|
||||
},
|
||||
{
|
||||
"id": "test-chunk-id",
|
||||
"choices": [{"delta": {"content": " there"}, "finish_reason": "stop"}],
|
||||
},
|
||||
]
|
||||
|
||||
def _stream_event_response_ids(self, mock_emit) -> list[str | None]:
|
||||
return [
|
||||
call.kwargs["event"].response_id
|
||||
for call in mock_emit.call_args_list
|
||||
if isinstance(call.kwargs.get("event"), LLMStreamChunkEvent)
|
||||
]
|
||||
|
||||
def test_sync_dict_chunk_id_propagates_to_stream_event(self, mock_emit):
|
||||
llm = LLM(model="gpt-4o-mini", is_litellm=True, stream=True)
|
||||
|
||||
with patch(
|
||||
"crewai.llm.litellm.completion",
|
||||
return_value=iter(self._dict_chunks()),
|
||||
):
|
||||
llm.call("anything")
|
||||
|
||||
ids = self._stream_event_response_ids(mock_emit)
|
||||
assert ids, "expected at least one LLMStreamChunkEvent"
|
||||
assert all(rid == "test-chunk-id" for rid in ids), ids
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_async_dict_chunk_id_propagates_to_stream_event(self, mock_emit):
|
||||
llm = LLM(model="gpt-4o-mini", is_litellm=True, stream=True)
|
||||
|
||||
async def _aiter():
|
||||
for chunk in self._dict_chunks():
|
||||
yield chunk
|
||||
|
||||
async def _acompletion(*_args, **_kwargs):
|
||||
return _aiter()
|
||||
|
||||
with patch("crewai.llm.litellm.acompletion", side_effect=_acompletion):
|
||||
await llm.acall("anything")
|
||||
|
||||
ids = self._stream_event_response_ids(mock_emit)
|
||||
assert ids, "expected at least one LLMStreamChunkEvent"
|
||||
assert all(rid == "test-chunk-id" for rid in ids), ids
|
||||
|
||||
|
||||
class TestEmitCallCompletedEventPassesFinishReasonAndResponseId:
|
||||
def test_passes_through_to_event(self, mock_emit):
|
||||
llm = _StubLLM(model="test-model")
|
||||
|
||||
llm._emit_call_completed_event(
|
||||
response="hi",
|
||||
call_type=LLMCallType.LLM_CALL,
|
||||
finish_reason="stop",
|
||||
response_id="resp_123",
|
||||
)
|
||||
|
||||
event = mock_emit.call_args[1]["event"]
|
||||
assert isinstance(event, LLMCallCompletedEvent)
|
||||
assert event.finish_reason == "stop"
|
||||
assert event.response_id == "resp_123"
|
||||
|
||||
def test_omitted_defaults_to_none(self, mock_emit):
|
||||
llm = _StubLLM(model="test-model")
|
||||
|
||||
llm._emit_call_completed_event(
|
||||
response="hi",
|
||||
call_type=LLMCallType.LLM_CALL,
|
||||
)
|
||||
|
||||
event = mock_emit.call_args[1]["event"]
|
||||
assert event.finish_reason is None
|
||||
assert event.response_id is None
|
||||
|
||||
|
||||
class TestLLMExtractFinishReasonAndResponseId:
|
||||
def test_non_streaming_litellm_shape(self):
|
||||
response = SimpleNamespace(
|
||||
id="chatcmpl-abc",
|
||||
choices=[SimpleNamespace(finish_reason="stop", message=SimpleNamespace())],
|
||||
)
|
||||
|
||||
finish_reason, response_id = LLM._extract_finish_reason_and_response_id(
|
||||
response
|
||||
)
|
||||
|
||||
assert finish_reason == "stop"
|
||||
assert response_id == "chatcmpl-abc"
|
||||
|
||||
def test_streaming_litellm_chunk_shape(self):
|
||||
last_chunk = SimpleNamespace(
|
||||
id="chatcmpl-stream-xyz",
|
||||
choices=[SimpleNamespace(finish_reason="tool_calls", delta=SimpleNamespace())],
|
||||
)
|
||||
|
||||
finish_reason, response_id = LLM._extract_finish_reason_and_response_id(
|
||||
last_chunk
|
||||
)
|
||||
|
||||
assert finish_reason == "tool_calls"
|
||||
assert response_id == "chatcmpl-stream-xyz"
|
||||
|
||||
def test_dict_shape(self):
|
||||
chunk = {
|
||||
"id": "chatcmpl-dict",
|
||||
"choices": [{"finish_reason": "length", "delta": {}}],
|
||||
}
|
||||
|
||||
finish_reason, response_id = LLM._extract_finish_reason_and_response_id(chunk)
|
||||
|
||||
assert finish_reason == "length"
|
||||
assert response_id == "chatcmpl-dict"
|
||||
|
||||
def test_missing_fields_return_none(self):
|
||||
finish_reason, response_id = LLM._extract_finish_reason_and_response_id(
|
||||
SimpleNamespace()
|
||||
)
|
||||
|
||||
assert finish_reason is None
|
||||
assert response_id is None
|
||||
|
||||
def test_non_string_values_coerced_to_none(self):
|
||||
response = SimpleNamespace(
|
||||
id=12345,
|
||||
choices=[SimpleNamespace(finish_reason=MagicMock(), delta=SimpleNamespace())],
|
||||
)
|
||||
|
||||
finish_reason, response_id = LLM._extract_finish_reason_and_response_id(
|
||||
response
|
||||
)
|
||||
|
||||
assert finish_reason is None
|
||||
assert response_id is None
|
||||
|
||||
def test_never_raises_on_unexpected_input(self):
|
||||
assert LLM._extract_finish_reason_and_response_id(None) == (None, None)
|
||||
assert LLM._extract_finish_reason_and_response_id(42) == (None, None)
|
||||
assert LLM._extract_finish_reason_and_response_id("string") == (None, None)
|
||||
|
||||
|
||||
class TestExtractChoicesFinishReasonAndIdHelper:
|
||||
# The shared extractor is consumed by LLM (LiteLLM), OpenAI Chat, and Azure.
|
||||
# TestLLMExtractFinishReasonAndResponseId exercises the choices-shape paths
|
||||
# transitively; these tests cover the direct-call surface and the
|
||||
# import contract.
|
||||
@pytest.mark.parametrize(
|
||||
"response, expected",
|
||||
[
|
||||
(
|
||||
SimpleNamespace(
|
||||
id="resp-1", choices=[SimpleNamespace(finish_reason="stop")]
|
||||
),
|
||||
("stop", "resp-1"),
|
||||
),
|
||||
(
|
||||
{"id": "resp-2", "choices": [{"finish_reason": "length"}]},
|
||||
("length", "resp-2"),
|
||||
),
|
||||
(
|
||||
SimpleNamespace(
|
||||
id="resp-3", choices=[{"finish_reason": "tool_calls"}]
|
||||
),
|
||||
("tool_calls", "resp-3"),
|
||||
),
|
||||
(
|
||||
{
|
||||
"id": "resp-4",
|
||||
"choices": [SimpleNamespace(finish_reason="content_filter")],
|
||||
},
|
||||
("content_filter", "resp-4"),
|
||||
),
|
||||
],
|
||||
)
|
||||
def test_extracts_choices_shape(
|
||||
self, response: Any, expected: tuple[str | None, str | None]
|
||||
) -> None:
|
||||
assert extract_choices_finish_reason_and_id(response) == expected
|
||||
|
||||
@pytest.mark.parametrize(
|
||||
"bad_input",
|
||||
[
|
||||
None,
|
||||
42,
|
||||
"string",
|
||||
{},
|
||||
SimpleNamespace(),
|
||||
SimpleNamespace(choices=[]),
|
||||
SimpleNamespace(choices=[SimpleNamespace()]),
|
||||
{"id": 12345, "choices": [{"finish_reason": MagicMock()}]},
|
||||
],
|
||||
)
|
||||
def test_never_raises_returns_nones_or_coerces(self, bad_input: Any) -> None:
|
||||
finish_reason, response_id = extract_choices_finish_reason_and_id(bad_input)
|
||||
assert finish_reason is None or isinstance(finish_reason, str)
|
||||
assert response_id is None or isinstance(response_id, str)
|
||||
@@ -122,6 +122,20 @@ def test_gemini_completion_initialization_parameters():
|
||||
assert llm.top_k == 40
|
||||
|
||||
|
||||
def test_gemini_started_event_surfaces_max_output_tokens():
|
||||
from crewai.events.event_bus import CrewAIEventsBus
|
||||
from crewai.events.types.llm_events import LLMCallStartedEvent
|
||||
|
||||
llm = LLM(model="google/gemini-2.0-flash-001", max_output_tokens=2000, api_key="test-key")
|
||||
|
||||
with patch.object(CrewAIEventsBus, "emit") as mock_emit:
|
||||
llm._emit_call_started_event(messages="hi")
|
||||
|
||||
event = mock_emit.call_args[1]["event"]
|
||||
assert isinstance(event, LLMCallStartedEvent)
|
||||
assert event.max_tokens == 2000
|
||||
|
||||
|
||||
def test_gemini_specific_parameters():
|
||||
"""
|
||||
Test Gemini-specific parameters like stop_sequences, streaming, and safety settings
|
||||
|
||||
96
lib/crewai/tests/test_llm_streaming_finish_reason.py
Normal file
96
lib/crewai/tests/test_llm_streaming_finish_reason.py
Normal file
@@ -0,0 +1,96 @@
|
||||
"""Regression: LiteLLM emits a final usage-only chunk (choices=[]) when
|
||||
``stream_options.include_usage`` is set. The old post-loop
|
||||
``_extract_finish_reason_and_response_id(last_chunk)`` then silently returned
|
||||
(None, None). These tests pin that we capture finish_reason/response_id
|
||||
incrementally during the stream loop instead.
|
||||
"""
|
||||
from __future__ import annotations
|
||||
|
||||
from typing import Any
|
||||
from unittest.mock import patch
|
||||
|
||||
import pytest
|
||||
|
||||
from crewai.events.event_bus import CrewAIEventsBus
|
||||
from crewai.events.types.llm_events import LLMCallCompletedEvent
|
||||
from crewai.llm import LLM
|
||||
|
||||
|
||||
@pytest.fixture
|
||||
def mock_emit():
|
||||
with patch.object(CrewAIEventsBus, "emit") as mock:
|
||||
yield mock
|
||||
|
||||
|
||||
def _completed_event(mock_emit) -> LLMCallCompletedEvent:
|
||||
matches = [
|
||||
call.kwargs["event"]
|
||||
for call in mock_emit.call_args_list
|
||||
if isinstance(call.kwargs.get("event"), LLMCallCompletedEvent)
|
||||
]
|
||||
assert matches, "expected an LLMCallCompletedEvent to be emitted"
|
||||
assert len(matches) == 1, f"expected one completed event, got {len(matches)}"
|
||||
return matches[0]
|
||||
|
||||
|
||||
def _chunks_with_usage_tail() -> list[dict[str, Any]]:
|
||||
"""Three-chunk stream mirroring LiteLLM's include_usage behavior:
|
||||
two content chunks where the second carries finish_reason="stop",
|
||||
then a final usage-only chunk with choices=[]."""
|
||||
return [
|
||||
{
|
||||
"id": "chatcmpl-stream-1",
|
||||
"choices": [
|
||||
{"delta": {"content": "hi"}, "finish_reason": None}
|
||||
],
|
||||
},
|
||||
{
|
||||
"id": "chatcmpl-stream-1",
|
||||
"choices": [
|
||||
{"delta": {"content": " there"}, "finish_reason": "stop"}
|
||||
],
|
||||
},
|
||||
{
|
||||
"id": "chatcmpl-stream-1",
|
||||
"choices": [],
|
||||
"usage": {
|
||||
"prompt_tokens": 1,
|
||||
"completion_tokens": 2,
|
||||
"total_tokens": 3,
|
||||
},
|
||||
},
|
||||
]
|
||||
|
||||
|
||||
def test_sync_stream_emits_finish_reason_and_response_id_from_loop(mock_emit):
|
||||
llm = LLM(model="gpt-4o-mini", is_litellm=True, stream=True)
|
||||
|
||||
with patch("crewai.llm.litellm.completion", return_value=iter(_chunks_with_usage_tail())):
|
||||
result = llm.call("anything")
|
||||
|
||||
assert result == "hi there"
|
||||
|
||||
event = _completed_event(mock_emit)
|
||||
assert event.finish_reason == "stop"
|
||||
assert event.response_id == "chatcmpl-stream-1"
|
||||
|
||||
|
||||
@pytest.mark.asyncio
|
||||
async def test_async_stream_emits_finish_reason_and_response_id_from_loop(mock_emit):
|
||||
llm = LLM(model="gpt-4o-mini", is_litellm=True, stream=True)
|
||||
|
||||
async def _aiter():
|
||||
for chunk in _chunks_with_usage_tail():
|
||||
yield chunk
|
||||
|
||||
async def _acompletion(*_args, **_kwargs):
|
||||
return _aiter()
|
||||
|
||||
with patch("crewai.llm.litellm.acompletion", side_effect=_acompletion):
|
||||
result = await llm.acall("anything")
|
||||
|
||||
assert result == "hi there"
|
||||
|
||||
event = _completed_event(mock_emit)
|
||||
assert event.finish_reason == "stop"
|
||||
assert event.response_id == "chatcmpl-stream-1"
|
||||
Reference in New Issue
Block a user