* feat(llm): llm_overlay maps models too, not only agent roles
A key `model:<provider/model>` maps every LLM built from that model string
inside the block, and `model:*` every LLM built from any model string. That
reaches what a role cannot: a flow step's own `LLM(...).call()`. The key is
read in `LLM.__new__` (and so `create_llm` and an agent's `llm="..."`), and
on an agent's declared llm built outside the block; a role key still wins for
that agent. The caller's settings follow the mapped model by the rule a
declared llm's do. `MODEL_KEY_PREFIX` is public so another package can
feature-detect the form.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(telemetry): allow-list cli_usage:eval_models attributes
The span `crewai eval --models` records carries whether the caller was
logged in, the models compared (provider/model names) and how many, in both
telemetry classes. Nothing about the run, its output or the organization.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(cli): crewai eval --models compares models on the deployment
`crewai eval --models "a,b"` (and `--deployment UUID` when AMP cannot tell
the deployment from the project id) asks AMP for a models evaluation: the
project id, the models from one comma-separated list (stripped, repeats
dropped, provider/model required, 1-5) and the project's eval.jsonc. It
needs the login, prints and opens the report link, says where the
comparison is while it runs, then prints one row per model (goal, tasks,
agents, tools, cost, time; the deployed one marked, the best per column
starred) and the top three suggestions. Exit 0 when it finished, 1 when it
failed or could not start. Everything from the wire is printed as Text.
The comparison is counted as `cli_usage:eval_models` once AMP accepts it.
Mode 1 now sends the project id with `create_evaluation` too, so every
evaluation is filed under the project it is about.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(cli): crewai deploy create and push send the project id
Every create (git or ZIP) and every push (redeploy by name or uuid, or a
ZIP update) carries `[tool.crewai].project_id` when the project has one, so
AMP knows which project a deployment runs and `crewai eval --models` can find
it. The id is read, never minted: a deploy does not rewrite pyproject.toml.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(llm): a model key maps an LLM whose declared SDK is not installed
Nothing is built on the declared model, so a missing SDK for it is no
reason to fail the mapped build; its credentials cannot be matched to the
new provider, so none are carried.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* feat(cli): crewai eval --models leaves its criteria behind as eval.jsonc
When the comparison's answer carries `eval_config` and the project has no
eval.jsonc, it is written through `write_eval_config` and the same one line
Mode 1 prints says so. An existing file is never overwritten.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(llm): a model key compares providers by provider, not by class
`_same_route_provider` now follows `_same_provider`'s rule case for case: a
LiteLLM-routed `LLM(...)` is on the same provider as a native route when its
provider names that native class, so its key and endpoint follow a model
key to another model of the same provider. Another provider still gets none.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* chore(llm): say why mark_mapped may skip an object, and one import form in its tests
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(cli): usage stats name a compared model only when it is a public one
`cli_usage:eval_models` carried the `--models` strings as typed, and those
can identify a customer: a fine-tune id, an Azure deployment name, a
self-hosted model or host. A model is now sent by name only when it is an
exact entry of crewAI's model catalog (`LLM_CONTEXT_WINDOW_SIZES`), else as
`<provider>/other`, and a provider crewAI does not route as `other/other`.
An `ft:` id is never sent.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(cli): a refused comparison's test no longer keeps a None return
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* test(llms): re-importing OpenAI's completion module restores its package attribute
`test_openai_completion_module_is_imported` deletes the module from
sys.modules and lets `LLM(...)` import it again, which binds the new module
on `crewai.llms.providers.openai` as well. monkeypatch restored only the
sys.modules entry, so every later test on that worker saw two module
objects for one name: `patch("crewai.llms.providers.openai.completion.X")`
patched the attribute's module while `from ... import X` read sys.modules'.
On Python 3.10 that made tests/llms/azure/test_azure_responses.py fail
whenever it shared a worker with test_openai.py and test_crew_loader.py,
which the new tests' even split now did. The attribute is restored too.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(llm): model keys strip only a model's own prefix; a role builds from the declaration
Two review findings on the model-for-model overlay:
- `overlay_model_for_model` took the text after the first slash as another
name for the model, so an aggregator's route (`openrouter/openai/gpt-4o`)
matched a `model:openai/gpt-4o` or `model:gpt-4o` key meant for the
native model. A prefix is now stripped only when it is a native provider's
own and what remains has no slash; an instance on an aggregator is compared
as `<provider>/<its model id>`.
- With a role key and a model key both active, the model key mapped the
declared llm first and the role's model was built like that mapped
instance, losing the caller's key and endpoint. The overlay now remembers
what the caller declared for every llm it built, and a role key is built
from that declaration.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(llm): model keys read a provider's other names; a LiteLLM choice survives a missing SDK
- The router's provider aliases are now one table, `crewai.llm.PROVIDER_ALIASES`,
and `llm_overlay` compares model names through it: `google/x` and
`gemini/x` are one model whether the llm is built in the block or was
built before it (a prebuilt instance records `gemini`). An exact key still
wins, and an aggregator's route is still compared whole.
- When the declared model's SDK is missing, a caller's explicit
`is_litellm=True` is kept for the mapped model instead of dropping to a
native route.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(deploy): a git create sends the project id once, at the top level
AMP reads project_id at the top level on every deploy route, so the git
create no longer repeats it inside `deploy`. The redeploy docstring records
why its optional body is safe on an AMP that predates project ids.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* fix(deps): pin instructor <1.16 to keep OpenAI Mode.TOOLS registered
instructor 1.16.0 removed the OpenAI Mode.TOOLS handler in favor of
Mode.RESPONSES_TOOLS. `InternalInstructor` calls `instructor.from_provider`,
which still defaults to Mode.TOOLS internally, so any native-OpenAI kickoff
that hits a structured-output path (guardrails, output_pydantic, LLM
guardrails) fails immediately with `ModeError: Mode.TOOLS is not registered
for provider Provider.OPENAI`.
Cap the range while we teach `_create_instructor_client` to negotiate a
supported mode against `mode_registry`.
Refs COR-861 (Docusign, deployment 134108).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
* fix: avoid redundant cache saves in tests workflow
Co-authored-by: vinibrsl <5093045+vinibrsl@users.noreply.github.com>
* docs: explain instructor compatibility pin
Co-authored-by: vinibrsl <5093045+vinibrsl@users.noreply.github.com>
---------
Co-authored-by: Daniel Minella <dminella@Mac.home>
Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Co-authored-by: copilot-swe-agent[bot] <198982749+Copilot@users.noreply.github.com>
* chore(ci): ignore unpatched nltk GHSA-8mgp-746c-j5xp
No patched PyPI release exists beyond 3.10.3. nltk is transitive via
crewai-tools[xml] -> unstructured; CrewAI does not call the vulnerable
model-artifact APIs.
Co-authored-by: Vidit Ostwal <Vidit-Ostwal@users.noreply.github.com>
* chore(ci): note dropping nltk GHSA ignore on the next bump
Leave an explicit TODO beside the ignore so GHSA-8mgp-746c-j5xp is
removed when nltk moves past the unpatched 3.10.3 floor.
---------
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Vidit Ostwal <Vidit-Ostwal@users.noreply.github.com>
* ci: require an open issue for first-time contributor PRs
Gate anyone who is not a returning contributor, and allow the PR only when a closing keyword points at an open issue in this repo.
* ci: accept any open issue mention for first-timer PRs
Drop the closing-keyword regex so #123, owner/repo#N, or an issue URL is enough when that issue is open.
* ci: ignore foreign owner/repo#N in first-timer issue gate
Bare #123 no longer matches the suffix of other/repo#123, so an open local issue cannot keep that PR open.
Force torch>=2.13.0 via override-dependencies so the transitive
docling/unstructured stack picks up the CVE-2025-3000 fix, and drop the
now-unnecessary pip-audit ignore. chromadb's CVE-2026-45829 remains
ignored: the upstream fix is merged but not released on PyPI.
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Rip&Tear <theCyberTech@users.noreply.github.com>
Skipped matrix jobs never post status for branch-protection names like
tests (3.10). Add lightweight skip jobs with the same names so Actions-only
and docs-only PRs can merge without waiting forever.
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Rip&Tear <theCyberTech@users.noreply.github.com>
Exclude .github/** from the code path filter used by tests, lint,
type-checker, and vulnerability-scan so workflow-only changes (including
CodeQL) do not run the Python matrix. CodeQL itself is unchanged and still
analyzes Actions YAML on those PRs.
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Rip&Tear <theCyberTech@users.noreply.github.com>
Co-authored-by: Vidit Ostwal <110953813+Vidit-Ostwal@users.noreply.github.com>
Add a patch-minor-updates group for routine version bumps while keeping
the existing security-updates grouping. Ignore semver-major updates so
breaking upgrades stay manual.
dorny/paths-filter treats exclusion-only patterns as matching every
non-excluded path under the default "some" quantifier, so docs-only
PRs still set code=true. Add an explicit "**" include and use
predicate-quantifier: every in tests, type-checker, and linter.
Also gate Vulnerability Scan the same way on pull_request while
keeping schedule and main push runs unconditional.
Co-authored-by: Cursor Agent <cursoragent@cursor.com>
Co-authored-by: Rip&Tear <theCyberTech@users.noreply.github.com>
`mintlify broken-links` walks every frozen version snapshot under
`docs/` (currently 22 of them), pushing docs PR checks past 14 minutes
and growing with each release. Prune the immutable snapshots — keeping
`edge` and the default (latest) version, which unprefixed links resolve
against — before running the checker, cutting the run to ~30 seconds.
`workflow_dispatch` still checks the full tree, and the deprecated
`mintlify` CLI is swapped for `mint`, which drops the `yes` prompt hack.
* chore: bump json-repair to 0.60.1 and un-ignore fixed vulns in scan
- json-repair 0.25.3 -> 0.60.1 (fixes GHSA-xf7x-x43h-rpqh)
- pyOpenSSL already at 26.2.0 in lock (covers CVE-2026-27448, CVE-2026-27459)
- remove the corresponding --ignore-vuln flags from vulnerability-scan.yml
* fix: adapt _safe_repair_json to json-repair 0.60 semantics
json-repair >= 0.60 returns an empty string for plain-text input and
wraps brace-enclosed junk in a single-element list instead of the old
""/{} sentinel values. Treat both as unrepairable so the original
tool input is preserved.
* chore: fix CI - bump gitpython/pyasn1, drop stale type ignores
- gitpython 3.1.50 -> 3.1.52 (GHSA-2f96-g7mh-g2hx, GHSA-v396-v7q4-x2qj,
GHSA-956x-8gvw-wg5v; fixed in 3.1.51)
- pyasn1 0.6.3 -> 0.6.4 (GHSA-8ppf-4f7h-5ppj, GHSA-hm4w-wwcw-mr6r)
- json-repair 0.60 ships type stubs; remove now-unused
type: ignore[import-untyped] comments flagged by mypy
The vulnerability scan started failing when PYSEC-2026-2132 (click) and
PYSEC-2026-2253..2257 (pillow) were published on Jul 12. Both have fixed
releases within our constraints, so `uv.lock` upgrades click to 8.4.2 and
pillow to 12.3.0. A newer json-repair advisory (GHSA-xf7x-x43h-rpqh) also
surfaced; its fix is outside the `json-repair~=0.25.2` pin and 0.25.x lacks
the vulnerable `schema_repair` module, so it joins the ignore list in
`vulnerability-scan.yml` with a justification.
* ci: ignore GHSA-rrmf-rvhw-rf47 (torch alias of PYSEC-2025-194)
pip-audit reports CVE-2025-3000 under its GHSA id, which the existing
PYSEC-2025-194 ignore does not match. Same advisory: memory corruption
in torch.jit.script, CVSS 1.9, local-only, no fix for torch 2.11.0.
* ci: sync GHSA-rrmf-rvhw-rf47 ignore into pre-commit pip-audit
uv 0.11.7 -> 0.11.17 patches GHSA-4gg8-gxpx-9rph. chromadb has no
patched release for GHSA-f4j7-r4q5-qw2c (server-only pre-auth RCE,
not reachable in our embedded use); ignore until upstream ships a fix.
* ci: pin third-party actions to commit SHAs
Pin third-party GitHub Actions in workflow files to immutable 40-char
commit SHAs per the org security policy. Mutable refs like @v4 can be
silently re-pointed by a compromised upstream; SHAs cannot. Trailing
version comments let Dependabot/Renovate continue to manage updates.
Related to [COR-51](https://linear.app/crewai/issue/COR-51).
* ci: disable persist-credentials in pip-audit checkout
Address CodeRabbit feedback on PR #5869: the pip-audit workflow is
read-only and never needs an authenticated git context, so opt out of
persisting the GITHUB_TOKEN in the local git config per the
actions/checkout security guidance.
OSV no longer flags pip 26.1.1 (GHSA-58qw-9mgm-455v) or paramiko
5.0.0 (GHSA-r374-rxx8-8654), so override both to those minimums
and remove the corresponding --ignore-vuln entries. paramiko is
pulled in transitively via composio-core.
Adds joblib, markdown, nltk, onnx, pyjwt, torch and transformers
advisories that have no fixed version available (or are disputed)
to the pip-audit ignore list. Rationale recorded next to each ID.
Replaces version tags (e.g. astral-sh/setup-uv@v6, slackapi/slack-github-action@v2.1.0)
with full commit SHAs across every workflow. Mitigates supply-chain risk from
mutable tags.
GitHub doesn't expose repo secrets to pull_request events from forks, so
${{ secrets.CREWAI_TOOL_SPECS_APP_ID }} resolves to an empty string and
tibdex/github-app-token@v2 errors with "Input required and not supplied:
app_id". The job also tries to push commits to the PR branch, which it
can't do on a fork regardless. Skip it for cross-repo PRs and keep it
for same-repo PRs and manual dispatch.
Co-authored-by: Greyson LaLonde <greyson.r.lalonde@gmail.com>
* perf: defer MCP SDK import by fixing import path in agent/core.py
- Change 'from crewai.mcp import MCPServerConfig' to direct path
'from crewai.mcp.config import MCPServerConfig' to avoid triggering
mcp/__init__.py which eagerly loads the full mcp SDK (~300-400ms)
- Move MCPToolResolver import into get_mcp_tools() method body since
it's only used at runtime, not in type annotations
Saves ~200ms on 'import crewai' cold start.
* perf: lazy-load heavy MCP imports in mcp/__init__.py
MCPClient, MCPToolResolver, BaseTransport, and TransportType now use
__getattr__ lazy loading. These pull in the full mcp SDK (~400ms) but
are only needed at runtime when agents actually connect to MCP servers.
Lightweight config and filter types remain eagerly imported.
* perf: lazy-load all event type modules in events/__init__.py
Previously only agent_events were lazy-loaded; all other event type
modules (crew, flow, knowledge, llm, guardrail, logging, mcp, memory,
reasoning, skill, task, tool_usage) were eagerly imported at package
init time. Since events/__init__.py runs whenever ANY crewai.events.*
submodule is accessed, this loaded ~12 Pydantic model modules
unnecessarily.
Now all event types use the same __getattr__ lazy-loading pattern,
with TYPE_CHECKING imports preserved for IDE/type-checker support.
Saves ~550ms on 'import crewai' cold start.
* chore: remove UNKNOWN.egg-info from version control
* fix: add MCPToolResolver to TYPE_CHECKING imports
Fixes F821 (ruff) and name-defined (mypy) from lazy-loading the
MCP import. The type annotation on _mcp_resolver needs the name
available at type-check time.
* fix: bump lxml to >=5.4.0 for GHSA-vfmq-68hx-4jfw
lxml 5.3.2 has a known vulnerability. Bump to 5.4.0+ which
includes the fix (libxml2 2.13.8). The previous <5.4.0 pin
was for etree import issues that have since been resolved.
* fix: bump exclude-newer to 2026-04-22 for lxml 6.1.0 resolution
lxml 6.1.0 (GHSA fix) was released April 17 but the exclude-newer
date was set to April 17, missing it by timestamp. Bump to April 22.
* perf: add import time benchmark script
scripts/benchmark_import_time.py measures import crewai cold start
in fresh subprocesses. Supports --runs, --json (for CI), and
--threshold (fail if median exceeds N seconds).
The companion GitHub Action workflow needs to be pushed separately
(requires workflow scope).
* new action
* Potential fix for pull request finding 'CodeQL / Workflow does not contain permissions'
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
---------
Co-authored-by: Joao Moura <joaomdmoura@gmail.com>
Co-authored-by: Copilot Autofix powered by AI <62310815+github-advanced-security[bot]@users.noreply.github.com>
* fix: bump litellm to >=1.83.0 to address CVE-2026-35030
Bump litellm from <=1.82.6 to >=1.83.0 to fix JWT auth bypass via
OIDC cache key collision (CVE-2026-35030). Also widen devtools openai
pin from ~=1.83.0 to >=1.83.0,<3 to resolve the version conflict
(litellm 1.83.0 requires openai>=2.8.0).
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
* fix: resolve mypy errors from litellm bump
- Remove unused type: ignore[import-untyped] on instructor import
- Remove all unused type: ignore[union-attr] comments (litellm types fixed)
- Add hasattr guard for tool_call.function — new litellm adds
ChatCompletionMessageCustomToolCall to the union which lacks .function
* fix: tighten litellm pin to ~=1.83.0 (patch-only bumps)
>=1.83.0,<2 is too wide — litellm has had breaking changes between
minors. ~=1.83.0 means >=1.83.0,<1.84.0 — gets CVE patches but won't
pull in breaking minor releases.
* ci: bump uv from 0.8.4 to 0.11.3
* fix: resolve mypy errors in openai completion from 2.x type changes
Use isinstance checks with concrete openai response types instead of
string comparisons for proper type narrowing. Update code interpreter
handling for outputs/OutputImage API changes in openai 2.x.
* fix: pre-cache tiktoken encoding before VCR intercepts requests
---------
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-authored-by: Alex <alex@crewai.com>
Co-authored-by: Greyson LaLonde <greyson@crewai.com>
* ci: add vulnerability scanning with pip-audit and Snyk
Add a new GitHub Actions workflow that runs on PRs, pushes to main, and weekly:
- pip-audit: scans all Python dependencies (direct + transitive) against
PyPI Advisory DB and OSV for known CVEs. Outputs JSON report as artifact
and posts results to the job summary.
- Snyk: optional enterprise-grade scanning (gated behind SNYK_ENABLED
repo variable and SNYK_TOKEN secret). Runs on high+ severity and
monitors main branch.
This addresses the need for automated pre-release vulnerability scanning
to catch dependency CVEs before cutting releases.
* ci: pin Snyk action to @v1 tag and remove continue-on-error
- Pin snyk/actions/python from @master to @v1 to prevent supply chain
risk from mutable branch references (matches convention of other
actions in the repo using versioned tags)
- Remove continue-on-error on the Snyk check step so high+ severity
vulnerabilities actually fail the build
* ci: fail build when pip-audit crashes without producing a report
If pip-audit exits abnormally without writing pip-audit-report.json,
the Display Results step now emits an error annotation and exits 1
instead of silently passing.
* ci: fix pip-audit failing on local packages
Replace --strict with --skip-editable to avoid pip-audit failing when
it encounters local/private packages (e.g. crewai-devtools) that are
not published on PyPI. The --skip-editable flag tells pip-audit to
skip packages installed in editable/development mode while still
auditing all published dependencies.
* fix: bump vulnerable dependencies and ignore unfixable CVEs
Dependency upgrades (via uv lock --upgrade-package):
- aiohttp 3.13.3 → 3.13.5 (fixes 10 CVEs)
- cryptography 46.0.5 → 46.0.6 (fixes CVE-2026-34073)
- pygments 2.19.2 → 2.20.0 (fixes CVE-2026-4539)
- onnx 1.20.1 → 1.21.0 (fixes 6 CVEs)
- couchbase 4.5.0 → 4.6.0 (fixes PYSEC-2023-235)
Temporarily ignored CVEs (cannot be fixed without upstream changes):
- CVE-2025-69872 (diskcache): no fix available, latest version
- CVE-2026-25645 (requests): needs 2.33.0, blocked by crewai-tools pin
- CVE-2026-27448/27459 (pyopenssl): needs 26.0.0, blocked by
snowflake-connector-python pin
- PYSEC-2023-235 (couchbase): advisory not yet updated for 4.6.0
* chore: remove accidentally committed egg-info files
* ci: remove Snyk job, pip-audit is sufficient
pip-audit covers Python dependency CVE scanning against PyPI Advisory DB
and OSV, which is all we need for pre-release checks. Snyk adds
complexity (account setup, token management) without meaningful
additional coverage for this use case.
---------
Co-authored-by: Greyson LaLonde <greyson.r.lalonde@gmail.com>
Vidit Ostwal