mirror of
https://github.com/crewAIInc/crewAI.git
synced 2026-10-22 18:03:10 +02:00
a8877f0cdfbca4fc4b95e3ce54aaa69aafca7688
3010
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
|
a8877f0cdf |
feat(cli): every crewai eval keeps the evaluation's full data in the project (#8326)
CodeQL Advanced / Analyze (actions) (push) Canceled after 0s
CodeQL Advanced / Analyze (python) (push) Canceled after 0s
Check Documentation Broken Links / Check broken links (push) Canceled after 0s
Vulnerability Scan / Detect changes (push) Canceled after 0s
Vulnerability Scan / pip-audit (push) Canceled after 0s
* feat(cli): every crewai eval keeps the evaluation's full data in the project
An evaluation's report opens for one five-minute sitting after it is first
read, and the verdict `crewai eval` waits for is that read. A coding agent
that wants a check's quotes, the outputs or the trace after its edits ran
`crewai eval export` and found it closed ("has been read already").
Right after the verdict, `crewai eval` now saves the export into
`.crewai/evaluations/<id>/` (report.md, run/checks.md, run/outputs.md,
run/trace.json, beside a `.gitignore` of `*`) and says where. Graded again
with nothing changed, it points at what it saved instead of asking AMP a
second time. Only inside a project; the ten newest are kept. A service that
does not export yet says nothing, and any other failure is one line: the
verdict and the exit code are never the save's.
The export writer now raises an error its callers decide about instead of
exiting, and writes into a nested folder the same way it wrote into one:
each folder opened inside the one before it, never through a link.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cli): a saved evaluation is marked whole, pruned only by its owner, and fetched briefly
Review on #8326:
- the save asks AMP with its own short timeout (20 s to read) instead of
export's two minutes: the verdict is already printed, and CI waits on the
command (`PlusAPI.export_evaluation(..., timeout=)`);
- a marker is written last, and only when every file was; it, not report.md,
says an evaluation is saved, so a save cut short is tried again;
- a save that skipped a file says "Part of its data was saved … N skipped"
and leaves no marker, instead of claiming the full data;
- only a folder this command saved whole (the marker, a plain file) is ever
removed to keep the newest ten; never a link or a junction, and nothing at
all when `.crewai/` or `.crewai/evaluations/` is a link;
- the path writer (Windows) refuses a link anywhere on the folder's way
before making anything through it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cli): AMP cannot send a saved evaluation's marker
Review on #8326: `_export_target` reserved the folder's `.gitignore` but not
the marker that says a save was whole. A payload carrying
`.crewai-eval-saved` and one file that is skipped would have left that
marker on a partial save: read as whole, never tried again, and kept over a
complete one. The marker is now reserved the same way, compared as a
case-insensitive filesystem that drops trailing dots sees it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
94508c971f |
feat(cli): with no terminal, crewai eval sends the crew's own output to a log (#8325)
* feat(cli): with no terminal, crewai eval sends the crew's own output to a log A coding agent reads every line `crewai eval` prints, and pays for it. When `crewai eval` ran the crew itself, the crew's panels, final answer and tool calls came first: 341 of 378 lines in a measured run, all before the verdict, and an agent that cuts long output could lose the verdict and the brief at the end. With no terminal, what the crew prints while it runs now goes to `.crewai/last_run.log`, and the agent gets one line saying so, one saying how long the run took, then the verdict and brief as before: 40 lines for the same crew. A run that leaves no trace to evaluate shows its last 40 lines, which usually hold the error. At a terminal nothing changes. It moves this process's stdout and stderr while the crew runs, which every child inherits, so every way a project runs is covered; anything that keeps them from moving leaves the output on screen. `.crewai/` is already left out of the project-changed check, so the log never triggers a re-run. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(cli): the run log is the owner's alone, and both streams always come back Review on #8325: - the run log holds final answers, tool output and tracebacks that can carry a credential: it is now opened owner-only (0600) and never through a link, and a log an earlier run left is emptied and tightened; - both descriptors are saved before either moves, and a move that fails halfway puts back what it moved, so the verdict still reaches the screen; - putting them back is best effort for each on its own and never raises, so the run's own error is the one that surfaces; - whether to keep the transcript off screen is decided by stdout, as for the brief (`nobody_watching()`): an agent can hand the command a terminal for stdin and still read a captured stdout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * refactor(cli): the run log is a file object closed by its with; a test's raise moves into a helper Code-quality review on #8325: the log descriptor was closed by hand on every path; it is now a file object (os.fdopen) the redirect holds in a with, so it is closed however the block ends. The put-back test raised inline inside pytest.raises, which reads as unreachable code after it; the raise is now in a helper. No behaviour changes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
dec075b5e4 |
fix(llms): use auto tool choice for a single Anthropic tool (#8282)
Newer Claude models reject a forced named tool_choice, and one executable tool should not require the model to call it. |
||
|
|
b523e6124f | docs: fix dead repo links in observability guides (Braintrust SDK, OpenInference) (#8310) | ||
|
|
4f4573706e |
fix: treat checkpoint paths literally in glob patterns (#8269)
JsonProvider.prune() and the checkpoint CLI joined the checkpoint location (and branch) with "*.json" or "**/*.json" and passed the result to glob.glob(), so glob characters in the path were read as a pattern. For "runs[1]", pruning deleted the checkpoints in "runs1" and kept its own, and the CLI list, info and prune commands read "runs1". Escape the directory part, so that only the file name part is a pattern. |
||
|
|
ecddbf2af7 | test(llms): cover earliest stop-word matching (#8287) | ||
|
|
37f0bd32ce | fix(cli): keep runtime input prompts on stderr (#8297) | ||
|
|
add3f7bf1e | fix(agents): restore task description when replanning fails (#8308) | ||
|
|
78cc9d9e95 |
fix(memory): preserve records when update serialization fails (#8303)
Build uv cache / build-cache (3.10) (push) Canceled after 0s
Build uv cache / build-cache (3.11) (push) Canceled after 0s
Build uv cache / build-cache (3.12) (push) Canceled after 0s
Build uv cache / build-cache (3.13) (push) Canceled after 0s
CodeQL Advanced / Analyze (actions) (push) Canceled after 0s
CodeQL Advanced / Analyze (python) (push) Canceled after 0s
Check Documentation Broken Links / Check broken links (push) Canceled after 0s
Vulnerability Scan / Detect changes (push) Canceled after 0s
Vulnerability Scan / pip-audit (push) Canceled after 0s
Serialize replacement rows before deleting saved memories so invalid metadata cannot erase the original record. Add regression coverage for serialization failures and successful replacement. Fixes #8302 Signed-off-by: vatsalyd <vatsal.y.official@gmail.com> |
||
|
|
c08008b624 |
fix(tools): default-deny non-global IPs in SSRF check (#7826)
* fix(tools): default-deny non-global IPs in SSRF check is_blocked_ip used hand-written denylists that missed CGNAT (100.64.0.0/10, incl. Alibaba metadata 100.100.100.200), 198.18.0.0/15, Azure WireServer (168.63.129.16), and IPv6 forms wrapping private IPv4 (NAT64, 6to4). Block anything that is not is_global, keep explicit entries for ranges is_global may allow or that vary by Python version, and unwrap IPv4-mapped and NAT64 (64:ff9b::/96) addresses so the embedded IPv4 is checked. 6to4, Teredo and local-use NAT64 are non-global and blocked. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * docs(tools): note CGNAT is blocked by SSRF-safe scrape tools Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> * fix(tools): block IPv6 transition ranges explicitly in is_blocked_ip 6to4 and local-use NAT64 were is_global=True before the CVE-2024-4032 fix (Python <3.10.15 / <3.11.10), so they relied on the interpreter to be blocked. Add them, Teredo, IPv4-compatible (::/96), IPv4-translated (::ffff:0:0:0/96) and deprecated site-local (fec0::/10) to the explicit IPv6 list, which also closes forms is_global allows on every version. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Sonnet 5.5 <noreply@anthropic.com> |
||
|
|
aaa33e03e0 |
fix(cli): crewai eval says why before it asks, and its help says when it asks (#8301)
At a terminal, the first time tracing is turned on, `crewai eval` showed
crewAI's trace consent screen ("Detailed execution traces are
available! ... Share this execution trace with CrewAI?") with nothing
before it, though no crew had run yet. It now says first why the crew
runs and where its trace goes ("No traced run of this crew yet. To
evaluate it, the crew runs now and its trace goes to CrewAI AMP, which
grades it."), and does not repeat the run line after a yes. Without a
terminal nothing changes: no screen, no extra line.
`crewai eval --help` still said "Nothing is asked, with or without a
terminal". It now says a person at a terminal is asked the first time
tracing is turned on, and that a project that turned tracing off is
asked at a terminal and told how without one.
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
87c2868123 |
feat(cli): rewrite the scaffolded AGENTS.md and fail PRs that teach coding agents less (#8298)
* feat(cli): rewrite the scaffolded AGENTS.md around the CrewAI + AMP workflow The template opened with an API reference and mentioned traces, eval and deployment as optional extras, so coding agents rarely used them. It now opens with the steps an agent follows in a session (set up, reuse what the organization has on AMP, run, trace and evaluate every change, deploy when the work has to run elsewhere), then a short reference of the facts models get wrong. About 3.3K tokens instead of 11.9K, with outdated claims corrected. Tests assert the behaviours the old wording carried, check every documented `crewai` command against the CLI, and keep numbered lists gapless with and without the evaluation blocks. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(cli): index the features AGENTS.md no longer details, and guard the file The workflow rewrite dropped references to guardrails, knowledge, MCP, conversational and declarative flows, checkpoints and more. An "Also available" index under Reference names each one with its API and docs path. Tests now guard the shipped file on every PR: a reference manifest, a token budget, text that is wrong on main, imports that must resolve, python blocks that must parse, and docs paths that must exist. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(cli): add a core API section to AGENTS.md for models that don't know CrewAI Measured with OpenAI models, the workflow rewrite lost the basics they don't know without the file: flow decorators and state, the @CrewBase class, task fields, CrewOutput and custom tools. A short Core API section restores them and folds in the two crew and router facts it would otherwise repeat. The reference manifest now guards those core references too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: run the AGENTS.md guards on PRs that touch the template The guards live in lib/cli/tests, which the main test workflow doesn't run. A path-filtered job runs them when the template, its tests or the docs pages it links change. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: run the AGENTS.md workflow only on pull requests against main Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(cli): close the gaps where AGENTS.md taught less than the file it replaces Question by question against today's file, V6 trailed on facts it had cut or worded loosely: the context window and date options, task output folders and human review, the guardrail's return shape, test targets before writes, the eval's structural fixes, logged-in trace visibility, partial selector misses, written IDs for silent failures, ZIP deploys with an origin, the flow memory root, and LiteLLM for providers outside the native set. Each is back, verified on main. Measured on Claude Code (Haiku, Sonnet) and Codex, no question now scores below today's file. The token budget rises to 4,800 to hold them. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(cli): teach the hooks and planning that telemetry counts, and three facts the old file had The index named kickoff hooks and Crew(planning=True), but the features most projects use are interception hooks (@on(InterceptionPoint...) in crewai.hooks) and agent planning (Agent(planning_config=PlanningConfig(...))). Both lines now lead with those. Three facts today's file taught come back: OpenAI's Responses API, agents judge while code computes, and answering a paused deployed flow through AMP's human_feedback_requests API. Measured on 221 questions in Claude Code (Haiku, Sonnet) and Codex, no question scores below today's file. The budget rises to 5,000 tokens, guards cover the new lines, and the never-teach list allows naming reasoning=True as replaced. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(devtools): agents-md-eval fails a PR whose AGENTS.md teaches coding agents less `devtools agents-md-eval` asks a 218-question bank in fresh, tool-less Claude Code and Codex sessions, once with the base branch's AGENTS.md and once with this branch's, grades each answer by its key, and exits non-zero when either agent scores more than 10% below the base. The AGENTS.md workflow runs it on PRs that change the template, the eval or its questions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(devtools): run the AGENTS.md eval per model and count sessions with no answer The action asks Claude Code with the latest Haiku and Sonnet and Codex with gpt-5.6-luna and gpt-5.6-sol, one job per model. A session that hangs, fails or returns nothing is retried once and then counted; past 2% unanswered the comparison is called unreliable instead of scored. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * ci: run the AGENTS.md eval only on pull requests against main Drops the manual trigger and the conditions it needed; a test pins the trigger. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(devtools): say why AGENTS.md eval sessions gave no answer Each unanswered session records its exit code and last error line (keys redacted) or a timeout, and the command prints the most common reasons, so a run called unreliable can be diagnosed from the CI log. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(devtools): grade 'is a different command' as a warning, accept 'call the agent directly' Two answer-key misses found reading V6's answers: a correct answer that warns off `crewai update` tripped its trap, and "No. Call the agent directly" didn't count as needing no crew. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(devtools): B-065's trap fires only on advice to run crewai update Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(devtools): read a failed session's reason from stdout when stderr is empty Claude Code prints some failures, such as 'Not logged in', on stdout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(devtools): accept 'never a fix' in C-018's key Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(devtools): read typographic apostrophes and 'replaces' as warnings in traps Codex writes "don’t"; the negation check knew only "don't", so its warnings counted as recommendations. "X replaces reasoning=True" now reads as a warning too. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(devtools): add hooks and agent planning questions; D-018 needs the tools, not the package name Three questions for features telemetry counts and the bank missed: tool-call hooks, LLM-call hooks and agent planning (221 questions). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * feat(cli): AGENTS.md teaches that crewai eval runs the crew itself, logged in or out Since #8270, `crewai eval` runs the crew, traced, when the project changed since its last traced run. The file still said to `crewai run` first, and that a run without a terminal is traced only when logged in, so logged-out coding agents logged in or stopped instead of evaluating. It now says `crewai eval` works logged in or out, with or without a terminal, with no `crewai run` between rounds and no wait on a login. The CLI test's command parser stops at a group that runs without a subcommand (`crewai eval`, `crewai eval --models ...`), which `crewai eval` became when it gained `export`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(devtools): ask whether crewai eval needs a login, and read "doesn't" as a warning Two questions join the bank: can a logged-out coding agent with no terminal get a verdict, and does it need `crewai run` first. Three keys follow main: C-004 and B-051 accept `crewai eval` (it runs and traces the crew), C-032 accepts "flow code". Negations now include doesn't, didn't, aren't and can't, so "evaluation doesn't require an account" no longer trips the trap it warns against. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(ci): run the AGENTS.md eval from the base commit, with the PR's files as data The eval job checked out the pull request and ran its devtools with both API keys in the environment, so a branch could print or send them. It now checks out the base commit, fetches the pull request's commit only to read its AGENTS.md and CLAUDE.md (`agents-md-eval --head <sha>`), and runs in the `agents-md-eval` environment, where the keys can sit behind a required reviewer. When the base branch has no eval yet, the job skips with a notice. The eval also names a session's real error: Claude Code can end its output with a JSON log line after the message ("Invalid API key"), which hid every Haiku failure. Item text and reasons print as text, so `[tool.crewai]` isn't dropped and `[/...]` can't raise MarkupError after the paid runs. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(cli): AGENTS.md says deploys build from any git origin, not only GitHub `crewai deploy create` sends whatever `origin` URL the project has, and AMP builds from GitHub or another git host (`any_git`). The file said an origin meant GitHub, which pointed GitLab and other users at the wrong fix. The GitHub integration note now applies to GitHub only, and a guard keeps it so. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(ci): run the AGENTS.md eval on pull_request_target, so its keys reach only reviewed code Under pull_request a branch's own copy of the workflow runs, so a pull request could edit the eval's steps and still get the keys. The eval now has its own workflow on pull_request_target: GitHub runs it from the base branch, the job checks out the base commit, and the pull request's AGENTS.md and CLAUDE.md are fetched and read as data, never run. It runs for this repository's branches only, since pull_request_target would hand the keys to forks too, and it fetches the pull request's commit only once it knows it will run, so a skip can't fail on the fetch. The guards stay on pull_request, with no secrets, and no longer persist the checkout's token for the pull request's code to read. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(devtools): accept "go-ahead" as asking first in C-052's key "Publishing writes to your organization, so it needs your go-ahead first" asks before publishing, but the key only matched ask, approve, confirm or permission, so correct answers scored wrong. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
d06c91401f |
feat(flow): add opt-in per-turn conversation tracing (#8291)
* feat(flow): add opt-in per-turn conversation tracing * fix(flow): avoid completing turns paused for human feedback |
||
|
|
ef11fa70e2 |
feat(tools): add Azure DocumentDB vector search integration (#8273)
* feat(tools): add Azure DocumentDB vector search integration Add native vector search, document ingestion, IVF indexing, and renewable Entra ID authentication for Azure DocumentDB and Azure OpenAI. Include optional dependencies, public exports, tool metadata, documentation, and unit, Podman, and live Azure tests. Refs crewAIInc/crewAI#7302 Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> * docs(tools): complete Azure DocumentDB function docstrings Document tool helpers, OIDC callbacks, and tests to satisfy the PR docstring coverage threshold. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> --------- Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> |
||
|
|
c6e500ab37 |
feat(rag): add read-only notebook text loader (#8188)
CodeQL Advanced / Analyze (actions) (push) Canceled after 0s
CodeQL Advanced / Analyze (python) (push) Canceled after 0s
Vulnerability Scan / Detect changes (push) Canceled after 0s
Vulnerability Scan / pip-audit (push) Canceled after 0s
Build uv cache / build-cache (3.10) (push) Canceled after 0s
Build uv cache / build-cache (3.11) (push) Canceled after 0s
Build uv cache / build-cache (3.12) (push) Canceled after 0s
Build uv cache / build-cache (3.13) (push) Canceled after 0s
* feat(rag): add read-only notebook text loader * fix(rag): validate notebook cells and metadata * docs(rag): document notebook loader functions and tests * fix(rag): accept utf-8 bom in notebook files --------- Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com> |
||
|
|
abb82030cb |
feat(rag): add standard-library jsonl loader (#8144)
* feat(rag): add standard-library jsonl loader * fix(rag): enforce strict jsonl record validation * docs(rag): document jsonl loader functions and tests --------- Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com> |
||
|
|
65cdd70ac0 | fix(deps): update aiohttp security floor (#8288) | ||
|
|
b1c9279918 |
fix: store deferred tool results in tmp (#8229)
* fix: store deferred tool results in tmp * chore: remove tool result artifact diagnostic |
||
|
|
2282ab658b |
Fix JSONLoader escaping non-ASCII text when rendering JSON documents (#8278)
* Preserve non-ASCII characters in JSON loading Updated json_loader.py to ensure non-ASCII characters are preserved when converting JSON data to text. * Fix BOM handling in JSON loader tests * Add tests for JSONLoader with non-ASCII characters Add tests to verify JSONLoader handles non-ASCII text correctly. * Preserve lone surrogate escapes in JSONLoader output Added encoding to keep lone surrogates escaped in JSON output. * Test JSONLoader scalar Unicode and surrogate handling |
||
|
|
0b84f8e18b |
fix(core): send AMP requests through the proxy the environment names (#8186)
* fix(core): send AMP requests through the proxy the environment names PlusAPI built its httpx clients with trust_env=False, so HTTP(S)_PROXY, NO_PROXY and SSL_CERT_FILE were ignored and every AMP call tried a direct connection. On a corporate network or inside a coding-agent sandbox (Claude Code, Codex), which only allow traffic through that proxy, deploy, eval, org, triggers, trace upload, skills and repository agents all failed with "[Errno 8] nodename nor servname provided", while `crewai login` (module-level httpx.post) worked. trust_env=False dates from #1455, when the client used requests, where it also stopped ~/.netrc credentials from replacing the AMP bearer token. httpx 0.28 no longer reads .netrc, so the flag now only disables proxy and CA-bundle variables. Remove it from the three clients (core request and multipart, and the CLI's multipart override). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(core): keep the direct-request proxy test off the OS proxy settings With no proxy variables set, httpx falls back to urllib's getproxies(), which reads the macOS or Windows system proxy settings. On a machine with a system proxy configured, the "no proxy variables" test would have routed through it. Model a machine with none configured. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(cli): keep a project's .env from choosing how crewai eval sends the login crewai eval already refuses to send the saved login to an AMP host that only the project's .env names. With AMP requests now honoring proxy and CA variables, the same .env could still pick the route: its own proxy and its own CA bundle, past that check. eval and eval --models now leave *_proxy, SSL_CERT_FILE and SSL_CERT_DIR out when they load the project's .env, say so, and keep the machine's own settings (shell exports, OS proxy). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(core): take AMP's proxy and CA from the machine, not a project's .env AMP requests carry the saved login. Honoring HTTP(S)_PROXY and SSL_CERT_FILE from the live environment let a cloned project's .env choose both: `crewai run` on a JSON crew or a declarative flow loads it with override=True, and `crewai.llm` loads it again on import, which also undid the filter `crewai eval` applied to its own loader. PlusAPI now pins the proxy and CA variables when crewai_core.plus_api is first imported, which the CLI does at startup, and hands them to httpx explicitly with trust_env=False. With none set, the OS proxy settings apply, read live. The eval loader goes back to main's, since the client now covers every command. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(cli): pin AMP's proxy and CA in the JSON-crew runner too `crewai run` runs a JSON crew in a fresh `uv run python -c` interpreter that never imports cli.py, and `_run_json_crew` loads the project's .env with override=True before tracing first imports crewai_core.plus_api, so the pin recorded the project's proxy and CA. The runner now imports plus_api before anything else. Declarative flows re-run through `uv run crewai run`, which already pins in cli.py. Also keeps `_requests_sent_by` off machine and OS proxies, so a developer's proxy can't mount a real transport ahead of its MockTransport. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(core): keep urllib's CGI proxy rule, and surface a broken plus_api import The pinned proxy map dropped urllib's REQUEST_METHOD rule: in a CGI process an uppercase HTTP_PROXY can come from a client's Proxy header, and AMP's login and deployment ZIPs would follow it. REQUEST_METHOD is now pinned with the other transport variables, and the map is read exactly as urllib reads it. The JSON-crew runner skipped any ImportError from crewai_core.plus_api. It now skips only a crewai-core too old to have the module; any other failure stops the run with its own error instead of running with nothing pinned. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(cli): import the proxy pin with the project folder off sys.path `python -c` puts the working directory, the project, first on sys.path, so a project folder named crewai_core could supply the module that pins where the saved login travels. The JSON-crew runner now imports the pin with the project off the path and restores the path right after, so project code (custom tools importing their neighbours) still finds its own modules. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(core): pin AMP's proxy and CA first thing in `import crewai` A crew run in its own process (`uv run run_crew`) never imports the CLI, so `import crewai` is where the pin happens there. It already preceded the project's .env, loaded by crewai.llm on import, but only by import order. The pin is now the first import in crewai/__init__.py, and a test runs a fresh interpreter in a project whose .env names a proxy and CA: the .env loads, and the pin still holds only what the machine set. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(cli): keep the machine's proxy and CA for the crew `crewai eval` runs `crewai eval` loads the project's .env into its own process and then runs the crew in a child that inherits that environment. The child pins what it inherits, so a .env naming HTTPS_PROXY and SSL_CERT_FILE became the run's machine transport, and that run's traces carry the saved login. The proxy and CA variables go back to the machine's pin right after the .env loads; the rest of the .env still applies, and the run loads the .env itself after its own pin, as under `crewai run`. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(core): keep a project's UV_ENV_FILE out of the crew `crewai eval` runs `crewai eval` loads the project's .env and starts the crew with `uv run`. A .env naming itself as UV_ENV_FILE had uv load it into the run before Python started, so the run pinned the project's proxy and CA anyway. UV_ENV_FILE now counts among the variables restored to the machine's. The pin test starts the run through `uv run`, as the CLI does. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
3da49f872b |
feat(cli): crewai eval runs the crew itself, so a coding agent gets a verdict from one command (#8270)
Vulnerability Scan / Detect changes (push) Canceled after 0s
Vulnerability Scan / pip-audit (push) Canceled after 0s
CodeQL Advanced / Analyze (actions) (push) Canceled after 0s
CodeQL Advanced / Analyze (python) (push) Canceled after 0s
Check Documentation Broken Links / Check broken links (push) Canceled after 0s
* feat(cli): crewai eval runs the crew itself, so a coding agent gets a verdict from one command
Most `crewai eval` attempts stopped before an evaluation began because the
project had no traced run. Telemetry since Sept 25: 117 projects stopped as
"untraced", 97 of them with no terminal (coding agents), and 67 of those never
got further. With no terminal the command could only print steps, and without
a login those steps could not work: an unattended anonymous run never uploads
its trace.
- With no traced run, or when the project's files changed since the last one,
`crewai eval` runs the crew and evaluates that run. No question, with or
without a terminal: running the command is the request to have the run
graded, and the grade is read off its trace. Tracing is on for that run only;
nothing is written to the project's `.env`.
- crewAI's tracing approves the upload of that run through the existing
handshake: a token in CREWAI_EVAL_AWAITING_RUN that counts only while its file
exists in the user's crewAI data folder, so a `.env` cannot claim it. Shared
as `crewai_core.eval_handshake`; the run app keeps its own copy because it
also runs against a project's older crewai-core, and a test pins the two.
- The run app's Evaluate on an untraced run closes the app and runs the crew
again, traced, through `crewai eval`'s own path, instead of a notice.
- Counts: `eval_ran_crew:{no_traced_run,project_changed,run_again}`,
`eval_finished:{passed,failed,inconclusive,unknown}`, and
`eval_stopped:untraced_after_run` replacing the catch-all `untraced`
(`declined` is gone with the question). The CLI tests no longer send usage
counts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cli): a project that turned tracing off is asked before crewai eval runs it
`CREWAI_TRACING_ENABLED=false` is the project's no, and nothing overrules it,
not even a run `crewai eval` started.
- `crewai eval` at a terminal asks to turn tracing on before running the crew.
A yes writes it to the project's `.env`; a no runs nothing (exit 0,
`eval_stopped:declined`). With no terminal it stops and says how to turn it
on (`eval_stopped:tracing_off`), which a coding agent can do itself.
- The run app's Evaluate on an untraced run asks the same question in a modal
(TurnTracingOnScreen) before running the crew again. Otherwise it still runs
it again with no question.
- crewAI's tracing respects an explicit off again: `should_enable_tracing`
checks it before a run `crewai eval` started, and `execution._tracing_disabled`
is back to what main has.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(tracing): the telemetry opt-out no longer turns tracing off; an evaluated run says where its trace goes
- `OTEL_SDK_DISABLED` is the telemetry opt-out, a separate setting from
tracing, which has its own (`CREWAI_TRACING_ENABLED`, `tracing=`).
`execution._tracing_disabled` no longer reads it. The trace session's own
TracerProvider records even when the OpenTelemetry SDK is switched off
(`session._trace_provider`), because the SDK otherwise hands out tracers that
record nothing. A provider a host passes in is left as it is. Tests that used
`OTEL_SDK_DISABLED` as a way to stop a trace now cover the two real switches;
new tests show a traced run, anonymous or logged in, recorded and uploaded
with it set.
- A run app opened to run the crew for an evaluation (`crewai eval`, or
Evaluate on a run that was not traced) shows one plain note on mount: "This
run will be evaluated: its trace is sent to CrewAI AMP, which grades it."
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cli): say where the trace goes only when the yes is inferred from crewai eval
The note, in the run app ("This run will be evaluated: its trace is sent to
CrewAI AMP, which grades it.") and in `crewai eval`'s line, is for the case
where nobody turned tracing on: not `tracing=True` on the crew or flow, not
`CREWAI_TRACING_ENABLED=true`, not a consent a person gave once
(`crewai_core.user_data.is_tracing_enabled`), and not the yes they just gave to
turn it on. A project or a person who opted in is told nothing new.
The run `crewai eval` starts no longer sets `CREWAI_TRACING_ENABLED=true`:
the handshake alone turns tracing on for it, so the setting a run reads is the
project's own.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cli): turn tracing on once with a note, pin the eval handshake, and keep the saved login to trusted AMPs
From the review on #8270, and João's call on the note:
- The first `crewai eval` that infers the yes turns tracing on for the
project (`CREWAI_TRACING_ENABLED=true` in `.env` and for the run) and says
so once, in the command and in the run app: "Tracing is now on for this
project…". Afterwards the project traces and nothing more is said. A run
whose crew or flow sets `tracing=False` is never told its trace is sent.
- The handshake's folder is found from the account's home in the account
database, never `HOME` or `XDG_DATA_HOME`, so a project's `.env` cannot plant
a token file that approves an upload. Both copies (crewai-core, the run app)
are pinned to the same folder and file by a test.
- The token file carries the AMP origins `crewai eval` trusted before the
project's `.env` was loaded. A run it started sends the saved login only
there over HTTPS or loopback; an AMP the `.env` chose gets an anonymous
grant.
- Evaluate on an untraced run carries that run's definition, inputs (prompted
ones included) and trained-agents file into the re-run. The conversational
flow consumes the request too, and a crew or flow that sets `tracing=False`
says why it cannot be evaluated instead of paying for another run.
- "The project changed" also sees deleted files: each evaluation keeps the
list of the project's files (`.crewai/eval_project_state.json`) and the next
one compares against it. That list is taken after this command's own
project-id write, so the write is no change.
- The OpenTelemetry override probes its provider and warns if an SDK version
still records nothing, instead of failing silently.
- The stale `sdk` case of `test_disabled_execution_does_not_collect_or_request_grants`
is gone; the handshake tests are typed (type-checker CI).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cli): keep the saved login off a project's AMP on every path crewai eval runs
From the second review round on #8270:
- Evaluate's re-run runs in a process that already loaded the project's
`.env`, so it passes this machine's AMPs (`_machine_amp_origins`) as the
trusted set instead of reading `CREWAI_PLUS_URL` there (`eval_crew(
trusted_amp_origins=…)`).
- A project's `.env` can neither set nor replace `crewai eval`'s token:
`run_crew.load_project_env` keeps the token the process was started with and
drops one a `.env` sets. The JSON-crew, declarative-flow and eval paths all
load `.env` through it.
- `crewai eval` turns tracing on only when every credential this machine holds
(the saved login, or a PAT or integration token from before the `.env`) may
go to the AMP the run would trace to. A project's crewai older than the
handshake would otherwise send it to an AMP the `.env` names. A newer crewai
still traces the run anonymously through the handshake.
- A conversational flow named by its definition is re-run by that definition.
- Code quality: no redundant assignment in `_awaiting_dir`, no unused names in
the credential test.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cli): crewai eval does not run a crew whose trace would carry the login to an untrusted AMP
When tracing is on for the run (the project's .env, or a consent given once)
and a credential of this machine would go to an AMP the command does not
trust, the crew is not run. A project's crewai older than the handshake would
send the login there, and the CLI cannot tell which crewai the project runs.
The command stops (`eval_stopped:untrusted_amp`) and says how to go on: log
in to that AMP, or remove CREWAI_PLUS_URL from the project's .env. A project
that turned tracing off is not asked to turn it on toward such an AMP. With
no credential on the machine, there is nothing to protect and the run goes on.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Ask at a terminal before tracing is first turned on
A person running `crewai eval` at a terminal, in a project that never
turned tracing on, is now asked on crewAI's trace consent screen before
the crew runs; a yes turns tracing on for the project, a no (or the
screen timing out) runs nothing and is counted as `declined`. The run
app's Evaluate on an untraced run asks on its own consent screen the same
way. With nobody at a terminal (a coding agent, a script) the yes is
still inferred from the command, tracing is turned on and said once.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Keep main's Iterator import, which the rebase over #8187 dropped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* A run-app test no longer leaves .crewai/last_eval.json in the repo
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Two run-app tests from #7963 no longer leave .crewai/last_eval.json in the repo
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Never run the crew toward an AMP the login must not reach; check before asking
`crewai eval` now refuses to run the crew whenever a credential of this
machine would go with the trace to an AMP it does not trust, whatever the
tracing setting: `tracing=True` in code is invisible to the CLI, and a
project's crewai older than the handshake sends the credential wherever
the run traces. The run app's Evaluate makes the same check before it
asks anything, so a yes can never write CREWAI_TRACING_ENABLED=true to
`.env` for a re-run that is then refused.
Which credentials are this machine's is read from the environment as it
was before any project `.env` was loaded (`run_crew.environment_before_project`),
so on Evaluate's re-run a CREWAI_USER_PAT the project's own `.env` brought
is the project's, as on any run of it.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* Run the crew only toward an AMP this machine trusts, login or not
Whether this machine holds a login right now no longer decides: with no
login yet, `crewai eval` still wrote CREWAI_TRACING_ENABLED=true toward a
CREWAI_PLUS_URL the project's `.env` chose, and the first run after
`crewai login` would have carried the login there. Toward an AMP this
command does not trust, the crew is now never run, nobody is asked and
nothing is written, in `crewai eval` and the run app's Evaluate alike.
That also drops the credential bookkeeping from
|
||
|
|
1410318b2e |
feat(cli): crewai eval export downloads an evaluation as files (#8187)
* crewai eval export downloads an evaluation's full data as files A coding agent that is asked to act on an evaluation needs more than the brief: the report, every setup's outputs, every check with the judge's words, and the traces. AMP serves them as one export; this writes them into ./crewai-eval-<id>/ with a .gitignore of `*`, so customer data is never committed by accident. `crewai eval` becomes a group so the subcommand can live under it; every existing invocation is unchanged. The paths come from whichever AMP answered, so only plain relative paths that resolve inside the folder are written; the rest are skipped and counted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * A .gitignore link already in the export folder is replaced, never followed Writing through it would have put `*` into whatever file it points at, outside the folder. The tests also read the export as UTF-8, as it is written, so a Windows locale cannot misread its dashes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * The export writes through folder handles and refuses Windows device names A name Windows keeps for a device (CON, PRN, AUX, NUL, COM0-9, LPT0-9, with any extension) is skipped like any other unsafe path, on every platform: on Windows `NUL.md` is the null device, and the export would report a file written that nobody can read. On Linux and macOS each folder of a path is opened inside the one before it, never through a link, and the file inside the last: no path is checked and then opened again, so a link another process swaps in while the export runs is never gone through. A link where a folder goes skips that file; a link at a file's own place, symbolic or hard, is removed and the file made anew; the export folder itself being a link stops the export with one sentence. Windows keeps the resolved-path check. Reading an evaluation sends `can_export=1`, so the brief names `crewai eval export` only when the CLI, AMP and the evaluation service all serve it. An AMP without the export route (a 404 that is not AMP's `not_found`) says it needs an update; AMP's `export_unsupported` and `export_too_large` answers are printed in AMP's own words. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * A folder where an exported file goes is skipped; the path writer replaces hard links and refuses a junction - A folder already standing where an exported file goes is skipped and counted, as a file where a folder goes already was, instead of stopping the export. - On the path writer (Windows), whatever is at a file's place is removed and the file created anew. A hard link there is no longer written through. - The export folder must resolve to itself. A Windows junction, which `is_symlink` does not see, stops the export like a link. - The file handle is closed if wrapping it fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Say why a missing file is no error before the export makes it Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * A folder where the export's .gitignore goes stops the export before any data The data never goes into the folder without the .gitignore that keeps it out of git. The Windows path writer's comment now states the race it leaves open, and why that loss is bounded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
9112709294 |
fix(cli): run JSON crews headless when there is no terminal (#8239)
`crewai run` on a JSON crew opened the Textual run screen unless CREWAI_DMN was set. Without an interactive terminal (a coding agent's shell, CI, piped input) the screen stayed open after the crew finished, so the command never exited. It now opens only when is_interactive() is true, the convention declarative flows already follow; otherwise the crew runs on plain output and the command exits. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
40bb2d3b58 |
fix(memory): escape scope and id values in LanceDB filters (#7820)
Vulnerability Scan / Detect changes (push) Has been cancelled
Vulnerability Scan / pip-audit (push) Has been cancelled
Nightly Canary Release / Check for new commits (push) Has been cancelled
Nightly Canary Release / Build nightly packages (push) Has been cancelled
Nightly Canary Release / Publish nightly to PyPI (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
* fix(memory): escape scope and id values in LanceDB filters LanceDBStorage built its where() filters by interpolating scope prefixes and record ids into SQL strings. Route every interpolated value through a quote-escaping helper, and match scope prefixes with LIKE ... ESCAPE '\' so %, _ and \ in scope names match literally while the trailing prefix wildcard is kept. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(memory): keep delete calls out of assert expressions Run each storage.delete() on its own line and assert on the returned count, so the deletion does not live inside an assert statement. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Craig Estep <craig.estep@crewai.com> Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
cf9f08a4a1 |
fix: resolve pip-audit dependency findings (#8221)
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Vulnerability Scan / Detect changes (push) Has been cancelled
Vulnerability Scan / pip-audit (push) Has been cancelled
Build uv cache / build-cache (3.10) (push) Has been cancelled
Build uv cache / build-cache (3.11) (push) Has been cancelled
Build uv cache / build-cache (3.13) (push) Has been cancelled
Build uv cache / build-cache (3.12) (push) Has been cancelled
Raise the MCP Python SDK and TinyTag version floors to releases patched for the two vulnerabilities reported by the pip-audit workflow. - mcp: ~=1.28.1 / >=1.28.1 → >=1.30.0,<2 - tinytag: ~=2.2.1 → >=2.3.0,<3 - Regenerated uv.lock Co-authored-by: Vidit <vidit@crewai.com> |
||
|
|
6115839576 |
fix(mcp): name tools from IP-addressed MCP servers so providers accept them (#8117)
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Vulnerability Scan / pip-audit (push) Has been cancelled
Nightly Canary Release / Check for new commits (push) Has been cancelled
Nightly Canary Release / Build nightly packages (push) Has been cancelled
Nightly Canary Release / Publish nightly to PyPI (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
Vulnerability Scan / Detect changes (push) Has been cancelled
* fix(mcp): name tools from IP-addressed MCP servers so providers accept them Tools from an MCP server addressed by IP (http://127.0.0.1:8800/mcp) were named 127_0_0_1_8800_mcp_<tool>. validate_function_name requires a leading letter, so every provider rejected them. _extract_server_name now prepends mcp_ when the derived name starts with anything other than an ASCII letter; hostname-based names keep their current form. The server name also copied the user:pass@ part of the URL into every tool name, which sends credentials to the model. It is now stripped. * fix(mcp): prefix stdio server names that start with a digit Stdio servers built their tool prefix from the command and args without the leading-letter check, so a command like 3d-server produced 3d_server_<tool>, which providers reject. Both stdio and URL server names now go through _ensure_letter_prefix. The check now looks at the sanitized name. A non-ASCII host such as ñandu.example sanitizes to a leading n and keeps its name, and a path like /usr/bin/python never gets a prefix. |
||
|
|
c661df9816 |
feat(agents): promote AgentExecutor out of experimental (#8192)
* feat(agents): promote AgentExecutor out of experimental The default executor was still imported from crewai.experimental. Move it to a stable module and keep the old import as a deprecation warning. * fix(agents): stabilize executor discriminator * feat(agents): remove CrewAgentExecutor AgentExecutor is the only executor. Selecting the deleted class fails immediately, with no compatibility path. * feat(agents): drop experimental AgentExecutor compatibility The experimental import path and the legacy experimental discriminator are no longer accepted. * docs: address executor review comments Translate the remaining hook comments, document the full executor type, and stop the removal test from hiding an eager export. * docs: keep translated hook sketches on AgentExecutor The locale class sketches only needed the CrewAgentExecutor rename. LiteAgent and None stay on the English page, which documents the full type. |
||
|
|
912f491b67 |
feat: proactively summarize oversized LLM context (#8191)
Build uv cache / build-cache (3.10) (push) Has been cancelled
Build uv cache / build-cache (3.11) (push) Has been cancelled
Build uv cache / build-cache (3.12) (push) Has been cancelled
Build uv cache / build-cache (3.13) (push) Has been cancelled
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Vulnerability Scan / Detect changes (push) Has been cancelled
Vulnerability Scan / pip-audit (push) Has been cancelled
* feat(llm): summarize before context limit * test(llm): cover proactive context compaction * test(agent): cover proactive compaction routes * test(agent): cover async proactive compaction * test(lite-agent): cover proactive compaction * test(agent): cover executor context preflight * feat(agent): show context summarization status * fix(llm): harden proactive context preflight * test(llm): configure proactive preflight mocks * test(agent): remove flaky async preflight coverage * feat(events): emit context summarization lifecycle * fix(agent): preserve sole pending request * fix(agent): fail open proactive compaction * fix(agent): offload proactive compaction * test(agent): align proactive compaction fixture * test(agent): add history to compaction fixtures * fix(events): close failed summarization lifecycle |
||
|
|
c069653b83 |
fix: stop an agent that repeats the same tool call (#8164)
* fix: stop an agent that repeats the same tool call * count tool calls within method only |
||
|
|
e9f682222a |
feat: defer oversized native tool results (#8110)
* feat: add temporary tool result artifacts * feat: detect oversized tool results * feat: allow deferred tool results in file reader * feat: preview deferred tool results * test: cover deferred native tool results * test: cover multiple deferred tool results * test: cover existing file reader with deferred results * feat: clean up deferred tool result artifacts * test: cover deferred tool result fallbacks * feat(agent): describe deferred tool result size * feat(agent): display deferred tool artifacts * style(agent): label deferred results with emoji * feat(agent): recommend deferred result chunks * feat(agent): expose deferred artifact directory * feat(agent): add better compaction flag * feat(agent): add deferred result reader * fix(agent): reference native deferred reader name * fix(agent): render deferred results as plain text * fix(agent): size deferred result reads by context * fix(agent): reuse overflow token count for deferred reads Keep the provider token count from the overflow check and store that same count so read-size recommendations do not recalculate it. * fix(agent): keep deferred reader available after text tool downgrade * fix: avoid re-deferring deferred tool results * revert(tools): remove deferred file reader support |
||
|
|
3da2d4e993 |
Filter platform actions while selecting them (#7991)
* Mark the platform action picker as searchable. The flag is off for every other multi-select, and filtering comes in the next change. * Filter picker labels by query while keeping their original indexes. An empty query matches every label, and a typed query can match the display name or the slug. * Let the platform action picker filter as the user types. The arrow-key list shows the query and hides non-matching rows, and the numbered fallback uses the same filter without dropping checked actions. * Erase leftover rows when a search result is shorter than the previous list. A filtered action picker now shows only the matching actions. * Show the action search in green under the navigation hint. The line includes how many actions the query matched. |
||
|
|
aa5f2acd9b |
deploy push reads AMP's comparison refusal by its code (#8161)
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Vulnerability Scan / Detect changes (push) Has been cancelled
Vulnerability Scan / pip-audit (push) Has been cancelled
Nightly Canary Release / Check for new commits (push) Has been cancelled
Nightly Canary Release / Build nightly packages (push) Has been cancelled
Nightly Canary Release / Publish nightly to PyPI (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
* deploy push reads AMP's comparison refusal by its code AMP's 409 for a deployment with a models comparison running now carries the sentence in `error`, because every released CLI prints a failed request's `error` and none has `--force`; the refusal is named in `code`. `crewai deploy push` recognises it by `code` and prints the sentence as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Recognise AMP's refusal named in `error` too, on every route An AMP from before the `code` field (a branch on staging, a factory that upgrades later) names the refusal in `error`; recognising both keeps the sentence for an upgraded CLI whichever AMP it talks to. A test covers deploy by id, by name and the ZIP update in both shapes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
c7c0d8969a |
Rename the platform token and preselect default app actions (#7990)
* Rename the platform integration token to the Enterprise Action Auth Token. The CLI now asks for that token under Settings → Account and stores it as CREWAI_ENTERPRISE_ACTION_AUTH_TOKEN. * Add the latest Clipper platform applications to the catalog. SharePoint Graph, Resend, Vercel, and Pinecone come from clipper v5 and are grouped in the platform picker. * Preselect three default actions for every platform application. The create wizard checks those actions in the picker, and a test keeps the defaults list the same length as the catalog. * Fail the Clipper client test when the Enterprise Action Auth Token is absent. A deployment instance id is not a substitute for that token, so the test no longer sets one. * fix(agent): name the Enterprise Action Auth Token when apps resolve to no tools |
||
|
|
aaaa312cba |
feat(llm): add provider input token count support (#8109)
* feat(llm): add token estimation interface * feat(openai): add input token count support * test(openai): cover input token counting * feat(anthropic): add input token count support * feat(llm): extend input token count providers * feat(litellm): add input token estimation * fix(llm): address token count provider gaps * fix(llm): count LiteLLM structured output tokens * fix(anthropic): align token count payload * fix(snowflake): reject unsupported token counts * fix(openai): reject compatible token counts * refactor(llm): simplify token count interface * fix(llm): fall back to character token estimates * fix(llm): warn on token count fallback * fix(anthropic): fall back when token counting fails |
||
|
|
b0c16c162a |
fix(cli): crewai eval waits out AMP's rate limit instead of quitting (#8148)
* fix(cli): crewai eval waits out AMP's rate limit instead of quitting
On prod (2026-10-14) a `crewai eval --models` comparison ended its CLI with
exit 1 on one poll AMP answered 429 ("Too many evaluation requests. Please
retry later." — AMP allows 30 evaluation requests a minute per IP), while
the deployment went on running the comparison. A 429 mid-poll is now waited
out: Retry-After seconds, never less than a poll's interval nor more than
RATE_LIMIT_MAX_WAIT_SECONDS (120), a missing or odd header a minute; then
the poll goes on, however many come. It is not counted as a miss: being
asked to slow down is no reason to stop waiting for a running evaluation.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cli): a rate limit that never lifts ends the wait; Retry-After as RFC 9110 has it
Review on #8148:
- Rate-limited without a break for RATE_LIMITED_FOR_AT_MOST_SECONDS (15
minutes, counted over the waits the 429s asked for and reset by any other
answer), the wait stops: "AMP kept rate-limiting this command for 15
minutes, so it stopped waiting; the evaluation keeps running at <url>."
A limit that passes is still ridden out; CI, agents and the TUI no longer
wait without end.
- Retry-After is read in both forms RFC 9110 allows, delay-seconds and an
HTTP-date (a date already past waits a poll's interval); a negative or
non-finite number, or words, waits a minute, as documented — NaN and the
infinities no longer slip past that.
- The test compares the whole list of waits.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(cli): the passing-limit case checks every poll it made
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cli): the rate-limit count never overruns, and any other answer restarts it
CodeRabbit on #8148 (outside the diff):
- A 5xx between 429s is an answer too: every non-429 answer now restarts
the count, before the 5xx branch continues.
- The last wait is what is left of RATE_LIMITED_FOR_AT_MOST_SECONDS, so the
wait never runs past fifteen minutes (it could overrun by up to 119 s).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cli): a 429 breaks a streak of unreachable or 5xx polls
CodeRabbit on #8148: POLL_RETRIES counts a consecutive streak of
unreachable or 5xx polls, and a 429 is AMP answering. Four 503s, a 429 and
one more 503 stopped the wait; the 429 now resets the streak.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cli): one deadline over every kind of refusal while waiting
iris on #8148: a 503 reset the rate-limit count and a 429 reset the 5xx
streak, so refusals taking turns could keep CI and agent callers waiting
without end. The two counts are now one monotonic deadline (_clock):
polls that get no answer about the evaluation — rate-limited, failing,
unreachable, in any mix — for UNANSWERED_FOR_AT_MOST_SECONDS (15 minutes)
without a break end the wait, and only AMP's answer (a 200) restarts it.
The last wait is still what is left of the deadline. The sentence names the
cause: "AMP kept rate-limiting this command for 15 minutes" when every
refusal was a 429, else "AMP gave this command no answer about the
evaluation for 15 minutes (rate limits, errors, no connection)".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
8dcb00584f |
feat(cli): crewai deploy push --force, and why AMP held a push back (#8150)
* feat(cli): crewai deploy push --force, and why AMP held a push back
AMP now holds back a redeploy while a models comparison is running on the
deployment (crewAIInc/crewai-plus#4776): a redeploy restarts the workers
and the comparison loses its runs (pr_security_reviewer, 2026-10-14). It
answers 409 {"error": "comparison_running", "message": ...}.
- `crewai deploy push --force` goes ahead anyway: deploy_by_uuid /
deploy_by_name send {"force": true} and update_crew_from_zip a force field
(crewai-core and the CLI's own copy), only when asked — an unforced push
sends exactly what it always did, and an older AMP ignores the field.
- A push AMP held back prints AMP's sentence (which names --force) and
"Nothing was deployed.", and exits 1 — not the bare error code.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(llms): Claude 3.5 Sonnet v2 on Bedrock has a 200k window in the refreshed catalog
The 1.15.28 release refreshed the packaged LiteLLM catalog, which now lists
anthropic.claude-3-5-sonnet-20241022-v2:0 at 200,000 input tokens (it had
1,000,000), the model's documented window. The test still expected 1M, so
every PR's test job failed on main's own catalog.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cli): AMP's hold-back sentence is printed literally
Review on #8150: the comparison_running message went through Rich markup,
so a bracketed word in it vanished and a stray closing tag raised
MarkupError before "Nothing was deployed." could print. It is printed with
markup=False now; a test sends one with both.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
|
acd4b33a17 |
test(llms): Claude 3.5 Sonnet v2 on Bedrock has a 200k window in the refreshed catalog (#8151)
The 1.15.28 release refreshed the packaged LiteLLM catalog, which now lists anthropic.claude-3-5-sonnet-20241022-v2:0 at 200,000 input tokens (it had 1,000,000), the model's documented window. The test still expected 1M, so every PR's test job failed on main's own catalog. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
eb15c66cef |
[docs-freeze] docs: snapshot and changelog for v1.15.28 (#8147)
Mark stale issues and pull requests / stale (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
Vulnerability Scan / Detect changes (push) Has been cancelled
Vulnerability Scan / pip-audit (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
|
||
|
|
7056fe1fe2 | feat: bump versions to 1.15.28 (#8146) | ||
|
|
75808b0eca |
fix(flow): learn from resumed human feedback and keep the reviewed output's type (#8142)
Nightly Canary Release / Check for new commits (push) Has been cancelled
Nightly Canary Release / Build nightly packages (push) Has been cancelled
Nightly Canary Release / Publish nightly to PyPI (push) Has been cancelled
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
Vulnerability Scan / Detect changes (push) Has been cancelled
Vulnerability Scan / pip-audit (push) Has been cancelled
* fix(flow): learn from human feedback that resumes a paused flow With learn=True, lessons were distilled only when the reply arrived during the same run. A reply that resumes a paused flow (resume / resume_async after from_pending, which is how async providers such as AMP's email review work) went through outcome collapse and routing but never reached _distill_and_store_lessons, so it taught nothing. Resume now stores lessons under the same conditions as an in-run reply: memory available, non-empty feedback, the method's learn_source and learn_strict, and the persisted LLM config. Both paths share one helper. A strict distillation failure fails the resume before the pending review is cleared, so it can be retried. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(flow): undo the recorded reply when strict learning fails on resume _finalize_human_feedback appends the reply to human_feedback_history and sets last_human_feedback before lessons are stored. With learn_strict, a distillation failure re-raises and the review stays pending, so a retry on the same instance recorded the reply twice. Restore both on failure. Lesson storage stays after outcome collapse, so a collapse failure never leaves lessons behind to be stored again on retry. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(flow): keep the reviewed output's type when learn=True pre-reviews it Pre-review always returned PreReviewResult.improved_output, a str, so a method returning a Pydantic model or a dict reached the reviewer, and last_human_feedback.output, as a repr string. Text output is still rewritten as text; a Pydantic model is now rewritten as the same model (response_model=type(output)); other outputs are shown unchanged. LiteLLM-routed models return structured output as a JSON string, which PreReviewResult/DistilledLessons.model_validate rejected, so pre-review and lesson storage silently did nothing for them. One parser now accepts an instance, a JSON string, or a dict. Docs (en, ar, ko, pt-BR): drop the nonexistent learn_limit, document learn_source and learn_strict, say lessons are also learned from replies that resume a paused flow, how each output type is pre-reviewed, and that downstream steps should read last_human_feedback.output. Correct the HumanFeedbackResult.output docstring and the learn/learn_strict field descriptions. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(flow): keep pre-review failures and unseen fields from leaking into the output model_dump_json() ran before the pre-review try, so a Pydantic output that cannot be serialized failed the feedback step even with learn_strict off. It now runs inside the try and falls back to the original output. A reviewed Pydantic model is rebuilt from what the LLM saw, so a SecretStr (sent masked) or an excluded field (not sent) came back as the mask or the default. Fields the review left unchanged now keep their original values. Also pin that a retry after a persistence failure that follows lesson storage offers the lessons to memory again. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(flow): keep nested hidden values when pre-review edits sibling fields The unchanged-field restore only looked at top-level fields, so editing a sibling inside a nested model, or one item of a list of models, kept the LLM's masked secret and default excluded values for that whole object. The restore now recurses into nested models and same-length lists of models. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(flow): skip pre-review for models holding secrets or excluded fields Restoring hidden values after the review kept growing edge cases: aliased fields compared by name discarded the review's edit, dicts of models were not recursed, and reordered lists paired the wrong items, which could copy one account's secret onto another. Drop the merge. A Pydantic output is pre-reviewed only when the LLM sees all of it; one holding a Secret, SecretStr, SecretBytes or Field(exclude=True) value at any depth reaches the reviewer unchanged. Docs updated in en, ar, ko and pt-BR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs(flow): say a resumed review's Pydantic output arrives as a dict PendingFeedbackContext persists method_output as JSON, so after from_pending and resume, last_human_feedback.output is a dict even when the method returned a model. The docstring and docs said "same type" without that qualification. Also say learn_strict covers distillation and submitting lessons, not the background memory write. Pin the dict behavior with a pause/resume test. Docs in en, ar, ko and pt-BR. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
ba36746169 |
fix(agent): warn when non-empty apps resolve to zero platform tools (#6469)
* fix(agent): warn when non-empty apps resolve to zero platform tools When an agent declares `apps` (enterprise integrations) but they resolve to no tools — unknown/misspelled app, app not connected, no actions, or a missing CREWAI_PLATFORM_INTEGRATION_TOKEN — the agent silently runs without them and can't do what those apps were for. CrewaiPlatformTools returns an empty list in all those cases, and both resolution sites dropped it silently (agent kickoff `if platform_tools:` and crew `_inject_platform_tools` merge), while the only existing signal is a verbose-gated Logger.log. Emit a loud UserWarning from Agent.get_platform_tools() — the shared choke point for the agent-kickoff path (core.py:_prepare_kickoff), the crew task path (crew.py:_inject_platform_tools), and declarative flows (agent actions route through the same kickoff). The warning names the unresolved apps and the likely causes. Tests: warns on zero-tool resolution (direct + crew-prepare paths), stays quiet when tools resolve. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RBYGqJHC2TMC6fonFziuuh * test(agent): cover empty platform-tool resolution through the real factory Address review on the empty-apps warning: document get_platform_tools, trim the inline comment, and use one import style for crewai_tools in the tests. Two new tests run the real CrewaiPlatformTools factory (it returns a plain list) with only requests.get mocked: a selector that matches no action, and a missing integration token. Both reach the warning, and messages are now asserted in full. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
|
ddfa8517b8 |
feat(cli): crewai eval --models says at once where a stopped model broke (#8105)
* feat(cli): crewai eval --models says at once where a stopped model broke The service's poll answer now lists the setups that stopped, with where in the project each broke (step → crew → task → agent), the whole reason and its plain words. The CLI says each one the moment that is known, while the others keep running: a coding agent gets a block it can start fixing from, the run's own words quoted as data; a person gets one red line. An older service sends nothing and nothing changes. The AGENTS.md templates say to run a comparison in the background and read its output as it goes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(cli): a stopped setup's location is a list of names, and the run's words come only from `reported` Review (#8105): `where` from the service was used whenever it was truthy, so a string skipped the wait and printed one character per place, and anything not iterable raised. Each stopped setup is now parsed once into a checked record (_Stop): only a list of non-empty strings is a location; the run's own words are read only from `reported`, where the service (crew-optimize #25) puts them beside its data-not-instructions note — top-level text is not the run's channel. The AGENTS.md templates say a location prints when the service knows it, and an unplaced stop after about 20 seconds or when the comparison ends. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(cli): pin when an unplaced stop is said, not just the order Review (#8105): the ordering assertion passed even if an invalid `where` were announced on the first poll, since the progress line prints first anyway. The test now records the fake clock when the stop is said and asserts 30 s — after the wait ran out, never at the first poll. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
7b07ee5b6a |
fix(utilities): stop json-schema models warning about BaseModel-shadowing fields (#8085)
* fix(utilities): stop json-schema models warning about BaseModel-shadowing fields Tool schemas legitimately name arguments schema, json or copy. Pydantic warns that these shadow BaseModel attributes every time create_model_from_schema builds such a model, which clutters CLI and agent output when platform or MCP tools load. The fields already validate and dump under their own names, so silence only that warning, and only while building a model that has a colliding name. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(utilities): serialize the shadow-warning filter swap across threads Two threads building colliding models could overlap their catch_warnings blocks; the later exit then restored a filter list that still held the earlier build's ignore rule, leaking it to every later model. A lock now serializes the swap. Also replace a class defined only for its warning with create_model in the test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(utilities): pin that rebuilt recursive models emit no shadow warning A recursive $defs schema builds one model incomplete and finishes it with model_rebuild outside the suppression context. Pydantic only warns while collecting fields at class creation, so the rebuild stays silent; this test keeps that true. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
c0eda9f538 |
docs: consolidate MongoDB documentation improvements (#8097)
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Vulnerability Scan / Detect changes (push) Has been cancelled
Vulnerability Scan / pip-audit (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
* docs: consolidate MongoDB documentation improvements Combines previously closed MongoDB docs PRs (#6847, #6845, #6721, #6626, #6760, #6759) into a single PR per the updated contribution guidelines requiring a linked open issue. RagTool only supports chromadb and qdrant as vector store backends; MongoDB Atlas is covered separately by MongoDBVectorSearchTool, which is already listed in the database-data tools overview, so the RagTool data-sources bullet was dropped rather than reintroduced. Fixes #8096 * revert gemini change * ai review feedback * memory wording * translations |
||
|
|
18473adc33 |
fix(cli): flow add-crew finds the package from pyproject (#8082)
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
Vulnerability Scan / Detect changes (push) Has been cancelled
Vulnerability Scan / pip-audit (push) Has been cancelled
Nightly Canary Release / Check for new commits (push) Has been cancelled
Nightly Canary Release / Build nightly packages (push) Has been cancelled
Nightly Canary Release / Publish nightly to PyPI (push) Has been cancelled
`crewai flow add-crew` looked for `src/<folder name>/crews`, so a project cloned or created under a different directory name failed with "Crews folder does not exist". It now resolves the package from `[project].name` (normalized like the scaffolder does) and falls back to a single `src/*/crews`; several candidates or none fail with a message naming what was expected or found. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
bea8c03649 |
fix: preserve native tool aborts and post-hook cache identity (#8107)
* fix: preserve native tool aborts and post-hook cache identity * fix: limit native hook changes to active executors |
||
|
|
ab423144c1 |
docs: correct deploy and platform-tool guidance in the project AGENTS.md (#8084)
* docs: correct deploy and platform-tool guidance in the project AGENTS.md The AGENTS.md that `crewai create` ships told coding agents that `crewai deploy remove <id>` works (the CLI only takes `-u <uuid>`), that a deploy needs a GitHub repository (with no origin it uploads a ZIP), that PLATFORM_APPS is the source of truth for action selectors (some of its action names resolve to no tool), and to measure quality with `crewai test`. Its router example also declared no labels, so `crewai flow plot` could not draw the branches. - Template: platform token, selector syntax, `apps:` YAML example and selector check; router labels; deploy prerequisites, `-y`, `.env` upload, org commands, `project_id`; best practice points at `crewai eval` inside the eval markers. - Docs: quickstart deploy prerequisite and `remove -u`, and router labels in concepts/flows, in en, ar, ko and pt-BR. - Tests parse every documented deploy/org command against the real CLI, run the routing example through the plot builder, and check the best practices stay numbered without gaps for crew and tool projects. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: base the quickstart ZIP-deploy note on any git origin The CLI deploys from git whenever an `origin` exists, GitHub or not, and uploads a ZIP only when there is none. The quickstart said "GitHub origin ... without one", which reads as if a non-GitHub origin takes the ZIP path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
744bf60367 |
fix(cli): show why a deployment failed and fix deploy remove and lockfile checks (#8083)
* fix(cli): show why a deployment failed and fix deploy remove and lockfile checks - deploy status prints AMP's error_message and error_suggested_fix when a deployment has failed, verbatim (no Rich markup), instead of only "Provisioning Failed, try again." - deploy remove -u names the removed UUID instead of the local project, and a failed removal prints the API's error. - deploy validate asks `uv lock --check` (offline) whether uv.lock is current instead of comparing file times, which flagged metadata-only edits that `uv lock` never clears. File times remain the fallback without uv or for poetry.lock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: note that deploy status prints the failure reason Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(cli): print deploy validate findings without Rich markup render_report printed codes, titles, details and hints as Rich markup, so bracketed text vanished: `[stale_lockfile]`, `[project]`, `[tool.hatch.build.targets.wheel]`, `crewai[tools]`. Raw `uv` stderr in the new unverified_lockfile detail could also raise MarkupError on a stray closing tag. Escape every finding field. Also bound the test fixture's `uv lock` with a timeout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
6b9e62180e |
fix(core): give trigger list and payload requests a 60s timeout (#8081)
`crewai triggers list` timed out on every real organization: the apps listing takes about 11s and these two calls fell back to httpx's 5s default. Other PlusAPI calls keep their current timeouts. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
|
13b7c3079f |
Refresh the LiteLLM catalog after the version-bump PR merges. (#8103)
The release docs PR now downloads the checked-in model catalog so context-window data ships with the tagged release. |