mirror of
https://github.com/crewAIInc/crewAI.git
synced 2026-10-21 17:36:42 +02:00
b26ccaf440e67eea04765ad10fe0c346fcbbaff2
171
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Joao MouraandClaude Opus 5.5
|
b26ccaf440 |
Never run the crew toward an AMP the login must not reach; check before asking
`crewai eval` now refuses to run the crew whenever a credential of this machine would go with the trace to an AMP it does not trust, whatever the tracing setting: `tracing=True` in code is invisible to the CLI, and a project's crewai older than the handshake sends the credential wherever the run traces. The run app's Evaluate makes the same check before it asks anything, so a yes can never write CREWAI_TRACING_ENABLED=true to `.env` for a re-run that is then refused. Which credentials are this machine's is read from the environment as it was before any project `.env` was loaded (`run_crew.environment_before_project`), so on Evaluate's re-run a CREWAI_USER_PAT the project's own `.env` brought is the project's, as on any run of it. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Joao MouraandClaude Opus 5.5
|
4745d1d390 |
Two run-app tests from #7963 no longer leave .crewai/last_eval.json in the repo
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Joao MouraandClaude Opus 5.5
|
4a3a590a97 |
A run-app test no longer leaves .crewai/last_eval.json in the repo
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Joao MouraandClaude Opus 5.5
|
56ff953124 |
Keep main's Iterator import, which the rebase over #8187 dropped
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Joao MouraandClaude Opus 5.5
|
b943e53383 |
Ask at a terminal before tracing is first turned on
A person running `crewai eval` at a terminal, in a project that never turned tracing on, is now asked on crewAI's trace consent screen before the crew runs; a yes turns tracing on for the project, a no (or the screen timing out) runs nothing and is counted as `declined`. The run app's Evaluate on an untraced run asks on its own consent screen the same way. With nobody at a terminal (a coding agent, a script) the yes is still inferred from the command, tracing is turned on and said once. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Joao MouraandClaude Opus 5.5
|
f1477fc6cb |
fix(cli): crewai eval does not run a crew whose trace would carry the login to an untrusted AMP
When tracing is on for the run (the project's .env, or a consent given once) and a credential of this machine would go to an AMP the command does not trust, the crew is not run. A project's crewai older than the handshake would send the login there, and the CLI cannot tell which crewai the project runs. The command stops (`eval_stopped:untrusted_amp`) and says how to go on: log in to that AMP, or remove CREWAI_PLUS_URL from the project's .env. A project that turned tracing off is not asked to turn it on toward such an AMP. With no credential on the machine, there is nothing to protect and the run goes on. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Joao MouraandClaude Opus 5.5
|
f8da6c5c9e |
fix(cli): keep the saved login off a project's AMP on every path crewai eval runs
From the second review round on #8270: - Evaluate's re-run runs in a process that already loaded the project's `.env`, so it passes this machine's AMPs (`_machine_amp_origins`) as the trusted set instead of reading `CREWAI_PLUS_URL` there (`eval_crew( trusted_amp_origins=…)`). - A project's `.env` can neither set nor replace `crewai eval`'s token: `run_crew.load_project_env` keeps the token the process was started with and drops one a `.env` sets. The JSON-crew, declarative-flow and eval paths all load `.env` through it. - `crewai eval` turns tracing on only when every credential this machine holds (the saved login, or a PAT or integration token from before the `.env`) may go to the AMP the run would trace to. A project's crewai older than the handshake would otherwise send it to an AMP the `.env` names. A newer crewai still traces the run anonymously through the handshake. - A conversational flow named by its definition is re-run by that definition. - Code quality: no redundant assignment in `_awaiting_dir`, no unused names in the credential test. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Joao MouraandClaude Opus 5.5
|
be02ce4593 |
fix(cli): turn tracing on once with a note, pin the eval handshake, and keep the saved login to trusted AMPs
From the review on #8270, and João's call on the note: - The first `crewai eval` that infers the yes turns tracing on for the project (`CREWAI_TRACING_ENABLED=true` in `.env` and for the run) and says so once, in the command and in the run app: "Tracing is now on for this project…". Afterwards the project traces and nothing more is said. A run whose crew or flow sets `tracing=False` is never told its trace is sent. - The handshake's folder is found from the account's home in the account database, never `HOME` or `XDG_DATA_HOME`, so a project's `.env` cannot plant a token file that approves an upload. Both copies (crewai-core, the run app) are pinned to the same folder and file by a test. - The token file carries the AMP origins `crewai eval` trusted before the project's `.env` was loaded. A run it started sends the saved login only there over HTTPS or loopback; an AMP the `.env` chose gets an anonymous grant. - Evaluate on an untraced run carries that run's definition, inputs (prompted ones included) and trained-agents file into the re-run. The conversational flow consumes the request too, and a crew or flow that sets `tracing=False` says why it cannot be evaluated instead of paying for another run. - "The project changed" also sees deleted files: each evaluation keeps the list of the project's files (`.crewai/eval_project_state.json`) and the next one compares against it. That list is taken after this command's own project-id write, so the write is no change. - The OpenTelemetry override probes its provider and warns if an SDK version still records nothing, instead of failing silently. - The stale `sdk` case of `test_disabled_execution_does_not_collect_or_request_grants` is gone; the handshake tests are typed (type-checker CI). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Joao MouraandClaude Opus 5.5
|
9226eee072 |
fix(cli): say where the trace goes only when the yes is inferred from crewai eval
The note, in the run app ("This run will be evaluated: its trace is sent to
CrewAI AMP, which grades it.") and in `crewai eval`'s line, is for the case
where nobody turned tracing on: not `tracing=True` on the crew or flow, not
`CREWAI_TRACING_ENABLED=true`, not a consent a person gave once
(`crewai_core.user_data.is_tracing_enabled`), and not the yes they just gave to
turn it on. A project or a person who opted in is told nothing new.
The run `crewai eval` starts no longer sets `CREWAI_TRACING_ENABLED=true`:
the handshake alone turns tracing on for it, so the setting a run reads is the
project's own.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
Joao MouraandClaude Opus 5.5
|
627c2c5c11 |
fix(tracing): the telemetry opt-out no longer turns tracing off; an evaluated run says where its trace goes
- `OTEL_SDK_DISABLED` is the telemetry opt-out, a separate setting from tracing, which has its own (`CREWAI_TRACING_ENABLED`, `tracing=`). `execution._tracing_disabled` no longer reads it. The trace session's own TracerProvider records even when the OpenTelemetry SDK is switched off (`session._trace_provider`), because the SDK otherwise hands out tracers that record nothing. A provider a host passes in is left as it is. Tests that used `OTEL_SDK_DISABLED` as a way to stop a trace now cover the two real switches; new tests show a traced run, anonymous or logged in, recorded and uploaded with it set. - A run app opened to run the crew for an evaluation (`crewai eval`, or Evaluate on a run that was not traced) shows one plain note on mount: "This run will be evaluated: its trace is sent to CrewAI AMP, which grades it." Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Joao MouraandClaude Opus 5.5
|
e7d5bf3c8e |
fix(cli): a project that turned tracing off is asked before crewai eval runs it
`CREWAI_TRACING_ENABLED=false` is the project's no, and nothing overrules it, not even a run `crewai eval` started. - `crewai eval` at a terminal asks to turn tracing on before running the crew. A yes writes it to the project's `.env`; a no runs nothing (exit 0, `eval_stopped:declined`). With no terminal it stops and says how to turn it on (`eval_stopped:tracing_off`), which a coding agent can do itself. - The run app's Evaluate on an untraced run asks the same question in a modal (TurnTracingOnScreen) before running the crew again. Otherwise it still runs it again with no question. - crewAI's tracing respects an explicit off again: `should_enable_tracing` checks it before a run `crewai eval` started, and `execution._tracing_disabled` is back to what main has. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Joao MouraandClaude Opus 5.5
|
9aed68d495 |
feat(cli): crewai eval runs the crew itself, so a coding agent gets a verdict from one command
Most `crewai eval` attempts stopped before an evaluation began because the
project had no traced run. Telemetry since Sept 25: 117 projects stopped as
"untraced", 97 of them with no terminal (coding agents), and 67 of those never
got further. With no terminal the command could only print steps, and without
a login those steps could not work: an unattended anonymous run never uploads
its trace.
- With no traced run, or when the project's files changed since the last one,
`crewai eval` runs the crew and evaluates that run. No question, with or
without a terminal: running the command is the request to have the run
graded, and the grade is read off its trace. Tracing is on for that run only;
nothing is written to the project's `.env`.
- crewAI's tracing approves the upload of that run through the existing
handshake: a token in CREWAI_EVAL_AWAITING_RUN that counts only while its file
exists in the user's crewAI data folder, so a `.env` cannot claim it. Shared
as `crewai_core.eval_handshake`; the run app keeps its own copy because it
also runs against a project's older crewai-core, and a test pins the two.
- The run app's Evaluate on an untraced run closes the app and runs the crew
again, traced, through `crewai eval`'s own path, instead of a notice.
- Counts: `eval_ran_crew:{no_traced_run,project_changed,run_again}`,
`eval_finished:{passed,failed,inconclusive,unknown}`, and
`eval_stopped:untraced_after_run` replacing the catch-all `untraced`
(`declined` is gone with the question). The CLI tests no longer send usage
counts.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
João MouraandClaude Opus 5.5
|
1410318b2e |
feat(cli): crewai eval export downloads an evaluation as files (#8187)
* crewai eval export downloads an evaluation's full data as files A coding agent that is asked to act on an evaluation needs more than the brief: the report, every setup's outputs, every check with the judge's words, and the traces. AMP serves them as one export; this writes them into ./crewai-eval-<id>/ with a .gitignore of `*`, so customer data is never committed by accident. `crewai eval` becomes a group so the subcommand can live under it; every existing invocation is unchanged. The paths come from whichever AMP answered, so only plain relative paths that resolve inside the folder are written; the rest are skipped and counted. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * A .gitignore link already in the export folder is replaced, never followed Writing through it would have put `*` into whatever file it points at, outside the folder. The tests also read the export as UTF-8, as it is written, so a Windows locale cannot misread its dashes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * The export writes through folder handles and refuses Windows device names A name Windows keeps for a device (CON, PRN, AUX, NUL, COM0-9, LPT0-9, with any extension) is skipped like any other unsafe path, on every platform: on Windows `NUL.md` is the null device, and the export would report a file written that nobody can read. On Linux and macOS each folder of a path is opened inside the one before it, never through a link, and the file inside the last: no path is checked and then opened again, so a link another process swaps in while the export runs is never gone through. A link where a folder goes skips that file; a link at a file's own place, symbolic or hard, is removed and the file made anew; the export folder itself being a link stops the export with one sentence. Windows keeps the resolved-path check. Reading an evaluation sends `can_export=1`, so the brief names `crewai eval export` only when the CLI, AMP and the evaluation service all serve it. An AMP without the export route (a 404 that is not AMP's `not_found`) says it needs an update; AMP's `export_unsupported` and `export_too_large` answers are printed in AMP's own words. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * A folder where an exported file goes is skipped; the path writer replaces hard links and refuses a junction - A folder already standing where an exported file goes is skipped and counted, as a file where a folder goes already was, instead of stopping the export. - On the path writer (Windows), whatever is at a file's place is removed and the file created anew. A hard link there is no longer written through. - The export folder must resolve to itself. A Windows junction, which `is_symlink` does not see, stops the export like a link. - The file handle is closed if wrapping it fails. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Say why a missing file is no error before the export makes it Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * A folder where the export's .gitignore goes stops the export before any data The data never goes into the folder without the .gitignore that keeps it out of git. The Windows path writer's comment now states the race it leaves open, and why that loss is bounded. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
João MouraandClaude Opus 5.5
|
9112709294 |
fix(cli): run JSON crews headless when there is no terminal (#8239)
`crewai run` on a JSON crew opened the Textual run screen unless CREWAI_DMN was set. Without an interactive terminal (a coding agent's shell, CI, piped input) the screen stayed open after the crew finished, so the command never exited. It now opens only when is_interactive() is true, the convention declarative flows already follow; otherwise the crew runs on plain output and the command exits. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Vidit Ostwal
|
3da2d4e993 |
Filter platform actions while selecting them (#7991)
* Mark the platform action picker as searchable. The flag is off for every other multi-select, and filtering comes in the next change. * Filter picker labels by query while keeping their original indexes. An empty query matches every label, and a typed query can match the display name or the slug. * Let the platform action picker filter as the user types. The arrow-key list shows the query and hides non-matching rows, and the numbered fallback uses the same filter without dropping checked actions. * Erase leftover rows when a search result is shorter than the previous list. A filtered action picker now shows only the matching actions. * Show the action search in green under the navigation hint. The line includes how many actions the query matched. |
||
|
João MouraandClaude Opus 5.5
|
aa5f2acd9b |
deploy push reads AMP's comparison refusal by its code (#8161)
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Vulnerability Scan / Detect changes (push) Has been cancelled
Vulnerability Scan / pip-audit (push) Has been cancelled
Nightly Canary Release / Check for new commits (push) Has been cancelled
Nightly Canary Release / Build nightly packages (push) Has been cancelled
Nightly Canary Release / Publish nightly to PyPI (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
* deploy push reads AMP's comparison refusal by its code AMP's 409 for a deployment with a models comparison running now carries the sentence in `error`, because every released CLI prints a failed request's `error` and none has `--force`; the refusal is named in `code`. `crewai deploy push` recognises it by `code` and prints the sentence as before. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * Recognise AMP's refusal named in `error` too, on every route An AMP from before the `code` field (a branch on staging, a factory that upgrades later) names the refusal in `error`; recognising both keeps the sentence for an upgraded CLI whichever AMP it talks to. A test covers deploy by id, by name and the ZIP update in both shapes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
Vidit Ostwal
|
c7c0d8969a |
Rename the platform token and preselect default app actions (#7990)
* Rename the platform integration token to the Enterprise Action Auth Token. The CLI now asks for that token under Settings → Account and stores it as CREWAI_ENTERPRISE_ACTION_AUTH_TOKEN. * Add the latest Clipper platform applications to the catalog. SharePoint Graph, Resend, Vercel, and Pinecone come from clipper v5 and are grouped in the platform picker. * Preselect three default actions for every platform application. The create wizard checks those actions in the picker, and a test keeps the defaults list the same length as the catalog. * Fail the Clipper client test when the Enterprise Action Auth Token is absent. A deployment instance id is not a substitute for that token, so the test no longer sets one. * fix(agent): name the Enterprise Action Auth Token when apps resolve to no tools |
||
|
João MouraandClaude Opus 5.5
|
b0c16c162a |
fix(cli): crewai eval waits out AMP's rate limit instead of quitting (#8148)
* fix(cli): crewai eval waits out AMP's rate limit instead of quitting
On prod (2026-10-14) a `crewai eval --models` comparison ended its CLI with
exit 1 on one poll AMP answered 429 ("Too many evaluation requests. Please
retry later." — AMP allows 30 evaluation requests a minute per IP), while
the deployment went on running the comparison. A 429 mid-poll is now waited
out: Retry-After seconds, never less than a poll's interval nor more than
RATE_LIMIT_MAX_WAIT_SECONDS (120), a missing or odd header a minute; then
the poll goes on, however many come. It is not counted as a miss: being
asked to slow down is no reason to stop waiting for a running evaluation.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cli): a rate limit that never lifts ends the wait; Retry-After as RFC 9110 has it
Review on #8148:
- Rate-limited without a break for RATE_LIMITED_FOR_AT_MOST_SECONDS (15
minutes, counted over the waits the 429s asked for and reset by any other
answer), the wait stops: "AMP kept rate-limiting this command for 15
minutes, so it stopped waiting; the evaluation keeps running at <url>."
A limit that passes is still ridden out; CI, agents and the TUI no longer
wait without end.
- Retry-After is read in both forms RFC 9110 allows, delay-seconds and an
HTTP-date (a date already past waits a poll's interval); a negative or
non-finite number, or words, waits a minute, as documented — NaN and the
infinities no longer slip past that.
- The test compares the whole list of waits.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(cli): the passing-limit case checks every poll it made
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cli): the rate-limit count never overruns, and any other answer restarts it
CodeRabbit on #8148 (outside the diff):
- A 5xx between 429s is an answer too: every non-429 answer now restarts
the count, before the 5xx branch continues.
- The last wait is what is left of RATE_LIMITED_FOR_AT_MOST_SECONDS, so the
wait never runs past fifteen minutes (it could overrun by up to 119 s).
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cli): a 429 breaks a streak of unreachable or 5xx polls
CodeRabbit on #8148: POLL_RETRIES counts a consecutive streak of
unreachable or 5xx polls, and a 429 is AMP answering. Four 503s, a 429 and
one more 503 stopped the wait; the 429 now resets the streak.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cli): one deadline over every kind of refusal while waiting
iris on #8148: a 503 reset the rate-limit count and a 429 reset the 5xx
streak, so refusals taking turns could keep CI and agent callers waiting
without end. The two counts are now one monotonic deadline (_clock):
polls that get no answer about the evaluation — rate-limited, failing,
unreachable, in any mix — for UNANSWERED_FOR_AT_MOST_SECONDS (15 minutes)
without a break end the wait, and only AMP's answer (a 200) restarts it.
The last wait is still what is left of the deadline. The sentence names the
cause: "AMP kept rate-limiting this command for 15 minutes" when every
refusal was a 429, else "AMP gave this command no answer about the
evaluation for 15 minutes (rate limits, errors, no connection)".
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
João MouraandClaude Opus 5.5
|
8dcb00584f |
feat(cli): crewai deploy push --force, and why AMP held a push back (#8150)
* feat(cli): crewai deploy push --force, and why AMP held a push back
AMP now holds back a redeploy while a models comparison is running on the
deployment (crewAIInc/crewai-plus#4776): a redeploy restarts the workers
and the comparison loses its runs (pr_security_reviewer, 2026-10-14). It
answers 409 {"error": "comparison_running", "message": ...}.
- `crewai deploy push --force` goes ahead anyway: deploy_by_uuid /
deploy_by_name send {"force": true} and update_crew_from_zip a force field
(crewai-core and the CLI's own copy), only when asked — an unforced push
sends exactly what it always did, and an older AMP ignores the field.
- A push AMP held back prints AMP's sentence (which names --force) and
"Nothing was deployed.", and exits 1 — not the bare error code.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* test(llms): Claude 3.5 Sonnet v2 on Bedrock has a 200k window in the refreshed catalog
The 1.15.28 release refreshed the packaged LiteLLM catalog, which now lists
anthropic.claude-3-5-sonnet-20241022-v2:0 at 200,000 input tokens (it had
1,000,000), the model's documented window. The test still expected 1M, so
every PR's test job failed on main's own catalog.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
* fix(cli): AMP's hold-back sentence is printed literally
Review on #8150: the comparison_running message went through Rich markup,
so a bracketed word in it vanished and a stray closing tag raised
MarkupError before "Nothing was deployed." could print. It is printed with
markup=False now; a test sends one with both.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
---------
Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com>
|
||
|
João Moura
|
7056fe1fe2 | feat: bump versions to 1.15.28 (#8146) | ||
|
João MouraandClaude Opus 5.5
|
ddfa8517b8 |
feat(cli): crewai eval --models says at once where a stopped model broke (#8105)
* feat(cli): crewai eval --models says at once where a stopped model broke The service's poll answer now lists the setups that stopped, with where in the project each broke (step → crew → task → agent), the whole reason and its plain words. The CLI says each one the moment that is known, while the others keep running: a coding agent gets a block it can start fixing from, the run's own words quoted as data; a person gets one red line. An older service sends nothing and nothing changes. The AGENTS.md templates say to run a comparison in the background and read its output as it goes. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(cli): a stopped setup's location is a list of names, and the run's words come only from `reported` Review (#8105): `where` from the service was used whenever it was truthy, so a string skipped the wait and printed one character per place, and anything not iterable raised. Each stopped setup is now parsed once into a checked record (_Stop): only a list of non-empty strings is a location; the run's own words are read only from `reported`, where the service (crew-optimize #25) puts them beside its data-not-instructions note — top-level text is not the run's channel. The AGENTS.md templates say a location prints when the service knows it, and an unplaced stop after about 20 seconds or when the comparison ends. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * test(cli): pin when an unplaced stop is said, not just the order Review (#8105): the ordering assertion passed even if an invalid `where` were announced on the first poll, since the progress line prints first anyway. The test now records the fake clock when the stop is said and asserts 30 s — after the wait ran out, never at the first poll. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
João MouraandClaude Opus 5.5
|
18473adc33 |
fix(cli): flow add-crew finds the package from pyproject (#8082)
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
Vulnerability Scan / Detect changes (push) Has been cancelled
Vulnerability Scan / pip-audit (push) Has been cancelled
Nightly Canary Release / Check for new commits (push) Has been cancelled
Nightly Canary Release / Build nightly packages (push) Has been cancelled
Nightly Canary Release / Publish nightly to PyPI (push) Has been cancelled
`crewai flow add-crew` looked for `src/<folder name>/crews`, so a project cloned or created under a different directory name failed with "Crews folder does not exist". It now resolves the package from `[project].name` (normalized like the scaffolder does) and falls back to a single `src/*/crews`; several candidates or none fail with a message naming what was expected or found. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
João MouraandClaude Opus 5.5
|
ab423144c1 |
docs: correct deploy and platform-tool guidance in the project AGENTS.md (#8084)
* docs: correct deploy and platform-tool guidance in the project AGENTS.md The AGENTS.md that `crewai create` ships told coding agents that `crewai deploy remove <id>` works (the CLI only takes `-u <uuid>`), that a deploy needs a GitHub repository (with no origin it uploads a ZIP), that PLATFORM_APPS is the source of truth for action selectors (some of its action names resolve to no tool), and to measure quality with `crewai test`. Its router example also declared no labels, so `crewai flow plot` could not draw the branches. - Template: platform token, selector syntax, `apps:` YAML example and selector check; router labels; deploy prerequisites, `-y`, `.env` upload, org commands, `project_id`; best practice points at `crewai eval` inside the eval markers. - Docs: quickstart deploy prerequisite and `remove -u`, and router labels in concepts/flows, in en, ar, ko and pt-BR. - Tests parse every documented deploy/org command against the real CLI, run the routing example through the plot builder, and check the best practices stay numbered without gaps for crew and tool projects. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: base the quickstart ZIP-deploy note on any git origin The CLI deploys from git whenever an `origin` exists, GitHub or not, and uploads a ZIP only when there is none. The quickstart said "GitHub origin ... without one", which reads as if a non-GitHub origin takes the ZIP path. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
João MouraandClaude Opus 5.5
|
744bf60367 |
fix(cli): show why a deployment failed and fix deploy remove and lockfile checks (#8083)
* fix(cli): show why a deployment failed and fix deploy remove and lockfile checks - deploy status prints AMP's error_message and error_suggested_fix when a deployment has failed, verbatim (no Rich markup), instead of only "Provisioning Failed, try again." - deploy remove -u names the removed UUID instead of the local project, and a failed removal prints the API's error. - deploy validate asks `uv lock --check` (offline) whether uv.lock is current instead of comparing file times, which flagged metadata-only edits that `uv lock` never clears. File times remain the fallback without uv or for poetry.lock. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * docs: note that deploy status prints the failure reason Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> * fix(cli): print deploy validate findings without Rich markup render_report printed codes, titles, details and hints as Rich markup, so bracketed text vanished: `[stale_lockfile]`, `[project]`, `[tool.hatch.build.targets.wheel]`, `crewai[tools]`. Raw `uv` stderr in the new unverified_lockfile detail could also raise MarkupError on a stray closing tag. Escape every finding field. Also bound the test fixture's `uv lock` with a timeout. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
João MouraandClaude Opus 5.5
|
6b9e62180e |
fix(core): give trigger list and payload requests a 60s timeout (#8081)
`crewai triggers list` timed out on every real organization: the apps listing takes about 11s and these two calls fell back to httpx's 5s default. Other PlusAPI calls keep their current timeouts. Co-authored-by: Claude Opus 5.5 <noreply@anthropic.com> |
||
|
João MouraandClaude Opus 5.5
|
9d5fd32661 |
fix(cli): deploy push reads AMP's source_type, so a ZIP deployment gets the new code (#7980)
#7345 made push follow how AMP holds a deployment's code, read from a `zip_deployment` boolean on the status. AMP's status never sends that field: it sends `source_type` ("zip", "studio", "github", "any_git"). So push always fell back to the local origin remote, and a ZIP deployment whose project has a git remote was redeployed by uuid — AMP rebuilt the ZIP it already held, not the code being pushed. Seen on a real deployment: after raising crewai to 1.15.26, `crewai deploy push` rebuilt the stored 1.15.23 ZIP ("CrewAI version detected: 1.15.23"), with no upload. `zip_deployment` is still read first when a server sends it. Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Lorenze Jay
|
d522c31339 | feat: bump versions to 1.15.27 (#8001) | ||
|
João MouraandClaude Opus 5.5
|
b3c4409bc5 |
feat(cli): record why an evaluation stopped before its verdict (#7963)
* feat(cli): record why an evaluation stopped before its verdict `cli_usage:evaluate` counts presses of Evaluate and `cli_usage:eval` counts evaluations that started, but about four in five presses never become an evaluation and nothing said why. Every stop now counts once as `cli_usage:eval_stopped:<reason>` from a closed list: untraced, declined, no_project, login_unreadable, network_error, amp_401/403/404/4xx/5xx, invalid_response, unexpected. The reason rides on EvaluationStoppedError from where it is raised; messages, exit codes and screens are unchanged. A run that `crewai eval` hands to the run app and that turns out untraced is counted by the command only, so one stop is never counted twice. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(cli): count each eval stop once, with the reason shown - the run app answers for a delegated run it could not start, so the waiting crewai eval does not start it again and count the stop twice - an unreadable saved login on the unattended path counts as login_unreadable, not untraced - an evaluation AMP itself failed counts as evaluation_failed, in the CLI and the app; --models unchanged Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
João MouraandClaude Opus 5.5
|
ed71a6f4d6 |
crewai eval --models: per-run cost and time, and no repeated progress lines (#7965)
* crewai eval --models: per-run cost and time, and no repeated progress lines The comparison table printed each setup's cost and time summed over all its runs, while the brief and the report say them per run. It now prints cost_per_run / seconds_per_run under 'cost / run' and 'time / run' (the totals, headed 'cost' / 'time', only when the service sends no per-run figure; a column never mixes the two). Progress deduped only against the previous line, so parallel setups made polls alternate and the same lines printed again and again. Each line is now said once per wait. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * Keep the per-run unit when a model is unpriced, and explain its dash Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
João MouraandClaude Opus 5.5
|
24a1ddf23f |
feat(cli): record what the run app's Deploy met before an attempt (#7964)
About half the presses of Deploy in the run app never reach a deployment attempt, and nothing said why. With no saved login the press raises before the attempt is counted and prompts for one. That prompt now counts once as `cli_usage:deploy_login_prompted`, and a press that still never reaches an attempt counts as `cli_usage:deploy_stopped:<reason>`: login_still_required, login_abandoned or login_failed. Counting re-raises; what the user sees is unchanged. Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
João Moura
|
1eff1d6bc9 | feat: bump versions to 1.15.26 (#7961) | ||
|
Sathyajith MCandVidit Ostwal
|
b90fa6e822 |
fix(cli): show actual task output in log-tasks-outputs (#7936)
* fix(cli): show actual task output in log-tasks-outputs * fix(cli): show task result instead of storage blob in log-tasks-outputs --------- Co-authored-by: Vidit Ostwal <110953813+Vidit-Ostwal@users.noreply.github.com> |
||
|
Lorenze Jay
|
7ed27d764a | feat: bump versions to 1.15.25 (#7945) | ||
|
Lorenze Jayandgithub-actions[bot]
|
82d09de711 |
feat: bump versions to 1.15.24 (#7941)
* feat: bump versions to 1.15.24 * chore: update tool specifications --------- Co-authored-by: github-actions[bot] <41898282+github-actions[bot]@users.noreply.github.com> |
||
|
João MouraandClaude Opus 5.5
|
00e2ff98e9 |
feat(cli): crewai eval prints a markdown brief when run by an agent (#7907)
* feat(cli): crewai eval prints a markdown brief when run by an agent When stdout is not a terminal (or CREWAI_DMN is on), `crewai eval` and `crewai eval --models` print the poll response's `brief_markdown` after the verdict and the report link — written straight to stdout, never through Rich markup — or "Brief (markdown): <brief_url>" when only the link came. An older AMP sends neither and nothing extra is printed. In a terminal nothing changes. Exit codes unchanged. `crewai eval --help` and the `--models` help say so, and the AGENTS.md the crew, flow and declarative-flow templates scaffold gains an "Evaluating" section telling a coding agent to run `crewai eval` after a change and act on the brief. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * docs(cli): an agent re-runs before it re-evaluates; each command's exit rule; the brief may be a link Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(cli): keep the eval guidance out of tool projects; the brief is conditional in --help `crewai create tool` copies the shared AGENTS.md too, but a tool project has no `crewai run` and no traced run for `crewai eval`. The evaluation blocks are marked in the template and dropped for a tool (`evaluation=False`); crew, flow and JSON-crew projects keep them, markers stripped. --help now says the brief prints when AMP provides it, else its link, else nothing more. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Sathyajith MCandVidit Ostwal
|
e836a191cf |
fix(cli): correct train filename error message (#7903)
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Vulnerability Scan / Detect changes (push) Has been cancelled
Vulnerability Scan / pip-audit (push) Has been cancelled
Nightly Canary Release / Check for new commits (push) Has been cancelled
Nightly Canary Release / Build nightly packages (push) Has been cancelled
Nightly Canary Release / Publish nightly to PyPI (push) Has been cancelled
* fix(cli): correct train filename error message * fix(cli): fail train on invalid input with nonzero exit --------- Co-authored-by: Vidit Ostwal <110953813+Vidit-Ostwal@users.noreply.github.com> |
||
|
João MouraandClaude Opus 5.5
|
1b9bbccbed |
feat(cli): crewai eval --models, and llm_overlay swaps models as well as roles (#7811)
Build uv cache / build-cache (3.10) (push) Has been cancelled
Build uv cache / build-cache (3.11) (push) Has been cancelled
Build uv cache / build-cache (3.12) (push) Has been cancelled
Build uv cache / build-cache (3.13) (push) Has been cancelled
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Vulnerability Scan / Detect changes (push) Has been cancelled
Vulnerability Scan / pip-audit (push) Has been cancelled
Nightly Canary Release / Check for new commits (push) Has been cancelled
Nightly Canary Release / Build nightly packages (push) Has been cancelled
Nightly Canary Release / Publish nightly to PyPI (push) Has been cancelled
* feat(llm): llm_overlay maps models too, not only agent roles A key `model:<provider/model>` maps every LLM built from that model string inside the block, and `model:*` every LLM built from any model string. That reaches what a role cannot: a flow step's own `LLM(...).call()`. The key is read in `LLM.__new__` (and so `create_llm` and an agent's `llm="..."`), and on an agent's declared llm built outside the block; a role key still wins for that agent. The caller's settings follow the mapped model by the rule a declared llm's do. `MODEL_KEY_PREFIX` is public so another package can feature-detect the form. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat(telemetry): allow-list cli_usage:eval_models attributes The span `crewai eval --models` records carries whether the caller was logged in, the models compared (provider/model names) and how many, in both telemetry classes. Nothing about the run, its output or the organization. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat(cli): crewai eval --models compares models on the deployment `crewai eval --models "a,b"` (and `--deployment UUID` when AMP cannot tell the deployment from the project id) asks AMP for a models evaluation: the project id, the models from one comma-separated list (stripped, repeats dropped, provider/model required, 1-5) and the project's eval.jsonc. It needs the login, prints and opens the report link, says where the comparison is while it runs, then prints one row per model (goal, tasks, agents, tools, cost, time; the deployed one marked, the best per column starred) and the top three suggestions. Exit 0 when it finished, 1 when it failed or could not start. Everything from the wire is printed as Text. The comparison is counted as `cli_usage:eval_models` once AMP accepts it. Mode 1 now sends the project id with `create_evaluation` too, so every evaluation is filed under the project it is about. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat(cli): crewai deploy create and push send the project id Every create (git or ZIP) and every push (redeploy by name or uuid, or a ZIP update) carries `[tool.crewai].project_id` when the project has one, so AMP knows which project a deployment runs and `crewai eval --models` can find it. The id is read, never minted: a deploy does not rewrite pyproject.toml. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(llm): a model key maps an LLM whose declared SDK is not installed Nothing is built on the declared model, so a missing SDK for it is no reason to fail the mapped build; its credentials cannot be matched to the new provider, so none are carried. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * feat(cli): crewai eval --models leaves its criteria behind as eval.jsonc When the comparison's answer carries `eval_config` and the project has no eval.jsonc, it is written through `write_eval_config` and the same one line Mode 1 prints says so. An existing file is never overwritten. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(llm): a model key compares providers by provider, not by class `_same_route_provider` now follows `_same_provider`'s rule case for case: a LiteLLM-routed `LLM(...)` is on the same provider as a native route when its provider names that native class, so its key and endpoint follow a model key to another model of the same provider. Another provider still gets none. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * chore(llm): say why mark_mapped may skip an object, and one import form in its tests Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(cli): usage stats name a compared model only when it is a public one `cli_usage:eval_models` carried the `--models` strings as typed, and those can identify a customer: a fine-tune id, an Azure deployment name, a self-hosted model or host. A model is now sent by name only when it is an exact entry of crewAI's model catalog (`LLM_CONTEXT_WINDOW_SIZES`), else as `<provider>/other`, and a provider crewAI does not route as `other/other`. An `ft:` id is never sent. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(cli): a refused comparison's test no longer keeps a None return Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * test(llms): re-importing OpenAI's completion module restores its package attribute `test_openai_completion_module_is_imported` deletes the module from sys.modules and lets `LLM(...)` import it again, which binds the new module on `crewai.llms.providers.openai` as well. monkeypatch restored only the sys.modules entry, so every later test on that worker saw two module objects for one name: `patch("crewai.llms.providers.openai.completion.X")` patched the attribute's module while `from ... import X` read sys.modules'. On Python 3.10 that made tests/llms/azure/test_azure_responses.py fail whenever it shared a worker with test_openai.py and test_crew_loader.py, which the new tests' even split now did. The attribute is restored too. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(llm): model keys strip only a model's own prefix; a role builds from the declaration Two review findings on the model-for-model overlay: - `overlay_model_for_model` took the text after the first slash as another name for the model, so an aggregator's route (`openrouter/openai/gpt-4o`) matched a `model:openai/gpt-4o` or `model:gpt-4o` key meant for the native model. A prefix is now stripped only when it is a native provider's own and what remains has no slash; an instance on an aggregator is compared as `<provider>/<its model id>`. - With a role key and a model key both active, the model key mapped the declared llm first and the role's model was built like that mapped instance, losing the caller's key and endpoint. The overlay now remembers what the caller declared for every llm it built, and a role key is built from that declaration. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(llm): model keys read a provider's other names; a LiteLLM choice survives a missing SDK - The router's provider aliases are now one table, `crewai.llm.PROVIDER_ALIASES`, and `llm_overlay` compares model names through it: `google/x` and `gemini/x` are one model whether the llm is built in the block or was built before it (a prebuilt instance records `gemini`). An exact key still wins, and an aggregator's route is still compared whole. - When the declared model's SDK is missing, a caller's explicit `is_litellm=True` is kept for the mapped model instead of dropping to a native route. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(deploy): a git create sends the project id once, at the top level AMP reads project_id at the top level on every deploy route, so the git create no longer repeats it inside `deploy`. The redeploy docstring records why its optional body is safe on an AMP that predates project ids. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
João MouraandClaude Opus 5.5
|
19d2ebcc8c |
fix(cli): crewai eval exits 1 unless the gate passed, and says to log in when nothing was traced unattended (#7806)
* fix(cli): `crewai eval` exits 1 unless the gate passed, and says to log in when nothing was traced unattended The exit code is what a CI job reads, and it said only whether the evaluation finished: a run whose goal gate FAILED exited 0, so a pipeline gating on `crewai eval` waved it through. It is the gate's now — 0 only for PASSED; a failed gate, one without a verdict, and an evaluation that stopped are 1. The command's help says so. With no terminal and no login, an anonymous run's trace stays on the machine even with tracing on (nobody is there to approve the upload). `crewai eval` then told the user to turn tracing on, which they had — the same loop again. When tracing is on and nobody is logged in, it now says what traces an unattended run: `crewai login`. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(cli): an unreadable login is the reason given when nothing was traced unattended Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> * fix(cli): print the unattended reason as text, never markup Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5.5 (1M context) <noreply@anthropic.com> |
||
|
Lorenze Jay
|
c018da6546 | feat: bump versions to 1.15.23 (#7804) | ||
|
João MouraandClaude Opus 5.5
|
4dcd19aa2e |
fix(tracing): turning tracing on is the answer, and an Evaluate button in the TUI (#7765)
* fix(tracing): turning tracing on is the answer; stop asking again
Two things a user who ran `crewai eval` sees today.
`crewai eval` offers to turn tracing on and run the crew. The run finishes —
and the ephemeral buffer asks "Share this execution trace with CrewAI?", which
is the same question again. Turning tracing on IS consent: `tracing_asked_for()`
is true when `CREWAI_TRACING_ENABLED` says so or when `tracing=True` was passed
in this context, and the buffer takes it as the answer. First-time
auto-collection still asks, because nobody asked for that one.
And the offer named the mechanism rather than the effect — "Turn tracing on
(CREWAI_TRACING_ENABLED=true stays in .env) and run the crew now?", then
"Tracing is on for this project (CREWAI_TRACING_ENABLED=true in .env)." The
variable is how it works, not what the user is agreeing to; both lines now say
what happens and leave the plumbing to the .env file it is written in.
491 telemetry tests and 62 eval-CLI tests green.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* fix(tracing): the TUI's modal is a prompt too, and say where traces go
Two follow-ups from running it.
The terminal UI installs its own consent callback, so turning tracing on still
bought a modal at the end of the run — the callback is consulted before the
default prompt, and the check only guarded the prompt. A host's callback stays
authoritative, because an embedder may answer no for reasons of its own; what
changed is that the TUI's callback, which exists to ASK, now answers for itself
when tracing was asked for.
And having taken the variable out of the offer, the offer should say what saying
yes means: "Turn tracing on and run the crew now? The run's trace is sent to
CrewAI AMP", and afterwards "Tracing is on for this project — its runs are
traced to CrewAI AMP."
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(tui): an Evaluate button beside View Traces and Deploy
A run finishes in the terminal UI and the next question is usually "was that
any good?" — which `crewai eval` answers, from the command line, about the run
that just happened. The button is there now, with `e` bound to it.
It behaves like Deploy rather than like View Traces: it leaves the UI and then
runs the command, because `crewai eval` talks — it prints the link to follow,
waits for the verdict and prints that — and none of it belongs inside a
full-screen app. It calls `eval_crew()` itself rather than reimplementing any
of it, so the button and the command can never say different things; a clean
SystemExit is the command deciding, and a failure is reported without failing
the run that already succeeded.
Declarative flows get it too — same button, same tail.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(telemetry): count `crewai eval`, not only the button that runs it
The TUI's buttons each record a `cli_usage:` feature span, so the new Evaluate
button was counted — but the command it runs was not, and most people will type
`crewai eval` rather than press a button. Every evaluation now records
`cli_usage:eval`; the button keeps its own `cli_usage:evaluate`, so the
difference between the two is how many were started from the TUI.
It counts the command and nothing about the run: no id, no project, no verdict.
And it is wrapped like every other telemetry call here — a missing tracer or a
refused span never stops an evaluation.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* feat(telemetry): say which run was evaluated, and by whom
`project_id` already rides every span. What only this command knows is which
run is being graded, whether the caller was logged in, and — when they are —
which organization they are logged in to. A feature span can carry attributes
now, and `cli_usage:eval` carries those three.
What it does not carry is anything about the work: no inputs, no output, no
verdict, no criteria. The execution id is the join key; what was found lives in
AMP, where the evaluation itself is.
An anonymous caller reports `authenticated=false` and an empty organization, and
is not asked for one.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Address the review: a terminal, a thread, and the right run
Three findings worth acting on, one already fixed, and ruff.
**The security review is right.** `prompt_user_for_trace_viewing` fails closed
where no one can answer, so an anonymous run in CI has never uploaded. Treating
the switch as consent everywhere would have changed that: a project `.env`
copied into CI carries the variable, not the person. `tracing_asked_for()` now
answers only where a prompt could have been shown — the double-ask this PR is
for happens in a terminal, and nothing else moves.
**The ContextVar never crosses the worker.** `set_tracing_enabled` runs in the
crew's validator, on the thread the crew was BUILT on, and a flow never sets it
at all — so `tracing=True` without the env var still opened the modal. The TUI
reads the declaration off the object as well: the same yes, wherever written.
**The button now names the run it watched.** `_chain_eval()` called `crewai eval`
with no id, which looks up the last recorded run — a PREVIOUS run when this one
was not traced, and an offer to run the crew again when there is no record at
all. The TUI remembers what was recorded before the run, hands the new id to the
command, and says "this run was not traced" rather than grading someone else's.
**Already fixed**: telemetry is recorded after `get_or_create_project_id()`, not
before — that moved when the span gained its attributes.
492 telemetry tests, 187 CLI tests. Ruff format and check clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Close all three doors the prompt closes, not just the terminal
The security review named three places `prompt_user_for_trace_viewing` fails
closed — no terminal, a suite under test, and a host that suppressed tracing
messages — and the first pass only honoured the terminal. A run under test, or
inside a host that asked for silence, could still have uploaded on the strength
of a variable.
The three conditions are one function now, `_prompt_can_be_shown()`, used by the
prompt itself and by `tracing_asked_for`, so the rule and the prompt it answers
cannot drift apart. Where the question could not have been put to anybody, there
is nothing to answer and nothing leaves the machine.
The terminal UI decides for itself, and says why: it silences crewAI's console
to keep it out of its own layout, which is a reason a PROMPT cannot be shown,
not a reason to ask a second time — it has a screen and somebody in front of it.
494 telemetry tests, 88 TUI tests. Ruff clean.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* Tests: a real session for the consent rule, and a laid-out screen for a click
The rule that a declared CREWAI_TRACING_ENABLED needs no modal only holds
where a prompt could have been shown, and a test suite is one of the places
it could not — so the three cases about a real session say a person is here.
The consent click waited for the screen, which is not the same as waiting for
its buttons to have a region; it now waits for the layout it clicks on.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* A run started for an evaluation closes itself, even in a child process
A project's crew and flow are run through `uv run …`, so the app lives in
another process and the ContextVar saying "an evaluation is waiting" never
reaches it: the screen stayed open and `crewai eval` waited behind it, which
is the trap the handshake was meant to remove. The environment is the one
thing that crosses into that process, so the command sets a variable there and
clears it after; the id travels back the way it already did, through the
project's last-run record. A run that FAILED leaves too — the command behind
the screen has its own way of saying there is nothing to grade, and it cannot
say it while somebody has to quit a screen first.
And the spans a run exports reach AMP a moment after the run ends, sometimes
later than that, so a command that just watched the run would ask for a trace
still in flight and be told, correctly and uselessly, that there is nothing
there. A run we know is fresh now waits up to two minutes and says it is
waiting; an id somebody typed fails at once, as it always has.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* The evaluation happens in the app, on the screen the reader is already on
Closing the app the moment the run ended and leaving somebody in a bare
terminal, waiting some seconds for a link to appear and a browser to open, is
worse than the screen it replaced. So the evaluation runs where the run does:
`Evaluating this run` under the run's own line, the report link beside it, the
browser opened from there, and `Goal gate PASSED · goal 5/5 · …` when it comes
back. Nothing closes; the reader leaves when they are done reading, and the
link is printed once more on the way out so it outlives the screen.
The Evaluate button does the same thing now instead of quitting the app and
chaining into the command, so there is one implementation of one experience —
`_chain_eval` and `_want_eval` are gone. Pressing it again opens the report it
already made rather than paying for a second evaluation.
Underneath, `crewai eval`'s two steps became callable as values: they raise
`EvaluationStoppedError` with the sentence to show instead of printing and
exiting, and `evaluate_run()` is the app's one door — the url as soon as AMP
gives it, every unfinished answer while it waits. The command keeps the
terminal wording it had; when it ran the crew itself, it now has nothing left
to say and steps aside for the app it opened.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* The button says which report it opens
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* The button works while it waits: it spins, and it stays pressable
A dimmed label that never moves reads as a screen that has stopped. The
Evaluate button now carries the same spinner the run's own line does, the
clock goes back up to eight frames a second while the evaluation runs, and the
button is no longer disabled — the report page is where the progress is, so
pressing it while it waits is the useful thing to do.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* View Traces opens the traces
The button never had a link. crewAI grants one for every exported run and
prints it after a run — but never under a TUI, where its console is kept out
of the layout — so the app had nothing to open and said so, which read as a
feature that had stopped working.
The link is now recorded beside the rest of what crewAI writes when a run's
spans reach AMP, and the button reads it off that record, matched to THIS
app's execution rather than to whatever ran last in the project. When there is
no link the notice says which of the two reasons it is: a run nobody traced,
or a traced run AMP granted no viewer for — only one of those is the reader's
to fix.
Verified against a real anonymous run: pressing `t` opens
app.crewai.com/crewai_plus/ephemeral_traces/<id> and shows it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* The review's five: a trust boundary, a fallback, a retry, an exit and a count
**A project's `.env` cannot lend itself the login.** `evaluate_run()` built its
AMP client after the project's `.env` had been loaded with override, so a
`CREWAI_PLUS_URL` a project put there landed in the set of origins trusted with
the saved `crewai login` bearer. The command reads that variable BEFORE the
`.env` — a shell export is this machine speaking — but inside the app there is
no such "before", so the variable is left out of the set entirely. The request
still follows it; the credential does not.
**A run no app evaluated is still evaluated.** The command returned as soon as
a trace existed, assuming the app had graded it — but a conversational session
never ends the way a crew does and a flow with human feedback takes the
terminal instead, so `crewai eval` could exit 0 having graded nothing. The app
now leaves a marker naming the run it started evaluating, and the command
grades anything that marker does not name.
**An evaluation that stopped can be tried again.** The button re-enabled itself
after a failure but the action still treated any existing evaluation as
finished, so a 502 could only be escaped by re-running the crew.
**A login that exits is shown on screen.** `saved_login()` fails by exiting,
which is not an `Exception` and so slipped past the worker's hands: the button
could sit on "Evaluating…" forever with the reason gone with the exit.
**An evaluation the app runs is counted.** `cli_usage:eval` is the count of
evaluations that begin, and the ones that begin in the app were missing from
it.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* A stop is a sentence, and the app's own word settles what it graded
**"1" is not a reason.** A credential that exists but cannot be read stopped
the evaluation by printing its sentence and exiting — under the app the print
lands beneath the layout and the exit carries only a code, so the screen said
`1`. `saved_login()` now raises the stop like everything else that stops an
evaluation, the command catches it where it builds the client and prints it as
before, and the app shows it. The test that claimed to cover this raised a
`SystemExit` carrying a sentence, which is not what `_fail` raises; it now
breaks the credential store and reads what reaches the screen. The worker
still catches an exit, and says that the reason did not survive rather than
echoing its code.
**The record is the project's last finish, not this run.** Another run
finishing in the same project replaces it while the app is open, so "is the
marker about the id in the record" could send the command off to evaluate a
different execution. The marker is now the app's own word about the run it
watched — the id it evaluated, or null when it looked and found nothing to
grade — and the command trusts it over the record, matching on time rather
than on an id that may already have been overwritten. No marker still means no
app got here, which is the conversational session and the flow that takes the
terminal, and the command grades those itself.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* A project can say what good means, and `crewai eval` carries it
An evaluation grades a run against the crew's own `expected_output`s — which
is the prompt that produced the output, so it catches a crew that ignored its
own contract and nothing else. What discriminates lives in criteria somebody
writes, and there was nowhere to write them.
Now there is. After the first evaluation the criteria it used are written to
`eval.jsonc` in the project — once, never over a file that exists, because the
one thing worse than no criteria is criteria that vanish whenever somebody
evaluates — and the line that says so points at what to do with it. From then
on the file travels with every evaluation, as it was written, comments and
all: what a criterion means is the grader's to read, not this command's.
Both doors do it, since most evaluations now happen inside the run app: the
app writes the file and names it in the line that outlives its screen, and
`crewai eval` in a terminal says it there.
The file is bounded at the same 64KB AMP holds the field to, so one that would
be refused is never uploaded and the run is graded anyway.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
* The review's asks: anonymous eval stats, a handshake a project cannot forge
Usage stats are anonymous, so the `crewai eval` count says whether the caller
was logged in and nothing else — no execution id, no organization (the product
decision on this PR). Which runs were evaluated, and by whom, is AMP's record.
A feature span now keeps only the keys its own feature lists, in both
telemetry classes; everything else — the dimensions every span carries
included — is dropped.
The run app's "an evaluation is waiting" handshake was an environment variable
any project's `.env` could set, which would start an evaluation with the
machine's login on a plain `crewai run`. Its value is now a random token that
counts only while the command's file of that name exists in the user's crewAI
data directory; a `.env` cannot create a file.
And three smaller ones: when no app evaluated the run, the command grades the
project's record only if it was written after this run began; the "Wrote
eval.jsonc" line after the app closes now appears (the key it read was never
set); an oversized eval.jsonc is said once, through the caller's note, rather
than printed under the TUI on every retry.
uv.lock is back to main's: the change was re-resolution drift, nothing this PR
needs.
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
* A report link is a web address, and an anonymous request names no organization
Two from the security review of
|
||
|
Vidit Ostwal
|
0e6440b397 |
docs(cli): guide assistants to platform tools (#7581)
* feat(cli): prioritize platform integrations * docs(cli): guide assistants to platform tools * test(platform): update missing token assertion * fix(platform): retain integration token environment key * chore: ignore Codex workspace files * feat(platform): expand integration catalog * feat(platform): show integration action counts * feat(platform): select agent actions separately * feat(platform): group integration applications * fix(cli): keep platform picker navigation in place * fix(cli): validate platform apps once * fix(platform): use one integration token variable * test(platform): update integration token assertion * fix(cli): clarify platform token validation * fix(cli): show platform token prompts * refactor(platform): expose application catalog * refactor(platform): export catalog from Python * docs(platform): document catalog regeneration * fix(platform): resolve catalog type export |
||
|
Vidit Ostwal
|
a2ea278210 |
feat(cli): prioritize popular platform integrations (#7573)
* feat(cli): prioritize platform integrations * test(platform): update missing token assertion * chore: ignore Codex workspace files * feat(platform): expand integration catalog * feat(platform): show integration action counts * feat(platform): select agent actions separately * feat(platform): group integration applications * fix(cli): keep platform picker navigation in place * fix(cli): validate platform apps once * fix(platform): use one integration token variable * test(platform): update integration token assertion * fix(cli): clarify platform token validation * fix(cli): show platform token prompts * refactor(platform): expose application catalog * refactor(platform): export catalog from Python * docs(platform): document catalog regeneration * fix(platform): resolve catalog type export |
||
|
João MouraandClaude Opus 5
|
bdd1bc6200 |
chore(cli): move crewai eval under experimental (#7707)
CodeQL Advanced / Analyze (python) (push) Has been cancelled
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
Vulnerability Scan / Detect changes (push) Has been cancelled
Vulnerability Scan / pip-audit (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
Nothing about the command changes. It is still `crewai eval`, still listed in `--help`, still takes `--run`, still prints the same lines. Only the module moved, from `crewai_cli/eval_crew.py` to `crewai_cli/experimental/eval_crew.py`, with its tests alongside. The package says what the command's location could not: its output and its options may change between releases without the deprecation cycle a settled command gets. That is honest about where it stands — most of what it depends on lives outside this repository, in an AMP endpoint and the evaluator behind it, and both are new enough that the report's shape is still moving. The command is already built to absorb some of that, printing whatever the evaluation graded rather than a list of its own, but the promise it makes to a user should match. `crewai/experimental/` has the same role on the library side, so this is the convention the repository already has rather than a new one. The CLI had no experimental package before; it does now, and `crewai eval` is its first tenant. Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com> |
||
|
João Moura
|
9de80df7f3 |
fix(cli): print whatever areas the evaluation graded (#7701)
`crewai eval` printed a fixed tuple of area names. The areas belong to the evaluation, not to the client, so a fixed list does two wrong things the moment the evaluator's vocabulary moves ahead of an installed CLI: it drops every area it has not heard of, and it invents "not measured" for ones that no longer exist. A user would see one real grade and three phantom blanks, with no sign that anything had been dropped. It now prints the areas it was sent, in the order they arrived. The verdict is still validated exactly as before — a grade is an int in 1..5 or null — so a malformed payload is still a protocol error rather than something printed. This lands before the evaluator's own change so that an installed CLI keeps working through the rollout rather than after it. |
||
|
João MouraandClaude Fable 5.1
|
0a3b8912b3 |
feat(cli): crewai eval evaluates the last traced run through AMP (#7649)
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Vulnerability Scan / Detect changes (push) Has been cancelled
Vulnerability Scan / pip-audit (push) Has been cancelled
Nightly Canary Release / Check for new commits (push) Has been cancelled
Nightly Canary Release / Build nightly packages (push) Has been cancelled
Nightly Canary Release / Publish nightly to PyPI (push) Has been cancelled
* feat(cli): crewai eval evaluates the last traced run through AMP `crewai eval` reads `.crewai/last_run.json` — the record crewAI writes when a traced run's spans reach Wharf — and asks AMP to evaluate that run: POST /crewai_plus/api/v1/tracing/evaluations with the execution id, sending the saved `crewai login` when there is one and nothing otherwise. AMP answers with an evaluation id and a URL; the command prints the URL, opens it, waits for the verdict and prints it (goal gate and the four grades), exit 1 only when the evaluation itself failed. `--run EXECUTION_ID` evaluates another run. With no traced run recorded it offers to turn tracing on for the project (`CREWAI_TRACING_ENABLED=true` in .env, set_key so nothing else in the file moves) and run the crew now with `crewai run`; without a terminal, or declined, it prints the three steps instead. A run that leaves no record behind is explained, never guessed at. AMP's refusals are printed in its own words: a run that needs an account (401 account_required), a refused credential (then `crewai login`), a run AMP does not hold (404), rate limiting (429 with Retry-After), and any other status with AMP's message. The two AMP calls live on the CLI's PlusAPI subclass, so no crewai-core release is needed. Who may evaluate what is AMP's decision, not the command's: an anonymous run once without an account, then it needs one; a run traced while logged in for that organization's members; a deployment execution for members who may see its traces. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(cli): crewai eval — guard the project, survive AMP blips, name the right subject, read the whole record Review findings, each reproduced before the fix: - The run-it-now offer wrote CREWAI_TRACING_ENABLED into ./.env before checking the directory is a crewAI project, then run_crew() died on a missing pyproject.toml with a traceback. Now: no pyproject.toml → one sentence, exit 1, nothing written. - httpx errors (AMP unreachable, a timeout) surfaced as tracebacks. Now the start says "Could not reach AMP to start the evaluation: …"; while waiting, an unreachable AMP or a 5xx is retried up to POLL_RETRIES consecutive times, then reported with the URL — the evaluation keeps running server-side either way. - The POST now carries a 120 s timeout (AMP reads the run's spans inside it), the poll 30 s. - A 200 whose body has no known status (a non-dict, no status, a status outside queued/running/done/failed) polled forever. Now it stops with the status it saw and the URL. - A 404 without a JSON message read "AMP holds no run <evaluation id>" while polling. _refused takes "run <id>" / "evaluation <id>" and says "AMP answered 404 for <subject>" — AMP's own message still wins. - The record's amp_base_url was ignored; the CLI now evaluates the run at the AMP it was traced to. --run keeps the configured AMP. - The post-run explanation names the third cause: a crewai older than the version that records the last run. Tests for each, plus the previously untested paths: Ctrl-C exits 130, a 2xx without an id, a refusal mid-poll, DMN opens no browser, --run skips the offer. 51 passed in lib/cli/tests (eval + plus_api). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(cli): crewai eval — the credential goes only to the configured AMP; an explicit yes before running the crew; a done answer needs a well-formed verdict Review findings (CodeRabbit, the PR gate): - The record's amp_base_url was handed to PlusAPI beside the saved login, so a modified .crewai/last_run.json could send the token to any origin. The client is now built from the configured AMP only (CREWAI_PLUS_URL, the saved settings, app.crewai.com); the project's .env is loaded first, as `crewai run` loads it, so the configured AMP is the one the run was traced to. A record naming another address gets a one-line note and no credential. - The offer to turn tracing on and run the crew defaults to no and says the .env change stays; Enter no longer spends a crew run. - A `done` answer whose verdict is missing or malformed (no gate, grades not an object, a grade not an int or null) is a protocol error, exit 1, instead of an INCONCLUSIVE line with exit 0 or an AttributeError. Tests for each: a foreign origin in the record with a saved token, the .env-loaded same-AMP case, seven malformed verdicts, the prompt's text and default. 59 passed (eval + plus_api); ruff and mypy clean. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(cli): crewai eval — Enter accepts the offer to run the crew (y/n, Y default; the prompt names both effects) João's call (2026-09-20): the confirm is y/n with Y as the default. The prompt still says tracing stays on in .env and that the crew runs now. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
João MouraandClaude Opus 5
|
6e1fa8cf2a |
feat(tracing): record the last traced run for crewai eval instead of printing it (#7648)
* feat(tracing): record the last traced run for crewai eval instead of printing it After a traced run, crewAI printed a panel with the execution id and a viewer link. The id is internal, and the panel exposed it for no purpose a user has. Nothing is printed now. Instead, once the run's spans have reached Wharf, crewAI writes `.crewai/last_run.json` in the project: the execution id, when the run started and ended, whether it was traced anonymously or under an account, and the AMP base url. `crewai eval` reads it back, so the user evaluates their last run without pasting anything. GrantSpanExporter.record_export replaces show_trace_summary at the two finish points (an authenticated run's shutdown, an anonymous run's share). Only a run whose every export succeeded is recorded — a partial export would name a run the grader could not read whole. Recording is silent in the TUI and under message suppression too, off under the test suite, and inert inside a deployment, where the platform binds the execution before crewAI's own tracing starts. The record is written atomically and a write failure never fails the run. The crew, flow and json_crew scaffolds now ignore `.crewai/` like the declarative flow scaffold already did. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(tracing): never record a truncated ephemeral run; resolve the record path inside the guard; pin the window and the once - An ephemeral buffer that dropped spans at its cap still uploaded the rest and recorded the run. The grader could not read that run whole, so it is no longer recorded (the upload still happens). Test added. - last_run_path() ran before the try in record_last_run; a cwd that vanished mid-run would have raised out of a function documented never to raise. It is inside now, with the same debug log and None. - Tests: the "recorded once" check counted an unchanged mtime, which proves nothing on a coarse filesystem — it now counts calls to record_last_run; the first-start/last-end fold across export batches had no value test — batches now arrive out of order and the ISO window is asserted. - Docs: the conversational-flows comment said "one trace link" for output that no longer exists; tracing.mdx now names .crewai/last_run.json, what it holds, who reads it, and when it is not written. en, ar, ko, pt-BR. 85 passed (test_session_trace_export + test_last_run); ruff and mypy clean. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(tracing): a temporary file of its own per writer; the docs name deployments - record_last_run wrote every record through one `.tmp` name, so two crews finishing together in one project could replace each other's temporary file and one record was lost. Each write now goes through tempfile.mkstemp in the same directory, then os.replace; a failed write removes its own temporary file. Tests: two writers use two names and leave only last_run.json; a failed replace leaves nothing behind. - tracing.mdx (en, ar, ko, pt-BR): nothing is recorded inside a deployment, where the platform owns the trace. 87 passed (test_last_run + test_session_trace_export); ruff and mypy clean. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(tracing): the run that finished last stays recorded, whichever writer comes last Two crews finishing together in one project: the older run's writer could reach the file after the newer one and leave the older run as "the last". record_last_run now compares finished_at (recorded_at when absent) with the record already there and keeps the later-finished run; the same run recorded again (a refreshed grant) and a record without a comparable time never block the run just finished. No lock file: the file is a convenience pointer and the compare closes the ordering, leaving a microsecond window between read and replace that two runs finishing in the same instant could hit — either is then a fair "last run". Test: the older writer replaces after the newer one; the newer stays. 88 passed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(tracing): a same-millisecond tie keeps the record already there; nothing is recorded inside a deployment - The times in the record are stored to the millisecond; two runs finishing in the same millisecond compared equal and the later writer won, which could leave the older run recorded. A tie now keeps what is there. No finer order is worth a field in the file. - recording_enabled() is false when the platform's integration token is present: a deployment normally never reaches this code (the host binds the trace before crewAI would start its own), but a container that did not would have written a file the platform never reads, against what the docs promise. Test added. 89 passed; ruff and mypy clean. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(tracing): only a run that provably finished later holds the record My previous commit compared `finished_at` and fell back to `recorded_at`, which broke two existing tests on CI: two records written in the same millisecond with no completion time compared equal, so the second write was dropped and the older run stayed. Locally the two writes happened to land in different milliseconds, so the suite passed — a timing-dependent bug, not a CI quirk. The rule is now narrow: a write stands unless the record already there is a DIFFERENT run with a later `finished_at`. No completion time on either side, the same run recorded again, or a tie to the millisecond all leave the write to stand, so "the last run recorded" stays what a reader gets. The case the rule exists for is unchanged: the older-finished writer arriving last does not replace the newer one. 485 passed in lib/crewai/tests/telemetry (test_last_run run five times for timing); ruff and mypy clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(tracing): serialise the write across processes; drop the platform-token guard, which was not a deployment test - Reading the record, deciding and replacing it are now one step, held under a lock file beside the record, so a writer can no longer decide against a record another process has already replaced. Where the platform has no flock the write goes ahead unserialised: the record is a convenience pointer for `crewai eval`, never worth failing a run over. The regression drives three writers at once and fails without the lock. - The deployment guard I added yesterday read CREWAI_PLATFORM_INTEGRATION_TOKEN, which is not a deployment marker: `crewai create crew` writes that token into a project's own .env for platform tools, and crew_base loads it, so the guard would have stopped recording for every developer who uses them. It is also unnecessary — a deployment kicks off with tracing off, so crewAI never builds the exporter and never reaches this module. Guard removed, with a test that a project carrying that token is still recorded. - The module docstring claimed the platform "binds the execution before crewAI would start its own tracing". It does not; it sets tracing off at kickoff. Docstring and the four tracing docs now say the real reason. 487 passed in lib/crewai/tests/telemetry; ruff and mypy clean. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * fix(tracing): a refused lock still records the run; the platform-token test exercises the real guard Two Cursor findings on the locking commit. - If flock itself failed — some network mounts refuse it — the OSError escaped _exclusive, the outer handler deleted the temporary file and record_last_run returned None, so the run was not recorded at all. A refused lock is now logged and the write goes ahead unserialised, matching what already happened when the lock file could not be opened or flock does not exist. Unserialised beats not recorded. - test_a_project_using_platform_tools_is_still_recorded used the `project` fixture, which stubs recording_enabled — the very thing under test — so the regression it guards could have come back green. It now stubs only project_dir and asserts the real guard. 13 in test_last_run, 489 across lib/crewai/tests/telemetry. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> |
||
|
Vidit Ostwal
|
977aa0c6ec |
feat(cli): improve platform integration setup UX (#7453)
* fix(cli): silence tool import warnings during platform setup * feat(cli): validate platform integrations concurrently * fix(cli): persist platform token during crew setup * fix(core): make settings write probe concurrency-safe * test(cli): simplify event loop fallback stub |
||
|
|
b9629cfb8f |
feat: add native Gemini 3.8 Flash support (#7284)
* feat(llm): add Gemini 3.8 Flash support - Register Gemini 3.8 Flash in CrewAI and CLI model catalogs - Add 1,048,576-token context window mapping - Add native Gemini provider support - Add Gemini 3.8 Flash to the CLI model picker - Add native model detection coverage for gemini/ and google/ formats - Add context window regression coverage Closes #7241 * test: add VCR cassette for gemini-3.8-flash * chore: stop tracking .env.test * Bringing back .env.test --------- Co-authored-by: Vidit Ostwal <110953813+Vidit-Ostwal@users.noreply.github.com> Co-authored-by: ViditOstwal <viditostwal@gmail.com> |
||
|
Lorenze Jay
|
b0343eb75f | feat: bump versions to 1.15.22 (#7520) | ||
|
|
5990ead5ec |
feat(cli): record why a deployment create failed (#7451)
* feat(cli): record why a deployment create failed `crewai deploy create` counts every attempt (`Create Crew Deployment`) and every success (`Crew Deployment Created`), but the gap between them carried no cause: among clients able to emit the success span, the CLI succeeds 96.7% of the time and the run TUI 36.4%, and nothing said why. A third span, `Crew Deployment Failed`, now fires for every failure after the attempt is counted, with a closed vocabulary `reason` (api_4xx, api_5xx, invalid_response, network_error, zip_error, user_declined, unexpected), the HTTP `status_code` when the API answered, and the existing `source`. Never the error message. The request path is factored into `_request_crew_creation`; every exception is classified, reported and re-raised unchanged, so CLI and TUI behaviour is the same as before. HTTP failures are classified before `_validate_response`, which still prints and exits as it did. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(cli): classify deploy create failures by status and by stage Review fixes on the failure span. Check the HTTP class before the body so a gateway's HTML page counts as api_4xx / api_5xx with its code. Treat a 2xx whose body is not a JSON object carrying uuid and status as invalid_response and exit cleanly, instead of emitting a success span and crashing in the display step. Recognise archive failures by a dedicated ArchiveError (a ValueError) raised from create_project_zip, so the git helpers' own ValueErrors no longer read as zip_error; a failed ZIP write is wrapped and its partial file removed. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * fix(cli): treat staging and temp-file failures as archive errors `create_project_zip` only wrapped the ZIP write, so an `OSError` while staging files or creating the temporary archive escaped as a bare `OSError` and the deploy command recorded it as `unexpected` instead of `zip_error`. The archive boundary now covers staging, temp-file creation and the write; the staging directory is removed on every path, and no partial archive is left behind. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> * feat(cli): tell a non-JSON 2xx apart from a non-creation 2xx A proxy's 200 HTML page and a JSON body missing the creation fields were both recorded as `invalid_response`. They are different failures, one in the network path and one in the API contract, so the deploy failure span now records `invalid_json` for the first and `invalid_creation_response` for the second. Requested in review. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com> Co-authored-by: Vidit Ostwal <110953813+Vidit-Ostwal@users.noreply.github.com> |