* feat(cli): crewai eval evaluates the last traced run through AMP
`crewai eval` reads `.crewai/last_run.json` — the record crewAI writes
when a traced run's spans reach Wharf — and asks AMP to evaluate that
run: POST /crewai_plus/api/v1/tracing/evaluations with the execution id,
sending the saved `crewai login` when there is one and nothing otherwise.
AMP answers with an evaluation id and a URL; the command prints the URL,
opens it, waits for the verdict and prints it (goal gate and the four
grades), exit 1 only when the evaluation itself failed. `--run
EXECUTION_ID` evaluates another run.
With no traced run recorded it offers to turn tracing on for the project
(`CREWAI_TRACING_ENABLED=true` in .env, set_key so nothing else in the
file moves) and run the crew now with `crewai run`; without a terminal, or
declined, it prints the three steps instead. A run that leaves no record
behind is explained, never guessed at.
AMP's refusals are printed in its own words: a run that needs an account
(401 account_required), a refused credential (then `crewai login`), a run
AMP does not hold (404), rate limiting (429 with Retry-After), and any
other status with AMP's message. The two AMP calls live on the CLI's
PlusAPI subclass, so no crewai-core release is needed.
Who may evaluate what is AMP's decision, not the command's: an anonymous
run once without an account, then it needs one; a run traced while logged
in for that organization's members; a deployment execution for members
who may see its traces.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(cli): crewai eval — guard the project, survive AMP blips, name the right subject, read the whole record
Review findings, each reproduced before the fix:
- The run-it-now offer wrote CREWAI_TRACING_ENABLED into ./.env before
checking the directory is a crewAI project, then run_crew() died on a
missing pyproject.toml with a traceback. Now: no pyproject.toml → one
sentence, exit 1, nothing written.
- httpx errors (AMP unreachable, a timeout) surfaced as tracebacks. Now
the start says "Could not reach AMP to start the evaluation: …"; while
waiting, an unreachable AMP or a 5xx is retried up to POLL_RETRIES
consecutive times, then reported with the URL — the evaluation keeps
running server-side either way.
- The POST now carries a 120 s timeout (AMP reads the run's spans inside
it), the poll 30 s.
- A 200 whose body has no known status (a non-dict, no status, a status
outside queued/running/done/failed) polled forever. Now it stops with
the status it saw and the URL.
- A 404 without a JSON message read "AMP holds no run <evaluation id>"
while polling. _refused takes "run <id>" / "evaluation <id>" and says
"AMP answered 404 for <subject>" — AMP's own message still wins.
- The record's amp_base_url was ignored; the CLI now evaluates the run at
the AMP it was traced to. --run keeps the configured AMP.
- The post-run explanation names the third cause: a crewai older than the
version that records the last run.
Tests for each, plus the previously untested paths: Ctrl-C exits 130, a
2xx without an id, a refusal mid-poll, DMN opens no browser, --run skips
the offer. 51 passed in lib/cli/tests (eval + plus_api).
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(cli): crewai eval — the credential goes only to the configured AMP; an explicit yes before running the crew; a done answer needs a well-formed verdict
Review findings (CodeRabbit, the PR gate):
- The record's amp_base_url was handed to PlusAPI beside the saved login,
so a modified .crewai/last_run.json could send the token to any origin.
The client is now built from the configured AMP only (CREWAI_PLUS_URL,
the saved settings, app.crewai.com); the project's .env is loaded first,
as `crewai run` loads it, so the configured AMP is the one the run was
traced to. A record naming another address gets a one-line note and no
credential.
- The offer to turn tracing on and run the crew defaults to no and says
the .env change stays; Enter no longer spends a crew run.
- A `done` answer whose verdict is missing or malformed (no gate, grades
not an object, a grade not an int or null) is a protocol error, exit 1,
instead of an INCONCLUSIVE line with exit 0 or an AttributeError.
Tests for each: a foreign origin in the record with a saved token, the
.env-loaded same-AMP case, seven malformed verdicts, the prompt's text and
default. 59 passed (eval + plus_api); ruff and mypy clean.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
* fix(cli): crewai eval — Enter accepts the offer to run the crew (y/n, Y default; the prompt names both effects)
João's call (2026-09-20): the confirm is y/n with Y as the default. The
prompt still says tracing stays on in .env and that the crew runs now.
Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
---------
Co-authored-by: Claude Fable 5.1 <noreply@anthropic.com>
Inline agent and crew actions can now use repository-backed agents
without duplicating role, goal, and backstory in each definition.
Examples:
* `agent.with.from_repository: support_specialist`
* `crew.with.agents.researcher.from_repository: researcher`
`PlusAPI.get_agent` now uses the shared synchronous request path so
project loaders can fetch repository agents without nested event loops.
* Update crewAI CLI with various enhancements and fixes
- Updated `create_json_crew.py` to require `crewai[tools]>=1.14.7`.
- Enhanced `git.py` with improved repository initialization, including automatic initial commit creation and exclusion patterns for initial commits.
- Modified `install_crew.py` to allow error handling during installation with an optional `raise_on_error` parameter.
- Expanded `plus_api.py` to include methods for creating and updating crews from ZIP files.
- Introduced a new `archive.py` for creating deployable ZIP archives of CrewAI projects, ensuring local artifacts are excluded.
- Updated `run_crew.py` to manage JSON crew dependencies and run crews in the project's environment.
- Enhanced deployment logic in `main.py` to handle ZIP uploads and improve user feedback during deployment processes.
- Added tests for new functionalities and ensured existing tests reflect recent changes in behavior and requirements.
* fix(cli): address deploy zip review feedback
* fix(cli): sync missing lockfile before deploy
* fix(cli): preserve remote deploy on git setup warnings
* test(cli): use single deploy main import style
* fix(cli): skip project install for json crew sync
* fix(cli): load json runner from source checkout
* fix(cli): skip json crew sync when locked
* fix(cli): address deploy zip review feedback
* fix(cli): pass env on zip redeploy
* fix(cli): harden json run and zip fallback
* fix(cli): validate before deploy lock install
* fix(cli): respect poetry lock for json runs
* fix(cli): align json zip wrapper detection
* fix(deps): bump starlette audit floor
* fix(cli): avoid auth retry for deploy exits
* fix(cli): update json zip script entrypoints
