mirror of
https://github.com/crewAIInc/crewAI.git
synced 2026-09-21 10:26:25 +00:00
fix(cli): crewai eval — guard the project, survive AMP blips, name the right subject, read the whole record
Review findings, each reproduced before the fix: - The run-it-now offer wrote CREWAI_TRACING_ENABLED into ./.env before checking the directory is a crewAI project, then run_crew() died on a missing pyproject.toml with a traceback. Now: no pyproject.toml → one sentence, exit 1, nothing written. - httpx errors (AMP unreachable, a timeout) surfaced as tracebacks. Now the start says "Could not reach AMP to start the evaluation: …"; while waiting, an unreachable AMP or a 5xx is retried up to POLL_RETRIES consecutive times, then reported with the URL — the evaluation keeps running server-side either way. - The POST now carries a 120 s timeout (AMP reads the run's spans inside it), the poll 30 s. - A 200 whose body has no known status (a non-dict, no status, a status outside queued/running/done/failed) polled forever. Now it stops with the status it saw and the URL. - A 404 without a JSON message read "AMP holds no run <evaluation id>" while polling. _refused takes "run <id>" / "evaluation <id>" and says "AMP answered 404 for <subject>" — AMP's own message still wins. - The record's amp_base_url was ignored; the CLI now evaluates the run at the AMP it was traced to. --run keeps the configured AMP. - The post-run explanation names the third cause: a crewai older than the version that records the last run. Tests for each, plus the previously untested paths: Ctrl-C exits 130, a 2xx without an id, a refusal mid-poll, DMN opens no browser, --run skips the offer. 51 passed in lib/cli/tests (eval + plus_api). Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
@@ -29,6 +29,7 @@ class TestPlusAPI(unittest.TestCase):
|
||||
"POST",
|
||||
"/crewai_plus/api/v1/tracing/evaluations",
|
||||
json={"execution_id": "6f31fe1a-20bd-4bfe-a011-25d6b9341f62"},
|
||||
timeout=120.0, # AMP reads the run's spans inside this request
|
||||
)
|
||||
self.assertEqual(response, mock_response)
|
||||
|
||||
@@ -40,7 +41,7 @@ class TestPlusAPI(unittest.TestCase):
|
||||
response = self.api.get_evaluation("ev-1")
|
||||
|
||||
mock_make_request.assert_called_once_with(
|
||||
"GET", "/crewai_plus/api/v1/tracing/evaluations/ev-1"
|
||||
"GET", "/crewai_plus/api/v1/tracing/evaluations/ev-1", timeout=30.0
|
||||
)
|
||||
self.assertEqual(response, mock_response)
|
||||
|
||||
|
||||
Reference in New Issue
Block a user