andre/crewAI
1
0
Fork 0
mirror of https://github.com/crewAIInc/crewAI.git synced 2026-07-02 21:58:11 +00:00
Code Issues Actions 6 Packages Projects Releases Wiki Activity

JSON first crews (#6131)
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
Details
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Details
Check Documentation Broken Links / Check broken links (push) Has been cancelled
Details
Vulnerability Scan / pip-audit (push) Has been cancelled
Details
Nightly Canary Release / Check for new commits (push) Has been cancelled
Details
Nightly Canary Release / Build nightly packages (push) Has been cancelled
Details
Nightly Canary Release / Publish nightly to PyPI (push) Has been cancelled
Details
Mark stale issues and pull requests / stale (push) Has been cancelled
Details

Browse Source 
* feat(cli): introduce JSON crew project support and TUI enhancements

- Added support for creating and running JSON-defined crew projects, allowing users to scaffold projects with a new `create_json_crew.py` file.
- Implemented a full-screen Textual TUI for crew execution in `crew_run_tui.py`, enhancing user interaction with a two-column layout.
- Updated `run_crew.py` to prioritize JSON crew projects and added daemon mode for running without TUI.
- Introduced interactive pickers in `tui_picker.py` for improved CLI prompts.
- Enhanced validation for JSON crew files in `validate.py` to ensure proper structure and agent definitions.
- Updated `.gitignore` to exclude demo and crewai directories.

* feat: update LLM model references to gpt-5.4-mini

- Changed default LLM model from gpt-4o-mini to gpt-5.4-mini across various files, including CLI options, JSON crew configurations, and agent definitions.
- Enhanced benchmark and human feedback functionalities to utilize the new model.
- Improved user interface elements in the TUI for better interaction and feedback during execution.
- Added support for new skills directory in JSON crew project creation.

* feat(benchmark): add crew-level benchmarking functionality

- Introduced a new `benchmark` command in the CLI for crew-level benchmarking, allowing users to specify agents, models, and timeout settings.
- Implemented `CrewBenchmarkCase` to handle crew-level benchmark cases with inputs and criteria.
- Enhanced the benchmark runner to support progress tracking and detailed reporting of results for multiple models.
- Added tests for loading crew benchmark cases and validating their structure.
- Updated existing benchmark functions to accommodate the new crew-level execution model.

* feat(cli): enhance JSON crew project functionality and TUI improvements

- Added optional agent-level guardrails and advanced options in JSON crew configurations to improve output validation and flexibility.
- Updated the TUI to better handle plan step statuses, including visual indicators for task completion and failure.
- Introduced methods for parsing and managing step observation events, ensuring accurate updates to task statuses during execution.
- Enhanced validation for JSON crew projects, ensuring proper structure and error handling for agent and task definitions.
- Added comprehensive tests for new features and validation logic, ensuring robustness in JSON crew project handling.

* refactor(cli): streamline JSON crew project handling and improve validation

- Refactored JSON crew project loading and validation logic to enhance clarity and maintainability.
- Introduced utility functions for finding JSON crew files, improving code reuse across modules.
- Removed deprecated benchmark functionality and associated tests to simplify the codebase.
- Updated CLI commands to utilize the new JSON project structure, ensuring compatibility with recent changes.
- Enhanced test coverage for JSON crew project features, ensuring robust validation and error handling.

* feat(cli): enhance activity log navigation and focus management

- Added functionality to focus on the activity log when navigating through log entries.
- Implemented refresh logic for the log panel to ensure updates are displayed correctly during navigation.
- Improved keyboard navigation for log entries, allowing users to expand and scroll through logs seamlessly.
- Added tests to verify the correct behavior of log navigation and focus management in the TUI.

* feat(cli): enhance JSON crew project interaction and input handling

- Introduced a new function to enable prompt line editing for better user experience during input prompts.
- Updated the JSON crew project wizards to show interpolation hints for dynamic values, improving user guidance.
- Enhanced the handling of missing input placeholders by prompting users for required values during crew setup.
- Refactored the crew run logic to ensure proper loading and preparation of JSON-defined crews, including runtime input management.
- Added tests to verify the correct behavior of new input handling features and JSON crew project interactions.

* feat(cli): improve crew project input prompts and event handling

- Enhanced the `_prompt_text` function to allow for configurable spacing before prompts, improving user experience during input collection.
- Updated the wizards for agent and task creation to utilize the new prompt configuration, ensuring a more compact and streamlined interaction.
- Introduced new plan step lifecycle events (`PlanStepStartedEvent`, `PlanStepCompletedEvent`) to better track the execution status of plan steps.
- Refactored the step executor to emit these events during the execution of tasks, improving observability and debugging capabilities.
- Added tests to verify the correct behavior of new prompt handling and event emissions during crew project execution.

* fix: refine json-first crew interactions

* fix: prioritize common json crew tools

* fix: make json crew more tools expandable

* fix: show json crew tools by category

* feat(memory): update default embedder to OpenAI text-embedding-3-large and enhance memory compatibility

- Changed the default embedding model for Memory to OpenAI text-embedding-3-large, which uses 3072-dimensional vectors.
- Added warnings regarding compatibility issues with existing local memory stores created with 1536-dimensional embeddings.
- Updated documentation to reflect the new default embedder and its configuration options.
- Enhanced the CLI and codebase to support the new embedding model across various components, ensuring a seamless transition for users.

* fix: address PR review feedback for JSON-first crews

Review blockers:
- Forward trained_agents_file to JSON crews: crewai run -f now exports
  CREWAI_TRAINED_AGENTS_FILE for the in-process JSON crew path
- Wizard agent picker: Esc/cancel now reprompts instead of silently
  assigning the first agent
- JSON tool resolution hard-fails: unknown tool names, missing custom
  tool files, and invalid custom tool modules raise JSONProjectError
  with actionable messages instead of warn-and-continue
- Embedding dimension mismatch: LanceDB and Qdrant Edge storages raise
  EmbeddingDimensionMismatchError with reset/pin guidance instead of
  silently zero-filling vectors or returning empty search results
- Custom tool code execution documented in loader docstring and the
  scaffolded project README

CI fixes:
- ruff format across lib/
- All 133 PR-introduced mypy errors fixed (llm.py lazy-litellm and
  cli.py lazy command shims now use TYPE_CHECKING imports; textual
  is_mounted misuse fixed; pick_many overloads; misc annotations)

Bot review comments:
- Empty except blocks now have explanatory comments or debug logging
- Removed unused _C_BG/_C_PANEL/_C_BORDER globals and redundant
  import re; tests use a single import style for create_json_crew

Tests: trained-agents propagation, wizard cancel, tool resolution
failures, and dimension mismatch guidance.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: address second round of PR review comments

Cursor Bugbot:
- Wizard agent slugs: strip to [a-z0-9_] and fall back to agent_<n> so
  symbol-only roles can't produce an empty agents/.jsonc filename
- Wizard task names: dedupe against prior task names and fall back to
  task_<n> for symbol-only descriptions

CodeRabbit:
- Agent.message(): import Task explicitly at runtime instead of relying
  on the namespace injection done by crewai/__init__
- Async executor: move the native-tools-unsupported fallback from
  _ainvoke_loop_react (self-recursion) to _ainvoke_loop_native_tools,
  mirroring the sync implementation
- StepExecutor downgrade: keep the in-step conversation and append the
  text-tooling instructions instead of rebuilding messages, so completed
  native tool calls are not re-executed
- crewai-files: extension-based MIME lookup now runs before byte
  sniffing so csv/xml types are not degraded to text/plain
- Memory storages: validate every record in a save() batch against a
  consistent embedding dimension (LanceDB previously checked only the
  first record); added mixed-batch tests
- _print_post_tui_summary now typed against CrewRunApp
- Docs: Azure OpenAI default embedder change called out in the memory
  migration warning and provider table

Code quality bots:
- Removed unused _C_YELLOW/_C_CYAN (crew_run_tui) and _GREEN (tui_picker)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* feat(cli): accordion tool picker in JSON crew wizard

The flat tool list had grown to ~90 rows. The picker now shows:
- Common tools always visible at the top
- Every other category as a single expandable row with tool and
  selection counts (e.g. "Search & Research  (27 tools, 2 selected)")
- Expanding a category collapses the previously expanded one
- Selections persist across expand/collapse via new preselected
  support in pick_many; cursor follows the toggled category row

tui_picker gains preselected + initial_cursor options on pick_many,
and Esc in multi-select now confirms the current selection instead of
discarding it (required so collapsing can't silently drop choices).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* refactor(cli): remove --daemon flag from crewai run

The flag only affected JSON crew projects — classic and flow projects
ignored it entirely, which made the behavior inconsistent. Removed the
option, the daemon code path (_run_json_crew_daemon), and its helper
(_load_json_crew_with_inputs).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* test: update run command tests after --daemon removal

lib/crewai/tests/cli/test_run_crew.py still asserted the old
run_crew(trained_agents_file=..., daemon=False) call signature.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(cli): exit codes, mid-run quit, async statuses, hyphen placeholders

Addresses the latest Bugbot review round:

- Failed JSON crew runs now exit non-zero (SystemExit(1)) so scripts
  and CI don't treat failures as success, mirroring the classic path
- Quitting the TUI mid-run now ends the process (os._exit(130));
  kickoff runs in a thread worker that cannot be force-cancelled, so
  letting the CLI return would leave LLM/tool work burning tokens in
  the background
- Sidebar task statuses are now async-safe: completion/failure events
  resolve the task's own row via identity instead of assuming the most
  recently started task, and starting a task no longer blanket-marks
  earlier active rows as done
- The runtime-input prompt regex now accepts hyphenated placeholder
  names ({my-topic}), matching kickoff's interpolation pattern

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix: validation safety, custom tool sandboxing, TUI log integrity, memory error surfacing

- Deploy validation no longer executes project code: validation mode
  checks tool declarations structurally (well-formed entries, custom
  tool file exists) without importing or instantiating anything.
  custom:<name> resolution only happens on the actual run path.
- custom:<name> is constrained to [A-Za-z_][A-Za-z0-9_]* and the
  resolved path must stay inside the project's tools/ directory, so
  custom:../foo or absolute-path names cannot execute code outside it.
  Tool paths resolve relative to the crew project root, not cwd.
- TUI task logs are built from per-task state captured at task start
  (idx, description, agent, start time); an out-of-order completion
  takes its output from the event and no longer steals or resets the
  current task's streamed steps/output.
- EmbeddingDimensionMismatchError now inherits ValueError instead of
  RuntimeError so background saves surface it through
  MemorySaveFailedEvent instead of silently dropping the save; the
  shutdown catch in _background_encode_batch is narrowed to the
  "cannot schedule new futures" case.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(cli): declared project type wins over crew.json presence

A flow project that also contains a crew.json(c) file now runs and
validates as the flow it declares in pyproject.toml instead of being
hijacked by the JSON crew path. Both crewai run (_has_json_crew) and
deploy validation (_is_json_crew) check tool.crewai.type; a missing or
unreadable pyproject still means a bare JSON crew project.

Also documents why StepObservationFailedEvent intentionally marks the
plan step "done": the event signals an observer failure, not a step
failure, and the executor continues past it.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

* fix(cli): type the declared_type locals so mypy stays clean

Comparing an Any-typed .get() chain returns Any, which tripped
no-any-return on the previous commit.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>

---------

Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
João Moura
2026-06-14 04:19:48 -03:00
committed by GitHub
parent d80719df81
commit bb477f8a91
81 changed files with 9088 additions and 235 deletions
Download Patch File Download Diff File Expand all files Collapse all files

35
lib/cli/tests/deploy/test_deploy_main.py
View File

@@ -150,6 +150,7 @@ class TestDeployCommand(unittest.TestCase):
@patch("crewai_cli.deploy.main.fetch_and_json_env_file")
@patch("crewai_cli.deploy.main.git.Repository.origin_url")
@patch("builtins.input")
@pytest.mark.timeout(180)
def test_create_crew(self, mock_input, mock_git_origin_url, mock_fetch_env):
mock_fetch_env.return_value = {"ENV_VAR": "value"}
mock_git_origin_url.return_value = "https://github.com/test/repo.git"
@@ -165,6 +166,40 @@ class TestDeployCommand(unittest.TestCase):
self.assertIn("Deployment created successfully!", fake_out.getvalue())
self.assertIn("new-uuid", fake_out.getvalue())
@patch("crewai_cli.deploy.main.fetch_and_json_env_file")
@patch("crewai_cli.deploy.main.git.Repository")
def test_create_crew_without_git_repo_shows_setup_help(
self, mock_repository, mock_fetch_env
):
mock_fetch_env.return_value = {"ENV_VAR": "value"}
mock_repository.side_effect = ValueError("not a Git repository")
with patch("sys.stdout", new=StringIO()) as fake_out:
self.deploy_command.create_crew(skip_validate=True)
output = fake_out.getvalue()
self.assertIn("Deployment requires a Git repository", output)
self.assertIn("git init", output)
self.assertIn("git remote add origin <your-repo-url>", output)
self.mock_client.create_crew.assert_not_called()
@patch("crewai_cli.deploy.main.fetch_and_json_env_file")
@patch("crewai_cli.deploy.main.git.Repository")
def test_create_crew_without_remote_shows_remote_help(
self, mock_repository, mock_fetch_env
):
mock_fetch_env.return_value = {"ENV_VAR": "value"}
mock_repository.return_value.origin_url.return_value = None
with patch("sys.stdout", new=StringIO()) as fake_out:
self.deploy_command.create_crew(skip_validate=True)
output = fake_out.getvalue()
self.assertIn("No remote repository URL found.", output)
self.assertIn("git remote add origin <your-repo-url>", output)
self.assertIn("git push -u origin HEAD", output)
self.mock_client.create_crew.assert_not_called()
def test_list_crews(self):
mock_response = MagicMock()
mock_response.status_code = 200

100
lib/cli/tests/deploy/test_validate.py
View File

@@ -110,6 +110,45 @@ def _run_without_import_check(root: Path) -> DeployValidator:
return v
def _scaffold_json_crew(root: Path, *, task_agent: str = "researcher") -> None:
(root / "pyproject.toml").write_text(_make_pyproject(name="json_crew"))
(root / "uv.lock").write_text("# dummy uv lockfile\n")
agents_dir = root / "agents"
agents_dir.mkdir()
(agents_dir / "researcher.jsonc").write_text(
dedent(
"""
{
"role": "Researcher",
"goal": "Research things",
"backstory": "Experienced researcher",
"llm": "openai/gpt-4o-mini"
}
"""
).strip()
+ "\n"
)
(root / "crew.jsonc").write_text(
dedent(
f"""
{{
"name": "json_crew",
"agents": ["researcher"],
"tasks": [
{{
"name": "research",
"description": "Research https://example.com/a//b",
"expected_output": "Findings",
"agent": "{task_agent}"
}}
]
}}
"""
).strip()
+ "\n"
)
@pytest.mark.parametrize(
"project_name, expected",
[
@@ -129,6 +168,38 @@ def test_valid_standard_crew_project_passes(tmp_path: Path) -> None:
assert v.ok, f"expected clean run, got {v.results}"
def test_valid_json_crew_project_passes(tmp_path: Path) -> None:
_scaffold_json_crew(tmp_path)
v = DeployValidator(project_root=tmp_path)
v.run()
assert "invalid_crew_json" not in _codes(v)
def test_json_task_agent_mismatch_is_error(tmp_path: Path) -> None:
_scaffold_json_crew(tmp_path, task_agent="missing_agent")
v = DeployValidator(project_root=tmp_path)
v.run()
finding = next(r for r in v.results if r.code == "invalid_crew_json")
assert finding.severity is Severity.ERROR
assert "missing_agent" in finding.detail
def test_json_runtime_fields_are_deploy_errors(tmp_path: Path) -> None:
_scaffold_json_crew(tmp_path)
crew_path = tmp_path / "crew.jsonc"
crew_path.write_text(
crew_path.read_text().replace(
'"name": "json_crew",',
'"name": "json_crew",\n "id": "00000000-0000-4000-8000-000000000000",',
)
)
v = DeployValidator(project_root=tmp_path)
v.run()
finding = next(r for r in v.results if r.code == "invalid_crew_json")
assert finding.severity is Severity.ERROR
assert "runtime-only" in finding.detail
def test_missing_pyproject_errors(tmp_path: Path) -> None:
v = _run_without_import_check(tmp_path)
assert "missing_pyproject" in _codes(v)
@@ -426,4 +497,31 @@ def test_create_crew_aborts_on_validation_error(tmp_path: Path) -> None:
cmd = DeployCommand()
cmd.create_crew()
assert not cmd.plus_api_client.create_crew.called
del mock_api # silence unused-var lint
del mock_api # silence unused-var lint
def test_is_json_crew_defers_to_declared_flow_type(tmp_path):
"""A flow project with a stray crew.jsonc must validate as a flow."""
(tmp_path / "crew.jsonc").write_text("{}")
(tmp_path / "pyproject.toml").write_text(
'[project]\nname = "demo"\nversion = "0.1.0"\n\n'
'[tool.crewai]\ntype = "flow"\n'
)
assert DeployValidator(project_root=tmp_path)._is_json_crew is False
def test_is_json_crew_true_for_declared_crew_type(tmp_path):
(tmp_path / "crew.jsonc").write_text("{}")
(tmp_path / "pyproject.toml").write_text(
'[project]\nname = "demo"\nversion = "0.1.0"\n\n'
'[tool.crewai]\ntype = "crew"\n'
)
assert DeployValidator(project_root=tmp_path)._is_json_crew is True
def test_is_json_crew_true_without_pyproject(tmp_path):
(tmp_path / "crew.jsonc").write_text("{}")
assert DeployValidator(project_root=tmp_path)._is_json_crew is True

4
lib/cli/tests/test_cli.py
View File

@@ -94,9 +94,9 @@ def test_version_command_with_tools(runner):
def test_test_default_iterations(evaluate_crew, runner):
result = runner.invoke(test)
evaluate_crew.assert_called_once_with(3, "gpt-4o-mini", trained_agents_file=None)
evaluate_crew.assert_called_once_with(3, "gpt-5.4-mini", trained_agents_file=None)
assert result.exit_code == 0
assert "Testing the crew for 3 iterations with model gpt-4o-mini" in result.output
assert "Testing the crew for 3 iterations with model gpt-5.4-mini" in result.output
@mock.patch("crewai_cli.cli.evaluate_crew")

440
lib/cli/tests/test_create_crew.py
View File

@@ -6,6 +6,8 @@ from unittest import mock
import pytest
from click.testing import CliRunner
import crewai_cli.create_json_crew as json_crew
import crewai_cli.tui_picker as tui_picker
from crewai_cli.create_crew import create_crew, create_folder_structure
@@ -345,3 +347,441 @@ def test_env_vars_are_uppercased_in_env_file(
env_file_path = crew_path / ".env"
content = env_file_path.read_text()
assert "MODEL=" in content
def test_json_wizard_defaults_to_sequential_and_memory_enabled(monkeypatch):
monkeypatch.setattr(
json_crew,
"_wizard_agent",
lambda **_: {
"name": "researcher",
"role": "Researcher",
"goal": "Research",
"backstory": "Researcher",
"llm": "openai/gpt-5.5",
"tools": [],
"planning": False,
"allow_delegation": False,
},
)
monkeypatch.setattr(
json_crew,
"_wizard_task",
lambda **_: {
"name": "research_task",
"description": "Research",
"expected_output": "Findings",
"agent": "researcher",
"context": [],
},
)
def confirm(label: str, default: bool = False) -> bool:
if label == "Enable crew memory?":
return default
return False
monkeypatch.setattr(json_crew, "_confirm", confirm)
monkeypatch.setattr(json_crew.click, "prompt", lambda *_, **__: "")
monkeypatch.setattr(
json_crew,
"pick_one",
lambda *_args, **_kwargs: pytest.fail("process should not be prompted"),
)
_agents, _tasks, settings = json_crew._wizard_agents_and_tasks(
skip_provider=True,
default_llm="openai/gpt-5.5",
)
assert settings == {"process": "sequential", "memory": True, "inputs": {}}
def test_json_wizard_shows_interpolation_hint(capsys):
json_crew._show_interpolation_hint("tasks")
output = capsys.readouterr().out
assert "{placeholder}" in output
assert "dynamic values" in output
assert "{topic}" not in output
assert "Description >" not in output
assert '"description"' not in output
def test_json_wizard_text_prompt_uses_full_prompt_for_readline(monkeypatch):
prompts: list[str] = []
monkeypatch.setattr(
json_crew, "_readline_safe_prompt", lambda prompt: f"safe:{prompt}"
)
monkeypatch.setattr(
"builtins.input", lambda prompt: prompts.append(prompt) or "Draft content"
)
assert json_crew._prompt_text("Goal", spacing_before=False) == "Draft content"
assert len(prompts) == 1
assert prompts[0].startswith("safe:")
assert "Goal" in prompts[0]
assert " > " in prompts[0]
def test_json_wizard_tool_picker_prioritizes_common_tools(monkeypatch):
picker_calls: list[tuple[str, list[str], dict[str, object]]] = []
def pick_many(title: str, labels: list[str], **kwargs):
picker_calls.append((title, labels, kwargs))
return [1, 3], None
monkeypatch.setattr(json_crew, "pick_many", pick_many)
tools = json_crew._select_tools()
assert tools == ["SerperDevTool", "DirectoryReadTool"]
assert len(picker_calls) == 1
labels = picker_calls[0][1]
assert 0 in picker_calls[0][2]["separator_indices"]
assert labels[0] == "── Common tools ──"
assert labels[1].strip().endswith("SerperDevTool")
assert labels[2].strip().endswith("ScrapeWebsiteTool")
assert labels[3].strip().endswith("DirectoryReadTool")
assert labels[4].strip().endswith("FileReadTool")
assert labels[5].strip().endswith("FileWriterTool")
assert labels[1].index("Google search") < labels[1].index("SerperDevTool")
assert "More tools" not in labels
def test_json_wizard_tool_picker_collapses_categories_by_default(monkeypatch):
picker_calls: list[tuple[str, list[str], dict[str, object]]] = []
def pick_many(title: str, labels: list[str], **kwargs):
picker_calls.append((title, labels, kwargs))
return [], None
monkeypatch.setattr(json_crew, "pick_many", pick_many)
json_crew._select_tools()
labels = picker_calls[0][1]
action_indices = picker_calls[0][2]["action_indices"]
# Categories show as collapsed action rows, not separators with tools
assert any(label.startswith("▸ Search & Research") for label in labels)
assert any(label.startswith("▸ Web Scraping") for label in labels)
assert not any(label.strip().endswith("BraveSearchTool") for label in labels)
assert len(action_indices) >= 4
# Only the common tools section is visible beyond the category rows
assert len(labels) == 1 + 5 + len(action_indices)
def test_json_wizard_tool_picker_expands_one_category_at_a_time(monkeypatch):
picker_calls: list[tuple[str, list[str], dict[str, object]]] = []
def find_category_row(labels: list[str], category: str) -> int:
return next(
idx for idx, label in enumerate(labels) if category in label
)
def pick_many(title: str, labels: list[str], **kwargs):
picker_calls.append((title, labels, kwargs))
call_num = len(picker_calls)
if call_num == 1:
return [], find_category_row(labels, "Search & Research")
if call_num == 2:
# Search & Research is expanded; select BraveSearchTool and
# expand Web Scraping instead
brave = next(
idx
for idx, label in enumerate(labels)
if label.strip().endswith("BraveSearchTool")
)
return [brave], find_category_row(labels, "Web Scraping")
return [], None
monkeypatch.setattr(json_crew, "pick_many", pick_many)
tools = json_crew._select_tools()
assert tools == ["BraveSearchTool"]
assert len(picker_calls) == 3
# Second render: Search & Research expanded, others collapsed
labels2 = picker_calls[1][1]
assert any(label.startswith("▾ Search & Research") for label in labels2)
assert any(label.strip().endswith("BraveSearchTool") for label in labels2)
assert any(label.startswith("▸ Web Scraping") for label in labels2)
# Third render: Web Scraping expanded, Search & Research collapsed again
labels3 = picker_calls[2][1]
assert any(label.startswith("▸ Search & Research") for label in labels3)
assert any(label.startswith("▾ Web Scraping") for label in labels3)
assert not any(label.strip().endswith("BraveSearchTool") for label in labels3)
# The collapsed Search & Research row reports its selection count
assert any(
"Search & Research" in label and "1 selected" in label for label in labels3
)
# Cursor returns to the toggled category row
assert picker_calls[2][2]["initial_cursor"] == next(
idx for idx, label in enumerate(labels3) if "Web Scraping" in label
)
def test_json_wizard_tool_picker_preserves_selection_across_renders(monkeypatch):
picker_calls: list[tuple[str, list[str], dict[str, object]]] = []
def pick_many(title: str, labels: list[str], **kwargs):
picker_calls.append((title, labels, kwargs))
call_num = len(picker_calls)
if call_num == 1:
# Select a common tool, then expand a category
category_row = next(
idx for idx, label in enumerate(labels) if "Web Scraping" in label
)
return [1], category_row
# Confirm without touching anything else
return sorted(kwargs["preselected"]), None
monkeypatch.setattr(json_crew, "pick_many", pick_many)
tools = json_crew._select_tools()
# The common-tool selection survived the expand re-render via preselected
assert tools == ["SerperDevTool"]
assert 1 in picker_calls[1][2]["preselected"]
def test_json_wizard_tool_picker_lists_builtin_tools_across_categories(monkeypatch):
picker_calls: list[tuple[str, list[str], dict[str, object]]] = []
expanded_labels: list[str] = []
def pick_many(title: str, labels: list[str], **kwargs):
picker_calls.append((title, labels, kwargs))
expanded_labels.extend(labels)
action_indices = sorted(kwargs["action_indices"])
call_num = len(picker_calls)
if call_num <= len(action_indices):
# Expand the n-th category (indices shift between renders, so
# recompute from this render's action rows)
return [], action_indices[call_num - 1]
return [], None
monkeypatch.setattr(json_crew, "pick_many", pick_many)
json_crew._select_tools()
tool_names = {
label.rsplit(maxsplit=1)[-1]
for label in expanded_labels
if not label.startswith(("▸", "▾", "──"))
}
assert {
"DirectorySearchTool",
"MDXSearchTool",
"XMLSearchTool",
"YoutubeVideoSearchTool",
"S3ReaderTool",
"E2BExecTool",
"TavilyResearchTool",
"SerplyNewsSearchTool",
"BrowserbaseLoadTool",
"PatronusEvalTool",
}.issubset(tool_names)
assert {
"MCPServerAdapter",
"MongoDBVectorSearchConfig",
"ScrapegraphScrapeToolSchema",
"SnowflakeConfig",
}.isdisjoint(tool_names)
def test_multi_picker_skips_separator_on_initial_cursor(monkeypatch):
cursors: list[int] = []
monkeypatch.setattr(tui_picker, "_read_key", lambda: "enter")
monkeypatch.setattr(
tui_picker,
"_draw_multi",
lambda _labels, cursor, *_args, **_kwargs: cursors.append(cursor),
)
monkeypatch.setattr(tui_picker, "_clear_lines", lambda *_args, **_kwargs: None)
assert tui_picker._arrow_select_multi(
["── Common tools ──", "Google search via Serper API SerperDevTool"],
separator_indices={0},
) == ([], None)
assert cursors == [1]
def test_json_wizard_agent_attribute_prompts_are_compact(monkeypatch):
prompt_calls: list[tuple[str, bool]] = []
prompt_values = {
"Role": "Senior Dev Rel",
"Goal": "Draft content",
"Backstory": "Knows developer communities",
}
def prompt_text(
label: str,
default: str = "",
*,
spacing_before: bool = True,
) -> str:
prompt_calls.append((label, spacing_before))
return prompt_values[label]
monkeypatch.setattr(json_crew, "_prompt_text", prompt_text)
monkeypatch.setattr(json_crew, "_select_model", lambda: "openai/gpt-5.5")
monkeypatch.setattr(json_crew, "pick_many", lambda *_args, **_kwargs: ([], None))
monkeypatch.setattr(json_crew, "_confirm", lambda *_args, **_kwargs: False)
agent = json_crew._wizard_agent(agent_num=1, existing_names=[])
assert agent is not None
assert prompt_calls == [
("Role", False),
("Goal", False),
("Backstory", False),
]
def test_json_wizard_task_attribute_prompts_are_compact(monkeypatch):
prompt_calls: list[tuple[str, bool]] = []
prompt_values = {
"Description": "Research latest release",
"Expected output": "Release summary",
}
def prompt_text(
label: str,
default: str = "",
*,
spacing_before: bool = True,
) -> str:
prompt_calls.append((label, spacing_before))
return prompt_values[label]
monkeypatch.setattr(json_crew, "_prompt_text", prompt_text)
task = json_crew._wizard_task(
task_num=1,
agent_names=["senior_dev_rel"],
prior_task_names=[],
)
assert task is not None
assert prompt_calls == [
("Description", False),
("Expected output", False),
]
def test_json_create_provider_preselects_default_model(tmp_path, monkeypatch):
monkeypatch.chdir(tmp_path)
with mock.patch(
"crewai_cli.create_json_crew._wizard_agents_and_tasks"
) as mock_wizard:
mock_wizard.return_value = (
[
{
"name": "researcher",
"role": "Researcher",
"goal": "Research",
"backstory": "Researcher",
"llm": "openai/gpt-5.5",
"tools": [],
"planning": False,
"allow_delegation": False,
}
],
[
{
"name": "research_task",
"description": "Research",
"expected_output": "Findings",
"agent": "researcher",
"context": [],
}
],
{"process": "sequential", "memory": False, "inputs": {}},
)
json_crew.create_json_crew("JSON Crew", provider="openai", skip_provider=True)
mock_wizard.assert_called_once_with(
skip_provider=True,
default_llm="openai/gpt-5.5",
)
assert (tmp_path / "json_crew" / "crew.jsonc").exists()
assert not (tmp_path / "json_crew" / "tests").exists()
assert not (tmp_path / "json_crew" / "config.jsonc").exists()
crew_template = (tmp_path / "json_crew" / "crew.jsonc").read_text()
assert (
'"guardrail": "Every factual claim needs context support."'
in crew_template
)
assert '"guardrails": [' in crew_template
assert '"guardrail_max_retries": 2' in crew_template
assert "Docs: https://docs.crewai.com/concepts/tasks" in crew_template
assert '"output_pydantic": null' in crew_template
assert '"markdown": false' in crew_template
assert "Docs: https://docs.crewai.com/concepts/crews" in crew_template
assert '"manager_agent": "researcher"' in crew_template
assert '"output_log_file": "crew.log"' in crew_template
assert "Crew-level LLM fields also accept object form" in crew_template
assert '"chat_llm": {"model": "llama3", "provider": "ollama"' in (
crew_template
)
assert "Use {placeholder} in agent or task text" in crew_template
assert "`crewai run` prompts for any placeholders" in crew_template
assert "Use {placeholder} inputs here" in crew_template
agent_template = (
tmp_path / "json_crew" / "agents" / "researcher.jsonc"
).read_text()
assert "You can use {placeholder} inputs in role, goal, or backstory" in (
agent_template
)
assert '"role": "Senior {industry} Researcher"' in agent_template
assert "Optional agent-level guardrail" in agent_template
assert '"guardrail_max_retries": 2' in agent_template
assert "Docs: https://docs.crewai.com/concepts/agents" in agent_template
assert '"reasoning": true' in agent_template
assert "For custom endpoints or deployment-based providers" in agent_template
assert '"deployment_name": "my-deployment", "provider": "azure"' in (
agent_template
)
assert '"planning_config": {' in agent_template
assert '"llm": {"model": "deepseek-chat", "provider": "deepseek"}' in (
agent_template
)
assert '"knowledge_sources": []' in agent_template
def test_json_provider_default_model_helper():
assert json_crew._default_model_for_provider("openai") == "openai/gpt-5.5"
assert json_crew._default_model_for_provider("anthropic/claude-custom") == (
"anthropic/claude-custom"
)
assert json_crew._default_model_for_provider("unknown") is None
def test_json_wizard_task_reprompts_on_cancelled_agent_pick(monkeypatch):
"""Esc on the agent picker must reprompt, not silently assign agent 0."""
prompts = iter(["Do the research", "A report"])
monkeypatch.setattr(json_crew, "_prompt_text", lambda *a, **k: next(prompts))
pick_calls: list[str] = []
picks = iter([-1, 1])
def fake_pick_one(title: str, labels: list[str]) -> int:
pick_calls.append(title)
return next(picks)
monkeypatch.setattr(json_crew, "pick_one", fake_pick_one)
task = json_crew._wizard_task(
task_num=1,
agent_names=["first_agent", "second_agent"],
prior_task_names=[],
)
assert len(pick_calls) == 2
assert task["agent"] == "second_agent"

796
lib/cli/tests/test_crew_run_tui.py Normal file
View File

@@ -0,0 +1,796 @@
from datetime import datetime
import time
import pytest
from crewai.events.event_bus import crewai_event_bus
from crewai.events.types.observation_events import (
GoalAchievedEarlyEvent,
PlanRefinementEvent,
PlanReplanTriggeredEvent,
PlanStepCompletedEvent,
PlanStepStartedEvent,
StepObservationCompletedEvent,
StepObservationFailedEvent,
StepObservationStartedEvent,
)
from crewai.events.types.tool_usage_events import (
ToolUsageErrorEvent,
ToolUsageFinishedEvent,
ToolUsageStartedEvent,
)
from crewai_cli import run_crew
from crewai_cli.crew_run_tui import CrewRunApp
def _app_with_plan() -> CrewRunApp:
app = CrewRunApp()
app._plan = {
"plan": "Demo plan",
"steps": [
{"step_number": 1, "description": "First"},
{"step_number": 2, "description": "Second"},
{"step_number": 3, "description": "Third"},
],
}
app._plan_step_status = {1: "pending", 2: "pending", 3: "pending"}
return app
def _log_entry(name: str) -> dict:
now = time.time()
return {
"tool_name": name,
"status": "success",
"args": None,
"result": f"{name} result",
"error": None,
"start_time": now,
"duration": 1.0,
"task_idx": 1,
}
def _emit_event(event: object) -> None:
future = crewai_event_bus.emit(None, event)
if future:
future.result(timeout=5)
def test_chain_deploy_skips_validation_after_auth_retry(monkeypatch) -> None:
create_calls: list[dict[str, object]] = []
login_calls: list[bool] = []
class FakeDeployCommand:
attempts = 0
def create_crew(self, **kwargs) -> None:
create_calls.append(kwargs)
FakeDeployCommand.attempts += 1
if FakeDeployCommand.attempts == 1:
raise SystemExit(1)
class FakeAuthenticationCommand:
def login(self) -> None:
login_calls.append(True)
monkeypatch.setattr("crewai_cli.deploy.main.DeployCommand", FakeDeployCommand)
monkeypatch.setattr(
"crewai_cli.authentication.main.AuthenticationCommand",
FakeAuthenticationCommand,
)
run_crew._chain_deploy()
assert create_calls == [
{"confirm": False, "skip_validate": True},
{"confirm": False, "skip_validate": True},
]
assert login_calls == [True]
def test_plan_step_status_updates_only_the_explicit_step() -> None:
app = _app_with_plan()
app._set_plan_step_status(2, "done")
assert app._plan_step_status == {
1: "pending",
2: "done",
3: "pending",
}
def test_step_observation_events_update_the_explicit_step() -> None:
app = _app_with_plan()
app._subscribe()
try:
future = crewai_event_bus.emit(
None,
StepObservationStartedEvent(
agent_role="Agent",
step_number=2,
step_description="Second",
),
)
if future:
future.result(timeout=5)
assert app._plan_step_status == {
1: "pending",
2: "active",
3: "pending",
}
future = crewai_event_bus.emit(
None,
StepObservationCompletedEvent(
agent_role="Agent",
step_number=2,
step_description="Second",
step_completed_successfully=True,
),
)
if future:
future.result(timeout=5)
finally:
app._unsubscribe()
assert app._plan_step_status == {
1: "pending",
2: "done",
3: "pending",
}
def test_plan_step_lifecycle_events_update_the_explicit_step() -> None:
app = _app_with_plan()
app._subscribe()
try:
_emit_event(
PlanStepStartedEvent(
agent_role="Agent",
step_number=2,
step_description="Second",
)
)
assert app._plan_step_status == {
1: "pending",
2: "active",
3: "pending",
}
_emit_event(
PlanStepCompletedEvent(
agent_role="Agent",
step_number=2,
step_description="Second",
success=True,
result="done",
)
)
finally:
app._unsubscribe()
assert app._plan_step_status == {
1: "pending",
2: "done",
3: "pending",
}
def test_failed_plan_step_lifecycle_event_marks_exact_step_failed() -> None:
app = _app_with_plan()
app._subscribe()
try:
_emit_event(
PlanStepCompletedEvent(
agent_role="Agent",
step_number=2,
step_description="Second",
success=False,
error="Step failed",
)
)
finally:
app._unsubscribe()
assert app._plan_step_status == {
1: "pending",
2: "failed",
3: "pending",
}
def test_tool_usage_events_do_not_advance_plan_steps() -> None:
app = _app_with_plan()
app._subscribe()
try:
future = crewai_event_bus.emit(
None,
ToolUsageStartedEvent(tool_name="search", tool_args={"query": "CrewAI"}),
)
if future:
future.result(timeout=5)
now = datetime.now()
future = crewai_event_bus.emit(
None,
ToolUsageFinishedEvent(
tool_name="search",
tool_args={"query": "CrewAI"},
started_at=now,
finished_at=now,
output="result",
),
)
if future:
future.result(timeout=5)
finally:
app._unsubscribe()
assert app._plan_step_status == {
1: "pending",
2: "pending",
3: "pending",
}
def test_next_tool_does_not_mark_unfinished_tool_successful() -> None:
app = _app_with_plan()
app._subscribe()
try:
_emit_event(
ToolUsageStartedEvent(tool_name="search", tool_args={"query": "CrewAI"}),
)
_emit_event(
ToolUsageStartedEvent(tool_name="scrape", tool_args={"url": "https://x"}),
)
finally:
app._unsubscribe()
assert app._log_entries[0]["status"] == "timeout"
assert app._log_entries[0]["result"] is None
assert app._log_entries[0]["error"] == (
"No result received before the next tool started"
)
assert app._log_entries[1]["status"] == "running"
assert app._plan_step_status == {
1: "pending",
2: "pending",
3: "pending",
}
def test_internal_reasoning_function_call_is_hidden_from_activity_log() -> None:
app = _app_with_plan()
app._subscribe()
try:
future = crewai_event_bus.emit(
None,
ToolUsageStartedEvent(
tool_name="create_reasoning_plan",
tool_args={"plan": "Plan", "steps": [], "ready": True},
),
)
if future:
future.result(timeout=5)
now = datetime.now()
future = crewai_event_bus.emit(
None,
ToolUsageFinishedEvent(
tool_name="create_reasoning_plan",
tool_args={"plan": "Plan", "steps": [], "ready": True},
started_at=now,
finished_at=now,
output='{"plan":"Plan","steps":[],"ready":true}',
),
)
if future:
future.result(timeout=5)
future = crewai_event_bus.emit(
None,
ToolUsageErrorEvent(
tool_name="create_reasoning_plan",
tool_args={"plan": "Plan", "steps": [], "ready": True},
error="internal planning fallback",
),
)
if future:
future.result(timeout=5)
finally:
app._unsubscribe()
assert app._log_entries == []
assert app._current_task_steps == []
def test_tool_failure_does_not_override_successful_plan_step_completion() -> None:
app = _app_with_plan()
app._subscribe()
try:
_emit_event(
PlanStepStartedEvent(
agent_role="Agent",
step_number=1,
step_description="First",
)
)
_emit_event(
ToolUsageStartedEvent(
tool_name="search_the_internet_with_serper",
tool_args={"search_query": "CrewAI release"},
plan_step_number=1,
plan_step_description="First",
)
)
_emit_event(
ToolUsageErrorEvent(
tool_name="search_the_internet_with_serper",
tool_args={"search_query": "CrewAI release"},
plan_step_number=1,
plan_step_description="First",
error="No results",
)
)
_emit_event(
PlanStepCompletedEvent(
agent_role="Agent",
step_number=1,
step_description="First",
success=True,
result="Recovered with another source",
)
)
finally:
app._unsubscribe()
assert app._plan_step_status == {
1: "done",
2: "pending",
3: "pending",
}
def test_tool_event_step_metadata_is_stored_in_activity_log() -> None:
app = _app_with_plan()
app._subscribe()
try:
_emit_event(
ToolUsageStartedEvent(
tool_name="search_the_internet_with_serper",
tool_args={"search_query": "CrewAI release"},
plan_step_number=2,
plan_step_description="Second",
)
)
now = datetime.now()
_emit_event(
ToolUsageFinishedEvent(
tool_name="search_the_internet_with_serper",
tool_args={"search_query": "CrewAI release"},
plan_step_number=2,
plan_step_description="Second",
started_at=now,
finished_at=now,
output="Found official source",
)
)
finally:
app._unsubscribe()
assert app._log_entries[0]["plan_step_number"] == 2
assert app._plan_step_status == {
1: "pending",
2: "pending",
3: "pending",
}
def test_starting_next_tool_does_not_infer_plan_step_progress() -> None:
app = _app_with_plan()
app._subscribe()
try:
_emit_event(
ToolUsageStartedEvent(
tool_name="search_the_internet_with_serper",
tool_args={"search_query": "CrewAI release"},
)
)
_emit_event(
ToolUsageErrorEvent(
tool_name="search_the_internet_with_serper",
tool_args={"search_query": "CrewAI release"},
error="No results",
)
)
_emit_event(
ToolUsageStartedEvent(
tool_name="read_website_content",
tool_args={"url": "https://example.com"},
)
)
finally:
app._unsubscribe()
assert app._log_entries[0]["status"] == "error"
assert app._log_entries[1]["status"] == "running"
assert app._plan_step_status == {
1: "pending",
2: "pending",
3: "pending",
}
@pytest.mark.asyncio
async def test_crew_done_does_not_mark_unfinished_tool_successful() -> None:
app = _app_with_plan()
async with app.run_test(size=(100, 40)) as pilot:
app._plan_step_status = {1: "failed", 2: "done", 3: "pending"}
app._log_entries = [
{
"tool_name": "search",
"status": "running",
"args": '{"query": "CrewAI"}',
"result": None,
"error": None,
"start_time": time.time() - 2,
"duration": None,
"task_idx": 1,
}
]
app._on_crew_done("final output")
await pilot.pause()
assert app._log_entries[0]["status"] == "timeout"
assert app._log_entries[0]["result"] is None
assert app._log_entries[0]["error"] == "No result received before crew completed"
assert app._plan_step_status == {1: "failed", 2: "done", 3: "done"}
def test_streamed_step_observation_updates_named_step_only() -> None:
app = _app_with_plan()
updated = app._try_parse_step_observation(
'{"step_completed_successfully":true,'
'"key_information_learned":"Step 2 succeeded with the official source."}'
)
assert updated is True
assert app._plan_step_status == {
1: "pending",
2: "done",
3: "pending",
}
def test_failed_streamed_step_observation_marks_named_step_failed() -> None:
app = _app_with_plan()
updated = app._try_parse_step_observation(
'{"step_completed_successfully":false,'
'"key_information_learned":"Step 2 failed because the tool failed."}'
)
assert updated is True
assert app._plan_step_status == {
1: "pending",
2: "failed",
3: "pending",
}
def test_streamed_goal_achieved_observation_collapses_remaining_steps_done() -> None:
app = _app_with_plan()
updated = app._try_parse_step_observation(
'{"step_number":2,'
'"step_completed_successfully":true,'
'"key_information_learned":"Goal is already satisfied.",'
'"goal_already_achieved":true}'
)
assert updated is True
assert app._plan_step_status == {
1: "done",
2: "done",
3: "done",
}
def test_task_completion_collapses_pending_plan_steps_but_preserves_failed() -> None:
app = _app_with_plan()
app._plan_step_status = {1: "failed", 2: "done", 3: "pending"}
app._collapse_plan_on_task_done()
assert app._plan_step_status == {1: "failed", 2: "done", 3: "done"}
def test_observation_failure_collapses_to_done_because_executor_continues() -> None:
app = _app_with_plan()
app._plan_step_status = {1: "done", 2: "active", 3: "pending"}
app._subscribe()
try:
future = crewai_event_bus.emit(
None,
StepObservationFailedEvent(
agent_role="Agent",
step_number=2,
step_description="Second",
error="observer timeout",
),
)
if future:
future.result(timeout=5)
finally:
app._unsubscribe()
assert app._plan_step_status == {
1: "done",
2: "done",
3: "pending",
}
def test_goal_achieved_event_collapses_remaining_steps_done() -> None:
app = _app_with_plan()
app._plan_step_status = {1: "done", 2: "active", 3: "pending"}
app._subscribe()
try:
future = crewai_event_bus.emit(
None,
GoalAchievedEarlyEvent(
agent_role="Agent",
step_number=2,
steps_completed=2,
steps_remaining=1,
),
)
if future:
future.result(timeout=5)
finally:
app._unsubscribe()
assert app._plan_step_status == {
1: "done",
2: "done",
3: "done",
}
def test_replan_event_keeps_old_plan_until_next_streamed_plan_replaces_it() -> None:
app = _app_with_plan()
app._subscribe()
try:
future = crewai_event_bus.emit(
None,
PlanReplanTriggeredEvent(
agent_role="Agent",
step_number=2,
replan_reason="Need updated sources",
replan_count=1,
completed_steps_preserved=1,
),
)
if future:
future.result(timeout=5)
finally:
app._unsubscribe()
assert app._plan is not None
assert app._plan_step_status == {1: "pending", 2: "pending", 3: "pending"}
assert app._awaiting_replan is True
app._try_parse_plan(
'{"plan":"Updated plan","steps":['
'{"step_number":1,"description":"Updated first"},'
'{"step_number":2,"description":"Updated second"}]}'
)
assert app._plan == {
"plan": "Updated plan",
"steps": [
{"step_number": 1, "description": "Updated first"},
{"step_number": 2, "description": "Updated second"},
],
}
assert app._plan_step_status == {1: "pending", 2: "pending"}
assert app._awaiting_replan is False
def test_plan_refinement_updates_descriptions_without_new_statuses() -> None:
app = _app_with_plan()
app._plan_step_status = {1: "done", 2: "active", 3: "pending"}
app._subscribe()
try:
future = crewai_event_bus.emit(
None,
PlanRefinementEvent(
agent_role="Agent",
step_number=2,
refined_step_count=1,
refinements=["Step 3: Write the final answer from verified facts"],
),
)
if future:
future.result(timeout=5)
finally:
app._unsubscribe()
assert app._plan_step_status == {
1: "done",
2: "done",
3: "pending",
}
assert app._plan["steps"][2]["description"] == (
"Write the final answer from verified facts"
)
def test_step_observation_json_is_hidden_from_streaming_text() -> None:
app = _app_with_plan()
assert (
app._strip_step_observation_json(
'Visible before {"step_completed_successfully":true,'
'"key_information_learned":"Step 2 succeeded."} visible after'
)
== "Visible before visible after"
)
@pytest.mark.asyncio
async def test_completed_run_keeps_activity_log_keyboard_navigation_active() -> None:
app = CrewRunApp()
async with app.run_test(size=(100, 40)) as pilot:
app._log_entries = [_log_entry("search"), _log_entry("scrape")]
app._on_crew_done("final output")
await pilot.pause()
assert app.focused is app.query_one("#log-panel")
await pilot.press("down", "enter")
await pilot.pause()
assert app._log_cursor == 1
assert app._log_expanded == {1}
await pilot.press("up")
await pilot.pause()
assert app._log_cursor == 0
class _FakeTask:
fingerprint = None
def __init__(self, task_id: str, name: str) -> None:
self.id = task_id
self.name = name
self.description = name
def test_async_task_completion_marks_the_right_sidebar_row() -> None:
"""Overlapping tasks: completing task 1 while task 2 runs must not
mark task 2 done, and starting task 2 must not mark task 1 done."""
from crewai.events.types.task_events import TaskCompletedEvent, TaskStartedEvent
from crewai.tasks.task_output import TaskOutput
app = CrewRunApp(total_tasks=2, task_names=["first", "second"])
app._subscribe()
try:
task1 = _FakeTask("id-1", "first")
task2 = _FakeTask("id-2", "second")
for task in (task1, task2):
future = crewai_event_bus.emit(
None, TaskStartedEvent(context=None, task=task)
)
if future:
future.result(timeout=5)
# Both started: neither prematurely done
assert app._task_statuses == {1: "active", 2: "active"}
future = crewai_event_bus.emit(
None,
TaskCompletedEvent(
output=TaskOutput(description="first", raw="done", agent="a"),
task=task1,
),
)
if future:
future.result(timeout=5)
assert app._task_statuses == {1: "done", 2: "active"}
finally:
app._unsubscribe()
def test_pop_task_state_falls_back_to_current_task() -> None:
app = CrewRunApp(total_tasks=2, task_names=["first", "second"])
app._current_task_idx = 2
app._current_task_desc = "second"
class _Evt:
task = None
task_name = "unknown"
state = app._pop_task_state(_Evt())
assert state["idx"] == 2
assert state["desc"] == "second"
def test_overlapping_task_logs_keep_their_own_state() -> None:
"""Task 1 completing after task 2 started must log its own description,
agent, and output — and must not steal or reset task 2's stream state."""
from crewai.events.types.task_events import TaskCompletedEvent, TaskStartedEvent
from crewai.tasks.task_output import TaskOutput
app = CrewRunApp(total_tasks=2, task_names=["first", "second"])
app._subscribe()
try:
task1 = _FakeTask("id-1", "first")
task2 = _FakeTask("id-2", "second")
for task in (task1, task2):
future = crewai_event_bus.emit(
None, TaskStartedEvent(context=None, task=task)
)
if future:
future.result(timeout=5)
# Task 2 is current and has streamed state in flight
app._task_full_output = "task two streaming output"
app._current_task_steps = [{"type": "llm", "summary": "thinking"}]
future = crewai_event_bus.emit(
None,
TaskCompletedEvent(
output=TaskOutput(
description="first", raw="task one result", agent="a1"
),
task=task1,
),
)
if future:
future.result(timeout=5)
# Task 1's entry carries its own identity and output
entry1 = app._task_logs[-1]
assert entry1["idx"] == 1
assert entry1["desc"] == "first"
assert entry1["output"] == "task one result"
assert entry1["steps"] == []
# Task 2's in-flight stream state was not consumed or reset
assert app._task_full_output == "task two streaming output"
assert app._current_task_steps == [{"type": "llm", "summary": "thinking"}]
future = crewai_event_bus.emit(
None,
TaskCompletedEvent(
output=TaskOutput(
description="second", raw="task two result", agent="a2"
),
task=task2,
),
)
if future:
future.result(timeout=5)
entry2 = app._task_logs[-1]
assert entry2["idx"] == 2
assert entry2["desc"] == "second"
assert entry2["output"] == "task two streaming output"
assert any(step.get("summary") == "thinking" for step in entry2["steps"])
finally:
app._unsubscribe()

144
lib/cli/tests/test_run_crew.py Normal file
View File

@@ -0,0 +1,144 @@
"""Tests for crewai_cli.run_crew JSON crew handling."""
import os
from pathlib import Path
import pytest
from crewai_core.constants import CREWAI_TRAINED_AGENTS_FILE_ENV
import crewai_cli.run_crew as run_crew_module
def test_run_crew_forwards_trained_agents_file_to_json_crews(monkeypatch):
"""crewai run -f must reach JSON crews, not only classic subprocess crews."""
monkeypatch.setattr(run_crew_module, "_has_json_crew", lambda: True)
called: dict = {}
def fake_run_json_crew(trained_agents_file=None):
called["trained_agents_file"] = trained_agents_file
monkeypatch.setattr(run_crew_module, "_run_json_crew", fake_run_json_crew)
run_crew_module.run_crew(trained_agents_file="some.pkl")
assert called == {"trained_agents_file": "some.pkl"}
def test_run_json_crew_exports_trained_agents_env(monkeypatch, tmp_path: Path):
"""JSON crews run in-process, so the pickle path must land in the env var."""
monkeypatch.chdir(tmp_path)
monkeypatch.delenv(CREWAI_TRAINED_AGENTS_FILE_ENV, raising=False)
try:
# No crew.json(c) in tmp_path: the loader fails *after* the env var
# export, which is the part under test.
with pytest.raises(FileNotFoundError):
run_crew_module._run_json_crew(trained_agents_file="some.pkl")
assert os.environ[CREWAI_TRAINED_AGENTS_FILE_ENV] == "some.pkl"
finally:
os.environ.pop(CREWAI_TRAINED_AGENTS_FILE_ENV, None)
def test_run_json_crew_leaves_env_untouched_without_flag(monkeypatch, tmp_path: Path):
monkeypatch.chdir(tmp_path)
monkeypatch.delenv(CREWAI_TRAINED_AGENTS_FILE_ENV, raising=False)
with pytest.raises(FileNotFoundError):
run_crew_module._run_json_crew()
assert CREWAI_TRAINED_AGENTS_FILE_ENV not in os.environ
def test_missing_input_names_accepts_hyphenated_placeholders():
"""The prompt regex must accept the same names kickoff interpolation does."""
from types import SimpleNamespace
crew = SimpleNamespace(
agents=[
SimpleNamespace(
role="Researcher", goal="Cover {my-topic}", backstory=""
)
],
tasks=[
SimpleNamespace(
description="Write about {my-topic} for {target-audience}",
expected_output="Post",
output_file=None,
)
],
)
assert run_crew_module._missing_input_names(crew, {}) == [
"my-topic",
"target-audience",
]
def _patch_tui_run(monkeypatch, status: str):
"""Stub the TUI pieces of _run_json_crew so only exit handling runs."""
class FakeApp:
def __init__(self, **kwargs):
self._status = status
self._crew_result = "result" if status == "completed" else None
self._want_deploy = False
def run(self):
pass
from types import SimpleNamespace
crew = SimpleNamespace(name="Demo", tasks=[], agents=[])
monkeypatch.setattr(
run_crew_module, "find_crew_json_file", lambda: Path("crew.jsonc")
)
monkeypatch.setattr(
run_crew_module,
"_load_json_crew_for_tui",
lambda _path: (FakeApp, crew, {}, [], []),
)
monkeypatch.setattr(
run_crew_module, "_prompt_for_missing_inputs", lambda _crew, inputs: inputs
)
monkeypatch.setattr(run_crew_module, "_print_post_tui_summary", lambda _app: None)
def test_run_json_crew_failed_status_exits_nonzero(monkeypatch, tmp_path: Path):
monkeypatch.chdir(tmp_path)
_patch_tui_run(monkeypatch, status="failed")
with pytest.raises(SystemExit) as exc_info:
run_crew_module._run_json_crew()
assert exc_info.value.code == 1
def test_run_json_crew_completed_status_returns_result(monkeypatch, tmp_path: Path):
monkeypatch.chdir(tmp_path)
_patch_tui_run(monkeypatch, status="completed")
assert run_crew_module._run_json_crew() == "result"
def test_has_json_crew_defers_to_declared_flow_type(monkeypatch, tmp_path: Path):
"""A flow project containing a stray crew.jsonc must still run as a flow."""
monkeypatch.chdir(tmp_path)
(tmp_path / "crew.jsonc").write_text("{}")
(tmp_path / "pyproject.toml").write_text('[tool.crewai]\ntype = "flow"\n')
assert run_crew_module._has_json_crew() is False
def test_has_json_crew_true_for_declared_crew_type(monkeypatch, tmp_path: Path):
monkeypatch.chdir(tmp_path)
(tmp_path / "crew.jsonc").write_text("{}")
(tmp_path / "pyproject.toml").write_text('[tool.crewai]\ntype = "crew"\n')
assert run_crew_module._has_json_crew() is True
def test_has_json_crew_true_without_pyproject(monkeypatch, tmp_path: Path):
monkeypatch.chdir(tmp_path)
(tmp_path / "crew.jsonc").write_text("{}")
assert run_crew_module._has_json_crew() is True

8
lib/cli/tests/tools/test_main.py
View File

@@ -157,14 +157,16 @@ def test_install_api_error(mock_get, capsys, tool_command):
mock_get.assert_called_once_with("error-tool")
@patch("crewai_cli.tools.main.git.Repository.fetch")
@patch("crewai_cli.tools.main.git.Repository.is_synced", return_value=False)
def test_publish_when_not_in_sync(mock_is_synced, mock_fetch, capsys, tool_command):
@patch("crewai_cli.tools.main.git.Repository")
def test_publish_when_not_in_sync(mock_repository, capsys, tool_command):
mock_repository.return_value.is_synced.return_value = False
with raises(SystemExit):
tool_command.publish(is_public=True)
output = capsys.readouterr().out
assert "Local changes need to be resolved before publishing" in output
mock_repository.return_value.is_synced.assert_called_once_with()
@patch("crewai_cli.tools.main.get_project_name", return_value="sample-tool")