32 Commits

Author	SHA1	Message	Date
João Moura	2b56dab813	feat(cli): pull latest LLM models dynamically in the crew wizard (#6462) ... * feat(cli): pull latest LLM models dynamically in the crew wizard The JSON-crew creation wizard hardcoded a short model list per provider, which goes stale as vendors ship new models every few weeks. Add a three-tier resolver that prefers live data and falls back to a curated list. - New `model_catalog.get_provider_models(provider, fallback)`: 1. Vendor API (openai/anthropic/gemini/groq/cerebras/ollama) when the provider key is already in the environment — the only reliably-fresh source (real release dates / display names). 2. Curated hardcoded fallback — hand-verified, used when no key is set. 3. LiteLLM feed — only for providers with no curated list; it lags real releases, so it must never preempt the curated fallback. - Rank by date/version parsed from model ids, humanize labels, 6h cache, short timeouts, silent fallback on any error. - Wire it into `create_json_crew._select_model()` (picker only). - Refresh the curated fallback against each vendor's official model docs (Anthropic Fable 5 / Opus 4.8 / Sonnet 5; OpenAI GPT-5.5(+pro); Gemini 3.5 Flash / 3.1 Pro preview / 3 Flash preview; Groq Llama 4 / GPT-OSS). - Tests for ranking, chat filtering, caching, and the tier order (17 tests). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RBYGqJHC2TMC6fonFziuuh * fix(cli): address model_catalog review findings - Key the Ollama catalog cache by its base URL so a changed OLLAMA_API_BASE / API_BASE no longer serves the previous host's models for up to the TTL. - Negatively cache the curated fallback after a failed/empty fetch (short _NEGATIVE_TTL) so the picker doesn't repeat a timeout-prone vendor/LiteLLM request on every call — most impactful for a down local Ollama server. - Guard _read_catalog_cache / _write_catalog_cache against a non-dict cache root (corrupt JSON array no longer raises AttributeError). - Replace the two empty `except OSError: pass` blocks with contextlib.suppress(OSError) plus an explanatory comment (CodeQL empty-except). - Tests: negative cache, base-keyed Ollama cache, corrupt-cache no-crash (20 total). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RBYGqJHC2TMC6fonFziuuh * fix(cli): guard null litellm_provider and paginate Gemini models - _from_litellm: coerce a present-but-null `litellm_provider` before string ops so it's skipped instead of raising AttributeError (keeps the documented "never raises" contract). - _fetch_gemini: walk models.list pages via nextPageToken (bounded to 10) — the API is paginated and not guaranteed newest-first, so a single page could drop models the ranking should consider. - Tests for both (22 total). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RBYGqJHC2TMC6fonFziuuh * ci: ignore nltk PYSEC-2026-597 in pip-audit (no fix, not reachable) pip-audit newly flags nltk 3.9.4 for PYSEC-2026-597 (CVE-2026-12243), a path traversal via percent-encoded `..%2f` in nltk.data.load()/find(). It affects all nltk versions <=3.9.4 with no patched release, so it can't be resolved by a version bump — same situation as the already-ignored PYSEC-2026-97. nltk is a transitive dependency (unstructured[local-inference, all-docs] in crewai-tools) used for text tokenization; we never pass untrusted resource URLs/paths to nltk.data, so the traversal is not reachable. Add it to the curated --ignore-vuln list with a justification, matching the existing pattern. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RBYGqJHC2TMC6fonFziuuh * fix(cli): address second Cursor review round on model_catalog - Cache ignores new API keys: include API-key presence in the cache key (`<provider>#key|#nokey`), so a key added after a no-key/negative-cached lookup triggers a fresh live fetch instead of serving the stale fallback. - Bad LiteLLM cache crashes picker: `_from_litellm` now requires a dict from `_load_litellm_data` (a non-mapping JSON root is skipped, not `.items()`'d). - Stale LiteLLM refetch loop: memoize the feed load once per process (`_litellm_memo` + `_reset_litellm_memo` test hook) so repeated uncurated- provider lookups don't each re-attempt a timed download when offline. - Tests: new-key bypass, corrupt-litellm-cache no-crash, one-fetch-per-process (25 total). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RBYGqJHC2TMC6fonFziuuh * fix(cli): keep partial Gemini results on later-page fetch error _fetch_gemini paginates; a network/HTTP error on page 2+ previously raised out through _from_vendor, discarding models already parsed from earlier pages and forcing the curated fallback. Catch per-page fetch errors and return the partial set instead (a first-page failure still yields an empty list -> fallback). Test: test_vendor_gemini_keeps_partial_on_later_page_error (26 total). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RBYGqJHC2TMC6fonFziuuh * fix(cli): don't let an invalid fresh LiteLLM cache block download _fetch_litellm_data treated any truthy JSON root in a fresh provider_cache.json as the feed and returned it, so a non-mapping root (e.g. a JSON array) was memoized and the tier never re-downloaded until the file aged out — leaving uncurated providers with an empty picker despite a recoverable cache. Only short-circuit on a usable dict; otherwise fall through to the download. Test renamed to test_invalid_litellm_cache_falls_through_to_download (asserts recovery via refetch). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RBYGqJHC2TMC6fonFziuuh * fix(cli): honor OLLAMA_HOST and treat empty vendor list as authoritative - Ollama host mismatch: _ollama_base now also reads OLLAMA_HOST (the Ollama runtime convention) after OLLAMA_API_BASE/API_BASE, normalizing a scheme-less value (e.g. "127.0.0.1:11434" -> "http://127.0.0.1:11434"), so users who set only OLLAMA_HOST see models from the server the crew will actually use. - Empty vendor list: a successful vendor fetch returning no models is now authoritative instead of collapsing to the curated fallback. A reachable Ollama with nothing installed yields an empty list (the picker prompts for manual entry) rather than offering hardcoded models that aren't installed; a failed fetch still falls back. _from_vendor now returns [] on success-empty and None only when the tier is unavailable. - Tests: ollama empty->manual, ollama down->fallback, OLLAMA_HOST resolution (29 total). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RBYGqJHC2TMC6fonFziuuh * fix(cli): Gemini first-page failure falls back instead of showing empty Interaction from the prior two fixes: _fetch_gemini swallowed a first-page error and returned [], which _from_vendor reported as a successful-empty result and get_provider_models treated as authoritative — skipping the curated Gemini fallback and jumping to manual entry. Now a first-page failure (nothing gathered yet) re-raises so _from_vendor returns None and the curated list is used; a later-page failure still keeps the partial results. Test: test_vendor_gemini_first_page_error_uses_fallback (30 total). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RBYGqJHC2TMC6fonFziuuh * fix(cli): Gemini GOOGLE_API_KEY + Ollama recovery not blocked by cache - Gemini ignores GOOGLE_API_KEY: _PROVIDER_KEY_ENV now maps each provider to a tuple of accepted env vars; Gemini accepts GEMINI_API_KEY or GOOGLE_API_KEY (matching crewai's own Gemini provider). A new _provider_api_key() resolver is used by both _from_vendor and the cache key, so a GOOGLE_API_KEY user gets the live models API instead of the stale curated fallback. - Ollama recovery blocked by cache: skip the negative (fallback) cache for Ollama. It's a local, fast-failing server, so re-probing each call is cheap and lets the picker pick up real installed models as soon as the server comes up, instead of serving suggestions for the negative-cache TTL. - Tests: GOOGLE_API_KEY live fetch, Ollama down->recover (32 total). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RBYGqJHC2TMC6fonFziuuh * style(cli): ruff format model_catalog.py Add the blank line ruff format expects after _provider_api_key; no behavior change. Fixes the lint-run `ruff format --check lib/` step. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RBYGqJHC2TMC6fonFziuuh * fix(cli): don't treat 'search' substring as a non-chat model marker The 'search' entry in _NON_CHAT_MARKERS matched anywhere in a model id, dropping legitimate completion models like gpt-4o-search-preview and anything containing 'research' (e.g. o3-deep-research, since 'search' is a substring). Remove it; the remaining markers (embedding/audio/image/moderation/etc.) still filter genuine non-chat models. Test: test_search_substring_not_treated_as_non_chat (33). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RBYGqJHC2TMC6fonFziuuh * Revert "ci: ignore nltk PYSEC-2026-597 in pip-audit" Do not suppress an unpatched security advisory to make CI green. Remove PYSEC-2026-597 from the pip-audit ignore list; leave the scan failing so it keeps surfacing the nltk path traversal (CVE-2026-12243). This PR should not be merged until nltk ships a fix (or the vulnerable transitive dep is otherwise resolved). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RBYGqJHC2TMC6fonFziuuh * fix(cli): exclude fine-tuned models and checkpoints from the picker With a live OPENAI_API_KEY, /v1/models returns the user's fine-tunes and training checkpoints (ft:..., ...:ckpt-step-N). Their recent `created` timestamps ranked them above the base models and filled every slot, so the picker showed a wall of `ft:gpt-4o-mini-...:crewai::...` with mangled labels and no foundation models at all. Skip fine-tunes/checkpoints in the OpenAI-shaped fetcher so clean base models surface; a user who wants a fine-tune can still enter it via the picker's "Other" option. Test: test_openai_excludes_fine_tunes_and_checkpoints (34 total). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RBYGqJHC2TMC6fonFziuuh * fix(cli): cleaner model labels + filter Ollama non-chat models Anthropic/Gemini already use vendor display names; OpenAI/Groq/Cerebras/Ollama fall to _humanize for anything outside the curated map, which produced mediocre labels ("GPT Oss 120b", "qwen3 32b", "Deepseek r1", "llama3.3:70b"). Improve _humanize: - split on ':' too (Ollama tags: llama3.3:70b -> "Llama3.3 70B") - uppercase size suffixes (70b -> 70B), acronyms OSS/IT, brand casing (DeepSeek, ChatGPT, QwQ) - capitalize the leading letter of fused family+version tokens (qwen3 -> Qwen3) while preserving OpenAI o-series lowercase (o3, o1-mini) Also fix _fetch_ollama: /api/tags lists everything installed, so filter non-chat (embedding) and fine-tune entries the same way the other tiers do. Tests: expanded test_humanize + test_ollama_excludes_embedding_models (35 total). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01RBYGqJHC2TMC6fonFziuuh --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com>	2026-07-06 20:49:52 -03:00
Vinicius Brasil	2b90117e88	Add repository agents to flow definitions (#6437) ... Inline agent and crew actions can now use repository-backed agents without duplicating role, goal, and backstory in each definition. Examples: * `agent.with.from_repository: support_specialist` * `crew.with.agents.researcher.from_repository: researcher` `PlusAPI.get_agent` now uses the shared synchronous request path so project loaders can fetch repository agents without nested event loops.	2026-07-02 14:28:01 -07:00
Vinicius Brasil	ba2dafdeda	Add text helper for flow CEL prompts (#6404) ... CEL string concatenation currently fails when prompt builders read missing or null fields. This commit adds `text(root, "path", "default")` custom CEL helper so prompt text can safely read nested state/output values.	2026-06-30 16:45:39 -07:00
Vinicius Brasil	1556dbea3e	Add generated Flow Definition authoring skill (#6393) ... * Add generated Flow Definition authoring skill Generate a portable skill from the Flow Definition schema so agents can author valid declarative flows with the same reference CrewAI uses to validate them. New declarative flow projects now write this skill. ```python from crewai.flow.flow_definition import FlowDefinition skill = FlowDefinition.skill(skips=(), examples_format="yaml") ``` * `examples_format` accepts `"yaml"` or `"json"`. * Supported skips: `conversational`, `non_linear_flows`, `each`, `hitl`, `persistence`, `config`, `expression_action`, `script_action`, `tool_action` The generated skill includes authoring rules, a routed crew example, and an API reference extracted from the Flow, action, state, agent, crew, and task Pydantic schemas. * Fix declarative flow scaffold without framework import * Fix skipped expression action guidance * Fix markdown links in skill	2026-06-30 09:13:33 -07:00
João Moura	04adff1e0e	Fix deployment page link id resolution (#6365)	2026-06-27 03:19:46 -03:00
Vinicius Brasil	e1ddb32e56	Initialize Git repositories for generated projects (#6364)	2026-06-26 16:29:49 -07:00
Vinicius Brasil	a149a30bc0	Fix JSON crew template rendering (#6359) ... JSON crews were not using existing CLI templates.	2026-06-26 13:48:48 -07:00
João Moura	8eaae40acf	Track TUI button telemetry (#6346) ... * feat(cli): track TUI button telemetry * fix(cli): use feature usage telemetry for TUI buttons --------- Co-authored-by: Lorenze Jay <63378463+lorenzejay@users.noreply.github.com>	2026-06-26 12:28:47 -07:00
Vinicius Brasil	596150188b	Require explicit CrewAI project definitions (#6358) ... * Require explicit CrewAI project definitions JSON crews and declarative flows now resolve from `[tool.crewai]` metadata instead of implicit filename discovery. This makes project type selection deterministic, prevents stray `crew.json(c)` files from changing CLI behavior, and centralizes definition path validation for run, install, deploy validation, plotting, and memory reset paths. `[tool.crewai].definition` must be a project-local file path. Absolute paths, `~`, missing files, directories, and paths escaping the project root are rejected so deploy and runtime commands use the same contract. Breaking changes and migration paths: * JSON crew projects are no longer discovered from `crew.json` or `crew.jsonc` alone. Add explicit metadata: ```toml [tool.crewai] type = "crew" definition = "crew.jsonc" ``` * Declarative flow projects must use a valid project-local definition path: ```toml [tool.crewai] type = "flow" definition = "flows/research.yaml" ``` * `Flow.from_definition(definition)` is removed. Use: ```python Flow.from_declaration(contents=definition) ``` * `FlowDefinition.to_json()` and `FlowDefinition.to_yaml()` are removed. Use `FlowDefinition.to_dict()` and serialize with the caller's JSON or YAML library. * `FlowDefinition.from_dict()` is removed. Use: ```python FlowDefinition.from_declaration(contents=data) ``` * `FlowDefinition.json_schema()` is removed. Use Pydantic's schema API only where schema generation is intentionally needed: ```python FlowDefinition.model_json_schema(by_alias=True) ``` * `crewai_cli.run_crew.find_crew_json_file()` and `_has_json_crew()` are removed. Use `configured_project_json_crew()` or the shared `crewai_core.project.configured_project_definition("crew")` helper. * `crewai reset-memories` now only loads JSON crews declared through `[tool.crewai].definition`, and invalid declared JSON crew definitions fail instead of silently falling back to classic crew discovery. * Address code review comments	2026-06-26 12:07:03 -07:00
João Moura	e10c17fcf6	Open deployment page after CLI deploy (#6343) ... * Open deployment page after CLI deploy * Format deploy browser URL helper * Handle browser launch failures * Prefer nested deployment identifiers	2026-06-26 14:34:07 -03:00
João Moura	f364a7d988	Fix JSON crew version pin (#6342) ... * Fix JSON crew version pin * Use bounded CrewAI dependency range	2026-06-26 05:19:14 -03:00
Lorenze Jay	9b31226494	Track conversational flow turn usage in telemetry (#6324) ... * Track conversational flow turn usage in telemetry * adjusted name to flow:conversation_turn * only mark on turn completed event * ensure tui also emits these events	2026-06-25 11:02:07 -07:00
Rip&Tear	654abcb40d	fix: enforce owner-only permissions on credential files (#6242) ... * fix: enforce owner-only permissions on credential files Credentials stored at rest were left world-readable on multi-user hosts: - TokenManager._get_secure_storage_path() documented its credential dir as mode 0o700 but created it via mkdir() with default perms (0o755), leaving the Fernet secret.key and encrypted tokens.enc in a traversable dir. - Settings.dump() persisted tool_repository_password (plaintext) to settings.json via open("w"), producing a 0o644 file, and created the config dir at 0o755 — despite the sibling token_manager already writing secrets atomically at 0o600. Fixes: - TokenManager: chmod the credential dir to 0o700 after mkdir (robust against umask and pre-existing dirs). - Settings: write settings.json atomically at 0o600 (mkstemp + chmod + os.replace) and chmod the dedicated config dir to 0o700. The /tmp and cwd fallback parents are deliberately not chmod'd; the 0o600 file mode protects the credential there. Adds regression tests asserting 0o600 files and 0o700 dirs, and that shared fallback dirs are not globally tightened. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Potential fix for pull request finding 'Empty except' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> * Potential fix for pull request finding 'Empty except' Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com> * Close temp fd on secure settings write failure * Log secure settings fd close failures --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com> Co-authored-by: Copilot Autofix powered by AI <223894421+github-code-quality[bot]@users.noreply.github.com>	2026-06-26 01:36:48 +08:00
Rip&Tear	fac3e3579b	Fix symlink path traversal in skill archive extraction (#6235) ... * Fix symlink path traversal in skill archive extraction `_safe_extractall` (the Python < 3.12 fallback used by `crewai skills` archive unpacking) validated each member's *name* against the destination but never validated symlink/hardlink *targets*. A malicious skill tarball could plant a symlink escaping the destination (e.g. `link -> /home/user/.ssh`) followed by a regular member written through it (`link/authorized_keys`), escaping `dest` even though every member name resolves inside it — the classic symlink-extraction traversal. The 3.12+ path (`extractall(..., filter="data")`) already blocks this; the fallback now mirrors it by rejecting absolute link targets and any link target that resolves outside the destination directory. Adds regression tests covering absolute and relative escaping symlinks plus benign in-tree symlinks and ordinary archives. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com> * Harden skill cache archive extraction * Reject special skill archive members --------- Co-authored-by: Claude Opus 4.8 <noreply@anthropic.com>	2026-06-24 08:50:41 -07:00
Vinicius Brasil	a046e6a50b	Validate declarative flow definition paths (#6311)	2026-06-23 19:28:35 -07:00
Lorenze Jay	1862ff8f6c	Support conversational flows in the CLI TUI (#6293) ... * Add conversational flow TUI support * properly support tui	2026-06-23 18:04:09 -07:00
Vinicius Brasil	3452e5c187	Add unified declarative flow loading (#6308) ... Add a single declaration loader shared by API and CLI callers. - Add FlowDefinition.from_declaration for FlowDefinition instances, dictionaries, YAML/JSON strings, and file paths - Add Flow.from_declaration to build runnable flows directly from the same inputs - Route declarative flow CLI loading through Flow.from_declaration so path handling and validation stay centralized ``` # Load just the serializable definition when you do not need to run it yet. definition = FlowDefinition.from_declaration(path="flows/research.crewai") definition = FlowDefinition.from_declaration(contents=flow_yaml) definition = FlowDefinition.from_declaration(contents=flow_dict) # Build a runnable flow directly from the same declaration inputs. flow = Flow.from_declaration(path="flows/research.crewai") flow = Flow.from_declaration(contents=flow_yaml) flow = Flow.from_declaration(contents=flow_dict) flow = Flow.from_declaration(contents=definition) # Run it like any other flow. result = flow.kickoff(inputs={"topic": "AI agents"}) # The CLI now goes through the same path-based loader. # crewai run --definition flows/research.crewai ```	2026-06-23 12:02:18 -07:00
Vinicius Brasil	2eb4e3a236	Improve crewai run startup UX (#6297) ... Remove redundant startup logs from `crewai run` and make the legacy flow command warning actionable. - Stop printing `Running the Flow` and `Running the Crew` before project execution. - Stop printing the redundant `Flow started with ID: ...` line while preserving flow lifecycle event emission. - Replace Click's generic `kickoff` deprecation warning with a clearer message that tells users to use `crewai run`.	2026-06-22 22:31:39 -07:00
Vinicius Brasil	221dfdb08e	Consolidate crewai run and crewai flow kickoff (#6296) ... Make `crewai run` the single execution path for crews and flows, with `crewai flow kickoff` kept as a deprecated compatibility alias.	2026-06-22 20:44:08 -07:00
Vinicius Brasil	4b2ce00a09	Add declarative Flow CLI support (#6294) ... * Add declarative Flow CLI support Currently, declarative flows can be loaded by the runtime, but the CLI still treats them as an experimental definition file instead of a first-class Flow project shape. With this PR, `crewai create flow --declarative` scaffolds a YAML-backed Flow project, and `crewai run`, `crewai flow kickoff`, and `crewai flow plot` can run against the configured definition. This also lets crew actions reference reusable crew definition files or folders and override their inputs from the Flow definition, so declarative flows can compose existing declarative crews without inlining everything. * Address code review comments	2026-06-22 19:58:17 -07:00
João Moura	4cbfbdb232	Keep JSON crew projects and deploy archives Python-free (#6228) ... * fix: scaffold deployable json crews * fix: keep json crew scaffolds python-free * fix: keep json deploy archives python-free * fix: tighten json crew deploy validation * fix: address json crew pr checks * fix: clear langsmith audit advisory	2026-06-22 13:22:46 -03:00
João Moura	504c5c9b04	JSON crew fixes (#6217) ... * feat: update pyproject.toml to specify wheel targets Added a new section to the pyproject.toml file to include only specific files in the wheel build, enhancing the packaging process. Updated tests to verify the inclusion of these targets. * feat: add memory save event handling to activity log Implemented event handlers for MemorySaveStartedEvent, MemorySaveCompletedEvent, and MemorySaveFailedEvent in the crew_run_tui module. This allows the application to log memory save operations, capturing their status and details in the activity log. Added corresponding tests to verify the correct logging behavior for successful and failed memory saves. * feat: enhance memory save event handling in activity log Added functionality to suppress nested memory save events and updated the handling of MemorySaveStartedEvent, MemorySaveCompletedEvent, and MemorySaveFailedEvent to improve logging accuracy. Introduced new tests to verify the correct behavior of memory save events, including scenarios for nested events and completion updates for timed-out entries. * Fix memory save activity log handling * Normalize alpha package versions * Update scaffolded crew dependency * feat: add button to copy setup instructions for CrewAI coding agents Introduced a button in the documentation that allows users to easily copy setup instructions for CrewAI coding agents. The instructions include installation steps, environment setup, and best practices for using the CrewAI CLI. This enhancement aims to streamline the onboarding process for new users. * Improve missing CrewAI install guidance * fix: address pr review feedback * fix: avoid mismatched memory save rows * fix: wait for queued memory save events * fix: avoid matching memory saves on missing ids * chore: normalize prerelease version to 1.14.8a1	2026-06-18 14:14:54 -03:00
João Moura	ebbc0998ef	Implement DMN mode support in crew creation and execution (#6194) ... * Implement DMN mode support in crew creation and execution - Added `is_dmn_mode_enabled` utility to check for enterprise non-interactive mode based on the `CREWAI_DMN` environment variable. - Updated `create` function in `cli.py` to enforce required parameters when DMN mode is active, raising appropriate usage errors. - Enhanced `create_crew` and `create_json_crew` functions to skip provider prompts and handle folder existence checks in DMN mode. - Introduced non-interactive defaults for agent and task creation in DMN mode, ensuring seamless project setup without user input. - Modified `run_crew` to bypass TUI and handle runtime inputs directly when in DMN mode, improving execution flow for JSON-defined crews. - Added tests to validate DMN mode behavior, ensuring correct handling of required inputs and non-interactive defaults. * Implement DMN mode support in crew creation and execution - Introduced `is_dmn_mode_enabled()` utility to check for non-interactive mode based on the `CREWAI_DMN` environment variable. - Updated `create` function to enforce required parameters when DMN mode is active, raising appropriate usage errors. - Modified `create_crew` and `create_json_crew` functions to skip provider prompts and utilize non-interactive defaults in DMN mode. - Enhanced `run_crew` to bypass TUI and handle runtime inputs directly in DMN mode, ensuring smooth execution without user interaction. - Added tests to validate DMN mode behavior, including requirements for type and name, and ensuring proper handling of existing folders and missing inputs.	2026-06-16 19:48:31 -03:00
João Moura	e9d568dc69	Deep Crew / Agent / Task attributes support on json (#6172) ... * Enhance JSON crew project handling and validation - Updated `create_json_crew.py` to specify input files with a brief path. - Refactored `crew_loader.py` to improve agent and task loading logic, including the introduction of a `build_agent` function and better handling of task classes. - Enhanced `json_loader.py` with additional validation for agent and task definitions, including support for Python references and conditional tasks. - Added tests in `test_crew_loader.py` and `test_json_loader.py` to ensure proper loading of agents, tasks, and validation of project structures, including custom types and conditional tasks. - Improved error handling and validation safety across the project loading process. * Enhance JSON crew configuration options in create_json_crew.py - Added optional fields for custom agent subclasses and advanced task options, including condition checks and output specifications. - Improved documentation comments for better clarity on agent and task configurations. - Updated JSON crew handling to support additional callbacks for pre- and post-execution processes. * Enhance JSON crew template tests in test_create_crew.py - Added assertions for new optional fields in crew and agent templates, including conditional tasks, custom converters, and input file specifications. - Improved validation checks for manager agents and callback references to ensure proper configuration in JSON crew definitions. - Expanded documentation references within the tests to provide clearer guidance on the expected structure and usage of crew templates. * Fix JSON crew PR review issues	2026-06-16 02:00:19 -03:00
João Moura	53c2284484	Support ZIP deployment fallback and JSON crew project env runs (#6166) ... * Update crewAI CLI with various enhancements and fixes - Updated `create_json_crew.py` to require `crewai[tools]>=1.14.7`. - Enhanced `git.py` with improved repository initialization, including automatic initial commit creation and exclusion patterns for initial commits. - Modified `install_crew.py` to allow error handling during installation with an optional `raise_on_error` parameter. - Expanded `plus_api.py` to include methods for creating and updating crews from ZIP files. - Introduced a new `archive.py` for creating deployable ZIP archives of CrewAI projects, ensuring local artifacts are excluded. - Updated `run_crew.py` to manage JSON crew dependencies and run crews in the project's environment. - Enhanced deployment logic in `main.py` to handle ZIP uploads and improve user feedback during deployment processes. - Added tests for new functionalities and ensured existing tests reflect recent changes in behavior and requirements. * fix(cli): address deploy zip review feedback * fix(cli): sync missing lockfile before deploy * fix(cli): preserve remote deploy on git setup warnings * test(cli): use single deploy main import style * fix(cli): skip project install for json crew sync * fix(cli): load json runner from source checkout * fix(cli): skip json crew sync when locked * fix(cli): address deploy zip review feedback * fix(cli): pass env on zip redeploy * fix(cli): harden json run and zip fallback * fix(cli): validate before deploy lock install * fix(cli): respect poetry lock for json runs * fix(cli): align json zip wrapper detection * fix(deps): bump starlette audit floor * fix(cli): avoid auth retry for deploy exits * fix(cli): update json zip script entrypoints	2026-06-15 18:46:54 -03:00
João Moura	bb477f8a91	JSON first crews (#6131) ... * feat(cli): introduce JSON crew project support and TUI enhancements - Added support for creating and running JSON-defined crew projects, allowing users to scaffold projects with a new `create_json_crew.py` file. - Implemented a full-screen Textual TUI for crew execution in `crew_run_tui.py`, enhancing user interaction with a two-column layout. - Updated `run_crew.py` to prioritize JSON crew projects and added daemon mode for running without TUI. - Introduced interactive pickers in `tui_picker.py` for improved CLI prompts. - Enhanced validation for JSON crew files in `validate.py` to ensure proper structure and agent definitions. - Updated `.gitignore` to exclude demo and crewai directories. * feat: update LLM model references to gpt-5.4-mini - Changed default LLM model from gpt-4o-mini to gpt-5.4-mini across various files, including CLI options, JSON crew configurations, and agent definitions. - Enhanced benchmark and human feedback functionalities to utilize the new model. - Improved user interface elements in the TUI for better interaction and feedback during execution. - Added support for new skills directory in JSON crew project creation. * feat(benchmark): add crew-level benchmarking functionality - Introduced a new `benchmark` command in the CLI for crew-level benchmarking, allowing users to specify agents, models, and timeout settings. - Implemented `CrewBenchmarkCase` to handle crew-level benchmark cases with inputs and criteria. - Enhanced the benchmark runner to support progress tracking and detailed reporting of results for multiple models. - Added tests for loading crew benchmark cases and validating their structure. - Updated existing benchmark functions to accommodate the new crew-level execution model. * feat(cli): enhance JSON crew project functionality and TUI improvements - Added optional agent-level guardrails and advanced options in JSON crew configurations to improve output validation and flexibility. - Updated the TUI to better handle plan step statuses, including visual indicators for task completion and failure. - Introduced methods for parsing and managing step observation events, ensuring accurate updates to task statuses during execution. - Enhanced validation for JSON crew projects, ensuring proper structure and error handling for agent and task definitions. - Added comprehensive tests for new features and validation logic, ensuring robustness in JSON crew project handling. * refactor(cli): streamline JSON crew project handling and improve validation - Refactored JSON crew project loading and validation logic to enhance clarity and maintainability. - Introduced utility functions for finding JSON crew files, improving code reuse across modules. - Removed deprecated benchmark functionality and associated tests to simplify the codebase. - Updated CLI commands to utilize the new JSON project structure, ensuring compatibility with recent changes. - Enhanced test coverage for JSON crew project features, ensuring robust validation and error handling. * feat(cli): enhance activity log navigation and focus management - Added functionality to focus on the activity log when navigating through log entries. - Implemented refresh logic for the log panel to ensure updates are displayed correctly during navigation. - Improved keyboard navigation for log entries, allowing users to expand and scroll through logs seamlessly. - Added tests to verify the correct behavior of log navigation and focus management in the TUI. * feat(cli): enhance JSON crew project interaction and input handling - Introduced a new function to enable prompt line editing for better user experience during input prompts. - Updated the JSON crew project wizards to show interpolation hints for dynamic values, improving user guidance. - Enhanced the handling of missing input placeholders by prompting users for required values during crew setup. - Refactored the crew run logic to ensure proper loading and preparation of JSON-defined crews, including runtime input management. - Added tests to verify the correct behavior of new input handling features and JSON crew project interactions. * feat(cli): improve crew project input prompts and event handling - Enhanced the `_prompt_text` function to allow for configurable spacing before prompts, improving user experience during input collection. - Updated the wizards for agent and task creation to utilize the new prompt configuration, ensuring a more compact and streamlined interaction. - Introduced new plan step lifecycle events (`PlanStepStartedEvent`, `PlanStepCompletedEvent`) to better track the execution status of plan steps. - Refactored the step executor to emit these events during the execution of tasks, improving observability and debugging capabilities. - Added tests to verify the correct behavior of new prompt handling and event emissions during crew project execution. * fix: refine json-first crew interactions * fix: prioritize common json crew tools * fix: make json crew more tools expandable * fix: show json crew tools by category * feat(memory): update default embedder to OpenAI text-embedding-3-large and enhance memory compatibility - Changed the default embedding model for Memory to OpenAI text-embedding-3-large, which uses 3072-dimensional vectors. - Added warnings regarding compatibility issues with existing local memory stores created with 1536-dimensional embeddings. - Updated documentation to reflect the new default embedder and its configuration options. - Enhanced the CLI and codebase to support the new embedding model across various components, ensuring a seamless transition for users. * fix: address PR review feedback for JSON-first crews Review blockers: - Forward trained_agents_file to JSON crews: crewai run -f now exports CREWAI_TRAINED_AGENTS_FILE for the in-process JSON crew path - Wizard agent picker: Esc/cancel now reprompts instead of silently assigning the first agent - JSON tool resolution hard-fails: unknown tool names, missing custom tool files, and invalid custom tool modules raise JSONProjectError with actionable messages instead of warn-and-continue - Embedding dimension mismatch: LanceDB and Qdrant Edge storages raise EmbeddingDimensionMismatchError with reset/pin guidance instead of silently zero-filling vectors or returning empty search results - Custom tool code execution documented in loader docstring and the scaffolded project README CI fixes: - ruff format across lib/ - All 133 PR-introduced mypy errors fixed (llm.py lazy-litellm and cli.py lazy command shims now use TYPE_CHECKING imports; textual is_mounted misuse fixed; pick_many overloads; misc annotations) Bot review comments: - Empty except blocks now have explanatory comments or debug logging - Removed unused _C_BG/_C_PANEL/_C_BORDER globals and redundant import re; tests use a single import style for create_json_crew Tests: trained-agents propagation, wizard cancel, tool resolution failures, and dimension mismatch guidance. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address second round of PR review comments Cursor Bugbot: - Wizard agent slugs: strip to [a-z0-9_] and fall back to agent_<n> so symbol-only roles can't produce an empty agents/.jsonc filename - Wizard task names: dedupe against prior task names and fall back to task_<n> for symbol-only descriptions CodeRabbit: - Agent.message(): import Task explicitly at runtime instead of relying on the namespace injection done by crewai/__init__ - Async executor: move the native-tools-unsupported fallback from _ainvoke_loop_react (self-recursion) to _ainvoke_loop_native_tools, mirroring the sync implementation - StepExecutor downgrade: keep the in-step conversation and append the text-tooling instructions instead of rebuilding messages, so completed native tool calls are not re-executed - crewai-files: extension-based MIME lookup now runs before byte sniffing so csv/xml types are not degraded to text/plain - Memory storages: validate every record in a save() batch against a consistent embedding dimension (LanceDB previously checked only the first record); added mixed-batch tests - _print_post_tui_summary now typed against CrewRunApp - Docs: Azure OpenAI default embedder change called out in the memory migration warning and provider table Code quality bots: - Removed unused _C_YELLOW/_C_CYAN (crew_run_tui) and _GREEN (tui_picker) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(cli): accordion tool picker in JSON crew wizard The flat tool list had grown to ~90 rows. The picker now shows: - Common tools always visible at the top - Every other category as a single expandable row with tool and selection counts (e.g. "Search & Research (27 tools, 2 selected)") - Expanding a category collapses the previously expanded one - Selections persist across expand/collapse via new preselected support in pick_many; cursor follows the toggled category row tui_picker gains preselected + initial_cursor options on pick_many, and Esc in multi-select now confirms the current selection instead of discarding it (required so collapsing can't silently drop choices). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(cli): remove --daemon flag from crewai run The flag only affected JSON crew projects — classic and flow projects ignored it entirely, which made the behavior inconsistent. Removed the option, the daemon code path (_run_json_crew_daemon), and its helper (_load_json_crew_with_inputs). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: update run command tests after --daemon removal lib/crewai/tests/cli/test_run_crew.py still asserted the old run_crew(trained_agents_file=..., daemon=False) call signature. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): exit codes, mid-run quit, async statuses, hyphen placeholders Addresses the latest Bugbot review round: - Failed JSON crew runs now exit non-zero (SystemExit(1)) so scripts and CI don't treat failures as success, mirroring the classic path - Quitting the TUI mid-run now ends the process (os._exit(130)); kickoff runs in a thread worker that cannot be force-cancelled, so letting the CLI return would leave LLM/tool work burning tokens in the background - Sidebar task statuses are now async-safe: completion/failure events resolve the task's own row via identity instead of assuming the most recently started task, and starting a task no longer blanket-marks earlier active rows as done - The runtime-input prompt regex now accepts hyphenated placeholder names ({my-topic}), matching kickoff's interpolation pattern Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: validation safety, custom tool sandboxing, TUI log integrity, memory error surfacing - Deploy validation no longer executes project code: validation mode checks tool declarations structurally (well-formed entries, custom tool file exists) without importing or instantiating anything. custom:<name> resolution only happens on the actual run path. - custom:<name> is constrained to [A-Za-z_][A-Za-z0-9_]* and the resolved path must stay inside the project's tools/ directory, so custom:../foo or absolute-path names cannot execute code outside it. Tool paths resolve relative to the crew project root, not cwd. - TUI task logs are built from per-task state captured at task start (idx, description, agent, start time); an out-of-order completion takes its output from the event and no longer steals or resets the current task's streamed steps/output. - EmbeddingDimensionMismatchError now inherits ValueError instead of RuntimeError so background saves surface it through MemorySaveFailedEvent instead of silently dropping the save; the shutdown catch in _background_encode_batch is narrowed to the "cannot schedule new futures" case. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): declared project type wins over crew.json presence A flow project that also contains a crew.json(c) file now runs and validates as the flow it declares in pyproject.toml instead of being hijacked by the JSON crew path. Both crewai run (_has_json_crew) and deploy validation (_is_json_crew) check tool.crewai.type; a missing or unreadable pyproject still means a bare JSON crew project. Also documents why StepObservationFailedEvent intentionally marks the plan step "done": the event signals an observer failure, not a step failure, and the executor continues past it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): type the declared_type locals so mypy stays clean Comparing an Any-typed .get() chain returns Any, which tripped no-any-return on the previous commit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-06-14 04:19:48 -03:00
Vini Brasil	d80719df81	Add experimental crewai run --definition for flows (#6147) ... Let users run a Flow from a Flow Definition YAML file or inline string without writing Python, passing kickoff inputs as `--inputs` JSON. The flag is gated behind an experimental warning since the definition format may still change.	2026-06-12 22:31:05 -07:00
devin-ai-integration[bot]	3010f1286f	chore: widen click dependency constraint to allow 8.2+ ... Addresses #6002	2026-06-02 00:06:25 -07:00
Greyson LaLonde	ed91100a0f	refactor(skills): move Skills Repository to experimental + CREWAI_EXPERIMENTAL gate ... Moves the registry/cache pieces of PR #5867 under crewai.experimental.skills and the CLI commands under `crewai experimental skill`. The stable local-file skills feature (loader, parser, validation, models) stays in crewai.skills. Both entry points now require CREWAI_EXPERIMENTAL=1: - resolve_registry_ref() calls require_experimental_skills() before resolving - The `crewai experimental` CLI group raises UsageError when the flag is unset SkillDownloadStarted/CompletedEvent move out of crewai.events.types.skill_events into crewai.experimental.skills.events. * refactor(skills): move 'version' off SkillFrontmatter into metadata The skill version is now stored as `metadata.version` rather than a top-level field on `SkillFrontmatter`. A `before` validator lifts any top-level YAML `version:` into `metadata['version']` so existing SKILL.md files keep parsing.	2026-05-28 09:38:10 -07:00
Greyson LaLonde	bad64b1ee6	chore(cli): drop self-explanatory comments	2026-05-26 01:05:25 -07:00
alex-clawd	418afd29e7	feat: Skills Repository — registry, cache, CLI, and SDK integration (#5867) ... * feat: add Skills Repository — registry, cache, CLI, and SDK integration Adds a Skills Repository feature allowing users to publish, install, and use skills from the CrewAI registry with @org/skill-name refs. ## What's New ### SDK (lib/crewai/) - SkillFrontmatter: added optional 'version' field (backward compatible) - SkillCacheManager: manages ~/.crewai/skills/{org}/{name}/ with .crewai_meta.json tracking, path-traversal-safe tar extraction - SkillRegistry: parse @org/skill-name refs, local-first resolution (./skills/ > cache > download), interactive prompt on first use, CI-mode guard (CREWAI_NONINTERACTIVE/CI env vars) - Agent.skills and Crew.skills widened to accept str refs (@org/name) - set_skills() resolves registry refs with org-prefixed dedup keys - New events: SkillDownloadStartedEvent, SkillDownloadCompletedEvent ### CLI (lib/cli/) - crewai skill create <name> — context-aware (project vs standalone) - crewai skill install @org/name — downloads to ./skills/ or cache - crewai skill publish — ZIP + upload to org registry - crewai skill list — show installed skills ### PlusAPI (lib/crewai-core/) - Added SKILLS_RESOURCE, get_skill(), publish_skill(), list_skills() ### Scaffolding - crew and flow templates now include skills/ directory ### Tests - 91 SDK skill tests + 15 CLI skill tests, all passing * fix: address all CI failures and CodeRabbit review comments Lint: - Remove unused imports (click, pytest, json) - Replace try-except-pass with logging (S110) - Fix unprotected zipfile.extractall (S202) Security: - Path traversal: startswith → is_relative_to for tar extraction - Add path traversal protection to ZIP extraction via _safe_extract_zip - Both cache.py and CLI main.py hardened Type checker: - Fix import path: crewai.events.event_bus (not crewai_event_bus) - Remove unused type: ignore comments - Fix type mismatches in set_skills() variable types Code quality: - Fix f-string interpolation in SkillNotCachedError - Use ValidationError instead of Exception in test * style: ruff format + autofix remaining lint errors * refactor: reuse SDK parser and SkillCacheManager in CLI - _parse_frontmatter() now delegates to crewai.skills.parser.parse_frontmatter when available, with a minimal fallback for CLI-only installs - install() global cache path now reuses SkillCacheManager.store() instead of duplicating metadata writing logic * refactor: add _print_current_organization to SkillCommand (matches ToolCommand pattern) * fix: write .crewai_meta.json in fallback install path CodeRabbit caught that the ImportError fallback in install() didn't write cache metadata, making skills invisible to 'crewai skill list'. * fix: tighten @org/name ref validation to prevent path traversal Reject refs with multiple slashes (@org/a/b), dot segments (@../skill), or leading dots in org/name. Applied to both CLI install() and SDK parse_registry_ref() so the contract is enforced consistently. * fix: update test assertions to match tightened error messages * fix: align OSS client with AMP API contract - download_skill(): fetch download_url (presigned URL) instead of expecting inline base64. Falls back to 'file' field for compat. - Read 'latest_version' field, fall back to 'version' - Same fixes applied to CLI install() command * fix: publish as tar.gz (matches AMP content_type validation) + add zip fallback to SDK cache CLI publish: - _build_skill_zip → _build_skill_tarball (tar.gz format) - Content type: application/x-gzip (matches SkillVersion validation) SDK cache: - store() now tries tar.gz first, falls back to zip extraction - Added _safe_extract_zip for path-traversal-safe zip handling - Both formats work for download/install regardless of server format --------- Co-authored-by: João Moura <joaomdmoura@gmail.com>	2026-05-20 14:38:25 -03:00
Greyson LaLonde	93e786d263	refactor: extract CLI into standalone crewai-cli package	2026-05-06 20:46:46 +08:00