9 Commits

Author	SHA1	Message	Date
João Moura	e9d568dc69	Deep Crew / Agent / Task attributes support on json (#6172) ... * Enhance JSON crew project handling and validation - Updated `create_json_crew.py` to specify input files with a brief path. - Refactored `crew_loader.py` to improve agent and task loading logic, including the introduction of a `build_agent` function and better handling of task classes. - Enhanced `json_loader.py` with additional validation for agent and task definitions, including support for Python references and conditional tasks. - Added tests in `test_crew_loader.py` and `test_json_loader.py` to ensure proper loading of agents, tasks, and validation of project structures, including custom types and conditional tasks. - Improved error handling and validation safety across the project loading process. * Enhance JSON crew configuration options in create_json_crew.py - Added optional fields for custom agent subclasses and advanced task options, including condition checks and output specifications. - Improved documentation comments for better clarity on agent and task configurations. - Updated JSON crew handling to support additional callbacks for pre- and post-execution processes. * Enhance JSON crew template tests in test_create_crew.py - Added assertions for new optional fields in crew and agent templates, including conditional tasks, custom converters, and input file specifications. - Improved validation checks for manager agents and callback references to ensure proper configuration in JSON crew definitions. - Expanded documentation references within the tests to provide clearer guidance on the expected structure and usage of crew templates. * Fix JSON crew PR review issues	2026-06-16 02:00:19 -03:00
João Moura	53c2284484	Support ZIP deployment fallback and JSON crew project env runs (#6166) ... * Update crewAI CLI with various enhancements and fixes - Updated `create_json_crew.py` to require `crewai[tools]>=1.14.7`. - Enhanced `git.py` with improved repository initialization, including automatic initial commit creation and exclusion patterns for initial commits. - Modified `install_crew.py` to allow error handling during installation with an optional `raise_on_error` parameter. - Expanded `plus_api.py` to include methods for creating and updating crews from ZIP files. - Introduced a new `archive.py` for creating deployable ZIP archives of CrewAI projects, ensuring local artifacts are excluded. - Updated `run_crew.py` to manage JSON crew dependencies and run crews in the project's environment. - Enhanced deployment logic in `main.py` to handle ZIP uploads and improve user feedback during deployment processes. - Added tests for new functionalities and ensured existing tests reflect recent changes in behavior and requirements. * fix(cli): address deploy zip review feedback * fix(cli): sync missing lockfile before deploy * fix(cli): preserve remote deploy on git setup warnings * test(cli): use single deploy main import style * fix(cli): skip project install for json crew sync * fix(cli): load json runner from source checkout * fix(cli): skip json crew sync when locked * fix(cli): address deploy zip review feedback * fix(cli): pass env on zip redeploy * fix(cli): harden json run and zip fallback * fix(cli): validate before deploy lock install * fix(cli): respect poetry lock for json runs * fix(cli): align json zip wrapper detection * fix(deps): bump starlette audit floor * fix(cli): avoid auth retry for deploy exits * fix(cli): update json zip script entrypoints	2026-06-15 18:46:54 -03:00
João Moura	bb477f8a91	JSON first crews (#6131) ... * feat(cli): introduce JSON crew project support and TUI enhancements - Added support for creating and running JSON-defined crew projects, allowing users to scaffold projects with a new `create_json_crew.py` file. - Implemented a full-screen Textual TUI for crew execution in `crew_run_tui.py`, enhancing user interaction with a two-column layout. - Updated `run_crew.py` to prioritize JSON crew projects and added daemon mode for running without TUI. - Introduced interactive pickers in `tui_picker.py` for improved CLI prompts. - Enhanced validation for JSON crew files in `validate.py` to ensure proper structure and agent definitions. - Updated `.gitignore` to exclude demo and crewai directories. * feat: update LLM model references to gpt-5.4-mini - Changed default LLM model from gpt-4o-mini to gpt-5.4-mini across various files, including CLI options, JSON crew configurations, and agent definitions. - Enhanced benchmark and human feedback functionalities to utilize the new model. - Improved user interface elements in the TUI for better interaction and feedback during execution. - Added support for new skills directory in JSON crew project creation. * feat(benchmark): add crew-level benchmarking functionality - Introduced a new `benchmark` command in the CLI for crew-level benchmarking, allowing users to specify agents, models, and timeout settings. - Implemented `CrewBenchmarkCase` to handle crew-level benchmark cases with inputs and criteria. - Enhanced the benchmark runner to support progress tracking and detailed reporting of results for multiple models. - Added tests for loading crew benchmark cases and validating their structure. - Updated existing benchmark functions to accommodate the new crew-level execution model. * feat(cli): enhance JSON crew project functionality and TUI improvements - Added optional agent-level guardrails and advanced options in JSON crew configurations to improve output validation and flexibility. - Updated the TUI to better handle plan step statuses, including visual indicators for task completion and failure. - Introduced methods for parsing and managing step observation events, ensuring accurate updates to task statuses during execution. - Enhanced validation for JSON crew projects, ensuring proper structure and error handling for agent and task definitions. - Added comprehensive tests for new features and validation logic, ensuring robustness in JSON crew project handling. * refactor(cli): streamline JSON crew project handling and improve validation - Refactored JSON crew project loading and validation logic to enhance clarity and maintainability. - Introduced utility functions for finding JSON crew files, improving code reuse across modules. - Removed deprecated benchmark functionality and associated tests to simplify the codebase. - Updated CLI commands to utilize the new JSON project structure, ensuring compatibility with recent changes. - Enhanced test coverage for JSON crew project features, ensuring robust validation and error handling. * feat(cli): enhance activity log navigation and focus management - Added functionality to focus on the activity log when navigating through log entries. - Implemented refresh logic for the log panel to ensure updates are displayed correctly during navigation. - Improved keyboard navigation for log entries, allowing users to expand and scroll through logs seamlessly. - Added tests to verify the correct behavior of log navigation and focus management in the TUI. * feat(cli): enhance JSON crew project interaction and input handling - Introduced a new function to enable prompt line editing for better user experience during input prompts. - Updated the JSON crew project wizards to show interpolation hints for dynamic values, improving user guidance. - Enhanced the handling of missing input placeholders by prompting users for required values during crew setup. - Refactored the crew run logic to ensure proper loading and preparation of JSON-defined crews, including runtime input management. - Added tests to verify the correct behavior of new input handling features and JSON crew project interactions. * feat(cli): improve crew project input prompts and event handling - Enhanced the `_prompt_text` function to allow for configurable spacing before prompts, improving user experience during input collection. - Updated the wizards for agent and task creation to utilize the new prompt configuration, ensuring a more compact and streamlined interaction. - Introduced new plan step lifecycle events (`PlanStepStartedEvent`, `PlanStepCompletedEvent`) to better track the execution status of plan steps. - Refactored the step executor to emit these events during the execution of tasks, improving observability and debugging capabilities. - Added tests to verify the correct behavior of new prompt handling and event emissions during crew project execution. * fix: refine json-first crew interactions * fix: prioritize common json crew tools * fix: make json crew more tools expandable * fix: show json crew tools by category * feat(memory): update default embedder to OpenAI text-embedding-3-large and enhance memory compatibility - Changed the default embedding model for Memory to OpenAI text-embedding-3-large, which uses 3072-dimensional vectors. - Added warnings regarding compatibility issues with existing local memory stores created with 1536-dimensional embeddings. - Updated documentation to reflect the new default embedder and its configuration options. - Enhanced the CLI and codebase to support the new embedding model across various components, ensuring a seamless transition for users. * fix: address PR review feedback for JSON-first crews Review blockers: - Forward trained_agents_file to JSON crews: crewai run -f now exports CREWAI_TRAINED_AGENTS_FILE for the in-process JSON crew path - Wizard agent picker: Esc/cancel now reprompts instead of silently assigning the first agent - JSON tool resolution hard-fails: unknown tool names, missing custom tool files, and invalid custom tool modules raise JSONProjectError with actionable messages instead of warn-and-continue - Embedding dimension mismatch: LanceDB and Qdrant Edge storages raise EmbeddingDimensionMismatchError with reset/pin guidance instead of silently zero-filling vectors or returning empty search results - Custom tool code execution documented in loader docstring and the scaffolded project README CI fixes: - ruff format across lib/ - All 133 PR-introduced mypy errors fixed (llm.py lazy-litellm and cli.py lazy command shims now use TYPE_CHECKING imports; textual is_mounted misuse fixed; pick_many overloads; misc annotations) Bot review comments: - Empty except blocks now have explanatory comments or debug logging - Removed unused _C_BG/_C_PANEL/_C_BORDER globals and redundant import re; tests use a single import style for create_json_crew Tests: trained-agents propagation, wizard cancel, tool resolution failures, and dimension mismatch guidance. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: address second round of PR review comments Cursor Bugbot: - Wizard agent slugs: strip to [a-z0-9_] and fall back to agent_<n> so symbol-only roles can't produce an empty agents/.jsonc filename - Wizard task names: dedupe against prior task names and fall back to task_<n> for symbol-only descriptions CodeRabbit: - Agent.message(): import Task explicitly at runtime instead of relying on the namespace injection done by crewai/__init__ - Async executor: move the native-tools-unsupported fallback from _ainvoke_loop_react (self-recursion) to _ainvoke_loop_native_tools, mirroring the sync implementation - StepExecutor downgrade: keep the in-step conversation and append the text-tooling instructions instead of rebuilding messages, so completed native tool calls are not re-executed - crewai-files: extension-based MIME lookup now runs before byte sniffing so csv/xml types are not degraded to text/plain - Memory storages: validate every record in a save() batch against a consistent embedding dimension (LanceDB previously checked only the first record); added mixed-batch tests - _print_post_tui_summary now typed against CrewRunApp - Docs: Azure OpenAI default embedder change called out in the memory migration warning and provider table Code quality bots: - Removed unused _C_YELLOW/_C_CYAN (crew_run_tui) and _GREEN (tui_picker) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * feat(cli): accordion tool picker in JSON crew wizard The flat tool list had grown to ~90 rows. The picker now shows: - Common tools always visible at the top - Every other category as a single expandable row with tool and selection counts (e.g. "Search & Research (27 tools, 2 selected)") - Expanding a category collapses the previously expanded one - Selections persist across expand/collapse via new preselected support in pick_many; cursor follows the toggled category row tui_picker gains preselected + initial_cursor options on pick_many, and Esc in multi-select now confirms the current selection instead of discarding it (required so collapsing can't silently drop choices). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * refactor(cli): remove --daemon flag from crewai run The flag only affected JSON crew projects — classic and flow projects ignored it entirely, which made the behavior inconsistent. Removed the option, the daemon code path (_run_json_crew_daemon), and its helper (_load_json_crew_with_inputs). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: update run command tests after --daemon removal lib/crewai/tests/cli/test_run_crew.py still asserted the old run_crew(trained_agents_file=..., daemon=False) call signature. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): exit codes, mid-run quit, async statuses, hyphen placeholders Addresses the latest Bugbot review round: - Failed JSON crew runs now exit non-zero (SystemExit(1)) so scripts and CI don't treat failures as success, mirroring the classic path - Quitting the TUI mid-run now ends the process (os._exit(130)); kickoff runs in a thread worker that cannot be force-cancelled, so letting the CLI return would leave LLM/tool work burning tokens in the background - Sidebar task statuses are now async-safe: completion/failure events resolve the task's own row via identity instead of assuming the most recently started task, and starting a task no longer blanket-marks earlier active rows as done - The runtime-input prompt regex now accepts hyphenated placeholder names ({my-topic}), matching kickoff's interpolation pattern Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix: validation safety, custom tool sandboxing, TUI log integrity, memory error surfacing - Deploy validation no longer executes project code: validation mode checks tool declarations structurally (well-formed entries, custom tool file exists) without importing or instantiating anything. custom:<name> resolution only happens on the actual run path. - custom:<name> is constrained to [A-Za-z_][A-Za-z0-9_]* and the resolved path must stay inside the project's tools/ directory, so custom:../foo or absolute-path names cannot execute code outside it. Tool paths resolve relative to the crew project root, not cwd. - TUI task logs are built from per-task state captured at task start (idx, description, agent, start time); an out-of-order completion takes its output from the event and no longer steals or resets the current task's streamed steps/output. - EmbeddingDimensionMismatchError now inherits ValueError instead of RuntimeError so background saves surface it through MemorySaveFailedEvent instead of silently dropping the save; the shutdown catch in _background_encode_batch is narrowed to the "cannot schedule new futures" case. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): declared project type wins over crew.json presence A flow project that also contains a crew.json(c) file now runs and validates as the flow it declares in pyproject.toml instead of being hijacked by the JSON crew path. Both crewai run (_has_json_crew) and deploy validation (_is_json_crew) check tool.crewai.type; a missing or unreadable pyproject still means a bare JSON crew project. Also documents why StepObservationFailedEvent intentionally marks the plan step "done": the event signals an observer failure, not a step failure, and the executor continues past it. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * fix(cli): type the declared_type locals so mypy stays clean Comparing an Any-typed .get() chain returns Any, which tripped no-any-return on the previous commit. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> --------- Co-authored-by: Claude Fable 5 <noreply@anthropic.com>	2026-06-14 04:19:48 -03:00
Vini Brasil	d80719df81	Add experimental crewai run --definition for flows (#6147) ... Let users run a Flow from a Flow Definition YAML file or inline string without writing Python, passing kickoff inputs as `--inputs` JSON. The flag is gated behind an experimental warning since the definition format may still change.	2026-06-12 22:31:05 -07:00
devin-ai-integration[bot]	3010f1286f	chore: widen click dependency constraint to allow 8.2+ ... Addresses #6002	2026-06-02 00:06:25 -07:00
Greyson LaLonde	ed91100a0f	refactor(skills): move Skills Repository to experimental + CREWAI_EXPERIMENTAL gate ... Moves the registry/cache pieces of PR #5867 under crewai.experimental.skills and the CLI commands under `crewai experimental skill`. The stable local-file skills feature (loader, parser, validation, models) stays in crewai.skills. Both entry points now require CREWAI_EXPERIMENTAL=1: - resolve_registry_ref() calls require_experimental_skills() before resolving - The `crewai experimental` CLI group raises UsageError when the flag is unset SkillDownloadStarted/CompletedEvent move out of crewai.events.types.skill_events into crewai.experimental.skills.events. * refactor(skills): move 'version' off SkillFrontmatter into metadata The skill version is now stored as `metadata.version` rather than a top-level field on `SkillFrontmatter`. A `before` validator lifts any top-level YAML `version:` into `metadata['version']` so existing SKILL.md files keep parsing.	2026-05-28 09:38:10 -07:00
Greyson LaLonde	bad64b1ee6	chore(cli): drop self-explanatory comments	2026-05-26 01:05:25 -07:00
alex-clawd	418afd29e7	feat: Skills Repository — registry, cache, CLI, and SDK integration (#5867) ... * feat: add Skills Repository — registry, cache, CLI, and SDK integration Adds a Skills Repository feature allowing users to publish, install, and use skills from the CrewAI registry with @org/skill-name refs. ## What's New ### SDK (lib/crewai/) - SkillFrontmatter: added optional 'version' field (backward compatible) - SkillCacheManager: manages ~/.crewai/skills/{org}/{name}/ with .crewai_meta.json tracking, path-traversal-safe tar extraction - SkillRegistry: parse @org/skill-name refs, local-first resolution (./skills/ > cache > download), interactive prompt on first use, CI-mode guard (CREWAI_NONINTERACTIVE/CI env vars) - Agent.skills and Crew.skills widened to accept str refs (@org/name) - set_skills() resolves registry refs with org-prefixed dedup keys - New events: SkillDownloadStartedEvent, SkillDownloadCompletedEvent ### CLI (lib/cli/) - crewai skill create <name> — context-aware (project vs standalone) - crewai skill install @org/name — downloads to ./skills/ or cache - crewai skill publish — ZIP + upload to org registry - crewai skill list — show installed skills ### PlusAPI (lib/crewai-core/) - Added SKILLS_RESOURCE, get_skill(), publish_skill(), list_skills() ### Scaffolding - crew and flow templates now include skills/ directory ### Tests - 91 SDK skill tests + 15 CLI skill tests, all passing * fix: address all CI failures and CodeRabbit review comments Lint: - Remove unused imports (click, pytest, json) - Replace try-except-pass with logging (S110) - Fix unprotected zipfile.extractall (S202) Security: - Path traversal: startswith → is_relative_to for tar extraction - Add path traversal protection to ZIP extraction via _safe_extract_zip - Both cache.py and CLI main.py hardened Type checker: - Fix import path: crewai.events.event_bus (not crewai_event_bus) - Remove unused type: ignore comments - Fix type mismatches in set_skills() variable types Code quality: - Fix f-string interpolation in SkillNotCachedError - Use ValidationError instead of Exception in test * style: ruff format + autofix remaining lint errors * refactor: reuse SDK parser and SkillCacheManager in CLI - _parse_frontmatter() now delegates to crewai.skills.parser.parse_frontmatter when available, with a minimal fallback for CLI-only installs - install() global cache path now reuses SkillCacheManager.store() instead of duplicating metadata writing logic * refactor: add _print_current_organization to SkillCommand (matches ToolCommand pattern) * fix: write .crewai_meta.json in fallback install path CodeRabbit caught that the ImportError fallback in install() didn't write cache metadata, making skills invisible to 'crewai skill list'. * fix: tighten @org/name ref validation to prevent path traversal Reject refs with multiple slashes (@org/a/b), dot segments (@../skill), or leading dots in org/name. Applied to both CLI install() and SDK parse_registry_ref() so the contract is enforced consistently. * fix: update test assertions to match tightened error messages * fix: align OSS client with AMP API contract - download_skill(): fetch download_url (presigned URL) instead of expecting inline base64. Falls back to 'file' field for compat. - Read 'latest_version' field, fall back to 'version' - Same fixes applied to CLI install() command * fix: publish as tar.gz (matches AMP content_type validation) + add zip fallback to SDK cache CLI publish: - _build_skill_zip → _build_skill_tarball (tar.gz format) - Content type: application/x-gzip (matches SkillVersion validation) SDK cache: - store() now tries tar.gz first, falls back to zip extraction - Added _safe_extract_zip for path-traversal-safe zip handling - Both formats work for download/install regardless of server format --------- Co-authored-by: João Moura <joaomdmoura@gmail.com>	2026-05-20 14:38:25 -03:00
Greyson LaLonde	93e786d263	refactor: extract CLI into standalone crewai-cli package	2026-05-06 20:46:46 +08:00