mirror of
https://github.com/crewAIInc/crewAI.git
synced 2026-08-10 00:11:57 +00:00
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Vulnerability Scan / pip-audit (push) Has been cancelled
Build uv cache / build-cache (3.10) (push) Has been cancelled
Build uv cache / build-cache (3.11) (push) Has been cancelled
Build uv cache / build-cache (3.12) (push) Has been cancelled
Build uv cache / build-cache (3.13) (push) Has been cancelled
Nightly Canary Release / Check for new commits (push) Has been cancelled
Nightly Canary Release / Build nightly packages (push) Has been cancelled
Nightly Canary Release / Publish nightly to PyPI (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
* feat: add project_id to link OSS usage to an enterprise account Adds a stable per-project identifier so a project's OSS traces and runs can be attributed to an account after signup. There was no such identifier before: [tool.crewai] held only `type`, the deploy UUID was printed to the console but never persisted, Settings.org_uuid is global rather than per-project, and trace batches carried only crew_fingerprint/crew_name. The id lives in the project's pyproject.toml, so it is committed with the repository and stays stable across machines, teammates, CI, and containers - unlike a machine- or user-derived identifier, which is unstable in exactly the containerized production environments that matter most. crewai-core: - get_project_id(): read-only lookup of [tool.crewai].project_id. Safe for library code; never creates or modifies anything. - get_or_create_project_id(): mints a uuid4 and persists it, returning (id, created) so callers can tell the user. Best-effort - returns (None, False) for a missing, malformed, or read-only pyproject.toml rather than raising. - Insertion edits the raw TOML text instead of round-tripping through a writer, so comments, key order, and formatting elsewhere survive. The key is placed at the end of the [tool.crewai] table, before the next table header, so it cannot land in a neighbouring section. - LoginPayload and TraceExecutionContext gain optional project_id. Sent on two paths: - Traces: project_id is added to execution_context, which is sent on both the ephemeral and authenticated paths, so a project's traces remain attributable before and after the user creates an account. - Login: `crewai login` already sends the pseudonymous user_identifier on an authenticated request; adding project_id means one request carries account + user + project, which is the link itself. Minting is restricted to CLI commands the user explicitly invoked - `crewai create` for new projects and `crewai run` to backfill existing ones - and is announced when it happens. Library code only ever reads. Silently rewriting a user's pyproject.toml during Crew.kickoff() would be surprising. Privacy: project_id is a random uuid4 in a file the user commits. It is visible in a diff, contains nothing personal, and identifies a project rather than a person - so this needs none of the notice changes that attaching a user identifier to all telemetry would require. Tests: 18 new tests covering minting, stability, table placement, comment and formatting preservation, five pyproject layouts, the neighbouring-table regression, and graceful handling of missing/malformed/read-only files. Verified end-to-end that both create paths mint distinct ids, that the trace payload carries project_id on both the ephemeral and authenticated paths, and that the login payload carries user_identifier and project_id together. Follow-ups, deliberately not included: adding project_id to telemetry spans, and backend persistence of the (account, user_identifier, project_id) triple. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UNumDnNbiyw3pv1WakAe6t * refactor: drop the console announcement when minting project_id Minting now happens silently. With no message to print, the (id, created) tuple had no consumer, so simplify the API rather than keep the flag around for a hypothetical caller: - get_or_create_project_id() returns `str | None` instead of `tuple[str | None, bool]`. - Remove crewai_cli.utils.ensure_project_id, which existed only to print the message and discard the flag. The four call sites (crewai create crew, crewai create flow, crewai run, and tool-repository login) now call get_or_create_project_id directly. - Update tests for the simplified signature; still 18 tests covering minting, stability, table placement, formatting preservation, five pyproject layouts, and missing/malformed/read-only handling. Behaviour is otherwise unchanged: minting stays restricted to CLI commands the user invoked, library code still only reads via get_project_id, and a missing or read-only pyproject.toml still returns None rather than raising. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UNumDnNbiyw3pv1WakAe6t * fix: harden project_id minting against TOML corruption; address review Several reviewers found ways the raw-text edit could produce invalid TOML. Each is now fixed and covered by a test that fails without the fix. Duplicate project_id key (Cursor bugbot, Copilot x2): - get_project_id() reports a blank or non-string value as "absent", so a file containing `project_id = ""` took the insert path and gained a second project_id line - a duplicate key, and therefore invalid TOML that no tomli-based tool could read afterwards. - _insert_project_id is now _set_project_id: it replaces an existing assignment inside [tool.crewai] instead of appending unconditionally. Table header with a trailing comment (CodeRabbit major, Cursor bugbot): - `[tool.crewai] # config` is valid TOML but failed exact string equality, so the fallback appended a second [tool.crewai] header - a redefined table, also invalid TOML, and silent because get_project_id swallows the resulting decode error. - Added _is_table_header(), which tolerates a trailing comment and does not match similar names such as [tool.crewai-extra]. Writing into malformed TOML (Cursor bugbot, Copilot): - get_or_create_project_id relied on get_project_id, which cannot distinguish "no id" from "unparsable file", so it appended to files it could not parse. - The locked path now parses explicitly and bails on a decode error, and re-parses the updated content before writing, so this feature can never be the reason a project's pyproject.toml stops parsing. Concurrency and atomicity (CodeRabbit major): - Two CLI processes could both see no id, mint different uuids, and clobber each other, leaving a caller holding an id that is not on disk. Minting now takes the existing crewai_core cross-process lock, re-reads under it, and returns the id that persists. - Writes go through a temp file in the same directory plus os.replace, so an interruption cannot truncate pyproject.toml. File mode is copied across, and the temp file is removed on failure. - os.replace only needs a writable directory, which would have let an atomic write silently overwrite a file the user marked read-only; writability is now checked explicitly so that case still returns None. Line endings (CodeRabbit): - Path.read_text/write_text normalized CRLF to LF, so minting would rewrite a CRLF-committed file entirely. Read and write now use newline="" and the inserted line ending is derived from the existing content. Default create path skipped minting (Cursor bugbot): - `crewai create crew` defaults to create_json_crew; only the --classic and flow paths minted, so most new projects had no id until a later command. Wired into create_json_crew as well. Verified all three paths now mint distinct ids. Do not mint during login (CodeRabbit major): - ToolCommand.login ran get_or_create_project_id, which is outside the sanctioned minting commands and is invoked by `crewai tools create` from a freshly scaffolded directory before the project is persisted. It now uses the read-only get_project_id. Verified login leaves pyproject.toml untouched. Not applied: Copilot asked for a console message when an id is written, in create_crew and create_flow. Minting was made deliberately silent in the previous commit, so the (id, created) tuple and the announcement are both gone by design. Tests: 32 in test_project_id.py, up from 18. New cases cover blank and non-string existing ids, three commented-header forms, similar table names, malformed input, CRLF and LF preservation, concurrent minting convergence, file-mode preservation, and temp-file cleanup. Confirmed the header and duplicate-key tests fail when the fixes are reverted. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UNumDnNbiyw3pv1WakAe6t * fix: never create [tool.crewai], treat whitespace ids as absent, harden test `crewai run` could rewrite unrelated projects (Cursor bugbot, high): - get_or_create_project_id ran before the cwd was established as a CrewAI project, and _set_project_id appended a [tool.crewai] table when none existed. Any directory with a pyproject.toml could therefore gain one - including on `crewai run --definition`, which may otherwise succeed. - _set_project_id no longer creates the table; it returns None when [tool.crewai] is absent, so a key is only ever added to a table the project already declares. The templates all ship the table, so no create path needs the old fallback. - The minting call in run_crew moved after the --definition early return, so an explicit-flow run does not touch the cwd at all. - Presence is checked, not truthiness: an empty [tool.crewai] is still a CrewAI marker, and get_crewai_project_config returns {} both for that and for an absent table. - Verified an unrelated project's pyproject.toml is byte-identical after a mint attempt. Whitespace-only project_id accepted as valid (CodeRabbit): - `project_id = " "` is truthy, so it was returned as an identity and would have propagated into login payloads and tracing context. It also meant the '" "' parameter of the replacement test asserted nothing. - Added _usable_project_id, which strips before deciding, used by both get_project_id and the locked mint path. Concurrency test could hang CI (CodeRabbit, major): - Neither the barrier nor the joins had timeouts, so a thread dying early or blocking on the lock would hang the job rather than fail it. The result count was also unchecked, so a dead thread still passed. - Added timeouts, an explicit liveness assertion, a result-count assertion, a lock around the shared result list, and corrected the docstring: this covers the read-modify-write race with threads, not the cross-process backend. Tests: 35, up from 32. New coverage for the absent-table refusal and three whitespace forms; the blank-id replacement case now asserts a real uuid replaced the blank value. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UNumDnNbiyw3pv1WakAe6t * chore(deps): force gitpython 3.1.57+ for GHSA-p538-c434-8v24 and GHSA-3f7w-8rr8-f37f Unrelated to project_id; bundled here only because it blocks this PR's vulnerability scan. Two advisories were published for gitpython 3.1.55 after main last passed the scan: - GHSA-p538-c434-8v24: arbitrary file truncation via `git rev-list --output` argument injection. Fixed in 3.1.56. - GHSA-3f7w-8rr8-f37f: unguarded git option forwarding in IndexFile.checkout() and TagReference. Fixed in 3.1.57. - Bump the override floor to gitpython>=3.1.57 and declare the same floor in crewai-tools, so consumers installing the published package are covered and not only this repo's lock. - 3.1.57 was published 2026-07-26, past gitpython's exclude-newer-package cutoff of 2026-07-24, so that cutoff moves to 2026-07-27. Without it the floor is unresolvable. pip-audit against the updated lock reports no known vulnerabilities. Verified gitpython 3.1.57 resolves and that crewai_tools and crewai_cli.git still import. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01UNumDnNbiyw3pv1WakAe6t --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
183 lines
5.9 KiB
Python
183 lines
5.9 KiB
Python
from __future__ import annotations
|
|
|
|
from collections.abc import Mapping
|
|
import os
|
|
from pathlib import Path
|
|
import re
|
|
import shutil
|
|
from typing import Any
|
|
|
|
import click
|
|
from crewai_core.project import (
|
|
get_or_create_project_id as get_or_create_project_id,
|
|
get_project_description as get_project_description,
|
|
get_project_id as get_project_id,
|
|
get_project_name as get_project_name,
|
|
get_project_version as get_project_version,
|
|
parse_toml as parse_toml,
|
|
read_toml as read_toml,
|
|
)
|
|
from crewai_core.tool_credentials import (
|
|
build_env_with_all_tool_credentials as build_env_with_all_tool_credentials,
|
|
build_env_with_tool_repository_credentials as build_env_with_tool_repository_credentials,
|
|
)
|
|
from rich.console import Console
|
|
|
|
from crewai_cli.version import get_crewai_tools_dependency
|
|
|
|
|
|
__all__ = [
|
|
"build_env_with_all_tool_credentials",
|
|
"build_env_with_tool_repository_credentials",
|
|
"copy_template",
|
|
"enable_prompt_line_editing",
|
|
"fetch_and_json_env_file",
|
|
"get_or_create_project_id",
|
|
"get_project_description",
|
|
"get_project_id",
|
|
"get_project_name",
|
|
"get_project_version",
|
|
"is_dmn_mode_enabled",
|
|
"load_env_vars",
|
|
"parse_toml",
|
|
"read_toml",
|
|
"render_template",
|
|
"tree_copy",
|
|
"tree_find_and_replace",
|
|
"write_env_file",
|
|
]
|
|
|
|
|
|
console = Console()
|
|
_TEMPLATE_TOKEN_RE = re.compile(r"{{([a-zA-Z_][a-zA-Z0-9_]*)}}")
|
|
|
|
|
|
def is_dmn_mode_enabled() -> bool:
|
|
"""Return True when the enterprise non-interactive mode is enabled."""
|
|
value = os.environ.get("CREWAI_DMN")
|
|
if value is None:
|
|
return False
|
|
return value.strip().lower() not in {"", "0", "false", "no", "off"}
|
|
|
|
|
|
def enable_prompt_line_editing() -> None:
|
|
"""Enable cursor movement/history editing for Click text prompts when available."""
|
|
try:
|
|
import readline
|
|
except ImportError:
|
|
return
|
|
|
|
try:
|
|
readline.parse_and_bind("set editing-mode emacs")
|
|
except Exception: # pragma: no cover - readline backends vary by platform
|
|
return
|
|
|
|
|
|
def copy_template(
|
|
src: Path, dst: Path, name: str, class_name: str, folder_name: str
|
|
) -> None:
|
|
"""Copy a file from src to dst."""
|
|
content = render_template(
|
|
src,
|
|
{
|
|
"name": name,
|
|
"crew_name": class_name,
|
|
"folder_name": folder_name,
|
|
"crewai_tools_dependency": get_crewai_tools_dependency(),
|
|
},
|
|
)
|
|
|
|
with open(dst, "w") as file:
|
|
file.write(content)
|
|
|
|
click.secho(f" - Created {dst}", fg="green")
|
|
|
|
|
|
def render_template(src: Path, replacements: Mapping[str, str]) -> str:
|
|
"""Render a template file using ``{{placeholder}}`` replacements."""
|
|
content = src.read_text(encoding="utf-8")
|
|
return _TEMPLATE_TOKEN_RE.sub(
|
|
lambda match: replacements.get(match.group(1), match.group(0)),
|
|
content,
|
|
)
|
|
|
|
|
|
def fetch_and_json_env_file(env_file_path: str = ".env") -> dict[str, Any]:
|
|
"""Fetch the environment variables from a .env file and return them as a dictionary."""
|
|
try:
|
|
with open(env_file_path, "r") as f:
|
|
env_content = f.read()
|
|
|
|
env_dict = {}
|
|
for line in env_content.splitlines():
|
|
if line.strip() and not line.strip().startswith("#"):
|
|
key, value = line.split("=", 1)
|
|
env_dict[key.strip()] = value.strip()
|
|
|
|
return env_dict
|
|
|
|
except FileNotFoundError:
|
|
console.print(f"Error: {env_file_path} not found.", style="bold red")
|
|
except Exception as e:
|
|
console.print(f"Error reading the .env file: {e}", style="bold red")
|
|
|
|
return {}
|
|
|
|
|
|
def tree_copy(source: Path, destination: Path) -> None:
|
|
"""Copies the entire directory structure from the source to the destination."""
|
|
for item in os.listdir(source):
|
|
source_item = os.path.join(source, item)
|
|
destination_item = os.path.join(destination, item)
|
|
if os.path.isdir(source_item):
|
|
shutil.copytree(source_item, destination_item)
|
|
else:
|
|
shutil.copy2(source_item, destination_item)
|
|
|
|
|
|
def tree_find_and_replace(directory: Path, find: str, replace: str) -> None:
|
|
"""Recursively searches through a directory, replacing a target string in
|
|
both file contents and filenames with a specified replacement string.
|
|
"""
|
|
for path, dirs, files in os.walk(os.path.abspath(directory), topdown=False):
|
|
for filename in files:
|
|
filepath = os.path.join(path, filename)
|
|
|
|
with open(filepath, "r", encoding="utf-8", errors="ignore") as file:
|
|
contents = file.read()
|
|
with open(filepath, "w") as file:
|
|
file.write(contents.replace(find, replace))
|
|
|
|
if find in filename:
|
|
new_filename = filename.replace(find, replace)
|
|
new_filepath = os.path.join(path, new_filename)
|
|
os.rename(filepath, new_filepath)
|
|
|
|
for dirname in dirs:
|
|
if find in dirname:
|
|
new_dirname = dirname.replace(find, replace)
|
|
new_dirpath = os.path.join(path, new_dirname)
|
|
old_dirpath = os.path.join(path, dirname)
|
|
os.rename(old_dirpath, new_dirpath)
|
|
|
|
|
|
def load_env_vars(folder_path: Path) -> dict[str, Any]:
|
|
"""Loads environment variables from a .env file in the specified folder path."""
|
|
env_file_path = folder_path / ".env"
|
|
env_vars = {}
|
|
if env_file_path.exists():
|
|
with open(env_file_path, "r") as file:
|
|
for line in file:
|
|
key, _, value = line.strip().partition("=")
|
|
if key and value:
|
|
env_vars[key] = value
|
|
return env_vars
|
|
|
|
|
|
def write_env_file(folder_path: Path, env_vars: dict[str, Any]) -> None:
|
|
"""Writes environment variables to a .env file in the specified folder."""
|
|
env_file_path = folder_path / ".env"
|
|
with open(env_file_path, "w") as file:
|
|
for key, value in env_vars.items():
|
|
file.write(f"{key.upper()}={value}\n")
|