diff --git a/docs/edge/ar/telemetry.mdx b/docs/edge/ar/telemetry.mdx index f3869107b0..21d22b997a 100644 --- a/docs/edge/ar/telemetry.mdx +++ b/docs/edge/ar/telemetry.mdx @@ -34,6 +34,20 @@ os.environ['CREWAI_DISABLE_TELEMETRY'] = 'true' os.environ['OTEL_SDK_DISABLED'] = 'true' ``` +### العزل عن إعداد OpenTelemetry الخاص بك + +يعمل القياس عن بُعد الخاص بـ CrewAI على `TracerProvider` خاص به ولا يسجل نفسه +أبدًا كمزوّد عام. هذا يفصل الاتجاهين: + +- لا تُرسَل أبدًا إلى CrewAI الامتدادات (spans) الصادرة عن المكتبات الأخرى + المزوّدة بأدوات القياس في عمليتك — أطر الويب، وعملاء قواعد البيانات، + وعملاء HTTP. +- لا تُرسَل امتدادات القياس عن بُعد الخاصة بـ CrewAI إلى نظام المراقبة لديك، لذا + لن تظهر في Langfuse أو Braintrust أو Phoenix أو أي مُجمِّع آخر تقوم بإعداده. + +لا تتأثر تكاملات المراقبة: فهي تقيس CrewAI عبر مزوّد التتبع الخاص بها، وهو +مستقل عن المزوّد الموصوف هنا. + ### شرح البيانات: | افتراضي | البيانات | السبب والتفاصيل | |:----------|:------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------| diff --git a/docs/edge/en/telemetry.mdx b/docs/edge/en/telemetry.mdx index 60e7f10312..074930adeb 100644 --- a/docs/edge/en/telemetry.mdx +++ b/docs/edge/en/telemetry.mdx @@ -34,6 +34,20 @@ os.environ['CREWAI_DISABLE_TELEMETRY'] = 'true' os.environ['OTEL_SDK_DISABLED'] = 'true' ``` +### Isolation from your own OpenTelemetry setup + +CrewAI's telemetry runs on its own private `TracerProvider` and never registers +itself as the global one. This keeps the two directions separate: + +- Spans from other instrumented libraries in your process — web frameworks, + database clients, HTTP clients — are never sent to CrewAI. +- CrewAI's telemetry spans are never sent to your observability backend, so they + will not appear in Langfuse, Braintrust, Phoenix, or any other collector you + configure. + +Observability integrations are unaffected: they instrument CrewAI through their +own tracer provider, which is independent of the one described here. + ### Data Explanation: | Defaulted | Data | Reason and Specifics | |:----------|:------------------------------------------|:----------------------------------------------------------------------------------------------------------------------------| diff --git a/docs/edge/ko/telemetry.mdx b/docs/edge/ko/telemetry.mdx index 9ec9397c76..1a61d308e0 100644 --- a/docs/edge/ko/telemetry.mdx +++ b/docs/edge/ko/telemetry.mdx @@ -33,6 +33,19 @@ os.environ['CREWAI_DISABLE_TELEMETRY'] = 'true' os.environ['OTEL_SDK_DISABLED'] = 'true' ``` +### 사용자 OpenTelemetry 설정과의 격리 + +CrewAI의 telemetry는 자체 전용 `TracerProvider`에서 실행되며 자신을 전역 +provider로 등록하지 않습니다. 이를 통해 양방향이 분리됩니다: + +- 프로세스 내 다른 계측된 라이브러리(웹 프레임워크, 데이터베이스 클라이언트, + HTTP 클라이언트)의 span은 CrewAI로 전송되지 않습니다. +- CrewAI의 telemetry span은 사용자의 관측 가능성 백엔드로 전송되지 않으므로 + Langfuse, Braintrust, Phoenix 또는 구성한 다른 수집기에 나타나지 않습니다. + +관측 가능성 통합은 영향을 받지 않습니다: 해당 통합은 여기서 설명한 provider와 +독립적인 자체 tracer provider를 통해 CrewAI를 계측합니다. + ### 데이터 설명: | 기본값 | 데이터 | 사유 및 세부 사항 | |:--------|:-------------------------------------------|:----------------------------------------------------------------------------------------------------------------------| diff --git a/docs/edge/pt-BR/telemetry.mdx b/docs/edge/pt-BR/telemetry.mdx index 580d8d2131..5b7c1ea69b 100644 --- a/docs/edge/pt-BR/telemetry.mdx +++ b/docs/edge/pt-BR/telemetry.mdx @@ -34,6 +34,20 @@ os.environ['CREWAI_DISABLE_TELEMETRY'] = 'true' os.environ['OTEL_SDK_DISABLED'] = 'true' ``` +### Isolamento da sua própria configuração do OpenTelemetry + +A telemetria do CrewAI roda em seu próprio `TracerProvider` privado e nunca se +registra como o provider global. Isso mantém as duas direções separadas: + +- Spans de outras bibliotecas instrumentadas no seu processo — frameworks web, + clientes de banco de dados, clientes HTTP — nunca são enviados ao CrewAI. +- Os spans de telemetria do CrewAI nunca são enviados ao seu backend de + observabilidade, portanto não aparecerão no Langfuse, Braintrust, Phoenix ou + em qualquer outro coletor que você configurar. + +As integrações de observabilidade não são afetadas: elas instrumentam o CrewAI +por meio do próprio tracer provider, que é independente do descrito aqui. + ### Explicação dos Dados: | Padrão | Dados | Razão e Especificidades | |--------|--------------------------------------------|-----------------------------------------------------------------------------------------------------------------------------------------------| diff --git a/lib/crewai-core/src/crewai_core/runtime_env.py b/lib/crewai-core/src/crewai_core/runtime_env.py new file mode 100644 index 0000000000..eb1c7d7b74 --- /dev/null +++ b/lib/crewai-core/src/crewai_core/runtime_env.py @@ -0,0 +1,268 @@ +"""Detection of the coding assistant and runtime a process is running under. + +Lives in ``crewai_core`` rather than ``crewai`` because both telemetry +implementations need it: the CLI emits deployment, template and flow-creation +spans through ``crewai_core.telemetry`` without importing ``crewai`` at all. + +``crewai.utilities.constants`` and ``crewai.telemetry.utils`` re-export from +here, so the original import paths keep working. +""" + +from __future__ import annotations + +import os +import sys +from typing import Final + + +CC_ENV_VAR: Final[str] = "CLAUDECODE" +CODEX_ENV_VARS: Final[tuple[str, ...]] = ( + "CODEX_CI", + "CODEX_MANAGED_BY_NPM", + "CODEX_SANDBOX", + "CODEX_SANDBOX_NETWORK_DISABLED", + "CODEX_THREAD_ID", +) +CC_ENV_VARS: Final[tuple[str, ...]] = (CC_ENV_VAR, "CLAUDE_CODE") +CURSOR_ENV_VARS: Final[tuple[str, ...]] = ( + "CURSOR_AGENT", + "CURSOR_EXTENSION_HOST_ROLE", + "CURSOR_SANDBOX", + "CURSOR_TRACE_ID", + "CURSOR_WORKSPACE_LABEL", +) +ANTIGRAVITY_ENV_VARS: Final[tuple[str, ...]] = ( + "ANTIGRAVITY_AGENT", + "ANTIGRAVITY_CLI_ALIAS", +) +AUGMENT_ENV_VARS: Final[tuple[str, ...]] = ("AUGMENT_AGENT",) +CLINE_ENV_VARS: Final[tuple[str, ...]] = ("CLINE_ACTIVE",) +GEMINI_CLI_ENV_VARS: Final[tuple[str, ...]] = ("GEMINI_CLI",) +JUNIE_ENV_VARS: Final[tuple[str, ...]] = ("JUNIE_DATA", "JUNIE_SHIM_PATH") +OPENCODE_ENV_VARS: Final[tuple[str, ...]] = ("OPENCODE", "OPENCODE_CLIENT") + +# Proposed cross-vendor marker (agentsmd/agents.md#136). Checked last and +# reported as "other": it says an assistant is present without naming one, and +# reading its value to find out would put an arbitrary string in telemetry. +GENERIC_AGENT_ENV_VARS: Final[tuple[str, ...]] = ("AI_AGENT",) + +# Ordered (name, env vars) pairs for identifying the AI coding assistant a +# process is running under. Reuses the sets above and keeps the same precedence +# as ``get_env_context()``, so the env-context events and telemetry never +# disagree about which assistant is present. +# +# Deliberately limited to assistants whose markers are verified. Guessing a +# variable name is worse than omitting the assistant: a wrong name never +# matches, so that assistant is silently counted as "unknown" while the table +# implies it is covered. +# +# Two rules for adding an entry: +# 1. Confirm the variable the tool actually sets - do not infer it from the +# product name. +# 2. Use only *session*-scoped variables the assistant sets for processes it +# spawns. Persistent user configuration (an ``AIDER_MODEL`` in a committed +# ``.env``, say) is unusable: crewai loads dotenv files on normal runs, so +# a leftover config value would mislabel ordinary human executions. +# +# Extend the shared sets above rather than adding a parallel tuple here, so both +# detection paths pick the new markers up together: ``get_env_context()`` walks +# this same table for its precedence and emits ``DefaultEnvEvent`` for the +# assistants that have no event class of their own. +# +# Markers below the first three were taken from the published detection matrix +# at vercel/detect-agent (agents.json), cross-checked against the proposal in +# agentsmd/agents.md#136 and microsoft/vscode#311734. Rule 2 excluded several +# entries those sources list: Goose's ``GOOSE_PROVIDER`` and Copilot's +# ``COPILOT_MODEL`` / ``COPILOT_GITHUB_TOKEN`` are user configuration, and a +# committed ``.env`` carrying one would relabel every ordinary run. Replit's +# ``REPL_ID`` is a hosted environment rather than an assistant, so it stays in +# HOSTED_IDE_ENV_VARS. +# +# The assistants that spawn inside another editor's terminal are ordered ahead +# of Cursor for the same reason Codex is: CURSOR_* is set for every integrated +# terminal, so checking Cursor first would mask anything running inside it. +CODING_AGENT_ENV_MARKERS: Final[tuple[tuple[str, tuple[str, ...]], ...]] = ( + ("claude_code", CC_ENV_VARS), + ("codex", CODEX_ENV_VARS), + ("cline", CLINE_ENV_VARS), + ("gemini_cli", GEMINI_CLI_ENV_VARS), + ("augment", AUGMENT_ENV_VARS), + ("opencode", OPENCODE_ENV_VARS), + ("antigravity", ANTIGRAVITY_ENV_VARS), + ("junie", JUNIE_ENV_VARS), + ("cursor", CURSOR_ENV_VARS), +) + +# Markers for *where* a process runs, kept separate from which assistant is +# driving it. The two answer different questions: a scheduled container run has +# no assistant to detect, and folding it into the assistant field made "no +# marker found" and "no assistant possible" indistinguishable. +# +# These are published platform contracts rather than per-tool observations, so +# they do not need the case-by-case verification the assistant table requires. +# Presence is all that is checked; no value is ever read. +CI_ENV_VARS: Final[tuple[str, ...]] = ( + "APPVEYOR", + "BITBUCKET_BUILD_NUMBER", + "BUILDKITE", + "CI", + "CIRCLECI", + "DRONE", + "GITHUB_ACTIONS", + "GITLAB_CI", + "JENKINS_URL", + "TEAMCITY_VERSION", + "TF_BUILD", + "TRAVIS", +) +SERVERLESS_ENV_VARS: Final[tuple[str, ...]] = ( + "AWS_LAMBDA_FUNCTION_NAME", + "FUNCTIONS_EXTENSION_VERSION", + "FUNCTIONS_WORKER_RUNTIME", + "FUNCTION_TARGET", + "K_SERVICE", + "VERCEL", +) +# Managed application platforms, kept apart from serverless: their markers are +# set for long-lived containers rather than per-invocation functions, and +# checking them under "serverless" would have claimed every Heroku dyno and +# Azure App Service instance before the container check could see them. +# +# Azure Functions run on the App Service host and inherit WEBSITE_INSTANCE_ID, +# so they would land here despite being serverless. The FUNCTIONS_* markers +# above are checked first to keep them out. +PAAS_ENV_VARS: Final[tuple[str, ...]] = ( + "DYNO", + "WEBSITE_INSTANCE_ID", +) +HOSTED_IDE_ENV_VARS: Final[tuple[str, ...]] = ( + "CODESPACES", + "GITPOD_WORKSPACE_ID", + "REPL_ID", +) +NOTEBOOK_ENV_VARS: Final[tuple[str, ...]] = ( + "COLAB_RELEASE_TAG", + "JPY_PARENT_PID", +) +CONTAINER_ENV_VARS: Final[tuple[str, ...]] = ("KUBERNETES_SERVICE_HOST",) + +# Ordered most specific first. CI jobs and hosted IDEs usually run inside +# containers, so a bare container match is only meaningful once the others have +# been ruled out. +RUNTIME_CONTEXT_ENV_MARKERS: Final[tuple[tuple[str, tuple[str, ...]], ...]] = ( + ("ci", CI_ENV_VARS), + ("serverless", SERVERLESS_ENV_VARS), + ("paas", PAAS_ENV_VARS), + ("hosted_ide", HOSTED_IDE_ENV_VARS), + ("notebook", NOTEBOOK_ENV_VARS), + ("container", CONTAINER_ENV_VARS), +) + + +# Editors whose integrated terminal a process can be launched from. Matched on +# an exact value rather than presence, since these variables name the terminal. +_EDITOR_TERM_MARKERS: Final[tuple[tuple[str, str, str], ...]] = ( + ("TERM_PROGRAM", "vscode", "vscode_terminal"), + ("TERMINAL_EMULATOR", "JetBrains-JediTerm", "jetbrains_terminal"), +) + +# Marks a process running inside a container when no more specific runtime +# marker is present. +_DOCKER_ENV_PATH: Final[str] = "/.dockerenv" + +_UNKNOWN: Final[str] = "unknown" +_OTHER: Final[str] = "other" + +# The complete set of values detect_coding_agent() can ever return. Every value +# is a literal from CODING_AGENT_ENV_MARKERS or this module, which is what makes +# the function structurally incapable of emitting PII: no environment value, +# path, hostname, or user-supplied string can reach the return value. +KNOWN_CODING_AGENTS: Final[frozenset[str]] = frozenset( + [name for name, _ in CODING_AGENT_ENV_MARKERS] + [_OTHER, _UNKNOWN] +) + +# The same guarantee for detect_runtime_context(): a closed set of literals. +KNOWN_RUNTIME_CONTEXTS: Final[frozenset[str]] = frozenset( + [name for name, _ in RUNTIME_CONTEXT_ENV_MARKERS] + + [name for _, _, name in _EDITOR_TERM_MARKERS] + + ["interactive", "non_interactive", _UNKNOWN] +) + + +def detect_coding_agent() -> str: + """Best-effort detection of the AI coding assistant running this process. + + Uses the shared ``CODING_AGENT_ENV_MARKERS`` table, so this agrees with the + env-context events emitted by ``get_env_context()`` rather than maintaining + a second, narrower set of markers. Precedence follows that table, which ends + with Cursor: ``CURSOR_*`` is set for every integrated terminal, so checking + it earlier would mask any assistant running inside one. + + Only the assistant's normalized name is returned - environment variable + values are never read into the return value or recorded anywhere. + + Two limits worth knowing. This is heuristic: markers change as tools + evolve, so "unknown" means "no known marker present", not "no agent". And + some markers (the Cursor set in particular) are set by the editor for any + integrated terminal, so a result names the environment the process is + running *under*, not proof that an agent authored the code. + + Answers only *which assistant*. Where the process runs is + :func:`detect_runtime_context`, so an "unknown" here is a genuine gap in + the marker table rather than a run that never had an assistant to find. + + Returns: + A normalized assistant name (e.g. "claude_code", "cursor", "codex"), or + "unknown" when no marker matches. The result is always a member of + KNOWN_CODING_AGENTS. + """ + for agent_name, env_vars in CODING_AGENT_ENV_MARKERS: + if any(os.environ.get(env_var) for env_var in env_vars): + return agent_name + + # Checked last and by presence: the cross-vendor marker establishes that an + # assistant is present without naming one, and an empty value still says so. + if any(env_var in os.environ for env_var in GENERIC_AGENT_ENV_VARS): + return _OTHER + + return _UNKNOWN + + +def detect_runtime_context() -> str: + """Best-effort detection of where this process is running. + + Separate from :func:`detect_coding_agent` because the two answer different + questions. Automated runs -- CI, containers, serverless -- have no + assistant to detect, and reporting them in the assistant field made an + unrecognized assistant indistinguishable from a run that could never have + had one. + + Precedence runs most specific first: CI and hosted IDEs typically run + inside containers, so a bare container match only applies once the more + specific markers have been ruled out. + + Returns: + One of the runtime names (e.g. "ci", "serverless", "container", + "notebook", "hosted_ide"), an editor terminal (e.g. "vscode_terminal"), + "interactive" or "non_interactive" when only a TTY check applies, or + "unknown" when that check cannot be made. The result is always a member + of KNOWN_RUNTIME_CONTEXTS. + """ + # Presence, not truthiness: a platform that exports an empty CI= is still + # CI, unlike the assistant markers where an empty value means the tool set + # a placeholder rather than claiming the session. + for context_name, env_vars in RUNTIME_CONTEXT_ENV_MARKERS: + if any(env_var in os.environ for env_var in env_vars): + return context_name + + for env_var, expected, context_name in _EDITOR_TERM_MARKERS: + if os.environ.get(env_var) == expected: + return context_name + + if os.path.exists(_DOCKER_ENV_PATH): + return "container" + + try: + return "interactive" if sys.stdout.isatty() else "non_interactive" + except (AttributeError, ValueError, OSError): + return _UNKNOWN diff --git a/lib/crewai-core/src/crewai_core/telemetry.py b/lib/crewai-core/src/crewai_core/telemetry.py index 18476fcf55..702d8a52f7 100644 --- a/lib/crewai-core/src/crewai_core/telemetry.py +++ b/lib/crewai-core/src/crewai_core/telemetry.py @@ -14,22 +14,26 @@ import asyncio import atexit from collections.abc import Callable import contextlib +from functools import cache import logging import os import threading from typing import Any, ClassVar, Final -from opentelemetry import trace +from opentelemetry.context import Context from opentelemetry.exporter.otlp.proto.http.trace_exporter import OTLPSpanExporter from opentelemetry.sdk.resources import SERVICE_NAME, Resource -from opentelemetry.sdk.trace import TracerProvider +from opentelemetry.sdk.trace import SpanProcessor, TracerProvider from opentelemetry.sdk.trace.export import ( BatchSpanProcessor, SpanExportResult, ) -from opentelemetry.trace import ProxyTracerProvider, Span, Status, StatusCode +from opentelemetry.trace import Span, Status, StatusCode from typing_extensions import Self +from crewai_core.project import get_project_id +from crewai_core.runtime_env import detect_coding_agent, detect_runtime_context + logger = logging.getLogger(__name__) @@ -37,6 +41,8 @@ logger = logging.getLogger(__name__) CREWAI_TELEMETRY_BASE_URL: Final[str] = "https://telemetry.crewai.com:4319" CREWAI_TELEMETRY_SERVICE_NAME: Final[str] = "crewAI-telemetry" +TRACER_NAME: Final[str] = "crewai.telemetry" + def close_span(span: Span) -> None: """Set span status to OK and end it.""" @@ -65,6 +71,89 @@ class SafeOTLPSpanExporter(OTLPSpanExporter): return SpanExportResult.FAILURE +class CommonAttributesSpanProcessor(SpanProcessor): + """Applies a fixed set of attributes to every span at start. + + Used for process-wide context that should appear on all spans (e.g. which + AI coding assistant is running the process) without each span-emitting + method having to set it. Attributes are applied as span attributes rather + than Resource attributes because the ingestion pipeline preserves only + serviceName from the resource. + """ + + def __init__(self, attributes: dict[str, str]) -> None: + """Initialize the processor. + + Args: + attributes: Attributes applied to every span. Values must not + contain user data - this is process-wide context only. + """ + self._attributes = attributes + + def on_start(self, span: Span, parent_context: Context | None = None) -> None: + """Apply the common attributes to a span as it starts. + + Args: + span: The span being started. + parent_context: Parent context, unused. + """ + try: + span.set_attributes(self._attributes) + except Exception: # noqa: S110 - telemetry must never break execution + pass + + def on_end(self, span: Any) -> None: + """No-op; export is handled by the batch processor.""" + + def shutdown(self) -> None: + """No-op; this processor holds no resources.""" + + def force_flush(self, timeout_millis: int = 30000) -> bool: + """No-op flush. + + Args: + timeout_millis: Unused. + + Returns: + Always True. + """ + return True + + +@cache +def common_span_attributes() -> dict[str, str]: + """Build the attributes every span carries, once per process. + + Cached because it reads the project's ``pyproject.toml``, and because a + process cannot change which assistant, runtime, or project it belongs to + partway through. + + Shared by both telemetry implementations so a CLI-only process, which never + imports ``crewai``, still reports the same process-wide context. + + Returns: + Attributes to stamp on every span. ``project_id`` is omitted for + projects that do not declare one. + """ + attributes = { + "coding_agent": detect_coding_agent(), + "runtime_context": detect_runtime_context(), + } + + try: + # Read-only: minting an id belongs to the CLI commands a user + # invoked, not to a library call during execution. + project_id = get_project_id() + except Exception as e: # Telemetry must never break execution. + logger.debug("Failed to read project id: %s", e) + project_id = None + + if project_id: + attributes["project_id"] = project_id + + return attributes + + class Telemetry: """Base telemetry: OTLP setup + the spans needed by the CLI. @@ -101,6 +190,13 @@ class Telemetry: with suppress_warnings(): self.provider = TracerProvider(resource=self.resource) + # Span attributes, not Resource attributes: the ingestion pipeline + # preserves only serviceName from the resource, so anything else set + # there is dropped before it reaches storage. + self.provider.add_span_processor( + CommonAttributesSpanProcessor(common_span_attributes()) + ) + processor = BatchSpanProcessor( SafeOTLPSpanExporter( endpoint=f"{CREWAI_TELEMETRY_BASE_URL}/v1/traces", @@ -145,20 +241,21 @@ class Telemetry: logger.debug("Telemetry shutdown failed: %s", e) def set_tracer(self) -> None: - """Install our TracerProvider as the global one (idempotent).""" - if self.ready and not self.trace_set: - try: - with suppress_warnings(): - existing_provider = trace.get_tracer_provider() - if not isinstance(existing_provider, ProxyTracerProvider): - self.trace_set = True - return - trace.set_tracer_provider(self.provider) - self.trace_set = True - except Exception as e: - logger.debug("Failed to set tracer provider: %s", e) - self.ready = False - self.trace_set = False + """Mark telemetry live (idempotent). + + Deliberately does not install a global TracerProvider. Doing so made + every OTel-instrumented library in the host process - HTTP servers, + Redis clients, ORMs - resolve ``trace.get_tracer()`` to our provider + and export to our collector. It also meant that when an application + had already installed its own provider, our spans went to *their* + collector instead of ours. Spans are now created from ``self.provider`` + directly, so neither can happen. + + Retained rather than removed because it is called across packages + (``crewai_cli.command``, ``crewai_cli.crew_run_tui``) and by the + ``crewai`` event listener. + """ + self.trace_set = self.ready def _safe_telemetry_operation( self, operation: Callable[[], Span | None] @@ -194,7 +291,7 @@ class Telemetry: """Records when an error occurs during the deployment signup process.""" def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Deploy Signup Error") close_span(span) @@ -204,7 +301,7 @@ class Telemetry: """Records the start of a deployment process.""" def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Start Deployment") if uuid: self._add_attribute(span, "uuid", uuid) @@ -216,7 +313,7 @@ class Telemetry: """Records the creation of a new crew deployment.""" def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Create Crew Deployment") close_span(span) @@ -228,7 +325,7 @@ class Telemetry: """Records the retrieval of crew logs.""" def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Get Crew Logs") self._add_attribute(span, "log_type", log_type) if uuid: @@ -241,7 +338,7 @@ class Telemetry: """Records the removal of a crew.""" def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Remove Crew") if uuid: self._add_attribute(span, "uuid", uuid) @@ -254,7 +351,7 @@ class Telemetry: from crewai_core.version import get_crewai_version def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Feature Usage") self._add_attribute(span, "crewai_version", get_crewai_version()) self._add_attribute(span, "feature", feature) @@ -267,7 +364,7 @@ class Telemetry: from crewai_core.version import get_crewai_version def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Flow Creation") self._add_attribute(span, "crewai_version", get_crewai_version()) self._add_attribute(span, "flow_name", flow_name) @@ -280,7 +377,7 @@ class Telemetry: from crewai_core.version import get_crewai_version def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Template Installed") self._add_attribute(span, "crewai_version", get_crewai_version()) self._add_attribute(span, "template_name", template_name) diff --git a/lib/crewai-core/tests/test_runtime_env.py b/lib/crewai-core/tests/test_runtime_env.py new file mode 100644 index 0000000000..08cd547424 --- /dev/null +++ b/lib/crewai-core/tests/test_runtime_env.py @@ -0,0 +1,137 @@ +"""crewai-core must report the same process context as crewai. + +The CLI emits deployment, template and flow-creation spans through +``crewai_core.telemetry`` in processes that never import ``crewai``. Those spans +previously carried ``coding_agent``/``runtime_context`` only by accident - they +rode the global TracerProvider that ``crewai`` installed at import - so a +CLI-only process reported neither. + +The detection and common-attribute behaviour itself is covered against the +shared implementation in ``lib/crewai/tests/telemetry/``; only what is specific +to crewai-core standing alone is tested here. +""" + +from __future__ import annotations + +from collections.abc import Iterator + +from crewai_core.runtime_env import ( + CODING_AGENT_ENV_MARKERS, + GENERIC_AGENT_ENV_VARS, + RUNTIME_CONTEXT_ENV_MARKERS, + detect_coding_agent, + detect_runtime_context, +) +from crewai_core.telemetry import Telemetry, common_span_attributes +from opentelemetry.sdk.trace.export import SpanExportResult +import pytest + + +class _NullExporter: + """Stands in for the OTLP exporter so no test attempts a real export.""" + + def export(self, spans: object) -> SpanExportResult: + return SpanExportResult.SUCCESS + + def shutdown(self) -> None: + pass + + def force_flush(self, timeout_millis: int = 30000) -> bool: + return True + + +@pytest.fixture(autouse=True) +def _reset_cache() -> Iterator[None]: + common_span_attributes.cache_clear() + yield + common_span_attributes.cache_clear() + + +@pytest.fixture +def clean_env(monkeypatch: pytest.MonkeyPatch) -> pytest.MonkeyPatch: + """Remove every marker the detectors read, so results are deterministic.""" + for _, env_vars in (*CODING_AGENT_ENV_MARKERS, *RUNTIME_CONTEXT_ENV_MARKERS): + for var in env_vars: + monkeypatch.delenv(var, raising=False) + for var in (*GENERIC_AGENT_ENV_VARS, "TERM_PROGRAM", "TERMINAL_EMULATOR"): + monkeypatch.delenv(var, raising=False) + return monkeypatch + + +def test_detection_works_from_crewai_core(clean_env: pytest.MonkeyPatch) -> None: + """The detectors resolve here, not only through the crewai re-export.""" + clean_env.setenv("CLAUDECODE", "1") + clean_env.setenv("GITHUB_ACTIONS", "true") + + assert detect_coding_agent() == "claude_code" + assert detect_runtime_context() == "ci" + + +def test_runtime_env_does_not_import_crewai() -> None: + """crewai-core is the leaf package; importing crewai here would invert it. + + Asserted on the module's imports rather than on ``sys.modules``, which + reflects whatever else the test session has already loaded and would make + this depend on ordering. + """ + import ast + import inspect + + from crewai_core import runtime_env + + tree = ast.parse(inspect.getsource(runtime_env)) + imported: set[str] = set() + for node in ast.walk(tree): + if isinstance(node, ast.Import): + imported |= {alias.name for alias in node.names} + elif isinstance(node, ast.ImportFrom) and node.module: + imported.add(node.module) + + assert not [ + module + for module in imported + if module == "crewai" or module.startswith("crewai.") + ] + + +def test_cli_spans_carry_the_process_context( + clean_env: pytest.MonkeyPatch, monkeypatch: pytest.MonkeyPatch +) -> None: + """The regression: CLI spans must carry coding_agent and runtime_context.""" + from opentelemetry.sdk.trace.export import SimpleSpanProcessor + from opentelemetry.sdk.trace.export.in_memory_span_exporter import ( + InMemorySpanExporter, + ) + + clean_env.setenv("CLAUDECODE", "1") + clean_env.setenv("GITHUB_ACTIONS", "true") + monkeypatch.delenv("OTEL_SDK_DISABLED", raising=False) + monkeypatch.delenv("CREWAI_DISABLE_TELEMETRY", raising=False) + monkeypatch.delenv("CREWAI_DISABLE_TRACKING", raising=False) + + Telemetry._instance = None + monkeypatch.setattr(Telemetry, "_register_shutdown_handlers", lambda self: None) + # Patched before construction: __init__ wires the real OTLP exporter. + monkeypatch.setattr( + "crewai_core.telemetry.SafeOTLPSpanExporter", + lambda **_kwargs: _NullExporter(), + ) + telemetry = Telemetry() + + exporter = InMemorySpanExporter() + telemetry.provider.add_span_processor(SimpleSpanProcessor(exporter)) + + try: + telemetry.feature_usage_span("cli_usage:deploy") + telemetry.template_installed_span("crew-template") + finally: + telemetry.provider.shutdown() + Telemetry._instance = None + + exported = exporter.get_finished_spans() + assert [span.name for span in exported] == ["Feature Usage", "Template Installed"] + for span in exported: + attributes = span.attributes + assert attributes is not None + assert attributes["coding_agent"] == "claude_code" + assert attributes["runtime_context"] == "ci" diff --git a/lib/crewai-core/tests/test_smoke.py b/lib/crewai-core/tests/test_smoke.py index 5731018475..f08a3df7f0 100644 --- a/lib/crewai-core/tests/test_smoke.py +++ b/lib/crewai-core/tests/test_smoke.py @@ -15,7 +15,6 @@ from crewai_core import ( user_data, version, ) -from opentelemetry.sdk.trace import TracerProvider import pytest @@ -176,36 +175,28 @@ def test_configured_project_definition_rejects_empty_definition( ) -def test_core_telemetry_skips_duplicate_tracer_provider( +def test_core_telemetry_never_installs_a_global_provider( monkeypatch: pytest.MonkeyPatch, ) -> None: + """Telemetry must leave the process-wide TracerProvider untouched. + + Installing it globally routed every OTel-instrumented library in the host + process - HTTP servers, Redis clients, ORMs - to CrewAI's collector. + """ from crewai_core.telemetry import Telemetry + import opentelemetry.trace as ot Telemetry._instance = None monkeypatch.delenv("OTEL_SDK_DISABLED", raising=False) monkeypatch.delenv("CREWAI_DISABLE_TELEMETRY", raising=False) monkeypatch.delenv("CREWAI_DISABLE_TRACKING", raising=False) - monkeypatch.setattr( - "crewai_core.telemetry.trace.get_tracer_provider", - lambda: TracerProvider(), - ) - - called = False - - def fail_if_called(provider: object) -> None: - nonlocal called - called = True - - monkeypatch.setattr( - "crewai_core.telemetry.trace.set_tracer_provider", - fail_if_called, - ) - + before = ot.get_tracer_provider() telemetry = Telemetry() telemetry.set_tracer() + after = ot.get_tracer_provider() - assert called is False + assert after is before assert telemetry.trace_set is True @@ -223,8 +214,8 @@ def test_core_telemetry_records_feature_usage( span = Mock() tracer.start_span.return_value = span monkeypatch.setattr( - "crewai_core.telemetry.trace.get_tracer", - lambda _name: tracer, + "crewai_core.telemetry.TracerProvider", + lambda **_kwargs: Mock(get_tracer=Mock(return_value=tracer)), ) telemetry = Telemetry() @@ -250,8 +241,8 @@ def test_core_telemetry_records_flow_creation_version( span = Mock() tracer.start_span.return_value = span monkeypatch.setattr( - "crewai_core.telemetry.trace.get_tracer", - lambda _name: tracer, + "crewai_core.telemetry.TracerProvider", + lambda **_kwargs: Mock(get_tracer=Mock(return_value=tracer)), ) telemetry = Telemetry() diff --git a/lib/crewai/src/crewai/telemetry/constants.py b/lib/crewai/src/crewai/telemetry/constants.py index 24ac608c9a..4082136b83 100644 --- a/lib/crewai/src/crewai/telemetry/constants.py +++ b/lib/crewai/src/crewai/telemetry/constants.py @@ -8,3 +8,5 @@ from typing import Final CREWAI_TELEMETRY_BASE_URL: Final[str] = "https://telemetry.crewai.com:4319" CREWAI_TELEMETRY_SERVICE_NAME: Final[str] = "crewAI-telemetry" + +TRACER_NAME: Final[str] = "crewai.telemetry" diff --git a/lib/crewai/src/crewai/telemetry/telemetry.py b/lib/crewai/src/crewai/telemetry/telemetry.py index 5b91c5af97..ea2052acbc 100644 --- a/lib/crewai/src/crewai/telemetry/telemetry.py +++ b/lib/crewai/src/crewai/telemetry/telemetry.py @@ -19,21 +19,21 @@ import platform import signal import threading from typing import TYPE_CHECKING, Any -import weakref -from crewai_core.project import get_project_id -from opentelemetry import trace -from opentelemetry.context import Context +from crewai_core.telemetry import ( + CommonAttributesSpanProcessor, + common_span_attributes, +) from opentelemetry.exporter.otlp.proto.http.trace_exporter import ( OTLPSpanExporter, ) from opentelemetry.sdk.resources import SERVICE_NAME, Resource -from opentelemetry.sdk.trace import SpanProcessor, TracerProvider +from opentelemetry.sdk.trace import TracerProvider from opentelemetry.sdk.trace.export import ( BatchSpanProcessor, SpanExportResult, ) -from opentelemetry.trace import ProxyTracerProvider, Span +from opentelemetry.trace import Span from typing_extensions import Self from crewai.events.event_bus import crewai_event_bus @@ -48,14 +48,13 @@ from crewai.llms.base_llm import BaseLLM from crewai.telemetry.constants import ( CREWAI_TELEMETRY_BASE_URL, CREWAI_TELEMETRY_SERVICE_NAME, + TRACER_NAME, ) from crewai.telemetry.utils import ( add_agent_fingerprint_to_span, add_crew_and_task_attributes, add_crew_attributes, close_span, - detect_coding_agent, - detect_runtime_context, ) from crewai.utilities.i18n import I18N_DEFAULT from crewai.utilities.logger_utils import suppress_warnings @@ -92,55 +91,6 @@ class SafeOTLPSpanExporter(OTLPSpanExporter): return SpanExportResult.FAILURE -class CommonAttributesSpanProcessor(SpanProcessor): - """Applies a fixed set of attributes to every span at start. - - Used for process-wide context that should appear on all spans (e.g. which - AI coding assistant is running the process) without each span-emitting - method having to set it. Attributes are applied as span attributes rather - than Resource attributes because the ingestion pipeline preserves only - serviceName from the resource. - """ - - def __init__(self, attributes: dict[str, str]) -> None: - """Initialize the processor. - - Args: - attributes: Attributes applied to every span. Values must not - contain user data - this is process-wide context only. - """ - self._attributes = attributes - - def on_start(self, span: Span, parent_context: Context | None = None) -> None: - """Apply the common attributes to a span as it starts. - - Args: - span: The span being started. - parent_context: Parent context, unused. - """ - try: - span.set_attributes(self._attributes) - except Exception: # noqa: S110 - telemetry must never break execution - pass - - def on_end(self, span: Any) -> None: - """No-op; export is handled by the batch processor.""" - - def shutdown(self) -> None: - """No-op; this processor holds no resources.""" - - def force_flush(self, timeout_millis: int = 30000) -> bool: - """No-op flush. - - Args: - timeout_millis: Unused. - - Returns: - Always True. - """ - return True - - class Telemetry: """Handle anonymous telemetry for the CrewAI package. @@ -171,10 +121,6 @@ class Telemetry: self._initialized: bool = True self._coding_agent_reported: bool = False self._coding_agent_lock = threading.Lock() - # Weak so instrumented apps' providers are not kept alive by telemetry. - self._common_attributes_providers: weakref.WeakSet[Any] = weakref.WeakSet() - self._common_attributes_lock = threading.Lock() - self._common_attributes: dict[str, str] | None = None if self._is_telemetry_disabled(): return @@ -186,7 +132,12 @@ class Telemetry: with suppress_warnings(): self.provider = TracerProvider(resource=self.resource) - self._attach_common_attributes(self.provider) + # Span attributes, not Resource attributes: the ingestion pipeline + # preserves only serviceName from the resource, so anything else set + # there is dropped before it reaches storage. + self.provider.add_span_processor( + CommonAttributesSpanProcessor(common_span_attributes()) + ) processor = BatchSpanProcessor( SafeOTLPSpanExporter( @@ -206,72 +157,6 @@ class Telemetry: raise self.ready = False - def _common_span_attributes(self) -> dict[str, str]: - """Build the attributes every span carries, once per process. - - Memoized because it is computed per provider and reads the project's - ``pyproject.toml``, and because a process cannot change which - assistant, runtime, or project it belongs to partway through. - - Returns: - Attributes to stamp on every span. ``project_id`` is omitted for - projects that do not declare one. - """ - if self._common_attributes is not None: - return self._common_attributes - - attributes = { - "coding_agent": detect_coding_agent(), - "runtime_context": detect_runtime_context(), - } - - try: - # Read-only: minting an id belongs to the CLI commands a user - # invoked, not to a library call during execution. - project_id = get_project_id() - except Exception as e: # Telemetry must never break execution. - logger.debug(f"Failed to read project id: {e}") - project_id = None - - if project_id: - attributes["project_id"] = project_id - - self._common_attributes = attributes - return attributes - - def _attach_common_attributes(self, provider: Any) -> None: - """Attach process-wide attributes to every span a provider emits. - - Applied as *span* attributes rather than Resource attributes: the - ingestion pipeline preserves only serviceName from the resource, so - anything else set there is dropped before it reaches storage. - - Tracked per provider rather than once globally: our own provider and an - application's pre-installed provider both need the processor, but - neither should receive it twice. - - Args: - provider: Tracer provider to attach the processor to. Ignored if it - does not accept span processors (e.g. a NoOp provider). - """ - add_span_processor = getattr(provider, "add_span_processor", None) - if add_span_processor is None: - return - - try: - # Locked: check-then-act. Crews and flows created from different - # threads can both reach set_tracer() before trace_set flips, and - # would otherwise each attach a processor to the same provider. - with self._common_attributes_lock: - if provider in self._common_attributes_providers: - return - add_span_processor( - CommonAttributesSpanProcessor(self._common_span_attributes()) - ) - self._common_attributes_providers.add(provider) - except Exception as e: # Telemetry must never break execution. - logger.debug(f"Failed to attach common span attributes: {e}") - @classmethod def _is_telemetry_disabled(cls) -> bool: """Check if telemetry should be disabled based on environment variables.""" @@ -286,25 +171,24 @@ class Telemetry: return self.ready and not self._is_telemetry_disabled() def set_tracer(self) -> None: - """Set the tracer provider if ready and not already set.""" - if self.ready and not self.trace_set: - try: - with suppress_warnings(): - existing_provider = trace.get_tracer_provider() - if not isinstance(existing_provider, ProxyTracerProvider): - # An application installed its own provider, so our - # spans are created by theirs. Attach the common - # attributes there too, otherwise every span emitted in - # an instrumented app would silently lose coding_agent. - self._attach_common_attributes(existing_provider) - self.trace_set = True - return - trace.set_tracer_provider(self.provider) - self.trace_set = True - except Exception as e: - logger.debug(f"Failed to set tracer provider: {e}") - self.ready = False - self.trace_set = False + """Mark telemetry live (idempotent). + + Deliberately does not install a global TracerProvider. Doing so made + every OTel-instrumented library in the host process - HTTP servers, + Redis clients, ORMs - resolve ``trace.get_tracer()`` to our provider + and export to our collector, which is neither data we asked for nor + data a user consented to send. It also meant that when an application + had already installed its own provider, our spans were created by + theirs and went to *their* collector instead of ours. + + Spans are now created from ``self.provider`` directly, so neither can + happen, and the common attributes only ever land on our own spans. + + Retained rather than removed because it is called across packages + (``crewai_cli.command``, ``crewai_cli.crew_run_tui``) and by + ``crewai.events.event_listener``. + """ + self.trace_set = self.ready def _register_shutdown_handlers(self) -> None: """Register handlers for graceful shutdown on process exit and signals.""" @@ -408,7 +292,7 @@ class Telemetry: """ def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Crew Created") self._add_attribute( span, @@ -621,7 +505,7 @@ class Telemetry: """ def _operation() -> Span: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) created_span = tracer.start_span("Task Created") @@ -715,7 +599,7 @@ class Telemetry: """ def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Tool Repeated Usage") self._add_attribute( span, @@ -743,7 +627,7 @@ class Telemetry: """ def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Tool Usage") self._add_attribute( span, @@ -772,7 +656,7 @@ class Telemetry: """ def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Tool Usage Error") self._add_attribute( span, @@ -803,7 +687,7 @@ class Telemetry: """ def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Crew Individual Test Result") self._add_attribute( @@ -838,7 +722,7 @@ class Telemetry: """ def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Crew Test Execution") self._add_attribute( @@ -863,7 +747,7 @@ class Telemetry: """Records when an error occurs during the deployment signup process.""" def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Deploy Signup Error") close_span(span) @@ -877,7 +761,7 @@ class Telemetry: """ def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Start Deployment") if uuid: self._add_attribute(span, "uuid", uuid) @@ -889,7 +773,7 @@ class Telemetry: """Records the creation of a new crew deployment.""" def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Create Crew Deployment") close_span(span) @@ -906,7 +790,7 @@ class Telemetry: """ def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Get Crew Logs") self._add_attribute(span, "log_type", log_type) if uuid: @@ -923,7 +807,7 @@ class Telemetry: """ def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Remove Crew") if uuid: self._add_attribute(span, "uuid", uuid) @@ -948,7 +832,7 @@ class Telemetry: self.crew_creation(crew, inputs) def _operation() -> Span: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Crew Execution") self._add_attribute( span, @@ -1081,7 +965,7 @@ class Telemetry: """ def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Flow Creation") self._add_attribute(span, "crewai_version", version("crewai")) self._add_attribute(span, "flow_name", flow_name) @@ -1099,7 +983,7 @@ class Telemetry: """ def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Flow Plotting") self._add_attribute(span, "flow_name", flow_name) self._add_attribute(span, "node_names", json.dumps(node_names)) @@ -1116,7 +1000,7 @@ class Telemetry: """ def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Flow Execution") self._add_attribute( span, @@ -1133,7 +1017,7 @@ class Telemetry: """Records the coding tool environment context.""" def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Environment Context") self._add_attribute( span, @@ -1164,7 +1048,7 @@ class Telemetry: """ def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Human Feedback") self._add_attribute(span, "event_type", event_type) self._add_attribute(span, "has_routing", has_routing) @@ -1187,7 +1071,7 @@ class Telemetry: """ def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Feature Usage") self._add_attribute(span, "crewai_version", version("crewai")) self._add_attribute(span, "feature", feature) @@ -1216,13 +1100,19 @@ class Telemetry: Emitted at most once per process as a feature usage event, so it lands in the existing feature-usage aggregation as "coding_agent:". Only the assistant's name is recorded - never any environment values. + + Read from the shared common attributes rather than detecting again, so + the feature event and the ``coding_agent`` span attribute cannot + disagree. """ with self._coding_agent_lock: if self._coding_agent_reported: return self._coding_agent_reported = True - self.feature_usage_span(f"coding_agent:{detect_coding_agent()}") + self.feature_usage_span( + f"coding_agent:{common_span_attributes()['coding_agent']}" + ) def template_installed_span(self, template_name: str) -> None: """Records when a template is downloaded and installed. @@ -1233,7 +1123,7 @@ class Telemetry: """ def _operation() -> None: - tracer = trace.get_tracer("crewai.telemetry") + tracer = self.provider.get_tracer(TRACER_NAME) span = tracer.start_span("Template Installed") self._add_attribute(span, "crewai_version", version("crewai")) self._add_attribute(span, "template_name", template_name) diff --git a/lib/crewai/src/crewai/telemetry/utils.py b/lib/crewai/src/crewai/telemetry/utils.py index ec3cafa70b..2782c2f96b 100644 --- a/lib/crewai/src/crewai/telemetry/utils.py +++ b/lib/crewai/src/crewai/telemetry/utils.py @@ -1,22 +1,24 @@ """Telemetry utility functions. This module provides utility functions for telemetry operations. + +Assistant and runtime detection moved to ``crewai_core.runtime_env`` so the CLI +can report the same context without importing ``crewai``; the four names are +re-exported here because this is their established import path. """ from __future__ import annotations from collections.abc import Callable -import os -import sys -from typing import TYPE_CHECKING, Any, Final +from typing import TYPE_CHECKING, Any -from opentelemetry.trace import Span, Status, StatusCode - -from crewai.utilities.constants import ( - CODING_AGENT_ENV_MARKERS, - GENERIC_AGENT_ENV_VARS, - RUNTIME_CONTEXT_ENV_MARKERS, +from crewai_core.runtime_env import ( + KNOWN_CODING_AGENTS as KNOWN_CODING_AGENTS, + KNOWN_RUNTIME_CONTEXTS as KNOWN_RUNTIME_CONTEXTS, + detect_coding_agent as detect_coding_agent, + detect_runtime_context as detect_runtime_context, ) +from opentelemetry.trace import Span, Status, StatusCode if TYPE_CHECKING: @@ -24,114 +26,6 @@ if TYPE_CHECKING: from crewai.task import Task -# Editors whose integrated terminal a process can be launched from. Matched on -# an exact value rather than presence, since these variables name the terminal. -_EDITOR_TERM_MARKERS: Final[tuple[tuple[str, str, str], ...]] = ( - ("TERM_PROGRAM", "vscode", "vscode_terminal"), - ("TERMINAL_EMULATOR", "JetBrains-JediTerm", "jetbrains_terminal"), -) - -# Marks a process running inside a container when no more specific runtime -# marker is present. -_DOCKER_ENV_PATH: Final[str] = "/.dockerenv" - -_UNKNOWN: Final[str] = "unknown" -_OTHER: Final[str] = "other" - -# The complete set of values detect_coding_agent() can ever return. Every value -# is a literal from CODING_AGENT_ENV_MARKERS or this module, which is what makes -# the function structurally incapable of emitting PII: no environment value, -# path, hostname, or user-supplied string can reach the return value. -KNOWN_CODING_AGENTS: Final[frozenset[str]] = frozenset( - [name for name, _ in CODING_AGENT_ENV_MARKERS] + [_OTHER, _UNKNOWN] -) - -# The same guarantee for detect_runtime_context(): a closed set of literals. -KNOWN_RUNTIME_CONTEXTS: Final[frozenset[str]] = frozenset( - [name for name, _ in RUNTIME_CONTEXT_ENV_MARKERS] - + [name for _, _, name in _EDITOR_TERM_MARKERS] - + ["interactive", "non_interactive", _UNKNOWN] -) - - -def detect_coding_agent() -> str: - """Best-effort detection of the AI coding assistant running this process. - - Uses the shared ``CODING_AGENT_ENV_MARKERS`` table, so this agrees with the - env-context events emitted by ``get_env_context()`` rather than maintaining - a second, narrower set of markers. Precedence follows that table: Claude - Code, then Codex, then Cursor, then the remaining assistants. - - Only the assistant's normalized name is returned - environment variable - values are never read into the return value or recorded anywhere. - - Two limits worth knowing. This is heuristic: markers change as tools - evolve, so "unknown" means "no known marker present", not "no agent". And - some markers (the Cursor set in particular) are set by the editor for any - integrated terminal, so a result names the environment the process is - running *under*, not proof that an agent authored the code. - - Answers only *which assistant*. Where the process runs is - :func:`detect_runtime_context`, so an "unknown" here is a genuine gap in - the marker table rather than a run that never had an assistant to find. - - Returns: - A normalized assistant name (e.g. "claude_code", "cursor", "codex"), or - "unknown" when no marker matches. The result is always a member of - KNOWN_CODING_AGENTS. - """ - for agent_name, env_vars in CODING_AGENT_ENV_MARKERS: - if any(os.environ.get(env_var) for env_var in env_vars): - return agent_name - - # Checked last and by presence: the cross-vendor marker establishes that an - # assistant is present without naming one, and an empty value still says so. - if any(env_var in os.environ for env_var in GENERIC_AGENT_ENV_VARS): - return _OTHER - - return _UNKNOWN - - -def detect_runtime_context() -> str: - """Best-effort detection of where this process is running. - - Separate from :func:`detect_coding_agent` because the two answer different - questions. Automated runs -- CI, containers, serverless -- have no - assistant to detect, and reporting them in the assistant field made an - unrecognized assistant indistinguishable from a run that could never have - had one. - - Precedence runs most specific first: CI and hosted IDEs typically run - inside containers, so a bare container match only applies once the more - specific markers have been ruled out. - - Returns: - One of the runtime names (e.g. "ci", "serverless", "container", - "notebook", "hosted_ide"), an editor terminal (e.g. "vscode_terminal"), - "interactive" or "non_interactive" when only a TTY check applies, or - "unknown" when that check cannot be made. The result is always a member - of KNOWN_RUNTIME_CONTEXTS. - """ - # Presence, not truthiness: a platform that exports an empty CI= is still - # CI, unlike the assistant markers where an empty value means the tool set - # a placeholder rather than claiming the session. - for context_name, env_vars in RUNTIME_CONTEXT_ENV_MARKERS: - if any(env_var in os.environ for env_var in env_vars): - return context_name - - for env_var, expected, context_name in _EDITOR_TERM_MARKERS: - if os.environ.get(env_var) == expected: - return context_name - - if os.path.exists(_DOCKER_ENV_PATH): - return "container" - - try: - return "interactive" if sys.stdout.isatty() else "non_interactive" - except (AttributeError, ValueError, OSError): - return _UNKNOWN - - def add_agent_fingerprint_to_span( span: Span, agent: Any, add_attribute_fn: Callable[[Span, str, Any], None] ) -> None: diff --git a/lib/crewai/src/crewai/utilities/constants.py b/lib/crewai/src/crewai/utilities/constants.py index 0f31c86aae..255d1d9a4b 100644 --- a/lib/crewai/src/crewai/utilities/constants.py +++ b/lib/crewai/src/crewai/utilities/constants.py @@ -8,6 +8,31 @@ from crewai_core.constants import ( TRAINING_DATA_FILE as TRAINING_DATA_FILE, ) from crewai_core.printer import PrinterColor + +# Coding-assistant and runtime marker tables live in crewai_core so the CLI can +# detect the same context without importing crewai. Re-exported here because +# these names are the established import path. +from crewai_core.runtime_env import ( + ANTIGRAVITY_ENV_VARS as ANTIGRAVITY_ENV_VARS, + AUGMENT_ENV_VARS as AUGMENT_ENV_VARS, + CC_ENV_VAR as CC_ENV_VAR, + CC_ENV_VARS as CC_ENV_VARS, + CI_ENV_VARS as CI_ENV_VARS, + CLINE_ENV_VARS as CLINE_ENV_VARS, + CODEX_ENV_VARS as CODEX_ENV_VARS, + CODING_AGENT_ENV_MARKERS as CODING_AGENT_ENV_MARKERS, + CONTAINER_ENV_VARS as CONTAINER_ENV_VARS, + CURSOR_ENV_VARS as CURSOR_ENV_VARS, + GEMINI_CLI_ENV_VARS as GEMINI_CLI_ENV_VARS, + GENERIC_AGENT_ENV_VARS as GENERIC_AGENT_ENV_VARS, + HOSTED_IDE_ENV_VARS as HOSTED_IDE_ENV_VARS, + JUNIE_ENV_VARS as JUNIE_ENV_VARS, + NOTEBOOK_ENV_VARS as NOTEBOOK_ENV_VARS, + OPENCODE_ENV_VARS as OPENCODE_ENV_VARS, + PAAS_ENV_VARS as PAAS_ENV_VARS, + RUNTIME_CONTEXT_ENV_MARKERS as RUNTIME_CONTEXT_ENV_MARKERS, + SERVERLESS_ENV_VARS as SERVERLESS_ENV_VARS, +) from pydantic_core import CoreSchema @@ -42,148 +67,6 @@ __all__ = [ EMITTER_COLOR: Final[PrinterColor] = "bold_blue" -CC_ENV_VAR: Final[str] = "CLAUDECODE" -CODEX_ENV_VARS: Final[tuple[str, ...]] = ( - "CODEX_CI", - "CODEX_MANAGED_BY_NPM", - "CODEX_SANDBOX", - "CODEX_SANDBOX_NETWORK_DISABLED", - "CODEX_THREAD_ID", -) -CC_ENV_VARS: Final[tuple[str, ...]] = (CC_ENV_VAR, "CLAUDE_CODE") -CURSOR_ENV_VARS: Final[tuple[str, ...]] = ( - "CURSOR_AGENT", - "CURSOR_EXTENSION_HOST_ROLE", - "CURSOR_SANDBOX", - "CURSOR_TRACE_ID", - "CURSOR_WORKSPACE_LABEL", -) -ANTIGRAVITY_ENV_VARS: Final[tuple[str, ...]] = ( - "ANTIGRAVITY_AGENT", - "ANTIGRAVITY_CLI_ALIAS", -) -AUGMENT_ENV_VARS: Final[tuple[str, ...]] = ("AUGMENT_AGENT",) -CLINE_ENV_VARS: Final[tuple[str, ...]] = ("CLINE_ACTIVE",) -GEMINI_CLI_ENV_VARS: Final[tuple[str, ...]] = ("GEMINI_CLI",) -JUNIE_ENV_VARS: Final[tuple[str, ...]] = ("JUNIE_DATA", "JUNIE_SHIM_PATH") -OPENCODE_ENV_VARS: Final[tuple[str, ...]] = ("OPENCODE", "OPENCODE_CLIENT") - -# Proposed cross-vendor marker (agentsmd/agents.md#136). Checked last and -# reported as "other": it says an assistant is present without naming one, and -# reading its value to find out would put an arbitrary string in telemetry. -GENERIC_AGENT_ENV_VARS: Final[tuple[str, ...]] = ("AI_AGENT",) - -# Ordered (name, env vars) pairs for identifying the AI coding assistant a -# process is running under. Reuses the sets above and keeps the same precedence -# as ``get_env_context()``, so the env-context events and telemetry never -# disagree about which assistant is present. -# -# Deliberately limited to assistants whose markers are verified. Guessing a -# variable name is worse than omitting the assistant: a wrong name never -# matches, so that assistant is silently counted as "unknown" while the table -# implies it is covered. -# -# Two rules for adding an entry: -# 1. Confirm the variable the tool actually sets - do not infer it from the -# product name. -# 2. Use only *session*-scoped variables the assistant sets for processes it -# spawns. Persistent user configuration (an ``AIDER_MODEL`` in a committed -# ``.env``, say) is unusable: crewai loads dotenv files on normal runs, so -# a leftover config value would mislabel ordinary human executions. -# -# Extend the shared sets above rather than adding a parallel tuple here, so both -# detection paths pick the new markers up together: ``get_env_context()`` walks -# this same table for its precedence and emits ``DefaultEnvEvent`` for the -# assistants that have no event class of their own. -# -# Markers below the first three were taken from the published detection matrix -# at vercel/detect-agent (agents.json), cross-checked against the proposal in -# agentsmd/agents.md#136 and microsoft/vscode#311734. Rule 2 excluded several -# entries those sources list: Goose's ``GOOSE_PROVIDER`` and Copilot's -# ``COPILOT_MODEL`` / ``COPILOT_GITHUB_TOKEN`` are user configuration, and a -# committed ``.env`` carrying one would relabel every ordinary run. Replit's -# ``REPL_ID`` is a hosted environment rather than an assistant, so it stays in -# HOSTED_IDE_ENV_VARS. -# -# The assistants that spawn inside another editor's terminal are ordered ahead -# of Cursor for the same reason Codex is: CURSOR_* is set for every integrated -# terminal, so checking Cursor first would mask anything running inside it. -CODING_AGENT_ENV_MARKERS: Final[tuple[tuple[str, tuple[str, ...]], ...]] = ( - ("claude_code", CC_ENV_VARS), - ("codex", CODEX_ENV_VARS), - ("cline", CLINE_ENV_VARS), - ("gemini_cli", GEMINI_CLI_ENV_VARS), - ("augment", AUGMENT_ENV_VARS), - ("opencode", OPENCODE_ENV_VARS), - ("antigravity", ANTIGRAVITY_ENV_VARS), - ("junie", JUNIE_ENV_VARS), - ("cursor", CURSOR_ENV_VARS), -) - -# Markers for *where* a process runs, kept separate from which assistant is -# driving it. The two answer different questions: a scheduled container run has -# no assistant to detect, and folding it into the assistant field made "no -# marker found" and "no assistant possible" indistinguishable. -# -# These are published platform contracts rather than per-tool observations, so -# they do not need the case-by-case verification the assistant table requires. -# Presence is all that is checked; no value is ever read. -CI_ENV_VARS: Final[tuple[str, ...]] = ( - "APPVEYOR", - "BITBUCKET_BUILD_NUMBER", - "BUILDKITE", - "CI", - "CIRCLECI", - "DRONE", - "GITHUB_ACTIONS", - "GITLAB_CI", - "JENKINS_URL", - "TEAMCITY_VERSION", - "TF_BUILD", - "TRAVIS", -) -SERVERLESS_ENV_VARS: Final[tuple[str, ...]] = ( - "AWS_LAMBDA_FUNCTION_NAME", - "FUNCTIONS_EXTENSION_VERSION", - "FUNCTIONS_WORKER_RUNTIME", - "FUNCTION_TARGET", - "K_SERVICE", - "VERCEL", -) -# Managed application platforms, kept apart from serverless: their markers are -# set for long-lived containers rather than per-invocation functions, and -# checking them under "serverless" would have claimed every Heroku dyno and -# Azure App Service instance before the container check could see them. -# -# Azure Functions run on the App Service host and inherit WEBSITE_INSTANCE_ID, -# so they would land here despite being serverless. The FUNCTIONS_* markers -# above are checked first to keep them out. -PAAS_ENV_VARS: Final[tuple[str, ...]] = ( - "DYNO", - "WEBSITE_INSTANCE_ID", -) -HOSTED_IDE_ENV_VARS: Final[tuple[str, ...]] = ( - "CODESPACES", - "GITPOD_WORKSPACE_ID", - "REPL_ID", -) -NOTEBOOK_ENV_VARS: Final[tuple[str, ...]] = ( - "COLAB_RELEASE_TAG", - "JPY_PARENT_PID", -) -CONTAINER_ENV_VARS: Final[tuple[str, ...]] = ("KUBERNETES_SERVICE_HOST",) - -# Ordered most specific first. CI jobs and hosted IDEs usually run inside -# containers, so a bare container match is only meaningful once the others have -# been ruled out. -RUNTIME_CONTEXT_ENV_MARKERS: Final[tuple[tuple[str, tuple[str, ...]], ...]] = ( - ("ci", CI_ENV_VARS), - ("serverless", SERVERLESS_ENV_VARS), - ("paas", PAAS_ENV_VARS), - ("hosted_ide", HOSTED_IDE_ENV_VARS), - ("notebook", NOTEBOOK_ENV_VARS), - ("container", CONTAINER_ENV_VARS), -) class _NotSpecified: diff --git a/lib/crewai/tests/telemetry/conftest.py b/lib/crewai/tests/telemetry/conftest.py new file mode 100644 index 0000000000..4db226b90c --- /dev/null +++ b/lib/crewai/tests/telemetry/conftest.py @@ -0,0 +1,19 @@ +"""Shared fixtures for the telemetry suite.""" + +from crewai_core.telemetry import common_span_attributes +import pytest + + +@pytest.fixture(autouse=True) +def _reset_common_attributes_cache(): + """Clear the process-wide common-attribute cache around every test. + + ``common_span_attributes`` is cached for the life of the process, which is + correct in production - a process cannot change which assistant, runtime or + project it belongs to. In tests it would outlive the environment each test + sets up, so a value computed under one test's markers would leak into the + next and make results depend on ordering. + """ + common_span_attributes.cache_clear() + yield + common_span_attributes.cache_clear() diff --git a/lib/crewai/tests/telemetry/test_coding_agent_detection.py b/lib/crewai/tests/telemetry/test_coding_agent_detection.py index 9c62808159..87f43fb097 100644 --- a/lib/crewai/tests/telemetry/test_coding_agent_detection.py +++ b/lib/crewai/tests/telemetry/test_coding_agent_detection.py @@ -5,6 +5,7 @@ from unittest.mock import patch import pytest +from crewai.telemetry.constants import TRACER_NAME from crewai.telemetry.utils import ( KNOWN_CODING_AGENTS, KNOWN_RUNTIME_CONTEXTS, @@ -276,49 +277,6 @@ def test_terminal_bound_assistants_outrank_cursor(clean_env): clean_env.delenv(marker) -def test_concurrent_attach_registers_the_processor_once(isolated_telemetry, clean_env): - """Check-then-act on the provider set must be locked. - - Crews and flows created from different threads can both reach set_tracer() - before trace_set flips, and without a lock each would attach its own - processor to the same provider for the life of the process. - """ - import threading - import time - - telemetry = isolated_telemetry() - threads_count = 8 - - class SlowProvider: - """Widens the check-then-act window so the race is deterministic. - - Sleeping inside add_span_processor guarantees every unlocked thread gets - past the membership check before any of them records the provider. - """ - - def __init__(self) -> None: - self.processors: list[object] = [] - - def add_span_processor(self, processor: object) -> None: - time.sleep(0.05) - self.processors.append(processor) - - provider = SlowProvider() - start = threading.Barrier(threads_count) - - def attach() -> None: - start.wait() - telemetry._attach_common_attributes(provider) - - threads = [threading.Thread(target=attach) for _ in range(threads_count)] - for thread in threads: - thread.start() - for thread in threads: - thread.join() - - assert len(provider.processors) == 1 - - def test_editor_terminal_requires_exact_value(clean_env): clean_env.setenv("TERM_PROGRAM", "vscode") assert detect_runtime_context() == "vscode_terminal" @@ -454,7 +412,7 @@ def test_coding_agent_lands_on_every_exported_span(clean_env, otel_enabled): InMemorySpanExporter, ) - from crewai.telemetry.telemetry import CommonAttributesSpanProcessor + from crewai_core.telemetry import CommonAttributesSpanProcessor exporter = InMemorySpanExporter() provider = TracerProvider() @@ -479,7 +437,7 @@ def test_coding_agent_lands_on_every_exported_span(clean_env, otel_enabled): def test_common_attributes_processor_never_breaks_span_creation(clean_env, otel_enabled): """A failure applying attributes must not propagate into user execution.""" - from crewai.telemetry.telemetry import CommonAttributesSpanProcessor + from crewai_core.telemetry import CommonAttributesSpanProcessor class ExplodingSpan: def set_attributes(self, _): @@ -505,15 +463,36 @@ def test_coding_agent_span_emits_once(isolated_telemetry, clean_env, monkeypatch assert emitted == ["coding_agent:claude_code"] -def test_attribute_survives_an_externally_installed_provider( +def test_common_attributes_land_on_our_own_spans( isolated_telemetry, clean_env, otel_enabled ): - """Spans must keep coding_agent when the app installs its own provider. + """Spans built from our own provider must carry the process-wide context.""" + from opentelemetry.sdk.trace.export import SimpleSpanProcessor + from opentelemetry.sdk.trace.export.in_memory_span_exporter import ( + InMemorySpanExporter, + ) - set_tracer() leaves an existing non-proxy provider in place, and telemetry - methods resolve their tracer through the global provider - so attaching the - processor only to our own provider would drop the attribute entirely in any - already-instrumented application. + clean_env.setenv("CLAUDECODE", "1") + + telemetry = isolated_telemetry() + exporter = InMemorySpanExporter() + telemetry.provider.add_span_processor(SimpleSpanProcessor(exporter)) + + telemetry.provider.get_tracer(TRACER_NAME).start_span("Crew Created").end() + + exported = exporter.get_finished_spans() + assert len(exported) == 1 + assert exported[0].attributes["coding_agent"] == "claude_code" + + +def test_an_app_provider_never_receives_our_processor( + isolated_telemetry, clean_env, otel_enabled +): + """An application's own provider must be left completely alone. + + Attaching our processor to it stamped CrewAI's process-wide attributes onto + that application's unrelated spans - HTTP handlers, DB queries - which is + both wrong data and data we were never given permission to annotate. """ from opentelemetry import trace as ot from opentelemetry.sdk.trace import TracerProvider @@ -532,46 +511,23 @@ def test_attribute_survives_an_externally_installed_provider( telemetry = isolated_telemetry() telemetry.set_tracer() - span = app_provider.get_tracer("crewai.telemetry").start_span("Crew Created") - span.end() + app_provider.get_tracer("some.app.library").start_span("GET /status").end() exported = exporter.get_finished_spans() assert len(exported) == 1 - assert exported[0].attributes["coding_agent"] == "claude_code" - - -def test_attaching_common_attributes_is_idempotent(isolated_telemetry, clean_env): - """Repeated set_tracer() calls must not stack duplicate processors.""" - from opentelemetry.sdk.trace import TracerProvider - - provider = TracerProvider() - telemetry = isolated_telemetry() - - before = len(provider._active_span_processor._span_processors) - telemetry._attach_common_attributes(provider) - telemetry._attach_common_attributes(provider) - after = len(provider._active_span_processor._span_processors) - - assert after - before == 1 - - -def test_attaching_to_a_provider_without_processors_is_safe(isolated_telemetry): - """A NoOp provider has no add_span_processor; this must not raise.""" - telemetry = isolated_telemetry() - - telemetry._attach_common_attributes(object()) + assert "coding_agent" not in (exported[0].attributes or {}) + assert "project_id" not in (exported[0].attributes or {}) def _common_attributes(monkeypatch, project_id=None): """Build the process-wide span attributes with a stubbed project id.""" - from crewai.telemetry.telemetry import Telemetry + from crewai_core.telemetry import common_span_attributes monkeypatch.setattr( - "crewai.telemetry.telemetry.get_project_id", lambda *a, **k: project_id + "crewai_core.telemetry.get_project_id", lambda *a, **k: project_id ) - telemetry = Telemetry.__new__(Telemetry) - telemetry._common_attributes = None - return telemetry._common_span_attributes() + common_span_attributes.cache_clear() + return common_span_attributes() def test_common_attributes_carry_agent_and_runtime(clean_env, monkeypatch): @@ -600,16 +556,15 @@ def test_project_id_is_omitted_when_absent(clean_env, monkeypatch): def test_project_id_lookup_never_breaks_telemetry(clean_env, monkeypatch): """A failed lookup degrades to omitting the attribute.""" - from crewai.telemetry.telemetry import Telemetry + from crewai_core.telemetry import common_span_attributes def boom(*args, **kwargs): raise OSError("unreadable") - monkeypatch.setattr("crewai.telemetry.telemetry.get_project_id", boom) - telemetry = Telemetry.__new__(Telemetry) - telemetry._common_attributes = None + monkeypatch.setattr("crewai_core.telemetry.get_project_id", boom) + common_span_attributes.cache_clear() - attributes = telemetry._common_span_attributes() + attributes = common_span_attributes() assert "project_id" not in attributes assert "coding_agent" in attributes @@ -617,7 +572,7 @@ def test_project_id_lookup_never_breaks_telemetry(clean_env, monkeypatch): def test_common_attributes_are_computed_once(clean_env, monkeypatch): """The project file must not be re-read for each provider.""" - from crewai.telemetry.telemetry import Telemetry + from crewai_core.telemetry import common_span_attributes calls = [] @@ -626,13 +581,12 @@ def test_common_attributes_are_computed_once(clean_env, monkeypatch): return "proj-123" monkeypatch.setattr( - "crewai.telemetry.telemetry.get_project_id", counting_get_project_id + "crewai_core.telemetry.get_project_id", counting_get_project_id ) - telemetry = Telemetry.__new__(Telemetry) - telemetry._common_attributes = None + common_span_attributes.cache_clear() - first = telemetry._common_span_attributes() - second = telemetry._common_span_attributes() + first = common_span_attributes() + second = common_span_attributes() assert first is second assert len(calls) == 1 @@ -646,7 +600,7 @@ def test_all_common_attributes_land_on_exported_spans(clean_env, monkeypatch, ot InMemorySpanExporter, ) - from crewai.telemetry.telemetry import CommonAttributesSpanProcessor + from crewai_core.telemetry import CommonAttributesSpanProcessor clean_env.setenv("CLAUDECODE", "1") clean_env.setenv("CI", "true") diff --git a/lib/crewai/tests/telemetry/test_detection_reexports.py b/lib/crewai/tests/telemetry/test_detection_reexports.py new file mode 100644 index 0000000000..5fb4604483 --- /dev/null +++ b/lib/crewai/tests/telemetry/test_detection_reexports.py @@ -0,0 +1,63 @@ +"""The detection helpers moved to crewai_core; their old paths must still work. + +``crewai.telemetry.utils`` and ``crewai.utilities.constants`` are the +established import paths for these names, and both are public surface for +anyone who reached for them. Moving the implementation must not break them. +""" + +from __future__ import annotations + +from crewai_core import runtime_env +import pytest + + +def test_detection_helpers_still_import_from_telemetry_utils() -> None: + from crewai.telemetry.utils import ( + KNOWN_CODING_AGENTS, + KNOWN_RUNTIME_CONTEXTS, + detect_coding_agent, + detect_runtime_context, + ) + + assert detect_coding_agent is runtime_env.detect_coding_agent + assert detect_runtime_context is runtime_env.detect_runtime_context + assert KNOWN_CODING_AGENTS is runtime_env.KNOWN_CODING_AGENTS + assert KNOWN_RUNTIME_CONTEXTS is runtime_env.KNOWN_RUNTIME_CONTEXTS + + +@pytest.mark.parametrize( + "name", + [ + "ANTIGRAVITY_ENV_VARS", + "AUGMENT_ENV_VARS", + "CC_ENV_VAR", + "CC_ENV_VARS", + "CI_ENV_VARS", + "CLINE_ENV_VARS", + "CODEX_ENV_VARS", + "CODING_AGENT_ENV_MARKERS", + "CONTAINER_ENV_VARS", + "CURSOR_ENV_VARS", + "GEMINI_CLI_ENV_VARS", + "GENERIC_AGENT_ENV_VARS", + "HOSTED_IDE_ENV_VARS", + "JUNIE_ENV_VARS", + "NOTEBOOK_ENV_VARS", + "OPENCODE_ENV_VARS", + "PAAS_ENV_VARS", + "RUNTIME_CONTEXT_ENV_MARKERS", + "SERVERLESS_ENV_VARS", + ], +) +def test_marker_tables_still_import_from_utilities_constants(name: str) -> None: + from crewai.utilities import constants + + assert getattr(constants, name) is getattr(runtime_env, name) + assert name in constants.__all__ + + +def test_env_context_still_reads_the_shared_table() -> None: + """get_env_context() walks the same table, so both paths stay in agreement.""" + from crewai.utilities.env import CODING_AGENT_ENV_MARKERS + + assert CODING_AGENT_ENV_MARKERS is runtime_env.CODING_AGENT_ENV_MARKERS diff --git a/lib/crewai/tests/telemetry/test_telemetry.py b/lib/crewai/tests/telemetry/test_telemetry.py index 5b565371f2..acc44bf9ef 100644 --- a/lib/crewai/tests/telemetry/test_telemetry.py +++ b/lib/crewai/tests/telemetry/test_telemetry.py @@ -53,20 +53,22 @@ def test_telemetry_enabled_by_default(): assert telemetry.ready is True -def test_set_tracer_skips_when_provider_already_configured(): - """A second telemetry instance must not re-install the global provider.""" - with ( - patch.dict(os.environ, {}, clear=True), - patch( - "crewai.telemetry.telemetry.trace.get_tracer_provider", - return_value=TracerProvider(), - ), - patch("crewai.telemetry.telemetry.trace.set_tracer_provider") as mock_set, - ): +def test_set_tracer_never_installs_a_global_provider(): + """Telemetry must not hijack the process-wide TracerProvider. + + Installing it globally made every OTel-instrumented library in the host + process export to CrewAI's collector, so the global provider must be left + exactly as it was found whether or not an application installed one. + """ + import opentelemetry.trace as ot + + with patch.dict(os.environ, {}, clear=True): + before = ot.get_tracer_provider() telemetry = Telemetry() telemetry.set_tracer() + after = ot.get_tracer_provider() - mock_set.assert_not_called() + assert after is before assert telemetry.trace_set is True @@ -84,8 +86,10 @@ def test_flow_execution_span_records_crewai_version(): "OTEL_SDK_DISABLED": "false", }, ), - patch("crewai.telemetry.telemetry.TracerProvider"), - patch("crewai.telemetry.telemetry.trace.get_tracer", return_value=tracer), + patch( + "crewai.telemetry.telemetry.TracerProvider", + return_value=Mock(get_tracer=Mock(return_value=tracer)), + ), patch("crewai.telemetry.telemetry.version", return_value="9.9.9"), ): telemetry = Telemetry() @@ -110,8 +114,10 @@ def test_flow_creation_span_records_crewai_version(): "OTEL_SDK_DISABLED": "false", }, ), - patch("crewai.telemetry.telemetry.TracerProvider"), - patch("crewai.telemetry.telemetry.trace.get_tracer", return_value=tracer), + patch( + "crewai.telemetry.telemetry.TracerProvider", + return_value=Mock(get_tracer=Mock(return_value=tracer)), + ), patch("crewai.telemetry.telemetry.version", return_value="9.9.9"), ): telemetry = Telemetry() diff --git a/lib/crewai/tests/telemetry/test_tracer_isolation.py b/lib/crewai/tests/telemetry/test_tracer_isolation.py new file mode 100644 index 0000000000..9ee96383df --- /dev/null +++ b/lib/crewai/tests/telemetry/test_tracer_isolation.py @@ -0,0 +1,128 @@ +"""Telemetry must export our spans and only our spans. + +Regression cover for the collector receiving third-party application traces: +``set_tracer()`` used to install CrewAI's ``TracerProvider`` as the global one, +so every OTel-instrumented library in the host process - HTTP servers, Redis +clients, ORMs - resolved ``trace.get_tracer()`` to our provider and shipped its +spans to CrewAI's endpoint. +""" + +from typing import Any +from unittest.mock import patch + +import opentelemetry.trace as ot +from opentelemetry.sdk.trace import TracerProvider +from opentelemetry.sdk.trace.export import SimpleSpanProcessor, SpanExportResult +from opentelemetry.sdk.trace.export.in_memory_span_exporter import ( + InMemorySpanExporter, +) +import pytest + +from crewai.telemetry.constants import TRACER_NAME +from crewai.telemetry.telemetry import Telemetry + + +class _NullExporter: + """Stands in for the OTLP exporter so no test attempts a real export.""" + + def export(self, spans: Any) -> SpanExportResult: + return SpanExportResult.SUCCESS + + def shutdown(self) -> None: + pass + + def force_flush(self, timeout_millis: int = 30000) -> bool: + return True + + +@pytest.fixture +def telemetry_with_exporter(monkeypatch): + """A fresh Telemetry whose provider exports into memory. + + Telemetry is a process-wide singleton that registers atexit and signal + handlers on init, so the instance is replaced for the duration of the test + and lifecycle registration is suppressed. + """ + monkeypatch.setattr(Telemetry, "_instance", None) + monkeypatch.setattr(Telemetry, "_register_shutdown_handlers", lambda self: None) + + # Set for the whole test: _is_telemetry_disabled() is re-read on every span + # call, and the suite runs with OTEL_SDK_DISABLED set. + monkeypatch.setenv("CREWAI_DISABLE_TELEMETRY", "false") + monkeypatch.setenv("CREWAI_DISABLE_TRACKING", "false") + monkeypatch.setenv("OTEL_SDK_DISABLED", "false") + + # Patched before construction: __init__ wires the real OTLP exporter, which + # would make every test here attempt a live export. + monkeypatch.setattr( + "crewai.telemetry.telemetry.SafeOTLPSpanExporter", + lambda **_kwargs: _NullExporter(), + ) + + telemetry = Telemetry() + + exporter = InMemorySpanExporter() + telemetry.provider.add_span_processor(SimpleSpanProcessor(exporter)) + + try: + yield telemetry, exporter + finally: + telemetry.provider.shutdown() + Telemetry._instance = None + + +def test_third_party_spans_never_reach_our_exporter(telemetry_with_exporter): + """A dependency instrumenting itself must not export to CrewAI.""" + telemetry, exporter = telemetry_with_exporter + telemetry.set_tracer() + + ot.get_tracer("redis.client").start_span("XLEN").end() + ot.get_tracer("opentelemetry.instrumentation.asgi").start_span( + "GET /status http send" + ).end() + + assert exporter.get_finished_spans() == () + + +def test_our_own_spans_still_reach_our_exporter(telemetry_with_exporter): + """The isolation must not cost us the telemetry we do want.""" + telemetry, exporter = telemetry_with_exporter + telemetry.set_tracer() + + telemetry.feature_usage_span("cli_usage:view_traces") + + assert [span.name for span in exporter.get_finished_spans()] == ["Feature Usage"] + + +def test_our_spans_are_unaffected_by_an_application_provider(telemetry_with_exporter): + """An app that installs its own provider must not divert our telemetry. + + Resolving our tracer globally meant that in an already-instrumented + application our spans were created by the application's provider and went + to its collector, so CrewAI received nothing at all from those processes. + """ + telemetry, exporter = telemetry_with_exporter + + app_exporter = InMemorySpanExporter() + app_provider = TracerProvider() + app_provider.add_span_processor(SimpleSpanProcessor(app_exporter)) + + with patch.object(ot, "get_tracer_provider", return_value=app_provider): + telemetry.set_tracer() + telemetry.feature_usage_span("cli_usage:deploy") + + assert [span.name for span in exporter.get_finished_spans()] == ["Feature Usage"] + assert app_exporter.get_finished_spans() == () + + +def test_set_tracer_is_idempotent(telemetry_with_exporter): + """Repeated calls must not stack processors or duplicate exports.""" + telemetry, exporter = telemetry_with_exporter + + telemetry.set_tracer() + telemetry.set_tracer() + telemetry.set_tracer() + + telemetry.provider.get_tracer(TRACER_NAME).start_span("Crew Created").end() + + assert len(exporter.get_finished_spans()) == 1