Files
crewAI/lib/crewai-core/tests/test_smoke.py
João Moura 094b94e8d0
Some checks failed
CodeQL Advanced / Analyze (actions) (push) Has been cancelled
CodeQL Advanced / Analyze (python) (push) Has been cancelled
Check Documentation Broken Links / Check broken links (push) Has been cancelled
Vulnerability Scan / Detect changes (push) Has been cancelled
Vulnerability Scan / pip-audit (push) Has been cancelled
Nightly Canary Release / Check for new commits (push) Has been cancelled
Nightly Canary Release / Build nightly packages (push) Has been cancelled
Nightly Canary Release / Publish nightly to PyPI (push) Has been cancelled
Mark stale issues and pull requests / stale (push) Has been cancelled
fix(core): scope span export to our own tracer provider (#6954)
* fix(telemetry): stop exporting third-party spans to the collector

set_tracer() installed CrewAI's TracerProvider as the global one, so every
OTel-instrumented library in the host process - HTTP servers, Redis clients,
ORMs - resolved trace.get_tracer() to our provider and exported to CrewAI's
endpoint. A 20M-row sample of the telemetry table found 18,866 distinct
operation names under our serviceName; CrewAI emits 21.

The same wiring lost data in the other direction: when an application had
already installed its own provider, our spans were created by theirs and went
to their collector, so CrewAI received nothing from instrumented processes.

Spans are now created from the private provider in both packages. Deletes
_attach_common_attributes and its WeakSet/lock, whose multi-provider dedupe
guarded a state that can no longer occur.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH

* fix(telemetry): keep process context on crewai-core spans

Isolating each package to its own TracerProvider removed an accident the CLI
spans depended on: crewai_core.telemetry had no CommonAttributesSpanProcessor,
so its spans only ever carried coding_agent/runtime_context/project_id by
riding the global provider that crewai installed at import.

A differential capture of every span reaching the exporter showed 8 of 54 spans
losing those attributes - Feature Usage (cli_usage:*), Start Deployment,
Template Installed, Create Crew Deployment, Get Crew Logs, Remove Crew,
Deploy Signup Error and Flow Creation.

Moves the marker tables and the detect_* helpers to crewai_core.runtime_env and
the processor plus common_span_attributes() to crewai_core.telemetry, so both
implementations share one source of truth. crewai.telemetry.utils and
crewai.utilities.constants re-export the moved names, so their import paths are
unchanged.

Also fixes a gap that predates the isolation change: a CLI-only process never
imports crewai, so it never reported either attribute. It does now.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH

* fix(core): type test helpers and stop tests reaching the collector

mypy runs over lib/crewai-core/tests (the one test tree not excluded), so the
new test file needed full annotations and a narrowed span.attributes.

Also patches SafeOTLPSpanExporter before Telemetry is constructed and shuts the
provider down afterwards: __init__ wires a BatchSpanProcessor around the real
OTLP exporter, so each test was attempting a live export and leaving its batch
worker thread running.

Corrects the marker-precedence docstring, which named Cursor third when the
table checks it last so that assistants running inside its terminal are not
masked.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH

* style(core): use one import form per module in telemetry tests

Both test modules imported their telemetry module twice - once aliased for the
monkeypatch target and once via from-import for the names. Dropping the alias
in favour of monkeypatch's dotted-string target leaves a single import form and
removes the need to qualify every reference.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH

* docs(core): drop comments that restate the code

The TRACER_NAME constants were annotated with what their name and
set_tracer()'s docstring already say, and the test fixtures narrated
provider.shutdown() and the exporter patch at more length than either needed.

Keeps the ones carrying something the code cannot: the resource-attribute
ingestion quirk, why the marker tables moved packages, and the two ordering
traps the fixtures exist to avoid.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01ASfWmW3RGy4qAQm6s8U9jH

---------

Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-11 00:16:27 +00:00

255 lines
7.9 KiB
Python

"""Smoke tests for the crewai-core leaf modules."""
from __future__ import annotations
import os
from pathlib import Path
from unittest.mock import Mock
from crewai_core import (
constants,
lock_store,
paths,
printer,
project,
user_data,
version,
)
import pytest
def test_version_returns_string() -> None:
v = version.get_crewai_version()
assert isinstance(v, str) and v
def test_paths_creates_storage_dir(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
monkeypatch.setenv("CREWAI_STORAGE_DIR", str(tmp_path / "store"))
monkeypatch.setattr(
"crewai_core.paths.appdirs.user_data_dir",
lambda app, author: str(tmp_path / app),
)
out = paths.db_storage_path()
assert Path(out).exists()
def test_constants_exposes_env_keys() -> None:
assert constants.CREWAI_TRAINED_AGENTS_FILE_ENV == "CREWAI_TRAINED_AGENTS_FILE"
def test_printer_emits_when_not_suppressed(capsys: pytest.CaptureFixture[str]) -> None:
printer.PRINTER.print("hello", color="green")
out = capsys.readouterr().out
assert "hello" in out
def test_printer_respects_suppression(capsys: pytest.CaptureFixture[str]) -> None:
token = printer.set_suppress_console_output(True)
try:
printer.PRINTER.print("hidden")
finally:
printer._suppress_console_output.reset(token) # type: ignore[arg-type]
assert "hidden" not in capsys.readouterr().out
def test_lock_acquires_and_releases() -> None:
with lock_store.lock("crewai_core.tests.smoke", timeout=5):
pass
def test_user_data_round_trip(tmp_path: Path, monkeypatch: pytest.MonkeyPatch) -> None:
monkeypatch.setenv("CREWAI_STORAGE_DIR", "crewai_core_test_user_data")
monkeypatch.setattr(
"crewai_core.paths.appdirs.user_data_dir",
lambda app, author: str(tmp_path / app),
)
user_data.update_user_data({"trace_consent": True, "first_execution_done": True})
data = user_data._load_user_data()
assert data == {"trace_consent": True, "first_execution_done": True}
assert user_data.has_user_declined_tracing() is False
monkeypatch.setenv("CREWAI_TRACING_ENABLED", "true")
assert user_data.is_tracing_enabled() is True
monkeypatch.delenv("CREWAI_TRACING_ENABLED", raising=False)
assert (
user_data.is_tracing_enabled() is True
) # consent alone enables (matches runtime)
def test_user_data_decline_blocks(
tmp_path: Path, monkeypatch: pytest.MonkeyPatch
) -> None:
monkeypatch.setenv("CREWAI_STORAGE_DIR", "crewai_core_test_decline")
monkeypatch.setattr(
"crewai_core.paths.appdirs.user_data_dir",
lambda app, author: str(tmp_path / app),
)
user_data.update_user_data({"trace_consent": False, "first_execution_done": True})
assert user_data.has_user_declined_tracing() is True
monkeypatch.delenv("CREWAI_TRACING_ENABLED", raising=False)
assert user_data.is_tracing_enabled() is False
monkeypatch.setenv("CREWAI_TRACING_ENABLED", "true")
assert user_data.is_tracing_enabled() is True # env-var override (matches runtime)
def test_unused_var_warning_silenced() -> None:
# Touch os to keep the import (used by env-var fixtures above)
assert os.environ is not None
def test_configured_project_definition_resolves_project_local_file(
tmp_path: Path,
) -> None:
definition = tmp_path / "crew.jsonc"
definition.write_text("{}\n")
resolved = project.configured_project_definition(
"crew",
pyproject_data={
"tool": {
"crewai": {
"type": "crew",
"definition": " crew.jsonc ",
}
}
},
project_root=tmp_path,
)
assert resolved == definition.resolve()
def test_configured_project_definition_rejects_project_escape(tmp_path: Path) -> None:
outside = tmp_path.parent / f"{tmp_path.name}-outside-crew.jsonc"
outside.write_text("{}\n")
with pytest.raises(project.ProjectDefinitionError):
project.configured_project_definition(
"crew",
pyproject_data={
"tool": {
"crewai": {
"type": "crew",
"definition": "../outside-crew.jsonc",
}
}
},
project_root=tmp_path,
)
def test_configured_project_definition_rejects_non_string_definition(
tmp_path: Path,
) -> None:
with pytest.raises(project.ProjectDefinitionError, match="must be a string"):
project.configured_project_definition(
"crew",
pyproject_data={
"tool": {
"crewai": {
"type": "crew",
"definition": ["crew.jsonc"],
}
}
},
project_root=tmp_path,
)
def test_configured_project_definition_rejects_empty_definition(
tmp_path: Path,
) -> None:
with pytest.raises(project.ProjectDefinitionError, match="non-empty"):
project.configured_project_definition(
"crew",
pyproject_data={
"tool": {
"crewai": {
"type": "crew",
"definition": " ",
}
}
},
project_root=tmp_path,
)
def test_core_telemetry_never_installs_a_global_provider(
monkeypatch: pytest.MonkeyPatch,
) -> None:
"""Telemetry must leave the process-wide TracerProvider untouched.
Installing it globally routed every OTel-instrumented library in the host
process - HTTP servers, Redis clients, ORMs - to CrewAI's collector.
"""
from crewai_core.telemetry import Telemetry
import opentelemetry.trace as ot
Telemetry._instance = None
monkeypatch.delenv("OTEL_SDK_DISABLED", raising=False)
monkeypatch.delenv("CREWAI_DISABLE_TELEMETRY", raising=False)
monkeypatch.delenv("CREWAI_DISABLE_TRACKING", raising=False)
before = ot.get_tracer_provider()
telemetry = Telemetry()
telemetry.set_tracer()
after = ot.get_tracer_provider()
assert after is before
assert telemetry.trace_set is True
def test_core_telemetry_records_feature_usage(
monkeypatch: pytest.MonkeyPatch,
) -> None:
from crewai_core.telemetry import Telemetry
Telemetry._instance = None
monkeypatch.delenv("OTEL_SDK_DISABLED", raising=False)
monkeypatch.delenv("CREWAI_DISABLE_TELEMETRY", raising=False)
monkeypatch.delenv("CREWAI_DISABLE_TRACKING", raising=False)
tracer = Mock()
span = Mock()
tracer.start_span.return_value = span
monkeypatch.setattr(
"crewai_core.telemetry.TracerProvider",
lambda **_kwargs: Mock(get_tracer=Mock(return_value=tracer)),
)
telemetry = Telemetry()
telemetry.feature_usage_span("cli_usage:view_traces")
tracer.start_span.assert_called_once_with("Feature Usage")
span.set_attribute.assert_any_call("feature", "cli_usage:view_traces")
span.end.assert_called_once()
def test_core_telemetry_records_flow_creation_version(
monkeypatch: pytest.MonkeyPatch,
) -> None:
from crewai_core.telemetry import Telemetry
Telemetry._instance = None
monkeypatch.delenv("OTEL_SDK_DISABLED", raising=False)
monkeypatch.delenv("CREWAI_DISABLE_TELEMETRY", raising=False)
monkeypatch.delenv("CREWAI_DISABLE_TRACKING", raising=False)
monkeypatch.setattr("crewai_core.version.get_crewai_version", lambda: "1.0.0")
tracer = Mock()
span = Mock()
tracer.start_span.return_value = span
monkeypatch.setattr(
"crewai_core.telemetry.TracerProvider",
lambda **_kwargs: Mock(get_tracer=Mock(return_value=tracer)),
)
telemetry = Telemetry()
telemetry.flow_creation_span("ResearchFlow")
tracer.start_span.assert_called_once_with("Flow Creation")
span.set_attribute.assert_any_call("crewai_version", "1.0.0")
span.set_attribute.assert_any_call("flow_name", "ResearchFlow")
span.end.assert_called_once()