Commit Graph

6 Commits

Author SHA1 Message Date
Joao Moura
e1b0ca1124 fix(telemetry): stop export assertions depending on test order
test_all_common_attributes_land_on_exported_spans failed in CI with an
IndexError on an empty span list, and only in one shard: the suite runs
with OTEL_SDK_DISABLED set, so TracerProvider hands out no-op tracers and
an export-based assertion sees zero spans rather than a wrong attribute.
It passed only when it happened to run after a test whose fixture flips
the variable, which random ordering decides.

Adds an otel_enabled fixture that sets the variable for the four tests
asserting on exported spans. Three of them predate this branch and had
the same latent dependency - they are fixed here because the new test
made the ordering hit reachable, and leaving them would keep the required
check red.

Verified by running every test in the file individually, all of which
previously exposed the dependency, and the telemetry suite three times
under random ordering.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 16:09:21 -07:00
Joao Moura
36070166cf fix(telemetry): share assistant precedence with the env-context path
Four findings from the Cursor and CodeRabbit reviews, three of them the
same root cause.

get_env_context restated the precedence the shared table already defines,
so every marker added for telemetry was invisible to it: a session
exposing only CLAUDE_CODE reported claude_code on spans while emitting
DefaultEnvEvent, and an assistant running inside a Cursor terminal
reported that assistant on spans while emitting CursorEnvEvent. It now
walks CODING_AGENT_ENV_MARKERS and maps the three assistants that have an
event class of their own, defaulting the rest to DefaultEnvEvent. A test
now asserts the two paths agree for every marker in the table, so they
cannot drift again.

The generic AI_AGENT marker was documented as presence-only but ran
through the truthiness loop with everything else, so an empty value fell
through to unknown. It moves out of the table and is checked by presence
after it, which also keeps the named markers' truthiness intact.

Azure Functions run on the App Service host and inherit
WEBSITE_INSTANCE_ID, so moving that marker to paas would have relabelled
them. The FUNCTIONS_* markers are checked first to keep them serverless.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 15:44:58 -07:00
Joao Moura
59c932cd68 feat(telemetry): widen assistant detection from published marker sets
The table previously covered three assistants because the rest were
unverified. They are documented after all: vercel/detect-agent publishes
a machine-readable detection matrix (agents.json), corroborated by the
proposal in agentsmd/agents.md#136 and by microsoft/vscode#311734.

Adds cline, gemini_cli, augment, opencode, antigravity and junie, plus
CLAUDE_CODE alongside CLAUDECODE. Gemini's marker is confirmed by its own
docs, which state that run_shell_command sets GEMINI_CLI=1 in the
subprocess environment.

Rule 2 excluded several entries those sources list. Goose's
GOOSE_PROVIDER and Copilot's COPILOT_MODEL and COPILOT_GITHUB_TOKEN are
user configuration, and a committed .env carrying one would relabel every
ordinary run - the AIDER_MODEL trap the guard test already pins, now
parametrized over all four. Replit's REPL_ID names a hosted environment
rather than an assistant, so it stays a runtime context. Copilot sets no
session marker at all today; that is an open request upstream.

The new assistants are ordered ahead of Cursor, since CURSOR_* is set for
every integrated terminal and would otherwise mask anything spawned
inside it - the same ordering Codex already needed.

Also adds the proposed cross-vendor AI_AGENT marker as a last resort,
reported as "other". It establishes that an assistant is present without
naming one, and its value is an arbitrary vendor string, so the value is
never read.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 15:33:41 -07:00
Joao Moura
96e40ed3a4 fix(telemetry): detect runtime markers by presence, split paas from serverless
Three findings from the CodeRabbit, code-quality and Cursor reviews.

The runtime loop tested truthiness while constants.py documented presence,
so a platform exporting a bare CI= fell through to the TTY fallback and
was mislabelled as an ordinary local run. Presence is now what it says.
The assistant markers keep truthiness deliberately: there an empty value
means the tool set a placeholder rather than claiming the session.

DYNO and WEBSITE_INSTANCE_ID marked Heroku dynos and Azure App Service
instances as serverless, and since serverless is checked first they could
never reach the container label. They move to a paas context, which is
what they are: long-lived containers rather than per-invocation
functions. AWS_EXECUTION_ENV is dropped entirely - it is set on ECS and
EC2 as well as Lambda, and AWS_LAMBDA_FUNCTION_NAME already covers Lambda
without the collision.

The container probe no longer wraps os.path.exists in a try/except.
os.path.exists handles OSError internally and returns False, so the
handler guarded a condition that cannot occur and only hid the intent.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 15:22:36 -07:00
Joao Moura
9a67ecf63b feat(telemetry): split runtime context from coding agent, add project id
The coding-agent field answered two questions at once. A run with no TTY
reported "non_interactive" and an editor's integrated terminal reported
"vscode_terminal", both in the same field as the assistant name, so a run
that never had an assistant to detect was indistinguishable from one
whose assistant we failed to recognize. Together those two values were
the majority of what the field reported.

detect_coding_agent now answers only which assistant, returning "unknown"
when no marker matches. detect_runtime_context answers where the process
runs: ci, serverless, hosted_ide, notebook, container, the editor
terminals, and the interactive/non_interactive fallback. Both ride on
every span, so an assistant running inside CI reports both rather than
one masking the other.

The runtime markers are published platform contracts - CI providers,
container and serverless runtimes, hosted IDEs - so unlike the assistant
table they need no per-tool verification step. Presence is checked; no
value is read. The assistant table is unchanged: its entries still
require a confirmed, session-scoped variable, and the existing guard test
still enforces that.

Spans also carry project_id when the project declares one. It is read
through the read-only accessor, since minting an id belongs to the CLI
commands a user invoked rather than to a library call during execution,
and it is omitted entirely for projects without one. The attributes are
computed once per process and memoized, so the project file is not
re-read for each provider.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
2026-08-07 15:12:51 -07:00
João Moura
766d71aefb feat: surface AMP in AGENTS.md and detect coding agents in telemetry (#6779) 2026-08-03 12:13:05 -07:00