Detect Otto and GitHub Copilot as calling agents - #249
Conversation
|
The Type checking job failure here is pre-existing and unrelated to this PR. It fails on I reproduced it on an untouched checkout of current The cause is the job itself. Two ways to fix, whichever the owners prefer: run it as |
`Type checking` fails on `main` as it stands. Any PR touching Python in this repo hits it — [#249](#249) is how I found it. I reproduced it by running the job's own command against an untouched checkout of `origin/main`: ``` error[unresolved-import]: Cannot resolve imported module `click` --> cli.py:26:8 ``` `main` reads green only because its last CI run was 2026-07-20, before the ty release that changed this. ## What broke The job at `.github/workflows/ci.yml:85-96` runs `uvx ty check`, unpinned, so every run picks up the latest ty. ty `0.0.62` (2026-07-21) shipped this, under Configuration: > Respect `rules` and `analysis` in PEP 723 script metadata configurations `cli.py` carries a PEP 723 inline metadata block. As of that release the block configures the file, and `ty.toml` no longer reaches it. The `unresolved-import = "ignore"` rule stopped applying and the `click` import surfaced as an error. Bisected: | ty | result | |---|---| | `0.0.61` | passes | | `0.0.62` | fails ← behaviour change | | `0.0.65` (what CI pulls today) | fails | ## What this does Repeats the suppression in the script's own metadata, which is where ty now reads it from, plus a note in `ty.toml` so the next person understands the duplication. The dependency is not actually missing — `cli.py` declares `click>=8.0.0` in its PEP 723 header and runs via `uv run cli.py`. ty does not install dependencies, which is the reason `ty.toml` suppressed the rule in the first place. This keeps that decision, it just states it where ty will now look. `cli.py` is the only PEP 723 script in the checked directory, so it is the only file affected. ## Testing `uvx ty check` passes on ty `0.0.65` and on `0.0.61`, so it works either side of the change. `uv run cli.py --help` still works, confirming the added block does not disturb PEP 723 execution. ## One thing for the owners to decide I did not pin ty, because that is a policy call rather than part of the fix. Worth considering though: an unpinned tool in CI means an upstream release can turn `main` red without anyone touching the repo, which is exactly what happened. A pin would make the job reproducible at the cost of someone bumping it. Happy to add it if you want. 🤖 Generated with [Claude Code](https://claude.com/claude-code) https://claude.ai/code/session_01NKSW26NfBtfq8hVeeLdfLj Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
…rkers _detect_invocation_context reads a fixed list of environment variables to work out which coding agent ran the command. Otto was missing, so the af commands Otto runs came through with no agent at all and looked like a person typing. Otto drives af heavily, so this is a real gap. Otto passes its environment to children untouched, so an Otto session started from inside another agent still carries that agent's marker. OTTO goes first in the dict, ahead of CLAUDECODE and the rest, because Otto is the nearer caller. A matching change in astronomer/otto sets it. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NKSW26NfBtfq8hVeeLdfLj
Copilot was missing from the list, so any af command run from Copilot looked like a person typing. Two variables carry it. COPILOT_CLI comes from the Copilot CLI, which is also the harness behind the cloud coding agent and, since 2026, Copilot for JetBrains, so one entry covers three surfaces; github's own gh reads the same variable. COPILOT_AGENT is set on the terminals VS Code builds for agent mode, where the source calls it backward compatibility for this kind of detection. AI_AGENT is a newer convention where the value names the agent rather than the variable, so VS Code reports github_copilot_vscode_agent and the desktop app github_copilot_app_agent. It is read after the list, so a specific marker still decides, and only prefixes we know are mapped. Values are not passed through: some vendors put a version in theirs, which would spread one agent over dozens of values in the telemetry. Two candidates were dropped on inspection. COPILOT_AGENT_SESSION_ID is in no changelog and gh does not read it. VSCODE_AGENT is named in the VS Code 1.121 release notes but is nowhere in the source; the note is wrong, and other tools have copied the mistake. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01NKSW26NfBtfq8hVeeLdfLj
41526f8 to
80e8d50
Compare
_detect_invocation_contextreads a fixed list of environment variables to work out which coding agent ran the command. Otto is not on it and sets no marker of its own, so everyafcommand Otto runs arrives with noagentand looks like a person typing.This matters more for
afthan forastro. Otto installs its own pinnedafwrapper and puts it first onPATH, so in an Otto session essentially allaftraffic is Otto-driven — and none of it is labelled.There is a second effect. Otto hands its environment to child processes untouched, so a user who starts Otto from inside another coding agent passes that agent's variable straight through to
af, and the commands get credited to it. Some share of theclaude-codevolume inaftelemetry is really Otto.What this does
Adds
"OTTO": "otto"toagent_env_vars, first in the dict. The loop breaks on the first match and dicts keep insertion order, so position is the fix for the second problem: withOTTOahead ofCLAUDECODE, an Otto session started inside Claude Code reportsotto, the nearer caller.Companion changes:
astronomer/otto#379setsOTTO=1;astronomer/astro-cli#2234andastronomer/astro-cli-internal#81do the same for theastroCLI. Until the Otto change ships this is inert.Testing
test_detects_ottoandtest_otto_wins_over_inherited_marker, plusOTTOadded to theclean_envfixture so it cannot leak in from the environment running the suite. 45 tests pass.🤖 Generated with Claude Code
https://claude.ai/code/session_01NKSW26NfBtfq8hVeeLdfLj
Update: also adds GitHub Copilot detection
Second commit on this branch. Copilot was missing from the same list, so anything run from Copilot arrived unlabelled too.
COPILOT_CLIghkeys on it:cli/cliinternal/agents/detect.goCOPILOT_AGENTmicrosoft/vscodetoolTerminalCreator.ts, where it is commented as backward compatibility for this kind of detectionAI_AGENTgithub_copilot_vscode_agent), Copilot desktop app (github_copilot_app_agent)microsoft/vscodeaiAgentEnv.ts, which calls it a wire contractAI_AGENTis a different shape from everything else in the list: the value names the agent, not the variable. It is read after the list so a specific marker still decides, and only prefixes we recognise are mapped.Values are deliberately not passed through, which is where this differs from
gh.ghreturns any value matching^[a-zA-Z0-9_-]+$. Some vendors put a version in theirs — Claude Code's isclaude-code_2-1-156_agent— so passing values through would spread one agent across a new telemetry value on every release and make the column useless for grouping. A new vendor adopting the convention is a one-line addition.Two candidates dropped after checking the sources
COPILOT_AGENT_SESSION_IDis widely cited but appears in no Copilot CLI changelog entry through1.0.76, andghdoes not read it.COPILOT_CLIcovers the same surface with a first-party source behind it.VSCODE_AGENTis named in the VS Code 1.121 release notes but does not exist in the source — onlyVSCODE_AGENT_ZSH_FIXUPSandVSCODE_AGENT_HOST_*. The release note is wrong and at least one other library has copied the error.Also avoided:
COPILOT_MODEL,COPILOT_ALLOW_ALL,COPILOT_GITHUB_TOKEN. Some detectors key on these, but they are user config the CLI reads, so anyone withexport COPILOT_MODEL=…in a shell profile would be misdetected forever.Worth a follow-up, not in this PR
CODEX_API_KEYin the existing list has that same flaw — it is a key the user sets, not a marker Codex exports.CODEX_THREAD_IDorCODEX_SANDBOXare what Codex actually sets per execution, and are whatghreads.