Skip to content

feat(import): add CodeTraceBench bulk conversion - #57

Merged
drewstone merged 1 commit into
mainfrom
feat/codetracebench-import
Jul 30, 2026
Merged

feat(import): add CodeTraceBench bulk conversion#57
drewstone merged 1 commit into
mainfrom
feat/codetracebench-import

Conversation

@drewstone

Copy link
Copy Markdown
Contributor

What changed

  • add traces import-codetracebench for official CodeTracer-normalized trajectories
  • preserve upstream step ids as step-<n> OTLP spans without importing labels
  • reject annotation leakage, unsafe paths, duplicate ids, malformed rows, and mismatched step counts
  • publish atomically with an adjacent ownership lock and a receipt containing input and output SHA-256 hashes
  • expose the importer through the SDK and document the CLI entry point

Why

agent-eval analyst-benchmark needs one deterministic, label-free trace per public trajectory. This imports the maintained upstream normalization output instead of duplicating CodeTracer parsers.

Proof

  • pnpm typecheck
  • pnpm test: 394/394 tests passed across 34 files
  • pnpm build
  • pnpm check:package: packed CLI and 2 skills passed
  • git merge-tree --write-tree origin/main HEAD
  • git diff origin/main...HEAD --check
  • two concurrent importers: exactly one output owner; the loser writes no output files
  • 32 public compatibility cases: 32 traces, 935 action steps, 1,897 spans, zero label leaks or hash mismatches

The 32-case input is a compatibility-filtered calibration set, not a leaderboard sample.

@drewstone
drewstone merged commit e413600 into main Jul 30, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant