Skip to content

Repository files navigation

selfsource: debuginfo for agents

Go Reference CI License

Embed a Go module's source, docs, and a build-time code index in the binary itself. The binary can then answer questions about its own code through a CLI, a web browser, MCP tools, or an AI chat loop.

Stack traces and pprof tell you where. Buildinfo tells you which build. Selfsource carries the what: the source text, doc comments, and cross-references for exactly that build.

Ask the binary about itself:

$ ./toy source chat
I'm github.com/oplehto/selfsource-go/examples/toy, AMA! (Ctrl-D to exit)
you> Where is the crash verb implemented, and what does it actually do?
  [search {"pattern":"crash"}]
  [read_file {"path":"main.go"}]
The crash verb is dispatched at main.go:63-64 and implemented by crash()
at main.go:78-82. It deliberately panics: g is a nil *Greeting and
g.Message() dereferences it ...

Or hand it its own crash:

$ ./toy crash 2>&1 | ./toy source report
binary   vcs.revision 95ba829... — MATCHES bundle
#0 main.(*Greeting).Message
   greet.go:11
       10 | func (g *Greeting) Message() string {
   >   11 | 	return "Hello, " + g.Name + "!"
   doc: Message returns the greeting text. ...

No repo checkout needed. And no guessing whether the source matches the binary: the bundle records the git revision it was generated at, and every surface checks it against the binary's own debug.ReadBuildInfo().

Quick start

go install github.com/oplehto/selfsource-go/cmd/selfsource@latest   # the CLI
go get github.com/oplehto/selfsource-go                             # the library

Integration is three pieces: generate, embed, expose.

// 1. at build time:  selfsource gen -root . -o bundle/app.selfsource

// 2. embed it (bundle/.gitkeep committed, the blob gitignored,
//    so a bare `go build` compiles before the bundle exists)
//go:embed all:bundle
var bundleFS embed.FS

func init() { auto.Enable(bundleFS, "bundle/app.selfsource") }   // 3a. always-on

case "source":                                                   // 3b. explicit verb
    r, _ := selfsource.Decode(mustRead(bundleFS, "bundle/app.selfsource"))
    return cli.Run(r, os.Args[2:])

examples/toy is the reference integration. make demo runs its whole flow: generate, build, crash, annotated self-report. docs/case-studies/ walks the same integration through three real programs — scc, hey, and croc — with captured output, each contrasted against doing the same job without selfsource: profiles that confirm a doc comment's hot-path claim versus pprof's source-path scavenger hunt, a live goroutine dump annotated back to source versus reading it by hand, and the secret scanner earning its keep.

A missing bundle never breaks the host. Every surface reports "no bundle embedded" instead.

The surfaces

cli.Run mounts the full verb set under one subcommand. The standalone CLI reaches the same verbs with selfsource -b app.selfsource <verb>.

Verb Does
info Manifest, counts, binary-vs-bundle revision check
ls / cat / grep List, read (loose path match), regexp search
sym / refs / deps [-r] Symbol detail, reference sites, import graph
extract [-o dir] Write the bundled tree to disk, for pprof -source_path, diffing, editors
report [-C n] Annotate a stack trace from stdin with source and docs
profile Analyze pprof profiles against the embedded source: top functions, per-line listings, diffs, self-collection
serve HTTP source browser; loopback default, -token required elsewhere
mcp The same queries as MCP tools over stdio, plus workflow prompts
chat Agent loop against your LLM endpoint (-api-key, -base-url, -model / $SELFSOURCE_MODEL)
skill Emit an agent skill (SKILL.md) for driving this binary, with its real path filled in

Profiles

The profile verb reads pprof profiles — the format runtime/pprof and net/http/pprof produce — and joins the samples against the bundle, so the hot paths come back with symbol kinds, signatures, doc comments, and annotated source lines instead of bare function names:

profile -cpu <file>           top functions, joined to the bundle
profile -heap <file>          same, for heap profiles
profile -collect <duration>   CPU-profile THIS process, then analyze
profile -cpu <file> -list <func>   pprof-style annotated listing, no -source_path
profile -diff <a> <b>         normalized per-function delta, regression triage

The decoder is hand-rolled stdlib (gzip plus the profile.proto subset Go emits) — the no-dependencies rule holds. Frames join by fully-qualified function name against the symbol index first; file-path suffix matching is the fallback, with -trimpath builds as the happy path, and a path match is only trusted when the file actually declares the function. Profiles carrying a GNU build ID are checked against the running binary and a mismatch warns loudly; on platforms where profiles carry no build ID (most), the existing bundle-vs-binary revision check is the identity story. make demo-profile shows the flagship loop: the selfsource CLI, carrying its own source, profiles itself and explains its own hot paths. The same queries are MCP tools (hot_functions, annotate_frame, profile_diff, collect_profile), response-capped so an agent asks narrow questions instead of drowning in samples.

Prompts and skills

The binary carries its own debugging expertise, not just tools. The MCP server advertises four workflow playbooks as prompts — diagnose_crash, hot_paths, perf_regression, orient — which MCP clients surface as slash commands; each one walks an agent through the right tool sequence, starting with the identity check. The chat verb folds the same discipline into its system prompt, and both frontends now share one tool set (the MCP server's), including annotate_stack for crash triage and the profile tools.

skill goes the other direction: the binary writes an agent skill describing itself —

./yourapp source skill -o ~/.claude/skills/yourapp-debug

— a ready-to-use SKILL.md with the binary's real path, verbs, and workflows, so a coding agent on the same machine knows how to interrogate the deployed build without being told.

auto.Enable is the net/http/pprof of selfsource. One call in init() gives every build three always-on surfaces:

  • /debug/source/ on http.DefaultServeMux. The browser is live wherever the host already serves its debug mux.
  • SELFSOURCE_MCP=1 ./yourapp serves MCP over stdio instead of running the app. Any integrated binary drops into an MCP client config as-is.
  • SELFSOURCE_SERVE=<addr> ./yourapp runs the browser standalone. Loopback only, unless SELFSOURCE_TOKEN is set.

Everything is compiled in either way; the environment picks what is live. SELFSOURCE=off is an operator kill switch that keeps every surface dormant (the bundle is not even decoded). auto.EnableOptIn is the inverse posture: zero always-on debug surface until SELFSOURCE=1 or one of the activation variables asks — for production binaries that want the capability one env var away instead of always mounted. Gating governs runtime exposure only: the bundle still ships inside the binary, and anyone holding the file can extract it.

The browser looks like this (/debug/source/, or serve, or SELFSOURCE_SERVE):

The source browser: packages, symbols with definition links, files

Every surface is read-only by construction. HTML output is escaped. The browser refuses non-loopback binds without bearer auth. MCP is stdio-only, so the ability to spawn the process is the access control. Chat talks only to the endpoint you point it at.

What goes into a bundle

selfsource gen -root . -o app.selfsource produces a deterministic blob. Generation at the same commit is byte-identical. Revision and build time default to the current git commit, with a warning on a dirty tree (a dirty bundle matches no commit).

Three things ship: files, the index, and the manifest.

Files. The default is a narrow allowlist: **/*.go, go.mod, build files, the license, and published docs by name (README, CHANGELOG, and so on). Deliberately not **/*.md, which would sweep up TODO.md and internal notes.

To customize, drop a .selfsource file in the module root. It works like gitignore: one glob per line, ! excludes, # comments.

# what goes into the bundle
**/*.go
**/*.md
go.mod
!vendor/**
!**/testdata/**

Generation prints what shipped and paste-ready globs for everything it skipped (-list / -list-skipped). Widening the list is a copy-paste, not archaeology.

The index. Always covers the whole module, whatever the file globs say. Every package-scope symbol, exported or not, including interface and aliased-type methods: kind, signature, doc comment, definition, and all reference sites as file:line:col (test files included). Plus the package import graph, both directions.

The index is computed by the same type checker as the build, so it cannot drift from the code. It can only age with the binary, which is what you want from debuginfo. Unexported symbols matter because panic frames usually are.

The manifest. Module path, revision, build time, Go toolchain, and the resolved dependency list (path@version).

What never ships

Four safety mechanisms hold regardless of your globs:

  • Dot directories are never descended (.git, .aws, .ssh, ...).
  • Symlinks are never followed. A link named notes.go pointing at ~/.aws/credentials is skipped, not read.
  • Credential-shaped paths never ship (.env, *.pem, *.key, id_rsa, *.tfstate, kubeconfig, ...) unless you pass -unsafe-include-all.
  • Included content is scanned for credential shapes: cloud and API keys, private key blocks, tokens in URLs. A finding aborts generation with masked excerpts. Override per line with a selfsource:allow-secret comment, or wholesale with -allow-secrets.

The build host never leaks either. No absolute paths, user names, or GOPATH appear anywhere in a bundle; TestNoHostPathsLeak holds the pipeline to it.

Design

Everything is a library function. The binaries are thin wrappers, and a host picks the pieces it wants:

Package Provides
selfsource (root) DecodeReader: files, search, symbols, references, import graph; ExecTool, AnnotateStack, Extract
gen The build-time generator (the one place golang.org/x/tools is used)
cli Run(r, args), the verb set above
browser The HTTP browser, separate so hosts that only query never link net/http
mcpserver The MCP server: a ~200-line stdio JSON-RPC implementation, no SDK
auto The init()-time integration
cmd/selfsource Standalone CLI: gen plus -b bundle <verb>

Every runtime piece is stdlib-only. The chat client is one HTTP POST. The module's single dependency (golang.org/x/tools) belongs to the generator and is never linked into a host.

Measured on the toy: a reader-only host is about 3.6 MB, of which 2.5 MB is the Go runtime any binary carries (the reader itself adds about 1.1 MB). The everything-integration is about 12.8 MB.

Why

An operator, or an agent, has only the deployed binary and a panic from its logs. No repo, and no doubt which commit:

  1. ./app crash-log | ./app source report resolves every frame to source and docs from the build that crashed.
  2. ./app source sym Foo / refs Foo explores the implicated code paths.
  3. pprof says something is hot? ./app source profile -cpu cpu.pb names the hot functions with their docs and definition sites, and -list <func> is the annotated per-line listing — straight from the binary, no external tooling. (d=$(./app source extract) && go tool pprof -source_path="$d" ... still works as the fallback when you want pprof's own UI.)

Prior art: zipizap/EmbeddedSource (GPL-3.0) embeds a directory of .go files and extracts them at runtime. The extract verb here owes it the reminder that plain files on disk are what existing tools consume. Selfsource adds the code index, deep trees via glob config, the identity check, and the browser/MCP/chat/report surfaces.

Development

make check    # fast gate: gofmt + vet + tests
make ci       # everything CI runs: check + race + staticcheck + govulncheck + fuzz smoke + example
make example  # generate the toy's bundle and build the toy
make demo     # example + crash → annotated self-report

Pre-1.0, the API may still move between minor versions; changes are called out in CHANGELOG.md. The bundle format is versioned independently (selfsource/v1 in the manifest). Readers reject formats they don't know rather than guessing.

License

Apache License 2.0; see LICENSE and NOTICE. Contributions are accepted under the same license (Apache-2.0 § 5, inbound = outbound). No CLA, no sign-off ceremony.

About

Give a Go binary its own source code and docs, so you or an AI agent can ask it to explain its code, its crashes, and its hot paths.

Topics

Resources

Stars

1 star

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages