Skip to content

Repository files navigation

Three Gates

Formal certificates and empirical audits for a question that current AI safety practice mostly leaves implicit: who can actually pause a deployed multi-agent system, and what must they know to do it?

Motivation

When many principals (developers, operators, platform owners, regulators) jointly oversee a population of interacting agents, no single one of them holds enough information to decide a shutdown alone. This project decomposes the distributed-pause problem into three necessary gates — detection (someone sees the danger), attribution (someone can report which agents/behaviors are responsible), and intervention (the system can be selectively stopped) — and asks what each principal must hold, formally, for a legitimate pause decision to exist. Observed real-world safety failures each map onto exactly one failing gate.

What is in this repository

  • artifact/ — the frozen research artifact, preserved byte for byte: finite distributed safety certificates, seeded graph and hypergraph monitoring, a stylized reporting game, machine-checkable witnesses, an exact integrated three-gate factorial, and offline audits of public AgentDojo task specifications and stored model trajectories. artifact/SHA256SUMS covers every file; artifact/README.md documents environment setup, tests, and each study. Nothing in it invokes an LLM or runs a live multi-agent system.
  • protocols/ — design memos and portability notes written before the corresponding runs (kept unedited; internal paths refer to the original working tree).
  • visualization_generators/ — figure and dashboard generators (requirements-visualization.txt lists their extra dependencies).

Reproducing

cd artifact
python3 -m venv .venv && source .venv/bin/activate
python3 -m pip install -r requirements.txt
shasum -a 256 -c SHA256SUMS   # integrity check

See artifact/README.md for running the test suite (requires the pinned AgentDojo source on PYTHONPATH) and for per-study entry points.

Integrity note

The artifact/ and protocols/ trees are a hash-attested release: result manifests record SHA-256 digests of inputs, code, and outputs, so their files are intentionally not edited here — including historical paths inside comments and manifests.

Status

Research code release. A paper reference will be added here once available.

License

MIT (see LICENSE). The artifact retains its original internal license file.

About

Formal certificates and empirical audits for pausing multi-principal agent systems: detection, attribution, intervention

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages