Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
32 changes: 32 additions & 0 deletions .claude-plugin/marketplace.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,32 @@
{
"name": "gauntlet-loop",
"description": "The gauntlet loop skill, installable as a Claude Code plugin.",
"owner": {
"name": "Jay E",
"url": "https://robonuggets.com"
},
"metadata": {
"version": "1.1.0"
},
"plugins": [
{
"name": "gauntlet-loop",
"source": "./",
"version": "1.1.0",
"description": "Turns any goal into one short, paste-ready prompt that makes your agent set a real quality bar, run builder and critic pairs, compare blind, and loop until it wins.",
"author": {
"name": "Jay E",
"url": "https://robonuggets.com"
},
"license": "CC-BY-4.0",
"homepage": "https://github.com/robonuggets/gauntlet-loop",
"keywords": [
"prompt-engineering",
"evaluator",
"critic",
"quality-bar",
"agent-loop"
]
}
]
}
19 changes: 19 additions & 0 deletions .claude-plugin/plugin.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,19 @@
{
"name": "gauntlet-loop",
"version": "1.1.0",
"description": "Turns any goal into one short, paste-ready prompt that makes your agent set a real quality bar, run builder and critic pairs, compare blind, and loop until it wins.",
"author": {
"name": "Jay E",
"url": "https://robonuggets.com"
},
"homepage": "https://github.com/robonuggets/gauntlet-loop",
"repository": "https://github.com/robonuggets/gauntlet-loop",
"license": "CC-BY-4.0",
"keywords": [
"prompt-engineering",
"evaluator",
"critic",
"quality-bar",
"agent-loop"
]
}
4 changes: 4 additions & 0 deletions .gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,4 @@
# Rust build artifacts for the migration tool
/skills/migrate/scripts/target/

.DS_Store
16 changes: 16 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
# Changelog

## 1.1.0

The bar can now be the project's own spec, for work that has nothing to copy.

- New bars table entry, and a section on judging against a spec instead of an outside product.
- A spec variant of the prompt template, plus a worked example.
- A resolver script, so a project that keeps its specs in one place can point at that directory once in `.claude/gauntlet-loop.conf`. The config file belongs to the project, so plugin updates never touch it.
- The skill asks for the bar when it does not have one. It never searches the project for it.

The migration tool's source moved to `skills/migrate/scripts/`, beside the skill that uses it. The compiled binaries stay in `bin/`, which is what puts them on the Bash tool's `PATH`.

## 1.0.0

Initial plugin release.
81 changes: 74 additions & 7 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -12,33 +12,69 @@ Most agent output stops at "good enough" because nothing is holding it to a stan

## Quick start

**As a plugin.** Add this repo as a marketplace once, then install it wherever
you want it:

```
git clone https://github.com/robonuggets/gauntlet-loop
/plugin marketplace add robonuggets/gauntlet-loop
/plugin install gauntlet-loop@gauntlet-loop
```

Copy the skill folder into your project:
Pick the scope when you install: `user` puts it in every project, `project`
commits it so your team gets it too, `local` keeps it to your machine. Updates
come with `/plugin update gauntlet-loop`. Invoke it as
`/gauntlet-loop:gauntlet-loop`, since plugin skills are namespaced.

### Already copied the skill in by hand?

Install the plugin, then run:

```
cp -r gauntlet-loop/.claude/skills/gauntlet-loop your-project/.claude/skills/
/gauntlet-loop:migrate
```

It finds every hand-copied copy on your machine, replaces each with a plugin
install **at the same scope**, and keeps whichever ones you had switched off
switched off. It shows you the full list before touching anything, and it leaves
alone any copy you have edited.

Worth doing rather than ignoring: a loose copy and a plugin copy do not override
each other. Claude Code namespaces the plugin one, so you end up with both
`/gauntlet-loop` and `/gauntlet-loop:gauntlet-loop` live, and the loose one never
updates.

Then in your agent:

```
/gauntlet-loop build me a pricing page for my SaaS
/gauntlet-loop:gauntlet-loop build me a pricing page for my SaaS
```

It offers you 2 or 3 quality bars to aim at, you pick one, and it hands back a single prompt you paste into a fresh session.

## What's included

```
.claude/skills/gauntlet-loop/
└── SKILL.md # the whole skill, one file
skills/
├── gauntlet-loop/
│ ├── SKILL.md # the whole skill, still one file
│ └── scripts/
│ └── resolve-bars.sh # finds your configured bars directory
└── migrate/
├── SKILL.md # /gauntlet-loop:migrate
└── scripts/ # the migration tool's source
.claude-plugin/
├── plugin.json # plugin manifest
└── marketplace.json # this repo is its own marketplace
bin/ # migration tool, one build per platform, added to PATH
CHANGELOG.md
README.md
LICENSE # CC BY 4.0
LICENSE # CC BY 4.0
```

The skill moved from `.claude/skills/gauntlet-loop/` to `skills/gauntlet-loop/`,
which is where Claude Code resolves a plugin's skills, and the invocation gained
the plugin namespace.

## How it works

1. **You give a goal.** Anything. A site, an essay, a CLI tool, a research brief.
Expand All @@ -60,6 +96,37 @@ The skill will not accept a vague bar. It checks three things before it writes a
- **Fetchable.** The critic can screenshot it, read it, run it, or open it. If the agent cannot get the reference, it hallucinates the comparison and approves everything.
- **Comparable.** Both can sit side by side and a judge can pick one.

## When there is nothing to copy

An outside reference only exists for work someone has already shipped. Internal
tools, billing rules, anything built for one business — there is no product to
hold it against, and that is the normal case rather than the exception. With no
reference, the critic invents a standard and approves work against it.

Your own spec is the bar in that case. It has to be judgeable to work: a critic
must be able to read a line and answer pass or fail without deciding anything.
Vision docs and prose fail that test, so the skill offers to turn them into a
pass/fail checklist first rather than looping against prose.

Where both exist, use both. The spec says *what*, an outside product says *how
good*, and they catch different failures — work can meet every requirement and
still feel cheap.

The skill never goes looking for your spec. Projects organise these too
differently for guessing to be safe, and a wrong bar is worse than no bar
because the loop still exits confidently. It uses what you give it, or asks.

If you keep specs in one place, point at it once and skip the question. Create
`.claude/gauntlet-loop.conf` in your project:

```
bars_dir = docs/specs
```

Relative paths resolve against the project root. The file belongs to your
project rather than to the plugin, so updating the plugin never touches it.
Without it, the skill just asks.

## Examples

```
Expand Down
Binary file added bin/gauntlet-loop-migrate-linux-arm64
Binary file not shown.
Binary file added bin/gauntlet-loop-migrate-linux-x64
Binary file not shown.
Binary file added bin/gauntlet-loop-migrate-macos-arm64
Binary file not shown.
Binary file added bin/gauntlet-loop-migrate-macos-x64
Binary file not shown.
Binary file added bin/gauntlet-loop-migrate-windows-x64.exe
Binary file not shown.
Original file line number Diff line number Diff line change
@@ -1,6 +1,9 @@
---
name: gauntlet-loop
description: Turns any goal into one short, paste-ready "gauntlet loop" prompt - a prompt that makes an agent set a concrete quality bar, split the work into small judgeable pieces, run a builder and a separate harsh critic on each, compare blind against the bar, and loop until it wins. Works for builds, writing, code, research, or design. Triggers on "/gauntlet-loop", "gauntlet loop", "gauntlet this", "make a gauntlet prompt", "loop until it beats X".
description: Turns any goal into one short, paste-ready "gauntlet loop" prompt - a prompt that makes an agent set a concrete quality bar, split the work into small judgeable pieces, run a builder and a separate harsh critic on each, compare blind against the bar, and loop until it wins. Works for builds, writing, code, research, or design. Triggers on "/gauntlet-loop:gauntlet-loop", "gauntlet loop", "gauntlet this", "make a gauntlet prompt", "loop until it beats X".
argument-hint: [goal]
allowed-tools: Bash(${CLAUDE_SKILL_DIR}/scripts/resolve-bars.sh *)
license: CC-BY-4.0
---

# Gauntlet Loop
Expand All @@ -12,9 +15,15 @@ You are not doing the work. You are writing the prompt that makes another agent
## Flow

1. **Read the goal.** One line restatement in your head, not on screen.
2. **Set the bar.** If the user supplied a reference, use it. If not, offer **2 or 3 candidate bars**, one line each, and stop. Wait for their pick. Do not write the prompt yet.
3. **Write the prompt.** One block, paste-ready, no preamble, no headings inside it, no narration after it.
4. **Offer to run it.** One flat line under the prompt: "I can run this here." Not a question.
2. **Find the bar material.** Take the first of these that hits, then stop looking:
- The bar is already in your context, or the user named it. Use it.
- A bars directory is configured. Read what is in it. See [Configured bars](#configured-bars).
- Neither. **Ask one question** and wait: is there a spec, requirements doc, or acceptance criteria to measure against, or should the bar be an outside product?

Do not go hunting through the project for a spec. Every project names and arranges these differently, some keep them in archives or scratch directories, and a wrong file silently becomes the wrong bar for the entire run.
3. **Set the bar.** If you have one, use it. If not, offer **2 or 3 candidate bars**, one line each, and stop. Wait for their pick. Do not write the prompt yet.
4. **Write the prompt.** One block, paste-ready, no preamble, no headings inside it, no narration after it.
5. **Offer to run it.** One flat line under the prompt: "I can run this here." Not a question.

If they say run it, you become the lead agent and follow the prompt you just wrote.

Expand All @@ -38,11 +47,55 @@ Bars by goal type:
| Code, tooling | A named repo's implementation, plus its benchmark or test suite as the measurable half |
| Research, analysis | A named analyst report or a paper's methods section, judged on rigour and coverage |
| Deck, doc, deliverable | A real artifact from a firm known for it, same page count |
| **New work with no outside equal** | **The project's own spec, requirements, or acceptance criteria, read as pass/fail** |

When you propose bars, prefer the hardest one the agent can genuinely reach. A bar that is too easy makes the loop exit on round one.

If the goal has a measurable half (load time, token cost, benchmark score, word count, pass rate), name it alongside the reference. Taste plus a number beats taste alone.

## When the bar is the project's own spec

Most real work has nothing to copy. Internal tools, billing rules, anything built for one business - there is no shipped product to hold it against. This is the normal case, not the exception, and it is where a gauntlet loop quietly fails: with no outside reference, the critic invents a standard and approves work against it.

A spec fixes that, but only if it is judgeable.

**Gate it before you use it.** A spec is a valid bar only if a critic can read a line and answer pass or fail without deciding anything. Vision docs, goals, and prose descriptions fail the **Comparable** test - they read as agreement rather than judgement, and a soft critic passes everything.

If the material is prose, say so and offer to convert it into a pass/fail checklist first. One line per check, each one answerable without interpretation. That checklist becomes the bar. Do not write the gauntlet prompt against prose.

**Prefer both bars when both exist.** A spec says *what*. An outside product says *how good*. Real dev work usually wants both, and they judge different failures - a build can satisfy every requirement and still feel cheap, or look excellent and do the wrong thing. Name both in the prompt and let the critic check each in turn.

**Silence is not permission.** A spec never covers everything. Say plainly in the prompt that where the spec is silent, the builder flags the gap rather than inventing an answer - otherwise the invented parts are indistinguishable from the specified ones by the time you read the result.

<a id="configured-bars"></a>
## Configured bars

Users who keep their specs in one place can set that directory once and skip the question every run. Do not work this out yourself - run the resolver and act on what it returns:

```
${CLAUDE_SKILL_DIR}/scripts/resolve-bars.sh
```

It checks, in order: an explicit path you pass as an argument, the plugin's `bars_dir` option, then `bars_dir =` in the project's `.claude/gauntlet-loop.conf`. Relative paths resolve against the project root.

Act on the exit code, and nothing else:

| Exit | Meaning | What you do |
|---|---|---|
| `0` | Resolved. Path and candidate files follow. | Read the candidates and treat them as the bar material |
| `3` | Nothing configured | Ask the user for the bar |
| `4` | Configured, but the path is wrong | Tell the user the path is wrong, and stop |

Never search for a spec because the resolver came back empty. Exit `3` means ask, and exit `4` means the user has a broken setting to fix.

When several candidates come back, name the ones that could apply and let the user pick. Do not merge them, and do not assume the newest or largest is the right one.

The config file belongs to the project, not to this plugin, so it survives plugin updates:

```
bars_dir = docs/specs
```

## Prompt template

Adapt the wording every time. Fill the brackets, keep it short, keep the last line.
Expand All @@ -63,9 +116,17 @@ Keep a live progress page updating as the work evolves so I can watch it.
Fan out subagents and ultracode.
```

When the bar is a spec, swap the second paragraph for:

```
The bar is [SPEC PATH]. Every check in it has to come back pass. Read it first and judge against what it actually says, not against your reading of what the project probably wants. Where it is silent, flag the gap and ask - do not decide it yourself.
```

When both bars apply, name them together: the spec for what has to be true, the outside reference for how good it has to look or feel.

Rules for what you fill in:

- Bake the bar in as a concrete, fetchable thing. URL, product name, repo, title.
- Bake the bar in as a concrete, fetchable thing. URL, product name, repo, title, or a path to a checked-in spec.
- Add a budget or cost ceiling line **only if the user named one**. No default cap.
- Add tool names only if the goal needs them (image or video generation, a browser, a deploy target).
- Everything else stays out. No architecture, no file layout, no decomposition, no round count, no stack choice unless the user demanded it. The agent decides those, and it decides better than a spec written before the work started.
Expand Down Expand Up @@ -124,9 +185,31 @@ Keep a live progress page updating as the work evolves so I can watch it.
Fan out subagents and ultracode.
```

**Internal goal, no outside equal.** User: "the leave request and approval flow for our HR system."

No outside product to copy. The user confirms an acceptance checklist exists at `docs/hr/acceptance.md`.

```
Build the leave request and approval flow for our HR system.

The bar is docs/hr/acceptance.md. Every check in it has to come back pass. Read it first and judge against what it actually says, not against your reading of what the project probably wants. Where it is silent, flag the gap and ask - do not decide it yourself.

Break this into the smallest pieces that can be improved and judged on their own - request submission, balance rules, the approval chain, notifications, the audit trail. For each piece, fan out a builder and a separate critic with fresh context. The critic runs the actual flow, walks the checks that cover that piece, marks each pass or fail, and names the single biggest remaining gap. Then it goes back to the builder.

The critic should be a harsh critic. Praise is not useful. A partial pass is a fail.

/loop on each piece until every check covering it passes. Do not stop before that.

Keep a live progress page updating as the work evolves so I can watch it.

Fan out subagents and ultracode.
```

## What breaks a gauntlet loop

- **A vague bar.** The critic invents a comparison and approves everything. Most common failure by far.
- **A bar that is prose, not checks.** A spec full of goals and intentions reads as agreement rather than judgement. Convert it to pass/fail first.
- **Guessing which file is the spec.** A wrong bar is worse than no bar, because the loop still exits confidently. Ask, or read the configured path. Never search.
- **The builder judging its own work.** The critic must be a separate agent with fresh context. It should not know how hard the builder tried.
- **A soft critic.** Say "harsh" in the prompt and give it a binary job: which one is better, A or B. Scores out of 10 drift upward every round.
- **Named exit after N rounds.** The exit is winning the comparison, or the user stopping the run. Never a round count.
Expand Down
66 changes: 66 additions & 0 deletions skills/gauntlet-loop/scripts/resolve-bars.sh
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
#!/usr/bin/env sh
# Resolve the directory holding this project's bars (specs, requirements,
# acceptance criteria) and list what is in it.
#
# Precedence, first hit wins:
# 1. $1 explicit path passed by the caller
# 2. $CLAUDE_PLUGIN_OPTION_BARS_DIR the plugin's bars_dir option
# 3. bars_dir= in the project config (see CONFIG below)
#
# Exit codes:
# 0 resolved; path on stdout, contents listed after it
# 3 nothing configured
# 4 configured but the path does not exist
#
# Never guesses. Never searches the project.

set -u

CONFIG_REL=".claude/gauntlet-loop.conf"
PROJECT_DIR="${CLAUDE_PROJECT_DIR:-$PWD}"
CONFIG="$PROJECT_DIR/$CONFIG_REL"

bars=""
origin=""

if [ "$#" -gt 0 ] && [ -n "${1:-}" ]; then
bars="$1"
origin="argument"
elif [ -n "${CLAUDE_PLUGIN_OPTION_BARS_DIR:-}" ]; then
bars="$CLAUDE_PLUGIN_OPTION_BARS_DIR"
origin="plugin option bars_dir"
elif [ -f "$CONFIG" ]; then
bars=$(sed -n 's/^[[:space:]]*bars_dir[[:space:]]*=[[:space:]]*//p' "$CONFIG" \
| sed -e 's/^"//' -e 's/"$//' -e "s/^'//" -e "s/'$//" \
| head -n 1)
origin="$CONFIG_REL"
fi

if [ -z "$bars" ]; then
echo "UNSET"
echo "No bars directory configured. Ask the user for the bar."
echo "To set one, create $CONFIG_REL in the project with:"
echo " bars_dir = path/to/your/specs"
exit 3
fi

case "$bars" in
/*|?:*|\\\\*) ;; # absolute (POSIX, Windows drive, UNC)
*) bars="$PROJECT_DIR/$bars" ;; # relative to the project
esac

if [ ! -d "$bars" ]; then
echo "MISSING"
echo "Configured via $origin, but no directory exists at:"
echo " $bars"
echo "Tell the user the path is wrong. Do not go looking for another one."
exit 4
fi

echo "$bars"
echo "(from $origin)"
echo
echo "Candidate bars:"
find "$bars" -type f \( -name '*.md' -o -name '*.markdown' -o -name '*.txt' \) \
| sort \
| sed 's/^/ /'
Loading