Skip to content
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
44 commits
Select commit Hold shift + click to select a range
5a4da12
Fail only on mature rules and levels by default
tauanbinato Sep 28, 2026
abe1bc9
Leave hardcoded values out of the default rules
tauanbinato Sep 28, 2026
199c9bc
Treat a model name without an x.y.z version as an alias, and accept g…
tauanbinato Sep 28, 2026
655894f
Price a run by the model that answered it, and accept answers without…
tauanbinato Sep 28, 2026
d52ae57
Move the transport tests to their own file
tauanbinato Sep 28, 2026
c06d73b
Record request ids, honor retry-after-ms and HTTP dates, and explain …
tauanbinato Sep 28, 2026
e0269be
Pace requests to 1,200 a minute, cap concurrency at 6, time out attem…
tauanbinato Sep 28, 2026
6034aea
Accept OpenRouter and Vercel AI Gateway keys
tauanbinato Sep 28, 2026
f32e7d4
Show a run with no paid requests as $0.0000, not $-0.0000
tauanbinato Sep 28, 2026
6e0ed2d
Say when the provider does not know the model
tauanbinato Sep 28, 2026
e2c29ab
Test the 422 message on TypeSafe's real answer to a request without s…
tauanbinato Sep 28, 2026
48c5264
Check the saved key's prefix too, bill unnamed models to "unknown", n…
tauanbinato Sep 28, 2026
013d30a
Count an empty key in a credential file as unset
tauanbinato Sep 28, 2026
7ffb23d
Find changed files when jevgate.toml sits below the Git top level
tauanbinato Sep 28, 2026
87d446e
Judge only what a change touches with --base
tauanbinato Sep 28, 2026
50f8e7f
Lower a concurrency above 6 to 6 with a notice instead of refusing it
tauanbinato Sep 28, 2026
941e66b
Say in the HTML report why a run's cost is unknown
tauanbinato Sep 28, 2026
26ced90
Make the docs agree on the 0.26 gate, scope and keys
tauanbinato Sep 28, 2026
1c6697e
Write the 0.26 changelog as one section, with what a pull request che…
tauanbinato Sep 28, 2026
71cf70f
Split the mock provider's request handling and share two tests' steps
tauanbinato Sep 28, 2026
2b2bb32
Put a blank line before the configuration page's key section
tauanbinato Sep 28, 2026
a705fcc
Test that a pull request check fails only on what the change touched
tauanbinato Sep 28, 2026
705eab6
Read a gateway's variable after every key given to JevGate
tauanbinato Sep 28, 2026
1b1ba62
Say in the agent text why files failed, as it says why they were skipped
tauanbinato Sep 28, 2026
ade1772
Refuse a Retry-After date whose year is not four digits
tauanbinato Sep 28, 2026
004e6e9
Keep only a checked request id when the body carries its own
tauanbinato Sep 28, 2026
d214eb7
Diff only the files a check judges, and ignore GIT_DIFF_OPTS
tauanbinato Sep 28, 2026
4edce15
Say that concurrency in jevgate.toml above 6 has no effect, and expla…
tauanbinato Sep 28, 2026
2f93b13
List reviews before considers after the findings that fail the gate
tauanbinato Sep 28, 2026
1ec0899
Price OpenRouter's dated Jev 1.13 endpoint
tauanbinato Sep 28, 2026
0e56c5e
Say that law findings were labeled on Bend 2 projects, not "none labe…
tauanbinato Sep 28, 2026
4a19246
Say in --model's short help that the default follows the key
tauanbinato Sep 28, 2026
3da0e48
Name the maintainer's repositories in the pull request numbers and co…
tauanbinato Sep 28, 2026
e0bc674
Split the baseline writer and share the saved-credential test helper
tauanbinato Sep 28, 2026
b061572
Accept as wrong the self-check's review of a hardcoded-values test fi…
tauanbinato Sep 28, 2026
010471d
Send 3 requests at once by default with a gateway's key
tauanbinato Sep 28, 2026
6e77c1c
Retry an overloaded request up to six times, pausing up to 8 s
tauanbinato Sep 28, 2026
83e435b
Keep the tests' Git out of the repository running them
tauanbinato Sep 28, 2026
75deef4
Test that --base reads the repository GIT_DIR names
tauanbinato Sep 28, 2026
6651cec
Tell agents to fix what fails the gate and weigh the rest
tauanbinato Sep 28, 2026
9edaf77
Say on the CI page that --rule alone can switch the gate off
tauanbinato Sep 28, 2026
141458e
Name the gate levels init wrote before 0.26 until their comments go
tauanbinato Sep 28, 2026
ee5b336
Show 0.26's output in the README and state what blocks and what it costs
tauanbinato Sep 28, 2026
91180ad
Hand Git plain paths in the GIT_DIR test on Windows
tauanbinato Sep 28, 2026
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
52 changes: 52 additions & 0 deletions CHANGELOG.md

Large diffs are not rendered by default.

1 change: 1 addition & 0 deletions CONTRIBUTING.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,6 +24,7 @@ The tests run offline and need no API key. `jevgate check --dry-run --show-reque
## Code conventions

- Unused code is deleted, not silenced, and long parameter lists are grouped into a type. `tests/lint_policy.rs` rejects `allow` or `expect` for `dead_code`, `unused`, `too_many_arguments` and `complexity`. Any other exception uses `#[expect(lint, reason = "…")]`.
- Tests run Git through `tests/support/git.rs`, which drops the variables that point Git at a repository (`GIT_DIR`, `GIT_INDEX_FILE` and the others Git exports to hooks and `git rebase --exec`), so `cargo test` run there leaves that repository alone. `tests/lint_policy.rs` rejects starting Git anywhere but there and `src/revision.rs`.
- Commit subjects say what changed, in the imperative ("Report overlapping tests by groups").
- Messages and findings are plain sentences that name the code and say what to do next.

Expand Down
20 changes: 10 additions & 10 deletions README.md
Original file line number Diff line number Diff line change
Expand Up @@ -7,19 +7,19 @@

**JevGate is a code-review gate. It asks small, precise questions about your code and turns the answers into findings you can act on.**

JevGate parses your repository locally and builds small units of evidence: a function, a file outline, a pair of copies, a test, a documentation section. It asks [TypeSafe Jev](https://docs.typesafe.ai) short, typed questions about each one. Code, not a chat model, combines the answers into a verdict. Each finding has a location, a probability and a concrete next step, so an agent or CI job can act on it and a person can check it quickly.
JevGate parses your repository locally and builds small units of evidence: a function, a file outline, a pair of copies, a test, a documentation section. It asks [TypeSafe Jev](https://docs.typesafe.ai) short, typed questions about each one. Code, not a chat model, combines the answers into a verdict. Each finding has a location, a probability and a concrete next step, so an agent or CI job can act on it and a person can check it quickly. By default the gate fails only on the rules and levels measured right at least 80% of the time on projects JevGate was never tuned on: among the default rules, function-simplification reviews, right 20 of the 23 times they were labeled there (87%).

![JevGate's terminal output on zoxide: the gate fails on 3 review findings (a function mixing separate jobs, and two sets of importers repeating the same steps), 3 consider findings (a group of file helpers that could be a module, branching that hides a main path, an unexplained 7) and optional notes on unnamed values](site/src/images/terminal.svg)
![JevGate's terminal output on zoxide: the gate fails on 1 function-simplification review (a function mixing separate jobs); 3 more reviews (a file holding several features, two sets of importers repeating the same steps) and a consider (branching that hides a main path) are reported without failing it, since their rules and levels are still being measured](site/src/images/terminal.svg)

JevGate 0.22.0 on [zoxide](https://github.com/ajeetdsouza/zoxide/tree/09a18b4424b3f1033094ffd97da6d47585e38259), rerun from its answer cache, so it cost nothing; 9 of the 11 notes are left out.
JevGate 0.26.0 on [zoxide](https://github.com/ajeetdsouza/zoxide/tree/09a18b4424b3f1033094ffd97da6d47585e38259), rerun from its answer cache, so it cost nothing.

**[Documentation](https://tech-byte-frontier.github.io/jevgate/)** · [Rules](https://tech-byte-frontier.github.io/jevgate/reference/rules.html) · [Configuration](https://tech-byte-frontier.github.io/jevgate/configuration.html) · [CI](https://tech-byte-frontier.github.io/jevgate/ci.html) · [Troubleshooting](https://tech-byte-frontier.github.io/jevgate/troubleshooting.html) · [Changelog](CHANGELOG.md)

## What it finds

| Group | Rules | On |
|---|---|---|
| Maintainability | File organization, function simplification, shared logic, hardcoded values | by default |
| Maintainability | File organization, function simplification, shared logic; hardcoded values (opt-in) | by default |
| Tests | Test value (mock-only checks, expected values recomputed with the code's own logic), test redundancy | with `--include-tests` |
| Security | Injection, sensitive data, unsafe settings, SQL access control, GitHub workflows; each finding names a CWE | `--rule security` |
| Documentation | Agent instruction files, large and stale docs, duplicated sections, code comments | `--rule documentation` |
Expand All @@ -35,21 +35,21 @@ cargo binstall jevgate # any platform, with cargo-binstall
cargo install jevgate --locked # build from source; needs Rust 1.90 or later
```

Releases have binaries for Linux, macOS and Windows with checksums and build provenance. Reviewing needs a [TypeSafe API key](https://console.typesafe.ai/settings/keys). [Install](https://tech-byte-frontier.github.io/jevgate/install.html) covers verifying a download, shell completions and man pages.
Releases have binaries for Linux, macOS and Windows with checksums and build provenance. Reviewing needs an API key from [TypeSafe](https://console.typesafe.ai/settings/keys) or [OpenRouter](https://openrouter.ai/settings/keys), which serve the same model at the same price; a [Vercel AI Gateway](https://vercel.com/docs/ai-gateway/authentication-and-byok/api-keys) key is accepted too, but has not been tried with a real key yet. [Install](https://tech-byte-frontier.github.io/jevgate/install.html) covers verifying a download, shell completions and man pages.

## Quick start

```sh
jevgate init # write a commented jevgate.toml for this repository
jevgate auth login # validate and save your TypeSafe API key
jevgate auth login # validate and save your API key: TypeSafe, OpenRouter or Vercel
jevgate check --dry-run --show-requests # see exactly what would be uploaded; free and offline
jevgate check --report # review, then open a local HTML dashboard
jevgate baseline # accept today's findings; later checks fail only on new ones
```

`jevgate check --report` writes the same findings to a local dashboard you can filter by path and classification, with each file's findings, undecided units and the answers behind them:

![JevGate's HTML report on zoxide: totals for files, review and consider findings, notes and cost, then a list of files by classification, with src/util.rs open to show its review and consider findings, their next steps and one undecided unit](site/src/images/report.png)
![JevGate's HTML report on zoxide: the gate's result and what fails it by default, totals for files, findings, notes and cost, then a list of files by classification, with src/util.rs open to show its two review findings, the one that fails the gate marked, and how each rule classified the file](site/src/images/report.png)

`jevgate --help` gives the workflow, exit codes, files and environment, and `jevgate check --help` explains each flag and the JSON report. Coding agents can also call JevGate as a tool through its MCP server, `jevgate mcp` ([coding agents](https://tech-byte-frontier.github.io/jevgate/coding-agents.html)).

Expand All @@ -75,7 +75,7 @@ jobs:
version: 0.25.0
```

It reviews only the changed files, annotates each finding on its line and writes a job summary; unchanged code is answered from the cache for free. [Continuous integration](https://tech-byte-frontier.github.io/jevgate/ci.html) covers pre-commit, other CI systems, pull requests from forks, budgets and a gate policy the change cannot edit.
It reviews only what the pull request changed (the functions, tests and comments on changed lines, and copies where either copy changed), annotates each finding on its line and writes a job summary; unchanged code is answered from the cache for free. [Continuous integration](https://tech-byte-frontier.github.io/jevgate/ci.html) covers pre-commit, other CI systems, pull requests from forks, budgets and a gate policy the change cannot edit.

## Output and exit codes

Expand All @@ -87,13 +87,13 @@ It reviews only the changed files, annotates each finding on its line and writes
| 1 | Gate failed |
| 2 | Run incomplete, invalid configuration or invalid usage |

Findings are `review` (act on it), `consider` (worth a look) or `note` (optional). A file whose answers stay undecided is `uncertain`, never hidden or counted as clear. `--fail-on` and `jevgate.toml` set what fails the gate, per rule and per path. `jevgate baseline` accepts today's findings, and a `jevgate: allow(RULE) reason` comment accepts one where it is.
Findings are `review` (act on it), `consider` (worth a look) or `note` (optional). A file whose answers stay undecided is `uncertain`, never hidden or counted as clear. By default only the rules and levels measured right at least 80% of the time on projects JevGate was never tuned on fail the gate (`jevgate rules` shows them); the other findings are reported without failing it. `--fail-on` and `jevgate.toml` set what fails the gate, per rule and per path. `jevgate baseline` accepts today's findings, and a `jevgate: allow(RULE) reason` comment accepts one where it is.

## Privacy and cost

- **What is uploaded:** only the selected units of source, bounded by `upload_allow` and `upload_deny`; `--dry-run --show-requests` prints every request body offline.
- **Secrets:** out of scope on purpose, because judging secrets would mean uploading them. Use a local secret scanner.
- **Cost:** every run prints its input tokens and an estimated cost, and cached answers cost nothing.
- **Cost:** every run prints its input tokens and an estimated cost, and cached answers cost nothing, so a rerun of unchanged code sends no request. Jev 1.13 costs $0.042 per million input tokens: checked as pull requests with every rule and nothing cached, the last commits of 118 corpus projects sent 4.55M first-pass input tokens, $0.19 for all 118.

[Privacy and cost](https://tech-byte-frontier.github.io/jevgate/privacy-and-cost.html) and [limits](https://tech-byte-frontier.github.io/jevgate/limits.html) say more, and [how it works](https://tech-byte-frontier.github.io/jevgate/how-it-works.html) explains the evidence units and how code turns answers into findings.

Expand Down
25 changes: 23 additions & 2 deletions docs/classification-cascade.md
Original file line number Diff line number Diff line change
Expand Up @@ -132,6 +132,22 @@ signatures, or one candidate pair.
a full pass but saved a half to two thirds as much per edit, and left
whole files in one run (`clones.rs`, `literals.rs`); one in four
overtakes it after 31 to 40 edits.
With `--base`, only what the change touched is asked: units whose lines
it added or modified, or removed lines inside; copies where either copy
changed; a file's outline, or a large document's, only when the change
adds a member or heading its base version lacks; and a document it left
alone only in a section that names a path it deleted or renamed. The
touched functions of one run share a pack, in runs that end where they
end for the whole file, so no other pack is sent. A later push that
changes another function of the run adds it to that pack, which is asked
again whole: on 16 corpus projects whose last two commits edit the same
file, the second push re-asked 93 units the first had asked, in 45 of
its 575 new packs and 1% of the bytes it sent (judging whole files, 363
units in 129 of 759 packs, 3%). A unit asked beside other functions can
answer differently: of 11,693 first-pass answers about the same units on
the corpus's last commits, 88% were the same as with whole-file packs,
the others moved 0.03 on average, and 36 crossed 0.50 or 0.80 (13 up, 23
down), which made three function-simplification considers notes.
Tests are sent one per request, because unrelated
tests in the same state left more answers undecided. State uses literal paths
such as `functions[2].source`; group IDs are Choice options. Stage and freshness
Expand Down Expand Up @@ -692,8 +708,13 @@ signatures, or one candidate pair.
rather than splitting it, so a consider left naming no group is a note
and a review says to split the whole file.
6. **Gate.** `--fail-on`, `[[scope]]` levels per path and the baseline act on
composed findings only. Baseline entries can carry a reason (`intended`,
`later`, `wrong`) that survives rewrites; `baseline stats` counts them.
composed findings only. The default level, `mature`, fails only on the
rules and levels whose findings were right at least 80% of the time on
projects never used for tuning, over at least 20 hand labels
(`maturity::TABLE`); a probability says how sure an answer is, not how
often such findings are right. Baseline entries can carry a reason
(`intended`, `later`, `wrong`) that survives rewrites; `baseline stats`
counts them.

## Constraints

Expand Down
14 changes: 12 additions & 2 deletions jevgate-baseline.json
Original file line number Diff line number Diff line change
@@ -1,5 +1,15 @@
{
"version": 1,
"created_at": 1790514454,
"findings": []
"created_at": 1790587702,
"findings": [
{
"fingerprint": "62ce38c353079b7b9773080b905d226e8d8e473ef040d8b1b828e25cca8fc4f1",
"rule": "maintainability/hardcoded-values",
"path": "src/units/tests/mod.rs",
"line": 312,
"strength": "review",
"message": "One of this file's constants fixes a value that differs between deployments (0.98). The constant is `HARDCODED`.",
"reason": "wrong"
}
]
}
14 changes: 9 additions & 5 deletions jevgate.schema.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,7 @@
"enum": [
"review",
"consider",
"mature",
"uncertain",
"report",
"none",
Expand All @@ -18,6 +19,7 @@
"enum": [
"review",
"consider",
"mature",
"uncertain",
"report",
"none",
Expand Down Expand Up @@ -168,6 +170,7 @@
"enum": [
"review",
"consider",
"mature",
"uncertain",
"report",
"none"
Expand Down Expand Up @@ -260,13 +263,13 @@
"description": "JevGate configuration. The command line wins over the file, except that upload patterns and budgets in the file are ceilings that flags can only narrow. Unknown keys are errors.",
"properties": {
"cache_ttl_secs": {
"description": "Cache lifetime in seconds for the `jev-latest` and `jev-preview` aliases; pinned versions never expire. Default: 3600.",
"description": "Cache lifetime in seconds for an alias, a model name without an x.y.z version such as `jev-latest`; pinned versions never expire. Default: 3600.",
"minimum": 0,
"type": "integer"
},
"concurrency": {
"description": "Ceiling on simultaneous requests (1-8). Default: 6.",
"maximum": 8,
"description": "Most simultaneous requests; flags can only lower it. JevGate sends at most 6 at once, so a higher value means 6. Default: 6 with a TypeSafe key, 3 with an OpenRouter or Vercel AI Gateway key.",
"maximum": 6,
"minimum": 1,
"type": "integer"
},
Expand All @@ -278,11 +281,12 @@
"type": "array"
},
"fail_on": {
"description": "The level for rules without their own, like `--fail-on`. Default: [\"review\"].",
"description": "The level for rules without their own, like `--fail-on`. Default: [\"mature\"], which fails only on the levels of a rule measured right at least 80% of the time on projects JevGate was never tuned on; `jevgate rules` shows them.",
"items": {
"enum": [
"review",
"consider",
"mature",
"uncertain",
"report",
"none"
Expand Down Expand Up @@ -318,7 +322,7 @@
"type": "integer"
},
"model": {
"description": "TypeSafe model; a pinned version keeps results repeatable. `--model` overrides it.",
"description": "Model, as the key's provider names it; a pinned version keeps results repeatable. `--model` overrides it. Default: `jev-1.13.0` with a TypeSafe key, `typesafe/jev-1.13` with an OpenRouter key, `typesafe-ai/jev` with a Vercel AI Gateway key.",
"type": "string"
},
"rules": {
Expand Down
32 changes: 28 additions & 4 deletions site/generate.py
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@

COMMANDS = ["auth", "check", "baseline", "rules", "init", "completions", "man", "serve", "mcp"]
GROUPS = {
"maintainability": "On by default.",
"maintainability": "On by default, except hardcoded values: add it with `--rule default --rule hardcoded-values`, or a level for it in `[rules]`.",
"tests": "On by default. Test value and test redundancy are judged with `--include-tests` or `include_tests = true`; the laws of Bend 2 code are judged without it.",
"security": "Opt-in: `--rule security`, or a level in `[rules]`.",
"documentation": "Opt-in: `--rule documentation`, or a level in `[rules]`.",
Expand All @@ -36,14 +36,24 @@ def rules_page(binary):
"its key or its group anywhere a rule is accepted: `--rule`, `--skip-rule`,",
"`--fail-on TARGET=LEVEL`, `[rules]`, `[[scope]]` and `jevgate: allow(…)` comments.",
"",
"| Rule | Key | Default | Question |",
"|---|---|---|---|",
"*Fails by default* names the levels that fail the check under the default gate level,",
"`mature`: those right at least 80% of the time over at least 20 findings labeled by hand on",
"projects JevGate was never tuned on; an opt-in rule's levels fail it once the rule is selected.",
"The other findings are reported without failing it.",
"*Reviews right* and *considers right* give the share of labeled findings that were right on",
"those projects, and how many were labeled; a debatable one counts as not right.",
"`tests/laws` is labeled only on Bend 2 projects, which these numbers leave out.",
"",
"| Rule | Key | Default | Fails by default | Reviews right | Considers right | Question |",
"|---|---|---|---|---|---|---|",
]
for rule in rules:
anchor = rule["id"].replace("/", "-")
default = "yes" if rule["default_enabled"] else "opt-in"
maturity = rule["maturity"]
lines.append(
f"| [`{rule['id']}`](#{anchor}) | `{rule['key']}` | {default} | {cell(rule['inspection'])} |"
f"| [`{rule['id']}`](#{anchor}) | `{rule['key']}` | {default} | {blocks(maturity)}"
f" | {right(maturity, 'review')} | {right(maturity, 'consider')} | {cell(rule['inspection'])} |"
)
group = None
for rule in rules:
Expand Down Expand Up @@ -78,6 +88,20 @@ def rules_page(binary):
return "\n".join(lines) + "\n"


def blocks(maturity):
"""The levels that fail the default gate, or "no"."""
return ", ".join(level for level in ("review", "consider") if maturity.get(level, {}).get("mature")) or "no"


def right(maturity, level):
""""87% of 23": labeled findings of a level right on unseen projects, or "-"."""
unseen = maturity.get(level, {}).get("unseen")
if not unseen or not unseen["labeled"]:
return "-"
percent = (200 * unseen["right"] + unseen["labeled"]) // (2 * unseen["labeled"]) # half up, as JevGate rounds
return f"{percent}% of {unseen['labeled']}"


def configuration_page(schema_path):
schema = json.loads(Path(schema_path).read_text())
lines = [
Expand Down
Loading
Loading