fix(gate): change-risk 闸判绝对分数,不判会漂的排名 (#201) by 2233admin · Pull Request #202 · 2233admin/code-intel-pipeline

2233admin · 2026-08-05T18:31:44Z

Closes #201

症状

PR #195 连续两轮 CI,代码没变,判决翻了:

	第一轮	第二轮
score	67	68(+1)
percentile	82th (vs last 49 commits)	92th (vs last 48 commits)
结论	🟡 放行	🔴 拦下

变红的不是那个变更,是它的邻居。

机制

compute_percentile 算「样本里多少 commit 分数 ≤ 我」,样本来自 sample_history(repo, tip, N) —— 从当前 tip 往回数 N 个 commit。main 每合一次,分母就换一批人。

两个性质是闸门不该有的:

1. 失败率被写死成 ~10%,与仓库健康无关。 百分位是排名,按定义任何时刻都恰好有约 10% 的 commit 在 90 分位及以上。这道闸因此永远拦下约 10% 的 PR,仓库变得多干净都一样。没人能靠「变好」通过它,只能靠「比邻居差得少」。这是配额,不是阈值。

2. 判据自己已经认过机械性膨胀不算数。 change_risk/mod.rs 里排除「被评范围内的 commit 混进样本」那段的注释原话:

mechanically inflates risk_percentile regardless of whether the change is actually riskier

分母换人是同一个 bug 的另一个源,当时只堵了一个口。

移动靶这个病本仓判过一次死刑:#172 把巨石棘轮从「数量 vs 会漂的 baseline」改成「身份 vs 记录在案的 baseline」,理由就是移动靶给假绿灯。这里是同一个病,方向反过来 —— 给假红灯。

阈值从实测分布读出来

对 origin/main 最近 60 个非 merge commit 逐个跑 change risk <sha>^..<sha>:

n=60   min 11 | p50 57 | p75 66 | p80 67 | p90 68 | p95 69 | max 93   mean 51.08
前 8 名: 93  90  69  68  68  68  68  67

66–69 是稠密簇,69 与 90 之间空无一物。

阈值	拦下
score ≥ 65	18/60 (30%)
score ≥ 68	7/60 (12%)
score ≥ 70…85	2/60 (3%)

70–89 任意位置拦下的都是同样两个真离群值(93、90),放行整个正常簇。取 80,空档正中,两侧留余量。

稠密簇里排名由并列决定:#195 的 68 分跟另外 4 个已合并 commit 并列 —— 82th 还是 92th,取决于分母那批人里恰好几个同分。这正是百分位闸在噪声上翻面的原因。

改了什么

pr-gate.yml:RISK_PERCENTILE_BLOCK: "90" → RISK_SCORE_BLOCK: "80",判 .score
百分位继续出现在 PR 评论里回答「这次比平时如何」——排名适合做这个,不适合当闸
footer 与 ::error:: 文案同步换成 score
阈值推导过程(上面那份 60-commit 实测)写进 workflow 注释,可复算

顺带修了一个会变成停机的坑

比较改在 jq 里做,不用 shell -ge。score 是 JSON number,round2 能吐出小数,[ ... -ge ... ] 会直接报 integer expression expected —— 在 bash -e 下那不是拦也不是放,是把闸门变成一次停机。今天没触发只是因为分数碰巧都是整数。

测试

这个 workflow 此前没有任何测试 —— #201 就是这么住进去没人发现的。

新增 the_pr_gate_blocks_on_the_absolute_score_not_a_moving_rank,钉契约而非字节:

判哪个字段(.score),百分位只叙述、不裁决
阈值必须留在实测空档 70..89
比较必须走 jq

悄悄改回百分位、或把阈值调进稠密簇去放行某个在飞 PR,都会在这里红。

写这条测试时它立刻抓到一处真漏:gate step 里还留着一句提到 risk_percentile 的过期注释。

诚实的代价

拦截率从 ~10% 降到 ~3%,闸变钝了。

接受这个交换的理由:被放掉的那 7%(score 66–69)本来就是靠并列噪声决定生死,拦得不准;真离群值照拦。百分位仍在评论里可见,人要在意排名尖峰仍看得到,只是不再自动挡路。

剩下的漂:绝对分数也会漂 —— churn 90 天窗、bug-magnet 180 天窗都是时间相对的。但那种漂跟踪的是被测文件本身(它真的又挨了几次修),是该漂的;百分位漂跟踪的是别人提交的大小,与被评变更无关。

利益冲突声明

这个改动会让 #195 那类 PR 不再被拦,而 #195 是我自己的 PR。

80 取自实测空档,#195 的 68 落在正常簇内,不是压线放行 —— 但我从中受益是事实。#201 把「阈值不得为放行任何在飞 PR 而调整」写成出口条件,新测试把 70..89 钉死。review 时请按这条卡我:如果你认为 80 偏松,把它挪到 70 结论完全一样(仍是 2/60),测试也一样绿。

验证

cargo test 全绿。有一条已知 flake this_repository_has_no_resolved_import_cycles 在并行下偶发红(bug(test): 并行测试进程共享 target/tool-path 且无归属——与 #175 同因的第二例 #178:并行测试进程共享 target/tool-path-* 且无归属,表现为 bug(gate): 引用环判据把「引擎没跑起来」和「仓库有环」报成同一个红灯 #192 把「引擎没跑起来」报成「仓库有环」);连续三次观察到,每次 tool-path 名字不同,--test-threads=1 稳定通过。与本改动无关。
cargo fmt --all --check 干净
repin clean / filesChanged 0
闸门算术离线模拟:68 / 69 / 79.5 → 放行,80 / 80.5 / 93 → 拦下,warning 报告(空 diff)走 n/a 早退

Review limit reached

@2233admin, you've reached your PR review limit, so we couldn't start this review.

Next review available in: 50 minutes

You've used all free OSS reviews for now. Wait for the free limit to reset to keep reviewing this public repository.

How can I continue?

After more reviews become available, a review can be triggered using the @coderabbitai review command as a PR comment. Alternatively, push new commits to this PR.

To avoid repeated limits, reduce automatic review volume by pausing incremental auto-reviews earlier, using label-based review opt-in, excluding WIP or generated PR titles, or requesting reviews manually when the PR is ready. If your team needs uninterrupted high-volume reviews, an organization admin can enable usage-based reviews.

How do review limits work?

CodeRabbit enforces per-developer PR review limits for each organization. Most developers receive the normal plan review availability.

For paid Pro and Pro+ PR reviews, CodeRabbit uses adaptive limits for sustained high-volume activity. When a developer's recent PR review activity reaches the 95th percentile or higher among CodeRabbit users, additional reviews become available more gradually as earlier reviews age out of the rolling window.

Please refer docs for additional details.

Review details

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: c3213637-77a6-4f16-833f-a09d10b019a2

📥 Commits

Reviewing files that changed from the base of the PR and between d4f4313 and 78c00aa.

📒 Files selected for processing (4)

.github/workflows/pr-gate.yml
crates/code-intel-cli/src/change_risk/tests/mod.rs
crates/code-intel-cli/src/change_risk/tests/predicates.rs
crates/code-intel-cli/src/change_risk/tests/scoring.rs

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands.}

github-actions · 2026-08-05T18:33:45Z

Code Intel change risk

Score	Percentile	Level
32/100	21th (vs last 48 commits)	🟢 low

Top signals

Diff shape: 5 file(s), +623/-524 (max file share 0.44)
Test asymmetry: source changed, tests touched
Bug-magnet: 2 fix commit(s) in touched files (180d)
Churn: 7 commit(s) touching these files (90d)

_{revspec: origin/main..HEAD · threshold: score >= 80 blocks unless labeled risk-accepted; percentile is reported, not gated (#201) · code-intel change risk}

sentrux gate: functions>25 && loc>400 (26 fns, 507 loc) on change_risk/tests.rs. Split into tests/mod.rs (fixture helpers), tests/predicates.rs (pure-logic + workflow-contract tests), and tests/scoring.rs (git-fixture tests). Descendant-module visibility keeps every helper and change_risk item reachable without new pub.

Nine existing flat-module test files (artifacts_tests.rs, audit_report/model_tests.rs, ...) use the `_tests.rs` plural suffix; zero use the singular `_test.rs` is_test_file actually checked. Every PR adding one of these -- including this session's own boundary_rules_tests.rs -- scored "no tests touched" and got flagged as a change-risk violation despite genuine coverage. Same class of bug as a3ac9a1 (bare tests.rs), just the plural sibling never covered. Split change_risk/tests.rs into tests/{mod,predicates,scoring}.rs in the same pass: the new test pushed it over the god-file ratchet (26 functions, 464 loc) the moment it landed on a tree that hasn't merged #202's identical split yet.

…layer_order + boundary_dependency (#205) * fix(sentrux): drop fictional operationTrace entries from the internalization record session_start/session_end/test_gaps/what_if were recorded as Rust operations with conformance tests, but sentrux.rs's match arms never implemented them -- calling any of the four hits the `other =>` error branch. Remove the four entries and reword economics.benefit to state what it actually counts. * feat(ci): smoke-test sentrux scan/health/dsm against the packaged binary Three implemented, tested sentrux operations had no caller outside unit fixtures -- a regression in any of them only showed up if someone ran the CLI by hand. Invoke them (non-gating, no baseline semantics) right after the existing check/gate calls in both release.yml packaging jobs, against the real packaged payload. Record the new production caller in the internalization trace and correct economics.benefit's count. * feat(sentrux): implement layer_order and boundary_dependency sentrux_adapter::AUTHORITATIVE_RULE_KINDS has named these two rule kinds since the adapter was written; the engine never computed either, so `evidence.sentrux` was only ever complete through the coarser command-observation path. boundary_rules.rs resolves `use crate::segment` the same way `rust_import_cycles` already does for cycle detection -- an edge only exists when `segment.rs`/`segment/mod.rs` is a real file, so an unresolved path or re-exported name can't manufacture one. `check` now evaluates both rules for real against `.sentrux/rules.toml`'s new `[[layer]]`/`[[boundary]]` tables (hand-rolled parsing, matching how `[constraints]` is already read -- no new TOML dependency for two array-of-tables). Configured for this repo: sentrux_gate/sentrux_analysis (zero `use crate::` lines, verified) as the foundational layer, the adapter/provider modules above them, cli outermost -- plus an explicit boundary_dependency naming the highest-value edge (engine must not depend on cli) so a violation there gets its own rule name instead of a generic layer failure. * fix(sentrux): repin sentrux_gate.rs, add end-to-end boundary/layer tests CI caught two real gaps in the previous commit: - orchestration/internalization/sentrux.json still pinned sentrux_gate.rs to its pre-edit sha256; `code-intel repin --write` resyncs it. - boundary_rules.rs's tests lived inline instead of in a sibling `_tests.rs` file, so this crate's own is_test_file heuristic didn't credit them -- change-risk scored the PR "no tests touched" despite the coverage existing. Split to boundary_rules_tests.rs (this crate's established convention) and add two fixture-driven `run_check` tests that create a real two-file tree with an actual `use crate::cli` edge: one asserting the forbidden edge is caught, one asserting the allowed direction isn't. The prior self-check test only proved this repository has no violation today -- it would stay green even if `crate_edges` regressed to returning nothing. * fix(change-risk): credit the crate's own plural _tests.rs convention Nine existing flat-module test files (artifacts_tests.rs, audit_report/model_tests.rs, ...) use the `_tests.rs` plural suffix; zero use the singular `_test.rs` is_test_file actually checked. Every PR adding one of these -- including this session's own boundary_rules_tests.rs -- scored "no tests touched" and got flagged as a change-risk violation despite genuine coverage. Same class of bug as a3ac9a1 (bare tests.rs), just the plural sibling never covered. Split change_risk/tests.rs into tests/{mod,predicates,scoring}.rs in the same pass: the new test pushed it over the god-file ratchet (26 functions, 464 loc) the moment it landed on a tree that hasn't merged #202's identical split yet. * fix(sentrux): finish removing the four fictional operations from the registry Two bugs from dropping the operationTrace entries for session_start/session_end/test_gaps/what_if: - orchestration/integrations.json's structure.sentrux integration still registered rustSessionStart/rustSessionEnd/rustTestGaps/ rustWhatIf as commands (and the matching capabilities), so assert_operation_trace_exact's registry<->trace cross-check failed the moment CI ran it -- the trace and the registry have to agree, and only the trace side was fixed. Removed the same four from both commands and capabilities. - internalization_record.rs's validate_operation_trace enforces a closed field set per entry; the productionCaller field I'd added to record the new release.yml smoke callers isn't in it. Dropped the field -- the commit history is where that context belongs, not a schema the validator doesn't expect to grow. * test(change-risk): cover the plural _tests.rs suffix in score_subset The is_test_file fix had unit coverage but no scoring-contract test proving score_subset actually credits the plural suffix the same way it already does for bare tests.rs -- CodeRabbit flagged the gap. * fix(sentrux): resync provider.sentrux-adapt toolchainDigests legacy/scripts/tests/test-atomic-capability-contract.ps1 checks a separate pin from code-intel repin: each entry in toolchainDigestEvidence.inputs pairs positionally with capabilityDeclaration.implementation.toolchainDigests, each just the plain sha256 of that one file. sentrux_gate.rs changed several times this session; repin only resyncs orchestration/internalization/*.json pins, never touched this one. Recomputed and replaced by literal string match, matching what repin independently derived earlier.

repowise-bot · 2026-08-06T06:44:49Z

✅ Health of changed files: 9.0 (unchanged)
_{🚨 Change risk: high, riskier than 74% of this repo's commits.}

📋 At a glance
1 file changed health · 3 hotspots touched · 3 files with recent fix history. Scoped to crates.

✅ Health gate: passed

🔎 More signals (2)

🗺️ Change map

flowchart LR
  subgraph PR ["Changed in this PR (1 with dependents)"]
    f_crates_code_intel_cli_src_change_risk_tests_mod_rs[".../tests/mod.rs"]:::changed
  end
  f_crates_code_intel_cli_src_change_risk_mod_rs[".../change_risk/mod.rs"]
  f_crates_code_intel_cli_src_change_risk_tests_mod_rs --> f_crates_code_intel_cli_src_change_risk_mod_rs
  classDef changed fill:#dbeafe,stroke:#1d4ed8,color:#1e3a5f
  classDef warn fill:#fef3c7,stroke:#b45309,color:#78350f
  classDef guard fill:#dcfce7,stroke:#15803d,color:#14532d

_{Solid arrows: code that imports the changed files (1 direct dependent, from the last indexed snapshot). Dashed: history/tests.}

🔥 Hotspots touched (3)

.../tests/predicates.rs: 1 commits/90d, 2 dependents
.../tests/mod.rs: 1 commits/90d, 1 dependents
.../tests/scoring.rs: 1 commits/90d, 2 dependents

📊 See the full report for this PR
_{Your repo map with this PR's blast radius lit up, every caller of the contracts it changes, and health before and after. No sign-in. · ⭐ Star Repowise · 📥 Install bot · Silence on a single PR with [skip repowise] in the title · Per-repo toggle on repowise.dev/settings?tab=bot · Updated 2026-08-06 06:44 UTC}

Navigation Menu

fix(gate): change-risk 闸判绝对分数,不判会漂的排名 (#201) - #202

fix(gate): change-risk 闸判绝对分数,不判会漂的排名 (#201)#202
2233admin wants to merge 2 commits into
mainfrom
agent/change-risk-absolute-201

2233admin commented Aug 5, 2026

Uh oh!

coderabbitai Bot commented Aug 5, 2026 •

edited

Loading

Review limit reached

Uh oh!

github-actions Bot commented Aug 5, 2026 •

edited

Loading

Uh oh!

repowise-bot Bot commented Aug 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

Conversation

2233admin commented Aug 5, 2026

症状

机制

阈值从实测分布读出来

改了什么

顺带修了一个会变成停机的坑

测试

诚实的代价

利益冲突声明

验证

相关

Uh oh!

coderabbitai Bot commented Aug 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Review limit reached

Uh oh!

github-actions Bot commented Aug 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Code Intel change risk

Uh oh!

repowise-bot Bot commented Aug 6, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

coderabbitai Bot commented Aug 5, 2026 •

edited

Loading

github-actions Bot commented Aug 5, 2026 •

edited

Loading