-
Notifications
You must be signed in to change notification settings - Fork 0
feat(automation): add DEV readiness policy evaluator #4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Merged
Merged
Changes from all commits
Commits
Show all changes
22 commits
Select commit
Hold shift + click to select a range
35fa226
test(automation): define DEV readiness behavior
syllik 36f2ab3
feat(automation): add pure DEV readiness evaluator
syllik f0bd092
test(automation): add DEV readiness dry-run scenarios
syllik c292f23
test(automation): execute dry-run scenario matrix
syllik 81fc351
docs(automation): document DEV readiness dry-run
syllik 25af919
ci(automation): add manual read-only DEV dry-run
syllik 3dedaf4
test(automation): require explicit readable state
syllik 5028ac2
fix(automation): fail closed when readability is unspecified
syllik 99945e6
test(automation): cover terminal statuses and blocker readiness
syllik fa9ed7e
ci(automation): run DEV policy tests on pull requests
syllik 20b9cbf
fix(automation): make DEV readiness monotonic and status-aware
syllik c35de05
test(automation): expand DEV readiness safety fixtures
syllik c0e8d96
docs(automation): clarify policy evaluator scope and safety
syllik 89314fd
test(automation): cover Codex fail-closed findings
syllik f691bf0
fix(automation): fail closed on unsupported and nullable state
syllik a27b047
test(automation): require explicit blocker relationship read
syllik 85d5c93
test(automation): make successful blocker reads explicit
syllik 13df201
fix(automation): require explicit blocker read
syllik 50d2104
test(automation): cover unsupported blocker state
syllik cf824fa
fix(automation): reject unsupported blocker state
syllik 0fe1745
test(automation): cover required-item read failures
syllik 7c15141
fix(automation): fail closed on required relationship reads
syllik File filter
Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
There are no files selected for viewing
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,25 @@ | ||
| name: DEV readiness policy tests | ||
|
|
||
| on: | ||
| pull_request: | ||
| paths: | ||
| - 'automation/**' | ||
| - '.github/workflows/dev-readiness-dry-run.yml' | ||
| workflow_dispatch: | ||
|
|
||
| permissions: | ||
| contents: read | ||
|
|
||
| jobs: | ||
| evaluate-fixtures: | ||
| runs-on: ubuntu-latest | ||
| timeout-minutes: 5 | ||
| steps: | ||
| - name: Checkout | ||
| uses: actions/checkout@v4 | ||
| - name: Use Node.js | ||
| uses: actions/setup-node@v4 | ||
| with: | ||
| node-version: '22' | ||
| - name: Run DEV readiness tests | ||
| run: node --test automation/*.test.mjs |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,34 @@ | ||
| # DEV readiness policy evaluator | ||
|
|
||
| This directory contains the pure, read-only decision logic for the future ChipIn Project `→ DEV` automation. | ||
|
|
||
| This PR is the policy layer only. It does not call GitHub APIs and cannot mutate issues, Projects, fields, statuses, PRs, or relationships. A later read-only adapter will feed real GitHub issue/PR/Project state into this evaluator before any live mutation is considered. | ||
|
|
||
| ## States | ||
|
|
||
| - `READY_FOR_DEV` — deterministic delivery requirements are integrated and the current Project status is still before `DEV`. | ||
| - `NOT_READY` — no DEV transition should happen now; required work may be incomplete, the task may be manual/non-code, or the item may already be at a terminal/integrated status. | ||
| - `BLOCKED_UNKNOWN` — required structured state is missing, unreadable, ambiguous, or unsupported. | ||
| - `INCONSISTENT` — an item already at `DEV` or `PROD` no longer satisfies DEV readiness and needs human review. | ||
|
|
||
| ## Integration branches | ||
|
|
||
| - frontend: `dev` | ||
| - backend: `develop` | ||
| - knowledge base: `main` | ||
|
|
||
| A blocking issue is considered satisfied for DEV readiness when the blocker itself is closed or its readable Project status is `DEV`, `PROD`, or `Done`. An unresolved/open blocker without an integrated Project status still blocks. | ||
|
|
||
| Deployment and informational `References` never gate `DEV`. `PROD`, `Done`, and parent closure remain manual in v1, and the evaluator never proposes a transition back from those statuses. | ||
|
|
||
| ## Why not Boardly | ||
|
|
||
| Boardly was reviewed before implementing this evaluator. It provides GitHub Projects v2 dry-run and sub-issue gating, but its built-in model does not express ChipIn's required Development-linked PR integration checks against repository-specific integration branches or the post-DEV inconsistency rule. Reusing it would require a larger customization layer than this isolated pure evaluator. | ||
|
|
||
| ## Tests | ||
|
|
||
| The GitHub Actions workflow runs on pull requests that touch `automation/**` or the workflow itself, and can also be started manually. It has only `contents: read` permission. | ||
|
|
||
| ```sh | ||
| node --test automation/*.test.mjs | ||
| ``` |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,205 @@ | ||
| const INTEGRATION_BRANCHES = Object.freeze({ | ||
| 'ChipIn-one/chipin-frontend': 'dev', | ||
| 'ChipIn-one/chipin-backend': 'develop', | ||
| 'ChipIn-one/chipin-knowledge-base': 'main', | ||
| }); | ||
|
|
||
| const KNOWN_PROJECT_STATUSES = new Set([ | ||
| 'Backlog', | ||
| 'Todo', | ||
| 'In Progress', | ||
| 'DEV', | ||
| 'PROD', | ||
| 'Done', | ||
| ]); | ||
|
|
||
| const KNOWN_WORK_KINDS = new Set(['Task', 'Feature', 'Bug']); | ||
| const KNOWN_DELIVERY_CLASSES = new Set(['code', 'non-code']); | ||
| const KNOWN_BLOCKER_STATES = new Set(['open', 'closed']); | ||
| const KNOWN_PR_STATES = new Set(['open', 'closed', 'merged']); | ||
| const COMPLETE_DEPENDENCY_STATUSES = new Set(['DEV', 'PROD', 'Done']); | ||
|
|
||
| function result(state, reason) { | ||
| return { state, reason }; | ||
| } | ||
|
|
||
| function hasUnreadableRelation(items) { | ||
| return items.some((item) => item?.readable === false || !item?.state); | ||
| } | ||
|
|
||
| function blockerState(input) { | ||
| if (hasUnreadableRelation(input.blockers)) { | ||
| return result('BLOCKED_UNKNOWN', 'A blocking relationship is unreadable.'); | ||
| } | ||
|
|
||
| for (const blocker of input.blockers) { | ||
| if (!KNOWN_BLOCKER_STATES.has(blocker.state)) { | ||
| return result('BLOCKED_UNKNOWN', 'A blocker has an unknown issue state.'); | ||
| } | ||
|
|
||
| if (blocker.state === 'closed') { | ||
| continue; | ||
| } | ||
|
|
||
| if ( | ||
| blocker.projectStatus !== undefined | ||
| && !KNOWN_PROJECT_STATUSES.has(blocker.projectStatus) | ||
| ) { | ||
| return result('BLOCKED_UNKNOWN', 'A blocker has an unknown Project status.'); | ||
| } | ||
|
|
||
| if (COMPLETE_DEPENDENCY_STATUSES.has(blocker.projectStatus)) { | ||
| continue; | ||
| } | ||
|
|
||
| return result('NOT_READY', 'At least one required blocker is not integrated yet.'); | ||
| } | ||
|
|
||
| return null; | ||
| } | ||
|
|
||
| function classifyDelivery(input) { | ||
| if (input.deliveryClass !== undefined) { | ||
| return input.deliveryClass; | ||
| } | ||
| if (input.workKind === 'Bug' || input.workKind === 'Feature') { | ||
| return 'code'; | ||
| } | ||
| return null; | ||
| } | ||
|
|
||
| function evaluateComposite(input) { | ||
| if (!input.requiredItems.length) { | ||
| return result('BLOCKED_UNKNOWN', 'Composite parent has no readable required sub-issues.'); | ||
| } | ||
| if (hasUnreadableRelation(input.requiredItems)) { | ||
| return result('BLOCKED_UNKNOWN', 'A required sub-issue is unreadable.'); | ||
| } | ||
|
|
||
| for (const item of input.requiredItems) { | ||
| if (item.kind !== 'subissue' || !KNOWN_PROJECT_STATUSES.has(item.state)) { | ||
| return result('BLOCKED_UNKNOWN', 'A required sub-issue has unsupported structured state.'); | ||
| } | ||
| } | ||
|
|
||
| const complete = input.requiredItems.every( | ||
| (item) => COMPLETE_DEPENDENCY_STATUSES.has(item.state), | ||
| ); | ||
| return complete | ||
| ? result('READY_FOR_DEV', 'All required sub-issues are complete for DEV roll-up.') | ||
| : result('NOT_READY', 'At least one required sub-issue is not complete.'); | ||
| } | ||
|
|
||
| function evaluateCodeDelivery(input) { | ||
| const expectedBranch = INTEGRATION_BRANCHES[input.repository]; | ||
| if (!expectedBranch) { | ||
| return result('BLOCKED_UNKNOWN', 'Repository has no configured integration branch.'); | ||
| } | ||
| if (!input.requiredItems.length) { | ||
| return result('NOT_READY', 'No required implementation PR is recorded.'); | ||
| } | ||
| if (hasUnreadableRelation(input.requiredItems)) { | ||
| return result('BLOCKED_UNKNOWN', 'A required implementation relationship is unreadable.'); | ||
| } | ||
|
|
||
| for (const item of input.requiredItems) { | ||
| if ( | ||
| item.kind !== 'pr' | ||
| || !KNOWN_PR_STATES.has(item.state) | ||
| || typeof item.baseBranch !== 'string' | ||
| || item.baseBranch.length === 0 | ||
| ) { | ||
| return result('BLOCKED_UNKNOWN', 'A required implementation PR has incomplete or unsupported structured state.'); | ||
| } | ||
| } | ||
|
|
||
| const allIntegrated = input.requiredItems.every( | ||
| (item) => item.state === 'merged' && item.baseBranch === expectedBranch, | ||
| ); | ||
| return allIntegrated | ||
| ? result('READY_FOR_DEV', `All required PRs are merged to ${expectedBranch}.`) | ||
| : result('NOT_READY', `At least one required PR is not merged to ${expectedBranch}.`); | ||
| } | ||
|
|
||
| export function evaluateDevReadiness(rawInput) { | ||
| const source = rawInput ?? {}; | ||
| const blockersRead = Object.hasOwn(source, 'blockers'); | ||
| const requiredItemsRead = Object.hasOwn(source, 'requiredItems'); | ||
| const input = { | ||
| requiredItems: [], | ||
|
syllik marked this conversation as resolved.
|
||
| blockers: [], | ||
|
syllik marked this conversation as resolved.
|
||
| metadataReadable: false, | ||
| projectReadable: false, | ||
| isCompositeParent: false, | ||
| ...source, | ||
| }; | ||
|
|
||
| if (!input.metadataReadable || !input.projectReadable) { | ||
| return result('BLOCKED_UNKNOWN', 'Required structured GitHub state is unreadable.'); | ||
| } | ||
|
|
||
| if (!input.repository || !input.currentStatus || !input.workKind) { | ||
| return result('BLOCKED_UNKNOWN', 'Required structured GitHub state is missing.'); | ||
|
syllik marked this conversation as resolved.
|
||
| } | ||
|
|
||
| if (!blockersRead) { | ||
| return result('BLOCKED_UNKNOWN', 'Blocking relationships were not read explicitly.'); | ||
| } | ||
|
|
||
| if (!Array.isArray(input.requiredItems) || !Array.isArray(input.blockers)) { | ||
| return result('BLOCKED_UNKNOWN', 'Required relationship collections are unreadable.'); | ||
| } | ||
|
|
||
| if (!KNOWN_PROJECT_STATUSES.has(input.currentStatus)) { | ||
| return result('BLOCKED_UNKNOWN', 'Project status is unknown.'); | ||
| } | ||
|
|
||
| if (!KNOWN_WORK_KINDS.has(input.workKind)) { | ||
| return result('BLOCKED_UNKNOWN', 'Work kind is unknown.'); | ||
| } | ||
|
|
||
| if ( | ||
| input.deliveryClass !== undefined | ||
| && !KNOWN_DELIVERY_CLASSES.has(input.deliveryClass) | ||
| ) { | ||
| return result('BLOCKED_UNKNOWN', 'Delivery class is unknown.'); | ||
| } | ||
|
|
||
| const blocked = blockerState(input); | ||
| const deliveryClass = classifyDelivery(input); | ||
| let readiness; | ||
|
|
||
| if (!deliveryClass) { | ||
| readiness = result('BLOCKED_UNKNOWN', 'Task delivery class is ambiguous.'); | ||
| } else if (deliveryClass === 'non-code') { | ||
| readiness = result('NOT_READY', 'Standalone non-code work terminates at Done manually, not DEV.'); | ||
| } else if (!requiredItemsRead) { | ||
| readiness = result('BLOCKED_UNKNOWN', 'Required implementation relationships were not read explicitly.'); | ||
| } else if (input.isCompositeParent) { | ||
| readiness = blocked ?? evaluateComposite(input); | ||
| } else { | ||
| readiness = blocked ?? evaluateCodeDelivery(input); | ||
| } | ||
|
|
||
| if (input.currentStatus === 'DEV' || input.currentStatus === 'PROD') { | ||
| if (readiness.state !== 'READY_FOR_DEV') { | ||
| return result( | ||
| 'INCONSISTENT', | ||
| `Current status is ${input.currentStatus} but readiness recomputation is ${readiness.state}.`, | ||
| ); | ||
| } | ||
| return result('NOT_READY', `Current status is already ${input.currentStatus}; no DEV transition is allowed.`); | ||
| } | ||
|
|
||
| if (input.currentStatus === 'Done') { | ||
| return result('NOT_READY', 'Current status is Done; DEV automation must not change a manual terminal state.'); | ||
| } | ||
|
|
||
| return readiness; | ||
| } | ||
|
|
||
| export { | ||
| INTEGRATION_BRANCHES, | ||
| KNOWN_PROJECT_STATUSES, | ||
| KNOWN_WORK_KINDS, | ||
| }; | ||
Oops, something went wrong.
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Uh oh!
There was an error while loading. Please reload this page.