Separate evidence status from ACTION outcome reporting

[Bashirloyan]

Context
In #36, I suggested separating evidence status from action outcomes. The maintainer called the idea reasonable and requested a separate issue because it changes reporting design.
This issue focuses on ACTION conformance reporting. It does not take over ACTION-014/015, change replay behavior, or add self-contained fixture bundles.
This issue proposes reporting evidence validity separately from authorization decisions or execution outcomes. Evidence of a denied or rejected action can remain valid when it passes the required verification checks.

Problem
The ACTION helper returns _ActionEvidenceResult with classification and code. These values already distinguish between provenance failures, authorization failures, and valid negative outcomes.
Today, readers and reporting tools must infer both evidence status and action outcome from those classifications. I propose making two separate questions explicit:

1. Did the evidence pass the required checks?
2. What decision or outcome does that evidence support?
   A denial can show that a control worked correctly. Evidence can pass its checks while policy denies the request or the controller rejects the action.

Proposal
Add an explicit evidence_status alongside the action decision or outcome. Define which evidence-verification checks it covers, separately from authorization decisions and execution outcomes. Retain existing classification and code values where possible, subject to compatibility review.

Case	What the report should show
Evidence passes its checks, but local policy denies the request	Report the evidence check result separately from the policy denial.
Evidence passes its checks, but the controller rejects the action	Preserve valid evidence of the negative outcome.
Evidence fails its checks, but the input claims the action was allowed	Flag the evidence failure. Do not present the claimed outcome as verified.

If evidence fails, the report could retain the claimed outcome separately while marking the verified outcome as unknown or not evaluated.
“Allowed” should describe an explicit authorization decision, not merely the absence of a block. The report must not present that decision as proof of successful execution.
Likewise, a passing conformance test means the system behaved as expected—not that the sample action succeeded.
The design should distinguish failed checks, missing required evidence, and checks the verifier did not perform. Each state carries a different meaning for readers and reporting tools. I leave the exact fields and values open for discussion.

Scope
Limit the initial change to the ACTION conformance reporting helper and related documentation.
The change should:
• Define which checks evidence_status covers.
• Preserve security checks, failure ordering, and reason codes.
• Never treat invalid evidence as permission to act.
• Review compatibility and document any required migration.

Questions for maintainers

1. Should the initial change stay within conformance reporting, or should it also expose evidence_status through the public verifier API?
2. Should we use separate fields for authorization decisions and controller outcomes, or would one shared outcome field suffice?
3. How should we distinguish invalid evidence, missing required evidence, and checks the verifier did not perform?
4. Can we add reporting fields while retaining existing classifications and reason codes for compatibility?

[carloshvp]

The direction is sound, and I would accept it with the following boundary before implementation.

1. Keep the first change in conformance reporting

Please limit the initial change to the private ACTION conformance helper, its cases, and tests/conformance/README.md. There is no public action-evidence verification result today: _ActionEvidence and _ActionEvidenceResult live only in the conformance test module. We should settle the semantics there before creating a public API that we would then need to preserve.

2. Report three independent axes

Authorization and controller execution should not share one outcome field. They answer different questions, and a controller cannot reject an action that authorization never allowed it to receive.

I would use a result shape along these lines:

• provenance_status: verified, invalid, missing, or not_evaluated
• authorization_decision: allowed, denied, or not_evaluated
• controller_outcome: accepted, rejected, or not_evaluated
• the existing classification and code, retained initially for compatibility and exact failure attribution

I prefer provenance_status over the broader evidence_status unless the documentation defines the checked evidence set precisely. The current helper verifies the delegation chain, provenance DAG, chain-to-record correspondence, record hash, credential id, and authorization replay. It explicitly does not reconstruct holder proof during offline replay, and it does not establish every possible attestation or execution-evidence property. A bare evidence_status=verified could therefore claim more than the helper proved.

3. Do not call the controller outcome verified yet

At current main af0d6f4c5223580b5d3a59b2cac60f1d1dee61b2, _ActionEvidence.controller_decision is a standalone test-helper input. It is not part of the DelegationRecord body whose hash is checked, and _verify_action_evidence() reads it only after the provenance and authorization checks.

That means the helper can report a claimed or scenario controller outcome, but cannot yet say cryptographically that the controller produced that outcome. If this result is named controller_outcome, the documentation should state that boundary. If we want a verified controller outcome, it must first be committed into signed ACTION or TRACE evidence and checked through that binding.

4. Preserve prerequisite ordering

If provenance is invalid or required evidence is missing, authorization and controller outcome must be not_evaluated. A claimed outcome may be retained separately for audit, but it must not become a verified outcome. The existing ACTION-008 ordering is the right precedent: an invalid signature preempts both authorization and controller classification.

The minimum load-bearing cases I would want are:

1. verified provenance + policy denial -> verified, denied, not_evaluated;
2. verified provenance + authorization allowed + controller rejection -> verified, allowed, rejected;
3. invalid signature + claimed rejection -> invalid, not_evaluated, not_evaluated;
4. missing required evidence -> missing, not_evaluated, not_evaluated;
5. a check deliberately outside the helper's scope -> not_evaluated, not verified or invalid.

I ran the existing ACTION group against that pinned main before writing this: all 13 current ACTION conformance tests pass. This proposal should change reporting semantics without weakening their security ordering or reason codes.

[Bashirloyan]

Thank you, Carlos. This boundary is clear and addresses the concern I raised.

I agree that provenance_status is more precise than evidence_status for this purpose. Separating provenance, authorization, and controller outcome avoids implying that permission proves execution.

I understand that the first change should stay within the private ACTION conformance helper, its cases, and the conformance README. The documentation should define exactly what provenance_status covers and note that it does not verify every possible evidence property.

I also understand that the current controller result is test input, not an outcome proven by signed evidence. The checks must run in order, while the existing classification, code, prerequisite ordering, and reason codes remain in place.

The five cases you listed provide a clear basis for the first reporting-only change. Would you like me to draft the field definitions and expected results for those cases next.?

[imran-siddique]

Ratifying @carloshvp's boundary as the agreed scope. Nothing to add to it, and @Bashirloyan has restated it back accurately, so this is a go rather than another round.

To be explicit about what is settled, so the PR has something to point at:

• Scope is the private ACTION conformance helper, its cases, and tests/conformance/README.md. No public API. _ActionEvidence and _ActionEvidenceResult live only in the conformance module today, and settling the semantics before there is a surface to preserve is the right order.
• provenance_status, not evidence_status. Carlos is right that it is more precise, and the imprecise name is the one that would have caused the confusion this issue exists to remove.
• Three independent axes: provenance, authorization, controller outcome. The reason to keep them apart is the one @Bashirloyan opened with, that a denial can be evidence a control worked, and it is worth putting in the README in those words.
• The documentation says what provenance_status covers and what it does not. A status that reads as "the evidence is good" when it means "these specific checks passed" is the failure this change is meant to prevent, so the boundary belongs next to the definition rather than in a commit message.

@Bashirloyan, please open the PR. Existing classification and code values retained where they can be, and where one has to move, say so in the PR body rather than in the diff alone.

Thanks to both of you: this started as a suggestion inside #36, got moved to its own issue because it changes reporting design, and arrived scoped. That is the process working.