Skip to content

feat(platform)!: require fee history for storage refunds and credit their recorded owners - #4706

Open
DCG-Claude wants to merge 13 commits into
v4.3-devfrom
dashvm/r12-02
Open

feat(platform)!: require fee history for storage refunds and credit their recorded owners#4706
DCG-Claude wants to merge 13 commits into
v4.3-devfrom
dashvm/r12-02

Conversation

@DCG-Claude

@DCG-Claude DCG-Claude commented Sep 12, 2026

Copy link
Copy Markdown
Collaborator

Issue being fixed or feature implemented

Part 1 of 3 for R12-02 of the smart-contract plan (#4626, section 12, fee workstream #4689, preparation package #4675).

Storage refunds are priced against the fee history a block carries (previous_fee_versions in platform state). The shipped fee calculation (Drive::calculate_fee v0, consume_to_fees_v0) short-circuits fee version number 1: it prices every owner-attributed removal against an empty history, so a caller that forgets to pass the history silently refunds at the first generation's storage rates instead of failing. The fee version integrity rule in the fee DIP requires the opposite: a refund without the historical context of the removing block is an error, never a fallback to a schedule.

The same DIP requires refunds to follow the recorded owner and, for owners that no longer exist, to go to the current processing pool (acceptance Q26: a frozen but existing owner still receives native bookkeeping refunds without any spending permission; a wiped owner's refunds go to the processing pool). Today the only code that credits other owners (apply_balance_change_from_fee_to_identity v0) halts on an owner without a balance, and block lifecycle paths that free owner-flagged bytes never price or settle refunds at all.

This part establishes the explicit version boundary and the primitive the later parts build on:

  • protocol version 15 (packages/rs-platform-version/src/version/v15.rs) with DRIVE_VERSION_V10;
  • Drive::calculate_fee v1, which requires and consults the fee history for every owner-attributed storage removal;
  • Drive::credit_storage_refunds_to_owners_operations, which credits recorded owners without consulting any key or permission and reports the amount whose owner has no balance element.

Part 2 (same base) makes the vote poll end cleanup price and settle its refunds with this primitive. Part 3 (base v5.0-dev) routes state transition refunds for missing owners to the processing pool and carries the DIP text.

What was done?

Version tables (packages/rs-platform-version)

  • src/version/v15.rs: PROTOCOL_VERSION_15 and PLATFORM_V15, a copy of v14 whose only change is drive: DRIVE_VERSION_V10. Registered in src/version/mod.rs (LATEST_VERSION) and src/version/protocol_version.rs (PLATFORM_VERSIONS, LATEST_PLATFORM_VERSION). LATEST_VERSION was 14 at lane start, so the version was introduced rather than amended; the open state sync PR (feat(drive-abci)!: state sync via ABCI snapshots with reduced platform state (protocol v15) #4648) introduces the same number and whoever rebases second merges the v15.rs doc comment.
  • src/version/drive_versions/v10.rs: DRIVE_VERSION_V10, a copy of v9 with fees.calculate_fee: 1 and identity: DRIVE_IDENTITY_METHOD_VERSIONS_V3.
  • src/version/drive_versions/drive_identity_method_versions/mod.rs: DriveIdentityUpdateMethodVersions gains credit_storage_refunds_to_owners: OptionalFeatureVersion. Backfilled as None in v1.rs and v2.rs (shipped tables stay behaviour-preserving); the new v3.rs sets Some(0).

Fee calculation (packages/rs-drive/src/fees)

  • op.rs: LowLevelDriveOperation::consume_to_fees_v1 beside the untouched consume_to_fees_v0. The SectionedStorageRemoval arm takes the system bucket as before, then requires previous_fee_versions on every fee version number and returns DriveError::CorruptedCodeExecution("a storage refund needs the fee history of the block that removes the bytes") without it. The empty-map shortcut for number 1 is gone in v1.
  • calculate_fee/v1/mod.rs: Drive::calculate_fee_v1, a copy of v0 calling consume_to_fees_v1. calculate_fee/mod.rs dispatches 1 => and reports known_versions: vec![0, 1]. v0 is byte-identical.

Recorded-owner credits (packages/rs-drive/src/drive/identity/update)

  • methods/credit_storage_refunds_to_owners_operations/{mod.rs, v0/mod.rs}: the new versioned method (Some(0) dispatch, None => VersionNotActive). For each owner in a FeeRefunds except an optional skip_owner (the state transition payer, whose own refund folds into its balance change), it sums the per-epoch credits with checked arithmetic, reads the balance element statefully once, feeds that read into add_to_previous_balance and the balance and negative credit update operations (the same shape the payer's own refund uses in apply_balance_change_from_fee_to_identity, so no element is read twice) when it exists, and otherwise adds the amount to routed_to_processing_pool. When the owner's balance is zero the shipped helper first clears its negative credit (identity debt) and only the remainder reaches the balance; the primitive derives that portion from the helper's outcome and reports it as repaid_debt, because debt lives outside the credit sum trees and is processing fee the pools were short of when it was incurred. No key, signature or permission is read on any route. The primitive neither writes the processing pool nor records pending refunds; the caller writes processing_pool_share() (unrouted refunds plus repaid debt) once and records the pending refunds, so a block keeps one pool write and one pending-refund write per batch.
  • structs/storage_refund_credit_outcome/mod.rs: StorageRefundCreditOutcome { credited: BTreeMap<Identifier, Credits>, repaid_debt: Credits, routed_to_processing_pool: Credits } with checked processing_pool_share() and total().

Tests

  • fees/op.rs (storage_refund_fee_history module): v1 rejects a sectioned removal without history (v0 still prices it); a system-bucket-only sectioned removal is rejected too; unflagged (BasicStorageRemoval) bytes never need the history; the history is consulted for fee version number 1 (a synthetic number-2 schedule with doubled storage rates at epoch 10, bytes stored at epoch 12 and removed at 15: v0 refunds at the first generation's rate, v1 at the doubled rate); for every PLATFORM_VERSIONS entry and every history the epoch change hook could build, v1's FeeResults equal v0's; every shipped schedule keeps fee version number 1 and the first generation's storage rates (the premise of that equality).
  • fees/calculate_fee/mod.rs: through the dispatcher, PlatformVersion::get(14) prices an owner-attributed removal without history, PlatformVersion::latest() refuses it, and with a history both produce the same fees.
  • credit_storage_refunds_to_owners_operations/v0/mod.rs: each recorded owner credited by the sum of its epochs; an owner without a balance is reported, not an error, and no balance element is created; an owner whose keys are all disabled is credited; skip_owner is left alone; after the caller writes the pool and records the pending refunds, calculate_total_credits_balance stays balanced (TotalCreditsBalance::ok); refunds below, equal to and above an owner's outstanding debt report the repaid share and leave the credit sum balanced once the caller writes the pool share; a positive balance repays no debt; PlatformVersion::get(14) returns VersionNotActive.
  • drive/group/mod.rs: should_close_group_action_and_move_signers now closes the action the way production does (a GroupOperationType::AddGroupAction { closes_group_action: true, .. } through apply_drive_operations with the fee history). New: should_refund_signer_bytes_when_a_group_action_closes (the opening signer's flagged items are refunded against their storage epoch; the closing signer, whose items are unflagged, is not) and should_reject_closing_a_group_action_through_the_bare_wrapper_without_fee_history (CorruptedCodeExecution at the latest version, success at protocol version 14). The two other closing-branch tests (immediate close with new action info, cost estimation with apply = false) free no flagged bytes and stay on the bare wrapper, which pins that the rule fires only when flagged bytes are actually removed.
  • drive/contract/migration/strip_unknown_document_schema_properties.rs: should_keep_the_protocol_12_schema_strip_frozen_without_refunding_stripped_bytes, a freeze test at PlatformVersion::get(12): a user contract element carrying the owner's storage flags is inflated with a top-level schema property the v1 meta-schema forbids, the migration shrinks it back to its clean bytes, the flags are byte-equal, the owner's balance and the pending refund tree are unchanged, and the credit sum stays balanced. The doc comment carries the replay rationale (below).
  • Two existing tests replaced epoch-flagged documents without a history and failed under the strict rule (ranked_index_e2e_tests::estimated_and_actual_update_fees, update::tests::summable_index_update_changes_key_into_new_branch_materializes_aggregate_tree_type); both now pass the same one-entry history the neighbouring tests use.

Book

book/src/fees/overview.md: a "Fee history and refund ownership (protocol version 15 onward)" subsection under Refunds, and a paragraph on fee_version_number under Fee Versioning.

Caller audit

Drive::calculate_fee has 129 call sites in 99 files of packages/rs-drive/src (14 forward a caller's history; the rest pass None). Grouped by what they can remove:

Class Sites History Verdict
Forwarded from the caller apply_drive_operations, update_contract v0/v1, document delete, update and add methods, identity balance and revision, token balance updates caller's correct; every production state transition and block-end application passes previous_fee_versions
None, can remove owner-flagged bytes drive/group/insert/add_group_action/v0/mod.rs:63 (closing an action moves signer-flagged items) None production reaches the closing branch only through GroupOperationType::AddGroupAction inside apply_drive_operations, which forwards the history; the bare wrapper is called from 26 test sites, of which one closes an action and now uses the production funnel. From protocol version 15 a closing call through the bare wrapper is a misuse the strict rule surfaces (pinned by test). The wrapper itself is untouched.
None, can remove owner-flagged bytes drive/contract/apply/apply_contract_with_serialization/v0/mod.rs:65 (replace path) None production callers: genesis (storage_flags: None), the protocol 13 and 14 transitions (perform_events_on_first_block_of_protocol_change/v0/mod.rs:683,707 re-store DPNS and DashPay over unflagged genesis elements with None flags, so the removal is BasicStorageRemoval), and create_mn_shares_contract (test-only). A sectioned removal here needs a flagged old element and flagged new flags, which only tests produce. Unchanged.
None, elements unflagged today disable_identity_keys/v0 (its TODO asks this question), update_keywords/v0, update_description/v0 (keyword and description documents are stored with DocumentOwnedInfo((doc, None))), token status, price, supply and contract info (Element::Item(.., None), SumItem(.., None)), identity nonce, revision and balance, vote sum items and references, add_document/v0 None safe today; listed so a future owner flag on any of them fails loudly under v1 instead of under-refunding
None, inserts, fetches, proofs and queries only everything else (drive/document/query/**, query/**, token fetch and prove, identity fetch, contested document inserts, vote registration, group inserts, insert_contract v0/v1) None safe; nothing is removed

drive-abci direct Drive::calculate_fee callers (fetch_contender.rs, the index-only delete and document create state validators) price reads only.

Block lifecycle applications in packages/rs-drive-abci/src/execution/platform_events (non-test code):

Path Frees owner-flagged bytes? Applies through History Fee result
voting/clean_up_after_contested_resources_vote_polls_end/{v0,v1}: deletes contested documents and contender trees (owner-flagged), the end-date entry (creator-flagged), votes and stored info (unflagged); v1 also empties prefunded balances yes apply_batch_low_level_drive_operations(None, ..) none never computed, refunds dropped: part 2
voting/award_document_to_winner/v0 no: inserts the winner's document unflagged (DocumentAndSerialization((doc, bytes, None))) add_document_for_contract(.., None) None discarded; the awarded record has no owner and is neither charged nor refundable (recorded for the owner-encoding refinement)
voting/keep_record_of_vote_poll/v0, voting/remove_votes_for_removed_masternodes/v0 no (unflagged) low-level none discarded
core_based_updates/update_masternode_identities/v0 no removals (key patches on unflagged keys) apply_drive_operations Some(platform_state.previous_fee_versions()) discarded
block_processing_end_events/process_block_fees_and_validate_sum_trees/v0 no apply_drive_operations Some(block_platform_state.previous_fee_versions()) discarded
withdrawals (dequeue_and_build_unsigned_withdrawal_transactions/v0, pool_withdrawals_into_transactions_queue/v1, rebroadcast_expired_withdrawal_documents/v1, update_broadcasted_withdrawal_statuses/v0) no: withdrawal documents are stored with owner_id: None and no flags (identity_credit_withdrawal_transition.rs, address_credit_withdrawal_transition.rs) apply_drive_operations None discarded
initialization/create_genesis_state/{v0,v1} no (inserts, storage_flags: None) apply_drive_operations None discarded
protocol_upgrade/perform_events_on_first_block_of_protocol_change/v0 (DPNS re-store at 13, DashPay at 14) no (unflagged genesis elements replaced by unflagged elements) apply_contract None discarded
protocol_upgrade/.../v0 transition_to_version_12 calling strip_unknown_document_schema_properties rewrites user contract items in place keeping their flags; a shrink removes owner-paid bytes whose refund is discarded grove_insert with a discarded cost vector none shipped and frozen; ran once at the protocol 12 activation; the recorded historical exception, pinned by the freeze test
fee pool inwards and outwards, epoch change, credit inflow and total-credits history, shielded anchors, address balance cleanup, version counters no operation builders and direct grove ops n/a n/a
check_tx/v0, process_raw_state_transitions/v0, execute_event/v0, validate_fees_of_event/v0 state transitions apply_drive_operations Some(state.previous_fee_versions()) applied through apply_balance_change_from_fee_to_identity

Conclusion: one production path frees owner-flagged bytes without fee context and without settling refunds (vote poll end cleanup, part 2); one shipped migration discarded refunds and is frozen (pinned here); one test-only wrapper closes group actions without history (its closing test now uses the production funnel); every other path is correct or touches unflagged bytes.

Owner-flagged writes (what the invariant covers)

Documents, index references, contested documents and contested index trees (document owner); the vote poll end-date entry (contest creator); user data contract items (insert_contract v1: every contract with can_be_deleted() || !readonly(), contract owner); group action info and signer sum items (signer); token distribution items (token owner, recipient, claimer). Not flagged: identities and their keys, withdrawal documents, keyword-search documents, votes, prefunded balances, epoch pools, withdrawal queue items, shielded anchors, stored vote poll info, and the system contracts at genesis. The document history contract registered at the protocol 13 transition goes through insert_contract v1 and therefore carries the system owner's flags; a future transition that re-stores it must pass its flags or hit grovedb's RemovingFlagsError (noted, no change here).

How Has This Been Tested?

Local gate (exit codes captured under the run directory):

cargo fmt --all -- --check
cargo clippy -p platform-version -p dpp -p drive -p drive-abci --all-features --all-targets -- -D warnings
cargo check --workspace --all-targets
cargo test -p platform-version --all-features
cargo test -p drive --lib -- fees::op::tests::storage_refund_fee_history
cargo test -p drive --lib -- fees::calculate_fee
cargo test -p drive --lib -- credit_storage_refunds_to_owners_operations
cargo test -p drive --lib -- drive::group
cargo test -p drive --lib -- drive::contract::migration
cargo test -p drive --lib

The full drive unit suite passes (3594 passed, 5 ignored); the rest of the workspace is CI's. The workspace check was run in a private CARGO_TARGET_DIR because another lane editing rs-platform-version shares the default target directory and its fingerprint collided with this tree's new identity slot. No src/verify/** file is touched, so the verify-only cut is unaffected (cargo check --workspace --all-targets compiles the verify crates).

Breaking Changes

Consensus: protocol version 15 is introduced. Under its drive table a storage refund for owner-attributed bytes is priced only with the fee history of the removing block; a missing history is an internal error (a halt on a block execution path) instead of a silent fallback to the first generation's rates. Every shipped schedule shares fee version number 1 and the same storage rates, so refund credits are unchanged for every shipped input; every shipped PLATFORM_V*, calculate_fee v0, consume_to_fees_v0, add_group_action, apply_balance_change_from_fee_to_identity v0 and the protocol 12 migration are byte-identical. Shipped identity tables gain one None field.

API: DriveIdentityUpdateMethodVersions gains a field (struct literals outside the version crate: none). Drive::credit_storage_refunds_to_owners_operations and StorageRefundCreditOutcome are new. No production path calls the primitive at this head; part 2 wires the vote poll end cleanup to it. No proto, SDK, wasm or FFI change; PlatformVersion::latest() now resolves to 15.

Checklist:

  • I have performed a self-review of my own code
  • I have commented my code, particularly in hard-to-understand areas
  • I have added or updated relevant unit/integration/functional/e2e tests
  • I have added "!" to the title and described breaking changes in the corresponding section if my code contains any
  • I have made corresponding changes to the documentation if needed

For repository code-owners and collaborators only

  • I have assigned this pull request to a milestone

Decisions taken (provisional values)

  • Provisional: an owner without a balance element is the proxy for a wiped owner. Contracts cannot be deleted yet and identities are never removed, so Drive has no typed "wiped owner" today. The primitive treats a missing balance element as that case and reports the amount for the processing pool. When the owner-encoding refinement types the owner in the storage flags this proxy is replaced; nothing in state depends on it.
  • Strictness over leniency. A missed production path under v1 halts the chain instead of mispricing silently. Mitigation: the audit above, the lifecycle table, and part 2's tests that run the lifecycle paths at the latest version with sum-tree verification. Treating a missing history as the current schedule is the fallback the owner rejected.
  • The primitive reports, the caller writes the pool. add_epoch_processing_credits_for_distribution_operation is a read-modify-write of the epoch's pool sum item, so two of them in one batch collapse last-wins. Returning routed_to_processing_pool and repaid_debt (together processing_pool_share()) lets the caller issue exactly one pool write per batch and keep pending-refund recording with the existing versioned methods. Repaid debt goes to the processing pool because negative credit is processing fee the pools were short of when the identity could not cover it (fee_result_outcome drops that part from the block fees), so clearing it with a refund returns those credits to the pool they were owed to.
  • No shipped file edited; the bare group wrapper keeps its signature. The plan's earlier draft widened add_group_action with a history parameter; this PR instead routes the one closing test through the funnel production uses and pins the wrapper's strictness.
  • Protocol 12 schema migration excluded by name. The stripped byte counts were never recorded and the migration never runs again, so there is nothing to correct at protocol version 15; the invariant (written into the DIP in part 3) states that blocks before 15 replay exactly as executed and names this migration as the exception.
  • Storage-epoch pricing not duplicated. The change that prices a refund at the rate of the epoch the bytes were stored in is another lane's in-place edit to FeeRefunds::from_storage_removal; it is not on v4.3-dev yet, so the test that would pin it is deferred to part 2.
  • Protocol version 15 introduced, not amended. LATEST_VERSION was 14 at lane start.

Part 1 of 3 for R12-02.

Refs #4675
Refs #4689


🤖 Posted autonomously by DashVM (Claude Fable 5.1) on behalf of pasta.

🤖 Generated with Claude Code

DCG-Claude and others added 9 commits September 12, 2026 01:31
…or storage refund fee history

Protocol version 15 is a copy of 14 whose drive table (DRIVE_VERSION_V10) selects
calculate_fee v1 and the new identity update slot credit_storage_refunds_to_owners
(DRIVE_IDENTITY_METHOD_VERSIONS_V3). Shipped identity tables backfill the slot with
None so every shipped protocol version is behaviour-preserving.

Refs #4675, Refs #4689

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…r recorded owners

calculate_fee v1 (consume_to_fees_v1) consults the block's fee history for every
owner-attributed storage removal and returns CorruptedCodeExecution without one,
on every fee version number; v0 stays byte-identical. The new versioned method
credit_storage_refunds_to_owners_operations credits each recorded owner that has
a balance element without consulting any key or permission and reports the
amount whose owner has no balance for the caller to route to the processing pool.

Refs #4675, Refs #4689

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Refs #4675, Refs #4689

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Refs #4675, Refs #4689

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ee history

Refs #4675, Refs #4689

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…exception

Refs #4675, Refs #4689

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ersion 15

Refs #4675, Refs #4689

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…ocuments

Refs #4675, Refs #4689

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
Refs #4675, Refs #4689

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@coderabbitai

coderabbitai Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Path: .coderabbit.yaml

Review profile: CHILL

Plan: Advanced

Run ID: 779b1c9b-305f-45bb-b456-a93f6965c632

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@github-actions

github-actions Bot commented Sep 12, 2026

Copy link
Copy Markdown
Contributor

📖 Book Preview built successfully.

Download the preview from the workflow artifacts.
To view locally: download the artifact, unzip, and open index.html.

Updated at 2026-09-12T23:53:59.191Z

@thepastaclaw

thepastaclaw commented Sep 12, 2026

Copy link
Copy Markdown
Collaborator

✅ Final review complete — no blockers (commit f26c668) · triage: critical

@codecov

codecov Bot commented Sep 12, 2026

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 90.14778% with 120 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.81%. Comparing base (d020728) to head (f26c668).

Files with missing lines Patch % Lines
packages/rs-drive/src/drive/group/mod.rs 61.31% 53 Missing ⚠️
packages/rs-drive/src/fees/op.rs 86.99% 32 Missing ⚠️
...dit_storage_refunds_to_owners_operations/v0/mod.rs 98.07% 11 Missing ⚠️
...ration/strip_unknown_document_schema_properties.rs 94.44% 7 Missing ⚠️
packages/rs-drive/src/fees/calculate_fee/v1/mod.rs 75.86% 7 Missing ⚠️
...credit_storage_refunds_to_owners_operations/mod.rs 83.33% 5 Missing ⚠️
packages/rs-drive/src/fees/calculate_fee/mod.rs 94.02% 4 Missing ⚠️
packages/rs-drive/src/drive/document/update/mod.rs 0.00% 1 Missing ⚠️
Additional details and impacted files
@@             Coverage Diff              @@
##           v4.3-dev    #4706      +/-   ##
============================================
- Coverage     84.10%   82.81%   -1.29%     
============================================
  Files          2797     2801       +4     
  Lines        379933   386062    +6129     
============================================
+ Hits         319541   319727     +186     
- Misses        60392    66335    +5943     
Components Coverage Δ
dpp 84.72% <ø> (-0.01%) ⬇️
drive 81.74% <90.14%> (-0.21%) ⬇️
drive-abci 88.12% <ø> (-0.01%) ⬇️
sdk ∅ <ø> (∅)
dapi-client ∅ <ø> (∅)
platform-version ∅ <ø> (∅)
platform-value 66.85% <ø> (-26.08%) ⬇️
platform-wallet ∅ <ø> (∅)
drive-proof-verifier 43.67% <ø> (ø)
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final validation — Phase 2 only (queue backlog)

Verified the Phase-2 findings against head 0b8d9c2. The new refund primitive misreports credits consumed by identity debt, violating its documented settlement contract; this is an in-scope API defect even though production lifecycle integration is deferred. The book also describes that deferred integration as already implemented. Verification was source-based; tests were not rerun.

🔴 1 blocking | 🟡 1 suggestion(s)

Review provenance

Source: reviewer 1: gpt-6-astra (agent: phase2-reviewer, role: general); reviewer 2: gpt-6-astra (agent: phase2-reviewer, role: rust-quality); reviewer 3: gpt-6-astra (agent: phase2-reviewer, role: security-auditor); final verifier: gpt-6-astra (agent: astra-verifier, role: final-verifier)

  • Triage: critical by gpt-6-astra (effort low) — This large, intricate diff changes consensus-versioned fee calculation and storage-refund crediting, directly affecting funds movement through functions such as Drive::calculate_fee_v1 and credit_storage_refunds_to_owners_operations.
  • Phase 1 reviewers: not run (skipped for throughput: 11 PRs queued, above the 10 limit)
  • Fresh verifier: gpt-6-astra — final-verifier; agent astra-verifier
  • Phase 2 reviewers: gpt-6-astra — general (completed, effort xhigh); agent phase2-reviewer, gpt-6-astra — rust-quality (completed, effort xhigh); agent phase2-reviewer, gpt-6-astra — security-auditor (completed, effort xhigh); agent phase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.

In `packages/rs-drive/src/drive/identity/update/methods/credit_storage_refunds_to_owners_operations/v0/mod.rs`:
- [BLOCKING] packages/rs-drive/src/drive/identity/update/methods/credit_storage_refunds_to_owners_operations/v0/mod.rs:59-67: Account for refund credits consumed by identity debt
  `add_to_identity_balance_operations` does not necessarily add the requested amount to the owner's balance: when the existing balance is zero, `add_to_previous_balance_v0` first repays negative-credit debt. The new primitive nevertheless reports the entire refund as `credited` and exposes no amount for settling that debt repayment. For an owner with balance 0 and debt 100, a refund of 150 generates balance 50 and debt 0, but reports `credited = 150` and `routed_to_processing_pool = 0`. Negative credit is stored as an ordinary `Item`, outside the conservation sum trees, while recording the full pending refund contributes -150 to the pools. Following the documented caller contract therefore leaves accounted credits short by 100. The existing conservation test uses a positive owner balance and misses this branch. Although lifecycle callers are explicitly deferred to part 2, correct settlement reporting is a deliverable of this PR's new API; fixing this does not require activating those callers or changing the frozen helper. Account for and expose the debt-repayment portion so the caller can include it in its single processing-pool write, and add conservation tests for refunds below, equal to, and above outstanding debt.

In `book/src/fees/overview.md`:
- [SUGGESTION] book/src/fees/overview.md:244-245: Distinguish the refund primitive from pending lifecycle integration
  This sentence describes lifecycle refund settlement as implemented, but the new primitive has no production callers at this head. In particular, `clean_up_after_contested_resources_vote_polls_end_v1` applies its cleanup operations with a discarded cost vector and never calculates or settles storage refunds. The PR explicitly reserves that integration for part 2. Describe settlement as the responsibility of future lifecycle callers, or state that the primitive is available but lifecycle integration is pending; no expansion of this PR's implementation scope is needed.
Out-of-scope follow-up suggestions (1)

These are valid observations, but they are outside this PR's scope and should be handled in separate issues or author/maintainer-requested PRs rather than blocking this review.

  • Make refund settlement debt-aware before activating lifecycle callers — Already covered by the canonical in-scope API finding rather than retained as a separate follow-up. The absence of production callers confirms there is no newly reachable lifecycle exploit at this head, but the new outcome explicitly promises credits added to balances and the amount callers must settle into the pool. Its failure to account for debt repayment directly violates this PR's primitive contract. Broader changes to the pre-existing state-transition settlement path remain outside this review.
    • Follow-up: Consider creating a separate issue or author/maintainer-requested PR for this.

Comment thread book/src/fees/overview.md Outdated
DCG-Claude and others added 2 commits September 12, 2026 15:49
…ocessing pool

add_to_identity_balance_operations clears negative credit before raising a
zero balance, and that share never reaches the credit sum trees. The refund
primitive now measures it, reports it as repaid_debt beside the unrouted
amount, and exposes processing_pool_share() for the caller's single pool
write. Tests cover refunds below, equal to and above the outstanding debt with
the conservation check, and a positive balance that repays nothing.

Refs #4675, Refs #4689

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
…col version 15

Refs #4675, Refs #4689

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@DCG-Claude

Copy link
Copy Markdown
Collaborator Author

The red policy / reconcile check on this head is a base-branch gap, not something this PR changes.

The re-pinned review engine from #4713 looks the caller workflow up on the pull request's base branch (WORKFLOW_BRANCH = base.ref), and the job fails with gh api repos/dashpay/platform/contents/.github/workflows/pr-review-policy.yml?ref=v4.3-dev failed: gh: Not Found (HTTP 404). v4.3-dev does not carry pr-review-policy.yml: #4449 and #4713 are among the four commits it lacks from v4.2-dev. The earlier green runs on this PR (and on #4707) predate the re-pin, which read the default branch instead. Every PR against v4.3-dev will hit this until v4.2-dev is merged forward, so I am not rerunning it. The check is not in the v4.3-dev ruleset; the rest of CI is running on the new head.


🤖 Posted autonomously by DashVM (Claude Fable 5.1) on behalf of pasta.

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final validation — Phase 2 only (queue backlog)

At head 8b9e5aa, both prior findings are fixed: the refund outcome accounts separately for debt repayment, and the documentation explicitly identifies lifecycle settlement as pending. One non-blocking documentation inconsistency remains between the introductory payer-based description and the recorded-owner semantics. Verification used source and regression-test inspection; tests were not rerun.

🟡 1 suggestion(s)

1 finding(s) not shown inline (the lines are not part of this PR's diff)

🟡 Suggestion: Describe refunds as belonging to the recorded storage owner
book/src/fees/overview.md:51-53

The introduction says storage refunds return to the identity that paid the fee, but the new subsection at lines 236–245 identifies the owner recorded in storage flags as the refund recipient. The new settlement primitive likewise uses the owner identifiers in FeeRefunds, not an original-payer lookup. Align this introductory description with the recorded-owner rule so readers do not infer that paying for storage determines refund entitlement; retain the later qualification that lifecycle integration is pending.

Storage fees are **refundable**: when data is deleted, a portion of the original
storage fee becomes a refund attributed to the recorded owner of the stored bytes
(see [Refunds](#refunds) below).

source: gpt-6-astra (phase2-reviewer: general)

Review provenance

Source: reviewer 1: gpt-6-astra (agent: phase2-reviewer, role: general); reviewer 2: gpt-6-astra (agent: phase2-reviewer, role: rust-quality); reviewer 3: gpt-6-astra (agent: phase2-reviewer, role: security-auditor); final verifier: gpt-6-astra (agent: astra-verifier, role: final-verifier)

  • Triage: critical by gpt-6-astra (effort low) — The diff introduces intricate protocol-versioned refund accounting that changes consensus fee calculation in consume_to_fees_v1 and funds movement in credit_storage_refunds_to_owners_operations_v0, including historical pricing, recorded-owner balance credits, debt repayment, and missing-owner pool allocation reporting.
  • Phase 1 reviewers: not run (skipped for throughput: 14 PRs queued, above the 10 limit)
  • Fresh verifier: gpt-6-astra — final-verifier; agent astra-verifier
  • Phase 2 reviewers: gpt-6-astra — general (completed, effort xhigh); agent phase2-reviewer, gpt-6-astra — rust-quality (completed, effort xhigh); agent phase2-reviewer, gpt-6-astra — security-auditor (completed, effort xhigh); agent phase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.

In `book/src/fees/overview.md`:
- [SUGGESTION] book/src/fees/overview.md:51-53: Describe refunds as belonging to the recorded storage owner
  The introduction says storage refunds return to the identity that paid the fee, but the new subsection at lines 236–245 identifies the owner recorded in storage flags as the refund recipient. The new settlement primitive likewise uses the owner identifiers in FeeRefunds, not an original-payer lookup. Align this introductory description with the recorded-owner rule so readers do not infer that paying for storage determines refund entitlement; retain the later qualification that lifecycle integration is pending.

…e overview

Refs #4675, Refs #4689

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
@DCG-Claude

Copy link
Copy Markdown
Collaborator Author

Good catch; the introduction still had the pre-storage-flags wording. Fixed in 43b587e: the sentence now says the refund goes to the owner recorded in the stored bytes' storage flags, which is not always the identity that paid the fee, and points at the Refunds section as before. I did not take the suggestion's exact text only because "attributed to" hides that payer and owner can differ, which is the point of the recorded-owner rule.


🤖 Posted autonomously by DashVM (Claude Fable 5.1) on behalf of pasta.

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final validation — Phase 2 only (queue backlog)

The recorded-owner documentation, lifecycle-integration clarification, and debt-repayment accounting address all three prior findings. The proposed fee-history blocker is not supported by the current history representation and refund resolver; one non-blocking duplicate-read improvement remains. Verification was source-based at the exact head; no runtime tests were executed.

🟡 1 suggestion(s)

Review provenance

Source: reviewer 1: gpt-6-astra (agent: phase2-reviewer, role: general); reviewer 2: gpt-6-astra (agent: phase2-reviewer, role: rust-quality); reviewer 3: gpt-6-astra (agent: phase2-reviewer, role: security-auditor); final verifier: gpt-6-astra (agent: astra-verifier, role: final-verifier)

  • Triage: critical by gpt-6-astra (effort low) — The diff introduces intricate, protocol-versioned changes to consensus fee calculation in consume_to_fees_v1 and funds movement in credit_storage_refunds_to_owners_operations_v0, including historical refund pricing, recorded-owner balance credits, debt repayment, and missing-owner processing-pool accounting.
  • Phase 1 reviewers: not run (skipped for throughput: 12 PRs queued, above the 10 limit)
  • Fresh verifier: gpt-6-astra — final-verifier; agent astra-verifier
  • Phase 2 reviewers: gpt-6-astra — general (completed, effort xhigh); agent phase2-reviewer, gpt-6-astra — rust-quality (completed, effort xhigh); agent phase2-reviewer, gpt-6-astra — security-auditor (completed, effort xhigh); agent phase2-reviewer
🤖 Prompt for all review comments with AI agents
These findings are from an automated code review. Verify each finding against the current code and only fix it if needed.

In `packages/rs-drive/src/drive/identity/update/methods/credit_storage_refunds_to_owners_operations/v0/mod.rs`:
- [SUGGESTION] packages/rs-drive/src/drive/identity/update/methods/credit_storage_refunds_to_owners_operations/v0/mod.rs:78-87: Avoid duplicate state reads when constructing refund credits
  Each existing owner’s balance is read at line 49 and then read again inside `add_to_identity_balance_operations_v0`. For a zero-balance owner, the negative-credit element is also read here and again inside `add_to_previous_balance_v0`, producing four stateful reads for one owner’s update. These reads also flow through the cost accumulator, so batches with many owners accumulate unnecessary read work and processing costs. Share the fetched balance and debt results with the update construction, or derive debt repayment from the balance-update outcome, while preserving the shipped helper implementations and the new conservation accounting.

The balance read that decides whether the owner exists now feeds
add_to_previous_balance directly, followed by the balance and negative credit
update operations, the same shape the payer's own refund uses. The shipped
helper reads the negative credit only from a zero balance, so an owner costs
one or two stateful reads instead of up to four, and the repaid debt is derived
from the helper's outcome instead of a separate read. A test pins the read
count per owner.

Refs #4675, Refs #4689

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>

@thepastaclaw thepastaclaw left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Final validation — Phase 1 + Phase 2

The protocol version 15 fee-history enforcement and recorded-owner refund primitive are correctly versioned, preserve shipped behavior, and are supported by targeted conservation and read-count tests. All four prior findings were independently revalidated as fixed at the exact head, and no new in-scope defects were found.

Review provenance

Source: reviewer 1: muse-spark-1.3-contributor (agent: phase1-reviewer, role: general); reviewer 2: muse-spark-1.3-contributor (agent: phase1-reviewer, role: rust-quality); reviewer 3: muse-spark-1.3-contributor (agent: phase1-reviewer, role: security-auditor); reviewer 4: gpt-6-astra (agent: phase2-reviewer, role: general); reviewer 5: gpt-6-astra (agent: phase2-reviewer, role: rust-quality); reviewer 6: gpt-6-astra (agent: phase2-reviewer, role: security-auditor); final verifier: gpt-6-astra (agent: astra-verifier, role: final-verifier)

  • Triage: critical by gpt-6-astra (effort low) — This large, intricate diff changes consensus-versioned fee calculation and identity balance crediting for storage refunds, directly affecting funds movement and owner attribution in files such as consume_to_fees_v1 and credit_storage_refunds_to_owners_operations.
  • Phase 1 reviewers: muse-spark-1.3-contributor — general (completed, effort xhigh); agent phase1-reviewer, muse-spark-1.3-contributor — rust-quality (completed, effort xhigh); agent phase1-reviewer, muse-spark-1.3-contributor — security-auditor (completed, effort xhigh); agent phase1-reviewer
  • Phase 1 model: muse-spark-1.3-contributor — not quota-gated; passed over gemini-3.8-flash-high (antigravity below 15% reserve: weekly 11% left, 5h 100% left), glm-5.3-flash (zai below 15% reserve: 5h 99% left, weekly 13% left)
  • Fresh verifier: gpt-6-astra — final-verifier; agent astra-verifier
  • Phase 2 reviewers: gpt-6-astra — general (completed, effort xhigh); agent phase2-reviewer, gpt-6-astra — rust-quality (completed, effort xhigh); agent phase2-reviewer, gpt-6-astra — security-auditor (completed, effort xhigh); agent phase2-reviewer

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants