From 94552187ce22bfc77443237d75089bd0ceb9e7df Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Thu, 13 Aug 2026 21:47:58 +0700 Subject: [PATCH 01/23] feat(platform)!: required document fields via contract updates (requiredSince) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A contract update may now add a new required property to a document type by annotating it with requiredSince equal to the contract version the update creates. Documents are stamped (serialization format 3) with the contract version their bytes conform to, so the latest contract alone reconstructs every stamp's byte layout — no historical contract lookups anywhere: - requiredSince property keyword in meta-schema v3, parsed onto DocumentProperty behind a new apply_required_since version slot (None on pre-v14 tables, so frozen parsers stay byte-identical) - document serialization format 3: a contract-version stamp varint after the format prefix; a property whose requiredSince exceeds the stamp keeps the presence-flagged layout it was written with (DOCUMENT_VERSIONS_V4, default 3, wired into v14 only; read dispatch stays prefix-driven) - legacy formats 0-2 read and write with required_at(None) — byte-identical for every schema without annotations (all shipped data), and it keeps old-format bytes readable under a schema that later gained a required field - validate_update v1 strips top-level required from the schema diff (the indices pattern) and judges it in dedicated Rust: additions only for brand-new properties carrying requiredSince == old version + 1; removals, promotions of existing properties, system fields, and retroactive values rejected with DataContractInvalidRequiredFieldsUpdateError (10276); the differ gets a frozen requiredSince rule so tampering is a clean consensus error instead of an unsupported-keyword hard error - Drive assigns the stamp at create/replace (beside creator_id); transfers and purchases re-serialize without touching it, so grandfathered documents stay transferable; contract creation rejects requiredSince other than 1 (basic_structure v2) Grandfathered documents remain valid and readable indefinitely; a replace re-supplies full content and must include the field (lazy migration). The stamp also gives clients an explicit staleness signal when a document is stamped above their cached contract version. Co-Authored-By: Claude Fable 5 --- .../document/v3/document-meta.json | 7 +- .../try_from_schema/common/mod.rs | 1 + .../class_methods/try_from_schema/mod.rs | 193 ++++ .../class_methods/try_from_schema/v0/mod.rs | 1 + .../methods/validate_update/common/mod.rs | 2 +- .../methods/validate_update/mod.rs | 8 +- .../methods/validate_update/v0/mod.rs | 4 +- .../methods/validate_update/v1/mod.rs | 352 ++++++- .../methods/versioned_methods.rs | 2 + .../src/data_contract/document_type/mod.rs | 1 + .../byte_array_encoding_flip_tests.rs | 1 + .../document_type/property/mod.rs | 44 + .../document_type/random_document.rs | 1 + .../validate_schema_compatibility/v1/mod.rs | 41 +- .../document_type/v0/random_document_type.rs | 2 + .../methods/validate_update/v0/mod.rs | 8 +- packages/rs-dpp/src/document/accessors/mod.rs | 1 + .../rs-dpp/src/document/document_event.rs | 1 + .../src/document/document_factory/v0/mod.rs | 1 + .../get_raw_for_document_type/v0/mod.rs | 2 + .../is_equal_ignoring_timestamps/v0/mod.rs | 1 + .../src/document/extended_document/mod.rs | 1 + packages/rs-dpp/src/document/mod.rs | 10 + .../deserialize/v0/mod.rs | 12 + .../serialize/v0/mod.rs | 9 + .../platform_value_conversion/mod.rs | 1 + .../rs-dpp/src/document/v0/cbor_conversion.rs | 4 + packages/rs-dpp/src/document/v0/mod.rs | 15 + .../document/v0/platform_value_conversion.rs | 2 + packages/rs-dpp/src/document/v0/serialize.rs | 855 +++++++++++++++++- .../src/errors/consensus/basic/basic_error.rs | 21 +- ...ct_invalid_required_fields_update_error.rs | 46 + .../consensus/basic/data_contract/mod.rs | 2 + packages/rs-dpp/src/errors/consensus/codes.rs | 1 + .../document_create_transition/v0/mod.rs | 2 + .../document_replace_transition/v0/mod.rs | 2 + packages/rs-dpp/src/tests/json_document.rs | 1 + packages/rs-dpp/src/tokens/token_event.rs | 1 + .../create_genesis_state/common.rs | 1 + .../basic_structure/mod.rs | 1 + .../basic_structure/v2/mod.rs | 82 ++ .../data_contract_create/mod.rs | 6 +- .../src/query/document_query/v0/mod.rs | 5 + .../src/query/document_query/v1/tests.rs | 1 + .../src/test/helpers/fee_pools.rs | 1 + .../benches/document_average_worst_case.rs | 1 + .../benches/document_count_worst_case.rs | 1 + .../benches/document_sum_worst_case.rs | 1 + .../contract/insert/add_description/v0/mod.rs | 1 + .../insert/add_new_keywords/v0/mod.rs | 1 + .../rs-drive/src/drive/document/update/mod.rs | 4 + packages/rs-drive/src/query/conditions.rs | 1 + .../drive_dispatcher.rs | 2 + .../query/drive_document_count_query/tests.rs | 5 + .../query/drive_document_sum_query/tests.rs | 1 + packages/rs-drive/src/query/mod.rs | 1 + .../address_credit_withdrawal_transition.rs | 1 + .../identity_credit_withdrawal_transition.rs | 1 + .../shielded_withdrawal_transition.rs | 1 + .../address_credit_withdrawal/mod.rs | 1 + .../v0/transformer.rs | 1 + .../v0/mod.rs | 34 + .../v0/mod.rs | 33 + .../identity_credit_withdrawal/mod.rs | 1 + .../v0/transformer.rs | 2 + .../shielded/shielded_withdrawal/mod.rs | 1 + .../shielded_withdrawal/v0/transformer.rs | 1 + .../util/object_size_info/document_info.rs | 1 + .../tests/drive_storage_ops_coverage.rs | 2 + .../src/rules/rule_set.rs | 48 + .../dpp_versions/dpp_contract_versions/mod.rs | 5 + .../dpp_versions/dpp_contract_versions/v1.rs | 1 + .../dpp_versions/dpp_contract_versions/v2.rs | 1 + .../dpp_versions/dpp_contract_versions/v3.rs | 1 + .../dpp_versions/dpp_contract_versions/v4.rs | 1 + .../dpp_versions/dpp_contract_versions/v5.rs | 1 + .../dpp_versions/dpp_contract_versions/v6.rs | 1 + .../dpp_versions/dpp_document_versions/mod.rs | 1 + .../dpp_versions/dpp_document_versions/v4.rs | 37 + .../drive_abci_validation_versions/v10.rs | 2 +- .../rs-platform-version/src/version/v14.rs | 15 +- .../rs-platform-wallet-ffi/src/document.rs | 1 + .../wallet/identity/network/contact_info.rs | 1 + .../identity/network/contact_requests.rs | 1 + .../src/wallet/identity/network/profile.rs | 2 + packages/rs-sdk-ffi/src/document/create.rs | 1 + packages/rs-sdk-ffi/src/document/delete.rs | 1 + packages/rs-sdk-ffi/src/document/price.rs | 1 + packages/rs-sdk-ffi/src/document/purchase.rs | 1 + packages/rs-sdk-ffi/src/document/put.rs | 1 + packages/rs-sdk-ffi/src/document/replace.rs | 1 + packages/rs-sdk-ffi/src/document/transfer.rs | 1 + .../src/platform/dashpay/contact_request.rs | 1 + .../platform/documents/transitions/delete.rs | 1 + .../documents/transitions/purchase.rs | 1 + .../documents/transitions/set_price.rs | 1 + .../documents/transitions/transfer.rs | 1 + .../rs-sdk/src/platform/dpns_usernames/mod.rs | 2 + .../src/errors/consensus/consensus_error.rs | 5 +- .../src/data_contract/document/model.rs | 1 + 100 files changed, 1907 insertions(+), 87 deletions(-) create mode 100644 packages/rs-dpp/src/errors/consensus/basic/data_contract/data_contract_invalid_required_fields_update_error.rs create mode 100644 packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/basic_structure/v2/mod.rs create mode 100644 packages/rs-platform-version/src/version/dpp_versions/dpp_document_versions/v4.rs diff --git a/packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json b/packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json index f5fb020a8b6..3d952b5663c 100644 --- a/packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json +++ b/packages/rs-dpp/schema/meta_schemas/document/v3/document-meta.json @@ -1,7 +1,7 @@ { "$schema": "https://json-schema.org/draft/2020-12/schema", "$id": "https://github.com/dashpay/platform/blob/master/packages/rs-dpp/schema/meta_schemas/document/v1/document-meta.json", - "$comment": "EDITABLE UNTIL THE RELEASE CARRYING PROTOCOL V14 SHIPS — FROZEN AFTER. This v3 document meta-schema activates with protocol v14 (CONTRACT_VERSIONS_V6). It is v2 plus the ranked index keywords (rankedCountable, rankedSummable, rankedAverageable) and the refersTo reference keyword on identifier properties, and admits every v14+ contract written to disk. v2 stays in place for protocol v13, where those keys still fail an index entry's `additionalProperties: false`. Once the release carrying protocol v14 ships, mutating it would change historical validation results and break consensus replay. After release, any new top-level property or rule MUST go in a newer meta-schema version (v4+). The $id above deliberately still names the v1 path: v1, v2 and v3 all share that identity, and it is the exact string `enrich_with_base_schema` injects as every PV12+ document schema's `$schema`, so bumping it here would be a wire-visible change rather than a documentation fix.", + "$comment": "EDITABLE UNTIL THE RELEASE CARRYING PROTOCOL V14 SHIPS — FROZEN AFTER. This v3 document meta-schema activates with protocol v14 (CONTRACT_VERSIONS_V6). It is v2 plus the ranked index keywords (rankedCountable, rankedSummable, rankedAverageable), the refersTo reference keyword on identifier properties, and the requiredSince property keyword (the contract version a property is required from), and admits every v14+ contract written to disk. v2 stays in place for protocol v13, where those keys still fail an index entry's `additionalProperties: false`. Once the release carrying protocol v14 ships, mutating it would change historical validation results and break consensus replay. After release, any new top-level property or rule MUST go in a newer meta-schema version (v4+). The $id above deliberately still names the v1 path: v1, v2 and v3 all share that identity, and it is the exact string `enrich_with_base_schema` injects as every PV12+ document schema's `$schema`, so bumping it here would be a wire-visible change rather than a documentation fix.", "type": "object", "$defs": { "documentProperties": { @@ -224,6 +224,11 @@ "position": { "type": "integer", "minimum": 0 + }, + "requiredSince": { + "type": "integer", + "minimum": 1, + "maximum": 4294967295 } }, "dependentSchemas": { diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/common/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/common/mod.rs index d54aac7b187..dc9e05b06fe 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/common/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/common/mod.rs @@ -722,6 +722,7 @@ fn parse_document_properties( &mut document_properties, &required_fields, &transient_fields, + true, property_key, property_value, root_schema, diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs index db391c3456e..45bf15e501f 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs @@ -128,6 +128,7 @@ fn insert_values( vec![(prefix, property_key, property_value)]; while let Some((prefix, property_key, property_value)) = to_visit.pop() { + let is_top_level = prefix.is_none(); let prefixed_property_key = match prefix { None => property_key, Some(prefix) => [prefix, property_key].join(".").to_owned(), @@ -143,6 +144,12 @@ fn insert_values( let is_required = known_required.contains(&prefixed_property_key); let is_transient = known_transient.contains(&prefixed_property_key); + let required_since = apply_required_since( + &inner_properties, + is_required, + is_top_level, + platform_version, + )?; match DocumentPropertyType::try_from_value_map(&inner_properties, &config.into())? { DocumentPropertyType::Object(_) => { @@ -179,6 +186,7 @@ fn insert_values( property_type, required: is_required, transient: is_transient, + required_since, }, ); } @@ -194,6 +202,7 @@ fn insert_values_nested( document_properties: &mut IndexMap, known_required: &BTreeSet, known_transient: &BTreeSet, + is_top_level: bool, property_key: String, property_value: &Value, root_schema: &Value, @@ -212,6 +221,13 @@ fn insert_values_nested( let is_transient = known_transient.contains(&property_key); + let required_since = apply_required_since( + &inner_properties, + is_required, + is_top_level, + platform_version, + )?; + let property_type = match DocumentPropertyType::try_from_value_map(&inner_properties, &config.into())? { DocumentPropertyType::Object(_) => { @@ -271,6 +287,7 @@ fn insert_values_nested( &mut nested_properties, &stripped_required, &stripped_transient, + false, object_property_string, object_property_value, root_schema, @@ -294,12 +311,79 @@ fn insert_values_nested( property_type, required: is_required, transient: is_transient, + required_since, }, ); Ok(()) } +/// Parses the `requiredSince` keyword: the contract version from which the +/// property is required. Only meaningful on top-level required properties — +/// the document wire format encodes a required property without a presence +/// flag, so requiredness that varies by contract version must be resolvable +/// per property from the current schema alone (see the per-document contract +/// version stamp in document serialization format 3). +/// +/// Versioned on `apply_required_since` in the platform version's document +/// type schema versions. `None` selects the behavior of the versions that +/// predate the keyword: it is ignored entirely, so their parses stay +/// byte-for-byte identical to what they always produced. +fn apply_required_since( + inner_properties: &BTreeMap, + is_required: bool, + is_top_level: bool, + platform_version: &PlatformVersion, +) -> Result, DataContractError> { + match platform_version + .dpp + .contract_versions + .document_type_versions + .schema + .apply_required_since + { + None => Ok(None), + Some(0) => apply_required_since_v0(inner_properties, is_required, is_top_level), + Some(version) => Err(DataContractError::Unsupported(format!( + "apply_required_since version {version} is not supported" + ))), + } +} + +fn apply_required_since_v0( + inner_properties: &BTreeMap, + is_required: bool, + is_top_level: bool, +) -> Result, DataContractError> { + let Some(required_since_value) = inner_properties.get(property_names::REQUIRED_SINCE) else { + return Ok(None); + }; + + if !is_top_level { + return Err(DataContractError::InvalidContractStructure( + "requiredSince is only allowed on top-level properties".to_string(), + )); + } + + if !is_required { + return Err(DataContractError::InvalidContractStructure( + "requiredSince is only allowed on properties listed in required".to_string(), + )); + } + + let required_since: u32 = required_since_value + .to_integer() + .map_err(|e| DataContractError::ValueWrongType(e.to_string()))?; + + if required_since == 0 { + return Err(DataContractError::InvalidContractStructure( + "requiredSince must be a contract version of at least 1".to_string(), + )); + } + + Ok(Some(required_since)) +} + /// Folds a `refersTo` declaration into the property type: an identifier property /// with `refersTo` becomes `IdentifierWithReference(target)`. Non-identifier /// properties cannot carry `refersTo`. @@ -804,4 +888,113 @@ mod tests { ) .expect("a parse predating refersTo should ignore the keyword entirely"); } + + // ================================================================ + // requiredSince + // ================================================================ + + #[test] + fn should_parse_required_since_on_top_level_required_property() { + let document_type = try_document_type_from_schema(json!({ + "type": "object", + "properties": { + "a": {"type": "string", "position": 0, "maxLength": 60}, + "b": {"type": "string", "position": 1, "maxLength": 60, "requiredSince": 3}, + }, + "required": ["a", "b"], + "additionalProperties": false + })) + .expect("should parse"); + + let properties = document_type.as_ref().flattened_properties().clone(); + assert_eq!(properties.get("a").unwrap().required_since, None); + assert_eq!(properties.get("b").unwrap().required_since, Some(3)); + assert!(properties.get("b").unwrap().required); + } + + #[test] + fn should_reject_required_since_on_optional_property() { + let result = try_document_type_from_schema(json!({ + "type": "object", + "properties": { + "a": {"type": "string", "position": 0, "maxLength": 60, "requiredSince": 2}, + }, + "required": [], + "additionalProperties": false + })); + + assert!( + result.is_err(), + "requiredSince on a property not listed in required must be rejected" + ); + } + + #[test] + fn should_reject_required_since_on_nested_property() { + let result = try_document_type_from_schema(json!({ + "type": "object", + "properties": { + "outer": { + "type": "object", + "position": 0, + "properties": { + "inner": {"type": "string", "position": 0, "maxLength": 60, "requiredSince": 2}, + }, + "required": ["inner"], + "additionalProperties": false + }, + }, + "required": [], + "additionalProperties": false + })); + + assert!( + result.is_err(), + "requiredSince on a nested property must be rejected" + ); + } + + #[test] + fn should_reject_required_since_of_zero() { + let result = try_document_type_from_schema(json!({ + "type": "object", + "properties": { + "a": {"type": "string", "position": 0, "maxLength": 60, "requiredSince": 0}, + }, + "required": ["a"], + "additionalProperties": false + })); + + assert!( + result.is_err(), + "requiredSince of 0 must be rejected (contract versions start at 1)" + ); + } + + #[test] + fn should_ignore_required_since_on_platform_versions_predating_it() { + // Platform versions whose tables carry `apply_required_since: None` + // predate the keyword: even if it appears in a schema they parse + // (only possible without full validation — their meta-schemas reject + // it), they must ignore it and keep producing the plain required + // property they always produced. + let platform_version = PlatformVersion::get(13).expect("platform version 13 should exist"); + + let document_type = try_document_type_from_schema_on_version( + json!({ + "type": "object", + "properties": { + "a": {"type": "string", "position": 0, "maxLength": 60, "requiredSince": 3}, + }, + "required": ["a"], + "additionalProperties": false + }), + platform_version, + ) + .expect("a parse predating requiredSince should ignore the keyword entirely"); + + let properties = document_type.as_ref().flattened_properties().clone(); + assert_eq!(properties.get("a").unwrap().required_since, None); + assert!(properties.get("a").unwrap().required); + } } diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v0/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v0/mod.rs index 686b8a42e76..fc41f482429 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v0/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/v0/mod.rs @@ -254,6 +254,7 @@ impl DocumentTypeV0 { &mut document_properties, &required_fields, &transient_fields, + true, property_key, property_value, &root_schema, diff --git a/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/common/mod.rs b/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/common/mod.rs index cae38edefce..ea05394a714 100644 --- a/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/common/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/common/mod.rs @@ -1712,7 +1712,7 @@ mod tests { let old = document_type_with_byte_array(old_ba, platform_version); let new = document_type_with_byte_array(new_ba, platform_version); old.as_ref() - .validate_update(new.as_ref(), platform_version) + .validate_update(new.as_ref(), 2, platform_version) .expect("validate_update should not error") } diff --git a/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/mod.rs b/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/mod.rs index a8fd6b443c5..6e4418eaeeb 100644 --- a/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/mod.rs @@ -9,10 +9,14 @@ mod v1; impl DocumentTypeRef<'_> { /// Verify that the update to the document type is valid. - /// We assume that new document type is valid + /// We assume that new document type is valid. + /// `new_contract_version` is the version the updated contract will have + /// (already validated to be the old version + 1): a newly added required + /// property must carry `requiredSince` equal to exactly that version. pub fn validate_update( &self, new_document_type: DocumentTypeRef, + new_contract_version: u32, platform_version: &PlatformVersion, ) -> Result { match platform_version @@ -22,7 +26,7 @@ impl DocumentTypeRef<'_> { .validate_update { 0 => self.validate_update_v0(new_document_type, platform_version), - 1 => self.validate_update_v1(new_document_type, platform_version), + 1 => self.validate_update_v1(new_document_type, new_contract_version, platform_version), version => Err(ProtocolError::UnknownVersionMismatch { method: "validate_update".to_string(), known_versions: vec![0, 1], diff --git a/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/v0/mod.rs b/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/v0/mod.rs index 46a4de01420..3f7616e6e47 100644 --- a/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/v0/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/v0/mod.rs @@ -132,7 +132,7 @@ mod tests { let early_result = old .as_ref() - .validate_update(new_early_name.as_ref(), platform_version) + .validate_update(new_early_name.as_ref(), 2, platform_version) .expect("early-name addition should produce a validation result"); assert_matches!( @@ -147,7 +147,7 @@ mod tests { // check ("schema keyword 'indices' ... is not supported"). let late_error = old .as_ref() - .validate_update(new_late_name.as_ref(), platform_version) + .validate_update(new_late_name.as_ref(), 2, platform_version) .expect_err("late-name addition should error in schema compatibility"); assert_matches!( diff --git a/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/v1/mod.rs b/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/v1/mod.rs index 8e872772233..76a6732fc20 100644 --- a/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/v1/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/methods/validate_update/v1/mod.rs @@ -20,7 +20,9 @@ //! was always rejected by the `indices` schema-compatibility hard error) — //! it makes the rejection deterministic, clean, and correctly labeled. -use crate::consensus::basic::data_contract::DataContractInvalidIndexDefinitionUpdateError; +use crate::consensus::basic::data_contract::{ + DataContractInvalidIndexDefinitionUpdateError, DataContractInvalidRequiredFieldsUpdateError, +}; use crate::data_contract::document_type::accessors::DocumentTypeV0Getters; use crate::data_contract::document_type::DocumentTypeRef; use crate::validation::SimpleConsensusValidationResult; @@ -32,6 +34,7 @@ impl DocumentTypeRef<'_> { pub(super) fn validate_update_v1( &self, new_document_type: DocumentTypeRef, + new_contract_version: u32, platform_version: &PlatformVersion, ) -> Result { // Validate configuration @@ -55,10 +58,102 @@ impl DocumentTypeRef<'_> { return Ok(result); } + // Validate required-field changes (the schema compatibility differ + // has the top-level `required` key stripped, so this is the only + // place top-level requiredness changes are judged) + let result = self.validate_required_fields_update(new_document_type, new_contract_version); + + if !result.is_valid() { + return Ok(result); + } + // Validate schema compatibility self.validate_schema(new_document_type, platform_version) } + /// Top-level requiredness may only change in one way: a brand-new + /// property may be added as required when it is annotated with + /// `requiredSince` equal to the contract version this update creates. + /// Everything else is frozen: requiredness is baked into the document + /// wire format (required properties serialize without a presence flag), + /// and the per-document contract-version stamp resolves layouts from the + /// latest schema alone only if annotations never change retroactively. + /// + /// Nested (dotted) required paths and the `requiredSince` keyword on + /// existing properties stay frozen by the schema compatibility differ; + /// this check judges the top-level `required` key, which is stripped + /// from the diff exactly like `indices`. + fn validate_required_fields_update( + &self, + new_document_type: DocumentTypeRef, + new_contract_version: u32, + ) -> SimpleConsensusValidationResult { + let old_required = self.required_fields(); + let new_required = new_document_type.required_fields(); + + for name in old_required { + // Nested paths are governed by the schema compatibility differ + if name.contains('.') { + continue; + } + if !new_required.contains(name) { + return SimpleConsensusValidationResult::new_with_error( + DataContractInvalidRequiredFieldsUpdateError::new( + self.name().to_string(), + format!("removed required field '{name}'"), + ) + .into(), + ); + } + } + + for name in new_required { + if name.contains('.') || old_required.contains(name) { + continue; + } + if name.starts_with('$') { + return SimpleConsensusValidationResult::new_with_error( + DataContractInvalidRequiredFieldsUpdateError::new( + self.name().to_string(), + format!("system field '{name}' cannot become required"), + ) + .into(), + ); + } + if self.properties().contains_key(name) { + return SimpleConsensusValidationResult::new_with_error( + DataContractInvalidRequiredFieldsUpdateError::new( + self.name().to_string(), + format!("existing property '{name}' cannot become required"), + ) + .into(), + ); + } + let Some(new_property) = new_document_type.properties().get(name) else { + return SimpleConsensusValidationResult::new_with_error( + DataContractInvalidRequiredFieldsUpdateError::new( + self.name().to_string(), + format!("added required field '{name}' references an unknown property"), + ) + .into(), + ); + }; + if new_property.required_since != Some(new_contract_version) { + return SimpleConsensusValidationResult::new_with_error( + DataContractInvalidRequiredFieldsUpdateError::new( + self.name().to_string(), + format!( + "new required field '{name}' must carry requiredSince {new_contract_version}, the contract version this update creates" + ), + ) + .into(), + ); + } + } + + SimpleConsensusValidationResult::new() + } + /// Index definitions are immutable once a document type is registered: /// Drive lays out the index trees at contract creation and never /// backfills them, so an added index would silently miss every @@ -196,7 +291,7 @@ mod tests { let early_result = old .as_ref() - .validate_update(new_early_name.as_ref(), platform_version) + .validate_update(new_early_name.as_ref(), 2, platform_version) .expect("validate_update should not error"); assert_matches!( @@ -208,7 +303,7 @@ mod tests { let late_result = old .as_ref() - .validate_update(new_late_name.as_ref(), platform_version) + .validate_update(new_late_name.as_ref(), 2, platform_version) .expect("validate_update should not error"); assert_matches!( @@ -242,7 +337,7 @@ mod tests { let result = old .as_ref() - .validate_update(new.as_ref(), platform_version) + .validate_update(new.as_ref(), 2, platform_version) .expect("validate_update should not error"); assert_matches!( @@ -268,7 +363,7 @@ mod tests { let result = old .as_ref() - .validate_update(new.as_ref(), platform_version) + .validate_update(new.as_ref(), 2, platform_version) .expect("validate_update should not error"); assert_matches!( @@ -295,7 +390,7 @@ mod tests { let result = old .as_ref() - .validate_update(new.as_ref(), platform_version) + .validate_update(new.as_ref(), 2, platform_version) .expect("validate_update should not error"); assert_matches!( @@ -325,7 +420,7 @@ mod tests { let result = old .as_ref() - .validate_update(new.as_ref(), platform_version) + .validate_update(new.as_ref(), 2, platform_version) .expect("validate_update should not error"); assert_matches!( @@ -359,7 +454,7 @@ mod tests { let result = old .as_ref() - .validate_update(new.as_ref(), platform_version) + .validate_update(new.as_ref(), 2, platform_version) .expect("validate_update should not error"); assert!( @@ -378,7 +473,7 @@ mod tests { let result = old .as_ref() - .validate_update(new.as_ref(), platform_version) + .validate_update(new.as_ref(), 2, platform_version) .expect("validate_update should not error"); assert!( @@ -465,7 +560,7 @@ mod tests { let result = old .as_ref() - .validate_update(new.as_ref(), platform_version) + .validate_update(new.as_ref(), 2, platform_version) .expect("validate_update should not error"); assert_matches!( @@ -484,7 +579,7 @@ mod tests { let result = old .as_ref() - .validate_update(new.as_ref(), platform_version) + .validate_update(new.as_ref(), 2, platform_version) .expect("validate_update should not error"); assert!( @@ -494,4 +589,239 @@ mod tests { ); } } + + // ================================================================ + // Required-field updates (`requiredSince`) + // ================================================================ + + mod required_fields_update { + use super::*; + + fn doc_type_with( + properties: Value, + required: Value, + platform_version: &PlatformVersion, + ) -> DocumentType { + let schema = platform_value!({ + "type": "object", + "properties": properties, + "required": required, + "additionalProperties": false, + }); + let config = DataContractConfig::default_for_version(platform_version) + .expect("should create a default config"); + DocumentType::try_from_schema( + Identifier::new([1; 32]), + 1, + config.version(), + "test", + schema, + None, + &BTreeMap::new(), + &config, + false, + &mut Vec::new(), + platform_version, + ) + .expect("failed to create document type") + } + + fn old_doc_type(platform_version: &PlatformVersion) -> DocumentType { + doc_type_with( + platform_value!({ + "a": {"type": "string", "position": 0, "maxLength": 60_u32}, + }), + platform_value!(["a"]), + platform_version, + ) + } + + #[test] + fn should_allow_adding_new_required_property_with_correct_required_since() { + let platform_version = PlatformVersion::latest(); + + let old = old_doc_type(platform_version); + let new = doc_type_with( + platform_value!({ + "a": {"type": "string", "position": 0, "maxLength": 60_u32}, + "b": {"type": "string", "position": 1, "maxLength": 60_u32, "requiredSince": 2}, + }), + platform_value!(["a", "b"]), + platform_version, + ); + + let result = old + .as_ref() + .validate_update(new.as_ref(), 2, platform_version) + .expect("validate_update should not error"); + + assert!( + result.is_valid(), + "a new required property annotated with the version this \ + update creates must be accepted, got {:?}", + result.errors + ); + } + + #[test] + fn should_reject_new_required_property_with_retroactive_required_since() { + let platform_version = PlatformVersion::latest(); + + let old = old_doc_type(platform_version); + // Contract moving to version 3, but the annotation claims 2: + // documents stamped 2 would misparse + let new = doc_type_with( + platform_value!({ + "a": {"type": "string", "position": 0, "maxLength": 60_u32}, + "b": {"type": "string", "position": 1, "maxLength": 60_u32, "requiredSince": 2}, + }), + platform_value!(["a", "b"]), + platform_version, + ); + + let result = old + .as_ref() + .validate_update(new.as_ref(), 3, platform_version) + .expect("validate_update should not error"); + + assert_matches!( + result.errors.as_slice(), + [ConsensusError::BasicError( + BasicError::DataContractInvalidRequiredFieldsUpdateError(e) + )] if e.details().contains("must carry requiredSince 3") + ); + } + + #[test] + fn should_reject_new_required_property_without_required_since() { + let platform_version = PlatformVersion::latest(); + + let old = old_doc_type(platform_version); + let new = doc_type_with( + platform_value!({ + "a": {"type": "string", "position": 0, "maxLength": 60_u32}, + "b": {"type": "string", "position": 1, "maxLength": 60_u32}, + }), + platform_value!(["a", "b"]), + platform_version, + ); + + let result = old + .as_ref() + .validate_update(new.as_ref(), 2, platform_version) + .expect("validate_update should not error"); + + assert_matches!( + result.errors.as_slice(), + [ConsensusError::BasicError( + BasicError::DataContractInvalidRequiredFieldsUpdateError(e) + )] if e.details().contains("must carry requiredSince 2") + ); + } + + #[test] + fn should_reject_promoting_existing_property_to_required() { + let platform_version = PlatformVersion::latest(); + + let old = doc_type_with( + platform_value!({ + "a": {"type": "string", "position": 0, "maxLength": 60_u32}, + "b": {"type": "string", "position": 1, "maxLength": 60_u32}, + }), + platform_value!(["a"]), + platform_version, + ); + let new = doc_type_with( + platform_value!({ + "a": {"type": "string", "position": 0, "maxLength": 60_u32}, + "b": {"type": "string", "position": 1, "maxLength": 60_u32}, + }), + platform_value!(["a", "b"]), + platform_version, + ); + + let result = old + .as_ref() + .validate_update(new.as_ref(), 2, platform_version) + .expect("validate_update should not error"); + + assert_matches!( + result.errors.as_slice(), + [ConsensusError::BasicError( + BasicError::DataContractInvalidRequiredFieldsUpdateError(e) + )] if e.details() == "existing property 'b' cannot become required" + ); + } + + #[test] + fn should_reject_removing_required_field() { + let platform_version = PlatformVersion::latest(); + + let old = doc_type_with( + platform_value!({ + "a": {"type": "string", "position": 0, "maxLength": 60_u32}, + "b": {"type": "string", "position": 1, "maxLength": 60_u32}, + }), + platform_value!(["a", "b"]), + platform_version, + ); + let new = doc_type_with( + platform_value!({ + "a": {"type": "string", "position": 0, "maxLength": 60_u32}, + "b": {"type": "string", "position": 1, "maxLength": 60_u32}, + }), + platform_value!(["a"]), + platform_version, + ); + + let result = old + .as_ref() + .validate_update(new.as_ref(), 2, platform_version) + .expect("validate_update should not error"); + + assert_matches!( + result.errors.as_slice(), + [ConsensusError::BasicError( + BasicError::DataContractInvalidRequiredFieldsUpdateError(e) + )] if e.details() == "removed required field 'b'" + ); + } + + #[test] + fn should_reject_mutating_required_since_on_existing_property() { + let platform_version = PlatformVersion::latest(); + + // The property was added as required at version 2; a later + // update must not move the annotation. This is caught by the + // compatibility differ's frozen `requiredSince` rule. + let old = doc_type_with( + platform_value!({ + "a": {"type": "string", "position": 0, "maxLength": 60_u32}, + "b": {"type": "string", "position": 1, "maxLength": 60_u32, "requiredSince": 2}, + }), + platform_value!(["a", "b"]), + platform_version, + ); + let new = doc_type_with( + platform_value!({ + "a": {"type": "string", "position": 0, "maxLength": 60_u32}, + "b": {"type": "string", "position": 1, "maxLength": 60_u32, "requiredSince": 3}, + }), + platform_value!(["a", "b"]), + platform_version, + ); + + let result = old + .as_ref() + .validate_update(new.as_ref(), 3, platform_version) + .expect("validate_update should not error"); + + assert_matches!( + result.errors.as_slice(), + [ConsensusError::BasicError( + BasicError::IncompatibleDocumentTypeSchemaError(e) + )] if e.operation() == "replace" && e.property_path() == "/properties/b/requiredSince" + ); + } + } } diff --git a/packages/rs-dpp/src/data_contract/document_type/methods/versioned_methods.rs b/packages/rs-dpp/src/data_contract/document_type/methods/versioned_methods.rs index 588c826c383..701a780e49f 100644 --- a/packages/rs-dpp/src/data_contract/document_type/methods/versioned_methods.rs +++ b/packages/rs-dpp/src/data_contract/document_type/methods/versioned_methods.rs @@ -166,6 +166,7 @@ pub trait DocumentTypeV0MethodsVersioned: DocumentTypeV0Getters + DocumentTypeBa { 0 => { let mut document = DocumentV0 { + contract_version: None, id: document_id, owner_id, properties: data @@ -328,6 +329,7 @@ pub trait DocumentTypeV0MethodsVersioned: DocumentTypeV0Getters + DocumentTypeBa .document_structure_version { 0 => Ok(DocumentV0 { + contract_version: None, id, owner_id, properties, diff --git a/packages/rs-dpp/src/data_contract/document_type/mod.rs b/packages/rs-dpp/src/data_contract/document_type/mod.rs index 3492b399571..46fae6efbe2 100644 --- a/packages/rs-dpp/src/data_contract/document_type/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/mod.rs @@ -62,6 +62,7 @@ pub(crate) mod property_names { pub const PROPERTIES: &str = "properties"; pub const POSITION: &str = "position"; pub const REQUIRED: &str = "required"; + pub const REQUIRED_SINCE: &str = "requiredSince"; pub const TRANSIENT: &str = "transient"; pub const TYPE: &str = "type"; pub const REF: &str = "$ref"; diff --git a/packages/rs-dpp/src/data_contract/document_type/property/byte_array_encoding_flip_tests.rs b/packages/rs-dpp/src/data_contract/document_type/property/byte_array_encoding_flip_tests.rs index 832ba61c864..948b0401352 100644 --- a/packages/rs-dpp/src/data_contract/document_type/property/byte_array_encoding_flip_tests.rs +++ b/packages/rs-dpp/src/data_contract/document_type/property/byte_array_encoding_flip_tests.rs @@ -105,6 +105,7 @@ fn build_document_with_ff_prefixed_bytes(_contract: &DataContract) -> Document { properties.insert(BYTE_ARRAY_FIELD.to_string(), Value::Bytes32(bytes)); DocumentV0 { + contract_version: None, id: Identifier::new([1; 32]), owner_id: Identifier::new([2; 32]), properties, diff --git a/packages/rs-dpp/src/data_contract/document_type/property/mod.rs b/packages/rs-dpp/src/data_contract/document_type/property/mod.rs index 7b1dc041afa..18a81473b07 100644 --- a/packages/rs-dpp/src/data_contract/document_type/property/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/property/mod.rs @@ -39,6 +39,25 @@ pub struct DocumentProperty { pub property_type: DocumentPropertyType, pub required: bool, pub transient: bool, + /// The contract version this property is required from (`requiredSince`). + /// `None` for plain-required properties (required at every version) and + /// for optional properties. Only ever `Some` when `required` is true. + #[serde(skip_serializing_if = "Option::is_none")] + pub required_since: Option, +} + +impl DocumentProperty { + /// Whether this property is required for a document whose bytes conform to + /// `contract_version` (the document's stamp). `None` means the document + /// was serialized before format 3, which predates every `requiredSince` + /// annotation, so only unconditionally required properties count. + pub fn required_at(&self, contract_version: Option) -> bool { + self.required + && match self.required_since { + None => true, + Some(since) => contract_version.is_some_and(|version| version >= since), + } + } } #[derive(Debug, PartialEq, Clone, Serialize)] @@ -2825,6 +2844,7 @@ mod tests { property_type: DocumentPropertyType::U32, required: true, transient: false, + required_since: None, }, ); sub_fields.insert( @@ -2833,6 +2853,7 @@ mod tests { property_type: DocumentPropertyType::U64, required: true, transient: false, + required_since: None, }, ); let obj = DocumentPropertyType::Object(sub_fields); @@ -5121,6 +5142,7 @@ mod tests { }), required: true, transient: false, + required_since: None, }, ); inner_fields.insert( @@ -5129,6 +5151,7 @@ mod tests { property_type: DocumentPropertyType::U32, required: true, transient: false, + required_since: None, }, ); let prop = DocumentPropertyType::Object(inner_fields); @@ -5179,6 +5202,7 @@ mod tests { }), required: true, transient: false, + required_since: None, }, ); let prop = DocumentPropertyType::Object(inner_fields); @@ -5198,6 +5222,7 @@ mod tests { property_type: DocumentPropertyType::U32, required: true, transient: false, + required_since: None, }, ); inner_fields.insert( @@ -5206,6 +5231,7 @@ mod tests { property_type: DocumentPropertyType::U64, required: false, transient: false, + required_since: None, }, ); let prop = DocumentPropertyType::Object(inner_fields); @@ -5638,6 +5664,7 @@ mod tests { property_type: DocumentPropertyType::U32, required: true, transient: false, + required_since: None, }, ); let prop = DocumentPropertyType::Object(inner_fields); @@ -5672,6 +5699,7 @@ mod tests { property_type: DocumentPropertyType::U32, required: true, transient: false, + required_since: None, }, ); sub_fields.insert( @@ -5680,6 +5708,7 @@ mod tests { property_type: DocumentPropertyType::U64, required: true, transient: false, + required_since: None, }, ); let obj = DocumentPropertyType::Object(sub_fields); @@ -5697,6 +5726,7 @@ mod tests { property_type: DocumentPropertyType::U16, required: true, transient: false, + required_since: None, }, ); sub_fields.insert( @@ -5705,6 +5735,7 @@ mod tests { property_type: DocumentPropertyType::Boolean, required: true, transient: false, + required_since: None, }, ); let obj = DocumentPropertyType::Object(sub_fields); @@ -5996,6 +6027,7 @@ mod tests { property_type: DocumentPropertyType::U32, required: true, transient: false, + required_since: None, }, ); sub_fields.insert( @@ -6004,6 +6036,7 @@ mod tests { property_type: DocumentPropertyType::U64, required: false, transient: false, + required_since: None, }, ); let prop = DocumentPropertyType::Object(sub_fields); @@ -6063,6 +6096,7 @@ mod tests { property_type: DocumentPropertyType::U32, required: true, transient: false, + required_since: None, }, ); sub_fields.insert( @@ -6071,6 +6105,7 @@ mod tests { property_type: DocumentPropertyType::U64, required: false, transient: false, + required_since: None, }, ); let prop = DocumentPropertyType::Object(sub_fields); @@ -6156,6 +6191,7 @@ mod tests { property_type: DocumentPropertyType::U8, required: true, transient: false, + required_since: None, }, ); sub_fields.insert( @@ -6164,6 +6200,7 @@ mod tests { property_type: DocumentPropertyType::Boolean, required: false, transient: false, + required_since: None, }, ); let prop = DocumentPropertyType::Object(sub_fields); @@ -6429,6 +6466,7 @@ mod tests { property_type: DocumentPropertyType::U32, required: true, transient: false, + required_since: None, }, ); let prop = DocumentPropertyType::Object(inner_fields); @@ -6457,6 +6495,7 @@ mod tests { property_type: DocumentPropertyType::U32, required: false, transient: false, + required_since: None, }, ); // Second field is required @@ -6466,6 +6505,7 @@ mod tests { property_type: DocumentPropertyType::U32, required: true, transient: false, + required_since: None, }, ); let prop = DocumentPropertyType::Object(inner_fields); @@ -6581,6 +6621,7 @@ mod tests { }), required: true, transient: false, + required_since: None, }, ); let prop = DocumentPropertyType::Object(inner_fields); @@ -6598,6 +6639,7 @@ mod tests { property_type: DocumentPropertyType::U32, required: false, transient: false, + required_since: None, }, ); let prop = DocumentPropertyType::Object(inner_fields); @@ -6903,6 +6945,7 @@ mod tests { property_type: DocumentPropertyType::U8, required: true, transient: false, + required_since: None, }, ); let prop = DocumentPropertyType::Object(sub_fields); @@ -7165,6 +7208,7 @@ mod tests { ), required: false, transient: false, + required_since: None, }; let value = serde_json::to_value(&property).expect("serialization should succeed"); diff --git a/packages/rs-dpp/src/data_contract/document_type/random_document.rs b/packages/rs-dpp/src/data_contract/document_type/random_document.rs index 39394248c1a..7797534e8dd 100644 --- a/packages/rs-dpp/src/data_contract/document_type/random_document.rs +++ b/packages/rs-dpp/src/data_contract/document_type/random_document.rs @@ -376,6 +376,7 @@ pub trait CreateRandomDocument: DocumentTypeV0Getters + DocumentTypeV0Methods { .document_structure_version { 0 => Ok(DocumentV0 { + contract_version: None, id, properties, owner_id, diff --git a/packages/rs-dpp/src/data_contract/document_type/schema/validate_schema_compatibility/v1/mod.rs b/packages/rs-dpp/src/data_contract/document_type/schema/validate_schema_compatibility/v1/mod.rs index bd369d8ed2b..b1bbcbca3cb 100644 --- a/packages/rs-dpp/src/data_contract/document_type/schema/validate_schema_compatibility/v1/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/schema/validate_schema_compatibility/v1/mod.rs @@ -16,6 +16,14 @@ //! diffing, so index definitions are validated in exactly one place. Only //! the document type's own `indices` keyword is removed; a *property* named //! `indices` lives under `/properties/indices` and is still validated. +//! +//! The top-level `required` key is stripped for the same reason: top-level +//! requiredness changes are judged by `validate_update` v1's +//! `validate_required_fields_update`, which admits exactly one change the +//! differ's frozen `required` rule cannot express — a brand-new property +//! added as required with `requiredSince` equal to the version the update +//! creates. Nested `required` arrays (under `/properties//required`) +//! remain frozen by the differ. use crate::data_contract::document_type::schema::IncompatibleJsonSchemaOperation; use crate::data_contract::errors::{DataContractError, JsonSchemaError}; @@ -48,30 +56,41 @@ static OPTIONS: Lazy = Lazy::new(|| { } }); -fn without_indices(schema: &JsonValue) -> Cow<'_, JsonValue> { +/// Strips the two top-level keys whose changes are validated by dedicated +/// checks in `validate_update` v1 instead of the JSON diff: `indices` +/// (index definitions compared by name) and `required` +/// (`validate_required_fields_update`, which admits new-property additions +/// annotated with `requiredSince`). Only the document type's own top-level +/// keys are removed; a nested object property's `required` array lives under +/// `/properties//required` and stays governed by the differ's frozen +/// `required` rule, as do properties named `indices` or `required`. +fn without_top_level_validated_keys(schema: &JsonValue) -> Cow<'_, JsonValue> { match schema { - JsonValue::Object(map) if map.contains_key("indices") => { + JsonValue::Object(map) if map.contains_key("indices") || map.contains_key("required") => { let mut map = map.clone(); map.remove("indices"); + map.remove("required"); Cow::Owned(JsonValue::Object(map)) } _ => Cow::Borrowed(schema), } } -/// Pairing invariant: stripping `indices` unconditionally is only safe -/// because every `PlatformVersion` that selects this generation -/// (`validate_schema_compatibility: 1`) also selects a `validate_update` -/// generation of at least 1 (`dpp.validation.document_type.validate_update`), -/// which rejects every real index change before this check runs. A future -/// version table that bumps one without the other would let index changes -/// bypass compatibility validation entirely. +/// Pairing invariant: stripping `indices` and top-level `required` +/// unconditionally is only safe because every `PlatformVersion` that selects +/// this generation (`validate_schema_compatibility: 1`) also selects a +/// `validate_update` generation of at least 1 +/// (`dpp.validation.document_type.validate_update`), which rejects every +/// real index change and every disallowed required-set change before this +/// check runs. A future version table that bumps one without the other +/// would let index or required changes bypass compatibility validation +/// entirely. pub(super) fn validate_schema_compatibility_v1( original_schema: &JsonValue, new_schema: &JsonValue, ) -> Result, ProtocolError> { - let original_schema = without_indices(original_schema); - let new_schema = without_indices(new_schema); + let original_schema = without_top_level_validated_keys(original_schema); + let new_schema = without_top_level_validated_keys(new_schema); validate_schemas_compatibility(&original_schema, &new_schema, OPTIONS.deref()) .map(|result| { diff --git a/packages/rs-dpp/src/data_contract/document_type/v0/random_document_type.rs b/packages/rs-dpp/src/data_contract/document_type/v0/random_document_type.rs index 17fdc0fe3a4..26231060fd9 100644 --- a/packages/rs-dpp/src/data_contract/document_type/v0/random_document_type.rs +++ b/packages/rs-dpp/src/data_contract/document_type/v0/random_document_type.rs @@ -197,6 +197,7 @@ impl DocumentTypeV0 { property_type: document_type, required, transient: false, + required_since: None, } }; @@ -526,6 +527,7 @@ impl DocumentTypeV0 { property_type: document_type, required, transient: false, + required_since: None, } }; diff --git a/packages/rs-dpp/src/data_contract/methods/validate_update/v0/mod.rs b/packages/rs-dpp/src/data_contract/methods/validate_update/v0/mod.rs index b269970d6e0..b71b6d2c09b 100644 --- a/packages/rs-dpp/src/data_contract/methods/validate_update/v0/mod.rs +++ b/packages/rs-dpp/src/data_contract/methods/validate_update/v0/mod.rs @@ -107,9 +107,11 @@ impl DataContract { }; // Validate document type update rules - let validate_update_result = old_document_type - .as_ref() - .validate_update(new_document_type, platform_version)?; + let validate_update_result = old_document_type.as_ref().validate_update( + new_document_type, + new_data_contract.version(), + platform_version, + )?; if !validate_update_result.is_valid() { return Ok(SimpleConsensusValidationResult::new_with_errors( diff --git a/packages/rs-dpp/src/document/accessors/mod.rs b/packages/rs-dpp/src/document/accessors/mod.rs index ecae8be34d1..0b52caebccd 100644 --- a/packages/rs-dpp/src/document/accessors/mod.rs +++ b/packages/rs-dpp/src/document/accessors/mod.rs @@ -223,6 +223,7 @@ mod tests { fn make_doc() -> Document { Document::V0(DocumentV0 { + contract_version: None, id: Identifier::new([1u8; 32]), owner_id: Identifier::new([2u8; 32]), properties: BTreeMap::new(), diff --git a/packages/rs-dpp/src/document/document_event.rs b/packages/rs-dpp/src/document/document_event.rs index b28086ce9a0..84477f13e93 100644 --- a/packages/rs-dpp/src/document/document_event.rs +++ b/packages/rs-dpp/src/document/document_event.rs @@ -103,6 +103,7 @@ impl DocumentEvent { } DocumentV0 { + contract_version: None, id: document_id, owner_id, properties, diff --git a/packages/rs-dpp/src/document/document_factory/v0/mod.rs b/packages/rs-dpp/src/document/document_factory/v0/mod.rs index 269dc16c092..55f82eedb26 100644 --- a/packages/rs-dpp/src/document/document_factory/v0/mod.rs +++ b/packages/rs-dpp/src/document/document_factory/v0/mod.rs @@ -574,6 +574,7 @@ mod test { platform_value::Value::Array(vec![]), ); let document_v0 = DocumentV0 { + contract_version: None, id: document_id, owner_id, properties, diff --git a/packages/rs-dpp/src/document/document_methods/get_raw_for_document_type/v0/mod.rs b/packages/rs-dpp/src/document/document_methods/get_raw_for_document_type/v0/mod.rs index d2e5464cf5a..e7c3eabb2e6 100644 --- a/packages/rs-dpp/src/document/document_methods/get_raw_for_document_type/v0/mod.rs +++ b/packages/rs-dpp/src/document/document_methods/get_raw_for_document_type/v0/mod.rs @@ -96,6 +96,7 @@ mod tests { fn make_document_with_known_ids() -> DocumentV0 { DocumentV0 { + contract_version: None, id: Identifier::new([0xAA; 32]), owner_id: Identifier::new([0xBB; 32]), properties: BTreeMap::new(), @@ -399,6 +400,7 @@ mod tests { fn minimal_doc() -> DocumentV0 { DocumentV0 { + contract_version: None, id: Identifier::new([1u8; 32]), owner_id: Identifier::new([2u8; 32]), properties: BTreeMap::new(), diff --git a/packages/rs-dpp/src/document/document_methods/is_equal_ignoring_timestamps/v0/mod.rs b/packages/rs-dpp/src/document/document_methods/is_equal_ignoring_timestamps/v0/mod.rs index a27dcb20c27..14c2a5ed610 100644 --- a/packages/rs-dpp/src/document/document_methods/is_equal_ignoring_timestamps/v0/mod.rs +++ b/packages/rs-dpp/src/document/document_methods/is_equal_ignoring_timestamps/v0/mod.rs @@ -56,6 +56,7 @@ mod tests { properties.insert("score".to_string(), Value::U64(100)); DocumentV0 { + contract_version: None, id: Identifier::new([1u8; 32]), owner_id: Identifier::new([2u8; 32]), properties, diff --git a/packages/rs-dpp/src/document/extended_document/mod.rs b/packages/rs-dpp/src/document/extended_document/mod.rs index 6b9fd509bde..19c2371a840 100644 --- a/packages/rs-dpp/src/document/extended_document/mod.rs +++ b/packages/rs-dpp/src/document/extended_document/mod.rs @@ -63,6 +63,7 @@ mod json_convertible_tests { let data_contract_id = data_contract.id(); let document = Document::V0(DocumentV0 { + contract_version: None, id: Identifier::new([0xa1; 32]), owner_id: Identifier::new([0xb2; 32]), properties: BTreeMap::new(), diff --git a/packages/rs-dpp/src/document/mod.rs b/packages/rs-dpp/src/document/mod.rs index dee2eef4bf3..ce4d97fba80 100644 --- a/packages/rs-dpp/src/document/mod.rs +++ b/packages/rs-dpp/src/document/mod.rs @@ -344,6 +344,7 @@ mod tests { #[test] fn display_document_with_no_properties() { let doc = Document::V0(DocumentV0 { + contract_version: None, id: platform_value::Identifier::new([0xAA; 32]), owner_id: platform_value::Identifier::new([0xBB; 32]), properties: Default::default(), @@ -371,6 +372,7 @@ mod tests { #[test] fn display_document_shows_transferred_at_fields() { let doc = Document::V0(DocumentV0 { + contract_version: None, id: platform_value::Identifier::new([1u8; 32]), owner_id: platform_value::Identifier::new([2u8; 32]), properties: Default::default(), @@ -409,6 +411,7 @@ mod tests { fn display_document_shows_creator_id() { let creator = platform_value::Identifier::new([0xCC; 32]); let doc = Document::V0(DocumentV0 { + contract_version: None, id: platform_value::Identifier::new([1u8; 32]), owner_id: platform_value::Identifier::new([2u8; 32]), properties: Default::default(), @@ -436,6 +439,7 @@ mod tests { #[test] fn display_document_shows_block_height_fields() { let doc = Document::V0(DocumentV0 { + contract_version: None, id: platform_value::Identifier::new([1u8; 32]), owner_id: platform_value::Identifier::new([2u8; 32]), properties: Default::default(), @@ -466,6 +470,7 @@ mod tests { #[test] fn increment_revision_works_on_mutable_document() { let mut doc = Document::V0(DocumentV0 { + contract_version: None, id: platform_value::Identifier::new([1u8; 32]), owner_id: platform_value::Identifier::new([2u8; 32]), properties: Default::default(), @@ -490,6 +495,7 @@ mod tests { #[test] fn increment_revision_fails_when_no_revision() { let mut doc = Document::V0(DocumentV0 { + contract_version: None, id: platform_value::Identifier::new([1u8; 32]), owner_id: platform_value::Identifier::new([2u8; 32]), properties: Default::default(), @@ -618,6 +624,7 @@ mod tests { #[test] fn increment_revision_errors_on_overflow() { let mut doc = Document::V0(DocumentV0 { + contract_version: None, id: platform_value::Identifier::new([1u8; 32]), owner_id: platform_value::Identifier::new([2u8; 32]), properties: Default::default(), @@ -647,6 +654,7 @@ mod tests { #[test] fn from_document_v0_produces_v0_variant() { let v0 = DocumentV0 { + contract_version: None, id: platform_value::Identifier::new([1u8; 32]), owner_id: platform_value::Identifier::new([2u8; 32]), properties: Default::default(), @@ -676,6 +684,7 @@ mod tests { #[test] fn document_display_has_version_prefix() { let doc = Document::V0(DocumentV0 { + contract_version: None, id: platform_value::Identifier::new([1u8; 32]), owner_id: platform_value::Identifier::new([2u8; 32]), properties: Default::default(), @@ -742,6 +751,7 @@ mod json_convertible_tests { fn fixture() -> Document { Document::V0(DocumentV0 { + contract_version: None, id: Identifier::new([0xa1; 32]), owner_id: Identifier::new([0xb2; 32]), properties: BTreeMap::new(), diff --git a/packages/rs-dpp/src/document/serialization_traits/platform_serialization_conversion/deserialize/v0/mod.rs b/packages/rs-dpp/src/document/serialization_traits/platform_serialization_conversion/deserialize/v0/mod.rs index 5db85a579fa..aa6686d6ccb 100644 --- a/packages/rs-dpp/src/document/serialization_traits/platform_serialization_conversion/deserialize/v0/mod.rs +++ b/packages/rs-dpp/src/document/serialization_traits/platform_serialization_conversion/deserialize/v0/mod.rs @@ -35,6 +35,18 @@ pub(in crate::document) trait DocumentPlatformDeserializationMethodsV0 { ) -> Result where Self: Sized; + + /// Reads a serialized document and creates a Document from it. + /// Version 3 has the contract version stamp, which selects each + /// `requiredSince` property's byte layout (raw when the stamp reaches the + /// property's `requiredSince`, presence-flagged otherwise). + fn from_bytes_v3( + serialized_document: &[u8], + document_type: DocumentTypeRef, + platform_version: &PlatformVersion, + ) -> Result + where + Self: Sized; } #[cfg(feature = "extended-document")] diff --git a/packages/rs-dpp/src/document/serialization_traits/platform_serialization_conversion/serialize/v0/mod.rs b/packages/rs-dpp/src/document/serialization_traits/platform_serialization_conversion/serialize/v0/mod.rs index 5f6238667b8..80700fd9a92 100644 --- a/packages/rs-dpp/src/document/serialization_traits/platform_serialization_conversion/serialize/v0/mod.rs +++ b/packages/rs-dpp/src/document/serialization_traits/platform_serialization_conversion/serialize/v0/mod.rs @@ -24,6 +24,15 @@ pub(in crate::document) trait DocumentPlatformSerializationMethodsV0 { /// id 32 bytes + owner_id 32 bytes + encoded values byte arrays /// Serialize v2 will serialize the creator id if the document can be transferred or sold fn serialize_v2(&self, document_type: DocumentTypeRef) -> Result, ProtocolError>; + + /// Serializes the document. + /// + /// The serialization of a document follows the pattern: + /// contract version stamp varint + id 32 bytes + owner_id 32 bytes + encoded values byte arrays + /// Serialize v3 stamps the document with the data contract version its + /// bytes conform to, and encodes a property whose `requiredSince` exceeds + /// the stamp with a presence flag instead of raw + fn serialize_v3(&self, document_type: DocumentTypeRef) -> Result, ProtocolError>; } #[cfg(feature = "extended-document")] diff --git a/packages/rs-dpp/src/document/serialization_traits/platform_value_conversion/mod.rs b/packages/rs-dpp/src/document/serialization_traits/platform_value_conversion/mod.rs index 5d451bce535..2e6c7ea9eb3 100644 --- a/packages/rs-dpp/src/document/serialization_traits/platform_value_conversion/mod.rs +++ b/packages/rs-dpp/src/document/serialization_traits/platform_value_conversion/mod.rs @@ -159,6 +159,7 @@ mod tests { let owner_id = Identifier::new([2u8; 32]); let doc_v0 = DocumentV0 { + contract_version: None, id, owner_id, properties: std::collections::BTreeMap::new(), diff --git a/packages/rs-dpp/src/document/v0/cbor_conversion.rs b/packages/rs-dpp/src/document/v0/cbor_conversion.rs index 190fa5ffa69..36c3621e2e1 100644 --- a/packages/rs-dpp/src/document/v0/cbor_conversion.rs +++ b/packages/rs-dpp/src/document/v0/cbor_conversion.rs @@ -80,6 +80,8 @@ impl TryFrom for DocumentForCbor { updated_at_core_block_height, transferred_at_core_block_height, creator_id, + // The CBOR document form predates the contract-version stamp + contract_version: _, } = value; Ok(DocumentForCbor { id: id.to_buffer(), @@ -148,6 +150,7 @@ impl DocumentV0 { // dev-note: properties is everything other than the id and owner id Ok(DocumentV0 { + contract_version: None, properties: document_map, owner_id: Identifier::new(owner_id), id: Identifier::new(id), @@ -229,6 +232,7 @@ mod tests { properties.insert("name".to_string(), Value::Text("Alice".to_string())); properties.insert("age".to_string(), Value::U64(30)); DocumentV0 { + contract_version: None, id, owner_id, properties, diff --git a/packages/rs-dpp/src/document/v0/mod.rs b/packages/rs-dpp/src/document/v0/mod.rs index 130908e1419..406d36023d8 100644 --- a/packages/rs-dpp/src/document/v0/mod.rs +++ b/packages/rs-dpp/src/document/v0/mod.rs @@ -98,6 +98,20 @@ pub struct DocumentV0 { /// The creator id. #[cfg_attr(feature = "serde-conversion", serde(rename = "$creatorId", default))] pub creator_id: Option, + /// The data contract version this document's bytes conform to — assigned + /// by Drive when document content is (re-)supplied (create/replace) and + /// preserved across server-side rewrites (transfer/purchase). Selects the + /// per-property byte layout when the document type carries `requiredSince` + /// annotations. `None` for documents serialized before format 3. + #[cfg_attr( + feature = "serde-conversion", + serde( + rename = "$contractVersion", + default, + skip_serializing_if = "Option::is_none" + ) + )] + pub contract_version: Option, } impl DocumentGetRawForContractV0 for DocumentV0 { @@ -197,6 +211,7 @@ mod tests { fn minimal_doc() -> DocumentV0 { DocumentV0 { + contract_version: None, id: Identifier::new([1u8; 32]), owner_id: Identifier::new([2u8; 32]), properties: BTreeMap::new(), diff --git a/packages/rs-dpp/src/document/v0/platform_value_conversion.rs b/packages/rs-dpp/src/document/v0/platform_value_conversion.rs index fa2902cd9eb..8811958b5d9 100644 --- a/packages/rs-dpp/src/document/v0/platform_value_conversion.rs +++ b/packages/rs-dpp/src/document/v0/platform_value_conversion.rs @@ -22,6 +22,7 @@ mod tests { fn minimal_doc() -> DocumentV0 { DocumentV0 { + contract_version: None, id: Identifier::new([1u8; 32]), owner_id: Identifier::new([2u8; 32]), properties: BTreeMap::new(), @@ -44,6 +45,7 @@ mod tests { props.insert("name".into(), Value::Text("Eve".into())); props.insert("score".into(), Value::U64(42)); DocumentV0 { + contract_version: None, id: Identifier::new([7u8; 32]), owner_id: Identifier::new([8u8; 32]), properties: props, diff --git a/packages/rs-dpp/src/document/v0/serialize.rs b/packages/rs-dpp/src/document/v0/serialize.rs index 14d0024b444..f187d5c05b9 100644 --- a/packages/rs-dpp/src/document/v0/serialize.rs +++ b/packages/rs-dpp/src/document/v0/serialize.rs @@ -216,7 +216,7 @@ impl DocumentPlatformSerializationMethodsV0 for DocumentV0 { .try_for_each(|(field_name, property)| { if let Some(value) = self.properties.get(field_name) { if value.is_null() { - if property.required && !property.transient { + if property.required_at(None) && !property.transient { Err(ProtocolError::DataContractError( DataContractError::MissingRequiredKey( "a required field is not present".to_string(), @@ -229,24 +229,24 @@ impl DocumentPlatformSerializationMethodsV0 for DocumentV0 { Ok(()) } } else { - if !property.required || property.transient { + if !property.required_at(None) || property.transient { // dbg!("we added 1", field_name); buffer.push(1); } let value = if property.property_type.is_integer() { DocumentPropertyType::I64 - .encode_value_ref_with_size(value, property.required) + .encode_value_ref_with_size(value, property.required_at(None)) } else { property .property_type - .encode_value_ref_with_size(value, property.required) + .encode_value_ref_with_size(value, property.required_at(None)) }?; // dbg!("we pushed {} with {}", field_name, hex::encode(&value)); buffer.extend(value.as_slice()); Ok(()) } - } else if property.required && !property.transient { + } else if property.required_at(None) && !property.transient { Err(ProtocolError::DataContractError( DataContractError::MissingRequiredKey(format!( "a required field {field_name} is not present" @@ -440,7 +440,7 @@ impl DocumentPlatformSerializationMethodsV0 for DocumentV0 { .try_for_each(|(field_name, property)| { if let Some(value) = self.properties.get(field_name) { if value.is_null() { - if property.required && !property.transient { + if property.required_at(None) && !property.transient { Err(ProtocolError::DataContractError( DataContractError::MissingRequiredKey( "a required field is not present".to_string(), @@ -453,18 +453,18 @@ impl DocumentPlatformSerializationMethodsV0 for DocumentV0 { Ok(()) } } else { - if !property.required || property.transient { + if !property.required_at(None) || property.transient { // dbg!("we added 1", field_name); buffer.push(1); } let value = property .property_type - .encode_value_ref_with_size(value, property.required)?; + .encode_value_ref_with_size(value, property.required_at(None))?; // dbg!("we pushed {} with {}", field_name, hex::encode(&value)); buffer.extend(value.as_slice()); Ok(()) } - } else if property.required && !property.transient { + } else if property.required_at(None) && !property.transient { Err(ProtocolError::DataContractError( DataContractError::MissingRequiredKey(format!( "a required field {field_name} is not present" @@ -668,7 +668,7 @@ impl DocumentPlatformSerializationMethodsV0 for DocumentV0 { .try_for_each(|(field_name, property)| { if let Some(value) = self.properties.get(field_name) { if value.is_null() { - if property.required && !property.transient { + if property.required_at(None) && !property.transient { Err(ProtocolError::DataContractError( DataContractError::MissingRequiredKey( "a required field is not present".to_string(), @@ -681,18 +681,18 @@ impl DocumentPlatformSerializationMethodsV0 for DocumentV0 { Ok(()) } } else { - if !property.required || property.transient { + if !property.required_at(None) || property.transient { // dbg!("we added 1", field_name); buffer.push(1); } let value = property .property_type - .encode_value_ref_with_size(value, property.required)?; + .encode_value_ref_with_size(value, property.required_at(None))?; // dbg!("we pushed {} with {}", field_name, hex::encode(&value)); buffer.extend(value.as_slice()); Ok(()) } - } else if property.required && !property.transient { + } else if property.required_at(None) && !property.transient { Err(ProtocolError::DataContractError( DataContractError::MissingRequiredKey(format!( "a required field {field_name} is not present" @@ -708,11 +708,469 @@ impl DocumentPlatformSerializationMethodsV0 for DocumentV0 { Ok(buffer) } + + /// Serializes the document. + /// + /// Serialize v3 is v2 plus the contract version stamp: a varint right + /// after the format prefix recording the data contract version the bytes + /// conform to (0 = unstamped, for pre-format-3 documents that are + /// re-serialized). A property whose `requiredSince` exceeds the stamp is + /// encoded with a presence flag exactly like an optional property, so + /// documents written before the property became required stay valid. + fn serialize_v3(&self, document_type: DocumentTypeRef) -> Result, ProtocolError> { + let mut buffer: Vec = 3u64.encode_var_vec(); //version 3 + + // the contract version stamp; 0 means unstamped + buffer.extend((self.contract_version.unwrap_or_default() as u64).encode_var_vec()); + + // $id + buffer.extend(self.id.as_slice()); + + // $ownerId + buffer.extend(self.owner_id.as_slice()); + + if document_type.trade_mode() != TradeMode::None + || document_type.documents_transferable().is_transferable() + { + if let Some(creator_id) = self.creator_id { + buffer.push(1); + buffer.extend(creator_id.as_slice()); + } else { + buffer.push(0); + } + } + + // $revision + if let Some(revision) = self.revision { + buffer.extend(revision.encode_var_vec()) + } else if document_type.requires_revision() { + buffer.extend((1 as Revision).encode_var_vec()) + } + + let mut bitwise_exists_flag: u16 = 0; + + let mut time_fields_data_buffer = vec![]; + + // $createdAt + if let Some(created_at) = &self.created_at { + bitwise_exists_flag |= 1; + time_fields_data_buffer.extend(created_at.to_be_bytes()); + } else if document_type.required_fields().contains(CREATED_AT) { + return Err(ProtocolError::DataContractError( + DataContractError::MissingRequiredKey( + "created at field is not present".to_string(), + ), + )); + } + + // $updatedAt + if let Some(updated_at) = &self.updated_at { + bitwise_exists_flag |= 2; + time_fields_data_buffer.extend(updated_at.to_be_bytes()); + } else if document_type.required_fields().contains(UPDATED_AT) { + return Err(ProtocolError::DataContractError( + DataContractError::MissingRequiredKey( + "updated at field is not present".to_string(), + ), + )); + } + + // $transferredAt + if let Some(transferred_at) = &self.transferred_at { + bitwise_exists_flag |= 4; + time_fields_data_buffer.extend(transferred_at.to_be_bytes()); + } else if document_type.required_fields().contains(TRANSFERRED_AT) { + return Err(ProtocolError::DataContractError( + DataContractError::MissingRequiredKey( + "transferred at field is not present".to_string(), + ), + )); + } + + // $createdAtBlockHeight + if let Some(created_at_block_height) = &self.created_at_block_height { + bitwise_exists_flag |= 8; + time_fields_data_buffer.extend(created_at_block_height.to_be_bytes()); + } else if document_type + .required_fields() + .contains(CREATED_AT_BLOCK_HEIGHT) + { + return Err(ProtocolError::DataContractError( + DataContractError::MissingRequiredKey( + "created_at_block_height field is not present".to_string(), + ), + )); + } + + // $updatedAtBlockHeight + if let Some(updated_at_block_height) = &self.updated_at_block_height { + bitwise_exists_flag |= 16; + time_fields_data_buffer.extend(updated_at_block_height.to_be_bytes()); + } else if document_type + .required_fields() + .contains(UPDATED_AT_BLOCK_HEIGHT) + { + return Err(ProtocolError::DataContractError( + DataContractError::MissingRequiredKey( + "updated_at_block_height field is not present".to_string(), + ), + )); + } + + // $transferredAtBlockHeight + if let Some(transferred_at_block_height) = &self.transferred_at_block_height { + bitwise_exists_flag |= 32; + time_fields_data_buffer.extend(transferred_at_block_height.to_be_bytes()); + } else if document_type + .required_fields() + .contains(TRANSFERRED_AT_BLOCK_HEIGHT) + { + return Err(ProtocolError::DataContractError( + DataContractError::MissingRequiredKey( + "transferred_at_block_height field is not present".to_string(), + ), + )); + } + + // $createdAtCoreBlockHeight + if let Some(created_at_core_block_height) = &self.created_at_core_block_height { + bitwise_exists_flag |= 64; + time_fields_data_buffer.extend(created_at_core_block_height.to_be_bytes()); + } else if document_type + .required_fields() + .contains(CREATED_AT_CORE_BLOCK_HEIGHT) + { + return Err(ProtocolError::DataContractError( + DataContractError::MissingRequiredKey( + "created_at_core_block_height field is not present".to_string(), + ), + )); + } + + // $updatedAtCoreBlockHeight + if let Some(updated_at_core_block_height) = &self.updated_at_core_block_height { + bitwise_exists_flag |= 128; + time_fields_data_buffer.extend(updated_at_core_block_height.to_be_bytes()); + } else if document_type + .required_fields() + .contains(UPDATED_AT_CORE_BLOCK_HEIGHT) + { + return Err(ProtocolError::DataContractError( + DataContractError::MissingRequiredKey( + "updated_at_core_block_height field is not present".to_string(), + ), + )); + } + + // $transferredAtCoreBlockHeight + if let Some(transferred_at_core_block_height) = &self.transferred_at_core_block_height { + bitwise_exists_flag |= 256; + time_fields_data_buffer.extend(transferred_at_core_block_height.to_be_bytes()); + } else if document_type + .required_fields() + .contains(TRANSFERRED_AT_CORE_BLOCK_HEIGHT) + { + return Err(ProtocolError::DataContractError( + DataContractError::MissingRequiredKey( + "transferred_at_core_block_height field is not present".to_string(), + ), + )); + } + + buffer.extend(bitwise_exists_flag.to_be_bytes().as_slice()); + buffer.append(&mut time_fields_data_buffer); + + // Now we serialize the price which might not be necessary unless called for by the document type + + if document_type.trade_mode().seller_sets_price() { + if let Some(price) = self.properties.get(PRICE) { + buffer.push(1); + let price_as_u64: u64 = price.to_integer().map_err(ProtocolError::ValueError)?; + buffer.append(&mut price_as_u64.to_be_bytes().to_vec()); + } else { + buffer.push(0); + } + } + + // User defined properties: requiredness is evaluated at this + // document's stamp, so a property that became required after the + // stamp keeps the presence-flagged layout it was written with + document_type + .properties() + .iter() + .try_for_each(|(field_name, property)| { + let required = property.required_at(self.contract_version); + if let Some(value) = self.properties.get(field_name) { + if value.is_null() { + if required && !property.transient { + Err(ProtocolError::DataContractError( + DataContractError::MissingRequiredKey( + "a required field is not present".to_string(), + ), + )) + } else { + // We don't have something that wasn't required + buffer.push(0); + Ok(()) + } + } else { + if !required || property.transient { + buffer.push(1); + } + let value = property + .property_type + .encode_value_ref_with_size(value, required)?; + buffer.extend(value.as_slice()); + Ok(()) + } + } else if required && !property.transient { + Err(ProtocolError::DataContractError( + DataContractError::MissingRequiredKey(format!( + "a required field {field_name} is not present" + )), + )) + } else { + // We don't have something that wasn't required + buffer.push(0); + Ok(()) + } + })?; + + Ok(buffer) + } } -impl DocumentPlatformDeserializationMethodsV0 for DocumentV0 { +impl DocumentPlatformDeserializationMethodsV0 for DocumentV0 { + /// Reads a serialized document and creates a Document from it. + fn from_bytes_v0( + serialized_document: &[u8], + document_type: DocumentTypeRef, + _platform_version: &PlatformVersion, + ) -> Result { + let mut buf = BufReader::new(serialized_document); + if serialized_document.len() < 64 { + return Err(DataContractError::DecodingDocumentError( + DecodingError::new( + "serialized document is too small, must have id and owner id".to_string(), + ), + )); + } + + // $id + let mut id = [0; 32]; + buf.read_exact(&mut id).map_err(|_| { + DataContractError::DecodingDocumentError(DecodingError::new( + "error reading from serialized document for id".to_string(), + )) + })?; + + // $ownerId + let mut owner_id = [0; 32]; + buf.read_exact(&mut owner_id).map_err(|_| { + DataContractError::DecodingDocumentError(DecodingError::new( + "error reading from serialized document for owner id".to_string(), + )) + })?; + + // $revision + // if the document type is mutable then we should deserialize the revision + let revision: Option = if document_type.requires_revision() { + let revision = buf.read_varint().map_err(|_| { + DataContractError::DecodingDocumentError(DecodingError::new( + "error reading revision from serialized document for revision".to_string(), + )) + })?; + Some(revision) + } else { + None + }; + + let timestamp_flags = buf.read_u16::().map_err(|_| { + DataContractError::CorruptedSerialization( + "error reading timestamp flags from serialized document".to_string(), + ) + })?; + + let created_at = if timestamp_flags & 1 > 0 { + Some(buf.read_u64::().map_err(|_| { + DataContractError::CorruptedSerialization( + "error reading created_at timestamp from serialized document".to_string(), + ) + })?) + } else { + None + }; + + let updated_at = if timestamp_flags & 2 > 0 { + Some(buf.read_u64::().map_err(|_| { + DataContractError::CorruptedSerialization( + "error reading updated_at timestamp from serialized document".to_string(), + ) + })?) + } else { + None + }; + + let transferred_at = if timestamp_flags & 4 > 0 { + Some(buf.read_u64::().map_err(|_| { + DataContractError::CorruptedSerialization( + "error reading transferred_at timestamp from serialized document".to_string(), + ) + })?) + } else { + None + }; + + let created_at_block_height = if timestamp_flags & 8 > 0 { + Some(buf.read_u64::().map_err(|_| { + DataContractError::CorruptedSerialization( + "error reading created_at_block_height from serialized document".to_string(), + ) + })?) + } else { + None + }; + + let updated_at_block_height = if timestamp_flags & 16 > 0 { + Some(buf.read_u64::().map_err(|_| { + DataContractError::CorruptedSerialization( + "error reading updated_at_block_height from serialized document".to_string(), + ) + })?) + } else { + None + }; + + let transferred_at_block_height = if timestamp_flags & 32 > 0 { + Some(buf.read_u64::().map_err(|_| { + DataContractError::CorruptedSerialization( + "error reading transferred_at_block_height from serialized document" + .to_string(), + ) + })?) + } else { + None + }; + + let created_at_core_block_height = if timestamp_flags & 64 > 0 { + Some(buf.read_u32::().map_err(|_| { + DataContractError::CorruptedSerialization( + "error reading created_at_core_block_height from serialized document" + .to_string(), + ) + })?) + } else { + None + }; + + let updated_at_core_block_height = if timestamp_flags & 128 > 0 { + Some(buf.read_u32::().map_err(|_| { + DataContractError::CorruptedSerialization( + "error reading updated_at_core_block_height from serialized document" + .to_string(), + ) + })?) + } else { + None + }; + + let transferred_at_core_block_height = if timestamp_flags & 256 > 0 { + Some(buf.read_u32::().map_err(|_| { + DataContractError::CorruptedSerialization( + "error reading updated_at_core_block_height from serialized document" + .to_string(), + ) + })?) + } else { + None + }; + + // Now we deserialize the price which might not be necessary unless called for by the document type + + let price = if document_type.trade_mode().seller_sets_price() { + let has_price = buf.read_u8().map_err(|_| { + DataContractError::CorruptedSerialization( + "error reading has price bool from serialized document".to_string(), + ) + })?; + if has_price > 0 { + let price = buf.read_u64::().map_err(|_| { + DataContractError::CorruptedSerialization( + "error reading price u64 from serialized document".to_string(), + ) + })?; + Some(price) + } else { + None + } + } else { + None + }; + + let mut finished_buffer = false; + + let mut properties = document_type + .properties() + .iter() + .filter_map(|(key, property)| { + if finished_buffer { + return if property.required_at(None) && !property.transient { + Some(Err(DataContractError::CorruptedSerialization( + "required field after finished buffer".to_string(), + ))) + } else { + None + }; + } + + // In version 0 all integers are encoded as I64 (in theory) + let read_value = if property.property_type.is_integer() { + DocumentPropertyType::I64.read_optionally_from( + &mut buf, + property.required_at(None) & !property.transient, + ) + } else { + property.property_type.read_optionally_from( + &mut buf, + property.required_at(None) & !property.transient, + ) + }; + + match read_value { + Ok(read_value) => { + finished_buffer |= read_value.1; + read_value.0.map(|read_value| Ok((key.clone(), read_value))) + } + Err(e) => Some(Err(e)), + } + }) + .collect::, DataContractError>>()?; + + if let Some(price) = price { + properties.insert(PRICE.to_string(), price.into()); + } + + Ok(DocumentV0 { + contract_version: None, + id: Identifier::new(id), + properties, + owner_id: Identifier::new(owner_id), + revision, + created_at, + updated_at, + transferred_at, + created_at_block_height, + updated_at_block_height, + transferred_at_block_height, + created_at_core_block_height, + updated_at_core_block_height, + transferred_at_core_block_height, + creator_id: None, + }) + } + /// Reads a serialized document and creates a Document from it. - fn from_bytes_v0( + fn from_bytes_v1( serialized_document: &[u8], document_type: DocumentTypeRef, _platform_version: &PlatformVersion, @@ -884,7 +1342,7 @@ impl DocumentPlatformDeserializationMethodsV0 for DocumentV0 { .iter() .filter_map(|(key, property)| { if finished_buffer { - return if property.required && !property.transient { + return if property.required_at(None) && !property.transient { Some(Err(DataContractError::CorruptedSerialization( "required field after finished buffer".to_string(), ))) @@ -892,16 +1350,10 @@ impl DocumentPlatformDeserializationMethodsV0 for DocumentV0 { None }; } - - // In version 0 all integers are encoded as I64 (in theory) - let read_value = if property.property_type.is_integer() { - DocumentPropertyType::I64 - .read_optionally_from(&mut buf, property.required & !property.transient) - } else { - property - .property_type - .read_optionally_from(&mut buf, property.required & !property.transient) - }; + let read_value = property.property_type.read_optionally_from( + &mut buf, + property.required_at(None) & !property.transient, + ); match read_value { Ok(read_value) => { @@ -918,6 +1370,7 @@ impl DocumentPlatformDeserializationMethodsV0 for DocumentV0 { } Ok(DocumentV0 { + contract_version: None, id: Identifier::new(id), properties, owner_id: Identifier::new(owner_id), @@ -936,7 +1389,7 @@ impl DocumentPlatformDeserializationMethodsV0 for DocumentV0 { } /// Reads a serialized document and creates a Document from it. - fn from_bytes_v1( + fn from_bytes_v2( serialized_document: &[u8], document_type: DocumentTypeRef, _platform_version: &PlatformVersion, @@ -966,6 +1419,31 @@ impl DocumentPlatformDeserializationMethodsV0 for DocumentV0 { )) })?; + // $creatorId + let creator_id: Option = if document_type.trade_mode() != TradeMode::None + || document_type.documents_transferable().is_transferable() + { + let has_creator_id = buf.read_u8().map_err(|_| { + DataContractError::CorruptedSerialization( + "error reading has creator id bool from serialized document".to_string(), + ) + })?; + if has_creator_id > 0 { + // $creatorId + let mut known_owner_id = [0; 32]; + buf.read_exact(&mut known_owner_id).map_err(|_| { + DataContractError::DecodingDocumentError(DecodingError::new( + "error reading from serialized document for creator id".to_string(), + )) + })?; + Some(known_owner_id.into()) + } else { + None + } + } else { + None + }; + // $revision // if the document type is mutable then we should deserialize the revision let revision: Option = if document_type.requires_revision() { @@ -1108,7 +1586,7 @@ impl DocumentPlatformDeserializationMethodsV0 for DocumentV0 { .iter() .filter_map(|(key, property)| { if finished_buffer { - return if property.required && !property.transient { + return if property.required_at(None) && !property.transient { Some(Err(DataContractError::CorruptedSerialization( "required field after finished buffer".to_string(), ))) @@ -1116,9 +1594,10 @@ impl DocumentPlatformDeserializationMethodsV0 for DocumentV0 { None }; } - let read_value = property - .property_type - .read_optionally_from(&mut buf, property.required & !property.transient); + let read_value = property.property_type.read_optionally_from( + &mut buf, + property.required_at(None) & !property.transient, + ); match read_value { Ok(read_value) => { @@ -1135,6 +1614,7 @@ impl DocumentPlatformDeserializationMethodsV0 for DocumentV0 { } Ok(DocumentV0 { + contract_version: None, id: Identifier::new(id), properties, owner_id: Identifier::new(owner_id), @@ -1148,25 +1628,42 @@ impl DocumentPlatformDeserializationMethodsV0 for DocumentV0 { created_at_core_block_height, updated_at_core_block_height, transferred_at_core_block_height, - creator_id: None, + creator_id, }) } /// Reads a serialized document and creates a Document from it. - fn from_bytes_v2( + /// Version 3 is version 2 plus the contract version stamp, which selects + /// each `requiredSince` property's byte layout: raw when the stamp has + /// reached the property's `requiredSince`, presence-flagged otherwise. + fn from_bytes_v3( serialized_document: &[u8], document_type: DocumentTypeRef, _platform_version: &PlatformVersion, ) -> Result { let mut buf = BufReader::new(serialized_document); - if serialized_document.len() < 64 { + if serialized_document.len() < 65 { return Err(DataContractError::DecodingDocumentError( DecodingError::new( - "serialized document is too small, must have id and owner id".to_string(), + "serialized document is too small, must have contract version, id and owner id" + .to_string(), ), )); } + // the contract version stamp; 0 means unstamped + let stamp: u64 = buf.read_varint().map_err(|_| { + DataContractError::DecodingDocumentError(DecodingError::new( + "error reading contract version stamp from serialized document".to_string(), + )) + })?; + if stamp > u32::MAX as u64 { + return Err(DataContractError::CorruptedSerialization( + "contract version stamp does not fit in a u32".to_string(), + )); + } + let contract_version = if stamp == 0 { None } else { Some(stamp as u32) }; + // $id let mut id = [0; 32]; buf.read_exact(&mut id).map_err(|_| { @@ -1349,8 +1846,9 @@ impl DocumentPlatformDeserializationMethodsV0 for DocumentV0 { .properties() .iter() .filter_map(|(key, property)| { + let required = property.required_at(contract_version); if finished_buffer { - return if property.required && !property.transient { + return if required && !property.transient { Some(Err(DataContractError::CorruptedSerialization( "required field after finished buffer".to_string(), ))) @@ -1360,7 +1858,7 @@ impl DocumentPlatformDeserializationMethodsV0 for DocumentV0 { } let read_value = property .property_type - .read_optionally_from(&mut buf, property.required & !property.transient); + .read_optionally_from(&mut buf, required & !property.transient); match read_value { Ok(read_value) => { @@ -1377,6 +1875,7 @@ impl DocumentPlatformDeserializationMethodsV0 for DocumentV0 { } Ok(DocumentV0 { + contract_version, id: Identifier::new(id), properties, owner_id: Identifier::new(owner_id), @@ -1428,9 +1927,13 @@ impl DocumentPlatformConversionMethodsV0 for DocumentV0 { // Document types now have properties that are known to be things like u8, i32 etc. 1 => self.serialize_v1(document_type), 2 => self.serialize_v2(document_type), + // Version 3 coincides with protocol version 14: it stamps the + // document with the contract version its bytes conform to, + // enabling `requiredSince` properties. + 3 => self.serialize_v3(document_type), version => Err(ProtocolError::UnknownVersionMismatch { method: "DocumentV0::serialize".to_string(), - known_versions: vec![0, 1, 2], + known_versions: vec![0, 1, 2, 3], received: version, }), } @@ -1458,9 +1961,10 @@ impl DocumentPlatformConversionMethodsV0 for DocumentV0 { 0 => self.serialize_v0(document_type), 1 => self.serialize_v1(document_type), 2 => self.serialize_v2(document_type), + 3 => self.serialize_v3(document_type), version => Err(ProtocolError::UnknownVersionMismatch { method: "DocumentV0::serialize".to_string(), - known_versions: vec![0, 1, 2], + known_versions: vec![0, 1, 2, 3], received: version, }), } @@ -1510,9 +2014,11 @@ impl DocumentPlatformConversionMethodsV0 for DocumentV0 { .map_err(ProtocolError::DataContractError), 2 => DocumentV0::from_bytes_v2(serialized_document, document_type, platform_version) .map_err(ProtocolError::DataContractError), + 3 => DocumentV0::from_bytes_v3(serialized_document, document_type, platform_version) + .map_err(ProtocolError::DataContractError), version => Err(ProtocolError::UnknownVersionMismatch { method: "Document::from_bytes (deserialization)".to_string(), - known_versions: vec![0, 1, 2], + known_versions: vec![0, 1, 2, 3], received: version, }), } @@ -1585,9 +2091,21 @@ impl DocumentPlatformConversionMethodsV0 for DocumentV0 { )), } } + 3 => { + match DocumentV0::from_bytes_v3( + serialized_document, + document_type, + platform_version, + ) { + Ok(document) => Ok(ConsensusValidationResult::new_with_data(document)), + Err(err) => Ok(ConsensusValidationResult::new_with_error( + ConsensusError::BasicError(BasicError::ContractError(err)), + )), + } + } version => Err(ProtocolError::UnknownVersionMismatch { method: "Document::from_bytes (deserialization)".to_string(), - known_versions: vec![0, 1, 2], + known_versions: vec![0, 1, 2, 3], received: version, }), } @@ -2147,6 +2665,7 @@ mod tests { fn doc_with_ids() -> DocumentV0 { DocumentV0 { + contract_version: None, id: Identifier::new([1u8; 32]), owner_id: Identifier::new([2u8; 32]), properties: BTreeMap::new(), @@ -2606,4 +3125,258 @@ mod tests { assert_eq!(deserialized.created_at, Some(1750244879636)); assert_eq!(deserialized.updated_at, Some(1750244879636)); } + + // ================================================================ + // Format 3: the contract-version stamp and requiredSince layouts + // ================================================================ + + /// A document type with: + /// - `a`: required at every version + /// - `b`: required since contract version 2 + /// - `c`: plain optional + fn required_since_document_type() -> crate::data_contract::document_type::DocumentType { + use crate::data_contract::config::DataContractConfig; + use crate::data_contract::document_type::DocumentType; + use platform_value::platform_value; + use std::collections::BTreeMap; + + let platform_version = PlatformVersion::latest(); + let schema = platform_value!({ + "type": "object", + "properties": { + "a": {"type": "string", "position": 0, "maxLength": 60_u32}, + "b": {"type": "string", "position": 1, "maxLength": 60_u32, "requiredSince": 2}, + "c": {"type": "string", "position": 2, "maxLength": 60_u32}, + }, + "required": ["a", "b"], + "additionalProperties": false, + }); + let config = DataContractConfig::default_for_version(platform_version) + .expect("should create a default config"); + DocumentType::try_from_schema( + platform_value::Identifier::new([1; 32]), + 1, + config.version(), + "test", + schema, + None, + &BTreeMap::new(), + &config, + false, + &mut Vec::new(), + platform_version, + ) + .expect("failed to create document type") + } + + fn stamped_document( + contract_version: Option, + properties: BTreeMap, + document_type: DocumentTypeRef, + ) -> DocumentV0 { + DocumentV0 { + contract_version, + id: Identifier::new([3; 32]), + owner_id: Identifier::new([4; 32]), + properties, + revision: document_type.initial_revision(), + ..Default::default() + } + } + + #[test] + fn serialize_v3_round_trips_document_stamped_at_required_since() { + let platform_version = PlatformVersion::latest(); + let document_type = required_since_document_type(); + let document_type_ref = document_type.as_ref(); + + let mut properties = BTreeMap::new(); + properties.insert("a".to_string(), Value::Text("alpha".to_string())); + properties.insert("b".to_string(), Value::Text("beta".to_string())); + + let document = stamped_document(Some(2), properties, document_type_ref); + + let serialized = document + .serialize_v3(document_type_ref) + .expect("stamped document with the required-since field should serialize"); + + let (version, _) = u64::decode_var(&serialized).expect("expected varint"); + assert_eq!(version, 3, "serialization version prefix should be 3"); + + let deserialized = DocumentV0::from_bytes(&serialized, document_type_ref, platform_version) + .expect("expected deserialization to succeed"); + + assert_eq!(deserialized.contract_version, Some(2)); + assert_eq!(deserialized, document); + } + + #[test] + fn serialize_v3_grandfathered_document_may_omit_required_since_field() { + let platform_version = PlatformVersion::latest(); + let document_type = required_since_document_type(); + let document_type_ref = document_type.as_ref(); + + let mut properties = BTreeMap::new(); + properties.insert("a".to_string(), Value::Text("alpha".to_string())); + + // Stamped at version 1, before `b` became required at version 2 + let document = stamped_document(Some(1), properties, document_type_ref); + + let serialized = document + .serialize_v3(document_type_ref) + .expect("grandfathered document without the required-since field should serialize"); + + let deserialized = DocumentV0::from_bytes(&serialized, document_type_ref, platform_version) + .expect("expected deserialization to succeed"); + + assert_eq!(deserialized.contract_version, Some(1)); + assert!(!deserialized.properties.contains_key("b")); + assert_eq!(deserialized, document); + } + + #[test] + fn serialize_v3_unstamped_document_treats_required_since_fields_as_optional() { + let platform_version = PlatformVersion::latest(); + let document_type = required_since_document_type(); + let document_type_ref = document_type.as_ref(); + + let mut properties = BTreeMap::new(); + properties.insert("a".to_string(), Value::Text("alpha".to_string())); + + // No stamp: a pre-format-3 document being re-serialized (e.g. on + // transfer). Every requiredSince annotation postdates its bytes. + let document = stamped_document(None, properties, document_type_ref); + + let serialized = document + .serialize_v3(document_type_ref) + .expect("unstamped document without the required-since field should serialize"); + + let deserialized = DocumentV0::from_bytes(&serialized, document_type_ref, platform_version) + .expect("expected deserialization to succeed"); + + assert_eq!(deserialized.contract_version, None); + assert_eq!(deserialized, document); + } + + #[test] + fn serialize_v3_stamped_at_required_since_missing_field_errors() { + let document_type = required_since_document_type(); + let document_type_ref = document_type.as_ref(); + + let mut properties = BTreeMap::new(); + properties.insert("a".to_string(), Value::Text("alpha".to_string())); + + // Stamped at version 2, where `b` is required — but `b` is absent + let document = stamped_document(Some(2), properties, document_type_ref); + + let result = document.serialize_v3(document_type_ref); + assert!( + matches!( + result, + Err(ProtocolError::DataContractError( + DataContractError::MissingRequiredKey(_) + )) + ), + "a document stamped at requiredSince must contain the field, got {result:?}" + ); + } + + #[test] + fn format_2_bytes_stay_readable_under_a_required_since_schema() { + use crate::data_contract::config::DataContractConfig; + use crate::data_contract::document_type::DocumentType; + use platform_value::platform_value; + + let platform_version = PlatformVersion::latest(); + + // The schema as it was at contract version 1, before `b` (required + // since version 2) and `c` (optional) were appended + let old_schema = platform_value!({ + "type": "object", + "properties": { + "a": {"type": "string", "position": 0, "maxLength": 60_u32}, + }, + "required": ["a"], + "additionalProperties": false, + }); + let config = DataContractConfig::default_for_version(platform_version) + .expect("should create a default config"); + let old_document_type = DocumentType::try_from_schema( + Identifier::new([1; 32]), + 1, + config.version(), + "test", + old_schema, + None, + &BTreeMap::new(), + &config, + false, + &mut Vec::new(), + platform_version, + ) + .expect("failed to create old document type"); + + let mut properties = BTreeMap::new(); + properties.insert("a".to_string(), Value::Text("alpha".to_string())); + + // A pre-stamp document serialized in format 2 under the old schema + // (as every document written before protocol v14 was, at the + // latest): its buffer ends before `b` and `c`, which must read back + // as absent under the updated schema, not as errors + let document = stamped_document(None, properties, old_document_type.as_ref()); + + let serialized = document + .serialize_v2(old_document_type.as_ref()) + .expect("format 2 serialization should succeed"); + + let (version, _) = u64::decode_var(&serialized).expect("expected varint"); + assert_eq!(version, 2); + + let new_document_type = required_since_document_type(); + let deserialized = + DocumentV0::from_bytes(&serialized, new_document_type.as_ref(), platform_version) + .expect("format 2 bytes must stay readable under a requiredSince schema"); + + assert_eq!(deserialized.contract_version, None); + assert!(!deserialized.properties.contains_key("b")); + assert!(!deserialized.properties.contains_key("c")); + assert_eq!(deserialized, document); + } + + #[test] + fn stamp_survives_the_wire_for_documents_stamped_past_required_since() { + let platform_version = PlatformVersion::latest(); + let document_type = required_since_document_type(); + let document_type_ref = document_type.as_ref(); + + // The same content stamped before and at the requiredSince boundary + // must produce different byte layouts (flagged vs raw), and each must + // round-trip through the layout its own stamp selects + let mut properties = BTreeMap::new(); + properties.insert("a".to_string(), Value::Text("alpha".to_string())); + properties.insert("b".to_string(), Value::Text("beta".to_string())); + + let stamped_before = stamped_document(Some(1), properties.clone(), document_type_ref); + let stamped_at = stamped_document(Some(2), properties, document_type_ref); + + let serialized_before = stamped_before + .serialize_v3(document_type_ref) + .expect("expected serialization"); + let serialized_at = stamped_at + .serialize_v3(document_type_ref) + .expect("expected serialization"); + + // The flagged layout carries one extra presence byte for `b`, and the + // two stamps differ in the prefix varint + assert_ne!(serialized_before, serialized_at); + + let before_back = + DocumentV0::from_bytes(&serialized_before, document_type_ref, platform_version) + .expect("expected deserialization"); + let at_back = DocumentV0::from_bytes(&serialized_at, document_type_ref, platform_version) + .expect("expected deserialization"); + + assert_eq!(before_back, stamped_before); + assert_eq!(at_back, stamped_at); + } } diff --git a/packages/rs-dpp/src/errors/consensus/basic/basic_error.rs b/packages/rs-dpp/src/errors/consensus/basic/basic_error.rs index 6791910923e..d1bed746659 100644 --- a/packages/rs-dpp/src/errors/consensus/basic/basic_error.rs +++ b/packages/rs-dpp/src/errors/consensus/basic/basic_error.rs @@ -7,15 +7,15 @@ use crate::consensus::basic::data_contract::data_contract_max_depth_exceed_error use crate::consensus::basic::data_contract::{ ContestedUniqueIndexOnMutableDocumentTypeError, ContestedUniqueIndexWithUniqueIndexError, DataContractHaveNewUniqueIndexError, DataContractImmutablePropertiesUpdateError, - DataContractInvalidIndexDefinitionUpdateError, DataContractTokenConfigurationUpdateError, - DataContractUniqueIndicesChangedError, DecimalsOverLimitError, DuplicateIndexError, - DuplicateIndexNameError, GroupExceedsMaxMembersError, GroupHasTooFewMembersError, - GroupMemberHasPowerOfZeroError, GroupMemberHasPowerOverLimitError, - GroupNonUnilateralMemberPowerHasLessThanRequiredPowerError, GroupPositionDoesNotExistError, - GroupRequiredPowerIsInvalidError, GroupTotalPowerLessThanRequiredError, - IncompatibleDataContractSchemaError, IncompatibleDocumentTypeSchemaError, - IncompatibleRe2PatternError, InvalidCompoundIndexError, InvalidDataContractIdError, - InvalidDataContractVersionError, InvalidDocumentTypeNameError, + DataContractInvalidIndexDefinitionUpdateError, DataContractInvalidRequiredFieldsUpdateError, + DataContractTokenConfigurationUpdateError, DataContractUniqueIndicesChangedError, + DecimalsOverLimitError, DuplicateIndexError, DuplicateIndexNameError, + GroupExceedsMaxMembersError, GroupHasTooFewMembersError, GroupMemberHasPowerOfZeroError, + GroupMemberHasPowerOverLimitError, GroupNonUnilateralMemberPowerHasLessThanRequiredPowerError, + GroupPositionDoesNotExistError, GroupRequiredPowerIsInvalidError, + GroupTotalPowerLessThanRequiredError, IncompatibleDataContractSchemaError, + IncompatibleDocumentTypeSchemaError, IncompatibleRe2PatternError, InvalidCompoundIndexError, + InvalidDataContractIdError, InvalidDataContractVersionError, InvalidDocumentTypeNameError, InvalidDocumentTypeRequiredSecurityLevelError, InvalidIndexPropertyTypeError, InvalidIndexedPropertyConstraintError, InvalidKeywordCharacterError, InvalidTokenBaseSupplyError, InvalidTokenDistributionFunctionDivideByZeroError, @@ -696,6 +696,9 @@ pub enum BasicError { #[error(transparent)] TokenPricingScheduleEmptyError(TokenPricingScheduleEmptyError), + + #[error(transparent)] + DataContractInvalidRequiredFieldsUpdateError(DataContractInvalidRequiredFieldsUpdateError), } impl From for ConsensusError { diff --git a/packages/rs-dpp/src/errors/consensus/basic/data_contract/data_contract_invalid_required_fields_update_error.rs b/packages/rs-dpp/src/errors/consensus/basic/data_contract/data_contract_invalid_required_fields_update_error.rs new file mode 100644 index 00000000000..db524161d38 --- /dev/null +++ b/packages/rs-dpp/src/errors/consensus/basic/data_contract/data_contract_invalid_required_fields_update_error.rs @@ -0,0 +1,46 @@ +use crate::consensus::basic::BasicError; +use crate::consensus::ConsensusError; +use crate::errors::ProtocolError; +use bincode::{Decode, Encode}; +use platform_serialization_derive::{PlatformDeserialize, PlatformSerialize}; +use thiserror::Error; + +#[derive( + Error, Debug, Clone, PartialEq, Eq, Encode, Decode, PlatformSerialize, PlatformDeserialize, +)] +#[error("Document type {document_type} required fields update is not allowed: {details}")] +#[platform_serialize(unversioned)] +pub struct DataContractInvalidRequiredFieldsUpdateError { + /* + + DO NOT CHANGE ORDER OF FIELDS WITHOUT INTRODUCING OF NEW VERSION + + */ + document_type: String, + details: String, +} + +impl DataContractInvalidRequiredFieldsUpdateError { + pub fn new(document_type: String, details: String) -> Self { + Self { + document_type, + details, + } + } + + pub fn document_type(&self) -> &str { + &self.document_type + } + + pub fn details(&self) -> &str { + &self.details + } +} + +impl From for ConsensusError { + fn from(err: DataContractInvalidRequiredFieldsUpdateError) -> Self { + Self::BasicError(BasicError::DataContractInvalidRequiredFieldsUpdateError( + err, + )) + } +} diff --git a/packages/rs-dpp/src/errors/consensus/basic/data_contract/mod.rs b/packages/rs-dpp/src/errors/consensus/basic/data_contract/mod.rs index a43ea4793a3..d15ccaca661 100644 --- a/packages/rs-dpp/src/errors/consensus/basic/data_contract/mod.rs +++ b/packages/rs-dpp/src/errors/consensus/basic/data_contract/mod.rs @@ -3,6 +3,7 @@ mod contested_unique_index_with_unique_index_error; mod data_contract_have_new_unique_index_error; mod data_contract_immutable_properties_update_error; mod data_contract_invalid_index_definition_update_error; +mod data_contract_invalid_required_fields_update_error; pub mod data_contract_max_depth_exceed_error; mod data_contract_token_configuration_update_error; mod data_contract_unique_indices_changed_error; @@ -62,6 +63,7 @@ mod unknown_transferable_type_error; pub use data_contract_have_new_unique_index_error::*; pub use data_contract_immutable_properties_update_error::*; pub use data_contract_invalid_index_definition_update_error::*; +pub use data_contract_invalid_required_fields_update_error::*; pub use data_contract_token_configuration_update_error::*; pub use data_contract_unique_indices_changed_error::*; pub use document_types_are_missing_error::*; diff --git a/packages/rs-dpp/src/errors/consensus/codes.rs b/packages/rs-dpp/src/errors/consensus/codes.rs index 6fa1b64d6ca..cc42a8e4d53 100644 --- a/packages/rs-dpp/src/errors/consensus/codes.rs +++ b/packages/rs-dpp/src/errors/consensus/codes.rs @@ -120,6 +120,7 @@ impl ErrorWithCode for BasicError { Self::InvalidTokenDistributionTimeIntervalTooShortError(_) => 10273, Self::InvalidTokenDistributionTimeIntervalNotMinuteAlignedError(_) => 10274, Self::RedundantDocumentPaidForByTokenWithContractId(_) => 10275, + Self::DataContractInvalidRequiredFieldsUpdateError { .. } => 10276, // Group Errors: 10350-10399 Self::GroupPositionDoesNotExistError(_) => 10350, diff --git a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_create_transition/v0/mod.rs b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_create_transition/v0/mod.rs index 8b7a528b7d0..876fc29bc10 100644 --- a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_create_transition/v0/mod.rs +++ b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_create_transition/v0/mod.rs @@ -374,6 +374,7 @@ impl DocumentFromCreateTransitionV0 for Document { .document_structure_version { 0 => Ok(DocumentV0 { + contract_version: None, id: base.id(), owner_id, properties: data, @@ -484,6 +485,7 @@ impl DocumentFromCreateTransitionV0 for Document { .document_structure_version { 0 => Ok(DocumentV0 { + contract_version: None, id: base.id(), owner_id, properties, diff --git a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_replace_transition/v0/mod.rs b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_replace_transition/v0/mod.rs index 0c2d1f95054..b501324e98d 100644 --- a/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_replace_transition/v0/mod.rs +++ b/packages/rs-dpp/src/state_transition/state_transitions/document/batch_transition/batched_transition/document_replace_transition/v0/mod.rs @@ -234,6 +234,7 @@ impl DocumentFromReplaceTransitionV0 for Document { .document_structure_version { 0 => Ok(DocumentV0 { + contract_version: None, id, owner_id, properties: data.clone(), @@ -315,6 +316,7 @@ impl DocumentFromReplaceTransitionV0 for Document { .document_structure_version { 0 => Ok(DocumentV0 { + contract_version: None, id, owner_id, properties: data, diff --git a/packages/rs-dpp/src/tests/json_document.rs b/packages/rs-dpp/src/tests/json_document.rs index 1f76d7431fd..feab00b5adc 100644 --- a/packages/rs-dpp/src/tests/json_document.rs +++ b/packages/rs-dpp/src/tests/json_document.rs @@ -154,6 +154,7 @@ pub fn json_document_to_document( } let mut document: DocumentV0 = DocumentV0 { + contract_version: None, id: data.remove_identifier("$id")?, owner_id: data.remove_identifier("$ownerId")?, properties: Default::default(), diff --git a/packages/rs-dpp/src/tokens/token_event.rs b/packages/rs-dpp/src/tokens/token_event.rs index 5982eb45520..5066b0f0479 100644 --- a/packages/rs-dpp/src/tokens/token_event.rs +++ b/packages/rs-dpp/src/tokens/token_event.rs @@ -860,6 +860,7 @@ impl TokenEvent { }; let document: Document = DocumentV0 { + contract_version: None, id: document_id, owner_id, properties, diff --git a/packages/rs-drive-abci/src/execution/platform_events/initialization/create_genesis_state/common.rs b/packages/rs-drive-abci/src/execution/platform_events/initialization/create_genesis_state/common.rs index a96ca521343..aa28c7fa3bf 100644 --- a/packages/rs-drive-abci/src/execution/platform_events/initialization/create_genesis_state/common.rs +++ b/packages/rs-drive-abci/src/execution/platform_events/initialization/create_genesis_state/common.rs @@ -66,6 +66,7 @@ impl Platform { .map_err(|e| Error::Protocol(ProtocolError::ValueError(e)))?; let document = DocumentV0 { + contract_version: None, id: DPNS_DASH_TLD_DOCUMENT_ID.into(), properties: document_stub_properties, owner_id: contract.owner_id(), diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/basic_structure/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/basic_structure/mod.rs index 008be12cc67..4598692d1d6 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/basic_structure/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/basic_structure/mod.rs @@ -1,2 +1,3 @@ pub(crate) mod v0; pub(crate) mod v1; +pub(crate) mod v2; diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/basic_structure/v2/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/basic_structure/v2/mod.rs new file mode 100644 index 00000000000..e59a2e06e07 --- /dev/null +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/basic_structure/v2/mod.rs @@ -0,0 +1,82 @@ +use crate::error::Error; +use dpp::consensus::basic::data_contract::DataContractInvalidRequiredFieldsUpdateError; +use dpp::dashcore::Network; +use dpp::state_transition::data_contract_create_transition::accessors::DataContractCreateTransitionAccessorsV0; +use dpp::state_transition::data_contract_create_transition::DataContractCreateTransition; +use dpp::validation::SimpleConsensusValidationResult; +use dpp::version::PlatformVersion; + +use super::v1::DataContractCreateStateTransitionBasicStructureValidationV1; + +const PROPERTIES: &str = "properties"; +const REQUIRED_SINCE: &str = "requiredSince"; + +pub(in crate::execution::validation::state_transition::state_transitions::data_contract_create) trait DataContractCreateStateTransitionBasicStructureValidationV2 +{ + fn validate_basic_structure_v2( + &self, + network_type: Network, + platform_version: &PlatformVersion, + ) -> Result; +} + +impl DataContractCreateStateTransitionBasicStructureValidationV2 for DataContractCreateTransition { + fn validate_basic_structure_v2( + &self, + network_type: Network, + platform_version: &PlatformVersion, + ) -> Result { + // First run all v1 (and transitively v0) validations + let v1_result = self.validate_basic_structure_v1(network_type, platform_version)?; + if !v1_result.is_valid() { + return Ok(v1_result); + } + + // `requiredSince` names the contract version a property is required + // from. A freshly created contract is version 1, so the only value + // that names an existing version is 1 (which is equivalent to plain + // membership in `required`). Later values would pre-schedule + // requiredness at a future version — coherent for the wire format, + // but banned: requiredness changes must arrive with the update that + // creates the version they name. + for (document_type_name, schema) in self.data_contract().document_schemas() { + let Some(properties) = schema + .get_optional_value(PROPERTIES) + .ok() + .flatten() + .and_then(|properties| properties.as_map()) + else { + continue; + }; + + for (property_name, property_schema) in properties { + let Some(required_since) = property_schema + .as_map() + .and_then(|map| { + map.iter() + .find(|(key, _)| key.as_text() == Some(REQUIRED_SINCE)) + }) + .and_then(|(_, value)| value.as_integer::()) + else { + continue; + }; + + if required_since != 1 { + return Ok(SimpleConsensusValidationResult::new_with_error( + DataContractInvalidRequiredFieldsUpdateError::new( + document_type_name.clone(), + format!( + "property '{}' of a newly created contract cannot carry requiredSince {} — a fresh contract is version 1", + property_name.as_text().unwrap_or_default(), + required_since + ), + ) + .into(), + )); + } + } + } + + Ok(SimpleConsensusValidationResult::new()) + } +} diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/mod.rs index 8983c91b396..df6c2b6e695 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/mod.rs @@ -6,6 +6,7 @@ mod state; use advanced_structure::v1::DataContractCreatedStateTransitionAdvancedStructureValidationV1; use basic_structure::v0::DataContractCreateStateTransitionBasicStructureValidationV0; use basic_structure::v1::DataContractCreateStateTransitionBasicStructureValidationV1; +use basic_structure::v2::DataContractCreateStateTransitionBasicStructureValidationV2; use dpp::address_funds::PlatformAddress; use dpp::block::block_info::BlockInfo; use dpp::dashcore::Network; @@ -99,14 +100,15 @@ impl StateTransitionBasicStructureValidationV0 for DataContractCreateTransition { Some(0) => self.validate_basic_structure_v0(network_type, platform_version), Some(1) => self.validate_basic_structure_v1(network_type, platform_version), + Some(2) => self.validate_basic_structure_v2(network_type, platform_version), Some(version) => Err(Error::Execution(ExecutionError::UnknownVersionMismatch { method: "data contract create transition: validate_basic_structure".to_string(), - known_versions: vec![0, 1], + known_versions: vec![0, 1, 2], received: version, })), None => Err(Error::Execution(ExecutionError::VersionNotActive { method: "data contract create transition: validate_basic_structure".to_string(), - known_versions: vec![0, 1], + known_versions: vec![0, 1, 2], })), } } diff --git a/packages/rs-drive-abci/src/query/document_query/v0/mod.rs b/packages/rs-drive-abci/src/query/document_query/v0/mod.rs index 2bd36c1fad1..ff5a9ab1686 100644 --- a/packages/rs-drive-abci/src/query/document_query/v0/mod.rs +++ b/packages/rs-drive-abci/src/query/document_query/v0/mod.rs @@ -886,6 +886,7 @@ mod tests { let created_at = base_time + i * 20000; // Create a Document with the desired properties let random_document: Document = DocumentV0 { + contract_version: None, id: Identifier::random_with_rng(&mut std_rng), owner_id: Identifier::random_with_rng(&mut std_rng), properties: { @@ -1052,6 +1053,7 @@ mod tests { let created_at = base_time + i * 20000; // Create a Document with the desired properties let random_document: Document = DocumentV0 { + contract_version: None, id: Identifier::random_with_rng(&mut std_rng), owner_id: Identifier::random_with_rng(&mut std_rng), properties: { @@ -1218,6 +1220,7 @@ mod tests { let created_at = base_time + i * 20000; // Create a Document with the desired properties let random_document: Document = DocumentV0 { + contract_version: None, id: Identifier::random_with_rng(&mut std_rng), owner_id: Identifier::random_with_rng(&mut std_rng), properties: { @@ -1377,6 +1380,7 @@ mod tests { let created_at = base_time + i * 20000; // Create a Document with the desired properties let random_document: Document = DocumentV0 { + contract_version: None, id: Identifier::random_with_rng(&mut std_rng), owner_id: Identifier::random_with_rng(&mut std_rng), properties: { @@ -1551,6 +1555,7 @@ mod tests { let created_at = base_time + i * 20000; // Create a Document with the desired properties let random_document: Document = DocumentV0 { + contract_version: None, id: Identifier::random_with_rng(&mut std_rng), owner_id: Identifier::random_with_rng(&mut std_rng), properties: { diff --git a/packages/rs-drive-abci/src/query/document_query/v1/tests.rs b/packages/rs-drive-abci/src/query/document_query/v1/tests.rs index 860c394c18f..2241e2ba5f7 100644 --- a/packages/rs-drive-abci/src/query/document_query/v1/tests.rs +++ b/packages/rs-drive-abci/src/query/document_query/v1/tests.rs @@ -1128,6 +1128,7 @@ mod ported_v0_count_tests { properties.insert("age".to_string(), Value::U64(age)); let document: Document = DocumentV0 { + contract_version: None, id: Identifier::from(id), owner_id: Identifier::from([0u8; 32]), properties, diff --git a/packages/rs-drive-abci/src/test/helpers/fee_pools.rs b/packages/rs-drive-abci/src/test/helpers/fee_pools.rs index 5dbe0cb0957..2a6dcdcf5d5 100644 --- a/packages/rs-drive-abci/src/test/helpers/fee_pools.rs +++ b/packages/rs-drive-abci/src/test/helpers/fee_pools.rs @@ -78,6 +78,7 @@ fn create_test_mn_share_document( properties.insert(String::from("percentage"), percentage.into()); let document = DocumentV0 { + contract_version: None, id, properties, owner_id: identity_id, diff --git a/packages/rs-drive/benches/document_average_worst_case.rs b/packages/rs-drive/benches/document_average_worst_case.rs index 3ec864dc2b5..3ad031fab16 100644 --- a/packages/rs-drive/benches/document_average_worst_case.rs +++ b/packages/rs-drive/benches/document_average_worst_case.rs @@ -547,6 +547,7 @@ fn insert_grade_document( properties.insert("instructor".to_string(), Value::Bytes(instructor.to_vec())); let document: Document = DocumentV0 { + contract_version: None, id: Identifier::from(document_id(row)), owner_id: Identifier::from([7u8; 32]), properties, diff --git a/packages/rs-drive/benches/document_count_worst_case.rs b/packages/rs-drive/benches/document_count_worst_case.rs index 2a00dd884c4..40e8953bc81 100644 --- a/packages/rs-drive/benches/document_count_worst_case.rs +++ b/packages/rs-drive/benches/document_count_worst_case.rs @@ -260,6 +260,7 @@ fn insert_widget_document( properties.insert("serial".to_string(), Value::U64(row)); let document: Document = DocumentV0 { + contract_version: None, id: Identifier::from(document_id(row)), owner_id: Identifier::from([7u8; 32]), properties, diff --git a/packages/rs-drive/benches/document_sum_worst_case.rs b/packages/rs-drive/benches/document_sum_worst_case.rs index e1953d40b2e..6d42c21dbf8 100644 --- a/packages/rs-drive/benches/document_sum_worst_case.rs +++ b/packages/rs-drive/benches/document_sum_worst_case.rs @@ -312,6 +312,7 @@ fn insert_tip_document( properties.insert("sentAt".to_string(), Value::U64(sent_at)); let document: Document = DocumentV0 { + contract_version: None, id: Identifier::from(document_id(row)), owner_id: Identifier::from([7u8; 32]), properties, diff --git a/packages/rs-drive/src/drive/contract/insert/add_description/v0/mod.rs b/packages/rs-drive/src/drive/contract/insert/add_description/v0/mod.rs index bbd6671fc90..c8a21e091d6 100644 --- a/packages/rs-drive/src/drive/contract/insert/add_description/v0/mod.rs +++ b/packages/rs-drive/src/drive/contract/insert/add_description/v0/mod.rs @@ -202,6 +202,7 @@ impl Drive { ]); let document: Document = DocumentV0 { + contract_version: None, id: document_id, owner_id, properties, diff --git a/packages/rs-drive/src/drive/contract/insert/add_new_keywords/v0/mod.rs b/packages/rs-drive/src/drive/contract/insert/add_new_keywords/v0/mod.rs index 89536fa16cb..950785dd05d 100644 --- a/packages/rs-drive/src/drive/contract/insert/add_new_keywords/v0/mod.rs +++ b/packages/rs-drive/src/drive/contract/insert/add_new_keywords/v0/mod.rs @@ -163,6 +163,7 @@ impl Drive { ]); let document: Document = DocumentV0 { + contract_version: None, id: document_id, owner_id, properties, diff --git a/packages/rs-drive/src/drive/document/update/mod.rs b/packages/rs-drive/src/drive/document/update/mod.rs index 9b223c9bdf2..226abd4ad94 100644 --- a/packages/rs-drive/src/drive/document/update/mod.rs +++ b/packages/rs-drive/src/drive/document/update/mod.rs @@ -2747,6 +2747,7 @@ mod tests { properties_initial.insert("color".to_string(), Value::Text("red".to_string())); properties_initial.insert("amount".to_string(), Value::U64(5)); let document_initial: dpp::document::Document = DocumentV0 { + contract_version: None, id: doc_id, owner_id: Identifier::from([0u8; 32]), properties: properties_initial, @@ -2795,6 +2796,7 @@ mod tests { properties_updated.insert("color".to_string(), Value::Text("red".to_string())); properties_updated.insert("amount".to_string(), Value::U64(42)); let document_updated: dpp::document::Document = DocumentV0 { + contract_version: None, id: doc_id, owner_id: Identifier::from([0u8; 32]), properties: properties_updated, @@ -2998,6 +3000,7 @@ mod tests { properties_initial.insert("color".to_string(), Value::Text("red".to_string())); properties_initial.insert("amount".to_string(), Value::U64(11)); let document_initial: dpp::document::Document = DocumentV0 { + contract_version: None, id: doc_id, owner_id: Identifier::from([0u8; 32]), properties: properties_initial, @@ -3043,6 +3046,7 @@ mod tests { properties_updated.insert("color".to_string(), Value::Text("blue".to_string())); properties_updated.insert("amount".to_string(), Value::U64(17)); let document_updated: dpp::document::Document = DocumentV0 { + contract_version: None, id: doc_id, owner_id: Identifier::from([0u8; 32]), properties: properties_updated, diff --git a/packages/rs-drive/src/query/conditions.rs b/packages/rs-drive/src/query/conditions.rs index 7fb394f3ef0..7d4dfd71591 100644 --- a/packages/rs-drive/src/query/conditions.rs +++ b/packages/rs-drive/src/query/conditions.rs @@ -1538,6 +1538,7 @@ mod tests { fn cursor_document(field: &str, value: Value) -> dpp::document::Document { DocumentV0 { + contract_version: None, id: Identifier::from([3u8; 32]), owner_id: Identifier::from([4u8; 32]), properties: BTreeMap::from([(field.to_string(), value)]), diff --git a/packages/rs-drive/src/query/drive_document_average_query/drive_dispatcher.rs b/packages/rs-drive/src/query/drive_document_average_query/drive_dispatcher.rs index 526a5c86b57..26bcd39f12e 100644 --- a/packages/rs-drive/src/query/drive_document_average_query/drive_dispatcher.rs +++ b/packages/rs-drive/src/query/drive_document_average_query/drive_dispatcher.rs @@ -495,6 +495,7 @@ mod tests { properties.insert("color".to_string(), Value::Text(color.to_string())); properties.insert("amount".to_string(), Value::U64(amount)); let document: Document = DocumentV0 { + contract_version: None, id: Identifier::from([(i + 1) as u8; 32]), owner_id: Identifier::from([0u8; 32]), properties, @@ -2096,6 +2097,7 @@ mod tests { let mut properties = std::collections::BTreeMap::new(); properties.insert("amount".to_string(), Value::U64(*amount)); let document: Document = DocumentV0 { + contract_version: None, id: Identifier::from([(i + 1) as u8; 32]), owner_id: Identifier::from([0u8; 32]), properties, diff --git a/packages/rs-drive/src/query/drive_document_count_query/tests.rs b/packages/rs-drive/src/query/drive_document_count_query/tests.rs index da54e064fc9..116147110a5 100644 --- a/packages/rs-drive/src/query/drive_document_count_query/tests.rs +++ b/packages/rs-drive/src/query/drive_document_count_query/tests.rs @@ -69,6 +69,7 @@ fn insert_person_doc( properties.insert("age".to_string(), Value::U64(age)); let document: Document = DocumentV0 { + contract_version: None, id: Identifier::from(id), owner_id: Identifier::from([0u8; 32]), properties, @@ -1262,6 +1263,7 @@ fn test_compound_range_in_summed_no_proof_uses_per_in_aggregate_fanout() { properties.insert("brand".to_string(), Value::Text(brand.to_string())); properties.insert("color".to_string(), Value::Text(color.to_string())); let document: Document = DocumentV0 { + contract_version: None, id: Identifier::from([(i + 1) as u8; 32]), owner_id: Identifier::from([0u8; 32]), properties, @@ -1536,6 +1538,7 @@ fn test_range_distinct_proof_uses_compile_time_default_query_limit_not_operator_ let mut properties = StdBTreeMap::new(); properties.insert("color".to_string(), Value::Text(color.to_string())); let document: Document = DocumentV0 { + contract_version: None, id: Identifier::from([(i + 1) as u8; 32]), owner_id: Identifier::from([0u8; 32]), properties, @@ -2767,6 +2770,7 @@ mod range_countable_point_lookup_tests { properties.insert("color".to_string(), Value::Text(c.to_string())); } let document: Document = DocumentV0 { + contract_version: None, id: Identifier::from(id), owner_id: Identifier::from([0u8; 32]), properties, @@ -2815,6 +2819,7 @@ mod range_countable_point_lookup_tests { let mut properties = StdBTreeMap::new(); properties.insert("category".to_string(), Value::Text(category.to_string())); let document: Document = DocumentV0 { + contract_version: None, id: Identifier::from(id), owner_id: Identifier::from([0u8; 32]), properties, diff --git a/packages/rs-drive/src/query/drive_document_sum_query/tests.rs b/packages/rs-drive/src/query/drive_document_sum_query/tests.rs index 1688669039b..aa1d0566be1 100644 --- a/packages/rs-drive/src/query/drive_document_sum_query/tests.rs +++ b/packages/rs-drive/src/query/drive_document_sum_query/tests.rs @@ -334,6 +334,7 @@ mod limit_policy_regression { properties.insert("color".to_string(), Value::Text(color.to_string())); properties.insert("amount".to_string(), Value::U64(amount)); let document: Document = DocumentV0 { + contract_version: None, id: Identifier::from([(i + 1) as u8; 32]), owner_id: Identifier::from([0u8; 32]), properties, diff --git a/packages/rs-drive/src/query/mod.rs b/packages/rs-drive/src/query/mod.rs index 53fad48e77c..c7fb41382d7 100644 --- a/packages/rs-drive/src/query/mod.rs +++ b/packages/rs-drive/src/query/mod.rs @@ -2808,6 +2808,7 @@ mod tests { // We intentionally omit 'transactionIndex' to simulate missing field let starts_at_document = DocumentV0 { + contract_version: None, id: Identifier::from([3u8; 32]), // The same as start_at owner_id: Identifier::random(), properties, diff --git a/packages/rs-drive/src/state_transition_action/action_convert_to_operations/address_funds/address_credit_withdrawal_transition.rs b/packages/rs-drive/src/state_transition_action/action_convert_to_operations/address_funds/address_credit_withdrawal_transition.rs index b93406ec832..0b76c034f34 100644 --- a/packages/rs-drive/src/state_transition_action/action_convert_to_operations/address_funds/address_credit_withdrawal_transition.rs +++ b/packages/rs-drive/src/state_transition_action/action_convert_to_operations/address_funds/address_credit_withdrawal_transition.rs @@ -96,6 +96,7 @@ mod tests { fn make_document() -> Document { Document::V0(DocumentV0 { + contract_version: None, id: Identifier::from([0x11; 32]), owner_id: Identifier::from([0x22; 32]), properties: Default::default(), diff --git a/packages/rs-drive/src/state_transition_action/action_convert_to_operations/identity/identity_credit_withdrawal_transition.rs b/packages/rs-drive/src/state_transition_action/action_convert_to_operations/identity/identity_credit_withdrawal_transition.rs index 520e54e765e..203b9d0ad10 100644 --- a/packages/rs-drive/src/state_transition_action/action_convert_to_operations/identity/identity_credit_withdrawal_transition.rs +++ b/packages/rs-drive/src/state_transition_action/action_convert_to_operations/identity/identity_credit_withdrawal_transition.rs @@ -77,6 +77,7 @@ mod tests { fn make_document() -> Document { Document::V0(DocumentV0 { + contract_version: None, id: Identifier::from([0xDD; 32]), owner_id: Identifier::from([0xAA; 32]), properties: Default::default(), diff --git a/packages/rs-drive/src/state_transition_action/action_convert_to_operations/shielded/shielded_withdrawal_transition.rs b/packages/rs-drive/src/state_transition_action/action_convert_to_operations/shielded/shielded_withdrawal_transition.rs index 6df19016c54..458fb0cd2ec 100644 --- a/packages/rs-drive/src/state_transition_action/action_convert_to_operations/shielded/shielded_withdrawal_transition.rs +++ b/packages/rs-drive/src/state_transition_action/action_convert_to_operations/shielded/shielded_withdrawal_transition.rs @@ -119,6 +119,7 @@ mod tests { fn make_document() -> Document { Document::V0(DocumentV0 { + contract_version: None, id: Identifier::from([0x11; 32]), owner_id: Identifier::from([0x22; 32]), properties: Default::default(), diff --git a/packages/rs-drive/src/state_transition_action/address_funds/address_credit_withdrawal/mod.rs b/packages/rs-drive/src/state_transition_action/address_funds/address_credit_withdrawal/mod.rs index 28f9438d2ee..85d1fee5e0d 100644 --- a/packages/rs-drive/src/state_transition_action/address_funds/address_credit_withdrawal/mod.rs +++ b/packages/rs-drive/src/state_transition_action/address_funds/address_credit_withdrawal/mod.rs @@ -86,6 +86,7 @@ mod tests { fn make_document() -> Document { Document::V0(DocumentV0 { + contract_version: None, id: Identifier::from([0x11; 32]), owner_id: Identifier::from([0x22; 32]), properties: Default::default(), diff --git a/packages/rs-drive/src/state_transition_action/address_funds/address_credit_withdrawal/v0/transformer.rs b/packages/rs-drive/src/state_transition_action/address_funds/address_credit_withdrawal/v0/transformer.rs index 862b2d052c9..0c44dcd19c5 100644 --- a/packages/rs-drive/src/state_transition_action/address_funds/address_credit_withdrawal/v0/transformer.rs +++ b/packages/rs-drive/src/state_transition_action/address_funds/address_credit_withdrawal/v0/transformer.rs @@ -93,6 +93,7 @@ impl AddressCreditWithdrawalTransitionActionV0 { }); let withdrawal_document = DocumentV0 { + contract_version: None, id: document_id, owner_id, properties: document_data.into_btree_string_map().unwrap(), diff --git a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_create_transition_action/v0/mod.rs b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_create_transition_action/v0/mod.rs index 8cb8c278692..0170963c753 100644 --- a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_create_transition_action/v0/mod.rs +++ b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_create_transition_action/v0/mod.rs @@ -164,6 +164,23 @@ impl DocumentFromCreateTransitionActionV0 for Document { None }; + // The contract-version stamp exists exactly when document + // serialization format 3 (the format that writes it) is in + // effect; earlier formats have no stamp on the wire, so + // building one into the struct would only desync in-memory + // documents from their deserialized counterparts on replay + let contract_version = if platform_version + .dpp + .document_versions + .document_serialization_version + .default_current_version + >= 3 + { + Some(data_contract.contract.version()) + } else { + None + }; + let is_created_at_required = required_fields.contains(CREATED_AT); let is_updated_at_required = required_fields.contains(UPDATED_AT); let is_transferred_at_required = required_fields.contains(TRANSFERRED_AT); @@ -188,6 +205,7 @@ impl DocumentFromCreateTransitionActionV0 for Document { .document_structure_version { 0 => Ok(DocumentV0 { + contract_version, id, owner_id, properties: data, @@ -296,6 +314,21 @@ impl DocumentFromCreateTransitionActionV0 for Document { None }; + // The contract-version stamp exists exactly when document + // serialization format 3 (the format that writes it) is in + // effect + let contract_version = if platform_version + .dpp + .document_versions + .document_serialization_version + .default_current_version + >= 3 + { + Some(data_contract.contract.version()) + } else { + None + }; + let is_created_at_required = required_fields.contains(CREATED_AT); let is_updated_at_required = required_fields.contains(UPDATED_AT); let is_transferred_at_required = required_fields.contains(TRANSFERRED_AT); @@ -320,6 +353,7 @@ impl DocumentFromCreateTransitionActionV0 for Document { .document_structure_version { 0 => Ok(DocumentV0 { + contract_version, id: *id, owner_id, properties: data, diff --git a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_replace_transition_action/v0/mod.rs b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_replace_transition_action/v0/mod.rs index 2a458a70f2e..336c3d091d1 100644 --- a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_replace_transition_action/v0/mod.rs +++ b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_replace_transition_action/v0/mod.rs @@ -1,5 +1,6 @@ pub mod transformer; +use dpp::data_contract::accessors::v0::DataContractV0Getters; use dpp::document::{Document, DocumentV0}; use dpp::identity::TimestampMillis; use dpp::platform_value::{Identifier, Value}; @@ -151,12 +152,28 @@ impl DocumentFromReplaceTransitionActionV0 for Document { let id = base.id(); + // A replace re-supplies the full document contents, so the document + // is re-stamped with the current contract version (the stamp exists + // exactly when document serialization format 3 is in effect) + let contract_version = if platform_version + .dpp + .document_versions + .document_serialization_version + .default_current_version + >= 3 + { + Some(base.data_contract_fetch_info_ref().contract.version()) + } else { + None + }; + match platform_version .dpp .document_versions .document_structure_version { 0 => Ok(DocumentV0 { + contract_version, id, owner_id, properties: data.clone(), @@ -205,12 +222,28 @@ impl DocumentFromReplaceTransitionActionV0 for Document { let id = base.id(); + // A replace re-supplies the full document contents, so the document + // is re-stamped with the current contract version (the stamp exists + // exactly when document serialization format 3 is in effect) + let contract_version = if platform_version + .dpp + .document_versions + .document_serialization_version + .default_current_version + >= 3 + { + Some(base.data_contract_fetch_info_ref().contract.version()) + } else { + None + }; + match platform_version .dpp .document_versions .document_structure_version { 0 => Ok(DocumentV0 { + contract_version, id, owner_id, properties: data, diff --git a/packages/rs-drive/src/state_transition_action/identity/identity_credit_withdrawal/mod.rs b/packages/rs-drive/src/state_transition_action/identity/identity_credit_withdrawal/mod.rs index d83af986f2d..9e508a14a06 100644 --- a/packages/rs-drive/src/state_transition_action/identity/identity_credit_withdrawal/mod.rs +++ b/packages/rs-drive/src/state_transition_action/identity/identity_credit_withdrawal/mod.rs @@ -75,6 +75,7 @@ mod tests { fn make_document() -> Document { Document::V0(DocumentV0 { + contract_version: None, id: Identifier::from([0xDD; 32]), owner_id: Identifier::from([0xAA; 32]), properties: Default::default(), diff --git a/packages/rs-drive/src/state_transition_action/identity/identity_credit_withdrawal/v0/transformer.rs b/packages/rs-drive/src/state_transition_action/identity/identity_credit_withdrawal/v0/transformer.rs index e7f8d622d16..3f74eb99ba8 100644 --- a/packages/rs-drive/src/state_transition_action/identity/identity_credit_withdrawal/v0/transformer.rs +++ b/packages/rs-drive/src/state_transition_action/identity/identity_credit_withdrawal/v0/transformer.rs @@ -52,6 +52,7 @@ impl IdentityCreditWithdrawalTransitionActionV0 { }); let withdrawal_document = DocumentV0 { + contract_version: None, id: document_id, owner_id: identity_credit_withdrawal.identity_id, properties: document_data.into_btree_string_map().unwrap(), @@ -176,6 +177,7 @@ impl IdentityCreditWithdrawalTransitionActionV0 { }); let withdrawal_document = DocumentV0 { + contract_version: None, id: document_id, owner_id: identity_credit_withdrawal.identity_id, properties: document_data.into_btree_string_map()?, diff --git a/packages/rs-drive/src/state_transition_action/shielded/shielded_withdrawal/mod.rs b/packages/rs-drive/src/state_transition_action/shielded/shielded_withdrawal/mod.rs index ed54c01c824..11ab32792bf 100644 --- a/packages/rs-drive/src/state_transition_action/shielded/shielded_withdrawal/mod.rs +++ b/packages/rs-drive/src/state_transition_action/shielded/shielded_withdrawal/mod.rs @@ -96,6 +96,7 @@ mod tests { fn make_document() -> Document { Document::V0(DocumentV0 { + contract_version: None, id: Identifier::from([0x11; 32]), owner_id: Identifier::from([0x22; 32]), properties: Default::default(), diff --git a/packages/rs-drive/src/state_transition_action/shielded/shielded_withdrawal/v0/transformer.rs b/packages/rs-drive/src/state_transition_action/shielded/shielded_withdrawal/v0/transformer.rs index d674e99c965..86166ffd29d 100644 --- a/packages/rs-drive/src/state_transition_action/shielded/shielded_withdrawal/v0/transformer.rs +++ b/packages/rs-drive/src/state_transition_action/shielded/shielded_withdrawal/v0/transformer.rs @@ -90,6 +90,7 @@ impl ShieldedWithdrawalTransitionActionV0 { }); let withdrawal_document = DocumentV0 { + contract_version: None, id: document_id, owner_id, properties: document_data diff --git a/packages/rs-drive/src/util/object_size_info/document_info.rs b/packages/rs-drive/src/util/object_size_info/document_info.rs index 086d34ed46d..8fe04f45b03 100644 --- a/packages/rs-drive/src/util/object_size_info/document_info.rs +++ b/packages/rs-drive/src/util/object_size_info/document_info.rs @@ -304,6 +304,7 @@ mod tests { /// Helper: build a minimal Document (V0) with a given 32-byte id. fn make_document(id_bytes: [u8; 32]) -> Document { Document::V0(DocumentV0 { + contract_version: None, id: Identifier::new(id_bytes), owner_id: Identifier::new([0xAA; 32]), properties: BTreeMap::new(), diff --git a/packages/rs-drive/tests/drive_storage_ops_coverage.rs b/packages/rs-drive/tests/drive_storage_ops_coverage.rs index 36342c497ae..25ea15b592e 100644 --- a/packages/rs-drive/tests/drive_storage_ops_coverage.rs +++ b/packages/rs-drive/tests/drive_storage_ops_coverage.rs @@ -887,6 +887,7 @@ mod document_operation_tests { use dpp::document::Document; let doc = Document::V0(dpp::document::DocumentV0 { + contract_version: None, id: dpp::prelude::Identifier::new([1u8; 32]), owner_id: dpp::prelude::Identifier::new([2u8; 32]), properties: Default::default(), @@ -920,6 +921,7 @@ mod document_operation_tests { use dpp::document::Document; let doc = Document::V0(dpp::document::DocumentV0 { + contract_version: None, id: dpp::prelude::Identifier::new([1u8; 32]), owner_id: dpp::prelude::Identifier::new([2u8; 32]), properties: Default::default(), diff --git a/packages/rs-json-schema-compatibility-validator/src/rules/rule_set.rs b/packages/rs-json-schema-compatibility-validator/src/rules/rule_set.rs index 6718805e19f..3984cfb261e 100644 --- a/packages/rs-json-schema-compatibility-validator/src/rules/rule_set.rs +++ b/packages/rs-json-schema-compatibility-validator/src/rules/rule_set.rs @@ -1369,6 +1369,54 @@ pub static KEYWORD_COMPATIBILITY_RULES: Lazy = Laz ], }, ), + // `requiredSince` (the contract version a property is required from) + // is frozen on existing properties: a document's byte layout is + // resolved from the latest schema by comparing each property's + // `requiredSince` against the document's contract-version stamp, so + // changing the annotation retroactively would misparse stored + // documents. A brand-new property carrying the keyword arrives as a + // single Add of the whole property subschema and never resolves this + // rule; introducing it there is judged by the document type's + // required-fields update validation, not by this differ. + ( + "requiredSince", + CompatibilityRules { + allow_addition: false, + allow_removal: false, + allow_replacement_callback: FALSE_CALLBACK.clone(), + subschema_levels_depth: None, + inner: None, + #[cfg(any(test, feature = "examples"))] + examples: vec![ + ( + json!({}), + json!({ "requiredSince": 2 }), + Some(JsonSchemaChange::Add(AddOperation { + path: "/requiredSince".to_string(), + value: json!(2), + })), + ) + .into(), + ( + json!({ "requiredSince": 2 }), + json!({}), + Some(JsonSchemaChange::Remove(RemoveOperation { + path: "/requiredSince".to_string(), + })), + ) + .into(), + ( + json!({ "requiredSince": 2 }), + json!({ "requiredSince": 3 }), + Some(JsonSchemaChange::Replace(ReplaceOperation { + path: "/requiredSince".to_string(), + value: json!(3), + })), + ) + .into(), + ], + }, + ), ( "$defs", CompatibilityRules { diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/mod.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/mod.rs index e66bb564083..ec80ba92ca8 100644 --- a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/mod.rs +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/mod.rs @@ -82,6 +82,11 @@ pub struct DocumentTypeSchemaVersions { /// `None` on versions that predate the keyword: they ignore it entirely, /// exactly as they parsed before it existed. pub apply_property_reference: OptionalFeatureVersion, + /// Parses the `requiredSince` property keyword (the contract version from + /// which a property is required). `None` on versions that predate the + /// keyword: they ignore it entirely, exactly as they parsed before it + /// existed. + pub apply_required_since: OptionalFeatureVersion, pub validate_max_depth: FeatureVersion, pub max_depth: u16, pub recursive_schema_validator_versions: RecursiveSchemaValidatorVersions, diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v1.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v1.rs index 156436fbd24..bd6eda1e693 100644 --- a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v1.rs +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v1.rs @@ -43,6 +43,7 @@ pub const CONTRACT_VERSIONS_V1: DPPContractVersions = DPPContractVersions { find_identifier_and_binary_paths: 0, // This version predates the `refersTo` reference keyword apply_property_reference: None, + apply_required_since: None, validate_max_depth: 0, max_depth: 256, recursive_schema_validator_versions: RecursiveSchemaValidatorVersions { diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v2.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v2.rs index 45a29d6b353..1aec3222db0 100644 --- a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v2.rs +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v2.rs @@ -43,6 +43,7 @@ pub const CONTRACT_VERSIONS_V2: DPPContractVersions = DPPContractVersions { find_identifier_and_binary_paths: 0, // This version predates the `refersTo` reference keyword apply_property_reference: None, + apply_required_since: None, validate_max_depth: 0, max_depth: 256, recursive_schema_validator_versions: RecursiveSchemaValidatorVersions { diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v3.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v3.rs index 143f2719c93..5d513cc8506 100644 --- a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v3.rs +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v3.rs @@ -45,6 +45,7 @@ pub const CONTRACT_VERSIONS_V3: DPPContractVersions = DPPContractVersions { find_identifier_and_binary_paths: 0, // This version predates the `refersTo` reference keyword apply_property_reference: None, + apply_required_since: None, validate_max_depth: 0, max_depth: 256, recursive_schema_validator_versions: RecursiveSchemaValidatorVersions { diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v4.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v4.rs index fd12634279e..3c9f484e8c4 100644 --- a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v4.rs +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v4.rs @@ -45,6 +45,7 @@ pub const CONTRACT_VERSIONS_V4: DPPContractVersions = DPPContractVersions { find_identifier_and_binary_paths: 0, // This version predates the `refersTo` reference keyword apply_property_reference: None, + apply_required_since: None, validate_max_depth: 0, max_depth: 256, recursive_schema_validator_versions: RecursiveSchemaValidatorVersions { diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v5.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v5.rs index 1ca91eedd68..674f8593997 100644 --- a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v5.rs +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v5.rs @@ -47,6 +47,7 @@ pub const CONTRACT_VERSIONS_V5: DPPContractVersions = DPPContractVersions { find_identifier_and_binary_paths: 0, // This version predates the `refersTo` reference keyword apply_property_reference: None, + apply_required_since: None, validate_max_depth: 0, max_depth: 256, recursive_schema_validator_versions: RecursiveSchemaValidatorVersions { diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v6.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v6.rs index f61a7db2c4b..5c4dbcd8f2f 100644 --- a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v6.rs +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v6.rs @@ -69,6 +69,7 @@ pub const CONTRACT_VERSIONS_V6: DPPContractVersions = DPPContractVersions { enrich_with_base_schema: 1, find_identifier_and_binary_paths: 0, apply_property_reference: Some(0), // changed: the meta-schema v3 `refersTo` keyword is folded into the parsed property type; None before this version means the keyword is ignored, as it was before it existed + apply_required_since: Some(0), // changed: the meta-schema v3 `requiredSince` keyword (contract version a property is required from) is parsed onto the property; None before this version means the keyword is ignored, as it was before it existed validate_max_depth: 0, max_depth: 256, recursive_schema_validator_versions: RecursiveSchemaValidatorVersions { diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_document_versions/mod.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_document_versions/mod.rs index 22148d7eeaf..bbb55fefa6d 100644 --- a/packages/rs-platform-version/src/version/dpp_versions/dpp_document_versions/mod.rs +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_document_versions/mod.rs @@ -3,6 +3,7 @@ use versioned_feature_core::{FeatureVersion, FeatureVersionBounds}; pub mod v1; pub mod v2; pub mod v3; +pub mod v4; #[derive(Clone, Debug, Default)] pub struct DPPDocumentVersions { diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_document_versions/v4.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_document_versions/v4.rs new file mode 100644 index 00000000000..dbcc34f6117 --- /dev/null +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_document_versions/v4.rs @@ -0,0 +1,37 @@ +use crate::version::dpp_versions::dpp_document_versions::{ + DPPDocumentVersions, DocumentMethodVersions, +}; +use versioned_feature_core::FeatureVersionBounds; + +/// Document serialization moves to format 3: the document is stamped with the +/// data contract version its bytes conform to, right after the format prefix. +/// The stamp lets a property carry `requiredSince` (required from a given +/// contract version) while documents written before that version keep the +/// presence-flagged layout they were serialized with. Formats 0-2 predate the +/// stamp and deserialize with an unstamped (pre-annotation) layout. +pub const DOCUMENT_VERSIONS_V4: DPPDocumentVersions = DPPDocumentVersions { + document_structure_version: 0, + document_serialization_version: FeatureVersionBounds { + min_version: 0, + max_version: 3, + default_current_version: 3, + }, + document_cbor_serialization_version: FeatureVersionBounds { + min_version: 0, + max_version: 0, + default_current_version: 0, + }, + extended_document_structure_version: 0, + extended_document_serialization_version: FeatureVersionBounds { + min_version: 0, + max_version: 0, + default_current_version: 0, + }, + document_method_versions: DocumentMethodVersions { + is_equal_ignoring_timestamps: 0, + hash: 0, + get_raw_for_contract: 0, + get_raw_for_document_type: 0, + try_into_asset_unlock_base_transaction_info: 0, + }, +}; diff --git a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v10.rs b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v10.rs index 9195558adb4..d0ccbccd3f6 100644 --- a/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v10.rs +++ b/packages/rs-platform-version/src/version/drive_abci_versions/drive_abci_validation_versions/v10.rs @@ -95,7 +95,7 @@ pub const DRIVE_ABCI_VALIDATION_VERSIONS_V10: DriveAbciValidationVersions = }, masternode_vote_state_transition_balance_pre_check: 0, contract_create_state_transition: DriveAbciStateTransitionValidationVersion { - basic_structure: Some(1), + basic_structure: Some(2), // changed: rejects `requiredSince` other than 1 on a newly created contract — the annotation must name the version the change arrives with, and a fresh contract is version 1 advanced_structure: Some(1), identity_signatures: None, nonce: Some(0), diff --git a/packages/rs-platform-version/src/version/v14.rs b/packages/rs-platform-version/src/version/v14.rs index 21000350ca7..e5f3e09c909 100644 --- a/packages/rs-platform-version/src/version/v14.rs +++ b/packages/rs-platform-version/src/version/v14.rs @@ -2,7 +2,7 @@ use crate::version::consensus_versions::ConsensusVersions; use crate::version::dpp_versions::dpp_asset_lock_versions::v1::DPP_ASSET_LOCK_VERSIONS_V1; use crate::version::dpp_versions::dpp_contract_versions::v6::CONTRACT_VERSIONS_V6; use crate::version::dpp_versions::dpp_costs_versions::v1::DPP_COSTS_VERSIONS_V1; -use crate::version::dpp_versions::dpp_document_versions::v3::DOCUMENT_VERSIONS_V3; +use crate::version::dpp_versions::dpp_document_versions::v4::DOCUMENT_VERSIONS_V4; use crate::version::dpp_versions::dpp_factory_versions::v1::DPP_FACTORY_VERSIONS_V1; use crate::version::dpp_versions::dpp_identity_versions::v1::IDENTITY_VERSIONS_V1; use crate::version::dpp_versions::dpp_method_versions::v2::DPP_METHOD_VERSIONS_V2; @@ -113,6 +113,17 @@ pub const PROTOCOL_VERSION_14: ProtocolVersion = 14; /// reference property names an identity or contract that does not exist /// is rejected. v13 keeps the v9 table and therefore keeps /// accepting all of these, so replay of pre-upgrade blocks is unchanged. +/// * `DOCUMENT_VERSIONS_V4` bumps `document_serialization_version` to +/// default 3: documents are stamped with the contract version their bytes +/// conform to (a varint after the format prefix), enabling the +/// `requiredSince` property keyword — a contract update may add a new +/// required property annotated with the version that update creates. +/// Documents stamped below a property's `requiredSince` keep the +/// presence-flagged layout they were written with, so the latest contract +/// alone reconstructs every stamp's layout and no historical contract +/// lookups are ever needed. Reads dispatch on the byte prefix, so +/// formats 0–2 (all pre-v14 documents) deserialize exactly as before with +/// an unstamped (pre-annotation) layout. /// /// The wire surface is deliberately unchanged: `GetDocumentsRequestV1` /// already carries `selects` / `group_by` / `order_by` / `limit` / @@ -137,7 +148,7 @@ pub const PLATFORM_V14: PlatformVersion = PlatformVersion { state_transition_method_versions: STATE_TRANSITION_METHOD_VERSIONS_V1, state_transitions: STATE_TRANSITION_VERSIONS_V3, contract_versions: CONTRACT_VERSIONS_V6, // changed: v3 document meta-schema hosts the ranked index keywords - document_versions: DOCUMENT_VERSIONS_V3, + document_versions: DOCUMENT_VERSIONS_V4, // changed: document serialization format 3 — the contract version stamp that enables `requiredSince` properties identity_versions: IDENTITY_VERSIONS_V1, voting_versions: VOTING_VERSION_V2, token_versions: TOKEN_VERSIONS_V2, diff --git a/packages/rs-platform-wallet-ffi/src/document.rs b/packages/rs-platform-wallet-ffi/src/document.rs index 32509bc3356..79c18bc2e1f 100644 --- a/packages/rs-platform-wallet-ffi/src/document.rs +++ b/packages/rs-platform-wallet-ffi/src/document.rs @@ -532,6 +532,7 @@ mod tests { ); let document = Document::V0(DocumentV0 { + contract_version: None, id: Identifier::from([1u8; 32]), owner_id: Identifier::from([2u8; 32]), properties, diff --git a/packages/rs-platform-wallet/src/wallet/identity/network/contact_info.rs b/packages/rs-platform-wallet/src/wallet/identity/network/contact_info.rs index 36e34dd3556..a0a7a919204 100644 --- a/packages/rs-platform-wallet/src/wallet/identity/network/contact_info.rs +++ b/packages/rs-platform-wallet/src/wallet/identity/network/contact_info.rs @@ -726,6 +726,7 @@ impl DashPayView<'_, B> { properties.insert("privateData".to_string(), Value::Bytes(private_data)); let document = Document::V0(DocumentV0 { + contract_version: None, id: doc_id.unwrap_or_else(|| Identifier::from([0u8; 32])), owner_id: *identity_id, properties, diff --git a/packages/rs-platform-wallet/src/wallet/identity/network/contact_requests.rs b/packages/rs-platform-wallet/src/wallet/identity/network/contact_requests.rs index c99018792cf..46fa521e4bf 100644 --- a/packages/rs-platform-wallet/src/wallet/identity/network/contact_requests.rs +++ b/packages/rs-platform-wallet/src/wallet/identity/network/contact_requests.rs @@ -4287,6 +4287,7 @@ mod sweep_tests { ); let doc = Document::V0(DocumentV0 { + contract_version: None, id: Identifier::from([9u8; 32]), owner_id: sender, properties, diff --git a/packages/rs-platform-wallet/src/wallet/identity/network/profile.rs b/packages/rs-platform-wallet/src/wallet/identity/network/profile.rs index b5582883845..9cc79a181db 100644 --- a/packages/rs-platform-wallet/src/wallet/identity/network/profile.rs +++ b/packages/rs-platform-wallet/src/wallet/identity/network/profile.rs @@ -197,6 +197,7 @@ impl DashPayView<'_, B> { }; let stub_document = Document::V0(DocumentV0 { + contract_version: None, id: Identifier::from([0u8; 32]), owner_id: *identity_id, properties, @@ -358,6 +359,7 @@ impl DashPayView<'_, B> { }; let updated_document = Document::V0(DocumentV0 { + contract_version: None, id: existing_doc_id, owner_id: *identity_id, properties, diff --git a/packages/rs-sdk-ffi/src/document/create.rs b/packages/rs-sdk-ffi/src/document/create.rs index 1412ea0d46e..6aa744398d6 100644 --- a/packages/rs-sdk-ffi/src/document/create.rs +++ b/packages/rs-sdk-ffi/src/document/create.rs @@ -341,6 +341,7 @@ pub unsafe extern "C" fn dash_sdk_document_make_handle( // Create the document let document = Document::V0(DocumentV0 { + contract_version: None, id: document_id, owner_id, properties, diff --git a/packages/rs-sdk-ffi/src/document/delete.rs b/packages/rs-sdk-ffi/src/document/delete.rs index b1dbb23d439..864bc8019bb 100644 --- a/packages/rs-sdk-ffi/src/document/delete.rs +++ b/packages/rs-sdk-ffi/src/document/delete.rs @@ -400,6 +400,7 @@ mod tests { properties.insert("name".to_string(), Value::Text("Test Document".to_string())); let document = Document::V0(DocumentV0 { + contract_version: None, id, owner_id, properties, diff --git a/packages/rs-sdk-ffi/src/document/price.rs b/packages/rs-sdk-ffi/src/document/price.rs index 18aea1485c7..10197aa6a2b 100644 --- a/packages/rs-sdk-ffi/src/document/price.rs +++ b/packages/rs-sdk-ffi/src/document/price.rs @@ -336,6 +336,7 @@ mod tests { properties.insert("price".to_string(), Value::U64(1000)); let document = Document::V0(DocumentV0 { + contract_version: None, id, owner_id, properties, diff --git a/packages/rs-sdk-ffi/src/document/purchase.rs b/packages/rs-sdk-ffi/src/document/purchase.rs index bf55bb6fc11..8d45477eb1f 100644 --- a/packages/rs-sdk-ffi/src/document/purchase.rs +++ b/packages/rs-sdk-ffi/src/document/purchase.rs @@ -382,6 +382,7 @@ mod tests { properties.insert("price".to_string(), Value::U64(1000)); let document = Document::V0(DocumentV0 { + contract_version: None, id, owner_id, properties, diff --git a/packages/rs-sdk-ffi/src/document/put.rs b/packages/rs-sdk-ffi/src/document/put.rs index fb2539df325..d492b6bb065 100644 --- a/packages/rs-sdk-ffi/src/document/put.rs +++ b/packages/rs-sdk-ffi/src/document/put.rs @@ -400,6 +400,7 @@ mod tests { properties.insert("name".to_string(), Value::Text("Test Document".to_string())); let document = Document::V0(DocumentV0 { + contract_version: None, id, owner_id, properties, diff --git a/packages/rs-sdk-ffi/src/document/replace.rs b/packages/rs-sdk-ffi/src/document/replace.rs index c552bd291b2..6be96648c02 100644 --- a/packages/rs-sdk-ffi/src/document/replace.rs +++ b/packages/rs-sdk-ffi/src/document/replace.rs @@ -413,6 +413,7 @@ mod tests { properties.insert("age".to_string(), Value::U64(25)); let document = Document::V0(DocumentV0 { + contract_version: None, id, owner_id, properties, diff --git a/packages/rs-sdk-ffi/src/document/transfer.rs b/packages/rs-sdk-ffi/src/document/transfer.rs index 9dfeeaab883..00523d400a1 100644 --- a/packages/rs-sdk-ffi/src/document/transfer.rs +++ b/packages/rs-sdk-ffi/src/document/transfer.rs @@ -391,6 +391,7 @@ mod tests { ); let document = Document::V0(DocumentV0 { + contract_version: None, id, owner_id, properties, diff --git a/packages/rs-sdk/src/platform/dashpay/contact_request.rs b/packages/rs-sdk/src/platform/dashpay/contact_request.rs index d595faaaed7..adf48b41150 100644 --- a/packages/rs-sdk/src/platform/dashpay/contact_request.rs +++ b/packages/rs-sdk/src/platform/dashpay/contact_request.rs @@ -524,6 +524,7 @@ impl Sdk { // Create the document from the result let document = Document::V0(DocumentV0 { + contract_version: None, id: result.id, owner_id: result.owner_id, properties: result.properties, diff --git a/packages/rs-sdk/src/platform/documents/transitions/delete.rs b/packages/rs-sdk/src/platform/documents/transitions/delete.rs index 2d44ec735a5..da43978f5bd 100644 --- a/packages/rs-sdk/src/platform/documents/transitions/delete.rs +++ b/packages/rs-sdk/src/platform/documents/transitions/delete.rs @@ -179,6 +179,7 @@ impl DocumentDeleteTransitionBuilder { // Create a minimal document for deletion let document = Document::V0(dpp::document::DocumentV0 { + contract_version: None, id: self.document_id, owner_id: self.owner_id, properties: Default::default(), diff --git a/packages/rs-sdk/src/platform/documents/transitions/purchase.rs b/packages/rs-sdk/src/platform/documents/transitions/purchase.rs index fa6be76a464..d9b245139e0 100644 --- a/packages/rs-sdk/src/platform/documents/transitions/purchase.rs +++ b/packages/rs-sdk/src/platform/documents/transitions/purchase.rs @@ -90,6 +90,7 @@ impl DocumentPurchaseTransitionBuilder { // Create a minimal document with just the required fields // The actual document will be fetched during the transition let document = Document::V0(dpp::document::DocumentV0 { + contract_version: None, id: document_id, owner_id: current_owner_id, properties: Default::default(), diff --git a/packages/rs-sdk/src/platform/documents/transitions/set_price.rs b/packages/rs-sdk/src/platform/documents/transitions/set_price.rs index 6750a6700cc..75f289c2fce 100644 --- a/packages/rs-sdk/src/platform/documents/transitions/set_price.rs +++ b/packages/rs-sdk/src/platform/documents/transitions/set_price.rs @@ -84,6 +84,7 @@ impl DocumentSetPriceTransitionBuilder { // Create a minimal document with just the required fields // The actual document will be fetched during the transition let document = Document::V0(dpp::document::DocumentV0 { + contract_version: None, id: document_id, owner_id, properties: Default::default(), diff --git a/packages/rs-sdk/src/platform/documents/transitions/transfer.rs b/packages/rs-sdk/src/platform/documents/transitions/transfer.rs index 490380cc3fa..8bd5d515921 100644 --- a/packages/rs-sdk/src/platform/documents/transitions/transfer.rs +++ b/packages/rs-sdk/src/platform/documents/transitions/transfer.rs @@ -83,6 +83,7 @@ impl DocumentTransferTransitionBuilder { // Create a minimal document with just the required fields // The actual document will be fetched during the transition let document = Document::V0(dpp::document::DocumentV0 { + contract_version: None, id: document_id, owner_id, properties: Default::default(), diff --git a/packages/rs-sdk/src/platform/dpns_usernames/mod.rs b/packages/rs-sdk/src/platform/dpns_usernames/mod.rs index 4d6ba1f660f..0f6ddefa7cd 100644 --- a/packages/rs-sdk/src/platform/dpns_usernames/mod.rs +++ b/packages/rs-sdk/src/platform/dpns_usernames/mod.rs @@ -278,6 +278,7 @@ impl Sdk { // Create preorder document let preorder_document = Document::V0(DocumentV0 { + contract_version: None, id: preorder_id, owner_id: identity_id, properties: BTreeMap::from([( @@ -299,6 +300,7 @@ impl Sdk { // Create domain document let domain_document = Document::V0(DocumentV0 { + contract_version: None, id: domain_id, owner_id: identity_id, properties: BTreeMap::from([ diff --git a/packages/wasm-dpp/src/errors/consensus/consensus_error.rs b/packages/wasm-dpp/src/errors/consensus/consensus_error.rs index 7a1b2245af5..33daad96a0e 100644 --- a/packages/wasm-dpp/src/errors/consensus/consensus_error.rs +++ b/packages/wasm-dpp/src/errors/consensus/consensus_error.rs @@ -62,7 +62,7 @@ use dpp::consensus::state::data_trigger::DataTriggerError::{ DataTriggerConditionError, DataTriggerExecutionError, DataTriggerInvalidResultError, }; use wasm_bindgen::{JsError, JsValue}; -use dpp::consensus::basic::data_contract::{ContestedUniqueIndexOnMutableDocumentTypeError, ContestedUniqueIndexWithUniqueIndexError, DataContractTokenConfigurationUpdateError, DecimalsOverLimitError, DuplicateKeywordsError, GroupExceedsMaxMembersError, GroupHasTooFewMembersError, GroupMemberHasPowerOfZeroError, GroupMemberHasPowerOverLimitError, GroupNonUnilateralMemberPowerHasLessThanRequiredPowerError, GroupPositionDoesNotExistError, GroupRequiredPowerIsInvalidError, GroupTotalPowerLessThanRequiredError, InvalidDescriptionLengthError, InvalidDocumentTypeRequiredSecurityLevelError, InvalidKeywordCharacterError, InvalidKeywordLengthError, InvalidTokenBaseSupplyError, InvalidTokenDistributionFunctionDivideByZeroError, InvalidTokenDistributionFunctionIncoherenceError, InvalidTokenDistributionFunctionInvalidParameterError, InvalidTokenDistributionFunctionInvalidParameterTupleError, InvalidTokenLanguageCodeError, InvalidTokenNameCharacterError, InvalidTokenNameLengthError, MainGroupIsNotDefinedError, NewTokensDestinationIdentityOptionRequiredError, NonContiguousContractGroupPositionsError, NonContiguousContractTokenPositionsError, RedundantDocumentPaidForByTokenWithContractId, TokenPaymentByBurningOnlyAllowedOnInternalTokenError, TooManyKeywordsError, UnknownDocumentActionTokenEffectError, UnknownDocumentCreationRestrictionModeError, UnknownGasFeesPaidByError, UnknownSecurityLevelError, UnknownStorageKeyRequirementsError, UnknownTradeModeError, UnknownTransferableTypeError}; +use dpp::consensus::basic::data_contract::{ContestedUniqueIndexOnMutableDocumentTypeError, DataContractInvalidRequiredFieldsUpdateError, ContestedUniqueIndexWithUniqueIndexError, DataContractTokenConfigurationUpdateError, DecimalsOverLimitError, DuplicateKeywordsError, GroupExceedsMaxMembersError, GroupHasTooFewMembersError, GroupMemberHasPowerOfZeroError, GroupMemberHasPowerOverLimitError, GroupNonUnilateralMemberPowerHasLessThanRequiredPowerError, GroupPositionDoesNotExistError, GroupRequiredPowerIsInvalidError, GroupTotalPowerLessThanRequiredError, InvalidDescriptionLengthError, InvalidDocumentTypeRequiredSecurityLevelError, InvalidKeywordCharacterError, InvalidKeywordLengthError, InvalidTokenBaseSupplyError, InvalidTokenDistributionFunctionDivideByZeroError, InvalidTokenDistributionFunctionIncoherenceError, InvalidTokenDistributionFunctionInvalidParameterError, InvalidTokenDistributionFunctionInvalidParameterTupleError, InvalidTokenLanguageCodeError, InvalidTokenNameCharacterError, InvalidTokenNameLengthError, MainGroupIsNotDefinedError, NewTokensDestinationIdentityOptionRequiredError, NonContiguousContractGroupPositionsError, NonContiguousContractTokenPositionsError, RedundantDocumentPaidForByTokenWithContractId, TokenPaymentByBurningOnlyAllowedOnInternalTokenError, TooManyKeywordsError, UnknownDocumentActionTokenEffectError, UnknownDocumentCreationRestrictionModeError, UnknownGasFeesPaidByError, UnknownSecurityLevelError, UnknownStorageKeyRequirementsError, UnknownTradeModeError, UnknownTransferableTypeError}; use dpp::consensus::basic::document::{ContestedDocumentsTemporarilyNotAllowedError, DocumentCreationNotAllowedError, DocumentFieldMaxSizeExceededError, MaxDocumentsTransitionsExceededError, MissingPositionsInDocumentTypePropertiesError}; use dpp::consensus::basic::group::GroupActionNotAllowedOnTransitionError; use dpp::consensus::basic::identity::{DataContractBoundsNotPresentError, DisablingKeyIdAlsoBeingAddedInSameTransitionError, InvalidIdentityCreditWithdrawalTransitionAmountError, InvalidIdentityUpdateTransitionDisableKeysError, InvalidIdentityUpdateTransitionEmptyError, InvalidKeyPurposeForContractBoundsError, TooManyMasterPublicKeyError, WithdrawalOutputScriptNotAllowedWhenSigningWithOwnerKeyError}; @@ -1005,6 +1005,9 @@ fn from_basic_error(basic_error: &BasicError) -> JsValue { BasicError::TokenPricingScheduleEmptyError(e) => { generic_consensus_error!(TokenPricingScheduleEmptyError, e).into() } + BasicError::DataContractInvalidRequiredFieldsUpdateError(e) => { + generic_consensus_error!(DataContractInvalidRequiredFieldsUpdateError, e).into() + } } } diff --git a/packages/wasm-dpp2/src/data_contract/document/model.rs b/packages/wasm-dpp2/src/data_contract/document/model.rs index d447196c486..b49d54ff9a3 100644 --- a/packages/wasm-dpp2/src/data_contract/document/model.rs +++ b/packages/wasm-dpp2/src/data_contract/document/model.rs @@ -235,6 +235,7 @@ impl DocumentWasm { )?; let document = Document::V0(DocumentV0 { + contract_version: None, id: doc_id, owner_id, properties, From fb5d97e17e877c9fcccd59693bd7fb347cd0ae7b Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Fri, 14 Aug 2026 00:38:36 +0700 Subject: [PATCH 02/23] fix(platform): address requiredSince review findings - CI: regenerate withdrawal query test root hashes (every document now carries the stamp byte) and latest-version estimated-fee pins - from_bytes_v3 hard-errors on unconsumed trailing bytes: a reader with a stale contract can no longer silently drop fields a newer-stamped document carries; the error directs it to refetch the contract - requiredSince <= contract version is now enforced on *parsed* document properties (validate_required_since_within_contract_version) at every serialization->struct conversion, closing the $defs $ref bypass of the raw-JSON creation scan; the basic_structure v2 scan remains as an early cheap rejection and is documented as non-authoritative - document types introduced by a contract update (which have no old counterpart for the per-type diff) must annotate requiredSince with exactly the version the update creates - create/replace stamping moved out of the shipped v0 action->Document conversions into new generation-1 modules dispatched on a new document_from_action version slot (DRIVE_STATE_TRANSITION_METHOD_VERSIONS_V4, selected only by protocol v14); v0 restored byte-identical - estimated_size v1 adds the format-3 stamp varint (worst case 5 bytes) to worst-case document size estimation, gated at protocol v14 - CBOR document form carries the stamp as an optional $contractVersion entry (skipped when absent, so pre-stamp CBOR stays byte-identical) - contract_version accessors on Document; regression tests for each fix Co-Authored-By: Claude Fable 5 --- .../class_methods/try_from_schema/mod.rs | 94 ++++++++++++++ .../document_type/methods/mod.rs | 3 +- .../methods/versioned_methods.rs | 7 ++ .../src/data_contract/document_type/mod.rs | 31 +++++ .../methods/validate_update/v0/mod.rs | 117 +++++++++++++++++- .../src/data_contract/v0/serialization/mod.rs | 12 ++ .../src/data_contract/v1/serialization/mod.rs | 12 ++ packages/rs-dpp/src/document/accessors/mod.rs | 12 ++ .../rs-dpp/src/document/accessors/v0/mod.rs | 6 + packages/rs-dpp/src/document/fields.rs | 1 + packages/rs-dpp/src/document/v0/accessors.rs | 11 ++ .../rs-dpp/src/document/v0/cbor_conversion.rs | 51 +++++++- packages/rs-dpp/src/document/v0/serialize.rs | 81 ++++++++++++ .../basic_structure/v2/mod.rs | 99 +++++++++++++++ .../rs-drive/src/drive/document/delete/mod.rs | 4 +- .../rs-drive/src/drive/document/insert/mod.rs | 4 +- .../document_create_transition_action/mod.rs | 42 ++++++- .../v0/mod.rs | 36 +----- .../v1/mod.rs | 70 +++++++++++ .../document_replace_transition_action/mod.rs | 46 ++++++- .../v0/mod.rs | 35 +----- .../v1/mod.rs | 63 ++++++++++ packages/rs-drive/tests/query_tests.rs | 12 +- .../dpp_versions/dpp_contract_versions/v6.rs | 2 +- .../mod.rs | 13 ++ .../v1.rs | 5 + .../v2.rs | 5 + .../v3.rs | 5 + .../v4.rs | 69 +++++++++++ .../src/version/drive_versions/v9.rs | 4 +- 30 files changed, 865 insertions(+), 87 deletions(-) create mode 100644 packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_create_transition_action/v1/mod.rs create mode 100644 packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_replace_transition_action/v1/mod.rs create mode 100644 packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v4.rs diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs index 45bf15e501f..ce6ba7e1755 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs @@ -954,6 +954,26 @@ mod tests { ); } + #[test] + fn should_reject_required_since_above_u32_max() { + // The meta-schema caps the value at u32::MAX too; this pins the + // parser-side rejection so it does not depend on meta-schema + // coverage (parses without full validation skip the meta-schema) + let result = try_document_type_from_schema(json!({ + "type": "object", + "properties": { + "a": {"type": "string", "position": 0, "maxLength": 60, "requiredSince": 4_294_967_296_u64}, + }, + "required": ["a"], + "additionalProperties": false + })); + + assert!( + result.is_err(), + "requiredSince above u32::MAX must be rejected" + ); + } + #[test] fn should_reject_required_since_of_zero() { let result = try_document_type_from_schema(json!({ @@ -971,6 +991,80 @@ mod tests { ); } + #[test] + fn should_parse_required_since_reached_through_a_ref() { + // A `$ref`'d property resolves to its `$defs` entry before keywords + // are read, so an annotation hidden behind a reference is parsed + // exactly like a direct one — any validation that only scans raw + // property JSON would miss it, which is why the + // `requiredSince <= contract version` invariant is enforced on + // parsed properties (validate_required_since_within_contract_version) + let platform_version = PlatformVersion::latest(); + let config = + DataContractConfig::default_for_version(platform_version).expect("config should build"); + + let schema_defs: BTreeMap = [( + "annotated".to_string(), + platform_value::to_value(json!({ + "type": "string", "maxLength": 60, "requiredSince": 2 + })) + .expect("defs should convert"), + )] + .into_iter() + .collect(); + + let schema = platform_value::to_value(json!({ + "type": "object", + "properties": { + "a": {"type": "string", "position": 0, "maxLength": 60}, + "b": {"$ref": "#/$defs/annotated", "position": 1}, + }, + "required": ["a", "b"], + "additionalProperties": false + })) + .expect("schema should convert"); + + let document_type = DocumentType::try_from_schema( + Identifier::random(), + 0, + config.version(), + "msg", + schema, + Some(&schema_defs), + &BTreeMap::new(), + &config, + false, + &mut vec![], + platform_version, + ) + .expect("should parse"); + + let properties = document_type.as_ref().flattened_properties().clone(); + assert_eq!(properties.get("b").unwrap().required_since, Some(2)); + + // The parsed-property invariant check sees the annotation the raw + // JSON hides: version 1 (too old for requiredSince 2) rejects, + // version 2 accepts + let mut document_types = BTreeMap::new(); + document_types.insert("msg".to_string(), document_type); + + assert!( + crate::data_contract::document_type::validate_required_since_within_contract_version( + &document_types, + 1 + ) + .is_err(), + "requiredSince 2 must be rejected on a version 1 contract even through $ref" + ); + assert!( + crate::data_contract::document_type::validate_required_since_within_contract_version( + &document_types, + 2 + ) + .is_ok() + ); + } + #[test] fn should_ignore_required_since_on_platform_versions_predating_it() { // Platform versions whose tables carry `apply_required_since: None` diff --git a/packages/rs-dpp/src/data_contract/document_type/methods/mod.rs b/packages/rs-dpp/src/data_contract/document_type/methods/mod.rs index 88ce7c0f39d..fd773b07c5f 100644 --- a/packages/rs-dpp/src/data_contract/document_type/methods/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/methods/mod.rs @@ -164,9 +164,10 @@ pub trait DocumentTypeV0Methods: DocumentTypeV0Getters + DocumentTypeV0MethodsVe .estimated_size { 0 => self.estimated_size_v0(platform_version), + 1 => self.estimated_size_v1(platform_version), version => Err(ProtocolError::UnknownVersionMismatch { method: "estimated_size".to_string(), - known_versions: vec![0], + known_versions: vec![0, 1], received: version, }), } diff --git a/packages/rs-dpp/src/data_contract/document_type/methods/versioned_methods.rs b/packages/rs-dpp/src/data_contract/document_type/methods/versioned_methods.rs index 701a780e49f..d2f18611bd1 100644 --- a/packages/rs-dpp/src/data_contract/document_type/methods/versioned_methods.rs +++ b/packages/rs-dpp/src/data_contract/document_type/methods/versioned_methods.rs @@ -442,6 +442,13 @@ pub trait DocumentTypeV0MethodsVersioned: DocumentTypeV0Getters + DocumentTypeBa Ok(total_size) } + /// Generation 0 plus the document serialization format 3 + /// contract-version stamp varint (worst case 5 bytes for a u32). + /// Selected together with format 3 by the version table. + fn estimated_size_v1(&self, platform_version: &PlatformVersion) -> Result { + Ok(self.estimated_size_v0(platform_version)?.saturating_add(5)) + } + fn max_size_v0(&self, platform_version: &PlatformVersion) -> Result { let mut total_size = 0u16; diff --git a/packages/rs-dpp/src/data_contract/document_type/mod.rs b/packages/rs-dpp/src/data_contract/document_type/mod.rs index 46fae6efbe2..60bbbb9e53f 100644 --- a/packages/rs-dpp/src/data_contract/document_type/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/mod.rs @@ -40,6 +40,37 @@ pub const EMPTY_TREE_STORAGE_SIZE: usize = 33; pub const MAX_INDEX_SIZE: usize = 255; pub const STORAGE_FLAGS_SIZE: usize = 2; +/// A `requiredSince` annotation may never exceed the version of the contract +/// carrying it — requiredness cannot be pre-scheduled at a future version. +/// Runs over the *parsed* properties, so annotations reached through `$ref` +/// are covered. Called wherever document types are built from a contract's +/// serialized form (creates, updates, and disk loads all pass through +/// there); a no-op for every contract predating the keyword, since their +/// properties carry no annotation. +pub(crate) fn validate_required_since_within_contract_version( + document_types: &std::collections::BTreeMap, + contract_version: u32, +) -> Result<(), crate::data_contract::errors::DataContractError> { + use crate::data_contract::document_type::accessors::DocumentTypeV0Getters; + + for (document_type_name, document_type) in document_types { + for (property_name, property) in document_type.as_ref().properties() { + if let Some(required_since) = property.required_since { + if required_since > contract_version { + return Err( + crate::data_contract::errors::DataContractError::InvalidContractStructure( + format!( + "property '{property_name}' of document type '{document_type_name}' carries requiredSince {required_since} which exceeds the contract version {contract_version}" + ), + ), + ); + } + } + } + } + Ok(()) +} + pub(crate) mod property_names { pub const DOCUMENTS_KEEP_HISTORY: &str = "documentsKeepHistory"; pub const KEEPS_TRANSFER_HISTORY: &str = "keepsTransferHistory"; diff --git a/packages/rs-dpp/src/data_contract/methods/validate_update/v0/mod.rs b/packages/rs-dpp/src/data_contract/methods/validate_update/v0/mod.rs index b71b6d2c09b..c7a38bb18d5 100644 --- a/packages/rs-dpp/src/data_contract/methods/validate_update/v0/mod.rs +++ b/packages/rs-dpp/src/data_contract/methods/validate_update/v0/mod.rs @@ -6,7 +6,8 @@ use crate::consensus::state::token::PreProgrammedDistributionTimestampInPastErro use crate::data_contract::accessors::v0::DataContractV0Getters; use crate::consensus::basic::data_contract::{ - DuplicateKeywordsError, IncompatibleDataContractSchemaError, InvalidDataContractVersionError, + DataContractInvalidRequiredFieldsUpdateError, DuplicateKeywordsError, + IncompatibleDataContractSchemaError, InvalidDataContractVersionError, InvalidDescriptionLengthError, InvalidKeywordCharacterError, InvalidKeywordLengthError, TooManyKeywordsError, }; @@ -17,6 +18,7 @@ use crate::data_contract::accessors::v1::DataContractV1Getters; use crate::data_contract::associated_token::token_configuration::accessors::v0::TokenConfigurationV0Getters; use crate::data_contract::associated_token::token_distribution_rules::accessors::v0::TokenDistributionRulesV0Getters; use crate::data_contract::associated_token::token_pre_programmed_distribution::accessors::v0::TokenPreProgrammedDistributionV0Methods; +use crate::data_contract::document_type::accessors::DocumentTypeV0Getters; use crate::data_contract::document_type::schema::validate_schema_compatibility; use crate::data_contract::schema::DataContractSchemaMethodsV0; use crate::data_contract::DataContract; @@ -120,6 +122,40 @@ impl DataContract { } } + // Document types introduced by this update have no old counterpart, + // so the per-type update validation above never sees them. Their + // `requiredSince` annotations must name the version this update + // creates — anything else would pre-schedule (or backdate) a + // wire-layout change without validation. Replay safety: this loop is + // a no-op for every contract that predates the `requiredSince` + // keyword (protocol v14's meta-schema), because such contracts can + // carry no annotation — older meta-schemas rejected the keyword at + // write time and older parsers ignore it entirely. + for (document_type_name, new_document_type) in new_data_contract.document_types() { + if self + .document_type_optional_for_name(document_type_name) + .is_some() + { + continue; + } + for (property_name, property) in new_document_type.as_ref().properties() { + if let Some(required_since) = property.required_since { + if required_since != new_data_contract.version() { + return Ok(SimpleConsensusValidationResult::new_with_error( + DataContractInvalidRequiredFieldsUpdateError::new( + document_type_name.clone(), + format!( + "new document type property '{property_name}' must carry requiredSince {}, the contract version this update creates", + new_data_contract.version() + ), + ) + .into(), + )); + } + } + } + } + // Schema $defs should be compatible if let Some(old_defs_map) = self.schema_defs() { // If new contract doesn't have $defs, it means that it's $defs was removed and compatibility is broken @@ -366,6 +402,85 @@ mod tests { use crate::identity::accessors::IdentityGettersV0; use crate::prelude::Identity; + #[test] + fn should_validate_required_since_on_document_types_added_by_the_update() { + let platform_version = PlatformVersion::latest(); + + let old_data_contract = get_data_contract_fixture( + None, + IdentityNonce::default(), + platform_version.protocol_version, + ) + .data_contract_owned(); + + let new_type_schema = |required_since: u32| { + platform_value!({ + "type": "object", + "properties": { + "message": { + "type": "string", + "position": 0, + "maxLength": 60_u32, + "requiredSince": required_since, + } + }, + "required": ["message"], + "additionalProperties": false + }) + }; + + // A new document type pre-scheduling requiredness at version 99 + // has no old counterpart, so the per-type update validation + // never runs on it — this pass must catch it + let mut new_data_contract = old_data_contract.clone(); + new_data_contract.set_version(old_data_contract.version() + 1); + new_data_contract + .set_document_schema( + "note", + new_type_schema(99), + false, + &mut Vec::new(), + platform_version, + ) + .expect("should add document type"); + + let result = old_data_contract + .validate_update(&new_data_contract, &BlockInfo::default(), platform_version) + .expect("failed validate update"); + + assert_matches!( + result.errors.as_slice(), + [ConsensusError::BasicError( + BasicError::DataContractInvalidRequiredFieldsUpdateError(e) + )] if e.details().contains("must carry requiredSince 2") + ); + + // The same new document type annotated with the version this + // update creates is accepted + let mut new_data_contract = old_data_contract.clone(); + new_data_contract.set_version(old_data_contract.version() + 1); + new_data_contract + .set_document_schema( + "note", + new_type_schema(old_data_contract.version() + 1), + false, + &mut Vec::new(), + platform_version, + ) + .expect("should add document type"); + + let result = old_data_contract + .validate_update(&new_data_contract, &BlockInfo::default(), platform_version) + .expect("failed validate update"); + + assert!( + result.is_valid(), + "a new document type annotated with the version this update \ + creates must be accepted, got {:?}", + result.errors + ); + } + #[test] fn should_return_invalid_result_if_owner_id_is_not_the_same() { let platform_version = PlatformVersion::latest(); diff --git a/packages/rs-dpp/src/data_contract/v0/serialization/mod.rs b/packages/rs-dpp/src/data_contract/v0/serialization/mod.rs index 2fcee85088d..beb38d035b5 100644 --- a/packages/rs-dpp/src/data_contract/v0/serialization/mod.rs +++ b/packages/rs-dpp/src/data_contract/v0/serialization/mod.rs @@ -101,6 +101,12 @@ impl DataContractV0 { platform_version, )?; + crate::data_contract::document_type::validate_required_since_within_contract_version( + &document_types, + version, + ) + .map_err(ProtocolError::DataContractError)?; + let data_contract = DataContractV0 { id, version, @@ -144,6 +150,12 @@ impl DataContractV0 { platform_version, )?; + crate::data_contract::document_type::validate_required_since_within_contract_version( + &document_types, + version, + ) + .map_err(ProtocolError::DataContractError)?; + let data_contract = DataContractV0 { id, version, diff --git a/packages/rs-dpp/src/data_contract/v1/serialization/mod.rs b/packages/rs-dpp/src/data_contract/v1/serialization/mod.rs index c044aaf0833..b12357388bc 100644 --- a/packages/rs-dpp/src/data_contract/v1/serialization/mod.rs +++ b/packages/rs-dpp/src/data_contract/v1/serialization/mod.rs @@ -100,6 +100,12 @@ impl DataContractV1 { platform_version, )?; + crate::data_contract::document_type::validate_required_since_within_contract_version( + &document_types, + version, + ) + .map_err(ProtocolError::DataContractError)?; + let data_contract = DataContractV1 { id, version, @@ -161,6 +167,12 @@ impl DataContractV1 { platform_version, )?; + crate::data_contract::document_type::validate_required_since_within_contract_version( + &document_types, + version, + ) + .map_err(ProtocolError::DataContractError)?; + let data_contract = DataContractV1 { id, version, diff --git a/packages/rs-dpp/src/document/accessors/mod.rs b/packages/rs-dpp/src/document/accessors/mod.rs index 0b52caebccd..d629cbffcbd 100644 --- a/packages/rs-dpp/src/document/accessors/mod.rs +++ b/packages/rs-dpp/src/document/accessors/mod.rs @@ -116,6 +116,12 @@ impl DocumentV0Getters for Document { Document::V0(v0) => v0.creator_id, } } + + fn contract_version(&self) -> Option { + match self { + Document::V0(v0) => v0.contract_version, + } + } } impl DocumentV0Setters for Document { @@ -213,6 +219,12 @@ impl DocumentV0Setters for Document { Document::V0(v0) => v0.creator_id = creator_id, } } + + fn set_contract_version(&mut self, contract_version: Option) { + match self { + Document::V0(v0) => v0.contract_version = contract_version, + } + } } #[cfg(test)] diff --git a/packages/rs-dpp/src/document/accessors/v0/mod.rs b/packages/rs-dpp/src/document/accessors/v0/mod.rs index b2de832bc36..b448e060016 100644 --- a/packages/rs-dpp/src/document/accessors/v0/mod.rs +++ b/packages/rs-dpp/src/document/accessors/v0/mod.rs @@ -48,6 +48,9 @@ pub trait DocumentV0Getters { fn updated_at_core_block_height(&self) -> Option; fn transferred_at_core_block_height(&self) -> Option; fn creator_id(&self) -> Option; + /// The data contract version this document's bytes conform to (the + /// serialization format 3 stamp); `None` for pre-stamp documents. + fn contract_version(&self) -> Option; } pub trait DocumentV0Setters: DocumentV0Getters { @@ -158,4 +161,7 @@ pub trait DocumentV0Setters: DocumentV0Getters { /// - `creator_id`: An `Option` to set as the document's creator ID. /// `None` indicates the creator ID is not available. fn set_creator_id(&mut self, creator_id: Option); + /// Sets the contract-version stamp: the data contract version this + /// document's bytes conform to. + fn set_contract_version(&mut self, contract_version: Option); } diff --git a/packages/rs-dpp/src/document/fields.rs b/packages/rs-dpp/src/document/fields.rs index 2d88483f034..93dbd0bc200 100644 --- a/packages/rs-dpp/src/document/fields.rs +++ b/packages/rs-dpp/src/document/fields.rs @@ -6,6 +6,7 @@ pub mod property_names { pub const REVISION: &str = "$revision"; pub const OWNER_ID: &str = "$ownerId"; pub const CREATOR_ID: &str = "$creatorId"; + pub const CONTRACT_VERSION: &str = "$contractVersion"; pub const PRICE: &str = "$price"; pub const CREATED_AT: &str = "$createdAt"; pub const UPDATED_AT: &str = "$updatedAt"; diff --git a/packages/rs-dpp/src/document/v0/accessors.rs b/packages/rs-dpp/src/document/v0/accessors.rs index 7d0b47f0de3..7ce2269f81c 100644 --- a/packages/rs-dpp/src/document/v0/accessors.rs +++ b/packages/rs-dpp/src/document/v0/accessors.rs @@ -160,6 +160,10 @@ impl DocumentV0Getters for DocumentV0 { fn creator_id(&self) -> Option { self.creator_id } + + fn contract_version(&self) -> Option { + self.contract_version + } } impl DocumentV0Setters for DocumentV0 { @@ -290,4 +294,11 @@ impl DocumentV0Setters for DocumentV0 { fn set_creator_id(&mut self, creator_id: Option) { self.creator_id = creator_id; } + + /// Sets the contract-version stamp: the data contract version this + /// document's bytes conform to. Assigned by Drive when document content + /// is (re-)supplied; `None` for pre-stamp documents. + fn set_contract_version(&mut self, contract_version: Option) { + self.contract_version = contract_version; + } } diff --git a/packages/rs-dpp/src/document/v0/cbor_conversion.rs b/packages/rs-dpp/src/document/v0/cbor_conversion.rs index 36c3621e2e1..632d9c5ea68 100644 --- a/packages/rs-dpp/src/document/v0/cbor_conversion.rs +++ b/packages/rs-dpp/src/document/v0/cbor_conversion.rs @@ -58,6 +58,15 @@ pub struct DocumentForCbor { #[serde(rename = "$creatorId")] pub creator_id: Option, + + /// The contract-version stamp. Skipped when absent so pre-stamp CBOR + /// output stays byte-identical; `default` keeps old CBOR readable. + #[serde( + rename = "$contractVersion", + default, + skip_serializing_if = "Option::is_none" + )] + pub contract_version: Option, } #[cfg(feature = "cbor")] @@ -80,10 +89,10 @@ impl TryFrom for DocumentForCbor { updated_at_core_block_height, transferred_at_core_block_height, creator_id, - // The CBOR document form predates the contract-version stamp - contract_version: _, + contract_version, } = value; Ok(DocumentForCbor { + contract_version, id: id.to_buffer(), properties: Value::convert_to_cbor_map(properties) .map_err(ProtocolError::ValueError)?, @@ -148,9 +157,12 @@ impl DocumentV0 { .remove_optional_identifier(property_names::CREATOR_ID) .map_err(ProtocolError::ValueError)?; + let contract_version = + document_map.remove_optional_integer(property_names::CONTRACT_VERSION)?; + // dev-note: properties is everything other than the id and owner id Ok(DocumentV0 { - contract_version: None, + contract_version, properties: document_map, owner_id: Identifier::new(owner_id), id: Identifier::new(id), @@ -254,6 +266,39 @@ mod tests { // Round-trip: to_cbor -> from_cbor preserves document data // ================================================================ + #[test] + fn cbor_round_trip_preserves_contract_version_stamp() { + use crate::document::Document; + + let platform_version = PlatformVersion::latest(); + let mut document = make_document_v0_with_timestamps(); + document.contract_version = Some(7); + + let cbor = document.to_cbor().expect("expected to serialize to cbor"); + let restored = Document::from_cbor(&cbor, None, None, platform_version) + .expect("expected to deserialize from cbor"); + + let Document::V0(restored) = restored; + assert_eq!(restored.contract_version, Some(7)); + assert_eq!(restored.id, document.id); + assert_eq!(restored.revision, document.revision); + // (full property equality is not asserted: CBOR decodes integers as + // I128, a pre-existing normalization of this legacy path) + assert_eq!( + restored.properties.get("name"), + document.properties.get("name") + ); + + // An unstamped document round-trips to no stamp (the key is skipped + // entirely when absent, keeping pre-stamp CBOR byte-identical) + let unstamped = make_document_v0_with_timestamps(); + let unstamped_cbor = unstamped.to_cbor().expect("expected to serialize to cbor"); + let restored_unstamped = Document::from_cbor(&unstamped_cbor, None, None, platform_version) + .expect("expected to deserialize from cbor"); + let Document::V0(restored_unstamped) = restored_unstamped; + assert_eq!(restored_unstamped.contract_version, None); + } + #[test] fn cbor_round_trip_with_random_dashpay_profile() { let platform_version = PlatformVersion::latest(); diff --git a/packages/rs-dpp/src/document/v0/serialize.rs b/packages/rs-dpp/src/document/v0/serialize.rs index f187d5c05b9..404022c1a5c 100644 --- a/packages/rs-dpp/src/document/v0/serialize.rs +++ b/packages/rs-dpp/src/document/v0/serialize.rs @@ -1874,6 +1874,25 @@ impl DocumentPlatformDeserializationMethodsV0 for DocumentV0 { properties.insert(PRICE.to_string(), price.into()); } + // Every property the document was serialized with must have been + // consumed. Trailing bytes mean the document was written under a + // newer contract version than the document type used to read it — a + // stale reader would otherwise silently drop the fields it does not + // know about. The stamp makes this detectable: callers should + // refetch the contract and retry. + let mut trailing_probe = [0u8; 1]; + let trailing = buf.read(&mut trailing_probe).map_err(|_| { + DataContractError::CorruptedSerialization( + "error probing for trailing bytes in serialized document".to_string(), + ) + })?; + if trailing > 0 { + return Err(DataContractError::CorruptedSerialization(format!( + "serialized document has trailing bytes: it was serialized under contract version {} with properties this document type does not know; refetch the contract", + stamp + ))); + } + Ok(DocumentV0 { contract_version, id: Identifier::new(id), @@ -3343,6 +3362,68 @@ mod tests { assert_eq!(deserialized, document); } + #[test] + fn stale_document_type_rejects_document_stamped_under_newer_contract() { + use crate::data_contract::config::DataContractConfig; + use crate::data_contract::document_type::DocumentType; + use platform_value::platform_value; + + let platform_version = PlatformVersion::latest(); + + // The reader's stale view: the schema as of contract version 1, + // before `b` and `c` were appended + let stale_schema = platform_value!({ + "type": "object", + "properties": { + "a": {"type": "string", "position": 0, "maxLength": 60_u32}, + }, + "required": ["a"], + "additionalProperties": false, + }); + let config = DataContractConfig::default_for_version(platform_version) + .expect("should create a default config"); + let stale_document_type = DocumentType::try_from_schema( + Identifier::new([1; 32]), + 1, + config.version(), + "test", + stale_schema, + None, + &BTreeMap::new(), + &config, + false, + &mut Vec::new(), + platform_version, + ) + .expect("failed to create stale document type"); + + // A document written under contract version 2, where `b` exists and + // is required + let current_document_type = required_since_document_type(); + let mut properties = BTreeMap::new(); + properties.insert("a".to_string(), Value::Text("alpha".to_string())); + properties.insert("b".to_string(), Value::Text("beta".to_string())); + let document = stamped_document(Some(2), properties, current_document_type.as_ref()); + + let serialized = document + .serialize_v3(current_document_type.as_ref()) + .expect("expected serialization"); + + // A stale reader must hard-error on the trailing bytes instead of + // silently dropping the field it does not know about + let result = + DocumentV0::from_bytes(&serialized, stale_document_type.as_ref(), platform_version); + assert!( + matches!( + &result, + Err(ProtocolError::DataContractError( + DataContractError::CorruptedSerialization(message) + )) if message.contains("trailing bytes") + ), + "a stale document type must reject a newer-stamped document, got {result:?}" + ); + } + #[test] fn stamp_survives_the_wire_for_documents_stamped_past_required_since() { let platform_version = PlatformVersion::latest(); diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/basic_structure/v2/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/basic_structure/v2/mod.rs index e59a2e06e07..b8c12c7c0c2 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/basic_structure/v2/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/basic_structure/v2/mod.rs @@ -39,6 +39,14 @@ impl DataContractCreateStateTransitionBasicStructureValidationV2 for DataContrac // requiredness at a future version — coherent for the wire format, // but banned: requiredness changes must arrive with the update that // creates the version they name. + // + // This raw-JSON scan is an early, cheap rejection only — it cannot + // see an annotation reached through a `$defs` `$ref`. The + // authoritative enforcement is + // `validate_required_since_within_contract_version` in dpp, which + // runs on the *parsed* properties (references resolved) whenever the + // contract is built from its serialized form, including this + // transition's transform into action. for (document_type_name, schema) in self.data_contract().document_schemas() { let Some(properties) = schema .get_optional_value(PROPERTIES) @@ -80,3 +88,94 @@ impl DataContractCreateStateTransitionBasicStructureValidationV2 for DataContrac Ok(SimpleConsensusValidationResult::new()) } } + +#[cfg(test)] +mod tests { + use super::*; + use assert_matches::assert_matches; + use dpp::consensus::basic::BasicError; + use dpp::consensus::ConsensusError; + use dpp::platform_value::platform_value; + use dpp::prelude::IdentityNonce; + use dpp::state_transition::data_contract_create_transition::DataContractCreateTransitionV0; + use dpp::tests::fixtures::get_data_contract_fixture; + use platform_version::version::PlatformVersion; + use platform_version::TryIntoPlatformVersioned; + + fn create_transition_with_required_since( + required_since: u32, + ) -> (DataContractCreateTransition, &'static PlatformVersion) { + let platform_version = PlatformVersion::latest(); + let identity_nonce = IdentityNonce::default(); + + let data_contract = + get_data_contract_fixture(None, identity_nonce, platform_version.protocol_version) + .data_contract_owned(); + + let mut data_contract_for_serialization: dpp::data_contract::serialized_version::DataContractInSerializationFormat = data_contract + .try_into_platform_versioned(platform_version) + .expect("failed to convert data contract"); + + data_contract_for_serialization + .document_schemas_mut() + .insert( + "note".to_string(), + platform_value!({ + "type": "object", + "properties": { + "message": { + "type": "string", + "position": 0, + "maxLength": 60, + "requiredSince": required_since, + } + }, + "required": ["message"], + "additionalProperties": false + }), + ); + + let transition: DataContractCreateTransition = DataContractCreateTransitionV0 { + data_contract: data_contract_for_serialization, + identity_nonce, + user_fee_increase: 0, + signature_public_key_id: 0, + signature: Default::default(), + } + .into(); + + (transition, platform_version) + } + + #[test] + fn should_accept_required_since_of_one_on_a_new_contract() { + let (transition, platform_version) = create_transition_with_required_since(1); + + let result = transition + .validate_basic_structure_v2(Network::Testnet, platform_version) + .expect("failed to validate basic structure"); + + assert!( + result.is_valid(), + "requiredSince 1 on a fresh contract is equivalent to plain \ + required and must be accepted, got {:?}", + result.errors + ); + } + + #[test] + fn should_reject_required_since_above_one_on_a_new_contract() { + let (transition, platform_version) = create_transition_with_required_since(2); + + let result = transition + .validate_basic_structure_v2(Network::Testnet, platform_version) + .expect("failed to validate basic structure"); + + assert_matches!( + result.errors.as_slice(), + [ConsensusError::BasicError( + BasicError::DataContractInvalidRequiredFieldsUpdateError(e) + )] if e.details().contains("cannot carry requiredSince 2") + ); + } +} diff --git a/packages/rs-drive/src/drive/document/delete/mod.rs b/packages/rs-drive/src/drive/document/delete/mod.rs index 9e074da93b2..717446bac3f 100644 --- a/packages/rs-drive/src/drive/document/delete/mod.rs +++ b/packages/rs-drive/src/drive/document/delete/mod.rs @@ -1047,7 +1047,9 @@ mod tests { assert!(fee_result.fee_refunds.0.is_empty()); assert_eq!(fee_result.storage_fee, 0); - assert_eq!(fee_result.processing_fee, 71994700); + // estimated_size v1 adds the contract-version stamp varint to the + // worst-case document size + assert_eq!(fee_result.processing_fee, 72064200); } #[test] diff --git a/packages/rs-drive/src/drive/document/insert/mod.rs b/packages/rs-drive/src/drive/document/insert/mod.rs index b02044b4781..fbaa278f2e5 100644 --- a/packages/rs-drive/src/drive/document/insert/mod.rs +++ b/packages/rs-drive/src/drive/document/insert/mod.rs @@ -455,7 +455,9 @@ mod tests { &EPOCH_CHANGE_FEE_VERSION_TEST, StorageDiskUsageCreditPerByte, ), - processing_fee: 73253660, + // estimated_size v1 adds the contract-version stamp varint to + // the worst-case document size + processing_fee: 73323060, ..Default::default() }; diff --git a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_create_transition_action/mod.rs b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_create_transition_action/mod.rs index e529985db76..5584aec2faf 100644 --- a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_create_transition_action/mod.rs +++ b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_create_transition_action/mod.rs @@ -1,6 +1,7 @@ /// transformer pub mod transformer; mod v0; +mod v1; use derive_more::From; @@ -14,6 +15,7 @@ use dpp::fee::Credits; use dpp::ProtocolError; pub use v0::*; +pub use v1::*; use crate::state_transition_action::batch::batched_transition::document_transition::document_base_transition_action::{DocumentBaseTransitionAction}; use dpp::version::PlatformVersion; use dpp::voting::vote_info_storage::contested_document_vote_poll_stored_info::ContestedDocumentVotePollStoredInfo; @@ -150,7 +152,21 @@ impl DocumentFromCreateTransitionAction for Document { ) -> Result { match document_create_transition_action { DocumentCreateTransitionAction::V0(v0) => { - Self::try_from_create_transition_action_v0(v0, owner_id, platform_version) + match platform_version + .drive + .methods + .state_transitions + .document_from_action + .document_from_create_transition_action + { + 0 => Self::try_from_create_transition_action_v0(v0, owner_id, platform_version), + 1 => Self::try_from_create_transition_action_v1(v0, owner_id, platform_version), + version => Err(ProtocolError::UnknownVersionMismatch { + method: "Document::try_from_create_transition_action".to_string(), + known_versions: vec![0, 1], + received: version, + }), + } } } } @@ -162,7 +178,29 @@ impl DocumentFromCreateTransitionAction for Document { ) -> Result { match document_create_transition_action { DocumentCreateTransitionAction::V0(v0) => { - Self::try_from_owned_create_transition_action_v0(v0, owner_id, platform_version) + match platform_version + .drive + .methods + .state_transitions + .document_from_action + .document_from_create_transition_action + { + 0 => Self::try_from_owned_create_transition_action_v0( + v0, + owner_id, + platform_version, + ), + 1 => Self::try_from_owned_create_transition_action_v1( + v0, + owner_id, + platform_version, + ), + version => Err(ProtocolError::UnknownVersionMismatch { + method: "Document::try_from_owned_create_transition_action".to_string(), + known_versions: vec![0, 1], + received: version, + }), + } } } } diff --git a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_create_transition_action/v0/mod.rs b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_create_transition_action/v0/mod.rs index 0170963c753..73193343490 100644 --- a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_create_transition_action/v0/mod.rs +++ b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_create_transition_action/v0/mod.rs @@ -164,23 +164,6 @@ impl DocumentFromCreateTransitionActionV0 for Document { None }; - // The contract-version stamp exists exactly when document - // serialization format 3 (the format that writes it) is in - // effect; earlier formats have no stamp on the wire, so - // building one into the struct would only desync in-memory - // documents from their deserialized counterparts on replay - let contract_version = if platform_version - .dpp - .document_versions - .document_serialization_version - .default_current_version - >= 3 - { - Some(data_contract.contract.version()) - } else { - None - }; - let is_created_at_required = required_fields.contains(CREATED_AT); let is_updated_at_required = required_fields.contains(UPDATED_AT); let is_transferred_at_required = required_fields.contains(TRANSFERRED_AT); @@ -205,7 +188,7 @@ impl DocumentFromCreateTransitionActionV0 for Document { .document_structure_version { 0 => Ok(DocumentV0 { - contract_version, + contract_version: None, id, owner_id, properties: data, @@ -314,21 +297,6 @@ impl DocumentFromCreateTransitionActionV0 for Document { None }; - // The contract-version stamp exists exactly when document - // serialization format 3 (the format that writes it) is in - // effect - let contract_version = if platform_version - .dpp - .document_versions - .document_serialization_version - .default_current_version - >= 3 - { - Some(data_contract.contract.version()) - } else { - None - }; - let is_created_at_required = required_fields.contains(CREATED_AT); let is_updated_at_required = required_fields.contains(UPDATED_AT); let is_transferred_at_required = required_fields.contains(TRANSFERRED_AT); @@ -353,7 +321,7 @@ impl DocumentFromCreateTransitionActionV0 for Document { .document_structure_version { 0 => Ok(DocumentV0 { - contract_version, + contract_version: None, id: *id, owner_id, properties: data, diff --git a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_create_transition_action/v1/mod.rs b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_create_transition_action/v1/mod.rs new file mode 100644 index 00000000000..d15e0792fad --- /dev/null +++ b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_create_transition_action/v1/mod.rs @@ -0,0 +1,70 @@ +//! Generation 1 of the create-action → `Document` conversion: generation 0 +//! plus the contract-version stamp. The built document records the version +//! of the contract it was validated against, which selects each +//! `requiredSince` property's byte layout in document serialization format +//! 3. This generation must only be selected by platform versions whose +//! document serialization format writes the stamp (format 3, protocol +//! v14+); the version table pairs the two. + +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::document::{Document, DocumentV0Setters}; +use dpp::platform_value::Identifier; +use dpp::version::PlatformVersion; +use dpp::ProtocolError; + +use crate::state_transition_action::batch::batched_transition::document_transition::document_base_transition_action::{DocumentBaseTransitionAction, DocumentBaseTransitionActionAccessorsV0}; +use super::{DocumentCreateTransitionActionV0, DocumentFromCreateTransitionActionV0}; + +/// documents from create transition v1 +pub trait DocumentFromCreateTransitionActionV1 { + /// Attempts to create a new `Document` from the given `DocumentCreateTransitionActionV0` + /// instance and `owner_id`, stamped with the contract version the + /// document was created against. + fn try_from_owned_create_transition_action_v1( + v0: DocumentCreateTransitionActionV0, + owner_id: Identifier, + platform_version: &PlatformVersion, + ) -> Result + where + Self: Sized; + /// Attempts to create a new `Document` from the given `DocumentCreateTransitionActionV0` + /// reference and `owner_id`, stamped with the contract version the + /// document was created against. + fn try_from_create_transition_action_v1( + v0: &DocumentCreateTransitionActionV0, + owner_id: Identifier, + platform_version: &PlatformVersion, + ) -> Result + where + Self: Sized; +} + +fn action_contract_version(base: &DocumentBaseTransitionAction) -> u32 { + base.data_contract_fetch_info_ref().contract.version() +} + +impl DocumentFromCreateTransitionActionV1 for Document { + fn try_from_owned_create_transition_action_v1( + v0: DocumentCreateTransitionActionV0, + owner_id: Identifier, + platform_version: &PlatformVersion, + ) -> Result { + let contract_version = action_contract_version(&v0.base); + let mut document = + Self::try_from_owned_create_transition_action_v0(v0, owner_id, platform_version)?; + document.set_contract_version(Some(contract_version)); + Ok(document) + } + + fn try_from_create_transition_action_v1( + v0: &DocumentCreateTransitionActionV0, + owner_id: Identifier, + platform_version: &PlatformVersion, + ) -> Result { + let contract_version = action_contract_version(&v0.base); + let mut document = + Self::try_from_create_transition_action_v0(v0, owner_id, platform_version)?; + document.set_contract_version(Some(contract_version)); + Ok(document) + } +} diff --git a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_replace_transition_action/mod.rs b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_replace_transition_action/mod.rs index 136c1f9e6dc..21836bef237 100644 --- a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_replace_transition_action/mod.rs +++ b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_replace_transition_action/mod.rs @@ -1,4 +1,5 @@ mod v0; +mod v1; use std::collections::{BTreeMap, BTreeSet}; @@ -10,6 +11,7 @@ use dpp::platform_value::{Identifier, Value}; use dpp::prelude::{BlockHeight, CoreBlockHeight, Revision}; use dpp::ProtocolError; pub use v0::*; +pub use v1::*; use crate::state_transition_action::batch::batched_transition::document_transition::document_base_transition_action::DocumentBaseTransitionAction; use dpp::version::PlatformVersion; @@ -172,7 +174,25 @@ impl DocumentFromReplaceTransitionAction for Document { ) -> Result { match document_replace_transition_action { DocumentReplaceTransitionAction::V0(v0) => { - Self::try_from_replace_transition_action_v0(v0, owner_id, platform_version) + match platform_version + .drive + .methods + .state_transitions + .document_from_action + .document_from_replace_transition_action + { + 0 => { + Self::try_from_replace_transition_action_v0(v0, owner_id, platform_version) + } + 1 => { + Self::try_from_replace_transition_action_v1(v0, owner_id, platform_version) + } + version => Err(ProtocolError::UnknownVersionMismatch { + method: "Document::try_from_replace_transition_action".to_string(), + known_versions: vec![0, 1], + received: version, + }), + } } } } @@ -184,7 +204,29 @@ impl DocumentFromReplaceTransitionAction for Document { ) -> Result { match document_replace_transition_action { DocumentReplaceTransitionAction::V0(v0) => { - Self::try_from_owned_replace_transition_action_v0(v0, owner_id, platform_version) + match platform_version + .drive + .methods + .state_transitions + .document_from_action + .document_from_replace_transition_action + { + 0 => Self::try_from_owned_replace_transition_action_v0( + v0, + owner_id, + platform_version, + ), + 1 => Self::try_from_owned_replace_transition_action_v1( + v0, + owner_id, + platform_version, + ), + version => Err(ProtocolError::UnknownVersionMismatch { + method: "Document::try_from_owned_replace_transition_action".to_string(), + known_versions: vec![0, 1], + received: version, + }), + } } } } diff --git a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_replace_transition_action/v0/mod.rs b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_replace_transition_action/v0/mod.rs index 336c3d091d1..ac82cd6d4f7 100644 --- a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_replace_transition_action/v0/mod.rs +++ b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_replace_transition_action/v0/mod.rs @@ -1,6 +1,5 @@ pub mod transformer; -use dpp::data_contract::accessors::v0::DataContractV0Getters; use dpp::document::{Document, DocumentV0}; use dpp::identity::TimestampMillis; use dpp::platform_value::{Identifier, Value}; @@ -152,28 +151,13 @@ impl DocumentFromReplaceTransitionActionV0 for Document { let id = base.id(); - // A replace re-supplies the full document contents, so the document - // is re-stamped with the current contract version (the stamp exists - // exactly when document serialization format 3 is in effect) - let contract_version = if platform_version - .dpp - .document_versions - .document_serialization_version - .default_current_version - >= 3 - { - Some(base.data_contract_fetch_info_ref().contract.version()) - } else { - None - }; - match platform_version .dpp .document_versions .document_structure_version { 0 => Ok(DocumentV0 { - contract_version, + contract_version: None, id, owner_id, properties: data.clone(), @@ -222,28 +206,13 @@ impl DocumentFromReplaceTransitionActionV0 for Document { let id = base.id(); - // A replace re-supplies the full document contents, so the document - // is re-stamped with the current contract version (the stamp exists - // exactly when document serialization format 3 is in effect) - let contract_version = if platform_version - .dpp - .document_versions - .document_serialization_version - .default_current_version - >= 3 - { - Some(base.data_contract_fetch_info_ref().contract.version()) - } else { - None - }; - match platform_version .dpp .document_versions .document_structure_version { 0 => Ok(DocumentV0 { - contract_version, + contract_version: None, id, owner_id, properties: data, diff --git a/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_replace_transition_action/v1/mod.rs b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_replace_transition_action/v1/mod.rs new file mode 100644 index 00000000000..9b1b631e1fc --- /dev/null +++ b/packages/rs-drive/src/state_transition_action/batch/batched_transition/document_transition/document_replace_transition_action/v1/mod.rs @@ -0,0 +1,63 @@ +//! Generation 1 of the replace-action → `Document` conversion: generation 0 +//! plus the contract-version stamp. A replace re-supplies the full document +//! contents, so the document is re-stamped with the current contract +//! version. This generation must only be selected by platform versions +//! whose document serialization format writes the stamp (format 3, protocol +//! v14+); the version table pairs the two. + +use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::document::{Document, DocumentV0Setters}; +use dpp::platform_value::Identifier; +use dpp::version::PlatformVersion; +use dpp::ProtocolError; + +use crate::state_transition_action::batch::batched_transition::document_transition::document_base_transition_action::DocumentBaseTransitionActionAccessorsV0; +use super::{DocumentFromReplaceTransitionActionV0, DocumentReplaceTransitionActionV0}; + +/// document from replace transition v1 +pub trait DocumentFromReplaceTransitionActionV1 { + /// Attempts to create a new `Document` from the given `DocumentReplaceTransitionAction` + /// reference and `owner_id`, re-stamped with the current contract version. + fn try_from_replace_transition_action_v1( + value: &DocumentReplaceTransitionActionV0, + owner_id: Identifier, + platform_version: &PlatformVersion, + ) -> Result + where + Self: Sized; + /// Attempts to create a new `Document` from the given `DocumentReplaceTransitionAction` + /// instance and `owner_id`, re-stamped with the current contract version. + fn try_from_owned_replace_transition_action_v1( + value: DocumentReplaceTransitionActionV0, + owner_id: Identifier, + platform_version: &PlatformVersion, + ) -> Result + where + Self: Sized; +} + +impl DocumentFromReplaceTransitionActionV1 for Document { + fn try_from_replace_transition_action_v1( + value: &DocumentReplaceTransitionActionV0, + owner_id: Identifier, + platform_version: &PlatformVersion, + ) -> Result { + let contract_version = value.base.data_contract_fetch_info_ref().contract.version(); + let mut document = + Self::try_from_replace_transition_action_v0(value, owner_id, platform_version)?; + document.set_contract_version(Some(contract_version)); + Ok(document) + } + + fn try_from_owned_replace_transition_action_v1( + value: DocumentReplaceTransitionActionV0, + owner_id: Identifier, + platform_version: &PlatformVersion, + ) -> Result { + let contract_version = value.base.data_contract_fetch_info_ref().contract.version(); + let mut document = + Self::try_from_owned_replace_transition_action_v0(value, owner_id, platform_version)?; + document.set_contract_version(Some(contract_version)); + Ok(document) + } +} diff --git a/packages/rs-drive/tests/query_tests.rs b/packages/rs-drive/tests/query_tests.rs index bdcd8ab5576..5393c6e4f71 100644 --- a/packages/rs-drive/tests/query_tests.rs +++ b/packages/rs-drive/tests/query_tests.rs @@ -6812,8 +6812,8 @@ mod tests { .expect("there is always a root hash"); let expected_app_hash = vec![ - 237, 198, 157, 236, 20, 182, 87, 85, 216, 64, 84, 25, 163, 231, 107, 173, 155, 152, 34, - 64, 34, 142, 234, 16, 99, 134, 153, 156, 24, 208, 150, 115, + 224, 113, 139, 209, 16, 163, 116, 151, 213, 152, 169, 13, 158, 228, 31, 124, 88, 139, + 165, 2, 152, 27, 85, 54, 21, 40, 183, 80, 104, 140, 198, 119, ]; assert_eq!(root_hash.as_slice(), expected_app_hash); @@ -6893,8 +6893,8 @@ mod tests { .expect("there is always a root hash"); let expected_app_hash = vec![ - 237, 198, 157, 236, 20, 182, 87, 85, 216, 64, 84, 25, 163, 231, 107, 173, 155, 152, 34, - 64, 34, 142, 234, 16, 99, 134, 153, 156, 24, 208, 150, 115, + 224, 113, 139, 209, 16, 163, 116, 151, 213, 152, 169, 13, 158, 228, 31, 124, 88, 139, + 165, 2, 152, 27, 85, 54, 21, 40, 183, 80, 104, 140, 198, 119, ]; assert_eq!(root_hash.as_slice(), expected_app_hash); @@ -6995,8 +6995,8 @@ mod tests { .expect("there is always a root hash"); let expected_app_hash = vec![ - 237, 198, 157, 236, 20, 182, 87, 85, 216, 64, 84, 25, 163, 231, 107, 173, 155, 152, 34, - 64, 34, 142, 234, 16, 99, 134, 153, 156, 24, 208, 150, 115, + 224, 113, 139, 209, 16, 163, 116, 151, 213, 152, 169, 13, 158, 228, 31, 124, 88, 139, + 165, 2, 152, 27, 85, 54, 21, 40, 183, 80, 104, 140, 198, 119, ]; assert_eq!(root_hash.as_slice(), expected_app_hash); diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v6.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v6.rs index 5c4dbcd8f2f..546494f144d 100644 --- a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v6.rs +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v6.rs @@ -82,7 +82,7 @@ pub const CONTRACT_VERSIONS_V6: DPPContractVersions = DPPContractVersions { create_document_with_prevalidated_properties: 0, prefunded_voting_balance_for_document: 0, contested_vote_poll_for_document: 0, - estimated_size: 0, + estimated_size: 1, // changed: adds the document serialization format 3 contract-version stamp varint (worst case 5 bytes) to the estimate index_for_types: 0, max_size: 0, serialize_value_for_key: 0, diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/mod.rs b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/mod.rs index 31387c4a28d..b2b4f08c977 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/mod.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/mod.rs @@ -1,6 +1,7 @@ pub mod v1; pub mod v2; pub mod v3; +pub mod v4; use crate::version::drive_versions::DriveDataContractOperationMethodVersions; use versioned_feature_core::FeatureVersion; @@ -10,6 +11,18 @@ pub struct DriveStateTransitionMethodVersions { pub operations: DriveStateTransitionOperationMethodVersions, pub convert_to_high_level_operations: DriveStateTransitionActionConvertToHighLevelOperationsMethodVersions, + pub document_from_action: DriveDocumentFromActionVersions, +} + +/// Versions of the action → `Document` conversions. Generation 1 stamps the +/// built document with the data contract version its bytes conform to +/// (create assigns, replace re-assigns); it must only be selected by +/// platform versions whose document serialization format writes the stamp +/// (format 3, protocol v14+). +#[derive(Clone, Debug, Default)] +pub struct DriveDocumentFromActionVersions { + pub document_from_create_transition_action: FeatureVersion, + pub document_from_replace_transition_action: FeatureVersion, } #[derive(Clone, Debug, Default)] diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v1.rs b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v1.rs index 204d3058747..aaebe7aa3f7 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v1.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v1.rs @@ -1,4 +1,5 @@ use crate::version::drive_versions::drive_state_transition_method_versions::{ + DriveDocumentFromActionVersions, DriveStateTransitionActionConvertToHighLevelOperationsMethodVersions, DriveStateTransitionMethodVersions, DriveStateTransitionOperationMethodVersions, }; @@ -56,4 +57,8 @@ pub const DRIVE_STATE_TRANSITION_METHOD_VERSIONS_V1: DriveStateTransitionMethodV shielded_withdrawal_transition: 0, identity_create_from_shielded_pool_transition: 0, }, + document_from_action: DriveDocumentFromActionVersions { + document_from_create_transition_action: 0, + document_from_replace_transition_action: 0, + }, }; diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v2.rs b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v2.rs index babf9bc6fd1..48e4c7d8f8e 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v2.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v2.rs @@ -1,4 +1,5 @@ use crate::version::drive_versions::drive_state_transition_method_versions::{ + DriveDocumentFromActionVersions, DriveStateTransitionActionConvertToHighLevelOperationsMethodVersions, DriveStateTransitionMethodVersions, DriveStateTransitionOperationMethodVersions, }; @@ -57,4 +58,8 @@ pub const DRIVE_STATE_TRANSITION_METHOD_VERSIONS_V2: DriveStateTransitionMethodV shielded_withdrawal_transition: 0, identity_create_from_shielded_pool_transition: 0, }, + document_from_action: DriveDocumentFromActionVersions { + document_from_create_transition_action: 0, + document_from_replace_transition_action: 0, + }, }; diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v3.rs b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v3.rs index 5993c538df9..801fdaf5437 100644 --- a/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v3.rs +++ b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v3.rs @@ -1,4 +1,5 @@ use crate::version::drive_versions::drive_state_transition_method_versions::{ + DriveDocumentFromActionVersions, DriveStateTransitionActionConvertToHighLevelOperationsMethodVersions, DriveStateTransitionMethodVersions, DriveStateTransitionOperationMethodVersions, }; @@ -61,4 +62,8 @@ pub const DRIVE_STATE_TRANSITION_METHOD_VERSIONS_V3: DriveStateTransitionMethodV shielded_withdrawal_transition: 0, identity_create_from_shielded_pool_transition: 0, }, + document_from_action: DriveDocumentFromActionVersions { + document_from_create_transition_action: 0, + document_from_replace_transition_action: 0, + }, }; diff --git a/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v4.rs b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v4.rs new file mode 100644 index 00000000000..466d006a06c --- /dev/null +++ b/packages/rs-platform-version/src/version/drive_versions/drive_state_transition_method_versions/v4.rs @@ -0,0 +1,69 @@ +use crate::version::drive_versions::drive_state_transition_method_versions::{ + DriveDocumentFromActionVersions, + DriveStateTransitionActionConvertToHighLevelOperationsMethodVersions, + DriveStateTransitionMethodVersions, DriveStateTransitionOperationMethodVersions, +}; +use crate::version::drive_versions::DriveDataContractOperationMethodVersions; + +// This started at protocol 14: document_from_action generation 1 stamps built documents with the contract version (paired with document serialization format 3) +pub const DRIVE_STATE_TRANSITION_METHOD_VERSIONS_V4: DriveStateTransitionMethodVersions = + DriveStateTransitionMethodVersions { + operations: DriveStateTransitionOperationMethodVersions { + finalization_tasks: 0, + contracts: DriveDataContractOperationMethodVersions { + finalization_tasks: 0, + }, + }, + convert_to_high_level_operations: + DriveStateTransitionActionConvertToHighLevelOperationsMethodVersions { + data_contract_create_transition: 0, + data_contract_update_transition: 0, + document_create_transition: 0, + document_delete_transition: 0, + // PROTOCOL_VERSION_13: v1 rewrites a transferred or purchased + // DPNS domain document's `records.identity` to the new owner + // so the username resolves to the buyer. v0 stays for + // PROTOCOL_VERSION_12 chain replay. + document_purchase_transition: 1, // changed + document_replace_transition: 0, + document_transfer_transition: 1, // changed + document_update_price_transition: 1, // changed + token_burn_transition: 0, + token_mint_transition: 0, + token_transfer_transition: 0, + documents_batch_transition: 0, + identity_create_transition: 0, + identity_create_from_addresses_transition: 0, + identity_credit_transfer_transition: 0, + identity_credit_withdrawal_transition: 0, + identity_top_up_transition: 0, + identity_top_up_from_addresses_transition: 0, + identity_update_transition: 1, + masternode_vote_transition: 0, + bump_identity_data_contract_nonce: 0, + bump_identity_nonce: 0, + partially_use_asset_lock: 0, + token_freeze_transition: 0, + token_unfreeze_transition: 0, + token_emergency_action_transition: 0, + token_destroy_frozen_funds_transition: 0, + token_config_update_transition: 0, + token_claim_transition: 0, + token_direct_purchase_transition: 0, + token_set_price_for_direct_purchase_transition: 0, + identity_credit_transfer_to_addresses_transition: 0, + address_funds_transfer_transition: 0, + address_credit_withdrawal_transition: 0, + address_funding_from_asset_lock_transition: 0, + shield_transition: 0, + shield_from_asset_lock_transition: 0, + shielded_transfer_transition: 0, + unshield_transition: 0, + shielded_withdrawal_transition: 0, + identity_create_from_shielded_pool_transition: 0, + }, + document_from_action: DriveDocumentFromActionVersions { + document_from_create_transition_action: 1, // changed + document_from_replace_transition_action: 1, // changed + }, + }; diff --git a/packages/rs-platform-version/src/version/drive_versions/v9.rs b/packages/rs-platform-version/src/version/drive_versions/v9.rs index 54f4e357c57..fade08c521d 100644 --- a/packages/rs-platform-version/src/version/drive_versions/v9.rs +++ b/packages/rs-platform-version/src/version/drive_versions/v9.rs @@ -6,7 +6,7 @@ use crate::version::drive_versions::drive_group_method_versions::v1::DRIVE_GROUP use crate::version::drive_versions::drive_group_method_versions::DriveShieldedMethodVersions; use crate::version::drive_versions::drive_grove_method_versions::v1::DRIVE_GROVE_METHOD_VERSIONS_V1; use crate::version::drive_versions::drive_identity_method_versions::v2::DRIVE_IDENTITY_METHOD_VERSIONS_V2; -use crate::version::drive_versions::drive_state_transition_method_versions::v3::DRIVE_STATE_TRANSITION_METHOD_VERSIONS_V3; +use crate::version::drive_versions::drive_state_transition_method_versions::v4::DRIVE_STATE_TRANSITION_METHOD_VERSIONS_V4; use crate::version::drive_versions::drive_structure_version::v1::DRIVE_STRUCTURE_V1; use crate::version::drive_versions::drive_token_method_versions::v1::DRIVE_TOKEN_METHOD_VERSIONS_V1; use crate::version::drive_versions::drive_verify_method_versions::v2::DRIVE_VERIFY_METHOD_VERSIONS_V2; @@ -98,7 +98,7 @@ pub const DRIVE_VERSION_V9: DriveVersion = DriveVersion { apply_batch_low_level_drive_operations: 0, apply_batch_grovedb_operations: 0, }, - state_transitions: DRIVE_STATE_TRANSITION_METHOD_VERSIONS_V3, // changed in v8: DPNS domain records.identity rewrite on transfer/purchase + state_transitions: DRIVE_STATE_TRANSITION_METHOD_VERSIONS_V4, // changed: document_from_action generation 1 stamps built documents with the contract version (create assigns, replace re-assigns; paired with document serialization format 3) batch_operations: DriveBatchOperationsMethodVersion { convert_drive_operations_to_grove_operations: 0, apply_drive_operations: 0, From c57720d2e929af13c0e5fcf2a48a09853873b23b Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Fri, 14 Aug 2026 01:08:57 +0700 Subject: [PATCH 03/23] test(drive-abci): update PV14 fee baselines for contract-version stamp Documents written at protocol v14 carry the contract-version stamp (one stored byte, five in worst-case estimation), which shifts byte-billed processing fees. Updates the latest-version baselines for document delete/replace/transfer and the token tests whose genesis system documents are now stamped; prior-version pins are untouched. Co-Authored-By: Claude Fable 5 --- .../state_transitions/batch/tests/document/deletion.rs | 4 +++- .../state_transitions/batch/tests/document/replacement.rs | 7 ++++--- .../state_transitions/batch/tests/document/transfer.rs | 5 +++-- .../state_transitions/batch/tests/token/burn/mod.rs | 4 +++- .../batch/tests/token/direct_selling/mod.rs | 5 ++++- 5 files changed, 17 insertions(+), 8 deletions(-) diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/deletion.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/deletion.rs index f69dfbaf868..3272136bbd2 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/deletion.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/deletion.rs @@ -10,7 +10,9 @@ mod deletion_tests { async fn test_document_delete_on_document_type_that_is_mutable_and_can_be_deleted() { run_document_delete_on_document_type_that_is_mutable_and_can_be_deleted_at_protocol_version( PlatformVersion::latest().protocol_version, - 1699160, + // v14: the deleted document carries the contract-version stamp + // (one stored byte, five estimated), shifting processing costs + 1699620, ) .await; } diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/replacement.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/replacement.rs index 9cf7dacdab0..5b6161d1b2b 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/replacement.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/replacement.rs @@ -333,7 +333,8 @@ mod replacement_tests { async fn test_document_replace_on_document_type_that_is_mutable() { run_document_replace_on_document_type_that_is_mutable_at_protocol_version( PlatformVersion::latest().protocol_version, - 1432760, + // v14: replaced documents carry the contract-version stamp + 1433220, ) .await; } @@ -1037,7 +1038,7 @@ mod replacement_tests { async fn test_document_replace_on_document_type_that_is_not_mutable() { run_document_replace_on_document_type_that_is_not_mutable_at_protocol_version( PlatformVersion::latest().protocol_version, - 460920, + 460940, // v14: stamped documents (see happy-path baseline note) ) .await; } @@ -1293,7 +1294,7 @@ mod replacement_tests { async fn test_document_replace_on_document_type_that_is_not_mutable_but_is_transferable() { run_document_replace_on_document_type_that_is_not_mutable_but_is_transferable_at_protocol_version( PlatformVersion::latest().protocol_version, - 457660, + 457680, // v14: stamped documents (see happy-path baseline note) ) .await; } diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/transfer.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/transfer.rs index cd03d32a7c5..cc761bad415 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/transfer.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/transfer.rs @@ -498,7 +498,8 @@ mod transfer_tests { async fn test_document_transfer_on_document_type_that_is_transferable() { run_document_transfer_on_document_type_that_is_transferable_at_protocol_version( PlatformVersion::latest().protocol_version, - 3643400, + // v14: transferred documents carry the contract-version stamp + 3643860, ) .await; } @@ -1478,7 +1479,7 @@ mod transfer_tests { async fn test_document_delete_after_transfer() { run_document_delete_after_transfer_at_protocol_version( PlatformVersion::latest().protocol_version, - 4004260, + 4004720, // v14: stamped documents (see transferable baseline note) ) .await; } diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/token/burn/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/token/burn/mod.rs index 5fba3b44ec7..1068a7ad348 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/token/burn/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/token/burn/mod.rs @@ -3958,7 +3958,9 @@ mod token_burn_tests { // sizes and therefore the byte-billed group-action contract reads. run_token_burn_group_action_confirmer_fee_includes_transformer_reads_at_protocol_version( PlatformVersion::latest().protocol_version, - 4_367_880, + // PROTOCOL_VERSION_14: +400 — genesis system documents now carry + // the contract-version stamp, shifting byte-billed subtree reads + 4_368_280, ) .await; } diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/token/direct_selling/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/token/direct_selling/mod.rs index 2fe0cb1727c..3fdde12cd7c 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/token/direct_selling/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/token/direct_selling/mod.rs @@ -24,7 +24,10 @@ mod token_selling_tests { // sizes and therefore the byte-billed contract reads. run_successful_direct_purchase_single_price_at_protocol_version( PlatformVersion::latest().protocol_version, - 699_868_073_580, + // PROTOCOL_VERSION_14: 27_400 credits more in fees — genesis system + // documents now carry the contract-version stamp, shifting + // byte-billed subtree reads + 699_868_046_180, ) .await; } From 72278c651cc815e8e321974d38e1425714106109 Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Fri, 14 Aug 2026 01:21:57 +0700 Subject: [PATCH 04/23] test(dpp): cover document serialization format 3 across all property types Round-trips every schema-reachable property type (all integer widths, f64, string, byteArray, identifier, boolean) through serialize_v3 / from_bytes_v3 in required, optional-present, and optional-absent positions, asserts byte determinism, and sweeps every truncated prefix of the serialized form through from_bytes to exercise the reader's error arms. u128/i128 have no schema-reachable serializer arm (integer bounds are i64-limited), so they stay uncovered by design. Co-Authored-By: Claude Fable 5 --- packages/rs-dpp/src/document/v0/serialize.rs | 163 +++++++++++++++++++ 1 file changed, 163 insertions(+) diff --git a/packages/rs-dpp/src/document/v0/serialize.rs b/packages/rs-dpp/src/document/v0/serialize.rs index 404022c1a5c..840e8733829 100644 --- a/packages/rs-dpp/src/document/v0/serialize.rs +++ b/packages/rs-dpp/src/document/v0/serialize.rs @@ -3153,6 +3153,169 @@ mod tests { /// - `a`: required at every version /// - `b`: required since contract version 2 /// - `c`: plain optional + /// A document type exercising every schema-reachable property type in + /// both required and optional positions (u128/i128 are not inferable + /// from i64-bounded schemas, so they have no reachable serializer arm). + fn kitchen_sink_document_type() -> crate::data_contract::document_type::DocumentType { + use crate::data_contract::config::DataContractConfig; + use crate::data_contract::document_type::DocumentType; + use platform_value::platform_value; + use std::collections::BTreeMap; + + let platform_version = PlatformVersion::latest(); + let schema = platform_value!({ + "type": "object", + "properties": { + "u8v": {"type": "integer", "position": 0, "minimum": 0, "maximum": 255}, + "u16v": {"type": "integer", "position": 1, "minimum": 0, "maximum": 65535}, + "u32v": {"type": "integer", "position": 2, "minimum": 0, "maximum": 4294967295_u64}, + "i8v": {"type": "integer", "position": 3, "minimum": -128, "maximum": 127}, + "i16v": {"type": "integer", "position": 4, "minimum": -32768, "maximum": 32767}, + "i32v": {"type": "integer", "position": 5, "minimum": -2147483648_i64, "maximum": 2147483647_i64}, + "i64v": {"type": "integer", "position": 6}, + "f64v": {"type": "number", "position": 7}, + "strv": {"type": "string", "position": 8, "maxLength": 60_u32}, + "bytv": {"type": "array", "position": 9, "byteArray": true, "minItems": 0, "maxItems": 32}, + "idv": {"type": "array", "position": 10, "byteArray": true, "minItems": 32, "maxItems": 32, "contentMediaType": "application/x.dash.dpp.identifier"}, + "boolv": {"type": "boolean", "position": 11}, + "u8o": {"type": "integer", "position": 12, "minimum": 0, "maximum": 255}, + "u16o": {"type": "integer", "position": 13, "minimum": 0, "maximum": 65535}, + "u32o": {"type": "integer", "position": 14, "minimum": 0, "maximum": 4294967295_u64}, + "i8o": {"type": "integer", "position": 15, "minimum": -128, "maximum": 127}, + "i16o": {"type": "integer", "position": 16, "minimum": -32768, "maximum": 32767}, + "i32o": {"type": "integer", "position": 17, "minimum": -2147483648_i64, "maximum": 2147483647_i64}, + "i64o": {"type": "integer", "position": 18}, + "f64o": {"type": "number", "position": 19}, + "stro": {"type": "string", "position": 20, "maxLength": 60_u32}, + "byto": {"type": "array", "position": 21, "byteArray": true, "minItems": 0, "maxItems": 32}, + "ido": {"type": "array", "position": 22, "byteArray": true, "minItems": 32, "maxItems": 32, "contentMediaType": "application/x.dash.dpp.identifier"}, + "boolo": {"type": "boolean", "position": 23}, + }, + "required": ["u8v", "u16v", "u32v", "i8v", "i16v", "i32v", "i64v", "f64v", "strv", "bytv", "idv", "boolv"], + "additionalProperties": false, + }); + let config = DataContractConfig::default_for_version(platform_version) + .expect("should create a default config"); + DocumentType::try_from_schema( + platform_value::Identifier::new([2; 32]), + 1, + config.version(), + "sink", + schema, + None, + &BTreeMap::new(), + &config, + false, + &mut Vec::new(), + platform_version, + ) + .expect("failed to create kitchen-sink document type") + } + + fn kitchen_sink_required_properties() -> BTreeMap { + let mut properties = BTreeMap::new(); + properties.insert("u8v".to_string(), Value::U8(200)); + properties.insert("u16v".to_string(), Value::U16(60000)); + properties.insert("u32v".to_string(), Value::U32(4000000000)); + properties.insert("i8v".to_string(), Value::I8(-100)); + properties.insert("i16v".to_string(), Value::I16(-30000)); + properties.insert("i32v".to_string(), Value::I32(-2000000000)); + properties.insert("i64v".to_string(), Value::I64(-9000000000000000000)); + properties.insert("f64v".to_string(), Value::Float(1.5)); + properties.insert("strv".to_string(), Value::Text("hello".to_string())); + properties.insert("bytv".to_string(), Value::Bytes(vec![1, 2, 3])); + properties.insert("idv".to_string(), Value::Identifier([7; 32])); + properties.insert("boolv".to_string(), Value::Bool(true)); + properties + } + + #[test] + fn serialize_v3_round_trips_every_property_type() { + let platform_version = PlatformVersion::latest(); + let document_type = kitchen_sink_document_type(); + + // Every optional present alongside every required + let mut properties = kitchen_sink_required_properties(); + properties.insert("u8o".to_string(), Value::U8(1)); + properties.insert("u16o".to_string(), Value::U16(2)); + properties.insert("u32o".to_string(), Value::U32(3)); + properties.insert("i8o".to_string(), Value::I8(-1)); + properties.insert("i16o".to_string(), Value::I16(-2)); + properties.insert("i32o".to_string(), Value::I32(-3)); + properties.insert("i64o".to_string(), Value::I64(-4)); + properties.insert("f64o".to_string(), Value::Float(-2.75)); + properties.insert("stro".to_string(), Value::Text(String::new())); + properties.insert("byto".to_string(), Value::Bytes(Vec::new())); + properties.insert("ido".to_string(), Value::Identifier([9; 32])); + properties.insert("boolo".to_string(), Value::Bool(false)); + + let document = stamped_document(None, properties, document_type.as_ref()); + let serialized = document + .serialize_v3(document_type.as_ref()) + .expect("expected to serialize all property types"); + let deserialized = + DocumentV0::from_bytes(&serialized, document_type.as_ref(), platform_version) + .expect("expected to deserialize all property types"); + assert_eq!(document, deserialized); + + // Determinism: same document, same bytes + let serialized_again = document + .serialize_v3(document_type.as_ref()) + .expect("expected to serialize again"); + assert_eq!(serialized, serialized_again); + + // Every optional absent (the flag-0 arm of each type), stamped + let document = stamped_document( + Some(1), + kitchen_sink_required_properties(), + document_type.as_ref(), + ); + let serialized = document + .serialize_v3(document_type.as_ref()) + .expect("expected to serialize with absent optionals"); + let deserialized = + DocumentV0::from_bytes(&serialized, document_type.as_ref(), platform_version) + .expect("expected to deserialize with absent optionals"); + assert_eq!(document, deserialized); + } + + #[test] + fn serialize_v3_missing_plain_required_property_errors() { + let document_type = kitchen_sink_document_type(); + let mut properties = kitchen_sink_required_properties(); + properties.remove("u16v"); + + let document = stamped_document(None, properties, document_type.as_ref()); + assert!( + document.serialize_v3(document_type.as_ref()).is_err(), + "serializing without a required property must error" + ); + } + + #[test] + fn from_bytes_v3_never_panics_on_truncated_input() { + let platform_version = PlatformVersion::latest(); + let document_type = kitchen_sink_document_type(); + + let mut properties = kitchen_sink_required_properties(); + properties.insert("stro".to_string(), Value::Text("tail".to_string())); + let document = stamped_document(Some(1), properties, document_type.as_ref()); + let serialized = document + .serialize_v3(document_type.as_ref()) + .expect("expected to serialize"); + + // Every strict prefix must produce a Result, never a panic. (Some + // prefixes legitimately succeed: format 3 tolerates EOF at property + // boundaries so appended properties stay readable by old data.) + for length in 0..serialized.len() { + let _ = DocumentV0::from_bytes( + &serialized[..length], + document_type.as_ref(), + platform_version, + ); + } + } + fn required_since_document_type() -> crate::data_contract::document_type::DocumentType { use crate::data_contract::config::DataContractConfig; use crate::data_contract::document_type::DocumentType; From 5c255b8742809d1afeb9553255604e6e58eb1d25 Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Fri, 14 Aug 2026 01:28:25 +0700 Subject: [PATCH 05/23] docs(dpp): clarify which property types are not schema-reachable in format-3 test Co-Authored-By: Claude Fable 5 --- packages/rs-dpp/src/document/v0/serialize.rs | 7 +++++-- 1 file changed, 5 insertions(+), 2 deletions(-) diff --git a/packages/rs-dpp/src/document/v0/serialize.rs b/packages/rs-dpp/src/document/v0/serialize.rs index 840e8733829..287d97b92df 100644 --- a/packages/rs-dpp/src/document/v0/serialize.rs +++ b/packages/rs-dpp/src/document/v0/serialize.rs @@ -3154,8 +3154,11 @@ mod tests { /// - `b`: required since contract version 2 /// - `c`: plain optional /// A document type exercising every schema-reachable property type in - /// both required and optional positions (u128/i128 are not inferable - /// from i64-bounded schemas, so they have no reachable serializer arm). + /// both required and optional positions. Not represented because no + /// document schema can produce them (`try_from_value_map` dispatches on + /// `"type"` only): `Date` (no `"date"` arm; only array item types and + /// system fields use it via `try_from_name`) and u128/i128 (integer + /// bound inference is i64-limited). fn kitchen_sink_document_type() -> crate::data_contract::document_type::DocumentType { use crate::data_contract::config::DataContractConfig; use crate::data_contract::document_type::DocumentType; From b837928b2469a409b0552ec0ec9f7d11344a17ff Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Tue, 25 Aug 2026 14:43:49 +0200 Subject: [PATCH 06/23] fix(dpp): reattach stray fixture doc comment tripping clippy doc_lazy_continuation The a/b/c property list belonged to required_since_document_type() but sat on top of kitchen_sink_document_type()'s comment, which clippy 1.92 rejects as an unindented doc list continuation under -D warnings. Co-Authored-By: Claude Fable 5 --- packages/rs-dpp/src/document/v0/serialize.rs | 8 ++++---- 1 file changed, 4 insertions(+), 4 deletions(-) diff --git a/packages/rs-dpp/src/document/v0/serialize.rs b/packages/rs-dpp/src/document/v0/serialize.rs index 287d97b92df..ad5a14c52b6 100644 --- a/packages/rs-dpp/src/document/v0/serialize.rs +++ b/packages/rs-dpp/src/document/v0/serialize.rs @@ -3149,10 +3149,6 @@ mod tests { // Format 3: the contract-version stamp and requiredSince layouts // ================================================================ - /// A document type with: - /// - `a`: required at every version - /// - `b`: required since contract version 2 - /// - `c`: plain optional /// A document type exercising every schema-reachable property type in /// both required and optional positions. Not represented because no /// document schema can produce them (`try_from_value_map` dispatches on @@ -3319,6 +3315,10 @@ mod tests { } } + /// A document type with: + /// - `a`: required at every version + /// - `b`: required since contract version 2 + /// - `c`: plain optional fn required_since_document_type() -> crate::data_contract::document_type::DocumentType { use crate::data_contract::config::DataContractConfig; use crate::data_contract::document_type::DocumentType; From 74295458203a73409d478e2213159e12750c62e2 Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Tue, 25 Aug 2026 14:44:03 +0200 Subject: [PATCH 07/23] refactor(dpp): version the contract-level requiredSince update validation The requiredSince orchestration (feeding the new contract version into per-document-type validation and validating annotations on document types introduced by the update) was added directly to the shipped DataContract::validate_update generation 0, leaving its behavior dependent on the separately versioned schema parser. Move it to a new generation 1, selected only by CONTRACT_VERSIONS_V6 (protocol v14); generation 0 is restored byte-identical apart from the widened document-type dispatcher call, which generation 0 ignores. Co-Authored-By: Claude Fable 5 --- .../methods/validate_update/mod.rs | 4 +- .../methods/validate_update/v0/mod.rs | 122 +---- .../methods/validate_update/v1/mod.rs | 454 ++++++++++++++++++ .../dpp_versions/dpp_contract_versions/v6.rs | 7 +- 4 files changed, 468 insertions(+), 119 deletions(-) create mode 100644 packages/rs-dpp/src/data_contract/methods/validate_update/v1/mod.rs diff --git a/packages/rs-dpp/src/data_contract/methods/validate_update/mod.rs b/packages/rs-dpp/src/data_contract/methods/validate_update/mod.rs index 2a40b26459f..e817ba86cec 100644 --- a/packages/rs-dpp/src/data_contract/methods/validate_update/mod.rs +++ b/packages/rs-dpp/src/data_contract/methods/validate_update/mod.rs @@ -3,6 +3,7 @@ use crate::prelude::DataContract; use platform_version::version::PlatformVersion; mod v0; +mod v1; use crate::validation::SimpleConsensusValidationResult; use crate::ProtocolError; pub use v0::*; @@ -21,9 +22,10 @@ impl DataContractUpdateValidationMethodsV0 for DataContract { .validate_update { 0 => self.validate_update_v0(data_contract, block_info, platform_version), + 1 => self.validate_update_v1(data_contract, block_info, platform_version), version => Err(ProtocolError::UnknownVersionMismatch { method: "DataContract::validate_update".to_string(), - known_versions: vec![0], + known_versions: vec![0, 1], received: version, }), } diff --git a/packages/rs-dpp/src/data_contract/methods/validate_update/v0/mod.rs b/packages/rs-dpp/src/data_contract/methods/validate_update/v0/mod.rs index c7a38bb18d5..5d07ed0c5e9 100644 --- a/packages/rs-dpp/src/data_contract/methods/validate_update/v0/mod.rs +++ b/packages/rs-dpp/src/data_contract/methods/validate_update/v0/mod.rs @@ -6,8 +6,7 @@ use crate::consensus::state::token::PreProgrammedDistributionTimestampInPastErro use crate::data_contract::accessors::v0::DataContractV0Getters; use crate::consensus::basic::data_contract::{ - DataContractInvalidRequiredFieldsUpdateError, DuplicateKeywordsError, - IncompatibleDataContractSchemaError, InvalidDataContractVersionError, + DuplicateKeywordsError, IncompatibleDataContractSchemaError, InvalidDataContractVersionError, InvalidDescriptionLengthError, InvalidKeywordCharacterError, InvalidKeywordLengthError, TooManyKeywordsError, }; @@ -18,7 +17,6 @@ use crate::data_contract::accessors::v1::DataContractV1Getters; use crate::data_contract::associated_token::token_configuration::accessors::v0::TokenConfigurationV0Getters; use crate::data_contract::associated_token::token_distribution_rules::accessors::v0::TokenDistributionRulesV0Getters; use crate::data_contract::associated_token::token_pre_programmed_distribution::accessors::v0::TokenPreProgrammedDistributionV0Methods; -use crate::data_contract::document_type::accessors::DocumentTypeV0Getters; use crate::data_contract::document_type::schema::validate_schema_compatibility; use crate::data_contract::schema::DataContractSchemaMethodsV0; use crate::data_contract::DataContract; @@ -108,7 +106,10 @@ impl DataContract { )); }; - // Validate document type update rules + // Validate document type update rules. The document-type + // dispatcher takes the new contract version for generation 1; + // generation 0, the only one this method's platform versions + // select, ignores it. let validate_update_result = old_document_type.as_ref().validate_update( new_document_type, new_data_contract.version(), @@ -122,40 +123,6 @@ impl DataContract { } } - // Document types introduced by this update have no old counterpart, - // so the per-type update validation above never sees them. Their - // `requiredSince` annotations must name the version this update - // creates — anything else would pre-schedule (or backdate) a - // wire-layout change without validation. Replay safety: this loop is - // a no-op for every contract that predates the `requiredSince` - // keyword (protocol v14's meta-schema), because such contracts can - // carry no annotation — older meta-schemas rejected the keyword at - // write time and older parsers ignore it entirely. - for (document_type_name, new_document_type) in new_data_contract.document_types() { - if self - .document_type_optional_for_name(document_type_name) - .is_some() - { - continue; - } - for (property_name, property) in new_document_type.as_ref().properties() { - if let Some(required_since) = property.required_since { - if required_since != new_data_contract.version() { - return Ok(SimpleConsensusValidationResult::new_with_error( - DataContractInvalidRequiredFieldsUpdateError::new( - document_type_name.clone(), - format!( - "new document type property '{property_name}' must carry requiredSince {}, the contract version this update creates", - new_data_contract.version() - ), - ) - .into(), - )); - } - } - } - } - // Schema $defs should be compatible if let Some(old_defs_map) = self.schema_defs() { // If new contract doesn't have $defs, it means that it's $defs was removed and compatibility is broken @@ -402,85 +369,6 @@ mod tests { use crate::identity::accessors::IdentityGettersV0; use crate::prelude::Identity; - #[test] - fn should_validate_required_since_on_document_types_added_by_the_update() { - let platform_version = PlatformVersion::latest(); - - let old_data_contract = get_data_contract_fixture( - None, - IdentityNonce::default(), - platform_version.protocol_version, - ) - .data_contract_owned(); - - let new_type_schema = |required_since: u32| { - platform_value!({ - "type": "object", - "properties": { - "message": { - "type": "string", - "position": 0, - "maxLength": 60_u32, - "requiredSince": required_since, - } - }, - "required": ["message"], - "additionalProperties": false - }) - }; - - // A new document type pre-scheduling requiredness at version 99 - // has no old counterpart, so the per-type update validation - // never runs on it — this pass must catch it - let mut new_data_contract = old_data_contract.clone(); - new_data_contract.set_version(old_data_contract.version() + 1); - new_data_contract - .set_document_schema( - "note", - new_type_schema(99), - false, - &mut Vec::new(), - platform_version, - ) - .expect("should add document type"); - - let result = old_data_contract - .validate_update(&new_data_contract, &BlockInfo::default(), platform_version) - .expect("failed validate update"); - - assert_matches!( - result.errors.as_slice(), - [ConsensusError::BasicError( - BasicError::DataContractInvalidRequiredFieldsUpdateError(e) - )] if e.details().contains("must carry requiredSince 2") - ); - - // The same new document type annotated with the version this - // update creates is accepted - let mut new_data_contract = old_data_contract.clone(); - new_data_contract.set_version(old_data_contract.version() + 1); - new_data_contract - .set_document_schema( - "note", - new_type_schema(old_data_contract.version() + 1), - false, - &mut Vec::new(), - platform_version, - ) - .expect("should add document type"); - - let result = old_data_contract - .validate_update(&new_data_contract, &BlockInfo::default(), platform_version) - .expect("failed validate update"); - - assert!( - result.is_valid(), - "a new document type annotated with the version this update \ - creates must be accepted, got {:?}", - result.errors - ); - } - #[test] fn should_return_invalid_result_if_owner_id_is_not_the_same() { let platform_version = PlatformVersion::latest(); diff --git a/packages/rs-dpp/src/data_contract/methods/validate_update/v1/mod.rs b/packages/rs-dpp/src/data_contract/methods/validate_update/v1/mod.rs new file mode 100644 index 00000000000..9786fa3b95e --- /dev/null +++ b/packages/rs-dpp/src/data_contract/methods/validate_update/v1/mod.rs @@ -0,0 +1,454 @@ +use std::collections::HashSet; + +use crate::block::block_info::BlockInfo; +use crate::consensus::state::state_error::StateError; +use crate::consensus::state::token::PreProgrammedDistributionTimestampInPastError; +use crate::data_contract::accessors::v0::DataContractV0Getters; + +use crate::consensus::basic::data_contract::{ + DataContractInvalidRequiredFieldsUpdateError, DuplicateKeywordsError, + IncompatibleDataContractSchemaError, InvalidDataContractVersionError, + InvalidDescriptionLengthError, InvalidKeywordCharacterError, InvalidKeywordLengthError, + TooManyKeywordsError, +}; +use crate::consensus::state::data_contract::data_contract_update_action_not_allowed_error::DataContractUpdateActionNotAllowedError; +use crate::consensus::state::data_contract::data_contract_update_permission_error::DataContractUpdatePermissionError; +use crate::consensus::state::data_contract::document_type_update_error::DocumentTypeUpdateError; +use crate::data_contract::accessors::v1::DataContractV1Getters; +use crate::data_contract::associated_token::token_configuration::accessors::v0::TokenConfigurationV0Getters; +use crate::data_contract::associated_token::token_distribution_rules::accessors::v0::TokenDistributionRulesV0Getters; +use crate::data_contract::associated_token::token_pre_programmed_distribution::accessors::v0::TokenPreProgrammedDistributionV0Methods; +use crate::data_contract::document_type::accessors::DocumentTypeV0Getters; +use crate::data_contract::document_type::schema::validate_schema_compatibility; +use crate::data_contract::schema::DataContractSchemaMethodsV0; +use crate::data_contract::DataContract; +use crate::validation::SimpleConsensusValidationResult; +use crate::ProtocolError; +use platform_value::Value; +use platform_version::version::PlatformVersion; +use serde_json::json; + +impl DataContract { + /// Generation 1 (protocol version 14, `requiredSince`). Differences from + /// generation 0: + /// - the new contract version is passed into per-document-type update + /// validation, whose own generation 1 admits required-set additions + /// annotated with `requiredSince` equal to that version; + /// - document types introduced by the update — which have no old + /// counterpart for the per-type pass to see — get their `requiredSince` + /// annotations validated here: each must name exactly the version this + /// update creates. + #[inline(always)] + pub(super) fn validate_update_v1( + &self, + new_data_contract: &DataContract, + block_info: &BlockInfo, + platform_version: &PlatformVersion, + ) -> Result { + // Check if the contract is owned by the same identity + if self.owner_id() != new_data_contract.owner_id() { + return Ok(SimpleConsensusValidationResult::new_with_error( + DataContractUpdatePermissionError::new(self.id(), new_data_contract.owner_id()) + .into(), + )); + } + + // Check version is bumped + // Failure (version != previous version + 1): Keep ST and transform it to a nonce bump action. + // How: A user pushed an update that was not the next version. + + let new_version = new_data_contract.version(); + let old_version = self.version(); + if new_version < old_version || new_version - old_version != 1 { + return Ok(SimpleConsensusValidationResult::new_with_error( + InvalidDataContractVersionError::new(old_version + 1, new_version).into(), + )); + } + + // Validate that the config was not updated + // * Includes verifications that: + // - Old contract is not read_only + // - New contract is not read_only + // - Keeps history did not change + // - Can be deleted did not change + // - Documents keep history did not change + // - Documents can be deleted contract default did not change + // - Documents mutable contract default did not change + // - Requires identity encryption bounded key did not change + // - Requires identity decryption bounded key did not change + // * Failure (contract does not exist): Keep ST and transform it to a nonce bump action. + // * How: A user pushed an update to a contract that changed its configuration. + + let config_validation_result = self.config().validate_update( + new_data_contract.config(), + self.id(), + platform_version, + )?; + + if !config_validation_result.is_valid() { + return Ok(SimpleConsensusValidationResult::new_with_errors( + config_validation_result.errors, + )); + } + + // Validate updates for existing document types to make sure that previously created + // documents will be still valid with a new version of the data contract + for (document_type_name, old_document_type) in self.document_types() { + // Make sure that existing document aren't removed + let Some(new_document_type) = + new_data_contract.document_type_optional_for_name(document_type_name) + else { + return Ok(SimpleConsensusValidationResult::new_with_error( + DocumentTypeUpdateError::new( + self.id(), + document_type_name, + "document type can't be removed", + ) + .into(), + )); + }; + + // Validate document type update rules + let validate_update_result = old_document_type.as_ref().validate_update( + new_document_type, + new_data_contract.version(), + platform_version, + )?; + + if !validate_update_result.is_valid() { + return Ok(SimpleConsensusValidationResult::new_with_errors( + validate_update_result.errors, + )); + } + } + + // Document types introduced by this update have no old counterpart, + // so the per-type update validation above never sees them. Their + // `requiredSince` annotations must name the version this update + // creates — anything else would pre-schedule (or backdate) a + // wire-layout change without validation. + for (document_type_name, new_document_type) in new_data_contract.document_types() { + if self + .document_type_optional_for_name(document_type_name) + .is_some() + { + continue; + } + for (property_name, property) in new_document_type.as_ref().properties() { + if let Some(required_since) = property.required_since { + if required_since != new_data_contract.version() { + return Ok(SimpleConsensusValidationResult::new_with_error( + DataContractInvalidRequiredFieldsUpdateError::new( + document_type_name.clone(), + format!( + "new document type property '{property_name}' must carry requiredSince {}, the contract version this update creates", + new_data_contract.version() + ), + ) + .into(), + )); + } + } + } + } + + // Schema $defs should be compatible + if let Some(old_defs_map) = self.schema_defs() { + // If new contract doesn't have $defs, it means that it's $defs was removed and compatibility is broken + let Some(new_defs_map) = new_data_contract.schema_defs() else { + return Ok(SimpleConsensusValidationResult::new_with_error( + IncompatibleDataContractSchemaError::new( + self.id(), + "remove".to_string(), + "/$defs".to_string(), + ) + .into(), + )); + }; + + // If $defs is updated we need to make sure that our data contract is still compatible + // with previously created data + if old_defs_map != new_defs_map { + // both new and old $defs already validated as a part of new and old contract + let old_defs_json = Value::from(old_defs_map) + .try_into_validating_json() + .map_err(ProtocolError::ValueError)?; + + let new_defs_json = Value::from(new_defs_map) + .try_into_validating_json() + .map_err(ProtocolError::ValueError)?; + + let old_defs_schema = json!({ + "$defs": old_defs_json + }); + + let new_defs_schema = json!({ + "$defs": new_defs_json + }); + + // We do not allow to remove or modify $ref in document type schemas + // it means that compatible changes in $defs won't break the overall compatibility + // Make sure that updated $defs schema is compatible + let compatibility_validation_result = validate_schema_compatibility( + &old_defs_schema, + &new_defs_schema, + platform_version, + )?; + + if !compatibility_validation_result.is_valid() { + let errors = compatibility_validation_result + .errors + .into_iter() + .map(|operation| { + IncompatibleDataContractSchemaError::new( + self.id(), + operation.name, + operation.path, + ) + .into() + }) + .collect(); + + return Ok(SimpleConsensusValidationResult::new_with_errors(errors)); + } + } + } + + if self.groups() != new_data_contract.groups() { + // No groups can have been removed + for old_group_position in self.groups().keys() { + if !new_data_contract.groups().contains_key(old_group_position) { + return Ok(SimpleConsensusValidationResult::new_with_error( + DataContractUpdateActionNotAllowedError::new( + self.id(), + "remove group".to_string(), + ) + .into(), + )); + } + } + + // Ensure no group has been changed + for (old_group_position, old_group) in self.groups() { + if let Some(new_group) = new_data_contract.groups().get(old_group_position) { + if old_group != new_group { + return Ok(SimpleConsensusValidationResult::new_with_error( + DataContractUpdateActionNotAllowedError::new( + self.id(), + format!( + "change group at position {} is not allowed", + old_group_position + ), + ) + .into(), + )); + } + } + } + } + + if self.tokens() != new_data_contract.tokens() { + for (token_position, old_token_config) in self.tokens() { + // Check if a token has been removed + if !new_data_contract.tokens().contains_key(token_position) { + return Ok(SimpleConsensusValidationResult::new_with_error( + DataContractUpdateActionNotAllowedError::new( + self.id(), + format!("remove token at position {}", token_position), + ) + .into(), + )); + } + + // Check if a token configuration has been changed + if let Some(new_token_config) = new_data_contract.tokens().get(token_position) { + if old_token_config != new_token_config { + return Ok(SimpleConsensusValidationResult::new_with_error( + DataContractUpdateActionNotAllowedError::new( + self.id(), + format!("update token at position {}", token_position), + ) + .into(), + )); + } + } + } + + // Validate any newly added tokens + for (token_contract_position, token_configuration) in new_data_contract.tokens() { + if !self.tokens().contains_key(token_contract_position) { + if let Some(distribution) = token_configuration + .distribution_rules() + .pre_programmed_distribution() + { + if let Some((timestamp, _)) = distribution.distributions().iter().next() { + if timestamp < &block_info.time_ms { + return Ok(SimpleConsensusValidationResult::new_with_error( + StateError::PreProgrammedDistributionTimestampInPastError( + PreProgrammedDistributionTimestampInPastError::new( + new_data_contract.id(), + *token_contract_position, + *timestamp, + block_info.time_ms, + ), + ) + .into(), + )); + } + } + } + } + } + } + + if self.keywords() != new_data_contract.keywords() { + // Validate there are no more than 50 contract keywords + if new_data_contract.keywords().len() > 50 { + return Ok(SimpleConsensusValidationResult::new_with_error( + TooManyKeywordsError::new(self.id(), new_data_contract.keywords().len() as u8) + .into(), + )); + } + + // Validate the keywords are all unique and between 3 and 50 characters + let mut seen_keywords = HashSet::new(); + for keyword in new_data_contract.keywords() { + // First check keyword length + if keyword.len() < 3 || keyword.len() > 50 { + return Ok(SimpleConsensusValidationResult::new_with_error( + InvalidKeywordLengthError::new(self.id(), keyword.to_string()).into(), + )); + } + + if !keyword + .chars() + .all(|c| !c.is_control() && !c.is_whitespace()) + { + // This would mean we have an invalid character + return Ok(SimpleConsensusValidationResult::new_with_error( + InvalidKeywordCharacterError::new( + new_data_contract.id(), + keyword.to_string(), + ) + .into(), + )); + } + + // Then check uniqueness + if !seen_keywords.insert(keyword) { + return Ok(SimpleConsensusValidationResult::new_with_error( + DuplicateKeywordsError::new(self.id(), keyword.to_string()).into(), + )); + } + } + } + + if self.description() != new_data_contract.description() { + // Validate the description is between 3 and 100 characters + if let Some(description) = new_data_contract.description() { + let char_count = description.chars().count(); + if !(3..=100).contains(&char_count) { + return Ok(SimpleConsensusValidationResult::new_with_error( + InvalidDescriptionLengthError::new(self.id(), description.to_string()) + .into(), + )); + } + } + } + + Ok(SimpleConsensusValidationResult::new()) + } +} + +#[cfg(test)] +mod tests { + use super::*; + use crate::consensus::basic::basic_error::BasicError; + use crate::consensus::ConsensusError; + use crate::data_contract::accessors::v0::DataContractV0Setters; + use crate::data_contract::methods::validate_update::DataContractUpdateValidationMethodsV0; + use crate::data_contract::schema::DataContractSchemaMethodsV0; + use crate::prelude::IdentityNonce; + use crate::tests::fixtures::get_data_contract_fixture; + use assert_matches::assert_matches; + use platform_value::platform_value; + + #[test] + fn should_validate_required_since_on_document_types_added_by_the_update() { + let platform_version = PlatformVersion::latest(); + + let old_data_contract = get_data_contract_fixture( + None, + IdentityNonce::default(), + platform_version.protocol_version, + ) + .data_contract_owned(); + + let new_type_schema = |required_since: u32| { + platform_value!({ + "type": "object", + "properties": { + "message": { + "type": "string", + "position": 0, + "maxLength": 60_u32, + "requiredSince": required_since, + } + }, + "required": ["message"], + "additionalProperties": false + }) + }; + + // A new document type pre-scheduling requiredness at version 99 + // has no old counterpart, so the per-type update validation + // never runs on it — this pass must catch it + let mut new_data_contract = old_data_contract.clone(); + new_data_contract.set_version(old_data_contract.version() + 1); + new_data_contract + .set_document_schema( + "note", + new_type_schema(99), + false, + &mut Vec::new(), + platform_version, + ) + .expect("should add document type"); + + let result = old_data_contract + .validate_update(&new_data_contract, &BlockInfo::default(), platform_version) + .expect("failed validate update"); + + assert_matches!( + result.errors.as_slice(), + [ConsensusError::BasicError( + BasicError::DataContractInvalidRequiredFieldsUpdateError(e) + )] if e.details().contains("must carry requiredSince 2") + ); + + // The same new document type annotated with the version this + // update creates is accepted + let mut new_data_contract = old_data_contract.clone(); + new_data_contract.set_version(old_data_contract.version() + 1); + new_data_contract + .set_document_schema( + "note", + new_type_schema(old_data_contract.version() + 1), + false, + &mut Vec::new(), + platform_version, + ) + .expect("should add document type"); + + let result = old_data_contract + .validate_update(&new_data_contract, &BlockInfo::default(), platform_version) + .expect("failed validate update"); + + assert!( + result.is_valid(), + "a new document type annotated with the version this update \ + creates must be accepted, got {:?}", + result.errors + ); + } +} diff --git a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v6.rs b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v6.rs index 546494f144d..37db8c03037 100644 --- a/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v6.rs +++ b/packages/rs-platform-version/src/version/dpp_versions/dpp_contract_versions/v6.rs @@ -48,7 +48,12 @@ pub const CONTRACT_VERSIONS_V6: DPPContractVersions = DPPContractVersions { }, methods: DataContractMethodVersions { validate_document: 0, - validate_update: 0, + // Generation 1 (requiredSince): feeds the new contract version into + // per-document-type update validation and validates requiredSince + // annotations on document types introduced by the update, which the + // per-type pass never sees. Generation 0 stays byte-identical for + // replay of pre-v14 blocks. + validate_update: 1, schema: 0, validate_groups: 0, equal_ignoring_time_fields: 0, From 879d21b014f711aa6da61fce64b20e3d5c62a67a Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Tue, 25 Aug 2026 14:44:03 +0200 Subject: [PATCH 08/23] test(drive): cover contract-version stamping in create/replace action conversions Regression tests through the version-dispatched entry points: protocol v13 (generation 0) leaves contract_version unset and protocol v14 (generation 1) stamps the fetched contract's version, for both borrowed and owned create and replace conversions. The fixture contract version is bumped to 7 so a hardcoded stamp cannot pass by accident. Co-Authored-By: Claude Fable 5 --- .../state_transition_action/batch/tests.rs | 149 +++++++++++++++++- 1 file changed, 144 insertions(+), 5 deletions(-) diff --git a/packages/rs-drive/src/state_transition_action/batch/tests.rs b/packages/rs-drive/src/state_transition_action/batch/tests.rs index f441b6d7636..8ffacdeb10b 100644 --- a/packages/rs-drive/src/state_transition_action/batch/tests.rs +++ b/packages/rs-drive/src/state_transition_action/batch/tests.rs @@ -8,10 +8,10 @@ use std::collections::{BTreeMap, BTreeSet}; use std::sync::Arc; use dpp::block::block_info::BlockInfo; -use dpp::data_contract::accessors::v0::DataContractV0Getters; +use dpp::data_contract::accessors::v0::{DataContractV0Getters, DataContractV0Setters}; use dpp::data_contract::associated_token::token_configuration_item::TokenConfigurationChangeItem; use dpp::data_contract::associated_token::token_distribution_key::TokenDistributionInfo; -use dpp::document::{Document, DocumentV0}; +use dpp::document::{Document, DocumentV0, DocumentV0Getters}; use dpp::identifier::Identifier; use dpp::platform_value::Value; use dpp::tokens::emergency_action::TokenEmergencyAction; @@ -28,15 +28,15 @@ use crate::state_transition_action::batch::batched_transition::document_transiti }; use crate::state_transition_action::batch::batched_transition::document_transition::document_create_transition_action::{ DocumentCreateTransitionAction, DocumentCreateTransitionActionAccessorsV0, - DocumentCreateTransitionActionV0, + DocumentCreateTransitionActionV0, DocumentFromCreateTransitionAction, }; use crate::state_transition_action::batch::batched_transition::document_transition::document_delete_transition_action::DocumentDeleteTransitionAction; use crate::state_transition_action::batch::batched_transition::document_transition::document_delete_transition_action::v0::{ DocumentDeleteTransitionActionAccessorsV0, DocumentDeleteTransitionActionV0, }; use crate::state_transition_action::batch::batched_transition::document_transition::document_replace_transition_action::{ - DocumentReplaceTransitionAction, DocumentReplaceTransitionActionAccessorsV0, - DocumentReplaceTransitionActionV0, + DocumentFromReplaceTransitionAction, DocumentReplaceTransitionAction, + DocumentReplaceTransitionActionAccessorsV0, DocumentReplaceTransitionActionV0, }; use crate::state_transition_action::batch::batched_transition::document_transition::document_transfer_transition_action::{ DocumentTransferTransitionAction, DocumentTransferTransitionActionAccessorsV0, @@ -2900,3 +2900,142 @@ fn test_batched_transition_from_bump_action() { let batched: BatchedTransitionAction = bump.into(); assert!(batched.as_bump_identity_nonce_action().is_ok()); } + +// ============================================================ +// 10. Contract-version stamp on create/replace conversions +// ============================================================ +// +// The document built from a create or replace action carries a +// contract-version stamp from generation 1 of the conversion (protocol +// v14, document serialization format 3) and no stamp under generation 0. +// These tests go through the version-dispatched entry points so they fail +// on a wrong Drive version-table selection, a missing stamp in one +// conversion path, or a stamp that isn't the fetched contract's version. + +/// Contract version deliberately different from the fixture default (1) so a +/// hardcoded stamp can't pass by accident. +const STAMP_TEST_CONTRACT_VERSION: u32 = 7; + +fn stamp_test_contract_info(protocol_version: u32) -> Arc { + let mut info = DataContractFetchInfo::dpns_contract_fixture(protocol_version); + info.contract.set_version(STAMP_TEST_CONTRACT_VERSION); + Arc::new(info) +} + +fn stamp_test_create_action(protocol_version: u32) -> DocumentCreateTransitionAction { + let base = DocumentBaseTransitionAction::V0(DocumentBaseTransitionActionV0 { + id: Identifier::from([0xAA; 32]), + identity_contract_nonce: 1, + document_type_name: "domain".to_string(), + data_contract: stamp_test_contract_info(protocol_version), + token_cost: None, + gas_fees_paid_by: GasFeesPaidBy::default(), + }); + DocumentCreateTransitionAction::V0(DocumentCreateTransitionActionV0 { + base, + block_info: BlockInfo::default(), + data: BTreeMap::from([("key".to_string(), Value::Text("value".to_string()))]), + prefunded_voting_balance: None, + current_store_contest_info: None, + should_store_contest_info: None, + }) +} + +fn stamp_test_replace_action(protocol_version: u32) -> DocumentReplaceTransitionAction { + let base = DocumentBaseTransitionAction::V0(DocumentBaseTransitionActionV0 { + id: Identifier::from([0xAA; 32]), + identity_contract_nonce: 1, + document_type_name: "domain".to_string(), + data_contract: stamp_test_contract_info(protocol_version), + token_cost: None, + gas_fees_paid_by: GasFeesPaidBy::default(), + }); + DocumentReplaceTransitionAction::V0(DocumentReplaceTransitionActionV0 { + base, + revision: 2, + created_at: Some(1000), + updated_at: Some(2000), + transferred_at: Some(3000), + created_at_block_height: Some(10), + updated_at_block_height: Some(20), + transferred_at_block_height: Some(30), + created_at_core_block_height: Some(100), + updated_at_core_block_height: Some(200), + transferred_at_core_block_height: Some(300), + data: BTreeMap::from([("field".to_string(), Value::U64(42))]), + changed_data_fields: BTreeSet::from(["field".to_string()]), + creator_id: Some(Identifier::from([0xCC; 32])), + }) +} + +#[test] +fn should_not_stamp_contract_version_on_create_conversion_before_format_3() { + let platform_version = PlatformVersion::get(13).expect("expected protocol version 13"); + let owner_id = Identifier::from([0xDD; 32]); + + let action = stamp_test_create_action(platform_version.protocol_version); + let borrowed = Document::try_from_create_transition_action(&action, owner_id, platform_version) + .expect("borrowed create conversion"); + assert_eq!(borrowed.contract_version(), None); + + let owned = + Document::try_from_owned_create_transition_action(action, owner_id, platform_version) + .expect("owned create conversion"); + assert_eq!(owned.contract_version(), None); +} + +#[test] +fn should_stamp_fetched_contract_version_on_create_conversion() { + let platform_version = PlatformVersion::latest(); + let owner_id = Identifier::from([0xDD; 32]); + + let action = stamp_test_create_action(platform_version.protocol_version); + let borrowed = Document::try_from_create_transition_action(&action, owner_id, platform_version) + .expect("borrowed create conversion"); + assert_eq!( + borrowed.contract_version(), + Some(STAMP_TEST_CONTRACT_VERSION) + ); + + let owned = + Document::try_from_owned_create_transition_action(action, owner_id, platform_version) + .expect("owned create conversion"); + assert_eq!(owned.contract_version(), Some(STAMP_TEST_CONTRACT_VERSION)); +} + +#[test] +fn should_not_stamp_contract_version_on_replace_conversion_before_format_3() { + let platform_version = PlatformVersion::get(13).expect("expected protocol version 13"); + let owner_id = Identifier::from([0xDD; 32]); + + let action = stamp_test_replace_action(platform_version.protocol_version); + let borrowed = + Document::try_from_replace_transition_action(&action, owner_id, platform_version) + .expect("borrowed replace conversion"); + assert_eq!(borrowed.contract_version(), None); + + let owned = + Document::try_from_owned_replace_transition_action(action, owner_id, platform_version) + .expect("owned replace conversion"); + assert_eq!(owned.contract_version(), None); +} + +#[test] +fn should_stamp_fetched_contract_version_on_replace_conversion() { + let platform_version = PlatformVersion::latest(); + let owner_id = Identifier::from([0xDD; 32]); + + let action = stamp_test_replace_action(platform_version.protocol_version); + let borrowed = + Document::try_from_replace_transition_action(&action, owner_id, platform_version) + .expect("borrowed replace conversion"); + assert_eq!( + borrowed.contract_version(), + Some(STAMP_TEST_CONTRACT_VERSION) + ); + + let owned = + Document::try_from_owned_replace_transition_action(action, owner_id, platform_version) + .expect("owned replace conversion"); + assert_eq!(owned.contract_version(), Some(STAMP_TEST_CONTRACT_VERSION)); +} From 3e1d8eccafab04c079f372604f1d6153533559eb Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Tue, 25 Aug 2026 17:58:31 +0200 Subject: [PATCH 09/23] fix(dpp): classify requiredSince contract-version invariant failures as consensus errors MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The parsed-property invariant (requiredSince may not exceed the version of the contract carrying it) returned a plain ProtocolError::DataContractError, which drive-abci's create/update flows treat as an execution error that aborts transition processing. Untrusted schema data must instead yield a consensus-invalid result and a nonce-bump action. The invariant now produces the dedicated DataContractInvalidRequiredFieldsUpdateError (code 10276) and all four V0/V1 serialization call sites classify it through the validation-aware consensus path, mirroring consensus_or_protocol_data_contract_error. New create/update state validation tests assert the consensus error and the bump action end-to-end — the create one through a $defs-referenced required property, the exact shape that bypasses the raw-JSON basic-structure scan. Co-Authored-By: Claude Fable 5 --- .../document_type/class_methods/mod.rs | 22 ++++ .../src/data_contract/document_type/mod.rs | 14 ++- .../src/data_contract/v0/serialization/mod.rs | 8 +- .../src/data_contract/v1/serialization/mod.rs | 8 +- .../data_contract_create/state/v0/mod.rs | 105 +++++++++++++++++- .../data_contract_update/state/v0/mod.rs | 102 ++++++++++++++++- 6 files changed, 250 insertions(+), 9 deletions(-) diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/mod.rs index 48b129985a4..5023b2e4d82 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/mod.rs @@ -1,3 +1,4 @@ +use crate::consensus::basic::data_contract::DataContractInvalidRequiredFieldsUpdateError; #[cfg(feature = "validation")] use crate::consensus::basic::BasicError; #[cfg(feature = "validation")] @@ -26,6 +27,27 @@ pub(crate) fn consensus_or_protocol_data_contract_error( } } +#[inline] +pub(crate) fn consensus_or_protocol_required_fields_error( + error: DataContractInvalidRequiredFieldsUpdateError, +) -> ProtocolError { + #[cfg(feature = "validation")] + { + ProtocolError::ConsensusError( + ConsensusError::BasicError(BasicError::DataContractInvalidRequiredFieldsUpdateError( + error, + )) + .into(), + ) + } + #[cfg(not(feature = "validation"))] + { + ProtocolError::DataContractError(DataContractError::InvalidContractStructure( + error.to_string(), + )) + } +} + #[inline] pub(crate) fn consensus_or_protocol_value_error( platform_value_error: platform_value::Error, diff --git a/packages/rs-dpp/src/data_contract/document_type/mod.rs b/packages/rs-dpp/src/data_contract/document_type/mod.rs index 60bbbb9e53f..3c151633407 100644 --- a/packages/rs-dpp/src/data_contract/document_type/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/mod.rs @@ -47,10 +47,17 @@ pub const STORAGE_FLAGS_SIZE: usize = 2; /// serialized form (creates, updates, and disk loads all pass through /// there); a no-op for every contract predating the keyword, since their /// properties carry no annotation. +/// +/// The failure is the dedicated consensus error, because the input is +/// untrusted schema data: state-transition processing must classify it as +/// consensus-invalid (nonce bump), never as an execution error. Callers map +/// it through +/// [`class_methods::consensus_or_protocol_required_fields_error`]. pub(crate) fn validate_required_since_within_contract_version( document_types: &std::collections::BTreeMap, contract_version: u32, -) -> Result<(), crate::data_contract::errors::DataContractError> { +) -> Result<(), crate::consensus::basic::data_contract::DataContractInvalidRequiredFieldsUpdateError> +{ use crate::data_contract::document_type::accessors::DocumentTypeV0Getters; for (document_type_name, document_type) in document_types { @@ -58,9 +65,10 @@ pub(crate) fn validate_required_since_within_contract_version( if let Some(required_since) = property.required_since { if required_since > contract_version { return Err( - crate::data_contract::errors::DataContractError::InvalidContractStructure( + crate::consensus::basic::data_contract::DataContractInvalidRequiredFieldsUpdateError::new( + document_type_name.clone(), format!( - "property '{property_name}' of document type '{document_type_name}' carries requiredSince {required_since} which exceeds the contract version {contract_version}" + "property '{property_name}' carries requiredSince {required_since} which exceeds the contract version {contract_version}" ), ), ); diff --git a/packages/rs-dpp/src/data_contract/v0/serialization/mod.rs b/packages/rs-dpp/src/data_contract/v0/serialization/mod.rs index beb38d035b5..ce860414232 100644 --- a/packages/rs-dpp/src/data_contract/v0/serialization/mod.rs +++ b/packages/rs-dpp/src/data_contract/v0/serialization/mod.rs @@ -105,7 +105,9 @@ impl DataContractV0 { &document_types, version, ) - .map_err(ProtocolError::DataContractError)?; + .map_err( + crate::data_contract::document_type::class_methods::consensus_or_protocol_required_fields_error, + )?; let data_contract = DataContractV0 { id, @@ -154,7 +156,9 @@ impl DataContractV0 { &document_types, version, ) - .map_err(ProtocolError::DataContractError)?; + .map_err( + crate::data_contract::document_type::class_methods::consensus_or_protocol_required_fields_error, + )?; let data_contract = DataContractV0 { id, diff --git a/packages/rs-dpp/src/data_contract/v1/serialization/mod.rs b/packages/rs-dpp/src/data_contract/v1/serialization/mod.rs index b12357388bc..19b28ef4fc9 100644 --- a/packages/rs-dpp/src/data_contract/v1/serialization/mod.rs +++ b/packages/rs-dpp/src/data_contract/v1/serialization/mod.rs @@ -104,7 +104,9 @@ impl DataContractV1 { &document_types, version, ) - .map_err(ProtocolError::DataContractError)?; + .map_err( + crate::data_contract::document_type::class_methods::consensus_or_protocol_required_fields_error, + )?; let data_contract = DataContractV1 { id, @@ -171,7 +173,9 @@ impl DataContractV1 { &document_types, version, ) - .map_err(ProtocolError::DataContractError)?; + .map_err( + crate::data_contract::document_type::class_methods::consensus_or_protocol_required_fields_error, + )?; let data_contract = DataContractV1 { id, diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/state/v0/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/state/v0/mod.rs index dc72d28a481..cf445d8841c 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/state/v0/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_create/state/v0/mod.rs @@ -443,7 +443,7 @@ mod tests { use dpp::data_contract::accessors::v0::DataContractV0Getters; use dpp::data_contract::errors::DataContractError; use dpp::data_contract::serialized_version::DataContractInSerializationFormat; - use dpp::platform_value::Value; + use dpp::platform_value::{platform_value, Value}; use dpp::prelude::IdentityNonce; use dpp::state_transition::data_contract_create_transition::DataContractCreateTransitionV0; use dpp::tests::fixtures::get_data_contract_fixture; @@ -546,6 +546,109 @@ mod tests { assert!(!execution_context.operations_slice().is_empty()); } + #[test] + fn should_return_invalid_result_when_required_since_reached_through_a_ref_exceeds_the_contract_version( + ) { + // The create-time basic-structure check scans only raw top-level + // property JSON, so a `requiredSince` annotation hidden behind + // `$ref` slips past it. Contract deserialization enforces the + // invariant on parsed properties instead, and its failure must be + // classified as a consensus error converted to a nonce bump — + // never as an execution error, which would abort processing. + let platform_version = PlatformVersion::latest(); + let identity_nonce = IdentityNonce::default(); + + let platform = TestPlatformBuilder::new() + .build_with_mock_rpc() + .set_genesis_state(); + + let data_contract = + get_data_contract_fixture(None, identity_nonce, platform_version.protocol_version) + .data_contract_owned(); + + let identity_id = data_contract.owner_id(); + + let mut data_contract_for_serialization = data_contract + .try_into_platform_versioned(platform_version) + .expect("failed to convert data contract"); + + let DataContractInSerializationFormat::V1(ref mut contract) = + data_contract_for_serialization + else { + panic!("expected serialization version 1") + }; + + contract + .schema_defs + .get_or_insert_with(Default::default) + .insert( + "annotated".to_string(), + platform_value!({ + "type": "string", + "maxLength": 60_u32, + "requiredSince": 2_u32, + }), + ); + contract.document_schemas.insert( + "note".to_string(), + platform_value!({ + "type": "object", + "properties": { + "message": {"$ref": "#/$defs/annotated", "position": 0_u32}, + }, + "required": ["message"], + "additionalProperties": false + }), + ); + + let transition: DataContractCreateTransition = DataContractCreateTransitionV0 { + data_contract: data_contract_for_serialization, + identity_nonce, + user_fee_increase: 0, + signature_public_key_id: 0, + signature: Default::default(), + } + .into(); + + let mut execution_context = + StateTransitionExecutionContext::default_for_platform_version(platform_version) + .expect("failed to create execution context"); + + let state = platform.state.load_full(); + + let platform_ref = PlatformRef { + drive: &platform.drive, + state: &state, + config: &platform.config, + core_rpc: &platform.core_rpc, + }; + + let result = transition + .validate_state_v0::( + &platform_ref, + &BlockInfo::default(), + ValidationMode::Validator, + None, + &mut execution_context, + platform_version, + ) + .expect("failed to validate state"); + + assert_matches!( + result.errors.as_slice(), + [ConsensusError::BasicError( + BasicError::DataContractInvalidRequiredFieldsUpdateError(e) + )] if e.document_type() == "note" + && e.details().contains("requiredSince 2 which exceeds the contract version 1") + ); + + assert_matches!( + result.data, + Some(StateTransitionAction::BumpIdentityNonceAction(action)) + if action.identity_id() == identity_id && action.identity_nonce() == identity_nonce + ); + } + #[test] fn should_return_invalid_result_when_transform_into_action_failed_latest() { let platform_version = PlatformVersion::latest(); diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_update/state/v0/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_update/state/v0/mod.rs index 776ca42b7a1..e7a04348217 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_update/state/v0/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/data_contract_update/state/v0/mod.rs @@ -510,7 +510,7 @@ mod tests { use dpp::data_contract::accessors::v0::DataContractV0Setters; use dpp::data_contract::errors::DataContractError; use dpp::data_contract::serialized_version::DataContractInSerializationFormat; - use dpp::platform_value::Value; + use dpp::platform_value::{platform_value, Value}; use dpp::prelude::IdentityNonce; use dpp::state_transition::data_contract_update_transition::DataContractUpdateTransitionV0; use dpp::tests::fixtures::get_data_contract_fixture; @@ -606,6 +606,106 @@ mod tests { assert!(!execution_context.operations_slice().is_empty()); } + #[test] + fn should_return_invalid_result_when_required_since_exceeds_the_contract_version() { + // Deserializing the updated contract enforces `requiredSince <= + // contract version` on parsed properties. That failure must be + // classified as a consensus error converted to a nonce bump — + // never as an execution error, which would abort processing. + let platform_version = PlatformVersion::latest(); + let identity_contract_nonce = IdentityNonce::default(); + + let platform = TestPlatformBuilder::new() + .build_with_mock_rpc() + .set_genesis_state(); + + let data_contract = get_data_contract_fixture( + None, + identity_contract_nonce, + platform_version.protocol_version, + ) + .data_contract_owned(); + + let identity_id = data_contract.owner_id(); + let data_contract_id = data_contract.id(); + + let mut data_contract_for_serialization = data_contract + .try_into_platform_versioned(platform_version) + .expect("failed to convert data contract"); + + let DataContractInSerializationFormat::V1(ref mut contract) = + data_contract_for_serialization + else { + panic!("expected serialization version 1") + }; + + contract.document_schemas.insert( + "note".to_string(), + platform_value!({ + "type": "object", + "properties": { + "message": { + "type": "string", + "position": 0_u32, + "maxLength": 60_u32, + "requiredSince": 99_u32, + }, + }, + "required": ["message"], + "additionalProperties": false + }), + ); + + let transition: DataContractUpdateTransition = DataContractUpdateTransitionV0 { + identity_contract_nonce, + data_contract: data_contract_for_serialization, + user_fee_increase: 0, + signature_public_key_id: 0, + signature: Default::default(), + } + .into(); + + let mut execution_context = + StateTransitionExecutionContext::default_for_platform_version(platform_version) + .expect("failed to create execution context"); + + let state = platform.state.load_full(); + + let platform_ref = PlatformRef { + drive: &platform.drive, + state: &state, + config: &platform.config, + core_rpc: &platform.core_rpc, + }; + + let result = transition + .validate_state_v0::( + &platform_ref, + &BlockInfo::default(), + ValidationMode::Validator, + &mut execution_context, + None, + platform_version, + ) + .expect("failed to validate state"); + + assert_matches!( + result.errors.as_slice(), + [ConsensusError::BasicError( + BasicError::DataContractInvalidRequiredFieldsUpdateError(e) + )] if e.document_type() == "note" + && e.details().contains("requiredSince 99 which exceeds the contract version 1") + ); + + assert_matches!( + result.data, + Some(StateTransitionAction::BumpIdentityDataContractNonceAction(action)) + if action.identity_id() == identity_id + && action.identity_contract_nonce() == identity_contract_nonce + && action.data_contract_id() == data_contract_id + ); + } + #[test] fn should_return_invalid_result_when_data_contract_does_not_exist() { let platform_version = PlatformVersion::latest(); From ccb60da7f84f4e1bc7a763ab6e7f7a28e6875a91 Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Wed, 26 Aug 2026 07:47:12 +0200 Subject: [PATCH 10/23] refactor(dpp): name the contract-version stamp size in estimated_size v1 Extract the format-3 stamp's worst-case varint length (5 bytes for a u32) into CONTRACT_VERSION_STAMP_MAX_SIZE next to the other document-type size constants, replacing the magic number flagged in review. Co-Authored-By: Claude Fable 5 --- .../document_type/methods/versioned_methods.rs | 11 +++++++---- .../rs-dpp/src/data_contract/document_type/mod.rs | 3 +++ 2 files changed, 10 insertions(+), 4 deletions(-) diff --git a/packages/rs-dpp/src/data_contract/document_type/methods/versioned_methods.rs b/packages/rs-dpp/src/data_contract/document_type/methods/versioned_methods.rs index d2f18611bd1..1501184adfb 100644 --- a/packages/rs-dpp/src/data_contract/document_type/methods/versioned_methods.rs +++ b/packages/rs-dpp/src/data_contract/document_type/methods/versioned_methods.rs @@ -4,7 +4,8 @@ use crate::data_contract::document_type::v0::DocumentTypeV0; use crate::data_contract::document_type::v1::DocumentTypeV1; use crate::data_contract::document_type::v2::DocumentTypeV2; use crate::data_contract::document_type::{ - DocumentPropertyType, DocumentType, DocumentTypeRef, Index, DEFAULT_HASH_SIZE, MAX_INDEX_SIZE, + DocumentPropertyType, DocumentType, DocumentTypeRef, Index, CONTRACT_VERSION_STAMP_MAX_SIZE, + DEFAULT_HASH_SIZE, MAX_INDEX_SIZE, }; use crate::data_contract::errors::DataContractError; use crate::document::property_names::{ @@ -443,10 +444,12 @@ pub trait DocumentTypeV0MethodsVersioned: DocumentTypeV0Getters + DocumentTypeBa } /// Generation 0 plus the document serialization format 3 - /// contract-version stamp varint (worst case 5 bytes for a u32). - /// Selected together with format 3 by the version table. + /// contract-version stamp varint. Selected together with format 3 by + /// the version table. fn estimated_size_v1(&self, platform_version: &PlatformVersion) -> Result { - Ok(self.estimated_size_v0(platform_version)?.saturating_add(5)) + Ok(self + .estimated_size_v0(platform_version)? + .saturating_add(CONTRACT_VERSION_STAMP_MAX_SIZE)) } fn max_size_v0(&self, platform_version: &PlatformVersion) -> Result { diff --git a/packages/rs-dpp/src/data_contract/document_type/mod.rs b/packages/rs-dpp/src/data_contract/document_type/mod.rs index 3c151633407..61d755827c0 100644 --- a/packages/rs-dpp/src/data_contract/document_type/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/mod.rs @@ -39,6 +39,9 @@ pub const DEFAULT_FLOAT_SIZE: usize = 8; pub const EMPTY_TREE_STORAGE_SIZE: usize = 33; pub const MAX_INDEX_SIZE: usize = 255; pub const STORAGE_FLAGS_SIZE: usize = 2; +/// Worst-case byte length of the contract-version stamp written by document +/// serialization format 3: a u32 varint. +pub const CONTRACT_VERSION_STAMP_MAX_SIZE: u16 = 5; /// A `requiredSince` annotation may never exceed the version of the contract /// carrying it — requiredness cannot be pre-scheduled at a future version. From 9908e804e800ad7b1b7e6e0c8353b8492295f5be Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Wed, 26 Aug 2026 09:31:19 +0200 Subject: [PATCH 11/23] refactor(dpp): move apply_required_since into its own versioned module Relocate the requiredSince parse dispatcher and its generation 0 from try_from_schema/mod.rs into class_methods/apply_required_since/{mod.rs,v0}, matching the versioned file layout used by the other class methods. No behavior change. Co-Authored-By: Claude Fable 5 --- .../class_methods/apply_required_since/mod.rs | 40 +++++++++++ .../apply_required_since/v0/mod.rs | 43 ++++++++++++ .../document_type/class_methods/mod.rs | 1 + .../class_methods/try_from_schema/mod.rs | 67 +------------------ 4 files changed, 85 insertions(+), 66 deletions(-) create mode 100644 packages/rs-dpp/src/data_contract/document_type/class_methods/apply_required_since/mod.rs create mode 100644 packages/rs-dpp/src/data_contract/document_type/class_methods/apply_required_since/v0/mod.rs diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/apply_required_since/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/apply_required_since/mod.rs new file mode 100644 index 00000000000..5e09e394782 --- /dev/null +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/apply_required_since/mod.rs @@ -0,0 +1,40 @@ +use std::collections::BTreeMap; + +use platform_value::Value; +use platform_version::version::PlatformVersion; + +use crate::data_contract::errors::DataContractError; + +mod v0; + +/// Parses the `requiredSince` keyword: the contract version from which the +/// property is required. Only meaningful on top-level required properties — +/// the document wire format encodes a required property without a presence +/// flag, so requiredness that varies by contract version must be resolvable +/// per property from the current schema alone (see the per-document contract +/// version stamp in document serialization format 3). +/// +/// Versioned on `apply_required_since` in the platform version's document +/// type schema versions. `None` selects the behavior of the versions that +/// predate the keyword: it is ignored entirely, so their parses stay +/// byte-for-byte identical to what they always produced. +pub(crate) fn apply_required_since( + inner_properties: &BTreeMap, + is_required: bool, + is_top_level: bool, + platform_version: &PlatformVersion, +) -> Result, DataContractError> { + match platform_version + .dpp + .contract_versions + .document_type_versions + .schema + .apply_required_since + { + None => Ok(None), + Some(0) => v0::apply_required_since_v0(inner_properties, is_required, is_top_level), + Some(version) => Err(DataContractError::Unsupported(format!( + "apply_required_since version {version} is not supported" + ))), + } +} diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/apply_required_since/v0/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/apply_required_since/v0/mod.rs new file mode 100644 index 00000000000..0932523d7ed --- /dev/null +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/apply_required_since/v0/mod.rs @@ -0,0 +1,43 @@ +use std::collections::BTreeMap; + +use platform_value::Value; + +use crate::data_contract::document_type::property_names; +use crate::data_contract::errors::DataContractError; + +/// Generation 0 parse rules: the keyword is admitted only on a top-level +/// property listed in `required`, and its value is a contract version of at +/// least 1 fitting in a u32. +pub(super) fn apply_required_since_v0( + inner_properties: &BTreeMap, + is_required: bool, + is_top_level: bool, +) -> Result, DataContractError> { + let Some(required_since_value) = inner_properties.get(property_names::REQUIRED_SINCE) else { + return Ok(None); + }; + + if !is_top_level { + return Err(DataContractError::InvalidContractStructure( + "requiredSince is only allowed on top-level properties".to_string(), + )); + } + + if !is_required { + return Err(DataContractError::InvalidContractStructure( + "requiredSince is only allowed on properties listed in required".to_string(), + )); + } + + let required_since: u32 = required_since_value + .to_integer() + .map_err(|e| DataContractError::ValueWrongType(e.to_string()))?; + + if required_since == 0 { + return Err(DataContractError::InvalidContractStructure( + "requiredSince must be a contract version of at least 1".to_string(), + )); + } + + Ok(Some(required_since)) +} diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/mod.rs index 5023b2e4d82..0fa5e3e27a7 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/mod.rs @@ -6,6 +6,7 @@ use crate::consensus::ConsensusError; use crate::data_contract::errors::DataContractError; use crate::ProtocolError; +pub(crate) mod apply_required_since; mod create_document_types_from_document_schemas; mod should_use_creator_id; mod system_properties; diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs index ce6ba7e1755..e711c5a3ca5 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs @@ -1,4 +1,5 @@ use crate::data_contract::config::DataContractConfig; +use crate::data_contract::document_type::class_methods::apply_required_since::apply_required_since; use crate::data_contract::document_type::v0::DocumentTypeV0; use crate::data_contract::document_type::v1::DocumentTypeV1; use crate::data_contract::document_type::{ @@ -318,72 +319,6 @@ fn insert_values_nested( Ok(()) } -/// Parses the `requiredSince` keyword: the contract version from which the -/// property is required. Only meaningful on top-level required properties — -/// the document wire format encodes a required property without a presence -/// flag, so requiredness that varies by contract version must be resolvable -/// per property from the current schema alone (see the per-document contract -/// version stamp in document serialization format 3). -/// -/// Versioned on `apply_required_since` in the platform version's document -/// type schema versions. `None` selects the behavior of the versions that -/// predate the keyword: it is ignored entirely, so their parses stay -/// byte-for-byte identical to what they always produced. -fn apply_required_since( - inner_properties: &BTreeMap, - is_required: bool, - is_top_level: bool, - platform_version: &PlatformVersion, -) -> Result, DataContractError> { - match platform_version - .dpp - .contract_versions - .document_type_versions - .schema - .apply_required_since - { - None => Ok(None), - Some(0) => apply_required_since_v0(inner_properties, is_required, is_top_level), - Some(version) => Err(DataContractError::Unsupported(format!( - "apply_required_since version {version} is not supported" - ))), - } -} - -fn apply_required_since_v0( - inner_properties: &BTreeMap, - is_required: bool, - is_top_level: bool, -) -> Result, DataContractError> { - let Some(required_since_value) = inner_properties.get(property_names::REQUIRED_SINCE) else { - return Ok(None); - }; - - if !is_top_level { - return Err(DataContractError::InvalidContractStructure( - "requiredSince is only allowed on top-level properties".to_string(), - )); - } - - if !is_required { - return Err(DataContractError::InvalidContractStructure( - "requiredSince is only allowed on properties listed in required".to_string(), - )); - } - - let required_since: u32 = required_since_value - .to_integer() - .map_err(|e| DataContractError::ValueWrongType(e.to_string()))?; - - if required_since == 0 { - return Err(DataContractError::InvalidContractStructure( - "requiredSince must be a contract version of at least 1".to_string(), - )); - } - - Ok(Some(required_since)) -} - /// Folds a `refersTo` declaration into the property type: an identifier property /// with `refersTo` becomes `IdentifierWithReference(target)`. Non-identifier /// properties cannot carry `refersTo`. From fc852d6b3c50a5ed9982278a78288918e06a6273 Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Wed, 26 Aug 2026 09:38:27 +0200 Subject: [PATCH 12/23] refactor(dpp): extract shared validate_update generation logic into common helpers MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The contract-level validate_update generations duplicated ~300 lines of generation-independent checks. Move them into validate_update/common/mod.rs as pub(super) helpers (ownership+version, config, existing document types, schema $defs, groups, tokens, keywords, description), mirroring the document-type-level validate_update common module. v0 and v1 become the orchestration sequence, with v1's new-document-type requiredSince check as its own named method. Pure extraction — same checks, same order, same short-circuit semantics in both generations. Co-Authored-By: Claude Fable 5 --- .../methods/validate_update/common/mod.rs | 393 ++++++++++++++++++ .../methods/validate_update/mod.rs | 1 + .../methods/validate_update/v0/mod.rs | 320 ++------------ .../methods/validate_update/v1/mod.rs | 349 +++------------- 4 files changed, 470 insertions(+), 593 deletions(-) create mode 100644 packages/rs-dpp/src/data_contract/methods/validate_update/common/mod.rs diff --git a/packages/rs-dpp/src/data_contract/methods/validate_update/common/mod.rs b/packages/rs-dpp/src/data_contract/methods/validate_update/common/mod.rs new file mode 100644 index 00000000000..81a786a9ee7 --- /dev/null +++ b/packages/rs-dpp/src/data_contract/methods/validate_update/common/mod.rs @@ -0,0 +1,393 @@ +//! Helpers shared by every generation of `DataContract::validate_update` +//! (`v0`, `v1`, …). Only the parts of the update-validation flow that differ +//! between generations live in the per-version modules; the checks below are +//! generation independent. Each helper returns a +//! `SimpleConsensusValidationResult`; generations run them in sequence and +//! short-circuit on the first invalid result, so extraction preserves the +//! original early-return semantics exactly. + +use std::collections::HashSet; + +use crate::block::block_info::BlockInfo; +use crate::consensus::state::state_error::StateError; +use crate::consensus::state::token::PreProgrammedDistributionTimestampInPastError; +use crate::data_contract::accessors::v0::DataContractV0Getters; + +use crate::consensus::basic::data_contract::{ + DuplicateKeywordsError, IncompatibleDataContractSchemaError, InvalidDataContractVersionError, + InvalidDescriptionLengthError, InvalidKeywordCharacterError, InvalidKeywordLengthError, + TooManyKeywordsError, +}; +use crate::consensus::state::data_contract::data_contract_update_action_not_allowed_error::DataContractUpdateActionNotAllowedError; +use crate::consensus::state::data_contract::data_contract_update_permission_error::DataContractUpdatePermissionError; +use crate::consensus::state::data_contract::document_type_update_error::DocumentTypeUpdateError; +use crate::data_contract::accessors::v1::DataContractV1Getters; +use crate::data_contract::associated_token::token_configuration::accessors::v0::TokenConfigurationV0Getters; +use crate::data_contract::associated_token::token_distribution_rules::accessors::v0::TokenDistributionRulesV0Getters; +use crate::data_contract::associated_token::token_pre_programmed_distribution::accessors::v0::TokenPreProgrammedDistributionV0Methods; +use crate::data_contract::document_type::schema::validate_schema_compatibility; +use crate::data_contract::schema::DataContractSchemaMethodsV0; +use crate::data_contract::DataContract; +use crate::validation::SimpleConsensusValidationResult; +use crate::ProtocolError; +use platform_value::Value; +use platform_version::version::PlatformVersion; +use serde_json::json; + +impl DataContract { + /// The update must come from the contract owner, and the new contract's + /// version must be exactly the old version plus one. + /// + /// Failure (version != previous version + 1): Keep ST and transform it to + /// a nonce bump action. How: A user pushed an update that was not the + /// next version. + pub(super) fn validate_update_ownership_and_version( + &self, + new_data_contract: &DataContract, + ) -> SimpleConsensusValidationResult { + // Check if the contract is owned by the same identity + if self.owner_id() != new_data_contract.owner_id() { + return SimpleConsensusValidationResult::new_with_error( + DataContractUpdatePermissionError::new(self.id(), new_data_contract.owner_id()) + .into(), + ); + } + + // Check version is bumped + let new_version = new_data_contract.version(); + let old_version = self.version(); + if new_version < old_version || new_version - old_version != 1 { + return SimpleConsensusValidationResult::new_with_error( + InvalidDataContractVersionError::new(old_version + 1, new_version).into(), + ); + } + + SimpleConsensusValidationResult::new() + } + + /// Validate that the config was not updated + /// * Includes verifications that: + /// - Old contract is not read_only + /// - New contract is not read_only + /// - Keeps history did not change + /// - Can be deleted did not change + /// - Documents keep history did not change + /// - Documents can be deleted contract default did not change + /// - Documents mutable contract default did not change + /// - Requires identity encryption bounded key did not change + /// - Requires identity decryption bounded key did not change + /// * Failure (contract does not exist): Keep ST and transform it to a nonce bump action. + /// * How: A user pushed an update to a contract that changed its configuration. + pub(super) fn validate_update_config( + &self, + new_data_contract: &DataContract, + platform_version: &PlatformVersion, + ) -> Result { + let config_validation_result = self.config().validate_update( + new_data_contract.config(), + self.id(), + platform_version, + )?; + + if !config_validation_result.is_valid() { + return Ok(SimpleConsensusValidationResult::new_with_errors( + config_validation_result.errors, + )); + } + + Ok(SimpleConsensusValidationResult::new()) + } + + /// Validate updates for existing document types to make sure that + /// previously created documents will be still valid with a new version + /// of the data contract. Document types can never be removed. The new + /// contract version is passed into the per-type dispatcher: its + /// generation 1 admits required-set additions annotated with + /// `requiredSince` equal to that version, while generation 0 — the only + /// one selected by pre-v14 platform versions — ignores it. + pub(super) fn validate_update_existing_document_types( + &self, + new_data_contract: &DataContract, + platform_version: &PlatformVersion, + ) -> Result { + for (document_type_name, old_document_type) in self.document_types() { + // Make sure that existing document aren't removed + let Some(new_document_type) = + new_data_contract.document_type_optional_for_name(document_type_name) + else { + return Ok(SimpleConsensusValidationResult::new_with_error( + DocumentTypeUpdateError::new( + self.id(), + document_type_name, + "document type can't be removed", + ) + .into(), + )); + }; + + // Validate document type update rules + let validate_update_result = old_document_type.as_ref().validate_update( + new_document_type, + new_data_contract.version(), + platform_version, + )?; + + if !validate_update_result.is_valid() { + return Ok(SimpleConsensusValidationResult::new_with_errors( + validate_update_result.errors, + )); + } + } + + Ok(SimpleConsensusValidationResult::new()) + } + + /// Schema $defs should be compatible: `$defs` may not be removed, and a + /// changed `$defs` must remain compatible with previously created data. + pub(super) fn validate_update_schema_defs( + &self, + new_data_contract: &DataContract, + platform_version: &PlatformVersion, + ) -> Result { + if let Some(old_defs_map) = self.schema_defs() { + // If new contract doesn't have $defs, it means that it's $defs was removed and compatibility is broken + let Some(new_defs_map) = new_data_contract.schema_defs() else { + return Ok(SimpleConsensusValidationResult::new_with_error( + IncompatibleDataContractSchemaError::new( + self.id(), + "remove".to_string(), + "/$defs".to_string(), + ) + .into(), + )); + }; + + // If $defs is updated we need to make sure that our data contract is still compatible + // with previously created data + if old_defs_map != new_defs_map { + // both new and old $defs already validated as a part of new and old contract + let old_defs_json = Value::from(old_defs_map) + .try_into_validating_json() + .map_err(ProtocolError::ValueError)?; + + let new_defs_json = Value::from(new_defs_map) + .try_into_validating_json() + .map_err(ProtocolError::ValueError)?; + + let old_defs_schema = json!({ + "$defs": old_defs_json + }); + + let new_defs_schema = json!({ + "$defs": new_defs_json + }); + + // We do not allow to remove or modify $ref in document type schemas + // it means that compatible changes in $defs won't break the overall compatibility + // Make sure that updated $defs schema is compatible + let compatibility_validation_result = validate_schema_compatibility( + &old_defs_schema, + &new_defs_schema, + platform_version, + )?; + + if !compatibility_validation_result.is_valid() { + let errors = compatibility_validation_result + .errors + .into_iter() + .map(|operation| { + IncompatibleDataContractSchemaError::new( + self.id(), + operation.name, + operation.path, + ) + .into() + }) + .collect(); + + return Ok(SimpleConsensusValidationResult::new_with_errors(errors)); + } + } + } + + Ok(SimpleConsensusValidationResult::new()) + } + + /// Groups can be neither removed nor changed by an update. + pub(super) fn validate_update_groups( + &self, + new_data_contract: &DataContract, + ) -> SimpleConsensusValidationResult { + if self.groups() != new_data_contract.groups() { + // No groups can have been removed + for old_group_position in self.groups().keys() { + if !new_data_contract.groups().contains_key(old_group_position) { + return SimpleConsensusValidationResult::new_with_error( + DataContractUpdateActionNotAllowedError::new( + self.id(), + "remove group".to_string(), + ) + .into(), + ); + } + } + + // Ensure no group has been changed + for (old_group_position, old_group) in self.groups() { + if let Some(new_group) = new_data_contract.groups().get(old_group_position) { + if old_group != new_group { + return SimpleConsensusValidationResult::new_with_error( + DataContractUpdateActionNotAllowedError::new( + self.id(), + format!( + "change group at position {} is not allowed", + old_group_position + ), + ) + .into(), + ); + } + } + } + } + + SimpleConsensusValidationResult::new() + } + + /// Existing tokens can be neither removed nor reconfigured; a newly + /// added token may not carry a pre-programmed distribution timestamp in + /// the past. + pub(super) fn validate_update_tokens( + &self, + new_data_contract: &DataContract, + block_info: &BlockInfo, + ) -> SimpleConsensusValidationResult { + if self.tokens() != new_data_contract.tokens() { + for (token_position, old_token_config) in self.tokens() { + // Check if a token has been removed + if !new_data_contract.tokens().contains_key(token_position) { + return SimpleConsensusValidationResult::new_with_error( + DataContractUpdateActionNotAllowedError::new( + self.id(), + format!("remove token at position {}", token_position), + ) + .into(), + ); + } + + // Check if a token configuration has been changed + if let Some(new_token_config) = new_data_contract.tokens().get(token_position) { + if old_token_config != new_token_config { + return SimpleConsensusValidationResult::new_with_error( + DataContractUpdateActionNotAllowedError::new( + self.id(), + format!("update token at position {}", token_position), + ) + .into(), + ); + } + } + } + + // Validate any newly added tokens + for (token_contract_position, token_configuration) in new_data_contract.tokens() { + if !self.tokens().contains_key(token_contract_position) { + if let Some(distribution) = token_configuration + .distribution_rules() + .pre_programmed_distribution() + { + if let Some((timestamp, _)) = distribution.distributions().iter().next() { + if timestamp < &block_info.time_ms { + return SimpleConsensusValidationResult::new_with_error( + StateError::PreProgrammedDistributionTimestampInPastError( + PreProgrammedDistributionTimestampInPastError::new( + new_data_contract.id(), + *token_contract_position, + *timestamp, + block_info.time_ms, + ), + ) + .into(), + ); + } + } + } + } + } + } + + SimpleConsensusValidationResult::new() + } + + /// Changed keywords must number at most 50, each between 3 and 50 + /// visible characters, all unique. + pub(super) fn validate_update_keywords( + &self, + new_data_contract: &DataContract, + ) -> SimpleConsensusValidationResult { + if self.keywords() != new_data_contract.keywords() { + // Validate there are no more than 50 contract keywords + if new_data_contract.keywords().len() > 50 { + return SimpleConsensusValidationResult::new_with_error( + TooManyKeywordsError::new(self.id(), new_data_contract.keywords().len() as u8) + .into(), + ); + } + + // Validate the keywords are all unique and between 3 and 50 characters + let mut seen_keywords = HashSet::new(); + for keyword in new_data_contract.keywords() { + // First check keyword length + if keyword.len() < 3 || keyword.len() > 50 { + return SimpleConsensusValidationResult::new_with_error( + InvalidKeywordLengthError::new(self.id(), keyword.to_string()).into(), + ); + } + + if !keyword + .chars() + .all(|c| !c.is_control() && !c.is_whitespace()) + { + // This would mean we have an invalid character + return SimpleConsensusValidationResult::new_with_error( + InvalidKeywordCharacterError::new( + new_data_contract.id(), + keyword.to_string(), + ) + .into(), + ); + } + + // Then check uniqueness + if !seen_keywords.insert(keyword) { + return SimpleConsensusValidationResult::new_with_error( + DuplicateKeywordsError::new(self.id(), keyword.to_string()).into(), + ); + } + } + } + + SimpleConsensusValidationResult::new() + } + + /// A changed description must be between 3 and 100 characters. + pub(super) fn validate_update_description( + &self, + new_data_contract: &DataContract, + ) -> SimpleConsensusValidationResult { + if self.description() != new_data_contract.description() { + // Validate the description is between 3 and 100 characters + if let Some(description) = new_data_contract.description() { + let char_count = description.chars().count(); + if !(3..=100).contains(&char_count) { + return SimpleConsensusValidationResult::new_with_error( + InvalidDescriptionLengthError::new(self.id(), description.to_string()) + .into(), + ); + } + } + } + + SimpleConsensusValidationResult::new() + } +} diff --git a/packages/rs-dpp/src/data_contract/methods/validate_update/mod.rs b/packages/rs-dpp/src/data_contract/methods/validate_update/mod.rs index e817ba86cec..5081e57bcd3 100644 --- a/packages/rs-dpp/src/data_contract/methods/validate_update/mod.rs +++ b/packages/rs-dpp/src/data_contract/methods/validate_update/mod.rs @@ -2,6 +2,7 @@ use crate::block::block_info::BlockInfo; use crate::prelude::DataContract; use platform_version::version::PlatformVersion; +mod common; mod v0; mod v1; use crate::validation::SimpleConsensusValidationResult; diff --git a/packages/rs-dpp/src/data_contract/methods/validate_update/v0/mod.rs b/packages/rs-dpp/src/data_contract/methods/validate_update/v0/mod.rs index 5d07ed0c5e9..6cea9ac97b6 100644 --- a/packages/rs-dpp/src/data_contract/methods/validate_update/v0/mod.rs +++ b/packages/rs-dpp/src/data_contract/methods/validate_update/v0/mod.rs @@ -1,30 +1,8 @@ -use std::collections::HashSet; - use crate::block::block_info::BlockInfo; -use crate::consensus::state::state_error::StateError; -use crate::consensus::state::token::PreProgrammedDistributionTimestampInPastError; -use crate::data_contract::accessors::v0::DataContractV0Getters; - -use crate::consensus::basic::data_contract::{ - DuplicateKeywordsError, IncompatibleDataContractSchemaError, InvalidDataContractVersionError, - InvalidDescriptionLengthError, InvalidKeywordCharacterError, InvalidKeywordLengthError, - TooManyKeywordsError, -}; -use crate::consensus::state::data_contract::data_contract_update_action_not_allowed_error::DataContractUpdateActionNotAllowedError; -use crate::consensus::state::data_contract::data_contract_update_permission_error::DataContractUpdatePermissionError; -use crate::consensus::state::data_contract::document_type_update_error::DocumentTypeUpdateError; -use crate::data_contract::accessors::v1::DataContractV1Getters; -use crate::data_contract::associated_token::token_configuration::accessors::v0::TokenConfigurationV0Getters; -use crate::data_contract::associated_token::token_distribution_rules::accessors::v0::TokenDistributionRulesV0Getters; -use crate::data_contract::associated_token::token_pre_programmed_distribution::accessors::v0::TokenPreProgrammedDistributionV0Methods; -use crate::data_contract::document_type::schema::validate_schema_compatibility; -use crate::data_contract::schema::DataContractSchemaMethodsV0; use crate::data_contract::DataContract; use crate::validation::SimpleConsensusValidationResult; use crate::ProtocolError; -use platform_value::Value; use platform_version::version::PlatformVersion; -use serde_json::json; pub trait DataContractUpdateValidationMethodsV0 { fn validate_update( @@ -43,291 +21,43 @@ impl DataContract { block_info: &BlockInfo, platform_version: &PlatformVersion, ) -> Result { - // Check if the contract is owned by the same identity - if self.owner_id() != new_data_contract.owner_id() { - return Ok(SimpleConsensusValidationResult::new_with_error( - DataContractUpdatePermissionError::new(self.id(), new_data_contract.owner_id()) - .into(), - )); - } - - // Check version is bumped - // Failure (version != previous version + 1): Keep ST and transform it to a nonce bump action. - // How: A user pushed an update that was not the next version. - - let new_version = new_data_contract.version(); - let old_version = self.version(); - if new_version < old_version || new_version - old_version != 1 { - return Ok(SimpleConsensusValidationResult::new_with_error( - InvalidDataContractVersionError::new(old_version + 1, new_version).into(), - )); - } - - // Validate that the config was not updated - // * Includes verifications that: - // - Old contract is not read_only - // - New contract is not read_only - // - Keeps history did not change - // - Can be deleted did not change - // - Documents keep history did not change - // - Documents can be deleted contract default did not change - // - Documents mutable contract default did not change - // - Requires identity encryption bounded key did not change - // - Requires identity decryption bounded key did not change - // * Failure (contract does not exist): Keep ST and transform it to a nonce bump action. - // * How: A user pushed an update to a contract that changed its configuration. - - let config_validation_result = self.config().validate_update( - new_data_contract.config(), - self.id(), - platform_version, - )?; - - if !config_validation_result.is_valid() { - return Ok(SimpleConsensusValidationResult::new_with_errors( - config_validation_result.errors, - )); + let result = self.validate_update_ownership_and_version(new_data_contract); + if !result.is_valid() { + return Ok(result); } - // Validate updates for existing document types to make sure that previously created - // documents will be still valid with a new version of the data contract - for (document_type_name, old_document_type) in self.document_types() { - // Make sure that existing document aren't removed - let Some(new_document_type) = - new_data_contract.document_type_optional_for_name(document_type_name) - else { - return Ok(SimpleConsensusValidationResult::new_with_error( - DocumentTypeUpdateError::new( - self.id(), - document_type_name, - "document type can't be removed", - ) - .into(), - )); - }; - - // Validate document type update rules. The document-type - // dispatcher takes the new contract version for generation 1; - // generation 0, the only one this method's platform versions - // select, ignores it. - let validate_update_result = old_document_type.as_ref().validate_update( - new_document_type, - new_data_contract.version(), - platform_version, - )?; - - if !validate_update_result.is_valid() { - return Ok(SimpleConsensusValidationResult::new_with_errors( - validate_update_result.errors, - )); - } + let result = self.validate_update_config(new_data_contract, platform_version)?; + if !result.is_valid() { + return Ok(result); } - // Schema $defs should be compatible - if let Some(old_defs_map) = self.schema_defs() { - // If new contract doesn't have $defs, it means that it's $defs was removed and compatibility is broken - let Some(new_defs_map) = new_data_contract.schema_defs() else { - return Ok(SimpleConsensusValidationResult::new_with_error( - IncompatibleDataContractSchemaError::new( - self.id(), - "remove".to_string(), - "/$defs".to_string(), - ) - .into(), - )); - }; - - // If $defs is updated we need to make sure that our data contract is still compatible - // with previously created data - if old_defs_map != new_defs_map { - // both new and old $defs already validated as a part of new and old contract - let old_defs_json = Value::from(old_defs_map) - .try_into_validating_json() - .map_err(ProtocolError::ValueError)?; - - let new_defs_json = Value::from(new_defs_map) - .try_into_validating_json() - .map_err(ProtocolError::ValueError)?; - - let old_defs_schema = json!({ - "$defs": old_defs_json - }); - - let new_defs_schema = json!({ - "$defs": new_defs_json - }); - - // We do not allow to remove or modify $ref in document type schemas - // it means that compatible changes in $defs won't break the overall compatibility - // Make sure that updated $defs schema is compatible - let compatibility_validation_result = validate_schema_compatibility( - &old_defs_schema, - &new_defs_schema, - platform_version, - )?; - - if !compatibility_validation_result.is_valid() { - let errors = compatibility_validation_result - .errors - .into_iter() - .map(|operation| { - IncompatibleDataContractSchemaError::new( - self.id(), - operation.name, - operation.path, - ) - .into() - }) - .collect(); - - return Ok(SimpleConsensusValidationResult::new_with_errors(errors)); - } - } + let result = + self.validate_update_existing_document_types(new_data_contract, platform_version)?; + if !result.is_valid() { + return Ok(result); } - if self.groups() != new_data_contract.groups() { - // No groups can have been removed - for old_group_position in self.groups().keys() { - if !new_data_contract.groups().contains_key(old_group_position) { - return Ok(SimpleConsensusValidationResult::new_with_error( - DataContractUpdateActionNotAllowedError::new( - self.id(), - "remove group".to_string(), - ) - .into(), - )); - } - } - - // Ensure no group has been changed - for (old_group_position, old_group) in self.groups() { - if let Some(new_group) = new_data_contract.groups().get(old_group_position) { - if old_group != new_group { - return Ok(SimpleConsensusValidationResult::new_with_error( - DataContractUpdateActionNotAllowedError::new( - self.id(), - format!( - "change group at position {} is not allowed", - old_group_position - ), - ) - .into(), - )); - } - } - } + let result = self.validate_update_schema_defs(new_data_contract, platform_version)?; + if !result.is_valid() { + return Ok(result); } - if self.tokens() != new_data_contract.tokens() { - for (token_position, old_token_config) in self.tokens() { - // Check if a token has been removed - if !new_data_contract.tokens().contains_key(token_position) { - return Ok(SimpleConsensusValidationResult::new_with_error( - DataContractUpdateActionNotAllowedError::new( - self.id(), - format!("remove token at position {}", token_position), - ) - .into(), - )); - } - - // Check if a token configuration has been changed - if let Some(new_token_config) = new_data_contract.tokens().get(token_position) { - if old_token_config != new_token_config { - return Ok(SimpleConsensusValidationResult::new_with_error( - DataContractUpdateActionNotAllowedError::new( - self.id(), - format!("update token at position {}", token_position), - ) - .into(), - )); - } - } - } - - // Validate any newly added tokens - for (token_contract_position, token_configuration) in new_data_contract.tokens() { - if !self.tokens().contains_key(token_contract_position) { - if let Some(distribution) = token_configuration - .distribution_rules() - .pre_programmed_distribution() - { - if let Some((timestamp, _)) = distribution.distributions().iter().next() { - if timestamp < &block_info.time_ms { - return Ok(SimpleConsensusValidationResult::new_with_error( - StateError::PreProgrammedDistributionTimestampInPastError( - PreProgrammedDistributionTimestampInPastError::new( - new_data_contract.id(), - *token_contract_position, - *timestamp, - block_info.time_ms, - ), - ) - .into(), - )); - } - } - } - } - } + let result = self.validate_update_groups(new_data_contract); + if !result.is_valid() { + return Ok(result); } - if self.keywords() != new_data_contract.keywords() { - // Validate there are no more than 50 contract keywords - if new_data_contract.keywords().len() > 50 { - return Ok(SimpleConsensusValidationResult::new_with_error( - TooManyKeywordsError::new(self.id(), new_data_contract.keywords().len() as u8) - .into(), - )); - } - - // Validate the keywords are all unique and between 3 and 50 characters - let mut seen_keywords = HashSet::new(); - for keyword in new_data_contract.keywords() { - // First check keyword length - if keyword.len() < 3 || keyword.len() > 50 { - return Ok(SimpleConsensusValidationResult::new_with_error( - InvalidKeywordLengthError::new(self.id(), keyword.to_string()).into(), - )); - } - - if !keyword - .chars() - .all(|c| !c.is_control() && !c.is_whitespace()) - { - // This would mean we have an invalid character - return Ok(SimpleConsensusValidationResult::new_with_error( - InvalidKeywordCharacterError::new( - new_data_contract.id(), - keyword.to_string(), - ) - .into(), - )); - } - - // Then check uniqueness - if !seen_keywords.insert(keyword) { - return Ok(SimpleConsensusValidationResult::new_with_error( - DuplicateKeywordsError::new(self.id(), keyword.to_string()).into(), - )); - } - } + let result = self.validate_update_tokens(new_data_contract, block_info); + if !result.is_valid() { + return Ok(result); } - if self.description() != new_data_contract.description() { - // Validate the description is between 3 and 100 characters - if let Some(description) = new_data_contract.description() { - let char_count = description.chars().count(); - if !(3..=100).contains(&char_count) { - return Ok(SimpleConsensusValidationResult::new_with_error( - InvalidDescriptionLengthError::new(self.id(), description.to_string()) - .into(), - )); - } - } + let result = self.validate_update_keywords(new_data_contract); + if !result.is_valid() { + return Ok(result); } - Ok(SimpleConsensusValidationResult::new()) + Ok(self.validate_update_description(new_data_contract)) } } @@ -337,7 +67,11 @@ mod tests { use crate::consensus::basic::basic_error::BasicError; use crate::consensus::state::state_error::StateError; use crate::consensus::ConsensusError; + use crate::data_contract::accessors::v0::DataContractV0Getters; + use crate::data_contract::accessors::v1::DataContractV1Getters; use crate::data_contract::config::v0::DataContractConfigSettersV0; + use crate::data_contract::methods::validate_update::DataContractUpdateValidationMethodsV0; + use crate::data_contract::schema::DataContractSchemaMethodsV0; use crate::prelude::IdentityNonce; use crate::tests::fixtures::get_data_contract_fixture; use assert_matches::assert_matches; diff --git a/packages/rs-dpp/src/data_contract/methods/validate_update/v1/mod.rs b/packages/rs-dpp/src/data_contract/methods/validate_update/v1/mod.rs index 9786fa3b95e..89b7c8f9dde 100644 --- a/packages/rs-dpp/src/data_contract/methods/validate_update/v1/mod.rs +++ b/packages/rs-dpp/src/data_contract/methods/validate_update/v1/mod.rs @@ -1,39 +1,21 @@ -use std::collections::HashSet; - use crate::block::block_info::BlockInfo; -use crate::consensus::state::state_error::StateError; -use crate::consensus::state::token::PreProgrammedDistributionTimestampInPastError; +use crate::consensus::basic::data_contract::DataContractInvalidRequiredFieldsUpdateError; use crate::data_contract::accessors::v0::DataContractV0Getters; - -use crate::consensus::basic::data_contract::{ - DataContractInvalidRequiredFieldsUpdateError, DuplicateKeywordsError, - IncompatibleDataContractSchemaError, InvalidDataContractVersionError, - InvalidDescriptionLengthError, InvalidKeywordCharacterError, InvalidKeywordLengthError, - TooManyKeywordsError, -}; -use crate::consensus::state::data_contract::data_contract_update_action_not_allowed_error::DataContractUpdateActionNotAllowedError; -use crate::consensus::state::data_contract::data_contract_update_permission_error::DataContractUpdatePermissionError; -use crate::consensus::state::data_contract::document_type_update_error::DocumentTypeUpdateError; -use crate::data_contract::accessors::v1::DataContractV1Getters; -use crate::data_contract::associated_token::token_configuration::accessors::v0::TokenConfigurationV0Getters; -use crate::data_contract::associated_token::token_distribution_rules::accessors::v0::TokenDistributionRulesV0Getters; -use crate::data_contract::associated_token::token_pre_programmed_distribution::accessors::v0::TokenPreProgrammedDistributionV0Methods; use crate::data_contract::document_type::accessors::DocumentTypeV0Getters; -use crate::data_contract::document_type::schema::validate_schema_compatibility; -use crate::data_contract::schema::DataContractSchemaMethodsV0; use crate::data_contract::DataContract; use crate::validation::SimpleConsensusValidationResult; use crate::ProtocolError; -use platform_value::Value; use platform_version::version::PlatformVersion; -use serde_json::json; impl DataContract { /// Generation 1 (protocol version 14, `requiredSince`). Differences from /// generation 0: /// - the new contract version is passed into per-document-type update /// validation, whose own generation 1 admits required-set additions - /// annotated with `requiredSince` equal to that version; + /// annotated with `requiredSince` equal to that version (shared + /// behavior — the per-type dispatcher resolves the generation from the + /// platform version, so generation 0 of this method never observes + /// it); /// - document types introduced by the update — which have no old /// counterpart for the per-type pass to see — get their `requiredSince` /// annotations validated here: each must name exactly the version this @@ -45,88 +27,59 @@ impl DataContract { block_info: &BlockInfo, platform_version: &PlatformVersion, ) -> Result { - // Check if the contract is owned by the same identity - if self.owner_id() != new_data_contract.owner_id() { - return Ok(SimpleConsensusValidationResult::new_with_error( - DataContractUpdatePermissionError::new(self.id(), new_data_contract.owner_id()) - .into(), - )); + let result = self.validate_update_ownership_and_version(new_data_contract); + if !result.is_valid() { + return Ok(result); } - // Check version is bumped - // Failure (version != previous version + 1): Keep ST and transform it to a nonce bump action. - // How: A user pushed an update that was not the next version. - - let new_version = new_data_contract.version(); - let old_version = self.version(); - if new_version < old_version || new_version - old_version != 1 { - return Ok(SimpleConsensusValidationResult::new_with_error( - InvalidDataContractVersionError::new(old_version + 1, new_version).into(), - )); + let result = self.validate_update_config(new_data_contract, platform_version)?; + if !result.is_valid() { + return Ok(result); } - // Validate that the config was not updated - // * Includes verifications that: - // - Old contract is not read_only - // - New contract is not read_only - // - Keeps history did not change - // - Can be deleted did not change - // - Documents keep history did not change - // - Documents can be deleted contract default did not change - // - Documents mutable contract default did not change - // - Requires identity encryption bounded key did not change - // - Requires identity decryption bounded key did not change - // * Failure (contract does not exist): Keep ST and transform it to a nonce bump action. - // * How: A user pushed an update to a contract that changed its configuration. + let result = + self.validate_update_existing_document_types(new_data_contract, platform_version)?; + if !result.is_valid() { + return Ok(result); + } - let config_validation_result = self.config().validate_update( - new_data_contract.config(), - self.id(), - platform_version, - )?; + let result = self.validate_update_new_document_types_required_since(new_data_contract); + if !result.is_valid() { + return Ok(result); + } - if !config_validation_result.is_valid() { - return Ok(SimpleConsensusValidationResult::new_with_errors( - config_validation_result.errors, - )); + let result = self.validate_update_schema_defs(new_data_contract, platform_version)?; + if !result.is_valid() { + return Ok(result); } - // Validate updates for existing document types to make sure that previously created - // documents will be still valid with a new version of the data contract - for (document_type_name, old_document_type) in self.document_types() { - // Make sure that existing document aren't removed - let Some(new_document_type) = - new_data_contract.document_type_optional_for_name(document_type_name) - else { - return Ok(SimpleConsensusValidationResult::new_with_error( - DocumentTypeUpdateError::new( - self.id(), - document_type_name, - "document type can't be removed", - ) - .into(), - )); - }; + let result = self.validate_update_groups(new_data_contract); + if !result.is_valid() { + return Ok(result); + } - // Validate document type update rules - let validate_update_result = old_document_type.as_ref().validate_update( - new_document_type, - new_data_contract.version(), - platform_version, - )?; + let result = self.validate_update_tokens(new_data_contract, block_info); + if !result.is_valid() { + return Ok(result); + } - if !validate_update_result.is_valid() { - return Ok(SimpleConsensusValidationResult::new_with_errors( - validate_update_result.errors, - )); - } + let result = self.validate_update_keywords(new_data_contract); + if !result.is_valid() { + return Ok(result); } - // Document types introduced by this update have no old counterpart, - // so the per-type update validation above never sees them. Their - // `requiredSince` annotations must name the version this update - // creates — anything else would pre-schedule (or backdate) a - // wire-layout change without validation. + Ok(self.validate_update_description(new_data_contract)) + } + + /// Document types introduced by this update have no old counterpart, + /// so the per-type update validation never sees them. Their + /// `requiredSince` annotations must name the version this update + /// creates — anything else would pre-schedule (or backdate) a + /// wire-layout change without validation. + fn validate_update_new_document_types_required_since( + &self, + new_data_contract: &DataContract, + ) -> SimpleConsensusValidationResult { for (document_type_name, new_document_type) in new_data_contract.document_types() { if self .document_type_optional_for_name(document_type_name) @@ -137,7 +90,7 @@ impl DataContract { for (property_name, property) in new_document_type.as_ref().properties() { if let Some(required_since) = property.required_since { if required_since != new_data_contract.version() { - return Ok(SimpleConsensusValidationResult::new_with_error( + return SimpleConsensusValidationResult::new_with_error( DataContractInvalidRequiredFieldsUpdateError::new( document_type_name.clone(), format!( @@ -146,217 +99,13 @@ impl DataContract { ), ) .into(), - )); + ); } } } } - // Schema $defs should be compatible - if let Some(old_defs_map) = self.schema_defs() { - // If new contract doesn't have $defs, it means that it's $defs was removed and compatibility is broken - let Some(new_defs_map) = new_data_contract.schema_defs() else { - return Ok(SimpleConsensusValidationResult::new_with_error( - IncompatibleDataContractSchemaError::new( - self.id(), - "remove".to_string(), - "/$defs".to_string(), - ) - .into(), - )); - }; - - // If $defs is updated we need to make sure that our data contract is still compatible - // with previously created data - if old_defs_map != new_defs_map { - // both new and old $defs already validated as a part of new and old contract - let old_defs_json = Value::from(old_defs_map) - .try_into_validating_json() - .map_err(ProtocolError::ValueError)?; - - let new_defs_json = Value::from(new_defs_map) - .try_into_validating_json() - .map_err(ProtocolError::ValueError)?; - - let old_defs_schema = json!({ - "$defs": old_defs_json - }); - - let new_defs_schema = json!({ - "$defs": new_defs_json - }); - - // We do not allow to remove or modify $ref in document type schemas - // it means that compatible changes in $defs won't break the overall compatibility - // Make sure that updated $defs schema is compatible - let compatibility_validation_result = validate_schema_compatibility( - &old_defs_schema, - &new_defs_schema, - platform_version, - )?; - - if !compatibility_validation_result.is_valid() { - let errors = compatibility_validation_result - .errors - .into_iter() - .map(|operation| { - IncompatibleDataContractSchemaError::new( - self.id(), - operation.name, - operation.path, - ) - .into() - }) - .collect(); - - return Ok(SimpleConsensusValidationResult::new_with_errors(errors)); - } - } - } - - if self.groups() != new_data_contract.groups() { - // No groups can have been removed - for old_group_position in self.groups().keys() { - if !new_data_contract.groups().contains_key(old_group_position) { - return Ok(SimpleConsensusValidationResult::new_with_error( - DataContractUpdateActionNotAllowedError::new( - self.id(), - "remove group".to_string(), - ) - .into(), - )); - } - } - - // Ensure no group has been changed - for (old_group_position, old_group) in self.groups() { - if let Some(new_group) = new_data_contract.groups().get(old_group_position) { - if old_group != new_group { - return Ok(SimpleConsensusValidationResult::new_with_error( - DataContractUpdateActionNotAllowedError::new( - self.id(), - format!( - "change group at position {} is not allowed", - old_group_position - ), - ) - .into(), - )); - } - } - } - } - - if self.tokens() != new_data_contract.tokens() { - for (token_position, old_token_config) in self.tokens() { - // Check if a token has been removed - if !new_data_contract.tokens().contains_key(token_position) { - return Ok(SimpleConsensusValidationResult::new_with_error( - DataContractUpdateActionNotAllowedError::new( - self.id(), - format!("remove token at position {}", token_position), - ) - .into(), - )); - } - - // Check if a token configuration has been changed - if let Some(new_token_config) = new_data_contract.tokens().get(token_position) { - if old_token_config != new_token_config { - return Ok(SimpleConsensusValidationResult::new_with_error( - DataContractUpdateActionNotAllowedError::new( - self.id(), - format!("update token at position {}", token_position), - ) - .into(), - )); - } - } - } - - // Validate any newly added tokens - for (token_contract_position, token_configuration) in new_data_contract.tokens() { - if !self.tokens().contains_key(token_contract_position) { - if let Some(distribution) = token_configuration - .distribution_rules() - .pre_programmed_distribution() - { - if let Some((timestamp, _)) = distribution.distributions().iter().next() { - if timestamp < &block_info.time_ms { - return Ok(SimpleConsensusValidationResult::new_with_error( - StateError::PreProgrammedDistributionTimestampInPastError( - PreProgrammedDistributionTimestampInPastError::new( - new_data_contract.id(), - *token_contract_position, - *timestamp, - block_info.time_ms, - ), - ) - .into(), - )); - } - } - } - } - } - } - - if self.keywords() != new_data_contract.keywords() { - // Validate there are no more than 50 contract keywords - if new_data_contract.keywords().len() > 50 { - return Ok(SimpleConsensusValidationResult::new_with_error( - TooManyKeywordsError::new(self.id(), new_data_contract.keywords().len() as u8) - .into(), - )); - } - - // Validate the keywords are all unique and between 3 and 50 characters - let mut seen_keywords = HashSet::new(); - for keyword in new_data_contract.keywords() { - // First check keyword length - if keyword.len() < 3 || keyword.len() > 50 { - return Ok(SimpleConsensusValidationResult::new_with_error( - InvalidKeywordLengthError::new(self.id(), keyword.to_string()).into(), - )); - } - - if !keyword - .chars() - .all(|c| !c.is_control() && !c.is_whitespace()) - { - // This would mean we have an invalid character - return Ok(SimpleConsensusValidationResult::new_with_error( - InvalidKeywordCharacterError::new( - new_data_contract.id(), - keyword.to_string(), - ) - .into(), - )); - } - - // Then check uniqueness - if !seen_keywords.insert(keyword) { - return Ok(SimpleConsensusValidationResult::new_with_error( - DuplicateKeywordsError::new(self.id(), keyword.to_string()).into(), - )); - } - } - } - - if self.description() != new_data_contract.description() { - // Validate the description is between 3 and 100 characters - if let Some(description) = new_data_contract.description() { - let char_count = description.chars().count(); - if !(3..=100).contains(&char_count) { - return Ok(SimpleConsensusValidationResult::new_with_error( - InvalidDescriptionLengthError::new(self.id(), description.to_string()) - .into(), - )); - } - } - } - - Ok(SimpleConsensusValidationResult::new()) + SimpleConsensusValidationResult::new() } } From c4bfc3f6d5b340277e7043073942a9afac853e81 Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Wed, 26 Aug 2026 09:38:27 +0200 Subject: [PATCH 13/23] style(dpp): import requiredSince helpers instead of spelling out crate paths MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Call sites used fully qualified crate::data_contract::document_type::… paths inline; import the items and call them bare (or with one module qualifier) per repo style. Co-Authored-By: Claude Fable 5 --- .../class_methods/try_from_schema/mod.rs | 15 +++----------- .../src/data_contract/document_type/mod.rs | 16 +++++++-------- .../src/data_contract/v0/serialization/mod.rs | 20 ++++++------------- .../src/data_contract/v1/serialization/mod.rs | 20 ++++++------------- 4 files changed, 22 insertions(+), 49 deletions(-) diff --git a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs index e711c5a3ca5..41e5da7c184 100644 --- a/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/class_methods/try_from_schema/mod.rs @@ -432,6 +432,7 @@ mod tests { use super::*; use crate::data_contract::config::DataContractConfig; use crate::data_contract::document_type::accessors::DocumentTypeV0Getters; + use crate::data_contract::document_type::validate_required_since_within_contract_version; use platform_value::string_encoding::Encoding; use serde_json::json; @@ -984,20 +985,10 @@ mod tests { document_types.insert("msg".to_string(), document_type); assert!( - crate::data_contract::document_type::validate_required_since_within_contract_version( - &document_types, - 1 - ) - .is_err(), + validate_required_since_within_contract_version(&document_types, 1).is_err(), "requiredSince 2 must be rejected on a version 1 contract even through $ref" ); - assert!( - crate::data_contract::document_type::validate_required_since_within_contract_version( - &document_types, - 2 - ) - .is_ok() - ); + assert!(validate_required_since_within_contract_version(&document_types, 2).is_ok()); } #[test] diff --git a/packages/rs-dpp/src/data_contract/document_type/mod.rs b/packages/rs-dpp/src/data_contract/document_type/mod.rs index 61d755827c0..0d07108d5df 100644 --- a/packages/rs-dpp/src/data_contract/document_type/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/mod.rs @@ -22,6 +22,7 @@ pub mod v2; #[cfg(feature = "validation")] pub(crate) mod validator; +use crate::consensus::basic::data_contract::DataContractInvalidRequiredFieldsUpdateError; use crate::data_contract::document_type::methods::{ DocumentTypeBasicMethods, DocumentTypeV0Methods, }; @@ -59,22 +60,19 @@ pub const CONTRACT_VERSION_STAMP_MAX_SIZE: u16 = 5; pub(crate) fn validate_required_since_within_contract_version( document_types: &std::collections::BTreeMap, contract_version: u32, -) -> Result<(), crate::consensus::basic::data_contract::DataContractInvalidRequiredFieldsUpdateError> -{ +) -> Result<(), DataContractInvalidRequiredFieldsUpdateError> { use crate::data_contract::document_type::accessors::DocumentTypeV0Getters; for (document_type_name, document_type) in document_types { for (property_name, property) in document_type.as_ref().properties() { if let Some(required_since) = property.required_since { if required_since > contract_version { - return Err( - crate::consensus::basic::data_contract::DataContractInvalidRequiredFieldsUpdateError::new( - document_type_name.clone(), - format!( - "property '{property_name}' carries requiredSince {required_since} which exceeds the contract version {contract_version}" - ), + return Err(DataContractInvalidRequiredFieldsUpdateError::new( + document_type_name.clone(), + format!( + "property '{property_name}' carries requiredSince {required_since} which exceeds the contract version {contract_version}" ), - ); + )); } } } diff --git a/packages/rs-dpp/src/data_contract/v0/serialization/mod.rs b/packages/rs-dpp/src/data_contract/v0/serialization/mod.rs index ce860414232..a17abd6d74d 100644 --- a/packages/rs-dpp/src/data_contract/v0/serialization/mod.rs +++ b/packages/rs-dpp/src/data_contract/v0/serialization/mod.rs @@ -1,3 +1,5 @@ +use crate::data_contract::document_type::class_methods::consensus_or_protocol_required_fields_error; +use crate::data_contract::document_type::validate_required_since_within_contract_version; use crate::data_contract::document_type::DocumentType; use crate::data_contract::serialized_version::v0::DataContractInSerializationFormatV0; use crate::data_contract::serialized_version::DataContractInSerializationFormat; @@ -101,13 +103,8 @@ impl DataContractV0 { platform_version, )?; - crate::data_contract::document_type::validate_required_since_within_contract_version( - &document_types, - version, - ) - .map_err( - crate::data_contract::document_type::class_methods::consensus_or_protocol_required_fields_error, - )?; + validate_required_since_within_contract_version(&document_types, version) + .map_err(consensus_or_protocol_required_fields_error)?; let data_contract = DataContractV0 { id, @@ -152,13 +149,8 @@ impl DataContractV0 { platform_version, )?; - crate::data_contract::document_type::validate_required_since_within_contract_version( - &document_types, - version, - ) - .map_err( - crate::data_contract::document_type::class_methods::consensus_or_protocol_required_fields_error, - )?; + validate_required_since_within_contract_version(&document_types, version) + .map_err(consensus_or_protocol_required_fields_error)?; let data_contract = DataContractV0 { id, diff --git a/packages/rs-dpp/src/data_contract/v1/serialization/mod.rs b/packages/rs-dpp/src/data_contract/v1/serialization/mod.rs index 19b28ef4fc9..dadcf2f8256 100644 --- a/packages/rs-dpp/src/data_contract/v1/serialization/mod.rs +++ b/packages/rs-dpp/src/data_contract/v1/serialization/mod.rs @@ -1,3 +1,5 @@ +use crate::data_contract::document_type::class_methods::consensus_or_protocol_required_fields_error; +use crate::data_contract::document_type::validate_required_since_within_contract_version; use crate::data_contract::document_type::DocumentType; use crate::data_contract::serialized_version::v0::DataContractInSerializationFormatV0; use crate::data_contract::serialized_version::DataContractInSerializationFormat; @@ -100,13 +102,8 @@ impl DataContractV1 { platform_version, )?; - crate::data_contract::document_type::validate_required_since_within_contract_version( - &document_types, - version, - ) - .map_err( - crate::data_contract::document_type::class_methods::consensus_or_protocol_required_fields_error, - )?; + validate_required_since_within_contract_version(&document_types, version) + .map_err(consensus_or_protocol_required_fields_error)?; let data_contract = DataContractV1 { id, @@ -169,13 +166,8 @@ impl DataContractV1 { platform_version, )?; - crate::data_contract::document_type::validate_required_since_within_contract_version( - &document_types, - version, - ) - .map_err( - crate::data_contract::document_type::class_methods::consensus_or_protocol_required_fields_error, - )?; + validate_required_since_within_contract_version(&document_types, version) + .map_err(consensus_or_protocol_required_fields_error)?; let data_contract = DataContractV1 { id, From fc476375fd124b1c63334038ecb1aadf150da6ee Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Wed, 26 Aug 2026 09:45:37 +0200 Subject: [PATCH 14/23] refactor(dpp): move validate_required_since_within_contract_version to its own file Relocate the invariant from document_type/mod.rs into a module named after it, re-exported so call sites are unchanged. No behavior change. Co-Authored-By: Claude Fable 5 --- .../src/data_contract/document_type/mod.rs | 39 +------------------ ..._required_since_within_contract_version.rs | 39 +++++++++++++++++++ 2 files changed, 41 insertions(+), 37 deletions(-) create mode 100644 packages/rs-dpp/src/data_contract/document_type/validate_required_since_within_contract_version.rs diff --git a/packages/rs-dpp/src/data_contract/document_type/mod.rs b/packages/rs-dpp/src/data_contract/document_type/mod.rs index 0d07108d5df..007486eba68 100644 --- a/packages/rs-dpp/src/data_contract/document_type/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/mod.rs @@ -16,13 +16,14 @@ pub mod restricted_creation; pub mod schema; mod token_costs; +mod validate_required_since_within_contract_version; +pub(crate) use validate_required_since_within_contract_version::validate_required_since_within_contract_version; pub mod v0; pub mod v1; pub mod v2; #[cfg(feature = "validation")] pub(crate) mod validator; -use crate::consensus::basic::data_contract::DataContractInvalidRequiredFieldsUpdateError; use crate::data_contract::document_type::methods::{ DocumentTypeBasicMethods, DocumentTypeV0Methods, }; @@ -44,42 +45,6 @@ pub const STORAGE_FLAGS_SIZE: usize = 2; /// serialization format 3: a u32 varint. pub const CONTRACT_VERSION_STAMP_MAX_SIZE: u16 = 5; -/// A `requiredSince` annotation may never exceed the version of the contract -/// carrying it — requiredness cannot be pre-scheduled at a future version. -/// Runs over the *parsed* properties, so annotations reached through `$ref` -/// are covered. Called wherever document types are built from a contract's -/// serialized form (creates, updates, and disk loads all pass through -/// there); a no-op for every contract predating the keyword, since their -/// properties carry no annotation. -/// -/// The failure is the dedicated consensus error, because the input is -/// untrusted schema data: state-transition processing must classify it as -/// consensus-invalid (nonce bump), never as an execution error. Callers map -/// it through -/// [`class_methods::consensus_or_protocol_required_fields_error`]. -pub(crate) fn validate_required_since_within_contract_version( - document_types: &std::collections::BTreeMap, - contract_version: u32, -) -> Result<(), DataContractInvalidRequiredFieldsUpdateError> { - use crate::data_contract::document_type::accessors::DocumentTypeV0Getters; - - for (document_type_name, document_type) in document_types { - for (property_name, property) in document_type.as_ref().properties() { - if let Some(required_since) = property.required_since { - if required_since > contract_version { - return Err(DataContractInvalidRequiredFieldsUpdateError::new( - document_type_name.clone(), - format!( - "property '{property_name}' carries requiredSince {required_since} which exceeds the contract version {contract_version}" - ), - )); - } - } - } - } - Ok(()) -} - pub(crate) mod property_names { pub const DOCUMENTS_KEEP_HISTORY: &str = "documentsKeepHistory"; pub const KEEPS_TRANSFER_HISTORY: &str = "keepsTransferHistory"; diff --git a/packages/rs-dpp/src/data_contract/document_type/validate_required_since_within_contract_version.rs b/packages/rs-dpp/src/data_contract/document_type/validate_required_since_within_contract_version.rs new file mode 100644 index 00000000000..98a8b025c94 --- /dev/null +++ b/packages/rs-dpp/src/data_contract/document_type/validate_required_since_within_contract_version.rs @@ -0,0 +1,39 @@ +use std::collections::BTreeMap; + +use crate::consensus::basic::data_contract::DataContractInvalidRequiredFieldsUpdateError; +use crate::data_contract::document_type::accessors::DocumentTypeV0Getters; +use crate::data_contract::document_type::DocumentType; + +/// A `requiredSince` annotation may never exceed the version of the contract +/// carrying it — requiredness cannot be pre-scheduled at a future version. +/// Runs over the *parsed* properties, so annotations reached through `$ref` +/// are covered. Called wherever document types are built from a contract's +/// serialized form (creates, updates, and disk loads all pass through +/// there); a no-op for every contract predating the keyword, since their +/// properties carry no annotation. +/// +/// The failure is the dedicated consensus error, because the input is +/// untrusted schema data: state-transition processing must classify it as +/// consensus-invalid (nonce bump), never as an execution error. Callers map +/// it through +/// [`class_methods::consensus_or_protocol_required_fields_error`](crate::data_contract::document_type::class_methods). +pub(crate) fn validate_required_since_within_contract_version( + document_types: &BTreeMap, + contract_version: u32, +) -> Result<(), DataContractInvalidRequiredFieldsUpdateError> { + for (document_type_name, document_type) in document_types { + for (property_name, property) in document_type.as_ref().properties() { + if let Some(required_since) = property.required_since { + if required_since > contract_version { + return Err(DataContractInvalidRequiredFieldsUpdateError::new( + document_type_name.clone(), + format!( + "property '{property_name}' carries requiredSince {required_since} which exceeds the contract version {contract_version}" + ), + )); + } + } + } + } + Ok(()) +} From dfd20d7bcd0838f9aa0ae9ab8697c5791ab6cb01 Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Wed, 26 Aug 2026 09:48:02 +0200 Subject: [PATCH 15/23] refactor(dpp): validate_update_v1 delegates to the frozen generation 0 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Generation 1 is generation 0 plus the new-document-type requiredSince check, so express it that way: run v0 and append the extra check when it passes. Safe because v0 is shipped and frozen, and the checks are independent and short-circuiting — appending changes only which error is reported when several rules are violated at once, never whether the update is rejected (PV14 is unreleased, so error precedence is still ours to pick). Co-Authored-By: Claude Fable 5 --- .../methods/validate_update/v1/mod.rs | 65 +++++-------------- 1 file changed, 15 insertions(+), 50 deletions(-) diff --git a/packages/rs-dpp/src/data_contract/methods/validate_update/v1/mod.rs b/packages/rs-dpp/src/data_contract/methods/validate_update/v1/mod.rs index 89b7c8f9dde..24eec3afaf5 100644 --- a/packages/rs-dpp/src/data_contract/methods/validate_update/v1/mod.rs +++ b/packages/rs-dpp/src/data_contract/methods/validate_update/v1/mod.rs @@ -8,18 +8,19 @@ use crate::ProtocolError; use platform_version::version::PlatformVersion; impl DataContract { - /// Generation 1 (protocol version 14, `requiredSince`). Differences from - /// generation 0: - /// - the new contract version is passed into per-document-type update - /// validation, whose own generation 1 admits required-set additions - /// annotated with `requiredSince` equal to that version (shared - /// behavior — the per-type dispatcher resolves the generation from the - /// platform version, so generation 0 of this method never observes - /// it); - /// - document types introduced by the update — which have no old - /// counterpart for the per-type pass to see — get their `requiredSince` - /// annotations validated here: each must name exactly the version this - /// update creates. + /// Generation 1 (protocol version 14, `requiredSince`): generation 0 + /// plus validation of `requiredSince` annotations on document types + /// introduced by the update, which have no old counterpart for the + /// per-type pass to see. (Required-set changes on *existing* document + /// types are judged inside the shared per-type dispatcher, which + /// resolves its own generation from the platform version, so this + /// method needs no logic of its own for them.) + /// + /// Delegating to generation 0 is safe because that generation is + /// shipped and therefore frozen. The checks are independent and + /// short-circuiting, so appending the extra one changes only which + /// error is reported when an update violates several rules at once — + /// never whether it is rejected. #[inline(always)] pub(super) fn validate_update_v1( &self, @@ -27,48 +28,12 @@ impl DataContract { block_info: &BlockInfo, platform_version: &PlatformVersion, ) -> Result { - let result = self.validate_update_ownership_and_version(new_data_contract); + let result = self.validate_update_v0(new_data_contract, block_info, platform_version)?; if !result.is_valid() { return Ok(result); } - let result = self.validate_update_config(new_data_contract, platform_version)?; - if !result.is_valid() { - return Ok(result); - } - - let result = - self.validate_update_existing_document_types(new_data_contract, platform_version)?; - if !result.is_valid() { - return Ok(result); - } - - let result = self.validate_update_new_document_types_required_since(new_data_contract); - if !result.is_valid() { - return Ok(result); - } - - let result = self.validate_update_schema_defs(new_data_contract, platform_version)?; - if !result.is_valid() { - return Ok(result); - } - - let result = self.validate_update_groups(new_data_contract); - if !result.is_valid() { - return Ok(result); - } - - let result = self.validate_update_tokens(new_data_contract, block_info); - if !result.is_valid() { - return Ok(result); - } - - let result = self.validate_update_keywords(new_data_contract); - if !result.is_valid() { - return Ok(result); - } - - Ok(self.validate_update_description(new_data_contract)) + Ok(self.validate_update_new_document_types_required_since(new_data_contract)) } /// Document types introduced by this update have no old counterpart, From 9e76e50d536f311f34925eaddfa22054687d2931 Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Wed, 26 Aug 2026 10:03:25 +0200 Subject: [PATCH 16/23] refactor(dpp): name the unconditional-requiredness check always_required MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit required_at(None) at the legacy-format call sites read as a puzzle; give the concept its own method on DocumentProperty — required without a requiredSince gate, the requiredness serialization formats 0-2 encode. Equivalent by definition to required_at(None), which stays for the stamp-aware format 3 read path. Co-Authored-By: Claude Fable 5 --- .../document_type/property/mod.rs | 9 +++++ packages/rs-dpp/src/document/v0/serialize.rs | 40 +++++++++---------- 2 files changed, 29 insertions(+), 20 deletions(-) diff --git a/packages/rs-dpp/src/data_contract/document_type/property/mod.rs b/packages/rs-dpp/src/data_contract/document_type/property/mod.rs index 18a81473b07..ba8f1fb1042 100644 --- a/packages/rs-dpp/src/data_contract/document_type/property/mod.rs +++ b/packages/rs-dpp/src/data_contract/document_type/property/mod.rs @@ -58,6 +58,15 @@ impl DocumentProperty { Some(since) => contract_version.is_some_and(|version| version >= since), } } + + /// Whether this property is required regardless of any document's + /// contract-version stamp: `required` without a `requiredSince` gate. + /// This is the requiredness the pre-stamp serialization formats (0–2) + /// encode, and it equals `required_at(None)` — an unstamped document + /// predates every `requiredSince` annotation. + pub fn always_required(&self) -> bool { + self.required && self.required_since.is_none() + } } #[derive(Debug, PartialEq, Clone, Serialize)] diff --git a/packages/rs-dpp/src/document/v0/serialize.rs b/packages/rs-dpp/src/document/v0/serialize.rs index ad5a14c52b6..65db82e7e6e 100644 --- a/packages/rs-dpp/src/document/v0/serialize.rs +++ b/packages/rs-dpp/src/document/v0/serialize.rs @@ -216,7 +216,7 @@ impl DocumentPlatformSerializationMethodsV0 for DocumentV0 { .try_for_each(|(field_name, property)| { if let Some(value) = self.properties.get(field_name) { if value.is_null() { - if property.required_at(None) && !property.transient { + if property.always_required() && !property.transient { Err(ProtocolError::DataContractError( DataContractError::MissingRequiredKey( "a required field is not present".to_string(), @@ -229,24 +229,24 @@ impl DocumentPlatformSerializationMethodsV0 for DocumentV0 { Ok(()) } } else { - if !property.required_at(None) || property.transient { + if !property.always_required() || property.transient { // dbg!("we added 1", field_name); buffer.push(1); } let value = if property.property_type.is_integer() { DocumentPropertyType::I64 - .encode_value_ref_with_size(value, property.required_at(None)) + .encode_value_ref_with_size(value, property.always_required()) } else { property .property_type - .encode_value_ref_with_size(value, property.required_at(None)) + .encode_value_ref_with_size(value, property.always_required()) }?; // dbg!("we pushed {} with {}", field_name, hex::encode(&value)); buffer.extend(value.as_slice()); Ok(()) } - } else if property.required_at(None) && !property.transient { + } else if property.always_required() && !property.transient { Err(ProtocolError::DataContractError( DataContractError::MissingRequiredKey(format!( "a required field {field_name} is not present" @@ -440,7 +440,7 @@ impl DocumentPlatformSerializationMethodsV0 for DocumentV0 { .try_for_each(|(field_name, property)| { if let Some(value) = self.properties.get(field_name) { if value.is_null() { - if property.required_at(None) && !property.transient { + if property.always_required() && !property.transient { Err(ProtocolError::DataContractError( DataContractError::MissingRequiredKey( "a required field is not present".to_string(), @@ -453,18 +453,18 @@ impl DocumentPlatformSerializationMethodsV0 for DocumentV0 { Ok(()) } } else { - if !property.required_at(None) || property.transient { + if !property.always_required() || property.transient { // dbg!("we added 1", field_name); buffer.push(1); } let value = property .property_type - .encode_value_ref_with_size(value, property.required_at(None))?; + .encode_value_ref_with_size(value, property.always_required())?; // dbg!("we pushed {} with {}", field_name, hex::encode(&value)); buffer.extend(value.as_slice()); Ok(()) } - } else if property.required_at(None) && !property.transient { + } else if property.always_required() && !property.transient { Err(ProtocolError::DataContractError( DataContractError::MissingRequiredKey(format!( "a required field {field_name} is not present" @@ -668,7 +668,7 @@ impl DocumentPlatformSerializationMethodsV0 for DocumentV0 { .try_for_each(|(field_name, property)| { if let Some(value) = self.properties.get(field_name) { if value.is_null() { - if property.required_at(None) && !property.transient { + if property.always_required() && !property.transient { Err(ProtocolError::DataContractError( DataContractError::MissingRequiredKey( "a required field is not present".to_string(), @@ -681,18 +681,18 @@ impl DocumentPlatformSerializationMethodsV0 for DocumentV0 { Ok(()) } } else { - if !property.required_at(None) || property.transient { + if !property.always_required() || property.transient { // dbg!("we added 1", field_name); buffer.push(1); } let value = property .property_type - .encode_value_ref_with_size(value, property.required_at(None))?; + .encode_value_ref_with_size(value, property.always_required())?; // dbg!("we pushed {} with {}", field_name, hex::encode(&value)); buffer.extend(value.as_slice()); Ok(()) } - } else if property.required_at(None) && !property.transient { + } else if property.always_required() && !property.transient { Err(ProtocolError::DataContractError( DataContractError::MissingRequiredKey(format!( "a required field {field_name} is not present" @@ -1114,7 +1114,7 @@ impl DocumentPlatformDeserializationMethodsV0 for DocumentV0 { .iter() .filter_map(|(key, property)| { if finished_buffer { - return if property.required_at(None) && !property.transient { + return if property.always_required() && !property.transient { Some(Err(DataContractError::CorruptedSerialization( "required field after finished buffer".to_string(), ))) @@ -1127,12 +1127,12 @@ impl DocumentPlatformDeserializationMethodsV0 for DocumentV0 { let read_value = if property.property_type.is_integer() { DocumentPropertyType::I64.read_optionally_from( &mut buf, - property.required_at(None) & !property.transient, + property.always_required() & !property.transient, ) } else { property.property_type.read_optionally_from( &mut buf, - property.required_at(None) & !property.transient, + property.always_required() & !property.transient, ) }; @@ -1342,7 +1342,7 @@ impl DocumentPlatformDeserializationMethodsV0 for DocumentV0 { .iter() .filter_map(|(key, property)| { if finished_buffer { - return if property.required_at(None) && !property.transient { + return if property.always_required() && !property.transient { Some(Err(DataContractError::CorruptedSerialization( "required field after finished buffer".to_string(), ))) @@ -1352,7 +1352,7 @@ impl DocumentPlatformDeserializationMethodsV0 for DocumentV0 { } let read_value = property.property_type.read_optionally_from( &mut buf, - property.required_at(None) & !property.transient, + property.always_required() & !property.transient, ); match read_value { @@ -1586,7 +1586,7 @@ impl DocumentPlatformDeserializationMethodsV0 for DocumentV0 { .iter() .filter_map(|(key, property)| { if finished_buffer { - return if property.required_at(None) && !property.transient { + return if property.always_required() && !property.transient { Some(Err(DataContractError::CorruptedSerialization( "required field after finished buffer".to_string(), ))) @@ -1596,7 +1596,7 @@ impl DocumentPlatformDeserializationMethodsV0 for DocumentV0 { } let read_value = property.property_type.read_optionally_from( &mut buf, - property.required_at(None) & !property.transient, + property.always_required() & !property.transient, ); match read_value { From f534bba5a6f3a6e39172fdf6a614f5117314dc16 Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Wed, 26 Aug 2026 10:16:31 +0200 Subject: [PATCH 17/23] test(drive-abci): pin protocol v13 fees on every path the stamp re-baselined Every latest-version fee baseline this PR changed for the contract-version stamp gets a v13 twin proving pre-v14 costs are untouched: document delete/replace (mutable, not-mutable, not-mutable-but-transferable), transfer, delete-after-transfer, token burn group-action confirmer, and direct purchase. Six paths pin the exact pre-stamp value; the mutable replace and delete paths pin lower v13 values whose delta against the pre-stamp v14 baseline predates this PR (the #4380 dashpay payment-address contract changes at v14). Co-Authored-By: Claude Fable 5 --- .../batch/tests/document/deletion.rs | 14 ++++++++ .../batch/tests/document/replacement.rs | 35 +++++++++++++++++++ .../batch/tests/document/transfer.rs | 21 +++++++++++ .../batch/tests/token/burn/mod.rs | 13 +++++++ .../batch/tests/token/direct_selling/mod.rs | 9 +++++ 5 files changed, 92 insertions(+) diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/deletion.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/deletion.rs index 3272136bbd2..76b143e295b 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/deletion.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/deletion.rs @@ -17,6 +17,20 @@ mod deletion_tests { .await; } + /// PROTOCOL_VERSION_13: fee predating every v14 change on this path — + /// both the contract-version stamp (this PR) and the dashpay + /// payment-address contract changes (#4380), which is why it differs + /// from the pre-stamp v14 baseline by more than the stamp bytes. Pinned + /// so v13 chain history stays bit-for-bit reproducible. + #[tokio::test] + async fn test_document_delete_on_document_type_that_is_mutable_and_can_be_deleted_protocol_version_13( + ) { + run_document_delete_on_document_type_that_is_mutable_and_can_be_deleted_at_protocol_version( + 13, 1678920, + ) + .await; + } + /// PROTOCOL_VERSION_11: pre-B7 fee — the transformer's local execution /// context was dropped, so the user wasn't charged for the per-transition /// grovedb reads `try_from_borrowed_*_with_contract_lookup` performs. diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/replacement.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/replacement.rs index 5b6161d1b2b..cc457303672 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/replacement.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/replacement.rs @@ -339,6 +339,17 @@ mod replacement_tests { .await; } + /// PROTOCOL_VERSION_13: fee predating every v14 change on this path — + /// both the contract-version stamp (this PR) and the dashpay + /// payment-address contract changes (#4380), which is why it differs + /// from the pre-stamp v14 baseline by more than the stamp bytes. Pinned + /// so v13 chain history stays bit-for-bit reproducible. + #[tokio::test] + async fn test_document_replace_on_document_type_that_is_mutable_protocol_version_13() { + run_document_replace_on_document_type_that_is_mutable_at_protocol_version(13, 1411320) + .await; + } + /// PROTOCOL_VERSION_11: pre-B7 happy-path fee — transformer's local /// execution context was dropped, so per-transition grovedb reads /// were not billed. Pinned so v11 chain history stays bit-for-bit @@ -1043,6 +1054,16 @@ mod replacement_tests { .await; } + /// PROTOCOL_VERSION_13: pre-stamp fee — document serialization format 3 + /// (the contract-version stamp) activates at v14, so v13 costs must be + /// exactly what they were before the `requiredSince` changes. Pinned so + /// v13 chain history stays bit-for-bit reproducible. + #[tokio::test] + async fn test_document_replace_on_document_type_that_is_not_mutable_protocol_version_13() { + run_document_replace_on_document_type_that_is_not_mutable_at_protocol_version(13, 460920) + .await; + } + /// PROTOCOL_VERSION_11: pre-fix bump-only fee (no charge for the fetch /// + validation work). Pinned so v11 chain history stays bit-for-bit /// reproducible. @@ -1299,6 +1320,20 @@ mod replacement_tests { .await; } + /// PROTOCOL_VERSION_13: pre-stamp fee — document serialization format 3 + /// (the contract-version stamp) activates at v14, so v13 costs must be + /// exactly what they were before the `requiredSince` changes. Pinned so + /// v13 chain history stays bit-for-bit reproducible. + #[tokio::test] + async fn test_document_replace_on_document_type_that_is_not_mutable_but_is_transferable_protocol_version_13( + ) { + run_document_replace_on_document_type_that_is_not_mutable_but_is_transferable_at_protocol_version( + 13, + 457660, + ) + .await; + } + /// PROTOCOL_VERSION_11: pre-B7 bump-only fee (transformer's local /// execution context dropped the per-transition reads). Pinned so /// v11 chain history stays bit-for-bit reproducible. diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/transfer.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/transfer.rs index cc761bad415..e8f51ec6344 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/transfer.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/transfer.rs @@ -504,6 +504,18 @@ mod transfer_tests { .await; } + /// PROTOCOL_VERSION_13: pre-stamp fee — document serialization format 3 + /// (the contract-version stamp) activates at v14, so v13 costs must be + /// exactly what they were before the `requiredSince` changes. Pinned so + /// v13 chain history stays bit-for-bit reproducible. + #[tokio::test] + async fn test_document_transfer_on_document_type_that_is_transferable_protocol_version_13() { + run_document_transfer_on_document_type_that_is_transferable_at_protocol_version( + 13, 3643400, + ) + .await; + } + /// PROTOCOL_VERSION_11: pre-B4 fee — query_documents cost was discarded. /// Pinned so v11 chain history stays bit-for-bit reproducible. #[tokio::test] @@ -1484,6 +1496,15 @@ mod transfer_tests { .await; } + /// PROTOCOL_VERSION_13: pre-stamp fee — document serialization format 3 + /// (the contract-version stamp) activates at v14, so v13 costs must be + /// exactly what they were before the `requiredSince` changes. Pinned so + /// v13 chain history stays bit-for-bit reproducible. + #[tokio::test] + async fn test_document_delete_after_transfer_protocol_version_13() { + run_document_delete_after_transfer_at_protocol_version(13, 4004260).await; + } + /// PROTOCOL_VERSION_11: pre-B4 fee — query_documents cost was discarded. /// Pinned so v11 chain history stays bit-for-bit reproducible. #[tokio::test] diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/token/burn/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/token/burn/mod.rs index 1068a7ad348..32d40c65338 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/token/burn/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/token/burn/mod.rs @@ -3965,6 +3965,19 @@ mod token_burn_tests { .await; } + /// PROTOCOL_VERSION_13: pre-stamp fee — genesis system documents are not + /// stamped before document serialization format 3 (v14), so v13 costs + /// must be exactly what they were before the `requiredSince` changes. + /// Pinned so v13 chain history stays bit-for-bit reproducible. + #[tokio::test] + async fn test_token_burn_group_action_confirmer_fee_includes_transformer_reads_protocol_version_13( + ) { + run_token_burn_group_action_confirmer_fee_includes_transformer_reads_at_protocol_version( + 13, 4_367_880, + ) + .await; + } + /// PROTOCOL_VERSION_11: pre-B7 fee — the transformer's local execution /// context was dropped, so the three group-action drive reads /// (fetch_action_is_closed + diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/token/direct_selling/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/token/direct_selling/mod.rs index 3fdde12cd7c..11451ddcbae 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/token/direct_selling/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/token/direct_selling/mod.rs @@ -32,6 +32,15 @@ mod token_selling_tests { .await; } + /// PROTOCOL_VERSION_13: pre-stamp buyer balance — genesis system documents + /// are not stamped before document serialization format 3 (v14), so v13 + /// costs must be exactly what they were before the `requiredSince` + /// changes. Pinned so v13 chain history stays bit-for-bit reproducible. + #[tokio::test] + async fn test_successful_direct_purchase_single_price_protocol_version_13() { + run_successful_direct_purchase_single_price_at_protocol_version(13, 699_868_073_580).await; + } + /// PROTOCOL_VERSION_11: pre-B4/B7 buyer balance — query_documents + /// transformer-phase reads were dropped, so the buyer paid 7,900 /// credits less in fees. Pinned so v11 chain history stays From 3b42f7f4e80a4834db018d08a0f69b2394cc4ab1 Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Wed, 26 Aug 2026 10:19:57 +0200 Subject: [PATCH 18/23] docs(drive-abci): name the version gate behind the v13 fee deltas The mutable replace/delete v13 pins sit below the pre-stamp v14 baseline because v14 genesis stores the larger dashpay v2 contract (payment addresses, #4380) gated behind SYSTEM_DATA_CONTRACT_VERSIONS_V3; v13 genesis stores dashpay v1. Spell that out so the delta reads as the gated upgrade it is, and so the pin's role as the gate's regression guard is explicit. Co-Authored-By: Claude Fable 5 --- .../batch/tests/document/deletion.rs | 12 ++++++++---- .../batch/tests/document/replacement.rs | 12 ++++++++---- 2 files changed, 16 insertions(+), 8 deletions(-) diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/deletion.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/deletion.rs index 76b143e295b..2a14056d93e 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/deletion.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/deletion.rs @@ -18,10 +18,14 @@ mod deletion_tests { } /// PROTOCOL_VERSION_13: fee predating every v14 change on this path — - /// both the contract-version stamp (this PR) and the dashpay - /// payment-address contract changes (#4380), which is why it differs - /// from the pre-stamp v14 baseline by more than the stamp bytes. Pinned - /// so v13 chain history stays bit-for-bit reproducible. + /// both the contract-version stamp and the dashpay payment-address + /// contract (#4380), whose v2 schema is gated behind + /// `SYSTEM_DATA_CONTRACT_VERSIONS_V3` (v14 only; v13 genesis stores + /// dashpay v1, so its smaller node shifts the byte-billed contracts- + /// subtree reads). That gate is why this value is below the pre-stamp + /// v14 baseline by more than the stamp bytes — and this pin is what + /// fails if the gate ever leaks into v13. Pinned so v13 chain history + /// stays bit-for-bit reproducible. #[tokio::test] async fn test_document_delete_on_document_type_that_is_mutable_and_can_be_deleted_protocol_version_13( ) { diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/replacement.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/replacement.rs index cc457303672..8429b932fea 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/replacement.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/replacement.rs @@ -340,10 +340,14 @@ mod replacement_tests { } /// PROTOCOL_VERSION_13: fee predating every v14 change on this path — - /// both the contract-version stamp (this PR) and the dashpay - /// payment-address contract changes (#4380), which is why it differs - /// from the pre-stamp v14 baseline by more than the stamp bytes. Pinned - /// so v13 chain history stays bit-for-bit reproducible. + /// both the contract-version stamp and the dashpay payment-address + /// contract (#4380), whose v2 schema is gated behind + /// `SYSTEM_DATA_CONTRACT_VERSIONS_V3` (v14 only; v13 genesis stores + /// dashpay v1, so its smaller node shifts the byte-billed contracts- + /// subtree reads). That gate is why this value is below the pre-stamp + /// v14 baseline by more than the stamp bytes — and this pin is what + /// fails if the gate ever leaks into v13. Pinned so v13 chain history + /// stays bit-for-bit reproducible. #[tokio::test] async fn test_document_replace_on_document_type_that_is_mutable_protocol_version_13() { run_document_replace_on_document_type_that_is_mutable_at_protocol_version(13, 1411320) From 820d73ebff8e1169c0c59c0b4ec4da1a7429acc0 Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Wed, 26 Aug 2026 10:40:41 +0200 Subject: [PATCH 19/23] test(dpp): cover requiredSince layouts across property types and annotation versions Two gaps in the format-3 matrix: annotations were only ever exercised on string properties, and only one annotation version ever existed in a document type. New fixture annotates variable and fixed byte arrays, an identifier, integers, a float, a bool, and a nested object at two requiredSince versions; round-trips at every stamp position (before, between, at, absent), asserts the presence-flag layout carries exactly one extra byte per annotated property, and errors on a missing annotated identifier at its stamp. Co-Authored-By: Claude Fable 5 --- packages/rs-dpp/src/document/v0/serialize.rs | 186 +++++++++++++++++++ 1 file changed, 186 insertions(+) diff --git a/packages/rs-dpp/src/document/v0/serialize.rs b/packages/rs-dpp/src/document/v0/serialize.rs index 65db82e7e6e..0273aede95b 100644 --- a/packages/rs-dpp/src/document/v0/serialize.rs +++ b/packages/rs-dpp/src/document/v0/serialize.rs @@ -3590,6 +3590,192 @@ mod tests { ); } + /// A document type whose `requiredSince` annotations sit on properties of + /// every distinct byte layout — variable and fixed byte arrays, an + /// identifier, integers, a float, a bool, and a nested object — at two + /// different annotation versions, so a stamp can fall before, between, + /// and after them: + /// - `a`: string, required at every version + /// - `bytv2`, `fixv2`, `objv2`: required since contract version 2 + /// - `idv3`, `intv3`, `fltv3`, `boolv3`: required since contract version 3 + fn multi_type_required_since_document_type() -> crate::data_contract::document_type::DocumentType + { + use crate::data_contract::config::DataContractConfig; + use crate::data_contract::document_type::DocumentType; + use platform_value::platform_value; + use std::collections::BTreeMap; + + let platform_version = PlatformVersion::latest(); + let schema = platform_value!({ + "type": "object", + "properties": { + "a": {"type": "string", "position": 0, "maxLength": 60_u32}, + "bytv2": {"type": "array", "position": 1, "byteArray": true, "minItems": 0, "maxItems": 32, "requiredSince": 2}, + "fixv2": {"type": "array", "position": 2, "byteArray": true, "minItems": 8, "maxItems": 8, "requiredSince": 2}, + "objv2": { + "type": "object", + "position": 3, + "properties": { + "inner": {"type": "string", "position": 0, "maxLength": 10_u32}, + }, + "required": ["inner"], + "additionalProperties": false, + "requiredSince": 2, + }, + "idv3": {"type": "array", "position": 4, "byteArray": true, "minItems": 32, "maxItems": 32, "contentMediaType": "application/x.dash.dpp.identifier", "requiredSince": 3}, + "intv3": {"type": "integer", "position": 5, "requiredSince": 3}, + "fltv3": {"type": "number", "position": 6, "requiredSince": 3}, + "boolv3": {"type": "boolean", "position": 7, "requiredSince": 3}, + }, + "required": ["a", "bytv2", "fixv2", "objv2", "idv3", "intv3", "fltv3", "boolv3"], + "additionalProperties": false, + }); + let config = DataContractConfig::default_for_version(platform_version) + .expect("should create a default config"); + DocumentType::try_from_schema( + platform_value::Identifier::new([5; 32]), + 3, + config.version(), + "multi", + schema, + None, + &BTreeMap::new(), + &config, + false, + &mut Vec::new(), + platform_version, + ) + .expect("failed to create multi-type document type") + } + + fn multi_type_properties_since_v2() -> BTreeMap { + let mut properties = BTreeMap::new(); + properties.insert("a".to_string(), Value::Text("alpha".to_string())); + properties.insert("bytv2".to_string(), Value::Bytes(vec![1, 2, 3])); + properties.insert("fixv2".to_string(), Value::Bytes(vec![9; 8])); + properties.insert( + "objv2".to_string(), + Value::Map(vec![( + Value::Text("inner".to_string()), + Value::Text("in".to_string()), + )]), + ); + properties + } + + fn multi_type_properties_since_v3() -> BTreeMap { + let mut properties = multi_type_properties_since_v2(); + properties.insert("idv3".to_string(), Value::Identifier([7; 32])); + properties.insert("intv3".to_string(), Value::I64(-42)); + properties.insert("fltv3".to_string(), Value::Float(1.5)); + properties.insert("boolv3".to_string(), Value::Bool(true)); + properties + } + + #[test] + fn serialize_v3_round_trips_annotated_non_string_types_at_every_stamp() { + let platform_version = PlatformVersion::latest(); + let document_type = multi_type_required_since_document_type(); + let document_type_ref = document_type.as_ref(); + + let mut base = BTreeMap::new(); + base.insert("a".to_string(), Value::Text("alpha".to_string())); + + // Unstamped and stamped-at-1: every annotation postdates the bytes, + // so all annotated properties may be absent + for stamp in [None, Some(1)] { + let document = stamped_document(stamp, base.clone(), document_type_ref); + let serialized = document + .serialize_v3(document_type_ref) + .expect("document predating every annotation should serialize"); + let deserialized = + DocumentV0::from_bytes(&serialized, document_type_ref, platform_version) + .expect("expected deserialization to succeed"); + assert_eq!(deserialized, document, "stamp {stamp:?}"); + } + + // Stamped between the two annotation versions: the version-2 group is + // required (raw layout), the version-3 group still optional and absent + let document = + stamped_document(Some(2), multi_type_properties_since_v2(), document_type_ref); + let serialized = document + .serialize_v3(document_type_ref) + .expect("document stamped between annotations should serialize"); + let deserialized = DocumentV0::from_bytes(&serialized, document_type_ref, platform_version) + .expect("expected deserialization to succeed"); + assert_eq!(deserialized, document); + + // Same stamp with the version-3 group present: still optional, so it + // rides the presence-flagged layout and must round-trip + let document = + stamped_document(Some(2), multi_type_properties_since_v3(), document_type_ref); + let serialized = document + .serialize_v3(document_type_ref) + .expect("optional-but-present annotated fields should serialize"); + let deserialized = DocumentV0::from_bytes(&serialized, document_type_ref, platform_version) + .expect("expected deserialization to succeed"); + assert_eq!(deserialized, document); + + // Stamped at the newest annotation: everything required, raw layouts + let document = + stamped_document(Some(3), multi_type_properties_since_v3(), document_type_ref); + let serialized = document + .serialize_v3(document_type_ref) + .expect("document stamped at the newest annotation should serialize"); + let deserialized = DocumentV0::from_bytes(&serialized, document_type_ref, platform_version) + .expect("expected deserialization to succeed"); + assert_eq!(deserialized, document); + + // A stamp at the newest annotation with one of its fields missing + // errors for non-string types just like for strings + let mut missing = multi_type_properties_since_v3(); + missing.remove("idv3"); + let document = stamped_document(Some(3), missing, document_type_ref); + assert!( + matches!( + document.serialize_v3(document_type_ref), + Err(ProtocolError::DataContractError( + DataContractError::MissingRequiredKey(_) + )) + ), + "a stamped-at-annotation document missing an annotated identifier must error" + ); + } + + #[test] + fn serialize_v3_layouts_diverge_between_stamps_across_property_types() { + let platform_version = PlatformVersion::latest(); + let document_type = multi_type_required_since_document_type(); + let document_type_ref = document_type.as_ref(); + + // Identical content, different stamps: at stamp 2 the version-3 group + // is presence-flagged, at stamp 3 it serializes raw — the bytes must + // differ beyond the stamp varint itself, and each layout must decode + // only under its own stamp + let at_2 = stamped_document(Some(2), multi_type_properties_since_v3(), document_type_ref) + .serialize_v3(document_type_ref) + .expect("stamp-2 document should serialize"); + let at_3 = stamped_document(Some(3), multi_type_properties_since_v3(), document_type_ref) + .serialize_v3(document_type_ref) + .expect("stamp-3 document should serialize"); + + // Four version-3 properties drop one presence byte each when the + // stamp makes them required; the stamp varint is one byte in both + assert_eq!( + at_2.len(), + at_3.len() + 4, + "the presence-flagged layout must carry one extra byte per annotated property" + ); + + let from_2 = DocumentV0::from_bytes(&at_2, document_type_ref, platform_version) + .expect("stamp-2 bytes should decode"); + let from_3 = DocumentV0::from_bytes(&at_3, document_type_ref, platform_version) + .expect("stamp-3 bytes should decode"); + assert_eq!(from_2.properties, from_3.properties); + assert_eq!(from_2.contract_version, Some(2)); + assert_eq!(from_3.contract_version, Some(3)); + } + #[test] fn stamp_survives_the_wire_for_documents_stamped_past_required_since() { let platform_version = PlatformVersion::latest(); From a26cea6e48b5cdc6557a934535abf116e213de1e Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Wed, 26 Aug 2026 10:40:41 +0200 Subject: [PATCH 20/23] test(drive-abci): end-to-end grandfathering flow for requiredSince MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The feature's full story through real state-transition processing: a contract update adds a required property with requiredSince 2; documents created before it stay stamped 1, transfer with the stamp preserved untouched, and delete; creates and replaces omitting the property are consensus-rejected while ones carrying it are accepted and stamped 2 — replace being the lazy-migration path that re-stamps a grandfathered document. Co-Authored-By: Claude Fable 5 --- .../batch/tests/document/mod.rs | 1 + .../batch/tests/document/required_since.rs | 517 ++++++++++++++++++ 2 files changed, 518 insertions(+) create mode 100644 packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/required_since.rs diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/mod.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/mod.rs index 714a696d900..06546a41465 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/mod.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/mod.rs @@ -4,6 +4,7 @@ mod dpns; mod nft; mod ranked_group_drain; mod replacement; +mod required_since; mod transfer; use super::*; diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/required_since.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/required_since.rs new file mode 100644 index 00000000000..e68fa9ec0bf --- /dev/null +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/required_since.rs @@ -0,0 +1,517 @@ +//! End-to-end coverage for `requiredSince`: a contract update adds a new +//! required property, and every document lifecycle op runs through real +//! state-transition processing before and after — proving grandfathered +//! documents keep working, new writes are held to the new schema, and the +//! contract-version stamp is assigned, preserved, and refreshed where the +//! design says it must be. + +use super::*; + +mod required_since_tests { + use super::*; + use crate::platform_types::platform_state::PlatformState; + use crate::rpc::core::MockCoreRPCLike; + use crate::test::helpers::setup::TempPlatform; + use dpp::data_contract::accessors::v0::DataContractV0Setters; + use dpp::data_contract::schema::DataContractSchemaMethodsV0; + use dpp::document::Document; + use dpp::identity::identity_public_key::accessors::v0::IdentityPublicKeyGettersV0; + use dpp::platform_value::platform_value; + use dpp::prelude::DataContract; + use dpp::state_transition::data_contract_update_transition::methods::DataContractUpdateTransitionMethodsV0; + use dpp::state_transition::data_contract_update_transition::DataContractUpdateTransition; + use dpp::state_transition::StateTransition; + use dpp::tests::fixtures::get_data_contract_fixture; + use drive::util::storage_flags::StorageFlags; + + /// The full grandfathering story, in order: + /// 1. contract v1 with a `note` document type (one required property); + /// 2. two documents created — both stamped with contract version 1; + /// 3. the update to v2 adds required `extra` with `requiredSince: 2`; + /// 4. a create omitting `extra` is consensus-rejected, one carrying it is + /// accepted and stamped 2; + /// 5. the grandfathered documents — which do not have `extra` — still + /// transfer (stamp 1 preserved through the server-side rewrite) and + /// still delete; + /// 6. replacing a grandfathered document must supply `extra` (rejected + /// without it) and re-stamps it to 2 — the lazy migration path. + #[tokio::test] + async fn test_contract_update_adding_required_field_grandfathers_existing_documents() { + let platform_version = PlatformVersion::latest(); + let mut platform = TestPlatformBuilder::new() + .build_with_mock_rpc() + .set_initial_state_structure(); + let platform_state = platform.state.load(); + + let (identity, signer, key) = setup_identity(&mut platform, 958, dash_to_credits!(0.5)); + let (receiver, receiver_signer, receiver_key) = + setup_identity(&mut platform, 450, dash_to_credits!(0.5)); + + // ------------------------------------------------------------------ + // Contract v1: `note` has a single required property `message` + // ------------------------------------------------------------------ + let mut contract = + get_data_contract_fixture(Some(identity.id()), 0, platform_version.protocol_version) + .data_contract_owned(); + + let note_schema_v1 = platform_value!({ + "type": "object", + "documentsMutable": true, + "canBeDeleted": true, + "transferable": 1, + "properties": { + "message": {"type": "string", "position": 0, "maxLength": 100_u32}, + }, + "required": ["message"], + "additionalProperties": false + }); + + contract + .set_document_schema( + "note", + note_schema_v1, + true, + &mut Vec::new(), + platform_version, + ) + .expect("expected to add the note document type"); + + platform + .drive + .apply_contract( + &contract, + BlockInfo::default(), + true, + StorageFlags::optional_default_as_cow(), + None, + platform_version, + ) + .expect("expected to apply contract"); + + let note_type_v1 = contract + .document_type_for_name("note") + .expect("expected the note document type"); + + let mut rng = StdRng::seed_from_u64(433); + + // ------------------------------------------------------------------ + // Two documents under contract v1 — the grandfathered generation + // ------------------------------------------------------------------ + let mut grandfathered = Vec::new(); + for (nonce, seed_text) in [(1, "first note"), (2, "second note")] { + let entropy = Bytes32::random_with_rng(&mut rng); + let mut document = note_type_v1 + .random_document_with_identifier_and_entropy( + &mut rng, + identity.id(), + entropy, + DocumentFieldFillType::DoNotFillIfNotRequired, + DocumentFieldFillSize::AnyDocumentFillSize, + platform_version, + ) + .expect("expected a random document"); + document.set("message", seed_text.into()); + + let transition = BatchTransition::new_document_creation_transition_from_document( + document.clone(), + note_type_v1, + entropy.0, + &key, + nonce, + 0, + None, + &signer, + platform_version, + None, + ) + .await + .expect("expected a creation transition"); + + let result = process_and_commit(&mut platform, &platform_state, &transition).await; + assert_matches!( + result, + StateTransitionExecutionResult::SuccessfulExecution { .. }, + "creating a document under contract v1 must succeed" + ); + grandfathered.push(document); + } + + let stored = query_notes(&platform, &contract, platform_version); + assert_eq!(stored.len(), 2); + for document in &stored { + assert_eq!( + document.contract_version(), + Some(1), + "documents created under contract v1 must be stamped 1" + ); + } + + // ------------------------------------------------------------------ + // The update: v2 adds required `extra` with `requiredSince: 2` + // ------------------------------------------------------------------ + let note_schema_v2 = platform_value!({ + "type": "object", + "documentsMutable": true, + "canBeDeleted": true, + "transferable": 1, + "properties": { + "message": {"type": "string", "position": 0, "maxLength": 100_u32}, + "extra": {"type": "string", "position": 1, "maxLength": 50_u32, "requiredSince": 2}, + }, + "required": ["message", "extra"], + "additionalProperties": false + }); + + let mut updated_contract = contract.clone(); + updated_contract.set_version(2); + updated_contract + .set_document_schema( + "note", + note_schema_v2, + true, + &mut Vec::new(), + platform_version, + ) + .expect("expected to update the note document type"); + + let update_transition = DataContractUpdateTransition::new_from_data_contract( + updated_contract.clone(), + &identity.clone().into_partial_identity_info(), + key.id(), + 3, + 0, + &signer, + platform_version, + None, + ) + .await + .expect("expected an update transition"); + + let result = process_and_commit_serialized( + &mut platform, + &platform_state, + update_transition + .serialize_to_bytes() + .expect("expected serialized update"), + ) + .await; + assert_matches!( + result, + StateTransitionExecutionResult::SuccessfulExecution { .. }, + "adding a required property with requiredSince = new version must be accepted" + ); + + let note_type_v2 = updated_contract + .document_type_for_name("note") + .expect("expected the updated note document type"); + + // ------------------------------------------------------------------ + // New creates: without `extra` rejected, with it accepted + stamped 2 + // ------------------------------------------------------------------ + let entropy = Bytes32::random_with_rng(&mut rng); + let mut incomplete = note_type_v2 + .random_document_with_identifier_and_entropy( + &mut rng, + identity.id(), + entropy, + DocumentFieldFillType::DoNotFillIfNotRequired, + DocumentFieldFillSize::AnyDocumentFillSize, + platform_version, + ) + .expect("expected a random document"); + incomplete.set("message", "no extra".into()); + incomplete.remove("extra"); + + let transition = BatchTransition::new_document_creation_transition_from_document( + incomplete, + note_type_v2, + entropy.0, + &key, + 4, + 0, + None, + &signer, + platform_version, + None, + ) + .await + .expect("expected a creation transition"); + + let result = process_and_commit(&mut platform, &platform_state, &transition).await; + assert_matches!( + result, + StateTransitionExecutionResult::PaidConsensusError { ref error, .. } + if error.to_string().contains("extra"), + "a create missing the newly required property must be consensus-rejected" + ); + + let entropy = Bytes32::random_with_rng(&mut rng); + let mut complete = note_type_v2 + .random_document_with_identifier_and_entropy( + &mut rng, + identity.id(), + entropy, + DocumentFieldFillType::DoNotFillIfNotRequired, + DocumentFieldFillSize::AnyDocumentFillSize, + platform_version, + ) + .expect("expected a random document"); + complete.set("message", "with extra".into()); + complete.set("extra", "present".into()); + let complete_id = complete.id(); + + let transition = BatchTransition::new_document_creation_transition_from_document( + complete, + note_type_v2, + entropy.0, + &key, + 5, + 0, + None, + &signer, + platform_version, + None, + ) + .await + .expect("expected a creation transition"); + + let result = process_and_commit(&mut platform, &platform_state, &transition).await; + assert_matches!( + result, + StateTransitionExecutionResult::SuccessfulExecution { .. }, + "a create carrying the newly required property must succeed" + ); + + let stored = query_notes(&platform, &updated_contract, platform_version); + let stored_complete = stored + .iter() + .find(|d| d.id() == complete_id) + .expect("expected the new document to be stored"); + assert_eq!( + stored_complete.contract_version(), + Some(2), + "documents created under contract v2 must be stamped 2" + ); + + // ------------------------------------------------------------------ + // Grandfathered transfer: succeeds, stamp 1 preserved untouched + // ------------------------------------------------------------------ + let transferred_id = grandfathered[0].id(); + let mut to_transfer = grandfathered[0].clone(); + to_transfer.set_revision(Some(2)); + let transition = BatchTransition::new_document_transfer_transition_from_document( + to_transfer, + note_type_v2, + receiver.id(), + &key, + 6, + 0, + None, + &signer, + platform_version, + None, + ) + .await + .expect("expected a transfer transition"); + + let result = process_and_commit(&mut platform, &platform_state, &transition).await; + assert_matches!( + result, + StateTransitionExecutionResult::SuccessfulExecution { .. }, + "a grandfathered document without the new property must stay transferable" + ); + + let stored = query_notes(&platform, &updated_contract, platform_version); + let transferred = stored + .iter() + .find(|d| d.id() == transferred_id) + .expect("expected the transferred document to be stored"); + assert_eq!( + transferred.owner_id(), + receiver.id(), + "the transfer must have moved ownership" + ); + assert_eq!( + transferred.contract_version(), + Some(1), + "a transfer re-serializes the fetched document without touching it, \ + so the stamp must stay at the version its bytes conform to" + ); + assert!( + !transferred.properties().contains_key("extra"), + "the grandfathered document must still omit the new property" + ); + + // ------------------------------------------------------------------ + // Grandfathered replace: must supply `extra`, and re-stamps to 2 + // ------------------------------------------------------------------ + let mut replacement_missing_extra = grandfathered[1].clone(); + replacement_missing_extra.set_revision(Some(2)); + replacement_missing_extra.set("message", "still no extra".into()); + + let transition = BatchTransition::new_document_replacement_transition_from_document( + replacement_missing_extra, + note_type_v2, + &key, + 7, + 0, + None, + &signer, + platform_version, + None, + ) + .await + .expect("expected a replacement transition"); + + let result = process_and_commit(&mut platform, &platform_state, &transition).await; + assert_matches!( + result, + StateTransitionExecutionResult::PaidConsensusError { ref error, .. } + if error.to_string().contains("extra"), + "a replace re-supplies full content, so it must carry the newly required property" + ); + + let replaced_id = grandfathered[1].id(); + let mut replacement = grandfathered[1].clone(); + replacement.set_revision(Some(2)); + replacement.set("message", "migrated".into()); + replacement.set("extra", "now present".into()); + + let transition = BatchTransition::new_document_replacement_transition_from_document( + replacement, + note_type_v2, + &key, + 8, + 0, + None, + &signer, + platform_version, + None, + ) + .await + .expect("expected a replacement transition"); + + let result = process_and_commit(&mut platform, &platform_state, &transition).await; + assert_matches!( + result, + StateTransitionExecutionResult::SuccessfulExecution { .. }, + "a replace carrying the newly required property must succeed" + ); + + let stored = query_notes(&platform, &updated_contract, platform_version); + let replaced = stored + .iter() + .find(|d| d.id() == replaced_id) + .expect("expected the replaced document to be stored"); + assert_eq!( + replaced.contract_version(), + Some(2), + "a replace re-supplies content, so the document must be re-stamped — lazy migration" + ); + assert_eq!( + replaced + .properties() + .get_str("extra") + .expect("expected the migrated property"), + "now present" + ); + + // ------------------------------------------------------------------ + // Grandfathered delete: the transferred stamp-1 document still deletes + // ------------------------------------------------------------------ + let mut to_delete = grandfathered[0].clone(); + to_delete.set_owner_id(receiver.id()); + + let transition = BatchTransition::new_document_deletion_transition_from_document( + to_delete, + note_type_v2, + &receiver_key, + 1, + 0, + None, + &receiver_signer, + platform_version, + None, + ) + .await + .expect("expected a deletion transition"); + + let result = process_and_commit(&mut platform, &platform_state, &transition).await; + assert_matches!( + result, + StateTransitionExecutionResult::SuccessfulExecution { .. }, + "a grandfathered document must stay deletable under the new schema" + ); + + let stored = query_notes(&platform, &updated_contract, platform_version); + assert!( + stored.iter().all(|d| d.id() != transferred_id), + "the deleted document must be gone" + ); + } + + async fn process_and_commit( + platform: &mut TempPlatform, + platform_state: &PlatformState, + transition: &StateTransition, + ) -> StateTransitionExecutionResult { + process_and_commit_serialized( + platform, + platform_state, + transition + .serialize_to_bytes() + .expect("expected serialized transition"), + ) + .await + } + + async fn process_and_commit_serialized( + platform: &mut TempPlatform, + platform_state: &PlatformState, + serialized: Vec, + ) -> StateTransitionExecutionResult { + let platform_version = platform_state + .current_platform_version() + .expect("expected the current platform version"); + let transaction = platform.drive.grove.start_transaction(); + let processing_result = platform + .platform + .process_raw_state_transitions( + &[serialized], + platform_state, + &BlockInfo::default(), + &transaction, + platform_version, + false, + None, + ) + .expect("expected to process the state transition"); + platform + .drive + .grove + .commit_transaction(transaction) + .unwrap() + .expect("expected to commit the transaction"); + processing_result.into_execution_results().remove(0) + } + + fn query_notes( + platform: &TempPlatform, + contract: &DataContract, + platform_version: &PlatformVersion, + ) -> Vec { + let query = DriveDocumentQuery::from_sql_expr( + "select * from note", + contract, + Some(&platform.config.drive), + platform_version, + ) + .expect("expected a document query"); + platform + .drive + .query_documents(query, None, false, None, None) + .expect("expected a query result") + .documents() + .to_vec() + } +} From 5bbdc5ddf738ed25bd9ffdac310c1543d71ebccf Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Wed, 26 Aug 2026 10:57:32 +0200 Subject: [PATCH 21/23] test(drive-abci): cover the v13-to-v14 upgrade boundary for requiredSince MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A document written at protocol v13 (serialization format 2, no stamp) crosses the upgrade: at v14 the contract gains a required requiredSince property, the pre-upgrade document still transfers — rewritten in format 3 but deliberately unstamped, since its bytes predate every annotation — and replacing it re-supplies content and stamps it at the current contract version. The exact shape of mainnet data crossing v14 activation. Co-Authored-By: Claude Fable 5 --- .../batch/tests/document/required_since.rs | 265 ++++++++++++++++++ 1 file changed, 265 insertions(+) diff --git a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/required_since.rs b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/required_since.rs index e68fa9ec0bf..211bddb97bf 100644 --- a/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/required_since.rs +++ b/packages/rs-drive-abci/src/execution/validation/state_transition/state_transitions/batch/tests/document/required_since.rs @@ -450,6 +450,271 @@ mod required_since_tests { ); } + /// The protocol-upgrade boundary: a document written at protocol v13 + /// (serialization format 2, no stamp) keeps working after the chain + /// upgrades to v14 and the contract gains a required `requiredSince` + /// property — it transfers with no stamp acquired (its bytes predate + /// every annotation), and replacing it re-stamps it at the current + /// contract version. This is the exact shape of mainnet data crossing + /// the v14 activation. + #[tokio::test] + async fn test_documents_created_at_v13_survive_the_v14_upgrade_and_required_field_update() { + let platform_version_13 = PlatformVersion::get(13).expect("expected protocol version 13"); + let mut platform = TestPlatformBuilder::new() + .with_initial_protocol_version(13) + .build_with_mock_rpc() + .set_initial_state_structure(); + let platform_state = platform.state.load(); + + let (identity, signer, key) = setup_identity(&mut platform, 958, dash_to_credits!(0.5)); + let (receiver, receiver_signer, receiver_key) = + setup_identity(&mut platform, 450, dash_to_credits!(0.5)); + + // Contract v1 at protocol v13 — no requiredSince anywhere (the v13 + // meta-schema does not admit the keyword) + let mut contract = + get_data_contract_fixture(Some(identity.id()), 0, platform_version_13.protocol_version) + .data_contract_owned(); + + let note_schema_v1 = platform_value!({ + "type": "object", + "documentsMutable": true, + "canBeDeleted": true, + "transferable": 1, + "properties": { + "message": {"type": "string", "position": 0, "maxLength": 100_u32}, + }, + "required": ["message"], + "additionalProperties": false + }); + + contract + .set_document_schema( + "note", + note_schema_v1, + true, + &mut Vec::new(), + platform_version_13, + ) + .expect("expected to add the note document type"); + + platform + .drive + .apply_contract( + &contract, + BlockInfo::default(), + true, + StorageFlags::optional_default_as_cow(), + None, + platform_version_13, + ) + .expect("expected to apply contract"); + + let note_type_v1 = contract + .document_type_for_name("note") + .expect("expected the note document type"); + + let mut rng = StdRng::seed_from_u64(433); + + // A document written at protocol v13: serialization format 2, no stamp + let entropy = Bytes32::random_with_rng(&mut rng); + let mut document = note_type_v1 + .random_document_with_identifier_and_entropy( + &mut rng, + identity.id(), + entropy, + DocumentFieldFillType::DoNotFillIfNotRequired, + DocumentFieldFillSize::AnyDocumentFillSize, + platform_version_13, + ) + .expect("expected a random document"); + document.set("message", "written at v13".into()); + let document_id = document.id(); + + let transition = BatchTransition::new_document_creation_transition_from_document( + document.clone(), + note_type_v1, + entropy.0, + &key, + 1, + 0, + None, + &signer, + platform_version_13, + None, + ) + .await + .expect("expected a creation transition"); + + let result = process_and_commit(&mut platform, &platform_state, &transition).await; + assert_matches!( + result, + StateTransitionExecutionResult::SuccessfulExecution { .. }, + "creating a document at protocol v13 must succeed" + ); + + let stored = query_notes(&platform, &contract, platform_version_13); + assert_eq!(stored.len(), 1); + assert_eq!( + stored[0].contract_version(), + None, + "a document written before serialization format 3 carries no stamp" + ); + + // ------------------------------------------------------------------ + // The chain upgrades to protocol v14 + // ------------------------------------------------------------------ + let mut upgraded_state = platform.state.load().as_ref().clone(); + upgraded_state.set_current_protocol_version_in_consensus(14); + upgraded_state.set_next_epoch_protocol_version(14); + platform.state.store(std::sync::Arc::new(upgraded_state)); + let platform_state = platform.state.load(); + let platform_version_14 = PlatformVersion::get(14).expect("expected protocol version 14"); + + // The contract update adding required `extra` now goes through + let note_schema_v2 = platform_value!({ + "type": "object", + "documentsMutable": true, + "canBeDeleted": true, + "transferable": 1, + "properties": { + "message": {"type": "string", "position": 0, "maxLength": 100_u32}, + "extra": {"type": "string", "position": 1, "maxLength": 50_u32, "requiredSince": 2}, + }, + "required": ["message", "extra"], + "additionalProperties": false + }); + + let mut updated_contract = contract.clone(); + updated_contract.set_version(2); + updated_contract + .set_document_schema( + "note", + note_schema_v2, + true, + &mut Vec::new(), + platform_version_14, + ) + .expect("expected to update the note document type"); + + let update_transition = DataContractUpdateTransition::new_from_data_contract( + updated_contract.clone(), + &identity.clone().into_partial_identity_info(), + key.id(), + 2, + 0, + &signer, + platform_version_14, + None, + ) + .await + .expect("expected an update transition"); + + let result = process_and_commit_serialized( + &mut platform, + &platform_state, + update_transition + .serialize_to_bytes() + .expect("expected serialized update"), + ) + .await; + assert_matches!( + result, + StateTransitionExecutionResult::SuccessfulExecution { .. }, + "the requiredSince update must be accepted after the upgrade" + ); + + let note_type_v2 = updated_contract + .document_type_for_name("note") + .expect("expected the updated note document type"); + + // The format-2 document transfers at v14: rewritten in format 3 but + // still unstamped — its bytes predate every annotation + let mut to_transfer = document.clone(); + to_transfer.set_revision(Some(2)); + let transition = BatchTransition::new_document_transfer_transition_from_document( + to_transfer, + note_type_v2, + receiver.id(), + &key, + 3, + 0, + None, + &signer, + platform_version_14, + None, + ) + .await + .expect("expected a transfer transition"); + + let result = process_and_commit(&mut platform, &platform_state, &transition).await; + assert_matches!( + result, + StateTransitionExecutionResult::SuccessfulExecution { .. }, + "a pre-upgrade document must stay transferable at v14" + ); + + let stored = query_notes(&platform, &updated_contract, platform_version_14); + let transferred = stored + .iter() + .find(|d| d.id() == document_id) + .expect("expected the transferred document"); + assert_eq!(transferred.owner_id(), receiver.id()); + assert_eq!( + transferred.contract_version(), + None, + "a transfer must not stamp a document whose bytes predate format 3" + ); + assert!(!transferred.properties().contains_key("extra")); + + // Replacing it re-supplies content and re-stamps — lazy migration + // across the upgrade boundary + let mut replacement = document.clone(); + replacement.set_owner_id(receiver.id()); + replacement.set_revision(Some(3)); + replacement.set("message", "migrated after upgrade".into()); + replacement.set("extra", "now present".into()); + + let transition = BatchTransition::new_document_replacement_transition_from_document( + replacement, + note_type_v2, + &receiver_key, + 1, + 0, + None, + &receiver_signer, + platform_version_14, + None, + ) + .await + .expect("expected a replacement transition"); + + let result = process_and_commit(&mut platform, &platform_state, &transition).await; + assert_matches!( + result, + StateTransitionExecutionResult::SuccessfulExecution { .. }, + "replacing a pre-upgrade document with the new property must succeed" + ); + + let stored = query_notes(&platform, &updated_contract, platform_version_14); + let replaced = stored + .iter() + .find(|d| d.id() == document_id) + .expect("expected the replaced document"); + assert_eq!( + replaced.contract_version(), + Some(2), + "the replace must stamp the pre-upgrade document at the current contract version" + ); + assert_eq!( + replaced + .properties() + .get_str("extra") + .expect("expected the migrated property"), + "now present" + ); + } + async fn process_and_commit( platform: &mut TempPlatform, platform_state: &PlatformState, From 499ce4c5b038e57cc83b7c97607ee177988e81d8 Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Wed, 26 Aug 2026 10:57:32 +0200 Subject: [PATCH 22/23] test(drive-abci): strategy run with a mid-chain requiredSince contract update A scheduled contract update adds a required property with requiredSince 2 at block 4 while random document inserts fire every block, with per-block proof verification on. Pre-update documents land stamped 1 and survive; post-update inserts generated from the pre-update schema are rejected with the expected JSON-schema code each block; the stored contract ends at version 2 and every surviving document is a grandfathered stamp-1 row. The verify harness rebuilds actions against post-block state, so a create or replace sharing a block with a contract update rebuilds with a stamp that postdates the stored one; the comparison now aligns the stamp in that direction only (a stored stamp exceeding the rebuilt one still fails). Co-Authored-By: Claude Fable 5 --- .../tests/strategy_tests/test_cases/mod.rs | 1 + .../test_cases/required_since_update_tests.rs | 205 ++++++++++++++++ .../verify_state_transitions.rs | 40 ++- ...-all-mutable-add-required-since-field.json | 227 ++++++++++++++++++ 4 files changed, 464 insertions(+), 9 deletions(-) create mode 100644 packages/rs-drive-abci/tests/strategy_tests/test_cases/required_since_update_tests.rs create mode 100644 packages/rs-drive-abci/tests/supporting_files/contract/dashpay/dashpay-contract-all-mutable-add-required-since-field.json diff --git a/packages/rs-drive-abci/tests/strategy_tests/test_cases/mod.rs b/packages/rs-drive-abci/tests/strategy_tests/test_cases/mod.rs index e24e19439e0..dcd6ddd7ea4 100644 --- a/packages/rs-drive-abci/tests/strategy_tests/test_cases/mod.rs +++ b/packages/rs-drive-abci/tests/strategy_tests/test_cases/mod.rs @@ -7,6 +7,7 @@ mod core_update_tests; mod data_contract_history_tests; mod identity_and_document_tests; mod identity_transfer_tests; +mod required_since_update_tests; // TODO: re-enable once OperationType shielded variants are implemented // mod shielded_tests; mod token_tests; diff --git a/packages/rs-drive-abci/tests/strategy_tests/test_cases/required_since_update_tests.rs b/packages/rs-drive-abci/tests/strategy_tests/test_cases/required_since_update_tests.rs new file mode 100644 index 00000000000..203877995ef --- /dev/null +++ b/packages/rs-drive-abci/tests/strategy_tests/test_cases/required_since_update_tests.rs @@ -0,0 +1,205 @@ +//! Multi-block strategy coverage for `requiredSince`: a scheduled contract +//! update adds a new required property mid-run while random document inserts +//! keep firing every block. Documents inserted before the update land stamped +//! with contract version 1 and survive in state; inserts generated from the +//! pre-update schema after it are consensus-rejected for missing the new +//! property. Every block's transitions are proof-verified and the chain's +//! app hashes stay deterministic — the cross-block wiring no scripted test +//! exercises. + +#[cfg(test)] +mod tests { + use crate::execution::run_chain_for_strategy; + use crate::strategy::{FailureStrategy, NetworkStrategy}; + use dash_platform_macros::stack_size; + use dpp::data_contract::accessors::v0::{DataContractV0Getters, DataContractV0Setters}; + use dpp::data_contract::document_type::random_document::{ + DocumentFieldFillSize, DocumentFieldFillType, + }; + use dpp::document::DocumentV0Getters; + use dpp::tests::json_document::json_document_to_created_contract; + use dpp::version::PlatformVersion; + use drive::drive::document::query::QueryDocumentsOutcomeV0Methods; + use drive::query::DriveDocumentQuery; + use drive_abci::config::{ + ChainLockConfig, ExecutionConfig, InstantLockConfig, PlatformConfig, PlatformTestConfig, + ValidatorSetConfig, + }; + use drive_abci::test::helpers::setup::TestPlatformBuilder; + use std::collections::{BTreeMap, HashMap}; + use strategy_tests::frequency::Frequency; + use strategy_tests::operations::{DocumentAction, DocumentOp, Operation, OperationType}; + use strategy_tests::{IdentityInsertInfo, StartAddresses, StartIdentities, Strategy}; + + #[stack_size(4 * 1024 * 1024)] + #[test] + async fn run_chain_contract_update_adds_required_field_mid_run() { + let platform_version = PlatformVersion::latest(); + let created_contract = json_document_to_created_contract( + "tests/supporting_files/contract/dashpay/dashpay-contract-all-mutable.json", + 1, + true, + platform_version, + ) + .expect("expected to get contract from a json document"); + + let mut contract_update = json_document_to_created_contract( + "tests/supporting_files/contract/dashpay/dashpay-contract-all-mutable-add-required-since-field.json", + 2, + true, + platform_version, + ) + .expect("expected to get the updated contract from a json document"); + contract_update.data_contract_mut().set_version(2); + + let contract = created_contract.data_contract(); + + // Inserts are generated from the document type captured here — the + // pre-update schema — every block. Before the update they are valid + // and get stamped with contract version 1; after it they lack the + // newly required `country` and must be consensus-rejected. + let document_op = DocumentOp { + contract: contract.clone(), + action: DocumentAction::DocumentActionInsertRandom( + DocumentFieldFillType::DoNotFillIfNotRequired, + DocumentFieldFillSize::AnyDocumentFillSize, + ), + document_type: contract + .document_type_for_name("contactRequest") + .expect("expected a contactRequest document type") + .to_owned_document_type(), + }; + + let strategy = NetworkStrategy { + strategy: Strategy { + start_contracts: vec![( + created_contract, + Some(BTreeMap::from([(4, contract_update)])), + )], + operations: vec![Operation { + op_type: OperationType::Document(document_op), + frequency: Frequency { + times_per_block_range: 1..2, + chance_per_block: None, + }, + }], + start_identities: StartIdentities::default(), + start_addresses: StartAddresses::default(), + identity_inserts: IdentityInsertInfo { + frequency: Frequency { + times_per_block_range: 1..2, + chance_per_block: None, + }, + ..Default::default() + }, + identity_contract_nonce_gaps: None, + signer: None, + }, + total_hpmns: 100, + extra_normal_mns: 0, + validator_quorum_count: 24, + chain_lock_quorum_count: 24, + upgrading_info: None, + proposer_strategy: Default::default(), + rotate_quorums: false, + failure_testing: Some(FailureStrategy { + deterministic_start_seed: None, + dont_finalize_block: false, + expect_every_block_errors_with_codes: vec![], + rounds_before_successful_block: None, + // From block 5 on, the pre-update generator's documents miss + // the newly required `country`: JSON-schema rejection (10101) + expect_specific_block_errors_with_codes: HashMap::from([ + (5, vec![10101]), + (6, vec![10101]), + (7, vec![10101]), + (8, vec![10101]), + ]), + }), + query_testing: None, + verify_state_transition_results: true, + ..Default::default() + }; + let config = PlatformConfig { + validator_set: ValidatorSetConfig::default_100_67(), + chain_lock: ChainLockConfig::default_100_67(), + instant_lock: InstantLockConfig::default_100_67(), + execution: ExecutionConfig { + verify_sum_trees: true, + ..Default::default() + }, + block_spacing_ms: 3000, + testing_configs: PlatformTestConfig::default_minimal_verifications(), + ..Default::default() + }; + let mut platform = TestPlatformBuilder::new() + .with_config(config.clone()) + .build_with_mock_rpc(); + + let outcome = + run_chain_for_strategy(&mut platform, 8, strategy, config, 15, &mut None, &mut None) + .await; + + // The runner re-derives the contract id at deployment, so read it + // back from the outcome's strategy + let contract_id = outcome + .strategy + .strategy + .start_contracts + .first() + .expect("expected the start contract") + .0 + .data_contract() + .id(); + + // The scheduled update landed: the stored contract is at version 2 + let fetched_contract = outcome + .abci_app + .platform + .drive + .fetch_contract(contract_id.to_buffer(), None, None, None, platform_version) + .unwrap() + .expect("expected to fetch the contract") + .expect("expected the contract to exist"); + assert_eq!( + fetched_contract.contract.version(), + 2, + "the scheduled requiredSince update must have been applied" + ); + + // Every surviving contactRequest document predates the update: stamped with + // contract version 1 and stored without the new required property — + // grandfathered rows living under the version-2 schema + let query = DriveDocumentQuery::from_sql_expr( + "select * from contactRequest", + &fetched_contract.contract, + None, + platform_version, + ) + .expect("expected a document query"); + let documents = outcome + .abci_app + .platform + .drive + .query_documents(query, None, false, None, None) + .expect("expected to query documents") + .documents() + .to_vec(); + + assert!( + !documents.is_empty(), + "documents inserted before the update must survive it" + ); + for document in &documents { + assert_eq!( + document.contract_version(), + Some(1), + "every surviving document predates the update and must be stamped 1" + ); + assert!( + !document.properties().contains_key("country"), + "grandfathered documents must not carry the new property" + ); + } + } +} diff --git a/packages/rs-drive-abci/tests/strategy_tests/verify_state_transitions.rs b/packages/rs-drive-abci/tests/strategy_tests/verify_state_transitions.rs index 6d4de616ad0..55cb580b173 100644 --- a/packages/rs-drive-abci/tests/strategy_tests/verify_state_transitions.rs +++ b/packages/rs-drive-abci/tests/strategy_tests/verify_state_transitions.rs @@ -3,7 +3,7 @@ use dpp::block::block_info::BlockInfo; use dpp::data_contract::accessors::v0::DataContractV0Getters; use dpp::data_contract::config::v0::DataContractConfigGettersV0; use dpp::data_contract::document_type::accessors::DocumentTypeV0Getters; -use dpp::document::{Document, DocumentV0Getters}; +use dpp::document::{Document, DocumentV0Getters, DocumentV0Setters}; use dpp::fee::Credits; use dpp::identity::accessors::IdentityGettersV0; use dpp::identity::identity_public_key::accessors::v0::IdentityPublicKeyGettersV0; @@ -541,15 +541,28 @@ pub(crate) fn verify_state_transitions_were_or_were_not_executed( // ) // .expect("expected to get document") // ); - assert_eq!( - document, + let mut expected_document = Document::try_from_create_transition_action( creation_action, batch_transition.owner_id(), platform_version, ) - .expect("expected to get document") - ); + .expect("expected to get document"); + // The contract-version stamp records the + // contract version at execution time inside + // the block, but this harness rebuilds the + // action against post-block state — so a + // same-block contract update makes the + // rebuilt stamp postdate the stored one. + // Align only in that direction; a stored + // stamp must never exceed the rebuilt one. + if document.contract_version() + < expected_document.contract_version() + { + expected_document + .set_contract_version(document.contract_version()); + } + assert_eq!(document, expected_document); } else { //there is the possibility that the state transition was not executed because it already existed, // we can discount that for now in tests @@ -560,15 +573,24 @@ pub(crate) fn verify_state_transitions_were_or_were_not_executed( if *was_executed { // it's also possible we deleted something we replaced if let Some(document) = document { - assert_eq!( - document, + let mut expected_document = Document::try_from_replace_transition_action( replace_action, batch_transition.owner_id(), platform_version, ) - .expect("expected to get document") - ); + .expect("expected to get document"); + // Same post-block rebuild skew as the + // create arm: align the stamp only when + // the rebuilt one postdates the stored one + if document.contract_version() + < expected_document.contract_version() + { + expected_document.set_contract_version( + document.contract_version(), + ); + } + assert_eq!(document, expected_document); } } else { //there is the possibility that the state transition was not executed and the state is equal to the previous diff --git a/packages/rs-drive-abci/tests/supporting_files/contract/dashpay/dashpay-contract-all-mutable-add-required-since-field.json b/packages/rs-drive-abci/tests/supporting_files/contract/dashpay/dashpay-contract-all-mutable-add-required-since-field.json new file mode 100644 index 00000000000..8e4847a6b7d --- /dev/null +++ b/packages/rs-drive-abci/tests/supporting_files/contract/dashpay/dashpay-contract-all-mutable-add-required-since-field.json @@ -0,0 +1,227 @@ +{ + "$formatVersion": "0", + "id": "8MjTnX7JUbGfYYswyuCtHU7ZqcYU9s1fUaNiqD9s5tEw", + "ownerId": "2QjL594djCH2NyDsn45vd6yQjEDHupMKo7CEGVTHtQxU", + "version": 2, + "documentSchemas": { + "profile": { + "type": "object", + "indices": [ + { + "name": "ownerId", + "properties": [ + { + "$ownerId": "asc" + } + ], + "unique": true + }, + { + "name": "ownerIdUpdatedAt", + "properties": [ + { + "$ownerId": "asc" + }, + { + "$updatedAt": "asc" + } + ] + } + ], + "properties": { + "avatarUrl": { + "type": "string", + "format": "uri", + "maxLength": 2048, + "position": 0 + }, + "publicMessage": { + "type": "string", + "maxLength": 140, + "position": 1 + }, + "displayName": { + "type": "string", + "maxLength": 25, + "position": 2 + } + }, + "required": [ + "$createdAt", + "$updatedAt" + ], + "additionalProperties": false + }, + "contactInfo": { + "type": "object", + "indices": [ + { + "name": "ownerIdKeyIndexes", + "properties": [ + { + "$ownerId": "asc" + }, + { + "rootEncryptionKeyIndex": "asc" + }, + { + "derivationEncryptionKeyIndex": "asc" + } + ], + "unique": true + }, + { + "name": "owner_updated", + "properties": [ + { + "$ownerId": "asc" + }, + { + "$updatedAt": "asc" + } + ] + } + ], + "properties": { + "encToUserId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "position": 0 + }, + "rootEncryptionKeyIndex": { + "type": "integer", + "position": 1 + }, + "derivationEncryptionKeyIndex": { + "type": "integer", + "position": 2 + }, + "privateData": { + "type": "array", + "byteArray": true, + "minItems": 48, + "maxItems": 2048, + "position": 3, + "description": "This is the encrypted values of aliasName + note + displayHidden encoded as an array in cbor" + } + }, + "required": [ + "$createdAt", + "$updatedAt", + "encToUserId", + "privateData", + "rootEncryptionKeyIndex", + "derivationEncryptionKeyIndex" + ], + "additionalProperties": false + }, + "contactRequest": { + "requiresIdentityEncryptionBoundedKey": 2, + "requiresIdentityDecryptionBoundedKey": 2, + "type": "object", + "indices": [ + { + "name": "owner_user_ref", + "properties": [ + { + "$ownerId": "asc" + }, + { + "toUserId": "asc" + }, + { + "accountReference": "asc" + } + ], + "unique": true + }, + { + "name": "ownerId_toUserId", + "properties": [ + { + "$ownerId": "asc" + }, + { + "toUserId": "asc" + } + ] + }, + { + "name": "toUserId_$createdAt", + "properties": [ + { + "toUserId": "asc" + }, + { + "$createdAt": "asc" + } + ] + }, + { + "name": "$ownerId_$createdAt", + "properties": [ + { + "$ownerId": "asc" + }, + { + "$createdAt": "asc" + } + ] + } + ], + "properties": { + "toUserId": { + "type": "array", + "byteArray": true, + "minItems": 32, + "maxItems": 32, + "position": 0 + }, + "encryptedPublicKey": { + "type": "array", + "byteArray": true, + "minItems": 96, + "maxItems": 96, + "position": 1 + }, + "senderKeyIndex": { + "type": "integer", + "position": 2 + }, + "recipientKeyIndex": { + "type": "integer", + "position": 3 + }, + "accountReference": { + "type": "integer", + "position": 4 + }, + "encryptedAccountLabel": { + "type": "array", + "byteArray": true, + "minItems": 48, + "maxItems": 80, + "position": 5 + }, + "country": { + "type": "string", + "maxLength": 50, + "position": 6, + "requiredSince": 2 + } + }, + "required": [ + "$createdAt", + "toUserId", + "encryptedPublicKey", + "senderKeyIndex", + "recipientKeyIndex", + "accountReference", + "country" + ], + "additionalProperties": false + } + } +} From 1500e5ee3bb4abc0d3c480340fc1577d0960fe87 Mon Sep 17 00:00:00 2001 From: Quantum Explorer Date: Wed, 26 Aug 2026 11:09:20 +0200 Subject: [PATCH 23/23] docs(book): document requiredSince, the contract version stamp, and format 3 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit - document-serialization.md: format 3 in the version table, layout diagram, and dispatch; a new section specifying the contract-version stamp and the per-property requiredness resolution rule; a pitfall on stamp-dependent layouts. - data-contracts.md: an authoring-facing section on adding required fields via requiredSince — the consensus rules, grandfathering, and lazy migration semantics. - documents.md: the DocumentV0 contract_version field and getter. - error-codes.md: the Data Contract range now ends at 10276 (DataContractInvalidRequiredFieldsUpdateError). Co-Authored-By: Claude Fable 5 --- book/src/data-model/data-contracts.md | 26 ++++++++++++++ book/src/data-model/documents.md | 4 +++ book/src/error-handling/error-codes.md | 2 +- .../serialization/document-serialization.md | 36 +++++++++++++++++-- 4 files changed, 64 insertions(+), 4 deletions(-) diff --git a/book/src/data-model/data-contracts.md b/book/src/data-model/data-contracts.md index 0f53667ea26..21c4840a6c0 100644 --- a/book/src/data-model/data-contracts.md +++ b/book/src/data-model/data-contracts.md @@ -217,6 +217,32 @@ This intermediate format is important because serialization versions and code st There is also `versioned_limit_deserialize`, which imposes a size limit and always performs full validation -- this is used for data coming from untrusted sources (anything not from Drive's own storage). +## Evolving a Contract: Adding Required Fields + +Contract updates are deliberately conservative: existing documents must stay valid and their stored bytes must stay readable, so most schema changes that would break either are rejected. Historically that froze the `required` set of a document type in both directions — requiredness is baked into the document wire format (required properties serialize raw, optional ones carry a presence flag), so changing it would desynchronize every stored document's bytes from the schema used to read them. + +From protocol v14, an update **may add a brand-new required property** by annotating it with `requiredSince` equal to the contract version the update creates: + +```json +"properties": { + "newField": { "type": "string", "maxLength": 63, "position": 4, "requiredSince": 3 } +}, +"required": ["existingField", "newField"] +``` + +The rules, enforced by consensus (`DataContractInvalidRequiredFieldsUpdateError`, code 10276, on violation): + +- The annotation must name **exactly the new contract version** — requiredness can be neither pre-scheduled for a future version nor backdated. +- Only **brand-new properties** can become required. Promoting an existing (optional) property is still rejected, as is removing anything from `required` or touching an existing `requiredSince` annotation. +- On contract **creation**, `requiredSince` may only be `1`. +- Annotations sit on **top-level properties** listed in `required`; nested properties cannot carry them. + +What happens to data: + +- **Existing documents are grandfathered.** Each document carries a *contract version stamp* recording the contract version its bytes conform to (see [Document Serialization](../serialization/document-serialization.md)); a document stamped below a property's `requiredSince` may omit that property and still reads, transfers, and deletes normally. +- **New writes are held to the new schema.** Creates must supply the property; replaces re-supply full content, so replacing a grandfathered document requires the new property and re-stamps the document at the current version — lazy migration, one document at a time. +- **Indexes are unaffected** because index additions on update remain banned — a newly added required field cannot be indexed retroactively (there is no backfill). + ## Rules and Guidelines **Do:** diff --git a/book/src/data-model/documents.md b/book/src/data-model/documents.md index 0b5d8f3a621..7ea45c75ff2 100644 --- a/book/src/data-model/documents.md +++ b/book/src/data-model/documents.md @@ -37,6 +37,7 @@ pub struct DocumentV0 { pub updated_at_core_block_height: Option, pub transferred_at_core_block_height: Option, pub creator_id: Option, + pub contract_version: Option, } ``` @@ -54,6 +55,8 @@ Let us walk through the key fields: - **`creator_id`**: The original creator of the document. This differs from `owner_id` when a document has been transferred to a new owner. +- **`contract_version`**: The data contract version this document's bytes conform to — the *contract version stamp* (protocol v14+, document serialization format 3). Drive assigns it whenever document content is supplied (create and replace) and preserves it through transfers and purchases. `None` means the document was serialized before format 3, which predates every `requiredSince` annotation. The stamp resolves per-property byte layouts when a document type gains required properties through contract updates — see the [Document Serialization](../serialization/document-serialization.md) chapter. + ## Document ID Generation Document IDs are not random -- they are derived deterministically. From `packages/rs-dpp/src/document/generate_document_id.rs`: @@ -99,6 +102,7 @@ pub trait DocumentV0Getters { fn created_at_block_height(&self) -> Option; fn updated_at_block_height(&self) -> Option; fn creator_id(&self) -> Option; + fn contract_version(&self) -> Option; // ... and more } ``` diff --git a/book/src/error-handling/error-codes.md b/book/src/error-handling/error-codes.md index 018dd014fd0..884dcfae15d 100644 --- a/book/src/error-handling/error-codes.md +++ b/book/src/error-handling/error-codes.md @@ -50,7 +50,7 @@ Error codes are organized into ranges that correspond to error categories and su |-------|----------|----------| | 10000-10099 | Versioning | `UnsupportedVersionError` (10000), `ProtocolVersionParsingError` (10001), `IncompatibleProtocolVersionError` (10004) | | 10100-10199 | Structure | `JsonSchemaCompilationError` (10100), `InvalidIdentifierError` (10102), `ValueError` (10103) | -| 10200-10275 | Data Contract | `DataContractMaxDepthExceedError` (10200), `DuplicateIndexError` (10201), `InvalidDataContractIdError` (10204) | +| 10200-10276 | Data Contract | `DataContractMaxDepthExceedError` (10200), `DuplicateIndexError` (10201), `InvalidDataContractIdError` (10204), `DataContractInvalidRequiredFieldsUpdateError` (10276) | | 10350-10359 | Groups | `GroupPositionDoesNotExistError` (10350), `GroupExceedsMaxMembersError` (10354) | | 10400-10418 | Documents | `DataContractNotPresentError` (10400), `DuplicateDocumentTransitionsWithIdsError` (10401) | | 10450-10460 | Tokens | `InvalidTokenIdError` (10450), `TokenTransferToOurselfError` (10456) | diff --git a/book/src/serialization/document-serialization.md b/book/src/serialization/document-serialization.md index a0b68c76b9e..8e24c32de44 100644 --- a/book/src/serialization/document-serialization.md +++ b/book/src/serialization/document-serialization.md @@ -11,7 +11,10 @@ Every serialized document follows this layout: ```text ┌──────────────────────┐ │ Serialization │ varint (1-2 bytes) -│ Version │ Currently: 0, 1, or 2 +│ Version │ Currently: 0, 1, 2, or 3 +├──────────────────────┤ +│ Contract version │ V3 only: varint +│ stamp │ (0 = unstamped) ├──────────────────────┤ │ $id │ 32 bytes ├──────────────────────┤ @@ -44,8 +47,9 @@ The first bytes of a serialized document are a **varint** encoding the serializa | 0 | Original format. All integers encoded as **i64** (8 bytes big-endian) regardless of their schema type. | | 1 | Integers encoded at their **native size** (u8 = 1 byte, u16 = 2 bytes, u32 = 4 bytes, etc.). Otherwise identical to v0. | | 2 | Same as v1, but adds **`$creatorId`** field after `$ownerId` for document types that support transfers or trading. | +| 3 | Same as v2, but adds a **contract version stamp** varint immediately after the version varint (protocol v14+). The stamp selects each property's layout when the document type carries `requiredSince` annotations — see below. | -The varint encoding uses the [`integer-encoding`](https://docs.rs/integer-encoding) crate's `VarInt` format. For values 0, 1, and 2, the varint is a single byte: `0x00`, `0x01`, or `0x02`. +The varint encoding uses the [`integer-encoding`](https://docs.rs/integer-encoding) crate's `VarInt` format. For values 0 through 3, the varint is a single byte: `0x00`, `0x01`, `0x02`, or `0x03`. ```rust // Serialization version is written first @@ -60,12 +64,36 @@ match serialized_version { 0 => DocumentV0::from_bytes_v0(serialized_document, document_type, platform_version), 1 => DocumentV0::from_bytes_v1(serialized_document, document_type, platform_version), 2 => DocumentV0::from_bytes_v2(serialized_document, document_type, platform_version), + 3 => DocumentV0::from_bytes_v3(serialized_document, document_type, platform_version), _ => Err(/* unknown version */), } ``` Note: version 0 has a fallback — if deserialization as v0 (all i64) fails, it retries as v1 (native integer types). This handles edge cases from protocol versions 1–8 where the version byte was 0 but non-i64 integer types may have been used. +## The contract version stamp (v3) + +Serialization version 3 (the default from protocol v14) writes one extra varint immediately after the version varint: the **contract version stamp** — the version of the data contract the document's bytes conform to. A value of `0` means *unstamped*: the document was originally serialized before format 3 existed and has merely been rewritten in the new envelope (for example by a transfer). + +The stamp exists because contract updates may add new **required** properties from a specific contract version onward, using the `requiredSince` schema keyword: + +```json +"properties": { + "newField": { "type": "string", "maxLength": 63, "position": 4, "requiredSince": 3 } +}, +"required": ["existingField", "newField"] +``` + +Requiredness is baked into the wire format — a required property serializes raw while an optional one carries a presence flag — so a property whose requiredness varies by contract version needs the stamp to resolve its layout. The rule, per property: + +> A property is encoded as **required** (no presence flag) if it is listed in `required` **and** either it has no `requiredSince` annotation, or the document's stamp is **at or above** the annotation. Otherwise it is encoded as optional (presence-flagged). + +An unstamped document (`0`) predates every `requiredSince` annotation, so only unconditionally required properties count as required for it. This means the **latest contract alone** reconstructs the byte layout of every document ever stored — no historical contract lookups are needed. + +The stamp is **platform-assigned**: Drive sets it to the current contract version whenever document content is supplied (create and replace), and preserves it untouched through server-side rewrites that do not re-supply content (transfer and purchase). A document created before a contract update therefore keeps its old stamp — and may legitimately omit properties the newest schema requires — until a replace re-supplies its content and re-stamps it. Clients can also use the stamp as a staleness signal: a document stamped above the client's cached contract version means the contract needs refetching. + +Formats 0–2 have no stamp; documents read from them deserialize with `contract_version = None`, equivalent to a `0` stamp. + ## Field-by-field breakdown ### `$id` (32 bytes) @@ -128,7 +156,7 @@ If the document type's `trade_mode` allows seller-set pricing: Properties are serialized **in schema position order** — each property in the data contract schema has a `position` field, and `document_type.properties()` returns an `IndexMap` sorted by that position. This is *not* alphabetical order. -Each property is encoded based on its type and whether it is required: +Each property is encoded based on its type and whether it is required. In serialization version 3, "required" means *required at the document's contract version stamp* (see above); in versions 0–2 — and for every property without a `requiredSince` annotation — it is simply whether the property is listed in `required`. **Required fields**: The value is written directly with no prefix byte. @@ -255,3 +283,5 @@ See `packages/rs-scripts/README.md` for full usage details. 5. **Optional fields have a presence byte.** If you forget to read the `0x00`/`0x01` prefix for optional fields, every subsequent field will be shifted by one byte. 6. **ByteArray encoding depends on size constraints.** Fixed-size byte arrays (where `minItems == maxItems` in the schema) have no length prefix. Variable-size byte arrays have a varint length prefix. Check the schema to know which encoding is used. + +7. **In version 3, the same document type can produce different property layouts.** A property annotated with `requiredSince` is presence-flagged in documents stamped below the annotation and raw in documents stamped at or above it. Two version-3 documents of the same type may therefore differ in layout — always read the stamp varint and resolve each property's requiredness against it before decoding the properties section.