Separate Databases

[emielsebastiaan]

PyJAMaz will require a number of separate databases.
This issue lists these databases:

State K/V Database

• Current K/V database with sigma.

State Trie Database

• Store state trie in separate DB and only update relevant branches #97

Block History Database

• Historical blocks from genesis to finalized head of the chain.
• Allows for reproducing the finalized state. of the chain.
• Allows other (new) nodes to sync.

DA-layer (erasure coding) database

• DA-layer is designed to hold data up to 28 days.
• This implies validators should keep their own data segments for at least this amount of time.

Validator Database

• Per epoch a list of validators is set containing public keys and metadata (ipv6).
• Although the state db contains the previous epoch's validator set, older validator sets are also required for auditing purposes.
• E.g. DA-layer is designed to hold data up to 28 days. This implies validators should keep their data segments for at least this amount of time. If you need to retrieve a erasure coded segment from a particular validator you would need to know its IPv6-address from the (possibly older) validator set. If you need to rebuild the data from the erasure-coded data you would need to know the index of the validator in that particular historical validator set and you would need to know whether this validator has assured data-availability (availability bitfield from assurance extrinsic).
• Discussion: https://matrix.to/#/!wBOJlzaOULZOALhaRh%3Apolkadot.io/%24GM5Zq6ksTbql5yPAA-ocsXcHG_N09UJL6IyGX14r5_A

[arjanz]

Currently, there are two separate databases (namespaces) in PyJAMaz, one for state (sigma) and one for block history.

TBD: the key structure of the block history DB.

Now blocks can be retrieved with key: b'block:[slotnumber]' and then value is the JAM encoded block. But I can also imagine key b'block_slotnumber:[block_hash]' with value slotnumber should be possible. Or to store block primary with block hash instead of slotnumber.