qkc/slave, cmd/slave: boot a slave's shards and run until interrupted - #38
Open
syntrust wants to merge 35 commits into
Open
qkc/slave, cmd/slave: boot a slave's shards and run until interrupted#38syntrust wants to merge 35 commits into
syntrust wants to merge 35 commits into
Conversation
Boot a slave from a pyquarkchain-compatible cluster_config.json. The slave hosts the shards assigned to one slave identity and performs no network I/O yet (tracking issue #17). - cmd/slave: `config` (validate + print a normalized per-slave summary) and `genesis` (derive the root genesis block, hash byte-identical to pyquarkchain's GenesisManager.create_root_block()) - qkc/config: cluster config loader (load.go) and checked-in singularity mainnet/devnet configs - qkc/genesis: root-block derivation - qkc/types: root block, token balances, helpers - Makefile: `make slave` target Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
…etadata Add the per-shard host the slave boots each of its shards through: - genesis.go: Genesis descriptor built from ShardGenesis (fields + parsed ALLOC) with a Petersburg-only ChainConfig, ChainID = BASE_ETH_CHAIN_ID + chain_id + 1 (pyquarkchain config.py:363); Fingerprint() gives the descriptor a deterministic identity the stub reports as its genesis hash. - services.go: the ShardChain seam (genesis hash, head, Stop) between the slave skeleton and the geth-core shard-chain task, plus the Options injection points for #4 (engine/miner), #5 (master conn), and sync; the stub chain reports head height 0 at the descriptor's identity. - rawdb.go: GenesisMeta record (root-genesis linkage + xshard cursor at (root_height, 0, 0), genesis.py:92) under a single QKC-prefixed key, encoded with qkc/serialize; ReconcileGenesisMeta passes reopen only on an exact match and hard-errors naming both genesis hashes and the db path. Marked temporary: re-implemented, not patched, once QKC block format (#1) lands. - shard.go: Shard{Branch, cfg, db, chain}; New opens an isolated pebble chaindb under {datadir}/shard-0x{full_shard_id}/, reconciles the metadata, and constructs the chain through the seam; Stop is idempotent and closes chain then db. qkc/config gains ETH_CHAIN_ID parsing on ChainConfig: pyquarkchain forces ETH_CHAIN_ID = BASE_ETH_CHAIN_ID + CHAIN_ID + 1 on load (config.py:534), so Validate() accepts a configured value only when consistent. Tests cover the descriptor (table-driven over the mainnet/devnet fixtures), fingerprint determinism/sensitivity, metadata round-trip, reconcile, the milestone demo (boot one shard into a temp dir, reopen idempotently), and the loud mismatch errors for shard- and root-genesis config changes. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
pyquarkchain always derives ETH_CHAIN_ID as BASE_ETH_CHAIN_ID + CHAIN_ID + 1 and overwrites any configured value with that derivation on load (config.py:534); accept a configured value only when consistent with it.
syntrust
changed the title
syntrust
marked this pull request as ready for review
July 14, 2026 11:11
Shard.New logs "existing genesis validated" on reopen and "genesis metadata written" on first boot, using the new existed return value.
An empty DB_PATH_ROOT means an ephemeral in-memory database in pyquarkchain (use_mem_db, cluster_config.py:247) and in goquarkchain (geth's ephemeral-node convention). Previously it silently created a persistent pebble db in the process working directory.
pyquarkchain's arithmetic is unbounded, so BASE_ETH_CHAIN_ID + CHAIN_ID + 1 may exceed uint32; the uint32 sum wrapped silently (worst case to 0, which reads as "absent" and passes any configured value).
… rule set - The ETH_CHAIN_ID derivation is computed in uint64, matching pyquarkchain's unbounded arithmetic; the uint32 sum wrapped silently (worst case to 0, which reads as "absent" and puts a wrong replay-protection chain id into the EVM rule set). - Fingerprint() now encodes the whole ChainConfig instead of just the chain id, so a change to the compiled-in fork schedule changes every fingerprint and forces a loud re-bootstrap instead of executing new semantics on an old db.
ReconcileGenesisMeta no longer writes on a fresh db: Shard.New commits the record once the chain stands at that genesis, so a boot that fails at chain construction leaves no record behind and the retry re-runs the fresh path instead of reporting a never-validated record as existing.
…boot The slave performs no network I/O of any kind, but registering all of geth's debug.Flags made --pprof start an HTTP listener and --pyroscope push to a remote server. Register an allowlisted subset instead: logging and file-based profiling only, fail-closed against upstream additions, with a test pinning the boundary in both directions. Install the SIGINT/SIGTERM handler before any resource opens, so a signal that lands during config load or shard boot still funnels into the blocking Stop() path instead of the OS default termination. A watcher restores default signal handling the moment the first signal lands, keeping the second-signal force-quit available even mid-boot. Covered by a reexec subprocess test that signals at the 'slave booting' line and asserts a clean exit plus a reopenable datadir. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The pattern was added with a leading space, which is significant in gitignore syntax, so it never matched anything. Remove it rather than fix it and restore the trailing newline. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
syntrust
requested review from
blockchaindevsh,
iteyelmp,
ping-ke,
qizhou and
qzhodl
July 15, 2026 11:11
Adopt the final CLI shape in the milestone that creates these files: newApp assembles the app for tests, the cluster_config/node_id flags register globally without Required (actions validate presence via loadClusterConfig), and only non-networked debug flags are exposed, pinned by TestNoNetworkedDebugFlags. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Backport the final shape of the shard package into the milestone that creates it: DBDirName/ParseDBDirName canonicalize the shard chaindb directory names, and the stub-genesis surrogate paths carry their follow-up TODO markers. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Take the finalized cmd/slave CLI shape from slave-m2 and relocate the boot tests it superseded: the fixture helpers, TestBootSlave, and TestBootSlaveRejectsBadNodeID move from slave_test.go (owned by slave-m1) into run_test.go, which also adopts its final goleak TestMain form; run.go and qkc/slave pick up their follow-up TODO markers. main.go keeps only the one-line app.Action wiring on top of slave-m2. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…stone Drop the milestone markers and the not-yet-implemented tracker, adopt the final fixtures/pyquarkchain cross-validation section (including the virtualenv note in the singularity regeneration command), so later milestones only add sections. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The README baseline now carries the final section structure; keep only the 'Running a slave' section on top (its milestone marker dropped with the rest). Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The pyquarkchain derivation snippet already lives (with seal/serialize output) in qkc/config/singularity/README.md; link to that section instead of carrying a trimmed copy. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
The record is a bootstrap stamp standing in for the genesis block, not a block "meta" in the pyquarkchain sense — half its fields live in the header there — so free the word before the QKC block format (#1) ports MinorBlockMeta. The chaindb key moves with it (QKC-genesis-meta -> QKC-genesis-record); no migration by design: the record is delete-not- migrate scaffolding, and a stub-era datadir simply re-stamps on its next boot. Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
syntrust
force-pushed
the
slave-m2
branch
2 times, most recently
from
July 31, 2026 07:36
679e101 to
b6dd7e9
Compare
#44 and #26 rewrote the shard package around the real QKC minor genesis block, so this merge takes the new base wholesale and re-applies only what this milestone owns, ported to it: - The root genesis comes from qkc.CreateRootBlock; the qkc/genesis package is gone. - The injected ChainService follows the current seam, New(db, *types.MinorBlock, *params.ChainConfig). - The README's "Running a slave" section is rewritten against the current output: genesis committed / existing genesis validated per shard, and the refusal that names both hashes when the config changed. Four commits on this branch were superseded upstream and are dropped rather than merged: the GenesisMeta -> GenesisRecord rename and its doc follow-up (no record exists any more), fingerprinting the whole rule set (replaced by storing the chain config under its own key and reconciling it with CheckCompatible), and committing the metadata only after the chain constructs (now carried by the timing of WriteGenesisBlock). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
syntrust
changed the title
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Milestone M3 of #17, on top of #26 (now merged, so this targets
goshard/base).Boots a slave identity end to end:
slave --cluster_config <file> --node_id S0starts every shard the identity owns, runs until SIGINT/SIGTERM, and shuts down cleanly. No network I/O.What's included
qkc/slave:SlaveBackend, a branch-keyed shard registry mirroring pyquarkchain'sSlaveServer. The slave process itself owns no database — all persistent state lives in the per-shard chaindbs, and the registry is rebuilt from config on every boot. A shard that fails to boot rolls back the ones already started (chains stopped, chaindbs closed) before the error returns, so the datadir stays reopenable;Stop()is idempotent and blocks until every shard is down.cmd/slave: the default run action — a drop-in for how pyquarkchain'scluster.pylaunches a slave. The SIGINT/SIGTERM handler is installed before any resource opens, so a signal landing mid-boot still funnels into the blocking shutdown instead of the OS default; a second signal force-quits.cmd/slave/README.md: a "Running a slave" section written against the current output.Test plan
go build ./... && go test -race ./qkc/... ./cmd/slave/— 423 tests / 13 packages green;gofmtandgo vetclean.New tests:
qkc/slave: boot + reopen over the {mainnet, devnet} fixtures, checking the registry against config order; boot rollback through a failingChainServiceinjected viaOptions.Chain(pebble's directory lock proves the databases were closed, since the reboot that follows would otherwise fail); idempotentStop.cmd/slave: thebootSlavepipeline withDB_PATH_ROOTredirected to a temp dir;TestRunHonorsSignalDuringStartupreexecs the binary, sends SIGTERM at theslave bootingline and asserts exit 0 plus a datadir that reopens.Manual run (devnet, S0): first boot logs
genesis committedandshard started genesis=… head=0per shard,^Cexits 0; rerunning against the same datadir logsexisting genesis validated; rerunning it with the mainnet config exits 1 withstored genesis … does not match config genesis … — cluster config changed since initialization; an emptyDB_PATH_ROOTruns both shards on in-memory databases and writes nothing to disk.🤖 Generated with Claude Code