diff --git a/README.md b/README.md index cc2342b..108dede 100644 --- a/README.md +++ b/README.md @@ -52,6 +52,13 @@ conventional neural-weight training. > unattended model execution, protected final custody, formal > MNCS/MNCDS conformance, or automatic RAVEL promotion. +The first one-step distillation reference is also implemented for the +`forge.evidence-relevance` role. It records bounded recurrent teacher observations, +independently checked fixture targets, a tiny affine/tanh student, a no-distillation +control, explicit calibration/OOD abstention, and lineage-bound teacher fallback. The +reference is experimental and diagnostic-only; its measurements do not show that the +student is faster end-to-end or ready to replace the teacher. + The repository also contains a dependency-aware MNCS-family integration reference: `mnel family-integration-reference` exposes a Forge Provider Protocol 0.1 adapter, validates pinned Forge/Fabric/Commons/RAVEL/Language compatibility shapes, executes a bounded local @@ -84,6 +91,17 @@ mnel recurrent-specialist-reference --workspace examples/recurrent-specialists These artifacts are diagnostic-only and do not issue verifier results, permissions, trust, or promotion decisions. +Run the one-step distilled specialist reference with: + +```bash +mnel one-step-specialist-reference --workspace examples/one-step-specialists +``` + +The checked-in example contains the recurrent teacher, distilled student and control +artifacts, retained distillation records, and measured study report. A successful +student call records one learned forward pass; preprocessing, calibration, OOD checks, +and recurrent fallback are measured outside that boundary. + ## Core rule **Investigators and learned providers may propose knowledge. They may not declare it true.** @@ -158,6 +176,10 @@ copy their authority or silently create substitute implementations. - `mnel distill-reference`, a no-network held-out study that writes an evidence ledger and exercises controls, hidden-transfer access, strategy transfer, and a tiny reloadable transition-frequency learned provider. Its observations remain diagnostic. +- `mnel one-step-specialist-reference`, a no-network study that distills the bounded + recurrent Forge relevance teacher into a one-pass affine/tanh student, compares a + classical nearest-centroid baseline and a same-architecture no-distillation control, + and records explicit abstention/fallback measurements. - `mnel provider-study-reference`, a no-network heterogeneous portfolio study that trains a transition-frequency provider and a structurally different tabular nearest-centroid provider, calibrates and reloads both, compares seeded-random and heuristic controls, diff --git a/docs/LEARNED_MICRO_PROVIDERS.md b/docs/LEARNED_MICRO_PROVIDERS.md index 5281c16..ef1de82 100644 --- a/docs/LEARNED_MICRO_PROVIDERS.md +++ b/docs/LEARNED_MICRO_PROVIDERS.md @@ -53,6 +53,31 @@ A learned result is a diagnostic observation. Agreement among several models doe become a verifier verdict. Disagreement is preserved because the pattern of disagreement may be more useful than a vote. +## One-step distilled specialist reference + +MNEL now contains an experimental one-step reference for `forge.evidence-relevance`. +The bounded recurrent specialist is the teacher: its provider/model/generation, +operating-envelope, context-state, query, decision, confidence, iterations, operations, +and decision identity are retained in `mnel-distillation-record/0.1` records. A fixture +label is accepted independently only for the checked-in deterministic reference cases; +teacher rejections, unknown targets, and teacher abstentions remain retained but are not +converted into positive training rows. + +The student is a dependency-free affine/tanh/softmax model. After bounded feature +preparation it performs one learned forward evaluation and emits a structured diagnostic +proposal with calibrated confidence. Low confidence, OOD distance, and incompatible +context cause `ABSTAIN`; the runtime records that abstention before explicitly invoking +the lineage-compatible recurrent teacher as fallback. The student cannot emit verifier, +evaluator, conformance, permission, trust, promotion, or verdict authority. + +`mnel one-step-specialist-reference` compares the iterative teacher, the one-step +student, a deterministic nearest-centroid baseline, and a same-architecture control +trained only from direct fixture labels. It reports correctness against independent +fixture targets, false accepts, abstentions, OOD behavior, disagreements, model/artifact +size, operations, one-pass counts, cold/warm latency, preprocessing cost, and escalated +end-to-end latency. The current synthetic study deliberately does not claim a speed or +accuracy win. + ## Registry contract Every declaration identifies: diff --git a/docs/MNCS_NATIVE_TRAINING_PIPELINE.md b/docs/MNCS_NATIVE_TRAINING_PIPELINE.md index ccee75e..d109f52 100644 --- a/docs/MNCS_NATIVE_TRAINING_PIPELINE.md +++ b/docs/MNCS_NATIVE_TRAINING_PIPELINE.md @@ -287,6 +287,12 @@ MNEL Some components may move downward over time. For example, tensor operations may prove sufficiently general to belong in the standard library rather than MNEL. +The first one-step distilled specialist implementation intentionally remains in MNEL's +Python/reference-runtime layer. It uses a dependency-free bounded numerical trainer and +identity-bound JSON artifacts while MNCS-native training representation and semantic +closure remain future work. No new MNCS syntax, compiler behavior, backend lowering, or +standard-library primitive is required by this reference slice. + ## Backend boundary MNCS-native training does **not** require MNCS to own every low-level implementation on diff --git a/docs/ROADMAP.md b/docs/ROADMAP.md index bc3bce0..5cb9ce0 100644 --- a/docs/ROADMAP.md +++ b/docs/ROADMAP.md @@ -74,6 +74,11 @@ measurements, and diagnostic-only evidence ledgers; - **Implemented:** provider candidate/admission, transfer-pending, quarantine, retirement, and rollback records with explicit evidence checklists; +- **Implemented:** first executable one-step distilled specialist reference for the + Forge evidence-relevance role, including retained recurrent teacher observations, + independently checked target status, a tiny reloadable affine/tanh student, + no-distillation control, calibrated/OOD abstention, explicit teacher fallback, and + measured reference comparisons. This remains diagnostic-only and experimental. - **Started:** broader provider portfolios and native export of Python-trained artifacts; - **Implemented:** a bounded Rust parser/reference inference surface for the existing transition-frequency artifact, with checked-in Python/Rust identity and score-equivalence diff --git a/examples/one-step-specialists/README.md b/examples/one-step-specialists/README.md new file mode 100644 index 0000000..159795f --- /dev/null +++ b/examples/one-step-specialists/README.md @@ -0,0 +1,32 @@ +# One-step distilled specialist reference + +This checked-in study is the first executable MNEL one-step distillation slice for +`forge.evidence-relevance`. + +```text +bounded recurrent teacher + -> retained teacher observations + -> independently checked fixture targets + -> tiny affine/tanh student + -> one learned forward pass or ABSTAIN + -> explicit recurrent teacher fallback +``` + +The files are generated with: + +```bash +mnel one-step-specialist-reference --workspace examples/one-step-specialists +``` + +- `forge-teacher-recurrent-g0.json` is the existing bounded recurrent teacher artifact. +- `forge-student-distilled-g0.json` is the one-step student trained from teacher-derived + decision/confidence targets. +- `forge-student-no-distillation-control-g0.json` uses the same architecture and rows, + but trains only from direct fixture labels. +- `distillation-records.json` retains every teacher observation, including rejected, + unknown, and abstaining cases, plus the selected training targets. +- `reference-study.json` reports the iterative teacher, distilled student, deterministic + nearest-centroid baseline, no-distillation control, calibration, and fallback costs. + +The measurements are diagnostic evidence, not a correctness or promotion verdict. The +synthetic fixture does not currently support a claim that distillation wins end-to-end. diff --git a/examples/one-step-specialists/distillation-records.json b/examples/one-step-specialists/distillation-records.json new file mode 100644 index 0000000..48a9cc1 --- /dev/null +++ b/examples/one-step-specialists/distillation-records.json @@ -0,0 +1,919 @@ +{ + "authority": "diagnostic-only", + "dataset": { + "authority": "diagnostic-only", + "dataset_identity": "sha256:835001ba074393893495cd064fc12eba14ec39d65b8c82fe13bfb47b47bb721c", + "distillation_identity": "sha256:c0d0b68d08870cc8e16c18d25c462d0388d07e5d4d506799f822e6a4e625c8ea", + "observations": [ + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:e18ebe977e94b354a2ce332e44fc6ff26233e8c2fdfd10146045f3799dd6e502", + "independent_target": "relevant", + "independent_target_identity": "sha256:26f920a30f9a5ff32cd2ec0f542087f92b41e5457de52024f028a93f0cba9153", + "observation_identity": "sha256:76610ac42c04b5a73e190f5f8942388587ca5cc7155441b35b5276137262ecd2", + "query_features": [ + 900, + 820, + 760, + 880 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:92c3c6414174102bf9054372a336a2d91124808a490dabecb4d92857e04e1205", + "source_observation_identities": [ + "sha256:92c3c6414174102bf9054372a336a2d91124808a490dabecb4d92857e04e1205" + ], + "target_role": "forge.evidence-relevance", + "target_status": "verified", + "teacher_abstained": false, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "relevant", + "teacher_escalation_reason": null, + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + }, + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:a062ba82f43e1c40695e2f64bae45c60eefe3b440f944788c03cb35dcb45b689", + "independent_target": "relevant", + "independent_target_identity": "sha256:1d5b239f67df7678b3c311fdd8c61b2571ae3444a6d04609f0e0121b7ce3fdc3", + "observation_identity": "sha256:5a74d5dbdc9affc79ed6d35421f89c85e68bf1375339b6e30dd53ecf9f62c9c6", + "query_features": [ + 820, + 760, + 700, + 800 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:326f4f543e4ff49f8e00b8bda62af7b866cdc2fb0c2b5f134c702d655031dc2f", + "source_observation_identities": [ + "sha256:326f4f543e4ff49f8e00b8bda62af7b866cdc2fb0c2b5f134c702d655031dc2f" + ], + "target_role": "forge.evidence-relevance", + "target_status": "verified", + "teacher_abstained": false, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "relevant", + "teacher_escalation_reason": null, + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + }, + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:e1f3597047706cc17a31f67a3da254ef90849ed36364e38d9dfb6d601b273a55", + "independent_target": "relevant", + "independent_target_identity": "sha256:51d1ad63fca894801e49a275dc72922c92993dcc72ca779cdc7aa54337b4e96d", + "observation_identity": "sha256:4241d223e76611c9ca974fc914ebb54124e335b00cf59fd9523a7b1894a088aa", + "query_features": [ + 760, + 700, + 660, + 720 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:0fb47451ee0e775dcd217a71623974f9bf9d2187d25d6dae7fe2c1aac10cc03e", + "source_observation_identities": [ + "sha256:0fb47451ee0e775dcd217a71623974f9bf9d2187d25d6dae7fe2c1aac10cc03e" + ], + "target_role": "forge.evidence-relevance", + "target_status": "verified", + "teacher_abstained": false, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "relevant", + "teacher_escalation_reason": null, + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + }, + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:2f22f7030d0f61016109be48f2e6174e8ca629f4e6e946018c273ae1c3d9c3bd", + "independent_target": "relevant", + "independent_target_identity": "sha256:c6efb29f4ed2fa7c23f7075771855795fe10dc027e95fe6a7e9267458551da05", + "observation_identity": "sha256:96d35d6368875ab858973b959bddae3d655144a0e05831096865f79bb9b86a61", + "query_features": [ + 700, + 740, + 720, + 760 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:249da5b607a1307f45f891644f43fa640c33ac64b2225faa2d572e54f354a2a9", + "source_observation_identities": [ + "sha256:249da5b607a1307f45f891644f43fa640c33ac64b2225faa2d572e54f354a2a9" + ], + "target_role": "forge.evidence-relevance", + "target_status": "verified", + "teacher_abstained": false, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "relevant", + "teacher_escalation_reason": null, + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + }, + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:8f153ac051b34ff023eaa24a47c1a6b8ea54462e906a8a2a9133fa68945718ac", + "independent_target": "irrelevant", + "independent_target_identity": "sha256:73ecb5fa044d13e6b9d53122398033e6457c5541d3b831c9a6b5c23f8e7902ae", + "observation_identity": "sha256:5c4743b99fd1e79eef773d1b826a3613fb9ca5c2e75499ce30eab6816a93702d", + "query_features": [ + 120, + 180, + 160, + 100 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:57d3bb4abc308769f4df3b821187b1caab26aaa998edfbf8a2c6dc8769005d0f", + "source_observation_identities": [ + "sha256:57d3bb4abc308769f4df3b821187b1caab26aaa998edfbf8a2c6dc8769005d0f" + ], + "target_role": "forge.evidence-relevance", + "target_status": "verified", + "teacher_abstained": false, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "irrelevant", + "teacher_escalation_reason": null, + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + }, + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:86a705551ab186e86dbb0d6ff68ac025e220ca1c30f411eecd940da3e5235799", + "independent_target": "irrelevant", + "independent_target_identity": "sha256:19b7551f9a99dfc52d0a25c7cc501e92531c7c18e225d2d295eda7dac4c3a286", + "observation_identity": "sha256:2ba03c5c8b3c8eba9085e24a29c0266cf7193755d5aaf336a751584dc35c0944", + "query_features": [ + 220, + 120, + 180, + 160 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:c4de69391023c32c9b669ce58a22dae6096d17219312c601839142a32dc880ba", + "source_observation_identities": [ + "sha256:c4de69391023c32c9b669ce58a22dae6096d17219312c601839142a32dc880ba" + ], + "target_role": "forge.evidence-relevance", + "target_status": "verified", + "teacher_abstained": false, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "irrelevant", + "teacher_escalation_reason": null, + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + }, + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:0a60c9fda1f3f62543d4956674078f2527163fe50e70a3c479483dd9b03e8845", + "independent_target": "irrelevant", + "independent_target_identity": "sha256:892fe78299e3d3ee47afd90446e9d0cb43b529b9047a6dd673d9dd21f28536a9", + "observation_identity": "sha256:2c8142f197514ba9fb6c922ec0c22cc6d14dbf7f12e54c9c79238feb08629d11", + "query_features": [ + 160, + 220, + 120, + 180 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:31f46b4a6417a9da7c9371a660bd26d61060ae1d808b5cdb1a23bcffdfd12444", + "source_observation_identities": [ + "sha256:31f46b4a6417a9da7c9371a660bd26d61060ae1d808b5cdb1a23bcffdfd12444" + ], + "target_role": "forge.evidence-relevance", + "target_status": "verified", + "teacher_abstained": false, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "irrelevant", + "teacher_escalation_reason": null, + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + }, + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:93d1fccef9e0bcf0f0ed6df9fa599da822c117122c443b280f518df0098d71ba", + "independent_target": "irrelevant", + "independent_target_identity": "sha256:376012d09ee04d438d27ef4af37c1a0ff9ebb247932677bed8019ecc0660e4d3", + "observation_identity": "sha256:d655cc21314aa9387b5e4aaf16dbdf7df996b37b3c94d26571dc5be9ccece880", + "query_features": [ + 260, + 180, + 220, + 200 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:3f57dcdc34c198a49f583ca6d5d3866d8be59f7f846c20de18092fbd136be28c", + "source_observation_identities": [ + "sha256:3f57dcdc34c198a49f583ca6d5d3866d8be59f7f846c20de18092fbd136be28c" + ], + "target_role": "forge.evidence-relevance", + "target_status": "verified", + "teacher_abstained": false, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "irrelevant", + "teacher_escalation_reason": null, + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + }, + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:65f72fde3014fbb6e47b3d2de1c285b4d036a2c6f922b9471d13f6e569649c51", + "independent_target": "irrelevant", + "independent_target_identity": "sha256:522facd6fcec510004975b9d49af6e34b9d85596d856fdeea8717b0230df65bf", + "observation_identity": "sha256:945b9c4c23775285ae76e24de93f4d00908d65195ccef4d61282dd24558d7234", + "query_features": [ + 900, + 820, + 760, + 880 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:f5fcc52f63a7244b93d76cf56c895224c813b7cea4580d4ca299562855ad2867", + "source_observation_identities": [ + "sha256:f5fcc52f63a7244b93d76cf56c895224c813b7cea4580d4ca299562855ad2867" + ], + "target_role": "forge.evidence-relevance", + "target_status": "rejected-teacher-target", + "teacher_abstained": false, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "relevant", + "teacher_escalation_reason": null, + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + }, + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:fadda7542632f09b913a84bf4b7369f72228d074b08affc0da2fdfb8ed65f0bc", + "independent_target": null, + "independent_target_identity": null, + "observation_identity": "sha256:96b62faf7d9c6142ec7fafdae312d1fc36331e9c039ba4a979a78b303f14bf5c", + "query_features": [ + 640, + 620, + 600, + 580 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:e9653b762c5d2d74a65d64a108f4b874add839bf5db72a77407f9527a9d69a55", + "source_observation_identities": [ + "sha256:e9653b762c5d2d74a65d64a108f4b874add839bf5db72a77407f9527a9d69a55" + ], + "target_role": "forge.evidence-relevance", + "target_status": "unknown", + "teacher_abstained": true, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "ABSTAIN", + "teacher_escalation_reason": "out-of-distribution-distance", + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + }, + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:c923379354c042d38f128a46affefd06f45033af1c22e4015b9c24ab52c70988", + "independent_target": "ABSTAIN", + "independent_target_identity": "sha256:08f6cfdfd4a287a9a3c37c7d2eeb5e0a8099a2705506b302619b3aa061b94a39", + "observation_identity": "sha256:de2f232c12960a19a533502991829b113ffcc52d9fa4e81d7f432d0398f11fb8", + "query_features": [ + 500, + 500, + 500, + 500 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:71c76f1756c96604c3e74b6c494922acb6089a17e47288258c44b55eaf34cd42", + "source_observation_identities": [ + "sha256:71c76f1756c96604c3e74b6c494922acb6089a17e47288258c44b55eaf34cd42" + ], + "target_role": "forge.evidence-relevance", + "target_status": "abstention", + "teacher_abstained": true, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "ABSTAIN", + "teacher_escalation_reason": "out-of-distribution-distance", + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + } + ], + "schema": "mnel-distillation-dataset/0.1", + "semantics": "source-preserving-distillation-dataset; verified-target-filtered; not-a-verdict", + "target_role": "forge.evidence-relevance", + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1", + "training_observation_identities": [ + "sha256:76610ac42c04b5a73e190f5f8942388587ca5cc7155441b35b5276137262ecd2", + "sha256:5a74d5dbdc9affc79ed6d35421f89c85e68bf1375339b6e30dd53ecf9f62c9c6", + "sha256:4241d223e76611c9ca974fc914ebb54124e335b00cf59fd9523a7b1894a088aa", + "sha256:96d35d6368875ab858973b959bddae3d655144a0e05831096865f79bb9b86a61", + "sha256:5c4743b99fd1e79eef773d1b826a3613fb9ca5c2e75499ce30eab6816a93702d", + "sha256:2ba03c5c8b3c8eba9085e24a29c0266cf7193755d5aaf336a751584dc35c0944", + "sha256:2c8142f197514ba9fb6c922ec0c22cc6d14dbf7f12e54c9c79238feb08629d11", + "sha256:d655cc21314aa9387b5e4aaf16dbdf7df996b37b3c94d26571dc5be9ccece880" + ], + "training_targets": [ + { + "features": [ + 900, + 820, + 760, + 880 + ], + "independent_target": "relevant", + "observation_identity": "sha256:76610ac42c04b5a73e190f5f8942388587ca5cc7155441b35b5276137262ecd2", + "target_construction": "teacher-decision-and-confidence", + "target_distribution": [ + 0.0, + 1.0 + ], + "teacher_confidence_milli": 1000, + "teacher_target": "relevant" + }, + { + "features": [ + 820, + 760, + 700, + 800 + ], + "independent_target": "relevant", + "observation_identity": "sha256:5a74d5dbdc9affc79ed6d35421f89c85e68bf1375339b6e30dd53ecf9f62c9c6", + "target_construction": "teacher-decision-and-confidence", + "target_distribution": [ + 0.0, + 1.0 + ], + "teacher_confidence_milli": 1000, + "teacher_target": "relevant" + }, + { + "features": [ + 760, + 700, + 660, + 720 + ], + "independent_target": "relevant", + "observation_identity": "sha256:4241d223e76611c9ca974fc914ebb54124e335b00cf59fd9523a7b1894a088aa", + "target_construction": "teacher-decision-and-confidence", + "target_distribution": [ + 0.0, + 1.0 + ], + "teacher_confidence_milli": 1000, + "teacher_target": "relevant" + }, + { + "features": [ + 700, + 740, + 720, + 760 + ], + "independent_target": "relevant", + "observation_identity": "sha256:96d35d6368875ab858973b959bddae3d655144a0e05831096865f79bb9b86a61", + "target_construction": "teacher-decision-and-confidence", + "target_distribution": [ + 0.0, + 1.0 + ], + "teacher_confidence_milli": 1000, + "teacher_target": "relevant" + }, + { + "features": [ + 120, + 180, + 160, + 100 + ], + "independent_target": "irrelevant", + "observation_identity": "sha256:5c4743b99fd1e79eef773d1b826a3613fb9ca5c2e75499ce30eab6816a93702d", + "target_construction": "teacher-decision-and-confidence", + "target_distribution": [ + 1.0, + 0.0 + ], + "teacher_confidence_milli": 1000, + "teacher_target": "irrelevant" + }, + { + "features": [ + 220, + 120, + 180, + 160 + ], + "independent_target": "irrelevant", + "observation_identity": "sha256:2ba03c5c8b3c8eba9085e24a29c0266cf7193755d5aaf336a751584dc35c0944", + "target_construction": "teacher-decision-and-confidence", + "target_distribution": [ + 1.0, + 0.0 + ], + "teacher_confidence_milli": 1000, + "teacher_target": "irrelevant" + }, + { + "features": [ + 160, + 220, + 120, + 180 + ], + "independent_target": "irrelevant", + "observation_identity": "sha256:2c8142f197514ba9fb6c922ec0c22cc6d14dbf7f12e54c9c79238feb08629d11", + "target_construction": "teacher-decision-and-confidence", + "target_distribution": [ + 1.0, + 0.0 + ], + "teacher_confidence_milli": 1000, + "teacher_target": "irrelevant" + }, + { + "features": [ + 260, + 180, + 220, + 200 + ], + "independent_target": "irrelevant", + "observation_identity": "sha256:d655cc21314aa9387b5e4aaf16dbdf7df996b37b3c94d26571dc5be9ccece880", + "target_construction": "teacher-decision-and-confidence", + "target_distribution": [ + 1.0, + 0.0 + ], + "teacher_confidence_milli": 1000, + "teacher_target": "irrelevant" + } + ], + "transform_identity": "sha256:842b068521daf45287696bd770f5479c163126e0b13fa154717f75ebf21dfa2b" + }, + "records": [ + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:e18ebe977e94b354a2ce332e44fc6ff26233e8c2fdfd10146045f3799dd6e502", + "independent_target": "relevant", + "independent_target_identity": "sha256:26f920a30f9a5ff32cd2ec0f542087f92b41e5457de52024f028a93f0cba9153", + "observation_identity": "sha256:76610ac42c04b5a73e190f5f8942388587ca5cc7155441b35b5276137262ecd2", + "query_features": [ + 900, + 820, + 760, + 880 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:92c3c6414174102bf9054372a336a2d91124808a490dabecb4d92857e04e1205", + "source_observation_identities": [ + "sha256:92c3c6414174102bf9054372a336a2d91124808a490dabecb4d92857e04e1205" + ], + "target_role": "forge.evidence-relevance", + "target_status": "verified", + "teacher_abstained": false, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "relevant", + "teacher_escalation_reason": null, + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + }, + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:a062ba82f43e1c40695e2f64bae45c60eefe3b440f944788c03cb35dcb45b689", + "independent_target": "relevant", + "independent_target_identity": "sha256:1d5b239f67df7678b3c311fdd8c61b2571ae3444a6d04609f0e0121b7ce3fdc3", + "observation_identity": "sha256:5a74d5dbdc9affc79ed6d35421f89c85e68bf1375339b6e30dd53ecf9f62c9c6", + "query_features": [ + 820, + 760, + 700, + 800 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:326f4f543e4ff49f8e00b8bda62af7b866cdc2fb0c2b5f134c702d655031dc2f", + "source_observation_identities": [ + "sha256:326f4f543e4ff49f8e00b8bda62af7b866cdc2fb0c2b5f134c702d655031dc2f" + ], + "target_role": "forge.evidence-relevance", + "target_status": "verified", + "teacher_abstained": false, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "relevant", + "teacher_escalation_reason": null, + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + }, + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:e1f3597047706cc17a31f67a3da254ef90849ed36364e38d9dfb6d601b273a55", + "independent_target": "relevant", + "independent_target_identity": "sha256:51d1ad63fca894801e49a275dc72922c92993dcc72ca779cdc7aa54337b4e96d", + "observation_identity": "sha256:4241d223e76611c9ca974fc914ebb54124e335b00cf59fd9523a7b1894a088aa", + "query_features": [ + 760, + 700, + 660, + 720 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:0fb47451ee0e775dcd217a71623974f9bf9d2187d25d6dae7fe2c1aac10cc03e", + "source_observation_identities": [ + "sha256:0fb47451ee0e775dcd217a71623974f9bf9d2187d25d6dae7fe2c1aac10cc03e" + ], + "target_role": "forge.evidence-relevance", + "target_status": "verified", + "teacher_abstained": false, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "relevant", + "teacher_escalation_reason": null, + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + }, + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:2f22f7030d0f61016109be48f2e6174e8ca629f4e6e946018c273ae1c3d9c3bd", + "independent_target": "relevant", + "independent_target_identity": "sha256:c6efb29f4ed2fa7c23f7075771855795fe10dc027e95fe6a7e9267458551da05", + "observation_identity": "sha256:96d35d6368875ab858973b959bddae3d655144a0e05831096865f79bb9b86a61", + "query_features": [ + 700, + 740, + 720, + 760 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:249da5b607a1307f45f891644f43fa640c33ac64b2225faa2d572e54f354a2a9", + "source_observation_identities": [ + "sha256:249da5b607a1307f45f891644f43fa640c33ac64b2225faa2d572e54f354a2a9" + ], + "target_role": "forge.evidence-relevance", + "target_status": "verified", + "teacher_abstained": false, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "relevant", + "teacher_escalation_reason": null, + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + }, + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:8f153ac051b34ff023eaa24a47c1a6b8ea54462e906a8a2a9133fa68945718ac", + "independent_target": "irrelevant", + "independent_target_identity": "sha256:73ecb5fa044d13e6b9d53122398033e6457c5541d3b831c9a6b5c23f8e7902ae", + "observation_identity": "sha256:5c4743b99fd1e79eef773d1b826a3613fb9ca5c2e75499ce30eab6816a93702d", + "query_features": [ + 120, + 180, + 160, + 100 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:57d3bb4abc308769f4df3b821187b1caab26aaa998edfbf8a2c6dc8769005d0f", + "source_observation_identities": [ + "sha256:57d3bb4abc308769f4df3b821187b1caab26aaa998edfbf8a2c6dc8769005d0f" + ], + "target_role": "forge.evidence-relevance", + "target_status": "verified", + "teacher_abstained": false, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "irrelevant", + "teacher_escalation_reason": null, + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + }, + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:86a705551ab186e86dbb0d6ff68ac025e220ca1c30f411eecd940da3e5235799", + "independent_target": "irrelevant", + "independent_target_identity": "sha256:19b7551f9a99dfc52d0a25c7cc501e92531c7c18e225d2d295eda7dac4c3a286", + "observation_identity": "sha256:2ba03c5c8b3c8eba9085e24a29c0266cf7193755d5aaf336a751584dc35c0944", + "query_features": [ + 220, + 120, + 180, + 160 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:c4de69391023c32c9b669ce58a22dae6096d17219312c601839142a32dc880ba", + "source_observation_identities": [ + "sha256:c4de69391023c32c9b669ce58a22dae6096d17219312c601839142a32dc880ba" + ], + "target_role": "forge.evidence-relevance", + "target_status": "verified", + "teacher_abstained": false, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "irrelevant", + "teacher_escalation_reason": null, + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + }, + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:0a60c9fda1f3f62543d4956674078f2527163fe50e70a3c479483dd9b03e8845", + "independent_target": "irrelevant", + "independent_target_identity": "sha256:892fe78299e3d3ee47afd90446e9d0cb43b529b9047a6dd673d9dd21f28536a9", + "observation_identity": "sha256:2c8142f197514ba9fb6c922ec0c22cc6d14dbf7f12e54c9c79238feb08629d11", + "query_features": [ + 160, + 220, + 120, + 180 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:31f46b4a6417a9da7c9371a660bd26d61060ae1d808b5cdb1a23bcffdfd12444", + "source_observation_identities": [ + "sha256:31f46b4a6417a9da7c9371a660bd26d61060ae1d808b5cdb1a23bcffdfd12444" + ], + "target_role": "forge.evidence-relevance", + "target_status": "verified", + "teacher_abstained": false, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "irrelevant", + "teacher_escalation_reason": null, + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + }, + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:93d1fccef9e0bcf0f0ed6df9fa599da822c117122c443b280f518df0098d71ba", + "independent_target": "irrelevant", + "independent_target_identity": "sha256:376012d09ee04d438d27ef4af37c1a0ff9ebb247932677bed8019ecc0660e4d3", + "observation_identity": "sha256:d655cc21314aa9387b5e4aaf16dbdf7df996b37b3c94d26571dc5be9ccece880", + "query_features": [ + 260, + 180, + 220, + 200 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:3f57dcdc34c198a49f583ca6d5d3866d8be59f7f846c20de18092fbd136be28c", + "source_observation_identities": [ + "sha256:3f57dcdc34c198a49f583ca6d5d3866d8be59f7f846c20de18092fbd136be28c" + ], + "target_role": "forge.evidence-relevance", + "target_status": "verified", + "teacher_abstained": false, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "irrelevant", + "teacher_escalation_reason": null, + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + }, + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:65f72fde3014fbb6e47b3d2de1c285b4d036a2c6f922b9471d13f6e569649c51", + "independent_target": "irrelevant", + "independent_target_identity": "sha256:522facd6fcec510004975b9d49af6e34b9d85596d856fdeea8717b0230df65bf", + "observation_identity": "sha256:945b9c4c23775285ae76e24de93f4d00908d65195ccef4d61282dd24558d7234", + "query_features": [ + 900, + 820, + 760, + 880 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:f5fcc52f63a7244b93d76cf56c895224c813b7cea4580d4ca299562855ad2867", + "source_observation_identities": [ + "sha256:f5fcc52f63a7244b93d76cf56c895224c813b7cea4580d4ca299562855ad2867" + ], + "target_role": "forge.evidence-relevance", + "target_status": "rejected-teacher-target", + "teacher_abstained": false, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "relevant", + "teacher_escalation_reason": null, + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + }, + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:fadda7542632f09b913a84bf4b7369f72228d074b08affc0da2fdfb8ed65f0bc", + "independent_target": null, + "independent_target_identity": null, + "observation_identity": "sha256:96b62faf7d9c6142ec7fafdae312d1fc36331e9c039ba4a979a78b303f14bf5c", + "query_features": [ + 640, + 620, + 600, + 580 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:e9653b762c5d2d74a65d64a108f4b874add839bf5db72a77407f9527a9d69a55", + "source_observation_identities": [ + "sha256:e9653b762c5d2d74a65d64a108f4b874add839bf5db72a77407f9527a9d69a55" + ], + "target_role": "forge.evidence-relevance", + "target_status": "unknown", + "teacher_abstained": true, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "ABSTAIN", + "teacher_escalation_reason": "out-of-distribution-distance", + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + }, + { + "authority": "diagnostic-only", + "context_state_identity": "sha256:c1167d46dbfd0fe5957a883da1359643df58f8b53e220a037b85f78ac7aaf4e8", + "decision_identity": "sha256:c923379354c042d38f128a46affefd06f45033af1c22e4015b9c24ab52c70988", + "independent_target": "ABSTAIN", + "independent_target_identity": "sha256:08f6cfdfd4a287a9a3c37c7d2eeb5e0a8099a2705506b302619b3aa061b94a39", + "observation_identity": "sha256:de2f232c12960a19a533502991829b113ffcc52d9fa4e81d7f432d0398f11fb8", + "query_features": [ + 500, + 500, + 500, + 500 + ], + "reasoning_iterations": 4, + "schema": "mnel-distillation-record/0.1", + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + "snapshot_identity": "sha256:71c76f1756c96604c3e74b6c494922acb6089a17e47288258c44b55eaf34cd42", + "source_observation_identities": [ + "sha256:71c76f1756c96604c3e74b6c494922acb6089a17e47288258c44b55eaf34cd42" + ], + "target_role": "forge.evidence-relevance", + "target_status": "abstention", + "teacher_abstained": true, + "teacher_architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "teacher_confidence": 1.0, + "teacher_confidence_milli": 1000, + "teacher_decision": "ABSTAIN", + "teacher_escalation_reason": "out-of-distribution-distance", + "teacher_generation_identity": "sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4", + "teacher_model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "teacher_operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "teacher_operations": 65, + "teacher_provider_identity": "mnel-bounded-recurrent-specialist/0.1" + } + ], + "schema": "mnel-distillation-record-set/0.1", + "semantics": "retained-teacher-observations-and-target-construction; not-a-verdict" +} diff --git a/examples/one-step-specialists/forge-student-distilled-g0.json b/examples/one-step-specialists/forge-student-distilled-g0.json new file mode 100644 index 0000000..295870a --- /dev/null +++ b/examples/one-step-specialists/forge-student-distilled-g0.json @@ -0,0 +1 @@ +{"architecture":{"context_state":"persistent-derived-summary","hidden_width":4,"input_dimensions":4,"kind":"tiny-affine-tanh","learned_forward_passes":1,"output_dimensions":2,"reasoning_state":"none-after-preparation"},"architecture_identity":"sha256:8ee72d1282eac8c091cb2ebd0522fcd72fc09a6ba98bf16f64290b317e6b0375","artifact_identity":"sha256:fb1994bc16a65c3a21cb3b95093befbdbf9d7f36c04a2ffaf1419c2aa12b84aa","authority":"diagnostic-only","calibration_identity":"sha256:db8431a90e74bce699d3ec0e6d22c4257af7f2bb3bd615e4cf554966cf3762d1","checkpoint_identity":"sha256:3190bbbcfb252fc4142abd092d0cd2102aa91cb33ce671c420ee511e4e403300","class_labels":["irrelevant","relevant"],"distillation_identity":"sha256:c0d0b68d08870cc8e16c18d25c462d0388d07e5d4d506799f822e6a4e625c8ea","feature_center":[492,465,440,475],"hidden_bias":[-1.1739808229969506,-0.16472067392733597,1.1388451212811772,-1.5905173435476538],"input_weights":[[0.6753969126884765,0.7733133513548675,0.7316958461465339,0.7355029715767041],[0.22402472478747482,0.33664501092106947,0.14766551701907907,0.22759995499124483],[-0.6844603549965731,-0.7449423386979389,-0.6423136469973355,-0.7696672553787475],[0.9871629807391983,0.8727829882593194,0.8083298481486876,1.0649903608367897]],"model_identity":"sha256:e03237586802d58adc9bbfb9ea786ddd1ba3e031cedc9e9a37d75db10f4593e3","operating_envelope":{"envelope_identity":"sha256:1ddb82b9ac71cd8bdc2080cca9bfc1b0a19e20269be6888b923a4aca205ac2f1","maximum_context_observations":32,"maximum_distance":1352,"maximum_query_abs":1000,"minimum_confidence":900},"output_bias":[0.5369485030077782,-0.4024485030077775],"output_weights":[[-1.4821676022730363,-0.345827650547979,1.3640041698234613,-2.05867492473175],[1.4550676022730369,0.38292765054797895,-1.4857041698234583,2.0011749247317483]],"provider_abi":"mnel-specialist-provider-abi/0.1","provider_id":"mnel-one-step-distilled-specialist/0.1","schema":"mnel-one-step-specialist-artifact/0.1","semantics":"identity-bound-one-step-learned-specialist; diagnostic-only; not-a-verdict","source_observation_identities":["sha256:0fb47451ee0e775dcd217a71623974f9bf9d2187d25d6dae7fe2c1aac10cc03e","sha256:249da5b607a1307f45f891644f43fa640c33ac64b2225faa2d572e54f354a2a9","sha256:31f46b4a6417a9da7c9371a660bd26d61060ae1d808b5cdb1a23bcffdfd12444","sha256:326f4f543e4ff49f8e00b8bda62af7b866cdc2fb0c2b5f134c702d655031dc2f","sha256:3f57dcdc34c198a49f583ca6d5d3866d8be59f7f846c20de18092fbd136be28c","sha256:57d3bb4abc308769f4df3b821187b1caab26aaa998edfbf8a2c6dc8769005d0f","sha256:71c76f1756c96604c3e74b6c494922acb6089a17e47288258c44b55eaf34cd42","sha256:92c3c6414174102bf9054372a336a2d91124808a490dabecb4d92857e04e1205","sha256:c4de69391023c32c9b669ce58a22dae6096d17219312c601839142a32dc880ba","sha256:e9653b762c5d2d74a65d64a108f4b874add839bf5db72a77407f9527a9d69a55","sha256:f5fcc52f63a7244b93d76cf56c895224c813b7cea4580d4ca299562855ad2867"],"target_role":"forge.evidence-relevance","teacher_architecture_identity":"sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4","teacher_generation_identity":"sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4","teacher_model_identity":"sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62","teacher_operating_envelope_identity":"sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e","teacher_provider_identity":"mnel-bounded-recurrent-specialist/0.1","training_code_identity":"sha256:a318ca2496556da8b591ab22f526531118d81284aa862c728f03538d8688b99c","training_dataset_identity":"sha256:835001ba074393893495cd064fc12eba14ec39d65b8c82fe13bfb47b47bb721c","training_record_ids":["sha256:2ba03c5c8b3c8eba9085e24a29c0266cf7193755d5aaf336a751584dc35c0944","sha256:2c8142f197514ba9fb6c922ec0c22cc6d14dbf7f12e54c9c79238feb08629d11","sha256:4241d223e76611c9ca974fc914ebb54124e335b00cf59fd9523a7b1894a088aa","sha256:5a74d5dbdc9affc79ed6d35421f89c85e68bf1375339b6e30dd53ecf9f62c9c6","sha256:5c4743b99fd1e79eef773d1b826a3613fb9ca5c2e75499ce30eab6816a93702d","sha256:76610ac42c04b5a73e190f5f8942388587ca5cc7155441b35b5276137262ecd2","sha256:96d35d6368875ab858973b959bddae3d655144a0e05831096865f79bb9b86a61","sha256:d655cc21314aa9387b5e4aaf16dbdf7df996b37b3c94d26571dc5be9ccece880"],"training_spec_identity":"sha256:e7965aceee1ad2d9e1a6e6442eb743a2edf2f1238a562c36b9de66141aa8c344"} \ No newline at end of file diff --git a/examples/one-step-specialists/forge-student-no-distillation-control-g0.json b/examples/one-step-specialists/forge-student-no-distillation-control-g0.json new file mode 100644 index 0000000..1ba4dfb --- /dev/null +++ b/examples/one-step-specialists/forge-student-no-distillation-control-g0.json @@ -0,0 +1 @@ +{"architecture":{"context_state":"persistent-derived-summary","hidden_width":4,"input_dimensions":4,"kind":"tiny-affine-tanh","learned_forward_passes":1,"output_dimensions":2,"reasoning_state":"none-after-preparation"},"architecture_identity":"sha256:8ee72d1282eac8c091cb2ebd0522fcd72fc09a6ba98bf16f64290b317e6b0375","artifact_identity":"sha256:179f9f38ab4b8645a782b9c07524584faa978ac966323aa294aba4c5361e64d8","authority":"diagnostic-only","calibration_identity":"sha256:db8431a90e74bce699d3ec0e6d22c4257af7f2bb3bd615e4cf554966cf3762d1","checkpoint_identity":"sha256:3190bbbcfb252fc4142abd092d0cd2102aa91cb33ce671c420ee511e4e403300","class_labels":["irrelevant","relevant"],"distillation_identity":"sha256:c0d0b68d08870cc8e16c18d25c462d0388d07e5d4d506799f822e6a4e625c8ea","feature_center":[492,465,440,475],"hidden_bias":[-1.1739808229969506,-0.16472067392733597,1.1388451212811772,-1.5905173435476538],"input_weights":[[0.6753969126884765,0.7733133513548675,0.7316958461465339,0.7355029715767041],[0.22402472478747482,0.33664501092106947,0.14766551701907907,0.22759995499124483],[-0.6844603549965731,-0.7449423386979389,-0.6423136469973355,-0.7696672553787475],[0.9871629807391983,0.8727829882593194,0.8083298481486876,1.0649903608367897]],"model_identity":"sha256:c6273f11e39aaf91181cfcdf3b9533acf02144d50d6feb7851af684cf6e091bb","operating_envelope":{"envelope_identity":"sha256:1ddb82b9ac71cd8bdc2080cca9bfc1b0a19e20269be6888b923a4aca205ac2f1","maximum_context_observations":32,"maximum_distance":1352,"maximum_query_abs":1000,"minimum_confidence":900},"output_bias":[0.5369485030077782,-0.4024485030077775],"output_weights":[[-1.4821676022730363,-0.345827650547979,1.3640041698234613,-2.05867492473175],[1.4550676022730369,0.38292765054797895,-1.4857041698234583,2.0011749247317483]],"provider_abi":"mnel-specialist-provider-abi/0.1","provider_id":"mnel-one-step-distilled-specialist/0.1","schema":"mnel-one-step-specialist-artifact/0.1","semantics":"identity-bound-one-step-learned-specialist; diagnostic-only; not-a-verdict","source_observation_identities":["sha256:0fb47451ee0e775dcd217a71623974f9bf9d2187d25d6dae7fe2c1aac10cc03e","sha256:249da5b607a1307f45f891644f43fa640c33ac64b2225faa2d572e54f354a2a9","sha256:31f46b4a6417a9da7c9371a660bd26d61060ae1d808b5cdb1a23bcffdfd12444","sha256:326f4f543e4ff49f8e00b8bda62af7b866cdc2fb0c2b5f134c702d655031dc2f","sha256:3f57dcdc34c198a49f583ca6d5d3866d8be59f7f846c20de18092fbd136be28c","sha256:57d3bb4abc308769f4df3b821187b1caab26aaa998edfbf8a2c6dc8769005d0f","sha256:71c76f1756c96604c3e74b6c494922acb6089a17e47288258c44b55eaf34cd42","sha256:92c3c6414174102bf9054372a336a2d91124808a490dabecb4d92857e04e1205","sha256:c4de69391023c32c9b669ce58a22dae6096d17219312c601839142a32dc880ba","sha256:e9653b762c5d2d74a65d64a108f4b874add839bf5db72a77407f9527a9d69a55","sha256:f5fcc52f63a7244b93d76cf56c895224c813b7cea4580d4ca299562855ad2867"],"target_role":"forge.evidence-relevance","teacher_architecture_identity":"sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4","teacher_generation_identity":"sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4","teacher_model_identity":"sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62","teacher_operating_envelope_identity":"sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e","teacher_provider_identity":"mnel-bounded-recurrent-specialist/0.1","training_code_identity":"sha256:a318ca2496556da8b591ab22f526531118d81284aa862c728f03538d8688b99c","training_dataset_identity":"sha256:835001ba074393893495cd064fc12eba14ec39d65b8c82fe13bfb47b47bb721c","training_record_ids":["sha256:2ba03c5c8b3c8eba9085e24a29c0266cf7193755d5aaf336a751584dc35c0944","sha256:2c8142f197514ba9fb6c922ec0c22cc6d14dbf7f12e54c9c79238feb08629d11","sha256:4241d223e76611c9ca974fc914ebb54124e335b00cf59fd9523a7b1894a088aa","sha256:5a74d5dbdc9affc79ed6d35421f89c85e68bf1375339b6e30dd53ecf9f62c9c6","sha256:5c4743b99fd1e79eef773d1b826a3613fb9ca5c2e75499ce30eab6816a93702d","sha256:76610ac42c04b5a73e190f5f8942388587ca5cc7155441b35b5276137262ecd2","sha256:96d35d6368875ab858973b959bddae3d655144a0e05831096865f79bb9b86a61","sha256:d655cc21314aa9387b5e4aaf16dbdf7df996b37b3c94d26571dc5be9ccece880"],"training_spec_identity":"sha256:29da696e319ae15cc49037d955b4c2bf39788f7af6d73e1d1588095d1cacecee"} \ No newline at end of file diff --git a/examples/one-step-specialists/forge-teacher-recurrent-g0.json b/examples/one-step-specialists/forge-teacher-recurrent-g0.json new file mode 100644 index 0000000..7bd2224 --- /dev/null +++ b/examples/one-step-specialists/forge-teacher-recurrent-g0.json @@ -0,0 +1 @@ +{"architecture":{"context_state":"persistent-derived-summary","kind":"bounded-recurrent-centroid","masked_select":true,"max_iterations":4,"reasoning_state":"per-query-fixed-point-lanes","width":4},"architecture_identity":"sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4","artifact_identity":"sha256:c0a860ad60b33a3cc6d77b55bf6bc5f5fc711232d0b31ca34adcaf2c7c6b3ce2","authority":"diagnostic-only","calibration_identity":"sha256:3a9c61b28bc33a51e5ae1ac50029830d48b8d310c6a539fc874e378ab6540d05","checkpoint_identity":"sha256:92a9795d8d45d9dd1146efcfc1efff9da98524422f194a077c5ad08ccb027c1b","class_centroids":{"irrelevant":[170,150,170,130],"relevant":[860,790,730,840]},"generation_identity":"sha256:2767857f27a108bee26d88acb05597c2233500dd81e69c89ac621d423c4917d4","inherited_strategies":[],"known_counterexamples":[],"model_identity":"sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62","negative_memory":["known-omission-is-escalation"],"operating_envelope":{"convergence_delta":2,"envelope_identity":"sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e","max_iterations":4,"maximum_context_observations":32,"maximum_distance":460,"maximum_query_abs":1000,"minimum_confidence":850},"parent_model_identity":null,"prior_failure_causes":[],"provider_abi":"mnel-specialist-provider-abi/0.1","provider_id":"mnel-bounded-recurrent-specialist/0.1","schema":"mnel-recurrent-specialist-artifact/0.1","semantics":"identity-bound-learned-specialist; diagnostic-only; not-a-verdict","source_evidence_references":["forge-train-irrelevant-1","forge-train-irrelevant-2","forge-train-relevant-1","forge-train-relevant-2"],"target_role":"forge.evidence-relevance","training_code_identity":"sha256:cead54014d62e920af40844af0c6556ea9ae7cde2b32bee77dcde0695c9b6d2c","training_dataset_identity":"sha256:1cec040071b9fb21036bcf75403b17766d14094a4418e4c5835fd29e82140c3e","training_record_ids":["forge-train-irrelevant-1","forge-train-irrelevant-2","forge-train-relevant-1","forge-train-relevant-2"],"training_spec_identity":"sha256:c9d1736ae21cc36285aaecc34b41f656bfc2edab61e99b0a59c93ea37db525ba"} \ No newline at end of file diff --git a/examples/one-step-specialists/reference-study.json b/examples/one-step-specialists/reference-study.json new file mode 100644 index 0000000..7374f40 --- /dev/null +++ b/examples/one-step-specialists/reference-study.json @@ -0,0 +1,270 @@ +{ + "authority": "diagnostic-only", + "claim_boundary": { + "one_step_means": "one learned affine/tanh/softmax forward evaluation after preparation", + "student_authority": "diagnostic-only", + "success_claim_supported": false, + "teacher_fallback_is_explicit": true + }, + "distillation": { + "dataset_identity": "sha256:835001ba074393893495cd064fc12eba14ec39d65b8c82fe13bfb47b47bb721c", + "distillation_identity": "sha256:c0d0b68d08870cc8e16c18d25c462d0388d07e5d4d506799f822e6a4e625c8ea", + "rejected_or_unresolved_observation_count": 3, + "rejected_or_unresolved_statuses": [ + "rejected-teacher-target", + "unknown", + "abstention" + ], + "retained_observation_count": 11, + "training_observation_count": 8, + "transform_identity": "sha256:842b068521daf45287696bd770f5479c163126e0b13fa154717f75ebf21dfa2b" + }, + "evaluations": { + "calibration_behavior": { + "known_coverage": 1.0, + "ood_abstentions": 2, + "ood_cases": 2, + "threshold_milli": 900 + }, + "classical_nearest_centroid_baseline": { + "abstentions": 2, + "correct_known_cases": 4, + "decisions": [ + "relevant", + "relevant", + "irrelevant", + "irrelevant", + "ABSTAIN", + "ABSTAIN" + ], + "false_accepts": 0, + "inference_operations": [ + 9, + 9, + 9, + 9, + 9, + 9 + ], + "kind": "nearest-centroid-deterministic-reference", + "latency_ns": [ + 10842, + 6518, + 5019, + 4149, + 4096, + 3702 + ], + "model_bytes": 81, + "out_of_distribution_cases": 2 + }, + "comparisons": { + "student_control_disagreements": 0, + "student_teacher_disagreements": 0 + }, + "fallback": { + "escalated_end_to_end_latency_ns": [ + 258212, + 374137 + ], + "fallback_invocations": 2, + "final_decisions": [ + "relevant", + "relevant", + "irrelevant", + "irrelevant", + "ABSTAIN", + "ABSTAIN" + ], + "student_abstentions": 2, + "student_only_end_to_end_latency_ns": [ + 75835, + 67426, + 66985, + 69178, + 54534, + 108602 + ], + "teacher_results_retained": 2 + }, + "iterative_teacher": { + "abstentions": 2, + "correct_known_cases": 4, + "decisions": [ + "relevant", + "relevant", + "irrelevant", + "irrelevant", + "ABSTAIN", + "ABSTAIN" + ], + "false_accepts": 0, + "latency_ns": [ + 65177, + 51519, + 49547, + 51266, + 50777, + 48437 + ], + "model_bytes": 2047, + "operations": [ + 65, + 65, + 65, + 65, + 65, + 65 + ], + "reasoning_iterations": [ + 4, + 4, + 4, + 4, + 4, + 4 + ] + }, + "no_distillation_control": { + "abstentions": 2, + "calibration_threshold_milli": 900, + "cold_latency_ns": 74199, + "correct_known_cases": 4, + "decisions": [ + "relevant", + "relevant", + "irrelevant", + "irrelevant", + "ABSTAIN", + "ABSTAIN" + ], + "false_accepts": 0, + "inference_operations": [ + 32, + 32, + 32, + 32, + 0, + 0 + ], + "learned_forward_passes": [ + 1, + 1, + 1, + 1, + 0, + 0 + ], + "model_bytes": 4233, + "out_of_distribution_cases": 2, + "preprocessing_operations": [ + 5, + 5, + 5, + 5, + 5, + 5 + ], + "teacher_disagreements": 0, + "warm_latency_ns": [ + 83477, + 64692, + 64230, + 54564, + 53217 + ] + }, + "one_step_distilled_student": { + "abstentions": 2, + "calibration_threshold_milli": 900, + "cold_latency_ns": 149644, + "correct_known_cases": 4, + "decisions": [ + "relevant", + "relevant", + "irrelevant", + "irrelevant", + "ABSTAIN", + "ABSTAIN" + ], + "false_accepts": 0, + "inference_operations": [ + 32, + 32, + 32, + 32, + 0, + 0 + ], + "learned_forward_passes": [ + 1, + 1, + 1, + 1, + 0, + 0 + ], + "model_bytes": 4233, + "out_of_distribution_cases": 2, + "preprocessing_operations": [ + 5, + 5, + 5, + 5, + 5, + 5 + ], + "teacher_disagreements": 0, + "warm_latency_ns": [ + 67189, + 150944, + 96321, + 53979, + 53765 + ] + } + }, + "heldout_examples": 6, + "known_cases": 4, + "limitations": [ + "synthetic forge relevance features do not establish production utility", + "latency measurements are host-dependent and excluded from semantic identities", + "the student is diagnostic-only and cannot verify evidence, grant permissions, or promote generations", + "the recurrent teacher remains available as an explicit fallback rather than being silently replaced" + ], + "schema": "mnel-one-step-specialist-reference-report/0.1", + "semantics": "experimental-reference-measurement; not-a-verdict", + "students": { + "distilled": { + "architecture_identity": "sha256:8ee72d1282eac8c091cb2ebd0522fcd72fc09a6ba98bf16f64290b317e6b0375", + "artifact_bytes": 4327, + "artifact_identity": "sha256:318da84380a14c3b0b90275e5a36e482cca62971d3b8e2b74b08cd68daac1b3b", + "calibration_identity": "sha256:db8431a90e74bce699d3ec0e6d22c4257af7f2bb3bd615e4cf554966cf3762d1", + "model_bytes": 4233, + "model_identity": "sha256:e03237586802d58adc9bbfb9ea786ddd1ba3e031cedc9e9a37d75db10f4593e3", + "training_dataset_identity": "sha256:835001ba074393893495cd064fc12eba14ec39d65b8c82fe13bfb47b47bb721c" + }, + "no_distillation_control": { + "architecture_identity": "sha256:8ee72d1282eac8c091cb2ebd0522fcd72fc09a6ba98bf16f64290b317e6b0375", + "artifact_bytes": 4327, + "artifact_identity": "sha256:12ea4ff910e1772c8dfc129620fefc84c94c8ce7cd3742889099ddad11e21d44", + "calibration_identity": "sha256:db8431a90e74bce699d3ec0e6d22c4257af7f2bb3bd615e4cf554966cf3762d1", + "model_bytes": 4233, + "model_identity": "sha256:c6273f11e39aaf91181cfcdf3b9533acf02144d50d6feb7851af684cf6e091bb", + "training_dataset_identity": "sha256:835001ba074393893495cd064fc12eba14ec39d65b8c82fe13bfb47b47bb721c" + } + }, + "study_identity": "sha256:263a91be80215f875f1b8317ec9a7659e065cf193f7d35fa2ae0de6431ead354", + "target_role": "forge.evidence-relevance", + "teacher": { + "architecture_identity": "sha256:0b854e10e4fc3843017c7a264620bcfb2ace844ade15820e313e2a0faf737ab4", + "artifact_bytes": 2141, + "artifact_identity": "sha256:402e249af9d4b0fe991a621aa1230dd570569730d3859406b77a757bd70c3ee0", + "model_bytes": 2047, + "model_identity": "sha256:b3c8ba2bd9672b2ea652a92d1fd81c4c2f15db76c59a4ee5ff5fb90b72d94f62", + "operating_envelope_identity": "sha256:7a7248d840ac9b048935ebdb6b34e5188526f4dea78d850357ed122d18d78a7e", + "provider_id": "mnel-bounded-recurrent-specialist/0.1" + }, + "total_reference_cases": 6, + "training_examples": 8 +} diff --git a/schemas/mnel-distillation-dataset.schema.json b/schemas/mnel-distillation-dataset.schema.json new file mode 100644 index 0000000..0283162 --- /dev/null +++ b/schemas/mnel-distillation-dataset.schema.json @@ -0,0 +1,32 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://github.com/epi13/Machine-Native-Experimental-Learning/schemas/mnel-distillation-dataset.schema.json", + "title": "MNEL source-preserving one-step distillation dataset", + "type": "object", + "additionalProperties": false, + "required": [ + "schema", "target_role", "teacher_provider_identity", "teacher_model_identity", + "teacher_generation_identity", "teacher_architecture_identity", + "teacher_operating_envelope_identity", "observations", "training_observation_identities", + "training_targets", "transform_identity", "authority", "semantics", "dataset_identity", + "distillation_identity" + ], + "properties": { + "schema": {"const": "mnel-distillation-dataset/0.1"}, + "target_role": {"type": "string", "minLength": 1}, + "teacher_provider_identity": {"type": "string", "minLength": 1}, + "teacher_model_identity": {"$ref": "#/$defs/identity"}, + "teacher_generation_identity": {"$ref": "#/$defs/identity"}, + "teacher_architecture_identity": {"$ref": "#/$defs/identity"}, + "teacher_operating_envelope_identity": {"$ref": "#/$defs/identity"}, + "observations": {"type": "array", "minItems": 1, "maxItems": 128, "items": {"type": "object"}}, + "training_observation_identities": {"type": "array", "minItems": 1, "items": {"$ref": "#/$defs/identity"}}, + "training_targets": {"type": "array", "minItems": 1, "items": {"type": "object"}}, + "transform_identity": {"$ref": "#/$defs/identity"}, + "authority": {"const": "diagnostic-only"}, + "semantics": {"const": "source-preserving-distillation-dataset; verified-target-filtered; not-a-verdict"}, + "dataset_identity": {"$ref": "#/$defs/identity"}, + "distillation_identity": {"$ref": "#/$defs/identity"} + }, + "$defs": {"identity": {"type": "string", "pattern": "^sha256:[0-9a-f]{64}$"}} +} diff --git a/schemas/mnel-distillation-record.schema.json b/schemas/mnel-distillation-record.schema.json new file mode 100644 index 0000000..86dcf03 --- /dev/null +++ b/schemas/mnel-distillation-record.schema.json @@ -0,0 +1,45 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://github.com/epi13/Machine-Native-Experimental-Learning/schemas/mnel-distillation-record.schema.json", + "title": "MNEL retained teacher observation distillation record", + "type": "object", + "additionalProperties": false, + "required": [ + "schema", "target_role", "teacher_provider_identity", "teacher_model_identity", + "teacher_generation_identity", "teacher_architecture_identity", + "teacher_operating_envelope_identity", "snapshot_identity", "query_features", + "context_state_identity", "source_observation_identities", "teacher_decision", + "teacher_confidence_milli", "teacher_confidence", "teacher_abstained", + "teacher_escalation_reason", "reasoning_iterations", "teacher_operations", + "decision_identity", "target_status", "independent_target", + "independent_target_identity", "authority", "semantics", "observation_identity" + ], + "properties": { + "schema": {"const": "mnel-distillation-record/0.1"}, + "target_role": {"type": "string", "minLength": 1}, + "teacher_provider_identity": {"type": "string", "minLength": 1}, + "teacher_model_identity": {"$ref": "#/$defs/identity"}, + "teacher_generation_identity": {"$ref": "#/$defs/identity"}, + "teacher_architecture_identity": {"$ref": "#/$defs/identity"}, + "teacher_operating_envelope_identity": {"$ref": "#/$defs/identity"}, + "snapshot_identity": {"$ref": "#/$defs/identity"}, + "query_features": {"type": "array", "minItems": 4, "maxItems": 4, "items": {"type": "integer"}}, + "context_state_identity": {"$ref": "#/$defs/identity"}, + "source_observation_identities": {"type": "array", "minItems": 1, "items": {"type": "string", "minLength": 1}}, + "teacher_decision": {"type": "string", "minLength": 1}, + "teacher_confidence_milli": {"type": "integer", "minimum": 0, "maximum": 1000}, + "teacher_confidence": {"type": "number", "minimum": 0, "maximum": 1}, + "teacher_abstained": {"type": "boolean"}, + "teacher_escalation_reason": {"type": ["string", "null"]}, + "reasoning_iterations": {"type": "integer", "minimum": 1}, + "teacher_operations": {"type": "integer", "minimum": 1}, + "decision_identity": {"$ref": "#/$defs/identity"}, + "target_status": {"enum": ["verified", "rejected-teacher-target", "unknown", "abstention"]}, + "independent_target": {"type": ["string", "null"]}, + "independent_target_identity": {"anyOf": [{"$ref": "#/$defs/identity"}, {"type": "null"}]}, + "authority": {"const": "diagnostic-only"}, + "semantics": {"const": "retained-teacher-observation; independently-targeted; not-a-verdict"}, + "observation_identity": {"$ref": "#/$defs/identity"} + }, + "$defs": {"identity": {"type": "string", "pattern": "^sha256:[0-9a-f]{64}$"}} +} diff --git a/schemas/mnel-one-step-specialist-decision.schema.json b/schemas/mnel-one-step-specialist-decision.schema.json new file mode 100644 index 0000000..f29892c --- /dev/null +++ b/schemas/mnel-one-step-specialist-decision.schema.json @@ -0,0 +1,60 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://github.com/epi13/Machine-Native-Experimental-Learning/schemas/mnel-one-step-specialist-decision.schema.json", + "title": "MNEL one-step specialist diagnostic decision", + "type": "object", + "additionalProperties": false, + "required": [ + "schema", "request_identity", "context_state_identity", "target_role", "decision", + "diagnostic_proposal", "confidence", "confidence_milli", "abstained", + "escalation_reason", "out_of_distribution", "student_model_identity", + "student_architecture_identity", "teacher_model_identity", "teacher_generation_identity", + "distillation_identity", "calibration_identity", "operating_envelope_identity", + "prepared_features", "logits", "learned_forward_passes", "preprocessing_operations", + "inference_operations", "preprocessing_elapsed_ns", "learned_elapsed_ns", "elapsed_ns", + "source_observation_identities", "authority", "semantics", "decision_identity" + ], + "properties": { + "schema": {"const": "mnel-one-step-specialist-decision/0.1"}, + "request_identity": {"$ref": "#/$defs/identity"}, + "context_state_identity": {"$ref": "#/$defs/identity"}, + "target_role": {"type": "string", "minLength": 1}, + "decision": {"type": "string", "minLength": 1}, + "diagnostic_proposal": { + "type": "object", + "additionalProperties": false, + "required": ["kind", "role", "label"], + "properties": { + "kind": {"const": "structured-diagnostic-proposal"}, + "role": {"type": "string", "minLength": 1}, + "label": {"type": "string", "minLength": 1}, + "class_index": {"type": "integer", "minimum": 0} + } + }, + "confidence": {"type": "number", "minimum": 0, "maximum": 1}, + "confidence_milli": {"type": "integer", "minimum": 0, "maximum": 1000}, + "abstained": {"type": "boolean"}, + "escalation_reason": {"type": ["string", "null"]}, + "out_of_distribution": {"type": "boolean"}, + "student_model_identity": {"$ref": "#/$defs/identity"}, + "student_architecture_identity": {"$ref": "#/$defs/identity"}, + "teacher_model_identity": {"$ref": "#/$defs/identity"}, + "teacher_generation_identity": {"$ref": "#/$defs/identity"}, + "distillation_identity": {"$ref": "#/$defs/identity"}, + "calibration_identity": {"$ref": "#/$defs/identity"}, + "operating_envelope_identity": {"$ref": "#/$defs/identity"}, + "prepared_features": {"type": "array", "minItems": 4, "maxItems": 4, "items": {"type": "integer"}}, + "logits": {"type": "array", "minItems": 2, "maxItems": 8, "items": {"type": "number"}}, + "learned_forward_passes": {"type": "integer", "minimum": 0, "maximum": 1}, + "preprocessing_operations": {"type": "integer", "minimum": 1}, + "inference_operations": {"type": "integer", "minimum": 0}, + "preprocessing_elapsed_ns": {"type": "integer", "minimum": 0}, + "learned_elapsed_ns": {"type": "integer", "minimum": 0}, + "elapsed_ns": {"type": "integer", "minimum": 0}, + "source_observation_identities": {"type": "array", "items": {"type": "string", "minLength": 1}}, + "authority": {"const": "diagnostic-only"}, + "semantics": {"const": "one-step-structured-diagnostic-proposal; not-a-verdict"}, + "decision_identity": {"$ref": "#/$defs/identity"} + }, + "$defs": {"identity": {"type": "string", "pattern": "^sha256:[0-9a-f]{64}$"}} +} diff --git a/schemas/mnel-one-step-specialist.schema.json b/schemas/mnel-one-step-specialist.schema.json new file mode 100644 index 0000000..0f119b9 --- /dev/null +++ b/schemas/mnel-one-step-specialist.schema.json @@ -0,0 +1,79 @@ +{ + "$schema": "https://json-schema.org/draft/2020-12/schema", + "$id": "https://github.com/epi13/Machine-Native-Experimental-Learning/schemas/mnel-one-step-specialist.schema.json", + "title": "MNEL one-step distilled specialist artifact", + "type": "object", + "additionalProperties": false, + "required": [ + "schema", "provider_id", "provider_abi", "target_role", + "teacher_provider_identity", "teacher_model_identity", "teacher_generation_identity", + "teacher_architecture_identity", "teacher_operating_envelope_identity", + "architecture_identity", "architecture", "training_code_identity", + "training_dataset_identity", "distillation_identity", "training_spec_identity", + "checkpoint_identity", "calibration_identity", "operating_envelope", "class_labels", + "input_weights", "hidden_bias", "output_weights", "output_bias", "feature_center", + "training_record_ids", "source_observation_identities", "authority", "semantics", + "model_identity", "artifact_identity" + ], + "properties": { + "schema": {"const": "mnel-one-step-specialist-artifact/0.1"}, + "provider_id": {"const": "mnel-one-step-distilled-specialist/0.1"}, + "provider_abi": {"const": "mnel-specialist-provider-abi/0.1"}, + "target_role": {"type": "string", "minLength": 1}, + "teacher_provider_identity": {"type": "string", "minLength": 1}, + "teacher_model_identity": {"$ref": "#/$defs/identity"}, + "teacher_generation_identity": {"$ref": "#/$defs/identity"}, + "teacher_architecture_identity": {"$ref": "#/$defs/identity"}, + "teacher_operating_envelope_identity": {"$ref": "#/$defs/identity"}, + "architecture_identity": {"$ref": "#/$defs/identity"}, + "architecture": { + "type": "object", + "additionalProperties": false, + "required": ["kind", "input_dimensions", "hidden_width", "output_dimensions", "learned_forward_passes", "context_state", "reasoning_state"], + "properties": { + "kind": {"const": "tiny-affine-tanh"}, + "input_dimensions": {"const": 4}, + "hidden_width": {"const": 4}, + "output_dimensions": {"type": "integer", "minimum": 2, "maximum": 8}, + "learned_forward_passes": {"const": 1}, + "context_state": {"const": "persistent-derived-summary"}, + "reasoning_state": {"const": "none-after-preparation"} + } + }, + "training_code_identity": {"$ref": "#/$defs/identity"}, + "training_dataset_identity": {"$ref": "#/$defs/identity"}, + "distillation_identity": {"$ref": "#/$defs/identity"}, + "training_spec_identity": {"$ref": "#/$defs/identity"}, + "checkpoint_identity": {"$ref": "#/$defs/identity"}, + "calibration_identity": {"$ref": "#/$defs/identity"}, + "operating_envelope": { + "type": "object", + "additionalProperties": false, + "required": ["minimum_confidence", "maximum_distance", "maximum_query_abs", "maximum_context_observations", "envelope_identity"], + "properties": { + "minimum_confidence": {"type": "integer", "minimum": 0, "maximum": 1000}, + "maximum_distance": {"type": "integer", "minimum": 1}, + "maximum_query_abs": {"type": "integer", "minimum": 1, "maximum": 1000}, + "maximum_context_observations": {"type": "integer", "minimum": 1, "maximum": 32}, + "envelope_identity": {"$ref": "#/$defs/identity"} + } + }, + "class_labels": {"type": "array", "minItems": 2, "maxItems": 8, "items": {"type": "string", "minLength": 1}}, + "input_weights": {"$ref": "#/$defs/matrix4x4"}, + "hidden_bias": {"$ref": "#/$defs/vector4"}, + "output_weights": {"type": "array", "minItems": 2, "maxItems": 8, "items": {"$ref": "#/$defs/vector4"}}, + "output_bias": {"type": "array", "minItems": 2, "maxItems": 8, "items": {"type": "number"}}, + "feature_center": {"type": "array", "minItems": 4, "maxItems": 4, "items": {"type": "integer", "minimum": -1000, "maximum": 1000}}, + "training_record_ids": {"type": "array", "minItems": 1, "items": {"$ref": "#/$defs/identity"}}, + "source_observation_identities": {"type": "array", "minItems": 1, "items": {"$ref": "#/$defs/identity"}}, + "authority": {"const": "diagnostic-only"}, + "semantics": {"const": "identity-bound-one-step-learned-specialist; diagnostic-only; not-a-verdict"}, + "model_identity": {"$ref": "#/$defs/identity"}, + "artifact_identity": {"$ref": "#/$defs/identity"} + }, + "$defs": { + "identity": {"type": "string", "pattern": "^sha256:[0-9a-f]{64}$"}, + "vector4": {"type": "array", "minItems": 4, "maxItems": 4, "items": {"type": "number"}}, + "matrix4x4": {"type": "array", "minItems": 4, "maxItems": 4, "items": {"$ref": "#/$defs/vector4"}} + } +} diff --git a/src/mnel/__init__.py b/src/mnel/__init__.py index 26ea896..a95cf56 100644 --- a/src/mnel/__init__.py +++ b/src/mnel/__init__.py @@ -14,6 +14,23 @@ LearnedProviderQuery, LearnedProviderRegistry, ) +from .one_step_specialist import ( + DistillationDataset, + FallbackResult, + OneStepSpecialistError, + OneStepStudentDecision, + OneStepStudentModel, + StudentCalibration, + StudentOperatingEnvelope, + TargetStatus, + TeacherObservation, + build_distillation_dataset, + build_one_step_reference_artifacts, + calibrate_one_step_student, + collect_teacher_observation, + infer_with_teacher_fallback, + train_one_step_student, +) from .placement import ( AcceleratorDiagnostics, ExecutionDevice, @@ -50,10 +67,12 @@ "DEFAULT_LEARNED_PROVIDER_REGISTRY", "AcceleratorDiagnostics", "CalibrationRecord", + "DistillationDataset", "EvidenceLedger", "ExecutionDevice", "ExecutionMode", "ExecutionTier", + "FallbackResult", "HardGateEvaluator", "ImplementationLanguage", "LearnedProviderDeclaration", @@ -62,6 +81,9 @@ "LearnedProviderRegistry", "NativeLanguageException", "OffloadMode", + "OneStepSpecialistError", + "OneStepStudentDecision", + "OneStepStudentModel", "OperatingEnvelope", "PlacementCapabilities", "PlacementDecision", @@ -73,14 +95,24 @@ "SpecialistContextState", "SpecialistDecision", "SpecialistError", + "StudentCalibration", + "StudentOperatingEnvelope", + "TargetStatus", + "TeacherObservation", "VerifiedExperienceDistiller", + "build_distillation_dataset", + "build_one_step_reference_artifacts", "build_reference_artifacts", + "calibrate_one_step_student", "calibrate_recurrent_specialist", "canonical_digest", + "collect_teacher_observation", "context_update", "decide_placement", "infer_batch", + "infer_with_teacher_fallback", "load_runtime_manifest", + "train_one_step_student", "train_recurrent_specialist", ] diff --git a/src/mnel/cli.py b/src/mnel/cli.py index 1ef4ba7..c5bb457 100644 --- a/src/mnel/cli.py +++ b/src/mnel/cli.py @@ -22,6 +22,7 @@ LearnedProviderQuery, OutputKind, ) +from .one_step_specialist import build_one_step_reference_artifacts from .provider_study import run_reference_portfolio_study from .recurrent_specialist import build_reference_artifacts @@ -91,6 +92,11 @@ def parser() -> argparse.ArgumentParser: description="Train and measure the bounded recurrent specialist reference artifacts", ) recurrent_reference.add_argument("--workspace", default="examples/recurrent-specialists") + one_step_reference = commands.add_parser( + "one-step-specialist-reference", + description="Train and measure the one-step distilled specialist reference study", + ) + one_step_reference.add_argument("--workspace", default="examples/one-step-specialists") fabric_run = commands.add_parser( "fabric-run", description="Dispatch an operator-supplied fixed-argv plan through remote Fabric", @@ -183,6 +189,9 @@ def main(argv: list[str] | None = None) -> int: if args.command == "recurrent-specialist-reference": print(json.dumps(build_reference_artifacts(args.workspace), indent=2, sort_keys=True)) return 0 + if args.command == "one-step-specialist-reference": + print(json.dumps(build_one_step_reference_artifacts(args.workspace), indent=2, sort_keys=True)) + return 0 if args.command == "fabric-run": try: result = run_network_fabric( diff --git a/src/mnel/one_step_specialist.py b/src/mnel/one_step_specialist.py new file mode 100644 index 0000000..52e7033 --- /dev/null +++ b/src/mnel/one_step_specialist.py @@ -0,0 +1,2052 @@ +"""A small, evidence-preserving one-step distilled specialist reference. + +The module intentionally keeps teacher observation, target construction, training, +student inference, and teacher fallback as separate operations. The student is a +tiny dependency-free affine/tanh classifier. Its learned boundary is exactly one +forward evaluation; preprocessing, calibration, OOD checks, and fallback are +measured separately and never become authority. +""" + +from __future__ import annotations + +import json +import math +import time +from collections.abc import Mapping, Sequence +from dataclasses import dataclass, replace +from enum import StrEnum +from pathlib import Path +from typing import Any + +from .core import canonical_digest, canonical_json +from .recurrent_specialist import ( + AUTHORITY, + DIMENSIONS, + MAX_CONTEXT_OBSERVATIONS, + RecurrentSpecialistModel, + SpecialistContextState, + SpecialistDecision, + SpecialistError, +) + +ARTIFACT_SCHEMA = "mnel-one-step-specialist-artifact/0.1" +DECISION_SCHEMA = "mnel-one-step-specialist-decision/0.1" +DISTILLATION_RECORD_SCHEMA = "mnel-distillation-record/0.1" +DISTILLATION_DATASET_SCHEMA = "mnel-distillation-dataset/0.1" +PROVIDER_ID = "mnel-one-step-distilled-specialist/0.1" +PROVIDER_ABI = "mnel-specialist-provider-abi/0.1" +HIDDEN_WIDTH = 4 +MAX_ROWS = 128 +MAX_EPOCHS = 256 + + +class OneStepSpecialistError(ValueError): + """A malformed lineage record, artifact, request, or bounded invocation.""" + + +class TargetStatus(StrEnum): + VERIFIED = "verified" + REJECTED_TEACHER = "rejected-teacher-target" + UNKNOWN = "unknown" + ABSTENTION = "abstention" + + +def _identity(value: object, label: str) -> str: + if not isinstance(value, str) or len(value) != 71 or not value.startswith("sha256:"): + raise OneStepSpecialistError(f"{label} must be a sha256 identity") + return value + + +def _text(value: object, label: str, maximum: int = 256) -> str: + if not isinstance(value, str) or not value.strip() or len(value) > maximum: + raise OneStepSpecialistError(f"{label} must be a bounded non-empty string") + return value + + +def _features(value: object, label: str) -> tuple[int, ...]: + if not isinstance(value, (list, tuple)) or len(value) != DIMENSIONS: + raise OneStepSpecialistError(f"{label} must contain exactly {DIMENSIONS} lanes") + result = tuple(value) + if any( + not isinstance(item, int) or isinstance(item, bool) or not -1000 <= item <= 1000 + for item in result + ): + raise OneStepSpecialistError(f"{label} contains an invalid lane") + return result + + +def _number(value: object, label: str, *, limit: float = 64.0) -> float: + if not isinstance(value, (int, float)) or isinstance(value, bool): + raise OneStepSpecialistError(f"{label} must be numeric") + result = float(value) + if not math.isfinite(result) or abs(result) > limit: + raise OneStepSpecialistError(f"{label} is outside its numeric bound") + return result + + +def _vector(value: object, label: str, length: int) -> tuple[float, ...]: + if not isinstance(value, (list, tuple)) or len(value) != length: + raise OneStepSpecialistError(f"{label} has the wrong width") + return tuple(_number(item, f"{label}[{index}]") for index, item in enumerate(value)) + + +def _matrix(value: object, label: str, rows: int, columns: int) -> tuple[tuple[float, ...], ...]: + if not isinstance(value, (list, tuple)) or len(value) != rows: + raise OneStepSpecialistError(f"{label} has the wrong row count") + return tuple(_vector(row, f"{label}[{index}]", columns) for index, row in enumerate(value)) + + +def _reject_authority(value: object) -> None: + forbidden = { + "verdict", + "evaluator_verdict", + "evaluator_eligible", + "promotion", + "promotion_authorized", + "conformance", + "permission", + "credentials", + "trust", + "mncs_verdict", + "mncds_verdict", + } + if isinstance(value, Mapping): + for key, child in value.items(): + if str(key).lower() in forbidden: + raise OneStepSpecialistError(f"specialist payload contains authority field: {key}") + if str(key).lower() == "authority" and child != AUTHORITY: + raise OneStepSpecialistError("specialist payload attempted to expand authority") + _reject_authority(child) + elif isinstance(value, (list, tuple)): + for child in value: + _reject_authority(child) + + +def _without_timing(value: Any) -> Any: + """Remove host-clock measurements before deriving a reproducible study identity.""" + + if isinstance(value, Mapping): + return { + key: _without_timing(child) + for key, child in value.items() + if not key.endswith("_ns") and key not in {"latency_ns", "warm_latency_ns"} + } + if isinstance(value, list): + return [_without_timing(child) for child in value] + return value + + +def _source_identity(row: Mapping[str, Any]) -> str: + candidate = row.get("source_observation_identity") + if isinstance(candidate, str) and candidate.startswith("sha256:") and len(candidate) == 71: + return candidate + return canonical_digest( + { + "record_id": _text(row.get("record_id"), "record_id"), + "features": list(_features(row.get("features"), "features")), + } + ) + + +def _context_for_student(model: OneStepStudentModel) -> SpecialistContextState: + context = SpecialistContextState( + provider_identity=model.teacher_provider_identity, + generation_identity=model.teacher_generation_identity, + role_identity=model.target_role, + source_observation_identities=(), + feature_mean=(0, 0, 0, 0), + ) + return replace( + context, update_identity=context.content_identity, state_identity=context.content_identity + ) + + +@dataclass(frozen=True, slots=True) +class TeacherObservation: + """One retained teacher proposal plus its independently supplied target status.""" + + target_role: str + teacher_provider_identity: str + teacher_model_identity: str + teacher_generation_identity: str + teacher_architecture_identity: str + teacher_operating_envelope_identity: str + snapshot_identity: str + query_features: tuple[int, ...] + context_state_identity: str + source_observation_identities: tuple[str, ...] + teacher_decision: str + teacher_confidence: int + teacher_abstained: bool + teacher_escalation_reason: str | None + reasoning_iterations: int + teacher_operations: int + decision_identity: str + target_status: TargetStatus + independent_target: str | None = None + independent_target_identity: str | None = None + observation_identity: str = "" + authority: str = AUTHORITY + + def __post_init__(self) -> None: + _text(self.target_role, "target_role") + _text(self.teacher_provider_identity, "teacher_provider_identity") + for name in ( + "teacher_model_identity", + "teacher_generation_identity", + "teacher_architecture_identity", + "teacher_operating_envelope_identity", + "snapshot_identity", + "context_state_identity", + "decision_identity", + ): + _identity(getattr(self, name), name) + _features(self.query_features, "query_features") + if not self.source_observation_identities or any( + not isinstance(item, str) or not item.strip() + for item in self.source_observation_identities + ): + raise OneStepSpecialistError("source observation identities are required") + _text(self.teacher_decision, "teacher_decision", 128) + if not 0 <= self.teacher_confidence <= 1000: + raise OneStepSpecialistError("teacher_confidence is outside [0, 1000]") + if self.teacher_abstained != (self.teacher_decision == "ABSTAIN"): + raise OneStepSpecialistError("teacher abstention and decision disagree") + if self.reasoning_iterations < 1 or self.teacher_operations < 1: + raise OneStepSpecialistError("teacher measurements are invalid") + if self.independent_target_identity is not None: + _identity(self.independent_target_identity, "independent_target_identity") + if self.target_status == TargetStatus.VERIFIED: + if self.independent_target is None or self.teacher_abstained: + raise OneStepSpecialistError("verified targets require a non-abstaining teacher") + if self.teacher_decision != self.independent_target: + raise OneStepSpecialistError("verified target does not agree with teacher proposal") + elif self.target_status == TargetStatus.ABSTENTION and not self.teacher_abstained: + raise OneStepSpecialistError("abstention target requires an abstaining teacher") + if self.authority != AUTHORITY: + raise OneStepSpecialistError("teacher observations are diagnostic-only") + if self.observation_identity and self.observation_identity != self.content_identity: + raise OneStepSpecialistError("teacher observation identity does not match content") + + @property + def content_identity(self) -> str: + return canonical_digest(self.to_dict(include_identity=False)) + + def to_dict(self, *, include_identity: bool = True) -> dict[str, Any]: + value: dict[str, Any] = { + "schema": DISTILLATION_RECORD_SCHEMA, + "target_role": self.target_role, + "teacher_provider_identity": self.teacher_provider_identity, + "teacher_model_identity": self.teacher_model_identity, + "teacher_generation_identity": self.teacher_generation_identity, + "teacher_architecture_identity": self.teacher_architecture_identity, + "teacher_operating_envelope_identity": self.teacher_operating_envelope_identity, + "snapshot_identity": self.snapshot_identity, + "query_features": list(self.query_features), + "context_state_identity": self.context_state_identity, + "source_observation_identities": list(self.source_observation_identities), + "teacher_decision": self.teacher_decision, + "teacher_confidence_milli": self.teacher_confidence, + "teacher_confidence": self.teacher_confidence / 1000, + "teacher_abstained": self.teacher_abstained, + "teacher_escalation_reason": self.teacher_escalation_reason, + "reasoning_iterations": self.reasoning_iterations, + "teacher_operations": self.teacher_operations, + "decision_identity": self.decision_identity, + "target_status": self.target_status.value, + "independent_target": self.independent_target, + "independent_target_identity": self.independent_target_identity, + "authority": self.authority, + "semantics": "retained-teacher-observation; independently-targeted; not-a-verdict", + } + if include_identity: + value["observation_identity"] = self.observation_identity or self.content_identity + return value + + def serialize(self) -> bytes: + return canonical_json(self.to_dict()) + + @classmethod + def load(cls, payload: bytes | Mapping[str, Any]) -> TeacherObservation: + try: + value = json.loads(payload) if isinstance(payload, bytes) else dict(payload) + except (TypeError, json.JSONDecodeError) as error: + raise OneStepSpecialistError("distillation record is not valid JSON") from error + if not isinstance(value, dict) or value.get("schema") != DISTILLATION_RECORD_SCHEMA: + raise OneStepSpecialistError("unsupported distillation record schema") + _reject_authority(value) + expected_keys = { + "schema", + "target_role", + "teacher_provider_identity", + "teacher_model_identity", + "teacher_generation_identity", + "teacher_architecture_identity", + "teacher_operating_envelope_identity", + "snapshot_identity", + "query_features", + "context_state_identity", + "source_observation_identities", + "teacher_decision", + "teacher_confidence_milli", + "teacher_confidence", + "teacher_abstained", + "teacher_escalation_reason", + "reasoning_iterations", + "teacher_operations", + "decision_identity", + "target_status", + "independent_target", + "independent_target_identity", + "authority", + "semantics", + "observation_identity", + } + if ( + set(value) != expected_keys + or value.get("semantics") + != "retained-teacher-observation; independently-targeted; not-a-verdict" + ): + raise OneStepSpecialistError("distillation record contains unknown or missing fields") + confidence = value.get("teacher_confidence") + confidence_milli = value.get("teacher_confidence_milli") + if ( + not isinstance(confidence, (int, float)) + or round(float(confidence) * 1000) != confidence_milli + ): + raise OneStepSpecialistError("teacher confidence representations disagree") + record = cls( + target_role=value.get("target_role"), + teacher_provider_identity=value.get("teacher_provider_identity"), + teacher_model_identity=value.get("teacher_model_identity"), + teacher_generation_identity=value.get("teacher_generation_identity"), + teacher_architecture_identity=value.get("teacher_architecture_identity"), + teacher_operating_envelope_identity=value.get("teacher_operating_envelope_identity"), + snapshot_identity=value.get("snapshot_identity"), + query_features=tuple(value.get("query_features", ())), + context_state_identity=value.get("context_state_identity"), + source_observation_identities=tuple(value.get("source_observation_identities", ())), + teacher_decision=value.get("teacher_decision"), + teacher_confidence=confidence_milli, + teacher_abstained=value.get("teacher_abstained"), + teacher_escalation_reason=value.get("teacher_escalation_reason"), + reasoning_iterations=value.get("reasoning_iterations"), + teacher_operations=value.get("teacher_operations"), + decision_identity=value.get("decision_identity"), + target_status=TargetStatus(value.get("target_status")), + independent_target=value.get("independent_target"), + independent_target_identity=value.get("independent_target_identity"), + authority=value.get("authority"), + observation_identity=value.get("observation_identity"), + ) + return record + + +def collect_teacher_observation( + teacher: RecurrentSpecialistModel, + row: Mapping[str, Any], + *, + independent_target: str | None = None, + context: SpecialistContextState | None = None, +) -> TeacherObservation: + """Run the bounded teacher once and retain, but do not promote, its proposal.""" + + features = _features(row.get("features"), "features") + record_id = _text(row.get("record_id"), "record_id") + source_identity = _source_identity(row) + snapshot_identity = canonical_digest({"record_id": record_id, "features": list(features)}) + context_value = context or SpecialistContextState( + provider_identity=teacher.provider_id, + generation_identity=teacher.generation_identity, + role_identity=teacher.target_role, + source_observation_identities=(), + feature_mean=(0, 0, 0, 0), + ) + context_value = replace( + context_value, + update_identity=context_value.update_identity or context_value.content_identity, + state_identity=context_value.state_identity or context_value.content_identity, + ) + if ( + context_value.role_identity != teacher.target_role + or context_value.generation_identity != teacher.generation_identity + ): + raise OneStepSpecialistError("teacher context is bound to another role or generation") + request_identity = canonical_digest( + { + "snapshot_identity": snapshot_identity, + "context_state_identity": context_value.state_identity, + } + ) + result = teacher.infer( + features, + context=context_value, + request_identity=request_identity, + source_observation_identities=(source_identity,), + ) + target = independent_target if independent_target is not None else row.get("expected") + if target is not None and not isinstance(target, str): + raise OneStepSpecialistError("independent_target must be a string or null") + target_identity = ( + canonical_digest({"snapshot_identity": snapshot_identity, "independent_target": target}) + if target is not None + else None + ) + if target is None: + status = TargetStatus.UNKNOWN + elif target == "ABSTAIN": + status = TargetStatus.ABSTENTION if result.abstained else TargetStatus.REJECTED_TEACHER + elif result.abstained: + status = TargetStatus.ABSTENTION + elif result.decision == target: + status = TargetStatus.VERIFIED + else: + status = TargetStatus.REJECTED_TEACHER + observation = TeacherObservation( + target_role=teacher.target_role, + teacher_provider_identity=teacher.provider_id, + teacher_model_identity=teacher.model_identity or teacher.content_identity, + teacher_generation_identity=teacher.generation_identity, + teacher_architecture_identity=teacher.architecture_identity, + teacher_operating_envelope_identity=teacher.operating_envelope.envelope_identity + or teacher.operating_envelope.content_identity, + snapshot_identity=snapshot_identity, + query_features=features, + context_state_identity=context_value.state_identity, + source_observation_identities=(source_identity,), + teacher_decision=result.decision, + teacher_confidence=result.confidence, + teacher_abstained=result.abstained, + teacher_escalation_reason=result.escalation_reason, + reasoning_iterations=result.reasoning_iterations, + teacher_operations=result.operations, + decision_identity=result.decision_identity or result.content_identity, + target_status=status, + independent_target=target, + independent_target_identity=target_identity, + ) + return replace(observation, observation_identity=observation.content_identity) + + +@dataclass(frozen=True, slots=True) +class DistillationDataset: + """A bounded dataset retaining every observation and selecting only verified rows.""" + + target_role: str + teacher_provider_identity: str + teacher_model_identity: str + teacher_generation_identity: str + teacher_architecture_identity: str + teacher_operating_envelope_identity: str + observations: tuple[TeacherObservation, ...] + training_observation_identities: tuple[str, ...] + transform_identity: str + dataset_identity: str = "" + distillation_identity: str = "" + authority: str = AUTHORITY + + def __post_init__(self) -> None: + _text(self.target_role, "target_role") + _text(self.teacher_provider_identity, "teacher_provider_identity") + for name in ( + "teacher_model_identity", + "teacher_generation_identity", + "teacher_architecture_identity", + "teacher_operating_envelope_identity", + "transform_identity", + ): + _identity(getattr(self, name), name) + if not self.observations or len(self.observations) > MAX_ROWS: + raise OneStepSpecialistError("distillation dataset is outside its row bound") + for observation in self.observations: + if ( + observation.target_role != self.target_role + or observation.teacher_provider_identity != self.teacher_provider_identity + or observation.teacher_model_identity != self.teacher_model_identity + or observation.teacher_generation_identity != self.teacher_generation_identity + or observation.teacher_architecture_identity != self.teacher_architecture_identity + or observation.teacher_operating_envelope_identity + != self.teacher_operating_envelope_identity + ): + raise OneStepSpecialistError( + "distillation observations have mismatched teacher lineage" + ) + available = { + item.observation_identity or item.content_identity for item in self.observations + } + if ( + not self.training_observation_identities + or not set(self.training_observation_identities) <= available + ): + raise OneStepSpecialistError("training observations must be retained dataset records") + if any( + item.target_status != TargetStatus.VERIFIED + for item in self.observations + if (item.observation_identity or item.content_identity) + in self.training_observation_identities + ): + raise OneStepSpecialistError( + "rejected, unknown, or abstaining observations cannot train the student" + ) + if self.authority != AUTHORITY: + raise OneStepSpecialistError("distillation datasets are diagnostic-only") + if self.dataset_identity and self.dataset_identity != self.content_identity: + raise OneStepSpecialistError("dataset identity does not match content") + if ( + self.distillation_identity + and self.distillation_identity != self._distillation_content_identity + ): + raise OneStepSpecialistError("distillation identity does not match content") + + @property + def content_identity(self) -> str: + return canonical_digest(self.to_dict(include_identity=False)) + + @property + def _distillation_content_identity(self) -> str: + return canonical_digest( + { + "schema": "mnel-distillation-transform/0.1", + "teacher_model_identity": self.teacher_model_identity, + "teacher_generation_identity": self.teacher_generation_identity, + "transform_identity": self.transform_identity, + "training_observation_identities": list(self.training_observation_identities), + } + ) + + @property + def training_observations(self) -> tuple[TeacherObservation, ...]: + selected = set(self.training_observation_identities) + return tuple( + item + for item in self.observations + if (item.observation_identity or item.content_identity) in selected + ) + + @property + def training_targets(self) -> tuple[dict[str, Any], ...]: + """Expose the exact teacher-derived target construction for inspection.""" + + labels = tuple( + sorted( + { + item.independent_target + for item in self.training_observations + if item.independent_target + } + ) + ) + return tuple( + { + "observation_identity": item.observation_identity or item.content_identity, + "features": list(item.query_features), + "independent_target": item.independent_target, + "teacher_target": item.teacher_decision, + "teacher_confidence_milli": item.teacher_confidence, + "target_distribution": list(_soft_target(item, labels, True)), + "target_construction": "teacher-decision-and-confidence", + } + for item in self.training_observations + ) + + def to_dict(self, *, include_identity: bool = True) -> dict[str, Any]: + value: dict[str, Any] = { + "schema": DISTILLATION_DATASET_SCHEMA, + "target_role": self.target_role, + "teacher_provider_identity": self.teacher_provider_identity, + "teacher_model_identity": self.teacher_model_identity, + "teacher_generation_identity": self.teacher_generation_identity, + "teacher_architecture_identity": self.teacher_architecture_identity, + "teacher_operating_envelope_identity": self.teacher_operating_envelope_identity, + "observations": [item.to_dict() for item in self.observations], + "training_observation_identities": list(self.training_observation_identities), + "training_targets": list(self.training_targets), + "transform_identity": self.transform_identity, + "authority": self.authority, + "semantics": "source-preserving-distillation-dataset; verified-target-filtered; not-a-verdict", + } + if include_identity: + value["dataset_identity"] = self.dataset_identity or self.content_identity + value["distillation_identity"] = ( + self.distillation_identity or self._distillation_content_identity + ) + return value + + def serialize(self) -> bytes: + return canonical_json(self.to_dict()) + + @classmethod + def load(cls, payload: bytes | Mapping[str, Any]) -> DistillationDataset: + try: + value = json.loads(payload) if isinstance(payload, bytes) else dict(payload) + except (TypeError, json.JSONDecodeError) as error: + raise OneStepSpecialistError("distillation dataset is not valid JSON") from error + if not isinstance(value, dict) or value.get("schema") != DISTILLATION_DATASET_SCHEMA: + raise OneStepSpecialistError("unsupported distillation dataset schema") + _reject_authority(value) + expected_keys = { + "schema", + "target_role", + "teacher_provider_identity", + "teacher_model_identity", + "teacher_generation_identity", + "teacher_architecture_identity", + "teacher_operating_envelope_identity", + "observations", + "training_observation_identities", + "training_targets", + "transform_identity", + "authority", + "semantics", + "dataset_identity", + "distillation_identity", + } + if ( + set(value) != expected_keys + or value.get("semantics") + != "source-preserving-distillation-dataset; verified-target-filtered; not-a-verdict" + ): + raise OneStepSpecialistError("distillation dataset contains unknown or missing fields") + observations = tuple( + TeacherObservation.load(item) for item in value.get("observations", ()) + ) + dataset = cls( + target_role=value.get("target_role"), + teacher_provider_identity=value.get("teacher_provider_identity"), + teacher_model_identity=value.get("teacher_model_identity"), + teacher_generation_identity=value.get("teacher_generation_identity"), + teacher_architecture_identity=value.get("teacher_architecture_identity"), + teacher_operating_envelope_identity=value.get("teacher_operating_envelope_identity"), + observations=observations, + training_observation_identities=tuple(value.get("training_observation_identities", ())), + transform_identity=value.get("transform_identity"), + dataset_identity=value.get("dataset_identity"), + distillation_identity=value.get("distillation_identity"), + authority=value.get("authority"), + ) + if tuple(value.get("training_targets", ())) != dataset.training_targets: + raise OneStepSpecialistError("distillation targets do not match retained observations") + return dataset + + +def build_distillation_dataset( + observations: Sequence[TeacherObservation], + *, + target_role: str | None = None, + transform_identity: str | None = None, +) -> DistillationDataset: + """Build a dataset with an explicit role binding and a stable transform identity.""" + + if not observations: + raise OneStepSpecialistError("teacher observation set is empty") + first = observations[0] + if any(item.target_role != (target_role or first.target_role) for item in observations): + raise OneStepSpecialistError("distillation observations have mismatched target roles") + transform = transform_identity or canonical_digest( + { + "schema": "mnel-distillation-transform/0.1", + "operation": "verified-teacher-decision-with-confidence-targets", + "version": "0.1", + } + ) + selected = tuple( + item.observation_identity or item.content_identity + for item in observations + if item.target_status == TargetStatus.VERIFIED + ) + if not selected: + raise OneStepSpecialistError( + "distillation dataset has no independently verified training rows" + ) + dataset = DistillationDataset( + target_role=target_role or first.target_role, + teacher_provider_identity=first.teacher_provider_identity, + teacher_model_identity=first.teacher_model_identity, + teacher_generation_identity=first.teacher_generation_identity, + teacher_architecture_identity=first.teacher_architecture_identity, + teacher_operating_envelope_identity=first.teacher_operating_envelope_identity, + observations=tuple(observations), + training_observation_identities=selected, + transform_identity=transform, + ) + return replace( + dataset, + dataset_identity=dataset.content_identity, + distillation_identity=dataset._distillation_content_identity, + ) + + +build_role_distillation_dataset = build_distillation_dataset + + +@dataclass(frozen=True, slots=True) +class StudentOperatingEnvelope: + minimum_confidence: int = 700 + maximum_distance: int = 1400 + maximum_query_abs: int = 1000 + maximum_context_observations: int = MAX_CONTEXT_OBSERVATIONS + envelope_identity: str = "" + + def __post_init__(self) -> None: + if not 0 <= self.minimum_confidence <= 1000: + raise OneStepSpecialistError("student minimum_confidence is invalid") + if ( + self.maximum_distance < 1 + or not 1 <= self.maximum_context_observations <= MAX_CONTEXT_OBSERVATIONS + ): + raise OneStepSpecialistError("student operating envelope is invalid") + if not 1 <= self.maximum_query_abs <= 1000: + raise OneStepSpecialistError("student maximum_query_abs is invalid") + if self.envelope_identity and self.envelope_identity != self.content_identity: + raise OneStepSpecialistError("student envelope identity does not match content") + + @property + def content_identity(self) -> str: + return canonical_digest(self.to_dict(include_identity=False)) + + def to_dict(self, *, include_identity: bool = True) -> dict[str, Any]: + value = { + "minimum_confidence": self.minimum_confidence, + "maximum_distance": self.maximum_distance, + "maximum_query_abs": self.maximum_query_abs, + "maximum_context_observations": self.maximum_context_observations, + } + if include_identity: + value["envelope_identity"] = self.envelope_identity or self.content_identity + return value + + +@dataclass(frozen=True, slots=True) +class StudentCalibration: + target_role: str + calibration_dataset_identity: str + minimum_confidence: int + maximum_distance: int + method: str = "heldout-confidence-and-distance-envelope" + calibration_identity: str = "" + + def __post_init__(self) -> None: + _text(self.target_role, "target_role") + _identity(self.calibration_dataset_identity, "calibration_dataset_identity") + if not 0 <= self.minimum_confidence <= 1000 or self.maximum_distance < 1: + raise OneStepSpecialistError("student calibration thresholds are invalid") + if self.calibration_identity and self.calibration_identity != self.content_identity: + raise OneStepSpecialistError("student calibration identity does not match content") + + @property + def content_identity(self) -> str: + return canonical_digest(self.to_dict(include_identity=False)) + + def to_dict(self, *, include_identity: bool = True) -> dict[str, Any]: + value = { + "schema": "mnel-one-step-specialist-calibration/0.1", + "target_role": self.target_role, + "calibration_dataset_identity": self.calibration_dataset_identity, + "minimum_confidence": self.minimum_confidence, + "maximum_distance": self.maximum_distance, + "method": self.method, + "authority": AUTHORITY, + "semantics": "calibration-observation; not-a-verdict", + } + if include_identity: + value["calibration_identity"] = self.calibration_identity or self.content_identity + return value + + +@dataclass(frozen=True, slots=True) +class OneStepStudentDecision: + request_identity: str + context_state_identity: str + target_role: str + decision: str + diagnostic_proposal: Mapping[str, Any] + confidence: int + abstained: bool + escalation_reason: str | None + out_of_distribution: bool + student_model_identity: str + student_architecture_identity: str + teacher_model_identity: str + teacher_generation_identity: str + distillation_identity: str + calibration_identity: str + operating_envelope_identity: str + prepared_features: tuple[int, ...] + logits: tuple[float, ...] + learned_forward_passes: int + preprocessing_operations: int + inference_operations: int + preprocessing_elapsed_ns: int + learned_elapsed_ns: int + elapsed_ns: int + source_observation_identities: tuple[str, ...] = () + authority: str = AUTHORITY + decision_identity: str = "" + + def __post_init__(self) -> None: + for name in ( + "request_identity", + "context_state_identity", + "student_model_identity", + "student_architecture_identity", + "teacher_model_identity", + "teacher_generation_identity", + "distillation_identity", + "calibration_identity", + "operating_envelope_identity", + ): + _identity(getattr(self, name), name) + _text(self.target_role, "target_role") + _text(self.decision, "decision", 128) + _reject_authority(self.diagnostic_proposal) + if self.diagnostic_proposal.get("kind") != "structured-diagnostic-proposal": + raise OneStepSpecialistError("student proposal kind is invalid") + if not 0 <= self.confidence <= 1000: + raise OneStepSpecialistError("student confidence is invalid") + if self.abstained != (self.decision == "ABSTAIN"): + raise OneStepSpecialistError("student abstention and decision disagree") + if not 0 <= self.learned_forward_passes <= 1: + raise OneStepSpecialistError("one-step student performed more than one learned pass") + if self.preprocessing_operations < 1 or self.inference_operations < 0: + raise OneStepSpecialistError("student operation measurements are invalid") + if any( + item < 0 + for item in (self.preprocessing_elapsed_ns, self.learned_elapsed_ns, self.elapsed_ns) + ): + raise OneStepSpecialistError("student latency measurements are invalid") + _features(self.prepared_features, "prepared_features") + _vector(self.logits, "logits", len(self.logits)) + if self.authority != AUTHORITY: + raise OneStepSpecialistError("student decisions are diagnostic-only") + if self.decision_identity and self.decision_identity != self.content_identity: + raise OneStepSpecialistError("student decision identity does not match content") + + @property + def content_identity(self) -> str: + value = self.to_dict(include_identity=False) + value.pop("elapsed_ns", None) + value.pop("preprocessing_elapsed_ns", None) + value.pop("learned_elapsed_ns", None) + return canonical_digest(value) + + def to_dict(self, *, include_identity: bool = True) -> dict[str, Any]: + value: dict[str, Any] = { + "schema": DECISION_SCHEMA, + "request_identity": self.request_identity, + "context_state_identity": self.context_state_identity, + "target_role": self.target_role, + "decision": self.decision, + "diagnostic_proposal": dict(self.diagnostic_proposal), + "confidence": self.confidence / 1000, + "confidence_milli": self.confidence, + "abstained": self.abstained, + "escalation_reason": self.escalation_reason, + "out_of_distribution": self.out_of_distribution, + "student_model_identity": self.student_model_identity, + "student_architecture_identity": self.student_architecture_identity, + "teacher_model_identity": self.teacher_model_identity, + "teacher_generation_identity": self.teacher_generation_identity, + "distillation_identity": self.distillation_identity, + "calibration_identity": self.calibration_identity, + "operating_envelope_identity": self.operating_envelope_identity, + "prepared_features": list(self.prepared_features), + "logits": list(self.logits), + "learned_forward_passes": self.learned_forward_passes, + "preprocessing_operations": self.preprocessing_operations, + "inference_operations": self.inference_operations, + "preprocessing_elapsed_ns": self.preprocessing_elapsed_ns, + "learned_elapsed_ns": self.learned_elapsed_ns, + "elapsed_ns": self.elapsed_ns, + "source_observation_identities": list(self.source_observation_identities), + "authority": self.authority, + "semantics": "one-step-structured-diagnostic-proposal; not-a-verdict", + } + if include_identity: + value["decision_identity"] = self.decision_identity or self.content_identity + return value + + +@dataclass(frozen=True, slots=True) +class OneStepStudentModel: + target_role: str + teacher_provider_identity: str + teacher_model_identity: str + teacher_generation_identity: str + teacher_architecture_identity: str + teacher_operating_envelope_identity: str + architecture_identity: str + training_code_identity: str + training_dataset_identity: str + distillation_identity: str + training_spec_identity: str + checkpoint_identity: str + calibration_identity: str + operating_envelope: StudentOperatingEnvelope + class_labels: tuple[str, ...] + input_weights: tuple[tuple[float, ...], ...] + hidden_bias: tuple[float, ...] + output_weights: tuple[tuple[float, ...], ...] + output_bias: tuple[float, ...] + feature_center: tuple[int, ...] + training_record_ids: tuple[str, ...] + source_observation_identities: tuple[str, ...] + model_identity: str = "" + artifact_identity: str = "" + provider_id: str = PROVIDER_ID + provider_abi: str = PROVIDER_ABI + authority: str = AUTHORITY + + def __post_init__(self) -> None: + _text(self.target_role, "target_role") + _text(self.teacher_provider_identity, "teacher_provider_identity") + for name in ( + "teacher_model_identity", + "teacher_generation_identity", + "teacher_architecture_identity", + "teacher_operating_envelope_identity", + "architecture_identity", + "training_code_identity", + "training_dataset_identity", + "distillation_identity", + "training_spec_identity", + "checkpoint_identity", + "calibration_identity", + ): + _identity(getattr(self, name), name) + if not self.class_labels or len(self.class_labels) > 8: + raise OneStepSpecialistError("student class label count is invalid") + if tuple(sorted(self.class_labels)) != self.class_labels or len( + set(self.class_labels) + ) != len(self.class_labels): + raise OneStepSpecialistError("student class labels must be sorted and unique") + _matrix(self.input_weights, "input_weights", HIDDEN_WIDTH, DIMENSIONS) + _vector(self.hidden_bias, "hidden_bias", HIDDEN_WIDTH) + _matrix(self.output_weights, "output_weights", len(self.class_labels), HIDDEN_WIDTH) + _vector(self.output_bias, "output_bias", len(self.class_labels)) + _features(self.feature_center, "feature_center") + if not self.training_record_ids or not self.source_observation_identities: + raise OneStepSpecialistError("student training/source identities are required") + if self.authority != AUTHORITY or self.provider_abi != PROVIDER_ABI: + raise OneStepSpecialistError("student authority or ABI is invalid") + if self.model_identity and self.model_identity != self.content_identity: + raise OneStepSpecialistError("student model identity does not match content") + + @property + def content_identity(self) -> str: + return canonical_digest(self.to_dict(include_identity=False)) + + @property + def model_size_bytes(self) -> int: + return len(canonical_json(self.to_dict())) + + def to_dict(self, *, include_identity: bool = True) -> dict[str, Any]: + value: dict[str, Any] = { + "schema": ARTIFACT_SCHEMA, + "provider_id": self.provider_id, + "provider_abi": self.provider_abi, + "target_role": self.target_role, + "teacher_provider_identity": self.teacher_provider_identity, + "teacher_model_identity": self.teacher_model_identity, + "teacher_generation_identity": self.teacher_generation_identity, + "teacher_architecture_identity": self.teacher_architecture_identity, + "teacher_operating_envelope_identity": self.teacher_operating_envelope_identity, + "architecture_identity": self.architecture_identity, + "architecture": { + "kind": "tiny-affine-tanh", + "input_dimensions": DIMENSIONS, + "hidden_width": HIDDEN_WIDTH, + "output_dimensions": len(self.class_labels), + "learned_forward_passes": 1, + "context_state": "persistent-derived-summary", + "reasoning_state": "none-after-preparation", + }, + "training_code_identity": self.training_code_identity, + "training_dataset_identity": self.training_dataset_identity, + "distillation_identity": self.distillation_identity, + "training_spec_identity": self.training_spec_identity, + "checkpoint_identity": self.checkpoint_identity, + "calibration_identity": self.calibration_identity, + "operating_envelope": self.operating_envelope.to_dict(), + "class_labels": list(self.class_labels), + "input_weights": [list(row) for row in self.input_weights], + "hidden_bias": list(self.hidden_bias), + "output_weights": [list(row) for row in self.output_weights], + "output_bias": list(self.output_bias), + "feature_center": list(self.feature_center), + "training_record_ids": list(self.training_record_ids), + "source_observation_identities": list(self.source_observation_identities), + "authority": self.authority, + "semantics": "identity-bound-one-step-learned-specialist; diagnostic-only; not-a-verdict", + } + if include_identity: + value["model_identity"] = self.model_identity or self.content_identity + return value + + def serialize(self) -> bytes: + value = self.to_dict() + artifact_identity = canonical_digest(value) + value["artifact_identity"] = artifact_identity + object.__setattr__(self, "artifact_identity", artifact_identity) + return canonical_json(value) + + @classmethod + def load(cls, payload: bytes | Mapping[str, Any]) -> OneStepStudentModel: + try: + value = json.loads(payload) if isinstance(payload, bytes) else dict(payload) + except (TypeError, json.JSONDecodeError) as error: + raise OneStepSpecialistError("student artifact is not valid JSON") from error + if not isinstance(value, dict) or value.get("schema") != ARTIFACT_SCHEMA: + raise OneStepSpecialistError("unsupported one-step artifact schema") + _reject_authority(value) + supplied_artifact = value.pop("artifact_identity", None) + if not isinstance(supplied_artifact, str): + raise OneStepSpecialistError("student artifact identity is missing") + if supplied_artifact != canonical_digest(value): + raise OneStepSpecialistError("student artifact bytes do not match artifact identity") + expected_keys = { + "schema", + "provider_id", + "provider_abi", + "target_role", + "teacher_provider_identity", + "teacher_model_identity", + "teacher_generation_identity", + "teacher_architecture_identity", + "teacher_operating_envelope_identity", + "architecture_identity", + "architecture", + "training_code_identity", + "training_dataset_identity", + "distillation_identity", + "training_spec_identity", + "checkpoint_identity", + "calibration_identity", + "operating_envelope", + "class_labels", + "input_weights", + "hidden_bias", + "output_weights", + "output_bias", + "feature_center", + "training_record_ids", + "source_observation_identities", + "authority", + "semantics", + "model_identity", + } + if set(value) != expected_keys: + raise OneStepSpecialistError("student artifact contains unknown or missing fields") + architecture = value.get("architecture") + if ( + not isinstance(architecture, dict) + or architecture.get("kind") != "tiny-affine-tanh" + or architecture.get("learned_forward_passes") != 1 + ): + raise OneStepSpecialistError( + "student artifact does not declare exactly one learned pass" + ) + envelope_value = value.get("operating_envelope") + if not isinstance(envelope_value, dict): + raise OneStepSpecialistError("student operating envelope is missing") + envelope = StudentOperatingEnvelope( + **{key: item for key, item in envelope_value.items() if key != "envelope_identity"}, + envelope_identity=envelope_value.get("envelope_identity", ""), + ) + model = cls( + target_role=value.get("target_role"), + teacher_provider_identity=value.get("teacher_provider_identity"), + teacher_model_identity=value.get("teacher_model_identity"), + teacher_generation_identity=value.get("teacher_generation_identity"), + teacher_architecture_identity=value.get("teacher_architecture_identity"), + teacher_operating_envelope_identity=value.get("teacher_operating_envelope_identity"), + architecture_identity=value.get("architecture_identity"), + training_code_identity=value.get("training_code_identity"), + training_dataset_identity=value.get("training_dataset_identity"), + distillation_identity=value.get("distillation_identity"), + training_spec_identity=value.get("training_spec_identity"), + checkpoint_identity=value.get("checkpoint_identity"), + calibration_identity=value.get("calibration_identity"), + operating_envelope=envelope, + class_labels=tuple(value.get("class_labels", ())), + input_weights=tuple(tuple(row) for row in value.get("input_weights", ())), + hidden_bias=tuple(value.get("hidden_bias", ())), + output_weights=tuple(tuple(row) for row in value.get("output_weights", ())), + output_bias=tuple(value.get("output_bias", ())), + feature_center=tuple(value.get("feature_center", ())), + training_record_ids=tuple(value.get("training_record_ids", ())), + source_observation_identities=tuple(value.get("source_observation_identities", ())), + model_identity=value.get("model_identity", ""), + artifact_identity=supplied_artifact, + provider_id=value.get("provider_id", ""), + provider_abi=value.get("provider_abi", ""), + authority=value.get("authority", ""), + ) + if model.model_identity != model.content_identity: + raise OneStepSpecialistError("student model identity is invalid") + return model + + def _prepare(self, query: Sequence[int], context: SpecialistContextState) -> tuple[int, ...]: + query_value = _features(query, "query") + if any(abs(item) > self.operating_envelope.maximum_query_abs for item in query_value): + raise OneStepSpecialistError("query exceeds student operating envelope") + if ( + context.role_identity != self.target_role + or context.generation_identity != self.teacher_generation_identity + ): + raise OneStepSpecialistError("context is bound to another student role or generation") + if context.provider_identity != self.teacher_provider_identity: + raise OneStepSpecialistError("context provider identity is incompatible with student") + if ( + len(context.source_observation_identities) + > self.operating_envelope.maximum_context_observations + ): + raise OneStepSpecialistError("context exceeds student operating envelope") + return tuple( + max(-1000, min(1000, query_value[index] + context.feature_mean[index] // 16)) + for index in range(DIMENSIONS) + ) + + def _forward(self, prepared: Sequence[int]) -> tuple[tuple[float, ...], tuple[float, ...]]: + normalized = [value / 1000.0 for value in prepared] + hidden = tuple( + math.tanh( + self.hidden_bias[row] + + sum(self.input_weights[row][col] * normalized[col] for col in range(DIMENSIONS)) + ) + for row in range(HIDDEN_WIDTH) + ) + logits = tuple( + self.output_bias[row] + + sum(self.output_weights[row][col] * hidden[col] for col in range(HIDDEN_WIDTH)) + for row in range(len(self.class_labels)) + ) + return logits, hidden + + def _raw_score( + self, prepared: Sequence[int] + ) -> tuple[tuple[float, ...], tuple[float, ...], tuple[float, ...]]: + logits, hidden = self._forward(prepared) + highest = max(logits) + exponentials = tuple(math.exp(value - highest) for value in logits) + total = sum(exponentials) + return logits, hidden, tuple(value / total for value in exponentials) + + def infer( + self, + query: Sequence[int], + *, + context: SpecialistContextState | None = None, + request_identity: str | None = None, + source_observation_identities: Sequence[str] = (), + ) -> OneStepStudentDecision: + started = time.perf_counter_ns() + context_value = context or _context_for_student(self) + invalid_context = not isinstance(context_value, SpecialistContextState) + context_identity = ( + canonical_digest({"invalid_context_type": type(context_value).__name__}) + if invalid_context + else context_value.state_identity or context_value.content_identity + ) + request = request_identity or canonical_digest( + {"query": list(query), "context_state_identity": context_identity} + ) + _identity(request, "request_identity") + prep_started = time.perf_counter_ns() + try: + if invalid_context: + raise OneStepSpecialistError("context must be a SpecialistContextState") + prepared = self._prepare(query, context_value) + except OneStepSpecialistError as error: + elapsed = time.perf_counter_ns() - started + result = OneStepStudentDecision( + request_identity=request, + context_state_identity=context_identity, + target_role=self.target_role, + decision="ABSTAIN", + diagnostic_proposal={ + "kind": "structured-diagnostic-proposal", + "role": self.target_role, + "label": "ABSTAIN", + }, + confidence=0, + abstained=True, + escalation_reason=f"unsupported-envelope:{error}", + out_of_distribution=False, + student_model_identity=self.model_identity or self.content_identity, + student_architecture_identity=self.architecture_identity, + teacher_model_identity=self.teacher_model_identity, + teacher_generation_identity=self.teacher_generation_identity, + distillation_identity=self.distillation_identity, + calibration_identity=self.calibration_identity, + operating_envelope_identity=self.operating_envelope.envelope_identity + or self.operating_envelope.content_identity, + prepared_features=(0, 0, 0, 0), + logits=tuple(0.0 for _ in self.class_labels), + learned_forward_passes=0, + preprocessing_operations=1, + inference_operations=0, + preprocessing_elapsed_ns=elapsed, + learned_elapsed_ns=0, + elapsed_ns=elapsed, + source_observation_identities=tuple(source_observation_identities), + ) + return replace(result, decision_identity=result.content_identity) + preprocessing_elapsed = time.perf_counter_ns() - prep_started + distance = sum( + abs(prepared[index] - self.feature_center[index]) for index in range(DIMENSIONS) + ) + if distance > self.operating_envelope.maximum_distance: + elapsed = time.perf_counter_ns() - started + result = OneStepStudentDecision( + request_identity=request, + context_state_identity=context_identity, + target_role=self.target_role, + decision="ABSTAIN", + diagnostic_proposal={ + "kind": "structured-diagnostic-proposal", + "role": self.target_role, + "label": "ABSTAIN", + }, + confidence=0, + abstained=True, + escalation_reason="out-of-distribution-distance", + out_of_distribution=True, + student_model_identity=self.model_identity or self.content_identity, + student_architecture_identity=self.architecture_identity, + teacher_model_identity=self.teacher_model_identity, + teacher_generation_identity=self.teacher_generation_identity, + distillation_identity=self.distillation_identity, + calibration_identity=self.calibration_identity, + operating_envelope_identity=self.operating_envelope.envelope_identity + or self.operating_envelope.content_identity, + prepared_features=prepared, + logits=tuple(0.0 for _ in self.class_labels), + learned_forward_passes=0, + preprocessing_operations=5, + inference_operations=0, + preprocessing_elapsed_ns=preprocessing_elapsed, + learned_elapsed_ns=0, + elapsed_ns=elapsed, + source_observation_identities=tuple(source_observation_identities), + ) + return replace(result, decision_identity=result.content_identity) + learned_started = time.perf_counter_ns() + logits, _hidden, probabilities = self._raw_score(prepared) + learned_elapsed = time.perf_counter_ns() - learned_started + best_index = max( + range(len(probabilities)), key=lambda index: (probabilities[index], -index) + ) + confidence = max(0, min(1000, round(probabilities[best_index] * 1000))) + abstention = None + decision = self.class_labels[best_index] + if confidence < self.operating_envelope.minimum_confidence: + abstention = "insufficient-calibrated-confidence" + decision = "ABSTAIN" + elapsed = time.perf_counter_ns() - started + result = OneStepStudentDecision( + request_identity=request, + context_state_identity=context_identity, + target_role=self.target_role, + decision=decision, + diagnostic_proposal={ + "kind": "structured-diagnostic-proposal", + "role": self.target_role, + "label": decision, + "class_index": best_index, + }, + confidence=confidence, + abstained=abstention is not None, + escalation_reason=abstention, + out_of_distribution=False, + student_model_identity=self.model_identity or self.content_identity, + student_architecture_identity=self.architecture_identity, + teacher_model_identity=self.teacher_model_identity, + teacher_generation_identity=self.teacher_generation_identity, + distillation_identity=self.distillation_identity, + calibration_identity=self.calibration_identity, + operating_envelope_identity=self.operating_envelope.envelope_identity + or self.operating_envelope.content_identity, + prepared_features=prepared, + logits=logits, + learned_forward_passes=1, + preprocessing_operations=5, + inference_operations=HIDDEN_WIDTH * (DIMENSIONS + 1) + + len(self.class_labels) * (HIDDEN_WIDTH + 1) + + len(self.class_labels), + preprocessing_elapsed_ns=preprocessing_elapsed, + learned_elapsed_ns=learned_elapsed, + elapsed_ns=elapsed, + source_observation_identities=tuple(source_observation_identities), + ) + return replace(result, decision_identity=result.content_identity) + + +def _initial_weight(seed: int, row: int, column: int) -> float: + value = (seed * 1103515245 + row * 12345 + column * 2654435761) & 0xFFFFFFFF + return ((value % 2001) - 1000) / 10000.0 + + +def _soft_target( + observation: TeacherObservation, labels: Sequence[str], distilled: bool +) -> tuple[float, ...]: + if observation.independent_target not in labels: + raise OneStepSpecialistError("training target label is not a student class") + target_index = labels.index(observation.independent_target) + if not distilled: + return tuple(1.0 if index == target_index else 0.0 for index in range(len(labels))) + teacher_index = labels.index(observation.teacher_decision) + confidence = observation.teacher_confidence / 1000.0 + mass = max(0.5, min(1.0, confidence)) + remainder = (1.0 - mass) / max(1, len(labels) - 1) + return tuple(mass if index == teacher_index else remainder for index in range(len(labels))) + + +def train_one_step_student( + dataset: DistillationDataset, + *, + distilled: bool = True, + seed: int = 17, + epochs: int = 120, + learning_rate: float = 0.35, +) -> OneStepStudentModel: + """Train the tiny student from retained teacher targets or direct labels. + + ``distilled=True`` consumes teacher decision/confidence targets. The control + uses the same architecture and examples but only the independent labels. + """ + + if not isinstance(seed, int) or seed < 0 or epochs < 1 or epochs > MAX_EPOCHS: + raise OneStepSpecialistError("student training budget or seed is invalid") + if not 0.0 < learning_rate <= 2.0 or len(dataset.training_observations) > MAX_ROWS: + raise OneStepSpecialistError("student training parameters are invalid") + rows = dataset.training_observations + labels = tuple( + sorted({item.independent_target for item in rows if item.independent_target is not None}) + ) + if len(labels) < 2: + raise OneStepSpecialistError("student training requires at least two target classes") + input_weights = [ + [_initial_weight(seed, row, column) for column in range(DIMENSIONS)] + for row in range(HIDDEN_WIDTH) + ] + hidden_bias = [_initial_weight(seed + 7, row, 0) for row in range(HIDDEN_WIDTH)] + output_weights = [ + [_initial_weight(seed + 13, row, column) for column in range(HIDDEN_WIDTH)] + for row in range(len(labels)) + ] + output_bias = [_initial_weight(seed + 19, row, 0) for row in range(len(labels))] + for _ in range(epochs): + for observation in rows: + x = [value / 1000.0 for value in observation.query_features] + hidden = [ + math.tanh( + hidden_bias[row] + + sum(input_weights[row][col] * x[col] for col in range(DIMENSIONS)) + ) + for row in range(HIDDEN_WIDTH) + ] + logits = [ + output_bias[row] + + sum(output_weights[row][col] * hidden[col] for col in range(HIDDEN_WIDTH)) + for row in range(len(labels)) + ] + highest = max(logits) + exponentials = [math.exp(value - highest) for value in logits] + total = sum(exponentials) + probabilities = [value / total for value in exponentials] + target = _soft_target(observation, labels, distilled) + gradient_logits = [probabilities[index] - target[index] for index in range(len(labels))] + gradient_hidden = [ + sum( + gradient_logits[row] * output_weights[row][column] for row in range(len(labels)) + ) + for column in range(HIDDEN_WIDTH) + ] + for row in range(len(labels)): + output_bias[row] -= learning_rate * gradient_logits[row] + for column in range(HIDDEN_WIDTH): + output_weights[row][column] -= ( + learning_rate * gradient_logits[row] * hidden[column] + ) + for row in range(HIDDEN_WIDTH): + gradient = gradient_hidden[row] * (1.0 - hidden[row] * hidden[row]) + hidden_bias[row] -= learning_rate * gradient + for column in range(DIMENSIONS): + input_weights[row][column] -= learning_rate * gradient * x[column] + center = tuple( + sum(item.query_features[index] for item in rows) // len(rows) for index in range(DIMENSIONS) + ) + radius = max( + sum(abs(item.query_features[index] - center[index]) for index in range(DIMENSIONS)) + for item in rows + ) + spec = { + "schema": "mnel-one-step-specialist-training-spec/0.1", + "architecture": "tiny-affine-tanh", + "target_role": dataset.target_role, + "distilled": distilled, + "seed": seed, + "epochs": epochs, + "learning_rate": learning_rate, + "deterministic": True, + "resource_budget": {"max_epochs": MAX_EPOCHS, "max_rows": MAX_ROWS}, + } + checkpoint = { + "input_weights": input_weights, + "hidden_bias": hidden_bias, + "output_weights": output_weights, + "output_bias": output_bias, + } + model = OneStepStudentModel( + target_role=dataset.target_role, + teacher_provider_identity=dataset.teacher_provider_identity, + teacher_model_identity=dataset.teacher_model_identity, + teacher_generation_identity=dataset.teacher_generation_identity, + teacher_architecture_identity=dataset.teacher_architecture_identity, + teacher_operating_envelope_identity=dataset.teacher_operating_envelope_identity, + architecture_identity=canonical_digest( + { + "kind": "tiny-affine-tanh", + "input": DIMENSIONS, + "hidden": HIDDEN_WIDTH, + "output": len(labels), + } + ), + training_code_identity=canonical_digest( + {"module": __name__, "algorithm": "bounded-sgd-tanh-softmax", "version": "0.1"} + ), + training_dataset_identity=dataset.dataset_identity or dataset.content_identity, + distillation_identity=dataset.distillation_identity + or dataset._distillation_content_identity, + training_spec_identity=canonical_digest(spec), + checkpoint_identity=canonical_digest(checkpoint), + calibration_identity=canonical_digest( + {"status": "pending", "dataset": dataset.dataset_identity} + ), + operating_envelope=StudentOperatingEnvelope(maximum_distance=max(600, radius + 240)), + class_labels=labels, + input_weights=tuple(tuple(row) for row in input_weights), + hidden_bias=tuple(hidden_bias), + output_weights=tuple(tuple(row) for row in output_weights), + output_bias=tuple(output_bias), + feature_center=center, + training_record_ids=tuple(sorted(dataset.training_observation_identities)), + source_observation_identities=tuple( + sorted( + { + identity + for item in dataset.observations + for identity in item.source_observation_identities + } + ) + ), + ) + return replace(model, model_identity=model.content_identity) + + +def calibrate_one_step_student( + model: OneStepStudentModel, + rows: Sequence[Mapping[str, Any]], +) -> tuple[OneStepStudentModel, StudentCalibration]: + if not rows: + raise OneStepSpecialistError("student calibration dataset is empty") + confidences: list[int] = [] + distances: list[int] = [] + dataset_identity = canonical_digest({"rows": [dict(row) for row in rows]}) + for row in rows: + features = _features(row.get("features"), "calibration features") + prepared = model._prepare(features, _context_for_student(model)) + _, _, probabilities = model._raw_score(prepared) + expected = row.get("expected") + if isinstance(expected, str) and expected in model.class_labels: + confidences.append(round(probabilities[model.class_labels.index(expected)] * 1000)) + distances.append( + sum(abs(prepared[index] - model.feature_center[index]) for index in range(DIMENSIONS)) + ) + minimum_confidence = max(550, min(900, (min(confidences) - 40) if confidences else 700)) + known_distances = [ + distance for row, distance in zip(rows, distances) if row.get("expected") != "ABSTAIN" + ] + maximum_distance = max( + 1, max(known_distances, default=model.operating_envelope.maximum_distance) + 160 + ) + calibration = StudentCalibration( + target_role=model.target_role, + calibration_dataset_identity=dataset_identity, + minimum_confidence=minimum_confidence, + maximum_distance=maximum_distance, + ) + calibration = replace(calibration, calibration_identity=calibration.content_identity) + envelope = StudentOperatingEnvelope( + minimum_confidence=calibration.minimum_confidence, + maximum_distance=calibration.maximum_distance, + maximum_query_abs=model.operating_envelope.maximum_query_abs, + maximum_context_observations=model.operating_envelope.maximum_context_observations, + ) + envelope = replace(envelope, envelope_identity=envelope.content_identity) + calibrated = replace( + model, + calibration_identity=calibration.calibration_identity, + operating_envelope=envelope, + model_identity="", + artifact_identity="", + ) + return replace(calibrated, model_identity=calibrated.content_identity), calibration + + +@dataclass(frozen=True, slots=True) +class FallbackResult: + student_decision: OneStepStudentDecision + teacher_decision: SpecialistDecision | None + final_decision: str + final_confidence: int + fallback_invoked: bool + fallback_error: str | None + student_elapsed_ns: int + teacher_elapsed_ns: int + total_elapsed_ns: int + authority: str = AUTHORITY + result_identity: str = "" + + def __post_init__(self) -> None: + if self.teacher_decision is None and self.fallback_invoked and self.fallback_error is None: + raise OneStepSpecialistError("fallback invocation must retain its result or error") + if self.authority != AUTHORITY: + raise OneStepSpecialistError("fallback results are diagnostic-only") + if self.result_identity and self.result_identity != self.content_identity: + raise OneStepSpecialistError("fallback result identity does not match content") + + @property + def content_identity(self) -> str: + return canonical_digest(self.to_dict(include_identity=False)) + + def to_dict(self, *, include_identity: bool = True) -> dict[str, Any]: + value: dict[str, Any] = { + "schema": "mnel-one-step-fallback-result/0.1", + "student_decision": self.student_decision.to_dict(), + "teacher_decision": self.teacher_decision.to_dict() if self.teacher_decision else None, + "final_decision": self.final_decision, + "final_confidence_milli": self.final_confidence, + "fallback_invoked": self.fallback_invoked, + "fallback_error": self.fallback_error, + "student_elapsed_ns": self.student_elapsed_ns, + "teacher_elapsed_ns": self.teacher_elapsed_ns, + "total_elapsed_ns": self.total_elapsed_ns, + "authority": self.authority, + "semantics": "explicit-student-abstention-and-teacher-fallback; not-a-verdict", + } + if include_identity: + value["result_identity"] = self.result_identity or self.content_identity + return value + + +def infer_with_teacher_fallback( + student: OneStepStudentModel, + teacher: RecurrentSpecialistModel, + query: Sequence[int], + *, + context: SpecialistContextState | None = None, + request_identity: str | None = None, + source_observation_identities: Sequence[str] = (), +) -> FallbackResult: + """Invoke the teacher only after an explicit student abstention.""" + + if ( + teacher.provider_id != student.teacher_provider_identity + or (teacher.model_identity or teacher.content_identity) != student.teacher_model_identity + or teacher.generation_identity != student.teacher_generation_identity + or teacher.architecture_identity != student.teacher_architecture_identity + or ( + teacher.operating_envelope.envelope_identity + or teacher.operating_envelope.content_identity + ) + != student.teacher_operating_envelope_identity + ): + raise OneStepSpecialistError("fallback teacher is not lineage-compatible with student") + started = time.perf_counter_ns() + student_decision = student.infer( + query, + context=context, + request_identity=request_identity, + source_observation_identities=source_observation_identities, + ) + teacher_decision = None + fallback_error = None + teacher_elapsed = 0 + if student_decision.abstained: + fallback_invoked = True + teacher_started = time.perf_counter_ns() + try: + teacher_decision = teacher.infer( + query, + context=context, + request_identity=student_decision.request_identity, + source_observation_identities=source_observation_identities, + lineage_identity=student.distillation_identity, + ) + except (SpecialistError, ValueError) as error: + fallback_error = str(error) + teacher_elapsed = time.perf_counter_ns() - teacher_started + else: + fallback_invoked = False + final_decision = teacher_decision.decision if teacher_decision else student_decision.decision + final_confidence = ( + teacher_decision.confidence if teacher_decision else student_decision.confidence + ) + result = FallbackResult( + student_decision=student_decision, + teacher_decision=teacher_decision, + final_decision=final_decision, + final_confidence=final_confidence, + fallback_invoked=fallback_invoked, + fallback_error=fallback_error, + student_elapsed_ns=student_decision.elapsed_ns, + teacher_elapsed_ns=teacher_elapsed, + total_elapsed_ns=time.perf_counter_ns() - started, + ) + return replace(result, result_identity=result.content_identity) + + +def _reference_teacher() -> RecurrentSpecialistModel: + from .recurrent_specialist import calibrate_recurrent_specialist, train_recurrent_specialist + + rows = ( + { + "record_id": "forge-train-relevant-1", + "features": [900, 820, 760, 880], + "label": "relevant", + }, + { + "record_id": "forge-train-relevant-2", + "features": [820, 760, 700, 800], + "label": "relevant", + }, + { + "record_id": "forge-train-irrelevant-1", + "features": [120, 180, 160, 100], + "label": "irrelevant", + }, + { + "record_id": "forge-train-irrelevant-2", + "features": [220, 120, 180, 160], + "label": "irrelevant", + }, + ) + teacher = train_recurrent_specialist( + rows, + target_role="forge.evidence-relevance", + generation_identity=canonical_digest( + {"role": "forge.evidence-relevance", "generation": "G0"} + ), + negative_memory=("known-omission-is-escalation",), + ) + teacher, _ = calibrate_recurrent_specialist( + teacher, + ( + *rows, + { + "record_id": "forge-calibration-boundary", + "features": [760, 700, 660, 720], + "label": "relevant", + }, + ), + ) + return teacher + + +def _study_training_rows() -> tuple[dict[str, Any], ...]: + return ( + { + "record_id": "study-train-relevant-1", + "features": [900, 820, 760, 880], + "expected": "relevant", + }, + { + "record_id": "study-train-relevant-2", + "features": [820, 760, 700, 800], + "expected": "relevant", + }, + { + "record_id": "study-train-relevant-3", + "features": [760, 700, 660, 720], + "expected": "relevant", + }, + { + "record_id": "study-train-relevant-4", + "features": [700, 740, 720, 760], + "expected": "relevant", + }, + { + "record_id": "study-train-irrelevant-1", + "features": [120, 180, 160, 100], + "expected": "irrelevant", + }, + { + "record_id": "study-train-irrelevant-2", + "features": [220, 120, 180, 160], + "expected": "irrelevant", + }, + { + "record_id": "study-train-irrelevant-3", + "features": [160, 220, 120, 180], + "expected": "irrelevant", + }, + { + "record_id": "study-train-irrelevant-4", + "features": [260, 180, 220, 200], + "expected": "irrelevant", + }, + # Deliberately mis-targeted and retained as rejected teacher evidence. + { + "record_id": "study-rejected-target", + "features": [900, 820, 760, 880], + "expected": "irrelevant", + }, + {"record_id": "study-unknown-target", "features": [640, 620, 600, 580]}, + { + "record_id": "study-teacher-abstention", + "features": [500, 500, 500, 500], + "expected": "ABSTAIN", + }, + ) + + +def _study_holdout_rows() -> tuple[dict[str, Any], ...]: + return ( + { + "record_id": "study-heldout-relevant", + "features": [780, 740, 700, 760], + "expected": "relevant", + }, + { + "record_id": "study-heldout-relevant-boundary", + "features": [720, 700, 680, 720], + "expected": "relevant", + }, + { + "record_id": "study-heldout-irrelevant", + "features": [160, 220, 120, 180], + "expected": "irrelevant", + }, + { + "record_id": "study-heldout-irrelevant-boundary", + "features": [260, 180, 220, 200], + "expected": "irrelevant", + }, + { + "record_id": "study-ood-alternating", + "features": [1000, -1000, 1000, -1000], + "expected": "ABSTAIN", + }, + { + "record_id": "study-ood-high", + "features": [1000, 1000, 1000, 1000], + "expected": "ABSTAIN", + }, + ) + + +def _evaluate_reference( + teacher: RecurrentSpecialistModel, + student: OneStepStudentModel, + rows: Sequence[Mapping[str, Any]], +) -> dict[str, Any]: + teacher_results: list[SpecialistDecision] = [] + student_results: list[OneStepStudentDecision] = [] + fallback_results: list[FallbackResult] = [] + for row in rows: + features = _features(row.get("features"), "study features") + source = _source_identity(row) + teacher_results.append(teacher.infer(features, source_observation_identities=(source,))) + student_results.append(student.infer(features, source_observation_identities=(source,))) + fallback_results.append( + infer_with_teacher_fallback( + student, teacher, features, source_observation_identities=(source,) + ) + ) + expected = [str(row.get("expected", "UNKNOWN")) for row in rows] + known = [index for index, item in enumerate(expected) if item not in {"UNKNOWN", "ABSTAIN"}] + ood = [index for index, item in enumerate(expected) if item == "ABSTAIN"] + teacher_decisions = [item.decision for item in teacher_results] + student_decisions = [item.decision for item in student_results] + baseline_results = [] + for row in rows: + started = time.perf_counter_ns() + features = _features(row.get("features"), "study features") + distances = sorted( + (teacher._distance(features, centroid), label) + for label, centroid in teacher.class_centroids.items() + ) + best_distance, best_label = distances[0] + second_distance = distances[1][0] if len(distances) > 1 else None + confidence = teacher._confidence(best_distance, second_distance) + abstained = ( + best_distance > teacher.operating_envelope.maximum_distance + or confidence < teacher.operating_envelope.minimum_confidence + ) + baseline_results.append( + { + "decision": "ABSTAIN" if abstained else best_label, + "abstained": abstained, + "out_of_distribution": best_distance > teacher.operating_envelope.maximum_distance, + "operations": 1 + len(teacher.class_centroids) * DIMENSIONS, + "elapsed_ns": time.perf_counter_ns() - started, + } + ) + baseline_decisions = [item["decision"] for item in baseline_results] + return { + "total_reference_cases": len(rows), + "known_cases": len(known), + "expected": expected, + "teacher": { + "decisions": teacher_decisions, + "correct_known_cases": sum( + teacher_decisions[index] == expected[index] for index in known + ), + "false_accepts": sum(teacher_decisions[index] != "ABSTAIN" for index in ood), + "abstentions": sum(item.abstained for item in teacher_results), + "reasoning_iterations": [item.reasoning_iterations for item in teacher_results], + "operations": [item.operations for item in teacher_results], + "latency_ns": [item.elapsed_ns for item in teacher_results], + "model_bytes": teacher.model_size_bytes, + }, + "student": { + "decisions": student_decisions, + "correct_known_cases": sum( + student_decisions[index] == expected[index] for index in known + ), + "false_accepts": sum(student_decisions[index] != "ABSTAIN" for index in ood), + "abstentions": sum(item.abstained for item in student_results), + "out_of_distribution_cases": sum(item.out_of_distribution for item in student_results), + "teacher_disagreements": sum( + student_decisions[index] != teacher_decisions[index] for index in range(len(rows)) + ), + "learned_forward_passes": [item.learned_forward_passes for item in student_results], + "inference_operations": [item.inference_operations for item in student_results], + "preprocessing_operations": [item.preprocessing_operations for item in student_results], + "cold_latency_ns": student_results[0].elapsed_ns, + "warm_latency_ns": [item.elapsed_ns for item in student_results[1:]], + "model_bytes": student.model_size_bytes, + "calibration_threshold_milli": student.operating_envelope.minimum_confidence, + }, + "baseline": { + "kind": "nearest-centroid-deterministic-reference", + "decisions": baseline_decisions, + "correct_known_cases": sum( + baseline_decisions[index] == expected[index] for index in known + ), + "false_accepts": sum(baseline_decisions[index] != "ABSTAIN" for index in ood), + "abstentions": sum(item["abstained"] for item in baseline_results), + "out_of_distribution_cases": sum( + item["out_of_distribution"] for item in baseline_results + ), + "inference_operations": [item["operations"] for item in baseline_results], + "latency_ns": [item["elapsed_ns"] for item in baseline_results], + "model_bytes": len(canonical_json({"class_centroids": teacher.class_centroids})), + }, + "fallback": { + "fallback_invocations": sum(item.fallback_invoked for item in fallback_results), + "final_decisions": [item.final_decision for item in fallback_results], + "student_abstentions": sum( + item.student_decision.abstained for item in fallback_results + ), + "teacher_results_retained": sum( + item.teacher_decision is not None for item in fallback_results + ), + "student_only_end_to_end_latency_ns": [ + item.student_elapsed_ns for item in fallback_results + ], + "escalated_end_to_end_latency_ns": [ + item.total_elapsed_ns for item in fallback_results if item.fallback_invoked + ], + }, + "calibration_behavior": { + "threshold_milli": student.operating_envelope.minimum_confidence, + "ood_cases": len(ood), + "ood_abstentions": sum(student_results[index].abstained for index in ood), + "known_coverage": sum(not student_results[index].abstained for index in known) + / max(1, len(known)), + }, + "decision_digest": canonical_digest( + {"teacher": teacher_decisions, "student": student_decisions} + ), + } + + +def build_one_step_reference_artifacts(output_dir: str | Path) -> dict[str, Any]: + """Generate the deterministic forge reference study and its lineage artifacts.""" + + destination = Path(output_dir) + destination.mkdir(parents=True, exist_ok=True) + teacher = _reference_teacher() + observations = tuple( + collect_teacher_observation(teacher, row) for row in _study_training_rows() + ) + dataset = build_distillation_dataset(observations, target_role=teacher.target_role) + student, calibration = calibrate_one_step_student( + train_one_step_student(dataset, distilled=True), + _study_holdout_rows()[:4], + ) + control, control_calibration = calibrate_one_step_student( + train_one_step_student(dataset, distilled=False), + _study_holdout_rows()[:4], + ) + student_path = destination / "forge-student-distilled-g0.json" + control_path = destination / "forge-student-no-distillation-control-g0.json" + teacher_path = destination / "forge-teacher-recurrent-g0.json" + records_path = destination / "distillation-records.json" + teacher_path.write_bytes(teacher.serialize()) + student_path.write_bytes(student.serialize()) + control_path.write_bytes(control.serialize()) + records_path.write_text( + json.dumps( + { + "schema": "mnel-distillation-record-set/0.1", + "records": [item.to_dict() for item in observations], + "dataset": dataset.to_dict(), + "authority": AUTHORITY, + "semantics": "retained-teacher-observations-and-target-construction; not-a-verdict", + }, + indent=2, + sort_keys=True, + ) + + "\n", + encoding="utf-8", + ) + distilled_evaluation = _evaluate_reference(teacher, student, _study_holdout_rows()) + control_evaluation = _evaluate_reference(teacher, control, _study_holdout_rows()) + evaluations = { + "iterative_teacher": distilled_evaluation["teacher"], + "one_step_distilled_student": distilled_evaluation["student"], + "classical_nearest_centroid_baseline": distilled_evaluation["baseline"], + "no_distillation_control": control_evaluation["student"], + "fallback": distilled_evaluation["fallback"], + "calibration_behavior": distilled_evaluation["calibration_behavior"], + "comparisons": { + "student_teacher_disagreements": distilled_evaluation["student"][ + "teacher_disagreements" + ], + "student_control_disagreements": sum( + left != right + for left, right in zip( + distilled_evaluation["student"]["decisions"], + control_evaluation["student"]["decisions"], + ) + ), + }, + } + study_body = { + "teacher_model_identity": teacher.model_identity, + "teacher_artifact_identity": canonical_digest(json.loads(teacher.serialize())), + "distillation_dataset_identity": dataset.dataset_identity, + "distillation_identity": dataset.distillation_identity, + "student_model_identity": student.model_identity, + "student_artifact_identity": canonical_digest(json.loads(student.serialize())), + "control_model_identity": control.model_identity, + "control_artifact_identity": canonical_digest(json.loads(control.serialize())), + "calibration_identity": calibration.calibration_identity, + "control_calibration_identity": control_calibration.calibration_identity, + "evaluations": _without_timing(evaluations), + } + report = { + "schema": "mnel-one-step-specialist-reference-report/0.1", + "study_identity": canonical_digest(study_body), + "target_role": teacher.target_role, + "total_reference_cases": distilled_evaluation["total_reference_cases"], + "known_cases": distilled_evaluation["known_cases"], + "heldout_examples": len(_study_holdout_rows()), + "training_examples": len(dataset.training_observations), + "teacher": { + "provider_id": teacher.provider_id, + "model_identity": teacher.model_identity, + "artifact_identity": canonical_digest(json.loads(teacher.serialize())), + "architecture_identity": teacher.architecture_identity, + "operating_envelope_identity": teacher.operating_envelope.envelope_identity, + "model_bytes": teacher.model_size_bytes, + "artifact_bytes": len(teacher.serialize()), + }, + "distillation": { + "dataset_identity": dataset.dataset_identity, + "distillation_identity": dataset.distillation_identity, + "transform_identity": dataset.transform_identity, + "retained_observation_count": len(dataset.observations), + "training_observation_count": len(dataset.training_observations), + "rejected_or_unresolved_observation_count": len(dataset.observations) + - len(dataset.training_observations), + "rejected_or_unresolved_statuses": [ + item.target_status.value + for item in dataset.observations + if item.target_status != TargetStatus.VERIFIED + ], + }, + "students": { + "distilled": { + "model_identity": student.model_identity, + "artifact_identity": canonical_digest(json.loads(student.serialize())), + "architecture_identity": student.architecture_identity, + "training_dataset_identity": student.training_dataset_identity, + "calibration_identity": student.calibration_identity, + "model_bytes": student.model_size_bytes, + "artifact_bytes": len(student.serialize()), + }, + "no_distillation_control": { + "model_identity": control.model_identity, + "artifact_identity": canonical_digest(json.loads(control.serialize())), + "architecture_identity": control.architecture_identity, + "training_dataset_identity": control.training_dataset_identity, + "calibration_identity": control.calibration_identity, + "model_bytes": control.model_size_bytes, + "artifact_bytes": len(control.serialize()), + }, + }, + "evaluations": evaluations, + "claim_boundary": { + "one_step_means": "one learned affine/tanh/softmax forward evaluation after preparation", + "student_authority": AUTHORITY, + "teacher_fallback_is_explicit": True, + "success_claim_supported": False, + }, + "limitations": [ + "synthetic forge relevance features do not establish production utility", + "latency measurements are host-dependent and excluded from semantic identities", + "the student is diagnostic-only and cannot verify evidence, grant permissions, or promote generations", + "the recurrent teacher remains available as an explicit fallback rather than being silently replaced", + ], + "authority": AUTHORITY, + "semantics": "experimental-reference-measurement; not-a-verdict", + } + report_path = destination / "reference-study.json" + report_path.write_text(json.dumps(report, indent=2, sort_keys=True) + "\n", encoding="utf-8") + return { + "artifacts": { + "teacher": str(teacher_path), + "student": str(student_path), + "control": str(control_path), + "records": str(records_path), + }, + "evidence": str(report_path), + "report": report, + } + + +__all__ = [ + "ARTIFACT_SCHEMA", + "DECISION_SCHEMA", + "DISTILLATION_DATASET_SCHEMA", + "DISTILLATION_RECORD_SCHEMA", + "DistillationDataset", + "FallbackResult", + "OneStepSpecialistError", + "OneStepStudentDecision", + "OneStepStudentModel", + "StudentCalibration", + "StudentOperatingEnvelope", + "TargetStatus", + "TeacherObservation", + "build_distillation_dataset", + "build_one_step_reference_artifacts", + "calibrate_one_step_student", + "collect_teacher_observation", + "infer_with_teacher_fallback", + "train_one_step_student", +] diff --git a/tests/test_one_step_specialist.py b/tests/test_one_step_specialist.py new file mode 100644 index 0000000..89d0508 --- /dev/null +++ b/tests/test_one_step_specialist.py @@ -0,0 +1,249 @@ +from __future__ import annotations + +import json +import tempfile +import unittest +from dataclasses import replace +from pathlib import Path + +from mnel.core import canonical_digest +from mnel.one_step_specialist import ( + ARTIFACT_SCHEMA, + DECISION_SCHEMA, + DISTILLATION_DATASET_SCHEMA, + DISTILLATION_RECORD_SCHEMA, + OneStepSpecialistError, + OneStepStudentModel, + StudentOperatingEnvelope, + TargetStatus, + build_distillation_dataset, + build_one_step_reference_artifacts, + calibrate_one_step_student, + collect_teacher_observation, + infer_with_teacher_fallback, + train_one_step_student, +) +from mnel.recurrent_specialist import ( + SpecialistContextState, + calibrate_recurrent_specialist, + train_recurrent_specialist, +) + +TRAINING_ROWS = ( + {"record_id": "train-relevant-a", "features": [900, 820, 760, 880], "expected": "relevant"}, + {"record_id": "train-relevant-b", "features": [820, 760, 700, 800], "expected": "relevant"}, + {"record_id": "train-relevant-c", "features": [760, 700, 660, 720], "expected": "relevant"}, + {"record_id": "train-relevant-d", "features": [700, 740, 720, 760], "expected": "relevant"}, + {"record_id": "train-irrelevant-a", "features": [120, 180, 160, 100], "expected": "irrelevant"}, + {"record_id": "train-irrelevant-b", "features": [220, 120, 180, 160], "expected": "irrelevant"}, + {"record_id": "train-irrelevant-c", "features": [160, 220, 120, 180], "expected": "irrelevant"}, + {"record_id": "train-irrelevant-d", "features": [260, 180, 220, 200], "expected": "irrelevant"}, +) + + +def make_teacher(): + base_rows = tuple( + {**row, "label": row["expected"]} for row in TRAINING_ROWS[:2] + TRAINING_ROWS[4:6] + ) + teacher = train_recurrent_specialist( + base_rows, + target_role="forge.evidence-relevance", + generation_identity=canonical_digest({"test": "teacher-generation"}), + ) + return calibrate_recurrent_specialist( + teacher, + ( + *base_rows, + {"record_id": "calibration", "features": [760, 700, 660, 720], "label": "relevant"}, + ), + )[0] + + +def make_dataset(): + teacher = make_teacher() + rows = ( + *TRAINING_ROWS, + {"record_id": "rejected", "features": [900, 820, 760, 880], "expected": "irrelevant"}, + {"record_id": "unknown", "features": [640, 620, 600, 580]}, + {"record_id": "abstention", "features": [500, 500, 500, 500], "expected": "ABSTAIN"}, + ) + observations = tuple(collect_teacher_observation(teacher, row) for row in rows) + return teacher, rows, observations, build_distillation_dataset(observations) + + +class OneStepSpecialistTests(unittest.TestCase): + def test_teacher_observations_retain_lineage_and_do_not_promote_targets(self) -> None: + teacher, _, observations, dataset = make_dataset() + statuses = [item.target_status for item in observations] + self.assertEqual(statuses[:8], [TargetStatus.VERIFIED] * 8) + self.assertEqual( + statuses[-3:], + [TargetStatus.REJECTED_TEACHER, TargetStatus.UNKNOWN, TargetStatus.ABSTENTION], + ) + self.assertEqual(len(dataset.observations), 11) + self.assertEqual(len(dataset.training_observations), 8) + self.assertEqual(dataset.teacher_model_identity, teacher.model_identity) + self.assertEqual(dataset.to_dict()["schema"], "mnel-distillation-dataset/0.1") + self.assertEqual(observations[0].to_dict()["schema"], DISTILLATION_RECORD_SCHEMA) + self.assertNotIn("verdict", observations[0].to_dict()) + self.assertEqual( + dataset.dataset_identity, build_distillation_dataset(observations).dataset_identity + ) + reloaded = type(dataset).load(dataset.serialize()) + self.assertEqual(reloaded.dataset_identity, dataset.dataset_identity) + self.assertEqual(reloaded.distillation_identity, dataset.distillation_identity) + self.assertEqual(reloaded.training_targets, dataset.training_targets) + broken = json.loads(dataset.serialize()) + broken["teacher_model_identity"] = canonical_digest({"wrong": "teacher"}) + with self.assertRaises(OneStepSpecialistError): + type(dataset).load(broken) + + def test_student_is_genuinely_trained_reloadable_and_one_pass(self) -> None: + teacher, _, _, dataset = make_dataset() + student, calibration = calibrate_one_step_student( + train_one_step_student(dataset, distilled=True, seed=23), + TRAINING_ROWS, + ) + reloaded = OneStepStudentModel.load(student.serialize()) + self.assertEqual(student.model_identity, reloaded.model_identity) + self.assertEqual(student.artifact_identity, reloaded.artifact_identity) + decision = student.infer([780, 740, 700, 760]) + replay = reloaded.infer([780, 740, 700, 760]) + self.assertEqual(decision.decision, "relevant") + self.assertEqual(decision.learned_forward_passes, 1) + self.assertGreater(decision.inference_operations, 0) + self.assertEqual(decision.decision_identity, replay.decision_identity) + self.assertEqual(decision.to_dict()["schema"], DECISION_SCHEMA) + self.assertEqual(student.calibration_identity, calibration.calibration_identity) + self.assertEqual(student.teacher_model_identity, teacher.model_identity) + self.assertEqual(student.to_dict()["architecture"]["learned_forward_passes"], 1) + self.assertNotIn("promotion", decision.to_dict()) + + def test_low_confidence_ood_and_incompatible_context_abstain(self) -> None: + _, _, _, dataset = make_dataset() + student = train_one_step_student(dataset) + zero = replace( + student, + input_weights=tuple((0.0,) * 4 for _ in range(4)), + hidden_bias=(0.0,) * 4, + output_weights=tuple((0.0,) * 4 for _ in range(2)), + output_bias=(0.0,) * 2, + checkpoint_identity=canonical_digest({"test": "zero-checkpoint"}), + operating_envelope=StudentOperatingEnvelope( + minimum_confidence=700, maximum_distance=1400 + ), + model_identity="", + artifact_identity="", + ) + zero = replace(zero, model_identity=zero.content_identity) + low = zero.infer([780, 740, 700, 760]) + self.assertTrue(low.abstained) + self.assertEqual(low.escalation_reason, "insufficient-calibrated-confidence") + self.assertEqual(low.learned_forward_passes, 1) + ood = student.infer([1000, -1000, 1000, -1000]) + self.assertTrue(ood.abstained) + self.assertTrue(ood.out_of_distribution) + self.assertEqual(ood.learned_forward_passes, 0) + wrong_context = SpecialistContextState( + provider_identity=student.teacher_provider_identity, + generation_identity=canonical_digest({"wrong": "generation"}), + role_identity=student.target_role, + source_observation_identities=(), + feature_mean=(0, 0, 0, 0), + ) + incompatible = student.infer([780, 740, 700, 760], context=wrong_context) + self.assertTrue(incompatible.abstained) + self.assertTrue(incompatible.escalation_reason.startswith("unsupported-envelope:")) + self.assertEqual(incompatible.learned_forward_passes, 0) + malformed = student.infer([780, 740, 700, 760], context={"feature_mean": [0, 0, 0, 0]}) # type: ignore[arg-type] + self.assertTrue(malformed.abstained) + self.assertEqual(malformed.learned_forward_passes, 0) + + def test_fallback_is_explicit_and_lineage_bound(self) -> None: + teacher, _, _, dataset = make_dataset() + student = calibrate_one_step_student(train_one_step_student(dataset), TRAINING_ROWS)[0] + result = infer_with_teacher_fallback(student, teacher, [1000, -1000, 1000, -1000]) + self.assertTrue(result.student_decision.abstained) + self.assertTrue(result.fallback_invoked) + self.assertIsNotNone(result.teacher_decision) + self.assertEqual(result.final_decision, "ABSTAIN") + self.assertEqual(result.to_dict()["student_decision"]["decision"], "ABSTAIN") + other = replace( + teacher, + generation_identity=canonical_digest({"test": "other-generation"}), + model_identity="", + ) + with self.assertRaises(OneStepSpecialistError): + infer_with_teacher_fallback(student, other, [1000, -1000, 1000, -1000]) + + def test_authority_fields_and_unknown_artifact_fields_are_rejected(self) -> None: + _, _, _, dataset = make_dataset() + student = train_one_step_student(dataset) + payload = json.loads(student.serialize()) + payload["verdict"] = "PASS" + with self.assertRaises(OneStepSpecialistError): + OneStepStudentModel.load(payload) + payload = json.loads(student.serialize()) + payload["architecture"]["authority"] = "verifier" + payload["artifact_identity"] = canonical_digest( + {key: value for key, value in payload.items() if key != "artifact_identity"} + ) + with self.assertRaises(OneStepSpecialistError): + OneStepStudentModel.load(payload) + payload = json.loads(student.serialize()) + payload["unexpected"] = True + payload["artifact_identity"] = canonical_digest( + {key: value for key, value in payload.items() if key != "artifact_identity"} + ) + with self.assertRaises(OneStepSpecialistError): + OneStepStudentModel.load(payload) + payload = json.loads(student.serialize()) + payload["architecture"]["learned_forward_passes"] = 2 + payload["artifact_identity"] = canonical_digest( + {key: value for key, value in payload.items() if key != "artifact_identity"} + ) + with self.assertRaises(OneStepSpecialistError): + OneStepStudentModel.load(payload) + + def test_reference_study_has_teacher_student_baseline_control_and_measurements(self) -> None: + first = build_one_step_reference_artifacts(tempfile.mkdtemp()) + second = build_one_step_reference_artifacts(tempfile.mkdtemp()) + report = first["report"] + self.assertEqual(report["study_identity"], second["report"]["study_identity"]) + self.assertEqual(report["target_role"], "forge.evidence-relevance") + self.assertEqual( + report["distillation"]["rejected_or_unresolved_statuses"], + ["rejected-teacher-target", "unknown", "abstention"], + ) + self.assertEqual(report["claim_boundary"]["success_claim_supported"], False) + self.assertEqual(report["evaluations"]["iterative_teacher"]["abstentions"], 2) + self.assertEqual( + report["evaluations"]["one_step_distilled_student"]["learned_forward_passes"][:4], + [1, 1, 1, 1], + ) + self.assertEqual( + report["evaluations"]["no_distillation_control"]["learned_forward_passes"][:4], + [1, 1, 1, 1], + ) + self.assertEqual( + report["evaluations"]["classical_nearest_centroid_baseline"]["abstentions"], 2 + ) + self.assertTrue(report["evaluations"]["fallback"]["fallback_invocations"] >= 1) + self.assertTrue(report["students"]["distilled"]["artifact_bytes"] > 0) + + def test_schema_files_are_explicit_and_strict(self) -> None: + root = Path(__file__).resolve().parents[1] + for name, schema, title in ( + ("mnel-one-step-specialist.schema.json", ARTIFACT_SCHEMA, "artifact"), + ("mnel-one-step-specialist-decision.schema.json", DECISION_SCHEMA, "decision"), + ("mnel-distillation-record.schema.json", DISTILLATION_RECORD_SCHEMA, "record"), + ("mnel-distillation-dataset.schema.json", DISTILLATION_DATASET_SCHEMA, "dataset"), + ): + value = json.loads((root / "schemas" / name).read_text()) + self.assertEqual(value["$schema"], "https://json-schema.org/draft/2020-12/schema") + self.assertFalse(value.get("additionalProperties", True)) + self.assertIn(title, value["title"].lower()) + + +if __name__ == "__main__": + unittest.main()