Port three torus candidate objectives to conditional U-Socket graphs

docs/action-flow-usocket-candidates.md

@@ -0,0 +1,104 @@
+# Conditional U-Socket ports of the torus candidates
+
+These are three research candidates, not three equivalent reconstruction-weight
+ablations. All use U-Socket BC, fresh seed42, H16 normalized `[x,y,cosθ,sinθ]`,
+latent8 per token, one observation frame, a shared 12×512 AdaLN field, private
+context-free two-layer width20 codecs, AdamW3e-5, 8K warmup, cosine floor3e-6,
+240K updates, batch32 and BF16 on one H100 or H200. No warm-start or
+reconstruction-only phase. The generic pipeline runner is unchanged.
+
+## Configs and gradients
+
+### FM-only clean stop-gradient (method 2)
+
+Config: `pusht/action_flow_bc_usocket_latent_fm_sg_recon1_s42`.
+
+- Reconstruction: `A → E → g → MSE(A)`; updates E and g, weight1.
+- Latent FM: `A → E → stop-gradient → latent bridge → v(z,t,O) → latent MSE`;
+  both the bridge endpoint and velocity target detach. Updates v and the
+  observation encoder, not E or g.
+- Action Flow: `A → E → attached latent bridge → v → residual → J_g(residual)`;
+  squared residual mean updates E, v, g and the observation encoder, weight1.
+
+The two v forwards share time, Gaussian noise and conditioning-dropout mask.
+Fourteen samples per content means 28 field sample-equivalents, not 14, for
+this variant. A zero FM/reconstruction parameter intersection is intentional;
+its gradient cosine is undefined, not evidence of broken training.
+Inference: `Gaussian latent → reverse Euler16(v,O) → g → action`.
+
+### Learned Gaussian-bridge likelihood (method 3)
+
+Config: `pusht/action_flow_bc_usocket_bridge_likelihood_s42`.
+
+Private reference mean `mu(A,t)=(1-t)E_raw(A,t)` ends exactly at zero. The
+reference stage, Gaussian noising stage, conditional reverse-mean transformer,
+decoder and objective are separate nodes.
+
+Interior: `A → mu_k,mu_(k-1) → Gaussian z_k and attached reverse target →
+M(z,k,O)=z+field(z,k/32,O) → equal-variance Gaussian KL`.
+Boundary: `A → mu_1 + .1*noise → M(z,1,O) → g → action Gaussian NLL`.
+Both mean and target gradients remain attached. Both terms update the private
+mean encoder, field and observation encoder; the boundary also updates g.
+There is no latent-FM, clean reconstruction, JVP or decoded-noise scale loss.
+
+K32, sigma1=.1, sigma32=1, geometric schedule, rho=.95, tau=.02. Draw 14
+interior levels/noises per content plus one independent boundary noise. The
+batched field sees 15 sample-equivalents in one forward. Interior loss uses the
+31× uniform-level correction and variance `(1-rho²)*sigma_(k-1)²`.
+
+All squared coordinate residuals are **summed over the full H16 chunk**, then
+averaged over examples/level samples. Boundary weighting is therefore 80000×
+per-coordinate action MSE for H16×D4, not the toy three-coordinate coefficient.
+Logged likelihood terms omit parameter-independent Gaussian constants; they
+are not exact reported marginal log-likelihood estimates.
+
+Inference: `z32~N(0,I) → 31 stochastic Gaussian reverse transitions →
+final learned mean M(z1,1,O) → g → add tau*action_noise`. Exactly32 field calls,
+31 latent innovations, and one action-noise draw. No deterministic Euler
+substitution. Finite output noise models a smoothed action distribution.
+
+`Train/MSE` is decoded noisy-boundary-mean error; `Valid/MSE` is actual generated
+action error. Neither is clean-codec reconstruction. Ordinary CFM trajectory/
+clean-latent/Jacobian diagnostics are disabled; normalized/native generated
+errors and fixed-bank stochastic EnergyScore@32 remain required.
+
+### Exact graph section (method 4, restricted diagnostic)
+
+Config: `pusht/action_flow_bc_usocket_graph_section_s42`.
+
+`E(A)=(A,f(A))`; split each latent token into x4,h4 and decode
+`g(x,h)=x + [R(x,h)-R(x,f(x))]`. The subtraction is grouped before adding x.
+The same f parameter objects are used at both boundaries. A model-specific
+Hydra factory injects them once: repeating a Hydra config is not weight sharing.
+E owns only f; decoder-only R is not included in E's parameter set.
+
+Training: `A → E → latent bridge → v → latent FM`, plus
+`latent residual → J_g → Action Flow loss`, both fully attached. No reconstruction
+optimizer term; `g(E(A))=A` is checked and logged as a diagnostic. Inference uses
+Gaussian latent → reverse Euler16 → g. Fixed-level/Jacobian diagnostics remain;
+activation matching is disabled with an empty map and cknna_k=0.
+
+**R6 FAIL / restricted diagnostic:** explicit action coordinates are embedded in
+the latent. Exact reconstruction is not a general flexible-interface solution
+and is not evidence of good generation.
+
+## Comparison and operational boundaries
+
+Methods2/4 retain the historical U-Socket scale-regularizer weight0; the torus
+winners used scale1. That difference is explicit, not a faithful toy ablation.
+Conditional quality, scale stability and transfer remain OPEN. R2–R5/R8 are
+implemented contracts, not demonstrated cross-embodiment transfer; R7 requires
+observed GPU cost. R1/R9/R10 are not passed merely by finite smoke metrics.
+
+Use the maintained `scripts/train/launch_action_flow_usocket.sbatch` typed
+entrypoint with exact source/runtime/config/data/norm bindings. Reuse hashed
+2999-episode content, 2970/29 split, and train-only normalization receipts;
+never recompute unchanged statistics to prepare another row. Validation10K,
+immutable checkpoint40K, all scheduled/signal saves retained. Full training
+requires that candidate's real optimizer+validation smoke and strict reload.
+
+An explicitly authorized ICE/Phoenix queue race needs distinct attempt paths and
+tracked scheduler IDs. Keep the first allocation, confirm loser cancellation,
+and never let two writers share W&B/output. Preserve the original queue deadline.
+Restart testing must prove checkpoint-based optimizer continuation and the same
+W&B ID; a passing ordinary smoke alone does not prove requeue continuity.

docs/cluster-pipeline.md

@@ -32,6 +32,12 @@ branch as its parent and carries shared runtime fixes without replacing its
 model code. Do not treat the synthetic Action Flow scripts on `main` as the
 real-data implementation or reset this stack to `main`.
 
+The `codex/torus-winners-usocket-system-test-20260907` child adds three explicit
+conditional candidate configs to that same launcher. Read
+[their objective and sampler contracts](action-flow-usocket-candidates.md)
+before selecting one. The likelihood arm is not ordinary latent FM; the exact
+graph-section arm is a restricted diagnostic, not an unrestricted solution.
+
 ## Cluster entry points
 
 | Host | Execution authority | Current-task discovery |

egomimic/hydra_configs/experiment/pusht/action_flow_bc_usocket_bridge_likelihood_s42.yaml

@@ -0,0 +1,44 @@
+# @package _global_
+defaults:
+  - /experiment/pusht/action_flow_usocket_candidate_common
+  - override /model: bf/us_action_flow_bc_bridge_likelihood
+  - _self_
+
+name: action_flow_bc_usocket_bridge_likelihood_s42
+description: "Learned Gaussian bridge likelihood; K32 stochastic reverse chain; full H16 coordinate-sum reduction"
+train:
+  action_flow_method: gaussian_bridge_likelihood
+
+evaluator:
+  # Straight-CFM clean endpoints, velocities and Euler trajectories are not
+  # defined for this model. Native MSE and stochastic EnergyScore32 remain on.
+  action_flow_diagnostics:
+    enabled: false
+
+run_provenance:
+  objective:
+    method: gaussian_bridge_likelihood
+    components: [interior_bridge_kl, action_boundary_nll]
+    target_mean_gradients: attached
+    interior_samples_per_content: ${model.interior_samples_per_content}
+    num_levels: ${model.num_levels}
+    sigma_min: ${model.sigma_min}
+    sigma_max: ${model.sigma_max}
+    rho: ${model.rho}
+    tau: ${model.tau}
+    reduction: sum_all_horizon_coordinates_then_batch_mean
+    interior_level_multiplier: 31
+    # H16 × D4 / (2 × .02²) = 80000 times per-coordinate MSE.
+    boundary_per_coordinate_mse_multiplier: 80000.0
+    reconstruction_only_warmup_steps: 0
+    decoded_noise_scale_weight: 0.0
+  inference:
+    sampler: gaussian_bridge_reverse_chain
+    steps: ${model.num_inference_steps}
+    latent_innovations: 31
+    output_noise_std: ${model.tau}
+    classifier_free_guidance: false
+  requirements:
+    r6: PASS_design
+    conditional_quality: OPEN
+    observation_noise_caveat: finite_tau_models_a_smoothed_action_distribution

egomimic/hydra_configs/experiment/pusht/action_flow_bc_usocket_graph_section_s42.yaml

@@ -0,0 +1,72 @@
+# @package _global_
+defaults:
+  - /experiment/pusht/action_flow_usocket_candidate_common
+  - override /model: bf/us_action_flow_bc_graph_section
+  - _self_
+
+name: action_flow_bc_usocket_graph_section_s42
+description: "Exact graph section; attached FM and Action Flow; R6 FAIL restricted diagnostic"
+train:
+  action_flow_method: graph_section_diagnostic
+
+evaluator:
+  action_flow_diagnostics:
+    enabled: true
+    noise_seed_bank_path: ${hydra:runtime.cwd}/egomimic/hydra_configs/evaluator/energy_score_seed_bank_k32_v1.json
+    noise_seed_bank_sha256: 88657b829905d4374823db145ded19b99cec4735f76694734473bcee068bb5b6
+    raw_noise_levels: [0.0, 0.25, 0.5, 0.75, 1.0]
+    max_batches_per_rank: 1
+    max_samples: 16
+    jacobian_samples: 2
+    capture_activations: false
+    activation_layer_map: {}
+    cknna_k: 0
+    native_error:
+      enabled: true
+      type: usocket_native_xy_wrapped_theta_mse_v1
+      space: decoded_native_x_y_theta_radians
+      native_theta_index: 2
+      wrap_period_radians: 6.283185307179586
+      reduction: mean_squared_error_over_horizon_and_native_coordinates
+      normalizer: bound_train_only_evaluator_normalizer
+      native_decoder: egomimic.pipeline.pushshapes.USocketRotVecNativeDecoder
+    artifact_root: ${paths.output_dir}/validation_predictions/action_flow_diagnostics
+    validation_view:
+      definition: first_deterministic_validation_batch_per_rank
+      split_manifest_sha256: ${run_provenance.split_manifest_sha256}
+      per_rank_batch_size: 16
+      world_size: 1
+    provenance:
+      source_commit: ${run_provenance.source_commit}
+      normalization_sha256: ${run_provenance.normalization_sha256}
+      split_manifest_sha256: ${run_provenance.split_manifest_sha256}
+      dataset_content:
+        manifest_sha256: ${run_provenance.content_manifest_sha256}
+        aggregate_sha256: ${run_provenance.dataset_content_aggregate_sha256}
+      action_representation: x_y_cos_theta_sin_theta
+      clean_endpoint: t0
+      gaussian_endpoint: t1
+      latent_shape: [16, 8]
+      flow_samples_per_content: ${model.flow_samples_per_content}
+      condition_dropout_probability: ${model.condition_dropout_probability}
+      sampler: reverse_euler
+      sampler_steps: ${model.num_inference_steps}
+      fixed_level_coupling: one_clean_latent_and_one_gaussian_per_condition
+      decoder_jacobian_evaluation: declared_bridge_state_at_each_fixed_noise_level
+      activation_capture: disabled_for_graph_section
+
+run_provenance:
+  objective:
+    method: graph_section_diagnostic
+    flow_clean_gradient_mode: full
+    action_velocity_clean_gradient_mode: full
+    reconstruction_is_optimizer_objective: false
+    flow_weight: ${model.flow_weight}
+    reconstruction_weight: ${model.reconstruction_weight}
+    action_velocity_weight: 1.0
+    flow_samples_per_content: ${model.flow_samples_per_content}
+    decoded_noise_scale_weight: 0.0
+    monotonic_weight: 0.0
+  requirements:
+    r6: FAIL_restricted_diagnostic
+    conditional_quality: OPEN

egomimic/hydra_configs/experiment/pusht/action_flow_bc_usocket_latent_fm_sg_recon1_s42.yaml

@@ -0,0 +1,72 @@
+# @package _global_
+defaults:
+  - /experiment/pusht/action_flow_usocket_candidate_common
+  - override /model: bf/us_action_flow_bc_latent_fm_sg
+  - _self_
+
+name: action_flow_bc_usocket_latent_fm_sg_recon1_s42
+description: "FM-only endpoint detach; fully attached Action Flow; recon1"
+train:
+  action_flow_method: latent_fm_stopgrad
+
+evaluator:
+  action_flow_diagnostics:
+    enabled: true
+    noise_seed_bank_path: ${hydra:runtime.cwd}/egomimic/hydra_configs/evaluator/energy_score_seed_bank_k32_v1.json
+    noise_seed_bank_sha256: 88657b829905d4374823db145ded19b99cec4735f76694734473bcee068bb5b6
+    raw_noise_levels: [0.0, 0.25, 0.5, 0.75, 1.0]
+    max_batches_per_rank: 1
+    max_samples: 16
+    jacobian_samples: 2
+    capture_activations: true
+    activation_layer_map: {0: 0, 1: 11}
+    cknna_k: 10
+    native_error:
+      enabled: true
+      type: usocket_native_xy_wrapped_theta_mse_v1
+      space: decoded_native_x_y_theta_radians
+      native_theta_index: 2
+      wrap_period_radians: 6.283185307179586
+      reduction: mean_squared_error_over_horizon_and_native_coordinates
+      normalizer: bound_train_only_evaluator_normalizer
+      native_decoder: egomimic.pipeline.pushshapes.USocketRotVecNativeDecoder
+    artifact_root: ${paths.output_dir}/validation_predictions/action_flow_diagnostics
+    validation_view:
+      definition: first_deterministic_validation_batch_per_rank
+      split_manifest_sha256: ${run_provenance.split_manifest_sha256}
+      per_rank_batch_size: 16
+      world_size: 1
+    provenance:
+      source_commit: ${run_provenance.source_commit}
+      normalization_sha256: ${run_provenance.normalization_sha256}
+      split_manifest_sha256: ${run_provenance.split_manifest_sha256}
+      dataset_content:
+        manifest_sha256: ${run_provenance.content_manifest_sha256}
+        aggregate_sha256: ${run_provenance.dataset_content_aggregate_sha256}
+      action_representation: x_y_cos_theta_sin_theta
+      clean_endpoint: t0
+      gaussian_endpoint: t1
+      latent_shape: [16, 8]
+      flow_samples_per_content: ${model.flow_samples_per_content}
+      condition_dropout_probability: ${model.condition_dropout_probability}
+      sampler: reverse_euler
+      sampler_steps: ${model.num_inference_steps}
+      fixed_level_coupling: one_clean_latent_and_one_gaussian_per_condition
+      decoder_jacobian_evaluation: declared_bridge_state_at_each_fixed_noise_level
+      activation_capture: real_forward_hooks_without_replay_or_replacement
+
+run_provenance:
+  objective:
+    method: latent_fm_stopgrad
+    flow_clean_gradient_mode: all_stopgrad
+    action_velocity_clean_gradient_mode: full
+    reconstruction_is_optimizer_objective: true
+    flow_weight: ${model.flow_weight}
+    reconstruction_weight: ${model.reconstruction_weight}
+    action_velocity_weight: 1.0
+    flow_samples_per_content: ${model.flow_samples_per_content}
+    decoded_noise_scale_weight: 0.0
+    monotonic_weight: 0.0
+  requirements:
+    r6: PASS_design
+    conditional_quality: OPEN