Skip to content

feat(abc): ABC-130k mcap->zarr pipeline + yam_bimanual embodiment - #10

Open
AnikethCheluva wants to merge 2 commits into
graph-hpt-qwenfrom
graph-abc
Open

feat(abc): ABC-130k mcap->zarr pipeline + yam_bimanual embodiment#10
AnikethCheluva wants to merge 2 commits into
graph-hpt-qwenfrom
graph-abc

Conversation

@AnikethCheluva

Copy link
Copy Markdown

Port of GaTech-RL2#606 onto the graph fork.

Adds the YAM two-arm teleoperation station behind ABC-130k as a first-class
embodiment (id 7) plus the converter, subset builder, uploader and metadata
backfill under egomimic/scripts/abc_process, with their sbatch launchers.

YAM is structurally close to Eva -- bimanual, parallel-jaw, per-arm
obs/cmd_ee_pose plus gripper -- with three differences that drive the code:
extrinsics come from the published station rig rather than the data (so
eef_frame, which needs none, is the mode that works on ZED-X episodes);
intrinsics are per-episode because two station types with different
calibration ship in the dataset, hence INTRINSICS = None; and the world
frame is station-anchored rather than egocentric, so there is no head pose or
keypoint stream.

A batch cannot mix image sizes and these datasets ship 640x480, 640x360,
1280x720 and 848x480, so image_hw pins every episode to one size. The
intrinsics are rescaled by the SAME per-axis factors -- row 0 with width, row 1
with height. Scaling by a single shared factor is what produced the anamorphic
K bug, and every projected overlay inherits it. Both helpers read the channel
layout rather than assuming HWC, because the decoder emits channels-first and
assuming otherwise turns each sample into a decode failure or a garbage scale
factor.

Graph integration: ActionTargetBuilder now takes action_key instead of
hardcoding actions. Planar names its chunk actions, but the Eva/Human/Yam
cartesian transform lists concatenate into actions_cartesian, and the graph
should read the key that exists rather than force embodiments to rename their
output. egomimic/hydra_configs/model/abc/yam_bimanual_dp.yaml is the
graph-native DP stage list for this data, and the ABC data configs declare
batch_keys for the keys their transform lists add -- tools/config_graph.py
derives seed keys from the key_map alone, so without that declaration it
reports the concatenated pose and action keys as unprovided. Both the train and
inference graphs lint clean.

Co-Authored-By: Claude Opus 5 noreply@anthropic.com
Claude-Session: https://claude.ai/code/session_01DcPCNAW3GKERANrqvfSCY1

@AnikethCheluva
AnikethCheluva marked this pull request as ready for review September 5, 2026 21:01
@AnikethCheluva
AnikethCheluva changed the base branch from graph-nested-val to graphite-base/10 September 8, 2026 05:01
@AnikethCheluva
AnikethCheluva changed the base branch from graphite-base/10 to graph-bounds-gate September 8, 2026 05:02
@AnikethCheluva
AnikethCheluva changed the base branch from graph-bounds-gate to graphite-base/10 September 8, 2026 05:02
@AnikethCheluva
AnikethCheluva changed the base branch from graphite-base/10 to graph-hpt-qwen September 8, 2026 21:07
@AnikethCheluva
AnikethCheluva changed the base branch from graph-hpt-qwen to graphite-base/10 September 8, 2026 21:07
@AnikethCheluva
AnikethCheluva changed the base branch from graphite-base/10 to graph-hpt-qwen September 9, 2026 22:21
Port of GaTech-RL2#606 onto the graph fork.

Adds the YAM two-arm teleoperation station behind ABC-130k as a first-class
embodiment (id 7) plus the converter, subset builder, uploader and metadata
backfill under egomimic/scripts/abc_process, with their sbatch launchers.

YAM is structurally close to Eva -- bimanual, parallel-jaw, per-arm
obs/cmd_ee_pose plus gripper -- with three differences that drive the code:
extrinsics come from the published station rig rather than the data (so
`eef_frame`, which needs none, is the mode that works on ZED-X episodes);
intrinsics are per-episode because two station types with different
calibration ship in the dataset, hence `INTRINSICS = None`; and the world
frame is station-anchored rather than egocentric, so there is no head pose or
keypoint stream.

A batch cannot mix image sizes and these datasets ship 640x480, 640x360,
1280x720 and 848x480, so `image_hw` pins every episode to one size. The
intrinsics are rescaled by the SAME per-axis factors -- row 0 with width, row 1
with height. Scaling by a single shared factor is what produced the anamorphic
K bug, and every projected overlay inherits it. Both helpers read the channel
layout rather than assuming HWC, because the decoder emits channels-first and
assuming otherwise turns each sample into a decode failure or a garbage scale
factor.

Graph integration: `ActionTargetBuilder` now takes `action_key` instead of
hardcoding `actions`. Planar names its chunk `actions`, but the Eva/Human/Yam
cartesian transform lists concatenate into `actions_cartesian`, and the graph
should read the key that exists rather than force embodiments to rename their
output. `egomimic/hydra_configs/model/abc/yam_bimanual_dp.yaml` is the
graph-native DP stage list for this data, and the ABC data configs declare
`batch_keys` for the keys their transform lists add -- tools/config_graph.py
derives seed keys from the key_map alone, so without that declaration it
reports the concatenated pose and action keys as unprovided. Both the train and
inference graphs lint clean.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01DcPCNAW3GKERANrqvfSCY1
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant