Autolife Toolkit

Intrinsic-first calibration for the three head cameras on the Autolife robot, followed by camera-to-head extrinsic calibration with a gripper-mounted ArUco target.

Camera	Model	ROS 2 image topic
forehead	pinhole	/camera/camera/color/image_raw
left_eye	fisheye	/robot/camera/head_left_eye/image_raw
right_eye	fisheye	/robot/camera/head_right_eye/image_raw

Python environments

Use uv for every command. The project environment (Python 3.12) runs OpenCV, optimization, IK, and BIT* planning:

uv sync --extra dev
uv run autolife-calib --help

This ROS 2 Humble build exposes CPython 3.8 bindings. The scripts/autolife-calib-ros wrapper uses a uv-managed Python 3.8 runtime; it does not invoke /usr/bin/python3:

./scripts/autolife-calib-ros --help

uv downloads that managed interpreter automatically if necessary.

1. Intrinsics

The measured intrinsic checkerboard has 7×9 physical squares, hence 6×8 OpenCV inner corners. Its measured 26.8×34.4 cm grid gives a mean pitch of 38.25 mm. Capture 30–40 sharp, distinct views per camera, including image edges and varied depth/tilt:

./scripts/autolife-calib-ros capture-intrinsics --camera forehead --target-views 35
./scripts/autolife-calib-ros capture-intrinsics --camera left_eye --target-views 35
./scripts/autolife-calib-ros capture-intrinsics --camera right_eye --target-views 35

Press Space to save a detected view and Q/Escape to finish. Compute all available intrinsics in the project environment:

uv run autolife-calib calibrate-intrinsics \
  --dataset datasets/intrinsics --out results/intrinsics

Extrinsic calibration always loads these results as fixed camera models; it does not re-optimize intrinsics.

2. Extrinsic target and equation

The extrinsic target matches AdaCompNUS/Franka-Toolkit:

• one DICT_6X6_250 ArUco marker;
• marker ID 0;
• 0.10 m outer marker side.

The complete backing board is 0.15×0.225 m. That physical size is used only for collision checking. A 2×3 cover of six spheres, each approximately 63 mm radius, includes the 5 mm board thickness and 10 mm safety margin.

For observation i, capture stores a fresh 24-DOF feedback vector while the robot is stationary. Offline FK reconstructs the moving head and gripper poses, and the solver uses

T_base_head(i) · T_head_camera · T_camera_target(i)
  = T_base_gripper(i) · T_gripper_target

and jointly estimates the two constant unknown transforms T_head_camera and T_gripper_target. Fixed-head observations are the same equation with repeated T_base_head values. Image-time ROS TF is retained in the dataset only as an audit; calibration reports its disagreement with measured-q24 FK but does not optimize from it.

The target is mounted once on the left gripper for all three cameras. This keeps one physical T_gripper_target definition throughout the dataset.

3. Tune the initial view

The defaults in configs/extrinsic_session.yaml already pass calibrated-image visibility, TRAC-IK, robot collision checks, attached-board clearance, and BIT* on the installed backend. Re-run the check after changing the physical mount:

uv run autolife-calib tune-extrinsic-start --camera forehead
uv run autolife-calib tune-extrinsic-start --camera left_eye
uv run autolife-calib tune-extrinsic-start --camera right_eye

Tune without editing YAML by overriding the desired marker center in the OpenCV camera frame (+X right, +Y down, +Z forward), neck RPY, or marker roll:

uv run autolife-calib tune-extrinsic-start --camera forehead \
  --center -0.004 0.037 0.312 --neck 0 -0.55 0 --roll-deg -166

This command only plans and writes a report; it never moves the robot.

4. Plan from the live robot state

Read feedback through the exact robot topic, then generate the complete motion plan. Keep the robot stationary between these two commands:

./scripts/autolife-calib-ros read-robot-state \
  --out results/live_robot_state.json

uv run autolife-calib plan-extrinsic-motion --camera forehead \
  --start-state results/live_robot_state.json \
  --out results/forehead_motion.json

The planner samples front-facing marker poses inside a sphere in the calibrated forehead-camera frame, then derives each required gripper pose through the configured T_gripper_target. It checks the complete printed marker again using FK of the actual IK endpoint in all three calibrated camera models. This makes sampling independent of how the target is mounted and guarantees one shared run is geometrically usable by all three cameras. Every accepted arm segment uses bitstar, is time-parameterized with TOPP-RA, and is rejected if any interpolated waypoint collides with the six-sphere board attachment. Neck-only transitions are separately checked with the full-body collision model. The default session uses 8 head configurations and targets 25 accepted arm poses per head configuration (200 shared observations). Each head group tests up to 10 candidates per requested accepted pose. A plan is executable only if all 200 poses are found. Planning is offline: it never commands the robot.

5. Preview in PyBullet

Inspect the complete planned motion and the attached 15 x 22.5 cm target before hardware execution:

uv run autolife-calib preview-extrinsic-motion \
  --plan results/forehead_motion.json

The viewer starts paused. Press SPACE to play or pause, use the left/right arrows to step, R to restart, and Q to exit. Use --speed 2 for twice-real- time playback. This command has no ROS imports and cannot move the robot.

6. Execute and capture

Close the left gripper on the mounted target (the driver uses 0 for fully open and 360 for fully closed):

./scripts/autolife-calib-ros set-gripper --side left --position 360

The following command moves the physical robot. Verify the target mount, clear the workspace, keep the emergency stop accessible, and inspect the plan report first:

./scripts/autolife-calib-ros execute-extrinsic-motion \
  --plan results/forehead_motion.json --out datasets/extrinsics

The execution backend follows the reference implementation under ~/work/calibration:

• state feedback: /topic_arm_whole_body_and_gripper_current_joints_status_0_281;
• neck position commands: /topic_arm_whole_body_target_joints_position_0_281;
• 18-DOF time-parameterized arm trajectories: /topic_arm_move_joints_trajectory_0_281.

Driver JSON angles are converted between degrees and the planner's radians. Motion is parameterized directly by TOPP-RA with 1.0 rad/s velocity and 1.0 rad/s² acceleration limits; both scale factors remain 1.0. Execution waits for the full parameterized duration and three stable goal-feedback messages, so camera settling cannot overlap the tail of a trajectory. After reaching each endpoint, the executor remains still for 2 seconds while continuously servicing camera and TF subscriptions, then captures the next fresh four-stream bundle:

• forehead RGB;
• left-eye RGB;
• right-eye RGB;
• 640×360 RealSense depth aligned to forehead RGB, losslessly stored as 16UC1 PNG with a configured 0.001 m/unit scale.

After each 25-observation head group, the arm stays at the final sample while the collision-validated neck transition moves to the next head configuration. The next arm segment is planned directly from that physical endpoint. There is no return-to-home transit or reverse-path fallback between groups, and execution stops at the final captured sample.

Each camera is saved as a first-class calibration dataset with its own ROS timestamp, the shared stationary q24 feedback sample, and image-time T_base_head/T_base_gripper TF values for auditing:

<dataset>/forehead/frame_NNNN.png
<dataset>/left_eye/frame_NNNN.png
<dataset>/right_eye/frame_NNNN.png
<dataset>/forehead/streams/depth/frame_NNNN.png

Independently published images are not assigned a made-up common timestamp. All three RGB frames are saved at every observation even if a particular marker detection fails; the offline solver filters each camera independently. Detection counts for all three cameras are printed during capture. The executor refuses a stale plan if live joints differ from its recorded start by more than 0.05 rad. Override the delay only when needed with --settle <seconds>. Run the motion trajectory only once; separate left-eye and right-eye motion runs are not required.

6. Solve extrinsics

uv run autolife-calib calibrate-extrinsics \
  --dataset datasets/extrinsics \
  --intrinsics results/intrinsics \
  --out results/extrinsics

The solver loads the three camera datasets, filters marker detections independently, and calibrates all three in one command. Depth remains auxiliary and is not treated as a fourth camera extrinsic.

Results include T_head_camera, the jointly estimated T_gripper_target, reprojection RMS, board-in-gripper consistency, and the deviation from the URDF initial camera pose.

Configuration

• configs/cameras.yaml: the three cameras, models, topics, and URDF guesses.
• configs/board.yaml: intrinsic checkerboard geometry only.
• configs/extrinsic_board.yaml: exact ArUco marker, physical backing size, attachment cover, and coarse gripper mount guess.
• configs/extrinsic_session.yaml: per-camera start view, local sampling, head perturbations, and BIT* parameters.

Development

PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 uv run pytest
uv run ruff check src tests

PYTEST_DISABLE_PLUGIN_AUTOLOAD=1 prevents unrelated ROS pytest entry points from being loaded into the project test environment.

License

MIT