Bridging Language and Physics: Automated Design of Continuum Robots with Large Language Models

Overview

AID-SR is an agentic workflow for generating continuum-robot designs from a natural-language objective and a MuJoCo task environment. It proposes multiple designs, generates MuJoCo XML, checks physical validity, integrates the robot into the task scene, applies semantic review, and optionally pauses for human feedback. This repository contains the design agent, interactive interface, and downstream control and evaluation tools.

Demo

The demo shows the original Web workflow, automatic loading in the native MuJoCo viewer, actuator interaction, and feedback-driven refinement.

Installation

AID-SR supports macOS, Linux, and Windows. It requires Python 3.10–3.12 and uv.

cd AID-SR
uv sync --locked

Set your OpenAI API key on the machine that will run AID-SR.

macOS and Linux:

export OPENAI_API_KEY="your-api-key"

Windows PowerShell:

$env:OPENAI_API_KEY = "your-api-key"

Quick Start

List the bundled tasks:

uv run aid-sr tasks

Generate designs for a task:

uv run aid-sr run --task grasp

Enable terminal human review:

uv run aid-sr run --task grasp --human-review

Run multiple tasks:

uv run aid-sr batch --tasks button grasp tunnel --repeats 1

Use uv run aid-sr run --help for custom objective/environment files and execution options.

Interactive Web Demo

Start the interface on the machine running the agent:

uv run aid-sr web

Open http://127.0.0.1:5000. The interface displays live per-design output and pauses at human review until Approve, Refine, or Quit is selected.

If AID-SR runs on a remote machine, forward the Web port from the local macOS, Linux, or Windows computer:

ssh -N -L 5000:127.0.0.1:5000 user@server

During human review, a local watcher opens the task environment and reloads MuJoCo whenever a design is ready for inspection.

On macOS:

uv run python scripts/macos/load_mujoco.py \
  --server-url http://127.0.0.1:5000

On Windows PowerShell:

.\scripts\windows\load_mujoco.ps1 `
  -ServerUrl "http://127.0.0.1:5000" `
  -SimulatePath "C:\path\to\mujoco\bin\simulate.exe"

Training and Evaluation

Controller training and task evaluation tools are available in evaluation.

Tasks

Category	Task	Objective	Environment
Reaching	Button pushing	button.txt	button.xml
Reaching	Surface adaptation	surface.txt	surface_adaptation.xml
Reaching	Pipe insertion	pipe.txt	pipe.xml
Grasping	Simple grasping	grasp.txt	grasp_env.xml
Grasping	Zero-gravity grasping	grasp_zero_gravity.txt	grasp_env_zero_gravity.xml
Grasping	Delicate grasping	grasp_delicate.txt	delicate_grasp.xml
Grasping	Dynamic catching	dynamic_catch_2d.txt	dynamic_catch_2d.xml
Locomotion	Flat ground	flat_ground.txt	flat_ground.xml
Locomotion	Stairs	stairs.txt	stairs.xml
Locomotion	Obstacles	obstacle.txt	obstacle.xml
Locomotion	Gap crossing	gap.txt	gap.xml
Locomotion	Tunnel traversal	tunnel.txt	tunnel.xml
Manipulation	Drawer opening	drawer.txt	drawer.xml
Manipulation	Force-controlled brushing	brushing.txt	brushing.xml

Outputs

Each run creates one timestamped directory:

outputs/<run-id>/
├── input/
│   ├── objective.txt
│   └── environment.xml
├── designs/
│   └── design_01/
│       ├── design.json
│       ├── round_01_generated.xml
│       ├── round_01_integrated.xml
│       ├── final.xml
│       ├── result.json
│       └── trace.jsonl
├── summary.json
└── trace.jsonl

The traces contain the actual model requests, responses, token usage, validation outcomes, judge decisions, and human feedback produced during execution.

Citation

@inproceedings{chen2026bridging,
  title     = {Bridging Language and Physics: Automated Design of Continuum Robots with Large Language Models},
  author    = {Chen, Jingyi and Zhang, Mohan and Yao, Laura and Ni, Yingtai and Ji, Jianmin and Peng, Jie and Wang, Song and Chen, Tianlong},
  booktitle = {Robotics: Science and Systems},
  year      = {2026}
}

License

Released under the MIT License.