Skip to content

Separate regression hosts, reuse one Unreal scene, and limit docs builds - #216

Merged
jonyMarino merged 5 commits into
mainfrom
codex/docs-main-push
Sep 17, 2026
Merged

jonyMarino merged 5 commits into
mainfrom
codex/docs-main-push

Conversation

@jonyMarino

@jonyMarino jonyMarino commented Sep 16, 2026

Copy link
Copy Markdown
Contributor

About

Separate Python regression tests by host so control, physics, clocks, and non-rendered sensors run on PAS Runtime before Unreal packaging. The final partition is 40 offline, 60 Runtime, and 83 Unreal cases, retaining all 180 original cases and adding three fixture contracts.

Unreal loads one composed scene for the whole suite. Clients attach without reloading; an RPC guard rejects a second LoadScene. Reset mutable world/vehicle/camera state between tests, use disposable material-test actors and a local texture HTTP server, and isolate generated data per case. Real voxel planning, images, and all five LiDAR variants remain covered. The hello-drone smoke moves to Runtime: its former image callback contained no assertions; actual image/pose checks remain in Unreal.

The combined workload uses 100k CPU LiDAR points/s, a 100 ms annotation camera interval, and a 640x480 streaming viewport. Camera benchmarks reuse DownCamera with its original capture interval. Existing throughput and object-detection assertions remain enabled; combined-scene performance is not directly comparable to the previous isolated workloads.

Also retain the earlier documentation change: Sphinx runs only on pushes to main affecting README.md, docs/, or client/python/projectairsim/docs/. Remove it from PR orchestration and remove all README workflow-status badges, including documentation. This workflow builds documentation; it does not deploy it.

Unity build opt-in

Normal builds omit the Unity native wrapper and Unity-specific ONNX/zlib staging. Add --unity to build.cmd or build.sh to include them. CMake exposes PROJECTAIRSIM_BUILD_UNITY (OFF by default); build scripts explicitly reset it on every invocation. Unity README documents Windows/Linux/macOS commands, cache behavior, and the existing experimental status.

Validated the real CMake gates with isolated artifacts on Windows and Linux: default OFF, ON, and reused-cache OFF; shared targets remain and disabled builds do not touch Unity outputs. Windows/Linux command dry runs cover flag placement and NMake assignments; macOS Make commands, Linux shell syntax/build-reuse checks, and all 45 workflow scenarios pass. No full SimLibs or Unity application build was performed for this build-selection change.

Runtime clock acceleration

Runtime steppable configurations now target 10x real time using real-time-update-rate = step-ns / 10. The physics step remains 3,000,000 ns; the wall-clock interval becomes 300,000 ns. Unreal clocks and source scene files are unchanged. The offline fixture contract verifies the step and source configuration are preserved.

A two-second clock probe on the existing local Debug Runtime measured 0.432 simulated seconds / 2.002 wall seconds with the old interval (0.22x), versus 8.199 / 2.002 with the new interval (4.10x effective). The configured target is 10x; achievable speed depends on host scheduling and compute capacity.

Local validation

  • Offline: 40 passed in 15.45 seconds after the clock change.

  • Runtime: the complete 60-case partition passed in one run in 188.62 seconds with the accelerated clock. This supersedes the earlier full-run/targeted-rerun evidence.

  • Full combined Unreal run: one observed LoadScene RPC; 81 passed. Two camera pub/sub benchmarks remain below the existing 15 FPS floor: RGB 12.19 FPS and depth 13.52 FPS. Request/reply RGB/depth passed at 6.06/7.28 FPS. CPU LiDAR passed at approximately 91,628 points/s; all five detection variants passed.

  • That full run also included hello-drone under its earlier Unreal classification and failed its flight assertion; it was subsequently moved to Runtime and validated there. It is not part of the final Unreal partition.

  • Runtime launcher: real process start/readiness/test/cleanup exercised, and occupied-port rejection preserved the existing Unreal process.

  • 45 workflow scenarios/contracts passed; all workflow YAML parsed; actionlint on all 11 workflows, strict Python compilation, and git diff checks passed.

Simulator tests used existing local Windows Debug Runtime and UE 5.7.4 DebugGame Editor binaries with the changed Python suite, not a native rebuild of this PR head. Linux Shipping, remote Actions, and the Sphinx build were not run. The two performance failures remain visible and need validation on the intended Shipping runner before treating this as green regression CI.

Regression restructuring, Unity opt-in, and Runtime clock/docs-badge follow-ups are each isolated in their own commit alongside the original documentation change.

@jonyMarino jonyMarino changed the title Run Sphinx on documentation pushes to main and remove PR build badges Separate regression hosts, reuse one Unreal scene, and limit docs builds Sep 16, 2026
@github-actions

Copy link
Copy Markdown

[fail] ProjectAirSim Layered CI

  • [fail] Final Linux regression: failure
  • [pass] Linux SimLibs: success
  • [pass] Config validation: success
  • [pass] Cheap Python checks: success
  • [pass] Windows SimLibs/C++: success
  • [pass] Change detection: success
  • [pass] macOS: success

View full workflow run

@jonyMarino
jonyMarino merged commit 643d723 into main Sep 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant