Separate regression hosts, reuse one Unreal scene, and limit docs builds - #216
Merged
Merged
Conversation
[fail] ProjectAirSim Layered CI
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
About
Separate Python regression tests by host so control, physics, clocks, and non-rendered sensors run on PAS Runtime before Unreal packaging. The final partition is 40 offline, 60 Runtime, and 83 Unreal cases, retaining all 180 original cases and adding three fixture contracts.
Unreal loads one composed scene for the whole suite. Clients attach without reloading; an RPC guard rejects a second LoadScene. Reset mutable world/vehicle/camera state between tests, use disposable material-test actors and a local texture HTTP server, and isolate generated data per case. Real voxel planning, images, and all five LiDAR variants remain covered. The hello-drone smoke moves to Runtime: its former image callback contained no assertions; actual image/pose checks remain in Unreal.
The combined workload uses 100k CPU LiDAR points/s, a 100 ms annotation camera interval, and a 640x480 streaming viewport. Camera benchmarks reuse DownCamera with its original capture interval. Existing throughput and object-detection assertions remain enabled; combined-scene performance is not directly comparable to the previous isolated workloads.
Also retain the earlier documentation change: Sphinx runs only on pushes to main affecting README.md, docs/, or client/python/projectairsim/docs/. Remove it from PR orchestration and remove all README workflow-status badges, including documentation. This workflow builds documentation; it does not deploy it.
Unity build opt-in
Normal builds omit the Unity native wrapper and Unity-specific ONNX/zlib staging. Add
--unityto build.cmd or build.sh to include them. CMake exposes PROJECTAIRSIM_BUILD_UNITY (OFF by default); build scripts explicitly reset it on every invocation. Unity README documents Windows/Linux/macOS commands, cache behavior, and the existing experimental status.Validated the real CMake gates with isolated artifacts on Windows and Linux: default OFF, ON, and reused-cache OFF; shared targets remain and disabled builds do not touch Unity outputs. Windows/Linux command dry runs cover flag placement and NMake assignments; macOS Make commands, Linux shell syntax/build-reuse checks, and all 45 workflow scenarios pass. No full SimLibs or Unity application build was performed for this build-selection change.
Runtime clock acceleration
Runtime steppable configurations now target 10x real time using real-time-update-rate = step-ns / 10. The physics step remains 3,000,000 ns; the wall-clock interval becomes 300,000 ns. Unreal clocks and source scene files are unchanged. The offline fixture contract verifies the step and source configuration are preserved.
A two-second clock probe on the existing local Debug Runtime measured 0.432 simulated seconds / 2.002 wall seconds with the old interval (0.22x), versus 8.199 / 2.002 with the new interval (4.10x effective). The configured target is 10x; achievable speed depends on host scheduling and compute capacity.
Local validation
Offline: 40 passed in 15.45 seconds after the clock change.
Runtime: the complete 60-case partition passed in one run in 188.62 seconds with the accelerated clock. This supersedes the earlier full-run/targeted-rerun evidence.
Full combined Unreal run: one observed LoadScene RPC; 81 passed. Two camera pub/sub benchmarks remain below the existing 15 FPS floor: RGB 12.19 FPS and depth 13.52 FPS. Request/reply RGB/depth passed at 6.06/7.28 FPS. CPU LiDAR passed at approximately 91,628 points/s; all five detection variants passed.
That full run also included hello-drone under its earlier Unreal classification and failed its flight assertion; it was subsequently moved to Runtime and validated there. It is not part of the final Unreal partition.
Runtime launcher: real process start/readiness/test/cleanup exercised, and occupied-port rejection preserved the existing Unreal process.
45 workflow scenarios/contracts passed; all workflow YAML parsed; actionlint on all 11 workflows, strict Python compilation, and git diff checks passed.
Simulator tests used existing local Windows Debug Runtime and UE 5.7.4 DebugGame Editor binaries with the changed Python suite, not a native rebuild of this PR head. Linux Shipping, remote Actions, and the Sphinx build were not run. The two performance failures remain visible and need validation on the intended Shipping runner before treating this as green regression CI.
Regression restructuring, Unity opt-in, and Runtime clock/docs-badge follow-ups are each isolated in their own commit alongside the original documentation change.