BenchScope

BenchScope is an experimental macOS desktop front end for reproducible GPU benchmarking, RenderDoc capture/replay, Mesa llvmpipe counter collection, RDC slicing, and offline performance analysis.

Warning

BenchScope is research software. Its build patches, artifact formats, and command-line interfaces may change. It is currently supported on macOS 13 or later only.

BenchScope does not ship prebuilt graphics runtimes, benchmark assets, RDC files, or PvrGPU. Build scripts fetch supported open-source dependencies at fixed revisions and apply the patches stored in this repository.

Modes and options

The top-level Mode menu has exactly four entries. The available Option values depend on the selected Mode:

Mode	Options
Direct Mode	Benchmark + Mesa/llvmpipe; Benchmark + Mesa/PvrGPU
Capture Mode	Benchmark + RenderDoc/Capture + Mesa/llvmpipe
Play Mode	RenderDoc/replay + Mesa/llvmpipe; RenderDoc/replay + Mesa/PvrGPU
Slice Mode	Slice + RenderDoc/replay + Mesa/llvmpipe

Play Mode is the default. BenchScope therefore opens on the Trace / RDC controls and keeps Play Mode selected when moving between Benchmark and Output artifacts.

Quick start

Install the Xcode Command Line Tools and Python 3, then clone and launch:

git clone https://github.com/cclwylin/BenchScope.git
cd BenchScope
./scripts/setup-benchmark-ui.sh
./scripts/run-benchscope.sh

The setup script creates an isolated Python environment, installs PySide6>=6.8,<7, and creates a Finder-launchable BenchScope.app beneath the configured work root. It does not build Mesa, RenderDoc, or the benchmarks.

For the native build workflows, install the common build tools first:

brew install python@3.12 llvm bison cmake meson ninja pkg-config \
  molten-vk libx11 libxcb glslang spirv-tools zstd libpng

The Build page runs the same scripts available from the command line. A typical llvmpipe + RenderDoc setup is:

./scripts/build-mesa-counter.sh
./scripts/build-renderdoc-mesa.sh

Build only the benchmark families you plan to use:

./scripts/build-macos-mesa-sw.sh   # external GFXBench checkout
./scripts/build-glbench-mesa.sh
./scripts/build-glmark2-mesa.sh
./scripts/build-deqp-mesa.sh

See the UI and workflow guide for runner behavior, artifacts, counter analysis, and troubleshooting.

External sources and pinned revisions

Upstream source trees are intentionally not vendored. The build scripts verify the following revisions before building:

Dependency	Revision used by BenchScope	Local integration
GFXBench	89aa956f43836798d1eaaab8ec32d3dc8b02eead	patches/gfxbench/gfxbench-89aa956f-macos-mesa.patch
Mesa	mesa-26.2.1, commit da14d65e4499e66468094be52bff9ea0915a695e	llvmpipe counter and shader-artifact patches in patches/mesa/
RenderDoc	v1.45, commit 2fc0bc04cb95499635f63986a55bc6f67849dd9f	Apple EGL/Mesa and RDC-split patches in patches/renderdoc/
VK-GL-CTS	opengl-es-cts-3.2.14.1, commit 067e8832315e79817ede1c4863804e440f5d1c80	null-window-system GLES/EGL build
GLBench	commit e99bc684272bffd68b06c998e272531c9c84330f	small Apple/Mesa adapter in tools/glbench-mesa/
glmark2	commit 22c527cb0556f3a1ac4445aaa52cc532760928d5	patches/glmark2/glmark2-22c527c-apple-mesa-capture.patch

GFXBench is external

GFXBench source code and assets are not included. Running scripts/build-macos-mesa-sw.sh prepares a separate checkout at the pinned commit and applies the repository patch exactly. The checkout URL, source cache, and source root can be changed with GFXBENCH_SOURCE_URL, GFXBENCH_SOURCE_CACHE_DIR, and GFXBENCH_SOURCE_ROOT.

Review and comply with GFXBench's code and asset licenses before downloading, using, or redistributing it. In particular, asset attribution requirements are separate from BenchScope's own license.

PvrGPU is optional and external

The two Mesa/PvrGPU options require a separately obtained and built PvrGPU project. BenchScope does not clone, pin, vendor, license, or redistribute PvrGPU or its SystemC runtime. By default the adapters look for a sibling PvrGPU directory; set PVRGPU_PROJECT_ROOT when it lives elsewhere.

The PvrGPU project can publish its runner locations through config/local.env or these variables:

• PVRGPU_WORK_ROOT and PVRGPU_BUILD_DIR
• PVRGPU_DEQP_RUNNER
• PVRGPU_RDC_PVRGPU_RUNNER
• PVRGPU_SYSTEMC_API_LIB

Configuration

BenchScope keeps generated files outside the checkout by default. The work root is resolved in this order:

1. BENCHSCOPE_WORK_ROOT
2. BENCHMARK_WORK_ROOT (compatibility)
3. MANHATTAN_OUTPUT_ROOT (legacy compatibility)
4. $XDG_CACHE_HOME/benchscope
5. $HOME/.cache/benchscope

Frequently useful overrides are:

Variable	Purpose
BENCHMARK_UI_ROOT	Qt virtual environment and app-bundle root
BENCHMARK_UI_VENV / BENCHMARK_UI_PYTHON	Use a specific UI environment or Python interpreter
BENCHMARK_UI_BOOTSTRAP_PYTHON	Python used by the setup script
MESA_COUNTER_ROOT / MESA_PREFIX	Mesa source/build root or installed prefix
RENDERDOC_MESA_ROOT	RenderDoc build and adapter root
GLBENCH_LOCAL_ROOT / GLMARK2_LOCAL_ROOT / DEQP_LOCAL_ROOT	Per-family source/build roots
GFXBENCH_LOCAL_ROOT	GFXBench build, install, and package root
GFXBENCH_SOURCE_URL / GFXBENCH_SOURCE_CACHE_DIR / GFXBENCH_SOURCE_ROOT	External GFXBench checkout settings
LLVM_PREFIX / OSX_ARCHS	Toolchain prefix and explicit native build architecture
PVRGPU_PROJECT_ROOT and PVRGPU_*	Optional PvrGPU project, runners, and bridge

Build scripts expose additional expert overrides near the top of each file. Paths used by native build tools should not contain spaces unless the script explicitly supports an alias.

Current limitations

• Linux and Windows are not supported; native builds target macOS 13+.
• CI validates the portable Python and shell control plane. It does not build Mesa/RenderDoc, download benchmark assets, or execute GPU workloads.
• Capture Mode uses llvmpipe only. dEQP capture is not currently exposed.
• Slice Mode uses llvmpipe only.
• Direct/PvrGPU accepts one exact dEQP GLES2 or EGL case per run. Wildcards, lists, GLES3/GLES31, and other benchmark families fail closed.
• Play/PvrGPU requires a compatible PvrGPU native RDC runner. dEQP replay also requires reliable case and surface metadata.
• The 17-counter analyzer parses Mesa reports, not PvrGPU's native JSON/JSONL counter format.
• The performance model is an analytical throughput ceiling, not measured GPU time or a cycle-accurate simulator.

Tests

The same checks used by CI can be run from a clean checkout:

for test_file in scripts/tests/test-*.py; do
  PYTHONDONTWRITEBYTECODE=1 python3 "$test_file"
done

for test_file in scripts/tests/test-*.sh; do
  /bin/bash "$test_file"
done

find scripts -type f -name '*.sh' -exec /bin/bash -n {} +

BENCHMARK_UI_PYTHON="$(python3 -c 'import sys; print(sys.executable)')" \
QT_QPA_PLATFORM=offscreen BENCHMARK_UI_SMOKE_TEST=1 \
  ./scripts/run-benchscope.sh

Install requirements-ui.txt before the Qt smoke test. Large native-build and replay integration tests remain manual because they require external source, toolchains, traces, and significant build time.

Licensing and project identity

BenchScope's original code and documentation are licensed under the BSD 3-Clause License. Third-party-derived patch files and external dependencies remain subject to their applicable upstream terms; see THIRD_PARTY_NOTICES.md and LICENSES/.

BenchScope is an independent project. It is not affiliated with, sponsored by, or endorsed by Kishonti, Mesa, RenderDoc, Khronos Group, ChromiumOS, glmark2, Qt, or the maintainers of PvrGPU. Product and project names belong to their respective owners.