Add CUDA device-pointer plane_codec ABI with explicit selection - #2
Merged
Conversation
…d Python/CLI surface Adds a device-pointer plane_codec ABI (encode and decode) so a GPU-resident codec can be supplied by a third-party plugin, reachable by explicit codec selection from Rust, Python, and the CLI. - Two new CUDA C-ABI symbols mirror the existing hardware_backend_v1_* pair, attached to the plane_codec kind; encode reports its required buffer size explicitly rather than through decode_rc's lossy -2 return. - DispatchedPlaneCodecCuda trait and native adapter, resolved through a capability-gated PlaneCodecCudaRouter (native flavor only, mirrors HardwareBackendRouter's cache and hardware_class gating). - Third-party encode, previously refused because the trial-encode loop cannot consume its flat ABI return shape, becomes reachable under explicit selection. - resolve_codec_selector turns a name or canonical id into a CanonicalId; an ambiguous or unknown selector is a loud error naming every candidate. - A Rust-only integration test pins that the decode path needs no Python bindings. - A PyO3 bridge exposes plane_codec_encode_cuda/decode_cuda, sharing DLPack parsing, buffer-protocol fallback, and device-ordinal checking with the existing hardware-backend bridge. - CompressionConfig/DecompressionConfig gain codec/device fields; Compressor gains a device-mismatch guard. CLI --codec/--device flags are wired through every config-construction site each command reaches. - Setting codec or device on a config with no dispatch path to consume it raises immediately, naming the field, instead of silently doing the wrong thing.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds a device-pointer
plane_codecABI (encode and decode) so aGPU-resident codec can operate on device memory directly, supplied
entirely by a third-party plugin. The new symbol pair mirrors the
existing
hardware_backend_v1_*CUDA ABI but attaches to theplane_codeckind, so it participates in container framing the sameway a built-in codec does.
Why
The existing
plane_codecdispatch path assumes a contribution'sdata already lives in host memory. A GPU-resident codec needs its
compressed and decompressed buffers to stay on-device: routing every
plane through a host round trip would erase most of the reason to use
a GPU codec at all. This adds the missing piece, a symbol pair that
takes device pointers directly, plus the plumbing to reach it from
Rust, Python, and the CLI.
What changed
ABI and dispatch (Rust core)
plane_codeckind. Encode reports its required output sizeexplicitly, since the existing
decode_rcerror code cannotround-trip an arbitrary byte count.
DispatchedPlaneCodecCudatrait and native adapter bridge theABI, resolved through a capability-gated router mirroring the
existing hardware-backend router's caching and
hardware_classgating.
selected by name or id. It stays unreachable through the automatic
trial-encode search, whose return shape the flat third-party ABI
cannot express.
resolve_codec_selector) turns a human-typedselector into a canonical id, erroring loudly and naming every
candidate on an ambiguous or unknown name, rather than picking one
silently.
never needs the Python bindings.
Python bridge and surface
DLPack tensor parsing, a buffer-protocol fallback, and
device-ordinal agreement checking with the existing hardware-backend
bridge rather than duplicating them.
CompressionConfig/DecompressionConfiggaincodec/devicefields; compression gains a guard against a mismatch between an
explicit device and an input tensor's own device.
--codec/--deviceflags, wired through everyconfig-construction path each command reaches.
codecordeviceon a config with no dispatch path toconsume it raises immediately, naming the field, instead of
silently falling back to the automatic codec search or CPU-only
decode.
Testing
cargo test --workspace: all passing, clippy clean.pytestacross the full suite (maturin developbuild): allpassing; CUDA-dependent paths skip without a GPU.