SOFIE_quantization Full Library v0

[shaunlee8]

An end-to-end quantized inference pipeline for SOFIE code generation. Support for ONNX quantized graphs, to self-contained generated headers, to low-precision execution. Structured as an installable codegen pass (SOFIE_quantization) over the existing core.

Frontend:

• ONNX QDQ: QuantizeLinear / DequantizeLinear boundary pairs
• QONNX: Quant custom ops

Data Types:

• INT8/UINT8 affine: symmetric and asymmetric, per-tensor and per-channel weights, INT32 accumulation
• FP8 E4M3/E5M2 with scale, FP16 carriers for movement

Operators:

• GEMM/MatMul: fused inter-layer requantization and activations
• Convolution: im2col, depthwise, per-channel weights
• Gather, Elementwise, movement ops (Transpose/Reshape/...)

Execution:

• cuBLASLt INT8 and FP8 paths with fused epilogues
• Region detection: boundary recognition, expanded region formation through value-preserving ops, node elimination rule ladder

Pipeline:

• Per-family passes (RQuantization_*Pass.cxx), RQuantization_*.cxx find regions where everything between a dequantize and a quantize can run directly on the codes (raw stored numbers, int or fp bit patterns). Regions grow through value-preserving ops, absorb activations, and detect boundaries between neighboring regions so chained layers requantize in place. Node elimination rules target leftover quantized ops.
• Each region becomes a lowering plan (RQuantization_Lowering.hxx) where operand quantization metadata lives in QuantizedOperandContract (INT8, FP8, etc.) and QuantizationGrid which maps codes to values.