Skip to content

fix: improve quantization and runtime correctness - #3055

Merged
Qubitium merged 2 commits into
mainfrom
fix/quantization-runtime-correctness
Sep 4, 2026
Merged

fix: improve quantization and runtime correctness#3055
Qubitium merged 2 commits into
mainfrom
fix/quantization-runtime-correctness

Conversation

@ZX-ModelCloud

Copy link
Copy Markdown
Collaborator

Summary:

  • Release Hessian partials.
  • Safe sharded-load capability defaults.
  • Instance loader/lm_head.
  • ERNIE projection paths.
  • GiB formatting.

Tests:

  • Targeted: 92 passed, 35 skipped.
  • Module-tree: 106 passed, 2 deselected.

@ModelCloud ModelCloud deleted a comment from Qubitium Sep 4, 2026
@ZX-ModelCloud

Copy link
Copy Markdown
Collaborator Author

Reviewed commit 39971cd3bd214fc60f7a4a87a72eec6baf35589f.

Result: no blocking findings.

The changes are internally consistent:

  • The fallback path resolves its compute device before releasing per-device Hessian partials, preserves the prior fallback arithmetic/results, and free() now drops lingering partial references.
  • The nested getattr in kernel selection avoids eager evaluation of a missing legacy capability attribute while preserving SUPPORTS_SHARDED_LOAD precedence.
  • get_model_with_quantize() now correctly honors definition-subclass loader and lm_head overrides through the instance.
  • The ERNIE expert projection correction matches the surrounding gate_proj/up_proj/down_proj layout.
  • math.prod keeps the column count as a plain Python int.

Validation performed in an isolated checkout:

  • Changed/related test files: 60 passed, 31 skipped
  • tests/test_looper_concurrency.py: 19 passed
  • Full tests/module_tree: 106 passed, plus 2 failures that reproduce unchanged on origin/main (one existing None vs [] assertion and one local CUDA-initialization/environment failure)
  • git diff --check origin/main...HEAD: passed
  • Current GitHub CodeQL and Ruff checks: green

Non-blocking cleanup nit: in get_max_memory(), the two bare expressions reading active_bytes.all.peak have discarded results. Since peak values are not used, those expressions can be deleted rather than retained as no-ops.

@Qubitium
Qubitium merged commit 3912254 into main Sep 4, 2026
6 checks passed
@Qubitium
Qubitium deleted the fix/quantization-runtime-correctness branch September 4, 2026 09:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants