Preserve Video VAE decoder cache across input changes - #68
Open
Siilent-Sea wants to merge 1 commit into
Open
Siilent-Sea wants to merge 1 commit into
Siilent-Sea wants to merge 1 commit into
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem and scope
In the interactive session, the resident cache is fully cleared whenever the model
input changes.
h3_cli.ccallsh3_cache_clear()at five input-change sites:clear_references()—!refs clearadd_reference_image()—!ref-image, and the other reference addersremove_reference()—!ref-remove Nset_anchor()—!first/!last, both theclearbranch and the set branchh3_cache_clear()drops the prompt/reference conditioning, the prepared DiT andthe Video VAE decoder. The decoder, however, is keyed independently in
h3_generate()byvae_path|latentWxH. When that key is unchanged — which is thecommon case for an anchor or reference edit, since neither changes the decoder model
path or the spatial latent shape — the decoder is still valid and is now needlessly
released. The next generation reloads it from scratch.
The change keeps the full clear as the explicit/default behavior and adds one narrower
invalidation for input changes.
Why the change is general
session path and the shared
h3_ctxcache.(
h3_acquire_video_decoder()and the key comparison inh3_generate()); no newresidency policy or heuristic is introduced.
!cache clearcommand,h3_cache_set_enabled(ctx, 0)andh3_free(ctx)still perform full clears, so the user-visible explicit-clear contractis unchanged.
is only called from the five input-change sites.
Public reproducer
Host-only and deterministic; no model weights, no GPU, no Metal device. See
public-repro/test_cache_invalidate.c.Implementation
h3.c: addh3_cache_invalidate_inputs()— clears conditioning and the preparedDiT, leaves
video_decoder/video_decoder_keyuntouched.h3.h: declare it with a contract comment.h3_cli.c: call it at the five input-change sites; keeph3_cache_clear()for!cache clear.Regression test
tests/test_cache_invalidate.c, 15 asserts:conditioning_present)and DiT (
dit,dit_key) but keeps both the residentvideo_decoderpointerand
video_decoder_keyunchanged;h3_cache_clear()still drops the decoder and both keys;Numerical validation
Not applicable. The change alters cache lifetime, not arithmetic; no tensor path is
touched.
Performance and memory validation
None claimed. The intended effect is to avoid a decoder reload across an input change.
No timing measurement accompanies this draft, and none is asserted.
Platforms tested / not tested
make -j8 libh3.a h3builds with no warnings under the project's-Wall -Wextra -Wpedantic -Wshadow -Wconversion;h3_testsreports 1768 checks OK;the new host test passes.
Risks and fallback
decoder resident. That is intentional and matches the decoder's own key contract;
!cache clearstill offers a hard reset.h3_cli.chunk; the new library function remains additive andharmless.
Privacy and asset provenance
No private media, prompts, latents, hashes or paths. The patch is original project code
under the repository's existing MIT license; it adds no third-party code.
Related issue or discussion
No existing Issue or PR found at 2026-09-16 for input-only cache invalidation
(
repo:antirez/h3.csearches for "invalidation" and "cache invalidate" return nomatch). PR #62 / Issue #25 concern non-TTY stdout flushing; PR #1 concerns VAE tile
size; PR #35 concerns decode progress. None owns this behavior.
Contribution status
No
.github/contribution policy, CLA or DCO requirement detected onantirez/h3.c. The account holder submits; no separate sign-off action is known to berequired. Recheck immediately before submission.
AI assistance disclosure
Submitted by the project owner with assistance from an owner-authorized AI engineering
agent. Reproduction, code and tests are included.