Skip to content

Preserve Video VAE decoder cache across input changes - #68

Open
Siilent-Sea wants to merge 1 commit into
antirez:mainfrom
Siilent-Sea:codex/input-cache-invalidation
Open

Siilent-Sea wants to merge 1 commit into
antirez:mainfrom
Siilent-Sea:codex/input-cache-invalidation

Conversation

@Siilent-Sea

Copy link
Copy Markdown

Problem and scope

In the interactive session, the resident cache is fully cleared whenever the model
input changes. h3_cli.c calls h3_cache_clear() at five input-change sites:

  • clear_references()!refs clear
  • add_reference_image()!ref-image, and the other reference adders
  • remove_reference()!ref-remove N
  • set_anchor()!first / !last, both the clear branch and the set branch

h3_cache_clear() drops the prompt/reference conditioning, the prepared DiT and
the Video VAE decoder. The decoder, however, is keyed independently in
h3_generate() by vae_path|latentWxH. When that key is unchanged — which is the
common case for an anchor or reference edit, since neither changes the decoder model
path or the spatial latent shape — the decoder is still valid and is now needlessly
released. The next generation reloads it from scratch.

The change keeps the full clear as the explicit/default behavior and adds one narrower
invalidation for input changes.

Why the change is general

  • It is not Mac-, shape- or workflow-specific: the affected code is the shared CLI
    session path and the shared h3_ctx cache.
  • It uses the decoder-key mechanism that already exists upstream
    (h3_acquire_video_decoder() and the key comparison in h3_generate()); no new
    residency policy or heuristic is introduced.
  • The existing !cache clear command, h3_cache_set_enabled(ctx, 0) and
    h3_free(ctx) still perform full clears, so the user-visible explicit-clear contract
    is unchanged.
  • Default behavior for one-shot callers is unchanged: the new function is additive and
    is only called from the five input-change sites.

Public reproducer

Host-only and deterministic; no model weights, no GPU, no Metal device. See
public-repro/test_cache_invalidate.c.

Implementation

  • h3.c: add h3_cache_invalidate_inputs() — clears conditioning and the prepared
    DiT, leaves video_decoder / video_decoder_key untouched.
  • h3.h: declare it with a contract comment.
  • h3_cli.c: call it at the five input-change sites; keep h3_cache_clear() for
    !cache clear.

Regression test

tests/test_cache_invalidate.c, 15 asserts:

  • input-only invalidation clears conditioning (key, values, conditioning_present)
    and DiT (dit, dit_key) but keeps both the resident video_decoder pointer
    and video_decoder_key unchanged;
  • h3_cache_clear() still drops the decoder and both keys;
  • NULL-argument safety for both entry points.

Numerical validation

Not applicable. The change alters cache lifetime, not arithmetic; no tensor path is
touched.

Performance and memory validation

None claimed. The intended effect is to avoid a decoder reload across an input change.
No timing measurement accompanies this draft, and none is asserted.

Platforms tested / not tested

  • Tested: macOS 26.5.1 (build 25F80), Apple clang 21.0.0, arm64.
    make -j8 libh3.a h3 builds with no warnings under the project's
    -Wall -Wextra -Wpedantic -Wshadow -Wconversion; h3_tests reports 1768 checks OK;
    the new host test passes.
  • Not tested: GPU/generation paths, non-Apple platforms, any model run.

Risks and fallback

  • Risk: a caller that relied on an input change also resetting the decoder keeps the
    decoder resident. That is intentional and matches the decoder's own key contract;
    !cache clear still offers a hard reset.
  • Fallback: drop the h3_cli.c hunk; the new library function remains additive and
    harmless.

Privacy and asset provenance

No private media, prompts, latents, hashes or paths. The patch is original project code
under the repository's existing MIT license; it adds no third-party code.

Related issue or discussion

No existing Issue or PR found at 2026-09-16 for input-only cache invalidation
(repo:antirez/h3.c searches for "invalidation" and "cache invalidate" return no
match). PR #62 / Issue #25 concern non-TTY stdout flushing; PR #1 concerns VAE tile
size; PR #35 concerns decode progress. None owns this behavior.

Contribution status

No .github/ contribution policy, CLA or DCO requirement detected on
antirez/h3.c. The account holder submits; no separate sign-off action is known to be
required. Recheck immediately before submission.

AI assistance disclosure

Submitted by the project owner with assistance from an owner-authorized AI engineering
agent. Reproduction, code and tests are included.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant