Skip to content
Open
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
5 changes: 5 additions & 0 deletions LICENSE
Original file line number Diff line number Diff line change
Expand Up @@ -19,6 +19,11 @@ FlashDreams is open-source software. Its licensing posture is:
is licensed under the Zlib license. Full text:
LICENSES/Zlib.txt.

* MiniMax H3 codec and conditioning ports under
integrations_v2/minimax_h3/impl/ retain their upstream Apache-2.0
attribution. Full text: LICENSES/Apache-2.0.txt; source details
are recorded in THIRD-PARTY-NOTICES.

Each first-party source file carries an inline SPDX license
identifier; the REUSE 3.3 manifest at REUSE.toml fills the gaps for
files (configuration, assets, generated outputs) that cannot
Expand Down
4 changes: 4 additions & 0 deletions NOTICE
Original file line number Diff line number Diff line change
Expand Up @@ -15,6 +15,10 @@ LICENSES/:
cudaraster/framework/3rdparty/lodepng/{lodepng.h,lodepng.cpp}
Zlib (see LICENSES/Zlib.txt)

The MiniMax/HuggingFace H3 codec and conditioning ports under
integrations_v2/minimax_h3/impl/ retain their upstream Apache-2.0
attribution (see LICENSES/Apache-2.0.txt).

Third-party software attributions, source-level redistribution
disclosures, and the full per-dependency license inventory are
documented in THIRD-PARTY-NOTICES at the repository root.
15 changes: 15 additions & 0 deletions REUSE.toml
Original file line number Diff line number Diff line change
Expand Up @@ -66,6 +66,21 @@ SPDX-FileCopyrightText = [
]
SPDX-License-Identifier = "Apache-2.0"

# --- MiniMax/HuggingFace inference mathematics adapted to native APIs. ---
[[annotations]]
path = [
"integrations_v2/minimax_h3/impl/video_vae.py",
"integrations_v2/minimax_h3/impl/audio_encoder.py",
"integrations_v2/minimax_h3/impl/conditioning.py",
"integrations_v2/minimax_h3/impl/layout.py",
]
precedence = "aggregate"
SPDX-FileCopyrightText = [
"Copyright (c) 2026 NVIDIA CORPORATION & AFFILIATES. All rights reserved.",
"Copyright 2026 The MiniMax and HuggingFace Teams. All rights reserved.",
]
SPDX-License-Identifier = "Apache-2.0"

# --- Generated protobuf stubs. The .proto sources are authoritative; the
# *_pb2.py / *_pb2.pyi / *_pb2_grpc.py outputs inherit the same license
# because the regeneration script (compile_protos.sh) is project-owned. ---
Expand Down
27 changes: 25 additions & 2 deletions THIRD-PARTY-NOTICES
Original file line number Diff line number Diff line change
Expand Up @@ -147,6 +147,30 @@ The following third-party source is physically present in this repository
(not just consumed at runtime). Each file retains its original copyright
notice inline; license texts are reproduced under LICENSES/.

--------------------------------------------------------------------------------
MiniMax H3 native codecs and conditioning
--------------------------------------------------------------------------------

Path: integrations_v2/minimax_h3/impl/{video_vae,audio_encoder,conditioning,layout}.py
License: Apache-2.0 (see LICENSES/Apache-2.0.txt)
Copyright 2026 The MiniMax and HuggingFace Teams. All rights reserved.
Upstream: https://github.com/huggingface/diffusers/tree/175fe6b2419a01db9c2ceabd01ec37d2c0305fc2/src/diffusers

The inference mathematics are adapted from the MiniMax H3 implementation.
FlashDreams replaces pipeline blocks, model mixins, attention processors,
checkpoint loading, and runtime orchestration with its native APIs. The
audio port contains only reference encoding, not generated-audio decoding.
Diffusers is not a runtime dependency of this integration. Model weights
are not redistributed. Native checkpoint assets are pinned to
MiniMaxAI/MiniMax-H3 revision 42ed227ee7df40d41602854ae760620d6eb651fe.

The integration dynamically uses Transformers and Accelerate (Apache-2.0;
https://github.com/huggingface/transformers and
https://github.com/huggingface/accelerate), PyAV (BSD-3-Clause;
https://github.com/PyAV-Org/PyAV), and Pillow (MIT-CMU;
https://github.com/python-pillow/Pillow), alongside the shared runtime
dependencies listed above. These dependencies are not vendored.

--------------------------------------------------------------------------------
HPG-2011 NVIDIA CUDA Rasterizer port
--------------------------------------------------------------------------------
Expand Down Expand Up @@ -176,8 +200,7 @@ License: Zlib (see LICENSES/Zlib.txt)
Upstream: https://lodev.org/lodepng/

LodePNG is a PNG codec implementation embedded by the upstream
cudaraster framework. It is the only non-NVIDIA source physically
redistributed in this repository. The inline notice in lodepng.h
cudaraster framework. The inline notice in lodepng.h
preserves the upstream copyright notice as required by the Zlib
license; LICENSES/Zlib.txt reproduces the license text in full.

Expand Down
16 changes: 16 additions & 0 deletions apps/t2v/t2v/application.py
Original file line number Diff line number Diff line change
Expand Up @@ -180,6 +180,15 @@ def create_session(self, session_desc: SessionDesc) -> ISession:
if prompt is not None and (not isinstance(prompt, str) or not prompt.strip()):
raise ValueError("A session prompt must be non-empty text.")
self._validate_frame_size(session_desc, pipeline)
cache_kwargs = self._cache_initialization_kwargs(session_desc)
if cache_kwargs:
return self.session_type(
pipeline,
prompt,
session_desc,
config.total_blocks,
cache_init_kwargs=cache_kwargs,
)
return self.session_type(pipeline, prompt, session_desc, config.total_blocks)

def close(self) -> None:
Expand All @@ -193,6 +202,13 @@ def close(self) -> None:

## Integration hooks

def _cache_initialization_kwargs(self, session_desc: SessionDesc) -> dict[str, Any]:
"""Return model-specific request inputs retained across session resets.

Standard text, image, height, and width inputs remain framework-owned.
"""
return {}

def _configure_argument_parser(self, parser: argparse.ArgumentParser) -> None:
"""Add arguments this integration takes beyond the shared ones."""

Expand Down
22 changes: 21 additions & 1 deletion apps/t2v/t2v/session.py
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@

"""One text-to-video rollout: a prompt in, a chunk of frames per step out."""

from dataclasses import dataclass
from dataclasses import dataclass, field
from typing import Any

from flashdreams.api_v2.loop import IModelLoop
Expand All @@ -25,6 +25,9 @@ class T2VModelState:
image: Any = None
"""Optional first-frame tensor for image-conditioned generation."""

cache_init_kwargs: dict[str, Any] = field(default_factory=dict)
"""Integration-specific request inputs reused when resetting the cache."""

blocks_generated: int = 0
cache: Any = None

Expand Down Expand Up @@ -91,6 +94,7 @@ def __init__(
total_blocks: int,
*,
image: Any = None,
cache_init_kwargs: dict[str, Any] | None = None,
) -> None:
"""
Args:
Expand All @@ -101,12 +105,20 @@ def __init__(
against what the model can produce.
total_blocks: Blocks this rollout generates before it is finished.
image: Optional first-frame tensor retained across session resets.
cache_init_kwargs: Additional request inputs, excluding standard
``text``, ``image``, ``height``, and ``width`` arguments.
"""
self._pipeline = pipeline
self._prompt = prompt
self._session_desc = session_desc
self._total_blocks = total_blocks
self._image = image
self._cache_init_kwargs = dict(cache_init_kwargs or {})
reserved = {"text", "image", "height", "width"} & self._cache_init_kwargs.keys()
if reserved:
raise ValueError(
f"Cache inputs cannot override framework arguments: {sorted(reserved)}"
)

def init(self) -> None:
"""Encode the prompt and prepare the rollout's cache.
Expand All @@ -119,6 +131,7 @@ def init(self) -> None:
session_desc=self._session_desc,
total_blocks=self._total_blocks,
image=self._image,
cache_init_kwargs=dict(self._cache_init_kwargs),
)
if state.prompt is not None:
state.cache = _new_cache(state)
Expand All @@ -139,15 +152,22 @@ def _new_cache(state: T2VModelState) -> Any:
"""Encode the prompt into a cache for one rollout."""
if state.prompt is None:
raise RuntimeError("Cannot initialize a text-to-video cache without a prompt.")
reserved = {"text", "image", "height", "width"} & state.cache_init_kwargs.keys()
if reserved:
raise ValueError(
f"Cache inputs cannot override framework arguments: {sorted(reserved)}"
)
if state.image is not None:
return state.pipeline.initialize_cache(
text=[state.prompt],
image=state.image,
**state.cache_init_kwargs,
)
ratio = state.pipeline.decoder.spatial_compression_ratio
return state.pipeline.initialize_cache(
text=[state.prompt],
image=None,
height=state.session_desc.video_height // ratio,
width=state.session_desc.video_width // ratio,
**state.cache_init_kwargs,
)
27 changes: 27 additions & 0 deletions apps/t2v/tests/test_session.py
Original file line number Diff line number Diff line change
Expand Up @@ -22,6 +22,33 @@

pytestmark = pytest.mark.ci_cpu


def test_request_kwargs_survive_reset_and_cannot_override_standard_fields():
from t2v.session import T2VSession

pipeline = _stand_in()
session = T2VSession(
pipeline,
_PROMPT,
_session_desc(),
1,
cache_init_kwargs={"reference_paths": ("first", "last")},
)
session.init()
session.model_loop.reset()
assert len(pipeline.caches) == 2
assert pipeline.caches[0]["reference_paths"] == ("first", "last")
assert pipeline.caches[1]["reference_paths"] == ("first", "last")
with pytest.raises(ValueError, match="framework arguments"):
T2VSession(
pipeline,
_PROMPT,
_session_desc(),
1,
cache_init_kwargs={"text": ["override"]},
)


_WIDTH = 128
"""Frame width the stand-in generates. Not square, so a transposed frame cannot
pass unnoticed, and a whole number of latents across."""
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -271,7 +271,7 @@ def compute_kv(
def forward(
self,
x: Tensor,
kv_cache: KVCacheT,
kv_cache: KVCacheT | None = None,
rope_freqs: Tensor | None = None,
) -> Tensor:
"""Apply the configured attention type to ``x`` and ``kv_cache``.
Expand All @@ -281,7 +281,8 @@ def forward(

Args:
x: Query tokens, shape ``[..., L, query_dim]``.
kv_cache: Streaming cache for self-attention or precomputed static
kv_cache: ``None`` for cacheless bidirectional self-attention,
streaming cache for self-attention or precomputed static
cache for cross-attention. A streaming cache must already be in
its current-chunk update phase.
rope_freqs: Optional positional data. Before-cache RoPE expects the
Expand Down
Loading