Program is run using server.cmd, so without --keep-loaded arg.
Windows 10, CUDA 12.4.1 toolkit, compiled from latest master using buildcuda.cmd (had to change hardcoded path to C:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat)
Expected behavior : non-fatal errors should not end the server process
But also: VRAM should be sufficient
Is this error because VRAM is not being freed properly? Feedback appreciated.
Here is the session log leading to the error:
[Server] Scanning models in .\models
[Registry] Qwen3-Embedding-0.6B-BF16.gguf -> Text-Enc
[Registry] acestep-5Hz-lm-4B-Q6_K.gguf -> LM
[Registry] acestep-v15-xl-sftturbo50-Q6_K.gguf -> DiT
[Registry] vae-BF16.gguf -> VAE
[Server] Scanning adapters in .\adapters
[Store] Created (policy=STRICT)
[Server] acestep.cpp fa33775 (2026-07-21)
[Server] Listening on 0.0.0.0:8085
[Server] Pipelines: /lm /synth /understand
[Server] Models: 1 LM, 1 Text-Enc, 1 DiT, 1 VAE, 0 Adapter
[Server] Job 2d2d6b8e2e85ad66 created (1 requests)
[Server] Loading synth: DiT=acestep-v15-xl-sftturbo50-Q6_K.gguf VAE=vae-BF16.gguf
[GGUF] .\models\acestep-v15-xl-sftturbo50-Q6_K.gguf: 830 tensors, data at offset 69120
[GGUF] .\models\acestep-v15-xl-sftturbo50-Q6_K.gguf: 830 tensors, data at offset 69120
[Synth-Load] Ready: turbo=yes, fa=yes, batch_cfg=yes
[Resolve-T] T=750, S=375
[Resolve-T] seed=4269250351, steps=8, guidance=1.0, shift=3.0, duration=30.0s
[BPE] Loaded from GGUF: 151643 vocab, 151387 merges
ggml_cuda_init: found 1 CUDA devices (Total VRAM: 6143 MiB):
Device 0: NVIDIA GeForce RTX 2060, compute capability 7.5, VMM: yes, VRAM: 6143 MiB
[Load] TextEncoder backend: CUDA0 (CPU threads: 4)
[GGUF] .\models\Qwen3-Embedding-0.6B-BF16.gguf: 310 tensors, data at offset 5337568
[Load] TextEncoder: 28L, H=1024, Nh=16/8
[Qwen3] Attn: Q+K+V fused
[Qwen3] MLP: gate+up fused
[WeightCtx] Loaded 310 tensors, 1136.5 MB into backend
[Store] Load TextEnc: 24422 ms
[Store] Unload TextEnc (1136.5 MB)
[Load] CondEncoder backend: CUDA0 (CPU threads: 4)
[GGUF] .\models\acestep-v15-xl-sftturbo50-Q6_K.gguf: 830 tensors, data at offset 69120
[Load] LyricEncoder: 8L
[Qwen3] Attn: Q+K+V fused
[Qwen3] MLP: gate+up fused
[Load] TimbreEncoder: 4L
[Qwen3] Attn: Q+K+V fused
[Qwen3] MLP: gate+up fused
[WeightCtx] Loaded 141 tensors, 476.3 MB into backend
[Load] CondEncoder: lyric(8L), timbre(4L, CLS), text_proj, null_cond
[Store] Load CondEnc: 27224 ms
[CondEnc] Lyric sliding mask: 11x11, window=128
[CondEnc] Timbre sliding mask: 2x2, window=128 (CLS)
[Encode] Packed: lyric=11 + timbre=1 + text=74 = 86 tokens
[Encode-Text Batch0] 74+11 tokens -> enc_S=86, 157363.4 ms
[Store] Unload CondEnc (476.3 MB)
[Init-Noise Batch0] Philox noise seed=4269250351, [750, 64] solver=euler
[Init-Noise] Starting: T=750, S=375, enc_S=86, steps=8, batch=1
[Load] DiT backend: CUDA0 (CPU threads: 4)
[GGUF] .\models\acestep-v15-xl-sftturbo50-Q6_K.gguf: 830 tensors, data at offset 69120
[DiT] Self-attn: Q+K+V fused
[DiT] Cross-attn: Q+K+V fused
[DiT] MLP: gate+up fused
[Load] null_condition_emb found (CFG available)
[WeightCtx] Loaded 630 tensors, 3267.8 MB into backend
[Load] DiT: 32 layers, H=2560, Nh=32/8, D=128
[Store] Load DiT: 71265 ms
[DiT] Batch N=1, T=750, S=375, enc_S=86
[DiT] Graph: 2569 nodes
[DiT] Solver: ODE Euler (1 NFE/step, order 1)
[DiT] Step 1/8 t=1.000
ggml_backend_cuda_graph_compute: CUDA graph warmup complete
[DiT] Step 2/8 t=0.955
[DiT] Step 3/8 t=0.900
[DiT] Step 4/8 t=0.833
[DiT] Step 5/8 t=0.750
[DiT] Step 6/8 t=0.643
[DiT] Step 7/8 t=0.500
[DiT] Step 8/8 t=0.300
[DiT-Generate] Total: 2803.0 ms (2803.0 ms/sample)
[Store] Unload DiT (3267.8 MB)
[GGUF] .\models\vae-BF16.gguf: 365 tensors, data at offset 30048
[Load] VAE backend: CUDA0 (CPU threads: 4)
[VAE] Backend: CUDA0, Weight buffer: 161.1 MB
[VAE] Loaded: 5 blocks, upsample=1920x, F32 activations
[Store] Load VAE-Dec: 4027 ms
[VAE] Graph: 479 nodes, T_latent=750
CUDA error: out of memory
Program is run using
server.cmd, so without--keep-loadedarg.Windows 10, CUDA 12.4.1 toolkit, compiled from latest master using
buildcuda.cmd(had to change hardcoded path toC:\Program Files\Microsoft Visual Studio\2022\Community\VC\Auxiliary\Build\vcvars64.bat)Expected behavior : non-fatal errors should not end the server process
But also: VRAM should be sufficient
Is this error because VRAM is not being freed properly? Feedback appreciated.
Here is the session log leading to the error: