Video Generation (Wan 2.1 / 2.2)

Wan turns a prompt — and, on the Wan 2.2 checkpoints, an uploaded first frame — into an H.264 MP4. This page covers the downloads, the run recipes, and the settings that decide the wall clock.

Wan 2.1 / 2.2 (prompt → video, image → video)

Download the Turbo DiT, not the base one. The base Wan2.2-TI2V-5B follows the official 50-step × CFG recipe — 100 DiT passes per video. The step-distilled Turbo build of the same model is trained to run guidance-free in 4 steps, TensorSharp recognizes it from the file name, and the identical request comes back in minutes instead of hours. The Turbo repository ships only the DiT, so the VAE and the text encoder still come from the base repositories:

# The fast lane: 4 DiT passes instead of 100. Note the underscore in "Wan2_2".
hf download hum-ma/Wan2.2-TI2V-5B-Turbo-GGUF Wan2_2-TI2V-5B-Turbo-Q8_0.gguf --local-dir models
hf download QuantStack/Wan2.2-TI2V-5B-GGUF VAE/Wan2.2_VAE.safetensors --local-dir models
hf download city96/umt5-xxl-encoder-gguf umt5-xxl-encoder-Q8_0.gguf --local-dir models

# text → video
dotnet TensorSharp.Cli/bin/TensorSharp.Cli.dll --model models/Wan2_2-TI2V-5B-Turbo-Q8_0.gguf \
    --wan-te models/umt5-xxl-encoder-Q8_0.gguf --wan-vae models/VAE/Wan2.2_VAE.safetensors \
    --prompt "A red fox trotting through falling snow, cinematic" \
    --video-frames 81 --fps 24 --output out.mp4 --backend ggml_cuda
# image → video (the image becomes the first frame; Wan 2.2 models only)
dotnet TensorSharp.Cli/bin/TensorSharp.Cli.dll --model models/Wan2_2-TI2V-5B-Turbo-Q8_0.gguf \
    --wan-te models/umt5-xxl-encoder-Q8_0.gguf --wan-vae models/VAE/Wan2.2_VAE.safetensors \
    --image first_frame.png --prompt "the camera pushes in as the waves rise" \
    --output out.mp4 --backend ggml_cuda
dotnet TensorSharp.Server/bin/TensorSharp.Server.dll --model models/Wan2_2-TI2V-5B-Turbo-Q8_0.gguf \
    --wan-te models/umt5-xxl-encoder-Q8_0.gguf --wan-vae models/VAE/Wan2.2_VAE.safetensors \
    --backend ggml_cuda --video-frames 121 --fps 24

At load the console confirms the detection with step-distilled checkpoint detected -> 4 steps, guidance off (--diffusion-steps / --cfg override). Measured on an M5 Pro with ggml_metal, a 1088×832 × 121-frame (5 s, 720p-class) image-to-video request costs 100 DiT passes ≈ 3 h 30 m on the base checkpoint and 4 passes = 17 m 30 s on the Turbo one — the same flags, the same resolution, only the --model path differs. At 480p (736×544 × 121f) the Turbo checkpoint finishes in 6 m 19 s.

To reproduce the base recipe instead — for a reference sample, or on a Wan 2.1 checkpoint that has no distilled build — swap the first download and the --model paths:

hf download QuantStack/Wan2.2-TI2V-5B-GGUF Wan2.2-TI2V-5B-Q8_0.gguf --local-dir models

# 50 steps x 2 CFG passes; --cfg-cache-stride trades a little accuracy for 1.30x / 1.43x
dotnet TensorSharp.Cli/bin/TensorSharp.Cli.dll --model models/Wan2.2-TI2V-5B-Q8_0.gguf \
    --wan-te models/umt5-xxl-encoder-Q8_0.gguf --wan-vae models/VAE/Wan2.2_VAE.safetensors \
    --prompt "A red fox trotting through falling snow, cinematic" \
    --video-frames 81 --fps 24 --cfg-cache-stride 2 --output out.mp4 --backend ggml_cuda

For Wan 2.2 A14B image-to-video, download both distilled experts under one --local-dir — the loader finds the low-noise partner from the high-noise file name, so only one --model path is passed. A14B uses the Wan 2.1 VAE:

hf download jayn7/WAN2.2-I2V_A14B-DISTILL-LIGHTX2V-4STEP-GGUF \
    high_noise/wan2.2_i2v_A14b_high_noise_lightx2v_4step-Q4_K_M.gguf --local-dir models
hf download jayn7/WAN2.2-I2V_A14B-DISTILL-LIGHTX2V-4STEP-GGUF \
    low_noise/wan2.2_i2v_A14b_low_noise_lightx2v_4step-Q4_K_M.gguf --local-dir models
hf download QuantStack/Wan2.2-I2V-A14B-GGUF VAE/Wan2.1_VAE.safetensors --local-dir models

dotnet TensorSharp.Cli/bin/TensorSharp.Cli.dll \
    --model models/high_noise/wan2.2_i2v_A14b_high_noise_lightx2v_4step-Q4_K_M.gguf \
    --wan-te models/umt5-xxl-encoder-Q8_0.gguf --wan-vae models/VAE/Wan2.1_VAE.safetensors \
    --image first_frame.png --prompt "the camera pushes in as the waves rise" \
    --output out.mp4 --backend ggml_cuda

Video generation (Wan 2.1 / 2.2)

Wan turns a prompt — and, on the Wan 2.2 checkpoints, an uploaded first frame — into an H.264 MP4. The --model GGUF is only the DiT; TensorSharp resolves two companions next to it (a VAE/, HighNoise/ or LowNoise/ subfolder is fine):

• UMT5-XXL text encoder — prompt → conditioning (--wan-te / TS_WAN_TE). Released from VRAM before the denoise starts.
• Causal 3D video VAE — latent ↔ frames (--wan-vae / TS_WAN_VAE). Which one is required is read off the DiT itself, not chosen by you: Wan 2.1 and A14B use wan_2.1_vae; TI2V-5B uses Wan2.2_VAE.
• The A14B second expert — the two 14B experts must both be present. Put them in one directory, or in HighNoise/ + LowNoise/ subdirectories, and the loader pairs them from the file names (TS_WAN_DIT2 overrides; there is no CLI flag).

Each denoise step is one resident-weight ggml graph (CUDA-graph-captured, flash attention, per-token-timestep modulation for TI2V image-to-video); the video VAE encode and decode are each a single graph. The stages hand VRAM over in sequence — text encoder, then DiT, then VAE — so peak usage is roughly max(TE, DiT + attention, VAE): TI2V-5B generates 81-frame 480p image-to-video on a 16 GB GPU in under 8 minutes, and both A14B experts run sequentially on the same card. Numerics are verified against diffusers (DiT cosine > 0.995, VAE encoders > 0.999, decode 59.9 dB PSNR), and Wan 2.1 runs 6.0× faster end-to-end than stable-diffusion.cpp on the identical workload.

The checkpoint decides the wall clock

Wan's DiT token count is latent_frames × (h/2) × (w/2) and self-attention costs O(tokens²), so a 5-second 720p video is a genuinely large job: 1088×832 × 121 frames is 27 404 tokens, and the official TI2V-5B recipe spends 50 steps × 2 classifier-free-guidance passes = 100 DiT passes on it. A step-distilled checkpoint is trained to run guidance-free in 4 steps, so the same video costs 4 passes — 1/25th of the denoising work. TensorSharp detects one from the DiT file name (turbo, distill, lightning, lightx2v, fastwan, -dmd, or an explicit …-4steps-… for 1–16 steps), prints step-distilled checkpoint detected -> 4 steps, guidance off at load, and applies the recipe itself; --diffusion-steps and --cfg still override.

The two levers are independent: ~1.7× per pass from keeping the DiT's flash-attention keys and values in F16 (2.02× on a single 27 k-token self-attention, at cosine 0.999964 against the diffusers reference — TS_WAN_DIT_KV_F16=0 restores F32) and from running the Metal VAE convolutions through MPSGraph instead of ggml's im2col+GEMM lowering (VAE decode 159 s → 80 s at 736×544 × 81f, numerics unchanged at 93.9 dB PSNR — TS_WAN_VAE_MPS_CONV=0 reverts), and 25× fewer passes from the distilled checkpoint. Once distilled, the VAE decode is the bottleneck at ~55% of the run, not the DiT.

Frames and resolution set the rest

480p (≈0.4 MP) is a resolution Wan is trained at, so the first two rows are in-distribution rather than a degraded mode — that is the setting to reach for when a few minutes matters. Quality falls off below ~0.3 MP, where the DiT is out of distribution and the video turns soft, unstable and color-drifting however many steps you spend; the pipeline prints a warning when you cross that line. Generate at a supported resolution and downscale afterwards (--width 480 --height 704 rather than 320×480).

To make a large request cheaper, in order of effect: (1) use a step-distilled checkpoint — 100 passes become 4, dwarfing everything else; (2) fewer frames — 121 → 61 roughly quarters the attention work and halves the VAE decode; (3) a smaller frame area, but not below ~0.3 MP; (4) fewer steps, on base checkpoints only — 30 instead of 50 is visibly close and 1.7× cheaper; (5) --cfg-cache-stride 2 or 3, which runs the unconditional pass on one step in N and reuses the cached guidance direction between — 77 of the 100 passes (1.30×) or 70 (1.43×) at 50 steps. It is an approximation, so leave it off when matching a reference sample matters, and it does nothing on a distilled checkpoint, which is already guidance-free.

On NVIDIA, ggml_cuda is the fastest Wan backend: on an RTX 2000 Ada 16 GB running the official 480p Wan2.1-1.3B recipe it takes 12.0 s/step against ggml_vulkan's 17.2 and the direct cuda backend's 19.3. Wan does not run on mlx at all, and the cpu / ggml_cpu backends are for functional use only.

Drive it from the CLI (--prompt, optional --image, --video-frames, --fps, --flow-shift, --sampler, --negative-prompt), the HTTP API (/v1/videos/generations), or the Web UI chat with an image upload. On the server, --video-frames and --fps set defaults rather than caps — a request carrying its own frames or fps overrides them independently, and with both omitted the model's own recipe applies (49 frames at 24 fps for Wan2.2-TI2V, 33 at 16 fps otherwise). Frame counts snap to the VAE's temporal grid (4k+1). Full details are in the repository's docs/models/wan.md card.