Supported Models

Choose the output you need, then check inputs, model files, backend and validation scope. Receiving images, video or audio and generating those media are separate capabilities; the table keeps them apart.

First run → · Compare families → · Read support status →

Browse by task

Use these five existing reference pages: choose a task while comparing models, or downloads when you already know the model. Each family's files, memory requirements and launch commands remain in its reference.

🔎

Embedding Models & Semantic Search

Snowflake, MiniLM and EmbeddingGemma 2: text and multimodal vectors, APIs, and retrieval quality.

⬇️

Model Downloads

Every GGUF, with the projector, VAE, encoder and drafter files each family needs.

🧠

Text & LLM Models

Supported model families and their run recipes, plus multimodal input, thinking and tool calling.

🖼️

Image Generation

Qwen-Image-2.1 text-to-image and image editing, its defaults and draft settings.

🎬

Video Generation

LTX 2.5 and MiniMax-H3 generate joint audio/video; Wan 2.1 / 2.2 generates video.

Reading support status

Implemented means a code path exists. Conditional requires a named variant, backend or companion. Experimental marks explicit implementation or validation limits. Fixture-only covers small synthetic/component checks. Measured refers to the recorded checkpoint, hardware, workload and build. Unsupported means refused or unimplemented. Read input capability and validation status separately.

Choose quantization and memory settings from the model card. A quantization name, component gate or successful load does not establish full-model quality on every device.

Compare family inputs, outputs and conditions

Family / cardInputsOutput / taskConditions and evidence
Gemma 4Text, image, video, audioTextConditional: matching projector for media. Start with the E4B Q8_0 native-GGML lane.
Qwen 3.5 / 3.6 / 3.8 27BText, imageTextConditional: NextN needs a GGUF retaining its draft block; check variant and backend.
Qwen 3.8 Flash NextText, image, videoTextConditional: memory placement, tensor parallelism and the separate MTP head depend on quantization/backend.
Bonsai2 27BText, image with projectorTextConditional: single-device GGML only. Recorded Metal and bounded CPU checks do not qualify other devices.
GPT OSSTextTextImplemented: thinking mode and tools; check the card for backend and cache requirements.
Nemotron-H / OmniText; image on Omni; conditional audioTextConditional: audio needs a Parakeet companion absent from public GGUFs. Speculation is unsupported.
Mistral 3Text, imageTextUnsupported: explicit thinking mode and tools. Check the named checkpoint/backend coverage.
Hunyuan DenseTextTextImplemented on the shared decoder runtime: --tp, --layer-split and their hybrid with batched serving; tools are unsupported.
GLM 5.xText; image only on 5.3-FlashTextConditional: 5.2/5.3 are text-only. NextN excludes active tensor parallelism; the CPU reference has numerical limits.
DeepSeek V4 FlashTextTextConditional: check memory and placement; optional DSpark speculation requires a matching drafter.
DeepSeek V4.1 Flash / 1017Text; conditional image/videoTextExperimental: routed-MoE TP/DSpark. Pure-C# CPU coverage is fixture-only; default 1017 YOCO prefill is approximate.
Muse SparkText, image with Spark projectorTextConditional: multi-GPU or large host memory. Recorded CUDA measurements do not qualify every backend; audio is unsupported.
MiniMax-M3.1-FlashText, image with a converted projectorTextConditional: ggml_cuda and several GPUs (232 GiB UD-Q4_K_XL); the projector and DSpark drafter must be converted. Recorded 8x A40 measurements do not qualify other backends; audio is unsupported.
Muse-GlimmerText, imageTextConditional: projector for images, ATEM for tools, separate DFlash drafter for optional speculation; check backend limits.
DiffusionGemmaText, imageTextUnsupported: chat audio, video_url and tools. Web UI video is supplied as sampled images.
Jev (DiffusionGemma)Text, images, extracted documents/frames/transcriptsTyped decisionsConditional: sampled video and configured ASR transcripts; see ingestion and probability limits.
Qwen-Image-2.1Text, optional reference/mask imagesImages / image editsConditional: matching encoder/VAE and sampling recipe; component coverage varies by backend.
LTX 2.5 DistilledText, image keyframes; optional identity sheetsVideo + 48 kHz stereoExperimental: current two-stage recipe and companions; Vulkan and multi-GPU are unsupported.
MiniMax-H3Text, keyframes or reference media by checkpointVideo + 32 kHz stereoConditional: distinct FL2VA/Ref2VA checkpoints. Component gates and whole-generation coverage differ.
Wan 2.1 / 2.2Text, image by variantVideo (no audio)Conditional: variant-specific encoders/VAE, sampler and step schedule; multi-GPU is unsupported.

Media input usually needs matching companion files. DeepSeek 1017 defaults to approximate YOCO prefill; public Nemotron Omni companions omit the Parakeet audio tower. Tool and multi-GPU requirements remain in the detailed reference below and each card.

Vector output: embedding encoders accept text/code and, conditionally, media. Snowflake, MiniLM and EmbeddingGemma 2 inputs, APIs and validation →

Choose a measured path

Match the checkpoint and backend before reading a measurement. DeepSeek 1017: exact and approximate YOCO · Muse Spark: prompt length and placement · MiniMax-M3.1-Flash: layer split, TP and DSpark. Keep conditions, quality costs and build scope with each result.

Full measurement conditions, tool protocols and multi-GPU limits

Choose the model and backend for the workload you need. The newest selected comparisons have different hardware, quantizations and timing methods; they do not rank every model or backend.

Model / taskRecorded path and scopeResult and limits
DeepSeek V4.1 Flash 1017 prefill2026-10-22–23, five A40s, UD-IQ2_XXS, TP5, one request, ~1.8K-token prompt, 256 output tokensExact 563.9 tok/s vs llama.cpp layer split 174.6 (3.23×); default approximate YOCO 960.5 (5.50×), with perplexity 2.4978 vs exact 2.4811 and reference 2.4697. Internal timers, not TTFT. Conditions and quality cost →
Muse Spark long prompts and generationReport refreshed 2026-10-22 (measurement date unrecorded), ten A40s, UD-IQ3_XXS, 32K contextLayer-split 4,075-token prefill 999.7 vs 483.2 tok/s (2.07×, output cap 32); TP solo decode 22.63 vs 16.19 (1.40×, 167-token prompt, output cap 256). Corresponding llama.cpp modes; no working P2P, reference without NCCL. Full scope →
MiniMax-M3.1-Flash long prompts, generation and DSpark2026-10-24, eight A40s, UD-Q4_K_XL, layer split (and --tp 8 for its row)32K-token server prefill 1,544 vs 240 tok/s (6.43×); decode at 4K depth 23.9 vs 19.2 (1.24×); DSpark 32.3 vs 25.8 tok/s; --tp 8 decode 35.7 tok/s at depth 0 with no llama.cpp counterpart. Short-chat time to first token slightly behind (0.41 vs 0.39 s). Full scope →
Other text modelsCheck the backend guide and model-specific support before enabling speculation, tensor parallelism or CPU expert offload.These controls change memory, latency and quality trade-offs. A result for one checkpoint and interconnect is not a prediction for another.
Image and video generationUse the checkpoint's matching sampling recipe, dimensions and companion models. Image recipes → · Video recipes →Step counts, component timings and total generation latency are different quantities; the text comparisons above do not cover media generation.

Tool support requires both a prompt renderer that declares the tools and a matching structured-output parser. Qwen 3.8 Flash Next (qwen4exp) returns structured calls through the Qwen 3.5 parser, so caller-defined tools, Agent Skills tools, code-execution tools and, on the server, sub-agent delegation are all offered to it. Mistral 3 and Hunyuan Dense render neither tool declarations nor tool results, and DiffusionGemma renders no declarations and refuses requests that carry tools, so none of those tools, and no sub-agent delegation, is offered to these three families.

LTX 2.5, MiniMax-H3, Wan and DiffusionGemma reject multi-GPU modes. Mistral 3 and Hunyuan Dense run on the shared decoder runtime, which gives them --tp T, --layer-split P and both together. Qwen 3.8 Flash Next supports local eligible-quantization GGML CUDA TP as well as --layer-split N; DeepSeek V4 uses whole-layer placement. GLM supports both explicit local modes; Qwen-Image supports local tensor parallelism of its diffusion transformer only. DeepSeek V4.1 supports --layer-split N and experimental routed-MoE --tp N. Muse Spark takes both on its native executor — --layer-split N on GGML CUDA/Vulkan and --tp N on GGML CUDA — single-node only; --tp N on GGML Vulkan runs its per-op path. MiniMax-M3.1 takes both on its native executor on GGML CUDA, single-node only; its --tp N mirrors attention and refuses host experts and the DSpark drafter.

Architecture and implementation reference

Model-selection index · Full capability and implementation matrix · Project status

Original architecture keys, protocols, loading and implementation conditions

TensorSharp loads models in GGUF format and auto-detects the architecture from the file's general.architecture metadata — except MiniMax-H3 and metadata-free Qwen-Image-2.1 GGUFs (such as Unsloth's Q8_0), which carry no metadata at all and are identified by their tensors instead. Pick a quantization that fits your hardware (Q4_K_M for low memory, Q8_0 for higher quality).

Embedding models: Snowflake Arctic Embed L v2.0, MiniLM, and EmbeddingGemma 2 with text and optional image/audio/video input. Serve normalized vectors through --embeddings and OpenAI/Ollama APIs; backend support depends on the encoder. Models, APIs, and validation →

ArchitectureGGUF arch keysExample modelsMultimodalThinkingToolsMTP spec
DeepSeek V4.1 Flashdeepseek41DeepSeek-V4.1-Flash (40 layers, 384 routed experts at top-6 plus one shared expert, four residual streams with delayed hyper-connection mixing, Engram n-gram features on layers 1 and 14, 1M declared context). Served on ggml_cuda, with cuda and ggml_cpu as portability and correctness paths and cpu the 100% pure-C# executor — no ggml, no native library and no GPU, the whole V4.1 graph held to an independent PyTorch oracle at atol=rtol=2e-5 plus greedy-argmax agreement (fixture-scale — a five-layer, 256-hidden synthetic model, so architectural agreement with the oracle rather than CPU/CUDA parity on the real 246 GiB Q2_K weights), a correctness and portability path rather than a serving one; current Q2_K and Q4_K_M GGUF releases embed Engram metadata and need no Engram preparationImage and video with the separately prepared vision companion (--mmproj); audio is refused, not ignored. The companion is a native ggml component, so it is unavailable on the pure-C# cpu backend, which is text-onlyYesYes (spaced DSML, grammar-constrained)Experimental — loads a deepseek41-dspark drafter via --draft-model on ggml_cuda / ggml_cpu; initial text/image HTTP probes with trained weights passed using two-GPU layer split on ggml_cuda; broad quality and throughput remain unqualified. V4 drafters are rejected
DeepSeek V4 Flashdeepseek4DeepSeek-V4-Flash (284B MoE, 256 experts, compressed sparse attention, 1M context)Text onlyYesYes (DSML)Yes (DSpark block drafter, separate GGUF)
GLM 5.xglm-dsa, glm5nextGLM-5.2 (744B-A40B MoE, 256 experts, MLA + DeepSeek Sparse Attention, 1M context); GLM-5.3 (the same 79-block glm-dsa shape as 5.2 — 78 trunk layers plus one NextN, 256 routed experts at top-8 with one shared expert, MLA with the lightning indexer, rope base 8e6, 1M declared context capped to device capacity unless MAX_CONTEXT names the number — so it loads on the GLM-5.2 path with no new code and no new flag; at UD-Q2_K_XL it is seven shards, 236.4 GiB, in unsloth/GLM-5.3-GGUF, with --model pointed at the -00001-of- shard); GLM-5.3-Flash (320B, 288 routed experts, KDA linear attention on 34 of 45 trunk layers + NoPE MLA/DSA with a pooled indexer on the other 11)Image on 5.3-Flash only; 5.2 and 5.3 are text-only — the GLM-5.3 repo publishes no mmproj at any quant, and on glm-dsa an --mmproj is warned about and ignored rather than refused, so the run is text-only instead of a failureYesYes (XML tool calls)Yes on glm-dsa (GLM-5.2 and GLM-5.3 both embed a complete NextN block at blk.78, so there is nothing extra to download; --spec has to be on the command line before load, and it engages on single-device or explicit --layer-split N placement — under --tp N>1 the loader prints a line on stderr and serves standard decode, because that block ships no LM head of its own and the trunk head it borrows is column-parallel). Not implemented on glm5next (GLM-5.3-Flash)
Gemma 4gemma4gemma-4-E4B, 12B, 31B, 26B-A4B (MoE)Image, Video, AudioYesYesYes (separate draft)
Qwen 3.5 / 3.6qwen35, qwen35moe, qwen3nextQwen3.5-9B, Qwen3.5/3.6-35B-A3B (MoE), Qwen3.8-27B (dense)ImageYesYesYes on 3.6 and Qwen3.8-27B (embedded NextN — only in GGUFs that retain the NextN block, e.g. the -MTP- repos); Qwen3.8-27B also takes a DFlash2 block drafter, a separate GGUF via --draft-model
Qwen 3.8 Flash Nextqwen4expQwen3.8-Flash-Next (hybrid MoE: GatedDeltaNet recurrent layers interleaved with full attention, some behind Qwen Sparse Attention's indexer, a PLE n-gram embedding block, ×4 hyper-connection streams, 512 experts / 10 used)Image, video (video_url)Yes (switchable)Yes (Qwen XML / JSON tool calls)Yes (separate shared MTP head GGUF via --draft-model, every backend it loads on)
GPT OSSgptoss, gpt-ossgpt-oss-20b (MoE)Text onlyYes (always)Yes—
Nemotron-Hnemotron_h, nemotron_h_moe, nemotron_h_omniNemotron-H-8B, 47B, Nemotron 3 Nano Omni, Nemotron 3.5 Lightning 30B-A3BImage (Omni); audio only with a separately prepared Parakeet companion GGUF, otherwise refusedYesYesNo (every speculator is refused: the verify and decode kernels differ, so speculation would change the output)
Hunyuan Densehunyuan-denseTencent dense Hunyuan decoders such as the Hy-MT2 releases — GQA whose per-head QK-norm runs after NeoX RoPE, then SwiGLU. Multi-GPU through the shared decoder runtimeText onlyNoNo—
Mistral 3mistral3; also llama-labelled Mistral Small 3.x files (Tekken tokenizer, [INST] / [SYSTEM_PROMPT] tokens)Mistral-Small-3.1-24B-InstructImageNoNo—
Muse-Glimmermuse-glimmer, muse_glimmerMuse-Glimmer-30B (interleaved SWA + NoPE, attention output gate)ImageYesYes (ATEM)Yes (DFlash block drafter, separate GGUF)
Muse Sparkmuse-spark, muse_sparkMuse-Spark-Open-816B-A42B (816B sparse MoE, about 42B active per token; 46 sliding-window RoPE layers and 16 global NoPE layers, 256 routed experts at top-8). A TensorSharp-owned native whole-model executor runs it by default on ggml_cuda / ggml_vulkan (only CUDA has a full-model measurement); it needs several GPUs — ten A40s hold the UD-IQ3_XXS quant entirely — or large host RAM for the expert stacks that do not fitImage, through the shared Muse-Glimmer vision encoder and the release's own mmproj; no audioYesYes (ATEM)No learned drafter — only the weight-free n-gram speculator (--spec --spec-type ngram)
MiniMax-M3.1minimax-m3, minimax_m3MiniMax-M3.1-Flash (428B sparse MoE, about 23B active per token; 60 layers, 3 dense + 57 MoE with 128 routed experts at top-4 plus one shared; MiniMax Sparse Attention on every layer, exact dense below position 2048). A TensorSharp-owned native whole-model executor runs it on ggml_cuda only (ggml_cpu as a correctness path), with --layer-split N or --tp N over several GPUs — eight A40s hold the UD-Q4_K_XL quantImage through a converted minimax_m3 mmproj; video as sampled frames; no audioYes (adaptive)Yes (namespaced XML)Yes (DSpark block drafter, separate converted GGUF; layer split only)
Bonsai2 27Bqwen35 plus prism.hadamard.* metadata and PQ2_0 / PTQ1_0 tensorsTernary-Bonsai-2-27B-PQ2_0.gguf / -PTQ1_0.gguf (dense Qwen 3.5 hybrid, 48 GatedDeltaNet + 16 full-attention layers). Weights are transcoded losslessly to GGML Q2_0 at load and PRISM's signed Hadamard transforms are applied; single-device GGML backends only (cpu, cuda, mlx and --tp are refused). Validated on Metal (M5 Pro) onlyImage with a companion mmprojYesYes—
DiffusionGemmadiffusion-gemma, diffusion_gemmadiffusion-gemma text-diffusion GGUFs; also serves the Jev typed-decision endpoint POST /v1/systemoneImage (chat and Jev decisions), through the Gemma 4 vision tower loaded from an mmproj GGUF or the upstream model-00011-of-00011.safetensors shard; audio and video are refusedNo (not prompted; a spontaneous thought block is returned as reasoning only with think: true)No (requests with tools are refused)—
Qwen-Image-2.1qwen_image, qwen-image (metadata-free GGUFs are recognised from their tensors)Qwen-Image-2.1 DiT (+ dedicated 2.1 VAE & Qwen3-VL-8B companions)Text → image and image editing (RGBA); LoRA plug-insNoNo—
LTX 2.5 Distilledltx-2.5Q4_K_M denoiser, dedicated Gemma 4 encoder/projections, convolutional video VAE, x2 latent upscaler and audio VAE; experimentalText/first-last/timed images → video + 48 kHz stereo; optional Ingredients identity adapterNoNo—
MiniMax-H3minimax-h3, minimax_h3MiniMax-H3 FL2VA / Ref2VA denoisers (+ Qwen3-VL-32B text encoder, video VAE & audio VAE companions). The published GGUFs carry no metadata, so they are recognised by their tensorsVideo + 32 kHz stereo audio out (text → video, image → video, first/last frame, reference → video)NoNo—
Wan videowan, wan2.1, wan2.2Wan 2.1 T2V 1.3B/14B, Wan 2.2 TI2V-5B, Wan 2.2 A14B T2V/I2V (two 14B experts) (+ UMT5-XXL & video-VAE companions)Video out (text → video, image → video)NoNo—

Detailed per-model architecture cards (forward graph, components, parameters, and how TensorSharp optimizes prefill/decode) live under docs/models/ in the repository.

📘

Gemma 4 E4B is the example model in From Tensors to Tokens. Read the book for a guided build from tensor fundamentals through multimodal inference; use this reference for current downloads and capability details. Explore the book →