Supported Models
Choose the output you need, then check inputs, model files, backend and validation scope. Receiving images, video or audio and generating those media are separate capabilities; the table keeps them apart.
First run → · Compare families → · Read support status →
Browse by task
Use these five existing reference pages: choose a task while comparing models, or downloads when you already know the model. Each family's files, memory requirements and launch commands remain in its reference.
Embedding Models & Semantic Search
Snowflake, MiniLM and EmbeddingGemma 2: text and multimodal vectors, APIs, and retrieval quality.
Model Downloads
Every GGUF, with the projector, VAE, encoder and drafter files each family needs.
Text & LLM Models
Supported model families and their run recipes, plus multimodal input, thinking and tool calling.
Image Generation
Qwen-Image-2.1 text-to-image and image editing, its defaults and draft settings.
Video Generation
LTX 2.5 and MiniMax-H3 generate joint audio/video; Wan 2.1 / 2.2 generates video.
Reading support status
Implemented means a code path exists. Conditional requires a named variant, backend or companion. Experimental marks explicit implementation or validation limits. Fixture-only covers small synthetic/component checks. Measured refers to the recorded checkpoint, hardware, workload and build. Unsupported means refused or unimplemented. Read input capability and validation status separately.
Choose quantization and memory settings from the model card. A quantization name, component gate or successful load does not establish full-model quality on every device.
Compare family inputs, outputs and conditions
| Family / card | Inputs | Output / task | Conditions and evidence |
|---|---|---|---|
| Gemma 4 | Text, image, video, audio | Text | Conditional: matching projector for media. Start with the E4B Q8_0 native-GGML lane. |
| Qwen 3.5 / 3.6 / 3.8 27B | Text, image | Text | Conditional: NextN needs a GGUF retaining its draft block; check variant and backend. |
| Qwen 3.8 Flash Next | Text, image, video | Text | Conditional: memory placement, tensor parallelism and the separate MTP head depend on quantization/backend. |
| Bonsai2 27B | Text, image with projector | Text | Conditional: single-device GGML only. Recorded Metal and bounded CPU checks do not qualify other devices. |
| GPT OSS | Text | Text | Implemented: thinking mode and tools; check the card for backend and cache requirements. |
| Nemotron-H / Omni | Text; image on Omni; conditional audio | Text | Conditional: audio needs a Parakeet companion absent from public GGUFs. Speculation is unsupported. |
| Mistral 3 | Text, image | Text | Unsupported: explicit thinking mode and tools. Check the named checkpoint/backend coverage. |
| Hunyuan Dense | Text | Text | Implemented on the shared decoder runtime: --tp, --layer-split and their hybrid with batched serving; tools are unsupported. |
| GLM 5.x | Text; image only on 5.3-Flash | Text | Conditional: 5.2/5.3 are text-only. NextN excludes active tensor parallelism; the CPU reference has numerical limits. |
| DeepSeek V4 Flash | Text | Text | Conditional: check memory and placement; optional DSpark speculation requires a matching drafter. |
| DeepSeek V4.1 Flash / 1017 | Text; conditional image/video | Text | Experimental: routed-MoE TP/DSpark. Pure-C# CPU coverage is fixture-only; default 1017 YOCO prefill is approximate. |
| Muse Spark | Text, image with Spark projector | Text | Conditional: multi-GPU or large host memory. Recorded CUDA measurements do not qualify every backend; audio is unsupported. |
| MiniMax-M3.1-Flash | Text, image with a converted projector | Text | Conditional: ggml_cuda and several GPUs (232 GiB UD-Q4_K_XL); the projector and DSpark drafter must be converted. Recorded 8x A40 measurements do not qualify other backends; audio is unsupported. |
| Muse-Glimmer | Text, image | Text | Conditional: projector for images, ATEM for tools, separate DFlash drafter for optional speculation; check backend limits. |
| DiffusionGemma | Text, image | Text | Unsupported: chat audio, video_url and tools. Web UI video is supplied as sampled images. |
| Jev (DiffusionGemma) | Text, images, extracted documents/frames/transcripts | Typed decisions | Conditional: sampled video and configured ASR transcripts; see ingestion and probability limits. |
| Qwen-Image-2.1 | Text, optional reference/mask images | Images / image edits | Conditional: matching encoder/VAE and sampling recipe; component coverage varies by backend. |
| LTX 2.5 Distilled | Text, image keyframes; optional identity sheets | Video + 48 kHz stereo | Experimental: current two-stage recipe and companions; Vulkan and multi-GPU are unsupported. |
| MiniMax-H3 | Text, keyframes or reference media by checkpoint | Video + 32 kHz stereo | Conditional: distinct FL2VA/Ref2VA checkpoints. Component gates and whole-generation coverage differ. |
| Wan 2.1 / 2.2 | Text, image by variant | Video (no audio) | Conditional: variant-specific encoders/VAE, sampler and step schedule; multi-GPU is unsupported. |
Media input usually needs matching companion files. DeepSeek 1017 defaults to approximate YOCO prefill; public Nemotron Omni companions omit the Parakeet audio tower. Tool and multi-GPU requirements remain in the detailed reference below and each card.
Vector output: embedding encoders accept text/code and, conditionally, media. Snowflake, MiniLM and EmbeddingGemma 2 inputs, APIs and validation →
Choose a measured path
Match the checkpoint and backend before reading a measurement. DeepSeek 1017: exact and approximate YOCO · Muse Spark: prompt length and placement · MiniMax-M3.1-Flash: layer split, TP and DSpark. Keep conditions, quality costs and build scope with each result.
Full measurement conditions, tool protocols and multi-GPU limits
Choose the model and backend for the workload you need. The newest selected comparisons have different hardware, quantizations and timing methods; they do not rank every model or backend.
| Model / task | Recorded path and scope | Result and limits |
|---|---|---|
| DeepSeek V4.1 Flash 1017 prefill | 2026-10-22–23, five A40s, UD-IQ2_XXS, TP5, one request, ~1.8K-token prompt, 256 output tokens | Exact 563.9 tok/s vs llama.cpp layer split 174.6 (3.23×); default approximate YOCO 960.5 (5.50×), with perplexity 2.4978 vs exact 2.4811 and reference 2.4697. Internal timers, not TTFT. Conditions and quality cost → |
| Muse Spark long prompts and generation | Report refreshed 2026-10-22 (measurement date unrecorded), ten A40s, UD-IQ3_XXS, 32K context | Layer-split 4,075-token prefill 999.7 vs 483.2 tok/s (2.07×, output cap 32); TP solo decode 22.63 vs 16.19 (1.40×, 167-token prompt, output cap 256). Corresponding llama.cpp modes; no working P2P, reference without NCCL. Full scope → |
| MiniMax-M3.1-Flash long prompts, generation and DSpark | 2026-10-24, eight A40s, UD-Q4_K_XL, layer split (and --tp 8 for its row) | 32K-token server prefill 1,544 vs 240 tok/s (6.43×); decode at 4K depth 23.9 vs 19.2 (1.24×); DSpark 32.3 vs 25.8 tok/s; --tp 8 decode 35.7 tok/s at depth 0 with no llama.cpp counterpart. Short-chat time to first token slightly behind (0.41 vs 0.39 s). Full scope → |
| Other text models | Check the backend guide and model-specific support before enabling speculation, tensor parallelism or CPU expert offload. | These controls change memory, latency and quality trade-offs. A result for one checkpoint and interconnect is not a prediction for another. |
| Image and video generation | Use the checkpoint's matching sampling recipe, dimensions and companion models. Image recipes → · Video recipes → | Step counts, component timings and total generation latency are different quantities; the text comparisons above do not cover media generation. |
Tool support requires both a prompt renderer that declares the tools and a matching structured-output parser. Qwen 3.8 Flash Next (qwen4exp) returns structured calls through the Qwen 3.5 parser, so caller-defined tools, Agent Skills tools, code-execution tools and, on the server, sub-agent delegation are all offered to it. Mistral 3 and Hunyuan Dense render neither tool declarations nor tool results, and DiffusionGemma renders no declarations and refuses requests that carry tools, so none of those tools, and no sub-agent delegation, is offered to these three families.
LTX 2.5, MiniMax-H3, Wan and DiffusionGemma reject multi-GPU modes. Mistral 3 and Hunyuan Dense run on the shared decoder runtime, which gives them --tp T, --layer-split P and both together. Qwen 3.8 Flash Next supports local eligible-quantization GGML CUDA TP as well as --layer-split N; DeepSeek V4 uses whole-layer placement. GLM supports both explicit local modes; Qwen-Image supports local tensor parallelism of its diffusion transformer only. DeepSeek V4.1 supports --layer-split N and experimental routed-MoE --tp N. Muse Spark takes both on its native executor — --layer-split N on GGML CUDA/Vulkan and --tp N on GGML CUDA — single-node only; --tp N on GGML Vulkan runs its per-op path. MiniMax-M3.1 takes both on its native executor on GGML CUDA, single-node only; its --tp N mirrors attention and refuses host experts and the DSpark drafter.
Architecture and implementation reference
Model-selection index · Full capability and implementation matrix · Project status
Original architecture keys, protocols, loading and implementation conditions
TensorSharp loads models in GGUF format and auto-detects the architecture from the file's general.architecture metadata — except MiniMax-H3 and metadata-free Qwen-Image-2.1 GGUFs (such as Unsloth's Q8_0), which carry no metadata at all and are identified by their tensors instead. Pick a quantization that fits your hardware (Q4_K_M for low memory, Q8_0 for higher quality).
Embedding models: Snowflake Arctic Embed L v2.0, MiniLM, and EmbeddingGemma 2 with text and optional image/audio/video input. Serve normalized vectors through --embeddings and OpenAI/Ollama APIs; backend support depends on the encoder. Models, APIs, and validation →
| Architecture | GGUF arch keys | Example models | Multimodal | Thinking | Tools | MTP spec |
|---|---|---|---|---|---|---|
| DeepSeek V4.1 Flash | deepseek41 | DeepSeek-V4.1-Flash (40 layers, 384 routed experts at top-6 plus one shared expert, four residual streams with delayed hyper-connection mixing, Engram n-gram features on layers 1 and 14, 1M declared context). Served on ggml_cuda, with cuda and ggml_cpu as portability and correctness paths and cpu the 100% pure-C# executor — no ggml, no native library and no GPU, the whole V4.1 graph held to an independent PyTorch oracle at atol=rtol=2e-5 plus greedy-argmax agreement (fixture-scale — a five-layer, 256-hidden synthetic model, so architectural agreement with the oracle rather than CPU/CUDA parity on the real 246 GiB Q2_K weights), a correctness and portability path rather than a serving one; current Q2_K and Q4_K_M GGUF releases embed Engram metadata and need no Engram preparation | Image and video with the separately prepared vision companion (--mmproj); audio is refused, not ignored. The companion is a native ggml component, so it is unavailable on the pure-C# cpu backend, which is text-only | Yes | Yes (spaced DSML, grammar-constrained) | Experimental — loads a deepseek41-dspark drafter via --draft-model on ggml_cuda / ggml_cpu; initial text/image HTTP probes with trained weights passed using two-GPU layer split on ggml_cuda; broad quality and throughput remain unqualified. V4 drafters are rejected |
| DeepSeek V4 Flash | deepseek4 | DeepSeek-V4-Flash (284B MoE, 256 experts, compressed sparse attention, 1M context) | Text only | Yes | Yes (DSML) | Yes (DSpark block drafter, separate GGUF) |
| GLM 5.x | glm-dsa, glm5next | GLM-5.2 (744B-A40B MoE, 256 experts, MLA + DeepSeek Sparse Attention, 1M context); GLM-5.3 (the same 79-block glm-dsa shape as 5.2 — 78 trunk layers plus one NextN, 256 routed experts at top-8 with one shared expert, MLA with the lightning indexer, rope base 8e6, 1M declared context capped to device capacity unless MAX_CONTEXT names the number — so it loads on the GLM-5.2 path with no new code and no new flag; at UD-Q2_K_XL it is seven shards, 236.4 GiB, in unsloth/GLM-5.3-GGUF, with --model pointed at the -00001-of- shard); GLM-5.3-Flash (320B, 288 routed experts, KDA linear attention on 34 of 45 trunk layers + NoPE MLA/DSA with a pooled indexer on the other 11) | Image on 5.3-Flash only; 5.2 and 5.3 are text-only — the GLM-5.3 repo publishes no mmproj at any quant, and on glm-dsa an --mmproj is warned about and ignored rather than refused, so the run is text-only instead of a failure | Yes | Yes (XML tool calls) | Yes on glm-dsa (GLM-5.2 and GLM-5.3 both embed a complete NextN block at blk.78, so there is nothing extra to download; --spec has to be on the command line before load, and it engages on single-device or explicit --layer-split N placement — under --tp N>1 the loader prints a line on stderr and serves standard decode, because that block ships no LM head of its own and the trunk head it borrows is column-parallel). Not implemented on glm5next (GLM-5.3-Flash) |
| Gemma 4 | gemma4 | gemma-4-E4B, 12B, 31B, 26B-A4B (MoE) | Image, Video, Audio | Yes | Yes | Yes (separate draft) |
| Qwen 3.5 / 3.6 | qwen35, qwen35moe, qwen3next | Qwen3.5-9B, Qwen3.5/3.6-35B-A3B (MoE), Qwen3.8-27B (dense) | Image | Yes | Yes | Yes on 3.6 and Qwen3.8-27B (embedded NextN — only in GGUFs that retain the NextN block, e.g. the -MTP- repos); Qwen3.8-27B also takes a DFlash2 block drafter, a separate GGUF via --draft-model |
| Qwen 3.8 Flash Next | qwen4exp | Qwen3.8-Flash-Next (hybrid MoE: GatedDeltaNet recurrent layers interleaved with full attention, some behind Qwen Sparse Attention's indexer, a PLE n-gram embedding block, ×4 hyper-connection streams, 512 experts / 10 used) | Image, video (video_url) | Yes (switchable) | Yes (Qwen XML / JSON tool calls) | Yes (separate shared MTP head GGUF via --draft-model, every backend it loads on) |
| GPT OSS | gptoss, gpt-oss | gpt-oss-20b (MoE) | Text only | Yes (always) | Yes | — |
| Nemotron-H | nemotron_h, nemotron_h_moe, nemotron_h_omni | Nemotron-H-8B, 47B, Nemotron 3 Nano Omni, Nemotron 3.5 Lightning 30B-A3B | Image (Omni); audio only with a separately prepared Parakeet companion GGUF, otherwise refused | Yes | Yes | No (every speculator is refused: the verify and decode kernels differ, so speculation would change the output) |
| Hunyuan Dense | hunyuan-dense | Tencent dense Hunyuan decoders such as the Hy-MT2 releases — GQA whose per-head QK-norm runs after NeoX RoPE, then SwiGLU. Multi-GPU through the shared decoder runtime | Text only | No | No | — |
| Mistral 3 | mistral3; also llama-labelled Mistral Small 3.x files (Tekken tokenizer, [INST] / [SYSTEM_PROMPT] tokens) | Mistral-Small-3.1-24B-Instruct | Image | No | No | — |
| Muse-Glimmer | muse-glimmer, muse_glimmer | Muse-Glimmer-30B (interleaved SWA + NoPE, attention output gate) | Image | Yes | Yes (ATEM) | Yes (DFlash block drafter, separate GGUF) |
| Muse Spark | muse-spark, muse_spark | Muse-Spark-Open-816B-A42B (816B sparse MoE, about 42B active per token; 46 sliding-window RoPE layers and 16 global NoPE layers, 256 routed experts at top-8). A TensorSharp-owned native whole-model executor runs it by default on ggml_cuda / ggml_vulkan (only CUDA has a full-model measurement); it needs several GPUs — ten A40s hold the UD-IQ3_XXS quant entirely — or large host RAM for the expert stacks that do not fit | Image, through the shared Muse-Glimmer vision encoder and the release's own mmproj; no audio | Yes | Yes (ATEM) | No learned drafter — only the weight-free n-gram speculator (--spec --spec-type ngram) |
| MiniMax-M3.1 | minimax-m3, minimax_m3 | MiniMax-M3.1-Flash (428B sparse MoE, about 23B active per token; 60 layers, 3 dense + 57 MoE with 128 routed experts at top-4 plus one shared; MiniMax Sparse Attention on every layer, exact dense below position 2048). A TensorSharp-owned native whole-model executor runs it on ggml_cuda only (ggml_cpu as a correctness path), with --layer-split N or --tp N over several GPUs — eight A40s hold the UD-Q4_K_XL quant | Image through a converted minimax_m3 mmproj; video as sampled frames; no audio | Yes (adaptive) | Yes (namespaced XML) | Yes (DSpark block drafter, separate converted GGUF; layer split only) |
| Bonsai2 27B | qwen35 plus prism.hadamard.* metadata and PQ2_0 / PTQ1_0 tensors | Ternary-Bonsai-2-27B-PQ2_0.gguf / -PTQ1_0.gguf (dense Qwen 3.5 hybrid, 48 GatedDeltaNet + 16 full-attention layers). Weights are transcoded losslessly to GGML Q2_0 at load and PRISM's signed Hadamard transforms are applied; single-device GGML backends only (cpu, cuda, mlx and --tp are refused). Validated on Metal (M5 Pro) only | Image with a companion mmproj | Yes | Yes | — |
| DiffusionGemma | diffusion-gemma, diffusion_gemma | diffusion-gemma text-diffusion GGUFs; also serves the Jev typed-decision endpoint POST /v1/systemone | Image (chat and Jev decisions), through the Gemma 4 vision tower loaded from an mmproj GGUF or the upstream model-00011-of-00011.safetensors shard; audio and video are refused | No (not prompted; a spontaneous thought block is returned as reasoning only with think: true) | No (requests with tools are refused) | — |
| Qwen-Image-2.1 | qwen_image, qwen-image (metadata-free GGUFs are recognised from their tensors) | Qwen-Image-2.1 DiT (+ dedicated 2.1 VAE & Qwen3-VL-8B companions) | Text → image and image editing (RGBA); LoRA plug-ins | No | No | — |
| LTX 2.5 Distilled | ltx-2.5 | Q4_K_M denoiser, dedicated Gemma 4 encoder/projections, convolutional video VAE, x2 latent upscaler and audio VAE; experimental | Text/first-last/timed images → video + 48 kHz stereo; optional Ingredients identity adapter | No | No | — |
| MiniMax-H3 | minimax-h3, minimax_h3 | MiniMax-H3 FL2VA / Ref2VA denoisers (+ Qwen3-VL-32B text encoder, video VAE & audio VAE companions). The published GGUFs carry no metadata, so they are recognised by their tensors | Video + 32 kHz stereo audio out (text → video, image → video, first/last frame, reference → video) | No | No | — |
| Wan video | wan, wan2.1, wan2.2 | Wan 2.1 T2V 1.3B/14B, Wan 2.2 TI2V-5B, Wan 2.2 A14B T2V/I2V (two 14B experts) (+ UMT5-XXL & video-VAE companions) | Video out (text → video, image → video) | No | No | — |
Detailed per-model architecture cards (forward graph, components, parameters, and how TensorSharp optimizes prefill/decode) live under docs/models/ in the repository.
Gemma 4 E4B is the example model in From Tensors to Tokens. Read the book for a guided build from tensor fundamentals through multimodal inference; use this reference for current downloads and capability details. Explore the book →