API Reference
Every flag, variable, endpoint, and public type in one place. Type in the box to filter all tables below instantly — or press / for wiki-wide search.
CLI flags — TensorSharp.Cli
| Flag | Description |
|---|---|
--model <path> | Path to a GGUF model file (required). |
--input <path> | Text file containing the user prompt. |
--input-jsonl <path> | JSONL file with batch requests (one JSON per line). |
--multi-turn-jsonl <path> | JSONL for multi-turn chat simulation with KV-cache reuse. |
--output <path> | Write generated text to this file. |
--image / --video / --audio <path> | Media for vision / video / audio inference. |
--pdf <path> | PDF document input (one-shot): born-digital PDFs are inlined as text; scanned PDFs become page images for a vision model (page cap: TS_PDF_MAX_PAGES). |
--mmproj <path|none> | Multimodal projector: an mmproj GGUF or, for the Gemma 4 family (DiffusionGemma included), a Hugging Face .safetensors shard holding the vision tower. Pass it explicitly; when it is omitted and an image, audio or video is given, the CLI looks beside the model only for the architecture's known projector file names (a .safetensors shard is never auto-detected). none (any case) loads no projector and skips that lookup, as on the server. |
--chat-template-file <path> | Jinja chat template that replaces the GGUF's tokenizer.chat_template, as llama.cpp's option of the same name does. Parsed before the model loads; an unreadable or unparsable file is a configuration error. Used even for families TensorSharp normally renders with its own renderer; the output parser, tool-call format and BOS handling stay the family's. |
--max-tokens <N> | Maximum tokens to generate (default 100). |
--backend <type> | cpu, cuda, mlx, ggml_cpu, ggml_metal, ggml_cuda, ggml_vulkan. Default ggml_cpu on every OS. |
--gpu-device <N> | Vulkan device index for ggml_vulkan on multi-GPU hosts (default 0; env TS_GGML_VULKAN_DEVICE). |
--list-gpus | List visible Vulkan devices (index + adapter name) and exit. |
--kv-cache-dtype <type> | KV cache precision: f32, f16, q8_0, q4_0 (default: auto per backend/model; q4_0 ~1/7 of f32, for very long 128K–256K contexts; the quantized tiers need the native GGML flash-attention path). |
--interactive / -i / --chat | Start the interactive REPL. |
--system <text> / --system-file <path> | Seed the system prompt. |
--think | Enable thinking / reasoning mode. |
--tools <path> | JSON file with tool / function definitions. |
--skills-dir <path> · --skill <name> · --list-skills · --no-skills · --skills-no-discovery | Discover/select/list/disable Agent Skills or restrict discovery. Selected skills start as metadata for tool-capable models; bodies are read on demand. Without --skills-dir, the roots are every existing .agents/skills from the working directory up to the Git root (nearest first), then skills/ beside the binary. Env TS_SKILLS_DIR / TS_NO_SKILLS. |
--skills-allow-exec · --skills-sandbox <off|preferred|required> · --skills-allow-network | Enable skills_run and choose script isolation/network policy. Execution and network are off by default; sandbox mode defaults to required. macOS uses Seatbelt, Linux needs bubblewrap 0.12+, and Windows job objects cannot confine files/network. |
--skills-max-rounds <1..64> | Internal model/tool round bound: default 8, or 24 when code execution is offered; an explicit value is kept. |
--code-exec | Enable the four server-owned agentic tools: read_file, write_file, shell, apply_patch. Off by default; env TS_CODE_EXEC. |
--code-exec-allow-install · --code-exec-packages <list> | Allow host-performed pip/npm installs and optionally restrict package names. Does not grant generated commands network access. |
--code-exec-install-domains <list> · --code-exec-install-index <url> | Installer host allow-list (default pypi.org,files.pythonhosted.org,registry.npmjs.org) and operator-selected index. |
--code-exec-allow-network | Give generated commands unrestricted host IP networking, including LAN/loopback and listening sockets. Off by default and independent of installs/skill networking. |
--code-exec-timeout <seconds> · --code-exec-max-output <bytes> · --code-exec-shell <path|name> · --code-exec-temperature <0..2> | Command deadline (default 120 s), middle-truncated output cap (32768 bytes), shell override, and optional coding-turn temperature override (only when the request temperature is still default). Coding turns always disable the built-in 1.1 repetition penalty while code tools can run. |
--code-exec-unconfined | Run despite inadequate OS confinement. Required on Windows; unsafe for users not trusted with the host. |
--code-exec-languages | Removed and rejected at startup; no replacement. A shell can reach every interpreter on its rebuilt PATH, so the host reports installed interpreters instead of claiming a language allow-list. |
--spec / --no-spec | Enable / disable speculative decoding: a drafter proposes the next few tokens and the trunk verifies them in one batched forward. Every emitted token is still drawn from a trunk row with the run's own sampler, so this is a speed path only. Must be passed before the model loads — it is what tells glm-dsa to page its ~3 GiB NextN layer into VRAM, which also leaves less room for the context. Not available under --tp N>1 on a checkpoint whose draft block borrows the trunk's LM head, which includes GLM-5.2 and GLM-5.3 — on those, speculation engages on single-device or explicit --layer-split N placement (no active tensor parallelism). The explicit opt-in for drafters embedded in the trunk checkpoint (GLM-5.2, GLM-5.3, Qwen 3.6 NextN); a drafter named with --draft-model engages without it. Default off. Env TS_SPEC. |
--spec-type <name> | Which speculation algorithm drafts: auto (default) uses whatever drafter the checkpoint carries, draft-head pins a per-token NextN/MTP head, block pins a block drafter, and ngram needs no trained weights at all — it drafts by finding where the last few tokens occurred earlier in the context and proposing what followed, and is strongest where the answer quotes the prompt (summarizing, editing, translating, repetitive structured output, agentic loops). It only picks the algorithm: pair it with --spec, because given without --spec or --draft-model (and with TS_SPEC unset) it — like --spec-draft and --spec-pmin — leaves speculation off, with a startup warning on both hosts. It still needs a model that can act as a speculative target: GPT-OSS, Mistral 3, Hunyuan Dense and DiffusionGemma have no speculative path, and Nemotron-H refuses every speculator. Some targets are conditional and otherwise serve plain decoding: DeepSeek V4 / V4.1 and Muse-Glimmer speculate only while their own drafter (DSpark / DFlash) is loaded, Gemma 4 only on the GGML backends or cuda, and Qwen 3.8 Flash Next only on the GGML backends. Env TS_SPEC_TYPE. |
--spec-draft <N> | Maximum tokens drafted per speculative step (range 1–64, default 8; a block drafter defaults to — and clamps the value to — its trained block size, 5 for DSpark). Also sizes the native graph cache at load, so it belongs on the same command line as --spec. Env TS_SPEC_DRAFT. |
--spec-pmin <f> | Confidence gate below which drafting stops. What the number means is the algorithm's business, so each brings its own default: 0.15 for a per-token head (top-1 probability), 0.35 for a block drafter (the cumulative prefix probability, so the same number is far stricter), 0 for n-gram, where it scales the required match length instead. 0 is accepted and means never gate. Env TS_SPEC_PMIN. |
--draft-model <path> | Drafter GGUF for every speculator that ships as its own file (DeepSeek V4's DSpark, the DFlash / DFlash2 block drafters for Muse-Glimmer and Qwen3.8-27B, Gemma 4's gemma4-assistant draft head, Qwen 3.8 Flash Next's shared MTP head). The file's own general.architecture decides how it loads; naming the file is the request — no --spec needed (an explicit --no-spec vetoes it). A block drafter has to be resident before the model's layer split runs; DeepSeek V4's DSpark engages on cuda / ggml_cuda only. DeepSeek V4.1 accepts a deepseek41-dspark drafter on ggml_cuda / ggml_cpu only — experimental; initial text/image HTTP probes with trained weights passed using two-GPU layer split on ggml_cuda; broad quality and throughput remain unqualified. A DFlash / DFlash2 drafter runs on one GPU: under --tp N > 1 the CLI declines it with a warning and serves standard decoding. Qwen 3.6, Qwen 3.8 27B, GLM-5.2 and GLM-5.3 embed their NextN block in the trunk GGUF and need no file (they take --spec) — GLM-5.3's is complete at blk.78, so there is nothing extra to download. Env TS_SPEC_DRAFT_MODEL. |
--temperature / --top-k / --top-p / --min-p | Sampling controls. |
--repeat-penalty / --presence-penalty / --frequency-penalty | Penalties (1.0 / 0 = off). |
--repeat-last-n <N> | How many recent tokens the repeat / presence / frequency penalties consider (default 64; 0 disables history penalties, -1 uses the whole history). The same spelling as the server and the repeat_last_n request field; the former CLI-only --penalty-last-n is removed. |
--seed <N> / --stop <string> | Random seed (-1 = random) / stop sequence (repeatable). |
--dump-prompt | Render prompt + tokenization and exit. |
--diffusion-steps / --diffusion-seed / --diffusion-blocks <N> | DiffusionGemma generation controls (48 steps by default). --diffusion-steps and --diffusion-seed also set Qwen-Image-2.1's step count (auto: 40, or a LoRA recipe's) and seed. |
--prompt <text> | Qwen-Image-2.1 prompt: generates an image, or edits the --image reference(s); the PNG goes to --output. |
--mask <path> · --mask-mode grayscale|alpha · --mask-invert · --mask-feather <px> · --mask-crop · --mask-crop-padding <px> | Qwen-Image-2.1 local editing: a mask matching the first reference selects editable pixels, with inversion, inward feathering and optional region cropping. Output retains the source dimensions and protected pixels. These are CLI flags; the server takes matching per-request fields. Semantics and ranges → |
--keep-source-size | Qwen-Image-2.1 edit: the result keeps the first --image's exact width and height, so an edit of an edit does not shrink. Sampling uses about the image's own area, at least 1 megapixel and at most the automatic area, at about its aspect ratio (the 32-pixel grid rounds both), and the result is resized to the source. Cannot be combined with --width/--height. The server takes the matching per-request field keepSourceSize, which the Web UI and TensorAgent send with every edit. |
--cfg <F> | True-CFG guidance scale. Qwen-Image-2.1: auto 1.0 (one transformer prediction per step); <= 1 disables the negative pass. Video generation reuses the flag — MiniMax-H3 ships CFG-distilled and refuses anything above 1.0, so --cfg 1.0 is the only setting it accepts. LTX 2.5 requires CFG 1 and rejects other explicit guidance recipes. |
--qwen-image-vae / --qwen-image-vl / --qwen-image-mmproj <path> | Override the resolved Qwen-Image-2.1 companions (dedicated 2.1 VAE / Qwen3-VL-8B text encoder / Qwen3-VL-8B mmproj). |
--qwen-image-variant base|turbo | Which Qwen-Image-2.1 checkpoint the DiT GGUF holds (the GGUFs carry no metadata). turbo samples Qwen-Image-2.1-Turbo's published 8-step schedule at CFG 1, refuses other step counts and refuses step-distillation --lora plug-ins. Without it, a file name with the word turbo is assumed to be Turbo and the load says so. Refused with another model. Env: TS_QWEN_IMAGE_VARIANT. |
--lora <path> | LoRA plug-in for Qwen-Image-2.1 or MiniMax-H3 (FL2VA): a TensorSharp plug-in config .json (config/lora/), or for Qwen-Image-2.1 also a LoRA .safetensors; repeatable to stack Qwen-Image-2.1 plug-ins. MiniMax-H3 takes one minimax-h3-lora config only, config/lora/minimax-h3-fl2v-turbo-8step.json or -4step.json, which locks --diffusion-steps, --flow-shift, --sampler and --cfg to its recipe and is refused on the Ref2VA checkpoint. Applied unmerged on top of the quantized transformer; a plug-in the model does not take is refused at startup, before the plug-in downloads (a model a --config still has to download is checked right after that download). → Qwen-Image-2.1 plug-ins · MiniMax-H3 Turbo |
--lora-scale <f> / --lora-config <path> | Strength (default: the plug-in's "scale", else 1.0) and companion config (a TensorSharp LoRA config, PEFT adapter_config.json or VideoX-Fun pdd_config.json) of the preceding --lora. A Qwen-Image-2.1 plug-in's sampling recipe applies unless --diffusion-steps / --cfg are given. A MiniMax-H3 plug-in's recipe stays locked, a plug-in config passed here must name the preceding --lora's weights file, and a negative strength is refused. |
--qwen-image-lora · --offload-cpu | Removed and rejected at startup. Both served only the earlier Qwen-Image-Edit pipeline. --qwen-image-lora is replaced by --lora; --offload-cpu has no replacement, because Qwen-Image-2.1 keeps its DiT weights resident. |
--penalty-last-n | Removed and rejected at startup, including as a --config key. It was the CLI-only name of the repeat-penalty window — use --repeat-last-n <N>. |
--paged-kv* · --paged-bench* · --paged-batching | Removed and rejected at startup, including as --config keys. The standalone paged-KV store they configured is gone: prompt reuse across requests is the engine's radix prefix cache (--no-prefix-cache turns it off), and its TS_KV_* variables are refused too. --paged-batching / --no-paged-batching were second spellings of --continuous-batching / --no-continuous-batching. |
--width <px> / --height <px> | Output canvas for the media families. Qwen-Image-2.1: 0 = auto (2048×2048 for generation, or about that area at the first reference's aspect ratio for editing; explicit sizes in multiples of 32). Video generation: MiniMax-H3 rounds both up to a multiple of 32. With no size, text-to-video and Ref2VA references run at 640×384 (a plain --image on Ref2VA keeps that area at the image's aspect); i2v/fl2v take the canvas from the first keyframe (the last one when it is the only one) by MiniMax's released 768p recipe — its aspect at a 768-pixel short edge, capped at 768×1344 pixels, each side on the nearest multiple of 32 (1152×768 for a 3:2 photo, 768×1152 for 2:3, 1024×768 for 4:3, 1344×768 for 16:9, 768×768 square; aspect clamped to 1:4–4:1, the excess centre-cropped). Resolution is the dominant cost lever, and faces need pixels: those modes keep the keyframe's framing, and fitted to 640×384 a full-body photo's ~60 px face is smeared by the VAE alone. The recipe costs 2.4–4.2× the video tokens per frame of 640×384, and attention cost grows faster than that. Both --width and --height pin one size for every mode, keyframes included; with only one, the other follows the image's aspect. LTX 2.5 defaults to 1024×1024 and requires dimensions divisible by 64. |
--video-frames <N> / --fps <N> | Output frame count, snapped up onto the model's temporal grid — 17k+5 for MiniMax-H3 (5, 22, 39, 56, 73, 90, 107 …; default 22), 4k+1 for Wan (default 33, or 49 for Wan2.2-TI2V) — and MP4 playback rate (Wan: 16, or 24 for Wan2.2-TI2V). A model trained at a fixed rate overrides the value asked for: MiniMax-H3 is pinned to 24 fps. LTX 2.5 defaults to 17 frames/24 fps and requires the 8k+1 frame grid. |
--video-mode <mode> | MiniMax-H3 conditioning mode, inferred from what you pass when omitted: t2v (text only), i2v (the image is the first frame and gets animated), fl2v (first and last frame), ref (the images are identity/appearance references for a new scene). i2v/fl2v need the fl2va checkpoint and ref needs ref2va — separate files, and each refuses the other's inputs by naming the file to load instead. On ref2va a plain --image is taken as a single reference. |
--end-image <file> | Last-frame conditioning image (MiniMax-H3 first/last-frame mode). Combined with --image the clip is steered to start and end on the two frames. LTX 2.5 also supports last-frame timeline guides. |
--ref-image <file> | Reference still for MiniMax-H3 Ref2VA: the subject carries over while camera, background and composition come from the prompt. Repeatable, up to nine references in total; the prompt refers to them positionally as <Picture 1>, <Picture 2>, … References are only ever scaled down and keep their own aspect ratio, so the output canvas still comes from --width/--height. On LTX 2.5 ordinary image lists become evenly spaced timeline guides; ref identity references require the optional Ingredients adapter. |
--ref-video <path> / --ref-video-audio <file> | Reference clip — a video file or a directory of frames, resampled onto MiniMax-H3's own 24 fps and canvas, referred to as <Video 1>, <Video 2>, … — and its soundtrack, which pairs by position: the first --ref-video-audio with the first --ref-video. Separate flags because a container's audio track is not readable through the frame decoder. A reference clip is the most expensive input H3 takes: a 22-frame 448×320 one adds 980 conditioning tokens on top of the 1680 the output needs. |
--ref-audio <file> | Standalone reference soundtrack (<Audio 1>, …), WAV / MP3 / Ogg, resampled to the audio VAE's 32 kHz stereo and truncated to the generated clip's duration. Repeatable; stills, clips and audio share the same nine-reference budget. |
--no-audio | Skip the audio decode on a model that generates a track jointly with the video (LTX 2.5 or MiniMax-H3), saving the audio VAE's time and memory. Ignored by video-only models. |
--flow-shift <F> / --sampler <name> | FlowMatch timestep shift (0 = the model's official recipe; 12.0 for MiniMax-H3, where it shifts the video stream only, or 6.0 under a MiniMax-H3 Turbo plug-in, which refuses any other value) and sampler: unipc (Wan default) or euler; LTX 2.5 defaults to euler_ancestral, where euler is a deterministic diagnostic. MiniMax-H3 runs its own flow-match Euler schedule and ignores --sampler, except that a Turbo plug-in refuses any sampler but euler. |
--negative-prompt <text> | Classifier-free-guidance negative prompt (default: the model's official one). No unconditional pass runs at --cfg 1.0, so it is inert on MiniMax-H3, which accepts nothing else, and on a step-distilled Wan checkpoint, which runs guidance-free. |
--cfg-cache-stride <N> | Wan guidance cache: run the unconditional CFG pass on one step in N and reuse the cached guidance direction in between (default 0 = off). At 50 steps, 2 = 1.30x and 3 = 1.43x. Approximate, and it has no effect at --cfg 1.0 — so nothing on MiniMax-H3 or a distilled Wan checkpoint. |
--video-vae / --video-text-encoder / --audio-vae <path> | Override the resolved video companions: the video VAE (minimax_h3_video_vae_fp16.safetensors for MiniMax-H3, the causal 3D VAE for Wan), the text encoder (Qwen3-VL-32B for MiniMax-H3, UMT5-XXL for Wan), and the audio VAE MiniMax-H3 decodes its soundtrack with — omit it and the model still runs, just silently. Env TS_VIDEO_VAE / TS_VIDEO_TEXT_ENCODER / TS_VIDEO_AUDIO_VAE; Wan A14B's second expert is --video-dit2. |
--n-cpu-moe <N|all> (-ncmoe) · --cpu-moe (-cmoe) · --cpu-moe-threads <N> | Keep the routed MoE expert weights of the first N layers (or all of them) in system RAM and multiply them on the CPU. Default 0 on every architecture, DeepSeek V4 included. Host threads default to 1 with two or fewer usable CPUs, all but one with up to eight, and otherwise half, capped at 64 (DeepSeek V4 / V4.1 and GLM-5.x on their native executors use every usable CPU once experts are offloaded, GLM-5.x also on a GPU-less run). Env TS_N_CPU_MOE / TS_CPU_MOE / TS_CPU_MOE_THREADS. |
--no-prefix-cache | Disable Radix prefix caching and the interactive system/tool prompt warm-up (also sets TS_SCHED_PREFIX_CACHE=0). Prefix caching is on by default and lives in memory for the CLI process. |
--continuous-batching / --no-continuous-batching | Enable (default) / disable the paged continuous-batching engine. |
--benchmark / --bench-prefill / --bench-decode / --bench-runs | Synthetic throughput benchmark. --bench-chunked uses the server-style chunked prefill; --bench-fixed-tokens times a predetermined decode stream without host sampling; --bench-random-tokens draws the prompt and decode ids at random, as llama-bench does. |
--bench-kvcache / --bench-kv-turns <N> | Multi-turn KV-cache reuse benchmark. |
--warmup-runs <N> | Throw-away forward passes before timing (default 0). |
--test / --test-templates <dir> | Built-in tokenizer/template tests; validate templates against GGUF Jinja2. |
--test-chunked-prefill / --correct-prefill <N> / --correct-decode <N> | Prefill one prompt single-pass and chunked and compare the decoded tokens (defaults 1500 prompt tokens, 8 decode steps). |
--config <path> | Read options from a JSON file keyed by long option names (repeatable). A single-valued option given on the command line (or in a later file) drops the file's entry before it is resolved, so the command line wins and that entry's download is skipped, with one [config] line on stderr; the repeatable options (--stop, --skills-dir, --skill, --lora*, --image, --ref-*) add to the file's values instead. See config/README.md. |
--tp <N> | Tensor parallelism degree — split the model across N GPUs in one process (default 1; requires --backend cuda, ggml_cuda, or ggml_vulkan). Also accepted by the server. |
--tp-node-id <N> / --tp-peers <list> | Multi-node distributed TP: this node's 0-based ID and the shared comma-separated host:port peer list. |
--log-level / --log-dir / --log-file / --log-console | Logger level, directory, and file/console toggles. CLI only: the server has no --log-* flags and reads the TENSORSHARP_LOG_* variables instead. |
Server flags — TensorSharp.Server.Host
| Flag | Description |
|---|---|
--model <path> | GGUF file to host. Required at startup; a model-less process cannot select a GGUF through /api/models/load. |
--mmproj <path|none> | Explicit multimodal projector: an mmproj GGUF or, for the Gemma 4 family (DiffusionGemma included), a Hugging Face .safetensors vision-tower shard. A bare file name is resolved next to the model; none disables it. The server does not auto-detect it. |
--chat-template-file <path> | Jinja chat template that replaces the --model GGUF's tokenizer.chat_template (requires --model; not with --embeddings). Parsed at startup; applies to the --model file only, so a model loaded later through /api/models/load keeps its own template. See agent clients. |
--embeddings · --embedding-threads <N> · --embedding-context-size <N> | Host a GGUF embedding encoder instead of a chat model (/v1/embeddings, /api/embed). Requires --model, forbids --mmproj, and runs on cpu, ggml_cpu, ggml_metal or ggml_cuda; the chat, Jev, image and video routes answer 400 in this mode, and an explicit --backend the machine lacks exits 2. See Embeddings. |
--port <N> · --host <address> · --urls <urls> | Listen port (default 5000; env PORT), bind address (default 0.0.0.0; env HOST), or full listen URLs (falls back to ASPNETCORE_URLS). |
--no-webui | Do not serve the bundled Web UI; the HTTP APIs and /uploads stay up. Env TS_NO_WEBUI. |
--backend <type> | Compute backend; defaults to ggml_metal on macOS and ggml_cpu elsewhere. |
--gpu-device <N> / --list-gpus | Vulkan device selection for ggml_vulkan / list visible Vulkan devices and exit. |
--tp <N> · --tp-node-id <N> · --tp-peers <list> | Tensor parallelism, same semantics as the CLI. Env TENSORSHARP_TP_DEGREE / TENSORSHARP_TP_NODE_ID / TENSORSHARP_TP_PEERS. |
--help | Print the parameter reference and exit (also shown when started with no arguments). An unknown option is refused at startup, with a did-you-mean suggestion when a known flag is close. |
--config <path> | Read options from a JSON file, as on the CLI (repeatable). |
--max-tokens <N> | Generation limit for every endpoint: fills in when a request omits it and caps a larger request (default 20000, which only fills in). |
--temperature / --top-k / --top-p / --min-p | Default sampling values. |
--repeat-penalty / --presence-penalty / --frequency-penalty / --seed | Default penalties and seed. |
--repeat-last-n <N> | Default penalty window (64 tokens); env TENSORSHARP_REPEAT_LAST_N. The CLI uses the same spelling; the former CLI-only --penalty-last-n is removed and refused on both hosts. |
--stop <string> | Stop sequence (repeatable); merged with a per-request list under config precedence, replaced by it under request. |
--sampling-precedence <config|request> | Whether configured sampling parameters outrank the ones a request sends (default config). Parameters left unconfigured always come from the request. |
--continuous-batching / --no-continuous-batching | Enable (default) / disable iteration-level paged batching. |
--no-prefix-cache | Disable the Radix prefix cache (on by default), the startup warm-up of the shared prompt and the prefix checkpoints saved between launches in prefix-cache/ beside the binary (TENSORSHARP_PREFIX_CACHE_DIR moves them). Also sets TS_SCHED_PREFIX_CACHE=0. |
--n-cpu-moe <N|all> · --cpu-moe · --cpu-moe-threads <N> | MoE expert offload to system RAM; same semantics and defaults as the CLI. |
--spec / --no-spec | Enable / disable speculative decoding (default off); engages for solo, non-concurrent sequences. The explicit opt-in for drafters embedded in the trunk checkpoint; a drafter named with --draft-model engages without it. |
--spec-type <name> | Speculation algorithm: auto (default) uses whatever drafter the checkpoint carries, draft-head and block pin one explicitly, and ngram needs no trained weights at all — it drafts by suffix match over the context — but still only runs on a speculative-target model (not GPT-OSS, Mistral 3, Hunyuan Dense or DiffusionGemma; Nemotron-H refuses every speculator; DeepSeek V4 / V4.1 and Muse-Glimmer only while their own drafter is loaded, Gemma 4 only on the GGML backends or cuda, Qwen 3.8 Flash Next only on the GGML backends). It never turns speculation on by itself: given without --spec or --draft-model, startup logs a WARNING that speculation stays off. |
--spec-draft <N> | Max tokens drafted per speculative step (1–64, default 8; a block drafter defaults to — and clamps the value to — its trained block size). |
--spec-pmin <f> | Draft-confidence gate in [0, 1] (0 = never gate); drafting stops below it. Default per algorithm — 0.15 for a per-token head, 0.35 for a block drafter (where the gate is the cumulative prefix probability), 0 for n-gram. |
--draft-model <path> | Drafter GGUF for every speculator that ships as its own file (DeepSeek V4's DSpark, the DFlash / DFlash2 block drafters for Muse-Glimmer and Qwen3.8-27B, Gemma 4 gemma4-assistant, Qwen 3.8 Flash Next's shared MTP head). The file's own general.architecture decides how it loads; naming the file is the request, so it needs no --spec (an explicit --no-spec vetoes it). Qwen 3.6, Qwen 3.8 27B, GLM-5.2 and GLM-5.3 embed their NextN block and need no draft file; an explicit drafter that cannot activate (a DFlash / DFlash2 drafter under --tp N > 1, any drafter on Nemotron-H) stops startup with exit code 2; DeepSeek V4's DSpark serves solo sequences on cuda / ggml_cuda. DeepSeek V4.1 accepts a deepseek41-dspark drafter on ggml_cuda / ggml_cpu only — experimental; initial text/image HTTP probes with trained weights passed using two-GPU layer split on ggml_cuda; broad quality and throughput remain unqualified. Env TS_SPEC_DRAFT_MODEL. |
--prefill-chunk-size <N> | Maximum prefill tokens per scheduler step (sets TS_SCHED_PREFILL_CHUNK). |
--kv-cache-dtype <type> | KV cache precision: f32, f16, q8_0, q4_0 (default: auto per backend/model; env KV_CACHE_DTYPE). |
--skills-dir · --skill · --list-skills · --no-skills · --skills-no-discovery | Server-side Agent Skills registry, startup selection and discovery policy; same semantics as the CLI, except that skills/ beside the binary (where POST /api/skills installs uploads) is always scanned first, even with explicit roots, so it wins a name clash. |
--skills-allow-exec · --skills-sandbox · --skills-allow-network · --skills-max-rounds | Script opt-in, isolation/network policy and 1–64 round cap. Defaults: execution/network off, sandbox required, rounds 8 or 24 with code execution. |
--code-exec · --code-exec-allow-install · --code-exec-allow-network · --code-exec-unconfined | Enable built-in code tools and their separate install/network/unconfined permissions. All are operator startup decisions; there is no per-command approval endpoint. |
--code-exec-packages · --code-exec-install-domains · --code-exec-install-index · --code-exec-timeout · --code-exec-max-output · --code-exec-shell · --code-exec-temperature | Package/install source policy, command limits, shell, and optional coding-turn temperature. When code tools can run, coding turns disable the built-in 1.1 repetition penalty independently of that temperature flag. See Agentic Work. |
--code-exec-languages | Removed and rejected at startup; no replacement. The shell reaches every interpreter on its rebuilt PATH. |
--no-multi-agent | Turn off sub-agent delegation, which is on by default for tool-capable models on /v1/chat/completions, /v1/responses, /api/chat/ollama and /api/chat. Env TS_NO_MULTI_AGENT (any value but 0). A request can only opt out, with "multi_agent": false. See Sub-agents. |
--agents-max-concurrent · --agents-max-count · --agents-max-depth · --agents-max-rounds · --agents-max-generations · --agents-timeout · --agents-max-result-chars | Per-request delegation limits: active children across the tree (default 3, range 1–32), children per request (8, 1–128), depth (2, 1–8), tool-loop rounds per child (8, 1–64), shared child-generation budget (48, 1–1024), seconds per child run (180, 1–3600), and characters per child report (8000, 256–64000). |
--agents-allow-worker-tools | Let worker children use the parent's enabled mutable tools. Without it every child is read-only; explorer and reviewer children always are. |
--upload-max-mb <N> · --upload-quota-mb <N> · --upload-ttl-hours <N> | Upload-directory limits: per-file cap on client-supplied files (default 500 MB; 413 when exceeded; POST /api/upload's request-body limit follows it and never drops below 500 MB, while every other route keeps 500 MB), a total quota that also counts generated outputs (507), and deletion of files older than N hours. Quota and TTL are off by default. Env TS_UPLOAD_MAX_MB / TS_UPLOAD_QUOTA_MB / TS_UPLOAD_TTL_HOURS. |
--qwen-image-vae / --qwen-image-vl / --qwen-image-mmproj <path> | Override the resolved Qwen-Image-2.1 companions (2.1 VAE / Qwen3-VL-8B text encoder / mmproj). |
--qwen-image-variant base|turbo | Which Qwen-Image-2.1 checkpoint the DiT GGUF holds (the GGUFs carry no metadata). turbo samples Qwen-Image-2.1-Turbo's published 8-step schedule at CFG 1, refuses other step counts and refuses step-distillation --lora plug-ins. Without it, a file name with the word turbo is assumed to be Turbo and the load says so. Another model loads without it, with a warning. Env: TS_QWEN_IMAGE_VARIANT. |
--lora · --lora-scale · --lora-config | LoRA plug-ins for the hosted Qwen-Image-2.1 or MiniMax-H3 (FL2VA) model; same semantics as the CLI. Checked at startup, where a plug-in the startup model does not take is refused before the plug-in downloads (a model a --config still has to download is checked right after that download), and applied to every image request or clip (no per-request selection). On Qwen-Image-2.1 a request's steps / cfg still override a plug-in's recipe; a MiniMax-H3 Turbo plug-in locks --video-steps and a request's steps, flowShift and sampler to its recipe. |
--qwen-image-lora · --offload-cpu | Removed and rejected at startup. Both served only the earlier Qwen-Image-Edit pipeline. --qwen-image-lora is replaced by --lora; --offload-cpu has no replacement, because Qwen-Image-2.1 keeps its DiT weights resident. |
--penalty-last-n | Removed and rejected at startup, including as a --config key. It was the CLI-only name of the repeat-penalty window — use --repeat-last-n <N>. |
--paged-kv* · --paged-bench* · --paged-batching | Removed and rejected at startup, including as --config keys. The standalone paged-KV store they configured is gone: prompt reuse across requests is the engine's radix prefix cache (--no-prefix-cache turns it off), and its TS_KV_* variables are refused too. --paged-batching / --no-paged-batching were second spellings of --continuous-batching / --no-continuous-batching. |
--video-width <px> / --video-height <px> | Default output size when a request omits width/height — the main quality lever, and a startup flag because the browser sends no size of its own. Aliases --width / --height, which additionally set the default Qwen-Image-2.1 image size (TS_QWEN_IMAGE_WIDTH / TS_QWEN_IMAGE_HEIGHT; used only when both are given, for image requests that name neither a size nor an area; an off-grid value is rounded down to a multiple of 32 with a one-time warning, and one side alone is ignored with a warning); rounded up to the model's grid (a multiple of 32 for MiniMax-H3). For MiniMax-H3 leave both unset: text-to-video and Ref2VA references then run at 640×384 (a picture the Web UI attaches on Ref2VA keeps that area at its own aspect), and i2v/fl2v take MiniMax's 768p recipe canvas from the first keyframe (1152×768 for a 3:2 photo; see the CLI --width row), which is what keeps faces in a keyframe intact. Setting both pins that one size for every mode, keyframes included, and turns off Studio's “From the first frame” size; give only one of the two and H3 takes the other from the conditioning image's aspect ratio. LTX 2.5 defaults to 1024×1024 and requires dimensions divisible by 64. |
--video-steps <N> | Default denoising steps when a request omits steps — the quality/time trade-off after resolution. MiniMax-H3 defaults to 20, is visibly cleaner at 16–24, and gains little past ~30; for fewer steps load a Turbo plug-in with --lora, which samples exactly its 8 or 4 steps and refuses any other --video-steps at startup (and any other request steps). The server has no --cfg at all. LTX 2.5 accepts the fixed eight-step first pass and automatically adds three refinement steps. |
--video-mode <mode> | Default conditioning mode when a request omits videoMode: t2v, i2v, fl2v or ref. Omit it and the mode is inferred per request, which is usually what you want; pin it for a deployment that only offers one. |
--video-frames <N> / --fps <N> | Default frame count and MP4 playback rate when a request omits frames/fps. The count is snapped to the model's temporal grid — 17k+5 for MiniMax-H3 (default 22), 4k+1 for Wan (default 33, or 49 for Wan2.2-TI2V). MiniMax-H3 is pinned to 24 fps whatever is asked for. LTX 2.5 defaults to 17 frames/24 fps and requires the 8k+1 frame grid. |
--video-vae / --video-text-encoder / --audio-vae <path> | Video companions: the video VAE, the text encoder (Qwen3-VL-32B for MiniMax-H3, UMT5-XXL for Wan), and the audio VAE MiniMax-H3 decodes its soundtrack with — without it the model still runs and produces silent video. Env TS_VIDEO_VAE / TS_VIDEO_TEXT_ENCODER / TS_VIDEO_AUDIO_VAE; Wan A14B's second expert is --video-dit2. |
--redis-url <url> | Sets TS_RESPONSES_STORE_REDIS_URL (only when unset), which moves the /v1/responses store to Redis (--help lists it under "Responses API store"). |
Environment variables
| Variable | Description |
|---|---|
TS_KNOWLEDGE_DIRECTORY | Standalone server store root; setting it enables /api/knowledge. TensorAgent uses its own data directory. Configuration → |
TS_KNOWLEDGE_EMBEDDING_MODEL / TS_KNOWLEDGE_EMBEDDING_BACKEND / TS_KNOWLEDGE_QUERY_PREFIX / TS_KNOWLEDGE_DOCUMENT_PREFIX | Optional local GGUF encoder, backend and query/document prefixes; changing its identity needs reindexing for semantic search. |
TS_KNOWLEDGE_RERANK | 0 disables chat hosts' bounded model relevance pass; fallback retains the original retrieval order. |
BACKEND | Default backend (ggml_metal on macOS, ggml_cpu elsewhere). |
MAX_TOKENS | Default max generation length (20000). |
TS_PDF_MAX_PAGES | Cap on PDF pages read (upload and CLI --pdf); 0 = all pages (default). |
VIDEO_SAMPLE_FPS / VIDEO_MAX_FRAMES | Video frame sampling rate / cap. |
TENSORSHARP_TEMPERATURE / _TOP_K / _TOP_P / _MIN_P | Default sampling values. |
TENSORSHARP_REPEAT_PENALTY / _REPEAT_LAST_N / _PRESENCE_PENALTY / _FREQUENCY_PENALTY / _SEED | Default penalties, penalty window (64) and seed. |
TENSORSHARP_SAMPLING_PRECEDENCE | config (default) or request: whether the values above outrank the ones a client sends (= --sampling-precedence). |
TENSORSHARP_LOG_LEVEL / _LOG_DIR / _LOG_FILE | Logging level, directory, file toggle (CLI + server). |
TS_SKILLS_DIR / TS_NO_SKILLS / TS_SKILLS_ALLOW_EXEC / TS_SKILLS_MAX_ROUNDS / TS_SKILLS_SANDBOX / TS_SKILLS_ALLOW_NETWORK | Agent Skills roots, disable switch, script opt-in, round bound, sandbox mode and script network opt-in. |
TS_CODE_EXEC / TS_CODE_EXEC_ALLOW_INSTALL / TS_CODE_EXEC_ALLOW_NETWORK / TS_CODE_EXEC_INSTALL_DOMAINS / TS_CODE_EXEC_INSTALL_INDEX | Code-tool opt-in, separate install/network grants, and host-installer source policy. |
TS_NO_MULTI_AGENT | Server only: any value but 0 turns sub-agent delegation off (= --no-multi-agent). |
TS_UPLOAD_MAX_MB / TS_UPLOAD_QUOTA_MB / TS_UPLOAD_TTL_HOURS | Upload-directory per-file cap (default 500), total quota and file lifetime (both off by default) (= --upload-max-mb / --upload-quota-mb / --upload-ttl-hours). |
TENSORSHARP_UPLOAD_DIR | Where the server stores uploads and generated files (default uploads/ beside the binary). |
DIFFUSION_STEPS / DIFFUSION_MAX_BATCH | DiffusionGemma steps per block / max batched requests. |
TS_JEV_MAX_BODY_MB / TS_JEV_MAX_CANVAS / TS_JEV_MAX_PENDING | /v1/systemone limits: request-body cap in MiB (default 8, range 1–64), answer-canvas width per question chunk (default 64, range 8–4096, also bounded by the checkpoint), and admitted requests before 529 (default 32, range 1–1024). |
KV_CACHE_DTYPE | KV cache precision (CLI + server): f32, f16, q8_0, q4_0; default auto (= --kv-cache-dtype). |
TS_SCHED_DISABLE_BATCHED | 1 forces per-sequence KV-swap (= --no-continuous-batching). |
TS_SCHED_MAX_BATCHED_TOKENS | Per-step token budget (4096). |
TS_SCHED_MAX_RUNNING_SEQS | Max in-flight sequences (16). |
TS_SCHED_PREFILL_CHUNK | Per-request mixed-step prefill cap (256); prefill-only steps fill the batch budget. |
TS_SCHED_SOLO_PREFILL_CHUNK | Prefill chunk for a solo / uncontended request (8192), bounded by TS_SCHED_MAX_BATCHED_TOKENS. Left unset, a model that sizes its own lone-prompt steps does so instead — Muse Spark with pipelined prefill takes 8192-token steps, which may exceed that budget; setting it turns the model's steps off. |
TS_SCHED_DECODE_QUANTUM | Decode tokens before a sequence switch (256 = block size). |
TS_SCHED_NUM_BLOCKS / TS_SCHED_BLOCK_SIZE | Engine block-pool size (256) / tokens per block (16 for a family that reuses only whole pages, otherwise 256). |
TS_SCHED_PREFIX_CACHE | 0 disables all prefix reuse (--no-prefix-cache sets it). |
TENSORSHARP_PREFIX_CACHE_DIR | Server: root for the prefix checkpoints saved between launches, one subdirectory per model (default prefix-cache/ beside the binary). |
TS_SPEC / TS_SPEC_TYPE / TS_SPEC_DRAFT / TS_SPEC_PMIN / TS_SPEC_DRAFT_MODEL | Speculative-decoding knobs, mirroring the --spec* flags: on/off, algorithm (auto, draft-head, block, ngram), draft length, confidence gate, and the separate draft-head GGUF. |
TS_DSV4_UBATCH / TS_DSV4_PERF | DeepSeek V4: prefill micro-batch, and throughput/stage logging. |
TS_DSV4_THREADS / TS_DSV4_CPU_TRACE_DIR | DeepSeek V4.1 Flash on --backend cpu, the pure-C# DeepSeek4CpuExecutor — no ggml, no native library and no GPU, so it runs anywhere .NET does. TS_DSV4_THREADS is the compute-thread count and defaults to Environment.ProcessorCount on this backend, against min(cores, 32) elsewhere; TS_DSV4_CPU_TRACE_DIR writes the same per-tensor files eng/dsv41-reference.py --output writes, so the two directories diff tensor by tensor. This is a correctness and portability path, not a serving path. Engram comes directly from the GGUF metadata, and MAX_CONTEXT caps to 65,536 by default here. |
TS_DSV41_ENGRAM_DEVICE | Native DeepSeek V4.1 loader placement: which device holds the Engram tables. On --backend cpu only 0 is accepted, and that backend also refuses --tp, distributed TP groups and any draft model, all before any weight is read. On V4.1 a drafter is accepted only on ggml_cuda or ggml_cpu and only as a deepseek41-dspark file (experimental: initial text/image HTTP probes with trained weights passed using two-GPU layer split on ggml_cuda; broad quality and throughput remain unqualified); anywhere else naming one is a hard refusal rather than DeepSeek V4's warn-and-continue. TS_DSV41_ENGRAM_WARM / _THREADS / _RANDOM, TS_DSV41_SPARSE_FA and TS_DSV41_COMPACT_RAW_GATHER are native-loader only and inert on cpu. |
TS_MUSE_SPARK_NATIVE / TS_MUSE_SPARK_CPU_MOE / TS_MUSE_SPARK_PLAN_SLOTS / TS_MUSE_SPARK_DEV0_RESERVE_MB | Muse Spark. The native whole-model executor is on by default for ggml_cuda / ggml_vulkan; 0 selects the per-op path (also for --tp N, which otherwise runs the native executor on upstream ggml's meta backend on ggml_cuda, and the per-op path on ggml_vulkan) and 1 opts the GGML CPU backend in. TS_MUSE_SPARK_CPU_MOE=auto places each layer's gate, up and down expert stacks separately, keeping as many on the GPUs as fit, and overrides --n-cpu-moe. TS_MUSE_SPARK_PLAN_SLOTS is the number of sliding-window rings the placement plan reserves (default 1; raise it for concurrent requests), and TS_MUSE_SPARK_DEV0_RESERVE_MB extra room on GPU 0, e.g. for the vision encoder (default 0). The full option table is in the model card. |
TS_MUSE_SPARK_PIPELINE / TS_MUSE_SPARK_PIPELINE_UBATCH | Muse Spark pipelined prefill on a layer split. Unset, it is automatic: on with two or more CUDA GPUs, no host experts and no tensor parallelism. 0 turns it off, 1 requires it (refused under --tp and on a multi-GPU non-CUDA backend), and any other value refuses the load. TS_MUSE_SPARK_PIPELINE_UBATCH fixes the micro-batch rows (1 up to the 2048-row prefill ubatch); unset, each call gets about a quarter of its length, between 512 and 2048 rows. |
TS_QWEN_IMAGE_VAE / TS_QWEN_IMAGE_TE / TS_QWEN_IMAGE_MMPROJ | Qwen-Image-2.1 companion paths (dedicated 2.1 VAE / Qwen3-VL-8B text encoder / Qwen3-VL-8B mmproj). |
TS_QWEN_IMAGE_WIDTH / TS_QWEN_IMAGE_HEIGHT | Qwen-Image-2.1 default output size for an image request that gives neither a width/height nor an explicit targetArea (a request with its own area keeps its own geometry). Used only when both are set; a value off the 32-pixel grid is rounded down to a multiple of 32 (minimum 32) with a one-time warning, and one alone — or an unparsable or negative value — is ignored with a one-time warning. The server's --width / --height set them. |
TS_QWEN21_EDIT_NOISE | Qwen-Image-2.1 edit noise. references (the default): an edit's initial noise follows the seed and its reference images, so an edit never restarts from the noise that drew its source. seed: edits draw the seed's text-to-image noise, as stable-diffusion.cpp does, for matched-noise comparisons; each such edit prints a line. Any other value fails the request. |
TS_QWEN21_PREFIX_CACHE / _PREFIX_CACHE_TYPE / _PREFIX_CACHE_MAX_MIB | Qwen-Image-2.1 prefix KV cache: the prompt and reference tokens' K/V are stored at step 1 and reused by every later step. On by default (0/false/off/no disables it); storage auto (what attention reads), f32, f16, q8_0 or q8_0_v; a MiB cap on top of the built-in limit of half the device's free memory. The model card measures 1024² single-reference edits at 1.73–1.86× per step on an M5 Pro (Metal) and 1.92× on an A40 (CUDA), with byte-identical PNGs; generation gains 1–3%. |
TS_QWEN21_GRAPH_REUSE / TS_QWEN21_FLASH / TS_QWEN21_PAD_MASK | Qwen-Image-2.1 DiT diagnostics: 0 stops retaining graphs between steps (retention is on by default and applies on CUDA and Metal only), 0 turns flash attention off, and 1 selects padded attention masks (off by default). |
TS_QWEN21_VAE_FUSED / TS_QWEN21_VISION_FUSED | Qwen-Image-2.1 whole-VAE decode graph (default on CUDA and Metal, never on Vulkan; 0 forces the per-convolution path, 1 opts in on another GGML backend) and the fused vision encoder on GGML CUDA (0 disables). |
TS_LORAS | The channel through which both hosts hand the parsed --lora / --lora-scale / --lora-config list to the Qwen-Image-2.1 or MiniMax-H3 model: a JSON array of {"path", "scale", "config"} objects. When the command line names no plug-in, an inherited TS_LORAS gets the same startup checks; entries the model does not take are skipped with a warning. The retired TS_QWEN_IMAGE_LORA is refused at load, with advice to use --lora. |
TS_VIDEO_VAE / TS_VIDEO_TEXT_ENCODER / TS_VIDEO_AUDIO_VAE / TS_VIDEO_DIT2 | Video companion paths (= --video-vae / --video-text-encoder / --audio-vae / --video-dit2): the video VAE, the text encoder (Qwen3-VL-32B for MiniMax-H3, UMT5-XXL for Wan), the audio VAE MiniMax-H3 decodes its soundtrack with, and Wan 2.2 A14B's second high/low-noise expert GGUF. |
TS_VIDEO_TOKENIZER | Folder holding MiniMax-H3's vocab.json and merges.txt. The text-encoder GGUF carries no tokenizer, and auto-download can only fill in options that are flags, so take the pair from MiniMaxAI/MiniMax-H3 under processor/ and either drop it beside the encoder or point this variable at it. |
TS_H3_TRACE | 1 prints MiniMax-H3's latent and velocity magnitudes for every denoise step. The sampler already fails a request whose velocity goes non-finite — naming the step it appeared at, rather than writing a black clip — and this shows where the magnitudes left the FP16 range. |
TS_H3_PREFAULT | Sequentially reads the MiniMax-H3 denoiser file so its first upload is a copy out of the page cache rather than a page-fault storm inside the host-to-device transfer (0.91 GB/s faulting as it copies against 5.97 GB/s from resident pages, RTX 3080 Laptop). Default 3: the read starts the moment the text trunk hands over its hidden states and is pipelined with the upload rather than joined before it. 1 is serial, 2 overlaps it with text conditioning — measured worse, because the encoder streams its own 17 GB through the same page cache and evicts the pages just placed — and 0 disables it. Output is byte-identical in every mode. |
TS_H3_PREFAULT_THREADS | Read streams the prefault uses (default 1). More measured worse here — 640x384 best of three on an RTX 3080 Laptop: 1 stream 63.9 s, 4 streams 64.9 s, 16 streams 66.6 s — because this read runs concurrently with the encoder teardown and the upload it is warming, not with the machine to itself. |
TS_H3_PHASE | 1 prints a per-stage MiniMax-H3 breakdown: text-encoder open / trunk / teardown, the prefault, every denoise step, and VAE open / decode. The one-line summaries tell you a phase was slow; this tells you which half of it. |
TS_H3_TE_GROUP | <n> runs MiniMax-H3's 50-layer text-encoder trunk in groups of n layers, releasing each group's device copy. Off by default: on a 16 GB RTX 3080 Laptop it does remove the trunk's spill (peak 16 041 -> 12 981 MiB) and is bit-identical, but the trunk is a one-shot prefill over a short prompt, so it was 3 s slower. |
TS_H3_KEEP_RESIDENT | 1 keeps MiniMax-H3's denoiser and both VAEs resident between clips on Metal, the old behaviour. By default they are released before each clip's text encoder, which held peak wired memory at 19.1 GB instead of 33.3 GB in the TensorAgent Mac app on an M5 Pro, for about 1.3 s more text conditioning. Discrete GPUs are unaffected. |
TS_WAN_DIT_KV_F16 | 0 restores F32 attention keys/values in the Wan DiT. F16 is the default and is 2.02x on a single 27k-token self-attention at the same 0.999964 DiT cosine. |
TS_WAN_VAE_MPS_CONV | Metal only: 0 restores ggml's im2col+GEMM lowering for the Wan VAE convolutions. MPSGraph is the default (a 736x544x81f decode: 159 s -> 80 s, numerics unchanged). |
TS_WAN_VAE_GEMM_MAX_MB / TS_WAN_VAE_TILE | Wan VAE im2col GEMM budget in MB (default: derived from free device memory) and 0 to disable the horizontal-band tiling used above ~0.5 MP. |
TS_WAN_METAL_TENSOR_API | Force the Metal 4 tensor API on (1) or off (0) for the Wan DiT. Default: on for A14B/14B-class DiTs, off for smaller ones such as TI2V-5B. |
TS_WAN_DIT_CAPTURE / TS_WAN_DIT_FLASH | 0 disables the persistent CUDA-graph-captured Wan DiT graph / forces materialized attention. Both are slower — A/B and debugging only. |
TS_WAN_HEARTBEAT_S | Wan progress heartbeat interval in seconds (default 30; 0 silences it). |
TS_FFMPEG | Path to the ffmpeg used to write the near-lossless MP4 (default: ffmpeg on PATH). |
TENSORSHARP_TP_DEGREE | Local tensor-parallel degree — GPUs to split the model across (= --tp on the CLI and the server). |
TENSORSHARP_TP_DEVICES | GPU ordinals the TP ranks map to, e.g. 0,2 (default 0..tp-1). GGML backends. |
TENSORSHARP_TP_NODE_ID / TENSORSHARP_TP_PEERS | Multi-node distributed TP: 0-based node ID and the shared host:port peer list (= --tp-node-id / --tp-peers). |
TENSORSHARP_TP_CONNECT_TIMEOUT_SECONDS / _RECV_TIMEOUT_SECONDS | Peer connect retry window (120 s) and per-receive timeout (300 s) for the distributed TP mesh. |
TENSORSHARP_TP_DISABLE_P2P / TENSORSHARP_TP_HOST_ALLREDUCE | 1 forces cross-GPU copies / the local AllReduce through host memory instead of CUDA P2P (diagnostics). |
TS_GGML_TP_DEVICE_AR_THRESHOLD / TS_GGML_TP_PARALLEL | GGML TP: element count above which AllReduce uses the device collective (default 262144), and 0 to drive the ranks sequentially (diagnostic). |
TS_GEMMA4_TP_FUSED_MOE | 0 falls back from Gemma 4's fused MoE trunk under TP to the whole-expert per-op path. |
GGML_CUDA_ALLREDUCE / GGML_CUDA_AR_BF16_THRESHOLD | ggml collective selection (nccl / internal / none) and the payload size above which F32 collectives convert to BF16 (raised to 1 MB by TensorSharp). |
TS_RESPONSES_STORE_REDIS_URL | Redis-backed OpenAI Responses API store (replaces the in-memory store); the live half of --redis-url. |
TS_GGML_VULKAN_DEVICE | Vulkan device index for ggml_vulkan (= --gpu-device). |
TS_MLX_* | MLX backend tuning: mlock GGUF, batched MoE decode, layer-eval cadence, memory caps. |
TENSORSHARP_MLX_LIBRARY / _LIBRARY_DIR | Override the search path for libmlxc. |
TENSORSHARP_GGML_NO_UPDATE / _GGML_GIT_REF | Skip / pin the ggml source clone on native builds. |
TENSORSHARP_GGML_NATIVE_ENABLE_VULKAN | Force the GGML Vulkan backend on/off at native build time (ON/OFF = build-script --vulkan/--no-vulkan). Auto-enabled when a Vulkan runtime is detected; an explicit choice sticks across rebuilds. |
HTTP endpoints
| Method & path | Style | Purpose |
|---|---|---|
/api/knowledge/collections · document routes · POST /api/knowledge/search · POST /api/knowledge/read | Opt-in | Collection/document lifecycle, scoped search/read and versioned original downloads. Search/read use collectionIds; supported inference requests use knowledgeCollections. Endpoint scope → |
GET / · GET /index.html | Web UI | Browser chat application when wwwroot is present; bare / falls back to liveness only in a headless deployment. |
GET /health | Utility | Stable plain-text liveness response. |
POST /api/generate | Ollama | Single-prompt completion (stream or not). |
POST /api/chat/ollama | Ollama | Multi-turn chat with optional think / tools / images. |
GET /api/tags | Ollama | List the hosted model. |
POST /api/show | Ollama | Model info. |
POST /v1/chat/completions | OpenAI | Chat Completions (stream, tools, response_format, skills, multi_agent). |
POST /v1/responses · GET /v1/responses/{id} | OpenAI | Responses API generation and stored-response lookup. Stateless per request: previous_response_id is refused. |
GET /v1/models | OpenAI | List models. |
POST /v1/embeddings · POST /api/embed | OpenAI / Ollama | Normalized sentence embeddings from a server started with --embeddings. |
POST /v1/systemone | Jev | Typed decisions (noul, choice, score) about a text and/or image state on a hosted DiffusionGemma model; jev-latest / jev-preview alias the loaded model. See HTTP API. |
POST /api/chat | Web UI | SSE chat stream with session + KV-reuse fields. |
POST /api/sessions · DELETE /api/sessions/{id} | Web UI | Create / dispose a per-tab session. |
POST /api/upload | Web UI | Upload an image / audio / video / PDF / text file. |
GET /v1/skills/{name?} | OpenAI | List registered skills or read one descriptor including its instructions. |
GET /api/skills · GET /api/skills/{name} · GET /api/skills/{name}/files/{*path} | Web UI | List/read skills and safely fetch a bundled file. |
POST /api/skills · POST /api/skills/rescan · DELETE /api/skills/{name} | Web UI | Install a skill ZIP, rescan roots, or remove an installed skill. |
GET /api/code/artifacts/{runId} · GET /api/code/artifacts/{runId}/{*path} | Web UI | List files retained from one code run or download one as a forced attachment. |
POST /api/image-generate | Web UI | Qwen-Image-2.1 text-to-image (JSON). |
POST /api/image-generate/stream | Web UI | SSE text-to-image with live denoising previews. |
POST /api/image-edit | Web UI | Qwen-Image-2.1 one-shot edit (multipart or JSON). |
POST /api/image-edit/stream | Web UI | SSE image edit with live denoising previews. |
POST /api/video-generate | Web UI | Video generation on the hosted model (MiniMax-H3, Wan). JSON { prompt, width, height, frames, steps, cfg, fps, imagePath, videoMode, generateAudio, endImage, referenceImages, referenceVideos, referenceAudios, referenceVideoAudios } — videoMode, generateAudio, endImage and the four reference* lists also accept a snake_case spelling (camelCase wins if both are sent); the rest are camelCase only, and an unrecognized spelling such as image_path is silently ignored. referenceVideoAudios pairs by index with referenceVideos, and each file field names something previously uploaded through /api/upload. Returns a URL to the MP4 plus audioUrl, null when the model produced no track. A model rejection (wrong checkpoint, keyframes together with references) comes back as a 400 carrying the model's own message. |
POST /api/video-generate/stream | Web UI | Same body, streamed as SSE progress ticks per denoise step. |
POST /v1/videos/generations | OpenAI | OpenAI-shaped envelope over the same parser (size: "640x384"); the sidecar track comes back as audio_url. |
GET /api/models | Web UI | Hosted model, supported backends, defaults. For a video model it also returns a video object — family, supportsAudio, supportsImageConditioning, supportsEndImageConditioning, supportsReferenceConditioning, maxReferenceImages — null for every non-video model, so a client tests one field instead of pattern-matching an architecture string. |
POST /api/models/load | Web UI | Re-load the startup model (optionally on a different backend). |
GET /api/version · GET /api/queue/status | Utility | Server version / legacy queue snapshot. |
GET /uploads/* | Utility | Stored uploads and generated images / videos, with an allow-listed content type and nosniff; still served under --no-webui. |
Sampling parameters
Ollama (options) | OpenAI (top-level) | Default | Meaning |
|---|---|---|---|
num_predict | max_tokens | --max-tokens / 20000 | Maximum tokens to generate; the server default applies to every endpoint when omitted. OpenAI's max_completion_tokens is accepted too. |
temperature | temperature | 0.8 | Sampling temperature (0 = greedy). |
top_k | top_k | 40 | Top-K filtering (0 = disabled). |
top_p | top_p | 0.9 | Nucleus sampling threshold (1.0 = disabled). |
min_p | min_p | 0 | Minimum probability filtering. |
repeat_penalty | repeat_penalty / repetition_penalty | 1.1 | Repetition penalty (1.0 = off). |
repeat_last_n | repeat_last_n | 64 | How many recent tokens the penalties look back over. |
presence_penalty / frequency_penalty | presence_penalty / frequency_penalty | 0 | Presence / frequency penalties. |
seed | seed | -1 | Random seed (-1 = random). |
stop | stop | null | Stop sequences. |
| — | response_format | null | text, json_object, or json_schema. |
C# public API
| Member | Signature / values | Notes |
|---|---|---|
KnowledgeStore · KnowledgeOptions · IKnowledgeExtractor · IKnowledgeEmbeddingProvider | TensorSharp.Knowledge | Independent personal store, configuration and extraction/embedding extension points. Coverage and limits. |
ModelBase.Create | static ModelBase Create(string ggufPath, BackendType backend, int tpDegree = 1, ITensorParallelGroup tpGroup = null, string draftModelPath = null) | Auto-detects architecture from GGUF metadata. |
ModelBase.Forward | float[] Forward(int[] tokens) | Returns next-token logits (length = vocab size). |
ModelBase.Sample | int Sample(float[] logits, SamplingConfig config, IList<int> generated = null) | Applies penalties + sampling. |
ModelBase.SampleGreedy | int SampleGreedy(float[] logits) | Deterministic argmax. |
ModelBase.Config / .Tokenizer | ModelConfig / ITokenizer | Config.VocabSize, context length, etc. |
BackendType | Cpu, Cuda, Mlx, GgmlCpu, GgmlMetal, GgmlCuda, GgmlVulkan | Backend selector enum. |
ITokenizer.Encode | List<int> Encode(string text, bool addSpecial = true) | Text → token ids. |
ITokenizer.Decode | string Decode(List<int> ids) | Token ids → text. |
ITokenizer.IsEos / .EosTokenIds | bool IsEos(int id) / int[] EosTokenIds | End-of-sequence detection. |
SamplingConfig | Temperature, TopK, TopP, MinP, penalties, Seed, StopSequences, MaxTokens | See C# Library. |
IBatchedPagedModel.ForwardBatch | batched/paged forward | Implemented by most architectures for continuous batching. |
GgufPromptRenderer.Render | string Render(string template, List<ChatMessage> messages, bool addGenerationPrompt = true, string? architecture = null, List<ToolFunction>? tools = null, bool enableThinking = false) | Renders a conversation with the model's chat template (IPromptRenderer). |
MultiAgentOptions | Enabled (true), MaxConcurrentAgents 3, MaxAgents 8, MaxDepth 2, MaxRoundsPerAgent 8, MaxTotalChildGenerations 48, AgentTimeoutSeconds 180, MaxResultCharacters 8000, MaxTaskCharacters 16000, AllowWorkerTools (false) | Sub-agent limits for SkillAgentLoop and SkillsChatClient (TensorSharp.AgentHost.Agents). See C# Library. |
REPL commands
| Command | Description |
|---|---|
/help, /? | Show all interactive commands. |
/exit, /quit | Leave the session. |
/reset, /new | Clear conversation history and KV cache. |
/history · /save <file> | Print / append the transcript. |
/system <text> | Set the system prompt (resets KV cache). |
/think on|off · /multiline on|off | Toggle reasoning mode / multi-line input. |
/info, /status | Show model, backend, architecture, context/vocab, projector, depth. |
/model <path> · /backend <name> · /mmproj <path> (/projector) | Hot-swap model, backend (cpu, cuda, ggml_cpu, ggml_metal, ggml_cuda, ggml_vulkan), or projector; unloading a projector takes /model <path>. |
/sampling, /show | Print current sampling configuration. |
/max · /temp · /topk · /topp · /minp | Set reply length / temperature / top-k / top-p / min-p (also /maxtokens, /temperature). |
/repeat · /presence · /frequency · /seed | Set penalties and seed. |
/stop <text> · /clearstop | Add / clear stop sequences. |
/image · /audio · /video · /text <path> · /clearattach | Attach media / text for the next turn; drop pending attachments. Aliases /img, /vid, /txt and /file. |
/skills · /skill <name> | List installed skills and mark the active ones / turn one on or off (either toggle resets the conversation). |
Community discovery uses the find-skills tools skills_search, skills_inspect and skills_install with skill-network permission; these are agent tools, not separate HTTP installation endpoints. The server/app can install; the default CLI registry is read-only with search/inspection. Conditions and current skill packs →
LTX 2.5: use --config config/ltx-2.5-distilled.json; NVIDIA uses cuda/ggml_cuda, Apple uses mlx/ggml_metal (the MLX executor), and CPU aliases select direct CPU. Defaults are 1024×1024×17 at 24 fps, dimensions divisible by 64 and frames 8k+1; eight half-size steps plus three refinement steps, CFG 1. Five files, audio, conditioning and limits →