Getting Started

From a source checkout to a verified CLI reply or local API. The quick start runs Gemma 4 E4B on a native GGML backend; a managed CPU path (no CMake or GPU toolchain) remains available when you cannot build native code.

1 · Prerequisites

Install the .NET 10 SDK

TensorSharp targets net10.0, so a source build requires the full .NET 10 SDK—installing only the .NET Runtime or ASP.NET Core Runtime is not enough. The SDK includes the runtimes needed to run the CLI and server. Start with Microsoft's cross-platform .NET installation guide, then use the instructions for your operating system:

Open a new terminal after installation and confirm that a 10.0.x SDK is listed:

dotnet --list-sdks

Other prerequisites

• git, curl, and network access — used to clone TensorSharp and download a model. A full native build also clones ggml into ExternalProjects/ggml/; set TENSORSHARP_GGML_NO_UPDATE=1 to skip later network updates.
• A GGUF model file — e.g. from Hugging Face. See Model downloads.

Per-platform toolchains (only for GPU acceleration)

2 · Clone and choose a build path

git clone https://github.com/zhongkaifu/TensorSharp.git
cd TensorSharp

Managed-CPU path (no native toolchain)

To skip every native build, pass both skip properties and use the managed cpu backend. The first run restores and builds the managed projects automatically.

-p:TensorSharpSkipGgmlNative=true -p:TensorSharpSkipMlxNative=true -- --backend cpu

Full native / GPU path

Build the whole solution when you want GGML CPU, Metal, CUDA, Vulkan, or MLX. The first build compiles the native bridge and can take several minutes; subsequent builds are incremental.

dotnet build TensorSharp.slnx -c Release

Or build just one application:

# Console application
dotnet build TensorSharp.Cli/TensorSharp.Cli.csproj -c Release

# Web application
dotnet build TensorSharp.Server/TensorSharp.Server.csproj -c Release

The CLI binary lands in TensorSharp.Cli/bin/... and the server in TensorSharp.Server/bin/.... For a CUDA- or Vulkan-enabled native build, manual native builds, or MLX, see Building the native libraries.

3 · Download a model

For the quick start, download the recommended benchmark-verified gemma-4-E4B-it-Q8_0.gguf file (7.48 GiB) from the public ggml-org/gemma-4-E4B-it-GGUF repository. The lower-memory gemma-4-E4B-it-Q4_K_M.gguf lives in the same repository, and the Models page lists other options.

curl --create-dirs --fail -L "https://huggingface.co/ggml-org/gemma-4-E4B-it-GGUF/resolve/main/gemma-4-E4B-it-Q8_0.gguf?download=true" -o models/gemma-4-E4B-it-Q8_0.gguf

Gemma 4 E4B backend and platform notes

The first-run commands below are for Linux + NVIDIA. On Windows + NVIDIA, set $env:TENSORSHARP_GGML_NATIVE_ENABLE_CUDA='ON' in PowerShell, then run the same commands without the Bash assignment prefix. On Apple Silicon, omit the CUDA environment assignment and use ggml_metal. For Vulkan on Windows/Linux, set TENSORSHARP_GGML_NATIVE_ENABLE_VULKAN=ON (PowerShell: $env:TENSORSHARP_GGML_NATIVE_ENABLE_VULKAN='ON') and use ggml_vulkan. Without a GPU, use ggml_cpu (native CPU kernels). Text needs no projector. For image, video, or audio, download mmproj-gemma-4-E4B-it-Q8_0.gguf from the same repository and add --mmproj models/mmproj-gemma-4-E4B-it-Q8_0.gguf.

4 · First run

The one-shot prompt comes from a file via --input; --prompt is reserved for Qwen-Image-Edit. The CLI defaults to ggml_cpu, greedy decoding, and 100 generated tokens. The environment assignment enables the CUDA native build; the first run compiles it and can take several minutes, later runs are incremental.

Option A — one-shot generation (CLI)

echo "What is TensorSharp? Answer briefly." > prompt.txt
TENSORSHARP_GGML_NATIVE_ENABLE_CUDA=ON dotnet run --project TensorSharp.Cli -c Release -p:TensorSharpSkipMlxNative=true -- --model models/gemma-4-E4B-it-Q8_0.gguf --input prompt.txt --max-tokens 128 --backend ggml_cuda

Option B — interactive chat (REPL)

dotnet run --project TensorSharp.Cli -c Release --no-build -- --model models/gemma-4-E4B-it-Q8_0.gguf -i --max-tokens 128 --backend ggml_cuda

Type messages turn-by-turn; drive the session with slash commands like /reset, /think on, or /image photo.png.

Option C — browser UI + HTTP APIs (server)

TENSORSHARP_GGML_NATIVE_ENABLE_CUDA=ON dotnet run --project TensorSharp.Server -c Release -p:TensorSharpSkipMlxNative=true -- --model models/gemma-4-E4B-it-Q8_0.gguf --backend ggml_cuda --max-tokens 128
# open http://localhost:5000/index.html

The health response is at http://localhost:5000/; the browser chat is at /index.html. The Ollama- and OpenAI-compatible endpoints use the same fixed port.

Where to go next