Embedding Models & Semantic Search

Generate normalized vectors locally through OpenAI- and Ollama-compatible APIs. Snowflake Arctic Embed and MiniLM encode text/code; EmbeddingGemma 2 also combines text, images, audio and video into one vector. This page describes current source; older release archives may not include these features.

Models

ModelOutput dimensionsInput windowPooling
Snowflake Arctic Embed L v2.0 Q8_010248192CLS
all-MiniLM-L6-v2 Q8_0384512 in this GGUFMean
EmbeddingGemma 2 Q8_0 / BF16768; MRL 512 / 256 / 1288192 including special/media tokensMean + learned projection

Supported architecture keys are bert, nomic-bert and gemma-embedding2; Vulkan embedding execution is unimplemented. The BERT/XLM-R path supports pure C# cpu and native ggml_cpu / ggml_metal / ggml_cuda. EmbeddingGemma 2 additionally implements direct mlx and cuda, for six backends. Implementation availability does not establish all-six-backend multimodal quality or performance parity.

Start the service

Each process keeps one embedding encoder resident; host chat separately on another port. config/embeddinggemma2.json supplies a checksum-pinned native CPU text recipe:

dotnet run --project TensorSharp.Server.Host -- \
  --config config/embeddinggemma2.json --host 127.0.0.1 --port 5000 --no-webui

EmbeddingGemma 2 image/audio/video needs the matching media projector; omit --mmproj for text-only loading. Download the pinned files:

mkdir -p models/embeddings
curl -fL --retry 3 -o models/embeddings/embeddinggemma-2-Q8_0.gguf \
  https://huggingface.co/unsloth/embeddinggemma-2-GGUF/resolve/031f0d4b35536f69ab3509d4893c923264fcf253/embeddinggemma-2-Q8_0.gguf
curl -fL --retry 3 -o models/embeddings/embeddinggemma-2-mmproj-Q8_0.gguf \
  https://huggingface.co/unsloth/embeddinggemma-2-GGUF/resolve/031f0d4b35536f69ab3509d4893c923264fcf253/mmproj-Q8_0.gguf
dotnet run --project TensorSharp.Server.Host -- \
  --model models/embeddings/embeddinggemma-2-Q8_0.gguf --embeddings --backend ggml_metal \
  --mmproj models/embeddings/embeddinggemma-2-mmproj-Q8_0.gguf --no-webui

HTTP APIs and multimodal sequences

POST /v1/embeddings accepts strings, string arrays and token-ID arrays; token inputs must include the model's special tokens. encoding_format is float or little-endian Float32 base64. POST /api/embed accepts strings/arrays and defaults text truncate to true; OpenAI rejects overflow. With an EmbeddingGemma 2 projector loaded, both endpoints also accept one {"content":[...]} object or an array of these objects; each preserves part order and returns one vector for the complete sequence:

{
  "model": "embeddinggemma-2-Q8_0",
  "input": [{"content": [
    {"type": "text", "text": "Waterproof running shoes. "},
    {"type": "image_url", "image_url": {"url": "data:image/png;base64,..."}}
  ]}],
  "dimensions": 256
}

Media uses base64 data URIs. Audio is {"type":"input_audio","input_audio":{"data":"...","format":"wav"}} (also MP3/OGG/FLAC); video is {"type":"input_video","input_video":{"url":"data:video/mp4;base64,..."}} (also WebM). Video samples 1 fps up to 32 frames; image preprocessing follows the HF aspect-preserving 280-soft-token target. Audio decodes to mono 16 kHz. Actual media tokens share the same 8192-token window. Oversized media is rejected, not cut by the text truncation option; host codecs also constrain formats.

A request accepts up to 2048 inputs and 262144 total tokens. Embedding/model-show JSON bodies are limited to 16 MiB (HTTP 413 above that), including encoded media. --embedding-context-size N lowers the per-input limit; 0 uses the model default. Embedding processes reject generation endpoints; /v1/models, /api/tags and /api/show report their capabilities.

Retrieval contracts and measured scope

EmbeddingGemma 2 search queries use task: search result | query: and untitled documents use title: none | text: ; code retrieval and sentence similarity have their own task prefixes. The API preserves caller text without adding prefixes. dimensions crops then L2-normalizes; queries and documents must use the same model and dimensions. Snowflake queries use query: and documents remain unprefixed.

Current Q8 text evidence uses an M5 Pro/48 GiB, identical model/input bytes and sequential steady-state HTTP runs, excluding model loading: all 16 native Metal scenarios beat llama.cpp; MLX met the 5% latency margin in all 16. Independent NumPy checks cover eight exact token sequences and 8192 tokens; the first 200 STS-B development pairs were checked at four dimensions. This is a subset regression, not a full retrieval evaluation.

Default-HF media checks cover seven synthetic cases through both APIs, 22 malformed requests, temporary-file cleanup, shutdown and an independent complete pooled-audio oracle. Controlled llama.cpp media comparisons align image/video preprocessing and audio mel floor; default preprocessing differs. Controlled Metal passed seven numerical comparisons but only text met the 5% latency margin; complete direct CPU/CUDA comparisons also contain failed or slower cases. Text and component results do not establish six-backend multimodal parity.

These runs use unchanged ggml 3612c0af93f2cfc8f048295300a61187723b12ae. Missing models/devices are explicit skips, not passes. See the full embedding guide and reproduction tools for pinned assets, numerical gates, failed reports and performance limits. Generated evidence stays in ignored docs/validation/.

C# integration

Text uses EmbeddingModel.Load; multimodal input uses EmbeddingGemma2MultimodalEncoder.Load(textGguf, projectorGguf, options) with EmbeddingGemma2InputPart.Text/Image/Audio/Video for local inputs. CPU selects pure C#; other names are GGML_CPU, METAL, CUDA, MLX and GGML_CUDA; see the guide for their mappings.