▦
1 · Make tensors concrete
Start with storage, shapes, strides, views, allocation, and operator dispatch—the small contracts every later model operation depends on.
The TensorSharp book
From Tensors to Tokens
Building a Multimodal LLM Inference Engine from Scratch with TensorSharp and Gemma 4 E4B
Go beyond calling a model API. Follow a real inference engine from the memory that holds a tensor to the code that streams the next token—and see how text, images, video, and audio meet inside one implementation.
One continuous path through the engine
The book follows a deliberately implementation-driven sequence. Each layer earns its place before the next abstraction is introduced, so the complete system stays understandable from first allocation to final response.
◫
2 · Turn model files into inputs
Read GGUF metadata and weights, understand quantization, tokenize prompts, and prepare the values that enter the network.
→
3 · Generate the next token
Trace transformer execution, logits, sampling, stopping, and streaming until a low-level forward pass becomes an interactive reply.
◉
4 · Add every modality
Follow Gemma 4 E4B as image, video, and audio encoders project their inputs into the language model's token stream.
✓
5 · Prove correctness first
Build a readable CPU reference, validate intermediate and final results, and create a trustworthy baseline before optimizing.
⚡
6 · Scale the working design
Move to accelerators, paged KV caches, continuous batching, prefix sharing, and speculative decoding without losing the original mental model.
Learn the architecture by implementing it
TensorSharp is not presented as a black box. The book connects the concepts directly to the responsibilities an inference engine must fulfill: owning memory, dispatching operators, decoding a portable model format, executing a multimodal transformer, choosing tokens, and serving results efficiently.
Gemma 4 E4B keeps that journey anchored in one concrete example. Its text, image, video, and audio path makes the boundaries between model loading, modality encoders, projection, transformer execution, and output generation visible in the same system.
📖
A companion to the source and reference docs. Read the book for the guided narrative, then use this wiki and the TensorSharp repository to inspect the current commands, APIs, architecture cards, and implementation details.
Who this book is for
.NET developers
Move from consuming AI services to understanding the C# systems underneath local inference.
ML and systems engineers
Connect model architecture to memory layout, execution, validation, acceleration, and serving.
Students and hands-on builders
Replace disconnected concepts with an end-to-end implementation you can read, run, and explore.
From the first tensor to the next token
Understand what your inference engine is actually doing
Take the implementation-driven route through TensorSharp and Gemma 4 E4B, then return to the repository ready to read the code as a complete system.