Building an AI model with Ollama — from scratch, then inspected end to end

Byte-Level Bard

A 10.8-million-parameter transformer, trained from random weights on one GPU, converted into a real Ollama model, then traced all the way down — real computation graphs, real GPU kernels, real sampling probabilities, and a plain-language explainer series for all of it. Every page below is real, captured data; nothing is staged.

Start here

Byte-Level BardThe original build: tokenizer, model, training, GGUF conversion, and getting it running in Ollama.pipeline-diagram.html The Model LifecycleA map of the whole project: the AI model lifecycle as a nine-stage loop, mapped to the SDLC, linking every stage to where it actually happened.the_model_lifecycle.html What "AI" Actually Means"AI" is an umbrella term for five unrelated approaches — where this whole project's real work actually sits within it.what_ai_means.html

Plain-language series

No jargon. Best read in order — each page builds on the last, using the same real numbers throughout.

The Unlabeled DialsA real dictionary for letters, and 384 real numbers with no dictionary at all.the_unlabeled_dials.html The Next LetterHow those unlabeled numbers turn back into something readable: real odds for what comes next.the_next_letter.html Getting Less WrongWhere the AI's numbers came from: the real training loop and loss curve.getting_less_wrong.html The Whole StoryAll of the above as one continuous walk, from training text to a finished sentence.the_whole_story.html A Much Bigger MachineThe same story again, scaled to a real frontier model — real published figures, clearly cited.a_much_bigger_machine.html Look at the MathsThe real equations behind the series, each with a plain-English translation.look_at_the_maths.html The Missing IndexA complete 65-entry dictionary next to a real, un-indexed 384-number hidden state.the_missing_index.html

The reasoning experiment

Can the same machine be given a <think> span? We trained it on worked additions instead of Shakespeare, then tested whether the working is real.

Showing Its WorkingThree identical models, three training formats; accuracy, length generalisation, low-data runs, and four causal tests of whether the visible steps produce the answer.showing_its_working.html Working, Token by TokenThe scratchpad model's real Ollama response to 472+38 as a sampling tree, every digit of its working with its probability.sampling_tree_tiny_adder.html

Technical deep dives

The same claims, at full resolution — real op graphs, real GPU kernels, real sampling trees.

The JourneyOne prompt traced end to end through tiny-shakespeare: tokens, all 201 real ggml ops, softmax, sampling.the_journey.html The Larger JourneyThe same journey for deepseek-r1:8b, with an honestly-named gap where the data runs out.the_larger_journey.html Live Computation GraphEvery PyTorch module that fired during a real forward pass, via forward hooks.graph_trace.html tiny-shakespeare GraphAll 201 real ggml ops, captured by running llama.cpp's eval-callback natively in WSL.ggml_graph.html deepseek-r1:8b GraphThe same trace for the 8B reasoning model — 1,266 real ops, RoPE, SwiGLU, Q/K-norm.ggml_graph_deepseek.html GPU Kernel TraceReal CUDA kernels from one inference call, profiled with Nsight Systems, matched back to the ggml ops.gpu_trace.html Sampling Treetiny-shakespeare's real token-by-token sampling as an interactive vis-network tree.sampling_tree.html The Sheep Riddledeepseek-r1's real chain-of-thought on the classic "all but 9 die" riddle.sampling_tree_sheep_riddle.html Before the ScriptThe 560 tokens deepseek-r1 spent reasoning before writing any code.sampling_tree_before_script.html Writing the ScriptJust the 261 tokens where it actually wrote the batch file.sampling_tree_batchfile_script.html

Every page here is a self-contained static HTML file — open any of them directly, no server required. Some pages load vis-network or Google Fonts from a CDN, so they need an internet connection to render fully.