Defining terms, precisely

What "AI" Actually Means

"AI" is an umbrella term — not a technology, a label stretched over a genuinely disparate pile of techniques that share a goal (get a computer to do something that looks like it needs intelligence) but not a mechanism. Every page in this project describes exactly one branch of that umbrella. This page shows which one, and what the others look like instead.

"AI" an umbrella term Symbolic / Rule-based Search & Planning Classical Statistical ML Neural Networks Reinforcement Learning Transformers tiny-shakespeare & deepseek-r1 everything this project traced
Five branches under one label, sharing a goal but no mechanism. This project lives entirely in one narrow path down the right side.
1

Symbolic / Rule-based e.g. early chess engines, expert systems

Hand-written if this, then that logic. No training, no data, no numbers to trace — just code a person wrote directly. Everything this project showed about tokens, hidden states, and sampling is meaningless here; there's nothing of the kind inside a rule-based system.

2

Search & Planning e.g. GPS routing, A*, chess minimax

Explore possible next moves against a fixed rule and pick the best one. No learning happens at all — the "intelligence" is entirely in a search strategy someone designed in advance, not in anything discovered from data.

3

Classical Statistical ML e.g. spam filters, credit scoring

Fit a mathematical model to data using well-understood statistics — linear regression, decision trees, random forests. Often small enough that a person really can point to what one specific part is doing, unlike the 384-number mystery this project spent so long on.

4

Neural Networks where this entire project lives

Trained, not hand-coded — layers of the exact matrix multiplications traced in Look at the Maths. A huge family covering far more than language: image recognition, protein folding, self-driving perception. This project's branch inside it, specifically: transformers, the architecture behind both tiny-shakespeare and deepseek-r1.

5

Reinforcement Learning e.g. AlphaGo, robot-arm control

Learn by trial and reward instead of by labeled examples — play a game millions of times, gradually favor moves that win. Often built from neural networks underneath, but organized around a completely different training signal than the "guess the next letter, check, nudge" loop in Getting Less Wrong.

So what has this whole project actually been about?

One narrow path down one branch of a five-way tree: neural networks, specifically transformers, specifically two of them. Every real number in this project — every token id, every logit, every CUDA kernel, every missing dictionary — describes that path. None of it describes a decision tree, an A* pathfinder, or a hand-written expert system, even though every one of those is, correctly and unremarkably, also called "AI."

The word was never precise. The mechanism always was — that's what the rest of this project actually measured.

Where this fits in the project

The Model LifecycleA map of the whole project, stage by stage. Byte-Level BardThe transformer this page keeps pointing back to, built from scratch. Look at the MathsThe actual equations behind the neural-network branch specifically.