title	How AI Works
description	A beginner-to-advanced series on how modern AI models are trained, generate responses, use tools, and remember information.

Segment 1: Introductory Concepts

Written for a general audience. Nothing here requires knowing how a model works internally, and the segment is meant to be complete on its own.

Part A: AI Basics

This series will focus primarily on large language models (LLMs) and the applications built around them
This will be interactive

----- ROBOT WAVING TO PERSON, ALLOW USER TO PRESS "WAVE" BUTTON TO HAVE THE ROBOT'S HAND WAVE BACK -----

What AI Is (and Isn't)

"AI isn't universal intelligence"
- AI is a broad category of systems trained to perform different tasks
- Different models are trained for different tasks
- Image generation models are different from text generation models
- Speech recognition, recommendation, image generation, and text generation may all use different model architectures and training processes
A model is different from an AI product
- A model receives input and produces output
- Products like ChatGPT, Gemini, and Claude combine a model with application instructions, context management, memory, tools, safety systems, and a user interface
- A product that appears "general purpose" may use one multimodal model, multiple specialized models, or a combination of both
"AI doesn't 'think'" in the same way a person does
- Instead, it reacts to input based on patterns it has learned from training data
- Some models can perform additional internal computation before answering, but that does not necessarily mean they reason like a human
Training and inference are different
- Training is when a model's parameters are changed based on examples
- Inference is when a trained model is used to respond to new input
- Talking to an AI usually does not retrain its underlying model

----- SHOW A SINGLE BOX THAT SAYS "AI", ALLOW THE USER TO PRESS A BUTTON TO MOVE A SMALLER "PRODUCT" BOX OUT OF IT, AND A SMALLER "MODEL" BOX OUT OF THAT -----

How Models Learn, in Plain Terms

A model is not programmed with a rule for every situation
- It is shown examples and adjusted until it handles them well
The basic learning loop, simplified
- Show the model an example
- Let it produce a guess
- Compare the guess against the desired result
- Nudge the model so the next guess is a little closer
- Repeat across an enormous number of examples
What actually changes are the model's parameters
- Parameters are numerical values stored inside the model, like knobs it can turn
- In a robot learning to walk, parameters could control how strongly each sensor reading influences each motor movement
- They are not physical traits like the length of the robot's legs; they are values inside the software controlling it
The examples matter as much as the process
- The quality and diversity of the data influence how well the model performs
- A model can be biased based on the data it is trained on
Learning patterns is the goal, not memorizing answers
- The model needs to handle input it has never seen before
- A model that memorizes its examples can fail on unfamiliar input
Bigger is not automatically better
- More parameters give a model more capacity to represent complex patterns
- They also require more data, memory, and computing power

----- SHOW A ROBOT TRYING TO WALK. LET THE USER PRESS "TRY AGAIN" AND WATCH THE KNOBS ADJUST AND THE ROBOT IMPROVE OVER MANY ATTEMPTS -----

How a Chatbot Writes a Sentence

Large language models (LLMs) are the most common type of AI used for text generation in products like ChatGPT, Gemini, Claude, et al.
They work by predicting the next word (or piece of a word) in a sequence based on the previous words
The model is trained on a large dataset of text, learning patterns and relationships within language
Predicting one word may sound simple, but the process can be repeated to generate sentences, articles, code, conversations, and other structured text
The model writes one small piece at a time
- It looks at everything it can currently see
- It predicts which piece of text fits best next
- That piece is added to the end of what it can see
- It repeats the process for the piece after that

----- SHOW A GAME OF "GUESS THE NEXT WORD" WITH A ROBOT AND A HUMAN -----

Those small pieces are called tokens
- A token can be a whole word, part of a word, or punctuation
- "ChatGPT is great!" might be split into ["Chat", "G", "PT", " is", " great", "!"]
The model does not plan the whole answer in advance and then type it out
- Each new token is chosen with all the previous ones already visible
Generation stops when the response reaches a stopping condition
- The model produces a special "I am finished" token
- The application stops it after a length limit
Everything the model can currently see is called its "context"
- Your messages, the model's earlier replies, and instructions written by the application
- The model does not remember anything that is not in front of it

----- LET THE USER PRESS "NEXT TOKEN" REPEATEDLY AND WATCH A SENTENCE BUILD UP ONE PIECE AT A TIME UNTIL IT STOPS -----

Why AI Gets Things Wrong

The model predicts likely text; it does not look answers up in a database of truth
A response can sound confident and well written while still being incorrect
- This is commonly called a "hallucination"
The model can only work from two sources
- Patterns it picked up during training
- Information placed in front of it right now
It learns from human-produced data, which contains unfair and inaccurate patterns alongside useful ones
Fluency is not accuracy
- Well-formed writing is exactly what the model was trained to produce
- Sounding convincing and being correct are separate properties
Not every mistake comes from the model
- The application around it, a search step, or an external tool can fail as well
- "The AI was wrong" does not identify which part actually failed
Practical habits
- Verify claims that matter
- Give the model the information it needs instead of assuming it already knows
- Ask where an answer came from when the source matters

----- SHOW TWO ANSWERS TO THE SAME QUESTION, ONE CORRECT AND ONE PLAUSIBLE BUT WRONG. LET THE USER PICK THE ONE THEY TRUST BEFORE REVEALING WHICH IS WHICH -----

Part B: AI Products

Everything here lives in the software around the model, not in the model itself
Part A described the model; this part describes what a product wraps around it

Memory

The word "memory" can refer to several different systems
"Parametric" memory is information represented within the model's trained parameters
- It was learned during training
- It is not a searchable database of exact documents or facts
"Contextual" memory is information currently present in the model's context window
- It is temporary
- The model can no longer directly use it after the application removes it from context
- Context may be shorter than the full conversation, because the application may summarize or omit earlier messages
"Persistent" memory is information stored by the surrounding application
- It may be stored in a database, file, profile, or search index
- This is how an AI product can remember things about you beyond the current conversation context
When you say something important to an AI, the application may decide to store it
Storing information and retrieving information are separate operations that may use different tools connected to the same database
Memories still need to be inserted into the model's context before the model can use them
- The database may contain thousands of memories while the model sees only a selected few
Another model or rules-based system may help decide what to store, retrieve, summarize, update, or delete
- This is one possible implementation rather than a requirement of memory systems

----- LET THE USER TYPE INTO A BOX, THEN PRESS A "REMEMBER" BUTTON. CLEAR THE CONVERSATION, SHOW THE DATABASE STILL CONTAINING THE MEMORY, THEN LET THE APPLICATION RETRIEVE IT BACK INTO CONTEXT -----

Retrieval and RAG

A trained model's parameters are not the only source of information an AI product can use
Retrieval-augmented generation (RAG) allows an application to search an external source before asking the model to answer
A basic retrieval flow is:
- Receive the user's question
- Search documents, databases, or the web for relevant information
- Add selected results to the model's context
- Ask the model to generate an answer using that information
Embeddings can be used to search for text with similar meaning rather than only exact matching words
Retrieval can give a model access to private or newly updated information without retraining the entire model
Retrieved information still consumes context-window space
Retrieval does not guarantee a correct answer
- Search may return the wrong information
- Useful information may be omitted
- The model may misunderstand or ignore the retrieved material

----- GIVE THE USER A SMALL LIBRARY OF DOCUMENTS AND A QUESTION. SHOW EXACT-WORD SEARCH VERSUS EMBEDDING-BASED SEARCH, THEN INSERT THE SELECTED RESULT INTO CONTEXT -----

Tool Calling

Tool calling allows an LLM to request that the surrounding application run code
The code is not usually executed by the model itself
A basic tool-calling flow is:
- The application tells the model which tools exist and what inputs they accept
- The model asks for one of those tools by name and supplies the inputs to use
- The application checks the request
- The application executes the code or API
- The result is returned to the model as additional context
- The model continues generating its response
Tools can be defined directly by the application, provided by the model platform, or exposed through protocols such as MCP
The LLM can input values to the tool and receive values back from it
Tool access does not mean the model can run any code it wants
- The surrounding application controls which tools exist and which actions are permitted

----- HAVE A CANVAS AND A TIMELINE OF TOKENS, THEN A "DRAW PICTURE" TOOL REQUEST. SHOW THE REQUEST CROSSING INTO THE APPLICATION, CODE DRAWING A HEART, AND THE RESULT RETURNING TO THE MODEL -----

Multimodal Input and Output

Some models can process more than text
- Images
- Audio
- Video
- Other structured sensor input
A multimodal model converts each supported kind of input into numerical representations that can be processed together
Some AI products use one multimodal model, while others route each kind of input to a separate specialized model
Output may also be produced in different ways
- A model may directly generate audio or image representations
- A language model may call a separate image, speech, or video tool

----- SHOW THE SAME OBJECT AS TEXT, AN IMAGE, AND AUDIO. LET THE USER SEND EACH TO THE SYSTEM AND SHOW WHETHER ONE MODEL OR MULTIPLE SPECIALIZED MODELS HANDLE IT -----

Reasoning Models

Some models are trained to perform more internal work before producing a final answer
- They may break a problem into parts
- Compare possible approaches
- Check intermediate results
- Use tools before responding
Products often describe these as "reasoning" or "thinking" models
Reasoning models do not require subagents
- The additional work may happen within one model invocation
Not all models show all of their internal work, even when requested
- A visible explanation may be a summary rather than a literal transcript of every internal computation
More internal work does not guarantee the answer is correct

----- CLICK "ASK" WITH A PREDEFINED QUESTION. SHOW A FAST MODEL ANSWERING IMMEDIATELY AND A REASONING MODEL SPENDING ADDITIONAL COMPUTE BEFORE REVEALING A FINAL ANSWER AND A SHORT REASONING SUMMARY -----

Agents

An agent is an application that allows a model to repeatedly choose and perform actions while working toward a goal
A basic agent loop is:
- Receive a goal and the current state
- Decide what action to take next
- Call a tool or produce another action
- Observe the result
- Update the state
- Decide whether to continue or stop
The important difference is that the model can react to the result of each action rather than generating the entire sequence in advance
Agents often need limits and stopping conditions
- Maximum number of steps
- Time or cost limits
- Detection of repeated actions
- A way to ask a person for help
- A way to declare the task complete
An agent can fail by choosing a poor action, misunderstanding an observation, repeatedly looping, or stopping too early

----- GIVE THE AGENT A SMALL MAZE OR SET OF CLOSED BOXES. AFTER EACH ACTION, REVEAL NEW INFORMATION AND LET THE AGENT CHOOSE WHAT TO DO NEXT UNTIL IT REACHES THE GOAL -----

Subagents

A parent agent may delegate part of a task to another agent
These delegated agents are often called subagents
A subagent may receive:
- A smaller goal
- Its own instructions
- Only the context needed for its task
- A limited set of tools
Subagents can run one at a time or in parallel
The parent agent can collect their results and decide what to do next
Some systems use a handoff instead
- A handoff transfers control to another specialized agent rather than returning a result to the original agent
Subagents are useful when a task contains separate kinds of work, but they are not required for an AI system to be considered an agent

----- "SHOULD WE HOLD THE OUTDOOR EVENT TOMORROW?" QUERY SPAWNS A WEATHER SUBAGENT, CALENDAR SUBAGENT, AND VENUE SUBAGENT. THE PARENT COLLECTS THEIR RESULTS AND PRODUCES ONE DECISION -----

Part C: AI Usage

Parts A and B described what the system is; this part is about the levers a normal user actually has

Prompting

The model responds to what you wrote, not to what you meant
A useful prompt usually includes:
- The goal, stated plainly
- Any material the model needs, pasted or attached
- The format you want back
- Constraints such as length, audience, or tone
- An example of a good answer, when the shape of it matters
Asking for a revision is usually better than starting over when an answer is close
Long threads accumulate everything said earlier, including mistakes
- A fresh conversation can help when a thread has gone off track
There is no magic phrasing
- Clear, specific, complete instructions beat "prompt tricks"

----- LET THE USER SEND A VAGUE PROMPT AND A SPECIFIC PROMPT TO THE SAME ROBOT AND COMPARE THE TWO ANSWERS SIDE BY SIDE -----

Choosing a Model

Most products offer more than one model
The usual trade-offs are:
- Speed against depth
- Cost against capability
- How much context it can hold
- Whether it accepts images, audio, or files
- Whether it can use tools or search
- Whether it does extra internal work before answering
The largest or newest model is not automatically the best choice
- A fast model is often better for short, well-defined work
The same task can behave differently across products even at similar capability

----- GIVE THE USER A TASK AND A SET OF MODEL CARDS. LET THEM PICK ONE AND SHOW THE RESULTING SPEED, COST, AND ANSWER -----

Custom Instructions and Skills

Most products let you save standing instructions that apply to every conversation
- Who you are, what you are working on, how you want answers formatted
A "skill" is a reusable bundle of instructions, and sometimes files, that the product loads when it looks relevant
None of this retrains the model
- It is inserted into the context before the model answers
Saved instructions and skills consume context-window space like everything else
Too many standing instructions can contradict each other or crowd out your actual question

----- SHOW A CONVERSATION WITH AND WITHOUT SAVED INSTRUCTIONS, THEN REVEAL THE HIDDEN TEXT THAT WAS ADDED TO THE CONTEXT -----

Connecting Tools and MCP

Part B described tool calling from the model's side; this is the same idea from yours
Products expose these under names like connectors, extensions, plugins, or integrations
MCP (Model Context Protocol) is an open standard for exposing tools and data to AI applications
- One connector can then be reused across any product that supports it
Every connection you add carries three things at once
- Capability gained
- Context space consumed by the tool descriptions
- Access granted on your behalf
Review what a connected tool is allowed to read and change
Anything the model reads can contain instructions aimed at the model
- A web page, document, or email can try to redirect it
- This is called prompt injection, and it is why permissions matter more than trusting the model

----- LET THE USER CONNECT AND DISCONNECT TOOLS FOR AN ASSISTANT, THEN SHOW WHICH REQUESTS IT CAN AND CANNOT COMPLETE -----

Knowing When Not to Use It

The riskiest tasks are the ones where you cannot check the answer
Confidential or regulated information may leave your control depending on the product
High-stakes decisions still need a person who is accountable for them
Speed is not a win if verifying the output takes longer than doing the work

----- PRESENT A SET OF TASKS AND LET THE USER SORT THEM INTO "GOOD FIT", "NEEDS REVIEW", AND "DON'T" -----

Segment 2: Intermediate Explainers

For readers who want to know why the behavior in Segment 1 happens. Useful on its own, but not required to use these tools well.

Part D: Components of an LLM

Part A described what an LLM does
This part is about the pieces that do it

Tokens and Tokenizers

Part A described an LLM as predicting the next piece of text
This section is about what those pieces actually are
A token is a unit of text that the model processes at a time
- It can be a word, part of a word, or even punctuation
- For example, one tokenizer might split "ChatGPT is great!" into the tokens ["Chat", "G", "PT", " is", " great", "!"]
Each model may break up text into tokens differently, which can affect how it understands and generates text
- For example, some models may treat "ChatGPT" as one token, while others may split it into multiple tokens
- The tool that breaks text into tokens is called a tokenizer
- Different tokenizers can lead to different model behaviors, even when trained on similar text
The model's vocabulary contains every token it knows how to accept and generate
Because the model is built around its token vocabulary, replacing its tokenizer usually requires retraining or substantially adapting the model
Tokens can represent things that are not ordinary visible text
- Models may use reserved tokens for message boundaries, tool calls, image placeholders, audio, or other internal control information

----- HAVE A "TRANSFORM TO TOKENS" BUTTON THAT CONVERTS TEXT TO TOKENS VISUALLY -----

Embeddings

The model cannot directly process a token like "robot" as written text
Each token is first converted into a list of numbers called an embedding
- These numbers give the model a mathematical representation it can process
- The embedding is learned during training rather than manually defined by a person
Tokens used in similar ways may develop related numerical representations
- "Cat" and "dog" may become more closely related than "cat" and "volcano"
The same token can take on a different contextual representation after it is processed alongside other tokens
- "Bank" in "river bank" should be treated differently from "bank account"
An embedding does not contain a neat human-readable definition
- Meaning is distributed across many numerical dimensions and transformed throughout the model

----- SHOW WORDS AS DOTS ON A MAP. LET THE USER TRAIN THE MODEL AND WATCH RELATED WORDS MOVE CLOSER TOGETHER. THEN SHOW THE SAME WORD MOVE DIFFERENTLY BASED ON ITS SENTENCE -----

Next-Token Prediction

Between the embeddings and the prediction, the model runs the whole context through many repeated processing layers
- This architecture is called a transformer
- Part G covers what happens inside those layers
For now, treat that step as a box that turns the current context into a set of scores
Once the model has processed the current context, it produces a score for every token that could come next
- These raw scores are often called logits
- The scores are converted into probabilities
The application then selects one of the possible tokens
- It may simply take the highest-scoring token
- It may pick from among the strong candidates with some controlled randomness
- Part H covers the settings that control this choice
That token is added to the end of the context
The model processes the updated context and predicts another token
This loop repeats until the response reaches a stopping condition
- The model may generate a special end token
- The application may stop it after a length limit
- A tool call or another structured output may temporarily interrupt ordinary text generation

----- SHOW THE FULL LOOP: CONTEXT -> TOKEN SCORES -> SELECTED TOKEN -> UPDATED CONTEXT. LET THE USER ADVANCE ONE TOKEN AT A TIME -----

The Context Window

The context window is the number of tokens the model can manage at once
The context contains the temporary information available to the model while generating its next response
- The user's visible messages
- Previous model responses
- Application instructions
- Tool definitions and tool results
- Retrieved documents or memories
- Any output space reserved by the application
For example, if a model has a context window of 2,048 tokens, the total supplied input and generated output must fit within that limit
The model does not independently choose which entire conversation to remember
- The surrounding application decides which information is placed into the context
If the limit is exceeded, AI usage may degrade or halt
- The application may reject the request
- It may remove older information
- It may summarize earlier information
- It may retrieve only information that appears relevant
Some AI applications "compact" conversations when they approach their limits before continuing
- Compaction creates a shorter representation of earlier conversation state
- Like a synopsis of everything talked about earlier
- Summaries can omit details, so compaction is not identical to preserving the original messages

----- ALLOW THE USER TO PRESS BUTTONS TO TYPE, THEN RUN OUT OF CONTEXT. LET THEM CHOOSE BETWEEN "STOP", "REMOVE OLDEST", "SUMMARIZE", AND "RETRIEVE RELEVANT" -----

Part E: How Models Learn

Part A described training as showing the model examples and nudging it toward better guesses
This part covers what each step in that loop actually is

Training Data

Training data is crucial
- The quality and diversity of the data influence how well the model performs
- AI can be biased based on the data it is trained on
- Bias can also be introduced by how data is selected, filtered, labeled, and evaluated
Training data is usually split into multiple groups
- A training set is used to adjust the model's parameters
- A validation set is used to check how well the model performs while it is being developed
- A test set is used to evaluate the completed model using examples it was not directly trained on
A model should learn patterns that work beyond the examples it has already seen
- If it memorizes the training data without learning useful patterns, it may perform poorly on unfamiliar input
- This is called overfitting

----- ALLOW THE USER TO SORT EXAMPLES INTO "TRAINING", "VALIDATION", AND "TEST" BOXES, THEN SHOW WHAT HAPPENS IF THE SAME EXAMPLES ARE USED FOR EVERYTHING -----

Parameters

Parameters are the "knobs" that the model can adjust to do its task better
- Parameters are numerical values stored inside the model
- In a robot trying to walk, parameters could control how strongly each sensor reading influences each motor movement
- They are not usually physical traits like the length of the robot's legs; they are values inside the software controlling it
A model learns by changing its parameters over time
- It may begin with parameters that are random or otherwise unhelpful
- Training gradually adjusts them so the model produces better output
The more parameters a model has, the more capacity it has to represent complex patterns
- More parameters do not automatically make a model better
- Training data, architecture, and the training process also determine how useful the model becomes
More parameters usually require more data, memory, and computational resources to train and run

---- SHOW A VISUAL OF A ROBOT WHOSE SENSOR-TO-MOTOR CONNECTIONS HAVE KNOBS AND DIALS, BUT ELECTRICAL DRAW INCREASES AS MORE CONNECTIONS ARE ADDED ----

Loss and Optimization

The model needs a way to measure how well or poorly it performed
- This measurement is called loss
- A lower loss means the model's output was closer to the desired result
During training, the model repeatedly:
- Receives an input
- Produces a prediction
- Compares the prediction against the expected result
- Calculates the loss
- Adjusts its parameters to reduce similar errors in the future
The model works backward from its mistake to estimate which adjustable values played a part in it (backpropagation)
It then uses a set of rules to decide how each value should change (optimizer)
- One common approach is to make many small adjustments in whichever direction seems to reduce the mistake (gradient descent)
The model repeats this process across an enormous number of examples, gradually making better predictions

----- SHOW A SIMPLE MODEL WITH ADJUSTABLE KNOBS, A TARGET OUTPUT, AND A LOSS METER. LET THE USER PRESS "TRAIN" REPEATEDLY AND WATCH THE LOSS FALL -----

Types of Learning

The training loop above needs a source for the "desired result"
Where that answer comes from is what separates the main styles of learning

Supervised Learning

In supervised learning, training examples include the desired result
- For example, an image may be labeled "cat" or "dog"
- The model predicts an answer and compares it against the provided label
The labels may be produced by people, existing systems, or another process

----- LET THE USER LABEL A FEW SHAPES, THEN SHOW A MODEL ATTEMPTING TO LABEL A NEW SHAPE -----

Self-Supervised Learning

In self-supervised learning, the desired result is created from the data itself
- Instead of requiring a person to label every example, part of an example can be hidden and used as the answer
- A language model can be given the beginning of a sentence and trained to predict the token that comes next
This allows a model to learn from extremely large collections of otherwise unlabeled text
Most of the initial language ability in a modern LLM is learned this way

----- SHOW A SENTENCE WITH THE FINAL WORD HIDDEN, THEN LET THE USER REVEAL IT AND SCORE THE MODEL'S PREDICTION -----

Reinforcement Learning

Consider a robot you want to train to walk
You can give it a reward for taking a step forward and a penalty for falling down
Over time, the robot learns behavior that increases its expected rewards and avoids penalties
The reward does not directly tell the robot which parameter to change
- The learning algorithm must determine which behavior contributed to the result
It is often trained using many copies of the robot in a simulated environment to gather experience faster
- These robots do not have to compete against one another
Reinforcement learning can also be used during later stages of language-model training to encourage responses that people or automated evaluators prefer

----- TIME TRAVEL SLIDER TO SEE GENERATIONS OF ROBOTS LEARNING TO WALK, WITH MANY SIMULATED ROBOTS TRAINING IN PARALLEL -----

Part F: Coding with LLMs

Coding is the most common professional use of these tools, and it exercises every piece from Part B at once
It is also where the surrounding software often matters more than the model

Where the Code Runs

A coding assistant does not run code inside the model
It runs somewhere concrete: your machine, a container, or a remote sandbox
A sandbox limits what generated code can reach
- Filesystem access
- Network access
- Credentials and environment variables
Generated code, and anything the assistant fetched, is untrusted until reviewed
Version control is the practical safety net
- Small commits make a bad change easy to throw away

----- SHOW THE SAME GENERATED SCRIPT RUNNING INSIDE AND OUTSIDE A SANDBOX, AND WHAT EACH ONE IS ABLE TO TOUCH -----

Structured Output

Part B said the model can ask the application to run a tool; this is the mechanism underneath that
An LLM does not have to produce ordinary prose
It can be trained or constrained to produce data in a predictable structure
- JSON objects
- Lists of values
- A fixed set of labels
- Arguments for a function
A tool call is just structured output the application knows how to act on
The surrounding application can validate the generated structure before using it
- A schema can reject a malformed or unexpected response before any code runs
Structured output is one of the basic bridges between language generation and ordinary software

----- LET THE USER TYPE A NATURAL-LANGUAGE REQUEST, THEN TRANSFORM IT INTO A VISIBLE JSON OBJECT THAT AN APPLICATION CAN VALIDATE -----

Giving the Model the Right Code

A repository is far larger than any context window
The assistant has to choose what to look at
- Search across the codebase
- Files you attach explicitly
- Files it opened while working
Project instruction files let you record conventions once instead of repeating them every session
Edits are usually returned as structured changes rather than whole rewritten files
- Smaller diffs are cheaper to generate and easier to review

----- SHOW A REPOSITORY WHERE ONLY A FEW FILES FIT IN CONTEXT. LET THE USER CHOOSE WHICH ONES TO INCLUDE AND SEE WHETHER THE ANSWER IS CORRECT -----

The Agentic Coding Loop

Compilers, tests, and linters give an agent something objective to react to
A typical loop is:
- Make a change
- Run the build or the tests
- Read the output
- Fix the next failure
This is why tooling quality matters more than prompt wording
- A project with fast, reliable tests is a project an agent can work in
The limits from Part B still apply: step counts, cost ceilings, and a person reviewing before anything ships

----- SHOW AN AGENT FIXING A FAILING TEST SUITE ONE ERROR AT A TIME, WITH THE TEST OUTPUT VISIBLE AFTER EACH STEP -----

Evals

"It looked right once" is not evidence that a change helped
An eval is a repeatable set of examples plus a way to score the output
Common scoring methods:
- Exact or pattern matching for well-defined answers
- Unit tests for generated code
- Human review for judgment calls
- Another model as a grader, with its own failure modes
Evals let you compare prompts, models, and settings instead of guessing
Tuning against a small eval set carries the same risk as overfitting in Part E
- Keep examples you never tune against

----- LET THE USER CHANGE A PROMPT AND WATCH THE SCORE MOVE ACROSS A WHOLE SET OF TEST CASES INSTEAD OF A SINGLE EXAMPLE -----

Secrets and Untrusted Content

Credentials pasted into a prompt become part of the context and may be stored
Prefer environment variables and short-lived tokens the assistant never sees
Issue text, web pages, and dependency files can all contain instructions aimed at the model
Treat anything the assistant read as data, not as orders

----- SHOW A TICKET CONTAINING HIDDEN INSTRUCTIONS AND LET THE USER DECIDE WHETHER THE AGENT SHOULD FOLLOW THEM -----

Segment 3: Advanced Internals

The mechanics behind the boxes the earlier segments skipped over.

Part G: Inside a Transformer Layer

Part D treated the middle of the model as a box that turns a context into token scores
This part opens that box
Most modern LLMs use an architecture called a transformer
The one job of everything in this part is to work out what each token means here, in this specific context, rather than what it means in general
Once the most recent token carries that, guessing the next token is comparatively easy
Part H covers that final guess and how one token gets chosen

What Moves Through the Layers

The model cannot revise a word; it can only revise numbers
Part D described each token being converted into an embedding: a list of numbers
That starting embedding says what a token means on its own, in isolation
- "Bank" arrives with the same numbers whether the sentence is about a river or a loan
- "It" arrives with almost no meaning at all, because its meaning is entirely borrowed from elsewhere
The transformer's job is to revise those numbers until they describe the token in context instead of in isolation
- Part D said "bank" should end up treated differently in "river bank" than in "bank account"; this part is where that actually happens
That revision is what the rest of this part is doing, so it is worth knowing the shape it happens in
Every token position in the context gets its own list, and every one of those lists is the same length
- That length is the model's hidden size, and it is fixed for a given model
- A model might use a few thousand numbers per token position
Every layer of the transformer takes in one list per position and hands back one list per position, at the same length
- Nothing gets longer or shorter in the middle of the model
- Each layer only edits the numbers in place
Because nothing about that arrangement changes, layers can be stacked as many times as the model designer wants
- Small models may stack a couple dozen layers; large ones stack a hundred or more
The running list at one token position is often called that position's residual stream
- Each layer reads from the residual stream and adds its result back into it
- Think of it as a scratchpad per token that every layer writes notes onto
The entire context is processed at once, not one position at a time
- Every position's scratchpad is updated by every layer, even the positions the model has already moved past

----- SHOW "I SAT DOWN BY THE RIVER BANK" WITH ONE ROW OF NUMBERS UNDER EACH WORD. LET THE USER STEP THROUGH THE LAYERS AND WATCH THE ROW UNDER "BANK" DRIFT AWAY FROM WHERE IT STARTED. THEN LET THEM SWAP THE SENTENCE FOR "I OPENED A BANK ACCOUNT" AND WATCH THE SAME WORD DRIFT SOMEWHERE ELSE ENTIRELY. THE NUMBER OF ROWS AND THE LENGTH OF EACH ROW NEVER CHANGE -----

What One Layer Does

No single pass can work out everything a token means in context
- Working out what "it" refers to can depend on having already worked out what the clause before it is describing
- Some conclusions are only reachable once other conclusions are available to build on
A layer is one round of that revision, and stacking layers lets each round build on the last
- Early layers tend to settle local matters like word sense and grammatical role
- Later layers tend to handle longer-range and more abstract relationships
- These tendencies are what researchers observe, not rules the architecture enforces
Each round needs to do two different things, which is why a layer has two sublayers
- Pull in whatever this position needs from the other positions
- Do something useful with what the position is now holding
A transformer layer is therefore made of two sublayers, run one after the other
- An attention sublayer, where token positions exchange information
- A feed-forward sublayer, where each token position is processed on its own
Attention is the only place where information moves between token positions
- Everything else in the model treats each position independently
Each sublayer is wrapped the same way
- Normalize the incoming numbers so they sit in a predictable range
- Run the sublayer
- Add the result back onto what came in, rather than replacing it
That "add back onto what came in" step is called a residual connection
- It is what makes a layer a revision rather than a replacement, so nothing established by earlier layers is thrown away
- It means a layer that has nothing useful to contribute can add close to nothing and leave the state intact
- It also gives training a short path from the output back to the early layers, which is what makes very deep stacks trainable at all
Normalization keeps the numbers from growing or shrinking out of control as they pass through a hundred layers
The next several sections take these two sublayers apart in turn

----- WALK THROUGH A SINGLE LAYER AS SIX PLAIN-LANGUAGE STEPS: TIDY THE NUMBERS, LOOK AROUND AT THE OTHER WORDS, ADD WHAT YOU FOUND, TIDY AGAIN, THINK IT OVER ON YOUR OWN, ADD THAT TOO. GIVE THE USER A SWITCH LABELED "REPLACE INSTEAD OF ADD" AND LET THEM WATCH EVERY EARLIER LAYER'S WORK GET WIPED OUT THE MOMENT THEY FLIP IT -----

Attention, Step by Step

Attention answers one question for each token position: "which other positions should I pull information from, and how much of each?"
Each position starts by producing three different vectors from its own residual stream
- A query: what this position is looking for
- A key: what this position offers to others that are looking
- A value: the information this position will actually hand over
These three are produced by multiplying the position's numbers by three separate learned weight matrices
- Those matrices are part of the model's parameters, learned during training as described in Part E
The model then compares one position's query against every position's key
- A high comparison score means "this other position looks relevant to me"
- The scores are scaled down based on vector size to keep them from getting extreme
- During text generation, positions are additionally blocked from looking at later positions; Part H covers that restriction
The scores are converted into weights that add up to 1
- This is the same softmax step Part D used to turn logits into probabilities
- A position can spread its attention across many positions or concentrate it on one
Each position's output is the weighted sum of every position's value vector
- Positions with high weights contribute most of the result
- That result is added back into the residual stream
Worked example: "The animal did not cross the street because it was tired"
- The query from "it" is compared against the keys from every earlier word
- "Animal" produces a high score, so most of the value pulled in comes from "animal"
- The position for "it" now carries information about the animal, without the word "animal" ever being copied
Change one word: "The animal did not cross the street because it was crowded"
- The same query-and-key machinery now scores "street" higher instead
- Nothing about the model changed; only the input did

----- PUT "THE ANIMAL DID NOT CROSS THE STREET BECAUSE IT WAS TIRED" ON SCREEN AND LET THE USER CLICK "IT". SHOW IT HOLDING UP A QUESTION WHILE EVERY OTHER WORD HOLDS UP AN ANSWER, AND LET THE BEST MATCHES GLOW BRIGHTEST. THEN LET THE USER SWAP "TIRED" FOR "CROWDED" AND WATCH THE GLOW JUMP FROM "ANIMAL" OVER TO "STREET" WITHOUT ANYTHING ABOUT THE MODEL CHANGING -----

Multiple Attention Heads

One set of query, key, and value matrices can only express one notion of relevance at a time
Transformers run several attention calculations side by side in the same sublayer
- Each one is called a head, and each has its own query, key, and value matrices
- A layer might have 32 or more heads
The hidden size is split across the heads, so each head works with a smaller slice of the numbers
- Running 32 heads does not cost 32 times as much as running one
Different heads tend to specialize during training
- One might track which noun a pronoun refers to
- One might track the nearest preceding punctuation
- One might track matching brackets or quotation marks
- Many do not correspond to anything a person would name
The heads' outputs are concatenated back together and passed through one more learned matrix before rejoining the residual stream
Because heads are independent, "what is the model attending to here?" rarely has a single answer

----- KEEP THE SAME SENTENCE, BUT ADD A ROW OF NUMBERED HEADS THE USER CAN FLIP BETWEEN. ONE TRACKS WHAT THE PRONOUN POINTS AT, ONE ONLY EVER LOOKS AT THE WORD IMMEDIATELY BEFORE IT, ONE PAIRS UP QUOTATION MARKS, AND SEVERAL LOOK LIKE NOTHING A PERSON WOULD HAVE A NAME FOR. MAKE IT CLEAR THAT NONE OF THEM IS THE ONE TRUE ANSWER -----

Why the Model Needs Position Information

The attention math described above has no sense of order
- It compares every query to every key, and nothing in that comparison says which token came first
- Without a fix, "the dog bit the man" and "the man bit the dog" would look identical to the model
So the model injects position information into the numbers themselves
Early transformers added a fixed position pattern to the embeddings before the first layer
Most current models instead rotate the query and key vectors by an amount based on their position
- This is usually called rotary position embedding, or RoPE
- The comparison between two positions then depends on how far apart they are, not just what they contain
This is also why extending a model past the context length it was trained on is not free
- The model has never seen position values that large
- Techniques exist to stretch position handling, but they trade off accuracy

----- SHOW "THE DOG BIT THE MAN" AND "THE MAN BIT THE DOG" SIDE BY SIDE WITH POSITION INFORMATION SWITCHED OFF, SO THE MODEL SEES ONE INDISTINGUISHABLE PILE OF WORDS FOR BOTH. LET THE USER SWITCH IT BACK ON AND WATCH THE TWO SENTENCES COME APART -----

The Feed-Forward Sublayer

After attention has moved information between positions, each position is processed on its own
The feed-forward sublayer expands each position's numbers to a much larger size, applies a non-linear function, then projects back down
- The expansion is often four times the hidden size
- The non-linear step is what lets the model represent relationships that are not just weighted sums
This sublayer holds the majority of the model's parameters in most transformers
- Attention gets most of the attention, but feed-forward layers are where most of the weights live
It is often described as the place where learned associations are stored and retrieved
- Attention decides what the current position is about; the feed-forward step brings in what the model knows about it
- This is a useful intuition rather than a strict architectural fact
Some larger models use a mixture-of-experts variant
- Many separate feed-forward blocks exist, but only a few run for any given token
- This raises the total parameter count without raising the cost of processing each token by the same amount
- It is one reason a model's advertised parameter count may not match how expensive it is to run

----- TAKE ONE WORD'S ROW OF NUMBERS, FAN IT OUT INTO A MUCH WIDER STRIP, RESHAPE IT, THEN FOLD IT BACK DOWN TO EXACTLY THE SIZE IT STARTED AT. ADD A "MIXTURE OF EXPERTS" SWITCH THAT SWAPS THE ONE WIDE STRIP FOR A ROW OF SPECIALISTS, WITH ONLY TWO OF THEM LIGHTING UP FOR ANY GIVEN WORD -----

Reading Attention Maps Carefully

Attention weights are easy to visualize, which makes them tempting to treat as an explanation
A single head's weights are only one of thousands of such calculations happening across the model
High attention weight means information was pulled from a position, not that the position caused the answer
Information also arrives through the residual stream from layers that ran earlier
Attention diagrams, including the ones in this part, are simplifications chosen for legibility
- They are good for building intuition and poor for proving why a model said something specific

Part H: Producing the Next Token

Part G ended with a stack of layers that has turned the context into a context-aware representation of every token position
This part covers the rest of the trip: turning that representation into one chosen token, then going around again
It also covers why that loop costs what it does

Looking Backward Only

In a text-generating model, each position is only allowed to attend to itself and to earlier positions
- This is called causal masking, and it is implemented by forcing the attention scores for later positions to zero weight
Without the mask, predicting the next token would be trivial, because the model could look at the answer
One useful consequence: the work done for a token position never changes once that position exists
- Its keys and values do not depend on anything that comes after it
Applications exploit this with a KV cache, storing each position's keys and values so they are not recomputed for every new token
- This is why generating the first token of a response is often slower than generating the rest
- It is also why a long context costs memory even when the model is idle mid-response

----- SHOW A TABLE OF WHICH WORDS ARE ALLOWED TO SEE WHICH OTHER WORDS, WITH EVERYTHING ABOVE THE DIAGONAL GREYED OUT AS OFF-LIMITS. LET THE USER GENERATE A FEW TOKENS AND WATCH THE ALREADY-FILLED CELLS GET REUSED RATHER THAN REDONE, WITH A RUNNING TALLY OF THE WORK THE CACHE SAVED -----

From the Last Layer to Token Scores

After the final layer, every position still has its own list of numbers, but those numbers now reflect the full context
To predict the next token, only the last position's list is used
- Every other position was processed too, but during generation their results are not what gets turned into the next token
Those numbers are normalized one last time, then multiplied by a matrix with one row per token in the vocabulary
- The result is one score per possible next token
- Those scores are the logits Part D mentioned
This matrix is frequently tied to the input embedding matrix, reusing the same learned weights in both directions
At this point the model's work is done; everything that follows is the application's choice

----- TAKE THE LAST WORD'S ROW OF NUMBERS AND RACE IT AGAINST EVERY WORD THE MODEL KNOWS, FINISHING WITH A LEADERBOARD OF THE TOP HANDFUL AND THEIR SCORES. LET THE USER SCROLL DOWN THE LEADERBOARD TO SEE HOW QUICKLY THE REST FALL AWAY -----

Decoding Strategies

Part D ended the generation loop with the application choosing one token out of many scored candidates
This section covers how that choice is actually configured
Tokens are ranked by how likely they are to appear next, then selected using a decoding strategy
Greedy decoding always selects the most likely remaining token
Sampling selects from a probability distribution, introducing controlled randomness
Temperature controls the size of the gaps between more-preferred and less-preferred tokens
- A lower temperature makes the most likely tokens dominate more strongly
- A temperature of 1 keeps the model's original relative probability distribution
- A temperature above 1 makes lower-ranked tokens more competitive
- Very low temperature usually gives more consistent or repetitive results, while high temperature gives more varied and error-prone results
- A temperature of 0 is often treated like greedy decoding, but does not assure true determinism across every system

----- GIVE THE USER A HALF-FINISHED SENTENCE, A TEMPERATURE DIAL, AND A "WRITE IT AGAIN" BUTTON THEY CAN PRESS AS MANY TIMES AS THEY LIKE. AT THE LOW END THE SAME ENDING COMES BACK EVERY SINGLE TIME. IN THE MIDDLE IT VARIES BUT STAYS SENSIBLE. AT THE HIGH END IT STOPS MAKING SENSE ALTOGETHER -----

Sampling Filters

Sampling filters allow the application to change which tokens are allowed to remain in the lottery system
- There are various strategies like top-k, top-p, and min-p
- top-k: Keep only the k most probable tokens, then sample among them. top_k: 2 only allows the top two most likely tokens to remain
- top-p: Keep the smallest group of tokens whose cumulative probability reaches p. top_p: 0.85 only allows the tokens that add up to at least 85% probability to remain
- min-p: Keep tokens whose probability is not too small relative to the best token. With min_p: 0.2, a token must have at least 20% of the probability of the most likely token. If the most likely token is 60% likely, another token must have 12% or more to stay because 60% × 0.2 = 12%
Multiple filters may be combined, and the exact order or behavior can differ between model runtimes
None of these settings add knowledge; they only change which of the model's own candidates survive

----- SHOW THE LEADERBOARD OF CANDIDATE WORDS AGAIN, THIS TIME WITH TOP-K, TOP-P, AND MIN-P AS THREE SWITCHES. AS THE USER FLIPS EACH ONE, CROSS OFF THE CANDIDATES IT ELIMINATES AND WRITE THE REASON BESIDE EACH CASUALTY. LET THEM STACK THE FILTERS AND FIND THE POINT WHERE ONLY ONE WORD IS LEFT STANDING -----

What Generation Costs

Every selected token sends the whole updated context back through every layer, so all of Part G's machinery runs once per token
Attention compares every position against every earlier position
- Doubling the context length roughly quadruples that comparison work
- This is the main reason long context windows are expensive rather than free
The feed-forward cost grows in step with the number of tokens, not with the square of it
- For short contexts, feed-forward dominates; for very long ones, attention does
The KV cache grows with context length, number of layers, and number of heads
- Long conversations occupy memory on the serving hardware for their whole lifetime
A great deal of engineering exists to reduce these costs
- Sharing keys and values across groups of heads
- Attention implementations that avoid writing the full score grid to memory
- Approximate or sparse attention patterns that skip most position pairs
Part D said the application decides what goes into the context; this is the bill it is managing

----- GIVE THE USER A SLIDER FOR HOW LONG THE CONVERSATION HAS GOTTEN, ALONGSIDE THREE METERS: TIME SPENT COMPARING WORDS AGAINST EACH OTHER, TIME SPENT THINKING ABOUT EACH WORD ON ITS OWN, AND MEMORY HELD OPEN FOR THE CONVERSATION. DOUBLING THE LENGTH SHOULD SEND THE FIRST METER UP FAR FASTER THAN THE SECOND -----

Part I: Fine-tuning and RLHF

Part E covered how a model is trained from nothing
This part covers what is done to an already-trained model to make it better at a task or better behaved

Base Models and Assistant Models

Predicting the next token teaches a model a great deal about language, but it does not automatically make the model a helpful assistant
A model trained only on broad text is often called a base model
- A base model may continue text rather than directly answer a request
Later training stages can teach the model to follow instructions and behave more like an assistant
- Supervised examples can show it how a helpful response should look
- Preference training can encourage responses that people or automated evaluators rank more highly
- Reinforcement learning is one possible preference-training method
Product-level instructions and safety systems can further shape how the trained model is used without changing all of its underlying parameters
The rest of this part covers each of those stages

----- GIVE THE SAME INPUT TO A SIMPLIFIED "BASE MODEL" AND "ASSISTANT MODEL." HAVE ONE CONTINUE THE TEXT AND THE OTHER ANSWER THE REQUEST -----

When Fine-tuning Is the Right Answer

Prompting, retrieval, and tools change what the model sees
Fine-tuning changes the model's parameters
Fine-tuning is a reasonable fit for:
- A consistent output format or style
- A narrow domain with its own vocabulary
- Shrinking a long prompt you would otherwise repeat every time
It is a poor fit for:
- Facts that change over time
- Anything retrieval already solves
A fine-tuned model is also a model you now have to maintain, re-evaluate, and re-create when the base model changes

----- LET THE USER PICK A PROBLEM AND CHOOSE BETWEEN "BETTER PROMPT", "RETRIEVAL", AND "FINE-TUNE", THEN SHOW THE COST AND RESULT OF EACH -----

Supervised Fine-tuning

The same training loop from Part E, applied to a much smaller curated dataset
Examples are pairs of input and the desired output, written for the target task
Data quality dominates data quantity
- A few hundred consistent examples can outperform thousands of sloppy ones
Training hard on a narrow set can degrade abilities the model used to have
- This is often called catastrophic forgetting

----- SHOW A MODEL BEFORE AND AFTER FINE-TUNING ON ONE FORMAT, INCLUDING A TASK IT USED TO HANDLE AND NOW HANDLES WORSE -----

Parameter-Efficient Fine-tuning

Updating every parameter is expensive in memory, compute, and storage
Instead, a small set of additional parameters can be trained while the original ones stay frozen
- LoRA and other adapter methods work this way
The result is a small file layered on top of a shared base model
Adapters can be swapped per task, and many of them can share one underlying model

----- SHOW ONE BASE MODEL WITH SWAPPABLE ADAPTER CARTRIDGES, EACH CHANGING WHAT IT IS GOOD AT -----

Preference Training and RLHF

Some qualities have no single correct answer to imitate
- Helpfulness, tone, refusal behavior, admitting uncertainty
Instead of one target output, the model is shown comparisons
- Two or more candidate responses, ranked by people or by another model
One common approach trains a separate reward model on those rankings
- The model is then optimized to score well against the reward model
- Using reinforcement learning for that step is what "RLHF" refers to
Other approaches optimize directly from the comparisons without a separate reward model
Optimizing against a proxy has predictable side effects
- Reward hacking: scoring well without actually being better
- Sycophancy: agreeing with the user because agreement was rated highly
Rankings can also come from automated evaluators or from a written set of principles instead of a person each time

----- LET THE USER RANK PAIRS OF RESPONSES, THEN SHOW THE MODEL SHIFTING TOWARD WHAT THEY REWARDED, INCLUDING A SIDE EFFECT THEY DID NOT INTEND -----

crutchcorn/bbb-outline-v1.md