The Lexicon
The eight words that show up in every AI conversation, defined plainly.
Most confusion about AI tools is vocabulary confusion. People nod along to words they have only half-defined, and then can't tell whether a bad result came from a bad prompt, a small context window, or a model that simply isn't good enough.
This module fixes the vocabulary. Nothing here is difficult; it is just rarely stated out loud.
LLM (Large Language Model)
A program that predicts the next chunk of text, over and over, until it has produced a response. That is genuinely the whole mechanism. It was trained by reading an enormous amount of text and learning which chunk tends to follow which.
Two consequences follow directly from that definition, and they explain most of what people find surprising:
- It has no memory between conversations. Each request starts cold. Anything it "remembers" was re-sent to it as part of the request.
- It is generating plausible text, not looking up facts. A confident, well-formatted, completely wrong answer is not a malfunction — it is the same mechanism working normally on a question it can't actually answer.
Tokens
Models don't read characters or words. They read tokens — fragments roughly
three to four characters long. training might be one token; AltZ might be
two or three.
Tokens matter for exactly two practical reasons:
- You are billed per token, both for what you send and what you get back.
- Limits are measured in tokens, not words or characters.
A useful rough conversion for English prose: 750 words is about 1,000 tokens. Code and JSON are denser — punctuation and indentation each cost tokens — so a file of code uses more tokens than a page of prose of the same length.
Context Window
The maximum number of tokens the model can consider at once, counting both your input and its output. Think of it as the size of the desk, not the size of the library.
When a conversation grows past the context window, something has to be dropped or summarized. This is why a long chat starts "forgetting" what you said at the beginning — the earlier turns fell off the desk.
Current models have very large windows (a million tokens is now common), but large is not the same as free: everything in the window is re-read and re-billed on every single turn.
System Prompt
Instructions that sit above the conversation and describe how the model should behave — its role, its constraints, its output format. The user's message says what to do this time; the system prompt says how you always behave.
The distinction matters because the system prompt is stable and the user turn is not. Put durable rules in the system prompt and per-request specifics in the user turn, and you will get far more consistent results:
System: You are a technical editor. Reply only with the corrected sentence.
Never add commentary.
User: Their going to the conference next week.
Prompt
Everything you send: the system prompt, the conversation history, and your current message. When someone says "the prompt was too long," they mean the total.
Temperature
A dial for randomness. At low values the model picks the most likely next token almost every time, producing repetitive but predictable output. At higher values it takes more chances.
Hallucination
The industry's term for the model stating something false with the same fluency and confidence it uses for something true.
The word is a little misleading, because it implies a malfunction. Given the definition of an LLM above, generating plausible-but-false text is the expected behavior whenever the model lacks the information to be correct. The fix is usually to supply the missing information, constrain the output, or verify the result — not to ask the model to try harder.
Structured Output
Getting a machine-readable response — usually JSON — instead of prose. This is the difference between a demo and something you can build on, because prose has to be parsed by a human and JSON can be parsed by a program.
It is important enough that a full advanced lab is devoted to it.
Check yourself
Before moving on, you should be able to answer these without scrolling up:
- Why does a long conversation cost more per message than a short one?
- Where would you put "always answer in British English" — system prompt or user message? Why?
- Your model returns a confident, wrong date. What is your first move?