LLM Fundamentals, Model Selection, and Prompt Engineering

A language model is trained to model sequences of **tokens**. For the autoregressive LLMs this module focuses on, the core pretraining objective is next-token prediction: given all the tokens before it, predict what comes next, across trillions of examples pulled from massive amounts of text. Modern LLMs then typically go through additional post-training for instruction following, reasoning, tool use, and safety, which is covered below. Much of what the model appears to "know" - facts, code syntax, reasoning patterns, tone - is a side effect of getting extremely good at that core prediction task. **Tokens** are the units the model actually processes. Not words, not characters - sub-word chunks from a fixed vocabulary. The word "tokenization" might split into `Token`, `ization`. A rare word might split down to individual characters. A common rough heuristic: for typical English text, one token is often around three-quarters of a word - but this ratio varies significantly by tokenizer, language, code, and text structure, so treat it as a starting estimate, not a universal rule. > 💡 **Tip:** Every model provider exposes a tokenizer you can test against. Paste a sentence in and look at how it splits - this builds intuition fast, is the most reliable way to measure actual token counts for your text, and explains why prompts in some languages cost more tokens than the same meaning expressed in English. Two things make up a model: the **weights** (billions of learned numbers) and the **architecture** (almost always a Transformer) that runs those weights to produce output. When people say "GPT-4" or "Claude," they mean a specific set of weights running on a specific architecture. ### How a model goes from text-predictor to assistant A freshly pretrained model - a **base model** - is primarily optimized to continue token sequences plausibly. It may answer questions or follow some instructions incidentally, since those patterns exist in its training data, but it has not been specifically post-trained to reliably behave like a helpful assistant. Ask it a question and it might answer, or it might just keep writing more questions, because "predict the next token" does not inherently mean "be helpful." Getting from base model to assistant takes additional post-training stages: * **Supervised fine-tuning (SFT)** - the model is shown curated examples of good assistant behavior: answering clearly, declining harmful requests, following instructions. This is what teaches it to act like an assistant instead of an autocomplete engine. * **Preference-based post-training** - human or other preference data is used to make desirable responses more likely. Traditional reinforcement learning from human feedback (RLHF) trains a reward model from human comparisons of response pairs, then optimizes the policy against that reward; newer approaches can use direct preference-optimization methods instead. Either way, this stage helps the model learn judgment calls with no single correct answer, like "is this response too terse or appropriately concise." * **Reasoning-focused post-training** - some models are further trained to spend additional computation on difficult reasoning tasks, particularly math, logic, and code, where there is a checkable right answer to reward. > 📌 **Remember:** A base model completes text without reliably following instructions. A post-trained assistant model follows instructions and reflects human preferences. A reasoning-trained model spends extra computation on hard problems, at the cost of more tokens and higher latency. Knowing what stages a model went through tells you what to expect from it before you write a single prompt. ### Why this matters for engineering, not trivia You do not need to reproduce this training pipeline. You need it to explain behavior you will see constantly: why a cheap, lightly-tuned model sometimes rambles or ignores instructions, why a reasoning model is slower and pricier but more reliable on multi-step logic, and why "just prompt harder" cannot fix a model that was never trained on the behavior you are asking for. ---