Large Language Models (LLM)
Quick reference: LLM Cheatsheet
An LLM is a next-token predictor trained on huge text corpora, then tuned to follow instructions. As a developer you call it over an HTTP API (or run open weights locally) and turn its text output into product features.
What a developer must know
- It is stateless: every request must carry the whole conversation (system prompt + messages + tool results).
- Everything is tokens: input and output are billed, limited and timed in tokens, not characters.
- Output is probabilistic: same prompt, different answers. Design for variance (validation, retries, evals).
- It does not know your data: give it context (RAG, tools) instead of expecting it to remember.
- It can be wrong confidently: ground answers in sources and verify anything with consequences.
- Untrusted text is code-like: anything in the context can steer the model (prompt injection).
- Measure, do not guess: evals decide model choice, prompt changes and regressions.
Request lifecycle
Pages in this section
| Page | Covers |
|---|---|
| Cheatsheet | One-page refresh, SDK versions, current models |
| Tokens, Context & Sampling | Tokens, context windows, temperature, top-p, max tokens |
| Prompt Engineering | System prompts, structure, examples, iteration |
| Structured Output & Tool Calling | JSON schemas, function calling, the tool loop |
| RAG | Embeddings, chunking, vector DBs, hybrid search, reranking |
| Agents & MCP | Agent loops, Model Context Protocol servers and clients |
| Evals | Test sets, graders, LLM-as-judge, regression checks |
| Security | Hallucination, prompt injection, data leakage |
| Cost & Latency | Prompt caching, batching, streaming, model routing |
| Prompting vs RAG vs Fine-tuning | Which lever to pull first |
| Local Models | Ollama, llama.cpp, vLLM, open weights |
| Model Landscape | Vendors and current model families |