---
id: "prereq-llm-context-tokenization"
type: "prereq"
source_timestamps: ["00:02:18", "00:05:08"]
tags: ["technical-literacy", "ai-fundamentals"]
related: ["concept-long-running-agents", "claim-consumer-hardware-upgrade-cycle"]
reason: "Required to grasp the compute costs and hardware constraints of long-running agents."
sources: ["s35-compounding-gap"]
sourceVaultSlug: "s35-compounding-gap"
originDay: 35
---
# Understanding of Context Windows and Tokenization

## Prerequisite: Context Windows and Tokenization

### What you need to know
The speaker assumes the audience understands:

- What a **token** is (a sub-word unit consumed by an LLM)
- What it means for a model to **"burn millions of tokens"** during a long run
- The significance of **"local tokenization"** via consumer GPUs (no cloud round-trip per token)
- Why **context window size** matters for sustained agent work

### Why this prerequisite matters
Without it, the references in [[concept-long-running-agents]] and [[claim-consumer-hardware-upgrade-cycle]] feel abstract. With it, the compute and hardware claims become concrete and falsifiable.