Applied AI / Forward Deployed Engineer · Fine-Tuning & Model Adaptation · Lesson 2 of 2
How fine-tuning works: SFT, LoRA/PEFT, DPO & distillation
- SFT (supervised fine-tuning): train on (prompt → ideal completion) pairs. The workhorse — you need a curated, high-quality dataset (hundreds to thousands of examples).
- LoRA / PEFT (parameter-efficient fine-tuning): instead of updating all weights, train small adapter matrices. Far cheaper, faster, smaller to store, and you can swap adapters per task. The default modern approach.
- Preference tuning (DPO / RLHF): train on preferred vs rejected response pairs to shape nuanced behavior/tone. DPO is a simpler, popular alternative to full RLHF.
- Distillation: use a big 'teacher' model to generate training data (or targets) for a small 'student' model — get near-big-model quality on a task at small-model cost/latency.
Data quality > data quantityFine-tuning success is dominated by dataset quality: clean, consistent, representative examples that demonstrate exactly the behavior you want. A few hundred great examples beat thousands of noisy ones. Garbage in, worse-and-confidently-wrong out.
Costs & catastrophic forgettingFine-tuning has real costs: data curation, training, hosting the custom model, re-tuning when the base updates, and catastrophic forgetting (the model gets worse at general tasks). Weigh this against staying on a prompted frontier model.
Fine-tuning in 2026 — the reality checkSelf-serve fine-tuning of frontier proprietary models is disappearing (OpenAI is sunsetting its self-serve fine-tuning; Anthropic never offered it for current models). In practice, 2026 fine-tuning means open-weight models — LoRA SFT and increasingly RFT / agent-RL on hosted platforms. And the ladder grew a rung zero: context-engineer first — a frontier model with well-managed context clears most quality bars at zero training cost. Fine-tune for form, latency, and unit cost; never for facts.
◆ Lock it in
- SFT = train on ideal completions; LoRA/PEFT = cheap adapters (modern default); DPO/RLHF = preference/behavior tuning.
- Distillation = teacher model trains a cheaper student for prod cost/latency.
- Success = data quality; beware hosting costs, re-tuning, and catastrophic forgetting.
Feynman drill — say it out loudExplain LoRA in one sentence and why it's cheaper than full fine-tuning.
Step 1 rate your confidence · Step 2 pick your answer
You want cheap, fast task-specific fine-tuning without paying to update all weights. Use:
Step 1 — how sure are you?