Fine-Tuning and Post-Training

A better prompt, few-shot examples, or retrieval beats a tune more often than anyone admits.

Format compliance, tone, latency from a smaller model, and private behavior no prompt encodes.

Turn 'make it better' into a scored task with a holdout set a stranger could grade the same way.

Measure the base model on your task, on IFEval, and on MMLU before touching a single weight.

Bytes per parameter, H100 near 2 to 3 dollars an hour, and what an 8B LoRA run really costs.

Production logs, paid writers, and teacher models, with the cost and the bias of each priced out.

A thousand curated examples usually beats a hundred thousand scraped ones, and why that holds.

Tokenizer chat templates and special tokens, plus the mismatch that trains a string you never wrote.

Loss on completion tokens only, and how to print the label tensor to prove the mask is where you think.

Generating traces from Ministral 3, plus the upstream terms that decide if you can ship the result.

Near-duplicate removal and an n-gram check of train against eval, run before you trust any number.

TRL 1.8 SFTTrainer on Ministral 3 8B, sequence packing, and what the loss curve does not tell you.

Adapters on all linear layers including MLP, not attention only, which is the 2024 advice that aged badly.

Rank 16 to 32 unless capacity-bound, and a learning rate near ten times the full fine-tune value.

4-bit NF4 base weights, the quality cost you sometimes pay, and when 24GB is simply not enough.

Unsloth for the fastest single GPU run, Axolotl YAML when the recipe has to survive four people.

FSDP and ZeRO-3 sharding, and the point where renting eight H100s beats another LoRA sweep.

Replay mixing and a retention pack, because LoRA reduces forgetting but does not prevent it.

Demonstrations cannot say 'this answer is worse', which is where preference and reward data start.

Chosen and rejected pairs, beta, the reference model, and the length bias that inflates your win rate.

Unit tests, exact match, and schema checks: RLVR only works where a program can grade the answer.

Group sampling with no value model, KL to the reference, and rollouts that dominate the GPU bill.

GSPO and DAPO as sequence-level and clipping fixes, and why PPO-style RLHF is now rarely the first choice.

The model finds the bug in your grader before you do, so read raw samples every run, not just curves.

Sample traces from a large teacher, keep only the verified ones, and train a student that still reasons.

Multi-LoRA serving off one base model against a single merged checkpoint, and what each choice costs.

FP8 first on Hopper and Blackwell, AWQ through llm-compressor when memory is the binding constraint.

Re-run the scorecard on the quantized artifact, because quantization undoes a tune very quietly.

Position swaps, a judge from a different model family, and a human-labeled calibration set of your own.

Task lift against the base model, retention deltas, and dollars per successful request, side by side.