Training

Fine-tuning

Training a pre-trained model further on a smaller, task-specific dataset to adapt it for a particular use case.

Fine-tuning is the process of taking a pre-trained model and continuing its training on a smaller, curated dataset to adapt it for a specific task or domain. Rather than training from scratch (which costs millions of dollars), you leverage the general knowledge already baked into the base model and specialize it efficiently.

For example, you might fine-tune an LLM on medical records to improve clinical note generation, or fine-tune a coding model on your company's internal codebase to make it follow your conventions. The fine-tuning process adjusts the model's weights to emphasize patterns relevant to the target domain.

Full fine-tune vs. PEFT: Full fine-tuning updates all parameters (expensive, risks catastrophic forgetting). Parameter-efficient fine-tuning methods like LoRA update only a small subset of parameters — much cheaper and often just as effective.

Fine-tuning Approaches

Supervised fine-tuning (SFT) — train on input/output example pairs
RLHF — use human feedback to steer model behavior
LoRA / QLoRA — efficient low-rank adaptation; update far fewer parameters
Instruction tuning — train on diverse instruction-following examples

Fine-tuning is increasingly accessible through platforms like OpenAI's fine-tuning API, Hugging Face, and Together AI. The main challenge is data quality — a small, high-quality dataset typically outperforms a large, noisy one. Common pitfalls include overfitting to the fine-tuning set and forgetting general capabilities.

Related Terms

← Back to Glossary