Fine-Tuning
Continuing training of a pre-trained model on domain-specific data to specialize its behavior.
Fine-tuning takes a general-purpose foundation model and updates its weights on a curated dataset of examples. The result is a model that follows a specific tone, format, or classification schema more reliably than prompting alone.
Modern fine-tuning is dominated by parameter-efficient methods like LoRA and QLoRA, which train only 0.1–1% of parameters. A useful fine-tune can be produced in hours on a single GPU for a few hundred dollars.
Fine-tuning is not the right tool for teaching a model new facts — that is what RAG is for. Fine-tuning teaches behaviors: 'always respond in this JSON schema', 'classify this ticket into these 12 categories', 'write in our brand voice'.
Key points
- Needs 100–10,000 high-quality labeled examples
- LoRA / QLoRA reduce cost 10–100x vs full fine-tuning
- Fine-tuned models can run on smaller base models, cutting inference cost
- Iteration cycle is slower than prompt engineering — plan evaluation harness first
Common use cases
Frequently asked
Related terms
A neural network trained on massive text corpora to understand and generate human-like language.
A pattern that grounds LLM answers in your private documents by retrieving relevant snippets at query time.
The practice of designing model inputs to produce reliable, task-specific outputs.
DevOps practices adapted to the lifecycle of machine learning and AI systems.
Applying Fine-Tuning in your business?
We help enterprises design, ship, and govern AI systems end-to-end.