LoRA (Low-Rank Adaptation)
A parameter-efficient fine-tuning technique that trains small adapter matrices instead of full model weights.
LoRA freezes the base model's weights and trains small low-rank matrices inserted into attention layers. The result is a fine-tuned model that's 100–1000x cheaper to train and store than full fine-tuning, at ~95% of full-tuning quality on most tasks.
LoRA (and its quantized variant QLoRA) are the default fine-tuning method for open-source LLMs in 2026. You can train a Llama 3.3 70B LoRA on a single A100 in hours; full fine-tuning would need a multi-GPU cluster and days.
Adapters are portable — you ship a small (~100MB) LoRA file that layers onto the base model. This makes per-tenant or per-task customisation practical.
Key points
- Trains small adapter matrices, not full weights
- 100–1000x cheaper than full fine-tuning
- QLoRA adds 4-bit quantization for further efficiency
- Portable adapters enable per-tenant customisation
Common use cases
Frequently asked
Related terms
Continuing training of a pre-trained model on domain-specific data to specialize its behavior.
A neural network trained on massive text corpora to understand and generate human-like language.
Compressing a model by reducing the numerical precision of its weights, cutting memory and speed cost.
Applying LoRA (Low-Rank Adaptation) in your business?
We help enterprises design, ship, and govern AI systems end-to-end.