Small Language Model (SLM)
A compact LLM (1B–15B parameters) tuned for cost, latency, and on-device deployment.
Small language models — Phi-3, Gemma, Llama 3 8B, Mistral 7B — deliver 70–90% of the quality of frontier models on well-scoped tasks at 10–100x lower cost. They are the practical foundation for high-volume enterprise inference.
The economics matter: at scale, an SLM running on your own GPUs can cost USD 0.01 per 1M tokens vs USD 5–15 for a frontier API. Over billions of tokens, that is the difference between a viable product and an unfundable one.
The design pattern is task specialization: use a frontier model to generate training data or handle escalations; use an SLM for the 95% of routine calls.
Key points
- 10–100x cheaper than frontier models per token
- Run on a single GPU or even CPU
- Fine-tuning yields large gains on narrow tasks
- Weakness: general reasoning and long context
Common use cases
Frequently asked
Related terms
A neural network trained on massive text corpora to understand and generate human-like language.
Continuing training of a pre-trained model on domain-specific data to specialize its behavior.
Running AI models on-device or on local hardware instead of in the cloud.
Compressing a model by reducing the numerical precision of its weights, cutting memory and speed cost.
Applying Small Language Model (SLM) in your business?
We help enterprises design, ship, and govern AI systems end-to-end.