Fine-tuning

Further training of an existing LLM on a narrower dataset to change or improve its behavior (for example instruction following, tool calling, style, safety, or domain expertise). Fine-tuning is typically cheaper than pretraining from scratch.

Details

Fine-tuning is one form of post-training; it changes the LLM's weights so that its behavior during inference is different for similar inputs. This contrasts with in-context learning, which adapts behavior through examples in the prompt without modifying weights. Preference-based methods (for example RLHF, DPO) are sometimes grouped under fine-tuning and sometimes treated as a separate post-training category; the boundary is context-dependent.

Full fine-tuning updates all model parameters, which is expensive in compute and memory and risks catastrophic forgetting (degrading capabilities the model already had). Parameter-efficient fine-tuning (PEFT) methods address this by updating only a small fraction of parameters while keeping the rest frozen. Fine-tuning is also used in distillation workflows, where a smaller student model is fine-tuned on outputs generated by a larger teacher model.

The most widely adopted PEFT technique is LoRA (Low-Rank Adaptation), which injects small trainable low-rank matrices into the model's existing layers rather than modifying the original weights directly. The trainable parameter count is orders of magnitude smaller than full fine-tuning, requiring significantly less GPU memory and training time. The resulting adapter weights are small enough to store, version, and swap independently of the base model. QLoRA combines LoRA with model quantization, enabling fine-tuning of large models on consumer-grade hardware. Other PEFT approaches exist (prefix tuning, adapter layers), though LoRA has become the dominant default.

Open-weight models can be fine-tuned directly on local or self-hosted infrastructure, giving full control over the training process and keeping data on-premises. Inference providers also increasingly offer hosted fine-tuning APIs that abstract away PEFT mechanics, letting application developers supply training data and receive a fine-tuned model endpoint without managing GPU infrastructure or adapter merging. Common practice is to fine-tune on hundreds to thousands of curated examples rather than the millions needed for pretraining.

In practice, fine-tuning targets concrete failures measured by evals, often using training examples sourced or prioritized from production observability. After fine-tuning, evals are often re-run and observability data is monitored to confirm improvements and catch regressions.

Examples

  • Fine-tuning a base model to follow a specific support tone and formatting
  • Fine-tuning on internal domain documents to improve terminology usage
  • Safety fine-tuning to reduce a specific class of policy violations
  • Fine-tuning on tool-call datasets to teach a model structured tool calling, or to train it on specific provider-defined or provider-executed tools
  • Using LoRA to adapt a model's response style with a small dataset while preserving its general capabilities
  • QLoRA fine-tuning of a 70B-parameter model on a single GPU that would otherwise require a multi-GPU cluster for full fine-tuning

Synonyms

SFT (supervised fine-tuning), PEFT (parameter-efficient fine-tuning)