LLM Fine-Tuning FAQ
When fine-tuning is the right lever — and when it is an expensive way to store documents in weights.
LLM fine-tuning continues training a base model on your labelled examples so it acquires a skill, a format, or a voice that prompting cannot hold. It is not how you keep facts current — that is RAG — and most teams should not fine-tune until a golden set shows the same errors after a serious prompt and retrieval pass.
What fine-tuning is — and is not
If the facts change weekly, you need retrieval. If the model must talk in a format it keeps missing, you may need a fine-tune.
What is LLM fine-tuning?
Fine-tuning continues training a pretrained model on your labelled input/output pairs so its weights (or a small adapter) shift toward your task. The result should be more reliable on that format, voice, or decision boundary. It does not give the model a live view of your documents. If the source of truth updates, you still retrieve or you ship yesterday’s policy inside the weights.
Fine-tuning vs RAG — which one do we need?
RAG when the source of truth is documents that change. Fine-tuning when the model must acquire a skill, a schema, or a tone that prompting cannot hold. Many production systems use both: RAG for facts, a light adapter for format. If your pain is “it does not know our refund policy,” start with retrieval. If your pain is “it will not emit our ticket schema,” consider a fine-tune.
Should I fine-tune ChatGPT?
Usually not first. Consumer ChatGPT is the wrong surface for enterprise training data. Vendor fine-tune APIs can make sense for a narrow, high-volume task once prompting plateaus — with the caveat that you may not export full weights. Open-weight models plus LoRA give you an artefact you can actually own. We will not start an engagement by fine-tuning ChatGPT because a wiki exists.
What is PEFT, and what is LoRA?
Parameter-efficient fine-tuning (PEFT) updates a small set of weights instead of the whole network. LoRA is the common method: low-rank adapters attached to attention layers, trained on your data, then served beside the frozen base or merged. You get most of the task lift with less compute, less forgetting, and an artefact you can turn off. QLoRA is the same idea with a quantized base to fit smaller GPUs.
How much data do we need to fine-tune?
Enough clean, representative examples that cover the decision boundary — often hundreds to a few thousand for LoRA, more if the task is diverse. Quality beats volume: each row needs the input the model will see and the output you would accept. A million noisy tickets with no accepted output will teach the model your mess. Start with the golden set, then expand the train split without poisoning the holdout.
Fine-tuning vs prompting — when is prompting enough?
Prompting is enough when a system policy, a schema, and a handful of examples already pass the golden set. Fine-tune when those examples keep growing, latency from long prompts hurts, or the same error class remains after retrieval is fixed. Prompting is cheaper to reverse. We treat a fine-tune as a product change with a rollback (serve the base, drop the adapter), not as a default.
How long does fine-tuning take?
A LoRA run on a prepared dataset is often hours, not weeks. The calendar is data: labelling, de-duplication, holdouts, and evals. Inside a four-week implementation, week one decides whether you even need a fine-tune; a first adapter can land in week two or three if the labels exist. If the labels do not exist, we do not pretend a training job will invent them.
How a fine-tune actually gets built
The training job is the short part. The dataset, the holdout, and the forgetting checks are the work.
What format should training data be in?
The same shape the model will see in production: messages with system, user, and assistant turns, or instruction/output pairs, plus any tool calls if you are tuning an agent. Include refusals and hard negatives, not only happy paths. JSONL with a frozen schema is the usual artefact. If production uses retrieval, train with retrieved context in the user turn or you will teach the model to ignore it.
Who owns the fine-tuned weights?
You do. Adapters, merged checkpoints, datasets, and eval suites are handed over in your registry and object store. Vendor-hosted fine-tunes may not export full weights — we say that before you pick that path. There is no lock-in that requires us to serve the model. Thirty days of on-call covers the serving and eval path we shipped.
What is catastrophic forgetting, and should we worry?
The model gets better at your task and worse at things the base still needs — general instruction following, other tools, languages you still serve. You worry if you serve one checkpoint for many jobs. Mitigations: PEFT instead of full updates, mixing a slice of general data, and a regression suite that includes tasks you did not train on. If the suite only contains the new task, forgetting is invisible.
Do we need to fine-tune the embedding model too?
Only if retrieval recall on your questions is the bottleneck after hybrid search and a re-ranker. Generator fine-tunes do not fix a missed chunk. Domain embedding models help on jargon-heavy corpora; they also force a re-index. Measure recall first. We do not fine-tune embeddings because a blog post said to.
Full fine-tune vs adapters — which should we use?
Adapters (LoRA/QLoRA) are the default: cheaper, reversible, less forgetting, easier to serve several task heads on one base. A full update is for when adapters plateau on a large, high-quality set and you accept the eval and serving cost. Merging an adapter into the base can help throughput; keeping it separate helps rollback. That choice is measured, not aesthetic.
How do you evaluate a fine-tune?
Hold out real cases the training set never saw. Score the base model and the adapter on the same suite: task quality, format validity, refusal correctness, and a slice of jobs you still need the base for. Training loss going down is not a ship signal. Online, sample live traffic after rollout and keep the previous artefact as a pin. Preference data (RLHF/DPO) gets its own pairwise eval, not a vibe check.
What do you need from our team to fine-tune?
A named task, an accepted-output definition, access to examples (or time to label them), and an engineer who can run the serving path in your cloud. A weekly 45-minute review is enough if labels move. If subject-matter experts cannot agree on the accepted output, we pause — that disagreement will be in the weights.
Serving and operating a fine-tune
An adapter that cannot roll back, and a dataset that cannot be rebuilt, is a liability even if the demo sparkles.
Where does training run — your GPUs or ours?
Yours, or a tenancy you control. Datasets and checkpoints stay in your bucket and registry. We do not train a shared ReinforcedX model on your tickets. If a vendor fine-tune API is the right tool, you hold that account and pay the provider; we still hand over the data, evals, and runbook. VPC is the default for anything that cannot leave the perimeter.
What happens when the base model is updated?
An adapter trained on v1 may not load on v2. You pin the base you trained against, re-run training on the new base when you choose to move, and compare evals before traffic shifts. For API fine-tunes, read the vendor’s compatibility rules; some force a retrain. Silent base swaps are why pins and CI exist. Do not assume the adapter travels for free.
Can we fine-tune on PII or customer records?
Only with a written lawful basis, minimisation, and a path that never sends raw PII to a provider that trains on your data. Redact or tokenize before labelling when the task does not need the raw value. Weights can memorise rare strings; that is a real risk on small, unique records. If we cannot meet the bar, we do not run the job.
How much does fine-tuning cost?
Compute is GPU-hours (or a vendor training job billed in tokens/steps), plus the people-time to label and evaluate — usually the larger number. Serving an adapter adds memory and a bit of latency versus the base. We do not quote stale list prices. You pay providers directly; our fee is the fixed-scope implementation, not a markup on training tokens.
When does RL or preference training come in?
After supervised fine-tuning plateaus and you have pairwise preferences or a reward signal you trust — not as a substitute for labels. DPO and similar methods need preference pairs and a regression suite; RL in an environment needs a simulator and a success metric. We use them when the task is a policy over sequences (agents, reasoning), not to polish a FAQ bot that still lacks RAG.
Can we serve adapters without merging them into the base?
Yes. Serving the adapter live lets you hot-swap task heads and roll back without rebuilding the base image. Merging can raise throughput and simplify ops if you have one task and a stable artefact. Multi-tenant or multi-task systems usually keep adapters separate. Measure KV-cache and latency on your hardware before standardising.
When not to fine-tune, and where it pays
The method is the same. The dataset and the definition of a wrong answer are not.
When should we not fine-tune?
When the knowledge changes faster than you can relabel, when you have fewer than a serious handful of clean examples, when a prompt plus RAG already hits the metric, or when you cannot export or own the resulting artefact and that is a requirement. Also when the “accepted output” is still an argument. Training will freeze the argument into production.
Does fine-tuning pay off more for support, code, or extraction?
Extraction and classification with a stable schema often show the clearest lift: the model stops chatting and emits the object. Code style and internal APIs can benefit if you have accepted diffs. Support policy is usually RAG first; a support fine-tune helps tone and ticket format, not last week’s shipping rule. Measure on your slice, not on a public leaderboard.
Open-source fine-tunes vs vendor fine-tune APIs?
Open-weight plus LoRA gives you a checkpoint in your registry, air-gap options, and no training-data surprise. Vendor APIs are faster to stand up and may win on quality for that family, with less export control. Hybrid is common: prototype on an API, production adapter on an open base if residency requires it. The eval suite, not the brand, decides.
What usually goes wrong with enterprise fine-tunes?
Training on the eval set, no forgetting checks, labels that do not match production context (missing retrieved chunks or tools), and serving without a pin so a base upgrade silently drops the adapter. The other classic is fine-tuning to store documents. Those failures look like a good loss curve and a worse product. The how-to on LoRA covers the mechanical antidotes.
What should we expect in the first 90 days after a fine-tune?
Drift as live traffic diverges from the train slice, a temptation to keep adding rows without a holdout, and at least one base-model bump you must decide to follow. Budget a labelling hour each week and a scheduled eval review. Thirty days of on-call is included; after that, a quarterly eval pass is how most teams catch the quiet regressions.
Fine-tuning changes how a model behaves on a narrow task. RAG changes what it is allowed to know today. Prompting is the first lever. Parameter-efficient methods such as LoRA are the usual enterprise path: smaller datasets, cheaper training, adapters you can serve or merge. You own the resulting weights, datasets, and evals. ReinforcedX fine-tunes inside your cloud, model-agnostic, and will say so if retrieval would hit the metric without a training run. Catastrophic forgetting, stale labels, and skipping evals are how fine-tunes fail after a promising loss curve.
Keep reading
Ready to refine
your workflow?
Share your current process. We’ll help you identify what can be automated — and where efficiency can be reclaimed.