How to Fine-Tune an LLM with LoRA
How to LoRA fine-tune an enterprise model with PEFT and QLoRA: data format, rank, merge vs serve adapters, evals, and how to limit catastrophic forgetting.
To LoRA fine-tune an LLM, format a clean instruction dataset, train a low-rank adapter (or QLoRA in 4-bit) on a pinned base model, mix in enough general data to limit catastrophic forgetting, and decide whether to merge the adapter or serve it — then keep evals that score both the new skill and the old ones.
What you’ll build
- A LoRA adapter trained with a recorded rank, alpha, targets, and seed
- A QLoRA path for large bases and a full-LoRA path when you have the VRAM
- A choice, documented, to merge the adapter or serve it side-by-side
- Evals that catch catastrophic forgetting, not only domain accuracy
- A registry so you can roll back to the previous adapter without retraining
Before you start
- 01A base model you are licensed to adapt, pinned by revision
- 02A few thousand high-quality instruction or preference examples in a documented schema
- 03A held-out eval that includes both your domain tasks and a general-ability slice
- 04A GPU budget: QLoRA on a single 24–48GB card is enough for 7–14B; larger needs more
- 05A decision that RAG or prompting cannot meet the style, format, or latency need
Key takeaways
- 01
LoRA is PEFT: you train small adapter matrices, not the full base, which is why a 7–70B enterprise tune is affordable.
- 02
QLoRA loads the base in 4-bit and still trains LoRA in higher precision; it is the default when VRAM is the constraint.
- 03
Rank, alpha, and target modules are the knobs; start small, measure, then scale rank.
- 04
Catastrophic forgetting is the usual failure: mix general data and score a held-out general eval every run.
- 05
Serve adapters when you need several domains on one base; merge when you want a single dumb checkpoint.
How to LoRA fine-tune only when RAG is not enough
LoRA (Low-Rank Adaptation) injects small trainable matrices into a frozen base model. You get a domain skill without a full fine-tune. That is the PEFT tutorial enterprise teams actually need: cheaper, reversible, and small enough to store many adapters against one base. It is the wrong tool for 'know our wiki' — retrieval handles that and stays current.
Good LoRA jobs: emit a house JSON schema, talk in a regulated tone, copy a tool-call dialect, or specialize a coder on an internal framework. Bad LoRA jobs: tomorrow's pricing page, per-user permissions, or anything that must cite a living document. If you are unsure, ship RAG first. Fine-tune when the eval still fails on format or skill with the documents in context.
Data format and hygiene
Pick one schema and do not improvise. Chat models want messages[] with roles. Base completion models want prompt/completion. Mix them and you train on noise. Each row should be a task a future user will actually run, with the output you would accept in production — not a scraped blog paragraph.
Volume is not a substitute for agreement. A few thousand consistent tickets beat 200k contradictory ones. Deduplicate, strip secrets, and keep a data hash in the training record. Mix 10–30% general instruction data unless you have measured that your domain set already preserves general ability. Label the mix ratio; you will need it when forgetting shows up.
Rank, targets, PEFT, and QLoRA
Rank (r) is adapter capacity. Start at 8 or 16. Alpha scales the update; a common default is alpha = 2r. Dropout around 0.05–0.1 if you overfit. Target the attention projections first (q, k, v, o); adding MLP targets can help stubborn format tasks and will raise VRAM. This is PEFT: the base weights stay frozen, the artifact is megabytes, not tens of gigabytes.
QLoRA quantizes the base to 4-bit (typically NF4) and trains LoRA in 16-bit. It is how a 70B tune fits on a reachable GPU budget. Expect a small quality gap versus bf16 LoRA; measure it. Do not QLoRA a 7B that already fits — you are adding quantization error for no reason. Pin bitsandbytes / PEFT versions; silent library upgrades change numerics.
- r=8–16, alpha=2r, attention targets: default first run
- QLoRA: large bases or tight VRAM; compare to bf16 LoRA on the same set
- Seed + data hash in the registry row, or you cannot reproduce a good run
- Early-stop on holdout loss; extra epochs are how memorization lands
Catastrophic forgetting is the default failure
Catastrophic forgetting is the model losing skills the base had: following a generic instruction, staying in the right language, refusing a basic unsafe request. It shows up when the domain set is narrow and the epoch count is high. Mix general data, lower the learning rate, and stop when the domain eval plateaus.
Score three buckets every run: domain task, general instruction, safety/regression. Set floors before you train. If domain accuracy rises 12 points and general drops 9, you did not ship a specialist — you shipped a model that can only do one trick. Replay a slice of the original instruction mix rather than hoping the system prompt will restore what the weights forgot.
Merge vs serve adapters
Serving the adapter next to the frozen base lets you hot-swap domains (support tone vs code style) and roll back in seconds. It needs a runtime that loads LoRA (vLLM, TGI, and several others do). Merging bakes the adapter into a new checkpoint: simpler ops, one file, no multi-adapter routing. Merge when you have one domain and a serving stack that hates extra files.
Keep the unmerged adapter even after a merge. The next domain will want to train from the original base plus a new LoRA, not from a merge of last quarter's support bot. Register adapter id, base revision, data hash, metrics, and who signed off. You own those artifacts — a vendor should not.
Enterprise delivery
ReinforcedX runs PEFT jobs in your cloud on your licensed base. You own the weights, datasets, eval suites, and runbooks. We are model-agnostic: Llama, Mistral, a client fine-tune, or a commercial base you are allowed to adapt. No token markup, because this path is mostly your GPUs.
Four weeks: data schema and whether LoRA is even justified, training plus QLoRA bake-off, evals including forgetting, then registry and serving. Thirty days on-call covers the first adapter swap. If RAG plus a schema-constrained prompt already hits the rubric, we will say so in week 1 and not train.
Step-by-step build
- 1
Confirm LoRA is the right tool
Use LoRA for style, schema, or a skill; use RAG for changing facts. If a 50-example prompt already passes the rubric, do not train.
- 2
Freeze a dataset schema
Write JSONL with instruction, input, and output (or messages[] for chat), dedupe, strip PII, and hold out 10–20% that the trainer never sees.
- 3
Pick base, rank, and QLoRA vs LoRA
Pin the base revision, start at rank 8–16 with alpha ≈ 2×rank, target q/k/v/o projections, and enable QLoRA when the base will not fit in VRAM in bf16.
- 4
Mix data and train with a seed
Blend domain examples with a general instruction slice, set a seed, cap epochs so you are not memorizing, and log loss on the holdout.
- 5
Evaluate domain, general, and safety
Score the new task, a general instruction suite, and a small safety/regression set; reject the adapter if general or safety drops past your floor.
- 6
Register, serve or merge, and roll back
Store the adapter, config, and data hash in a registry; serve it next to the base or merge for a single checkpoint; keep the previous adapter hot.
Common pitfalls
The mistakes that show up in real deployments — each one costs a week if you learn it the hard way.
Fine-tuning because the wiki is stale
Weights do not update when a policy changes. If the need is current facts, use RAG. LoRA is for format, tone, tool-call shape, and skills that retrieval cannot supply.
Rank 64 on 400 examples
High rank on a tiny set memorizes. Start at rank 8–16, measure, then climb. More rank is not more quality; it is more capacity to overfit.
Domain-only data and no mix-in
Training solely on tickets teaches the model to forget how to follow a generic instruction. Mix a slice of general instruction data or you will watch general evals collapse.
Merging into base and losing the adapter
A merged checkpoint is convenient to serve and painful to combine with the next domain adapter. Keep the adapter artifact even if you ship a merge.
No eval for the behavior you did not want to change
If you only score your new JSON format, you will not see that the model now refuses harmless questions. Hold out a general suite and a safety suite every run.