PEFT and Fine-Tuning Pipelines
Enterprise PEFT pipeline: datasets, LoRA/QLoRA training, forgetting checks, adapter registry, merge vs multi-adapter serve, failure modes, and when RAG is enough.
- Read time
- 16 min
- Level
- Advanced
- Updated
- 2026-08-25
- Components
- Dataset registry · PEFT trainer (LoRA/QLoRA) · Eval harness
A PEFT fine-tuning pipeline is the production path that versions a dataset, trains a small adapter (usually LoRA or QLoRA) on a frozen base model, evaluates it for task gain and catastrophic forgetting, registers the artifact, and serves it — either merged into weights or loaded beside other adapters. It is how you change how a model behaves on your tasks without operating a full-model training stack, and it is the wrong tool when retrieval would supply the missing facts.
Key takeaways
- 01
Fine-tune for behavior, format, and domain language; use RAG for facts that change. Most “train on our docs” requests are retrieval problems.
- 02
The dataset is the product: schema, de-duplication, PII handling, and a frozen eval split matter more than rank or learning rate.
- 03
Always measure the base-model eval suite after training; a win on the domain set that collapses general instruction following is a failed run.
- 04
Register adapters like binaries: base model hash, tokenizer, rank, data snapshot, eval report, and promotion state.
- 05
Serve merged weights when you have one adapter; serve multi-LoRA when many teams need isolated behaviors on the same base.
When PEFT is the right lever
PEFT (parameter-efficient fine-tuning) updates a small set of weights — LoRA adapters, QLoRA on quantized bases, sometimes prompt/prefix tuners — while the base model stays frozen. Use it when you need the model to adopt a house style, a tool-calling schema, a language or dialect, or a skill the base fails at even with good retrieval. Do not use it as a dumping ground for documents; an adapter does not get weekly policy updates for free, and it will still hallucinate facts it “saw” in training.
Full-weight fine-tuning is for labs with a pretraining budget and a reason PEFT cannot fit the skill. For enterprise delivery, LoRA-class PEFT is the default because artifacts are small, rollbacks are a file swap, and one base can carry many adapters. If you cannot describe the behavior change in a sentence and a golden set, you are not ready to train.
Pipeline data flow
Data lands in a registry: each snapshot has a schema (messages, tools, labels), a license/PII report, a hash, and a frozen eval split that never enters training. A job loads a pinned base model and tokenizer, trains the adapter with logged hyperparameters (rank, alpha, dropout, target modules, steps, learning rate), and writes checkpoints only to the job’s artifact store. The eval harness scores the candidate against the domain set and against a held-out general suite (instruction following, safety, schema validity). A human or a gate promotes the adapter into the registry with a pointer to that report.
Serving reads only promoted artifacts. Shadow traffic compares adapter vs base on live traces before a cutover. Rollback is “point the route back at the previous adapter id,” not “we will retrain.”
- Snapshot data → hash → train split / frozen eval split
- Pin base model, tokenizer, and PEFT config
- Train → domain eval + forgetting eval + safety eval
- Register adapter with lineage; promote explicitly
- Serve merged or multi-LoRA; shadow, then cut over
Training choices that survive production
LoRA rank is a capacity knob, not a quality trophy. Ranks in the 8–64 range cover most enterprise style and tool-schema jobs; if you need rank 256 and still underfit, the data is noisy or the task is a full-finetune candidate. Target attention (and sometimes MLP) modules consistently with how you will serve. QLoRA exists to fit training on smaller GPUs; the serving dtype can still be higher — document both. Pack examples so the loss is on the assistant spans you care about, not on the system prompt.
Catastrophic forgetting is the default unless you mix replay data or regularize against the base. A domain-only run that teaches invoice JSON may break “refuse medical advice” and “follow the tool schema on a different API.” Mix a slice of general instruction data, or run a distillation-style KL term, and keep the general suite as a hard gate.
Registry and serving adapters
The adapter registry is a model registry with extra fields: base model id and hash, tokenizer id, LoRA rank/alpha/targets, data snapshot id, eval report id, and who promoted it. Do not store “latest.safetensors” on a share. Downstream serving pulls by immutable id. When the base model version changes, adapters do not silently apply; they rebuild or they stay pinned to the old base pool.
Merged weights are operationally simple: one file, one replica, slightly better latency. Multi-adapter serving (one base, many LoRAs) is how you keep teams isolated without multiplying GPU fleets. Cap concurrent loaded adapters, map request → adapter in the gateway, and never let the client pass an arbitrary adapter path. If two adapters fight (style vs safety), that is a product conflict — merge them in data, not at request time, unless you have measured composition.
Failure modes
Training on eval data, or on documents that should have been RAG, produces demos that die in production. Label leakage from near-duplicate tickets does the same. Tokenizer mismatch between train and serve truncates or retokenizes tools. Serving a LoRA on the wrong base is a silent personality change. QLoRA trained at 4-bit and evaluated at 16-bit reports a quality you will not ship. An adapter that memorizes PII from the train set is a privacy incident, not a win.
Operationally: jobs that overwrite artifacts, missing promotion states, and “we fine-tuned again” as the response to a prompt bug. If a regression is one example, add it to the eval set and the train mix deliberately; do not run a one-off job from a laptop against production.
- Facts stuffed into adapters that should be retrieved
- Forgotten general skills; safety or schema regressions
- Train/serve tokenizer or base-hash mismatch
- PII memorization from unredacted traces
- Unregistered “latest” files as the deploy artifact
Evals, and when not to fine-tune
Gate on three numbers: domain task score vs the base (must rise), general suite score (must not fall past a set delta), and safety/schema hard fails (must stay at zero). Add a memorization check: can the model regurgitate unique train-set strings it should not. Keep the judge and the exact-match checkers versioned with the pipeline. Offline goldens first; then shadow comparison on live traffic before promotion.
Do not fine-tune when prompting plus tools plus RAG already meet the golden set, when the corpus changes weekly, or when you cannot staff a dataset owner. Do fine-tune when the base persistently fails a stable skill — dialect, house JSON, tool choreography — after those cheaper layers are in place. ReinforcedX treats adapters, datasets, and eval suites as client-owned artifacts in the client’s registry, not as a hosted model product.