Synthetic Data Generation Systems
Synthetic data pipeline architecture: generators, critics, diversity checks, contamination controls, collapse detection, evals, and when not to train on synthetic text.
- Read time
- 15 min
- Level
- Intermediate
- Updated
- 2026-08-25
- Components
- Seed and schema · Generator models · Filter / critic
A synthetic data generation system is a pipeline that expands a small set of seeds into training or eval examples using generator models, then filters those examples with critics, diversity checks, and contamination screens so the resulting dataset is usable — and that watches for model collapse when synthetic output is fed back as the next generation’s input. Synthetic data is a multiplier on seeds you already trust, not a substitute for domain reality.
Key takeaways
- 01
Start from real seeds (traces, tickets, documents, expert templates); unconstrained generation produces fluent garbage with hidden duplicates.
- 02
Filters are the product: schema validity, groundedness to a source, toxicity, PII, and a critic that can reject, not just score.
- 03
Measure diversity (embedding clusters, n-gram novelty, template coverage) or you will train on 50 paraphrases of one example.
- 04
Recursive training on a model’s own output collapses rare skills and sharpens artifacts; mix real data and cap the synthetic fraction.
- 05
Keep a frozen human-labeled eval set that never passes through the generator; it is the only honest score.
What synthetic data is for
Synthetic data fills coverage holes that collection cannot fill cheaply: rare error codes, under-represented languages, adversarial phrasings, structured tool traces, or teacher-labeled rationales for distillation. It is also how you build eval variants around a gold case without waiting for production to stumble into them. It is not a way to avoid legal access to real data, and it is not automatically “privacy-safe” — generators memorize, and seeds leak.
The pipeline mindset: every record has a seed id, a generator version, a filter verdict, and a lineage hash. If you cannot say which seed a row came from, you cannot debug a collapse or a contamination incident.
Data flow: seed → generate → filter → mix
Seeds and a schema go in: task type, fields, difficulty, and constraints (“must mention field X,” “SQL must parse,” “answer span must be in the passage”). A generator — often a stronger teacher model, sometimes a specialist — emits candidates, preferably with structured outputs. Critics score or binary-reject: schema, execution (did the SQL run, did the unit test pass), groundedness, safety, PII. Survivors hit a diversity stage that drops near-duplicates. A mixer then samples a training batch with a declared real:synthetic ratio and writes a snapshot to the dataset registry.
Generation without an executable check is how you get plausible SQL that never ran and plausible citations that never existed. If the task has a verifier, the verifier sits in this pipeline, not only in RL later.
- Seed + schema + difficulty dial
- Generate candidates (structured when possible)
- Deterministic filters, then model critics
- Diversity / de-dup against seeds and prior snapshots
- Mix with real data at a declared ratio; version the snapshot
Filters, diversity, and contamination
Layer cheap filters first. Parse JSON, validate JSON Schema, compile code, execute tests in a sandbox, check that RAG answers quote a provided span. Route only the remainder to an LLM critic with binary rubrics. A 1–10 “quality” score without a threshold is not a filter. Log reject reasons; a spike in schema rejects is a generator bug, a spike in PII rejects is a seed bug.
Diversity: embed survivors and drop pairs below a cosine threshold, cap examples per template and per seed, and track unique n-grams. Contamination: screen against the frozen eval set and against known public benchmarks you claim to hold out. If a synthetic eval item is a paraphrase of a train item, the number is fake. Dedup across snapshots so yesterday’s run does not re-enter as “new” data.
Collapse and recursive generation
Model collapse is the loss of tail skills and the amplification of generator artifacts when each generation trains on the last generation’s output. It shows up as falling n-gram entropy, shrinking embedding volume, and eval scores that stay high on synthetic tests while dropping on real ones. The controls are dull: keep a floor of real data, cap synthetic fraction per mix, periodically refresh seeds from production, and stop when the collapse detector trips.
Teacher–student distillation is a controlled form of this loop. It works when the teacher is better on a verifiable task and the student is evaluated on real held-out cases. It fails when the teacher is the same size as the student, when there is no verifier, or when the student is later used as the teacher. Document the generation depth the way you document epochs.
Failure modes
Mode collapse into a few templates that look diverse to the eye. Teacher hallucination written into “ground truth.” Eval contamination. PII from seeds or from the teacher’s pretraining surfacing in student outputs. Style artifacts (the teacher’s favorite transition words) that the student then treats as the house voice. A critic that likes verbosity, so the dataset drifts long. A legal assumption that synthetic equals uncopyrighted or unconsented — it does not automatically.
Operationally: generator model upgrades that silently change the distribution mid-corpus, and mixing ratios that are not recorded. Treat a data snapshot like a build artifact. If you cannot rebuild it from seeds, generator version, and filter version, do not train on it.
- Unverified fluent labels (hallucinated ground truth)
- Near-duplicate paraphrases counted as diversity
- Train/eval contamination via paraphrases
- Recursive generation without a real-data floor
- PII and license issues inherited from seeds or teachers
Evals, and when not to use synthetic data
The frozen human-labeled (or production-verified) eval set is sacred: it never goes through the generator. Report student scores on that set, plus dataset diagnostics: reject rates by filter, cluster count, duplicate rate, n-gram entropy, PII hits, and contamination hits. If synthetic-only evals rise while real evals fall, you are measuring the generator talking to itself.
Do not use synthetic data as the only train set for a customer-facing skill, as a privacy strategy without a threat model, or as an eval set for the same generator that produced it. Do use it to oversample rare verified tails, to build tool traces you can execute, and to stress-test agents with attack paraphrases that a red-team pipeline already labeled. The seeds and the filters, not the teacher brand, decide whether the snapshot is safe to train on.