Anthropic
S18
HardPremiumDesign a Synthetic Data Generation & Curation Agent
Design a pipeline that generates and curates synthetic data for fine-tuning or evaluation, and proves it helps on real held-out data.
DataEvaluationFine-tuningQualityContamination
Key Requirements
- Control coverage across tasks, personas and difficulty, seeded from real data
- Verify correctness with a signal independent of the generator
- Remove duplicates and near-duplicates, and any overlap with eval sets
- Prove value on a held-out real eval before any set ships
- Avoid model collapse: keep real data and track each row's lineage
Interviewer Follow-ups
- Q1How do you prevent model collapse from training on synthetic data?
- Q2How do you validate that generated examples are realistic?
- Q3How do you ensure diversity across edge cases?
Loading...