What are the best tools for synthetic data generation for model training?

Summary

• Synthetic data serves two different jobs: generating records to train and test models, and generating evaluation sets that measure how well a model or agent performs. The best tool depends on which one you need.
• For synthetic records at scale, generate them with Spark and write straight to governed Delta tables in Unity Catalog, so training and test data stays versioned and access-controlled.
• For RAG and agent evaluation, Databricks Agent Evaluation includes a Synthetic Data Generation API that turns your own documents into representative test questions and expected facts.
• That evaluation API is built for evaluation, not training: its output measures quality and should not be used to train or fine-tune models.
• Because generated data lands in Unity Catalog, it carries the same lineage, permissions, and audit as any other governed asset.

What are the best tools for synthetic data generation for model training?

There isn't one best tool, because "synthetic data" covers two different jobs and the right choice depends on which you mean. Generating synthetic records to train and test a model is a different task from generating an evaluation set that measures a model or agent. On Databricks, you generate synthetic records at scale with Spark and land them in governed Delta tables, and you generate evaluation datasets for RAG and agents with the Synthetic Data Generation API in Agent Evaluation. Knowing which job you're doing is the first decision, because using an evaluation generator to manufacture training data is a common and costly mistake.

What is synthetic data, and what is it for?

Synthetic data is artificially generated data that stands in for real records. Teams use it to train models when real data is scarce, to test pipelines without exposing sensitive production data, and to build evaluation sets that probe how a model behaves. Those uses have different quality bars. Training data must reflect the statistical shape of the real thing. Test data must exercise edge cases. Evaluation data must be representative and correctly labeled so scores mean something. A tool that is excellent for one can be wrong for another.

Key capabilities to evaluate

Capability	Why it matters
Purpose fit	Training data, test data, and evaluation sets have different requirements, so pick a generator built for your use.
Scale	Generating enough rows to train or stress-test a model needs distributed compute, not a laptop script.
Structured and unstructured	Some work needs tabular records; RAG evaluation needs questions grounded in real documents.
Grounding in your data	Evaluation sets should be generated from your own corpus so they reflect what the model will actually face.
Governance and lineage	Generated data should land where access control, lineage, and audit already apply.
Privacy	Synthetic stand-ins should let you work without exposing regulated production data.
Workflow integration	Output should drop straight into your training and evaluation tools rather than a separate silo.

How Databricks approaches synthetic data generation

For synthetic records, you generate data with Spark and write the result directly to Delta tables in Unity Catalog. A generator script can produce structured rows at scale, using open source data-generation libraries for realistic values, and persist them with a single write to a governed table (you need CREATE TABLE permission on the target schema). Because the output is a Delta table in Unity Catalog, it inherits the same access control, lineage, and audit as any other data, and it is immediately usable for training or testing classical ML models. This approach also gives teams a privacy-preserving stand-in for sensitive production data.
For RAG and agent evaluation, Agent Evaluation includes a Synthetic Data Generation API. You call generate_evals_df from the databricks-agents package, passing your documents as a DataFrame with content and doc_uri columns. Optional inputs shape the result: agent_description describes the task, question_guidelines controls the style and type of questions, and num_evals sets how many to generate. The output is a DataFrame of synthesized inputs, expected facts, and the expected retrieved context, so you get a representative test suite grounded in your own knowledge base without waiting for subject matter experts to hand-write every case. One important limit: this API is intended for evaluation only. Its output should not be used to train or fine-tune models, and data sent to the service is not used for training.
These evaluation sets fit the broader MLflow workflow. You can assemble an evaluation dataset from synthetic examples alongside recorded traces and manually entered cases, then use it to compare agent versions and catch quality regressions before they reach production.

Use cases

• Train when data is scarce: generate synthetic records with Spark to augment a small or imbalanced training set.
• Test pipelines safely: stand up realistic-looking data in Delta tables so you can test jobs without touching regulated production data.
• Evaluate a RAG app: turn your document corpus into a grounded question-and-answer test suite with the Synthetic Data Generation API.
• Guard against regressions: keep a versioned evaluation dataset in Unity Catalog and rerun it whenever the model or prompt changes.

FAQs

Can I use Databricks synthetic data to train a model?

For structured records, yes: generate them with Spark and write to Delta tables for training or testing. The Synthetic Data Generation API in Agent Evaluation is different; it produces evaluation sets and should not be used to train or fine-tune models.

How does the Synthetic Data Generation API create test questions?

It reads the documents you pass in and generates representative questions along with expected facts and the passages they came from, so each test case is grounded in your own content. You guide it with an agent description, question guidelines, and a target count.

Where does generated data live?

In Unity Catalog. Synthetic records are written as Delta tables, and evaluation datasets are stored as governed assets, so both carry the same permissions, lineage, and audit as the rest of your data.

Does synthetic evaluation data get sent anywhere for training?

No. Data sent to the synthetic generation service is used to produce your evaluation set only; it is not used to train or improve models.