Data sourcing for post-training teams: SFT, preference and RL data from real work

Which supply route fits each post-training objective?

The right route depends on whether you need volume, realism or verifiable outcomes, because each route is strong on only one or two of these. Recent work on post-training data argues that how data is built and mixed often matters more than the choice of SFT, DPO or PPO, and that uncontrolled source mixtures and missing lineage degrade results [2]. Treat route selection as a data-quality decision, not only a budget line.

Illustrative example: invented to show structure; it does not describe an available dataset.

Synthetic routes scale well but inherit the coverage of their seeds. The Synatra work notes that direct demonstrations gathered through exploration or RL are expensive and often poorly covered, and it converts indirect knowledge such as tutorials into synthetic demonstrations [6]. Licensed real records are a strong seed for exactly that kind of generation, if the license allows it. For a deeper comparison in agent settings, see licensed vs commissioned vs synthetic trajectories.

How business records map to SFT, preference and RL data

Operational records already encode demonstrations, comparisons and outcomes; the work is recognizing which record type yields which training signal. The mapping below is where most of the value in licensed work product sits.

• Resolved support tickets and engineering fixes to SFT demonstrations. The customer request is the prompt, the final agent reply or merged patch is the target, and the resolution code filters out failed attempts. See expert demonstration data for SFT.
• Draft-to-final edits to preference pairs. A first-draft contract clause or reply and the version a senior reviewer approved form a natural chosen/rejected pair. That is the input format DPO consumes directly, without a separate reward model [3].
• QA scorecards and review outcomes to reward labels. Supervisor scores, rubric checkboxes and correction notes behave like graded comparisons. Llama 2 trained its reward model on binary choices between two outputs [4]. Scorecards give graded signal and, often, the reason. See human feedback datasets of QA scores and corrections and training data for reward models.
• Workflow histories with outcomes to RL tasks with verifiers. A purchase-order exception, a claims adjudication or a maintenance work order has a start state, a sequence of system actions and a checkable end state. Those end states become verifiers. See enterprise workflow datasets and agent trajectories and training data for RL environments.
• Step-level review comments to process supervision. Reviewers who flag the exact line or step that went wrong supply step labels. See training data for process supervision.

Two cautions apply. Real preference signals are noisy: published reward-modeling work reports inter-annotator agreement typically between 63 and 72 percent [5], and an edit may reflect house style rather than quality. Keep the reviewer role and rubric so you can filter.

RL environments from business workflows: real records vs simulated systems

Most commercial RL environments today are simulations of enterprise software, so real workflow records are most useful as task seeds, state distributions and verifiers. Vendors describe an RL environment as a structured simulation in which an agent completes multi-step tasks and is scored on results [8]. Benchmarks such as CRMArena-Pro (Salesforce AI Research) build their CRM environments on synthetic records using 21 latent variables to induce implicit causal structure [7].

The gap is distributional. Synthetic tenants rarely contain the duplicated accounts, half-filled fields, reopened tickets and policy exceptions that dominate real queues. Licensed histories let you sample task frequencies, field-null rates and failure modes from production, then populate the simulator to match. Ask the supplier for state snapshots, action logs with timestamps and the final outcome field, not only the narrative record.

What to require from every post-training data supplier

Give commissioned, licensed and synthetic suppliers the same written spec, so you can compare batches and trace lineage later [2]. The request below works as a template; adjust fields by objective.

Illustrative example: invented to show structure; it does not describe an available dataset.

post_training_data_request:
  objective: preference_pairs        # sft | preference_pairs | reward_labels | rl_tasks | process_labels
  record_types: [contract_redlines, reviewer_approvals]
  outcome_labels:
    field: final_status              # approved | revised | rejected
    definition: "status recorded by the approving reviewer"
  expert_metadata:                   # role and credentials only, no personal details
    role: "senior commercial counsel"
    years_experience_band: "10+"
  rubric:
    provided: true
    version: "v3, effective dates attached"
  time_span: "2021-01 to 2025-12"
  deidentification:
    method: "names, emails, phones, account numbers replaced with typed tokens"
    method_documented: true
    sample_checked: true
  lineage:
    per_record_source_id: true
    synthetic_or_model_generated_share: "declared per batch"
  delivery_format: jsonl

For health records, require HIPAA de-identification by Safe Harbor or Expert Determination and ask which was used [9]. Require a declaration of any model-generated content in every batch, since undeclared distilled data contaminates both training mixtures and evaluation sets.

Rights questions specific to post-training uses

Post-training uses go beyond plain "training," so the license should name each one explicitly. Check these before signing:

• Use of records to train reward models and to build RL environments and verifiers, not only for SFT.
• Ownership of model outputs derived from licensed records, including preference pairs you generate by sampling your own model against licensed targets.
• Whether synthetic data generated from licensed seeds may be kept after the term ends.
• Whether any supplier batch includes outputs from third-party models. Some model providers' terms restrict using outputs to develop competing models; read the current terms before mixing distilled data.
• Transparency duties as of October 2026: EU AI Act Article 53 obligations for general-purpose model providers have applied since 2 August 2025 [10], and California AB 2013 training-data disclosures were due 1 January 2026 [11]. Keep lineage that supports both.

A fine-tuning-only data license may be enough for SFT but too narrow for reward models or RL. Have counsel read scope language against your actual pipeline; the in-house counsel guide covers review and sign-off.

This page is general information, not legal advice. Confirm requirements with counsel for your jurisdiction and use case.

Test a licensed sample before committing to volume

A small ablation on a sample is the cheapest way to learn whether a source moves your evals. Hold out your evaluation set first, fine-tune a small model or a LoRA adapter on the sample alone and mixed with your current data, and compare against an equal-size synthetic baseline. Check near-duplicate overlap with your eval prompts and with public benchmarks before reading the results. Our data quality assessment guide and the guide on combining licensed and synthetic data cover contamination checks and mixing ratios.

How SourceX fits a post-training data program

SourceX sources operational datasets from US companies on request, including support and sales histories, engineering records, documents, finance and legal workflows, and new recordings of hands-on work. Nothing is held in stock, and a request does not guarantee a match. Every dataset is rights-reviewed for ownership and consents and delivered under a license that defines records, uses, term and delivery. Personal details are removed or replaced before delivery, the method is recorded and a sample is checked, though no method is perfect. Post-training leads can describe the records they need to SourceX. Other team guides are on the data sourcing by buyer team hub, and the term itself is defined in the post-training glossary entry.

Sourcing real-work data for post-training

Describe the data you need, such as record types, outcome labels and time span, not the businesses that hold it. SourceX looks for US businesses that hold matching data, assesses data and licensing permissions, and agrees pricing and allowed uses in a license; every release is approved by the supplying company. Start a post-training data request.