Skip to content

Question

What is RLHF data?

By SourceX Editorial · Updated

Short answer

RLHF data is the human input used in reinforcement learning from human feedback. It usually includes demonstrations (people writing ideal responses) and preference comparisons (people choosing the better of two model outputs). The preferences train a reward model, which then guides further model training. The method was described in research by Christiano et al. (2017) and applied to language models by Ouyang et al. (2022).

Part of: AI training data

Grader rubric

RLHF data

This page covers what is RLHF data.Illustrative

Explanation

RLHF data is produced against a model's own outputs, so it is usually commissioned rather than licensed from existing archives.

Existing business data can still support related steps, such as supervised fine-tuning with expert demonstrations.

Examples

  • Prompt + two responses + which is better
  • Prompt + expert-written ideal response

Rights and privacy

  • Annotator agreements
  • Confidential content in prompts

How licensing works

A business describes its data, Acquisition Partner demand is assessed, scope and permitted use are proposed, the business approves the scope and terms and signs an agreement, and the data is delivered and de-identified before it is used or shared. See data licensing for detail.

Related questions

Sources

  1. Christiano et al., Deep reinforcement learning from human preferences (2017)
  2. Ouyang et al., Training language models to follow instructions with human feedback (2022)

General information, not legal advice. Editorial policy.

See if your company qualifies

A short company assessment. No data uploads are needed.

See if your company qualifies