Question
What is RLHF data?
By SourceX Editorial · Updated
Short answer
RLHF data is the human input used in reinforcement learning from human feedback. It usually includes demonstrations (people writing ideal responses) and preference comparisons (people choosing the better of two model outputs). The preferences train a reward model, which then guides further model training. The method was described in research by Christiano et al. (2017) and applied to language models by Ouyang et al. (2022).
Part of: AI training data
Grader rubric
RLHF data
Explanation
RLHF data is produced against a model's own outputs, so it is usually commissioned rather than licensed from existing archives.
Existing business data can still support related steps, such as supervised fine-tuning with expert demonstrations.
Examples
- Prompt + two responses + which is better
- Prompt + expert-written ideal response
Rights and privacy
- Annotator agreements
- Confidential content in prompts
How licensing works
A business describes its data, Acquisition Partner demand is assessed, scope and permitted use are proposed, the business approves the scope and terms and signs an agreement, and the data is delivered and de-identified before it is used or shared. See data licensing for detail.
Relevant industries
Relevant data types
Related questions
Sources
General information, not legal advice. Editorial policy.
See if your company qualifies
A short company assessment. No data uploads are needed.