Rights and contracts
AI dataset distillation rights: does permission to train allow distillation?
By SourceX Editorial · Reviewed by Noah Loul ·
10 sections
Short answer
A data license that permits training does not automatically permit distillation. The answer depends on whether the grant covers models built from a trained model's outputs, whether licensed records leave the licensee during the process, and whether the student is used and distributed within the license's limits. Where the contract is silent on any of these, record an assumption and settle the scope in writing first.
License terms
AI dataset distillation
Key takeaways
- Distillation moves value from your data into a student that may never see the records, so training-only wording may not reach it.
- Map the data path first: compression, a new product model, third-party teachers, customer delivery and synthetic data raise different clauses.
- Sending licensed records to an outside teacher is a confidentiality and subprocessor question before it is a scope question.
- Show uncertainty as a status backed by quoted text or a written assumption, never as a percentage.
- Model licenses such as Open RAIL-M and the Gemma terms already define distillation as producing derivatives, so it can be drafted precisely.
Does a data license that permits training also permit model distillation?
Not by default. Whether a training permission reaches distillation depends on three things in the agreement: whether the grant covers models built from a trained model's outputs or only models trained directly on the data, whether the process sends the data to anyone the license does not allow, and whether the resulting student is used and distributed within the license's limits. If the text is silent on any of the three, treat it as an open scope decision.
The reason is that distillation moves what a model learned into a second model without that second model ever seeing the records. A license written around the act of training on the data can be read to cover only the first model, to cover anything downstream of it, or to forbid the student entirely. Each reading has supporters, and the disagreement tends to surface after the student has shipped.
For a company licensing its records, the worksheet in this guide turns the question into inputs that can be filled in from the proposed use and the contract text, a decision rule, and a clear marker for the points where counsel has to decide.
What counts as model distillation, and why does it matter to a data licensor?
Distillation is training a smaller or different model, the student, to reproduce the behavior of an existing model, the teacher, using the teacher's outputs or internal signals as training targets. In the paper that popularized the term, Hinton, Vinyals and Dean describe using the class probabilities produced by a large model as soft targets for training a small model.
Modern variants go further. A teacher can generate full responses to a pool of prompts, which become supervised examples for the student. It can also supply probabilities over its vocabulary, intermediate representations or rankings. Some approaches use the teacher to create synthetic data, and the student trains on that.
For a licensor this matters for two reasons. First, the student carries value derived from your data without being trained on it, so a license that only restricts training on the data may not reach it. Second, some distillation workflows use your records directly as prompts, which can send them to a third-party model provider. Those are different risks and need different contract answers.
Distillation is also not the same as fine-tuning a copy of the teacher. Fine-tuning updates the original weights, so the result is plainly a version of the covered model. A distilled student usually has a different architecture and starts from different weights, which is exactly why parties disagree about whether it is the same model for license purposes. The worksheet treats that question as an input to decide, not a fact to assume.
Finally, the student's value does not depend on how many records it saw. A small student can capture much of a teacher's behavior on the narrow task the licensed data taught, which is why licensors care about it even when no record is copied.
How can distillation touch your licensed data?
Map the proposed use to one of these patterns before reading the contract; the data path decides which clauses matter. The default readings in the last column are assumptions to test with counsel, not conclusions.
Real proposals often combine patterns. A vendor may compress its own model and also hand the student to a customer, or generate synthetic data with a covered model and then distill a third model from it. Fill the worksheet below once per pattern involved, not once per proposal, so each data path gets its own answer.
Ask the licensee to describe the pipeline in writing, including which systems hold the records, which model generates targets and where the student is stored. Engineers can usually answer these questions in a paragraph, and the answers determine which rows of the table apply.
| Pattern | Data path | Clauses that decide it | Default reading if the license is silent |
|---|---|---|---|
| Compress a covered model | Licensee's own teacher trained on your data teaches a smaller student | Covered model definition, permitted uses | Arguable both ways; often treated as part of the licensed model |
| Distill into an unrelated product model | Covered teacher's outputs train a model for a different purpose | Field of use, derived model clause | Risky for the licensor; the student may sit outside every restriction |
| Use records as prompts to a third-party teacher | Your records are sent to an outside model provider to generate answers | Confidentiality, processors and subcontractors, data location | Usually a disclosure question first, a distillation question second |
| Distill for a customer | Student is delivered to or hosted for a licensee's customer | Distribution, sublicensing | Often outside a grant limited to the licensee's internal use |
| Generate synthetic data, then train | Covered model generates data that trains another model | Derived data clause, synthetic data terms | Depends on whether synthetic data is defined as derived data |
Which contract language decides whether distillation is allowed?
Read the grant verb, the definition of the licensed or covered model, any output or derived data clause, and the distribution clause together. The wording below is illustrative, written to show how small differences change the reading; it is not recommended clause text.
Verbs carry more weight than they appear to. Train usually describes updating a model on the data itself. Improve, develop and create can reach further, because a student that serves the same task arguably develops the licensee's capability. When the grant uses broad verbs, the restrictions elsewhere in the agreement do most of the work.
Definitions are the cleanest fix. A covered model definition that names students, plus a sentence on whether students may be distributed, answers most distillation questions before they arise and keeps the grant itself short.
| Illustrative wording | Likely reading on distillation | What is left open |
|---|---|---|
| Licensee may use the Data to train machine learning models | Covers models trained on the data; silent on students trained on outputs | Whether a student is a model trained using the Data |
| Licensee may use the Data to train and improve the Licensed Model | Limited to one named model; a student is arguably a different model | Whether compression of the Licensed Model is improvement |
| Covered Models include any model trained on the outputs of a Covered Model | Students are covered and inherit all restrictions | How long coverage lasts and whether customers may receive students |
| Licensee shall not use any Covered Model output to train any other model | Distillation outside the covered set is prohibited | Whether internal compression counts as another model |
| Data may be processed only by Licensee and its listed subprocessors | Sending records to an outside teacher needs that provider to be listed | Whether a hosted model provider is a subprocessor |
What should the use case and contract language review worksheet include?
It should capture the facts of the proposed use, the contract text that governs each fact, an explicit assumption where the text is silent, and a status for each line. Fill one worksheet per proposed distillation use. The rows below are the input dictionary; the illustrative entries show the level of detail that makes the worksheet useful.
- Decision rule 1: if licensed records leave the licensee during distillation and the license does not allow that recipient, stop; this is a disclosure issue before it is a scope issue.
- Decision rule 2: if the contract defines students as covered models or expressly permits distillation, the use is permitted subject to the covered model's restrictions.
- Decision rule 3: if the contract prohibits output training outside the covered set, a student used outside that set is prohibited.
- Decision rule 4: if the contract is silent, record the assumption, mark the status as needs explicit decision, and resolve it in writing before the student is trained.
| Input | What to record | Illustrative entry | Changes the decision if |
|---|---|---|---|
| Proposed use | One sentence on what is distilled and why | Compress the ticket triage model for on-device use | The purpose falls outside the field of use |
| Teacher | Which model, and whether it is a covered model | Triage model v2, trained on the licensed tickets | The teacher is a third-party model |
| Student inputs | Does the student or the process see licensed records? | No; prompts are synthetic, teacher outputs only | Licensed records are used as prompts |
| Outside parties | Who receives records or outputs during distillation | None | Records go to a provider not allowed by the license |
| Student destination | Who will use, host or receive the student | Licensee's own app, weights not distributed | The student is delivered to a customer or released |
| Grant text | Exact grant and covered model wording, quoted | Train machine learning models using the Data | The grant names a single model |
| Output and derived clauses | Exact wording, or silent | Silent | A clause prohibits output training |
| Term and survival | Do students survive termination? | Models trained in term survive | Students must be deleted with the data |
| Assumption | What you assume where the text is silent, in one line | Student treated as a covered model | Counsel rejects the assumption |
| Status | Permitted, prohibited, or needs explicit decision | Needs explicit decision | Any line above changes |
Is the student a model trained using the data, and how do you record doubt?
The assumption that most often flips the answer is whether the student counts as a model trained using the data. Change that one line and a permitted compression can become an unlicensed derivative, or the reverse. The second is whether licensed records leave the licensee during the process, because that turns a scope question into a confidentiality question.
Show uncertainty as a status, not a probability. A worksheet line is either supported by quoted text, contradicted by quoted text, or silent with a written assumption. Avoid percentages or confidence scores; they suggest a precision the contract does not have and tend to be read as legal conclusions.
A useful habit is to run the worksheet twice: once with the assumption that the student is a covered model and once without it. If the status is the same both ways, the assumption does not matter for this use. If it differs, that line is the one to resolve with counsel before anything else.
When two lines disagree, for example a grant broad enough to cover the student and a distribution clause that forbids giving models to customers, the narrower clause usually governs the specific act. Record both and let counsel confirm the order of precedence in your agreement.
How do model providers' own terms treat distillation?
Several providers and model licenses address distillation directly, which shows the question is recognized and draftable. These terms govern the models they license, not your data, but they are useful reference points and they can also apply to the licensee's own workflow.
The CreativeML Open RAIL-M license defines Derivatives of the Model to include models created by transfer of patterns of the weights, parameters, activations or output of the Model, including distillation methods and methods based on synthetic data generated by the Model. Google's Gemma Terms of Use similarly define Model Derivatives to include models built using distillation or synthetic data Outputs from Gemma, while stating that Outputs themselves are not Model Derivatives.
Other terms restrict output training by purpose or attach conditions. OpenAI's Terms of Use list using Output to develop models that compete with OpenAI among the things users may not do. The Llama 3.1 Community License requires anyone who uses Llama outputs to train an AI model that is distributed or made available to include Llama at the beginning of that model's name.
Open data licenses can point the other way. The Community Data License Agreement Permissive 2.0 defines Results to include machine learning models and imposes no restriction on them. If your records have ever been shared under a license like that, the licensee may argue that models and anything distilled from them are unrestricted.
One practical consequence for licensors: if the licensee plans to use a third-party teacher on prompts drawn from your records, the teacher's own terms add a second layer of restrictions on the student, separate from your license.
Worked scenario: compressing a triage model for a customer deployment
This scenario is illustrative. A field service company licenses de-identified work orders to a software vendor under a grant to train machine learning models for dispatch and triage, for internal use and in the vendor's hosted product. The license is silent on distillation and on delivering models to customers.
The vendor proposes to distill its triage model into a small student that runs on a large customer's own servers. On the worksheet, the teacher is a covered model, no records leave the vendor, and the student inputs are synthetic prompts. Two lines fail: the student's destination is a customer, not the hosted product, and the grant does not say whether students are covered models.
Applying the decision rules, the status is needs explicit decision on the scope question and prohibited on the distribution question as drafted, because the license limits use to the vendor's internal use and hosted product. The parties sign an amendment that defines students as covered models, permits delivery of compiled students to named customers under flow-down terms, and requires a log of each delivery.
The limit of this scenario: had the vendor proposed to send the work orders themselves to an outside model provider to generate training pairs, the analysis would start with the confidentiality and subprocessor clauses, and a distillation amendment alone would not resolve it.
What are the limits of this worksheet, and when is specialist review needed?
The worksheet organizes facts and text; it does not decide how a court or arbitrator would read your agreement under its governing law. Treat every needs explicit decision line as a question for qualified counsel, and every technical claim about what the student sees as a question for the licensee's engineers to confirm in writing.
Request specialist review when records include personal or sensitive data, when a third-party teacher is involved, when the student will be released or sold, or when the licensee asks for perpetual rights in students. Narrow the scope rather than guess: permit compression of covered models now, and leave other distillation uses for a later amendment.
This guide is general information, not legal advice, and it does not predict any legal outcome or performance gain from distillation.
Where distillation belongs in a SourceX license review
Companies licensing through SourceX should fill in the worksheet before signing. Scope and terms are approved by the company ahead of any delivery, and SourceX's later use of the de-identified dataset, sublicensing and distribution included, is capped by the signed text, so a distillation question answered there never needs arguing afterward.
Finding out whether your records qualify starts with a quick company questionnaire; no files are requested at that stage.
Frequently asked questions
Is distillation the same as training on the data?
No. In distillation the student learns from a teacher model's outputs or internal signals rather than from the original records, although some workflows use the records as prompts. That difference is why a license written around training on the data may not clearly reach the student.
Can a licensee distill a model trained on my data without asking?
It depends on the agreement. If the license defines covered models to include students or expressly permits distillation, it may not need to ask; if the license is silent, the use is an open scope question that should be settled in writing before the student is trained.
Does sending my records to an outside AI provider for distillation need permission?
Usually that is a confidentiality and subprocessor question before it is a distillation question. Check whether the license limits who may process the data and whether the provider is an allowed recipient.
Are students covered by the same restrictions as the teacher?
Only if the agreement says so or a court reads it that way. A clause defining models trained on a covered model's outputs as covered models makes students inherit the same field-of-use, distribution and termination terms.
Do model providers' output terms affect my license?
They govern the licensee's use of the provider's model, not your data. They matter when the licensee uses a third-party teacher, because the student may then be subject to both your license and the provider's terms.
Sources
- Hinton, Vinyals and Dean describe distillation as using the class probabilities produced by a large model as soft targets for training a small model. Source
- The CreativeML Open RAIL-M license defines Derivatives of the Model to include distillation methods and methods based on synthetic data generated by the Model. Source
- Google's Gemma Terms of Use define Model Derivatives to include models built through distillation or synthetic data Outputs and state that Outputs are not Model Derivatives. Source
- OpenAI's Terms of Use prohibit using Output to develop models that compete with OpenAI. Source
- The Llama 3.1 Community License requires distributed models trained using Llama outputs to include Llama at the beginning of the model name. Source
- The Community Data License Agreement Permissive 2.0 defines Results to include machine learning models and imposes no restriction on Results. Source
See if your company qualifies
A short company assessment. No data uploads are needed.