Leadership and readiness
Supplier education and data transaction quality: the four-level test
By SourceX Editorial · Reviewed by Noah Loul ·
10 sections
Short answer
Supplier education improves data transaction quality only if it changes what suppliers send, not how many finish the form. Judge it on results: fewer permission gaps found after a deal is confirmed, fewer amendments and more closed datasets per review hour, on cohorts assigned to education. In this article's illustrative cohorts, a guide lowers completion from 80% to 70% yet cuts hours per closed dataset by 12.2%.
Checklist
Supplier education data
Key takeaways
- Judge supplier education on results: late permission gaps, amendments and closed datasets per review hour.
- Completion can fall when education works; in the illustrative model it drops from 80% to 70% while closed datasets rise from 34 to 47.
- A reassurance page that lifts completion to 90% adds 6.3% to hours per closed dataset in the same model, the cost of applications that add no supply.
- Assign education by week or at random and report intention-to-treat effects, because readers select themselves.
- Make completeness a property of the intake format and spend teaching effort on what a form cannot check.
What would show that supplier education improves data transaction quality?
Evidence that educated suppliers send cleaner deals: fewer permission gaps discovered after a deal is confirmed, fewer amendments and more closed datasets per hour of review, measured on cohorts assigned to education rather than on suppliers who chose it. A higher completion rate is not that evidence, and it can point the wrong way.
For a data licensing company, a high-quality transaction reaches Closing on the dataset and terms first agreed, without a permission problem surfacing late, without a re-scope and without a withdrawal over terms the supplier misunderstood. Each failure costs more the later it appears. A missing permission found at intake costs a conversation; the same gap found after the dataset has been fixed in a signed confirmation can cost the deal and the preparation work already done.
Supplier education is anything that teaches a prospective supplier, before or during qualification, what permissions it needs and what the deal will ask of it: a pre-application guide, a permission self-check, worked examples of contract clauses, a plain account of the terms. The investor question is whether it changes what suppliers send, measured against what it costs to write and maintain.
Four levels of evidence, and where completion fits
Use the four levels that training evaluators already apply, and treat completion as a separate and weaker signal. Kirkpatrick Partners describes its model as four levels, Reaction, Learning, Behavior and Results, and defines Learning as the degree to which participants acquire the intended knowledge, skills, attitude, confidence and commitment. Mapped to supplier education, only the last two levels touch transaction quality.
Most supplier education reporting stops at the first level, because opens and ratings are easy to count. Ask for the third and fourth: did applications arrive with the evidence the guide asked for, and did late gaps and amendments fall among closed deals? A company that cannot report the fourth level has not yet shown the effect an investor would pay for. The levels, their measures and their limits:
| Level | What to measure for supplier education | Denominator | What it cannot show |
|---|---|---|---|
| Reaction | Share who open the guide and rate it useful | Suppliers offered the guide | Whether anything they send changes |
| Learning | Correct answers on a short permission self-check, such as who owns client records | Suppliers who took the check | Whether they act on it with real documents |
| Behavior | Applications that arrive with the permission evidence asked for: contract terms, notice versions, platform terms | Completed applications | Whether that evidence survives rights review |
| Results | Permission gaps found after confirmation, amendments, closed datasets per review hour | Closed datasets and confirmations signed | Effort, unless staff hours are logged |
| Completion (not a level) | Applications finished | Applicants | Whether the finishers were the right suppliers |
Which supplier mistakes can education prevent, and when do they surface?
Five mistakes cause most avoidable rework, and each has a cheap point of discovery and an expensive one. Education works by preventing the mistake outright or by moving its discovery earlier.
Permissions are the costliest. SourceX's published deal terms ask a supplier to confirm four things: it holds the rights to license the data, any personal data in it was collected lawfully, nobody has licensed it for AI training before, and AI was not used to make or change it for the sale. A supplier that learns what those statements require before applying can check its customer contracts and notices first. It cannot repair a gap by rewriting its terms afterward: FTC staff wrote in February 2024 that a company that adopts more permissive data practices, such as using consumer data for AI training, and informs consumers only through a surreptitious, retroactive change to its terms of service or privacy policy may be acting unfairly or deceptively.
Expectations are the second lever. SourceX's how-it-works page says pricing stays indicative until a Dataset Deal Confirmation is signed, that licenses are exclusive for AI training, that payment is usually a one-time amount for the dataset and that data which does not reach Closing earns no payment. A supplier that learns this before qualification either proceeds knowingly or leaves early, and both outcomes cost less than a withdrawal after rights review. The five mistakes and their discovery points:
| Mistake | What education teaches | Found early at | Found late at |
|---|---|---|---|
| Customer contracts restrict reuse | Which clauses matter and how to pull them before applying | Rights review | Preparation or acceptance, after the dataset is fixed |
| Privacy notice never covered the use | That a notice cannot be widened quietly after the fact | Rights review | A buyer's diligence |
| Client-owned material in the inventory | Which records belong to clients rather than the company | Qualification | Extraction, when the records sit in client systems |
| Inventory misstated | How to count systems, years of history and exportable fields | Inventory completion | Delivery, forcing an amendment |
| Terms misunderstood | Indicative pricing, exclusivity for AI training, payment only after Closing | The offer | After signature, as a withdrawal or dispute |
The education cohort model: three arms, one ledger
The model compares three illustrative cohorts of 200 applicants assigned by week: a control arm with the standard application, a guide arm that receives a pre-application guide on permissions and terms, and a reassurance arm that receives a page making the process sound easy without teaching anything. Every input is invented to show the arithmetic; the model uses no SourceX data and describes no real company.
At a glance
The education cohort model: three arms, one ledger
Read the guide column against the control. Fewer applicants finish (140 against 160) because some learn they lack the rights or dislike the terms, yet more qualify (63 against 56), about half as many permission gaps reach rights review (9 against 17) and late gaps fall from 4 to 2. Closed datasets rise from 34 to 47. The guide arm spends more staff hours in total, 2,498 against 2,057, because it carries more deals through preparation, but hours per closed dataset fall from 60.5 to 53.1, a 12.2% drop that already includes 80 hours of guide upkeep and supplier questions.
The reassurance column is the warning. Completion rises to 90%, yet qualified suppliers, gaps and closed datasets match the control exactly, so the extra 20 applications and the page's own upkeep add hours and nothing else, and hours per closed dataset rise 6.3% to 64.3. More applications can raise expense without raising accepted supply, and a completion dashboard would have scored this arm the winner. The three arms side by side:
| Measure (illustrative) | Control | Guide | Reassurance page |
|---|---|---|---|
| Completed applications, of 200 | 160 (80%) | 140 (70%) | 180 (90%) |
| Qualified suppliers | 56 | 63 | 56 |
| Permission gaps found at rights review | 17 | 9 | 17 |
| Withdrawals over terms before confirmation | 6 | 3 | 6 |
| Confirmations signed | 42 | 56 | 42 |
| Permission gaps found after confirmation | 4 (2 deals lost) | 2 (1 deal lost) | 4 (2 deals lost) |
| Amendments | 10 of 42 | 7 of 56 | 10 of 42 |
| Closed datasets | 34 | 47 | 34 |
| Staff hours (per closed dataset) | 2,057 (60.5) | 2,498 (53.1) | 2,187 (64.3) |
Why completion rates mislead, and what a better gate looks like
Completion measures whether suppliers finished a form, not whether they sent a licensable dataset, so it rewards the reassurance page and punishes the guide. Separate two gates: completeness, which a well-designed intake can enforce, and quality, which only review can judge.
Medical device regulation draws the same line. FDA's refuse-to-accept guidance for 510(k) submissions says the acceptance review checks whether a submission is administratively complete, leaving the adequacy of its data to the substantive review. FDA recommends that submitters include the acceptance checklist, showing where each required element sits, and says submissions prepared with its eSTAR template should represent a complete submission and are therefore not expected to go through the acceptance review. The lesson for supplier education is to make completeness a property of the format, with a guided intake that asks for the permission evidence in a fixed place, and to spend teaching effort on what a form cannot check.
Then measure the quality gate in the units that matter: late permission gaps per 10 closed datasets (1.2 in the control arm, 0.4 with the guide), amendments per confirmation signed (10 of 42 against 7 of 56) and hours per closed dataset. If those do not move, the education is decoration, however many suppliers finish.
Is it education or selection? Test it with assignment, not choice
The strongest alternative is selection: suppliers who read guides may already be better prepared, so comparing readers with non-readers credits the guide for their quality. The remedy is to assign the guide, by week or at random, and to analyze every assigned supplier whether or not it opened the material.
That intention-to-treat comparison is the honest headline. In the illustrative model it is 6.5 more closed datasets per 100 applicants (23.5 against 17.0). If 60% of assigned suppliers opened the guide, dividing by that share estimates the effect on suppliers who opened it because they were assigned: 10.8 per 100. Imbens and Wooldridge's lecture notes on instrumental variables are explicit that such a ratio identifies the average effect only for the subpopulation the assignment moved, the compliers, so suppliers who would always or never have read a guide sit outside it.
Guardrails keep the test honest. Fix the outcome before starting, closed datasets per 100 applicants, with hours per closed dataset and late gaps as secondary measures. Keep review standards identical across arms, count suppliers who leave after reading the guide rather than dropping them, and wait until enough deals have closed or been censored to report an interval instead of a point.
What does supplier education do to review capacity?
It moves work earlier, where it can be scheduled, and shrinks the repair work that interrupts preparation. In the model, hours spent repairing deals after confirmation, on late-gap handling and amendments, fall from 108 to 66 even though the guide arm signs 14 more confirmations; per closed dataset that is 3.2 hours against 1.4.
Early permission work grows with volume instead. Rights review and the rework of gaps found there take 381 hours in the control arm and 403 with the guide, because more suppliers qualify, but per closed dataset the load falls from 11.2 hours to 8.6. A capacity plan built on the control arm would understaff rights review the month education starts and overstaff the repair queue, so plan each stage from the cohort that will actually arrive and recheck after the first closed deals.
What could make the education thesis wrong
Five developments could erase the effect, and a company should be able to say which worries it most.
At a glance
What could make the education thesis wrong
- Selection dressed as teaching: a guide that mostly repels weak applicants lowers rework but may also repel good suppliers who misread it. Review a sample of leavers to count how many would have qualified.
- Stale content: contract practice and privacy enforcement move, and a guide written a year ago can teach the wrong check. Date every page and review it on a schedule.
- Answers instead of evidence: suppliers learn what a self-check wants to hear. Score the documents they send, not the answers they give.
- Small cohorts: with a few dozen closed deals per arm, a difference of a handful of deals can be noise. Report intervals and resist quarterly conclusions.
- Shifting demand: if buyers change what they accept, gains measured against the old standard stop mattering. Tie the results level to the current acceptance test.
Questions to ask any data company about supplier education, SourceX included
Ask for results-level numbers on assigned cohorts, using definitions you set. SourceX is one application of the question, because its public pages already carry supplier education of this kind.
Its application says no files, passwords or API tokens are needed, that nothing is licensed or transferred by applying and that an application alone does not mean an offer or payment will follow. Its how-it-works page sets out what a supplier confirms, when pricing stops being indicative and what happens to data that does not close. The screening guidance it publishes for referral partners treats the rights to license as a core check and says client-owned material is usually not a fit. Whether those pages lower late permission gaps is exactly what the measures above would show. SourceX has not published such results; it shares company and commercial metrics with investors directly, and its Rule 506(c) raise is open only to accredited investors, with materials available on request through its investors page.
Questions to bring to that conversation, or to any other data company:
Checklist · 5 items
- Which suppliers were assigned education, by what rule, and how did assigned and unassigned cohorts compare on closed datasets per 100 applicants?
- How many permission gaps surfaced after a deal was confirmed, per 10 closed datasets, before and after the education began?
- How many amendments per confirmation signed, and what caused them?
- How many staff hours per closed dataset, including the hours spent writing and maintaining the education?
- Of suppliers who left after reading the material, how many would have qualified?
0 of 5 ticked. Tick each item as you go.
Methods note: the cohort model's inputs, rules and limits
The cohort model was built for this article on 2026-10-28, and every input is illustrative. Each arm starts with 200 applicants. Rates for completion, qualification, permission gaps at rights review, withdrawals over terms, permission gaps after confirmation, re-scopes and acceptance are set per arm: control 0.80, 0.35, 0.30, 0.12, 0.10, 0.20 and 0.85; guide 0.70, 0.45, 0.15, 0.05, 0.04, 0.10 and 0.85; reassurance 0.90 and 0.31, then the control's rates. Each rate applies to the suppliers still in the cohort at that stage: completion to applicants, qualification to completed applications, rights-review gaps to qualified suppliers, withdrawals to qualified suppliers left after early drops, late gaps to confirmations, and re-scopes and acceptance to deals still alive after late gaps. Counts are rounded to whole suppliers at each stage, halves up. Half of the gaps found at rights review are fixed with rework, the rounded half going to fixed, and the rest drop out; 60% of gaps found after confirmation lose the deal, rounded, and the rest are amended, as are re-scoped deals.
Early permission work is rights review plus rework of gaps found there; late repair work is late-gap handling plus amendments. Staff hours are 2.5 per completed application for intake and qualification review, 6 per qualified supplier for rights review, 5 per gap fixed at rights review, 4 per confirmation, 12 per late gap, 6 per amendment and 25 per dataset delivered (every deal still alive after late gaps) for preparation and acceptance. The guide and reassurance arms each add 20 hours of upkeep and half an hour per supplier who opened the material, with an opened share of 60%. The complier estimate divides the intention-to-treat difference by that share. Ratios are computed from unrounded values and rounded once. A script, xs0192_model.py, reproduces every figure in the article.
Sources: Kirkpatrick Partners' description of its four levels, FTC staff's February 2024 post on retroactive changes to terms, FDA's refuse-to-accept guidance for 510(k)s, Imbens and Wooldridge's lecture notes on local average treatment effects, and SourceX's application, how-it-works, investors and screening pages, read on 2026-10-28. Not measured: any real education effect at SourceX or elsewhere, the share of leavers who would have qualified, or how long an effect lasts. No SourceX results are reported.
Frequently asked questions
Does supplier education raise or lower application completion?
It can lower it, and that can be good news. A guide that explains permissions and terms leads some unsuitable suppliers to stop early, so completion falls while closed datasets per applicant rise. Judge it on results, not completion.
What is the best single metric for supplier education?
Closed datasets per 100 assigned applicants, read alongside hours per closed dataset and late permission gaps per 10 closed datasets. Completion and guide opens are useful diagnostics but say nothing about transaction quality.
How do you separate the effect of education from supplier selection?
Assign the education by week or at random and compare every assigned supplier with every unassigned one, whether or not they read it. Dividing that difference by the share who opened the material estimates the effect on those who read it because they were assigned.
Why are late permission gaps so costly?
By then the dataset and terms are fixed in a signed confirmation and preparation work may already be done. A gap found at intake costs a conversation; the same gap after confirmation can cost an amendment or the whole deal.
Has SourceX published supplier education results?
No. SourceX publishes its application, deal steps and screening guidance, and it shares company and commercial metrics with investors directly. The measures in this article are the ones to ask for.
Sources
- SourceX's how-it-works page lists what a supplier confirms, says pricing stays indicative until a Dataset Deal Confirmation is signed, that licenses are exclusive for AI training and that data that does not reach Closing earns no payment. Source
- SourceX's application says no files, passwords or API tokens are needed, nothing is licensed or transferred, and an application alone does not mean an offer or payment will follow. Source
- SourceX's screening guidance for referral partners lists the rights to license as a check and treats client-owned material as usually not a fit. Source
- SourceX's investors page says the company shares company and commercial metrics with investors directly. Source
- FTC staff wrote in February 2024 that adopting more permissive data practices, such as AI training, through a surreptitious, retroactive change to terms or a privacy policy may be unfair or deceptive. Source
- Kirkpatrick Partners describes four levels of training evaluation, Reaction, Learning, Behavior and Results, and defines Learning as acquiring the intended knowledge, skills, attitude, confidence and commitment. Source
- FDA's refuse-to-accept guidance says the 510(k) acceptance review checks administrative completeness rather than data adequacy, recommends submitting the acceptance checklist and expects eSTAR submissions to be complete. Source
- Imbens and Wooldridge's lecture notes explain that an instrument identifies the average effect only for the subpopulation it induces to change treatment, the compliers. Source
Request SourceX investor materials
SourceX shares company and commercial metrics with investors directly. Any offering is made only to verified accredited investors.