Getting started
Data supplier experience research: what to count after a deal closes
By SourceX Editorial · Reviewed by Noah Loul ·
10 sections
Short answer
Data supplier experience research should compare what each company signed with what actually happened, on three measures: clarity (could it predict scope, conditions and payment triggers from the documents), effort (logged internal hours by role against the estimate) and outcome (delivered scope and payment events reconciled to the signed terms). Count every completed transaction in a fixed window, report non-response, and let approval govern quoting, never inclusion.
Process
Data supplier experience
Key takeaways
- A testimonial answers a marketing question; supplier experience research answers a decision question, and only a denominator turns one into the other.
- Measure clarity as the match between what a company expected before signing and what the signed terms produced, not as a satisfaction rating.
- Count effort in logged hours by role, with the source of each number, against an estimate made before work started.
- Approval to publish should decide whether a company is named or quoted, never whether its result is counted.
- Results from completed transactions describe survivors; deals that ended without Closing belong in the same report.
What should data supplier experience research measure?
It should measure whether a licensing transaction went the way the company's signed documents said it would, seen from the supplier's side, on three dimensions: clarity, effort and outcome. Done well, it tells a company owner considering data licensing what preparing, signing and delivering will ask of the business, and how often the terms held, without implying that any one company's experience will repeat.
Two readers need it. A company weighing its first deal wants to judge the evidence an intermediary or a buyer shows it: quotes, case studies, satisfaction scores. A company that has just finished a deal wants a debrief that tells it whether to do another and what to change. Both are asking a decision question rather than looking for reassurance, so both need the same discipline.
The decision itself is narrow: is this route to licensing worth the internal effort and the commitments in its documents, for a company like ours? The evidence that answers it is equally specific: the signed terms, dated records of what happened, logged hours, and a count of every comparable company, including the ones whose deals went badly or never closed.
Why does a glowing supplier quote prove so little?
Because a quote is one experience that someone chose to show. Without the number of transactions it was drawn from, a reader cannot tell whether it describes the usual result or the best one.
U.S. advertising law names the same problem. The FTC's Endorsement Guides (16 CFR 255.2) say an endorsement describing a consumer's experience with a central attribute is likely to be read as representing what consumers generally achieve, so an advertiser without substantiation for that impression should clearly disclose the generally expected performance. The examples in the guides add that a bare 'results not typical' line does not fix the impression.
The FTC's final rule on consumer reviews and testimonials, announced in August 2024 and in effect since October 21, 2024, targets selection directly. It bars suppressing negative reviews through threats or intimidation, and it bars claiming that displayed reviews are representative when negative ones have been suppressed. Both texts are written for consumer marketing, and a business-to-business licensing debrief may sit outside them. Their logic still holds for anyone reading supplier evidence: a curated quote shows what a company was willing to say in public, not what happened to the companies that stayed quiet.
Clarity: could the company predict the result from what it signed?
Clarity is the share of material terms whose real result the company predicted correctly before signing. Ask about each term twice, once as an expectation and once as an outcome, and score the match; a pleasant process that produced surprising terms was not clear.
A data licensing transaction has few material terms, and they are concrete: exactly which records, systems and years were in scope; which conditions had to be met and by whom; which event made payment due; which restrictions, such as exclusivity and term, applied after signing; and who de-identified the data and by what route. A company that misread any of these was surprised, however friendly the calls felt.
Unclear terms are a known weakness of data licensing in general. The MIT-led audit of dataset licensing and attribution (Longpre and colleagues, Nature Machine Intelligence, 2024) traced more than 1,800 text datasets and found licenses frequently omitted or miscategorized on widely used hosting sites. If public datasets often travel without accurate terms, a private transaction's terms deserve to be checked line by line rather than assumed understood.
Capture expectations before the outcome is known, ideally at signing, and store them with a date. Questions asked only after Closing invite hindsight, because people remember having expected whatever happened.
Effort: how many internal hours did the deal really take?
Count effort in hours by role, taken from records, and compare it with the estimate made before work began. A score on a five-point ease scale is a perception; it belongs in the debrief only next to the logged hours it is meant to explain.
Five roles carry most of the internal work in a licensing deal, and each has a natural record of its time. Log rework in its own column: a second export because a field was missing, a scope question answered twice, or a privacy notice re-read after the dataset changed are the costs a better process removes, and a single total hides them.
The worksheet below is editable. The roles are typical rather than required, and a company with outside counsel or a one-person IT team will merge or split rows. What matters is that every hour has a source:
| Role | Work to log | Source of the hours | Rework to log separately |
|---|---|---|---|
| Executive sponsor | Scope decisions, approvals and signatures | Calendar entries for deal meetings and reviews | Decisions reopened after a scope change |
| Systems owner | Inventory, sample and full exports, transfer | Ticket or project log entries | Repeat exports for missing fields or years |
| Counsel or contract reader | Customer contracts, privacy notices and the deal documents | Outside counsel invoices or an internal time log | Documents re-read after the dataset changed |
| Privacy or security reviewer | Personal-data review, de-identification questions, transfer security | Review tickets and sign-off records | Re-review after a new field or source was added |
| Subject expert | Explaining what fields, statuses and codes mean | Calendar and message threads with the deal team | The same question answered twice |
Outcome: did scope, conditions and payment triggers match the signed terms?
Outcome is reconciliation, not satisfaction: list what the signed documents fixed and check each item against dated evidence of what happened. The debrief records matches, documented changes and unexplained differences.
Scope compares the records, systems and years delivered with those the signed documents named, and notes each change with the signed document that authorized it. Conditions lists every condition, who had to meet it and the date it was met or waived. The payment trigger names the event that made payment due under the terms and the date it occurred. Money itself belongs in a separate payout bridge, from the agreed price to what reached the bank, which the payout reconciliation guide linked from this page covers.
Use the terms as signed, including later amendments, never the first estimate. At SourceX, for example, pricing stays indicative until the deal's Dataset Deal Confirmation is signed, and data that does not reach Closing earns no payment, so an early estimate is the wrong baseline for judging outcome.
The post-closing debrief worksheet
One row per measure, each a comparison between what was signed or expected and what happened. Fill the evidence column from records rather than memory, and agree the flag rules before anyone discusses results, so a disappointing row cannot be argued away afterwards.
The same worksheet serves one company's own debrief and, with identical definitions, a study across many companies. Six rows:
| Measure | Compare | Evidence to keep | Flag when |
|---|---|---|---|
| Scope | Records, systems and years expected at signing vs delivered | Signed documents, amendments, delivery manifest | Delivered scope that no signed document authorized |
| Conditions | Each condition's owner and expected date vs actual date or waiver | Condition list, notices, dated emails | A condition the company did not know it owned |
| Payment trigger | The event that made payment due vs when it occurred | Closing record or equivalent, invoice date | The company could not name the trigger before it happened |
| Internal effort | Estimated vs logged hours by role, with rework | Effort worksheet, calendars, counsel invoices | Actual hours exceed the estimate by a margin set in advance |
| Explanations | Questions the company asked vs those answered in writing | A dated question log | A material question answered only verbally |
| Approval to cite | What the company agreed may be published, named or anonymous | Signed approval with scope and date | Any published use beyond the approval |
Running the study across many suppliers: denominators, response and approval
A study across companies needs a population fixed in advance, a reported response rate and a separate approval step for publication. Without all three, the report is a set of testimonials with a chart attached.
Population: every transaction that reached Closing in a stated window, plus, as a comparison group, every transaction that ended without Closing in the same window. Invite all of them at the same interval after the outcome, chosen before the study starts, and give each a final disposition: completed, refused, never reached or ineligible. AAPOR's Standard Definitions (10th edition, 2023) set out how to code final dispositions and calculate outcome rates, and reporting those codes is part of meeting its minimal disclosure standard.
Non-response: report the response rate with its numerator and denominator, then compare responders with non-responders on facts already known, such as company size, record types and whether the deal closed. The U.S. Office of Management and Budget's standards for federal statistical surveys call for a nonresponse bias analysis whenever the overall unit response rate is below 80 percent. A small supplier study will rarely reach that rate, so the comparison is part of the method, not an extra.
Approval: ask for three separate permissions, to take part, to be quoted without a name and to be named, and record each with its scope and date. The ICC/ESOMAR code defines consent as freely given, specific and informed, and requires separate consent before research data feeds promotional or customer-experience follow-up. A company that declines to be quoted is still counted.
Report every measure as a count over its stated denominator, alongside the window, the response rate and the share of deals that did not close. SourceX has not published results from a study like this; the design is published so that any intermediary's supplier evidence, its own included, can be checked against it.
Could a strong score mislead? Five rival explanations to rule out
Yes. Before reading a good result as proof that a licensing route works, rule out five explanations that produce the same score with no merit in the process:
At a glance
Could a strong score mislead? Five rival explanations to rule out
- Survivorship: only completed deals were asked. Eligibility, an indicative estimate or even an offer does not mean a deal will be selected, close or pay a particular amount, so companies whose deals stopped early hold the missing half of the evidence.
- Company mix: larger companies with in-house counsel report less effort for reasons unrelated to the process. Compare like with like, by size and record type.
- Timing: memories fade and a recent payment improves moods. Ask everyone at the same interval after the outcome.
- Sponsor effect: feedback requested by the person who ran the deal, on the intermediary's letterhead, tends to be kinder. Use a neutral collector where possible and say who asked.
- Approval filtering: if only responses approved for publication are counted, the score measures willingness to be quoted. Count everyone, and let approval govern attribution only.
The decision memo: proceed, ask for more, or wait
Write the conclusion in a short memo so the evidence produces a decision instead of a mood. The memo states what the company concluded, on which evidence, and what it still needs before acting.
Its limits belong in it: it covers one licensing route on one date, reports no study result of its own, and is neither legal advice nor a forecast of any payment. The fields:
- Decision: proceed to a first assessment, ask for specific evidence first, or wait for a stated change.
- Evidence seen: each item with its date, source and denominator, or a note that it had none.
- Clarity check: the material terms the company can explain in its own words before signing.
- Effort estimate: hours by role for the company's own systems, taken from the effort worksheet.
- Rival explanations: which of the five the evidence could not rule out.
- Next evidence needed: for example a written answer on payment triggers, or a count of deals that did not close.
- Owner of the memo and the date it will be reviewed.
What SourceX publishes, and what it has not yet published
SourceX has not published supplier experience results, so there is no score of its own to weigh here. What it does publish is the order of commitments. It starts with a short company assessment that asks for no data uploads, and the company's approval of scope and terms, followed by its signature, comes ahead of any delivery of data.
The How it works page names the documents a debrief would reconcile against: one Master Supplier Licensing Agreement signed after qualification, a short Dataset Deal Confirmation for each deal that fixes the exact dataset, the payout, rights, the processing route and the deadlines, and a Closing Notice once SourceX has accepted the data in writing and every condition is met. An assessment is a fit check, not an offer, and it promises no payout.
Frequently asked questions
Is a supplier testimonial evidence that data licensing works?
Only weakly. A testimonial shows one company's experience that someone chose to publish, and without the number of comparable transactions and how many companies were asked, a reader cannot tell whether it is typical. Treat it as a reason to ask for counts, not as a result.
What response rate does a supplier experience survey need?
No rate makes a survey trustworthy on its own. Report it with its numerator and denominator, using standard disposition codes such as AAPOR's, and compare responders with non-responders on known facts such as company size and whether the deal closed. Federal statistical standards in the U.S. call for that comparison whenever unit response falls below 80 percent.
When should a company collect feedback after a data licensing deal?
At a fixed interval after the outcome, chosen before anyone is asked, so every company answers at the same distance from the event. Record expectations earlier, ideally at signing, because questions asked only after Closing invite hindsight. Companies whose deals did not close should be asked at the same interval.
Can an intermediary publish only the feedback companies approve?
It can and should limit names and quotes to what each company approves. It should not limit the count: a result built only from approved responses measures willingness to be quoted, not experience. Every response belongs in the totals, and approval decides only what is attributed.
Does SourceX publish supplier satisfaction or experience scores?
No. SourceX has not published supplier experience results. A company can begin with a short assessment that needs no data uploads and ask to see the documents a deal would use; the assessment is a fit check, not an offer and not a promise of payout.
Sources
- The FTC's Endorsement Guides (16 CFR 255.2) say an endorsement about a central attribute is likely read as representing generally achieved results, so without substantiation the generally expected performance should be disclosed; 'results not typical' alone does not suffice. Source
- The FTC announced its final rule on consumer reviews and testimonials in August 2024 (effective October 21, 2024); it bars review suppression through threats or intimidation and claims that displayed reviews are representative when negative ones were suppressed. Source
- The MIT-led audit of dataset licensing and attribution (Nature Machine Intelligence, 2024) traced more than 1,800 text datasets and found frequent license omission and miscategorization on widely used hosting sites. Source
- AAPOR's Standard Definitions (10th edition, 2023) give a standard way to code final case dispositions and calculate outcome rates; reporting the codes meets its minimal disclosure standard. Source
- OMB's Standards and Guidelines for Statistical Surveys (Guideline 3.2.9) call for a nonresponse bias analysis when a survey's overall unit response rate is below 80 percent. Source
- The ICC/ESOMAR International Code (2025) defines consent as freely given, specific and informed, and requires separate consent before research data is used for promotional or customer-experience follow-up. Source
- SourceX's How it works page: pricing stays indicative until a Dataset Deal Confirmation is signed; one Master after qualification; a Confirmation fixes dataset, payout, rights, processing route and deadlines; Closing after written acceptance with a Closing Notice; data that does not reach Closing earns no payment. Source
See if your company qualifies
A short company assessment. No data uploads are needed.