AI data market
AI data procurement requirements template: write lines suppliers can answer precisely
By SourceX Editorial · Reviewed by Noah Loul ·
10 sections
Short answer
A dataset RFP gets precise answers when every requirement names a unit, a threshold and the test that will check it, and tells the supplier exactly what form the answer takes: a number, a coded status and the evidence attached. Write ten or so such lines covering the record, fields, volume, coverage, label accuracy, language, duplicates, personal data, rights and documentation, then reuse the same lines as pilot and acceptance tests.
Checklist
AI data procurement
Key takeaways
- A requirement is answerable when it has a unit, a threshold, a test method and a fixed answer form; anything else invites a sales paragraph.
- Describe the records you need, not the model gain you hope for: suppliers can measure fields and coverage, not your future accuracy.
- Ask suppliers to answer each line as met, partly met with a stated deviation, or not offered, plus a number and an evidence reference.
- The pilot sample should test the same lines with the same methods the final acceptance will use, so the pilot result predicts the delivery.
- Mark which lines are gates and which are weighted preferences before responses arrive, so no supplier's answer reshapes the criteria.
What makes a dataset requirement answerable?
A requirement is answerable when two suppliers reading it would produce answers you can put side by side without interpretation. That takes four parts: the object being measured, the unit, the threshold, and the method that will check it. Representative support conversations fails all four; at least 95 percent of conversations include the final resolution code, checked on a random sample of 300, passes all four.
Data quality standards offer a vocabulary for the measured object. ISO/IEC 25012 sets out a general data quality model for structured data with fifteen characteristics, seen from an inherent and a system-dependent point of view, and lists setting data quality requirements among its uses. ISO/IEC 5259-2, published in 2024, extends that work with data quality measures for analytics and machine learning. You do not need to cite the standards in the RFP, but borrowing terms such as completeness and accuracy keeps requirements from overlapping.
The fourth part, the method, is what most RFPs leave out. If the requirement does not say who measures, on what sample and with which rule, each supplier will pick the method that flatters its data, and the numbers will not compare.
Which parts of the business need belong in the requirements?
Put the properties of the records in the requirements and keep the model outcome out. A supplier can measure how many records have an outcome field, what date range they span and what share are in English. It cannot measure what your model will score after training, because that depends on your base model, your mix and your training run.
Translate the need in three steps. State the task in one sentence, such as classify inbound claims by line of business. Name the record that teaches it, such as a closed claim with the adjuster's final coding. Then list the properties of that record that decide usefulness: fields, history, coverage across the cases that matter, label quality and rights.
Regulated uses add a reason to be explicit. For high-risk systems, Article 10 of the EU AI Act requires training, validation and testing data sets to be relevant, sufficiently representative and, to the best extent possible, free of errors and complete in view of the intended purpose. Even outside that scope, those words are a useful checklist for what your requirement lines should make measurable.
The dataset requirements template with acceptance tests
Copy the ten lines below into the requirements section of your RFP and replace the illustrative thresholds with your own. Each line tells the supplier what to answer and tells your team how the answer will be checked at the pilot and again at delivery.
Two cautions apply when adapting it. Keep each line to one measurable property, because a line that bundles fill rate and accuracy can only be answered as partly met, which tells you nothing. And state the measurement base in every line, whether the full offered pool, a random sample or the delivered files, since the same fill rate computed on different bases is not the same claim.
Add lines only where a specific risk justifies them, such as a timestamp precision line for a forecasting model or a speaker-separation line for call audio. Ten to fifteen lines is a workable ceiling; past that, suppliers start answering the easy lines carefully and the hard ones loosely.
| Requirement line | Measurable statement (illustrative threshold) | Supplier answers with | Checked at pilot and acceptance by |
|---|---|---|---|
| R1 Record unit | One record equals one closed case with its full message thread and final outcome | Definition of the record in its source system and a sample record with values masked | Comparing 20 sample records with the stated definition |
| R2 Required fields | Outcome code, timestamps and channel present in at least 95 percent of records | Field list with fill rate per field, measured on the full set | Recomputing fill rates on the pilot sample and on delivery |
| R3 Volume and history | At least 50,000 records spanning 24 or more consecutive months | Record count by month for the offered window | Count by month on the delivery manifest |
| R4 Coverage strata | No single customer above 5 percent of records; each of 6 named categories at least 3 percent | Distribution table across the named strata | Recomputing the distribution on pilot and delivery |
| R5 Outcome or label accuracy | Outcome code agrees with a reviewer's reading in at least 90 percent of audited records | How outcomes were recorded and any prior audit result | Blind audit of a random sample by your reviewers |
| R6 Language | At least 95 percent of records in English, measured per record | Language share with the detection method named | Running the same detector on pilot and delivery |
| R7 Duplicates and overlap | Exact and near-duplicate rate below 2 percent; no overlap with your held-out set | Deduplication method and measured rate | Your dedup run, including against your evaluation data |
| R8 Personal data handling | Direct identifiers removed or replaced; residual rate below an agreed limit on audit | De-identification method, who performs it and residual check results | Audit sample checked for residual identifiers |
| R9 Rights and permitted uses | Supplier holds the right to license the records for model training, evaluation and the stated retention | Rights basis, permitted uses and any carve-outs, with document references | Legal review before the pilot and before signature |
| R10 Documentation and format | Datasheet answered in full; JSONL or Parquet with a field dictionary | Completed datasheet and schema file | Schema validation of every delivered file |
How do you rewrite a vague ask into a measurable one?
Most vague requirements hide a real concern that can be measured once it is named. Ask what would make you reject a delivery, then write that rejection condition as the requirement.
Test each rewrite by asking a colleague who did not write it to describe how they would check it. If their method differs from yours, the line still leaves room for interpretation, and a supplier will find the same room.
| Vague ask | Measurable rewrite | Why suppliers can now answer |
|---|---|---|
| High-quality data | Outcome code agrees with reviewer reading in at least 90 percent of a random audit of 300 records | Names the field, the threshold and the audit method |
| Diverse records | At least 3 percent of records in each of six named categories; no single customer above 5 percent | Gives strata the supplier can count in its own system |
| Recent data | At least half of records created in the last 24 months, by record creation date | Specifies which date field and what share |
| Clean of personal data | Direct identifiers replaced; residual identifiers in fewer than 1 in 500 audited records | Turns a promise into a sampled check |
| Usable format | JSONL, one record per line, field dictionary supplied, UTF-8 | Lets the supplier confirm or quote the conversion work |
How should suppliers format their answers?
Give suppliers a response form with one row per requirement line and three fixed columns: a status code, a measured value with its unit, and an evidence reference. Free text goes in a fourth column for deviations only. Without this form, responses arrive as narrative and someone on your team spends days converting them back into the table you should have sent.
Use three status codes and define them in the RFP. Met means the measured value meets the threshold by the stated method. Partly met means the supplier offers something close and states the deviation in numbers, such as an 88 percent fill rate against a 95 percent requirement. Not offered means the supplier cannot meet the line at all, which is still useful because it narrows the field honestly.
For the documentation line, point suppliers at an existing structure rather than inventing one. Datasheets for Datasets groups its questions by motivation, composition, collection process, preprocessing and labeling, uses, distribution and maintenance. The MLCommons Croissant RAI vocabulary adds machine-readable properties such as dataCollection, dataCollectionTimeframe and dataAnnotationProtocol. Asking for either lets you compare documentation completeness across suppliers instead of judging prose.
What must the pilot sample establish before the larger commitment?
The pilot has to show that the measured values in the supplier's response hold on records you check yourself, using the same methods your acceptance will use. If the response says 95 percent fill rate and the pilot shows 80, the response is the problem, and the larger commitment should wait.
At a glance
What must the pilot sample establish before the larger commitment
Treat the pilot like an incoming inspection, not a demo. Industrial sampling standards such as ISO 2859-1, which provides sampling schemes indexed by an acceptance quality limit for lot-by-lot inspection by attributes, rest on the same idea: decide in advance how many items you will inspect and how many defects you will accept. For a dataset, the sample should be drawn at random from the offered pool by a method both sides agree, not hand-picked by the supplier.
If the pilot also includes a training or evaluation run, size it for the effect you care about. Card and colleagues found that small, underpowered comparisons are common in language research and that their significant results tend to overstate, or even reverse, the true effect. A pilot that is too small for a model test should be limited to the data checks in R1 through R10.
- Draw method: random sample from the full offered pool, with the seed or query recorded.
- Checks: every requirement line that can be tested on a sample, using the acceptance method.
- Pass rule: written before the sample arrives, line by line.
- Rights: a pilot license covering only evaluation of the sample, with deletion terms.
- Outcome: proceed, renegotiate named lines, or stop.
Which costs belong in the acquisition model besides the license?
Ask suppliers to state what they will deliver and in what condition, then estimate your own costs for everything else. A license price that excludes conversion, de-identification review or evaluation compute will look cheaper than an offer that includes them.
List the cost lines in the RFP so every supplier states which ones its offer covers: extraction from the source system, format conversion, de-identification and the residual audit, delivery and storage, your reviewers' audit hours, legal review, evaluation and training compute for the pilot, and the ongoing cost of refreshes if the deal includes them. Supplier continuity belongs here too: ask how long the supplier expects to be able to supply records of the same kind, and what happens if its source system changes.
Which lines are gates and which are preferences?
Decide before responses arrive which lines eliminate a supplier and which only affect the score. Rights, personal data handling and the record unit are usually gates, because no amount of volume compensates for records you cannot use. Coverage, history length and format are usually weighted, because a shortfall can be priced or fixed.
Write the exception rule too. A supplier that partly meets a gate line should be able to propose a remedy with a date, such as completing a missing field from another system, and your team decides once, in writing, whether the remedy is acceptable. That keeps one strong response from quietly rewriting the requirements for everyone.
Publish the weights for the scored lines in the RFP itself. Suppliers then know where a shortfall costs them, and a supplier that cannot reach the history threshold can decide early whether to compete on coverage or format instead of spending weeks on a response that a gate will end.
Worked example (illustrative): contact-center records for a routing model
Everything in this example is invented to show the method. A team training a ticket-routing model issues an RFP with lines R1 through R10. Three suppliers respond. One answers in narrative and is asked to resubmit on the form; the other two return coded rows.
Supplier A marks every line met. Its pilot sample of 300 records shows outcome agreement of 92 percent and a fill rate of 96 percent, consistent with its response. Supplier B marks R3 partly met, offering 18 months of history instead of 24, and R4 met. Its pilot confirms both.
The meaningful exception is R9. Supplier A's rights basis covers evaluation but leaves training uses unclear, so legal review holds it at the gate even though its data checks are strongest. The team proceeds with Supplier B on a narrower history window and asks Supplier A to come back with a clarified rights basis, rather than relaxing the gate.
Because both remaining suppliers answered on the same form, the comparison took one review meeting rather than a week of reconciling narratives, and the reasons for each decision were already written next to the line they concerned.
Sending a finished requirements set to a sourcing partner
Once the ten lines are written, they also work as a request to a sourcing partner rather than a formal RFP. SourceX keeps no catalog; it assembles each dataset from supplying U.S. businesses after a buyer asks, and its request form takes the template's own inputs: scope, volume, language, format, timing and licensing needs. Very large deliveries remain in the supplying company's own storage or travel on encrypted drives, so state your delivery and access expectations in R10.
Which unit should the record count use: messages, cases, decisions or task episodes?
Count the unit your model will be judged on. For most enterprise work that is the task episode: one attempt at one piece of work, from the trigger to an observed outcome, even when the work spans several threads or record IDs. Choosing the AI dataset unit of work before anyone counts examples keeps R1 and R3 describing the same thing.
Evaluation teams already count this way. In Anthropic's engineering guide to agent evals, published January 9, 2026, the unit is the task: one test whose inputs and success criteria are spelled out in advance. Each attempt at it is a trial, and the outcome is whatever state the environment is left in when the trial ends. A dataset counted in messages or ticket IDs cannot be lined up against tasks scored that way until it is reconciled.
The table reconciles one supplier export against an R3 floor of 50,000 complete episodes; every number in it is invented to show the method. Ask each supplier to report all four counts on the response form, and name in R3 which count the threshold applies to.
- Message or event: one email, chat turn, comment or log line. It inflates coverage on long threads and lets one talkative customer dominate. Count it only when the model's job is one reply or one label per message.
- Case or ticket ID: one record in the source system. Merges, follow-ups and duplicates split or double a single piece of work. Count it after reconciliation, when the system's case boundary matches the work.
- Decision: one choice point with its inputs and the option taken. A case with ten routine approvals becomes ten examples of judgment. Count it when the model makes or recommends one choice inside a longer process.
- Task episode: one attempt at a piece of work from trigger to observed outcome, across IDs. It undercounts only if incomplete attempts are silently dropped. Count it when the model completes, or is evaluated on, work end to end.
| Step (illustrative) | Rule applied | Count after the step | What the count means |
|---|---|---|---|
| Raw export | None | 412,000 messages in 61,500 ticket IDs | A message count, not a record count |
| Merged tickets | 3,100 IDs merged into another ticket fold into the surviving ticket | 58,400 IDs | Zendesk closes a merged ticket and a merge cannot be undone, so its history sits with the survivor |
| Follow-up tickets | Of 4,400 follow-ups opened by replies to closed tickets, the 2,900 that continue the same problem fold into their parent | 55,500 episodes | One piece of work under two IDs, now counted once |
| Incomplete episodes | 6,200 with no observed outcome (open, abandoned, or closed without a resolution code) are flagged, not deleted | 49,300 complete episodes | Incomplete cases stay for escalation and abstention tests but do not count toward R3 |
| R3 answer | Compare complete episodes with the 50,000 floor | Partly met: 49,300 against 50,000 | The honest status, where the message count would have claimed met |
When is the missing input a decision rather than a document?
The missing input is a decision when the correct output is a choice that depends on the situation at that moment and is judged by what happened afterwards. A document holds the rule or the result; a decision episode holds the state, the options, the choice and the outcome, and those four elements are what decision evidence dataset requirements have to name.
Documents hide the gap between a stated result and the real state. Anthropic's eval guide gives the example of a flight-booking agent that ends its transcript with 'Your flight has been booked' when the outcome that counts is whether a reservation exists in the environment's database. Benchmarks built from real work products name the same limit: OpenAI says GDPval's tasks come with reference files and context, yet its first version is limited to one-shot evaluations and does not capture cases where a model must build context or improve through multiple drafts.
The table writes R1 both ways for an illustrative credit-decision model. The decision-episode version becomes measurable through its own completeness line: at least 90 percent of a random 300-episode sample carry state, options, choice with the decider's role and outcome, with no input timestamped after the decision, and incomplete episodes are delivered flagged rather than dropped. Both numbers are placeholders to set against your task.
Test the cheaper explanation first. Rerun the failing cases with the governing policy, price list or prior correspondence pasted into the input. If accuracy recovers, the gap is retrieval of a document, and the fix is a document set or a retrieval change, not decision episodes.
| Element | Document-only line (illustrative) | Decision-episode line (illustrative) | Completeness check on the sample |
|---|---|---|---|
| Record unit | One credit memo PDF with its approval stamp | One credit request from receipt to posted result, joined across the ticketing and billing systems | Every row carries one episode key; 20 records match the definition |
| State at decision time | Absent | Account balance, order history and policy version as they stood at the decision timestamp | No input is timestamped after the decision |
| Options open | Absent | Approve, partial credit, deny or escalate, plus any limit that removed an option | At least two options, or the rule that removed them, recorded |
| Choice and authority | Final amount on the memo | Option chosen, the decider's role and any override of a system recommendation | Choice and role both present |
| Outcome observed later | Absent | What followed within 90 days: repeat request, chargeback, account closed or nothing | Outcome filled, or marked not yet observable with the window end date |
What rejection rule should be written before the first candidate is reviewed?
Write down, before any response arrives, which evidence ends a candidate's review outright, which removes only some records, and who may approve a deviation. Dataset procurement rejection criteria drafted after the first strong response tend to bend around it, which is the failure the gates above exist to prevent.
Public procurement law shows the discipline. FAR 15.305(a) requires U.S. agencies to evaluate competitive proposals solely on the factors and subfactors specified in the solicitation and to document strengths, deficiencies, significant weaknesses and risks in the contract file, and FAR 15.304(d) requires the factors and their relative importance to be stated clearly in the solicitation. A private data buyer is not bound by the FAR, but the same two habits, criteria fixed in advance and reasons written down, make a rejection defensible to the supplier and to your own reviewers.
Record every deviation in one line: the requirement, the measured value against the threshold, the decision, the approver, the date, and whether it now applies to every candidate. A deviation granted to one candidate only is a changed requirement the others never saw; issue it to all of them in writing or do not grant it.
| Failed line | Evidence that triggers the rule | Decision written in advance | Deviation allowed? |
|---|---|---|---|
| R9 Rights | Rights basis covers evaluation but not training | Reject for the training lot; the candidate may compete for an evaluation-only lot | No; rights are rescoped in writing, never waived |
| R1 Record unit | Records are messages with no case or episode key the supplier can rebuild | Reject | No |
| R8 Personal data | Residual direct identifiers above the agreed limit in the audit sample | Stop the pilot, delete the sample, allow one retest after reprocessing | Once, with a dated retest |
| R2 or R3 misstatement | Pilot value below the response beyond the stated tolerance, such as 80 percent measured against 95 percent claimed | Reject the response; a resubmission is treated as a new response | No |
| R7 Overlap | Candidate records match your held-out evaluation set | Exclude the matching records; reject if the overlap exceeds the stated share | Exclusion only |
| Episode completeness | Complete share below the threshold on the 300-episode sample | Partly met: narrow the scope or price the shortfall | Yes, if the line was set as a preference |
Which five questions need an owner's answer before an enterprise records dataset request goes out?
Five questions need a named owner and a written answer before any company records are requested: what the records are for, what is in scope, how the data will be validated, what permissions the license needs, and what would make you reject a candidate. Each answer becomes requirement lines and goes into a specific field of the request.
Owners matter because each answer commits a different budget or risk. NIST's AI Risk Management Framework (AI RMF 1.0) asks organizations to document roles and responsibilities for mapping, measuring and managing AI risks so they are clear to teams (GOVERN 2.1), and to document a system's intended purposes, context-specific laws, norms and expectations, and deployment settings (MAP 1.1). The Use answer should also name the test that ruled out tooling, retrieval or process design as the cause, because more training records do not fix those.
The request form at sourcex.si/request-data marks three fields as required: organization, work email and dataset description. Selecting Workflow datasets opens six more: business task, workflow type, input or context required, desired action trajectory, required outcome labels and evaluation criteria, which carry the decision-episode elements above. Because the description is the only required field about the data, put the task sentence, the unit of work, the completeness test and the rejection rule there. SourceX sources each dataset on request rather than holding stock, so a submitted brief starts a search; it does not mean matching records exist or will pass your test.
Every threshold, count and example in these four sections is illustrative and should be replaced with your own; SourceX has not published measured results from this template.
| Question | Decision owner | Minimum written answer | Where it goes on the request form | Lines it feeds |
|---|---|---|---|---|
| Use: what task or decision will the records train or test? | Model or product owner | One task sentence, excluded uses, and the test that ruled out tooling or retrieval | Dataset description; Business task | R9 |
| Scope: which unit, systems, period and strata? | Data lead with a domain expert | Unit of work, source systems, date window, strata and the complete-episode floor | Source systems; Approximate volume; Historical timeframe; Language; Format | R1, R3, R4, R6, R10 |
| Validation: how will each line be measured? | Evaluation lead | Method, sample size, reviewer and the episode completeness test | Evaluation criteria; Required outcome labels; Additional specifications | R2, R5, R7 |
| Permissions: what must the license and privacy handling allow? | Counsel and privacy lead | Permitted uses, exclusivity, duration and de-identification standard | Licensing requirements; De-identification requirements | R8, R9 |
| Acceptance: what ends a candidate's review, and who signs? | Budget holder with procurement | Gates, weights, the rejection rule and the deviation approver | Additional specifications; Desired timeline; Budget range | Gate lines and the rejection table |
Sources for this section
- Anthropic's engineering guide to agent evals (January 9, 2026) defines a task as a single test with defined inputs and success criteria, a trial as each attempt and the outcome as the final state of the environment, and contrasts an agent saying a flight is booked with whether a reservation exists in the database. Source
- OpenAI says GDPval tasks come with reference files and context, and that its first version is limited to one-shot evaluations that do not capture cases where a model must build context or improve through multiple drafts. Source
- FAR 15.305(a) requires agencies to evaluate competitive proposals solely on the factors and subfactors specified in the solicitation and to document strengths, deficiencies, significant weaknesses and risks; FAR 15.304(d) requires the factors and their relative importance to be stated clearly in the solicitation. Source
- NIST AI RMF 1.0 calls for documented, clear roles and responsibilities for mapping, measuring and managing AI risks (GOVERN 2.1) and for documenting a system's intended purposes, context-specific laws, norms and expectations, and deployment settings (MAP 1.1). Source
- Zendesk does not reopen closed tickets; a response to a closed ticket creates a new, linked follow-up ticket. Source
- In Zendesk, merging moves the merged ticket to closed status, and a merge cannot be undone. Source
Frequently asked questions
What is the difference between a data RFP and a data requirements template?
The RFP is the whole procurement document, including instructions, timeline, price form and scoring. The requirements template is the part that describes the data line by line, with thresholds and test methods. A good template can be reused across RFPs, direct requests and contract schedules.
Should the RFP ask suppliers to commit to a model accuracy gain?
Usually not. Model results depend on your base model, data mix and training run, which the supplier does not control. Ask suppliers to commit to measurable record properties and test the model effect yourself in a pilot.
How large should the pilot sample be?
Large enough to check each requirement line with the precision you need, and larger still if you plan a model test. For data checks, a few hundred randomly drawn records often suffice to detect a large gap between a claimed and actual fill rate; for model tests, size the sample with a power calculation.
What if no supplier meets every requirement?
Use the status codes to see which lines are the problem. If many suppliers miss the same line, the threshold may be unrealistic and can be relaxed once, for everyone, in a written addendum. If a gate line is missed, consider splitting the purchase or sourcing that slice separately.
Can the same requirement lines go into the contract?
Yes, and they should. Moving the lines and test methods into the acceptance schedule means the pilot, the evaluation of responses and the delivery check all measure the same thing.
Sources
- ISO/IEC 25012 defines a data quality model for structured data with fifteen characteristics from inherent and system-dependent points of view. Source
- ISO/IEC 5259-2:2024 covers data quality measures for analytics and machine learning. Source
- Article 10(3) of the EU AI Act requires high-risk training, validation and testing data to be relevant, sufficiently representative and, to the best extent possible, free of errors and complete. Source
- Datasheets for Datasets groups documentation questions by motivation, composition, collection, preprocessing and labeling, uses, distribution and maintenance. Source
- The Croissant RAI vocabulary defines machine-readable properties for data collection, collection time frame and annotation protocol. Source
- ISO 2859-1 provides sampling schemes indexed by acceptance quality limit for lot-by-lot inspection by attributes. Source
- Underpowered comparisons on small samples tend to overstate or reverse true effects. Source
Tell us what data you need
Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.