AI uses for records
A revenue forecast uncertainty dataset grades the range, not the number
By SourceX Editorial · Reviewed by Noah Loul ·
9 sections
Short answer
A revenue forecast uncertainty dataset pairs each forecast question, frozen when it was asked, with a reference answer stating a range and how often such ranges should hold, the evidence behind it, the deals or assumptions that move it, and what would settle them. Grade answers across many quarters, never one. In a seeded illustrative simulation, quarters finished within 2.5 points of the weighted pipeline figure only 8.8% of the time.
Record timeline
Revenue forecast uncertainty
Key takeaways
- A weighted pipeline figure is an average, not a quarter anyone will see: in the illustrative simulation only 8.8% of quarters finished within 2.5 points of it.
- A useful revenue answer carries five parts, a range with coverage, a labeled center, grounds, swing items and open assumptions, all stamped with an as-of time.
- Name the swing deal: in the simulation, whether one deal signed moved the chance of reaching plan from 25.6% to 92.1%.
- Stated odds that run high break a range quietly: with true odds a quarter lower, the stated 80% range held only 63.3% of outcomes.
- Grade uncertainty across many as-of questions, by coverage and width together; one quarter landing inside a range proves little.
- Leave contact identities out: role tokens, account tokens and dated documents carry every ground the grader needs.
What should a revenue agent say when the honest answer is a range?
It should give the range and what the range rests on, not a single figure. Asked whether the quarter will reach plan, a well-trained agent states a range and how often ranges like it should contain the result, the center it would pick and what kind of center that is, the evidence behind its odds, the one or two deals or assumptions that swing the answer, and the record or date that would settle them. A revenue forecast uncertainty dataset exists to teach that answer and to grade it.
Most training and evaluation data pushes the other way. OpenAI's September 2025 research on hallucination argues that accuracy-only scoring rewards a model for guessing instead of acknowledging uncertainty, because a confident guess sometimes scores and an admission of doubt never does. A revenue dataset whose reference answers are single numbers, marked right or wrong, builds the same incentive into a forecasting agent.
The better behavior is learnable. Lin, Hilton and Evans showed in 2022 that a language model can be fine-tuned to state its confidence in words, which they call verbalized probability, and that those statements stayed reasonably calibrated when the type of question shifted. For revenue teams the open question is less whether a model can say 70 percent than whether the training records show what a defensible 70 percent rests on.
So the buying decision turns on three properties of the candidate records: questions frozen at the moment they were asked, the evidence that existed at that moment, and outcomes defined before anyone labels an answer. Records missing any of the three can teach an agent to sound careful without being careful.
Why a weighted pipeline figure is the wrong reference answer
Because it is an average of outcomes, not an outcome. A weighted forecast multiplies each open deal by its stated probability and adds the products; the quarter itself books each deal or misses it, so it lands on lumpy totals that can sit well away from the average.
A seeded simulation of one illustrative pipeline shows the size of the gap. The pipeline is expressed in percent of the quarter's plan: 30 points already booked and 24 open deals worth 146 points, including two large deals of 30 and 20 points at stated odds of 50 and 40 percent. In each of 100,000 simulated quarters every deal closed independently with its stated probability (Python standard library, seed 20261027). The inputs are invented to show the mechanism and describe no company.
The weighted forecast came to 104.2 percent of plan. Only 8.8 percent of simulated quarters finished within 2.5 points of it, while the middle 80 percent of outcomes ran from 76 to 132, and the quarter reached plan 58.7 percent of the time. An agent trained to answer 104 is being taught a figure that, on these inputs, it will rarely see.
The odds are the second trap. With the true probability of every open deal set to three quarters of the stated one, the median outcome fell to 84 and the stated 76-to-132 range held only 63.3 percent of quarters, with 35.5 percent below its floor. The weighted forecast did not move, since it is computed from the stated odds. An answer that does not say where its odds came from cannot be checked.
The two cases side by side, in percent of plan:
| Measure | Odds as stated | True odds a quarter lower | What the answer should carry |
|---|---|---|---|
| Weighted forecast | 104.2 | 104.2, unchanged | A labeled center, never the whole answer |
| Middle 80 percent of outcomes (P10 to P90) | 76 to 132 | 60 to 113 | The range and how often it should hold |
| Quarters reaching plan | 58.7 percent | 25.9 percent | The chance of the event the asker cares about |
| Quarters inside the stated 76-to-132 range | 80.8 percent | 63.3 percent | The source of the odds, so coverage can be audited |
Name the deal that swings the quarter
The most useful sentence in a revenue answer is often conditional. In two further seeded runs, when the 30-point deal signed, the middle 80 percent of outcomes ran from 100 to 139 and the quarter reached plan 92.1 percent of the time; when it did not, the range was 70 to 109 and plan was reached 25.6 percent of the time. Blending those two worlds into one range hides the decision the asker can act on.
Revenue accounting faces the same choice and spells it out. IFRS 15, paragraph 53, tells a company estimating variable consideration to use either the expected value, a probability-weighted sum across possible amounts, or the most likely amount, whichever better predicts what it will receive. The standard notes that an expected value can suit a large number of contracts with similar characteristics, while the most likely amount can suit a contract with only two possible outcomes, and it then limits the estimate to an amount for which it is highly probable that a significant reversal will not occur.
For a revenue agent the lesson is to label the center and pick the right one. A quarter that hinges on one or two large deals behaves like the two-outcome case, so the answer should give a range for each outcome of the swing deal and name it. A quarter made of many small, similar deals behaves like the portfolio case, where an expected value with a range is a fair summary.
Five parts of an answer that keeps its uncertainty
Every reference answer in the dataset should contain the same five parts, each tied to a record that existed on the as-of date, plus the as-of stamp itself. Climate assessment already works this way. The IPCC's guidance for authors of its Fifth Assessment Report keeps two measures apart: confidence, a qualitative judgment built from the type, amount, quality and consistency of evidence and the degree of agreement, and likelihood, a quantified probability. It also encourages authors to report the tails of key variables, because low-probability outcomes can have large impacts.
U.S. securities law treats a revenue projection's assumptions as part of the statement: its definition of a forward-looking statement covers projections of financial items together with the assumptions underlying them. Public companies publish guidance under that framing, and an internal answer to a chief financial officer deserves the same discipline.
The parts, the record each rests on and the failure each prevents:
| Part of the answer | What it states | Record it rests on | Failure it prevents |
|---|---|---|---|
| Range with coverage | Low and high values and how often such ranges should contain the outcome, such as 8 in 10 | A distribution built from as-of snapshots or closed-cohort history | A single figure heard as a promise |
| Labeled center | Expected value, most likely amount or median, named as such | The same distribution and the deal mix | An average reported as a likely result |
| Grounds | Where the odds came from: stage defaults, cohort close rates or buyer documents | Field history, closed cohorts, dated buyer records | Odds no one can audit |
| Swing items | The deals or assumptions that move the answer most, with a range for each outcome | Deal amounts and odds on the as-of date | A range that hides the decision |
| Open assumptions | What is unresolved, which record would settle it and by when | Pending approvals, scheduled reviews, renewal dates | Hedges that never resolve |
| As-of stamp | When the answer was given and the outcome rule it will be graded on | Snapshot time and the written outcome definition | Grading with facts that arrived later |
Seven labels for grading a revenue answer
Each candidate answer, whether written by a person or a model, gets one primary label. The labels judge how the answer handled uncertainty, not whether the quarter came in, so a grader can apply them on the day the question was asked.
Two labels reward restraint correctly. A justified abstention, which says the evidence cannot support a range and names what would, should outscore a confident figure built on stage defaults. An empty hedge, words like might and could with no range and no basis, should score no better than a bare point.
The label set, with the test a grader applies to each:
| Label | Test the grader applies | Illustrative answer | Use in training |
|---|---|---|---|
| Grounded range | Range, coverage, labeled center, grounds and swing items present and consistent with as-of records | 76 to 132 percent of plan, 8 in 10; above plan if deal A signs | Target |
| Bare point | One figure with no range and no basis | We land at 104 percent | Negative example |
| Unanchored range | A range without a coverage level or without stated grounds | Somewhere between 80 and 130 | Negative example; add the basis |
| Hidden swing | A range is given but the deal or assumption that dominates it is not named | 76 to 132, broadly on track | Partial credit; add the condition |
| Hindsight basis | Cites a record dated after the as-of time | Deal A slipped, so expect 89 | Excluded; flags leakage in the set |
| Empty hedge | Uncertainty words with no range, odds or basis | It could go either way | Negative example |
| Justified abstention | Declines a range, says why and names the record that would allow one | Too few closed deals in this segment to set odds; the approval meeting settles deal A | Target when evidence is thin |
Which records ground each part, and which to leave out
The grounds come from dated commercial records and the outcomes from documents, not from people's identities. A pipeline snapshot gives amounts, stages and stated odds; field history shows who set each probability and when; quarters that closed earlier give observed close rates; signed orders, approvals and review dates settle the open assumptions; and the written outcome rule says what counts as booked.
Who set a number matters as much as the number. A probability a manager overrode on the last day of the quarter is a different ground from a stage default an administrator configured, and the dataset should record which one an answer relied on. Annotating those values belongs to the CRM data itself; this dataset sits one step later and grades the answer built on them.
Leave out what the grader does not need. Contact names, email addresses and call recordings add privacy exposure and no grading signal. Replace people with role tokens such as account executive or approver, and accounts with stable tokens, so a swing deal stays traceable across snapshots without naming anyone.
A short map of the records, the as-of rule for each and what to drop:
- Pipeline snapshot: amount, stage, stated probability and expected close date as of the question time, never rebuilt from today's record.
- Probability provenance: stage default, rep override, manager override or model score, taken from field history.
- Cohort history: close rates for comparable deals from quarters that had fully closed before the as-of date.
- Resolution records: signed orders, approval decisions and review dates that later settle each open assumption, used for grading only.
- Outcome rule: one written definition set before labeling, such as signed and not cancelled within the quarter.
- Drop: contact names, emails, phone numbers, call audio and free-text notes that name people; keep role and account tokens.
Can one quarter that landed inside the range prove the answer was right?
No. A won deal can come from a competitor's stumble or a budget released early, and a lost one from an acquisition nobody could see, so one outcome says little about whether the stated range was honest. In the simulation, a range built from correct odds still missed 19.2 percent of quarters, as an 80 percent range should.
Grade uncertainty in aggregate. Across many as-of questions, an 8-in-10 range should contain roughly 8 in 10 outcomes, and the grader should score width and misses together so an agent cannot win by answering zero to 200 percent of plan. Gneiting and Raftery's 2007 paper on proper scoring rules sets out the principle: under a strictly proper score, a forecaster does best in expectation by reporting its true distribution, and the paper applies an interval score to interval forecasts. With 50 questions, an honest 80 percent range would be expected to miss about 10 times, enough to expose gross overconfidence but not a small drift.
Keep the reasoning grade apart from the outcome grade. The seven labels judge the answer on the day it was given; coverage and width judge a set of answers after the quarters close. A model that earns grounded-range labels while its ranges hold 60 percent of outcomes has learned the form without the substance, and the dataset should be able to show that.
Writing the request for a revenue forecast uncertainty dataset
A request should describe the questions, the snapshots, the outcome rule and the acceptance test, not just ask for a CRM export. Name the question types the agent will face, such as plan attainment, one deal's close or a segment total; the snapshot cadence; two or more years of closed history where it exists; the written outcome definition; and the identity fields to exclude.
Add an acceptance test the supplier can run before delivery: every snapshot rebuilt from field history, no record in a snapshot dated after its as-of time, probability provenance filled for every open deal, and enough closed quarters to compute cohort rates for each segment the agent will be asked about.
SourceX does not hold datasets like this in stock; it sources them from U.S. businesses once a buyer's request arrives, and the request form takes scope, volume, language, format, timing and licensing needs alongside the intended use and the acceptance test. Whether any company's records pass that test is decided deal by deal, and records alone imply no gain in a model.
Limits of this design, and the next evidence to collect
The simulation is illustrative and simplified on purpose. Deals closed independently, with no shared shock such as a budget freeze that hits many buyers at once; correlation of that kind would widen the spread, so a range computed as if deals were independent would hold fewer outcomes than shown here. Partial closes and deals pulled in from later quarters were not modeled. The simulated figures are estimates for seed 20261027, with one random draw per deal per quarter in list order and every run taken in turn from that one stream. Computed exactly, the same inputs give a P90 of 133 rather than 132 (114 rather than 113 with overstated odds) and reach plan 92.0, 25.5 and 26.0 percent of the time rather than 92.1, 25.6 and 25.9; every other reported figure is within 0.1 point of its exact value.
Nothing here is measured on real pipelines. The label set has not been tested for agreement between graders, and the right coverage level, 8 in 10 or 9 in 10, depends on how answers will be used. The next evidence is a pilot: two graders labeling the same answers blind, agreement reported per label, and coverage and width computed on quarters held out from training. No results from such a pilot have been published. The simulation script, with its inputs and seed, is kept for reviewers.
Frequently asked questions
What is a revenue forecast uncertainty dataset?
It is a set of revenue questions, each frozen at the moment it was asked, paired with reference answers that state a range, its coverage, the grounds, the swing items and the open assumptions, plus the outcome that later resolved each question. It trains and grades how an agent expresses uncertainty, not only whether its number was right.
Should a revenue agent answer with a probability or a range?
Each fits a different question. Whether the quarter reaches plan calls for a probability with its grounds; how much will book calls for a range with its coverage and a labeled center. When one deal dominates, a range for each outcome of that deal is clearer than either.
How many questions are needed to check whether ranges are calibrated?
More than most teams expect. An honest 80 percent range should miss about 2 times in 10, so 50 questions give about 10 expected misses, enough to catch badly overconfident ranges but not a small drift. Pool questions across quarters and segments, and report how many fed each figure.
Can CRM stage probabilities serve as the grounds for an answer?
Only if the answer says they are stage defaults. Stage probabilities are configured assumptions, not measured rates, so an answer built on them should say so and, where closed-quarter history exists, set them against observed close rates.
Does the dataset need customer or salesperson names?
No. Grading needs amounts, stages, dates, probability provenance and outcomes. Role tokens and stable account tokens keep swing deals traceable across snapshots without contact identities.
Sources
- OpenAI's September 2025 research on why language models hallucinate argues that accuracy-only scoring rewards guessing over acknowledging uncertainty, because abstaining never scores. Source
- Lin, Hilton and Evans (2022) fine-tuned GPT-3 to express its confidence in words (verbalized probability) and found reasonable calibration both in and out of distribution. Source
- The IPCC AR5 uncertainty guidance note separates qualitative confidence, based on evidence and agreement, from quantified likelihood, and encourages authors to report the tails of key variables. Source
- IFRS 15 paragraph 53 requires variable consideration to be estimated by the expected value or the most likely amount, whichever better predicts the amount; the constraint limits it to amounts where a significant reversal is highly improbable. Source
- 15 U.S.C. 78u-5 defines forward-looking statements to include projections of financial items and the assumptions underlying them. Source
- Gneiting and Raftery (2007, Journal of the American Statistical Association) formalize strictly proper scoring rules, which the true distribution uniquely optimizes, and use an interval score for interval forecasts. Source
Tell us what data you need
Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.