AI uses for records
A financial model assumption dataset grades the driver before the math
By SourceX Editorial · Reviewed by Noah Loul ·
10 sections
Short answer
A financial model assumption dataset should grade whether an agent finds every driver, such as growth, churn, price and timing, and checks each against the evidence in its packet before calculating. Arithmetic checks pass models built on contradicted inputs. Label each driver as supported, contradicted, unsupported or an approved judgment, have qualified accounting reviewers set those labels, and score only the flags an agent raises before its first formula.
Checklist
Financial model assumption
Key takeaways
- A recalculated workbook proves the formula, not the input; drivers carry the judgment in a financial model.
- The UK's AQuA Book separates verification (error-free, meets its specification) from validation (fit for purpose); only validation catches a bad assumption.
- PCAOB guidance and inspection findings make the same point for auditors: tracing an assumption to data is not evaluating whether it is reasonable.
- Label every driver supported, contradicted, unsupported or approved judgment, using records in the packet and signed approvals.
- Score flags raised before the first calculation, penalize silent overrides of approved assumptions, and include control packets so blanket flagging does not pay.
What should a financial model assumption dataset test before the arithmetic?
It should test whether an agent finds every driver the requested output depends on, checks each against the evidence in the task packet and flags the unsupported or contradicted ones before it calculates anything. A correct formula applied to a growth rate nobody can support is still a wrong answer; it is just a wrong answer that recalculates cleanly.
In a financial model the formulas are the visible part and the inputs carry the judgment. A reviewer who recalculates a workbook confirms that revenue equals customers times price times retention. That check says nothing about whether 3 percent monthly churn, a renewal price increase or a go-live date three months out is backed by any record. An agent asked to finish a model can fill a missing input with a plausible default and hand back a tidy file, and nothing in the arithmetic reveals that the input was invented.
Evidence that would support a useful answer comes in three parts: task packets that hold the model, the records bearing on each driver and any owner approvals; driver labels set by qualified accounting reviewers; and a scoring rule that rewards flags raised before calculation. This article sets out that design, a casebook and a rubric. It is a method, and no results from it have been published.
Verification is not validation: what a cancelled rail franchise taught
Checking that a model does what it was specified to do is verification; checking that its inputs and design suit the decision is validation, and only validation catches a bad assumption. UK government analysts built that distinction into their guidance after a modeling failure forced the cancellation of a major rail franchise competition.
On 3 October 2012 the Department for Transport cancelled the InterCity West Coast franchise competition after finding significant technical flaws, saying in its announcement that mistakes had been made in how inflation and passenger numbers were taken into account and in how much risk capital bidders were then asked to commit. The Laidlaw inquiry, reporting that December, traced the failure to an accumulation of significant errors linked to inadequate planning and preparation, a complex organizational structure and a weak governance framework.
The government then commissioned a review of quality assurance across its analytical models, published in March 2013 with eight recommendations, one of which asked departments to publish lists of their business-critical models. The AQuA Book guidance followed. Its current GOV.UK text says analytical quality assurance "is more than checking that the analysis is error-free and satisfies its specification (verification). It should also include checks that the analysis is appropriate and fit for the purpose for which it is being used (validation)."
The same guidance defines an assumptions log as "a register of assumptions, whether provided by the commissioner or gathered from the analysis, that have been risk assessed and signed off by an appropriate governance group or stakeholder." For a dataset, that maps cleanly: a grader that only recalculates performs verification, a driver label performs validation, and the assumptions log is the reference structure the labels should follow.
Growth, churn, price and timing: the evidence that can carry each
Each driver class has records that can support it and stand-ins that routinely take their place, and a useful packet holds both so the agent has to tell them apart. Labels judge support in the packet, not whether the future will turn out as assumed.
Reviewers assign one of four labels to every driver, defined before labeling starts:
- Supported: a record in the packet bears on the driver and agrees with it within a tolerance the reviewers set for that driver class.
- Contradicted: a record in the packet bears on the driver and shows a materially different value.
- Unsupported: nothing in the packet bears on the driver, so the value rests on the modeler's choice alone.
- Approved judgment: no record settles the driver, but an owner with authority signed it off as an assumption, with the approval in the packet.
| Driver | Records that can support it | Common stand-in that does not | Typical label for the stand-in |
|---|---|---|---|
| Growth in customers or volume | Pipeline conversion history and signed orders by cohort | A board target or an industry-wide growth rate | Unsupported, unless an owner signed the target as an assumption |
| Churn and retention | Cohort retention from billing or subscription records with cancellation dates | Last year's company-wide rate applied to a new segment | Unsupported for the new segment |
| Price | Signed price lists, contracts and realized price after discounts and credits | List price, or a planned increase not yet agreed with customers | Contradicted when invoices show a lower realized price |
| Timing of start, ramp and collection | Contract start dates, delivery schedules and collection history by customer | The planning calendar or standard payment terms | Unsupported until a delivery or collection record backs it |
Tracing an input to its source is not testing it
An agent that shows where an input came from has not shown that the input is reasonable, and auditing standards draw the same line. The dataset should score the trace and the test separately.
PCAOB auditing standard AS 2501 requires auditors to evaluate whether significant assumptions are reasonable, individually and in combination, including whether the company has a reasonable basis for them. PCAOB staff guidance from August 2019 frames the question as whether an assumption rests on an analysis of relevant information or was set with little or no such analysis, and asks whether evidence that contradicts it was considered. PCAOB staff material on auditing estimates also describes inspection findings in which auditors checked assumptions only by tracing them to internal or external data, without evaluating whether they were reasonable.
Securities rules point the same way for forward-looking numbers. Regulation S-K Item 10(b) encourages management projections "that have a reasonable basis and are presented in an appropriate format" and cautions against selective projection of only favorable items. Item 1609, which covers projections in de-SPAC transactions, requires disclosure of all material bases of the projections and all material assumptions underlying them.
Translated into a rubric, a trace earns credit only for naming the record or approval behind a driver. A test earns credit for comparing the value with that record and with the other drivers it interacts with, such as growth and churn read together, because a growth rate that is plausible alone can be impossible beside the retention the records show.
Casebook: six packets where the formula was never the problem
Each packet pairs a formula that works with a driver whose evidence fails in one specific way, and records the move a correct agent makes before calculating. Two packets are patterned on public records and four are Illustrative; none reproduces any company's actual records.
The public patterns are documents, not adjudications. The risk-capital packet takes only the outline of the rail case from the department's announcement, an inflation input feeding the risk capital a bidder must commit; its contract clause is invented. The projection packet follows a pattern described in settled SEC charges announced in September 2024, which alleged that Romeo Power's former chief executive misled investors about a shortage of battery cells, the main component of its products, and that a January 2021 filing he signed kept a risk disclosure the SEC described as materially false and misleading, despite his knowing or having reason to know that cell supply was deteriorating. A revenue projection is only as good as the supply behind it, and the packet tests whether the agent asks for that supply record.
| Case | Driver at issue | What the packet shows | Label | Correct move before calculating |
|---|---|---|---|---|
| Risk-capital sizing (patterned on the 2012 rail case) | Inflation input used to size the risk capital a bidder commits | Contract indexation on a different basis from the model | Contradicted | Stop and route the input to the model owner for validation |
| Revenue projection (patterned on SEC allegations, 2024) | Component supply behind a revenue projection | Supplier allocations falling while the projection stays unchanged | Contradicted | Flag the projection; offer only a supply-constrained scenario |
| New segment churn (Illustrative) | Monthly churn for an enterprise tier with no history | Retention cohorts for small accounts only | Unsupported | Ask for segment evidence; show a labeled range |
| Renewal price (Illustrative) | A planned 8 percent list-price rise | Renewal invoices at a lower realized price after discounts | Contradicted | Use realized price or flag the gap to the price owner |
| Go-live timing (Illustrative) | Revenue from month one of each contract | Delivery schedules showing go-live in month four | Contradicted | Shift the start to the scheduled month and say so |
| Planning target (Illustrative) | Growth set by the finance lead as a planning assumption | A signed entry in the assumptions log | Approved judgment | Use it, cite the approval and show sensitivity |
Reviewer rubric: points for flags raised before the first formula
Score the agent on how it handles drivers, in the order it handles them, and give no credit for a flag raised after the results. The figure shows the gap this rubric closes: the same Illustrative packet passes a formula-only check and fails on two drivers.
Score per driver and per packet, and weight load-bearing drivers more heavily: a driver is load-bearing when moving it to the value its evidence supports changes the decision the model informs. Report the criteria separately rather than as one blended number, so a buyer can see whether an agent fails at finding drivers, testing them or knowing when to stop. The criteria are:
- Driver inventory: lists every input the requested output depends on, including implicit ones such as ramp time and collection lag. Missing a load-bearing driver fails the packet.
- Evidence trace: names the packet record or approval behind each driver, or states that none exists.
- Test, not just trace: compares each value with its record and with related drivers in combination.
- Label match: assigns supported, contradicted, unsupported or approved judgment in agreement with the adjudicated reviewer label.
- Order: raises every flag before the first calculation; flags added after results earn nothing.
- Output scope: calculates on supported and approved drivers only and delivers anything else as a labeled scenario or a question to the owner.
- No silent override: leaves an owner-approved assumption in place, cites the approval and shows sensitivity instead of substituting its own value.
- Authority and treatment: does not present the output as approved, and does not treat correct arithmetic as settling the accounting treatment; both go to the named owner.
Assembling the task packets: authority, anonymization and qualified labels
Build packets from records a company has authorized for this use, anonymize them without breaking the links between drivers and evidence, and have qualified accounting reviewers set and adjudicate every label. The structure below is a design; no packets built to it have been scored, and no results are published.
At a glance
Assembling the task packets: authority, anonymization and qualified labels
Anonymization has to preserve the test. Customer and employee identifiers come out, but dates shift together so that go-live, invoice and collection intervals survive, and amounts keep their ratios so that a realized price still sits below list. A packet that loses those relationships can no longer contradict anything.
Labels need two independent reviewers with accounting qualifications, adjudication of disagreements by a third and reported agreement per driver class, because unsupported and approved judgment are the labels most likely to be confused. Hold out whole companies for evaluation so that an agent cannot learn one company's planning habits, and report counts per driver class and per label, since a set that is mostly supported drivers rewards an agent that never flags.
- Model: the workbook or a written specification of the requested output and its formulas.
- Driver list: every input, with the value the model uses and where the modeler took it from.
- Evidence: billing or subscription exports, contracts and price lists, delivery schedules, collection history, each dated.
- Approvals: the assumptions log or sign-off records, with the approver's role rather than name.
- Label sheet: reviewer labels, tolerances per driver class, load-bearing flags and adjudication notes.
When the arithmetic is right and the answer still lacks approval
Correct arithmetic establishes neither that the accounting treatment is appropriate nor that anyone authorized the assumptions, so the dataset grades both separately from the calculation. This is the strongest objection to formula-centered grading, and the design treats it as a requirement rather than a footnote.
Authorization is the easier half to label. An approved judgment can still prove wrong, but the agent's job is to surface it with its approval and its sensitivity, not to quietly replace it, which is why the rubric penalizes silent overrides as heavily as missed flags. Treatment is harder: a forecast can be computed perfectly on bookings when the decision needs recognized revenue, and only a qualified reviewer can say which basis applies.
The opposite failure needs a guard too. An agent that flags every driver would score well on a set full of weak packets and be useless in practice, so the set needs control packets in which every driver is supported, and the rubric charges for flags those packets did not warrant.
What the design cannot show yet
The design measures whether an agent recognizes unsupported drivers; it cannot yet say how well any agent does, and several limits remain until packets are built and scored:
At a glance
What the design cannot show yet
- No results: reviewer agreement, label distributions and agent scores are unknown until packets are assembled and labeled.
- Support is not accuracy: a supported driver can still miss, so a later phase should attach outcomes from the forecast period to each packet.
- Coverage: early packets will come from a few industries and model types, and growth, churn, price and timing do not exhaust the drivers that matter.
- Public cases are allegations or official statements, not court findings, and serve only as patterns for the Illustrative packets.
- Reviewer variation: tolerances for supported differ by driver class and by reviewer, so they must be written down per class and reported with the results.
Requesting assumption packets through SourceX
Buyers who need packets like these can ask SourceX to source them, because it finds datasets on request for each buyer rather than keeping a catalogue in stock. A useful request names the decision the model informs, the driver classes and industries needed, the volume of packets, the language and format, the timing and the licensing terms, and it includes the acceptance test the buyer will apply.
Candidate records would be planning workbooks with assumption logs or sign-offs, billing and subscription exports with cancellation dates, contracts and price lists, delivery schedules and collection histories from companies that choose to license them. A company approves the scope and terms and signs its agreement before any of its records reach SourceX, and none of this implies that a given set exists today or will pass a particular buyer's test.
- The decision the model informs and the output the agent must produce.
- Driver classes, industries and model types the packets must cover.
- The labels and tolerances the buyer expects reviewers to apply.
- Volume, language, format and the share of control packets.
- Intended use, licensing needs and the acceptance test for delivery.
Frequently asked questions
What is a financial model assumption dataset?
It is a set of task packets, each holding a financial model, the drivers it depends on, the records that bear on each driver and any approvals, plus labels from qualified reviewers. An agent is scored on whether it finds and tests the drivers before calculating, not only on whether its arithmetic is right.
How is checking an assumption different from checking a formula?
A formula check confirms the model computes what it specifies; an assumption check asks whether each input is supported by evidence for the decision at hand. The UK's AQuA Book calls the first verification and the second validation, and only the second catches a wrong growth or churn input.
Can an AI agent judge whether a growth assumption is reasonable?
It can compare the value with records in its packet, such as pipeline conversion history and signed orders, and flag a gap or a conflict. Whether a growth assumption is acceptable as a planning choice is a judgment for an authorized owner, so the agent should surface that choice rather than settle it.
What records support a churn assumption in a forecast?
Cohort retention computed from billing or subscription records with cancellation dates, for the same segment the forecast covers. A company-wide or industry average applied to a new segment with no history is a stand-in, and a reviewer would label the driver unsupported.
Should an agent change an assumption that management approved?
No. It should keep the approved value, cite the approval and show how sensitive the output is to it. Replacing an approved assumption silently is as serious a failure as missing an unsupported one, because it hides a decision that belongs to someone else.
Sources
- The AQuA Book on GOV.UK distinguishes verification (error-free, meets specification) from validation (fit for purpose) and defines an assumptions log as a risk-assessed, signed-off register of assumptions. Source
- The Department for Transport cancelled the InterCity West Coast competition on 3 October 2012 over technical flaws, citing mistakes in how inflation and passenger numbers were taken into account and in the risk capital bidders were asked to commit. Source
- The Laidlaw inquiry's final report (December 2012) attributed the failure to an accumulation of significant errors linked to inadequate planning and preparation, complex organizational structure and weak governance. Source
- The March 2013 review of quality assurance of government analytical models, commissioned after the West Coast problems, made eight recommendations including publishing lists of business-critical models. Source
- PCAOB AS 2501 requires auditors to evaluate the reasonableness of significant assumptions individually and in combination, including whether the company has a reasonable basis for them. Source
- PCAOB staff guidance (August 2019) asks whether an assumption rests on analysis of relevant information or was set with little or no such analysis, and whether contrary evidence was considered. Source
- The PCAOB's Audit Focus on estimates reports inspection findings of auditors who tested assumptions only by tracing them to data without evaluating reasonableness. Source
- Regulation S-K Item 10(b) encourages management projections that have a reasonable basis and are presented in an appropriate format, and cautions against selective projection of favorable items. Source
- Regulation S-K Item 1609 requires de-SPAC filings to disclose all material bases of projections and all material assumptions underlying them. Source
- SEC Litigation Release 26137 (September 2024) announced settled charges alleging that Romeo Power's former CEO misled investors about a battery cell shortage while a January 2021 filing kept a materially false and misleading risk disclosure. Source
Tell us what data you need
Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.