Audit judgment data: risk assessments, sampling decisions and reviewer notes

What audit workpapers actually record

Audit workpapers record judgment, not just numbers, which is why they are valuable for agents. Under PCAOB AS 1215, documentation must let an experienced auditor with no previous connection to the engagement follow the procedures performed, the evidence obtained and the conclusions reached; private-company audits follow the parallel AICPA standard, AU-C 230 [1]. Both standards also expect the file to show who prepared and reviewed the work.

That reviewability standard is what makes the files trainable. A well-kept file already pairs inputs (trial balance, account population, prior-year findings) with decisions (risk rating, procedure, sample size and method) and outcomes (exceptions, proposed adjustments, conclusion). Unlike the close work on our reconciliations and close datasets page, audit files capture a second party's skeptical evaluation of that work.

Typical workpaper components buyers ask for:

• Planning and risk assessment: materiality calculations, significant-account and assertion mapping, fraud-risk brainstorming notes, inherent and control risk ratings.
• Procedures and tickmarks: lead schedules, test-of-details sheets, walkthrough memos, control test results, tickmark legends that explain what each mark means.
• Sampling decisions: population definition, sampling unit, method (statistical, haphazard, targeted key items), sample size rationale and selection seed or listing.
• Exceptions and evaluation: deviation logs, projected misstatement, summary of unadjusted differences, follow-up procedures.
• Review notes: open points raised by seniors, managers and engagement quality reviewers, preparer responses, clearance dates.

Why reviewer notes are the highest-value layer

Reviewer notes are the closest thing in audit to labeled corrections, so they deserve the most attention in a sourcing request. Each note identifies a gap ("population not tied to GL," "sample excludes Q4 manual entries," "conclusion not supported by testing"), and the preparer's response shows the fix. That pairing is a natural source for critique models, preference pairs and evaluation rubrics.

The failure modes are specific. Many firms clear notes inside engagement software and archive only the final state, so the original deficient draft disappears. Others export notes without a pointer to the workpaper version they refer to, which breaks the link between critique and artifact. Ask whether note history, timestamps and the referenced document version survive export before anything else.

For agent builders, the pattern generalizes beyond audit: it is the same correction structure covered in human override and correction logs and in decision records with rationale, with stricter documentation norms.

How agentic audit tools use this data

Current agentic audit designs keep humans in charge of conclusions, so training and evaluation data should mirror that split. Wolters Kluwer describes agentic audits as assistive, with approval gates, exception logs and evidence links that make each agent step traceable and human-reviewed [2]. Workpaper automation vendors describe the same boundary: automation handles extraction, evidence organization and first drafts, while practitioners keep methodology and conclusions [3]. Professional bodies are cataloging agentic audit workflow use cases as well [4].

That suggests three practical training targets:

1. Drafting tasks: generate a test-of-details sheet or walkthrough memo from source evidence. Supervision comes from final, reviewed workpapers.
2. Critique tasks: given a draft workpaper, predict the reviewer notes. Supervision comes from note history.
3. Judgment-support tasks: given account characteristics, propose a risk rating, procedure and sampling approach with rationale. Supervision comes from planning memos and sampling worksheets, judged against the firm's methodology.

For evaluation, the AS 1215 standard is itself a rubric: could a reviewer with no prior connection reconstruct what the agent did, on what evidence, and why [1]? Fieldguide notes that AI-assisted files are expected to document source-data completeness and preserve outputs [1], so an evaluation set should check those fields, not only the final conclusion.

Sampling decision data: fields that make it usable

Sampling data is only useful when the population, method and rationale travel together. A sample listing without the population definition cannot teach an agent why 25 items were enough, or why the auditor added targeted selections above a threshold.

Ask suppliers whether each sampling record includes: population source and record count, the tolerable and expected misstatement or deviation rate used, confidence or assurance factor, method, selection seed or interval, items selected, results per item, and the evaluation of results. Also ask whether key-item and residual-population selections are separated. Agents that mix the two learn the wrong sample sizes.

Expect variation across firms. Methodologies differ in assurance tables, terminology and documentation templates, so a multi-firm dataset needs a normalization map, and a single-firm dataset carries that firm's methodology as an implicit label. Neither is wrong, but your evaluation design should know which one it has.

Illustrative example: invented to show structure; it does not describe an available dataset.

{
  "engagement_id": "ENG-7F3A",
  "client_industry": "wholesale distribution",
  "framework": "AU-C (private company)",
  "workpaper_ref": "C-4.2",
  "workpaper_version": 3,
  "area": "Revenue",
  "assertion": "occurrence",
  "risk_rating": {"inherent": "significant", "control": "moderate", "rationale": "new pricing system; manual credit memos"},
  "procedure": "test of details: vouch sales invoices to shipping documents and cash receipts",
  "sampling": {
    "population": "sales invoices FY, excluding intercompany",
    "population_count": 18412,
    "key_items": {"threshold": "> 0.5 x performance materiality", "count": 6},
    "residual_method": "monetary unit sampling",
    "residual_sample_size": 40,
    "selection_seed": "recorded"
  },
  "exceptions": [{"item": 17, "type": "cutoff", "amount_bucket": "low", "resolution": "proposed adjustment"}],
  "conclusion": "assertion supported after adjustment",
  "preparer": {"role": "staff_2", "signoff_date_offset_days": 41},
  "review_notes": [
    {"reviewer_role": "manager", "note": "Population not tied to GL revenue; tie out before concluding.", "response": "Tied to GL; difference is intercompany, now documented.", "cleared_offset_days": 44}
  ],
  "deidentification": {"client_name": "removed", "person_names": "role tokens", "dates": "offset from period end", "amounts": "bucketed"}
}

Confidentiality and consent come before privacy review

Audit files are confidential client information first, so the client's specific consent is the gating question. CPA professional-conduct rules on confidential client information generally bar disclosure without the client's consent, and state accountancy boards and firm policies add their own limits. Expect a CPA firm that wants to license workpapers to need each audited client's consent, not only its own decision; have counsel confirm the exact rule text that applies.

Tax workpapers add a second regime. Federal restrictions on the use and disclosure of tax-return information apply to return preparers separately from professional ethics rules. Keep tax provision and return-preparation files out of scope unless counsel has cleared them separately.

Other layers to check with counsel:

• Engagement letters and firm policy: many letters restrict use of client data beyond the engagement; network firms may add global policies.
• Issuer audit files: these sit inside PCAOB inspection and retention obligations, so archived files should be copied for preparation, never altered.
• Third-party data inside the file: confirmations from banks and customers, payroll registers and vendor master data carry other parties' information.
• Material nonpublic information: issuer audit evidence can include pre-release financial results; treat timing and access controls accordingly.

This page is general information, not legal advice. Confirm requirements with counsel for your jurisdiction and use case.

De-identification that preserves audit reasoning

De-identification for audit data must remove client and personal identity while keeping the relationships the reasoning depends on. Masking every amount destroys materiality logic; masking nothing exposes the client.

Practical choices: replace client and entity names with stable tokens; convert personal names to role tokens (staff, senior, manager, EQR); express dates as offsets from period end so cutoff tests still work; scale or bucket amounts while preserving ratios to materiality; and strip free-text references to customers, vendors and locations. Test the result against small populations: a regional client with a distinctive revenue profile can be re-identified from industry, size and period alone.

Document every transformation in a dataset card. Data Cards cover upstream sources, collection and annotation methods, intended use and decisions that affect model performance [5], which maps well to firm mix, framework, years, transformation rules and known gaps.

Buyer checklist for audit judgment data

Use this checklist before you request samples or price a set.

Illustrative example: invented to show structure; it does not describe an available dataset.

Related agent data shapes you may want alongside audit files: exception handling records, spreadsheet task trajectories for lead-schedule work, and fraud investigation case notes for forensic judgment. The agent training data hub and the AI data hub list the rest.

How SourceX sources audit workpaper data

SourceX sources operational datasets from US companies on request, including finance and legal workflow records; nothing is held in stock and a request does not guarantee a match. Buyers describe the data, not the firms, and SourceX looks for US businesses that hold it, with every release approved by the supplying company. See how buyers in this sector start on our accounting buyers page, or submit a data request.

Each dataset is rights-reviewed for ownership and consents and delivered under a license that defines records, uses, term and delivery. Personal details such as names, emails, phones and account numbers are removed or replaced before delivery, the method is recorded and a sample is checked, though no method is perfect. Delivery runs through private, access-controlled workflows only after an executed agreement and supplier approval.

Request audit judgment data for your agents

SourceX finds US companies that hold the operational records you describe, assesses data and licensing permissions, and agrees pricing and allowed uses in a license before anything is delivered. Nothing is contracted until a supplier agrees. Describe the workpapers, sampling records and reviewer notes you need at sourcex.si/buyers.