AI uses for records
Insurance customer correction dataset: when a material fact changes after the quote
By SourceX Editorial · Reviewed by Noah Loul ·
10 sections
Short answer
Training cases should capture a post-quote correction as its own event between two frozen quote versions: the original inputs and decision, the corrected field with old and new values and source, every rating, eligibility, form and referral field that depends on it, the evidence on file, the authority boundary, and the reevaluated decision. Score models on stale values left behind and on matching the recorded disposition, not on plausible output.
Evaluation split
Insurance customer
Key takeaways
- Treat a correction as an event between two complete quote versions, never as a silent edit to the final policy.
- Tag materiality from the record at the time; legal materiality is a human and counsel judgment, not a model label.
- The propagation map of dependent fields is the answer key that makes this a benchmark rather than a before-and-after diff.
- Score stale-value leakage, propagation recall, overreach, disposition match, authority compliance and evidence citation separately.
- Label timing: before bind, after bind and after a loss produce different jobs, and post-loss cases stay out of automated scoring.
How should a training case record a correction made after the quote?
Record the correction as its own event between two complete quote versions, not as an edit to the original. A usable case holds the quote as the customer first saw it, the corrected field with its old value, new value, source and time, every downstream field the correction should have changed, and the reevaluated decision with the person or rule that made it.
Many policy administration systems overwrite the working copy and keep the history in audit tables, notes or a separate job. If you export only the final policy, the correction disappears and the record teaches a model that the corrected value was there from the start. That is exactly the wrong lesson for an agent that will meet corrections mid-conversation.
The question an insurance customer correction dataset should answer is narrow: given a quote and a late change to a material fact, did the decision get rebuilt on the new fact, and did anything stale survive? Everything in the benchmark below serves that one question.
Which corrections are material enough to reopen the decision?
A correction is material for this benchmark when it can change the rating, eligibility, coverage forms, required evidence or the authority needed to approve the risk. A corrected phone number is not material; a corrected construction type, garaging address, payroll class or loss history usually is.
Materiality also has a legal meaning, which is why the benchmark labels it rather than inferring it. California Insurance Code section 359 lets the injured party rescind a contract when a representation is false in a material point, from the time the representation becomes false, and section 331 says concealment, whether intentional or unintentional, entitles the injured party to rescind. Other states use their own wording, so the label in a case is the underwriter's or counsel's call recorded at the time, never a model's guess.
Use the table to tag each correction before it enters the benchmark. The dependents column is what a propagation check will look for later.
| Correction type | Typical dependent fields | Likely reevaluation | Who usually decides |
|---|---|---|---|
| Contact or mailing detail | Correspondence address, notices | None to the decision; update only | Service staff |
| Location or garaging address | Territory, protection class, catastrophe zone, distance to coast or hydrant | Requote; possible appetite check | Underwriter or rating rule |
| Construction, occupancy or protection | Construction class, sprinkler credit, inspection requirement, eligible forms | Requote and refer if outside appetite | Underwriter |
| Exposure base (payroll, revenue, units, drivers) | Class rates, minimum premium basis, audit flags | Requote; audit note | Underwriter or rating rule |
| Prior losses or claims history | Experience modifier, eligibility, referral triggers | Refer; possible decline | Underwriter with referral authority |
| Ownership, insured entity or named insured | Named insured, additional insureds, prior carrier data | Reissue or rewrite review | Underwriter and operations |
What must the original quote preserve before the correction lands?
The original quote must survive as a frozen version with the inputs, the rating result, the forms selected and the decision state at the moment the customer saw it. Without that snapshot there is nothing to compare the reevaluated decision against.
Policy systems already model this separation. In Guidewire PolicyCenter's Cloud API, editing a quoted job means calling a make-draft action that returns the job to Draft, after which it has to be quoted again before binding. Multi-version quoting lets a team create and compare several quotes for one job, and a quoted version can be made draft, modified and requoted. Each of those transitions is a natural boundary for a benchmark case.
Export the version identifiers along with the values. A case that says the sprinkler answer changed is weaker than one that says quote version 1 carried full sprinklers, version 2 carried partial sprinklers, and version 2 was quoted at a recorded time by a recorded user or rule set.
Keep the rating and rules version too. If the rate table or the underwriting guideline changed between the original quote and the correction, a model cannot tell whether the new premium reflects the corrected fact or the new rates, and the case should record both changes separately.
The correction propagation and decision reevaluation benchmark
The benchmark is a set of correction cases, each a self-contained packet that a model reads in order and acts on. Every field below is required; a case missing one goes to a review queue rather than into the scored set. Field names are illustrative and should be mapped to your own system's labels.
- Accept a case only when the original version, the correction event and the final disposition all come from the record, not from memory.
- Store the propagation map as the answer key; it is what separates this benchmark from a plain before-and-after comparison.
- Tag each case with the correction type from the earlier table so results can be read per type.
| Packet section | Required content | Where it usually lives | Check before acceptance |
|---|---|---|---|
| Case header | Case ID, line of business, jurisdiction, policy or submission ID (pseudonymized), job type | Policy admin system job record | IDs resolve to one job; jurisdiction present |
| Original quote version | All rating inputs, forms and endorsements selected, rating result, decision state, quote time, rules version | Quote version or job snapshot | Version is frozen and complete, not the current working copy |
| Correction event | Field, old value, new value, who reported it, channel, time, supporting document reference | Notes, email, call log, change request | Old value matches the original version exactly |
| Propagation map | Every field, rule result, form and referral flag that depends on the corrected field | Rating worksheet, rules engine output, forms logic | Reviewer confirms map against the rules version in force |
| Evidence on file | Inspection report, application answers, prior carrier data, photos, with dates | Document management, underwriting file | Each item dated; conflicts flagged, not resolved silently |
| Authority boundary | Who may approve the corrected risk and at what limits | Underwriting authority letter or referral rules | Boundary stated in plain terms, current as of the case date |
| Reevaluated decision | New quote version, referral, decline or no change, with rationale text | Requoted job, referral notes | Rationale names the corrected fact |
| Final disposition | Bound, not taken, declined, withdrawn or still open, with date | Job status history | Disposition observed, not inferred |
How do you score a model that receives the correction?
Score the model on whether it rebuilt the decision from the corrected fact, not on whether its final premium or wording looks right. A model can land on a plausible answer while carrying a stale value into one form or ignoring a referral trigger, and those are the failures that matter.
Run each case as a sequence: give the model the original quote state, then the correction event, then ask for its next action and the fields it would change. Compare its output to the propagation map and the recorded reevaluated decision. Report counts per correction type with the number of cases behind each figure, and treat any per-type result built on a handful of cases as directional only.
| Measure | What it counts | Pass condition | Why it matters |
|---|---|---|---|
| Stale-value leakage | Dependent fields the model left at the pre-correction value | Zero stale dependents | One stale rating input can misstate the whole quote |
| Propagation recall | Share of mapped dependents the model identified | Every mapped dependent named | Shows whether the model understands the rule chain |
| Overreach | Fields the model changed that the correction does not touch | No unmapped changes | Unneeded edits create new errors and audit noise |
| Disposition match | Model's next action versus the recorded reevaluation | Same action class: requote, refer, decline or no change | The business outcome, not the text |
| Authority compliance | Whether the model acted beyond the stated authority boundary | Refers whenever the boundary requires it | An agent that binds outside authority is unusable |
| Evidence citation | Whether the rationale points to the documents on file | Cites the correction and any conflicting evidence | Lets a reviewer audit the decision quickly |
What happens when the evidence disagrees with the customer?
When a document on file contradicts the correction, the benchmark records the conflict and the resolution the business actually made, and it scores the model on flagging the conflict, not on picking a winner. Choosing between a customer's statement and an inspection report is an underwriting judgment with legal consequences.
Typical conflicts include an inspection report that shows a different roof age than the corrected answer, prior carrier data that lists losses the customer now says did not occur, or a payroll figure that disagrees with a filed audit. In each case the correct model behavior is to hold the requote, list the disagreeing sources with dates, and route to the person with authority.
Keep these cases in a separate slice. They behave differently from clean corrections, and mixing them into the main score hides whether a model escalates when it should.
Worked example: a sprinkler correction on a small commercial property quote
This example is illustrative and invented to show the packet shape; it does not describe a real account, carrier or dataset. A small commercial property submission is quoted with the building marked as fully sprinklered. Two days later the customer's broker emails that only the warehouse portion is sprinklered; the office wing is not.
| Step | Recorded state | Expected model action | Answer-key note |
|---|---|---|---|
| Quote version 1 | Full sprinkler, protection credit applied, standard form set, quoted and sent | None; this is the baseline | Freeze inputs, forms and rules version |
| Correction event | Broker email: partial sprinkler, office wing unprotected | Recognize a material protection change | Old value matches version 1 exactly |
| Propagation | Sprinkler credit, protection class input, inspection flag, eligible forms | Name all four dependents and mark the quote stale | Leaving the credit in place counts as leakage |
| Evidence check | Inspection report on file is older and silent on the office wing | Note the gap and request current inspection | No conflict, but missing evidence |
| Authority | Partial sprinkler on this occupancy needs referral under the illustrative guideline | Refer instead of requoting directly | Requoting without referral fails authority compliance |
| Reevaluated decision | Underwriter requotes as version 2 without the full credit, adds an inspection condition | Match the action class: refer, then requote | Rationale must cite the broker email |
| Final disposition | Bound on version 2 with the condition | Not scored; used to confirm the case closed | Observed in job status history |
Does the answer change before bind, after bind or after a loss?
Yes. The same corrected fact creates a different job and a different answer key depending on when it arrives, so every case must carry its timing label.
At a glance
Does the answer change before bind, after bind or after a loss
Before bind, the correction usually means returning the job to draft and requoting. After bind, it becomes a policy change, and timing conflicts appear: Guidewire's documentation describes preemption, where two jobs built on the same base policy period collide, and notes that handling a preemption typically modifies the job, so it must be quoted again before binding even if it was quoted earlier. Its glossary describes rewrites as the route for changes a policy change cannot handle or to correct major errors. After a loss, a corrected fact may raise questions about the original representation, which is a coverage and legal matter that belongs with counsel, not with a model.
Consumer-initiated corrections add another layer. The NAIC Insurance Information and Privacy Protection Model Act (Model 670), adopted in some states, gives individuals a route to request correction, amendment or deletion of recorded personal information, requires a response within 30 business days, and requires notice of a correction to certain recipients that received the information in the prior two years. It also sets requirements when an insurer makes an adverse underwriting decision. Check which version, if any, applies in your state before treating these steps as required.
- Before bind: label the case requote or refer; the answer key is the next quote version.
- After bind, before renewal: label the case policy change; include preemption or out-of-sequence notes if present.
- After a loss: exclude from the automated benchmark unless counsel has cleared the case; keep it for human review training only.
- Consumer correction request: include the request date and response date so timing can be checked.
Which correction histories should stay out of the benchmark?
Leave out any case where the correction is tied to an open claim, a fraud referral, a rescission discussion or litigation. Those records carry legal holds and judgments that a training set should not freeze.
Also leave out cases where the original quote version cannot be reconstructed, where the correction arrived by phone with no note, or where the rules version in force is unknown. A partial case looks like a correction example but trains guesswork.
Governance matters if the benchmark will evaluate a model an insurer uses on consumers. The NAIC model bulletin on the use of artificial intelligence systems by insurers, adopted in December 2023, expects insurers to govern AI systems that support consumer-impacting decisions and to be able to show documentation to regulators. A benchmark with a clear answer key and dated cases is the kind of evidence that supports that work, but it does not satisfy the bulletin on its own.
Finally, personal details in correction notes, such as names, addresses and loss descriptions, need de-identification before records leave the business, and that step must keep the old and new values comparable. Replace a street address with a stable territory or zone code at both versions rather than removing it from one.
Licensing an agency or carrier correction history
Correction histories are valuable to AI teams because they show decisions being rebuilt, which final policy records never show. An agency, broker, MGA or carrier that holds versioned quotes, change notes and dispositions has the raw material for this benchmark, provided it has the right to license it; carrier contracts and agency agreements often decide who controls quote data, and that question comes before any preparation work.
Turning selected operational records like these into licensed datasets for AI is SourceX's work, including rights review, contracting, preparation, delivery and payment. No data reaches SourceX before the business approves the scope and terms and signs, and the first step is a short company assessment with no upload.
Frequently asked questions
What is an insurance customer correction dataset?
It is a set of cases in which a customer or broker corrected a fact after a quote, each holding the original quote version, the correction event, the fields that depend on it and the reevaluated decision. It trains or tests whether a model rebuilds an insurance decision from the corrected fact instead of carrying stale values forward.
Which corrections should send a quote back to an underwriter?
A correction changes the decision when it moves a rating input, an eligibility rule, a required form or the level of authority needed to approve the risk. An evidence conflict changes it when a dated document disagrees with the correction, and an authority boundary changes it when the corrected risk falls outside what the current handler may approve.
Can I use final bound policies instead of quote versions?
Not for this purpose. A bound policy shows only the end state, so the correction and the decision it triggered are invisible. You need the frozen quote as first presented and the version that followed.
How many correction cases does a benchmark need?
There is no fixed number; what matters is enough cases per correction type to read results by type, plus a separate slice for evidence conflicts. Report the case count next to every result so readers can judge how stable it is.
Which correction cases need an underwriter, compliance or counsel before they enter a dataset?
Stop when a correction touches an open claim, a possible misrepresentation, a fraud referral or a consumer's formal correction request, or when the rules version in force cannot be established. Those cases need an underwriter, compliance or counsel before they enter any dataset.
Sources
- California Insurance Code section 359: if a representation is false in a material point, the injured party is entitled to rescind the contract from the time the representation becomes false. Source
- California Insurance Code section 331: concealment, whether intentional or unintentional, entitles the injured party to rescind insurance. Source
- In Guidewire PolicyCenter's Cloud API, edits to a quoted job require the make-draft action, which returns the job to Draft, and the job must be requoted. Source
- PolicyCenter multi-version quoting lets a caller create and compare several quotes for one job; a quoted version can be made draft, modified and requoted. Source
- Handling a preemption in PolicyCenter typically modifies the job, so the job must be quoted before binding even if it was quoted before the preemption. Source
- Guidewire's business glossary describes a rewrite as used for changes a policy change cannot handle or to correct major errors. Source
- NAIC Model 670 requires a response to a correction, amendment or deletion request within 30 business days, notice of corrections to certain recipients from the prior two years, and sets requirements for adverse underwriting decisions. Source
- NAIC members approved a model bulletin on the use of AI systems by insurers in December 2023, setting governance and documentation expectations for consumer-impacting AI. Source
See if your company qualifies
A short company assessment. No data uploads are needed.