AI uses for records
Voice agent correction recovery: did the address fix reach every system?
By SourceX Editorial · Reviewed by Noah Loul ·
10 sections
Short answer
To test whether an address correction reached every affected system, list each record that holds the address, decide from the call which ones should change, and compare each record's state after the call with an expected state written before grading. Prove from the audio what the caller said, including any repair turn, and recheck downstream outcomes later. The agent's confirmation message is never the evidence.
Agent run
Voice agent correction
Key takeaways
- A customer usually has several address fields across systems, and a correction applies to some of them, not all.
- Grade record states against an expected end state, the way tau-bench compares database results, not the agent's closing sentence.
- The audio must establish who said the corrected value and when, with address spans checked by a person.
- Over-propagation is a failure too: changing every address for a one-order request is wrong.
- Outcomes such as invoices, deliveries and returned mail need a maturity window before a task counts as passed.
How do you test whether an address correction reached every affected system?
Start from the systems, not the call. List every record that holds the customer's address and decide, from what the customer actually said, which of those records should change. Then compare each record's state after the call with an expected state written before grading. An agent that says "I've updated your address everywhere" has produced a sentence, not an outcome; only the records prove propagation.
Public agent benchmarks already grade this way. In the tau-bench retail environment, the reward check replays the ground-truth actions on a fresh copy of the database and compares a hash of the result with the database the agent left behind; if the changes differ, the task scores zero regardless of what the agent said.
A voice version adds two layers the text benchmarks skip: proving from the audio what the customer said (including the moment they corrected themselves), and waiting long enough to see whether the corrected address held up downstream. The task pack below covers both.
Why one customer address is really several fields
Most business stacks store more than one address per customer, and a correction usually applies to some of them, not all. The tau-bench retail environment makes the split explicit with two separate tools: one modifies the user's default address, the other modifies the shipping address of a single pending order. Stripe's Customer object keeps a billing address and a separate shipping address. Salesforce standard objects carry their own address fields, such as MailingAddress on Contact and BillingAddress and ShippingAddress on Account.
That means "update my address" is ambiguous until the agent settles scope. Is this a move, so every current address changes? Is it a one-time delivery to a relative, so only one order changes? Is it a fix to a typo the agent itself introduced earlier in the call? The correct end state differs in each case, and so does the evaluation.
Write the scope decision into the task as its own labeled step. Grading only the final records cannot distinguish an agent that asked and chose correctly from one that guessed and got lucky.
What must the recording establish before the outcome is graded?
The audio has to support the transcript, and the transcript has to support the expected end state. If either link is weak, a failure might be the grader's, not the agent's.
At a glance
What must the recording establish before the outcome is graded
Speaker diarization tools such as pyannote.audio output turns as start time, stop time and speaker label, which is the shape the task pack needs: who said the corrected address, and when relative to the agent's confirmation. Speech recognition quality also varies; the Whisper README notes that its performance differs widely by language, so street names, unit numbers and accented speech deserve a human-checked transcript for the spans that matter.
- Audio rights: confirmation that the recording was captured and may be used under the consent rules that applied to the call.
- Speaker turns with start and end times, labeled customer or agent.
- Transcript aligned to those turns, with the address spans checked by a person against the audio.
- Repair turns marked explicitly: the original wrong value, the correction, and the words that signal it, such as "no, it's 14, not 40".
- Transfer context: whether the call was handed between agents or systems, and what the receiving side was told.
- Read-back: whether the agent repeated the corrected address and the customer confirmed it.
The correction propagation voice task pack
Each task in the pack is one call plus everything needed to grade it. The components below are the minimum; a reviewer should be able to rebuild the expected outcome from them without listening to the call twice.
| Component | Contents | Evidence reference | Check before release |
|---|---|---|---|
| Starting state | Snapshot of every address-bearing record before the call | Record ids and field values with timestamps | Snapshot taken from source systems, not reconstructed |
| Call audio and rights | Recording plus consent and use basis | Audio file hash; consent record | Rights confirmed by the data owner |
| Aligned transcript | Diarized turns with times | Turn ids mapped to audio offsets | Address spans verified by a person |
| Repair span | Original value, corrected value, signal words | Turn ids for both values | Annotator agreement on what was corrected |
| Scope decision | Which addresses the customer meant to change | Turn ids where scope was asked or stated | Reviewer agrees scope is unambiguous, or task is dropped |
| Fan-out map | Every system and field that holds the address | System inventory for the business | Signed off by the operations owner |
| Expected end state | Value each field should hold after the call | Derived from scope decision and fan-out map | Written before grading |
| Observed end state | Value each field actually holds, with change time | Audit log or change history entries | Pulled at a fixed delay after the call |
| Outcome maturity | What happened next, such as a delivery or returned mail | Shipment, invoice or mail return record | Graded only after the maturity window closes |
Which records should change for each kind of correction?
Use a fan-out map per business, filled in from its own systems. The rows below are illustrative system types, not a claim about any one company's stack. The point is that each correction type has a different right answer for each record.
| Record | Customer moved | One order to a different place | Typo fixed during the call |
|---|---|---|---|
| CRM contact mailing address | Change | Leave | Change if the typo was saved there |
| Billing address on the payment profile | Change only if the customer confirms billing moved too | Leave | Change if the typo was saved there |
| Default shipping address | Change | Leave | Change if the typo was saved there |
| Open order not yet shipped | Change | Change that order only | Change |
| Order already shipped | Do not edit; start a carrier or return process | Do not edit; same | Do not edit; same |
| Subscription or recurring delivery | Change | Leave | Change if affected |
Worked case: a corrected unit number on a moving customer
This case is illustrative and invented. A customer calls a home goods retailer to report a move. Starting state: the CRM contact, the payment profile's billing address, the default shipping address and one unshipped order all show the old address. A second order shipped yesterday.
At turn 6 the customer gives the new address with unit 40. At turn 9, after the agent reads it back, the customer says "no, unit 14, not 40". At turn 11 the agent asks whether billing should change too; the customer says yes. At turn 13 the agent confirms and closes.
- Decision: the task fails. The agent's closing summary read back unit 14, which is exactly why a plausible final message is not evidence; the billing write used the value captured before the repair turn.
- Outcome check: after the maturity window, the next invoice is generated with unit 40, and the shipped order arrives at the old address. Both confirm the grade.
- Counterexample: on a different call the customer asks to send one gift order to a friend. An agent that changes the default shipping address and CRM contact as well has over-propagated, and that task also fails even though every system now agrees.
| Record | Expected end state | Observed end state | Grade |
|---|---|---|---|
| CRM contact mailing address | New address, unit 14 | New address, unit 14 | Pass |
| Payment profile billing address | New address, unit 14 | New address, unit 40 | Fail: correction did not reach this write |
| Default shipping address | New address, unit 14 | New address, unit 14 | Pass |
| Unshipped order | New address, unit 14 | New address, unit 14 | Pass |
| Shipped order | Unchanged; carrier or return process started | Unchanged; no process started | Fail: agent told the customer it was handled |
How to score a pack of correction calls, not just one
One graded call shows the mechanics; a pack of them shows where an agent tends to fail. Score each record in each task as pass, fail, or not applicable, then roll results up by correction type and by record, never into one success rate. An agent that always updates the CRM but rarely the billing profile has a specific, fixable defect that a single blended number would hide.
At a glance
How to score a pack of correction calls, not just one
Keep under-propagation and over-propagation as separate counts. They have different causes (a missed write versus a mis-scoped request) and different business costs, and a model tuned to reduce one often raises the other. Track repair-turn handling separately as well: the share of calls where the value written matched the corrected value rather than the first one spoken.
Repeat each task more than once where the agent is nondeterministic, and report how consistently it passes rather than whether it passed at least once. Report the spread across repetitions alongside each rate.
Checklist · 6 items
- Per-record pass rate for each correction type.
- Under-propagation count: records that should have changed and did not.
- Over-propagation count: records that changed but should not have.
- Repair fidelity: writes that used the corrected value, not the first value spoken.
- Scope questioning: calls where the agent asked which addresses to change when the request was ambiguous.
- Outcome agreement: tasks whose later delivery or invoice evidence matched the record-level grade.
0 of 6 ticked. Tick each item as you go.
When is an outcome mature enough to grade?
Grade the records at a fixed delay after the call, long enough for normal asynchronous syncs to finish, then grade the downstream outcome again after a longer window. A status field that says "address updated" can be true in one system while a nightly job overwrites it from another.
Outside signals help. In the U.S., the Postal Service's NCOALink product lets licensed providers match mailing lists against change-of-address filings so mailers can update addresses; a mismatch between a customer's filing and the address your systems hold after the call is a useful flag, though not proof of an agent error. Returned mail, failed deliveries and corrected invoices are stronger evidence because they come from the business's own records.
Record which outcome signals were available for each task and how long after the call they were observed. A task whose outcome window has not closed should be labeled pending, not passed.
Where privacy rules shape the evaluation
Address corrections are personal data corrections, and some privacy laws treat propagation as an obligation, not a quality goal. Under Article 19 of the EU General Data Protection Regulation, a controller must communicate a rectification to each recipient to whom the data were disclosed, unless that proves impossible or involves disproportionate effort. A test that measures fan-out is therefore close to a test of that duty for EU customers, though passing it is not a compliance determination.
Call recordings carry their own consent and retention rules, which vary by jurisdiction, and voiceprints can raise biometric questions. Counsel should review the specific recording practices before any calls are used for evaluation or training, and the task pack should include only recordings whose use basis has been confirmed.
Sourcing call recordings tied to system outcomes
The scarce ingredient in this task pack is the link between a real call and the system records it changed. Contact centers keep recordings; operations teams keep change logs; the two rarely sit in one dataset. A buyer needs both, joined by call and customer, with timestamps that let a grader see which write followed which turn.
Linked call-and-record data of this kind is something a buyer commissions through SourceX rather than picks from a catalog: SourceX looks for U.S. businesses that hold it once the buyer sets out scope, volume, language, format, timing and licensing needs. With the business that agrees to supply, SourceX handles qualification, rights review, contracting, preparation and delivery. A request for correction-recovery tasks should specify the correction types, the record systems in the fan-out map, and how long an outcome window you need after each call.
Frequently asked questions
What is voice agent correction recovery?
It is an agent's ability to notice when a caller corrects information mid-call, replace the earlier value everywhere it was used, and confirm the corrected value. A full test checks the audio, the transcript, every affected system and the later outcome.
Why isn't the agent's confirmation message enough to grade the task?
The agent can read back the right value while a system write used the earlier one. Only the record states, compared with an expected end state written before grading, show whether the correction propagated.
How should a test handle a customer who changes only one order's address?
Treat it as a different correction type with its own expected end state: that one order changes and the default and CRM addresses stay as they were. Updating everything is over-propagation and should fail.
How long after a call should the outcome be checked?
Check records after a fixed delay that covers routine syncs, then check downstream outcomes such as deliveries, invoices or returned mail after a longer window set per business. Label tasks pending until that window closes.
Do I need the original audio, or is a transcript enough?
Keep the audio for any span that carries the address or the correction, because transcription errors on numbers and street names are common. A person should verify those spans against the recording before the task is used.
Sources
- tau-bench grades a task by replaying the ground-truth actions and comparing database hashes; incorrect database changes score zero. Source
- The tau-bench retail environment has separate tools to modify a user's default address and a pending order's shipping address. Source
- Stripe's Customer object keeps a billing address and a separate shipping address. Source
- Salesforce standard objects expose address compound fields such as MailingAddress on Contact and BillingAddress and ShippingAddress on Account. Source
- pyannote.audio is a speaker diarization toolkit whose output lists turns with start time, stop time and speaker. Source
- The Whisper README says its performance varies widely depending on the language. Source
- USPS NCOALink lets licensed providers update mailing lists using change-of-address records. Source
- GDPR Article 19 requires a controller to communicate rectification to each recipient of the data unless impossible or disproportionate. Source
Tell us what data you need
Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.