Skip to content

AI uses for records

Business voicemail workflow dataset: from message to verified outcome

By SourceX Editorial · Reviewed by Noah Loul ·

9 sections

Short answer

Voicemail recordings become useful training data when each one travels with its workflow: the audio and an aligned transcript, the record the phone system created, every callback attempt, the work that followed and a verified outcome from a system of record. Keep what the caller said separate from what was later confirmed, and never treat a closed status as proof the request was resolved.

Call to transcript

Business voicemail workflow

This page covers business voicemail workflow dataset: from message to verified outcome.Illustrative

Key takeaways

  • A voicemail is a request that starts a workflow; the dataset should carry that workflow, not just the audio.
  • Keep three things apart: what the caller said, what the business recorded about it and what actually happened.
  • A ticket's closed status is a field someone set, not evidence the caller's problem was solved.
  • Phone platforms may delete voicemail audio while keeping the transcript, so check what survives before scoping.
  • Matching voicemails to records by phone number alone produces confident, wrong links on shared lines.

What workflow context should accompany voicemail recordings in training data?

Each recording should arrive with the chain that shows what the business did about it: the record the phone system created, the person or queue it was routed to, every callback attempt and its result, the work order or case that followed, and an outcome taken from the system that would know. Audio alone teaches transcription. Audio with its chain teaches the task.

That chain is what an AI developer needs to build an agent that listens to a message, decides what it is about, creates the right record and schedules the right follow-up. It is also what an evaluator needs to check whether such an agent got it right, because the business already did the task once and wrote down how it ended.

The rule that keeps the dataset honest is separation. What the caller said is a claim. What the business recorded about the call is a second claim, made by an employee or a system. What happened afterwards, confirmed by an invoice, a delivery scan or a closed case with a resolution code, is the outcome. A good record keeps the three in different fields and says which one each label came from.

Why a voicemail is not just a short phone call

A voicemail has one human speaker, no agent to ask a clarifying question and a system around it that shapes what gets kept. Those differences change what a dataset has to capture.

The caller corrects themselves without prompting, saying a callback number and then a different one, or naming one site and then another. The greeting and the beep are system audio that a transcript may or may not include. The message can end because the caller hung up, pressed a key or hit the platform's length limit, and the ending matters when a request is cut off mid-sentence.

The phone platform also leaves its own records. On Twilio's Record verb, the action callback receives the recording URL, its duration and a Digits value that shows which key ended the recording or that the caller hung up; Twilio notes the file may not be ready at that moment and points to a separate status callback. In Zendesk Talk, a voicemail creates a ticket with the recording attached, transcription is an optional setting, and audio can be set to delete after a period while the transcription stays on the ticket. Before scoping a dataset, find out which of those settings applied in which years, because they decide whether audio, transcript or both still exist.

The voicemail to action linkage schema

Model each voicemail as six linked layers, each with its own identifiers and its own check. The examples name common systems for orientation; field names differ by platform and by how a company configured it.

The voicemail to action linkage schema
LayerFields to keepWhere it usually livesVerification check
1. AudioRecording id, start time, duration, end reason (key, hangup or limit), line or number dialed, channel layoutTelephony platform recording log, such as Twilio or the phone system in a help deskDuration and end reason match the file; audio still exists under the retention settings
2. Transcript and segmentsTranscript text with word timings, system greeting marked separately, caller segment, any self-corrections taggedPlatform transcription, a speech-to-text job output or a human transcriptA sampled reference transcript scored for substitutions, deletions and insertions, with numbers and names checked by hand
3. Extracted requestIntent, entities the caller stated (account, site, callback number, deadline), urgency in the caller's own wordsTicket subject and first comment, or an agent's summaryEvery extracted entity points to a time range in the audio
4. Created record and routingTicket or case id, creation time, queue or assignee, priority set, link back to the recording idZendesk ticket, Salesforce case, ServiceTitan call record or a CRM taskLink is a stored id, not an inferred match on phone number and time
5. Follow-up actionsEach callback attempt with time, result (reached, no answer, wrong number), any appointment or order createdTicket comments and events, dialer logs, scheduling systemAttempt times are later than the voicemail; the reached party matches the caller
6. Verified outcomeOutcome code, completion evidence (invoice, delivery, resolution note), time to outcome, outcome maturity dateBilling, field service, order or case resolution recordsOutcome comes from a system that records completion, not from a status field alone

Worked case (illustrative): a no-heat voicemail traced to a repaired furnace

This case is invented to show the schema in use. A heating and cooling contractor receives an evening voicemail of about one minute. The caller says the furnace stopped overnight, gives a street address, reads a callback number, then says to use a different number because the first is a work line.

The table traces the message through its records. The last column is the point of the exercise: each record proves something specific and leaves something else unproven.

  • Label for an agent task: create an urgent ticket and call the second number, citing the audio time range of the correction.
  • Label for an evaluation of the human workflow: one wasted attempt caused by the missed self-correction.
  • Outcome label: resolved, with completion evidence from the work order and invoice, matured after the review window.
Worked case (illustrative): a no-heat voicemail traced to a repaired furnace
StepEvidence recordWhat it provesWhat it does not prove
Message leftRecording with end reason hangup and a transcript segment for the callerThe request, the address and both numbers as spokenThat the transcript got the second number right
Record createdHelp desk ticket created by the phone system, linked to the recording idThe business received the message and whenWhich number staff would call back
TriageTicket priority raised to urgent with a dispatcher note citing no heatA person read or heard it and judged it urgentThat the judgment matched company policy
Callback attempt 1Dialer log: first number, no answerAn attempt was made at a recorded timeContact; it also shows the self-correction was missed
Callback attempt 2Dialer log: second number, reached; appointment createdContact with the caller and a booked visitThat the visit fixed the problem
OutcomeWork order completed with parts used and an invoice issued, no return visit within the review windowThe furnace was repaired and stayed repaired for that windowCustomer satisfaction, unless a survey was recorded

Checking that the transcript says what the caller said

Score a sample of transcripts against a human reference, and score the fields that drive action separately from the rest. A transcript can read cleanly and still swap one digit of a callback number, which is the error that breaks the workflow.

NIST's sclite tool is a standard way to do the general part: it aligns a hypothesis with a reference and reports substituted, inserted and deleted words as percentages of the reference words. Add a field-level check on numbers, names, addresses and dates, because a single substitution there matters more than several in filler words.

Two cautions apply to automatic transcripts. OpenAI's Whisper model card says its predictions may include text that was not spoken, and that accuracy varies across languages, accents and dialects, so a fluent transcript is not proof of a faithful one. And when a transcript carries speaker labels, Amazon Transcribe's documentation describes them as labels such as spk_0 assigned by voice, randomly and inconsistently across files, without identifying anyone. In a voicemail that means the label can separate greeting from caller, but it cannot tell you who the caller is; identity has to come from the linked record.

Why a closed ticket is not a resolved request

Take the outcome from evidence of completion, not from the ticket's status. Statuses get set for many reasons: an automation closes tickets after a period without reply, an agent merges a duplicate, or a team closes a ticket when the work moves to another system.

For each outcome label, record which system it came from and when it matured. A repair is only known to have held after a return-visit window passes; a billing question is only resolved once the corrected invoice is paid or credited. Pick a maturity rule for each workflow before labeling, and leave items whose outcome has not matured out of the outcome set rather than labeling them resolved.

This is the distinction between a status field and a task outcome that evaluation sets can easily blur. A model trained on status will learn when tickets are closed, which is a different skill from learning when problems are solved.

This counterexample is illustrative. A property management office shares one main number. Two tenants from the same building leave voicemails ten minutes apart, one about a leak and one about a lockout. The help desk creates both tickets, but a later import that matched recordings to tickets by caller number and time attached both recordings to the leak ticket.

The resulting record looks complete: audio, transcript, ticket, callback and a plumber's invoice. It is still wrong for the lockout message, whose outcome is now a plumbing repair. Nothing in the audio or the outcome contradicts the link on its own; only the stored recording id, or a transcript-to-subject check, catches it.

Two rules follow. Require links built from ids the platform stored at creation time, and treat inferred matches as a separate, lower-confidence tier. And run a consistency check that compares the extracted request with the ticket subject and the outcome category, sending mismatches to a person.

Caller rights and personal data come before any export

Voicemails carry the caller's voice and often names, numbers, addresses and account details. Whether recordings may be used or licensed for AI depends on the recording notices given, the company's own policies and contracts, and the laws where callers and the business are located, so counsel should review the specific facts before any audio leaves the company.

Keep three questions apart. Having access to the recordings is not the same as having permission under privacy rules to use them this way, and neither is the same as holding the contractual right to license them. De-identifying a transcript does not settle the voice itself, which can identify a person on its own.

Practical scoping often starts with transcripts and workflow records, with audio added only for the lines and years where notices and retention settings are clear.

Requesting voicemail with its workflow attached

If you are sourcing this data, ask for the linkage, not just the files: recording ids that join to tickets, callback logs, the downstream work record and an outcome field with its source system. Say which layers are required and which are optional, and how outcome maturity should be decided.

Voicemail datasets of this kind are not sitting on a shelf: SourceX finds them on request among U.S. businesses that hold the records. A request can set scope, volume, language, format, timing and licensing needs, and nothing is delivered until the supplying company has signed off on scope and terms.

Frequently asked questions

Is voicemail audio useful for training without the tickets it created?

It is useful for speech recognition, but not for training or grading an agent that has to act on a message. Without the linked ticket, callbacks and outcome, nothing shows what the right action was or whether it worked.

What is the minimum linkage for a voicemail workflow record?

A stored recording id, the ticket or case created from it, at least one follow-up action with its result, and an outcome taken from a system that records completion. Anything less shows a request without showing how it was handled.

How do you check a voicemail transcript's accuracy?

Score a sample against human reference transcripts for substituted, inserted and deleted words, then check the action fields such as numbers, names and addresses by hand. Errors in those fields break the workflow even when the overall error rate looks low.

Can speaker labels identify who left a voicemail?

No. Speech services that label speakers assign labels by voice and, for example in Amazon Transcribe, the labels are not consistent across files and do not identify anyone. The caller's identity has to come from the linked business record.

What if the phone system deleted the audio but kept the transcript?

Then the transcript and the workflow records can still form a dataset, but nobody can re-check the transcript against the audio. Record that limit on each item, and keep audio-based and transcript-only items in separate subsets.

Sources

  • Twilio's Record verb sends the recording URL, duration and a Digits value to the action URL, and the recording file may not be available until the recording status callback. Source
  • Zendesk Talk voicemails create a ticket with the recording attached, transcription is optional, and audio can be auto-deleted while the transcription remains on the ticket. Source
  • Amazon Transcribe speaker diarization labels each unique speaker's text with a value such as spk_0. Source
  • Amazon Transcribe speaker labels are randomly assigned, not consistent across files and do not identify speakers. Source
  • The Whisper model card warns that predictions may include text not actually spoken and that accuracy varies across languages, accents and dialects. Source
  • NIST sclite scores speech recognition output against a reference and reports substituted, inserted and deleted words as percentages of reference words. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies