Spoken Intent Data: IVR and Self-Service Utterances Labeled with Intents and Slots

What a labeled IVR utterance contains

A labeled IVR utterance is a two-to-ten-second caller turn, usually the answer to an open prompt such as "Briefly tell me why you're calling," plus the labels needed to train intent classification and slot filling directly from speech. Research resources such as SLURP frame spoken language understanding as inferring meaning directly from audio rather than from a pipeline of ASR then text NLU, which can reduce error propagation [1]. The same paper notes that open SLU resources are scarce [1]; SLURP itself covers home-assistant style commands, not billing disputes or prescription refills.

That gap is why contact-center teams look for utterances from real self-service systems. The useful unit is not the full call; it is the caller's turn at a specific prompt node, with the system's reaction attached. Full conversations belong to a different purchase, covered in our guide to specifying contact-center audio data and the call center audio datasets page.

Platforms often store these turns separately from agent audio. Amazon Connect, for example, lets an operator choose whether to record customer and system audio during IVR interactions, independently of agent-interaction recording [2]. Ask a supplier which platform produced the files, because that determines whether the utterance is already segmented or must be cut out of a longer recording.

Label layers that make utterances trainable

Each utterance needs four label layers: transcript, intent, slots and outcome. Missing any one of them limits what you can train or evaluate.

• Transcript. Verbatim, including disfluencies, false starts and "uh, yeah, um, my bill." Clean transcripts hide exactly the noise an end-to-end model must learn; see verbatim vs clean transcription standards.
• Intent. One primary label from the supplier's taxonomy, plus a secondary label for multi-intent turns such as "pay my bill and change my address." Record the taxonomy version.
• Slots. Character or token spans with a type (account_number, date, amount, product) and a normalized value. "The fifteenth" becomes 2026-03-15 only if the reference date is stored.
• Outcome. The routing destination, whether the call was contained in self-service, transferred, or abandoned, and whether the caller re-prompted or hit a "no match" event.

The outcome layer is what separates operational data from crowd-collected command sets. It lets you measure misroutes against what actually happened, the same logic behind outcome-labeled evaluation data. Prior task-oriented dialogue work also notes that slot-and-value labels alone miss the actions agents take under policy [6], so ask whether downstream transfer reasons were captured.

Illustrative utterance record

The record below shows the minimum fields to request. Treat it as a spec you can paste into a data request, not a description of any dataset.

Illustrative example: invented to show structure; it does not describe an available dataset.

{
  "utterance_id": "ivr-000184-n02",
  "audio_uri": "audio/ivr-000184-n02.wav",
  "codec_original": "G.711 mu-law",
  "sample_rate_hz": 8000,
  "channels": 1,
  "duration_s": 3.4,
  "prompt_node": "main_menu_open_prompt",
  "prompt_text": "In a few words, tell me why you're calling.",
  "attempt": 2,
  "asr_hypothesis_production": "I need to pay my pill",
  "asr_confidence": 0.61,
  "transcript_verbatim": "uh I need to pay my bill from last month",
  "intent": "billing.make_payment",
  "intent_secondary": null,
  "taxonomy_version": "routing-v14",
  "slots": [
    {"type": "billing_period", "text": "last month", "value": "2026-02", "reference_date": "2026-03-09"}
  ],
  "redactions": [],
  "routing_destination": "payments_selfservice",
  "contained": true,
  "event_flags": ["no_match_attempt_1"],
  "language": "en-US",
  "consent_notice_version": "ivr-greeting-2025-11"
}

Two fields deserve attention. Keeping the production ASR hypothesis and confidence beside the human transcript lets you study where the current engine fails, and the attempt counter exposes re-prompts, which are often the hardest and most valuable utterances.

Audio specs: match the telephony channel

IVR utterances should be bought at the sample rate and codec your production system receives, which for most PSTN traffic is 8 kHz narrowband. Training a routing model on 16 kHz studio speech then deploying on G.711 audio is a common source of accuracy loss. Our guide to telephony vs wideband ASR training covers the tradeoffs in detail.

Ask for the original codec (G.711, G.729, Opus over WebRTC) rather than a re-encoded MP3, and for the source sample rate before any upsampling. IVR audio also carries artifacts that a robust model needs: speakerphone echo, car noise, DTMF tones cut into speech, and barge-in over the prompt. Confirm whether the supplier trimmed prompt audio or left system speech in the file; for container and channel conventions, see audio file specs for speech datasets.

Plan the intent taxonomy mapping before you buy

Every company's intent taxonomy is different, so plan a mapping from the supplier's labels to yours before you sign. One utility's "billing.dispute" may be split into three intents elsewhere, while a bank's "card.lost" may fold together lost, stolen and damaged. A mapping that is discovered after delivery often means relabeling.

Illustrative example: invented to show structure; it does not describe an available dataset.

Ask for the full label list with counts per intent, the labeling guideline, and inter-annotator agreement on a held-out sample. Watch for "other" buckets that hold a large share of traffic, and for intents whose definitions changed mid-collection when the taxonomy version incremented.

Consent, notices and privacy in IVR recordings

IVR recordings are only licensable for training if the recording notice and the supplier's privacy commitments support that reuse. All-party consent statutes such as California Penal Code section 632 prohibit recording confidential communications without the consent of all parties [3] (section 632.7 separately covers calls involving cellular or cordless phones), which is why IVR greetings typically announce recording. Whether that notice mentioned AI or model training is a separate question; see call-recording consent for AI training.

The FTC has warned that quietly adopting more permissive data practices, such as using customer data for AI training, may be unfair or deceptive [4]. Ask the supplier for the greeting script and privacy notice versions in effect during collection, and map each utterance to its notice version. If the IVR was operated by an outsourcer or CCaaS vendor, check who holds the right to license; see client data held by service providers.

Slots are where personal data concentrates: account numbers, dates of birth, ZIP codes and card digits spoken aloud. Ask how spoken PII was removed from both transcript and audio (bleeping, silence insertion or segment drop), and whether payment-card prompts were excluded. Voice itself can identify a caller; voice anonymization remains an open research problem whose effect on ASR utility is still being benchmarked [5], and biometric exposure is covered in voiceprints and BIPA.

This page is general information, not legal advice. Confirm requirements with counsel for your jurisdiction and use case.

Buyer checklist for IVR utterance sourcing

Use this checklist to scope a request and to review samples before any license is signed.

Illustrative example: invented to show structure; it does not describe an available dataset.

1. Source system and prompt nodes: platform, prompt texts, and whether utterances come from open prompts, directed menus or both.
2. Audio: original codec, sample rate, channels, trimming policy, and duration distribution.
3. Labels: verbatim transcript, intent with taxonomy version, slots with normalized values, outcome and event flags.
4. Coverage: intent distribution, long-tail intents, language and accent mix, time span, and seasonal events such as outages.
5. Quality: guideline document, agreement scores, and a reviewer sample you can relabel yourself.
6. Rights: recording notice versions, privacy policy history, authority of the party licensing the data.
7. Privacy: spoken PII handling in audio and text, card-data exclusions, and residual-risk statement.
8. Documentation: a data statement covering speaker and language characteristics [7] and a dataset card with license and size metadata [8].

For multilingual or code-switched callers, extend item 4 with the guidance in multilingual ASR training data. Teams building conversational voice agents rather than routers should also review the voice agent training data page and the broader speech and audio datasets buyer's guide.

How SourceX handles requests for spoken intent data

SourceX sources operational datasets, including support and sales histories, from US companies and manages the commercial process, from licensing agreements to ongoing purchases. Data is sourced on request rather than held in stock, so a request is not a promise of a match. You describe the utterances, labels and outcomes you need, and SourceX looks for US businesses that hold that data; every release is approved by the supplying company.

Each dataset is rights-reviewed for ownership and consents and delivered under a license that defines records, uses, term and delivery. Personal details such as names, emails, phone numbers and account numbers are removed or replaced before delivery, the method is recorded and a sample is checked, though no method is perfect. Delivery runs through private, access-controlled workflows only after an executed agreement and supplier approval. You can start by describing the IVR data you need on the buyers page.

Source labeled IVR and self-service utterances

If your routing or voice self-service model needs real caller utterances with intents, slots and outcomes, describe the data and the label layers you require. SourceX assesses data and licensing permissions with suppliers, and nothing is contracted until a supplier agrees. Submit a spoken intent data request.