Skip to content

AI uses for records

Healthcare referral routing dataset: an AI test built on a misrouted referral

By SourceX Editorial · Reviewed by Noah Loul ·

10 sections

Short answer

Build the test as a packet: freeze the referral at the moment it reached the wrong work queue, show only administrative evidence such as service code, specialty, location, network status and attachment flags, include the routing rule in force, and hide the correct decision and the linked outcome for scoring. Proof comes from acceptance, a linked appointment or closure, not a rerouted status, and every packet needs a counterexample twin.

License terms

Healthcare referral routing

This page covers healthcare referral routing dataset: an AI test built on a misrouted referral.Illustrative

Key takeaways

  • A misrouted referral makes a strong administrative test when it is frozen at the moment it reached the wrong queue.
  • Route on service, specialty, location, network status, urgency category and attachment flags; clinical narrative stays out.
  • A queue change is not proof; acceptance, a linked appointment or a closure back to the referrer is.
  • Pair every packet with a near twin that was correctly placed, so moving everything unusual does not score well.
  • When routing depends on clinical content, the correct answer is escalation to triage, and the packet should score it that way.

How do you build an administrative AI test around a misrouted referral?

Build it as a packet with five parts: the frozen starting state when the referral entered the wrong work queue, the administrative evidence a scheduler could see, the routing rule in force that day, the correct decision, and a linked record that shows what actually happened next. The model reads the starting state and evidence, proposes an action, and is scored against the decision and the observed outcome.

The test is administrative by design. It asks whether an agent can tell that a referral sits with the wrong team, name the right team or the missing item, and move it without clinical judgment. It does not ask whether the referral was clinically appropriate, how urgent the condition is, or what care the patient needs; those questions belong to clinicians.

A healthcare referral routing dataset built this way is small and dense. Each packet takes real effort to assemble, but one well-linked case teaches more than thousands of referrals that only show a final queue name.

Where is the nonclinical line for a routing task?

Keep the fields a scheduler or referral coordinator uses to route work, and leave out clinical narrative. Routing usually depends on the requested service, specialty, location, payer network status, required attachments and the urgency category the referring office selected, not on the reasons behind them.

That boundary also matches how HHS describes the minimum necessary standard: for routine and recurring disclosures, a covered entity's policies may use standard protocols that limit the information to the minimum necessary for that type of disclosure, and when an entire medical record is needed the policies must say so explicitly with a justification. A routing test never needs the entire record. The table is a starting point for the field review a privacy officer will do.

Where is the nonclinical line for a routing task?
Field groupKeep, transform or excludeReasonTypical source
Requested service code and specialtyKeepPrimary routing inputReferral order or intake form
Receiving location and queueKeep, pseudonymize queue names if they reveal staffDefines the routing targetWork queue configuration
Payer and plan network statusKeep as category (in network, out of network, unknown)Often decides which location can acceptEligibility check result
Urgency category chosen by the referrerKeep as stated, never re-derivedAffects queue priority, not clinical assessmentReferral form
Attachment checklist (present or missing)Keep as flagsMissing items cause send-backsDocument index
Diagnosis narrative, notes, resultsExcludeClinical content outside the taskClinical documents
Patient name, contact details, exact datesExclude or transformIdentifiers; dates become intervalsRegistration and audit logs

What starting state should the packet freeze?

Freeze the referral as it stood at the moment it reached the wrong queue: the queue name, who or what assigned it, the status, the routing rule version, and the work-queue history up to that point. Anything that happened later is hidden from the model and kept for the answer key.

The HL7 FHIR Task resource is a useful vocabulary even when the source system is not FHIR based. Its specification describes a Task as tracking the state of completion of an activity, with an owner defined as the party responsible for managing task execution and a business status for nuances of the business state. It separates focus, the request being fulfilled, from basedOn, a higher-level authorization. In a routing packet, focus is the referral, owner is the queue, and business status carries labels such as awaiting records or pending eligibility.

Capture who made the first assignment as well. An interface rule, an intake clerk and a self-scheduling portal can fail in different ways: an interface rule may fail consistently on one code range, staff on unfamiliar services, and a portal when patients pick the closest-sounding option. A test set that mixes all three without a label cannot tell an AI team which failure it is learning to fix.

Record the routing rule as text plus a version or effective date. Rules change when a clinic opens, a provider leaves or a payer contract ends, and a packet that pairs an old referral with today's rule produces a false error.

The administrative routing and correction task packet

Each packet below is self-contained and labeled real or illustrative. Field names are suggestions to map onto your own referral management or EHR work queue export.

The administrative routing and correction task packet
Packet partContentsVisible to the modelAcceptance check
HeaderPacket ID, real or illustrative label, source system, routing rule version, review sign-offRule version onlyPrivacy and administrative reviewers signed
Starting stateCurrent queue, assigning actor (rule, staff role, interface), status, time since receipt as an intervalYesMatches the work-queue history at that moment
Administrative evidenceService code, specialty, location requested, network category, urgency category, attachment flagsYesEvery field passes the boundary table
Applicable ruleRouting rule text that decides the correct queue or the send-back reasonYes, as policy textRule was in force on the referral date
Expected decisionReroute to named queue, return for missing item, or escalate to clinical triageNoAgreed by two administrative reviewers
Outcome linkLater work-queue status, acceptance or decline, appointment status or closureNoObserved in the record, not inferred
Counterexample pairA similar packet where the starting queue was correctYes, scored separatelyDiffers from the main packet by one rule-relevant field

What proves the reroute worked, beyond a changed queue name?

The proof is a later record showing that the receiving team accepted the referral and acted on it, not a status field that says rerouted. A queue change proves only that someone moved the referral; it does not prove the new queue was right.

Several records can serve, in rising order of strength. The FHIR Task status set includes rejected, where a potential performer decides not to execute the task before any action, and completed. The FHIR Appointment resource has a basedOn element for the request an appointment is allocated to assess, such as an incoming referral, which is the cleanest link from referral to booking. The ONC 360X closed-loop referral project requires the referral recipient to send either an accept or a decline, and its referral outcome concept covers care that was not rendered or ended early as well as completed results.

What proves the reroute worked, beyond a changed queue name?
EvidenceWhat it provesWhat it does not proveUse in the packet
Queue change in the work-queue logSomeone moved the referralThat the destination was correctContext only
Acceptance by the receiving teamThe destination took ownershipThat the patient was seenMinimum outcome link
Decline or send-back with reasonThe destination refused and whyWhere it should have goneNegative outcome; keep the reason code
Appointment linked to the referralBooking happened against this requestThat the visit occurredStrong outcome link
Appointment completed or closure sent to referrerThe loop closed administrativelyAnything clinical about the visitStrongest administrative outcome

Worked case: an orthopedic referral lands in the general surgery queue

This case is illustrative and invented to show the packet; it does not describe a real provider or patient. A primary care office sends an electronic referral for a knee evaluation. An interface rule maps the service code to general surgery because an older code table grouped it there.

Worked case: an orthopedic referral lands in the general surgery queue
Packet partRecorded contentExpected model outputScoring note
Starting stateGeneral surgery intake queue, assigned by interface rule, status received, interval since receipt two business daysDetect that the service code points to orthopedics under the current ruleRule text is visible; code table version is in the header
EvidenceService code for joint evaluation, specialty orthopedics, in-network category, routine urgency, imaging report flagged presentUse only these fieldsAny request for clinical notes is overreach
RuleJoint evaluation codes route to the orthopedics referral queue at the requested locationCite the ruleCitation required for full credit
Expected decisionReroute to orthopedics intake; no send-back neededReroute with a one-line reasonWrong queue or added send-back fails
Outcome linkOrthopedics accepted the next business day; appointment booked against the referralHidden from the modelConfirms the answer key

The counterexample: when the receiving queue was right after all

Every main packet needs a near twin where the referral looked misrouted but was not. Without it, a model can score well by moving everything that looks unusual.

In the illustrative twin, the same knee referral sits in a different location's orthopedics queue than the one requested. It looks wrong, but the patient's plan has no network agreement at the requested site, and the routing rule sends out-of-network requests to the nearest in-network location with a notice to the referrer. The correct action is to leave it where it is and confirm the notice went out.

A second kind of counterexample marks the edge of the task. If the correct queue depends on reading clinical notes, for example choosing between two subspecialty clinics by symptom, the expected decision is escalation to clinical triage. The model earns credit for recognizing that the routing question is clinical and handing it on, and loses credit for answering it.

  • Twin packets differ from the main packet by exactly one rule-relevant field.
  • Escalation packets name the point where administrative evidence runs out.
  • Report scores on main packets, twins and escalation packets separately.

How should a routing agent be scored on these packets?

Score the action, the reason and the restraint. An agent that reroutes correctly but cites the wrong rule will fail on the next rule change, and one that pulls clinical detail into an administrative decision fails the boundary even when its queue choice is right.

Run the packets in a fixed order with no memory between them, and freeze the rule text the model sees so a later edit to a rule cannot change an old score. When a rule does change, add new packets under the new version rather than editing old ones; the history of both versions is itself useful test material.

Keep the scoring at packet level and publish counts alongside any rate, because packet sets are small and a few cases move a percentage a long way.

How should a routing agent be scored on these packets?
MeasurePass conditionFails whenApplies to
DetectionFlags a misrouted packet as needing actionLeaves a misrouted referral in placeMain packets
RestraintLeaves a correctly placed twin aloneMoves a twin that was rightTwin packets
DestinationNames the queue in the answer keyPicks a different queue or invents oneMain packets
Rule citationQuotes or names the applicable ruleGives no rule or the wrong oneAll packets
BoundaryUses only visible administrative fieldsRequests or infers clinical detailAll packets
EscalationHands clinical routing questions to triageAnswers a clinical routing question itselfEscalation packets

What has to be settled before packets leave the provider?

Three questions come first: whether the organization may use these records for building a test at all, which fields the task needs, and how the released packets are de-identified. A privacy officer and counsel answer the first; the field table helps with the second; the third needs a documented method.

HHS guidance describes two de-identification methods under the HIPAA Privacy Rule. Under Safe Harbor, all elements of dates other than the year that relate to an individual are removed, which is why the packet uses intervals such as business days since receipt instead of timestamps. Expert Determination lets a qualified expert assess risk for a specific dataset and recipient. Free-text routing notes are the usual leak, because staff write names and clinical hints into them; review or remove them before any packet leaves.

Stop, narrow the scope or ask for specialist review when routing depends on behavioral health or substance use program status, when the referral involves a minor or a sensitive service, when the only proof of outcome sits in clinical notes, or when the rule version cannot be reconstructed. Those packets are better left out than half built.

Turning routing histories into a licensed dataset

The value of referral routing histories with work-queue logs, rule versions and linked outcomes sits in the corrections, the send-backs and the declines, not in the clean referrals.

Rights review, contracting, preparation, delivery and payment are handled by SourceX when a provider or vendor licenses selected routing records to AI developers. Records move only once the business has approved the scope and terms and signed, and a short company assessment with no upload shows whether an organization qualifies.

Frequently asked questions

What is a healthcare referral routing dataset?

It is a set of administrative records showing how referrals were assigned to work queues, moved, accepted, declined or sent back, linked to the rule in force and the eventual administrative outcome. Used as a test, it checks whether an agent can route referrals and correct mistakes without clinical judgment.

What can a routing packet show the agent without crossing into clinical detail?

Service code, specialty, location, network category, the referrer's urgency category and attachment flags are usually enough to route. Diagnosis narrative, clinical notes and results stay out; if routing truly depends on them, the packet becomes an escalation case for clinical triage.

Why not use the final queue name as the label?

The final queue only shows where a referral ended up, which may itself be wrong or the result of several moves. Acceptance by the receiving team, a linked appointment or a closure back to the referrer is the evidence that the routing worked.

Can the test use illustrative packets instead of real ones?

Yes, as long as each one is labeled illustrative and built from the organization's real rules and field structure. Illustrative packets are useful for edge cases that are rare in the record, but scores on them should be reported apart from scores on real packets.

Does a routing test need HIPAA review if it excludes clinical notes?

Routing records still describe a patient's care request, so the privacy officer and counsel should review the use and the de-identification method. Excluding clinical notes narrows the review; it does not remove it.

Sources

  • HHS guidance on the minimum necessary standard: routine and recurring disclosures may follow standard protocols limited to the minimum necessary, and policies must explicitly justify use of an entire medical record. Source
  • The HL7 FHIR Task resource describes an activity that can be performed and tracks the state of completion of that activity. Source
  • In the FHIR Task definition, owner is the party responsible for managing task execution, businessStatus holds business-specific nuances, focus is the request being fulfilled and basedOn is a distinct higher-level authorization. Source
  • The FHIR Task status code system defines rejected as a potential performer deciding not to execute the task before any action, and completed as the task having been completed. Source
  • FHIR Appointment.basedOn is the request this appointment is allocated to assess, for example an incoming referral. Source
  • In the ONC 360X closed-loop referral workflow, the referral recipient must transmit an accept or a decline, and referral outcome covers care not rendered or ended before completion. Source
  • HHS de-identification guidance: under Safe Harbor, elements of dates other than year directly related to an individual are removed; Expert Determination is the alternative method. Source

See if your company qualifies

A short company assessment. No data uploads are needed.

See if your company qualifies