Provider roster and directory data for provider data management AI

This page is general information, not legal advice. Confirm requirements with counsel for your jurisdiction and use case.

Why roster ingestion is a hard ML problem

Roster ingestion is hard because every delegated group, IPA and health system sends a different spreadsheet for the same underlying facts. One file puts each provider on one row with locations in numbered columns; another repeats the provider per location; a third merges cells, hides tabs and embeds term dates in a comments column. The model has to map arbitrary headers to a canonical schema, split and pivot rows, and decide whether "Dr. J. Smith, MD" at "Suite 200" is an existing practitioner-location record or a new one.

The canonical grain is usually practitioner × service location × plan product or network, with effective and term dates on each relationship. Typical fields include individual (type 1) NPI, organization (type 2) NPI, TIN, specialty taxonomy code, license number and state, practice address, phone, hours, languages, accepting-new-patients status and hospital affiliations. Errors propagate: a wrong TIN-to-location link mis-assigns a provider to a network, and a stale address becomes a ghost listing.

Directory accuracy also has a compliance clock. The No Surprises Act provisions of the Consolidated Appropriations Act, 2021 added federal directory verification and update duties for group health plans and issuers with networks; confirm the current cadence and scope with counsel as of October 2026. That recurring verification generates a steady stream of attestations, outreach attempts and removals, which is exactly the labeled history a directory model needs.

Where the labels come from: corrections, not the registry

Gold labels in this domain are the plan's own match and correction decisions, not the public registry. CMS publishes the NPPES registry as a free public download, and the taxonomy codes in it need the separate NUCC code set to interpret. Anyone can join those; a model trained only on them learns what providers self-reported to CMS, which often lags real practice locations.

The licensable signal sits downstream, in provider data management systems and roster ingestion queues:

• Roster-to-load diffs: the original file, the normalized staging rows and the rows that reached the directory, linked by line.
• Match decisions: for each incoming row, the matched provider and location IDs, match method (exact NPI, fuzzy name plus address, manual) and the analyst's override.
• Rejection and pend reasons: missing license, NPI-taxonomy mismatch, address failing USPS validation, TIN not on contract.
• Directory change requests and attestations: who asked, which field changed, the old and new values, the verification channel and the outcome, including "unable to verify, suppressed."
• Credentialing milestones: application received, primary-source verification complete, committee decision and recredentialing dates, which explain why a provider appears on a roster but not yet in the directory.

Public entity-resolution benchmarks are a weak substitute. Research on ER evaluation shows that benchmark construction and pairwise metrics can misstate real performance [1], and label noise in test sets is common enough to reorder models [2]. Real analyst decisions carry their own noise, so ask for reviewer IDs and override flags; see verifying outcome labels in operational records.

Provider data versus PHI: what to minimize

Provider rosters are generally provider data, not patient data, but they still contain personal information that should be minimized before licensing. Individual practitioners' home addresses, personal cell numbers, dates of birth, SSNs on credentialing applications, DEA numbers and malpractice histories are sensitive even when no patient appears. Decide which of these the model actually needs; most extraction and matching tasks work with NPI, license number, practice address and a hashed or replaced personal identifier.

Patient data can still leak in. Panel-size reports, attached member lists and free-text notes on change requests sometimes name members, and those fragments are protected health information that must be de-identified under HIPAA Safe Harbor or Expert Determination [3] or removed. Run a scan for member IDs and dates of service before any file leaves the source environment.

Decision table: which records support which model

Match the record type to the task before you request data; roster files alone will not train a directory-accuracy agent.

Illustrative example: invented to show structure; it does not describe an available dataset.

For roster-to-load pairs, request as-of snapshots so the directory state matches what the analyst saw at the time; the pattern is covered in point-in-time correct training data.

Illustrative roster-to-directory record

A useful delivery links each raw row to its resolved directory outcome in a flat, documented record, often as JSON Lines.

Illustrative example: invented to show structure; it does not describe an available dataset.

{"roster_id":"R-2025-0412-07","source_row":34,"raw":{"Provider Name":"Smith, Jane MD","Location":"200 Oak St Ste 2","Spec":"Peds","NPI":"1XXXXXXXXX","TIN":"XX-XXXXXXX","Accepting":"Y"},
 "normalized":{"last_name":"[REPLACED]","credential":"MD","taxonomy":"208000000X","address_std":"200 OAK ST STE 2","accepting_new_patients":true},
 "match":{"practitioner_id":"P-88213","location_id":"L-4410","method":"fuzzy_name_address","analyst_override":false},
 "outcome":"loaded","pend_reason":null,"effective_date":"2025-05-01",
 "directory_events":[{"date":"2025-08-02","channel":"portal_attestation","field":"accepting_new_patients","old":true,"new":false,"result":"changed"}]}

Check that every pend reason comes from a controlled list, that override flags exist, and that roster IDs tie back to the original file hash.

How to scope a provider data request

Scope by task, network mix and time window rather than by "all directory data." Specify the plan types (commercial, Medicare Advantage, Medicaid managed care), whether delegated rosters are in scope, the number of distinct roster templates you need for generalization, and the date range covering at least several directory verification cycles. Ask the source to document its provider data management system, its canonical schema and how match decisions were recorded.

Licensing terms matter as much as fields. Confirm the supplier owns or may license the corrections and outreach outcomes, whether delegated-group contracts restrict reuse of their rosters, and what uses, term and delivery the license defines. For broader health-plan context, see utilization management review records and SourceX's healthcare administration buyer page and healthcare administration AI use cases. Spreadsheet-heavy sources are also covered under licensing spreadsheets for AI training.

How SourceX approaches provider roster requests

SourceX sources operational datasets from US companies on request and manages the commercial process, including licensing agreements and ongoing purchases. Nothing is held in stock and a request does not guarantee a match; buyers describe the data, and SourceX looks for US businesses that hold it, with every release approved by the supplying company. Each dataset is rights-reviewed for ownership and consents and delivered under a license defining records, uses, term and delivery.

Personal details such as names, emails, phones and account numbers are removed or replaced before delivery, the method is recorded and a sample is checked, though no method is perfect; health records require HIPAA de-identification. You can start a buyer request or browse the industry-specific operational data hub and the AI data guides.

Request provider roster data for AI

Describe the roster files, match decisions and directory history your model needs, and SourceX will look for US companies that hold them and agree to license. Every dataset is rights-reviewed and delivered through private, access-controlled workflows after an executed agreement and supplier approval. Describe your provider roster data request.