Validation Checks for Structured Dataset Deliveries: Schema, Missingness, Ranges and Referential Integrity

Why arrival validation comes before data profiling

Arrival validation answers a narrower question than profiling: did the supplier ship what the data dictionary and manifest promised? ISO 8000 frames this as syntactic quality, which can be checked automatically against a specification, as distinct from semantic and pragmatic quality, which need domain judgment [2]. ISO/IEC 5259-2 adds a measurable quality model for ML data that builds on ISO/IEC 25012 and ISO 8000, which gives you vocabulary for reporting completeness and accuracy consistently across deliveries [1].

The practical reason is cost. A shifted column in a ticket export or an Excel-mangled date in a billing table will propagate into every derived feature, embedding and text-to-SQL pair. The BIRD benchmark exists precisely because real databases contain dirty values that schema-only checks miss [8], so for agents and text-to-SQL work, cell-level validation is part of the task definition, not housekeeping. Profiling for distribution and coverage belongs later, in training data quality metrics and coverage gap analysis.

Stage 1: Reconcile the delivery against the manifest

The first check is that every expected file arrived intact, before you parse a single row. Compare the file list, byte sizes, SHA-256 checksums and row counts against the supplier's manifest; any checksum mismatch means a truncated or altered transfer, and object stores such as Amazon S3 expose checksums for exactly this purpose [7]. A good manifest also records the extraction query window, source system and export timestamp, which you will need when a later check fails. See the sample manifest for the fields worth asking for.

Row counts deserve a second look. A count that matches the manifest but is suspiciously round (exactly 1,000,000) often signals a LIMIT clause in the export job. A count that differs between the manifest and the parsed file often points to line breaks inside text fields, counted as rows by a naive line count or broken by an unquoted export.

Stage 2: Schema and type conformance for CSV and Parquet

Schema validation means every column in the data dictionary is present, nothing undocumented appeared, and each column parses to its declared type. The mechanics differ by format.

• CSV. RFC 4180 describes the common format: optional header line, comma separators, double-quote escaping and CRLF line breaks [4]. Check encoding (UTF-8 with or without BOM), delimiter, quote character and header names explicitly; never let a reader infer types, because "00123" postal codes and account IDs lose leading zeros and "1/2/2025" is ambiguous. The CSV delivery specification covers what to put in the contract.
• Parquet. Schema and column metadata live in the file footer [5], so read the footer of every part file and diff it against the dictionary before scanning data. Watch for physical-versus-logical type drift: INT96 legacy timestamps, decimals written as DOUBLE, timezone-naive versus UTC-adjusted timestamps, and string columns that silently became BINARY.
• JSON or JSONL sidecars. JSON Schema 2020-12 splits Core and Validation, and the Validation vocabulary is where type, required and enum live [6]. The deeper treatment is in JSON Schema dataset validation.

Treat column order as informational for Parquet but binding for headerless CSV. Treat renamed or added columns as a schema-change event, handled under schema evolution across recurring deliveries, not as a silent pass.

Stage 3: Missingness by field, not by file

Missingness checks should measure null rates per column against a documented expectation, because a 2% null rate is fine for secondary_phone_hash and fatal for ticket_created_at. Vendor guidance commonly recommends profiling completeness and anomaly rates per field before training [3]. Count every encoding of "missing": SQL NULL, empty string, whitespace, "N/A", "null", "-", 0 in a numeric field where zero is impossible, and sentinel dates such as 1900-01-01 or 9999-12-31.

Then check structure, not just totals. Nulls that cluster in one month, one region or one source system usually mean an upstream migration or a field added mid-period, which is a temporal coverage problem rather than random noise. Conditional completeness matters too: if status = 'closed', then closed_at and resolution_code should be populated.

Stage 4: Ranges, allowed codes and cross-field rules

Range and domain checks confirm that values are plausible for the business process that produced them. Typical rules for operational exports:

• Numeric bounds: invoice_amount non-negative except on credit memos; quantity an integer; discount_pct between 0 and 100.
• Dates: no future created_at relative to the export timestamp; closed_at >= created_at; all timestamps inside the manifest's extraction window.
• Code lists: priority in the documented enum; ISO 4217 currency codes; ISO 3166 country codes; ERP status codes matching the supplier's dictionary rather than free text.
• Cross-field logic: line totals summing to header totals within rounding; currency present wherever an amount is present.

Flag, do not clip. Out-of-range values in business records are often real (a refund larger than the original charge after a fee reversal), and the outcome-label verification work depends on seeing them. Also watch for pseudonymization artifacts: if names, emails or account numbers were replaced before delivery, confirm the replacement tokens are consistent and that the format check does not reject them.

Stage 5: Key uniqueness and referential integrity across tables

Referential integrity is the check most often skipped and most often broken in CSV exports. Each table's primary key should be unique and non-null; composite keys (for example order_id plus line_no) need uniqueness on the combination. Then every foreign key should resolve: each order_line.order_id exists in orders, each ticket.account_id exists in accounts.

Orphans typically come from extracts filtered on different date windows per table, from soft-deleted parent rows, or from a sampling job that subset child tables without following keys. Report the orphan rate per relationship rather than a pass/fail, and inspect whether orphans concentrate at window boundaries. When the delivery spans several source systems joined on a shared identifier, extend this into record linkage verification; when a subset was drawn, check it against sampling relational extracts without breaking keys.

A validation suite you can adapt

The table below is a starting rule set; tools such as Great Expectations, pandera, Soda or plain SQL against DuckDB all implement these checks.

Illustrative example: invented to show structure; it does not describe an available dataset.

Pair the suite with a machine-readable report returned to the supplier. A minimal record per failed check:

{
  "delivery_id": "dlv-2026-10-batch-07",
  "check_id": 10,
  "table": "order_lines",
  "column": "order_id",
  "rule": "fk:orders.order_id",
  "rows_evaluated": 412880,
  "rows_failed": 913,
  "failure_rate": 0.0022,
  "severity": "warning",
  "sample_keys": ["OL-88121", "OL-90457"],
  "suite_version": "3.1.0"
}

Version the suite alongside the data dictionary so a later dispute can be replayed against the exact rules that ran. If a delivery passes structurally but you still need a statistical accept/reject decision on content defects, layer acceptance sampling on top.

Where these checks sit in governance and contracts

Arrival validation is evidence, not just plumbing. For high-risk systems under the EU AI Act, Article 10 requires training, validation and testing data to meet quality criteria and be subject to data governance practices [9]; as of October 2026 the high-risk application dates have reportedly moved under Regulation (EU) 2026/1744, which also amends Article 10 [10]. A versioned suite and stored reports are the simplest way to show what was checked and when; see EU AI Act Article 10 data governance for the wider picture.

Put the dictionary, the format choice and the blocking rules in the delivery specification before the first export, so failures are a defined remedy rather than a negotiation. The file formats AI buyers accept page and the quality cluster hub cover the surrounding decisions.

When you source structured operational data through SourceX, each dataset is rights-reviewed and delivered under a license that defines records, uses, term and delivery, and personal details are removed or replaced before delivery, with the method recorded and a sample checked. No de-identification method is perfect, so keep your own checks. You can describe the tables you need on the SourceX buyers page.

Sourcing structured exports for validation-ready training data

SourceX sources operational datasets, such as support and sales histories, engineering records and finance workflows, from US companies on request, and every release is approved by the supplying company. Nothing is held in stock, and a request does not guarantee a match. Describe the schema, fields and coverage you need at sourcex.si/buyers.