Tabular, Time-Series and Transactional Data for AI: A Buyer's Guide

This hub covers the structured data cluster of the SourceX guide to AI data. For definitions, read structured vs unstructured AI training data.

Eight families of structured data and the models they train

Scope structured data for AI training by family, because each family comes from different systems, trains different models and fails acceptance for different reasons. Settle the last column before discussing volume.

Several record types have their own SourceX pages: spreadsheets and financial models, financial transaction data, sensor and IoT data, sales CRM histories and accounting reconciliations. Licensing operational data for AI covers the wider category.

Why real enterprise tables are hard to find

Real multi-table business data rarely reaches the open market because it sits inside ERP, billing and payment systems under privacy, confidentiality and commercial constraints.

• Scarcity. When SAP released SALT, anonymized sales tables from one customer's ERP system, it said text is plentiful but data with multiple linked tables is scarce, because privacy, confidentiality and commercial interests make company datasets hard to procure [1].
• Tabular foundation models. TabDPT's authors report that existing TFMs are predominantly trained on synthetic data, and that adding real data to pretraining can give significantly faster training and better generalization [2]. Real-TabPFN reports that continued pretraining on a small, curated set of large real datasets beat broader, potentially noisier corpora such as CommonCrawl or GitTables [3].
• Time series. Google Research describes TimesFM as pretrained on 100 billion real-world time points [4], so fair evaluation needs held-out business time series outside public corpora.

Specify grain, keys and history before row counts

A structured data request should define what one row means, how tables join, how history is kept and when each label became known; row counts mean little until those are fixed. The tabular license specification guide adds clause-level detail.

• Grain. "One row per invoice line" and "one row per invoice" train different models; mixed grains double-count amounts.
• Keys. Name primary and foreign keys and agree a maximum orphan-row rate. Subsets must keep whole entity graphs (sampling relational extracts without breaking keys).
• History. Ask for change history or dated snapshots, with both business event time and the time each row was written; current-state tables cannot support point-in-time correct training data.
• Labels. Record each label's source and when it became known: a dispute flag set after settlement did not exist when the payment was scored.
• Metadata. Spider 2.0 tasks require database metadata and dialect documentation [7]; ask for column descriptions, code lists, units and metric definitions (data dictionaries as grounding data).
• Coverage. State entities and time span: ten million rows from 200 customers is a narrow sample.

Acceptance thresholds can borrow the data quality model and measures of ISO/IEC 5259-2:2024 [13]; validation checks for structured deliveries turns them into tests. The manifest below applies these points to a four-table extract.

Illustrative example: invented to show structure; it does not describe an available dataset.

dataset: order_to_cash_extract_example
history: change_log                 # every update kept with valid_from / valid_to
time_zone: UTC
coverage: {from: 2021-01-01, to: 2025-12-31}
tables:
  customers:
    grain: one row per customer version
    primary_key: [customer_token, valid_from]
    customer_token: {method: keyed_hash, key_held_by: supplier, same_key_all_tables: true}
    postal_code: {method: generalize, keep_digits: 3}
  orders:
    grain: one row per order
    primary_key: [order_id]
    foreign_keys: {customer_token: customers.customer_token}   # as-of join on valid_from / valid_to
    event_time: order_created_at
    written_at: row_inserted_at
  order_lines:
    grain: one row per order line
    primary_key: [order_id, line_no]
    foreign_keys: {order_id: orders.order_id}
  payments:
    grain: one row per payment attempt
    primary_key: [payment_attempt_id]
    foreign_keys: {order_id: orders.order_id}
    label: {column: disputed, source: processor_dispute_file, known_at: dispute_opened_at}
    free_text_columns: {payment_memo: pii_scanned_and_masked}
acceptance:
  duplicate_primary_keys: 0
  orphan_foreign_keys_max_pct: 0.1
license:
  license_id: LIC-EXAMPLE-01
  permitted_uses: [model_training, internal_evaluation]
  derived_artifacts: [features, embeddings]   # name them explicitly

• known_at lets you drop any label or field that did not exist at prediction time.
• One keyed hash applied across all tables pseudonymizes customers without breaking joins.
• event_time and written_at separate when something happened from when the system recorded it.

Leakage: the defect that makes a weak table look strong

Leakage means a training table holds information that would not exist at prediction time. In licensed business data it usually enters through post-outcome fields, random splits over time-ordered rows, or the same entity on both sides of a split.

• Post-outcome fields. A write_off_reason in a credit table or resolved_by in a ticket table is filled only after the outcome. Ask which columns are written after the event you predict.
• Time. Split chronologically; for time-series validation, one library recommends a gap between training and test equal to the forecast horizon [14]. RelBench uses time-based splits so models cannot use future data to predict earlier events [5].
• Entities and preprocessing. The same customer on both sides inflates scores, and encoders fitted on the full dataset before splitting leak statistics; fit preprocessing on training folds only [15].

Write timestamps make these checks possible, so history belongs in the license (target leakage audit).

De-identifying tables without breaking joins

Removing names and account numbers does not de-identify a table, because combinations of ordinary columns single people out, and inconsistent key replacement destroys the joins that give relational data its value. Treat direct identifiers, quasi-identifiers (ZIP code, birth date, job title) and free-text columns as separate problems.

• Quasi-identifiers. Rocher and colleagues estimated with a generative model that 99.98% of Americans would be correctly re-identified in any dataset using 15 demographic attributes; it is a model estimate, not a count of actual re-identifications [16]. Generalize dates and locations, then test residual risk, for example with k-anonymity.
• Keys. Use one keyed pseudonymization scheme across all tables, with the key held by the supplier, so joins survive and tokens cannot be recomputed from known identifiers.
• Free text. Memo and description columns carry identifiers that column rules miss.
• Alternatives. NIST SP 800-188 cautions about the limits of traditional de-identification compared with formal privacy methods, and describes synthetic data and protected enclaves as other sharing models [17].

As of October 2026, US rules attach to particular families:

• Health tables. HIPAA de-identification uses Safe Harbor, which removes 18 listed identifiers, or Expert Determination [18]. SourceX requires one of these methods for health records before anything is considered for a license.
• Consumer records. California Civil Code section 1798.140(m) treats information as deidentified only if the business takes reasonable measures, publicly commits not to re-identify it and contractually obligates recipients to comply [19], so expect no-re-identification terms.
• Financial transactions. Under Regulation P (12 CFR 1016.11), a recipient of nonpublic personal information from a nonaffiliated financial institution faces reuse and redisclosure limits even if it is not itself a financial institution [20].

For datasets sourced through SourceX, personal details such as names and account numbers are removed or replaced before delivery, the method is recorded per dataset, and a sample is checked after processing. No de-identification method is perfect. Techniques are compared in de-identifying tabular and transactional data and the de-identified data guide.

This page is general information, not legal advice. Confirm requirements with counsel for your jurisdiction and use case.

Public, synthetic or licensed: choosing a route per use case

Public benchmarks suit reproducing baselines, synthetic tables suit pipeline tests and privacy-constrained prototypes, and licensed real records are needed when a model must learn real distributions, dirty values and cross-table behavior.

Synthetic tables are scored against real data: the SDMetrics library measures statistical differences, machine-learning efficacy and privacy [21]. See evaluating synthetic tabular data and combining licensed and synthetic data.

SourceX works the licensed route: it sources operational datasets from US companies, including sales histories and finance workflows, and manages the licensing agreement and later purchases. Datasets are sourced on request, not held in stock, so a request does not guarantee a match. You can send SourceX a structured data specification.

Delivery formats that keep types, keys and metadata intact

Deliver structured data as typed columnar files or a governed warehouse share with the data dictionary alongside, not as CSV dumps that lose types and keys.

• Columnar files. Apache Parquet writes file metadata in a footer that records where each column chunk starts, so readers fetch only the columns they need [22]. Open table formats add snapshots on top (Iceberg and Delta Lake delivery).
• Warehouse shares. In Snowflake Secure Data Sharing no data is copied between accounts and shared objects are read-only for the consumer [23] (zero-copy warehouse data sharing).
• Metadata. Croissant describes a dataset in JSON-LD built on schema.org, covering dataset metadata, file resources and record-level structure [24]. Transfer channels for every data type are in the dataset delivery guide.

Start here: structured data guides by family

Read the guide that matches your family and model goal.

• Tables and relational databases: real-world tables for tabular foundation models, multi-table data for relational deep learning, how much tabular data you need.
• SQL, schemas and spreadsheets: text-to-SQL training data, private text-to-SQL evaluation sets, spreadsheet benchmarks from real workbooks, messy multi-table spreadsheets.
• Transactions and ERP: ERP transaction and master data, fraud-labeled transactions, journal entries for anomaly detection.
• Time series, logs and events: time-series foundation model data, order histories for demand forecasting, production logs for anomaly detection, labeled equipment failure data for predictive maintenance.
• Master data and graphs: entity resolution data, product attribute data, enterprise knowledge graph data.

Mistakes that sink structured data purchases

Beyond leakage and broken keys, five errors routinely leave purchased structured data unusable or under-licensed.

• Joining systems without a crosswalk. CRM, billing and ERP rarely share a customer ID; ask for the mapping table and its match method.
• Ignoring currencies, units and time zones. Mixed currencies and local timestamps across daylight-saving changes corrupt amounts and sequences.
• Keeping test and demo records. Business systems hold test transactions and demo accounts; ask the supplier to flag or remove them.
• Evaluating on public benchmark data. It may already sit in a model's training mix; keep a private held-out set (LLM evaluation datasets).
• Leaving derived artifacts out of the license. Features, embeddings, fine-tuned weights and synthetic tables made from licensed data need explicit terms (AI training data licensing).

Describe the tables, series or transactions your model needs

At the SourceX buyer page you can submit a data request, talk to SourceX or browse the dataset catalog. Describe the data family, grain, history, labels and permitted uses you need; SourceX looks for US companies that hold matching records, checks the data and the supplier's licensing permissions, and manages the license and delivery. Nothing is contracted until a supplier agrees.

Describe your structured data needs to SourceX