ERP Transaction and Master Data for AI Training

Why public ERP datasets rarely cover a production use case

Public ERP data exists, but it is narrow, simulated or single-company. SAP framed SALT as a research release precisely because large, cleansed company datasets are hard to procure [1]. The BPI Challenge 2019 log covers purchase-to-pay at one paints and coatings group [5], and the Zenodo OCEL 2.0 logs for order-to-cash and procure-to-pay are simulated [6], using the object-centric OCEL 2.0 format that links each event to several business objects [7].

These are good for prototyping process-mining features or smoke-testing a pipeline. They fall short when a model must generalize across company codes, chart-of-accounts designs, custom Z-fields, pricing procedures and industry-specific document types. Academic work on S/4HANA process data reports classification accuracy around 90% with room to grow as more data becomes available [4]. That points to a data-volume and data-diversity problem, which history from multiple real companies helps address.

Which ERP document flows to request

The useful unit is an end-to-end chain with stable foreign keys, so a model can learn that a delivery belongs to an order and an invoice clears against a goods receipt. Specify the chains you need, not "all ERP data."

• Order-to-cash: sales order header and items (SAP VBAK/VBAP), deliveries (LIKP/LIPS), billing (VBRK/VBRP), the document-flow table (VBFA), and accounting and clearing entries. In NetSuite terms, sales orders, item fulfillments, invoices and customer payments linked through transaction and transaction-line records.
• Procure-to-pay: purchase requisitions with their release status and approvals, which follow the configured release strategy [8], purchase orders (EKKO/EKPO), PO history (EKBE), goods receipts (material documents, MATDOC in S/4HANA), and supplier invoices (RBKP/RSEG). Ask whether the flows include 3-way match, 2-way match and consignment cases, as BPI 2019 does [5].
• Record to report: journal lines (BSEG, or ACDOCA in S/4HANA) when the use case touches posting or reconciliation. Journal-entry modelling has its own page on journal entry data for anomaly detection.
• Change history: change documents (CDHDR/CDPOS) give old and new values per field, which is the closest thing to labels for "what a user corrected."

For agent tasks that act on these flows, see procure-to-pay and order-to-cash records for finance and operations agents. For PO-only needs, the owner page is license purchase orders for AI training.

Master data is half of the dataset

Transaction tables without master data are hard to learn from, because the predictive signal often sits in the customer, vendor or material record. Field recommendation in particular depends on it: SAP's own Data Attribute Recommendation tutorials train on uploaded historical records with a defined object schema [3], and SALT targets sales-document fields that are predicted from linked customer and header context [1].

Request customer and vendor masters (KNA1/LFA1, or business partner tables in S/4HANA), material masters (MARA and plant-level views), pricing conditions and payment terms. Require the supplier to keep keys joinable after de-identification, so a pseudonymized vendor ID still matches across EKKO, RBKP and LFA1. If you are building matching or deduplication models, the related page is entity resolution data from real master data.

Schema realities that break ERP models

Real ERP tables differ from vendor documentation, and those differences are what your model must handle. Expect multiple company codes and currencies, units of measure with conversion factors, fiscal year variants, custom fields and tables, reversed and cancelled documents, and archived years that sit outside the live database.

Common failure modes to test for:

• Silent leakage: a status field such as a billing block or clearing date is populated after the event you are predicting. Ask for change-document timestamps so you can rebuild point-in-time snapshots.
• Orphaned keys: extraction by date window cuts flows in half, leaving invoices without orders. Specify extraction by document flow, not by table and date.
• Undocumented customization: Z-fields and custom document types with no definitions. Require a data dictionary; see data dictionaries and schema documentation for AI.
• Authorization gaps: extracts pulled under a restricted user silently miss plants or company codes. Vendors stress that extraction should run within SAP authorizations [2], so ask which scope the extract user had.

Text-to-SQL work over these schemas is harder than public benchmarks suggest: Spider 2.0 uses 632 problems over real application databases with large schemas [9]. If that is your use case, read text-to-SQL training data from real enterprise schemas.

ERP data request template

A precise request lets a supplier judge quickly whether they hold the data, and gives your counsel a clear scope for the license. Adapt the template below; general grain and key guidance is on what to specify when licensing tabular data.

Illustrative example: invented to show structure; it does not describe an available dataset.

Privacy, confidentiality and rights in ERP extracts

ERP extracts carry personal data and commercial secrets together, so de-identification and licensing need equal attention. Customer and vendor masters hold names, addresses, contact persons, tax IDs and bank details; sole-proprietor vendors make even "business" records personal data. Pricing conditions, margins and supplier terms are confidential to the company and sometimes to its counterparties under NDAs.

Ask suppliers how identifiers were replaced, whether replacement is consistent across tables, and how free-text fields (header texts, item notes, invoice reference lines) were handled, since these leak names most often. Also confirm the supplier has the right to release the data: ERP vendor agreements, customer contracts and data processing agreements can each restrict reuse. For the supplier-side view of that question, see can I license my ERP data to AI companies.

How SourceX sources ERP data for buyers

SourceX sources operational datasets from US companies on request and manages the commercial process, including the licensing agreement and ongoing purchases. Data is not held in stock, so a request describes the data you need, and SourceX looks for US businesses that hold it; a request does not guarantee a match. The process runs Find, Assess (data and licensing permissions), Agree (pricing and allowed uses in a license), Transact and Manage, and nothing is contracted until a supplier agrees.

Every dataset is rights-reviewed for ownership and consents and delivered under a license that defines records, uses, term and delivery. Personal details such as names, emails, phones and account numbers are removed or replaced before delivery, the method is recorded and a sample is checked, though no method is perfect. Related finance work is covered on finance operations AI, accounting reconciliations and supply chain and logistics datasets. You can describe your ERP data requirement to SourceX at any stage of scoping.

What to check before you sign for ERP data

Before signing, confirm coverage, joinability and documentation against your evaluation plan. Run a sample through your own pipeline and check:

• Share of invoices that join back to an order or PO, and share of payments that clear an invoice.
• Distribution of document types, company codes, currencies and years against your target customers.
• Whether change documents allow point-in-time reconstruction for every label you plan to predict.
• Whether held-out companies or years can be reserved for evaluation, separate from training.

More structured-data guidance is in the tabular, time-series and transactional data buyer's guide, forecasting-specific needs are on sales and order histories for demand forecasting, and the full buying process is on AI training data procurement.

Request ERP transaction and master data for AI training

SourceX sources operational ERP datasets from US companies on request, with every release approved by the supplying company and delivered under a license that defines records, uses, term and delivery. Describe the flows, master data and history you need, and SourceX will look for businesses that hold them. Start an ERP data request.