Skip to content

Privacy and preparation

Workflow dataset reproducible build: prove it from approved exports

By SourceX Editorial · Updated

9 sections

Short answer

To prove a workflow dataset can be rebuilt from the approved exports, freeze those exports with content hashes, run a versioned transformation recipe in a pinned environment with fixed parameters, and record inputs, recipe version, stage counts and output hashes in a reproducibility manifest. A second person then reruns it, gets matching hashes and counts, and reconciles episode totals to the source system's own reports.

Provenance trail

Workflow dataset

This page covers workflow dataset reproducible build: prove it from approved exports.Illustrative

Key takeaways

  • A rebuild only proves something when it starts from stored, hashed exports, never from a fresh query against the live system.
  • The transformation recipe must declare case identity, clock mapping, ordering, duplicate handling, concurrency and episode boundaries as explicit rules.
  • A reproducibility manifest pins inputs, code, environment and parameters, and records stage counts, source reconciliation and output hashes.
  • Unsorted queries, naive timestamps, relative date filters and unseeded sampling are the usual reasons a rebuild differs.
  • Deletions and corrections should be applied as recorded recipe steps so the new build differs from the old one by exactly the intended records.

How do you prove a workflow dataset can be rebuilt from the approved exports?

You prove it by rebuilding it. Freeze the approved source exports and record their content hashes, run a versioned transformation recipe in a pinned environment, and write a reproducibility manifest that lists every input hash, the recipe version, the parameters and the output hashes. A second person then reruns the recipe from the same exports and gets the same outputs, and the episode counts reconcile back to the source system's own totals.

The software world has a precise version of this test. The Reproducible Builds project defines a build as reproducible when, given the same source code, build environment and build instructions, any party can recreate bit-by-bit identical copies of the specified artifacts, verified by comparing cryptographic hashes. A workflow dataset is a build whose source is the approved exports and whose instructions are the transformation recipe.

This matters for a supplier because a licensed dataset is a promise about what was delivered and where it came from. If the dataset cannot be rebuilt, a correction, a deletion request or a buyer's question about one episode turns into an investigation instead of a rerun. This page gives the recipe structure, the manifest schema, a validator and a worked example with one passing and one failing rebuild.

Freezing the approved exports so the inputs cannot drift

A rebuild is only meaningful if the inputs are byte-for-byte the files that were approved. Store each export once, in write-once or access-controlled storage, and record its hash, row count, export time and the query or report that produced it. Never rebuild from a live query against the source system; the system will have changed.

Existing tools already work this way. The Frictionless Data Package standard gives each data resource a hash property, MD5 by default or another algorithm named as a prefix. DVC keeps a dvc.lock file that lists each stage's dependencies and outputs with a content hash field, and its repro command regenerates pipeline results by restoring the dependency graph and running only the stages whose inputs changed. Either pattern, or a plain manifest with SHA-256 values, gives the rebuild a fixed starting point.

Record approval with the hashes, not with file names. A file called tickets_final.csv tells a reviewer nothing; a hash listed next to the approver, date and scope decision shows which bytes were cleared for licensing.

Incremental exports need one more rule. When a system can only be exported in monthly slices, or a later top-up adds new months, treat each slice as its own approved input with its own hash and row count, and list all of them in the manifest. Never overwrite an earlier slice with a corrected one; store the correction as a new file and record which slice it replaces, so an older build can still be rerun exactly as it was delivered.

The transformation recipe: from raw events to task episodes

The recipe is the ordered list of rules that turns exported events into episodes, each written so that a second engineer would make the same choice. Process mining tools need the same core: pm4py's format_dataframe function, for example, takes a case identifier column, an activity column and a timestamp column. The recipe has to say how each of those is derived and what happens at the edges.

The transformation recipe: from raw events to task episodes
Recipe stepRule to declareDeterminism requirementCheck after the step
Case identityWhich ID defines an episode and how merged or split records mapMapping table is an input file, not logic inside a queryEvery event has exactly one case ID
Clock mappingSource time zone per system and conversion to UTCNo naive timestamps after this stepMin and max time per source within the export window
Activity and state namesMapping from raw status codes to named activitiesMapping table versioned with the recipeNo unmapped codes; counts per activity logged
Event orderingSort keys including a tie-breaker for equal timestampsExplicit sort on every ordered outputTimestamps never decrease within a case
Retries and duplicatesWhen repeated events are collapsed and which copy is keptRule depends only on event contentDuplicate count logged per source
Concurrent workHow parallel tasks inside one case are representedSame representation for every caseOverlapping intervals flagged, not dropped silently
Episode boundariesStart event, terminal states and handling of still-open casesCut-off is a fixed date in the parameters, not today's dateEach closed episode has one terminal state
Redaction and de-identificationFields removed, masked or replaced and the method usedSeeded or keyed methods with keys held outside the datasetScan results stored with the run

The reproducibility manifest: fields and what each proves

The manifest is a small file that travels with every build. It holds enough to rerun the build and to check that the rerun matched. Example values below are illustrative.

At a glance

The reproducibility manifest: fields and what each proves

The reproducibility manifest: fields and what each proves
FieldPurposeIllustrative valueValidator
dataset_id and build_idName the dataset and this specific buildsupport-episodes, build 2026-10-18.1Unique per build
source_exportsList each approved export with system, query, export time, rows and hashhelpdesk_tickets.csv, 412,000 rows, sha256 valueEvery listed hash matches the stored file
approval_refLink the inputs to the rights and scope decisionScope decision record ID and datePresent for every source export
recipe_versionPin the transformation code and mapping tablesGit commit of the recipe repositoryCommit exists and the working tree was clean
environmentPin interpreter, libraries and container imageLock file hash and image digestRebuild uses the same digests
parametersRecord cut-off date, time zones, seeds and thresholdscutoff 2026-09-30, seed 20261018No parameter depends on the run date
stage_countsRows in and out of every recipe stepevents 1.9 million, episodes 380,000Counts identical on rebuild
source_reconciliationCompare episode totals with the source system's own reportClosed tickets in period versus closed episodesDifference within the declared tolerance, each gap explained
output_filesList each output with rows and hashepisodes.parquet with sha256 valueHashes match, or canonical comparison passes
known_limitsState gaps and reconstruction confidenceTwo weeks of missing status history in MarchField is never empty

Where nondeterminism hides in a rebuild

Most failed rebuilds are not caused by bad data but by steps that were never deterministic. The PostgreSQL documentation states that if no sort is chosen, rows are returned in an unspecified order that depends on the plan and the order on disk and must not be relied on. Python's documentation notes that a naive datetime does not carry enough information to locate itself relative to other times, so whether it is UTC or local time is up to the program.

Check the recipe for each of these before the first build:

  • Queries or dataframe operations without an explicit sort, especially before taking the first or last event in a case.
  • Naive timestamps from systems in different time zones, and daylight saving transitions handled differently by two libraries.
  • Relative filters such as the last 90 days, which change meaning every time the recipe runs.
  • Random sampling, shuffling or train and test splits without a recorded seed.
  • Iteration over unordered collections, such as sets or dictionary keys built from external input.
  • Floating point aggregation whose order changes with parallelism.
  • Library or model upgrades, including language detection or redaction models, that change outputs without changing code.
  • Hand edits to mapping tables or outputs made outside the recipe.

Could clock drift, an ID mismatch or a missing transition change a case?

Yes, and the manifest should show where. Clock differences between systems can reorder events in a case, for example putting a resolution before the escalation it answered. Declare each source's time zone and any known offset, and flag cases where cross-system events fall within that offset of each other instead of trusting the order.

Identifier mismatches split or merge cases. A ticket reopened under a new number, or an order line that changes ID after an amendment, will look like two short episodes unless the mapping table joins them. Count episodes that start without a creation event or end without a terminal state; a jump in either count after a recipe change points to an identity problem.

Unrecorded transitions are the hardest. If a system only logs some status changes, the reconstructed path shows jumps the real process never made. Record the share of cases with a gap of this kind as the reconstruction confidence in known_limits, and reconcile episode totals to the source system's own report so a missing block of events cannot hide.

Worked example: one rebuild passes, one fails

This example is illustrative; the systems are generic and the counts are invented to show the method. A software company prepares support episodes from a helpdesk export and an engineering issue tracker export. Build 1 produces 380,000 closed episodes. The manifest lists both export hashes, the recipe commit, the container image digest, a fixed cut-off date and a seed for the sampled evaluation split.

Passing rebuild: a second engineer pulls the two stored exports, confirms their hashes, checks out the recipe commit and runs it in the recorded image. Stage counts match at every step, the output hashes match, and closed episodes reconcile to the helpdesk's closed-ticket report for the period within the declared tolerance, with the difference explained by test tickets the recipe excludes by rule.

Failing rebuild: months later, a buyer asks for a corrected build after a deletion request. The engineer re-pulls the helpdesk data from the live system instead of the stored export, and the recipe's filter still says last 24 months. The source hash no longer matches, the episode count drops, and a timezone setting changed in the new environment shifts a few thousand events across midnight. The validator stops the build at its first check, the stored export is restored, the deletion is applied as a recorded step with its own count, and the rebuild then differs from build 1 by exactly the deleted episodes.

The validator to run on every build

Automate these checks so a build cannot be published without them. Each check writes its result into the manifest, and a failed check stops the run rather than producing a warning.

At a glance

The validator to run on every build

  • Every source export hash in the manifest matches the stored file.
  • The recipe commit exists, the working tree was clean and mapping tables are included in that commit.
  • The environment digest matches the one recorded, or the difference is a new build with a new build_id.
  • No parameter is derived from the current date.
  • Stage counts are recorded for every step and match on rebuild.
  • Within each episode, timestamps never decrease and exactly one terminal state exists for closed cases.
  • Episode totals reconcile to the source system's report within the declared tolerance, with each gap explained.
  • Output hashes match; where formats embed write times, a canonical comparison of sorted rows is run instead and recorded.
  • A small random sample of episodes is printed side by side with the raw events for a human spot check.

Rebuildable datasets in a licensing workflow

A reproducible build is also good documentation. The Datasheets for Datasets questions ask whether raw data was saved alongside preprocessed data and whether the preprocessing software is available, and a manifest answers both for a licensee.

Companies that license selected records through SourceX prepare and deliver the dataset together with it, and the company approves the scope and terms and signs before any record leaves. Because SourceX does not host very large datasets, they can remain in the company's storage or travel on encrypted drives, which suits a build that is rerun where the approved exports live.

Frequently asked questions

Does a reproducible dataset need bit-identical output?

Bit-identical output is the strongest proof and is achievable for most tabular builds once sorting, time zones and seeds are fixed. Where a file format embeds write times or compression differs, compare a canonical form, such as rows sorted by key, and record that the comparison method changed.

Can we rebuild after a deletion request?

Yes, and the rebuild should show it. Apply the deletion as a recorded recipe step against the stored exports, log the number of affected episodes, and confirm the new build differs from the previous one by exactly those episodes.

Is DVC required?

No. DVC, a data package with hashes, or a plain manifest file with SHA-256 values all work. What matters is that inputs, code, environment, parameters and outputs are pinned and checked on every build.

How long should we keep the approved exports?

For as long as you may need to rebuild, correct or answer questions about the dataset, subject to your retention schedule, legal holds and any deletion obligations. Record the retention decision in the manifest so a licensee knows how long rebuilds remain possible.

What tolerance is acceptable when reconciling to source totals?

There is no universal figure. Set it per dataset based on known exclusions, such as test records or merged duplicates, and require each difference to be explained rather than accepted as noise.

Sources

  • A build is reproducible when any party can recreate bit-by-bit identical copies of the specified artifacts from the same source, environment and instructions, verified by comparing hashes. Source
  • The Frictionless Data Resource standard gives each resource a hash property, MD5 by default, with other algorithms named by prefix. Source
  • DVC's dvc.lock file records each stage's dependencies and outputs with a content hash, letting DVC detect when stages must be reproduced. Source
  • dvc repro regenerates data pipeline results by restoring the dependency graph and running the needed stages in order. Source
  • pm4py's format_dataframe prepares an event log from a case identifier column, an activity column and a timestamp column. Source
  • PostgreSQL returns rows in an unspecified order when no sort is chosen, and that order must not be relied on. Source
  • A naive Python datetime does not contain enough information to locate itself relative to other date and time objects. Source
  • Datasheets for Datasets asks whether the raw data was saved in addition to the preprocessed data and whether the preprocessing software is available. Source

See if your company qualifies

A short company assessment. No data uploads are needed.

See if your company qualifies