Skip to content

Privacy and preparation

Dataset incident scope mapping: start from the run, not the record

By SourceX Editorial · Reviewed by Noah Loul ·

10 sections

Short answer

Dataset incident scope mapping for a processing fault starts from the run, not the record: find every run that used the faulty step between its introduction and its fix, then every version, sample and intermediate copy those runs produced, then every recipient. Release notes will not supply the start: in TensorFlow Datasets, 2 of 32 data-changing fix notes say where the fault began and 4 say how many records it touched.

Record timeline

Dataset incident scope

This page covers dataset incident scope mapping: start from the run, not the record.Illustrative

Key takeaways

  • A processing fault lives in a step, so its scope is every record the step touched between the faulty change and the fix, not the records someone reported.
  • Build the impact map in four layers, runs, versions, copies and recipients, and widen the scope wherever a layer has no record.
  • Of 51 fix notes in TensorFlow Datasets release notes, 32 change data users read; 2 of those name where the fault began, 4 quantify it and 26 do neither.
  • When the start is undocumented, test retained versions for the fault's signature, newest first, and record what each cleared version was tested for.
  • Processor-held extracts and recipients' existing copies sit outside any release manifest, so they need their own register and a status per entry.

What does dataset incident scope mapping have to establish?

Dataset incident scope mapping has to establish four things for a processing incident: which runs used the faulty step, which dataset versions those runs produced, which samples, splits and intermediate copies were cut from them, and who received each copy and what they did with it. Until those four lists exist, nobody can contain the incident, decide whether a notice is due or plan a repair.

Regulators ask for the output of that map in plain numbers. In the UK, the Information Commissioner's Office guide to personal data breaches says a notification should describe the nature of the breach, including the categories and approximate number of individuals and of personal data records concerned, and that every breach must be recorded with its facts, effects and remedial action whether or not it is reported; notifiable breaches go to the ICO within 72 hours of becoming aware. The ICO also says the guide is under review following the Data (Use and Access) Act.

In the US, the HHS Breach Notification Rule requires a HIPAA business associate to notify the covered entity without unreasonable delay and no later than 60 days from discovery, identifying, where possible, each individual affected. A business associate that cannot say which records went where cannot meet that request. This is general information about UK and US rules, not legal advice; whether any notice is required is for counsel.

Why a processing fault is scoped by its run window, not by reported records

A processing fault is scoped by its run window because the defect sits in a step, not in a record: every record that passed through the step between the faulty change and the fix is suspect, including records nobody has reported. The first complaint is a symptom, not the boundary.

Typical processing incidents look like this. A configuration push turns off one redaction rule on one export path. A de-identification job is skipped when a retry runs from an older script. A processor's staging folder is readable by the wrong group for a month. A pseudonym table is reused across two datasets that should never link. In each case the question is not which record is wrong but which runs touched what while the fault was live.

That makes the map different from tracing one bad source record forward, which the page on dataset issue record lineage covers. A record trace starts from a key and follows it through derivations. A run-window trace starts from two dates, the first run with the fault and the first run with the fix, and pulls in everything those runs wrote. The end date is usually easy to find, because someone shipped the fix. The start date is the hard one, and public practice shows why.

What 51 fix notes in TensorFlow Datasets record about processing faults

Public dataset release notes record when a processing fault was fixed, rarely when it began and almost never how many records it touched. This article read every release-notes mapping in the TensorFlow Datasets repository at commit 24c0f20 (26 October 2026), a public dataset collection with a written versioning policy, and coded each note that describes a fix.

The repository holds 210 release-notes mappings with 351 version entries, and 79 mappings list two or more versions. Fifty-one entries, in 43 mappings, use a fix word such as fix, bug, corrupt, wrong or missing. Read by hand, 32 of those 51 describe a change to what users read: labels, values, alignment, identifiers or images. The rest change storage or naming only (10), concern download sources (5), withdraw a subset for licence reasons (1), fix an unnamed bug (1) or use the word for something else (2).

Of the 32 data-changing fix notes, two name the version where the fault began: one says labels were inverted in an earlier numbered release, another says labels were wrong in V1. Four quantify the damage, with figures such as about 12 images, 19 examples, or roughly 14 and 25 percent of the examples in the affected splits. The remaining 26 say neither. A reader holding an older copy learns that a fix exists and must work out alone whether their version carried the fault.

The project's own versioning guide explains the gap. It describes its version semantics as best effort and warns that "there might be un-noticed bugs impacting a dataset while the version was not incremented." It also says that only the latest version can be generated while all previously generated datasets can still be read. A fault can live across versions without a version change, and copies made before the fix stay usable after it.

What 51 fix notes in TensorFlow Datasets record about processing faults
What the 51 fix notes describeEntriesName where the fault beganQuantify affected records
Change to labels, values, alignment, ids or images3224
Storage type, encoding, serialization or a field name only10Not codedNot coded
Download source, checksum or archive5Not codedNot coded
Subset withdrawn for licence reasons1Not codedNot coded
Bug fixed, change not described1Not codedNot coded
Fix word used for something else2Not codedNot coded

The dataset impact map: four layers and the record behind each

A dataset impact map has four layers, runs, versions, copies and recipients, and each layer is only as good as the record that settles it. The table below names the record, the gap that forces the scope wider and the person who owns the layer. It is an editorial tool drawn up for this article, not a regulator's template.

Build the layers in order. Runs come first because they carry the step version and configuration that define the fault; versions inherit the runs' status; copies inherit the versions'; recipients inherit the copies'. When a layer has no record, the honest move is to widen the scope to everything that layer could contain and narrow it only as evidence arrives.

The dataset impact map: four layers and the record behind each
LayerQuestion it answersRecord that settles itGap that widens the scopeOwner
Processing runsWhich runs executed the faulty step or configuration, and when?Run log with step version, configuration hash, input and output identifiersRuns logged without step version: every run of that pipeline is in scopePipeline owner
Dataset versionsWhich released versions were built by those runs?Release manifest linking each version to its run IDs and file hashesVersion labels reused or not tied to runs: every version since the step changedDataset owner
Derived copiesWhich samples, splits, evaluation sets and extracts came from those versions?Copy register with source version and record keys per copyCopies made by hand or outside the pipeline: search shared drives and ticketsDataset owner
Intermediate copiesWhich processors or staging areas held pre-processing data during the window?Processor list, staging retention log, deletion confirmationsUnknown staging retention: assume the copy still existsSecurity owner
Recipients and useWho received each affected copy, and did they store, evaluate or train on it?Delivery and access log keyed by version, plus recipient confirmationsShared credentials or unlogged reads: every holder of access is in scopeContract owner
An illustrative impact map in four lanes, runs, versions, copies and recipients, across builds 2.2 to 2.7, with a shaded fault window over builds 2.3 to 2.6 whose builds link to a sample, an evaluation split and a staging extract and on to two buyers, reviewers and a processor, above a line noting that 2 of 32 data-changing fix notes in TensorFlow Datasets name where the fault began.
Illustrative: an impact map for one disabled redaction rule; builds, copies and recipients are invented. The footer count is from TensorFlow Datasets release notes at commit 24c0f20.

Where does the fault window start when nobody wrote it down?

When no note records the start, find it by testing outputs, not by reading history: rerun a detector for the fault's signature over each retained version, and take the earliest version that fails as the provisional start. The fault's own definition tells you what to test for, such as phone-shaped strings in one field or a missing pseudonymization pass.

Search in order of cost. First, read the configuration and code history for the step and note every change that could have introduced the fault. Second, check run logs for the step version each run used. Third, run the detector on stored versions, newest first, until one passes; then check one version earlier to confirm the boundary. A version that passes the detector is cleared for that signature only, which the incident record should state.

If older versions were deleted or never retained, the start cannot be proven. Set the window at the earliest version the faulty code could have produced and say so in the record. A conservative start widens the copies and recipients layers, and that cost is the reason to keep run logs with step versions from the first delivery.

Recipients and intermediate copies: the layer release notes cannot hold

The recipients layer lists every party that held an affected copy, including processors and internal reviewers, with the version each received and what they did with it. Public release notes cannot carry this layer at all: a public dataset is downloaded by people the publisher never sees, while a licensed dataset goes to named parties, so the list can exist if someone keeps it.

Intermediate copies are the part most often missed. A processing incident can sit upstream of de-identification, in an extract staged for a processor, so the most sensitive copy may never appear in any release manifest. In the EU, the European Data Protection Board's guidelines on controllers and processors restate that a processor must notify the controller without undue delay after becoming aware of a personal data breach, under Article 33(2) of the GDPR. The processor list belongs in the map from the start.

Timing questions depend on the same map. The EDPB's Guidelines 9/2022 on breach notification say a controller should be regarded as aware when it has a reasonable degree of certainty that a security incident has led to personal data being compromised, and allow a short period of investigation to establish that. A map built from run logs shortens that investigation; a map rebuilt from email lengthens it.

Replacement files do not reach copies already made. TensorFlow Datasets notes that previously generated datasets remain readable, and the same holds for any file delivered to a recipient. Each recipient entry should end with a status: replaced, deleted with confirmation, retained under an agreed reason, or open, and what a recipient that trained or evaluated on the affected version must do is a contract question for counsel.

Before a corrected version goes to anyone, the map should let the incident owner tick every line of this delivery checklist:

  • The fix runs on every path that can execute the step, including retries and alternate export paths, and the detector passes on the corrected build.
  • The corrected build carries a new version identifier, and its manifest names the runs that produced it.
  • Every in-window version, derived copy and staging extract on the map has a status: replaced, deleted with confirmation, retained with a reason, or open.
  • Each recipient has been told what changed and asked to confirm replacement or deletion, and each processor has confirmed deletion of intermediate copies.
  • Counsel has the record counts by version and recipient, and the notification decision is written into the incident record.
  • The incident record keeps the window evidence: configuration history, run logs, detector results and the signature each cleared version was tested for.

An illustrative impact map for one disabled redaction rule

This example is invented to show the map at work; it describes no real company, dataset or incident. A company prepares a support-ticket dataset in numbered builds. A configuration change for one help desk brand turns off the phone-number rule in the redaction step, and nobody notices until a reviewer spots phone-shaped strings in an evaluation split.

Runs layer: the configuration history shows the change entering the pipeline before build 2.3 and the fix shipping in build 2.7. The detector rerun finds phone-shaped strings for that brand in builds 2.3 to 2.6 and none in 2.2 or 2.7, so the window holds four builds and the run logs confirm which runs produced them.

Versions and copies layers: four affected builds and three derived copies, a sample prepared for buyers from 2.4, the evaluation split cut from 2.5 where the fault surfaced, and a processor staging extract used for 2.6. Recipients layer: one buyer received build 2.3, a second buyer received 2.6, the processor still holds the 2.6 extract, and two internal reviewers opened the 2.4 sample.

The map turns a vague alarm into a work list: replace four builds, withdraw the sample and the evaluation split, delete one staging extract with confirmation, collect deletion or retention statements from two buyers, and give counsel the record count per build so it can decide on any notices. Without run logs, the window would have reached back to the first build of the pipeline, and every recipient of every build would be on the list.

If the data was de-identified or encrypted, is the map still needed?

Yes. De-identification and encryption change how serious an incident is, not whether its scope must be known, and a processing incident is often a failure of the de-identification itself.

The strongest objection is that de-identified records leaving the building are low risk, so mapping them is wasted effort. That holds only when the de-identification worked in the affected runs, which is exactly what a processing incident calls into question. SourceX's legal framework page makes the general point that de-identification "reduces risk but can't remove it entirely," and its supplier agreement records that re-identification may still be possible.

Encryption protects a copy in transit or at rest, not the copy a recipient has decrypted and loaded. And under the ICO guide, the duty to record a breach with its facts, effects and remedial action applies whether or not it is reported, so even an incident judged unlikely to cause harm needs the map that supports the record.

Incident scoping in a licensing deal with SourceX

For a company licensing records, the map's two hardest layers, method and recipients, line up with what SourceX publishes about its own handling. Its legal framework page says the de-identification method used is recorded for each dataset, and its trust page says SourceX authorizes customers' access to de-identified datasets explicitly for each delivery and records every status change with who made it and when.

On incidents, the trust page says suspected security issues are triaged by the SourceX team, affected access links can be revoked immediately, and affected suppliers and Acquisition Partners are notified, with the full escalation process shared on request. A supplier preparing records can make that process faster by keeping its own side of the map: run logs with step versions, a copy register and the list of anyone who saw extracts before the agreement.

A company checking initial fit can start with the short assessment, which needs no data uploads. The assessment is not an offer and commits no payment.

Methods note: the release-note corpus, coding rules and limits

Corpus: tensorflow/datasets at commit 24c0f2026885d78fae097736587bc47486507aab (committed 2026-10-26), sparse clone read on 2026-10-28. Files: every .py file under tensorflow_datasets/, except names ending in _test.py, files in dummy_data folders (not checked out) and the library folders core/, scripts/ and testing/. Script: bin/tfds_fix_notes.py in this batch's scratch folder, with hand codes in bin/tfds_fix_codes.csv.

Mappings: every RELEASE_NOTES dictionary assigned in a class body and every release_notes= keyword argument in a call. A name that points to a module-level dictionary is resolved, and strings joined with + are folded; no mapping was left unparsed. Entries are the version keys with string notes: 351 in 210 mappings. Codes are keyed by file, mapping line and version, because one file can hold several mappings with the same version (two audio configurations share a note).

Fix notes: entries whose text contains, as a whole word in any case, fix, fixes, fixed, fixing, bug, bugs, bugfix, corrupt, corrupted, corruption, incorrect, incorrectly, wrong, wrongly, error, errors, erroneous, duplicate, duplicates, duplicated, missing, leak, leaks, leaked, leakage, broken, mislabeled, mislabelled, invalid, issue or issues: 51 entries. Each was coded by hand as data (the note says labels, values, records, identifiers, alignment or images users read were wrong or changed; a changed format of identifier values counts as data), representation (storage type, encoding, serialization or a field name only), source (download URL, checksum, archive or manual-download files), withdrawal, unspecified or other. Five of the 32 data entries are one RL Unplugged pixel-colour fix repeated in five sibling datasets, each counted once.

Flags on data entries, applied by script: names a start when the note contains an x.y.z version other than its own key, or, in any case, V followed by digits as a whole word, or previous, earlier, prior or older version(s). Quantifies when, after URLs and x.y.z version strings are removed, a number not attached to a letter, digit, underscore or period (optionally with ~ or %) has at most 40 characters between it and one of the whole words example(s), image(s), record(s), sample(s), row(s), file(s), episode(s), item(s), label(s), annotation(s), instance(s), document(s), video(s) or sentence(s), in either order and any case, or any percentage appears.

Not measured: changes described without a fix word, which the corpus also contains; whether any fix had privacy consequences; and how other dataset publishers write notes. The impact map, the start-finding procedure and the worked example are editorial; the example's builds and recipients are invented.

Frequently asked questions

How is a processing incident different from a data error?

A data error sits in specific records and is traced from their keys. A processing incident sits in a step, such as a disabled redaction rule or an exposed staging folder, so everything the step touched while the fault was live is in scope. Its map starts from the run window, not from the reported record.

How far back should the scope go if nobody knows when the fault started?

Back to the earliest version the faulty code could have produced, until testing narrows it. Rerun a detector for the fault's signature over retained versions, newest first, and take the earliest failing version as the provisional start. Record which signature each cleared version passed.

Do copies held by a processor belong in the impact map?

Yes, and they are often the most sensitive, because a processor may hold an extract taken before de-identification. In the EU, a processor must tell the controller about a personal data breach without undue delay after becoming aware of it. List processors and their retention in the map from the start.

Does shipping a fixed version solve the problem for recipients?

No. A replacement does not change files already delivered, and public tooling such as TensorFlow Datasets keeps previously generated versions readable. Each recipient needs a status: replaced, deleted with confirmation, retained for an agreed reason, or still open.

What numbers does a breach notification need from the map?

Under the ICO guide for UK GDPR, the categories and approximate number of individuals and of records concerned, where possible. Under the HIPAA rule, a business associate should identify each affected individual where possible. Counsel decides whether a notice is due; the map supplies the counts.

Sources

  • ICO (UK GDPR): notify notifiable breaches within 72 hours of awareness; describe categories and approximate numbers of individuals and records; record every breach with facts, effects and remedial action; guide under review after the Data (Use and Access) Act. Source
  • HHS Breach Notification Rule (US HIPAA): a business associate notifies the covered entity no later than 60 days from discovery and identifies each affected individual where possible. Source
  • EDPB Guidelines 9/2022 (EU GDPR): a controller is aware when it has a reasonable degree of certainty that a security incident compromised personal data; a short investigation period is allowed. Source
  • EDPB Guidelines 07/2020 on controllers and processors (EU GDPR): a processor must notify the controller without undue delay after becoming aware of a personal data breach (Article 33(2)). Source
  • TensorFlow Datasets release notes at commit 24c0f20, analyzed for this article: 51 fix notes, 32 data-changing, 2 naming the start version, 4 quantifying affected records. Source
  • The TensorFlow Datasets versioning guide calls its version semantics best effort, warns of unnoticed bugs without a version change, and says previously generated datasets remain readable. Source
  • SourceX's trust page: suspected security issues are triaged, affected access links can be revoked, affected suppliers and Acquisition Partners are notified; access is authorized per delivery; every status change is recorded. Source
  • SourceX's legal framework page: the de-identification method is recorded for each dataset; de-identification reduces risk but cannot remove it entirely. Source

See if your company qualifies

A short company assessment. No data uploads are needed.

See if your company qualifies