Skip to content

Privacy and preparation

Dataset source deletion dependencies: which releases used the record?

By SourceX Editorial · Reviewed by Noah Loul ·

9 sections

Short answer

To find released artifacts that depend on a deleted source record, walk a derivation map forward: from the record's stable source key to every derived item, to every release manifest containing one, to every recipient of that release. First record why the record was deleted, since only some reasons reach releases. If any join was never recorded, scope the response to every release in the window.

Provenance trail

Dataset source deletion

This page covers dataset source deletion dependencies: which releases used the record.Illustrative

Key takeaways

  • Deleting a record in the live system changes nothing in extracts, derived files or delivered copies.
  • The deletion reason decides whether releases need action: retention purges usually do not, erasure requests and withdrawn rights often do.
  • Four joins make the dependency query possible: source to derived item, derived item to run, derived item to release, release to recipient.
  • Run the query in a fixed order and keep zero results with the same evidence as hits.
  • Repair and verify a new release before counsel approves any recipient notice.
  • Tracing ends at the recipient; what happens to trained models belongs in the license.

Which released artifacts depend on a deleted source record?

You find them by walking a derivation map forward: from the deleted record's stable source key, to every derived item built from it, to every release manifest that contains one of those items, to every recipient who received that release. If any link in that chain was never recorded, the honest answer is that you cannot say, and the response has to be scoped as if every release in the relevant window were affected.

That is why the question is really about what you wrote down at release time. A deletion in Zendesk, Salesforce or NetSuite removes the row from the live system. It does nothing to the extract you took last spring, the de-identified copy your processing partner produced from it, the label file built on top, the evaluation split carved out of that, or the copies already delivered to a licensee.

A released artifact, for this purpose, is anything that left the source system and was frozen under a version name: a delivered dataset file, a derived record (de-identified text, a summary, a chunk, a label, a translation), an evaluation or holdout split, an aggregate table in release notes, and any recipient copy of those. Models a recipient trained are downstream of all of them, but they sit outside what a supplier can inspect directly, which the last sections deal with.

Why was the record deleted, and does the reason reach your releases?

Start with the reason, because it decides whether releases need any action at all. A routine retention purge in the source system usually does not oblige you to recall a licensed copy, while an erasure request, a withdrawn right or a discovered error often does. Record the reason in the issue before anyone touches a release.

Under the GDPR, Article 19 says a controller must communicate an erasure to each recipient to whom the personal data were disclosed, unless that proves impossible or involves disproportionate effort, and must tell the data subject about those recipients if asked. Whether a de-identified release still holds personal data, and so whether that duty reaches it, is a question for counsel on the specific facts. The table below is a starting map, not a legal conclusion.

Why was the record deleted, and does the reason reach your releases?
Reason the source record was deletedDoes it reach released artifacts?Who decidesUsual response
Routine retention schedule purgeUsually no, if the license and notices allow retention of released copiesRecords owner, with counsel confirming the license termsLog it; no recipient notice; keep the ledger entry
Data subject erasure requestOften yes, where the released copy still identifies the personPrivacy lead and counselRun the dependency query; notify recipients as the law and contract require
Right withdrawn (customer contract ended, consent revoked, content owner objects)Yes, if the right was the basis for licensing that recordCounsel and the deal ownerWithdraw the record from future releases; notice under the license terms
Record was wrong, duplicated or misfiledYes, as a quality correction rather than a legal oneDataset operations ownerRepair or remove in the next release; notify if a recipient relies on it
Record merged into another (CRM dedupe)Sometimes; derived items may now point to a key that no longer existsDataset operations ownerRemap keys in the crosswalk; no recipient action unless content changed
Sensitive content found after releaseYes, urgentlyPrivacy lead, counsel and the release ownerHighest severity; stop further delivery; notify and request deletion

What lineage has to exist before any record is deleted?

You need four joins recorded at release time: source key to derived item, derived item to transformation run, derived item to release version, and release version to recipient. Without all four the dependency query breaks somewhere, and the break is usually discovered on the day it matters.

Open standards already describe most of this. The W3C PROV ontology defines derivation as the transformation of one entity into another and gives the relation prov:wasDerivedFrom for it, which is the shape of a source-to-derived link. The OpenLineage specification models jobs that consume and produce datasets as their inputs and outputs, with each run identified by a UUID, so a de-identification or labeling step can be tied to the exact run that produced a derived file. DVC keeps a dvc.lock file per pipeline that lists each stage's dependencies and outputs with a content hash field, which is a practical way to pin a release to exact bytes.

None of those tools knows who received a release. The recipient log is a business record you keep yourself, next to the delivery confirmations.

Two design choices save the most pain later. First, key the crosswalk on the source system's immutable ID, not on a ticket number or invoice number that a merge or renumbering can change, and keep the source modified date beside it so you know which version of the record each extract saw. Second, never let a derived item carry the source key in the released file itself; the crosswalk stays private, which is what lets a de-identified release stay de-identified while you can still trace it.

What lineage has to exist before any record is deleted?
JoinMinimum fields to keepTypical homeWhat breaks without it
Source key to derived itemSource system, object type, stable source ID, source version or modified date, derived item IDA private crosswalk table held by the supplierYou cannot find which derived items came from the deleted record
Derived item to transformation runJob name, run ID, code version, parametersLineage events (for example OpenLineage) or pipeline logsYou cannot tell which processing version produced the item, or rerun it
Derived item to releaseRelease ID, manifest row, file path, content hashRelease manifest plus a lock file or checksum listYou cannot tell which delivered files contain the item
Release to recipientRecipient, agreement reference, delivery date, release ID, delivery methodDelivery log kept with signed agreementsYou cannot say who must be told or asked to delete
Aggregate to inputsStatistic name, release ID, input query or filter, record countRelease notes appendixYou cannot judge whether a published count is now wrong

Running the dependency query from one record to every recipient

Run the query in a fixed order and save each intermediate result to the issue, so that a reviewer can repeat it. The order matters because fan-out happens at every step: one ticket can become several conversation turns, a label, a chunk in a retrieval corpus and a contribution to a published count.

Set the lookback window explicitly. It should start at the first extract that could have contained the record (its creation date in the source system) and end today, and every release inside it is checked even when the crosswalk suggests it cannot contain the record. Releases that were superseded still count if any recipient may hold a copy.

  • Resolve the deleted record to its stable source key. If the source system reuses IDs or a merge changed the key, find the key that was current at each extract date from the crosswalk history.
  • Query the crosswalk for every derived item whose source key matches, including items derived from derived items (a label on a de-identified turn is two hops away).
  • Expand each derived item to the transformation runs that produced it and note the code version, so the repair can be run with the same logic.
  • Join derived items to release manifests and list every release ID, file and row that contains one, plus every evaluation or holdout split cut from those releases.
  • List aggregates in release notes whose inputs included the record, with the record count each one rests on.
  • Join the affected releases to the delivery log to produce the recipient list: recipient, release, delivery date and agreement reference.
  • Write the result as a dependency report on the issue, with the query text, the run date and the count of items, releases and recipients found. A count of zero needs the same evidence as any other result.

The lineage and response workflow, field by field

The workflow below is the asset: one issue record per deleted source record (or per batch with a shared reason), with a field for each decision and a clear done condition. The two counsel checkpoints are marked; everything else can be run by dataset operations. Adapt the owners to your organization, but keep the order, because a notice sent before the repair is verified tends to need a second notice.

At a glance

The lineage and response workflow, field by field

The lineage and response workflow, field by field
FieldWhat to recordOwnerDone when
Issue ID and triggerIssue number, deleted source key, source system, deletion date, who reported itDataset operationsThe key resolves in the crosswalk or is logged as untraceable
Reason and legal basis (counsel checkpoint 1)Retention purge, erasure request, withdrawn right, error, merge or sensitive content; the clause or law involvedCounsel with the privacy leadCounsel has recorded whether released artifacts are in scope
SeverityS1 to S4 using the severity table in this articleRelease ownerSeverity set and the response clock started
Source and derived linksEvery derived item ID with its hop count from the source recordDataset operationsCrosswalk query saved with run date
Affected releases and splitsRelease IDs, files, rows, evaluation splits, aggregatesDataset operationsEvery release in the lookback window checked, including zero hits
RepairRemove, replace, re-derive or mask; the transformation run usedDataset operationsA corrected release candidate exists with a new version ID
Regression checksRow counts, hash diff against the prior release, split leakage check, schema check, aggregate recomputeQuality reviewerOnly the intended items changed and counts reconcile
Recipient and version logRecipient, release received, delivery date, agreement referenceRelease ownerList matches the delivery log line by line
Change notice (counsel checkpoint 2)Notice text, recipients, requested action (replace, delete, confirm), response deadlineCounsel approves; release owner sendsNotice sent and receipt recorded for each recipient
Reconciliation and closureRecipient confirmations, outstanding replies, residual risk, lessons for the ledgerRelease ownerEvery recipient confirmed or escalated, and the issue closed with sign-off

Severity: notify recipients now, or fix it in the next release?

Notify immediately when the deleted record exposes someone or was never licensable; otherwise, correct it in the next scheduled release and log the change. Severity is set from what the record contains and why it was deleted, not from how many releases it touched. A low-risk error found in ten releases is still a routine fix.

Write the timing targets into your own policy and the license, and treat the column below as prompts, not deadlines. Where a law or contract sets a period, that period wins; where none does, a target your team can actually meet is better than an ambitious one that is missed and then has to be explained.

Severity: notify recipients now, or fix it in the next release?
SeverityTypical triggerRecipient action requestedTarget timing to set in your own policy
S1Identifying or sensitive content found in a released itemStop use, delete affected files, confirm deletionSame day notice once the dependency report is complete
S2Erasure request or withdrawn right reaching an identifiable released itemReplace with corrected release and delete prior copiesWithin the period the law or contract sets
S3Quality error or duplicate that changes labels or evaluation resultsReplace the release at next convenienceNext scheduled release, with a change note
S4Retention purge or key merge with no change to released contentNoneLedger entry only

Worked example (illustrative): one erased support ticket across three releases

Illustrative only; no real company or person. A field services company licensed de-identified support tickets from its helpdesk. Eleven months later a former customer submits an erasure request, and the helpdesk ticket is deleted. The privacy lead opens an issue and counsel records that one released item may still identify the person, because the ticket text named a rare equipment model at a single site.

The crosswalk resolves the ticket to one de-identified ticket record, four conversation turns, one resolution label and one chunk in a retrieval corpus file: seven derived items in total. The release manifests show those items in release 1.0 (delivered to two recipients), release 1.1 (one recipient) and the evaluation split cut from 1.1. A count in the release 1.0 notes, the number of tickets per product line, included the ticket.

The team sets severity S2, re-derives the affected items with the equipment model generalized, and builds release 1.2. Regression checks show seven items changed, nothing else, and the evaluation split still has no overlap with training. The product line count moves by one, so the release note is corrected. Counsel approves a notice asking all three recipients to replace 1.0 and 1.1 with 1.2 and confirm deletion of the earlier copies; two confirm within the window and one is escalated to the deal owner.

The exception: an earlier pilot extract was de-identified with a crosswalk that was destroyed by design after delivery. The team cannot tell whether the ticket was in it. The issue records that fact, cites the de-identification assessment that justified destroying the crosswalk, and leaves the decision on whether to contact that pilot recipient to counsel.

Where tracing ends: trained models, recipient copies and destroyed crosswalks

Supplier-side tracing ends at the recipient's door. You can prove which releases held the record and that you notified each recipient; you cannot inspect what a recipient trained. What happens to models is a contract question that should be settled before delivery, not during an incident.

Regulators have treated models as derived artifacts. In its 2021 Everalbum order, the FTC required the company to delete models and algorithms it developed using photos and videos uploaded by its users. On the technical side, the SISA training approach described by Bourtoule and colleagues limits the influence of each data point during training so that unlearning is faster than retraining from scratch, but it only helps if a recipient trained that way from the start.

Stop and escalate rather than improvise when any of these is true: the crosswalk is missing or was destroyed; the reason for deletion is disputed; the recipient is outside the agreement you can find; the record may have been used in published research or a public benchmark; or the deletion affects so many records that sampling, not a per-record query, is the only practical check. Each of those needs a deliberate decision from counsel and the release owner, written into the issue.

The Datasheets for Datasets questionnaire asks dataset creators to say whether a dataset will be updated to correct errors or delete instances, how often, by whom and how updates will be communicated, and whether older versions stay supported. Answering those questions in writing before the first release is the cheapest form of deletion readiness there is.

Building deletion traceability into a licensing project

If your company is preparing records for an AI licensing deal, the dependency ledger is easiest to build at the first extract, when the crosswalk, manifests and delivery log are created anyway. Retrofitting it after three releases means reconstructing joins from file names and memory.

In a SourceX deal the supplying company approves scope and terms and signs before the first delivery, so the license negotiation is where deletion handling, corrections and recipient notices get written down. Because SourceX performs the de-identification through approved processing routes, the same negotiation should settle who keeps the private crosswalk and for how long. Eligibility is checked first through a brief company assessment that needs no record uploads.

Frequently asked questions

Does deleting a record in the source system remove it from datasets already released?

No. Deleting a ticket, case or invoice in Zendesk, Salesforce or NetSuite changes the live system only. Extracts, derived files and delivered copies keep their content until someone repairs and replaces the release and asks recipients to act.

Do I have to tell every recipient when a source record is deleted?

Not always. A routine retention purge often needs no recipient action, while an erasure request or withdrawn right may. Under the GDPR, Article 19 requires a controller to communicate erasures to recipients unless that is impossible or involves disproportionate effort; counsel decides how that applies to your releases.

What is the difference between data lineage and a release ledger?

Lineage records how data moved through jobs and transformations, for example as OpenLineage run events. A release ledger adds what lineage tools do not hold: which frozen release versions contained which derived items, and which recipients received each version under which agreement.

What if the record was de-identified before it was released?

Keep the private crosswalk that links source keys to derived item IDs, held only by your team, if you want to trace deletions later. If the crosswalk was destroyed on purpose, record that, rely on the documented de-identification assessment, and let counsel decide whether any recipient contact is needed.

Can a recipient's trained model be made to forget one record?

Sometimes, at a cost. Retraining without the record is the clear method; approaches such as SISA training reduce that cost only when the model was trained that way from the start. What recipients must do with models is best fixed in the license before delivery.

Sources

  • GDPR Article 19 requires the controller to communicate any erasure to each recipient to whom the personal data were disclosed, unless impossible or involving disproportionate effort, and to inform the data subject about those recipients on request. Source
  • The W3C PROV ontology defines a derivation as a transformation of one entity into another, expressed with prov:wasDerivedFrom. Source
  • The OpenLineage specification defines a job as a process that consumes and produces datasets (its inputs and outputs) and identifies each run with a UUID. Source
  • DVC maintains a dvc.lock file per dvc.yaml that lists each stage's dependencies and outputs with a content hash field. Source
  • The FTC's 2021 Everalbum order required the company to delete models and algorithms it developed using photos and videos uploaded by its users. Source
  • SISA training expedites unlearning by strategically limiting the influence of a data point in the training procedure. Source
  • Datasheets for Datasets asks whether a dataset will be updated to correct errors, add or delete instances, by whom and how often, how updates are communicated, and whether older versions remain supported. Source

See if your company qualifies

A short company assessment. No data uploads are needed.

See if your company qualifies