Privacy and preparation
Dataset source deletion dependencies: which releases used the record?
By SourceX Editorial · Reviewed by Noah Loul ·
9 sections
Short answer
To find released artifacts that depend on a deleted source record, walk a derivation map forward: from the record's stable source key to every derived item, to every release manifest containing one, to every recipient of that release. First record why the record was deleted, since only some reasons reach releases. If any join was never recorded, scope the response to every release in the window.
Provenance trail
Dataset source deletion
Key takeaways
- Deleting a record in the live system changes nothing in extracts, derived files or delivered copies.
- The deletion reason decides whether releases need action: retention purges usually do not, erasure requests and withdrawn rights often do.
- Four joins make the dependency query possible: source to derived item, derived item to run, derived item to release, release to recipient.
- Run the query in a fixed order and keep zero results with the same evidence as hits.
- Repair and verify a new release before counsel approves any recipient notice.
- Tracing ends at the recipient; what happens to trained models belongs in the license.
Which released artifacts depend on a deleted source record?
You find them by walking a derivation map forward: from the deleted record's stable source key, to every derived item built from it, to every release manifest that contains one of those items, to every recipient who received that release. If any link in that chain was never recorded, the honest answer is that you cannot say, and the response has to be scoped as if every release in the relevant window were affected.
That is why the question is really about what you wrote down at release time. A deletion in Zendesk, Salesforce or NetSuite removes the row from the live system. It does nothing to the extract you took last spring, the de-identified copy your processing partner produced from it, the label file built on top, the evaluation split carved out of that, or the copies already delivered to a licensee.
A released artifact, for this purpose, is anything that left the source system and was frozen under a version name: a delivered dataset file, a derived record (de-identified text, a summary, a chunk, a label, a translation), an evaluation or holdout split, an aggregate table in release notes, and any recipient copy of those. Models a recipient trained are downstream of all of them, but they sit outside what a supplier can inspect directly, which the last sections deal with.
Why was the record deleted, and does the reason reach your releases?
Start with the reason, because it decides whether releases need any action at all. A routine retention purge in the source system usually does not oblige you to recall a licensed copy, while an erasure request, a withdrawn right or a discovered error often does. Record the reason in the issue before anyone touches a release.
Under the GDPR, Article 19 says a controller must communicate an erasure to each recipient to whom the personal data were disclosed, unless that proves impossible or involves disproportionate effort, and must tell the data subject about those recipients if asked. Whether a de-identified release still holds personal data, and so whether that duty reaches it, is a question for counsel on the specific facts. The table below is a starting map, not a legal conclusion.
| Reason the source record was deleted | Does it reach released artifacts? | Who decides | Usual response |
|---|---|---|---|
| Routine retention schedule purge | Usually no, if the license and notices allow retention of released copies | Records owner, with counsel confirming the license terms | Log it; no recipient notice; keep the ledger entry |
| Data subject erasure request | Often yes, where the released copy still identifies the person | Privacy lead and counsel | Run the dependency query; notify recipients as the law and contract require |
| Right withdrawn (customer contract ended, consent revoked, content owner objects) | Yes, if the right was the basis for licensing that record | Counsel and the deal owner | Withdraw the record from future releases; notice under the license terms |
| Record was wrong, duplicated or misfiled | Yes, as a quality correction rather than a legal one | Dataset operations owner | Repair or remove in the next release; notify if a recipient relies on it |
| Record merged into another (CRM dedupe) | Sometimes; derived items may now point to a key that no longer exists | Dataset operations owner | Remap keys in the crosswalk; no recipient action unless content changed |
| Sensitive content found after release | Yes, urgently | Privacy lead, counsel and the release owner | Highest severity; stop further delivery; notify and request deletion |
What lineage has to exist before any record is deleted?
You need four joins recorded at release time: source key to derived item, derived item to transformation run, derived item to release version, and release version to recipient. Without all four the dependency query breaks somewhere, and the break is usually discovered on the day it matters.
Open standards already describe most of this. The W3C PROV ontology defines derivation as the transformation of one entity into another and gives the relation prov:wasDerivedFrom for it, which is the shape of a source-to-derived link. The OpenLineage specification models jobs that consume and produce datasets as their inputs and outputs, with each run identified by a UUID, so a de-identification or labeling step can be tied to the exact run that produced a derived file. DVC keeps a dvc.lock file per pipeline that lists each stage's dependencies and outputs with a content hash field, which is a practical way to pin a release to exact bytes.
None of those tools knows who received a release. The recipient log is a business record you keep yourself, next to the delivery confirmations.
Two design choices save the most pain later. First, key the crosswalk on the source system's immutable ID, not on a ticket number or invoice number that a merge or renumbering can change, and keep the source modified date beside it so you know which version of the record each extract saw. Second, never let a derived item carry the source key in the released file itself; the crosswalk stays private, which is what lets a de-identified release stay de-identified while you can still trace it.
| Join | Minimum fields to keep | Typical home | What breaks without it |
|---|---|---|---|
| Source key to derived item | Source system, object type, stable source ID, source version or modified date, derived item ID | A private crosswalk table held by the supplier | You cannot find which derived items came from the deleted record |
| Derived item to transformation run | Job name, run ID, code version, parameters | Lineage events (for example OpenLineage) or pipeline logs | You cannot tell which processing version produced the item, or rerun it |
| Derived item to release | Release ID, manifest row, file path, content hash | Release manifest plus a lock file or checksum list | You cannot tell which delivered files contain the item |
| Release to recipient | Recipient, agreement reference, delivery date, release ID, delivery method | Delivery log kept with signed agreements | You cannot say who must be told or asked to delete |
| Aggregate to inputs | Statistic name, release ID, input query or filter, record count | Release notes appendix | You cannot judge whether a published count is now wrong |
Running the dependency query from one record to every recipient
Run the query in a fixed order and save each intermediate result to the issue, so that a reviewer can repeat it. The order matters because fan-out happens at every step: one ticket can become several conversation turns, a label, a chunk in a retrieval corpus and a contribution to a published count.
Set the lookback window explicitly. It should start at the first extract that could have contained the record (its creation date in the source system) and end today, and every release inside it is checked even when the crosswalk suggests it cannot contain the record. Releases that were superseded still count if any recipient may hold a copy.
- Resolve the deleted record to its stable source key. If the source system reuses IDs or a merge changed the key, find the key that was current at each extract date from the crosswalk history.
- Query the crosswalk for every derived item whose source key matches, including items derived from derived items (a label on a de-identified turn is two hops away).
- Expand each derived item to the transformation runs that produced it and note the code version, so the repair can be run with the same logic.
- Join derived items to release manifests and list every release ID, file and row that contains one, plus every evaluation or holdout split cut from those releases.
- List aggregates in release notes whose inputs included the record, with the record count each one rests on.
- Join the affected releases to the delivery log to produce the recipient list: recipient, release, delivery date and agreement reference.
- Write the result as a dependency report on the issue, with the query text, the run date and the count of items, releases and recipients found. A count of zero needs the same evidence as any other result.
The lineage and response workflow, field by field
The workflow below is the asset: one issue record per deleted source record (or per batch with a shared reason), with a field for each decision and a clear done condition. The two counsel checkpoints are marked; everything else can be run by dataset operations. Adapt the owners to your organization, but keep the order, because a notice sent before the repair is verified tends to need a second notice.
At a glance
The lineage and response workflow, field by field
| Field | What to record | Owner | Done when |
|---|---|---|---|
| Issue ID and trigger | Issue number, deleted source key, source system, deletion date, who reported it | Dataset operations | The key resolves in the crosswalk or is logged as untraceable |
| Reason and legal basis (counsel checkpoint 1) | Retention purge, erasure request, withdrawn right, error, merge or sensitive content; the clause or law involved | Counsel with the privacy lead | Counsel has recorded whether released artifacts are in scope |
| Severity | S1 to S4 using the severity table in this article | Release owner | Severity set and the response clock started |
| Source and derived links | Every derived item ID with its hop count from the source record | Dataset operations | Crosswalk query saved with run date |
| Affected releases and splits | Release IDs, files, rows, evaluation splits, aggregates | Dataset operations | Every release in the lookback window checked, including zero hits |
| Repair | Remove, replace, re-derive or mask; the transformation run used | Dataset operations | A corrected release candidate exists with a new version ID |
| Regression checks | Row counts, hash diff against the prior release, split leakage check, schema check, aggregate recompute | Quality reviewer | Only the intended items changed and counts reconcile |
| Recipient and version log | Recipient, release received, delivery date, agreement reference | Release owner | List matches the delivery log line by line |
| Change notice (counsel checkpoint 2) | Notice text, recipients, requested action (replace, delete, confirm), response deadline | Counsel approves; release owner sends | Notice sent and receipt recorded for each recipient |
| Reconciliation and closure | Recipient confirmations, outstanding replies, residual risk, lessons for the ledger | Release owner | Every recipient confirmed or escalated, and the issue closed with sign-off |
Severity: notify recipients now, or fix it in the next release?
Notify immediately when the deleted record exposes someone or was never licensable; otherwise, correct it in the next scheduled release and log the change. Severity is set from what the record contains and why it was deleted, not from how many releases it touched. A low-risk error found in ten releases is still a routine fix.
Write the timing targets into your own policy and the license, and treat the column below as prompts, not deadlines. Where a law or contract sets a period, that period wins; where none does, a target your team can actually meet is better than an ambitious one that is missed and then has to be explained.
| Severity | Typical trigger | Recipient action requested | Target timing to set in your own policy |
|---|---|---|---|
| S1 | Identifying or sensitive content found in a released item | Stop use, delete affected files, confirm deletion | Same day notice once the dependency report is complete |
| S2 | Erasure request or withdrawn right reaching an identifiable released item | Replace with corrected release and delete prior copies | Within the period the law or contract sets |
| S3 | Quality error or duplicate that changes labels or evaluation results | Replace the release at next convenience | Next scheduled release, with a change note |
| S4 | Retention purge or key merge with no change to released content | None | Ledger entry only |
Worked example (illustrative): one erased support ticket across three releases
Illustrative only; no real company or person. A field services company licensed de-identified support tickets from its helpdesk. Eleven months later a former customer submits an erasure request, and the helpdesk ticket is deleted. The privacy lead opens an issue and counsel records that one released item may still identify the person, because the ticket text named a rare equipment model at a single site.
The crosswalk resolves the ticket to one de-identified ticket record, four conversation turns, one resolution label and one chunk in a retrieval corpus file: seven derived items in total. The release manifests show those items in release 1.0 (delivered to two recipients), release 1.1 (one recipient) and the evaluation split cut from 1.1. A count in the release 1.0 notes, the number of tickets per product line, included the ticket.
The team sets severity S2, re-derives the affected items with the equipment model generalized, and builds release 1.2. Regression checks show seven items changed, nothing else, and the evaluation split still has no overlap with training. The product line count moves by one, so the release note is corrected. Counsel approves a notice asking all three recipients to replace 1.0 and 1.1 with 1.2 and confirm deletion of the earlier copies; two confirm within the window and one is escalated to the deal owner.
The exception: an earlier pilot extract was de-identified with a crosswalk that was destroyed by design after delivery. The team cannot tell whether the ticket was in it. The issue records that fact, cites the de-identification assessment that justified destroying the crosswalk, and leaves the decision on whether to contact that pilot recipient to counsel.
Where tracing ends: trained models, recipient copies and destroyed crosswalks
Supplier-side tracing ends at the recipient's door. You can prove which releases held the record and that you notified each recipient; you cannot inspect what a recipient trained. What happens to models is a contract question that should be settled before delivery, not during an incident.
Regulators have treated models as derived artifacts. In its 2021 Everalbum order, the FTC required the company to delete models and algorithms it developed using photos and videos uploaded by its users. On the technical side, the SISA training approach described by Bourtoule and colleagues limits the influence of each data point during training so that unlearning is faster than retraining from scratch, but it only helps if a recipient trained that way from the start.
Stop and escalate rather than improvise when any of these is true: the crosswalk is missing or was destroyed; the reason for deletion is disputed; the recipient is outside the agreement you can find; the record may have been used in published research or a public benchmark; or the deletion affects so many records that sampling, not a per-record query, is the only practical check. Each of those needs a deliberate decision from counsel and the release owner, written into the issue.
The Datasheets for Datasets questionnaire asks dataset creators to say whether a dataset will be updated to correct errors or delete instances, how often, by whom and how updates will be communicated, and whether older versions stay supported. Answering those questions in writing before the first release is the cheapest form of deletion readiness there is.
Building deletion traceability into a licensing project
If your company is preparing records for an AI licensing deal, the dependency ledger is easiest to build at the first extract, when the crosswalk, manifests and delivery log are created anyway. Retrofitting it after three releases means reconstructing joins from file names and memory.
In a SourceX deal the supplying company approves scope and terms and signs before the first delivery, so the license negotiation is where deletion handling, corrections and recipient notices get written down. Because SourceX performs the de-identification through approved processing routes, the same negotiation should settle who keeps the private crosswalk and for how long. Eligibility is checked first through a brief company assessment that needs no record uploads.
Frequently asked questions
Does deleting a record in the source system remove it from datasets already released?
No. Deleting a ticket, case or invoice in Zendesk, Salesforce or NetSuite changes the live system only. Extracts, derived files and delivered copies keep their content until someone repairs and replaces the release and asks recipients to act.
Do I have to tell every recipient when a source record is deleted?
Not always. A routine retention purge often needs no recipient action, while an erasure request or withdrawn right may. Under the GDPR, Article 19 requires a controller to communicate erasures to recipients unless that is impossible or involves disproportionate effort; counsel decides how that applies to your releases.
What is the difference between data lineage and a release ledger?
Lineage records how data moved through jobs and transformations, for example as OpenLineage run events. A release ledger adds what lineage tools do not hold: which frozen release versions contained which derived items, and which recipients received each version under which agreement.
What if the record was de-identified before it was released?
Keep the private crosswalk that links source keys to derived item IDs, held only by your team, if you want to trace deletions later. If the crosswalk was destroyed on purpose, record that, rely on the documented de-identification assessment, and let counsel decide whether any recipient contact is needed.
Can a recipient's trained model be made to forget one record?
Sometimes, at a cost. Retraining without the record is the clear method; approaches such as SISA training reduce that cost only when the model was trained that way from the start. What recipients must do with models is best fixed in the license before delivery.
Sources
- GDPR Article 19 requires the controller to communicate any erasure to each recipient to whom the personal data were disclosed, unless impossible or involving disproportionate effort, and to inform the data subject about those recipients on request. Source
- The W3C PROV ontology defines a derivation as a transformation of one entity into another, expressed with prov:wasDerivedFrom. Source
- The OpenLineage specification defines a job as a process that consumes and produces datasets (its inputs and outputs) and identifies each run with a UUID. Source
- DVC maintains a dvc.lock file per dvc.yaml that lists each stage's dependencies and outputs with a content hash field. Source
- The FTC's 2021 Everalbum order required the company to delete models and algorithms it developed using photos and videos uploaded by its users. Source
- SISA training expedites unlearning by strategically limiting the influence of a data point in the training procedure. Source
- Datasheets for Datasets asks whether a dataset will be updated to correct errors, add or delete instances, by whom and how often, how updates are communicated, and whether older versions remain supported. Source
See if your company qualifies
A short company assessment. No data uploads are needed.