Skip to content

Privacy and preparation

Excluded data documentation: what an exclusion log should disclose

By SourceX Editorial · Reviewed by Noah Loul ·

10 sections

Short answer

A dataset exclusion log should disclose, for each exclusion rule, the reason code, the authority behind it, the rule version, the pipeline stage, the approver, how many records it removed and which segments it thinned out. It should not disclose the excluded content. Keep hashed record keys with the data owner so any removal can be traced to its release.

Provenance trail

Excluded data documentation

This page covers excluded data documentation: what an exclusion log should disclose.Illustrative

Key takeaways

  • An exclusion log documents what a dataset cannot teach and why, which buyers need to judge fitness for a task.
  • Record one row per exclusion rule with a reason code, authority, stage, approver and counts, plus a record-level key file.
  • Keep exclusions separate from field-level transformations and from defects shipped with a note.
  • Concentration matters more than volume: a small rule that removes most of one segment must be disclosed or the scope narrowed.
  • Give the buyer reasons, counts and coverage notes; keep keys and rule queries with the data owner for audit.

What should a dataset exclusion log disclose to a buyer?

A dataset exclusion log should tell a buyer how many records were left out, for which approved reason, under which rule version, at which preparation stage, who approved each rule, and what the removals did to coverage. It should not hand over the excluded content itself. The buyer needs to understand the shape of the gap, not see what is inside it.

That distinction drives every field in the template below. An exclusion log is documentation of absence: the delivered files show what a model can learn from, and the log shows what it cannot, and why. A buyer who receives only the files has no way to tell a deliberate privacy removal from a broken export, or a contractual carve-out from a sampling accident.

The log also protects the supplier. When someone later asks why a customer segment, a date range or a ticket category is thin, the answer is already written down with an approver's name on it, instead of reconstructed from memory months after the extraction scripts changed.

Why does the record of what you removed matter as much as the files?

Removals change what a dataset represents, so a buyer cannot judge fitness for a task without knowing them. The Datasheets for Datasets questionnaire asks creators whether any preprocessing or cleaning was done, naming removal of instances as an example, and to describe it. The Croissant responsible AI vocabulary from MLCommons gives the same idea machine-readable homes: a property for preprocessing steps such as filtering out incomplete entries, and a separate property for describing missing data. The Hugging Face dataset card template likewise asks authors to describe data selection criteria and filtering methods.

Regulation points the same way for some buyers. Article 10 of the EU AI Act lists data-preparation operations such as cleaning among the practices a high-risk system provider's data governance must address, and asks providers to identify relevant data gaps or shortcomings. A supplier who cannot say what was removed and why makes that buyer's documentation harder, whatever the files contain.

There is a plainer reason too. Exclusions are where judgment happens. Format conversion and field mapping are mechanical; deciding that a legal-hold mailbox, an opted-out customer or a contract-restricted account stays home is a decision someone made under a rule. Decisions are what reviewers, auditors and buyers ask about.

Which removals count as exclusions, and which are something else?

Log a removal as an exclusion when whole records, files or entities are deliberately kept out of scope by a rule. Field-level redaction, value replacement and format repair are transformations and belong in the preparation record, not the exclusion log. Defects found after a build, such as truncated text or broken encoding that you decide to ship with a note, belong in a validation exception log.

Keeping the three apart stops the exclusion log from turning into a junk drawer. If a record was removed because a redaction failed on it, record it as an exclusion with a quality reason code and point to the failed check, so the count of privacy-driven removals is not inflated by tooling problems.

Which removals count as exclusions, and which are something else?
What happenedExample in business recordsWhere it is recordedBuyer sees
Whole record kept out by a ruleSupport tickets from customers who opted out of sale or sharingExclusion logCount by reason and rule version
Whole source kept out by a decisionHR mailboxes and legal channels never exportedExclusion log, source-level entryNamed source category and scope statement
Field removed or masked inside kept recordsPhone numbers replaced with surrogates in CRM notesPreparation or de-identification recordMethod description, not record counts
Defect shipped with a noteAttachments missing for a block of older ticketsValidation exception logDefect, affected count and impact
Record dropped because preparation failedTicket where the redaction pass erroredExclusion log, quality reason codeCount and the check that failed
Record removed after deliveryCustomer erasure request received after releaseWithdrawal notice and next release diffNotice with affected record keys

What should the exclusion reason and approval template include?

Use one row per exclusion rule, not per record, and attach a record-level key file to each rule. Rules are what get approved; the key file is what lets a reviewer trace a single record. The fields below are written so a data owner can fill them in a spreadsheet today and a provenance engineer can load them into a pipeline later without renaming anything.

Fill the approval fields before the rule runs on the release build. A rule approved after the fact is a finding, not an approval, and the log should say so.

Two fields deserve extra care. The authority field is what separates a defensible exclusion from a convenient one: if nobody can name the policy, clause, request or decision a rule implements, the rule is either unnecessary or undocumented, and both are worth knowing before a buyer asks. The coverage note is the field most often left blank, because it requires someone to look at the distribution of removed records by customer type, product line, region and period rather than just the total. Run that breakdown once per rule; it takes minutes in the source system and it is the single most useful line a buyer reads.

Keep the template in the same repository or folder as the extraction code, versioned with it. When an engineer edits a filter, the change to rule_logic and the bump in version happen in one commit, and the release notes can list rule changes without anyone reconciling two documents by hand.

What should the exclusion reason and approval template include?
FieldWhat it recordsIllustrative entryFilled by
rule_id and versionStable identifier for the exclusion rule and its revisionEX-07 v3Data engineer
reason_codeOne code from the controlled list in the next sectionCONSENT-OPTOUTData owner
rule_logicPlain-language test plus the query or filter referenceAccount flag do_not_sell is true on extraction dateData engineer
authorityPolicy, contract clause, request or law the rule implementsCustomer privacy preference registerPrivacy or legal reviewer
stageWhere in the pipeline the rule runsBefore de-identification, at extractionData engineer
scope_effectRecords, entities and date range affected, with countsIllustrative: 1,240 tickets, 2% of the windowData engineer
coverage_noteSegments, periods or categories the removal thins outSkews toward enterprise accountsData owner
approver and dateNamed role who approved the rule for this releaseHead of support operations, before buildApprover
key_file and release_idHashed record keys removed, tied to the releaseex07_keys.csv, release R2Data engineer
buyer_disclosureWhat the buyer receives about this ruleReason, count, coverage note; no keysPrivacy or legal reviewer

Which reason codes hold up when a reviewer asks why?

A short controlled list beats free text, because counts by reason only mean something when every row uses the same vocabulary. Keep the list under a dozen codes and require a named authority for each. If an engineer needs a new code mid-project, the data owner adds it to the list first, with a definition, and the release notes say the list changed.

At a glance

Which reason codes hold up when a reviewer asks why

Order matters when a record qualifies under more than one code. A ticket from an opted-out customer that also sits in a restricted contract should be counted once, under the code that would have removed it first in the pipeline, with the overlap reported separately if it is large. Otherwise the reason totals add up to more than the records removed and a careful buyer will stop trusting the summary.

  • CONSENT-OPTOUT: the person or account withheld permission for this use, for example a sale or sharing opt-out flag. Authority: the preference register.
  • CONTRACT-RESTRICT: a customer, vendor or partner agreement limits reuse of these records. Authority: the clause, cited by agreement and section.
  • LEGAL-HOLD: records are preserved for litigation or investigation and kept out of any copy. Authority: the hold notice.
  • SENSITIVE-SOURCE: a whole source is out by policy, such as HR, legal or executive mailboxes. Authority: the source exclusion decision.
  • SPECIAL-CATEGORY: records reveal health, biometric or similar sensitive attributes that preparation cannot reliably remove. Authority: the privacy review.
  • THIRD-PARTY-CONTENT: material owned by someone else, such as customer drawings or licensed reports. Authority: the rights review.
  • OUT-OF-SCOPE: outside the agreed systems, dates, languages or record types. Authority: the signed scope.
  • QUALITY-FAIL: preparation could not process the record safely or the record is unusable. Authority: the failed check and its run ID.
  • DUPLICATE: an exact or agreed near-duplicate of a kept record. Authority: the deduplication rule and match threshold.

Can a reviewer trace an exclusion to the exact record and release?

Yes, if every rule writes a key file at run time and the key file is bound to a release identifier. Store a stable hashed key for each removed record rather than the record itself, so the trail survives without keeping a second copy of sensitive material in the evidence folder. Keep the salt or key used for hashing with the data owner, not with the evidence.

Provenance standards give useful vocabulary for this. The W3C PROV ontology links an entity to the activity that generated it and, separately, to the activity that invalidated it, and it relates derived entities to their sources. Read an exclusion as an activity under a rule version that removed specific source entities from a derived release, and the trail becomes a small set of joins: release to rule run, rule run to rule version and approver, rule run to key file.

Test the trail before the buyer does. Pick a handful of removed keys at random, follow each back to its source system record, and confirm the rule logic would remove it today. Then pick a handful of kept records near the rule boundary and confirm the rule would keep them. Write down who ran the test and when; that note is the evidence a governance lead will ask for.

Worked example: an exclusion log for a help desk export (illustrative)

Every number in this example is invented to show the method. A mid-size software company prepares two years of help desk tickets from Zendesk for licensing. The approved scope covers English tickets with their public replies and internal notes, excluding attachments. The table shows the buyer-facing summary the data owner approved for release R1.

Read the table from right to left. The counts tell the buyer how big each hole is, but the coverage notes tell them where the holes are, and that is what changes a modeling decision. Behind each row sits a full template entry with the rule logic, authority, approver and key file, which stays with the company. Before release, the support operations lead signed off each rule, the privacy reviewer signed off EX-03 and EX-04 because their authority is a preference register and customer contracts, and the engineer attached a random-sample trace test for every rule.

Worked example: an exclusion log for a help desk export (illustrative)
Rule and reasonRecords removed (illustrative)Share of windowCoverage note for the buyer
EX-01 OUT-OF-SCOPE: non-English tickets6,1008%Removes most tickets from two regional support queues
EX-03 CONSENT-OPTOUT: accounts with an opt-out flag1,2402%Skews slightly toward enterprise accounts
EX-04 CONTRACT-RESTRICT: three customers with reuse bans2,9004%Removes the healthcare vertical almost entirely
EX-06 SENSITIVE-SOURCE: HR and security queues8001%No HR or security incident tickets in the dataset
EX-08 QUALITY-FAIL: redaction pass errored310under 1%Long tickets with pasted logs are underrepresented

When does an exclusion change what the dataset can be used for?

The meaningful exception in the example is rule EX-04. On paper it is a small share of the window, but it removes nearly every ticket from one industry. A buyer building a support model for healthcare software would find the dataset unfit, and a buyer building a general model would inherit a blind spot without knowing it. Counts alone hide this; the coverage note exposes it.

So the release decision turns on concentration, not volume. When a single rule removes most of a segment the buyer cares about, the data owner has three honest options: disclose the gap prominently in the dataset card, narrow the stated scope so the claim matches the files, or reopen the restriction with the customer before release. Shipping silently is not one of them.

The second exception runs the other way. Rule EX-08 is tiny, but quality failures cluster in long, technical tickets, which are often the most useful for training. If the failure share grows between releases, fix the preparation step rather than accepting a growing hole.

What stays internal, and what happens to exclusions after delivery?

The buyer receives reasons, counts, rule versions, stages and coverage notes. The key files, the rule queries and any reference to identifiable people stay with the data owner, available to an auditor under the agreement. That split lets a buyer document its training data honestly without the supplier disclosing the very information the exclusions exist to protect.

Exclusions do not stop at the first release. A record that must leave the dataset after delivery is a withdrawal, handled by a notice to the recipient and reflected as a new exclusion row in the next release. Under the GDPR, for instance, Article 19 requires a controller to communicate rectification or erasure to each recipient to whom the personal data were disclosed, unless that proves impossible or involves disproportionate effort. Whether that duty applies to a particular dataset is a question for counsel, but the log should be ready for it either way.

Version the log with the dataset. Each release carries its own exclusion summary and a diff from the previous one, so a buyer who trained on release R1 can see exactly which rules changed in R2 and decide whether to retrain.

How SourceX treats exclusion records in a licensing deal

SourceX runs rights review, contracting, preparation and delivery for businesses that license selected operational records directly to SourceX, so the exclusion decisions described here are part of preparing a dataset rather than an afterthought. The company signs off on scope and terms before anything is delivered, which is the natural point to approve the exclusion rules and the buyer-facing summary together. If you want to know whether your records are a fit, the first step is a short company assessment that needs no data upload.

Frequently asked questions

Is an exclusion log the same as a data deletion log?

No. A deletion log records data destroyed in a source system, while an exclusion log records data deliberately kept out of a derived dataset that still exists at the source. The two connect when a deletion or erasure request arrives after delivery, which becomes a new exclusion in the next release.

Should the buyer receive the hashed keys of excluded records?

Usually not. Counts, reasons and coverage notes let the buyer document the dataset; hashed keys add little for them and may still allow matching against other data. Keep keys with the data owner and make them available to an auditor under the agreement.

How detailed should reason codes be?

Detailed enough that two reviewers would assign the same code to the same record, and short enough to fit on one page. Nine or ten codes, each tied to a named authority such as a policy, clause or request register, cover most business datasets.

Who should approve an exclusion rule?

The data owner for the source system approves scope and coverage effects, and a privacy or legal reviewer approves rules whose authority is a law, contract or consent record. The approval should be dated before the rule runs on the release build.

What if an exclusion is discovered after the dataset was delivered?

Treat it as a withdrawal: notify the recipient as the agreement requires, identify the affected records by key, and add the rule to the next release with a diff. Record why the rule was missed so the template catches similar cases earlier.

Sources

  • Datasheets for Datasets asks dataset creators whether any preprocessing or cleaning was done, listing removal of instances as an example, and to describe it. Source
  • The Hugging Face dataset card template asks for data selection criteria and filtering methods and includes a bias, risks and limitations section. Source
  • The Croissant RAI vocabulary defines rai:dataPreprocessingProtocol (for example filtering out incomplete entries) and rai:dataCollectionMissingData. Source
  • EU AI Act Article 10(2) covers data-preparation operations such as cleaning and the identification of relevant data gaps or shortcomings. Source
  • GDPR Article 19 requires a controller to communicate rectification or erasure to each recipient of the personal data unless impossible or disproportionate. Source
  • The W3C PROV ontology links entities to the activities that generated or invalidated them and relates derived entities to their sources. Source

See if your company qualifies

A short company assessment. No data uploads are needed.

See if your company qualifies