Systems and records
Dataset documentation issue tracking for data dictionary errors
By SourceX Editorial · Updated
10 sections
Short answer
To repair a data dictionary error that affects buyers, log it as a tracked defect: record the entry as released, the correct statement and its evidence, and every documentation version that carried it. Rate severity by what a buyer could have concluded, fix it in a new version rather than editing in place, add a check that would have caught it, and send a dated notice to every recipient of an affected version.
Provenance trail
Dataset documentation issue
Key takeaways
- Decide first whether the values or only their description are wrong; the repair paths differ.
- Write dictionary claims as checks (unique, not blank, allowed codes, valid references) and run them on every release.
- Rate severity by what a buyer could have done with the wrong entry; privacy or license-scope errors go straight to the top.
- Never edit a released dictionary in place: publish a new version with a Fixed changelog line naming the old and new text.
- The recipient list comes from the delivery log, so a defect is also a test of whether that log exists.
How do you identify and repair data dictionary errors affecting buyers?
Treat a wrong data dictionary entry as a tracked defect, not a quiet edit. Open an issue with an ID, the exact entry and the correct statement, the evidence for the correction, and the list of released versions that carried the wrong text. Rate severity by what a buyer could have concluded from it, fix the entry in a new documentation version, run checks that would have caught it, and send a dated change notice to every recipient of an affected version.
Dictionary errors matter more than they look because buyers rarely see the source system. They read the dictionary, write their parsing and filtering code against it, and build evaluation sets on top. One wrong unit, code meaning or null rule can bend every downstream use without anyone looking at a raw record.
This guide covers the defect itself: how to recognize it, record it, size its reach, repair it and tell people. It assumes the dataset has released versions and a record of who received which one; if that record does not exist, building it is the first fix.
Is the data wrong, or only its description?
Decide first whether the values are wrong or the words describing them are wrong, because the repair paths differ. If the data is correct and the description is not, you can fix the documentation and reissue it without touching the records. If the data itself is wrong, you are dealing with a data correction, which needs a new data release and usually a wider trace through derived examples.
Many reported problems turn out to be documentation defects. The table lists the common types, how they tend to surface and the harm each does to a buyer who trusted the entry.
| Defect type | Illustrative example | How it usually surfaces | Harm to a buyer who trusted it |
|---|---|---|---|
| Wrong unit or scale | Handle time documented in minutes, stored in seconds | Values look implausibly large in a buyer's profile | Thresholds, filters and model features off by a constant factor |
| Wrong code meaning | Status CLOSED described as resolved, but also covers duplicates and spam | Outcome rates disagree with the buyer's expectations | Labels built on the code mean something different from what was assumed |
| Wrong null or blank rule | Blank described as not applicable, actually means not recorded | Buyer finds blanks concentrated in older records | Missing data treated as a valid negative answer |
| Wrong time basis | Timestamps described as UTC, stored in branch local time | Events appear out of order across regions | Durations and sequences wrong for part of the data |
| Wrong derivation | Resolved flag described as customer-confirmed, actually set by agent | A buyer asks how the flag is produced and the owner checks | Ground truth is a proxy the buyer did not choose |
| Wrong key or join | ticket_id described as unique, repeats after a system migration | Join produces more rows than expected | Duplicate or crossed examples across tasks |
| Stale or missing entry | Field removed in a later export still documented | Parser fails or silently fills defaults | Code written for a field that no longer exists |
Turning dictionary claims into checks that find defects early
The cheapest way to find dictionary errors is to make the dictionary testable. Every entry makes claims (this field is unique, never blank, takes these codes, points to that table), and each claim can be run against the data before release.
At a glance
Turning dictionary claims into checks that find defects early
Analytics teams already work this way. dbt describes its data tests as select queries that look for failing records, and ships four generic tests out of the box: unique, not_null, accepted_values and relationships. The Frictionless Table Schema standard lets a field descriptor carry a name, type, format, description, example and constraints, and lets string and integer fields declare a closed set of categories. A dictionary written in a machine-readable form like that can be validated against each export rather than trusted.
Tests catch the claims that can be expressed as rules. They do not catch a description that is wrong in meaning, such as a flag that is set by the agent rather than the customer. Those surface through buyer questions, owner handoffs and source system changes, so treat each of those events as a prompt to recheck the entries they touch.
- Unit and range claims: compare the documented unit with the observed distribution and a few raw records opened in the source system.
- Code lists: list every distinct value in the export and diff it against the documented codes in both directions.
- Null rules: count blanks by period and source; a sudden change in blank rate usually means the rule changed.
- Keys and joins: test uniqueness and referential integrity on every documented key.
- Meaning claims: ask the system owner to confirm how the field is set, and record who confirmed it and when.
What should the documentation defect and impact management guide include?
The guide is one issue record per defect, kept in the same tracker as data issues but tagged as documentation. These fields are the minimum that lets someone else pick up the issue cold and finish it.
| Field | What it holds | Illustrative entry |
|---|---|---|
| Issue ID and type | Stable identifier and defect type from the table above | DOC-0412, wrong unit |
| Entry as released | Exact text of the dictionary entry and the documentation versions that carried it | handle_time: minutes; dictionary 1.0.0 to 1.2.0 |
| Correct statement and evidence | What the entry should say and how that was established | Seconds; confirmed by source schema and owner on a dated call |
| Data affected | Whether values are wrong or only the description | Description only; values unchanged |
| Severity and rationale | Level from the severity table with the reason | S2: buyers could filter or threshold on a wrong unit |
| Recipients and versions | Every recipient of every affected version, from the release log | Recipients of data releases 1.0 to 1.2 |
| Repair and new version | What changed and the new documentation version | Entry corrected; dictionary 1.2.1 |
| Regression check | The test that now guards the entry | Range test on handle_time added to release checks |
| Notice and reconciliation | Date sent, to whom, acknowledgements, open follow-ups | Sent to all recipients; two acknowledgements pending |
How severe is a wrong entry, and what does each level require?
Rate severity by what a careful buyer could have done with the wrong entry, not by how small the edit is. A one-word fix to a unit can be the most serious defect in the backlog.
| Level | Test | Required response | Notice |
|---|---|---|---|
| S1 critical | The entry misstates privacy, de-identification, consent or license scope | Stop new deliveries of affected versions; escalate to privacy and contract owners | Immediate, individual notice |
| S2 major | A buyer relying on the entry would likely compute wrong labels, features or filters | Fix in a patch documentation release with a regression check | Individual notice to every affected recipient |
| S3 moderate | The entry is wrong but other documentation or the data itself makes the error evident | Fix in the next scheduled documentation release | Changelog entry and release note |
| S4 minor | Wording, typo or formatting with no change in meaning | Batch with other edits | Changelog entry only |
Which released versions and recipients does the defect reach?
The affected set is every recipient who received any version of the documentation that carried the wrong entry, together with any data release shipped alongside it. That answer is only as good as the release log, so keep one row per delivery: recipient, data release ID, documentation version, date and the agreement it was delivered under.
Work backward from the defect. Find the first documentation version where the wrong text appeared, which is often the first release, then list every later version until the fix. Join that version range to the delivery log and you have the recipient list. Then ask each internal team that built on the dataset (evaluation sets, derived task files, sample packs) whether their work copied the wrong statement.
Expect the trace to stop at your boundary. You can say which recipients received the wrong entry; you cannot see what they built with it. The notice is how that downstream reach gets handled, by giving recipients enough detail to check their own use.
Repairing the entry without rewriting history
Never edit a released document in place. The Semantic Versioning specification states that once a versioned package is released its contents must not be modified, and that any change must be released as a new version, with the patch number reserved for backward compatible bug fixes. Applying the same rule to documentation means a buyer can always see what text they relied on and what replaced it.
Record the fix where people look for changes. Keep a Changelog groups entries under Added, Changed, Deprecated, Removed, Fixed and Security, and recommends marking a pulled release with a visible YANKED tag rather than dropping it from the log. A dictionary correction belongs under Fixed; a release withdrawn because of a critical defect should stay listed and be marked.
Then update the human-readable card. Hugging Face's dataset card template has sections for dataset structure, uses and out-of-scope use, and bias, risks and limitations; a defect that changes how a field should be used often belongs in more than one of them. Datasheets for Datasets asks creators whether the dataset will be updated, for example to correct labeling errors, by whom and how updates will be communicated, which is the question this repair process answers in practice.
Use this order for the repair itself:
- Freeze the issue facts: entry as released, correct statement, evidence, versions.
- Write the corrected entry and have the field's system owner sign off on it.
- Add the regression check that would have caught the defect, and run it on every release still in use.
- Publish the new documentation version with a Fixed changelog line that names the field and the old and new statement.
- Send notices by severity, log acknowledgements, and close the issue only when every affected recipient has been reached or the gap is recorded.
Worked example (illustrative): a duration documented in minutes but stored in seconds
This example is illustrative; the field, versions and counts are invented. A support-operations dataset documents handle_time as active handling time in minutes. A recipient profiling the data reports that the median looks far too long for chat contacts.
The dataset owner opens a sample of records in the source help desk and confirms the field is stored in seconds; the source schema says so too. The values are right, so this is a documentation defect, not a data defect. The wrong entry has been present since the first release, so three documentation versions are affected and the release log shows four recipients across them.
Severity is S2. A buyer could reasonably have filtered for contacts under some number of minutes, or used handle time as a difficulty feature, and either would be off by a factor of sixty. The fix is a patch documentation release with the corrected entry, a range check that fails if the median falls outside a plausible band for the channel, a Fixed changelog line, and individual notices that state the old text, the new text and a suggested check: divide any thresholds by sixty and rerun.
One exception surfaced during the trace. An internal evaluation set had copied the wrong unit into its task instructions, which made it a derived artifact with its own defect. It got a linked issue and its own patch release, because recipients of the evaluation set needed a separate notice.
Where a documentation fix is not enough
Some dictionary defects cannot be repaired with words alone. Stop and widen the scope in these situations:
- The entry is right and the data is wrong, or both are wrong. Treat it as a data correction with a new data release and a trace through derived examples.
- The defect touches privacy or rights, such as a field documented as removed or masked that was delivered in full. That is an S1 issue for privacy and contract owners, and it may carry obligations the documentation process cannot satisfy.
- The correct meaning cannot be established because the system owner has left or the source system is retired. Record the uncertainty in the entry and in limitations instead of guessing.
- Recipients cannot be identified because no delivery log exists. Notify everyone who could plausibly have received the data and build the log before the next release.
- The fix would change what the license covers, such as a field that was never in scope. Send it to the people who own the agreement before anything ships.
Keeping dictionaries trustworthy before a dataset is licensed
For a company licensing its records, a defect found after delivery costs far more trust than the same defect found before. Running the checks above on the first release, and naming a person who owns each field's meaning, is most of the protection.
For businesses that license selected operational records directly to SourceX, the rights review, contracting, preparation and delivery are SourceX's work, and the company's sign-off on scope and terms before delivery is a natural checkpoint for confirming that the dictionary matches the data. A short assessment with no record upload shows whether the company qualifies at all.
Frequently asked questions
What is dataset documentation issue tracking?
It is the practice of logging errors in a dataset's documentation, such as its data dictionary or dataset card, as tracked issues with severity, affected versions, a repair and a notice. It keeps documentation fixes separate from data fixes and makes each correction traceable to the recipients who relied on the wrong text.
Should a typo in a data dictionary get a new version?
If the typo changes no meaning, batch it into the next documentation release with a changelog line. If it could change how someone parses or interprets a field, treat it as a defect, release a patch version and notify recipients. Editing the released file silently is the one option to avoid.
How do you tell recipients about a documentation correction?
Send a dated notice that names the field, the old text, the new text, the affected versions and a practical check they can run on their own work. Log who received it and whether they acknowledged it. Critical defects that touch privacy or license scope need individual and immediate notice.
Can automated tests find every data dictionary error?
No. Tests catch claims that can be written as rules, such as uniqueness, allowed codes, required values and valid references. They miss errors of meaning, such as who sets a flag or what a status really covers, which need the system owner's confirmation.
Who should own a data dictionary entry?
The person or team that controls how the field is set in the source system, with the dataset owner responsible for publishing it. Record both names on the entry. When either changes jobs, recheck the entries they owned.
Sources
- dbt data tests are select queries that return failing records, and dbt ships four generic tests: unique, not_null, accepted_values and relationships. Source
- A Table Schema field descriptor can carry name, title, type, format, example, description and constraints, and string or integer fields may declare categories. Source
- Semantic Versioning: released contents must not be modified, changes go in a new version, and patch versions are for backward compatible bug fixes. Source
- Keep a Changelog 1.1.0 groups changes as Added, Changed, Deprecated, Removed, Fixed and Security, and marks pulled releases YANKED. Source
- The Hugging Face dataset card template includes Dataset Structure, Uses, Out-of-Scope Use, and Bias, Risks, and Limitations sections. Source
- Datasheets for Datasets (Gebru et al.) asks whether a dataset will be updated, for example to correct labeling errors, by whom, and how updates will be communicated. Source
See if your company qualifies
A short company assessment. No data uploads are needed.