Getting started
What is dark data, and how do you find it in your company?
By SourceX Editorial · Updated
Short answer
Dark data is information a company collects and stores during normal operations but never uses again, such as closed tickets, retired system databases, departed staff drives and old project folders. Finding it is an inventory job, not a technology project: ask each department what it stopped looking at, then record the system, owner, years and contents.
Key takeaways
- Dark data is defined by neglect, not by format: databases, files, mailboxes and paper can all be dark.
- The largest pockets usually sit in retired systems, departed employees' accounts and closed project folders.
- Discovery starts with interviews and existing system lists, then moves to storage scans.
- Every pocket needs an owner, a date range and a decision: delete, keep, archive or review for value.
What is dark data?#
Dark data is information a company gathers and keeps as a by-product of operations but does not use for any current purpose. The term is commonly attributed to IT research analysts describing storage that costs money and carries risk while nobody looks at it.
Dark data is not the same as bad data. Much of it is ordinary operational history: support conversations, dispatch notes, quality reports and project correspondence that served their purpose and were left in place. Some of it is redundant or trivial and should go. Some of it records years of expert work.
For a COO, the working definition is simpler: anything stored that no current process, report or person relies on.
Ten places dark data hides in a mid-size company#
Dark data hides wherever a process ended but the storage did not. These ten locations come up again and again in companies of a few hundred people, across software, services and operating businesses.
- Retired help desks, such as an old Zendesk or Freshdesk instance kept for read-only lookups.
- Legacy ERP, WMS or accounting databases left running or backed up after a migration.
- Mailboxes and drives of former employees, often converted to shared or archived accounts.
- Closed project folders on file servers, SharePoint sites and Box or Dropbox accounts.
- Chat history in Slack or Teams channels for finished projects and disbanded teams.
- Call recordings and transcripts in phone systems and contact center tools.
- Exports and spreadsheets saved to desktops during past reporting cycles or audits.
- Quality, inspection and maintenance logs kept outside the main system.
- Backups and snapshots kept long after the systems they protected changed.
- Paper archives in offsite storage, including job folders and signed forms.
How do you find dark data? A discovery checklist#
Finding dark data starts with people and system lists, then moves to technical scans. Department leaders know which tools were retired and where old work lives; finance knows what was paid for; IT knows what is still running, backed up or licensed.
Automated tools help with the scan step. Google Cloud's Sensitive Data Protection, for example, describes itself as a platform for inspecting, classifying and de-identifying sensitive data across text, images and storage repositories. Tools find content and flag risk; people still decide what it means.
| Step | Who | Output |
|---|---|---|
| List every system paid for now and in recent years | Finance and IT | A system list built from invoices and contracts |
| Ask each department what it stopped using | COO with department heads | Retired tools, old folders and archives |
| Check identity records for departed users who still hold data | IT | Accounts and drives still holding content |
| Scan file shares and cloud storage for size and last-access dates | IT | Large, untouched folders and stores |
| Review backup and offsite storage inventories | IT and facilities | Backups and boxes with their contents |
| Record owner, years, contents and restrictions for each pocket | Each department lead | One inventory row per pocket |
| Assign a decision to every row | COO with counsel | Delete, keep, archive or review for value |
What to record for each pocket you find#
Recording each pocket of dark data in the same format turns a pile of findings into decisions. One row per pocket is enough, and every field below can be filled from interviews and scans without opening individual records or moving files.
- System or location, and whether it is still running, backed up only or on paper.
- Business owner and IT custodian.
- Record types inside, in plain words such as resolved tickets or inspection sheets.
- First and last year covered.
- Approximate volume, as the system or storage report shows it.
- Known restrictions: client ownership, contract terms, holds and personal data.
- Proposed decision and the date it will be reviewed.
Which dark data matters, and which is clutter?#
Dark data matters when it records real decisions with outcomes and the company owns it; it is clutter when it duplicates other records or captures nothing a person decided. Sorting the two early keeps discovery from turning into a storage cleanup alone.
When in doubt, ask whether a new employee could learn how the company works from the pocket. Resolved tickets and inspection reports teach; notification emails and duplicate exports do not.
| Usually worth a closer look | Usually clutter |
|---|---|
| Resolved support tickets linked to fixes | Auto-generated notifications and alerts |
| Job records with diagnosis, parts and callbacks | Duplicate exports of the same report |
| NCRs and CAPAs with root causes | System logs with no human input |
| Proposals with win and loss notes | Drafts superseded by final versions |
| RFI responses and submittal reviews | Stock images and old marketing assets |
What does leaving dark data alone cost?#
Leaving dark data alone carries cost and risk even if nobody opens it. Old accounts and file shares hold customer and employee personal details under weaker controls than live systems, and they can be requested in litigation like any other record.
Automatic deletion settings can also remove dark data before anyone decides on it. Zendesk lets admins create ticket deletion schedules that delete archived tickets after a set period, and deleted tickets cannot be restored. Zoom lets admins delete cloud recordings automatically after a set number of days. Check these settings in every system on your list before the inventory is finished.
There is a quieter cost too. When a retired system's last administrator leaves, the knowledge of how to export or interpret its data leaves with them, and the archive becomes harder to use for anything at all, including a defensible deletion.
Illustrative: a manufacturer maps its dark data#
Illustrative: a fictional precision machining company with two plants asks its COO to cut storage spending. Discovery interviews surface an old MES database kept after an upgrade, quality folders full of NCRs and CAPAs on a shared drive, a retired help desk for distributor questions and the drives of several retired engineers.
The scan shows customer drawings stored in the same shared drive as the quality reports. The COO separates them: customer-owned designs and any export-controlled work are marked out of scope for anything beyond retention. The NCRs, CAPAs and help desk tickets are company-written records with clear outcomes.
The company deletes duplicate exports and stale backups, archives the MES database with an index and keeps the quality and support records for a value review.
How SourceX looks at dark data#
In the SourceX five-step transaction, a dark data inventory feeds Supply, the step where candidate records are described before anything else happens. The fit check uses only descriptions of each pocket: system, years, record types and known restrictions. Pockets that show expert work with outcomes are rated with the SourceX Enterprise Data Value Framework before any rights review or preparation begins.
Frequently asked questions
Is dark data the same as unstructured data?
No. Much dark data is unstructured, such as documents, email and chat, but whole structured databases can be dark when a retired system keeps running untouched. Unstructured data can also be in active daily use. Dark data describes whether information is used, not how it is formatted.
Should we delete dark data to reduce risk?
Delete what duplicates other records, holds personal details you no longer need or has no plausible use, after checking legal holds and retention rules. Keep or review what records real work with outcomes. A blanket purge can destroy records you are required to keep or that have business value.
How much effort does dark data discovery take?
Effort depends on how many systems and sites a company has, but the first pass is mostly interviews and existing lists rather than new software. Start with the departments that retired a system most recently, because their archives are best remembered and most at risk of being lost.
Who should own dark data once it is found?
Each pocket should have a business owner who understands its contents, such as the head of support for an old help desk or the quality manager for NCR folders. IT keeps custody and access controls; the business owner decides whether to keep, delete or review for value.
Can dark data be licensed to AI developers?
Some can. Records that show expert work, such as resolved tickets, job histories or quality investigations, may be licensed for a defined use after a rights review and removal of personal and confidential details. The company keeps ownership and approves scope. Client-owned and export-controlled material is excluded.
Sources
- Google's DLP API v2 definition states that Sensitive Data Protection provides access to a sensitive data inspection, classification, and de-identification platform that works on text, images, and Google Cloud storage repositories. Source
- Zendesk admins can create ticket deletion schedules that delete archived tickets after a set period; deleted tickets cannot be restored. Source
- Zoom lets account owners and admins enable deleting cloud recordings after a specified number of days. Source
Related resources
See if your company qualifies
A short company assessment. No data uploads are needed.