What is dark data, and what should you do with it?

What counts as dark data?#

Dark data is any information a company created or collected while doing its work and then stopped using, reviewing or managing. It is usually still sitting in a live system, a backup, a file server or a cloud account that keeps renewing.

Dark data is not the same as unstructured data. A legacy ERP table of closed service orders is perfectly structured and can still be dark if nobody has queried it in years. Equally, an active knowledge base made of free-text articles is not dark because people use it every day.

The useful test is ownership. If nobody can say who is responsible for a store, what it covers and how long it should be kept, it is dark, whatever its format.

Dark data examples in operating companies#

Dark data looks different in each kind of business, but the pattern repeats: the work finished, the records stayed and nobody went back. The table below lists common examples by company type.

What these examples share is that people created them while doing skilled work. An exception email shows how a dispatcher recovered a missed pickup; an NCR shows how an inspector traced a defect to its cause. That is why some dark data is worth a second look before anyone deletes it.

What are the risks of dark data?#

The first risk of dark data is personal information nobody tracks. Old support tickets, HR files saved to a shared drive by mistake and call recordings can contain customer and employee details well past any retention schedule.

The second is security. Retired systems that still run tend to miss patches, keep stale admin accounts and sit outside current monitoring. The third is obligation: records a company keeps may have to be produced in a dispute, and contracts may have required them to be returned or destroyed long ago.

Cost is real but usually the smallest of the four risks. Storage is cheap; not knowing what you store is not.

How to find dark data#

Finding dark data is an inventory exercise, and it works best when several sources are combined. No single list, whether from IT, finance or the people who have been around longest, catches everything.

Expect the first pass to miss things. Treat the result as a living register that is reviewed whenever a system is added, migrated or retired, and whenever a team is reorganized, because those are the moments when new dark data is created.

• 1Pull every software subscription from accounts payable and company card statements.
• 2Review admin consoles for inactive workspaces, archived projects and unused shared drives.
• 3Ask long-tenured staff which old systems and folders they still remember.
• 4List systems retired in past migrations and confirm whether anything still runs or renews.
• 5Check shared inboxes, team mailboxes and departed users' drives.
• 6Note on-premises servers, backup drives and storage closets with physical media.
• 7Record for each store: system, owner, date range, record family, personal data present and known obligations.

Four options for each dark data store#

Each dark data store should end with one of four decisions, made by its owner with counsel checking holds and obligations. Applying one answer to everything, such as deleting it all or keeping it all, usually gets at least one store wrong.

A practical decision rule: delete what has no legal, contractual or business reason to exist; archive what must be kept; de-identify what is useful but full of personal details; and assess for licensing only records the company owns that show how real work was done.

Deferring is a legitimate choice as long as it is recorded. An archive with an owner and a review date is a decision; a folder left alone because nobody wanted to choose is simply dark data with a longer life.

Illustrative: an engineering firm's project server#

Illustrative: a fictional structural engineering firm is replacing an on-premises project server that holds folders for every project since the firm began, alongside Deltek Vantagepoint for project accounting. The COO uses the replacement as the moment to sort the server's contents.

The inventory finds four kinds of material. HR files saved to a shared folder by mistake move into the HR system with restricted access. Client drawings and deliverables are archived under the terms of each client agreement. Duplicate scans and superseded drafts are deleted after a check for holds.

The firm's own RFI responses and internal review comments are de-identified, with client names and project addresses removed, and set aside for a licensing assessment. The new server starts with an owner for every folder.

Where SourceX fits#

SourceX becomes relevant only for stores that land in the assess-for-licensing column. The fit check uses metadata, such as systems, years of history and record families, and nothing is shared during the initial assessment.

In the SourceX Enterprise Data Value Framework, dark operational records often rate well on human-generated signal and domain expertise, because they capture people solving real problems. They often need work on data cleanliness and carry privacy burden and preparation cost, which reduce net value. Records that are easy to reproduce from public sources rate lower, because reproducibility reduces value. The framework is a SourceX-developed methodology with qualitative ratings, not a price list.