Process historian data: what to keep, archive and license

Why historian data matters for AI now#

Historian data matters for AI because models of physical processes learn from how signals move around events: start-ups, upsets, alarms, grade changes and failures. Those moments are short, and they are exactly what aggressive compression and summary-only archives tend to flatten.

Historians are often configured to save storage, with deadbands and compression tuned for trend displays rather than analysis. Plants also tend to keep hourly or shift averages longer than raw values. Both choices were sensible for operations, but they leave gaps for anyone who wants to learn from the full sequence of an event.

AI developers working on process control, predictive maintenance and operator assistance all need this sequence-level detail. A summary row showing that a batch finished out of specification is far less useful than the stretch of signals that led there.

Keep, compress or discard: a decision table by signal type#

The decision table below sorts historian signals by how much value they lose under compression. Check regulatory, quality and customer retention requirements before discarding anything, since those set the floor regardless of AI value.

What makes historian data usable by someone else#

Historian data becomes usable by someone else when its context travels with it. A list of tag names and values means little to an AI developer without the information that explains what each tag measures and what the plant was doing at the time.

Most of this context already exists somewhere, in the historian configuration, the control system documentation, batch records and shift logs. Historians such as AVEVA PI, Proficy Historian, Canary and the historian modules built into SCADA platforms each store tag configuration and compression settings in their own way, so export that configuration alongside the values and check that it matches the period being shared.

• 1A tag dictionary with descriptions, engineering units, ranges and the asset each tag belongs to.
• 2Time zone and clock source, plus notes on daylight saving changes and known clock drift.
• 3Compression and sampling settings in force for each period, since they change what the data can show.
• 4Links to batch or run IDs, product codes, work orders and nonconformance records.
• 5Operator logs and shift notes that explain upsets, changes and interventions.
• 6Records of instrument replacements and calibrations that cause shifts in the data.

Who controls historian data in a plant?#

The plant operator usually controls historian data from its own equipment, but three situations need a closer look. Vendor-hosted historians and cloud data platforms run under subscription terms that govern export. Skids and machines supplied with remote monitoring may send data to the equipment maker under its own agreement. Toll processing and contract manufacturing customers may claim rights in process data from their products.

Map each source to the agreement that covers it before planning an archive or a license. A short table of historian, assets covered, hosting model and governing contract is enough to show where the open questions sit.

What to keep out of a historian license#

A historian license should keep out the parameters and details that are competitively or operationally sensitive: recipes, setpoints, tuning constants, safety instrumented system details, network addresses and operator identities. Removing them is usually possible without destroying the value of the remaining signals.

Process variables can often be shared as measured values while the targets that drove them stay private. Where customer-specific processes run under contract, check whether the customer's terms cover process data as well as product specifications. Security-sensitive details, such as control network layouts and system versions, stay out even when everything else is cleared.

How to protect history before a historian migration#

Protecting history before a historian migration means exporting raw archives and configuration first, while the old system and the people who know it are still available. A migration may move only tag configuration and a recent window of data, leaving older archives on a server due for decommissioning.

Ask the people who configured the system what they know about gaps, renamed tags and periods of instrument trouble. That knowledge is rarely written down, and it leaves with them. Extract from a historian replica on the business network or from copied archive files, never by opening a new path into the control network.

• 1Step 1: list every historian, SCADA archive and data collector, including retired ones still on servers.
• 2Step 2: export tag configuration, compression settings and the asset model.
• 3Step 3: copy raw archive files to storage the company controls, with checksums.
• 4Step 4: confirm the archive can be read outside the old system, or keep a working reader.
• 5Step 5: record date ranges, gaps and known instrument changes in a short archive note.

Illustrative: a coatings plant rethinks its historian settings#

Illustrative: a fictional specialty coatings plant runs batch reactors and dispersion mills. During a historian upgrade, the OT director finds that compression deadbands had flattened reactor temperature transients for years, while shift averages had been kept indefinitely.

The team changes the settings so reactor and mill signals are stored at full resolution during batch windows, keeps the alarm journal in full and compresses utility tags. It exports the old archive with its configuration before the previous server is decommissioned.

When the company later reviews licensing, it scopes alarm journals, operator logs and batch outcome records linked to process variables, with recipes, setpoints and customer-specific formulations removed. The flattened years are documented as lower resolution rather than hidden.

How SourceX approaches historian archives#

SourceX starts a historian review with metadata only: which historians and SCADA archives exist, how many years they cover, at what resolution in each period, and which batch records, alarm journals and work orders can be linked to them. No tag data is shared at that stage.

Packages then move through the SourceX five-step transaction of Supply, Rights, Preparation, Approval and Delivery, with recipes, setpoints and control-network details removed during Preparation. Historian archives are often large, so they are delivered from the company's own servers or on encrypted drives; SourceX never hosts multi-terabyte datasets. A SourceX Evidence Packet records provenance, licensing rights, permitted use, the privacy record and release authorization, and the archive note on resolution and gaps travels with it.