Skip to content

Systems and records

How to measure workflow history lost in a migration: a study protocol

By SourceX Editorial · Updated

10 sections

Short answer

A business data preservation loss study measures workflow history lost in a software migration by sampling records from the old system before cutover, counting defined history elements such as original dates, owners, status changes, comments and attachments, and checking the same elements in the new system and any archive afterward. Report preservation rates per element with intervals. This page is a study protocol; it reports no results.

Record timeline

Business data preservation

This page covers how to measure workflow history lost in a migration: a study protocol.Illustrative

Key takeaways

  • Measure preservation element by element, because a migrated record can keep its fields and still lose dates, owners, notes and status history.
  • Sample records from the source system before cutover; sampling from the target can never observe what was lost.
  • Every rate needs the source element count as its denominator, an interval and the number of migrations behind it.
  • Keep history preserved in the target separate from history that survives only in an archive or export.
  • This is a protocol only: no results are reported, and any future figure describes the migrations studied, not migrations in general.

How could a study measure workflow history lost during a migration?

Measure it element by element, not record by record. A migrated support ticket can arrive with its subject and requester intact and still lose its original creation date, its internal notes and the history of who changed its priority, so a simple count of migrated records says almost nothing about preserved history.

The design is a paired before and after audit. For a random sample of records, an auditor records which history elements existed in the source system before cutover, then looks for each element in the target system and in any archive or export after go-live. Each element gets one of three outcomes: preserved in the target, preserved only in an archive or export, or lost.

No results are reported here. Any figure in this page is part of the method or labeled illustrative, and the protocol is published so that a study run later can be checked against what it promised to measure.

The research question, and the conclusions the design cannot support

The primary question is: for completed migrations between business systems of the same category, what share of the history elements present in the source before cutover is retrievable afterward, in the target system or in a retained archive? Secondary questions compare those shares by element type, by record age and by migration method.

The design describes migrations that companies agree to share and that still have a readable source. It cannot support a claim that a named vendor's tool loses a given amount of history in general, because samples per tool will be small and self-selected. It cannot support causal claims about migration methods, because companies choose methods for reasons that also affect outcomes. And it cannot be extrapolated to all companies, since companies that kept their old system readable may be the more careful ones.

Stating these limits before collection matters because the most quotable sentence a study like this could produce, a single loss percentage for all migrations, is the one the design does not license.

Documented ways history gets dropped, which the element list must cover

Vendors document several mechanisms by which history fails to move, and each one becomes a named element in the audit. The documentation shows the mechanisms; it does not say how often they cause loss, which is what the study would measure.

Salesforce's help documentation says that, without Field Audit Trail, field history data is retained for up to 18 months, and up to 24 months through the API. Its knowledge articles describe a Set Audit Fields upon Record Creation permission that allows fields such as CreatedDate and CreatedById to be set when records are inserted, and only on insert, which is how a migration into Salesforce keeps original dates and authors at all.

Zendesk's Ticket Import API lets an importer set created_at, updated_at and solved_at on imported tickets, rounds created_at or updated_at values before 1970 up to 1970, and does not run triggers on an imported ticket whose status is not closed. Atlassian's page on what the Jira Cloud Migration Assistant moves lists items it does not migrate, including some custom field types and data from apps whose vendors provide no migration path.

  • Original timestamps overwritten by the import time unless the target allows them to be set.
  • Original authors and owners replaced by the migration user.
  • Field and status change history kept for a limited window, or not exported at all.
  • Comments, internal notes and attachments moved without their links to the parent record.
  • Custom fields and add-on data that the migration tool does not support.

The before and after preservation study: elements and outcome codes

This table is the core instrument. Each row is a history element with an operational definition precise enough for two auditors to code the same record the same way; the outcome codes are shared across all rows.

At a glance

The before and after preservation study: elements and outcome codes

The before and after preservation study: elements and outcome codes
History elementOperational definitionBefore cutover checkAfter go-live checkOutcome codes
Original creation dateCreation timestamp of the record in the source, to the dayRead from source record or exportCompare target creation field and any legacy-date fieldP preserved, A archive only, L lost, X not applicable
Original author or ownerUser who created or owned the record at closureSource user ID and nameTarget owner, mapped user or legacy-owner fieldP, A, L, X
Status or stage historyEvery status transition with its timestampCount of transitions in source historyCount of transitions retrievable after cutoverP if all, partial noted as count, A, L
Field change historyChanges to tracked fields with old and new valuesCount from source history tableCount in target history or archiveP, partial count, A, L
Comments and internal notesEvery comment with author, time and visibilityCount and visibility flagsCount, author and visibility in targetP, partial count, A, L
AttachmentsFiles linked to the recordFile count and namesFile count retrievable and still linkedP, A, L
Links between recordsParent, child, duplicate or related linksCount of linksLinks resolvable in targetP, partial count, A, L
Closed and archived recordsWhether the record was in scope at allRecord exists in sourceRecord exists in target or archiveP, A, L
Custom and add-on fieldsValues in fields outside the standard schemaCount of populated custom fieldsCount retrievable after cutoverP, partial count, A, L

Population, recruitment and sampling within each migration

The unit of recruitment is a migration; the unit of measurement is a history element within a sampled record. Recruit migrations completed within a stated window before the study starts, between systems of the same category, where the source system is still readable through a read-only instance, a full export or a backup.

Stratify recruitment by system category, for example CRM, help desk, issue tracker and ERP, and by migration method: the vendor's own tool, a consultant or integration script, or manual file import. Fix the strata and target counts in the pre-registration, and report how many migrations were approached, how many agreed and how many were excluded.

Within each migration, draw a simple random sample of records from the source, stratified by record age band so that old records are not crowded out by recent ones. Draw the sample from the source, never from the target, because sampling from what arrived can never observe what did not.

  • Inclusion: migration completed within the window, same system category, source still readable, owner consents to an audit.
  • Exclusion: partial migrations still running, sources already purged, systems where the auditor cannot see history tables.
  • Disclose: recruitment channel, response rate and reasons for exclusion.
  • Freeze: the sampling seed and record list before any target lookups.

Variables, denominators and comparisons

The denominator for every rate is the number of element instances present in the source before cutover, not the number of records migrated. Define it first, because a rate with the wrong denominator is the most common way a preservation figure misleads.

  • Migration-level variables: system category, source and target product, migration method, cutover date, archive retained or not, record count in scope.
  • Record-level variables: record type, creation year, status at cutover, number of history entries in the source.
  • Comparisons: element types against each other, record age bands, and migration methods, all reported as descriptive differences, not effects.
Variables, denominators and comparisons
MeasureNumeratorDenominatorReported by
Target preservation rateElement instances found in the targetElement instances present in the sourceElement, system category, method
Retrievable preservation rateInstances found in the target or a retained archiveElement instances present in the sourceElement, system category, method
Complete-history record rateSampled records with every applicable element preservedSampled recordsSystem category, record age band
Silent loss shareLost instances the migration owner did not expect to loseAll lost instancesElement

Uncertainty: intervals, clustering and coder agreement

Report every rate with an interval, the number of migrations and the number of element instances behind it. A rate from one large migration and a rate from many small ones can look the same and mean very different things.

For a single proportion, the NIST Engineering Statistics Handbook presents an interval based on Wilson's method and an exact binomial method as an alternative to the simple normal approximation. Because records within one migration share a tool and a team, intervals for pooled rates should also account for clustering by migration, for example with a cluster bootstrap that resamples whole migrations.

Coding is a measurement too. Have two auditors code a fixed subset of records independently, report their agreement per element, and resolve disagreements by rule rather than by discussion of the expected answer. Suppress or merge cells with too few migrations to support an interval, and say which ones.

Pre-registration checklist for a migration preservation study

Pre-registration means posting a time-stamped, read-only version of the study plan to a public repository before data collection begins, as the Center for Open Science describes it. For this study, the plan should be posted before the first migration is recruited, and any later change noted with its date and reason.

At a glance

Pre-registration checklist for a migration preservation study

Checklist · 10 items

  1. Primary and secondary questions, worded exactly.
  2. Recruitment window, strata, target counts and stopping rule.
  3. Element list and operational definitions, with the coding manual.
  4. Sampling method within migrations, age bands and the random seed procedure.
  5. Denominators and formulas for every rate.
  6. Interval methods, clustering approach and the minimum cell size for reporting.
  7. Coder training, the double-coded subset and the agreement statistic.
  8. Handling of partial outcomes, records with no applicable elements and unreadable sources.
  9. The comparisons that will be reported and those that will not.
  10. A commitment to publish the element-level tables, including null and unfavorable results.

0 of 10 ticked. Tick each item as you go.

Limits, and how to read a result if one is ever published

Read any eventual result as a description of the migrations that took part, not a forecast for yours. The sample will favor companies willing to be audited and able to read their old system, and both traits probably go with better preservation.

Archive-only preservation is a real outcome but a fragile one. An element that survives only in an export file is preserved for a records request and may not be usable for analysis or for building training episodes, so the study keeps the target and retrievable rates separate rather than adding them.

Product behavior changes between releases. Retention windows, import options and migration tool coverage cited here come from current vendor documentation and should be rechecked at the start of any study wave, with the documentation version recorded alongside the data.

Running the audit on your own migration before licensing old records

The same element table works as a one-off audit of a single migration: sample from the old system, code each element and see which history survives in the target and which only in an archive. That audit tells an owner what is still licensable before an old system is switched off.

SourceX turns chosen business records into licensed AI datasets, taking a company through rights review, preparation, contracting, delivery and payment. Because buyers usually look for at least two years of consistent history and tens of thousands of records in one system, whether dates, owners and status history survived a migration is a practical question, and SourceX weighs it company by company. It begins with a short assessment that needs no upload.

Frequently asked questions

What is a business data preservation loss study?

It is a study that measures how much of the history attached to business records, such as original dates, owners, status changes, comments and attachments, survives a move from one system to another. It compares the source before cutover with the target and any archive afterward, element by element, and reports preservation rates with their uncertainty.

Why sample from the old system rather than the new one?

Sampling from the target can only find what arrived, so it cannot observe anything that was lost. Drawing the sample from the source before cutover makes every missing element visible as a loss, which is the quantity the study exists to measure.

Does keeping an export count as preserving history?

It counts as retrievable, and the protocol reports it separately from history preserved in the target system. An export may satisfy a records request but lose links between records or be hard to analyze, so adding the two together would overstate what is usable.

How many migrations would a defensible study need?

Enough per stratum that each reported rate has an interval narrow enough to be useful once clustering by migration is accounted for. The number should be fixed in the pre-registration from a precision target, and cells that fall short should be merged or suppressed rather than reported.

Can I use this protocol on a single migration?

Yes. Run the element table on a random sample of records from your old system and check each element in the new system and any archive. You will get a description of your own migration, not a benchmark, and it shows which history you still need to export before the old system is retired.

Sources

  • Salesforce retains field history for up to 18 months, and up to 24 months through the API, when Field Audit Trail is not enabled. Source
  • Salesforce's Set Audit Fields upon Record Creation permission allows audit fields such as CreatedDate and CreatedById to be set on insert only. Source
  • Zendesk's Ticket Import API allows created_at, updated_at and solved_at to be set, rounds created_at or updated_at values before 1970 up to 1970, and does not run triggers on imported tickets that are not closed. Source
  • Atlassian lists data the Jira Cloud Migration Assistant does not migrate, including some custom field types and app data without a vendor migration path. Source
  • The NIST Engineering Statistics Handbook presents a Wilson-based interval for a single proportion and an exact binomial method as an alternative to the normal approximation. Source
  • The Center for Open Science describes preregistration as posting a time-stamped, read-only study plan to a public repository before data collection or analysis. Source

See if your company qualifies

A short company assessment. No data uploads are needed.

See if your company qualifies