Skip to content

Systems and records

Business archive completeness index: what to measure, a study protocol

By SourceX Editorial · Updated

10 sections

Short answer

A business archive completeness index should measure, against a written reference, whether every period, record type, source total, decision field, link, change history and attachment the systems held is present and provable for a declared period. This page is a study protocol, not a result: it defines seven components with numerators and denominators, the sampling, scoring and uncertainty rules, and the limits. No results are reported.

Retrieval

Business archive

This page covers business archive completeness index: what to measure, a study protocol.Illustrative

Key takeaways

  • This is a protocol page; no index results have been collected or reported.
  • Completeness is measured against a written reference of what each system should hold, not by impression.
  • Report seven component scores first; a single number must never hide a failed component.
  • A gap in document numbers is not a loss until a source report shows the missing record.
  • Volunteered archives describe themselves, not all businesses, so cross-archive figures carry no margin of sampling error.

What should a business archive completeness index measure?

A business archive completeness index should measure how much of what a company's systems were supposed to record is actually present, readable and linked for a declared period. This page is the protocol for such an index: seven components, how each is counted, how archives are sampled and how uncertainty is stated. It reports no results; none have been collected.

Completeness here is narrower than quality and broader than a file count. It asks whether every period, record type, decision field, link, change history and attachment that the source systems held is present in the archive, and whether a reader can prove it. An archive can be complete and still be messy; it can also look tidy and be missing years.

The audience is two groups. Business owners want to know whether an old system export or backup still holds the history they think it does. Journalists and researchers want a method they can inspect before they quote any figure about how complete business archives tend to be. Both need the method before the number.

Why must completeness be measured against a declared reference?

Completeness is only measurable against a reference that says what should be there, so every archive in the study gets a written reference before anything is counted. Without one, a missing year and a year with no activity look the same.

Records management law and archival standards already frame completeness this way. The U.S. federal electronic records rule at 36 CFR 1236.10 asks agencies for controls that ensure records are a full and accurate representation of the transactions they document, and integrity controls, such as audit trails, to ensure records are complete and unaltered. The OAIS reference model, CCSDS 650.0-M-3, groups the information needed to preserve content into provenance, context, reference, fixity and access rights information. The index borrows that structure for private business archives, which have no such obligations but face the same questions.

The reference for each archive lists the source systems, the period, the record types each system produced, the fields that carry decisions and outcomes, and the source totals that can be checked, such as document counts by month from the system's own reports.

Writing the reference is the step most likely to bias the index, because the company knows its systems best and also has an interest in the result. The protocol therefore requires each reference item to cite where it came from, such as a system report, a configuration screen or a vendor document, and marks items that rest only on someone's recollection.

  • Source systems and instances, with go-live and retirement dates.
  • Declared period, and the reason it was chosen.
  • Record types each system produced, from configuration or documentation.
  • Decision and outcome fields to be checked for fill, named per record type.
  • Source totals available for reconciliation, such as monthly document counts, with where each came from.
  • Known events that changed recording: migrations, purges, archive runs, module changes.
  • Items supported only by recollection, flagged for sensitivity analysis.

Which seven components make up the index, and how is each counted?

Each component is a ratio with a stated numerator and denominator, so two researchers counting the same archive should get the same answer. Components are reported separately; the combined index is secondary.

Which seven components make up the index, and how is each counted?
ComponentWhat it measuresNumeratorDenominator
Period coverageWhether every month in the declared period has recordsMonths with at least one record of the main typeMonths in the declared period where the business was operating
Record-type coverageWhether each record type the system produced is presentRecord types present in the archiveRecord types listed in the reference
Count reconciliationWhether record counts match the source system's own totalsPeriods where archive counts match source totals within tolerancePeriods with an available source total
Decision-field fillWhether fields carrying decisions and outcomes are populatedSampled records with all named decision fields filledSampled records
LinkageWhether records join to the related records they referenceSampled references that resolve to a record in the archiveSampled references
History and attachmentsWhether change history and attached files survivedSampled records whose history and attachments are present and openSampled records that had history or attachments in the source
Fixity and documentationWhether integrity can be proven and fields can be interpretedChecklist items met: checksums, manifest, schema, field dictionary, export logChecklist items

Which question, population and sampling plan does the study fix in advance?

The research question is fixed in advance: among archives of business system records offered for review by participating companies in a declared window, what share of each component's denominator is met? The unit is one archive of one source system, not one company, because one company can hold a complete ERP history and a broken help desk export.

At a glance

Which question, population and sampling plan does the study fix in advance

The population is defined by how archives arrive, which is the main limit on what the study can claim. If archives come from companies that chose to take part, the sample is not a probability sample of businesses and must not be described as one. The protocol records how each archive was recruited, its system type, the company's sector and size band, and the archive's period, so readers can see the composition.

Sampling inside each archive is random and documented. Record-level components such as decision-field fill and linkage are measured on a simple random sample of records drawn with a recorded seed, stratified by year so that thin early years are not swamped by recent ones.

  • Population: archives of named system types offered by participating companies during the collection window.
  • Unit of analysis: one archive from one source system covering a declared period.
  • Recruitment: how each archive entered the study, recorded per archive.
  • Collection dates: the window in which archives were received and measured, published with the results.
  • Within-archive sampling: simple random sample of records per year with a recorded seed.
  • Exclusions: archives that cannot be opened at all, counted and reported, not silently dropped.

How is each archive measured without false alarms?

Measure each component with a check that can fail for a reason the team can name. The common false alarms are gaps that are not losses and counts that are not comparable.

Gaps in document numbers are a weak signal on their own. Microsoft's documentation for Dynamics 365 notes that noncontinuous number sequences use numbers sequentially but may skip numbers, so a missing number can be normal. The protocol therefore counts a gap as a loss only when the source system's own report or log shows a record that the archive lacks.

Quick checks come before slow ones. The Library of Congress bagit-python tool, for example, offers a fast validation that compares a bag's payload-oxum, its byte count and number of files, before full checksum validation. The index uses the same order: file counts and byte totals against the export manifest first, then checksums, then record-level sampling.

Record-level checks are made by two measurers working independently on the same sampled records, with a written rule for what counts as filled, linked or present. Disagreements go to an adjudicator, and agreement is reported per component, so a reader can see where the definitions were hard to apply. Measurers do not see the company's name or sector while scoring sampled records, which limits expectations leaking into judgments about borderline fields.

The declared period is not chosen to flatter the archive. Where the period is set by retention practice, the reference cites it; for example, the IRS says business records generally should be kept for three years, longer in specific situations, and employment tax records for at least four years, so an archive claiming a tax-driven period can be checked against what the company says it kept.

Why is the index a profile of scores before it is a single number?

Publish the seven component scores as the primary result and treat any single index number as a summary that must never hide a failed component. A weighted average lets a strong count reconciliation mask a missing decade of attachments.

If a single number is reported, its construction is fixed in the pre-registration: each component scored from zero to one as its ratio, the composite set to the minimum component or a pre-declared weighted mean shown next to the minimum, and any archive with a component that could not be measured reported as not scored, not as zero.

Sensitivity checks sit beside the scores. Each archive is rescored with reference items that rest only on recollection removed, and with the count tolerance halved and doubled, so readers can see which components move when an assumption changes and which stay put.

Why is the index a profile of scores before it is a single number?
RuleWhat it preventsHow it is disclosed
Report all seven components for every archiveA composite hiding a failed componentComponent table published with the index
Weights fixed before measurementChoosing weights that favor a resultWeights in the dated pre-registration
Not measurable is not zeroPenalizing archives for missing reference totalsSeparate count of unmeasured components
Minimum shown beside any meanA high mean with one empty componentTwo columns in every summary table
Version every scoring changeSilent rescoring after publicationChange log with old and new scores

Which intervals suit record-level scores, and why not cross-archive figures?

A defensible result states the sample, the baseline and the uncertainty separately for record-level and archive-level components, because they rest on different kinds of sampling.

Record-level components are estimated from random samples inside each archive, so they can carry a confidence interval. The NIST/SEMATECH engineering statistics handbook gives the Wilson method for a confidence interval on a proportion rather than the simple normal approximation, so the protocol pre-declares the interval method and a minimum sample per stratum, and uses an exact binomial interval below that minimum.

Archive-level summaries across companies are a different matter. If archives come from volunteers, AAPOR's guidance on nonprobability samples applies by analogy: a margin of sampling error is not appropriate, so cross-archive figures are reported as descriptions of the archives measured, with their composition, never as estimates for all businesses. The baseline for any comparison, such as one system type against another, is fixed in advance, and differences are described with the archive counts behind them.

What would an index built this way still fail to show?

The design describes the completeness of the archives measured, as of the collection window, against references the companies helped write. Several tempting conclusions are outside it.

  • Not supported: a statement about how complete business archives are in general, unless archives were drawn by probability sampling from a defined frame.
  • Not supported: a value or price for any archive; completeness is one input to fit, not a valuation.
  • Not supported: legal conclusions about whether a company met a retention obligation.
  • Not supported: comparisons between vendors' systems, because archives differ in age, configuration and how they were exported.
  • Not supported: claims about records that were never created; the index measures what was recorded and kept, not what should have happened in the business.
  • Supported: a reproducible, component-level description of each measured archive, with stated references, samples and uncertainty.

What must be registered before the first archive is measured?

Register the protocol on a dated public page before the first archive is measured. The Center for Open Science describes preregistration as setting out the research plan before data collection so that planned, confirmatory analysis stays distinct from exploratory work, which can still be reported if it is labeled as such.

At a glance

What must be registered before the first archive is measured

  • Research question and unit of analysis.
  • Population definition, recruitment route and collection window.
  • Reference template: systems, period, record types, decision fields and source totals.
  • Component definitions with numerators and denominators.
  • Within-archive sampling method, sample size per stratum and random seed handling.
  • Interval method for record-level components and the minimum sample for the large-sample method.
  • Composite rule and weights, or a statement that no composite will be published.
  • Tolerance for count reconciliation and the rule for treating number gaps.
  • Exclusion rules and how excluded archives are reported.
  • Revision and correction policy, and conflicts of interest of the team and funders.

Should a business run the protocol on its own archive before licensing it?

A business owner can run a reduced version of this protocol on one system before any outside conversation: write the reference, compare monthly counts with the system's own reports, sample records for decision fields, links and attachments, and note what could not be checked. The result is a component table, not a grade, and it shows where to spend restore or cleanup effort first.

That table also helps when the archive might be licensed. If the owner later licenses the archive through SourceX, which runs qualification, rights review, contracting, preparation, delivery and payment, the fit check looks at much the same evidence: as a general guide, two or more years of consistent history and tens of thousands of records or more in one system, with rarer work sometimes qualifying with less, judged deal by deal. That check begins with a brief questionnaire about the company, not a data transfer. Completeness and the right to license are separate questions, and counsel reviews the second.

Frequently asked questions

What should an original index methodology with transparent limitations include?

A fixed research question, the unit of analysis, the population and how it was recruited, component definitions with numerators and denominators, the sampling method, the interval method, the composite rule if any, exclusions and a list of conclusions the design cannot support. Publish it before any results.

Is a business archive complete if the file count matches?

Not necessarily. A matching file count shows the export arrived intact, but records can still be missing inside files, decision fields can be empty and links or attachments can be broken. File counts are the first check, not the last.

Does a gap in invoice or order numbers mean records are missing?

Not on its own. Some systems use number sequences that skip numbers by design. Treat a gap as a loss only when the source system's report or log shows a record the archive does not have.

Can the index say how complete most companies' archives are?

Only if archives are drawn by probability sampling from a defined frame of businesses. If companies volunteer their archives, results describe the archives measured and should not carry a margin of sampling error.

Has this index been run, and are there results?

No. This page publishes the protocol only. Any future results would be reported with their collection window, sample composition and uncertainty, separately from this method.

Sources

  • 36 CFR 1236.10 asks for controls ensuring records fully and accurately represent the transactions they document, and integrity controls such as audit trails ensuring records are complete and unaltered. Source
  • The OAIS reference model (CCSDS 650.0-M-3) groups preservation description information into provenance, context, reference, fixity and access rights information. Source
  • The IRS says to keep records for three years in general, longer in listed situations, and employment tax records for at least four years. Source
  • Dynamics 365 noncontinuous number sequences use numbers sequentially but may skip numbers. Source
  • The Library of Congress bagit-python tool offers a fast validation that compares a bag's payload-oxum, its byte count and number of files, before full checksum validation. Source
  • The NIST/SEMATECH e-Handbook gives the Wilson method for a confidence interval on a proportion. Source
  • AAPOR states that statistically valid margins of sampling error cannot be developed for nonprobability samples such as opt-in polls. Source
  • Preregistration sets out the research plan before data collection and keeps confirmatory analysis distinct from exploratory analysis. Source

See if your company qualifies

A short company assessment. No data uploads are needed.

See if your company qualifies