Systems and records
Business archive completeness index: what to measure, a study protocol
By SourceX Editorial · Updated
10 sections
Short answer
A business archive completeness index should measure, against a written reference, whether every period, record type, source total, decision field, link, change history and attachment the systems held is present and provable for a declared period. This page is a study protocol, not a result: it defines seven components with numerators and denominators, the sampling, scoring and uncertainty rules, and the limits. No results are reported.
Retrieval
Business archive
Key takeaways
- This is a protocol page; no index results have been collected or reported.
- Completeness is measured against a written reference of what each system should hold, not by impression.
- Report seven component scores first; a single number must never hide a failed component.
- A gap in document numbers is not a loss until a source report shows the missing record.
- Volunteered archives describe themselves, not all businesses, so cross-archive figures carry no margin of sampling error.
What should a business archive completeness index measure?
A business archive completeness index should measure how much of what a company's systems were supposed to record is actually present, readable and linked for a declared period. This page is the protocol for such an index: seven components, how each is counted, how archives are sampled and how uncertainty is stated. It reports no results; none have been collected.
Completeness here is narrower than quality and broader than a file count. It asks whether every period, record type, decision field, link, change history and attachment that the source systems held is present in the archive, and whether a reader can prove it. An archive can be complete and still be messy; it can also look tidy and be missing years.
The audience is two groups. Business owners want to know whether an old system export or backup still holds the history they think it does. Journalists and researchers want a method they can inspect before they quote any figure about how complete business archives tend to be. Both need the method before the number.
Why must completeness be measured against a declared reference?
Completeness is only measurable against a reference that says what should be there, so every archive in the study gets a written reference before anything is counted. Without one, a missing year and a year with no activity look the same.
Records management law and archival standards already frame completeness this way. The U.S. federal electronic records rule at 36 CFR 1236.10 asks agencies for controls that ensure records are a full and accurate representation of the transactions they document, and integrity controls, such as audit trails, to ensure records are complete and unaltered. The OAIS reference model, CCSDS 650.0-M-3, groups the information needed to preserve content into provenance, context, reference, fixity and access rights information. The index borrows that structure for private business archives, which have no such obligations but face the same questions.
The reference for each archive lists the source systems, the period, the record types each system produced, the fields that carry decisions and outcomes, and the source totals that can be checked, such as document counts by month from the system's own reports.
Writing the reference is the step most likely to bias the index, because the company knows its systems best and also has an interest in the result. The protocol therefore requires each reference item to cite where it came from, such as a system report, a configuration screen or a vendor document, and marks items that rest only on someone's recollection.
- Source systems and instances, with go-live and retirement dates.
- Declared period, and the reason it was chosen.
- Record types each system produced, from configuration or documentation.
- Decision and outcome fields to be checked for fill, named per record type.
- Source totals available for reconciliation, such as monthly document counts, with where each came from.
- Known events that changed recording: migrations, purges, archive runs, module changes.
- Items supported only by recollection, flagged for sensitivity analysis.
Which seven components make up the index, and how is each counted?
Each component is a ratio with a stated numerator and denominator, so two researchers counting the same archive should get the same answer. Components are reported separately; the combined index is secondary.
| Component | What it measures | Numerator | Denominator |
|---|---|---|---|
| Period coverage | Whether every month in the declared period has records | Months with at least one record of the main type | Months in the declared period where the business was operating |
| Record-type coverage | Whether each record type the system produced is present | Record types present in the archive | Record types listed in the reference |
| Count reconciliation | Whether record counts match the source system's own totals | Periods where archive counts match source totals within tolerance | Periods with an available source total |
| Decision-field fill | Whether fields carrying decisions and outcomes are populated | Sampled records with all named decision fields filled | Sampled records |
| Linkage | Whether records join to the related records they reference | Sampled references that resolve to a record in the archive | Sampled references |
| History and attachments | Whether change history and attached files survived | Sampled records whose history and attachments are present and open | Sampled records that had history or attachments in the source |
| Fixity and documentation | Whether integrity can be proven and fields can be interpreted | Checklist items met: checksums, manifest, schema, field dictionary, export log | Checklist items |
Which question, population and sampling plan does the study fix in advance?
The research question is fixed in advance: among archives of business system records offered for review by participating companies in a declared window, what share of each component's denominator is met? The unit is one archive of one source system, not one company, because one company can hold a complete ERP history and a broken help desk export.
At a glance
Which question, population and sampling plan does the study fix in advance
The population is defined by how archives arrive, which is the main limit on what the study can claim. If archives come from companies that chose to take part, the sample is not a probability sample of businesses and must not be described as one. The protocol records how each archive was recruited, its system type, the company's sector and size band, and the archive's period, so readers can see the composition.
Sampling inside each archive is random and documented. Record-level components such as decision-field fill and linkage are measured on a simple random sample of records drawn with a recorded seed, stratified by year so that thin early years are not swamped by recent ones.
- Population: archives of named system types offered by participating companies during the collection window.
- Unit of analysis: one archive from one source system covering a declared period.
- Recruitment: how each archive entered the study, recorded per archive.
- Collection dates: the window in which archives were received and measured, published with the results.
- Within-archive sampling: simple random sample of records per year with a recorded seed.
- Exclusions: archives that cannot be opened at all, counted and reported, not silently dropped.
How is each archive measured without false alarms?
Measure each component with a check that can fail for a reason the team can name. The common false alarms are gaps that are not losses and counts that are not comparable.
Gaps in document numbers are a weak signal on their own. Microsoft's documentation for Dynamics 365 notes that noncontinuous number sequences use numbers sequentially but may skip numbers, so a missing number can be normal. The protocol therefore counts a gap as a loss only when the source system's own report or log shows a record that the archive lacks.
Quick checks come before slow ones. The Library of Congress bagit-python tool, for example, offers a fast validation that compares a bag's payload-oxum, its byte count and number of files, before full checksum validation. The index uses the same order: file counts and byte totals against the export manifest first, then checksums, then record-level sampling.
Record-level checks are made by two measurers working independently on the same sampled records, with a written rule for what counts as filled, linked or present. Disagreements go to an adjudicator, and agreement is reported per component, so a reader can see where the definitions were hard to apply. Measurers do not see the company's name or sector while scoring sampled records, which limits expectations leaking into judgments about borderline fields.
The declared period is not chosen to flatter the archive. Where the period is set by retention practice, the reference cites it; for example, the IRS says business records generally should be kept for three years, longer in specific situations, and employment tax records for at least four years, so an archive claiming a tax-driven period can be checked against what the company says it kept.
Why is the index a profile of scores before it is a single number?
Publish the seven component scores as the primary result and treat any single index number as a summary that must never hide a failed component. A weighted average lets a strong count reconciliation mask a missing decade of attachments.
If a single number is reported, its construction is fixed in the pre-registration: each component scored from zero to one as its ratio, the composite set to the minimum component or a pre-declared weighted mean shown next to the minimum, and any archive with a component that could not be measured reported as not scored, not as zero.
Sensitivity checks sit beside the scores. Each archive is rescored with reference items that rest only on recollection removed, and with the count tolerance halved and doubled, so readers can see which components move when an assumption changes and which stay put.
| Rule | What it prevents | How it is disclosed |
|---|---|---|
| Report all seven components for every archive | A composite hiding a failed component | Component table published with the index |
| Weights fixed before measurement | Choosing weights that favor a result | Weights in the dated pre-registration |
| Not measurable is not zero | Penalizing archives for missing reference totals | Separate count of unmeasured components |
| Minimum shown beside any mean | A high mean with one empty component | Two columns in every summary table |
| Version every scoring change | Silent rescoring after publication | Change log with old and new scores |
Which intervals suit record-level scores, and why not cross-archive figures?
A defensible result states the sample, the baseline and the uncertainty separately for record-level and archive-level components, because they rest on different kinds of sampling.
Record-level components are estimated from random samples inside each archive, so they can carry a confidence interval. The NIST/SEMATECH engineering statistics handbook gives the Wilson method for a confidence interval on a proportion rather than the simple normal approximation, so the protocol pre-declares the interval method and a minimum sample per stratum, and uses an exact binomial interval below that minimum.
Archive-level summaries across companies are a different matter. If archives come from volunteers, AAPOR's guidance on nonprobability samples applies by analogy: a margin of sampling error is not appropriate, so cross-archive figures are reported as descriptions of the archives measured, with their composition, never as estimates for all businesses. The baseline for any comparison, such as one system type against another, is fixed in advance, and differences are described with the archive counts behind them.
What would an index built this way still fail to show?
The design describes the completeness of the archives measured, as of the collection window, against references the companies helped write. Several tempting conclusions are outside it.
- Not supported: a statement about how complete business archives are in general, unless archives were drawn by probability sampling from a defined frame.
- Not supported: a value or price for any archive; completeness is one input to fit, not a valuation.
- Not supported: legal conclusions about whether a company met a retention obligation.
- Not supported: comparisons between vendors' systems, because archives differ in age, configuration and how they were exported.
- Not supported: claims about records that were never created; the index measures what was recorded and kept, not what should have happened in the business.
- Supported: a reproducible, component-level description of each measured archive, with stated references, samples and uncertainty.
What must be registered before the first archive is measured?
Register the protocol on a dated public page before the first archive is measured. The Center for Open Science describes preregistration as setting out the research plan before data collection so that planned, confirmatory analysis stays distinct from exploratory work, which can still be reported if it is labeled as such.
At a glance
What must be registered before the first archive is measured
- Research question and unit of analysis.
- Population definition, recruitment route and collection window.
- Reference template: systems, period, record types, decision fields and source totals.
- Component definitions with numerators and denominators.
- Within-archive sampling method, sample size per stratum and random seed handling.
- Interval method for record-level components and the minimum sample for the large-sample method.
- Composite rule and weights, or a statement that no composite will be published.
- Tolerance for count reconciliation and the rule for treating number gaps.
- Exclusion rules and how excluded archives are reported.
- Revision and correction policy, and conflicts of interest of the team and funders.
Should a business run the protocol on its own archive before licensing it?
A business owner can run a reduced version of this protocol on one system before any outside conversation: write the reference, compare monthly counts with the system's own reports, sample records for decision fields, links and attachments, and note what could not be checked. The result is a component table, not a grade, and it shows where to spend restore or cleanup effort first.
That table also helps when the archive might be licensed. If the owner later licenses the archive through SourceX, which runs qualification, rights review, contracting, preparation, delivery and payment, the fit check looks at much the same evidence: as a general guide, two or more years of consistent history and tens of thousands of records or more in one system, with rarer work sometimes qualifying with less, judged deal by deal. That check begins with a brief questionnaire about the company, not a data transfer. Completeness and the right to license are separate questions, and counsel reviews the second.
Frequently asked questions
What should an original index methodology with transparent limitations include?
A fixed research question, the unit of analysis, the population and how it was recruited, component definitions with numerators and denominators, the sampling method, the interval method, the composite rule if any, exclusions and a list of conclusions the design cannot support. Publish it before any results.
Is a business archive complete if the file count matches?
Not necessarily. A matching file count shows the export arrived intact, but records can still be missing inside files, decision fields can be empty and links or attachments can be broken. File counts are the first check, not the last.
Does a gap in invoice or order numbers mean records are missing?
Not on its own. Some systems use number sequences that skip numbers by design. Treat a gap as a loss only when the source system's report or log shows a record the archive does not have.
Can the index say how complete most companies' archives are?
Only if archives are drawn by probability sampling from a defined frame of businesses. If companies volunteer their archives, results describe the archives measured and should not carry a margin of sampling error.
Has this index been run, and are there results?
No. This page publishes the protocol only. Any future results would be reported with their collection window, sample composition and uncertainty, separately from this method.
Sources
- 36 CFR 1236.10 asks for controls ensuring records fully and accurately represent the transactions they document, and integrity controls such as audit trails ensuring records are complete and unaltered. Source
- The OAIS reference model (CCSDS 650.0-M-3) groups preservation description information into provenance, context, reference, fixity and access rights information. Source
- The IRS says to keep records for three years in general, longer in listed situations, and employment tax records for at least four years. Source
- Dynamics 365 noncontinuous number sequences use numbers sequentially but may skip numbers. Source
- The Library of Congress bagit-python tool offers a fast validation that compares a bag's payload-oxum, its byte count and number of files, before full checksum validation. Source
- The NIST/SEMATECH e-Handbook gives the Wilson method for a confidence interval on a proportion. Source
- AAPOR states that statistically valid margins of sampling error cannot be developed for nonprobability samples such as opt-in polls. Source
- Preregistration sets out the research plan before data collection and keeps confirmatory analysis distinct from exploratory analysis. Source
See if your company qualifies
A short company assessment. No data uploads are needed.