Skip to content

Systems and records

Contract version reconciliation dataset: testing the wrong-version deal

By SourceX Editorial · Reviewed by Noah Loul ·

10 sections

Short answer

To test a deal workflow that used the wrong contract version, give it the agreement's full version lineage, the version the team relied on and the decision date. Ask it to name the controlling executed version, list only material differences and identify the decision they change. Score each answer against signing events and downstream records, and exclude privileged material.

License terms

Contract version

This page covers contract version reconciliation dataset: testing the wrong-version deal.Illustrative

Key takeaways

  • The controlling text is the executed version plus executed amendments, read through the agreement's own precedence clause, never simply the newest file.
  • Record lineage as explicit parent links between hashed, immutable versions with a state and an effective date.
  • Comparison tools find differences; a recorded human label decides which ones are material to the decision under test.
  • Every packet needs a starting state, authority rule, change register, expected answer, outcome check and exclusion record.
  • Score controlling-version accuracy, material change recall, noise, decision linkage and abstention separately.

How do you test a deal workflow that used the wrong contract version?

Give the workflow the full version history of one agreement, the record of what the deal team actually relied on, and a decision date, then ask it to name the controlling version, list the material differences and say which downstream decision those differences change. Score it against linked evidence, not against how confident its final message sounds.

That is the core of a contract version reconciliation dataset: each task is a frozen starting state, a set of documents with known lineage, an applicable rule about which document controls, and a verified outcome. The hard part is not the comparison itself. It is proving which file was executed, which was a draft circulated afterward, and whether the difference a reviewer spotted mattered to anything the business did next.

The rest of this page builds that task: how to establish authority, how to record lineage, how to separate material changes from noise, what the evaluation packet contains, a worked illustrative case with a counterexample, how to keep privileged material out, and how to score the result.

Which document controls on the decision date?

The controlling document is the executed version plus every executed amendment effective on or before the decision date, read through the agreement's own order-of-precedence and amendment clauses. The most recent file in a shared drive, the last redline in an email thread and the version attached to the CRM opportunity are all candidates to test, not answers.

At a glance

Which document controls on the decision date

Execution evidence should come from the signing system, not from file names. Docusign, for example, describes a Certificate of Completion that records the envelope ID, signer events and timestamped envelope events. Its help pages say a correction is recorded in both the envelope history, which notes who started it, what changed and when it finished, and the certificate, as an Envelope Updated event. That is exactly the trace that reveals a document changed while it was out for signature.

Write the authority rule into each task in plain words before anyone labels it. A typical rule reads: the executed master agreement controls, an executed amendment overrides the clauses it names from its effective date, an order form controls pricing and quantities only, and unsigned drafts never control. If the agreement says something different, the task follows the agreement and the rule is rewritten for that task.

  • Executed agreement: signature evidence from the signing platform or a wet-ink scan logged by the contracts team.
  • Amendments and change orders: each with its own signature evidence and effective date.
  • Precedence clause: the sentence that says which document wins when two conflict.
  • Relied-on version: the file the deal team opened, quoted or attached when it made the decision under test.
  • Decision record: the approval, quote, invoice run or system change that used the relied-on version.

Where does a superseded version usually slip into a deal?

Most wrong-version errors enter through a copy made for convenience: an attachment saved to a CRM record before signing, a clean copy rebuilt from an older redline, or a template refreshed after the counterparty's edits were accepted. Each entry point leaves a different trace, so the task should record which one applies.

Knowing the entry point changes what the workflow must check. A stale CRM attachment is caught by comparing its hash with the executed file. A rebuilt clean copy is caught only by lineage, because its text can look finished and carries no signature events. A missed amendment is caught by searching for executed documents that modify the agreement after its effective date.

  • Pre-signature attachment: the version uploaded to the opportunity or account record while terms were still moving.
  • Rebuilt clean copy: a draft regenerated from an earlier redline after signature, often without the counterparty's last change.
  • Missed amendment: an executed change stored with email or finance records instead of with the agreement.
  • Template drift: a renewal or order form generated from a template that predates negotiated terms.
  • Wrong counterparty entity: the right text for an affiliate or predecessor entity, not the party actually bound.

How should the version lineage be recorded?

Record lineage as explicit parent links between immutable file versions, each with a state and a date, so a reviewer can walk from any draft to the executed text without guessing. Do not infer lineage from file names such as final_v3_clean; give every file a content hash and a version ID instead.

The W3C PROV ontology offers a vocabulary that fits: an entity can be marked as a revision of an earlier entity with prov:wasRevisionOf, which the specification treats as a more specific kind of derivation than prov:wasDerivedFrom. Using a standard term matters less than using one term consistently, but borrowing PROV keeps the lineage readable by tools that already understand provenance graphs.

Two links are easy to miss. A draft circulated after signature is a revision of the executed text, not a new parent of it, and it must carry a state of post-execution draft. An amendment is not a revision of the master agreement at all; it is a separate executed entity that modifies named clauses, so it gets its own node with a modifies link.

How should the version lineage be recorded?
Lineage fieldWhat it holdsWhere it comes fromCommon error
version_id and sha256Stable ID and hash of the exact fileComputed at intakeTwo different files sharing one name
parent_version_idThe version this one revisesDocument management history or redline metadataLinking by modified date instead of content
stateDraft, sent for signature, executed, superseded, post-execution draftSigning platform events and contracts logTreating the last sent file as executed
effective_dateDate the version or amendment takes effectThe document text, checked against signature datesUsing the signature date when the text sets another date
modifiesClauses an amendment changesAmendment textApplying an amendment to the whole agreement
relied_on_byDecision records that used this versionApproval, quote or system change logsNo link, so the error cannot be shown

Which differences between versions are material, and which are noise?

A difference is material for the task when it changes an obligation, a date, a right, a quantity or a condition that the decision under test depended on. Formatting, numbering, typo corrections and added signature blocks are noise unless they move meaning.

Comparison tools find differences; they do not judge them. Word's legal blackline comparison, for instance, produces a third document showing the revised file's changes as tracked changes against the original, and lets the user compare at character or word level. That output is the raw material for labeling, and the label is a human judgment recorded with the clause it applies to.

Securities rules show the same line drawn in practice. Item 601 of Regulation S-K requires amendments to previously filed exhibits to be filed, yet says a registrant need not refile an exhibit amended only to correct typographical errors, insert signatures or make similar immaterial changes. That is a useful default for labeling: typographical and signature-only changes are immaterial unless the reviewer records why one is not.

For coverage, the Contract Understanding Atticus Dataset (CUAD) lists 41 expert-defined clause categories, including Effective Date, Expiration Date, Renewal Term, Notice Period to Terminate Renewal, Anti-Assignment and Change of Control. Those categories make a practical checklist of clause types a material change label should be able to name.

Which differences between versions are material, and which are noise?
Change classExampleDefault labelWho confirms
Dates and termEffective date moved, renewal term shortenedMaterialContracts manager
Notice and terminationNotice period to end renewal changed from 60 to 90 daysMaterialContracts manager, counsel if disputed
Scope and quantitiesService level target or unit count changedMaterial when the decision used itDeal owner
Rights and restrictionsData-use, assignment or exclusivity clause addedMaterialCounsel
Formatting and numberingSection renumbered with no text changeImmaterialLabeler, spot-checked
Typos and signaturesSpelling fixed, signature block addedImmaterial unless meaning movesLabeler, spot-checked

What should the version lineage and material change evaluation packet include?

Each task packet holds enough evidence for an independent reviewer to reach the same answer without asking anyone. If a reviewer needs to email the deal team to understand the task, the packet is not finished.

  • Version and date the packet itself, so later corrections create a new packet version rather than silently changing the answer.
  • Keep the relied-on version even when it is wrong; it is the most informative file in the task.
  • Store the authority rule as text, not as a flag, so a later reviewer can disagree with it on the record.
What should the version lineage and material change evaluation packet include?
Packet componentRequired contentEvidence that proves itFailure it catches
Starting stateDecision date, decision under test, the version the team relied onApproval or system change record with timestampTesting against hindsight instead of what was known
Version setEvery version and amendment with ID, hash, state and parentSigning platform events, document historyMissing amendment or post-signature draft
Authority rulePlain statement of which document controls and whyPrecedence and amendment clauses, quotedWorkflow picks the newest file
Material change registerEach difference with clause, change class, label and reviewerBlackline output plus labeler decisionNoise flagged as material, or the reverse
Expected answerControlling version ID, material changes, affected decisionReviewer sign-off with datePlausible but wrong final message
Outcome checkWhat happened downstream and whether it followed the controlling termsInvoice, notice, renewal or delivery recordStatus field read as proof of outcome
Exclusion recordWhat was removed and why: privileged, personal, out of scopeCounsel screen logLeaked privileged or personal content
Real or illustrative flagWhether the case is a real cleared record or constructedRights review noteIllustrative case presented as real

How does an illustrative notice-period change after signature play out?

This case is constructed for illustration; it is not a real client record. A services agreement went through four versions. Version 2 was sent for signature; during signing the envelope was corrected and the counterparty's change to the notice clause was accepted, producing version 3, which both parties executed. Two weeks later a paralegal circulated a clean copy built from version 2, saved as version 4 in the deal folder.

Starting state: 11 months later the account team checked the renewal window using version 4, which said 60 days' notice to stop renewal. The executed version 3 said 90 days. The team scheduled its non-renewal notice for day 70 before term end.

Evidence: the signing platform's history shows an Envelope Updated event and completion on version 3; the hash of the executed PDF matches version 3; version 4 has version 2 as its parent and no signature events. The blackline between versions 3 and 4 shows two differences: the notice period and a corrected spelling in the definitions.

Applicable rule and decision: the executed agreement controls, so the controlling version is 3. The notice-period difference is material because the non-renewal decision depended on it; the spelling fix is immaterial. Expected answer: controlling version 3, one material change, notice sent 20 days after the contractual deadline of day 90.

Outcome check: the packet links the sent notice and the counterparty's reply treating the agreement as renewed. That downstream record, not a CRM stage reading closed-lost, is what proves the wrong version had an effect.

  • Counterexample: in a second packet the post-signature copy differs from the executed text only in a corrected spelling and an added signature line. The expected answer is no material change, and a workflow that flags it as a version conflict fails the task.
  • Exception: if the executed copy cannot be identified because the signing record is missing, the expected answer is to stop and report insufficient evidence, not to pick the latest file.

How do you keep privileged and confidential material out of the dataset?

Build tasks from nonprivileged process evidence that counsel has screened: executed agreements, signing events, approval records and downstream system changes. Leave out legal advice, internal counsel commentary on redlines and anything a privilege review has not cleared.

The reason is practical. Federal Rule of Evidence 502(b) protects against waiver after an inadvertent disclosure only when the holder took reasonable steps to prevent it and promptly took reasonable steps to fix it. A dataset assembled without a documented screen makes both conditions harder to show, so the screen log belongs in the packet as its exclusion record.

Executed commercial agreements are usually confidential even when they are not privileged, and they contain personal data in signature blocks, notices sections and contact schedules. Confidentiality clauses and the counterparty's rights need their own review before any agreement leaves the company. Where a team wants illustrative cases without that exposure, public filings are one option: Item 601 requires certain material contracts and their amendments to be filed as exhibits, which gives versioned agreements whose public status is easy to show. Constructed cases, labeled as such, are the other option.

How should a reconciliation task be scored?

Score the parts a reviewer can check separately: whether the controlling version is right, whether the material changes are found without noise, whether the affected decision is named, and whether the workflow stops when evidence is missing. A single pass or fail hides which of those broke.

Report every score with the number of tasks behind it and an interval, and break results out by change class. A workflow that finds date changes reliably but misses added restrictions is not ready for rights questions, whatever its average says. No results are reported on this page; the metrics below are the design.

How should a reconciliation task be scored?
MetricDefinitionPassing evidenceTypical failure
Controlling version accuracyShare of tasks where the named version ID matches the reviewer'sVersion ID plus the signing event that proves executionNewest file chosen
Material change recallShare of labeled material changes the workflow foundClause reference for each changeAdded restriction missed
Noise rateShare of flagged changes the reviewer labeled immaterialLow count on counterexample packetsRenumbering flagged as a conflict
Decision linkageShare of tasks where the affected downstream decision is named correctlyLink to the decision recordCorrect diff, wrong consequence
Appropriate abstentionShare of insufficient-evidence tasks where the workflow stoppedStated missing evidenceGuessing the latest file

Can cleared contract version histories be licensed?

Contract histories with signing events, amendments and linked downstream decisions are the kind of operational record that can become a licensed dataset once rights, confidentiality and privilege questions are answered. A contracts team that wants to license such histories can hand SourceX the qualification, rights review, contract, preparation and delivery steps; the opening step is a brief company assessment with no agreements uploaded.

The company signs off on scope and terms and signs its own license before any file leaves, and the de-identified result may then be used only within that license. Because SourceX, through approved processing routes, performs the de-identification, signature blocks and notice contacts in the executed agreements are part of what it removes.

Frequently asked questions

What is a contract version reconciliation dataset?

It is a set of tasks, each built from one agreement's version history, the version a team relied on, a decision date and a verified outcome. The model or workflow must name the controlling version, find the material differences and identify the decision they affect. Each answer is checked against linked evidence such as signing events and downstream records.

Is the latest version of a contract always the one that controls?

No. The executed version plus executed amendments controls, read through the agreement's precedence clause. Drafts circulated after signature are often the newest files in a folder and still have no legal effect, which is why lineage and signature evidence belong in every task.

What if an amendment or the signing record is missing from the packet?

The executed agreement and any executed amendment effective on the decision date control, as the agreement's own clauses order them. When the signing record or an amendment is missing, the correct task answer is to stop and report the gap. Counsel confirms the authority rule for any disputed case.

What record shows the team acted on the wrong version, and what followed?

The decision is proved by the approval, quote, notice or system change that used the relied-on version, with its timestamp. The outcome is proved by what happened next, such as the counterparty's response, an invoice run or a renewal, not by a CRM status field or a confident summary.

Can real client contracts be used to build these tasks?

Only after rights, confidentiality and privilege review. Executed agreements are usually confidential and contain personal data, and legal advice must be excluded. Public filings and clearly labeled constructed cases are alternatives when cleared real records are not available.

Sources

  • Docusign's Certificate of Completion records the envelope ID, signer events and timestamped envelope summary events, with an Envelope Updated event for each correction. Source
  • Docusign records an envelope correction in the envelope history, including who started it, what changed and when it finished, and in the Certificate of Completion. Source
  • The W3C PROV ontology defines prov:wasRevisionOf as a more specific kind of prov:wasDerivedFrom for an entity revised from an earlier one. Source
  • Word's legal blackline comparison shows a revised document's differences from the original as tracked changes in a third document, at character or word level. Source
  • Regulation S-K Item 601 requires material contracts and amendments to filed exhibits to be filed, except amendments that only fix typographical errors, add signatures or make similar immaterial changes. Source
  • The CUAD category list defines 41 clause categories for contract review, including Renewal Term and Notice Period to Terminate Renewal. Source
  • Federal Rule of Evidence 502(b) limits waiver after inadvertent disclosure when the holder took reasonable steps to prevent it and promptly took reasonable steps to rectify it. Source

See if your company qualifies

A short company assessment. No data uploads are needed.

See if your company qualifies