Skip to content

AI uses for records

Human agent collaboration dataset: keep human edits when agents resume

By SourceX Editorial · Updated

9 sections

Short answer

A trajectory preserves human edits when it records three artifact versions, not one: the base the agent last saw, the version the human left, and what the agent wrote on resuming. Each human edit carries its span, author, authority and intent. The agent reconciles against the human's version, writes only if that version is still current, and a verifier confirms every human edit survives in the final artifact.

Agent run

Human agent collaboration

This page covers human agent collaboration dataset: keep human edits when agents resume.Illustrative

Key takeaways

  • Record three artifact versions per pause: base, human and final, linked by hashes.
  • An agent checkpoint restores the agent's state, not the shared document; re-read before writing.
  • Give every human edit a span, author, intent and an authority rule that defaults to locked.
  • Reconcile three ways and write conditionally so a stale copy cannot overwrite newer work.
  • Verify per edit; a done status or a polished final draft proves nothing about kept edits.

How should an agent trajectory preserve human edits made before it resumes?

Treat the human's edits as new ground truth that the agent must read, respect and prove it kept. The trajectory should store the artifact version the agent last worked from, the version the human produced while the agent was paused, the intent behind each human change, and the agent's write on resumption, with a check that it was made against the human's version rather than the agent's stale copy.

A useful human agent collaboration dataset makes those versions and checks explicit. Without them, a trajectory that ends with a polished document and a status of done can hide the most common collaboration failure: the agent resumed from its own memory and quietly overwrote what the person changed.

This guide gives a collaborative artifact version and intent task packet, the field-level record for one pause and resume cycle, plus failure modes, reconciliation rules, an illustrative worked case with a counterexample and the evidence that proves the outcome.

Why the agent's checkpoint is not the shared artifact

Agent frameworks persist the agent's state, not the world's. In LangGraph, a checkpointer saves the graph state so a run can resume later, and the thread_id tells it which saved state to load. When a run paused by an interrupt resumes, the node restarts from its beginning, so any code before the interrupt runs again.

That design is correct for the agent, but it means a document, spreadsheet or ticket that a person edited during the pause is not in the checkpoint unless the agent reads it again. If the node re-runs a step that regenerates content from the saved state, the person's work can be replaced by the agent's earlier draft without any error being raised.

LangGraph's time travel feature shows the same principle from the other side: update_state does not roll a thread back; it creates a new checkpoint that branches from the chosen point and leaves the original history intact. A dataset should keep history the same way, as branches with parents, so a reader can see which version each step was built on.

Research environments are beginning to treat this as a first-class problem. Collaborative Gym, from Stanford's SALT lab, lets humans and agents both take actions in a shared workspace and expects the human to intervene in real time rather than waiting for the agent to finish.

Six ways a resumed agent loses or distorts a human's edit

Each pattern below can produce a final artifact that looks complete. Name the pattern in the task packet when it occurs, because a model trained on the trajectory needs to learn the failure, not just the polished end state.

Six ways a resumed agent loses or distorts a human's edit
FailureWhat happensWhat the final artifact showsEvidence that exposes it
Stale overwriteAgent writes its cached version over the human'sAgent's earlier text; human change goneWrite made against base version, not human version
Regeneration on re-runResumed node regenerates a section it had already draftedFresh wording that drops human edits in that sectionSpan-level diff of human edits against final
Partial mergeAgent keeps some human edits and reverts othersMixed content that reads plausiblyPer-edit survival check, not a whole-file comparison
Intent reversalAgent treats a deliberate change as an error and fixes it backThe original agent text, restored on purposeEdit intent label plus the agent's stated reason
Authority overreachAgent changes a span the human locked or ownsAgent wording in a reviewed sectionAuthority rule in the packet and the span it covered
Repeated side effectRe-run step repeats an external action, such as a notificationCorrect document, duplicate message sentAction log with idempotency keys across the restart

The collaborative artifact version and intent task packet

The packet is one record per pause and resume cycle. It links the artifact versions, the human's edits and their intent, the authority rules, the agent's resumed actions and the verification. Values below are illustrative and do not describe a real system or dataset.

At a glance

The collaborative artifact version and intent task packet

The collaborative artifact version and intent task packet
FieldWhat it holdsIllustrative valueRule
task_and_constraintsGoal and the constraints stated at the start, each with its sourceDraft the incident summary; no customer names; plain languageConstraints carry an ID so later steps can cite them
base_versionContent hash of the artifact when the agent pausedsha256 of draft v4Required; every diff is computed from it
human_versionContent hash after the human's edits, before the agent resumedsha256 of draft v5Must differ from base when human_edits is not empty
human_editsOne entry per edit: path or span, before, after, author role, timeSection 2, sentence 3 replaced by the on-call leadSpans are stable anchors, not character offsets alone
edit_intentWhy the human changed it, and whether stated or inferredCorrection, stated in a commentInferred intent is labeled inferred, with reviewer name
authorityWhich spans the agent may change after resumingEdited spans locked unless the instruction names themDefault is locked
resume_instructionWhat the human asked the agent to do nextAdd a remediation sectionStored verbatim
agent_writeThe agent's changes, as a diff from human_versionNew section 5 onlyWrite is conditional on human_version still being current
final_versionContent hash of the artifact after the agent finishedsha256 of draft v6Parent must be human_version
verificationPer-edit survival, constraint checks and outcome verdict3 of 3 human edits present; constraint C2 holdsVerdict per edit, not one flag for the task

How should the resume step reconcile the two versions?

Reconcile with a three-way comparison, then write conditionally. The three inputs are the base the agent last saw, the human's version and the agent's intended version. Git's merge-file command describes the standard model: given a current file, a base and another file, it incorporates the changes from base to other into current, and it reports a conflict when both sides changed a common segment of lines.

For collaborative data, how a conflict was resolved matters as much as whether one occurred. Git can resolve conflicts automatically by favoring one side or keeping both, and the trajectory should record which rule applied and who chose it. A sensible default is that the human's version wins on any span both touched, unless the resume instruction explicitly hands that span back to the agent.

The write itself should fail if the artifact changed again after the agent read it. HTTP already defines this guard: RFC 9110 describes conditional requests with If-Match as a way to prevent the lost update problem, where one client accidentally overwrites another's parallel work, and a server must not perform the method when the condition is false. A rejected conditional write in the trajectory is good evidence; it shows the agent noticed the change and re-read before writing.

Checklist · 5 items

  1. Read the current artifact on resume; never write from the checkpointed copy.
  2. Compute base to human and base to agent diffs and list overlaps.
  3. Apply the authority rule to each overlap and record the decision.
  4. Write with a precondition on the human version's hash or entity tag.
  5. On a failed precondition, re-read, re-reconcile and log the retry.

0 of 5 ticked. Tick each item as you go.

Recording the intent behind each human edit

Store why the human changed something, because the same edit means different things for the next step. A corrected fact should be kept and propagated; a stylistic preference should be kept locally and perhaps applied elsewhere; a removed sentence may signal a new constraint the agent must apply to everything it writes next.

At a glance

Recording the intent behind each human edit

Keep the lineage machine-readable. The W3C PROV ontology has the relations a packet needs: a revision relation for a new version that keeps substantial content from an earlier one, and an attribution relation linking an entity to the agent responsible for it, whether a person or software. Using those relations, or fields that map to them, lets a buyer merge packets from different suppliers without guessing who wrote which version.

  • Correction: the agent's content was wrong; the fix must survive and inform later sections.
  • New constraint: the edit implies a rule, such as no customer names, that applies from now on.
  • Preference: wording or format choice; keep the span, follow the pattern only if the instruction says so.
  • Scope cut: content removed on purpose; the agent must not reintroduce it.
  • Fact update: the world changed during the pause; downstream values must be recomputed.

Worked case: an incident summary the agent nearly reverted

This case is illustrative and invented. An agent drafts a six-section incident summary in a shared document and pauses for review. During a 40-minute pause, the on-call lead makes three edits: corrects the root cause sentence, deletes a customer's name from the timeline, and changes one timeline entry's time. She then asks the agent to add a remediation section.

Failure run: the agent resumes, its drafting node restarts, and it regenerates the timeline section from its checkpointed notes before adding the new section. The final document reads well. The per-edit survival check shows 1 of 3 human edits present: the root cause correction survived, while the deleted customer name and the original timestamp both came back. The packet records the failure as regeneration on re-run, with the spans.

Correct run: the agent reads draft v5, diffs it against v4, marks the three edited spans as locked, infers a new constraint from the deleted name and records it as inferred, then writes only section 5 with a precondition on the v5 hash. The survival check reports 3 of 3 edits present, and a constraint check confirms no customer names appear in the new section.

Counterexample: in another packet, the human's edit changed a figure that the resume instruction later asked the agent to update from a newer report. Here the agent was right to change a human-edited span, because the instruction granted authority for it. The packet shows that by citing the instruction in the authority decision, so a reviewer does not mark the change as an overreach.

The version chain that shows the agent respected the edits

The proof is the chain of version hashes plus the per-edit verdicts, not the task's status field or the agent's closing message. A reader should be able to start from final_version, follow its parent to human_version, confirm the agent's write was conditioned on it, and check each human edit by span.

Collaborative Gym makes the same separation between outcome and process: it scores outcomes with delivery and task performance, and audits the process separately, for example with a measure of how initiative was distributed between human and agent. A purchased dataset should allow both readings. The outcome tells you whether the artifact was finished and correct; the packet tells you whether the agent respected the person who worked on it.

Where real collaboration records come from, and what to clear first

Business systems already hold much of the raw material: document version histories, ticket and case edits by agents and supervisors, review comments with accepted or rejected suggestions, and field history in CRMs. What they usually lack is the intent label and the authority rule, which have to be added by people who know the workflow, and the link to an AI agent's actions, which exists only where a business has deployed one.

These records carry employee authorship, customer details inside the edited text and the business's own confidential material. Ownership of the records, permission from the business's customers, privacy treatment of personal data and the license granted to the buyer are separate questions that need separate answers before anything is shared.

A buyer looking for collaborative trajectories can specify the packet fields, volume, format and licensing needs in SourceX's request form, and SourceX will look for U.S. businesses with matching records; it holds no ready-made stock.

Frequently asked questions

What should a collaborative artifact version and intent task packet include?

It should include the base, human and final version hashes, one entry per human edit with span, before and after, author role and intent, the authority rule, the resume instruction, the agent's write as a diff and per-edit verification. Constraints stated at the start should carry IDs so later decisions can cite them.

Which requirement, permission or state conflict is most often lost at the next handoff?

Constraints implied by a human edit are lost most often, because they were never written down as rules. A deleted customer name, for example, implies no names anywhere, and an agent that only keeps the edited span will reintroduce names in new content. Record inferred constraints explicitly and label them inferred.

Is a checkpoint enough to resume an agent safely?

A checkpoint restores the agent's own state, not the shared artifact. In LangGraph, a resumed node restarts from its beginning, so the agent must re-read the artifact and reconcile before writing. Side effects in re-run code should be idempotent.

Should the human's version always win a conflict?

It is a sound default for spans both sides changed, because the human acted later and with authority. The exception is when the resume instruction explicitly hands a span back to the agent. Record the rule applied and its source either way.

Can simulated humans produce this kind of data?

Simulated collaborators are useful for scale and for testing reconciliation logic, and research environments support both simulated and real users. Real edits carry the intent patterns and shortcuts that simulations miss, so label which packets came from real people.

Sources

  • Collaborative Gym lets humans and agents act in a shared workspace with real-time human intervention, and evaluates both outcomes and process. Source
  • LangGraph checkpoints save graph state keyed by thread_id, and a resumed node restarts from its beginning so code before the interrupt runs again. Source
  • LangGraph update_state creates a new branching checkpoint rather than rolling a thread back, leaving history intact. Source
  • git merge-file combines changes from a base into a current file, reports conflicts on common segments, and can resolve them by favoring one side or both. Source
  • RFC 9110 conditional requests with If-Match prevent the lost update problem; the server must not perform the method when the condition is false. Source
  • PROV-O defines revision as derivation into a revised version and attribution as linking an entity to the responsible agent. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies