Validating LLM-Extracted Procedure Steps

The buyer's problem

Preparation vendors increasingly run documents through a language model to produce structured steps, then spot-check a share. Output looks fluent and well-formed, which is the hazard: a record with a plausible invented precondition reads the same as a correct one. A single headline accuracy number hides what kind of errors occur and where. For procedure data used as planning input or as an evaluation set, a systematic error, such as steps merged whenever a table interrupts them, can quietly corrupt every downstream result.

This checklist is for the buyer or evaluator who receives extracted data. It is not a build guide. For the numeric layer see parameters and tolerances extraction; for a whole-corpus assessment, the SOP corpus evaluation checklist.

Before you sample: define the claim

Say what the extracted data will be used for, because it sets the tolerance. A retrieval index tolerates a missed warning more easily than a gold evaluation set does. State the unit of accuracy: step, field, document, or plan. Set thresholds before the vendor delivers, and write them into the acceptance terms.

Record five things about the delivery first: the source documents and their formats; the extraction method (model, prompts, post-processing, human review stages); the output schema; what was excluded; and the sampling plan the vendor used internally. A dataset datasheet that documents motivation, composition and collection process is a useful vehicle for this; the Datasheets for Datasets proposal describes that documentation practice (Gebru et al., 2018). The ISO/IEC 5259 series frames data quality across the data life cycle and offers measures and process frameworks that can supply vocabulary for acceptance criteria (ISO/IEC 5259-1:2024); it does not set thresholds for you.

Error classes to count

Report counts per class, not only a total. Two datasets with the same overall error rate can differ greatly in usefulness if one fails by omission and the other by paraphrase.

Sampling plan

• Stratify by document type, source format (native versus scanned), length, language, presence of tables and figures, and site or origin. Language models tend to fail in particular strata.
• Oversample hard cases: pages with tables, nested lists, multi-column layout, footnotes and warnings, since layout errors cascade. Layout benchmarks exist because this is difficult: DocLayNet reports 80,863 manually annotated pages, with 7,059 pages double-annotated and 1,591 triple-annotated to measure annotator agreement (Pfitzmann et al., 2022). That design is a reminder that even human labels need measured agreement.
• Use blind review. Reviewers should see source and output side by side without knowing whether a record was previously flagged.
• Draw your own sample. Do not accept only a vendor-chosen sample. Ask for the full set and pick records yourself.
• Re-sample after fixes. A rerun of the extraction can introduce new errors; repeat the test on a fresh draw.

Sample size depends on the confidence you need and the error rate you expect. Decide it with someone who handles sampling statistics, and say in advance what happens if the observed rate exceeds the threshold.

Illustrative acceptance checklist

Thresholds below are examples a buyer would set, not recommendations.

Illustrative error-log record

# Illustrative - synthetic
record_id: rec_004211
source: {doc: doc_0231, page: 14, bbox: [72, 310, 520, 346]}
extracted_step: "Remove cover and replace filter."
reviewer_verdict: fail
error_class: merge
note: "Source has two numbered steps (6.3, 6.4); safety check between them lost"
severity: high
reviewer: reviewer_04
double_reviewed: true

Ask for logs in this spirit, with a severity tag, so you can see whether failures cluster on high-consequence steps.

Red flags

• Single aggregate accuracy with no definition of unit, sample or error classes.
• No source spans. If steps cannot be traced, they cannot be audited.
• Review done by the same system that produced the extraction.
• Uniform fluency. Output with no flagged uncertainty suggests missing confidence handling.
• Duplicates. A repeated boilerplate procedure inflates counts; see deduplicating procedures.
• Silent normalization of units or terminology with no original text kept.
• Evaluation leakage. Records for evaluation drawn from the same documents or families as training material.
• No version control. Corrections applied without a changelog.

What validation cannot establish

Passing this checklist shows that the extracted records faithfully represent the documents. It does not show that the documents are correct, current or followed in practice; see procedural drift and tacit knowledge gaps in SOPs. It also does not make procedural text into physical-control training data. Faithful steps remain language and structure. Quality scoring of the source procedures themselves is a different exercise, covered by procedure quality scoring.

When extraction feeds a formal representation, the tests multiply: see converting procedures to PDDL or behavior trees.

Limitations and additional preparation

• Reviewer expertise. Judging whether "isolate unit" omits a lockout step requires domain knowledge; hire or borrow reviewers who know the trade.
• Cost. Careful double review is slow. Concentrate on high-consequence strata.
• Moving targets. Model updates change error profiles; fix the extraction version for the delivery.
• Scanned sources. Poor scans raise error rates; see parsing scanned manuals.
• Redaction effects. Masking may itself remove or alter content; check redacted fields were not mistaken for omissions.

Questions to ask suppliers

1. Which model, version and prompts produced the records, and what human review followed?
2. What share of records was reviewed, by whom, and with what disagreement rate?
3. Can I draw my own sample from the full set?
4. Are source spans provided for every step and attribute?
5. How are errors logged and severity classified?
6. Were corrections made after review, and is there a changelog?
7. What documents failed extraction or were excluded?
8. Which rights govern derived records versus source documents?

Discuss your requirements

If you are specifying or checking extracted procedure data, tell SourceX what your evaluation set needs to cover. SourceX looks for US businesses that hold suitable material, checks the data and the supplier's licensing permissions, and coordinates the license and delivery; annotation and review are scoped case by case. A request does not guarantee a match, and SourceX follows up about potential sources. See the humanoid training data guide and the procedures hub, and the general training data due diligence checklist.

Tell SourceX what your evaluation set needs to cover.

Sources

• DocLayNet: A Large Human-Annotated Dataset for Document-Layout Analysis — page counts, layout classes and multi-annotated subsets.
• ISO/IEC 5259-1:2024 Artificial intelligence - Data quality for analytics and machine learning (ML) - Part 1: Overview, terminology, and examples — data-quality framework across the life cycle.
• Datasheets for Datasets — documentation of motivation, composition and collection.