AI data market
Engineering change data opportunity: the value is in linked chains
By SourceX Editorial · Reviewed by Noah Loul ·
9 sections
Short answer
The engineering change data opportunity rests on linked chains, not record counts: a problem report, the revision, its test evidence and its approval, joined. In SourceX's count of 77,540 Linux kernel commits from 2025, 0.9% recorded a reporter, a tester and a reviewer together; in Go, where tooling stamps links, 39.9% linked an issue, a test run and an approval. Public software already supplies such chains; private and hardware histories are unproven.
Value drivers
Engineering change data
Key takeaways
- Value sits in the join between problem, revision, test and approval; a count of commits or change orders measures none of it.
- In 2025, 0.9% of Linux kernel commits named a reporter, a tester and a reviewer together, while 39.9% of Go commits linked an issue, a trybot result and a reviewer.
- SWE-bench kept 2,294 tasks from roughly 90,000 pull requests, about 2.5%, after requiring an issue link, test changes and a test that fails before the fix.
- Automated mining and synthetic generation already supply tens of thousands of software tasks, which limits the premium on ordinary private code histories.
- The open question is histories with no public analog, such as hardware change orders, where buyer tasks and measured utility are still unpublished.
What would an investor be buying in engineering change data?
A set of linked chains, not a pile of change records. Each chain joins the problem that prompted a change, the revision itself, the evidence that it was tested and the approval that let it ship, and ideally what happened afterward. That join is what turns history into a task an AI system can attempt and a grader can check.
The distinction decides the case. A software company with a million commits, or a manufacturer with twenty years of engineering change orders, owns a large archive. Whether it owns supply depends on how many of those changes can be joined end to end, whether anyone may license them, and whether a buyer has a task that needs them. Record volume answers none of the three.
This article tests the thesis on public evidence: how benchmark builders already turn change histories into tasks, how often public projects record a complete chain, a scorecard of five sources of engineering change evidence with the gaps marked, and the developments that could make the opportunity smaller than it looks.
How SWE-bench turned public change histories into a demand signal
It showed that linked change histories make gradable tasks, and that only a small share of raw history qualifies. The SWE-bench authors collected roughly 90,000 pull requests from 12 popular Python repositories, kept merged ones that resolve a GitHub issue and change the repository's test files, then kept only those with at least one test that fails before the fix and passes after it. That left 2,294 task instances, about 2.5% of the starting pool.
Even that filtered set needed a second screen. For SWE-bench Verified, OpenAI had 93 experienced Python developers review a random sample of 1,699 test items for underspecified issue descriptions and unit tests that could reject correct fixes, and released a 500-item subset judged free of those problems. Every original item had its links; not every link was good enough to grade against.
For an investor the signal cuts two ways. A model developer ran an expert screening campaign on change-history tasks, which is evidence that such chains are valued as evaluation material. The same work shows that most raw records fail the link test, so yield, not archive size, is the number to underwrite.
Linkage yield: 77,540 kernel commits against 3,157 Go commits
Complete chains are rare where people add the links by hand and common where tools record them. SourceX counted the trailer lines in every non-merge commit the Linux kernel and Go projects applied during 2025. In the kernel, 0.9% of 77,540 commits named a problem reporter or bug report, a tester and a reviewer together, and 0.7% also named the earlier commit being fixed. In Go, 39.9% of 3,157 commits linked an issue, a trybot test result and a reviewer.
The kernel's own guide defines each link. A Closes: tag should point to the bug report, a Fixes: tag says the patch fixes a bug in a previous commit, a Tested-by: tag says the named person tested the patch successfully, and a Reviewed-by: tag means the reviewer found it acceptable under a written reviewer's statement of oversight. People type these tags, and most commits carry only some: 61.4% named a reviewer or acker, but only 9.2% a tester.
Go's links come largely from machinery. Its contribution guide says maintainers trigger trybots that run the full test suite on several architectures, and that a change needs a Code-Review +2 from a maintainer before it is submitted. In 2025 every Go commit carried a Reviewed-by line and 95.5% a trybot result line signed by a build-system account. The weak link was the problem statement: 42.1% referenced an issue.
The lesson carries to private archives. Where a company's tracker, CI and review tools write links automatically, much of its history may be chain-ready; where engineers were expected to add links by hand, yield can fall below 1%. Kernel commits that name the faulty earlier commit show the same gradient: of 14,370, 60.9% named a reviewer, 22.8% a reporter and 12.5% a tester.
Share of 2025 commits carrying each link, with definitions in the methods note:
| Link | Linux kernel, 77,540 commits | Go, 3,157 commits | How it is recorded |
|---|---|---|---|
| Problem or issue | 6.2% (Reported-by or Closes) | 42.1% (Fixes, Updates or For an issue) | Typed by authors in both projects |
| Earlier revision being fixed | 18.5% (Fixes tag) | Not counted | Kernel tag names the faulty commit |
| Test evidence | 9.2% (Tested-by) | 95.5% (trybot result) | A named person in the kernel; a build-system account in Go |
| Review or approval | 61.4% (Reviewed-by or Acked-by) | 100% (Reviewed-by) | Optional in the kernel; present on every Go commit |
| Problem, test and review together | 0.9% | 39.9% | The chain a gradable task needs |
Engineering change opportunity scorecard, with the evidence gaps marked
Scored on four tests, only one source of engineering change evidence has proven buyer tasks, and it is the one with the least scarcity. The scorecard rates each source on buyer task evidence, linkage and preparation burden, and rights and measured utility, and names the gap an investor would need closed before the source can carry a thesis.
Open-source software is the proven case and the crowded one. SWE-rebench's automated pipeline built more than 21,000 interactive Python tasks, 21,336 from 3,468 repositories, and SWE-smith generated 50,000 task instances from 128 repositories by synthesizing changes that break existing tests. Hardware sits at the other end. FAA rules classify every change in an aircraft type design as minor or major, minor meaning no appreciable effect on weight, balance, structural strength, reliability, operational characteristics or other airworthiness characteristics, so the records exist by rule; no public buyer task or utility study built on them was found.
The scorecard, as of October 27, 2026:
| Source | Buyer task evidence | Linkage and preparation | Rights and measured utility | Evidence gap |
|---|---|---|---|---|
| Open-source software histories | Strong: SWE-bench and SWE-rebench turn issue, patch and test into graded tasks | Mining is automated; link coverage depends on project tooling (0.9% to 39.9% above) | Public repositories under their own licenses; benchmark gains from training reported | Scarcity: public and synthetic supply keep growing |
| Private software histories | Same task family, but no published buyer specification for private histories | High where tracker, CI and review must be joined by hand; low where tools stamp links | Owner may license, minus third-party and customer material; no published utility study | Measured lift over public and synthetic tasks |
| Manufacturing change orders | No public benchmark built on change-order chains found | High: drawings, bills of materials, effectivity and approvals sit in separate records | Customer-owned designs excluded; utility unmeasured | A buyer task with an acceptance test |
| Aircraft type design changes | No public benchmark found | Each change must be classified minor or major and approved, so records exist | Confidential, possibly controlled technical data; utility unmeasured | A rights path and a named task |
| Synthetic change tasks | Heavy research use: 50,000 SWE-smith instances | Low: generated by breaking existing tests | Built from public code; a trained model reached 40.2% on SWE-bench Verified | Transfer to real, messier defects |
Does a large archive of changes prove there is supply?
No. Archive size sets a ceiling and says little else. The kernel applied 77,540 commits in 2025, and 718 of them carried the three-link chain; SWE-bench turned roughly 90,000 pull requests into 2,294 tasks. Filters that look modest on paper removed more than 97% of records in both cases.
Private archives face three further filters that public projects never meet. Rights: the company must be able to license the records, and change histories can include customer-owned designs, which the guide to licensing engineering change orders linked below separates out before anything else, as well as third-party code. Context: a hardware change order points to drawings, bills of materials and effectivity dates held in other systems. Demand: a buyer needs a task the chain serves, with an acceptance test.
The honest supply estimate therefore multiplies down from the archive: records, times the share with complete chains, times the share that clears rights review, times the share a buyer's acceptance test accepts. None of those shares has been published for private engineering archives, and that absence is the central evidence gap.
The pilot that would settle the private-history question
A three-arm pilot would show whether private chains add anything that public and synthetic chains do not. Hold the model, budget and grader fixed; train or evaluate on public chains, on synthetic chains and on private chains from one supplier; and score all three on held-out later changes from that supplier.
At a glance
The pilot that would settle the private-history question
SourceX has not run this pilot and has published no results from one. The design is offered so an investor can ask any supplier for evidence in this shape, with these elements fixed in advance:
- Population: changes from one company and one product line over a stated window, with the selection rule written down.
- Unit: a chain of problem, revision, test evidence and approval, admitted only when all four can be joined.
- Metrics: share of held-out changes resolved, chains per 1,000 records, rights-cleared share and preparation hours per accepted chain.
- Comparison: the private arm against public and synthetic arms of equal size, with the decision threshold set before the run.
- Reporting: denominators at every filter and the reasons chains failed, including those that failed rights review.
What could make the engineering change thesis wrong
Five developments would shrink the opportunity, and each can be watched in public sources.
- Public supply keeps scaling. SWE-rebench mined more than 21,000 interactive tasks automatically and tracks creation dates to flag contamination; if mining spreads to more languages and task types, ordinary private code histories add little.
- Synthetic chains substitute. A model trained with SWE-smith data reached 40.2% on SWE-bench Verified; if generated chains pass buyers' acceptance tests as well as real ones, the premium for real history falls.
- Link quality fails review. Curated public items still needed screening for underspecified issues and unfair tests in SWE-bench Verified, and private chains may fail that screen more often.
- Linkage costs too much. If a company's tools never recorded links, rebuilding chains by hand may cost more than buyers will pay, as the kernel's 0.9% three-link rate for hand-added links suggests.
- Rights clear too little. Customer-owned designs, third-party code and controlled technical data can remove whole product lines from a license.
What to ask a supplier of change histories, SourceX among them
Ask for yields and denominators, never archive sizes, and hold SourceX to the same list.
SourceX is one application of this thesis, not evidence for it. It takes records that a business chooses and turns them into datasets licensed for AI, handling the sourcing and qualification, rights review, contract, preparation, delivery and payment on the way. Its operating and commercial figures are not published; it discusses them with investors directly. Its raise is open only to accredited investors under Rule 506(c), and investors can request its materials through the investors page with a question such as how it would measure chain yield in a change archive.
Checklist · 5 items
- What share of your engineering change records join a problem, a revision, test evidence and an approval, and how was each link detected?
- Which links did the source systems record automatically, and which were rebuilt by hand, at how many hours per chain?
- What share of chains cleared rights review, and which record types failed most often?
- Has a buyer compared your chains with public or synthetic alternatives against a threshold set before the run?
- How are customer-owned designs and controlled technical data excluded before preparation starts?
0 of 5 ticked. Tick each item as you go.
Methods note: repositories, commits, trailer definitions and gaps
Data: commit messages of github.com/torvalds/linux reachable from d135ab453a7400a34930bdda1ef4bdc8acfac80e and of github.com/golang/go reachable from 6c49ac311e5642fa5399b46c736295a9dc4eeec9, read on October 27, 2026 from commit-only clones. Population: non-merge commits with a committer date in calendar 2025, UTC. Kernel links, matched in any letter case at the start of a line after any leading spaces: Reported-by: or Closes: (problem), Fixes: (earlier revision), Tested-by: (test), Reviewed-by: or Acked-by: (review). Go links, matched at the start of a line after any leading spaces: Fixes, Updates or For in any letter case followed by an issue number (problem), TryBot-Result: or LUCI-TryBot-Result: (test), and Reviewed-by: (approval).
The two projects record links differently, so the comparison shows how recording practice changes yield, not which project tests more. Trailers miss evidence nobody wrote down: a kernel patch can be tested by automated systems without a Tested-by line, and many commit messages state the problem in prose. Market evidence comes from the SWE-bench, SWE-rebench and SWE-smith papers, OpenAI's SWE-bench Verified announcement and the FAA rule as published in the eCFR, read the same day. The scorecard ratings are SourceX's judgment from that evidence.
Not measured: any private archive's chain yield, rights-cleared share, preparation hours or buyer acceptance, and the size of any market for engineering change data. No public figure supports a market size, so none is given, and SourceX has published no results for the pilot design above.
Frequently asked questions
Is engineering change data a real AI training data opportunity?
For software it is proven: benchmarks such as SWE-bench are built from linked issues, fixes and tests. For private and hardware change histories it is plausible but unproven, because no public study measures what they add over public and synthetic chains. Underwrite chain yield and a pilot result rather than archive size.
Why do linked change chains matter more than commit volume?
A task needs the problem, the fix, a way to test it and evidence it was accepted. In 2025, only 0.9% of Linux kernel commits recorded a reporter, a tester and a reviewer together, so where links are added by hand, raw volume overstates usable supply by about a hundredfold.
Can public open-source histories meet AI developers' demand on their own?
For general software tasks, increasingly yes: SWE-rebench mined more than 21,000 tasks automatically and SWE-smith generated 50,000. Private histories matter most where public analogs are thin, such as proprietary systems, regulated design changes and hardware change orders.
How do engineering change orders differ from software changes as AI data?
A change order ties a design change to drawings, bills of materials, effectivity dates and approvals held in several systems, and often to customer-owned designs. That makes chains costlier to assemble and rights harder to clear, and no public benchmark built on change orders was found as of October 2026.
What metrics should investors request from an engineering change data supplier?
Chains per 1,000 records with the link definitions, the share that cleared rights review, preparation hours per accepted chain, and any buyer acceptance result with a threshold set in advance. Each figure should come with its denominator and its date range.
Sources
- Linux kernel commit history, from which SourceX counted problem, revision, test and review trailers on 77,540 non-merge commits from 2025. Source
- The kernel's patch submission guide defines the Closes:, Fixes:, Tested-by: and Reviewed-by: tags. Source
- Go commit history, from which SourceX counted issue, trybot-result and Reviewed-by lines on 3,157 non-merge commits from 2025. Source
- Go's contribution guide: trybots run the full test suite on several architectures, and a change needs a Code-Review +2 from a maintainer to be submitted. Source
- SWE-bench collected roughly 90,000 pull requests from 12 Python repositories and kept 2,294 task instances that link an issue, a fix and fail-to-pass tests. Source
- SWE-bench Verified: experienced developers screened 1,699 random test items for underspecified issues and problematic tests, yielding a 500-item subset. Source
- SWE-rebench's automated pipeline produced more than 21,000 interactive Python tasks from 3,468 repositories and tracks creation dates for contamination. Source
- SWE-smith generated 50,000 task instances from 128 repositories by breaking existing tests; a model trained with that data reached 40.2% on SWE-bench Verified. Source
- FAA rules classify each change in an aircraft type design as minor or major. Source
Request SourceX investor materials
SourceX shares company and commercial metrics with investors directly. Any offering is made only to verified accredited investors.