AI data market
Agent environment or dataset: 5 questions that set the business model
By SourceX Editorial · Reviewed by Noah Loul ·
9 sections
Short answer
A dataset records what happened; an environment computes what happens next when an agent acts. That difference sets the agent environments dataset business model: a dataset seller's duties mostly end at delivery, while an environment seller keeps funding state transitions, resets, verifier fixes and version upkeep for as long as buyers train on it. Classify any offering with five questions before comparing prices, margins or moats.
Agent run
Agent environment or dataset
Key takeaways
- The test is the off-record action: if an agent can do something no record contains and get a computed consequence back, the product is an environment.
- Five offerings share the environment label, and only simulated applications and hosted services compute consequences; record datasets, task sets and replay harnesses do not.
- Across six post-delivery duties, a seller carries none on an ongoing basis for a record dataset, four for a simulated environment and six for a hosted service.
- Verifier upkeep is real work: an audit of SWE-bench estimated that about a third of passing patches had leaked solutions and nearly another third passed on weak tests.
- Records and environments are complements with separate economics, so a company selling both should show unit economics for each line.
What separates an agent environment from a dataset?
An environment answers actions; a dataset only answers questions about the past. If an agent can do something no record contains and get a consequence back, such as a changed database row, an error or a reply, you are looking at an environment. If every response was fixed before the agent arrived, it is a dataset, however interactive the packaging looks.
The definition predates language-model agents. OpenAI's 2016 Gym paper (arXiv 1606.01540) organized reinforcement learning around environments behind one common interface: experience is split into episodes, each begins from an initial state, and after every action the environment hands back an observation and a reward until a terminal state ends the episode. Nothing in that loop is a stored answer. The environment computes it.
Commercial environments for language-model agents keep the loop and wrap a dataset inside it. Prime Intellect's documentation describes each environment on its Environments Hub as a self-contained Python module that bundles a dataset, a harness and a rubric of reward functions, distributed as versioned Python wheels. The dataset supplies tasks and starting states; the harness and the rubric turn it into something an agent can act in and be scored by. Before comparing two companies, an investor should ask which of those three parts each one actually sells.
Five offerings sold under the environment label
Five kinds of product reach agent builders under the environment label, and only two of them compute consequences. The other three are datasets with progressively more structure, a respectable business with different economics.
Public research shows the whole range. SWE-Gym (arXiv 2412.21139) turns GitHub issues into 2,438 Python tasks, each paired with a codebase, an executable runtime environment, unit tests and a task written in natural language, and its authors report absolute gains of up to 19 points in resolve rate on SWE-Bench Verified and Lite after training agents on it. The issues on their own were a dataset; the runtimes and tests made them an environment.
AppWorld (arXiv 2407.18901) goes further: about 60,000 lines of code simulate nine everyday apps behind 457 APIs, populated with the digital lives of roughly 100 fictitious people, and its 750 tasks are scored by state-based unit tests that accept different routes to the goal while checking for collateral damage. WorkArena (arXiv 2403.07718) runs 33 knowledge-work tasks with 19,912 unique instances on real ServiceNow personal developer instances. TheAgentCompany packages 175 workplace tasks as Docker images that act on self-hosted GitLab, Plane, ownCloud and RocketChat services.
The table places each offering by what the buyer receives and by who decides whether the agent succeeded:
| Offering | What the buyer receives | Can the agent act off the record? | Who decides success | Public example |
|---|---|---|---|---|
| Record dataset | Fixed records with documentation and a license | No: records answer only what was recorded | Labels or outcomes stored in the records | The GitHub issues SWE-Gym starts from |
| Task set with reference outcomes | Prompts, inputs and expected results, graded offline | No: the agent's output is compared afterwards | A stored answer or a script run on the final output | A frozen evaluation split with answer keys |
| Replay harness | Recorded tool responses served back in sequence | Only along recorded paths; an unrecorded call has no reply | A match against the recorded trajectory | Illustrative: API logs from one company replayed to an agent |
| Simulated or sandboxed application | Running software with state, tools, resets and a verifier | Yes: the software computes each consequence | A program that inspects end state, sometimes a model or a person | AppWorld, WorkArena, TheAgentCompany, SWE-Gym |
| Hosted environment service | Access to environments plus rollouts, evaluation and compute | Yes, on the host's infrastructure | The host's rubric, versioned with the environment | Prime Intellect's Lab with its Environments Hub |
Five questions that classify any offering
Ask five questions in order, and the first no tells you which class you are buying. A product that passes all five is a hosted environment; one that fails the first is a dataset whatever it is called.
At a glance
Five questions that classify any offering
Question four matters most to an investor, because it separates assets that hold their value on a shelf from assets that need a maintenance team to stay true to the systems they imitate.
The five questions:
- Off-record action: can the agent take an action that appears in no record and observe a computed consequence? If not, it is a dataset or a replay harness.
- Reset: can the starting state be restored exactly, so an episode can be run again and compared? If not, results cannot be reproduced and the training signal is hard to trust.
- Success: who decides it, a stored label, a program that inspects the end state or a person? A stored label points to a task set; a state check points to an environment.
- Seller absence: what still works if the seller stops all work tomorrow? A dataset keeps its value as a file, while a simulator of live software starts drifting from the real thing.
- Transfer: does the buyer receive a copy, a license to run code or access to a service? Each implies a different revenue form, from a one-time license to usage billing.
Who carries which duty after delivery?
Duties pile up as an offering moves from records toward a hosted service. A dataset seller's main obligations end at signing and delivery, while an environment seller keeps transitions, verifiers, resets and uptime working for as long as buyers train. Counted across the six rows of the matrix, a seller carries no ongoing duties for a record dataset or a task set, four for a simulated environment and six for a hosted service.
Verifier upkeep is the duty most often underpriced. When the authors of SWE-Bench+ (arXiv 2410.06992) audited patches that had passed SWE-bench, they estimated that 32.67 percent passed because the solution already sat in the issue text or comments and 31.08 percent passed because the tests were too weak; filtering those cases cut one agent's resolution rate from 12.47 to 3.97 percent. A grader that accepts wrong work is worse than no grader, because training follows it.
Resets and isolation are just as concrete. TheAgentCompany's evaluation guide warns that initializing a task can take up to 10 minutes because the script resets all the data in the dependent services, and the benchmark encrypts its grading functions to stop agents from reading the evaluator code. Both are engineering choices that someone has to keep working through every release.
Version upkeep appears even in open tooling. Prime Intellect's documentation says its hub is moving from version 0 of the verifiers library to version 1, with version 0 deprecated and older environments listed as legacy. An environment is software, and software comes with a maintenance schedule.
The duty matrix by offering class, an editorial classification of typical terms:
| Duty | Record dataset | Task set | Simulated environment | Hosted service |
|---|---|---|---|---|
| Rights and provenance of the records | Seller, at delivery | Seller, at delivery | Seller, ongoing | Seller, ongoing |
| Task validity and solvability | Buyer | Seller, at delivery | Seller, ongoing | Seller, ongoing |
| State transitions that behave like the real system | Does not exist | Does not exist | Seller, ongoing | Seller, ongoing |
| Verifier correctness and exploit fixes | Buyer | Seller, at delivery | Seller, ongoing | Seller, ongoing |
| Resets, isolation and reproducibility | Does not exist | Does not exist | Seller, at delivery | Seller, ongoing |
| Compute and uptime for rollouts | Buyer | Buyer | Buyer | Seller, ongoing |
How the value chain splits, and what wears out at each link
The chain runs from source records to tasks, to state and transitions, to verifiers, to rollout infrastructure and finally to a trained or evaluated agent, and each link loses value for its own reason. Pricing power sits with the link that wears out slowest relative to how hard it is to replace.
At a glance
How the value chain splits, and what wears out at each link
The links are already traded separately. micro1 said in October 2026 that it plans to spend 1 billion US dollars over 12 months acquiring and licensing enterprise operational data through a Company Data Partnerships program, using de-identified company data to build reinforcement learning environments that reflect real business operations. It is a stated plan rather than money spent, and its structure is what matters here: an environment builder treating records as a purchased input.
At the other end of the chain, Prime Intellect's Lab joins its Environments Hub with hosted training and hosted evaluations, built so customers can run agentic post-training without managing GPU clusters. Hosting turns an environment from a deliverable into a service whose revenue depends on usage and uptime rather than on a single transfer.
What wears out at each link:
- Source records: the history stays true, but relevance fades as workflows, policies and systems change; value lasts longest where records capture rare decisions and outcomes that are hard to stage.
- Tasks: they saturate as agents improve and leak into training corpora, so task sets need fresh items to keep separating strong agents from weak ones.
- State and transitions: a simulator that imitates real software drifts as soon as that software ships a new version, and someone has to follow it.
- Verifiers: agents find whatever graders fail to check, and each exploit found means a patch to write and past scores to revisit.
- Rollout infrastructure: compute and orchestration face price competition and open tooling, so margins rest on scale and utilization rather than scarcity.
Do agents need the records at all, or only the simulator?
Often the simulator does most of the work, and a record collection on its own may lack both the environment and the outcome evidence an agent needs. AppWorld populates its apps with fictitious users, WorkArena initializes its instances with demonstration data, and SWE-Gym builds on public code, so capable environments exist without any private operating records.
Records earn their place where invented state stops being realistic: the true mix of exceptions, the order in which real cases unfold, the outcomes that followed each decision and the policy in force at the time. An environment seeded only with invented cases tends to test its own simulator rather than the work, which is the gap micro1's stated plan targets. The investor question is not records versus environments but which offering supplies realistic state and outcomes, and whether its owner may license them.
Treat the two as complements with separate prices. A record supplier earns on rights, coverage and outcome evidence; an environment builder earns on fidelity, verifiers and upkeep. A company that claims both should be able to show separate unit economics for each line.
Risks that could break the environment-versus-dataset thesis
Five developments would shift who captures value, and each has a public marker to watch:
- Open environments could commoditize harness code. Prime Intellect distributes environments as installable packages on a public hub; if generic harnesses are shared freely, sellers can charge mainly for scarce data, rubrics or hosting rather than code.
- Synthetic state could close the realism gap. If generated users and cases, such as AppWorld's fictitious people, pass a buyer's acceptance tests, the premium for real records narrows.
- Verifier failures carry exposure. The SWE-Bench+ audit shows that passing tests can certify wrong work, and a seller of reward signals inherits that risk at scale.
- Imitating third-party software raises rights questions. Recreating a SaaS product's behavior for training touches contracts and intellectual property that a dataset license does not settle.
- Buyers may build in-house. If the largest agent developers staff their own environment teams, outside sellers keep pricing power only for the parts a buyer cannot rebuild.
What to ask a seller of environments or datasets, SourceX included
Ask each seller to classify its own products with the five questions, then ask for evidence behind every duty it claims to carry. SourceX belongs on the same list.
On its own description, SourceX operates at the record layer of this chain. It helps companies turn business records they select into licensed AI datasets, running sourcing and qualification, rights review, contracting, data preparation, delivery and payment. The supplying company approves scope and terms and signs before any data reaches SourceX, and very large deliveries stay in the supplier's own storage or travel on encrypted drives instead of being hosted. SourceX prints no deal or commercial figures; it gives company metrics to investors directly, and its Rule 506(c) raise is limited to accredited investors, with materials requested through the investor page.
Questions for any seller in this market:
Checklist · 6 items
- Which of the five offering classes does each product line fall into, and how much revenue comes from each class?
- For every duty in the matrix that you carry, how many engineers maintain it per active environment version?
- How often were verifiers patched in the last year, and were past scores recomputed after each patch?
- How long does a reset take, and how do you show that two runs started from the same state?
- Which source records feed your tasks and starting states, and what do your licenses allow you to do with derived tasks?
- If you stopped all work for six months, which products would still pass their acceptance tests?
0 of 6 ticked. Tick each item as you go.
Methods note: how the offering classes and duty matrix were drawn
The offering classes, the five questions and the duty matrix were drawn up for this article on 2026-10-27 as editorial tools. They rest on the Gym paper (2016), SWE-Gym, AppWorld, WorkArena and SWE-Bench+ on arXiv, TheAgentCompany's evaluation guide in its GitHub repository, Prime Intellect's environment documentation and Lab announcement, and micro1's October 2026 announcement.
Not measured: market sizes, prices or margins for any class, how often each class is sold, and engineering effort per environment at any company. The matrix shows a typical split of duties; a contract can move any cell to either side. No SourceX data informs the analysis.
Frequently asked questions
Is an agent trajectory dataset an environment?
No. Recorded trajectories show what one agent did and what followed, but they cannot answer an action nobody took. They become part of an environment only when a harness can compute consequences for new actions and a verifier can score the result.
Why do environments cost more to sell than datasets?
Because the duties continue after delivery. An environment seller has to keep transitions faithful to the software they imitate, reset episodes reliably, patch verifiers when agents find exploits and, when hosting, keep compute available. A dataset seller's main duties end at signing and delivery.
Can one company sell both datasets and environments?
Yes, but the two lines have different cost structures. Ask for revenue and engineering effort per line, because environment upkeep can hide inside a blended margin that looks like licensing.
Why would an environment builder buy business records?
Invented state rarely reproduces the real mix of exceptions, sequences and outcomes. micro1, for example, has announced a plan to acquire and license de-identified enterprise operational data to build reinforcement learning environments that reflect real operations.
How should an investor value an agent environment company?
Read revenue net of upkeep. Count engineers per active environment version, verifier patch frequency and reset reliability, and separate one-time builds from recurring hosting or maintenance, since each behaves differently as agents improve.
Sources
- OpenAI's Gym paper describes environments behind a common interface, with episodes that start from an initial state and return an observation and reward after each action until a terminal state. Source
- Prime Intellect's documentation describes each environment as a self-contained Python module bundling a dataset, a harness and a rubric, distributed as versioned wheels, and says verifiers v0 is deprecated in favor of v1. Source
- Prime Intellect's Lab combines the Environments Hub with hosted training and hosted evaluations for agentic post-training without managing GPU clusters. Source
- SWE-Gym contains 2,438 Python task instances, each with a codebase, executable runtime environment, unit tests and a natural-language task, and reports up to 19 points of resolve-rate gains. Source
- AppWorld simulates nine apps behind 457 APIs with about 100 fictitious users in about 60,000 lines of code, with 750 tasks scored by state-based unit tests that also check collateral damage. Source
- WorkArena has 33 tasks and 19,912 unique instances running on ServiceNow personal developer instances initialized with demonstration data. Source
- TheAgentCompany's evaluation guide describes 175 Docker-image tasks, an initialization script that resets data in dependent services in up to 10 minutes, and encrypted evaluators. Source
- SWE-Bench+ estimates 32.67 percent of passing SWE-bench patches had solution leakage and 31.08 percent passed on weak tests; filtering cut one agent's resolution rate from 12.47 to 3.97 percent. Source
- micro1 announced a stated plan to spend 1 billion US dollars over 12 months acquiring and licensing enterprise operational data, using de-identified company data to build RL environments. Source
Request SourceX investor materials
SourceX shares company and commercial metrics with investors directly. Any offering is made only to verified accredited investors.