Skip to content

AI uses for records

RL environment task dependency graphs: model the work before the prompt

By SourceX Editorial · Updated

9 sections

Short answer

Model the business task as a dependency graph before writing a prompt. Each node is a step with the state it reads and writes, its tool and a completion check; each edge is typed as data, policy, approval or external. The environment's initial state, actions, reset and state-based success checks follow from the graph, and the prompt only states the goal.

Grader rubric

RL environment task

This page covers RL environment task dependency graphs: model the work before the prompt.Illustrative

Key takeaways

  • Write the task as a graph of steps with reads, writes, tools and completion invariants before writing any prompt.
  • Type every edge as data, policy, approval or external so the grader knows which orderings fail and which are fine.
  • Derive initial state, actions, reset and success checks from the graph instead of inventing them.
  • Run acceptance tests such as acyclicity, reference replay, order sensitivity and collateral-damage diffs before training.
  • Model loops as events that reopen a node, and have the process owner sign off on edges and invariants.

How do you model business task dependencies before writing RL prompts?

Draw the task as a dependency graph first. Each node is one business step with the state it reads, the state it writes, the tool that performs it and the check that proves it is done; each edge says why one step must wait for another. From that graph you can derive the initial state, the allowed actions, the reset snapshot and the end-state checks. The prompt comes last and only describes the goal.

Teams that start with the prompt tend to hide the hard part of a business task inside prose. A sentence such as resolve the customer's backorder sounds complete, but it leaves out which records must be checked before a hold is placed, which refund cannot happen until a line is cancelled and which notice depends on both. An agent trained against that prompt is graded on whatever the grader author remembered.

This guide builds one RL environment task dependency graph in full, for an illustrative order exception, and shows how each part of the environment falls out of it. It ends with the acceptance checks the graph makes possible, the cases where a graph is the wrong model and the points where a process owner has to sign off.

What belongs on a node, and what an edge has to mean

A node is the smallest step that changes or confirms business state on its own. Reading an order is a node; deciding between a split shipment and a hold is a node; sending the customer notice is a node. Steps the agent might take for its own convenience, such as listing open tickets, are actions, not nodes, unless the business process requires them.

Every edge needs a type, because the type decides how the grader treats an out-of-order trajectory. A data edge means the later step cannot be computed without the earlier one's output. A policy edge means the later step is possible but forbidden until the earlier one is complete, such as refunding before the cancellation is recorded. An approval edge waits on a person. An external edge waits on a system or party outside the agent's control, such as a supplier's ship date.

What belongs on a node, and what an edge has to mean
ElementWhat to recordWhy the environment needs it
Node id and nameA stable id and a verb phraseTrajectories and logs reference steps by id
ReadsTables or fields the step depends onDefines what the initial state must contain
WritesFields the step changes and their allowed valuesDefines the end-state checks and collateral-damage checks
Tool or actionThe API call or UI action that performs the stepDefines the action space
Completion invariantA condition on state that is true once the step is doneLets the grader check state instead of transcript text
Edge typeData, policy, approval or externalTells the grader which orderings are failures and which are only inefficient

The worked order exception dependency graph

The example is illustrative and uses invented records. An online retailer receives an order with three lines. Payment is authorized, one line is out of stock with a supplier date two weeks out, and the shipping address failed validation. The business policy says customers choose between a split shipment and waiting for the full order, partial refunds follow a recorded cancellation, and the order is closed with a resolution code.

At a glance

The worked order exception dependency graph

The status vocabulary borrows from a real commerce API to keep it concrete. Shopify's Admin API defines fulfillment order statuses including OPEN and ON_HOLD, and states that fulfillment cannot be initiated for an order on hold until the hold is released; its hold mutation changes a fulfillment order's status to ON_HOLD and creates a fulfillment hold. The graph below does not depend on Shopify; any order system with holds, splits and partial captures maps onto it.

  • N2 and N3 are independent; an agent may do them in either order, and the grader must not penalize that.
  • N8 has a policy edge from N7: the refund is computable from N4 alone, but issuing it before the cancellation is recorded is a failure even if the final balance is right.
  • N4 is the only node that waits on a person; in the environment it is answered by a scripted or simulated customer.
The worked order exception dependency graph
NodeReads, then writesDepends on (edge type)Completion invariant
N1 Load orderOrder, lines, payment status; writes nothingNone (start)Exception reasons listed for this order id
N2 Check stock per lineInventory and supplier dates; writes nothingN1 (data)Each line marked available or short with a date
N3 Validate addressAddress record; writes corrected address or flagN1 (data)Address valid or a customer query is open
N4 Get customer choiceContact record; writes split or wait preferenceN2 (data), N3 (policy: confirm address in the same contact)Preference stored with a timestamp
N5 Hold short lineFulfillment order; writes status ON_HOLD with reasonN4 (data)Short line on hold; other lines unchanged
N6 Release available linesFulfillment order; writes OPEN for in-stock linesN3 (policy), N4 (data)Only in-stock lines open and address valid
N7 Cancel or keep short lineOrder line; writes cancelled flag if customer chose not to waitN4 (data)Line state matches customer choice
N8 Adjust paymentPayment; writes partial capture or refundN7 (policy: never before cancellation)Captured amount equals the value of lines shipped or kept
N9 Notify customerMessage log; writes one noticeN6 and N8 (data)One notice listing shipped, held and refunded lines
N10 Close with resolution codeOrder; writes resolution codeN9 (policy)Code set and no open holds without a reason

Turning the graph into state, actions, reset and success checks

Each part of the environment is read off the graph rather than invented. The initial state is the union of everything the nodes read, at the values the scenario requires. The action space is the set of tools named on the nodes, plus any read-only tools a real operator would have. Success is the conjunction of the completion invariants of the terminal nodes, plus a check that nothing outside the nodes' write sets changed.

Reset deserves explicit design because business state is large and stateful. Gymnasium's reference API describes reset as returning the environment to an initial internal state and says that passing an integer seed resets the random number generator; its step API separates terminated, when the task reaches a terminal state, from truncated, when an outside limit such as a time limit ends the episode. Keep that split: a run that exhausts its step budget at N7 is truncated, not failed for a wrong decision.

Grading on state rather than transcript has precedent in agent benchmarks. The tau-bench authors compare the database state at the end of a conversation with an annotated goal state, so different phrasings that reach the same state pass. AppWorld scores with state-based unit tests that allow different ways to complete a task while also checking for unexpected changes, which it calls collateral damage. The graph's write sets give you exactly the list of fields such tests should inspect.

  • Initial state: one order, three lines, an authorized payment, inventory with one short line and a supplier date, an address that fails validation, a customer simulator seeded with a preference.
  • Actions: read order, read inventory, validate address, message customer, place hold, release hold, cancel line, capture or refund, send notice, set resolution code.
  • Reset: restore a database snapshot and reseed the customer simulator; record the seed in the episode log.
  • Success: invariants of N5 to N10 hold, the refund was issued after the cancellation, and no field outside the write sets changed.
  • Prompt, written last: a short operator request naming the order id and the goal, without listing the steps.

Acceptance tests the graph lets you run before training

A graph makes the environment testable before any agent touches it. Run these checks on every task variant and keep the results with the environment version.

Graph tooling helps with the first check. NetworkX's topological sort documentation notes that a topological ordering exists only for a graph with no directed cycles and that the function raises an error when the graph is not acyclic. If a task's graph fails that check, either an edge is wrong or the task needs the event-driven treatment described below.

Acceptance tests the graph lets you run before training
CheckHow to run itWhat a failure means
AcyclicTopological sort of the task graphA mislabeled edge, or a loop that needs event modeling
ReachableEvery node reachable from the start nodeA step that no trajectory can perform
SolvableReplay a reference trajectory from reset and assert all invariantsThe environment cannot be completed as specified
Order-sensitiveReplay with a policy edge violated and confirm the grader fails itThe grader checks only the final balance
Order-tolerantReplay with independent nodes swapped and confirm it still passesThe grader enforces an order the business does not
Collateral damageDiff all fields outside write sets after the reference runA tool writes more than its node allows
Deterministic resetReset twice with the same seed and diff stateHidden randomness that makes failures unreproducible

When a dependency graph is the wrong model for the task

Some business tasks loop, and forcing them into an acyclic graph hides the part worth learning. If the customer changes their mind after the hold is placed, the process re-enters N4 and may undo N5. Model that as an event that reopens a node, with its own trigger and invariant, rather than as a back edge; the graph for each episode stays acyclic, and the scenario set includes episodes with the event.

Other tasks have little order at all, such as tidying records where each fix is independent. A graph still documents reads, writes and invariants, but edges add nothing and the grader should check the end state only. Teams that already document processes in BPMN, the Object Management Group's process notation published as version 2.0.2, can start from those diagrams, but should expect to add read and write sets and invariants that a process diagram does not carry.

Which settings outside the graph can flip the acceptance result?

The graph fixes order and state, but several environment settings sit outside it and can turn a passing task into a failing one. Pin each of them in the environment version and vary them only on purpose.

At a glance

Which settings outside the graph can flip the acceptance result

The most common surprise is tool semantics. If the simulated hold tool also blocks the in-stock lines, N6 can never succeed, and the task fails for the environment's reasons rather than the agent's. Check each tool's writes against the node's write set before blaming the policy.

  • Customer simulator behavior: a simulator that never answers makes N4 unreachable.
  • Step and time budgets: a budget below the reference path length truncates every run.
  • Tool side effects: a tool that writes beyond its node's write set causes false collateral-damage failures.
  • Visibility: if the agent cannot read supplier dates, N2 cannot complete and the task is unsolvable as written.
  • Policy text given to the agent: an edge the business enforces but the agent is never told about is a hidden rule, not a test of judgment.

Who must sign off before the graph becomes training data

The process owner signs off on edges and invariants; the environment engineer signs off on reset, tools and grading. Neither can do the other's part. An engineer can tell that N8 depends on N7, but only the person who runs order exceptions knows whether partial captures are ever allowed before cancellation for a wholesale account.

Stop and narrow the scope when the edges depend on contracts, regulated steps or approvals whose rules the team cannot see. A refund rule set by a payments agreement, or a hold required by a compliance screening, should be stubbed as an approval node with a scripted answer until the owner of that rule has reviewed how it is modeled.

Order exception histories as evidence for the graph

The graph is only as good as the edges, and edges are easiest to justify from real histories: order logs with status changes, hold reasons, refund timestamps and resolution codes show which steps actually wait on which. Environment builders who need those histories can describe the workflow, volume, format, timing and licensing terms they need on the SourceX request form; SourceX looks for matching U.S. businesses per request instead of holding datasets in stock. Whichever business supplies the histories signs off on their scope and terms, and signs its agreement, before any order history is handed over.

Frequently asked questions

What is a task dependency graph in an RL environment?

It is a directed graph whose nodes are business steps, each with the state it reads and writes, the tool that performs it and a completion check, and whose edges say why one step must wait for another. The environment's initial state, actions, reset and success checks are derived from it.

Why build the graph before writing the prompt?

The prompt describes the goal, but the graph defines what correct behavior is. Writing it first exposes hidden rules, such as a refund that must follow a cancellation, and gives the grader concrete state to check instead of transcript text.

How do you grade an agent that does steps in a different order?

Use edge types. Independent steps can run in any order without penalty, data edges are enforced by state, and policy edges are checked explicitly from the trajectory. A refund issued before the cancellation fails even if the final balance is right.

What if the business process loops back on itself?

Model the loop as an event that reopens a node, such as a customer changing their choice, rather than as a back edge. Each episode's graph stays acyclic, and the scenario set includes episodes where the event fires.

Can existing BPMN diagrams be reused?

They are a good starting point for steps and order. Expect to add what a process diagram usually lacks: the exact fields each step reads and writes, completion invariants and the tools the agent will call.

Sources

  • Shopify's Admin API defines fulfillment order statuses including OPEN and ON_HOLD, and fulfillment cannot be initiated until a hold is released. Source
  • Shopify's fulfillmentOrderHold mutation changes a fulfillment order's status to ON_HOLD and creates a fulfillment hold. Source
  • Gymnasium's Env API describes reset as returning an initial state with optional seeding, and step as returning separate terminated and truncated flags. Source
  • tau-bench evaluates agents by comparing the database state at the end of a conversation with an annotated goal state. Source
  • AppWorld scores agents with state-based unit tests that allow different ways of completing a task and check for collateral damage. Source
  • NetworkX documents that a topological sort is valid only for graphs with no directed cycles and raises an error otherwise. Source
  • The Object Management Group publishes the Business Process Model and Notation specification as version 2.0.2. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies