AI uses for records
RL environment hidden information: testing agents on facts revealed later
By SourceX Editorial · Updated
10 sections
Short answer
Build a staged disclosure task: keep the full truth in hidden environment state, release each fact only when a defined trigger fires (an agent action, a clock tick, a counterparty reply or a change notification), and log what the agent could observe at every step. Then grade the final records and the moment the agent committed, so acting before the needed fact arrived counts as a failure even when the guess was right.
Grader rubric
RL environment hidden
Key takeaways
- Keep every fact in hidden state from reset, and release scheduled facts only through a written trigger: an action, a clock, a counterparty reply or a change notification.
- Separate ground truth the grader uses, facts scheduled for later and facts the agent must never see; each needs a different test.
- Run leak tests before scoring, starting with the info channel that many environments use for hidden variables.
- List the facts each commit action requires and score commits made before those facts were observable, even if the guess was right.
- Record the patience budget, waiting cost and seed with every result, because they move scores more than agent quality does.
How do you test agents working with information that becomes available later?
You make the arrival of information part of the environment's design, not an accident of the data. The environment holds every fact the task depends on from the start, but the agent's observation contains only what has been disclosed so far, and a written schedule says which trigger releases which fact.
That turns a vague question, whether the agent copes with uncertainty, into checkable ones. Did it ask for the missing fact or wait for it instead of inventing it? Did it hold back irreversible actions until the fact that justifies them was observable? When the fact arrived, did it revise earlier work that the fact changed?
This page sets out the staged disclosure task template, the triggers that real tool protocols already provide, a worked example with its stage table, the leak tests that prove the agent cannot see ahead, and the scoring and exceptions that keep the test honest.
Hidden, not yet disclosed, or never disclosable?
Separate three kinds of withheld information before writing any task, because each one tests something different. Hidden ground truth is what the grader uses and the agent never sees, such as the correct final state. Scheduled information is hidden now but will be observable after a trigger, such as a supplier's reply. Undisclosable information is something the agent must never learn in this role, such as another customer's record, and it belongs to a permission test, not a timing test.
Reinforcement learning interfaces already allow for this split. Gymnasium's core environment class notes that an environment can be partially or fully observed, and its step method returns an observation alongside an info dictionary that may contain variables hidden from observations. That second channel is useful for logging and grading and dangerous for staged disclosure: if an agent wrapper passes info to the policy, every scheduled fact leaks on step one.
The staged disclosure task template
Each task is defined by the fields below before anyone runs an agent against it. The example column describes an illustrative supplier onboarding task; replace it with your own process and keep the acceptance checks.
Two fields do most of the work. The disclosure schedule is the contract between the task author and the grader: it names each scheduled fact, the trigger that releases it and the earliest step it can appear, so nobody argues afterward about whether the agent could have known. The commit list ties each irreversible action to the facts that justify it, which is what lets the grader call an action premature without reading the agent's reasoning.
Validate the task itself before using it. Solve it by hand or with a scripted policy that follows the schedule, confirm the success invariants hold at the end, and confirm that a scripted policy which commits at step one fails. A task that cannot be solved inside its budget, or that a reckless policy passes, measures nothing.
| Field | What it holds | Illustrative example | Acceptance check |
|---|---|---|---|
| Hidden state | Every fact the task depends on, disclosed or not | Supplier record, tax form status, sanctions screen result, approver's answer | Grader can compute success from hidden state alone |
| Observation function | What the agent can see at each step | Inbox, supplier portal fields already submitted, tool results | No hidden field reachable through any observation or tool |
| Disclosure schedule | Fact, trigger and earliest step for each scheduled fact | Tax form appears after the agent requests it and five steps pass | Every scheduled fact has exactly one trigger and is reachable |
| Allowed actions | Tools and arguments the agent may use | Request document, check status, draft record, ask approver, activate supplier | Each action's effect on state is specified |
| Commit actions | Actions that cannot be cleanly undone | Activate supplier, send confirmation to the supplier | Each commit lists the facts that must be observed first |
| Reset and seed | How the episode starts and what randomness exists | Fixed seed for reply delays; snapshot ID for records | Two resets with the same seed give the same schedule |
| Patience budget | Step or time limit before truncation | Twenty steps or five simulated business days | Budget allows the slowest scheduled fact to arrive |
| Success invariants | Conditions true at the end of a good episode | Supplier active only if tax form valid and screen clear | Checked by query over final state |
| Replay log | Ordered observations, actions, disclosures and commits | Step, time, observation hash, action, disclosed fact IDs | A second team can rebuild what the agent knew at each step |
Four triggers that release information, and where each comes from
Use triggers that mirror how facts reach an agent in production, because each one exercises a different habit. Real tool protocols already give the patterns.
- Counterparty disclosure. A simulated user or supplier reveals facts only when asked. The tau-bench user simulator is instructed not to give away the whole instruction at once, to provide only what the current step needs and not to invent details it was not given, such as an order ID. This trigger tests whether the agent asks the right question.
- Server request for more input. In the Model Context Protocol's multi round-trip pattern, a server that needs more information returns a request for it, the client gathers the answer from the user or another source and retries the original request. The fact does not exist for the agent until that round trip completes.
- Change notification. MCP subscriptions let a client receive resource-updated notifications for the resources it subscribed to. In a task, a document or status changing mid-episode tests whether the agent re-reads and revises.
- Clock. Some facts arrive after time passes regardless of what the agent does, such as a background check result. A pinned simulated clock with a seeded delay makes these reproducible.
Worked example (illustrative): the tax form that arrives on step six
The task asks the agent to onboard a new supplier. At reset, the hidden state already contains a valid tax form and a clear sanctions screen, but neither is observable. The tax form is released when the supplier replies, five steps after the agent requests it; the screen result is released when the agent calls the screening tool and one simulated day passes.
The stage table shows what a well-behaved agent sees and does. The failure column is what the grader looks for at each stage. The scenario, steps and timings are illustrative.
| Step | Observable to the agent | Still hidden | Correct behavior | Premature-commit failure |
|---|---|---|---|---|
| 1 | Request from buyer team; supplier name and contact | Tax form, screen result | Request tax form; start screening | Activating the supplier from the name alone |
| 2-3 | Request acknowledged; screen pending | Tax form, screen result | Draft the record; mark fields pending | Filling tax fields with guessed values |
| 4 | Screen result: clear | Tax form | Record the result with its time | Treating a clear screen as enough to activate |
| 5 | Nothing new | Tax form | Wait or send one reminder within the budget | Repeated reminders or inventing a form number |
| 6 | Tax form received | Approver's decision | Validate the form; ask the approver | Activating without approval |
| 7 | Approver approves | Nothing task-relevant | Activate and confirm to the supplier | Confirming before activation succeeds |
Leak tests: proving the agent cannot see what it should not
Run leak tests on the environment before any agent scores are trusted. A staged task that leaks is worse than none, because it rewards agents for reading the answer key.
At a glance
Leak tests: proving the agent cannot see what it should not
- Info channel: confirm the agent wrapper never passes the environment's info dictionary, grader fields or hidden state into the prompt or observation.
- Tool side doors: call every read tool with every plausible argument at step one and confirm no scheduled fact comes back, including through error messages.
- Identifiers and timestamps: check that sequential IDs, file names or modification times do not reveal that a later document already exists.
- Simulator discipline: sample counterparty turns and confirm the simulator does not volunteer scheduled facts early or invent facts absent from its instruction.
- Reset determinism: Gymnasium documents that reset with a seed initializes the environment's random generator, and that calling reset without a seed later does not re-seed it. Pass the seed explicitly on every evaluation reset so disclosure delays repeat.
- Replay: rebuild the agent's knowledge at each step from the log and confirm it matches the disclosure schedule.
Patience budgets, waiting costs and seeds: the assumptions that swing scores
Several design choices move scores more than agent quality does, so record them with every result. The patience budget is the largest: if the step limit ends the episode before the slowest fact can arrive, every agent fails for a reason that has nothing to do with judgment. Gymnasium separates terminated, meaning the task reached a terminal state, from truncated, meaning a limit outside the task cut it short; keep that distinction in your logs so a timeout is not scored as a wrong decision.
Whether waiting has a cost matters too. If waiting is free, the safest policy is to stall; if every step is penalized, agents learn to guess. Set the cost from the real process, for example a supplier left inactive for days has a business cost, and document it. Disclosure randomness, the counterparty simulator's verbosity and whether the agent may ask the same question twice all change outcomes and belong in the task card.
Scoring the final state and the moment of commitment
Grade from the records, then add timing. tau-bench grades an episode by comparing the final database state with the annotated goal state, which means a fluent closing message earns nothing if the records are wrong. Staged disclosure needs one more signal: for each commit action, was every fact on its prerequisite list observable when the agent took it?
Report final-state success, premature commits, fabricated values written before disclosure, revisions made after late facts and wasted steps. Run each task several times. The tau-bench authors introduced pass^k, the chance that all of k independent trials of a task succeed, because agents that succeed once often fail on repeats; a staged task with seeded delays is exactly where that inconsistency shows up.
When acting before the information arrives is the right call
Not every early action is a failure, and the task set should include cases that prove it. Reversible, provisional steps are often correct: placing a hold, drafting a record with fields marked pending or notifying a person that a decision is waiting. Some processes also have deadlines where acting on the best available evidence, with the uncertainty stated, beats waiting.
Encode these as explicit exceptions in the commit list, with the deadline or reversibility that justifies them, and include at least one task where waiting is the failure. If a real process has no clear rule for when acting early is acceptable, stop and ask the business process owner to write one before building the task; the environment cannot grade a judgment nobody has defined.
Where real disclosure schedules come from
The most believable schedules come from real case histories: when supplier documents actually arrived, how long approvals took, which replies changed the outcome. Business records such as onboarding tickets, procurement threads and approval logs carry that timing, so licensed records give more believable schedules than invented delays.
Teams that want real schedules can request them through SourceX, which sources records from U.S. businesses on request; the request form captures the scope, volume, language, format, timing and licensing terms. Rights, privacy, quality and buyer-fit checks apply to every dataset before it is shared.
Frequently asked questions
What is hidden information in an RL environment?
It is environment state that affects outcomes but is not in the agent's current observation. Some of it is ground truth only the grader sees; some is scheduled to become observable after a trigger. Staged disclosure tasks make the second kind explicit.
Is a staged disclosure task the same as a POMDP?
It is a practical case of a partially observable problem: the agent acts on observations rather than full state. The difference is that the disclosure schedule is written down per fact, so a grader can tell whether an action was taken before its justifying fact was observable.
How do you stop the agent from seeing scheduled facts early?
Run leak tests before scoring: keep the info channel and grader fields out of the prompt, probe every read tool at step one, check identifiers and timestamps for hints, sample the counterparty simulator and replay the log against the schedule.
Should waiting be penalized?
Only as much as waiting costs in the real process. Free waiting rewards stalling and heavy step penalties reward guessing. Set the cost from the business case and record it with the results, because it changes scores.
What should the staged disclosure task with observable state boundaries include?
Hidden state, an observation function, a disclosure schedule with one trigger per fact, allowed and commit actions with prerequisite facts, a seeded reset, a patience budget, success invariants checked over final state and a replay log that rebuilds what the agent knew at each step.
Sources
- Gymnasium's Env class says an environment can be partially or fully observed; step returns an info dictionary that may contain variables hidden from observations, separates terminated from truncated, and reset with a seed initializes the random generator while a later reset without a seed does not re-seed it. Source
- The tau-bench user simulator prompt tells the simulated user not to give away all the instruction at once, to provide only the information needed for the current step and not to invent information absent from the instruction. Source
- tau-bench computes an episode's reward by comparing a hash of the final database state with the hash of the ground-truth state. Source
- The tau-bench paper evaluates agents by comparing the end-of-conversation database state with an annotated goal state and proposes pass^k to measure reliability across repeated trials. Source
- MCP's multi round-trip request pattern lets a server respond that it needs additional information, after which the client gathers it from the user or other sources and retries the original request. Source
- MCP's subscriptions/listen stream delivers notifications the client asked for, including resource-updated notifications for subscribed resource URIs. Source
Tell us what data you need
Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.