AI uses for records
RL environment multi-tenant isolation: many companies, no mixing
By SourceX Editorial · Updated
10 sections
Short answer
To build RL tasks from several companies without mixing them, give each company its own data snapshot, tools, policies, identifiers, simulated users and grader reference, all keyed to a tenant ID the reset sets. Prove it with an isolation test that plants a unique marker in each company's world and fails if any marker crosses, then split training and evaluation data by company.
Grader rubric
RL environment multi tenant
Key takeaways
- Key every part of an episode's world to a tenant ID: snapshot, tools, policies, identifiers, simulated users and grader reference.
- Separate processes are not isolation; databases, indexes, caches and vendor sandboxes behind them can still be shared.
- If you pool state in PostgreSQL, run under a role that owns nothing and cannot bypass row policies, and test constraint side channels.
- Prove isolation with planted markers in each company's world and re-run the probes after any storage, tool or reset change.
- Split training and evaluation by company when testing generalization, and check near-duplicates and shared templates across companies.
How do you build RL tasks from several companies without mixing their worlds?
Give every company its own world inside the environment: its own data snapshot, tool bindings, policy text, simulated users, identifiers and grader reference, all keyed to a tenant ID that the reset sets and nothing in the episode can change. Then prove the separation with an isolation test that plants a unique marker in each company's world and fails if any marker appears where it does not belong.
The same discipline applies to training data. An environment can isolate perfectly at run time and still mix companies if the tasks, trajectories and evaluation items are split carelessly, so the isolation test comes with a contamination checklist for splits.
Mixing matters for two reasons. An agent that learns company A's refund rule and applies it in company B's episode is learning the wrong lesson and being graded against the wrong answer. And each company's records usually come under a separate agreement, so one company's data surfacing in another's episode can be a contract problem as well as a modeling one.
In which six ways does one company's world leak into another's episode?
Leaks rarely come from the obvious place. A shared table is easy to spot; a search index, a cache or a simulated user with memory is not. List every component that holds state and ask which company's state it holds during an episode.
| Leak type | Where it hides | What the agent experiences | Typical cause |
|---|---|---|---|
| State | Shared database tables, files or queues | Sees another company's orders, tickets or customers | Queries without a tenant filter, or a role that bypasses it |
| Tool | Search, retrieval or lookup tools | A search returns another company's documents | One index built over all companies' content |
| Policy | Policy text, business rules, prompts | Follows company A's rule inside company B's task | Policy loaded once per process rather than per episode |
| Memory | Agent memory, conversation caches, simulated user history | Remembers a customer from a different company's episode | Caches not cleared on reset |
| Identity | Customer, order and product IDs | Order 1001 resolves to the wrong company's order | Raw source IDs reused without a tenant prefix |
| Grading | Reference outcomes and grader state | Correct work scored as wrong, or wrong work as right | Grader replays against the wrong company's snapshot |
One instance per company, or one shared store with a tenant key?
Use a separate instance per company where you can afford it, and a shared store with an enforced tenant key only where you must, with the isolation test run against whichever you choose. SaaS architects call these the silo and pool models, and the same trade-off applies to environments.
AWS's tenant isolation guidance describes pooled compute running work for many tenants side by side, with storage commonly shared in tables indexed by tenant ID, and notes that the more resources are shared, the more ways one tenant can affect another. Its bridge model mixes the two layer by layer, for example a shared web tier in front of per-tenant business logic and storage. For an environment, a sensible bridge is a shared agent harness in front of per-company state and graders.
Process separation helps but is not isolation on its own. Gymnasium's AsyncVectorEnv runs each sub-environment in its own multiprocessing process and communicates through pipes, so two companies' environments in one vector do not share Python memory. They can still share a database, an index or a vendor sandbox behind those processes.
If you pool state in PostgreSQL, row security is a reasonable guard but has edges you must test. The PostgreSQL documentation says a table with row security enabled and no policy defaults to deny, but superusers and roles with BYPASSRLS always bypass it, table owners normally do too, and referential integrity checks such as unique keys always bypass it, which the documentation warns can leak information through a covert channel. Run the environment under a role that owns nothing and cannot bypass policies.
Why must the reset and the grader know which company they serve?
A reset has to reload the right company's world, and the grader has to compare against that same world. If either one silently defaults to a shared or previous snapshot, the episode is graded against a reality the agent never saw.
The tau-bench reference code shows why. Its reset reloads the environment's data from a loader function, and its reward step reloads the data again, replays the task's ground-truth actions and compares a hash of the resulting data with the hash the agent produced. That pattern is sound for one world. With several companies, the loader must take the tenant ID as an argument, the hash should cover the tenant ID as well as the data, and the reset must also clear the agent's tool sessions, caches and simulated user history.
Record the tenant ID, snapshot version and policy version in every episode log. When an isolation probe fails later, those three fields tell you whether the fault was in the reset, the store or the grader.
Which probes make up the tenant isolation test?
This is the asset: ten probes to run before any training run and after any change to storage, tools or reset code. Each plants or requests something that should be impossible if isolation holds. Run every probe for every pair of companies, not just one pair, because a leak can be specific to one tenant's configuration.
At a glance
Which probes make up the tenant isolation test
| Probe | How to run it | Pass condition | A failure points to |
|---|---|---|---|
| Canary rows | Insert a unique marker record into each company's snapshot, such as a customer named with a random token | No company's episode can retrieve another company's marker by any tool | State or tool leak |
| ID collision | Give two companies a record with the same raw ID and request it in each | Each episode resolves the ID to its own company's record | Identifiers without a tenant prefix |
| Search probe | Search for each company's marker token from every other company's episode | Zero hits outside the marker's own company | A shared or unfiltered index |
| Policy swap | Give two companies conflicting rules for the same action and run the same task in each | Each episode enforces its own company's rule | Policy cached per process |
| Grader swap | Grade company A's finished episode with the tenant ID changed to B | The grader refuses or the score changes as expected | Grader ignores tenant ID |
| Reset residue | Run company A, then reset to company B in the same worker and inspect state, caches and memory | No trace of A's records, messages or tool sessions | Incomplete reset |
| Concurrent episodes | Run all companies at once at full parallelism | Every probe still passes under load | Shared connection pools or race conditions |
| Privilege check | Confirm the environment's database role owns no tables and cannot bypass row policies | The role is neither owner, superuser nor policy-exempt | Row security silently bypassed |
| Log and trace scan | Search episode logs, traces and stored trajectories for other companies' markers | Markers appear only in their own company's logs | Leak into training artifacts |
| Constraint side channel | Try to create a record whose unique key matches another company's record | The error reveals nothing about the other company | Shared unique constraints across tenants |
How should training and evaluation splits be checked for cross-company contamination?
Split by company first and by task second, and decide in writing whether the evaluation asks if the agent generalizes to a new company or performs well inside known ones. Those are different questions and need different splits.
- Group split: assign each company wholly to train or to evaluation when the question is generalization to unseen companies; never let one company's tasks land on both sides by random row sampling.
- Declared within-company split: when the question is performance inside known companies, split by time inside each company and record that the evaluation companies were seen in training.
- Near-duplicate check: compare evaluation items with training items using overlapping word sequences; the GPT-3 authors flagged benchmark items that shared a 13-gram (a run of 13 words) with the pretraining data and reported results on clean subsets without them.
- Shared templates: look for vendor email templates, form letters and standard terms that appear across companies and would let a model recognize a task without learning it.
- Canary strings: place a unique marker string in each company's evaluation material, as BIG-bench does in every task file to keep tasks out of web-scraped training data, then test whether trained models reproduce it.
- Trajectory stores: check that recorded episodes from evaluation companies are not fed back into training through a replay buffer or a fine-tuning export.
- Policy documents: keep each company's policy text out of shared pretraining or retrieval corpora unless that company's agreement permits it.
- Version log: record which snapshot, task set and split each model was trained and evaluated on, so a later leak can be traced to the runs it affected.
Which setup changes turn a passing isolation test into a failing one?
An isolation test only holds for the setup it ran on. These changes most often break a previously passing test, so re-run the full probe set after any of them.
Moving from one worker per company to a shared worker pool brings back reset residue and connection reuse. Switching the database role, for example to a migration role with owner rights, can bypass row policies without any code change. Rebuilding a retrieval index over all companies' content for speed recreates the search leak. Upgrading the simulated user to a version with conversation memory adds a memory channel the old one did not have.
Vendor sandboxes deserve their own line. If two companies' episodes call the same sandbox account of a payment or ticketing service, isolation now depends on that vendor's data model, which you do not control. Use one sandbox account per company or stub the service.
What do the probes catch when two illustrative distributors share one returns environment?
This example is illustrative; the companies, counts and outcomes are invented to show the method. Two wholesale distributors each license a year of returns tickets, order records and their written returns policies. The environment builds return-handling tasks from both.
The first isolation run fails two probes. The search probe finds distributor A's marker customer from distributor B's episodes, because one embedding index covered both companies' knowledge-base articles. The ID collision probe fails on order numbers, since both distributors number orders from the same starting value. The team splits the index per company, prefixes every ID with a tenant code, and re-runs all ten probes, which then pass.
The contamination check then finds that about one in ten of distributor B's evaluation tickets (illustrative) quote a carrier's standard damage-claim text also present in distributor A's training tickets. Because the evaluation question is generalization to an unseen distributor, those tickets are kept but the shared passage is masked in both splits, and the masking is recorded in the version log.
What if crossing companies is the point of the task?
Some tasks are cross-company by nature: a buyer and a supplier exchanging a purchase order, a broker matching a shipper with a carrier, a marketplace resolving a dispute between two sellers. Isolating the two companies completely would make the task impossible.
In those cases, define the shared surface explicitly, such as the purchase order and the messages exchanged, and isolate everything else. The isolation test then checks that only the declared fields cross the boundary. Get each company's written agreement to the exact shared fields before building the task, because the right to use a record in one company's environment does not by itself extend to showing it to another.
Stop and narrow the scope if a company's records cannot be separated cleanly from another's, for example a shared mailbox or a reseller's system holding several clients' data. Bring in the business process owner to confirm which records belong to whom, and counsel if the agreements differ on permitted use.
How do licensed records from several companies arrive for an environment?
Multi-company environments are only as clean as the records behind them, and each company's records arrive under its own agreement. Through SourceX, every contributing company signs its own agreement, after approving its own scope and terms, before any of its records are delivered, so the environment team gets a dated boundary per company to hang the tenant key on.
An environment builder who wants several companies' workflows sets out the scope, volume, language, format, timing and licensing needs in SourceX's request-data form, and SourceX sources the records from U.S. businesses for that request rather than from anything it already holds.
Frequently asked questions
What does multi-tenant isolation mean in an RL environment?
It means each company whose records feed the environment has its own state, tools, policies, identifiers and grader reference, and nothing from one company's world can reach another company's episode. It is proven with probes that plant unique markers and fail if any marker crosses over.
Is running each environment in a separate process enough?
No. Separate processes stop shared memory, but the processes can still read the same database, search index, cache or vendor sandbox. The isolation test has to probe every store behind the process, not just the process boundary.
Can PostgreSQL row-level security isolate tenants in an environment?
It can help, but only if the environment's role does not own the tables, is not a superuser and lacks BYPASSRLS. PostgreSQL also documents that unique and foreign key checks bypass row security, so test for side channels through constraint errors.
Should the same company appear in both training and evaluation?
Only if the evaluation question is performance inside known companies, and then split by time and say so. To test generalization to new companies, keep each company wholly on one side of the split.
What is a canary string and why use one per company?
A canary is a unique marker placed in data that should never reach a model or another tenant. BIG-bench places one in every task file to keep tasks out of web-scraped training data; one canary per company lets you trace which company's material leaked.
Sources
- In a pooled model, tenants run side by side on shared compute and storage is commonly shared in tables indexed by tenant ID; the more that is shared, the more one tenant can affect another. Source
- The bridge model mixes silo and pool, for example a shared web tier in front of per-tenant business logic and storage. Source
- Gymnasium's AsyncVectorEnv runs multiple environments in parallel using multiprocessing processes and pipes for communication. Source
- PostgreSQL row security defaults to deny when no policy exists; superusers and BYPASSRLS roles always bypass it, table owners normally do, and referential integrity checks bypass it and can leak information through covert channels. Source
- tau-bench's reset reloads environment data from a loader function, and its reward reloads the data, replays ground-truth actions and compares data hashes. Source
- The GPT-3 paper built clean benchmark subsets by removing examples with a 13-gram overlap with the pretraining data. Source
- BIG-bench task files contain a canary string intended to prevent benchmark tasks from leaking into web-scraped training data. Source
Tell us what data you need
Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.