Skip to content

AI uses for records

RL environment multi-tenant isolation: many companies, no mixing

By SourceX Editorial · Updated

10 sections

Short answer

To build RL tasks from several companies without mixing them, give each company its own data snapshot, tools, policies, identifiers, simulated users and grader reference, all keyed to a tenant ID the reset sets. Prove it with an isolation test that plants a unique marker in each company's world and fails if any marker crosses, then split training and evaluation data by company.

Grader rubric

RL environment multi tenant

This page covers RL environment multi-tenant isolation: many companies, no mixing.Illustrative

Key takeaways

  • Key every part of an episode's world to a tenant ID: snapshot, tools, policies, identifiers, simulated users and grader reference.
  • Separate processes are not isolation; databases, indexes, caches and vendor sandboxes behind them can still be shared.
  • If you pool state in PostgreSQL, run under a role that owns nothing and cannot bypass row policies, and test constraint side channels.
  • Prove isolation with planted markers in each company's world and re-run the probes after any storage, tool or reset change.
  • Split training and evaluation by company when testing generalization, and check near-duplicates and shared templates across companies.

How do you build RL tasks from several companies without mixing their worlds?

Give every company its own world inside the environment: its own data snapshot, tool bindings, policy text, simulated users, identifiers and grader reference, all keyed to a tenant ID that the reset sets and nothing in the episode can change. Then prove the separation with an isolation test that plants a unique marker in each company's world and fails if any marker appears where it does not belong.

The same discipline applies to training data. An environment can isolate perfectly at run time and still mix companies if the tasks, trajectories and evaluation items are split carelessly, so the isolation test comes with a contamination checklist for splits.

Mixing matters for two reasons. An agent that learns company A's refund rule and applies it in company B's episode is learning the wrong lesson and being graded against the wrong answer. And each company's records usually come under a separate agreement, so one company's data surfacing in another's episode can be a contract problem as well as a modeling one.

In which six ways does one company's world leak into another's episode?

Leaks rarely come from the obvious place. A shared table is easy to spot; a search index, a cache or a simulated user with memory is not. List every component that holds state and ask which company's state it holds during an episode.

In which six ways does one company's world leak into another's episode?
Leak typeWhere it hidesWhat the agent experiencesTypical cause
StateShared database tables, files or queuesSees another company's orders, tickets or customersQueries without a tenant filter, or a role that bypasses it
ToolSearch, retrieval or lookup toolsA search returns another company's documentsOne index built over all companies' content
PolicyPolicy text, business rules, promptsFollows company A's rule inside company B's taskPolicy loaded once per process rather than per episode
MemoryAgent memory, conversation caches, simulated user historyRemembers a customer from a different company's episodeCaches not cleared on reset
IdentityCustomer, order and product IDsOrder 1001 resolves to the wrong company's orderRaw source IDs reused without a tenant prefix
GradingReference outcomes and grader stateCorrect work scored as wrong, or wrong work as rightGrader replays against the wrong company's snapshot

One instance per company, or one shared store with a tenant key?

Use a separate instance per company where you can afford it, and a shared store with an enforced tenant key only where you must, with the isolation test run against whichever you choose. SaaS architects call these the silo and pool models, and the same trade-off applies to environments.

AWS's tenant isolation guidance describes pooled compute running work for many tenants side by side, with storage commonly shared in tables indexed by tenant ID, and notes that the more resources are shared, the more ways one tenant can affect another. Its bridge model mixes the two layer by layer, for example a shared web tier in front of per-tenant business logic and storage. For an environment, a sensible bridge is a shared agent harness in front of per-company state and graders.

Process separation helps but is not isolation on its own. Gymnasium's AsyncVectorEnv runs each sub-environment in its own multiprocessing process and communicates through pipes, so two companies' environments in one vector do not share Python memory. They can still share a database, an index or a vendor sandbox behind those processes.

If you pool state in PostgreSQL, row security is a reasonable guard but has edges you must test. The PostgreSQL documentation says a table with row security enabled and no policy defaults to deny, but superusers and roles with BYPASSRLS always bypass it, table owners normally do too, and referential integrity checks such as unique keys always bypass it, which the documentation warns can leak information through a covert channel. Run the environment under a role that owns nothing and cannot bypass policies.

Why must the reset and the grader know which company they serve?

A reset has to reload the right company's world, and the grader has to compare against that same world. If either one silently defaults to a shared or previous snapshot, the episode is graded against a reality the agent never saw.

The tau-bench reference code shows why. Its reset reloads the environment's data from a loader function, and its reward step reloads the data again, replays the task's ground-truth actions and compares a hash of the resulting data with the hash the agent produced. That pattern is sound for one world. With several companies, the loader must take the tenant ID as an argument, the hash should cover the tenant ID as well as the data, and the reset must also clear the agent's tool sessions, caches and simulated user history.

Record the tenant ID, snapshot version and policy version in every episode log. When an isolation probe fails later, those three fields tell you whether the fault was in the reset, the store or the grader.

Which probes make up the tenant isolation test?

This is the asset: ten probes to run before any training run and after any change to storage, tools or reset code. Each plants or requests something that should be impossible if isolation holds. Run every probe for every pair of companies, not just one pair, because a leak can be specific to one tenant's configuration.

At a glance

Which probes make up the tenant isolation test

Which probes make up the tenant isolation test?
ProbeHow to run itPass conditionA failure points to
Canary rowsInsert a unique marker record into each company's snapshot, such as a customer named with a random tokenNo company's episode can retrieve another company's marker by any toolState or tool leak
ID collisionGive two companies a record with the same raw ID and request it in eachEach episode resolves the ID to its own company's recordIdentifiers without a tenant prefix
Search probeSearch for each company's marker token from every other company's episodeZero hits outside the marker's own companyA shared or unfiltered index
Policy swapGive two companies conflicting rules for the same action and run the same task in eachEach episode enforces its own company's rulePolicy cached per process
Grader swapGrade company A's finished episode with the tenant ID changed to BThe grader refuses or the score changes as expectedGrader ignores tenant ID
Reset residueRun company A, then reset to company B in the same worker and inspect state, caches and memoryNo trace of A's records, messages or tool sessionsIncomplete reset
Concurrent episodesRun all companies at once at full parallelismEvery probe still passes under loadShared connection pools or race conditions
Privilege checkConfirm the environment's database role owns no tables and cannot bypass row policiesThe role is neither owner, superuser nor policy-exemptRow security silently bypassed
Log and trace scanSearch episode logs, traces and stored trajectories for other companies' markersMarkers appear only in their own company's logsLeak into training artifacts
Constraint side channelTry to create a record whose unique key matches another company's recordThe error reveals nothing about the other companyShared unique constraints across tenants

How should training and evaluation splits be checked for cross-company contamination?

Split by company first and by task second, and decide in writing whether the evaluation asks if the agent generalizes to a new company or performs well inside known ones. Those are different questions and need different splits.

  • Group split: assign each company wholly to train or to evaluation when the question is generalization to unseen companies; never let one company's tasks land on both sides by random row sampling.
  • Declared within-company split: when the question is performance inside known companies, split by time inside each company and record that the evaluation companies were seen in training.
  • Near-duplicate check: compare evaluation items with training items using overlapping word sequences; the GPT-3 authors flagged benchmark items that shared a 13-gram (a run of 13 words) with the pretraining data and reported results on clean subsets without them.
  • Shared templates: look for vendor email templates, form letters and standard terms that appear across companies and would let a model recognize a task without learning it.
  • Canary strings: place a unique marker string in each company's evaluation material, as BIG-bench does in every task file to keep tasks out of web-scraped training data, then test whether trained models reproduce it.
  • Trajectory stores: check that recorded episodes from evaluation companies are not fed back into training through a replay buffer or a fine-tuning export.
  • Policy documents: keep each company's policy text out of shared pretraining or retrieval corpora unless that company's agreement permits it.
  • Version log: record which snapshot, task set and split each model was trained and evaluated on, so a later leak can be traced to the runs it affected.

Which setup changes turn a passing isolation test into a failing one?

An isolation test only holds for the setup it ran on. These changes most often break a previously passing test, so re-run the full probe set after any of them.

Moving from one worker per company to a shared worker pool brings back reset residue and connection reuse. Switching the database role, for example to a migration role with owner rights, can bypass row policies without any code change. Rebuilding a retrieval index over all companies' content for speed recreates the search leak. Upgrading the simulated user to a version with conversation memory adds a memory channel the old one did not have.

Vendor sandboxes deserve their own line. If two companies' episodes call the same sandbox account of a payment or ticketing service, isolation now depends on that vendor's data model, which you do not control. Use one sandbox account per company or stub the service.

What do the probes catch when two illustrative distributors share one returns environment?

This example is illustrative; the companies, counts and outcomes are invented to show the method. Two wholesale distributors each license a year of returns tickets, order records and their written returns policies. The environment builds return-handling tasks from both.

The first isolation run fails two probes. The search probe finds distributor A's marker customer from distributor B's episodes, because one embedding index covered both companies' knowledge-base articles. The ID collision probe fails on order numbers, since both distributors number orders from the same starting value. The team splits the index per company, prefixes every ID with a tenant code, and re-runs all ten probes, which then pass.

The contamination check then finds that about one in ten of distributor B's evaluation tickets (illustrative) quote a carrier's standard damage-claim text also present in distributor A's training tickets. Because the evaluation question is generalization to an unseen distributor, those tickets are kept but the shared passage is masked in both splits, and the masking is recorded in the version log.

What if crossing companies is the point of the task?

Some tasks are cross-company by nature: a buyer and a supplier exchanging a purchase order, a broker matching a shipper with a carrier, a marketplace resolving a dispute between two sellers. Isolating the two companies completely would make the task impossible.

In those cases, define the shared surface explicitly, such as the purchase order and the messages exchanged, and isolate everything else. The isolation test then checks that only the declared fields cross the boundary. Get each company's written agreement to the exact shared fields before building the task, because the right to use a record in one company's environment does not by itself extend to showing it to another.

Stop and narrow the scope if a company's records cannot be separated cleanly from another's, for example a shared mailbox or a reseller's system holding several clients' data. Bring in the business process owner to confirm which records belong to whom, and counsel if the agreements differ on permitted use.

How do licensed records from several companies arrive for an environment?

Multi-company environments are only as clean as the records behind them, and each company's records arrive under its own agreement. Through SourceX, every contributing company signs its own agreement, after approving its own scope and terms, before any of its records are delivered, so the environment team gets a dated boundary per company to hang the tenant key on.

An environment builder who wants several companies' workflows sets out the scope, volume, language, format, timing and licensing needs in SourceX's request-data form, and SourceX sources the records from U.S. businesses for that request rather than from anything it already holds.

Frequently asked questions

What does multi-tenant isolation mean in an RL environment?

It means each company whose records feed the environment has its own state, tools, policies, identifiers and grader reference, and nothing from one company's world can reach another company's episode. It is proven with probes that plant unique markers and fail if any marker crosses over.

Is running each environment in a separate process enough?

No. Separate processes stop shared memory, but the processes can still read the same database, search index, cache or vendor sandbox. The isolation test has to probe every store behind the process, not just the process boundary.

Can PostgreSQL row-level security isolate tenants in an environment?

It can help, but only if the environment's role does not own the tables, is not a superuser and lacks BYPASSRLS. PostgreSQL also documents that unique and foreign key checks bypass row security, so test for side channels through constraint errors.

Should the same company appear in both training and evaluation?

Only if the evaluation question is performance inside known companies, and then split by time and say so. To test generalization to new companies, keep each company wholly on one side of the split.

What is a canary string and why use one per company?

A canary is a unique marker placed in data that should never reach a model or another tenant. BIG-bench places one in every task file to keep tasks out of web-scraped training data; one canary per company lets you trace which company's material leaked.

Sources

  • In a pooled model, tenants run side by side on shared compute and storage is commonly shared in tables indexed by tenant ID; the more that is shared, the more one tenant can affect another. Source
  • The bridge model mixes silo and pool, for example a shared web tier in front of per-tenant business logic and storage. Source
  • Gymnasium's AsyncVectorEnv runs multiple environments in parallel using multiprocessing processes and pipes for communication. Source
  • PostgreSQL row security defaults to deny when no policy exists; superusers and BYPASSRLS roles always bypass it, table owners normally do, and referential integrity checks bypass it and can leak information through covert channels. Source
  • tau-bench's reset reloads environment data from a loader function, and its reward reloads the data, replays ground-truth actions and compares data hashes. Source
  • The GPT-3 paper built clean benchmark subsets by removing examples with a 13-gram overlap with the pretraining data. Source
  • BIG-bench task files contain a canary string intended to prevent benchmark tasks from leaking into web-scraped training data. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies