Skip to content

AI uses for records

Spatial reasoning operational datasets: tasks bound by real site rules

By SourceX Editorial · Updated

10 sections

Short answer

To ground spatial reasoning tasks in real operating constraints, build each task from one moment in a real operation: the layout snapshot with declared units and frames, the rules in force then (clearances, exit routes, capacity, time windows, holds), the action actually taken and what followed. Ask for a permitted route or placement, score feasibility before agreement with the record, and flag records that break the rules.

Video and sensors

Spatial reasoning

This page covers spatial reasoning operational datasets: tasks bound by real site rules.Illustrative

Key takeaways

  • A spatial task is grounded when its correct answers are fixed by rules the site actually enforced at that moment, not by what looks plausible in an image.
  • Every task carries a versioned constraint set, each rule tied to the record or document it came from.
  • Score in two steps: is the answer permitted, then does it match or beat the recorded choice; the recorded choice is evidence, not the optimum.
  • Declare units, coordinate frames and clocks in the task itself; a frame mix-up turns a permitted placement into a violation.
  • When the record breaks a rule, decide whether it is source error, a missing observation or a real violation before the task enters a set.

How do you ground spatial reasoning tasks in real operating constraints?

You anchor each task to one recorded moment in a real operation and let that moment's rules decide which answers are allowed. A placement question is not "where would this pallet fit" but "given this layout, this load, these clearances and this capacity at 14:20 on this shift, which slots are permitted, and which did the team use."

That requires four linked pieces from the operator's own systems: a state (layout, occupancy, equipment positions), a constraint set (written rules and live limits such as holds or closed aisles), an action (the route or placement actually executed) and a consequence (what happened next: a clean pick, a relocation, a blocked aisle, a missed time window). A task without the constraint set tests guessing; a task without the consequence cannot tell a good permitted answer from a bad one.

The rest of this page specifies that task format, shows where each constraint comes from and covers the hard part: records that contradict the rules they were made under.

Why scene-only spatial benchmarks leave out the rules

Most spatial benchmarks ask about geometry a model can see, such as distance, size or relative position, and leave out the operating rules that decide what is allowed. VSI-Bench, for example, builds over 5,000 question-answer pairs from about 290 real indoor videos, and its authors identify spatial reasoning rather than perception as the main bottleneck for multimodal models.

Constraints change the problem. SSI-Bench, which asks about structures whose arrangement is fixed by geometry, topology and physical feasibility, reports the strongest closed-source model at 33.6 percent accuracy against 91.6 percent for humans on its 1,000 ranking questions. Operations add another layer those benchmarks do not carry: rules that are written down, change by shift and are enforced by people with consequences. Routing a forklift through an aisle that is open in the image but closed for a spill that morning is a wrong answer only the record can reveal.

Which operating constraints decide the answer

The constraints worth encoding are the ones a supervisor would cite when rejecting a move. Many already exist in written form. In U.S. general industry, OSHA requires safe clearances for aisles where mechanical handling equipment is used and stable, secure stacking; exit routes must stay free with no materials placed in them even temporarily; and sprinkler systems need at least 18 inches of vertical clearance to material below. Routing tools encode the same kind of limits: Google's OR-Tools treats vehicle load and time windows as dimensions the solver must keep within bounds.

These rules are examples of the type, not the full set. A site's own rules, its fire code edition, racking load ratings and customer handling requirements govern, and the task should cite whichever document the site actually follows.

Which operating constraints decide the answer
Constraint typeTypical operating recordHow it is encoded in the taskHard or soft
Aisle clearance for equipmentFacility layout drawing, equipment turning radiusMinimum free width per aisle segment, by equipment classHard
Exit routesEmergency plan and marked egress pathsPolygons where no placement or staging is permitted at any timeHard
Sprinkler clearanceFire protection drawings, ceiling and head heightsMaximum load top height per zoneHard
Slot capacity and weightWMS location master, racking load ratingsPer-slot dimensions and maximum weightHard
Holds and closuresWMS holds, maintenance tickets, spill reportsTime-bounded exclusions on slots or aisle segmentsHard while active
Vehicle loadVehicle or equipment masterCumulative load limit along the routeHard
Time windowsDock appointments, order cut-offsEarliest and latest arrival per stopHard or soft, as the site treats it
Handling preferencesSlotting rules, velocity class, product familyRanked preferences that break ties among permitted answersSoft

The permitted routing and placement task specification

This is the asset: a field-level specification for one task record. A team can implement it as a JSON or Parquet schema as is. Each record answers one question at one moment; a shift with many moves becomes many records that share a layout snapshot.

At a glance

The permitted routing and placement task specification

The permitted routing and placement task specification
FieldContentRequiredValidation rule
task_id and task_type; as_ofStable ID; route, placement or sequence. Timestamp of the decision moment with UTC offsetYesUnique; type from a fixed list. Falls inside the snapshot and constraint validity windows
layout_snapshot; occupancy_stateReference to geometry: aisles, slots, zones, doors, with units and frame. Which slots and aisle segments are occupied or blocked at as_ofYesUnits declared; frame named; snapshot version dated. Reconciles with WMS inventory within a stated tolerance
agent and objectEquipment class and load, or item dimensions and weightYesUnits declared; values within equipment and item master ranges
start and goalOrigin, destination or candidate slot listYesAll locations exist in the layout snapshot
constraint_setRules in force, each with type, parameters, source document and versionYesEvery hard rule has a source; no rule without a validity window
permitted_answersSet of routes or slots that satisfy every hard rule, computed by a checkerYesRecomputed by an independent checker; non-empty or flagged infeasible
recorded_actionRoute or slot the operation actually used, from system logsYesTimestamped; linked to a log event ID
consequenceWhat followed within a stated horizon: relocation, delay, blocked aisle, on-time pickYesHorizon declared; outcome from a system event, not an assumption
evidence_flags; record_statusOcclusion, sensor gap, manual override or late scan, each naming the affected field. Consistent, source error, missing observation or real violationYesFlags present when evidence is weak. Status set by the diagnosis rules below; real violations never used as positive examples
rights; provenanceLicensing scope, client inventory restrictions, security redactions. Source systems, export dates and transformation stepsYesMatches the license schedule for the source site. Every field traceable to a system and export

Units, frames and clocks the task must declare

A placement that is permitted in meters can be a violation if a reader assumes feet, and a route that clears an aisle in one coordinate frame can clip a rack in another. Declare all three in every record rather than in a separate readme.

Robotics practice gives a usable default. ROS convention REP 103 standardizes on SI units, meters for length, and right-handed coordinate systems. REP 105 explains why the frame matters: a pose in the map frame should not drift but can jump when localization updates, while a pose in the odom frame is continuous but drifts without bound. For constraint checks against fixed geometry such as exit routes and rack faces, use a map-type frame tied to the layout drawing, and record the transform from any sensor frame.

Clocks are the third trap. WMS events, equipment telemetry and camera timestamps often come from different clocks and time zones. Store as_of with its UTC offset and record the measured offset between sources, so a hold that ended at 14:00 is not applied to a move at 13:58 on a fast clock.

Worked example: an illustrative pallet put-away task

This example is illustrative; the site, dimensions and outcome are invented to show the specification. A distribution center must put away one pallet, 1.2 m long, 1.0 m wide, 1.9 m tall and 700 kg, at 14:20. The task offers four candidate slots.

Slot A is high bay space where, after the pallet is placed, its top would sit about 30 cm below the sprinkler heads, inside the 18-inch clearance, so it is not permitted. Slot B is a floor location inside a marked exit route that staff sometimes use for overflow; it is not permitted at any time. Slot C fits by size but is rated for 500 kg, so it is not permitted. Slot D fits by size, weight and clearance, so it is permitted. The recorded action was slot D, and the consequence field shows the pallet was picked two days later without relocation.

A model that answers D passes both checks. A model that answers A has made a spatial error a scene-only task would not catch, because the slot looks open.

  • permitted_answers: D
  • recorded_action: D, log event linked
  • consequence: picked after two days, no relocation, horizon seven days
  • record_status: consistent
  • evidence_flags: none

When the record breaks the rules: error, gap or real violation?

Some records will show a move the constraint set forbids: a pallet in an exit route, a truck at a dock after its window closed. Before using that record, decide which of three things it is, because each needs a different treatment. In an illustrative companion task from the same shift, the log showed a pallet placed in slot B; review found a late scan, because the pallet had been staged there for four minutes before moving to its real slot, so the record was marked source error.

When the record breaks the rules: error, gap or real violation?
What you seeLikely explanationCheckTreatment in the dataset
Placement in a forbidden zone with a later moveLate or wrong scan; source errorCompare scan times with equipment telemetry or videoCorrect the record if evidence supports it; otherwise mark source error and exclude
Route through a closed aisleClosure ended earlier than the ticket shows; missing observationFind the reopening event or ask the siteMark missing observation; exclude until resolved
Load above slot ratingItem master weight wrongCheck receiving weights and the item master historyMark source error; fix the item master value in a new version
Impossible state, such as two pallets in one slotDuplicate or merged recordsDeduplicate on pallet and license plate IDsExclude both until reconciled
Confirmed forbidden moveReal violationEvidence agrees across systemsKeep only as a negative example labeled violation; never as a positive

Scoring: permitted first, recorded second

Score feasibility before anything else. An answer that breaks a hard rule fails regardless of how close it is to the recorded action. Among permitted answers, compare with the recorded action and its consequence: matching a choice that led to a clean outcome is good; differing from it is not automatically wrong, because operators make suboptimal but permitted choices every day.

For routing tasks, an optimization solver gives a useful reference. OR-Tools, for example, can compute a feasible route under capacity and time windows, which lets you report whether a model's route is permitted, how it compares with the solver's route and how it compares with the route the crew drove. Report the three numbers separately rather than blending them into one score.

Keep the checker independent of the task author. If the same code that wrote permitted_answers also grades responses, an error in a rule passes silently.

Where to narrow the task set or bring in a domain expert

Narrow the set when the constraint record is thinner than the task assumes. Not every site documents holds, closures or clearance heights well enough to build hard rules from them.

  • Most hard rules cannot be tied to a source document: build feasibility-only tasks for the rules that can be, and drop the rest.
  • Clearance or racking ratings come from memory rather than drawings: ask a facility engineer to confirm before any task depends on them.
  • Video is the only evidence of the state: flag occlusions per task and expect a higher share of missing-observation records.
  • Layouts reveal security features such as cash rooms or controlled-substance cages: redact those zones from the snapshot and exclude tasks that touch them.
  • Client-owned inventory appears in occupancy data: confirm the licensing scope covers it or replace item identifiers with categories.

Requesting operating records for constraint-bound tasks

Tasks like these need records most operators never publish: layout versions, location masters, hold and closure tickets, equipment telemetry and the event logs that show what was actually done. SourceX keeps no stock of layout snapshots or hold tickets; it finds a U.S. operator holding them for each buyer request, and that request, covering scope, volume, format, timing and licensing needs, is where to name the constraint types and consequence horizon the tasks require.

A warehouse or yard operator that supplies such records signs off on scope and terms first, and the de-identified layouts and logs are then used only within what that agreement permits.

Frequently asked questions

What is a spatial reasoning operational dataset?

It is a set of spatial tasks, such as routing or placement, built from records of a real operation so that the correct answers are fixed by the rules and limits in force at that moment. Each task links a state, a constraint set, the action taken and its consequence.

Why not use the recorded action as the only correct answer?

Operators often make permitted but suboptimal choices, and records sometimes contain errors. Scoring against the full set of permitted answers first, then comparing with the recorded action and its outcome, separates rule-following from imitation.

Could source error, missing observations or impossible states explain a surprising result?

Yes, and they often do. Late scans, stale closure tickets and wrong item weights all produce records that appear to break rules. Diagnose each one against a second system before using it, and keep confirmed violations only as labeled negative examples.

Do OSHA rules define the constraints for every site?

No. They are examples of hard rules that already exist in writing for U.S. general industry. Each site's own rules, fire code edition, racking ratings and customer requirements govern, and the task should cite the document the site follows.

Which coordinate frame should placement tasks use?

Use a fixed, map-type frame tied to the layout drawing for checks against racks and exit routes, declare units in every record and store the transform from any sensor frame. A drifting odometry frame is fine for short motions but not for long-term geometry.

Sources

  • VSI-Bench builds over 5,000 question-answer pairs from about 290 real indoor videos and identifies spatial reasoning as the main bottleneck for multimodal models. Source
  • SSI-Bench tests reasoning in constraint-governed structures and reports the strongest closed-source model at 33.6 percent accuracy against 91.6 percent for humans. Source
  • OSHA requires safe aisle clearances where mechanical handling equipment is used, clear aisles and stable, secure stacking. Source
  • OSHA sets a minimum 18-inch vertical clearance between sprinklers and material below. Source
  • OSHA requires exit routes to be free and unobstructed, with no materials placed in them even temporarily. Source
  • OR-Tools enforces vehicle capacity through a dimension that tracks cumulative load along each route. Source
  • OR-Tools enforces time windows through a time dimension that allows waiting slack. Source
  • ROS REP 103 standardizes on SI units, meters for length, and right-handed coordinate systems. Source
  • ROS REP 105 says map-frame poses should not drift but can jump, while odom-frame poses are continuous but drift without bound. Source

Tell us what data you need

Share scope, volume, format, timing and licensing needs. Datasets are sourced on request.

See if your company qualifies