What is physical AI, and why does it need factory data?

What is physical AI?#

Physical AI is artificial intelligence built to perceive, reason about and act in the physical world, rather than working only with text or images on a screen. In a factory the term usually means robots that pick, place and assemble, autonomous mobile robots that move material, and vision systems that inspect parts; some definitions also include autonomous vehicles and models that predict or control machine behavior.

The term is newer than the technology. Industrial robots and machine vision have long been on factory floors; what has changed is the attempt to give them general skills learned from large amounts of data, so they adapt to new parts and tasks instead of being programmed step by step.

That shift changes what developers need. A robot programmed by an integrator needs a program; a robot that learns needs many examples of the task done well and done badly, recorded with enough context to tell the difference.

Why physical AI needs real factory data#

Physical AI needs real factory data because public text and video rarely show industrial work with its context and outcome, and simulation alone misses much of the variation in a real plant. A video of someone assembling a gearbox does not say which work instruction applied, what torque was used, whether the unit passed test or why the operator stopped and restarted.

Factories hold that context. The same task is performed many times, documented in work instructions and routings, measured by sensors and inspections, and corrected when it goes wrong. Repetition with known results is what developers need to train systems and to show that they work.

Simulation helps but does not close the gap. Developers train many skills in simulated environments, then find that real parts, lighting, wear and human habits behave differently. Measurements and records from real operations are how they tune those simulations and check that a system trained there holds up on an actual line.

Developers are already collecting real-world video at scale. In September 2025, Figure announced a partnership with Brookfield to capture human video across Brookfield environments, a portfolio that includes residential, office and logistics space, to build training data for its Helix model. Build AI's Egocentric-10K dataset card on Hugging Face describes about 10,000 hours of head-mounted video collected in real factories and released under the Apache 2.0 license. As video of work becomes easier to obtain, the records that explain it, such as work instructions, test results and maintenance outcomes, may become the scarcer part.

The data physical AI uses, and which of it manufacturers hold#

Physical AI uses several kinds of data, and manufacturers already hold more of them than they expect. The table matches common data types to what they teach and to how often a mid-sized plant tends to hold them.

Two patterns stand out. Video of manual work is the type plants hold least often, and the one that needs the most rights and privacy work. The records that define tasks and outcomes already sit in ERP, MES, CMMS and quality systems, and they are what give any recording its meaning.

How developers use factory data: training, testing and simulation#

Developers use factory data in four ways: to train systems on examples, to test them against real cases with known results, to tune simulations, and to define what a correct task looks like. Each use favors different records, which is why a plant without cameras can still hold something useful.

Testing is the use owners most often overlook. A developer needs real cases it did not train on to show that a system works, and a documented history of inspections or fault diagnoses with confirmed outcomes can serve as that kind of test set. Consistent timestamps, stable machine and station names and part numbers that mean the same thing in every system decide whether those cases can be lined up without guessing.

• 1Training: demonstrations and records of real tasks, including mistakes and recoveries, that a system learns to imitate or improve on.
• 2Testing: held-back real cases with known outcomes, such as inspection results or confirmed repairs, used to check a system before it reaches a line.
• 3Simulation: real cycle times, part variation, machine behavior and failure patterns used to make simulated environments more realistic.
• 4Task definition: work instructions, routings and acceptance criteria that tell a developer what correct looks like.

What supplying physical AI data involves for a manufacturer#

Supplying physical AI data involves more than handing over files, especially when new capture is planned on the shop floor. Existing archives raise rights and privacy questions; new capture also raises safety, production and workforce questions.

Compensation is set package by package in the license. Volume plays a part, but documentation, linkage and fit with what a specific buyer needs usually matter as much, so value is known only once a buyer engages.

• 1Worker notice, and consent where the applicable laws or agreements call for it
• 2Union and employment terms that address cameras or monitoring
• 3Site access rules, safety training and escorts for any outside capture team
• 4Production impact of added sensors, cameras or recording steps
• 5Exclusion of customer-owned parts, prints and export-controlled work
• 6Redaction of faces, badges and screens in video and images
• 7Agreements with machine and software vendors that address controller or sensor data

Adopting physical AI and supplying data are separate decisions#

Adopting physical AI in your plant and supplying data to physical AI developers are separate decisions, and the first can quietly shape the second. When a plant buys robots, autonomous mobile robots or vision inspection systems, the vendor's contract and software terms may let the vendor collect and use what the equipment records, such as images, floor maps, fault logs and cycle data.

That can be reasonable, but it is a data decision as well as a purchasing one. Before signing, ask what the equipment records, where it is sent, whether the vendor may use it to improve products for other customers, and whether you can export and keep your own copy. A plant that settles these points keeps more of its options, including licensing the same kind of data later.

Customer mix matters on both sides. Images captured on lines that run customer-designed parts may be restricted under customer terms, whether the equipment vendor or the plant wants to use them.

Illustrative: a lighting fixture maker adds mobile robots#

Illustrative: a fictional maker of commercial LED lighting fixtures plans to add autonomous mobile robots to move kits from its stockroom to its assembly lines. It also wonders whether its own records could interest physical AI developers. It keeps maintenance work orders in a CMMS, routings and work instructions in its ERP, and test-station results for every fixture in its MES.

The robot vendor's draft agreement lets the vendor keep floor maps, camera images and navigation logs and use them across its customer base. The CEO negotiates a narrower clause: the vendor may use the data to run and support the fleet, the company keeps its own copy, and images showing private-label fixtures made for a retail customer are not retained.

Separately, a metadata review shows that test-station results link to work orders and maintenance history by line and date. The company takes those records into a licensing review and leaves any new video capture for a later program with its own worker notice.

How SourceX approaches physical AI data#

SourceX approaches physical AI data as physical-world supply within the SourceX five-step transaction: Supply, Rights, Preparation, Approval and Delivery. The fit check uses metadata about machines, systems, years of history and any recordings, and nothing is shared during the initial assessment.

Large sensor and video archives stay in the manufacturer's own storage or ship on encrypted drives. Equipment vendor terms are part of the Rights review, and each approved package carries a SourceX Evidence Packet; the manufacturer grants use rights while keeping ownership of its records.