Short answer. Physical AI is AI that acts in the physical world — robot arms, humanoids, warehouse systems, autonomous machines — and its models learn the way language models learned from text: from enormous volumes of recorded experience. The data arrives in four forms: robot demonstrations (usually teleoperated), egocentric human video, simulation rollouts, and the annotation layer that makes all of it learnable. The scarcest input is not hardware but human demonstration at scale, which is why a supply industry has formed around it in under two years.
Key takeaways
- Physical AI applies the foundation-model recipe to machines that act, and its binding constraint is demonstration data: there is no internet of actions, so every episode has to be made by someone.
- Four data types carry the field — teleoperated demonstrations, egocentric human video, simulation rollouts, and the annotation layer that makes those streams learnable.
- Scale demands are the reason for the gold rush: DROID, a flagship open dataset, holds 76,000 episodes, against field estimates that humanoid generalisation may need 100,000 to 1,000,000 demonstrations per task category.
- Providers split four ways: frontier labs running in-house collection floors, robotics-data pure-plays, established AI-data companies extending into embodiment, and open ecosystems supplying the shared floor.
- Buy it the way you buy annotation, because most of it is annotation: operator training, QA evidence, consent handling and per-episode economics, with provenance kept from the first episode.
What exactly is physical AI?
Physical AI, also called embodied AI, is the family of systems whose outputs are actions in the world rather than words on a screen: manipulation arms picking and assembling, humanoids built for warehouses and homes, mobile robots navigating sites, machines that see, decide and move.
What changed — and why the term surged through 2025 and 2026 — is the architecture underneath. Robotics used to be task-specific engineering: one robot, one programmed routine. The current generation applies the foundation-model recipe instead, training large models on vast demonstration corpora that generalise across tasks, objects and embodiments. That is the approach behind the vision-language-action models from the frontier robotics labs, and behind open cross-embodiment datasets like Open X-Embodiment, which pool demonstrations across dozens of robot types. CB Insights now maintains a physical AI models market map of the companies building these foundation layers.
The analogy the field uses on itself is the useful one: language models had the internet as a free corpus; physical AI has no internet of actions. Every demonstration has to be made. That is why the data story, not the robot story, is where the economics live, and why the sector reads so much like the early annotation market did — the subject of our piece on how multilingual collection economics work.
Why does it come down to training data?
Because action data is scarce in a way text never was, and the scale requirements are brutal.
The reference numbers tell the story. DROID, one of the field's flagship open manipulation datasets, holds 76,000 episodes across 564 tasks — a monumental collection effort, and a rounding error against the field's own estimate that humanoid generalisation may require 100,000 to 1,000,000 demonstrations per task category. Frontier players behave accordingly, with Physical Intelligence, Google DeepMind and Skild AI each reported to be investing tens of millions in collection.
Scarcity has minted a new supply industry. Dedicated teleoperation-data startups are raising at pace — xDOF launched with $70M in June 2026 specifically to sell robot demonstration data — and "data factories" filming human labour to train machines have become a recognised category. Scale AI's physical-AI arm now advertises 1,000+ hours of demonstration data collected daily through data factories, residential collection and in-field platforms. The strategic read is consistent across the analyst coverage, including the ISF's framing of robotics data pipelines as a national-competition issue: whoever solves demonstration supply at quality owns the bottleneck.
For readers of this blog the pattern will be familiar. It is the high-resource versus low-resource language divide replayed in motion: capability follows data density, and the market pays whoever can field trained humans to make the data.
What are the four kinds of physical AI data?
Four types carry the field: teleoperated robot demonstrations, egocentric human video, simulation rollouts, and the annotation layer that makes all three learnable. Serious programmes buy a mix rather than betting on one.
Robot demonstrations (teleoperation). A trained human drives the robot through a task — pick, place, fold, assemble — producing synchronised streams of camera frames, joint states and actions. This is the gold standard for manipulation learning, priced per episode, with quality hanging on operator skill and task diversity. Dexterous, fine-fingered demonstration is the scarcest and highest-value tier.
Egocentric human video. Head- or chest-mounted cameras recording humans doing tasks with their own hands. Cheaper and faster than teleoperation, embodiment-mismatched but increasingly usable as pretraining signal, which is why Scale now fields a robotless egocentric collection platform alongside its teleoperation work. Centific and Shaip document the same pipeline shape for humanoid and manipulation data.
Simulation rollouts. Synthetic episodes from physics engines are effectively free at the margin and indispensable for locomotion and rare-event coverage, but they are gapped by the sim-to-real divide: policies trained purely in simulation stumble on real friction, light and mess. The working recipe everywhere is simulation for breadth, real demonstrations for grounding — the same synthetic-plus-human-anchor logic we set out for synthetic text data.
The annotation layer. Every stream above needs labelling to be learnable: task segmentation, success and failure judgment, object and hand annotation, quality filtering of demonstrations, language instructions paired to episodes. This is classic data-operations work — gold sets, agreement, QA tiers, the machinery of scaling annotation from pilot to production — applied to video and sensor streams, and it is the layer most buyers under-budget. Our piece on annotation for robotics and physical AI covers that tier in detail.
| Data type | Role | Scarce ingredient |
|---|---|---|
| Teleoperated demonstrations | Grounding manipulation | Trained operators |
| Egocentric human video | Pretraining breadth | Task diversity, usage rights |
| Simulation rollouts | Breadth, rare events | Sim-to-real fidelity |
| Annotation layer | Makes the rest learnable | QA discipline |
How big is this market, and what does the data cost?
The robot training-data market is small in absolute terms but compounding quickly, and roughly 60% of it is collection services rather than software. Per-demonstration costs are falling steeply year on year as operations professionalise.
The most-cited trajectory comes from a robot-data market analysis putting the segment at roughly $300M in 2024 and around $500M in 2025, heading toward several billion by the end of the decade. Per-unit economics have moved the way early annotation pricing did.
Honest caveats belong beside those numbers. They come from a provider's own market write-up, the analyst sources behind them are unnamed, and the projections assume humanoid deployment timelines that remain contested. Treat the shape of the curve as the signal and the decimals as marketing. What is independently corroborated is the direction: dedicated funding rounds such as the $70M for a teleoperation-data startup, frontier labs building in-house collection farms, and a provider tier forming fast enough that round-ups of physical AI data providers are now their own genre.
Who collects and annotates physical AI data?
Supply splits four ways: frontier robotics labs running in-house collection floors, robotics-data pure-plays selling demonstrations, established AI-data companies extending into embodiment, and open ecosystems supplying a shared pretraining floor.
Frontier robotics labs, in-house. Figure, Physical Intelligence, 1X, Agility, Apptronik and the big-tech labs run their own collection floors. This is the capability bar, not the open market.
Robotics-data specialists. The new pure-plays: teleoperation farms and egocentric collectors such as xDOF and its peers, quoting tens of thousands of demonstrations per month, plus Scale AI's physical-AI arm running data factories and distributed collection.
AI data companies extending into embodiment. The established data industry is moving in because most of the stack is its home turf. Centific and Shaip publish humanoid and manipulation offerings. Lifewood operates the collection-and-annotation layer — managed human data collection and the video and sensor annotation tier — through a delivery network across 30+ countries, where the transferable assets are exactly the ones this market is short of: recruiting and training large human workforces, running collection to spec, and holding QA discipline over the resulting streams. The large-scale video annotation experience ports directly. For egocentric programmes specifically, geographic spread matters, because task diversity across homes, kitchens, warehouses and cultures is a data dimension rather than a logistics detail — the same reason locale-level scoping governs speech collection.
Open ecosystems. Open X-Embodiment, DROID and the LeRobot hub supply the shared floor. Essential pretraining substrate, differentiating no one.
Buying guidance mirrors annotation's. Judge providers on demonstrated throughput, operator training, QA evidence and rights and consent handling — egocentric video is people's working lives on camera, so the consent and privacy bar of secure collection on sensitive data applies in full — rather than on brochure counts of robots.
What should you do if your roadmap touches physical AI?
Decide which layer you occupy, budget for a mix of data types rather than one, build the evaluation set before buying training data, and pilot collection at a few hundred episodes before committing to volume.
Decide your layer first. Building models means you need demonstration scale. Deploying robots means you need task-specific fine-tuning data and evaluations. Supplying the ecosystem means you need collection capability. The data plan differs by layer, and programmes go wrong when the layer is left implicit.
Budget the mix, not one type. Simulation for breadth, egocentric for pretraining volume, teleoperation for grounding, annotation across all of it. Programmes fail by over-buying one lane.
Start with the evaluation set. A few hundred human-judged task episodes against a stable rubric, before any training-data spend.
Pilot collection at small scale — a few hundred episodes — to find task-specification and QA problems while they are cheap. The pilot-before-production rule transfers intact.
Write rights and consent into collection from day one: operator contracts, filming consent, and residency terms for footage of real workplaces.
Track per-episode economics quarterly. Prices are falling steadily, and long fixed-price commitments age badly in your counterparty's favour.
Keep provenance labels — simulation versus egocentric versus teleoperation, site, operator — because tomorrow's debugging and audits depend on them.