Learning Robot Behavior Priors from Non-Robotic Synthetic Data

Abstract

Real robot data, rich across tasks and morphologies, is often treated as the bottleneck for training robot learning models, including embodied foundation models that aim for zero-shot physical transfer. In this paper, we investigate the degree to which such models can acquire necessary structure using data from systems without embodiment. To this end, we test if inexpensive, procedurally generated synthetic data can yield useful priors before a model observes any real-world behavior. We study this through robot action prediction, a natural pretraining objective and a direct test of whether structure learned from arbitrary synthetic tasks transfers to embodied settings. We introduce three data generation pipelines that model mathematical abstractions of robotic control, and use them as pretraining priors for a Transformer. We demonstrate that a model trained purely on this non-robotic data attains non-trivial zero-shot prediction performance on unseen physical robot tasks via an in-context learning mechanism, and that our specialized control priors notably improve upon generic causal priors. We also compare against a compute-matched baseline trained on real robot data, and find that it outperforms our model but only on the datasets it was trained on. These findings suggest that procedurally generated, mathematical abstractions can yield useful robotic priors, offering a potential complement to expensive embodied data collection.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Robotics

• arXiv preprint · 22 October 2026

  Know When to Escalate: Critic-Guided Adaptive Execution for Cloud-Local Embodied Agents

  Wei Zhang, Lirong Xiang and Charles Fleming
• arXiv preprint · 23 October 2026

  CEMIR vs. FISER: a comparative study of reasoning-based and enrichment-based approaches to pragmatic intent recognition

  Ziya Hosseynzadeh, Farjad Fallah, Farnoosh Fallah and 3 others
• arXiv preprint · 22 October 2026

  OASIS: Online Anchor-centric Scene Inference System for Open-Vocabulary Retrieval and Navigation in Dynamic Environments

  Kaiyao Liao, Qingfeng Li, Chen Chen and 3 others
• arXiv preprint · 23 October 2026

  DiscoDemo: Discovering Efficient and Diverse Robot Demonstrations for Imitation Learning

  Minho Park, Kinam Kim, Donghu Kim and 6 others
• arXiv preprint · 22 October 2026

  PATH: Preference-Aware Traversability Estimation for Robotic Teleoperation in Off-Road Navigation

  Xianghao Meng, Wenshuo Wang, Cheng Shao and 4 others

All new papers in Robotics