Towards VLA-Dreamer: Refining VLA Behavior Using World Models

Abstract

Vision-Language-Action models (VLAs), while showing strong potential for robot control, require massive amounts of high-quality imitation learning data. Moreover, the absence of an explicit world model casts further doubt on their control capabilities. In this concept paper, we propose a novel architecture that addresses sample efficiency in VLAs by training a predictive world model on the embedding space of the VLA's vision encoder. We hypothesize that these embeddings are action-relevant and usable for future prediction. To this end, we propose using the suggested architecture to investigate how well these embeddings predict the future based on actions, as the inability to do so would mark a key limitation of VLA architectures: the lack of a non-lossy implicit world model to simulate real-world dynamics. The proposed architecture differs from the standard world model dynamics as the loss comes from the embedding space rather than the pixel space, similar to joint embedding predictive architectures. Furthermore, the trained world model can be utilized for short-term planning tasks by sampling VLA actions given goal images. We intend to examine the richness of vision embeddings in VLAs and reduce their high data requirements through a world model that can also generate plans during inference.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Robotics

• arXiv preprint · 25 September 2026

  MR. POP: Multi-Robot Parallel Optimizing Planner for Almost-Surely Asymptotically Optimal Planning

  Chih H. Huang, Roy Xing, Brian Plancher and 1 other
• arXiv preprint · 26 September 2026

  Active 3D weaves for load-bearing and damage-resilient locomotion

  Guowei Wayne Tu and Evgueni T. Filipov
• arXiv preprint · 25 September 2026

  Can a Robot Read Braille? - Learning to Adapt Contact via Imitation Learning for Tactile Braille Recognition

  Xi Chen, Yunlong Shan, Sihan Chen and 7 others
• arXiv preprint · 26 September 2026

  SAMBAR: Selective Anchoring via Method of Multipliers for Balanced Knowledge Acquisition and Retention in Vision-Language-Action Models

  Aayushi Shrivastava, Xunlan Zhou, Hongrui Zhao and 2 others
• arXiv preprint · 25 September 2026

  Multi-Objective Human-in-the-Loop Bayesian Optimization of a Lower-Limb Exoskeleton

  Neil Janwani, Matthew T. Lerner, Aaron J. Young and 1 other

All new papers in Robotics