Robot-WATCHDOG : failure detection through object-centric graph representation

Abstract

Reliable real-time failure detection is critical for deploying learned robotic policies, yet defining failures explicitly is infeasible in open-world settings. This motivates failure detection from nominal demonstrations alone. Existing methods, however, struggle to balance robustness to benign variations with the ability to capture semantic and interaction-level errors. We introduce Robot-WATCHDOG, a self-supervised framework that models manipulation scenes as object-centric spatio-temporal graphs. Building on this representation, we investigate two complementary failure scores: Trajectory Projection and Correlation (TPC), a non-parametric measure of geometric deviation and relational consistency, and GnnT, a Graph Attention Network with a Transformer that predicts future object states. We evaluate both on a public benchmark and on a new dataset of semantically complex manipulation tasks, across six tasks and ten random seeds. Both scores outperform image-based failure detectors, particularly on semantic and interaction-level failures. TPC provides the strongest performance in AUPR and MCC on both dataset averages, with low variance across seeds. Qualitative results are made available at https://cea-list.github.io/Robot-WATCHDOG/.

The authors’ abstract, as arXiv lists it. arXiv’s listings, abstracts included, are free for anyone to reuse.

More in Robotics

• arXiv preprint · 9 October 2026

  Cubic Spline Interpolation for Smooth Trajectory Generation in Autonomous Motion Systems

  Aarush Mandayam, Sriram Aditya Gundla and Elliott Tammaro
• arXiv preprint · 10 October 2026

  Red-Teaming of Vision-Language-Action Models via Quality Diversity Environment Generation

  Shihan Zhao, Henry Chen, Ya-Chuan Hsu and 7 others
• arXiv preprint · 9 October 2026

  VLMs are Native Action Models: Unleashing the Manipulation Capabilities of VLMs with Explicit Action-Space Grounding

  Jingkun Yi, Shujie Zhang, Zirui Zhou and 8 others
• arXiv preprint · 10 October 2026

  Shared-Control Teleoperation of Omnidirectional Aerial Robots for Continuous Valve Rotation

  Jinjie Li, Yicheng Chen, Haokun Liu and 1 other
• arXiv preprint · 9 October 2026

  Reinforcement Learning via Fixed-point Flow Matching

  Denis Blessing, Onur Celik and Gerhard Neumann

All new papers in Robotics