This repository hosts the project webpage and supplementary materials for the paper:
Robot-WATCHDOG: Failure Detection through Object-Centric Graph Representation Quentin Rolland, Fabrice Mayran de Chamisso, Jean-Baptiste Mouret Conference on Robot Learning (CoRL), 2026
Link to the site: https://cea-list.github.io/Robot-WATCHDOG/
Reliable real-time failure detection is critical for deploying learned robotic policies in the real world, but enumerating failures explicitly is intractable in open-world settings. Purely visual monitors are highly sensitive to benign background variation; kinematic monitors are blind to the surrounding environment. Both struggle with relational errors — grasping the wrong object, swapping target locations, violating task order.
Robot-WATCHDOG is a self-supervised failure detection framework built on an object-centric representation of dynamic scenes as spatio-temporal graphs over tracked labeled objects and their pairwise interactions. Within that representation it deploys two complementary detectors:
- GnnT — a predictive graph transformer (GATv2 + Transformer encoder) for data-rich settings, which flags failures as deviations from learned object-centric spatio-temporal dynamics.
- TPC (Trajectory Projection and Correlation) — a non-parametric detector for extreme few-shot regimes (< 25 episodes), combining time-weighted orthogonal projection distance to expert trajectories with a pairwise object-correlation penalty.
Both are calibrated by conformal prediction, giving a guaranteed upper bound on the false positive rate without manual threshold tuning and without ever seeing failure data.
Parts of this project page were adopted from the Nerfies page.
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
