Seeing Touch from Motion

Visuo-tactile policies leveraging optical tactile sensors have shown great promise in contact-rich manipulation. These sensors achieve high spatial resolution and multi-dimensional force sensing by using an internal camera to monitor the deformation of their elastic gel surface, thereby indirectly inferring tactile cues. Despite their advantages, extracting the fine-grained contact states necessary for contact-rich manipulation remains an open challenge. Existing methods typically use either raw images or cumulative motion fields, both of which are prone to perception ambiguity: raw images mainly capture appearance changes, while cumulative motion only reflects aggregate gel deformation. As a result, distinct contact states can exhibit highly similar patterns.

To address this, we explore the dynamic priors of tactile motion and discover that the correlation between transient and cumulative motion can explicitly distinguish fine-grained contact states. Based on this insight, we propose a motion-aware tactile representation, Tactile Motion Correlation (TMC). Beyond representation, effective fusion of tactile and visual modalities is also critical. We take advantage of the Mixture-of-Transformers (MoT) architecture and propose ViTacMotor, a unified yet modality-aware visuo-tactile policy that captures cross-modal complementarity while preserving modality-specific properties. Extensive experiments on four challenging contact-rich manipulation tasks demonstrate the superior performance of our method.