The paper "TT4D: A Pipeline and Dataset for Table Tennis 4D Reconstruction From Monocular Videos" authored by Nima Rahmanian, Dvij Kalaria, Daniel Kienzle, Thomas Gossard, Rainer Lienhart, and S. Shankar Sastry has been accepted at ACM Multimedia (ACM MM) 2026. The conference will take place from November 10 to 14, 2026, in Rio de Janeiro (Brazil).

With TT4D, the authors present both a novel reconstruction pipeline and an extensive multimodal table tennis dataset featuring over 140 hours of gameplay extracted from regular broadcast footage. The dataset contains 3D ball trajectories, ball spin annotations, precise temporal segmentation of individual rallies, and human meshes.

Previous reconstruction approaches often failed on uncut videos due to error-prone temporal segmentation of individual rallies based on 2D information. The presented "Lift-First" pipeline overcomes this issue:
1. Full-Sequence Lifting: A transformer network directly lifts unsegmented 2D ball detections into 3D space while simultaneously estimating spin. Missing detections caused by occlusions are robustly interpolated.
2.3D Time Segmentation & Annotation: Within the unambiguous 3D space, hit events can be reliably localized.

TT4D enables entirely new applications in sports analytics and robotics with over 210,000 reconstructed rallies.

https://arxiv.org/abs/2605.01234