Das Paper mit dem Titel "Uplift and Upsample: Efficient 3D Human Pose Estimation with Uplifting Transformers" von Moritz Einfalt, Katja Ludwig und Rainer Lienhart wurde für die IEEE/CVF Winter Conference on Applications in Computer Vision (WACV) 2023 akzeptiert. In diesem Paper beschreiben die Autoren eine Methode, mit der die Berechnungskomplexität von menschlicher Posenschätzung im 3D Raum mit Transformern erheblich reduziert werde kann. Dennoch können damit weiterhin zeitlich konsistente und präzise 3D Bewegungssequenzen rekonstruiert werden.

The state-of-the-art for monocular 3D human pose estimation in videos is dominated by the paradigm of 2D-to-3D pose uplifting. While the uplifting methods themselves are rather efficient, the true computational complexity depends on the per-frame 2D pose estimation. In this paper, we present a Transformer-based pose uplifting scheme that can operate on temporally sparse 2D pose sequences but still produce temporally dense 3D pose estimates. We show how masked token modeling can be utilized for temporal upsampling within Transformer blocks. This allows to decouple the sampling rate of input 2D poses and the target frame rate of the video and drastically decreases the total computational complexity. Additionally, we explore the option of pre-training on large motion capture archives, which has been largely neglected so far. We evaluate our method on two popular benchmark datasets: Human3.6M and MPI-INF-3DHP. With an MPJPE of 45.0 mm and 46.9 mm, respectively, our proposed method can compete with the state-of-the-art while reducing inference time by a factor of 12. This enables real-time throughput with variable consumer hardware in stationary and mobile applications. We release our code and models on  GitHub.

Moritz Einfalt, Katja Ludwig, Rainer Lienhart
Uplift and Upsample: Efficient 3D Human Pose Estimation with Uplifting Transformers
IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). Waikoloa, USA, January 2023. [ arXiv][ CVF][ GitHub]