Top: Important flight parameters of a ski jumper measured to the flight trajectory: upper body angle (yellow), lower body angle (orange), total body angle (purple), average ski angle (blue) and angle difference between lower body and skis (white) .

Bottom Left: Originally detected poses from multiple frames centered by the hip joint and the effect of the first robust estimation step which filters unrealistic poses.

Bottom Right: Clustering of jumps based by their flight parameters and colored according to their jumping distance.

In a joint effort with the Institute of Applied Training Science in Leipzig (Institut für angewandte Trainingswissenschaften, IAT) we develop a training feedback system for improving the jump posture of professional ski jumpers. In this project, we research deep learning models with different architectures for a continuous athlete and ski pose estimation and for tracking the body gravity center of the athletes. We use this tracking and the estimated poses to infer flight parameters, mainly the body angles, that are crucial for a long jumping distance. In order to determine these angles precisely, we use robust estimation techniques. Based on these angles, it is possible to differentiate between short, medium and long jumping distances.

We further enhance our system to detect more than the hand-annotated keypoints. In a first step, we train a VisionTransformer based model to detect intermediate keypoints, meaning any keypoint that is located on the bone of arms, legs, and the center of the skis during the in-run and the flight phase. In a next step, we extend the system to detect arbitrary points on the arms, legs and skis of ski jumpers. In order to do that, we leverage automatically generated segmentation masks which are often imprecise or incorrect. Our system is designed in a way that it can cope with these low-quality masks.

This joint project was funded by the Federal Institute for Sports Science (Bundesinstitut für Sportwissenschaft, BISp) based on a resolution of the German Bundestag.

For more information please contact  Katja Ludwig.

References:

• Katja Ludwig, Daniel Kienzle, Julian Lorenz, Rainer Lienhart. 
  Detecting arbitrary keypoints on limbs and skis with sparse partly correct segmentation masks. 
  In IEEE/CVF Winter Conference on Applications of Computer Vision Workshops (WACVW), Jan. 3 2023 to Jan. 7 2023, Waikoloa, HI, USA. 
• Katja Ludwig, Philipp Harzig, Rainer Lienhart. 
  Detecting arbitrary intermediate keypoints for human pose estimation with vision transformers.
  Winter Conference on Applications of Computer Vision (WACV), Waikoloa, Hawaii, USA, January 4-8, 2022.
• Katja Ludwig, Moritz Einfalt, Rainer Lienhart. 
  Robust estimation of flight parameters for ski jumpers. 
  IEEE International Conference on Multimedia and Expo 2020 (ICME). London, UK, July 2020. [ PDF]
• Dan Zecha, Christian Eggert, Moritz Einfalt, Stephan Brehm, Rainer Lienhart.
  A Convolutional Sequence to Sequence Model for Multimodal Dynamics Prediction in Ski Jumps.
  First International ACM Workshop on Multimodal Content Analysis in Sports (ACM MMSports'18), part of ACM Multimedia 2018. Seoul, Korea, October 2018. [ PDF]
• Dan Zecha, Moritz Einfalt, Christian Eggert, Rainer Lienhart.
  Kinematic Pose Rectification for Performance Analysis and Retrieval in Sports.
  IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops 2018. Salt Lake City, USA, June 2018. [ PDF]