Running Efficiency with Computer Vision: A Comparative Analysis with Eliud Kipchoge

How to improve your running efficiency with Computer Vision?

TLDR: I have created an experiment — iterating through various shoe types to achieve Eliud Kipchoge Running Efficiency. By surprise — it turns out that Flip-flops suck for running and Carbon-fiber shoes are epic!

Abstract

Running efficiency — the ability to expend minimal energy while covering a defined distance — is a major factor in athletic performance.

Traditional methods for assessing running efficiency mostly rely on subjective evaluations or invasive physiological measures — thus, often limiting applicability and objectivity. This experiment introduces a modern approach to measuring running efficiency using AI. In other words, I am using computer vision (CV) technology to evaluate the running efficiency. Employing MoveNet from TensorFlow, I extract 17 keypoints from video footage of professional runners, including Eliud Kipchoge. Eliud is the GOAT in marathon running and especially known for his exceptional efficiency. By analyzing keypoints that are extracted from the video material, I have developed a similarity regression function that quantifies the similarity between a runner’s gait and Kipchoge’s exemplary form. This function provides objective feedback on both posture (keypoints position relative to each other) and movement (keypoints movement from frame to frame). Based on this approach, I have conducted a comparative analysis of my own running style against Eliud while iterating various shoe types to get closer to Eliud performance.

Introduction

Running efficiency is a complex concept that encompasses various biomechanical factors. Those include: (1) stride length, (2) cadence, (3) ground contact time, and (4) energy expenditure. Theory tells us that optimizing these factors can significantly enhance running performance and at the same time reduce the risk of running injuries. Traditionally research assesses running efficiency by using subjective methods such as visual observation or expert opinions. Obviously this can result in high variability and may not provide detailed quantitative insights. The goal is to retrieve invasive physiological measurements such as metabolic testing and gait analysis which offer more precise assessments, however, before the era of AI, these measurements were often impractical and expensive to establish.

Clearly, the advent of ML technology has opened up new possibilities for objective and quantitative assessment of running efficiency. In detail, CV models can extract detailed information about a runner’s movement, including joint angles, body positions, and movement patterns. By analyzing this data, we can gain a deeper understanding of the biomechanical underpinnings of running efficiency and identify areas for improvement more easily, reliable, and faster.

Methodology

In this study, I used the MoveNet model from TensorFlow. MoveNet is a state-of-the-art CV model for body pose estimation. As mentioned above, MoveNet extracts 17 keypoints from video footage, providing a comprehensive representation of a runner’s body posture and movement.

Technical functionality: MoveNet’s architecture comprises two fundamental components: (1) a feature extractor and (2) a set of prediction heads. The feature extractor in MoveNet is built on MobileNetV2 and complemented by a feature pyramid network (FPN). Overall, the MoveNet architecture incorporates four prediction heads. Every head is dedicated to estimating specific pose-related information, as seen in the four steps below:

Step 1: Locate person centers: Identify the center of the ‘runner’ in the frame using the person center heatmap. This is done by evaluating the probability of each pixel being the center of a person. The location of the highest score is then selected and weighted by the inverse-distance from the frame center. The outcome is the estimated center for the ‘runner’.

Step 2: Generate initial keypoints: Create an initial set of keypoints for the ‘runner’ by slicing the keypoint regression output from the pixel corresponding to the estimated center.

Step 3: Refined keypoint localization: Next I apply a weighting scheme to each pixel in the keypoint heatmap. The weight is inversely proportional to the distance between the pixel and the corresponding regressed keypoint. This weighting ensures that only keypoints from the actual ‘runner’ are accepted — while background keypoints typically have low weights due to their distance from regressed keypoints.

Step 4: Refine and select keypoints: Lastly, I identify the coordinates of the maximum heatmap values in each keypoint channel. These represent the most likely locations for each keypoint. We add the corresponding local 2D offset predictions to these coordinates and therefore obtain refined estimates for the ‘runner’s’ keypoints. I later make use of this output to develop a similarity regression function that quantifies the similarity between a runner’s gait and Kipchoge’s exemplary form.

The Model to extract the keypoints

model_name = "movenet_thunder"
module = hub.load("https://tfhub.dev/google/movenet/singlepose/thunder/4")
input_size = 256

def movenet(input_image):
    model = module.signatures['serving_default']
    input_image = tf.cast(input_image, dtype=tf.int32)
    outputs = model(input_image)
    keypoints_with_scores = outputs['output_0'].numpy()
    return keypoints_with_scores

To train the similarity regression function, I created a dataset of video clips of Kipchoge running at various speeds and distances. These clips were carefully annotated by me to identify the corresponding keypoints for each frame. I then trained the regression function on this labeled dataset, using the extracted keypoints as features and the similarity to Kipchoge’s gait as the target variable. Further note that running outdoors or even on a treadmill in not static in the sense that every stride is equal or in the same position in continuous camera frames. Hence, normalization of stride ‘starting position’ and ‘ending position’ is required to make strides comparable. In other words, the keypoint normalization procedure ensured that the keypoints from different strides were comparable. This was important because the length of a stride can vary depending on the runner’s speed and cadence.

I ended up using a support vector regression (SVR) model for this task to better handle non-linear relationships between the input features (keypoints) and the output variable (similarity to Kipchoge’s gait). Additionally, I found that SVR is a robust model that is not serverly sensitive to outliers, which is important in this context as gait data can be noisy due to various factors, such as, (a) camera angle and (b) lighting conditions, etc. I ended up receiving a RMSE of 0.02 which indicates that the method is able to predict the similarity to Kipchoge’s gait with good accuracy. In other words, we now have a model that is able to effectively capture the relationship between the keypoints and the similarity to Kipchoge’s gait.

Finally, I weighted the foot/knee movement twice as high as the upper body keypoints. Theory says that (the lower body) these keypoints are more important for running efficiency than the upper body keypoints. This weighting scheme allowed the model to focus on the key aspects of gait that contribute most to running efficiency. I called the final Model — Kipchoge Efficiency Index (KEI) — This name directly references the model’s ability to assess running efficiency by benchmarking against Eliud Kipchoge, one of the most efficient runners in history.

Correlation coefficient: The correlation coefficient between the SVR model’s predictions and the ground truth was 0.98, which indicates a strong correlation. In other words, there is a strong linear relationship between the keypoints and the similarity to Kipchoge’s gait.

Mean absolute error (MAE): The MAE between the SVR model’s predictions and the ground truth similarity to Kipchoge’s gait was 0.02. This result also indicates that the model’s predictions are very close to the ground truth.

Root mean squared error (RMSE): As mentioed above, the RMSE between the SVR model’s predictions and the ground truth similarity to Kipchoge’s gait was 0.02, which is a good value. This indicates that the model’s predictions are accurate.

External validation: The KEI was validated on a separate dataset of four other video clips of Kipchoge running at various speeds and distances. The results showed that the KEI was able to predict the similarity to Kipchoge’s gait with reasonable accuracy.

Overall, these results provide evidence that the methodology to measure similarities of movement and posture measured by keypoints is a valid and reliable approach to assessing running efficiency. The use of multiple measurement metrics, including correlation coefficient, MAE, RMSE, and external validation, demonstrates the robustness and significance of the findings. Thus, we can derive that machine learning is a valid methodology the evaluate body movements, such as for runners.

The Experiment

After building the KEI-Model, I was able to predict on my running efficiency. Therefore, I collected a bunch of video material of myself running on a treadmill with from 2 to 20kmh. The camera angle and lighting was kept consistent throughout the experiment.

As an experiment, I iterated various shoes to evaluate any potential changes in running efficiency:

Results

The trained similarity regression function effectively measures the similarity between a runner’s gait and Kipchoge’s. I applied the function to my own running footage and obtained a quantitative score representing the level of efficiency relative to Kipchoge. I identified several areas for improvement, including increasing my stride length and increasing my cadence. Beyond the running technique, I found that professional running shoes severely help to increase running efficiency. In detail, I found that running in Carbon Fiber shoes give me the highest similarity to Eliud’s running style, whereas running in FlipFlops gave me a significant negative effect.

Barefoot running resulted in a high running efficiency (74%). This can be attributed to the natural gait and maximum proprioception allowed by barefoot running, which promote efficient movement patterns. Although, I felt other negative effect- which are not measured here, such us blisters.

Flip-flops exhibited the lowest running efficiency (38%). The minimal cushioning and support contribute to inefficient running mechanics and increased impact forces, leading to decreased efficiency. This could clearly be felt by me — even at low running speeds.

Normal running shoes provided a moderate level of running efficiency (62%). The balance of cushioning and support offers a comfortable and somewhat efficient running experience. I clearly felt that these shoes are built for running.

Carbon fiber professional running shoes surpassed all other shoe types in running efficiency (82%). In theory, the extensive cushioning and support, along with advanced features such as motion control and stability elements, optimize running mechanics and minimize energy expenditure. I subjectively also felt best-running the in the carbon fiber shoes.

Army boots, while providing stability and support, hampered running efficiency (45%). The rigid design and limited ground feel restrict natural movement and make it challenging to adapt to different terrains, ultimately reducing efficiency. Although, the results of the army boots are better than the flip-flops — the subjective feeling was the worst due to the heavy build and unflexible structure of the shoes.

These results indicate that shoe type plays a significant role in influencing running efficiency. Barefoot running, with its focus on natural gait and proprioception, emerges as an efficient approach — but could not be sustained on the treadmill due to imminent injury by belt friction (blisters). Normal running shoes did as expected while carbon fiber professional running shoes outperformed every other shoe type. Flip-flops and army boots, while providing specific benefits, fall short in terms of running efficiency — and are clearly not recommended for running especially longer distances.

Some potential advantages and disadvantages are listed below based on theory and my experiences:

Barefoot Running: Natural Gait: I feel that barefoot running promotes a natural gait pattern that aligns with the body’s biomechanics, reducing muscle strain and improving running economy. Maximum Proprioception: Theory says that running barefoot enhances proprioception, the body’s awareness of its position and movement, allowing for better balance, coordination, and efficient energy expenditure. Maximal Impact Forces: Lastly, barefoot running increases impact forces as no cushion can absorb shock. Thus muscles and tendons are attacked the most during the run — which ultimately leads to a decreased running efficiency over time.

Flip-Flops: Minimal Cushioning: I feel that the lack of cushioning in flip-flops exposes the foot to direct impact with the ground, increasing stress on the joints, crates blisters — thus reducing running efficiency. Limited Support: Clearly, Flip-flops provide minimal support for the foot and ankle, increasing the risk of instability and injury, potentially leading to inefficient running mechanics. Uneven Ground Contact: According to theory, the flexible sole of flip-flops can contribute to uneven ground contact and increased impact forces, potentially hindering running efficiency.

Normal Running Shoes: Balanced Cushioning and Support: I genuinely feel, that almost all ‘normal running shoes’ offer a acceptable cushioning to absorb impact and support to stabilize the foot and ankle, promoting a comfortable and somewhat efficient running experience. Consideration of Pronation: In addition, some normal running shoes offer features to control pronation, which can be beneficial for runners with excessive inward rolling of the foot, potentially improving running efficiency.

Carbon Fiber Professional Running Shoes: Extensive Cushioning: Theory and marketing tells us that Carbon fiber professional running shoes provide extensive cushioning to minimize impact forces and protect the joints, while also promoting a smooth and efficient running stride. This is due to the fact that Carbon soles are composed of lightweight yet rigid carbon fiber plates that generally provide superior cushioning and energy return compared to traditional foam soles- which can be found in our normal running shoes. Ultimately, this means that the high-tech shoes can absorb more impact energy during each stride and then release it back to the runner, propelling them forward with more force- thus improving running efficiency. Precision Fit: For Paul Childs/Reutersmy personal experiences, I can tell that the precise fit of carbon fiber professional running shoes allows for a more personalized and efficient running experience. However, one has to find the ‘right’ Carbon fiber shoes for oneself. Accelerated Recovery: Theory mentions that Carbon soles can also aid in the recovery process after running. The reduced muscle fatigue and improved running mechanics can help the body to recover more quickly from intense running or interval workouts.

Army Boots: Rigid Design: The rigid design of army boots restricts natural movement and can hinder the runner’s ability to adapt to different terrains, potentially reducing running efficiency on treadmills. Limited Ground Feel: The thick soles of army boots can impede ground feel, reducing the runner’s ability to make subtle adjustments to their gait, ultimately affecting efficiency. Weight and Bulkiness: The added weight and bulkiness of army boots can increase the effort required for running and potentially lead to inefficient movement patterns.

Discussion and Conclusion

My running study demonstrates the potential of CV technology for objective and quantitative assessment of running efficiency. My approach, using the MoveNet model and a similarity regression function, provides a practical and effective tool for evaluating running form and identifying areas for improvement. By benchmarking my own running style against that of a world-class athlete like Eliud Kipchoge, I could gain valuable insights into optimal biomechanical patterns and strive for greater efficiency — maybe breaking the World Record one day ;)

Future research directions include further refining the similarity regression function to incorporate additional biomechanical parameters, such as stride length, cadence, and ground contact time. Additionally, I highly recommend developing real-time applications that can provide feedback to runners during their workouts would be highly beneficial for optimizing their training and performance.

In conclusion, the use of AI-powered CV models offers a promising approach to objectively assess running efficiency and guide training strategies. By leveraging ML technology, we can empower athletes to achieve their full potential and enhance their overall performance.

Final remark — although I came close to Eliud Running Efficiency — I am still seeking for the perfect shoes that give me the cutting edge to tackle the impossible.

Limitations

While the developed model exhibits promising results in evaluating running efficiency, it is crucial to acknowledge from my side that there are certain limitations that require further investigation.

Human Limitations:

1. Individual Variability: Human attributes such as body structure, limb length, and muscle composition most likely significantly impact running style and efficiency. Thus, the model’s assessment may be less accurate for individuals with unique anatomical features. Hence, a more generalized model would be beneficial that could process and rule out potential bias due to the runner’s individual physic.
2. Variable Running Speed: Running speed can influence running mechanics, and the model’s evaluation may not be directly applicable to individuals running at different speeds. Thus, it is highly recommended for future research to include running speeds as a parameter in the analysis.
3. Individual Stride Length: Stride length is a highly variable parameter that is influenced by factors such as running speed, terrain, and individual biomechanics. My model’s assessment may not be fully sensitive to variations in stride length.

Technical Limitations:

1. Camera Angle Influence: The angle of the camera capturing the running motion can affect the detection of keypoints and consequently impact the model’s evaluation. In addtion, the resolution of the video material was limited as well to allow faster inferences times.
2. Lighting Conditions: Changes in lighting conditions, such as low illumination or harsh shadows, can decrease the accuracy of keypoint detection, potentially leading to inaccurate efficiency assessments. Thus, it is highly recommended to leverage good lighting conditions and contrast rich clothing.
3. Frame Rate Consistency: Consistent frame rates are essential for reliable keypoint detection and tracking. Variations in frame rate can introduce artifacts and inaccuracies in the model’s evaluation. In other words, a constant framerate is required.

Model and Data Limitations:

1. Subjective Efficiency Measurement: The concept of running efficiency is inherently subjective, and there is no universally accepted definition or method for its quantification. The model’s evaluation may not fully capture the nuances of human running efficiency as perceived by experts.
2. Limited Training Data: The model’s training data was limited to videos of one elite runner (Eliud Kipchoge) and one amateur runner. This limited data set may not fully represent the diversity of running styles and efficiency levels among individuals. Furthermore, the World Record holder (Kelvin Kiptum) was neglected in the model training due to limited video training material.
3. Model Bias and Similarity Function: The similarity function used to compare running styles may introduce model bias, potentially overemphasizing certain aspects of running mechanics and undervaluing others. Although similarity regression functions are well understood in research, it is curial to evaluate the importance of each keypoint individually regarding similarity significance — to achieve meaningful results.
4. Short Video Sequences: The model’s evaluation was based on short video sequences, which may not capture the full range of a runner’s biomechanics and running style. This limitations esepcially holds true for long-distance runners-where running style might vary thoughout the run to leverage different muscles groups.
5. Custom Loss Functions and Keypoint Weights: It is clear that the custom loss functions and keypoint weights used in the model’s training may introduce biases that reflect the specific characteristics of the training data.

Despite these limitations, the developed model represents an interesting tool and first step for analyzing running efficiency and identifying areas for improvement. Further research is necessary to address the identified limitations, expand the training data set, and refine the model’s evaluation metrics. This will enable the model to provide more accurate and generalizable assessments of running efficiency among a wider range of individuals and sports. Nevertheless, I hope my experiment is a good starting point and gives potential ideas for future developments.

Disclaimer

Don’t try this at home. This experiment was conducted for educational purposes only and should not be construed as medical, athletic, or personal advice. The final model is not a Google product or affiliated with any brand or company. The results of this experiment (including the shoes) may not be generalizable to all runners and may vary depending on individual factors such as running experience, biomechanics, and injury history. I highly recommend to consult with a healthcare professional before making any changes to your running routine or footwear.

References:

MoveNet-Paper: https://ieeexplore.ieee.org/document/9406043

MoveNet on TF-Hub: https://www.tensorflow.org/hub/tutorials/movenet

Next-Generation Pose Detection with MoveNet and TensorFlow.js: https://blog.tensorflow.org/2021/05/next-generation-pose-detection-with-movenet-and-tensorflowjs.html

[1] https://www.nike.com/de/laufen/breaking2