Bench press exercise detection and repetition counting

Martin Barus

Overview

Given set of device movement measurements, consisting from acceleration in X,Y,Z directions, gyroscope measurement in X,Y,Z directions, and gravity vector in X,Y,Z directions, tell when the person was really doing bench press exercising and how many repetitions of bench press occurred.

People go to gym and in order to track their progress, they write down the exercises and number of repetitions with the weight they exercised. Purpose of this project is to automate counting repetitions for bench press exercise and tell when the exercise really happened using wearable electronic device connected via blue tooth to mobile application, so that this data can be used for tracking and checking the progress of working out.

The Berkley research paper achieves 90 % accuracy in exercise classification (but they focus on classifying different types of exercises) along with average 5% miscount of repetitions (averaged for all exercises), however they achieved 16% miscount for the bench press exercise using Naive Bayes and Hidden Markov models for classification and Hidden Markov Models for repetition counting.

The Microsoft research paper does 3 thing, 1. segmentation into exercise – non exercise, 2.classification of exercise type and 3. repetition counting.

They report 3 different precisions for segmentation, based on overlapping true exercise and predicted exercise:

100% for traditional - precision requires only that a predicted set uniquely overlap with a ground truth set, i.e. that the segmenter said a set happened here

98.8% for close - precision requires both boundaries of a predicted set to fall within five seconds of the corresponding ground truth

85.6% for tight - precision requires both boundaries of a predicted set to fall within two seconds of the corresponding ground truth set

They process many exercises, however bench press is surprisingly not one of them.

For the repetition counting, they present that for 77% of all exercises they have counted the number of repetitions exactly, 93% percent are within 1 repetition and 97% are within 2 repetitions miscount.

They do the counting by projecting 3D signal into 1D using PCA and counting the number of autocorrelation peaks of this projected signal.

There are many more papers about exercise type classification (bench press vs squat f.e.), however, as in our application people create their workout plan beforehand and we know what exercise are they going to do, we don't need to do that.


Preliminary results: what you have done so far

We wish to be able to count the number of bench press repetitions, right now it is implemented by state machine based on the values of the input signal. The problem is that it also counts repetitions when people are doing other movements, like walking or swinging their arms, that is why we want to be able to predict (classify) when is the exercise happening and when the movements are not the movements of bench press.

So far we have collected 100 minutes (300 000 discrete signal samples, 50 Hz sampling rate) of bench pres exercise data consisting of 9 signals (acceleration,gyro and gravity in X,Y, Z) using a mobile application connected to the sensor device recording device output while people were exercising.

We have then manually labeled whether each signal data sample belongs to actual bench press exercise or not using following notation: Exercising (1), Non Exercising (-1).

For the purpose of later classification of bench press exercising, we have extracted some features of the signal windows, as it is beneficial to look at wider range of signal than 1 data sample. We used 5 second window (having 250 data samples for each of the 9 signals).

Features we extracted so far are RMS, standard deviation and variance of each signal in the window independently. I will extract more features (autocorrelation features and powerbands as described later in the Work plan section).

This is the visualization of the raw data, some of it's corresponding extracted features from 5 second windows and than the label whether or not corresponding window corresponds to exercise.

Raw data: gravity vector in X (Green) Y (Red) Z (Blue)

Transformed data: few of the extracted features (RMS and others). I'll extract more features as a part of the project.

Ground Truth: manual label of exercise (1) vs non exercise (-1)

For each exercise (measurement) we have also stored the number of repetitions that was made in that exercise.

Work plan: what do you plan to achieve in this project and how you plan to achieve them.

1) Classification of bench press exercising

We plan to be able to distinguish, if certain window of signals corresponds to bench pres exercise or not.

Using sliding windows of length 5 seconds, I will extract following features of the windows

for each of the 9 signals the RMS, mean, standard deviation, variance of first and second half separately (microsoft research paper does this for smoother transition from exercising to nonexercising for windows overlapping both parts), also powerbands (frequency bin counts), and autocorrelation features: number of autocorrelation peaks, number of prominent peaks (thresholded peaks only), number of weak peaks (nonprominent), maximum autocorrelation value, hight of the first autocorrelation peak after zero crossing.

Then I will combine these features with the labels of the windows (Exercise/Nonexercise) to build SVM, Naive Bayes and Multilayer Perceptron classifiers.

I will then try to find small subset of the features, such that when I apply K-means and maybe other types of clustering algorithms with number of clusters = 2, it would best divide the windows into exercising - cluster1 and non exercising - cluster 2.

I will than compare the results of all used methods. (Confusion matrices).

There is no classification implemented yet, so there is nothing else to compare to, but the result of this classification will be used in next step, which will be quantified.

2) Repetition counting

We also want to tell, how many repetitions occurred in one exercise.

Again we have labels for each measured exercise (csv file), telling how many repetitions were made in corresponding exercise.

For the repetition counting, I will only use the part of the signal corresponding to the bench press exercise (result of the last step).

As the repetitions can be best seen on input gravity signal in X and Y direction, I will do the PCA of the Gravity vector in X and Y, project this 2D signal onto 1-st principal component (to get 1 D signal) than threshold the signal using percentiles (f.e. 33% and 67%) into peaks, middle values and valleys and then count the number of transitions from starting state back, ignoring the middle values. (Peak valley Peak, or Valley Peak Valley transitions, based on the firs non middle state).

So far, the task is done by using state machine implemented in the device itself. The counting for the bench press is now very bad. (On average 60% miss count).

I hope to achieve on average less than 2 (ideally less than 1) repetitions miscount on the dataset of the 100 minutes of exercising (of which 25% is real exercising and 75% are other movements).

Reference

http://research.microsoft.com/en-us/um/redmond/groups/cue/publications/Morris_Workout_CHI_2014.pdf

http://www.cs.berkeley.edu/~kenghao/publications/freeweight_ubicomp2007.PDF