Forty-three recreational runners (30 males, 13 females, age: 25 ± 3 years, height: 180 ± 16 cm, weight 82 ± 20 kg) volunteered to participate in the study. The sample size in this study is in accordance with current recommendations for the conduction of running biomechanics studies (Oliveira and Pirscoveanu 2021) and may provide sufficient data for machine learning regression model predictions. The group presented 8 ± 4 years of running experience and a weekly running volume of 24 ± 15 km. Their self-reported 5-km race pace was 4:40 ± 0:38 min/km. Inclusion criteria included the current practice of running training protocol and being injury-free for a minimum of 6 months before the test. Participants were asked to avoid performing any strenuous exercise 24 h before the test, as well as avoid consuming caffeine and alcohol within the 12 h preceding the experiment. Participants were verbally informed about the experimental procedure and provided verbal and written informed consent to participate in this study. The local ethical committee (Region Nordjylland, Denmark) approved the procedures applied in the study, and all methods were carried out in accordance with relevant guidelines and regulations from the Declaration of Helsinki (2004).

Experimental design

In a single session, participants were initially provided with a 10-min warm-up consisting of 2 laps on a 400-m outdoor running track, walking lunges, running with high knees, and leg swings (Pirscoveanu et al. 2021). Subsequently, runners were asked to perform a simulated running race until exhaustion on the running track using their preferred regular running shoes. Runners were asked to maintain a stable running speed based on the reported 5-km race pace throughout the test. The running speed was measured continuously through the embedded GPS on the smartwatch. Moreover, the consistency of the running speed throughout the test was assured by the experimenters checking if the runner was within ± 3 s of the expected time at every 200. RPE was assessed at the end of every 400 m using the 6–20 Borg scale. The test was terminated when the runner could not maintain a constant speed in two consecutive speed checks. Therefore, some participants exceeded the 5 km distance requirement. Figure 1A illustrates the experimental design and the variables acquired during the experiments.

Fig. 1

Data acquisition and analysis. In A, the rating of perceived exertion (RPE) data from 5 km simulated running was extracted at every 400 m, being subsequently interpolated to match the same number of samples in the running biomechanical variables. Both RPE and running mechanics data were reduced to 5-s windows for machine learning predictions. In B, the subject-independent machine learning models were applied using a leave-one-out approach where all runners (marked as “R”) but one were used for training the model. The excluded runner was the test dataset. In C, the subject-dependent machine learning models were applied by splitting the dataset of a given subject into a training set and a test set. The test set comprised 5%, 10%, or 20% of the total amount of data of the runner (the figure exemplifies 10% of the total data). A similar amount of test data was extracted from four different splits of the dataset (0–25%, 26–50%, 51–75%, and 76–100%), assuring balanced exposure to biomechanical behavior across the entire exercise

Data acquisition and analysis

A commercial smartwatch (Garmin Forerunner 735XT, Garmin International, Kansas City, MO) was used to assess running speed, heart rate, and running biomechanical parameters as described elsewhere (Pirscoveanu et al. 2023). A compatible chest strap containing heart rate sensors and a tri-axial accelerometer was used to acquire instantaneous heart rate and trunk accelerometry. The data from both heart rate sensors and accelerometry cannot be accessed for customized data processing. However, the smartwatch utilizes data processing algorithms that outputs heart rate and running biomechanical parameters such as running cadence, running speed, vertical oscillation, stride length, foot contact time, and foot contact time symmetry. All smartwatch data were sampled at 1 Hz. The validity of this type of device has been previously verified for the assessment of heart rate (Price et al. 2017; Støve et al. 2019), as well as running cadence, ground contact time, and vertical oscillation (Adams et al. 2016; Carrier et al. 2020). The data processing was performed using custom-made scripts (Matlab 2020b, The Mathworks Inc., Natick, MA, USA). The RPE values were interpolated between laps with a 1 Hz frequency to match the number of samples extracted from the running parameters. The heart rate was normalized by the maximum expected heart rate using previous literature (Tanaka et al. 2001). Subsequently, the biomechanical and RPE data from all runners were reduced to averaged points for every 5 s.

Machine learning—model validation

The regression learning app from Matlab (Matlab 2020b, The Mathworks Inc., Natick, MA, USA) was used to run several machine learning algorithms using a fivefold validation method. Feature scaling was applied to each feature to prevent the feature magnitude from affecting the learning process. Scaling of features to:

$$\left[0, 1\right]: ^=\frac\left(x\right)}\left(x\right)-\mathrm\left(x\right)},$$

where x is the original value and xʹ is the scaled value.

We allocated the data from 26 runners (~ 60% of the sample) to validate machine learning models in the three different datasets. We first trained and validated models to predict the RPE using the runner’s age, body mass, and body height, as well as the running biomechanical data (running distance, heart rate, cadence, vertical oscillation, stride frequency, ground contact time, and ground contact time symmetry). Running speed was excluded from the training models since it was set to be a constant value across the experiment. The training models using all ten variables presented the best performance (Supplementary Fig. 1, white bars). However, the implementation of a leave-one-out design to predict the RPE from a single individual cannot include invariant features such as age, body mass, and body height, since they have no variability across the running trial. Therefore, two other models were validated: a model with only biomechanical variables except for running speed (Supplementary Fig. 1, gray bars) and a model with only biomechanical variables except running speed and running distance (Supplementary Fig. 1, black bars). This third model was tested since running distance is a cumulative variable that may highly correlate with RPE.

We found that the inclusion of all variables resulted in the lowest prediction errors, followed by the models using biomechanical data except running speed. As expected, removing the running distance from the training dataset substantially increased the prediction errors regardless of the applied algorithm. Moreover, classic linear models such as linear regression and linear support vector machine presented substantial errors > 1 RPE, whereas the best-performing model that included running distance (gray bars) was the Gaussian process regression (GPR, rational quadratic) with an error = 0.22. Therefore, further RPE predictions were conducted using the GPR rational quadratic algorithm, including all biomechanical variables except running speed. The GPR applies Bayesian non-parametric regressions to compute joint multivariate Gaussian posterior distributions of a test set when given a training set (Schulz et al. 2018). The GPR can be advantageous for its ability to make predictions using fewer parameters.

Machine learning—subject-independent model

Predictions of RPE over time were processed using a leave-one-out approach (Fig. 1B), where the training dataset consisted of all available data, except for the participant being tested (e.g., 42 participants in the training set, 1 participant in the testing set). The leave-one-out approach assures that no RPE information from the predicted participant was included in the training data (Derie et al. 2020).

Machine learning—subject-dependent model

Subject-dependent models trained a personalized model for each runner, albeit only using data from that runner. Such type of model may be relevant for individualized predictions in case generalized subject-independent models do not provide sufficient accuracy. The data from a given runner was split into four-time sectors (0–25%, 26–50%, 51–75%, and 76–100% of the total amount of data). The data split into four sectors assured a balanced amount of data points throughout the − 5-km simulated race (Fig. 1C). Three subject-dependent models were evaluated in this study, allocating random samples of 5%, 10% or 20% of each data split into the model test set. Models that require only a fraction of the data but still provide acceptable accuracy may be more relevant to implementing the technology in real-time settings. The prediction of RPE was repeated ten times for each runner in each of the three test set dimensions to increase the variability of the test datasets. The final prediction accuracy result was an average across the ten predictions for each runner.

Model evaluation and statistical analysis

The prediction quality from the proposed models to predict RPE using running biomechanical data was tested using Pearson’s correlation coefficient (r), absolute root-mean-square error (RMSE), and relative RMSE (rRMSE). The real and predicted RPE from the subject-independent models were averaged from five sequential measurements (~ 20 s of continuous recordings) at 25%, 50% 75%, and 100% of the total running time from each runner. Subsequently, the real and predicted RPEs in each running time percentage were compared using Bland–Altman plots displaying mean biases and the limits of agreement (e.g., 95% confidence interval). Moreover, the two-tailed t Student test and the respective Cohen’s D effect size (“small” values around 0.2, “medium” for 0.5, and “large” above 0.8 (Durlak 2009) were computed for each pairwise comparison. Regarding the subject-dependent models, we assessed the effect of the RPE extraction method (real vs predicted with 5%, 10%, and 20% training data) on the RPE at 25%, 50%, 75%, and 100% of the total running time using a one-way ANOVA, with Bonferroni pairwise post hoc tests if necessary. Bland–Altman plots were also generated for the pairwise comparisons of the subject-dependent models (shown in Supplementary Fig. 2 and Supplementary Table 1) The significance level for all statistical tests was set at p < 0.05.