Trajectory Forecasting for Powered Two Wheelers by Roll Angle Prediction with an LSTM Network

Type of the Paper: Conference Paper

Trajectory Forecasting for Powered Two Wheelers by Roll Angle Prediction with an LSTM Network

Karl Ludwig Stolle^1,*, Anja Wahl¹, and Stephan Schmidt²

¹ Robert Bosch GmbH, Germany; karlludwig.stolle@bosch.com, ORCID 0000-0002-6535-1894

² University of Applied Science Merseburg, Germany

*corresponding author

Name of Editor: Jason Moore

Submitted: 25/08/2023

Accepted: 07/09/2023

Published: 07/09/2023

Citation: Stolle, K., Wahl, A. & Schmidt, S. (2023). Trajectory Forecasting for Powered Two Wheelers by Roll Angle Prediction with an LSTM Network. The Evolving Scholar - BMD 2023, 5th Edition.

This work is licensed under a Creative Commons Attribution License (CC-BY).

Abstract:

Active safety systems for powered two wheelers (PTWs) are considered a key pillar to further reduce the number of accidents and thus of injured riders and fatalities. Enhanced awareness for the current riding situation is required to improve the performance of current systems as well as to enable new ones; this includes the detection of the rider’s intention – the action that is planned by the rider for the short-term future. The prediction of a continuous trajectory for the upcoming seconds of the ride is one way to express rider intention. Our work pursues the prediction of the PTW lateral dynamic state by means of a roll angle trajectory over the upcoming 4 s of riding. It thus considers the special vehicle dynamics characteristics of single-track vehicles that negotiate bends at a roll angle compared to cars. A deep learning (DL) prediction model that is based on a Long-Short Term Memory (LSTM) layer is optimized and trained for this task using a broad on-road riding dataset that focuses on the rural road environment. Inputs to the prediction model are PTW internal signals only, that are measurements of vehicle dynamics, rider inputs, and rider behavior. The latter two groups of signals are non-common for current series production PTWs and were especially added to our test bike before gathering the riding data set. The prediction performance of the optimized DL model is compared to a simple heuristic algorithm using multiple metrics in the roll angle and position trajectory domain. Evaluation on a representative test data set shows a significantly improved detection of rider intention by the DL model in all metrics. Reasonable lateral trajectory accuracy is achieved for at least 2 s of the total 4 s prediction horizon in 99 % of all test cases, given the chosen model architecture and input features. Furthermore, the feature importance of the especially added non-common measurement signals of steering and rider behavior is investigated in an ablation study. It reveals the importance of steering signals in the first second of the prediction horizon whereas the rider behavior signals aid trajectory prediction performance for up to 2.5 s.

Keywords: Deep Learning, Trajectory Prediction, Powered Two Wheeler, Rider Behavior, Riding Intention

Introduction

Three quarter of all PTW accidents in Europe are non-single accidents (Brown et al., 2021). Riders are especially prone to heavy injuries or fatalities in case of collision due to the inherently low passive safety of PTWs. Riding gear including airbags, that offer an additional cushioning space in case of an accident, are an ongoing trend that increases passive safety on PTWs, but their benefit in the field is not investigated yet (Tissot, Ballester, & Honoré, 2022). So, accident avoidance and mitigation by the help of active safety systems are objective of research and development in the field of PTWs.

The overall active safety systems landscape for PTWs can be differentiated into three levels as illustrated in Figure 1. On a first level, we find stabilization systems like Antilock Braking System (ABS) and Motorcycle Stability Control (MSC) that help avoiding accidents by intervening into the longitudinal dynamics of PTWs when instabilities are occurring, e.g., preventing the wheels from slipping or locking (Lich, Block, Prashanth, & Heiler, 2016). Stabilization systems successfully found their way into series PTWs for decades, reaching a broad spread nowadays. On a second level of active safety systems, Advanced Rider Assistance Systems (ARAS) are emerging in PTWs since a few years (Savino et al., 2020). Continuously assisting the riding task (e.g., distance-controlled cruise control ACC) and predictive warning ahead of critical situations (e.g., forward collision warning FCW) can reduce riding errors and thus help to lower the chance of accidents, being it single accidents or collisions with other traffic participants. ARAS systems rely on sensors that are on-board the PTWs capturing its environment. The third level of active safety systems utilizes connectivity to receive information about the environment from a backend or exchange information with other vehicles. The latter case is developed under the term Cooperative Intelligent Transport Systems (C-ITS), where traffic participants share their driving intentions with each other (Connected Motorcycle Consortium (CMC), 2023). By warning the rider and drivers of surrounding vehicles in case of an imminent risk of collision, such systems are suited to reach better collision avoidance potential than ARAS systems; they are yet not available in production.

Figure 1. Three levels of active safety systems for powered two wheelers with exemplary functions.

Beside the mentioned collisions, single vehicle accidents make up of a quarter of PTW accidents with fatalities or serious injured riders in Europe, while 64 % of them occur during cornering (Brown et al., 2021). Most single accidents are primarily caused by rider error and could therefore be avoided (Biral, Bosetti, & Lot, 2014). A prominent scenario of a single accident during cornering is known as ‘undercornering’ where a rider fails to follow the lane and leaves the curve towards its outside. It is hereby important to mention that undercornering is not caused by reaching the physical limit in most cases – e.g., the maximum roll angle for the given road conditions – but riders fail to increase the roll angle by their own error, e.g., due to corner fear (Scherer et al., 2021).

Future safety systems for PTWs, being it ARAS or C-ITS or a combination of both, strongly require information about the ego rider intention, either to share it with other traffic participants (C-ITS) and identify collision critical situations, or to identify riding errors. A performant detection of the rider intention will enable a multitude of functions that warn the rider or intervene in the controls to enhance riding safety. This challenging task is therefore the subject of current research. As an example for the required look-ahead time of a rider intention detection, the Connected Motorcycle Consortium (CMC) states that 6.5 s time before a collision (TTC) is needed to properly process and display a warning to riders for the left turn collision assistance system (CMC, 2023).

Recent work in the field of vehicle behavior prediction is introduced in the first section of this paper. Some investigations regarding rider behavior and intention detection for PTWs are presented as well. The specific problem statement and our approach to it are explained in a second section. Following is the description of the riding data used for training and testing a deep learning (DL) trajectory prediction model. Afterwards, the chosen DL prediction model architecture is described, and metrics used for model training and testing are explained. As first part of the results, the prediction performance of our DL model is contextualized by comparison to a heuristic approach. Secondly, the findings of our ablation study on the feature importance of the non-common measurement signals of rider steering inputs and rider behavior are presented.

Related work

Early research on vehicle behavior prediction was mostly concerned with physics-based approaches like filters (Lefèvre, Vasquez, & Laugier, 2014). These are not regarded as state-of-the-art anymore in the current research, which is mainly driven by autonomous driving applications (Mozaffari, Al-Jarrah, Dianati, Jennings, & Mouzakitis, 2022). Instead, DL based models are incorporated to predict intended behavior of traffic participants surrounding an autonomous agent that needs the information to plan and act in a safe and comfortable way. These surrounding vehicles are mostly cars due to their high share of representation in the available public datasets. Prediction of vehicle behavior can either be understood as classification problem where the output is a maneuver intention (high-level), or it is understood as regression problem where a continuous trajectory is output of the prediction model (Mozaffari et al., 2022). Either problem setting comes with advantages and disadvantages. Prediction of maneuver intention provides high-level understanding of the traffic scene but is restricted to a-priori defined maneuver classes. Whereas the biggest benefit of continuous trajectory prediction lies in its universality to traffic scenarios and the explicit statement of future vehicle states. The latter is very useful input to the downstream planning tasks. But the universality of regression may be of theoretical nature as specific model approaches are narrowing the problem statement down again to certain maneuvers or behaviors.

Recent examples of vehicle behavior prediction algorithms use information coming from environmental sensors of the ego vehicle or on infrastructure side as inputs, as they are mostly predicting the behavior of surrounding vehicles. Some approaches directly use the processed information of relative positions, velocities (or similar) of the surrounding that is assumed observable from the raw sensor data. As driver behavior is not assumed to fulfill the first-order Markov assumption, it is necessary to feed a time history of measurements instead of current state information only to any prediction model. Information about the road infrastructure like lanes and traffic rules are incorporated as additional inputs in some approaches (Mozaffari et al., 2022). In contrast, information that is assumed to contain valuable cues about the driving intention, like detailed measurements of vehicle dynamics and driver inputs, is not used in autonomous driving applications, because it is only available in the ego-trajectory prediction use case. Therefore, the transferability of the current state-of-the-art algorithms to the ego-trajectory prediction for PTW must be proven in our study.

Beside the classification type of behavior prediction model, Mozaffari et al. (2022) cluster regression models according to the number of predicted behavior modes (unimodal vs. multimodal) and differentiate whether the prediction model incorporates maneuver information (maneuver intention awareness). The latter type of approach can be seen as a combination of high-level maneuver classification and trajectory regression problem in a single model. So, a distinct split into classification and regression models is not applicable to a variety of more complex model types that incorporate both. A high share of DL based trajectory prediction models is based on Recurrent Neural Networks (RNN) because they are very powerful in extracting temporal information from time-series data (Mozaffari et al., 2022).

There is some related work available that aims directly on the behavior prediction of PTWs. Scherer and Basten (2022) present a heuristic algorithm in the form of a parameterizable mathematical model for the calculation of rider- and curve-individual roll angle trajectories. Their approach relies on the hypothesis that an individual rides through a specific curve in a repeatable way. Any explicit measurements on rider inputs or rider behavior are not incorporated in their work. The algorithm is presented for a single curve on an enclosed test track that was ridden multiple times by the participants of a riding study. A current whitepaper of CMC discusses the need for trajectory prediction for PTWs (CMC, 2023). They investigate the position trajectory prediction capability of simple algorithms that rely on current vehicle dynamic state information only; in summary, those algorithms assume either ‘constant heading’ or ‘constant curvature’ over the prediction horizon. Different turning maneuvers in an intersection scenario are simulated using a motorcycle multi-body model. A metric called evaluation index (EI) is introduced that is the maximum look-ahead time in the current riding state that can be predicted without the lateral distance error between the algorithm’s prediction and the ground truth trajectory getting larger than 2 m. The investigation shows insufficient prediction accuracy of all approaches tested as the EI metric declines to only about 1 s during the course of any turning maneuvers tested. Additionally, small in-lane corrections of the simulated rider during ‘going straight’ scenarios cause the EI to fall to 2-3 s already. Based on those results, the authors motivate the necessity of detecting rider intention and predicting the behavior based on the riding history to improve the accuracy, e.g., using machine learning techniques. The authors are not aware of any such research activities on trajectory prediction of PTW using machine-learned algorithms.

Approach

Our research approach on rider intention detection by roll angle trajectory prediction has been outlined in an extended abstract earlier (Stolle, Wahl, & Schmidt, 2023). The following describes it in greater detail. We aim for predicting cornering of PTW, as it is the case for the two PTW references presented above; thus, the future lateral dynamic state is predicted, not the longitudinal dynamics, e.g., forward velocity. We choose a machine learning approach by training a DL neural network on the prediction task as we are aiming to utilize typical rider intention patterns in riding data. Any prediction approach based on a physical model is limited to the time delay between rider inputs and the lateral dynamic vehicle states. A previous study revealed that the time delay between a steer torque rider input and the roll angle state ranges between 0.45 s and 2 s for the given test motorcycle, dependent on velocity (Stolle, Wahl, & Schmidt, 2022a). Consequently, empirical methods need to be applied to achieve further predictions. Upon reviewing the on-road riding data to be used in this work, which is highly variable in road and rider, it became apparent that the effort required to develop a heuristic model – like Scherer and Basten (2022) – that generalizes on the highly variable set of situations in the data set is unforeseeable high, coupled with uncertain chances of success. Therefore, it was decided to use the exploratory capability of a deep learning (DL) model to investigate the possibilities in predicting the rider cornering behavior.

Scope of our prediction algorithm is to use only internal PTW signals and no environmental sensing as input features, which means there is no information about road infrastructure or surrounding traffic participants. We can access a broad variety of vehicle dynamics signals and have special measurement signals of steering and rider behavior available; those are explained in more detail in the following section about the dataset. The decision to disregard any environmental information is made for multiple reasons. We focus on the rural road riding environment, especially as most single riding accidents occur there, where any infrastructure-based information is not expected to be available in the future. There is also no incorporation of surrounding vehicle information captured by on-board sensors as it is of minor importance for the rural road use case and single accidents. We also forgo to use any course information (e.g., from maps) as input for the rider intention detection for two main reasons. First, PTWs show a high variety of position within the own lane compared to non-single-track vehicles due to their narrow size; this makes their trajectories less tied to the course and thus poses challenges to localization. Secondly, using course information as input would hinder any prediction model to address riding errors where the course of the road is left, like the undercornering use case. Any learned algorithm would very likely not predict such a behavior as we fortunately don’t have major riding errors in the data set.

The task of detecting the rider intention for lateral dynamics is broken down to a regression problem. A DL model based on RNN layers is predicting the future trajectory of the roll angle state which strongly correlates with the lateral dynamics of a PTW – except for tilted roads, side wind or uneven loading of the PTW. Maximum prediction time, that is referred to as length of the prediction horizon, is defined to be 4 s. This lies in the range of other use cases in literature, and it is also expected to cover the maximum available performance that is feasible with the given input data. The detailed design of the DL model is introduced in the model section. Three research questions are to be answered using the developed trajectory prediction model for PTWs:

How much is the prediction improving over a simple benchmark algorithm when using a machine learning approach?

What is the importance of the especially added steering and rider behavior measurement signals?

Which prediction horizon is feasible with the developed DL model given the chosen input signals?

Riding data

A KTM 1290 Super Adventure motorcycle is equipped with data logging and several additional sensors for gathering vehicle dynamics, rider input and rider behavior data. Not all the measured signals are used as input features of the DL model, but a set of influential features is experimentally identified during model development. Table 1 provides an overview of the set of 16 signals and their respective sensor or source that are used as input features in the final prediction model.

Three ‘steering signals’ are measured at the steering system of the motorcycle that are non-common for series-production PTW. A linear potentiometer measures steering angle, an additional Inertial Measurement Unit (IMU) reads steering rate, and strain gauges applied to the handlebar measure the rider’s steering torque input. Furthermore, rider upper body and head movements are captured by a camera-based measurement system that was introduced by the authors for previous tests and is described in (Stolle, Wahl, & Schmidt, 2022b); those signals are referred to as ‘rider behavior’ measurements. Lateral upper body position is described by the two variables of relative lean angle between rider and motorcycle and a lateral offset of the hip point from the motorcycle center plane. Head movement is given as the rotational angle of the rider’s helmet around the vertical axis in the motorcycle’s frame coordinate system.

Table 1. List of measurements signals that are input features of the best DL prediction model and their source.
Abbreviations: IMU, Inertial Measurement Unit; MSC, Motorcycle Stability Control algorithm.

The usage of on-road measurement data poses additional challenges to any prediction model due to noise and disturbances in comparison to synthetically created data sets. Especially the steering torque signal suffers due to a low signal to noise ratio; one finds it to be smaller than unity in straight riding and also in most constant cornering situations. Filtering was addressed during model development with the ultimate result that the DL model’s best performance is with unfiltered data, which is probably due to the time delay that is added by the filtering process.

Figure 2. Left: definition of maneuver segments based on the lateral dynamic state (roll angle and roll rate) of a powered two-wheeler. Right: share of the lateral dynamic maneuver segments in overall prepared data (training, validation & test).

The on-road riding data set was gathered on various routes all over Southwest Germany and with a major focus on rural roads, which are typical for the sports/leisure use-case of PTWs in Europe; but still riding through towns, small cities, and on highway is also present to some extent. Having different routes for each ride ensures that a machine learning approach is not misled to learn course information. Overall, it comprises more than 70 h of riding by 21 riders of different riding experience, skills, and style. During data preparation, it is ensured that only riding with a velocity > 30 km/h is regarded for the prediction task to ensure the motorcycle is in the stable regime: this leaves ~ 65 h of data remaining. Besides, the data needs further preparation as one finds a strong bias towards straight riding in the data set due to its high appearance in real-world road design. Such a bias affects any learning based unimodal prediction model negatively. We are thus reducing straight riding by deleting 90 % of all data samples that are pure straight riding. This condenses the overall amount of data to ~ 49 h used for training, validation, and test of the DL model. The split into the latter three categories is chosen to be 75 %, 18 %, and 7 %.

The composition of the remaining prepared data set is further analyzed by assigning maneuver segment labels. The idea of labeling a PTW ride with maneuver segments according to the lateral dynamic states of roll angle and roll rate is taken from Magiera (2020); nine different maneuver segment classes are defined based on the PTW roll dynamic, those are illustrated in the left of Figure 2. The maneuver segment labeling process is realized with a state machine that distinguishes six transient from three quasi-stationary maneuver segments, mainly by the parameter roll rate. Quasi-stationary segments of straight riding and stationary cornering are separated by the roll angle state. Thresholds, hystereses, and other conditions of the state machine are chosen by expert knowledge with the objective of creating reasonable labels during cornering, i.e., a single corner without major riding line corrections should consist of a stationary cornering phase that is surrounded by roll-in and roll-out phase that start and end in straight riding. The distribution of maneuver segments over the whole prepared data set is presented in the pie chart on the right of Figure 2. One still notices a high share of straight riding which is caused by the length of samples (the sampling process is described in the ‘model & metrics’ section). Only samples with pure straight riding classification were deleted during data preparation. Samples that feature multiple maneuver segments over time still contain a high share of straight riding.

Model & Metrics

According to the previously described approach to rider intention detection, the specific trajectory prediction task of the DL model is introduced. Figure 3 illustrates the following description schematically. At a current point in time \(t_{\text{i}}\), a regression type DL model predicts future roll angle values \(P\) up to a certain maximum prediction time \(t_{\text{i+P}}\) at multiple discrete points that are evenly spaced along the prediction horizon with \(T_{\text{P}}\). Input to the model at time \(t_{\text{i}}\) is a time series of \(\text{n}\) features \(F_{\text{n}}\) from \(t_{\text{i-H}}\) in the past up to the current point in time \(t_{\text{i}}\), evenly sampled at discrete points with \(T_{\text{H}}\). The number and choice of the input features, input sampling time, and length of the input time series are all subject of hyperparameter optimization, whereas the prediction horizon length and output sampling time are preset to 4 s and 0.2 s each. The riding data is sliced into so-called samples during data preparation, where each sample ranges from \(t_{\text{i-H}}\) to \(t_{\text{i+P}}\) and contains all input features \(F_{\text{n}}\) and the desired ground truth output \(P\). One sample is created every 0.2 s, so samples are overlapping each other.

Figure 3. Illustration of the DL models prediction task at single time point ti. Discrete time series data of \(\text{n}\) features \(\mathbf{F}_{\text{n}}\) is input to the DL model. Its output is a prediction P at multiple discrete time points over a given prediction horizon.

The DL model architecture consists of one long short-term memory (LSTM) layer that is followed by a multilayer perceptron (MLP), which is a common neural network architecture for time-series prediction (Altche & De La Fortelle, 2017). The LSTM layer is a specific type of RNN, and it interprets the time-series of all input features into a hidden state. After processing the whole input time series, the LSTM’s final hidden state is fed into the MLP, which is a serial sequence of fully connected linear layers in our model. Definition of the network’s architecture is realized by hyperparameter optimization with the Python library Optuna. The following parameters were altered: size/number of LSTM layers, size/number of layers in the MLP, and the model’s training parameters of learning rate, batch size, weight initialization, and regularization. Optimization resulted in a single LSTM layer featuring 64 cells and two hidden linear layers of 64 and 32 neurons in the MLP before the output layer. All linear layers use the ReLU (Rectified Linear Unit) activation function.

Multiple metrics are used to evaluate the prediction model performance in training and testing, they are introduced hereinafter. The Mean Squared Error (MSE) of all predicted roll angle values along the prediction horizon is used as the DL model’s cost function during training; in combination with the Adam adaptive gradient descent learning strategy, this is a common choice for regression models (Bianchi, Maiorino, Kampffmeyer, Rizzi, & Jenssen, 2017). Calculating its root generates the ‘overall RMSE’, a more descriptive metric for model testing. In addition to a single cumulative RMSE value for the whole prediction, the roll angle error calculation is unraveled over the prediction horizon and the ‘RMSE over prediction horizon’ is analyzed for each of the 20 prediction points along the 4 s prediction horizon separately in testing. Furthermore, the prediction model’s performance will be compared to the results of the CMC investigations (CMC, 2023). This requires calculation of the EI metric that was mentioned in the related work. The roll angle trajectory output of our prediction model as well as the ground truth roll angle trajectory need to be transformed into position trajectories for this. A lateral position offset is then calculated based on these two position trajectories, which serves as basis for the EI.

Transforming a roll angle into a position trajectory means that we leave the lateral only domain as it requires the assumption of longitudinal velocity over the prediction horizon. We are using the ground truth future velocities for the transformation of the ground truth roll angles, and a constant velocity assumption for the transformation of the DL model’s predictions. First step of the transformation itself is the calculation of curvature, where a simple ‘single wheel’ model is used; it considers the width of the wheels

Figure 4. Example curve riding maneuver on rural roads. The prediction is analyzed in the marked blue section of 8.4 s length. Additional 1.6 s of riding history at the beginning and 4 s of prediction horizon at the end are displayed. A single situation of interest is marked with red star at 5.6 s.

as well as the center of gravity height of the PTW. The curvature is then propagated constantly between the 20 discrete point along the prediction horizon and relative positions (\(x\), \(y\)) are calculated accordingly. The RMSE of lateral distance between ground truth and predicted trajectory is evaluated at each point along the prediction horizon separately and the EI is calculated for each sample with a lateral error threshold of 2 m as in the CMC reference (CMC, 2023). The longitudinal distance error along the predicted relative position trajectory is not evaluated as the focus lies on lateral prediction.

Figure 5. Top: predictions at a single point of interest from the example curve riding situation, marked with red start in Figure 4, in further analysis. Top left: Roll angle prediction of machine learning (ML) model (‘best model’) compared to ground truth future roll angle and constant roll angle assumption. Top right: Relative position trajectories calculated from roll angle and velocity. EI metric with 2 m lateral distance error threshold marked. Bottom: EI metric evaluated over the whole example curve riding maneuver of Figure 4.

The procedure from roll angle prediction to EI metric shall be demonstrated for a single exemplary maneuver from the test data set to establish understanding of the just described. Figure 4 shows the course of the example cornering maneuver on rural roads in the map and the corresponding plots of roll angle and velocity. Data points that are underlined in blue are subject of the evaluation while the attachments at the beginning and end illustrate the DL model’s input history in the first and prediction horizon in the last test data point. The maneuver is a right-left S-curve that is already approached in a slight right turn. The following left curve can already be seen in the prediction horizon at the end of the maneuver.

The red star in Figure 4 marks a single point of interest at time 5.6 s for which the concrete predictions are presented in the upper two diagrams of Figure 5. Herein, the top left plot shows the ground truth roll angle over the 4 s prediction horizon beside the DL model’s prediction, denoted as ‘best model’, and a simple constant roll angle algorithm, that is equivalent to the assumption of constant curvature. Transformation into position trajectories results in longitudinal and lateral positions relative to the current state. Those trajectories are presented in the top right plot in Figure 5. Lateral distance errors are calculated by determining the distance of predictions perpendicular to the ground truth and the EI metrics for the black and blue trajectories in this single point in time are labelled in the plot too. The DL model reaches a prediction time of 2 s without the lateral distance error being larger than 2 m, while the simple assumption achieves 1 s only. The lower plot in Figure 5 shows the EI evaluated at each point of the exemplary test manoeuvre of Figure 4, where our best prediction model never goes below 1.8 s while the simple algorithm drops to 1 s in minimum.

Trajectory prediction compared to simple heuristic approach

The prediction performance of the optimized DL trajectory prediction model is presented in the following and is denoted as ‘best model’ in all graphics. This model has the 16 variables presented in Table 1 as inputs, sampled with a \(T_{\text{H}}\) of 20 ms over a signal history of 1.6 s. Its performance is compared to the simple heuristic algorithm of constant roll angle; this assumption equals a constant curvature logic which is presented as baseline in related work. Both predictions are evaluated on the same test data set that was drawn randomly before any DL training or optimization.

The left plot in Figure 6 shows the ‘RMSE over prediction horizon’ calculated for the roll angle predictions. One sees an advantage for the DL model over the simple approach for the whole prediction horizon as the gap between dotted blue and dashed black curve is ever increasing, while the biggest improvement, i.e., the strongest build-up of the gap, occurs within the first second of prediction. The ‘overall RMSE’ of roll angle is 56 % higher for the constant roll angle logic compared to the best model. Transforming roll angle into position trajectories, using the constant velocity assumption for both models, results in the lateral distance ‘RMSE over prediction horizon’ that is presented in the right plot in Figure 6. The gap between DL model and simple algorithm increases progressively as the transformation is integrating in nature. The constant roll angle algorithm is 83 % worse than the best model in ‘overall RMSE’ of lateral distance.

Figure 6. ‘RMSE over prediction horizon’ metric evaluated on test data for the best deep learning prediction model (blue dots) and the simple constant roll angle model (black). Left: roll angle predictions. Right: lateral distance evaluated after transforming roll angle into relative position trajectories.

Evaluating the EI metric for each sample of the test data set shows smallest occurring EIs of 1 s for the simple constant roll angle algorithm and 1.4 s for the best model. Comparing the occurrence of EIs in the histogram in Figure 7 reveals a distinct improvement of EI for the DL model in blue. Coming from the grey striped constant roll angle baseline, the number of test samples with EI smaller than 2 s decreases by 91 %, whereas the samples with an EI greater or equal 3 s increase by 26 %. In absolute numbers, the best model achieves at least 2 s of EI for 99.1 % of all test samples and for 80.5 % of all test samples an EI of 3 s or more is realized.

Figure 7. Relative frequency of Evaluation Index (EI) metric evaluated for each sample of the test data set with a lateral error threshold of 2 m. Best deep learning prediction model displayed in blue and simple constant roll angle model in striped grey.

Ablation study of non-common measurement signals

An ablation study on the importance of the non-common measurement signals as input features for the DL model is carried out. Ablation models are trained without specific input signals and are then compared to the best model for this. The results of three ablation models, where groups of input signals were removed, are illustrated in three individual diagrams in Figure 8. Each diagram shows the ‘RMSE over prediction horizon’ curves of the roll angle predictions for the ablation model and the best model in comparison. The difference between those two curves is shown separately as a dashed line with dots to aid interpretation. A performance degradation caused by the removal of input features is visible for all three configurations that were tested. Those were the ablation of: a) steering signals (steering angle, rate & torque), b) rider behavior signals (rider head yaw angle, rider upper body lean angle & lateral offset of the hip point) and c) both steering and rider behavior signals. The observed effects are described in more detail below.

Removing the steering signals causes the ‘overall RMSE’ of roll angle to increase by 1 % (3.5 % in ‘overall RMSE’ of lateral distance) compared to the best model. One can see the green curve of the ablated model moving upwards away from the blue curve towards larger errors right at the beginning of the prediction horizon in Figure 8a. The dashed curve, showing the difference between both models, reaches its maximum at a prediction horizon of 0.8 s. For the further prediction horizon, the difference in prediction performance is decreasing again and is almost identical from 3 s onwards. This means that the ablated model can almost make up for the disadvantage it suffered between 0.2 and 0.8 s due to the absence of steering signals. We conclude from the observation, that the distance between the error curves of both models only increases in the range from 0.2 to 0.8 s, that the steering signals contain valuable information for prediction only for this time horizon.

Removing the rider behavior features leads to a 5 % increase in ‘overall RMSE’ of roll angle (6.2 % in ‘overall RMSE’ of lateral distance) compared to the best model. Following the same line of argumentation as for the steering signals, the behavior of the ‘RMSE over prediction horizon’ curves in Figure 8b reveals that rider behavior signals contain valuable information for the roll angle trajectory prediction in between 0.4 and ~2.5 s of the prediction horizon. Interestingly, there is even a slight improvement from 0.2 to 0.4 s without the mentioned signals, but this has no significant influence on the ‘overall RMSE’. Unlike before, the performance disadvantage cannot be recovered before reaching the maximum prediction horizon of 4 s.

A roll angle trajectory prediction model without the steering system and rider behavior signals has only common measurement signals (regarding state-of-the-art PTWs) remaining. It’s performance difference compared to the best model is shown in Figure 8c where the gap between ablated and best model builds up between 0.2 and ~1.9 s of the prediction horizon. Having only the standard signals causes the highest degradation of prediction performance of all three configurations tested. The ‘overall RMSE’ of roll angle increases by 6.6 % (11.1 % in ‘overall RMSE’ of lateral distance) which is even a bit more than the sum of the single ablation effects presented before.

Conclusion

Research on PTW active safety systems aims for an improved detection of rider intention, with this work focusing on cornering. Prediction of the lateral dynamic state of roll angle over 4 s of future riding is realized by optimization of a DL model based on a LSTM layer. A diverse and broad riding data set, that focuses on the rural road environment, is used for development and testing. Only PTW internal measurements of vehicle dynamics and especially added signals of steering and rider behavior are applied as features to the DL model. The performance of the developed prediction model is evaluated using different metrics in the roll angle as well as position trajectory domain and is compared to a simple heuristic algorithm assuming constant roll angle. An ablation study on the importance of the especially added measurements is performed as well, to find answer to the research questions posed.

How much is the prediction improving over a simple benchmark algorithm when using a machine learning approach?

Testing on a representative 7 % of all riding data available reveals strong improvements of the DL model over the simple heuristic of constant roll angle that are visible in all metrics evaluated. E.g., the occurrence of riding situations that have less than 2 s of feasible look-ahead time (EI metric) – defined by a lateral trajectory offset < 2 m – is lowered by over 90 %. In total numbers, an EI of at least 2 s is achieved in 99.1 % of all test cases.

What is the importance of the especially added steering and rider behavior measurement signals?

The results of the ablation study reveal the importance of the non-common input features of steering and rider behavior signals. The three signals of steering torque, angle, and rate do only benefit in first second of prediction, their overall effect is thus small. Removing rider head yaw angle, rider upper body lean angle and lateral offset of the hip point from the set of input features has a stronger effect on prediction performance as those signals incorporate valuable information for trajectory prediction up to a prediction horizon of 2.5 s. Removing all non-common signals causes the biggest deterioration of prediction performance of 11 % in overall lateral distance error.

Which prediction horizon is feasible with the developed DL model given the chosen input signals?

The distribution of the EI metric over all test samples as well as the ablation study give indications on the prediction horizon that is feasible with reasonable accuracy. The EI shows that a minimum of 2 s of prediction horizon without a lateral distance error > 2 m is achievable in most cornering situations as 99 % of all test samples achieve this value. Similarly, the ablation study indicates that there is valuable information in the rider behavior input features for a prediction horizon of up to 2.5 s.

In summary, the LSTM based DL model predicting a roll angle trajectory is a promising approach for the detection of the rider intention regarding lateral dynamics. The method demonstrates that there is information in the history of the time-series vehicle dynamics and rider behavior measurement signals of a PTW that is valuable for the prediction of cornering. It significantly improves over simple heuristic approaches that only take the current state of the PTW into account. However, the method presented did not consider longitudinal dynamics, for which the simple assumption of constant velocity was made. It remains to investigate the subsequent drawback in the metrics under consideration. In addition, the extension of the method to include prediction of the longitudinal dynamics will be tested. Future work on the DL trajectory prediction model will also investigate in detail how different maneuvers and riding styles affect the prediction.

References

Altche, F., & De La Fortelle, A. (2017). An LSTM network for highway trajectory prediction. In IEEE 20th International Conference on Intelligent Transportation Systems (ITSC) (pp. 353–359). IEEE.

Bianchi, F. M., Maiorino, E., Kampffmeyer, M. C., Rizzi, A., & Jenssen, R. (2017). Recurrent neural networks for short-term load forecasting: An overview and comparative analysis. SpringerBriefs in computer science. Springer.

Biral, F., Bosetti, P., & Lot, R. (2014). Experimental evaluation of a system for assisting motorcyclists to safely ride road bends. European Transport Research Review, 6(4), 411–423. https://doi.org/10.1007/s12544-014-0140-6

Brown, L., Morris, A., Thomas, P., Ekambaram, K., Margaritis, D., Davidse, R., Usami, D. S., Robibaro, M., Persia, L., Buttler, I., Ziakopoulos, A., Theofilatos, A., Yannis, G., Martin, A., & Wadji, F. (2021). Investigation of accidents involving powered two wheelers and bicycles - A European in-depth study. Journal of Safety Research, 76, 135–145. https://doi.org/10.1016/j.jsr.2020.12.015

Connected Motorcycle Consortium. (2023, April 7). Path Prediction for PTWs: CMC Whitepaper. https://www.cmc-info.net/uploads/1/2/1/4/121453783/whitepaper_path_prediction.pdf

Lefèvre, S., Vasquez, D., & Laugier, C. (2014). A survey on motion prediction and risk assessment for intelligent vehicles. ROBOMECH Journal, 1(1). https://doi.org/10.1186/s40648-014-0001-z

Lich, T., Block, W. G., Prashanth, S. N., & Heiler, B. (2016). Motorcycle Stability Control - The Next Generation of Motorcycle Safety and Riding Dynamics. SAE International Journal of Engines, 9(1), 491–498. https://doi.org/10.4271/2015-32-0834

Magiera, N. (2020). Identifikation des Fahrfertigkeitsniveaus von Motorradfahrern in Kurvenfahrt im Realverkehr, Technische Universität Darmstadt. 10.25534/tuprints-00014011.

Mozaffari, S., Al-Jarrah, O. Y., Dianati, M., Jennings, P., & Mouzakitis, A. (2022). Deep Learning-Based Vehicle Behavior Prediction for Autonomous Driving Applications: A Review. IEEE Transactions on Intelligent Transportation Systems, 23(1), 33–47. https://doi.org/10.1109/TITS.2020.3012034

Savino, G., Lot, R., Massaro, M., Rizzi, M., Symeonidis, I., Will, S., & Brown, J. (2020). Active safety systems for powered two-wheelers: A systematic review. Traffic Injury Prevention, 21(1), 78–86. https://doi.org/10.1080/15389588.2019.1700408

Scherer, F., & Basten, T. (2022). Entwicklung eines Motorradfahrendenmodells zur Trajektorienprädiktion. In Insitut für Zweiradsicherheit (ifz) e.V. (Ed.), ifz-Research Publication Series: Nr. 20, Sicherheit - Umwelt - Zukunft XIV: Tagungsband der 14. Internationalen Motorradkonferenz 2022 (pp. 265–299). Institut für Zweiradsicherheit.

Scherer, F., Winner, H., Pleß, R., Will, S., Neukum, A., Stanglmaier, M., Bäumler, M., Siebke, C., & Prokop, G. (Eds.). (2021). Berichte der Bundesanstalt für Straßenwesen F, Fahrzeugtechnik: Heft 142. Schräglagenangst: = Corner fear. Fachverlag NW in Carl Ed. Schünemann KG. https://bast.opus.hbz-nrw.de

Stolle, K. L., Wahl, A., & Schmidt, S. (2022a). Importance of motorcycle rider upper body movement for rider intention detection and motorcycle state prediction. In Insitut für Zweiradsicherheit (ifz) e.V. (Ed.), ifz-Research Publication Series: Nr. 20, Sicherheit - Umwelt - Zukunft XIV: Tagungsband der 14. Internationalen Motorradkonferenz 2022 (pp. 32–41). Institut für Zweiradsicherheit.

Stolle, K. L., Wahl, A., & Schmidt, S. (2022b). Motorcycle rider posture measurement for on-road experiments on rider intention detection. In 2022 IEEE 1st International Conference on Cognitive Mobility (CogMob) (pp. 51–56). IEEE. https://doi.org/10.1109/CogMob55547.2022.10118004

Stolle, K. L., Wahl, A., & Schmidt, S. (2023). Trajectory Prediction for Powered Two Wheelers with Deep Learning. The Evolving Scholar - BMD 2023, 5th Edition. https://doi.org/10.24404/63fdc49b743b257fd7f65c53

Tissot, P.-F., Ballester, O. C., & Honoré, V. (2022). Smart wearable airbags: Benefits for users and remaining challenges after 60 million kilometres and 3000 accidents. In Insitut für Zweiradsicherheit (ifz) e.V. (Ed.), ifz-Research Publication Series: Nr. 20, Sicherheit - Umwelt - Zukunft XIV: Tagungsband der 14. Internationalen Motorradkonferenz 2022 (pp. 202–207). Institut für Zweiradsicherheit.