Validation of a bicycle simulator based on objective criteria

Type of the Paper: Conference Paper

Validation of a bicycle simulator based on objective criteria

Donaji Martinez Garcia¹*, Kilian Gröne¹, Martin Fischer¹, Min Zhao¹ and Melina Bergen¹

¹German Aerospace Center - DLR; Donaji.MartinezGarcia@dlr.de, ORCID 0000-0002-0839-0627; Kilian.Groene@dlr.de, ORCID 0000-0001-9035-4440; Ma.Fischer@dlr.de, ORCID 0000-0001-8435-4321; Min.Zhao@dlr.de, ORCID 0000-0002-0485-5255; Melina.Bergen@dlr.de, ORCID 0009-0009-0727-0218

*corresponding author.

Name of Editor: Jason Moore

Submitted: 22/09/2023

Accepted: 22/09/2023

Published: 03/10/2023

Martinez Garcia, D., Gröne, K., Fischer, M., Zhao, M. & Bergen, M. (2023). Validation of a bicycle simulator based on objective criteria. The Evolving Scholar - BMD 2023, 5th Edition.

This work is licensed under a Creative Commons Attribution License (CC-BY).

Abstract:

Statistics show, that bicycles become more and more popular as transportation method, e.g. 25% increase in Germany between 2019 and 2021 (Sinus, 2021). To ensure the safety of bicycle riders as vulnerable road users (VRUs), analysing critical traffic situations is essential (Wendel, 2020). To be able to explore such situations in a safe environment, a bicycle simulator was built at DLR that can be used stand-alone or in combination with other simulators in order to integrate other traffic participants such as pedestrians or car drivers (Fischer et al., 2022; Martinez Garcia, 2021). This work describes the development of the simulator with the goal of creating a realistic and therefore immersive cycling experience (Jacobi, 2022; Janssen, 2022). A detailed description of the implementation of the recent improvements is provided as well as an objective evaluation for validation of the simulator.

Keywords: Bicycle Simulator, Vr, Bicycle Dynamics, Validation, Objective Criteria

Introduction

Simulator studies are valid when they provide results that can be generalized to real-world situations and the occurrence of unwanted symptoms such as simulator sickness doesn´t influence the results (Shoman & Imine, 2021). The validation of simulators is very important, as it determines to which extent the results of a simulator study can be transferred to the real world. The goal of simulator designers is to evoke realistic riding behaviour. Behavioural validity is a measure of how the participants feel and act while driving on the simulator. To evaluate behavioural validity, performance measures of a simulator can be compared with those of a real bicycle riding on-road (O’Hern et al., 2017). A realistic perception of an environment is crucial to evoke valid and reliable behaviour. This includes infrastructure, other traffic participants, and the experience of cycling.

The study presented in this paper aimed to evaluate a new control logic together with hardware changes done to the DLR bicycle simulator and to identify in which areas it is possible to fine-tune the simulator so that it allows for a more realistic behaviour. For this purpose, two different set-ups of the simulator will be compared based on objective criteria first in a simulator study and then to data obtained with an equipped research bicycle. A parameterization of the simulator based on real-life information enables the delivery of realistic feedback to the users by adjusting the dynamic model, motion cueing and hardware modules. These initial steps are used as a basis for the process of obtaining an immersive and realistic cycling behaviour and motion perception. This should help to improve the bicycle simulator to represent real traffic situations in a virtual scenario and create an immersive environment.

Experimental set-up

With the goal of improving the cycling sensation and on the basis of previous evaluations (Martinez Garcia, 2022), a series of developmental changes were performed to the simulator. For the validation of the simulator data, an e-bike was instrumented with sensors with the main purpose of measuring the steering angle, leaning angle and cycling velocity. The final set-up of both the simulator and the real bicycle are described below.

Bicycle simulator

The dynamic simulator setup is depicted in Figure 1. The main objective is to achieve a high quality of the simulator and therefore provide a realistic experience. To achieve this, the force feedback modules and the real-time simulation of the bicycle must have a corresponding performance. The right fine-tuning is crucial for creating a highly immersive driving experience.

Figure 1. Bicycle simulator at the MoSAIC-VRU-Lab from the German Aerospace Center (DLR)

Two different set-ups of the DLR bicycle simulator (Martinez Garcia et al., 2023; Martínez García et al., 2022) were tested in different scenarios (s. Table 2).

For the lateral dynamics on V1.1, the force feedback of the steering motor was calculated by a modified steering force simulation based on motorized vehicles, whereas on V2.0 it was calculated based on the Whipple-bicycle physics model (Meijaard et al., 2007). The lean angle behaviour was controlled either by the force of the body (V1.1) or by the position of the steering angle (V2.0). For the longitudinal dynamics, on V2.0 the measurement of cycling velocity and braking was performed with an incremental encoder that has higher resolution and lower latency than the bicycle trainer used in V 1.1. In addition, the wind simulator was configured to deliver wind dynamically (depending on cycling velocity), whereas in V1.1 it was static and constantly delivered the same amount of air at the same velocity.

Within the next paragraphs, the underlying formulas and mechanical set-ups for both versions are described in more detail.

Table 1. Features of both versions of the bicycle simulator concerning lateral and longitudinal behaviour

	Feature	V1.1	V2.0
Lateral	Steering	Look-up table with adapted vehicle dynamics steering forces	Whipple-bicycle physics model
	Leaning	Force control	Position control
Longitudinal	Acceleration	Bike trainer	Incremental encoder
	Braking	Disc brake + information of bike trainer	Disc brake + information of incremental encoder
	Wind	Static	Dynamic

Visualization - The visualization was developed by using Unreal Engine 4. The visual system, which generates visual stimuli by displaying a virtual reality (VR), is implemented in two different ways. One possibility for visualizing the driving environment is transmission on a single monitor and another is the use of an HMD. Through the interaction of internal and external sensors, the position and orientation of the VR goggles in space can be detected. In contrast to the monitor, it is thus possible to adopt a 360° viewing angle independent of the direction of travel. For the presented study, only the VR googles were used.

Haptics - With the use of haptic gloves, the movements and position of the hands are tracked and displayed in the visualization, which is displayed as part of the virtual environment.

Bike trainer - To create adjustable resistance when pedalling, a Tacx® FLUX 2 Smart trainer is used to replace the rear wheel. This system provides a measurement of the driving velocity which was first utilized for the visualization for V1.1. Due to abrupt changes in the perceived movements in the visualization, a more precise measuring method was implemented for V2.0. An optical incremental encoder (Kübler Automation 8.KIH40.5462.2048) was placed on the shaft of the bike trainer, which allows a maximal resolution of 250 ppr with a maximal pulse frequency of 250 kHz. Figure 2 shows the assembly of the incremental encoder and a disc brake to the shaft of the bike trainer (Jacobi, 2022).

Figure 2. Assembly of the incremental encoder and a disc brake

To calculate the velocity value for V2.0, the number of measured increments is recorded in a specific time interval. The calculation for the virtual cycling velocity \(\mathbf{v}_{\mathbf{F\ }}\)is based on a bicycle with a wheel radius \(\mathbf{r}_{\mathbf{wheel}}\) of 28 ":

\[\mathbf{v}_{\mathbf{F}}\mathbf{=}\mathbf{r}_{\mathbf{wheel}}\mathbf{\bullet \ }\mathbf{\omega}_{\mathbf{wheel}}\] \[\mathbf{with\ \ }\mathbf{\omega}_{\mathbf{wheel}}\mathbf{= 2\ \bullet \ \pi\ \bullet}\mathbf{n}_{\mathbf{pinion}}\mathbf{\ }\]	(1)

The internal transmission ratio of the roller trainer \(\mathbf{i}\) is:

\[\mathbf{i = \ }\frac{\mathbf{z}_{\mathbf{downthrust}}}{\mathbf{z}_{\mathbf{propulsion}}}\mathbf{= \ }\frac{\mathbf{1}}{\mathbf{7}}\mathbf{= \ }\frac{\mathbf{n}_{\mathbf{pinion}}}{\mathbf{n}_{\mathbf{SR}}}\]	(2)

This now allows the rotation velocity of the flywheel to be determined considering the resolution of the incremental encoder:

\[\mathbf{n}_{\mathbf{SR}}\mathbf{= \ }\frac{\mathbf{rotations}}{\mathbf{\mathrm{\Delta}t}}\mathbf{=}\frac{\frac{\mathbf{\mathrm{\Delta}increments}}{\mathbf{resolution}}}{\mathbf{t}_{\mathbf{now}}\mathbf{-}\mathbf{t}_{\mathbf{end}}}\]	(3)

By substituting (2) and (3) in the basic formula (1), we can now calculate the cycling velocity \(\mathbf{v}_{\mathbf{F}}\):

\[\mathbf{v}_{\mathbf{F}}\mathbf{= \ }\frac{\mathbf{r}_{\mathbf{wheel}}\mathbf{\bullet 2 \bullet \pi \bullet \mathrm{\Delta}increments}}{\mathbf{7 \bullet (}\mathbf{t}_{\mathbf{now}}\mathbf{-}\mathbf{t}_{\mathbf{end}}\mathbf{) \bullet resolution}}\]	(4)

Wind - A WAHOO® KICKR Headwind Simulator is used to aid immersion and reduce motion sickness. This device is capable of providing velocity-based airflow by communicating with the bike trainer via Bluetooth Low Energy (BLE). For V1.1 only static wind was used, which was improved for V2.0 by varying the wind depending on the cycling velocity.

Steering motor - A servomotor (Schneider Electric BSH1003P31A2A) is used to control the steering behaviour. Direct coupling with the handlebar creates a haptic steering system which, in conjunction with the dynamics model, provides a steering-angle-dependent resistance torque. Where V1.1 used a simple lookup-table with adapted vehicle dynamics steering forces, a new approach for the dynamics model based on the Whipple-bicycle physics model (Meijaard et al., 2007) was developed (Janssen, 2022) for V2.0. The mathematical model of Schwab and Recuero, based on the findings of Meijaard et al. was used to calculate the force feedback \(\mathbf{T}_{\mathbf{f}}\) for the handlebars (Schwab & Recuero, 2013). The starting formula for the counter moment is:

\[\mathbf{M}\ddot{\mathbf{q}}\mathbf{+ C}\dot{\mathbf{q}}\mathbf{+ Kq = M}\ddot{\mathbf{q}}\mathbf{+}\mathbf{vC}_{\mathbf{1}}\dot{\mathbf{q}}\mathbf{+}\left( \mathbf{gK}_{\mathbf{0}}\mathbf{+}\mathbf{v}^{\mathbf{2}}\mathbf{K}_{\mathbf{2}} \right)\mathbf{q = f}\] with the time-varying quantities: \(\mathbf{q =}\mathbf{(\phi,\ \delta)}^{\mathbf{T}}\mathbf{\ and\ f =}{\mathbf{(}\mathbf{T}_{\mathbf{\phi}}\mathbf{,}\mathbf{T}_{\mathbf{\delta}}\mathbf{)}}^{\mathbf{T}}\)

(5)

From this basic equation and its matrix form, the counter torque of the handlebar or the force feedback \(\mathbf{T}_{\mathbf{f}}\) can be calculated:

\[\mathbf{T}_{\mathbf{f}}\mathbf{= - (}\mathbf{M}_{\mathbf{\delta\phi}}\ddot{\mathbf{\phi}}\mathbf{+}\mathbf{C}_{\mathbf{\delta\phi}}\dot{\mathbf{\phi}}\mathbf{+}\mathbf{C}_{\mathbf{\delta\delta}}\dot{\mathbf{\delta}}\mathbf{+}\mathbf{K}_{\mathbf{\delta\phi}}\mathbf{\phi +}\mathbf{K}_{\mathbf{\delta\delta}}\mathbf{\delta)}\]	(6)

The calculations for the individual matrix values were performed by using the values given by Meijaard et al., 2007 and compared to the determined values for the counter torque or force feedback \(\mathbf{T}_{\mathbf{f}}\) according to Schwab and Recuero, 2013. A further factor for the counter-torque has been included, which serves as compensation for the fine-tuning between the servomotor and the rectifier.

Platform – The simulator is assembled on top of a 2-DOF motion platform (pitch and roll) that allows for leaning behaviour and additionally stimulates the vestibular sensory system during incline and acceleration. The leaning angle is measured on the base of the platform. For V1.1, the platform is controlled by a force/torque application from the cyclist (force control). In addition, the motion platform pushes the rider back to the center with a spring force, which can lead to a pendulum motion and a safety risk when the rider is involved. This setting is also extremely dependent on the height and weight of the riders. Therefore, a position control approach was designed for V2.0 in which the leaning angle is dependent on the steering angle and the driving velocity to provide more safety to the rides while counteracting the platform to build up a pendulum moment (Janssen, 2022). The basic formula for the leaning behaviour is as follows (Meijaard et al., 2007):

\(\mathbf{-}\mathbf{m}_{\mathbf{T}}{\ddot{\mathbf{y}}}_{\mathbf{P}}\mathbf{z}_{\mathbf{T}}\mathbf{+}\mathbf{I}_{\mathbf{T}_{\mathbf{XX}}}\ddot{\mathbf{\phi}}\mathbf{+}\mathbf{I}_{\mathbf{T}_{\mathbf{XZ}}}\ddot{\mathbf{\psi}}\mathbf{+}\mathbf{I}_{\mathbf{A}_{\mathbf{\lambda X}}}\ddot{\mathbf{\delta}}\mathbf{+}\dot{\mathbf{\psi}}\mathbf{v}\mathbf{S}_{\mathbf{T}}\mathbf{+}\dot{\mathbf{\delta}}\mathbf{v}\mathbf{S}_{\mathbf{F}}\mathbf{\cos}\mathbf{\lambda}\mathbf{\ =}\mathbf{T}_{\mathbf{B}_{\mathbf{\phi}}}\mathbf{- g}\mathbf{m}_{\mathbf{T}}\mathbf{z}_{\mathbf{T}}\mathbf{\phi + g}\mathbf{S}_{\mathbf{A}}\mathbf{\delta}\) \[\mathbf{with\ }\dot{\mathbf{\psi}}\mathbf{=}\left( \frac{\mathbf{v}\mathbf{\delta + c}\dot{\mathbf{\delta}}}{\mathbf{w}} \right)\mathbf{\cos}\mathbf{\lambda}\mathbf{\ \rightarrow \ }\ddot{\mathbf{\psi}}\mathbf{=}\left( \frac{\mathbf{v}\dot{\mathbf{\delta}}\mathbf{+ c}\ddot{\mathbf{\delta}}}{\mathbf{w}} \right)\mathbf{\cos}\mathbf{\lambda}\] \[\mathbf{and\ }{\ddot{\mathbf{y}}}_{\mathbf{P}}\mathbf{=}\left( \frac{\mathbf{v}^{\mathbf{2}}\mathbf{\delta +}\mathbf{vc}\dot{\mathbf{\delta}}}{\mathbf{w}} \right)\mathbf{\cos}\mathbf{\lambda}\]

(7)

The 2-DOF motion platform used did not include a sensor system to measure the applied leaning moment of the driver \(\mathbf{T}_{\mathbf{B}_{\mathbf{\phi}}}\) which would provide feedback for the calculation basis. Therefore, a different approach was developed to ensure the control of the platform. The calculation was created as a function of velocity and steering angle for the deflection acceleration \({\ddot{\mathbf{y}}}_{\mathbf{P}}\mathbf{\ }\)at the rear contact point 𝑃. Figure 3 illustrates the development of the concept (adapted from Astrom et al., 2005; Meijaard et al., 2007).

Figure 3. Concept of the leaning behaviour (adapted from Astrom et al., 2005; Meijaard et al., 2007)

For the calculation of the lean angle 𝜙, first the displacement \(\mathbf{y}_{\mathbf{P}}\) of the rear contact point 𝑃 is determined as follows:

\[\mathbf{\sin}\mathbf{\delta}\mathbf{=}\frac{\mathbf{y}_{\mathbf{P}}}{\mathbf{\mathrm{\Delta}s}}\mathbf{=}\frac{\mathbf{y}_{\mathbf{P}}}{\mathbf{v \bullet \mathrm{\Delta}t}}\mathbf{\rightarrow}\mathbf{y}_{\mathbf{P}}\mathbf{=}\mathbf{\sin}\mathbf{\delta}\mathbf{\bullet v \bullet \mathrm{\Delta}t}\]	(8)

Finally, based on (8), the lean angle 𝜙 can be determined in the next step:

\[\mathbf{\tan}\mathbf{\phi}\mathbf{=}\frac{\mathbf{y}_{\mathbf{P}}}{\mathbf{- z}_{\mathbf{B}}}\mathbf{\rightarrow \phi =}\mathbf{\arctan}\left( \frac{\mathbf{y}_{\mathbf{P}}}{\mathbf{-}\mathbf{z}_{\mathbf{B}}} \right)\]	(9)

To compensate for the displacement 𝑦𝑃, a "bleeding" factor is set in the calculation, which resets an occurrence of errors in the integral and differential calculation and thus allows the platform to move smoother. Furthermore, the inputs of the steering angle 𝛿 and the measured cycling velocity 𝑣 were parameterized to allow for later adjustment for an even finer or smoother leaning behaviour. Likewise, a range for the steering angle 𝛿 was set by a trial in order not to move the platform at the smallest angle changes. This also applies to the travel velocity 𝑣. The limits for this are ±0.05 rad or approx. ±3° and above 2.5 m/s. Additionally, the maximum lean angle 𝜙 was limited to ±15° and the velocity of the platform was reduced to prevent a pendulum motion as in the previous setup.

Research bicycle

The real-world study aimed to gather validation data with a 28” e-bike, shown in Figure 4.

Figure 4. Research bicycle instrumented with sensors

The bicycle is equipped with an Elipse-D sensor that includes a 9-DOF Inertial Measurement Unit (IMU) with GNSS receiver with which we measured the leaning angle, position and velocity of the bicycle. The IMU was placed under the seat of the bicycle and is connected to a Raspberry Pi. An incremental encoder (Bourns EMS22D) with 512 pulses per revolution was attached to the handlebar and was used to measure the steering angle. It is read out by an Arduino Mega Board via digital inputs. The Arduino counts the increments and calculates the increment velocity. Afterwards, it is sent to the Raspberry via the serial output. All values (incremental encoder and the IMU) run together and are logged in Node-RED.

Study design

Study participants were recruited via the online tool “Probandenpool” (pool of subjects) of the Institute of Transportation Systems. They received a financial compensation for their participation. The study was approved by the "Research Ethics" office at the German Aerospace Center.

Research questions and hypothesis

The combination of a simulator study and a real-world study was designed to answer the following research questions:

• RQ1: Does the mechanical structure and control of the bicycle simulator allow the feeling of riding on a real bicycle?

• RQ2: Is the data collected from the bicycle simulator similar to the data of a real bicycle?

• RQ3: Which of the simulator set-ups is better suited in terms of RQ1 and RQ2?

To answer these questions, the following hypotheses were evaluated:

• H1: The mechanical and algorithmic changes made to the simulator will deliver a more realistic steering behaviour.

• H2: The mechanical and algorithmic changes made to the simulator will deliver a more realistic leaning behaviour.

• H3: The cycling velocity choice will be more realistic with V2.0.

Simulator study design

The simulator study was performed in September 2022. Both versions of the simulator set-up were successfully tested by 27 participants (5 female, 22 male, mean age 29.4 years) in a within-subject study design. 8 participants could not finish the study due to simulator sickness or not being able to control the simulator. After a training on the simulator, 6 scenarios with different tasks (s. Table 2) were conducted. The participants answered questionnaires on simulator sickness, presence, acceptance and realism. Simulation data on driving velocity, steer angle, and lean angle was collected to evaluate the differences between the simulator set-ups (V1.1 and V2.0).

Table 2. Scenarios of the evaluation study

Scenario	Task
A	Stop at a traffic light and go around a construction site
B	Slalom
C	Double left turn at intersection
D	Interaction with pedestrian + Right turn
E	Overtaking an E-Scooter + Left turn
F	Turning area

The scenarios took place on 2 different tracks depending on the task. Most of the scenarios of the main experiment (A, C, D, E) were performed on a virtual replication of the research intersection in Braunschweig, Germany, whereas scenarios B and F were performed on road sections of the city Cremlingen (Figure 5). The tasks performed on each scenario were elected to be realistic and comparable with data from the real track.

Figure 5. Scenario A (left), B (middle) and F (right)

More details about the simulator setup are presented by (Fischer et al., 2022) and (Martínez García et al., 2022). Additional analyses of the subjective evaluation of the two simulator set-ups can be found in (Martínez García et al., 2023).

Real-world study design

The study was performed in August 2023 with 11 participants (2 female, 9 male, mean age 34 years). The main goal of the study was to collect data about the driving velocity, steer angle and lean angle to compare it with the simulator data on a between-subject design. Scenarios A, C, D and E were performed at the research intersection in Braunschweig (Figure 6, left). Although these scenarios were identical in terms of route, there were many variables concerning the traffic conditions that could not be controlled or simulated. These conditions include the state of traffic lights, other road users and their behaviours, weather conditions and sirens of ambulances or police. For scenario D there was no instructed pedestrian to cross at the intersection and in scenario E there was no instructed E-Scooter. Therefore, these scenarios consisted only on turning to the right or to the left.

Scenarios B and F were recreated in a space with no other road users (Figure 6, right). Due to space limitations, the measurements used in the simulation could not be replicated in real life. For scenario B (Slalom) the cones on the real track were placed with 4 meters of separation between each other, whereas in the simulation, the cones had a separation of 9 meters. Furthermore, on the real track there were 8 cones, whereas on the simulation there were 11. Due to asphalt space limitations the turning area in scenario F had a diameter of only 12 meters on the real track and not 20 meters as in the simulation.

Figure 6. Real world study. Image of the research intersection (left) and slalom course of scenario B (right)

Results

Even though the simulator study was carried out in a balanced order design, due to the drop outs, in the end one group with 16 test subjects experienced V1.1 first and V2.0 afterwards, whereas the second group with the reversed order of simulator set-ups contained only 11 subjects. Since some order effects in the subjective ratings could be found (s. Martinez Garcia, 2023), this limits the expressiveness of the results. However, the objective analysis does not reveal similar order effects and thus the results seem to be trustworthy nevertheless. During the real-world study some major technical problems occurred, so that only a few data sets were usable for the analysis (between 2-4 depending on the scenario). Especially the steering recordings showed a strong drift, which made quantitative conclusions impossible. Thus, the steering angle is only analysed comparing the two simulator set-ups. The validation hence focuses on the driving velocity and the leaning angle.

A multivariate analysis of variance (MANOVA) was conducted, in which the differences in means between the profiles were compared for the three dependent variables (leaning angle, steering angle and velocity) together. The data of many participants in the real-world study was not usable, so that the remaining data is not sufficient to base reliable conclusions on. Therefore, only the simulator data was included in the statistical analysis. Furtherly, the analysis only includes datapoints where the velocity is greater than 1 m/s in order to eliminate the data produced when standing waiting at a traffic light. The multivariate test results for scenario A indicate that there are significant differences among the profiles when considering all dependent variables together. The p-values are very close to zero, suggesting a highly significant effect (Wilks' Lambda = 0.1951, F(2, 46 = 63.2775, p < 0.001). Similarly, the MANOVA showed significant effects of the profile on the dependent variables for scenario D (Wilks' Lambda = 0.5786, F(3, 46) = 11.1661, p < 0.001), scenario E (Wilks' Lambda = 0.6856, F(3, 46) = 7.0308, p < 0.001) and scenario F (Wilks' Lambda 0.3498, F(3, 46) = 28.5019, p < 0.001). In order to examine how the two profiles influences the variables leaning angle, steering angle and velocity individually, pairwise comparisons were conducted.

Steering behaviour

In the simulator study, no clear differences could be found in the steering behaviour of both versions. Figure 7 shows the frequency density of the steering angle of all the participants across all scenarios combined. This information is consistent with the analysis of individual scenarios.

Figure 7. Distribution of leaning angle for the simulator study

As the data of the incremental encoder used during the real-world study cannot be trusted (see above) the data was excluded from the further analysis.

Leaning behaviour

The analysis of the leaning angle shows significant differences between V1.1 and V2.0 in all scenarios (Table 3).

Table 3 . Post-hoc pairwise comparison of V1.1 and V2.0 for leaning angle

Scenario	Mean V1.1	Mean V2.0	se	T	df	p-val
A	1.843	0.870	0.073	13.330	44.848	< 0.001
B	2.110	1.728	0.157	2.430	36.054	0.020
C	1.959	1.615	0.150	2.299	42.548	0.026
D	1.847	1.231	0.111	5.573	46.980	< 0.001
E	1.941	1.488	0.107	4.249	37.959	< 0.001
F	1.953	3.478	0.180	-8.462	46.814	< 0.001

A more controlled steering behaviour and a more even distribution of leaning angles (Figure 8, bottom) was observed with V2.0. A small offset (between 1.5° and 2°) was often observed on V1.1, which means that the participants were not precisely cycling with 0° while being in upright position (Figure 8, top). This effect was not present in V2.0.

Figure 8. Leaning angle on a slalom course. Distribution for all participants (top) and timeline of a single participant (bottom)

Comparing the simulator data to the real-world data, we find that there is greater variation in the leaning angle in the real-world condition compared to simulator runs. Figure 9 displays the distribution of the leaning angle for the slalom (B) and the turning area (F). The absolute values of the leaning angles were used, as we are interested in how much the participants were leaning rather than the direction of the leaning. For both scenarios the plots show a large number of outliers for simulator version V1.1, which may be an indication that the participants were less steady in their leaning movements. For V2.0 the leaning angle is smaller than in the real-world condition, however the variance is more similar to the real data. The deviation in the leaning angle might be related to the differences in cone distances and turning area diameter (see above), which does not allow for a real validation in this context.

Figure 9. Boxplot of leaning angles for simulator version V1.1, V2.0 and real-world study for scenario B (slalom, right) and scenario F (turning area, left)

However, to get a better impression of the distribution of the leaning angle, Figure 10 shows the leaning angle per participant over distance for the slalom. The plots support the notion that the leaning behaviour was more unstable in V1.1 condition, compared to the V2.0 condition.

Figure 10. Leaning angle over distance for simulator version V1.1, V2.0 and real-world study for slalom scenario B.
Data for all participants with mean (black, dotted line)

Figure 11 shows the leaning angle over distance for the turning area. The simulator version V2.0 produces a smoother leaning pattern, which has a greater resemblance to the real-world data than V1.1, even though that the leaning into the curve seems to happen more gradually in simulation than in the real-world trial.

Figure 11. Leaning angle over distance for simulator version V1.1, V2.0 and real-world study for turning area scenario F.
Data for all participants with mean (black, dotted line)

Cycling velocity

The cycling velocity of the participants was evaluated to solve H3. Overall, no significant differences were found between V1.1 and V2.0 regarding the velocity for any of the scenarios. Figure 12 illustrates the distribution of velocity per profile across scenarios. The velocity in the simulator was on average roughly 0.5 m/s higher than in the real-world condition. In addition, the variance in velocity is slightly smaller in the V2.0 condition than in the V1.1 condition.

Figure 12. Boxplot of velocity for simulator version V1.1, V2.0 and real-world study based on data from all scenarios

Figure 13 compares the velocity for scenario C, the double left-turn at the intersection, for both simulation set-ups and the real-world study. The velocity keeping seems to be less steady and on average the breaking behaviour shows stronger deceleration for V1.1 than for V2.0. The same effect can be observed looking at individual data sets.

Figure 13: Driving velocity over distance for simulator version V1.1, V2.0 and real-world study for scenario D. Distribution for all participants with mean (black, dotted line)

Summary, discussion and conclusion

The studies performed in this paper provide insight into the realism of the simulator in comparison to a real bicycle. The adaptations to the configuration of the simulator were presented. The analyses of two different versions of the simulator were displayed and the resemblance to real-world data gathered with a bicycle while cycling in the same scenarios was examined. The study aimed to evaluate a new control logic and hardware changes in the DLR bicycle simulator. The version V2.0 was therefore compared to the older V1.1 version of the simulator set-up.

The analysis of the subjective ratings collected within the presented study (Martinez Garcia et al., 2023) showed, that the mechanical set-up and control algorithms of the bicycle simulator to some extend support the feeling of riding a real bicycle. Aspects related to the longitudinal cycling behaviour, thus braking, accelerating and wind effects, were rated more realistic in V2.0 than in V1.1. With regard to the lateral behaviour, namely the steering and leaning, no significant differences were found in the subjective rating of the participants.

This study aimed to compare the objective simulator data to real-world data. In order to do that, participants were asked to ride similar scenarios as in simulator on a real bicycle at the research intersection in Braunschweig. Unfortunately, due to major technical problems a lot of the data was not usable, so that the comparisons can be seen as a first indication, but no solid conclusions can be drawn from the real-world data. The steering angle data from the real-world condition was not usable due to strong drifts in the data. Therefore, H1, the assumption that the mechanical and algorithmic changes made to the simulator deliver a more realistic steering behaviour, cannot be answered conclusively. Comparing the steering data of V1.1 and V2.0 revealed no significant differences between the two versions. With respect to the leaning behaviour, we found a steadier behaviour in V2.0, which more closely resembled to the leaning behaviour observed in the real-world condition. These findings support the hypothesis H2, that the changes made to simulator promote a more realistic leaning behaviour. However, the subjective ratings of the lateral behaviour in V2.0 was not significantly better than the rating for V1.1. Regarding the longitudinal behaviour we found that even though the mean velocity does not significantly differ between the simulator versions, the velocity was steadier in V2.0. Further, we observed stronger, more abrupt decelerations for V1.1 than for V2.0. The notion, that V2.0 allowed for a smoother and steadier longitudinal behaviour was supported by the subjective ratings of the participants, which showed significantly higher ratings for V2.0 compared to V1.1. Given these results, we can see a small indication that the cycling velocity choice is more realistic with V2.0 (H3). However, further research will be needed to confirm this indication.

Some limitations of this study include that too many changes were performed on the simulator in parallel and therefore, it is partly difficult to evaluate them individually and identify the differences between both versions. For this reason, more specific research will be performed on each feature which requires further fine-tuning. As the steering angle analysis of the changes performed on the simulator delivered no clear results and the measurements of the steering angle during the real study were not precise due to a drift of the sensor, a more precise analysis should be performed to evaluate this behaviour. Another important consideration is that in the simulator study, a within design was used, whereas between the real bicycle and the simulator a between design was used. The data gathered with the real bicycle was not highly dynamic since the participants had to wait long times at the red traffic lights. Also, there were a lot more road users and aspects we couldn’t control as in the simulation. Furthermore, the results of the data recorded with the real bicycle was extremely limited since the sensors didn’t work properly and many datasets had to be dismissed. The findings from these analyses will be incorporated into the further development of the simulator.

This study provided valuable insights into the behaviour of persons riding the DLR bicycle simulator (V2.0) compared to real-world cycling and in contrast to the previous simulator set-up (V1.1). Even though technical issues prevent definitive conclusions in some aspects, the findings suggest some promising improvements in the simulator's performance. Specifically, for the longitudinal behaviour version V2.0 demonstrated smoother velocity profiles and less abrupt decelerations. Additionally, indications could be found that V2.0 exhibits a more realistic leaning behaviour although subjective ratings did not show significant differences in lateral behaviour compared to V1.1. However, it is essential to acknowledge the study's limitations, including the challenges in data collection, the need for more fine-tuned studies of individual simulator features, and the differences in scenario design between the simulator and real-world experiments. Hence, this research serves as a foundation for further refinement and development of the DLR bicycle simulator. Future studies will focus on isolating specific simulator features for improvement and conducting more precise analyses, especially regarding the comparison to real-world data, e.g. by using non-public test tracks for gathering real-world data. By addressing these limitations and building upon the current findings, enhancing the simulator's ability to replicate real-world cycling experiences will be continued, ultimately contributing to safer and more effective bicycle research and training.

References

Astrom, K. J., Klein, R. E., & Lennartsson, A. (2005). Bicycle Dynamics and Control: Adapted Bicycles for Education and Research. IEEE Control Systems, 25(4), 26–47. https://doi.org/10.1109/MCS.2005.1499389

Fischer, M., Temme, G., Gröne, K., Martinez Garcia, D., Grolms, G., & Rehm, J. (2022). A VRU-simulator for the evaluation of pedestrian- and cyclist-vehicle interaction – Design criteria and implementation. Proceedings of the Driving Simulation Conference, 153–160.

Jacobi, D. (2022). Konzeptentwicklung, Umsetzung und Evaluation der Längsdynamik eines Fahrradsimulators für eine realitätsnahe, interaktive Simulation. Bachelor thesis, DIPLOMA Hochschule.

Janssen, V. (2022). Konzeptentwicklung, Umsetzung und Evaluation der Querdynamik eines Fahrradsimulators für eine realitätsnahe, interaktive Simulation. Bachelor thesis, DIPLOMA Hochschule.

Martinez Garcia, D. (2021). Construction, parameterization and evaluation of a bicycle simulator for a realistic and interactive simulation environment. Master thesis, Technische Universität Braunschweig.

Martinez Garcia, D., Gröne, K., Janssen, V., Jacobi, D., Ackermann, S., Zhao, M., Nicolay, E., Bergen, M., & Fischer, M. (2023). Technical adjustments of a bicycle simulator . Impact on simulator sickness , presence , acceptance and realism. Proceedings of the Driving Simulation Conference 2023 Europe, 2023, 201–203.

Martínez García, D., Gröne, K., Quante, L., Fischer, M., Thal, S., & Henze, R. (2022). Parameter tuning of a bicycle simulator for a realistic riding behaviour and motion perception. Proceedings of the Driving Simulation Conference 2022 Europe, Driving Simulation Association, Strasbourg, France, 1–2.

Meijaard, J. P., Papadopoulos, J. M., Ruina, A., & Schwab, A. L. (2007). Linearized Dynamics Equations for the Balance and Steer of a Bicycle: A Benchmark and Review. Proceedings of the Royal Society A: Mathematical, Physical and Engineering Sciences, 463, 1955–1982. https://doi.org/10.1098/rspa.2007.1857

O’Hern, S., Oxley, J., & Stevenson, M. (2017). Validation of a bicycle simulator for road safety research. Accident Analysis and Prevention, 100, 53–58. https://doi.org/10.1016/j.aap.2017.01.002

Schwab, A. L., & Recuero, A. M. (2013). Design and experimental validation of a haptic steering interface for the control input of a bicycle simulator. Proceedings of the ECCOMAS Thematic Conference on Multibody Dynamics 2013, 103–110.

Shoman, M. M., & Imine, H. (2021). Bicycle Simulator Improvement and Validation. IEEE Access, 9, 55063–55076. https://doi.org/10.1109/ACCESS.2021.3071214

Sinus-Studie (2021). Fahrrad-Monitor Deutschland 2021 - Ergebnisse einer repräsentativen Online-Befragung. https://www.bmdv.bund.de/SharedDocs/DE/Artikel/StV/Radverkehr/fahrradmonitor-2019.html, Nov. 2021

Wendel, K. D. (2020). Bicycle-rider control identification. Technische Universiteit Delft.