Validation of a bicycle simulator based on objective criteria

Type of the Paper: Extended Abstract

Validation of a bicycle simulator based on objective criteria

Donaji Martinez Garcia¹*, Kilian Gröne¹, Martin Fischer¹, Min Zhao¹ and Melina Bergen¹

¹German Aerospace Center - DLR; Donaji.MartinezGarcia@dlr.de, ORCID 0000-0002-0839-0627; Kilian.Groene@dlr.de, ORCID 0000-0001-9035-4440; Ma.Fischer@dlr.de, ORCID 0000-0001-8435-4321; Min.Zhao@dlr.de, ORCID 0000-0002-0485-5255 ; Melina.Bergen@dlr.de, ORCID 0009-0009-0727-0218

*corresponding author.

Name of Editor: Jason Moore

Submitted: 26/06/2023

Accepted: 27/06/2023

Published: 27/06/2023

Citation: Martinez Garcia, D., Gröne, K., Fischer, M. & Zhao, M. (2023). Validation of a bicycle simulator based on objective criteria. The Evolving Scholar - BMD 2023, 5th Edition.
This work is licensed under a Creative Commons Attribution License (CC-BY).

Introduction

The validation of simulators is very important, as it determines to which extent the results of a simulator study can be transferred to the real world. Simulator studies are valid when they provide results that can be generalized to real-world situations and the occurrence of unwanted symptoms such as simulator sickness doesn´t influence the results (Shoman und Imine 2021). The goal is to provoke a realistic riding behavior. Behavioral validity is a measure for how the participants feel and act while driving on the simulator. To evaluate behavioral validity, performance measures of a simulator can be compared with those of a real bicycle riding on-road (O'Hern et al. 2017). The aim of the study presented in this paper was to first evaluate a new control logic together with hardware changes done to the DLR bicycle simulator and identify in which areas it is possible to fine tune the simulator so that it allows for a more realistic behavior. For this purpose, two different versions of the simulator will be compared based on objective criteria first in a simulator study and then to data of an equipped research bicycle.

Method

For the simulation, a VR visualization based on real geographical data of the research intersection was created in Unreal Engine 4. Two different set-ups of the DLR bicycle simulator (Martinez Garcia et al. 2022; Fischer et al. 2022) were tested in different scenarios. For the longitudinal dynamics, the measurement of vehicle speed on V 2.0 and braking was performed with an incremental encoder that has higher resolution and lower latency than the bicycle trainer used in V 1.1. In addition, the wind simulator was configured to deliver wind dynamically (depending on cycling speed), whereas in V1.1 it was static and constantly delivered the same amount of air at the same speed. For the lateral dynamics on V1.1, the force feedback of the steering motor was calculated by a modified steering force simulation based on motorized vehicles, whereas on V 2.0 it was calculated based on the Whipple-bicycle physics model (Meijaard et al. 2007). The lean angle behavior was controlled either by the force of the body (V 1.1) or by the position of the steering angle (V 2.0). In the full paper, further details of the implementation will be provided.

The real-world study aims to gather validation data with the research bicycle BoBBi, which is equipped with a series of sensors such as a radar, lidar, steer angle sensor and an Inertial Measurement Unit (IMU). The study will take place in April at the research intersection in Braunschweig.

Experimental set-up

A within-subject study was designed in which both versions of the simulator were successfully tested by 27 participants (5 female, 22 male, mean age 29.4 years). 8 participants could not finish the study due to simulator sickness or not being able to control the simulator. After a training, 6 scenarios (4 at the intersection, a slalom and a turning head track) with different tasks were conducted. Beside questionnaires on simulator sickness and presence, simulation data on e.g. driving velocity, steer angle, and lean angle was collected in order to answer the following research questions: RQ1: Does the mechanical structure and control of the bicycle simulator allow the feeling of riding on a real bike? RQ2: Is the data collected from the bicycle simulator similar to the data of a real bicycle? To answer these questions, the following hypotheses were evaluated. H1: The mechanical and algorithmic changes will improve the steering behavior. H2: The mechanical algorithmic changes will improve the leaning behavior. H3: The cycling speed choice will be more realistic with V 2.0.

Results

For the steering behavior (H1) no clear differences between the two versions could be seen. The analysis of the leaning angle (H2) shows a more controlled behavior with V 2.0 and a more even distribution of leaning angles (s. Figure 1).

Figure 1. Leaning angle on a slalom course. Distribution for all participants (top) and a single participant (bottom)

The cycling speed of the participants was evaluated to solve H3. These analyses show that the braking behavior was influenced by the delays provoked by the bike trainer in version V 1.1, this means, that riders slowed their velocity earlier or drove slower in order to be able to brake on time. Furthermore, the speed choice behavior on V 2.0 was more continuous than on V 1.1. The subjective evaluation gave no clear result, but a distinct preference of V 2.0 for the longitudinal dynamics.

Discussion and conclusion

Some limitations of this study include that too many changes were performed on the simulator in parallel. Therefore, it is partly difficult to evaluate them individually. For this reason, more specific research will be performed on each feature which requires further fine-tuning. As the steering angle analysis gave no clear results, the steering angular velocity will be analyzed. In the full paper, the real-world statistical data obtained by utilizing the DLR research intersection and BoBBi will be analyzed and compared with the simulator data. This will provide insights in the behavioral validity of the different versions. The findings from these analyses will be incorporated into the further development of the simulator.

References

Fischer, M.; Temme, G.; Gröne, K.; Martinez Garcia, D.; Grolms, G.; Rehm, J. (2022): A VRU-simulator for the evaluation of pedestrian and cyclist-vehicle interaction – Design criteria and implementation. Proceedings of the Driving Simulation Conference 2022 Europe, Strasbourg, France, 153–159.

Martinez Garcia, D.; Gröne, K.; Quante, L.; Fischer, M.; Thal, S.; Henze, R. (2022): Parameter tuning of a bicycle simulator for a realistic riding behaviour and motion perception. In: Product Solutions Book - Driving Simulation & Virtual Reality Conference & Exhibition 2022 Europe, Strasbourg, France, 101–102.

Meijaard, J.P; Papadopoulos, Jim M.; Ruina, Andy; Schwab, A.L (2007): Linearized dynamics equations for the balance and steer of a bicycle: a benchmark and review. In: Proc. R. Soc. A. 463 (2084), 1955–1982. DOI: 10.1098/rspa.2007.1857.

O'Hern, Steve; Oxley, Jennie; Stevenson, Mark (2017): Validation of a bicycle simulator for road safety research. Accident analysis and prevention 100, 53–58. DOI: 10.1016/j.aap.2017.01.002.

Shoman, Murad M.; Imine, Hocine (2021): Bicycle Simulator Improvement and Validation. IEEE Access 9, 55063–55076. DOI: 10.1109/ACCESS.2021.3071214.