Embodied & Intuitive Flying for VR, Gaming, and TeleOperation


Flying has been a dream for mankind for mil­lenia — but flying inter­faces for VR, gaming, and tele­op­er­a­tion (e.g., drones) typ­i­cally rely on cum­ber­some double-joystick/gamepads and do not allow for intu­itive and embod­ied flying expe­ri­ences. Here, we develop low-cost embod­ied flying inter­faces that adapt leaning-based motion cueing par­a­digms thus free­ing up hands for addi­tional tasks beyond just navigating.

Flying in vir­tual real­ity (VR) using stan­dard hand­held con­trollers can be cum­ber­some and con­tribute to unwanted side effects such as motion sick­ness and dis­ori­en­ta­tion. This paper inves­ti­gates a novel hands-free flying interface—HeadJoystick, where the user moves their head sim­i­lar to a joy­stick handle toward the target direc­tion to con­trol vir­tual trans­la­tion veloc­ity. The user sits on a reg­u­lar office swivel chair and rotates it phys­i­cally to con­trol vir­tual rota­tion using 1:1 map­ping. We eval­u­ated short-term (Study 1) and extended usage effects through repeated usage (Study 2) of the HeadJoystick versus hand­held inter­faces in two within-subject stud­ies, where par­tic­i­pants flew through a sequence of increas­ingly dif­fi­cult tun­nels in the sky. Using the HeadJoystick instead of hand­held inter­faces improved both user expe­ri­ence and per­for­mance, in terms of accu­racy, pre­ci­sion, ease of learn­ing, ease of use, usabil­ity, long-term use, pres­ence, immer­sion, sen­sa­tion of self-motion, work­load, and enjoy­ment in both stud­ies. These find­ings demon­strate the ben­e­fits of using leaning-based inter­faces for VR flying and poten­tially sim­i­lar telep­res­ence appli­ca­tions such as remote flight with quad­copter drones. From a the­o­ret­i­cal per­spec­tive, we also show how leaning-based motion cueing inter­acts with full phys­i­cal rota­tion to improve user expe­ri­ence and per­for­mance com­pared to the gamepad.

YouTube Preview Image


Adhikari, Ashu, Abraham M. Hashemian, Thinh Nguyen-Vo, Ernst Kruijff, Markus von der Heyde, and Bernhard E. Riecke. 2021. “Lean to Fly: Leaning-Based Embodied Flying Can Improve Performance and User Experience in 3D Navigation.” Frontiers in Virtual Reality 2: 1–22. https://doi.org/10.3389/frvir.2021.730334.
Hashemian, A., M. Lotfaliei, A. Adhikari, E. Kruijff, and Bernhard E. Riecke. 2020. “HeadJoystick: Improving Flying in VR Using a Novel Leaning-Based Interface.” IEEE Transactions on Visualization and Computer Graphics 28 (4): 1792–1809. https://doi.org/10.1109/TVCG.2020.3025084.