Embodied Self-Motion Illusions in VR

profile

How can we pro­vide humans with a believ­able sen­sa­tion of being in and moving through computer-generated envi­ron­ments (like VR, com­puter games, or movies) with­out the need for costly and cum­ber­some motion plat­forms or large free-space walk­ing areas? That is, how can we “cheat intel­li­gently” by pro­vid­ing a com­pelling, embod­ied self-motion illu­sion (“vec­tion”) with­out the need for full phys­i­cal motion?

In a series of exper­i­ments, we inves­ti­gated the con­tri­bu­tion and inter­ac­tion of dif­fer­ent sen­sory modal­i­ties as well as higher-level, cog­ni­tive influences.

Below is a selec­tion of such work [more will come as the web­site evolves]

Spatialized Sound Enhances Biomechanically-Induced Self-Motion Illusion (Vection)

The use of vec­tion, the illu­sion of self-movement, has recently been explored as a novel way to immerse observers in medi­ated envi­ron­ments through illu­sory yet com­pelling self-motion with­out phys­i­cally moving. This pro­vides advan­tages over exist­ing sys­tems that employ costly, cum­ber­some, and poten­tially haz­ardous motion plat­forms, which are often sur­pris­ingly inad­e­quate to pro­vide life-like motion expe­ri­ences. This study inves­ti­gates whether spa­tial­ized sound rotat­ing around the sta­tion­ary, blind­folded lis­tener can facil­i­tate bio­me­chan­i­cal vec­tion, the illu­sion of self-rotation induced by step­ping along a rotat­ing floor plate. For the first time, inte­grat­ing simple audi­tory and bio­me­chan­i­cal cues for turn­ing in place evoked con­vinc­ing cir­cu­lar vec­tion. In an audi­tory base­line con­di­tion, par­tic­i­pants expe­ri­enced only spa­tial­ized audi­tory cues. In a purely bio­me­chan­i­cal con­di­tion, seated par­tic­i­pants stepped along side­ways on a rotat­ing plate while lis­ten­ing to mono mask­ing sounds. Scores of the bi-modal con­di­tion (bin­au­ral bio­me­chan­i­cal cues) exceeded the sum of both single cue con­di­tions, which may imply super-additive or syn­er­gis­tic effects.

YouTube Preview Image

Stepping along a rotat­ing cir­cu­lar tread­mill while being blind­folded can (after a few sec­onds) create a com­pelling & embod­ied illu­sion of rotat­ing in the oppo­site direc­tion[ERROR: No URL was passed to the generic video BBCode]. Studying such “biomechnically-induced cir­cu­lar vec­tion” can help us to  better under­stand human multi-modal infor­ma­tion pro­cess­ing and even­tu­ally help us to improve motion sim­u­la­tion in Virtual Reality.

 

YouTube Preview Image

First demo of our new cir­cu­lar tread­mill in the iSpace lab. Since then the setup has had some major updates.

 

Why not build an omni-directional tread­mill so people can walk in all direc­tion in the Virtual Environment? While recent progress has been amaz­ing, such omni-directional tread­mills are still huge, loud, expen­sive, and need many safety mea­sures (see video below for an exam­ple from the Cyberwalk project). In addi­tion, it seems that walking-in-place can only reli­ably induce cir­cu­lar vec­tion, but not translational/linear vec­tion. Hence, we decided to focus on using out under­stand­ing if human multi-modal cue inte­gra­tion to “cheat intel­li­gently” and aiming for a com­pelling embod­ied illu­sion of self-motion (i.e., “per­cep­tual realism/effectiveness”) instead of aiming for biomechanical/stimulus realism.

YouTube Preview Image

Media Gallery

Related Publications

Feuereissen, D. (2008). VR: Getting the Reality Part Straight – Does Jitter and Suspension of the Human Body Increase Auditory Circular Vection? [Bachelor’s Thesis, Department of Computer Science in Media]. http://www.kyb.mpg.de/publication.html?publ=5071
Palmisano, S., Nakamura, S., Allison, R. S., & Riecke, B. E. (2020). The Stereoscopic Advantage for Vection Persists Despite Reversed Disparity. Attention, Perception, & Psychophysics, 82(4), 2098–2118. https://doi.org/10.3758/s13414-019–01886-2)
Keshavarz, B., Phillip-Muller, A. E., Hemmerich, W., Riecke, B. E., & Campos, J. J. (2018). The effect of visual motion stim­u­lus char­ac­ter­is­tics on vec­tion and visu­ally induced motion sick­ness. Displays, 58, 71–81. https://doi.org/10.1016/j.displa.2018.07.005
Palmisano, S., & Riecke, B. E. (2018). The search for instan­ta­neous vec­tion: An oscil­lat­ing visual prime reduces vec­tion onset latency. PLOS ONE, 13(5), 1–26. https://doi.org/10.1371/journal.pone.0195886
Hashemian, A. M., Kitson, A., Nquyen-Vo, T., Benko, H., Stuerzlinger, W., & Riecke, B. E. (2018). Investigating a Sparse Peripheral Display in a Head-Mounted Display for VR Locomotion. 2018 IEEE Conference on Virtual Reality and 3D User Interfaces (VR), 571–572. https://doi.org/10.1109/VR.2018.8446345
Mursic, R. A., Riecke, B. E., Apthorp, D., & Palmisano, S. (2017). The Shepard-Risset Glissando: Music that Moves You. Experimental Brain Research, 235(10), 3111–3127. https://doi.org/10.1007/s00221-017‑5033-1
Kruijff, E., Marquardt, A., Trepkowski, C., Lindemann, R., Hinkenjann, A., Maiero, J., & Riecke, B. E. (2016). On Your Feet! Enhancing Self-Motion Perception in Leaning-Based Interfaces through Multisensory Stimuli. Proceedings of ACM Symposium on Spatial User Interaction (SUI ’16), 149–158. https://doi.org/10.1145/2983310.2985759
Riecke, B. E. (2016). Using Spatialized Sound to Enhance Self-Motion Perception in Virtual Environments and Beyond: Auditory and Multi-Modal Contributions. Canadian Acoustics, 33(3), 148–149.
Keshavarz, B., Riecke, B. E., Hettinger, L. J., & Campos, J. L. (2015). Vection and visu­ally induced motion sick­ness: How are they related? Frontiers in Psychology, 6(413), 1–11. https://doi.org/10.3389/fpsyg.2015.00413
Riecke, B. E., Feuereissen, D., Rieser, J. J., & McNamara, T. P. (2015). More than a Cool Illusion? Functional Significance of Self-Motion Illusion (Circular Vection) for Perspective Switches. Frontiers in Psychology, 6(1174), 1–13. https://doi.org/10.3389/fpsyg.2015.01174
Riecke, B. E., Freiberg, J., & Grechkin, T. Y. (2015). Can Walking Motions Improve Visually Induced Rotational Self-Motion Illusions in Virtual Reality? Journal of Vision, 15(2), 1–15. https://doi.org/10.1167/15.2.3
Riecke, B. E., & Jordan, J. D. (2015). Comparing the effec­tive­ness of dif­fer­ent dis­plays in enhanc­ing illu­sions of self-movement (vec­tion). Frontiers in Psychology, 6(713). https://doi.org/10.3389/fpsyg.2015.00713
Riecke, B. E., & Schulte-Pelkum, J. (2015). An Integrative Approach to Presence and Self-Motion Perception Research. In F. Biocca, J. Freeman, W. IJsselsteijn, M. Lombard, & R. J. Schaevitz (Eds.), Immersed in Media: Telepresence Theory, Measurement and Technology (pp. 187–235). Springer. doi: 10.1007/978–3-319–10190-3_9.
Lawson, B. D., & Riecke, B. E. (2014). The Perception of Body Motion. In K. S. Hale & K. M. Stanney (Eds.), Handbook of Virtual Environments: Design, Implementation, and Applications (2nd ed., pp. 163–195). CRC Press.
Riecke, B. E., Feuereissen, D., Rieser, J. J., & McNamara, T. P. (2014). Can self-motion illu­sions (cir­cu­lar vec­tion) facil­i­tate spa­tial updat­ing? [Poster]. Spatial Cognition 2014 Conference, Bremen, Germany. http://conference.spatial-cognition.de/SC2014/
Riecke, B. E., Jordan, J. D., Prpa, M., & Feuereissen, D. (2014). Underlying Perceptual Issues in Virtual Reality Systems: Does Display Type Affect Self-Motion Perception? [Talk]. 55th Annual Meeting of the Psychonomic Society (Psychonomics), Los Angeles, USA.
Seno, T., Palmisano, S., Riecke, B. E., & Nakamura, S. (2014). Walking with­out optic flow reduces sub­se­quent vec­tion. Experimental Brain Research, 233(1), 275–281. https://doi.org/doi:10.1007/s00221-014‑4109-4
Riecke, B. E. (2013). Creating a moving expe­ri­ence  with­out moving the observer: Perceptual aspects & dis­play fac­tors in immer­sive Virtual Reality (and else­where) [Invited Talk]. International Toronto Stereoscopic Film Conference, Toronto, Canada. https://youtu.be/98hq_a_nvso
Riecke, B. E., & Schulte-Pelkum, J. (2013). Perceptual and Cognitive Factors for Self-Motion Simulation in Virtual Environments: How Can Self-Motion Illusions (“Vection”) Be Utilized? In F. Steinicke, Y. Visell, J. Campos, & A. Lécuyer (Eds.), Human Walking in Virtual Environments (pp. 27–54). Springer. doi: 10.1007/978–1-4419–8432-6_2.
Freiberg, J., & Riecke, B. E. (2013). Its your turn: Enhancing visu­ally induced self motion illu­sions (“vec­tion”) with walk­ing motions in vir­tual real­ity [Poster]. 54th Annual Meeting of the Psychonomic Society (Psychonomics), Toronto, Canada.
Riecke, B. E., & Feuereissen, D. (2012). To Move or Not to Move: Can Active Control and User-Driven Motion Cueing Enhance Self-Motion Perception (“Vection”) in Virtual Reality? ACM Symposium on Applied Perception SAP, 17–24. https://doi.org/10.1145/2338676.2338680
Riecke, B. E., Sigurdarson, S., & Milne, A. P. (2012). Moving Through Virtual Reality Without Moving? Cognitive Processing, 13(1), 293–297. https://doi.org/10.1007/s10339-012‑0491-7
Riecke, B. E. (2011). Compelling Self-Motion Through Virtual Environments Without Actual Self-Motion – Using Self-Motion Illusions (“Vection”) to Improve User Experience in VR. In J. Kim (Ed.). In J.-J. Kim (Ed.), Virtual Reality (pp. 149–176). InTechOpen. https://www.intechopen.com/books/virtual-reality/compelling-self-motion-through-virtual-environments-without-actual-self-motion-using-self-motion-ill
Riecke, B. E., Feuereissen, D., Rieser, J. J., & McNamara, T. P. (2011). Spatialized sound enhances biomechanically-induced self-motion illu­sion (vec­tion). Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems, 2799–2802. https://doi.org/10.1145/1978942.1979356
Riecke, B. E., Feuereissen, D., & Rieser, J. J. (2010). Spatialized sound influ­ences bio­me­chan­i­cal self-motion illu­sion (“vec­tion”). Proceedings of the 7th Symposium on Applied Perception in Graphics and Visualization, 158–158. https://doi.org/10.1145/1836248.1836280
Riecke, B. E., Väljamäe, A., & Schulte-Pelkum, J. (2009). Moving sounds enhance the visually-induced self-motion illu­sion (cir­cu­lar vec­tion) in vir­tual real­ity. ACM Transactions on Applied Perception (TAP), 6, 7:1–7:27. https://doi.org/10.1145/1498700.1498701
Riecke, B. E. (2009). Cognitive and higher-level con­tri­bu­tions to illu­sory self-motion per­cep­tion (“vec­tion”): does the pos­si­bil­ity of actual motion affect vec­tion? Japanese Journal of Psychonomic Science, 28(1), 135–139. http://ci.nii.ac.jp/naid/110007482465
Riecke, B. E., Feuereissen, D., & Rieser, J. J. (2009). Rotating sound fields can facil­i­tate bio­me­chan­i­cal self-motion illu­sion (“cir­cu­lar vec­tion”). Journal of Vision, 9(8), 714–714. https://doi.org/10.1167/9.8.714
Riecke, B. E., Feuereissen, D., & Rieser, J. J. (2009). Auditory self-motion sim­u­la­tion is facil­i­tated by haptic and vibra­tional cues sug­gest­ing the pos­si­bil­ity of actual motion. ACM Transactions on Applied Perception (TAP), 6, 20:1–20:22. https://doi.org/10.1145/1577755.1577763
Contribution and inter­ac­tion of audi­tory and bio­me­chan­i­cal cues for self-motion illu­sions (“cir­cu­lar vec­tion”). (2008). [Poster]. CyberWalk work­shop, Tübingen, Germany.
Riecke, B. E., Feuereissen, D., & Rieser, J. J. (2008). Auditory self-motion illu­sions (“cir­cu­lar vec­tion”) can be facil­i­tated by vibra­tions and the poten­tial for actual motion. Proceedings of the 5th Symposium on Applied Perception in Graphics and Visualization, 147–154. https://doi.org/10.1145/1394281.1394309
Riecke, B. E., Cunningham, D. W., & Bülthoff, H. H. (2007). Spatial updat­ing in vir­tual real­ity: the suf­fi­ciency of visual infor­ma­tion. Psychological Research, 71(3), 298–313. https://doi.org/10.1007/s00426-006‑0085-z
Schulte-Pelkum, J. (2007). Perception of self-motion: Vection exper­i­ments in multi-sensory Virtual Environments [PhD thesis, Ruhr-Universität Bochum]. https://hss-opus.ub.ruhr-uni-bochum.de/opus4/frontdoor/deliver/index/docId/2735/file/diss.pdf
Riecke, B. E., Schulte-Pelkum, J., Avraamides, M. N., Heyde, M. V. D., & Bülthoff, H. H. (2006). Cognitive fac­tors can influ­ence self-motion per­cep­tion (vec­tion) in vir­tual real­ity. ACM Transactions on Applied Perception (TAP), 3, 194–216. https://doi.org/10.1145/1166087.1166091
Riecke, B. E. (2006). Simple user-generated motion cueing can enhance self-motion per­cep­tion (Vection) in vir­tual real­ity. Proceedings of the ACM Symposium on Virtual Reality Software and Technology, 104–107. ACM. https://doi.org/10.1145/1180495.1180517
Riecke, B. E., & Schulte-Pelkum, J. (2006). Using the per­cep­tu­ally ori­ented approach to opti­mize spa­tial pres­ence & ego-motion sim­u­la­tion (153). MPI for Biological Cybernetics. http://www.kyb.mpg.de/publication.html?publ=4186
Riecke, B. E., Schulte-Pelkum, J., & Caniard, F. (2006). Visually induced linear vec­tion is enhanced by small phys­i­cal accel­er­a­tions [Poster]. 7th International Multisensory Research Forum (IMRF), Dublin, Ireland.
Mohler, B. J., Thompson, W. B., Riecke, B., & Bülthoff, H. H. (2005). Measuring vec­tion in a large screen vir­tual envi­ron­ment. Proceedings of the 2nd Symposium on Applied Perception in Graphics and Visualization, 103–109. https://doi.org/http://doi.acm.org/10.1145/1080402.1080421
Riecke, B. E., Schulte-Pelkum, J., Avraamides, M. N., von der Heyde, M., & Bülthoff, H. H. (2005). Scene con­sis­tency and spa­tial pres­ence increase the sen­sa­tion of self-motion in vir­tual real­ity. Proceedings of the 2nd Symposium on Applied Perception in Graphics and Visualization, 111–118. https://doi.org/10.1145/1080402.1080422
Riecke, B. E., Schulte-Pelkum, J., Caniard, F., & Bülthoff, H. H. (2005). Auditory cues can facil­i­tate the visually-induced self-motion illu­sion (cir­cu­lar vec­tion) in Virtual Reality [Poster]. 8. Tübingen Perception Conference (TWK), Max Planck Institute for Biological Cybernetics, Germany.
Riecke, B. E., Schulte-Pelkum, J., Caniard, F., & Bülthoff, H. H. (2005). Spatialized audi­tory cues enhance the visually-induced self-motion illu­sion (cir­cu­lar vec­tion) in Virtual Reality. (138). MPI for Biological Cybernetics. http://www.kyb.mpg.de/publication.html?publ=4187
Riecke, B. E., Schulte-Pelkum, J., Caniard, F., & Bülthoff, H. H. (2005). Influence of Auditory Cues on the visually-induced Self-Motion Illusion (Circular Vection) in Virtual Reality. Proceedings of 8th Annual Workshop Presence 2005, 49–57. http://en.scientificcommons.org/20596230
Riecke, B. E., Schulte-Pelkum, J., Caniard, F., & Bülthoff, H. H. (2005). Towards Lean and Elegant Self-Motion Simulation in Virtual Reality. Proceedings of the 2005 IEEE Conference 2005 on Virtual Reality, 131–138. https://doi.org/10.1109/VR.2005.83
Riecke, B. E., Västfjäll, D., Larsson, P., & Schulte-Pelkum, J. (2005). Top-Down and Multi-Modal Influences on Self-Motion Perception in Virtual Reality. Proceedings of HCI International 2005, 1–10. http://en.scientificcommons.org/20596227
Schulte-Pelkum, J., Riecke, B. E., Caniard, F., & Bülthoff, H. H. (2005). Can audi­tory cues influ­ence the visu­ally induced self-motion illu­sion? [Poster]. European Conference on Visual Perception (ECVP).
Riecke, B. E., Schulte-Pelkum, J., Avraamides, M. N., & Bülthoff, H. H. (2004). Enhancing the Visually Induced Self-Motion Illusion (Vection) under Natural Viewing Conditions in Virtual Reality. Proceedings of 7th Annual Workshop Presence 2004, 125–132. https://doi.org/10.1.1.122.5636
Riecke, B. E., Schulte-Pelkum, J., Avraamides, M. N., Heyde, M. von der, & Bülthoff, H. H. (2004). Top-down influ­ence on visu­ally induced self-motion per­cep­tion (vec­tion) [Poster]. 7. Tübingen Perception Conference (TWK), Tübingen, Germany.
Riecke, B. E., Schulte-Pelkum, J., Avraamides, M. N., von der Heyde, M., & Bülthoff, H. H. (2004). The effect of cog­ni­tion on the visu­ally induced illu­sion of self-motion (vec­tion). Journal of Vision, 4(8), 891a. https://doi.org/10.1167/4.8.891
Schulte-Pelkum, J., Riecke, B. E., & Bülthoff, H. H. (2004). Vibrational cues enhance believ­abil­ity of ego-motion sim­u­la­tion [Poster]. International Multisensory Research Forum (IMRF), Barcelona, Spain.
Schulte-Pelkum, J., Riecke, B. E., Heyde, M. von der, & Bülthoff, H. H. (2003). Screen cur­va­ture does influ­ence the per­cep­tion of visu­ally sim­u­lated ego-rotations. Journal of Vision, 3(9). https://doi.org/doi: 10.1167/3.9.411
Schulte-Pelkum, J., Riecke, B. E., von der Heyde, M., & Bülthoff, H. H. (2003). Circular vec­tion is facil­i­tated by a con­sis­tent pho­to­re­al­is­tic scene [Talk]. Presence 2003 Conference, Aalborg, Denmark.