Embodied Self-Motion Illusions in VR

profile

How can we pro­vide humans with a believ­able sen­sa­tion of being in and moving through computer-generated envi­ron­ments (like VR, com­puter games, or movies) with­out the need for costly and cum­ber­some motion plat­forms or large free-space walk­ing areas? That is, how can we “cheat intel­li­gently” by pro­vid­ing a com­pelling, embod­ied self-motion illu­sion (“vec­tion”) with­out the need for full phys­i­cal motion?

In a series of exper­i­ments, we inves­ti­gated the con­tri­bu­tion and inter­ac­tion of dif­fer­ent sen­sory modal­i­ties as well as higher-level, cog­ni­tive influences.

Below is a selec­tion of such work [more will come as the web­site evolves]

Spatialized Sound Enhances Biomechanically-Induced Self-Motion Illusion (Vection)

The use of vec­tion, the illu­sion of self-movement, has recently been explored as a novel way to immerse observers in medi­ated envi­ron­ments through illu­sory yet com­pelling self-motion with­out phys­i­cally moving. This pro­vides advan­tages over exist­ing sys­tems that employ costly, cum­ber­some, and poten­tially haz­ardous motion plat­forms, which are often sur­pris­ingly inad­e­quate to pro­vide life-like motion expe­ri­ences. This study inves­ti­gates whether spa­tial­ized sound rotat­ing around the sta­tion­ary, blind­folded lis­tener can facil­i­tate bio­me­chan­i­cal vec­tion, the illu­sion of self-rotation induced by step­ping along a rotat­ing floor plate. For the first time, inte­grat­ing simple audi­tory and bio­me­chan­i­cal cues for turn­ing in place evoked con­vinc­ing cir­cu­lar vec­tion. In an audi­tory base­line con­di­tion, par­tic­i­pants expe­ri­enced only spa­tial­ized audi­tory cues. In a purely bio­me­chan­i­cal con­di­tion, seated par­tic­i­pants stepped along side­ways on a rotat­ing plate while lis­ten­ing to mono mask­ing sounds. Scores of the bi-modal con­di­tion (bin­au­ral bio­me­chan­i­cal cues) exceeded the sum of both single cue con­di­tions, which may imply super-additive or syn­er­gis­tic effects.

Stepping along a rotat­ing cir­cu­lar tread­mill while being blind­folded can (after a few sec­onds) create a com­pelling & embod­ied illu­sion of rotat­ing in the oppo­site direc­tion[ERROR: No URL was passed to the generic video BBCode]. Studying such “biomechnically-induced cir­cu­lar vec­tion” can help us to  better under­stand human multi-modal infor­ma­tion pro­cess­ing and even­tu­ally help us to improve motion sim­u­la­tion in Virtual Reality.

 

First demo of our new cir­cu­lar tread­mill in the iSpace lab. Since then the setup has had some major updates.

 

Why not build an omni-directional tread­mill so people can walk in all direc­tion in the Virtual Environment? While recent progress has been amaz­ing, such omni-directional tread­mills are still huge, loud, expen­sive, and need many safety mea­sures (see video below for an exam­ple from the Cyberwalk project). In addi­tion, it seems that walking-in-place can only reli­ably induce cir­cu­lar vec­tion, but not translational/linear vec­tion. Hence, we decided to focus on using out under­stand­ing if human multi-modal cue inte­gra­tion to “cheat intel­li­gently” and aiming for a com­pelling embod­ied illu­sion of self-motion (i.e., “per­cep­tual realism/effectiveness”) instead of aiming for biomechanical/stimulus realism.

Media Gallery

Related Publications

Feuereissen, D. (2008, February). VR: Getting the Reality Part Straight – Does Jitter and Suspension of the Human Body Increase Auditory Circular Vection? (Bachelor’s Thesis). Department of Computer Science in Media. Retrieved from http://www.kyb.mpg.de/publication.html?publ=5071
Mursic, R., Riecke, B. E., Apthorp, D., & Palmisano, S. (2017). The Shepard-Risset Glissando: Music that Moves You. Experimental Brain Research (accepted), (pub­lished online). https://doi.org/10.1007/s00221-017‑5033-1
Kruijff, E., Marquardt, A., Trepkowski, C., Lindemann, R., Hinkenjann, A., Maiero, J., & Riecke, B. E. (2016). On Your Feet! Enhancing Self-Motion Perception in Leaning-Based Interfaces through Multisensory Stimuli (pp. 149–158). Presented at the ACM Symposium on Spatial User Interaction (SUI ’16), Tokyo, Japan: ACM. https://doi.org/10.1145/2983310.2985759
Riecke, B. E. (2016). Using Spatialized Sound to Enhance Self-Motion Perception in Virtual Environments and Beyond: Auditory and Multi-Modal Contributions. Canadian Acoustics, 33(3), 148–149.
Keshavarz, B., Riecke, B. E., Hettinger, L. J., & Campos, J. L. (2015). Vection and visu­ally induced motion sick­ness: How are they related? Frontiers in Psychology, 6(413). https://doi.org/10.3389/fpsyg.2015.00413
Riecke, B. E., Feuereissen, D., Rieser, J. J., & McNamara, T. P. (2015). More than a Cool Illusion? Functional Significance of Self-Motion Illusion (Circular Vection) for Perspective Switches. Frontiers in Psychology, 6(1174). https://doi.org/10.3389/fpsyg.2015.01174
Riecke, B. E., Freiberg, J., & Grechkin, T. Y. (2015). Can Walking Motions Improve Visually Induced Rotational Self-Motion Illusions in Virtual Reality? Journal of Vision, 15(2), 1–15. https://doi.org/10.1167/15.2.3
Riecke, B. E., & Jordan, J. D. (2015). Comparing the effec­tive­ness of dif­fer­ent dis­plays in enhanc­ing illu­sions of self-movement (vec­tion). Frontiers in Psychology, 6(713). https://doi.org/10.3389/fpsyg.2015.00713
Riecke, B. E., & Schulte-Pelkum, J. (2015). An Integrative Approach to Presence and Self-Motion Perception Research. In F. Biocca, J. Freeman, W. IJsselsteijn, M. Lombard, & R. J. Schaevitz (Eds.), Immersed in Media: Telepresence Theory, Measurement and Technology (pp. 187–235). Springer. doi: 10.1007/978–3-319–10190-3_9.
Lawson, B. D., & Riecke, B. E. (2014). The Perception of Body Motion. In K. S. Hale & K. M. Stanney (Eds.), Handbook of Virtual Environments: Design, Implementation, and Applications (2nd ed., pp. 163–195). CRC Press.
Riecke, B. E., Feuereissen, D., Rieser, J. J., & McNamara, T. P. (2014). Can self-motion illu­sions (cir­cu­lar vec­tion) facil­i­tate spa­tial updat­ing? Poster pre­sented at the Spatial Cognition 2014 Conference, Bremen, Germany. Retrieved from http://conference.spatial-cognition.de/SC2014/
Riecke, B. E., Jordan, J. D., Prpa, M., & Feuereissen, D. (2014). Underlying Perceptual Issues in Virtual Reality Systems: Does Display Type Affect Self-Motion Perception? Talk pre­sented at the 55th Annual Meeting of the Psychonomic Society (Psychonomics), Los Angeles, USA.
Seno, T., Palmisano, S., Riecke, B. E., & Nakamura, S. (2014). Walking with­out optic flow reduces sub­se­quent vec­tion. Experimental Brain Research, 233(1), 275–281. https://doi.org/doi:10.1007/s00221-014‑4109-4
Riecke, B. E. (2013, September). Creating a moving expe­ri­ence  with­out moving the observer: Perceptual aspects & dis­play fac­tors in immer­sive Virtual Reality (and else­where). Invited Talk pre­sented at the International Toronto Stereoscopic Film Conference, Toronto, Canada.
Riecke, B. E., & Schulte-Pelkum, J. (2013). Perceptual and Cognitive Factors for Self-Motion Simulation in Virtual Environments: How Can Self-Motion Illusions (“Vection”) Be Utilized? In F. Steinicke, Y. Visell, J. Campos, & A. Lécuyer (Eds.), Human Walking in Virtual Environments (pp. 27–54). New York: Springer. doi: 10.1007/978–1-4419–8432-6_2.
Freiberg, J., & Riecke, B. E. (2013). Its your turn: Enhancing visu­ally induced self motion illu­sions (“vec­tion”) with walk­ing motions in vir­tual real­ity. Poster pre­sented at the 54th Annual Meeting of the Psychonomic Society (Psychonomics), Toronto, Canada.
Riecke, B. E., & Feuereissen, D. (2012). To Move or Not to Move: Can Active Control and User-Driven Motion Cueing Enhance Self-Motion Perception (“Vection”) in Virtual Reality? In ACM Symposium on Applied Perception SAP (pp. 17–24). Los Angeles, USA: ACM. https://doi.org/10.1145/2338676.2338680
Riecke, B. E., Sigurdarson, S., & Milne, A. P. (2012). Moving Through Virtual Reality Without Moving? Cognitive Processing, 13(1), 293–297. https://doi.org/10.1007/s10339-012‑0491-7
Riecke, B. E. (2011). Compelling Self-Motion Through Virtual Environments Without Actual Self-Motion – Using Self-Motion Illusions (“Vection”) to Improve User Experience in VR. In J. Kim (Ed.). In J.-J. Kim (Ed.), Virtual Reality (pp. 149–176). InTech. Retrieved from http://www.intechopen.com/articles/show/title/compelling-self-motion-through-virtual-environments-without-actual-self-motion-using-self-motion-ill
Riecke, B. E., Feuereissen, D., Rieser, J. J., & McNamara, T. P. (2011). Spatialized sound enhances biomechanically-induced self-motion illu­sion (vec­tion). In Proceedings of the 2011 annual con­fer­ence on Human fac­tors in com­put­ing sys­tems (pp. 2799–2802). Vancouver, Canada. https://doi.org/10.1145/1978942.1979356
Riecke, B. E., Feuereissen, D., & Rieser, J. J. (2010). Spatialized sound influ­ences bio­me­chan­i­cal self-motion illu­sion (“vec­tion”). In Proceedings of the 7th Symposium on Applied Perception in Graphics and Visualization (pp. 158–158). https://doi.org/10.1145/1836248.1836280
Riecke, B. E., Väljamäe, A., & Schulte-Pelkum, J. (2009). Moving sounds enhance the visually-induced self-motion illu­sion (cir­cu­lar vec­tion) in vir­tual real­ity. ACM Transactions on Applied Perception (TAP), 6, 7:1–7:27. https://doi.org/10.1145/1498700.1498701
Riecke, B. E. (2009). Cognitive and higher-level con­tri­bu­tions to illu­sory self-motion per­cep­tion (“vec­tion”): does the pos­si­bil­ity of actual motion affect vec­tion? Japanese Journal of Psychonomic Science, 28(1), 135–139. Retrieved from http://ci.nii.ac.jp/naid/110007482465
Riecke, B. E., Feuereissen, D., & Rieser, J. J. (2009). Rotating sound fields can facil­i­tate bio­me­chan­i­cal self-motion illu­sion (“cir­cu­lar vec­tion”). Journal of Vision, 9(8), 714–714. https://doi.org/10.1167/9.8.714
Riecke, B. E., Feuereissen, D., & Rieser, J. J. (2009). Auditory self-motion sim­u­la­tion is facil­i­tated by haptic and vibra­tional cues sug­gest­ing the pos­si­bil­ity of actual motion. ACM Transactions on Applied Perception (TAP), 6, 20:1–20:22. https://doi.org/10.1145/1577755.1577763
Contribution and inter­ac­tion of audi­tory and bio­me­chan­i­cal cues for self-motion illu­sions (“cir­cu­lar vec­tion”). (2008). Poster pre­sented at the CyberWalk work­shop, Tübingen, Germany.
Riecke, B. E., Feuereissen, D., & Rieser, J. J. (2008). Auditory self-motion illu­sions (“cir­cu­lar vec­tion”) can be facil­i­tated by vibra­tions and the poten­tial for actual motion. In Proceedings of the 5th sym­po­sium on Applied per­cep­tion in graph­ics and visu­al­iza­tion (pp. 147–154). https://doi.org/10.1145/1394281.1394309
Riecke, B. E., Cunningham, D. W., & Bülthoff, H. H. (2007). Spatial updat­ing in vir­tual real­ity: the suf­fi­ciency of visual infor­ma­tion. Psychological Research, 71(3), 298–313. https://doi.org/10.1007/s00426-006‑0085-z
Schulte-Pelkum, J. (2007). Perception of self-motion: Vection exper­i­ments in multi-sensory Virtual Environments (PhD thesis). Ruhr-Universität Bochum.
Riecke, B. E., Schulte-Pelkum, J., Avraamides, M. N., Heyde, M. V. D., & Bülthoff, H. H. (2006). Cognitive fac­tors can influ­ence self-motion per­cep­tion (vec­tion) in vir­tual real­ity. ACM Transactions on Applied Perception (TAP), 3, 194–216. https://doi.org/10.1145/1166087.1166091
Riecke, B. E. (2006). Simple user-generated motion cueing can enhance self-motion per­cep­tion (Vection) in vir­tual real­ity. In Proceedings of the ACM sym­po­sium on Virtual real­ity soft­ware and tech­nol­ogy (pp. 104–107). Limassol, Cyprus: ACM. https://doi.org/10.1145/1180495.1180517
Riecke, B. E., & Schulte-Pelkum, J. (2006). Using the per­cep­tu­ally ori­ented approach to opti­mize spa­tial pres­ence & ego-motion sim­u­la­tion (No. 153). MPI for Biological Cybernetics. Retrieved from http://www.kyb.mpg.de/publication.html?publ=4186
Riecke, B. E., Schulte-Pelkum, J., & Caniard, F. (2006). Visually induced linear vec­tion is enhanced by small phys­i­cal accel­er­a­tions. Poster pre­sented at the 7th International Multisensory Research Forum (IMRF), Dublin, Ireland.
Mohler, B. J., Thompson, W. B., Riecke, B., & Bülthoff, H. H. (2005). Measuring vec­tion in a large screen vir­tual envi­ron­ment. In Proceedings of the 2nd sym­po­sium on Applied per­cep­tion in graph­ics and visu­al­iza­tion (pp. 103–109). https://doi.org/http://doi.acm.org/10.1145/1080402.1080421
Riecke, B. E., Schulte-Pelkum, J., Avraamides, M. N., von der Heyde, M., & Bülthoff, H. H. (2005). Scene con­sis­tency and spa­tial pres­ence increase the sen­sa­tion of self-motion in vir­tual real­ity. In Proceedings of the 2nd sym­po­sium on Applied per­cep­tion in graph­ics and visu­al­iza­tion (pp. 111–118). https://doi.org/10.1145/1080402.1080422
Riecke, B. E., Schulte-Pelkum, J., Caniard, F., & Bülthoff, H. H. (2005). Auditory cues can facil­i­tate the visually-induced self-motion illu­sion (cir­cu­lar vec­tion) in Virtual Reality. Poster pre­sented at the 8. Tübingen Perception Conference (TWK), Max Planck Institute for Biological Cybernetics, Germany.
Riecke, B. E., Schulte-Pelkum, J., Caniard, F., & Bülthoff, H. H. (2005). Influence of Auditory Cues on the visually-induced Self-Motion Illusion (Circular Vection) in Virtual Reality. In Proceedings of 8th Annual Workshop Presence 2005 (pp. 49–57). Retrieved from http://en.scientificcommons.org/20596230
Riecke, B. E., Schulte-Pelkum, J., Caniard, F., & Bülthoff, H. H. (2005). Towards Lean and Elegant Self-Motion Simulation in Virtual Reality. In Proceedings of the 2005 IEEE Conference 2005 on Virtual Reality (pp. 131–138). https://doi.org/10.1109/VR.2005.83
Riecke, B. E., Schulte-Pelkum, J., Caniard, F., & Bülthoff, H. H. (2005). Spatialized audi­tory cues enhance the visually-induced self-motion illu­sion (cir­cu­lar vec­tion) in Virtual Reality. (No. 138). MPI for Biological Cybernetics. Retrieved from http://www.kyb.mpg.de/publication.html?publ=4187
Riecke, B. E., Västfjäll, D., Larsson, P., & Schulte-Pelkum, J. (2005). Top-Down and Multi-Modal Influences on Self-Motion Perception in Virtual Reality. In Proceedings of HCI inter­na­tional 2005 (pp. 1–10). Las Vegas, NV, USA. Retrieved from http://en.scientificcommons.org/20596227
Schulte-Pelkum, J., Riecke, B. E., Caniard, F., & Bülthoff, H. H. (2005). Can audi­tory cues influ­ence the visu­ally induced self-motion illu­sion? Poster pre­sented at the European Conference on Visual Perception (ECVP).
Riecke, B. E., Schulte-Pelkum, J., Avraamides, M. N., & Bülthoff, H. H. (2004). Enhancing the Visually Induced Self-Motion Illusion (Vection) under Natural Viewing Conditions in Virtual Reality. In Proceedings of 7th Annual Workshop Presence 2004 (pp. 125–132). https://doi.org/10.1.1.122.5636
Riecke, B. E., Schulte-Pelkum, J., Avraamides, M. N., Heyde, M. von der, & Bülthoff, H. H. (2004). Top-down influ­ence on visu­ally induced self-motion per­cep­tion (vec­tion). Poster pre­sented at the 7. Tübingen Perception Conference (TWK), Tübingen, Germany.
Riecke, B. E., Schulte-Pelkum, J., Avraamides, M. N., von der Heyde, M., & Bülthoff, H. H. (2004). The effect of cog­ni­tion on the visu­ally induced illu­sion of self-motion (vec­tion). Journal of Vision, 4(8), 891a. https://doi.org/10.1167/4.8.891
Schulte-Pelkum, J., Riecke, B. E., & Bülthoff, H. H. (2004). Vibrational cues enhance believ­abil­ity of ego-motion sim­u­la­tion. Poster pre­sented at the International Multisensory Research Forum (IMRF), Barcelona, Spain.
Schulte-Pelkum, J., Riecke, B. E., Heyde, M. von der, & Bülthoff, H. H. (2003). Screen cur­va­ture does influ­ence the per­cep­tion of visu­ally sim­u­lated ego-rotations. Journal of Vision, 3(9). https://doi.org/doi: 10.1167/3.9.411
Schulte-Pelkum, J., Riecke, B. E., von der Heyde, M., & Bülthoff, H. H. (2003). Circular vec­tion is facil­i­tated by a con­sis­tent pho­to­re­al­is­tic scene. Talk pre­sented at the Presence 2003 Conference, Aalborg, Denmark.