Immersive Installation for Creative Expression and Public Performance: Transcending Perception

  Artist Statement Transcending Perception is an inter­ac­tive Virtual Reality (VR) instal­la­tion devel­oped by John Desnoyers-Stewart that allows par­tic­i­pants to col­lab­o­rate in the cre­ative, impro­vi­sa­tional pro­duc­tion of mul­ti­sen­sory expe­ri­ences. Bodies and space are trans­formed into instru­ments which trans­late pres­ence into per­for­mance. This instal­la­tion reminds par­tic­i­pants that they are cre…

Body RemiXer

Body RemiXer con­nects bodies through move­ment. It is an expe­ri­en­tial pro­jec­tion based Virtual Reality instal­la­tion that explores novel forms of embod­ied inter­ac­tion between mul­ti­ple par­tic­i­pants where their bodies mix into a shared embod­ied rep­re­sen­ta­tion pro­duc­ing a play­ful inter­ac­tion that aims to sup­port the feel­ing of con­nec­tion and self-transcendence. Body RemiXer is both an artis­tic installa…

Lucid Loop: A Virtual Deep Learning Biofeedback System for Lucid Dreaming Practice

Can VR and neu­ro­feed­back deep learn­ing art help enhance atten­tion and lucid dream­ing prac­tice? Lucid dream­ing, know­ing one is dream­ing while dream­ing, is an impor­tant tool for explor­ing con­scious­ness and bring­ing aware­ness to dif­fer­ent aspects of life. We cre­ated a system called Lucid Loop: a vir­tual real­ity expe­ri­ence where one can prac­tice lucid aware­ness via neu­ro­feed­back. Visuals are creativ…

Connecting through JeL – bio-responsive VR for interpersonal synchronization

Can a bio-responsive gen­er­a­tive art instal­la­tion foster inter­per­sonal syn­chro­niza­tion and con­nec­tion? JeL is a bio-responsive, immer­sive, inter­ac­tive, gen­er­a­tive art instal­la­tion designed to encour­age phys­i­o­log­i­cal syn­chro­niza­tion between the imm­er­sants. In this project, we will be explor­ing how novel forms of inter­ac­tion can be included in immer­sive tech­nol­ogy to foster the feel­ing of connection…

Embodied & Intuitive Flying for VR, Gaming, and TeleOperation

Flying has been a dream for mankind for mil­lenia — but flying inter­faces for VR, gaming, and tele­op­er­a­tion (e.g., drones) typ­i­cally rely on cum­ber­some double-joystick/gamepads and do not allow for intu­itive and embod­ied flying expe­ri­ences. Here, we develop low-cost embod­ied flying inter­faces that adapt leaning-based motion cueing par­a­digms thus free­ing up hands for addi­tional tasks beyond just na…

Navigation Interface Tutorial

Navigation Interfaces for Virtual Reality and Gaming: Theory and Practice First ver­sion held at IEEE VR 2017, Sunday, March 19, 1:30pm — 5:00pm, updated vari­ants of the tuto­r­ial will be pre­sented at ACM Chi 2018 (slides) and IEEE VR 2018. At The Spatial Cognition 2018 con­fer­ence we will present a new tuto­r­ial on Spatial Navigation Interfaces for Immersive Environments focus­ing more on the…

Gamified Research

Gamifying Research — Researchifying Games While tra­di­tional exper­i­men­tal par­a­digms offer tight stim­u­lus con­trol and repeata­bil­ity, then tend to be a bit boring and removed from many real-world sit­u­a­tions, which can limit real-world trans­fer­abil­ity of results. How can we bring together the method­olog­i­cal strenghs of research with the intrin­sic moti­va­tion of play­ful­ness and gaming? The …

Navigational Search in VR: Do Reference Frames Help?

Would the rec­tan­gu­lar ref­er­ence frame of a CAVE help to reduce dis­ori­en­ta­tion and improve nav­i­ga­tion per­for­mance in VR? Here, we show that simply pro­vid­ing the rec­tan­gu­lar ref­er­ence frame of a room (as a simple wire­frame cuboid), but not a CAVE improved nav­i­ga­tional search per­for­mance.   Despite recent advances in vir­tual real­ity, loco­mo­tion in a vir­tual envi­ron­ment is still restricted becau…

Lean and Elegant Motion Cueing in VR

How do we best design loco­mo­tion inter­faces for VR that pro­vide “enough” phys­i­cal motion cues (vestibular/proprioceptive) while still being effec­tive, afford­able, com­pact, and safe? Despite amaz­ing progress in com­puter graph­ics and VR dis­plays, most afford­able and room-sized VR loco­mo­tion inter­faces pro­vide only little phys­i­cal motion cues (e.g., vestibu­lar & pro­pri­o­cep­tive cues). To provide…

Pulse Breath Water

Pulse Breath Water is an immer­sive vir­tual envi­ron­ment manip­u­lated by the pulse of a participant’s breath that pro­vokes and chal­lenges the inter­ac­tion between a user and the sub­stan­tial ele­ment of the envi­ron­ment: water. The system “senses” the par­tic­i­pant, while the participant’s breath­ing feeds the system. The process is a sym­bi­otic play between inter­nal human processes [biosens­ing t…

Immersive & Embodied Teleoperation Interfaces

Developing vir­tual inter­faces for embod­ied tele-operation and loco­mo­tion. How can we best design and imple­ment an embod­ied telep­res­ence system for tele-robotics, so we can safely explore remote, hard-to-reach, or poten­tially haz­ardous areas or sit­u­a­tions? The goal of the “TeleSpider” project is to design and imple­ment a telep­res­ence system where users can remotely oper­ate a robotic spid…

state.scape: EEG-based Responsive Art Installation

Biofeedback in VR - SOLAR

Resonance in Virtual Environments: hack­ing biofeed­back for alter­ing user’s affec­tive states How can we com­bine immer­sive vir­tual envi­ron­ments (VE) with biofeed­back and gam­i­fi­ca­tion to foster relax­ation, de-stressing and med­i­ta­tive states? That is, instead of increas­ing sen­sory over­load, can we use the immer­sive and affec­tive poten­tial of VE and gam­i­fi­ca­tion to assist espe­cially novice meditato…

Motion Seats for VR

Using motion seats for enhanc­ing loco­mo­tion and immer­sion in VR How can we pro­vide a “moving expe­ri­ence” through VR with­out having to use a full-scale motion plat­form? Could a com­pact and rel­a­tively low-cost “motion seat” pro­vide some of the same ben­e­fits, thus reduc­ing cost, com­plex­ity, space & safety require­ments? Despite con­sid­er­able advances in Simulation and Virtual Real…

VR in Architecture Design & Review

How can we use immer­sive Virtual Reality and embod­ied loco­mo­tion inter­faces to to design more  cost– and space-efficient solu­tions for effec­tive pre­sen­ta­tion and com­mu­ni­ca­tion of archi­tec­tural designs and ideas?  Our over­all goal is to iter­a­tively design and eval­u­ate a novel embod­ied VR system that enables users to quickly, intu­itively, and pre­cisely posi­tion their vir­tual view­point in 3D space…

Cross-Disciplinary 'Immersion' Framework

Describing media as ‘immer­sive’ is ambigu­ous.  From debil­i­tat­ing addic­tion to ther­a­peu­tic relief, media engage­ment holds a clear dual­ity in its effect on human­ity… Without an inter­dis­ci­pli­nary char­ac­ter­i­za­tion of “immer­sion”, why do we allow this con­cept to be so read­ily invoked in dis­cus­sions of books, visual art, video games, vir­tual real­ity sys­tems and more? While “immer­sion” into tradit…

The Tabletop Makerspace

The table­top Makerspace was a Mitacs intern­ship project con­ducted in col­lab­o­ra­tion with Science World. A set of class­room tools was devel­oped to sup­port ‘Maker’ activ­i­ties at the museum. The tools included a home-built 3D printer and a set of elec­tron­ics kits for work­ing with the Arduino micro­con­troller. An intro­duc­tion to elec­tron­ics work­shop was devel­oped with local makers and Science World …

Gyroxus Gaming Chair for Motion Cueing in VR

Can self-motion per­cep­tion in vir­tual real­ity (VR) be enhanced by pro­vid­ing afford­able, user-powered min­i­mal motion cueing? Introduction & Motivation:  Can self-motion per­cep­tion in vir­tual real­ity (VR) be enhanced by pro­vid­ing afford­able, user-powered min­i­mal motion cueing? To inves­ti­gate this, we com­pared the effect of dif­fer­ent inter­ac­tion and motion par­a­digms on onset latency and intensi…

Embodied Self-Motion Illusions in VR

How can we pro­vide humans with a believ­able sen­sa­tion of being in and moving through computer-generated envi­ron­ments (like VR, com­puter games, or movies) with­out the need for costly and cum­ber­some motion plat­forms or large free-space walk­ing areas? That is, how can we “cheat intel­li­gently” by pro­vid­ing a com­pelling, embod­ied self-motion illu­sion (“vec­tion”) with­out the need for full phys­i­cal mo…

Navigational Search in VR: Do we need to walk?

Do we need full phys­i­cal motions for effec­tive nav­i­ga­tion through Virtual Environments? Recent results sug­gest that trans­la­tions might not be as impor­tant as pre­vi­ously believed, which could enable us to reduce over­all sim­u­la­tion effort and cost Physical rota­tions and trans­la­tions are the basic con­stituents of nav­i­ga­tion behav­ior, yet there is mixed evi­dence about their rel­a­tive impor­tance for co…

Spatial Cognition in VR vs. real world

Comparing spa­tial perception/cognition in real versus immer­sive vir­tual envi­ron­ments: it doesn’t com­pare! Virtual real­ity (VR) is increas­ingly used in psy­cho­log­i­cal research and appli­ca­tions – but does VR really afford nat­ural human spa­tial perception/cognition, which is a pre­req­ui­site for effec­tive spa­tial behav­ior? Using judg­ment of rel­a­tive direc­tion (JRD) tasks, Riecke & McNamara (Psychonom…

iSpaceMecha

Collaboration between the iSpace lab at SIAT and Mechatronics Undergraduate Interns to design and build a unique, vir­tual real­ity multi-modal motion sim­u­la­tor The iSpace pro­gram is cen­tered on inves­ti­gat­ing what con­sti­tutes effec­tive, robust, and intu­itive human spa­tial ori­en­ta­tion and behav­iour. This fun­da­men­tal knowl­edge will be applied to design novel, more effec­tive human-computer interfaces …

Sympathetic Guitar

Do humans response socially to abstract, expres­sive human-computer inter­faces? To inter­act with the Sympathetic Guitar is to use a famil­iar and com­fort­able Western musi­cal inter­face to feel an instant con­nec­tion to musi­cal cul­ture and style of the East.  The pro­to­type senses gui­tarists’ hand motions and per­for­mance dynam­ics to aug­ment a stan­dard clas­si­cal guitar with a dig­i­tal drone…

Spatial Updating With(out) Physical Motions?

How impor­tant are phys­i­cal motions for effec­tive spa­tial ori­en­ta­tion in VR? Most vir­tual real­ity sim­u­la­tors have a  seri­ous flaw: Users tend to get easily lost and dis­ori­ented as they nav­i­gate. According to the pre­vail­ing opin­ion, this is because phys­i­cal motion cues are absolutely required for stay­ing ori­ented while moving. In this study, we inves­ti­gated how phys­i­cal motion cues contribute …

Sonic Cradle

Sonic Cradle sus­pends the body is a com­pletely dark cham­ber which encour­ages expe­ri­ences com­pa­ra­ble to mind­ful­ness med­i­ta­tion.  Users com­pose peace­ful sound­scapes in real-time using only their breath­ing. Introduction and demo of the Sonic Cradle Sonic Cradle is a relax­ing human-computer inter­ac­tion par­a­digm designed to foster med­i­ta­tive atten­tional pat­terns.  The cur­rent p…

Immersive Installation for Creative Expression and Public Performance: Transcending Perception

  Artist Statement Transcending Perception is an interactive Virtual Reality (VR) installation developed by John Desnoyers-Stewart that allows participants to collaborate in the creative, improvisational production of multisensory experiences. Bodies and space are transformed into instruments which translate presence into performance. This installation reminds participants that they are cre...

Body RemiXer

Body RemiXer connects bodies through movement. It is an experiential projection based Virtual Reality installation that explores novel forms of embodied interaction between multiple participants where their bodies mix into a shared embodied representation producing a playful interaction that aims to support the feeling of connection and self-transcendence. Body RemiXer is both an artistic installa...

Lucid Loop: A Virtual Deep Learning Biofeedback System for Lucid Dreaming Practice

Can VR and neurofeedback deep learning art help enhance attention and lucid dreaming practice? Lucid dreaming, knowing one is dreaming while dreaming, is an important tool for exploring consciousness and bringing awareness to different aspects of life. We created a system called Lucid Loop: a virtual reality experience where one can practice lucid awareness via neurofeedback. Visuals are creativ...

Connecting through JeL – bio-responsive VR for interpersonal synchronization

Can a bio-responsive generative art installation foster interpersonal synchronization and connection? JeL is a bio-responsive, immersive, interactive, generative art installation designed to encourage physiological synchronization between the immersants. In this project, we will be exploring how novel forms of interaction can be included in immersive technology to foster the feeling of connection...

Embodied & Intuitive Flying for VR, Gaming, and TeleOperation

Flying has been a dream for mankind for millenia - but flying interfaces for VR, gaming, and teleoperation (e.g., drones) typically rely on cumbersome double-joystick/gamepads and do not allow for intuitive and embodied flying experiences. Here, we develop low-cost embodied flying interfaces that adapt leaning-based motion cueing paradigms thus freeing up hands for additional tasks beyond just na...

Navigation Interface Tutorial

Navigation Interfaces for Virtual Reality and Gaming: Theory and Practice First version held at IEEE VR 2017, Sunday, March 19, 1:30pm - 5:00pm, updated variants of the tutorial will be presented at ACM Chi 2018 (slides) and IEEE VR 2018. At The Spatial Cognition 2018 conference we will present a new tutorial on Spatial Navigation Interfaces for Immersive Environments focusing more on the...

Gamified Research

Gamifying Research - Researchifying Games While traditional experimental paradigms offer tight stimulus control and repeatability, then tend to be a bit boring and removed from many real-world situations, which can limit real-world transferability of results. How can we bring together the methodological strenghs of research with the intrinsic motivation of playfulness and gaming? The ...

Navigational Search in VR: Do Reference Frames Help?

Would the rectangular reference frame of a CAVE help to reduce disorientation and improve navigation performance in VR? Here, we show that simply providing the rectangular reference frame of a room (as a simple wireframe cuboid), but not a CAVE improved navigational search performance.   Despite recent advances in virtual reality, locomotion in a virtual environment is still restricted becau...

Lean and Elegant Motion Cueing in VR

How do we best design locomotion interfaces for VR that provide "enough" physical motion cues (vestibular/proprioceptive) while still being effective, affordable, compact, and safe? Despite amazing progress in computer graphics and VR displays, most affordable and room-sized VR locomotion interfaces provide only little physical motion cues (e.g., vestibular & proprioceptive cues). To provide...

Pulse Breath Water

Pulse Breath Water is an immersive virtual environment manipulated by the pulse of a participant’s breath that provokes and challenges the interaction between a user and the substantial element of the environment: water. The system “senses” the participant, while the participant’s breathing feeds the system. The process is a symbiotic play between internal human processes [biosensing t...

Immersive & Embodied Teleoperation Interfaces

Developing virtual interfaces for embodied tele-operation and locomotion. How can we best design and implement an embodied telepresence system for tele-robotics, so we can safely explore remote, hard-to-reach, or potentially hazardous areas or situations? The goal of the "TeleSpider" project is to design and implement a telepresence system where users can remotely operate a robotic spid...

state.scape: EEG-based Responsive Art Installation

State.scape: Using EEG-based brain-computer interfaces for a responsive art installation State.scape is an interactive installation in which audio-visuals are generated from users affective states (engagement, excitement, and meditation). The installation relies on a brain-computer interface based virtual environment and sonification, which both served as a platform for the exploration of users�...

Biofeedback in VR - SOLAR

Resonance in Virtual Environments: hacking biofeedback for altering user's affective states How can we combine immersive virtual environments (VE) with biofeedback and gamification to foster relaxation, de-stressing and meditative states? That is, instead of increasing sensory overload, can we use the immersive and affective potential of VE and gamification to assist especially novice meditato...

Motion Seats for VR

Using motion seats for enhancing locomotion and immersion in VR How can we provide a "moving experience" through VR without having to use a full-scale motion platform? Could a compact and relatively low-cost "motion seat" provide some of the same benefits, thus reducing cost, complexity, space & safety requirements? Despite considerable advances in Simulation and Virtual Real...

VR in Architecture Design & Review

How can we use immersive Virtual Reality and embodied locomotion interfaces to to design more  cost- and space-efficient solutions for effective presentation and communication of architectural designs and ideas?  Our overall goal is to iteratively design and evaluate a novel embodied VR system that enables users to quickly, intuitively, and precisely position their virtual viewpoint in 3D space...

Cross-Disciplinary 'Immersion' Framework

Describing media as 'immersive' is ambiguous.  From debilitating addiction to therapeutic relief, media engagement holds a clear duality in its effect on humanity... Without an interdisciplinary characterization of "immersion", why do we allow this concept to be so readily invoked in discussions of books, visual art, video games, virtual reality systems and more? While "immersion" into tradit...

The Tabletop Makerspace

The tabletop Makerspace was a Mitacs internship project conducted in collaboration with Science World. A set of classroom tools was developed to support ‘Maker’ activities at the museum. The tools included a home-built 3D printer and a set of electronics kits for working with the Arduino microcontroller. An introduction to electronics workshop was developed with local makers and Science World ...

Gyroxus Gaming Chair for Motion Cueing in VR

Can self-motion perception in virtual reality (VR) be enhanced by providing affordable, user-powered minimal motion cueing? Introduction & Motivation:  Can self-motion perception in virtual reality (VR) be enhanced by providing affordable, user-powered minimal motion cueing? To investigate this, we compared the effect of different interaction and motion paradigms on onset latency and intensi...

Embodied Self-Motion Illusions in VR

How can we provide humans with a believable sensation of being in and moving through computer-generated environments (like VR, computer games, or movies) without the need for costly and cumbersome motion platforms or large free-space walking areas? That is, how can we "cheat intelligently" by providing a compelling, embodied self-motion illusion ("vection") without the need for full physical mo...

Navigational Search in VR: Do we need to walk?

Do we need full physical motions for effective navigation through Virtual Environments? Recent results suggest that translations might not be as important as previously believed, which could enable us to reduce overall simulation effort and cost Physical rotations and translations are the basic constituents of navigation behavior, yet there is mixed evidence about their relative importance for co...

Spatial Cognition in VR vs. real world

Comparing spatial perception/cognition in real versus immersive virtual environments: it doesn't compare! Virtual reality (VR) is increasingly used in psychological research and applications – but does VR really afford natural human spatial perception/cognition, which is a prerequisite for effective spatial behavior? Using judgment of relative direction (JRD) tasks, Riecke & McNamara (Psychonom...

iSpaceMecha

Collaboration between the iSpace lab at SIAT and Mechatronics Undergraduate Interns to design and build a unique, virtual reality multi-modal motion simulator The iSpace program is centered on investigating what constitutes effective, robust, and intuitive human spatial orientation and behaviour. This fundamental knowledge will be applied to design novel, more effective human-computer interfaces ...

Sympathetic Guitar

Do humans response socially to abstract, expressive human-computer interfaces? To interact with the Sympathetic Guitar is to use a familiar and comfortable Western musical interface to feel an instant connection to musical culture and style of the East.  The prototype senses guitarists' hand motions and performance dynamics to augment a standard classical guitar with a digital drone...

Spatial Updating With(out) Physical Motions?

How important are physical motions for effective spatial orientation in VR? Most virtual reality simulators have a  serious flaw: Users tend to get easily lost and disoriented as they navigate. According to the prevailing opinion, this is because physical motion cues are absolutely required for staying oriented while moving. In this study, we investigated how physical motion cues contribute ...

Sonic Cradle

Sonic Cradle suspends the body is a completely dark chamber which encourages experiences comparable to mindfulness meditation.  Users compose peaceful soundscapes in real-time using only their breathing. [vimeo 35764652] Introduction and demo of the Sonic Cradle Sonic Cradle is a relaxing human-computer interaction paradigm designed to foster meditative attentional patterns.  The current p...