Research Criteria

 In order to ensure that my practical tests link correctly to my research, I started to compile research criteria, to do this I re-read my dissertation and highlighted sections that I feel are important and can be used to test theories and hopefully prove them. I will use this to begin to build a research framework that can be used for my PhD proposal. Below are the sections highlighted from my dissertation: 

Immersion is a fundamental part of any VR experience 

“ The word ‘immersive’ is used to describe experiences that completely surround a person to make them feel part of an alternative environment” (Digital Catapult, 2020, p.1). 

They describe the 3Is of VR and list them as: Immersion, Imagination and Interaction. It is only with the combination of the 3I’s working together that VR experiences can be effective. 

immersion, which is considered to be the most important of the three elements to make a successful VR experience: if an experience isn't immersive then imagination and interaction cannot be achieved. 

“mental immersion and physical (or sensory) immersion” (Sherman and Craig, 2018, p.9). Mental immersion is associated with “having a sense of presence within an environment” (Ibid.) and being completely engaged with the environment and story that is being watched, whereas physical immersion requires the use of technology to stimulate the body's senses. 

Disorientation and lack of immersion can occur if an insubstantial amount of senses are stimulated. 

“Sound can be used to increase the verisimilitude of the experience, supply additional information, help set a mood or indicate a situation… Realistic sounds help with mental immersion, but they also provide practical information about the environment” (Sherman and Craig, 2018, p.226).

“Physical immersion is accomplished by presenting a virtual world to users based on their location and orientation and providing synthetic stimuli to one or more of their senses in response to their position and actions. 

This occurs when the viewer does not feel present in the VR experience. “presence is currently used to indicate a successful feeling ‘being there’ in a synthetic environment” (Mcmahan, 2003 p.72) 

in VR, presence is the “result of perceptual and psychological immersion” (Mcmahan, 2003 p.77) 

“quality of social interaction, realism in the environment (graphics, sound, etc.), the effect of ‘transportation,’ the degree of immersiveness… actions within the environment… social impact… and responding to the computer itself.” (Mcmahan, 2003 p72-73) 

“‘perhaps presence and narrative cancel each other out.” (Burdette, 2015, p.1) 

Nothing works in VR that works in film… It’s a medium where you can’t punish people for looking in the wrong direction and that makes it a really limited medium, and for storytelling, almost impossible. (Lavis, cited in  Robinson, 2019, p.1). 

. It is possible that haptic stories and the use of embodied memories could be one of the new techniques that assist in telling stories in VR experiences. 

This research so far indicates that some key aspects of immersion in VR experiences are the stimulation of the senses with relevant feedback, creating a sense of presence/mental immersion as well as the interaction between immersant and characters.

. While challenging the addition of narrative in VR experiences covers another point of the 3Is by adding imagination 

Some VR experiences such as ‘Baba Yaga’ (2021a) use audience participation, lighting, narrators, and characters as aids to tell the story. 

. All of these methods help to position the viewer in the experience, create a sense of presence, immersion and tell the narrative effectively. 

New methods such as active viewer participation, lighting, and interaction all ensure that narratives play out as the creator intends whilst also making the experience immersive “to establish relationships between the narrative, environment, and the viewer that creates a sense of being there.” (Burdette, 2015, p.1). In order to create a sense of being there, a creator must establish a presence and ensure that the sensory needs for physical immersion are met. As discussed above some strategies that achieve this are Proximity, additional time to orient, character interaction (Burdette, 2015), sensory stimuli, sound (Sherman and Craig, 2018) lighting and active view participation (Befores and Afters, 2021a). 

The addition of a third source of sensory stimuli in the form of haptics could help to further immerse the viewer within an experience. The inclusion of haptics within VR also allows a greater level of interaction, which we know from above is one of the 3I’s in the VR triangle (Burdea and Coiffet, 2017).

“The user should receive haptic feedback to replicate the feel of real-world interaction. 

describes touch as being “a fundamental form of non–visual perception, one that plays a crucial role in nearly all of our sensory experiences (a feature noted by Gibson 1966)” (Fulkerson, 2020, p.1). Haptics are a form of communication, similar to how sound and sight (gestures) are used to communicate. 

is used to explore the world around us, touch is the crucial sense used to understand an environment (Jewitt et al, 2022) it is how we confirm our surrounding are real (Fulkerson, 2020) it verifies the existence of the environment, as well as our presence within it (Sherman and Craig, 2018), it also provides feedback about texture, solidity and size. 

The participant can receive different types of haptic feedback, this feedback from the technology is categorised into five types: “force, vibrotactile, electrotactile, ultrasound and thermal feedback.” (Teslasuit. 2022, p.1). Haptics are integrated into our daily life, they exist in many forms of technology as a way to help their users interact with them to a greater degree. Examples of haptics are the vibrations in mobile phones and game controllers (Blenkinsopp, 2019), the touch screen of mobile phones (Paterson, 2007), as well as replicated interactions in VR (Wigmore, 2016). 

. “Other reasons people add haptics include its use as an alternative channel of communication when visual or auditory channels are busy, and the emotional component of touch.” (ibid.).

By providing the audience with a natural way to interact with the environment, you are helping the immersant to believe the virtual world is real. 

An audience goes into a VR experience knowing what they are going to see isn't real, it is the job of the creators and the technology to trick the brain into believing it is real, this is achieved through the data received from the senses, “Consciousness of our immediate surroundings necessarily depends on the data picked up by our sensory systems – vision, sound, touch, force, taste, and smell.” (ibid.). As people mostly rely on their sense of vision, this is the primary sense that is stimulated in the majority of applications (ibid.) but in VR, this can be pushed further with the addition of sound and haptics, 

Disney. (2022) Avatar Flight of Passage. [Image] Available from: <https://www.disneyworld.co.uk/attractions/animal-kingdom/avatar-flight-of-passage/> [Accessed 03 June 2022]

Haptics are hard to fool” 

“Seeing is believing, but touching is knowing.” 

The speed at which touch is processed can become a useful tool in VR experiences, as it can be used to make an audience believe what they are experiencing is real and very quickly immerse them in the VR experience.

Now that it has been established that a sense of presence, interaction, sensory stimuli, synthetic feedback, haptics, proximity, time to orient, imagination in the form of narrative and design, effective sound and visuals are required to make an immersive VR experience,

, Aardman purposefully leave fingerprint marks on the puppets to emulate when they were made with clay 

the fact that they are made and animated by hand, this remnant of touch can help invoke memories in the viewer

 the addition of the handmade that show touch like the fingerprints in Aardman's puppets could add another level of haptic experience as it is bringing the creator into the piece, 

It also brings in visual haptics, which is where “one can experience the sensation of touch through vision alone” (Brown, 2009) which means that when a viewer sees a texture or item they are able to feel it (Vasseleu, 2009) 

“that memories are imparted to the objects we touch” (Wilson, 2018). His theory on embodied memory is that objects and places we touch are charged with emotions and retain memories that can be retrieved in the future (Stehlikova, 2012). 

This theory can be linked to psychology and the understanding of haptic/tactile memory, which is a form of sensory memory related to touch, this is where a memory forms due to physical contact. “When you feel a raindrop on your skin, your haptic memory records that sensation, helping you recognize what‘s happening” 

When animating using Stop Motion techniques it requires the animator to touch the puppet thousands of times (Vasseleu 2009; Stehlikova 2012) this act of animating is essentially charging the puppet with emotions and memories that can be seen on screen 

When an audience can see that an object has been touched it can invoke the memory it holds “The materiality and tactility present in the animation… stimulates an embodied reaction and involvement between the viewer and the film” (Rocha, 2016). It is my aim to translate this embodied tactility into the VR environment.

Embodied memories and haptic/tactile touch have the potential to add a further immersive element to VR experiences, if these are implemented within the experience, not only will it stimulate the senses of the audience, but has the potential to stimulate the immersants own embodied memories. It also brings the physicality of the maker into the experience, because the embodied memory of their touch can be seen within their creation, it potentially could invoke a memory in the immersant and they then become a participant in the experience.   

The 3Is of VR: Immersion, Interaction, and imagination (Burdea and Coiffet, 2017) must be covered. 

Physical immersion is created through the use of sensory stimuli to the visual, aural and haptic senses and also supplying the necessary sensory feedback. Mental immersion is accomplished by creating a sense of presence in the environment. This is achieved by creating a realistic environment in which a user can position themselves and interact with, in a natural way. Imagination is achieved through the inclusion of narrative which makes the audience imagine elements of the story, and character interaction which allows the immersant to imagine who the character is and the environment's/experiences design, which allows the immersant to imagine the virtual world is real. 

Physical interaction is created through the inclusion of haptics which allows an immersant to touch objects, affect them, and receive sensory feedback this also helps mental immersion as it verifies the existence of the environment making it feel real. Mental interaction is achieved through making story decisions which have an effect on the outcomes, and character interaction, where virtual characters acknowledge the immersant’s presence.

This chapter has also identified that the use of embodied memories, stop motion and tactile visuals could add an additional element to VR experiences to make them more immersive. This idea will be explored further in chapter three by completing a case study which will critically analyse existing VR experience

create a sense of presence/immersion 

• include appropriate sensory stimuli and feedback 

• use/show the imagination

• have a level of interaction

• Immersion/presence  

• Includes sensory stimuli

• Interaction

• Imagination 

The experience stimulates the visual and aural senses which were identified previously as the primary sensory stimuli needed for immersion 

The sensory stimuli used make this feel more like a 360 video rather than an immersive experience. Because it does not include haptics I was unable to interact with the environment on a physical level and was purely an observer. The lack of haptics affected the degree of presence I felt in the experience as I was unable to confirm anything was real.

‘Mad God’ does have visual haptics (Brown, 2009), as the puppets have an odd texture that almost looks as though they are melting, they have no faces, crooked limbs and their movement is jerky/unsmooth. The use of stop motion and visual haptics started to have an effect on the experience, it was like I could feel the texture of the puppets even though I had no way to (Vasseleu, 2009). This made me realise that the stop motion was using embodied memories, looking at the puppets made me remember being a child and getting slimy mud on my hands, this made me no longer miss haptics as I didn't want to feel that texture again or have the ability to touch the puppets. 

The experience uses visual stimuli to engage and direct the immersant, it uses lighting and object movement to draw the viewer's attention. Sections of the story are only visible to the viewer if that is what the creator wants them to look at, 

It uses aural stimuli to direct, inform and immerse, audio is used to narrate the story, have characters interact and speak to you, used to draw your attention, and used for natural sound effects which help immerse you further by creating heightened emotions, in the same way, that music when playing a game or watching a film creates. It uses haptic stimuli to interact with the environment, characters and objects, the immersant is given tasks to perform which means they begin to interact with the virtual world and helps create a deeper feeling of presence as you are able to use touch to confirm the world is real 

For me, the use of embodied memories did add another dimension to the immersion of the ‘Mad God’ experience, this was one of the reasons that I felt discomfort, I knew the objects were real, and my brain linked what I was seeing to memory where I had experienced touching odd textures and made me want to pull away from these puppets, even though I couldn't touch them I knew the experience had I been able to wouldn't be pleasant. 

Using these case studies I have found that the use of embodied memories did add another level of immersion. This offers new possibilities for my own work, if I am able to combine the narrative, interaction and haptics present in ‘Baba Yaga’ with the visual haptics and embodied memories from Stop Motion in ‘Mad God’ then what I'm doing next will be able to create a greater sense of immersion in my VR experience

identified in this contextual review that in order to make an immersive VR experience a creator must consider the 3Is of VR: immersion, interaction and imagination

. The research found that in order to fulfil the 3Is, the creator should include: sensory stimuli and feedback to the visual, aural and haptic senses, create a realistic environment in which a user can interact and position themselves in a natural way, and include a form of narrative and have character interaction. These findings became the criteria for the case study. The contextual review also identified that embodied memories are formed in physical objects which have been charged with emotion and memories through the act of being physically touched (Stehlikova, 2012). It further identified that embodied memories are present in stop motion animation due to its tactile nature and started to form the theory of how these could be used to further enhance VR experiences. 

This dissertation found that the inclusion of embodied memory and tactile visuals present in stop motion animation in general, and in the case study ‘Mad God’, did add another level of immersion to the experience that was not present in the CG experience ‘Baba Yaga’. It found

that the use of stop motion made the ‘Mad God’ experience immersive, even though it did not utilise a number of the identified criteria, nor did it evidence the use of as many of the criteria as was shown in ‘Baba Yaga’. The use of stop motion in ‘Mad God’ triggered embodied memories in the immersant which then made the experience feel more realistic and heightened the level of immersion that was felt by them. 

This dissertation has found that embodied memories which are present in stop motion animation, can add an additional immersive element to VR experiences, the next steps that will be taken are to create a practical test VR animation. The practical application of this dissertation is to create a VR experience that aims to prove the theory and will be produced in stop motion animation for VR. Various production pipelines will be tested to find to best input method for the stop motion footage, including the use of photogrammetry, compositing and 3D scanning (See Appendix 1). As well as including embodied memory, the finished experience will use the criteria identified in the contextual review as being essential to an immersive VR experience. The finished piece will be shown to industry professionals and the public and feedback from them will be collected, this seeks to further confirm the findings in this dissertation. 

A limitation for these methods would be that a wider selection of research could have been used in order to further explore the theories and existing knowledge behind them. Another might be that technical aspects of the production could also have been taken into consideration for why some of the case study examples did not include all the criteria that had been identified as essential. These limitations will be acted upon in the next steps of the project. Research into the technical aspects of stop motion in VR will be collected during the creation of the experience and will then be used to take into consideration the technical limitations of stop motion and VR. 

. This dissertation, along with the practical testing will be the foundation for a larger investigation into the uses of stop motion animation in VR, that will further explore how they can be used together, how they can strengthen each other, their possibilities, as well as the benefits this could bring to the animation industry.