This project has been a journey and getting to the final stage that I am at now has been a process within itself. At the start of this unit my idea was driven by excitement and personal interest. As I have started to progress along with this unit the idea began to develop and as I started to research and learn all that would be required to create it my project became something unmanageable for myself, it became overly abstract, and an impossibly ginormous mountain that I could not climb over, I began to become so overwhelmed by the project that I did not even know where to begin. It has taken a lot of work refining this idea down and developing an achievable but still challenging outcome. I now have a very different project to my initial proposal but it is one that I am very excited to develop and one that I finally feel like I have a neat, organised and refined project proposal to go forward with.
As I end the 701 unit my proposed idea is to create two stop motion puppets and conduct photogrammetry tests using them, I will then test input methods that I can use to determine the best method of getting stop motion footage into a 360 or virtual reality environment.
Due to the time the test animation cannot be a full story, I had to think of a way to integrate the characters together quickly but still give a bit of background. I decided that opening the animation with a text prelog was best and would then go into the animated sequence.
Opening with a text prelog:
After Selina had to run from guards, she dropped her precious log book that contains a record of all her and Jack's adventures. She is retracing her steps looking for the log around the bridge. Just as she spots the log she has to hide behind a tree as a Troll appears, the troll picks up the log and looks in, when he hears the noise of approaching guards he puts the log in his pocket and enters a hidden door underneath the bridge just as the guards pass over. Selina watches from her hiding spot, once the guards have passed, her and Jack start to think of a plan to get the log back from the troll.
Behind the hidden door there is a home that Rowen has been hiding in for years, it is safe here as the magic door cannot be seen by humans, only magical creatures are able to see it and Rowen has nothing to fear from them, as trolls are strong, fearless warriors, only the elves were equal to them but the elves relied too heavily on the ancient royal family and the their mystical powers that protected all, since the family was lost the elves have long since been driven from the land, no Rowan had nothing fear about someone finding him here… or so he thought.
Animation:
The scenario will play from here and is Selina and Jack breaking into the Rowans home after nightfall in order to retrieve her log book. She must sneak around so as not to wake up Rowen, who has fallen asleep reading the book. As she reaches for the book she notices a necklace that has slipped out from Rowens top, she has an irresistible urge to touch the stone pendant and reaches for it, Rowen wakes up and a fight breaks out. They both hear guards over foot and while they can't see the magical door they would be able to hear the noise of the fight. Hearing them, Rowen signals Salina to be quiet, they both wait as the guards pass overhead.
Continuation:
This would then lead to a conversation between the two.
As I cannot animate a full story, I picked a scenario in which the two could interact and would allow movement, so that I can test the puppets to their maximum capabilities. I have however enjoyed writing this scenario and character bios and the story maybe something that I develop further in my spare time.
As part of my 701 Project I needed to create a way that all the different changes that I have made and how the project has developed can be seen clearly. I needed to create a project map that gave an overview to the project but that could also be expanded upon for certain sections.
At first I wasn't sure how to do this and I made a draft map in photoshop and I also started to make a presentation in prezi.
Neither of these methods worked completely, the photoshop document was too small and while it gave a nice overview and map outline of the project I couldn't add more detail that allowed sections to be elaborated on more, as when more text was added it looked cramped and messy. The prezi did allow me to add more detailed notes about different sections but it did not allow me to keep the overall map outline of the project as when you zoom into a section to expand on you lose the overall map. After some thought and speaking to people at work, I decided to create a google sites for my project, this allowed me to create menus for different sections, add images, add documents and presentations and the most useful bit for me was to have sections that have a title and then have a drop down, this allowed me to add the overall map on display and then allows readers to go further into the work through using drop downs and links to further reading. It also allowed me to present all the different work nicely in a single location.
The intended audience for my project differs from most of the other MA projects that I have seen this year. As my project does not result in a finished animation with a complete narrative it isn't meant to be submitted at film and animation festivals as a short. Because it isn't narrative lead and instead focuses on developing new processes it does not have a target age range or group of people in the traditional sense. My project is aimed at people who are already working in the animation industry.
My project is a research and development based project its finished outcome is intended to be a completed production pipeline, with a guide of how to create stop motion puppets intended for use in virtual reality, which will be accompanied by a written research proposal that outlines the potential development of the project into a virtual reality interactivity piece. The finished pipeline and research proposal will be presented at animation conferences to professional animators and companies, where I can hopefully acquire the funding to progress the idea to the next stages of development.
As part of this project I will need to identify animation conferences, talks, studios, professionals and possibly festivals which I can attend and promote my project. At the end of my MA course I will also look at the possibility of using this completed project as a PHD proposal in order to further develop the idea as part of a formal academic qualification.
Another area that I wanted to clarify in my head before the submission of 701 was the context of my project and practice. I decided to look at my project as a whole again and define what context was informing my work.
I think this was important for me to do this as my project has changed so much since the start that being able to redefine my context will really help me as I start to move forwards and progress with the project.
Why am I making it?
I am making this project as I enjoy making stop motion and stop motion puppets.
I want to upskill and learn how to use digital sculpting and animation softwares such as ZBrush, 3DsMax and Maya.
I want to learn how to make and print stop motion puppet heads that use replacement faces. I like VR animation, games and VR painting.
Having a stop motion in VR is really interesting to me, I want to see how it is done, does it give the same emotions when viewed, if it feels different to see stop motion in VR compared to the usual clean cut animation that is found in it.
I am a stop motion puppet maker and animator working with the area of stop motion, virtual reality, games, photogrammetry, interactivity, materials testing and more.
I am investigating the combination of traditional animation skills and techniques with digital and modern processes. With the aim of using this mixture of traditional and digital to test stop motion in virtual reality.
I am looking at the use of modern softwares such as ZBrush, Maya and 3DS Max and use of VR with Adobe medium to sculpt and create 3D printed puppet heads with replacement faces systems and possibly replacement hand gestures, combined with the traditional processes of puppet production, such as armature building, costume, casting ect to create a fully functioning stop motion puppet.
The designs and creating of the two puppets for my project will focus on character expression and interaction between the characters developing and leading towards audience and character interaction.
The testing of various input methods that can be used to capture stop motion footage and then input the footage to be viewed in a virtual reality environment.
Identifying a production pipeline that successfully works to produce, shoot, edit and composite a finished stop motion animation that has been created which can then be viewed in virtual reality. That also has the possibility to progress into an interactive, stop motion and virtual reality animation.
Other research -
To have looked at the theories of the importance of touch in the making of animation, if traditional processes still have relevance in a digital world and how characters can be used to interact with an audience to make the viewing of animations more immersive. I will see if these theories impact my stop motion footage. An example would be that if I create a puppet using 3D printing and traditional techniques which is then 3D scanned, rigged and animated digitally if I compared this to a puppet created using the same method but that was animated using traditional stop motion techniques would the amount of touch involved in the production stages impact the finished outcome.
Seeing if stop motion animation has place and relevance in a digital world.
Seeing if character interaction with the audience has an impact on the viewer experience and if this experience is a positive or negative one.
Research and learning of how to use the various softwares, what do I need to learn? Do I need to be competent in it, or do I just need to learn to use certain tools?
Haptic technology where is it going? What's happening in it?
Photogrammetry - how to do it, software, equipment?
Stop motion - using 3D CGI sets, using 3D digital rigging, 3D scanning
What's happening now in haptics, touch, photogrammetry, stop motion,
Ethical Issues
Woman working in the tech industry
Largely traditionally male dominated areas of stop motion and virtual reality.
Is this going to impact how it's received? Will I need to prove that I know what I am talking about before people will listen?
Is it going to impact me when I research? Will I be able to get the same answers from people compared to if I was male?
Sustainability of stop motion puppet production, and materials used.
This week I wanted to look at the methodology that I use when working. After being advised to look at the action research structure, this methodology seems to fit how I work. I decided that I would research further into this area as having a clearer understanding of how methodologies can affect my work, will help me as my project progresses.
Action research is a methodology that brings together practice and theory (Beever, 2018) to deliver a practical outcome. Action research involves a number of steps that help the user go from theory to testing and practical outcomes. The stages involved are:
“Reflect
Describe the problem or opportunity
Give the background to the problem or opportunity to the industry and identify the stakeholders involved.
Define the objectives for the action research, ensure they are S.M.A.R.T (specific, measurable, attainable, relevant and timely)
Review the current situation, the knowledge around the problem or opportunity, the needs of the different stakeholders and as a group identify recommended actions
Plan
Develop an action plan for the actions: What is going to be done by whom and when
Develop a monitoring and evaluation plan; how are the actions and outcomes achieved (against the objectives) are going to be monitored and evaluated
Act
Undertake identified actions
Observe
Monitor and evaluate, make sense of the outcomes
Reflect
Undertake critical review of the outcomes and use this to inform future plans and associated actions against the objectives.”(Beever, 2018)
This method follows the Kolb learning cycle (Harb, Durrant and Terry, 1993), realising that action research follows this theory, helped me understand it better, Kolb's learning cycle is a theory that I learned about when doing my teacher training, this means it is a theory I am already very familiar with so I didn't need to do much research about this theory again, it also helped me better understand action research and identify that it was a method I have been working in for a long time without even realising.
I looked into action research further and found this image that describes the process nicely
(TerBorg, 2016)
Now that I have identified that I already work using this method I decided to use the stages outlined and applied them to my own research.
I decided to fill out this list of stages for where I am now in this project. If I completed this now from the start of my project to date it would take too long, and as I have already moved on for these stages they are no longer relevant to my work now. In future I think that completing a rough set of questions and working more clearly to these stages will help me, as it helps me to visually see where I started to where I am now, it also helps me plan clearly what I need to do, which then means I know exactly what I need to act on, I can then collect what I have found, reflect on it and then start the process again until I reach the finished conclusion.
Identify
What is my question?
How do you create 3D printed stop motion puppets, whose actions focus on expressions and interactions, that are intended for use in a virtual reality space?
How do you collect stop motion footage and input it into a virtual reality space?
What Problems do I see?
Time frame - this project is going to be very time consuming and take a long time to make.
Skill - what skills do I need to make this work and do I have them?
Plan
What research to collect?
Theory - character expressions, haptics, importance of touch in animation, traditional animation vs digital animation, stop motion and VR
Industry - Who is working in this field? What work already exists? What companies are working in this area? What conversations have been had about stop motion, VR and photogrammetry? What software and methods have been used?
Practical - Software that I need, software and skills in it that I need to learn, methods and techniques to input
Time - How long will it take? What is the production pipeline?
What do I need to learn?
Zbrush, Maya, 3DsMax, Photogrammetry, 3D printing, 3D printed replacement face system, 360 editing, 3D scanning, 3D rigging, 3D environments,
Act
How will I implement what I have learnt?
Theory - use findings to inform how the characters are made, how their expressions and interactions work in the tests, what they are made of.
Industry - Use information on methods and softwares that have been used in industry to make test animations, make contact with people who are working in this area.
Practical - learn software and specific skills in software to create the project
Time - create a production pipeline, undertake tests to understand the time frame needed, create a production schedule.
Observe
Collect what I have learnt
Reflect
Share what I have learnt
Reflect on work
Revisit
This method works as a loop and once stages have been completed and you have reflected on them the process starts over again. I am currently in the planning and acting stages for the theory section as I am collecting the theoretical research needed for my project. For the practical stage I will move on to the acting stage next year in 703.
This has helped me layout my work more clearly, as my project is so different it becomes confusing to not only other people but also myself, laying out the stages like this really helped me organise all the details in my head and understand what I need to do in each stage. In future I will use this method to outline questions I want to answer and then break them down into stages.
References
Beever, G., 2018. Action Research - Extension Practice. [online] Extension Practice. Available at: <https://extensionaus.com.au/extension-practice/action-research/> [Accessed 8 March 2022].
Harb, J., Durrant, S. and Terry, R., 1993. Use of the Kolb Learning Cycle and the 4MAT System in Engineering Education. Journal of Engineering Education, 82(2), pp.70-77.
TerBorg, S., 2016. Action Research. [online] sonya terborg. Available at: <https://sonyaterborg.com/2016/02/17/action-research/> [Accessed 8 March 2022].
This week I wanted to further define where my projects sits in relation to the animation industry. My project branches out over many sectors of the animation and games industry and does not solely sit in a single category. This means that there is simultaneously a large research base and a very niche research base. Each of the individual elements that my project sits with in: stop motion, VR, game, interactivity, animated audience participation, tactile animations, traditional animation and modern computer animation, all have countless journal articles and research conducted in to a variety of subcategories within themselves creating a very large and broad area in which to conduct research. It is when these areas are combined into a single idea that the amount of papers and people working within this area becomes considerably smaller. Even if I only combine two of these elements such as stop motion animation in virtual reality, the amount of sources to draw upon is limited. I was able to find a few examples of stop motion animation being used in virtual reality but these are very limited.
In the examples of animations that use both stop motion and virtual reality that I found, I noticed that they can visually differ considerably. Some of these animations aim to up the creepy factor that is often associated with traditional stop motion films, while the others play on the more modern stop motion techniques and ideas, by simply aiming to create visually stunning and technologically advanced animation. These two visuals are almost polar opposites when viewed. One aims to invoke a sense of unease in the audience while the other aims to be enjoyable and show off the skill of the animators who produced it. Both of these rely on the audience's emotions and their reactions to being fully immersed in an animated experience.
In order to further situate my work, I decided to study the examples that I could find further. This will help me place my own work, in terms of visuals, audience, technology and distribution. By looking at existing pieces of work I will be able to decide which of the visual styles I want to aim to achieve, who my intended audience is, this will then help determine how and where the animation is distributed.
This example of an animation that relies on the creepy factor, is Gymnasia (NFB/marketing, 2019) which is a stop motion animation produced in 2019 that was created using stop motion puppets and sets and then had 3D elements added. The most interesting element of this piece is that it was created without using photogrammetry. This animation experience was created by Felix and Paul studios (Felix & Paul Studios, 2022) which is a studio who has created many different virtual reality experiences, they are the studio responsible for creating the virtual reality behind the scenes tour of the Isle of dogs (SearchlightPictures, 2018). When interviewed the studio said "The shooting process was difficult," (Low, 2019). The article and studio website don't go into more detail about how the filming process for this was achieved, but this is definitely a studio that I need to try and contact when I am starting production and testing for my own animation. This example shows how animation in VR can be used to interact with an audience and invoke different emotions in them, it has elements of visual interaction that between the stop motion characters and the audience, these interactions are initiated by the stop motion motion element. This combined with a sudden character appearance add an element of shock to the animation.
The style of the animation appears heavily inspired by the work of Jan Svankmajer (Film Forum, 2020), and the Brothers Quay(BFI, 2016), due to the creepy aesthetic and use of dolls. This aesthetic was likely selected to add the feelings of unease in the audience, both Svankmajer and Quay's work, aimed at making the audience uncomfortable and unnerving, make the viewing of their animations not as pleasant when compared to work produced by studios like Disney. This animation also plays on the idea of the Svankmajer touch (Vasseleu, 2009), Svankmajer work is known for being tactile, with touch being an essential element to his work, he also held the theory that objects retain memory that has been imparted upon them when touched, and that these object can then recall that memory.
“This is the magical ingredient in Švankmajer’s alchemy. He cites Merleau-Ponty’s studies of Goldstein’s neurological cases, which discredited the idea that touch only occurs as a result of direct physical contact, as proof of the existence of ‘tactile memory’ (Švankmajer1994: 234). The tactile imagination is capable of retaining and transforming tactile memories into analogies that are charged with psychical intensity. For Švankmajer, seemingly inert objects have the mutability to arouse this tactile sensibility, which can be recalled from earliest childhood, and resurfaces in states of extreme emotional agitation.”(Vasseleu, 2009)
Svankmajer's theory about objects retaining memory and touch helps add to the creepiness that both his own animations and Gymnasia play on. This factor is further augmented in animations like gymnasia that are viewed in a virtual reality space, because these spaces are immersive, the audience becomes an active participant in the experience which only adds to the feeling of unease as it becomes more real to the viewer.
References
BFI, 2016. Street of Crocodiles (1986) clip. [video] Available at: <https://www.youtube.com/watch?v=nW3dW4yMLfE> [Accessed 5 March 2022].
Felix & Paul Studios. 2022. Felix & Paul Studios. [online] Available at: <https://www.felixandpaul.com/?gymnasia> [Accessed 5 March 2022].
Film Forum, 2020. Jan Svankmajer's FAUST Trailer. [video] Available at: <https://www.youtube.com/watch?v=UyW46Pbsp9k> [Accessed 5 March 2022].
Low, C., 2019. VR is a strangely fitting home for stop-motion animation. [online] Engadget.com. Available at: <https://www.engadget.com/2019-05-01-vr-stop-motion-gymnasia-tribeca-2019.html?guccounter=1&guce_referrer=aHR0cHM6Ly93d3cuZ29vZ2xlLmNvbS8&guce_referrer_sig=AQAAAHphhI0psEaV_x661jy2ln8MD8bnw_9WxMH418K9wo-6rD8igy-tPeUFXmPREDuWeEhZzBPVOkZdeubUQJpd2B2PGC5dOfU7Uf0k5kOcalwcaJKIIkPvxBPDpZyqpmZsT7UDa_mvE2irqToDVBoey5e0qCiRrYhIyWqC-0wsJOih> [Accessed 5 March 2022].
NFB/marketing, 2019. Gymnasia - behind the scenes. [video] Available at: <https://vimeo.com/332552376?embedded=true&source=vimeo_logo&owner=44219905> [Accessed 5 March 2022].
SearchlightPictures, 2018. 360° | ISLE OF DOGS | Behind The Scenes (in Virtual Reality) | FoxNext VR Studio. [video] Available at: <https://www.youtube.com/watch?v=JqXC46b1uUg> [Accessed 5 March 2022].
Vasseleu, C., 2009. The Svankmajer Touch. [online] Available at: <https://journal.animationstudies.org/cathryn-vasseleu-the-svankmajer-touch/> [Accessed 5 March 2022].
Example 2
A second example of a stop motion for VR is Mad god by Phil Tippet (Wevr, 2017), this is one of the more well known stop motion animation in VR, this animation was directed by Phil Tippet and was made by a combination of studios working together. The different studios that worked on this project were Tippett Studios, Wevr, Kaleidoscope and Foundry. Some of the people that helped make this project were Michael Breymann - head of kaleidoscope, Anthony Batt - executive producer - Wevr, Alex Hessler - VR supervisor - Tippett Studio, Chris Morley - VFX supervisor Tippett studios.
Mad god, was produced using stop motion puppets, stop motion sets, 3D CGI sets, Photogrammetry and lots of editing. The camera for this is animation is centrally positioned with the stop motion set around it, each frame of animation includes a number of images taken by the camera been rotate 360 degrees, the resulting footage is put into digital softwares, 3D CGI elements are added and then it is edited to achieve the finished VR animation (Kaleidoscope, 2017). This animation uses a different and more complex production method than example one, but the resulting animation is just as immersive, similar to the first example it plays on the creepy factor of stop motion and is meant to unsettle the audience. I would need to find out more about how the first example was produced before I am able to compare the production pipelines used to create them. I will need to research both production methods more in depth as I come to produce my own tests, this research will help me identify processes that I can use in my own work. From researching Mad God I have already found studios and professionals to contact and I have also started to identify further production methods, techniques, stages and softwares used.
From watching these behind the scenes videos of the making of Mad God(Foundry, 2016), I started to get a better understanding of the massive scope of the project and how many people were involved in making this single short animation. When I compared the finished footage from Mad God to my own ideas and ambitions that I wanted to achieve I started to realise that I had way, way over estimated my own ability, timescale for the project and the workload that would be involved. It was after watching this that I started to worry about my initial project idea and when I ultimately decided to cut back considerably on my plans. The behind the scenes was really informative and helped me gather a wide range of information as well as give me areas to investigate and research in further. It also helped me identify the amount of skills and software that I will need to use to create my idea, some area I will need to somewhat master, other I will need to be at least very competent in, these skills range from one that I already have such as stop motion, puppet and set building, to one that I would need to learn these included 3D CGI modelling, rigging, animating, Photogrammetry - both the process and the software, coding, compositing, editing and possibly more. While this is all something that I would like to upskill myself on eventually, learning and mastering these skills in two years while also completing the masters work and teaching full time, is not possible for me to do. Had I not watched the making of Mad God I don't think I would have come to this realisation when I did.
References
Foundry, 2016. Tippett Studios head into the unknown with stop-motion VR. [video] Available at: <https://www.youtube.com/watch?v=oP0HmNZjn18> [Accessed 5 March 2022].
Foundry. 2022. VR, AR and MR | Insights | Foundry. [online] Available at: <https://www.foundry.com/insights/vr-ar-mr> [Accessed 5 March 2022].
Kaleidoscope, 2017. FEVER DREAMS - Phil Tippett & The Making of Mad God VR. [video] Available at: <https://www.youtube.com/watch?v=3tQSPRifnuE> [Accessed 5 March 2022].
Wevr, 2017. Mad God 360 VR. [image] Available at: <https://www.youtube.com/watch?v=h6CTrqlpKFk> [Accessed 5 March 2022].
Example 3
The third example animation is a behind the scenes look at the making of The Isle of Dogs (Searchlight pictures, 2018), this example differs from the first two as it isn't a complete stop motion animation in VR this experience instead puts the audience in the middle of an animation studio and gives them an up close look at the animating process. It has the puppets and voice cast narrating, where they are telling the audience details about the film and their characters, as the audience turns in VR you can see a timelapse of the puppets being animated and then footage of some of the puppets being made. This animation also differs from the first two example in visual style, this animation is not meant to unsettle the audience and is instead meant to give them an insight into the stop motion animation process, the characters used in this animation are much more visually appealing and have the modern refined aesthetic that has started to be shown in recent stop motion animations. This change in aesthetic is most often found in stop motions intended for a young target audience both in TV shows and feature films.
I prefer the aesthetic in this animation mush more and is definitely the style I will be aiming for, while my intended finished piece does involve audience interaction, I want that interaction to be a pleasant one not an unsettling one, with the focus instead been on the characters and how they interact with each other as well as the audience.
References
Searchlight pictures, 2018. 360° | ISLE OF DOGS | Behind The Scenes (in Virtual Reality) | FoxNext VR Studio. [video] Available at: <https://www.youtube.com/watch?v=JqXC46b1uUg> [Accessed 5 March 2022].
VRScout. 2022. Wes Anderson Unveils 'Isle of Dogs' Stop Motion VR Experience at Sundance - VRScout. [online] Available at: <https://vrscout.com/news/wes-anderson-isle-of-dogs-stop-motion-vr-sundance/> [Accessed 5 March 2022].
Example 4
The forth example is one that I could not actually find the full animation to watch in VR but is instead on that I stumbled upon while researching photogrammetry and stop motion(MIZ Babelsberg, 2017). This video outlines some of the steps used to create a section of stop motion animation that is then put into a 3D environment to be watched in VR. This animation was created using photogrammetry and turn table style devices to rotate around a puppet after each movement, essentially creating a 3D scan of the puppet after each action. When I researched further I was able to find a website for the device which is called Orbital (Hagen, 2022).From what I can see of the finished animation, the resulting footage using this method is very good. I would like to try and find the full experience and watch it in VR but as of now I have not been able to. The methods used in this video are something that I will consider using to test my own puppets, as the method is a much smaller scale than the others I have researched, using a device like an orbital to capture the puppets and stop motion footage to then put into a 3D environment is something that I am more likely to be able to achieve as opposed to the full productions seen in the other examples. This example is also another one that visually is meant to be pleasing like the isle of dogs example, this is good to see achieved at a smaller scale as it means that how I plan to present my animation tests and puppets has a chance of working well.
References
Hagen, G., 2022. Gard Hagen - Orbital. [online] Gardhagen.com. Available at: <https://gardhagen.com/orbital-1> [Accessed 5 March 2022].
MIZ Babelsberg, 2017. Stop-Motion VR. [video] Available at: <https://www.youtube.com/watch?v=GNiyjnNhxAM> [Accessed 5 March 2022].
Conclusion
This research has helped me to see what existing animation produced using stop motion and VR are available and has helped me position my own work in the animation field. While there are not many people working in this combination of areas there are people and studios that are producing and advancing this work. I haven't been able to find anyone talking about the inclusion of interactivity within these fields yet, but it is something that there are definitely people having discussions about, this is an area that I will keep researching to try and find. My own work and project is positions along the edges of many different areas and draws from them all, while the area it sits is small alone when viewed with all the elements combined I have wide base of knowledge that I can research and draw upon to help me achieve the intended outcome.
