Photogrammetry test - Doll

As part of the testing with photogrammetry, I wanted to start to look at how it works capturing puppets, rather than build a full puppet I decided to use a doll. I blue tac this down to a base and took photos using the same lighting set-up as before. 

This test could have worked better. When I put the photos into agisoft it didn't recognise certain areas. Like the legs and arms, and the rest of the model came out really bumpy. I'm not sure if this is because the doll I use has a slight glitter layer on the skin and it was too reflective for the software.

Here is the point cloud

From this point cloud, I create a dense point cloud. 

Here is the dense point cloud, at this point, I could see that it was having issues with the legs and arms, but I finished the test to see the results. I cleaned up the cloud and then build the model. 

Here is the model solid

The model shaded, you can see on this that the model is all bumpy and has a really uneven surface. 

The model mesh

The model confident

And the model textured. 

While this test hasn't worked, I have learnt a few things for the next test. 

1. I need to try and get closer when I take photos. 

2. I will find another doll that does not have glitter skin. (I believe this is the reason the arms and legs look so bad as these are the bits with the most glitter, which means as the camera is taking the photos the light will be reflecting all over which agisoft then cannot read.)

3. I will find a ND filter for my camera, which means that I should be able to get images of shiny surfaces. 

I will conduct another test, using the above things and see if I can get better results. I am also going to research how other companies have conducted 3D scans similar to this. I will try to email Matt (old tutor) and see what methods he has used as he has done similar before and captured the wind in the willows puppets.

Wacom camera test

 I went to complete the test with 3D scanner that is built into the Wacom over an hour of arguing with the software and the camera it would not work, so unfortunately I will not be testing with this. Instead I will focus on testing using agisoft and then inputting into Maya

Photogrammetry test 3

 As part of the testing in photogrammetry, I decided to practice and learn how to decimate a mesh, this is an important part of the production pipeline as it enables me to create a model that looks good but is also easy to handle. 

This is a low-density point cloud

 

This is a high-density point cloud

I then made it into a model. You can see the missing sections clearly here this is where images of the diorama were missing or unsuitable, it has also struggled to capture any of the resin water, this is most likely due to it being shiny, I will have to conduct tests with more resin objects as this will most likely be the material that the puppet heads are made of. 

This is the model solid

This is the model's confidence, here I can see where I need to collect more images from

This is the mesh for the model you can see it's really dense. 

I created a texture for the dense mesh to see what the finished model would look like.

I then decided to play around with the mesh and see if I could reduce the number of polygons in it to make it more optimised.

I adjusted the mesh for the model, and reduced the number of polygons, this made sure that it is a manageable object but I also need to make sure that it looks good. I played around with decimating the mesh to different amounts, I decided to reduce it right down to 10,000 polygons, this was to test just how low I could take the mesh before I had problems with the appearance.

Here are the shaded model and the model solid for the optimised mesh, while this doesn't look good, I can still use this mesh and models and reapply the textures to the model. I reapplied the textures and got this result.

I reapplied the textures to the model and got this result, while it isn't as high a quality as the dense mesh it is good enough quality that I will be able to use it for background objects, this means I can use dense meshes for the object of importance and detail and can use lower mesh density for background objects. While this isn't as detailed as the first render it is still of a high enough quality. 

I then experimented with creating a texture map from the agisoft software. This is the texture map for the dice tower.

This isn't a good texture map if I wanted to paint an area, say add graffiti to the tower wall, on this map the tower wall is all over. I now know that to create a texture map that I can use I need to put the scan into 3D software, and rebuild and re-UV the mesh, this will mean that I can create a texture map that I can use in photoshop and repaint and modify. I will also need to put the object into 3D software if I intend to animate it and again rebuild the mesh and add UVs, as this is the only way I will be able to animate the scanned objects. This will be an important step for the puppets I capture in photogrammetry. 

Photogrammetry test 2

Building on from my first attempt in photogrammetry, I wanted to learn how to improve the point cloud and remove unnecessary points. I watched some guides and had help from an industry friend. From this I was able to re-input the same images and work through the entire workflow again, this time the results were much better, there are still images that didn't work but overall the results were better. I learn to tidy up the point cloud and also remove any points that I can't see. From this, I learn how to adjust the model mesh and add textures.

For this test, I learnt how to decimate the mesh to reduce the polygon count, which creates an optimised and efficient model ready for export to an additional 3D software for clean-up and UV mapping, this is the next stage of testing. This is an important factor for me to consider in my project because it will enable me to create a model that is light enough to work as intended and that is easy to handle, but also that has a level of detail that looks good and is appropriate for the project as I will have a lot of individual assets. 

These are the results below: 

I am much happier with the results this time, I have learnt a lot more and feel much more confident to create these in future, I may try and capture some generic items to practice taking the correct photos, and I am also going to take some more images of the dice tower diorama and attempt to capture the whole model.

Practical Tests and Research Methods

 Do the tests, test pipeline, theory, or both?

As part of the 703 unit I began to conduct practical tests, as part of this testing phase I started to identify if my planned tests were been used to test purely pipeline, purely theory or a combination of both, this is an important distinction to make throughout the project and something that I will have re-evaluate often. To help with this process I decide to explain what I was going to do in each test, I found that when I explained the tests this way it was easier for me to understand which area they were been used to test. The 3 tests I originally planned are listed below, I have also added a 4th test which was an idea I came about recently as I worked more on this unit. 

Test 1

The first test idea is to create a 360 camera test, this would be using a 360 camera where objects would be placed around it, these materials could be made items such as a dice tower diorama, and it could also be a collection of objects that have different textures and qualities, these will be captured using the camera and the resulting footage will be used to learn and test how to composite for VR in After Effects. 

This will test a section of the production pipeline and will see if After Effects is suitable to use for my proposed project.

Test 2

The second test is to create a 3D scan using the 3D scan camera on my Wacom mobile studio, I have never used this camera before but as it is made specifically for capturing 3D objects it may be suitable to use in my project. For the test, I will try and capture the dice tower diorama, an existing stop motion puppet or toy such as a doll, or try and capture the textures of different materials. This footage will then be put into 3D software (I'm not sure if the camera has native software yet) and I will see what results I can get. 

This will also test a section of the production pipeline as if the camera is suitable I may be able to use it as a quick and easy way to capture physical objects for use in 3D. 

Test 3

The third test is a photogrammetry test, for this test I will take multiple images of the dice tower diorama that I made using a canon camera and soft lighting set-up. I will then put the images into the photogrammetry software Agisoft Metashape, where I will create a photogrammetry scan and 3D model of the tower. I will then take the resulting 3D model into 3D software, After Effects, or possibly Unreal Engine, with the hope of getting it into a VR space. 

This will again test a section of the production pipeline as I will be able to see how photogrammetry can capture and map the textures of practical objects that are on a small scale. If this method works it could be used to then test areas of theory, as I can use it to capture objects and texture and put the results into VR, this would then allow me to record how participants react to the textures in what is commonly known as a digital space, this reaction would test how participants use visual tactility and Haptic memory.  

Test 4

The fourth test is the creation of a practical materials test board or test box. This will be where different items that have a variety of different tactile surfaces. This could be materials applied to a foam board such as different fabrics, shiny/reflective materials, objects of materials with textures (like glued sand, egg boxes etc.), latex samples, silicone samples, paints and colours. Or it could be loose items to be placed and scanned individually, such as slime, velvet, toys (dolls, teddy bears etc.), metal, hot items or items that smoke (cigarette, cup of coffee), cold items (ice, snow), clay, etc. These would be used to develop practical outputs, where they would be taken from physical objects to digital ones. 

This test would test the production pipeline and would expand on the above tests, but its main function would be to test the theory. 

One example of the practical test is to create a table in 3D, the objects and materials would be captured using photogrammetry. The scans of the objects would then be added to the table (testing Photogrammetry methods and 3D methods) These would then be composited into Unreal Engine where the existing VR presets could be tested (testing a new production pipeline method) This would then result in a VR piece of work that people could interact with. 

This practical output could then be used to test different theories. Because these materials and objects are ones that most people know, it would be able to test my theories about tactile visuals, haptic memory and embodied memory in VR. The practical VR piece would be used by a participant group where they would be asked to interact with the objects. This interaction would be recorded, with specific attention paid to if there are any objects that users avoided touching or objects they touched more. This interaction would be uninterrupted and the participants would not be guided at all, just allowed to interact freely. The time in the space would then be followed with a questionnaire, which would ask the users: why they had chosen certain objects, why they had avoided certain objects, how they felt about objects' textures, etc. 

The practical interactions with the VR piece and the questionnaire would look at my theories about how much visual haptics and embodied memory play a part in VR experiences, and if they do make them more immersive. It will question if even though a participant can't really touch the object, if they still imagine they can feel it and avoid it for that reason, or if they are more drawn to certain items because of their own embodied memories (such as toys and teddies). It will also look at if users who do touch items they avoided how much they can feel them (if at all) through visual haptics and haptic memory. This feedback will help me to answer if participants can still feel through the use of visual haptics do they feel like they know what the objects would feel like, or do they experience the feeling even though they can't actually touch them?

Next steps 

The next steps are to identify the research methods that I will use to conduct these tests, this will help inform my project more, and identify how I will record and use the findings of the tests. 

What Research Methods do the Tests Use?

Once I had identified if the tests were testing production pipeline, theories or both, I then needed to identify what research methods I would use to conduct these tests. As part of the 703 unit, I am writing the methodologies chapter for a PhD proposal, as part of this I have been looking at various research methods. I also spoke with my project supervisor who explained that the method that I use will be unique to my project, while it will draw on areas of all these existing methodologies the actual wording of my method will be outlined by myself. In order to do this I first wanted to identify the existing methods that I am using for each test, as well as how I will use them to conduct the test, record the findings, and evaluate to then act on and develop the work further. 

Test	What is it testing?	Method
Test 1 - 360 Camera	Pipeline	Action Research  Experimental making Analysis Reflective Practice
Test 2 - 3D Scan	Pipeline	Action Research  Experimental making Analysis Reflective Practice
Test 3 - Photogrammetry	Pipeline and potentially theory	Action Research  Experimental making Analysis Reflective Practice
Test 4 - Materials	Theory - Also refining pipeline and testing new pipeline.	Action Research  Focus group research Participant observation research Experimental making Questionnaires Analysis Reflective Practice

The tests all use Action Research, Experimental making, Analysis and Reflective Practice. The test with the biggest set of research methods is the one that tests theory and practice, this is because this not only is looking at testing both theory and practice but also has numerous stages to its development. 

How will these methods be used?

Action research will be used to test the efficiency of the production pipelines, which includes: if the software is appropriate, how hard or easy it is to do, is there a better software, is there a better way/method, are there any problems that need solving, is there anything that needs investigating further, reflecting on it and refining. This method allows me to:

1. Research, plan and identifies what I will do

2. Test what I have identified

3. Evaluate how well it went or didn't go

4. Reflect on these findings, make adaptions and start the cycle over again. 

While I have identified the next 3 methods they are also linked to action research as they are very similar to some of the stages it uses, while I will be using these as part of action research I will never be using them solely/alone. 

Experimental making will be used as all the practical work and tests are experiments, I don't yet know which methods will work, I can only find this out by experimenting and making different pieces of work that allow me to test different software, methods and pipelines. Similar to stage 2 of action research.

Analysis and reflective practice will be used to reflect on how the practical tests and experiments have gone, I will use these to look at the tests, and see what went well, what didn't go well, and what could be better. I will then reflect on the tests and implement changes that should improve them, and test again. Similar to stages 3 and 4 of action research. 

The next 3 methods I have identified are: Focus group research, Participant observation research, and Questionnaires. These research methods will be used once I start to work with other people and get their feedback and are the methods that will be used to help prove identified theories. All of these methods will be used in test 4 where a practical piece of VR work has been produced. The VR piece will have a group of participants interact within the VR environment (Focus group research, different groups eg other animators [MA group and staff] younger people who are not professional animators [my students]) These participants will be observed and their interactions in the VR environment recorded (both what they see and their reactions) no guidance will be given for this other than the odd prompt to encourage them to do what they want (participant observation) following this interaction they will then be asked different questions to see why they integrated with the piece the way they did this feedback will be recorded (questionnaire)

All of these methods will enable me to test the production pipeline and test theories that I have identified as well as record the findings from these tests. I will also use other research methods as I continue to conduct background research into the subject areas that my project sits within, these are Qualitative research, media analysis, case studies, and contextual reviews. It is important that I also continue to research as I test even though the proposed project is new it sits within such a broad spectrum of areas and disciplines there is lots of theory, and existing pieces that I can pull from, which will expand my base of knowledge and impact the projects practical outcome as well as its theoretical grounding. 

Next Steps

I now need to refine these methods and start to develop my own method that I will use in the production and progression of the idea.

What is a Research Framework - Paper Structures

 As part of the 703 unit, I had to write the methods chapter of a research proposal for a PhD or conference paper to begin this I first had to research what was involved in these and collect points that I would cover. To start this research I looked at the overall structures of the papers and what chapters they include. 

For the PhD proposal I found these points:

Site 1 points - 

• Highlight how it is original or significant

• Explain how it will develop or challenge current knowledge of your subject

• Identify the importance of your research

• Show why you are the right person to do this research

Site 2 points -

Your proposal should contain at least the following elements:

• A provisional title

• A key question, hypothesis or the broad topic for investigation

• An outline of the key aims of the research

• A brief outline of key literature in the area [what we already know]

• A description of the topic and an explanation of why further research in the area is important [the gap in the literature - what we need to know]

• Details of how the research will be carried out, including any special facilities / resources etc. which would be required and any necessary skills which you either have already or would need to acquire [the tools that will enable us to fill the gap you have identified]

• A plan and timetable of the work you will carry out

Site 3 points -

1. TITLE

• Your title should give a clear indication of your proposed research approach or key question

2. BACKGROUND AND RATIONALE

You should include:

• the background and issues of your proposed research

• identify your discipline

• a short literature review

• a summary of key debates and developments in the field

3. RESEARCH QUESTION(S)

You should formulate these clearly, giving an explanation as to what problems and issues are to be explored and why they are worth exploring

4. RESEARCH METHODOLOGY

You should provide an outline of:

• the theoretical resources to be drawn on

• the research approach (theoretical framework)

• the research methods appropriate for the proposed research

• a discussion of advantages as well as limits of particular approaches and methods

5. PLAN OF WORK & TIME SCHEDULE

You should include an outline of the various stages and corresponding time lines for developing and implementing the research, including writing up your thesis.

For full-time study your research should be completed within three years, with writing up completed in the fourth year of registration.

For part-time study your research should be completed within six years, with writing up completed by the eighth year.

6. BIBLIOGRAPHY

You should include:

• a list of references to key articles and texts discussed within your research proposal

• a selection of sources appropriate to the proposed research

 

I then broke all these points down into sections I would need to include in a PhD proposal, these could differ on where I am applying and what they require. 

1. Title

2. Supervisor

3. Proposed mode of research

4. Aims and objectives

5. Synopsis

6. Background

7. Expected research contribution

8. Proposed methodology 

9. Work plan

10. Resources 

11. Bibliography 

I then looked at the structure of conference papers, this was a bit easier as Micheala had provided me with a document that outlined the structure, from the document provided I made these points for what I would need to cover 

1. Abstract

2. Authors keywords

3. ACM Classification Keywords

4. Introduction

5. Related work - what already exists 

6. Design of your Widget - description of your work

7. Subsections - technical 

8. Methods

9. Results

10. Discussion

11. Future work

12. Conclusion 

13. Acknowledgements

14. References 

From this research, I have found that the points I will need to cover are very similar in some aspects and different in other places. This means that in order to produce these documents I can keep some sections the same and make only minor alterations, other sections will have to be specific to the document that I am writing. 

The next steps of this work is to collect all the research method I have been collecting and plan how I will use them in my work, I will need to adjust these as relevant to my project. I can then begin to write the methods chapter. 

Bibliography 

York St John University. (No Date) Examples of Research proposals - York St John University [Online]. Available from: <https://yorksj.ac.uk/study/postgraduate/research/apply/examples-of-research-proposals/> [Accessed November 15, 2022].

McMeekin, N. Wu, O. Germeni, E. Briggs, A. (2020) How methodological frameworks are being developed: evidence from a scoping review. BMC Medical Research Methodology | Full Text [Online]. Available from: <https://bmcmedresmethodol.biomedcentral.com/articles/10.1186/s12874-020-01061-4> [Accessed November 15, 2022].

McCombes, S. George, T. (No Date) How to Write a Research Proposal, Examples & Templates. [Online]. Available from: <https://scribbr.co.uk/the-research-process/research-proposal-explained/> [Accessed November 15, 2022].

The University of Sydney. (No Date) How to write a research proposal, The University of Sydney [Online]. Available from: <https://sydney.edu.au/study/how-to-apply/postgraduate-research/how-to-write-a-research-proposal-for-a-strong-phd-application.html> [Accessed November 15, 2022].

Higginbotham, D. (2022) How to write a successful research proposal | Prospects.ac.uk [Online]. Available from: <https://prospects.ac.uk/postgraduate-study/phd-study/how-to-write-a-successful-research-proposal> [Accessed November 15, 2022].

University of Westminster. (No Date) How to write your research proposal | University of Westminster, London [Online]. Available from: <https://www.westminster.ac.uk/study/postgraduate/research-degrees/entry-requirements/how-to-write-your-research-proposal> [Accessed November 15, 2022].

Mills, A, J. Durepos, G. Wiebe, E. (2010) Research framework. In Encyclopedia of case study research (pp. 814-816). SAGA Publications, Inc., https://dx.doi.org/10.4135/9781412957397.n299 [Online]. Available from: <https://methods.sagepub.com/reference/encyc-of-case-study-research/n299.xml> [Accessed November 15, 2022].

Godfrey, M. (2019) What is a research framework and why do we need one? [Online]. Available from: <https://uxdesign.cc/what-is-a-research-framework-and-why-do-we-need-one-b3fac8351d46> [Accessed November 15, 2022].

The University of Sheffield (No Date) Writing a research proposal [Online]. Available from: <https://sheffield.ac.uk/socstudies/phd/research-proposal> [Accessed November 15, 2022].

Wobbrock, J O. (No Date) Catchy Titles Are Good: But Avoid Being Cute. The Information School - DUB Group, University of Washington [PDF] Available from: <https://faculty.washington.edu/wobbrock/pubs/Wobbrock-2015.pdf> [Accessed November 15, 2022].