January 29th-February 4th, 2018:

This week, I had to turn in the Explainer Video on Tuesday, and then continue my research for our four week project on Thursday. Maria gave me some helpful feedback on where I needed to take my research and how I should document it (write down what it means to me, do more research on who their influences were, how does this pertain specifically to my role in the project, etc.)

From there I got to work on the basic foundations of motion capture and why this would be a good medium to work with to tell this story.

To me it’s obvious I want to use this medium, because it is the one I am most knowledgeable with, and where I want my future occupation to be. But as for the medium with the story, I think it will help show the movement of the story, just like film does, but from real people, not idolized ones that we aspire to be. I can direct that motion too, to define the story, giving me experience in direction and design.

I needed to focus our idea to a more manageable goals, such as the ones Abby addressed:

  • Recreating scenes from these books and placing students in a position to explore at their own pace

  • Interacting with elements through the scene for further information and clarity,

  • Provide another medium for students to understand their readings

After thinking about these and discussing them with Abby, we are doing our research for the week and then from there deciding which interactive parts we want and which scene we want to focus on.


My approach had me thinking over:

  • How do I want to show my research?

  • Articles?

  • Videos?

  • Personal discussions?

  • Documentation?

  • What steps do I need to do to start research first?

  • Motion Capture (as a whole)?

  • VR (as a whole)?

  • What other projects are out there like ours?

  • What have they found in their research?

  • What is our main factor? Emotions? Immersiveness? (Big picture)

  • What is it we are wanting to produce in four weeks?

  • Individually?

  • Together?

Moving forward with the project, Maria suggested not making timelines so linear but more continuously growing so that research is involved in every aspect. There isn’t a point where I should just get all of my information from articles or videos and then stop. I must continue to grow with the project, aiding in it’s outcome.


Given that Abby is my partner on the project, she and I have been working together to figure out our specific needs per each portion of our project, both as a pair and as individuals. We will be moving forward to figure out our next steps, starting with live stream into VR and then capturing data for the project. 

This week we met with Lakshika to figure out the one to one process of MotionBuilder into Unity for our software needs, and then from that session, reached out to the performing arts department to get some actors to help out with the scene.


When/Where: My time researching this was mainly in class, but I spent most of my time at ACCAD doing so, along with time during the weekends.


I sought out more visual media this week rather than simply articles, because if I want to tell a story through visual learning, why not learn visually?

I’ve attached articles, videos, and interesting websites dealing with the mediums we’ve chosen to use to give a little light on to where our heads are at and what they are exploring.

More notes will be posted (later this week) to explain my thought process on each of them.


From here I would like to continue to do research on immersiveness within virtual reality and how experiencing a (mocap) performance within it can change the way people perceive stories. This is going to take work in reading more articles that deal with how kids learn, why they are learning what they are, if immersiveness is more likely to work on them as a target audience or not, along with many other technology and reaction related questions. Also this requires Abby and I to go out and discuss things with groups involved in whatever children’s story we choose along with getting permissions and grants to help grow this area of study.

I also want to gain more technical knowledge within Unity and streaming into it for motion capture and virtual reality. Those are the two primary mediums being used for the Ruby Bridges project we want to do, so I need to understand the software more to get it working in our prototype.

Also, I am anxious to hear back from those in the Theater department to see what their take on the story is, and document our process of learning how to direct their skills.


What was the most difficult thing about the work this week?

Narrowing down a specific area and thinking about these four weeks rather than the whole outcome is hard for me. I like to work on big scale, not step by step. So coming back down to reality was challenging.

What was the most satisfying thing about the work this week?

Working with Lakshika and Abby to see how our two mediums would work together was fascinating. It opened up the doors to what we can do with this project and direct our actors in real time.

Here Leap Motion Orion works with the HTCVive in order to use your hands as the controllers. This is something Abby and I are looking into as a possible choice for our project.

I found this real-time motion capture with a virtual human turned video game, giving inspiration to projects like ours. Although they consist of a game and different fantasy-like topics, it still shows relevance to what we are doing and how creative thinking on this subject is already out there.

Here, famous motion capture actor, Andy Serkis describes the breakdown of how motion capture works in a production film, focusing on his knowledge of performance over the years. This helps in terms of what actors need in terms of environment, direction, and reactions to acted out scenarios.

This explanation of BEYOND: Two Souls, the actors describe how performance capture for a video game is different to traditional movie acting. This relates to my side of how to direct the actors in the scene rather than just for a live audience. We want the user to experience it in real-time rather than taking a story experience for a brief moment.

Actually, this is a mixed reality experience, introduced to our class by Maria, showcasing 3D interactive visuals of an 'immersive digital playground'. This shows the collaboration of immersion and interaction - a key concept in our upcoming project.

Recent Posts

See All

October 7th-11th, 2019:

I'm getting terrible at consistently doing a post of my work! But I am going to try and really focus on it from here on out (insert all of you reading this rolling your eyes right about now). For the