February 5th-11th, 2018:

Updated: Mar 9, 2018

On Thursday (02/08/2018), Abby and I documented our first test of the prototype Unity scene to see if the functionality would work. She built a mock-up version of the outdoor scene we would like to use, and placed the viewer inside it, be we needed to know what mechanics worked/didn't work. Abby set it up so that teleportation, menu buttons, and pick-ups would be implemented into the scene. Using the HTCVive remote we were able to use the trigger on the back to show where we wanted to point/click, the button on the top to turn on and off the menu, and the cursor to teleport to different locations in the scene.

We also set some time aside to discuss what we wanted in this four week prototype. We did a threaded board of the bigger question for our project, and started nailing down the language/purpose for each of our explorations. After giving it some time, we will return with our own questions and try to find a way to combine our ideas and subject matter into one formulated question/statement.



ARTICLES


Below are links to some of the articles I read this week. They include my highlights and notes in the margins of parts that stood out to me as both a designer and human being. I will use this gathered material and data to discuss my own thoughts on what the articles are about, and how they influence the work that I am doing on this project.


"Action Video Game Modifies Visual Selective Attention"

(Original Link: https://www.nature.com/articles/nature01647)


With this article, I kept coming back to ask myself, with our audience, will those who have played video games before have a better understanding/response from the story or just understand the mechanics more so that others? This article was more concentrated on video game players, but I took aspects from it on thinking more about who our audience is.

- How we will gear it towards people who know the story, who don't know the story, and who've used this medium before?

- What defines attention for us?

These are things I want to continue thinking about, especially it being geared towards the other side of the interactive game design portion of the project rather than the story telling side.


"VR and Empathy: Tread Carefully"

(Original Link: http://ithrivegames.org/vr-and-empathy-tread-carefully/)


Essentially, the focus was on how to be cautious with the stories we want to tell in virtual reality - is it necessary to say? We want to include our players/audiences in the design process, but how are we thinking about their morals/ethics in these ideas? Story telling ranges from drama, comedy, horror, and love. So if we want VR to emphasize immersiveness, we need a key factor to be our audience's response.

- How can VR build practices of emotion and empathy?

- Who are my players/users?

- What do I really want my players to do?

- What does the research/science say?


"Mixed Reality Environments As Collaborative and Constructive Learning Spaces for Elementary School Children"

(Original Link: https://docs.wixstatic.com/ugd/f1daf1_cc0921899beb4d0e8ccc4d86155736f7.pdf)


Here, the message involved bringing together disciplines of arts and computer science, and how to integrate those two elements into one. Children can use mixed reality to optimize both of their primary learning skills - through their senses and through physical activity. So in our project, are we implementing the senses and physical activity? Yes, we have immersion and interactivity via mocap and VR, but as for our audience, will this be beneficial and broken down enough?


The articles states, "It is important that the learning environment is authentic and situated in a real-life situation. Learners must get an opportunity to build multiple contexts and perspectives in a social context." Because we chose a children's book that is non-fictional, the authenticity remains and is relatable, but we continuously need to be conscious of the negative impacts along with the positive ones.

- How does this pertain to us? Even when we aren't doing mixed reality, but mixed media? Are they one in the same? How does virtual reality relate to this?

- We are using a singular user application, is this benefiting or harming our story-telling?


"Collaborative and Constructive Learning of Elementary School Children in Experiential Learning Spaces Along the Virtuality Continuum"

(Original Link:

https://pdfs.semanticscholar.org/1d0b/60262ca2e6d24ba061f9066e0b6b232ed888.pdf)


This articles was a different publication of the one above (mixed reality environments), but had some extra points to discuss. Specifically, they mentioned what other mixed reality exhibits inspired their research and work. Some include the Virtual Museum Project, Nice, Gorilla World, and Agora.


I want to look into these projects and see the response not only children, but audiences of all ages had to the technological design and interactivity.


"Imagination and Creativity in Childhood"

(Original Link:

https://pdfs.semanticscholar.org/6b4d/caaad97f277f9d08c2f5bbf1c0d735eeecac.pdf)


"Our brains retain previous experiences and tries to reproduce them with altercations or mimicry. This type of learning is just responsive to what we see. But from a young age, we can learn in another way - via imagination and creativity. Not all learning is from stuff we have seen before, but can stem from 'the creation of new images or actions [...] in combinatorial behavior."


"But in actuality, imagination, as the basis of all creative activity, is an important component of absolutely all aspects of cultural life, enabling artistic, scientific, and technical creation alike."


I want to take aspects of this and tie it back into our storytelling. Reproductive activity and behavior can explain how the adults may have felt in the mob scene. But combinatorial behavior can explain how Ruby may have felt in that scenario. There is more research to be done on this subject in terms of the narrative, not so much the mechanics.


"Advances In Interaction With 3D Environments"

(Original Link: http://onlinelibrary.wiley.com/doi/10.1111/cgf.12466/full)


This article covered how there has been poor design thus far in 3D interactive environments, and provide aid in how to change these designs to become better. I enjoyed that before going into each topic, they defined what they were going to discuss in broad terms, and then went narrower into what their focus was. Jankowski and Hachet also explained their research process before presenting their findings - what they studied, how they came about it, and the evaluation of these implementation methods.


The article begins with the multiple ways of navigation via interaction.

- General movement (using hands, using controls, using arrows/mouse, etc.)

- Targeted movement (point of interest, path drawing, bookmarks, query, etc.)

- Specified movement (using precise position/orientation)

- Specified trajectory movement ( more control by maker than player)

In these areas of navigation, I learned that game-play has more to it than just point and click. It takes a lot of thought and process in the mechanics to design exactly how you want your viewer to interact with the game, the routes you want them to follow, and exactly how much control you want to give them.


It continues with object selection and manipulation.

- Tools

- Dialogue boxes

- Menus

- Pointing/pick-ups

- Touch based manipulation

Here is more of the style of choice. how are we going to afford our players the opportunity to interact and play? I learned that the further we advance in technology, the interface is changing from icons, symbols, words, and menus and even those designs have to fit into the aesthetic of the environment you're building. But these are also things that succeed by being unnoticeable.


The section that inspired me the most was 7.2, involving head and human pose interaction. It discussed, "Motion capture and analysis for more 'novel' interaction techniques based on tracking, pose estimation and understanding of human action and behavior." Motion capture and is unique way of story telling will hep shed light on human behavior from the past, present, and future.


Section 7.3, gaze-based interaction was a section I thought would suit Abby's research involving diverse ways of eye-tracking equipment. We've discussed using head direction for point interaction and teleportation, so here is a good catalyst for that research.


"Creature Interactions"

(Original Link: http://delivery.acm.org/10.1145/3080000/3072965/p360-bluff.pdf?ip=128.146.18.60&id=3072965&acc=ACTIVE%20SERVICE&key=1D8E1CA5B8D7D8DD%2ED6B9658582656CB8%2E4D4702B0C3E38B35%2E4D4702B0C3E38B35&__acm__=1518125671_79aa302e7c16c6ac23524b655784d9f2)


Mixed reality is booming and here, we discover its power with children as an audience. Maria showed us this article as one of our readings, and then I went back to it some more to discover how it will help our own project. The technology "incorporates immersive 360 stereoscopic visuals [and] interactive technology" with actor performance. Combining all of these mediums in VR for us is the goal, rather than mixed reality.


Bluff and Johnston discuss works that they studied before tackling their own project (Very Nervous System and Messa Di Voce) to better involve physical interaction, audio, and connection.


"The free-form motion tracking, physical simulations, and attraction systems allow a complex but understandable response to movement...[it] gives the audience a range of interaction aesthetics that can be explored through full-bodied, free-form interaction." This particular piece of information relates to my involvement in the motion data capture because I want an understandable movement. If it isn't understandable, then the narrative won't come through.


VIDEOS


Below are links to some relevant videos I watched this week focusing on a multitude of topics from our mediums of motion capture and virtual reality, and also the history of Ruby Bridges. This process will document my gathering of material and what my thoughts were in response to these videos.


"The African Americans: Many Rivers to Cross - Ruby Bridges Interview, School Integration - PBS"

Ruby describes her experience in a PBS interview, showing that she thought the mob wasn't a negative group, but a celebration of Mardi Gras. She states, "They didn't see a child, they saw change, on what they thought was being taken from them, they never saw a child."


This gives light into her world on not having full comprehension of the situation, but behaving enough to be strong about it and understand the fear people had.

"Civil Rights Pioneer on First-Grad Teachers: "She Showed Me Her Heart" - Where Are They Now - OWN

This is the first time in 35 years that Ruby and her 1st grade teacher, Barbara Henry, are reunited since 1960. It sheds light onto the woman who saw a child that needed an education, she didn't see color. she was doing her job, thinking she'd have classroom full of students rather than an individualized session. But, they made the best of it.

"Civil Rights - Ruby Bridges"

In this video, Ruby discusses that she was actually one of four, six-year-old girls who were placed in white schools that school year. She says, "I crossed a picket line, a mob, every day for a whole year [and] every day when I entered the building, there was a woman there to greet me, and she was also white." It's just remarkable thinking about mobs against a child, but then also the strength Ruby had, even at age six.


MOTION CAPTURE

On Sunday (02/11/2018), we are lucky enough to have three members of the performing arts department to come and do some improvisational performance work for us. We will be live streaming them into the Unity scene and having them perform based on the placement of the Ruby character. This way we can direct them into the space location wise.


I plan on having the actors act out as a multitude of characters:

- Police officer(s)

- Mob

- Mrs. Barbara Henry

- Ruby

- Mrs. Bridges

This way, we have a large amount of character to play-test data with.