VR for Art’s Sake

 

Alex and my latest project is a virtual reality poem, working title “My Brother’s Keeper.” The story revolves around two brothers, and their relationship, with the viewer in the role of the younger brother. It begins at the funeral of the older brother, who was killed in combat somewhere far away. The narrative explores themes of self-understanding, heroism, memory, and time.

It is an attempt to create an emotionally transporting fictional experience in VR, along with creating a VR-native narrative structure, neither of which Alex and I have seen much of in VR yet.

We both have an interest in using virtual reality for things outside of gaming, and we’ve done a lot of experiments with things like 360 video/ VR fusion, mechanics, and even VR journalism. But everything we’d done had, in some way (besides Alex’s genre-shattering SpaceFrog game), been a commentary on the medium and what it could do, not a transporting art experience that took the medium as a given. Some beautiful art is a commentary on a medium, but in most movies, books, paintings, or photographs we’d seen that really affected us, the medium had faded away...it transported us deeply into another world or idea. It is ironic that this doesn’t often happen in VR, given that total immersion would seem to be an easy opportunity to accomplish this.

Alex and I both write a lot of poetry and short stories (Alex has also written an excellent YA novel), and we are great lovers of literature. So we decided to start with the medium of poetry. Poetry affords a particular kind of freedom of thought and flexibility of composition that we were interested in leveraging. We spent a long time talking about the narrative and the world around it. Both of us grew up in families with two other brothers so we drew a lot from our childhood memories (although neither of our experiences were anything like what appear in this experience). We had also been having a lot of discussions about heroism and American identity, so we knew this would come through as well.

We also wanted a form of narrative that would leverage the potential of VR. When a new medium emerges, the first attempts to use it are almost always just replications of a previous medium. Early uses of film were just “moving pictures,” and they looked like that--steady, repetitive shots of a single viewpoint. Early cars were “horseless carriages,” and looked the part. VR has done basically the same thing, porting linear narratives into the medium, so we have “VR movies” and “VR video games.” Creating something native to VR at a conceptual level meant developing a non linear narrative...specifically, a three dimensional narrative.

Screen Shot 2018-04-02 at 3.58.30 PM.png

We constructed the narrative with three axes: symbolic, chronological, and emotional. This formed a cube with many intersections. Since we already knew the key events in the brothers’ lives and emotions we were trying to evoke, we discussed at length recurring symbols we could use to represent these ideas. This gave us many different intersections of the three axies that we used as guides when writing the prose poems that would populate the space.

 

In our initial plans there were 6 time periods, 4 key symbols, and two core emotional tones, which would make 48 individual intersections. Ultimately we didn’t actually procedurally create a poem or short prose piece for each of those intersections, but they served as guides, and we have been mapping them along this as we create them using a table like this:

Screen Shot 2018-04-02 at 3.58.45 PM.png


 

We have tried to be careful about length for the poems, since they will need to be read in VR:

Screen Shot 2018-04-02 at 3.58.52 PM.png

 

The protagonist, embodying the younger brother, will navigate through space via walking and teleportation (although we are searching for a new navigation tool...teleportation is kind of the lesser of evils in terms of navigating a space at this point). No one will navigate chronologically through the spaces...they will navigate back and forth through time, triggered by the different symbols. The connective tissue is not chronology, however...it is the connection of the symbols and content. Sometimes when people reappear in a scene, the narratives will have changed. This replicates the way that we actually navigate identity and memory...relationally and iteratively.

The idea is that the person, along with getting to know and exploring the house in which the narrative takes place, will also “wander around” within the narrative, building an increasingly clear picture of what is happening the longer that they spend in there. The idea itself is three dimensional, and along with wandering around in the three dimensional space, the protagonist is wandering around a three dimensional idea...a sculptural concept.

“Sticking the landing” on the idea will involve doing some of the banal details really well. How the text appears to people, and what size, will make or break the experience for people. Also, how and when people end the experience will be important...we want to give people “permission to exit” and avoid an “infinite scroll” kind of fatigue-based ending. The navigation between scenes will be important too. We want these mechanics to be invisible and so simple that someone using VR for the first time could flawlessly navigate them. And as any designer knows, making a simple mechanic invisible and seamless is much harder than making something creative and obvious.

The cutting edge of virtual reality is not the latest technology or haptics, it is the new emotional experiences that we can create for people that were not before possible. We can create incredible immersive experiences with simple a paintbrush, a canvas, and paint. The point of strapping a screen to our face and connecting it to a 2000 dollar supercomputer must only to be to achieve something that goes beyond this, that transports us in a way that way simply not possible before. If we can’t achieve this, then the project of virtual reality itself is pointless, and we would be better off sticking to painting and movies. Prognosticators are right that technology will continue to advance, but not at the speed of Moore’s Law or the speed of light. Technology will fly forward at the speed of art.

Crafting a literary VR game part 4) Room layout + atmosphere testing!

Crafting a literary VR game, part 1: Concepting + Story Design

Crafting a literary VR game, part 2) from Concept to Narrative Timeline to Script to Set Design!

Crafting a literary VR game, part 3) Nailing down the story / Design document drafting!

Greetings world!

When last we checked in, I'd been working on the story for this crazy game involving inter-dimensional gods who eat happiness, 70's and 80's retro decor, and a grizzled Midwestern cop who's forced to investigate the disappearance of his estranged father.

Next on my design/development schedule for this game was supposed to be mocking up all the interactions with code. But I was having trouble persuading myself to get back into development, so I decided to take a few hours to focus on something simpler than prototyping the player interactions. And something a bit more immediately rewarding: environment design.

So I went on Sketchfab and the Unity Asset Store to find models to populate the estranged father's apartment. This is about a lot more than just decorating though: environment design is about finding visual elements and creating a space that heightens immersion, resonates back the game's primary themes and tones, and supports way-finding (navigation) and the player journey.

You can see, first of all, the color palette of muted greens, yellows, and blues, with a flair of orange/red for the kitchen for contrast.

Through this design experience, I encountered a few challenges / learned some new things:

1) The estranged father (who went missing) was an emotionally remote and repressed man for most of his life, but in the past couple months went through a dramatic personal blossoming. Figuring out how to reflect that in his space is challenging. I'll have to add some elements that are in blatant contrast with each-other, yet are still in keeping with the overall ethos of the game.

2) Almost any color, when used throughout a scene, has the effect of being depressing. Color tone variation alone is enough to spice up the composition to help the player feel that the environment is energized, rather than subdued.

3) It sounds  stupid, but homes have a lot of things in them. That's a huge opportunity for storytelling, and a major challenge / timesink as well. Sure, you can always place cabinets with closed doors, but the space will still feel empty.

NEXT UP: Interaction prototyping for the main scene!

Crafting a literary VR game, part 3) Nailing down the story, Design Document drafting

Crafting a literary VR game, part 1: Concepting + Story Design

Crafting a literary VR game, part 2) from Concept to Narrative Timeline to Script to Set Design!

Hello strange internet people! I've been in Denmark for consulting work this past week, so I haven't been able to devote sufficient cognitive cycles to this project. Accordingly, I've shifted the project timeline to have a working prototype I can test with by End of November, and then finish the game before New Years.

Here's the most significant accomplishment from the past week: building a high-level design document that integrates both narrative and interaction. It explains what the player can do, how to solve the mystery-- the entire narrative and interaction arc for the game. Check it out, parse through it, and if you're curious you can read through the rest of the blog post explaining how I got there. (Click on the image below for a zoomable, legible version). 

From a high level, the game begins with a brief, linear experience, then branches out into non-linear exploration of the player's vanished father's apartment, then returns to a linear climax where the player is presented with a simple task, which, depending on their performance, ends in either a horrific loss state or victory.

 

So how did we get here, and what does this design document DO?

So far, this November, we began with a challenge (crafting a literary-feeling VR game) and, through research and the design process, wound up with a game concept centering around exploring the apartment of your estranged father, who mysteriously went missing. The game now has a strong Stranger Things vibe, crossed with K-Pax and Big Fish.

This past week I nailed down the story. To do that, I first needed to answer some big questions. The biggest of which, and really the key question, had to do with the element of Mystery. How can the protagonist (named James) discover what happened to his disappeared Father? Solving this mystery involves discovering and synthesizing clues into a conclusive timeline of events (the solution to the mystery). 

Clues in the apartment can be divided into 2 categories: Clues from before the event, and clues from during / after the event. Clues from before the event give us background information on the people involved. Clues from during/after the event would be, for instance, a chair knocked over, spilled blood, that kind of thing. If all the clues for the player to find in the game are 'before the event' clues, then why didn't the protagonist's father piece together that something was going to happen to him? If all the clues are from during/after the event, then there is no significance to characters' backstories.

To answer the remaining plot questions, figure out this balance of before/during/after clues, and wrap the interactions around those, I came up with and answered a list of questions.

QUESTIONS:

  1. How does player's father piece it together that the Axixu might have malevolent intentions?

    1. He didn’t-- he misread the clues, because of his trustingness. He thought the Axixu would take him somewhere wonderful. His naivety is a character flaw stemming from weakness, and his desparation to find a treatment for his PTSD.

    2. It was only because of the obituary of the Crazy dude who disappeared from same apartment years ago (obituary appeared in newspaper 2 days previous) that struck player's father as queer. He confronted the Axixu about it, and they dragged him away. There is evidence of a struggle in the apartment, and odd claw marks on the walls...

    3. Obituary has drawing of ‘gods eat happiness’?

  2. Why should James want to go through the portal?

    1. Three days ago the Axixu visited and told player's fatherhe had nothing new to learn, were going to show him their world, where he would stay for a few days, and he would be the guest of honor at a feast. player's father excited. (this is in journal) (this shows James that player's father is still alive)

  3. What did player's father and Linda believe?

    1. He did tell Linda- Linda calls James to tell him about her most recent conversation with player's father.

    2. He told her he was going on a vacation with the Axixu. Linda warned him not to go, that she never fully trusted them. Linda explains the Axixu wear masks with no eyeholes, says how player's father was so excited to help her with her problems, the Axixu sat across from them in the living room and spoke with a soft, gentle voice. (creation of narrative) They tell you beautiful things.. Things that make you wise, that change the way you see the world…

  4. Did player's father know about the portal?

    1. player's father never pieced together how the Axixu entered the apartment. He didn’t realize that they had to come in through a portal. He thought they just magically appeared. The Axixu were worried that if he found out, he would destroy the portal. This, James deduces, when he learns there is a portal, is further proof they were conniving and evil.

  5. How/why can James piece together something his Dad didn’t?

    1. The fact that the crazy dude (obituary) was terrified of showers indicates where the portal is/was, a fact that player's father never pieced together.

    2. The fact that player's father went missing confirms Fa’s fears that player's fatherwas too uncertain to act upon.

    3. player's father mentions that the Axixu showed up for the first time after he cried in the shower, audio recording. The axixu said it was because, as a god, it sensed Fa’s suffering from afar. Further, there are scattered thing in the bathroom, proof he was dragged in there.

    4. James was always skeptical of religions, and never took people at face value, whereas player's father was always trusting and into organized religion.

    5. The struggle and mess that player's father left behind is a confirmation

    6. player's father, having been kidnapped, is stuck in the between world, and can kind of communicate with James. This new information- of what it’s like where he is now-- is information that he completely didn’t have before. However, it’s very hard to piece this barely audible communications together. It’s not clear who or what is communicating.

      1. player's father tells James there is a portal (says so through the TV), says he was dragged to it, didn't see where it was, but it was wet.

    7. The Axixu left behind some clues that James must parse and figure out.

      1. A mask player's fatherhad knocked off as he was being dragged

      2. player's fatherwas reading a book about Mayan temples, but the knocked over chair indicates he got dragged away part-way through. The player can turn the page, but only to the next page, that has a codex for translating the runes from the man's obituary, which would have helped player's father piece together the Axixu's true intentions!

    8. Who IS player's father, and how is his story told? How does it wrap into the narrative?

      1. player's father was an emotionally distraught veteran who ignored his family and wallowed in self-pity. He turned in every direction possible to find salvation except within. He just never had the strength to see that he was really in control. He bought into organized religion, grasping for a treatment to his PTSD; he turned to cults, and quack medicine, and all that stuff. He was thusly naturally susceptible to being seduced by the Axixu. However, as he got wiser, he became more suspicious...

      2. The game contains 2 kinds of interactions: Narrative and Progressive (and the 2 often combine in single interactions). Narrative are more for world-building, telling the tale, helping to situate and build an emotional relationship with the characters. Progressive are more about moving the story forward through plot reveal and actions.

      3. player's father was a war photographer, and after the war, he worked for a nature publication.

      4. As James is investigating the disappearance, he's also coming to terms with his anger at his father. Each interaction moves in the direction of redemption and reconciliation for both of them.

    9. How does the emotional reconciliation play into the story?

      1. Maybe James doesn't feel comfortable looking into Fa's diaries and such until he reconnects with him? Maybe his diary is kept locked, and the hint for the key is something like “favorite vacation spot when the kids were young”, pointing to the vacation photo.

         

Next up: technical prototyping!