VR vs. Visual Style

There are generally two ways when creating an interactive VR project. An expensive but quicker method is to select and purchase assets from a 3D asset supplier to build a suitable virtual environment that can convey the story effectively. Second is to custom build our own models using 3D software, such as, Blender or Maya.

When selecting the right assets for my environment, I should keep in mind that assets made up the environment, environment delivers the experience, and that embodies the story as a virtual form. The environment becomes the virtual medium of the story. The virtual environment develops the story. It transmits a unique mood to the users (like each film has their own visual style), determines their responses, interactions, and action sequences.

In Dan’s Solo, due to the technical team’s resources and availability, our decision was made to build our own assets. This only kept the project’s originality, but also provided a certain degree of artistic freedom. While the technical team delivered the first draft of Dan’s virtual room on August 3rd 2016. The first draft presents a modern and trendy style of a couple’s bedroom. While it is an aesthetically pleasing VR environment for participants to interact, but it is not driven by Dan’s Solo’s project content. There are some key points that this environment needs to be adjusted. When the storytelling medium becomes virtual space, the environment delivers to participants a sense of “presence”, that is, its visual language. Dan’s story is expected to provide each participant an experience of standing in another person’s shoes in order to sense the power of privacy and enactment from an individual to the protagonist. To conclude, I proposed a major stylish change to a low-poly, private, and intimate virtual room.

First Draft of Dan’s Virtual Room

Second Draft of Dan’s Virtual Room

Character Model: Dan and His Husband (Claude)

PIVOT! Here Comes the Technological Difference

Frist week of July 2016, I travelled from Beijing to Henan Province in China, where Mirage Gaming Studio locates. I met the with my collaborative partner and started to produce this interactive project.

For an independent filmmaker, it is the first time for me to work with a studio company, and it is same for them. In a mass connected society I believe in cooperation, it creates efficiency and perhaps will deliver unexpected value. But first of all, one premise before starting any collaboration is we must have a shared vision. Luckily, we do.

By saying a shared vision doesn’t mean how much they could understand me or Dan’s personal story. A personal narrative is quite niche, and that can make the story and a first person interactive narrative enchanted. Mirage Gaming Studio is a start up company, two co-founders who are friends from high school began explore the myth of VR when DK1 just launched their first Kickstarter campaign in 2014. They’ve come a long way from making different VR content experiments to a technical-driven studio dedicated in content creations, especially VR games. Hours-long conversations proved that we are mutually complemented, I stand in the creative side and they are trying to develop a technical frontier in VR. They’ve confessed in order to make progression they have to prioritize market needs or their investor’s demand. From a technical point of view, neither have they done such a project nor tried to develop an interactive VR narrative, Dan’s Solo interests them as well.

We’ve worked on a new script that not only integrated lots of minor actions to work on users’ perceptions and actions, but also left one interactive object instead of two, which means there’s only one interactive scenario. What I learned the most from pivoting is the technical difference for VR vs. traditional screen-based narrative mediums.

Here is what I’ve found out:

1. Even though the virtual environment is 360 degrees, less informational message is better. Users tend to experience, enjoy the sense of “presence” rather than absorbing the texts or pictures which presents the narrative.
2. We know that most of the headset tracks your perceptions (Oculus, Gear, etc.), except HTC Vive that detects your movements. Still, users who are new to this technology and experience are likely to sit or stand without moving back and forth when putting on the headset. Therefore, before writing the script and explains what users would see in a virtual world, we should try to break down our script by users’ different POVs, or simply try to limit the amount of POVs.
3. Also because there is no edge for VR perception, creators should justify everything before adding any transition cut, for example, in Dan’s Solo, I add black screen to make transition from the prelude to the interactive scenario when Dan’s first person POV changes from sitting up in bed to standing in the centre of his bedroom. A black screen is gradually sweeps from the upper side to the lower side indicating that Dan falls asleep again.

Pivoting regarded as a necessary process for a new project, people from different aspects generated pros and cons resulting in a new script. We’ve selected one interactive object (the computer) to present as the only interactive scenario in this story, and therefore I derived essence from initial documentations (texts, photos, interviews, etc.) to inject in this computer interactive scenario.

VR vs. First Person POV

When we talk about developing virtual reality stories in games or films we talk about point-of-views, which in this case, instead of the mise-en-scene. There are two objectives for making Dan’s Solo.

1. Telling stories in first person POV. Utilize the uniqueness of the VR platform which enhances viewers by taking them into a more immersive and engaging experience. Allowing the possibility for each viewer to perceive the protagonist’s POV in a specific time and space. Viewers could “become” the protagonist when putting on the head-mounted device.
2. The interactive nature of virtual reality platform allows the viewers to become users by navigating the virtual/narrative space when perceiving different points of environment and receiving different experiences. This project pioneers the virtual object to reveal more information that let the audience to become users to perceive more stories and make decision. The virtual environment, which contains the interactive narrative object, becomes the new medium of storytelling.

When writing Dan’s story for the virtual reality platform, I tried to limit myself as a content creator. Of course it is an interactive personal narrative about Dan, it is also a digital documentation about his past led by a decision-making question with his future. I tried not to expose my conscious mind when writing this project, or more precisely, establishing this environment. When users immerse in the story, everything is virtual, but the story is real. The experience is not tangible but the feeling is real. We can all use the mouse like we always do in the real life, but this time, sitting in front of Dan’s desk (virtually speaking), and given the right to answer the question “Shall I go back to my normal life or stay in my fantasy world?” Perhaps it may create resonation to different users in real time.

Creating a personal narrative in first person POV:

1. I tried to ask questions in first person. Or perhaps from the plural form to single form. Let the users feel the transition from themselves to the protagonist.
2. Leave the question and discussion open-ended to every possible scenario. Think about different paths that users may choose that will reflect a different loop, or the possibility of same user going back to see different possible endings. Like a labyrinth, audiences ultimately feel the power of controlling interactive object; they become users to make any decision and look for the way out.

VR vs. Narrative – How to write a script for VR?

A script for an interactive story possesses different elements. Traditional script requires the writer to write everything happens on the screen. However, content creators for virtual reality platforms need to pay attention not only what happens in the virtual environment (how it looks), but also describe possible reactions of the user, and their action sequences (how they interact). Just like the storyboard is accompanied by a pipeline to visually explain how the user interact in real time. When the narrative is intervening with the virtual platforms, one thing to keep in mind is we can not determine users’ perceptions, in another word, we don’t know where they look at. In that case, it is equally important to add an interactive sequence accompanied with each visual section, in order to explain how and why an user could interact, also let the production team to better understand a vision.

When experimental narrative kicks in…

As my last post mentioned, I regard the virtual reality stories as the stories that have 360 degree scopes, also because the immersive nature of this particular platform allow users to guide themselves. When I am incorporating any Dan’s photo, writings and audio messages to reveal his memories and personality as ways of documentation, I imagine those “objects” in his bedroom are interactive, informative and narratable. In another word, those interactive objects are like doors opened up more information about the protagonist. And of course, there are more story trajectories being presented.

When I am writing the script for Dan’s Solo, I regard each interactive virtual environment as an interactive photo journal of Dan. Because there are different photos, texts, sometime interview audios emerging within the environment, each photos and texts compile different narratives.  A use of parataxis could be applied in VR. Parataxis is originally used as a literature technique but also commonly seen in experimental films. Video artists used parataxis to accompany split screens. Or by narrating different stories in visual and audio tracks to mingle audience’s perception. I found much similarities between VR and experimental storytelling. Telling stories in VR could even satisfy what movie screens aren’t capable of – the possibility of multi-linear storytelling. For example, when creating two different interactive scenarios, I kept the balance between the audio and visual cues (more visual less audio vs. more audio less visual) to better tell a personal narrative.

Narrating a story in VR, or is it possible?

I could still recall the first time when I immersed myself in a VR experience. It was last year’s FIVAR (The Festival of Virtual & Augmented Reality). I came to the festival with no knowledge about virtual reality stories. LOVR produced by Immersive Ltd. was my first experience and my perception was suddenly grasped this real time rendered visual magic.

Poster of LOVR

Although I have the visual storytelling background, I have to confess that the experience given by the VR platform was unprecedented. And as far as I began to dig into such an interdisciplinary study regarding to virtual reality storytelling, I am surprised about how sophisticated it is. Not only am I referring to the technological requirement, but also the quality with those storytellers, or I should call them, innovators.

Regardless of all these changes and challenges, one thing that can’t be doubt is, no matter what technological medium that might be seen as the future trend or the social innovation, we go back to its essence, the story. Just like the gospel around Oculus Story Studio is to “deliver a compelling, immersive, evocative VR experience by telling a great story”, we believe in a simple mantra: “Story is King!”

The process of creating a VR story still seems different than the traditional. Another mantra from Story Studio, which is also as important as the first one, that is: “Presence is magic.” I strongly agree with the ability of VR that can transform you into a different world in real time where make you feel present. In that sense, a story within this virtual space isn’t just about sitting passively in our sofas engaging with the visual language, but actually we are immersed in the world. We are part of the experience, feeling empowering and connecting, maybe we are the protagonist who can interact with stories by our conscious activities.

Therefore, rather than composing the story as a visual form, I envisioned this project as a story-orientated visual experience. In the process of script writing, I changed some of the techniques to better compile all the visual information to tell the story, also let the technical team to understand my goal and vision. While I were writing for this interactive project, I proposed some questions explaining why VR narrative is different than the traditional storytelling. For example, how does the creator/director eliminate the sense of isolation among actors and audiences (in another word, break the “forth wall”)? How do we tell a personal narrative, that will make each user feel private, intimate and resonated? According to Dan’s Solo, which is a personal narrative for VR platforms. It is also a first person, real time rendered experience. Its interactive component not only lead the audience become users to interact with certain object in the environment which could narrate different story trajectory and let users make decisions as they are the protagonist. The question for me to think about before constructing this virtual environment, is that how can I possibly lead the audience, like holding their hands, and let them experience the process of standing in someone’s shoes and experiencing his/her life?

So my answer to the first question is, yes, it is possible to tell a story in VR, and I believe the immersive space will enhance the audience’s experience when they read the story. But VR stories are driven by the content, which derives specific questions regarding to the stories, and what I call it as a 360 degree of the story scope. As we producing a VR narrative, we are not creating the mise-en-scene, we are building a virtual environment, which does not give us the power to direct the audience gaze but only guide them through strategies. For me I think this may develop a trust-worthy relationship between creators and users. We give them an experience and let them make decisions. And now, we may start to think about, how to polish or reinforce their experiences which give them an more enjoyable time.

References:

Oculus Story Studio Blog

Immersive Ltd. Virtual Reality

Prelude – A New Beginning

“It is a hot summer again.” I said this to myself when I arrived in Beijing International Airport. After three-month research and development. I started the journey to produce my first virtual reality narrative.

The past three-month I have been focusing on theoretical and practical researches in virtual reality storytelling, a multi-disciplinary subject which refines stories in a newly-emerging platform. It possessed the core value of the story by deconstructing the visual syntax and reconstructing it to an interactive and immersive 360-degree screen. For a visual storyteller with experimental film background, this couldn’t be more exciting.

It all started from a simple beginning. The first encounter with Dan happened in the closing ceremony of Toronto Urban Film Festival. After a short but deep conversation with him I found this married young homosexual man is with great passion, big dream, but the reality of his life is too simple to make him fulfilled. I knew that he dated with a girl for four years in university in China. He majored in piano and always wants to become a pianist. They engaged right after they graduated from university. They ended up broke up at the end and thus he came to Toronto in search of his new life, where he met his husband Claude, a middle-aged Caucasian man from Quebec. Due to the cultural difference, this marriage was kept in secret from Dan’s parents in China, and because the identity of a newcomer to Canadian society, he is facing trouble of getting a job to support himself.

The alignment between me and Dan sparks an urge working with him. Given the fact that the virtual reality platforms alter the spectator’s role into an active user, as a visual storyteller, I would like to tell an interactive personal narrative for the VR platform.

May 29^th 2016, I met with my collaborative partner, Jie, CEO of Mirage Gaming Company in China. We brief exchange our idea about the current and future market in virtual reality between north America and Asia, we all agree that no matter what kind of technology advancement proceed in current or future market, the original IP is more valuable.

We, as the content creators for emerging platforms, have to consider both market experiences with different new technologies, but also focus on narrating compelling stories which are suitable for more immersive and engaging medium.