The first two weeks of April 2016 saw Andrew Sage (author) and Xanthe Bodington (illustrator) mount an exhibition in Aberdeen, Scotland based around their graphic novel, The Price. When planning the exhibition, they wanted to make it more than just pictures on a wall. This raised the questions:
How do you bring the static panels of a graphic novel to life?
How do you let the reader step into the main character's shoes and feel what they are feeing?
This article tells the story of how they set out to answer these questions.
Initially we experimented with making some of the story artwork panels interactive using Bare Conductive's Touch Board and Electric Paint. Despite having some success with these experiments we didn't feel it brought anything to the experience. It would very much have been a case of tech for tech's sake. Not wasted though as experience gained and something that would be of use in a future project.
The next experiment was to use Virtual Reality to bring an element of the story to life. Part of The Price deals with the main character's sense of being overwhelmed by the unwanted attention of fans and paparazzi. VR could give us the opportunity to place the viewer in a scene where they were surrounded by an ever-increasing number of people.
Using Unity's Target Gallery VR sample project as a starting point a project began to take shape. The sample code was stripped down and rebuilt to create avatars appearing in the distance and moving towards the viewer. Upon reaching the viewer they would start to move away to the back of the scene before once again advancing on the viewer. Over the duration of the experience the number of avatars kept increasing.
To give the viewer the impression they could maybe halt the onslaught of the crowd a virtual hand was added to the scene.
A virtual reality does not have to match this reality. One of the joys of VR is that worlds beyond imagination can be created. Using 3D avatars to harass the viewer was briefly considered but that would not have taken full advantage of Xanthe's artistic style. Using cut outs of figures from one of the graphic novel's panels, billboards were created to represent the avatars. As the viewer's position was fixed with the only movement being their head position, the 2D cut-outs approaching their point of view worked very well.
Just for some set dressing a couple of 3D bars were added to the scene by sticking some flat artwork panels on to some cubes.
Visuals in a virtual world is the key factor in enabling the viewer to immerse themselves. However, vision is not the only sense that can be tricked into believing we are experiencing something we are not. Sound can greatly enhance the experience. Xanthe recorded herself screaming out like an obsessed fan. This audio was added to each of the avatars and created a nightmarish soundtrack as they all overlapped and echoed around the 3D environment.
The initial development was done using an Oculus Rift DK2. Although a convenient headset during development it is not the easiest to use when demonstrating to the public. Luckily, just prior to the exhibition's opening, the Samsung S7 Edge was released and one arrived with the Samsung Gear VR headset. This allowed a much more lightweight and portable viewing experience. The downside was that spectators couldn't see what the wearer was seeing.
Did it work?
Watching a group of people experience VR for the first time is an interesting spectacle. As the first person tries it out the others all look on and laugh at the wearer moving like a robot having a breakdown as they get to grips with the VR and their hands wave around even though they can't see them. Then as each person tries it themselves they forget how stupid everyone else looked as they too do the same robot breakdown dance. If they are trying out the tethered Oculus Rift DK2 then at least everyone else can see those invisible flies the wearer appears to be fighting off. The tethered nature also stops the wearer from wandering off. However, if they are trying it out wearing the Samsung Gear VR they have no wires to keep them in place and some do end up wandering around the room, to the amusement of onlookers.
So, did it work? Judging by the reactions of some of the people who stepped into the scene, it would be safe to say yes it did. Seeing the wearer flailing about with their hands trying to push away the onslaught of people shows just how immersed they became. Having someone rip the headset off before the experience was completed because it was getting too much was also a sign of success.
From the illustrator's perspective
Xanthe was asked what it felt like to meet her drawings life size face to face in VR:
It felt very exciting as it was something I had never experienced before. It made me think of my 2D illustrations in another way as I started to see them as more dynamic and interactive and how can I push that further in my work. The VR experience with my own illustrations brought them to life taking the reader from the pages of the novel to the world of the main character and becoming truly immersive.
Bringing stories to life
The VR experience created to allow the viewer to experience some of the emotions of The Price shows how VR can be used to bring a story to life without the need to create a photorealistic environment. Targeted immersive scenes like these can really enhance a project originating in another medium. We look forward to future collaborations to help turn peoples' imaginations into experiences. If you would like us to work with you then please get in touch.