Murder on the YokAI Line is a transmedia immersive theatre production as my final project of MA Digital Direction in Royal College of Art, based on Narrative Mixed Reality App built in Unity with ARKit, integrated with a video art installation to make the seamless shifting from physical world to digital world, presents a story of murder that explores new forms of prejudice in a world of sentient AI. The project uses the figure of the Yokai I found in the myth painting "Erlang and his soldiers driving out animal spirits (Erlang Soushan Tu)" which is a mythical being found in Chinese and Japanese culture that can be thought of as a mischievous apparition. The project is trying to explore a new cultural mythology of artificial intelligence.
This project shows the possibility of transmedia storytelling in the near future when our life is covered by Digital contents, critically discusses post-human problems such as Transhumanism、Robot Rights、Smart Prosthetic limbs, and leaves a simple but strong question: What is the reason and truth, if a YokAI murdered a human?
Fake News: The audience is in a real space, watching virtual event
In this case, I mainly use Cinema 4D to model and Substance Designer to make the materials.
Guide (role): Give the audience an identity and throw out questions
As the transitional immersive theatre, audiences need to follow one character to enter the special storyline. This project uses Fungus to create this character as the guide leading audiences experience the story.
Video Installation (theatre): The audience is located in the gap between reality and virtual
Augmented Reality (Torch): Audiences are given the identity to participate in virtual events
This part mainly use ARKit's image anchor function.
Mixed Reality (Portal): The audience enters the virtual space and gets the answer.
There is an ARKit demonstration named Portal created by Japanese developer Kei Wakizuka. By touching screen, the portal will appear in the real space in the screen with correct perspective and space distortion effect. Through the portal, there is a digital world, but is an immersive one because the audience can also enter the world, and the real world will be behind the portal.
Now there is a variety of similar projects, and most of them are named as Dokodemo Door which is a magical tool in the Japanese manga Doraemon. In the manga, by the Dokodemo Door, Doraemon and his friends can go to anywhere.
Most Dokodemo Door projects are based on SLAM technology, for example, ARKit, the reason is if the audience use Image-Recognization-based AR technology, it is not so easy to enter into the digital world because the audience has to hold the device aiming on the image which will be a terrible experience. But I will keep my experience on both technology, there must be more than one possibility as Dokodemo Door.
My experiment is based on Vuforia and ARKit, here are two results, and you may see the difference of two design logics:
A target image, which is only necessary toImagine-Recognization-based AR, and it should be a little bit lower than the hole;
A closed 3D object with a hole (I created in C4D) which is used as the entrance for audiences, in my example, it is a cylinder. DepthMask.shader should affect this closed space, to make it as an invisibility cloak to hide other component inside;
An inner box with out top side, it will be the most important part, and may have better construction method if you wanna realize a virtual world, but which is not necessary in my example;
A ball with ShowInside.shader.
2 closed objects with holes in the same direction and with different sizes: the outside one should be a little bit larger than the inside one, as the outside one’s function is to hide the inside one by DepthMask.shader, and the skybox material should be apply on the inside one to create a scene with depth of field;
A doorframe for the hole to prevent the edge from being too shape. It can be a real doorframe with high quality texture or a black hole with dynamic shader;
A scene with several objects to create spatial sense of hierarchy.