The Ark
Updated a year ago
A violinist and the Nephalem in a sunken city in 2050.
Final submission video of The Ark:
Creative Process Overview: Since I come to 3D from the theatre/film world, I began with theme, narrative and character. My cousin runs Pew's Arctic program and has made me acutely aware of climate change. His world is literally melting away. I wanted to create a piece in which the future reflected the breakdown of the Gulf Stream. In addition, even though the British side of my family landed in New Amsterdam in 1652, we still view ourselves as immigrants. Quakers. Refugees from a despotic system. The current political climate is as disturbing as the health of the planet. That said, I spent an inordinate amount of time doing point-to-point animation of the violinist... down to her fingering and frown... to the music score.
Cinemachine/Timeline made it possible to realize a story in Unity that flowed beat-to-beat. Once I grew more comfortable with Cinemachine, I created a shot list and then composed the shots to reveal the narrative. In turn, the shots had to be orchestrated with the score. The rhythms of the music dictated cuts. So I danced between Timeline/Cinemachine and Koreographer (text triggers placed on the audio track)...and attached the Post Processing Stack to the main camera.
List of Assets Used: The Wasteland Package (Asset Store) Uber shaders (Asset Store) Amplify Color (Asset Store) Volumetric Fog (Asset Store) Koreographer (Asset Store) AVPro Video (Asset Store) Ocean - community shader Speed Tree - Haunted Tree Body Bags - Purchased DIY Windmills - Violinist with Landfill Violin - The Coracle - Landfill Drum Set - (made but not used) Landfill Cello - (made but not used) As per the characters, I had to retopologize the base meshes from Fuse in Maya to then take advantage of Mixamo's full body/face rigs. Fuse doesn't offer ready-made kids. Then I had to rig the violin to the Maya Auto Control rig - which meant having to bypass their FBX export system - and using a generic rig in Unity to get the violinist animation to work properly. Somehow, through all the work-arounds, the violinist rig and animation worked smoothly in Unity.
WIP17 - Editing film with Cinemachine. Would prefer to have control over alpha of layered shots. May have to use AVPro video transparency to layer a shot with semi-transparency over another shot. Then enable/disable video playback in the Timeline to control use of transparent shot. Trying to figure out how to do this with some finesse!
Still have to bring in one more vertex animation. Using Megafiers to import point cache data into Unity.
WIP16 - Storyboarding shots, staging, filling holes in narrative.
One child is ill, another watches over him in the shelter of an old cargo container... while the violinist summons the Nephalem for aid. Establish the dramatic situation from the get-go.
Adding cargo containers to rooftops and signs of a city that's been abandoned by the military to the rising tide. Will put a corroded tank on a roof.
Created large banners (cloth) that warn refugees - the boat people - to leave.
Again, using theatrical staging technigues to steamline what is absolutely necessary (re game objects) to reveal the beat-to-beat narrative.
WIP15 - There's an old maxim in the theatre: "Less is more."
I've been running the XR Lab in partnership with the NYU Game Center for the past few years. Last autumn, we had a few CUNY TV people in the mix. The Director of Special Projects spent an afternoon using Unity to create and animate the opening of her show (she's an old theatre chum and took to Unity and VR like a fly to a glue stick). By using Unity, she saved a huge amount of time and money, not to mention her thrill at doing the show's opening without having to tap another creative.
The point is: this game engine has a utility far beyond "games" and can reach a creative community that includes theatre and film people. Cinemachine is mind-blowing - the next leap. I took a step back and realized... wait a minute... I'm trying to cram too many game objects in this scene... and it has nothing to do with telling the story. "Less is more." An overhead shot of the coracle (ancient round boat) with body bags... simply floating... is powerful. And enough.
WIP14 - Using Koreographer to trigger events from the music track. Will see how it works with Timeline and Cinemachine.
WIP13 - The Nephalem arrive via movie textures on building walls. Played with transparency on the material which affects the movie texture and gives it a more ghostly quality. Need to avoid any more point-to-point animation at this point. I'll script the transition from the regular wall material to the movie texture material so that parts of the buildings come in randomly... but in sync with the music score and in progression until the whole city is a mass of pulsing wings.
Note: May not need transparency. The buildings with movie textures at full alpha are more striking. Experimenting to achieve the right balance and effect.
WIP12 - Adding trees and working on the city.

WIP11 - Took far too long to animate the violinist to a 2.5 minute score that I may be off point with regard to this competition. Just figuring out rigging constraints for an instrument is enough to throw someone off track. A one-line tip on an obscure BBS gave me an inkling on how to rig the violin. There's no time to complete animations for the other characters. But will continue to forge on. This gesture is the seed of a larger piece that won't be fully realized until after the deadline.

Contemplating scale as well. A tiny violinist is getting buried in this cityscape. After getting sidetracked on the minutiae of animation, I'll take a stab at scoring changes in the environment according to the (royalty-free!) music.
Thinking of going abstract on appearance of the fallen angels - that is, using either animated light cookies or movie textures to capture the movement of wings. Because the violinist is so concrete - real - the abstraction of the angels may serve as a visual counterpoint.
WIP10 - Still working on animations of the Ark kids in Maya.
Finished modeling and texturing the instruments.
WIP9 - Since I'm using the Maya Auto Control Rig script, the integration of custom facial rigs with Mixamo body rigs is "knotty" to say the least. Instead I retopologized the children of ark in Maya and then imported them as custom characters into Fuse. Then, I auto-rigged them with facial blendshapes. Now that the face and body rigs are integrated, I can animate the main characters. Just tested the rigs with idle animations in Unity as well as tested the face/body rigs in Maya.
WIP8 - Completing custom facial rigs for the main characters - these joint-based rigs work well in Unity. Time-consuming and tedious, but determined to use Cinemachine for close-up shots of facial expressions.
WIP7 - Albino boy joins the Asian girl and African boy with a cello made from refuse (inspired by the Landfill Harmonic) - barrels and cans from Asset Store packs being used in other projects. Shrank a Fuse body to create the child's shape and morphed a free child's head off the web... and voila, Albino boy. Rigging the faces of the kids (on top of body rigs) because Cinemachine enables close-ups. A single shot of an expressive face can make or break a piece... as seen in Unity's Adam videos. We empathize with the robot because we can see his eyes - still human - embedded in machine and circuitry.
Opting for point2point animation with the cello. So I may be reclusive and underwater massaging keyframes for a few days. Once I finish animating the main characters to the music, I'll loop back to environment and progress it according to story.
WIP6 - The narrative continues to evolve. Three poor kids - an Asian girl (violinist), African boy (drummer), and another child (cellist) - find each other in this sunken landscape. The music will wake the city up, light after light, a rhythmic call to action. Thus, the evolution of the environment will now take cues from the soundscape and characters, not vice versa.
Using the Maya Auto Control Rig to and add and tweak animations. And reach out to Pratt for mocap data for the cellist (to be modeled and animated).
In the meantime, I have to morph the instruments into those made from landfill refuse.
WIP5 - Modeled and animated a child playing a violin in the flooded city of the future. Will add characters and morph the instruments according to the vision of Landfill Harmonic. But sought to test the narrative vision in Unity on this round. Added the Post Processing Stack to the camera and getting up to speed on Timeline and Cinemachine.
WIP4 - Narrative discovery! What does the blind girl in the ark do to enlist the aid of the Nephalem? After doing research on how people recycle garbage in creative ways, I discovered the Landfill Harmonic:
A group in Paraguay turns the tons of trash (that people dump in their village) into instruments.
As the blind girl's ark begins to sink, the girl and her friends pull out instruments made of rubbish and begin to play. The music lures the fallen angels to their boat, casting a spell that leads to the group's rescue. Today, I will turn debris (so easily found free in the Asset Store!) and turn it into musical instruments.
WIP 3 - Will blog from bottom to top so early sketch work won't send folks to rafters. Again, I had to model the DIY windmills - one from car alternator parts and the other from snow shovels. These makeshift windmills power the flooded cities.
WIP 1: "The biblical flood narrative was derived from stories that had been embedded in Sumerian and Babylonian society and literature for thousands of years. The book revolves around a clay tablet dating from about 1800BC with 60 lines of cuneiform (the tiny, wedge-shaped script on the tablets), which relate part of the flood story." According to a tablet that's almost 4000 years old, Noah's ark was round. (
2050 AD. Yet another forlorn metropolis. Deserted. Destitute. And submerged. While the wealthy have fled to mountaintops, the poor have scrambled to create coracles (ancient round boats) from rotted wood and reeds. Asylum seekers who once sought refuge in the Western world now circle skyscrapers in search of supplies. Food. Clean clothes. Fresh water.
A blind girl withers on a boat, her parents long lost in the frantic race to stay afloat. But she sees what others can not see... the Nephalem (fallen angels) hovering on rooftops. These winged creatures watch the drift of humanity as the seas continue to rise and swallow the earth. As her flimsy boat begins to sink, she must befriend the fallen angels and persuade them to lift her boat to a rooftop. Will they heed her plea for help or not?
Day 1 was spent just playing with a sunken city and conceiving a short narrative. Sketching out the bare bones of a scene. While I'm playing with Asset Store packs, shaders and Mixamo models (attaching some wings here and there), I'm contemplating the little arks. The reconstruction of Noah's Ark as spelled out by the Sumerian tablet is shown below:
For Day 2, I'll model the boat and figure out tactics used by the blind girl to enlist the Nephalem. I'll continue to work on the underwater city-scape.
WIP2 - Custom modeled the gufas - "arks" - and placed them in the scene to test float. No free or paid assets begin to resemble these round boats, which are strange in an interesting way.
Placed proto-characters with RBs and colliders on the boats to see if they'd fall off. As long as I freeze constraints, they seem to weather rocking and sliding on water. Populating the boats and animating characters - still in the mind as to how to do it.
Honing the concept further. In contrast to sleek sci-fi, this dystopic vision presents a world made of refuse (a touch of the madness of Mad Max). Everything that's been abandoned by the mountain-top elite is being turned into devices by people who have been left to drown. Started to put rope bridges (free 3d models) across buildings. Envisioning a nest of planks and ropes that glue together passageways between skyscrapers, a web-like chaos known only to sea-level dwellers.
Next, I'm going to model some DIY windmills made from snow shovels, washing machine rotors and car alternators.
Going to place these contraptions on ledges and rooftops. They will power the neon signage to come.
Testing use of light cookies to create the overhead web of planks as well as volumetric fog.
J Dakota Powell
Founder, TimeWave / XR Lab - Artist