Something that often goes unnoticed (consciously, that is) to the average viewer of movies is the cinematography. This probably holds 10 times the truth in video games. With all the other things happening in any given sequence, its usually the furthest thing from a player's mind while they are playing any given game. However, in such a narratively driven game like ours, it is a critical element and a big part of the production pipeline.
A little background on myself. I was educated in 3D animation as a generalist and minored in cinema studies as well. Though I had produced and created an entire short by myself and have helped on many live action sets, cinematography was never something I had tackled head on. In fact, probably one of the weakest components of my aforementioned short film was the camera work. Upon joining iNK Stories though, this would be my sole role.
Learning On The Job
At the beginning of the process, I was all sorts of rough around the edges. Though I was familiar with composition and the principles/language of cinematography, I had never had to consider them and work with it on this level. Luckily, there was a clear set of rules from the start set out by Navid and the team as to how this game/story would be shot. With these constraints and my prior knowledge, I started to block out the cameras for the sequence that is now known as “Chapter 8: Just a Knock.”
(My First Camera Scene)
At this point in the process, production was well underway but environments and characters were still not set in stone. I was setting up the cameras using the animation from the motion capture placed in generic generic Mo-Cap rigs and blocked in environments. We began a process of experimentation to get the cameras set up correctly and eye-balled based on where environments would be refined in the future.
As time went on, I was becoming increasingly comfortable with the process and the Graph Editor (a representation of the keyframes on animated objects, shown as animation curves - this allows you to manipulate the curves to smooth out and better control the motion of your cameras) but we soon came up against our first big obstacle. We had already sent off several scenes of cameras and were finally starting to see them implemented into the game, but something was off about what we were seeing. The framing didn’t match what I worked endlessly to get right and the characters were constantly getting cut off at the top of the frame. We realized that the Mocap rigs I was framing off of were set to a slightly different scale than our character models’ height. At this point we had “shot” maybe about a third of the game but of course it happened to be the meatiest third as we had wanted to start with the longer scenes. This meant we had to adjust all of those cameras again for the new (and correct) height of our characters.
(Side By Side Comparison)
A Second Wind
We had put a lot of time into the work of making these cameras initially and knowing how much tedium it would take to fix all of this manually, we set out to fix this issue in a procedural manner (you’ve gotta let the computer work for you now and then). We tried all sorts of solutions both in Maya as well as Unity but it just never looked quite right. It started to become clear that it wasn’t just the height difference in the models but the overall mass of the two rigs that made the framing no longer work. This meant I would have to adjust each scene, camera by camera, for the framing to be right.
(The Graph Editor)
This actually turned out to be a blessing in disguise. Sure, there was a good amount of time lost, but this allowed me to take all that I’d learned up to this point and apply it to the solid blue-print we already had, making the work that much better. I had a professor in college who used to always say that you would always work better and faster on the second time around after a crash. This rang extremely true in this process as I think all of the shots were improved upon.
As we started finishing more and more scenes, our work flow got tighter and more refined. I would lay out all of the cameras and animate them for the entirety of a scene, getting as much coverage as possible. This was a more broad stroke approach, trying to capture the action and emotion of the scene from all of the characters’ perspectives and getting a range of shots, from close ups to wides. I would then playblast the footage (sort of a quick screen captured video that doesn’t require the full rendering process) and give it to Sam who did the editing for all of the cinematics. From there we would work on the edit and get it as tight as possible at that point. During this process, we were able to see what was working and what shots we still needed to get to make the scene come alive. This back and forth became essential to streamlining the cinematic process of the game.
Something From Nothing
Up until this point we were working on scenes that were fairly 1:1 with the motion captured animation, meaning what was shot on the floor of the mo-cap was exactly what played out in that scene. If there were 3 characters interacting in the volume then there were 3 characters in that scene in closed rooms or environments by themselves. Now it was time to put together some of our larger scale scenes that had a lot of overlap and different functioning elements. The two main scenes of this stage were what went on to become Chapters 5 and 16, two pivotal moments in the game. These were scenes where groups of characters were motion captured separately and where we really started to fill out the scene with FBX files that held all sorts of generic animation and animation of specific “non-primary” actions. This ranged from crowd members cheering and picketing to soldiers marching and managing the large crowds.
Now it wasn’t just about the camera work but about the staging of scenes as well and how these elements played off of each other. If the graph editor was my best friend up until now, then the dope sheet would fulfill that role in this phase. The dope sheet is a holdover from traditional animation that was used to block out timing of action and dialogue prior to animating. I would be using it here to move around the animation information obtained from the Motion capture so that character's actions aligned correctly. I would often have as many as 10 or so Mo capped rigs in a scene and having to move around all of their key frames, often testing the power of my computer.
(The Dope Sheet on a Long Set of Animation)
This process was truly tested when we started working on Chapter 11. This scene had a whole slew of problems working against it. For starters, this scene had more characters interacting with each other directly than any other scene, so much so that we weren’t able to capture all of the actors together for this scene. Some of the actors had to play several characters and had to keep switching positions and roles throughout. On top of that, the data was slightly corrupted for this scene so there were weird gaps mid action for certain sections. All of these factors made this one of the hardest scenes to piece together. I also had to import another character that had to be in the scene for continuity's sake. On an initial pass when we received the in game version, the cameras and timing didn’t match up to what we had set up. It was extremely difficult to get it to work the same way in Maya and Unity with the approach we were taking. We went back and forth with the developers trying to get the scene working as we intended with all of the shots framed correctly.
(An Top-down view of Chapter 11)
We ultimately had to take each of the characters with the baked animation and individually place them into the scene and redo the cameras once more. I tracked everything I did in a chart with the keyframes and positions of the characters so it could be recreated correctly in unity. Once we got the setup correct, this extra step was necessary to account for the areas with corrupted data. This scene was made difficult enough with the technical issues but was also a challenge for the cinematography as well. The amount of characters and the accusatory tone combined with how tight of a space they created made this an artistic challenge as well. We finally got the scene working correctly and sat with the developers to polish the problem areas of the scene. Interestingly, I feel this turned out to be one of the better flowing scenes in the game - no small feat considering all of the issues.
(A Shot From Chapter 11)
Looking back on this process, which took more than five months to complete cameras from start to finish, I learned a great deal – not just about the technical process of cinematography in Maya/Unity, but the art of cinematography itself. The camera has the ability to tell story as much as dialogue or action, the framing of a shot can make you feel happy, scared, sad, emboldened, and more. The world of virtual cameras lets you do things you could never dream possible in traditional physical media, but that doesn’t always mean you should do it. By sticking to a defined, period accurate rule-set, and making a game that had its roots in period filmmaking, we were able to make the cameras themselves an asset in the construction of the game. Moving forward, I’m excited to push the boundaries on what’s possible through cinematography and jump into the unknown again.
You can find 1979 Revolution: Black Friday on Steam and the Apple AppStore.