When working with a team full of artists, one of their first questions when they start to work is “Where is the file?” or “Where should I save?”

For our team at TAOS, the very first challenge in the production was a big one - meeting the deadline for our Motion Capture Shoot in early March.

For our team, it was an honor to have a partnership with the Swedish studio @Goodbye Kansas, based in Stockholm, who supported us tremendously during the entire preparation process. Here you will find in detail how our team put together a pipeline to meet this challenge.

Pipeline Set Up

On TAOS, we started the project by creating a very organized structure that is built to support a USD publishing workflow and that works with a tracking system. In the future we will use Ayon and Maya USD. At the moment, it is manually set up to mimic those systems. This becomes very important when versioning and publishing so that the area for other disciplines to reference from is isolated away from the working areas. This is the difference between versioning up a file that is referencing many elements or redoing things from scratch when there are changes.

Storyboard

The storyboard team works hand in hand with the layout team to plan camera angles, content, and functionality. How does something work? How should something move in camera to keep momentum moving in the right direction? These storyboards feed into boardomatics so that we can figure out timings. This means that the early edits are part 3D previsualization and part 2D drawings.

Asset Build - Characters and Props

So, what do you do when characters and props must wait to be defined but you also need to deliver for a shoot with accuracy? There are parameters set in place that make the structures and bones consistent and viable for future use while still allowing for flexibility. For example, the drone has completely changed from an egg shape to a sleek hard surface model with vials that feel like spoilers. The layout team was able to update all shots with the drone in a day because the pivots and proportions stayed the same. Similarly, the characters we are sending to the shoot have the requirement set that the joint positions where the character will move cannot change. They can, however, change clothes, accessories, textures, and styles around these base proportions.

Environments

Similar to characters and props, the basic proportions of the set should stay the same so we can send basic locations of walls, floors, and furniture, to the mocap shoot. For example, a character can not run completely left to right if there is a narrow alleyway or hallway. They would run through the wall. On set, the closer you get to recreating the environment you want to interact with later, the more accurate mocap animation data you will receive. It will look more purposeful and physically believable if it is not random. In the next phase, these simple structures will have set dressings and more detail. What is important in planning is proportions, placement, and functionality.

Rigging

When rigging for mocap shoots, it is important to include a joint that has channels for timecode. When the layout team receives the mocap data from set, they will use the timecode to match a mocap tech edit and sync the characters and props to the action that the director wants to see in each shot. For rigging a prop like a door, the functionality and design becomes very important for capturing the correct action of the actors. Is the handle on the door a turning knob or a handle that pivots up and down? Typically, props should have a root control and a handle control so that the artist can move the character around globally and then, also, animate underneath that. For characters, the rigging team decided to go with a Metahuman face which works really well with motion capture data.

Previsualization and Layout

In the previsualization phase, we had the pleasure and freedom to read the script and animate the characters actions to what we believed the story was aiming to achieve. Since previsualization starts very early on, the layout team uses downloadable free rigs and props to get going in storytelling right away. When published with a rig, that prop geometry can easily be replaced and versioned up in multiple Maya files across many artists. Clean and clear setups keep the pipeline efficient and smooth. The temporary character rigs will later be replaced with the final character rigs with mocap data applied. The previsualization files are also the basis of the layout phase because right from the start, the layout team is designing frame ranges, actions, lenses, and camera angles. The editor uses these playblasts inspired by the storyboards and script to make an edit. After the shoot, there will be a mocap edit made and the layout team will conform their previsualization files to the timings and timecode defined in that edit. The cameras will also be adjusted to work with the new actions.

Editorial

TAOS has a previsualization edit that is sent to Goodbye Kansas before the shoot so that the physical prop team can understand the actions, sets, and story. This lets the mocap team know what they need to have ready on shoot day in terms of props to interact with and how to set the limitations of the environment. Since the actions and stunts become clearer when there is a visual representation of what is desired, the team can decide if a stunt actor is needed and if it is required to use wires, cages, and mats. The edit is also an integral part of cutting down the story to the most intriguing content within the guidelines of the film length. The edits are also a feedback guide to the layout team.

Goodbye Kansas Deliverables

There are guidelines to delivering to set and a shoot doc to help keep it all in order. The TAOS team has prepped all props and characters as skinned fbx files that match the requirements:

1. There should not be groups in between joints.
2. There must be a timecode node.
3. The joint rigs should be able to directly drive the control rigs later to not sacrifice any mocap data quality. This means joint positions and namings should not change later.
4. Environments are sent as fbx files as well. There is a color coding system where if something is touched, it is marked red, and if something is to be an eyeline object, it is marked in teal.

The next challenge we have ahead of us is to plan every take that we aim to capture on shoot day. By looking at the previs edit, we can determine which actions we need to capture in sequence, which actor will be marked to play them, and which environment that action will happen in. Ready, set, go!