Animating PanoMoments in Unity
Updated 2 years ago
1.7 K
Introducing the Unity PanoMoments toolset
PanoMoments are a new short-form VR medium that associates time and space. However, describing them with words is nearly impossible. The best way to understand what they are is to experience one yourself:
For the highest performance experience, see our native apps, and note that we're actively working on reducing the overall file size in addition to a streaming optimization that will drastically cut load times.
PanoMoments exist because we felt VR deserves something more engaging than a 360 photo. And while 360 video is perfect for live events and cinematic stories, sometimes you just want something more bite-sized like a photo. PanoMoments capture the slice of time that defines a moment, whether that moment is a 1/10,000th of a second hyper-slowmo or a 24 hour timelapse. It’s this constraint, much like the border of a painter’s canvas, that brings out the creativity. The simple association of time + space guarantees that each and every frame captured will be seen by the viewer, giving the content creator full control of the experience. And with rotating cameras and animated captures, the ability to show motion parallax brings the experience to life whether in a headset or on the web.
Check out the this link to see some examples of what animated PanoMoments feel like, but note that all of these were converted from 360 video available on the Internet, rather than created directly for the PanoMoments format - which also means that they have no motion parallax.

Unity as the animation toolset for PanoMoments

Animating 360 video content is still far from easy. There just aren't many tools available to facilitate the process. And while you can easily convert 360 video to the PanoMoments format, creating an animated PanoMoment with this workflow is a bit of a roundabout solution. Today we're releasing a very simple (yet slightly rough) Unity 2018.1 project that is pre-configured to directly export PanoMoments from within Unity. We see Unity becoming the primary toolset for animating PanoMoments, and as a way for the creative community to explore an entirely new VR medium.
To give it a try, follow these instructions:
  1. Download and install Git, FFMpeg, and Unity 2018.1 or later.
  2. Clone this repo into a local folder by typing "git clone"
  3. Launch the Unity project in the examples folder.
  4. Open the Demo scene.
  5. Expand the "Camera Holder" -> "Main Camera" in the Scene Hierarchy and specify your desired number of frames in the "Total Frames" field of the "Camera Rotation" script.
  6. Hit Play and frames will be exported to the project's "Recordings" folder as a PNG sequence. Exporting will be slow as it's currently configured to output 8K textures. Once you notice the editor preview speed up the export has completed.
  7. Encode the PNG sequence with FFMpeg: ffmpeg -y -framerate 30 -i "PanoMoment%4d.png" -vf scale=4096:2048 -c:v libx265 -crf 12 hevc.mp4
  8. Upload to with "Contains Parallax" set True, "Alignment" set to 0 degrees, and "Clockwise" set True.
  9. You're done!
Note - The cat animation isn't currently synchronized to the camera rotation. Most likely when you'd actually compose a real PanoMoments animation, the timing would be more orchestrated with the camera rotation.
Check out this link to see what the demo Unity project feels like in its final form. It's quite ugly :) but should hopefully serve as a springboard for more talented animators to do their magic.
When you're ready to animate your own PanoMoment, just open the Empty scene to start from scratch. Note that the "Camera Rotation" script is already configured to export frames and for capturing with parallax. Also keep in mind that the exporter will overwrite any files in the "Recordings" folder, so make sure it's cleaned out before exporting a new PanoMoment or you will end up losing your previous exports.

Motion Parallax

One of the big benefits of the PanoMoments format is the ability to show motion parallax. We're not talking about a full 6-DOF volumetric capture (PanoMoments only capture depth along the rotational axis), however, it's enough to break the flat feeling inherent to 360 photos/videos. And unlike stereo content, you can appreciate this effect whether you're viewing in a headset or on the 2D web. Capturing parallax is done by rotating the camera offset from the optical center-point. See the GIF below for what this looks like when capturing with a physical rotation:

In Unity, this is done in the exact same way, but with a virtual camera. Depending on your Unity scene's scale, you'll need to modify the X position value of the "Main Camera" to find a value that makes sense. Too high of a value will feel unnatural, and too small (or zero) will feel flat.

Opening up a whole new creative VR medium

Since the beginning, our goal has been to provide a platform where people can discover, create, and share PanoMoments. Starting today, with the release of the sample Unity project, we're hoping to inspire the creative community, challenge them to think in a new way, and create something amazing.
Make sure to check out the website for much more info, and feel free to reach out to us at if you have any questions at all.