3D on the Cheap, Part 2

Figure 1. The 3D Panasonic AG-3DA1 camera.

Last time out, I detailed a consumer oriented workflow for 3D production, culminating with uploading a 3D file to YouTube. Since the YouTube video has only been viewed about 50 times so far, I'm getting the message that consumer isn't exactly the level that you care about. So, this time out, we're stepping up a few levels. Specifically, I'll take footage shot with the Panasonic AG-3DA1, mux the two streams into the CineForm format and perform some nondestructive 2D and 3D editing in CineForm First Light, then edit in Adobe Premiere Pro CS5 with realtime 3D preview courtesy of Nvidia 3D Vision.

Of course, the workflow is only part of the equation, and what many readers care most about is who's using these technologies, This brings the questions back to who's doing 3D, how, and for what types of productions. So I asked three executives from Adobe, CineForm and Nvidia to answer a few questions that highlight these issues. These email interviews are presented at the end of the workflow article.

Related Links

3D on the Cheap, Part 1 In this article, I'll discuss how to shoot, edit, and upload 3D videos to YouTube, and what you'll need to view them from that site... Stereo Hype This past December, Vic Love was headed to Jamaica to shoot footage for a documentary about the history of reggae music. With him traveled a tech-averse... Low-cost 3D Cameras 2010 is really starting to look like the year of 3D. How's that? January started out with any number of companies at CES touting 3D playback capability... The State of 3D I wanted to know who's doing 3D, how, and for what types of productions. Dave Helmly, North American tech sales manager for pro video/audio at Adobe; CineForm CEO David Taylor; and Sean Kilbride, Nvidia's technical marketing manager, workstation products provided the answers in email interviews...

I couldn't have come up to speed anywhere near as quickly without the help of Adobe evangelist Dave Helmly, who is Adobe's North American tech sales manager for pro video/audio. For the complete picture on 3D production, check out this episode of Adobe TV: 3D Stereoscopic editing with Premiere Pro CS4/Win/OS X. Though the 62 minute duration sounds intimidating, Helmly covers a lot of ground, with very good depth, everything from practical tips about marking your SD cards to "how to" alternatives for 3D preview while editing.

In this article, I'll work through the high points of his suggested workflow, verifying that the techniques generally work as advertised. I won't go into the detail or depth that Helmly provides, but you can probably skim through this in 10 minutes and get a feel for the workflow. That said, if you're seriously interested in producing 3D, I strongly recommend that you check out the video in addition to this article.

Figure 2. MediaInfo, showing the video parameters of the AG-3DA1 files used in this tutorial.

Last time out, I used footage shot with the Fujifilm FinePix Real 3D W1. Obviously, that didn't impress, so I asked Helmly to supply some footage that he shot around Annapolis using the Panasonic AG-3DA1, which he graciously agreed to. Helmly makes some good points in his tutorial video about how a 3D camera provides much more flexibility than using two cameras on a single rig. For example, you can't zoom, change focus, or adjust aperture with a two-camera rig while shooting, since precise synchronization would be impossible. You can probably work around these limitations on an indy film type production, but for live events, it's near impossible. At $21,000, the Panny isn't cheap, but it may the cheapest legitimate point of entry for many producers.

As you probably know, the AG-3DA1 captures two streams of AVCHD footage with two independent optical systems and two CCDs. The product specs don't mention the data rate of the footage, but according to MediaInfo (Figure 2), the video Helmly supplied was full-resolution (1920x1080) AVCHD, with a data rate of 16Mbps. I asked Helmly about higher-data-rate options, and he said you can configure the video up to 21Mbps.

Figure 3. Converting the .MTS files to AVI in CineForm HDLink.

Helmly sent me two AVCHD .MTS files, which I can certainly load into Premiere Pro side by side and edit the way I showed in the last article. But then I'm not editing in 3D, and I can't adjust any aspects of the 3D files, like convergence. The key benefits of converting to CineForm include working with a single file that can display the video in 3D in Premiere Pro, working with a less CPU-intensive format than AVCHD, and the ability to nondestructively adjust 2D and 3D characteristics in CineForm's First Light program. More on First Light in a moment. If you'd like to work along with me, you can download Neo3D here, and download some 3D test footage from CineForm here. Note that Neo3D costs $2,500, which is certainly reasonable for corporate and indy producers, but likely out of the reach of most hobbyists.

If you're working with files shot with the AG-3DA1, you can convert the two .MTS files to a muxed stereo CineForm file in CineForm's First Light application using a procedure you can read about here. Since most readers won't be working with footage from this camera, I'll work through the procedure you would use for other footage, which starts in CineForm HDLink (ReMaster on the Mac), one of the applets that gets installed when you setup Neo3D. In this program, I'll convert the two .MTS files to CineForm AVI format, then mux the files in First Light.

The conversion process is simple: You run the conversion program, load the two source clips, and press Start. On my HP Z800 workstation with dual processor, 3.33GHz 6-core Intel Xeon chips running 64-bit Windows 7, it took 1:28 (min:sec) to convert the two 2:36 clips to CineForm, which felt pretty snappy to me.

Figure 4. The newly created CineForm file.

As you can see in Figure 4, the data rate of the CineForm file is about 133Mbps, and the file is roughly eight times larger than the original AVCHD file. To put this in perspective, the data rate of 1080p ProRes 422 (LT) is 102Mbps, while ProRes 422 is 147Mbps, and ProRes 422 (HQ) 220Mbps. So, the files are big, but they're in the ballpark with other digital intermediate formats.

Figure 5. Here I tell First Light that this is video from the left eye and choose the Sync Frame.

To mux the files in First Light, load them into the program, then identify both the eye and the Sync Frame in each video, which First Light uses to synchronize the two clips. Though synchronization seemed pretty much assured when shooting with the Panasonic AG-3DA1, Helmly still used a clapper with his video, which you can see in Figure 5.

As you can see in the figure, I clicked the 00007_right.avi in the Clip Library, then the Right Eye checkbox beneath the player, then moved the playhead to the sync frame where the clapper clapped and clicked Set. Like all well-mannered video programs, you can use your left and right arrow keys to precisely navigate to the sync frame. Interestingly, when I reached the same frame in the left eye stream, I noticed that it was frame 137, not frame 138. Even if you're working with 3D, two-stream camcorder like the 3DA1, you should use a clapper or other visual cue to clearly mark a sync frame, and sync the two streams before editing.

Figure 6. Muxing to the two clips.

To mux the two streams into a single stream, you choose the two streams in the Clip Library, choose 3D > 3D Mux from the First Light menu, then click Add to Queue and Start Queue. If you're working with multiple source files, and you number them properly, you can click Auto Mux to mux multiple pairs, though this would only work if the clips were already synced. If you have to choose sync frames for the clip pairs, you'll have to do each one manually.

I worked through the same procedure on the Mac, and found ReMaster much slower, taking about 12 minutes to convert from .MTS to CineForm format on dual processor, 2.93GHz quad-core Mac. More disturbing was that the work flow in First Light was completely different than the Windows version and much less visual. I asked CineForm CEO David Taylor whether the company intended to conform the two interfaces in the near term, and he responded that the Mac version will be revised to work more like the Windows version "soon."

On both operating systems, the muxed file turned out to be 5.85GB, slightly less than twice the size of the two individual files, which were 2.58GB each. Once muxing is complete, you import the muxed file back into First Light to perform 2D and 3D adjustments.

Figure 7. Making 3D adjustments in First Light.

CineForm calls the process the "Active Metadata" workflow. Basically, this means that the adjustments made in First Light are not rendered until you actually render the project in your editor, so you can go back and change them at any time. In fact, if you make any adjustments to the file in First Light after you import the file into an editor, they're applied to these muxed files in the editor in realtime. To close the loop, these adjustments exist only as metadata—hence the Active Metadata moniker. More on Active Metadata in the David Taylor interview below.

In Figure 7, you can see the 3D corrections that First Light enables, which are invaluable when working with two camera rigs, since some adjustments is almost always necessary. Obviously, when shooting with 3D cameras like the Panasonic AG-3DA1, most parameters should be identical, and the adjustments minimal. Note that First Light lets you key frame these adjustments, which is critical for many shots where the point of focus, or other compositional elements may change.

Figure 8. Choosing a look in First Light.

Beyond these 3D adjustments, First Light feels like a cross between Apple Color and Red Giant Magic Bullet Looks, with lots of color adjustments and multiple preset looks to choose from, again which are not rendered and exist only as metadata. Check out this CineForm-created tutorial on First Light's 2D capabilities and this tutorial on 3D workflow.

When you're working in First Light, you can preview the 3D file any number of ways, from anaglyph to side-by-side to onion skin. However, if you have an Nvidia 3D Vision system in your computer, or other 3D visualization system, you can view the video in 3D in realtime. Not to gush, but this is the glue that holds the process all together, because while the ability to adjust 3D footage is nice, you're flying blind unless you can see the changes in real time.

Figure 9. Choosing the Nvidia 3D Vision system from within First Light.

Briefly, the 3D Vision system combines active shutter glasses with an infrared controller. The system costs less than $200, but you'll need a 120Hz LCD panel to make it work. I used the Acer GD235HZ monitor, which you can find for less than $400. Helmly does a nice job in his video outlining other options for both your primary and secondary 3D monitor, including 3D TV sets, so if you're curious about your options here, check out his video. Overall, the 3D Vision/Acer combination worked well for me, with very clear and distinctive 3D.

Figure 10. New CineForm presets in Premiere Pro.

Once you've got your video synced, muxed, and adjusted, it's time to edit. When you install Neo3D on your computer, you'll find a number of new CineForm presets in Premiere Pro, which makes sequence setup easy (see Figure 10).

Figure 11. Customizing Premiere Pro to display 3D video in the preview monitor.

Choose a new preset, then import your content and start editing. If you've got a 3D Vision or other 3D visualization system installed on your computer, you'll have to choose that display mode within Premiere Pro to watch your video in 3D in the preview monitor and/or full screen. You accomplish this using the CineForm Playback Settings dialog accessed in the wing menu for the preview monitor, or via the output button below the playback controls in the preview monitor.

Figure 12. Converting a 2D title to 3D with CineForm's Parallex 3D effect.

A couple of noteworthy items to keep in mind while editing. If you add elements to the sequence, like titles, CineForm supplies a Parallax 3D effect you can use to add depth to the title (see Figure 12). CineForm also includes Color Balance, Color Corrector, and Levels effects, and a Chroma Noise Filter, though Adobe's Helmly states in his video that any 32-bit Premiere Pro effect will preserve the 3D nature of the source files. The obvious corollary is to avoid using non-32 bit effects in CineForm-based 3D projects.

When it's time to output, you can output in CineForm AVI or MOV format, which preserves the active metadata in the file. Otherwise, you can access all of Premiere Pro's traditional formats, like H.264, MPEG-2 and the like, though you'll need to configure your video in First Light (and accordingly, in Premiere Pro) to support the desired 3D output.

Figure 13. Switching the display to side-by-side in First Light (bottom) also switches the display in Premiere Pro (top).

For example, as discussed last time out, YouTube wants a side-by-side display for uploading 3D footage. If you were editing with Nvidia's 3D Vision system, and were ready to render, you would switch the display in First Light to side-by-side, which would change the display in Premiere Pro as well. You can see this in Figure 13, where I'm changing the 3D Display Type to Side-by-Side in First Light in the bottom image, which updates the display in Premiere Pro. Then, after selecting the display you need, you'd output your video.

Overall, the integration is impressive, and the work flow logical and well within the reach of reasonably technical editors. This brings the questions back to who's doing 3D, how, and for what types of productions. For answers to these questions, we'll transition over to our email interviews.