I’m sure many of you remember NAB Show 1996 where Sony and Panasonic introduced their DVCAM and DVCPRO formats. What was fascinating is both companies made great efforts to distance their new “pro” formats from “consumer DV.”
Now that quite a few raw cameras have come to market, we are in a different situation. Rather than companies working hard to create proprietary differences between their products—when at their codec heart—they are quite similar; we have companies not differentiating on the basis for their recording “format.” Instead, sensor size, sensor resolution, lens mount, and dynamic range are now key differentiators.
Raw: What It Is and What It Is Not
Unless a camera has three sensors, the data from a single chip camera that has a Bayer filter, will not have been “debayered” before it is stored. Put another way, a raw recorded serial RGB data stream from a sensor is neither video (Y’CrCb) nor RGB (4:4:4 or 4:2:2). And that’s the fundamental definition of raw.
A Bayer filter has thousands and thousands of groups each with four filters. Within each group, there are two green filters, plus one red and one blue filter. Figure 1 shows a group of filters. A raw camera stores each signal value as it is read from the sensor.
Figure 1: Bayer Filter
Keeping the Image Clean
Although a raw camera does not deBayer the sensor’s signal, there are other causes of image degradation that can afflict a raw camera. To avoid quality loss, the entire sensor readout and processing stage must be fast enough to handle all the photosites within the video recording chip’s window. This is a demanding task because the higher the clock rate, the greater the heat produced, and the more power consumed. Obviously, the simplest way to keep a sensor’s clock rate low is to greatly reduce the amount of data coming from a chip. To accomplish this goal, during each readout, some DSLRs skip lines of photosites, thereby generating aliasing and moiré.
An adequate clock rate is not enough to prevent aliasing and moiré. To prevent these artifacts, the camera needs an OLPF (optical low-pass filter) carefully tuned to the Nyquist frequency of the number of pixels employed when shooting video. (DSLR filters are designed for very much higher photo resolutions.) Some raw cameras, in an effort to capture more fine detail, do not even use an OLPF.
Unless a camera uses CCDs, it will exhibit rolling shutter and “jello” artifacts. A CMOS sensor’s rows are processed—reset, integrated and output—in a sequence that occurs over time. Therefore, a CMOS sensor exposes each frame in a top-to-bottom pattern. In future, CMOS sensors will likely be equipped with global shutters to prevent artifacts.
Once a serial RGB signal has been obtained from a sensor, the next step is to write it to solid-state media. A copy of the serial RGB signal is internally debayered for the camera’s LCD/VF, HDMI/SDI ports, and with the Blackmagic cameras, their ProRes/DNxHD encoders.
Trio of Low-Cost Raw Cameras
Three low-cost raw cameras have begun shipping over the past year: Blackmagic’s Cinema Camera (BMCC), Pocket Cinema Camera (BMPCC) and Production Camera (BMPC4K), plus the D16 from Digital Bolex. While all record raw data, the way they do differs. Table 1 presents options available for raw cameras.
Table 1: Raw Camera Processing Options
The Digital Bolex D16 wraps uncompressed serial RGB data from its Super 16-sized CCD into CinemaDNG files. The Blackmagic Cinema Camera also stores uncompressed data into CinemaDNG files.
There are two types of lossless compression: “visually lossless” (or “virtually lossless”) and “lossless.” During encoding to a visually lossless stream, data is discarded in a manner that should not be perceptually evident to a viewer. When lossless encoding is performed, data reduction occurs without the loss of any image information. Fundamentally this is like ZIP compression. Both the Blackmagic Pocket Cinema Camera and the Production Camera 4K store lossless compressed data into CinemaDNG files.
Rec. 709 Gamma Correction
A century ago, when television was being invented, one of the critical concepts involved in picture presentation was “gamma.” We still have gamma with us today. Moreover, video cameras still employ something called “gamma correction.” Gamma correction is now an HD standard called Rec. 709. To understand both gamma and gamma correction, we need to come understand how they work together.
Before we delve deeper, some definitions. First, “log” is short for logarithm. Second, a “function” inputs a signal and outputs a signal. Third, a function need not be something we calculate. One can graph the input and output of a device and infer from the graph the function that describes their relation.
Figure 2 is a plot of cathode ray tube (CRT) light intensity as its input voltage is increased. This relation is valid for every CRT, from the earliest days of television. The curve is called gamma. Excel has plotted a power trend line along it. Excel also computed the function’s slope to be 2.385, very close to the video gamma standard of 2.2 to 2.5. (LCDs and plasma monitors emulate a CRT’s gamma.) A CRT’s “Power” response, by coincidence, complements the logarithmic nature of the human visual system. (A CRT’s response curve is the inverse of our visual system’s response curve.)
Figure 2: CRT Gamma Curve
Cameras deal with CRT gamma by applying gamma correction before recording a signal. Knowing gamma is a power function, I had Excel plot a Power function, POWER(input-signal, 0.45). Figure 3 presents the curve, computed with a power of 0.45, 1.0 ÷ 2.2 (gamma), and a linear signal from 0.0 to 1.0. The curve represents gamma correction with a slope of 0.45.
Figure 3: Gamma Correction Curve
Figure 4 shows a gamma correction (gold) curve and a gamma (purple) curve, plus the product of the gamma correction and gamma values. The product (blue) is linear, which demonstrates ideal (unity) information transfer.
Figure 4: Light Output from CRT (blue line)
When a camera’s recording will not be viewed directly, the camera need not process image data to implement Rec. 709 gamma correction. Nevertheless, other processing may be done within a camera.
So far, we have talked without any references to photography. We’ll now correct this lack. When brightness increases by a factor of two, light is said to have increased by one F-stop. The signal (voltage) from a sensor likewise increases by a factor of two. Moreover, the binary value from the sensor’s A/D doubles.
Assume we are working with a 16-bit device that has a 12-stop dynamic range. With 16 bits, 65,536 unique “binary codes” represent its black to white range. At the point of maximum light input, it outputs a binary code of 65,535. When light is lowered by one F-stop, output codes decrease from 65,535 to 32,767. Using 32,768 codes for one stop is inefficient because highlights carry little perceptible information.
For all remaining stops, only 32,768 codes remain unclaimed. Code shortage forces the difference between, for example, stop 6 (a “1023” code) and stop 7 (a “511” code) to be 512 steps, while the difference between stop 11 (a “31” code) and stop 12 (a “15” code) is only 16 steps. Several alternatives are available for treating these A/D data.
The simplest alternative, “linear light” enables a camera to store serial RGB data after either no, or minimal, processing. When the stored data word-length is shorter than A/D word-length, the number of binary codes defining a one-stop light change is reduced. For example, when storing 12-bit data from a 16-bit A/D, the difference between stop 11 and stop 12 is reduced from 16 steps to four steps. The linear option preserves binary value ratios representing one-stop light changes. Moreover, no dynamic range is lost when stored data have the number of bits required to carry a sensor’s dynamic range.
The more complex alternative, log, applies one of a family of nonlinear curves to A/D data to shape these data to meet design goals. In some cases, a camera will be explicitly marketed as offering specific “log” or “gamma” curves. In other cases, such as Rec. 709, the curve has not traditionally been considered a “feature.”
Expensive raw cameras employ log to enable a camera designer to provide shooters with a variety gamma curves, some of which mimic the response of different film stocks while others enable a camera’s image to more closely match scanned film. Log curves, such as Sony’s S-Log, maximize recorded dynamic range.
S-Log enables a sensor with a very large dynamic range to squeeze its signal into the standard 0 to 109 percent range. Figure 5 shows how S-Log enables over a 1,000 percent dynamic range to be recorded. As you can see, the lowest ten increments, representing only 10 percent of the sensor’s full range, are transferred by 65 percent of the available output codes. The remainder of the sensor’s range is transferred by the remaining 35 percent.
Figure 5: Sony S-Log Curve
Less expensive cameras use log processing to support a more limited range of tasks. For example, log enables all, or most all, of a sensor’s dynamic range to be retained when 10- or 12-bit stored data are shorter than 14- or 16-bit data from the camera’s A/D.
Another task: applying traditional Rec. 709 gamma correction. Although dynamic range will be slightly reduced, the result requires minimal grading and makes good sense when the production target is HD television.
Figure 6 presents a simulation of “gamma encoding.” When equal signal increases are passed through the simulation, LOG10(sensor-signal, 0.25), output increases are large, medium, and small from shadow, midtone, and highlight levels, respectively. Important visual information thus receives more binary codes than does relatively unimportant highlight information.
Figure 6: Gamma Encoding
When a camera such as the Pocket Cinema Camera and Production Camera require maximum data compression, lossless compression accomplishes the goal. When the goal is to deliver to the person performing color grade maximum sensor information, uncompressed provides this capability.