Blue Screen Matting
📜 Abstract
A classical problem of imaging—the matting problem—is separation of a non-rectangular foreground image from a (usually) rectangular background image—for example, in a film frame, extraction of an actor from a background scene to allow substitution of a different background. Of the several attacks on this difficult and persistent problem, we discuss here only the special case of separating a desired foreground image from a background of a constant, or almost constant, backing color. This backing color has often been blue, so the problem, and its solution, have been called blue screen matting. However, other backing colors, such as yellow or (increasingly) green, have also been used, so we often generalize to constant color matting. The mathematics of constant color matting is presented and proven to be unsolvable as generally practiced. This, of course, flies in the face of the fact that the technique is commonly used in film and video, so we demonstrate constraints on the general problem that lead to solutions, or at least significantly prune the search space of solutions. We shall also demonstrate that an algorithmic solution is possible by allowing the foreground object to be shot against two constant backing colors—in fact, against two completely arbitrary backings so long as they differ everywhere.
✨ Summary
Summary
The paper formulates blue-screen matting as the recovery of an uncomposited foreground color and opacity from a foreground image composited over a known backing color. Using premultiplied-alpha compositing, it writes the image-formation model as (C_f=C_o+(1-\alpha_o)C_k), where the unknown foreground has three premultiplied color components and one alpha value.
The central theoretical result is that ordinary single-background matting is underdetermined: three color equations must recover four unknowns, producing infinitely many mathematically valid foregrounds and mattes. Consequently, a fully general single-image algorithm cannot identify the intended solution without additional assumptions, constraints, or human guidance.
The authors then derive several solvable or partially solvable cases. If the foreground contains no blue and the backing is pure blue, alpha can be recovered directly from the blue channel. If the foreground obeys a linear color relationship—covering gray, near-gray, and flesh-colored objects—alpha can also be obtained analytically. These constraints provide a mathematical interpretation of practical color-difference methods associated with Petro Vlahos and Ultimatte.
The principal algorithmic contribution is triangulation matting. By photographing the same, registered foreground against two different known backing colors, the unknown foreground color can be eliminated and alpha recovered directly. The paper generalizes this result using linear algebra to arbitrary backing colors, provided the two backings differ at each pixel. It further describes least-squares solutions for overdetermined systems involving more than two backing images.
The method is effective for static or repeatable scenes, but sensitive to camera misregistration, exposure changes, brightness variation, and noise. These errors generally bias the result toward excess transparency and can create thin transparent artifacts around fine lines. The authors also discuss physical alpha bounds that reduce, but do not eliminate, the solution space; blue-spill modeling; backing shadows; backing impurities; and limitations of color-space separating-surface approaches.
Influence and subsequent use
The paper placed traditional blue-screen matting on an explicit algebraic footing and introduced the term and method commonly known as triangulation matting. Later research has cited its two-background formulation as a basis for recovering mattes and foreground colors in static scenes. For example, subsequent work on boundary matting identifies Smith and Blinn’s method as a seminal multi-background approach, while later real-time systems adapt the same mathematical idea to simultaneous image capture for video. (graphics.stanford.edu)
The triangulation equations have also been used in practical reconstruction and data-acquisition systems, including opacity-hull and image-based 3D-scanning work that obtains opacity mattes from multiple controlled backgrounds. The method is additionally used in computer-vision and computer-graphics instruction as a way to generate matte ground truth for evaluating natural-image matting algorithms. (people.csail.mit.edu)
A Stanford retrospective records that an equivalent two-background technique had been independently used in Hanna-Barbera’s production pipeline before the paper, while noting that Smith and Blinn’s publication provided a formal derivation and broader generalization. (graphics.stanford.edu)