A Bayesian Approach to Digital Matting
đź“ś Abstract
This paper proposes a new Bayesian framework for solving the matting problem, i.e. extracting a foreground element from a background image by estimating an opacity for each pixel of the foreground element. Our approach models both the foreground and background color distributions with spatially-varying sets of Gaussians, and assumes a fractional blending of the foreground and background colors to produce the final output. It then uses a maximum-likelihood criterion to estimate the optimal opacity, foreground and background simultaneously. In addition to providing a principled approach to the matting problem, our algorithm effectively handles objects with intricate boundaries, such as hair strands and fur, and provides an improvement over existing techniques for these difficult cases.
✨ Summary
The paper formulates digital matting as a Bayesian estimation problem under the compositing model (C = \alpha F + (1-\alpha)B). Given a user-provided trimap, it estimates foreground color, background color, and opacity jointly. Local foreground and background color distributions are represented with spatially varying, oriented Gaussian models. The method uses opacity-weighted neighborhood samples, spatial falloff, color clustering, and a maximum a posteriori objective. Optimization alternates between solving a linear system for foreground and background colors with fixed opacity and projecting the observed color onto the estimated foreground–background line segment to update opacity. The approach supports constant-color, difference, and natural-image matting, and the experiments report improved handling of fine structures such as hair and fur compared with contemporaneous methods.
The work influenced subsequent matting research in two documented ways. First, the authors extended the approach to video matting in Video Matting of Complex Scenes (SIGGRAPH 2002), combining Bayesian matting with optical-flow-based trimap interpolation for moving objects and complex boundaries. (csie.ntu.edu.tw) Second, later matting literature identifies Bayesian matting as a foundational traditional sampling-based approach and continues to cite it when describing the progression from classical methods to closed-form and deep-learning-based matting. For example, Deep Video Matting via Spatio-Temporal Alignment and Aggregation explicitly describes the video extension and contrasts it with later learned video-matting methods, while Background Matting: The World Is Your Green Screen cites Bayesian matting as a classical method requiring trimaps. (openaccess.thecvf.com) The paper’s publication metadata is corroborated by the Microsoft Research record and the authors’ project page. (microsoft.com)