REAL-TIME DEPTH MAP ESTIMATION FROM INFRARED STEREO IMAGES OF RGB-D CAMERAS
đź“ś Abstract
RGB-D cameras are novel sensing systems that can rapidly provide accurate depth information for 3D perception, among which the type based on active stereo vision has been widely used. However, there are some problems exiting in use, such as the short measurement range and incomplete depth maps. This paper presents a robust and efficient matching algorithm based on semi-global matching to obtain more complete and accurate depth maps in real time. Considering characteristics of captured infrared speckle images, the Gaussian filter is performed firstly to restrain noise and enhance the relativity. It also adopts the idea of block matching for reliability, and a dynamic threshold selection of the block size is used to adapt to various situation. Moreover, several optimizations are applied to improve precision and reduce error. Through experiments on the Intel Realsense R200, the excellent capability of our proposed method is verified.
✨ Summary
Paper summary
The paper addresses incomplete depth maps and limited measurement range in the Intel RealSense R200, whose commercial stereo-matching pipeline uses local Census-based matching. The authors develop a real-time infrared stereo-matching method based on semi-global matching (SGM).
The method combines several processing stages:
- Gaussian filtering to reduce noise and increase similarity between the left and right infrared speckle images.
- Mutual-information-based dynamic selection of the block size used for matching.
- Birchfield–Tomasi matching costs augmented with horizontal Sobel-gradient information.
- Semi-global cost aggregation to incorporate information from multiple paths while retaining real-time efficiency.
- Uniqueness testing, subpixel interpolation, left–right consistency checking, and point-cloud growth to remove or reduce unreliable disparities.
- Conversion of disparity to depth using the calibrated stereo relationship (z=fB/d).
Experiments on three indoor scenes compare the RealSense commercial algorithm, standard SGM, and the proposed method. The proposed approach produces visually more complete depth maps, fewer abnormal values, and more complete object boundaries. In the wall-distance experiment, both methods have relative errors below 1% within 2 m; the commercial algorithm’s error increases more rapidly at longer distances and it fails to produce valid data at approximately 5 m or beyond, whereas the proposed method obtains usable measurements beyond 7 m. The paper reports that Gaussian filtering increases inter-image correlation by approximately 9% and mutual information by approximately 13%. The quantitative comparison is limited by the absence of a standard ground-truth dataset: error rates for the scene comparison are obtained through manual statistics.
Evidence of subsequent influence
The publisher’s metrics page records one Crossref citation for this paper: Huaizhou Li et al., “Research on 3D Reconstruction of Binocular Vision Based on Thermal Infrared,” Sensors (2023). That later study addresses a related problem involving low-texture infrared stereo imagery and uses preprocessing, semi-global block matching, and 3D reconstruction. The available citation record supports a documented connection to subsequent infrared stereo research, but it does not establish that the later work directly adopted this paper’s complete algorithm or that the paper had measurable industry deployment. (isprs-annals.copernicus.org)