Realistic Integration of Virtual Objects in Marker-less Augmented Reality Environment

Abstract

Dense stereo matching is the problem of determining per-pixel disparity given two images taken of the same scene from different viewpoints. A dense disparity map with precis edges is required for many applications including accurate visual occlusion in Augmented Reality. Well-established descriptors are proved successful in many computer vision applications based on sparse matching. Due to the robustness of these descriptors, they are getting utilized in dense matching which relies heavily on patch-intensity matching and aggregation techniques. This thesis evaluates the well-established binary descriptors for dense matching based on the resulting disparity maps. We choose the most adopted descriptors whose OpenCV implementations are widely used. Most of the available descriptor evaluation research is in the sparse domain. There are no available comparisons for per-pixel dense matching, to the best of the author’s knowledge. BRIEF is shown to produce the smoothest disparity map, while FREAK and BRISK achieved smaller overall error ratio. This finding is consistent with the existing body of research. However, all the tested binary descriptors clearly suffer near edges which necessitates the need for adopting standard aggregation that is originally introduced for patch-intensity matching. In the same context, we also propose to supplement the matching criteria with a second metric that makes up for the information loss associated with aggregation. The hybrid metric complements the binary descriptor by providing more information about the pixel to be matched. Adding the hybrid metric causes the error ratio to decrease by up to 2.86% if compared to the results without it. Also, the radiometric invariance of BRIEF is maintained because the weight controlling the contribution of each metric is evaluated adaptively depending on the intensity of the pixel and its neighbors instead of fixing the weights. The proposed hybrid metric does not cause an increase in the running time because of the simplicity of the added metric. Moreover, the weights are calculated by reusing pre-calculated values to avoid adding computational complexity. Furthermore, we propose to parallelize the sequential implementation using CUDA. Speedup of up to 109x is achieved for some of the datasets.