Depth Enhancement Using Coupled RGB Camera

Student(s): Daniel Rotman
Supervisor(s): Prof. Guy Gilboa

In the past few years, due to the miniaturization of technology, depth sensing cameras have become a commodity. The release of Kinect revealed a marketable depth camera with relatively high resolution and frame-rate which acts as enormous potential to advance the field of image research using depth information.That said, there is still a large gap between the resolution and frame-rate capabilities of regular RGB cameras and of the Kinect. In order to overcome this weakness a number of methods [1,2,3,4] were proposed to improve the Kinect spatial and temporal resolution using a coupled RGB camera. With regard to the spatial resolution the basic idea is to use the coupled RGB camera to perform a filtering which matches the color of a pixel to the appropriate depth of the object presented by this color. The basic assumptions are that different objects will have naturally different colors, and that object boundaries can be sensed with much better precision using the color camera than with the depth camera which has known difficulties regarding edge pixels. In this fashion it is possible to enlarge and refine images received from a low resolution depth camera with the aid of a high resolution RGB camera. In this project we will implement state-of-the-art techniques to improve the spatial resolution of the Kinect depth camera. A number of different methods will be implemented and we will discuss their limitations. In addition we will attempt to overcome any limitations encountered when possible.