---

Research Projects

Differential Geometric Inference in Surface Stereo Many two-view stereo algorithms explicitly or implicitly use the frontal parallel plane assumption when exploiting contextual information, e.g. the smoothness prior biases towards constant disparity (depth) over a neighborhood. This introduces systematic errors for slanted or curved surfaces. We use contextual information geometrically, which is based on a differential geometric study of smooth surfaces. This is encoded in Cartan's moving frame model over local quadratic approximations. The result enforces geometric consistency for both depth and surface normal, providing additional constraint for stereo.

Exploiting Occluding Contours for Real-Time 3D Tracking: A Unified Approach Traditional edge-based 3D tracking methods have difficulty handling occluding contours of curved surfaces, since they are not static model edges but change with the viewpoint. We propose a unified approach to edge-based tracking where 3D edges including occluding contours are utilized. This is achieved through an analysis of local surface differential geometry. This approach uses a simple parametrization of both types of model edges within the same framework. Our system can track both types of model edges in a very fast and robust manner.

Surface Geometric Constraints for Stereo in Belief Propagation Classical formulation of belief propagation implicitly imposes the frontal parallel plane assumption in the compatibility matrix for exploiting contextual information, since the priors perfer no depth (disparity) change in surrounding neighborhoods. This results in systematic errors for slanted or curved surfaces. To eliminate these errors we propose to use contextual information geometrically, and show how to encode surface differential geometric properties in the compatibility matrix. This enforces consistency for both depth and surface normal, extending the traditional formulation beyond consistency for (constant) depth.

Differential Geometric Consistency Extends Stereo to Curved Surfaces We perform a differential geometric study of smooth surfaces for stereo vision, and argue that geometric contextual information should be encoded in Cartan's moving frame model over local quadratic approximations of the smooth surfaces. The result enforces geometric consistency for both depth and surface normal. We develop a stereo algorithm to illustrate the importance of using such geometric contextual information and demonstrate its power on images of the human face.

Contextual Inference in Contour-Based Stereo Correspondence We relate the 2D differential structure (position, tangent, and curvature) of curves in the left and right images to the Frenet approximation of the (3D) space curve. A compatibility function is defined via transport of the Frenet frames, and they are matched by relaxing this compatibility function on overlapping neighborhoods along the curve. The remaining false matches are concurrently eliminated by a model of "near" and "far" neurons derived from neurobiology.

Stereo for Slanted Surfaces: First Order Disparities and Normal Consistency Traditional stereo algorithms either explicitly or implicitly use the frontal parallel plane assumption, which introduces both structural and geometric errors. We extend stereo matching to include first-order disparities. Contextual information is then expressed geometrically by transporting surface normals over overlapping neighborhoods for slanted surfaces. A novel stereo algorithm that combines first-order disparity with position (zero-order) disparity is developed.

---

Previous Projects at Tsinghua

Projective Transformation Based Stereo Vision for Obstacle Detection We exploit the "recognition by alignment" technique based on "Plane+Parallax" to solve the obstacle detection problem. Knowing the general formula constraining two corresponding points in stereo images by projective geometry: the Relative Affine Structure, we use a special case: the planar projection stereopsis, which detects any obstacle extending from the road plane.

---

Course Projects at Yale CS

Person Verification by Support Vector Machines - A Face Recognition Approach CPSC663 Maching Learning, joint work with Peishen Qi How to do identity verification by Appearance-Based Face Recognition using only one Support Vector Machine? Working in the difference space, we demonstrate our appearance based method using SVM can not only interpolate correctly, but also extrapolate to novel viewing conditions, i.e. different poses and different illumination conditions.

Robust Principal Component Analysis Based on M-estimation CPSC577 Neural Networks for Computing Implement and refine the PCA method for learning the low order linear subspace for face recognition. Work mainly based on F. De la Torre and M. Black's ICCV'01 work on Robust PCA and Z. Zhang's IVC'97 review work on Parameter Estimation.

Shape from Shading Using Symmetric Property CPSC575 Computational Vision and Biological Perception For symmetric objects with (unknown) varying albedo (e.g. faces), how to do shape from shading? Inspired by W. Zhao's Ph.D. thesis and CVPR'00 work, the varying unknown albedo term is cancelled, and a toy demo of the front end of an illumination insensitive face recognition system is demonstrated.

Parallel Image Segmentation via Graph Partitioning CPSC524 Parallel Programming Techniques, joint work with Jiang Chen How to do the popular Normalized Cuts (Shi & Malik, PAMI'00) in a parallel fashion? We show that to get an approximate solution to the NP-complete exact normalized cuts algorithm, the Lanczos method could be used. Specifically, we describe various issuses regarding the parallelization of the algorithm, and implement the parallel Lanczos method in a distributed memory environment using MPI (Zoo cluster of Yale CS Dept.).