Pose- and occlusion-invariant face recognition behind production identity at scale.
A line of work on face recognition that stays robust under the conditions that break naive 2D pipelines: large pose variation, partial occlusion, expression, and dataset shift between training and deployment. The thread runs through my Ph.D. and shipped components for production identity systems handling billion-level annual checks.
The 3D-aided 2D face recognition system UR2D(Xu et al., 2017; Xu et al., 2017) uses a 3D model registered against the 2D image to handle pose variation up to 90°. Joint head pose estimation and face alignment (Xu & Kakadiaris, 2017) share global and local CNN features, which we extended into face reconstruction with proper feature aggregation (Xu et al., 2019). Under occlusions specifically, OREO (Xu et al., 2020) improved generalization by 10.17% rank-1 accuracy in single-image settings. The FaRE package (Xu & Kakadiaris, 2019) packaged consistent open-source evaluation across these benchmarks.
The thread connecting all of it: robustness, not raw accuracy, is the binding constraint of deployed face recognition. That lesson carried into the production identity work and the lifelong-learning research that followed.
@inproceedings{xu2020generalization,title={On Improving the Generalization of Face Recognition in the Presence of Occlusions},author={Xu, Xiang and Sarafianos, Nikolaos and Kakadiaris, Ioannis A.},booktitle={IEEE/CVF Conference on Computer Vision and Pattern Recognition Workshops (CVPRW)},year={2020},}
@inproceedings{xu2019feature,title={On the Importance of Feature Aggregation for Face Reconstruction},author={Xu, Xiang and Le, Ha and Kakadiaris, Ioannis A.},booktitle={IEEE/CVF Winter Conference on Applications of Computer Vision (WACV)},year={2019},}
@inproceedings{xu2019fare,title={{FaRE}: Open Source Face Recognition Performance Evaluation Package},author={Xu, Xiang and Kakadiaris, Ioannis A.},booktitle={IEEE International Conference on Image Processing (ICIP)},pages={3272--3276},year={2019},}
2017
IJCB
Evaluation of a 3D-Aided Pose-Invariant 2D Face Recognition System
Xiang Xu, Ha Le, Pengfei Dou, and 2 more authors
In International Joint Conference on Biometrics (IJCB), 2017
@inproceedings{xu2017evaluation,title={Evaluation of a 3D-Aided Pose-Invariant 2D Face Recognition System},author={Xu, Xiang and Le, Ha and Dou, Pengfei and Wu, Yuhang and Kakadiaris, Ioannis A.},booktitle={International Joint Conference on Biometrics (IJCB)},year={2017},}
arXiv
When 3D-Aided 2D Face Recognition Meets Deep Learning: An Extended UR2D for Pose-Invariant Face Recognition
Xiang Xu, Pengfei Dou, Ha A. Le, and 1 more author
@article{xu2017ur2d,title={When 3D-Aided 2D Face Recognition Meets Deep Learning: An Extended UR2D for Pose-Invariant Face Recognition},author={Xu, Xiang and Dou, Pengfei and Le, Ha A. and Kakadiaris, Ioannis A.},journal={arXiv preprint arXiv:1709.06532},year={2017},}
FG
Joint Head Pose Estimation and Face Alignment Framework Using Global and Local CNN Features
Xiang Xu and Ioannis A. Kakadiaris
In IEEE Conference on Automatic Face and Gesture Recognition (FG), 2017
@inproceedings{xu2017joint,title={Joint Head Pose Estimation and Face Alignment Framework Using Global and Local CNN Features},author={Xu, Xiang and Kakadiaris, Ioannis A.},booktitle={IEEE Conference on Automatic Face and Gesture Recognition (FG)},year={2017},}