2016, 6(3): 263-296. doi: 10.3934/naco.2016011

Robust and flexible landmarks detection for uncontrolled frontal faces in the wild

1. 

Department of Computing, Curtin University, Perth, Western Australia

Received  June 2015 Revised  August 2016 Published  September 2016

In this paper, we propose a robust facial landmarking scheme for frontal faces which can be applied on both controlled and uncontrolled environment. This scheme is based on improvement/extension of the tree-structured facial landmarking scheme proposed by Zhu and Ramanan. The whole system is divided into two main parts: face detection and face landmarking. In the face detection part, we proposed a Tree-structured Filter Model (TFM) combined with Viola and Jones face detector to significantly reduce the false positives while maintaining high accuracy. For the facial landmarking step, we improve the accuracy and the amount of the facial landmarks by readjusting the face structure to provide better geometrical information. Furthermore, we expand the face models into Multi-Resolution (MR) models with the adaptive landmark approach via landmark reduction to train the face models to be able to detect facial landmarks on face images with resolutions as low as 30x30 pixels. Our experiments show that our proposed approaches can improve the accuracy of facial landmark detection on both controlled and uncontrolled environment. Furthermore, they also show that our MR models are more robust on detecting facial components (eyebrows, eyes, nose, and mouth) on very small faces.
Citation: A. Liang, C. Wang, W. Liu, L. Li. Robust and flexible landmarks detection for uncontrolled frontal faces in the wild. Numerical Algebra, Control & Optimization, 2016, 6 (3) : 263-296. doi: 10.3934/naco.2016011
References:
[1]

T. L. Berg, A. C. Berg, J. Edwards and D. A. Forsyth, Who's in the picture?, in NIPS, (2004).

[2]

O. Çeliktutan, S. Ulukaya and B. Sankur, A comparative study of face landmarking techniques,, EURASIP Journal on Image and Video Processing, 2013 (2013).

[3]

P. Conilione and D. Wang, Fuzzy approach for semantic face image retrieval,, The Computer Journal, 55 (2012), 1130.

[4]

T. F. Cootes, C. J. Taylor, D. H. Cooper and J. Graham, Active shape models-their training and application,, Computer vision and image understanding, 61 (1995), 38.

[5]

N. Dalal and B. Triggs, Histograms of oriented gradients for human detection,, in Computer Vision and Pattern Recognition, (2005), 886.

[6]

L. Ding and A. M. Martinez, Features versus context: An approach for precise and detailed detection and delineation of faces and facial features,, IEEE Trans. Pattern Anal. Mach. Intell., 32 (2010), 2022.

[7]

P. F. Felzenszwalb, R. B. Girshick, D. McAllester and D. Ramanan, Object detection with discriminatively trained part-based models,, Pattern Analysis and Machine Intelligence, 32 (2010), 1627.

[8]

P. F. Felzenszwalb and D. P. Huttenlocher, Pictorial structures for object recognition,, International Journal of Computer Vision, 61 (2005), 55.

[9]

G. D. Forney Jr, The viterbi algorithm,, Proceedings of the IEEE, 61 (1973), 268.

[10]

Y. Freund and R. E. Schapire, A desicion-theoretic generalization of on-line learning and an application to boosting,, in Computational learning theory, (1995), 23.

[11]

J. H. Friedman, Multivariate adaptive regression splines,, The annals of statistics, 19 (1991), 1. doi: 10.1214/aos/1176347963.

[12]

R. Gross, I. Matthews, J. Cohn, T. Kanade and S. Baker, Multi-pie,, Image Vision Computing, 28 (2010), 807.

[13]

V. Jain and E. Learned-Miller, FDDB: A Benchmark for Face Detection in Unconstrained Settings,, Technical Report UM-CS-2010-009, (2010), 2010.

[14]

A. Kasinski, A. Florek and A. Schmidt, The PUT face database,, Image Processing and Communications, 13 (2008), 59.

[15]

J. C. Klontz and A. K. Jain, A case study of automated face recognition: The boston marathon bombings suspects,, IEEE Computer, 46 (2013), 91.

[16]

M. Koestinger, P. Wohlhart, P. M. Roth and H. Bischof, Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization,, in First IEEE International Workshop on Benchmarking Facial Image Analysis Technologies, (2011).

[17]

V. Le, J. Brandt, Z. Lin, L. Bourdev and T. S. Huang, Interactive facial feature localization,, in Proceedings of the 12th European conference on Computer Vision - Volume Part III, (2012), 679.

[18]

A. Liang, W. Liu, L. Li, M. R. Farid and V. Le, Accurate facial landmarks detection for frontal faces with extended tree-structured models,, in Pattern Recognition (ICPR), (2014), 538.

[19]

A. Liang, C. Wang, W. Liu and L. Li, A novel landmark detector system for multi resolution frontal faces,, in Digital lmage Computing: Techniques and Applications (DlCTA), (2014), 1.

[20]

D. G. Lowe, Distinctive image features from scale-invariant keypoints,, International journal of computer vision, 60 (2004), 91.

[21]

A. M. Martínez and R. Benavente, The AR Face Database,, Technical Report 24, (1998).

[22]

S. Milborrow, Stasm User Manual,, , (2013).

[23]

S. Milborrow and F. Nicolls, Active Shape Models with SIFT Descriptors and MARS,, VISAPP, ().

[24]

S. Milborrow and F. Nicolls, Locating facial features with an extended active shape model,, in Computer Vision-ECCV 2008, (2008), 504.

[25]

A. Torralba, K. P. Murphy and W. T. Freeman, Sharing visual features for multiclass and multiview object detection,, Pattern Analysis and Machine Intelligence, 29 (2007), 854.

[26]

P. Viola and M. J. Jones, Robust real-time face detection,, International journal of computer vision, 57 (2004), 137.

[27]

X. Xiong and F. De la Torre, Supervised descent method and its applications to face alignment,, in Computer Vision and Pattern Recognition (CVPR), (2013), 532.

[28]

W. Zhao, R. Chellappa, P. J. Phillips and A. Rosenfeld, Face recognition: A literature survey,, ACM Computing Surveys (CSUR), 35 (2003), 399.

[29]

X. Zhu and D. Ramanan, Face detection, pose estimation and landmark localization in the wild,, , ().

[30]

X. Zhu and D. Ramanan, Face detection, pose estimation, and landmark localization in the wild,, in Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2012), 2879.

show all references

References:
[1]

T. L. Berg, A. C. Berg, J. Edwards and D. A. Forsyth, Who's in the picture?, in NIPS, (2004).

[2]

O. Çeliktutan, S. Ulukaya and B. Sankur, A comparative study of face landmarking techniques,, EURASIP Journal on Image and Video Processing, 2013 (2013).

[3]

P. Conilione and D. Wang, Fuzzy approach for semantic face image retrieval,, The Computer Journal, 55 (2012), 1130.

[4]

T. F. Cootes, C. J. Taylor, D. H. Cooper and J. Graham, Active shape models-their training and application,, Computer vision and image understanding, 61 (1995), 38.

[5]

N. Dalal and B. Triggs, Histograms of oriented gradients for human detection,, in Computer Vision and Pattern Recognition, (2005), 886.

[6]

L. Ding and A. M. Martinez, Features versus context: An approach for precise and detailed detection and delineation of faces and facial features,, IEEE Trans. Pattern Anal. Mach. Intell., 32 (2010), 2022.

[7]

P. F. Felzenszwalb, R. B. Girshick, D. McAllester and D. Ramanan, Object detection with discriminatively trained part-based models,, Pattern Analysis and Machine Intelligence, 32 (2010), 1627.

[8]

P. F. Felzenszwalb and D. P. Huttenlocher, Pictorial structures for object recognition,, International Journal of Computer Vision, 61 (2005), 55.

[9]

G. D. Forney Jr, The viterbi algorithm,, Proceedings of the IEEE, 61 (1973), 268.

[10]

Y. Freund and R. E. Schapire, A desicion-theoretic generalization of on-line learning and an application to boosting,, in Computational learning theory, (1995), 23.

[11]

J. H. Friedman, Multivariate adaptive regression splines,, The annals of statistics, 19 (1991), 1. doi: 10.1214/aos/1176347963.

[12]

R. Gross, I. Matthews, J. Cohn, T. Kanade and S. Baker, Multi-pie,, Image Vision Computing, 28 (2010), 807.

[13]

V. Jain and E. Learned-Miller, FDDB: A Benchmark for Face Detection in Unconstrained Settings,, Technical Report UM-CS-2010-009, (2010), 2010.

[14]

A. Kasinski, A. Florek and A. Schmidt, The PUT face database,, Image Processing and Communications, 13 (2008), 59.

[15]

J. C. Klontz and A. K. Jain, A case study of automated face recognition: The boston marathon bombings suspects,, IEEE Computer, 46 (2013), 91.

[16]

M. Koestinger, P. Wohlhart, P. M. Roth and H. Bischof, Annotated facial landmarks in the wild: A large-scale, real-world database for facial landmark localization,, in First IEEE International Workshop on Benchmarking Facial Image Analysis Technologies, (2011).

[17]

V. Le, J. Brandt, Z. Lin, L. Bourdev and T. S. Huang, Interactive facial feature localization,, in Proceedings of the 12th European conference on Computer Vision - Volume Part III, (2012), 679.

[18]

A. Liang, W. Liu, L. Li, M. R. Farid and V. Le, Accurate facial landmarks detection for frontal faces with extended tree-structured models,, in Pattern Recognition (ICPR), (2014), 538.

[19]

A. Liang, C. Wang, W. Liu and L. Li, A novel landmark detector system for multi resolution frontal faces,, in Digital lmage Computing: Techniques and Applications (DlCTA), (2014), 1.

[20]

D. G. Lowe, Distinctive image features from scale-invariant keypoints,, International journal of computer vision, 60 (2004), 91.

[21]

A. M. Martínez and R. Benavente, The AR Face Database,, Technical Report 24, (1998).

[22]

S. Milborrow, Stasm User Manual,, , (2013).

[23]

S. Milborrow and F. Nicolls, Active Shape Models with SIFT Descriptors and MARS,, VISAPP, ().

[24]

S. Milborrow and F. Nicolls, Locating facial features with an extended active shape model,, in Computer Vision-ECCV 2008, (2008), 504.

[25]

A. Torralba, K. P. Murphy and W. T. Freeman, Sharing visual features for multiclass and multiview object detection,, Pattern Analysis and Machine Intelligence, 29 (2007), 854.

[26]

P. Viola and M. J. Jones, Robust real-time face detection,, International journal of computer vision, 57 (2004), 137.

[27]

X. Xiong and F. De la Torre, Supervised descent method and its applications to face alignment,, in Computer Vision and Pattern Recognition (CVPR), (2013), 532.

[28]

W. Zhao, R. Chellappa, P. J. Phillips and A. Rosenfeld, Face recognition: A literature survey,, ACM Computing Surveys (CSUR), 35 (2003), 399.

[29]

X. Zhu and D. Ramanan, Face detection, pose estimation and landmark localization in the wild,, , ().

[30]

X. Zhu and D. Ramanan, Face detection, pose estimation, and landmark localization in the wild,, in Proceedings of the 2012 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), (2012), 2879.

[1]

Maide Bucolo, Federica Di Grazia, Luigi Fortuna, Mattia Frasca, Francesca Sapuppo. An environment for complex behaviour detection in bio-potential experiments. Mathematical Biosciences & Engineering, 2008, 5 (2) : 261-276. doi: 10.3934/mbe.2008.5.261

[2]

Monika Muszkieta. A variational approach to edge detection. Inverse Problems & Imaging, 2016, 10 (2) : 499-517. doi: 10.3934/ipi.2016009

[3]

Michael Dellnitz, O. Junge, B Thiere. The numerical detection of connecting orbits. Discrete & Continuous Dynamical Systems - B, 2001, 1 (1) : 125-135. doi: 10.3934/dcdsb.2001.1.125

[4]

Elena Beretta, Markus Grasmair, Monika Muszkieta, Otmar Scherzer. A variational algorithm for the detection of line segments. Inverse Problems & Imaging, 2014, 8 (2) : 389-408. doi: 10.3934/ipi.2014.8.389

[5]

Liming Zhang, Tao Qian, Qingye Zeng. Edge detection by using rotational wavelets. Communications on Pure & Applied Analysis, 2007, 6 (3) : 899-915. doi: 10.3934/cpaa.2007.6.899

[6]

Jian Zhao, Fang Deng, Jian Jia, Chunmeng Wu, Haibo Li, Yuan Shi, Shunli Zhang. A new face feature point matrix based on geometric features and illumination models for facial attraction analysis. Discrete & Continuous Dynamical Systems - S, 2019, 12 (4&5) : 1065-1072. doi: 10.3934/dcdss.2019073

[7]

Yuying Shi, Ying Gu, Li-Lian Wang, Xue-Cheng Tai. A fast edge detection algorithm using binary labels. Inverse Problems & Imaging, 2015, 9 (2) : 551-578. doi: 10.3934/ipi.2015.9.551

[8]

Xiangying Meng, Gemma Huguet, John Rinzel. Type III excitability, slope sensitivity and coincidence detection. Discrete & Continuous Dynamical Systems - A, 2012, 32 (8) : 2729-2757. doi: 10.3934/dcds.2012.32.2729

[9]

Ephraim Agyingi, Tamas Wiandt, Sophia A. Maggelakis. Thermal detection of a prevascular tumor embedded in breast tissue. Mathematical Biosciences & Engineering, 2015, 12 (5) : 907-915. doi: 10.3934/mbe.2015.12.907

[10]

Dominique Zosso, Jing An, James Stevick, Nicholas Takaki, Morgan Weiss, Liane S. Slaughter, Huan H. Cao, Paul S. Weiss, Andrea L. Bertozzi. Image segmentation with dynamic artifacts detection and bias correction. Inverse Problems & Imaging, 2017, 11 (3) : 577-600. doi: 10.3934/ipi.2017027

[11]

H. Thomas Banks, Shuhua Hu, Zackary R. Kenz, Carola Kruse, Simon Shaw, John Whiteman, Mark P. Brewin, Stephen E. Greenwald, Malcolm J. Birch. Model validation for a noninvasive arterial stenosis detection problem. Mathematical Biosciences & Engineering, 2014, 11 (3) : 427-448. doi: 10.3934/mbe.2014.11.427

[12]

Habib Ammari, Josselin Garnier, Vincent Jugnon. Detection, reconstruction, and characterization algorithms from noisy data in multistatic wave imaging. Discrete & Continuous Dynamical Systems - S, 2015, 8 (3) : 389-417. doi: 10.3934/dcdss.2015.8.389

[13]

Chao Xu, Yimeng Dong, Zhigang Ren, Huachen Jiang, Xin Yu. Sensor deployment for pipeline leakage detection via optimal boundary control strategies. Journal of Industrial & Management Optimization, 2015, 11 (1) : 199-216. doi: 10.3934/jimo.2015.11.199

[14]

Martene L. Fair, Stephen L. Campbell. Active incipient fault detection in continuous time systems with multiple simultaneous faults. Numerical Algebra, Control & Optimization, 2011, 1 (2) : 211-224. doi: 10.3934/naco.2011.1.211

[15]

Marie Levakova. Effect of spontaneous activity on stimulus detection in a simple neuronal model. Mathematical Biosciences & Engineering, 2016, 13 (3) : 551-568. doi: 10.3934/mbe.2016007

[16]

Evans K. Afenya, Calixto P. Calderón. Growth kinetics of cancer cells prior to detection and treatment: An alternative view. Discrete & Continuous Dynamical Systems - B, 2004, 4 (1) : 25-28. doi: 10.3934/dcdsb.2004.4.25

[17]

Jason R. Scott, Stephen Campbell. Auxiliary signal design for failure detection in differential-algebraic equations. Numerical Algebra, Control & Optimization, 2014, 4 (2) : 151-179. doi: 10.3934/naco.2014.4.151

[18]

David L. Russell. Coefficient identification and fault detection in linear elastic systems; one dimensional problems. Mathematical Control & Related Fields, 2011, 1 (3) : 391-411. doi: 10.3934/mcrf.2011.1.391

[19]

Lok Ming Lui, Tsz Wai Wong, Wei Zeng, Xianfeng Gu, Paul M. Thompson, Tony F. Chan, Shing Tung Yau. Detection of shape deformities using Yamabe flow and Beltrami coefficients. Inverse Problems & Imaging, 2010, 4 (2) : 311-333. doi: 10.3934/ipi.2010.4.311

[20]

Fioralba Cakoni, Rainer Kress. Integral equations for inverse problems in corrosion detection from partial Cauchy data. Inverse Problems & Imaging, 2007, 1 (2) : 229-245. doi: 10.3934/ipi.2007.1.229

 Impact Factor: 

Metrics

  • PDF downloads (84)
  • HTML views (0)
  • Cited by (0)

Other articles
by authors

[Back to Top]