Convolutional Neural Network (CNN) Based Facial Aesthetics Analysis through Dynamic Robust Losses and Ensemble Regression

The search for beauty has been pursued by mankind since its beginnings. The attempt to trace the secret of beauty has been a goal for philosophers, artists and scientists throughout human history. Nowadays, the beauty of the face receives even more interest due to the rapid development of plastic surgery and the cosmetics industry. In the last decade, several studies have shown that facial attractiveness can be learned by machines. Indeed, facial beauty prediction is sophisticated task even for human, where for one face distinct facial beauty scores can be given by different people. Thus, facial beauty prediction (FBP) has high subjective bias. On the other hand, big labelled data is required to create an efficient machine learning system for Facial beauty prediction, especially for deep learning methods.

The goal of this work is to leverage the advances in Deep Learning architectures to provide stable and accurate face beauty estimation from static face images.

In this work, we propose a system that exploits the diversity of learners as shown in Figure 1. We present two main proposals. First, we propose to combine two different CNN architectures into a single architecture (called the two-branch architecture) that is trained end-to-end. Second, we propose to build an ensemble of regressions where the final prediction is given by the average of all predictions. The latter solution does not need to be trained on new validation sets. More specifically, we propose ensemble regressions using one-branch architectures (ResneXt-50 and Inception-v3) and our proposed two-branch architecture (REX-INCEP) trained with different loss functions. Four loss functions are used in our approach, namely MSE, dynamic ParamSmoothL1, dynamic Huber and dynamic Tukey.

 Our approach is evaluated on the SCUT-FBP5500 database using the two evaluation scenarios provided by the database creators: 60%-40% split and five-fold cross-validation. In both evaluation scenarios, our approach outperforms the state of the art on several metrics. These comparisons highlight the effectiveness of the proposed solutions for FBP. They also show that the proposed dynamic robust losses lead to more flexible and accurate estimators (

This collaborative work are between several research teams from :

  • Institute of Applied Sciences and Intelligent Systems, National Research Council of Italy, Lecce, 73100, Italy. ( Dr Fares Bougourzi, PostDoc)
  • University of the Basque Country UPV/EHU, San Sebastian 20018, Basque Country, Spain, IKERBASQUE, Basque Foundation for Science, Bilbao, 48012, Basque Country, Spain (Pr Fadi Dornaika).
  • Universite Polytechnique Hauts-de-France, Université de Lille, CNRS, UMR 8520, Valenciennes, 59313, Hauts-de-France, France ( Pr Abdelmalik Taleb-Ahmed).

The approach developed and the first results obtained have been published in reference [1].

 [1] F. Bougourzi, F. Dornaika, A. Taleb-Ahmed,   Deep learning based face beauty prediction via dynamic robust losses and ensemble regression, Knowledge-Based Systems, Vol  242, pp 108246, 2022.

Figure 1: Our Proposed EN-CNN approach.