Explainable Uncertainty-Aware Deep Learning. Application to emotion perception

Financiador: MINISTERIO DE CIENCIA, INNOVACIÓN Y UNIVERSIDADES

Convocatoria: Convocatoria 2018 "Retos Investigación": Proyectos I+D+I, Del Programa de I+D+I Orientada a los Retos de la Sociedad, en el marco del Plan Estatal de Investigación Científica y Técnica y de Innovación 2017-2020 (MINISTERIO DE CIENCIA, INNOVACIÓN Y UNIVERSIDADES)

De ámbito Nacional.

We build this project on top of our previous research (joint grants TIN2015-66951-C2 & 2-R), where we developed and published highly robust DL methods. We propose to continue these lines dealing essentially with three topics: 1.- Uncertainty modelling, where will try to develop models able to deliver a second output linked to the prediction, that encodes its degree of certainty. 2.- The incorporation of cross-modality to bridge information from different domains (Domain Adaptation) and improve DL methods with all the data available. 3.- The consideration of the Explainability in DL models, i.e. not only obtain a prediction and a confidence of this prediction, but also gain insights about the reason behind each decision, moving from the current black box paradigm inherent from DL approaches to more explainable, accountable and auditable models. We focus the project on the interaction of these three topics rather that proposing deep and complete research on each one separately. Our work plan combines the experience of both subproject researchers on uncertainty modeling, explainability and Domain Adaptation. In this particular subproject we will apply the theory jointly developed in the consortium from these three topics to the automated emotion perception problem. DL methods are eager for labelled training data, and annotation costs by expert Psychologists of large scale data are unaffordable. Nevertheless, there exist large amounts of unlabelled data publicly available (movies, Internet images, etc.), usually from different modalities (video, image, text). We conjecture that uncertainty modelling can be used inside an active learning framework to perform semi-supervised learning strategies on labelled and large sets of unlabelled samples. In addition, we propose to take benefit from all the informational cues related to emotion perception (especially video and sound) and use domain adaptation techniques to further enhance the accuracies.