Ensemble neural network approach detecting pain intensity from facial expressions

Bargshady, Ghazal ORCID: https://orcid.org/0000-0002-2557-7928 and Zhou, Xujuan and Deo, Ravinesh C. ORCID: https://orcid.org/0000-0002-2290-6749 and Soar, Jeffrey ORCID: https://orcid.org/0000-0002-4964-7556 and Whittaker, Frank and Wang, Hua (2020) Ensemble neural network approach detecting pain intensity from facial expressions. Artificial Intelligence in Medicine:101954. pp. 1-27. ISSN 0933-3657


Abstract

This paper reports on research to design an ensemble deep learning framework that integrates fine-tuned, three-stream hybrid deep neural network (i.e., Ensemble Deep Learning Model, EDLM), employing Convolutional Neural Network (CNN) to extract facial image features, detect and accurately classify the pain. To develop the approach, the VGGFace is fine-tuned and integrated with Principal Component Analysis and employed to extract features in images from the Multimodal Intensity Pain database at the early phase of the model fusion. Subsequently, a late fusion, three layers hybrid CNN and recurrent neural network algorithm is developed with their outputs merged to produce image-classified features to classify pain levels. The EDLM model is then benchmarked by means of a single-stream deep learning model including several competing models based on deep learning methods. The results obtained indicate that the proposed framework is able to outperform the competing methods, applied in a multi-level pain detection database to produce a feature classification accuracy that exceeds 89%, with a receiver operating characteristic of 93%. To evaluate the generalization of the proposed EDLM model, the UNBC-McMaster Shoulder Pain dataset is used as a test dataset for all of the modelling experiments, which reveals the efficacy of the proposed method for pain classification from facial images. The study concludes that the proposed EDLM model can accurately classify pain and generate multi-class pain levels for potential applications in the medical informatics area, and should therefore, be explored further in expert systems for detecting and classifying the pain intensity of patients, and automatically evaluating the patients’ pain level accurately.


Statistics for USQ ePrint 39525
Statistics for this ePrint Item
Item Type: Article (Commonwealth Reporting Category C)
Refereed: Yes
Item Status: Live Archive
Faculty/School / Institute/Centre: Current - Faculty of Business, Education, Law and Arts - School of Management and Enterprise (1 July 2013 -)
Faculty/School / Institute/Centre: Current - Faculty of Health, Engineering and Sciences - School of Sciences (6 Sept 2019 -)
Date Deposited: 14 Sep 2020 00:27
Last Modified: 14 Sep 2020 01:55
Uncontrolled Keywords: ensemble neural network, pain detection, facial expression, deep learning
Fields of Research (2008): 08 Information and Computing Sciences > 0801 Artificial Intelligence and Image Processing > 080109 Pattern Recognition and Data Mining
08 Information and Computing Sciences > 0801 Artificial Intelligence and Image Processing > 080104 Computer Vision
08 Information and Computing Sciences > 0801 Artificial Intelligence and Image Processing > 080199 Artificial Intelligence and Image Processing not elsewhere classified
Socio-Economic Objectives (2008): E Expanding Knowledge > 97 Expanding Knowledge > 970108 Expanding Knowledge in the Information and Computing Sciences
Funding Details:
Identification Number or DOI: 10.1016/j.artmed.2020.101954
URI: http://eprints.usq.edu.au/id/eprint/39525

Actions (login required)

View Item Archive Repository Staff Only