Enhanced deep learning algorithm development to detect pain intensity from facial expression images

Bargshady, Ghazal ORCID: https://orcid.org/0000-0002-2557-7928 and Zhou, Xujuan and Deo, Ravinesh C. ORCID: https://orcid.org/0000-0002-2290-6749 and Soar, Jeffery ORCID: https://orcid.org/0000-0002-4964-7556 and Whittaker, Frank and Wang, Hua (2020) Enhanced deep learning algorithm development to detect pain intensity from facial expression images. Expert Systems with Applications, 149:113305. pp. 1-10. ISSN 0957-4174


Automated detection of pain intensity from facial expressions, especially from face images that show a patient's health, remains a significant challenge in the medical diagnostics and health informatics area. Expert systems that prudently analyse facial expression images, utilising an automated machine learning algorithm, can be a promising approach for pain intensity analysis in health domain. Deep neural networks and emerging machine learning techniques have made significant progress in both the feature identification, mapping and the modelling of pain intensity from facial images, with great potential to aid health practitioners in the diagnosis of certain medical conditions. Consequently, there has been significant research within the pain recognition and management area that aim to adopt facial expression datasets into deep learning algorithms to detect the pain intensity in binary classes, and also to identify pain and non-pain faces. However, the volume of research in identifying pain intensity levels in multi-classes remains rather limited. This paper reports on a new enhanced deep neural network framework designed for the effective detection of pain intensity, in four-level thresholds using a facial expression image. To explore the robustness of the proposed algorithms, the UNBC-McMaster Shoulder Pain Archive Database, comprised of human facial images, was first balanced, then used for the training and testing of the classification model, coupled with the fine-tuned VGG-Face pre-trainer as a feature extraction tool. To reduce the dimensionality of the classification model input data and extract most relevant features, Principal Component Analysis was applied, improving its computational efficiency. The pre-screened features, used as model inputs, are then transferred to produce a new enhanced joint hybrid CNN-BiLSTM (EJH-CNN-BiLSTM) deep learning algorithm comprised of convolutional neural networks, that were then linked to the joint bidirectional LSTM, for multi-classification of pain. The resulting EJH-CNN-BiLSTM classification model, tested to estimate four different levels of pain, revealed a good degree of accuracy in terms of different performance evaluation techniques. The results indicated that the enhanced EJH-CNN-BiLSTM classification algorithm was explored as a potential tool for the detection of pain intensity in multi-classes from facial expression images, and therefore, can be adopted as an artificial intelligence tool in the medical diagnostics for automatic pain detection and subsequent pain management of patients.

Statistics for USQ ePrint 38321
Statistics for this ePrint Item
Item Type: Article (Commonwealth Reporting Category C)
Refereed: Yes
Item Status: Live Archive
Additional Information: This study was funded by ARC Linkage Grant (LP150100673) held by Professor Jeffery Soar (CI) and the project was co-supervised by A/Prof Ravinesh Deo & Dr Xujuan Zhou. The industry partner was Nexus eCare led by Dr Frank Whittaker.
Faculty/School / Institute/Centre: Historic - Faculty of Business, Education, Law and Arts - School of Management and Enterprise (1 Jul 2013 - 17 Jan 2021)
Faculty/School / Institute/Centre: Current - Institute for Resilient Regions - Centre for Health Research (1 Apr 2020 -)
Date Deposited: 26 Feb 2020 02:09
Last Modified: 02 Jun 2021 04:34
Uncontrolled Keywords: facial expression; pain detection; deep neural networks; expert systems in healthcare; machine learning
Fields of Research (2008): 08 Information and Computing Sciences > 0801 Artificial Intelligence and Image Processing > 080105 Expert Systems
Fields of Research (2020): 46 INFORMATION AND COMPUTING SCIENCES > 4605 Data management and data science > 460510 Recommender systems
Socio-Economic Objectives (2008): E Expanding Knowledge > 97 Expanding Knowledge > 970111 Expanding Knowledge in the Medical and Health Sciences
E Expanding Knowledge > 97 Expanding Knowledge > 970108 Expanding Knowledge in the Information and Computing Sciences
Funding Details:
Identification Number or DOI: https://doi.org/10.1016/j.eswa.2020.113305
URI: http://eprints.usq.edu.au/id/eprint/38321

Actions (login required)

View Item Archive Repository Staff Only