The modeling of human facial pain intensity based on Temporal Convolutional Networks trained with video frames in HSV color space

Bargshady, Ghazal ORCID: https://orcid.org/0000-0002-2557-7928 and Zhou, Xujuan and Deo, Ravinesh C. ORCID: https://orcid.org/0000-0002-2290-6749 and Soar, Jeffrey ORCID: https://orcid.org/0000-0002-4964-7556 and Whittaker, Frank and Wang, Hua (2020) The modeling of human facial pain intensity based on Temporal Convolutional Networks trained with video frames in HSV color space. Applied Soft Computing, 97 (Part A):106805. pp. 1-14. ISSN 1568-4946


Abstract

An accurate detection and management of pain, measured through its relative intensity, plays an important role in the treatment of disease and reducing a patient’s discomfort. As it is relatively difficult to assess, describe, evaluate and manage the pain level using a patient’s self-report, automated pain-detecting tools can provide useful information to assist in the management of pain intensity. This study proposes a new predictive modeling framework that employs a modified Temporal Convolutional Network (TCN) algorithm to recognize the pain intensity prevalent in patients’ video frames collected as part of UNBC-McMaster Shoulder Pain Archive and MIntPAIN databases. The inputs of the proposed TCN network is composed of the extracted and reduced face image features from a fine-tuned VGG-Face and principal component analysis (PCA) with Hue, Saturation, Value (HSV) color spaces video images. The results of TCN based predictive model, employing a long short-term memory (LSTM) model as well as other state-of-the art models, show that the proposed approach performs faster with a high level of efficiency. This is demonstrated by the low magnitude of error metrics (i.e., Mean Squared Error = 0.0629, Mean Absolute Error = 0.1021, correctness validation results represented by Area under Curve = 85% and accuracy metric = 92.44%). Considering the efficiency of the proposed TCN framework, integrating fine-tuned VGG-Face and PCA with Hue, Saturation, Value (HSV) color spaces video images for pain intensity estimation, the present study affirms that the new method can be adopted as an automatic health informatics tool, mainly for pain detection, and subsequently, implemented in the pain management area.


Statistics for USQ ePrint 39938
Statistics for this ePrint Item
Item Type: Article (Commonwealth Reporting Category C)
Refereed: Yes
Item Status: Live Archive
Faculty/School / Institute/Centre: Historic - Faculty of Business, Education, Law and Arts - School of Management and Enterprise (1 Jul 2013 - 17 Jan 2021)
Faculty/School / Institute/Centre: Current - Faculty of Health, Engineering and Sciences - School of Sciences (6 Sep 2019 -)
Date Deposited: 20 Oct 2020 04:27
Last Modified: 19 Jul 2021 01:35
Uncontrolled Keywords: Temporal convolutional network; Facial expression; Pain detection; HSV color space; Video analysis
Fields of Research (2008): 08 Information and Computing Sciences > 0801 Artificial Intelligence and Image Processing > 080105 Expert Systems
08 Information and Computing Sciences > 0801 Artificial Intelligence and Image Processing > 080109 Pattern Recognition and Data Mining
08 Information and Computing Sciences > 0801 Artificial Intelligence and Image Processing > 080104 Computer Vision
08 Information and Computing Sciences > 0801 Artificial Intelligence and Image Processing > 080199 Artificial Intelligence and Image Processing not elsewhere classified
01 Mathematical Sciences > 0104 Statistics > 010402 Biostatistics
08 Information and Computing Sciences > 0801 Artificial Intelligence and Image Processing > 080108 Neural, Evolutionary and Fuzzy Computation
08 Information and Computing Sciences > 0801 Artificial Intelligence and Image Processing > 080106 Image Processing
Fields of Research (2020): 46 INFORMATION AND COMPUTING SCIENCES > 4605 Data management and data science > 460510 Recommender systems
46 INFORMATION AND COMPUTING SCIENCES > 4699 Other information and computing sciences > 469999 Other information and computing sciences not elsewhere classified
46 INFORMATION AND COMPUTING SCIENCES > 4603 Computer vision and multimedia computation > 460304 Computer vision
46 INFORMATION AND COMPUTING SCIENCES > 4699 Other information and computing sciences > 469999 Other information and computing sciences not elsewhere classified
49 MATHEMATICAL SCIENCES > 4905 Statistics > 490502 Biostatistics
46 INFORMATION AND COMPUTING SCIENCES > 4699 Other information and computing sciences > 469999 Other information and computing sciences not elsewhere classified
46 INFORMATION AND COMPUTING SCIENCES > 4603 Computer vision and multimedia computation > 460306 Image processing
Funding Details:
Identification Number or DOI: https://doi.org/10.1016/j.asoc.2020.106805
URI: http://eprints.usq.edu.au/id/eprint/39938

Actions (login required)

View Item Archive Repository Staff Only