Enhanced sequence labeling based on latent variable conditional random fields

Lin, Jerry Chun-Wei and Shao, Yinan and Zhang, Ji and Yun, Unil (2020) Enhanced sequence labeling based on latent variable conditional random fields. Neurocomputing, 403. pp. 431-440. ISSN 0925-2312

[img]
Preview
Text (Published Version)
Pub version.pdf
Available under License Creative Commons Attribution 4.0.

Download (1MB) | Preview

Abstract

Natural language processing is a useful processing technique of language data, such as text and speech. Sequence labeling represents the upstream task of many natural language processing tasks, such as machine translation, text classification, and sentiment classification. In this paper, the focus is on the sequence labeling task, in which semantic labels are assigned to each unit of a given input sequence. Two frameworks of latent variable conditional random fields (CRF) models (called LVCRF-I and LVCRF-II) are proposed, which use the encoding schema as a latent variable to capture the latent structure of the hidden variables and the observed data. Among the two designed models, the LVCRF-I model focuses on the sentence level, while the LVCRF-II works in the word level, to choose the best encoding schema for a given input sequence automatically without handcraft features. In the experiments, the two proposed models are verified by four sequence prediction tasks, including named entity recognition (NER), chunking, reference parsing and POS tagging. The proposed frameworks achieve better performance without using other handcraft features than the conventional CRF model. Moreover, these designed frameworks can be viewed as a substitution of the conventional CRF models. In the commonly used LSTM-CRF models, the CRF layer can be replaced with our proposed framework as they use the same training and inference procedure. The experimental results show that the proposed models exhibit latent variable and provide competitive and robust performance on all three sequence prediction tasks.


Statistics for USQ ePrint 38695
Statistics for this ePrint Item
Item Type: Article (Commonwealth Reporting Category C)
Refereed: Yes
Item Status: Live Archive
Additional Information: This is an open access article under the CC BY license (http://creativecommons.org/licenses/by/4.0/).
Faculty/School / Institute/Centre: Current - Faculty of Health, Engineering and Sciences - School of Sciences (6 Sept 2019 -)
Faculty/School / Institute/Centre: Current - Institute for Resilient Regions - Centre for Health Research (1 Apr 2020 -)
Date Deposited: 09 Jun 2020 06:16
Last Modified: 12 Jun 2020 04:56
Uncontrolled Keywords: encoding schema, latent CRF, natural language processing, sequence labeling
Fields of Research (2008): 08 Information and Computing Sciences > 0801 Artificial Intelligence and Image Processing > 080109 Pattern Recognition and Data Mining
Identification Number or DOI: 10.1016/j.neucom.2020.04.102
URI: http://eprints.usq.edu.au/id/eprint/38695

Actions (login required)

View Item Archive Repository Staff Only