What is Next when Sequential Prediction Meets Implicitly Hard Interaction?

Hu, Kaixi and Li, Lin and Xie, Qing and Liu, Jianquan and Tao, Xiaohui ORCID: https://orcid.org/0000-0002-0020-077X (2021) What is Next when Sequential Prediction Meets Implicitly Hard Interaction? In: 30th ACM International Conference on Information and Knowledge Management (CIKM 2021), 1 Nov - 5 Nov 2021, Queensland, Australia.

[img]
Preview
Text (Accepted Version)
CIKM-2021.pdf

Download (1MB) | Preview

Abstract

Hard interaction learning between source sequences and their next targets is challenging, which exists in a myriad of sequential prediction tasks. During the training process, most existing methods focus on explicitly hard interactions caused by wrong responses. However, a model might conduct correct responses by capturing a subset of learnable patterns, which results in implicitly hard interactions with some unlearned patterns. As such, its generalization performance is weakened. The problem gets more serious in sequential prediction due to the interference of substantial similar candidate targets.

To this end, we propose a Hardness Aware Interaction Learning framework (HAIL) that mainly consists of two base sequential learning networks and mutual exclusivity distillation (MED). The base networks are initialized differently to learn distinctive view patterns, thus gaining different training experiences. The experiences in the form of the unlikelihood of correct responses are drawn from each other by MED, which provides mutual exclusivity knowledge to figure out implicitly hard interactions. Moreover, we deduce that the unlikelihood essentially introduces additional gradients to push the pattern learning of correct responses. Our framework can be easily extended to more peer base networks. Evaluation is conducted on four datasets covering cyber and physical spaces. The experimental results demonstrate that our framework outperforms several state-of-the-art methods in terms of top-k based metrics.


Statistics for USQ ePrint 46209
Statistics for this ePrint Item
Item Type: Conference or Workshop Item (Commonwealth Reporting Category E) (Paper)
Refereed: Yes
Item Status: Live Archive
Additional Information: Files associated with this item cannot be displayed due to copyright restrictions.
Faculty/School / Institute/Centre: Historic - Faculty of Health, Engineering and Sciences - School of Sciences (6 Sep 2019 - 31 Dec 2021)
Faculty/School / Institute/Centre: Historic - Faculty of Health, Engineering and Sciences - School of Sciences (6 Sep 2019 - 31 Dec 2021)
Date Deposited: 11 May 2022 03:49
Last Modified: 30 May 2022 03:47
Uncontrolled Keywords: sequential prediction, hard interaction, unlikelihood, knowledge distillation
Fields of Research (2008): 08 Information and Computing Sciences > 0801 Artificial Intelligence and Image Processing > 080109 Pattern Recognition and Data Mining
Fields of Research (2020): 46 INFORMATION AND COMPUTING SCIENCES > 4605 Data management and data science > 460501 Data engineering and data science
46 INFORMATION AND COMPUTING SCIENCES > 4603 Computer vision and multimedia computation > 460308 Pattern recognition
Socio-Economic Objectives (2008): E Expanding Knowledge > 97 Expanding Knowledge > 970108 Expanding Knowledge in the Information and Computing Sciences
Socio-Economic Objectives (2020): 28 EXPANDING KNOWLEDGE > 2801 Expanding knowledge > 280115 Expanding knowledge in the information and computing sciences
22 INFORMATION AND COMMUNICATION SERVICES > 2204 Information systems, technologies and services > 220403 Artificial intelligence
Identification Number or DOI: https://doi.org/10.1145/3459637.3482492
URI: http://eprints.usq.edu.au/id/eprint/46209

Actions (login required)

View Item Archive Repository Staff Only