Detecting outlying subspaces for high-dimensional data: the new task, algorithms and performance

Zhang, Ji and Wang, Hai (2006) Detecting outlying subspaces for high-dimensional data: the new task, algorithms and performance. Knowledge and Information Systems, 10 (3). pp. 333-355. ISSN 0219-1377

PDF (Accepted Version)

Download (213Kb)


[Abstract]: In this paper, we identify a new task for studying the outlying degree (OD) of high-dimensional data, i.e. finding the subspaces (subsets of features)
in which the given points are outliers, which are called their outlying subspaces. Since the state-of-the-art outlier detection techniques fail to handle this
new problem, we propose a novel detection algorithm, called High-Dimension Outlying subspace Detection (HighDOD), to detect the outlying subspaces of
high-dimensional data efficiently. The intuitive idea of HighDOD is that we measure the OD of the point using the sum of distances between this point and its k nearest neighbors. Two heuristic pruning strategies are proposed to realize fast pruning in the subspace search and an efficient dynamic subspace search method with a sample-based learning process has been implemented. Experimental results show that HighDOD is efficient and outperforms other searching alternatives such as the naive top–down, bottom–up and random search methods, and the existing
outlier detection methods cannot fulfill this new task effectively.

Statistics for USQ ePrint 5642
Statistics for this ePrint Item
Item Type: Article (Commonwealth Reporting Category C)
Refereed: Yes
Item Status: Live Archive
Additional Information (displayed to public): Author's version deposited in accordance with the copyright policy of the publisher. The original publication is available at Publisher permission for book chapters in email dated 21/8/07
Depositing User: Dr Ji Zhang
Faculty / Department / School: Historic - Faculty of Sciences - Department of Maths and Computing
Date Deposited: 28 Sep 2009 05:11
Last Modified: 02 Jul 2013 23:23
Uncontrolled Keywords: outlying subspace; high-dimensional data; outlier detection; dynamic subspace search
Fields of Research (FoR): 08 Information and Computing Sciences > 0801 Artificial Intelligence and Image Processing > 080109 Pattern Recognition and Data Mining

Actions (login required)

View Item Archive Repository Staff Only