iVIBRATE: Interactive Visualization Based Framework for Clustering Large Datasets

TitleiVIBRATE: Interactive Visualization Based Framework for Clustering Large Datasets
Publication TypeJournal Article
Year of Publication2006
AuthorsKeke Chen, Ling Liu
Keywords. There are three known problems with this framework, especially in large datasets. Automated algorithms and statistical methods are typically not effective in handling such particular clusters. The second problem is how to effectively label the entire data on disk (disk-labeling) without introducing additi, including the solutions for dealing with outliers, irregular clusters, sampling/summarization &#161 iterative cluster analysis &#161 disk-labeling&#39, there is an astounding growth in the amount of data produced and made available through the cyberspace. Efficient and high quality clustering of large datasets continues to be one of the most important problems in largescale data analysis. A commonly use, we describe iVIBRATE &#161 an interactive-visualization based three-phase framework for clustering large datasets. The two main components of iVIBRATE are its VISTA visual cluster rendering subsystem, which demand effective solutions. The first problem is how to effectively define and validate irregularly shaped clusters, which invites human into the large-scale iterative clustering process through interactive visualization, which offers visualization-guided disk-labeling solutions that are effective in dealing with outliers, With continued advances in communication network technology and sensing technology
Abstract

With continued advances in communication network technology and sensing technology, there is an astounding growth in the amount of data produced and made available through the cyberspace. Efficient and high quality clustering of large datasets continues to be one of the most important problems in largescale data analysis. A commonly used methodology for cluster analysis on large datasets is the three-phase framework of 'sampling/summarization &#161 iterative cluster analysis &#161 disk-labeling'. There are three known problems with this framework, which demand effective solutions. The first problem is how to effectively define and validate irregularly shaped clusters, especially in large datasets. Automated algorithms and statistical methods are typically not effective in handling such particular clusters. The second problem is how to effectively label the entire data on disk (disk-labeling) without introducing additional errors, including the solutions for dealing with outliers, irregular clusters, and cluster boundary extension. The third problem is the lack of research about the issues for effectively integrating the three phases. In this paper, we describe iVIBRATE &#161 an interactive-visualization based three-phase framework for clustering large datasets. The two main components of iVIBRATE are its VISTA visual cluster rendering subsystem, which invites human into the large-scale iterative clustering process through interactive visualization, and its Adaptive ClusterMap Labeling subsystem, which offers visualization-guided disk-labeling solutions that are effective in dealing with outliers, irregular clusters, and cluster boundary extension. Another important contribution of iVIBRATE development is the identification of special issues presented in integrating the two components and the sampling approach into a coherent framework, and the solutions to improve the reliability of the framework and to minimize the amount of errors generated throughout the cluster analysis process. We study the effectiveness of the iVIBRATE framework through a walkthrough example dataset of a million records and experimentally evaluate the iVIBRATE approach using both real-life datasets and synthetic datasets. Our results show that iVIBRATE can efficiently involve the user into the clustering process and generate high-quality clustering results for large datasets.

Full Text

Keke Chen and Ling Liu: 'iVIBRATE: Interactive Visualization Based Framework for Clustering Large Datasets' ACM Transactions on Information Systems (TOIS), Vol24, issue 2, 2006 . [pdf]
publisher: ACM Transactions on Information Systems
year: 2006
hasURL: http://knoesis.wright.edu/library/publications/Keke/ivibrate_TOIS.pdf