RUS  ENG
Full version
JOURNALS // Informatsionnye Tekhnologii i Vychslitel'nye Sistemy // Archive

Informatsionnye Tekhnologii i Vychslitel'nye Sistemy, 2019 Issue 4, Pages 60–69 (Mi itvs363)

DATA PROCESSING AND ANALYSIS

Effective clustering of a text sample depending on the different parameterization of this sample

E. A. Golovastovaa, D. N. Krasotinb

a Lomonosov Moscow State University, Moscow, Russia
b CJSC “MNITI”, Moscow, Russia

Abstract: The Internet becomes the primary means of receiving text news. As a result, there is a necessity in automated processing of large data amount. One of the most important tasks is the automated cultivation of text information. In this paper we will consider the problem of effective clustering for objects from text sample. The most common representation of the text set is the matrix, which elements are the statistical measure values calculated on the basis of the word frequency. In opposition to we suggest parametrization by the text key words. We use two methods to provide the clustering: K-means and Dbscan. This paper considers the analysis of mentioned methods and provide comparison of the clustering quality results, which depend on various text parameterization and the used algorithm.

Keywords: Clustering, text set, sample parameterization, tf-idf-measure, keywords, effective method.

DOI: 10.14357/20718632190406



© Steklov Math. Inst. of RAS, 2024