RUS  ENG
Full version
JOURNALS // Artificial Intelligence and Decision Making // Archive

Artificial Intelligence and Decision Making, 2015 Issue 4, Pages 81–88 (Mi iipr340)

Natural language processing

Comprehensive semantic analysis flow of news texts

A. Zaboleeva-Zotovaa, J. A. Orlovab, V. Rozalievb

a Russian Foundation for Basic Research, Moscow
b Volgograd State Technical University

Abstract: This work is devoted to the question of adaptation of text information to persons with disabilities. Deals with the extraction of key entities from the text of the news article and their visualization. Briefly reviewed and analyzed existing methods and algorithms for determining near-duplicate texts, such as TF-IDF and its modifications, Long Sent, Shingles, Lex Rand. To solve the problem of separation of the news topic the algorithm including a method of shingles. Presented several options for its parallel implementation: using technologies like CUDA, Open CL and Google App Engine, the estimated parameters of the algorithm (time, speedup compared to sequential processing), applied to the problem of analysis of news texts. Presents the example of software implementation of complex analysis of news text, based on a combination of semantic analysis and subsequent annotation of the text view in its compressed form in the format of so-called mind map.

Keywords: news text, fuzzy duplicates, shingles, TF-IDF, annotation, mind map, CUDA, Open CL, Google App Engine.



Bibliographic databases:


© Steklov Math. Inst. of RAS, 2024