Road pavement assessment of the North-West Federal District using sentiment analysis of the Internet user reviews

As a result of the analysis, it was revealed that social networks, thematic communities, transport portals are a source of actual information about the traffic situation. The article deals with the task of analyzing the road pavement assessment of the North-West Federal District from reviews posted in the web. To solve this problem, a system for automatic classification of reviews based on the sentiment classifier has been developed. The crawler was developed using the Scrapy framework in Python3 and collected reviews from the site http://autostrada.info/ru. The methods of vectorization and lemmatization of texts and their implementation in the Scikit-Learn library are considered: Bag-of-Words, N-gram, CountVectorizer and TF-IDF Vectorizer. For the classification, a naive Bayes algorithm and a linear classifier model with optimization of stochastic gradient descent were used. As a training sample, a base of marked reviews from the Twitter resource was used. The classifier was trained, during which the cross-validation strategy and the ShuffleSplit method were used. According to the results of validation, the linear model with the N-gram scheme and the TF-IDF Vectorizer turned out to be the best. During the approbation of the developed system, the collection and analysis of feedback related to the quality of transport networks in the North-West Federal District was conducted. Based on the results, a color marking of the roads was produced, reflecting the visibility of the research results. Conclusions and prospects for the further development of this study are given.