The use of web-crawlers in technology of concrete historical investigation support

The article is devoted to the further development of the distributed technology of concrete historical investigation support based on the principles of crowdsourcing and focused on a wide range of users which are nonprofessional historians and biographers. Development is carried out through the automation of one of the main types of Internet searches (indirect Internet search) used in biographical research. The article analyzes the possible approaches to the automation of Internet search taking into account the specifics of concrete historical investigation. The use of web-crawlers is substantiated and the requirements for them arising from the distinctive of this technology are formulated. The possibility of using ready-made solutions is estimated. The necessary changes in the object model of the technology and the modifications of its algorithms related to indirect Internet search are described. As an additional measure to reduce the difficulty of indirect Internet search, the new mechanism for automating of the interaction of the technology users which execute their investigations in similar directions is proposed and described in detail.