Alternative assessment of significance of the results of expert assessment in strict ranking method

The paper presents an alternative embodiment of assessment of the coherence of the experts’ opinions in processing the expert estimations obtained on the basis of strict implementation of the ranking method. The existing approaches (based on concordance coefficient, hypothesis testing, calculation of the variation factor) consider the expert opinions as independent identically distributed random variables that fundamentally do not correspond to the essence of expert evaluation, which is aimed at identifying the deterministic component in the evaluated objects or criteria. A new approach to building up a significance factor of expert assessment is based on the algebraic methods of the framing theory. In the proposed method, a pair of expert assessments is presented in the form of substitution and is decomposed into transpositions. The minimum number of transpositions in the formulation is considered as a measure of the discordance of expert opinions. The paper provides a visual geometric interpretation of the proposed method and justification of its correctness and finding of substitution, providing the maximum assessment value. The obtained value is normalized by its maximum value for the changes in the result ranging from 0 to 1. The maximum significance coefficient for two experts is found. The results can be generalized in case, when it is necessary to take into account the competence of experts, as well as the level of the importance of the objects.