RUS  ENG
Full version
JOURNALS // Informatics and Automation // Archive

Informatics and Automation, 2022 Issue 21, volume 2, Pages 275–310 (Mi trspy1191)

Information Security

Comprehensive defense system against vulnerabilities based on return-oriented programming

I. Lubkin, V. Zolotarev

Reshetnev Siberian State University of Science and Technology

Abstract: It is difficult or impossible to develop software without included errors. Errors can lead to an abnormal order of machine code execution during data transmission to a program. Program splitting into routines causes possible attacks by using return instructions from these routines. Most of existing security tools need to apply program source codes to protect against such attacks. The proposed defensive method is intended to a comprehensive solution to the problem. Firstly, it makes it difficult for an attacker to gain control over program execution, and secondly, the number of program routines, which can be used during the attack, decreases. Specific security code insertion is used at the beginning and end of the routines to make it complicated to gain control over the program execution. The return address is kept secure during a call of the protected routine, and the protected routine is restored after its execution if it was damaged by the attacker. To reduce the number of suitable routines for attacks, it was suggested to use synonymous substitutions of instructions that contain dangerous values. It should be mentioned that proposed defensive measures do not affect the original application's algorithm. To confirm the effectiveness of the described defensive method, software implementation and its testing were accomplished. Acknowledging controls were conducted using synthetic tests, performance tests and real programs. Results of testing have demonstrated the reliability of the proposed measures. It ensures the elimination of program routines suitable for attacks and ensures the impossibility of using standard return instructions for conducting attacks. Performance tests have shown a 14% drop in the operating speed, which approximately matches the level of the nearest analogues. The application of the proposed solution declines the number of possible attack scenarios, and its applicability level is higher in comparison with analogues.

Keywords: vulnerability, remote code execution, code protection, RoP, code insertion.

UDC: 006.72

Received: 05.09.2021

DOI: 10.15622/ia.21.2.3



© Steklov Math. Inst. of RAS, 2024