RICF: Dynamic analysis of integer arithmetic overflow vulnerability via finite state machine

Integer arithmetic overflow vulnerabilities detection has close relationship with execute program running status. Finite state machine is a behavior model composed of a finite number of states, transitions between those states, and actions. A dynamic analysis method of Run-time Integer Checking via Finite state machine (RICF) is proposed. The proposed method also includes designing the decision tree according to integer arithmetic overflow features; decompiling executing programs into assembly instructions and selected the arithmetic instructions such as add, sub, multiply and division instances from the assembly language to a program test. After these arithmetic instructions with status flag are transformed into finite state machine grammar, a data arithmetic procedure is clearly formed between variables and registers via RICF. The program test results indicate the proposed method is effective for dynamic analysis of integer arithmetic overflow.