Detecting metamorphic malwares using code graphs

Malware writers and detectors have been running an endless battle. Self-defense is the weapon most malware writers prepare against malware detectors. Malware writers have tried to evade the improved detection techniques of anti-virus(AV) products. Packing and code obfuscation are two popular evasion techniques. When these techniques are applied to malwares, they are able to change their instruction sequence while maintaining their intended function. We propose a detection mechanism defeating these self-defense techniques to improve malware detection. Since an obfuscated malware is able to change the syntax of its code while preserving its semantics, the proposed mechanism uses the semantic invariant. We convert the API call sequence of the malware into a graph, commonly known as a call graph, to extract the semantic of the malware. The call graph can be reduced to a code graph used for semantic signatures of the proposed mechanism. We show that the code graph can represent the characteristics of a program exactly and uniquely. Next, we evaluate the proposed mechanism by experiment. The mechanism has an 91% detection ratio of real-world malwares and detects 300 metamorphic malwares that can evade AV scanners. In this paper, we show how to analyze malwares by extracting program semantics using static analysis. It is shown that the proposed mechanism provides a high possibility of detecting malwares even when they attempt self-protection.