UnThemida: Commercial obfuscation technique analysis with a fully obfuscated program

Jae Hyuk Suk, Jae Yung Lee, Hongjoo Jin, In-Seok Kim, Dong Hoon Lee

Research output: Contribution to journalArticle

1 Citation (Scopus)

Abstract

The main goal of code obfuscation is to make software more difficult to reverse engineer. These techniques modify data structures and control flow while retaining the functionality of the original program. Although obfuscation is a useful method for protecting programs, it can also be used to protect malware. This raises concerns that malware could use code obfuscation to avoid detection by antivirus software. It is very difficult to analyze the functionality of obfuscated malware before it has been deobfuscated. Furthermore, commercial obfuscation tools allow malware authors to apply multiple obfuscation options simultaneously, and current deobfuscation techniques cannot handle this situation. In this study, we analyzed a well-known commercial obfuscation tool called Themida. We applied its many obfuscation options to a program and implemented a tool to recover the original code and data. We extracted features from obfuscated programs and analyzed their control flow. Our tool is based on these features and the control flow patterns and can identify whether Themida has been applied to the program and which obfuscation options have been used. Finally, we suggested a method for recovering the import address table of programs by using dynamic binary instrumentation. The proposed rules and algorithms can almost completely recover the APIs of programs even though they are hidden by obfuscation options provided by Themida.

Original languageEnglish
JournalSoftware - Practice and Experience
DOIs
Publication statusAccepted/In press - 2018 Jan 1

Fingerprint

Flow control
Application programming interfaces (API)
Flow patterns
Data structures
Engineers
Malware

Keywords

  • Debugging
  • Packer
  • Reverse engineering
  • Software implementation
  • Software protection

ASJC Scopus subject areas

  • Software

Cite this

UnThemida : Commercial obfuscation technique analysis with a fully obfuscated program. / Suk, Jae Hyuk; Lee, Jae Yung; Jin, Hongjoo; Kim, In-Seok; Lee, Dong Hoon.

In: Software - Practice and Experience, 01.01.2018.

Research output: Contribution to journalArticle

@article{37c7d1fb3d2c44a6af925587d398ffaf,
title = "UnThemida: Commercial obfuscation technique analysis with a fully obfuscated program",
abstract = "The main goal of code obfuscation is to make software more difficult to reverse engineer. These techniques modify data structures and control flow while retaining the functionality of the original program. Although obfuscation is a useful method for protecting programs, it can also be used to protect malware. This raises concerns that malware could use code obfuscation to avoid detection by antivirus software. It is very difficult to analyze the functionality of obfuscated malware before it has been deobfuscated. Furthermore, commercial obfuscation tools allow malware authors to apply multiple obfuscation options simultaneously, and current deobfuscation techniques cannot handle this situation. In this study, we analyzed a well-known commercial obfuscation tool called Themida. We applied its many obfuscation options to a program and implemented a tool to recover the original code and data. We extracted features from obfuscated programs and analyzed their control flow. Our tool is based on these features and the control flow patterns and can identify whether Themida has been applied to the program and which obfuscation options have been used. Finally, we suggested a method for recovering the import address table of programs by using dynamic binary instrumentation. The proposed rules and algorithms can almost completely recover the APIs of programs even though they are hidden by obfuscation options provided by Themida.",
keywords = "Debugging, Packer, Reverse engineering, Software implementation, Software protection",
author = "Suk, {Jae Hyuk} and Lee, {Jae Yung} and Hongjoo Jin and In-Seok Kim and Lee, {Dong Hoon}",
year = "2018",
month = "1",
day = "1",
doi = "10.1002/spe.2622",
language = "English",
journal = "Software - Practice and Experience",
issn = "0038-0644",
publisher = "John Wiley and Sons Ltd",

}

TY - JOUR

T1 - UnThemida

T2 - Commercial obfuscation technique analysis with a fully obfuscated program

AU - Suk, Jae Hyuk

AU - Lee, Jae Yung

AU - Jin, Hongjoo

AU - Kim, In-Seok

AU - Lee, Dong Hoon

PY - 2018/1/1

Y1 - 2018/1/1

N2 - The main goal of code obfuscation is to make software more difficult to reverse engineer. These techniques modify data structures and control flow while retaining the functionality of the original program. Although obfuscation is a useful method for protecting programs, it can also be used to protect malware. This raises concerns that malware could use code obfuscation to avoid detection by antivirus software. It is very difficult to analyze the functionality of obfuscated malware before it has been deobfuscated. Furthermore, commercial obfuscation tools allow malware authors to apply multiple obfuscation options simultaneously, and current deobfuscation techniques cannot handle this situation. In this study, we analyzed a well-known commercial obfuscation tool called Themida. We applied its many obfuscation options to a program and implemented a tool to recover the original code and data. We extracted features from obfuscated programs and analyzed their control flow. Our tool is based on these features and the control flow patterns and can identify whether Themida has been applied to the program and which obfuscation options have been used. Finally, we suggested a method for recovering the import address table of programs by using dynamic binary instrumentation. The proposed rules and algorithms can almost completely recover the APIs of programs even though they are hidden by obfuscation options provided by Themida.

AB - The main goal of code obfuscation is to make software more difficult to reverse engineer. These techniques modify data structures and control flow while retaining the functionality of the original program. Although obfuscation is a useful method for protecting programs, it can also be used to protect malware. This raises concerns that malware could use code obfuscation to avoid detection by antivirus software. It is very difficult to analyze the functionality of obfuscated malware before it has been deobfuscated. Furthermore, commercial obfuscation tools allow malware authors to apply multiple obfuscation options simultaneously, and current deobfuscation techniques cannot handle this situation. In this study, we analyzed a well-known commercial obfuscation tool called Themida. We applied its many obfuscation options to a program and implemented a tool to recover the original code and data. We extracted features from obfuscated programs and analyzed their control flow. Our tool is based on these features and the control flow patterns and can identify whether Themida has been applied to the program and which obfuscation options have been used. Finally, we suggested a method for recovering the import address table of programs by using dynamic binary instrumentation. The proposed rules and algorithms can almost completely recover the APIs of programs even though they are hidden by obfuscation options provided by Themida.

KW - Debugging

KW - Packer

KW - Reverse engineering

KW - Software implementation

KW - Software protection

UR - http://www.scopus.com/inward/record.url?scp=85050934585&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85050934585&partnerID=8YFLogxK

U2 - 10.1002/spe.2622

DO - 10.1002/spe.2622

M3 - Article

AN - SCOPUS:85050934585

JO - Software - Practice and Experience

JF - Software - Practice and Experience

SN - 0038-0644

ER -