Toward detecting advanced persistent threat using malicious non-executable files

Young Han Choi, Hyoung Chun Kim, Dong Hoon Lee

Research output: Contribution to journalArticle

Abstract

Advanced Persistent Threat (APT) attacks vulnerable applications in client PC using social engineering and steals its information secretly. Most of APT controls the client's system by executing malicious code in non-executable file that the applications read and parse. In this paper, we propose a novel technique to detect a malicious non-executable file regardless of file format By regarding all nonexecutable files as byte sequences and executing the sequences forcibly from beginning to end, we target on detecting only an executable code in the byte sequences. Because it takes a long time to execute all bytes, we select suspicious bytes in a file using patterns of invalid instructions. We implement a tool to evaluate our idea using a debugger engine to change flow of execution freely. The experimental results show mat our idea is effective. Our idea can prevent APT by detecting malicious files in Honeynet or an email server beforehand.

Original languageEnglish
Pages (from-to)1735-1740
Number of pages6
JournalInformation (Japan)
Volume17
Issue number5
Publication statusPublished - 2014 Jan 1

    Fingerprint

ASJC Scopus subject areas

  • General

Cite this