Efficient file fuzz testing using automated analysis of binary file format

Hyoung Chun Kim, Young Han Choi, Dong Hoon Lee

Research output: Contribution to journalArticle

21 Citations (Scopus)

Abstract

Fuzz testing is regarded as the most useful technique in finding serious security holes in a software system. It inserts unexpected data into the input of the software system and finds the system's bugs or errors. However, one of the disadvantages that fuzz testing executed using binary files has is that it requires a large number of fault-inserted files to cover every test case, which could be up to 28×FILESIZE files. In order to overcome this drawback, we propose a novel algorithm that efficiently reduces the number of fault-inserted files, yet still maintain the maximum test case coverage. The proposed approach enables the automatic analysis of fields of binary files by tracking and analyzing stack frames, assembly codes, and registers as the software system parses the files. We evaluate the efficacy of the new method by implementing a practical tool, the Binary File Analyzer and Fault Injector (BFAFI), which traces the program execution and analyzes the fields in binary file format. Our experiments demonstrate that the BFAFI reduced the total number of fault-inserted files with maximum test case coverage as well as detected approximately 14 times more exceptions than did the general fuzzer. Also, the BFAFI found 11 causes of exceptions; five of them were found only by BFAFI. Ten of the 11 causes of exceptions that we found were generated by a graphic rendering engine (GDI32.dll); the other was generated by the system library (kernel32.dll) in Windows XP SP2.

Original languageEnglish
Pages (from-to)259-268
Number of pages10
JournalJournal of Systems Architecture
Volume57
Issue number3
DOIs
Publication statusPublished - 2011 Mar 1

Fingerprint

Testing
Engines
Experiments

Keywords

  • Fuzzing
  • Security testing
  • Software testing

ASJC Scopus subject areas

  • Hardware and Architecture
  • Software

Cite this

Efficient file fuzz testing using automated analysis of binary file format. / Kim, Hyoung Chun; Choi, Young Han; Lee, Dong Hoon.

In: Journal of Systems Architecture, Vol. 57, No. 3, 01.03.2011, p. 259-268.

Research output: Contribution to journalArticle

@article{af7323cc64a446b9b5a0e564217d5369,
title = "Efficient file fuzz testing using automated analysis of binary file format",
abstract = "Fuzz testing is regarded as the most useful technique in finding serious security holes in a software system. It inserts unexpected data into the input of the software system and finds the system's bugs or errors. However, one of the disadvantages that fuzz testing executed using binary files has is that it requires a large number of fault-inserted files to cover every test case, which could be up to 28×FILESIZE files. In order to overcome this drawback, we propose a novel algorithm that efficiently reduces the number of fault-inserted files, yet still maintain the maximum test case coverage. The proposed approach enables the automatic analysis of fields of binary files by tracking and analyzing stack frames, assembly codes, and registers as the software system parses the files. We evaluate the efficacy of the new method by implementing a practical tool, the Binary File Analyzer and Fault Injector (BFAFI), which traces the program execution and analyzes the fields in binary file format. Our experiments demonstrate that the BFAFI reduced the total number of fault-inserted files with maximum test case coverage as well as detected approximately 14 times more exceptions than did the general fuzzer. Also, the BFAFI found 11 causes of exceptions; five of them were found only by BFAFI. Ten of the 11 causes of exceptions that we found were generated by a graphic rendering engine (GDI32.dll); the other was generated by the system library (kernel32.dll) in Windows XP SP2.",
keywords = "Fuzzing, Security testing, Software testing",
author = "Kim, {Hyoung Chun} and Choi, {Young Han} and Lee, {Dong Hoon}",
year = "2011",
month = "3",
day = "1",
doi = "10.1016/j.sysarc.2010.03.002",
language = "English",
volume = "57",
pages = "259--268",
journal = "Journal of Systems Architecture",
issn = "1383-7621",
publisher = "Elsevier",
number = "3",

}

TY - JOUR

T1 - Efficient file fuzz testing using automated analysis of binary file format

AU - Kim, Hyoung Chun

AU - Choi, Young Han

AU - Lee, Dong Hoon

PY - 2011/3/1

Y1 - 2011/3/1

N2 - Fuzz testing is regarded as the most useful technique in finding serious security holes in a software system. It inserts unexpected data into the input of the software system and finds the system's bugs or errors. However, one of the disadvantages that fuzz testing executed using binary files has is that it requires a large number of fault-inserted files to cover every test case, which could be up to 28×FILESIZE files. In order to overcome this drawback, we propose a novel algorithm that efficiently reduces the number of fault-inserted files, yet still maintain the maximum test case coverage. The proposed approach enables the automatic analysis of fields of binary files by tracking and analyzing stack frames, assembly codes, and registers as the software system parses the files. We evaluate the efficacy of the new method by implementing a practical tool, the Binary File Analyzer and Fault Injector (BFAFI), which traces the program execution and analyzes the fields in binary file format. Our experiments demonstrate that the BFAFI reduced the total number of fault-inserted files with maximum test case coverage as well as detected approximately 14 times more exceptions than did the general fuzzer. Also, the BFAFI found 11 causes of exceptions; five of them were found only by BFAFI. Ten of the 11 causes of exceptions that we found were generated by a graphic rendering engine (GDI32.dll); the other was generated by the system library (kernel32.dll) in Windows XP SP2.

AB - Fuzz testing is regarded as the most useful technique in finding serious security holes in a software system. It inserts unexpected data into the input of the software system and finds the system's bugs or errors. However, one of the disadvantages that fuzz testing executed using binary files has is that it requires a large number of fault-inserted files to cover every test case, which could be up to 28×FILESIZE files. In order to overcome this drawback, we propose a novel algorithm that efficiently reduces the number of fault-inserted files, yet still maintain the maximum test case coverage. The proposed approach enables the automatic analysis of fields of binary files by tracking and analyzing stack frames, assembly codes, and registers as the software system parses the files. We evaluate the efficacy of the new method by implementing a practical tool, the Binary File Analyzer and Fault Injector (BFAFI), which traces the program execution and analyzes the fields in binary file format. Our experiments demonstrate that the BFAFI reduced the total number of fault-inserted files with maximum test case coverage as well as detected approximately 14 times more exceptions than did the general fuzzer. Also, the BFAFI found 11 causes of exceptions; five of them were found only by BFAFI. Ten of the 11 causes of exceptions that we found were generated by a graphic rendering engine (GDI32.dll); the other was generated by the system library (kernel32.dll) in Windows XP SP2.

KW - Fuzzing

KW - Security testing

KW - Software testing

UR - http://www.scopus.com/inward/record.url?scp=79952575071&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=79952575071&partnerID=8YFLogxK

U2 - 10.1016/j.sysarc.2010.03.002

DO - 10.1016/j.sysarc.2010.03.002

M3 - Article

VL - 57

SP - 259

EP - 268

JO - Journal of Systems Architecture

JF - Journal of Systems Architecture

SN - 1383-7621

IS - 3

ER -