Encoding function pointers and memory arrangement checking against buffer overflow attack

Changwoo Pyo, Kyung Ho Lee

Research output: Chapter in Book/Report/Conference proceedingConference contribution

9 Citations (Scopus)

Abstract

Program counter is the only mechanism for processor to access instruction to execute. Protecting program counter is the fundamental defense for securing computer system. This paper presents a scheme of protecting program counter by encoding function pointers. In the scheme, every function address is encoded by linker. Compiler inserts instructions for decoding function addresses before call instruction. Encoding code pointers, function addresses as well as return addresses in stack frame, provides substantial coverage of protecting program counter. Several suggestions are also made to detect compromised code pointers at run-time without memory space for sensor mechanism. A demo Linux system has been under construction with the proposed scheme. Experimental data shows performance slowdown less than 10% when all return addresses and function addresses are encoded. With a Pentium III processor of 866MHz, the overhead for each function call is on the order of nanoseconds. We plan to migrate parts of our code pointer encoding scheme from linker to dynamic linker, which should improve security and performance.

Original languageEnglish
Title of host publicationLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
PublisherSpringer Verlag
Pages25-36
Number of pages12
Volume2513
ISBN (Print)3540001646
Publication statusPublished - 2002
Externally publishedYes
Event4th International Conference on Information and Communications Security, ICICS 2002 - Singapore, Singapore
Duration: 2002 Dec 92002 Dec 12

Publication series

NameLecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume2513
ISSN (Print)03029743
ISSN (Electronic)16113349

Other

Other4th International Conference on Information and Communications Security, ICICS 2002
CountrySingapore
CitySingapore
Period02/12/902/12/12

    Fingerprint

Keywords

  • Buffer overflow attack
  • Function pointer encoding
  • Memory arrangement checking
  • Program counter

ASJC Scopus subject areas

  • Computer Science(all)
  • Theoretical Computer Science

Cite this

Pyo, C., & Lee, K. H. (2002). Encoding function pointers and memory arrangement checking against buffer overflow attack. In Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics) (Vol. 2513, pp. 25-36). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 2513). Springer Verlag.