Detecting buffer overflows using testcase synthesis and code instrumentation

The research presented in this thesis aims to improve existing approaches to dynamic buffer overflow detection by developing a system that utilizes code instrumentation and adaptive test case synthesis to find buffer overflows and corresponding failure-inducing inputs automatically. An evaluation of seven modern dynamic buffer overflow detection tools determined that C Range Error Detector (CRED) is capable of providing fine-grained buffer access information necessary for the creation of this system. CRED was also selected because of its ability to provide comprehensive error reports and compile complex programs with reasonable performance overhead. CRED was extended to provide appropriate code instrumentation for the adaptive testing system, which also includes a test case synthesizer that uses data perturbation techniques on legal inputs to produce new test cases, and an analytical module that evaluates the effectiveness of these test cases. Using information provided by code instrumentation in further test case generation creates a feedback loop that enables a focused exploration of the input space and faster buffer overflow detection. Applying the adaptive testing system to jabberd, a Jabber Instant Messaging server, demonstrates its effectiveness in finding buffer overflows and its advantages over existing dynamic testing systems.

Department dc:contributor.department

Massachusetts Institute of Technology. Dept. of Electrical Engineering and Computer Science.

Grantor dc:publisher

Massachusetts Institute of Technology

Year dc:date.issued

2005

Author dc:creator

• Zhivich, Michael A

Advisor dc:contributor.advisor

• Richard Lippmann.

• Electrical Engineering and Computer Science.

Statement dc:rights

• M.I.T. theses are protected by copyright. They may be viewed from this source for any purpose, but reproduction or distribution in any format is prohibited without written permission. See provided URL for inquiries about permission.

Licence dc:rights.uri

• http://dspace.mit.edu/handle/1721.1/7582

Language dc:language.iso

eng

Handle dc:identifier.uri

http://hdl.handle.net/1721.1/32112

OAI identifier oai:identifier

oai:dspace.mit.edu:1721.1/32112