Analysis of Evasion Techniques in Web-based Malware

Web-based mechanisms, often mediated by malicious JavaScript code, play an important role in malware delivery today, making defenses against web-based malware crucial for system security. To make it even more challenging, malware authors often take advantage of various evasion techniques to evade detection. As a result, a constant arms race of evasion and detection techniques between malware authors and security analysts has led to advancement in code obfuscation and anti-analysis techniques. This dissertation focuses on the defenses against web-based malware protected by advanced evasion techniques from both defensive and offensive perspectives. From a defensive perspective, we examine existing evasion techniques and propose deobfuscation and detection approaches to defeating some popular techniques used by web-based malware today. In the case of code-unfolding based obfuscation, we use a semantics-based approach to simplify away obfuscations by identifying code that is relevant to the behavior of the original program. In the case of environment-dependent malware, we propose environmental predicate, which detects behavior discrepancy of JavaScript program between targeted browser and detector sandbox, therefore protecting users from possible detection false negatives caused by environmental triggers. From an offensive perspective, we analyze existing detection techniques to examining their assumptions and study how these assumptions can be broken. We also propose a combination of obfuscation and anti-analysis techniques, targeting these limitations, which can hide existing web-based malware from state-of-the-art detectors.

Name thesis:degree_name

Ph.D.

Level thesis:degree_level

doctoral

Discipline thesis:degree_discipline

Graduate College

Grantor dc:publisher

The University of Arizona.

Year dc:date.issued

2013

Author dc:creator

• Lu, Gen

Advisor dc:contributor.advisor

• Debray, Saumya

Committee members dc:contributor.committeemember

• Debray, Saumya
• Lowenthal, David
• Hartman, John
• Gniady, Christopher

• obfuscation
• web security
• Computer Science
• malware detection

Statement dc:rights

• Copyright © is held by the author. Digital access to this material is made possible by the University Libraries, University of Arizona. Further transmission, reproduction or presentation (such as public display or performance) of protected items is prohibited except with permission of the author.

Language dc:language.iso

en_US

Handle dc:identifier.uri

http://hdl.handle.net/10150/312567

OAI identifier oai:identifier

oai:repository.arizona.edu:10150/312567