Classifying malicious windows executables using anomaly based detection

A malicious executable is broadly defined as any program or piece of code designed to cause damage to a system or the information it contains, or to prevent the system from being used in a normal manner. A generic term used to describe any kind of malicious software is Maiware, which includes Viruses, Worms, Trojans, Backdoors, Root-kits, Spyware and Exploits. Anomaly detection is technique which builds a statistical profile of the normal and malicious data and classifies unseen data based on these two profiles. A detection system is presented here which is anomaly based and focuses on the Windows platform. Several file infection techniques were studied to understand what particular features in the executable binary are more susceptible to being used for the malicious code propagation. A framework is presented for collecting data for both static (non-execution based) as well as dynamic (execution based) analysis of the malicious executables. Two specific features are extracted using static analysis, Windows API (from the Import Address Table of the Portable Executable Header) and the hex byte frequency count (collected using Hexdump utility) which have been explained in detail. Dynamic analysis features which were extracted are briefly mentioned and the major challenges faced using this data is explained. Classification results using Support Vector Machines for anomaly detection is shown for the two static analysis features. Experimental results have provided classification results with up to 94% accuracy for new, previously unseen executables.

Name thesis:degree_name

Master of Science in Computer Science - (M.S.)

Discipline thesis:degree_discipline

Computer Science

Year

2006

Author dc:creator

• Sutaria, Ronak

Contributors dc:contributor

• Constantine N. Manikopoulos
• Robert Statica
• Jie Hu

• Malicious executables
• Malware
• Malware detection
• Computer Sciences

Repository record dc:identifier

https://digitalcommons.njit.edu/theses/415

OAI identifier oai:identifier

oai:digitalcommons.njit.edu:theses-1414