Theses and Dissertations

Author

Dae Glendowne

Issuing Body

Mississippi State University

Advisor

Dampier, David A.

Committee Member

Archibald, Christopher

Committee Member

Ramkumar, Mahalingham

Committee Member

McGrew, Robert Wesley

Date of Degree

5-9-2015

Document Type

Dissertation - Open Access

Major

Computer Science

Degree Name

Doctor of Philosophy

College

James Worth Bagley College of Engineering

Department

Department of Computer Science and Engineering

Abstract

Malicious software, or malware, is often employed as a tool to maintain access to previously compromised systems. It enables the intruders to utilize system resources, harvest legitimate credentials, and maintain a level of stealth throughout the process. During incident response, identifying systems infected with malware is necessary for effective remediation of an attack. When analysts lack sufficient indicators of compromise they are forced to conduct a comprehensive examination to identify anomalous behavior on a system, a time consuming and challenging task. Malware authors use several techniques to conceal malware on a system, with a common method being DLL injection. In this dissertation we present a system for automatically generating Windows 7 x86 memory images infected with malware, identifying the malicious DLLs injected into a process, and extracting the features associated with those DLLs. A set of 3,240 infected memory images was produced and analyzed to identify common characteristics of malicious DLLs in memory. From this analysis a feature set was constructed and two datasets were used to evaluate five classification algorithms. The ZeroR method was used as a baseline for comparison with accuracy and false positive rate (misclassifying malicious DLLs as legitimate) being the two metrics of interest. The results of the experiments showed that learning using the feature set is viable and that the performance of the classifiers can be further improved through the use of feature selection. Each of the classification methods outperformed the ZeroR method with the J48 Decision Tree obtaining the, overall, best results.

URI

https://hdl.handle.net/11668/18170

Share

COinS