Dampier, David A.
McGrew, Robert Wesley
Date of Degree
Dissertation - Open Access
Doctor of Philosophy
Department of Computer Science and Engineering
Malicious software, or malware, is often employed as a tool to maintain access to previously compromised systems. It enables the intruders to utilize system resources, harvest legitimate credentials, and maintain a level of stealth throughout the process. During incident response, identifying systems infected with malware is necessary for effective remediation of an attack. When analysts lack sufficient indicators of compromise they are forced to conduct a comprehensive examination to identify anomalous behavior on a system, a time consuming and challenging task. Malware authors use several techniques to conceal malware on a system, with a common method being DLL injection. In this dissertation we present a system for automatically generating Windows 7 x86 memory images infected with malware, identifying the malicious DLLs injected into a process, and extracting the features associated with those DLLs. A set of 3,240 infected memory images was produced and analyzed to identify common characteristics of malicious DLLs in memory. From this analysis a feature set was constructed and two datasets were used to evaluate five classification algorithms. The ZeroR method was used as a baseline for comparison with accuracy and false positive rate (misclassifying malicious DLLs as legitimate) being the two metrics of interest. The results of the experiments showed that learning using the feature set is viable and that the performance of the classifiers can be further improved through the use of feature selection. Each of the classification methods outperformed the ZeroR method with the J48 Decision Tree obtaining the, overall, best results.
Glendowne, Dae, "Automating Malware Detection in Windows Memory Images using Machine Learning" (2015). Theses and Dissertations MSU. 830.