Existing research on malware classification focuses almost exclusively on two tasks: distinguishing between malicious and benign files, and classifying malware by family. Malware, however, can be categorized according to many other types of attributes, and the ability to identify these attributes in newly-emerging malware using machine learning will provide significant value to analysts. In particular, we have identified four tasks which are under-represented in prior work: classification by behaviors that malware exhibit, platforms that malware run on, vulnerabilities that malware exploit, and packers that packed the malware. To obtain labels for training and evaluating ML classifiers on these tasks, we created an antivirus (AV) tagging tool called ClarAVy. ClarAVy's sophisticated AV label parser distinguishes itself from prior AV-based taggers with the ability to parse 882 different AV label formats used by 90 different AV products accurately. We are releasing benchmark datasets for each of these four classification tasks, tagged using ClarAVy and comprising nearly 5.5 million malicious files in total. Our malware behavior dataset includes 75 distinct tags -- nearly seven times more than the only prior benchmark dataset with behavioral tags. To our knowledge, we are the first to release datasets with malware platform, exploitation, and packer tags.

RJ Joyce (joyce8@umbc.edu) is a PhD student at UMBC under the supervision of Dr. Charles Nicholas and Dr. Edward Raff. Presently, RJ works as a data scientist at Booz Allen Hamilton performing research at the intersection of malware analysis and machine learning. RJ is also a visiting lecturer at UMBC and is teaching CMSC-426 Principles of Computer Security course this semester.

Host: Alan T. Sherman, sherman@umbc.edu. Support for this event was provided in part by the National Science Foundation under SFS grant DGE-1753681.  The UMBC Cyber Defense Lab meets biweekly Fridays 12-1pm.  All meetings are open to the public.  Upcoming CDL meetings: Oct. 20 (1-2pm) Josh Benaloh (Microsoft), ElectionGuard; Nov. 3, Jason Rheinhart (Sandia), Risk analysis; Nov. 17 (1-2pm) Austin Murdoch (Sixmap); Dec. 1, Enis Golaszewski (UMBC), Automatic cryptographic bindings; Jan. 16-19, 2024, UMBC SFS/CySP Research Study.

We introduce AVScan2Vec, a sequence-to-sequence autoencoder that can ingest AV scan data, extract semantic meaning, and produce meaningful feature vectors for malware. AVScan2Vec is able to bypass several limitations of prior malware feature-extraction methods, while simultaneously showing noteworthy improvement in several relevant ML tasks. Our implementation of AVScan2Vec in combination with Dynamic Continuous Indexing is especially potent, enabling 10 nearest-neighbor lookup queries in ~16ms on a dataset containing over seven million malware samples. Automation has become increasingly more vital to the field of malware analysis due to manual effort being slow and costly. To improve common tasks such as classification, clustering, and nearest-neighbor lookup of malware, improving malware feature extraction has been a significant research focus. Many approaches rely on features that can only be obtained using prolonged analysis. Due to the enormous quantity and variety of malware, however, applying these feature extraction techniques to a production-size malware corpus would be infeasible. Other, more scalable feature-extraction methods are hindered by static obfuscation, restricted to a single file format, and/or limited in their capacity to identify higher-level malware features. Our work explores the under-recognized potential of antivirus (AV) scan data, which is relatively cheap to acquire and contains rich features.

RJ Joyce (joyce8@umbc.edu) is a PhD student at UMBC under the supervision of Dr. Charles Nicholas and Dr. Edward Raff. Presently, RJ works as a data scientist at Booz Allen Hamilton performing research at the intersection of malware analysis and machine learning. RJ is also a visiting lecturer at UMBC and is teaching the Principles of Computer Security course this semester.

Host: Alan T. Sherman, sherman@umbc.edu. Support for this event was provided in part by the National Science Foundation under SFS grant DGE-1753681.  The UMBC Cyber Defense Lab meets biweekly Fridays 12-1pm. All meetings are open to the public.  Upcoming CDL meetings: April 28, Roberto Yus (UMBC), Privacy; May 5, CSEE Research Day (ECS Atrium); May 12, Kia-Won-Tia von Wrex (UMBC), Cyberdawgs