12:00 noon–1pm Friday, April 11, 2025 online
Joint work with Yuechun Gu and Jiajie He
To appear, 2015 Int. Conf. on Knowledge Discovery and Data Mining
With increasingly deployed deep neural networks in sensitive application domains, such as healthcare and security, it is essential to understand what kind of sensitive information can be inferred from these models. Most known model-targeted attacks assume attackers have learned the application domain or training data distribution to ensure successful attacks. Can removing the domain information from model APIs protect models from these attacks? Our work studies this critical problem. Unfortunately, even with minimal knowledge, i.e., accessing the model as an unnamed function without leaking the meaning of input and output, the proposed adaptive domain inference (ADI) attack can still successfully estimate relevant subsets of training data. We show that the extracted relevant data can significantly improve the performance of model-inversion attacks, for instance. Specifically, the ADI method uses the concept hierarchy extracted from the public and private datasets that the attacker can access, and it applies a novel algorithm to adaptively tune the likelihood of leaf concepts in the hierarchy showing up in the unseen training data. For comparison, we also designed a straightforward hypothesis-testing-based attack called LDI. Among all candidate methods, the ADI attack extracts partial training data at the concept level, converges fastest, and requires the fewest target-model accesses.
Dr. Keke Chen is an associate professor in the UMBC CSEE Department. His recent research focuses on privacy and security issues with AI model training and deployment. He earned his PhD in computer science from Georgia Tech in 2006. Before joining UMBC, he was a Northwestern Mutual associate professor of computer science at Marquette University.
Support for this event was provided in part by the NSF under SFS grant DGE-1753681.