Assumption Stress-Testing for Machine Learning Security

Abstract

Security is guided by the principle that a system is only as strong as its weakest link. Despite the undeniable benefits brought by the growing integration of machine learning (ML) into nearly all aspects of modern life, decades of research and real-world deployment have established a broad consensus that systems oblivious to malicious behavior are fundamentally unsafe. In practice, ill-intended users can abuse access to powerful generative models to create harmful deepfakes capable of facilitating fraud and impersonation, or instigating societal chaos. At the same time, the rapid adoption of ML in high-stakes settings such as online moderation, media authenticity verification, and biometric authentication introduces the risk of these mechanisms being manipulated or bypassed through carefully crafted inputs with potentially catastrophic consequences, as repeatedly demonstrated in practice.

This thesis approaches these threats from an assumption-driven perspective, arguing that critical failures in ML security do not arise from isolated weaknesses of individual models, but from flawed or insufficiently scrutinized assumptions underlying broader robustness concepts. To study this problem, the thesis advances the perspective of assumption stress-testing, a framework for systematically analyzing the assumptions on which security mechanisms rely and constructing targeted approaches that subject them to worst-case behavior. Across four projects spanning multiple ML security domains, the thesis demonstrates how violating these assumptions can make the underlying concepts themselves inapplicable, causing entire classes of mechanisms built upon them to fail regardless of how they are instantiated. Specifically, the thesis investigates the robustness of voice authentication and image watermarking technologies designed to support media authenticity, revisits the reliability of a state-of-the-art defense paradigm against adversarial examples, and explores the constructive use of assumption-aware analysis to devise systems whose robustness remains meaningful under rigorous evaluation.