The challenges of machine learning in adversarial settings: a systems perspective
Advances in machine learning have enabled new applications that border on science fiction. Autonomous cars, data analytics, adaptive communication, and self-aware software systems are now revolutionizing markets by achieving or exceeding human performance. In this talk, I discuss the rapidly evolving use of machine learning in security-sensitive contexts and explore why many systems are vulnerable to nonobvious and potentially dangerous manipulation. We will examine sensitivity in applications where misuse might lead to harm—for instance, forcing adaptive networks into an unstable state, crashing an autonomous vehicle, or bypassing an adult content filter. I explore how currently accepted wisdom about threats and defenses should be viewed (and sometimes refuted) in light of the functional and security challenges of real-world systems. The talk concludes with a discussion of the technological, economic, and societal challenges we face as a result of the rise of machine learning as fundamental construct of computational systems.