TY - GEN
T1 - Effective white-box testing of deep neural networks with adaptive neuron-selection strategy
AU - Lee, Seokhyun
AU - Cha, Sooyoung
AU - Lee, Dain
AU - Oh, Hakjoo
N1 - Funding Information:
This work was partly supported by Institute of Information & communications Technology Planning & Evaluation (IITP) grant funded by the Korea government(MSIT) (No.2020-0-01337,(SW STAR LAB) Research on Highly-Practical Automated Software Repair) and Sam-sung Research Funding & Incubation Center of Samsung Electronics under Project Number SRFC-IT1701-09.
Publisher Copyright:
© 2020 ACM.
PY - 2020/7/18
Y1 - 2020/7/18
N2 - We present Adapt, a new white-box testing technique for deep neural networks. As deep neural networks are increasingly used in safety-first applications, testing their behavior systematically has become a critical problem. Accordingly, various testing techniques for deep neural networks have been proposed in recent years. However, neural network testing is still at an early stage and existing techniques are not yet sufficiently effective. In this paper, we aim to advance this field, in particular white-box testing approaches for neural networks, by identifying and addressing a key limitation of existing state-of-the-arts. We observe that the so-called neuron-selection strategy is a critical component of white-box testing and propose a new technique that effectively employs the strategy by continuously adapting it to the ongoing testing process. Experiments with real-world network models and datasets show that Adapt is remarkably more effective than existing testing techniques in terms of coverage and adversarial inputs found.
AB - We present Adapt, a new white-box testing technique for deep neural networks. As deep neural networks are increasingly used in safety-first applications, testing their behavior systematically has become a critical problem. Accordingly, various testing techniques for deep neural networks have been proposed in recent years. However, neural network testing is still at an early stage and existing techniques are not yet sufficiently effective. In this paper, we aim to advance this field, in particular white-box testing approaches for neural networks, by identifying and addressing a key limitation of existing state-of-the-arts. We observe that the so-called neuron-selection strategy is a critical component of white-box testing and propose a new technique that effectively employs the strategy by continuously adapting it to the ongoing testing process. Experiments with real-world network models and datasets show that Adapt is remarkably more effective than existing testing techniques in terms of coverage and adversarial inputs found.
KW - Deep neural networks
KW - Online learning
KW - White-box testing
UR - http://www.scopus.com/inward/record.url?scp=85088914984&partnerID=8YFLogxK
U2 - 10.1145/3395363.3397346
DO - 10.1145/3395363.3397346
M3 - Conference contribution
AN - SCOPUS:85088914984
T3 - ISSTA 2020 - Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis
SP - 165
EP - 176
BT - ISSTA 2020 - Proceedings of the 29th ACM SIGSOFT International Symposium on Software Testing and Analysis
A2 - Khurshid, Sarfraz
A2 - Pasareanu, Corina S.
PB - Association for Computing Machinery, Inc
T2 - 29th ACM SIGSOFT International Symposium on Software Testing and Analysis, ISSTA 2020
Y2 - 18 July 2020 through 22 July 2020
ER -