Baidu AI model testing tool AI Model-Mutator debuts at Black Hat Europe 2021

AI technology has been widely used in all walks of life. On the one hand, the hot research trend and efficient development iteration speed in the field of AI technology can cope with the ever-increasing demand. On the other hand, frequent technology updates and high-speed iterations of development have had an impact on the security of the underlying computing framework. How to ensure the code development quality of the AI computing framework has become an indispensable part of the code development process.

At the just-concluded Black Hat Europe 2021, Baidu Security proposed a new type of model mutation-based fuzzing tool AI Model-Mutator. It uses the model file as a data input file and performs mutations to test the code quality of the underlying computing framework. Unlike functional testing, it uses random mutation, so it can cover some boundary issues that functional testing cannot take into account. Unlike single API testing, model testing starts from model files, so it is not for a single API, but for a set of functional paths. Therefore, it can cover some paths that cannot be tested by a single API, such as the Python Binder part.

AI Model-Mutator proposes a graph-based model mutation method. In more detail, it uses static code analysis to obtain the types and constraints of model-related operations, and then randomly traverses and rewrites the graph according to these operation types and constraints. Finally, use the generated mutated model file as a seed to test the calculation framework. In addition, AI Model-Mutator proposes a series of mutation rules for model input to effectively mutate model files. It has been verified that these mutation rules can effectively reproduce the 66 vulnerabilities found in TensorFlow. In addition, AI Model-Mutator discovered 6 new vulnerabilities in TensorFlow and was confirmed by the TensorFlow team.

Baidu Security Lab has been committed to system security research for many years. The launch of the new AI model testing tool this time at Black Hat Europe 2021 is just the beginning. In the future, more and deeper research will focus on the AI computing framework to escort the foundation of Baidu's AI model framework.