【Model Reasoning】Tengine model conversion and quantification

Welcome to pay attention to my public account [极智视界], reply 001 to get Google programming specifications

  O_o   >_<   o_O   O_o   ~_~   o_O

This article introduces the Tengine model conversion and quantification process.

Tengine, like ncnn, is also an excellent end-to-side reasoning framework. I wrote an article " ncnn model conversion and quantification process ", and interested students can check it out.

Start below.

1. Compile Tengine Tools

Turn on the CONVERT_TOOL and QUANT_TOOL switches in CmakeList.txt:

Start to compile:

cd <Tengine-path>
mkdir build

cd build

# 开始编译
camke ..
make -j32
make install

After compilation, convert_tool and quant_tool ./build/install/bin/ directory, as follows:

2. Tengine model conversion

Use the following command to see what kind of parameters convert_tool

cd ./build/install/bin

./convert_tool -h

As above, the front is the input and the back is the output.

The front-end support of the model conversion tool provided by Tengine is very rich, and it basically takes all the mainstream deep learning frameworks at home and abroad.

The front end is Caffe:

./convert_tool -f caffe -p ./mobilenet.prototxt -m ./mobilenet.caffemodel -o ./mobilenet.tmfile

The front end is Onnx:

./convert_tool -f onnx -m ./mobilenet.onnx -o ./mobilenet.tmfile

The front end is Mxnet:

./convert_tool -f mxnet -p ./mobilenet.params -m ./mobilenet.json -o ./mobilenet.tmfile

The front end is Darknet:

./convert_tool -f darknet -p ./yolov3.weights -m ./yolov3.cfg -o yolov3.tmfile

The front end is TensorFlow:

./convert_tool -f tensorflow -m mobielenet_v1_1.0_224_frozen.pb -o mobilenet.tmfile

The front end is TFLite:

./convert_tool -f tflite -m mobielenet.tflite -o mobilenet.tmfile

The front end is MegEngine:

./convert_tool -f megengine -m mobilenet.pkl -o mobilenet.tmfile

The front end is OneFlow:

./convert_tool -f oneflow -p mobilenet.prototxt -m mobilenet/ -o mobilenet.tmfile

The front end is ncnn:

./convert_tool -f ncnn -p mobilenet.param -m mobilenet.bin -o mobilenet.tmfile

3. Tengine quantitative reasoning

You can see that there are three quantization tools provided by Tengine: quant_tool_int8 , quant_tool_uint8 , quant_tool_uint8_perchannel , which are symmetric quantization, asymmetric quantization, and asymmetric channel-by-channel quantization, respectively. The quantization strategies inside are similar. Here, we pick quant_tool_int8 .

First install the dependent libraries:

apt update
apt install libopencv-dev

Use the following command to view the parameters of the quantization tool:

./quant_tool_int8 -h

It can be seen that the transfer of parameters is quite abundant.

The front-end input is fp32 tmfile, and the quantization example is performed:

./quant_tool_int8  -m ./mobilenet_fp32.tmfile -i ./dataset -o ./mobilenet_int8.tmfile -g 3,224,224 -w 104.007,116.669,122.679 -s 0.017,0.017,0.017

Successful execution will output the following log:

And generate the following int8 model file:

Then it can be used to perform inference.

The above shared Tengine model conversion and quantification methods, I hope my sharing can be a little helpful to your study.

---

【Public Account Transmission】
" [Model Reasoning] Tengine model conversion and quantification "