Etc/Etc
ONNX, TVM
light_meal
2022. 8. 18. 23:51
728x90
ONNX, TVM
(ONNX + ONNX Runtime) & TVM
| ONNX + ONNX Runtime | TVM | |
|---|---|---|
| Develop & Support | Microsoft & facebook & AWS | Apache |
| System | 1. ONNX Runtime Quantization 2. ONNX Runtime Compile |
1. Integrated System( Lightweight Model + Compile ) |
| Support ML Framework | caffe24 Keras Tensorflow PyTorch CoreML mxnet XGBoost NCNN ... |
Pytorch CoreML Tensorflow Keras ONNX mxnet … |
| Deploy Target Device or Environment | Arm Arm NN CoreML CUDA Windows AMD Android Intel RKNPU TensorRT Vitis FPGA NPU |
Arm Android TensorRT Vitis BNNS DLPack FPGA NPU |
ONNX + ONNX Runtime
https://blog.ml6.eu/bert-is-eating-your-cash-quantization-and-onnxruntime-to-save-ea6dc84dcd88
- Resnet50 model → ONNX Runtime Quantization BenchmarkAvg: 23.95ms → Avg: 10.91ms
TVM
728x90