icemelon/bert_distilbert_cpu_inference.md

Last active July 7, 2020 21:14

Star () You must be signed in to star a gist
Fork () You must be signed in to fork a gist

Learn more about clone URLs
Clone this repository at <script src="https://gist.github.com/icemelon/cf0e7f354773ac6893d9dce62690b5be.js"></script>
Save icemelon/cf0e7f354773ac6893d9dce62690b5be to your computer and use it in GitHub Desktop.

Raw

    

model
batch
seq length
MXNet latency (ms)
TVM latency (ms)
speedup


BERT
1
64
26.1
12.6
2.1

BERT
1
128
45.8
19.2
2.4

BERT
1
256
99.4
35.3
2.8

DistilBERT
1
64
13.4
6.2
2.2

DistilBERT
1
128
23.2
9.5
2.5

DistilBERT
1
256
50.1
17.5
2.9



  

model	batch	seq length	MXNet latency (ms)	TVM latency (ms)	speedup
BERT	1	64	26.1	12.6	2.1
BERT	1	128	45.8	19.2	2.4
BERT	1	256	99.4	35.3	2.8
DistilBERT	1	64	13.4	6.2	2.2
DistilBERT	1	128	23.2	9.5	2.5
DistilBERT	1	256	50.1	17.5	2.9