Skip to content

Instantly share code, notes, and snippets.

@rwightman

rwightman/BENCHMARK.md

Last active July 12, 2023 09:40
Show Gist options
  • Save rwightman/826027cd544b936c959bae3761540f7d to your computer and use it in GitHub Desktop.
Save rwightman/826027cd544b936c959bae3761540f7d to your computer and use it in GitHub Desktop.
Download ZIP
timm model benchmark compare
Raw
BENCHMARK.md

NCHW and NHWC benchmark numbers for some common image classification models in timm.

For NCHW: python benchmark.py --model-list model.txt --amp -b 128

For NHWC: python benchmark.py --model-list model.txt --amp -b 128 --channels-last

Note the test res for efficientnet_b1/b2/b3/b4 and regnety_160 were adjusted in timm to match original paper and not timm defaults. Benchmark script in root of timm https://github.com/rwightman/pytorch-image-models/blob/master/benchmark.py

Raw
models.txt
resnet18
resnet34
resnet50
resnet101
resnet152
regnety_040
regnety_080
regnety_160
regnety_320
seresnet50
senet154
resnet50d
resnext50_32x4d
efficientnet_b0
efficientnet_b1
efficientnet_b2
efficientnet_b3
nfnet_f0
nfnet_f1
nfnet_f2
vit_tiny_patch16_224
vit_small_patch16_224
vit_base_patch16_224
ecaresnet50t
efficientnetv2_rw_s
efficientnetv2_rw_m
ecaresnet269d
Raw
results_nchw_rxt3090.csv
model infer_samples_per_sec infer_step_time infer_batch_size infer_img_size train_samples_per_sec train_step_time train_batch_size train_img_size param_count
resnet18 7836.71 16.32 128 224 2375.94 53.496 128 224 11.69
efficientnet_b0 5450.6 23.472 128 224 1284.98 98.423 128 224 5.29
resnet34 4511.76 28.357 128 224 1378.08 92.213 128 224 21.8
vit_tiny_patch16_224 4181.0 30.6 128 224 1197.97 105.906 128 224 5.72
efficientnet_b1 3289.75 38.895 128 240 788.17 160.716 128 240 7.79
efficientnet_b2 2542.63 50.327 128 260 579.16 219.281 128 260 9.11
resnet50 2431.07 52.637 128 224 793.04 160.45 128 224 25.56
resnet50d 2187.19 58.508 128 224 698.06 182.398 128 224 25.58
seresnet50 2128.41 60.124 128 224 664.59 191.31 128 224 28.09
vit_small_patch16_224 1931.46 66.256 128 224 624.52 203.977 128 224 22.05
regnety_040 1767.08 72.42 128 224 521.38 243.786 128 224 20.65
resnext50_32x4d 1710.41 74.821 128 224 604.65 210.757 128 224 25.03
efficientnet_b3 1550.98 82.512 128 300 358.84 354.726 128 300 12.23
resnet101 1446.82 88.455 128 224 492.24 258.259 128 224 44.55
regnety_080 1151.59 111.135 128 224 335.06 380.596 128 224 39.18
ecaresnet50t 1053.0 121.538 128 320 292.95 435.769 128 320 25.57
resnet152 1005.8 127.244 128 224 348.45 364.715 128 224 60.19
vit_base_patch16_224 827.29 154.704 128 224 268.79 475.155 128 224 86.57
efficientnetv2_rw_s 800.87 159.807 128 384 220.18 288.001 64 384 23.94
regnety_160 788.85 162.243 128 224 238.8 534.506 128 224 83.59
nfnet_f0 700.47 182.714 128 256 264.11 483.175 128 256 71.49
senet154 516.53 247.789 128 224 146.02 434.51 64 224 115.09
regnety_320 486.66 262.994 128 224 156.35 817.025 128 224 145.05
efficientnetv2_rw_m 374.12 342.119 128 416 99.07 319.099 32 416 53.24
nfnet_f1 262.09 488.357 128 320 93.72 680.34 64 320 132.63
ecaresnet269d 217.64 588.096 128 352 59.39 533.664 32 352 102.09
nfnet_f2 148.47 862.123 128 352 51.79 614.207 32 352 193.78
Raw
results_nhwc_rtx3090.csv
model infer_samples_per_sec infer_step_time infer_batch_size infer_img_size train_samples_per_sec train_step_time train_batch_size train_img_size param_count
resnet18 8647.22 14.791 128 224 3049.43 41.614 128 224 11.69
efficientnet_b0 6635.1 19.279 128 224 1580.9 79.728 128 224 5.29
resnet34 5473.07 23.374 128 224 1809.32 70.121 128 224 21.8
vit_tiny_patch16_224 3863.28 33.117 128 224 1208.08 105.055 128 224 5.72
efficientnet_b1 3730.41 34.299 128 240 976.82 129.407 128 240 7.79
resnet50 3224.27 39.685 128 224 1098.64 115.609 128 224 25.56
efficientnet_b2 3093.81 41.359 128 260 764.43 165.815 128 260 9.11
resnet50d 2797.9 45.735 128 224 960.88 132.274 128 224 25.58
seresnet50 2625.66 48.736 128 224 841.43 150.868 128 224 28.09
resnext50_32x4d 2620.92 48.824 128 224 776.9 163.863 128 224 25.03
vit_small_patch16_224 1904.9 67.18 128 224 623.67 204.282 128 224 22.05
resnet101 1851.93 69.102 128 224 661.96 191.696 128 224 44.55
efficientnet_b3 1817.91 70.396 128 300 369.7 344.342 128 300 12.23
resnet152 1292.63 99.007 128 224 456.08 278.158 128 224 60.19
regnety_040 1250.48 102.343 128 224 188.06 678.874 128 224 20.65
ecaresnet50t 1215.67 105.275 128 320 372.56 342.434 128 320 25.57
efficientnetv2_rw_s 968.67 132.123 128 384 276.98 228.605 64 384 23.94
regnety_080 810.93 157.825 128 224 103.47 1235.649 128 224 39.18
vit_base_patch16_224 798.68 160.245 128 224 265.8 480.503 128 224 86.57
nfnet_f0 709.4 180.414 128 256 267.2 477.653 128 256 71.49
regnety_160 624.0 205.109 128 224 64.31 1989.047 128 224 83.59
senet154 608.49 210.341 128 224 179.72 352.442 64 224 115.09
efficientnetv2_rw_m 455.68 280.879 128 416 119.58 263.862 32 416 53.24
regnety_320 379.15 337.572 128 224 63.78 2005.178 128 224 145.05
ecaresnet269d 263.75 485.289 128 352 72.74 434.843 32 352 102.09
nfnet_f1 260.5 491.343 128 320 93.58 681.337 64 320 132.63
nfnet_f2 148.89 859.658 128 352 51.02 623.635 32 352 193.78
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment