test-run-vision_int8_cmd_output.txt

python ./run.py  --tolerance 0.001 0.001 --cachedir /proj/gdba/shark/cache --ireebuild ../../iree-build -f onnx -g models --mode onnx --report -j 12  -r test-ru
n-vision_int8 --testsfile list1_vision_int8_run
/proj/gdba/shark/chi/src/SHARK-TestSuite/e2eshark/e2e_venv/lib/python3.10/site-packages/torchvision/io/image.py:14: UserWarning: Failed to load image Python extension: '/proj/gdba/shark/chi/src/SHARK-TestSuite/e2eshark/e2e_venv/lib/python3.10/site-packages/torchvision/image.so: undefined symbol: _ZNK3c1011StorageImpl27throw_data_ptr_access_errorEv'If you don't plan on using image functionality from `torchvision.io`, you can ignore this warning. Otherwise, there might be something wrong with your environment. Did you have `libjpeg` or `libpng` installed before building `torchvision` from source?
  warn(
Starting e2eshark tests. Using 12 processes
Cache Directory: /proj/gdba/shark/cache
Tolerance for comparing floating point (atol, rtol) = (0.001, 0.001)
Note: No Torch MLIR build provided using --torchmlirbuild. iree-import-onnx will be used to convert onnx to torch onnx mlir
IREE build: /proj/gdba/shark/chi/src/iree-build
Test run directory: /proj/gdba/shark/chi/src/SHARK-TestSuite/e2eshark/test-run-vision_int8
Since --tests or --testsfile was specified, --groups ignored
Framework:onnx mode=onnx backend=llvm-cpu runfrom=model-run runupto=inference
Test list: ['onnx/models/DarkNet53_vaiq', 'onnx/models/dla169_vaiq', 'onnx/models/efficientnet_b0.ra_in1k_vaiq', 'onnx/models/EfficientNet_b0_vaiq', 'onnx/models/efficientnet_b1.ft_in1k_vaiq', 'onnx/models/EfficientNet_b1_vaiq', 'onnx/models/efficientnet_b2.ra_in1k_vaiq', 'onnx/models/EfficientNet_b2_vaiq', 'onnx/models/efficientnet_b3.ra2_in1k_vaiq', 'onnx/models/EfficientNet_b3_vaiq', 'onnx/models/efficientnet_b4.ra2_in1k_vaiq', 'onnx/models/EfficientNet_b4_vaiq', 'onnx/models/efficientnet_b5.sw_in12k_vaiq', 'onnx/models/EfficientNet_b5_vaiq', 'onnx/models/EfficientNet_b6_vaiq', 'onnx/models/EfficientNet_b7_vaiq', 'onnx/models/efficientnet_el_pruned.in1k_vaiq', 'onnx/models/efficientnet_el.ra_in1k_vaiq', 'onnx/models/efficientnet_em.ra2_in1k_vaiq', 'onnx/models/efficientnet_es_pruned.in1k_vaiq', 'onnx/models/efficientnet_es.ra_in1k_vaiq', 'onnx/models/efficientnet_lite0.ra_in1k_vaiq', 'onnx/models/EfficientNet_v2_l_vaiq', 'onnx/models/EfficientNet_v2_m_vaiq', 'onnx/models/efficientnetv2_rw_m.agc_in1k_vaiq', 'onnx/models/efficientnetv2_rw_s.ra2_in1k_vaiq', 'onnx/models/efficientnetv2_rw_t.ra2_in1k_vaiq', 'onnx/models/EfficientNet_v2_s_vaiq', 'onnx/models/fbnetc_100.rmsp_in1k_vaiq', 'onnx/models/gernet_l.idstcv_in1k_vaiq', 'onnx/models/gernet_m.idstcv_in1k_vaiq', 'onnx/models/gernet_s.idstcv_in1k_vaiq', 'onnx/models/GoogLeNet_vaiq', 'onnx/models/inception_v3.tf_in1k_vaiq', 'onnx/models/Inception_v3_vaiq', 'onnx/models/inception_v4.tf_in1k_vaiq', 'onnx/models/lcnet_050.ra2_in1k_vaiq', 'onnx/models/lcnet_075.ra2_in1k_vaiq', 'onnx/models/lcnet_100.ra2_in1k_vaiq', 'onnx/models/mnasnet_100.rmsp_in1k_vaiq', 'onnx/models/mnasnet_small.lamb_in1k_vaiq', 'onnx/models/mobilenetv2_050.lamb_in1k_vaiq', 'onnx/models/mobilenetv2_100.ra_in1k_vaiq', 'onnx/models/mobilenetv2_110d.ra_in1k_vaiq', 'onnx/models/mobilenetv2_120d.ra_in1k_vaiq', 'onnx/models/mobilenetv2_140.ra_in1k_vaiq', 'onnx/models/MobileNetV2_vaiq', 'onnx/models/mobilenetv3_large_100.ra_in1k_vaiq', 'onnx/models/MobileNetV3_large_vaiq', 'onnx/models/mobilenetv3_small_050.lamb_in1k_vaiq', 'onnx/models/mobilenetv3_small_075.lamb_in1k_vaiq', 'onnx/models/mobilenetv3_small_100.lamb_in1k_vaiq', 'onnx/models/MobileNetV3_small_vaiq', 'onnx/models/ResNet101_vaiq', 'onnx/models/ResNet152_vaiq', 'onnx/models/ResNet18_vaiq', 'onnx/models/resnet32ts.ra2_in1k_vaiq', 'onnx/models/resnet33ts.ra2_in1k_vaiq', 'onnx/models/ResNet34_vaiq', 'onnx/models/resnet50.a1_in1k_vaiq', 'onnx/models/ResNet50_vaiq', 'onnx/models/RRDB_ESRGAN_pro_vaiq', 'onnx/models/RRDB_ESRGAN_vaiq', 'onnx/models/SqueezeNet_1_0_vaiq', 'onnx/models/SqueezeNet_1_1_vaiq', 'onnx/models/VGG11_bn_vaiq', 'onnx/models/VGG11_vaiq', 'onnx/models/VGG13_bn_vaiq', 'onnx/models/VGG13_vaiq', 'onnx/models/VGG16_bn_vaiq', 'onnx/models/VGG16_vaiq', 'onnx/models/VGG19_bn_vaiq', 'onnx/models/VGG19_vaiq', 'onnx/models/WideResNet_101_2_vaiq', 'onnx/models/WideResNet_50_2_vaiq', 'onnx/models/YoloNetV3_vaiq', 'onnx/models/Yolov8m_vaiq', 'onnx/models/Yolov8n_vaiq']
/proj/gdba/shark/chi/src/SHARK-TestSuite/e2eshark/_run_helper.py:118: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  loaded = torch.load(buf)
Test onnx/models/efficientnet_es.ra_in1k_vaiq passed
/proj/gdba/shark/chi/src/SHARK-TestSuite/e2eshark/_run_helper.py:118: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  loaded = torch.load(buf)
Test onnx/models/efficientnet_em.ra2_in1k_vaiq passed
/proj/gdba/shark/chi/src/SHARK-TestSuite/e2eshark/_run_helper.py:118: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  loaded = torch.load(buf)
Test onnx/models/efficientnet_el_pruned.in1k_vaiq passed
/proj/gdba/shark/chi/src/SHARK-TestSuite/e2eshark/_run_helper.py:118: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  loaded = torch.load(buf)
Test onnx/models/DarkNet53_vaiq passed
Test onnx/models/efficientnet_el.ra_in1k_vaiq passed
Test onnx/models/efficientnet_es_pruned.in1k_vaiq passed
Test onnx/models/efficientnet_lite0.ra_in1k_vaiq passed
/proj/gdba/shark/chi/src/SHARK-TestSuite/e2eshark/_run_helper.py:118: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  loaded = torch.load(buf)
Test onnx/models/efficientnet_b0.ra_in1k_vaiq failed [mismatch]
/proj/gdba/shark/chi/src/SHARK-TestSuite/e2eshark/_run_helper.py:118: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  loaded = torch.load(buf)
Test onnx/models/efficientnet_b2.ra_in1k_vaiq failed [mismatch]
/proj/gdba/shark/chi/src/SHARK-TestSuite/e2eshark/_run_helper.py:118: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  loaded = torch.load(buf)
Test onnx/models/efficientnet_b1.ft_in1k_vaiq passed
/proj/gdba/shark/chi/src/SHARK-TestSuite/e2eshark/_run_helper.py:118: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  loaded = torch.load(buf)
Test onnx/models/efficientnet_b4.ra2_in1k_vaiq failed [mismatch]
/proj/gdba/shark/chi/src/SHARK-TestSuite/e2eshark/_run_helper.py:118: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  loaded = torch.load(buf)
Test onnx/models/efficientnet_b3.ra2_in1k_vaiq failed [mismatch]
Test onnx/models/fbnetc_100.rmsp_in1k_vaiq passed
Test onnx/models/EfficientNet_b0_vaiq failed [mismatch]
Test onnx/models/dla169_vaiq passed
Test onnx/models/gernet_l.idstcv_in1k_vaiq passed
/proj/gdba/shark/chi/src/SHARK-TestSuite/e2eshark/_run_helper.py:118: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  loaded = torch.load(buf)
Test onnx/models/efficientnet_b5.sw_in12k_vaiq failed [mismatch]
/proj/gdba/shark/chi/src/SHARK-TestSuite/e2eshark/_run_helper.py:118: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  loaded = torch.load(buf)
Test onnx/models/EfficientNet_b6_vaiq passed
Test onnx/models/efficientnetv2_rw_t.ra2_in1k_vaiq passed
Test onnx/models/gernet_m.idstcv_in1k_vaiq passed
Test onnx/models/EfficientNet_b2_vaiq failed [mismatch]
Test onnx/models/EfficientNet_b1_vaiq passed
Test onnx/models/GoogLeNet_vaiq passed
Test onnx/models/efficientnetv2_rw_m.agc_in1k_vaiq failed [mismatch]
Test onnx/models/gernet_s.idstcv_in1k_vaiq passed
Test onnx/models/EfficientNet_b3_vaiq failed [mismatch]
Test onnx/models/lcnet_100.ra2_in1k_vaiq failed [mismatch]
Test onnx/models/lcnet_050.ra2_in1k_vaiq failed [mismatch]
Test onnx/models/Inception_v3_vaiq passed
Test onnx/models/EfficientNet_b4_vaiq failed [mismatch]
/proj/gdba/shark/chi/src/SHARK-TestSuite/e2eshark/_run_helper.py:118: FutureWarning: You are using `torch.load` with `weights_only=False` (the current default value), which uses the default pickle module implicitly. It is possible to construct malicious pickle data which will execute arbitrary code during unpickling (See https://github.com/pytorch/pytorch/blob/main/SECURITY.md#untrusted-models for more details). In a future release, the default value for `weights_only` will be flipped to `True`. This limits the functions that could be executed during unpickling. Arbitrary objects will no longer be allowed to be loaded via this mode unless they are explicitly allowlisted by the user via `torch.serialization.add_safe_globals`. We recommend you start setting `weights_only=True` for any use case where you don't have full control of the loaded file. Please open an issue on GitHub for any issues related to this experimental feature.
  loaded = torch.load(buf)
Test onnx/models/EfficientNet_v2_l_vaiq failed [mismatch]
Test onnx/models/lcnet_075.ra2_in1k_vaiq failed [mismatch]
Test onnx/models/mobilenetv2_100.ra_in1k_vaiq passed
Test onnx/models/mnasnet_small.lamb_in1k_vaiq passed
Test onnx/models/mnasnet_100.rmsp_in1k_vaiq passed
Test onnx/models/inception_v3.tf_in1k_vaiq passed
Test onnx/models/mobilenetv2_120d.ra_in1k_vaiq passed
Test onnx/models/EfficientNet_v2_s_vaiq failed [mismatch]
Test onnx/models/MobileNetV2_vaiq passed
Test onnx/models/mobilenetv2_050.lamb_in1k_vaiq failed [mismatch]
Test onnx/models/EfficientNet_b5_vaiq failed [mismatch]
Test onnx/models/mobilenetv2_110d.ra_in1k_vaiq passed
Test onnx/models/efficientnetv2_rw_s.ra2_in1k_vaiq passed
Test onnx/models/inception_v4.tf_in1k_vaiq passed
Test onnx/models/mobilenetv3_small_075.lamb_in1k_vaiq failed [mismatch]
Test onnx/models/mobilenetv2_140.ra_in1k_vaiq passed
Test onnx/models/MobileNetV3_large_vaiq passed
Test onnx/models/ResNet34_vaiq passed
Test onnx/models/MobileNetV3_small_vaiq failed [mismatch]
Test onnx/models/resnet32ts.ra2_in1k_vaiq passed
Test onnx/models/SqueezeNet_1_1_vaiq failed [mismatch]
Test onnx/models/EfficientNet_b7_vaiq passed
Test onnx/models/ResNet50_vaiq passed
Test onnx/models/mobilenetv3_large_100.ra_in1k_vaiq passed
Test onnx/models/RRDB_ESRGAN_pro_vaiq failed [model-run]
Test onnx/models/VGG11_bn_vaiq passed
Test onnx/models/VGG11_vaiq passed
Test onnx/models/resnet50.a1_in1k_vaiq passed
Test onnx/models/resnet33ts.ra2_in1k_vaiq passed
Test onnx/models/mobilenetv3_small_050.lamb_in1k_vaiq failed [mismatch]
Test onnx/models/mobilenetv3_small_100.lamb_in1k_vaiq failed [mismatch]
Test onnx/models/VGG13_vaiq passed
Test onnx/models/VGG16_vaiq passed
Test onnx/models/ResNet152_vaiq passed
Test onnx/models/EfficientNet_v2_m_vaiq failed [mismatch]
Test onnx/models/ResNet101_vaiq passed
Test onnx/models/VGG13_bn_vaiq passed
Test onnx/models/VGG19_vaiq passed
Test onnx/models/ResNet18_vaiq passed
Test onnx/models/VGG16_bn_vaiq passed
Test onnx/models/VGG19_bn_vaiq passed
Test onnx/models/WideResNet_50_2_vaiq passed
Test onnx/models/Yolov8m_vaiq failed [inference]
Test onnx/models/WideResNet_101_2_vaiq passed
Test onnx/models/YoloNetV3_vaiq passed
Test onnx/models/Yolov8n_vaiq failed [inference]
Test onnx/models/RRDB_ESRGAN_vaiq failed [mismatch]
Test onnx/models/SqueezeNet_1_0_vaiq failed [mismatch]
Generated status report /proj/gdba/shark/chi/src/SHARK-TestSuite/e2eshark/test-run-vision_int8/statusreport.md
Generated time report /proj/gdba/shark/chi/src/SHARK-TestSuite/e2eshark/test-run-vision_int8/timereport.md
Generated summary report /proj/gdba/shark/chi/src/SHARK-TestSuite/e2eshark/test-run-vision_int8/summaryreport.md

Completed run of e2e shark tests