vuiseng9

cheatsheet

# based on following
https://github.com/openvinotoolkit/openvino/wiki/BuildingForLinux
(new) https://github.com/openvinotoolkit/openvino/blob/master/docs/dev/build_linux.md

# create conda env and activate environment (optional but recommended, use python 3.8/3.9)

git clone https://github.com/openvinotoolkit/openvino
# checkout tag or commit according

vuiseng9

Setup

pip install transformers torch
git clone https://huggingface.co/EleutherAI/gpt-j-6b # depends on git-lfs 

Run following as python script

from transformers import AutoTokenizer, pipeline

vuiseng9

import os
import logging as log
from openvino.runtime import Core, PartialShape, serialize

log.info = print

def get_input_output_names(ports):
    return [port.any_name for port in ports]

vuiseng9

# Openvino==2022.1.0
import sys

from openvino.runtime import Core

DELIMITER = ' | '

if len(sys.argv) < 3:
    print("Please provide path to model xml file as a first arg and"
          " path to output text file to dump model constants.")

vuiseng9

*.pyc
.pytest_cache

vuiseng9

#!/bin/sh

set -e

# Auto-Get the latest commit sha via command line.
get_latest_release() {
    tag=$(curl --silent "https://api.github.com/repos/${1}/releases/latest" | # Get latest release from GitHub API
          grep '"tag_name":'                                              | # Get tag line
          sed -E 's/.*"([^"]+)".*/\1/'                                    ) # Pluck JSON value

vuiseng9

Objective: train a resnet18 for CIFAR10 dataset

Top 1 accuracy of resnet18/CIFAR10 in this repo achieves 93%. We are not using this because it defines/implements its own Resnet. We would like to use the out-of-the-box torchvision resnet18 definition. NNCF provides an image classification example which utilizes torchvision resnet definition.

# Step 1: Create a new virtualenv or conda environment, make sure the env is activated

# Step 2: Install VS's fork of NNCF
git clone https://github.com/vuiseng9/nncf
cd nncf

vuiseng9

import time
import numpy as np
import logging as log

from openvino.runtime import AsyncInferQueue, Core, PartialShape
from openvino.tools.benchmark.utils.constants import CPU_DEVICE_NAME

log.info = print

model_path="/data1/vchua/jpqd-bert/r0.010-squad-bert-b-mvmt-8bit/ir/squad-BertForQuestionAnswering.cropped.8bit.onnx"

vuiseng9

Evaluation of different BERT models on SQuADv1.1

https://github.com/huggingface/transformers

# following has been validated with transformers v4.18

# 24 Layers
# https://huggingface.co/bert-large-uncased-whole-word-masking-finetuned-squad
model=bert-large-uncased-whole-word-masking-finetuned-squad

vuiseng9

#!/usr/bin/env python3
# -*- coding: utf-8 -*-
# Copyright (C) 2018-2022 Intel Corporation
# SPDX-License-Identifier: Apache-2.0

import argparse
import logging as log
import sys

import time