gengwg

PyTorch Conference 2024

Key Takeaways

• Exciting times ahead for Generative AI (GenAI) on edge devices.
• Growing compute and rapid pace of model innovation for the edge.
• The PyTorch ecosystem provides key tools to support development, including:
  • AI Edge Generative API: AI Edge Torch
  • Visualize and Debug: Model Explorer
  • Tune / Experiment with new models (not covered today but worth noting):
• Torchchat

gengwg

```
$ sudo apt upgrade 
Reading package lists... Done
Building dependency tree... Done
Reading state information... Done
Calculating upgrade... Done
The following NEW packages will be installed:
  linux-headers-6.1.0-18-amd64 linux-headers-6.1.0-18-common
The following packages have been kept back:
  ca-certificates-java

gengwg

Clone

gengwg@gengwg-mbp:~$ git clone https://github.com/ggerganov/llama.cpp.git
Cloning into 'llama.cpp'...
remote: Enumerating objects: 5267, done.
remote: Counting objects: 100% (2065/2065), done.
remote: Compressing objects: 100% (320/320), done.
remote: Total 5267 (delta 1878), reused 1870 (delta 1745), pack-reused 3202
Receiving objects: 100% (5267/5267), 4.24 MiB | 13.48 MiB/s, done.

gengwg

gengwg@gengwg-mbp:~$ curl -L https://iterm2.com/shell_integration/install_shell_integration_and_utilities.sh | bash
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
100  5255  100  5255    0     0  27971      0 --:--:-- --:--:-- --:--:-- 28715
Downloading script from https://iterm2.com/shell_integration/zsh and saving it to /Users/gengwg/.iterm2_shell_integration.zsh...
Checking if /Users/gengwg/.zshrc contains iterm2_shell_integration...
Appending source command to /Users/gengwg/.zshrc...
Downloading imgcat...
Downloading imgls...
Downloading it2api...

gengwg

$ https_proxy="http://fwdproxy:8080" k deprecations --k8s-version=v1.26.0
W0610 10:16:52.170210 3539450 warnings.go:70] v1 ComponentStatus is deprecated in v1.19+
W0610 10:16:53.178949 3539450 warnings.go:70] policy/v1beta1 PodSecurityPolicy is deprecated in v1.21+, unavailable in v1.25+
RESULTS:
Deprecated APIs:

ComponentStatus found in /v1
	 ├─ ComponentStatus (and ComponentStatusList) holds the cluster validation info. Deprecated: This API is deprecated in v1.19+
		-> GLOBAL: scheduler
		-> GLOBAL: controller-manager

gengwg

$ brew install --cask docker
==> Downloading https://raw.githubusercontent.com/Homebrew/homebrew-cask/fbcbfbdf7379500b5647725f560bb538f367a931/Casks/docker.rb
######################################################################################################################################################################################################### 100.0%
==> Downloading https://desktop.docker.com/mac/main/arm64/106363/Docker.dmg
######################################################################################################################################################################################################### 100.0%
==> Installing Cask docker
==> Moving App 'Docker.app' to '/Applications/Docker.app'
==> Linking Binary 'docker' to '/usr/local/bin/docker'
Password:
==> Linking Binary 'docker-compose' to '/usr/local/bin/docker-compose'

gengwg

$ lsmod | grep kvm
kvm_intel             335872  0
kvm                  1036288  1 kvm_intel
irqbypass              16384  1 kvm

$ curl -Lo minikube https://storage.googleapis.com/minikube/releases/latest/minikube-linux-amd64
$ chmod +x minikube 
$ ./minikube version
minikube version: v1.24.0
commit: 76b94fb3c4e8ac5062daf70d60cf03ddcc0a741b

gengwg

~/volcano$ ./hack/local-up-volcano.sh 
Checking prerequisites
Found kubectl, version: Client Version: v1.20.0
Preparing...
Checking helm
Found helm, version: version.BuildInfo{Version:"v3.3.4", GitCommit:"a61ce5633af99708171414353ed49547cf05013d", GitTreeState:"clean", GoVersion:"go1.14.9"}
Generating volcano deploy yaml
which: no controller-gen in (/home/gengwg/.krew/bin:/usr/local/bin:/usr/local/sbin:/usr/bin:/usr/sbin:/home/gengwg/bin:/home/gengwg/nc//bin:/home/gengwg/.local/bin)
mkdir -p _output/bin
mkdir -p _output/release

gengwg

$ cat tf-mnist.yaml 
apiVersion: batch.volcano.sh/v1alpha1
kind: Job
metadata:
  name: tensorflow-dist-mnist
spec:
  minAvailable: 3
  schedulerName: volcano
  plugins:
    env: []

gengwg

Problem

Despite modifying my hosts file, the DNS address for hosts is still displayed when using the host command. For instance:

$ grep -E "hosts|ipnodes" /etc/nsswitch.conf
hosts: files dns mymachines myhostname

$ grep google.com /etc/hosts
2401:db00:25ff:c83:: google.com