Download Packer binary from here and add it to your path. Qemu and associated utilities will be needed.
Its better to use Bento project as a starting point and customize with extra scripts we need, rather than maintaining all of the configs needed to build with Packer. QEMU is supported even though its not listed. Command would be
packer build -only=qemu -var headless=true ubuntu-16.04-amd64.json
The below json is a packer-template to create an ubuntu 16.04 based appliance image, that would output qcow2 image for production, as well as box for developers to work with vagrant. User can pass variables using -var flag from packer build command-line to override defaults.
mkdir http scripts
cat << "EOF" > ubuntu-16.04-amd64.json
{
"builders": [{
"type": "qemu",
"iso_url": "{{user `mirror`}}/16.04/ubuntu-16.04.3-server-amd64.iso",
"iso_checksum": "{{user `iso_checksum`}}",
"iso_checksum_type": "{{user `iso_checksum_type`}}",
"output_directory": "output",
"vm_name": "packer-ubuntu-16.04-amd64",
"disk_size": "{{user `disk_size`}}",
"headless": "{{user `headless`}}",
"http_directory": "http",
"format": "{{user `format`}}",
"boot_wait": "5s",
"boot_command": [
"<enter><wait><f6><esc><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
"<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
"<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
"<bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs><bs>",
"/install/vmlinuz<wait>",
" auto<wait>",
" console-setup/ask_detect=false<wait>",
" console-setup/layoutcode=us<wait>",
" console-setup/modelcode=pc105<wait>",
" debconf/frontend=noninteractive<wait>",
" debian-installer=en_US.UTF-8<wait>",
" fb=false<wait>",
" initrd=/install/initrd.gz<wait>",
" kbd-chooser/method=us<wait>",
" keyboard-configuration/layout=USA<wait>",
" keyboard-configuration/variant=USA<wait>",
" locale=en_US.UTF-8<wait>",
" netcfg/get_domain=vm<wait>",
" netcfg/get_hostname=vagrant<wait>",
" grub-installer/bootdev=/dev/vda<wait>",
" noapic<wait>",
" preseed/url=http://{{ .HTTPIP }}:{{ .HTTPPort }}/preseed.cfg<wait>",
" -- <wait>",
"<enter><wait>"
],
"ssh_timeout": "{{user `ssh_timeout`}}",
"ssh_username": "vagrant",
"ssh_password": "vagrant",
"shutdown_command": "echo 'vagrant'|sudo -S shutdown -P now",
"qemuargs": [
["-cpu", "Haswell"],
["-m", "{{user `memory`}}"],
["-smp", "{{user `cpus`}}"]
]
}],
"provisioners": [{
"type": "shell",
"environment_vars": [
"HOME_DIR=/home/vagrant",
"http_proxy={{user `http_proxy`}}",
"https_proxy={{user `https_proxy`}}",
"no_proxy={{user `no_proxy`}}"
],
"execute_command": "echo 'vagrant' | {{.Vars}} sudo -S -E sh -eux '{{.Path}}'",
"scripts": [
"scripts/sudo.sh",
"scripts/dpdk.sh",
"scripts/network.sh"
]
}],
"post-processors": [{
"type": "vagrant",
"keep_input_artifact": true,
"compression_level": "{{user `compression_level`}}",
"output": "output/ubuntu-16.04-amd64-{{.Provider}}.box"
}],
"variables": {
"compression_level": "6",
"cpus": "10",
"disk_size": "5000",
"format": "qcow2",
"headless": "true",
"iso_checksum": "a06cd926f5855d4f21fb4bc9978a35312f815fbda0d0ef7fdc846861f4fc4600",
"iso_checksum_type": "sha256",
"memory": "4096",
"mirror": "http://releases.ubuntu.com",
"ssh_timeout": "60m"
}
}
EOF
The above template expects a preseed.cfg file in the http directory to bootstrap the automated install.
cat << "EOF" > http/preseed.cfg
choose-mirror-bin mirror/http/proxy string
d-i base-installer/kernel/override-image string linux-server
d-i clock-setup/utc boolean true
d-i clock-setup/utc-auto boolean true
d-i finish-install/reboot_in_progress note
d-i grub-installer/only_debian boolean true
d-i grub-installer/with_other_os boolean true
d-i partman-auto-lvm/guided_size string max
d-i partman-auto/choose_recipe select atomic
d-i partman-auto/method string lvm
d-i partman-lvm/confirm boolean true
d-i partman-lvm/confirm boolean true
d-i partman-lvm/confirm_nooverwrite boolean true
d-i partman-lvm/device_remove_lvm boolean true
d-i partman/choose_partition select finish
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true
d-i partman/confirm_write_new_label boolean true
d-i passwd/user-fullname string vagrant
d-i passwd/user-uid string 1000
d-i passwd/user-password password vagrant
d-i passwd/user-password-again password vagrant
d-i passwd/username string vagrant
d-i pkgsel/include string cloud-init openssh-server
d-i pkgsel/install-language-support boolean false
d-i pkgsel/update-policy select none
d-i pkgsel/upgrade select full-upgrade
d-i time/zone string UTC
d-i user-setup/allow-password-weak boolean true
d-i user-setup/encrypt-home boolean false
tasksel tasksel/first multiselect standard, server
EOF
There are 3 scripts under scripts directory registered in the provisioners section of the template. Users can add/delete to customize the provisioning steps to build their appliance image. One helpful script is to setup systemd to bringup all detected network interfaces with DHCP on boot as shown below. This helps solve some of the cloud deployment problems with multi-nic use-cases.
cat << "END" > scripts/network.sh
#!/bin/sh
cat > /etc/systemd/network/dhcp.network << "EOF"
[Match]
Name=e*
[Network]
DHCP=yes
EOF
systemctl enable systemd-networkd
systemctl restart systemd-networkd
END
Installing DPDK and ensuring the required settings.
cat << "END" > scripts/dpdk.sh
#!/bin/bash
export DEBIAN_FRONTEND=noninteractive
wget -q --spider http://google.com
if [ $? -ne 0 ]; then
echo "ERROR: You seem to be offline or behind a proxy"
exit 1
fi
# Dependencies
apt-get update && apt-get -y install \
build-essential git python libpcap-dev \
linux-headers-$(uname -r)
# Isolate CPUs, allocate hugepages in GRUB
sed -i '/GRUB_CMDLINE_LINUX_DEFAULT/c\GRUB_CMDLINE_LINUX_DEFAULT="console=tty0 console=ttyS0,115200n8 hugepagesz=1G hugepages=1 isolcpus=1,2 rcu_nocbs=1,2 nohz_full=1,2 nmi_watchdog=0"' /etc/default/grub
cat /etc/default/grub
update-grub
# Script for system optimization
mkdir -p /opt/scripts
cat << "EOF" > /opt/scripts/isolate.sh
#!/bin/bash
echo 1 > /sys/bus/workqueue/devices/writeback/cpumask
clear_mask=0x6 #Isolate CPU1 and CPU2 from IRQs
for i in /proc/irq/*/smp_affinity
do
echo "obase=16;$(( 0x$(cat $i) & ~$clear_mask ))" | bc > $i
done
EOF
chmod 744 /opt/scripts/isolate.sh
# Systemd unit to run the above script
cat << "EOF" > /etc/systemd/system/myisolate.service
[Unit]
Description=PVP isolate linux stuff
[Service]
ExecStart=/opt/scripts/isolate.sh
[Install]
WantedBy=default.target
EOF
echo 'nodev /dev/hugepages hugetlbfs pagesize=1GB 0 0' | tee -a /etc/fstab
RTE_SDK=$HOME/dpdk
RTE_TARGET=build
echo "export RTE_SDK=$RTE_SDK" | tee -a $HOME/.profile
echo "export RTE_TARGET=$RTE_TARGET" | tee -a $HOME/.profile
cat $HOME/.profile
git clone http://dpdk.org/git/dpdk -b v17.05 $RTE_SDK
#Enable PCAP PMD
sed -ri 's,(PMD_PCAP=).*,\1y,' $RTE_SDK/config/common_base
make config T=x86_64-native-linuxapp-gcc -C $RTE_SDK
make -j4 -C $RTE_SDK
# Kernel modules to load at boot
modprobe -v uio
insmod $RTE_SDK/$RTE_TARGET/kmod/igb_uio.ko
cp -f $RTE_SDK/$RTE_TARGET/kmod/igb_uio.ko /lib/modules/$(uname -r)
echo "uio" | tee -a /etc/modules
echo "igb_uio" | tee -a /etc/modules
depmod
chown vagrant:vagrant /home/vagrant -R
END
The below script would solve some of the vagrant related issues when using the box generated at the end
cat << "EOF" > scripts/sudo.sh
#!/bin/bash
sed -i -e '/Defaults\s\+env_reset/a Defaults\texempt_group=sudo' /etc/sudoers;
cat /etc/sudoers;
# Set up password-less sudo for the vagrant user
echo 'vagrant ALL=(ALL) NOPASSWD:ALL' > /etc/sudoers.d/99_vagrant;
chmod 440 /etc/sudoers.d/99_vagrant;
cat /etc/sudoers.d/99_vagrant;
EOF
Your directory must now look like this
$ tree
.
├── http
│ └── preseed.cfg
├── scripts
│ ├── dpdk.sh
│ ├── network.sh
│ └── sudo.sh
└── ubuntu-16.04-amd64.json
2 directories, 5 files
Run packer
packer build ubuntu-16.04-amd64.json
# faster download (uncached) # packer build -var mirror=http://mirrors.cat.pdx.edu/ubuntu-releases ubuntu-16.04-amd64.json
...
qemu: The VM will be run headless, without a GUI. If you want to
qemu: view the screen of the VM, connect via VNC without a password to
qemu: 127.0.0.1:5981
...
(Optional) Use VNC Viewer to watch the automated install.
Upon completion you should see 2 files under output directory. One will be qcow2 file and the other will be box file.
ll -h output
total 2.9G
-rw-r--r-- 1 stack stack 2.1G Jul 2 09:55 packer-ubuntu-16.04-amd64
-rw-rw-r-- 1 stack stack 877M Jul 2 09:55 ubuntu-16.04-amd64-libvirt.box
qemu-img info output/packer-ubuntu-16.04-amd64
image: output/packer-ubuntu-16.04-amd64
file format: qcow2
virtual size: 4.9G (5242880000 bytes)
disk size: 2.0G
cluster_size: 65536
Format specific information:
compat: 1.1
lazy refcounts: false
refcount bits: 16
corrupt: false
For developers to use the box output with vagrant they would need to install vagrant and vagrant-libvirt plugin. Steps can be found here along with a sample Vagrantfile.
The following command adds the above box to vagrant with a name that can be consumed through the Vagrantfile
vagrant box add --name krsna1729/dpdk-testpmd output/ubuntu-16.04-amd64-libvirt.box