Skip to content

Instantly share code, notes, and snippets.

@jdmichaud
Forked from smoser/README.md
Last active December 11, 2017 12:53
Show Gist options
  • Save jdmichaud/863cd34739d0a3f3ab0588caa955f920 to your computer and use it in GitHub Desktop.
Save jdmichaud/863cd34739d0a3f3ab0588caa955f920 to your computer and use it in GitHub Desktop.
ubuntu-auto-install: install a ubuntu image with kvm

Purpose

For testing purposes, you will want to quickly spawn blank VMs with an ssh access to try out some stuff. This Gist from @smoser allows you to do just that. It:

  1. Provides a script to automatically download, install and configure an ubuntu base VM from your terminal using qemu/kvm
  2. Spawn blank and dispensable VMs from that base read-only VM

Quick start

Create the base VM

/!\ Prerequisites: You need qemu and kvm installed. Tested on ubuntu 16.04.

Create a new ubuntu xenial-based (should work with any recent version) VM by cloning this repo, cd into it and then:

./ubuntu-auto-install xenial
cp xenial-amd64.img xenial-amd64.pristine.img
chmod -w xenial-amd64.pristine.img

This creates a base VM with sshd preinstalled and make it read only.

Span blank VMs from the base VM

Then base a qcow2 copy-on-write VM on it:

qemu-img create -f qcow2 -b disk.img.pristine disk.img

Then launch a VM on this blank image. Here we launch it with:

  • 1G RAM (-m 1024)
  • In an isolated network
  • with the local port 2222 mapped to the VM port 22 (in which sshd has been started)
  • with no graphic
qemu-system-x86_64 -enable-kvm -m 1024 \
      -device virtio-net-pci,netdev=net00 -netdev type=user,id=net00 \
      -drive if=virtio,file=disk.img,cache=unsafe \
      -redir tcp:2222::22 -nographic

Gist content

notes.txt

Just some notes on how to boot a d-i kernel and initramfs in qemu for installation and preseed.

preseed

Fully automated installation preseed.

script-to-latecommand

Running a full script inside a late command in d-i can be tricky, as you have to get quoting correct for shell and for d-i preseed syntax. This script will output a preseed string that will run your script.

 $ ./script-to-latecommand any-script >> preseed
 $ tail -n 1 preseed
 d-i     preseed/late_command string in-target sh -c ' set -ef; f=$1; .... > $f.output 2>&1; ' IyEvYm...dWIK /root/latecommand

After install output of your command will be in /root/latecommand.output and the script itself in /root/latecommand.

ubuntu-auto-install

This will download a mini-iso and extract kernel and initramfs and then boot it with automated install using the provided preseed file. No root is required.

The nifty thing about this script is that root is not required nor any web serving of the preseed. It extracts the kernel/initramfs from the mini iso, and re-packs the initramfs with the preseed inside it.

#!/bin/sh
set -e
echo blacklist vga16fb > /etc/modprobe.d/novga16fb.conf;
cat >> /etc/default/grub <<"EOF"
GRUB_CMDLINE_LINUX_DEFAULT=""
GRUB_TERMINAL=console
EOF
update-initramfs -u
update-grub
## how i verified hwe-t kernel boots / functions.
$ sudo apt-get update
$ sudo apt-get install qemu-system-x86
$ sudo chmod 666 /dev/kvm
$ burl="http://archive.ubuntu.com/ubuntu/dists/precise-proposed/main/installer-amd64/current/images/trusty-netboot/ubuntu-installer/"
$ wget "$burl/amd64/linux" -O kernel
$ wget "$burl/amd64/initrd.gz" -O initrd
$ md5sum kernel initrd
eabacd97fa75fcf0946531f597390107 kernel
ccf58028315106c22b7187e1026a988a initrd
$ qemu-img create -f qcow2 disk.img 4G
$ PRESEED_URL=http://some.url
$ qemu-system-x86_64 -enable-kvm -m 1024 -curses \
-device virtio-net-pci,netdev=net00 -netdev type=user,id=net00 \
-drive if=virtio,file=disk.img,cache=unsafe \
-no-reboot -kernel kernel -initrd initrd \
-append "apt-setup/proposed=true nomodeset fb=false priority=critical locale=en_US url=$PRESEED_URL"
# the kernel command line params are:
# apt-setup/proposed=true: required per LP: #1172101
# nomodeset fb=false: for -curses friendliness
#
## took defaults for everything other than
## * http_proxy (put in local proxy: http://192.168.1.130:3128/ )
## * user: ubuntu passwd: ubuntu
## Then boot the system, removing the '-kernel and -initrd'
## verify that I'm running a 3.13 kernel
$ uname -r
3.13.0-30-generic
## just for my random information, you can then make it boot
## sanely in curses mode only by:
## echo "blacklist vga16fb" | sudo tee /etc/modprobe.d/novga16fb.conf
## sudo sed -i -e 's,\(GRUB_CMDLINE_LINUX_DEFAULT\)=.*,\1="quiet nomodeset",' \
## -e 's,#GRUB_TERMINAL=console,GRUB_TERMINAL=console,' \
## /etc/default/grub
## sudo update-grub
## sudo update-initramfs -u
# based on http://bit.ly/uquick-doc
#d-i mirror/country string manual
#d-i mirror/http/hostname string mymirror.org
#d-i mirror/http/directory string /rep
#d-i mirror/http/proxy string http://myprox:port/
#d-i mirror/http/mirror select mymirror.org
d-i debian-installer/locale string en_US.UTF-8
d-i debian-installer/splash boolean false
d-i console-setup/ask_detect boolean false
d-i console-setup/layoutcode string us
d-i console-setup/variantcode string
d-i netcfg/get_nameservers string
d-i netcfg/get_ipaddress string
d-i netcfg/get_netmask string 255.255.255.0
d-i netcfg/get_gateway string
d-i netcfg/confirm_static boolean true
d-i clock-setup/utc boolean true
d-i partman-auto/method string regular
d-i partman-lvm/device_remove_lvm boolean true
d-i partman-lvm/confirm boolean true
d-i partman/confirm_write_new_label boolean true
d-i partman/choose_partition select Finish partitioning and write changes to disk
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true
d-i partman/default_filesystem string ext3
d-i clock-setup/utc boolean true
d-i clock-setup/ntp boolean true
d-i clock-setup/ntp-server string ntp.ubuntu.com
d-i passwd/root-login boolean false
d-i passwd/make-user boolean true
d-i passwd/user-fullname string ubuntu
d-i passwd/username string ubuntu
d-i passwd/user-password-crypted password $6$.1eHH0iY$ArGzKX2YeQ3G6U.mlOO3A.NaL22Ewgz8Fi4qqz.Ns7EMKjEJRIW2Pm/TikDptZpuu7I92frytmk5YeL.9fRY4.
d-i passwd/user-uid string
d-i user-setup/allow-password-weak boolean false
d-i user-setup/encrypt-home boolean false
d-i passwd/user-default-groups string adm cdrom dialout lpadmin plugdev sambashare
d-i apt-setup/services-select multiselect security
d-i apt-setup/security_host string security.ubuntu.com
d-i apt-setup/security_path string /ubuntu
d-i debian-installer/allow_unauthenticated string false
d-i pkgsel/upgrade select safe-upgrade
d-i pkgsel/language-packs multiselect
d-i pkgsel/update-policy select none
d-i pkgsel/updatedb boolean true
d-i grub-installer/skip boolean false
d-i lilo-installer/skip boolean false
d-i grub-installer/only_debian boolean true
d-i grub-installer/with_other_os boolean true
d-i finish-install/keep-consoles boolean false
d-i finish-install/reboot_in_progress note
d-i cdrom-detect/eject boolean true
d-i debian-installer/exit/halt boolean false
d-i debian-installer/exit/poweroff boolean false
d-i pkgsel/include string vim openssh-server
#d-i preseed/late_command string my-late-command
#!/bin/sh
Usage() {
cat <<EOF
Usage: ${0##*/} file [name]
write a d-i safe command to run 'file' in a late command.
useful for complex commands and not dealing with shell quoting.
EOF
}
file="$1"
name="${2:-/root/latecommand}"
[ $# -eq 0 ] && { Usage 1>&2; exit 1; }
[ "$1" = "-h" -o "$1" = "--help" ] && { Usage; exit 0; }
if [ $# -eq 1 ]; then
shift
else
shift 2
fi
error() { echo "$@" 1>&2; }
dump_tmpl() {
cat <<"EOF"
in-target sh -c '
set -ef;
f=$1;
shift;
[ ${f#/} = $f ] && f=./$f;
echo $0 | base64 --decode > $f;
chmod u+x $f;
$f $* > $f.output 2>&1;
'
EOF
}
get_tmpl() {
dump_tmpl | sed ':a; N; $!ba; s/\n[ ]*/ /g'
}
writecmd() {
local file="$1" name="$2" shellcmd="" b64=""
shift 2
[ -f "$file" ] || { error "$file: not a file"; return 1; }
shellcmd=$(get_tmpl) || { error "getting template failed"; return 1; }
b64=$(base64 --wrap=0 "$file") ||
{ error "base64 encode of $fail failed"; return 1; }
printf "d-i\t preseed/late_command string %s %s %s %s\n" "$shellcmd" "$b64" "$name" "$*"
}
writecmd "$file" "$name" "$@"
#!/bin/bash
VERBOSITY=0
TEMP_D=""
MY_PATH=$(readlink -f "$0")
MY_D=$(cd "${MY_PATH%/*}" && pwd)
DEF_PRESEED="$MY_D/preseed"
DEF_SIZE=4
DEF_MIRROR="http://archive.ubuntu.com/ubuntu"
DEF_ARCH=$(uname -m)
[ "$DEF_ARCH" = "x86_64" ] && DEF_ARCH=amd64
error() { echo "$@" 1>&2; }
errorp() { printf "$@" 1>&2; }
fail() { [ $# -eq 0 ] || error "$@"; exit 1; }
failp() { [ $# -eq 0 ] || errorp "$@"; exit 1; }
Usage() {
cat <<EOF
Usage: ${0##*/} [ options ] release [ arch [ size ] ]
Do an install of Ubuntu for release.
arch : the arch to use (amd64 i386). Default: ${DEF_ARCH}
size : size of the image (in GigaBytes). Default: ${DEF_SIZE}
options:
-o | --output IMAGE_FILE write the image to IMAGE_FILE
default: <release>-<arch>.img
-s | --preseed PRESEED use the preseed at preseed
default: ${DEF_PRESEED}
-m | --mirror MIRROR mirror to download iso from MIRROR
default: ${DEF_MIRROR}
--iso ISO use ISO file rather than downloading
--late-command-file F execute file F in target as late command
EOF
}
bad_Usage() { Usage 1>&2; [ $# -eq 0 ] || error "$@"; exit 1; }
cleanup() {
[ -z "${TEMP_D}" -o ! -d "${TEMP_D}" ] || rm -Rf "${TEMP_D}"
}
dl() {
local url="$1" out="${2}" opts=""
[ "$url" = "$out" ] && return
[ $VERBOSITY -lt 1 ] && opts="-q"
local tfile="" tdir="" ret=""
tdir=$(dirname "$out")
[ -d "$tdir" ] || mkdir -p "$tdir" || return 1
tdir=$(cd "$tdir" &&pwd)
tfile=$(mktemp $tdir/${out##*/}.XXXXXX)
case "$url" in
http://*|ftp://*) wget $opts "$url" -O "$tfile";;
*) cat "${url}" > "$tfile";;
esac
ret=$?
[ $ret -eq 0 ] && mv "$tfile" "$out" && return 0
rm -f "$tfile"
return $ret
}
debug() {
local level=${1}; shift;
[ "${level}" -ge "${VERBOSITY}" ] && return
error "${@}"
}
extract_file_from_iso() {
# extract_file_from_iso(iso, file, out)
# extract file from iso and store in output.
# does not work for zero length files
local iso="$1" file="$2" out="$3" ret=$?
local tmp="$out.tmp.$$"
isoinfo -RJ -x "$file" -i "$iso" > "$tmp" && [ -s "$tmp" ]
ret=$?
[ $ret -eq 0 ] && mv "$tmp" "$out" && return 0
rm -f "$tmp"
return $ret
}
search_iso_for_file() {
# search_iso_for_file(iso, output, files)
# search iso for each file in files. store the first found in output
local iso="$1" out="$2" i=""
shift 2
for i in "$@"; do
extract_file_from_iso "$iso" "$i" "$out" && return 0
done
error "did not find $out on $iso. searched $*"
return 1
}
short_opts="hm:o:p:v"
long_opts="help,iso:,late-command-file:,mirror:,output:,preseed:,verbose"
getopt_out=$(getopt --name "${0##*/}" \
--options "${short_opts}" --long "${long_opts}" -- "$@") &&
eval set -- "${getopt_out}" ||
bad_Usage
release="${DEF_RELEASE}"
preseed="${DEF_PRESEED}"
output=""
mirror="${DEF_MIRROR}"
arch="${DEF_ARCH}"
iso=""
late_command_file=""
while [ $# -ne 0 ]; do
cur=${1}; next=${2};
case "$cur" in
-h|--help) Usage ; exit 0;;
-i|--iso) iso=${2}; shift;;
-m|--mirror) mirror=${2}; shift;;
-o|--output) output=${2}; shift;;
-p|--preseed) preseed=${2}; shift;;
-v|--verbose) VERBOSITY=$((${VERBOSITY}+1));;
--late-command-file) late_command_file="$2";;
--) shift; break;;
esac
shift;
done
[ $# -ne 0 ] || bad_Usage "must provide arguments"
[ $# -lt 1 -o $# -gt 3 ] && bad_Usage "must provide 1,2, or 3 args"
release=$1
arch=${2:-${DEF_ARCH}}
size=${3:-${DEF_SIZE}}
size=${size%G}
[ -n "$output" ] || output="${release}-${arch}.img"
TEMP_D=$(mktemp -d "${TMPDIR:-/tmp}/${0##*/}.XXXXXX") ||
fail "failed to make tempdir"
trap cleanup EXIT
if [ -z "$iso" ]; then
iso="${release}-${arch}-mini.iso"
if [ -f "$iso" ]; then
debug 1 "using existing iso ${iso}"
else
found=false
for pocket in "$release-updates" "$release"; do
url="${mirror}/dists/$pocket/main/installer-$arch/current/images/netboot/mini.iso"
debug 1 "downloading ${url}"
dl "${url}" "${iso}" && found=true && break
done
$found || fail "failed to download $iso from $url"
fi
fi
dl "$preseed" preseed.cfg ||
fail "failed to download preseed: ${preseed}"
late_command=""
if [ -n "${late_command_file}" ]; then
[ -f "$late_command_file" ] || fail "$late_command_file: not a file"
sed -i '/^d-i.*preseed.late_command/d' "$preseed.cfg" ||
fail "failed to remove late command from $preseed.cfg"
out=$("${MY_D}/script-to-latecommand" "$late_command_file") ||
fail "failed script-to-latecommand $late_command_file"
printf "d-i\tpreseed/late_command\tstring\t%s" "$out" >> "$preseed.cfg"
fi
debug 1 "extracting kernel and ramdisk"
kernel_paths="/linux /install/vmlinuz"
initrd_paths="/initrd.gz /install/initrd.gz"
search_iso_for_file "$iso" "linux.dist" $kernel_paths || fail
search_iso_for_file "$iso" "initrd.gz.dist" $initrd_paths || fail
debug 1 "repacking initramfs"
zcat initrd.gz.dist > initrd &&
echo "./preseed.cfg" | cpio -o --format=newc --append -F initrd &&
gzip -9 initrd -c > initrd.gz && rm -f initrd ||
fail "failed to repack initrd"
cp "linux.dist" "linux"
debug 1 "creating a ${size}G disk in ${output}"
qemu-img create -f qcow2 "${output}" "${size}G" ||
fail "failed to create image"
kparms="nomodeset fb=false"
set -x
${KVM:-kvm} -kernel linux -initrd initrd.gz \
-append "priority=critical locale=en_US --- $kparms" \
-drive "file=${output},if=virtio,cache=unsafe" \
-cdrom "${iso}" -m 512 -boot d -no-reboot \
-vga std -curses
# vi: ts=4 noexpandtab
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment