Skip to content

Instantly share code, notes, and snippets.

@tai271828
Forked from smoser/README.md
Created October 12, 2020 17:14
Show Gist options
  • Save tai271828/1adea534577f02553c6f567ce19a2197 to your computer and use it in GitHub Desktop.
Save tai271828/1adea534577f02553c6f567ce19a2197 to your computer and use it in GitHub Desktop.
ubuntu-auto-install: install a ubuntu image with kvm

Ubuntu Auto Install

This allows user to do automated installation of ubuntu. Provided here a a few different files/scripts for making this easier.

I've used this to verify that d-i kernel and initramfs from -proposed work as shown in bug 1511497.

The provided 'preseed' below can be used as a fully automated installation of ubuntu. Boot with qemu like:

# note python web server is useful for this (python -m SimpleHTTPServer 9999)
$ PRESEED_URL=http://some.ip:9999/preseed
$ qemu-img create -f raw disk.img 4G
$ qemu-system-x86_64 -enable-kvm -m 1024 -curses \
  -device virtio-net-pci,netdev=net00  -netdev type=user,id=net00 \
  -drive if=virtio,file=disk.img,cache=unsafe  \
  -no-reboot -kernel kernel -initrd initrd \
  -append "apt-setup/proposed=true nomodeset fb=false priority=critical locale=en_US url=$PRESEED_URL"

# system will shut off, then you move the 'disk.img' to pristine
$ mv disk.img disk.img.pristine
$ chmod -w disk.img.pristine
$ qemu-img create -f qcow2 -b disk.img.pristine disk.img
$ qemu-system-x86_64 -enable-kvm -m 1024 -curses \
  -device virtio-net-pci,netdev=net00  -netdev type=user,id=net00 \
  -drive if=virtio,file=disk.img,cache=unsafe

notes.txt

Just some notes on how to boot a d-i kernel and initramfs in qemu for installation and preseed.

preseed

Fully automated installation preseed.

script-to-latecommand

Running a full script inside a late command in d-i can be tricky, as you have to get quoting correct for shell and for d-i preseed syntax. This script will output a preseed string that will run your script.

 $ ./script-to-latecommand any-script >> preseed
 $ tail -n 1 preseed
 d-i     preseed/late_command string in-target sh -c ' set -ef; f=$1; .... > $f.output 2>&1; ' IyEvYm...dWIK /root/latecommand

After install output of your command will be in /root/latecommand.output and the script itself in /root/latecommand.

ubuntu-auto-install

This will download a mini-iso and extract kernel and initramfs and then boot it with automated install using the provided preseed file. No root is required.

The nifty thing about this script is that root is not required nor any web serving of the preseed. It extracts the kernel/initramfs from the mini iso, and re-packs the initramfs with the preseed inside it.

#!/bin/sh
set -e
echo blacklist vga16fb > /etc/modprobe.d/novga16fb.conf;
cat >> /etc/default/grub <<"EOF"
GRUB_CMDLINE_LINUX_DEFAULT=""
GRUB_TERMINAL=console
EOF
update-initramfs -u
update-grub
## how i verified hwe-t kernel boots / functions.
$ sudo apt-get update
$ sudo apt-get install qemu-system-x86
$ sudo chmod 666 /dev/kvm
$ burl="http://archive.ubuntu.com/ubuntu/dists/precise-proposed/main/installer-amd64/current/images/trusty-netboot/ubuntu-installer/"
$ wget "$burl/amd64/linux" -O kernel
$ wget "$burl/amd64/initrd.gz" -O initrd
$ md5sum kernel initrd
eabacd97fa75fcf0946531f597390107 kernel
ccf58028315106c22b7187e1026a988a initrd
$ qemu-img create -f qcow2 disk.img 4G
$ PRESEED_URL=http://some.url
$ qemu-system-x86_64 -enable-kvm -m 1024 -curses \
-device virtio-net-pci,netdev=net00 -netdev type=user,id=net00 \
-drive if=virtio,file=disk.img,cache=unsafe \
-no-reboot -kernel kernel -initrd initrd \
-append "apt-setup/proposed=true nomodeset fb=false priority=critical locale=en_US url=$PRESEED_URL"
# the kernel command line params are:
# apt-setup/proposed=true: required per LP: #1172101
# nomodeset fb=false: for -curses friendliness
#
## took defaults for everything other than
## * http_proxy (put in local proxy: http://192.168.1.130:3128/ )
## * user: ubuntu passwd: ubuntu
## Then boot the system, removing the '-kernel and -initrd'
## verify that I'm running a 3.13 kernel
$ uname -r
3.13.0-30-generic
## just for my random information, you can then make it boot
## sanely in curses mode only by:
## echo "blacklist vga16fb" | sudo tee /etc/modprobe.d/novga16fb.conf
## sudo sed -i -e 's,\(GRUB_CMDLINE_LINUX_DEFAULT\)=.*,\1="quiet nomodeset",' \
## -e 's,#GRUB_TERMINAL=console,GRUB_TERMINAL=console,' \
## /etc/default/grub
## sudo update-grub
## sudo update-initramfs -u
# based on http://bit.ly/uquick-doc
#d-i mirror/country string manual
#d-i mirror/http/hostname string mymirror.org
#d-i mirror/http/directory string /rep
#d-i mirror/http/proxy string http://myprox:port/
#d-i mirror/http/mirror select mymirror.org
d-i debian-installer/locale string en_US.UTF-8
d-i debian-installer/splash boolean false
d-i console-setup/ask_detect boolean false
d-i console-setup/layoutcode string us
d-i console-setup/variantcode string
d-i netcfg/get_nameservers string
d-i netcfg/get_ipaddress string
d-i netcfg/get_netmask string 255.255.255.0
d-i netcfg/get_gateway string
d-i netcfg/confirm_static boolean true
d-i clock-setup/utc boolean true
d-i partman-auto/method string regular
d-i partman-lvm/device_remove_lvm boolean true
d-i partman-lvm/confirm boolean true
d-i partman/confirm_write_new_label boolean true
d-i partman/choose_partition select Finish partitioning and write changes to disk
d-i partman/confirm boolean true
d-i partman/confirm_nooverwrite boolean true
d-i partman/default_filesystem string ext3
d-i clock-setup/utc boolean true
d-i clock-setup/ntp boolean true
d-i clock-setup/ntp-server string ntp.ubuntu.com
d-i passwd/root-login boolean false
d-i passwd/make-user boolean true
d-i passwd/user-fullname string ubuntu
d-i passwd/username string ubuntu
d-i passwd/user-password-crypted password $6$.1eHH0iY$ArGzKX2YeQ3G6U.mlOO3A.NaL22Ewgz8Fi4qqz.Ns7EMKjEJRIW2Pm/TikDptZpuu7I92frytmk5YeL.9fRY4.
d-i passwd/user-uid string
d-i user-setup/allow-password-weak boolean false
d-i user-setup/encrypt-home boolean false
d-i passwd/user-default-groups string adm cdrom dialout lpadmin plugdev sambashare
d-i apt-setup/services-select multiselect security
d-i apt-setup/security_host string security.ubuntu.com
d-i apt-setup/security_path string /ubuntu
d-i debian-installer/allow_unauthenticated string false
d-i pkgsel/upgrade select safe-upgrade
d-i pkgsel/language-packs multiselect
d-i pkgsel/update-policy select none
d-i pkgsel/updatedb boolean true
d-i grub-installer/skip boolean false
d-i lilo-installer/skip boolean false
d-i grub-installer/only_debian boolean true
d-i grub-installer/with_other_os boolean true
d-i finish-install/keep-consoles boolean false
d-i finish-install/reboot_in_progress note
d-i cdrom-detect/eject boolean true
d-i debian-installer/exit/halt boolean false
d-i debian-installer/exit/poweroff boolean false
d-i pkgsel/include string vim openssh-server
#d-i preseed/late_command string my-late-command
#!/bin/sh
Usage() {
cat <<EOF
Usage: ${0##*/} file [name]
write a d-i safe command to run 'file' in a late command.
useful for complex commands and not dealing with shell quoting.
EOF
}
file="$1"
name="${2:-/root/latecommand}"
[ $# -eq 0 ] && { Usage 1>&2; exit 1; }
[ "$1" = "-h" -o "$1" = "--help" ] && { Usage; exit 0; }
if [ $# -eq 1 ]; then
shift
else
shift 2
fi
error() { echo "$@" 1>&2; }
dump_tmpl() {
cat <<"EOF"
in-target sh -c '
set -ef;
f=$1;
shift;
[ ${f#/} = $f ] && f=./$f;
echo $0 | base64 --decode > $f;
chmod u+x $f;
$f $* > $f.output 2>&1;
'
EOF
}
get_tmpl() {
dump_tmpl | sed ':a; N; $!ba; s/\n[ ]*/ /g'
}
writecmd() {
local file="$1" name="$2" shellcmd="" b64=""
shift 2
[ -f "$file" ] || { error "$file: not a file"; return 1; }
shellcmd=$(get_tmpl) || { error "getting template failed"; return 1; }
b64=$(base64 --wrap=0 "$file") ||
{ error "base64 encode of $fail failed"; return 1; }
printf "d-i\t preseed/late_command string %s %s %s %s\n" "$shellcmd" "$b64" "$name" "$*"
}
writecmd "$file" "$name" "$@"
#!/bin/bash
VERBOSITY=0
TEMP_D=""
MY_PATH=$(readlink -f "$0")
MY_D=$(cd "${MY_PATH%/*}" && pwd)
DEF_PRESEED="$MY_D/preseed"
DEF_SIZE=4
DEF_MIRROR="http://archive.ubuntu.com/ubuntu"
DEF_ARCH=$(uname -m)
[ "$DEF_ARCH" = "x86_64" ] && DEF_ARCH=amd64
error() { echo "$@" 1>&2; }
errorp() { printf "$@" 1>&2; }
fail() { [ $# -eq 0 ] || error "$@"; exit 1; }
failp() { [ $# -eq 0 ] || errorp "$@"; exit 1; }
Usage() {
cat <<EOF
Usage: ${0##*/} [ options ] release [ arch [ size ] ]
Do an install of Ubuntu for release.
arch : the arch to use (amd64 i386). Default: ${DEF_ARCH}
size : size of the image (in GigaBytes). Default: ${DEF_SIZE}
options:
-o | --output IMAGE_FILE write the image to IMAGE_FILE
default: <release>-<arch>.img
-s | --preseed PRESEED use the preseed at preseed
default: ${DEF_PRESEED}
-m | --mirror MIRROR mirror to download iso from MIRROR
default: ${DEF_MIRROR}
--iso ISO use ISO file rather than downloading
--late-command-file F execute file F in target as late command
EOF
}
bad_Usage() { Usage 1>&2; [ $# -eq 0 ] || error "$@"; exit 1; }
cleanup() {
[ -z "${TEMP_D}" -o ! -d "${TEMP_D}" ] || rm -Rf "${TEMP_D}"
}
dl() {
local url="$1" out="${2}" opts=""
[ "$url" = "$out" ] && return
[ $VERBOSITY -lt 1 ] && opts="-q"
local tfile="" tdir="" ret=""
tdir=$(dirname "$out")
[ -d "$tdir" ] || mkdir -p "$tdir" || return 1
tdir=$(cd "$tdir" &&pwd)
tfile=$(mktemp $tdir/${out##*/}.XXXXXX)
case "$url" in
http://*|ftp://*) wget $opts "$url" -O "$tfile";;
*) cat "${url}" > "$tfile";;
esac
ret=$?
[ $ret -eq 0 ] && mv "$tfile" "$out" && return 0
rm -f "$tfile"
return $ret
}
debug() {
local level=${1}; shift;
[ "${level}" -ge "${VERBOSITY}" ] && return
error "${@}"
}
extract_file_from_iso() {
# extract_file_from_iso(iso, file, out)
# extract file from iso and store in output.
# does not work for zero length files
local iso="$1" file="$2" out="$3" ret=$?
local tmp="$out.tmp.$$"
isoinfo -RJ -x "$file" -i "$iso" > "$tmp" && [ -s "$tmp" ]
ret=$?
[ $ret -eq 0 ] && mv "$tmp" "$out" && return 0
rm -f "$tmp"
return $ret
}
search_iso_for_file() {
# search_iso_for_file(iso, output, files)
# search iso for each file in files. store the first found in output
local iso="$1" out="$2" i=""
shift 2
for i in "$@"; do
extract_file_from_iso "$iso" "$i" "$out" && return 0
done
error "did not find $out on $iso. searched $*"
return 1
}
short_opts="hm:o:p:v"
long_opts="help,iso:,late-command-file:,mirror:,output:,preseed:,verbose"
getopt_out=$(getopt --name "${0##*/}" \
--options "${short_opts}" --long "${long_opts}" -- "$@") &&
eval set -- "${getopt_out}" ||
bad_Usage
release="${DEF_RELEASE}"
preseed="${DEF_PRESEED}"
output=""
mirror="${DEF_MIRROR}"
arch="${DEF_ARCH}"
iso=""
late_command_file=""
while [ $# -ne 0 ]; do
cur=${1}; next=${2};
case "$cur" in
-h|--help) Usage ; exit 0;;
-i|--iso) iso=${2}; shift;;
-m|--mirror) mirror=${2}; shift;;
-o|--output) output=${2}; shift;;
-p|--preseed) preseed=${2}; shift;;
-v|--verbose) VERBOSITY=$((${VERBOSITY}+1));;
--late-command-file) late_command_file="$2";;
--) shift; break;;
esac
shift;
done
[ $# -ne 0 ] || bad_Usage "must provide arguments"
[ $# -lt 1 -o $# -gt 3 ] && bad_Usage "must provide 1,2, or 3 args"
release=$1
arch=${2:-${DEF_ARCH}}
size=${3:-${DEF_SIZE}}
size=${size%G}
[ -n "$output" ] || output="${release}-${arch}.img"
TEMP_D=$(mktemp -d "${TMPDIR:-/tmp}/${0##*/}.XXXXXX") ||
fail "failed to make tempdir"
trap cleanup EXIT
if [ -z "$iso" ]; then
iso="${release}-${arch}-mini.iso"
if [ -f "$iso" ]; then
debug 1 "using existing iso ${iso}"
else
found=false
for pocket in "$release-updates" "$release"; do
url="${mirror}/dists/$pocket/main/installer-$arch/current/images/netboot/mini.iso"
debug 1 "downloading ${url}"
dl "${url}" "${iso}" && found=true && break
done
$found || fail "failed to download $iso from $url"
fi
fi
dl "$preseed" preseed.cfg ||
fail "failed to download preseed: ${preseed}"
late_command=""
if [ -n "${late_command_file}" ]; then
[ -f "$late_command_file" ] || fail "$late_command_file: not a file"
sed -i '/^d-i.*preseed.late_command/d' "$preseed.cfg" ||
fail "failed to remove late command from $preseed.cfg"
out=$("${MY_D}/script-to-latecommand" "$late_command_file") ||
fail "failed script-to-latecommand $late_command_file"
printf "d-i\tpreseed/late_command\tstring\t%s" "$out" >> "$preseed.cfg"
fi
debug 1 "extracting kernel and ramdisk"
kernel_paths="/linux /install/vmlinuz"
initrd_paths="/initrd.gz /install/initrd.gz"
search_iso_for_file "$iso" "linux.dist" $kernel_paths || fail
search_iso_for_file "$iso" "initrd.gz.dist" $initrd_paths || fail
debug 1 "repacking initramfs"
zcat initrd.gz.dist > initrd &&
echo "./preseed.cfg" | cpio -o --format=newc --append -F initrd &&
gzip -9 initrd -c > initrd.gz && rm -f initrd ||
fail "failed to repack initrd"
cp "linux.dist" "linux"
debug 1 "creating a ${size}G disk in ${output}"
qemu-img create -f qcow2 "${output}" "${size}G" ||
fail "failed to create image"
kparms="nomodeset fb=false"
set -x
${KVM:-kvm} -kernel linux -initrd initrd.gz \
-append "priority=critical locale=en_US --- $kparms" \
-drive "file=${output},if=virtio,cache=unsafe" \
-cdrom "${iso}" -m 512 -boot d -no-reboot \
-vga std -curses
# vi: ts=4 noexpandtab
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment