Create FreeBSD virtual machine using qemu. Run the VM using xhyve.

freebsd-qemu-xhyve-mac-os-x-virtual-machine.md

TL;DR

• Create 10GB FreeBSD image using QEMU.
• Run the VM using xhyve.
• Mount host directory.
• Resize the image.

Requisites

• OSX
• Homebrew
• xhyve

$ brew install xhyve --HEAD

• QEMU

$ brew install qemu

• FreeBSD installer. Download from https://www.freebsd.org/where.html

Create Virtual Machine

• This Example creates 10GB image.

$ cd /Path/to/workdir
$ qemu-img create -f raw VM10G.raw 10G
Formatting 'VM10G.raw', fmt=raw size=10737418240
$ ls
FreeBSD-10.2-RELEASE-amd64-bootonly.iso	VM10G.raw

Install FreeBSD

$ qemu-system-x86_64 -m 256 -hda VM10G.raw -cdrom FreeBSD-10.2-RELEASE-amd64-bootonly.iso
WARNING: Image format was not specified for 'VM10G.raw' and probing guessed raw.
         Automatically detecting the format is dangerous for raw images, write operations on block 0 will be restricted.
         Specify the 'raw' format explicitly to remove the restrictions.

QEMU window will open. Install FreeBSD.

Run The VM Using xhyve

#1

Before using xhyve, run the vm using qemu.

$ qemu-system-x86_64 -m 256 -hda VM10G.raw

Change GEOM name 'ada0' to 'vtbd0'.

Change /etc/fstab:

# Device        Mountpoint      FStype  Options Dump    Pass#
/dev/ada0p2    /               ufs     rw      1       1
/dev/ada0p3    none            swap    sw      0       0

to

# Device        Mountpoint      FStype  Options Dump    Pass#
/dev/vtbd0p2    /               ufs     rw      1       1
/dev/vtbd0p3    none            swap    sw      0       0

Note: We're changing the devices from /dev/ada*** to /dev/vtbd***.

And set DHCP to vtnet0 interface.

Edit /etc/rc.conf

ifconfig_vtnet0="DHCP"

Shut it down and close the qemu window.

#2

Create running script.

xhyverun-freebsd.sh

#!/bin/sh

USERBOOT="/Users/YOUR_USERNAME/Library/Caches/Homebrew/xhyve--git/test/userboot.so"
BOOTVOLUME="VM10G.raw"
KERNELENV=""

MEM="-m 2G"
SMP="-c 2"
IMG_HDD="-s 4:0,virtio-blk,$BOOTVOLUME"
PCI_DEV="-s 0:0,hostbridge -s 31,lpc"
LPC_DEV="-l com1,stdio"
NET="-s 2:0,virtio-net"

# UUID="-U deaddead-dead-dead-dead-deaddeaddead"

xhyve -A $MEM $SMP $PCI_DEV $LPC_DEV $NET $IMG_CD $IMG_HDD $UUID -f fbsd,$USERBOOT,$BOOTVOLUME,"$KERNELENV"

The testboot.so file can also be found in the xhyve repository under the test directory, if you'd like to store it somewhere else.

Run the FreeBSD VM.

$ sudo ./xhyverun-freebsd.sh


______               ____   _____ _____
|  ____|             |  _ \ / ____|  __ \
| |___ _ __ ___  ___ | |_) | (___ | |  | |
|  ___| '__/ _ \/ _ \|  _ < \___ \| |  | |
| |   | | |  __/  __/| |_) |____) | |__| |
| |   | | |    |    ||     |      |      |
|_|   |_|  \___|\___||____/|_____/|_____/    ```                        `
                                           s` `.....---.......--.```   -/
+============Welcome to FreeBSD===========+ +o   .--`         /y:`      +.
|                                         |  yo`:.            :o      `+-
|  1. Boot Multi User [Enter]             |   y/               -/`   -o/
|  2. Boot [S]ingle User                  |  .-                  ::/sy+:.
|  3. [Esc]ape to loader prompt           |  /                     `--  /
|  4. Reboot                              | `:                          :`
|                                         | `:                          :`
|  Options:                               |  /                          /
|  5. [K]ernel: kernel (1 of 2)           |  .-                        -.
|  6. Configure Boot [O]ptions...         |   --                      -.
|                                         |    `:`                  `:`
|                                         |      .--             `--.
|                                         |         .---.....----.
+=========================================+

If you got following mount error, you may be forgeting #1.

Trying to mount root from ufs:/dev/ada0p2 [rw]...
mountroot: waiting for device /dev/ada0p2 ...
Mounting from ufs:/dev/ada0p2 failed with error 19.

Loader variables:
  vfs.root.mountfrom=ufs:/dev/ada0p2
  vfs.root.mountfrom.options=rw

Manual root filesystem specification:
  <fstype>:<device> [options]
      Mount <device> using filesystem <fstype>
      and with the specified (optional) option list.

    eg. ufs:/dev/da0s1a
        zfs:tank
        cd9660:/dev/acd0 ro
          (which is equivalent to: mount -t cd9660 -o ro /dev/acd0 /)

  ?               List valid disk boot devices
  .               Yield 1 second (for background tasks)
  <empty line>    Abort manual input

mountroot>

But you can mount device manually.

mountroot> ?

List of GEOM managed disk devices:
  gptid/5bebeba9-6364-11e5-abe5-bd56998f3bd1 gptid/5be6a257-6364-11e5-abe5-bd56998f3bd1 ufsid/56050e05aac93398 gptid/5be40c9f-6364-11e5-abe5-bd56998f3bd1 diskid/DISK-BHYVE-1C5B-53F3-EA71p3 diskid/DISK-BHYVE-1C5B-53F3-EA71p2 diskid/DISK-BHYVE-1C5B-53F3-EA71p1 vtbd0p3 vtbd0p2 vtbd0p1 diskid/DISK-BHYVE-1C5B-53F3-EA71 vtbd0

mountroot> ufs:ufsid/56050e05aac93398

Mount Host Directory

• Reference machyve/xhyve#45

On Host(192.168.64.1)

$ sudo touch /etc/exports
$ sudo emacs /etc/exports

Edit /etc/exports (exports(5))

/PATH/TO/EXPORTDIR -mapall=501 -network 192.168.64.0 -mask 255.255.255.0

Reread file (nfsd(8))

$ sudo nfsd update

On VM

Mount

# mkdir /usr/home/YOU/host-shared
# sudo mount 192.168.64.1:/PATH/TO/EXPORTDIR /usr/home/YOU/host-shared

Unmount

# mount
/dev/vtbd0p2 on / (ufs, local, journaled soft-updates)
devfs on /dev (devfs, local, multilabel)
192.168.64.1:/PATH/TO/EXPORTDIR on /usr/home/YOU/host-shared (nfs)
# sudo umount 192.168.64.1:/PATH/TO/EXPORTDIR

Resizing and Growing Disks

• Reference https://www.freebsd.org/doc/handbook/disks-growing.html

Resize VM size on Host. This example 10GB -> 15GB.

$ qemu-img resize VM10G.raw 15GB

Fix disk partitions configuration on VM

# gpart show
=>      34  20971453  vtbd0  GPT  (15G) [CORRUPT]
        34      1024      1  freebsd-boot  (512K)
      1058  19919872      2  freebsd-ufs  (9.5G)
  19920930   1048576      3  freebsd-swap  (512M)
  20969506      1981         - free -  (991K)

# gpart recover vtbd0
vtbd0 recovered
# gpart show
=>      34  31457213  vtbd0  GPT  (15G)
        34      1024      1  freebsd-boot  (512K)
      1058  19919872      2  freebsd-ufs  (9.5G)
  19920930   1048576      3  freebsd-swap  (512M)
  20969506  10487741         - free -  (5.0G)

Delete swap partition

# swapoff /dev/vtbd0p3
# gpart delete -i 3 vtbd0
vtbd0p3 deleted
# gpart show
=>      34  31457213  vtbd0  GPT  (15G)
        34      1024      1  freebsd-boot  (512K)
      1058  19919872      2  freebsd-ufs  (9.5G)
  19920930  11536317         - free -  (5.5G)

Resize freebsd-ufs partition

# gpart resize -i 2 -a 4k -s 14G vtbd0
vtbd0p2 resized
root@xhyve-freebsd:~ # gpart show
=>      34  31457213  vtbd0  GPT  (15G)
        34      1024      1  freebsd-boot  (512K)
      1058  29360126      2  freebsd-ufs  (14G)
  29361184   2096063         - free -  (1.0G)

Recreate swap partition

#  gpart add -t freebsd-swap -a 4k -s 512M vtbd0
vtbd0p3 added
# gpart show
=>      34  31457213  vtbd0  GPT  (15G)
        34      1024      1  freebsd-boot  (512K)
      1058  29360126      2  freebsd-ufs  (14G)
  29361184   1048576      3  freebsd-swap  (512M)
  30409760   1047487         - free -  (511M)

# swapon /dev/vtbd0p3

Grow the UFS file system

# growfs /dev/vtbd0p2
Device is mounted read-write; resizing will result in temporary write suspension for /.
It's strongly recommended to make a backup before growing the file system.
OK to grow filesystem on /dev/vtbd0p2, mounted on /, from 9.5GB to 14GB? [Yes/No] Yes
super-block backups (for fsck_ffs -b #) at:
 20516032, 21798272, 23080512, 24362752, 25644992, 26927232, 28209472

Check the resized file system.

# df -h
Filesystem      Size    Used   Avail Capacity  Mounted on
/dev/vtbd0p2     14G    1.9G     11G    16%    /
devfs           1.0K    1.0K      0B   100%    /dev

You might want to consider renaming the raw file since it is no longer 10 GB.

Enjoy!

In step 1, how do I change the GEOM name?

Nice! Going to try this soon.

USERBOOT="/Users/${USER}/Library/Caches/Homebrew/xhyve--git/test/userboot.so"

Using ${USER} means it should work for anyone running it.

Why start with QEMU then switch to xhyve?

people start with qemu to boot off an iso or cd device. its not actually required; if the first (by pci path) device is a blank disk, and the 2nd is a CDROM or mfs device, then xhyve will boot from the 2nd one for install, and next time use the first device.