Skip to content

Instantly share code, notes, and snippets.

@TyrfingMjolnir

TyrfingMjolnir/README.md

Last active September 22, 2023 18:57
Show Gist options
  • Save TyrfingMjolnir/8d6da3e5ef5d3167fefa673039552f83 to your computer and use it in GitHub Desktop.
Save TyrfingMjolnir/8d6da3e5ef5d3167fefa673039552f83 to your computer and use it in GitHub Desktop.
Download ZIP
Some example bhyve zone files for Joyent SmartOS
Raw
README.md

A minor list of how to create and boot a Bhyve-zone in Joyent SmartOS

cd /opt/local/artifacts/
vim zonemanifestBhyveFreeBSD.json
vmadm create < zonemanifestBhyveFreeBSD.json
curl -kLo /zones/UUID/root/FreeBSD-12.0-RC3-amd64-bootonly.iso

The following line is for KVM, not BHYVE, hence obsoleted for non-legacy boot options

cd /zones/<<UUID>>/root/
vmadm boot <<UUID>> order=cd,once=d cdrom=/FreeBSD-12.0-RC3-amd64-bootonly.iso,ide

BHYVE prefers something like this

vmadm update <<UUID>> bhyve_extra_opts="-w -s 3:0,ahci-cd,/FreeBSD-12.0-RC3-amd64-bootonly.iso"

Or for Ubuntu

vmadm update <<UUID>> bhyve_extra_opts="-w -s 0:3:0,ahci-cd,/mini.iso"

Common

vmadm start  <<UUID>>

This is when you login using VNC and do the actual install

When rebooting after install or when Ubuntu asks for you to remove USB or CDVD-ROM

vmadm update <<UUID>> bhyve_extra_opts="-w"

From [email protected] @ 2023-04-21

export UUID=your_uuid
export ISO="NetBSD-9.3-amd64.iso"
cp $ISO /zones/$UUID/root/
vmadm update $UUID bhyve_extra_opts="-s 7:0,ahci-cd,/$ISO"
vmadm start $UUID
vmadm update $UUID bhyve_extra_opts=""

One of the examples is from here: https://pastebin.com/tN1f5VCd

Below are my starting points for creating these VMs.

NB! There is a calculation of the number of vcpus from the parameters of bhyve_extra_opts in which should match up. Thank you, Gyula Bibernáth, for bringing this up on the SmartOS mailing list.

Raw
sn00bhyveCatalina01.json
{
"brand": "bhyve",
"bhyve_extra_opts": "-c sockets=1,cores=2,threads=2",
"vcpus": 4,
"autoboot": false,
"alias": "sn06catalina01",
"hostname": "sn06catalina01",
"bootrom": "uefi",
"ram": 40960,
"vnc_port": 15910,
"resolvers": [
"172.12.0.1",
"208.67.222.222",
"208.67.220.220"
],
"disks": [
{
"boot": false,
"model": "virtio",
"size": 40960
},
{
"boot": true,
"model": "ahci",
"media": "cdrom",
"path": "/Catalina.iso"
}
],
"nics": [
{
"nic_tag": "admin",
"model": "virtio",
"ip": "172.12.0.212",
"netmask": "255.255.255.0",
"gateway": "172.12.0.1",
"primary": "1"
}
]
}
Raw
sn00BhyveCentOS7.json
{
"alias": "centos7",
"brand": "bhyve",
"hostname": "test00",
"resolvers": [
"192.168.1.1",
"208.67.222.222",
"208.67.220.220"
],
"ram": 6024,
"vcpus": 2,
"nics": [
{
"model": "virtio",
"nic_tag": "admin",
"ip": "192.168.1.5",
"netmask": "255.255.255.0",
"gateway": "192.168.1.1",
"primary": true
}
],
"disks": [
{
"image_uuid": "462d1d03-8457-e134-a408-cf9ea2b9be96",
"boot": true,
"model": "virtio"
}
]
}
Raw
sn00BhyveFreeBSD.json
{
"brand": "bhyve",
"bhyve_extra_opts": "-c sockets=1,cores=2,threads=2",
"vcpus": 4,
"autoboot": false,
"alias": "FreeBSD12a",
"hostname": "fbsd12kvm",
"bootrom": "uefi",
"ram": 4096,
"vnc_port": 15910,
"resolvers": [
"172.12.0.1",
"208.67.222.222",
"208.67.220.220"
],
"disks": [
{
"boot": true,
"model": "virtio",
"size": 10960
},
{
"boot": true,
"model": "ahci",
"media": "cdrom",
"path": "/FreeBSD-12.0-RC3-amd64-bootonly.iso"
}
],
"nics": [
{
"nic_tag": "admin",
"model": "virtio",
"ip": "172.12.0.212",
"netmask": "255.255.255.0",
"gateway": "172.12.0.1",
"primary": "1"
}
]
}
Raw
sn00KVMw2k16.json
{
"brand": "kvm",
"alias": "w2k16",
"cpu_type": "Nehalem",
"vcpus": 2,
"autoboot": false,
"ram": 16384,
"resolvers": ["10.0.0.10","208.67.222.222","208.67.220.220"],
"disks": [
{
"boot": true,
"model": "ide",
"size": 40960
}
],
"nics": [
{
"nic_tag": "admin",
"model":"e1000",
"ip": "10.0.0.4",
"netmask": "255.255.255.0",
"gateway": "10.0.0.1",
"primary": 1
}
]
}
Raw
windows-10-pro-smartos-bhyve.md

Windows 10 Pro

Source: https://gist.github.com/skirmess/9b1aebde9c72d3fb23a1efd521107cc3

Windows 10 Pro ISO

Create a fresh Windows 10 Pro ISO file with Microsofts Media Creation Tool. (I created a Windows 10 Pro 64 bit German ISO image)

Save the created image on your SmartOS global zone.

cp ... /zones/Windows10_64de.iso

Virtio driver

Download latest stable Virtio driver ISO from https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/stable-virtio/ and save it in the global zone.

# cd /zones
# wget --no-check-certificate https://fedorapeople.org/groups/virt/virtio-win/direct-downloads/archive-virtio/virtio-win-0.1.185-2/virtio-win-0.1.185.iso

Create a new filesystem to install Windows

All virtual machines created from this image will have this size. Choose a size that fits your needs.

# zfs create -V 80G zones/windows

Install Windows 10 Pro

# export WINDOWS_INSTALL_CD=/zones/Windows10_64de.iso
# export VIRTIO_DRIVER_CD=/zones/virtio-win-0.1.185.iso

# pfexec /usr/sbin/bhyve -c 1 -m 3G -H \
    -l com1,stdio \
    -l bootrom,/usr/share/bhyve/uefi-rom.bin \
    -s 2,ahci-cd,$WINDOWS_INSTALL_CD \
    -s 3,virtio-blk,/dev/zvol/rdsk/zones/windows \
    -s 4,ahci-cd,$VIRTIO_DRIVER_CD \
    -s 28,fbuf,vga=off,tcp=0.0.0.0:5900,w=1024,h=768,wait \
    -s 29,xhci,tablet \
    -s 31,lpc \
    windows

Bhyve will wait until you connect with VNC to port 5900 of your global zone. After you connect you have a few seconds to press any key to boot from CD.

Select the install now option and answer that you don't have a product key (license). Pick the Windows version you want to create the image for (Windows 10 Pro).

When asked where to install it to, load the Virtio SCSI driver from this location E:\amd64\w10\viostor.inf.

Whenever Windows wants to reboot the bhyve command terminates. Run the same ommand again but don't press any key. These times we want to boot from the hard disk.

Repeat this until Windows starts to ask you questions. Press Ctrl + Shift + F3 to reboot to audit mode (run the same bhyve command again). Audit mode logs you automatically into the Administrator account. (This takes forever, twice, to log you in. Just be patient.) Keep the window from sysprep open, do not click OK as this will reboot out of audit mode.

Run E:\virtio-win-gt-x64.msi and install all the Virtio drivers - or at least the networking one.

Click OK in the window from the sysprep tool and wait for the system to automatically shut down.

Create the image

# zfs send zones/windows | tee /zones/windows.zvol | digest -a sha1
5989a20371dbb29ab7e04d32f124eccfe5749a4b
# zfs destroy zones/windows
# /usr/sbin/bhyvectl --destroy --vm=windows
# ls -l /zones/windows.zvol
-rw-r--r--   1 root     root     10891713416 Jan 16 14:25 /zones/windows.zvol

Generate a random GUID with uuidgen(1) and create the windows.imgmanifest file.

{
    "v": 2,
    "uuid": "47e05f5c-a9c3-4779-b01a-a94d16f3a404",
    "name": "win-10-pro-de",
    "version": "0.0.1",
    "type": "zvol",
    "os": "windows",
    "files": [ {
        "sha1": "5989a20371dbb29ab7e04d32f124eccfe5749a4b",
        "size": 10891713416,
        "compression": "none"
    } ]
}

Import the image.

# imgadm install -m /zones/windows.imgmanifest -f /zones/windows.zvol

We're done, you can now create virtual machines from this image.

Create a Windows 10 Pro VM

The vcpus option from bhyve creates that number of CPUs, not cores. Unfortunately, Windows 10 Pro doesn't support more than 2 CPUs. That's why we only define a single vcpus but add multiple cores and threads with the bhyve_extra_opts option.

The nics options should be adjusted to match your needs. The example works with an external DHCP server to automatically configure networking.

{
  "alias": "wintest",
  "brand": "bhyve",
  "vcpus": 8,
  "bhyve_extra_opts": "-c sockets=1,cores=4,threads=2",
  "autoboot": false,
  "ram": 8192,
  "bootrom": "uefi",
  "disks": [ {
    "boot": true,
    "model": "virtio",
    "image_uuid": "47e05f5c-a9c3-4779-b01a-a94d16f3a404",
    "image_size": 15360
  } ],
  "nics": [
    {
      "nic_tag": "admin",
      "vlan_id": 150,
      "ip": "dhcp",
      "primary": "true",
      "model": "virtio"
    }
  ]
}

Now we create and start the VM.

# vmadm create -f wintest.json
Successfully created VM 3f23f273-f4dc-4cab-bd8d-8aeac66fe79e
# vmadm start 3f23f273-f4dc-4cab-bd8d-8aeac66fe79e
# vmadm info 3f23f273-f4dc-4cab-bd8d-8aeac66fe79e vnc

Connect with VNC and answer the questions.

Tip: When you create an offline account, don't set a password. You can set the password after you log in the first time. That way you don't have to answer the 3 idiotic recovery questions.

After you've set the password, enable the remote desktop and have fun.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment