Install windows with bhyve

README.md

These are bare-bones instructions for creating a Windows image for bhyve on SmartOS. You will need a platform image that has the fix for OS-7117. Platform images that I've used in testing are here: iso tgz usb

Create a volume that will be the disk.

zfs create -o volblocksize=4k -V 16g -s zones/hdd-windows

This script will be used for booting from the CD.

#!/bin/sh

bootrom=/usr/share/bhyve/uefi-rom.bin

[ "$1" = "-k" ] || pfexec bhyvectl --vm=windows --destroy
bhyve -H -w \
        -s 0,hostbridge \
        -B "1,product=SmartOS HVM" \
        -s 31,lpc \
        -l bootrom,$bootrom \
        -l com1,stdio \
        -c 2 \
        -m 2G \
        -s 3:0,ahci-cd,/zones/media/win2016eval.iso \
        -s 4:0,ahci-hd,/dev/zvol/rdsk/zones/hdd-windows \
        -s 28,fbuf,vga=off,tcp=0.0.0.0:5900,w=1024,h=768,wait -s 29,xhci,tablet \
        windows

You will need to connect via vnc and hit the any key to tell it to boot from the CD. After doing the first chunk of work, Windows willl reboot. You will need to re-run the script one or more times to complete the installation. In the subsequent boots, you do not want to boot from the CD, so don't hit the any key. Note that the ,wait on the fbuf argument will cause bhyve to wait for the VNC connection before booting.

Once the installation is complete, we need to associate zones/hdd-windows with a VM. The simplest way to do that is with the following.

First, stop the windows VM. Shutdown from within the VM or kill the bhyve process (-TERM for an orderly shutdown, -KILL or -9 for pull the cord).

Next, create an empty vm and swap disks.

vmadm create -f bhyve-empty.json
uuid=...
zfs destroy zones/$uuid/disk0
zfs rename zones/hdd-windows zones/$uuid/disk0

Now you should be able to start the Windows VM with vmadm start $uuid

Limitiations

Some things are not fully baked. We're working on them.

• Currently you must configure networking statically or have an external DHCP server.
• The procedure above does not include virtio drivers for disks or networking. It is unknown whether any other networking emulation works with bhyve on SmartOS. The virtio drivers we use on kvm and will likely use with bhyve are on this iso. It should be possible to add that iso to the command line during installation and adjust hdd-windows to use virtio-blk instead of ahci.

bhyve-empty.json

{
  "brand": "bhyve",
  "alias": "bhyve-uefi",
  "vcpus": 2,
  "autoboot": false,
  "ram": 2048,
  "disks": [
    {
      "boot": true,
      "model": "ahci",
      "size": 10240
    }
  ]
}

-s 28,fbuf,vga=off,tcp=0.0.0.0:5900,w=1024,h=768,wait -s 29,xhci,tablet \

Does this actually work for you? I can't get vnc after starting the bhyve guest, the port is listening but it times out when I try to access via vncclient, I noticed more folks on smartos mailing list saying the same so I'm curious how you got that to work

The VNC console only works with bootrom=uefi. Even then, the bhyve VNC server implementation is quite limited and does not work well with all clients. I find Chicken of the VNC works well.

If you are trying to have GUI access to a guest for any purpose other than installation, I highly recommend that you use any method other than the bhyve VNC console.

Thanks thats what Im using and as for client just the native macOS one.

What other methods exist for GUI install, didn't see anything else in the smartos docs, but then again they are super vague on bhyve

ah with that client I see the smartos "bios"/"uefi" loader quickly and than move to black screen so maybe its something wrong with the instalation media

btw this is running the bhyve cmd not vmadm yet... I was also using bootrom=/usr/share/bhyve/uefi-csm-rom.bin and switch to bootrom=/usr/share/bhyve/uefi-rom.bin and in this case I get the UEFI shell

Tigher vnc seems to work for me 🤷‍♂️ mac native still fails and the other is black or uefi shell (guess It doesn't connect on type to press the key to go fro cd)

Looks like that running bhyve in global zone requires a clean machine that do not have any KVM instances running. I got vm_open: Invalid argument when I run bhyve in a machine with KVM instance without any hint.

I only resolved this problem after I find another machine without any KVM instances.

is there a script or tip on how to convert a kvm to byhve ?

What does the -B "1,product=SmartOS HVM" do? Can't find it in the bhyve help.

It's been a while (I no longer work on SmartOS), but I believe it sets the BIOS product name, which is visible in a Linux guest with smbios -t 1. I think this was done so that cloud-init will recognize that it should use the SmartOS datasource, but it looks like that is looking for "SmartDC" not "SmartOS".

https://github.com/canonical/cloud-init/blob/0af1ff1eaf593c325b4f53181a572110eb016c50/cloudinit/sources/DataSourceSmartOS.py#L775