Installing pfSense on KVM in CentOS 8

pfsense-on-centos-8.md

Installing pfSense on KVM in CentOS 8

We download the latest release of pfSense as a gzipped ISO, then extract it and pass it to virt-install to get the VM up and running. Interactive portions of setup are done with virt-install's native console redirection. This guide assumes you've already installed and configured KVM, and that you've created network bridges already.

Instructions

Find the latest release here.

Download the gzipped memstick-serial image from a mirror into a directory that KVM can access, then unzip it:

mkdir -p /opt/vm/pfSense
cd /opt/vm
curl -fSOJL https://nyifiles.pfsense.org/mirror/downloads/pfSense-CE-memstick-serial-2.4.5-RELEASE-p1-amd64.img.gz
gunzip pfSense-CE-memstick-serial-2.4.5-RELEASE-p1-amd64.img.gz

Now call virt-install to begin your installation.

• Any --network lines will show up in this order as your virtual NICs.

virt-install \
--name pfSense \
--memory 1024 \
--autostart \
--os-type=freebsd \
--os-variant=freebsd11.0 \
--disk /opt/vm/pfSense-CE-memstick-serial-2.4.5-RELEASE-p1-amd64.img \
--boot hd,menu=on,useserial=on \
--disk /opt/vm/pfSense/disk0.qcow2,size=10,format=qcow2 \
--network bridge=brteam0,model=virtio \
--graphics none \
--console target_type=serial \
--debug

Install

Install as normal, I suggest using Guided ZFS on Root.

NOTE: disk0 is the boot media here. Install to disk 1.

When installation is finished, choose Yes when it asks you to drop to a shell, then give it the poweroff command:

# poweroff
Shutdown NOW!
poweroff: [pid 1295]
# Oct 17 16:20:47 shutdown: power-down by root: 

System shutdown time has arrived
Waiting (max 60 seconds) for system process `vnlru' to stop... done
Waiting (max 60 seconds) for system process `bufdaemon' to stop... done
Waiting (max 60 seconds) for system process `syncer' to stop... 
Syncing disks, vnodes remaining... 0 0 0 done
All buffers synced.
Uptime: 1m57s
acpi0: Powering system off

[Sat, 17 Oct 2020 12:20:57 virt-install 24937] DEBUG (virt-install:709) Domain state after install: 5
[Sat, 17 Oct 2020 12:20:57 virt-install 24937] DEBUG (cli:272) Domain creation completed.
Domain creation completed.

Cleanup

Add the EPEL repo if you don't have it already:

sudo yum install -y epel-release

Install xmlstarlet:

sudo yum install -y xmlstarlet

Remove the installation media from the VM config:

virsh dumpxml pfSense | xmlstarlet ed -d "domain/devices/disk[1]" > /tmp/pfSense.xml
virsh define /tmp/pfSense.xml 
rm -f /tmp/pfSense.xml

Restart pfSense and connect to it:

virsh start pfSense && virsh console pfSense

Assign interfaces and disconnect ^] when finished.

@Casper042 funnily enough, I did this for my VM to troubleshoot some performance issues with my 5 gbps service. Not too sure it made a difference. Regardless:

• use lspci to find the BDF id of your NIC.
  • You'll get an ID like 00:ab.0. For PCI there are leading 0000 corresponding to the local PCI root, so that device's complete ID would be 0000:00:ab.0.
• run virsh nodedev-list to find the corresponding libvirt node device name.
  • The example device would be like pci_0000_00_ab_0
• Replace --network ... with --host-device <id> in the virt-install command
  • The example device might look like --host-device pci_0000_00_ab_0

[Tue, 09 Jan 2024 01:38:46 virt-install 16154] ERROR (cli:256) unsupported configuration: host doesn't support passthrough of host PCI devices
[Tue, 09 Jan 2024 01:38:46 virt-install 16154] DEBUG (cli:258)
Traceback (most recent call last):
File "/usr/share/virt-manager/virtinst/virtinstall.py", line 941, in start_install
domain = installer.start_install(
File "/usr/share/virt-manager/virtinst/install/installer.py", line 695, in start_install
domain = self._create_guest(
File "/usr/share/virt-manager/virtinst/install/installer.py", line 637, in _create_guest
domain = self.conn.createXML(initial_xml or final_xml, 0)
File "/usr/lib/python3/dist-packages/libvirt.py", line 4400, in createXML
raise libvirtError('virDomainCreateXML() failed')
libvirt.libvirtError: unsupported configuration: host doesn't support passthrough of host PCI devices

I've double checked VT and VT-d are both indeed enabled in the BIOS.
And my CPU says it supports VT-d on Intel ARK as well.
I read something about possibly needing to first detach the device from the host?
Ring any bells?

@Casper042 pci passthrough has to be enabled in the kernel. Usually involves editing the kernel command line in grub: https://wiki.gentoo.org/wiki/GPU_passthrough_with_libvirt_qemu_kvm#GRUB_bootloader

That's not for centos but the process is similar. I think you'd need to edit the grub config and then rebuild the files with:

vim /etc/sysconfig/grub
grub2-mkconfig -o /boot/grub2/grub.cfg