Hardware Setup / Proxmox Base Install

proxmox-base.md

Hardware Setup / Proxmox Base Install

this gist is part of this series

Hardware Setup (perform on all 3 nodes)

• Connect all 3 NUC to power etc
• Disable secure boot in BIOS on all nodes
• Connect all 3 TB4 cables in a rinng topology:

using the numbers printed on the case of the intel13 nucs connect cables as follows (this is important):
- node 1 port 1 > node 2 port 2
- node 2 port 1 > node 3 port 2
- node 3 port 1 > node 1 port 2

• Prepare bootable usb key by downloading proxmox iso and tools such as 'rufus'
• Boot from USB

Proxmox Base Install (repeat on each node)

• choose GUI (why not!)
• accept terms
• choose the SSD NOT the NVME for the install
• set country, keyboard, etc
• choose a password (this will be for the user - root) and set email
• choose enp86s0 as the management interface (on proxmox 8 you will see the thundebolt netwrok too if cables are connected)
• set a name for each node - i used pveX.mydomain.com where X = the node number i wanted
• set fixed IPv4 address for the management network
  • 192.168.1.81/24 for node 1
  • 192.168.1.82/24 for node 2
  • 192.168.1.83/24 for node 3
• set external DNS server and gateway
• let the software partition disk as it sees fit
• review summary page
• removed the USB key and reboot when prompted

You should now have 3 nodes. Each should be accessible over the network with the IP addresses you specified. e.g 192.168.1.81:8006, 192.168.1.82:8006, 192.168.1.83:8006

The default username is root and you use the password you defined earlier. Consider making constrained users and doing security properly. I won't be doing that, i will be using root as this a PoC.

Set Package Repositories (only needed if you don't use a subscription)

Note: i will not be using enterprise packages as this is a PoC for evaluation / homeLab purposes only

In node-name > updates > repositories perform the following:

• add No-Subscription
• add Ceph Quincy No-Subscription
• disable pve-enterprise
• disable ceph-quincy enterprise too

Open the shell and perform apt update && apt upgrade to update all components and then reboot.

after installation my nvme ssd is not showing in boot devices as uefi is enabled , what to do ? i am unable to boot

Curious with Proxmox 8.1 supporting Secure boot have you tested this with Secure boot at all now?

Hi, why do you recommend: „choose the SSD NOT the NVME for the install“?
I have three NUCs as well and had Talos OS installed on the NVME to form a cluster and kept the 4TB SSD disk for cluster storage. Now I would like to use your setup.

Hi, why do you recommend: „choose the SSD NOT the NVME for the install“? I have three NUCs as well and had Talos OS installed on the NVME to form a cluster and kept the 4TB SSD disk for cluster storage. Now I would like to use your setup.

I think this is likely because proxmox as a base, is not heavy on the I/O and disk performance, another popular alternative Unraid, actually runs off a USB key. Thus, you have more "options" if you save your full NVME drive for after install, then it's easier to have that drive up for Ceph, or as partition it and wipe it as needed, and your actual containers and VMs can go on the NVME.

what are the disk recommendations for example in a situation where I have two 1TB NVME's per node... just one for Proxmox and the other for ceph ?
at the moment I've created a 1TB mirror and assigned 130gb to Proxmox and the remainder to Ceph.