Proxmox VE 3-Node Cluster
with GPU & HBA Passthrough

Proxmox VE is a Debian-based Type 1 hypervisor with a full web UI. It supports KVM virtual machines and LXC containers in the same management interface. The cluster feature lets multiple nodes share HA (High Availability) policies and shared storage pools.

Our setup: 3 physical nodes, each running Proxmox VE 8, connected to an NFS share on the ODIN storage server and a Ceph distributed storage cluster spread across all 3 nodes.

The Proxmox ISO written with most tools uses MBR partitioning and may not boot on UEFI-only systems. The fix: use Rufus in DD mode, which writes the raw disk image byte-for-byte preserving the original GPT partition table.

1. 1

   Download Proxmox ISO

   Get the latest from proxmox.com/downloads — Proxmox VE ISO Installer.

2. 2

   Write with Rufus (DD Mode)

   Open Rufus, select your USB drive and the Proxmox ISO. When Rufus prompts "Write in ISO Image mode or DD Image mode?" — choose DD Image mode. This preserves the original GPT/UEFI boot structure.

   If you use ISO mode (default), the installer may boot but fail to detect UEFI targets correctly. Always use DD mode for Proxmox.
3. 3

   BIOS/UEFI Settings

   Before booting: Enable VT-d (Intel) or AMD-Vi (AMD) in BIOS for PCIe passthrough. Disable Secure Boot. Set boot order to USB first.

4. 4

   Install Proxmox VE

   Follow the graphical installer. Set a static IP on your management interface — you'll need this to be stable for cluster formation. Use a dedicated management VLAN if possible.

Cluster formation requires all nodes to be able to reach each other on port 2224 (Corosync). Do this on all 3 nodes before adding VMs.

1. 1

   Create Cluster on Node 1

   In the Proxmox web UI on your first node: Datacenter → Cluster → Create Cluster. Give it a name (e.g., homelab). Note the join token shown after creation.

2. 2

   Join Remaining Nodes

   On Node 2 and Node 3: Datacenter → Cluster → Join Cluster. Paste the join token from Node 1. The joining nodes will sync cluster configuration automatically.

3. 3

   Remove Enterprise Subscription Warning

   On each node, via SSH:

   # Disable enterprise repo sed -i 's/^deb/# deb/' /etc/apt/sources.list.d/pve-enterprise.list # Add no-subscription repo echo "deb http://download.proxmox.com/debian/pve bookworm pve-no-subscription" \ >> /etc/apt/sources.list apt update && apt dist-upgrade -y

Shared NFS storage lets all nodes access the same VM disk images and ISO files — required for live migration without shared local storage.

1. 1

   Configure NFS on the Storage Server

   On ODIN (FreeBSD NFS server), add the Proxmox nodes to /etc/exports:

   /zdata/proxmox 10.10.50.0/24(rw,sync,no_subtree_check,no_root_squash)

   exportfs -ra systemctl restart nfs-server
2. 2

   Add to Proxmox (Cluster-Wide)

   Datacenter → Storage → Add → NFS. Set the NFS server IP, export path, and select content types (Disk image, ISO image, Container template). Adding it at the Datacenter level makes it available on all nodes simultaneously.

A common issue: SAS drives aren't showing up in a Proxmox VM even after adding the HBA via PCIe passthrough. The root cause is that the HBA needs to be the only controller managing those drives — if the host OS sees the drives first, the card initializes differently.

1. 1

   Identify the HBA PCI Address

   lspci | grep -i sas # Example output: 03:00.0 Serial Attached SCSI controller: LSI Logic SAS9211-8i
2. 2

   Blacklist the HBA Driver on Host

   Prevent Proxmox (the host) from claiming the HBA so it's available for passthrough:

   # /etc/modprobe.d/blacklist.conf blacklist mpt3sas blacklist mpt2sas update-initramfs -u reboot
3. 3

   Enable VFIO for the HBA

   # /etc/modprobe.d/vfio.conf options vfio-pci ids=1000:0072 # replace with your HBA vendor:device IDs # Get IDs with: lspci -n -s 03:00.0
4. 4

   Add to VM as PCIe Device

   In the VM hardware tab, click Add → PCI Device. Select the HBA. Enable "All Functions" and "Primary GPU" if applicable. Boot the VM — the guest OS will see the HBA natively and drive discovery will work as expected.

Pass an NVIDIA GPU through to a VM running Emby so it can use hardware transcoding. The same approach works for any GPU workload (gaming, ML inference).

1. 1

   Enable IOMMU in GRUB

   Edit /etc/default/grub:

   # Intel CPU GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt" # AMD CPU GRUB_CMDLINE_LINUX_DEFAULT="quiet amd_iommu=on iommu=pt"

   update-grub && reboot
2. 2

   Add VFIO Modules

   # /etc/modules vfio vfio_iommu_type1 vfio_pci vfio_virqfd update-initramfs -u -k all
3. 3

   Blacklist NVIDIA on Host

   # /etc/modprobe.d/blacklist-nvidia.conf blacklist nouveau blacklist nvidia blacklist nvidiafb
4. 4

   Add GPU to VM

   VM → Hardware → Add → PCI Device → select NVIDIA GPU. Set the VM to use OVMF (UEFI) and add a ROM file if driver refuses to load (Error 43 in Windows). Linux VMs typically don't have the Error 43 issue.

Create a base Debian VM, convert it to a template, then clone from it — far faster than installing from ISO each time.

In Proxmox UI: right-click the VM → Convert to Template. Then: right-click template → Clone → set type to "Full Clone" for independent copies.

Proxmox VE 3-node cluster dashboard — Datacenter view showing all nodes, VMs, and storage pools