Home / Walkthroughs / FreeBSD + ZFS
 Storage / Hypervisors

FreeBSD + ZFS
Mass Storage Server with bhyve

Build ODIN — a 27TB+ FreeBSD storage node with a 30-disk ZFS pool, LAGG bonded network, NFS exports, bhyve VM hosting a Debian SMB gateway, and SLOG/L2ARC cache acceleration.

Advanced 4–8 hours 30+ drives
FreeBSD ZFS bhyve LAGG Bonding NFS SMB SLOG L2ARC

 Table of Contents

  1. Why FreeBSD for Storage?
  2. Hardware Layout
  3. Install FreeBSD
  4. Configure LAGG Network Bonding
  5. Build the ZFS Pool
  6. Configure NFS Exports
  7. bhyve VM — Debian SMB Gateway
  8. SMB/CIFS Shares via Debian VM
  9. ZFS Snapshots & Replication
  10. References
📖
Why FreeBSD for Storage?

FreeBSD has the most mature ZFS implementation outside of OpenZFS on Solaris. ZFS-on-Linux (ZoL) has caught up significantly, but FreeBSD's native ZFS integration, network stack stability, and bhyve hypervisor make it an excellent choice for a mass-storage node that also hosts VMs.

The design philosophy here is: FreeBSD owns the disks and the network. Everything else (SMB, additional services) runs as bhyve VMs that get NFS mounts from the FreeBSD host. This way, a misbehaving VM cannot corrupt your ZFS pool.

This server is called ODIN in the homelab — the central storage node that all other hypervisors mount. Named for the all-seeing, all-knowing Norse god. Appropriately, it sees all the data.
🔧
Hardware Layout
Component Details Role
OS Disks 2× SSDs (mirror) FreeBSD boot volume — mirrored for redundancy
SLOG 2× SSDs (mirror) ZFS Intent Log — write acceleration, crash protection
L2ARC 1× Large SSD Read cache — dramatically speeds up frequent reads
Data Pool 30× HDDs Primary data storage — 27TB+ usable across RAIDZ
Network 2× 1GbE (LAGG) LACP bonding — 2Gbps throughput + failover
💿
Install FreeBSD
  1. 1

    Download FreeBSD 14

    Get the amd64 installer ISO from freebsd.org. Write to USB with dd or Rufus (DD mode).

  2. 2

    Install to Mirrored SSDs

    During install, choose Auto (ZFS) partitioning. Select both OS SSDs and choose mirror. This puts the FreeBSD OS on a redundant ZFS mirror — if one OS disk dies, the system keeps running.

  3. 3

    Enable Required Services

    In /etc/rc.conf, enable the services you'll use:

    hostname="odin" zfs_enable="YES" nfs_server_enable="YES" nfsv4_server_enable="YES" rpcbind_enable="YES" mountd_enable="YES" vm_enable="YES" # bhyve VM framework vm_dir="zfs:data/vms" # store bhyve VMs on ZFS
🔗
Configure LAGG Network Bonding

LAGG (Link Aggregation) bonds two NICs for 2Gbps throughput and automatic failover if one link drops.

# /etc/rc.conf — LAGG configuration cloned_interfaces="lagg0" ifconfig_igb0="up" ifconfig_igb1="up" ifconfig_lagg0="laggproto lacp laggport igb0 laggport igb1 \ inet 10.10.3.234 netmask 255.255.255.0" defaultrouter="10.10.3.1"
Your switch must support LACP (802.3ad) for link aggregation. Configure the corresponding switch ports as a LAG/trunk group. Most managed switches support this — even cheap ones.
💾
Build the ZFS Pool

With 30 data disks you have options. RAIDZ2 (dual parity) gives you 2 drive failure tolerance — recommended for drives of this count.

# Create a RAIDZ2 pool across 30 data disks # Replace da0-da29 with your actual device names zpool create -f data \ raidz2 da0 da1 da2 da3 da4 da5 da6 da7 da8 da9 \ raidz2 da10 da11 da12 da13 da14 da15 da16 da17 da18 da19 \ raidz2 da20 da21 da22 da23 da24 da25 da26 da27 da28 da29 # Add SLOG (ZFS Intent Log) — mirrored SSDs zpool add data log mirror sda sdb # Add L2ARC read cache zpool add data cache sdc

Create your primary datasets:

zfs create data/media zfs create data/appdata zfs create data/vms zfs create data/backups # Set compression — massive space savings, minimal CPU cost zfs set compression=lz4 data zfs set atime=off data # improves performance, disable access time writes
Always enable compression=lz4 on ZFS datasets. LZ4 compression is so fast it actually improves performance on most workloads (fewer bytes to read/write from disk) while reducing space usage 20–50% for typical media and data files.
📡
Configure NFS Exports

Export your ZFS datasets over NFS so other hypervisors and VMs can mount them as shared storage.

# /etc/exports /data/media -alldirs -ro -network 10.10.0.0 -mask 255.255.0.0 /data/appdata -alldirs -network 10.10.50.0 -mask 255.255.255.0 /data/vms -alldirs -network 10.10.50.0 -mask 255.255.255.0
# Reload exports service nfsd restart service mountd reload

Other hosts can now mount:

mount -t nfs 10.10.3.234:/data/media /mnt/media
🖥️
bhyve VM — Debian SMB Gateway

Windows clients need SMB, not NFS. Rather than running Samba directly on FreeBSD, a cleaner approach is to run a Debian VM via bhyve that mounts the NFS shares and re-exports them as SMB. This isolates Samba's complexity from the storage host.

  1. 1

    Install vm-bhyve Framework

    pkg install vm-bhyve bhyve-firmware vm init vm switch create public vm switch add public lagg0 # bridge to your LAGG
  2. 2

    Create and Install Debian VM

    vm iso https://cdimage.debian.org/debian-cd/current/amd64/iso-cd/debian-12.x-amd64-netinst.iso vm create -s 30G -m 2G -c 2 smb-gateway vm install smb-gateway debian-12.x-amd64-netinst.iso
  3. 3

    Mount NFS Shares in the VM

    Inside the Debian VM, add to /etc/fstab:

    10.10.3.234:/data/media /mnt/media nfs defaults 0 0 10.10.3.234:/data/appdata /mnt/appdata nfs defaults 0 0
🗂️
SMB/CIFS Shares via Debian VM

Install Samba in the Debian VM and share the NFS-mounted paths:

apt install samba
# /etc/samba/smb.conf (excerpts) [global] workgroup = HOMELAB server string = ODIN Storage security = user [Media] path = /mnt/media browseable = yes writable = no valid users = @storage [AppData] path = /mnt/appdata browseable = yes writable = yes valid users = @storage

Add users and start Samba:

smbpasswd -a <username> systemctl enable --now smbd nmbd

Windows clients can now map: \\10.10.3.234\Media

📸
ZFS Snapshots & Replication

ZFS snapshots are instantaneous and space-efficient. Set up automated snapshots with a cron job:

# Take hourly snapshots, keep 24 hours zfs snapshot data/media@$(date +%Y%m%d-%H%M%S) # Or use zfstools for automated management pkg install zfstools echo "*/15 * * * * root /usr/local/sbin/zfs-auto-snapshot frequent 4" >> /etc/crontab echo "0 * * * * root /usr/local/sbin/zfs-auto-snapshot hourly 24" >> /etc/crontab echo "0 0 * * * root /usr/local/sbin/zfs-auto-snapshot daily 30" >> /etc/crontab

ZFS Send/Receive for off-site backup:

# Incremental send to remote backup server over SSH zfs send -i data/media@snapshot1 data/media@snapshot2 | \ ssh backup-server "zfs receive backup/media"
📚
References