Proxmox ZFS Upgrade Issues and Fixes

ZFS and Proxmox VE Upgrades: Why Issues Occur

ZFS is deeply integrated into Proxmox VE as both a storage backend and a root filesystem option. During major version upgrades, the ZFS kernel module, userspace tools, and pool feature flags can all change. These changes occasionally create compatibility issues that prevent pools from importing, cause boot failures, or trigger performance degradation. Understanding the relationship between these components is key to a smooth upgrade.

Checking Your Current ZFS State Before Upgrading

Before any Proxmox upgrade, document your ZFS configuration:

# Check ZFS module version
modinfo zfs | grep ^version

# Check userspace tools version
zfs --version
zpool --version

# List all pools and their feature flags
zpool status
zpool get all rpool | grep feature

# List any pool feature flags not yet enabled
zpool upgrade
zpool upgrade -v

Save this output. If something goes wrong after the upgrade, you will need to compare these values to diagnose the problem.

Understanding zpool Upgrade and Feature Flags

ZFS pools have feature flags that enable new functionality. When you run zpool upgrade, it enables all feature flags supported by the current ZFS version. This is a one-way operation -- once a feature flag is enabled, it cannot be disabled, and older ZFS versions that do not support that flag will be unable to import the pool.

# See which features are available but not yet enabled
zpool upgrade

# Example output:
# This system supports ZFS pool feature flags.
# The following pools could be upgraded:
#   rpool

# Upgrade a specific pool (IRREVERSIBLE)
zpool upgrade rpool

# Or upgrade all pools
zpool upgrade -a

Important: Do not run zpool upgrade until you are certain you will not need to import this pool on a system running an older ZFS version. In a cluster with mixed Proxmox versions during a rolling upgrade, premature pool upgrades can prevent other nodes from accessing shared storage.

Pool Import Failures After Upgrade

The most common ZFS issue after a Proxmox upgrade is a pool that refuses to import. This typically happens when the ZFS kernel module version does not match what the pool expects, or when the module fails to load entirely.

# Try importing the pool manually
zpool import rpool

# If it fails, try with force
zpool import -f rpool

# If it reports missing features, check module version
modinfo zfs | grep version
zpool import -d /dev/disk/by-id rpool

# If the module is not loaded
modprobe zfs
# Check for errors
dmesg | tail -30 | grep -i zfs

If modprobe zfs fails, the ZFS kernel module was not built for the running kernel. This requires rebuilding DKMS modules or installing the correct pre-built module package:

# Rebuild DKMS modules for the current kernel
dkms autoinstall

# Or reinstall the matching module package
apt install --reinstall pve-kernel-$(uname -r)

# Load the module
modprobe zfs

# Try importing again
zpool import -f rpool

ZFS Module Version Mismatch

A module version mismatch occurs when the kernel module (loaded in memory) is a different version from the userspace tools (zfs, zpool commands). This can cause unpredictable behavior, failed operations, and cryptic error messages.

# Compare kernel module and userspace versions
echo "Kernel module: $(cat /sys/module/zfs/version)"
echo "Userspace:     $(zfs --version | head -1)"

# If they differ, reinstall both
apt install --reinstall zfsutils-linux zfs-zed

# Unload and reload the module (WARNING: unmount all ZFS filesystems first)
# Only do this if no ZFS pools are actively in use
zfs unmount -a
zpool export -a
modprobe -r zfs
modprobe zfs
zpool import -a

Rebuilding initramfs for ZFS Boot

If your Proxmox host boots from a ZFS root pool, the initramfs must include the correct ZFS module. A corrupted or outdated initramfs is the most common reason for boot failures after kernel upgrades.

# Rebuild initramfs for the current kernel
update-initramfs -u -k $(uname -r)

# Rebuild initramfs for all installed kernels
update-initramfs -u -k all

# Refresh the boot tool (critical for ZFS boot)
proxmox-boot-tool refresh

# Verify the initramfs contains the ZFS module
lsinitramfs /boot/initrd.img-$(uname -r) | grep zfs

If lsinitramfs shows no ZFS files, the module was not included during the initramfs build. This usually means the ZFS DKMS module was not properly installed for that kernel version. Install it first, then rebuild:

# Ensure DKMS module is built
dkms status
dkms install zfs/$(dkms status | grep zfs | head -1 | awk -F'[ ,]' '{print $2}') -k $(uname -r)

# Rebuild initramfs again
update-initramfs -u -k $(uname -r)
proxmox-boot-tool refresh

Boot from ZFS Issues and Recovery

If your system fails to boot after an upgrade, you will need to use a rescue environment. Boot from the Proxmox VE installer ISO and open a shell:

# From the Proxmox installer rescue shell:
# Import the root pool
zpool import -f -R /mnt rpool

# Mount the root filesystem
zfs mount rpool/ROOT/pve-1

# Mount additional required filesystems
mount --rbind /dev /mnt/dev
mount --rbind /proc /mnt/proc
mount --rbind /sys /mnt/sys

# Chroot into the installed system
chroot /mnt

# Rebuild initramfs and refresh boot
update-initramfs -u -k all
proxmox-boot-tool refresh

# Exit and reboot
exit
umount -R /mnt
zpool export rpool
reboot

If the ZFS pool was upgraded with feature flags incompatible with the installed kernel module, you may need to install the newer kernel package from within the chroot to get a matching ZFS module.

ZFS upgrade issues can be time-consuming to debug, especially when managing multiple nodes. Tracking which pools have been upgraded and which kernel/ZFS module combinations are running across your infrastructure is essential. ProxmoxR can help you maintain visibility into your nodes' storage health, so you can spot mismatches before they cause boot failures or data access issues.