Understanding and Managing IOMMU Groups in Proxmox VE

What Are IOMMU Groups?

IOMMU groups are collections of PCI devices that the hardware treats as a single isolation unit. When you pass through a PCI device to a VM in Proxmox, you must pass through all devices in the same IOMMU group. This is a hardware-level security boundary — devices in the same group can access each other's memory, so the IOMMU cannot isolate them individually. Understanding your IOMMU groups is the first step before any PCI passthrough configuration.

List All IOMMU Groups

Use this script to enumerate all IOMMU groups and their devices on your Proxmox host:

#!/bin/bash
# Save as /usr/local/bin/iommu-groups.sh and chmod +x
for g in $(find /sys/kernel/iommu_groups/ -maxdepth 1 -mindepth 1 -type d | sort -V); do
    echo "IOMMU Group ${g##*/}:"
    for d in "$g"/devices/*; do
        echo -e "\t$(lspci -nns ${d##*/})"
    done
done

Run it and examine the output:

chmod +x /usr/local/bin/iommu-groups.sh
/usr/local/bin/iommu-groups.sh

# Example output:
# IOMMU Group 1:
#     00:01.0 PCI bridge [0604]: Intel Corporation ... [8086:1901]
#     01:00.0 VGA compatible controller [0300]: NVIDIA Corporation ... [10de:2484]
#     01:00.1 Audio device [0403]: NVIDIA Corporation ... [10de:228b]
# IOMMU Group 2:
#     00:14.0 USB controller [0c03]: Intel Corporation ... [8086:a36d]
#     00:14.2 Signal processing controller [1180]: Intel Corporation ... [8086:a371]

In the example above, IOMMU Group 1 contains a PCI bridge, the GPU, and its audio controller — all three must be passed through together. Group 2 contains a USB controller and a signal processing controller sharing the same group.

The ACS Override Patch

Consumer motherboards often place many devices in a single large IOMMU group because they lack proper ACS (Access Control Services) support. The ACS override patch forces the kernel to split these groups artificially. This is a security trade-off — it weakens DMA isolation between devices — but it is sometimes the only way to pass through a specific device on consumer hardware.

# Edit GRUB to add the ACS override:
nano /etc/default/grub

# Add to GRUB_CMDLINE_LINUX_DEFAULT:
GRUB_CMDLINE_LINUX_DEFAULT="quiet intel_iommu=on iommu=pt pcie_acs_override=downstream,multifunction"

# Update GRUB and reboot:
update-grub
reboot

After reboot, run the IOMMU groups script again. You should see previously grouped devices now split into separate groups. Only use the ACS override if your devices are not in clean groups by default.

Isolate Devices with VFIO-PCI Binding

Once your target device is in a usable IOMMU group, bind it to the vfio-pci driver so the host does not claim it. Identify the device and vendor IDs from the IOMMU groups script output:

# For an NVIDIA GPU with IDs 10de:2484 and audio 10de:228b:
echo "options vfio-pci ids=10de:2484,10de:228b" > /etc/modprobe.d/vfio.conf

# Ensure VFIO modules load early:
nano /etc/modules
# Add these lines:
vfio
vfio_iommu_type1
vfio_pci

# Update initramfs:
update-initramfs -u -k all

# Reboot the host:
reboot

After reboot, verify the device is bound to vfio-pci:

lspci -nnk -s 01:00
# Should show: Kernel driver in use: vfio-pci

Handling Multifunction Devices

Many PCI devices are multifunction — a GPU typically appears as two functions (video and audio) at the same PCI address (e.g., 01:00.0 and 01:00.1). You must pass through all functions of a multifunction device together. In the Proxmox VM configuration:

# /etc/pve/qemu-server/100.conf
hostpci0: 01:00,pcie=1,x-vga=1

# The "01:00" without a function number passes through ALL functions
# at that address (01:00.0, 01:00.1, etc.)

# If you only want a specific function (not recommended for GPUs):
hostpci0: 01:00.0,pcie=1

Troubleshooting IOMMU Issues

Common issues and solutions:

• No IOMMU groups found: Verify IOMMU is enabled in BIOS (VT-d or AMD-Vi) and the kernel parameter (intel_iommu=on or amd_iommu=on) is set.
• Device still bound to host driver: Check that the vfio-pci IDs are correct and that initramfs was regenerated. Blacklist the host driver if needed: echo "blacklist nouveau" >> /etc/modprobe.d/blacklist.conf
• Large groups on server hardware: Server boards usually have better ACS support. Update your BIOS firmware before resorting to the ACS override.

# Verify IOMMU is active:
dmesg | grep -e DMAR -e IOMMU

# Check if VFIO loaded:
dmesg | grep vfio

# Verify kernel parameters:
cat /proc/cmdline

Working through IOMMU group issues often involves multiple reboots and configuration changes. ProxmoxR lets you check whether your Proxmox host has come back online after each reboot and review task logs from your phone, which helps when you are iterating through BIOS changes at the console.

Summary

IOMMU groups are the foundation of PCI passthrough in Proxmox VE. Use the IOMMU groups script to understand your hardware layout, apply the ACS override patch only when necessary, bind target devices to vfio-pci, and remember to pass through all functions of multifunction devices. Getting your IOMMU groups right is the most important step toward a successful passthrough configuration.