Zero-copy data path / reduce `memcpy` overhead for zvol direct I/O (PCIe 5.0 / NVMeoF use case)

When driving high-bandwidth workloads via OpenZFS `zvols` (specifically for exporting block storage over NVMe-oF), the CPU becomes a massive bottleneck due to memory copying overhead. Even when `primarycache=metadata` is enforced and the application uses `O_DIRECT`. On modern PCIe 5.0 NVMe SSDs hardware capable of 14+ GB/s, this `memcpy` operation consumes a staggering amount of CPU cycles and memory bandwidth, severely impacting latency profiles at higher queue depths. I would like to request an enhancement or architectural exploration into a zero-copy or optimize data path for direct I/O reads on `zvols`. Hardware: PCIe 5.0 NVMe SSDs, AMD Zen5, DDR5 DRAM 6400 Software: Rocky Linux 10.2 (6.12.0-211.18.1.el10_2.x86_64), OpenZFS 2.4.2 [root@memverge4 anton]# echo 1 > /sys/module/zfs/parameters/zfs_prefetch_disable [root@memverge4 anton]# echo 64 > /sys/module/zfs/parameters/zfs_vdev_async_write_max_active [root@memverge4 anton]# echo 64 > /sys/module/zfs/parameters/zfs_vdev_async_read_max_active [root@memverge4 anton]# echo 64 > /sys/module/zfs/parameters/zfs_vdev_sync_write_max_active [root@memverge4 anton]# echo 64 > /sys/module/zfs/parameters/zfs_vdev_sync_read_max_active [root@memverge4 anton]# echo 1 > /sys/module/zfs/parameters/zfs_txg_timeout zvol properties: primarycache=metadata secondarycache=none compression=off dedup=off checksum=off volblocksize=1M [root@memverge4 anton]# zfs get volblocksize tank/fiotest NAME PROPERTY VALUE SOURCE tank/fiotest volblocksize 1M - [root@memverge4 anton]# zpool status tank pool: tank state: ONLINE config: NAME STATE READ WRITE CKSUM tank ONLINE 0 0 0 raidz2-0 ONLINE 0 0 0 nvme0n1 ONLINE 0 0 0 nvme1n1 ONLINE 0 0 0 nvme2n1 ONLINE 0 0 0 nvme3n1 ONLINE 0 0 0 nvme4n1 ONLINE 0 0 0 nvme5n1 ONLINE 0 0 0 errors: No known data errors Observed behavior and performance data [root@memverge4 anton]# fio --name=test --rw=read --bs=1m --filename=/dev/zvol/tank/fiotest --direct=1 --numjobs=1 --iodepth=32 --exitall --group_reporting --ioengine=libaio --runtime=60 --time_based test: (g=0): rw=read, bs=(R) 1024KiB-1024KiB, (W) 1024KiB-1024KiB, (T) 1024KiB-1024KiB, ioengine=libaio, iodepth=32 fio-3.42-39-g5e65 Starting 1 process Jobs: 1 (f=1): [R(1)][100.0%][r=33.9GiB/s][r=34.7k IOPS][eta 00m:00s] test: (groupid=0, jobs=1): err= 0: pid=36982: Sun Jun 7 20:25:24 2026 read: IOPS=34.5k, BW=33.7GiB/s (36.2GB/s)(2024GiB/60001msec) slat (nsec): min=1562, max=249454, avg=3849.49, stdev=3721.13 clat (usec): min=196, max=2499, avg=922.25, stdev=130.74 lat (usec): min=202, max=2749, avg=926.10, stdev=130.60 Samples: 282K of event 'cycles:P', 4000 Hz, Event count (approx.): 126312343664 lost: 0/0 drop: 0/0 Overhead Shared Object Symbol 37.83% [kernel] [k] __memcpy 3.62% [kernel] [k] read_tsc 2.96% [kernel] [k] vdev_disk_check_alignment_cb 2.25% [kernel] [k] vbio_fill_cb 2.00% [kernel] [k] __slab_free 1.48% [kernel] [k] zfs_uiomove_bvec_rq 1.17% [kernel] [k] vmalloc_to_page 1.12% [kernel] [k] abd_iter_page 0.86% [kernel] [k] vdev_disk_io_done 0.86% [kernel] [k] enqueue_task_fair 0.83% [kernel] [k] select_task_rq_fair 0.73% [kernel] [k] zio_create 0.68% [kernel] [k] _raw_spin_lock_irqsave 0.65% [kernel] [k] zfs_copy_bvec 0.65% [kernel] [k] dbuf_compare 0.64% [kernel] [k] percpu_counter_add_batch 0.62% [kernel] [k] put_prev_task_idle 0.59% [kernel] [k] bvec_split_segs 0.58% [kernel] [k] default_wake_function 0.54% [kernel] [k] mutex_lock 0.54% [kernel] [k] _raw_spin_lock 0.53% [kernel] [k] wbt_done 0.52% [kernel] [k] kfree Samples: 720K of event 'cycles:P', 4000 Hz, Event count (approx.): 262052518971 __memcpy /proc/kcore [Percent: local period] 0.02 │ endbr64 0.03 │ nop 0.03 │ movq %rdi,%rax 0.02 │ movq %rdx,%rcx 98.44 │ rep movsb (%rsi), %es:(%rdi) 1.46 │ ← retq │ int3 │ int3 │ int3 │ int3 │ nop │ nop Is it possible to optimize execution path for zvols initialized with primarycache=metadata where read data can be DMAd directly into the bio_vec structures ?, or any other possibilities. It will allow unlock huge bandwidth potential 14GB/s for PCIe 5.0 and 28GB/s for upcoming PCIe 6.0