From 50867516b8ba0177bed01eb05ac450de4d050387 Mon Sep 17 00:00:00 2001
From: MattJackson <1085847+MattJackson@users.noreply.github.com>
Date: Fri, 15 May 2026 12:02:40 -0700
Subject: [PATCH] io/writeback: WRITEBACK_CHUNK_BYTES 32 -> 128 MiB
MIME-Version: 1.0
Content-Type: text/plain; charset=UTF-8
Content-Transfer-Encoding: 8bit

The pipeline's adaptive autotuner grows chunk_bytes only when the p95
WAIT_AFTER latency exceeds 200 ms. On NFS, sync_file_range(WAIT_AFTER)
translates to an NFS COMMIT RPC whose ack lands within ~10 ms — so
the autotuner never triggered and the pipeline stayed at the original
32 MiB initial value forever.

That capped sustained mux throughput by paying NFS COMMIT-RPC overhead
roughly once per second of writes. Bidirectional mountstats on rip1
2026-05-15: write side 29 MB/s (RTT 30 ms but exec_time 257 ms — 220 ms
queue/serial waiting) while concurrent dd on the same disk shows
~91 MB/s write + ~65 MB/s read available.

128 MiB initial drops COMMIT cadence 4x while keeping the bounded-
cache invariant intact (worst-case dirty pages ~2 x chunk = 256 MiB,
well under vm.dirty_ratio = 6.6 GB on the 32 GB rig). The adaptive
autotuner can still grow further (up to 256 MiB) or shrink if
WAIT_AFTER ever measures sub-20 ms p95 on faster media.
---
 src/io/writeback_file/mod.rs | 22 ++++++++++++++++++----
 1 file changed, 18 insertions(+), 4 deletions(-)

diff --git a/src/io/writeback_file/mod.rs b/src/io/writeback_file/mod.rs
index 086ec4f..c9cbdba 100644
--- a/src/io/writeback_file/mod.rs
+++ b/src/io/writeback_file/mod.rs
@@ -121,10 +121,24 @@ use std::thread::{self, JoinHandle};
 
 use super::writeback::WritebackPipeline;
 
-/// Granularity at which the Linux writeback pipeline issues
-/// `sync_file_range` / `posix_fadvise(DONTNEED)` pairs. 32 MiB is the
-/// historical default — bounded-cache pressure stays at ~2 × this size.
-const WRITEBACK_CHUNK_BYTES: u64 = 32 * 1024 * 1024;
+/// Initial granularity at which the Linux writeback pipeline issues
+/// `sync_file_range` / `posix_fadvise(DONTNEED)` pairs. The pipeline's
+/// adaptive autotuner grows/shrinks this between
+/// [`super::writeback::linux::CHUNK_BYTES_MIN`] and `CHUNK_BYTES_MAX`
+/// based on p95 of `WAIT_AFTER` latency.
+///
+/// 0.21.14: bumped from 32 MiB to 128 MiB. On NFS,
+/// `sync_file_range(WAIT_AFTER)` translates to an NFS COMMIT RPC whose
+/// completion ack arrives within ~10 ms (measured via mountstats), so
+/// the adaptive autotuner — which only grows when p95 > 200 ms —
+/// never triggered and the pipeline sat at 32 MiB forever, paying
+/// COMMIT-RPC overhead per ~1 s of writes. Empirical 2026-05-15:
+/// bidirectional mux on rip1 capped at ~29 MB/s write side while the
+/// rig's bidirectional dd ceiling is ~91 MB/s write + ~65 MB/s read.
+/// At a larger initial chunk the COMMIT cadence drops 4×, leaving
+/// the dirty-cache invariant intact (bounded at ~2 × chunk = 256 MiB,
+/// well under the kernel's `vm.dirty_ratio` cap on a 32 GB rig).
+const WRITEBACK_CHUNK_BYTES: u64 = 128 * 1024 * 1024;
 
 /// Maximum bytes outstanding in the muxer → writer-thread ring. Sized
 /// to cover ~4 s of muxer output at a 32 MB/s peak — enough to absorb a