io/writeback: WRITEBACK_CHUNK_BYTES 32 -> 128 MiB

The pipeline's adaptive autotuner grows chunk_bytes only when the p95
WAIT_AFTER latency exceeds 200 ms. On NFS, sync_file_range(WAIT_AFTER)
translates to an NFS COMMIT RPC whose ack lands within ~10 ms — so
the autotuner never triggered and the pipeline stayed at the original
32 MiB initial value forever.

That capped sustained mux throughput by paying NFS COMMIT-RPC overhead
roughly once per second of writes. Bidirectional mountstats on rip1
2026-05-15: write side 29 MB/s (RTT 30 ms but exec_time 257 ms — 220 ms
queue/serial waiting) while concurrent dd on the same disk shows
~91 MB/s write + ~65 MB/s read available.

128 MiB initial drops COMMIT cadence 4x while keeping the bounded-
cache invariant intact (worst-case dirty pages ~2 x chunk = 256 MiB,
well under vm.dirty_ratio = 6.6 GB on the 32 GB rig). The adaptive
autotuner can still grow further (up to 256 MiB) or shrink if
WAIT_AFTER ever measures sub-20 ms p95 on faster media.
This commit is contained in:
2026-05-15 12:02:40 -07:00
parent 5498280ee7
commit c802cd95b0
+18 -4
View File
@@ -121,10 +121,24 @@ use std::thread::{self, JoinHandle};
use super::writeback::WritebackPipeline;
/// Granularity at which the Linux writeback pipeline issues
/// `sync_file_range` / `posix_fadvise(DONTNEED)` pairs. 32 MiB is the
/// historical default — bounded-cache pressure stays at ~2 × this size.
const WRITEBACK_CHUNK_BYTES: u64 = 32 * 1024 * 1024;
/// Initial granularity at which the Linux writeback pipeline issues
/// `sync_file_range` / `posix_fadvise(DONTNEED)` pairs. The pipeline's
/// adaptive autotuner grows/shrinks this between
/// [`super::writeback::linux::CHUNK_BYTES_MIN`] and `CHUNK_BYTES_MAX`
/// based on p95 of `WAIT_AFTER` latency.
///
/// 0.21.14: bumped from 32 MiB to 128 MiB. On NFS,
/// `sync_file_range(WAIT_AFTER)` translates to an NFS COMMIT RPC whose
/// completion ack arrives within ~10 ms (measured via mountstats), so
/// the adaptive autotuner — which only grows when p95 > 200 ms —
/// never triggered and the pipeline sat at 32 MiB forever, paying
/// COMMIT-RPC overhead per ~1 s of writes. Empirical 2026-05-15:
/// bidirectional mux on rip1 capped at ~29 MB/s write side while the
/// rig's bidirectional dd ceiling is ~91 MB/s write + ~65 MB/s read.
/// At a larger initial chunk the COMMIT cadence drops 4×, leaving
/// the dirty-cache invariant intact (bounded at ~2 × chunk = 256 MiB,
/// well under the kernel's `vm.dirty_ratio` cap on a 32 GB rig).
const WRITEBACK_CHUNK_BYTES: u64 = 128 * 1024 * 1024;
/// Maximum bytes outstanding in the muxer → writer-thread ring. Sized
/// to cover ~4 s of muxer output at a 32 MB/s peak — enough to absorb a