io/writeback: WRITEBACK_CHUNK_BYTES 32 -> 128 MiB
The pipeline's adaptive autotuner grows chunk_bytes only when the p95 WAIT_AFTER latency exceeds 200 ms. On NFS, sync_file_range(WAIT_AFTER) translates to an NFS COMMIT RPC whose ack lands within ~10 ms — so the autotuner never triggered and the pipeline stayed at the original 32 MiB initial value forever. That capped sustained mux throughput by paying NFS COMMIT-RPC overhead roughly once per second of writes. Bidirectional mountstats on rip1 2026-05-15: write side 29 MB/s (RTT 30 ms but exec_time 257 ms — 220 ms queue/serial waiting) while concurrent dd on the same disk shows ~91 MB/s write + ~65 MB/s read available. 128 MiB initial drops COMMIT cadence 4x while keeping the bounded- cache invariant intact (worst-case dirty pages ~2 x chunk = 256 MiB, well under vm.dirty_ratio = 6.6 GB on the 32 GB rig). The adaptive autotuner can still grow further (up to 256 MiB) or shrink if WAIT_AFTER ever measures sub-20 ms p95 on faster media.
This commit is contained in:
@@ -121,10 +121,24 @@ use std::thread::{self, JoinHandle};
|
|||||||
|
|
||||||
use super::writeback::WritebackPipeline;
|
use super::writeback::WritebackPipeline;
|
||||||
|
|
||||||
/// Granularity at which the Linux writeback pipeline issues
|
/// Initial granularity at which the Linux writeback pipeline issues
|
||||||
/// `sync_file_range` / `posix_fadvise(DONTNEED)` pairs. 32 MiB is the
|
/// `sync_file_range` / `posix_fadvise(DONTNEED)` pairs. The pipeline's
|
||||||
/// historical default — bounded-cache pressure stays at ~2 × this size.
|
/// adaptive autotuner grows/shrinks this between
|
||||||
const WRITEBACK_CHUNK_BYTES: u64 = 32 * 1024 * 1024;
|
/// [`super::writeback::linux::CHUNK_BYTES_MIN`] and `CHUNK_BYTES_MAX`
|
||||||
|
/// based on p95 of `WAIT_AFTER` latency.
|
||||||
|
///
|
||||||
|
/// 0.21.14: bumped from 32 MiB to 128 MiB. On NFS,
|
||||||
|
/// `sync_file_range(WAIT_AFTER)` translates to an NFS COMMIT RPC whose
|
||||||
|
/// completion ack arrives within ~10 ms (measured via mountstats), so
|
||||||
|
/// the adaptive autotuner — which only grows when p95 > 200 ms —
|
||||||
|
/// never triggered and the pipeline sat at 32 MiB forever, paying
|
||||||
|
/// COMMIT-RPC overhead per ~1 s of writes. Empirical 2026-05-15:
|
||||||
|
/// bidirectional mux on rip1 capped at ~29 MB/s write side while the
|
||||||
|
/// rig's bidirectional dd ceiling is ~91 MB/s write + ~65 MB/s read.
|
||||||
|
/// At a larger initial chunk the COMMIT cadence drops 4×, leaving
|
||||||
|
/// the dirty-cache invariant intact (bounded at ~2 × chunk = 256 MiB,
|
||||||
|
/// well under the kernel's `vm.dirty_ratio` cap on a 32 GB rig).
|
||||||
|
const WRITEBACK_CHUNK_BYTES: u64 = 128 * 1024 * 1024;
|
||||||
|
|
||||||
/// Maximum bytes outstanding in the muxer → writer-thread ring. Sized
|
/// Maximum bytes outstanding in the muxer → writer-thread ring. Sized
|
||||||
/// to cover ~4 s of muxer output at a 32 MB/s peak — enough to absorb a
|
/// to cover ~4 s of muxer output at a 32 MB/s peak — enough to absorb a
|
||||||
|
|||||||
Reference in New Issue
Block a user