io/file_sector_source: throttle readahead for NFS bidirectional workloads
Empirical regression observed during 0.21.1 mux test on rip1/unraid-1: historical 0.20.7 baseline averaged ~18 MB/s mux throughput; 0.21.1 dropped to ~7-8 MB/s flat. Same NFS source + destination, same disc. Suspect: 32 MiB FileSectorSource readahead + posix_fadvise(SEQUENTIAL) together saturate the TCP connection on read bursts, starving the writer thread's concurrent NFS writes (mux reads the source ISO and writes the MKV over the same connection). - READAHEAD_BUF_BYTES: 32 MiB -> 4 MiB. Matches NFS rsize=1 MiB * 4 round-trips per refill, interleaves cleanly with writes. - linux/hint_sequential: now no-op. Kernel's default ~128 KiB readahead is what we want on NFS-backed ISOs (the dominant case). Per-OS file stays so we can re-enable a hint cleanly later if a different path benefits.
This commit is contained in:
@@ -1,15 +1,18 @@
|
||||
//! Linux: hint the kernel that this fd will be read sequentially so
|
||||
//! readahead widens. `posix_fadvise(POSIX_FADV_SEQUENTIAL)` is a hint,
|
||||
//! not a guarantee — the kernel still owns the policy decision.
|
||||
//! Linux: kernel readahead hint for the ISO file.
|
||||
//!
|
||||
//! Originally `POSIX_FADV_SEQUENTIAL` to widen the readahead window.
|
||||
//! On NFS that turned out to cause aggressive multi-MB readahead bursts
|
||||
//! that saturated the TCP connection and starved concurrent writes
|
||||
//! during mux — observed empirically as a ~3× drop in mux throughput
|
||||
//! on the rip1/unraid-1 setup (0.21.0 vs 0.20.7 baseline). The kernel's
|
||||
//! default readahead (~128 KiB on Linux) interleaves more naturally
|
||||
//! with the muxer's concurrent NFS writes, so we no longer issue any
|
||||
//! hint here. The per-OS file stays so the convention is honoured and
|
||||
//! we can re-enable a hint cleanly if a different storage path benefits.
|
||||
|
||||
use std::fs::File;
|
||||
use std::os::unix::io::AsRawFd;
|
||||
|
||||
pub(super) fn hint_sequential(file: &File, _len_bytes: u64) {
|
||||
// Best-effort: return value ignored. A fadvise failure has no
|
||||
// user-observable consequence (reads still work, just without the
|
||||
// widened readahead window).
|
||||
unsafe {
|
||||
libc::posix_fadvise(file.as_raw_fd(), 0, 0, libc::POSIX_FADV_SEQUENTIAL);
|
||||
}
|
||||
pub(super) fn hint_sequential(_file: &File, _len_bytes: u64) {
|
||||
// No-op: see module-level comment. Kernel default readahead is
|
||||
// what we want on NFS-backed ISOs, which is the dominant case.
|
||||
}
|
||||
|
||||
@@ -65,8 +65,15 @@ use crate::sector::SectorSource;
|
||||
/// than per-sector pread, and large enough to coast through a typical
|
||||
/// NFS server commit blip.
|
||||
///
|
||||
/// 0.21.2: shrunk from 32 MiB → 4 MiB. On NFS-backed ISOs with
|
||||
/// concurrent NFS writes (the mux phase), a 32 MiB refill bursts the
|
||||
/// TCP connection hard enough to starve the writer thread, observed
|
||||
/// empirically as a ~3× drop in sustained mux throughput on the
|
||||
/// rip1/unraid-1 setup. 4 MiB matches `rsize=1 MiB` × 4 round-trips
|
||||
/// and interleaves cleanly with writes.
|
||||
///
|
||||
/// Tweakable. Named const, not a magic number.
|
||||
pub const READAHEAD_BUF_BYTES: usize = 32 * 1024 * 1024;
|
||||
pub const READAHEAD_BUF_BYTES: usize = 4 * 1024 * 1024;
|
||||
|
||||
const SECTOR_SIZE: usize = 2048;
|
||||
/// Sectors per refill: [`READAHEAD_BUF_BYTES`] / [`SECTOR_SIZE`]. The
|
||||
|
||||
Reference in New Issue
Block a user