file_sector_source: restore read-side DONTNEED + SEQUENTIAL (the actual fix)

Empirical: isolated NFS read 70 MB/s + write 93 MB/s on the rip1 setup
right now, but mux throughput pinned at 2.7 MB/s on 0.21.5. NOT
environmental — code regression.

Root cause: Phase 1 silently dropped the read-side
posix_fadvise(POSIX_FADV_DONTNEED) eviction that the pre-Phase-1 (0.20.7)
hot path had. Without it, an 85 GB streaming ISO read pins the entire
file in the kernel page cache, starving concurrent MKV writeback. 0.21.2
then also dropped the POSIX_FADV_SEQUENTIAL hint on the same theory,
compounding the regression.

Restored both, per-OS split:
- linux: posix_fadvise(SEQUENTIAL) at open + posix_fadvise(DONTNEED)
  on consumed 32 MiB windows
- macos: F_RDADVISE hint at open (kept); drop_window no-op (macOS unified
  buffer cache less prone to the pin pathology)
- windows / other: both no-op stubs

Target mux speed restored to 20+ MB/s (per concurrent-NFS math:
70/2 read × 0.73 MKV/ISO ratio ≈ 25 MB/s achievable).
This commit is contained in:
MattJackson
2026-05-14 09:16:23 -07:00
parent a383200ef1
commit e3b2c9d850
5 changed files with 91 additions and 13 deletions
+41 -13
View File
@@ -1,18 +1,46 @@
//! Linux: kernel readahead hint for the ISO file.
//! Linux read-side platform hooks: sequential-access hint at open +
//! periodic page-cache eviction during streaming reads.
//!
//! Originally `POSIX_FADV_SEQUENTIAL` to widen the readahead window.
//! On NFS that turned out to cause aggressive multi-MB readahead bursts
//! that saturated the TCP connection and starved concurrent writes
//! during mux — observed empirically as a ~3× drop in mux throughput
//! on the rip1/unraid-1 setup (0.21.0 vs 0.20.7 baseline). The kernel's
//! default readahead (~128 KiB on Linux) interleaves more naturally
//! with the muxer's concurrent NFS writes, so we no longer issue any
//! hint here. The per-OS file stays so the convention is honoured and
//! we can re-enable a hint cleanly if a different storage path benefits.
//! ## Why both
//!
//! `POSIX_FADV_SEQUENTIAL` at open widens the kernel's readahead window
//! so each pread aggregates into fewer NFS round-trips. `DONTNEED` on
//! the consumed window (called periodically by the caller) drops the
//! already-read pages from the page cache so an 85 GB streaming ISO
//! read doesn't fill memory and starve concurrent writes (the MKV
//! output during mux). Together they mirror the write-side
//! WritebackPipeline's policy.
//!
//! ## History
//!
//! Pre-Phase-1 (0.20.7 baseline) had both. Phase 1's introduction of
//! `FileSectorSource` silently dropped the read-side DONTNEED, and
//! 0.21.2's revert of `SEQUENTIAL` (mistakenly attributing a regression
//! to it) removed the hint. Net effect: 85 GB of ISO reads pinned in
//! the page cache + no readahead widening → mux throughput collapse
//! from 18 MB/s historical to 2.7-8 MB/s on 0.21.x. Restored in 0.21.6.
use std::fs::File;
use std::os::unix::io::AsRawFd;
pub(super) fn hint_sequential(_file: &File, _len_bytes: u64) {
// No-op: see module-level comment. Kernel default readahead is
// what we want on NFS-backed ISOs, which is the dominant case.
pub(super) fn hint_sequential(file: &File, _len_bytes: u64) {
// Best-effort: return value ignored. A fadvise failure has no
// user-observable consequence.
unsafe {
libc::posix_fadvise(file.as_raw_fd(), 0, 0, libc::POSIX_FADV_SEQUENTIAL);
}
}
/// Drop pages in the half-open byte range `[start, start+len)` from
/// the page cache. Called periodically by `read_sectors` to bound the
/// read-side page cache pressure.
pub(super) fn drop_window(file: &File, start: u64, len: u64) {
unsafe {
libc::posix_fadvise(
file.as_raw_fd(),
start as i64,
len as i64,
libc::POSIX_FADV_DONTNEED,
);
}
}