pq/libfreemkv
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

io/file_sector_source: bypass app-level buffer — direct pread per call

Browse Source 
The 32 MiB readahead window (0.21.0–0.21.1) regressed mux throughput
on NFS bidirectional workloads vs the pre-Phase-1 0.20.7 baseline
(18 -> 7-8 MB/s). The 0.21.2 4 MiB shrink made it worse (5-6 MB/s).
Both signs point at the application-level buffer itself, not the size.

This commit bypasses the buffer entirely on the read path — every
read_sectors call seeks and pread()s direct to the file. That matches
0.20.7's hot path. Kernel readahead handles the policy; on NFS that
interleaves naturally with concurrent writes on the same TCP
connection.

Buffer state fields and refill/buffer_covers are kept so the
structure is preserved for a future per-source-type policy (e.g. a
local-disk source where batched reads ARE beneficial), and so the
existing tests still exercise that machinery.
This commit is contained in:
MattJackson
2026-05-13 23:57:46 -07:00
parent a34c419521
commit 55219070f1
1 changed files with 22 additions and 27 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/io/file_sector_source/mod.rs
+22 -27
View File
@@ -207,33 +207,28 @@ impl SectorSource for FileSectorSource {
if count == 0 { if count == 0 {
return Ok(0); return Ok(0);
} }
// Refill if the requested range isn't entirely buffered. // 0.21.3: bypass the application-level buffer entirely.
// `buffer_covers` also handles the empty-buffer case //
// (buf_len_sectors == 0). // Empirically the 32 MiB readahead window (0.21.0–0.21.1) and the
if !self.buffer_covers(lba, count) { // 4 MiB shrink (0.21.2) both regressed mux throughput vs the
// A request larger than the buffer itself can never fit; // pre-Phase-1 0.20.7 baseline on NFS bidirectional workloads
// fall back to a one-shot direct pread for that pathological // (sweep ~25 MB/s OK; mux dropped from 18 → 7-8 → 5-6 MB/s).
// case so callers can't deadlock the source. // Direct pread per call lets the kernel's own readahead policy
if count > BUF_SECTORS { // run, which interleaves naturally with concurrent NFS writes on
let offset = lba as u64 * SECTOR_SIZE as u64; // the same TCP connection.
self.file //
.seek(SeekFrom::Start(offset)) // Buffer fields are retained (currently unused on this path) so
.map_err(|e| Error::IoError { source: e })?; // any future per-source policy can be reintroduced without
self.file // re-plumbing structure. `refill` / `buffer_covers` are kept too
.read_exact(&mut out[..bytes]) // (still exercised by the tests so the API contract is locked).
.map_err(|e| Error::IoError { source: e })?; let offset = lba as u64 * SECTOR_SIZE as u64;
// Invalidate buffer state — we bypassed it, the self.file
// window is no longer authoritative for this LBA. .seek(SeekFrom::Start(offset))
self.buf_len_sectors = 0; .map_err(|e| Error::IoError { source: e })?;
return Ok(bytes); self.file
} .read_exact(&mut out[..bytes])
self.refill(lba)?; .map_err(|e| Error::IoError { source: e })?;
} self.buf_len_sectors = 0;
// Slice the buffer at the requested LBA's offset within it.
let off_sectors = (lba - self.buf_start_lba) as usize;
let off_bytes = off_sectors * SECTOR_SIZE;
out[..bytes].copy_from_slice(&self.buf[off_bytes..off_bytes + bytes]);
Ok(bytes) Ok(bytes)
} }
} }