io: phase 1 buffering — read-side flatness
Three changes targeting 0.20.9's "muxer never read-stalls on NFS read latency" invariant: A. FileSectorSource gets a 32 MiB internal read-ahead buffer (READAHEAD_BUF_BYTES). Splits out from src/sector/file.rs into src/io/file_sector_source/ with per-OS open hints (Linux posix_fadvise(SEQUENTIAL), macOS fcntl(F_RDADVISE) with 64 MiB cap, Windows TODO stub, BSD/illumos no-op). Backward seeks rebuffer; partial reads at EOF return only the bytes that exist; oversize-request bypass for count > BUF_SECTORS. B. WritebackFile inline #[cfg(target_os = "linux")] blocks split into per-OS files under src/io/writeback_file/. Linux unchanged (fallocate KEEP_SIZE, fsync via bounded_syscall). macOS gets a real F_PREALLOCATE + F_FULLFSYNC impl (was a "skipped (non-linux)" debug log before). Windows is a stub (FlushFileBuffers via std sync_all; TODO for SetFileValidData). BSDs/illumos fall back to std sync_all. C. New byte_channel module — byte-bounded producer/consumer wrapping std sync_channel with Mutex/Condvar byte accounting. Sender blocks when used_bytes + item.byte_size() > capacity. HasByteSize impl for PesFrame. Default cap BYTE_CHANNEL_DEFAULT_CAPACITY = 64 MiB, sized to absorb worst-case NFS read p99 (~2 s × UHD peak compressed ~15 MB/s). The mux call site lives in autorip (out of scope here); this lands the primitive in libfreemkv for autorip to adopt. Test counts: byte_channel +6, file_sector_source +5, sector::file round-trip suite (3) preserved. passn_handler_ab.rs A/B fixture (8 profiles) still green. precommit.sh libfreemkv: fmt + clippy + test all green on Rust 1.86. No version bump; no Cargo.lock changes; no forbidden-file edits (disc/patch.rs, disc/read_error.rs, io/pipeline.rs, tests/passn_handler_ab.rs).
This commit is contained in:
@@ -0,0 +1,66 @@
|
||||
//! Linux platform impl for [`super::WritebackFile`].
|
||||
//!
|
||||
//! - `preallocate`: `fallocate(FALLOC_FL_KEEP_SIZE)` — reserve extents
|
||||
//! without growing the reported file size. Reduces extent
|
||||
//! fragmentation on large sequential writes (mux output on NFS in
|
||||
//! particular).
|
||||
//! - `durable_sync`: `fsync` wrapped in
|
||||
//! [`crate::io::bounded::bounded_syscall`] with a 60 s deadline so a
|
||||
//! wedged NFS server can't trap the calling thread indefinitely.
|
||||
|
||||
use std::fs::File;
|
||||
use std::io;
|
||||
use std::os::unix::io::AsRawFd;
|
||||
use std::time::Duration;
|
||||
|
||||
/// Pre-reserve extents for `size_bytes` of upcoming sequential writes.
|
||||
/// Best-effort: a non-zero rc is logged but not propagated, since the
|
||||
/// caller would just continue with the unreserved file anyway.
|
||||
pub(super) fn preallocate(file: &File, size_bytes: u64) {
|
||||
// FALLOC_FL_KEEP_SIZE = 0x01 — keep the reported file size at 0
|
||||
// (writes grow it normally) while still pre-reserving the extents.
|
||||
let rc = unsafe {
|
||||
libc::fallocate(
|
||||
file.as_raw_fd(),
|
||||
libc::FALLOC_FL_KEEP_SIZE,
|
||||
0,
|
||||
size_bytes as i64,
|
||||
)
|
||||
};
|
||||
tracing::debug!(
|
||||
target: "mux",
|
||||
"WritebackFile fallocate size_hint={size_bytes} rc={rc} ok={}",
|
||||
rc == 0
|
||||
);
|
||||
}
|
||||
|
||||
/// Run `fsync` on `file` with a 60 s deadline. On timeout we log loudly
|
||||
/// and return `Ok(())` — the kernel will still flush on close, so the
|
||||
/// data is best-effort durable; the alternative (trap the thread for
|
||||
/// the rest of the rip) defeats `/api/stop`.
|
||||
pub(super) fn durable_sync(file: &File) -> io::Result<()> {
|
||||
let fd = file.as_raw_fd();
|
||||
match crate::io::bounded::bounded_syscall(
|
||||
None,
|
||||
Duration::from_secs(60),
|
||||
move || -> io::Result<()> {
|
||||
let rc = unsafe { libc::fsync(fd) };
|
||||
if rc == 0 {
|
||||
Ok(())
|
||||
} else {
|
||||
Err(io::Error::last_os_error())
|
||||
}
|
||||
},
|
||||
) {
|
||||
Ok(inner) => inner,
|
||||
Err(crate::io::bounded::BoundedError::Timeout) => {
|
||||
tracing::error!(
|
||||
target: "mux",
|
||||
"WritebackFile::sync_all fsync timed out after 60s; kernel will flush on close (best-effort)"
|
||||
);
|
||||
Ok(())
|
||||
}
|
||||
Err(crate::io::bounded::BoundedError::Halted) => Ok(()),
|
||||
Err(crate::io::bounded::BoundedError::WorkerLost) => Ok(()),
|
||||
}
|
||||
}
|
||||
Reference in New Issue
Block a user