pq/libfreemkv
1
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

0.31.0: hardening and correctness pass across mux, codec, AACS/CSS, UDF/MPLS/CLPI, recovery, drive/SCSI, labels, and I/O

Browse Source 
Library-wide review-and-fix pass: tightened AACS keydb/handshake/variant
handling and trailing-partial-unit policy, corrected MPLS mark offset and
added UDF allocation bounds, hardened the mux/codec framing and M2TS paths,
guarded SCSI READ CAPACITY short transfers and unified error mapping, added
overflow guards on untrusted disc input, and made prefetch shutdown
deterministic. Release profile now builds with thin LTO + single codegen unit.
This commit is contained in:
Matthew Jackson
2026-06-07 17:37:38 -07:00
parent 5b6ea8f5c4
commit 061f68594a
128 changed files with 11838 additions and 3831 deletions
Download Patch File Download Diff File Expand all files Collapse all files
src/io/bounded.rs
+12 -4
View File
@@ -67,10 +67,12 @@ pub(crate) enum BoundedError {
/// The deadline elapsed before the syscall returned. Same leak
/// semantics as `Halted`.
Timeout,
/// The worker thread panicked, or its sender disconnected before
/// sending a result. Treat as a benign no-op (callers usually
/// log and continue) rather than a hard error — by definition no
/// syscall observably ran to completion in this case.
/// The worker thread panicked, the OS rejected the thread spawn,
/// or its sender disconnected before sending a result. Treat as a
/// benign no-op (callers usually log and continue) rather than a
/// hard error — by definition no syscall observably ran to
/// completion in this case. In the spawn-failure case no thread is
/// leaked.
WorkerLost,
}
@@ -101,6 +103,12 @@ where
F: FnOnce() -> R + Send + 'static,
R: Send + 'static,
{
// If the caller already requested halt, don't spawn (and leak) a
// worker that would run `op` to completion in the background.
if halt.is_some_and(|h| h.is_cancelled()) {
return Err(BoundedError::Halted);
}
// Rendezvous channel: the worker sends exactly one value (the
// op's return) and then exits. Capacity-0 means the send blocks
// until we receive — fine on the happy path; on the timeout /
src/io/byte_channel.rs
-327
View File
@@ -1,327 +0,0 @@
//! Byte-sized bounded producer/consumer channel.
//!
//! Wraps `std::sync::mpsc::sync_channel` with a byte-accounting
//! `Mutex<usize> + Condvar` cap. Sender blocks (cooperatively) when
//! `used_bytes + item.byte_size() > capacity_bytes`. Receiver
//! decrements `used_bytes` when it takes the item.
//!
//! Why: the existing producer→consumer channel between `DiscStream`
//! (PES producer) and `MuxSink` (PES consumer) is bounded by frame
//! count. Frame sizes vary 100× between metadata and keyframes, so a
//! count-based cap either starves on small frames or buffers far too
//! much memory on big ones. Byte-sized accounting sizes the buffer for
//! the worst-case input stall (NFS read p99 ≈ 1–2 s × ~15 MB/s peak
//! compressed bitrate ≈ ~30 MB) directly.
//!
//! The underlying mpsc channel is created with a very large slot count
//! so the byte cap (not the slot count) is the real backpressure. Slot
//! count is only there to give the kernel a small chunk to wake on.
use std::sync::mpsc::{Receiver as MpscReceiver, RecvError, SendError, SyncSender, sync_channel};
use std::sync::{Arc, Condvar, Mutex};
/// Default byte cap for the muxer's input channel. Sized to hide a
/// worst-case ~2 s NFS read refill at UHD peak compressed bitrate
/// (~15 MB/s); 64 MiB gives headroom. Tweakable; not magic.
pub const BYTE_CHANNEL_DEFAULT_CAPACITY: usize = 64 * 1024 * 1024;
/// Slot capacity of the inner `sync_channel`. Large so the byte cap is
/// the real backpressure mechanism — the mpsc slot count only exists
/// to give the kernel a chunk to wake on. PES frames are typically
/// ~700 B each, so 64 MiB ≈ 90 k frames; 200 k is comfortable headroom.
const INNER_SLOT_CAPACITY: usize = 200_000;
/// Anything whose in-memory cost can be accounted by a single
/// `usize`. Implement on the item type sent through [`Sender`].
pub trait HasByteSize {
/// Bytes this item contributes to the channel's used budget.
/// Must be > 0 to make progress (a 0-byte item would never
/// block the sender no matter the cap; see send_blocks_at_capacity
/// test).
fn byte_size(&self) -> usize;
}
impl HasByteSize for crate::pes::PesFrame {
fn byte_size(&self) -> usize {
// Frame data + the fixed header overhead the serializer
// writes (track + pts + keyframe + len). The `Vec<u8>` heap
// allocation also has alloc-header overhead but that's
// <0.1 % at typical frame sizes — folding it in would just
// add noise to the budget.
self.data.len() + 14
}
}
/// Shared book-keeping between [`Sender`] and [`Receiver`]. Wrapped in
/// an `Arc` because both halves hold it independently.
struct Accounting {
used: Mutex<usize>,
cv: Condvar,
capacity: usize,
}
/// Send half of the byte-bounded channel.
///
/// `send` blocks (on a `Condvar`) when adding the item would push
/// `used_bytes` past `capacity_bytes`. Unblocks when the receiver
/// `recv`s items out and notifies. Returns `Err(item)` if the
/// receiver has been dropped — mirrors `mpsc::SyncSender::send`.
pub struct Sender<T: HasByteSize> {
tx: SyncSender<T>,
acct: Arc<Accounting>,
}
impl<T: HasByteSize> Clone for Sender<T> {
fn clone(&self) -> Self {
Sender {
tx: self.tx.clone(),
acct: self.acct.clone(),
}
}
}
impl<T: HasByteSize> Sender<T> {
/// Push one item. Blocks until adding it would not exceed the
/// capacity, then sends through the inner mpsc channel.
pub fn send(&self, item: T) -> Result<(), SendError<T>> {
let sz = item.byte_size();
// Reserve capacity first. The reservation is observable to
// other senders via `used`; only after we win the slot do we
// hand the item to the inner mpsc channel. That ordering means
// `used` is always a conservative upper bound on what's in the
// mpsc queue + about-to-be-sent.
{
let mut used = self.acct.used.lock().expect("byte_channel poisoned");
// An item bigger than the whole capacity will never fit; let
// it through anyway as a one-shot reservation, otherwise the
// sender deadlocks forever waiting for `used == 0` AND
// nothing in flight. The receiver will drain it on the
// other side. Same behaviour as `std::sync::mpsc` for
// arbitrarily large messages.
while *used + sz > self.acct.capacity && *used > 0 {
used = self.acct.cv.wait(used).expect("byte_channel cv poisoned");
}
*used += sz;
}
match self.tx.send(item) {
Ok(()) => Ok(()),
Err(SendError(returned)) => {
// Receiver dropped — refund the reservation so a later
// sender on a clone doesn't observe phantom used bytes
// (the receiver is gone so nobody will decrement).
let mut used = self.acct.used.lock().expect("byte_channel poisoned");
*used = used.saturating_sub(sz);
self.acct.cv.notify_all();
Err(SendError(returned))
}
}
}
}
/// Receive half of the byte-bounded channel.
///
/// `recv` blocks on the inner mpsc until an item is available, then
/// decrements the byte-accounting and wakes any sender waiting on
/// capacity.
pub struct Receiver<T: HasByteSize> {
rx: MpscReceiver<T>,
acct: Arc<Accounting>,
}
impl<T: HasByteSize> Receiver<T> {
/// Take the next item. Returns `Err(RecvError)` when all senders
/// have been dropped and the channel is empty.
pub fn recv(&self) -> Result<T, RecvError> {
let item = self.rx.recv()?;
let sz = item.byte_size();
let mut used = self.acct.used.lock().expect("byte_channel poisoned");
*used = used.saturating_sub(sz);
// Notify all so multi-sender setups wake every blocked sender,
// not just one. Wasted wakeups are cheap; missed wakeups would
// be a deadlock.
self.acct.cv.notify_all();
Ok(item)
}
}
/// Create a byte-bounded channel with the given capacity in bytes.
/// Returns a `(Sender, Receiver)` pair; clone the `Sender` for
/// multi-producer setups.
pub fn channel<T: HasByteSize>(capacity_bytes: usize) -> (Sender<T>, Receiver<T>) {
let (tx, rx) = sync_channel::<T>(INNER_SLOT_CAPACITY);
let acct = Arc::new(Accounting {
used: Mutex::new(0),
cv: Condvar::new(),
capacity: capacity_bytes,
});
(
Sender {
tx,
acct: acct.clone(),
},
Receiver { rx, acct },
)
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::thread;
use std::time::{Duration, Instant};
/// Test payload — its `byte_size` returns whatever we passed at
/// construction so capacity math is exact and predictable.
#[derive(Clone, Debug, PartialEq, Eq)]
struct Item {
sz: usize,
tag: u32,
}
impl HasByteSize for Item {
fn byte_size(&self) -> usize {
self.sz
}
}
#[test]
fn send_recv_round_trip() {
let (tx, rx) = channel::<Item>(1024);
for i in 0..5 {
tx.send(Item { sz: 100, tag: i }).unwrap();
}
for i in 0..5 {
let got = rx.recv().unwrap();
assert_eq!(got, Item { sz: 100, tag: i });
}
}
#[test]
fn byte_accounting_decrements_on_recv() {
// Internal book-keeping check via observable side-effect: after
// sending K items totalling N bytes and receiving them all, a
// subsequent send of an N-byte item must NOT block (no items
// in flight, all capacity refunded).
let (tx, rx) = channel::<Item>(1024);
for _ in 0..4 {
tx.send(Item { sz: 256, tag: 0 }).unwrap();
}
for _ in 0..4 {
rx.recv().unwrap();
}
// Cap is now fully available again. Send a 1024-byte item; the
// `used > 0` guard means it goes through alone (no wait).
let start = Instant::now();
tx.send(Item { sz: 1024, tag: 99 }).unwrap();
assert!(start.elapsed() < Duration::from_millis(100));
let got = rx.recv().unwrap();
assert_eq!(got.tag, 99);
}
#[test]
fn send_blocks_at_capacity_unblocks_on_recv() {
// Cap = 200 bytes, item = 100 bytes. First two sends fit
// exactly; the third must block until a recv frees capacity.
let (tx, rx) = channel::<Item>(200);
tx.send(Item { sz: 100, tag: 0 }).unwrap();
tx.send(Item { sz: 100, tag: 1 }).unwrap();
let tx2 = tx.clone();
let sent_at = Arc::new(Mutex::new(None::<Instant>));
let sent_at2 = sent_at.clone();
let h = thread::spawn(move || {
tx2.send(Item { sz: 100, tag: 2 }).unwrap();
*sent_at2.lock().unwrap() = Some(Instant::now());
});
// Give the sender thread a head start; it should be parked in
// `cv.wait` because used (200) + 100 > capacity (200).
thread::sleep(Duration::from_millis(100));
assert!(
sent_at.lock().unwrap().is_none(),
"third send should be blocked at capacity"
);
// Drain one. Sender wakes and completes.
let recv_at = Instant::now();
let got = rx.recv().unwrap();
assert_eq!(got.tag, 0);
h.join().unwrap();
let sent_when = sent_at.lock().unwrap().unwrap();
assert!(
sent_when >= recv_at,
"sender must complete AFTER receiver freed capacity"
);
// Drain the remaining two.
assert_eq!(rx.recv().unwrap().tag, 1);
assert_eq!(rx.recv().unwrap().tag, 2);
}
#[test]
fn item_larger_than_capacity_still_goes_through() {
// Pathological case: a single item bigger than the capacity.
// The guard `*used > 0` lets it through when the channel is
// empty (otherwise the sender deadlocks forever). Matches
// `mpsc::SyncSender` semantics for oversize messages.
let (tx, rx) = channel::<Item>(100);
tx.send(Item { sz: 1000, tag: 7 }).unwrap();
let got = rx.recv().unwrap();
assert_eq!(got, Item { sz: 1000, tag: 7 });
}
#[test]
fn concurrent_send_recv_stress() {
// 4 sender threads × 1k items each, 1 receiver. Verify byte
// accounting stays sane (channel never deadlocks, every item
// arrives exactly once) under contention.
const SENDERS: u32 = 4;
const PER_SENDER: u32 = 1000;
const TOTAL: u32 = SENDERS * PER_SENDER;
let (tx, rx) = channel::<Item>(8 * 1024);
let sent = Arc::new(AtomicUsize::new(0));
let mut handles = Vec::new();
for s in 0..SENDERS {
let tx = tx.clone();
let sent = sent.clone();
handles.push(thread::spawn(move || {
for i in 0..PER_SENDER {
// Vary item size so accounting actually has to
// multiplex differently-sized blockers. 1B → 256B.
let sz = 1 + ((i as usize) % 256);
tx.send(Item {
sz,
tag: s * PER_SENDER + i,
})
.unwrap();
sent.fetch_add(1, Ordering::SeqCst);
}
}));
}
// Drop our local sender so the receiver can eventually see
// RecvError once all sender clones are done. Cloning the
// sender into each producer means each clone Drop'd separately.
drop(tx);
let mut received = 0u32;
while let Ok(_item) = rx.recv() {
received += 1;
}
for h in handles {
h.join().unwrap();
}
assert_eq!(received, TOTAL);
assert_eq!(sent.load(Ordering::SeqCst) as u32, TOTAL);
}
#[test]
fn send_after_recv_dropped_returns_err() {
let (tx, rx) = channel::<Item>(1024);
drop(rx);
let r = tx.send(Item { sz: 10, tag: 0 });
assert!(r.is_err());
}
}
src/io/byte_prefetcher.rs
+153 -22
View File
@@ -10,11 +10,12 @@
//!
//! This is the byte-stream half of the freemkv mux highway —
//! `BytePrefetcher` feeds [`crate::mux::demux_thread::DemuxThread`]
//! for `m2ts://`, `network://`, `stdio://`, and any other stream
//! whose source is an `io::Read` rather than a `SectorSource`.
//! for `m2ts://` (the only in-tree caller today, via
//! [`crate::mux::resolve`]), and works for any stream whose source is
//! an `io::Read` rather than a `SectorSource`.
use crate::halt::Halt;
use crossbeam_channel::{Receiver, Sender, bounded};
use crate::halt::{Halt, POLL_INTERVAL};
use crossbeam_channel::{Receiver, RecvTimeoutError, SendTimeoutError, Sender, bounded};
use std::io::Read;
use std::thread::JoinHandle;
@@ -38,6 +39,13 @@ pub const DEFAULT_CHUNK_BYTES: usize = 16 * 1024 * 1024;
/// Returned from [`BytePrefetcher::into_channels`]. Owns the
/// producer-thread join handle so dropping the shell joins the
/// producer.
///
/// Drop blocks the calling thread until the producer exits. To
/// guarantee a prompt exit, drop the forward receiver and the recycle
/// sender first so the producer observes channel disconnection (or
/// cancel the [`Halt`] passed to [`BytePrefetcher::new`], which the
/// producer polls at [`POLL_INTERVAL`] granularity even while parked
/// on a channel op).
pub struct PrefetchShell {
producer: Option<JoinHandle<()>>,
}
@@ -66,7 +74,13 @@ impl BytePrefetcher {
mut reader: R,
chunk_bytes: usize,
halt: Option<Halt>,
) -> Self {
) -> std::io::Result<Self> {
// A zero-length chunk makes every recycled buffer an empty
// slice; `reader.read(&mut [])` returns Ok(0), which the loop
// below treats as EOF — the consumer would see a clean,
// silent zero-byte stream. Callers pass the downstream
// demuxer's batch size, which is always > 0.
debug_assert!(chunk_bytes > 0, "BytePrefetcher chunk_bytes must be > 0");
let (tx, rx) = bounded::<Batch>(FORWARD_DEPTH);
let (recycle_tx, recycle_rx) = bounded::<Vec<u8>>(RECYCLE_DEPTH);
@@ -80,16 +94,32 @@ impl BytePrefetcher {
let producer = std::thread::Builder::new()
.name("freemkv-byte-prefetch".into())
.spawn(move || {
let cancelled = || halt.as_ref().map(|h| h.is_cancelled()).unwrap_or(false);
loop {
if halt.as_ref().map(|h| h.is_cancelled()).unwrap_or(false) {
if cancelled() {
return;
}
let mut buf = match recycle_rx.recv() {
Ok(b) => b,
Err(_) => return, // consumer dropped both channels
// Park on the recycle channel, but re-poll halt
// every POLL_INTERVAL: a pure-AtomicBool Halt does
// not disconnect the channel, so a blocking recv()
// would never re-reach the cancel check.
let mut buf = loop {
match recycle_rx.recv_timeout(POLL_INTERVAL) {
Ok(b) => break b,
Err(RecvTimeoutError::Timeout) => {
if cancelled() {
return;
}
}
// Consumer dropped both channels.
Err(RecvTimeoutError::Disconnected) => return,
}
};
// Re-expose the full extent (previous iteration
// may have truncated after a short read).
// Re-expose the full extent. After a short read the
// prior iteration truncated to n < chunk_bytes, so
// this regrows the length back to chunk_bytes
// without reallocating (capacity was fixed at
// construction and never shrinks).
if buf.len() < chunk_bytes {
buf.resize(chunk_bytes, 0);
} else {
@@ -109,18 +139,31 @@ impl BytePrefetcher {
}
};
buf.truncate(n);
if tx.send(Ok(buf)).is_err() {
return; // consumer dropped
// Hand off the filled buffer, re-polling halt on
// each timeout slice so a cancel can interrupt a
// producer parked on a saturated forward channel.
let mut pending = Ok(buf);
loop {
match tx.send_timeout(pending, POLL_INTERVAL) {
Ok(()) => break,
Err(SendTimeoutError::Timeout(returned)) => {
if cancelled() {
return;
}
pending = returned;
}
// Consumer dropped.
Err(SendTimeoutError::Disconnected(_)) => return,
}
}
}
})
.expect("freemkv-byte-prefetch thread spawn failed");
})?;
Self {
Ok(Self {
rx,
recycle_tx,
producer: Some(producer),
}
})
}
/// Peel off the channels for zero-copy pipeline consumption. The
@@ -128,11 +171,30 @@ impl BytePrefetcher {
/// drains `rx`, runs the demuxer in place on each filled buffer,
/// and recycles back through `recycle_tx`.
pub fn into_channels(self) -> (Receiver<Batch>, Sender<Vec<u8>>, PrefetchShell) {
let mut me = self;
let producer = me.producer.take();
let rx = me.rx.clone();
let recycle = me.recycle_tx.clone();
std::mem::forget(me);
// MOVE the three fields out cleanly — never clone. Each of
// `rx` and `recycle_tx` ends up with exactly ONE live copy:
// the one in the returned tuple. The pre-1.0.0 implementation
// cloned both and then `mem::forget`-ed `self`, leaking the
// originals so an extra live receiver + sender survived
// forever. That defeated the channel-disconnection shutdown:
// when the demux consumer exited early (halt, or a `tx.send`
// error in `demux_thread`), the producer's `recycle_rx.recv()`
// and `tx.send()` never saw all-peers-dropped, so the producer
// never returned and `PrefetchShell::drop`'s `join()` hung.
//
// `ManuallyDrop` + `ptr::read` reads each field out by value
// and suppresses `self`'s own `Drop` (which would otherwise
// double-`join`), leaving NO extra live endpoint behind. This
// is the panic-free equivalent of the `Option::take` approach
// and mirrors `sector::prefetched::into_channels`.
let me = std::mem::ManuallyDrop::new(self);
// SAFETY: `me` is `ManuallyDrop`, so none of these fields will
// be dropped by `me`. Each `ptr::read` performs exactly one
// bitwise move out; every field is read exactly once and never
// touched again, so there are no double-frees and no aliasing.
let producer = unsafe { std::ptr::read(&me.producer) };
let rx = unsafe { std::ptr::read(&me.rx) };
let recycle = unsafe { std::ptr::read(&me.recycle_tx) };
(rx, recycle, PrefetchShell { producer })
}
}
@@ -144,3 +206,72 @@ impl Drop for BytePrefetcher {
}
}
}
#[cfg(test)]
mod tests {
use super::*;
/// Endless reader: every `read` fills the whole buffer and never
/// hits EOF, so the producer keeps trying to push batches forward
/// until the forward channel disconnects. Exactly the shape that
/// wedged the pre-1.0.0 `clone + mem::forget` `into_channels`.
struct EndlessReader;
impl Read for EndlessReader {
fn read(&mut self, buf: &mut [u8]) -> std::io::Result<usize> {
buf.fill(0);
Ok(buf.len())
}
}
/// Run `f` on a helper thread and fail if it does not finish within
/// `secs`. Turns a join-deadlock into a test failure instead of a
/// hung CI run.
fn within<F: FnOnce() + Send + 'static>(secs: u64, f: F) {
let (done_tx, done_rx) = bounded::<()>(1);
std::thread::spawn(move || {
f();
let _ = done_tx.send(());
});
assert!(
done_rx
.recv_timeout(std::time::Duration::from_secs(secs))
.is_ok(),
"operation did not complete within {secs}s (deadlock)"
);
}
/// The CRITICAL regression: after `into_channels`, dropping the
/// returned forward receiver + recycle sender must let the producer
/// observe disconnection and exit, so dropping the `PrefetchShell`
/// (which joins the producer) returns promptly. With the old
/// clone+forget the leaked endpoints kept the producer blocked and
/// this join hung forever.
#[test]
fn into_channels_drop_releases_producer() {
within(10, || {
// Small chunk so the producer cycles quickly and fills the
// forward channel without allocating much.
let pf = BytePrefetcher::new(EndlessReader, 4096, None).expect("spawn");
let (rx, recycle_tx, shell) = pf.into_channels();
// Consumer goes away early (halt / abort analogue): drop
// both channel endpoints without draining to EOF.
drop(rx);
drop(recycle_tx);
// Joining the producer must not hang.
drop(shell);
});
}
/// Same property via the halt path: cancel the token, then the
/// producer must exit and the shell join must complete.
#[test]
fn halt_releases_producer() {
within(10, || {
let halt = Halt::new();
let pf = BytePrefetcher::new(EndlessReader, 4096, Some(halt.clone())).expect("spawn");
let (_rx, _recycle_tx, shell) = pf.into_channels();
halt.cancel();
drop(shell);
});
}
}
src/io/file_sector_source/macos.rs
+7 -16
View File
@@ -8,32 +8,22 @@
use std::fs::File;
use std::os::unix::io::AsRawFd;
/// `F_RDADVISE` opcode — not in libc's named constants on all SDKs.
const F_RDADVISE: libc::c_int = 44;
/// Cap on the byte length we pass to `F_RDADVISE`. Asking for a
/// multi-GB readahead window is counterproductive — the OS doesn't
/// have that much cache to throw at one fd. 64 MiB is generous for
/// our use case (sweep, mux) and matches the byte-channel cap so the
/// kernel's prefetch ≥ our app-level pipeline depth.
/// our use case (sweep, mux) so the kernel's prefetch ≥ our app-level
/// pipeline depth.
const RDADVISE_MAX_BYTES: i64 = 64 * 1024 * 1024;
/// `radvisory` per `<sys/fcntl.h>`. repr(C) layout is stable.
#[repr(C)]
struct RadAdvisory {
ra_offset: libc::off_t,
ra_count: libc::c_int,
}
pub(super) fn hint_sequential(file: &File, len_bytes: u64) {
let bytes = (len_bytes as i64).min(RDADVISE_MAX_BYTES);
let mut ra = RadAdvisory {
let mut ra = libc::radvisory {
ra_offset: 0,
ra_count: bytes as libc::c_int,
};
// Best-effort.
unsafe {
libc::fcntl(file.as_raw_fd(), F_RDADVISE, &mut ra);
libc::fcntl(file.as_raw_fd(), libc::F_RDADVISE, &mut ra);
}
}
@@ -52,11 +42,12 @@ pub(super) fn drop_window(_file: &File, _start: u64, _len: u64) {}
/// returns immediately.
pub(super) fn prefetch(file: &File, offset: u64, len: u64) {
let bytes = (len as i64).min(RDADVISE_MAX_BYTES);
let mut ra = RadAdvisory {
let mut ra = libc::radvisory {
ra_offset: offset as libc::off_t,
ra_count: bytes as libc::c_int,
};
// Best-effort — kernel hint only.
unsafe {
libc::fcntl(file.as_raw_fd(), F_RDADVISE, &mut ra);
libc::fcntl(file.as_raw_fd(), libc::F_RDADVISE, &mut ra);
}
}
src/io/file_sector_source/mod.rs
+23 -10
View File
@@ -17,9 +17,17 @@
//! Without page-cache eviction an 85 GB streaming ISO read pins the
//! entire file in memory, starves the concurrent writer, and collapses
//! mux throughput (observed: 2.7 MB/s mux on 0.21.5 vs. 70 MB/s
//! isolated NFS reads). Every [`READ_DROP_CHUNK_BYTES`] of consumed
//! bytes we call `posix_fadvise(DONTNEED)` over that window, mirroring
//! the write-side [`crate::io::writeback::WritebackPipeline`] policy.
//! isolated NFS reads). Every [`READ_DROP_CHUNK_BYTES_DEFAULT`] of
//! consumed bytes we call `posix_fadvise(DONTNEED)` over that window,
//! mirroring the write-side [`crate::io::writeback::WritebackPipeline`]
//! policy.
//!
//! The drop window is accounted by a monotonic forward byte counter,
//! which matches the sequential streaming pattern the mux highway
//! drives. Under random or backward access the dropped range no longer
//! lines up with the bytes actually read — but `DONTNEED` is purely an
//! advisory cache hint with no correctness impact, so this degrades to
//! a slightly imprecise hint rather than a bug.
//!
//! ## Platform open hint
//!
@@ -102,7 +110,10 @@ pub struct FileSectorSource {
bytes_read_since_drop: u64,
/// File offset at which the current drop window starts. The next
/// DONTNEED drops from `drop_window_start` for
/// `bytes_read_since_drop` bytes.
/// `bytes_read_since_drop` bytes. This advances monotonically with
/// the byte count, so it tracks the actual reads only under the
/// forward-sequential access the mux highway uses; under random
/// access it degrades to a harmless, imprecise advisory hint.
drop_window_start: u64,
/// Cached drop chunk size (resolved from env once at open).
drop_chunk_bytes: u64,
@@ -116,16 +127,18 @@ impl FileSectorSource {
///
/// Issues the platform's "sequential access expected" hint on the
/// fd (Linux `posix_fadvise(SEQUENTIAL)`, macOS `fcntl(F_RDADVISE)`,
/// Windows TODO stub) so the kernel's readahead widens.
pub fn open(path: &Path) -> std::io::Result<Self> {
let file = File::open(path)?;
let len = file.metadata()?.len();
/// Windows no-op) so the kernel's readahead widens.
pub fn open(path: &Path) -> Result<Self> {
let file = File::open(path).map_err(|e| Error::IoError { source: e })?;
let len = file
.metadata()
.map_err(|e| Error::IoError { source: e })?
.len();
let sectors = len / SECTOR_SIZE as u64;
if sectors > u32::MAX as u64 {
return Err(Error::IsoTooLarge {
path: path.to_string_lossy().into_owned(),
}
.into());
});
}
let capacity = sectors as u32;
src/io/file_sector_source/windows.rs
+8 -10
View File
@@ -1,20 +1,18 @@
//! Windows: the canonical sequential-access hint is
//! `FILE_FLAG_SEQUENTIAL_SCAN` passed to `CreateFile` at open time —
//! it cannot be set after the fact via `SetFileInformationByHandle`.
//! Routing the open call through this module would mean a custom
//! `File::from_raw_handle` plumb for every `FileSectorSource::open`
//! caller, which is more invasive than the Phase 1 scope.
//!
//! TODO: replumb `FileSectorSource::open` to take an
//! `OpenOptions`-style builder so the Windows path can flip the flag
//! at open time. For now this is a no-op stub.
//! `FILE_FLAG_SEQUENTIAL_SCAN`, which must be passed to `CreateFile`
//! at open time and cannot be set afterward via
//! `SetFileInformationByHandle`. Since `FileSectorSource::open` uses a
//! plain `File::open`, the hints in this module are no-op stubs.
use std::fs::File;
/// No-op stub. `FILE_FLAG_SEQUENTIAL_SCAN` can only be set at
/// `CreateFile` open time, which the plain `File::open` path does not
/// do, so there is no post-open hint to issue here.
pub(super) fn hint_sequential(_file: &File, _len_bytes: u64) {
tracing::debug!(
target: "mux",
"FileSectorSource hint_sequential: windows stub (TODO: FILE_FLAG_SEQUENTIAL_SCAN at open)"
"FileSectorSource hint_sequential: windows no-op stub"
);
}
src/io/mod.rs
+5 -6
View File
@@ -18,15 +18,16 @@
//! consumed window so an 85 GB streaming ISO read doesn't fill the
//! page cache and starve the concurrent MKV write.
//!
//! `Pipeline` + `Sink` (0.18) is the generic producer/consumer primitive
//! `Pipeline` + `Sink` is the generic producer/consumer primitive
//! used by sweep, patch, and mux to overlap reads with writes via a
//! bounded channel + dedicated consumer thread.
//!
//! `byte_channel` is a byte-sized producer/consumer channel for the
//! mux pipeline, sized to absorb worst-case input read stalls.
//! `byte_prefetcher` is the read-ahead producer feeding the mux
//! pipeline for `io::Read`-backed sources: a worker thread fills a
//! recycled pool of buffers and ships them through a channel, exposing
//! `BytePrefetcher` / `PrefetchShell`.
pub(crate) mod bounded;
pub mod byte_channel;
pub mod byte_prefetcher;
pub mod file_sector_source;
pub mod sink;
@@ -40,8 +41,6 @@ pub mod pipeline;
pub(crate) use writeback_file::WritebackFile;
// Re-exports for the 0.18 redesign. Sweep, patch, and mux are all
// wired up (disc/sweep.rs, disc/patch.rs, autorip's ripper/mux.rs).
pub use pipeline::{
DEFAULT_PIPELINE_DEPTH, Flow, Pipeline, READ_PIPELINE_DEPTH, Sink, WRITE_PIPELINE_DEPTH,
WRITE_THROUGH_DEPTH,
src/io/pipeline.rs
+126 -130
View File
@@ -5,10 +5,9 @@
//! The consumer's behaviour is supplied by a [`Sink`] implementation:
//! `apply` is called once per item, `close` is called once at the end.
//!
//! Three call sites in libfreemkv want a producer/consumer split —
//! sweep (migrated to `disc/sweep.rs::SweepSink`), patch, and mux.
//! 0.18 collapses all three onto this primitive; sweep is in,
//! patch and mux migrate in later 0.18 slices.
//! Call sites in libfreemkv that want a producer/consumer split —
//! sweep (`disc/sweep.rs::SweepSink`) and the file-backed mux highway
//! — are built on this primitive.
//!
//! ## Cancellation and error semantics
//!
@@ -24,7 +23,8 @@
//! blocks on a dead receiver, and the first error is propagated as
//! the `JoinHandle` result.
//! - Consumer panic is converted into
//! `Error::IoError { source: io::Error::other(...) }`.
//! [`Error::PipelineConsumerPanicked`] (the panic message is logged,
//! not embedded in the error value).
//!
//! ## Debug logging
//!
@@ -32,7 +32,6 @@
//! logging throughout the pipeline (channel sends/receives, backpressure,
//! consumer lag detection). This is critical for diagnosing stalls.
use std::io;
use std::thread::{self, JoinHandle};
use std::time::{Duration, Instant};
@@ -41,15 +40,14 @@ use crossbeam_channel::{Sender, TrySendError, bounded};
use crate::error::Error;
use crate::halt::Halt;
/// Deadline for [`Pipeline::finish_with_halt`]'s polling join. Chosen
/// to be comfortably longer than the autorip hard watchdog
/// (`HARD_WATCHDOG_STALL_SECS = 300s`) so the watchdog's `exit(1)`
/// fires first when both are racing on the same wedged consumer.
/// Deadline for [`Pipeline::finish_with_halt`]'s polling join.
///
/// 10 minutes is a backstop, not a normal timeout — the consumer is
/// expected to drain in seconds. If we hit this, something is wedged
/// inside a kernel call the consumer thread can't unwind from, and the
/// caller has already lost the rip.
/// inside a kernel call the consumer thread can't unwind from. It is
/// deliberately long so a consuming application's own (shorter) stall
/// watchdog gets the first chance to escalate; this join only fires
/// when no such watchdog intervenes.
pub const JOIN_TIMEOUT_SECS: u64 = 600;
/// Halt-check cadence for the send loop. Producer blocks on
@@ -72,11 +70,40 @@ use crate::halt::POLL_INTERVAL;
const SEND_HALT_CHECK_INTERVAL: Duration = POLL_INTERVAL;
/// Check if verbose debug logging is enabled via FREEMKV_DEBUG env var.
///
/// The value cannot change mid-run, and this is called multiple times
/// per item on the mux highway hot loop, so the env lookup (a String
/// allocation behind the global env lock) is cached after the first
/// call.
pub fn debug_enabled() -> bool {
std::env::var("FREEMKV_DEBUG")
.ok()
.map(|v| v == "1" || v == "true" || v == "yes")
.unwrap_or(false)
static ENABLED: std::sync::OnceLock<bool> = std::sync::OnceLock::new();
*ENABLED.get_or_init(|| {
std::env::var("FREEMKV_DEBUG")
.ok()
.map(|v| v == "1" || v == "true" || v == "yes")
.unwrap_or(false)
})
}
/// Turn a consumer-thread panic payload into the numeric
/// [`Error::PipelineConsumerPanicked`] variant. The original panic
/// message (the two stdlib formats `panic!` produces: `&str` /
/// `String`) is logged at the join site for diagnostics — it is NOT
/// baked into the error value, since the library carries no English
/// text in its errors. Callers discriminate on the variant.
fn consumer_panicked(payload: Box<dyn std::any::Any + Send>) -> Error {
let msg = payload
.downcast_ref::<&'static str>()
.copied()
.or_else(|| payload.downcast_ref::<String>().map(|s| s.as_str()))
.unwrap_or("(no message)");
tracing::error!(
target: "freemkv::pipeline",
phase = "consumer_panicked",
panic_message = msg,
"pipeline consumer thread panicked"
);
Error::PipelineConsumerPanicked
}
/// Default channel depth for callers without a specific reason to
@@ -104,11 +131,12 @@ pub const WRITE_THROUGH_DEPTH: usize = 1;
/// ([`Flow::Continue`]), or stop the pipeline early and run `close()`
/// ([`Flow::Stop`]).
///
/// `Stop` has no in-tree caller in this slice — sweep never returns
/// it (it always processes the producer's full work-list before the
/// channel is dropped). Patch and mux are the intended consumers and
/// migrate in later 0.18 slices. The variant ships now so the contract
/// is fixed; the targeted `#[allow]` is removed when patch lands.
/// `Stop` currently has no in-tree caller — sweep never returns it (it
/// always processes the producer's full work-list before the channel
/// is dropped), and the mux highway drains to EOF. The variant is part
/// of the fixed `Sink` contract for early-stop consumers, so the
/// `#[allow(dead_code)]` is intentional and permanent until such a
/// consumer lands.
pub enum Flow {
Continue,
#[allow(dead_code)]
@@ -184,47 +212,54 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
let mut stopped = false;
while let Ok(item) = rx.recv() {
if debug_enabled() {
let debug = debug_enabled();
if debug {
tracing::debug!("Pipeline receive: item={}", std::any::type_name::<I>());
}
let apply_start = std::time::Instant::now();
if first_err.is_some() || stopped {
// Drain remaining items so the producer never
// blocks on a dead receiver. `apply` is not
// called once we've decided to stop.
continue;
}
// Only pay for the timestamp when debug tracing is
// on — this runs per item on the mux highway hot
// path.
let apply_start = debug.then(Instant::now);
match sink.apply(item) {
Ok(Flow::Continue) => {}
Ok(Flow::Stop) => {
stopped = true;
if debug_enabled() {
if debug {
tracing::debug!("Pipeline: consumer returned Flow::Stop");
}
}
Err(e) => {
if debug_enabled() {
if debug {
tracing::debug!("Pipeline: apply error, stopping, err={:?}", e);
}
first_err = Some(e);
}
}
let apply_elapsed = apply_start.elapsed();
if debug_enabled() && apply_elapsed > std::time::Duration::from_millis(100) {
tracing::debug!(
"Pipeline apply: took {:.2}s, item={}",
apply_elapsed.as_secs_f64(),
std::any::type_name::<I>()
);
} else if debug_enabled() {
tracing::debug!(
"Pipeline apply: OK in {:.3}ms, item={}",
apply_elapsed.as_micros(),
std::any::type_name::<I>()
);
if let Some(start) = apply_start {
let apply_elapsed = start.elapsed();
if apply_elapsed > Duration::from_millis(100) {
tracing::debug!(
"Pipeline apply: took {:.2}s, item={}",
apply_elapsed.as_secs_f64(),
std::any::type_name::<I>()
);
} else {
tracing::debug!(
"Pipeline apply: OK in {:.3}ms, item={}",
apply_elapsed.as_micros(),
std::any::type_name::<I>()
);
}
}
}
@@ -250,31 +285,37 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
/// independent signal (e.g. `Halt`) — `send` alone is not the
/// notification edge.
pub fn send(&self, item: I) -> Result<(), I> {
let start = std::time::Instant::now();
// Only timestamp when debug tracing is on — `send` runs per
// item on the mux highway hot path.
let start = debug_enabled().then(Instant::now);
match self.tx.send(item) {
Ok(()) => {
let elapsed = start.elapsed();
if debug_enabled() && elapsed > std::time::Duration::from_millis(10) {
tracing::debug!(
"Pipeline send: blocked {:.2}s, item={}",
elapsed.as_secs_f64(),
std::any::type_name::<I>()
);
} else if debug_enabled() {
tracing::debug!("Pipeline send: OK in {:.3}ms", elapsed.as_micros());
if let Some(start) = start {
let elapsed = start.elapsed();
if elapsed > Duration::from_millis(10) {
tracing::debug!(
"Pipeline send: blocked {:.2}s, item={}",
elapsed.as_secs_f64(),
std::any::type_name::<I>()
);
} else {
tracing::debug!("Pipeline send: OK in {:.3}ms", elapsed.as_micros());
}
}
Ok(())
}
Err(e) => {
let elapsed = start.elapsed();
if debug_enabled() && elapsed > std::time::Duration::from_millis(10) {
tracing::debug!(
"Pipeline send: blocked {:.2}s before channel closed, item={}",
elapsed.as_secs_f64(),
std::any::type_name::<I>()
);
} else if debug_enabled() {
tracing::debug!("Pipeline send: failed after {:.3}ms", elapsed.as_micros());
if let Some(start) = start {
let elapsed = start.elapsed();
if elapsed > Duration::from_millis(10) {
tracing::debug!(
"Pipeline send: blocked {:.2}s before channel closed, item={}",
elapsed.as_secs_f64(),
std::any::type_name::<I>()
);
} else {
tracing::debug!("Pipeline send: failed after {:.3}ms", elapsed.as_micros());
}
}
Err(e.0)
}
@@ -367,9 +408,9 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
/// Drop the producer-side channel and wait for the consumer
/// thread to finish. Returns whatever the consumer's `close()`
/// produced, or the first `apply` error, or — on consumer panic —
/// an `Error::IoError` whose source is `io::Error::other(...)`
/// with a "pipeline consumer panicked: <payload>" message
/// (callers can match on the constant prefix).
/// [`Error::PipelineConsumerPanicked`]. The panic payload is
/// logged at the join site (the library carries no English in its
/// error values), so callers discriminate on the variant.
pub fn finish(self) -> Result<R, Error> {
let Pipeline { tx, handle } = self;
// Explicit drop, although the destructure already drops `tx`
@@ -377,20 +418,7 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
drop(tx);
match handle.join() {
Ok(result) => result,
Err(payload) => {
// Preserve the original panic message when the
// consumer's panic payload was a `&str` or `String`
// (the two stdlib formats that `panic!` produces).
// Anything else falls back to "(no message)".
let msg = payload
.downcast_ref::<&'static str>()
.copied()
.or_else(|| payload.downcast_ref::<String>().map(|s| s.as_str()))
.unwrap_or("(no message)");
Err(Error::IoError {
source: io::Error::other(format!("pipeline consumer panicked: {msg}")),
})
}
Err(payload) => Err(consumer_panicked(payload)),
}
}
@@ -402,19 +430,18 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
/// [`JOIN_TIMEOUT_SECS`] deadline. Returns:
///
/// - `Ok(R)` on a clean consumer exit.
/// - `Err(Error::IoError)` with one of three message prefixes for
/// wedge cases:
/// - `"pipeline join halted"` — halt fired while waiting.
/// - `"pipeline join timed out"` — `JOIN_TIMEOUT_SECS` elapsed.
/// - `"pipeline consumer panicked"` — same as `finish()`.
/// - One of three numeric error variants for the wedge cases:
/// - [`Error::Halted`] — halt fired while waiting.
/// - [`Error::PipelineJoinTimeout`] — `JOIN_TIMEOUT_SECS` elapsed.
/// - [`Error::PipelineConsumerPanicked`] — same as `finish()`.
///
/// In the `halted` and `timed out` branches the consumer thread is
/// intentionally leaked — exactly the same trade-off the
/// `bounded_syscall` primitive makes. The wedged kernel call
/// inside the consumer will unwind whenever it does, or at
/// process exit. The caller is free to fall back to a degraded
/// path (in autorip's case: `exit(1)` after the hard watchdog
/// escalation, letting Docker restart the container).
/// path (e.g. abort the session and let a supervisor restart the
/// process).
///
/// Plain [`Pipeline::finish`] is preserved for callers without a
/// halt-token plumbed through; that path still blocks indefinitely
@@ -427,31 +454,18 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
if handle.is_finished() {
return match handle.join() {
Ok(result) => result,
Err(payload) => {
let msg = payload
.downcast_ref::<&'static str>()
.copied()
.or_else(|| payload.downcast_ref::<String>().map(|s| s.as_str()))
.unwrap_or("(no message)");
Err(Error::IoError {
source: io::Error::other(format!("pipeline consumer panicked: {msg}")),
})
}
Err(payload) => Err(consumer_panicked(payload)),
};
}
if let Some(h) = halt {
if h.is_cancelled() {
// Consumer thread is intentionally leaked.
return Err(Error::IoError {
source: io::Error::other("pipeline join halted"),
});
return Err(Error::Halted);
}
}
if Instant::now() >= deadline {
// Consumer thread is intentionally leaked.
return Err(Error::IoError {
source: io::Error::other("pipeline join timed out"),
});
return Err(Error::PipelineJoinTimeout);
}
thread::sleep(POLL_INTERVAL);
}
@@ -700,25 +714,13 @@ mod tests {
std::panic::set_hook(prev);
match res {
Err(Error::IoError { source }) => {
let msg = source.to_string();
// Constant prefix lets callers match without parsing
// the variable payload tail.
assert!(
msg.contains("pipeline consumer panicked"),
"expected constant panic prefix, got: {msg}"
);
// The original `panic!` payload (a `&'static str`) must
// be preserved — without the downcast the message
// would just be the prefix.
assert!(
msg.contains("synthetic test panic"),
"expected original panic payload, got: {msg}"
);
}
other => panic!("expected Err(IoError), got {other:?}"),
}
// A consumer panic surfaces as the numeric variant, not an
// English-carrying io::Error. The original panic payload is
// logged at the join site, not embedded in the error value.
assert!(
matches!(res, Err(Error::PipelineConsumerPanicked)),
"expected Err(PipelineConsumerPanicked), got {res:?}"
);
}
/// Never-completing sink — `apply` blocks until cancelled. Signals
@@ -810,8 +812,8 @@ mod tests {
#[test]
fn send_with_halt_returns_item_on_halt() {
// Same setup, but the halt fires before the deadline elapses.
// The send loop must observe the halt within ~50 ms (the
// SEND_POLL_INTERVAL) and return the item.
// The send loop must observe the halt within ~250 ms (the
// SEND_HALT_CHECK_INTERVAL) and return the item.
let cancel = Arc::new(std::sync::atomic::AtomicBool::new(false));
let started = Arc::new(std::sync::atomic::AtomicBool::new(false));
let pipe = Pipeline::spawn(
@@ -850,8 +852,7 @@ mod tests {
#[test]
fn finish_with_halt_returns_halted_when_consumer_wedged() {
// Consumer wedges on the first apply; halt fires; finish
// returns the documented "pipeline join halted" error rather
// than blocking forever.
// returns Error::Halted rather than blocking forever.
let cancel = Arc::new(std::sync::atomic::AtomicBool::new(false));
let started = Arc::new(std::sync::atomic::AtomicBool::new(false));
let pipe = Pipeline::spawn(
@@ -879,15 +880,10 @@ mod tests {
// Release the leaked consumer so the test process exits cleanly.
cancel.store(true, Ordering::SeqCst);
match res {
Err(Error::IoError { source }) => {
assert!(
source.to_string().contains("pipeline join halted"),
"expected halt-prefix error, got: {source}"
);
}
other => panic!("expected Err(IoError) halted, got {other:?}"),
}
assert!(
matches!(res, Err(Error::Halted)),
"expected Err(Halted), got {res:?}"
);
// Bailed out within ~1 second of the halt firing (worst case
// one POLL_INTERVAL = 250 ms of slack).
assert!(
src/io/sink/local_file.rs
+20 -3
View File
@@ -12,14 +12,18 @@
//! size patch, Cues index, segment header backpatch) to land on the
//! right offset.
//!
//! `RandomAccessSink` is satisfied via the blanket impl in
//! [`super::mod`]; no explicit impl needed here.
//! [`SequentialSink`](super::SequentialSink) is implemented explicitly
//! (not via a blanket impl) so its `finish()` flushes the `BufWriter`
//! and `fsync`s the file even when called through a `dyn` trait object;
//! [`RandomAccessSink`](super::RandomAccessSink) is implemented over the
//! `Seek` impl below.
use std::fs::{File, OpenOptions};
use std::io::{self, BufWriter, Seek, SeekFrom, Write};
use std::path::Path;
use super::preallocate;
use super::{RandomAccessSink, SequentialSink};
const BUFFER_BYTES: usize = 4 * 1024 * 1024;
@@ -80,13 +84,26 @@ impl LocalFileSink {
/// Drain the internal buffer and `fsync` the underlying file.
/// Idempotent with `Drop` (the `BufWriter` also flushes on drop;
/// this call additionally surfaces fsync errors to the caller).
#[allow(dead_code)] // exposed for parity with WritebackFile::sync_all
/// [`SequentialSink::finish`](super::SequentialSink::finish)
/// delegates here so the durable flush happens through a trait
/// object too.
pub fn sync_all(&mut self) -> io::Result<()> {
self.inner.flush()?;
self.inner.get_ref().sync_all()
}
}
impl SequentialSink for LocalFileSink {
/// Flush the 4 MiB `BufWriter` and `fsync` the file. Overriding the
/// trait default is what makes a `dyn SequentialSink` / `dyn
/// RandomAccessSink` `finish()` actually durable instead of a no-op.
fn finish(&mut self) -> io::Result<()> {
self.sync_all()
}
}
impl RandomAccessSink for LocalFileSink {}
impl Write for LocalFileSink {
fn write(&mut self, buf: &[u8]) -> io::Result<usize> {
self.inner.write(buf)
src/io/sink/mod.rs
+52 -43
View File
@@ -35,14 +35,22 @@ pub use socket::{SocketSink, UdpSocketSink};
/// trait does not impose or hide any buffering of its own.
///
/// `finish` drains any internal buffering and signals end-of-stream to
/// the underlying transport (close-write on a socket, flush on a
/// buffered writer, etc.). The default impl is a no-op; concrete
/// implementations that need explicit shutdown can override it but the
/// blanket impl below keeps it optional for adapter types like
/// `&mut File`.
/// the underlying transport (close-write on a socket, flush + fsync on
/// a buffered file, etc.). The default impl flushes via [`Write::flush`]
/// — correct for an unbuffered destination — but every concrete sink in
/// this module overrides it to drain its own buffer and run its
/// transport-specific finalisation (socket `shutdown(Write)`, file
/// `fsync`). There is deliberately NO blanket `impl SequentialSink for
/// T`: a blanket impl would force the no-op-style default on every
/// concrete sink (a blanket impl cannot be overridden per-type without a
/// coherence conflict), so a `Box<dyn SequentialSink>` / `&mut dyn
/// SequentialSink` `finish()` call would silently skip the flush and
/// transport shutdown. With explicit per-type impls the vtable dispatches
/// `finish` to the real implementation, so flush + durable-finish
/// actually happen through a trait object.
pub trait SequentialSink: Write + Send {
fn finish(&mut self) -> std::io::Result<()> {
Ok(())
self.flush()
}
}
@@ -51,15 +59,6 @@ pub trait SequentialSink: Write + Send {
/// random-access sink is always usable as a sequential sink.
pub trait RandomAccessSink: SequentialSink + Seek {}
// Blanket impls so any `Write + Send` type acts as a `SequentialSink`
// (with default `finish`), and any sink that also impls `Seek` is
// automatically a `RandomAccessSink`. Keeps call-site ergonomics simple
// — `&mut File`, `LocalFileSink`, `WritebackFile`, `BufWriter<File>`,
// and `Cursor<Vec<u8>>` all satisfy the right trait without per-type
// boilerplate.
impl<T: Write + Send + ?Sized> SequentialSink for T {}
impl<T: SequentialSink + Seek + ?Sized> RandomAccessSink for T {}
/// Pick the right `RandomAccessSink` impl for `dest` based on its
/// filesystem type.
///
@@ -82,13 +81,9 @@ pub fn open_for_mkv(
dest: &std::path::Path,
size_hint: Option<u64>,
) -> std::io::Result<Box<dyn RandomAccessSink>> {
#[cfg(not(target_os = "linux"))]
use crate::platform::fs_type::detect;
#[cfg(target_os = "linux")]
use crate::platform::fs_type::{FsType, detect};
#[cfg(target_os = "linux")]
{
use crate::platform::fs_type::{FsType, detect};
if detect(dest) == FsType::Nfs {
let wf = match size_hint {
Some(n) => crate::io::WritebackFile::create_with_size_hint(dest, n)?,
@@ -97,12 +92,13 @@ pub fn open_for_mkv(
return Ok(Box::new(wf));
}
}
// Silence the unused-binding warning on non-Linux where the only
// branch above is cfg-gated out.
// Only Linux differentiates the sink by filesystem type (NFS gets
// the WritebackFile machinery); every other OS always uses
// `LocalFileSink`. Reference `detect` as a value (no call, no
// `statfs` syscall) so it isn't flagged dead on non-Linux while
// still avoiding the wasted probe whose result we'd discard.
#[cfg(not(target_os = "linux"))]
{
let _ = detect(dest);
}
let _ = crate::platform::fs_type::detect;
let sink = match size_hint {
Some(n) => LocalFileSink::with_size_hint(dest, n)?,
@@ -114,32 +110,25 @@ pub fn open_for_mkv(
#[cfg(test)]
mod tests {
use super::*;
use std::fs::File;
// Type-level assertion: the blanket impls cover the shapes we care
// about. These functions never run; they just have to type-check.
fn _assert_file_is_sequential(_: &mut dyn SequentialSink) {}
fn _assert_file_is_random_access(_: &mut dyn RandomAccessSink) {}
// Type-level assertion: the concrete sinks satisfy the trait
// objects. These functions never run; they just have to type-check.
fn _assert_is_sequential(_: &mut dyn SequentialSink) {}
fn _assert_is_random_access(_: &mut dyn RandomAccessSink) {}
#[test]
fn blanket_impls_cover_file_and_localfilesink() {
// `File` directly via blanket impls.
fn concrete_sinks_satisfy_traits() {
let dir = tempfile::tempdir().unwrap();
let mut f = File::create(dir.path().join("a.bin")).unwrap();
_assert_file_is_sequential(&mut f);
_assert_file_is_random_access(&mut f);
// `LocalFileSink` ditto.
// `LocalFileSink` is a random-access (and thus sequential) sink.
let mut s = LocalFileSink::create(&dir.path().join("b.bin")).unwrap();
_assert_file_is_sequential(&mut s);
_assert_file_is_random_access(&mut s);
_assert_is_sequential(&mut s);
_assert_is_random_access(&mut s);
// `WritebackFile` — confirms the Phase 1 type still satisfies
// the trait via the blanket impl without needing an explicit
// `impl RandomAccessSink for WritebackFile {}`.
// `WritebackFile` ditto, via its explicit per-type impls.
let mut wf = crate::io::WritebackFile::create(&dir.path().join("c.bin")).unwrap();
_assert_file_is_sequential(&mut wf);
_assert_file_is_random_access(&mut wf);
_assert_is_sequential(&mut wf);
_assert_is_random_access(&mut wf);
}
#[test]
@@ -155,4 +144,24 @@ mod tests {
let bytes = std::fs::read(&p).unwrap();
assert_eq!(&bytes[..5], b"hello");
}
/// finish() through a `dyn SequentialSink` trait object must
/// dispatch to the concrete sink's override (flush + fsync), not a
/// no-op default. This is the regression test for the silent-no-op
/// finish() bug.
#[test]
fn finish_through_trait_object_flushes_local_file() {
let dir = tempfile::tempdir().unwrap();
let p = dir.path().join("trait-finish.bin");
let sink = LocalFileSink::create(&p).unwrap();
// Box as the trait object the production path uses.
let mut boxed: Box<dyn SequentialSink> = Box::new(sink);
boxed.write_all(b"buffered-tail").unwrap();
// finish() through the vtable must drain the 4 MiB BufWriter and
// fsync; the bytes must be visible to a separate reader BEFORE
// we drop the sink (drop-flush must not be what saves us).
boxed.finish().unwrap();
let bytes = std::fs::read(&p).unwrap();
assert_eq!(&bytes[..], b"buffered-tail");
}
}
src/io/sink/preallocate/linux.rs
+5 -2
View File
@@ -5,13 +5,16 @@
//! the file naturally.
use std::fs::File;
#[cfg(unix)]
use std::os::unix::io::AsRawFd;
pub(super) fn preallocate_impl(file: &File, size_bytes: u64) {
let fd = file.as_raw_fd();
// Clamp to the signed `off_t` range fallocate expects; an unchecked
// `as i64` cast would wrap a >= 2^63 size to a negative length that
// fallocate rejects with EINVAL (silent no-op).
let len = i64::try_from(size_bytes).unwrap_or(i64::MAX);
// FALLOC_FL_KEEP_SIZE = 0x01.
let rc = unsafe { libc::fallocate(fd, libc::FALLOC_FL_KEEP_SIZE, 0, size_bytes as i64) };
let rc = unsafe { libc::fallocate(fd, libc::FALLOC_FL_KEEP_SIZE, 0, len) };
tracing::debug!(
target: "mux",
"LocalFileSink fallocate size_hint={size_bytes} rc={rc} ok={}",
src/io/sink/preallocate/macos.rs
+14 -5
View File
@@ -1,7 +1,8 @@
//! macOS `F_PREALLOCATE` extent reservation.
//!
//! `fcntl(F_PREALLOCATE)` with `F_ALLOCATECONTIG` first (try for a
//! contiguous run) and fall back to `F_ALLOCATEALL` (non-contig OK).
//! `fcntl(F_PREALLOCATE)` with `F_ALLOCATECONTIG | F_ALLOCATEALL` first
//! (prefer a contiguous run but accept scattered extents to satisfy the
//! full length) and fall back to `F_ALLOCATEALL` alone on failure.
//! Reported file size is unchanged — the muxer's writes still grow it.
use std::fs::File;
@@ -13,16 +14,24 @@ use crate::io::platform_macos::{
pub(super) fn preallocate_impl(file: &File, size_bytes: u64) {
let fd = file.as_raw_fd();
// Clamp to the signed `off_t` range; an unchecked `as off_t` cast
// would wrap a >= 2^63 size to a negative length.
let len = i64::try_from(size_bytes).unwrap_or(i64::MAX) as libc::off_t;
let mut store = Fstore {
fst_flags: F_ALLOCATECONTIG,
// Prefer a contiguous run but accept scattered extents to
// satisfy the full length. Without F_ALLOCATEALL the first
// attempt is best-effort and can return rc=0 with a partial
// allocation, so the fallback below would never fire. Matches
// writeback_file/macos.rs.
fst_flags: F_ALLOCATECONTIG | F_ALLOCATEALL,
fst_posmode: F_PEOFPOSMODE,
fst_offset: 0,
fst_length: size_bytes as libc::off_t,
fst_length: len,
fst_bytesalloc: 0,
};
let mut rc = unsafe { libc::fcntl(fd, F_PREALLOCATE, &mut store as *mut Fstore) };
if rc == -1 {
// Fall back to non-contiguous.
// Fall back to non-contiguous only.
store.fst_flags = F_ALLOCATEALL;
rc = unsafe { libc::fcntl(fd, F_PREALLOCATE, &mut store as *mut Fstore) };
}
src/io/sink/socket.rs
+41 -25
View File
@@ -11,16 +11,21 @@
//! conventional choice). `finish()` is a no-op; UDP has no end-of-stream
//! marker.
//!
//! Both types satisfy [`SequentialSink`] via the blanket impl in
//! `super::mod`. Neither implements `Seek`, so neither satisfies
//! [`RandomAccessSink`] — using one with `MkvMux` is a compile error,
//! which is the design intent.
//! Both types implement [`SequentialSink`] explicitly so their
//! `finish()` dispatches correctly through a `dyn SequentialSink` trait
//! object (the `SocketSink` override drains the buffer and
//! `shutdown(Write)`s; the `UdpSocketSink` override flushes only).
//! Neither implements `Seek`, so neither satisfies [`RandomAccessSink`]
//! — using one with `MkvMux` is a compile error, which is the design
//! intent.
//!
//! [`SequentialSink`]: super::SequentialSink
//! [`RandomAccessSink`]: super::RandomAccessSink
use std::io::{self, BufWriter, Write};
use std::net::{Shutdown, TcpStream, ToSocketAddrs, UdpSocket};
use std::net::{Shutdown, SocketAddr, TcpStream, ToSocketAddrs, UdpSocket};
use super::SequentialSink;
/// `BufWriter` capacity for [`SocketSink`]. 1 MiB matches the typical
/// kernel send-buffer ceiling and keeps small-write amplification from
@@ -54,8 +59,10 @@ impl SocketSink {
// `set_nodelay(true)` keeps small writes (TS packet trains, fMP4
// moof headers) from sitting in Nagle's algorithm until the buffer
// fills. The BufWriter already absorbs syscall overhead; Nagle
// would just add latency without coalescing more.
stream.set_nodelay(true)?;
// would just add latency without coalescing more. It is a latency
// hint, not a correctness requirement, so a platform that rejects
// TCP_NODELAY must not fail the connect — demote the error.
let _ = stream.set_nodelay(true);
if let Some(n) = sndbuf_bytes {
set_send_buffer(&stream, n)?;
}
@@ -76,17 +83,12 @@ impl Write for SocketSink {
}
}
impl SocketSink {
impl SequentialSink for SocketSink {
/// Drain the BufWriter and `shutdown(Write)` the underlying socket
/// so the peer sees a clean EOF.
///
/// Note: [`SequentialSink::finish`](super::SequentialSink::finish)'s
/// blanket-impl default is a no-op. Trait-object call sites that
/// need socket shutdown should call this inherent method directly
/// before dropping the sink, or hold the concrete `SocketSink` type
/// (typical pattern: each muxer's `finish()` calls the appropriate
/// inherent close method on its captured concrete sink).
pub fn finish(&mut self) -> io::Result<()> {
/// so the peer sees a clean EOF. Overriding the trait default is
/// what makes a `dyn SequentialSink` `finish()` send the buffered
/// tail and the EOF instead of silently dropping them.
fn finish(&mut self) -> io::Result<()> {
self.buf.flush()?;
// `shutdown(Write)` signals clean EOF to the peer. Errors here
// are non-fatal — the connection may have already been torn down
@@ -117,10 +119,22 @@ impl UdpSocketSink {
///
/// `sndbuf_bytes`, when present, is a hint to `SO_SNDBUF`.
pub fn connect<A: ToSocketAddrs>(peer: A, sndbuf_bytes: Option<usize>) -> io::Result<Self> {
// Bind to all-zeros / any port. The kernel picks an ephemeral
// source port and the source IP at first send.
let socket = UdpSocket::bind("0.0.0.0:0")?;
socket.connect(peer)?;
// Resolve the peer first so the local bind matches its address
// family. Binding `0.0.0.0:0` (IPv4) and then connecting to an
// IPv6 peer fails with EAFNOSUPPORT, so pick the wildcard that
// matches the resolved family.
let peer_addr = peer
.to_socket_addrs()?
.next()
.ok_or_else(|| io::Error::from(io::ErrorKind::AddrNotAvailable))?;
let bind_addr = match peer_addr {
SocketAddr::V4(_) => "0.0.0.0:0",
SocketAddr::V6(_) => "[::]:0",
};
// Bind to the matching wildcard / any port. The kernel picks an
// ephemeral source port and the source IP at first send.
let socket = UdpSocket::bind(bind_addr)?;
socket.connect(peer_addr)?;
if let Some(n) = sndbuf_bytes {
set_udp_send_buffer(&socket, n)?;
}
@@ -140,10 +154,12 @@ impl Write for UdpSocketSink {
}
}
impl UdpSocketSink {
/// No-op — UDP has no end-of-stream marker. Provided for parity
/// with [`SocketSink::finish`] so call sites can treat them uniformly.
pub fn finish(&mut self) -> io::Result<()> {
impl SequentialSink for UdpSocketSink {
/// UDP has no end-of-stream marker, so there is nothing to shut
/// down; `write` already sent each datagram unbuffered. Flush is a
/// no-op but kept explicit so the trait-object `finish()` matches
/// the concrete behaviour.
fn finish(&mut self) -> io::Result<()> {
Ok(())
}
}
src/io/writeback/linux.rs
+34 -21
View File
@@ -54,7 +54,6 @@
use std::collections::VecDeque;
use std::fs::File;
use std::os::unix::io::{AsRawFd, RawFd};
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use std::time::{Duration, Instant};
@@ -94,13 +93,11 @@ pub(crate) struct WritebackPipeline {
is_nfs: bool,
/// Set the first time WAIT_AFTER exceeds [`WAIT_AFTER_TIMEOUT`].
/// Once set, behaviour matches the NFS path for the rest of the
/// pipeline's life. Wrapped in `Arc` only because both this
/// struct and the spawned worker thread (which itself doesn't
/// touch the flag) share-via-fd patterns might one day need it;
/// today it's effectively a single-owner cell — the `Arc` shape
/// keeps the door open for moving the read side into a worker
/// without re-plumbing types.
degraded: Arc<AtomicBool>,
/// pipeline's life. A plain `AtomicBool`: the flag is only ever
/// touched on the owning thread (the spawned WAIT_AFTER worker never
/// reads or writes it). `AtomicBool` over `bool` only because the
/// load/store sites read cleanly; no sharing is needed today.
degraded: AtomicBool,
}
impl WritebackPipeline {
@@ -124,7 +121,7 @@ impl WritebackPipeline {
wait_after_window: VecDeque::with_capacity(ADAPTIVE_WINDOW),
chunk_count: 0,
is_nfs,
degraded: Arc::new(AtomicBool::new(false)),
degraded: AtomicBool::new(false),
}
}
@@ -143,8 +140,12 @@ impl WritebackPipeline {
if pos < self.last_flush_pos.saturating_add(self.chunk_bytes) {
return;
}
let chunk_off = self.last_flush_pos as i64;
let chunk_len = (pos - self.last_flush_pos) as i64;
// Byte offsets are unsigned throughout; the signed cast happens
// only at the libc call boundary where the kernel ABI requires
// `i64`. `saturating_sub` documents and hardens the line-above
// guard that `pos >= last_flush_pos`.
let chunk_off: u64 = self.last_flush_pos;
let chunk_len: u64 = pos.saturating_sub(self.last_flush_pos);
let mut wait_ms: u64 = 0;
let mut fadvise_ms: u64 = 0;
// Async kickoff for the just-completed chunk runs on every
@@ -152,7 +153,12 @@ impl WritebackPipeline {
// by spec and gives the kernel an early hint that this range
// is ready to flush.
unsafe {
libc::sync_file_range(self.fd, chunk_off, chunk_len, libc::SYNC_FILE_RANGE_WRITE);
libc::sync_file_range(
self.fd,
chunk_off as i64,
chunk_len as i64,
libc::SYNC_FILE_RANGE_WRITE,
);
}
if let Some((prev_off, prev_len)) = self.pending.take() {
if self.skip_wait() {
@@ -198,12 +204,12 @@ impl WritebackPipeline {
}
}
}
self.pending = Some((chunk_off as u64, chunk_len as u64));
self.pending = Some((chunk_off, chunk_len));
self.last_flush_pos = pos;
self.chunk_count += 1;
tracing::trace!(
target: "mux",
"WritebackPipeline chunk off={} len={} sync_file_range_ms={wait_ms} fadvise_ms={fadvise_ms} chunk_bytes={} skip_wait={}",
"WritebackPipeline chunk off={} len={} wait_after_ms={wait_ms} fadvise_ms={fadvise_ms} chunk_bytes={} skip_wait={}",
chunk_off,
chunk_len,
self.chunk_bytes,
@@ -231,10 +237,14 @@ impl WritebackPipeline {
if self.wait_after_window.len() < ADAPTIVE_WINDOW {
return;
}
// p95 of 16 samples ≈ sorted[14] (5 % of 16 = 0.8 ≈ 1 above).
// p95 index, derived from the window size so it stays valid if
// ADAPTIVE_WINDOW changes (a hard-coded `[14]` would panic OOB
// for a window <= 14). For the default 16 this is index 15
// (ceil(16 * 95 / 100) - 1 = 15), i.e. the top sample.
let mut sorted: Vec<u64> = self.wait_after_window.iter().copied().collect();
sorted.sort_unstable();
let p95 = sorted[14];
let p95_idx = (ADAPTIVE_WINDOW * 95).div_ceil(100).min(ADAPTIVE_WINDOW) - 1;
let p95 = sorted[p95_idx];
let old = self.chunk_bytes;
let new = if p95 > ADAPTIVE_GROW_MS && self.chunk_bytes < CHUNK_BYTES_MAX {
(self.chunk_bytes * 2).min(CHUNK_BYTES_MAX)
@@ -321,11 +331,14 @@ fn detect_nfs(fd: RawFd) -> bool {
/// Run `sync_file_range(WAIT_AFTER)` on a worker thread and wait up
/// to [`WAIT_AFTER_TIMEOUT`] for it to return. `Some(elapsed_ms)` on
/// success; `None` on timeout. On timeout the worker thread is
/// 0.20.6 generalizes the worker-thread + recv_timeout pattern into
/// [`crate::io::bounded::bounded_syscall`]; this helper now just adapts
/// the generic primitive to the WAIT_AFTER call shape (returns elapsed_ms
/// instead of the syscall's `()` return, treats `WorkerLost` as a benign
/// no-op to match the original semantics).
/// intentionally leaked — it unwinds whenever the syscall eventually
/// returns or the process exits.
///
/// This delegates to [`crate::io::bounded::bounded_syscall`], the
/// generic worker-thread + `recv_timeout` primitive, and just adapts it
/// to the WAIT_AFTER call shape: it returns `elapsed_ms` instead of the
/// syscall's `()`, and treats `WorkerLost` as a benign no-op to match
/// the original semantics.
fn wait_after_with_timeout(fd: RawFd, off: u64, len: u64) -> Option<u64> {
let started = Instant::now();
match crate::io::bounded::bounded_syscall(None, WAIT_AFTER_TIMEOUT, move || unsafe {
src/io/writeback_file/linux.rs
+26 -14
View File
@@ -19,14 +19,10 @@ use std::time::Duration;
pub(super) fn preallocate(file: &File, size_bytes: u64) {
// FALLOC_FL_KEEP_SIZE = 0x01 — keep the reported file size at 0
// (writes grow it normally) while still pre-reserving the extents.
let rc = unsafe {
libc::fallocate(
file.as_raw_fd(),
libc::FALLOC_FL_KEEP_SIZE,
0,
size_bytes as i64,
)
};
// Clamp to the signed `off_t` range; an unchecked `as i64` cast
// would wrap a >= 2^63 size to a negative length (EINVAL no-op).
let len = i64::try_from(size_bytes).unwrap_or(i64::MAX);
let rc = unsafe { libc::fallocate(file.as_raw_fd(), libc::FALLOC_FL_KEEP_SIZE, 0, len) };
tracing::debug!(
target: "mux",
"WritebackFile fallocate size_hint={size_bytes} rc={rc} ok={}",
@@ -34,10 +30,14 @@ pub(super) fn preallocate(file: &File, size_bytes: u64) {
);
}
/// Run `fsync` on `file` with a 60 s deadline. On timeout we log loudly
/// and return `Ok(())` — the kernel will still flush on close, so the
/// data is best-effort durable; the alternative (trap the thread for
/// the rest of the rip) defeats `/api/stop`.
/// Run `fsync` on `file` with a 60 s deadline. On timeout — and
/// likewise on halt or a lost worker — we log and return `Ok(())`: the
/// kernel will still flush on close, so the data is best-effort durable.
/// The alternative (trap the thread for the rest of the rip, or return
/// an error that aborts an otherwise-complete mux) is worse, so all
/// three fallbacks return `Ok(())`. `Ok(())` from these paths is NOT a
/// durability barrier — the durable flush did not complete; only the
/// hang is bounded.
pub(super) fn durable_sync(file: &File) -> io::Result<()> {
let fd = file.as_raw_fd();
match crate::io::bounded::bounded_syscall(
@@ -60,7 +60,19 @@ pub(super) fn durable_sync(file: &File) -> io::Result<()> {
);
Ok(())
}
Err(crate::io::bounded::BoundedError::Halted) => Ok(()),
Err(crate::io::bounded::BoundedError::WorkerLost) => Ok(()),
Err(crate::io::bounded::BoundedError::Halted) => {
tracing::warn!(
target: "mux",
"WritebackFile::sync_all fsync skipped (halt requested); data not durably flushed, kernel will flush on close"
);
Ok(())
}
Err(crate::io::bounded::BoundedError::WorkerLost) => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all fsync worker lost before completion; data not durably flushed, kernel will flush on close"
);
Ok(())
}
}
}
src/io/writeback_file/macos.rs
+10 -5
View File
@@ -1,10 +1,12 @@
//! macOS platform impl for [`super::WritebackFile`].
//!
//! - `preallocate`: `fcntl(F_PREALLOCATE)` — macOS's fallocate-equiv.
//! Reserves a contiguous extent when possible, falling back to a
//! non-contiguous reservation if the FS can't satisfy it. Reported
//! file size is unchanged (`F_ALLOCATEALL` is not set, so allocation
//! is "best effort up to length"; growth happens via writes).
//! First attempt requests `F_ALLOCATECONTIG | F_ALLOCATEALL` (prefer a
//! contiguous run but accept scattered extents to satisfy the full
//! length), falling back to `F_ALLOCATEALL` alone on failure.
//! `F_PREALLOCATE` never advances EOF regardless of the flags — only
//! `ftruncate`/writes grow the file — so the reported file size is
//! unchanged; `F_ALLOCATEALL` governs the contiguity fallback, not size.
//! - `durable_sync`: `fcntl(F_FULLFSYNC)` wrapped in
//! [`crate::io::bounded::bounded_syscall`] with a 60 s deadline.
//! F_FULLFSYNC is HFS+/APFS's true-fsync (flushes the disk's own
@@ -25,11 +27,14 @@ use crate::io::platform_macos::{
const F_FULLFSYNC: libc::c_int = 51;
pub(super) fn preallocate(file: &File, size_bytes: u64) {
// Clamp to the signed `off_t` range; an unchecked `as off_t` cast
// would wrap a >= 2^63 size to a negative length.
let len = i64::try_from(size_bytes).unwrap_or(i64::MAX) as libc::off_t;
let mut fst = Fstore {
fst_flags: F_ALLOCATECONTIG | F_ALLOCATEALL,
fst_posmode: F_PEOFPOSMODE,
fst_offset: 0,
fst_length: size_bytes as libc::off_t,
fst_length: len,
fst_bytesalloc: 0,
};
// First attempt: contiguous.
src/io/writeback_file/mod.rs
+68 -22
View File
@@ -24,29 +24,32 @@
//! ## Platform split
//!
//! The platform-specific pieces of this wrapper — extent preallocation
//! (Linux `fallocate(KEEP_SIZE)`, macOS `F_PREALLOCATE`, Windows
//! `SetFileValidData`) and the durable-flush primitive (Linux/macOS
//! `fsync`/`F_FULLFSYNC` wrapped in a bounded syscall, Windows
//! `FlushFileBuffers`) — live in per-OS sibling modules. The dispatch
//! happens once at the bottom of this file via cfg-gated `mod` decls.
//! No inline `#[cfg(target_os = "...")]` in the business-logic above.
//! (Linux `fallocate(KEEP_SIZE)`, macOS `F_PREALLOCATE`, Windows no-op
//! today) and the durable-flush primitive (Linux/macOS
//! `fsync`/`F_FULLFSYNC` wrapped in a bounded syscall; Windows plain
//! `FlushFileBuffers`, unbounded) — live in per-OS sibling modules. The
//! dispatch happens once at the bottom of this file via cfg-gated `mod`
//! decls. No inline `#[cfg(target_os = "...")]` in the business-logic
//! above.
//!
//! ## Write path
//!
//! Writes are direct passthrough to the underlying `File` (no writer
//! thread, no ring, no batching). Empirically the Phase-2.5
//! writer-thread architecture introduced a ~60% mux throughput
//! regression on NFS bidirectional workloads; reverting the write path
//! to direct passthrough restores the 0.20.7 baseline. The writeback
//! pipeline still runs (it's called inline from `write` / `write_all` /
//! `seek`) so the bounded-cache invariant on Linux is preserved.
//! thread, no ring, no batching). Empirically a writer-thread
//! architecture introduced a ~60% mux throughput regression on NFS
//! bidirectional workloads; the direct-passthrough write path is faster.
//! The writeback pipeline still runs (it's called inline from `write` /
//! `write_all` / `seek`) so the bounded-cache invariant on Linux is
//! preserved.
//!
//! ## Halt-safety
//!
//! `sync_all` runs the per-OS durable-flush primitive, which on
//! Linux/macOS is wrapped in [`crate::io::bounded::bounded_syscall`]
//! with a 60 s deadline. A wedged NFS server cannot trap the muxer
//! indefinitely on the final fsync.
//! `sync_all` runs the per-OS durable-flush primitive. On Linux/macOS
//! it is wrapped in [`crate::io::bounded::bounded_syscall`] with a 60 s
//! deadline, so a wedged NFS server cannot trap the muxer indefinitely
//! on the final fsync. Windows is a known deviation: its `durable_sync`
//! calls `File::sync_all` (`FlushFileBuffers`) directly and is NOT
//! bounded — a wedged UNC/SMB share can block the final flush there.
#[cfg(target_os = "linux")]
mod linux;
@@ -73,18 +76,23 @@ use std::path::Path;
use super::writeback::WritebackPipeline;
/// Granularity at which the Linux writeback pipeline issues
/// `sync_file_range` pairs. 32 MiB is the empirically best value on
/// the rip1 test bed (NFS to unraid-1 over 1 GbE, single-disk SAS):
/// 8 MiB / 64 MiB / 128 MiB all measured worse in the 0.21.x mux
/// iteration runs. Override via `FREEMKV_WRITEBACK_CHUNK_MIB` —
/// faster backends (NVMe, RAID) may tolerate larger windows.
/// `sync_file_range` pairs. 32 MiB is the empirically best value on a
/// 1 GbE NFS mount backed by a single spinning disk: 8 MiB / 64 MiB /
/// 128 MiB all measured worse. Override via `FREEMKV_WRITEBACK_CHUNK_MIB`
/// — faster backends (NVMe, RAID) may tolerate larger windows.
const WRITEBACK_CHUNK_BYTES_DEFAULT: u64 = 32 * 1024 * 1024;
/// Upper bound (in MiB) accepted from `FREEMKV_WRITEBACK_CHUNK_MIB`.
/// 64 GiB — far above `CHUNK_BYTES_MAX` (256 MiB), generous for any
/// real backend, and small enough that `n * 1024 * 1024` cannot wrap
/// `u64`. Out-of-range values fall back to the default.
const WRITEBACK_CHUNK_MIB_MAX: u64 = 64 * 1024;
fn writeback_chunk_bytes() -> u64 {
std::env::var("FREEMKV_WRITEBACK_CHUNK_MIB")
.ok()
.and_then(|v| v.parse::<u64>().ok())
.filter(|&n| n > 0)
.filter(|&n| n > 0 && n <= WRITEBACK_CHUNK_MIB_MAX)
.map(|n| n * 1024 * 1024)
.unwrap_or(WRITEBACK_CHUNK_BYTES_DEFAULT)
}
@@ -160,6 +168,13 @@ impl WritebackFile {
/// trap the calling thread indefinitely. On timeout the page cache
/// is left to the kernel's normal flush-on-close path — best
/// effort, but bounded.
///
/// IMPORTANT: on Linux/macOS a successful `Ok(())` does NOT
/// guarantee the data is durable if the bounded fsync timed out or
/// was halted — only the hang is bounded, the fsync may not have
/// completed. Callers needing crash-consistency (e.g. mux-finish
/// then external commit/DB update) must not treat `Ok(())` as a
/// durability barrier.
pub(crate) fn sync_all(&mut self) -> io::Result<()> {
self.pipeline.finalize();
platform::durable_sync(&self.file)
@@ -215,6 +230,21 @@ impl Seek for WritebackFile {
}
}
impl super::sink::SequentialSink for WritebackFile {
/// Drain the writeback pipeline and run the bounded durable flush —
/// the same work [`Self::sync_all`] does. Implemented explicitly (no
/// blanket impl) so a `dyn SequentialSink` / `dyn RandomAccessSink`
/// `finish()` actually finalises + fsyncs instead of hitting a no-op
/// default. Note the bounded-fsync caveat from [`Self::sync_all`]
/// applies: `Ok(())` is not a durability barrier if the fsync timed
/// out or was halted.
fn finish(&mut self) -> io::Result<()> {
self.sync_all()
}
}
impl super::sink::RandomAccessSink for WritebackFile {}
impl Drop for WritebackFile {
fn drop(&mut self) {
// Run the pipeline's tail finalize so the last in-flight chunk
@@ -312,4 +342,20 @@ mod tests {
drop(w);
assert_eq!(read_back(&p), b"onetwothree");
}
/// finish() through a `dyn RandomAccessSink` trait object must
/// dispatch to WritebackFile's override (finalize + durable_sync),
/// not a no-op default. Bytes must be visible to a separate reader
/// before drop.
#[test]
fn finish_through_trait_object_persists() {
use crate::io::sink::RandomAccessSink;
let dir = tempfile::tempdir().unwrap();
let p = dir.path().join("finish-dyn.bin");
let w = WritebackFile::create(&p).unwrap();
let mut boxed: Box<dyn RandomAccessSink> = Box::new(w);
boxed.write_all(b"durable-tail").unwrap();
boxed.finish().unwrap();
assert_eq!(read_back(&p), b"durable-tail");
}
}
src/io/writeback_file/windows.rs
+15 -16
View File
@@ -1,16 +1,18 @@
//! Windows platform impl for [`super::WritebackFile`].
//!
//! TODO: this stub matches the design's "validate without a Windows
//! build env, leave a stub" carve-out. The real impl should use:
//! Current behaviour:
//!
//! - `SetEndOfFile` + `SetFileValidData` for extent preallocation
//! (caller needs `SE_MANAGE_VOLUME_NAME` privilege; if unavailable
//! fall back to a write-zero path or just skip).
//! - `FlushFileBuffers` for fsync-equivalent durable flush.
//!
//! Until then: preallocate is a debug-logged no-op; durable_sync calls
//! the std `File::sync_all` (which on Windows maps to
//! `FlushFileBuffers` internally).
//! - `preallocate` is a debug-logged no-op. Windows has no
//! `fallocate`-equivalent that keeps the reported size, so extent
//! reservation is not wired up.
//! - `durable_sync` delegates to the std `File::sync_all`, which on
//! Windows maps to `FlushFileBuffers`. Unlike the Linux/macOS impls
//! this is NOT wrapped in the bounded-syscall primitive (that would
//! need an `unsafe impl Send` for `RawHandle`, which cannot be
//! validated without a Windows test env), so a wedged UNC/SMB share
//! can block the final flush. This deviation is documented on
//! [`super::WritebackFile::sync_all`] and the parent module's
//! Halt-safety section.
use std::fs::File;
use std::io;
@@ -18,15 +20,12 @@ use std::io;
pub(super) fn preallocate(_file: &File, size_bytes: u64) {
tracing::debug!(
target: "mux",
"WritebackFile preallocate size_hint={size_bytes} skipped (windows stub; TODO: SetFileValidData)"
"WritebackFile preallocate size_hint={size_bytes} skipped (no-op on windows)"
);
}
pub(super) fn durable_sync(file: &File) -> io::Result<()> {
// `File::sync_all` on Windows is `FlushFileBuffers`. Acceptable
// for now; the bounded-syscall wrapper is not used here because
// the stub also skips the worker-thread + leak machinery (the
// wrapper would need an `unsafe impl Send` for `RawHandle`, and
// designing that without a Windows test env is asking for it).
// `File::sync_all` on Windows is `FlushFileBuffers`. Not wrapped in
// the bounded-syscall primitive (see the module doc) — unbounded.
file.sync_all()
}