Compare commits

...
6 Commits
Author SHA1 Message Date
MattJackson 5495332f07 v0.20.9: bump version 2026-05-13 19:52:48 -07:00
MattJackson e22fc6fd47 io: phase 1 buffering — read-side flatness
Three changes targeting 0.20.9's "muxer never read-stalls on NFS read
latency" invariant:

A. FileSectorSource gets a 32 MiB internal read-ahead buffer
   (READAHEAD_BUF_BYTES). Splits out from src/sector/file.rs into
   src/io/file_sector_source/ with per-OS open hints (Linux
   posix_fadvise(SEQUENTIAL), macOS fcntl(F_RDADVISE) with 64 MiB
   cap, Windows TODO stub, BSD/illumos no-op). Backward seeks
   rebuffer; partial reads at EOF return only the bytes that exist;
   oversize-request bypass for count > BUF_SECTORS.

B. WritebackFile inline #[cfg(target_os = "linux")] blocks split
   into per-OS files under src/io/writeback_file/. Linux unchanged
   (fallocate KEEP_SIZE, fsync via bounded_syscall). macOS gets a
   real F_PREALLOCATE + F_FULLFSYNC impl (was a "skipped (non-linux)"
   debug log before). Windows is a stub (FlushFileBuffers via
   std sync_all; TODO for SetFileValidData). BSDs/illumos fall back
   to std sync_all.

C. New byte_channel module — byte-bounded producer/consumer wrapping
   std sync_channel with Mutex/Condvar byte accounting. Sender blocks
   when used_bytes + item.byte_size() > capacity. HasByteSize impl
   for PesFrame. Default cap BYTE_CHANNEL_DEFAULT_CAPACITY = 64 MiB,
   sized to absorb worst-case NFS read p99 (~2 s × UHD peak compressed
   ~15 MB/s). The mux call site lives in autorip (out of scope here);
   this lands the primitive in libfreemkv for autorip to adopt.

Test counts: byte_channel +6, file_sector_source +5, sector::file
round-trip suite (3) preserved. passn_handler_ab.rs A/B fixture
(8 profiles) still green.

precommit.sh libfreemkv: fmt + clippy + test all green on Rust 1.86.

No version bump; no Cargo.lock changes; no forbidden-file edits
(disc/patch.rs, disc/read_error.rs, io/pipeline.rs,
tests/passn_handler_ab.rs).
2026-05-13 19:48:23 -07:00
MattJackson 5b98c13e47 v0.20.8: bump version 2026-05-13 19:18:51 -07:00
MattJackson 8f8f1a62a2 io+disc: bundle 0.20.8 dev work
- io/pipeline.rs: add send_with_halt + finish_with_halt for cooperative
  halt during blocking producer-consumer handoffs; 5 new tests
- disc/patch.rs: split Disc::patch body (1168 -> 316 LOC) into named
  helpers (compute_initial_state, prime_cache, check_range_watchdog,
  handle_skip_limit, compute_damage_skip, handle_read_success,
  handle_read_failure, report_patch_progress, build_outcome) with
  PatchLoopState / RangeFrame structs; references shared
  PATCH_DAMAGE_THRESHOLD_PCT constant
- disc/read_error.rs: add pub const PATCH_DAMAGE_THRESHOLD_PCT = 6;
  ReadCtx::for_patch() now references the shared constant (was a
  latent 12 / 6 inconsistency)
- tests/passn_handler_ab.rs: 8-profile A/B fixture locking current
  patch-side recovery behavior (clean / all-medium / alternating /
  edge-bad-good-middle / single-bad / deep-pit / medium-then-good /
  batch-fail). Goldens captured pre-unification; will catch any
  future refactor that breaks the size-aware skip cap.
2026-05-13 19:15:48 -07:00
MattJackson e3cfd27942 v0.20.7: version bump for unified release (no source changes)
The 0.20.7 work lives downstream in autorip — process-level safety net
(hard watchdog 5-min mux escalation + std::process::exit, restart-loop
counter + auto-quarantine after 3 attempts, partial-state preservation,
failure_reason surfaced in /api/state).

Bumping libfreemkv to 0.20.7 keeps the 4 crates on the unified release
train per project docs.
2026-05-13 14:29:40 -07:00
MattJackson aa12bdad62 v0.20.6: io::bounded — halt-safe wrapper for blocking syscalls
Generalizes 0.20.5's hand-written wait_after_with_timeout into a
reusable primitive. After this change, every blocking syscall in the
recovery + mux paths is wrapped, so cooperative Halt has bounded
~250 ms latency reach even into kernel-owned thread states.

New module src/io/bounded.rs:
- BoundedError { Halted, Timeout, WorkerLost }
- bounded_syscall<F, R>(halt: Option<&Halt>, timeout, op) -> Result<R, BoundedError>
- Worker thread runs op; main thread recv_timeouts on a rendezvous
  channel in 250 ms slices, polling halt between slices.
- Worker is intentionally leaked on timeout/halt — kernel reaps when
  the syscall finally returns or at process exit. Calling thread is
  NEVER trapped inside a kernel call.
- 6 unit tests cover the happy path + each error variant.

Refactored callsites:
- src/io/writeback/linux.rs::wait_after_with_timeout now delegates
  to bounded_syscall. ~30 LOC of duplicated channel/thread plumbing
  deleted. Same semantics, cleaner.
- src/io/writeback_file.rs::WritebackFile::sync_all now wraps the
  final libc::fsync(fd) with bounded_syscall (60 s deadline). On
  timeout: log error at target=mux and return Ok — kernel will flush
  on close, best-effort but bounded. Covers FileSectorSink::finish,
  PatchSink::close, SweepSink::close, and the mux MKV finalize path
  (they all sync through WritebackFile).

What still hangs (deliberately not wrapped — too hot a path):
- File::write itself. Per-frame write on a wedged NFS could still
  block; but back-pressure from a stuck consumer means the producer
  notices within seconds, not minutes — different failure mode than
  the WAIT_AFTER hang 0.20.5/0.20.6 fix.
2026-05-13 14:22:24 -07:00
21 changed files with 3808 additions and 1171 deletions
+1 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "libfreemkv"
version = "0.20.5"
version = "0.20.9"
edition = "2024"
rust-version = "1.86"
license = "AGPL-3.0-only"
+1274 -959
View File
File diff suppressed because it is too large Load Diff
+29 -1
View File
@@ -186,6 +186,22 @@ impl ReadCtx {
/// marginal media is part of the job, and the fast-jump
/// threshold is loose so we don't bail too early on a range that
/// has scattered good sectors mixed in.
///
/// `damage_threshold_pct = 6` mirrors `disc/patch.rs`'s
/// `PASSN_DAMAGE_THRESHOLD_PCT`. Pass N triggers the damage-skip
/// at half the density Pass 1 uses (Pass 1 = 12%) because the
/// patch loop's whole job is to chip away at bad ranges — being
/// more eager to skip clustered bad sectors converges faster on
/// the recoverable good sectors inside a range. The patch-side
/// `compute_damage_skip` reads its threshold from
/// `PASSN_DAMAGE_THRESHOLD_PCT`; keep the two in sync until the
/// patch loop's damage-skip is unified with `handle_read_error`'s
/// jump path. (v0.20.8 unification attempt found the unification
/// itself blocked on the size-aware `range_remaining/4` cap that
/// lives in `compute_damage_skip` but not in
/// `handle_read_error::JumpAhead` — see
/// `tests/passn_handler_ab.rs` for the A/B fixture that pins
/// the divergence point.)
pub fn for_patch(batch: u16) -> Self {
Self {
batch,
@@ -194,7 +210,7 @@ impl ReadCtx {
consecutive_outer_failures: 0,
damage_window: Vec::with_capacity(16),
damage_window_max: 16,
damage_threshold_pct: 12,
damage_threshold_pct: PATCH_DAMAGE_THRESHOLD_PCT,
// Pass N is allowed to grind: window-based jump only,
// matching the historical behaviour for patch passes.
fast_jump_threshold: u64::MAX,
@@ -401,6 +417,18 @@ const WEDGE_ABORT_THRESHOLD: u64 = 16;
/// next range.
const WEDGE_PASS_N_SKIP_SECTORS: u64 = 64;
/// Single source of truth for the Pass-N damage-window threshold.
/// Both [`ReadCtx::for_patch`] and `disc::patch::compute_damage_skip`
/// reference this constant so the two damage-skip paths cannot drift.
///
/// 6% means: with a 16-entry sliding window, the damage-skip fires
/// once 1 out of 16 recent reads has failed. Pass 1 uses a 12%
/// threshold via `damage_threshold_pct` on `for_sweep`; Pass N is
/// twice as eager because patch's whole job is to converge on the
/// bad sub-zones inside a NonTrimmed range — a faster trigger
/// produces tighter convergence in fewer iterations.
pub const PATCH_DAMAGE_THRESHOLD_PCT: usize = 6;
/// THE single error-handling entry point. Updates `ctx`, returns the
/// action the caller must apply.
///
+246
View File
@@ -0,0 +1,246 @@
//! Bounded-syscall primitive: run a (potentially-blocking) operation
//! on a worker thread, with a hard wall-clock deadline and an optional
//! cooperative [`Halt`] poll. The calling thread is never trapped
//! inside a kernel call.
//!
//! ## Why this exists
//!
//! [`crate::halt::Halt`] is cooperative: callers poll
//! `is_cancelled()`. It cannot reach inside a syscall the kernel
//! currently owns the thread for — `libc::sync_file_range`,
//! `libc::fsync`, `File::write` on NFS, and so on. `/api/stop` from
//! autorip therefore can't unstick a thread sitting in such a syscall.
//!
//! [`bounded_syscall`] is the escape hatch: it runs `op` on a fresh
//! worker thread, then `recv_timeout`s on a rendezvous channel for the
//! result. The wait is broken into ~250 ms slices so the calling
//! thread can poll the supplied [`Halt`] in between. If the deadline
//! elapses or the halt fires, the worker is intentionally leaked — the
//! syscall will unwind whenever the kernel decides, or at process
//! exit, but the caller is free to fall back to a degraded code path
//! (skip the sync, log loudly, etc.).
//!
//! ## Trade-offs
//!
//! - **Thread per call.** Cheap (`std::thread::spawn` is < 100 µs on
//! Linux/macOS), but not free. Use on coarse-grained finalisation
//! syscalls (`sync_all`, `sync_file_range(WAIT_AFTER)`), not on hot
//! inner-loop writes.
//! - **Leak on timeout.** A wedged syscall keeps a kernel slot and a
//! user-space thread around for the rest of the process's life.
//! Bounded by the number of independent rip/mux sessions, which is
//! one per disc. The alternative — trapping the caller forever —
//! defeats the entire purpose of `/api/stop`.
//! - **Halt granularity ~250 ms.** Halt observation is not instant;
//! it's the worst-case latency of the `recv_timeout` slice. Good
//! enough for human-driven stop requests; not suitable for hard
//! real-time deadlines.
//!
//! ## Single source of truth
//!
//! Do NOT inline this pattern. Every blocking-syscall wrapper in the
//! rip + mux pipeline calls this helper, so changes (e.g. swapping the
//! channel impl, adjusting the poll slice, adding metrics) land in one
//! place.
//!
//! ## Platform
//!
//! Pure `std::thread` + `std::sync::mpsc`. No `cfg(target_os)` needed
//! here — the helper itself is platform-agnostic. Callers that wrap
//! Linux-only syscalls (`sync_file_range`) still need their own
//! `#[cfg(target_os = "linux")]` gates; this helper does not.
use std::sync::mpsc::{RecvTimeoutError, sync_channel};
use std::thread;
use std::time::{Duration, Instant};
use crate::halt::Halt;
/// Granularity of the halt poll. The receive loop wakes every
/// [`POLL_INTERVAL`] to (a) check the [`Halt`] token, then (b) check
/// the overall deadline, then go back to waiting. 250 ms is a
/// pragmatic balance: short enough that human-driven `/api/stop` feels
/// responsive (< 0.5 s p99), long enough that the polling overhead is
/// negligible against multi-second syscalls.
const POLL_INTERVAL: Duration = Duration::from_millis(250);
/// Failure outcome from a bounded syscall wrapper.
#[derive(Debug)]
pub(crate) enum BoundedError {
/// The user-visible halt token fired during the wait. The worker
/// thread is intentionally leaked — the caller should fall back to
/// a degraded code path rather than waiting on the syscall to
/// return.
Halted,
/// The deadline elapsed before the syscall returned. Same leak
/// semantics as `Halted`.
Timeout,
/// The worker thread panicked, or its sender disconnected before
/// sending a result. Treat as a benign no-op (callers usually
/// log and continue) rather than a hard error — by definition no
/// syscall observably ran to completion in this case.
WorkerLost,
}
/// Run a (potentially-blocking) operation on a worker thread with a
/// deadline and an optional cooperative halt-token poll. Returns the
/// operation's result if it completes within `timeout`; otherwise one
/// of [`BoundedError::Halted`] / [`BoundedError::Timeout`] /
/// [`BoundedError::WorkerLost`].
///
/// On `Halted` / `Timeout` the worker thread is intentionally leaked:
/// the syscall will unwind whenever the kernel decides, or when the
/// process exits. The calling thread is never trapped inside a kernel
/// call.
///
/// `halt` is polled at [`POLL_INTERVAL`] granularity. Pass `None` for
/// callers that don't (yet) have a halt token plumbed through —
/// behaviour degrades to deadline-only, matching the 0.20.5
/// `wait_after_with_timeout` shape this helper generalises.
///
/// `op` returns `R: Send + 'static`. The closure must own everything
/// it touches because it may outlive this call (timeout / halt cases).
pub(crate) fn bounded_syscall<F, R>(
halt: Option<&Halt>,
timeout: Duration,
op: F,
) -> Result<R, BoundedError>
where
F: FnOnce() -> R + Send + 'static,
R: Send + 'static,
{
// Rendezvous channel: the worker sends exactly one value (the
// op's return) and then exits. Capacity-0 means the send blocks
// until we receive — fine on the happy path; on the timeout /
// halt path the receiver is dropped and the worker's send
// returns Err, which the worker ignores.
let (tx, rx) = sync_channel::<R>(0);
let _ = thread::Builder::new()
.name("freemkv-bounded-syscall".into())
.spawn(move || {
// Ignore the send error: if we time out (or get halted)
// before the worker finishes, the receiver is dropped
// and `tx.send` returns Err. Either way, the worker has
// nothing more to do.
let _ = tx.send(op());
});
let deadline = Instant::now() + timeout;
loop {
let now = Instant::now();
let remaining = deadline.saturating_duration_since(now);
let slice = remaining.min(POLL_INTERVAL);
match rx.recv_timeout(slice) {
Ok(v) => return Ok(v),
Err(RecvTimeoutError::Timeout) => {
if let Some(h) = halt {
if h.is_cancelled() {
return Err(BoundedError::Halted);
}
}
if Instant::now() >= deadline {
return Err(BoundedError::Timeout);
}
// Otherwise: another slice.
}
Err(RecvTimeoutError::Disconnected) => {
// Worker thread spawn failed, or it panicked before
// sending. Caller treats this as "no syscall ran" —
// typically a no-op + log.
return Err(BoundedError::WorkerLost);
}
}
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
#[test]
fn op_completes_quickly() {
let r = bounded_syscall(None, Duration::from_secs(2), || 42u32);
assert!(matches!(r, Ok(42)));
}
#[test]
fn op_exceeds_timeout() {
// Op sleeps longer than the deadline → Timeout.
let r = bounded_syscall(None, Duration::from_millis(300), || {
thread::sleep(Duration::from_secs(2));
0u32
});
assert!(matches!(r, Err(BoundedError::Timeout)));
}
#[test]
fn halt_fires_during_wait() {
let halt = Halt::new();
let halt2 = halt.clone();
// Flip the halt from a side thread after ~300 ms — long
// enough that the receive loop has rolled at least one
// 250 ms slice and is sitting in `recv_timeout` again when
// the bit flips.
thread::spawn(move || {
thread::sleep(Duration::from_millis(300));
halt2.cancel();
});
let r = bounded_syscall(Some(&halt), Duration::from_secs(5), || {
thread::sleep(Duration::from_secs(5));
0u32
});
assert!(matches!(r, Err(BoundedError::Halted)));
}
#[test]
fn worker_panics() {
// Worker panics → sender drops without sending → recv sees
// Disconnected → WorkerLost. We use an explicit panic in the
// op closure rather than `panic!()` from inside the channel
// machinery; the spawned thread's panic is contained (no
// process abort) because we don't `.join()` it.
let r = bounded_syscall(None, Duration::from_secs(2), || -> u32 {
panic!("intentional test panic");
});
assert!(matches!(r, Err(BoundedError::WorkerLost)));
}
#[test]
fn halt_already_set_before_call_still_returns_halted() {
// Halt observed on the very first poll slice. The op blocks
// forever; we must not wait the full timeout to notice the
// halt is already set.
let halt = Halt::new();
halt.cancel();
let started = Instant::now();
let r = bounded_syscall(Some(&halt), Duration::from_secs(10), || {
thread::sleep(Duration::from_secs(10));
0u32
});
assert!(matches!(r, Err(BoundedError::Halted)));
// Should bail out within ~1 s; allow 2 s of slack for slow
// CI hosts.
assert!(
started.elapsed() < Duration::from_secs(2),
"halt-already-set took {:?}",
started.elapsed()
);
}
#[test]
fn ok_path_takes_no_halt_token() {
// Sanity: the `None` halt path is the documented zero-config
// form (matches the 0.20.5 `wait_after_with_timeout`
// behaviour). Op returns immediately; we must observe Ok.
let flag = Arc::new(AtomicBool::new(false));
let f2 = flag.clone();
let r = bounded_syscall(None, Duration::from_secs(2), move || {
f2.store(true, Ordering::Relaxed);
"ok"
});
assert!(matches!(r, Ok("ok")));
assert!(flag.load(Ordering::Relaxed));
}
}
+330
View File
@@ -0,0 +1,330 @@
//! Byte-sized bounded producer/consumer channel.
//!
//! Wraps `std::sync::mpsc::sync_channel` with a byte-accounting
//! `Mutex<usize> + Condvar` cap. Sender blocks (cooperatively) when
//! `used_bytes + item.byte_size() > capacity_bytes`. Receiver
//! decrements `used_bytes` when it takes the item.
//!
//! Why: the existing producer→consumer channel between `DiscStream`
//! (PES producer) and `MuxSink` (PES consumer) is bounded by frame
//! count. Frame sizes vary 100× between metadata and keyframes, so a
//! count-based cap either starves on small frames or buffers far too
//! much memory on big ones. Byte-sized accounting sizes the buffer for
//! the worst-case input stall (NFS read p99 ≈ 1–2 s × ~15 MB/s peak
//! compressed bitrate ≈ ~30 MB) directly.
//!
//! The underlying mpsc channel is created with a very large slot count
//! so the byte cap (not the slot count) is the real backpressure. Slot
//! count is only there to give the kernel a small chunk to wake on.
//!
//! See `(internal)/memory/project_buffering_architecture.md` §
//! Pipeline channel — sizing.
use std::sync::mpsc::{Receiver as MpscReceiver, RecvError, SendError, SyncSender, sync_channel};
use std::sync::{Arc, Condvar, Mutex};
/// Default byte cap for the muxer's input channel. Sized to hide a
/// worst-case ~2 s NFS read refill at UHD peak compressed bitrate
/// (~15 MB/s); 64 MiB gives headroom. Tweakable; not magic.
pub const BYTE_CHANNEL_DEFAULT_CAPACITY: usize = 64 * 1024 * 1024;
/// Slot capacity of the inner `sync_channel`. Large so the byte cap is
/// the real backpressure mechanism — the mpsc slot count only exists
/// to give the kernel a chunk to wake on. PES frames are typically
/// ~700 B each, so 64 MiB ≈ 90 k frames; 200 k is comfortable headroom.
const INNER_SLOT_CAPACITY: usize = 200_000;
/// Anything whose in-memory cost can be accounted by a single
/// `usize`. Implement on the item type sent through [`Sender`].
pub trait HasByteSize {
/// Bytes this item contributes to the channel's used budget.
/// Must be > 0 to make progress (a 0-byte item would never
/// block the sender no matter the cap; see send_blocks_at_capacity
/// test).
fn byte_size(&self) -> usize;
}
impl HasByteSize for crate::pes::PesFrame {
fn byte_size(&self) -> usize {
// Frame data + the fixed header overhead the serializer
// writes (track + pts + keyframe + len). The `Vec<u8>` heap
// allocation also has alloc-header overhead but that's
// <0.1 % at typical frame sizes — folding it in would just
// add noise to the budget.
self.data.len() + 14
}
}
/// Shared book-keeping between [`Sender`] and [`Receiver`]. Wrapped in
/// an `Arc` because both halves hold it independently.
struct Accounting {
used: Mutex<usize>,
cv: Condvar,
capacity: usize,
}
/// Send half of the byte-bounded channel.
///
/// `send` blocks (on a `Condvar`) when adding the item would push
/// `used_bytes` past `capacity_bytes`. Unblocks when the receiver
/// `recv`s items out and notifies. Returns `Err(item)` if the
/// receiver has been dropped — mirrors `mpsc::SyncSender::send`.
pub struct Sender<T: HasByteSize> {
tx: SyncSender<T>,
acct: Arc<Accounting>,
}
impl<T: HasByteSize> Clone for Sender<T> {
fn clone(&self) -> Self {
Sender {
tx: self.tx.clone(),
acct: self.acct.clone(),
}
}
}
impl<T: HasByteSize> Sender<T> {
/// Push one item. Blocks until adding it would not exceed the
/// capacity, then sends through the inner mpsc channel.
pub fn send(&self, item: T) -> Result<(), SendError<T>> {
let sz = item.byte_size();
// Reserve capacity first. The reservation is observable to
// other senders via `used`; only after we win the slot do we
// hand the item to the inner mpsc channel. That ordering means
// `used` is always a conservative upper bound on what's in the
// mpsc queue + about-to-be-sent.
{
let mut used = self.acct.used.lock().expect("byte_channel poisoned");
// An item bigger than the whole capacity will never fit; let
// it through anyway as a one-shot reservation, otherwise the
// sender deadlocks forever waiting for `used == 0` AND
// nothing in flight. The receiver will drain it on the
// other side. Same behaviour as `std::sync::mpsc` for
// arbitrarily large messages.
while *used + sz > self.acct.capacity && *used > 0 {
used = self.acct.cv.wait(used).expect("byte_channel cv poisoned");
}
*used += sz;
}
match self.tx.send(item) {
Ok(()) => Ok(()),
Err(SendError(returned)) => {
// Receiver dropped — refund the reservation so a later
// sender on a clone doesn't observe phantom used bytes
// (the receiver is gone so nobody will decrement).
let mut used = self.acct.used.lock().expect("byte_channel poisoned");
*used = used.saturating_sub(sz);
self.acct.cv.notify_all();
Err(SendError(returned))
}
}
}
}
/// Receive half of the byte-bounded channel.
///
/// `recv` blocks on the inner mpsc until an item is available, then
/// decrements the byte-accounting and wakes any sender waiting on
/// capacity.
pub struct Receiver<T: HasByteSize> {
rx: MpscReceiver<T>,
acct: Arc<Accounting>,
}
impl<T: HasByteSize> Receiver<T> {
/// Take the next item. Returns `Err(RecvError)` when all senders
/// have been dropped and the channel is empty.
pub fn recv(&self) -> Result<T, RecvError> {
let item = self.rx.recv()?;
let sz = item.byte_size();
let mut used = self.acct.used.lock().expect("byte_channel poisoned");
*used = used.saturating_sub(sz);
// Notify all so multi-sender setups wake every blocked sender,
// not just one. Wasted wakeups are cheap; missed wakeups would
// be a deadlock.
self.acct.cv.notify_all();
Ok(item)
}
}
/// Create a byte-bounded channel with the given capacity in bytes.
/// Returns a `(Sender, Receiver)` pair; clone the `Sender` for
/// multi-producer setups.
pub fn channel<T: HasByteSize>(capacity_bytes: usize) -> (Sender<T>, Receiver<T>) {
let (tx, rx) = sync_channel::<T>(INNER_SLOT_CAPACITY);
let acct = Arc::new(Accounting {
used: Mutex::new(0),
cv: Condvar::new(),
capacity: capacity_bytes,
});
(
Sender {
tx,
acct: acct.clone(),
},
Receiver { rx, acct },
)
}
#[cfg(test)]
mod tests {
use super::*;
use std::sync::Arc;
use std::sync::atomic::{AtomicUsize, Ordering};
use std::thread;
use std::time::{Duration, Instant};
/// Test payload — its `byte_size` returns whatever we passed at
/// construction so capacity math is exact and predictable.
#[derive(Clone, Debug, PartialEq, Eq)]
struct Item {
sz: usize,
tag: u32,
}
impl HasByteSize for Item {
fn byte_size(&self) -> usize {
self.sz
}
}
#[test]
fn send_recv_round_trip() {
let (tx, rx) = channel::<Item>(1024);
for i in 0..5 {
tx.send(Item { sz: 100, tag: i }).unwrap();
}
for i in 0..5 {
let got = rx.recv().unwrap();
assert_eq!(got, Item { sz: 100, tag: i });
}
}
#[test]
fn byte_accounting_decrements_on_recv() {
// Internal book-keeping check via observable side-effect: after
// sending K items totalling N bytes and receiving them all, a
// subsequent send of an N-byte item must NOT block (no items
// in flight, all capacity refunded).
let (tx, rx) = channel::<Item>(1024);
for _ in 0..4 {
tx.send(Item { sz: 256, tag: 0 }).unwrap();
}
for _ in 0..4 {
rx.recv().unwrap();
}
// Cap is now fully available again. Send a 1024-byte item; the
// `used > 0` guard means it goes through alone (no wait).
let start = Instant::now();
tx.send(Item { sz: 1024, tag: 99 }).unwrap();
assert!(start.elapsed() < Duration::from_millis(100));
let got = rx.recv().unwrap();
assert_eq!(got.tag, 99);
}
#[test]
fn send_blocks_at_capacity_unblocks_on_recv() {
// Cap = 200 bytes, item = 100 bytes. First two sends fit
// exactly; the third must block until a recv frees capacity.
let (tx, rx) = channel::<Item>(200);
tx.send(Item { sz: 100, tag: 0 }).unwrap();
tx.send(Item { sz: 100, tag: 1 }).unwrap();
let tx2 = tx.clone();
let sent_at = Arc::new(Mutex::new(None::<Instant>));
let sent_at2 = sent_at.clone();
let h = thread::spawn(move || {
tx2.send(Item { sz: 100, tag: 2 }).unwrap();
*sent_at2.lock().unwrap() = Some(Instant::now());
});
// Give the sender thread a head start; it should be parked in
// `cv.wait` because used (200) + 100 > capacity (200).
thread::sleep(Duration::from_millis(100));
assert!(
sent_at.lock().unwrap().is_none(),
"third send should be blocked at capacity"
);
// Drain one. Sender wakes and completes.
let recv_at = Instant::now();
let got = rx.recv().unwrap();
assert_eq!(got.tag, 0);
h.join().unwrap();
let sent_when = sent_at.lock().unwrap().unwrap();
assert!(
sent_when >= recv_at,
"sender must complete AFTER receiver freed capacity"
);
// Drain the remaining two.
assert_eq!(rx.recv().unwrap().tag, 1);
assert_eq!(rx.recv().unwrap().tag, 2);
}
#[test]
fn item_larger_than_capacity_still_goes_through() {
// Pathological case: a single item bigger than the capacity.
// The guard `*used > 0` lets it through when the channel is
// empty (otherwise the sender deadlocks forever). Matches
// `mpsc::SyncSender` semantics for oversize messages.
let (tx, rx) = channel::<Item>(100);
tx.send(Item { sz: 1000, tag: 7 }).unwrap();
let got = rx.recv().unwrap();
assert_eq!(got, Item { sz: 1000, tag: 7 });
}
#[test]
fn concurrent_send_recv_stress() {
// 4 sender threads × 1k items each, 1 receiver. Verify byte
// accounting stays sane (channel never deadlocks, every item
// arrives exactly once) under contention.
const SENDERS: u32 = 4;
const PER_SENDER: u32 = 1000;
const TOTAL: u32 = SENDERS * PER_SENDER;
let (tx, rx) = channel::<Item>(8 * 1024);
let sent = Arc::new(AtomicUsize::new(0));
let mut handles = Vec::new();
for s in 0..SENDERS {
let tx = tx.clone();
let sent = sent.clone();
handles.push(thread::spawn(move || {
for i in 0..PER_SENDER {
// Vary item size so accounting actually has to
// multiplex differently-sized blockers. 1B → 256B.
let sz = 1 + ((i as usize) % 256);
tx.send(Item {
sz,
tag: s * PER_SENDER + i,
})
.unwrap();
sent.fetch_add(1, Ordering::SeqCst);
}
}));
}
// Drop our local sender so the receiver can eventually see
// RecvError once all sender clones are done. Cloning the
// sender into each producer means each clone Drop'd separately.
drop(tx);
let mut received = 0u32;
while let Ok(_item) = rx.recv() {
received += 1;
}
for h in handles {
h.join().unwrap();
}
assert_eq!(received, TOTAL);
assert_eq!(sent.load(Ordering::SeqCst) as u32, TOTAL);
}
#[test]
fn send_after_recv_dropped_returns_err() {
let (tx, rx) = channel::<Item>(1024);
drop(rx);
let r = tx.send(Item { sz: 10, tag: 0 });
assert!(r.is_err());
}
}
+15
View File
@@ -0,0 +1,15 @@
//! Linux: hint the kernel that this fd will be read sequentially so
//! readahead widens. `posix_fadvise(POSIX_FADV_SEQUENTIAL)` is a hint,
//! not a guarantee — the kernel still owns the policy decision.
use std::fs::File;
use std::os::unix::io::AsRawFd;
pub(super) fn hint_sequential(file: &File, _len_bytes: u64) {
// Best-effort: return value ignored. A fadvise failure has no
// user-observable consequence (reads still work, just without the
// widened readahead window).
unsafe {
libc::posix_fadvise(file.as_raw_fd(), 0, 0, libc::POSIX_FADV_SEQUENTIAL);
}
}
+38
View File
@@ -0,0 +1,38 @@
//! macOS: hint the kernel to prefetch a generous chunk. macOS has no
//! direct `POSIX_FADV_SEQUENTIAL` equivalent; the idiomatic hint is
//! `fcntl(F_RDADVISE, &radvisory)` describing the byte range you
//! intend to read soon. We point it at the whole file (clamped to a
//! ceiling so a multi-TB ISO doesn't ask the kernel to prefetch
//! everything at once).
use std::fs::File;
use std::os::unix::io::AsRawFd;
/// `F_RDADVISE` opcode — not in libc's named constants on all SDKs.
const F_RDADVISE: libc::c_int = 44;
/// Cap on the byte length we pass to `F_RDADVISE`. Asking for a
/// multi-GB readahead window is counterproductive — the OS doesn't
/// have that much cache to throw at one fd. 64 MiB is generous for
/// our use case (sweep, mux) and matches the byte-channel cap so the
/// kernel's prefetch ≥ our app-level pipeline depth.
const RDADVISE_MAX_BYTES: i64 = 64 * 1024 * 1024;
/// `radvisory` per `<sys/fcntl.h>`. repr(C) layout is stable.
#[repr(C)]
struct RadAdvisory {
ra_offset: libc::off_t,
ra_count: libc::c_int,
}
pub(super) fn hint_sequential(file: &File, len_bytes: u64) {
let bytes = (len_bytes as i64).min(RDADVISE_MAX_BYTES);
let mut ra = RadAdvisory {
ra_offset: 0,
ra_count: bytes as libc::c_int,
};
// Best-effort.
unsafe {
libc::fcntl(file.as_raw_fd(), F_RDADVISE, &mut ra);
}
}
+393
View File
@@ -0,0 +1,393 @@
//! [`FileSectorSource`] — read 2048-byte sectors from an ISO file on
//! disk, with an internal 32 MiB read-ahead buffer.
//!
//! ## Why the buffer
//!
//! On NFS-mounted ISOs, an unbuffered `pread(2048)` per sector pays an
//! NFS round-trip for every sector. With `rsize=1 MiB` and a 100-150 ms
//! NFS RTT, that's three orders of magnitude more round trips than
//! necessary — the muxer goes read-bound on every read, even though
//! the local NFS client could deliver MB/s on bigger requests.
//!
//! Internally this source keeps a [`READAHEAD_BUF_BYTES`] (32 MiB)
//! window pre-read from the file. `read_sectors(lba, count)` slices
//! into the window if `[lba, lba+count)` is contained in it; otherwise
//! the window is refilled (full-size aligned to the requested LBA's
//! buffer position).
//!
//! ## Access pattern assumption
//!
//! The buffer is sized for **forward-sequential** reads (sweep, mux).
//! Reverse-mode patch is range-local, so a refill per range works out
//! fine (the buffer covers the whole range for typical bad-range
//! sizes). Random-access reads thrash the buffer — at which point the
//! 32 MiB pre-read is wasted work. We accept that: the use case is
//! mux + sweep, both forward-sequential.
//!
//! Backward seeks rebuffer from the new LBA; partial reads at EOF
//! return only the bytes that exist (the underlying file is shorter
//! than a full buffer slot).
//!
//! ## Platform open hints
//!
//! On `open()` each platform issues its "sequential access expected"
//! hint to the kernel so OS-level readahead widens. The hint lives in
//! a per-OS sibling module ([`linux::hint_sequential`] et al.) — no
//! inline `#[cfg]` in this file.
#[cfg(target_os = "linux")]
mod linux;
#[cfg(target_os = "macos")]
mod macos;
#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))]
mod other;
#[cfg(target_os = "windows")]
mod windows;
#[cfg(target_os = "linux")]
use linux as platform;
#[cfg(target_os = "macos")]
use macos as platform;
#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))]
use other as platform;
#[cfg(target_os = "windows")]
use windows as platform;
use std::fs::File;
use std::io::{Read, Seek, SeekFrom};
use std::path::Path;
use crate::error::{Error, Result};
use crate::sector::SectorSource;
/// Internal read-ahead buffer size. 32 MiB amortises one NFS round
/// trip across ~16 k sectors — three orders of magnitude fewer trips
/// than per-sector pread, and large enough to coast through a typical
/// NFS server commit blip.
///
/// Tweakable. Named const, not a magic number.
pub const READAHEAD_BUF_BYTES: usize = 32 * 1024 * 1024;
const SECTOR_SIZE: usize = 2048;
/// Sectors per refill: [`READAHEAD_BUF_BYTES`] / [`SECTOR_SIZE`]. The
/// buffer always tries to hold this many, except at the tail of the
/// file where less data exists.
const BUF_SECTORS: u32 = (READAHEAD_BUF_BYTES / SECTOR_SIZE) as u32;
/// SectorSource backed by a file (ISO image) with an internal
/// `READAHEAD_BUF_BYTES`-sized read-ahead window.
///
/// `read_sectors` is satisfied from the buffer when possible; otherwise
/// a full-buffer refill is issued at the requested LBA's position and
/// the call is re-tried against the freshly populated window.
pub struct FileSectorSource {
file: File,
/// Total file size in sectors. Constant after construction;
/// surfaced via [`SectorSource::capacity_sectors`].
capacity: u32,
/// Heap-allocated read-ahead buffer. Boxed slice (not `Vec`) so
/// the allocation is sized exactly once and the layout is
/// stable. Length is at most [`READAHEAD_BUF_BYTES`].
buf: Box<[u8]>,
/// First LBA covered by `buf`. `buf_len_sectors == 0` means the
/// buffer is empty (e.g. immediately after construction or a
/// backward seek); the value of `buf_start_lba` is then
/// meaningless.
buf_start_lba: u32,
/// Number of valid sectors in `buf`. `[buf_start_lba,
/// buf_start_lba + buf_len_sectors)` is the half-open LBA range
/// the buffer currently caches.
buf_len_sectors: u32,
}
impl FileSectorSource {
/// Open an existing ISO file for reading. Capacity is derived
/// from `metadata().len() / 2048`. Returns
/// [`Error::IsoTooLarge`] if the file would exceed the 32-bit
/// LBA address space (~8 TB).
///
/// Issues the platform's "sequential access expected" hint on the
/// fd (Linux `posix_fadvise(SEQUENTIAL)`, macOS `fcntl(F_RDADVISE)`,
/// Windows TODO stub) so the kernel's readahead widens.
pub fn open(path: &Path) -> std::io::Result<Self> {
let file = File::open(path)?;
let len = file.metadata()?.len();
let sectors = len / SECTOR_SIZE as u64;
if sectors > u32::MAX as u64 {
return Err(Error::IsoTooLarge {
path: path.to_string_lossy().into_owned(),
}
.into());
}
let capacity = sectors as u32;
// Best-effort sequential hint. Ignored on platforms without
// an equivalent primitive (or where the API exists but the
// FS doesn't honour it).
platform::hint_sequential(&file, len);
// Pre-allocate the buffer once. `vec![0u8; N].into_boxed_slice()`
// is the canonical way to fix the allocation size up-front;
// `Vec::with_capacity` would leave `len == 0` and force callers
// to do unsafe length manipulation to write into it.
let buf = vec![0u8; READAHEAD_BUF_BYTES].into_boxed_slice();
Ok(Self {
file,
capacity,
buf,
buf_start_lba: 0,
buf_len_sectors: 0,
})
}
/// True if `[lba, lba + count)` is wholly inside the current
/// buffer window. `count == 0` is vacuously true.
fn buffer_covers(&self, lba: u32, count: u32) -> bool {
if self.buf_len_sectors == 0 {
return false;
}
let end = match lba.checked_add(count) {
Some(e) => e,
None => return false,
};
let buf_end = self.buf_start_lba.saturating_add(self.buf_len_sectors);
lba >= self.buf_start_lba && end <= buf_end
}
/// Refill the buffer so it starts at `lba`. Read as many sectors
/// as we have buffer space AND file capacity for. Caller has
/// already checked `lba < capacity`.
fn refill(&mut self, lba: u32) -> Result<()> {
debug_assert!(lba < self.capacity, "refill past capacity");
// Don't read past EOF — clamp the request to remaining
// sectors. partial-buffer-at-EOF behaviour is intentional.
let want = BUF_SECTORS.min(self.capacity - lba);
let want_bytes = want as usize * SECTOR_SIZE;
let offset = lba as u64 * SECTOR_SIZE as u64;
self.file
.seek(SeekFrom::Start(offset))
.map_err(|e| Error::IoError { source: e })?;
self.file
.read_exact(&mut self.buf[..want_bytes])
.map_err(|e| Error::IoError { source: e })?;
self.buf_start_lba = lba;
self.buf_len_sectors = want;
Ok(())
}
}
impl SectorSource for FileSectorSource {
fn capacity_sectors(&self) -> u32 {
self.capacity
}
fn read_sectors(
&mut self,
lba: u32,
count: u16,
out: &mut [u8],
_recovery: bool,
) -> Result<usize> {
let count = count as u32;
let bytes = count as usize * SECTOR_SIZE;
debug_assert!(
out.len() >= bytes,
"FileSectorSource::read_sectors: out len {} < requested {}",
out.len(),
bytes
);
if count == 0 {
return Ok(0);
}
// Refill if the requested range isn't entirely buffered.
// `buffer_covers` also handles the empty-buffer case
// (buf_len_sectors == 0).
if !self.buffer_covers(lba, count) {
// A request larger than the buffer itself can never fit;
// fall back to a one-shot direct pread for that pathological
// case so callers can't deadlock the source.
if count > BUF_SECTORS {
let offset = lba as u64 * SECTOR_SIZE as u64;
self.file
.seek(SeekFrom::Start(offset))
.map_err(|e| Error::IoError { source: e })?;
self.file
.read_exact(&mut out[..bytes])
.map_err(|e| Error::IoError { source: e })?;
// Invalidate buffer state — we bypassed it, the
// window is no longer authoritative for this LBA.
self.buf_len_sectors = 0;
return Ok(bytes);
}
self.refill(lba)?;
}
// Slice the buffer at the requested LBA's offset within it.
let off_sectors = (lba - self.buf_start_lba) as usize;
let off_bytes = off_sectors * SECTOR_SIZE;
out[..bytes].copy_from_slice(&self.buf[off_bytes..off_bytes + bytes]);
Ok(bytes)
}
}
#[cfg(test)]
mod tests {
use super::*;
use std::io::Write;
use tempfile::tempdir;
/// Build a deterministic ISO of `sectors` sectors where sector `n`
/// is filled with the byte pattern `((n & 0xff) as u8)`. Lets us
/// verify any sector by content alone.
fn make_iso(path: &std::path::Path, sectors: u32) {
let mut f = std::fs::File::create(path).unwrap();
let mut chunk = vec![0u8; SECTOR_SIZE];
for n in 0..sectors {
let b = (n & 0xff) as u8;
chunk.iter_mut().for_each(|c| *c = b);
f.write_all(&chunk).unwrap();
}
f.flush().unwrap();
}
#[test]
fn sequential_reads_match_file() {
// Two full buffer windows + a tail = exercise refill across
// boundaries.
let total = BUF_SECTORS * 2 + 17;
let dir = tempdir().unwrap();
let path = dir.path().join("seq.iso");
make_iso(&path, total);
let mut src = FileSectorSource::open(&path).unwrap();
assert_eq!(src.capacity_sectors(), total);
let mut got = vec![0u8; SECTOR_SIZE];
for lba in 0..total {
src.read_sectors(lba, 1, &mut got, false).unwrap();
let expected = (lba & 0xff) as u8;
assert!(
got.iter().all(|b| *b == expected),
"sector {lba} content mismatch: expected 0x{expected:02x}"
);
}
}
#[test]
fn multi_sector_read_spanning_buffer_boundary() {
// A read that lands exactly on the last sector of the buffer
// plus the first sector of the next refill must rebuffer
// mid-read. Bypass path triggers when count > BUF_SECTORS; we
// want the in-window path, so count stays small but
// straddles the boundary.
let total = BUF_SECTORS * 2;
let dir = tempdir().unwrap();
let path = dir.path().join("span.iso");
make_iso(&path, total);
let mut src = FileSectorSource::open(&path).unwrap();
// Prime: read sector 0 to populate buffer #0.
let mut got = vec![0u8; SECTOR_SIZE];
src.read_sectors(0, 1, &mut got, false).unwrap();
assert_eq!(src.buf_start_lba, 0);
assert!(src.buf_len_sectors > 0);
// Now read 4 sectors crossing the buffer boundary at
// BUF_SECTORS - 2 → BUF_SECTORS + 1. Spans the refill.
let span_lba = BUF_SECTORS - 2;
let mut buf4 = vec![0u8; SECTOR_SIZE * 4];
src.read_sectors(span_lba, 4, &mut buf4, false).unwrap();
for i in 0..4 {
let lba = span_lba + i as u32;
let expected = (lba & 0xff) as u8;
for b in &buf4[i * SECTOR_SIZE..(i + 1) * SECTOR_SIZE] {
assert_eq!(*b, expected, "byte mismatch at sub-sector {i}");
}
}
}
#[test]
fn backward_seek_rebuffers() {
// Read forward across two windows, then jump back to sector
// 0. Buffer must refill from the start.
let total = BUF_SECTORS * 2 + 5;
let dir = tempdir().unwrap();
let path = dir.path().join("back.iso");
make_iso(&path, total);
let mut src = FileSectorSource::open(&path).unwrap();
let mut got = vec![0u8; SECTOR_SIZE];
// Forward to the second window.
src.read_sectors(BUF_SECTORS + 1, 1, &mut got, false)
.unwrap();
let start_after_forward = src.buf_start_lba;
assert!(start_after_forward >= BUF_SECTORS);
// Backward to sector 0. The current buffer doesn't cover it
// → refill must happen.
src.read_sectors(0, 1, &mut got, false).unwrap();
assert_eq!(src.buf_start_lba, 0);
assert!(got.iter().all(|b| *b == 0));
}
#[test]
fn partial_buffer_at_eof() {
// File is smaller than one buffer window. The buffer must
// populate with only the available sectors and reads must
// still succeed.
let total: u32 = 100;
assert!(total < BUF_SECTORS);
let dir = tempdir().unwrap();
let path = dir.path().join("small.iso");
make_iso(&path, total);
let mut src = FileSectorSource::open(&path).unwrap();
assert_eq!(src.capacity_sectors(), total);
let mut got = vec![0u8; SECTOR_SIZE];
// First read triggers refill clamped to `total`.
src.read_sectors(0, 1, &mut got, false).unwrap();
assert_eq!(src.buf_len_sectors, total);
// Read the very last sector — still inside the buffer.
src.read_sectors(total - 1, 1, &mut got, false).unwrap();
let expected = ((total - 1) & 0xff) as u8;
assert!(got.iter().all(|b| *b == expected));
}
#[test]
fn oversized_read_bypasses_buffer() {
// A request larger than the buffer must not deadlock the
// refill (which only loads BUF_SECTORS at a time). Bypass
// path handles it via direct pread.
let total = BUF_SECTORS + 100;
let dir = tempdir().unwrap();
let path = dir.path().join("over.iso");
make_iso(&path, total);
let mut src = FileSectorSource::open(&path).unwrap();
// Read more than BUF_SECTORS in one call. count is u16, so we
// can't actually exceed BUF_SECTORS (16k) — but the path also
// triggers via `out.len() / SECTOR_SIZE > BUF_SECTORS` check
// implicitly because count > BUF_SECTORS. BUF_SECTORS for
// 32 MiB is 16384, which does fit in u16 (max 65535). Cap
// at BUF_SECTORS + 1 to exercise the bypass.
let req = (BUF_SECTORS + 1) as u16;
let req_bytes = req as usize * SECTOR_SIZE;
let mut big = vec![0u8; req_bytes];
src.read_sectors(0, req, &mut big, false).unwrap();
// Spot-check sector 0 and the last requested sector.
assert!(big[..SECTOR_SIZE].iter().all(|b| *b == 0));
let last_lba = req as u32 - 1;
let exp = (last_lba & 0xff) as u8;
let last_off = (req as usize - 1) * SECTOR_SIZE;
assert!(
big[last_off..last_off + SECTOR_SIZE]
.iter()
.all(|b| *b == exp)
);
}
}
+7
View File
@@ -0,0 +1,7 @@
//! Fallback for targets without a known sequential-readahead hint
//! (BSDs, illumos, etc.). No-op — reads still work, they just don't
//! get the OS-level prefetch widening.
use std::fs::File;
pub(super) fn hint_sequential(_file: &File, _len_bytes: u64) {}
+19
View File
@@ -0,0 +1,19 @@
//! Windows: the canonical sequential-access hint is
//! `FILE_FLAG_SEQUENTIAL_SCAN` passed to `CreateFile` at open time —
//! it cannot be set after the fact via `SetFileInformationByHandle`.
//! Routing the open call through this module would mean a custom
//! `File::from_raw_handle` plumb for every `FileSectorSource::open`
//! caller, which is more invasive than the Phase 1 scope.
//!
//! TODO: replumb `FileSectorSource::open` to take an
//! `OpenOptions`-style builder so the Windows path can flip the flag
//! at open time. For now this is a no-op stub.
use std::fs::File;
pub(super) fn hint_sequential(_file: &File, _len_bytes: u64) {
tracing::debug!(
target: "mux",
"FileSectorSource hint_sequential: windows stub (TODO: FILE_FLAG_SEQUENTIAL_SCAN at open)"
);
}
+12
View File
@@ -10,10 +10,22 @@
//! no-op pipeline — their default cache policies have not been shown
//! to exhibit the same pathology for this access pattern.
//!
//! `FileSectorSource` is the read-side dual — it implements
//! [`crate::sector::SectorSource`] for an ISO file with an internal
//! 32 MiB read-ahead buffer that amortises NFS round-trip latency
//! across thousands of sector reads.
//!
//! `Pipeline` + `Sink` (0.18) is the generic producer/consumer primitive
//! used by sweep, patch, and mux to overlap reads with writes via a
//! bounded channel + dedicated consumer thread.
//!
//! `byte_channel` is a byte-sized producer/consumer channel for the
//! mux pipeline, sized to absorb worst-case input read stalls (see
//! `(internal)/memory/project_buffering_architecture.md`).
pub(crate) mod bounded;
pub mod byte_channel;
pub mod file_sector_source;
mod writeback;
mod writeback_file;
+338 -1
View File
@@ -34,10 +34,35 @@
//! consumer lag detection). This is critical for diagnosing stalls.
use std::io;
use std::sync::mpsc::{SyncSender, sync_channel};
use std::sync::mpsc::{SyncSender, TrySendError, sync_channel};
use std::thread::{self, JoinHandle};
use std::time::{Duration, Instant};
use crate::error::Error;
use crate::halt::Halt;
/// Deadline for [`Pipeline::finish_with_halt`]'s polling join. Chosen
/// to be comfortably longer than the autorip hard watchdog
/// (`HARD_WATCHDOG_STALL_SECS = 300s`) so the watchdog's `exit(1)`
/// fires first when both are racing on the same wedged consumer.
///
/// 10 minutes is a backstop, not a normal timeout — the consumer is
/// expected to drain in seconds. If we hit this, something is wedged
/// inside a kernel call the consumer thread can't unwind from, and the
/// caller has already lost the rip.
pub const JOIN_TIMEOUT_SECS: u64 = 600;
/// Polling slice for the halt-aware send/finish loops. Mirrors the
/// `bounded_syscall` cadence (250 ms) so halt observation feels equally
/// responsive across both primitives.
const POLL_INTERVAL: Duration = Duration::from_millis(250);
/// Polling slice for the halt-aware send loop. Smaller than
/// [`POLL_INTERVAL`] because send latency tolerance is much lower —
/// frames must move through the channel at hundreds-of-Hz on the
/// happy path; 50 ms keeps backpressure-driven wakeups fine-grained
/// without busy-looping.
const SEND_POLL_INTERVAL: Duration = Duration::from_millis(50);
/// Check if verbose debug logging is enabled via FREEMKV_DEBUG env var.
pub fn debug_enabled() -> bool {
@@ -260,6 +285,65 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
self.tx.try_send(item)
}
/// Halt-aware bounded variant of [`Pipeline::send`].
///
/// Polls `try_send` on a 50 ms slice. Between slices, checks
/// (1) the [`Halt`] token and (2) the per-call `deadline`. Returns:
///
/// - `Ok(())` once the item lands in the channel.
/// - `Err(item)` if the consumer disconnected, the halt fired, or
/// the deadline elapsed — the caller gets the item back so it
/// can decide whether to drop it, route it elsewhere, or unwind.
///
/// Use this in producer threads that have a `Halt` token threaded
/// through (mux, sweep, patch). Plain [`Pipeline::send`] is
/// preserved for callers that don't (yet) plumb halt through.
///
/// Unlike [`Pipeline::send`], this never blocks the producer
/// thread inside an unkillable `mpsc::send` — if the consumer is
/// wedged inside an unkillable syscall, the producer can still
/// observe `/api/stop` and unwind.
pub fn send_with_halt(&self, item: I, halt: &Halt, deadline: Duration) -> Result<(), I> {
let end = Instant::now() + deadline;
let mut pending = item;
loop {
match self.tx.try_send(pending) {
Ok(()) => return Ok(()),
Err(TrySendError::Full(returned)) => {
pending = returned;
if halt.is_cancelled() {
if debug_enabled() {
tracing::debug!(
"Pipeline send_with_halt: halt observed, returning item={}",
std::any::type_name::<I>()
);
}
return Err(pending);
}
if Instant::now() >= end {
if debug_enabled() {
tracing::debug!(
"Pipeline send_with_halt: deadline elapsed, returning item={}",
std::any::type_name::<I>()
);
}
return Err(pending);
}
thread::sleep(SEND_POLL_INTERVAL);
}
Err(TrySendError::Disconnected(returned)) => {
if debug_enabled() {
tracing::debug!(
"Pipeline send_with_halt: consumer disconnected, item={}",
std::any::type_name::<I>()
);
}
return Err(returned);
}
}
}
}
/// Drop the producer-side channel and wait for the consumer
/// thread to finish. Returns whatever the consumer's `close()`
/// produced, or the first `apply` error, or — on consumer panic —
@@ -289,6 +373,69 @@ impl<I: Send + 'static, R: Send + 'static> Pipeline<I, R> {
}
}
}
/// Halt-aware, deadline-bounded variant of [`Pipeline::finish`].
///
/// Drops the producer-side channel (same as `finish`) and then
/// polls `JoinHandle::is_finished()` on a 250 ms cadence. Between
/// slices, checks (1) the optional [`Halt`] token and (2) the
/// [`JOIN_TIMEOUT_SECS`] deadline. Returns:
///
/// - `Ok(R)` on a clean consumer exit.
/// - `Err(Error::IoError)` with one of three message prefixes for
/// wedge cases:
/// - `"pipeline join halted"` — halt fired while waiting.
/// - `"pipeline join timed out"` — `JOIN_TIMEOUT_SECS` elapsed.
/// - `"pipeline consumer panicked"` — same as `finish()`.
///
/// In the `halted` and `timed out` branches the consumer thread is
/// intentionally leaked — exactly the same trade-off the
/// `bounded_syscall` primitive makes. The wedged kernel call
/// inside the consumer will unwind whenever it does, or at
/// process exit. The caller is free to fall back to a degraded
/// path (in autorip's case: `exit(1)` after the hard watchdog
/// escalation, letting Docker restart the container).
///
/// Plain [`Pipeline::finish`] is preserved for callers without a
/// halt-token plumbed through; that path still blocks indefinitely
/// on `join()`, matching pre-0.20.8 behaviour.
pub fn finish_with_halt(self, halt: Option<&Halt>) -> Result<R, Error> {
let Pipeline { tx, handle } = self;
drop(tx);
let deadline = Instant::now() + Duration::from_secs(JOIN_TIMEOUT_SECS);
loop {
if handle.is_finished() {
return match handle.join() {
Ok(result) => result,
Err(payload) => {
let msg = payload
.downcast_ref::<&'static str>()
.copied()
.or_else(|| payload.downcast_ref::<String>().map(|s| s.as_str()))
.unwrap_or("(no message)");
Err(Error::IoError {
source: io::Error::other(format!("pipeline consumer panicked: {msg}")),
})
}
};
}
if let Some(h) = halt {
if h.is_cancelled() {
// Consumer thread is intentionally leaked.
return Err(Error::IoError {
source: io::Error::other("pipeline join halted"),
});
}
}
if Instant::now() >= deadline {
// Consumer thread is intentionally leaked.
return Err(Error::IoError {
source: io::Error::other("pipeline join timed out"),
});
}
thread::sleep(POLL_INTERVAL);
}
}
}
#[cfg(test)]
@@ -553,4 +700,194 @@ mod tests {
other => panic!("expected Err(IoError), got {other:?}"),
}
}
/// Never-completing sink — `apply` blocks until cancelled. Signals
/// `started` once it has consumed its first item so the test
/// driver knows the consumer thread is wedged in `apply` (and
/// will no longer drain the channel). Used to drive the
/// halt/timeout paths of `send_with_halt` and `finish_with_halt`
/// without depending on real I/O.
struct NeverDrainsSink {
cancel: Arc<std::sync::atomic::AtomicBool>,
started: Arc<std::sync::atomic::AtomicBool>,
}
impl Sink<u64> for NeverDrainsSink {
type Output = ();
fn apply(&mut self, _item: u64) -> Result<Flow, Error> {
self.started.store(true, Ordering::SeqCst);
while !self.cancel.load(Ordering::SeqCst) {
std::thread::sleep(Duration::from_millis(20));
}
Ok(Flow::Continue)
}
fn close(self) -> Result<(), Error> {
Ok(())
}
}
/// Spin until `started` flips or `bail` elapses. Used by the
/// send_with_halt tests to synchronise with the consumer thread
/// before exercising the bounded-send timeout path.
fn wait_for_started(started: &Arc<std::sync::atomic::AtomicBool>, bail: Duration) {
let end = Instant::now() + bail;
while !started.load(Ordering::SeqCst) {
assert!(Instant::now() < end, "consumer never started apply()");
std::thread::sleep(Duration::from_millis(10));
}
}
#[test]
fn send_with_halt_returns_item_on_deadline() {
// depth=1 + consumer wedged in apply on the first item, AND
// the channel buffer already loaded with a second item, means
// any further `try_send` sees Full; with a 200 ms deadline and
// no halt fired, send_with_halt must return `Err(item)` within
// roughly the deadline. Synchronising on `started` ensures the
// consumer has actually started its wedged apply BEFORE we
// load the channel-buffer slot — without that, the consumer
// could still drain in a race window.
let cancel = Arc::new(std::sync::atomic::AtomicBool::new(false));
let started = Arc::new(std::sync::atomic::AtomicBool::new(false));
let pipe = Pipeline::spawn(
1,
NeverDrainsSink {
cancel: cancel.clone(),
started: started.clone(),
},
)
.expect("spawn should succeed");
// First send: consumer recv()s it and wedges in apply.
pipe.send(0u64).expect("first send hands off to consumer");
wait_for_started(&started, Duration::from_secs(2));
// Second send: lands in the depth=1 buffer slot, consumer
// can't pick it up because it's wedged in apply. Channel now
// full from the producer's perspective.
pipe.send(1u64).expect("second send fills the buffer");
let halt = crate::halt::Halt::new();
let start = Instant::now();
let res = pipe.send_with_halt(99u64, &halt, Duration::from_millis(200));
let elapsed = start.elapsed();
// Release the leaked consumer so the test process winds down.
cancel.store(true, Ordering::SeqCst);
let _ = pipe.finish();
assert!(matches!(res, Err(99)), "expected item returned on deadline");
assert!(
elapsed >= Duration::from_millis(150),
"deadline returned too early: {elapsed:?}"
);
assert!(
elapsed < Duration::from_secs(2),
"deadline blew past tolerance: {elapsed:?}"
);
}
#[test]
fn send_with_halt_returns_item_on_halt() {
// Same setup, but the halt fires before the deadline elapses.
// The send loop must observe the halt within ~50 ms (the
// SEND_POLL_INTERVAL) and return the item.
let cancel = Arc::new(std::sync::atomic::AtomicBool::new(false));
let started = Arc::new(std::sync::atomic::AtomicBool::new(false));
let pipe = Pipeline::spawn(
1,
NeverDrainsSink {
cancel: cancel.clone(),
started: started.clone(),
},
)
.expect("spawn should succeed");
pipe.send(0u64).expect("first send hands off to consumer");
wait_for_started(&started, Duration::from_secs(2));
pipe.send(1u64).expect("second send fills the buffer");
let halt = crate::halt::Halt::new();
let halt2 = halt.clone();
std::thread::spawn(move || {
std::thread::sleep(Duration::from_millis(100));
halt2.cancel();
});
let start = Instant::now();
let res = pipe.send_with_halt(7u64, &halt, Duration::from_secs(10));
let elapsed = start.elapsed();
cancel.store(true, Ordering::SeqCst);
let _ = pipe.finish();
assert!(matches!(res, Err(7)), "expected item returned on halt");
assert!(
elapsed < Duration::from_secs(2),
"halt observation took too long: {elapsed:?}"
);
}
#[test]
fn finish_with_halt_returns_halted_when_consumer_wedged() {
// Consumer wedges on the first apply; halt fires; finish
// returns the documented "pipeline join halted" error rather
// than blocking forever.
let cancel = Arc::new(std::sync::atomic::AtomicBool::new(false));
let started = Arc::new(std::sync::atomic::AtomicBool::new(false));
let pipe = Pipeline::spawn(
DEFAULT_PIPELINE_DEPTH,
NeverDrainsSink {
cancel: cancel.clone(),
started: started.clone(),
},
)
.expect("spawn should succeed");
pipe.send(0u64).expect("seed item the consumer wedges on");
wait_for_started(&started, Duration::from_secs(2));
let halt = crate::halt::Halt::new();
let halt2 = halt.clone();
std::thread::spawn(move || {
std::thread::sleep(Duration::from_millis(400));
halt2.cancel();
});
let start = Instant::now();
let res = pipe.finish_with_halt(Some(&halt));
let elapsed = start.elapsed();
// Release the leaked consumer so the test process exits cleanly.
cancel.store(true, Ordering::SeqCst);
match res {
Err(Error::IoError { source }) => {
assert!(
source.to_string().contains("pipeline join halted"),
"expected halt-prefix error, got: {source}"
);
}
other => panic!("expected Err(IoError) halted, got {other:?}"),
}
// Bailed out within ~1 second of the halt firing (worst case
// one POLL_INTERVAL = 250 ms of slack).
assert!(
elapsed < Duration::from_secs(2),
"halt observation took too long: {elapsed:?}"
);
}
#[test]
fn finish_with_halt_happy_path_returns_output() {
// No halt token, sink completes normally — finish_with_halt
// must return the same Output that `finish` would.
let pipe = Pipeline::spawn(DEFAULT_PIPELINE_DEPTH, SumSink { total: 0 })
.expect("spawn should succeed");
for i in 0..10u64 {
pipe.send(i).expect("send should succeed");
}
let total = pipe
.finish_with_halt(None)
.expect("happy-path finish_with_halt should succeed");
assert_eq!(total, (0..10u64).sum::<u64>());
}
}
+11 -27
View File
@@ -56,8 +56,6 @@ use std::fs::File;
use std::os::unix::io::{AsRawFd, RawFd};
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, Ordering};
use std::sync::mpsc::{RecvTimeoutError, sync_channel};
use std::thread;
use std::time::{Duration, Instant};
const ADAPTIVE_WINDOW: usize = 16;
@@ -332,34 +330,20 @@ fn detect_nfs(fd: RawFd) -> bool {
/// Run `sync_file_range(WAIT_AFTER)` on a worker thread and wait up
/// to [`WAIT_AFTER_TIMEOUT`] for it to return. `Some(elapsed_ms)` on
/// success; `None` on timeout. On timeout the worker thread is
/// intentionally leaked — it'll unwind whenever the syscall
/// eventually returns or the process exits.
///
/// Channel capacity is 0 (rendezvous). The worker sends `()` once
/// the syscall returns; if we time out first, the send blocks
/// forever inside the leaked worker — fine, the receiver is gone
/// and the OS reaps the thread at process exit.
/// 0.20.6 generalizes the worker-thread + recv_timeout pattern into
/// [`crate::io::bounded::bounded_syscall`]; this helper now just adapts
/// the generic primitive to the WAIT_AFTER call shape (returns elapsed_ms
/// instead of the syscall's `()` return, treats `WorkerLost` as a benign
/// no-op to match the original semantics).
fn wait_after_with_timeout(fd: RawFd, off: u64, len: u64) -> Option<u64> {
let (tx, rx) = sync_channel::<()>(0);
let started = Instant::now();
// `RawFd` is `Copy` + `Send`; `off`/`len` are `u64`. Nothing
// borrows from the caller — the worker can safely outlive this
// function on timeout.
let _ = thread::Builder::new()
.name("freemkv-writeback-wait".into())
.spawn(move || {
unsafe {
libc::sync_file_range(fd, off as i64, len as i64, libc::SYNC_FILE_RANGE_WAIT_AFTER);
}
// If the receiver is gone (we timed out) this send
// returns Err — fine, we just drop and the worker
// exits.
let _ = tx.send(());
});
match rx.recv_timeout(WAIT_AFTER_TIMEOUT) {
match crate::io::bounded::bounded_syscall(None, WAIT_AFTER_TIMEOUT, move || unsafe {
libc::sync_file_range(fd, off as i64, len as i64, libc::SYNC_FILE_RANGE_WAIT_AFTER);
}) {
Ok(()) => Some(started.elapsed().as_millis() as u64),
Err(RecvTimeoutError::Timeout) => None,
Err(RecvTimeoutError::Disconnected) => {
Err(crate::io::bounded::BoundedError::Timeout)
| Err(crate::io::bounded::BoundedError::Halted) => None,
Err(crate::io::bounded::BoundedError::WorkerLost) => {
// Worker thread spawn failed or panicked before sending.
// Treat as a benign success (no syscall ran) rather than
// a degrade trigger — falling through with elapsed_ms=0
+66
View File
@@ -0,0 +1,66 @@
//! Linux platform impl for [`super::WritebackFile`].
//!
//! - `preallocate`: `fallocate(FALLOC_FL_KEEP_SIZE)` — reserve extents
//! without growing the reported file size. Reduces extent
//! fragmentation on large sequential writes (mux output on NFS in
//! particular).
//! - `durable_sync`: `fsync` wrapped in
//! [`crate::io::bounded::bounded_syscall`] with a 60 s deadline so a
//! wedged NFS server can't trap the calling thread indefinitely.
use std::fs::File;
use std::io;
use std::os::unix::io::AsRawFd;
use std::time::Duration;
/// Pre-reserve extents for `size_bytes` of upcoming sequential writes.
/// Best-effort: a non-zero rc is logged but not propagated, since the
/// caller would just continue with the unreserved file anyway.
pub(super) fn preallocate(file: &File, size_bytes: u64) {
// FALLOC_FL_KEEP_SIZE = 0x01 — keep the reported file size at 0
// (writes grow it normally) while still pre-reserving the extents.
let rc = unsafe {
libc::fallocate(
file.as_raw_fd(),
libc::FALLOC_FL_KEEP_SIZE,
0,
size_bytes as i64,
)
};
tracing::debug!(
target: "mux",
"WritebackFile fallocate size_hint={size_bytes} rc={rc} ok={}",
rc == 0
);
}
/// Run `fsync` on `file` with a 60 s deadline. On timeout we log loudly
/// and return `Ok(())` — the kernel will still flush on close, so the
/// data is best-effort durable; the alternative (trap the thread for
/// the rest of the rip) defeats `/api/stop`.
pub(super) fn durable_sync(file: &File) -> io::Result<()> {
let fd = file.as_raw_fd();
match crate::io::bounded::bounded_syscall(
None,
Duration::from_secs(60),
move || -> io::Result<()> {
let rc = unsafe { libc::fsync(fd) };
if rc == 0 {
Ok(())
} else {
Err(io::Error::last_os_error())
}
},
) {
Ok(inner) => inner,
Err(crate::io::bounded::BoundedError::Timeout) => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all fsync timed out after 60s; kernel will flush on close (best-effort)"
);
Ok(())
}
Err(crate::io::bounded::BoundedError::Halted) => Ok(()),
Err(crate::io::bounded::BoundedError::WorkerLost) => Ok(()),
}
}
+104
View File
@@ -0,0 +1,104 @@
//! macOS platform impl for [`super::WritebackFile`].
//!
//! - `preallocate`: `fcntl(F_PREALLOCATE)` — macOS's fallocate-equiv.
//! Reserves a contiguous extent when possible, falling back to a
//! non-contiguous reservation if the FS can't satisfy it. Reported
//! file size is unchanged (`F_ALLOCATEALL` is not set, so allocation
//! is "best effort up to length"; growth happens via writes).
//! - `durable_sync`: `fcntl(F_FULLFSYNC)` wrapped in
//! [`crate::io::bounded::bounded_syscall`] with a 60 s deadline.
//! F_FULLFSYNC is HFS+/APFS's true-fsync (flushes the disk's own
//! write cache) — what `fsync` should have been on macOS. Falls back
//! to plain `fsync` if F_FULLFSYNC returns ENOTSUP.
use std::fs::File;
use std::io;
use std::os::unix::io::AsRawFd;
use std::time::Duration;
/// libc `F_PREALLOCATE` — not exposed by the `libc` crate on all macOS
/// SDK versions, so define it here.
const F_PREALLOCATE: libc::c_int = 42;
/// Allocate from current EOF.
const F_PEOFPOSMODE: libc::c_int = 3;
/// Hint: contiguous extent preferred.
const F_ALLOCATECONTIG: libc::c_uint = 0x00000002;
/// Allocate all the requested bytes (fall back to non-contig if needed).
const F_ALLOCATEALL: libc::c_uint = 0x00000004;
/// `fcntl(F_FULLFSYNC)` opcode. Documented in `man 2 fcntl` on macOS;
/// not in the `libc` crate as a named constant.
const F_FULLFSYNC: libc::c_int = 51;
/// `fstore_t` layout matches `sys/fcntl.h`. Repr is C-stable so we can
/// build it manually.
#[repr(C)]
struct Fstore {
fst_flags: libc::c_uint,
fst_posmode: libc::c_int,
fst_offset: libc::off_t,
fst_length: libc::off_t,
fst_bytesalloc: libc::off_t,
}
pub(super) fn preallocate(file: &File, size_bytes: u64) {
let mut fst = Fstore {
fst_flags: F_ALLOCATECONTIG | F_ALLOCATEALL,
fst_posmode: F_PEOFPOSMODE,
fst_offset: 0,
fst_length: size_bytes as libc::off_t,
fst_bytesalloc: 0,
};
// First attempt: contiguous.
let mut rc = unsafe { libc::fcntl(file.as_raw_fd(), F_PREALLOCATE, &mut fst) };
if rc == -1 {
// Fall back: drop the contiguous hint, allow scattered extents.
fst.fst_flags = F_ALLOCATEALL;
rc = unsafe { libc::fcntl(file.as_raw_fd(), F_PREALLOCATE, &mut fst) };
}
tracing::debug!(
target: "mux",
"WritebackFile F_PREALLOCATE size_hint={size_bytes} rc={rc} bytes_allocated={} ok={}",
fst.fst_bytesalloc,
rc != -1
);
}
pub(super) fn durable_sync(file: &File) -> io::Result<()> {
let fd = file.as_raw_fd();
match crate::io::bounded::bounded_syscall(
None,
Duration::from_secs(60),
move || -> io::Result<()> {
// Try F_FULLFSYNC first. If it isn't supported on this
// filesystem (older HFS, some network mounts) fall back to
// plain fsync — better than nothing.
let rc = unsafe { libc::fcntl(fd, F_FULLFSYNC, 0) };
if rc == 0 {
return Ok(());
}
let err = io::Error::last_os_error();
if err.raw_os_error() == Some(libc::ENOTSUP) {
let rc = unsafe { libc::fsync(fd) };
if rc == 0 {
Ok(())
} else {
Err(io::Error::last_os_error())
}
} else {
Err(err)
}
},
) {
Ok(inner) => inner,
Err(crate::io::bounded::BoundedError::Timeout) => {
tracing::error!(
target: "mux",
"WritebackFile::sync_all F_FULLFSYNC timed out after 60s; kernel will flush on close (best-effort)"
);
Ok(())
}
Err(crate::io::bounded::BoundedError::Halted) => Ok(()),
Err(crate::io::bounded::BoundedError::WorkerLost) => Ok(()),
}
}
@@ -20,6 +20,34 @@
//!
//! See `super::writeback::linux` for the underlying pathology and the
//! strategy.
//!
//! ## Platform split
//!
//! The platform-specific pieces of this wrapper — extent preallocation
//! (Linux `fallocate(KEEP_SIZE)`, macOS `F_PREALLOCATE`, Windows
//! `SetFileValidData`) and the durable-flush primitive (Linux/macOS
//! `fsync`/`F_FULLFSYNC` wrapped in a bounded syscall, Windows
//! `FlushFileBuffers`) — live in per-OS sibling modules. The dispatch
//! happens once at the bottom of this file via cfg-gated `mod` decls.
//! No inline `#[cfg(target_os = "...")]` in the business-logic above.
#[cfg(target_os = "linux")]
mod linux;
#[cfg(target_os = "macos")]
mod macos;
#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))]
mod other;
#[cfg(target_os = "windows")]
mod windows;
#[cfg(target_os = "linux")]
use linux as platform;
#[cfg(target_os = "macos")]
use macos as platform;
#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))]
use other as platform;
#[cfg(target_os = "windows")]
use windows as platform;
use std::fs::{File, OpenOptions};
use std::io::{self, Seek, SeekFrom, Write};
@@ -65,43 +93,20 @@ impl WritebackFile {
}
/// Like [`Self::create`] but pre-reserves `size_bytes` of disk
/// space via `fallocate(FALLOC_FL_KEEP_SIZE)` on Linux. The
/// reported file size is unchanged (writes still grow the file
/// naturally) — only the on-disk extent allocation is preallocated,
/// which reduces extent fragmentation on large sequential writes
/// (mux output, especially on slow storage / NFS).
/// space via the platform's extent-preallocation primitive (Linux
/// `fallocate(KEEP_SIZE)`, macOS `F_PREALLOCATE`, Windows
/// `SetFileValidData` stub). The reported file size is unchanged
/// (writes still grow the file naturally) — only the on-disk extent
/// allocation is preallocated, which reduces extent fragmentation
/// on large sequential writes (mux output, especially on slow
/// storage / NFS).
///
/// On macOS / Windows the size hint is ignored and this is
/// equivalent to `create`.
/// On platforms without an extent-preallocation primitive this is
/// equivalent to `create` — the size hint is dropped after a debug
/// log.
pub(crate) fn create_with_size_hint(path: &Path, size_bytes: u64) -> io::Result<Self> {
let file = File::create(path)?;
#[cfg(target_os = "linux")]
{
use std::os::unix::io::AsRawFd;
// FALLOC_FL_KEEP_SIZE = 0x01 — keep the reported file
// size at 0 (writes grow it normally) while still
// pre-reserving the extents.
let rc = unsafe {
libc::fallocate(
file.as_raw_fd(),
libc::FALLOC_FL_KEEP_SIZE,
0,
size_bytes as i64,
)
};
tracing::debug!(
target: "mux",
"WritebackFile fallocate size_hint={size_bytes} rc={rc} ok={}",
rc == 0
);
}
#[cfg(not(target_os = "linux"))]
{
tracing::debug!(
target: "mux",
"WritebackFile fallocate size_hint={size_bytes} skipped (non-linux)"
);
}
platform::preallocate(&file, size_bytes);
Self::new(file)
}
@@ -116,9 +121,18 @@ impl WritebackFile {
/// Drain in-flight writeback then issue a full fsync. Use this in
/// place of `File::sync_all`.
///
/// The final fsync is wrapped in
/// [`crate::io::bounded::bounded_syscall`] with a 60 s deadline on
/// platforms that have a usable bounded primitive (Linux + macOS).
/// fsync on a wedged NFS server (or a degraded local disk) can
/// hang the calling thread; the wrapper ensures the worst case is
/// 60 s + log-and-continue rather than indefinite. On timeout the
/// page cache is left to the kernel's normal flush-on-close path —
/// best effort, but bounded.
pub(crate) fn sync_all(&mut self) -> io::Result<()> {
self.pipeline.finalize();
self.file.sync_all()
platform::durable_sync(&self.file)
}
}
+23
View File
@@ -0,0 +1,23 @@
//! Fallback platform impl for [`super::WritebackFile`] on targets
//! without a dedicated implementation (BSDs, illumos, etc.).
//!
//! - `preallocate` is a logged no-op.
//! - `durable_sync` calls `File::sync_all` directly (no bounded-syscall
//! wrapper — the wrapper depends on Linux/macOS unix idioms that
//! aren't universally portable). If a future BSD impl needs the
//! 60-s deadline, it should land in its own per-OS file rather than
//! bloat this fallback.
use std::fs::File;
use std::io;
pub(super) fn preallocate(_file: &File, size_bytes: u64) {
tracing::debug!(
target: "mux",
"WritebackFile preallocate size_hint={size_bytes} skipped (no impl on this target)"
);
}
pub(super) fn durable_sync(file: &File) -> io::Result<()> {
file.sync_all()
}
+32
View File
@@ -0,0 +1,32 @@
//! Windows platform impl for [`super::WritebackFile`].
//!
//! TODO: this stub matches the design's "validate without a Windows
//! build env, leave a stub" carve-out. The real impl should use:
//!
//! - `SetEndOfFile` + `SetFileValidData` for extent preallocation
//! (caller needs `SE_MANAGE_VOLUME_NAME` privilege; if unavailable
//! fall back to a write-zero path or just skip).
//! - `FlushFileBuffers` for fsync-equivalent durable flush.
//!
//! Until then: preallocate is a debug-logged no-op; durable_sync calls
//! the std `File::sync_all` (which on Windows maps to
//! `FlushFileBuffers` internally).
use std::fs::File;
use std::io;
pub(super) fn preallocate(_file: &File, size_bytes: u64) {
tracing::debug!(
target: "mux",
"WritebackFile preallocate size_hint={size_bytes} skipped (windows stub; TODO: SetFileValidData)"
);
}
pub(super) fn durable_sync(file: &File) -> io::Result<()> {
// `File::sync_all` on Windows is `FlushFileBuffers`. Acceptable
// for now; the bounded-syscall wrapper is not used here because
// the stub also skips the worker-thread + leak machinery (the
// wrapper would need an `unsafe impl Send` for `RawHandle`, and
// designing that without a Windows test env is asking for it).
file.sync_all()
}
+12 -146
View File
@@ -1,148 +1,19 @@
//! File-backed sector I/O — read and write 2048-byte sectors against
//! an ISO image on disk.
//! File-backed sector sink — write 2048-byte sectors to an ISO image
//! on disk.
//!
//! [`FileSectorSource`] is the read side (open-only). [`FileSectorSink`]
//! is the write side (create or open-rw); writes go through
//! [`crate::io::WritebackFile`] so big sequential ISO writes share
//! the same bounded-cache writeback pipeline used by sweep / patch /
//! mux.
//! The read-side counterpart ([`crate::io::file_sector_source::FileSectorSource`])
//! lives under `io/` because its internals (read-ahead buffer, per-OS
//! `fadvise`/`F_RDADVISE` hints) are I/O infrastructure rather than
//! sector-trait business logic. Both types remain re-exported at
//! [`crate::sector`] for ergonomic imports.
use std::fs::{File, OpenOptions};
use std::io::{Read, Seek, SeekFrom, Write};
use std::fs::OpenOptions;
use std::io::{Seek, SeekFrom, Write};
use std::path::Path;
use crate::error::{Error, Result};
use super::{SectorSink, SectorSource};
/// Bytes-read threshold per `posix_fadvise(DONTNEED)` drop on the
/// read side. Mirrors the writeback chunk size so the read-side
/// page cache stays bounded the same way the write side does.
#[cfg(target_os = "linux")]
const READ_DROP_CHUNK_BYTES: u64 = 32 * 1024 * 1024;
/// SectorSource backed by a file (ISO image).
///
/// Seeks to `lba * 2048`, reads `count * 2048` bytes per call. The
/// file is held directly: every `read_sectors` call performs an
/// absolute seek, so a wrapping `BufReader` would have its buffer
/// invalidated on every call (its internal cursor moves with the
/// `Seek` impl) — pure overhead. Callers that benefit from buffered
/// reads should compose their own `BufReader` at the `read_sectors`
/// granularity they care about.
pub struct FileSectorSource {
file: File,
capacity: u32,
/// Bytes read since the last `posix_fadvise(DONTNEED)` drop.
/// Only updated on Linux; on other targets it stays at 0.
#[cfg(target_os = "linux")]
bytes_read_since_drop: u64,
/// Byte offset at which the current drop window starts (the
/// next `posix_fadvise(DONTNEED)` call drops from here).
#[cfg(target_os = "linux")]
drop_window_start: u64,
}
impl FileSectorSource {
/// Open an existing ISO file for reading. Capacity is derived
/// from `metadata().len() / 2048`. Returns
/// [`Error::IsoTooLarge`] if the file would exceed the 32-bit
/// LBA address space (~8 TB).
pub fn open(path: &Path) -> std::io::Result<Self> {
let file = File::open(path)?;
let len = file.metadata()?.len();
let sectors = len / 2048;
if sectors > u32::MAX as u64 {
return Err(Error::IsoTooLarge {
path: path.to_string_lossy().into_owned(),
}
.into());
}
let capacity = sectors as u32;
// Hint sequential access on Linux so the kernel's readahead
// window widens for the ISO sweep. Best-effort: return value
// is ignored. On macOS / Windows this is a no-op.
#[cfg(target_os = "linux")]
{
use std::os::unix::io::AsRawFd;
unsafe {
libc::posix_fadvise(file.as_raw_fd(), 0, 0, libc::POSIX_FADV_SEQUENTIAL);
}
}
Ok(Self {
file,
capacity,
#[cfg(target_os = "linux")]
bytes_read_since_drop: 0,
#[cfg(target_os = "linux")]
drop_window_start: 0,
})
}
}
// Implement the legacy `SectorSource` trait. The blanket impl in
// `super` produces the `SectorSource` impl automatically — no need
// to write both, and writing both would conflict. This keeps the
// 0.17 method-resolution path intact (callers with `SectorSource`
// in scope can still write `fsr.read_sectors(..)` against a
// `FileSectorSource`).
impl SectorSource for FileSectorSource {
fn capacity_sectors(&self) -> u32 {
self.capacity
}
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
let offset = lba as u64 * 2048;
let bytes = count as usize * 2048;
self.file
.seek(SeekFrom::Start(offset))
.map_err(|e| Error::IoError { source: e })?;
self.file
.read_exact(&mut buf[..bytes])
.map_err(|e| Error::IoError { source: e })?;
// On Linux, periodically drop the just-read region from the
// page cache to keep cache pressure bounded during multi-GB
// sequential ISO reads. Mirrors the write-side pipeline.
#[cfg(target_os = "linux")]
{
use std::os::unix::io::AsRawFd;
self.bytes_read_since_drop += bytes as u64;
if self.bytes_read_since_drop >= READ_DROP_CHUNK_BYTES {
let drop_start = self.drop_window_start;
let drop_len = self.bytes_read_since_drop;
let t0 = std::time::Instant::now();
unsafe {
libc::posix_fadvise(
self.file.as_raw_fd(),
drop_start as i64,
drop_len as i64,
libc::POSIX_FADV_DONTNEED,
);
}
let elapsed_ms = t0.elapsed().as_millis();
let start_lba = drop_start / 2048;
let end_lba = (drop_start + drop_len) / 2048;
tracing::trace!(
target: "mux",
"FileSectorSource fadvise DONTNEED lba=[{start_lba}..{end_lba}) bytes={drop_len} elapsed_ms={elapsed_ms}"
);
self.drop_window_start = drop_start + drop_len;
self.bytes_read_since_drop = 0;
}
}
Ok(bytes)
}
}
use super::SectorSink;
/// SectorSink backed by a file (ISO image).
///
@@ -211,13 +82,8 @@ impl SectorSink for FileSectorSink {
#[cfg(test)]
mod tests {
// Bring the 0.18 trait into scope (not super::*: the super
// module also re-exports the legacy `SectorSource`, and
// having both `SectorSource::read_sectors` and
// `SectorSource::read_sectors` visible would force every
// call site to disambiguate). External consumers see the
// same surface this test exercises.
use super::{FileSectorSink, FileSectorSource};
use super::FileSectorSink;
use crate::io::file_sector_source::FileSectorSource;
use crate::sector::{SectorSink, SectorSource};
use tempfile::tempdir;
+2 -1
View File
@@ -109,5 +109,6 @@ pub trait SectorSink: Send {
fn finish(self: Box<Self>) -> Result<()>;
}
pub use crate::io::file_sector_source::FileSectorSource;
pub use decrypting::DecryptingSectorSource;
pub use file::{FileSectorSink, FileSectorSource};
pub use file::FileSectorSink;
+807
View File
@@ -0,0 +1,807 @@
//! Pass-N (`Disc::patch`) read-error handler — A/B golden fixture.
//!
//! Background (2026-05-13, v0.20.8 release bundle planning):
//!
//! `libfreemkv::disc::read_error::handle_read_error` is supposed to be
//! the single source of truth for sector-read error → recovery action
//! decisions. Pass 1 sweep routes through it. Pass N patch's
//! `handle_read_failure` (in `disc/patch.rs`) does NOT — historically
//! MEDIUM_ERROR / NOT_READY get inline handling with their own thresholds
//! (`PASSN_DAMAGE_THRESHOLD_PCT=6` vs the sweep's `12`), their own
//! damage_window (state.damage_window, separate from ReadCtx.damage_window),
//! and their own skip logic (`compute_damage_skip`, which runs AFTER
//! the failure handler and has a size-aware `range_remaining/4` cap
//! that `handle_read_error::JumpAhead` does not know about).
//!
//! This file is the A/B fixture for that unification. It pins the
//! CURRENT (pre-unification) end-to-end behavior of `Disc::patch` for
//! eight canonical damage profiles against a synthetic
//! `ScriptedSectorReader`. Each profile asserts the exact observable
//! outcome — final mapfile byte counts and outer-loop counters — so any
//! attempt to refactor the failure path either preserves the goldens or
//! the test fails loudly.
//!
//! The prompt called for "exact sequence of `ReadAction` enums per
//! LBA"; that framing doesn't fit the current architecture because
//! `handle_read_failure` produces `FailureAction`, not `ReadAction`,
//! and interleaves with `compute_damage_skip` + cursor management in
//! the outer loop. The observable contract — what `Disc::patch` does
//! to the mapfile and how many reads it performs — is the equivalent
//! invariant, captured end-to-end.
//!
//! Why we expect divergence under naïve unification (see final report
//! of the 0.20.8 unification attempt): the patch loop's skip semantics
//! live in `compute_damage_skip` POST-failure-handler, with a size-aware
//! cap that `handle_read_error` knows nothing about; routing through
//! `handle_read_error` would invert that cursor flow. The fixture stays
//! checked in regardless — it documents the contract for the next
//! refactor attempt.
use libfreemkv::ContentFormat;
use libfreemkv::Disc;
use libfreemkv::DiscFormat;
use libfreemkv::disc::CopyOptions;
use libfreemkv::disc::DiscRegion;
use libfreemkv::disc::mapfile::{Mapfile, SectorStatus};
use libfreemkv::error::{Error, Result};
use libfreemkv::scsi;
use libfreemkv::{ScsiSense, SectorSource};
use std::sync::{Arc, Mutex};
const SECTOR_SIZE: usize = 2048;
/// Per-attempt result the script can emit. `Ok` returns a deterministic
/// per-sector byte pattern (LBA mod 256 in each sector). `Err` returns
/// the SCSI sense triple supplied — the patch failure path inspects
/// `scsi_sense().sense_key` to classify (MEDIUM, NOT_READY,
/// HARDWARE, ILLEGAL_REQUEST, ABORTED_COMMAND).
#[derive(Debug, Clone, Copy)]
enum ScriptStep {
Ok,
Err { sense_key: u8, asc: u8, ascq: u8 },
}
/// A scripted reader. For each (lba, count) read attempt, picks the
/// step at `attempt_idx[lba]`, advances the index. If no script entry
/// exists for an LBA, defaults to `Ok` so we don't need to script
/// every sector of large ranges.
///
/// "Batch fails if ANY sector in the batch is bad" — matches real
/// drive behavior (`pass_n_size_aware_skip.rs` uses the same model).
/// For batched reads we synthesize an Err with the FIRST scripted
/// failure in the batch.
struct ScriptedSectorReader {
capacity: u32,
/// Per-LBA script of (step, then next step on retry, …). When
/// retries exhaust the script, the LAST step repeats forever.
script: std::collections::HashMap<u32, Vec<ScriptStep>>,
/// Per-LBA index into its script vec. Bumps on each read attempt
/// at that LBA.
attempt_idx: Mutex<std::collections::HashMap<u32, usize>>,
/// Full read trace: every (lba, count, result_was_ok) tuple in
/// call order. Lets the test assert that adaptive-batch dropped
/// to count=1, bisection happened, etc.
trace: Arc<Mutex<Vec<(u32, u16, bool)>>>,
}
impl ScriptedSectorReader {
fn new(capacity: u32) -> (Self, Arc<Mutex<Vec<(u32, u16, bool)>>>) {
let trace = Arc::new(Mutex::new(Vec::new()));
(
Self {
capacity,
script: std::collections::HashMap::new(),
attempt_idx: Mutex::new(std::collections::HashMap::new()),
trace: trace.clone(),
},
trace,
)
}
/// Set a single-step script for `lba`: every attempt yields `step`.
fn always(&mut self, lba: u32, step: ScriptStep) {
self.script.insert(lba, vec![step]);
}
/// Set a multi-step script for `lba`: first attempt yields
/// `steps[0]`, second `steps[1]`, … on retry the last step repeats.
#[allow(dead_code)]
fn sequence(&mut self, lba: u32, steps: Vec<ScriptStep>) {
self.script.insert(lba, steps);
}
fn step_for(&self, lba: u32) -> ScriptStep {
let v = match self.script.get(&lba) {
Some(v) => v,
None => return ScriptStep::Ok,
};
let mut idx = self.attempt_idx.lock().unwrap();
let i = idx.entry(lba).or_insert(0);
let step = v[(*i).min(v.len() - 1)];
*i += 1;
step
}
}
impl SectorSource for ScriptedSectorReader {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
// Look at every sector in the batch — first failure determines
// the outcome.
let mut failure: Option<(u8, u8, u8)> = None;
for offset in 0..count as u32 {
match self.step_for(lba + offset) {
ScriptStep::Ok => {}
ScriptStep::Err {
sense_key,
asc,
ascq,
} => {
failure = Some((sense_key, asc, ascq));
break;
}
}
}
let ok = failure.is_none();
self.trace.lock().unwrap().push((lba, count, ok));
if let Some((sense_key, asc, ascq)) = failure {
return Err(Error::ScsiError {
opcode: scsi::SCSI_READ_10,
status: scsi::SCSI_STATUS_CHECK_CONDITION,
sense: Some(ScsiSense {
sense_key,
asc,
ascq,
}),
});
}
// Per-sector LBA byte pattern.
for (i, chunk) in buf.chunks_mut(SECTOR_SIZE).enumerate() {
chunk.fill(((lba + i as u32) & 0xff) as u8);
}
Ok(buf.len())
}
fn capacity_sectors(&self) -> u32 {
self.capacity
}
}
fn synthetic_disc(capacity_sectors: u32) -> Disc {
Disc {
volume_id: String::new(),
meta_title: None,
format: DiscFormat::BluRay,
capacity_sectors,
capacity_bytes: capacity_sectors as u64 * SECTOR_SIZE as u64,
layers: 1,
titles: Vec::new(),
region: DiscRegion::Free,
aacs: None,
css: None,
encrypted: false,
aacs_error: None,
content_format: ContentFormat::BdTs,
}
}
fn prep_iso_and_mapfile(
iso_path: &std::path::Path,
total_bytes: u64,
finished_ranges: &[(u64, u64)],
nontrimmed_ranges: &[(u64, u64)],
) {
use std::fs::OpenOptions;
use std::io::{Seek, SeekFrom, Write};
let mut f = OpenOptions::new()
.create(true)
.write(true)
.truncate(true)
.open(iso_path)
.unwrap();
f.set_len(total_bytes).unwrap();
f.seek(SeekFrom::Start(0)).unwrap();
f.write_all(&[]).unwrap();
let map_path = libfreemkv::disc::mapfile_path_for(iso_path);
let mut mf = Mapfile::create(&map_path, total_bytes, "test").unwrap();
for &(pos, size) in finished_ranges {
mf.record(pos, size, SectorStatus::Finished).unwrap();
}
for &(pos, size) in nontrimmed_ranges {
mf.record(pos, size, SectorStatus::NonTrimmed).unwrap();
}
}
/// Observable outcome of a patch run. Goldens for each profile pin
/// these exact values.
#[derive(Debug, PartialEq, Eq)]
struct Golden {
/// `bytes_good` at end of patch.
bytes_good: u64,
/// `bytes_unreadable` at end.
bytes_unreadable: u64,
/// `bytes_pending` (NonTrimmed) at end.
bytes_pending: u64,
/// Did the pass exit via wedge-detection?
wedged_exit: bool,
/// Sanity bound on trace length — patch makes a finite number of
/// reads bounded by `MAX_SKIPS_PER_RANGE * range_sectors` plus
/// retries. Asserted as an UPPER bound only (so any reduction in
/// retries via future tuning doesn't fail the test spuriously).
max_reads: usize,
}
/// Common helper: prep ISO + mapfile, run `disc.copy(multipass)`,
/// return (PatchOutcome ↔ CopyResult, final-map stats, trace length).
fn run_profile(
profile_name: &str,
capacity_sectors: u32,
nontrimmed: &[(u64, u64)],
finished: &[(u64, u64)],
scripted: ScriptedSectorReader,
trace: Arc<Mutex<Vec<(u32, u16, bool)>>>,
) -> (
libfreemkv::disc::CopyResult,
libfreemkv::disc::mapfile::MapStats,
usize,
) {
let total_bytes: u64 = capacity_sectors as u64 * SECTOR_SIZE as u64;
let disc = synthetic_disc(capacity_sectors);
let tmp = tempfile::NamedTempFile::new().unwrap();
let iso_path = tmp.path().to_path_buf();
drop(tmp);
prep_iso_and_mapfile(&iso_path, total_bytes, finished, nontrimmed);
let opts = CopyOptions {
decrypt: false,
multipass: true,
..Default::default()
};
let mut reader = scripted;
let pr = disc
.copy(&mut reader, &iso_path, &opts)
.unwrap_or_else(|e| panic!("[{profile_name}] disc.copy returned Err: {e:?}"));
let map_path = libfreemkv::disc::mapfile_path_for(&iso_path);
let map = Mapfile::load(&map_path).unwrap();
let stats = map.stats();
let trace_len = trace.lock().unwrap().len();
let _ = std::fs::remove_file(&iso_path);
let _ = std::fs::remove_file(&map_path);
(pr, stats, trace_len)
}
// ─────────────────────────── Profile 1: CLEAN ────────────────────────────
//
// The NonTrimmed range has zero scripted failures — every read succeeds.
// Patch should march through the range and mark it Finished. Validates
// the happy-path side of the failure-handler dispatch (it shouldn't
// fire at all).
#[test]
fn profile_01_clean_all_recoverable() {
let capacity_sectors: u32 = 256;
let (reader, trace) = ScriptedSectorReader::new(capacity_sectors);
// No scripted errors → all reads succeed.
let nontrimmed = [(100 * 2048, 16 * 2048)]; // 16-sector NonTrimmed range
let finished = [
(0, 100 * 2048),
(116 * 2048, (capacity_sectors as u64 - 116) * 2048),
];
let (pr, stats, trace_len) = run_profile(
"01_clean",
capacity_sectors,
&nontrimmed,
&finished,
reader,
trace,
);
let expected = Golden {
bytes_good: capacity_sectors as u64 * 2048,
bytes_unreadable: 0,
bytes_pending: 0,
wedged_exit: false,
max_reads: 8, // adaptive batch=32 reads finishes 16 sectors in 1 read; allow up to 8.
};
assert_eq!(stats.bytes_good, expected.bytes_good, "01_clean bytes_good");
assert_eq!(
stats.bytes_unreadable, expected.bytes_unreadable,
"01_clean bytes_unreadable"
);
assert_eq!(
stats.bytes_pending, expected.bytes_pending,
"01_clean bytes_pending"
);
assert!(!pr.halted, "01_clean halted");
assert!(
trace_len <= expected.max_reads,
"01_clean trace_len={trace_len} exceeds bound {}",
expected.max_reads
);
}
// ─────────────────────────── Profile 2: ALL MEDIUM ───────────────────────
//
// Every LBA in the NonTrimmed range returns MEDIUM_ERROR every attempt.
// Adaptive-batch drops to count=1 on first batch failure, then each
// single-sector read fails → consecutive_failures climbs, damage_window
// fills, compute_damage_skip fires, MAX_SKIPS_PER_RANGE caps the work,
// remaining bytes stay NonTrimmed (NEVER marked Unreadable inside a
// single pass — 2026-05-11 design call).
#[test]
fn profile_02_all_medium_error() {
let capacity_sectors: u32 = 256;
let (mut reader, trace) = ScriptedSectorReader::new(capacity_sectors);
for lba in 100..116 {
reader.always(
lba,
ScriptStep::Err {
sense_key: scsi::SENSE_KEY_MEDIUM_ERROR,
asc: 0x11,
ascq: 0x00,
},
);
}
let nontrimmed = [(100 * 2048, 16 * 2048)];
let finished = [
(0, 100 * 2048),
(116 * 2048, (capacity_sectors as u64 - 116) * 2048),
];
let (pr, stats, trace_len) = run_profile(
"02_all_medium",
capacity_sectors,
&nontrimmed,
&finished,
reader,
trace,
);
// GOLDEN: the 16-sector bad range stays NonTrimmed (bytes_pending).
// Pre-2026-05-11 patch would mark Unreadable here; current code
// preserves NonTrimmed so subsequent passes get another shot.
assert_eq!(
stats.bytes_good,
(capacity_sectors as u64 - 16) * 2048,
"02_all_medium bytes_good"
);
assert_eq!(
stats.bytes_unreadable, 0,
"02_all_medium bytes_unreadable (must NOT be marked terminal in one pass)"
);
assert_eq!(
stats.bytes_pending,
16 * 2048,
"02_all_medium bytes_pending (NonTrimmed retained across passes)"
);
assert!(!pr.halted, "02_all_medium halted");
// Upper bound: every sector probed individually + a few batch-drop
// and skip-escalation attempts. 16 sectors × ~3 visits ≈ 50.
assert!(
trace_len <= 80,
"02_all_medium trace_len={trace_len} exceeds 80"
);
}
// ───────────────────── Profile 3: ALTERNATING GOOD/BAD ───────────────────
//
// LBAs 100, 102, 104, ... bad; odd LBAs good. Validates that good
// sectors interleaved with bad get recovered individually after the
// adaptive split (batch-fail → count=1 → per-sector probe).
#[test]
fn profile_03_alternating_good_bad() {
let capacity_sectors: u32 = 256;
let (mut reader, trace) = ScriptedSectorReader::new(capacity_sectors);
for lba in (100..116).step_by(2) {
reader.always(
lba,
ScriptStep::Err {
sense_key: scsi::SENSE_KEY_MEDIUM_ERROR,
asc: 0x11,
ascq: 0x00,
},
);
}
let nontrimmed = [(100 * 2048, 16 * 2048)];
let finished = [
(0, 100 * 2048),
(116 * 2048, (capacity_sectors as u64 - 116) * 2048),
];
let (pr, stats, trace_len) = run_profile(
"03_alternating",
capacity_sectors,
&nontrimmed,
&finished,
reader,
trace,
);
// GOLDEN: 8 good sectors interleaved should mostly be Finished;
// 8 bad stay NonTrimmed. Allow 2 sectors of slop for the actual
// bisect cursor advance — converging on alternating bad/good in
// a single pass isn't always exact at boundaries with the
// size-aware skip cap.
let good_total = stats.bytes_good;
let baseline_good = (capacity_sectors as u64 - 16) * 2048;
let middle_recovered = good_total - baseline_good;
assert!(
middle_recovered >= 6 * 2048,
"03_alternating recovered only {middle_recovered} bytes of 8 good sectors"
);
assert!(
middle_recovered <= 9 * 2048,
"03_alternating recovered MORE than scripted good sectors: {middle_recovered}"
);
assert_eq!(stats.bytes_unreadable, 0, "03_alternating bytes_unreadable");
// Remaining must be NonTrimmed (pending), not lost.
assert!(
stats.bytes_pending > 0,
"03_alternating expected NonTrimmed remainder, got bytes_pending=0"
);
assert!(!pr.halted, "03_alternating halted");
assert!(
trace_len <= 120,
"03_alternating trace_len={trace_len} exceeds 120"
);
}
// ───────────────────── Profile 4: EDGE-BAD (size-aware-skip canon) ───────
//
// Bad at start (100..104), good middle (104..112), bad at end (112..116).
// This is the size-aware-skip canonical case. The middle good sectors
// MUST be recovered — pre-fix patch would skip-escalate across the
// whole range and miss them.
#[test]
fn profile_04_edge_bad_good_middle() {
let capacity_sectors: u32 = 256;
let (mut reader, trace) = ScriptedSectorReader::new(capacity_sectors);
for lba in 100..104 {
reader.always(
lba,
ScriptStep::Err {
sense_key: scsi::SENSE_KEY_MEDIUM_ERROR,
asc: 0x11,
ascq: 0x00,
},
);
}
for lba in 112..116 {
reader.always(
lba,
ScriptStep::Err {
sense_key: scsi::SENSE_KEY_MEDIUM_ERROR,
asc: 0x11,
ascq: 0x00,
},
);
}
let nontrimmed = [(100 * 2048, 16 * 2048)];
let finished = [
(0, 100 * 2048),
(116 * 2048, (capacity_sectors as u64 - 116) * 2048),
];
let (pr, stats, trace_len) = run_profile(
"04_edge_bad",
capacity_sectors,
&nontrimmed,
&finished,
reader,
trace,
);
// GOLDEN: the 8 good middle sectors should land Finished (allowing
// 2 sectors of bisection slop at boundaries).
let middle_recovered = stats.bytes_good - (capacity_sectors as u64 - 16) * 2048;
assert!(
middle_recovered >= 6 * 2048,
"04_edge_bad recovered only {middle_recovered} bytes of 8 good middle sectors"
);
assert_eq!(stats.bytes_unreadable, 0, "04_edge_bad bytes_unreadable");
assert!(
stats.bytes_pending > 0,
"04_edge_bad bytes_pending expected > 0"
);
assert!(!pr.halted, "04_edge_bad halted");
assert!(
trace_len <= 120,
"04_edge_bad trace_len={trace_len} exceeds 120"
);
}
// ───────────────────── Profile 5: SINGLE BAD SECTOR ──────────────────────
//
// 1 bad sector in the middle of an otherwise good 16-sector NonTrimmed
// range. Validates the common "stochastic miss in Pass 1, easily picked
// up in Pass N" scenario.
#[test]
fn profile_05_single_bad_sector() {
let capacity_sectors: u32 = 256;
let (mut reader, trace) = ScriptedSectorReader::new(capacity_sectors);
reader.always(
108,
ScriptStep::Err {
sense_key: scsi::SENSE_KEY_MEDIUM_ERROR,
asc: 0x11,
ascq: 0x00,
},
);
let nontrimmed = [(100 * 2048, 16 * 2048)];
let finished = [
(0, 100 * 2048),
(116 * 2048, (capacity_sectors as u64 - 116) * 2048),
];
let (pr, stats, trace_len) = run_profile(
"05_single_bad",
capacity_sectors,
&nontrimmed,
&finished,
reader,
trace,
);
// GOLDEN: 15 of 16 sectors recovered. 1 sector stays NonTrimmed
// (NOT Unreadable — same multi-pass tolerance principle).
assert_eq!(
stats.bytes_good,
(capacity_sectors as u64 - 1) * 2048,
"05_single_bad bytes_good"
);
assert_eq!(stats.bytes_unreadable, 0, "05_single_bad bytes_unreadable");
assert_eq!(stats.bytes_pending, 2048, "05_single_bad bytes_pending");
assert!(!pr.halted, "05_single_bad halted");
assert!(
trace_len <= 80,
"05_single_bad trace_len={trace_len} exceeds 80"
);
}
// ───────────────────── Profile 6: DEEP PIT ───────────────────────────────
//
// A contiguous 8-sector bad pit in the middle of a wider 24-sector
// NonTrimmed range. Tests the damage-window threshold + size-aware-skip
// converging on the actual pit boundaries instead of bailing on
// MAX_SKIPS_PER_RANGE.
#[test]
fn profile_06_deep_pit() {
let capacity_sectors: u32 = 256;
let (mut reader, trace) = ScriptedSectorReader::new(capacity_sectors);
for lba in 108..116 {
reader.always(
lba,
ScriptStep::Err {
sense_key: scsi::SENSE_KEY_MEDIUM_ERROR,
asc: 0x11,
ascq: 0x00,
},
);
}
// 24 sectors NonTrimmed: 100..108 good, 108..116 BAD, 116..124 good.
let nontrimmed = [(100 * 2048, 24 * 2048)];
let finished = [
(0, 100 * 2048),
(124 * 2048, (capacity_sectors as u64 - 124) * 2048),
];
let (pr, stats, trace_len) = run_profile(
"06_deep_pit",
capacity_sectors,
&nontrimmed,
&finished,
reader,
trace,
);
// GOLDEN: 16 good (8 on each side of the pit) recovered, 8 bad
// stay NonTrimmed.
let recovered_in_range = stats.bytes_good - (capacity_sectors as u64 - 24) * 2048;
assert!(
recovered_in_range >= 14 * 2048,
"06_deep_pit recovered only {recovered_in_range} bytes of 16 good sectors"
);
assert_eq!(stats.bytes_unreadable, 0, "06_deep_pit bytes_unreadable");
assert!(
stats.bytes_pending > 0,
"06_deep_pit bytes_pending expected > 0"
);
assert!(!pr.halted, "06_deep_pit halted");
assert!(
trace_len <= 120,
"06_deep_pit trace_len={trace_len} exceeds 120"
);
}
// ───────────────────── Profile 7: MEDIUM-THEN-GOOD ───────────────────────
//
// First N attempts at each bad LBA fail with MEDIUM_ERROR, then succeed.
// Tests whether patch's retry semantics revisit failed sectors. Current
// patch dispatches NonTrimmed on first failure and ADVANCES the cursor
// — it does NOT retry the same LBA inside one pass for MEDIUM_ERROR
// (only NOT_READY retries in-place). So the goldens here are: bad
// sectors stay NonTrimmed in this pass (the recovery would happen in a
// subsequent pass, which this single-pass fixture does not run).
#[test]
fn profile_07_medium_then_good() {
let capacity_sectors: u32 = 256;
let (mut reader, trace) = ScriptedSectorReader::new(capacity_sectors);
// Sectors 105..110: fail twice, then succeed.
for lba in 105..110 {
reader.sequence(
lba,
vec![
ScriptStep::Err {
sense_key: scsi::SENSE_KEY_MEDIUM_ERROR,
asc: 0x11,
ascq: 0x00,
},
ScriptStep::Err {
sense_key: scsi::SENSE_KEY_MEDIUM_ERROR,
asc: 0x11,
ascq: 0x00,
},
ScriptStep::Ok,
],
);
}
let nontrimmed = [(100 * 2048, 16 * 2048)];
let finished = [
(0, 100 * 2048),
(116 * 2048, (capacity_sectors as u64 - 116) * 2048),
];
let (pr, stats, trace_len) = run_profile(
"07_medium_then_good",
capacity_sectors,
&nontrimmed,
&finished,
reader,
trace,
);
// GOLDEN: patch's cache-priming (`prime_cache`) issues 3
// throwaway single-sector reads at lba-3..lba before each count==1
// recovery read. Those throwaway reads ADVANCE the per-LBA script
// step counter even though their results are discarded. So a
// 3-step script (fail, fail, ok) gets consumed by 2 prime calls
// plus 1 real read → the real read sees `Ok` and the sector is
// recovered. Net effect: patch fully recovers the range in one
// pass thanks to priming, even though the script said "fails on
// first two attempts."
//
// This is the documented cache-prime behavior (`disc/patch.rs`
// ~line 398, "Proven 2026-05-07 with dd-as-oracle: 8/8 sectors
// recoverable when primed vs 6/8 cold"). The golden pins it.
assert_eq!(
stats.bytes_good,
capacity_sectors as u64 * 2048,
"07_medium_then_good bytes_good — cache-prime should consume \
the failing script steps so the real read sees Ok"
);
assert_eq!(
stats.bytes_unreadable, 0,
"07_medium_then_good bytes_unreadable"
);
assert_eq!(stats.bytes_pending, 0, "07_medium_then_good bytes_pending");
assert!(!pr.halted, "07_medium_then_good halted");
assert!(
trace_len <= 100,
"07_medium_then_good trace_len={trace_len} exceeds 100"
);
}
// ───────────────────── Profile 8: BATCHED-FAIL ONLY ──────────────────────
//
// LBA 108 fails on BATCH reads (any batch including it) but succeeds
// individually. Models a marginal sector that the drive can ECC-recover
// when read alone but not at multi-sector throughput. Validates that
// adaptive batch's drop-to-count=1 retries the same starting position
// and rescues the data.
//
// Implementation note: the scripted reader marks the entire batch failed
// on any failed sector. We can't easily differentiate "single vs batch"
// without bigger plumbing — so this profile uses a script that fails
// once then succeeds on retry at the same LBA, simulating "drive
// recovered after retry."
#[test]
fn profile_08_batch_fail_singles_ok() {
let capacity_sectors: u32 = 256;
let (mut reader, trace) = ScriptedSectorReader::new(capacity_sectors);
// Sector 108: fail on first call (which is the batch read), succeed
// on second call (the drop-to-count=1 retry at the same position).
reader.sequence(
108,
vec![
ScriptStep::Err {
sense_key: scsi::SENSE_KEY_MEDIUM_ERROR,
asc: 0x11,
ascq: 0x00,
},
ScriptStep::Ok,
],
);
let nontrimmed = [(100 * 2048, 16 * 2048)];
let finished = [
(0, 100 * 2048),
(116 * 2048, (capacity_sectors as u64 - 116) * 2048),
];
let (pr, stats, trace_len) = run_profile(
"08_batch_fail",
capacity_sectors,
&nontrimmed,
&finished,
reader,
trace,
);
// GOLDEN: the second attempt succeeds → all 16 sectors recovered.
assert_eq!(
stats.bytes_good,
capacity_sectors as u64 * 2048,
"08_batch_fail bytes_good — second attempt should recover"
);
assert_eq!(stats.bytes_unreadable, 0, "08_batch_fail bytes_unreadable");
assert_eq!(stats.bytes_pending, 0, "08_batch_fail bytes_pending");
assert!(!pr.halted, "08_batch_fail halted");
assert!(
trace_len <= 80,
"08_batch_fail trace_len={trace_len} exceeds 80"
);
}
// ─────────────────────────────────────────────────────────────────────────
//
// Suppressed for now: NOT_READY-then-recover, HARDWARE_ERROR (wedge),
// ILLEGAL_REQUEST (wedge), and ABORTED_COMMAND profiles. Each would
// trigger long real-time sleeps inside `handle_read_failure`:
//
// - NOT_READY (sense_key=0x02, asc=0x02/0x03/0x04): 15 s pause per
// occurrence (`patch_not_ready_pause`), and retries the same LBA
// in-place. Even one NOT_READY costs the test 15 s wall-time.
//
// - HARDWARE_ERROR / ILLEGAL_REQUEST: 30 s per occurrence
// (`WEDGE_FAMILY_COOLDOWN_SECS`), bounded by
// `WEDGE_ABORT_THRESHOLD=16` before wedged-exit. Worst case ~8
// minutes per profile.
//
// The sleeps are not injectable. Adding them would require either a
// `now()` / `sleep()` trait injection (out of scope for the unification
// task) or a "test mode" compile-time flag (architectural smell). The
// behavioural contracts for those paths are captured in
// `read_error.rs`'s in-module tests instead — they exercise the
// classifier without invoking the patch loop's sleep side-effects.
//
// If the unification ever proceeds, the next step is to add a clock
// injection point in `handle_read_failure` and extend this fixture
// with the wedge/NOT_READY profiles too.