v0.13.12 — Fix 1+2+4 + cross-platform SCSI parity (RIP_DESIGN.md §6, §7, §15.1)
Fix 1: delete stall guard from Disc::copy. Pass 1 must sweep end-to-end
per ddrescue model (RIP_DESIGN.md §2.1, §3, §9). The v0.13.9 guard at
disc/mod.rs broke Pass 1 at 30% on Dune 2 with 56 GB still NonTried.
Removed stall_secs field, narrative comment in scsi/linux.rs, and the
broken regression test. Replaced with test_disc_copy_completes_full_disc_
with_failing_reader and test_disc_copy_halts_promptly_on_failing_reader.
Fix 2: async SCSI transport recovery. Added Arc<AtomicI32> fd_recovery
on SgIoTransport. On poll timeout: spawn close + spawn open in
background, return Err immediately. Top of execute() swaps fd from
recovery atomic. Main thread never blocked beyond ~1.5s poll budget
(was up to ~60s per timeout because kernel serialized main-thread
open() against in-flight close()). Drop drains pending recovery fd.
§15.1 cross-platform parity: Windows + macOS now have the same
observable recovery contract. SptiTransport gets try_recover()
(synchronous CloseHandle + CreateFileW; Windows close is fast, no
in-flight CDB drain like Linux). MacScsiTransport gets try_recover()
(release IOKit interface + reacquire via new acquire_device_iface()
helper); stores bsd_name for re-resolution. Drop guards null'd-out
interfaces. Stripped English error strings ("try as root" / "run as
administrator") on Linux + Windows. Fixed Windows TimeOutValue
ms→s ceiling so 1500ms gets 2s (was 1s; broke Drive::read fast path).
Fix 4: instrument Disc::patch arms. PatchResult exposes
blocks_attempted, blocks_read_ok, blocks_read_failed so the v0.13.11
mystery (Dune 2 Pass 2 recovered 0 bytes in 100 min) is diagnosable
from the live device log without re-instrumenting from outside.
Cleanup: honor PatchOptions::full_recovery (was read into _ and
ignored; now routed to read_sectors recovery arg). Updated
CopyOptions::batch_sectors doc to describe the actual production
path (sysfs detect_max_batch_sectors, typically 60 sectors / ~120 KB
on BU40N) rather than the test-only 32-sector internal default.
All four crates clippy-clean and tests green on the host targets
(macOS native + cargo check on Linux). Cross-platform CI watches
Linux + Windows + macOS builds + tests.
This commit is contained in:
MattJackson
@@ -10,7 +10,7 @@ use libfreemkv::{
|
||||
};
|
||||
use std::io::Write;
|
||||
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::sync::Arc;
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
const SECTOR_SIZE: usize = 2048;
|
||||
@@ -339,75 +339,52 @@ fn test_file_sector_reader_round_trip() {
|
||||
assert_eq!(all, data, "bulk read mismatch");
|
||||
}
|
||||
|
||||
// ── 6. Disc::copy stall detection triggers skip-forward (TDD red) ─────────
|
||||
// ── 6. Pass 1 sweeps the entire disc even when every read fails ───────────
|
||||
//
|
||||
// Regression guard for the Dell-host hang where `read_sectors` blocked inside
|
||||
// a kernel-level USB stall and `Disc::copy` sat frozen for 10+ minutes with
|
||||
// no progress and no error. The fix introduces `CopyOptions::stall_secs:
|
||||
// Option<u64>` — when elapsed-since-last-`bytes_good`-advance exceeds the
|
||||
// threshold, `Disc::copy` treats the current block as a read failure and
|
||||
// triggers the skip-forward path so the rip can advance.
|
||||
//
|
||||
// THIS TEST IS EXPECTED TO FAIL UNTIL THE PARALLEL FIX LANDS.
|
||||
// - Until `stall_secs` exists on `CopyOptions`, the test will not compile.
|
||||
// - Once the field exists but the stall guard isn't wired, the spawned copy
|
||||
// thread will never exit (test fails on the 5s join bound).
|
||||
// - Once the guard is wired, copy returns within ~stall_secs with
|
||||
// `complete=false` and `bytes_pending>0`.
|
||||
// Per RIP_DESIGN.md §2.1 + §3: Disc::copy must reach the end of the disc
|
||||
// regardless of how many reads fail. The only legitimate early exit is the
|
||||
// halt flag. With `skip_on_error + skip_forward` and a reader that returns
|
||||
// Err for every read, Pass 1 must:
|
||||
// - mark every sector NonTrimmed (so Pass 2 can retry them)
|
||||
// - return cleanly (no panic, no hang)
|
||||
// - bytes_good = 0
|
||||
// - bytes_pending = total_bytes (NonTrimmed counts as pending in mapfile
|
||||
// accounting; see disc/mapfile.rs::stats)
|
||||
// - bytes_unreadable = 0 (only Pass 2 marks Unreadable)
|
||||
// - complete = false (work remains for Pass 2)
|
||||
// - halted = false (no user stop)
|
||||
// - ISO file is `total_bytes` size on disk (sparse zeros)
|
||||
|
||||
/// Reader that returns Ok for sectors `< block_after`, then returns Err for
|
||||
/// any sector `>= block_after` after a small per-call delay. Models the
|
||||
/// realistic Dell-host symptom: reads keep returning Err (skip-forward fires)
|
||||
/// but no `bytes_good` ever accrues; without a stall guard, Pass 1 grinds
|
||||
/// silently for tens of minutes.
|
||||
struct StallingSectorReader {
|
||||
/// Reader that returns Err for every read. Models the worst case where the
|
||||
/// drive can read nothing on this disc — Pass 1 must still walk to end of disc.
|
||||
struct FailingSectorReader {
|
||||
capacity: u32,
|
||||
block_after: u32,
|
||||
/// Per-call delay for sectors >= block_after (simulates slow reads).
|
||||
/// Per-call delay so the test exercises the skip-forward path realistically
|
||||
/// without burning real wallclock.
|
||||
err_delay_ms: u64,
|
||||
release: Arc<AtomicBool>,
|
||||
/// Retained so callers can release the reader; unused now that the
|
||||
/// reader returns Err instead of blocking, but kept so the test's
|
||||
/// existing release plumbing compiles.
|
||||
park: Arc<(Mutex<()>, std::sync::Condvar)>,
|
||||
}
|
||||
|
||||
impl StallingSectorReader {
|
||||
fn new(capacity: u32, block_after: u32) -> Self {
|
||||
impl FailingSectorReader {
|
||||
fn new(capacity: u32) -> Self {
|
||||
Self {
|
||||
capacity,
|
||||
block_after,
|
||||
err_delay_ms: 100,
|
||||
release: Arc::new(AtomicBool::new(false)),
|
||||
park: Arc::new((Mutex::new(()), std::sync::Condvar::new())),
|
||||
err_delay_ms: 0,
|
||||
}
|
||||
}
|
||||
|
||||
fn release_handle(&self) -> (Arc<AtomicBool>, Arc<(Mutex<()>, std::sync::Condvar)>) {
|
||||
(self.release.clone(), self.park.clone())
|
||||
}
|
||||
}
|
||||
|
||||
impl SectorReader for StallingSectorReader {
|
||||
impl SectorReader for FailingSectorReader {
|
||||
fn read_sectors(
|
||||
&mut self,
|
||||
lba: u32,
|
||||
count: u16,
|
||||
buf: &mut [u8],
|
||||
_count: u16,
|
||||
_buf: &mut [u8],
|
||||
_recovery: bool,
|
||||
) -> Result<usize> {
|
||||
if lba >= self.block_after {
|
||||
// Realistic stall model: read takes err_delay_ms then returns
|
||||
// Err. With skip_on_error+skip_forward, Disc::copy will keep
|
||||
// skip-forwarding through this region — no bytes_good accrues.
|
||||
// The stall guard fires when bytes_good is unchanged for
|
||||
// stall_secs.
|
||||
if self.err_delay_ms > 0 {
|
||||
std::thread::sleep(Duration::from_millis(self.err_delay_ms));
|
||||
return Err(libfreemkv::error::Error::DiscRead { sector: lba as u64 });
|
||||
}
|
||||
let bytes = count as usize * SECTOR_SIZE;
|
||||
buf[..bytes].fill(0);
|
||||
Ok(bytes)
|
||||
Err(libfreemkv::error::Error::DiscRead { sector: lba as u64 })
|
||||
}
|
||||
|
||||
fn capacity(&self) -> u32 {
|
||||
@@ -416,89 +393,135 @@ impl SectorReader for StallingSectorReader {
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_disc_copy_stall_detection_triggers_skip_forward() {
|
||||
// 1024 sectors total. Reader serves the first 64 sectors instantly, then
|
||||
// every later read blocks forever. With stall_secs=2, copy should bail
|
||||
// out of the stalled block within ~2s and either skip forward or finish
|
||||
// with bytes_pending > 0 / complete=false.
|
||||
fn test_disc_copy_completes_full_disc_with_failing_reader() {
|
||||
// 1024 sectors = 2 MB. Reader fails every read. With skip_on_error +
|
||||
// skip_forward, Pass 1 must mark every sector NonTrimmed and return
|
||||
// cleanly — no bail, no hang.
|
||||
let capacity_sectors: u32 = 1024;
|
||||
let block_after: u32 = 64;
|
||||
|
||||
let reader = StallingSectorReader::new(capacity_sectors, block_after);
|
||||
let (release_flag, park) = reader.release_handle();
|
||||
let mut reader = reader;
|
||||
let total_bytes: u64 = capacity_sectors as u64 * SECTOR_SIZE as u64;
|
||||
|
||||
let mut reader = FailingSectorReader::new(capacity_sectors);
|
||||
let disc = synthetic_disc(capacity_sectors);
|
||||
|
||||
let tmp = tempfile::NamedTempFile::new().expect("tempfile create");
|
||||
let iso_path = tmp.path().to_path_buf();
|
||||
drop(tmp);
|
||||
|
||||
let iso_path_for_thread = iso_path.clone();
|
||||
|
||||
let join = std::thread::spawn(move || {
|
||||
let opts = CopyOptions {
|
||||
decrypt: false,
|
||||
skip_on_error: true,
|
||||
skip_forward: true,
|
||||
// ASSUMPTION: parallel fix adds `pub stall_secs: Option<u64>` to
|
||||
// CopyOptions. If the field name differs, update here.
|
||||
stall_secs: Some(2),
|
||||
..Default::default()
|
||||
};
|
||||
let t0 = Instant::now();
|
||||
let res = disc.copy(&mut reader, &iso_path_for_thread, &opts);
|
||||
(res, t0.elapsed())
|
||||
});
|
||||
|
||||
// Bound the join to ~5s. With stall_secs=2 the copy should exit well
|
||||
// within this window. If it doesn't, the stall guard isn't working.
|
||||
let started = Instant::now();
|
||||
let mut joined = None;
|
||||
while started.elapsed() < Duration::from_millis(5000) {
|
||||
if join.is_finished() {
|
||||
joined = Some(join.join().expect("thread join"));
|
||||
break;
|
||||
}
|
||||
std::thread::sleep(Duration::from_millis(50));
|
||||
}
|
||||
|
||||
// Whether or not the join succeeded, release the parked reader thread so
|
||||
// it can exit (its &mut reader is owned by the spawned thread; releasing
|
||||
// lets that thread unwind cleanly).
|
||||
release_flag.store(true, Ordering::Relaxed);
|
||||
park.1.notify_all();
|
||||
|
||||
let (result, elapsed) = match joined {
|
||||
Some(v) => v,
|
||||
None => {
|
||||
// Wait a bit longer for the thread to drain after release so we
|
||||
// don't leave it dangling, then fail the test.
|
||||
std::thread::sleep(Duration::from_millis(500));
|
||||
panic!(
|
||||
"Disc::copy did not return within 5s of stall_secs=2 — \
|
||||
stall guard not wired (TDD red until fix lands)"
|
||||
);
|
||||
}
|
||||
let opts = CopyOptions {
|
||||
decrypt: false,
|
||||
skip_on_error: true,
|
||||
skip_forward: true,
|
||||
..Default::default()
|
||||
};
|
||||
|
||||
let t0 = Instant::now();
|
||||
let result = disc
|
||||
.copy(&mut reader, &iso_path, &opts)
|
||||
.expect("copy returns Ok");
|
||||
let elapsed = t0.elapsed();
|
||||
|
||||
// Cleanup
|
||||
let _ = std::fs::remove_file(&iso_path);
|
||||
let _ = std::fs::remove_file(libfreemkv::disc::mapfile_path_for(&iso_path));
|
||||
|
||||
let copy_result = result.expect("copy returns Ok with stall handling");
|
||||
// Hard bound — even at 0 ms per read, 1024 sectors with skip-forward
|
||||
// should complete in well under a second on any host. If this test runs
|
||||
// for minutes, something has regressed (e.g. stall guard reintroduced
|
||||
// with infinite-loop semantics, or Pass 1 is hanging on each read).
|
||||
assert!(
|
||||
elapsed < Duration::from_secs(5),
|
||||
"Pass 1 took {elapsed:?} on a 2 MB synthetic disc — expected < 5 s"
|
||||
);
|
||||
|
||||
// Per RIP_DESIGN.md §2.1: Pass 1 must reach end of disc regardless of
|
||||
// read outcomes.
|
||||
assert_eq!(
|
||||
result.bytes_total, total_bytes,
|
||||
"bytes_total must match disc capacity"
|
||||
);
|
||||
assert_eq!(
|
||||
result.bytes_good, 0,
|
||||
"no reads succeeded, bytes_good must be 0"
|
||||
);
|
||||
assert_eq!(
|
||||
result.bytes_unreadable, 0,
|
||||
"Pass 1 does not mark Unreadable; only Pass 2 (Disc::patch) does"
|
||||
);
|
||||
assert_eq!(
|
||||
result.bytes_pending, total_bytes,
|
||||
"every sector must be NonTrimmed → counted as pending. \
|
||||
Got bytes_pending={} of total {}",
|
||||
result.bytes_pending, total_bytes
|
||||
);
|
||||
assert!(
|
||||
!result.complete,
|
||||
"complete=false because NonTrimmed regions remain (work for Pass 2)"
|
||||
);
|
||||
assert!(
|
||||
!result.halted,
|
||||
"no halt was set; halted must be false"
|
||||
);
|
||||
|
||||
// ISO file should be the full disc size on disk (sparse zeros where
|
||||
// reads failed).
|
||||
// Note: tempfile was dropped above; the file may or may not still exist
|
||||
// depending on cleanup ordering. We only assert what we can observe in
|
||||
// the CopyResult.
|
||||
}
|
||||
|
||||
// ── 7. Halt during Pass 1 of an all-failing-read sweep returns promptly ───
|
||||
//
|
||||
// Per RIP_DESIGN.md §3: halt is the only legitimate early exit from Pass 1.
|
||||
// Even when every read is failing (skip-forward path), a halt must be
|
||||
// honored within a small bounded time.
|
||||
|
||||
#[test]
|
||||
fn test_disc_copy_halts_promptly_on_failing_reader() {
|
||||
let capacity_sectors: u32 = 1024 * 1024; // 2 GB synthetic disc — plenty of work
|
||||
let mut reader = FailingSectorReader {
|
||||
capacity: capacity_sectors,
|
||||
err_delay_ms: 1, // small per-read delay so halt has something to interrupt
|
||||
};
|
||||
let disc = synthetic_disc(capacity_sectors);
|
||||
|
||||
let tmp = tempfile::NamedTempFile::new().expect("tempfile create");
|
||||
let iso_path = tmp.path().to_path_buf();
|
||||
drop(tmp);
|
||||
|
||||
let halt = Arc::new(AtomicBool::new(false));
|
||||
let halt_setter = halt.clone();
|
||||
|
||||
// Trigger halt after 200 ms.
|
||||
std::thread::spawn(move || {
|
||||
std::thread::sleep(Duration::from_millis(200));
|
||||
halt_setter.store(true, Ordering::Relaxed);
|
||||
});
|
||||
|
||||
let opts = CopyOptions {
|
||||
decrypt: false,
|
||||
skip_on_error: true,
|
||||
skip_forward: true,
|
||||
halt: Some(halt),
|
||||
..Default::default()
|
||||
};
|
||||
|
||||
let t0 = Instant::now();
|
||||
let result = disc
|
||||
.copy(&mut reader, &iso_path, &opts)
|
||||
.expect("copy returns Ok on halt");
|
||||
let elapsed = t0.elapsed();
|
||||
|
||||
// Cleanup
|
||||
let _ = std::fs::remove_file(&iso_path);
|
||||
let _ = std::fs::remove_file(libfreemkv::disc::mapfile_path_for(&iso_path));
|
||||
|
||||
assert!(
|
||||
elapsed < Duration::from_millis(5000),
|
||||
"copy elapsed {elapsed:?} exceeded 5s bound (stall_secs=2)"
|
||||
elapsed < Duration::from_secs(2),
|
||||
"halt must return within 2 s; took {elapsed:?}"
|
||||
);
|
||||
assert!(result.halted, "result.halted must be true");
|
||||
assert!(
|
||||
copy_result.bytes_pending > 0,
|
||||
"expected bytes_pending > 0 after stall-triggered skip; got {}",
|
||||
copy_result.bytes_pending
|
||||
);
|
||||
assert!(
|
||||
!copy_result.complete,
|
||||
"expected complete=false after stall-triggered skip"
|
||||
!result.complete,
|
||||
"halted run cannot be complete (bytes_pending > 0 likely)"
|
||||
);
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user