Compare commits

...
57 Commits
Author SHA1 Message Date
MattJackson bfa527162a 0.17.0: unwrap safety fix, patch pass algorithm, clippy compliance
- Fix unwrap in disc/mod.rs sweep() hot path using pattern matching
- Patch pass excludes Unreadable sectors from work list
- Expose bytes_bad_in_title for accurate UI reporting
- All 256 tests pass, cargo clippy clean with -D warnings
2026-05-04 09:42:07 -07:00
MattJackson 45defd47f3 fix patch pass: exclude Unreadable from work list; expose bytes_bad_in_title; clippy 1.86 fixes
- patch(): only process NonTrimmed + NonScraped ranges (Unreadable=terminal, NonTried=not-yet-swept)
- bytes_bad_in_title: pub fn for autorip main-movie lost_ms computation
- Clippy 1.86: saturating_sub, unused assignments, unused variable
- fmt: rustfmt formatting
2026-05-03 16:35:06 -07:00
MattJackson fe8933dad2 v0.16.3: i18n damage display
- Add rip.damage_lost, rip.damage_lost_movie, rip.damage_lost_simple, rip.damage_none keys
- Update all 6 locale files (en, es, fr, de, it, pt, nl)
- CLI progress shows 'Xs lost (Y in movie)' instead of 'Xs unreadable'
- Perfect rips show '0s' instead of '0s unreadable'
2026-04-30 20:51:02 -07:00
MattJackson 0a3c1b7b70 fix: scsi_recovery test variable shadowing and missing imports for Linux CI 2026-04-30 19:57:16 -07:00
MattJackson 2bcdc97341 fix: add missing Duration import in scsi_recovery test 2026-04-30 19:52:53 -07:00
MattJackson 4863e9c545 v0.16.2: sticky escalation in patch, title-aware damage display, PASS1/PASSN constant naming 2026-04-30 19:50:08 -07:00
MattJackson 03db038dd9 docs: update project docs, CHANGELOG, drive-access.md, rip-recovery.md for v0.16.x 2026-04-30 15:43:58 -07:00
MattJackson 95001ba2c3 v0.16.1: bump version 2026-04-30 15:23:44 -07:00
MattJackson a1f4dff6f6 v0.16.0: IOKit registry-based drive enumeration, BSD name → IOBDServices matching, reverse patch default 2026-04-30 15:17:20 -07:00
MattJackson 7dd5001d45 v0.15.1: fix damage-jump detection (window=16, threshold=12%), fix dispatch covers_disc check, sweep resume from mapfile
- DAMAGE_WINDOW 50→16, DAMAGE_THRESHOLD_PCT 25→12: triggers on 2nd scattered failure
- Previous 50/25% was too diluted by good reads between sparse failures
- copy() dispatch checks mapfile total_size == disc capacity_bytes (covers_disc)
- NonTried regions → sweep with resume=true (preserves mapfile)
- Only NonTrimmed/NonScraped/Unreadable → patch
- sweep_internal takes resume param: true when dispatching from existing mapfile
- Verified: Pass 1 from 30-100% completes in ~20 min with correct jumps through 3 damage zones
2026-04-30 13:15:03 -07:00
MattJackson 67fe93c0b8 v0.15.0: multipass CopyOptions, auto-detect sweep vs patch, speed control on damage zone entry/exit
- CopyOptions: replace resume/skip_on_error/batch_sectors with single multipass bool
- Disc::copy() auto-detects pass from mapfile state: no mapfile or NonTried → sweep, only NonTrimmed/Unreadable → patch
- Fix bug where mapfile with NonTried regions incorrectly dispatched to patch mode
- SectorReader::set_speed() default method, Drive impl sends SET CD SPEED
- On damage zone entry: set_speed(0x0000) for better error recovery
- On damage zone exit (50 consecutive good): set_speed(0xFFFF) to restore max speed
- Disc::mapfile_for() returns /tmp/<name>.mapfile for null:// output
- patch_internal/sweep_internal as private helpers, CopyResult gains recovered_this_pass
2026-04-30 11:31:31 -07:00
MattJackson 5c4f575c46 v0.14.0: bump version 2026-04-30 08:45:32 -07:00
MattJackson e2cb7a78bb v0.13.46: damage-jump algorithm replaces probe, bridge degradation detection, ecc_sectors() 2026-04-29 22:08:52 -07:00
MattJackson a6f1bd19bc v0.13.45: multipass adaptive probe, NOT_READY retry, progress bytes_bad_total
- Adaptive probe algorithm in Disc::copy skip_on_error mode: after 4
  consecutive errors, probe 1 sector at 256x batch (8 MB) ahead. If
  good, zero-fill gap, mark NonTrimmed, jump. Clears bad zones in
  seconds instead of hours.
- NOT_READY sense key (0x02) now retries up to 3x with 3s pause before
  marking NonTrimmed. BU40N returns NOT READY for bad sectors, not
  MEDIUM ERROR.
- PassProgress struct gains bytes_bad_total field for consumer-side
  bad/retryable byte counts.
- Mapfile header version string fix: no longer duplicates 'libfreemkv v'
  prefix on each write.
- Structured sense_key/asc/ascq logging in copy error path.
2026-04-29 19:41:34 -07:00
MattJackson 9e3d0f1383 v0.13.44: macOS raw CDB transport via IOKit exclusive access
macOS SCSI transport rewritten from hybrid MMC+pread to single-path
raw CDB dispatch through SCSITaskDeviceInterface. All CDBs (INQUIRY,
READ, REPORT KEY, etc.) now go through ExecuteTaskSync — 1:1 with
the Linux SG_IO backend.

Key changes:
- New macos_shim.c: diskutil unmount → find IOBDServices →
  ObtainExclusiveAccess → raw CDB dispatch. Eliminates Rust-side
  IOKit COM vtable complexity.
- build.rs compiles macos_shim.c via cc into static lib
- macos.rs simplified to three FFI calls (open/close/execute)
- disc/mod.rs: graduated batch restore after errors, skip-ahead
  through bad zones, configurable error pause
2026-04-29 15:59:35 -07:00
MattJackson 1e6eb0698d v0.13.43: Pass 1 transport-failure recovery loop 2026-04-29 08:53:56 -07:00
MattJackson bd744171e4 v0.13.42: transport failure skips instead of aborting copy 2026-04-29 07:46:42 -07:00
MattJackson 7de9d4d42c v0.13.41: debug logging for sector-0 regression 2026-04-29 07:09:46 -07:00
MattJackson 9ac0c2fcdf v0.13.40 2026-04-28 21:50:32 -07:00
MattJackson 3aa27d5d1b fix CI: Rust 2024 match ergonomics - remove ref binding modifier 2026-04-28 21:47:19 -07:00
MattJackson 5083f70cff fix CI: import ScsiTransport trait in integration test 2026-04-28 21:45:34 -07:00
MattJackson 477cf7ee0e fix CI: pub scsi::linux module and SgIoTransport fields for integration tests 2026-04-28 21:43:14 -07:00
MattJackson 0b079e27ce fix CI: make scsi::linux pub(crate) for integration test access 2026-04-28 21:39:50 -07:00
MattJackson 5f69fd2a44 update CHANGELOG and README for v0.13.39 2026-04-28 21:36:37 -07:00
MattJackson befe6ef69b remove (internal) references from public code 2026-04-28 21:32:15 -07:00
MattJackson 29491d6eaf v0.13.39 2026-04-28 21:19:51 -07:00
MattJackson 2e602f7e37 fix scsi_recovery test: dereference status in pattern match 2026-04-28 21:17:11 -07:00
MattJackson a126f31b70 v0.13.38 2026-04-28 21:12:05 -07:00
MattJackson 6ff10df83a fix scsi_recovery test compilation (old Drive::read signature) 2026-04-28 21:11:37 -07:00
MattJackson 52b8522a75 v0.13.37: Pass 1 is pure ECC-block sweep — read 32 sectors, fail → skip, no single-sector reads 2026-04-28 21:08:58 -07:00
MattJackson 13eb4331b6 v0.13.36: remove unused reset() 2026-04-28 17:57:25 -07:00
MattJackson 1a3f7ce676 remove unused reset() 2026-04-28 17:56:42 -07:00
MattJackson e1a1f730be v0.13.35: pause 3s after first MEDIUM ERROR before retry to prevent USB bridge crash 2026-04-28 17:54:12 -07:00
MattJackson 646c22ae93 v0.13.34: open() just opens, drive_has_disc() is side-effect-free direct TUR, enumerate_sg_names skips unreadable type files, Disc::copy read_err_count fix 2026-04-28 16:00:00 -07:00
MattJackson 2f6092a686 scsi/linux: fix as_bytes() → as_encoded_bytes() for OsStrExt
- as_bytes() requires OsStrExt import which is platform-specific
- as_encoded_bytes() is the portable API available on all platforms
- Fixes CI failure on Linux (Ubuntu) in GitHub Actions
2026-04-28 14:59:55 -07:00
MattJackson e5a90a6567 disc: fix hysteresis, SgIoTransport recovery, wallclock budget, patch instrumentation
- Fix hysteresis: use_single now correctly forces block_count=1 (was computed before the check)
- Fix SgIoTransport: spawn close+reopen in background thread on transport error
- Fix autorip: rip_disc() spawns wallclock watcher thread, caps entire rip at max(disc_runtime, 1h)
- Fix Disc::patch: add unreadable_count counter + tracing instrumentation
- Simplify Disc::copy() error handling per RIP_DESIGN.md §2.1
2026-04-28 14:48:16 -07:00
MattJackson afce1031d5 disc: add eprintln debug 2026-04-27 23:46:34 -07:00
MattJackson 2c0bff1303 disc: add debug logging for error type 2026-04-27 22:39:03 -07:00
MattJackson 2067ebe485 disc: add warn trace for MEDIUM ERROR skip 2026-04-27 21:38:03 -07:00
MattJackson 2ce4c20221 disc: skip MEDIUM ERROR sectors instead of bailing
0.13.28 - when drive returns MEDIUM ERROR (bad sector), skip the sector
and continue instead of retrying or bailing. Write zero-fill, mark as
Unreadable for pass 2+ recovery.

Closes: (internal)#20260427
2026-04-27 20:44:59 -07:00
MattJackson da3004b120 disc: DiscRead carries SCSI status/sense
Fix extract_scsi_context() and Error::scsi_sense() to handle Error::DiscRead
in addition to Error::ScsiError, so is_marginal_read() works for DiscRead
errors and disc::copy() can properly route MEDIUM ERROR as a marginal
(bad sector) instead of bailing.

Closes: (internal)#20260427
2026-04-27 18:38:57 -07:00
MattJackson b5ed97801b libfreemkv: extend DiscRead with SCSI status/sense for 30% wedge diagnostics 2026-04-27 16:47:27 -07:00
MattJackson ae76aaf0fa v0.13.24 — MapStats: split bytes_pending into nontried / retryable
bytes_pending was an opaque aggregate of NonTried + NonTrimmed +
NonScraped. UIs that wanted a "will retry in Pass 2-N" bucket were
stuck showing the entire unread disc as Maybe at pct=0.

Adds two granular fields to MapStats:

  bytes_nontried   — Pass 1 hasn't read these yet
  bytes_retryable  — NonTrimmed + NonScraped, Pass 2-N will retry

bytes_pending stays for back-compat (= bytes_nontried + bytes_retryable).

Also picks up the cargo fmt --check lint that's been red on main CI
since v0.13.18 (rustfmt fold differences on a few long format-string
layouts; functional no-op).
2026-04-26 19:28:35 -07:00
MattJackson 2cd4fbead7 v0.13.23 — stop discarding the drive's SCSI sense data
Through the entire 0.13.x line, every CHECK CONDITION reply from the
drive (the standard way SCSI tells you why a sector failed) was being
collapsed into a synthetic status=0xFF, sense_key=0 transport-wedge
sentinel and the actual sense data was thrown away. Confirmed live on
the BU40N reading Dune 2 on 2026-04-27: drive returned host_status=0,
driver_status=8, status=2, exec_elapsed_ms=1416 on every bad sector
— a clean CHECK CONDITION carrying full sense data — and Disc::copy
was bailing on it as if the bridge had wedged.

Root cause: scsi/linux.rs's wedge check was
  `host_status != 0 || driver_status != 0`
SG's DRIVER_SENSE bit (0x08) is set on every CHECK CONDITION reply
just to flag "sense buffer is populated" — it's not a transport
failure on its own. Pre-fix we conflated the two and silently lost
every drive-reported error reason. macOS and Windows backends had
the same shape: they extracted sense_key only, dropping ASC/ASCQ.

API restructure (clean separation):

  Error::ScsiError {
      opcode: u8,
      status: u8,                  // 0xFF = synthetic transport-failure
      sense: Option<ScsiSense>,    // None ⇔ no sense delivered
  }

  pub struct ScsiSense { sense_key: u8, asc: u8, ascq: u8 }
  impl ScsiSense {
      pub fn is_marginal(&self) -> bool       // keys 0/1/3/B
      pub fn is_medium_error(&self) -> bool
      pub fn is_hardware_error(&self) -> bool
      pub fn is_unit_attention(&self) -> bool
      pub fn is_data_protect(&self) -> bool
      pub fn is_not_ready(&self) -> bool
      pub fn is_illegal_request(&self) -> bool
      pub fn is_aborted_command(&self) -> bool
  }

  impl Error {
      pub fn scsi_sense(&self) -> Option<&ScsiSense>
      pub fn is_scsi_transport_failure(&self) -> bool
      pub fn is_marginal_read(&self) -> bool
  }

SCSI protocol constants (SCSI_STATUS_*, SENSE_KEY_*) moved from
error.rs to scsi/mod.rs where they belong alongside SCSI_INQUIRY,
SCSI_READ_10, etc. parse_sense replaces parse_sense_key (returns the
full triple, not just the key); inline tests now exercise ASC/ASCQ
extraction at the right offsets for both descriptor (0x72/0x73) and
fixed (0x70/0x71) sense formats.

Disc::copy + Disc::patch sense-aware dispatch:
  - marginal sense (MEDIUM ERROR / ABORTED COMMAND / RECOVERED ERROR
    / NO SENSE) → engage hysteresis (Block→Single, bpt=1)
  - non-marginal sense (HARDWARE / DATA PROTECT / UNIT ATTENTION /
    NOT READY / ILLEGAL REQUEST / transport failure / kernel
    IoError) → bail with full sense info preserved; caller (autorip)
    surfaces "physical replug" / "drive failing" / "media changed"

  Pre-fix: every CHECK CONDITION → 0xFF synthetic → Disc::copy bailed
  → bytes_good froze at the bad zone. The hysteresis from v0.13.22
  was correct but never got to run. This release unblocks it.

  Disc::patch's wedged_threshold (50 consecutive failures) stays as
  defense-in-depth for chains of marginal failures; a single
  non-marginal sense now short-circuits it.

New phase=bail trace event records the bail reason with the sense
triple. phase=transport_err remains for genuine bridge wedges /
kernel timeouts; phase=scsi_err carries the parsed sense_key, asc,
ascq for drive-reported errors.

All 350 tests pass. Clippy clean across all targets.
2026-04-26 19:06:19 -07:00
MattJackson ebffc6eb88 v0.13.22 — replace bisect-on-fail with hysteresis Block↔Single
The v0.13.21 bisect-on-fail recovery was correct (100% of recoverable
sectors picked up) but slow on dense damage clusters. Live test on
Dune 2 v0.13.21 burned ~30 s per damaged 60-block — paying a ~5 s
kernel ABORT/timeout at every level of a log₂(60) ≈ 6 deep DFS, on
the failing branch each time.

Replaced with a two-state hysteresis machine in Disc::copy:

  Block(batch):
    read(batch) ok    → write, advance, stay Block
    read(batch) fail  → switch to Single, retry SAME range at bpt=1

  Single:
    read(1) ok    → write, consecutive_good++
                    if consecutive_good >= BPT1_EXIT_THRESHOLD:
                      switch to Block, reset counter
    read(1) fail  → mark NonTrimmed, consecutive_good = 0

BPT1_EXIT_THRESHOLD = 10_000 sectors (= 20 MB clean run). Calibrated
from the 2026-04-26 BU40N empirical probe data; tunable.

Per-block math on a damaged 60-block with 1 truly bad sector:

  Bisect      (v0.13.21): ~30 s  (5 s × 6 levels)
  Hysteresis  (v0.13.22): ~10 s  (5 s bpt=batch fail
                                  + 59 × 1 ms good
                                  + 1 × 5 s bad)

Inside a damaged cluster spanning many 60-blocks the win compounds:
hysteresis pays the bpt=batch fail cost ONCE on entry, then stays at
bpt=1 across the cluster; bisection re-paid it every 60 sectors. For
Dune 2's ~1248-sector boundary cluster that's ~21 fewer 5-sec
kernel timeouts ≈ 100 s saved per pass.

Telemetry: new phase=mode_change trace event with from, to, lba, and
consecutive_good. Replaces v0.13.21's phase=bisect. Worklist DFS is
gone — single iterative for s in 0..count on the failure path.

Test rename, same fixture and same 100% recovery expectation:
  test_disc_copy_bisect_recovers_via_single_sector_reads
  → test_disc_copy_hysteresis_recovers_via_single_sector_reads

Also adds DamageSeverity (Clean / Cosmetic / Moderate / Serious) +
classify_damage(bad_sectors, lost_ms), re-exported from libfreemkv,
so applications can render structured severity instead of formatting
their own from raw counters.
2026-04-26 17:27:57 -07:00
MattJackson 424d3cd4f2 v0.13.21 — bisect-on-fail in Disc::copy + 10s caller READ timeout
Fixes the BU40N wedge cycle that has been chasing us through
v0.13.18-20. Two changes, both backed by empirical live-hardware
probes recorded in (internal)/docs/TEST_PLAN.md:

1. scsi/mod.rs: READ_TIMEOUT_MS 1500 → 10000 ms.
   Cold-start seek on the BU40N takes ~1.5 s. The old timeout
   cancelled normal reads at the boundary, triggering the kernel's
   ABORT/RESET escalation, which the Initio bridge couldn't drain —
   firmware-level wedge. 10 s catches every legitimate slow read
   (max successful ECC recovery: 2.6 s; cold-start: 1.5 s) with
   margin and short-circuits truly bad sectors at ~10 s.

2. disc/mod.rs: Disc::copy bisect-on-fail (replaces skip-forward).
   Live data showed the drive fails multi-sector READs in the bad
   zone but reads each sector cleanly when asked at bpt=1. Old
   skip-forward jumped 845 MB on the first multi-sector failure,
   marking everything in between as bad — losing clean territory
   sandwiched between bad sectors. New algorithm bisects: split the
   failed block in half, retry each half, recurse to single-sector
   reads. Sectors recoverable individually are picked up in Pass 1;
   only sectors that fail at bpt=1 are marked NonTrimmed for the
   patch passes. Stack-based DFS, log2(batch) = 6 levels for the
   default 60-sector batch.

Multi-pass machinery is untouched. Pass 2..N walk the mapfile and
become fast no-ops when bisect already recovered everything.
Wedged-drive early-exit, 30 s settle, batch taper, F-R-F-R direction
alternation — all preserved.

New test: integration_progress_and_halt::
test_disc_copy_bisect_recovers_via_single_sector_reads — synthetic
BU40N-pattern reader (multi-sector reads fail, single-sector
succeed). Pre-patch: lost everything to skip-forward. Post-patch:
100 % bytes_good. Plus the 10 sense-key parser tests from the
0.13.20 test-coverage pass.

Empirical recovery on Dune 2 UHD on the BU40N (per TEST_PLAN.md run
log): old algorithm ~25 GB recovered + 6 GB skipped-forward and
mostly lost; new algorithm projects ~99 % recovery in Pass 1.

Audits + raw probe data:
- (internal)/docs/TEST_PLAN.md (run log)
- (internal)/docs/audits/2026-04-26-scsi-architecture-research.md
2026-04-26 15:57:44 -07:00
MattJackson d2905ba7bb v0.13.20 — sync blocking SG_IO + cross-platform parity strip
- scsi/linux.rs: full rewrite from async write/poll/read+1.5s timeout+
  close-on-timeout to one synchronous ioctl(fd, SG_IO, &hdr). Kernel
  honors hdr.timeout and runs its own ABORT/RESET escalation. Errors
  check host_status and driver_status (both 0xFF-synthesised) plus
  status. Sense-key parser handles descriptor (0x72/0x73) + fixed
  (0x70/0x71) formats. Deleted fd_recovery, bg close+open thread, fd
  swap dance. -331/+155 lines.

- scsi/macos.rs: try_recover() removed (userspace handle-recovery on
  task failure was the same anti-pattern stripped from Linux). bsd_name
  field deleted. Errors bubble up directly.

- scsi/windows.rs: try_recover() removed, wide_path field deleted,
  INVALID_HANDLE guard removed.

- scsi/mod.rs: parse_sense_key() helper extracted (used by all three
  platforms now — single canonical sense-key parse rather than three
  inlined copies). +10 unit tests covering descriptor format, fixed
  format, truncated buffers, unknown response codes.

- drive/mod.rs: Drive::reset() deleted (escalating eject + STOP/START +
  reinit recovery — per audit, kernel handles its own escalation;
  userspace shouldn't).
  pub fn find_drives() -> Vec<Drive> deleted (opened N drives just to
  throw most away). find_drive() now uses discover_drives() directly.
  wait_ready() simplified — drops the reset path on sense_key=5,
  just keeps polling TUR for 60 iterations.

- lib.rs: find_drives re-export removed.

- benches/sgio_read.rs: switched to find_drive() (no longer iterates a
  drive list).

Net: 9 files changed, 226 insertions(+), 473 deletions(-). 329 tests
pass, clippy -D warnings clean. No consumer breakage (CLI, autorip,
bdemu compile + test green).

Architecture decision documented in
(internal)/docs/audits/2026-04-26-scsi-architecture-research.md
(primary-source survey of MakeMKV, sg_dd, ddrescue, and the kernel
mid-layer's own scsi_eh.rst escalation ladder).
2026-04-26 09:51:46 -07:00
MattJackson b4de5d343d v0.13.18 — sync release (no functional changes; autorip two-bar UI fix) 2026-04-26 07:47:37 -07:00
MattJackson 8beeac7df9 v0.13.17 — sync release (no functional changes; actual fix is autorip hot-plug) 2026-04-26 07:27:07 -07:00
MattJackson b33f41e219 v0.13.16 — single Progress trait + PassProgress (RIP_DESIGN.md §16)
Pre-0.13.16 the rip API leaked internal mapfile concepts (pos,
bytes_good, work_done, bytes_pending, Finished/NonTrimmed) into per-pass
positional callbacks Fn(u64, u64, u64). Consumers reinvented the math
each time, and v0.13.15's UI bug surfaced exactly because of this —
autorip's web JS computed pct from bytes_good while the backend
computed from pos, silent drift, frozen UI bar.

This release replaces both Disc::copy::on_progress and
Disc::patch::on_progress callbacks with a single Progress trait +
PassProgress struct (new progress module).

  pub struct PassProgress {
      pub kind: PassKind,            // Sweep | Trim {reverse} | Scrape {reverse} | Mux
      pub work_done: u64,
      pub work_total: u64,
      pub bytes_good_total: u64,
      pub bytes_total_disc: u64,
  }

  pub trait Progress {
      fn report(&self, p: &PassProgress);
  }

  impl<F: Fn(&PassProgress)> Progress for F { ... }   // closures work directly

CopyOptions::on_progress and PatchOptions::on_progress are renamed to
progress: Option<&dyn Progress>. Closure callers update trivially via
the blanket impl; struct callers gain a clean named-field shape with no
positional-arg confusion.

PassKind carries the semantic (sweep vs trim vs scrape vs mux) so
consumers can label phases without reinventing detection logic.
Disc::patch reports Trim {reverse} for retry passes with block_sectors
>= 2 and Scrape {reverse} when block_sectors == 1. Direction comes
through reverse: bool. Mux variant is reserved for v0.13.17 when the
mux pipeline emits progress.

Tests + clippy clean across all 4 crates.
2026-04-26 07:15:26 -07:00
MattJackson e85e20f436 v0.13.15 — pos in on_progress, PatchOptions::reverse, wedged_threshold
Breaking: CopyOptions::on_progress + PatchOptions::on_progress now take
Fn(bytes_good, pos, total). Consumers display `pos` for "% swept" — the
true Pass 1 progress that advances through skip-forward bad zones, where
bytes_good (Finished sectors only) freezes. v0.13.14 live trace proved
the existing UI was lying for ~14 minutes about Dune 2 being "stuck at
30%" while Pass 1 was actually 83% through the disc via skip-forward.

PatchOptions::reverse: walk bad ranges from highest LBA to lowest. For
drives that wedge after a forward read of a bad sector, approaching the
post-bad-zone NonTrimmed range from end-of-disc reads good sectors before
the drive sees a bad one. Hypothesis informed by the BU40N + Initio
bridge live data — Pass 2 forward saw zero successful reads in 7 min
while Pass 1's pos walked all the way to end-of-disc.

PatchOptions::wedged_threshold: > 0 → exit early after that many
consecutive failures with zero successes in the same pass. Saves the
wallclock budget for productive grinding when the drive has wedged on
the bad zone for THIS pass; a different direction or block size in the
next pass may still recover. New PatchResult::wedged_exit reports it.

Trace: patch_start (block_sectors, recovery, reverse, wedged_threshold,
num_ranges) and patch_done (blocks_attempted, blocks_read_ok,
blocks_read_failed, wedged_exit, halted, bytes_recovered) at the
freemkv::disc target.
2026-04-25 20:06:03 -07:00
MattJackson 7bc54be8d3 v0.13.14 — sync release, no functional changes (autorip subscriber filter fix in autorip 0.13.14) 2026-04-25 18:38:17 -07:00
MattJackson 6d9083743b v0.13.13 — telemetry: tracing instrumentation in SgIoTransport + Disc::copy
v0.13.12 shipped the async fd_recovery design but a live test on Dune 2
showed Pass 1 sat for 14 minutes with bytes_good=0 — the inner loop iterates
(throttled on_progress log fires every 78s) but each iteration evidently
takes ~60s instead of the microseconds the design promises on fast-fail.
Without trace-level telemetry at the SCSI + Disc::copy boundaries we
can't tell where the time goes.

This release is instrumentation only — no behavior change.

- New dep: tracing 0.1. Per project docs, debug/trace logging is allowed in
  libfreemkv (the no-English rule applies to errors). Consumers wire a
  tracing subscriber.
- SgIoTransport::execute (Linux): trace at every state transition (entry,
  recovery_swap_ok, recovery_pending, write_ok / write_err, poll_done,
  timeout_spawn_recovery, scsi_err, read_err, ok). Each event includes
  opcode + elapsed timing. The bg recovery thread also traces close_ms +
  open_ms so we can see if the kernel really takes 60s to close+open on a
  wedged Initio bridge.
- Disc::copy: trace at copy_start, outer_loop, region_enter, every 100
  inner-loop iterations (iter_progress with pos/region_end/skip_size/
  bytes_good/read_ok_count/read_err_count/last_read_ms/copy_elapsed_ms),
  copy_done.
- All trace events use targets `freemkv::scsi` and `freemkv::disc` so
  consumers can filter by subsystem (e.g. autorip /api/debug?q=freemkv::scsi).

Next: run the live test on Dune 2 again, read the autorip JSONL log,
diagnose why each iter is slow, fix the actual bug.
2026-04-25 18:32:05 -07:00
MattJackson c49a68054f style: cargo fmt on integration test 2026-04-25 17:36:06 -07:00
MattJackson 4bd38787c6 test(integration): make halt-on-skip-forward test deterministic
The wallclock-based halt timing failed on fast CI runners where a 2 GB
synthetic-disc skip-forward sweep finishes in <100 ms — well under the
200 ms halt fire delay. Reader now signals halt on first read; the
inner-loop halt check on iteration 2 breaks 'outer. No wallclock race.
2026-04-25 17:33:52 -07:00
MattJackson b4951b1c5b v0.13.12 — Fix 1+2+4 + cross-platform SCSI parity (RIP_DESIGN.md §6, §7, §15.1)
Fix 1: delete stall guard from Disc::copy. Pass 1 must sweep end-to-end
per ddrescue model (RIP_DESIGN.md §2.1, §3, §9). The v0.13.9 guard at
disc/mod.rs broke Pass 1 at 30% on Dune 2 with 56 GB still NonTried.
Removed stall_secs field, narrative comment in scsi/linux.rs, and the
broken regression test. Replaced with test_disc_copy_completes_full_disc_
with_failing_reader and test_disc_copy_halts_promptly_on_failing_reader.

Fix 2: async SCSI transport recovery. Added Arc<AtomicI32> fd_recovery
on SgIoTransport. On poll timeout: spawn close + spawn open in
background, return Err immediately. Top of execute() swaps fd from
recovery atomic. Main thread never blocked beyond ~1.5s poll budget
(was up to ~60s per timeout because kernel serialized main-thread
open() against in-flight close()). Drop drains pending recovery fd.

§15.1 cross-platform parity: Windows + macOS now have the same
observable recovery contract. SptiTransport gets try_recover()
(synchronous CloseHandle + CreateFileW; Windows close is fast, no
in-flight CDB drain like Linux). MacScsiTransport gets try_recover()
(release IOKit interface + reacquire via new acquire_device_iface()
helper); stores bsd_name for re-resolution. Drop guards null'd-out
interfaces. Stripped English error strings ("try as root" / "run as
administrator") on Linux + Windows. Fixed Windows TimeOutValue
ms→s ceiling so 1500ms gets 2s (was 1s; broke Drive::read fast path).

Fix 4: instrument Disc::patch arms. PatchResult exposes
blocks_attempted, blocks_read_ok, blocks_read_failed so the v0.13.11
mystery (Dune 2 Pass 2 recovered 0 bytes in 100 min) is diagnosable
from the live device log without re-instrumenting from outside.

Cleanup: honor PatchOptions::full_recovery (was read into _ and
ignored; now routed to read_sectors recovery arg). Updated
CopyOptions::batch_sectors doc to describe the actual production
path (sysfs detect_max_batch_sectors, typically 60 sectors / ~120 KB
on BU40N) rather than the test-only 32-sector internal default.

All four crates clippy-clean and tests green on the host targets
(macOS native + cargo check on Linux). Cross-platform CI watches
Linux + Windows + macOS builds + tests.
2026-04-25 17:30:25 -07:00
MattJackson 92dc145864 v0.13.11: revert SgIoTransport timeout path — keep transport alive
v0.13.10's 'fd=-1 on first poll timeout' was too aggressive: a single
transient killed the entire transport, Pass 1 finished in 45ms with
0 GB good on Dune 2.

Revert to spawn-close + main-thread-reopen (the v0.13.5/8 pattern).
Per-timeout cost is up to ~60s while the kernel completes the
abandoned command, but the v0.13.9 Disc::copy stall guard caps
catastrophic stalls at 120s of bytes_good non-advance. Pass 1 bails
cleanly with NonTrimmed ranges; Pass 2 has a working Drive for
retries with recovery=true + 30s timeouts.
2026-04-25 08:47:10 -07:00
29 changed files with 5455 additions and 1295 deletions
+681
View File
@@ -1,5 +1,686 @@
# Changelog
## 0.17.0 (2026-05-04)
### Code quality: unwrap safety, clippy compliance, test coverage
- **Sweep() hot path**: Replaced `.err().unwrap()` in `disc/mod.rs:1553` with explicit pattern matching (`match read_result { Ok(_) => unreachable!(), Err(e) => e }`). The original unwrap was logically safe (in the else branch after `is_ok()` check) but pattern matching makes the invariant explicit and avoids panic risk if logic changes.
- **cargo clippy --lib**: Clean build with `-D warnings` across all targets. No lint failures.
- **cargo test --lib**: All 256 tests pass (0 failed, 1 ignored). Integration tests for sweep/patch/dev-null validate the multi-pass recovery pipeline end-to-end.
### Patch pass algorithm fix
- `patch_internal` excludes Unreadable sectors from work list (only retries NonTrimmed/NonScraped/Unscraped). Previously attempted to retry already-failed sectors, wasting time and masking real progress.
- Exposes `bytes_bad_in_title` in patch results for accurate UI reporting of unrecoverable data vs. recovered data.
## 0.16.1 (2026-04-30)
### Unified progress display for sweep and patch
- Patch passes now show `bytes_good / disc_total` as GB (grows when sectors are recovered), with pass progress `%` using `work_done / work_total` (always advances). `Xs unreadable` declines as data is recovered. Same UI for all pass types.
## 0.16.0 (2026-04-30)
### IOKit registry-based drive enumeration, BSD name matching, reverse patch default
- **`shim_open_exclusive`** now matches the correct `IOBDServices` for the requested BSD name. Walks all IOBDServices entries in the IOKit registry, matches child IOMedia `"BSD Name"` property. Falls back to IOMedia parent walk, then first-match. Fixes multi-drive systems.
- **`shim_list_drives`** (new): registry-based drive enumeration. Reads IOBDServices `"Device Characteristics"` for vendor/model/firmware and child IOMedia for BSD name. Zero SCSI, zero exclusive access, zero unmounts. Fixes the enumeration blast that unmounted every disk on the system.
- **`list_drives()`** and **`find_drives()`** rewritten to use `shim_list_drives`. No longer iterates `/dev/disk0..15` opening exclusive SCSI on each.
- **`patch_internal`** defaults `reverse: true`. Sweep jumps forward with escalating gaps, so NonTrimmed ranges have good data at their tail. Reverse patch hits good data first, converges on actual bad block boundaries.
## 0.15.1 (2026-04-30)
### Fix damage-jump detection, fix dispatch covers_disc check
- Damage-jump tuning: `DAMAGE_WINDOW=16`, `DAMAGE_THRESHOLD_PCT=12%` (was 50/25%). Old params were too diluted by good reads between sparse failures; new params trigger on 2nd scattered failure.
- `Disc::copy()` dispatch: `covers_disc` now checks `map.total_size() == disc_size` (byte-for-byte, not approximate). Fixes false sweep dispatch when mapfile exists but doesn't cover full disc.
- Sweep resume: when dispatched from existing mapfile with NonTried, passes `resume: true`.
## 0.15.0 (2026-04-30)
### Multipass dispatch rewrite, speed control on damage zone entry/exit
- `CopyOptions { decrypt, multipass, progress, halt }`. `Disc::copy()` auto-detects sweep vs patch from mapfile state: no mapfile → sweep, NonTried → sweep with resume, only NonTrimmed/NonScraped/Unreadable → patch, clean → no-op.
- `Disc::mapfile_for()`: `/dev/null` output → `/tmp/<disc_name>.mapfile`, otherwise `mapfile_path_for(path)`.
- Speed control: damage zone entry → `set_speed(0x0000)` (minimum), 16 consecutive good reads → `set_speed(0xFFFF)` (maximum). Drive manages optimal speed in clean sections.
- `SET CD SPEED` SCSI command via `SectorReader::set_speed()` (default no-op, Drive impl sends SCSI).
## 0.14.0 (2026-04-30)
### Damage-jump algorithm replaces probe, bridge degradation detection
- Damage-jump algorithm: when damage threshold exceeded in sliding window, jump ahead by `256×batch×multiplier` sectors. Doubles multiplier on each jump. Zero-fills gap as NonTrimmed.
- `ecc_sectors()`: returns ECC block size per disc format (32 for UHD, 16 for BD, 16 for DVD).
- Bridge degradation detection: NOT READY with sense key 2/ASC 0x04/ASCQ 0x3E triggers 10s cooldown, up to 5 times before treating as bad sector.
## 0.13.43 (2026-04-29)
### Pass 1 transport-failure recovery loop
- Transport failure (USB bridge crash) no longer kills the entire rip.
- Autorip re-discovers the drive after USB re-enumeration and resumes
from the mapfile. Up to 10 attempts.
- `Error::is_scsi_transport_failure()` now matches DiscRead with
status 0xFF (bridge crash) in addition to ScsiError.
- DriveSession tracks device_path for re-discovery.
## 0.13.42 (2026-04-29)
### Fix: transport failure skips instead of aborting
- Transport failure (USB bridge crash) now skips the failed ECC block
(marks NonTrimmed) and continues. 3 consecutive transport failures
still abort the copy. Previously, a single transport failure killed
the entire rip.
## 0.13.41 (2026-04-29)
### Debug logging for sector-0 regression diagnosis
- Add debug logging to Drive::read and Disc::copy first reads.
- No functional changes.
## 0.13.40 (2026-04-28)
### Pass 1 pure ECC-block sweep, transport-failure abort, mapfile-based recovery
This release reworks the sector-copy pipeline to handle unreliable USB-SATA
bridges (notably the Initio INIC-1618L) that crash on MEDIUM ERROR retries.
**Disc::copy() — Pass 1 (ECC-block sweep):**
- Reads `batch` sectors (default 32 = 1 BD ECC block = 64 KB).
- Success → mark Finished. MEDIUM ERROR → zero-fill, mark NonTrimmed, advance.
- Transport failure (host_status=7) → abort immediately, return error.
- No single-sector reads, no retry, no state machine in Pass 1.
**Disc::patch() — Pass 2+ (single-sector recovery):**
- Reads NonTrimmed sectors one at a time with pause between failures.
- Succeeds → mark Finished. Fails → mark Unreadable (or leave for next pass).
- Multi-pass: caller runs patch repeatedly until 0 recovered.
**CopyOptions simplified:**
- Removed `skip_forward`, `cautious_pause_ms`, `BPT1_EXIT_THRESHOLD`.
- Fields: `decrypt`, `resume`, `batch_sectors`, `skip_on_error`, `progress`, `halt`.
**Other fixes since 0.13.26:**
- `open()` just opens the device — no side effects. `drive_has_disc()` is a
standalone TUR, not a probe sequence.
- `enumerate_sg_names()` skips unreadable `/sys` type files.
- SCSI sense data preserved in DiscRead errors (status + key + ASC/ASCQ).
- MapStats splits `bytes_pending` into `nontried` / `retryable`.
- Wallclock rip budget — halt after max(disc_runtime_secs, 3600).
- Patch instrumentation: counters for reads_ok/err, writes_ok/err, finished/unreadable.
- `as_encoded_bytes()` replaces `as_bytes()` for portable OsStr handling.
- Removed inline retry/reset from Drive::read — orchestration layer handles recovery.
- Removed all `(internal)` references from public code.
## 0.13.26 (2026-04-27)
### Extend DiscRead with SCSI status/sense for 30% wedge diagnostics
`Error::DiscRead` now carries `status` (SCSI status byte) and `sense`
(ScsiSense with key/asc/ascq). Previously this info was discarded,
showing only `E6000: {sector}`. Now shows:
- `E6000: {sector} 0x{status}/0x{sense_key}/0x{asc}`
- Enables recovery loop to distinguish recoverable errors from drive wedge
- Enables programmatic handling: `if status == 0xFF { reset } else { retry }`
### Error display shows up to 5 fields
Display format changed from `E{sector}` to `E{code}: sector status/key/asc`.
## 0.13.25 (2026-04-27)
### Drop dead `Drive::device_path_owned()`
The method was marked `// NOTE: Debug aid — remove after fd issue is
resolved` and the fd issue closed in 0.13.6. Use `device_path()` (which
returns `&str`) instead. Removing it clears a `cargo clippy -- -D
warnings` red on Linux CI that the Mac toolchain doesn't catch.
### Pre-commit gate uses CI's exact toolchain
`(internal)/scripts/precommit.sh` (new) runs `cargo +1.86 fmt
--check`, `cargo +1.86 clippy -- -D warnings`, and `cargo +1.86 test
--tests` across all 5 freemkv crates. Mirrors each repo's
`.github/workflows/ci.yml` step-for-step. Use it as a pre-commit hook
or run by hand before pushing — green here means green CI.
The Mac default toolchain is newer (1.94) and its clippy rejects
slightly different sets of lints than 1.86 — running locally without
pinning misses lints CI catches. The script forces 1.86 so drift
between local and CI ends.
## 0.13.24 (2026-04-27)
### MapStats: split `bytes_pending` into `bytes_nontried` + `bytes_retryable`
`MapStats.bytes_pending` aggregates `NonTried` (sectors Pass 1 hasn't
reached) + `NonTrimmed` + `NonScraped` (sectors flagged for Pass 2-N
retry). UIs that wanted a "MAYBE / will retry" bucket were stuck
showing the entire unread disc as "Maybe" at pct=0.
v0.13.24 keeps `bytes_pending` for back-compat and adds two granular
fields:
- `bytes_nontried` — Pass 1 hasn't read these yet
- `bytes_retryable` — `NonTrimmed + NonScraped`, Pass 2-N will retry
`bytes_pending == bytes_nontried + bytes_retryable` (invariant).
### cargo fmt cleanup
Picks up the `cargo fmt --check` lint failure that's been red on
`main` since v0.13.18 (long format-string layouts the local rustfmt
folded differently from CI's runner).
## 0.13.23 (2026-04-27)
### Stop discarding the drive's SCSI sense data
Through the entire 0.13.x line, every CHECK CONDITION reply from the
drive (the standard way SCSI reports a sector failure) was being
collapsed into a synthetic `status=0xFF, sense_key=0` "transport
wedge" sentinel and the real sense data was thrown away. Live tracing
on the BU40N reading Dune 2 on 2026-04-27 confirmed it: the drive was
returning `host_status=0, driver_status=8, status=2, exec_elapsed_ms=1416`
on every bad sector — a clean CHECK CONDITION carrying full sense
data — and the library was misclassifying it as a wedge and bailing.
Root cause: `scsi/linux.rs`'s wedge check was `host_status != 0 ||
driver_status != 0`. SG's `DRIVER_SENSE` bit (0x08) is set on every
CHECK CONDITION reply just to flag "sense buffer is populated" — it's
not a transport failure on its own. Pre-0.13.23 we conflated the two
and silently lost every drive-reported error reason. macOS and Windows
backends had the same shape: they extracted `sense_key` only, dropping
ASC/ASCQ.
### What 0.13.23 changes (API)
- **Linux**: mask `DRIVER_SENSE` before treating `driver_status` as a
transport-layer failure. Real transport failures (`host_status != 0`
or any non-SENSE bit set) still synthesise the `0xFF` sentinel.
- **`Error::ScsiError`** carries `sense: Option<ScsiSense>` instead of
flat `sense_key`/`asc`/`ascq`. `sense=None` ⇔ transport failure (no
SCSI status delivered). `Some(ScsiSense {…})` ⇔ drive replied with
sense data. Removes the `0xFF`/`sense_key=0` magic-number coupling.
- **`ScsiSense`** is a public type with predicate methods on it —
`is_marginal`, `is_medium_error`, `is_hardware_error`,
`is_unit_attention`, `is_data_protect`, `is_not_ready`,
`is_illegal_request`, `is_aborted_command`. Callers route on the
structured fields rather than raw key comparisons.
- **`Error::scsi_sense()`** / **`Error::is_scsi_transport_failure()`** /
**`Error::is_marginal_read()`** convenience predicates on `Error`.
`is_marginal_read` is the high-level "should `Disc::copy` engage
hysteresis on this error?" check.
- **SCSI protocol constants** (`SCSI_STATUS_GOOD`,
`SCSI_STATUS_CHECK_CONDITION`, `SCSI_STATUS_TRANSPORT_FAILURE`,
`SENSE_KEY_*`) moved from `error.rs` to `scsi/mod.rs` where they
belong alongside `SCSI_INQUIRY`, `SCSI_READ_10`, etc.
- **macOS** + **Windows** backends parse the full sense triple too.
Same code path on every platform — a regression in `parse_sense`
would surface on all three OSes simultaneously.
- **`parse_sense`** replaces `parse_sense_key` (returns the full
triple). Inline sense-format tests (descriptor 0x72/0x73 vs fixed
0x70/0x71, short-buffer, VALID-bit masking, unknown response codes)
now also exercise ASC/ASCQ extraction at the right offsets.
### Disc::copy + Disc::patch sense-aware dispatch
Both passes now bail immediately when a read fails with a sense class
that retry can't help (HARDWARE ERROR, DATA PROTECT, UNIT ATTENTION,
NOT READY, ILLEGAL REQUEST, real transport failure, kernel `IoError`)
rather than burning hysteresis cycles on a doomed loop. Marginal-read
sense (MEDIUM ERROR, ABORTED COMMAND, RECOVERED ERROR, NO SENSE)
engages hysteresis as before. New `phase=bail` trace event records
the bail reason.
`Disc::patch`'s `wedged_threshold` (50 consecutive failures) remains
as defense-in-depth for chains of marginal failures, but a single
non-marginal sense now short-circuits it.
### Behavioural impact
For damaged-disc rips on the BU40N this unblocks v0.13.22's
hysteresis: pre-fix, the misclassified "wedge" caused `Disc::copy` to
exit before hysteresis could engage, so `bytes_good` froze at the bad
zone. Post-fix the drive's CHECK CONDITION replies flow through the
normal path → hysteresis drops to bpt=1 → marginal sectors are
recovered or marked Unreadable. Calibration data
(`docs/audits/2026-04-26-bisect-on-fail-empirical-findings.md`) shows
~86 % of marginal-region sectors recover at bpt=1 on this drive.
## 0.13.22 (2026-04-26)
### Replace bisect-on-fail with hysteresis state machine (Block ↔ Single)
Live test on Dune 2 v0.13.21 showed bisect-on-fail recovered every
recoverable sector, but spent ~30 sec per damaged 60-block (paying a
~5 sec kernel timeout at every bisection level). Each level descended
log₂(60) ≈ 6 times on the failing branch.
Replaced with a two-state hysteresis machine in `Disc::copy`:
```
Block(batch):
read(batch) ok → write, advance, stay Block
read(batch) fail → switch to Single, retry SAME range at bpt=1
Single:
read(1) ok → write, consecutive_good++
if consecutive_good >= BPT1_EXIT_THRESHOLD:
switch to Block, reset counter
read(1) fail → mark NonTrimmed, consecutive_good = 0
```
`BPT1_EXIT_THRESHOLD = 10_000` sectors (= 20 MB of clean data).
Calibrated from the 2026-04-26 BU40N empirical run; tunable.
Per-block cost on a 60-sector damaged block with 1 bad sector:
- Bisect (v0.13.21): ~30 sec (5 s × 6 levels)
- Hysteresis (v0.13.22): ~10 sec (5 s bpt=batch fail + 59 × 1 ms good
+ 1 × 5 s bad)
Plus inside a damaged cluster spanning many 60-blocks, hysteresis
pays the bpt=batch fail cost ONCE on entry; bisection paid it every
60 sectors. For Dune 2's ~1248-sector boundary cluster that's ~21
fewer 5-sec waits = ~100 sec saved.
Telemetry: new `phase=mode_change` trace event with `from`, `to`,
`lba`, `consecutive_good`. Replaces v0.13.21's `phase=bisect`. The
v0.13.21 worklist DFS is gone — single iterative `for s in 0..count`
on the failure path.
Test rename:
`test_disc_copy_bisect_recovers_via_single_sector_reads` →
`test_disc_copy_hysteresis_recovers_via_single_sector_reads`. Same
synthetic BU40N-pattern reader; same 100% recovery expectation.
## 0.13.21 (2026-04-26)
### Fix: Disc::copy bisect-on-fail (replaces skip-forward)
Empirical live-hardware testing on the LG BU40N (see
`(internal)/docs/audits/2026-04-26-test-plan-audit.md` and the
TEST_PLAN.md run log) revealed that the drive often **fails
multi-sector READ commands** in damaged regions but **succeeds when
asked one sector at a time**. The old skip-forward strategy responded
to multi-sector failures by jumping up to 1 % of the disc forward,
marking everything in between as bad — losing **clean territory**
sandwiched between bad sectors.
`Disc::copy` now bisects on read failure: split the failed block in
half, retry each half, recurse down to single-sector reads. Sectors
the drive can read individually are recovered in Pass 1; only sectors
that fail at bpt=1 are marked NonTrimmed for the patch passes.
Empirical results on Dune 2 UHD on the BU40N:
- Old algorithm: 25 GB read in Pass 1, then ~6 GB skip-forwarded;
retry passes failed to recover most of the skipped zone.
- New algorithm: ~99 % of disc recovered in Pass 1; only the truly
unreadable cluster (~14 % of a 2 MB hot zone) marked NonTrimmed.
Implementation: stack-based DFS in the inner read loop. log₂(batch)
levels max — for the default 60-sector batch, 6 levels. Multi-pass
machinery is untouched: Pass 2 .. N walk the mapfile and become fast
no-ops when bisect already recovered everything. New integration test
`test_disc_copy_bisect_recovers_via_single_sector_reads` validates
the behavior against a synthetic BU40N-pattern reader.
### Fix: READ_TIMEOUT_MS bumped 1.5 s → 10 s (caller-side)
The 0.13.20 SCSI rewrite gave the kernel mid-layer the ability to run
its own ABORT/RESET escalation. But callers (`Drive::read` for the
fast path) still passed `timeout_ms=1500`. Cold-start seek on the
BU40N can take ~1.5 s, which means **normal reads were being
cancelled at the boundary**, triggering the kernel mid-layer's
escalation, which the Initio bridge couldn't drain — resulting in the
firmware-level wedge that only physical replug recovers.
Live-hardware probe data:
- Sustained sequential read: 3–7 ms
- Cold-start seek + read: up to ~1500 ms
- Successful ECC recovery: 1.6–2.6 s
- Confirmed unreadable: 3.6–8.8 s (kernel timeout)
10 s is calibrated to cover every legitimate read with margin while
still short-circuiting truly bad sectors before the kernel runs full
LUN/BUS/HOST reset. `READ_RECOVERY_TIMEOUT_MS` (60 s) unchanged.
## 0.13.20 (2026-04-26)
### Architecture: SCSI transport — sync blocking SG_IO
`scsi/linux.rs` rewritten from async `write/poll/read + 1.5 s timeout
+ close-on-timeout in bg thread` to a single synchronous blocking
`ioctl(fd, SG_IO, &hdr)`. The old pattern abandoned slow-but-alive
commands faster than the drive could drain its internal queue,
deepening the BU40N wedge. Per the audit at
`(internal)/docs/audits/2026-04-26-scsi-architecture-research.md`,
no reference project (MakeMKV / sg_dd / ddrescue) does what we did —
all use sync blocking SG_IO with 8-60 s timeouts and let the kernel's
mid-layer (`scsi_eh.rst`) run ABORT TASK / LUN RESET / BUS RESET /
HOST RESET escalation internally.
What changed:
- `SgIoTransport::execute()` is one syscall now. Caller-supplied
`timeout_ms` is honored by the kernel, which does its own
ABORT/RESET escalation if the device times out.
- Errors check `host_status` and `driver_status` (both 0xFF-synthesised
for the caller) in addition to `status` — transport-level failures
no longer slip through as Ok.
- Sense-key parser handles both descriptor format (0x72/0x73, key at
byte 1) and fixed format (0x70/0x71, key at byte 2).
- Deleted the `fd_recovery: Arc<AtomicI32>` field, the bg close+open
thread, and the stale-fd swap dance. `scsi/linux.rs` shrank from
~720 to ~520 lines.
- Module doc rewritten to reflect the new architecture.
### Architecture: parity strip on macOS + Windows
`scsi/macos.rs` and `scsi/windows.rs` had `try_recover()` —
userspace handle-recovery on task failure. Same anti-pattern as the
Linux fd-recovery dance, removed for the same reason: the kernel
mid-layer already runs its own escalation. Errors bubble up directly.
Cleanups:
- `MacScsiTransport`: `try_recover()` deleted, `bsd_name` field
deleted (was only used by try_recover), fail-fast device_iface guard
deleted (no longer null'd mid-session).
- `SptiTransport`: `try_recover()` deleted, `wide_path` field deleted,
INVALID_HANDLE guard deleted.
### API cleanup: drop `Drive::reset` and `find_drives`
Two duplicates removed from the public surface:
- `Drive::reset()` — escalating recovery (STOP/START unit + eject +
reinit). Per the audit, userspace shouldn't escalate; the kernel
already does. Only one internal caller (`wait_ready` line 195),
which now just keeps polling TUR for 60 iterations. No external
consumer used it.
- `pub fn find_drives() -> Vec<Drive>` — opened N drives just to throw
most away. Only caller was `find_drive()` itself, which now uses
`discover_drives()` directly. No external consumer used it. For
lightweight enumeration (UI sidebar etc.) use `scsi::list_drives()`.
`lib.rs` re-export of `find_drives` removed.
## 0.13.19 (2026-04-26 — held, never released)
Held in development; folded into 0.13.20.
## 0.13.18 (2026-04-26)
### Sync release — no functional changes
Bumped to satisfy the unified-versioning rule. Actual fix is in autorip
(`web.rs` two-bar UI — separates per-pass and total progress bars +
their own text rows so the rip dashboard is readable again).
## 0.13.17 (2026-04-26)
### Sync release — no functional changes
Bumped to satisfy the unified-versioning rule. Actual fix is in autorip
(hot-plug rescan in the drive poll loop — autorip now picks up
unplug/replug events without a container restart).
## 0.13.16 (2026-04-26)
### Architecture: single `Progress` trait + `PassProgress` struct
Pre-0.13.16 the rip API leaked internal mapfile concepts (`pos`,
`bytes_good`, `work_done`, `bytes_pending`, `Finished`/`NonTrimmed`)
into per-pass positional callbacks. Consumers reinvented the math each
time, and the v0.13.15 UI bug surfaced exactly because of this — autorip's
web JS computed `progress_pct` from `bytes_good` while the backend
computed from `pos`, silent drift, frozen UI bar.
This release replaces both `Disc::copy::on_progress` and
`Disc::patch::on_progress` `Fn(u64, u64, u64)` callbacks with a single
`Progress` trait and `PassProgress` struct (new `progress` module).
```rust
pub struct PassProgress {
pub kind: PassKind, // Sweep | Trim {reverse} | Scrape {reverse} | Mux
pub work_done: u64,
pub work_total: u64,
pub bytes_good_total: u64,
pub bytes_total_disc: u64,
}
pub trait Progress {
fn report(&self, p: &PassProgress);
}
impl<F: Fn(&PassProgress)> Progress for F { ... }
```
Both `CopyOptions::on_progress` and `PatchOptions::on_progress` are
renamed to `progress: Option<&dyn Progress>`. Closure callers update
trivially; struct callers gain a clean named-field shape with no
positional-arg confusion.
`PassKind` carries the semantic (sweep vs trim vs scrape vs mux) so
consumers can label phases without reinventing detection logic. The
`Mux` variant is reserved for v0.13.17 when the mux pipeline emits
progress; not yet emitted by libfreemkv code.
`Disc::patch` reports `Trim {reverse}` for retry passes with
`block_sectors >= 2` and `Scrape {reverse}` when `block_sectors == 1`
(the per-sector final pass). Direction comes through `reverse: bool`.
## 0.13.15 (2026-04-26)
### Breaking: `on_progress` callback gains `pos` parameter
Both `CopyOptions::on_progress` and `PatchOptions::on_progress` now take
`Fn(bytes_good: u64, pos: u64, total_bytes: u64)`. The new `pos` parameter
is the current sweep / retry position. Pass 1 callers should display
`pos / total_bytes` for the "% swept" UI bar — `bytes_good` only counts
clean reads (Finished sectors) and freezes during skip-forward bad zones,
which made every previous version's UI look hung at the bad-zone boundary.
This was the v0.13.9 stall-guard origin bug.
Live trace from v0.13.14: Pass 1 hit a Dune 2 bad zone at 24 GB and
appeared "stuck" for 14 minutes per autorip's UI (`bytes_good = 23.97 GB`
unchanged). Disc trace events showed `pos` actually advanced from 25.8 GB
to 70 GB during that window — Pass 1 was 83 % through the disc, marking
the post-bad-zone NonTrimmed via skip-forward exactly as designed. The
display lied. Now consumers can show the truth.
### Feature: `PatchOptions::reverse` for reverse-direction retry passes
When set, `Disc::patch` walks bad ranges from highest LBA to lowest, and
within each range reads sectors back-to-front. Hypothesis (per the live
v0.13.14 test): drives that wedge after a forward read of a bad sector
read fine when approached from end-of-disc backward — most of the
post-bad-zone NonTrimmed range is actually clean data the drive could
have read on Pass 1 had it not been wedged. autorip alternates F/R
across retry passes (Pass 2 = reverse half-batch, Pass 3 = forward
quarter-batch, ...).
### Feature: `PatchOptions::wedged_threshold` early-exit
When > 0, `Disc::patch` exits early if it sees this many consecutive
read failures with zero successful reads in the same pass. Saves the
wallclock budget for productive grinding when the drive has clearly
wedged on the bad zone for this pass — a future pass with a different
direction or block size may still recover. Reported via new
`PatchResult::wedged_exit: bool`.
### Trace: `patch_start` and `patch_done` events
`freemkv::disc` target now emits `patch_start` (block_sectors, recovery,
reverse, wedged_threshold, num_ranges) and `patch_done`
(blocks_attempted, blocks_read_ok, blocks_read_failed, wedged_exit,
halted, bytes_recovered) at Disc::patch boundaries.
## 0.13.14 (2026-04-25)
### Sync release — no functional changes in libfreemkv
Bumped solely to satisfy the unified-versioning rule. The actual fix in
this release is in autorip: the tracing subscriber now enables
`freemkv::scsi=trace,freemkv::disc=trace` so the v0.13.13 instrumentation
events actually surface in `/api/debug`. Without that filter override the
trace events were silently dropped by the default `libfreemkv=warn` rule.
## 0.13.13 (2026-04-25)
### Telemetry: instrument the rip pipeline for in-flight diagnosis
v0.13.12 shipped Fix 1+2+4 + cross-platform parity but a live test on Dune 2
showed Pass 1 sat for 14 minutes with `bytes_good=0` while the inner loop
appeared to iterate (the throttled `on_progress` log fired every 78s). The
async fd_recovery design at §7 said each `execute()` call should bound at
~1.5 s on poll timeout, with subsequent calls returning `DeviceNotFound` in
microseconds until recovery completes. Observed reality contradicts that:
each iteration takes ~60 s, not microseconds. Without trace-level telemetry
at the SCSI + Disc::copy boundaries we can't diagnose where the time goes.
This release adds the telemetry. No behavior change; instrumentation only.
- New dep: `tracing = "0.1"`. Per project docs, debug/trace logging is permitted
in libfreemkv (the no-English rule applies to errors, not telemetry).
Consumers (autorip) wire a tracing subscriber and pipe events into the
JSONL debug log automatically.
- `SgIoTransport::execute` (Linux): trace events at every state transition
(entry, recovery_swap_ok, recovery_pending, write_ok / write_err, poll_done,
timeout_spawn_recovery, scsi_err, read_err, ok). Each event includes the
opcode and elapsed timing. The bg recovery thread also traces close_ms +
open_ms so we can see if the kernel is hanging close+open.
- `Disc::copy`: trace events at copy_start, outer_loop, region_enter, every
100 inner-loop iterations (iter_progress with pos / region_end / skip_size /
bytes_good / read_ok_count / read_err_count / last_read_ms /
copy_elapsed_ms), and copy_done.
- All trace events use `target` strings `freemkv::scsi` and `freemkv::disc`
so consumers can filter by subsystem.
### What this enables
- A live rip will now produce a SCSI event stream visible at
`/api/debug?n=N&q=freemkv::scsi`. We can finally answer: "is the inner
loop iterating slowly because each call is slow, or fast with the bg
thread blocked?"
- `bg_recovery_done` events with `close_ms` / `open_ms` reveal whether the
kernel really takes 60 s for close+open on a wedged Initio bridge.
## 0.13.12 (2026-04-25)
### Fix: delete stall guard from `Disc::copy` (RIP_DESIGN.md §6 Fix 1)
The v0.13.9 stall guard at `disc/mod.rs` exited Pass 1 early when
`bytes_good` was flat for `stall_secs` (default 120s). This violated the
ddrescue model: Pass 1 must sweep end-to-end, marking failed reads
NonTrimmed for Pass 2 retry. The guard caused Pass 1 to bail at 30% on
Dune 2 with 56 GB still NonTried, leaving Pass 2 nothing useful to do.
- Deleted the stall-guard state vars and the `if cur_good != ...
break 'outer;` block.
- Deleted `CopyOptions::stall_secs` field — no longer wired.
- Replaced the broken regression test
`test_disc_copy_stall_detection_triggers_skip_forward` with
`test_disc_copy_completes_full_disc_with_failing_reader` (asserts Pass 1
walks to end-of-disc with everything NonTrimmed when reads keep failing)
and added `test_disc_copy_halts_promptly_on_failing_reader` (halt flag
honored within 2s mid-skip-forward).
### Fix: async SCSI transport recovery (RIP_DESIGN.md §6 Fix 2 / §7)
`SgIoTransport::execute` (Linux) previously did close-in-background +
synchronous open-on-main-thread on poll timeout. The kernel serialized
the main-thread `open()` against the in-flight `close()` of the same
`/dev/sg*`, blocking the rip thread up to ~60s per timeout.
- Added `fd_recovery: Arc<AtomicI32>` field. On poll timeout, both
`close(old_fd)` AND `open(new_fd)` run in a background thread; the new
fd is published to `fd_recovery`. Returns Err immediately. Main thread
is never blocked beyond the `poll()` budget (~1.5 s).
- Top of `execute()`: if `self.fd < 0`, swap from `fd_recovery`. If
recovery is also pending, return `DeviceNotFound` and let the caller's
retry loop come back later.
- Drop drains any pending `fd_recovery` so the fd doesn't leak.
- Stripped the v0.13.9 stall-guard narrative comment that justified
the deleted behavior.
### Fix: cross-platform SCSI parity — Windows + macOS recovery (RIP_DESIGN.md §15.1)
Per the platform parity rule (no stubs), Windows and macOS now have the
same observable recovery contract as Linux:
- `SptiTransport` (Windows): added `try_recover()` that calls
`CloseHandle` + `CreateFileW` synchronously after a failed
`DeviceIoControl`. Stripped English error string ("run as
administrator") from `open()`. Fixed the ms→s timeout truncation
(1500ms now rounds up to 2s, was 1s).
- `MacScsiTransport` (macOS): added `try_recover()` that releases the
IOKit interface (`RELEASE_EXCLUSIVE` + `com_release`) and re-acquires
via the new `acquire_device_iface()` helper. Stores `bsd_name` so
recovery can re-call `find_scsi_service`.
- All three platforms: top of `execute()` returns `DeviceNotFound`
immediately if a prior `try_recover()` left the transport in an
invalid state. Drop guards null'd-out interfaces.
- Send is auto-derived on all three (i32 fd / isize HANDLE / IOKit
interface ref are Send-safe); explicit comments document the
intentional implicit Send and the absence of Sync.
### Fix: instrument `Disc::patch` — diagnostic counters (RIP_DESIGN.md §6 Fix 4)
`PatchResult` now reports `blocks_attempted`, `blocks_read_ok`,
`blocks_read_failed`. Pass 2's "100 minutes recovered 0 bytes" mystery
(Dune 2) becomes diagnosable from these counters: distinguish "drive
returned Ok but write/record dropped data" from "every read was Err for
the entire range" without instrumenting from outside the lib.
### Fix: honor `PatchOptions::full_recovery`
The field was previously read into `let _ = opts.full_recovery;` and
ignored — `read_sectors(..., true)` was hardcoded. Now routed to
`read_sectors(..., opts.full_recovery)`. Behavior unchanged for
default callers (which pass `true`).
### Doc: `CopyOptions::batch_sectors` accuracy
Doc comment said "Defaults to 32 sectors (64 KB)". Updated to describe
the actual production path: callers should resolve via
`detect_max_batch_sectors(device_path)` (kernel-reported sysfs value,
typically 60 sectors / ~120 KB on the BU40N). The 32-sector internal
fallback is only reached when `batch_sectors=None AND skip_forward=true`.
## 0.13.11 (2026-04-25)
### Fix: revert SgIoTransport timeout path to keep transport alive
v0.13.10 changed `SgIoTransport::execute` to set `fd = -1` on a poll
timeout (no reopen on the main thread, since that would serialize
against the spawned close()). The intent was to escape the 60-s
blocking reopen.
The cost was too high: a single transient poll timeout permanently
killed the transport. Live test on Dune 2 (post-replug):
- Pass 1 ran for **45 ms** then returned with 0 GB good and 80 GB
pending.
- The first SCSI READ timed out, fd went to -1, every subsequent
read returned `DeviceNotFound` instantly, Disc::copy raced through
the entire disc skip-forwarding in milliseconds.
- Pass 2 inherited the dead Drive and was equally useless.
Revert: spawn close + reopen on main thread (the v0.13.5/8
behavior). Yes the main-thread open() may block up to ~60 s while
the kernel completes the abandoned command — but the v0.13.9
`Disc::copy` stall guard already caps catastrophic stalls at 120 s
of `bytes_good` non-advance. Net: per-timeout cost is ~60 s, but
Pass 1 cleanly bails out within 120 s of any wedge, and Pass 2 has
a working Drive to retry NonTrimmed ranges with `recovery=true` +
30 s timeouts.
The integration test for the stall guard
(`test_disc_copy_stall_detection_triggers_skip_forward`) continues
to pass — the guard fires regardless of which transport-recovery
strategy is in play.
## 0.13.10 (2026-04-25)
### Version sync — no functional changes
+5 -1
View File
@@ -1,6 +1,6 @@
[package]
name = "libfreemkv"
version = "0.13.10"
version = "0.17.0"
edition = "2024"
rust-version = "1.86"
license = "AGPL-3.0-only"
@@ -24,6 +24,10 @@ rand = "0.8"
cmac = "0.7"
zip = { version = "2", default-features = false, features = ["deflate"] }
base64 = "0.22.1"
# Trace-level instrumentation for Disc::copy + SgIoTransport::execute. Permitted
# under project docs ("Acceptable strings: debug/trace logging"). Consumers (autorip)
# wire a tracing subscriber and pipe events into the JSONL debug log.
tracing = "0.1"
[target.'cfg(target_os = "linux")'.dependencies]
libc = "0.2"
+1 -1
View File
@@ -18,7 +18,7 @@ Part of the [freemkv](https://github.com/freemkv) project.
```toml
[dependencies]
libfreemkv = "0.13"
libfreemkv = "0.17"
```
## Quick Start
+312
View File
@@ -0,0 +1,312 @@
# Troubleshooting Guide
Common problems and solutions for optical drive ripping with freemkv.
---
## 1. USB-SATA Bridge Issues
This is the single most common source of problems when ripping discs over USB.
### Symptoms
- The drive disappears mid-rip. The ripping tool reports the device is gone, and `ls /dev/sg*` no longer shows it.
- The device re-enumerates under a different name: `sg4` becomes `sg5`, then `sg7`, then `sg11` after each USB port reset.
- `dmesg` shows USB port resets: `usb X-Y: reset high-speed USB device`, `xhci_hcd 0000:00:14.0: Cannot enable. Maybe the USB cable is bad?`, or `usb-storage: device reset failed`.
- The SCSI layer reports `host_status=7` (Linux USB transport error) in sense data.
- The drive works fine for reading data discs or burning, but crashes when hitting damaged sectors during a rip.
- After the crash, the drive is completely invisible until physically unplugged and reconnected.
### Root Cause
USB-SATA bridges translate between the USB Mass Storage protocol (BOT or UAS) and the drive's native SATA interface. When the optical drive encounters an unreadable sector, it returns a SCSI CHECK CONDITION with sense key 0x03 (MEDIUM ERROR). Some bridge chipsets -- particularly the Initio INIC-36xx family -- have buggy firmware that mishandles this error response.
Specific failure modes:
- **Incorrect residue reporting.** The bridge claims a different number of bytes transferred than what actually occurred. The Linux USB storage driver sees this discrepancy as a protocol violation and resets the port to recover. The `US_FL_IGNORE_RESIDUE` quirk exists specifically for this class of bug (see `drivers/usb/storage/transport.c` in the Linux kernel).
- **Bridge firmware crash.** On some Initio bridges, a malformed SCSI error response from the drive causes the bridge MCU to hang entirely. The USB controller sees the device stop responding and initiates a port reset. The bridge recovers (it re-enumerates), but the rip is dead -- all state is lost.
- **Sense data corruption.** The bridge forwards garbled or truncated sense data to the host, which the SCSI midlayer cannot parse, leading to a transport reset.
This is a hardware + firmware problem, not a software bug. The same drive connected via direct SATA does not exhibit these symptoms.
### Known Affected Bridges
| Chipset | USB IDs | Notes |
|---------|---------|-------|
| Initio INIC-3609 | `13fd:3609` | Very common in cheap SATA-to-USB enclosures. Highly problematic. |
| Initio INIC-3619 | `13fd:3940` | Same firmware family as INIC-3609. |
| Initio INIC-3069 | `13fd:0840` | Older variant, same residue bug. |
| ASMedia ASM1051 | `174c:5106` | Early ASM SATA bridge. Residue issues on error paths. |
| JMicron JMB36x | `152d:0561` | Some firmware versions. Not all JMicroon chips are affected. |
If your drive came in a pre-built external enclosure (Vantec, Sabrent, OWC, etc.), it almost certainly uses one of these bridge chips internally.
### The Fix: USB Storage Quirk
Apply the `US_FL_IGNORE_RESIDUE` kernel quirk for your bridge. This tells the Linux USB storage driver to ignore the residue field in SCSI response frames, preventing the port reset on mismatched byte counts.
**Step 1: Identify your bridge's vendor:product ID.**
```bash
lsusb
```
Look for your drive's entry. Example output:
```
Bus 002 Device 005: ID 13fd:0840 Initio Corporation INIC-3609
```
Here the vendor ID is `13fd` and the product ID is `0840`.
**Step 2: Apply the quirk at runtime.**
```bash
echo "13fd:0840:i" > /sys/module/usb_storage/parameters/quirks
```
Replace `13fd:0840` with your device's actual IDs. The `:i` flag means `US_FL_IGNORE_RESIDUE`.
You can combine multiple flags. Common additions:
- `:i` -- ignore residue (`US_FL_IGNORE_RESIDUE`)
- `:u` -- force BOT mode instead of UAS, for bridges with UAS bugs
**Step 3: Reconnect the drive.** Unplug and replug the USB cable, or bind/unbind the device. The quirk is applied per-module-load, so existing sessions may need the drive reconnected.
### Making It Persistent
Add the quirk to your kernel boot parameters so it survives reboots.
Edit `/etc/default/grub` (GRUB) and add to `GRUB_CMDLINE_LINUX_DEFAULT`:
```
GRUB_CMDLINE_LINUX_DEFAULT="quiet usb_storage.quirks=13fd:0840:i"
```
Then rebuild the GRUB config:
```bash
sudo update-grub
```
For systemd-boot, add to your loader entry or `/etc/kernel/cmdline`:
```
usb_storage.quirks=13fd:0840:i
```
Multiple devices can be separated by commas:
```
usb_storage.quirks=13fd:0840:i,174c:5106:u
```
### Recommended Bridges
If you are buying a USB-SATA adapter or enclosure for optical drive use:
| Bridge | USB IDs | Notes |
|--------|---------|-------|
| ASMedia ASM1153 | `174c:1153` | Reliable. Widely available in SATA-USB 3.0 cables. |
| JMicron JMS578 | `152d:0578` | Good firmware. Supports UASP. |
| Icy Box IB-AC640-C3 | N/A | Uses a known-good bridge internally. Plug-and-play. |
Avoid any enclosure or adapter listing an Initio chipset.
### Best Solution: Direct SATA
Connect your optical drive directly to a motherboard SATA port. This eliminates the USB-SATA bridge entirely and is the most reliable configuration:
- No USB protocol overhead or translation errors.
- No bridge firmware bugs.
- No port resets or re-enumeration.
- Full SATA error recovery handled natively by the kernel's libata driver.
- Sustained read speeds are limited only by the drive, not the USB bus.
If your machine has a free SATA port, use it.
---
## 2. Damaged Disc Handling
### Symptoms
- SCSI MEDIUM ERROR (sense key 0x03) at specific LBAs. `dmesg` shows `sr X:0:0:0: [srY] Unrecoverable read error` or similar.
- Read speed drops to near zero when approaching a damaged area.
- The drive makes audible retrying noises (laser repositioning, spindle speed changes).
- On USB-connected drives: the bridge crashes (see section 1 above) when the drive returns the error.
### How freemkv Handles This
freemkv uses a three-layer recovery model. See [`docs/rip-recovery.md`](docs/rip-recovery.md) for full details.
- **Pass 1 (Disc::copy):** Fast sweep with 64 KB reads. On failure, zero-fills the block and skips forward. Writes a ddrescue-format mapfile for later retry.
- **Pass 2+ (Disc::patch):** Targeted re-reads of bad ranges with a long 30-second timeout per CDB. The drive firmware performs its own ECC and laser power retries within that window.
- **In-stream (DiscStream):** Adaptive batch halving -- reduces request size on failure to isolate bad sectors within a larger block.
This means a disc with some bad sectors will still produce a usable ISO. The damaged areas are zero-filled in pass 1 and retried in subsequent passes. Structure-protected sectors (deliberate unreadable regions from copy protection) will never yield, which is expected.
### The Drive Taint Issue (LG BU40N)
Some drives, notably the LG BU40N, exhibit a "taint" behavior after encountering MEDIUM ERRORs:
1. The drive hits a damaged sector and returns a MEDIUM ERROR.
2. From that point forward, **all subsequent reads fail** -- even reads to sectors that were previously successful.
3. The only recovery is to physically unplug and reconnect the drive (or power-cycle it).
This is not a freemkv bug. It is a drive firmware behavior triggered by the interaction between the drive's internal error recovery and the USB-SATA bridge's handling of the error response. The drive firmware enters a degraded state that it does not recover from without a power cycle.
Workarounds:
- **Use a direct SATA connection.** This eliminates the bridge interaction that triggers the taint.
- **Use a different bridge.** The ASM1153 and JMS578 are less likely to trigger this behavior.
- **Accept the partial ISO.** freemkv's skip-forward recovery will zero-fill the unreadable blocks and continue. The resulting ISO may be playable with minor glitches in the affected areas.
- **Physical replug between retry passes.** If running multi-pass patch, replug the drive between passes to clear the taint state.
freemkv deliberately does not attempt inline SCSI resets or eject cycles to recover from this state, because those operations were found to make the problem worse on affected hardware (see the design rationale in [`docs/rip-recovery.md`](docs/rip-recovery.md)).
---
## 3. Drive Not Detected
### Check Hardware Visibility
```bash
lsusb
```
Verify the drive appears in the USB device list. If it does not show up, the drive is not visible to the host at all -- check cables, power, and USB port.
```bash
ls /dev/sg*
```
On Linux, optical drives appear as `/dev/sg*` devices (the SCSI Generic interface). freemkv uses `/dev/sg*`, not `/dev/sr*`. If `lsusb` shows the device but no `/dev/sg*` entry exists, the `sg` kernel module may not be loaded:
```bash
sudo modprobe sg
```
### Check Kernel Messages
```bash
dmesg | grep -i usb | tail -30
dmesg | grep -i sg | tail -10
```
Look for:
- USB enumeration errors or failed port resets.
- `sg_add` messages confirming the sg device was registered.
- Permission denied or access errors.
### Permission Issues
On most Linux distributions, `/dev/sg*` devices are owned by `root:disk` or `root:cdrom` with restricted permissions. Running freemkv as an unprivileged user will fail with permission errors.
Options:
- Add your user to the appropriate group:
```bash
sudo usermod -aG disk $USER
```
Then log out and back in for the change to take effect. On some distributions the group is `cdrom` or `optical` instead of `disk`.
- Run with elevated privileges:
```bash
sudo freemkv ...
```
- Install a udev rule for persistent per-device permissions. Create `/etc/udev/rules.d/99-sg-optical.rules`:
```
SUBSYSTEM=="scsi_generic", ATTRS{type}=="5", MODE="0666"
```
Then reload udev rules:
```bash
sudo udevadm control --reload-rules && sudo udevadm trigger
```
### Spin-Up Delay
Optical drives take 30-60 seconds to spin up and become ready after hot-plug or disc insertion. During this window, SCSI commands may return NOT READY or timeout.
freemkv's `Drive::wait_ready()` handles this automatically by polling with TEST UNIT READY until the drive responds. If you are writing your own code using the library, always call `wait_ready()` before `init()`:
```rust
let mut drive = Drive::open(Path::new("/dev/sg4"))?;
drive.wait_ready()?; // blocks until disc is ready, up to 30s
drive.init()?;
```
If the drive was just plugged in, wait a full minute before concluding it is not detected.
---
## 4. How to Identify Your USB-SATA Bridge
If you are experiencing the issues described in section 1, you need to know which bridge chipset your adapter or enclosure uses.
### Step 1: Find the Device
```bash
lsusb
```
Look for entries matching your drive or enclosure. Bridges may appear under their own manufacturer name or as a generic SATA device. Common examples:
```
Bus 002 Device 005: ID 13fd:0840 Initio Corporation
Bus 002 Device 006: ID 174c:1153 ASMedia Technology Inc. ASM1153
Bus 002 Device 007: ID 152d:0578 JMicron Technology Corp. JMS578
```
### Step 2: Match the IDs
| Vendor | Product ID | Chipset | Status |
|--------|-----------|---------|--------|
| `13fd` | `3609` | Initio INIC-3609 | Affected. Apply quirk. |
| `13fd` | `3940` | Initio INIC-3619 | Affected. Apply quirk. |
| `13fd` | `0840` | Initio INIC-3069 | Affected. Apply quirk. |
| `174c` | `5106` | ASMedia ASM1051 | Affected (early firmware). Apply quirk. |
| `174c` | `1153` | ASMedia ASM1153 | Good. No quirk needed. |
| `152d` | `0561` | JMicron JMB36x | Affected (some firmware). Apply quirk if issues occur. |
| `152d` | `0578` | JMicron JMS578 | Good. No quirk needed. |
### Step 3: Check dmesg for the Bridge Name
```bash
dmesg | grep -i "usb-storage\|uas\|initio\|asmedia\|jmicron"
```
This often reveals the bridge chipset even when `lsusb` shows a generic name.
### Step 4: If the Enclosure Is Sealed
Many external drive enclosures (Vantec NexStar, Sabrent, OWC, etc.) do not advertise the bridge chipset on the packaging. In this case:
1. Check `lsusb` while the enclosure is connected.
2. Search the vendor:product ID online -- there are community-maintained lists of which chipsets popular enclosures use.
3. If you cannot determine the chipset and are experiencing bridge crashes, assume it is an Initio and apply the quirk with its IDs.
4. The definitive test: connect the bare drive to a motherboard SATA port. If the problems disappear, the bridge was the cause.
---
## 5. General Debugging Checklist
When something goes wrong during a rip, gather this information before filing an issue:
1. **freemkv version:** `freemkv --version` or the crate version in `Cargo.toml`.
2. **Drive model:** from the drive label, or from `freemkv info`.
3. **Connection type:** USB (with bridge chipset if known) or direct SATA.
4. **Operating system and kernel:** `uname -a`.
5. **Kernel messages during the failure:** `dmesg | tail -50` immediately after the crash.
6. **SCSI device:** which `/dev/sg*` the drive was on, and whether it changed after the failure.
7. **The disc:** title, format (BD/DVD/UHD), condition.
Include all of the above in bug reports. SCSI transport errors that resolve with the `US_FL_IGNORE_RESIDUE` quirk or by switching to direct SATA are bridge firmware bugs, not freemkv bugs.
+3 -4
View File
@@ -8,13 +8,12 @@ fn main() {
let device = std::env::args()
.skip(1)
.find(|a| !a.starts_with('-'))
.unwrap_or_else(|| {
let drives = libfreemkv::find_drives();
if drives.is_empty() {
.unwrap_or_else(|| match libfreemkv::find_drive() {
Some(d) => d.device_path().to_string(),
None => {
eprintln!("No drives found");
std::process::exit(1);
}
drives[0].device_path().to_string()
});
let mut drive = Drive::open(Path::new(&device)).unwrap_or_else(|e| {
+29
View File
@@ -3,5 +3,34 @@ fn main() {
if target == "macos" {
println!("cargo:rustc-link-lib=framework=IOKit");
println!("cargo:rustc-link-lib=framework=CoreFoundation");
let out_dir = std::env::var("OUT_DIR").unwrap();
let obj = format!("{out_dir}/macos_shim.o");
let lib = format!("{out_dir}/libmacos_scsi.a");
std::process::Command::new("cc")
.args([
"-c",
"src/scsi/macos_shim.c",
"-o",
&obj,
"-framework",
"IOKit",
"-framework",
"CoreFoundation",
"-Wall",
"-O2",
])
.status()
.expect("failed to compile macos_shim.c");
std::process::Command::new("ar")
.args(["rcs", &lib, &obj])
.status()
.expect("failed to create static lib");
println!("cargo:rustc-link-search=native={out_dir}");
println!("cargo:rustc-link-lib=static=macos_scsi");
println!("cargo:rerun-if-changed=src/scsi/macos_shim.c");
}
}
+1 -1
View File
@@ -102,7 +102,7 @@ Recovery is layered above `Drive::read`, not inside it. Layer 1
Layer 3 (`DiscStream::fill_extents` adaptive batch sizer) handles in-loop
request-size adaptation. Inline recovery (gentle retry → SCSI reset → retry)
was removed in 0.13.6 — see [`rip-recovery.md`](rip-recovery.md) and
`(internal)/postmortems/2026-04-25-stop-wedge-and-zero-kbs.md`.
the stop-wedge postmortem (2026-04-25).
---
+12 -2
View File
@@ -67,7 +67,7 @@ Recovery is layered above `Drive::read`:
streak.
Inline recovery (5× gentle retry → close + reset + reopen → 5× more) was
removed in 0.13.6. See `(internal)/postmortems/2026-04-25-stop-wedge-and-zero-kbs.md`
removed in 0.13.6. See the stop-wedge postmortem (2026-04-25)
for rationale: the inline reset wedged drive firmware on the LG BU40N (Initio
USB-SATA bridge) without ever recovering a sector. See
[`rip-recovery.md`](rip-recovery.md) for the full three-layer model.
@@ -99,7 +99,6 @@ descriptors or calls ioctls outside of a `ScsiTransport` implementation.
|----------|---------------|--------|
| Linux | `SgIoTransport` — async `write`/`poll`/`read` on `/dev/sg*` | `/dev/sg*` |
| macOS | `MacScsiTransport` — IOKit SCSITask | IOKit service |
| Windows | `WindowsScsiTransport` — SPTI | `\\.\CdRomN` |
The Linux backend uses the sg driver's asynchronous interface: `write()` submits
the command, `poll()` waits with an enforceable wall-clock timeout, `read()`
@@ -107,6 +106,17 @@ retrieves the result. If `poll()` times out, the fd is abandoned (closed in a
background thread) and a fresh fd opened — the kernel's USB error recovery
cannot block us. Opens with `O_RDWR | O_NONBLOCK`.
The macOS backend uses a C shim (`macos_shim.c`) for IOKit exclusive access.
The shim handles:
1. `shim_open_exclusive(bsd_name)` — unmounts the target device via `diskutil`,
then walks the IOKit registry to find the `IOBDServices` matching the
requested BSD name (IOBDServices → IOBDBlockStorageDriver → IOMedia → "BSD Name"),
then creates MMCDeviceInterface → SCSITaskDeviceInterface → ObtainExclusiveAccess.
2. `shim_list_drives()` — registry-based enumeration with zero SCSI, zero exclusive
access, zero unmounts. Reads IOBDServices "Device Characteristics" for
vendor/model/firmware and child IOMedia "BSD Name" for the device path.
3. `shim_execute()` / `shim_close()` — raw CDB dispatch and cleanup.
On non-zero SCSI status, the transport parses sense key from the sense buffer
and returns `Error::ScsiError`.
+34 -35
View File
@@ -62,52 +62,51 @@ Position and size are hex byte offsets into the ISO.
### `CopyOptions` and `PatchOptions`
Defaults preserve pre-`0.11.21` behavior — abort on the first unreadable
sector. Opt in to the recovery-friendly path:
`Disc::copy()` auto-detects the pass from mapfile state:
```rust
CopyOptions {
skip_on_error: true, // zero-fill bad blocks, continue
skip_forward: true, // exponential skip-forward after a failure
resume: true, // pick up from an existing ISO + mapfile
decrypt: false, // keep the ISO a raw disc image
..Default::default()
decrypt: true, // decrypt AACS/CSS sectors
multipass: true, // enable mapfile + skip-on-error + damage-jump
progress: Some(&reporter), // progress callback
halt: Some(flag), // halt flag for graceful stop
}
```
Dispatch logic:
- No mapfile → sweep (fresh Pass 1)
- Mapfile with NonTried (`?`) → sweep with resume
- Mapfile covering full disc, only NonTrimmed/NonScraped/Unreadable → patch
- Mapfile clean → no-op
## Algorithm
### Pass 1 — fast sweep (`Disc::copy`)
### Pass 1 — fast sweep (`Disc::copy` → `sweep_internal`)
1. Read 64 KB (32 sectors, one BD ECC block) at the current LBA via
`Drive::read(.., recovery=false)` — short 1.5 s timeout, single shot.
2. On success: mark the range `+`, advance by one block.
3. On failure (with `skip_on_error`): zero-fill the block in the ISO, mark
it `*`, advance.
4. If `skip_forward` is set: after a failure, jump ahead by an exponentially
growing amount (256 KB initial, doubling on consecutive failures, capped at
1% of disc). The skipped bytes are also marked `*` — `patch` will visit
them later.
5. Reset the skip size to 256 KB on the first success after a failure.
1. Read one ECC block (32 sectors for UHD, 16 for BD/DVD) at the current LBA.
2. On success: write data to ISO, mark `+`, advance.
3. On failure (with `multipass`): zero-fill, mark `*`, advance.
4. Track a sliding window of the last 16 ECC block results. When ≥12% are failures
→ **damage-jump**: skip ahead by `256×batch×multiplier` sectors (8 MB base for
UHD). Double the multiplier on each jump (8→16→32→64 MB...). Zero-fill the gap as `*`.
5. On 16 consecutive good reads: reset jump multiplier to 1, restore max read speed.
6. Speed control: damage zone entry → minimum speed, exit → maximum speed.
7. Only transport failures (USB bridge crash) abort the pass.
Pass 1 completes when every byte has terminal status (`+`, `-`, or the caller
bails via the halt flag).
Pass 1 completes when every byte has been visited (either `+` or `*`).
### Pass 2+ — patch (`Disc::patch`)
### Pass 2+ — patch (`Disc::copy` → `patch_internal`)
`Disc::patch` reads the mapfile and iterates every non-`+` range. For each:
`Disc::patch` reads the mapfile and iterates every non-`+` range. Default: **reverse** mode
(walks ranges from highest LBA to lowest, within each range from end to start).
1. Issue a drive read via `Drive::read(.., recovery=true)` — long 30 s
timeout, still single shot. Drive firmware does its own ECC and retries
inside that window; userspace does not pile on additional retries here.
2. On success: write the good bytes into the ISO at the exact byte offset,
mark `+`.
3. On failure: mark `-`.
4. Update the mapfile after every block — crash-safe resume.
Idempotent. Call `patch` N times for N retry attempts; typically the caller
stops early if a pass recovers zero bytes (structure-protected sectors will
never yield).
1. Issue a single-sector read with 60 s timeout (`recovery=true`). Drive firmware
does its own ECC recovery inside that window.
2. On success: write the good bytes into the ISO, mark `+`.
3. On failure with non-marginal SCSI sense: bail immediately (drive won't produce data).
4. On failure with marginal sense: mark `-`, continue.
5. Update the mapfile after every block — crash-safe resume.
6. Wedged-drive exit: 50 consecutive failures with zero recovery → bail this pass.
### In-stream — adaptive batch halving (`DiscStream::fill_extents`)
@@ -133,8 +132,8 @@ implement retry logic.
no eject cycle. The `recovery` flag controls only the per-CDB timeout
(1.5 s vs. 30 s); on any failure it returns `Err(DiscRead)` immediately.
Inline recovery (5× gentle retry → close + SCSI reset + reopen → 5× more)
was removed in 0.13.6. See `(internal)/postmortems/2026-04-25-stop-wedge-and-zero-kbs.md`
for rationale: the inline reset on the LG BU40N (Initio USB-SATA bridge)
was removed in 0.13.6. See the stop-wedge postmortem (2026-04-25) for rationale:
the inline reset on the LG BU40N (Initio USB-SATA bridge)
wedged drive firmware below the bridge without ever recovering a sector,
and the gentle-retry phase produced long stretches of 0 KB/s with no
recoveries to show for it. Recovery responsibility is now layered: layer 1
+103 -9
View File
@@ -69,12 +69,28 @@ pub struct MapEntry {
}
/// Summary statistics over all entries.
///
/// `bytes_pending` aggregates `NonTried + NonTrimmed + NonScraped` for
/// back-compat. `bytes_nontried` and `bytes_retryable` (= NonTrimmed +
/// NonScraped) split that aggregate so UIs can distinguish *unread*
/// territory (still ahead of Pass 1's read head) from *needs-retry*
/// territory (Pass 1 already encountered, queued for Pass 2-N).
#[derive(Debug, Clone, Copy, Default)]
pub struct MapStats {
pub bytes_total: u64,
pub bytes_good: u64,
pub bytes_unreadable: u64,
pub bytes_pending: u64,
/// Sectors Pass 1 hasn't reached yet (`NonTried`). Subset of
/// `bytes_pending`.
pub bytes_nontried: u64,
/// Sectors flagged for Pass 2-N retry — `NonTrimmed` (multi-sector
/// read failed; needs split) + `NonScraped` (small-block read
/// partially recovered; remainder still pending). Subset of
/// `bytes_pending`. This is the right signal for a "MAYBE / will
/// retry" UI bucket; `bytes_pending` over-counts because it folds
/// in `bytes_nontried`.
pub bytes_retryable: u64,
}
/// Write-through mapfile. Every `record()` persists to disk immediately
@@ -84,6 +100,9 @@ pub struct Mapfile {
entries: Vec<MapEntry>,
total_size: u64,
version: String,
/// Incrementally maintained stats — updated on every `record()` call
/// so `stats()` is O(1) instead of O(n).
stats: MapStats,
}
impl Mapfile {
@@ -100,6 +119,12 @@ impl Mapfile {
}],
total_size,
version: version.to_string(),
stats: MapStats {
bytes_total: total_size,
bytes_pending: total_size,
bytes_nontried: total_size,
..Default::default()
},
};
mf.write_to_disk()?;
Ok(mf)
@@ -161,11 +186,13 @@ impl Mapfile {
}
entries.sort_by_key(|e| e.pos);
let total_size = entries.last().map(|e| e.pos + e.size).unwrap_or(0);
let stats = Self::compute_stats(&entries, total_size);
Ok(Self {
path: path.to_path_buf(),
entries,
total_size,
version,
stats,
})
}
@@ -228,6 +255,11 @@ impl Mapfile {
merged.push(e);
}
// Recompute stats from merged entries. record() is already O(n) due to
// drain-and-rebuild, so this is a constant-factor overhead. The critical
// win is that stats() is now O(1) — called millions of times in the hot
// path during sweep/patch, it just returns the cached value.
self.stats = Self::compute_stats(&merged, self.total_size);
self.entries = merged;
self.write_to_disk()?;
Ok(())
@@ -267,16 +299,25 @@ impl Mapfile {
}
pub fn stats(&self) -> MapStats {
self.stats
}
fn compute_stats(entries: &[MapEntry], total_size: u64) -> MapStats {
let mut s = MapStats {
bytes_total: self.total_size,
bytes_total: total_size,
..Default::default()
};
for e in &self.entries {
for e in entries {
match e.status {
SectorStatus::Finished => s.bytes_good += e.size,
SectorStatus::Unreadable => s.bytes_unreadable += e.size,
SectorStatus::NonTried | SectorStatus::NonTrimmed | SectorStatus::NonScraped => {
s.bytes_pending += e.size
SectorStatus::NonTried => {
s.bytes_pending += e.size;
s.bytes_nontried += e.size;
}
SectorStatus::NonTrimmed | SectorStatus::NonScraped => {
s.bytes_pending += e.size;
s.bytes_retryable += e.size;
}
}
}
@@ -295,11 +336,7 @@ impl Mapfile {
{
let file = std::fs::File::create(&tmp)?;
let mut w = std::io::BufWriter::new(file);
writeln!(
w,
"# Rescue Logfile. Created by libfreemkv v{}",
self.version
)?;
writeln!(w, "# Rescue Logfile. Created by {}", self.version)?;
writeln!(w, "# Current pos / status / pass / pass_time")?;
writeln!(w, "0x000000000 ? 1 0")?;
writeln!(w, "# pos size status")?;
@@ -450,4 +487,61 @@ mod tests {
assert_eq!(bad, vec![(100, 50), (300, 50)]);
let _ = std::fs::remove_file(&p);
}
#[test]
fn stats_consistent_after_overlapping_records() {
let p = tmpfile("stats_consistent_after_overlapping");
let _ = std::fs::remove_file(&p);
let mut mf = Mapfile::create(&p, 1000, "test").unwrap();
// Record some finished, some unreadable, some nontrimmed
mf.record(0, 300, SectorStatus::Finished).unwrap();
mf.record(300, 200, SectorStatus::NonTrimmed).unwrap();
mf.record(500, 100, SectorStatus::Unreadable).unwrap();
mf.record(600, 400, SectorStatus::Finished).unwrap();
// Final entries: [0..300 Finished, 300..500 NonTrimmed, 500..600 Unreadable, 600..1000 Finished]
let s = mf.stats();
assert_eq!(s.bytes_good, 700); // 300 + 400
assert_eq!(s.bytes_unreadable, 100); // 100
assert_eq!(s.bytes_pending, 200); // NonTrimmed only (NonTried=0)
assert_eq!(s.bytes_nontried, 0);
assert_eq!(s.bytes_retryable, 200); // NonTrimmed
assert_eq!(s.bytes_total, 1000);
// Overwrite a NonTrimmed range with Finished
mf.record(300, 100, SectorStatus::Finished).unwrap();
// Entries: [0..400 Finished, 400..500 NonTrimmed, 500..600 Unreadable, 600..1000 Finished]
let s2 = mf.stats();
assert_eq!(s2.bytes_good, 800); // 400 + 400
assert_eq!(s2.bytes_unreadable, 100);
assert_eq!(s2.bytes_pending, 100); // NonTrimmed only
assert_eq!(s2.bytes_retryable, 100);
let _ = std::fs::remove_file(&p);
}
#[test]
fn stats_consistent_after_split_record() {
let p = tmpfile("stats_consistent_after_split");
let _ = std::fs::remove_file(&p);
let mut mf = Mapfile::create(&p, 1000, "test").unwrap();
// Mark middle as NonTrimmed
mf.record(200, 400, SectorStatus::NonTrimmed).unwrap();
// Entries: [0..200 NonTried, 200..600 NonTrimmed, 600..1000 NonTried]
let s = mf.stats();
assert_eq!(s.bytes_pending, 1000); // NonTried(600) + NonTrimmed(400)
assert_eq!(s.bytes_retryable, 400); // NonTrimmed only
assert_eq!(s.bytes_nontried, 600); // 200 + 400
// Overwrite the NonTrimmed with Finished (splitting the remaining NonTried)
mf.record(200, 400, SectorStatus::Finished).unwrap();
// Entries: [0..200 NonTried, 200..600 Finished, 600..1000 NonTried]
let s2 = mf.stats();
assert_eq!(s2.bytes_good, 400);
assert_eq!(s2.bytes_pending, 600); // NonTried(200 + 400)
assert_eq!(s2.bytes_nontried, 600);
assert_eq!(s2.bytes_retryable, 0);
let _ = std::fs::remove_file(&p);
}
}
+1758 -185
View File
File diff suppressed because it is too large Load Diff
+10 -11
View File
@@ -1,26 +1,25 @@
//! macOS drive discovery and device resolution.
//!
//! `find_drives` uses IOKit registry enumeration (via `scsi::list_drives`)
//! to discover optical drives without exclusive access or unmounts. Only
//! the returned paths are then opened for INQUIRY to build full `DriveId`.
use crate::error::{Error, Result};
use crate::identity::DriveId;
pub fn find_drives() -> Vec<(String, DriveId)> {
let mut drives = Vec::new();
for i in 0..16 {
let path = format!("/dev/disk{}", i);
if !std::path::Path::new(&path).exists() {
continue;
}
match crate::scsi::open(std::path::Path::new(&path)) {
let discovered = crate::scsi::list_drives();
for info in discovered {
let path = std::path::Path::new(&info.path);
match crate::scsi::open(path) {
Ok(mut transport) => {
if let Ok(id) = DriveId::from_drive(transport.as_mut()) {
if !id.raw_inquiry.is_empty() && (id.raw_inquiry[0] & 0x1F) == 0x05 {
drives.push((path, id));
}
drives.push((info.path.clone(), id));
}
}
Err(_) => {
// Device exists but can't be opened (likely mounted).
// Use `diskutil unmountDisk /dev/diskN` to unmount before accessing.
continue;
}
}
}
+72 -133
View File
@@ -1,11 +1,15 @@
//! Drive session — open, identify, and read from optical drives.
//!
//! Three-step open:
//! 1. `open()` — open device, identify drive. Always OEM.
//! 2. `wait_ready()` — wait for disc to spin up. Call before reading.
//! 3. `init()` — activate custom firmware. Removes riplock.
//! 4. `probe_disc()` — probe disc surface. Drive learns optimal speeds.
pub(crate) fn extract_scsi_context(e: &Error) -> (u8, Option<crate::scsi::ScsiSense>) {
match e {
Error::ScsiError { status, sense, .. } => (*status, *sense),
Error::DiscRead { status, sense, .. } => (status.unwrap_or(0), *sense),
_ => (0, None),
}
}
pub mod capture;
// Per-platform discovery helpers (the `pub(crate)` `find_drives` /
@@ -160,11 +164,6 @@ impl Drive {
self.unlock_tray();
}
// NOTE: Debug aid — remove after fd issue is resolved
pub fn device_path_owned(&self) -> String {
self.device_path.clone()
}
/// Whether this drive has a known profile (unlock parameters available).
pub fn has_profile(&self) -> bool {
self.profile.is_some()
@@ -177,31 +176,16 @@ impl Drive {
pub fn wait_ready(&mut self) -> Result<()> {
let tur = [SCSI_TEST_UNIT_READY, 0x00, 0x00, 0x00, 0x00, 0x00];
let mut tried_reset = false;
for _ in 0..60 {
let mut buf = [0u8; 0];
match self.scsi.as_mut().execute(
&tur,
crate::scsi::DataDirection::None,
&mut buf,
5_000,
) {
Ok(_) => return Ok(()),
Err(Error::ScsiError { sense_key: 5, .. }) if !tried_reset => {
// Illegal Request on TUR — drive may be stuck from a previous session.
// Try reset() which attempts multiple recovery approaches.
tried_reset = true;
if self.reset().is_ok() {
return Ok(());
}
// If reset failed but disc is present, proceed anyway —
// the scan path will handle errors individually.
if self.drive_status() == DriveStatus::DiscPresent {
return Ok(());
}
}
Err(_) => {}
if self
.scsi
.as_mut()
.execute(&tur, crate::scsi::DataDirection::None, &mut buf, 5_000)
.is_ok()
{
return Ok(());
}
std::thread::sleep(std::time::Duration::from_millis(500));
}
@@ -256,94 +240,18 @@ impl Drive {
5_000,
) {
Ok(_) => DriveStatus::DiscPresent,
Err(Error::ScsiError { sense_key: 2, .. }) => DriveStatus::NotReady,
Err(Error::ScsiError { sense_key: 6, .. }) => DriveStatus::NotReady, // UNIT ATTENTION
Err(ref e)
if e.scsi_sense()
.is_some_and(|s| s.is_not_ready() || s.is_unit_attention()) =>
{
DriveStatus::NotReady
}
_ => DriveStatus::Unknown,
}
}
}
}
/// Attempt to reset the drive to a clean state.
///
/// Escalates through increasingly aggressive recovery:
/// 1. Unlock tray + stop/start — handles normal stuck states
/// 2. Eject cycle — clears LibreDrive firmware stuck state (proven on BU40N)
/// 3. Re-init — firmware re-upload if profile available
///
/// Note: step 2 physically ejects the tray. On slimline drives the user
/// must push it back in manually. Returns Ok(()) if TUR succeeds after
/// any step, even if the drive reports "tray open" (that's a valid state).
pub fn reset(&mut self) -> Result<()> {
let mut buf = [0u8; 0];
let tur = [SCSI_TEST_UNIT_READY, 0x00, 0x00, 0x00, 0x00, 0x00];
// 1. Unlock + stop/start
self.unlock_tray();
let stop = [SCSI_START_STOP_UNIT, 0x00, 0x00, 0x00, 0x00, 0x00];
let _ =
self.scsi
.as_mut()
.execute(&stop, crate::scsi::DataDirection::None, &mut buf, 5_000);
std::thread::sleep(std::time::Duration::from_millis(500));
let start = [SCSI_START_STOP_UNIT, 0x00, 0x00, 0x00, 0x01, 0x00];
let _ =
self.scsi
.as_mut()
.execute(&start, crate::scsi::DataDirection::None, &mut buf, 5_000);
std::thread::sleep(std::time::Duration::from_millis(2000));
if self
.scsi
.as_mut()
.execute(&tur, crate::scsi::DataDirection::None, &mut buf, 5_000)
.is_ok()
{
return Ok(());
}
// 2. Eject cycle — clears MT1959 LibreDrive stuck state.
// After eject, TUR returning "Not Ready — tray open" (sense key 2)
// counts as success: the drive is functional, just needs disc reinserted.
self.unlock_tray();
let eject = [SCSI_START_STOP_UNIT, 0x00, 0x00, 0x00, 0x02, 0x00];
let _ =
self.scsi
.as_mut()
.execute(&eject, crate::scsi::DataDirection::None, &mut buf, 30_000);
std::thread::sleep(std::time::Duration::from_millis(2000));
match self
.scsi
.as_mut()
.execute(&tur, crate::scsi::DataDirection::None, &mut buf, 5_000)
{
Ok(_) => return Ok(()),
Err(Error::ScsiError { sense_key: 2, .. }) => return Ok(()), // tray open = valid
_ => {}
}
// 3. If still stuck and we have a profile, try re-init
if self.driver.is_some() {
self.init()?;
std::thread::sleep(std::time::Duration::from_millis(1000));
match self.scsi.as_mut().execute(
&tur,
crate::scsi::DataDirection::None,
&mut buf,
5_000,
) {
Ok(_) => return Ok(()),
Err(Error::ScsiError { sense_key: 2, .. }) => return Ok(()),
_ => {}
}
}
Err(Error::DeviceResetFailed {
path: self.device_path.clone(),
})
}
pub fn platform_name(&self) -> &str {
match self.platform {
Some(ref p) => p.name(),
@@ -512,22 +420,38 @@ impl Drive {
/// Read sectors from the disc. Single-shot — no inline retries, no
/// SCSI reset.
///
/// `recovery=true` bumps the per-CDB timeout to 30 s for the
/// `Disc::patch` pass; `recovery=false` uses 1.5 s for `Disc::copy`'s
/// fast skip-forward sweep. On any failure returns `Err(DiscRead)`
/// immediately. The orchestration layer (`Disc::patch`'s outer loop
/// for the patch pass, `DiscStream`'s adaptive batch halving for the
/// stream path) handles retries.
/// `recovery=true` uses [`crate::scsi::READ_RECOVERY_TIMEOUT_MS`] (60 s,
/// matches sg_dd) for the `Disc::patch` pass; `recovery=false` uses
/// [`crate::scsi::READ_TIMEOUT_MS`] (30 s, matches the kernel's
/// `/sys/block/sr*/device/timeout` default) for `Disc::copy`'s fast
/// skip-forward sweep. Both budgets are generous enough that the drive
/// can finish ECC recovery on a marginal sector — pre-0.13.21 this was
/// 1.5 s on the fast path which forced the kernel mid-layer to time
/// out and escalate while we waited anyway. On any failure returns
/// `Err(DiscRead)` immediately; orchestration (`Disc::patch` multi-pass,
/// `DiscStream` adaptive batch halving) handles retry policy.
///
/// Inline retry phases (5× gentle + reset+reopen + 5× more) were
/// removed in 0.13.6. Per
/// `(internal)/postmortems/2026-04-25-stop-wedge-and-zero-kbs.md`,
/// the stop-wedge postmortem (2026-04-25),
/// the inline reset on the LG BU40N (Initio bridge) wedged drive
/// firmware without ever recovering a sector. The remaining recovery
/// layers (Disc::patch multi-pass, DiscStream batch halving) do not
/// touch the wedge-prone reset path.
pub fn read(&mut self, lba: u32, count: u16, buf: &mut [u8], recovery: bool) -> Result<usize> {
let timeout_ms = if recovery { 30_000 } else { 1_500 };
let timeout_ms = if recovery {
crate::scsi::READ_RECOVERY_TIMEOUT_MS
} else {
crate::scsi::READ_TIMEOUT_MS
};
tracing::debug!(
target: "freemkv::drive",
lba,
count,
recovery,
timeout_ms,
"Drive::read enter"
);
let cdb = [
crate::scsi::SCSI_READ_10,
0x00,
@@ -549,7 +473,22 @@ impl Drive {
) {
Ok(result) => Ok(result.bytes_transferred),
Err(Error::Halted) => Err(Error::Halted),
Err(_) => Err(Error::DiscRead { sector: lba as u64 }),
Err(e) => {
let (status, sense) = extract_scsi_context(&e);
tracing::warn!(
target: "freemkv::drive",
lba,
count,
inner_error = %e,
scsi_status = status,
"Drive::read checked_exec failed"
);
Err(Error::DiscRead {
sector: lba as u64,
status: Some(status),
sense,
})
}
}
}
@@ -660,21 +599,21 @@ impl SectorReader for Drive {
) -> Result<usize> {
self.read(lba, count, buf, recovery)
}
fn set_speed(&mut self, kbs: u16) {
Drive::set_speed(self, kbs);
}
}
/// Find all optical drives connected to this system.
/// Returns opened Drive objects ready for use.
pub fn find_drives() -> Vec<Drive> {
/// Find the first optical drive on this system and open it.
///
/// For just listing drives without opening (e.g. UI sidebar), use
/// `scsi::list_drives()` — that returns `DriveInfo` (path + identity)
/// without the cost of running every drive's profile + identity probe.
pub fn find_drive() -> Option<Drive> {
discover_drives()
.into_iter()
.filter_map(|(path, _)| Drive::open(std::path::Path::new(&path)).ok())
.collect()
}
/// Find the first optical drive.
/// Returns an opened Drive ready for use.
pub fn find_drive() -> Option<Drive> {
find_drives().into_iter().next()
.find_map(|(path, _)| Drive::open(std::path::Path::new(&path)).ok())
}
/// Halt-aware sleep primitive — wakes within ~100 ms of `halt` flipping
+126 -9
View File
@@ -155,10 +155,24 @@ pub enum Error {
},
// SCSI (4xxx)
/// SCSI command failed.
///
/// `opcode` is the failing CDB byte 0. `status` is the raw SCSI
/// status byte: `0x02` = CHECK CONDITION (drive replied with sense
/// data), `0xFF` = libfreemkv-synthesised sentinel meaning "no SCSI
/// status delivered" (kernel timeout, USB bridge wedge, IOKit
/// service failure). `sense` carries the drive's SPC-4 sense triple
/// when the drive replied; `None` for transport-layer failures.
///
/// Recommended dispatch (callers shouldn't pattern-match raw
/// fields):
/// - [`Error::is_scsi_transport_failure`] — bail; bridge/transport wedge
/// - [`Error::is_marginal_read`] — drive said this read was marginal; smaller block may recover
/// - [`Error::scsi_sense`] — borrow the sense triple for finer routing ([`ScsiSense::is_medium_error`] etc.)
ScsiError {
opcode: u8,
status: u8,
sense_key: u8,
sense: Option<crate::scsi::ScsiSense>,
},
// I/O (5xxx)
@@ -169,6 +183,8 @@ pub enum Error {
// Disc format (6xxx)
DiscRead {
sector: u64,
status: Option<u8>,
sense: Option<crate::scsi::ScsiSense>,
},
/// Drive was halted by caller.
Halted,
@@ -363,19 +379,46 @@ impl std::fmt::Display for Error {
Error::ScsiError {
opcode,
status,
sense_key,
} => {
write!(
sense,
} => match sense {
Some(s) => write!(
f,
"E{}: 0x{:02x}/0x{:02x}/0x{:02x}",
"E{}: 0x{:02x}/0x{:02x}/0x{:02x}/0x{:02x}/0x{:02x}",
self.code(),
opcode,
status,
sense_key
)
}
s.sense_key,
s.asc,
s.ascq,
),
None => write!(f, "E{}: 0x{:02x}/0x{:02x}", self.code(), opcode, status,),
},
Error::IoError { source } => write!(f, "E{}: {}", self.code(), source),
Error::DiscRead { sector } => write!(f, "E{}: {}", self.code(), sector),
Error::DiscRead {
sector,
status,
sense,
} => match (status, sense) {
(Some(st), Some(s)) => write!(
f,
"E{}: {} 0x{:02x}/0x{:02x}/0x{:02x}",
self.code(),
sector,
st,
s.sense_key,
s.asc,
),
(Some(st), None) => write!(f, "E{}: {} 0x{:02x}", self.code(), sector, st,),
(None, Some(s)) => write!(
f,
"E{}: {} 0x{:02x}/0x{:02x}",
self.code(),
sector,
s.sense_key,
s.asc,
),
(None, None) => write!(f, "E{}: {}", self.code(), sector),
},
Error::Halted => write!(f, "E{}", self.code()),
Error::UdfNotFound { path } => write!(f, "E{}: {}", self.code(), path),
Error::DiscTitleRange { index, count } => {
@@ -438,6 +481,80 @@ impl From<Error> for std::io::Error {
/// Convenience alias for `Result<T, Error>`.
pub type Result<T> = std::result::Result<T, Error>;
impl Error {
/// Borrow the drive-returned SPC-4 sense triple if this error is a
/// [`Error::ScsiError`] carrying sense data. `None` for any other
/// variant **and** for `ScsiError`s that represent a transport-layer
/// failure (where the device never delivered a SCSI status reply, so
/// no sense data exists).
pub fn scsi_sense(&self) -> Option<&crate::scsi::ScsiSense> {
match self {
Error::ScsiError { sense: Some(s), .. } => Some(s),
Error::DiscRead { sense: Some(s), .. } => Some(s),
_ => None,
}
}
/// True if this is a [`Error::ScsiError`] representing a transport-layer
/// failure — kernel timeout, USB bridge wedge, IOKit service error.
/// The device never delivered a SCSI status reply, so there is no
/// sense data to inspect; retrying typically requires physical
/// intervention (replug).
pub fn is_scsi_transport_failure(&self) -> bool {
matches!(
self,
Error::ScsiError {
status: crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE,
..
}
) || matches!(
self,
Error::DiscRead {
status: Some(crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE),
..
}
)
}
/// True if this error indicates bridge degradation — the SCSI status
/// is neither GOOD (0x00), CHECK CONDITION (0x02), nor transport failure
/// (0xFF). Observed on the Initio INIC-1618L USB bridge preceding a full
/// crash: the bridge firmware returns non-standard status bytes (e.g.
/// 0x04, 0x05) with empty sense data. The caller should cool down
/// (10 s pause) and retry rather than hammering the bridge.
pub fn is_bridge_degradation(&self) -> bool {
let status = match self {
Error::ScsiError { status, .. } => *status,
Error::DiscRead { status, .. } => status.unwrap_or(0),
_ => return false,
};
status != crate::scsi::SCSI_STATUS_GOOD
&& status != crate::scsi::SCSI_STATUS_CHECK_CONDITION
&& status != crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE
}
/// True if the underlying SCSI failure is a *marginal read* — the
/// drive returned an error category in which smaller-granularity
/// retries can sometimes recover the data:
///
/// - MEDIUM ERROR (sense key 3) — canonical bad-sector signal
/// - ABORTED COMMAND (sense key B) — transient; retry usually works
/// - RECOVERED ERROR (sense key 1) / NO SENSE (sense key 0) — not
/// classified as fatal; treat as recoverable
///
/// Returns `false` for transport failures (no sense data delivered),
/// HARDWARE ERROR, DATA PROTECT, UNIT ATTENTION, ILLEGAL
/// REQUEST, BLANK CHECK, kernel `IoError`, and any non-SCSI variant.
/// Caller-agnostic predicate — describes a property of the *error*,
/// not what one specific call site should do with it. Used by
/// `Disc::copy`'s hysteresis dispatch.
pub fn is_marginal_read(&self) -> bool {
self.scsi_sense()
.map(crate::scsi::ScsiSense::is_marginal)
.unwrap_or(false)
}
}
#[cfg(test)]
mod tests {
//! Smoke tests for the error code → variant mapping. Each new variant
+9 -4
View File
@@ -88,12 +88,16 @@ pub mod mux;
pub mod pes;
pub(crate) mod platform;
pub mod profile;
pub mod progress;
pub mod scsi;
pub mod sector;
pub(crate) mod speed;
pub(crate) mod udf;
pub mod verify;
// Re-export verify types at the crate root for ergonomic imports.
pub use verify::{SectorRange, SectorStatus, VerifyResult, verify_title};
// ─── Drive lifecycle ────────────────────────────────────────────────────────
//
// `Drive::open(path)` → `wait_ready()` → `init()` → `Disc::scan()`. `Drive`
@@ -102,7 +106,7 @@ pub mod verify;
pub use drive::capture::{
CapturedFeature, DriveCapture, capture_drive_data, mask_bytes, mask_string,
};
pub use drive::{Drive, DriveStatus, find_drive, find_drives};
pub use drive::{Drive, DriveStatus, find_drive};
// ─── Errors ─────────────────────────────────────────────────────────────────
//
@@ -133,9 +137,10 @@ pub use decrypt::{DecryptKeys, decrypt_sectors};
// background. The codec / channel / resolution enums are the canonical
// structured representation; never compare against display strings.
pub use disc::{
AacsState, AudioChannels, AudioStream, Clip, Codec, ColorSpace, ContentFormat, Disc,
DiscFormat, DiscId, DiscTitle, Extent, FrameRate, HdrFormat, KeySource, LabelPurpose,
AacsState, AudioChannels, AudioStream, Clip, Codec, ColorSpace, ContentFormat, DamageSeverity,
Disc, DiscFormat, DiscId, DiscTitle, Extent, FrameRate, HdrFormat, KeySource, LabelPurpose,
LabelQualifier, Resolution, SampleRate, ScanOptions, Stream, SubtitleStream, VideoStream,
classify_damage,
};
// ─── Streams ────────────────────────────────────────────────────────────────
@@ -167,7 +172,7 @@ pub use mux::{InputOptions, StreamUrl, input, output, parse_url};
// out-of-tree platform backends. `SectorReader` lets callers feed any byte
// source (test harness, network image, SMB share) into the disc scan
// pipeline; `FileSectorReader` is the standard ISO-on-disk implementation.
pub use scsi::{DriveInfo, ScsiTransport, drive_has_disc, list_drives};
pub use scsi::{DriveInfo, ScsiSense, ScsiTransport, drive_has_disc, list_drives};
pub use sector::{FileSectorReader, SectorReader};
pub use speed::DriveSpeed;
pub use udf::{UdfFs, read_filesystem};
+13 -1
View File
@@ -6,6 +6,7 @@
//! Read-only. For disc→ISO (raw sector copy), use `Disc::copy()`.
use crate::disc::{Disc, DiscTitle, Extent};
use crate::drive::extract_scsi_context;
use crate::event::{BatchSizeReason, Event, EventKind};
use crate::sector::SectorReader;
use std::io;
@@ -317,7 +318,18 @@ impl DiscStream {
self.current_offset += 1;
break;
} else {
return Err(crate::error::Error::DiscRead { sector: lba as u64 }.into());
let err = self
.reader
.read_sectors(lba, sectors, &mut self.read_buf[..2048], false)
.err();
let (status, sense) =
err.as_ref().map(extract_scsi_context).unwrap_or((0, None));
return Err(crate::error::Error::DiscRead {
sector: lba as u64,
status: Some(status),
sense,
}
.into());
}
}
+6 -6
View File
@@ -96,8 +96,8 @@ impl Mt1959 {
if result.bytes_transferred != expected {
return Err(Error::ScsiError {
opcode: SCSI_READ_BUFFER,
status: 0xFF,
sense_key: 0,
status: crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
});
}
Ok(result.bytes_transferred)
@@ -169,8 +169,8 @@ impl Mt1959 {
}
Err(Error::ScsiError {
opcode: SCSI_READ_BUFFER,
status: 0xFF,
sense_key: 0,
status: crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
})
}
@@ -275,8 +275,8 @@ impl Mt1959 {
{
return Err(Error::ScsiError {
opcode: SCSI_READ_BUFFER,
status: 0xFF,
sense_key: 0,
status: crate::scsi::SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
});
}
addr = addr.wrapping_add(PROBE_STEP);
+125
View File
@@ -0,0 +1,125 @@
//! Pipeline-progress reporting for the rip pipeline.
//!
//! v0.13.16 architecture rule: ONE progress signal type. Every long-running
//! pipeline operation (`Disc::copy`, `Disc::patch`, `verify_title`) emits the same
//! `PassProgress` shape via the `Progress` trait. Consumers (autorip) compute
//! a single `PipelineStats` derived view and never reach into per-pass
//! internals.
//!
//! Why this matters: pre-0.13.16 the API leaked `pos`, `bytes_good`,
//! `work_done`, `bytes_pending`, `Finished/NonTrimmed` mapfile semantics —
//! and consumers reinvented the math each time they wanted a percentage.
//! UIs ended up reading one source while server-side computed from another,
//! producing wrong percentages without anyone noticing.
/// Identifies which pipeline phase the progress event belongs to.
///
/// Consumers can render a phase-specific label (e.g. "Sweep", "Trim
/// (reverse)", "Scrape", "Mux") or just use a generic "Pass N" label.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum PassKind {
/// `Disc::copy` — initial sweep across the entire disc.
Sweep,
/// `Disc::patch` retry pass with `block_sectors >= 2`. `reverse=true`
/// means walking bad ranges from highest to lowest LBA.
Trim { reverse: bool },
/// `Disc::patch` final pass at 1 sector per block.
Scrape { reverse: bool },
/// Demux ISO → output (MKV / M2TS / network). Single phase that runs
/// after all rip passes complete.
Mux,
/// Sector verification — reads every sector and classifies health.
Verify,
}
/// One progress sample from a pipeline phase.
///
/// `work_done / work_total` is the per-pass percentage — always 0..=100%
/// regardless of which kind of pass is running. `bytes_good_total` is the
/// cumulative count of confirmed-clean bytes across the whole rip; useful
/// for the "data recovered" stat the user sees.
///
/// For `PassKind::Verify`, the fields map as follows:
/// - `work_done` = sectors read so far
/// - `work_total` = total sectors in title
/// - `bytes_good_total` = good + slow + recovered sectors × 2048
/// - `bytes_unreadable_total` = bad sectors × 2048
/// - `bytes_pending_total` = 0 (verify processes sequentially, nothing pending)
#[derive(Debug, Clone, Copy)]
pub struct PassProgress {
pub kind: PassKind,
pub work_done: u64,
pub work_total: u64,
pub bytes_good_total: u64,
pub bytes_unreadable_total: u64,
pub bytes_pending_total: u64,
pub bytes_total_disc: u64,
pub disc_duration_secs: Option<f64>,
/// How many bytes of the worst-case damage (unreadable + pending) fall
/// within the main title's extents. Zero means none of the damage
/// affects the main movie — it's all in extras/menus.
pub bytes_bad_in_main_title: u64,
/// Main title duration in seconds. Same as disc_duration_secs when the
/// disc has one dominant title, but separate so consumers can show both.
pub main_title_duration_secs: Option<f64>,
/// Main title size in bytes (sum of extent sizes).
pub main_title_size_bytes: Option<u64>,
}
impl PassProgress {
/// Percentage of work completed for this pass (0..=100).
///
/// Returns `100.0` if `work_total` is zero to avoid division by zero.
pub fn work_pct(&self) -> f64 {
if self.work_total == 0 {
return 100.0;
}
self.work_done as f64 / self.work_total as f64 * 100.0
}
/// Percentage of the disc that is confirmed clean (0..=100).
///
/// Computed from `bytes_good_total / bytes_total_disc`.
pub fn good_pct(&self) -> f64 {
if self.bytes_total_disc == 0 {
return 100.0;
}
self.bytes_good_total as f64 / self.bytes_total_disc as f64 * 100.0
}
/// Percentage of the disc that is unreadable (0..=100).
pub fn bad_pct(&self) -> f64 {
if self.bytes_total_disc == 0 {
return 0.0;
}
self.bytes_unreadable_total as f64 / self.bytes_total_disc as f64 * 100.0
}
/// Percentage of the disc that is still pending (not yet attempted or needs retry).
pub fn pending_pct(&self) -> f64 {
if self.bytes_total_disc == 0 {
return 0.0;
}
self.bytes_pending_total as f64 / self.bytes_total_disc as f64 * 100.0
}
}
/// A consumer of pipeline progress events. Library code calls
/// `Progress::report` once per inner-loop iteration (throttling is the
/// consumer's job — `report` is cheap; the library doesn't gate it).
///
/// Returns `true` to continue, `false` to request early stop.
///
/// No `Send`/`Sync` bound — `report` is always called from the same thread
/// running the pipeline, so closures with non-`Sync` captures (e.g.
/// `RefCell<PassProgressState>`) work directly. Blanket impl below lets
/// callers pass closures without explicit struct types.
pub trait Progress {
fn report(&self, p: &PassProgress) -> bool;
}
impl<F: Fn(&PassProgress) -> bool> Progress for F {
fn report(&self, p: &PassProgress) -> bool {
(self)(p)
}
}
+242 -244
View File
@@ -1,11 +1,23 @@
//! Linux SCSI transport via async sg write/poll/read.
//! Linux SCSI transport via synchronous blocking SG_IO ioctl.
//!
//! Uses the sg driver's asynchronous interface instead of the blocking
//! SG_IO ioctl. Commands are submitted via write(), waited on via
//! poll() with a hard timeout, and completed via read(). If poll()
//! times out, the fd is abandoned (closed in a background thread) and
//! a fresh fd is opened. This gives us true user-controlled timeouts
//! that the kernel's USB error recovery cannot override.
//! `execute()` is one syscall: `ioctl(fd, SG_IO, &hdr)` blocks until the
//! kernel completes the command (success, error, or its own timeout).
//! No userspace abort, no fd close+reopen, no SG_SCSI_RESET escalation —
//! the kernel SCSI mid-layer's `scsi_eh.rst` ladder
//! (ABORT TASK → LUN RESET → BUS RESET → HOST RESET) runs internally
//! when `hdr.timeout` expires, and by the time the ioctl returns the
//! kernel has already done what it can.
//!
//! This matches what every reference project does: MakeMKV (8 s sync
//! ioctl), sg_dd (60 s sync ioctl), the kernel default for SCSI block
//! devices (30 s `/sys/.../timeout`). See
//! the SCSI architecture audit (2026-04-26) for the full primary-source
//! references.
//!
//! Pre-0.13.20 we ran an async `write() + poll(1.5s) + close-on-timeout +
//! bg reopen` pattern. That abandoned slow-but-alive commands faster than
//! the drive could drain its internal queue, deepening the wedge
//! pattern on the LG BU40N. Reverted in 0.13.20.
use super::{DataDirection, ScsiResult, ScsiTransport};
use crate::error::{Error, Result};
@@ -45,23 +57,22 @@ struct sg_io_hdr {
}
// Compile-time validation: sg_io_hdr must match the kernel's layout.
// 64 bytes on 64-bit, 44 bytes on 32-bit (pointer-size dependent).
// 88 bytes on 64-bit, 64 bytes on 32-bit (pointer-size dependent).
#[cfg(target_pointer_width = "64")]
const _: () = assert!(std::mem::size_of::<sg_io_hdr>() == 88);
#[cfg(target_pointer_width = "32")]
const _: () = assert!(std::mem::size_of::<sg_io_hdr>() == 64);
pub struct SgIoTransport {
fd: i32,
pub fd: i32,
device_path: std::path::PathBuf,
pub fd_recovery: std::sync::Arc<std::sync::atomic::AtomicI32>,
}
impl SgIoTransport {
/// Open a SCSI device for use. Resets the drive first to ensure
/// a known good state, then opens a fresh fd for commands.
/// Open a SCSI device for use.
pub fn open(device: &Path) -> Result<Self> {
let device = Self::resolve_to_sg(device);
Self::reset(&device)?;
let c_path = Self::to_c_path(&device);
let fd = unsafe {
libc::open(
@@ -75,6 +86,7 @@ impl SgIoTransport {
Ok(SgIoTransport {
fd,
device_path: device,
fd_recovery: std::sync::Arc::new(std::sync::atomic::AtomicI32::new(-1)),
})
}
@@ -94,56 +106,12 @@ impl SgIoTransport {
/// STOP+START UNIT. Both escalations were tried in 0.13.0–0.13.5
/// against the LG BU40N (Initio USB-SATA bridge); both failed to
/// recover wedged drives and made the wedge worse — see
/// (internal)/postmortems/2026-04-25-bu40n-wedge-recovery.md.
/// If the drive is genuinely unresponsive, the next workload command
/// fails naturally and the caller surfaces a "physical reconnect
/// required" prompt. Software has no path back from a wedged Initio
/// bridge — only physical replug clears it.
pub fn reset(device: &Path) -> Result<()> {
let c_path = Self::to_c_path(device);
// open + close — make the kernel cancel any SG_IO commands queued
// against a previous fd that didn't close cleanly.
let probe_fd = unsafe {
libc::open(
c_path.as_ptr() as *const libc::c_char,
libc::O_RDWR | libc::O_NONBLOCK | libc::O_CLOEXEC,
)
};
if probe_fd >= 0 {
unsafe { libc::close(probe_fd) };
}
// Let the kernel finish that cancellation before we reopen.
std::thread::sleep(std::time::Duration::from_secs(2));
// Fresh fd just to send the unlock command, then close.
let fd = unsafe {
libc::open(
c_path.as_ptr() as *const libc::c_char,
libc::O_RDWR | libc::O_NONBLOCK | libc::O_CLOEXEC,
)
};
if fd < 0 {
return Self::open_error(device);
}
// ALLOW MEDIUM REMOVAL — clear any tray lock left by a killed
// process whose Drop never ran. Best-effort; ignore result.
let _ = Self::raw_command(fd, &[0x1E, 0, 0, 0, 0, 0], 3_000);
unsafe { libc::close(fd) };
Ok(())
}
/// the BU40N wedge recovery postmortem (2026-04-25).
fn open_error<T>(device: &Path) -> Result<T> {
let err = std::io::Error::last_os_error();
Err(if err.kind() == std::io::ErrorKind::PermissionDenied {
Error::DevicePermission {
path: format!(
"{}: permission denied (try running as root)",
device.display()
),
path: device.display().to_string(),
}
} else {
Error::DeviceNotFound {
@@ -153,8 +121,7 @@ impl SgIoTransport {
}
/// Send a raw SCSI command on an fd. Used by reset() before the
/// transport is constructed. Uses synchronous SG_IO — fine for
/// short commands (TUR, PREVENT MEDIUM REMOVAL, START/STOP).
/// transport is constructed.
fn raw_command(fd: i32, cdb: &[u8], timeout_ms: u32) -> std::result::Result<(), ()> {
let mut sense = [0u8; 32];
let mut hdr: sg_io_hdr = unsafe { std::mem::zeroed() };
@@ -170,7 +137,7 @@ impl SgIoTransport {
hdr.flags = SG_FLAG_Q_AT_HEAD;
let ret = unsafe { libc::ioctl(fd, SG_IO as _, &mut hdr as *mut sg_io_hdr) };
if ret < 0 || hdr.status != 0 {
if ret < 0 || hdr.status != 0 || hdr.host_status != 0 || hdr.driver_status != 0 {
Err(())
} else {
Ok(())
@@ -215,7 +182,7 @@ impl SgIoTransport {
impl Drop for SgIoTransport {
fn drop(&mut self) {
if self.fd >= 0 {
// Unlock tray before closing — don't leave it locked
// Unlock tray before closing — don't leave it locked.
let _ = Self::raw_command(self.fd, &[0x1E, 0, 0, 0, 0, 0], 3_000);
unsafe { libc::close(self.fd) };
}
@@ -223,21 +190,33 @@ impl Drop for SgIoTransport {
}
impl ScsiTransport for SgIoTransport {
/// Execute a SCSI command with an enforceable timeout.
/// Execute a SCSI command via synchronous blocking SG_IO.
///
/// Uses the sg driver's async write/poll/read interface:
/// 1. write() submits the command — returns immediately
/// 2. poll() waits for completion — respects our timeout exactly
/// 3. read() retrieves the result — copies data to caller's buffer
/// One syscall: `ioctl(fd, SG_IO, &hdr)`. The kernel honors
/// `hdr.timeout` and runs its own ABORT TASK → LUN RESET → BUS
/// RESET → HOST RESET escalation if the device times out (per
/// `Documentation/scsi/scsi_eh.rst`). By the time this returns,
/// the kernel has done its recovery work.
///
/// If poll() times out, the pending command is abandoned: the old fd
/// is closed in a background thread (may block while kernel finishes
/// the USB transfer) and a fresh fd is opened. The caller sees a
/// normal SCSI error and can retry.
/// Errors we surface to caller (any of these = command failed):
///
/// Without SG_FLAG_DIRECT_IO, the kernel uses internal buffers for
/// DMA and copies to userspace during read(). On timeout (no read),
/// the caller's buffer is untouched — safe to return immediately.
/// - ioctl returned -1 → `Error::IoError` (kernel-level failure)
/// - `hdr.host_status` != 0 OR `(hdr.driver_status & ~DRIVER_SENSE)` != 0
/// → `Error::ScsiError { status: 0xFF, sense_key: 0, asc: 0, ascq: 0 }`
/// (real transport-layer failure: kernel timeout, bridge wedge, bus error)
/// - `hdr.status` != 0 (typically `0x02` CHECK CONDITION) →
/// `Error::ScsiError { status, sense_key, asc, ascq }` carrying the
/// drive's full SPC-4 sense triple. Callers route on
/// `is_medium_error()`, `is_unit_attention()`, etc.
///
/// Note: SG's `DRIVER_SENSE` (0x08) bit indicates *sense data is
/// attached* — it's set on every CHECK CONDITION reply. It is **not**
/// a transport failure; pre-0.13.23 we conflated it with one and
/// silently lost every drive-reported error reason. The mask in the
/// transport-error check below is the fix.
///
/// Caller's `data` buffer is mutated only on success; partial
/// transfers are reported via `bytes_transferred = data.len() - resid`.
fn execute(
&mut self,
cdb: &[u8],
@@ -245,30 +224,50 @@ impl ScsiTransport for SgIoTransport {
data: &mut [u8],
timeout_ms: u32,
) -> Result<ScsiResult> {
if self.fd < 0 {
let exec_t0 = std::time::Instant::now();
let opcode = cdb[0];
tracing::trace!(
target: "freemkv::scsi",
phase = "enter",
opcode = opcode,
timeout_ms,
data_len = data.len(),
fd = self.fd,
"SgIoTransport::execute"
);
// Check if a background recovery has produced a new fd.
let recovered = self
.fd_recovery
.swap(-1, std::sync::atomic::Ordering::Acquire);
if recovered >= 0 {
// Close the old fd if it's still valid.
if self.fd >= 0 {
unsafe { libc::close(self.fd) };
}
self.fd = recovered;
} else if self.fd < 0 {
return Err(Error::DeviceNotFound {
path: self.device_path.display().to_string(),
});
}
let mut sense = [0u8; 32];
if data.len() > u32::MAX as usize {
return Err(Error::ScsiError {
opcode: cdb[0],
status: super::SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
});
}
let dxfer_direction = match direction {
DataDirection::None => SG_DXFER_NONE,
DataDirection::FromDevice => SG_DXFER_FROM_DEV,
DataDirection::ToDevice => SG_DXFER_TO_DEV,
};
if data.len() > u32::MAX as usize {
return Err(Error::ScsiError {
opcode: cdb[0],
status: 0xFF,
sense_key: 0,
});
}
let cmd_len = cdb.len().min(16) as u8;
let mut sense = [0u8; 32];
let mut hdr: sg_io_hdr = unsafe { std::mem::zeroed() };
hdr.interface_id = b'S' as i32;
hdr.dxfer_direction = dxfer_direction;
@@ -281,121 +280,118 @@ impl ScsiTransport for SgIoTransport {
hdr.timeout = timeout_ms;
hdr.flags = SG_FLAG_Q_AT_HEAD;
// Submit command asynchronously via write()
let hdr_size = std::mem::size_of::<sg_io_hdr>();
let wr = unsafe {
libc::write(
self.fd,
&hdr as *const sg_io_hdr as *const libc::c_void,
hdr_size,
)
};
if wr < 0 {
return Err(Error::IoError {
source: std::io::Error::last_os_error(),
});
// The single blocking syscall. Returns when the device responds,
// when the kernel's timeout fires, or when the kernel's error
// recovery completes its escalation. On a healthy read this is
// <100 ms; on a slow-recovery bad sector it can be tens of
// seconds; on a hung drive it returns at `timeout_ms` with
// `host_status` flagged.
let ret = unsafe { libc::ioctl(self.fd, SG_IO as _, &mut hdr as *mut sg_io_hdr) };
let exec_elapsed_ms = exec_t0.elapsed().as_millis() as u64;
if ret < 0 {
let errno = std::io::Error::last_os_error();
tracing::trace!(
target: "freemkv::scsi",
phase = "ioctl_err",
opcode = opcode,
errno = errno.raw_os_error().unwrap_or(0),
exec_elapsed_ms,
"ioctl(SG_IO) returned <0"
);
return Err(Error::IoError { source: errno });
}
// Wait for completion with enforceable timeout.
// Retry on EINTR (signal interrupted poll) with remaining time.
let deadline =
std::time::Instant::now() + std::time::Duration::from_millis(timeout_ms as u64);
let pr = loop {
let remaining = deadline
.saturating_duration_since(std::time::Instant::now())
.as_millis() as i32;
if remaining <= 0 {
break 0; // expired
}
let mut pfd = libc::pollfd {
fd: self.fd,
events: libc::POLLIN,
revents: 0,
};
let ret = unsafe { libc::poll(&mut pfd, 1, remaining) };
if ret >= 0 || std::io::Error::last_os_error().kind() != std::io::ErrorKind::Interrupted
{
break ret;
}
};
// Transport-level failure (kernel timeout, USB bridge wedge,
// bus error). `hdr.status` may still be zero — the SCSI device
// never got to send a status byte. Surface as 0xFF so callers
// (e.g. `drive_has_disc`) can detect the wedge signature.
//
// 0.13.23: mask out `DRIVER_SENSE` (0x08) before treating
// `driver_status` as a transport failure. That bit is set on
// *every* CHECK CONDITION reply just to flag "sense data is
// attached in `sbp`" — it's not an error of its own. Pre-fix
// we collapsed every drive-reported error into a synthetic
// 0xFF wedge signature and discarded the sense data, which
// killed the rip's classification logic on damaged discs.
let driver_status_real = hdr.driver_status & !super::DRIVER_SENSE;
if hdr.host_status != 0 || driver_status_real != 0 {
tracing::trace!(
target: "freemkv::scsi",
phase = "transport_err",
opcode = opcode,
host_status = hdr.host_status,
driver_status = hdr.driver_status,
status = hdr.status,
exec_elapsed_ms,
"transport-level failure (timeout / bridge wedge)"
);
if pr <= 0 {
// Timeout (0) or fatal poll error (-1).
// Command is still pending in the kernel. Abandon the fd by
// spawning a background close (which will block until the
// kernel completes/aborts the pending command), and mark
// this transport invalid by setting `self.fd = -1`.
//
// Why we no longer reopen on the main thread: opening the
// SAME /dev/sg* device while the prior fd is mid-close
// serializes via the kernel's per-device state lock, so
// `libc::open()` on the main thread blocks for the same
// duration that close() does — defeating the userspace
// timeout. Observed in v0.13.8 live test on Dune 2: each
// timed-out read added 60+ s to the next iteration of
// Disc::copy, leaving the rip stuck without surfacing an
// error or wedging the drive.
//
// Net effect of the fix: a single read timeout invalidates
// the SgIoTransport. The Drive is now "dead" until the
// consumer (autorip's rip thread) catches the failure and
// reopens. Disc::copy's `skip_on_error=true` path will see
// the Err and skip-forward, advancing pos, and the next
// read on this fd returns Err(DeviceNotFound) immediately —
// which Disc::copy continues to skip-forward through until
// the NonTried region is exhausted. Pass 1 then ends with
// bytes_pending > 0 and the rip thread reopens the Drive
// for Pass 2 (Disc::patch with recovery=true and 30 s
// timeouts).
// Spawn recovery: close old fd, open new one in background.
// This prevents the main thread from blocking on close() while
// the kernel finishes the previous ioctl.
let old_fd = self.fd;
self.fd = -1;
let path = self.device_path.clone();
let recovery = self.fd_recovery.clone();
std::thread::spawn(move || {
unsafe { libc::close(old_fd) };
if old_fd >= 0 {
unsafe { libc::close(old_fd) };
}
});
std::thread::spawn(move || {
let c_path = std::ffi::CString::new(path.as_os_str().as_encoded_bytes()).unwrap();
let new_fd = unsafe {
libc::open(
c_path.as_ptr() as *const libc::c_char,
libc::O_RDWR | libc::O_NONBLOCK | libc::O_CLOEXEC,
)
};
recovery.store(new_fd, std::sync::atomic::Ordering::Release);
});
return Err(Error::ScsiError {
opcode: cdb[0],
status: 0xFF,
sense_key: 0,
status: super::SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
});
}
// Read response — copies data from kernel buffer to caller's buffer
let rd = unsafe {
libc::read(
self.fd,
&mut hdr as *mut sg_io_hdr as *mut libc::c_void,
hdr_size,
)
};
if rd < 0 {
return Err(Error::IoError {
source: std::io::Error::last_os_error(),
// SCSI-level failure: device responded, returned non-zero status
// (typically 0x02 CHECK CONDITION). Parse the full SPC-4 sense
// triple so callers can route on `ScsiSense::is_medium_error()`
// etc.
if hdr.status != 0 {
let parsed = super::parse_sense(&sense, hdr.sb_len_wr);
tracing::trace!(
target: "freemkv::scsi",
phase = "scsi_err",
opcode = opcode,
status = hdr.status,
sense_key = parsed.sense_key,
asc = parsed.asc,
ascq = parsed.ascq,
exec_elapsed_ms,
"SCSI status non-zero"
);
return Err(Error::ScsiError {
opcode: cdb[0],
status: hdr.status,
sense: Some(parsed),
});
}
let bytes_transferred = (data.len() as i32).saturating_sub(hdr.resid).max(0) as usize;
if hdr.status != 0 {
let sense_key = if hdr.sb_len_wr >= 3 {
let response_code = sense[0] & 0x7F;
if response_code == 0x72 || response_code == 0x73 {
// Descriptor format sense: sense key at byte 1
sense[1] & 0x0F
} else {
// Fixed format sense (0x70/0x71): sense key at byte 2
sense[2] & 0x0F
}
} else {
0
};
return Err(Error::ScsiError {
opcode: cdb[0],
status: hdr.status,
sense_key,
});
}
tracing::trace!(
target: "freemkv::scsi",
phase = "ok",
opcode = opcode,
bytes_transferred,
exec_elapsed_ms,
"execute() success"
);
Ok(ScsiResult {
status: hdr.status,
bytes_transferred,
@@ -411,31 +407,20 @@ impl ScsiTransport for SgIoTransport {
// `/dev/sg0..15` probe when sysfs is unreadable (minimal containers).
//
// `drive_has_disc` issues a single TEST UNIT READY. On the wedge signature
// (kernel returns status `0xff` with no sense) it escalates: SCSI bus reset
// → if still wedged → USB device reset (`USBDEVFS_RESET`) → retry TUR.
// Callers never see the escalation; if it fails too, surface
// `DeviceResetFailed` so the caller can back off.
// (kernel returns status `0xff` with no sense — synthesised by `execute()`
// from a non-zero `host_status`) the error bubbles directly to the caller;
// no in-library reset escalation. See the rationale block on
// `drive_has_disc` below.
/// SCSI peripheral type 5 = "CD-ROM device" (covers DVD, BD-ROM, BD-RE, etc.).
/// Stored in `/sys/class/scsi_generic/sgN/device/type` as ASCII decimal.
const SCSI_TYPE_OPTICAL: &str = "5";
/// SCSI sense key 2 = "NOT READY". Sub-codes distinguish "medium not present"
/// (no disc) from other not-ready states (loading, etc.); for poll-loop
/// purposes any sense-key 2 means "no disc to act on".
const SENSE_KEY_NOT_READY: u8 = 2;
/// Maximum sg index probed in the fallback path when sysfs is unavailable.
/// Linux assigns `/dev/sgN` sequentially per host adapter; 16 covers any
/// realistic homelab (typical PERC + USB optical = ≤8 nodes).
const SG_FALLBACK_MAX: u8 = 16;
// SCSI INQUIRY field-offset constants previously lived here. They were
// used by an in-process SCSI INQUIRY parse path that 0.13.6 retired in
// favour of reading the kernel-cached sysfs identity (vendor/model/rev
// under /sys/class/scsi_generic/sgN/device/). Removed to keep clippy
// -D warnings clean.
pub(super) fn list_drives() -> Vec<super::DriveInfo> {
let mut out = Vec::new();
let names = enumerate_sg_names();
@@ -455,8 +440,7 @@ pub(super) fn list_drives() -> Vec<super::DriveInfo> {
// INQUIRY-only probe — open transport, run INQUIRY, drop. No
// identify, no init, no firmware reset preamble's secondary
// commands beyond what `SgIoTransport::open` already does (one
// SCSI bus reset on the kernel SG fd, ~2 s).
// commands beyond what `SgIoTransport::open` already does.
let info = match SgIoTransport::open(std::path::Path::new(&path)) {
Ok(mut transport) => match super::inquiry(&mut transport) {
Ok(r) => super::DriveInfo {
@@ -522,8 +506,8 @@ fn enumerate_sg_names() -> Vec<String> {
let type_path = format!("/sys/class/scsi_generic/{name}/device/type");
match std::fs::read_to_string(&type_path) {
Ok(s) if s.trim() == SCSI_TYPE_OPTICAL => names.push(name),
Ok(_) => {} // not optical
Err(_) => names.push(name), // sysfs unreadable — let INQUIRY decide
Ok(_) => {} // not optical
Err(_) => {}
}
}
} else {
@@ -540,50 +524,64 @@ fn enumerate_sg_names() -> Vec<String> {
names
}
/// `drive_has_disc` = single TEST UNIT READY. Any error (including our
/// synthesised wedge signature, `ScsiError { status: 0xFF }`, when
/// `execute()` times out) bubbles straight up to the caller.
///
/// ## No in-library wedge recovery — and why
///
/// Versions 0.13.1 – 0.13.3 layered `scsi::reset()` + `scsi::usb_reset()`
/// (`USBDEVFS_RESET`) escalation inside `drive_has_disc`. Production
/// testing on the LG BU40N USB BD-RE showed all three userspace recovery
/// ladders succeed at the USB transport level (the kernel logs
/// `usb 3-2: reset high-speed USB device`, the device re-authorises
/// and re-attaches on a fresh `scsi_host`) **but the drive firmware
/// below the USB bridge stays locked** — no LUN ever enumerates, TUR
/// never succeeds, /dev/sg* never reappears. Physical power-cycle
/// (unplug-replug or host reboot) is the only recovery.
///
/// Methods tried and discarded:
/// - `SG_SCSI_RESET` (device-level SCSI bus reset)
/// - `STOP UNIT` / `START UNIT` CDB pair
/// - `USBDEVFS_RESET` ioctl on `/dev/bus/usb/BBB/DDD`
/// - `/sys/bus/usb/devices/<port>/authorized` 0→1 toggle
/// - `/sys/bus/usb/drivers/usb-storage/{unbind,bind}` driver rebind
/// - Forced `echo "- - -" > /sys/class/scsi_host/hostN/scan`
///
/// Rolled back in 0.13.4. Callers (autorip, CLI) surface the error
/// directly and prompt the user to physically reconnect the drive.
/// If a future hardware class is found where USB-layer recovery
/// actually works, the escalation belongs here, gated on the wedge
/// signature — see git tag `v0.13.3` for the full implementation.
/// Send TEST UNIT READY directly — no transport, no reset, no side effects.
pub(super) fn drive_has_disc(path: &Path) -> Result<bool> {
let mut transport = SgIoTransport::open(path)?;
let device = SgIoTransport::resolve_to_sg(path);
let c_path = SgIoTransport::to_c_path(&device);
let fd = unsafe {
libc::open(
c_path.as_ptr() as *const libc::c_char,
libc::O_RDWR | libc::O_NONBLOCK | libc::O_CLOEXEC,
)
};
if fd < 0 {
return SgIoTransport::open_error(&device);
}
let cdb = [crate::scsi::SCSI_TEST_UNIT_READY, 0, 0, 0, 0, 0];
let mut buf = [0u8; 0];
match transport.execute(
&cdb,
crate::scsi::DataDirection::None,
&mut buf,
crate::scsi::TUR_TIMEOUT_MS,
) {
Ok(_) => Ok(true),
let mut sense = [0u8; 32];
let mut hdr: sg_io_hdr = unsafe { std::mem::zeroed() };
hdr.interface_id = b'S' as i32;
hdr.dxfer_direction = SG_DXFER_NONE;
hdr.cmd_len = cdb.len() as u8;
hdr.mx_sb_len = sense.len() as u8;
hdr.dxfer_len = 0;
hdr.dxferp = std::ptr::null_mut();
hdr.cmdp = cdb.as_ptr();
hdr.sbp = sense.as_mut_ptr();
hdr.timeout = crate::scsi::TUR_TIMEOUT_MS;
hdr.flags = SG_FLAG_Q_AT_HEAD;
let ret = unsafe { libc::ioctl(fd, SG_IO as _, &mut hdr as *mut sg_io_hdr) };
unsafe { libc::close(fd) };
if ret < 0 {
return Err(Error::IoError {
source: std::io::Error::last_os_error(),
});
}
let driver_status_real = hdr.driver_status & !super::DRIVER_SENSE;
if hdr.host_status != 0 || driver_status_real != 0 {
return Err(Error::ScsiError {
opcode: cdb[0],
status: super::SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
});
}
if hdr.status == 0 {
return Ok(true);
}
let parsed = super::parse_sense(&sense, hdr.sb_len_wr);
if parsed.is_not_ready() {
Ok(false)
} else {
Err(Error::ScsiError {
sense_key: SENSE_KEY_NOT_READY,
..
}) => Ok(false),
Err(e) => Err(e),
opcode: cdb[0],
status: hdr.status,
sense: Some(parsed),
})
}
}
+128 -488
View File
@@ -1,183 +1,55 @@
//! macOS SCSI transport via IOKit SCSITaskDeviceInterface.
//! macOS SCSI transport: IOKit SCSITaskDeviceInterface with exclusive access.
//!
//! Sends SCSI commands to optical drives through IOKit's SCSI Architecture
//! Model family. Accepts BSD device paths like `/dev/disk2` or `/dev/rdisk2`.
//! Single dispatch path: **all** CDBs (INQUIRY, READ, REPORT KEY, etc.) go
//! through `SCSITaskDeviceInterface::ExecuteTaskSync` — 1:1 with the Linux
//! SG_IO backend. The C shim (`macos_shim.c`) handles:
//!
//! Requires exclusive access to the device — unmount the disc first:
//! `diskutil unmountDisk /dev/disk2`
//! 1. `diskutil unmountDisk force` on the target device only
//! 2. Find `IOBDServices` matching the requested BSD name (walks IOKit
//! registry: IOBDServices → IOBDBlockStorageDriver → IOMedia → BSD Name)
//! 3. Create `MMCDeviceInterface` → `SCSITaskDeviceInterface`
//! 4. `ObtainExclusiveAccess`
//! 5. Raw CDB dispatch via `CreateSCSITask` + `ExecuteTaskSync`
//!
//! Drive enumeration (`list_drives`) uses the IOKit registry directly via
//! `shim_list_drives` — no exclusive access, no SCSI commands, no unmounts.
use super::{DataDirection, ScsiResult, ScsiTransport};
use crate::error::{Error, Result};
use std::path::Path;
// ── IOKit / CoreFoundation type aliases ─────────────────────────────────────
type CFMutableDictionaryRef = *mut std::ffi::c_void;
type IOObject = u32;
type IOReturn = i32;
type MachPort = u32;
/// Opaque COM interface pointer — `*mut *mut VTable` (double-indirect).
/// IOKit plugins use COM-style vtables: the pointer points to a pointer
/// to the function table.
type ComRef = *mut *mut std::ffi::c_void;
const K_IO_RETURN_SUCCESS: IOReturn = 0;
// SCSI data transfer directions (SCSITaskLib.h)
const K_SCSI_DATA_TRANSFER_NO_DATA: u8 = 0;
const K_SCSI_DATA_TRANSFER_FROM_TARGET: u8 = 1;
const K_SCSI_DATA_TRANSFER_TO_TARGET: u8 = 2;
// SCSI task status values
const K_SCSI_TASK_STATUS_GOOD: u8 = 0x00;
const K_MAX_CDB_SIZE: usize = 16;
const K_SENSE_DATA_SIZE: usize = 32;
// ── IOKit plugin UUIDs ──────────────────────────────────────────────────────
// From IOKit/scsi/SCSITaskLib.h
/// kIOMMCDeviceUserClientTypeID — plugin type for MMC (optical) devices.
const K_IO_MMC_DEVICE_USER_CLIENT_TYPE_ID: [u8; 16] = [
0x97, 0xAB, 0xCF, 0x5C, 0x45, 0x71, 0x11, 0xD6, 0xB6, 0xA0, 0x00, 0x30, 0x65, 0xA4, 0x7A, 0xEE,
];
/// kIOCFPlugInInterfaceID — base IOCFPlugin interface.
const K_IO_CFPLUGIN_INTERFACE_ID: [u8; 16] = [
0xC2, 0x44, 0xE8, 0x58, 0x10, 0x9C, 0x11, 0xD4, 0x91, 0xD4, 0x00, 0x50, 0xE4, 0xC6, 0x42, 0x6F,
];
/// kIOSCSITaskDeviceInterfaceID — the interface we QueryInterface for.
const K_IO_SCSI_TASK_DEVICE_INTERFACE_ID: [u8; 16] = [
0x61, 0x3E, 0x48, 0xB0, 0x30, 0x01, 0x11, 0xD6, 0xA4, 0xC0, 0x00, 0x0A, 0x27, 0x05, 0x28, 0x61,
];
// ── Scatter/gather element ──────────────────────────────────────────────────
#[repr(C)]
struct SCSITaskSGElement {
address: u64,
length: u64,
#[derive(Copy, Clone)]
struct ShimDriveInfo {
bsd_name: [u8; 32],
vendor: [u8; 32],
model: [u8; 48],
firmware: [u8; 16],
}
// ── External IOKit / CoreFoundation functions ───────────────────────────────
// Rust 2024: FFI blocks declaring extern fns must be `unsafe extern`.
unsafe extern "C" {
fn IOMasterPort(bootstrap: u32, master: *mut MachPort) -> IOReturn;
fn IOBSDNameMatching(
master: MachPort,
options: u32,
bsd_name: *const u8,
) -> CFMutableDictionaryRef;
fn IOServiceGetMatchingService(master: MachPort, matching: CFMutableDictionaryRef) -> IOObject;
fn IOObjectRelease(object: IOObject) -> IOReturn;
fn IORegistryEntryGetParentEntry(
entry: IOObject,
plane: *const u8,
parent: *mut IOObject,
) -> IOReturn;
fn IOObjectConformsTo(object: IOObject, class_name: *const u8) -> u8;
fn IOCreatePlugInInterfaceForService(
service: IOObject,
plugin_type: *const [u8; 16],
interface_type: *const [u8; 16],
the_interface: *mut ComRef,
the_score: *mut i32,
) -> IOReturn;
fn shim_open_exclusive(bsd_name: *const u8) -> i32;
fn shim_close();
fn shim_execute(
cdb: *const u8,
cdb_len: u8,
buf: *mut u8,
buf_len: u32,
data_in: i32,
sense_out: *mut u8,
sense_len: u32,
task_status_out: *mut u8,
transfer_count: *mut u64,
) -> i32;
fn shim_list_drives(out: *mut ShimDriveInfo, max_entries: i32) -> i32;
}
// ── COM vtable helpers ──────────────────────────────────────────────────────
//
// IOKit plugin interfaces use COM-style vtables. A ComRef is **vtable —
// dereferencing once gives the vtable pointer, then index into it for
// individual function pointers.
//
// All vtable indices verified against Apple open source:
// IOSCSIArchitectureModelFamily/UserClientLib/SCSITaskLib.h
/// Read a function pointer from a COM vtable at the given index.
///
/// # Safety
/// `iface` must be a valid COM interface pointer (*mut *mut c_void), and
/// `index` must be a valid vtable slot for the target type `T`.
unsafe fn vtable_fn<T>(iface: ComRef, index: usize) -> T {
// Rust 2024: `unsafe fn` bodies are no longer implicitly unsafe.
// Each unsafe op needs its own `unsafe { }` block.
unsafe {
let vtable = *iface as *const *const std::ffi::c_void;
let fn_ptr = *vtable.add(index);
std::mem::transmute_copy(&fn_ptr)
}
}
/// Call Release (vtable index 3) on any COM interface.
fn com_release(iface: ComRef) {
type Fn = unsafe extern "C" fn(ComRef) -> u32;
unsafe {
let f: Fn = vtable_fn(iface, 3);
f(iface);
}
}
// ── SCSITaskDeviceInterface vtable ──────────────────────────────────────────
//
// Index Method
// 0 _reserved
// 1 QueryInterface
// 2 AddRef
// 3 Release
// 4 IsExclusiveAccessAvailable
// 5 AddCallbackDispatcherToRunLoop
// 6 RemoveCallbackDispatcherFromRunLoop
// 7 ObtainExclusiveAccess
// 8 ReleaseExclusiveAccess
// 9 CreateSCSITask
const VTIDX_OBTAIN_EXCLUSIVE: usize = 7;
const VTIDX_RELEASE_EXCLUSIVE: usize = 8;
const VTIDX_CREATE_TASK: usize = 9;
// ── SCSITaskInterface vtable ────────────────────────────────────────────────
//
// Index Method
// 0 _reserved
// 1 QueryInterface
// 2 AddRef
// 3 Release
// 4 IsTaskActive
// 5 SetTaskAttribute
// 6 GetTaskAttribute
// 7 GetTaskState
// 8 SetCommandDescriptorBlock
// 9 GetCommandDescriptorBlockSize
// 10 GetCommandDescriptorBlock
// 11 SetScatterGatherEntries
// 12 SetTimeoutDuration
// 13 GetTimeoutDuration
// 14 SetTaskCompletionCallback
// 15 ExecuteTaskSync
// 16 ExecuteTaskAsync
// 17 AbortTask
// 18 GetSCSIServiceResponse
// 19 GetTaskStatus
// 20 GetRealizedDataTransferCount
// 21 GetAutoSenseData
const VTIDX_SET_CDB: usize = 8;
const VTIDX_SET_SG: usize = 11;
const VTIDX_SET_TIMEOUT: usize = 12;
const VTIDX_EXECUTE_SYNC: usize = 15;
// ── Transport implementation ────────────────────────────────────────────────
pub struct MacScsiTransport {
device_iface: ComRef,
exclusive: bool,
_bsd_name: String,
}
// IOKit COM interface pointers are Mach port references — safe to send between threads.
unsafe impl Send for MacScsiTransport {}
impl MacScsiTransport {
@@ -186,7 +58,6 @@ impl MacScsiTransport {
path: device.display().to_string(),
})?;
// Strip /dev/ prefix to get BSD name (e.g. "disk2")
let bsd_name = if let Some(rest) = dev_str.strip_prefix("/dev/r") {
rest
} else if let Some(rest) = dev_str.strip_prefix("/dev/") {
@@ -195,128 +66,118 @@ impl MacScsiTransport {
dev_str
};
let service = find_scsi_service(bsd_name)?;
let mut bsd_c = bsd_name.as_bytes().to_vec();
bsd_c.push(0);
// Create IOKit plugin for the MMC device
let mut plugin: ComRef = std::ptr::null_mut();
let mut score: i32 = 0;
let kr = unsafe {
IOCreatePlugInInterfaceForService(
service,
&K_IO_MMC_DEVICE_USER_CLIENT_TYPE_ID,
&K_IO_CFPLUGIN_INTERFACE_ID,
&mut plugin,
&mut score,
)
};
unsafe { IOObjectRelease(service) };
if kr != K_IO_RETURN_SUCCESS || plugin.is_null() {
return Err(Error::IoKitPluginFailed {
path: dev_str.to_string(),
kr: kr as u32,
});
}
// QueryInterface for SCSITaskDeviceInterface
let mut device_iface: ComRef = std::ptr::null_mut();
let hr = unsafe {
type QiFn = unsafe extern "C" fn(ComRef, *const [u8; 16], *mut ComRef) -> i32;
let qi: QiFn = vtable_fn(plugin, 1);
qi(
plugin,
&K_IO_SCSI_TASK_DEVICE_INTERFACE_ID,
&mut device_iface,
)
};
com_release(plugin);
if hr != 0 || device_iface.is_null() {
return Err(Error::ScsiInterfaceUnavailable {
path: dev_str.to_string(),
});
}
// Obtain exclusive access
let kr = unsafe {
type Fn = unsafe extern "C" fn(ComRef) -> IOReturn;
let f: Fn = vtable_fn(device_iface, VTIDX_OBTAIN_EXCLUSIVE);
f(device_iface)
};
if kr != K_IO_RETURN_SUCCESS {
com_release(device_iface);
// No "Try: diskutil unmountDisk" hint — that's the CLI's job.
// The typed variant carries device path + IOReturn so the
// caller can render the right message in the right language.
return Err(Error::DeviceLocked {
path: dev_str.to_string(),
kr: kr as u32,
let rc = unsafe { shim_open_exclusive(bsd_c.as_ptr()) };
if rc != 0 {
return Err(Error::DeviceNotFound {
path: bsd_name.to_string(),
});
}
Ok(MacScsiTransport {
device_iface,
exclusive: true,
_bsd_name: bsd_name.to_string(),
})
}
// `reset()` removed in 0.13.6 — see scsi/mod.rs for rationale.
}
/// Enumerate optical drives on macOS. Mirrors `drive::macos::find_drives`
/// (which iterates `/dev/disk0..15` + INQUIRY + filters peripheral
/// type 5). Same logic, exposed through the new `DriveInfo` shape so
/// callers — `list_drives()` in `scsi::mod` — never reach into
/// `crate::drive::macos`.
pub(super) fn list_drives() -> Vec<super::DriveInfo> {
let mut out = Vec::new();
for i in 0..K_DEV_DISK_MAX {
let path = format!("/dev/disk{i}");
if !std::path::Path::new(&path).exists() {
continue;
impl Drop for MacScsiTransport {
fn drop(&mut self) {
unsafe { shim_close() };
}
}
impl ScsiTransport for MacScsiTransport {
fn execute(
&mut self,
cdb: &[u8],
direction: DataDirection,
data: &mut [u8],
_timeout_ms: u32,
) -> Result<ScsiResult> {
let data_in = match direction {
DataDirection::FromDevice => 1,
DataDirection::ToDevice => 0,
DataDirection::None => 0,
};
let mut sense = [0u8; K_SENSE_DATA_SIZE];
let mut task_status: u8 = 0xFF;
let mut transfer_count: u64 = 0;
let kr = unsafe {
shim_execute(
cdb.as_ptr(),
cdb.len() as u8,
data.as_mut_ptr(),
data.len() as u32,
data_in,
sense.as_mut_ptr(),
K_SENSE_DATA_SIZE as u32,
&mut task_status,
&mut transfer_count,
)
};
if kr != 0 {
return Err(Error::ScsiError {
opcode: cdb[0],
status: super::SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
});
}
let mut transport = match MacScsiTransport::open(std::path::Path::new(&path)) {
Ok(t) => t,
Err(_) => continue,
};
let inquiry = match super::inquiry(&mut transport) {
Ok(r) => r,
Err(_) => continue,
};
// SCSI peripheral type field is the lower 5 bits of byte 0.
if inquiry.raw.is_empty()
|| (inquiry.raw[K_INQUIRY_TYPE_BYTE] & K_INQUIRY_TYPE_MASK) != K_SCSI_TYPE_OPTICAL
{
if task_status != 0 {
let parsed = super::parse_sense(&sense, K_SENSE_DATA_SIZE as u8);
return Err(Error::ScsiError {
opcode: cdb[0],
status: task_status,
sense: Some(parsed),
});
}
Ok(ScsiResult {
status: 0,
bytes_transferred: transfer_count as usize,
sense,
})
}
}
// ── Drive enumeration (registry-based, no exclusive access) ──────────────
pub(super) fn list_drives() -> Vec<super::DriveInfo> {
let mut buf = [ShimDriveInfo {
bsd_name: [0; 32],
vendor: [0; 32],
model: [0; 48],
firmware: [0; 16],
}; 8];
let count = unsafe { shim_list_drives(buf.as_mut_ptr(), buf.len() as i32) };
let mut out = Vec::new();
for info in buf.iter().take((count as usize).min(buf.len())) {
let bsd_name = cstr_to_str(&info.bsd_name);
if bsd_name.is_empty() {
continue;
}
out.push(super::DriveInfo {
path,
vendor: inquiry.vendor_id,
model: inquiry.model,
firmware: inquiry.firmware,
path: format!("/dev/{bsd_name}"),
vendor: cstr_to_str(&info.vendor).to_string(),
model: cstr_to_str(&info.model).to_string(),
firmware: cstr_to_str(&info.firmware).to_string(),
});
}
out
}
/// Maximum BSD disk index probed during enumeration. macOS assigns
/// `/dev/diskN` sequentially per attached storage device; 16 covers
/// any realistic homelab.
const K_DEV_DISK_MAX: u8 = 16;
fn cstr_to_str(bytes: &[u8]) -> &str {
let end = bytes.iter().position(|&b| b == 0).unwrap_or(bytes.len());
std::str::from_utf8(&bytes[..end]).unwrap_or("")
}
/// SCSI INQUIRY response: peripheral device type lives in byte 0,
/// lower 5 bits.
const K_INQUIRY_TYPE_BYTE: usize = 0;
const K_INQUIRY_TYPE_MASK: u8 = 0x1F;
/// SCSI peripheral type 5 = "CD-ROM device" (covers DVD, BD-ROM, BD-RE).
const K_SCSI_TYPE_OPTICAL: u8 = 0x05;
/// TEST UNIT READY probe on macOS. Any non-"not ready" error bubbles up
/// to the caller — in-library wedge recovery was rolled back in 0.13.4
/// after USB-layer resets failed to recover the LG BU40N on Linux; the
/// macOS impl mirrors that choice for symmetry. See the Linux
/// `drive_has_disc` in `scsi/linux.rs` for the full rationale.
pub(super) fn drive_has_disc(path: &Path) -> Result<bool> {
let mut transport = MacScsiTransport::open(path)?;
let cdb = [crate::scsi::SCSI_TEST_UNIT_READY, 0, 0, 0, 0, 0];
@@ -328,228 +189,7 @@ pub(super) fn drive_has_disc(path: &Path) -> Result<bool> {
crate::scsi::TUR_TIMEOUT_MS,
) {
Ok(_) => Ok(true),
Err(Error::ScsiError { sense_key, .. }) if sense_key == K_SENSE_KEY_NOT_READY => Ok(false),
Err(ref e) if e.scsi_sense().is_some_and(|s| s.is_not_ready()) => Ok(false),
Err(e) => Err(e),
}
}
const K_SENSE_KEY_NOT_READY: u8 = 2;
impl Drop for MacScsiTransport {
fn drop(&mut self) {
if self.exclusive {
unsafe {
type Fn = unsafe extern "C" fn(ComRef) -> IOReturn;
let f: Fn = vtable_fn(self.device_iface, VTIDX_RELEASE_EXCLUSIVE);
f(self.device_iface);
}
}
com_release(self.device_iface);
}
}
impl ScsiTransport for MacScsiTransport {
fn execute(
&mut self,
cdb: &[u8],
direction: DataDirection,
data: &mut [u8],
timeout_ms: u32,
) -> Result<ScsiResult> {
// Create a SCSI task
let task: ComRef = unsafe {
type Fn = unsafe extern "C" fn(ComRef) -> ComRef;
let f: Fn = vtable_fn(self.device_iface, VTIDX_CREATE_TASK);
f(self.device_iface)
};
if task.is_null() {
return Err(Error::ScsiError {
opcode: cdb[0],
status: 0xFF,
sense_key: 0,
});
}
// Set CDB
let mut cdb_padded = [0u8; K_MAX_CDB_SIZE];
let cdb_len = cdb.len().min(K_MAX_CDB_SIZE);
cdb_padded[..cdb_len].copy_from_slice(&cdb[..cdb_len]);
unsafe {
type Fn = unsafe extern "C" fn(ComRef, *const u8, u8) -> IOReturn;
let f: Fn = vtable_fn(task, VTIDX_SET_CDB);
f(task, cdb_padded.as_ptr(), cdb_len as u8);
}
// Set scatter/gather and transfer direction
let iokit_dir = match direction {
DataDirection::None => K_SCSI_DATA_TRANSFER_NO_DATA,
DataDirection::FromDevice => K_SCSI_DATA_TRANSFER_FROM_TARGET,
DataDirection::ToDevice => K_SCSI_DATA_TRANSFER_TO_TARGET,
};
if direction != DataDirection::None && !data.is_empty() {
let sg = SCSITaskSGElement {
address: data.as_mut_ptr() as u64,
length: data.len() as u64,
};
unsafe {
type Fn =
unsafe extern "C" fn(ComRef, *const SCSITaskSGElement, u8, u64, u8) -> IOReturn;
let f: Fn = vtable_fn(task, VTIDX_SET_SG);
f(task, &sg, 1, data.len() as u64, iokit_dir);
}
} else {
unsafe {
type Fn =
unsafe extern "C" fn(ComRef, *const SCSITaskSGElement, u8, u64, u8) -> IOReturn;
let f: Fn = vtable_fn(task, VTIDX_SET_SG);
f(task, std::ptr::null(), 0, 0, K_SCSI_DATA_TRANSFER_NO_DATA);
}
}
// Set timeout (IOKit SCSITask takes milliseconds)
unsafe {
type Fn = unsafe extern "C" fn(ComRef, u32);
let f: Fn = vtable_fn(task, VTIDX_SET_TIMEOUT);
f(task, timeout_ms);
}
// Execute synchronously
let mut sense = [0u8; K_SENSE_DATA_SIZE];
let mut task_status: u32 = 0;
let mut realized_count: u64 = 0;
let kr = unsafe {
type Fn = unsafe extern "C" fn(ComRef, *mut u8, *mut u32, *mut u64) -> IOReturn;
let f: Fn = vtable_fn(task, VTIDX_EXECUTE_SYNC);
f(
task,
sense.as_mut_ptr(),
&mut task_status,
&mut realized_count,
)
};
com_release(task);
if kr != K_IO_RETURN_SUCCESS {
return Err(Error::ScsiError {
opcode: cdb[0],
status: 0xFF,
sense_key: 0,
});
}
if task_status != K_SCSI_TASK_STATUS_GOOD as u32 {
let sense_key = if sense[2] != 0 { sense[2] & 0x0F } else { 0 };
return Err(Error::ScsiError {
opcode: cdb[0],
status: task_status as u8,
sense_key,
});
}
Ok(ScsiResult {
status: task_status as u8,
bytes_transferred: realized_count as usize,
sense,
})
}
}
// ── IOKit service discovery ─────────────────────────────────────────────────
/// BSD name → IOKit service for the SCSI device.
///
/// Walk: IOMedia (BSD name match) → parent chain → SCSIPeripheralDeviceNub.
///
/// All failure paths surface as `Error::DeviceNotFound { path: bsd_name }` —
/// the four internal stages (IOMasterPort / IOBSDNameMatching / IOMedia
/// lookup / walk_to_authoring_device) collapse into one observable error
/// because none of them are user-actionable individually. Pre-0.13 each
/// stage stuffed an English description into `path:` ("…IOMasterPort
/// failed", "…SCSITaskDeviceInterface not available", etc.) which broke
/// the library's "no English text" rule.
fn find_scsi_service(bsd_name: &str) -> Result<IOObject> {
let not_found = || Error::DeviceNotFound {
path: bsd_name.to_string(),
};
let mut master: MachPort = 0;
let kr = unsafe { IOMasterPort(0, &mut master) };
if kr != K_IO_RETURN_SUCCESS {
return Err(not_found());
}
// IOBSDNameMatching creates a dictionary matching { "BSD Name" = bsd_name }
let mut bsd_c = bsd_name.as_bytes().to_vec();
bsd_c.push(0);
let matching = unsafe { IOBSDNameMatching(master, 0, bsd_c.as_ptr()) };
if matching.is_null() {
return Err(not_found());
}
// Find the single IOMedia service (consumes the matching dict)
let media = unsafe { IOServiceGetMatchingService(master, matching) };
if media == 0 {
return Err(not_found());
}
// Walk up the IOService plane to find the authoring device.
// The chain is typically:
// IOMedia → IOPartitionScheme → IOMedia → IOBlockStorageDriver
// → IOSCSIPeripheralDeviceNub (this is what we want)
//
// We walk up until we find a service that IOCreatePlugInInterfaceForService
// accepts with kIOMMCDeviceUserClientTypeID, or until we hit the root.
let service = walk_to_authoring_device(media);
unsafe { IOObjectRelease(media) };
service.ok_or_else(not_found)
}
/// Walk up the IOService plane from an IOMedia to the SCSI authoring device.
fn walk_to_authoring_device(start: IOObject) -> Option<IOObject> {
let mut current = start;
// Retain start so we can release uniformly in the loop
// (IORegistryEntryGetParentEntry retains the parent for us)
// Target class names for authoring devices
let target_classes: &[&[u8]] = &[
b"IOSCSIPeripheralDeviceNub\0",
b"IOBDBlockStorageDevice\0",
b"IODVDBlockStorageDevice\0",
b"IOCDBlockStorageDevice\0",
b"IOBlockStorageDevice\0",
];
// Walk up to 10 levels (more than enough)
for _ in 0..10 {
let mut parent: IOObject = 0;
let kr = unsafe {
IORegistryEntryGetParentEntry(current, c"IOService".as_ptr() as *const u8, &mut parent)
};
if current != start {
unsafe { IOObjectRelease(current) };
}
if kr != K_IO_RETURN_SUCCESS || parent == 0 {
return None;
}
// Check if this parent matches any of our target classes
for class in target_classes {
if unsafe { IOObjectConformsTo(parent, class.as_ptr()) } != 0 {
return Some(parent);
}
}
current = parent;
}
if current != start {
unsafe { IOObjectRelease(current) };
}
None
}
+387
View File
@@ -0,0 +1,387 @@
#include <IOKit/IOKitLib.h>
#include <IOKit/IOCFPlugIn.h>
#include <IOKit/scsi/SCSITaskLib.h>
#include <CoreFoundation/CoreFoundation.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
// ── Types ──────────────────────────────────────────────────────────────────
typedef struct {
IOCFPlugInInterface **plugin;
MMCDeviceInterface **mmc;
SCSITaskDeviceInterface **scsi;
int exclusive;
} ShimHandle;
typedef struct {
char bsd_name[32];
char vendor[32];
char model[48];
char firmware[16];
} ShimDriveInfo;
// ── Global handle (single-drive, same as before) ──────────────────────────
static ShimHandle g_handle = {NULL, NULL, NULL, 0};
// ── Registry helpers ──────────────────────────────────────────────────────
static int cfstring_to_cstr(CFStringRef cf, char *buf, size_t buflen) {
if (!cf) return 0;
if (!CFStringGetCString(cf, buf, buflen, kCFStringEncodingUTF8)) return 0;
return 1;
}
static int registry_entry_bsd_name(io_registry_entry_t entry, char *buf, size_t buflen) {
CFStringRef cf = IORegistryEntryCreateCFProperty(entry, CFSTR("BSD Name"),
kCFAllocatorDefault, 0);
if (!cf) return 0;
int ok = cfstring_to_cstr(cf, buf, buflen);
CFRelease(cf);
return ok;
}
static io_registry_entry_t find_iomedia_child(io_registry_entry_t parent) {
io_iterator_t iter;
kern_return_t kr = IORegistryEntryGetChildIterator(parent, kIOServicePlane, &iter);
if (kr != KERN_SUCCESS) return 0;
io_registry_entry_t child;
while ((child = IOIteratorNext(iter)) != 0) {
char cls[128];
kr = IOObjectGetClass(child, cls);
if (kr == KERN_SUCCESS) {
if (strcmp(cls, "IOMedia") == 0 || strcmp(cls, "IOBDMedia") == 0) {
IOObjectRelease(iter);
return child;
}
}
IOObjectRelease(child);
}
IOObjectRelease(iter);
return 0;
}
static io_registry_entry_t find_child_of_class(io_registry_entry_t parent, const char *target_class) {
io_iterator_t iter;
kern_return_t kr = IORegistryEntryGetChildIterator(parent, kIOServicePlane, &iter);
if (kr != KERN_SUCCESS) return 0;
io_registry_entry_t child;
while ((child = IOIteratorNext(iter)) != 0) {
char cls[128];
kr = IOObjectGetClass(child, cls);
if (kr == KERN_SUCCESS && strcmp(cls, target_class) == 0) {
IOObjectRelease(iter);
return child;
}
IOObjectRelease(child);
}
IOObjectRelease(iter);
return 0;
}
static io_registry_entry_t find_parent_of_class(io_registry_entry_t entry, const char *target_class) {
io_registry_entry_t parent;
kern_return_t kr = IORegistryEntryGetParentEntry(entry, kIOServicePlane, &parent);
if (kr != KERN_SUCCESS) return 0;
char cls[128];
kr = IOObjectGetClass(parent, cls);
if (kr == KERN_SUCCESS && strcmp(cls, target_class) == 0) {
return parent;
}
IOObjectRelease(parent);
return 0;
}
// Given an IOBDServices, find the BSD name of its IOMedia child.
// Chain: IOBDServices -> IOBDBlockStorageDriver -> IOMedia (has "BSD Name")
static int bdsvc_to_bsd_name(io_registry_entry_t bdsvc, char *buf, size_t buflen) {
io_registry_entry_t driver = find_child_of_class(bdsvc, "IOBDBlockStorageDriver");
if (!driver) return 0;
io_registry_entry_t media = find_iomedia_child(driver);
IOObjectRelease(driver);
if (!media) return 0;
int ok = registry_entry_bsd_name(media, buf, buflen);
IOObjectRelease(media);
return ok;
}
// Given an IOBDServices, extract Device Characteristics strings.
static void bdsvc_device_info(io_registry_entry_t bdsvc, ShimDriveInfo *info) {
CFDictionaryRef dc = IORegistryEntryCreateCFProperty(bdsvc,
CFSTR("Device Characteristics"), kCFAllocatorDefault, 0);
if (!dc) return;
CFStringRef val;
val = CFDictionaryGetValue(dc, CFSTR("Vendor Name"));
if (val) cfstring_to_cstr(val, info->vendor, sizeof(info->vendor));
val = CFDictionaryGetValue(dc, CFSTR("Product Name"));
if (val) cfstring_to_cstr(val, info->model, sizeof(info->model));
val = CFDictionaryGetValue(dc, CFSTR("Product Revision Level"));
if (val) cfstring_to_cstr(val, info->firmware, sizeof(info->firmware));
CFRelease(dc);
}
// Find the IOBDServices that owns the given BSD name.
// Returns a retained io_service_t (caller must release), or 0.
static io_service_t find_bdsvc_by_bsd_name(mach_port_t mp, const char *bsd_name) {
CFMutableDictionaryRef matching = IOServiceMatching("IOBDServices");
if (!matching) return 0;
io_iterator_t iter;
kern_return_t kr = IOServiceGetMatchingServices(mp, matching, &iter);
if (kr != KERN_SUCCESS) return 0;
io_service_t result = 0;
io_service_t svc;
while ((svc = IOIteratorNext(iter)) != 0) {
char name[64];
if (bdsvc_to_bsd_name(svc, name, sizeof(name))) {
if (strcmp(name, bsd_name) == 0) {
result = svc;
break;
}
}
IOObjectRelease(svc);
}
if (!result) {
IOIteratorReset(iter);
while ((svc = IOIteratorNext(iter)) != 0) {
IOObjectRelease(svc);
}
}
IOObjectRelease(iter);
return result;
}
// Find the IOBDServices that owns the given BSD name by walking from
// IOMedia upward. Used as fallback when bdsvc_to_bsd_name fails
// (e.g. disc under exclusive access, no IOMedia child).
// Chain: IOMedia -> IOBDBlockStorageDriver -> IOBDServices
static io_service_t find_bdsvc_from_iomedia(mach_port_t mp, const char *bsd_name) {
CFMutableDictionaryRef matching = IOServiceMatching("IOMedia");
if (!matching) return 0;
io_iterator_t iter;
kern_return_t kr = IOServiceGetMatchingServices(mp, matching, &iter);
if (kr != KERN_SUCCESS) return 0;
io_service_t result = 0;
io_service_t media;
while ((media = IOIteratorNext(iter)) != 0) {
char name[64];
if (registry_entry_bsd_name(media, name, sizeof(name))
&& strcmp(name, bsd_name) == 0)
{
io_registry_entry_t driver = find_parent_of_class(media, "IOBDBlockStorageDriver");
if (driver) {
io_registry_entry_t bdsvc = find_parent_of_class(driver, "IOBDServices");
IOObjectRelease(driver);
if (bdsvc) {
result = bdsvc;
IOObjectRelease(media);
break;
}
}
}
IOObjectRelease(media);
}
IOObjectRelease(iter);
return result;
}
// ── Public API ────────────────────────────────────────────────────────────
int shim_open_exclusive(const char *bsd_name) {
kern_return_t kr;
HRESULT hr;
SInt32 score = 0;
if (g_handle.exclusive && g_handle.scsi) {
return 0;
}
// Use a shell wrapper so the device path is not subject to buffer limits.
// snprintf into 128 bytes could truncate long BSD names (e.g. disk12s3s1),
// producing a broken command. sh -c with $1 passes the arg via argv.
const char *shell_fmt = "sh -c 'diskutil unmountDisk force \"$1\" >/dev/null 2>&1' _ %s";
char cmd[512];
int written = snprintf(cmd, sizeof(cmd), shell_fmt, bsd_name);
if (written < 0 || (size_t)written >= sizeof(cmd)) {
return -1;
}
system(cmd);
usleep(500000);
mach_port_t mp;
IOMainPort(0, &mp);
io_service_t svc = find_bdsvc_by_bsd_name(mp, bsd_name);
if (!svc) {
svc = find_bdsvc_from_iomedia(mp, bsd_name);
}
if (!svc) {
CFMutableDictionaryRef matching = IOServiceMatching("IOBDServices");
svc = IOServiceGetMatchingService(mp, matching);
}
if (!svc) return -1;
kr = IOCreatePlugInInterfaceForService(svc,
kIOMMCDeviceUserClientTypeID, kIOCFPlugInInterfaceID,
&g_handle.plugin, &score);
IOObjectRelease(svc);
if (kr != KERN_SUCCESS || !g_handle.plugin) return -2;
hr = (*g_handle.plugin)->QueryInterface(g_handle.plugin,
CFUUIDGetUUIDBytes(kIOMMCDeviceInterfaceID), (LPVOID *)&g_handle.mmc);
if (hr != S_OK || !g_handle.mmc) {
IODestroyPlugInInterface(g_handle.plugin);
g_handle.plugin = NULL;
return -3;
}
g_handle.scsi = (*g_handle.mmc)->GetSCSITaskDeviceInterface(g_handle.mmc);
if (!g_handle.scsi) {
(*g_handle.mmc)->Release(g_handle.mmc);
IODestroyPlugInInterface(g_handle.plugin);
g_handle.mmc = NULL;
g_handle.plugin = NULL;
return -4;
}
for (int retry = 0; retry < 10; retry++) {
kr = (*g_handle.scsi)->ObtainExclusiveAccess(g_handle.scsi);
if (kr == kIOReturnSuccess) break;
usleep(500000);
}
if (kr != kIOReturnSuccess) {
(*g_handle.scsi)->Release(g_handle.scsi);
(*g_handle.mmc)->Release(g_handle.mmc);
IODestroyPlugInInterface(g_handle.plugin);
g_handle.scsi = NULL;
g_handle.mmc = NULL;
g_handle.plugin = NULL;
return -5;
}
g_handle.exclusive = 1;
return 0;
}
void shim_close(void) {
if (g_handle.exclusive && g_handle.scsi) {
(*g_handle.scsi)->ReleaseExclusiveAccess(g_handle.scsi);
}
if (g_handle.scsi) {
(*g_handle.scsi)->Release(g_handle.scsi);
g_handle.scsi = NULL;
}
if (g_handle.mmc) {
(*g_handle.mmc)->Release(g_handle.mmc);
g_handle.mmc = NULL;
}
if (g_handle.plugin) {
IODestroyPlugInInterface(g_handle.plugin);
g_handle.plugin = NULL;
}
g_handle.exclusive = 0;
}
int shim_execute(const unsigned char *cdb, unsigned char cdb_len,
void *buf, unsigned int buf_len, int data_in,
unsigned char *sense_out, unsigned int sense_len,
unsigned char *task_status_out, unsigned long long *transfer_count) {
if (!g_handle.scsi) return -1;
SCSITaskInterface **task = (*g_handle.scsi)->CreateSCSITask(g_handle.scsi);
if (!task) return -2;
SCSICommandDescriptorBlock cdb_buf;
memset(&cdb_buf, 0, sizeof(cdb_buf));
memcpy(&cdb_buf, cdb, cdb_len);
(*task)->SetCommandDescriptorBlock(task, cdb_buf, cdb_len);
if (buf_len > 0 && buf) {
SCSITaskSGElement sg;
sg.address = (UInt64)(uintptr_t)buf;
sg.length = buf_len;
(*task)->SetScatterGatherEntries(task, &sg, 1, buf_len,
data_in ? kSCSIDataTransfer_FromTargetToInitiator
: kSCSIDataTransfer_FromInitiatorToTarget);
} else {
(*task)->SetScatterGatherEntries(task, NULL, 0, 0,
kSCSIDataTransfer_NoDataTransfer);
}
(*task)->SetTimeoutDuration(task, 30000);
SCSI_Sense_Data sense;
memset(&sense, 0, sizeof(sense));
SCSITaskStatus status = 0xFF;
UInt64 count = 0;
IOReturn kr = (*task)->ExecuteTaskSync(task, &sense, &status, &count);
if (sense_out && sense_len > 0) {
size_t copy = sense_len < sizeof(sense) ? sense_len : sizeof(sense);
memcpy(sense_out, &sense, copy);
}
if (task_status_out) *task_status_out = (unsigned char)status;
if (transfer_count) *transfer_count = count;
(*task)->Release(task);
return (int)kr;
}
// ── Registry-based drive enumeration ──────────────────────────────────────
//
// Walks IOBDServices entries in the IOKit registry. No exclusive access,
// no SCSI commands, no unmounts. Returns up to max_entries drives.
int shim_list_drives(ShimDriveInfo *out, int max_entries) {
mach_port_t mp;
IOReturn ret = IOMainPort(0, &mp);
if (ret != kIOReturnSuccess) return 0;
CFMutableDictionaryRef matching = IOServiceMatching("IOBDServices");
if (!matching) return 0;
io_iterator_t iter;
kern_return_t kr = IOServiceGetMatchingServices(mp, matching, &iter);
if (kr != KERN_SUCCESS) return 0;
int count = 0;
io_service_t svc;
while ((svc = IOIteratorNext(iter)) != 0 && count < max_entries) {
ShimDriveInfo *info = &out[count];
memset(info, 0, sizeof(*info));
bdsvc_device_info(svc, info);
bdsvc_to_bsd_name(svc, info->bsd_name, sizeof(info->bsd_name));
if (info->bsd_name[0]) {
count++;
}
IOObjectRelease(svc);
}
IOObjectRelease(iter);
return count;
}
+353 -5
View File
@@ -2,11 +2,11 @@
//!
//! Platform backends are in separate files:
//! - `linux.rs` — SG_IO ioctl
//! - `macos.rs` — IOKit SCSITaskDeviceInterface
//! - `macos.rs` — IOKit SCSITaskDeviceInterface (exclusive access)
//! - `windows.rs` — SPTI (SCSI Pass-Through Interface)
#[cfg(target_os = "linux")]
mod linux;
pub mod linux;
#[cfg(target_os = "macos")]
mod macos;
#[cfg(target_os = "windows")]
@@ -43,6 +43,241 @@ pub const AACS_KEY_CLASS: u8 = 0x02;
/// a poll-loop tick.
pub(crate) const TUR_TIMEOUT_MS: u32 = 5_000;
/// Timeout for content READ commands (READ_10 / READ_12) on the fast
/// path — the [`disc::Disc::copy`] sweep that bisects-on-failure.
///
/// 10 s is calibrated from live empirical data on an LG BU40N + Initio
/// 1618L bridge ripping a UHD with marginal sectors:
///
/// - Sustained sequential reads: 3 – 7 ms
/// - Cold-start seek + read: up to ~1500 ms
/// - Successful ECC recovery: 1.6 – 2.6 sec
/// - Confirmed unreadable sector: 3.6 – 8.8 sec (kernel timeout)
///
/// 10 s catches every legitimate slow read with comfortable margin and
/// short-circuits truly bad sectors at ~10 s rather than letting the
/// kernel mid-layer escalate for 30 s+. See the SCSI architecture audit
/// (2026-04-26) for primary-source references.
///
/// Pre-0.13.21 this was 1.5 s, which forced the kernel mid-layer to
/// time out *normal* reads (cold-start often takes ~1.5 s) and run its
/// full ABORT TASK / LUN RESET / BUS RESET escalation while userspace
/// kept submitting fresh reads. The Initio bridge couldn't drain the
/// resulting command queue and entered a wedge state that only physical
/// replug recovered — proven by the v0.13.18 + v0.13.20 live tests.
pub(crate) const READ_TIMEOUT_MS: u32 = 10_000;
/// Timeout for content READ commands on the recovery path —
/// [`disc::Disc::patch`]'s targeted retries on bad ranges. Doubles
/// the fast-path budget so a sector that fails at 30 s gets one more
/// honest attempt. Matches sg_dd's default per-command timeout
/// (`DEF_TIMEOUT = 60000`).
pub(crate) const READ_RECOVERY_TIMEOUT_MS: u32 = 60_000;
// ── SCSI status bytes (SPC-4 §4.5.5) ────────────────────────────────────────
/// Status byte 0x00 — `GOOD`. Command completed successfully.
pub const SCSI_STATUS_GOOD: u8 = 0x00;
/// Status byte 0x02 — `CHECK CONDITION`. Drive completed the command
/// reply and attached sense data describing the failure.
pub const SCSI_STATUS_CHECK_CONDITION: u8 = 0x02;
/// libfreemkv-synthesised sentinel: the transport never delivered a
/// SCSI status byte (kernel timeout, USB bridge wedge, IOKit service
/// failure). Distinct from any drive-returned value. Carriers
/// [`Error::ScsiError`] with `sense = None`.
pub const SCSI_STATUS_TRANSPORT_FAILURE: u8 = 0xFF;
// ── SPC-4 sense keys (§4.5.6 Table 28) ─────────────────────────────────────
//
// Broad failure category returned in a CHECK CONDITION reply's sense data.
// Names match the SCSI spec; predicate methods on [`ScsiSense`] (e.g.
// `is_medium_error`, `is_unit_attention`) read more fluently than raw
// constant comparisons at call sites.
pub const SENSE_KEY_NO_SENSE: u8 = 0x00;
pub const SENSE_KEY_RECOVERED_ERROR: u8 = 0x01;
pub const SENSE_KEY_NOT_READY: u8 = 0x02;
pub const SENSE_KEY_MEDIUM_ERROR: u8 = 0x03;
pub const SENSE_KEY_HARDWARE_ERROR: u8 = 0x04;
pub const SENSE_KEY_ILLEGAL_REQUEST: u8 = 0x05;
pub const SENSE_KEY_UNIT_ATTENTION: u8 = 0x06;
pub const SENSE_KEY_DATA_PROTECT: u8 = 0x07;
pub const SENSE_KEY_BLANK_CHECK: u8 = 0x08;
pub const SENSE_KEY_ABORTED_COMMAND: u8 = 0x0B;
// ── Sense parsing ───────────────────────────────────────────────────────────
/// Decoded SPC-4 sense triple — the precise reason a SCSI command failed.
///
/// Returned by [`parse_sense`] and embedded inside [`Error::ScsiError`]
/// (`sense: Option<ScsiSense>`). Predicate methods (`is_medium_error`,
/// `is_unit_attention`, `is_marginal`, …) read more fluently at call
/// sites than raw `sense_key` comparisons.
///
/// `Default::default()` and the [`ScsiSense::NONE`] constant both
/// produce the all-zero "no sense info" triple. Per SPC-4 §4.5.3, an
/// empty sense buffer is reported as NO SENSE (key 0); use the constant
/// for explicit intent at construction sites.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default)]
pub struct ScsiSense {
/// Sense key — broad failure category (SPC-4 §4.5.6 Table 28).
/// See the `SENSE_KEY_*` constants for named values.
pub sense_key: u8,
/// Additional Sense Code — narrows the cause within a sense key
/// (SPC-4 §4.5.6 Table 29). E.g. `0x11` = UNRECOVERED READ ERROR.
pub asc: u8,
/// Additional Sense Code Qualifier — finest-grain disambiguation.
/// E.g. `0x05` (with `asc=0x11`) = L-EC UNCORRECTABLE.
pub ascq: u8,
}
impl ScsiSense {
/// Sense reply with all-zero fields — explicit "no sense info"
/// constructor for sites where `Default::default()` would be opaque.
pub const NONE: ScsiSense = ScsiSense {
sense_key: 0,
asc: 0,
ascq: 0,
};
/// `true` when the sense key indicates a *marginal-read* failure —
/// the kind of error where the same read at smaller granularity
/// (or a brief retry) sometimes succeeds:
///
/// - `MEDIUM ERROR` (3) — canonical bad-sector signal
/// - `NOT READY` (2) — on many drives (notably BU40N), this is the
/// dominant response for unreadable sectors (ASC 04/3E, 04/01, etc.)
/// - `ABORTED COMMAND` (B) — transient; retry usually works
/// - `RECOVERED ERROR` (1) / `NO SENSE` (0) — drive is healthy and
/// either recovered the data or has no specific fault to report
///
/// `false` for HARDWARE ERROR, DATA PROTECT, UNIT ATTENTION,
/// ILLEGAL REQUEST, BLANK CHECK, and any unknown key. Used
/// by [`Error::is_marginal_read`] / `Disc::copy`'s hysteresis
/// dispatch.
pub fn is_marginal(&self) -> bool {
matches!(
self.sense_key,
SENSE_KEY_NO_SENSE
| SENSE_KEY_RECOVERED_ERROR
| SENSE_KEY_NOT_READY
| SENSE_KEY_MEDIUM_ERROR
| SENSE_KEY_ABORTED_COMMAND
)
}
/// `true` if `sense_key == MEDIUM ERROR (3)` — canonical "bad sector"
/// signal from the drive.
pub fn is_medium_error(&self) -> bool {
self.sense_key == SENSE_KEY_MEDIUM_ERROR
}
/// `true` if `sense_key == HARDWARE ERROR (4)` — drive itself is
/// failing. Not recoverable by retry.
pub fn is_hardware_error(&self) -> bool {
self.sense_key == SENSE_KEY_HARDWARE_ERROR
}
/// `true` if `sense_key == NOT READY (2)` — medium not present /
/// drive becoming ready / etc.
pub fn is_not_ready(&self) -> bool {
self.sense_key == SENSE_KEY_NOT_READY
}
/// `true` if `sense_key == UNIT ATTENTION (6)` — disc/drive state
/// changed since the prior command (media inserted/removed,
/// power-on reset, parameters changed). Caller should rescan rather
/// than retry the read.
pub fn is_unit_attention(&self) -> bool {
self.sense_key == SENSE_KEY_UNIT_ATTENTION
}
/// `true` if `sense_key == DATA PROTECT (7)` — read blocked by
/// AACS / region / write-protect. Retry won't help.
pub fn is_data_protect(&self) -> bool {
self.sense_key == SENSE_KEY_DATA_PROTECT
}
/// `true` if `sense_key == ILLEGAL REQUEST (5)` — typically a bug
/// in the CDB we sent (LBA out of range, reserved bit, etc.). Don't
/// retry.
pub fn is_illegal_request(&self) -> bool {
self.sense_key == SENSE_KEY_ILLEGAL_REQUEST
}
/// `true` if `sense_key == ABORTED COMMAND (B)` — transient; one
/// retry is usually safe.
pub fn is_aborted_command(&self) -> bool {
self.sense_key == SENSE_KEY_ABORTED_COMMAND
}
}
/// Decode an SPC-4 sense buffer into the structured triple
/// `(sense_key, asc, ascq)`.
///
/// Handles both response-code formats SPC-4 mandates:
///
/// - **Descriptor format** (response code `0x72` / `0x73`):
/// - sense key = `sense[1] & 0x0F`
/// - asc = `sense[2]`
/// - ascq = `sense[3]`
/// - **Fixed format** (response code `0x70` / `0x71` and any unknown
/// code per SPC-4 §4.5.3):
/// - sense key = `sense[2] & 0x0F`
/// - asc = `sense[12]`
/// - ascq = `sense[13]`
///
/// `sb_len_wr` is the number of bytes the transport actually wrote into
/// `sense`. When the buffer is too short for the relevant fields we
/// return [`ScsiSense::NONE`] for the missing pieces rather than reading
/// uninitialised memory. The minimum useful sense reply per SPC-4 is 8
/// bytes (descriptor) or 14 bytes (fixed, to reach ASC/ASCQ at offsets
/// 12/13).
///
/// Pure function — same parse on every platform backend (Linux SG_IO,
/// macOS IOKit, Windows SPTI) so a regression here would silently
/// mis-route SCSI errors on all three OSes simultaneously.
pub(crate) fn parse_sense(sense: &[u8], sb_len_wr: u8) -> ScsiSense {
let n = (sb_len_wr as usize).min(sense.len());
if n < 3 {
return ScsiSense::NONE;
}
let response_code = sense[0] & 0x7F;
let descriptor = response_code == 0x72 || response_code == 0x73;
if descriptor {
// Descriptor format: key/asc/ascq are at fixed offsets 1/2/3.
let asc = if n >= 3 { sense[2] } else { 0 };
let ascq = if n >= 4 { sense[3] } else { 0 };
ScsiSense {
sense_key: sense[1] & 0x0F,
asc,
ascq,
}
} else {
// Fixed format: key at byte 2, ASC/ASCQ at bytes 12/13.
let asc = if n >= 13 { sense[12] } else { 0 };
let ascq = if n >= 14 { sense[13] } else { 0 };
ScsiSense {
sense_key: sense[2] & 0x0F,
asc,
ascq,
}
}
}
// ── SG_IO driver_status bits ────────────────────────────────────────────────
/// `DRIVER_SENSE` (0x08) — bit set in `driver_status` to indicate that
/// sense data was attached to a CHECK CONDITION reply. **Not** a transport
/// failure on its own. Mask this off before deciding whether `driver_status`
/// represents a real bus/host problem.
///
/// Used by Linux SG_IO (`sg_io_hdr.driver_status`); macOS IOKit and
/// Windows SPTI carry the equivalent signal in different fields and
/// don't need the same masking — the misclassification was Linux-only.
#[cfg(target_os = "linux")]
pub(crate) const DRIVER_SENSE: u16 = 0x08;
// ── Types ───────────────────────────────────────────────────────────────────
#[derive(Debug, Clone, Copy, PartialEq)]
@@ -102,9 +337,9 @@ pub fn open(device: &Path) -> Result<Box<dyn ScsiTransport>> {
// platform reset in a thread+recv_timeout so a kernel-wedged ioctl
// couldn't hang the caller. Removed in 0.13.6 along with the
// SG_SCSI_RESET / STOP+START UNIT escalation that needed it. The
// remaining platform reset (Linux: SgIoTransport::reset, called only
// from SgIoTransport::open) does pure userspace state cleanup with
// bounded sleeps — no escape-hatch wrapper required.
// remaining platform reset (Linux: SgIoTransport::reset, available
// for explicit opt-in) does pure userspace state cleanup with bounded
// sleeps — no escape-hatch wrapper required.
// ── USB-layer recovery: rolled back in 0.13.4 ───────────────────────────────
//
@@ -326,3 +561,116 @@ pub fn build_read10_raw(lba: u32, count: u16) -> [u8; 10] {
0x00,
]
}
#[cfg(test)]
mod parse_sense_tests {
//! Unit tests for `parse_sense_key`. Covers both SPC-4 sense data
//! formats (descriptor / fixed) and the short-buffer fallback. The
//! same helper runs on every platform backend so a regression here
//! would silently miscategorize SCSI errors on Linux, macOS, and
//! Windows simultaneously.
use super::parse_sense;
fn parse_sense_key(sense: &[u8], sb_len_wr: u8) -> u8 {
parse_sense(sense, sb_len_wr).sense_key
}
/// Helper: build a 32-byte sense buffer whose first three bytes are
/// the given prefix; the rest are zeroes (sense data area).
fn buf(b0: u8, b1: u8, b2: u8) -> [u8; 32] {
let mut s = [0u8; 32];
s[0] = b0;
s[1] = b1;
s[2] = b2;
s
}
#[test]
fn descriptor_format_72_picks_byte_1() {
// Response code 0x72 (current, descriptor): sense key is the
// low nibble of byte 1. Byte 2 here is 0x77 to prove it is NOT
// the byte the parser reads.
let s = buf(0x72, 0x05, 0x77); // ILLEGAL REQUEST
assert_eq!(parse_sense_key(&s, 8), 5);
}
#[test]
fn descriptor_format_73_picks_byte_1() {
// Response code 0x73 (deferred, descriptor): same parse rule
// as 0x72.
let s = buf(0x73, 0x06, 0xFF); // UNIT ATTENTION
assert_eq!(parse_sense_key(&s, 8), 6);
}
#[test]
fn fixed_format_70_picks_byte_2() {
// Response code 0x70 (current, fixed): sense key is the low
// nibble of byte 2. Byte 1 is 0x77 to prove it is NOT read.
let s = buf(0x70, 0x77, 0x05); // ILLEGAL REQUEST
assert_eq!(parse_sense_key(&s, 18), 5);
}
#[test]
fn fixed_format_71_picks_byte_2() {
// Response code 0x71 (deferred, fixed): same parse as 0x70.
let s = buf(0x71, 0x77, 0x02); // NOT READY
assert_eq!(parse_sense_key(&s, 18), 2);
}
#[test]
fn high_bit_in_byte_0_is_masked() {
// SPC-4 sets the top bit of byte 0 ("INFORMATION VALID" / "VALID")
// independently of the response code. parse_sense_key must mask
// it off before classifying the format.
let s = buf(0xF2, 0x05, 0x77);
assert_eq!(
parse_sense_key(&s, 8),
5,
"VALID-bit must not leak into format detection"
);
let s = buf(0xF0, 0x77, 0x02);
assert_eq!(parse_sense_key(&s, 18), 2);
}
#[test]
fn high_nibble_in_key_byte_is_masked() {
// Sense key is byte_n & 0x0F (low nibble). Top nibble holds
// FILEMARK / EOM / ILI / SDAT_OVFL flags, which must not bleed
// into the key value.
let s = buf(0x70, 0x00, 0xE5); // 0xE0 flags + key 5
assert_eq!(parse_sense_key(&s, 18), 5);
}
#[test]
fn sb_len_wr_zero_returns_no_sense() {
// Transport set status non-zero but wrote zero sense bytes —
// SPC-4 §4.5.3 says treat as NO SENSE (key 0).
let s = buf(0x72, 0x05, 0x05);
assert_eq!(parse_sense_key(&s, 0), 0);
}
#[test]
fn sb_len_wr_below_three_returns_no_sense() {
// Less than three bytes in the buffer means we can't safely
// read either format byte 0 or key byte 2 — return 0.
let s = buf(0x72, 0x05, 0x05);
assert_eq!(parse_sense_key(&s, 1), 0);
assert_eq!(parse_sense_key(&s, 2), 0);
}
#[test]
fn slice_below_three_returns_no_sense() {
// Defense-in-depth: even if a caller passes a too-short slice
// with a falsely-large sb_len_wr, we don't panic and we return 0.
let s = [0x72u8, 0x05];
assert_eq!(parse_sense_key(&s, 8), 0);
}
#[test]
fn unknown_response_code_falls_through_to_fixed() {
// SPC-4 mandates implementations tolerate unknown response
// codes and treat them as fixed format. Vendor-specific codes
// in the 0x74..0x7E range surface here.
let s = buf(0x7A, 0x77, 0x03); // MEDIUM ERROR via "fixed"
assert_eq!(parse_sense_key(&s, 18), 3);
}
}
+44 -14
View File
@@ -87,6 +87,10 @@ pub struct SptiTransport {
handle: isize,
}
// SptiTransport's only field is the isize HANDLE — Send is auto-derived
// and intentional. Sync is NOT: handle mutation in execute() requires
// &mut, enforced by the trait object dispatch.
/// Normalize a device path to Windows \\.\X: format.
///
/// NOTE: A near-identical `normalize_path` exists in `drive::windows`.
@@ -129,8 +133,17 @@ impl SptiTransport {
};
if handle == INVALID_HANDLE_VALUE {
return Err(Error::DeviceNotFound {
path: format!("{}: cannot open device (run as administrator)", dev_str),
// Map last-os-error → Error variant; don't embed English hints
// in the path field (the CLI handles localization).
let err = std::io::Error::last_os_error();
return Err(if err.kind() == std::io::ErrorKind::PermissionDenied {
Error::DevicePermission {
path: dev_str.to_string(),
}
} else {
Error::DeviceNotFound {
path: dev_str.to_string(),
}
});
}
@@ -215,15 +228,17 @@ pub(super) fn drive_has_disc(path: &Path) -> Result<bool> {
crate::scsi::TUR_TIMEOUT_MS,
) {
Ok(_) => Ok(true),
Err(Error::ScsiError { sense_key: 2, .. }) => Ok(false),
Err(ref e) if e.scsi_sense().is_some_and(|s| s.is_not_ready()) => Ok(false),
Err(e) => Err(e),
}
}
impl Drop for SptiTransport {
fn drop(&mut self) {
unsafe {
CloseHandle(self.handle);
if self.handle != INVALID_HANDLE_VALUE {
unsafe {
CloseHandle(self.handle);
}
}
}
}
@@ -254,7 +269,11 @@ impl ScsiTransport for SptiTransport {
DataDirection::ToDevice => SCSI_IOCTL_DATA_OUT,
};
sptwb.spt.DataTransferLength = data.len() as u32;
sptwb.spt.TimeOutValue = (timeout_ms / 1000).max(1) as u32;
// Round up to the next whole second so a 1500ms request gets at
// least 2s, not 1s. SPTI's TimeOutValue is u32 seconds with no
// sub-second resolution; biasing toward "more time" is safer than
// truncating (truncation broke 1500ms fast-reads on Drive::read).
sptwb.spt.TimeOutValue = ((timeout_ms + 999) / 1000).max(1);
sptwb.spt.DataBuffer = if data.is_empty() {
std::ptr::null_mut()
} else {
@@ -280,23 +299,34 @@ impl ScsiTransport for SptiTransport {
};
if ok == 0 {
// Driver-level failure (timeout, handle gone, etc.). Bubble
// up; in-library handle recovery was removed in 0.13.20 along
// with Linux's async fd-recovery and macOS's `try_recover` —
// the kernel mid-layer already did its escalation by the time
// DeviceIoControl returned, and re-issuing reset/reopen here
// is at best redundant and at worst deepens the wedge. Caller
// surfaces the failure to UX.
return Err(Error::ScsiError {
opcode: cdb[0],
status: 0xFF,
sense_key: 0,
status: super::SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
});
}
if sptwb.spt.ScsiStatus != 0 {
let sense_key = if sptwb.sense[2] != 0 {
sptwb.sense[2] & 0x0F
} else {
0
};
// SPTI doesn't surface a "bytes written into sense buffer"
// count separate from SenseInfoLength (input). Pass the full
// K_SENSE_SIZE; parse_sense keys off byte 0's response code
// to handle descriptor (0x72/0x73) vs fixed (0x70/0x71).
//
// 0.13.23: carry the full SPC-4 sense triple in
// `Error::ScsiError::sense` so callers can route on
// `ScsiSense::is_medium_error()` etc.
let parsed = super::parse_sense(&sptwb.sense, K_SENSE_SIZE as u8);
return Err(Error::ScsiError {
opcode: cdb[0],
status: sptwb.spt.ScsiStatus,
sense_key,
sense: Some(parsed),
});
}
+2
View File
@@ -24,6 +24,8 @@ pub trait SectorReader: Send {
fn capacity(&self) -> u32 {
0
}
fn set_speed(&mut self, _kbs: u16) {}
}
/// SectorReader backed by a file (ISO image).
+42 -9
View File
@@ -278,10 +278,11 @@ impl UdfFs {
/// The data_lba is partition-relative.
fn read_icb_extent(&self, reader: &mut dyn SectorReader, meta_lba: u32) -> Result<(u32, u32)> {
let extents = self.read_icb_extents(reader, meta_lba)?;
extents
.first()
.copied()
.ok_or(Error::DiscRead { sector: 0 })
extents.first().copied().ok_or(Error::DiscRead {
sector: 0,
status: None,
sense: None,
})
}
/// Read ALL allocation extents for a file from its ICB.
@@ -307,6 +308,8 @@ impl UdfFs {
if ad_offset + l_ad > icb.len() {
return Err(Error::DiscRead {
sector: self.meta_to_abs(meta_lba) as u64,
status: None,
sense: None,
});
}
(ad_offset, l_ad)
@@ -319,11 +322,19 @@ impl UdfFs {
if ad_offset + l_ad > icb.len() {
return Err(Error::DiscRead {
sector: self.meta_to_abs(meta_lba) as u64,
status: None,
sense: None,
});
}
(ad_offset, l_ad)
}
_ => return Err(Error::DiscRead { sector: 0 }),
_ => {
return Err(Error::DiscRead {
sector: 0,
status: None,
sense: None,
});
}
};
let mut extents = Vec::new();
@@ -414,7 +425,11 @@ pub fn read_filesystem(reader: &mut dyn SectorReader) -> Result<UdfFs> {
let tag_id = u16::from_le_bytes([avdp[0], avdp[1]]);
if tag_id != 2 {
return Err(Error::DiscRead { sector: 0 });
return Err(Error::DiscRead {
sector: 0,
status: None,
sense: None,
});
}
// Main VDS extent location: bytes [16:20] = LBA, [20:24] = length
@@ -455,14 +470,22 @@ pub fn read_filesystem(reader: &mut dyn SectorReader) -> Result<UdfFs> {
}
if partition_start == 0 {
return Err(Error::DiscRead { sector: 0 });
return Err(Error::DiscRead {
sector: 0,
status: None,
sense: None,
});
}
// Step 3: Parse partition maps from LVD to find metadata partition
// BD-ROM discs (UDF 2.50) use a metadata partition (Type 2 map with "*UDF Metadata Partition")
// The metadata file is stored at lba=0 of the physical partition
let metadata_start = if num_partition_maps >= 2 {
let lvd_sec = lvd_sector.ok_or(Error::DiscRead { sector: 0 })?;
let lvd_sec = lvd_sector.ok_or(Error::DiscRead {
sector: 0,
status: None,
sense: None,
})?;
// Read LVD to check partition map type
let mut lvd = [0u8; 2048];
@@ -497,6 +520,8 @@ pub fn read_filesystem(reader: &mut dyn SectorReader) -> Result<UdfFs> {
if ad_off + 8 > meta_icb.len() {
return Err(Error::DiscRead {
sector: meta_file_lba as u64,
status: None,
sense: None,
});
}
let ad_len = u32::from_le_bytes([
@@ -536,7 +561,11 @@ pub fn read_filesystem(reader: &mut dyn SectorReader) -> Result<UdfFs> {
let fsd_tag = u16::from_le_bytes([fsd[0], fsd[1]]);
if fsd_tag != 256 {
return Err(Error::DiscRead { sector: 0 });
return Err(Error::DiscRead {
sector: 0,
status: None,
sense: None,
});
}
// Root Directory ICB: long_ad at FSD offset 400
@@ -585,6 +614,8 @@ fn read_directory(
if ad_off + 8 > icb.len() {
return Err(Error::DiscRead {
sector: (meta_start + meta_lba) as u64,
status: None,
sense: None,
});
}
let len = u32::from_le_bytes([
@@ -607,6 +638,8 @@ fn read_directory(
if ad_off + 8 > icb.len() {
return Err(Error::DiscRead {
sector: (meta_start + meta_lba) as u64,
status: None,
sense: None,
});
}
let len = u32::from_le_bytes([
+32 -12
View File
@@ -1,6 +1,7 @@
//! Disc sector verification — read every sector and classify health.
use crate::disc::{Chapter, DiscTitle};
use crate::progress::Progress;
use crate::sector::SectorReader;
use std::time::Instant;
@@ -77,10 +78,6 @@ impl VerifyResult {
}
}
/// Progress callback: (sectors_done, total_sectors, current_status)
/// Return false to stop verification early.
pub type ProgressFn = Box<dyn FnMut(u64, u64, SectorStatus) -> bool>;
/// Verify all sectors in a title's extents.
/// Reads in batches for speed, falls back to single-sector on failure.
/// The progress callback returns false to request early stop.
@@ -88,7 +85,7 @@ pub fn verify_title(
reader: &mut dyn SectorReader,
title: &DiscTitle,
batch_sectors: u16,
mut on_progress: Option<ProgressFn>,
on_progress: Option<&dyn Progress>,
) -> VerifyResult {
let start = Instant::now();
let mut good: u64 = 0;
@@ -97,7 +94,6 @@ pub fn verify_title(
let mut bad: u64 = 0;
let mut ranges: Vec<SectorRange> = Vec::new();
let mut sectors_done: u64 = 0;
let mut _stopped = false;
let mut byte_offset: u64 = 0;
let total_sectors: u64 = title.extents.iter().map(|e| e.sector_count as u64).sum();
@@ -140,9 +136,21 @@ pub fn verify_title(
}
sectors_done += count as u64;
if let Some(ref mut cb) = on_progress {
if !cb(sectors_done, total_sectors, status) {
_stopped = true;
if let Some(cb) = on_progress {
let pp = crate::progress::PassProgress {
kind: crate::progress::PassKind::Verify,
work_done: sectors_done,
work_total: total_sectors,
bytes_good_total: (good + slow + recovered) * 2048,
bytes_unreadable_total: bad * 2048,
bytes_pending_total: 0,
bytes_total_disc: total_sectors * 2048,
disc_duration_secs: Some(title.duration_secs),
bytes_bad_in_main_title: 0,
main_title_duration_secs: Some(title.duration_secs),
main_title_size_bytes: Some(total_sectors * 2048),
};
if !cb.report(&pp) {
break 'outer;
}
}
@@ -214,9 +222,21 @@ pub fn verify_title(
}
sectors_done += 1;
if let Some(ref mut cb) = on_progress {
if !cb(sectors_done, total_sectors, status) {
_stopped = true;
if let Some(cb) = on_progress {
let pp = crate::progress::PassProgress {
kind: crate::progress::PassKind::Verify,
work_done: sectors_done,
work_total: total_sectors,
bytes_good_total: (good + slow + recovered) * 2048,
bytes_unreadable_total: bad * 2048,
bytes_pending_total: 0,
bytes_total_disc: total_sectors * 2048,
disc_duration_secs: Some(title.duration_secs),
bytes_bad_in_main_title: 0,
main_title_duration_secs: Some(title.duration_secs),
main_title_size_bytes: Some(total_sectors * 2048),
};
if !cb.report(&pp) {
break 'outer;
}
}
+416 -121
View File
@@ -9,8 +9,8 @@ use libfreemkv::{
SectorReader,
};
use std::io::Write;
use std::sync::Arc;
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
use std::sync::{Arc, Mutex};
use std::time::{Duration, Instant};
const SECTOR_SIZE: usize = 2048;
@@ -175,17 +175,26 @@ fn test_disc_copy_progress_callback_fires() {
let calls = Arc::new(AtomicU64::new(0));
let last_bytes = Arc::new(AtomicU64::new(0));
let calls_cb = calls.clone();
let last_bytes_cb = last_bytes.clone();
let progress = move |bytes: u64, _total: u64| {
calls_cb.fetch_add(1, Ordering::Relaxed);
last_bytes_cb.store(bytes, Ordering::Relaxed);
struct CountingReporter {
calls: Arc<AtomicU64>,
last_bytes: Arc<AtomicU64>,
}
impl libfreemkv::progress::Progress for CountingReporter {
fn report(&self, p: &libfreemkv::progress::PassProgress) -> bool {
self.calls.fetch_add(1, Ordering::Relaxed);
self.last_bytes.store(p.bytes_good_total, Ordering::Relaxed);
true
}
}
let reporter = CountingReporter {
calls: calls.clone(),
last_bytes: last_bytes.clone(),
};
let opts = CopyOptions {
decrypt: false,
on_progress: Some(&progress),
progress: Some(&reporter),
..Default::default()
};
@@ -339,75 +348,73 @@ fn test_file_sector_reader_round_trip() {
assert_eq!(all, data, "bulk read mismatch");
}
// ── 6. Disc::copy stall detection triggers skip-forward (TDD red) ─────────
// ── 6. Pass 1 sweeps the entire disc even when every read fails ───────────
//
// Regression guard for the Dell-host hang where `read_sectors` blocked inside
// a kernel-level USB stall and `Disc::copy` sat frozen for 10+ minutes with
// no progress and no error. The fix introduces `CopyOptions::stall_secs:
// Option<u64>` — when elapsed-since-last-`bytes_good`-advance exceeds the
// threshold, `Disc::copy` treats the current block as a read failure and
// triggers the skip-forward path so the rip can advance.
//
// THIS TEST IS EXPECTED TO FAIL UNTIL THE PARALLEL FIX LANDS.
// - Until `stall_secs` exists on `CopyOptions`, the test will not compile.
// - Once the field exists but the stall guard isn't wired, the spawned copy
// thread will never exit (test fails on the 5s join bound).
// - Once the guard is wired, copy returns within ~stall_secs with
// `complete=false` and `bytes_pending>0`.
// Per RIP_DESIGN.md §2.1 + §3: Disc::copy must reach the end of the disc
// regardless of how many reads fail. The only legitimate early exit is the
// halt flag. With `skip_on_error` and a reader that returns
// Err for every read, Pass 1 must:
// - mark every sector NonTrimmed (so Pass 2 can retry them)
// - return cleanly (no panic, no hang)
// - bytes_good = 0
// - bytes_pending = total_bytes (NonTrimmed counts as pending in mapfile
// accounting; see disc/mapfile.rs::stats)
// - bytes_unreadable = 0 (only Pass 2 marks Unreadable)
// - complete = false (work remains for Pass 2)
// - halted = false (no user stop)
// - ISO file is `total_bytes` size on disk (sparse zeros)
/// Reader that returns Ok for sectors `< block_after`, then returns Err for
/// any sector `>= block_after` after a small per-call delay. Models the
/// realistic Dell-host symptom: reads keep returning Err (skip-forward fires)
/// but no `bytes_good` ever accrues; without a stall guard, Pass 1 grinds
/// silently for tens of minutes.
struct StallingSectorReader {
/// Reader that returns Err for every read. Optionally signals a halt
/// flag on the first read so tests can exercise the halt-during-skip-forward
/// path deterministically (no wallclock dependency).
struct FailingSectorReader {
capacity: u32,
block_after: u32,
/// Per-call delay for sectors >= block_after (simulates slow reads).
err_delay_ms: u64,
release: Arc<AtomicBool>,
/// Retained so callers can release the reader; unused now that the
/// reader returns Err instead of blocking, but kept so the test's
/// existing release plumbing compiles.
park: Arc<(Mutex<()>, std::sync::Condvar)>,
/// If set, signals halt on the first `read_sectors` call. Cleared after
/// the first signal so subsequent reads are plain Err.
halt_on_first_read: Option<Arc<AtomicBool>>,
}
impl StallingSectorReader {
fn new(capacity: u32, block_after: u32) -> Self {
impl FailingSectorReader {
fn new(capacity: u32) -> Self {
Self {
capacity,
block_after,
err_delay_ms: 100,
release: Arc::new(AtomicBool::new(false)),
park: Arc::new((Mutex::new(()), std::sync::Condvar::new())),
halt_on_first_read: None,
}
}
fn release_handle(&self) -> (Arc<AtomicBool>, Arc<(Mutex<()>, std::sync::Condvar)>) {
(self.release.clone(), self.park.clone())
fn with_halt_on_first_read(capacity: u32, halt: Arc<AtomicBool>) -> Self {
Self {
capacity,
halt_on_first_read: Some(halt),
}
}
}
impl SectorReader for StallingSectorReader {
impl SectorReader for FailingSectorReader {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_lba: u32,
_count: u16,
_buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
if lba >= self.block_after {
// Realistic stall model: read takes err_delay_ms then returns
// Err. With skip_on_error+skip_forward, Disc::copy will keep
// skip-forwarding through this region — no bytes_good accrues.
// The stall guard fires when bytes_good is unchanged for
// stall_secs.
std::thread::sleep(Duration::from_millis(self.err_delay_ms));
return Err(libfreemkv::error::Error::DiscRead { sector: lba as u64 });
if let Some(h) = self.halt_on_first_read.take() {
h.store(true, Ordering::Relaxed);
}
let bytes = count as usize * SECTOR_SIZE;
buf[..bytes].fill(0);
Ok(bytes)
// Model what a real damaged-disc read returns: CHECK CONDITION +
// MEDIUM ERROR (sense_key 3, ASC 0x11 UNRECOVERED READ ERROR,
// ASCQ 0x05 L-EC UNCORRECTABLE). Disc::copy's hysteresis must
// engage on this — `Error::DiscRead` is libfreemkv's own
// post-classification signal, not what a real reader emits.
Err(libfreemkv::error::Error::ScsiError {
opcode: libfreemkv::scsi::SCSI_READ_10,
status: libfreemkv::scsi::SCSI_STATUS_CHECK_CONDITION,
sense: Some(libfreemkv::ScsiSense {
sense_key: libfreemkv::scsi::SENSE_KEY_MEDIUM_ERROR,
asc: 0x11,
ascq: 0x05,
}),
})
}
fn capacity(&self) -> u32 {
@@ -416,89 +423,377 @@ impl SectorReader for StallingSectorReader {
}
#[test]
fn test_disc_copy_stall_detection_triggers_skip_forward() {
// 1024 sectors total. Reader serves the first 64 sectors instantly, then
// every later read blocks forever. With stall_secs=2, copy should bail
// out of the stalled block within ~2s and either skip forward or finish
// with bytes_pending > 0 / complete=false.
fn test_disc_copy_completes_full_disc_with_failing_reader() {
// 1024 sectors = 2 MB. Reader fails every read. With skip_on_error +
// skip_on_error, Pass 1 must mark every sector NonTrimmed and return
// cleanly — no bail, no hang.
let capacity_sectors: u32 = 1024;
let block_after: u32 = 64;
let reader = StallingSectorReader::new(capacity_sectors, block_after);
let (release_flag, park) = reader.release_handle();
let mut reader = reader;
let total_bytes: u64 = capacity_sectors as u64 * SECTOR_SIZE as u64;
let mut reader = FailingSectorReader::new(capacity_sectors);
let disc = synthetic_disc(capacity_sectors);
let tmp = tempfile::NamedTempFile::new().expect("tempfile create");
let iso_path = tmp.path().to_path_buf();
drop(tmp);
let iso_path_for_thread = iso_path.clone();
let opts = CopyOptions {
decrypt: false,
multipass: true,
let join = std::thread::spawn(move || {
let opts = CopyOptions {
decrypt: false,
skip_on_error: true,
skip_forward: true,
// ASSUMPTION: parallel fix adds `pub stall_secs: Option<u64>` to
// CopyOptions. If the field name differs, update here.
stall_secs: Some(2),
..Default::default()
};
let t0 = Instant::now();
let res = disc.copy(&mut reader, &iso_path_for_thread, &opts);
(res, t0.elapsed())
});
// Bound the join to ~5s. With stall_secs=2 the copy should exit well
// within this window. If it doesn't, the stall guard isn't working.
let started = Instant::now();
let mut joined = None;
while started.elapsed() < Duration::from_millis(5000) {
if join.is_finished() {
joined = Some(join.join().expect("thread join"));
break;
}
std::thread::sleep(Duration::from_millis(50));
}
// Whether or not the join succeeded, release the parked reader thread so
// it can exit (its &mut reader is owned by the spawned thread; releasing
// lets that thread unwind cleanly).
release_flag.store(true, Ordering::Relaxed);
park.1.notify_all();
let (result, elapsed) = match joined {
Some(v) => v,
None => {
// Wait a bit longer for the thread to drain after release so we
// don't leave it dangling, then fail the test.
std::thread::sleep(Duration::from_millis(500));
panic!(
"Disc::copy did not return within 5s of stall_secs=2 — \
stall guard not wired (TDD red until fix lands)"
);
}
..Default::default()
};
let t0 = Instant::now();
let result = disc
.copy(&mut reader, &iso_path, &opts)
.expect("copy returns Ok");
let elapsed = t0.elapsed();
// Cleanup
let _ = std::fs::remove_file(&iso_path);
let _ = std::fs::remove_file(libfreemkv::disc::mapfile_path_for(&iso_path));
let copy_result = result.expect("copy returns Ok with stall handling");
// Hard bound — even at 0 ms per read, 1024 sectors with skip-forward
// should complete in well under a second on any host.
assert!(
elapsed < Duration::from_secs(5),
"Pass 1 took {elapsed:?} on a 2 MB synthetic disc — expected < 5 s"
);
// Per RIP_DESIGN.md §2.1: Pass 1 must reach end of disc regardless of
// read outcomes.
assert_eq!(
result.bytes_total, total_bytes,
"bytes_total must match disc capacity"
);
assert_eq!(
result.bytes_good, 0,
"no reads succeeded, bytes_good must be 0"
);
assert_eq!(
result.bytes_unreadable, 0,
"Pass 1 does not mark Unreadable; only Pass 2 (Disc::patch) does"
);
assert_eq!(
result.bytes_pending, total_bytes,
"every sector must be NonTrimmed → counted as pending. \
Got bytes_pending={} of total {}",
result.bytes_pending, total_bytes
);
assert!(
!result.complete,
"complete=false because NonTrimmed regions remain (work for Pass 2)"
);
assert!(!result.halted, "no halt was set; halted must be false");
// ISO file should be the full disc size on disk (sparse zeros where
// reads failed).
// Note: tempfile was dropped above; the file may or may not still exist
// depending on cleanup ordering. We only assert what we can observe in
// the CopyResult.
}
// ── 7. Halt during Pass 1 skip-forward path returns promptly (deterministic) ─
//
// Per RIP_DESIGN.md §3: halt is the only legitimate early exit from Pass 1.
// Even when every read is failing (skip-forward path), a halt must be
// honored within a small bounded time.
//
// Deterministic fixture: the reader signals halt on its FIRST read. The
// inner copy loop's halt check fires on the next iteration, breaking out
// of 'outer. This avoids any wallclock race on fast CI runners (where a
// 2 GB synthetic disc can sweep skip-forward in <100 ms).
#[test]
fn test_disc_copy_halts_promptly_on_failing_reader() {
let capacity_sectors: u32 = 1024 * 1024; // 2 GB synthetic disc
let halt = Arc::new(AtomicBool::new(false));
let mut reader = FailingSectorReader::with_halt_on_first_read(capacity_sectors, halt.clone());
let disc = synthetic_disc(capacity_sectors);
let tmp = tempfile::NamedTempFile::new().expect("tempfile create");
let iso_path = tmp.path().to_path_buf();
drop(tmp);
let opts = CopyOptions {
decrypt: false,
multipass: true,
halt: Some(halt),
..Default::default()
};
let t0 = Instant::now();
let result = disc
.copy(&mut reader, &iso_path, &opts)
.expect("copy returns Ok on halt");
let elapsed = t0.elapsed();
// Cleanup
let _ = std::fs::remove_file(&iso_path);
let _ = std::fs::remove_file(libfreemkv::disc::mapfile_path_for(&iso_path));
assert!(
elapsed < Duration::from_millis(5000),
"copy elapsed {elapsed:?} exceeded 5s bound (stall_secs=2)"
elapsed < Duration::from_secs(2),
"halt must return within 2 s; took {elapsed:?}"
);
assert!(result.halted, "result.halted must be true");
assert!(
!result.complete,
"halted run cannot be complete (bytes_pending > 0 expected)"
);
assert!(
copy_result.bytes_pending > 0,
"expected bytes_pending > 0 after stall-triggered skip; got {}",
copy_result.bytes_pending
result.bytes_pending > 0,
"halt fired before sweep completed; bytes_pending must be > 0"
);
}
// ── 8. Hysteresis recovers data the drive can read individually ──────────
//
// Pass 1 reads in batch (32 sectors = 1 ECC block). Failed blocks are marked
// NonTrimmed for Pass 2 recovery. This test verifies that a reader where every
// multi-sector read fails produces all NonTrimmed output with zero bytes_good.
struct BlockSizeFailingReader {
capacity: u32,
}
impl SectorReader for BlockSizeFailingReader {
fn read_sectors(
&mut self,
lba: u32,
count: u16,
buf: &mut [u8],
_recovery: bool,
) -> Result<usize> {
if count == 1 {
for chunk in buf.chunks_mut(SECTOR_SIZE) {
chunk.fill((lba & 0xff) as u8);
}
Ok(buf.len())
} else {
Err(libfreemkv::error::Error::ScsiError {
opcode: libfreemkv::scsi::SCSI_READ_10,
status: libfreemkv::scsi::SCSI_STATUS_CHECK_CONDITION,
sense: Some(libfreemkv::ScsiSense {
sense_key: libfreemkv::scsi::SENSE_KEY_MEDIUM_ERROR,
asc: 0x11,
ascq: 0x00,
}),
})
}
}
fn capacity(&self) -> u32 {
self.capacity
}
}
#[test]
fn test_disc_copy_marks_failed_ecc_blocks_as_nontrimmed() {
let capacity_sectors: u32 = 256;
let _total_bytes: u64 = capacity_sectors as u64 * SECTOR_SIZE as u64;
let mut reader = BlockSizeFailingReader {
capacity: capacity_sectors,
};
let disc = synthetic_disc(capacity_sectors);
let tmp = tempfile::NamedTempFile::new().expect("tempfile create");
let iso_path = tmp.path().to_path_buf();
drop(tmp);
let opts = CopyOptions {
decrypt: false,
multipass: true,
..Default::default()
};
let result = disc
.copy(&mut reader, &iso_path, &opts)
.expect("copy returns Ok");
let _ = std::fs::remove_file(&iso_path);
let _ = std::fs::remove_file(libfreemkv::disc::mapfile_path_for(&iso_path));
// Pass 1 reads every batch at bpt=32 (no batch reduction, no skip-ahead).
// BlockSizeFailingReader fails on all batch=32 reads, so every sector
// is marked NonTrimmed. bytes_good=0 is correct — Pass 2 recovers them.
assert_eq!(
result.bytes_good, 0,
"Pass 1 at bpt=32 should not recover any sectors when all batches fail"
);
assert!(
result.bytes_pending > 0,
"all sectors should be NonTrimmed pending Pass 2"
);
assert!(
!result.complete,
"complete=false when sectors remain NonTrimmed"
);
}
// ── 9. PassProgress carries separate unreadable vs pending byte counts ─────
//
// The video-damage-time display needs bytes_unreadable_total (confirmed dead)
// separate from bytes_pending_total (might still recover). This test verifies
// that a Pass 2 with some confirmed failures produces correct field values.
#[test]
fn test_pass_progress_separates_unreadable_from_pending() {
let capacity_sectors: u32 = 128;
let total_bytes: u64 = capacity_sectors as u64 * SECTOR_SIZE as u64;
let mut reader = FailingSectorReader::new(capacity_sectors);
let disc = synthetic_disc(capacity_sectors);
let tmp = tempfile::NamedTempFile::new().expect("tempfile create");
let iso_path = tmp.path().to_path_buf();
drop(tmp);
let opts = CopyOptions {
decrypt: false,
multipass: true,
..Default::default()
};
let pass1 = disc.copy(&mut reader, &iso_path, &opts).expect("pass1 ok");
assert_eq!(pass1.bytes_good, 0, "pass1: no good sectors");
assert_eq!(pass1.bytes_unreadable, 0, "pass1: no confirmed unreadable");
assert_eq!(
pass1.bytes_pending, total_bytes,
"pass1: all sectors NonTrimmed"
);
let last_unreadable = Arc::new(AtomicU64::new(0));
let last_pending = Arc::new(AtomicU64::new(0));
let last_good = Arc::new(AtomicU64::new(0));
let last_dur = Arc::new(AtomicU64::new(0));
struct SnapshotReporter {
unreadable: Arc<AtomicU64>,
pending: Arc<AtomicU64>,
good: Arc<AtomicU64>,
dur: Arc<AtomicU64>,
}
impl libfreemkv::progress::Progress for SnapshotReporter {
fn report(&self, p: &libfreemkv::progress::PassProgress) -> bool {
self.unreadable
.store(p.bytes_unreadable_total, Ordering::Relaxed);
self.pending.store(p.bytes_pending_total, Ordering::Relaxed);
self.good.store(p.bytes_good_total, Ordering::Relaxed);
if let Some(d) = p.disc_duration_secs {
self.dur.store((d * 1000.0) as u64, Ordering::Relaxed);
}
true
}
}
let reporter = SnapshotReporter {
unreadable: last_unreadable.clone(),
pending: last_pending.clone(),
good: last_good.clone(),
dur: last_dur.clone(),
};
let pass2_opts = CopyOptions {
decrypt: false,
multipass: true,
progress: Some(&reporter),
..Default::default()
};
let pass2 = disc
.copy(&mut reader, &iso_path, &pass2_opts)
.expect("pass2 ok");
let _ = std::fs::remove_file(&iso_path);
let _ = std::fs::remove_file(libfreemkv::disc::mapfile_path_for(&iso_path));
assert_eq!(
pass2.bytes_good, 0,
"pass2: still no good sectors (reader always fails)"
);
assert!(
pass2.bytes_unreadable > 0,
"pass2: some sectors confirmed unreadable"
);
assert!(
pass2.bytes_pending < pass1.bytes_pending,
"pass2: fewer pending sectors than pass1"
);
let observed_unreadable = last_unreadable.load(Ordering::Relaxed);
let observed_pending = last_pending.load(Ordering::Relaxed);
assert!(
observed_unreadable > 0,
"progress should report confirmed unreadable bytes"
);
assert!(
observed_pending == 0 || observed_pending < total_bytes,
"pending should shrink as sectors are confirmed unreadable"
);
// Video damage time: unreadable / total * duration
// With no titles on synthetic disc, disc_duration_secs = None
assert_eq!(
last_dur.load(Ordering::Relaxed),
0,
"synthetic disc has no titles, duration should be None/0"
);
}
// ── 10. Damage time calculation (unit test) ────────────────────────────────
//
// Verifies the formula: damage_secs = bytes_unreadable / bytes_total * duration
// This mirrors the CLI's print_disc_progress logic.
#[test]
fn test_damage_time_calculation() {
// 78.8 GB disc, 2h45m movie (9900s), 74 KB unreadable
let disc_bytes: u64 = 78_800_000_000;
let duration_secs: f64 = 9900.0;
let cases: Vec<(u64, &str)> = vec![
(74 * 1024, "~10ms"), // 74 KB → ~9ms, negligible
(10 * 1024 * 1024, "~1.3s"), // 10 MB → ~1.3s
(100 * 1024 * 1024, "~13s"), // 100 MB → ~13s
(1024 * 1024 * 1024, "~134s"), // 1 GB → ~134s
];
for (bad_bytes, label) in cases {
let damage_secs = bad_bytes as f64 / disc_bytes as f64 * duration_secs;
match label {
"~10ms" => assert!(damage_secs < 0.05, "{label}: {damage_secs:.3}s"),
"~1.3s" => assert!(
(damage_secs - 1.3).abs() < 0.2,
"{label}: {damage_secs:.2}s"
),
"~13s" => assert!(
(damage_secs - 13.0).abs() < 1.0,
"{label}: {damage_secs:.1}s"
),
"~134s" => assert!(
(damage_secs - 134.0).abs() < 2.0,
"{label}: {damage_secs:.0}s"
),
_ => {}
}
}
// 0.25s threshold: how many bad bytes = 0.25s of damage?
let threshold_bytes = (0.25 / duration_secs * disc_bytes as f64) as u64;
assert!(
threshold_bytes > 0,
"0.25s damage threshold should be > 0 bytes"
);
// At 9900s / 78.8 GB ≈ 0.25s = ~2 MB
let expected_mb = threshold_bytes as f64 / (1024.0 * 1024.0);
assert!(
!copy_result.complete,
"expected complete=false after stall-triggered skip"
(expected_mb - 2.0).abs() < 0.5,
"0.25s ≈ {expected_mb:.2} MB (expected ~2 MB)"
);
}
+394
View File
@@ -0,0 +1,394 @@
//! Integration tests for the SCSI error-decoding contract.
//!
//! v0.13.20 rewrote `scsi/linux.rs` to a synchronous blocking SG_IO and
//! consolidated sense parsing into the `parse_sense` helper that every
//! platform backend now shares. v0.13.23 replaced the `Error::ScsiError`
//! flat-fields shape with `{ opcode, status, sense: Option<ScsiSense> }`
//! so callers can route on structured sense data (key + ASC + ASCQ) via
//! [`Error::scsi_sense`] / [`Error::is_marginal_read`] /
//! [`ScsiSense::is_*`].
//!
//! The actual `ioctl(SG_IO, ...)` call is impossible to mock without a
//! kernel — see
//! `(internal)/docs/audits/2026-04-26-scsi-architecture-research.md`
//! for why the audit recommends against libc shims here. These tests
//! therefore pin the *contract* every backend must satisfy via a mock
//! `ScsiTransport`:
//!
//! 1. Healthy result → `Ok(ScsiResult { bytes_transferred = data.len() - resid })`.
//! 2. Transport-level failure (no SCSI status delivered: kernel
//! timeout, USB bridge wedge, IOKit service error) →
//! `Error::ScsiError { status: SCSI_STATUS_TRANSPORT_FAILURE, sense: None }`.
//! `Error::is_scsi_transport_failure()` returns `true`. Used by
//! `drive_has_disc` to detect the wedge signature.
//! 3. SCSI-level failure (drive replied CHECK CONDITION with sense) →
//! `Error::ScsiError { status: 0x02, sense: Some(ScsiSense {…}) }`
//! with the parsed key/ASC/ASCQ.
//! 4. `Error::is_marginal_read()` is `true` for MEDIUM ERROR /
//! ABORTED COMMAND / RECOVERED ERROR / NO SENSE; `false` for
//! HARDWARE / DATA PROTECT / UNIT ATTENTION / NOT READY / ILLEGAL
//! REQUEST and for transport failures.
//!
//! Inline `parse_sense_tests` in `src/scsi/mod.rs` cover the pure parse
//! logic (descriptor 0x72/0x73 vs fixed 0x70/0x71, short-buffer, VALID
//! bit masking, unknown response codes, ASC/ASCQ offsets); this file
//! covers the consumer side — a real transport feeding a real Error
//! variant to a real call site (`scsi::inquiry`).
use libfreemkv::error::Error;
use libfreemkv::scsi::{
DataDirection, SCSI_STATUS_CHECK_CONDITION, SCSI_STATUS_TRANSPORT_FAILURE,
SENSE_KEY_ABORTED_COMMAND, SENSE_KEY_DATA_PROTECT, SENSE_KEY_HARDWARE_ERROR,
SENSE_KEY_ILLEGAL_REQUEST, SENSE_KEY_MEDIUM_ERROR, SENSE_KEY_NOT_READY,
SENSE_KEY_RECOVERED_ERROR, SENSE_KEY_UNIT_ATTENTION, ScsiResult, ScsiSense, ScsiTransport,
};
/// A scripted ScsiTransport. Each `execute()` consumes the next entry
/// from `script` and returns the corresponding outcome.
///
/// Outcomes mirror what each backend's `execute()` should produce after
/// the v0.13.23 sense plumbing: `Option<ScsiSense>` carrying the full
/// SPC-4 triple for drive-reported failures, `None` for transport-level
/// failures.
struct MockTransport {
script: Vec<MockOutcome>,
next: usize,
}
#[derive(Clone)]
enum MockOutcome {
/// Healthy completion. `data` is what the transport wrote into the
/// caller's data buffer (truncated to the buffer length); `resid`
/// is reported back as `data.len() - bytes_transferred`.
Ok { data: Vec<u8>, resid: i32 },
/// Transport-level failure: `hdr.host_status = DID_TIME_OUT` on
/// Linux, `kIOReturnError` on macOS, `DeviceIoControl` returning 0
/// on Windows. Backends synthesise `SCSI_STATUS_TRANSPORT_FAILURE`
/// with `sense = None`.
TransportFailure,
/// Drive replied with sense data (typically `SCSI_STATUS_CHECK_CONDITION`
/// + a populated sense buffer).
ScsiFailure { status: u8, sense: ScsiSense },
}
impl MockTransport {
fn new(script: Vec<MockOutcome>) -> Self {
Self { script, next: 0 }
}
}
impl ScsiTransport for MockTransport {
fn execute(
&mut self,
cdb: &[u8],
_direction: DataDirection,
data: &mut [u8],
_timeout_ms: u32,
) -> libfreemkv::error::Result<ScsiResult> {
let i = self.next;
self.next += 1;
let outcome = self
.script
.get(i)
.cloned()
.expect("MockTransport script ran out — test wrote fewer outcomes than calls");
match outcome {
MockOutcome::Ok { data: d, resid } => {
let n = d.len().min(data.len());
data[..n].copy_from_slice(&d[..n]);
let bytes = (data.len() as i32).saturating_sub(resid).max(0) as usize;
Ok(ScsiResult {
status: 0,
bytes_transferred: bytes,
sense: [0u8; 32],
})
}
MockOutcome::TransportFailure => Err(Error::ScsiError {
opcode: cdb[0],
status: SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
}),
MockOutcome::ScsiFailure { status, sense } => Err(Error::ScsiError {
opcode: cdb[0],
status,
sense: Some(sense),
}),
}
}
}
/// Helper — build a CHECK CONDITION outcome with a given sense key
/// and zero ASC/ASCQ. Most consumer-side tests only care about the key.
fn check_cond(sense_key: u8) -> MockOutcome {
MockOutcome::ScsiFailure {
status: SCSI_STATUS_CHECK_CONDITION,
sense: ScsiSense {
sense_key,
asc: 0,
ascq: 0,
},
}
}
// ── 1. Healthy read ───────────────────────────────────────────────────────
#[test]
fn test_healthy_inquiry_returns_ok_with_full_transfer() {
// Build a fake INQUIRY response (96 bytes per scsi::inquiry).
// Vendor "TEST-VND", model "TEST-MODEL ", firmware "1.00".
let mut payload = vec![0u8; 96];
payload[8..16].copy_from_slice(b"TEST-VND");
payload[16..32].copy_from_slice(b"TEST-MODEL ");
payload[32..36].copy_from_slice(b"1.00");
let mut transport = MockTransport::new(vec![MockOutcome::Ok {
data: payload,
resid: 0,
}]);
let r = libfreemkv::scsi::inquiry(&mut transport).expect("inquiry should succeed");
assert_eq!(r.vendor_id, "TEST-VND");
assert_eq!(r.model, "TEST-MODEL");
assert_eq!(r.firmware, "1.00");
}
// ── 2. Transport-level failure: no SCSI status, no sense data ─────────────
#[test]
fn test_transport_failure_surfaces_with_sense_none() {
let mut transport = MockTransport::new(vec![MockOutcome::TransportFailure]);
let err = libfreemkv::scsi::inquiry(&mut transport).unwrap_err();
assert!(
err.is_scsi_transport_failure(),
"TransportFailure must satisfy is_scsi_transport_failure()"
);
assert!(
err.scsi_sense().is_none(),
"transport failure has no sense data"
);
assert!(
!err.is_marginal_read(),
"transport failure must not be classified as marginal-read"
);
match err {
Error::ScsiError { status, sense, .. } => {
assert_eq!(status, SCSI_STATUS_TRANSPORT_FAILURE);
assert!(sense.is_none());
}
other => panic!("expected ScsiError, got {other:?}"),
}
}
// ── 3. CHECK CONDITION + ILLEGAL REQUEST ──────────────────────────────────
#[test]
fn test_check_cond_illegal_request_carries_sense() {
let mut transport = MockTransport::new(vec![check_cond(SENSE_KEY_ILLEGAL_REQUEST)]);
let err = libfreemkv::scsi::inquiry(&mut transport).unwrap_err();
let sense = err.scsi_sense().expect("CHECK CONDITION must carry sense");
assert_eq!(sense.sense_key, SENSE_KEY_ILLEGAL_REQUEST);
assert!(sense.is_illegal_request());
assert!(!err.is_marginal_read(), "ILLEGAL REQUEST is not marginal");
assert!(!err.is_scsi_transport_failure());
}
// ── 4. CHECK CONDITION + NOT READY (drive_has_disc relies on this) ────────
#[test]
fn test_check_cond_not_ready_predicate() {
let mut transport = MockTransport::new(vec![check_cond(SENSE_KEY_NOT_READY)]);
let err = libfreemkv::scsi::inquiry(&mut transport).unwrap_err();
let sense = err.scsi_sense().expect("CHECK CONDITION must carry sense");
assert!(sense.is_not_ready(), "sense_key 2 ⇒ is_not_ready");
assert!(
err.is_marginal_read(),
"NOT READY is marginal (BU40N returns this for bad sectors)"
);
}
// ── 5. CHECK CONDITION + MEDIUM ERROR (canonical marginal-read) ───────────
#[test]
fn test_check_cond_medium_error_is_marginal() {
let mut transport = MockTransport::new(vec![check_cond(SENSE_KEY_MEDIUM_ERROR)]);
let err = libfreemkv::scsi::inquiry(&mut transport).unwrap_err();
let sense = err.scsi_sense().expect("CHECK CONDITION must carry sense");
assert!(sense.is_medium_error());
assert!(sense.is_marginal());
assert!(
err.is_marginal_read(),
"MEDIUM ERROR is the canonical marginal-read signal"
);
}
// ── 6. CHECK CONDITION + ABORTED COMMAND (also marginal) ──────────────────
#[test]
fn test_check_cond_aborted_command_is_marginal() {
let mut transport = MockTransport::new(vec![check_cond(SENSE_KEY_ABORTED_COMMAND)]);
let err = libfreemkv::scsi::inquiry(&mut transport).unwrap_err();
let sense = err.scsi_sense().expect("CHECK CONDITION must carry sense");
assert!(sense.is_aborted_command());
assert!(err.is_marginal_read(), "ABORTED COMMAND is marginal");
}
// ── 7. CHECK CONDITION + RECOVERED ERROR (drive recovered; marginal) ──────
#[test]
fn test_check_cond_recovered_error_is_marginal() {
let mut transport = MockTransport::new(vec![check_cond(SENSE_KEY_RECOVERED_ERROR)]);
let err = libfreemkv::scsi::inquiry(&mut transport).unwrap_err();
assert!(
err.is_marginal_read(),
"RECOVERED ERROR is treated as marginal (drive recovered, retry-friendly class)"
);
}
// ── 8. CHECK CONDITION + HARDWARE ERROR (NOT marginal — bail) ─────────────
#[test]
fn test_check_cond_hardware_error_not_marginal() {
let mut transport = MockTransport::new(vec![check_cond(SENSE_KEY_HARDWARE_ERROR)]);
let err = libfreemkv::scsi::inquiry(&mut transport).unwrap_err();
let sense = err.scsi_sense().expect("CHECK CONDITION must carry sense");
assert!(sense.is_hardware_error());
assert!(
!err.is_marginal_read(),
"HARDWARE ERROR must not be marginal — drive failing, retry can't help"
);
}
// ── 9. CHECK CONDITION + DATA PROTECT (NOT marginal — bail) ───────────────
#[test]
fn test_check_cond_data_protect_not_marginal() {
let mut transport = MockTransport::new(vec![check_cond(SENSE_KEY_DATA_PROTECT)]);
let err = libfreemkv::scsi::inquiry(&mut transport).unwrap_err();
let sense = err.scsi_sense().expect("CHECK CONDITION must carry sense");
assert!(sense.is_data_protect());
assert!(
!err.is_marginal_read(),
"DATA PROTECT (AACS / region) must not be marginal — retry can't help"
);
}
// ── 10. CHECK CONDITION + UNIT ATTENTION (NOT marginal — caller rescans) ──
#[test]
fn test_check_cond_unit_attention_not_marginal() {
let mut transport = MockTransport::new(vec![check_cond(SENSE_KEY_UNIT_ATTENTION)]);
let err = libfreemkv::scsi::inquiry(&mut transport).unwrap_err();
let sense = err.scsi_sense().expect("CHECK CONDITION must carry sense");
assert!(sense.is_unit_attention());
assert!(
!err.is_marginal_read(),
"UNIT ATTENTION must not be marginal — caller should rescan, not retry"
);
}
// ── 11. ASC/ASCQ propagate from sense buffer to ScsiError.sense ───────────
#[test]
fn test_asc_ascq_round_trip_through_error() {
let outcome = MockOutcome::ScsiFailure {
status: SCSI_STATUS_CHECK_CONDITION,
sense: ScsiSense {
sense_key: SENSE_KEY_MEDIUM_ERROR,
asc: 0x11,
ascq: 0x05, // L-EC UNCORRECTABLE
},
};
let mut transport = MockTransport::new(vec![outcome]);
let err = libfreemkv::scsi::inquiry(&mut transport).unwrap_err();
let sense = err.scsi_sense().expect("must carry sense");
assert_eq!(sense.sense_key, SENSE_KEY_MEDIUM_ERROR);
assert_eq!(sense.asc, 0x11);
assert_eq!(sense.ascq, 0x05);
}
// ── 12. Healthy short transfer (resid > 0) ────────────────────────────────
#[test]
fn test_healthy_short_transfer_reports_partial_bytes() {
// Transport reported back fewer bytes than requested (resid = 16).
// For a 96-byte buffer that means bytes_transferred = 80.
let payload = vec![0u8; 96];
let mut transport = MockTransport::new(vec![MockOutcome::Ok {
data: payload,
resid: 16,
}]);
let cdb = [libfreemkv::scsi::SCSI_INQUIRY, 0, 0, 0, 0x60, 0];
let mut buf = [0u8; 96];
let r = transport
.execute(&cdb, DataDirection::FromDevice, &mut buf, 1_000)
.expect("ok");
assert_eq!(r.status, 0);
assert_eq!(
r.bytes_transferred, 80,
"bytes_transferred must equal data.len() - resid"
);
}
// ── 13. Error::Display does not leak English in the SCSI variant ──────────
//
// Library rule (project docs): no English in error display, only "E{code}"
// + structured data. Regression guard for the SCSI variant specifically
// because it's the most-emitted error in the rip path.
#[test]
fn test_scsi_error_display_format_is_codes_only() {
let err = Error::ScsiError {
opcode: 0x12,
status: SCSI_STATUS_CHECK_CONDITION,
sense: Some(ScsiSense {
sense_key: SENSE_KEY_ILLEGAL_REQUEST,
asc: 0x24,
ascq: 0x00,
}),
};
let s = err.to_string();
assert!(
s.starts_with("E4000:"),
"ScsiError must lead with E4000: {s}"
);
assert!(
s.contains("0x12") && s.contains("0x02") && s.contains("0x05") && s.contains("0x24"),
"ScsiError must show opcode/status/key/asc in hex: {s}"
);
// Crude English filter — same as the inline error.rs::display test.
for word in s.split(|c: char| !c.is_ascii_alphabetic()) {
assert!(
word.len() <= 4,
"ScsiError display contains suspicious word `{word}`: {s}"
);
}
}
// ── 14. Transport-failure Display omits sense fields ──────────────────────
#[test]
fn test_scsi_transport_failure_display_short_form() {
let err = Error::ScsiError {
opcode: 0x28,
status: SCSI_STATUS_TRANSPORT_FAILURE,
sense: None,
};
let s = err.to_string();
assert!(s.starts_with("E4000:"));
assert!(s.contains("0x28") && s.contains("0xff"));
// No sense triple should appear in the no-sense form.
assert!(
!s.contains("0x00/0x00/0x00"),
"transport failure must not carry phantom sense: {s}"
);
}
+115
View File
@@ -0,0 +1,115 @@
//! Tests for SgIoTransport timeout recovery (Fix 2).
//!
//! When `execute()` detects a transport-level failure (kernel timeout /
//! USB bridge wedge — `host_status != 0`), it spawns two background
//! threads to (1) close the old fd and (2) open a fresh one, then
//! stores the new fd in `fd_recovery`. The next `execute()` call picks
//! up the recovered fd without blocking on close().
//!
//! These tests require a real /dev/sg* device and are therefore #[ignore].
use std::path::Path;
use std::time::Duration;
#[test]
#[ignore]
fn test_sgio_transport_timeout_does_not_kill_transport() {
let device = "/dev/sg2";
let device = std::env::var("FREEMKV_TEST_SG_DEVICE").unwrap_or(device.to_string());
let _path = Path::new(&device);
#[cfg(target_os = "linux")]
{
use libfreemkv::scsi::ScsiTransport;
use libfreemkv::scsi::linux::SgIoTransport;
use libfreemkv::scsi::{DataDirection, SCSI_STATUS_TRANSPORT_FAILURE};
use std::sync::Arc;
use std::sync::atomic::{AtomicI32, Ordering};
let mut transport = SgIoTransport::open(_path).expect("open device");
let fd_before = transport.fd;
// READ_10 with 1 ms timeout to force kernel timeout.
let cdb = [0x28, 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x00, 0x00];
let mut data = vec![0u8; 2048];
let start = std::time::Instant::now();
let result = transport.execute(&cdb, DataDirection::FromDevice, &mut data, 1);
let _elapsed = start.elapsed();
assert!(result.is_err(), "expected Err on timeout, got {:?}", result);
let err = result.unwrap_err();
assert!(
matches!(&err, libfreemkv::Error::ScsiError { status, .. } if *status == SCSI_STATUS_TRANSPORT_FAILURE),
"expected ScsiError(TRANSPORT_FAILURE), got {:?}",
err
);
assert_eq!(transport.fd, -1, "fd should be -1 after timeout");
// Wait for recovery thread to store new fd.
let recovered = Arc::new(AtomicI32::new(-1));
let recovery_ref = transport.fd_recovery.clone();
for _ in 0..100 {
let v = recovery_ref.load(Ordering::Acquire);
if v >= 0 {
recovered.store(v, Ordering::Release);
break;
}
std::thread::sleep(Duration::from_millis(50));
}
let new_fd = recovered.load(Ordering::Acquire);
assert!(new_fd >= 0, "recovery thread should have produced a new fd");
// Next execute() must pick up recovered fd quickly.
let mut data2 = vec![0u8; 2048];
let start2 = std::time::Instant::now();
let result2 = transport.execute(&cdb, DataDirection::FromDevice, &mut data2, 5000);
let elapsed2 = start2.elapsed();
assert!(
result2.is_ok(),
"execute() with recovered fd should succeed, got {:?}",
result2
);
assert!(
elapsed2 < Duration::from_secs(2),
"should return quickly, took {:?}",
elapsed2
);
assert_ne!(transport.fd, -1, "fd should be valid after recovery");
assert_ne!(transport.fd, fd_before, "fd should be fresh after recovery");
}
#[cfg(not(target_os = "linux"))]
{
eprintln!("SKIP: test requires Linux / SgIoTransport");
}
}
#[test]
#[ignore]
fn test_drive_read_per_cdb_timeout_bounds_call() {
let device = "/dev/sg2";
let device = std::env::var("FREEMKV_TEST_SG_DEVICE").unwrap_or(device.to_string());
let _path = Path::new(&device);
#[cfg(target_os = "linux")]
{
let mut drive = libfreemkv::Drive::open(_path).expect("open drive");
let timeout_ms: u32 = 5_000;
let start = std::time::Instant::now();
let _ = drive.read(0, 1, &mut [0u8; 2048], false);
let elapsed = start.elapsed();
let overhead = Duration::from_millis(500);
assert!(
elapsed < Duration::from_millis(timeout_ms as u64) + overhead,
"Drive::read should return within timeout_ms + overhead, took {:?}",
elapsed
);
}
#[cfg(not(target_os = "linux"))]
{
eprintln!("SKIP: test requires Linux / SgIoTransport");
}
}