Fix extract_scsi_context() and Error::scsi_sense() to handle Error::DiscRead
in addition to Error::ScsiError, so is_marginal_read() works for DiscRead
errors and disc::copy() can properly route MEDIUM ERROR as a marginal
(bad sector) instead of bailing.
Closes: freemkv-private#20260427
bytes_pending was an opaque aggregate of NonTried + NonTrimmed +
NonScraped. UIs that wanted a "will retry in Pass 2-N" bucket were
stuck showing the entire unread disc as Maybe at pct=0.
Adds two granular fields to MapStats:
bytes_nontried — Pass 1 hasn't read these yet
bytes_retryable — NonTrimmed + NonScraped, Pass 2-N will retry
bytes_pending stays for back-compat (= bytes_nontried + bytes_retryable).
Also picks up the cargo fmt --check lint that's been red on main CI
since v0.13.18 (rustfmt fold differences on a few long format-string
layouts; functional no-op).
Through the entire 0.13.x line, every CHECK CONDITION reply from the
drive (the standard way SCSI tells you why a sector failed) was being
collapsed into a synthetic status=0xFF, sense_key=0 transport-wedge
sentinel and the actual sense data was thrown away. Confirmed live on
the BU40N reading Dune 2 on 2026-04-27: drive returned host_status=0,
driver_status=8, status=2, exec_elapsed_ms=1416 on every bad sector
— a clean CHECK CONDITION carrying full sense data — and Disc::copy
was bailing on it as if the bridge had wedged.
Root cause: scsi/linux.rs's wedge check was
`host_status != 0 || driver_status != 0`
SG's DRIVER_SENSE bit (0x08) is set on every CHECK CONDITION reply
just to flag "sense buffer is populated" — it's not a transport
failure on its own. Pre-fix we conflated the two and silently lost
every drive-reported error reason. macOS and Windows backends had
the same shape: they extracted sense_key only, dropping ASC/ASCQ.
API restructure (clean separation):
Error::ScsiError {
opcode: u8,
status: u8, // 0xFF = synthetic transport-failure
sense: Option<ScsiSense>, // None ⇔ no sense delivered
}
pub struct ScsiSense { sense_key: u8, asc: u8, ascq: u8 }
impl ScsiSense {
pub fn is_marginal(&self) -> bool // keys 0/1/3/B
pub fn is_medium_error(&self) -> bool
pub fn is_hardware_error(&self) -> bool
pub fn is_unit_attention(&self) -> bool
pub fn is_data_protect(&self) -> bool
pub fn is_not_ready(&self) -> bool
pub fn is_illegal_request(&self) -> bool
pub fn is_aborted_command(&self) -> bool
}
impl Error {
pub fn scsi_sense(&self) -> Option<&ScsiSense>
pub fn is_scsi_transport_failure(&self) -> bool
pub fn is_marginal_read(&self) -> bool
}
SCSI protocol constants (SCSI_STATUS_*, SENSE_KEY_*) moved from
error.rs to scsi/mod.rs where they belong alongside SCSI_INQUIRY,
SCSI_READ_10, etc. parse_sense replaces parse_sense_key (returns the
full triple, not just the key); inline tests now exercise ASC/ASCQ
extraction at the right offsets for both descriptor (0x72/0x73) and
fixed (0x70/0x71) sense formats.
Disc::copy + Disc::patch sense-aware dispatch:
- marginal sense (MEDIUM ERROR / ABORTED COMMAND / RECOVERED ERROR
/ NO SENSE) → engage hysteresis (Block→Single, bpt=1)
- non-marginal sense (HARDWARE / DATA PROTECT / UNIT ATTENTION /
NOT READY / ILLEGAL REQUEST / transport failure / kernel
IoError) → bail with full sense info preserved; caller (autorip)
surfaces "physical replug" / "drive failing" / "media changed"
Pre-fix: every CHECK CONDITION → 0xFF synthetic → Disc::copy bailed
→ bytes_good froze at the bad zone. The hysteresis from v0.13.22
was correct but never got to run. This release unblocks it.
Disc::patch's wedged_threshold (50 consecutive failures) stays as
defense-in-depth for chains of marginal failures; a single
non-marginal sense now short-circuits it.
New phase=bail trace event records the bail reason with the sense
triple. phase=transport_err remains for genuine bridge wedges /
kernel timeouts; phase=scsi_err carries the parsed sense_key, asc,
ascq for drive-reported errors.
All 350 tests pass. Clippy clean across all targets.
Fixes the BU40N wedge cycle that has been chasing us through
v0.13.18-20. Two changes, both backed by empirical live-hardware
probes recorded in freemkv-private/docs/TEST_PLAN.md:
1. scsi/mod.rs: READ_TIMEOUT_MS 1500 → 10000 ms.
Cold-start seek on the BU40N takes ~1.5 s. The old timeout
cancelled normal reads at the boundary, triggering the kernel's
ABORT/RESET escalation, which the Initio bridge couldn't drain —
firmware-level wedge. 10 s catches every legitimate slow read
(max successful ECC recovery: 2.6 s; cold-start: 1.5 s) with
margin and short-circuits truly bad sectors at ~10 s.
2. disc/mod.rs: Disc::copy bisect-on-fail (replaces skip-forward).
Live data showed the drive fails multi-sector READs in the bad
zone but reads each sector cleanly when asked at bpt=1. Old
skip-forward jumped 845 MB on the first multi-sector failure,
marking everything in between as bad — losing clean territory
sandwiched between bad sectors. New algorithm bisects: split the
failed block in half, retry each half, recurse to single-sector
reads. Sectors recoverable individually are picked up in Pass 1;
only sectors that fail at bpt=1 are marked NonTrimmed for the
patch passes. Stack-based DFS, log2(batch) = 6 levels for the
default 60-sector batch.
Multi-pass machinery is untouched. Pass 2..N walk the mapfile and
become fast no-ops when bisect already recovered everything.
Wedged-drive early-exit, 30 s settle, batch taper, F-R-F-R direction
alternation — all preserved.
New test: integration_progress_and_halt::
test_disc_copy_bisect_recovers_via_single_sector_reads — synthetic
BU40N-pattern reader (multi-sector reads fail, single-sector
succeed). Pre-patch: lost everything to skip-forward. Post-patch:
100 % bytes_good. Plus the 10 sense-key parser tests from the
0.13.20 test-coverage pass.
Empirical recovery on Dune 2 UHD on the BU40N (per TEST_PLAN.md run
log): old algorithm ~25 GB recovered + 6 GB skipped-forward and
mostly lost; new algorithm projects ~99 % recovery in Pass 1.
Audits + raw probe data:
- freemkv-private/docs/TEST_PLAN.md (run log)
- freemkv-private/docs/audits/2026-04-26-scsi-architecture-research.md
- scsi/linux.rs: full rewrite from async write/poll/read+1.5s timeout+
close-on-timeout to one synchronous ioctl(fd, SG_IO, &hdr). Kernel
honors hdr.timeout and runs its own ABORT/RESET escalation. Errors
check host_status and driver_status (both 0xFF-synthesised) plus
status. Sense-key parser handles descriptor (0x72/0x73) + fixed
(0x70/0x71) formats. Deleted fd_recovery, bg close+open thread, fd
swap dance. -331/+155 lines.
- scsi/macos.rs: try_recover() removed (userspace handle-recovery on
task failure was the same anti-pattern stripped from Linux). bsd_name
field deleted. Errors bubble up directly.
- scsi/windows.rs: try_recover() removed, wide_path field deleted,
INVALID_HANDLE guard removed.
- scsi/mod.rs: parse_sense_key() helper extracted (used by all three
platforms now — single canonical sense-key parse rather than three
inlined copies). +10 unit tests covering descriptor format, fixed
format, truncated buffers, unknown response codes.
- drive/mod.rs: Drive::reset() deleted (escalating eject + STOP/START +
reinit recovery — per audit, kernel handles its own escalation;
userspace shouldn't).
pub fn find_drives() -> Vec<Drive> deleted (opened N drives just to
throw most away). find_drive() now uses discover_drives() directly.
wait_ready() simplified — drops the reset path on sense_key=5,
just keeps polling TUR for 60 iterations.
- lib.rs: find_drives re-export removed.
- benches/sgio_read.rs: switched to find_drive() (no longer iterates a
drive list).
Net: 9 files changed, 226 insertions(+), 473 deletions(-). 329 tests
pass, clippy -D warnings clean. No consumer breakage (CLI, autorip,
bdemu compile + test green).
Architecture decision documented in
freemkv-private/docs/audits/2026-04-26-scsi-architecture-research.md
(primary-source survey of MakeMKV, sg_dd, ddrescue, and the kernel
mid-layer's own scsi_eh.rst escalation ladder).
Drive::read is now single-shot. Phase 1/2/3 retries + scsi::reset+reopen
removed (~80 lines). recovery=true bumps timeout to 30s; recovery=false
stays at 1.5s. On any failure returns Err(DiscRead) immediately — caller
(Disc::patch outer loop, DiscStream batch halver) handles retries.
Inline reset+reopen WAS the wedge primitive on the LG BU40N. Per prior
post-mortem, every USB/SCSI reset path tested fails to recover the
wedged Initio bridge — the inline retry was pure cost.
SgIoTransport::reset (Linux) trimmed to kernel SG_IO state flush +
ALLOW MEDIUM REMOVAL. SG_SCSI_RESET ioctl + STOP/START UNIT escalation
removed. macOS reset removed (no-op). scsi::reset() top-level family
removed (no callers).
EventKind::BytesRead { bytes, total } now actually emitted from
DiscStream::fill_extents after each successful sector read. Was
declared in 0.13.0, never fired. Drives autorip per-device progress
in direct mode.
EventKind::Retry / SectorRecovered no longer emitted (variants kept
for forward compat). SpeedChange still emitted via Drive::set_speed
public path.
Tests: new tests/integration_progress_and_halt.rs (5 tests). 233 unit
tests + 5 integration green.
Architectural cleanup. autorip + freemkv CLI were reimplementing drive
discovery (sysfs walking, type-5 filtering, sg-path construction) and
calling SCSI reset primitives directly. All of that hardware-aware code
moves into libfreemkv with two cheap public probes:
- DriveInfo + list_drives() — multi-OS enumeration (Linux/macOS/Windows)
with peripheral-type-5 filtering and INQUIRY identity. Cheap.
- drive_has_disc(path) — single TUR with internal wedge recovery
escalation (SCSI reset → USB reset → retry) hidden from callers.
USB-layer reset (USBDEVFS_RESET / IOUSBDeviceInterface::ResetDevice /
storport's combined reset) wired across all three platforms.
Visibility tightening — scsi::reset, scsi::usb_reset, and the timeout
constants are now pub(crate). Compile-time guarantee that no consumer
crate can issue SCSI commands directly.
233 lib tests pass; clippy clean.
Audit pass against the CLAUDE.md "no English text in library code" rule.
Found 9 call sites that violated the contract by stuffing English into
io::Error::new(kind, "…") or by abusing Error::DeviceNotFound { path }
as a free-form description field. Each is now a typed Error variant.
New variants and codes: ScsiInterfaceUnavailable (E1004), DeviceLocked
(E1005), IoKitPluginFailed (E1006), UnsupportedPlatform (E2003),
PlatformNotImplemented (E2004), MapfileInvalid (E6011), DiscUrlNotDirect
(E9009).
labels::apply() previously pushed Commentary/Descriptive/Score/IME and
" (Secondary)" English literals into AudioStream.label, leaking into
MKV titles + autorip UI. AudioStream now exposes structured `purpose:
LabelPurpose`, SubtitleStream `qualifier: LabelQualifier`. Callers
translate to localized text. label keeps codec-formatting only.
API hygiene: 11 mux/* modules dropped from `pub` to `pub(crate)` —
their *types* are still re-exported from lib.rs, but the modules were
leaking low-level EBML/TS/network primitives. Stream trait gets a real
rustdoc explaining read-vs-write split. lib.rs grouped re-exports into
documented sections. ScanOptions::with_keydb() removed (one-method-per-
action rule); use struct literal.
Dead-code sweep: removed lookahead.rs (orphan, never declared as mod),
tsreader.rs (TsDemuxReader unused), ebml::{write_int,read_vint,SEEK_*},
ts::{scan_first/last_pts,scan_duration,SCAN_HEAD/TAIL_SIZE,take/set_
remainder}, MkvMuxer codec_private_slots/filled fields and
fill_codec_private method (deferred-codecPrivate path never used since
the v0.10 PES rewrite). cargo clippy --all-targets -D warnings clean.
Tests: new error::tests for variant codes + Display "no English" guard +
io::ErrorKind mapping. 233 lib tests, all green (was 230).
Breaking: ScanOptions::with_keydb removed; mux/* modules pub(crate);
AudioStream and SubtitleStream gained required fields; UnsupportedDrive
{ product_revision: "Renesas not yet implemented" } no longer produced
(use PlatformNotImplemented).
Drive::read is now halt-check-free in its body. Previously, the halt
flag was checked in 4 places and the sleep logic was scattered across
4 "if halt_aware_sleep { return Halted }" call sites — correct, but
the ugliness invited drift: a new sleep added by someone unfamiliar
with the pattern would silently swallow Stop requests.
Two private primitives now own halt awareness:
checked_sleep(Duration) -> Result<()>
checked_exec(cdb, dir, buf, timeout_ms) -> Result<ScsiResult>
Both return Err(Halted) instead of a bool. The ? operator in read()
then propagates halts for free. The recovery path reads top-to-bottom
with no halt vocabulary.
sleep_until_halted lives as a free function so it's unit-testable
without a live Drive. 4 new tests: completes normally, bails on
pre-set flag within one slice, wakes mid-sleep, zero-duration no-op.
Public API unchanged — halt_flag/halt/clear_halt still exposed, the
refactor is entirely internal.
Stop was waiting up to 30 s to register when the drive hit L-EC
recovery mid-read — the recovery phase does 30 s sleeps between
retries and the halt flag was only checked at the start of each sleep.
UI feels broken ("stop isnt working") even though the halt was set.
halt_aware_sleep breaks each wait into 100 ms slices and returns
early on halt. Applied to all 4 sleeps in the recovery path (both
30 s retry delays, both 5 s reset-phase delays).
Disc::copy fast pass (skip_on_error=true, recovery=false) was giving
the drive 5 s per 64 KB block. On structure-protected / marginal UHD
sectors the drive grinds L-EC for nearly the full budget per block,
pinning throughput at ~13 KB/s even though skip_forward would happily
skip past the region.
1500ms bounds the floor at ~43 KB/s/block. Recoverable sectors that
would have succeeded at 3-5 s get picked up on Disc::patch (pass 2+)
where recovery=true and the per-read budget is 30 s.
Architecture:
- One stream per format, bidirectional PES (read/write on same type)
- IsoStream merged into DiscStream (one type, any SectorReader)
- Disc::copy() for disc→ISO raw sector dump
- IOStream trait deleted, all byte-level Read/Write removed
- ContentReader/OpenDisc/open_title/open_input/open_output deleted
- CountingStream wrapper for progress tracking
Error codes:
- All io::Error English strings replaced with Error enum variants
- From<Error> for io::Error conversion
- Unused variants removed, new stream/mux variants added
Deleted: mkvout.rs, pesout.rs, isowriter.rs, mkv-muxer-plan.md
Updated: all docs, README stream table, CHANGELOG
238 tests, 0 clippy warnings.
- Remove unused pes_buf field from M2tsStream and unused TS_PACKET/BD_TS_PACKET constants
- Replace match-with-single-pattern with if let (3 instances in drive/mod.rs)
- Replace match-can-be-? with ? operator for scsi::open call
- Add type aliases PesSetup and MkvHeaderResult to reduce type complexity
- Collapse identical if/else branches in tsmux.rs build_pes_header
- Use RangeInclusive::contains instead of manual range checks
- Make WriteSeek trait pub (was pub(crate) but leaked through pub fn)
- Remove empty line after doc comment in disc.rs
- Fix doc list item indentation in scsi/linux.rs (12 instances)
- Rename DriveSession → Drive across entire codebase
- find_drives() returns Vec<Drive>, find_drive() returns Option<Drive>
- resolve_device() now pub(crate) — internal only
- StreamUrl is now a typed enum (Disc, Mkv, M2ts, Iso, Network, Stdio, Null)
with scheme() and path_str() accessors, replacing struct of Strings
- Add lock_tray() / unlock_tray() for safe disc access during rips
- Improve reset() with eject cycle that clears LibreDrive stuck state
- Add Send bounds to ScsiTransport and PlatformDriver traits
- DiscOptions uses PathBuf instead of String for device/keydb paths
- Update doc example to use new Drive API
- DriveStatus enum: TrayOpen, NoDisc, DiscPresent, NotReady, Unknown
- drive_status(): GET EVENT STATUS NOTIFICATION with TUR fallback
- reset(): PREVENT ALLOW → START STOP → init() escalation
- wait_ready(): tries reset on Illegal Request, falls back to drive_status
- Remaining: standard READ(10) still fails in LibreDrive stuck state
- wait_ready: detects MMkv vendor probe when TUR returns Illegal Request
- scan: capacity fallback to 0 when READ CAPACITY fails
- Still needs: re-init to restore standard SCSI commands, or use raw reads for UDF
matthew
Matt Jackson