mirror of
https://github.com/freemkv/libfreemkv.git
synced 2026-10-07 18:13:10 +00:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
ca8ebf418f | ||
|
|
34182d956a | ||
|
|
0f967a2084 | ||
|
|
0341995c0d | ||
|
|
fe0ec0c5cf | ||
|
|
022657d481 | ||
|
|
fc8eca44e1 | ||
|
|
43836865be | ||
|
|
5e22199441 |
@@ -21,7 +21,7 @@ jobs:
|
||||
steps:
|
||||
- uses: actions/checkout@v5
|
||||
- uses: dtolnay/rust-toolchain@1.86.0
|
||||
- run: cargo test
|
||||
- run: cargo test --tests
|
||||
|
||||
check-macos:
|
||||
runs-on: macos-latest
|
||||
|
||||
+116
@@ -1,5 +1,121 @@
|
||||
# Changelog
|
||||
|
||||
## 0.13.9 (2026-04-25)
|
||||
|
||||
### Fix: Disc::copy silent stall + SgIoTransport reopen-after-timeout serialization
|
||||
|
||||
Two correlated fixes for a hang observed live on the LG BU40N during a
|
||||
v0.13.8 rip of Dune: Part Two. At ~30 % progress through Pass 1
|
||||
(disc → ISO), `bytes_good` froze for 10+ minutes with `errs=0`,
|
||||
no error surfaced, drive not wedged.
|
||||
|
||||
Root cause: `SgIoTransport::execute` (linux.rs) attempted to recover from
|
||||
a `poll()` timeout by spawning a background `close()` of the old fd and
|
||||
opening a fresh `/dev/sg*` fd on the main thread. On Linux, opening the
|
||||
SAME device while a prior fd is mid-close serializes via the kernel's
|
||||
per-device state lock — so the fresh `open()` blocks for as long as the
|
||||
close does (until the kernel completes the in-flight CDB). This undid
|
||||
the userspace 1.5 s timeout: each timed-out read added 60+ s to the
|
||||
next iteration. From `Disc::copy`'s perspective, reads kept returning
|
||||
Err slowly, the skip-forward path advanced `pos` but never `bytes_good`.
|
||||
|
||||
Fixes:
|
||||
- `SgIoTransport::execute` no longer reopens on timeout. Spawns the
|
||||
close, sets `self.fd = -1`, returns Err immediately. Subsequent
|
||||
calls fail with `DeviceNotFound` (already gated at line 248).
|
||||
Caller (Drive) is invalidated until reopened. Pass 2's
|
||||
`Disc::patch` would need a fresh Drive; that's a v0.14 follow-up.
|
||||
- `Disc::copy` adds a stall guard. New `CopyOptions::stall_secs:
|
||||
Option<u64>` (default 120 s). If `bytes_good` doesn't advance for
|
||||
the threshold, breaks the outer loop with `complete: false,
|
||||
bytes_pending > 0` so the caller's retry path picks up.
|
||||
|
||||
Tests: new `test_disc_copy_stall_detection_triggers_skip_forward` in
|
||||
`tests/integration_progress_and_halt.rs` proves the guard fires within
|
||||
the configured threshold.
|
||||
|
||||
Other:
|
||||
- Cosmetic: warning text "rip thread did not drain within 35s" updated
|
||||
to 60s (matches the v0.13.8 timeout bump).
|
||||
|
||||
## 0.13.8 (2026-04-25)
|
||||
|
||||
### Version sync — no functional changes
|
||||
|
||||
Sync bump for the ecosystem. 0.13.8 carries autorip-side fixes:
|
||||
post-stop "error" leak (halt-aware Err handling in Pass 1/2+),
|
||||
60 s drain timeout, and a structural spawn_rip_thread helper.
|
||||
|
||||
## 0.13.7 (2026-04-25)
|
||||
|
||||
### Version sync — no functional changes
|
||||
|
||||
Sync bump for the ecosystem. All four freemkv crates (libfreemkv,
|
||||
freemkv CLI, bdemu, autorip) always share a version number; 0.13.7
|
||||
carries an autorip-side fix (HTTP-spawned rip/scan threads now
|
||||
register for stop-drain).
|
||||
|
||||
## 0.13.6 (2026-04-25)
|
||||
|
||||
### Inline retry/reset stripped from `Drive::read`; `BytesRead` now emitted
|
||||
|
||||
Two related changes that close the loop on the BU40N wedge work from
|
||||
0.13.1–0.13.4 and on the long-standing autorip "0 KB/s, 0%" UI bug.
|
||||
|
||||
**`Drive::read` is now single-shot.** The phase 1 / 2 / 3 retry loop
|
||||
(reset → reopen → repeat) inside `Drive::read` is gone (~80 lines
|
||||
deleted). `recovery=true` only bumps the per-CDB timeout to 30 s;
|
||||
`recovery=false` keeps the 1.5 s timeout. On a failed read the
|
||||
function returns `Err(DiscRead)` immediately. Per the BU40N
|
||||
post-mortem, every USB / SCSI reset path tested in 0.13.1–0.13.3
|
||||
resets the bridge but not the drive firmware, and the inline
|
||||
reset+reopen *was* the wedge primitive itself — issuing it from
|
||||
inside `Drive::read` produced multi-minute hangs and made the wedge
|
||||
class harder to surface to the user. The correct retry layer is
|
||||
`Disc::patch`'s outer multi-pass loop, which is unaffected. A stuck
|
||||
drive now surfaces as a clean `DiscRead` to the caller, who can
|
||||
prompt physical replug.
|
||||
|
||||
**SCSI reset surface trimmed.** `SgIoTransport::reset` (Linux) drops
|
||||
the `SG_SCSI_RESET` ioctl and the STOP / START UNIT escalation; it
|
||||
keeps the kernel `SG_IO` state flush plus `ALLOW MEDIUM REMOVAL`.
|
||||
`MacScsiTransport::reset` is removed entirely (was open + drop +
|
||||
sleep, no SCSI). The top-level `scsi::reset` /
|
||||
`scsi::reset_with_timeout` / `scsi::reset_blocking` family is
|
||||
removed — no callers remain after the `Drive::read` strip.
|
||||
|
||||
**`EventKind::BytesRead` now emitted.** The variant was declared in
|
||||
0.13.0 but never fired. `DiscStream::fill_extents` now emits
|
||||
`BytesRead { bytes_read_total, total_extents_bytes }` after every
|
||||
successful sector read, so consumers in direct (no-mapfile) mode can
|
||||
drive a real-time progress bar without polling `output.bytes_written`.
|
||||
Multi-pass mode continues to use `Disc::copy`'s `on_progress`
|
||||
callback unchanged. Drives the autorip per-device live progress UI.
|
||||
|
||||
`Drive::checked_sleep` is removed (only used by the recovery loop);
|
||||
`Drive::sleep_until_halted` is `#[cfg(test)]`-only; `Drive::emit` is
|
||||
retained because `BytesRead` uses it.
|
||||
|
||||
### Tests
|
||||
- New `tests/integration_progress_and_halt.rs` (5 tests): `BytesRead`
|
||||
emission, `Disc::copy` `on_progress` regression guard, halt aborts
|
||||
copy, Drop safety, `FileSectorReader` round-trip.
|
||||
- 233 unit tests + 5 integration tests pass.
|
||||
|
||||
### Net diff
|
||||
~80 lines deleted, ~20 added.
|
||||
|
||||
### Version sync
|
||||
0.13.6 ecosystem release (libfreemkv + freemkv + bdemu + autorip all
|
||||
on 0.13.6).
|
||||
|
||||
## 0.13.5 (2026-04-25)
|
||||
|
||||
### Version sync — no functional changes
|
||||
Sync bump for the ecosystem. All four freemkv crates (libfreemkv,
|
||||
freemkv CLI, bdemu, autorip) always share a version number; 0.13.5
|
||||
carries autorip-side fixes (stop-is-reset, startup staging sweep).
|
||||
|
||||
## 0.13.4 (2026-04-25)
|
||||
|
||||
### Wedge recovery rolled back + sysfs identity fallback
|
||||
|
||||
+4
-1
@@ -1,6 +1,6 @@
|
||||
[package]
|
||||
name = "libfreemkv"
|
||||
version = "0.13.4"
|
||||
version = "0.13.9"
|
||||
edition = "2024"
|
||||
rust-version = "1.86"
|
||||
license = "AGPL-3.0-only"
|
||||
@@ -31,6 +31,9 @@ libc = "0.2"
|
||||
[target.'cfg(target_os = "macos")'.dependencies]
|
||||
libc = "0.2"
|
||||
|
||||
[dev-dependencies]
|
||||
tempfile = "3"
|
||||
|
||||
[[bench]]
|
||||
name = "sgio_read"
|
||||
harness = false
|
||||
|
||||
@@ -0,0 +1,20 @@
|
||||
# libfreemkv — local dev helper.
|
||||
# Mirrors the cross-crate scripts in (internal)/scripts/test-all.sh
|
||||
# but scoped to this single crate.
|
||||
|
||||
.PHONY: test build check ci clean
|
||||
|
||||
test:
|
||||
cargo test --tests
|
||||
|
||||
build:
|
||||
cargo build --release
|
||||
|
||||
check:
|
||||
cargo fmt --check
|
||||
cargo clippy --all-targets -- -D warnings
|
||||
|
||||
ci: check build test
|
||||
|
||||
clean:
|
||||
cargo clean
|
||||
@@ -18,7 +18,7 @@ Part of the [freemkv](https://github.com/freemkv) project.
|
||||
|
||||
```toml
|
||||
[dependencies]
|
||||
libfreemkv = "0.11"
|
||||
libfreemkv = "0.13"
|
||||
```
|
||||
|
||||
## Quick Start
|
||||
@@ -84,7 +84,7 @@ while result.bytes_unreadable + result.bytes_pending > 0 {
|
||||
- **Stream labels** — 5 BD-J format parsers (Paramount, Criterion, Pixelogic, CTRM, Deluxe)
|
||||
- **AACS decryption** — transparent key resolution and content decrypt (1.0 + 2.0 bus decryption)
|
||||
- **KEYDB updates** — download, verify, save from any HTTP URL (zero deps, raw TCP)
|
||||
- **Content reading** — adaptive batch reads with automatic decryption and error recovery
|
||||
- **Content reading** — adaptive batch reads with automatic decryption
|
||||
- **Stream I/O** — unified stream pipeline for reading and writing any format
|
||||
|
||||
### Streams
|
||||
@@ -92,21 +92,21 @@ while result.bytes_unreadable + result.bytes_pending > 0 {
|
||||
| Stream | Input | Output | Transport |
|
||||
|--------|-------|--------|-----------|
|
||||
| DiscStream | Yes | -- | Optical drive via SCSI |
|
||||
| IsoStream | Yes | Yes | Blu-ray ISO image file |
|
||||
| IsoStream | Yes | -- | Blu-ray ISO image file (read via stream pipeline; written via `Disc::copy()`) |
|
||||
| MkvStream | Yes | Yes | Matroska container |
|
||||
| M2tsStream | Yes | Yes | BD transport stream with FMKV metadata header |
|
||||
| NetworkStream | Yes (listen) | Yes (connect) | TCP with FMKV metadata header |
|
||||
| StdioStream | Yes (stdin) | Yes (stdout) | Raw byte pipe |
|
||||
| NullStream | -- | Yes | Discard sink (byte counter for benchmarks) |
|
||||
|
||||
Streams implement `IOStream` (byte-level) and `pes::Stream` (frame-level). `input()` / `output()` resolve URL strings to PES stream instances. `open_input()` / `open_output()` resolve to byte-level IOStream instances. All URLs use the `scheme://path` format — bare paths are rejected.
|
||||
Streams implement `pes::Stream` (frame-level). `input()` / `output()` resolve URL strings to PES stream instances. All URLs use the `scheme://path` format — bare paths are rejected.
|
||||
|
||||
AACS decryption requires a KEYDB.cfg file. If available at `~/.config/aacs/KEYDB.cfg` or passed via `ScanOptions`, the library handles everything — handshake, key derivation, and per-sector decryption — without the application needing to know anything about encryption.
|
||||
|
||||
## Architecture
|
||||
|
||||
```text
|
||||
Drive — open, identify, init, unlock, read (with recovery)
|
||||
Drive — open, identify, init, unlock, single-shot read
|
||||
├── ScsiTransport — SG_IO (Linux), IOKit (macOS), SPTI (Windows)
|
||||
├── DriveProfile — per-drive unlock parameters (bundled)
|
||||
└── PlatformDriver — MediaTek (supported), Renesas (planned)
|
||||
|
||||
@@ -12,6 +12,7 @@ Technical documentation for [libfreemkv](https://github.com/freemkv/libfreemkv),
|
||||
|----------|---------------|
|
||||
| [Architecture](architecture.md) | Module map, design principles, error codes, platform support |
|
||||
| [Drive Access](drive-access.md) | Drive, SCSI transport, profiles, unlock, why raw mode is needed |
|
||||
| [Rip Recovery](rip-recovery.md) | Three-layer recovery model: Disc::patch, single-shot Drive::read, DiscStream batch halving |
|
||||
| [AACS Encryption](aacs.md) | Key resolution (4 paths), content decryption, bus encryption, SCSI handshake |
|
||||
| [UDF Filesystem](udf.md) | UDF 2.50 with metadata partitions, pointer chain, how files are read from disc |
|
||||
| [MPLS Playlists](mpls.md) | Playlist format, play items, STN stream table, coding types |
|
||||
|
||||
+33
-3
@@ -126,15 +126,45 @@ pub struct Event {
|
||||
}
|
||||
|
||||
pub enum EventKind {
|
||||
// Init / scan
|
||||
DriveOpened { device: String },
|
||||
DriveReady,
|
||||
InitComplete { success: bool },
|
||||
ProbeComplete { success: bool },
|
||||
ScanComplete { titles: usize },
|
||||
|
||||
// Read pipeline
|
||||
BytesRead { bytes: u64, total: u64 },
|
||||
ReadError { sector: u64, error: Error },
|
||||
Retry { attempt: u32 },
|
||||
SpeedChange { speed_kbs: u16 },
|
||||
ExtentStart { index: usize, start_sector: u64, sector_count: u64 },
|
||||
SectorSkipped { sector: u64 },
|
||||
BatchSizeChanged { new_size: u16, reason: BatchSizeReason },
|
||||
Complete { bytes: u64, errors: u32 },
|
||||
|
||||
// Kept for forward-compat; not emitted in 0.13.6+
|
||||
Retry { attempt: u32 },
|
||||
SectorRecovered { sector: u64 },
|
||||
}
|
||||
```
|
||||
|
||||
Emission notes:
|
||||
|
||||
- `BytesRead { bytes, total }` is emitted from `DiscStream::fill_extents`
|
||||
after each successful sector read. `bytes` is the cumulative running
|
||||
total; `total` is the precomputed extent sum (0 if unknown).
|
||||
- `SpeedChange` is emitted from the public `Drive::set_speed` API path.
|
||||
It is no longer emitted from a recovery hot loop (recovery loop removed
|
||||
in 0.13.6).
|
||||
- `BatchSizeChanged` fires from the `DiscStream` adaptive sizer on shrink
|
||||
(read failed at a larger size) and on probe-up (clean-read streak hit
|
||||
the threshold). Consumers use it to display a "recovering" state
|
||||
distinct from "ripping normally".
|
||||
- `Retry` and `SectorRecovered` are NOT emitted in 0.13.6+. They were
|
||||
tied to the inline `Drive::read` recovery phases that were removed; the
|
||||
variants are kept for forward compatibility so consumers' match arms
|
||||
don't need conditional compilation.
|
||||
|
||||
Events report what happened. App decides what to do. GUI shows a dialog. CLI
|
||||
prints a line. Server logs to file.
|
||||
|
||||
@@ -145,8 +175,8 @@ libfreemkv/src/
|
||||
├── lib.rs Public exports
|
||||
├── error.rs Error codes (no English)
|
||||
├── event.rs Event types for callbacks
|
||||
├── drive/ Drive (open, init, read with recovery)
|
||||
│ ├── mod.rs Drive struct, init, read, reset, eject
|
||||
├── drive/ Drive (open, init, single-shot read)
|
||||
│ ├── mod.rs Drive struct, init, read (single-shot), reset, eject
|
||||
│ ├── capture.rs Drive profile capture for contribution
|
||||
│ ├── linux.rs Linux drive discovery
|
||||
│ ├── macos.rs macOS drive discovery
|
||||
|
||||
@@ -41,7 +41,7 @@ AACS keys are derived internally, and all SCSI communication is handled in-proce
|
||||
libfreemkv (lib.rs)
|
||||
│
|
||||
├── Drive Access
|
||||
│ ├── drive Drive — open, identify, init, unlock, read (with recovery)
|
||||
│ ├── drive Drive — open, identify, init, unlock, single-shot read
|
||||
│ ├── scsi ScsiTransport trait + platform backends (sg async, IOKit, SPTI)
|
||||
│ ├── platform/ Platform trait — per-chipset command handlers
|
||||
│ │ └── mt1959 MediaTek MT1959 driver (LG, ASUS, HP)
|
||||
@@ -93,10 +93,17 @@ Drive::open(Path::new("/dev/sg4"))
|
||||
After open:
|
||||
- `init()` -- unlock + firmware upload + speed calibration
|
||||
- `probe_disc()` -- probe disc surface for optimal speeds
|
||||
- `read(lba, count, buf)` -- single read method with built-in error recovery
|
||||
- `read(lba, count, buf, recovery)` -- single-shot read; `recovery` only selects the per-CDB timeout (1.5 s vs. 30 s)
|
||||
- `wait_ready()` -- wait for disc insertion
|
||||
- `eject()` -- eject tray
|
||||
|
||||
Recovery is layered above `Drive::read`, not inside it. Layer 1
|
||||
(`Disc::patch`) handles bad-range retry by replaying the ddrescue mapfile.
|
||||
Layer 3 (`DiscStream::fill_extents` adaptive batch sizer) handles in-loop
|
||||
request-size adaptation. Inline recovery (gentle retry → SCSI reset → retry)
|
||||
was removed in 0.13.6 — see [`rip-recovery.md`](rip-recovery.md) and
|
||||
`(internal)/postmortems/2026-04-25-stop-wedge-and-zero-kbs.md`.
|
||||
|
||||
---
|
||||
|
||||
## Disc Scanning Flow
|
||||
|
||||
+4
-2
@@ -77,7 +77,9 @@ Insert disc
|
||||
10. Stream content (mux/disc.rs → DiscStream)
|
||||
│ Read sectors → decrypt → TS demux → PES frames
|
||||
│ Or: read sectors → decrypt → raw bytes (for ISO output)
|
||||
│ Drive::read() handles error recovery (min speed → reset → retry)
|
||||
│ Drive::read() is single-shot. DiscStream::fill_extents adapts the
|
||||
│ batch size on failure (halve / probe-up). Bad-range retry is layer
|
||||
│ 1 above this — Disc::patch re-runs against the mapfile.
|
||||
│
|
||||
▼
|
||||
PES frames → output stream (MKV, M2TS, network, etc.)
|
||||
@@ -110,7 +112,7 @@ output.finish()?;
|
||||
|
||||
| Module | Doc | Purpose |
|
||||
|--------|-----|---------|
|
||||
| drive/ | [drive-access.md](drive-access.md) | Open, identify, init, unlock, read (with recovery) |
|
||||
| drive/ | [drive-access.md](drive-access.md) | Open, identify, init, unlock, single-shot read |
|
||||
| scsi/ | [drive-access.md](drive-access.md) | Platform SCSI transport (Linux, macOS, Windows) |
|
||||
| udf.rs | [udf.md](udf.md) | UDF 2.50 filesystem |
|
||||
| mpls.rs | [mpls.md](mpls.md) | MPLS playlists + STN streams |
|
||||
|
||||
+37
-15
@@ -26,8 +26,8 @@ platform driver. The drive is ready for `wait_ready()` and `init()`.
|
||||
| `wait_ready()` | Wait for disc insertion (30s timeout, TUR polling) |
|
||||
| `init()` | Firmware upload + unlock + speed calibration |
|
||||
| `probe_disc()` | Probe disc surface for optimal speeds |
|
||||
| `read(lba, count, buf)` | Read sectors with built-in error recovery |
|
||||
| `reset()` | Close/reopen device, TUR, escalate if needed |
|
||||
| `read(lba, count, buf, recovery)` | Read sectors. Single-shot — no inline retries or reset. |
|
||||
| `reset()` | Eject-cycle escape hatch. Caller-invoked only; not on the read path. |
|
||||
| `lock_tray()` | Prevent tray ejection during rip |
|
||||
| `unlock_tray()` | Allow tray ejection (also runs on Drop) |
|
||||
| `eject()` | Eject disc tray |
|
||||
@@ -44,22 +44,33 @@ platform driver. The drive is ready for `wait_ready()` and `init()`.
|
||||
3. Speed calibration after unlock
|
||||
4. Max 3 attempts before giving up
|
||||
|
||||
### read() with Recovery
|
||||
### read() — single-shot
|
||||
|
||||
`Drive::read(lba, count, buf, recovery)` is the single read method. The
|
||||
`recovery` parameter controls whether to attempt multi-phase recovery on
|
||||
failure or return immediately (used by DiscStream's binary search for
|
||||
single-sector probes).
|
||||
`Drive::read(lba, count, buf, recovery)` is the single read method. It issues
|
||||
exactly one READ(10) CDB and returns the result. The `recovery` parameter only
|
||||
selects the per-CDB timeout:
|
||||
|
||||
On error with `recovery = true`:
|
||||
| `recovery` | Timeout | Used by |
|
||||
|------------|----------|------------------------------------------|
|
||||
| `false` | 1.5 s | `Disc::copy` fast skip-forward sweep, `DiscStream::fill_extents` |
|
||||
| `true` | 30 s | `Disc::patch` multi-pass over the mapfile |
|
||||
|
||||
1. **Phase 1 — gentle retry (5 attempts):** set min speed, sleep 30s, retry.
|
||||
Each retry has a hard wall-clock timeout via async SG_IO.
|
||||
2. **Phase 2 — fresh start:** close transport, reset device, reopen, reinit.
|
||||
3. **Phase 3 — gentle retry on fresh connection (5 attempts).**
|
||||
4. If all fail: return `Err(DiscRead)`. DiscStream handles it (binary search,
|
||||
skip, zero-fill).
|
||||
5. Stay at min speed for 500 MB after any recovery (recovery window).
|
||||
On any SCSI failure or timeout, `read` returns `Err(DiscRead)` immediately.
|
||||
There are no inline retries, no SCSI reset, no Phase 1/2/3 escalation.
|
||||
|
||||
Recovery is layered above `Drive::read`:
|
||||
|
||||
- **Layer 1 — `Disc::patch`** loops over the ddrescue mapfile and re-issues
|
||||
`read(.., recovery=true)` against each non-`+` range.
|
||||
- **Layer 3 — `DiscStream::fill_extents`** halves the request size on
|
||||
failure, retries at the same LBA, and probes back up on a clean-read
|
||||
streak.
|
||||
|
||||
Inline recovery (5× gentle retry → close + reset + reopen → 5× more) was
|
||||
removed in 0.13.6. See `(internal)/postmortems/2026-04-25-stop-wedge-and-zero-kbs.md`
|
||||
for rationale: the inline reset wedged drive firmware on the LG BU40N (Initio
|
||||
USB-SATA bridge) without ever recovering a sector. See
|
||||
[`rip-recovery.md`](rip-recovery.md) for the full three-layer model.
|
||||
|
||||
---
|
||||
|
||||
@@ -99,6 +110,17 @@ cannot block us. Opens with `O_RDWR | O_NONBLOCK`.
|
||||
On non-zero SCSI status, the transport parses sense key from the sense buffer
|
||||
and returns `Error::ScsiError`.
|
||||
|
||||
`SgIoTransport::reset` (Linux) does pure userspace state cleanup: an open +
|
||||
close pair to make the kernel cancel any SG_IO commands queued against a
|
||||
previous fd, a 2 s sleep to let the kernel finish that cancellation, then a
|
||||
fresh fd to send ALLOW MEDIUM REMOVAL to clear any stale tray lock. It does
|
||||
NOT issue `SG_SCSI_RESET` or escalate via STOP+START UNIT. Both were tried
|
||||
in 0.13.0–0.13.5 against the LG BU40N (Initio USB-SATA bridge); both failed
|
||||
to recover wedged drives and made the wedge worse. The macOS reset (which
|
||||
had been a no-op) was removed entirely in 0.13.6, and the top-level
|
||||
`scsi::reset()` / `reset_with_timeout()` / `reset_blocking()` wrappers were
|
||||
removed at the same time (no callers).
|
||||
|
||||
### CDB Builders
|
||||
|
||||
The `scsi` module provides platform-agnostic CDB constructors:
|
||||
|
||||
+74
-20
@@ -1,12 +1,27 @@
|
||||
# Rip recovery — multi-pass architecture
|
||||
# Rip recovery — three-layer architecture
|
||||
|
||||
`libfreemkv` supports a two-stage rip model for damaged or protection-bearing
|
||||
discs: a fast forward sweep that tolerates read failures, followed by targeted
|
||||
`libfreemkv` supports a multi-stage rip model for damaged or protection-bearing
|
||||
discs: a fast forward sweep that tolerates read failures, in-loop request-size
|
||||
adaptation that survives transient drive trouble without bailing, and targeted
|
||||
retry passes against a persistent bad-range map. The stream pipeline
|
||||
(`DiscStream` + `input`/`output`) operates against the resulting ISO image, so
|
||||
the mux stage never touches the drive.
|
||||
|
||||
Three primitives compose the flow:
|
||||
Recovery is layered cleanly. Each layer has one responsibility and does not
|
||||
reach into the others.
|
||||
|
||||
| Layer | Where it lives | What it does |
|
||||
|-------|---------------|--------------|
|
||||
| 1 — Bad-range retry | `Disc::patch` (multi-pass over the mapfile) | Re-reads non-`+` ranges with the long timeout. Idempotent; call N times. |
|
||||
| 2 — Single-shot primitive | `Drive::read` in `src/drive/mod.rs` | One CDB, one timeout, one result. No inline retries, no SCSI reset. |
|
||||
| 3 — In-loop request adaptation | `DiscStream::fill_extents` adaptive batch sizer | Halves the batch on failure, retries at the same LBA, walks back up on a clean-read streak. |
|
||||
|
||||
The caller orchestrates layer 1. Autorip's `rip_disc` loops `copy` then
|
||||
N × `patch` per the `MAX_RETRIES` config, then hands the ISO off to the
|
||||
existing mux pipeline. Layer 3 runs inside any consumer of `DiscStream`
|
||||
(direct PES pipeline, ISO playback, etc.) without caller involvement.
|
||||
|
||||
Three primitives compose the disc-side flow:
|
||||
|
||||
| Primitive | What it does |
|
||||
|---------------------------|-----------------------------------------------------------------------|
|
||||
@@ -14,9 +29,6 @@ Three primitives compose the flow:
|
||||
| `Disc::patch` | Re-reads bad ranges from the drive. Idempotent; call N times. |
|
||||
| `DiscStream` (ISO source) | Reads sectors from the ISO, feeds decrypt → demux → codec → mux. |
|
||||
|
||||
The caller orchestrates. Autorip's `rip_disc` loops `copy` then N × `patch` per
|
||||
the `MAX_RETRIES` config, then hands the ISO off to the existing mux pipeline.
|
||||
|
||||
## Data model
|
||||
|
||||
### Mapfile
|
||||
@@ -26,7 +38,7 @@ plain text, greppable, tool-interoperable. Flushed to disk on every `record()`
|
||||
so a crashed rip loses at most one block.
|
||||
|
||||
```
|
||||
# Rescue Logfile. Created by libfreemkv v0.11.22
|
||||
# Rescue Logfile. Created by libfreemkv v0.13.6
|
||||
# Current pos / status / pass / pass_time
|
||||
0x000000000 ? 1 0
|
||||
# pos size status
|
||||
@@ -50,8 +62,8 @@ Position and size are hex byte offsets into the ISO.
|
||||
|
||||
### `CopyOptions` and `PatchOptions`
|
||||
|
||||
Defaults preserve pre-`0.11.21` behavior — full drive recovery on every read,
|
||||
abort on the first unreadable sector. Opt in to the recovery-friendly path:
|
||||
Defaults preserve pre-`0.11.21` behavior — abort on the first unreadable
|
||||
sector. Opt in to the recovery-friendly path:
|
||||
|
||||
```rust
|
||||
CopyOptions {
|
||||
@@ -65,9 +77,10 @@ CopyOptions {
|
||||
|
||||
## Algorithm
|
||||
|
||||
### Pass 1 — fast sweep
|
||||
### Pass 1 — fast sweep (`Disc::copy`)
|
||||
|
||||
1. Read 64 KB (32 sectors, one BD ECC block) at the current LBA.
|
||||
1. Read 64 KB (32 sectors, one BD ECC block) at the current LBA via
|
||||
`Drive::read(.., recovery=false)` — short 1.5 s timeout, single shot.
|
||||
2. On success: mark the range `+`, advance by one block.
|
||||
3. On failure (with `skip_on_error`): zero-fill the block in the ISO, mark
|
||||
it `*`, advance.
|
||||
@@ -80,12 +93,13 @@ CopyOptions {
|
||||
Pass 1 completes when every byte has terminal status (`+`, `-`, or the caller
|
||||
bails via the halt flag).
|
||||
|
||||
### Pass 2+ — patch
|
||||
### Pass 2+ — patch (`Disc::patch`)
|
||||
|
||||
`Disc::patch` reads the mapfile and iterates every non-`+` range. For each:
|
||||
|
||||
1. Issue a drive read with full recovery enabled (SCSI-level retries,
|
||||
ECC recovery, the lot).
|
||||
1. Issue a drive read via `Drive::read(.., recovery=true)` — long 30 s
|
||||
timeout, still single shot. Drive firmware does its own ECC and retries
|
||||
inside that window; userspace does not pile on additional retries here.
|
||||
2. On success: write the good bytes into the ISO at the exact byte offset,
|
||||
mark `+`.
|
||||
3. On failure: mark `-`.
|
||||
@@ -95,21 +109,61 @@ Idempotent. Call `patch` N times for N retry attempts; typically the caller
|
||||
stops early if a pass recovers zero bytes (structure-protected sectors will
|
||||
never yield).
|
||||
|
||||
### In-stream — adaptive batch halving (`DiscStream::fill_extents`)
|
||||
|
||||
When a consumer reads a `DiscStream` directly (no ISO intermediate),
|
||||
`fill_extents` runs an adaptive sizer in front of `Drive::read`:
|
||||
|
||||
1. Try the current preferred batch size (e.g. 32 sectors, one BD ECC block).
|
||||
2. On failure: halve the batch and retry at the same LBA. Emit
|
||||
`EventKind::BatchSizeChanged { reason: Shrunk }`.
|
||||
3. On a clean-read streak: probe back up toward the preferred size. Emit
|
||||
`EventKind::BatchSizeChanged { reason: Probed }`.
|
||||
4. If a single-sector read fails: skip (zero-fill, emit
|
||||
`EventKind::SectorSkipped`) when `skip_errors` is set, otherwise return
|
||||
`Err(DiscRead)`.
|
||||
|
||||
This is layer 3. It exists so a transient single-sector glitch in a 32-sector
|
||||
batch can be isolated and read individually without the caller needing to
|
||||
implement retry logic.
|
||||
|
||||
## Design choices
|
||||
|
||||
**`Drive::read` is single-shot.** No inline retry phases, no SCSI reset,
|
||||
no eject cycle. The `recovery` flag controls only the per-CDB timeout
|
||||
(1.5 s vs. 30 s); on any failure it returns `Err(DiscRead)` immediately.
|
||||
Inline recovery (5× gentle retry → close + SCSI reset + reopen → 5× more)
|
||||
was removed in 0.13.6. See `(internal)/postmortems/2026-04-25-stop-wedge-and-zero-kbs.md`
|
||||
for rationale: the inline reset on the LG BU40N (Initio USB-SATA bridge)
|
||||
wedged drive firmware below the bridge without ever recovering a sector,
|
||||
and the gentle-retry phase produced long stretches of 0 KB/s with no
|
||||
recoveries to show for it. Recovery responsibility is now layered: layer 1
|
||||
handles ranges, layer 3 handles request size, neither touches the
|
||||
wedge-prone reset path.
|
||||
|
||||
**No `MODE SELECT` to disable drive retries.** Research showed neither ddrescue
|
||||
nor MakeMKV does this. Drive firmware has access to raw analog signal, laser
|
||||
power control, and drive-specific ECC tuning that userspace can't replicate —
|
||||
disabling it throws away recovery headroom on marginal sectors. We fail fast
|
||||
via short SG_IO timeouts instead, and we avoid per-sector probing on first
|
||||
contact by using large blocks + skip-forward.
|
||||
via short SG_IO timeouts in pass 1 and let the firmware work the long timeout
|
||||
in pass 2 / patch.
|
||||
|
||||
**No SCSI reset from any retry path.** `SgIoTransport::reset` (Linux) is
|
||||
trimmed to a kernel SG_IO state flush plus ALLOW MEDIUM REMOVAL — the
|
||||
`SG_SCSI_RESET` ioctl and STOP/START UNIT escalation were removed in 0.13.6.
|
||||
The macOS reset (which had been a no-op) was removed entirely. The top-level
|
||||
`scsi::reset()` / `reset_with_timeout()` / `reset_blocking()` wrappers were
|
||||
also removed (no callers). The remaining `Drive::reset()` is only invoked
|
||||
explicitly by callers that need an eject-cycle escape hatch — it is never
|
||||
reached from a read path.
|
||||
|
||||
**ISO intermediate, even for single-pass.** Pass 1 always writes an ISO. The
|
||||
mux stage reads the ISO via `IsoSectorReader`. For single-pass (no retries),
|
||||
this adds ~2-3 min (local disk mux) but gains resumability across crashes,
|
||||
re-muxability without re-ripping, and a persistent forensic artifact. Callers
|
||||
who need pure speed can bypass and use `DiscStream::new(Box::new(drive), …)`
|
||||
directly — the lib doesn't forbid it.
|
||||
directly — the lib doesn't forbid it, and layer 3 (adaptive batch halving)
|
||||
still applies there.
|
||||
|
||||
**Mapfile in ddrescue format.** Plain text so users can `less` it, `diff` it,
|
||||
or feed it to ddrescue's own tooling. Crash-safe (flush-per-record). Entries
|
||||
@@ -117,11 +171,11 @@ coalesce on adjacent same-status ranges so files stay small.
|
||||
|
||||
**Patches target `-`, `*`, `/`, and `?` alike.** The status state machine is
|
||||
ddrescue's but `patch` collapses the distinction — it just tries every
|
||||
non-finished range with full recovery. Future work can specialize (trim vs.
|
||||
non-finished range with the long timeout. Future work can specialize (trim vs.
|
||||
scrape vs. retry with direction reversal) if there's measured benefit.
|
||||
|
||||
## References
|
||||
|
||||
- [ddrescue manual, Algorithm chapter](https://www.gnu.org/software/ddrescue/manual/ddrescue_manual.html)
|
||||
- [ddrescue optical media notes](https://www.electric-spoon.com/doc/gddrescue/html/Optical-media.html)
|
||||
- Source: [`src/disc/mapfile.rs`](../src/disc/mapfile.rs), [`src/disc/mod.rs`](../src/disc/mod.rs) (`Disc::copy`, `Disc::patch`)
|
||||
- Source: [`src/disc/mapfile.rs`](../src/disc/mapfile.rs), [`src/disc/mod.rs`](../src/disc/mod.rs) (`Disc::copy`, `Disc::patch`), [`src/drive/mod.rs`](../src/drive/mod.rs) (`Drive::read`), [`src/mux/disc.rs`](../src/mux/disc.rs) (`DiscStream::fill_extents`).
|
||||
|
||||
@@ -1286,6 +1286,9 @@ impl Disc {
|
||||
let mut buf = vec![0u8; batch as usize * 2048];
|
||||
let mut bytes_done = 0u64;
|
||||
let mut halt_requested = false;
|
||||
let stall_threshold = std::time::Duration::from_secs(opts.stall_secs.unwrap_or(120));
|
||||
let mut last_good_advance = std::time::Instant::now();
|
||||
let mut last_observed_good: u64 = 0;
|
||||
|
||||
// Iterate over not-yet-finished regions from the mapfile. We re-read the
|
||||
// mapfile after each block because record() mutates the region list.
|
||||
@@ -1315,6 +1318,30 @@ impl Disc {
|
||||
break 'outer;
|
||||
}
|
||||
}
|
||||
// Stall guard. The signal is "bytes_good (Finished sectors)
|
||||
// hasn't advanced for stall_threshold." pos may still be
|
||||
// advancing via the skip_on_error branch; that's not real
|
||||
// progress because skip-forward only marks ranges
|
||||
// NonTrimmed for Pass 2 to retry. If we go stall_threshold
|
||||
// without ANY successful read, the drive is grinding
|
||||
// unproductively (or kernel is silently stalling reads
|
||||
// past their per-CDB timeout — observed live on Dell with
|
||||
// SgIoTransport's reopen-after-timeout serializing
|
||||
// against close). Bail Pass 1; Pass 2 (Disc::patch with
|
||||
// recovery=true, 30 s timeouts) will retry the
|
||||
// NonTrimmed ranges.
|
||||
let cur_good = map.stats().bytes_good;
|
||||
if cur_good != last_observed_good {
|
||||
last_observed_good = cur_good;
|
||||
last_good_advance = std::time::Instant::now();
|
||||
} else if last_good_advance.elapsed() > stall_threshold {
|
||||
// Stall: bail Pass 1. Return cleanly with
|
||||
// bytes_pending > 0 and complete = false so the
|
||||
// caller's retry path (Disc::patch with
|
||||
// recovery=true, 30s timeouts) gets a shot at the
|
||||
// NonTrimmed ranges.
|
||||
break 'outer;
|
||||
}
|
||||
let block_bytes = (region_end - pos).min(batch as u64 * 2048);
|
||||
let lba = (pos / 2048) as u32;
|
||||
let count = (block_bytes / 2048) as u16;
|
||||
@@ -1404,6 +1431,15 @@ pub struct CopyOptions<'a> {
|
||||
pub skip_forward: bool,
|
||||
pub on_progress: Option<&'a dyn Fn(u64, u64)>,
|
||||
pub halt: Option<std::sync::Arc<std::sync::atomic::AtomicBool>>,
|
||||
/// Wall-clock stall threshold in seconds. If the inner copy loop runs
|
||||
/// this long without `pos` advancing, the current block is treated as
|
||||
/// a read failure (skip-forward in `skip_on_error` mode, else
|
||||
/// `Err(DiscRead)`). Defaults to 120 s. Defensive guard for
|
||||
/// kernel-level hangs that bypass the per-CDB SCSI timeout — e.g. the
|
||||
/// v0.13.8 case where SgIoTransport's reopen-after-timeout serialized
|
||||
/// against the in-flight close, blocking the main thread for tens of
|
||||
/// seconds per read.
|
||||
pub stall_secs: Option<u64>,
|
||||
}
|
||||
|
||||
/// Result of `Disc::copy`. `complete=true` means every byte reached a terminal
|
||||
|
||||
+43
-123
@@ -53,9 +53,6 @@ const SCSI_GET_EVENT_STATUS: u8 = 0x4A;
|
||||
const SCSI_MODE_SENSE: u8 = 0x5A;
|
||||
const SCSI_REPORT_KEY: u8 = 0xA4;
|
||||
|
||||
/// Recovery state after a read error — stay at min speed for N bytes.
|
||||
const RECOVERY_WINDOW: u64 = 500 * 1024 * 1024; // 500 MB
|
||||
|
||||
/// Optical disc drive session -- open, identify, unlock, and read.
|
||||
pub struct Drive {
|
||||
scsi: Box<dyn ScsiTransport>,
|
||||
@@ -64,11 +61,9 @@ pub struct Drive {
|
||||
pub platform: Option<profile::Platform>,
|
||||
pub drive_id: DriveId,
|
||||
device_path: String,
|
||||
/// Bytes remaining in the min-speed recovery window.
|
||||
recovery_bytes_remaining: u64,
|
||||
/// Halt flag — when set, Drive::read() bails at the next check point.
|
||||
halt: Arc<AtomicBool>,
|
||||
/// Event handler — fires during read recovery.
|
||||
/// Event handler — fires for read errors and library-level state changes.
|
||||
event_fn: Option<Box<dyn Fn(Event) + Send>>,
|
||||
}
|
||||
|
||||
@@ -95,7 +90,6 @@ impl Drive {
|
||||
profile,
|
||||
drive_id,
|
||||
device_path: device.to_string_lossy().to_string(),
|
||||
recovery_bytes_remaining: 0,
|
||||
halt: Arc::new(AtomicBool::new(false)),
|
||||
event_fn: None,
|
||||
})
|
||||
@@ -121,6 +115,9 @@ impl Drive {
|
||||
self.event_fn = Some(Box::new(f));
|
||||
}
|
||||
|
||||
#[allow(dead_code)] // public on_event registration kept; Drive currently
|
||||
// has no internal emission sites after the 0.13.6 recovery strip.
|
||||
// DiscStream is the BytesRead source. Plan to drop on_event in 0.14.
|
||||
fn emit(&self, kind: EventKind) {
|
||||
if let Some(ref f) = self.event_fn {
|
||||
f(Event { kind });
|
||||
@@ -131,13 +128,6 @@ impl Drive {
|
||||
self.halt.load(Ordering::Relaxed)
|
||||
}
|
||||
|
||||
/// Halt-aware sleep. Returns `Err(Halted)` if the flag fires before or
|
||||
/// during the wait. The only sleep the drive's recovery path is allowed
|
||||
/// to use — makes it impossible to accidentally block through a Stop.
|
||||
fn checked_sleep(&self, total: std::time::Duration) -> Result<()> {
|
||||
sleep_until_halted(&self.halt, total)
|
||||
}
|
||||
|
||||
/// Halt-aware SCSI execute. Returns `Err(Halted)` if the flag is set
|
||||
/// before the command dispatches or by the time it completes. The only
|
||||
/// path to talk to the drive in the recovery hot loop; keeps Drive::read
|
||||
@@ -519,29 +509,25 @@ impl Drive {
|
||||
}
|
||||
}
|
||||
|
||||
/// Read sectors from the disc with automatic error recovery.
|
||||
/// Read sectors from the disc. Single-shot — no inline retries, no
|
||||
/// SCSI reset.
|
||||
///
|
||||
/// On failure: drops to min speed, waits with escalating patience
|
||||
/// (5s, 10s, 15s, 30s, 60s), resets drive between attempts.
|
||||
/// After recovery, stays at min speed for 500 MB before ramping up.
|
||||
/// `recovery=true` bumps the per-CDB timeout to 30 s for the
|
||||
/// `Disc::patch` pass; `recovery=false` uses 1.5 s for `Disc::copy`'s
|
||||
/// fast skip-forward sweep. On any failure returns `Err(DiscRead)`
|
||||
/// immediately. The orchestration layer (`Disc::patch`'s outer loop
|
||||
/// for the patch pass, `DiscStream`'s adaptive batch halving for the
|
||||
/// stream path) handles retries.
|
||||
///
|
||||
/// Returns Err only after all attempts exhausted — user should clean
|
||||
/// the disc and resume.
|
||||
/// Inline retry phases (5× gentle + reset+reopen + 5× more) were
|
||||
/// removed in 0.13.6. Per
|
||||
/// `(internal)/postmortems/2026-04-25-stop-wedge-and-zero-kbs.md`,
|
||||
/// the inline reset on the LG BU40N (Initio bridge) wedged drive
|
||||
/// firmware without ever recovering a sector. The remaining recovery
|
||||
/// layers (Disc::patch multi-pass, DiscStream batch halving) do not
|
||||
/// touch the wedge-prone reset path.
|
||||
pub fn read(&mut self, lba: u32, count: u16, buf: &mut [u8], recovery: bool) -> Result<usize> {
|
||||
// Non-recovery mode is the Disc::copy fast pass — intent is "fail
|
||||
// fast, let skip_forward advance past bad regions." A 5s budget let
|
||||
// the drive grind L-EC on structure-protected / marginal sectors for
|
||||
// nearly the full interval per 64 KB block, pinning throughput at
|
||||
// ~13 KB/s on difficult discs. 1500ms kills slow reads early so
|
||||
// skip_forward can double its stride; recoverable sectors are picked
|
||||
// up on Disc::patch where recovery=true and the timeout is 30s.
|
||||
let timeout_ms = if !recovery {
|
||||
1_500
|
||||
} else if self.recovery_bytes_remaining > 0 {
|
||||
30_000
|
||||
} else {
|
||||
10_000
|
||||
};
|
||||
let timeout_ms = if recovery { 30_000 } else { 1_500 };
|
||||
let cdb = [
|
||||
crate::scsi::SCSI_READ_10,
|
||||
0x00,
|
||||
@@ -555,82 +541,16 @@ impl Drive {
|
||||
0x00,
|
||||
];
|
||||
|
||||
// Normal read. Fast path: drive returns data, we return it.
|
||||
// checked_exec short-circuits to Err(Halted) if Stop was requested;
|
||||
// read() never touches the halt flag directly.
|
||||
match self.checked_exec(
|
||||
&cdb,
|
||||
crate::scsi::DataDirection::FromDevice,
|
||||
buf,
|
||||
timeout_ms,
|
||||
) {
|
||||
Ok(result) => {
|
||||
if self.recovery_bytes_remaining > 0 {
|
||||
let bytes_read = count as u64 * 2048;
|
||||
self.recovery_bytes_remaining =
|
||||
self.recovery_bytes_remaining.saturating_sub(bytes_read);
|
||||
if self.recovery_bytes_remaining == 0 {
|
||||
self.emit(EventKind::SpeedChange { speed_kbs: 0xFFFF });
|
||||
self.set_speed(0xFFFF);
|
||||
}
|
||||
}
|
||||
return Ok(result.bytes_transferred);
|
||||
}
|
||||
Err(Error::Halted) => return Err(Error::Halted),
|
||||
Err(_) => { /* fall through to recovery */ }
|
||||
Ok(result) => Ok(result.bytes_transferred),
|
||||
Err(Error::Halted) => Err(Error::Halted),
|
||||
Err(_) => Err(Error::DiscRead { sector: lba as u64 }),
|
||||
}
|
||||
|
||||
if !recovery {
|
||||
return Err(Error::DiscRead { sector: lba as u64 });
|
||||
}
|
||||
|
||||
// Enter recovery
|
||||
self.emit(EventKind::ReadError {
|
||||
sector: lba as u64,
|
||||
error: Error::DiscRead { sector: lba as u64 },
|
||||
});
|
||||
self.emit(EventKind::SpeedChange { speed_kbs: 0 });
|
||||
self.set_speed(0);
|
||||
|
||||
// Phase 1: gentle — sleep 30 s, retry. 5 times.
|
||||
for attempt in 1..=5u32 {
|
||||
self.emit(EventKind::Retry { attempt });
|
||||
self.checked_sleep(std::time::Duration::from_secs(30))?;
|
||||
if let Ok(result) =
|
||||
self.checked_exec(&cdb, crate::scsi::DataDirection::FromDevice, buf, 30_000)
|
||||
{
|
||||
self.emit(EventKind::SectorRecovered { sector: lba as u64 });
|
||||
self.recovery_bytes_remaining = RECOVERY_WINDOW;
|
||||
return Ok(result.bytes_transferred);
|
||||
}
|
||||
}
|
||||
|
||||
// Phase 2: fresh start — close, reset, open, init.
|
||||
let device = std::path::PathBuf::from(&self.device_path);
|
||||
self.checked_sleep(std::time::Duration::from_secs(5))?;
|
||||
let _ = crate::scsi::reset(&device);
|
||||
self.checked_sleep(std::time::Duration::from_secs(5))?;
|
||||
self.scsi = crate::scsi::open(&device)?;
|
||||
let _ = self.init();
|
||||
let _ = self.wait_ready();
|
||||
self.set_speed(0);
|
||||
|
||||
// Phase 3: gentle again on fresh connection — sleep 30 s, retry. 5 times.
|
||||
for attempt in 6..=10u32 {
|
||||
self.emit(EventKind::Retry { attempt });
|
||||
self.checked_sleep(std::time::Duration::from_secs(30))?;
|
||||
if let Ok(result) =
|
||||
self.checked_exec(&cdb, crate::scsi::DataDirection::FromDevice, buf, 30_000)
|
||||
{
|
||||
self.emit(EventKind::SectorRecovered { sector: lba as u64 });
|
||||
self.recovery_bytes_remaining = RECOVERY_WINDOW;
|
||||
return Ok(result.bytes_transferred);
|
||||
}
|
||||
}
|
||||
|
||||
// Both phases failed.
|
||||
self.recovery_bytes_remaining = RECOVERY_WINDOW;
|
||||
Err(Error::DiscRead { sector: lba as u64 })
|
||||
}
|
||||
|
||||
/// Read the disc capacity in sectors (2048 bytes each).
|
||||
@@ -742,11 +662,26 @@ impl SectorReader for Drive {
|
||||
}
|
||||
}
|
||||
|
||||
/// Halt-aware sleep primitive. Returns `Err(Halted)` if the flag is set
|
||||
/// before or during the wait. Extracted as a free function so it can be
|
||||
/// unit-tested without constructing a full `Drive`.
|
||||
///
|
||||
/// Wakes within ~100 ms of a halt, regardless of the requested duration.
|
||||
/// Find all optical drives connected to this system.
|
||||
/// Returns opened Drive objects ready for use.
|
||||
pub fn find_drives() -> Vec<Drive> {
|
||||
discover_drives()
|
||||
.into_iter()
|
||||
.filter_map(|(path, _)| Drive::open(std::path::Path::new(&path)).ok())
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Find the first optical drive.
|
||||
/// Returns an opened Drive ready for use.
|
||||
pub fn find_drive() -> Option<Drive> {
|
||||
find_drives().into_iter().next()
|
||||
}
|
||||
|
||||
/// Halt-aware sleep primitive — wakes within ~100 ms of `halt` flipping
|
||||
/// to true. Kept for the unit tests that cover the slicing behaviour;
|
||||
/// production code paths no longer sleep on the recovery hot path
|
||||
/// (recovery loop removed in 0.13.6).
|
||||
#[cfg(test)]
|
||||
fn sleep_until_halted(halt: &AtomicBool, total: std::time::Duration) -> Result<()> {
|
||||
const SLICE: std::time::Duration = std::time::Duration::from_millis(100);
|
||||
let deadline = std::time::Instant::now() + total;
|
||||
@@ -763,21 +698,6 @@ fn sleep_until_halted(halt: &AtomicBool, total: std::time::Duration) -> Result<(
|
||||
}
|
||||
}
|
||||
|
||||
/// Find all optical drives connected to this system.
|
||||
/// Returns opened Drive objects ready for use.
|
||||
pub fn find_drives() -> Vec<Drive> {
|
||||
discover_drives()
|
||||
.into_iter()
|
||||
.filter_map(|(path, _)| Drive::open(std::path::Path::new(&path)).ok())
|
||||
.collect()
|
||||
}
|
||||
|
||||
/// Find the first optical drive.
|
||||
/// Returns an opened Drive ready for use.
|
||||
pub fn find_drive() -> Option<Drive> {
|
||||
find_drives().into_iter().next()
|
||||
}
|
||||
|
||||
/// Internal: discover drive paths + IDs without opening full Drive objects.
|
||||
fn discover_drives() -> Vec<(String, DriveId)> {
|
||||
#[cfg(target_os = "linux")]
|
||||
|
||||
@@ -127,6 +127,14 @@ pub struct DiscStream {
|
||||
event_fn: Option<Box<dyn Fn(Event) + Send>>,
|
||||
eof: bool,
|
||||
|
||||
// Cumulative bytes successfully read from the source. Drives
|
||||
// EventKind::BytesRead emission and autorip's per-device progress.
|
||||
bytes_read_total: u64,
|
||||
// Pre-computed total of all extents in bytes (or 0 if extents are
|
||||
// empty). Carried in EventKind::BytesRead.total so consumers can show
|
||||
// a percent without a separate API call.
|
||||
bytes_total_extents: u64,
|
||||
|
||||
// PES output
|
||||
ts_demuxer: Option<super::ts::TsDemuxer>,
|
||||
ps_demuxer: Option<super::ps::PsDemuxer>,
|
||||
@@ -149,6 +157,7 @@ impl DiscStream {
|
||||
content_format: crate::disc::ContentFormat,
|
||||
) -> Self {
|
||||
let extents = title.extents.clone();
|
||||
let bytes_total_extents: u64 = extents.iter().map(|e| e.sector_count as u64 * 2048).sum();
|
||||
|
||||
let mut pids = Vec::new();
|
||||
let mut parsers = Vec::new();
|
||||
@@ -194,6 +203,8 @@ impl DiscStream {
|
||||
halt: None,
|
||||
event_fn: None,
|
||||
eof: false,
|
||||
bytes_read_total: 0,
|
||||
bytes_total_extents,
|
||||
ts_demuxer,
|
||||
ps_demuxer,
|
||||
parsers,
|
||||
@@ -287,6 +298,11 @@ impl DiscStream {
|
||||
}
|
||||
self.buf_valid = bytes;
|
||||
self.current_offset += sectors as u32;
|
||||
self.bytes_read_total = self.bytes_read_total.saturating_add(bytes as u64);
|
||||
self.emit(EventKind::BytesRead {
|
||||
bytes: self.bytes_read_total,
|
||||
total: self.bytes_total_extents,
|
||||
});
|
||||
break;
|
||||
}
|
||||
|
||||
|
||||
+56
-84
@@ -12,8 +12,6 @@ use crate::error::{Error, Result};
|
||||
use std::path::Path;
|
||||
|
||||
const SG_IO: u32 = 0x2285;
|
||||
const SG_SCSI_RESET: u32 = 0x2284;
|
||||
const SG_SCSI_RESET_DEVICE: i32 = 1;
|
||||
const SG_DXFER_NONE: i32 = -1;
|
||||
const SG_DXFER_TO_DEV: i32 = -2;
|
||||
const SG_DXFER_FROM_DEV: i32 = -3;
|
||||
@@ -80,49 +78,32 @@ impl SgIoTransport {
|
||||
})
|
||||
}
|
||||
|
||||
/// Reset the drive to a known good state — equivalent to unplug/replug.
|
||||
/// After reset, the drive is clean and no fd is held open.
|
||||
/// Clean up kernel SG_IO state and unlock the tray. NOT a hardware
|
||||
/// reset — purely software cleanup before this process opens the
|
||||
/// device for real work.
|
||||
///
|
||||
/// ## Why each step exists
|
||||
/// When a previous process is killed (SIGKILL) mid-SG_IO, the kernel
|
||||
/// may hold queued commands against the dead fd, and `Drop` never
|
||||
/// ran so the tray may still be locked via PREVENT MEDIUM REMOVAL.
|
||||
/// This routine handles both: open + close flushes the kernel SG
|
||||
/// queue (sg_release cancels commands tied to the fd), the 2 s sleep
|
||||
/// gives the kernel time to finish that cleanup, then a fresh fd
|
||||
/// sends ALLOW MEDIUM REMOVAL to clear any stale tray lock.
|
||||
///
|
||||
/// When a process is killed (SIGKILL/kill -9) mid-SG_IO ioctl, two things
|
||||
/// go wrong: (1) the kernel's SG driver may have stale pending commands
|
||||
/// queued for the dead process's fd, and (2) the drive firmware may still
|
||||
/// be mid-operation (seeking, reading, processing a vendor command).
|
||||
///
|
||||
/// A new process opening the same /dev/sg* device gets a fresh fd, but the
|
||||
/// kernel doesn't automatically abort the dead process's commands — the
|
||||
/// drive can appear hung on the first SCSI command.
|
||||
///
|
||||
/// Additionally, killed processes skip Drop, so the tray may be locked
|
||||
/// via PREVENT MEDIUM REMOVAL with no process alive to unlock it.
|
||||
///
|
||||
/// ## Sequence
|
||||
///
|
||||
/// 1. **open** — allocates kernel SG state for this fd
|
||||
/// 2. **close** — triggers kernel cleanup: aborts any pending SG_IO
|
||||
/// commands associated with this fd. The key operation —
|
||||
/// the kernel's sg_release() cancels queued commands.
|
||||
/// 3. **sleep 2s** — the drive firmware needs time to finish/abort whatever
|
||||
/// it was doing when the previous process died. Without
|
||||
/// this, the next command may block on drive-internal state.
|
||||
/// 4. **open** — fresh fd with no stale commands in the kernel queue
|
||||
/// 5. **unlock** — ALLOW MEDIUM REMOVAL (CDB 0x1E, prevent=0). Clears
|
||||
/// any tray lock left by a killed process that never
|
||||
/// ran its Drop/cleanup.
|
||||
/// 6. **TUR** — TEST UNIT READY (CDB 0x00) with 3s timeout. If the
|
||||
/// drive responds, it's in a good state.
|
||||
/// 7. **escalate** — if TUR fails:
|
||||
/// - SG_SCSI_RESET (device level) — kernel sends a SCSI
|
||||
/// bus reset to the device, clearing all firmware state.
|
||||
/// - STOP + START UNIT (CDB 0x1B) — power-cycles the
|
||||
/// drive's logical unit, like pressing the eject button
|
||||
/// and reinserting.
|
||||
/// 8. **close** — release the fd. Drive is clean, nobody holds it.
|
||||
/// We do NOT verify the drive with TUR or escalate to SG_SCSI_RESET /
|
||||
/// STOP+START UNIT. Both escalations were tried in 0.13.0–0.13.5
|
||||
/// against the LG BU40N (Initio USB-SATA bridge); both failed to
|
||||
/// recover wedged drives and made the wedge worse — see
|
||||
/// (internal)/postmortems/2026-04-25-bu40n-wedge-recovery.md.
|
||||
/// If the drive is genuinely unresponsive, the next workload command
|
||||
/// fails naturally and the caller surfaces a "physical reconnect
|
||||
/// required" prompt. Software has no path back from a wedged Initio
|
||||
/// bridge — only physical replug clears it.
|
||||
pub fn reset(device: &Path) -> Result<()> {
|
||||
let c_path = Self::to_c_path(device);
|
||||
|
||||
// Step 1-2: open + close — flush stale kernel SG_IO state
|
||||
// open + close — make the kernel cancel any SG_IO commands queued
|
||||
// against a previous fd that didn't close cleanly.
|
||||
let probe_fd = unsafe {
|
||||
libc::open(
|
||||
c_path.as_ptr() as *const libc::c_char,
|
||||
@@ -133,10 +114,10 @@ impl SgIoTransport {
|
||||
unsafe { libc::close(probe_fd) };
|
||||
}
|
||||
|
||||
// Step 3: let drive settle
|
||||
// Let the kernel finish that cancellation before we reopen.
|
||||
std::thread::sleep(std::time::Duration::from_secs(2));
|
||||
|
||||
// Step 4: open clean fd
|
||||
// Fresh fd just to send the unlock command, then close.
|
||||
let fd = unsafe {
|
||||
libc::open(
|
||||
c_path.as_ptr() as *const libc::c_char,
|
||||
@@ -147,27 +128,10 @@ impl SgIoTransport {
|
||||
return Self::open_error(device);
|
||||
}
|
||||
|
||||
// Step 5: unlock tray
|
||||
// ALLOW MEDIUM REMOVAL — clear any tray lock left by a killed
|
||||
// process whose Drop never ran. Best-effort; ignore result.
|
||||
let _ = Self::raw_command(fd, &[0x1E, 0, 0, 0, 0, 0], 3_000);
|
||||
|
||||
// Step 6: TUR — if drive responds, we're done
|
||||
if Self::raw_command(fd, &[0, 0, 0, 0, 0, 0], 3_000).is_err() {
|
||||
// Step 7: escalate — SG_SCSI_RESET
|
||||
let mut reset_type: i32 = SG_SCSI_RESET_DEVICE;
|
||||
unsafe { libc::ioctl(fd, SG_SCSI_RESET as _, &mut reset_type) };
|
||||
std::thread::sleep(std::time::Duration::from_secs(3));
|
||||
|
||||
if Self::raw_command(fd, &[0, 0, 0, 0, 0, 0], 3_000).is_err() {
|
||||
// STOP + START
|
||||
let _ = Self::raw_command(fd, &[0x1B, 0, 0, 0, 0x00, 0], 3_000);
|
||||
std::thread::sleep(std::time::Duration::from_secs(1));
|
||||
let _ = Self::raw_command(fd, &[0x1B, 0, 0, 0, 0x01, 0], 3_000);
|
||||
std::thread::sleep(std::time::Duration::from_secs(3));
|
||||
let _ = Self::raw_command(fd, &[0, 0, 0, 0, 0, 0], 3_000);
|
||||
}
|
||||
}
|
||||
|
||||
// Step 8: close — drive is clean
|
||||
unsafe { libc::close(fd) };
|
||||
Ok(())
|
||||
}
|
||||
@@ -357,26 +321,38 @@ impl ScsiTransport for SgIoTransport {
|
||||
|
||||
if pr <= 0 {
|
||||
// Timeout (0) or fatal poll error (-1).
|
||||
// Command is still pending in the kernel. Abandon this fd and
|
||||
// open a fresh one. The old fd is closed in a background thread
|
||||
// because close() blocks until the kernel completes/aborts the
|
||||
// pending command.
|
||||
// Command is still pending in the kernel. Abandon the fd by
|
||||
// spawning a background close (which will block until the
|
||||
// kernel completes/aborts the pending command), and mark
|
||||
// this transport invalid by setting `self.fd = -1`.
|
||||
//
|
||||
// Why we no longer reopen on the main thread: opening the
|
||||
// SAME /dev/sg* device while the prior fd is mid-close
|
||||
// serializes via the kernel's per-device state lock, so
|
||||
// `libc::open()` on the main thread blocks for the same
|
||||
// duration that close() does — defeating the userspace
|
||||
// timeout. Observed in v0.13.8 live test on Dune 2: each
|
||||
// timed-out read added 60+ s to the next iteration of
|
||||
// Disc::copy, leaving the rip stuck without surfacing an
|
||||
// error or wedging the drive.
|
||||
//
|
||||
// Net effect of the fix: a single read timeout invalidates
|
||||
// the SgIoTransport. The Drive is now "dead" until the
|
||||
// consumer (autorip's rip thread) catches the failure and
|
||||
// reopens. Disc::copy's `skip_on_error=true` path will see
|
||||
// the Err and skip-forward, advancing pos, and the next
|
||||
// read on this fd returns Err(DeviceNotFound) immediately —
|
||||
// which Disc::copy continues to skip-forward through until
|
||||
// the NonTried region is exhausted. Pass 1 then ends with
|
||||
// bytes_pending > 0 and the rip thread reopens the Drive
|
||||
// for Pass 2 (Disc::patch with recovery=true and 30 s
|
||||
// timeouts).
|
||||
let old_fd = self.fd;
|
||||
self.fd = -1;
|
||||
|
||||
std::thread::spawn(move || {
|
||||
unsafe { libc::close(old_fd) };
|
||||
});
|
||||
|
||||
let c_path = Self::to_c_path(&self.device_path);
|
||||
let new_fd = unsafe {
|
||||
libc::open(
|
||||
c_path.as_ptr() as *const libc::c_char,
|
||||
libc::O_RDWR | libc::O_NONBLOCK | libc::O_CLOEXEC,
|
||||
)
|
||||
};
|
||||
self.fd = if new_fd >= 0 { new_fd } else { -1 };
|
||||
|
||||
return Err(Error::ScsiError {
|
||||
opcode: cdb[0],
|
||||
status: 0xFF,
|
||||
@@ -454,15 +430,11 @@ const SENSE_KEY_NOT_READY: u8 = 2;
|
||||
/// realistic homelab (typical PERC + USB optical = ≤8 nodes).
|
||||
const SG_FALLBACK_MAX: u8 = 16;
|
||||
|
||||
/// SCSI INQUIRY response field offsets (SPC-4, 6-byte standard CDB
|
||||
/// returning 96 bytes). Used to populate `DriveInfo` fields without
|
||||
/// magic-number arithmetic at the call site.
|
||||
const INQUIRY_VENDOR_OFFSET: usize = 8;
|
||||
const INQUIRY_VENDOR_LEN: usize = 8;
|
||||
const INQUIRY_MODEL_OFFSET: usize = 16;
|
||||
const INQUIRY_MODEL_LEN: usize = 16;
|
||||
const INQUIRY_FIRMWARE_OFFSET: usize = 32;
|
||||
const INQUIRY_FIRMWARE_LEN: usize = 4;
|
||||
// SCSI INQUIRY field-offset constants previously lived here. They were
|
||||
// used by an in-process SCSI INQUIRY parse path that 0.13.6 retired in
|
||||
// favour of reading the kernel-cached sysfs identity (vendor/model/rev
|
||||
// under /sys/class/scsi_generic/sgN/device/). Removed to keep clippy
|
||||
// -D warnings clean.
|
||||
|
||||
pub(super) fn list_drives() -> Vec<super::DriveInfo> {
|
||||
let mut out = Vec::new();
|
||||
|
||||
+1
-17
@@ -260,23 +260,7 @@ impl MacScsiTransport {
|
||||
})
|
||||
}
|
||||
|
||||
/// Reset the drive to a known good state.
|
||||
/// On macOS, we open the device, release exclusive access, wait for
|
||||
/// the system to reclaim it, then the next open() re-acquires.
|
||||
/// IOKit's USB layer handles device-level resets internally when the
|
||||
/// exclusive access is released and re-acquired.
|
||||
///
|
||||
/// NOTE: untested — macOS reset may need IOUSBDeviceInterface::ResetDevice()
|
||||
/// for USB drives. This is a best-effort implementation.
|
||||
pub fn reset(device: &Path) -> Result<()> {
|
||||
// Opening and immediately dropping triggers release of exclusive access
|
||||
// which forces IOKit to reset the device state.
|
||||
if let Ok(transport) = Self::open(device) {
|
||||
drop(transport); // Drop releases exclusive access + closes plugin
|
||||
}
|
||||
std::thread::sleep(std::time::Duration::from_secs(2));
|
||||
Ok(())
|
||||
}
|
||||
// `reset()` removed in 0.13.6 — see scsi/mod.rs for rationale.
|
||||
}
|
||||
|
||||
/// Enumerate optical drives on macOS. Mirrors `drive::macos::find_drives`
|
||||
|
||||
+7
-97
@@ -98,103 +98,13 @@ pub fn open(device: &Path) -> Result<Box<dyn ScsiTransport>> {
|
||||
}
|
||||
}
|
||||
|
||||
/// Default upper bound on `scsi::reset()`. The platform-specific reset
|
||||
/// sequence does ~15 s of bounded sleeps + ioctls in the happy path, so
|
||||
/// 30 s is roughly 2× the worst-case happy time — long enough that a
|
||||
/// healthy-but-slow drive isn't false-positively timed out, short enough
|
||||
/// that a kernel-wedged ioctl doesn't take down the caller's poll loop
|
||||
/// for minutes.
|
||||
pub(crate) const DEFAULT_RESET_TIMEOUT_SECS: u64 = 30;
|
||||
|
||||
/// Reset a SCSI device to a known good state, with a hard wallclock
|
||||
/// bound (`DEFAULT_RESET_TIMEOUT_SECS`). On Linux: open/close fd cycle +
|
||||
/// TUR + SG_SCSI_RESET escalation.
|
||||
///
|
||||
/// **Why the timeout matters.** SG_SCSI_RESET is an ioctl that can block
|
||||
/// indefinitely on a kernel-wedged USB target (the kernel waits for the
|
||||
/// SCSI subsystem to ack the reset, which never comes from a dead-bus
|
||||
/// device). Without an outer bound, the call hangs the caller's thread
|
||||
/// forever — observed in production on a wedged BU40N where the autorip
|
||||
/// poll loop sat in the ioctl for 60+ s. The bounded version returns
|
||||
/// `DeviceResetFailed` after `DEFAULT_RESET_TIMEOUT_SECS`; the inner
|
||||
/// thread keeps running until the kernel eventually unblocks it (we
|
||||
/// can't cancel a Linux ioctl from userspace), so this leaks one OS
|
||||
/// thread per wedge — acceptable cost for a daemon that recovers
|
||||
/// instead of hanging.
|
||||
///
|
||||
/// `pub(crate)` — outside callers use the higher-level `drive_has_disc`
|
||||
/// (which folds in recovery escalation) or `Drive::reset` (instance-level).
|
||||
/// Direct primitive exposure removed in 0.13.2 to enforce the
|
||||
/// architectural rule that no consumer crate issues SCSI commands.
|
||||
pub(crate) fn reset(device: &Path) -> Result<()> {
|
||||
reset_with_timeout(
|
||||
device,
|
||||
std::time::Duration::from_secs(DEFAULT_RESET_TIMEOUT_SECS),
|
||||
)
|
||||
}
|
||||
|
||||
/// Reset with a caller-specified timeout. See [`reset`] for the full
|
||||
/// rationale on why an outer wallclock bound is required.
|
||||
pub(crate) fn reset_with_timeout(device: &Path, timeout: std::time::Duration) -> Result<()> {
|
||||
let device_owned = device.to_path_buf();
|
||||
let (tx, rx) = std::sync::mpsc::channel();
|
||||
|
||||
// Detach a worker thread for the actual reset. We never `join` it —
|
||||
// if the kernel ioctl is wedged, the join would block forever, which
|
||||
// is the very thing we're protecting the caller from. The thread will
|
||||
// exit on its own when the kernel eventually returns from the ioctl
|
||||
// (or never, if the device is permanently dead — process exit cleans
|
||||
// it up).
|
||||
std::thread::Builder::new()
|
||||
.name("scsi-reset".into())
|
||||
.spawn(move || {
|
||||
let r = reset_blocking(&device_owned);
|
||||
let _ = tx.send(r);
|
||||
})
|
||||
.map_err(|_| Error::DeviceResetFailed {
|
||||
path: device.display().to_string(),
|
||||
})?;
|
||||
|
||||
match rx.recv_timeout(timeout) {
|
||||
Ok(result) => result,
|
||||
Err(std::sync::mpsc::RecvTimeoutError::Timeout) => Err(Error::DeviceResetFailed {
|
||||
path: device.display().to_string(),
|
||||
}),
|
||||
Err(std::sync::mpsc::RecvTimeoutError::Disconnected) => {
|
||||
// The worker thread panicked before sending. Surface as a
|
||||
// reset failure rather than a generic error.
|
||||
Err(Error::DeviceResetFailed {
|
||||
path: device.display().to_string(),
|
||||
})
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/// Inner reset — runs on the worker thread. May block indefinitely if
|
||||
/// the kernel's SCSI subsystem is wedged. Callers must use the bounded
|
||||
/// `reset()` wrapper above; this raw function isn't exposed.
|
||||
fn reset_blocking(device: &Path) -> Result<()> {
|
||||
#[cfg(target_os = "linux")]
|
||||
{
|
||||
linux::SgIoTransport::reset(device)
|
||||
}
|
||||
|
||||
#[cfg(target_os = "macos")]
|
||||
{
|
||||
macos::MacScsiTransport::reset(device)
|
||||
}
|
||||
|
||||
#[cfg(target_os = "windows")]
|
||||
{
|
||||
windows::SptiTransport::reset(device)
|
||||
}
|
||||
|
||||
#[cfg(not(any(target_os = "linux", target_os = "macos", target_os = "windows")))]
|
||||
{
|
||||
let _ = device;
|
||||
Ok(())
|
||||
}
|
||||
}
|
||||
// Note: a top-level `scsi::reset()` used to live here, wrapping a
|
||||
// platform reset in a thread+recv_timeout so a kernel-wedged ioctl
|
||||
// couldn't hang the caller. Removed in 0.13.6 along with the
|
||||
// SG_SCSI_RESET / STOP+START UNIT escalation that needed it. The
|
||||
// remaining platform reset (Linux: SgIoTransport::reset, called only
|
||||
// from SgIoTransport::open) does pure userspace state cleanup with
|
||||
// bounded sleeps — no escape-hatch wrapper required.
|
||||
|
||||
// ── USB-layer recovery: rolled back in 0.13.4 ───────────────────────────────
|
||||
//
|
||||
|
||||
@@ -0,0 +1,504 @@
|
||||
//! Integration tests for progress reporting, halt behavior, drop safety,
|
||||
//! and the file-backed sector reader round trip.
|
||||
|
||||
use libfreemkv::disc::{CopyOptions, DiscRegion};
|
||||
use libfreemkv::error::Result;
|
||||
use libfreemkv::pes::Stream as PesStream;
|
||||
use libfreemkv::{
|
||||
ContentFormat, Disc, DiscFormat, DiscStream, DiscTitle, EventKind, Extent, FileSectorReader,
|
||||
SectorReader,
|
||||
};
|
||||
use std::io::Write;
|
||||
use std::sync::atomic::{AtomicBool, AtomicU64, Ordering};
|
||||
use std::sync::{Arc, Mutex};
|
||||
use std::time::{Duration, Instant};
|
||||
|
||||
const SECTOR_SIZE: usize = 2048;
|
||||
|
||||
// ── helpers ────────────────────────────────────────────────────────────────
|
||||
|
||||
/// Returns zeroed sectors. Always succeeds. Counts each call.
|
||||
struct ZeroSectorReader {
|
||||
capacity: u32,
|
||||
calls: Arc<AtomicU64>,
|
||||
}
|
||||
|
||||
impl ZeroSectorReader {
|
||||
fn new(capacity: u32) -> Self {
|
||||
Self {
|
||||
capacity,
|
||||
calls: Arc::new(AtomicU64::new(0)),
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl SectorReader for ZeroSectorReader {
|
||||
fn read_sectors(
|
||||
&mut self,
|
||||
_lba: u32,
|
||||
count: u16,
|
||||
buf: &mut [u8],
|
||||
_recovery: bool,
|
||||
) -> Result<usize> {
|
||||
self.calls.fetch_add(1, Ordering::Relaxed);
|
||||
let bytes = count as usize * SECTOR_SIZE;
|
||||
buf[..bytes].fill(0);
|
||||
Ok(bytes)
|
||||
}
|
||||
|
||||
fn capacity(&self) -> u32 {
|
||||
self.capacity
|
||||
}
|
||||
}
|
||||
|
||||
/// Like ZeroSectorReader but sleeps a configurable duration per call.
|
||||
/// Used by the halt test so the copy takes >1 s.
|
||||
struct SlowZeroSectorReader {
|
||||
capacity: u32,
|
||||
sleep_per_call: Duration,
|
||||
}
|
||||
|
||||
impl SlowZeroSectorReader {
|
||||
fn new(capacity: u32, sleep_per_call: Duration) -> Self {
|
||||
Self {
|
||||
capacity,
|
||||
sleep_per_call,
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
impl SectorReader for SlowZeroSectorReader {
|
||||
fn read_sectors(
|
||||
&mut self,
|
||||
_lba: u32,
|
||||
count: u16,
|
||||
buf: &mut [u8],
|
||||
_recovery: bool,
|
||||
) -> Result<usize> {
|
||||
std::thread::sleep(self.sleep_per_call);
|
||||
let bytes = count as usize * SECTOR_SIZE;
|
||||
buf[..bytes].fill(0);
|
||||
Ok(bytes)
|
||||
}
|
||||
|
||||
fn capacity(&self) -> u32 {
|
||||
self.capacity
|
||||
}
|
||||
}
|
||||
|
||||
/// Build a Disc instance with a known capacity, no titles, no encryption.
|
||||
/// Sufficient for `Disc::copy` (which only uses capacity_sectors + decrypt keys).
|
||||
fn synthetic_disc(capacity_sectors: u32) -> Disc {
|
||||
Disc {
|
||||
volume_id: String::new(),
|
||||
meta_title: None,
|
||||
format: DiscFormat::BluRay,
|
||||
capacity_sectors,
|
||||
capacity_bytes: capacity_sectors as u64 * SECTOR_SIZE as u64,
|
||||
layers: 1,
|
||||
titles: Vec::new(),
|
||||
region: DiscRegion::Free,
|
||||
aacs: None,
|
||||
css: None,
|
||||
encrypted: false,
|
||||
content_format: ContentFormat::BdTs,
|
||||
}
|
||||
}
|
||||
|
||||
/// Build a DiscTitle with a single extent of `sector_count` sectors and no
|
||||
/// streams (DiscStream still iterates sectors and would emit BytesRead).
|
||||
fn synthetic_title(sector_count: u32) -> DiscTitle {
|
||||
DiscTitle {
|
||||
playlist: String::new(),
|
||||
playlist_id: 0,
|
||||
duration_secs: 0.0,
|
||||
size_bytes: sector_count as u64 * SECTOR_SIZE as u64,
|
||||
clips: Vec::new(),
|
||||
streams: Vec::new(),
|
||||
chapters: Vec::new(),
|
||||
extents: vec![Extent {
|
||||
start_lba: 0,
|
||||
sector_count,
|
||||
}],
|
||||
content_format: ContentFormat::BdTs,
|
||||
codec_privates: Vec::new(),
|
||||
}
|
||||
}
|
||||
|
||||
// ── 1. BytesRead events emitted during disc copy (TDD red) ────────────────
|
||||
|
||||
#[test]
|
||||
fn test_bytes_read_emitted_during_disc_copy() {
|
||||
// Build a tiny synthetic disc and stream it through DiscStream.
|
||||
let reader = ZeroSectorReader::new(64);
|
||||
let title = synthetic_title(64);
|
||||
let keys = libfreemkv::DecryptKeys::None;
|
||||
|
||||
let mut stream = DiscStream::new(Box::new(reader), title, keys, 60, ContentFormat::BdTs);
|
||||
|
||||
let count = Arc::new(AtomicU64::new(0));
|
||||
let count_cb = count.clone();
|
||||
stream.on_event(move |ev| {
|
||||
if let EventKind::BytesRead { .. } = ev.kind {
|
||||
count_cb.fetch_add(1, Ordering::Relaxed);
|
||||
}
|
||||
});
|
||||
|
||||
// Drive the stream to EOF. With no streams configured, read() returns
|
||||
// Ok(None) once all extents are exhausted.
|
||||
loop {
|
||||
match stream.read() {
|
||||
Ok(Some(_frame)) => {}
|
||||
Ok(None) => break,
|
||||
Err(e) => panic!("stream read failed: {e:?}"),
|
||||
}
|
||||
}
|
||||
|
||||
let n = count.load(Ordering::Relaxed);
|
||||
// EXPECTED TO FAIL until BytesRead emission is wired up. TDD red.
|
||||
assert!(
|
||||
n > 0,
|
||||
"expected at least one BytesRead event, got {n} (lib does not yet emit BytesRead)"
|
||||
);
|
||||
}
|
||||
|
||||
// ── 2. Disc::copy on_progress callback fires (regression guard) ───────────
|
||||
|
||||
#[test]
|
||||
fn test_disc_copy_progress_callback_fires() {
|
||||
let disc = synthetic_disc(64);
|
||||
let mut reader = ZeroSectorReader::new(64);
|
||||
|
||||
let tmp = tempfile::NamedTempFile::new().expect("tempfile create");
|
||||
let iso_path = tmp.path().to_path_buf();
|
||||
drop(tmp); // we want the path, not the file handle
|
||||
|
||||
let calls = Arc::new(AtomicU64::new(0));
|
||||
let last_bytes = Arc::new(AtomicU64::new(0));
|
||||
let calls_cb = calls.clone();
|
||||
let last_bytes_cb = last_bytes.clone();
|
||||
|
||||
let progress = move |bytes: u64, _total: u64| {
|
||||
calls_cb.fetch_add(1, Ordering::Relaxed);
|
||||
last_bytes_cb.store(bytes, Ordering::Relaxed);
|
||||
};
|
||||
|
||||
let opts = CopyOptions {
|
||||
decrypt: false,
|
||||
on_progress: Some(&progress),
|
||||
..Default::default()
|
||||
};
|
||||
|
||||
let result = disc.copy(&mut reader, &iso_path, &opts).expect("copy ok");
|
||||
|
||||
// Cleanup any sidecar mapfile + ISO before assertions.
|
||||
let _ = std::fs::remove_file(&iso_path);
|
||||
let _ = std::fs::remove_file(libfreemkv::disc::mapfile_path_for(&iso_path));
|
||||
|
||||
assert!(result.complete, "copy should be complete");
|
||||
let n = calls.load(Ordering::Relaxed);
|
||||
let last = last_bytes.load(Ordering::Relaxed);
|
||||
assert!(n > 0, "on_progress should fire at least once, got {n}");
|
||||
assert!(
|
||||
last > 0,
|
||||
"final progress bytes should be non-zero, got {last}"
|
||||
);
|
||||
}
|
||||
|
||||
// ── 3. Halt aborts disc copy promptly ─────────────────────────────────────
|
||||
|
||||
#[test]
|
||||
fn test_halt_aborts_disc_copy_promptly() {
|
||||
// 6000 sectors, 60-sector batches → 100 read_sectors() calls.
|
||||
// 10 ms sleep per call → ~1 s total without halt.
|
||||
let capacity_sectors: u32 = 6000;
|
||||
let mut reader = SlowZeroSectorReader::new(capacity_sectors, Duration::from_millis(10));
|
||||
let disc = synthetic_disc(capacity_sectors);
|
||||
|
||||
let tmp = tempfile::NamedTempFile::new().expect("tempfile create");
|
||||
let iso_path = tmp.path().to_path_buf();
|
||||
drop(tmp);
|
||||
|
||||
let halt = Arc::new(AtomicBool::new(false));
|
||||
let halt_for_thread = halt.clone();
|
||||
let iso_path_for_thread = iso_path.clone();
|
||||
|
||||
let join = std::thread::spawn(move || {
|
||||
let opts = CopyOptions {
|
||||
decrypt: false,
|
||||
halt: Some(halt_for_thread),
|
||||
..Default::default()
|
||||
};
|
||||
let t0 = Instant::now();
|
||||
let res = disc.copy(&mut reader, &iso_path_for_thread, &opts);
|
||||
(res, t0.elapsed())
|
||||
});
|
||||
|
||||
// Let copy run, then halt.
|
||||
std::thread::sleep(Duration::from_millis(200));
|
||||
halt.store(true, Ordering::Relaxed);
|
||||
|
||||
// Bound the join: should exit far before the full 1 s otherwise needed.
|
||||
let started = Instant::now();
|
||||
let mut joined = None;
|
||||
while started.elapsed() < Duration::from_millis(2000) {
|
||||
if join.is_finished() {
|
||||
joined = Some(join.join().expect("thread join"));
|
||||
break;
|
||||
}
|
||||
std::thread::sleep(Duration::from_millis(20));
|
||||
}
|
||||
let (result, elapsed) = joined.expect("copy thread did not exit within 2s of halt");
|
||||
|
||||
// Cleanup
|
||||
let _ = std::fs::remove_file(&iso_path);
|
||||
let _ = std::fs::remove_file(libfreemkv::disc::mapfile_path_for(&iso_path));
|
||||
|
||||
let copy_result = result.expect("copy returns Ok with halted=true on halt");
|
||||
assert!(
|
||||
copy_result.halted,
|
||||
"copy_result.halted should be true after halt"
|
||||
);
|
||||
assert!(
|
||||
!copy_result.complete,
|
||||
"copy_result.complete should be false when halted"
|
||||
);
|
||||
assert!(
|
||||
elapsed < Duration::from_millis(2000),
|
||||
"copy thread exit elapsed {elapsed:?} exceeded 2s"
|
||||
);
|
||||
}
|
||||
|
||||
// ── 4. DiscStream Drop does not panic or block ────────────────────────────
|
||||
|
||||
#[test]
|
||||
fn test_drop_impls_do_not_panic_or_block() {
|
||||
let reader = ZeroSectorReader::new(64);
|
||||
let title = synthetic_title(64);
|
||||
let keys = libfreemkv::DecryptKeys::None;
|
||||
let stream = DiscStream::new(Box::new(reader), title, keys, 60, ContentFormat::BdTs);
|
||||
|
||||
// Drop on a worker thread; main thread enforces the timeout.
|
||||
let handle = std::thread::spawn(move || {
|
||||
drop(stream);
|
||||
});
|
||||
|
||||
let started = Instant::now();
|
||||
while started.elapsed() < Duration::from_millis(100) {
|
||||
if handle.is_finished() {
|
||||
handle.join().expect("drop thread join");
|
||||
return;
|
||||
}
|
||||
std::thread::sleep(Duration::from_millis(5));
|
||||
}
|
||||
panic!("DiscStream drop did not complete within 100ms");
|
||||
}
|
||||
|
||||
// ── 5. FileSectorReader round trip ────────────────────────────────────────
|
||||
|
||||
#[test]
|
||||
fn test_file_sector_reader_round_trip() {
|
||||
// Build 8 sectors of pseudo-random bytes (sector-aligned).
|
||||
const N_SECTORS: usize = 8;
|
||||
let mut data = vec![0u8; N_SECTORS * SECTOR_SIZE];
|
||||
for (i, b) in data.iter_mut().enumerate() {
|
||||
// Cheap PRNG: just a multiplicative pattern, deterministic for asserts.
|
||||
*b = ((i as u64).wrapping_mul(2654435761) >> 16) as u8;
|
||||
}
|
||||
|
||||
let mut tmp = tempfile::NamedTempFile::new().expect("tempfile create");
|
||||
tmp.write_all(&data).expect("write data");
|
||||
tmp.flush().expect("flush");
|
||||
|
||||
let path = tmp.path().to_str().expect("path utf-8").to_string();
|
||||
let mut fsr = FileSectorReader::open(&path).expect("open FileSectorReader");
|
||||
|
||||
assert_eq!(fsr.capacity(), N_SECTORS as u32, "capacity mismatch");
|
||||
|
||||
// Read each sector individually and compare.
|
||||
let mut buf = vec![0u8; SECTOR_SIZE];
|
||||
for lba in 0..N_SECTORS as u32 {
|
||||
let n = fsr
|
||||
.read_sectors(lba, 1, &mut buf, false)
|
||||
.expect("read_sectors");
|
||||
assert_eq!(n, SECTOR_SIZE);
|
||||
let off = lba as usize * SECTOR_SIZE;
|
||||
assert_eq!(
|
||||
&buf[..],
|
||||
&data[off..off + SECTOR_SIZE],
|
||||
"sector {lba} mismatch"
|
||||
);
|
||||
}
|
||||
|
||||
// Read all sectors at once and compare.
|
||||
let mut all = vec![0u8; N_SECTORS * SECTOR_SIZE];
|
||||
let n = fsr
|
||||
.read_sectors(0, N_SECTORS as u16, &mut all, false)
|
||||
.expect("read all sectors");
|
||||
assert_eq!(n, N_SECTORS * SECTOR_SIZE);
|
||||
assert_eq!(all, data, "bulk read mismatch");
|
||||
}
|
||||
|
||||
// ── 6. Disc::copy stall detection triggers skip-forward (TDD red) ─────────
|
||||
//
|
||||
// Regression guard for the Dell-host hang where `read_sectors` blocked inside
|
||||
// a kernel-level USB stall and `Disc::copy` sat frozen for 10+ minutes with
|
||||
// no progress and no error. The fix introduces `CopyOptions::stall_secs:
|
||||
// Option<u64>` — when elapsed-since-last-`bytes_good`-advance exceeds the
|
||||
// threshold, `Disc::copy` treats the current block as a read failure and
|
||||
// triggers the skip-forward path so the rip can advance.
|
||||
//
|
||||
// THIS TEST IS EXPECTED TO FAIL UNTIL THE PARALLEL FIX LANDS.
|
||||
// - Until `stall_secs` exists on `CopyOptions`, the test will not compile.
|
||||
// - Once the field exists but the stall guard isn't wired, the spawned copy
|
||||
// thread will never exit (test fails on the 5s join bound).
|
||||
// - Once the guard is wired, copy returns within ~stall_secs with
|
||||
// `complete=false` and `bytes_pending>0`.
|
||||
|
||||
/// Reader that returns Ok for sectors `< block_after`, then returns Err for
|
||||
/// any sector `>= block_after` after a small per-call delay. Models the
|
||||
/// realistic Dell-host symptom: reads keep returning Err (skip-forward fires)
|
||||
/// but no `bytes_good` ever accrues; without a stall guard, Pass 1 grinds
|
||||
/// silently for tens of minutes.
|
||||
struct StallingSectorReader {
|
||||
capacity: u32,
|
||||
block_after: u32,
|
||||
/// Per-call delay for sectors >= block_after (simulates slow reads).
|
||||
err_delay_ms: u64,
|
||||
release: Arc<AtomicBool>,
|
||||
/// Retained so callers can release the reader; unused now that the
|
||||
/// reader returns Err instead of blocking, but kept so the test's
|
||||
/// existing release plumbing compiles.
|
||||
park: Arc<(Mutex<()>, std::sync::Condvar)>,
|
||||
}
|
||||
|
||||
impl StallingSectorReader {
|
||||
fn new(capacity: u32, block_after: u32) -> Self {
|
||||
Self {
|
||||
capacity,
|
||||
block_after,
|
||||
err_delay_ms: 100,
|
||||
release: Arc::new(AtomicBool::new(false)),
|
||||
park: Arc::new((Mutex::new(()), std::sync::Condvar::new())),
|
||||
}
|
||||
}
|
||||
|
||||
fn release_handle(&self) -> (Arc<AtomicBool>, Arc<(Mutex<()>, std::sync::Condvar)>) {
|
||||
(self.release.clone(), self.park.clone())
|
||||
}
|
||||
}
|
||||
|
||||
impl SectorReader for StallingSectorReader {
|
||||
fn read_sectors(
|
||||
&mut self,
|
||||
lba: u32,
|
||||
count: u16,
|
||||
buf: &mut [u8],
|
||||
_recovery: bool,
|
||||
) -> Result<usize> {
|
||||
if lba >= self.block_after {
|
||||
// Realistic stall model: read takes err_delay_ms then returns
|
||||
// Err. With skip_on_error+skip_forward, Disc::copy will keep
|
||||
// skip-forwarding through this region — no bytes_good accrues.
|
||||
// The stall guard fires when bytes_good is unchanged for
|
||||
// stall_secs.
|
||||
std::thread::sleep(Duration::from_millis(self.err_delay_ms));
|
||||
return Err(libfreemkv::error::Error::DiscRead { sector: lba as u64 });
|
||||
}
|
||||
let bytes = count as usize * SECTOR_SIZE;
|
||||
buf[..bytes].fill(0);
|
||||
Ok(bytes)
|
||||
}
|
||||
|
||||
fn capacity(&self) -> u32 {
|
||||
self.capacity
|
||||
}
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn test_disc_copy_stall_detection_triggers_skip_forward() {
|
||||
// 1024 sectors total. Reader serves the first 64 sectors instantly, then
|
||||
// every later read blocks forever. With stall_secs=2, copy should bail
|
||||
// out of the stalled block within ~2s and either skip forward or finish
|
||||
// with bytes_pending > 0 / complete=false.
|
||||
let capacity_sectors: u32 = 1024;
|
||||
let block_after: u32 = 64;
|
||||
|
||||
let reader = StallingSectorReader::new(capacity_sectors, block_after);
|
||||
let (release_flag, park) = reader.release_handle();
|
||||
let mut reader = reader;
|
||||
|
||||
let disc = synthetic_disc(capacity_sectors);
|
||||
|
||||
let tmp = tempfile::NamedTempFile::new().expect("tempfile create");
|
||||
let iso_path = tmp.path().to_path_buf();
|
||||
drop(tmp);
|
||||
|
||||
let iso_path_for_thread = iso_path.clone();
|
||||
|
||||
let join = std::thread::spawn(move || {
|
||||
let opts = CopyOptions {
|
||||
decrypt: false,
|
||||
skip_on_error: true,
|
||||
skip_forward: true,
|
||||
// ASSUMPTION: parallel fix adds `pub stall_secs: Option<u64>` to
|
||||
// CopyOptions. If the field name differs, update here.
|
||||
stall_secs: Some(2),
|
||||
..Default::default()
|
||||
};
|
||||
let t0 = Instant::now();
|
||||
let res = disc.copy(&mut reader, &iso_path_for_thread, &opts);
|
||||
(res, t0.elapsed())
|
||||
});
|
||||
|
||||
// Bound the join to ~5s. With stall_secs=2 the copy should exit well
|
||||
// within this window. If it doesn't, the stall guard isn't working.
|
||||
let started = Instant::now();
|
||||
let mut joined = None;
|
||||
while started.elapsed() < Duration::from_millis(5000) {
|
||||
if join.is_finished() {
|
||||
joined = Some(join.join().expect("thread join"));
|
||||
break;
|
||||
}
|
||||
std::thread::sleep(Duration::from_millis(50));
|
||||
}
|
||||
|
||||
// Whether or not the join succeeded, release the parked reader thread so
|
||||
// it can exit (its &mut reader is owned by the spawned thread; releasing
|
||||
// lets that thread unwind cleanly).
|
||||
release_flag.store(true, Ordering::Relaxed);
|
||||
park.1.notify_all();
|
||||
|
||||
let (result, elapsed) = match joined {
|
||||
Some(v) => v,
|
||||
None => {
|
||||
// Wait a bit longer for the thread to drain after release so we
|
||||
// don't leave it dangling, then fail the test.
|
||||
std::thread::sleep(Duration::from_millis(500));
|
||||
panic!(
|
||||
"Disc::copy did not return within 5s of stall_secs=2 — \
|
||||
stall guard not wired (TDD red until fix lands)"
|
||||
);
|
||||
}
|
||||
};
|
||||
|
||||
// Cleanup
|
||||
let _ = std::fs::remove_file(&iso_path);
|
||||
let _ = std::fs::remove_file(libfreemkv::disc::mapfile_path_for(&iso_path));
|
||||
|
||||
let copy_result = result.expect("copy returns Ok with stall handling");
|
||||
|
||||
assert!(
|
||||
elapsed < Duration::from_millis(5000),
|
||||
"copy elapsed {elapsed:?} exceeded 5s bound (stall_secs=2)"
|
||||
);
|
||||
assert!(
|
||||
copy_result.bytes_pending > 0,
|
||||
"expected bytes_pending > 0 after stall-triggered skip; got {}",
|
||||
copy_result.bytes_pending
|
||||
);
|
||||
assert!(
|
||||
!copy_result.complete,
|
||||
"expected complete=false after stall-triggered skip"
|
||||
);
|
||||
}
|
||||
Reference in New Issue
Block a user