diff --git a/CHANGELOG.md b/CHANGELOG.md index bd78b23..bb7bee9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -1,2712 +1,109 @@ # Changelog -## [1.0.0-rc.1] - UNRELEASED +## [1.0.0-rc.2] -First release candidate for 1.0. +Second release candidate for 1.0. libfreemkv is the core library: disc scan, +multipass sector recovery, content decryption (CSS, AACS 1.0/2.0), and the +threaded mux pipeline that turns a disc or ISO into an MKV. This candidate adds +keyless DVD/CSS support and correct DVD video, on top of security and recovery +hardening. ### Added -- **Keyless DVD/CSS title-key recovery.** The CSS title key is now recovered - directly from the scrambled disc data via the Stevenson known-plaintext - attack (ported from libdvdcss), so a CSS-protected DVD decrypts with no key - database. The recovered key is validated by descrambling a scrambled sector - and confirming the known plaintext reappears, so a wrong key fails cleanly - instead of producing silent garbage (`src/css/stevenson.rs`, `lfsr.rs`, - `tables.rs`). The bus-authentication handshake in `src/css/auth.rs` is - retained solely to unlock scrambled-sector reads on CSS-enforcing drives; - its result is not used for title-key derivation. +- **Keyless DVD/CSS title-key recovery.** A CSS-protected DVD decrypts with no + key database — the title key is recovered directly from the scrambled disc + data via the Stevenson known-plaintext attack (ported from libdvdcss) and + validated by descrambling a sector and confirming the known plaintext + reappears, so a wrong key fails cleanly instead of producing silent garbage + (`src/css/stevenson.rs`). `Disc::scan_image` recovers the same title key from + a raw, still-scrambled CSS ISO, so a raw image can be muxed without + pre-decryption. - **MPEG-2 Program-Stream access-unit reassembler** (`src/mux/codec/mpeg2.rs`). Buffers elementary-stream bytes across PES packets and emits exactly one - coded picture per MKV block. Reconstructs presentation timestamps from - `temporal_reference` and the sequence-header frame rate, anchored to the - PES PTS; frames emitted before the first timestamp anchor are buffered and - back-anchored. Includes an 8 MiB buffer cap so a corrupt stream cannot - exhaust memory. -- `Disc::scan_image` recovers the CSS title key from a raw, still-scrambled - DVD image, so a raw CSS ISO can be muxed without pre-decryption. + coded picture per MKV block, with presentation timestamps reconstructed from + the stream — fixing corrupted DVD video. Bounded buffer so a malformed stream + cannot exhaust memory. ### Changed -- DVD CSS authentication is now driven directly off the main title's first - sector instead of detecting CSS via an unauthenticated scrambled read, which - a CSS-enforcing drive rejects before auth can run. UHD/Blu-ray is unaffected. -- Drive initialization skips the firmware unlock when the loaded disc is a DVD - profile, running the drive in stock mode so CSS authentication succeeds. - Blu-ray/UHD is unchanged (`src/drive/mod.rs`). -- Param-set application emits self-contained keyframes: the active VPS/SPS/PPS - (HEVC), SPS/PPS (H.264), and sequence/entry headers (VC-1) are re-asserted - at every keyframe/RAP, and any param-set change (including a revert to the - codecPrivate set) is emitted in-band. Fixes whole-segment HEVC/H.264/VC-1 - corruption when a source stops repeating an unchanged param set or reverts - one mid-title. -- Strictly-monotonic block-timestamp adjustment is keyed on track type rather - than index, so a second video track (e.g. a Dolby Vision enhancement layer) - keeps its true B-frame presentation timestamps instead of being clobbered. -- Mux unit alignment is scheme-aware (AACS = 3 sectors, CSS/none = 1 sector), - so DVD IFO extents that are not 3-sector multiples are no longer rejected - with `ExtentNotUnitAligned`. +- Self-contained keyframes: the active param sets (HEVC VPS/SPS/PPS, H.264 + SPS/PPS, VC-1 sequence/entry headers) are re-asserted at every keyframe and + any mid-title param-set change is emitted in-band, fixing whole-segment + HEVC/H.264/VC-1 corruption when a source stops repeating or reverts a param + set. +- Block timestamps use presentation order keyed on track type, so B-frame video + (including a Dolby Vision enhancement layer) keeps its true presentation + timestamps instead of decode-order timecodes. +- Mux unit alignment is scheme-aware (AACS vs CSS/none), so DVD extents are no + longer rejected for unit misalignment. - MKV output records `freemkv ` in the Muxing/Writing application - fields, making every output file traceable to its build. -- Matroska codec-ID literals are centralized as `ebml::CODEC_*` constants - (single source of truth for both mux encode and demux decode). -- `MkvStream` `BlockDuration` values are scaled by the segment's `ts_scale_ns` - before being written, so subtitle display durations are correct when the - timecode scale is not 1 ms. -- The keydb decompressed-plaintext reader is capped at 64 MiB, preventing a - malformed or zip-bombed download from exhausting memory. -- The NOT_READY retry pause in the patch (Pass N) loop is now halt-responsive: - a stop request interrupts the 15-second drive-recovery wait immediately - instead of blocking the shutdown path. -- CSS Stevenson attack: periodic-extension crib and first-match-and-break on - a valid key candidate, matching the libdvdcss reference implementation. + fields, so every output file is traceable to its build. +- Subtitle `BlockDuration` values are scaled by the segment timecode scale, so + display durations are correct when the scale is not 1 ms. +- The NOT_READY retry pause in the patch (Pass N) loop is halt-responsive: a + stop request interrupts the drive-recovery wait immediately instead of + blocking shutdown. +- Bounded the keydb decompressed-plaintext reader (caps a malformed or + zip-bombed download). ### Fixed -- A `READ(10)` that returns GOOD status with a residual underrun is now treated - as a failed read (routed to NonTrimmed/retry) instead of committing the stale - buffer tail as recovered data — closing a silent-corruption hole in both the - sweep and patch paths. -- `raw_command` on Linux now masks the `DRIVER_SENSE` bit (0x08) from - `driver_status` before treating the result as an error. `DRIVER_SENSE` only - signals that sense data is present, not that the command failed; masking it - prevents false transport errors on commands that return sense alongside a - GOOD response. -- `decode_read_capacity` rejects a `READ CAPACITY (10)` response whose - `last_lba` field is `u32::MAX` (the "capacity exceeds 32-bit" sentinel), - returning `Error::DiscCapacityOverflow` instead of silently wrapping to 0 - and misreporting disc size. +- A `READ(10)` that returns GOOD status with a residual underrun is treated as a + failed read (routed to retry) instead of committing stale buffer data — + closing a silent-corruption hole in the sweep and patch paths. +- `raw_command` on Linux masks the `DRIVER_SENSE` bit before treating a result + as an error, preventing false transport errors on commands that return sense + alongside a GOOD response. +- `READ CAPACITY (10)` rejects the "capacity exceeds 32-bit" sentinel instead of + silently wrapping to 0 and misreporting disc size. ### Security -- CSS disc/title keys are redacted in log output (logged as `` with - a 1-byte fingerprint); a test guards against any key field (`title_key`, - `disc_key`, `player_key`, `unit_key`, `vuk`, `bus_key`) being logged with a - raw value. -- The macOS SCSI shim (`src/scsi/macos_shim.c`) no longer shells out via - `system()` / `sh -c` to invoke `diskutil unmountDisk`. It now uses - `posix_spawn` directly, eliminating a command-injection vector for a - device-path string that contains shell metacharacters. - - -## 0.31.10 (2026-06-18) - -### Performance -- `recover_dk_position`: parallelise the subset-difference slot scan with rayon - (`find_map_any`, first match cancels the rest). A UHD MKB's ~181k-slot - no-match scan drops from ~26s to ~4.6s on an 8-core box. Identical result. - - -## 0.31.9 (2026-06-17) - -### Performance -- Subset-difference PK walk (`calc_pk_from_dk`): at each tree level derive only - the child actually descended into (not both siblings) and compute the - Processing Key once at the final node — ~3x fewer AES block ops per walk. - Bit-for-bit identical output; speeds every Device-Key → Media-Key derivation - (disc decryption AND unpositioned-DK recovery). - - -## 0.31.8 (2026-06-17) - -### Added -- `recover_dk_position(mkb, key) -> Option`: recover an unpositioned - device key's subset-difference position (`node`/`uv`/`u_mask_shift`) by - scanning a disc MKB — the zero-descent slot probe and ancestor walk-up, with - the verify reduced to the hoisted terminal Processing Key. Lets a caller boil - a position-less device key against a real disc. - -### Changed -- Consolidate the SD walk surface: remove the research-only - `probe::walk_pk_against_tables` (the single PK→MK engine is reached via - `derive_media_key_from_pk`); make `derive_media_key_from_pk_walked` internal. - - -## 0.31.7 (2026-06-17) - -### Added -- `derive_media_key_and_pk_from_dk(mkb, dks) -> Option<(media_key, processing_key)>`: - expose the intermediate Processing Key the subset-difference walk already - computes, so callers can record the full key lineage on a Device-Key walk. - -### Changed -- Unify the subset-difference walk: the duplicate `aesg3`/`calc_v_mask`/ - `calc_pk_from_dk` copies are consolidated to one definition. No behavior change. - - -## 0.31.6 (2026-06-09) - -### Fixed - -- UDF `read_file`: the AACS Media Key Block (`MKB_RO.inf`) is allocated to a - fixed ~128 MiB on UHD discs and zero-padded, but the real record stream is - only a few MiB. 0.31.0 added a 64 MiB `MAX_FILE_BYTES` cap on `read_file`, - which rejected the padded MKB outright — so `read_aacs_inputs` failed and the - online key-resolve path reported "could not read this disc's key files" and - **never contacted the keyserver** (a regression on every disc whose MKB - exceeds 64 MiB). The MKB is now read length-aware: a bounded prefix is read, - the real record length is found via the MKB header, and exactly that is - returned — never the padding, and never tripping the cap. Validated - end-to-end against a real UHD ISO (MKB reads and trims to its record length). -- UDF `read_file`: honor inline/embedded allocation descriptors (ICB Tag flags - low bits == 3). Tiny files (some AACS `*.inf` key files) store their data - embedded directly in the ICB with no out-of-line extents; these are now read - from the ICB payload instead of misparsing the embedded bytes as allocation - descriptors (which could hard-error since 0.31.0). - -## 0.31.5 (2026-06-08) - -### Fixed - -- MKV mux: stop forcing strictly-monotonic block timestamps on the **video** - track. The monotonic nudge (added for audio PES that collide on a - millisecond) was clobbering B-frame video PTS — which are legitimately - non-monotonic in decode/storage order — to prev+1ms. A `copy` remux - preserved the wrong value, but decoding derived DTS from the HEVC POC and - found them colliding, emitting thousands of "non monotonically increasing - dts" warnings per title. Video now keeps its true PES PTS (Matroska - SimpleBlock permits non-monotonic block timestamps); audio/subtitle tracks - still get the nudge. Verified: a re-mux drops the warning count to zero. - -## 0.31.4 (2026-06-08) - -Test cleanup — no runtime changes. Removed 144 unit tests flagged in adversarial -review as vacuous (assert-current-behavior / duplicate / non-failing), leaving -the spec-grounded, mutation-verified subset. 1927 tests remain, all green. - -## 0.31.3 (2026-06-08) - -Test fixture hygiene — no runtime changes. Replaced a private-range example -address in a `network://` URL test with an RFC 5737 documentation address so -the public-repo leak-guard stays green. - -## 0.31.2 (2026-06-08) - -Test-hardening release — no runtime changes. Adds a comprehensive, -spec-grounded unit-test suite across the silent-corruption surfaces -(UDF/MPLS/CLPI/IFO parsing, BD/DVD title + extent assembly, AACS/CSS key -handling, TS/PS demux + codec parsers, MKV/EBML container output, the mux -pipeline, sector prefetch + decrypt decorator, drive/SCSI sense decoding, -disc-label extraction, and core I/O). Each test is grounded in the format -spec or real on-disc behavior and verified to fail under a targeted source -mutation, so a future refactor cannot silently regress these paths (the -class of defect behind the 0.31.0 UDF allocation-descriptor truncation). -~950 tests added; no behavior changed. - -## 0.31.1 (2026-06-08) - -Correctness fixes for the file-backed mux and disc AACS-input read paths, -plus regression tests. - -### Fixed - -- UDF: honor the ICB allocation-descriptor type flag. The extent reader - hardcoded an 8-byte Short-AD stride, so files using 16-byte Long ADs (large - BD-ROM `.m2ts` streams) were mis-strided: descriptor #0 read correctly, then - the allocation-list terminator tripped on the zero bytes mid-descriptor and - parsing stopped after the first extent. Every multi-extent title truncated - at ~1 GiB. Short/Long/Extended ADs are now strided correctly (8/16/20 bytes; - Extended ADs carry the extent LBA at offset +12). This repaired both the - file mux (`iso://`) and the disc AACS-input read (`/AACS/*.inf`). -- AACS: never zero an MKB whose content length the parser cannot determine - (`mkb_content_len == 0` now leaves the buffer intact instead of truncating - it to empty). - -### Tests - -- Long-AD `read_icb_extents` and `read_file` coverage, Extended-AD stride, - sparse/terminator handling, continuation-loop bound, UDF name decoding, and - `trim_mkb` empty-guard; bad-sector recovery damage-skip range bounds and - bridge-degradation budget. Each verified to fail under a targeted mutation. - -## 0.31.0 (2026-06-08) - -Hardening and correctness release: a library-wide review-and-fix pass across -the mux pipeline, codec parsers, AACS/CSS decryption, UDF/MPLS/CLPI parsing, -sector prefetch, multi-pass recovery, drive/SCSI, disc labels, and I/O. - -### Fixed - -- AACS: hardened keydb title parsing, redacted secrets from handshake debug - output, and tightened the media-key variant verification gate. A trailing - partial aligned unit is now tolerated when clear and rejected (fail-loud) - when scrambled, rather than passed through. -- Parsers: corrected the MPLS playlist-mark type offset, added per-extent and - cumulative allocation bounds in the UDF reader, and made the drive identity - probe best-effort on a CHECK CONDITION. -- Mux/codec: start-code de-duplication, framing fixes, M2TS packet hardening, - and bounds guards across the HEVC/H.264/VC-1/MPEG-2/TrueHD/DTS/PGS paths. -- Recovery/drive/SCSI: guarded READ CAPACITY short transfers, unified the - error-code / io-kind mapping, and hardened the platform unlock path. -- Robustness: overflow/underflow guards on values derived from untrusted disc - input, deterministic prefetch shutdown, and error display remains code-only. - -### Changed - -- Release profile now builds with thin LTO and a single codegen unit. - -## 0.29.0 (2026-06-06) - -### Fixed - -- **Video frames now carry their presentation timestamp (PTS), not the decode - timestamp (DTS), as the MKV block timecode.** The HEVC, H.264, VC-1 and - MPEG-2 parsers stamped each frame with `dts.or(pts)`, so B-frame titles were - muxed with monotonic decode-order timecodes: players presented frames in - decode order (visible judder / apparent "corruption") and PTS-based seeking - landed on the wrong frame. The compressed video was always byte-correct — - this was purely a timestamp defect, and it affected every B-frame title. The - parsers now use `pts.or(dts)`. Verified against a reference muxer on a real - UHD title: the emitted frame sequence is now identical. -- **DTS-HD MA tracks no longer downgrade to the lossy DTS core** — the access - unit is delimited by the next *validated* core sync, so a `0x7FFE8001` byte - pattern inside the XLL extension can no longer truncate the unit. (#10) -- **A resolved AACS key is verified against the content before it is applied**, - so a stale or wrong VUK can no longer silently produce garbage output. -- **`iso://` mux fails fast when no usable AACS key is available** instead of - writing a garbage MKV. - -### Changed - -- Codec / EBML / TS hardening: bounds-checked PMT and PES-header parsing, EBML - length validation, and bounded codec buffers (DTS, TrueHD, PGS). - -## 0.27.3 (2026-06-04) - -### Added - -- **The mapfile now persists resolved AACS unit keys, not just the Volume ID.** - A keyed disc writes its decrypted unit keys as `# freemkv-uk: :` - comment headers; an unresolved disc writes only the `# freemkv-vid:` marker. - The two are **mutually exclusive** (`Mapfile::set_unit_keys` clears the VID): - unit keys are the final answer, so a deferred-mux / resume decrypts directly - with no key-service round-trip, while the VID alone means "still unresolved — - retry the key service." `CopyOptions`/`SweepOptions` gain a `unit_keys` field - (written when non-empty, else the VID), and `Disc::inject_unit_keys` applies - mapfile-recovered keys to a scanned disc (marking the source `ExternalUk`). - -## 0.27.0 (2026-06-03) - -### Changed - -- **AACS unit-encryption detection now reads the raw MPEG-TS sync bytes - instead of header flag bits.** `is_unit_encrypted` is renamed - `is_aacs_scrambled`: a unit is encrypted iff its body TS packet sync bytes - (`0x47` at offset 4 and every 192 bytes) are not intact — the encrypted - body destroys them. The previous check read the TP_extra copy-control bits - (byte 0) or TS transport_scrambling_control bits (byte 7), which discs and - players do not set reliably. A single predicate now backs both the decrypt - gate and out-of-band key validation, so all callers agree on what - "encrypted" means. Decryption restores the syncs, so a decrypted unit reads - as clear and there is no encryption flag to clear. - -## 0.26.1 (2026-05-22) - -### Added - -- **AACS resolver path 5 — KEYDB unit-keys direct fallback.** When a - KEYDB entry for the disc has no VUK field but does have pre-decrypted - unit keys, the resolver now consumes those unit keys directly instead - of treating the entry as unusable. Covers ~4,572 entries in the - public keydb (≈2.5% of the database, heavily skewed toward MKBv76+ - UHD discs that DVDFab/FindVUK can no longer extract VUKs for). - Surfaces as `KeySource::KeyDbUnitKeys`; `AacsState::vuk` and - `ResolvedKeys::vuk` are `None` on this path because no VUK exists to - return. Partial CPS-unit coverage is rejected so a disc is never - half-decrypted. - -### Changed - -- **AACS resolver path order reordered root-to-leaf.** Paths now run in - derivation-strength order: device-key (1) → processing-key (2) → - KEYDB-derived from MK+VID (3) → KEYDB VUK direct (4) → KEYDB unit - keys direct (5). Previous order was leaf-first (KEYDB hash lookup, - then derivation). Resolution runs once per disc, so the few extra - milliseconds of MKB walking when a KEYDB VUK would have answered are - invisible against the rip itself; the new order matches the AACS - derivation hierarchy. `KeySource` enum variants are reordered to - match. -- **`AacsState::vuk` type is now `Option<[u8; 16]>`** (was `[u8; 16]`). - Required by path 5, which has no VUK. Public-API break. -- **`ResolvedKeys::vuk` type is now `Option<[u8; 16]>`** (was - `[u8; 16]`). Same rationale. - -## 0.25.14 (2026-05-21) - -### Changed - -- `Drive::is_raw_read_active()` is the current name for the - raw-read-capability probe (an older internal name was retired in this - pass). Same semantics; the old name was removed. Mirrored on the - internal `PlatformDriver::is_raw_read_active()` trait method. -- `Error::AacsRawReadUnsupported` is the current name for the - raw-read-unsupported variant; the underlying numeric code (E7016) - is unchanged. The corresponding constant is - `E_AACS_RAW_READ_UNSUPPORTED`. - -No behavioural change — purely a rename pass. - -## 0.25.13 (2026-05-21) - -### Added - -- **`DrmScheme` top-level dispatcher.** New `drm` module with a - `DrmScheme` enum (`Css`, `Aacs10`, `Aacs20`, `Aacs21`) and a - `detect` + `load` pair that uniformly handles all four content - protection schemes. Replaces the inlined dispatch in - `disc::encrypt::resolve_encryption` and the scattered CSS routing - in `disc::mod`. Both CSS call sites now route through the same - entry point. -- **AACS 2.1 Media Key Variant framework.** New `aacs::variants` - module implementing the Media Key Variant derivation chain - (`Kp + C → Kmp → ⊕KCD → Kpnew → Km → VUK`), Variant-scheme MKB - record parsing (record types `0x82` / `0x83`), bit-0x02 SoftKCD - and bit-0x04 online-challenge detection with dedicated error - variants. Wired into `DrmScheme::Aacs21` but the dispatcher arm - is commented out pending validation against a Variant-scheme - disc. Per-manufacturer Key Correction Data must be supplied by - the integrator; `KEY_CORRECTION_DATA_PLACEHOLDER` is the empty - placeholder slot. -- **`AacsVersion` enum.** Replaces the `aacs2: bool` field on - `ContentCertificate`, `UnitKeyFile`, and `ResolvedKeys`. - `parse_unit_key_ro` and `parse_content_cert` now take/emit the - enum. `resolve_keys` is split into `resolve_keys_v1`, - `resolve_keys_v2`, and `resolve_keys_v21` (the last not reachable - from the dispatcher today). - -### Fixed - -- **Raw-read VID shortcut deleted.** v0.25.11 introduced a - `do_handshake` branch that, on raw-read-capable drives, skipped the - AACS cert handshake and issued `READ_DISC_STRUCTURE` format 0x80 - with AGID=0 directly. The hypothesis was that raw-read-capable - drives would serve VID without auth. Empirical test (BU40N - + UHD disc, 2026-05-21) showed the drive returns - `0x05 / 0x6F / 0x02` (`ILLEGAL_REQUEST / Copy protection key - exchange failure: KEY NOT ESTABLISHED`) to that CDB regardless of - raw-read state. The AACS spec requires a successful - `REPORT_KEY` / `SEND_KEY` exchange to establish an AGID before - format 0x80 returns VID; that requirement is enforced by the drive - itself and isn't bypassed by raw-read mode. The shortcut - fired for every raw-read-capable drive, so v0.25.11 / v0.25.12 - MOVIE scans were stuck at E7017 instead of progressing to the - real wall (no DK walks MKB v77). -- `Disc::do_handshake` now always routes through `do_handshake_cert`. - `Drive::is_raw_read_active()` and the Mt1959 marker - detection are kept as informational signals (logged in the - `handshake_entry` warn line) but no longer steer the auth path. -- The raw-read VID read helper was deleted (~50 LOC). - -The corollary: AACS resolution on HRL-burned drives + UHD discs now -fails honestly. Either cert auth succeeds (drive unlock may or -may not bypass the HRL — that's the new empirical question) and we -hit the actual DK wall (E7018 "No DK that walks this MKB" for v77+ -UHD without a v77+ DK in keydb), or cert auth fails and we surface -E7015. Both are real verdicts; E7017's previous spurious dispatch -is gone. - -## 0.25.12 (2026-05-21) - -No libfreemkv source changes — unified sync bump for autorip's -`aacs_failure_message` two-line wording rewrite. See the autorip -v0.25.12 release for details. - -## 0.25.11 (2026-05-21) - -### Added - -- **Raw-read VID path.** When the Mt1959 unlock response - confirms both the active-mode and mode-ID markers, - `Drive::is_raw_read_active()` returns true and `do_handshake` - skips the AACS cert dance entirely — VID is retrieved via - `READ_DISC_STRUCTURE` format 0x80 with AGID=0, and bus encryption - is already off. This is what unblocks UHD ripping on drives whose - host cert is on the AACS HRL. -- New `Error` variants for finer-grained AACS failure reporting: - `AacsHostCertRejected` (E7015), `AacsRawReadUnsupported` - (E7016), `AacsVidUnavailable` (E7017), `AacsMkUnavailable` - (E7018), `AacsVukNotInKeydb` (E7019). Lets CLIs/UIs render which - piece of the AACS chain failed instead of always saying "no keys." - -### Fixed - -- `validate_processing_key` now matches libaacs `_validate_pk` - exactly: XORs `uv` into `mk[12..16]` (was omitted), AES-decrypts - `mk_dv` and checks the `01 23 45 67 89 AB CD EF` magic (was - AES-encrypt + 12-zero check). Pre-fix, every non-zero-uv - processing key was rejected — i.e. essentially every real disc. -- `mkb_find_cvalues` now prefers record type `0x07` (AACS 2.x) and - falls back to `0x05` (AACS 1.0), so the walker handles both - generations without an out-of-band version flag. -- `resolve_keys` short-circuits paths 2/3/4 when VID is the zero - sentinel — saves cycles and emits an honest "VID unavailable" log - instead of the misleading "all paths failed." -- AES-CMAC VID verification gains NIST SP 800-38B KAT + round-trip - + mutation + all-zero-rejection tests. - -### Removed - -- **Built-in AACS keys** (added in 0.25.9). `src/aacs/builtin_keys.rs` - deleted; `KeyDb::with_builtins`, `KeyDb::load_or_builtins`, - `KeyDb::merge_from`, and internal dedup helpers gone. The - compiled-in shortcut was a slim convenience that didn't move the - hard problem (no v77+ DKs) and added a maintenance surface. -- **Operator plugin slot** at `~/.config/freemkv/local_keys.cfg` - (added in 0.25.9). `local_plugin_path` and `KeyDb::merge_local_plugin` - removed. Single source: the main `keydb.cfg`. -- `KeyDb::load` reverts to the pre-0.25.9 form — read the file or - return I/O error; no layering, no fallback. -- `Disc::scan` reverts to surfacing `KeydbLoad { path: "" }` for an encrypted disc with no keydb — same - sentinel autorip's message switch already handles. - -CSS player keys in `src/css/auth.rs` remain compiled in; they're -1999-era public inputs separate from the AACS pipeline and were -never part of the 0.25.9 additions. - -## 0.25.9 (2026-05-20) - -### Fixed - -- **MKB record-type identification.** `mkb_find_mk_dv` and `mkb_version` - had their record-type constants swapped. The Verify Media Key record - is Type 0x81 (AACS 1.0) or Type 0x86 (AACS 2.0/2.1), not 0x10. The - Type-and-Version record is Type 0x10, not 0x81. PK and DK derivation - paths therefore failed silently on every disc, masking how often the - fallback paths could have succeeded. Fixed; mk_dv extracted at - offset 4 of the verify record, MKB version read at offset 8 of the - Type 0x10 record body. Tests added for both forms. - -### Added - -- **Built-in AACS keys.** Four device keys (covering MKB v01-v82+) and - three processing keys (covering v63-v68) are now compiled into - `libfreemkv` directly. DVDs and Blu-rays (AACS 1.0) decrypt with - zero external files. Combined with the existing 31 CSS player keys - in `css/auth.rs`, the library is self-sufficient for all DVD and - AACS 1.0 content. -- **Operator plugin slot.** `~/.config/freemkv/local_keys.cfg` is - loaded automatically (same format as `keydb.cfg`) and layered - additively on top of the built-ins and the main keydb.cfg. For - operators who derive their own keys, this is the additive surface - to drop them into — main keydb.cfg from upstream auto-update stays - separate and overwritable. -- `KeyDb::with_builtins()`, `KeyDb::load_or_builtins()`, - `KeyDb::empty()` constructors. - -### Changed - -- `Disc::scan` no longer errors when keydb.cfg is absent. With - built-ins covering DVD/BD, missing keydb is treated as "no UHD - keys available" — the AACS 2.0 / UHD code path surfaces a - specific error only when the disc actually needs keys that - aren't in built-ins, main keydb, or local plugin. - -## 0.25.8 (2026-05-20) - -### Changed - -- Synchronized to autorip 0.25.8. No libfreemkv code changes — - unified versioning bump only. autorip 0.25.7 image was DOA - (missing +x on the binary); see autorip changelog. - -## 0.25.7 (2026-05-20) - -### Fixed — BU40N firmware wedge on KEYDB-miss discs - -`disc::encrypt::do_handshake` used to fire up to 16 AACS authenticate -attempts back-to-back with no pause between them. Each attempt is -5-10 SCSI REPORT_KEY/SEND_KEY exchanges. On a disc whose host cert -isn't in our KEYDB (or one the drive rejects), that's 80-160 SCSI -commands hammered at the drive in a few hundred milliseconds — and -the BU40N (plus most consumer optical drives) responds by entering -a fast-fail firmware wedge state where every subsequent CDB returns -`ILLEGAL_REQUEST/INVALID_FIELD_IN_CDB` (sense 0x05/0x24) until the -drive is physically power-cycled. - -Live wedge event 2026-05-20 during a UHD disc scan -(KEYDB miss) confirmed the diagnosis and motivated this fix. - -Defence-in-depth: -- `MAX_CERT_ATTEMPTS` capped at 3 (was 16). If three different - host certs all fail, more won't help — the drive doesn't have a - match. -- 1-second sleep between attempts. Gives the drive's firmware time - to recover internal state between auth challenges. -- Bail immediately on any sense code with sense_key = 0x05 - (`ILLEGAL_REQUEST`). The drive isn't merely rejecting our cert - — it's saying "I won't talk to you anymore" — so trying more - certs would deepen the wedge. - -## 0.25.6 (2026-05-20) - -### Changed - -- Synchronized to autorip 0.25.6 release. No behavioural changes. - -## 0.25.5 (2026-05-20) - -### Changed - -- Synchronized to autorip 0.25.5 release. No behavioural changes. - -## 0.25.4 (2026-05-20) - -### Changed - -- Synchronized to autorip 0.25.4 release. No behavioural changes - in the library — the v0.25.4 release ships the autorip-side - in-container NFS mount option. - -## 0.25.3 (2026-05-19) - -### Changed - -- Synchronized to autorip 0.25.3 release. No behavioural changes - in the library — the v0.25.3 release ships the autorip-side - parallel mux pipeline. - -## 0.25.2 (2026-05-19) - -### Fixed - -- **DTS-HD codec ID** — `MkvTrack::audio` now emits `A_DTS/MA` for - DTS-HD MA and `A_DTS/HR` for DTS-HD HR instead of mislabelling - both as plain `A_DTS`. Strict players (Plex transcoder, some - hardware decoders, AV receivers) reject lossless DTS-HD MA - payload when the track advertises `A_DTS` because the ID - implies the 1.5 Mbps core-only bitstream. -- **PGS subtitle BlockDuration** — the PGS parser is now stateful: - it pairs each display PCS with the following empty/clear PCS to - compute a duration, and the MKV muxer emits a `BlockGroup` + - `BlockDuration` for subtitles that carry one. Without this the - last bitmap lingered on screen until the next display set - replaced it (or until end of file). - -### Changed - -- `codec::Frame` gains `duration_ns: Option` (set by parsers - that can compute one; currently only PGS). -- `pes::PesFrame` gains `duration_ns: Option` (in-memory only; - not part of the on-wire serialization). -- `MkvMuxer::write_frame` now takes a final `duration_ns: Option` - parameter. When `Some`, the frame is emitted as a `BlockGroup` - with `BlockDuration` instead of a `SimpleBlock`. - -## 0.25.1 (2026-05-19) - -### New — autorip event_fn plumbing - -- `PrefetchedSectorSource::new_with_events(reader, extents, batch, - halt, event_fn)` — same producer-thread pipeline as `new()`, plus - an optional callback fired from the producer with `BytesRead` - events after every successful batch. Lets the autorip multipass - + resume mux paths drive their progress UI from the highway - without polling the consumer side. -- `build_iso_pipeline` gains an `event_fn` parameter so callers can - wire the same callback through one ctor. - -### Changed - -- `pes::Stream` trait: new default `errors() -> u64` method - (default returns 0). Lets `Box` callers query the - skip-on-error counter without downcasting. `DiscStream` overrides - to surface its existing `errors` field. - -### Removed (breaking) - -- `DiscStream::new_pipeline` and `DiscStream::read_pipeline` - deleted. All file-backed mux now uses `build_iso_pipeline` → - `PipelinedPesStream`. `DiscStream` is now the single-threaded - inline reader used by autorip's live-drive single-pass path only. -- `DiscStream::demux_thread` and `demux_rx` fields removed. -- `M2tsStream::open` deleted; `Mode::Read` variant removed. - `m2ts://` URLs go through the internal `build_m2ts_pipeline` - helper in `mux/resolve.rs` → `PipelinedPesStream`. `M2tsStream` - is a write-only sink now. - -## 0.25.0 (2026-05-19) - -### New — the freemkv mux throughput "highway" - -Three-stage pipelined PES read path. Read+decrypt runs on a -producer thread, M2TS demux runs on a second thread, codec parse -runs on the caller's thread. Communication between stages is via -bounded `crossbeam_channel` with a recycled buffer pool — no -allocations or memcpys in the steady-state hot loop. - -**Throughput on the test bed (a UHD disc, 62 GiB ISO → -`null://`, single-thread caller):** - -| | MB/s | -| ------------------------------------- | -----: | -| 0.23.2 baseline | 60 | -| + memchr SIMD HEVC start-code scan | 69 | -| + ts.feed no-copy boundary | 72 | -| + `PrefetchedSectorSource` (producer) | 124 | -| + `DemuxThread` (3-stage pipeline) | 135 | -| + zero-copy recycled buffer pool | 148 | -| + 16 KiB initial PesAssembler buffer | 162 | -| + mimalloc allocator in freemkv CLI | 200+ | -| **+ warm cache** | **660** | - -The new public API: - -- `libfreemkv::PrefetchedSectorSource` — wraps any `SectorSource`, - spawns a producer thread, exposes recycled-buffer channels. -- `libfreemkv::io::byte_prefetcher::BytePrefetcher` — - `std::io::Read` analogue for byte-stream sources (m2ts files, - sockets, stdin). -- `libfreemkv::mux::demux_thread::DemuxThread` — the M2TS demux - worker; `spawn_zero_copy` takes either prefetcher's channels. -- `libfreemkv::PipelinedPesStream` — the read-side `Stream` impl - that runs codec parse on the caller thread. -- `libfreemkv::build_iso_pipeline(reader, title, keys, batch, - format, halt)` — the canonical ctor that wires all three stages - for an ISO file source. - -### Changed (breaking) - -- `IsoSectorReader` (the naive duplicate of `FileSectorSource`) - deleted. `FileSectorSource` is the sole file-backed sector - source; it carries the SEQUENTIAL fadvise hint, the periodic - DONTNEED page-cache eviction, and (new) the per-read - `readahead()` async-prefetch syscall. -- `mux::input("iso://...", &opts)` now returns a - `PipelinedPesStream`. Function signature - `input(&str, &InputOptions) -> io::Result>` is - unchanged; callers that treated the return value as - `Box` keep working. - -### Other - -- `FileSectorSource` exposes per-OS `prefetch()` hooks - (`readahead(2)` on Linux, `fcntl(F_RDADVISE)` on macOS, no-op on - Windows + other). -- AACS decrypt thread-pool: env var renamed from - `FREEMKV_DECRYPT_THREADS` to `FREEMKV_THREADS`; default raised - to `cores.clamp(1, 64)`. -- `PesAssembler` initial buffer capacity 256 KiB → 16 KiB (avoids - the 64-page first-touch fault tax on every PES boundary). -- HEVC / H.264 `find_start_code` swapped onto - `memchr::memmem::find` (SIMD). - -## 0.18.4 (2026-05-09) - -### Build / CI hardening — no library code changes - -- All `cargo` invocations in `.github/workflows/*.yml` now use - `--locked`. Previously `cargo build` / `cargo test` could silently - re-resolve `Cargo.lock` if a lock-pinned version wasn't yet on - crates.io — masking dependency races between same-tag releases of - sibling crates. The 0.18.3 release hit this: autorip's docker image - built `libfreemkv v0.18.2` because libfreemkv 0.18.3 hadn't - published yet at the time autorip's CI ran, and cargo silently fell - back to the previous version. With `--locked`, that race now hard- - fails the build with a clear "package X is not in registry" error - and we retrigger after the upstream lands. - -- `cargo publish --locked` in the release workflow guards against - publishing a lockfile-mismatched release. - -## 0.18.3 (2026-05-09) - -### Behaviour change - -- **`Disc::titles[0]` is now the canonical main feature on branching - discs, not the longest playlist.** Previously titles were sorted - purely by `duration_secs` descending, which puts a "play-all" - virtual playlist (alternate angles / seamless branching) at index 0 - on UHDs that ship one — its inflated duration overshoots the real - movie. Concrete example: *The Amateur (2025)* 4K UHD on a 58.5 GB - disc has Title 1 = `00020.mpls` 4h13m / 92.4 GB / 253 clips - (impossible — the size exceeds the disc capacity, proving it's a - virtual composite) and Title 2 = `00800.mpls` 2h02m / 57.2 GB / - 1 clip (the actual film, matching TMDB). - - New sort priority: - 1. Real titles (`size_bytes <= capacity_bytes`) before virtual - composites. - 2. Among real titles, fewer clips first (1-clip wins). - 3. Tiebreak on longer duration first. - - **Migration:** consumers calling `disc.titles.first()` / - `disc.titles[0]` automatically get the corrected title — no code - changes needed. CLI users invoking `freemkv -t 1 disc:// …` now hit - the actual main feature on branching discs (this was the user- - visible bug). On non-branching discs the order is unchanged. - - The comparator is exposed as - [`Disc::canonical_title_order`](#) for callers that need to - re-sort a custom title set with the same logic. - - Regression tests: `disc::tests::canonical_order_*` (three cases: - branching-UHD, normal disc, clip-count tiebreak). - -## 0.18.2 (2026-05-09) - -### Bug fixes - -- **Nav-file scramble during AACS rip** (`decrypt::decrypt_sectors`). 0.18's - `DecryptingSectorSource` decorator broadened the call surface of - `decrypt_sectors` to every sector flowing through sweep, including UDF - navigation files (MPLS playlists, CLPI clip-info). The byte-0 heuristic - in `aacs::is_unit_encrypted` correctly fires on m2ts source-packet copy - markers but false-positives on any binary file whose first byte happens - to have the top 2 bits set — most notably MPLS files (start with 'M' = - 0x4D) and CLPI files (start with 'H' = 0x48). `decrypt_unit_full` - already self-checks the result via TS-sync verification and returns - `false` on a misfire, but the chunk had been mutated by then; - `decrypt_sectors` discarded the return value, leaving scrambled bytes - in the ISO. Fix: snapshot the chunk before decryption and restore on - verification failure (same pattern `decrypt_unit_try_keys` already used - for multi-key discs). Symptom: `freemkv info iso://UHD.iso` and - `iso:// → mkv://` returned E6009 NoStreams on freshly-ripped UHD ISOs; - affected only the iso-source path, not the disc-source path or the - m2ts video payload itself. Regression test: - `decrypt::tests::nav_file_unit_survives_decrypt_attempt`. - -- **Sweep progress display can regress to zero** (`Disc::sweep`). The - consumer-side `bytes_good` snapshot lags producer-side `bytes_done` - whenever the consumer is behind on draining the work channel. Until - the first snapshot arrived, the placeholder branch reported - `bytes_done` correctly; once a stale snapshot landed, the report - switched to `snap.bytes_good` and could regress below `bytes_done`. - Fix: `bytes_good_total = max(snap.bytes_good, bytes_done)`, so the - user-visible counter never moves backward. - -## 0.18.1 (2026-05-09) - -### I/O stack redesign — primitives over orchestration - -0.18 reshapes the read/write surface from "library does the multipass dance" -to "library hands the caller flat verbs and a few composable primitives." All -in-tree consumers (autorip, the `freemkv` CLI) drove their own multipass -loops in 0.18 rounds 1-3; 0.18.1 lands the cleanup with the deprecated names -still alive for one minor-version window. - -The 0.18 design notes are kept privately — this entry sticks to -what changed at the public surface. - -#### Flat verbs - -`Disc::sweep` is the forward Pass 1. `Disc::patch` is one retry pass over the -mapfile. Neither knows about pass index, retry budget, or accept-loss policy -— the caller invokes them in whatever sequence its use case dictates. The -old multipass-aware `Disc::copy` dispatcher is deprecated and slated for -deletion in 0.18.2; no in-tree caller still uses it. - -#### Trait splits — direction-typed at compile time - -- `pes::Stream` (combined read+write) is split into `FrameSource` and - `FrameSink`. Calling `read()` on a write-only sink is now a compile error, - not the runtime `E9001` (`StreamWriteOnly`) it used to be. -- `SectorReader` is split into `SectorSource` (read) and `SectorSink` - (write). `Drive` impls `SectorSource` only; `FileSectorSource` / - `FileSectorSink` replace `FileSectorReader` for ISO-backed I/O. -- A blanket impl bridges legacy `SectorReader` callers onto the new - `SectorSource` so existing code keeps compiling through the deprecation - window. - -#### New primitives - -- `Halt` — one cancellation token (cloneable, `Arc` under the - hood) replaces the three near-duplicate halt flags scattered through the - workspace. Threaded through every long-running loop. -- `Pipeline` + `Sink` — generic producer/consumer primitive in - `crate::io`. Replaces the bespoke `disc/sweep_pipeline.rs` and now also - drives `Disc::patch` and the autorip mux loop. `DEFAULT_PIPELINE_DEPTH` - for streaming reads (`4`); `WRITE_THROUGH_DEPTH` for write-through patch - semantics (`1`). -- `WritebackFile` — was `crate::io::Writer`. The renamed type makes its job - explicit: a `File` wrapper that runs continuous `sync_file_range` + - `posix_fadvise(DONTNEED)` to keep the kernel dirty-page cache bounded on - long sequential writes. -- `DecryptingSectorSource` — a single decorator wrapping any - `SectorSource` to yield plaintext sectors. One audit surface for AACS / - CSS / passthrough; the previous two-site decrypt (sweep producer + - `DiscStream` demux) is gone. - -#### Throughput - -The round-2 producer/consumer split is now applied uniformly to sweep, -patch, and mux. Mux on NFS-staged UHD measured ~16 MB/s sustained on the -test bed (was ~12 MB/s pre-round-2). - -#### Module reorg - -- `sector/` and `io/` are now module directories. -- `disc/sweep.rs`, `disc/patch.rs`, and `disc/mapfile.rs` split out of the - monolithic `disc/mod.rs`. - -#### Renames (no behavior change) - -- `crate::io::Writer` → `crate::io::WritebackFile`. -- `Apply` → `Flow` (`Sink::apply` return value). -- `DEFAULT_DEPTH` → `DEFAULT_PIPELINE_DEPTH`. -- `PatchOpts` → `PatchOptions`. - -#### Deprecated (alive in 0.18.1, deletion target 0.18.2) - -`Disc::copy`, `pes::Stream`, `SectorReader`, `FileSectorReader`, -`CopyOptions`, `CopyResult`, `DiscStream::set_halt`. Each compiles with a -deprecation warning; all in-tree call sites have migrated. - -## 0.17.13 (2026-05-09) - -### Use `crate::io::Writer` uniformly for all binary file output - -0.17.10 introduced the bounded-cache writeback wrapper (`sync_file_range` -+ `posix_fadvise(DONTNEED)` per chunk) and 0.17.11 wired it into -`Disc::sweep`. The other two big-write paths in the crate were still -opening raw `std::fs::File` and would have hit the same dirty-page -burst pathology against slow / network-attached staging. - -This release threads `crate::io::Writer` through the remaining sites: - -- **`Disc::patch`** (`disc/mod.rs:1981`): the ISO file reopened for - Pass-N recovery now wraps in `Writer` before any seek / write. - Recovery writes are sparse, but the wrapper costs nothing when - there's no chunk crossing — and on heavily-damaged discs it - matters as patch accumulates GB of recovered data. -- **MKV mux output** (`mux/resolve.rs:243`): `BufWriter::with_capacity` - now wraps `Writer::new(file)` instead of a raw `File`. UHD MKVs - routinely exceed 70 GB of sequential writes; pre-0.17.13 those - bursts went straight to the kernel writeback queue. -- **M2TS mux output** (`mux/resolve.rs:251`): same change for - parity with the MKV path — anyone using `m2ts://` URLs gets it - too. - -No new public surface. `Writer::sync_all()` is called from `Disc::patch` -and `sweep_pipeline`'s consumer at end of pass; mux exits via Drop on -the wrapping `BufWriter`, which propagates flush down to `Writer`'s -final `note_progress` (kernel finishes the in-flight chunk on file -close — there's no explicit `sync_all` for mux today, same behaviour -as before). - -## 0.17.12 (2026-05-09) - -### Mapfile time-batched persistence — unblock NFS staging - -Pre-0.17.12 every `Mapfile::record()` wrote the entire mapfile to disk -via tempfile-create + buffered-write + atomic-rename. On local LVM -that's effectively free (page cache + microsecond-scale renames). On -NFS (autorip's intended staging path for centralised media) each -record() became three RPCs through the unraid user share's shfs-fuse -layer — measured end-to-end at multiple ms each. With ~170 record() -calls per second of sustained sweep, the mapfile path alone burned -multiple seconds of wall time per real-world second of work, dragging -the rip from ~11 MB/s on local to ~1.5 MB/s on NFS. - -Fix: time-batch the persistence inside `Mapfile`. - -- `record()` always updates in-memory state and stats (so callers' - `stats()` reads stay coherent in the same process). -- The `write_to_disk()` rename only fires when ≥ `FLUSH_INTERVAL` - (1 s) has elapsed since the last persist. -- New `flush()` method forces an out-of-band persist; called by - `sweep_pipeline`'s consumer at end-of-sweep and by `Disc::patch` at - end-of-patch, after the file's `sync_all()`. -- `Drop` impl best-effort flushes so an early-return / unwind doesn't - silently lose pending state. - -Crash-safety changes from "lose at most one block" to "lose at most -1 s of recorded progress" — the ISO file's payload bytes are -unaffected; only the mapfile's authority over which sectors are -already-good is at risk, and a resume re-reads anything Pass 1 had -already covered. Acceptable for a 7× throughput recovery on the -target deployment. - -The internal `round_trip_load` test now calls `flush()` before -`Mapfile::load` to read back what the in-memory state asserts. -External patch / copy tests are unaffected: `patch` and -`sweep_pipeline` flush at completion before returning, so any -`Mapfile::load` at the call-site sees fully persisted state. - -## 0.17.11 (2026-05-09) - -### Sweep producer/consumer split — overlap drive read with file write - -Pre-0.17.11 the sweep loop ran strictly serialised: SCSI read → decrypt -→ seek + write → mapfile.record → next read. The drive idled for the -post-read work; throughput capped at the **sum** of both costs. On a -healthy disc that's ~7-12 ms read + ~5-15 ms write/record per 64 KB -batch — limiting sustained throughput to ~10-12 MB/s on the test bed -(BU40N + UHD inner zone), well below the drive's ceiling of ~14-16 MB/s. - -This release decouples them with a producer / consumer split: - -- **Producer** (caller's thread): owns the `SectorReader`, the entire - `read_error` state machine (Retry, Bisect, SkipBlock, JumpAhead, - AbortPass), `set_speed` damage-zone transitions, halt check, and - decrypt. Hands plaintext bytes to the consumer. -- **Consumer** (one spawned thread): owns the `crate::io::Writer` and - the `Mapfile`. Receives `WorkItem` messages and applies the file - write + mapfile record per item. -- **Channel**: bounded `mpsc::sync_channel(4)` — natural back-pressure - via blocking `send` when consumer falls behind. - -While the consumer is writing batch N to disk and updating the mapfile, -the producer is already reading batch N+1 from the drive. Steady-state -throughput is now bound by the slower of the two pipelines (the drive, -on a healthy disc) instead of their sum. - -Side effects of the refactor: - -- **Bisect path now decrypts.** Pre-0.17.11 the bisect inner loop wrote - raw cyphertext when `decrypt=true` and a single sector was recovered - via single-sector retry — a quiet correctness bug exercised only by - the (rare) batch-fail-then-bisect-succeed path on encrypted discs. - The new producer-side decrypt covers both the main success path and - the bisect inner success path. -- All `read_error::ReadCtx` state stays single-threaded on the producer - (damage window, jump multiplier, consecutive-good count, etc.). No - locking added. -- Mapfile remains single-writer (consumer-only). No locking. -- Halt-flag responsiveness unchanged: producer breaks the loop on - signal, sends `Finish`, consumer drains its ≤4 in-flight items and - exits within ~1 batch (~12 ms typical). -- BU40N + Initio bridge wedge concern unchanged: still one SCSI command - in flight, error-path timing identical, no new retry logic. - -New module: `src/disc/sweep_pipeline.rs` (`WorkItem`, `ProgressSnapshot`, -`ConsumerInputs`, `spawn_consumer`, `consumer_loop`, send/recv helpers). -Public API surface unchanged — `Disc::copy` / `CopyOptions` / -`CopyResult` look identical to callers. - -Patch (Pass N) is **not** affected by this release. Patch is bound by -drive recovery time (60 s timeouts on bad sectors), not the read↔write -serialisation; a similar split there would yield negligible benefit. - -## 0.17.10 (2026-05-09) - -### Bounded-cache writeback for big sequential writes - -Pass 1 sweep speed on a healthy disc previously dipped from ~15 MB/s -to ~1 MB/s every ~30 s on a host with default Linux dirty-page -settings. Cause: the kernel's `vm.dirty_ratio` (~20 % of RAM) lets -hundreds of MB of dirty pages accumulate, then bursts a flush at -99 % disk utilisation that blocks app writes for ~1 s. Confirmed -empirically on the BU40N test bed: dirty pages grew 112 → 563 MB -between bursts; lowering `vm.dirty_bytes` to 64 MB at the host -sysctl level eliminated the dips. - -This release ships the equivalent inside libfreemkv so users don't -need to tune the host kernel: - -- New `crate::io::Writer` — drop-in `File` wrapper implementing - `Write` + `Seek`. Wraps a per-platform `WritebackPipeline` that on - Linux schedules `sync_file_range(WRITE)` + lagging - `sync_file_range(WAIT_AFTER)` + `posix_fadvise(DONTNEED)` calls in - 32 MB chunks, keeping dirty cache bounded at ~64 MB. macOS and - Windows ship a no-op stub — their default cache policies don't - exhibit the same pathology for our access pattern. -- Disc::sweep wraps its output `File` in `Writer`. No changes to the - loop body — `Writer` forwards `seek`/`write_all` to `File` and - drives the pipeline transparently. -- Module is purpose-built for any large sequential output (sweep, - patch, future mux) — they can adopt `crate::io::Writer` with a - one-line wrapper and inherit the same behaviour. - -## 0.17.7 (2026-05-08) - -### Sync release — no functional libfreemkv changes - -Version bump to keep the four freemkv crates at unified versioning. -autorip shipped UI smoothness + audit fixes today (v0.17.6, v0.17.7); -libfreemkv carries no code changes, but is bumped + republished to -crates.io so downstream `libfreemkv = "0.17"` consumers stay aligned -on the latest patch version. See autorip CHANGELOG for the -user-visible work in this release window. - -## 0.17.5 (2026-05-08) - -### Pass N recovery — kernel block-device fallback + per-range fixes - -Live testing on direct-SATA BU40N + AACS-encrypted UHD disc revealed that -the v0.17.3 single-shot SCSI READ path matched 0/22 of the small bad-sector -LBAs that `dd if=/dev/sr0` recovers on the same drive. This release closes -that gap and fixes several adjacent bugs that were silently capping -recovery. - -- **`/dev/sr0` pread fallback in `Drive::read` (Linux)**: when a SCSI READ - via `/dev/sg*` returns Err, fall back to `posix_fadvise(POSIX_FADV_DONTNEED) - + pread()` against the corresponding block device. The kernel `sr_mod` - driver runs ~5 internal retries per command without the per-attempt - error-escalation overhead that userspace SG_IO retries pay, which is the - source of dd's recovery advantage. End-to-end byte-verification confirms - the fallback path returns real disc data (md5-equivalent to fresh dd from - the same drive session). The block fd is opened in `Drive::open` by - resolving `/sys/class/scsi_generic/sgN/device/block`; best-effort, with no - fallback if open fails. - -- **`Disc::patch` per-range watchdog fix**: when a range hit `MAX_RANGE_SECS`, - the old code did `wedged_exit = true; break 'outer;` — a single slow range - killed the entire patch. Now `break;` (skip this range, advance the outer - for loop). Pre-fix, patch was dying after 4 sectors of the first slow - range and never reaching the other 46. - -- **Per-sector range budget**: replaced flat `MAX_RANGE_SECS = 180` with - `range_budget_secs = (range_sectors × SECONDS_PER_SECTOR).min(RANGE_BUDGET_CAP_SECS)`. - Tiny ranges exit fast (1-sector range = 25 s); medium ranges get - proportional time (51-sector range ≈ 1275 s); large ranges still bounded - by the 1800 s cap so they cannot monopolise pass 1. - -- **`consecutive_failures` resets per range**: the wedge-exit detector - (`>= 50 consecutive failures`) is for "stuck on the same range"; pre-fix - the counter persisted across ranges, so 50 small post-bisection ranges - with one failure each falsely tripped the wedge mid-pass. Reset at every - range boundary. - -- **Reverted inline 5× retry in patch**: a brief experiment that was - measurably harmful — each "2 s" SCSI timeout paid ~1.5 s kernel SCSI - mid-layer error-escalation overhead, so 5× retry took ~17 s per LBA and - triggered the per-range watchdog in 4 sectors. Restored - `READ_RECOVERY_TIMEOUT_MS = 60_000` (the v0.17.3 baseline; the kernel- - auto-retry pattern is now provided by the `/dev/sr0` fallback above). - -### Empirical results (a UHD disc, BU40N direct SATA) - -- Pass 1: **94.6 MB recovered** (28% of formerly-bad data, 33 sr0 fallback - saves), **11 s of main-title content** restored. Patch completed all 47 - retryable ranges naturally (was wedging at range 1 of 47 in v0.17.3). -- Pass 2 cumulative: 95.2 MB (+0.6 MB; diminishing returns curve). -- Remaining ~233 MB on the test disc appears physically unrecoverable on - this hardware (kernel auto-retry can't decode it either). - -### Behavioural notes - -- The `/dev/sr0` fallback is Linux only; macOS and Windows fall back to - the existing single-shot SCSI behaviour. The fallback is gated to - `recovery=true` reads (only fires from the patch path, not the sweep - path) to avoid page-cache pressure during multi-GB sequential ripping. - -## 0.17.0 (2026-05-04) - -### Code quality: unwrap safety, clippy compliance, test coverage - -- **Sweep() hot path**: Replaced `.err().unwrap()` in `disc/mod.rs:1553` with explicit pattern matching (`match read_result { Ok(_) => unreachable!(), Err(e) => e }`). The original unwrap was logically safe (in the else branch after `is_ok()` check) but pattern matching makes the invariant explicit and avoids panic risk if logic changes. -- **cargo clippy --lib**: Clean build with `-D warnings` across all targets. No lint failures. -- **cargo test --lib**: All 256 tests pass (0 failed, 1 ignored). Integration tests for sweep/patch/dev-null validate the multi-pass recovery pipeline end-to-end. - -### Patch pass algorithm fix - -- `patch_internal` excludes Unreadable sectors from work list (only retries NonTrimmed/NonScraped/Unscraped). Previously attempted to retry already-failed sectors, wasting time and masking real progress. -- Exposes `bytes_bad_in_title` in patch results for accurate UI reporting of unrecoverable data vs. recovered data. - -## 0.16.1 (2026-04-30) - -### Unified progress display for sweep and patch - -- Patch passes now show `bytes_good / disc_total` as GB (grows when sectors are recovered), with pass progress `%` using `work_done / work_total` (always advances). `Xs unreadable` declines as data is recovered. Same UI for all pass types. - -## 0.16.0 (2026-04-30) - -### IOKit registry-based drive enumeration, BSD name matching, reverse patch default - -- **`shim_open_exclusive`** now matches the correct `IOBDServices` for the requested BSD name. Walks all IOBDServices entries in the IOKit registry, matches child IOMedia `"BSD Name"` property. Falls back to IOMedia parent walk, then first-match. Fixes multi-drive systems. -- **`shim_list_drives`** (new): registry-based drive enumeration. Reads IOBDServices `"Device Characteristics"` for vendor/model/firmware and child IOMedia for BSD name. Zero SCSI, zero exclusive access, zero unmounts. Fixes the enumeration blast that unmounted every disk on the system. -- **`list_drives()`** and **`find_drives()`** rewritten to use `shim_list_drives`. No longer iterates `/dev/disk0..15` opening exclusive SCSI on each. -- **`patch_internal`** defaults `reverse: true`. Sweep jumps forward with escalating gaps, so NonTrimmed ranges have good data at their tail. Reverse patch hits good data first, converges on actual bad block boundaries. - -## 0.15.1 (2026-04-30) - -### Fix damage-jump detection, fix dispatch covers_disc check - -- Damage-jump tuning: `DAMAGE_WINDOW=16`, `DAMAGE_THRESHOLD_PCT=12%` (was 50/25%). Old params were too diluted by good reads between sparse failures; new params trigger on 2nd scattered failure. -- `Disc::copy()` dispatch: `covers_disc` now checks `map.total_size() == disc_size` (byte-for-byte, not approximate). Fixes false sweep dispatch when mapfile exists but doesn't cover full disc. -- Sweep resume: when dispatched from existing mapfile with NonTried, passes `resume: true`. - -## 0.15.0 (2026-04-30) - -### Multipass dispatch rewrite, speed control on damage zone entry/exit - -- `CopyOptions { decrypt, multipass, progress, halt }`. `Disc::copy()` auto-detects sweep vs patch from mapfile state: no mapfile → sweep, NonTried → sweep with resume, only NonTrimmed/NonScraped/Unreadable → patch, clean → no-op. -- `Disc::mapfile_for()`: `/dev/null` output → `/tmp/.mapfile`, otherwise `mapfile_path_for(path)`. -- Speed control: damage zone entry → `set_speed(0x0000)` (minimum), 16 consecutive good reads → `set_speed(0xFFFF)` (maximum). Drive manages optimal speed in clean sections. -- `SET CD SPEED` SCSI command via `SectorReader::set_speed()` (default no-op, Drive impl sends SCSI). - -## 0.14.0 (2026-04-30) - -### Damage-jump algorithm replaces probe, bridge degradation detection - -- Damage-jump algorithm: when damage threshold exceeded in sliding window, jump ahead by `256×batch×multiplier` sectors. Doubles multiplier on each jump. Zero-fills gap as NonTrimmed. -- `ecc_sectors()`: returns ECC block size per disc format (32 for UHD, 16 for BD, 16 for DVD). -- Bridge degradation detection: NOT READY with sense key 2/ASC 0x04/ASCQ 0x3E triggers 10s cooldown, up to 5 times before treating as bad sector. - -## 0.13.43 (2026-04-29) - -### Pass 1 transport-failure recovery loop - -- Transport failure (USB bridge crash) no longer kills the entire rip. -- Autorip re-discovers the drive after USB re-enumeration and resumes - from the mapfile. Up to 10 attempts. -- `Error::is_scsi_transport_failure()` now matches DiscRead with - status 0xFF (bridge crash) in addition to ScsiError. -- DriveSession tracks device_path for re-discovery. - -## 0.13.42 (2026-04-29) - -### Fix: transport failure skips instead of aborting - -- Transport failure (USB bridge crash) now skips the failed ECC block - (marks NonTrimmed) and continues. 3 consecutive transport failures - still abort the copy. Previously, a single transport failure killed - the entire rip. - -## 0.13.41 (2026-04-29) - -### Debug logging for sector-0 regression diagnosis - -- Add debug logging to Drive::read and Disc::copy first reads. -- No functional changes. - -## 0.13.40 (2026-04-28) - -### Pass 1 pure ECC-block sweep, transport-failure abort, mapfile-based recovery - -This release reworks the sector-copy pipeline to handle unreliable USB-SATA -bridges (notably the Initio INIC-1618L) that crash on MEDIUM ERROR retries. - -**Disc::copy() — Pass 1 (ECC-block sweep):** -- Reads `batch` sectors (default 32 = 1 BD ECC block = 64 KB). -- Success → mark Finished. MEDIUM ERROR → zero-fill, mark NonTrimmed, advance. -- Transport failure (host_status=7) → abort immediately, return error. -- No single-sector reads, no retry, no state machine in Pass 1. - -**Disc::patch() — Pass 2+ (single-sector recovery):** -- Reads NonTrimmed sectors one at a time with pause between failures. -- Succeeds → mark Finished. Fails → mark Unreadable (or leave for next pass). -- Multi-pass: caller runs patch repeatedly until 0 recovered. - -**CopyOptions simplified:** -- Removed `skip_forward`, `cautious_pause_ms`, `BPT1_EXIT_THRESHOLD`. -- Fields: `decrypt`, `resume`, `batch_sectors`, `skip_on_error`, `progress`, `halt`. - -**Other fixes since 0.13.26:** -- `open()` just opens the device — no side effects. `drive_has_disc()` is a - standalone TUR, not a probe sequence. -- `enumerate_sg_names()` skips unreadable `/sys` type files. -- SCSI sense data preserved in DiscRead errors (status + key + ASC/ASCQ). -- MapStats splits `bytes_pending` into `nontried` / `retryable`. -- Wallclock rip budget — halt after max(disc_runtime_secs, 3600). -- Patch instrumentation: counters for reads_ok/err, writes_ok/err, finished/unreadable. -- `as_encoded_bytes()` replaces `as_bytes()` for portable OsStr handling. -- Removed inline retry/reset from Drive::read — orchestration layer handles recovery. -- Removed all internal references from public code. - -## 0.13.26 (2026-04-27) - -### Extend DiscRead with SCSI status/sense for 30% wedge diagnostics - -`Error::DiscRead` now carries `status` (SCSI status byte) and `sense` -(ScsiSense with key/asc/ascq). Previously this info was discarded, -showing only `E6000: {sector}`. Now shows: -- `E6000: {sector} 0x{status}/0x{sense_key}/0x{asc}` -- Enables recovery loop to distinguish recoverable errors from drive wedge -- Enables programmatic handling: `if status == 0xFF { reset } else { retry }` - -### Error display shows up to 5 fields - -Display format changed from `E{sector}` to `E{code}: sector status/key/asc`. - -## 0.13.25 (2026-04-27) - -### Drop dead `Drive::device_path_owned()` - -The method was marked `// NOTE: Debug aid — remove after fd issue is -resolved` and the fd issue closed in 0.13.6. Use `device_path()` (which -returns `&str`) instead. Removing it clears a `cargo clippy -- -D -warnings` red on Linux CI that the Mac toolchain doesn't catch. - -### Pre-commit gate uses CI's exact toolchain - -A new pre-commit script runs `cargo +1.86 fmt ---check`, `cargo +1.86 clippy -- -D warnings`, and `cargo +1.86 test ---tests` across all 5 freemkv crates. Mirrors each repo's -`.github/workflows/ci.yml` step-for-step. Use it as a pre-commit hook -or run by hand before pushing — green here means green CI. - -The Mac default toolchain is newer (1.94) and its clippy rejects -slightly different sets of lints than 1.86 — running locally without -pinning misses lints CI catches. The script forces 1.86 so drift -between local and CI ends. - -## 0.13.24 (2026-04-27) - -### MapStats: split `bytes_pending` into `bytes_nontried` + `bytes_retryable` - -`MapStats.bytes_pending` aggregates `NonTried` (sectors Pass 1 hasn't -reached) + `NonTrimmed` + `NonScraped` (sectors flagged for Pass 2-N -retry). UIs that wanted a "MAYBE / will retry" bucket were stuck -showing the entire unread disc as "Maybe" at pct=0. - -v0.13.24 keeps `bytes_pending` for back-compat and adds two granular -fields: - - - `bytes_nontried` — Pass 1 hasn't read these yet - - `bytes_retryable` — `NonTrimmed + NonScraped`, Pass 2-N will retry - -`bytes_pending == bytes_nontried + bytes_retryable` (invariant). - -### cargo fmt cleanup - -Picks up the `cargo fmt --check` lint failure that's been red on -`main` since v0.13.18 (long format-string layouts the local rustfmt -folded differently from CI's runner). - -## 0.13.23 (2026-04-27) - -### Stop discarding the drive's SCSI sense data - -Through the entire 0.13.x line, every CHECK CONDITION reply from the -drive (the standard way SCSI reports a sector failure) was being -collapsed into a synthetic `status=0xFF, sense_key=0` "transport -wedge" sentinel and the real sense data was thrown away. Live tracing -on the BU40N reading a UHD disc on 2026-04-27 confirmed it: the drive was -returning `host_status=0, driver_status=8, status=2, exec_elapsed_ms=1416` -on every bad sector — a clean CHECK CONDITION carrying full sense -data — and the library was misclassifying it as a wedge and bailing. - -Root cause: `scsi/linux.rs`'s wedge check was `host_status != 0 || -driver_status != 0`. SG's `DRIVER_SENSE` bit (0x08) is set on every -CHECK CONDITION reply just to flag "sense buffer is populated" — it's -not a transport failure on its own. Pre-0.13.23 we conflated the two -and silently lost every drive-reported error reason. macOS and Windows -backends had the same shape: they extracted `sense_key` only, dropping -ASC/ASCQ. - -### What 0.13.23 changes (API) - -- **Linux**: mask `DRIVER_SENSE` before treating `driver_status` as a - transport-layer failure. Real transport failures (`host_status != 0` - or any non-SENSE bit set) still synthesise the `0xFF` sentinel. -- **`Error::ScsiError`** carries `sense: Option` instead of - flat `sense_key`/`asc`/`ascq`. `sense=None` ⇔ transport failure (no - SCSI status delivered). `Some(ScsiSense {…})` ⇔ drive replied with - sense data. Removes the `0xFF`/`sense_key=0` magic-number coupling. -- **`ScsiSense`** is a public type with predicate methods on it — - `is_marginal`, `is_medium_error`, `is_hardware_error`, - `is_unit_attention`, `is_data_protect`, `is_not_ready`, - `is_illegal_request`, `is_aborted_command`. Callers route on the - structured fields rather than raw key comparisons. -- **`Error::scsi_sense()`** / **`Error::is_scsi_transport_failure()`** / - **`Error::is_marginal_read()`** convenience predicates on `Error`. - `is_marginal_read` is the high-level "should `Disc::copy` engage - hysteresis on this error?" check. -- **SCSI protocol constants** (`SCSI_STATUS_GOOD`, - `SCSI_STATUS_CHECK_CONDITION`, `SCSI_STATUS_TRANSPORT_FAILURE`, - `SENSE_KEY_*`) moved from `error.rs` to `scsi/mod.rs` where they - belong alongside `SCSI_INQUIRY`, `SCSI_READ_10`, etc. -- **macOS** + **Windows** backends parse the full sense triple too. - Same code path on every platform — a regression in `parse_sense` - would surface on all three OSes simultaneously. -- **`parse_sense`** replaces `parse_sense_key` (returns the full - triple). Inline sense-format tests (descriptor 0x72/0x73 vs fixed - 0x70/0x71, short-buffer, VALID-bit masking, unknown response codes) - now also exercise ASC/ASCQ extraction at the right offsets. - -### Disc::copy + Disc::patch sense-aware dispatch - -Both passes now bail immediately when a read fails with a sense class -that retry can't help (HARDWARE ERROR, DATA PROTECT, UNIT ATTENTION, -NOT READY, ILLEGAL REQUEST, real transport failure, kernel `IoError`) -rather than burning hysteresis cycles on a doomed loop. Marginal-read -sense (MEDIUM ERROR, ABORTED COMMAND, RECOVERED ERROR, NO SENSE) -engages hysteresis as before. New `phase=bail` trace event records -the bail reason. - -`Disc::patch`'s `wedged_threshold` (50 consecutive failures) remains -as defense-in-depth for chains of marginal failures, but a single -non-marginal sense now short-circuits it. - -### Behavioural impact - -For damaged-disc rips on the BU40N this unblocks v0.13.22's -hysteresis: pre-fix, the misclassified "wedge" caused `Disc::copy` to -exit before hysteresis could engage, so `bytes_good` froze at the bad -zone. Post-fix the drive's CHECK CONDITION replies flow through the -normal path → hysteresis drops to bpt=1 → marginal sectors are -recovered or marked Unreadable. Calibration data -(`docs/audits/2026-04-26-bisect-on-fail-empirical-findings.md`) shows -~86 % of marginal-region sectors recover at bpt=1 on this drive. - -## 0.13.22 (2026-04-26) - -### Replace bisect-on-fail with hysteresis state machine (Block ↔ Single) - -Live test on a UHD disc v0.13.21 showed bisect-on-fail recovered every -recoverable sector, but spent ~30 sec per damaged 60-block (paying a -~5 sec kernel timeout at every bisection level). Each level descended -log₂(60) ≈ 6 times on the failing branch. - -Replaced with a two-state hysteresis machine in `Disc::copy`: - -``` -Block(batch): - read(batch) ok → write, advance, stay Block - read(batch) fail → switch to Single, retry SAME range at bpt=1 - -Single: - read(1) ok → write, consecutive_good++ - if consecutive_good >= BPT1_EXIT_THRESHOLD: - switch to Block, reset counter - read(1) fail → mark NonTrimmed, consecutive_good = 0 -``` - -`BPT1_EXIT_THRESHOLD = 10_000` sectors (= 20 MB of clean data). -Calibrated from the 2026-04-26 BU40N empirical run; tunable. - -Per-block cost on a 60-sector damaged block with 1 bad sector: -- Bisect (v0.13.21): ~30 sec (5 s × 6 levels) -- Hysteresis (v0.13.22): ~10 sec (5 s bpt=batch fail + 59 × 1 ms good - + 1 × 5 s bad) - -Plus inside a damaged cluster spanning many 60-blocks, hysteresis -pays the bpt=batch fail cost ONCE on entry; bisection paid it every -60 sectors. For the ~1248-sector boundary cluster that's ~21 -fewer 5-sec waits = ~100 sec saved. - -Telemetry: new `phase=mode_change` trace event with `from`, `to`, -`lba`, `consecutive_good`. Replaces v0.13.21's `phase=bisect`. The -v0.13.21 worklist DFS is gone — single iterative `for s in 0..count` -on the failure path. - -Test rename: -`test_disc_copy_bisect_recovers_via_single_sector_reads` → -`test_disc_copy_hysteresis_recovers_via_single_sector_reads`. Same -synthetic BU40N-pattern reader; same 100% recovery expectation. - -## 0.13.21 (2026-04-26) - -### Fix: Disc::copy bisect-on-fail (replaces skip-forward) - -Empirical live-hardware testing on the LG BU40N (see -the internal test-plan audit and run log) revealed that the drive often **fails -multi-sector READ commands** in damaged regions but **succeeds when -asked one sector at a time**. The old skip-forward strategy responded -to multi-sector failures by jumping up to 1 % of the disc forward, -marking everything in between as bad — losing **clean territory** -sandwiched between bad sectors. - -`Disc::copy` now bisects on read failure: split the failed block in -half, retry each half, recurse down to single-sector reads. Sectors -the drive can read individually are recovered in Pass 1; only sectors -that fail at bpt=1 are marked NonTrimmed for the patch passes. - -Empirical results on a UHD disc on the BU40N: -- Old algorithm: 25 GB read in Pass 1, then ~6 GB skip-forwarded; - retry passes failed to recover most of the skipped zone. -- New algorithm: ~99 % of disc recovered in Pass 1; only the truly - unreadable cluster (~14 % of a 2 MB hot zone) marked NonTrimmed. - -Implementation: stack-based DFS in the inner read loop. log₂(batch) -levels max — for the default 60-sector batch, 6 levels. Multi-pass -machinery is untouched: Pass 2 .. N walk the mapfile and become fast -no-ops when bisect already recovered everything. New integration test -`test_disc_copy_bisect_recovers_via_single_sector_reads` validates -the behavior against a synthetic BU40N-pattern reader. - -### Fix: READ_TIMEOUT_MS bumped 1.5 s → 10 s (caller-side) - -The 0.13.20 SCSI rewrite gave the kernel mid-layer the ability to run -its own ABORT/RESET escalation. But callers (`Drive::read` for the -fast path) still passed `timeout_ms=1500`. Cold-start seek on the -BU40N can take ~1.5 s, which means **normal reads were being -cancelled at the boundary**, triggering the kernel mid-layer's -escalation, which the Initio bridge couldn't drain — resulting in the -firmware-level wedge that only physical replug recovers. - -Live-hardware probe data: -- Sustained sequential read: 3–7 ms -- Cold-start seek + read: up to ~1500 ms -- Successful ECC recovery: 1.6–2.6 s -- Confirmed unreadable: 3.6–8.8 s (kernel timeout) - -10 s is calibrated to cover every legitimate read with margin while -still short-circuiting truly bad sectors before the kernel runs full -LUN/BUS/HOST reset. `READ_RECOVERY_TIMEOUT_MS` (60 s) unchanged. - -## 0.13.20 (2026-04-26) - -### Architecture: SCSI transport — sync blocking SG_IO - -`scsi/linux.rs` rewritten from async `write/poll/read + 1.5 s timeout -+ close-on-timeout in bg thread` to a single synchronous blocking -`ioctl(fd, SG_IO, &hdr)`. The old pattern abandoned slow-but-alive -commands faster than the drive could drain its internal queue, -deepening the BU40N wedge. Per the SCSI-architecture review, -no existing consumer ripper or dd-based tool does what we did — -all use sync blocking SG_IO with 8-60 s timeouts and let the kernel's -mid-layer (`scsi_eh.rst`) run ABORT TASK / LUN RESET / BUS RESET / -HOST RESET escalation internally. - -What changed: -- `SgIoTransport::execute()` is one syscall now. Caller-supplied - `timeout_ms` is honored by the kernel, which does its own - ABORT/RESET escalation if the device times out. -- Errors check `host_status` and `driver_status` (both 0xFF-synthesised - for the caller) in addition to `status` — transport-level failures - no longer slip through as Ok. -- Sense-key parser handles both descriptor format (0x72/0x73, key at - byte 1) and fixed format (0x70/0x71, key at byte 2). -- Deleted the `fd_recovery: Arc` field, the bg close+open - thread, and the stale-fd swap dance. `scsi/linux.rs` shrank from - ~720 to ~520 lines. -- Module doc rewritten to reflect the new architecture. - -### Architecture: parity strip on macOS + Windows - -`scsi/macos.rs` and `scsi/windows.rs` had `try_recover()` — -userspace handle-recovery on task failure. Same anti-pattern as the -Linux fd-recovery dance, removed for the same reason: the kernel -mid-layer already runs its own escalation. Errors bubble up directly. - -Cleanups: -- `MacScsiTransport`: `try_recover()` deleted, `bsd_name` field - deleted (was only used by try_recover), fail-fast device_iface guard - deleted (no longer null'd mid-session). -- `SptiTransport`: `try_recover()` deleted, `wide_path` field deleted, - INVALID_HANDLE guard deleted. - -### API cleanup: drop `Drive::reset` and `find_drives` - -Two duplicates removed from the public surface: - -- `Drive::reset()` — escalating recovery (STOP/START unit + eject + - reinit). Per the audit, userspace shouldn't escalate; the kernel - already does. Only one internal caller (`wait_ready` line 195), - which now just keeps polling TUR for 60 iterations. No external - consumer used it. -- `pub fn find_drives() -> Vec` — opened N drives just to throw - most away. Only caller was `find_drive()` itself, which now uses - `discover_drives()` directly. No external consumer used it. For - lightweight enumeration (UI sidebar etc.) use `scsi::list_drives()`. - -`lib.rs` re-export of `find_drives` removed. - -## 0.13.19 (2026-04-26 — held, never released) - -Held in development; folded into 0.13.20. - -## 0.13.18 (2026-04-26) - -### Sync release — no functional changes - -Bumped to satisfy the unified-versioning rule. Actual fix is in autorip -(`web.rs` two-bar UI — separates per-pass and total progress bars + -their own text rows so the rip dashboard is readable again). - -## 0.13.17 (2026-04-26) - -### Sync release — no functional changes - -Bumped to satisfy the unified-versioning rule. Actual fix is in autorip -(hot-plug rescan in the drive poll loop — autorip now picks up -unplug/replug events without a container restart). - -## 0.13.16 (2026-04-26) - -### Architecture: single `Progress` trait + `PassProgress` struct - -Pre-0.13.16 the rip API leaked internal mapfile concepts (`pos`, -`bytes_good`, `work_done`, `bytes_pending`, `Finished`/`NonTrimmed`) -into per-pass positional callbacks. Consumers reinvented the math each -time, and the v0.13.15 UI bug surfaced exactly because of this — autorip's -web JS computed `progress_pct` from `bytes_good` while the backend -computed from `pos`, silent drift, frozen UI bar. - -This release replaces both `Disc::copy::on_progress` and -`Disc::patch::on_progress` `Fn(u64, u64, u64)` callbacks with a single -`Progress` trait and `PassProgress` struct (new `progress` module). - -```rust -pub struct PassProgress { - pub kind: PassKind, // Sweep | Trim {reverse} | Scrape {reverse} | Mux - pub work_done: u64, - pub work_total: u64, - pub bytes_good_total: u64, - pub bytes_total_disc: u64, -} - -pub trait Progress { - fn report(&self, p: &PassProgress); -} - -impl Progress for F { ... } -``` - -Both `CopyOptions::on_progress` and `PatchOptions::on_progress` are -renamed to `progress: Option<&dyn Progress>`. Closure callers update -trivially; struct callers gain a clean named-field shape with no -positional-arg confusion. - -`PassKind` carries the semantic (sweep vs trim vs scrape vs mux) so -consumers can label phases without reinventing detection logic. The -`Mux` variant is reserved for v0.13.17 when the mux pipeline emits -progress; not yet emitted by libfreemkv code. - -`Disc::patch` reports `Trim {reverse}` for retry passes with -`block_sectors >= 2` and `Scrape {reverse}` when `block_sectors == 1` -(the per-sector final pass). Direction comes through `reverse: bool`. - -## 0.13.15 (2026-04-26) - -### Breaking: `on_progress` callback gains `pos` parameter - -Both `CopyOptions::on_progress` and `PatchOptions::on_progress` now take -`Fn(bytes_good: u64, pos: u64, total_bytes: u64)`. The new `pos` parameter -is the current sweep / retry position. Pass 1 callers should display -`pos / total_bytes` for the "% swept" UI bar — `bytes_good` only counts -clean reads (Finished sectors) and freezes during skip-forward bad zones, -which made every previous version's UI look hung at the bad-zone boundary. -This was the v0.13.9 stall-guard origin bug. - -Live trace from v0.13.14: Pass 1 hit a bad zone at 24 GB and -appeared "stuck" for 14 minutes per autorip's UI (`bytes_good = 23.97 GB` -unchanged). Disc trace events showed `pos` actually advanced from 25.8 GB -to 70 GB during that window — Pass 1 was 83 % through the disc, marking -the post-bad-zone NonTrimmed via skip-forward exactly as designed. The -display lied. Now consumers can show the truth. - -### Feature: `PatchOptions::reverse` for reverse-direction retry passes - -When set, `Disc::patch` walks bad ranges from highest LBA to lowest, and -within each range reads sectors back-to-front. Hypothesis (per the live -v0.13.14 test): drives that wedge after a forward read of a bad sector -read fine when approached from end-of-disc backward — most of the -post-bad-zone NonTrimmed range is actually clean data the drive could -have read on Pass 1 had it not been wedged. autorip alternates F/R -across retry passes (Pass 2 = reverse half-batch, Pass 3 = forward -quarter-batch, ...). - -### Feature: `PatchOptions::wedged_threshold` early-exit - -When > 0, `Disc::patch` exits early if it sees this many consecutive -read failures with zero successful reads in the same pass. Saves the -wallclock budget for productive grinding when the drive has clearly -wedged on the bad zone for this pass — a future pass with a different -direction or block size may still recover. Reported via new -`PatchResult::wedged_exit: bool`. - -### Trace: `patch_start` and `patch_done` events - -`freemkv::disc` target now emits `patch_start` (block_sectors, recovery, -reverse, wedged_threshold, num_ranges) and `patch_done` -(blocks_attempted, blocks_read_ok, blocks_read_failed, wedged_exit, -halted, bytes_recovered) at Disc::patch boundaries. - -## 0.13.14 (2026-04-25) - -### Sync release — no functional changes in libfreemkv - -Bumped solely to satisfy the unified-versioning rule. The actual fix in -this release is in autorip: the tracing subscriber now enables -`freemkv::scsi=trace,freemkv::disc=trace` so the v0.13.13 instrumentation -events actually surface in `/api/debug`. Without that filter override the -trace events were silently dropped by the default `libfreemkv=warn` rule. - -## 0.13.13 (2026-04-25) - -### Telemetry: instrument the rip pipeline for in-flight diagnosis - -v0.13.12 shipped Fix 1+2+4 + cross-platform parity but a live test -showed Pass 1 sat for 14 minutes with `bytes_good=0` while the inner loop -appeared to iterate (the throttled `on_progress` log fired every 78s). The -async fd_recovery design at §7 said each `execute()` call should bound at -~1.5 s on poll timeout, with subsequent calls returning `DeviceNotFound` in -microseconds until recovery completes. Observed reality contradicts that: -each iteration takes ~60 s, not microseconds. Without trace-level telemetry -at the SCSI + Disc::copy boundaries we can't diagnose where the time goes. - -This release adds the telemetry. No behavior change; instrumentation only. - -- New dep: `tracing = "0.1"`. Per CLAUDE.md, debug/trace logging is permitted - in libfreemkv (the no-English rule applies to errors, not telemetry). - Consumers (autorip) wire a tracing subscriber and pipe events into the - JSONL debug log automatically. -- `SgIoTransport::execute` (Linux): trace events at every state transition - (entry, recovery_swap_ok, recovery_pending, write_ok / write_err, poll_done, - timeout_spawn_recovery, scsi_err, read_err, ok). Each event includes the - opcode and elapsed timing. The bg recovery thread also traces close_ms + - open_ms so we can see if the kernel is hanging close+open. -- `Disc::copy`: trace events at copy_start, outer_loop, region_enter, every - 100 inner-loop iterations (iter_progress with pos / region_end / skip_size / - bytes_good / read_ok_count / read_err_count / last_read_ms / - copy_elapsed_ms), and copy_done. -- All trace events use `target` strings `freemkv::scsi` and `freemkv::disc` - so consumers can filter by subsystem. - -### What this enables - -- A live rip will now produce a SCSI event stream visible at - `/api/debug?n=N&q=freemkv::scsi`. We can finally answer: "is the inner - loop iterating slowly because each call is slow, or fast with the bg - thread blocked?" -- `bg_recovery_done` events with `close_ms` / `open_ms` reveal whether the - kernel really takes 60 s for close+open on a wedged Initio bridge. - -## 0.13.12 (2026-04-25) - -### Fix: delete stall guard from `Disc::copy` (RIP_DESIGN.md §6 Fix 1) - -The v0.13.9 stall guard at `disc/mod.rs` exited Pass 1 early when -`bytes_good` was flat for `stall_secs` (default 120s). This violated the -ddrescue model: Pass 1 must sweep end-to-end, marking failed reads -NonTrimmed for Pass 2 retry. The guard caused Pass 1 to bail at 30% on -a UHD disc with 56 GB still NonTried, leaving Pass 2 nothing useful to do. - -- Deleted the stall-guard state vars and the `if cur_good != ... - break 'outer;` block. -- Deleted `CopyOptions::stall_secs` field — no longer wired. -- Replaced the broken regression test - `test_disc_copy_stall_detection_triggers_skip_forward` with - `test_disc_copy_completes_full_disc_with_failing_reader` (asserts Pass 1 - walks to end-of-disc with everything NonTrimmed when reads keep failing) - and added `test_disc_copy_halts_promptly_on_failing_reader` (halt flag - honored within 2s mid-skip-forward). - -### Fix: async SCSI transport recovery (RIP_DESIGN.md §6 Fix 2 / §7) - -`SgIoTransport::execute` (Linux) previously did close-in-background + -synchronous open-on-main-thread on poll timeout. The kernel serialized -the main-thread `open()` against the in-flight `close()` of the same -`/dev/sg*`, blocking the rip thread up to ~60s per timeout. - -- Added `fd_recovery: Arc` field. On poll timeout, both - `close(old_fd)` AND `open(new_fd)` run in a background thread; the new - fd is published to `fd_recovery`. Returns Err immediately. Main thread - is never blocked beyond the `poll()` budget (~1.5 s). -- Top of `execute()`: if `self.fd < 0`, swap from `fd_recovery`. If - recovery is also pending, return `DeviceNotFound` and let the caller's - retry loop come back later. -- Drop drains any pending `fd_recovery` so the fd doesn't leak. -- Stripped the v0.13.9 stall-guard narrative comment that justified - the deleted behavior. - -### Fix: cross-platform SCSI parity — Windows + macOS recovery (RIP_DESIGN.md §15.1) - -Per the platform parity rule (no stubs), Windows and macOS now have the -same observable recovery contract as Linux: - -- `SptiTransport` (Windows): added `try_recover()` that calls - `CloseHandle` + `CreateFileW` synchronously after a failed - `DeviceIoControl`. Stripped English error string ("run as - administrator") from `open()`. Fixed the ms→s timeout truncation - (1500ms now rounds up to 2s, was 1s). -- `MacScsiTransport` (macOS): added `try_recover()` that releases the - IOKit interface (`RELEASE_EXCLUSIVE` + `com_release`) and re-acquires - via the new `acquire_device_iface()` helper. Stores `bsd_name` so - recovery can re-call `find_scsi_service`. -- All three platforms: top of `execute()` returns `DeviceNotFound` - immediately if a prior `try_recover()` left the transport in an - invalid state. Drop guards null'd-out interfaces. -- Send is auto-derived on all three (i32 fd / isize HANDLE / IOKit - interface ref are Send-safe); explicit comments document the - intentional implicit Send and the absence of Sync. - -### Fix: instrument `Disc::patch` — diagnostic counters (RIP_DESIGN.md §6 Fix 4) - -`PatchResult` now reports `blocks_attempted`, `blocks_read_ok`, -`blocks_read_failed`. Pass 2's "100 minutes recovered 0 bytes" mystery -(a UHD disc) becomes diagnosable from these counters: distinguish "drive -returned Ok but write/record dropped data" from "every read was Err for -the entire range" without instrumenting from outside the lib. - -### Fix: honor `PatchOptions::full_recovery` - -The field was previously read into `let _ = opts.full_recovery;` and -ignored — `read_sectors(..., true)` was hardcoded. Now routed to -`read_sectors(..., opts.full_recovery)`. Behavior unchanged for -default callers (which pass `true`). - -### Doc: `CopyOptions::batch_sectors` accuracy - -Doc comment said "Defaults to 32 sectors (64 KB)". Updated to describe -the actual production path: callers should resolve via -`detect_max_batch_sectors(device_path)` (kernel-reported sysfs value, -typically 60 sectors / ~120 KB on the BU40N). The 32-sector internal -fallback is only reached when `batch_sectors=None AND skip_forward=true`. - -## 0.13.11 (2026-04-25) - -### Fix: revert SgIoTransport timeout path to keep transport alive - -v0.13.10 changed `SgIoTransport::execute` to set `fd = -1` on a poll -timeout (no reopen on the main thread, since that would serialize -against the spawned close()). The intent was to escape the 60-s -blocking reopen. - -The cost was too high: a single transient poll timeout permanently -killed the transport. Live test on a UHD disc (post-replug): -- Pass 1 ran for **45 ms** then returned with 0 GB good and 80 GB - pending. -- The first SCSI READ timed out, fd went to -1, every subsequent - read returned `DeviceNotFound` instantly, Disc::copy raced through - the entire disc skip-forwarding in milliseconds. -- Pass 2 inherited the dead Drive and was equally useless. - -Revert: spawn close + reopen on main thread (the v0.13.5/8 -behavior). Yes the main-thread open() may block up to ~60 s while -the kernel completes the abandoned command — but the v0.13.9 -`Disc::copy` stall guard already caps catastrophic stalls at 120 s -of `bytes_good` non-advance. Net: per-timeout cost is ~60 s, but -Pass 1 cleanly bails out within 120 s of any wedge, and Pass 2 has -a working Drive to retry NonTrimmed ranges with `recovery=true` + -30 s timeouts. - -The integration test for the stall guard -(`test_disc_copy_stall_detection_triggers_skip_forward`) continues -to pass — the guard fires regardless of which transport-recovery -strategy is in play. - -## 0.13.10 (2026-04-25) - -### Version sync — no functional changes - -Sync bump for the autorip-side fix in 0.13.10 (Pass 1 batch reporting). - -## 0.13.9 (2026-04-25) - -### Fix: Disc::copy silent stall + SgIoTransport reopen-after-timeout serialization - -Two correlated fixes for a hang observed live on the LG BU40N during a -v0.13.8 rip of a UHD disc. At ~30 % progress through Pass 1 -(disc → ISO), `bytes_good` froze for 10+ minutes with `errs=0`, -no error surfaced, drive not wedged. - -Root cause: `SgIoTransport::execute` (linux.rs) attempted to recover from -a `poll()` timeout by spawning a background `close()` of the old fd and -opening a fresh `/dev/sg*` fd on the main thread. On Linux, opening the -SAME device while a prior fd is mid-close serializes via the kernel's -per-device state lock — so the fresh `open()` blocks for as long as the -close does (until the kernel completes the in-flight CDB). This undid -the userspace 1.5 s timeout: each timed-out read added 60+ s to the -next iteration. From `Disc::copy`'s perspective, reads kept returning -Err slowly, the skip-forward path advanced `pos` but never `bytes_good`. - -Fixes: -- `SgIoTransport::execute` no longer reopens on timeout. Spawns the - close, sets `self.fd = -1`, returns Err immediately. Subsequent - calls fail with `DeviceNotFound` (already gated at line 248). - Caller (Drive) is invalidated until reopened. Pass 2's - `Disc::patch` would need a fresh Drive; that's a v0.14 follow-up. -- `Disc::copy` adds a stall guard. New `CopyOptions::stall_secs: - Option` (default 120 s). If `bytes_good` doesn't advance for - the threshold, breaks the outer loop with `complete: false, - bytes_pending > 0` so the caller's retry path picks up. - -Tests: new `test_disc_copy_stall_detection_triggers_skip_forward` in -`tests/integration_progress_and_halt.rs` proves the guard fires within -the configured threshold. - -Other: -- Cosmetic: warning text "rip thread did not drain within 35s" updated - to 60s (matches the v0.13.8 timeout bump). - -## 0.13.8 (2026-04-25) - -### Version sync — no functional changes - -Sync bump for the ecosystem. 0.13.8 carries autorip-side fixes: -post-stop "error" leak (halt-aware Err handling in Pass 1/2+), -60 s drain timeout, and a structural spawn_rip_thread helper. - -## 0.13.7 (2026-04-25) - -### Version sync — no functional changes - -Sync bump for the ecosystem. All four freemkv crates (libfreemkv, -freemkv CLI, bdemu, autorip) always share a version number; 0.13.7 -carries an autorip-side fix (HTTP-spawned rip/scan threads now -register for stop-drain). - -## 0.13.6 (2026-04-25) - -### Inline retry/reset stripped from `Drive::read`; `BytesRead` now emitted - -Two related changes that close the loop on the BU40N wedge work from -0.13.1–0.13.4 and on the long-standing autorip "0 KB/s, 0%" UI bug. - -**`Drive::read` is now single-shot.** The phase 1 / 2 / 3 retry loop -(reset → reopen → repeat) inside `Drive::read` is gone (~80 lines -deleted). `recovery=true` only bumps the per-CDB timeout to 30 s; -`recovery=false` keeps the 1.5 s timeout. On a failed read the -function returns `Err(DiscRead)` immediately. Per the BU40N -post-mortem, every USB / SCSI reset path tested in 0.13.1–0.13.3 -resets the bridge but not the drive firmware, and the inline -reset+reopen *was* the wedge primitive itself — issuing it from -inside `Drive::read` produced multi-minute hangs and made the wedge -class harder to surface to the user. The correct retry layer is -`Disc::patch`'s outer multi-pass loop, which is unaffected. A stuck -drive now surfaces as a clean `DiscRead` to the caller, who can -prompt physical replug. - -**SCSI reset surface trimmed.** `SgIoTransport::reset` (Linux) drops -the `SG_SCSI_RESET` ioctl and the STOP / START UNIT escalation; it -keeps the kernel `SG_IO` state flush plus `ALLOW MEDIUM REMOVAL`. -`MacScsiTransport::reset` is removed entirely (was open + drop + -sleep, no SCSI). The top-level `scsi::reset` / -`scsi::reset_with_timeout` / `scsi::reset_blocking` family is -removed — no callers remain after the `Drive::read` strip. - -**`EventKind::BytesRead` now emitted.** The variant was declared in -0.13.0 but never fired. `DiscStream::fill_extents` now emits -`BytesRead { bytes_read_total, total_extents_bytes }` after every -successful sector read, so consumers in direct (no-mapfile) mode can -drive a real-time progress bar without polling `output.bytes_written`. -Multi-pass mode continues to use `Disc::copy`'s `on_progress` -callback unchanged. Drives the autorip per-device live progress UI. - -`Drive::checked_sleep` is removed (only used by the recovery loop); -`Drive::sleep_until_halted` is `#[cfg(test)]`-only; `Drive::emit` is -retained because `BytesRead` uses it. - -### Tests -- New `tests/integration_progress_and_halt.rs` (5 tests): `BytesRead` - emission, `Disc::copy` `on_progress` regression guard, halt aborts - copy, Drop safety, `FileSectorReader` round-trip. -- 233 unit tests + 5 integration tests pass. - -### Net diff -~80 lines deleted, ~20 added. - -### Version sync -0.13.6 ecosystem release (libfreemkv + freemkv + bdemu + autorip all -on 0.13.6). - -## 0.13.5 (2026-04-25) - -### Version sync — no functional changes -Sync bump for the ecosystem. All four freemkv crates (libfreemkv, -freemkv CLI, bdemu, autorip) always share a version number; 0.13.5 -carries autorip-side fixes (stop-is-reset, startup staging sweep). - -## 0.13.4 (2026-04-25) - -### Wedge recovery rolled back + sysfs identity fallback - -**What changed.** The in-library USB / SCSI wedge-recovery escalation -added in 0.13.1 – 0.13.3 has been removed. `drive_has_disc` now returns -the raw TUR result (or the `0xFF` poll-timeout wedge error) directly to -the caller. `scsi::usb_reset()` / `usb_reset_with_timeout()` / -`DEFAULT_USB_RESET_TIMEOUT_SECS` and the per-platform -`SgIoTransport::usb_reset` / `MacScsiTransport::usb_reset` / -`SptiTransport::usb_reset` are gone. All three platform backends pass -transport errors through verbatim, keeping the public -`list_drives` + `drive_has_disc` contract symmetric -(Linux / macOS / Windows). - -**Why.** Production testing against the LG BU40N USB BD-RE (the drive -that drove the whole 0.13.1–0.13.3 recovery push) showed: -- `SG_SCSI_RESET`, `STOP UNIT` + `START UNIT`, and `USBDEVFS_RESET` - all succeed at the USB transport layer (kernel logs - `usb 3-2: reset high-speed USB device`, device re-authorises). -- But the drive firmware *below* the USB bridge stays locked: no - LUN enumerates on the fresh `scsi_host`, TUR never succeeds, - `/dev/sg*` never reappears. -- Also tried (outside the lib): `/sys/bus/usb/devices//authorized` - toggle, `usb-storage` driver unbind/rebind, forced SCSI host rescan. - All same outcome. - -Only physical unplug-replug (or host reboot) clears this wedge class. -The library was logging 2-minute-per-tick escalation cycles for nothing, -and consumers had no way to surface "drive needs physical intervention" -to users because the escalation was masking the real failure. Upper -layers (autorip, CLI) now see the wedge error directly and prompt the -user. - -A breadcrumb in `scsi/linux.rs::drive_has_disc` catalogues every -recovery method tried and points to git tag `v0.13.3` for the full -implementation, in case a future hardware class is found where -USB-layer recovery actually works. - -**New: sysfs-cached identity fallback (Linux).** `list_drives` now -populates empty INQUIRY vendor/model/firmware fields from -`/sys/class/scsi_generic/sgN/device/{vendor,model,rev}` — the kernel -runs its own INQUIRY at device probe time and stashes the answer there, -so even a mid-wedge INQUIRY still yields the UI a human-readable -identity. The drive surface on screen doesn't suddenly go blank the -moment the drive firmware locks up. - -## 0.13.3 (2026-04-24) - -### Bug fix — `drive_has_disc` wedge recovery was dead code for TUR errors - -The wedge-signature predicate introduced in 0.13.2 gated on -`opcode == SCSI_INQUIRY (0x12)` — a holdover from when enumerate-time -INQUIRY was the only path wedges surfaced on. `drive_has_disc` issues -`TEST UNIT READY (0x00)`, so its wedge errors (`E4000: 0x00/0xff/0x00`) -never matched the predicate and the SCSI-reset + USB-reset escalation -never fired. Production result: the BU40N USB BD-RE stayed wedged -indefinitely, with autorip logging `recovery exhausted` on the raw -pass-through error while no recovery had actually been attempted. - -Fix: drop the opcode constraint. Status byte `0xFF` is synthesised by -our own `execute()` path when `poll()` on the SG fd times out; it's -the ground-truth wedge marker regardless of which opcode was in flight. -Doc comments on `is_wedge_signature` and `WEDGE_STATUS_BYTE` updated -accordingly. Linux-only — macOS / Windows use sense-key-based wedge -detection and are unaffected. - -## 0.13.2 (2026-04-24) - -### Public discovery + presence APIs; SCSI/USB primitives no longer -### exposed to consumer crates - -The autorip / freemkv-CLI side of the ecosystem was reimplementing -hardware discovery (sysfs walking, SCSI type-5 filtering, sg-path -construction) and SCSI recovery primitives in their own crates — a -direct violation of the architectural rule that ALL hardware-aware -code lives in libfreemkv. 0.13.2 closes that gap with two cheap public -probes that absorb everything consumers were doing themselves, plus -visibility tightening to make future violations a compile error. - -#### New public APIs - -- `pub struct DriveInfo { path, vendor, model, firmware }` — a single - enumerated optical drive's identity. Returned by `list_drives()`, - populated from a single SCSI INQUIRY at enumeration time. No - firmware reset, no `init`. -- `pub fn list_drives() -> Vec` — one-shot enumeration - across Linux/macOS/Windows. Linux walks `/sys/class/scsi_generic/` - with the SCSI type-5 filter and `/dev/sg0..15` fallback; macOS - walks `/dev/disk0..15` with the INQUIRY peripheral-type-5 filter; - Windows iterates `CdRom0..15`. Cheap (~10 ms / drive); cache the - result and refresh on udev events. -- `pub fn drive_has_disc(path: &Path) -> Result` — single TEST - UNIT READY. Returns `Ok(true)` when ready / `Ok(false)` on sense-key 2 - ("medium not present") / `Err` only after recovery has been exhausted. - **Internal wedge recovery is hidden from callers** — when the kernel - returns the wedge-signature pattern (status 0xFF, no sense), this - function transparently escalates: SCSI bus reset → if still wedged → - USB device reset → retry TUR. Consumers never see the escalation. - -#### USB-layer reset, multi-platform - -`USBDEVFS_RESET` (Linux) is the only thing that recovers a kernel- -level USB Mass Storage wedge — software equivalent of unplug-replug. -Now wired for all three OSes: - -- **Linux**: `USBDEVFS_RESET` ioctl on `/dev/bus/usb/BBB/DDD`. Resolves - sg → USB device via sysfs walk (`busnum`/`devnum` parents). -- **macOS**: `IOUSBDeviceInterface::ResetDevice()`. Walks IORegistry - parents from the SCSI service to the USB device, queries the IOKit - USB plugin, calls ResetDevice. -- **Windows**: existing `IOCTL_STORAGE_RESET_DEVICE` covers both SCSI - and USB layers via storport, so `usb_reset` returns `DeviceNotFound` - by design — the recovery escalation in `drive_has_disc` falls - through cleanly. (See the `windows::usb_reset` doc comment for - why a separate cycle-port IOCTL isn't needed on Windows.) - -All wrapped in a thread + `mpsc::recv_timeout` so a kernel ioctl that -hangs forever can't lock up the caller (the inner thread leaks one OS -thread per hard wedge — acceptable for a daemon that recovers vs. one -that wedges the whole poll loop). - -#### Visibility tightening (architectural enforcement) - -These were `pub` in 0.13.1; consumer crates could (and did) call them -directly, leaking SCSI knowledge across the lib boundary: - -- `scsi::reset` → `pub(crate)` -- `scsi::reset_with_timeout` → `pub(crate)` -- `scsi::usb_reset` → `pub(crate)` -- `scsi::usb_reset_with_timeout` → `pub(crate)` -- `DEFAULT_RESET_TIMEOUT_SECS` / `DEFAULT_USB_RESET_TIMEOUT_SECS` → - `pub(crate)` - -Consumers now reach recovery exclusively through `drive_has_disc`, -which folds the escalation in. **Compile-time guarantee** that no -future autorip/CLI/bdemu commit can reintroduce direct SCSI access. - -#### Why this design - -`Drive::open(path)` runs a ~2 s firmware-reset preamble + identify -sequence; suitable for ripping but wasteful for a poll loop probing -"is there a disc?". Pre-0.13.2 autorip called `Drive::open` 4 × every -5 s = ~17 000 speculative SCSI sessions/day, hammering the drives -between actual rips. The wedge in production at 23:51 UTC was -triggered by exactly this hot-loop pattern. With `drive_has_disc`, -the same poll cadence costs ~50 ms / drive (one TUR) — 40× cheaper -and side-effect-free on a healthy drive. - -#### Tests - -- 233 lib tests pass (no change in count; APIs covered indirectly via - the existing transport tests + a new `device_key` test on the - autorip side). -- `cargo clippy --all-targets -D warnings` clean across Linux/macOS. - -## 0.13.1 (2026-04-24) - -### `scsi::reset()` now has a hard wallclock timeout - -Production incident on a wedged BU40N USB drive: autorip's poll loop -called `scsi::reset()` and the call hung for 60+ seconds before the -operator manually intervened. Root cause: the Linux `SG_SCSI_RESET` -ioctl can block indefinitely when the kernel SCSI subsystem is waiting -on a bus-wedged device that will never ack — there's no kernel-side -timeout on this ioctl. Without an outer wallclock bound the caller's -thread is stuck in the kernel until the device unwedges (which, for a -permanently-dead USB target, may be never). - -`scsi::reset()` is now a wrapper that runs the platform-specific reset -on a detached worker thread and bounds the caller's wait via -`mpsc::recv_timeout(DEFAULT_RESET_TIMEOUT_SECS)` (30 s). Returns -`DeviceResetFailed` on timeout. The worker thread keeps running until -the kernel eventually unblocks (we can't cancel a Linux ioctl from -userspace) — this leaks one OS thread per hard wedge, an acceptable -cost for a daemon that recovers vs. one that hangs. - -- New `pub const DEFAULT_RESET_TIMEOUT_SECS: u64 = 30;` -- New `pub fn reset_with_timeout(device, Duration) -> Result<()>` for - callers that want a different bound. -- Existing `pub fn reset(device) -> Result<()>` keeps the same - signature; behaviour change is the timeout, not the API. - -### Follow-up flagged - -`SG_SCSI_RESET` only resets at the SCSI layer. For USB-attached drives -(the BU40N case), the wedge is often in the USB Mass Storage layer -*below* SCSI — `SG_SCSI_RESET` doesn't help. The proper escalation is -`USBDEVFS_RESET` (the `usbreset.c` ioctl), which re-enumerates the -device at the USB layer. Tracked for 0.13.2: a `scsi::usb_reset(path)` -that resolves sg → USB device and issues `USBDEVFS_RESET`. That would -have recovered tonight's BU40N without operator intervention. - -## 0.13.0 (2026-04-24) - -### Zero English in library — typed variants for every error path - -Audit pass against the `CLAUDE.md` rule (no English text in library code). -Found nine call sites that violated the contract by stuffing English into -`io::Error::new(kind, "…")` or by abusing `Error::DeviceNotFound { path }` -as a free-form description field. Each is now a typed variant with -structured fields; the CLI / autorip translates to localized text. - -New `Error` variants and codes: - -- `ScsiInterfaceUnavailable { path }` — `E1004` (macOS - `SCSITaskDeviceInterface` couldn't be obtained) -- `DeviceLocked { path, kr }` — `E1005` (replaces an English - "exclusive access denied. Try: diskutil unmountDisk" message) -- `IoKitPluginFailed { path, kr }` — `E1006` -- `UnsupportedPlatform { target }` — `E2003` (built on an OS without an - SCSI backend) -- `PlatformNotImplemented { platform }` — `E2004` (replaces the - `product_revision: "Renesas not yet implemented"` string-stuffing in - `drive::mod`) -- `MapfileInvalid { kind }` — `E6011` (ddrescue mapfile parse, with a - stable `&'static str` kind: `"status_char"` or `"hex"`) -- `DiscUrlNotDirect` — `E9009` (replaces the full English sentence - `"Use Drive::open() + Disc::scan() + DiscStream::new() for disc sources"` - that `mux::input(disc://…)` returned to callers) - -Migrated call sites: - -- `mux/resolve.rs` — disc URL → `DiscUrlNotDirect`; the four - `format!("m2ts://…")` / `format!("mkv://…")` IO error wraps now - propagate the inner `io::Error` unchanged (the URL-prefix wrap added - no semantic information). -- `mux/iso.rs` — same `format!("iso://…")` wrap dropped. -- `sector.rs` — image-too-large now uses the existing `IsoTooLarge` - variant instead of `format!("…image too large, max ~8 TB")`. -- `disc/mapfile.rs` — bad-status-char and bad-hex parser errors now use - `MapfileInvalid { kind }`. -- `scsi/mod.rs` — unsupported-platform path uses `UnsupportedPlatform`. -- `scsi/macos.rs` — IOKit plugin failure → `IoKitPluginFailed`, - `SCSITaskDeviceInterface` missing → `ScsiInterfaceUnavailable`, - exclusive-access denied → `DeviceLocked` with the IOReturn code as a - structured `kr` field. The `find_scsi_service` four-stage failure path - is now a single `DeviceNotFound { path }` (none of the prior - per-stage English descriptions were user-actionable individually). -- `drive/mod.rs` — Renesas platform → `PlatformNotImplemented`. - -### Stripped English from `labels` module - -`labels::apply()` previously pushed `"Commentary"`, `"Descriptive Audio"`, -`"Score"`, `"IME"`, and `" (Secondary)"` directly into -`AudioStream.label`. Those English strings then leaked into MKV titles -and into autorip's UI. The data was already structured upstream -(`LabelPurpose` enum on `StreamLabel`); the lib was downcasting it for -the caller's convenience. - -- `AudioStream` gains `purpose: LabelPurpose` (re-exported from - `crate::disc` next to the struct, alongside `LabelQualifier`). -- `SubtitleStream` gains `qualifier: LabelQualifier`. -- `apply()` writes structured fields, never English. `label` keeps - codec-formatting only (`"Dolby TrueHD 5.1"`). -- `generate_audio_label` drops the `" (Secondary)"` suffix; `secondary` - is already a `bool` field, callers render it. -- `generate_video_label` drops the `"Secondary Video"` fallback. - `"Dolby Vision EL"` kept (brand identifier, not translatable). - -### API hygiene - -- **mux module visibility tightened**. `pub mod ebml`, `m2ts`, `mkv`, - `network`, `null`, `ps`, `stdio`, `ts`, `tsmux` are now `pub(crate)`. - Their *types* are still re-exported from `lib.rs` — the modules - themselves were leaking low-level EBML primitives, TS muxer - internals, and network/stdio implementations that no external caller - used. `mux::codec`, `mux::disc`, `mux::iso`, `mux::resolve`, and - `mux::meta` stay public (genuine APIs). -- **Stream trait rustdoc**. The keystone PES `Stream` trait (in - `pes.rs`) had per-method docs but no trait-level doc. Now explains - read-vs-write split, error contracts, the role of `info()` / - `codec_private()` / `headers_ready()`. -- **lib.rs re-export sections**. Eight grouped sections with a - paragraph each (Drive lifecycle, Errors, Decryption, Disc structure, - Streams, Lower-level surfaces) so `cargo doc` tells callers when to - reach for what. -- **Dropped `ScanOptions::with_keydb()`**. The `_with_X` constructor - pattern was banned by `CLAUDE.md` (one method per action). Use the - struct literal: `ScanOptions { keydb_path: Some(p.into()) }`. Five - external call sites (autorip ×3, freemkv CLI ×3) and three test - fixtures migrated. -- **`pid_index` allocation documented**. The `TsDemuxer::new` flat - lookup table was flagged by audit as "unbounded for adversarial - PIDs"; on closer reading it's bounded by `u16::MAX × 2 bytes ≈ 128 KB`. - Doc comment now states the bound explicitly so future contributors - don't re-flag it. - -### Dead-code sweep - -Pre-PES-rewrite leftovers that were `pub` but unreachable: - -- Deleted `mux/lookahead.rs` entirely (orphan file — never had a `mod` - declaration; only used by its own tests). -- Deleted `mux/tsreader.rs` (`TsDemuxReader` struct + four methods, - used nowhere). -- Deleted `mux::ebml::write_int`, `read_vint`, `SEEK_HEAD`, `SEEK`, - `SEEK_ID`, `SEEK_POSITION` (unused). -- Deleted `mux::ts::scan_first_pts`, `scan_last_pts`, `scan_duration`, - `SCAN_HEAD_SIZE`, `SCAN_TAIL_SIZE`, `take_remainder`, `set_remainder` - (unused since the v0.10 PES rewrite). -- Deleted `MkvMuxer::codec_private_slots` / - `codec_private_filled` fields and `fill_codec_private` method — - deferred-codecPrivate path was never exercised once codec_privates - flowed through `DiscTitle`. - -`cargo clippy --all-targets -- -D warnings` is clean. - -### Tests - -- New `error::tests` module — code distinctness, Display has no English - words, `io::ErrorKind` mapping for every new variant. -- 233 lib tests (was 230), all green. - -### Breaking changes - -Source-compatible for callers who use `Error` opaquely (handle -`Result` and `error.code()` only). The following are breaking: - -- `ScanOptions::with_keydb()` removed — use struct literal. -- `mux::ebml`, `mux::mkv`, `mux::ts`, etc. modules no longer accessible - externally — use the re-exported types from the crate root instead. -- `AudioStream` and `SubtitleStream` gained required fields (`purpose`, - `qualifier`). Construction-by-struct-literal must include them. -- `Error::UnsupportedDrive { product_revision: "Renesas not yet - implemented" }` no longer produced — match `PlatformNotImplemented`. - -### Magic-number policy - -Per the v0.13 audit directive: new code in this release uses named -documented constants (e.g. `POLL_INTERVAL_SECS` in autorip, the wedge -signature literals in `ripper.rs`, the `BD_TS_PID_SPACE` floor in -`ts.rs`'s table allocation). A comprehensive retrofit of pre-existing -magic numbers across the older codebase is queued as follow-up work for -0.13.1+ — too large to absorb into this release without scope creep. - -## 0.12.0 (2026-04-24) - -### Rust 2024 edition migration -- Bumped `edition = "2024"`. Required code changes: - - FFI declarations in `src/scsi/macos.rs` wrapped in `unsafe extern "C" { … }` per the 2024 FFI safety rules. - - `unsafe_op_in_unsafe_fn` lint: `vtable_fn()` body now has an explicit `unsafe { … }` block rather than relying on implicit unsafe of the containing `unsafe fn`. - - Match-ergonomics: removed redundant `ref`/`ref mut` bindings in `mux/meta.rs`, `mux/mkvstream.rs`, `mux/network.rs`, `mux/stdio.rs` — 2024 tightens "cannot explicitly borrow within an implicitly-borrowing pattern." -- No behavior change. MSRV stays at 1.86. - -### Minor / version sync -- Part of the 0.12.0 ecosystem release. The autorip-side fixes (progress regressions, UI redesign, regression-guard tests) drove the minor bump. - -## 0.11.22 (2026-04-24) - -### Version sync — no functional changes -Part of the 0.11.22 ecosystem release. autorip 0.11.22 ships full multi-pass UI (bad-range viz, live mapfile stats, Recovery settings); the library API is unchanged from 0.11.21. - -## 0.11.21 (2026-04-24) - -### Multi-pass rip architecture — disc → ISO → patch → ISO - -New primitives for two-stage rip: fast forward pass with zero-fill on failures, then targeted retries of bad ranges. Keeps the library API stream-based; the multi-pass model lives entirely in caller-orchestrated function composition. - -- **New `Disc::copy(reader, path, &CopyOptions)`** replaces the positional-arg version. Always produces a ddrescue-format mapfile at `path + ".mapfile"` as a side-effect. With `skip_on_error=true` + `skip_forward=true`, does ddrescue-style fast sweep: 64 KB block reads, exponential skip-forward (256 KB → cap at 1% of disc) on failure, zero-fill bad blocks, record ranges in the mapfile. With defaults (both false), matches pre-0.11.21 behavior — uses drive-level recovery, aborts on bad sector. Mapfile is produced either way. -- **New `Disc::patch(reader, path, &PatchOptions)`** — idempotent retry pass. Reads the mapfile, re-reads every non-`+` range with full drive recovery enabled, writes successful bytes back into the ISO at exact offsets, updates mapfile. Call N times for N retry attempts. -- **New `disc::mapfile` module** — ddrescue-compatible plain-text format. Crash-safe (flushes on every `record()`), greppable, human-editable, tool-interoperable. Status chars match ddrescue: `?` non-tried · `*` non-trimmed · `/` non-scraped · `-` unreadable · `+` finished. -- **Re-exports:** `FileSectorReader` from the crate root for ISO readers. - -### Breaking changes -- `Disc::copy`'s signature changes from positional args (`decrypt, resume, batch, on_progress`) to `CopyOptions`. Previous callers must migrate. `freemkv` CLI updated in lockstep. - -### Version sync -- Part of the 0.11.21 ecosystem release (libfreemkv + freemkv + bdemu + autorip all on 0.11.21). - -## 0.11.18 (2026-04-24) - -### DiscStream halt flag — Stop works during dense bad-sector regions - -`DiscStream::fill_extents` loops internally when the demuxer hasn't accumulated enough data to emit a PES frame — during a dense bad-sector run, that loop can spend many minutes shrinking batch sizes and zero-filling sectors without ever returning to the outer read() call. Without an internal halt check, the caller's Stop request goes unserviced until the demuxer eventually emits a frame, which may be very far away. - -- **`DiscStream::set_halt(Arc)`** — share a halt flag with the stream. Typically wired to `Drive::halt_flag()` so Stop propagates across both the drive's recovery phases and the stream's sector processing. -- **`fill_extents()` checks the halt flag** at the top of every retry iteration (before each attempt at every size level). Raising the flag aborts within one read round-trip — at most the current SCSI command's timeout. -- Returns `Err(Error::Halted)` (E6010) so the outer rip pipeline terminates cleanly. - -No behavior change for callers that don't call `set_halt`. Unblocks the architectural fix for the "Stop doesn't stop" bug observed on a damaged UHD disc where the stream was stuck in a 12+ hour bad-sector grind. - -## 0.11.17 (2026-04-23) - -### Adaptive batch sizer in DiscStream — no more per-sector descent - -Rip recovery rewritten. The old binary-search-per-bad-sector model paid the full descent (batch → half → quarter → … → single) for every bad sector in a region. On a damaged disc with 600 consecutive bad sectors this took 12+ hours. The new algorithm pays the descent once, remembers the working size, and ramps back up only after a sustained clean streak. - -- **`BatchSizeChanged { new_size, reason }` event** — fires on shrink (read failed) and probe-up (clean streak threshold hit). Consumers use this to distinguish a "recovering" rip from a normal one. -- **Removed `BinarySearch` and `SectorRecovered` emissions from DiscStream** — no longer produced by the rip path. `SectorRecovered` still fires from `Drive::read`'s multi-phase recovery (unused by rips today, but kept for scan/other callers). -- **Removed `read_with_binary_search` and the 3×5s light-recovery loop** — no retry loops, no sleeps. One 5s attempt per read. On size-1 failure, skip (zero-fill) or error. -- **Probe-up threshold: 100 MiB (51,200 sectors) of clean reading at current size** before doubling toward preferred. Ramp 1 → preferred on good reading takes ~100 seconds for a typical BD — trivial vs. rip duration, conservative enough that a single lucky sector in a marginal zone can't trigger a premature probe. -- **Bad-region math**: ~600 consecutive bad sectors now complete in ~50 min (600 × 5s) instead of ~12h. The descent is O(log preferred) one time, not per sector. - -### macOS - -- Fix new clippy lint (`manual_c_str_literals`) in `scsi/macos.rs`. - -## 0.11.16 (2026-04-21) - -### API cleanup — one method per action -- **SectorReader::read_sectors(lba, count, buf, recovery)** — single method with `recovery: bool`. Removes `read_sectors_recover()`. -- **parser_for_codec(codec, codec_data)** — single constructor. Removes `parser_for_codec_with_data()`. -- **DvdSubParser::new(codec_data)** — single constructor. Removes `with_codec_data()`. -- **MkvMuxer::new(writer, tracks, title, duration, chapters)** — single constructor. Removes `new_with_chapters()`. - -## 0.11.15 (2026-04-21) - -### Lint cleanup -- Fix all `cargo fmt` and `cargo clippy -D warnings` across codebase. -- Remove unused imports, dead code, collapsible if-statements, div_ceil reimplementation. - -## 0.11.14 (2026-04-21) - -### Audit fixes: read recovery, verify, SCSI -- **Fix: trailing sectors at extent boundaries** — extents with sector_count not divisible by 3 no longer drop 1-2 trailing sectors. decrypt_sectors() safely skips partial AACS units. -- **Fix: verify_title stop support** — progress callback now returns bool. Return false to stop verification early instead of running to completion. -- **Fix: O_CLOEXEC on all SCSI fd opens** — prevents fd leak to child processes. -- **Fix: SCSI sense descriptor format** — correctly detect response code 0x72/0x73 (descriptor format) and extract sense key from byte 1 instead of byte 2. -- **Fix: DecryptFailed on missing unit key** — decrypt_sectors() returns Err(DecryptFailed) instead of silently using a zero key. - -## 0.11.13 (2026-04-21) - -### Fix: all rip reads use fast timeout -- Initial batch read changed from full Drive::read() recovery to fast 5s timeout. Binary search starts immediately on failure instead of after 10 minutes of retries. -- Max 15 seconds per bad sector (3 x 5s attempts). Max 23 seconds per batch with 1 bad sector. - -## 0.11.12 (2026-04-21) - -### Drive halt + sector events + light recovery -- **Drive.halt()** — AtomicBool flag checked between retry phases. Max 30s to stop. -- **Drive.on_event()** — callback for ReadError, Retry, SpeedChange, SectorRecovered events. -- **Error::Halted (E6010)** — distinct from DiscRead, indicates intentional stop. -- **Binary search light recovery** — single sectors get 3 attempts x 5s (15s max) instead of full 10-min Drive::read() recovery. Marginal disc zones complete in minutes not hours. -- **DiscStream.on_event()** — BinarySearch, SectorRecovered, SectorSkipped events. - -## 0.11.11 (2026-04-20) - -### Binary search error recovery -- **fill_extents binary search** — when a batch read fails, binary search to isolate the failing sector(s). Good sectors read in sub-batches at full speed. Only truly bad sectors get individual recovery. 60-sector batch with 1 bad sector: ~5 seconds instead of 10+ minutes. - -## 0.11.10 (2026-04-20) - -### Skip errors + clean verify API -- **DiscStream.skip_errors** — when true, zero-fills unreadable sectors and continues instead of aborting. Caller sets based on user preference. -- **read_sectors_recover(recovery: bool)** — single API for recovery vs fast reads. Replaces separate read_sectors_fast method. - -## 0.11.9 (2026-04-20) - -### Fast verify reads -- **read_sectors_fast()** — single-attempt 5s timeout SCSI read for verify. No recovery loop. Bad sectors detected in seconds instead of 10+ minutes. -- **SectorReader trait** — added read_sectors_fast() with default fallback to read_sectors(). - -## 0.11.8 (2026-04-20) - -### Disc verify -- **verify::verify_title()** — sector-by-sector health check. Classifies sectors as Good/Slow/Recovered/Bad. Progress callback, chapter mapping, sector ranges. - -## 0.11.7 (2026-04-19) - -### TrueHD parser rewrite -- **12-bit length mask** — access unit length is lower 12 bits of first 2 bytes, not full 16. Upper 4 bits are parity nibble. Wrong mask caused misaligned frame splits. -- **AC-3 frame skipping** — BD-TS TrueHD PES contains interleaved AC-3 frames (same PID). Parser now detects AC-3 sync word (0x0B77) and skips those frames. -- **Cross-PES buffering** — access units that span PES packet boundaries are correctly reassembled. -- **Per-unit timestamps** — each access unit gets incrementing PTS (1/1200th second apart) instead of all units in one PES sharing the same timestamp. -- **Major sync detection** — keyframe flag set when access unit contains MLP major sync (0xF8726FBA). -- Result: zero TrueHD decode errors on UHD and BD (was ~19 per 30 seconds). - -## 0.11.6 (2026-04-18) - -### TrueHD fix (incomplete) -- Initial attempt at TrueHD header stripping — wrong approach, superseded by 0.11.7. - -## 0.11.5 (2026-04-18) - -### MKV container fixes — Jellyfin/player compatibility -- **Timestamp normalization** — MKV and M2TS output starts at 0.000s instead of raw disc PTS offset. Fixes playback failures in Jellyfin and other players. -- **DefaultDuration** — correct frame rate written to MKV track header. Fixes wrong avg_frame_rate (was 293/12, now 24000/1001). -- **HDR Colour metadata** — MatrixCoefficients, TransferCharacteristics, Primaries, Range written to MKV video track. Enables HDR tone mapping in players. -- **DisplayWidth/DisplayHeight** — aspect ratio fields in MKV video track. -- **Chapters (Blu-ray)** — accept mark_type 0 as chapter entry (was filtering to type 1 only, which no disc uses). -- **Chapters (DVD)** — extract chapter timestamps from PGC program map + cell durations. -- **Default disposition** — only first video and first audio track marked default. Fixes wrong auto-selection in players. - -## 0.11.3 (2026-04-18) - -### Unified versioning -- All freemkv repos now share the same version number. No functional changes from 0.10.10. - -## 0.10.10 (2026-04-18) - -### Dual-layer disc fix -- **UDF extent allocation** — use actual UDF allocation descriptors (`file_extents()`) instead of assuming m2ts files are contiguous from `file_start_lba`. Dual-layer UHD discs split large files across many extents (~1 GB each). The old single-extent assumption truncated rips at ~37% on affected discs. -- **Read error propagation** — `fill_extents()` returns `io::Result` so SCSI read errors propagate to the caller instead of being silently treated as EOF. - -## 0.10.9 (2026-04-17) - -### Fast disc identification -- **Disc::identify()** — reads UDF filesystem only (name, format, layers, encrypted). ~3s on USB vs 18s for full scan. No AACS handshake or playlist parsing. -- **KEYDB path fix** — added `~/.config/freemkv/keydb.cfg` to search paths. Fixes silent rip hang when KEYDB exists but isn't found by `resolve_keydb()`. - -## 0.10.8 (2026-04-17) - -### Buffered UDF reads -- **BufferedSectorReader** — prefetches batch sectors on single-sector reads. USB drives have ~500ms per SCSI command; this eliminates scan hangs. -- **Metadata partition pre-read** — loads entire UDF metadata partition into memory after initial parse. -- Scan time reduced from 10+ minutes to ~18 seconds on USB. - -## 0.10.7 (2026-04-17) - -### DiscStream::new() -- Replaced open_drive(), open_iso(), from_reader() with single new() constructor -- Stream accepts ContentFormat and sets up demuxer internally -- Removed disc:// case from input() — callers use primitives directly - -## 0.10.6 (2026-04-16) - -### Docker compatibility -- **Drive discovery** — removed sysfs check that blocked detection inside Docker containers. Device nodes are sufficient; INQUIRY command validates the device is an optical drive. - -## 0.10.5 (2026-04-16) - -### Audio parser buffering -- **AC3** — buffer across PES boundaries with frame size from fscod/frmsizecod table. Eliminates all AC3 decode errors on BD and UHD. -- **DTS** — buffer with core sync detection + frame size from header. DTS-HD extension frames handled correctly. -- **TrueHD** — buffer with unit length field parsing. Incomplete units held for next PES. -- All audio parsers now emit complete frames only. When PES boundaries align (normal case), buffering is a no-op. - -## 0.10.4 (2026-04-16) - -### CSS decryption — full key hierarchy -- **Bus auth → disc key → title key** — complete CSS key chain. Bus authentication with CSSCryptKey challenge-response, disc key decryption using 31 player keys via READ DVD STRUCTURE, title key extraction via REPORT KEY format 0x04. -- **CSS descramble cipher** — correct LFSR keystream generation with TAB5 for LFSR1 output and TAB4 for LFSR0 output. Per-sector key derivation from title key XOR sector seed. -- **Stevenson plaintext attack** — expanded pattern set (padding, video, audio, nav pack headers), scans up to 50K scrambled sectors for ISO key recovery. -- **Disc::copy() CSS decrypt** — sector-level decryption during disc→ISO copy produces clean ISOs with zero scramble flags. - -### MPEG-2 PS demuxer fixes -- **DVD PS path routes through codec parsers** — was bypassing parser.parse(), producing raw PES frames without codec_private extraction or keyframe detection. -- **MPEG-2 sequence header extraction** — calculates exact header size including quantizer matrices (intra/non-intra flags), captures sequence extension from subsequent PES packets. -- **TsDemuxer dynamic PID table** — Vec instead of fixed [i16; 8192] for DVD PIDs that may exceed 8192. - -## 0.10.3 (2026-04-16) - -### DVD CSS authentication -- **CSS drive authentication** — full SCSI REPORT KEY / SEND KEY handshake with 6-round substitution-permutation cipher (CSSCryptKey). Brute-forces variant from 32 possibilities. Drive serves scrambled sectors after auth completes. -- **CSS auth runs before scan** — chicken-and-egg fix: auth must happen before reading VOB sectors for title key cracking, not after. -- **Remove debug output** — strip temporary eprintln from drive reads and CSS auth. - -## 0.10.2 (2026-04-15) - -### Fixes -- **Disc::copy() batch overflow** — hardcoded 64-sector batch exceeded BU40N's 60-sector hardware limit, causing every read to fail and trigger 5×30s recovery sleep. Now accepts detected batch size from caller, defaults to 60. -- **IFO PGC parsing** — playback time read from offset 0x04 (correct) instead of 0x02 (nr_programs). Cell BCD time at cell+4 not cell+0. DVD durations now correct. -- **Demuxer flush at EOF** — TS and PS demuxers flushed when source reaches EOF, preventing loss of last PES frame. Applied to DiscStream and M2tsStream. -- **DiscStream demuxer selection** — demuxer set by caller based on content_format (TS for Blu-ray, PS for DVD) instead of unconditionally creating TsDemuxer in from_reader() -- **StdioStream FMKV header** — writes/reads metadata header for roundtrip compatibility through stdio pipes - -## 0.10.1 (2026-04-15) - -### Architecture: streams are PES, disc.copy() for sector dumps -- **One stream per format, bidirectional PES** — MkvStream, M2tsStream, NetworkStream, StdioStream, NullStream each handle read and write -- **IsoStream merged into DiscStream** — one type for physical drives and ISO files, different SectorReader -- **Disc::copy()** — raw sector dump for disc→ISO, not a stream operation -- **IOStream deleted** — no more byte-level Read/Write on streams -- **ContentReader/OpenDisc deleted** — replaced by DiscStream + PES pipeline -- **CountingStream** — wrapper for progress tracking, no state in streams - -### Error codes only — zero English in library -- All `io::Error::new(kind, "english")` replaced with `Error` enum variants -- New error variants: StreamReadOnly, StreamWriteOnly, StreamUrlInvalid, MkvInvalid, NoStreams, etc. -- `From for io::Error` — clean conversion at system boundaries -- Removed unused error variants: WriteError, ProfileNotFound, NotUnlocked, NotCalibrated, ScsiTimeout, etc. - -### Deleted dead code -- `mkvout.rs`, `pesout.rs`, `isowriter.rs` — merged into parent stream types -- `lookahead.rs` usage in MkvStream — replaced by PES direct write -- ContentReader, OpenDisc, open_title() — replaced by PES pipeline -- `open_input()`, `open_output()` — replaced by `input()`, `output()` - -## 0.10.0 (2026-04-15) - -### PES pipeline -- **Unified Stream trait** — `read()` returns PES frames, `write()` accepts them. One trait for all streams. -- **All streams produce/consume PES frames** — DiscStream, IsoStream, MkvStream, M2tsStream, NetworkStream, StdioStream, NullStream -- **DVD PS demux** — MPEG-2 Program Stream demuxer produces PES frames -- **MKV input stream** — MKV demux produces PES frames -- **Network/stdio PES** — PES serialization over TCP and pipes -- **FileSectorReader** — ISO files implement SectorReader for unified disc/ISO handling - -### PES pipeline audit (20 fixes) -- PES serialize: track/length validation, OOM cap (256 MB), stuffing compliance -- TsDemuxer: AF length validation, find_start_code verified -- PTS: marker bit validation, ns→90kHz saturating_mul, round-to-nearest -- AC3/DTS: debug_assert promoted to runtime check -- MKV: block_vint 3-4 byte support, track bounds check -- FMKV: JSON 10 MB cap, PAT section_len underflow guard - -### codec_privates refactor -- **codec_privates on DiscTitle** — no separate parameter passing, no `_with_X` method variants -- **Streams-not-files** — MkvStream and M2tsStream take `impl Read`, not `File`/`Seek` -- **M2TS roundtrip fix** — TsMuxer Annex B conversion + codec_private in FMKV header -- **MKV remux fix** — MkvStream returns codec_privates from EBML header -- **Network codec_private fix** — FMKV header carries base64 codec_privates - -### Cleanup -- Remove Seek/File dependencies from stream interfaces -- Remove eprintln from library code -- Fix all clippy warnings -- 342 tests pass - -## 0.9.0 (2026-04-14) - -### Drive recovery + decrypt architecture -- **Drive::read()** — single read method with built-in error recovery (min speed → reset → retry) -- **Decrypt in streams** — streams handle their own decryption via `decrypt_sectors()`. Pipeline just moves bytes. -- **keys() on IOStream** — streams report their own decrypt keys -- **InputOptions** — `--raw` wired through to streams, skips decrypt only -- **decrypt_sectors returns Result** — fail instead of silent corruption -- **Handshake fix** — no longer returns fake success on failure -- **Drive::read_capacity()** — for raw sector dump (disc→ISO) -- **Reset on open** — SgIoTransport resets device on every open -- **Simplified DiscStream** — removed on_error/on_success/Recovery enum - -### Platform -- **Rust 1.86 MSRV** pinned in Cargo.toml and CI -- **macOS build fix** — MacScsiTransport marked Send -- **is_multiple_of** — replaced nightly API with stable equivalent - -### API changes -- **Drive object** — typed DriveSession API -- **Typed StreamUrl** — URL parsing returns enum, not strings -- **DriveStatus API** — reset(), wait_ready with fallback -- **Granular SCSI queries** — individual methods on DriveSession for capture -- **Profile module public** — for external tools (bdemu) -- **Tray lock/unlock** — exposed on Drive - -## 0.8.0 (2026-04-11) - -### DVD support -- **Full DVD pipeline** — VIDEO_TS detection, IFO parsing, CSS decryption, MPEG-2 PS demuxing -- **CSS cipher** — Stevenson 1999 table-driven implementation, no keys needed -- **IFO parser** — title sets, PGC chains, cell addresses, audio/subtitle attributes, palette -- **MPEG-2 PS demuxer** — pack headers, PES extraction, private stream 1 sub-streams -- **MPEG-2 video parser** — sequence headers, I-frame detection, codec_private - -### 100% codec coverage -- **E-AC-3 (Dolby Digital Plus)** — bsid detection, frame size calculation -- **DTS-HD MA/HR** — extension substream detection and inclusion -- **LPCM** — BD header skip, raw PCM extraction -- **DVD subtitles (VobSub)** — passthrough with IFO palette extraction (YCbCr→RGB) -- **Dolby Vision** — verified RPU NAL type 62 preserved in HEVC passthrough - -### MKV improvements -- **Chapters** — MPLS PlayList marks → MKV Chapters element -- **Track flags** — FlagDefault, FlagForced, Language correctly set -- **HEVC codec_private** — profile compatibility and constraint flags from SPS -- **VC-1 codec_private** — resolution parsed from sequence header - -### Architecture -- **SectorReader trait** — decouples disc scanning from SCSI -- **Disc::scan_image()** — scan ISO images or any SectorReader -- **resolve_encryption()** — single function handles AACS 1.0/2.0/CSS/none -- **Module refactors** — disc/ (4 files), aacs/ (5 files), drive/ (3 files) -- **Module visibility** — internal modules pub(crate), explicit AACS re-exports - -### Streams -- **StdioStream** — stdin/stdout pipe -- **IsoStream** — read/write Blu-ray ISO images with UDF 2.50 filesystem -- **Strict URLs** — all URLs require scheme:// prefix, bare paths rejected -- **total_bytes()** — IOStream reports content size for progress display - -### Platform -- **Windows SPTI** — SCSI Pass-Through Interface backend -- **Windows builds** — CI + release workflow for x86_64-pc-windows-msvc -- **macOS drive discovery** — separate from Linux (drive/macos.rs) -- **Stable download URLs** — /latest/download/ with version-free filenames - -### Audit fixes (4 rounds, 14→0 critical) -- UDF bounds checking on all disc-sourced offsets -- SCSI: Linux residual underflow, macOS task_status type, Windows buffer zeroing -- AACS: EC mod_inv safe, key reduced mod n, host cert fallback -- DiscStream: persistent read state (was recreating ContentReader per call) -- ISO writer: UDF tag checksums, multi-extent >4GB, reserve AVDP placement -- CSS crack: labeled loop break, polynomial match -- 0 clippy warnings - -### Testing -- **327 tests** (was 64 at start) -- CSS/AACS cross-validation against independent AES implementation -- End-to-end MKV mux test with H.264 codec headers - -## 0.7.2 (2026-04-11) - -### Windows support - -- **SPTI backend** (`scsi/windows.rs`) — SCSI_PASS_THROUGH_DIRECT via DeviceIoControl -- **Windows drive discovery** (`drive/windows.rs`) — scans CdRom0-15 + drive letters -- **Platform file separation** — `drive/unix.rs` and `drive/windows.rs`, no inline cfg branches -- **CI** — `cargo check` on windows-latest, actions/checkout@v5 - -### Test suite - -- **177 tests** (was 64) — MPLS, CLPI, H.264, HEVC, AC3, VC1, DTS, TrueHd, PGS, EBML, UDF, disc scanning, streams -- **FEATURES.md** created - -### Improvements - -- **Stable download URLs** — `/latest/download/freemkv-x86_64-unknown-linux-musl.tar.gz` works forever - -## 0.7.1 (2026-04-11) - -### SectorReader trait - -- **`SectorReader` trait** — decouples disc scanning from SCSI. UDF, MPLS, CLPI, labels, and AACS resolution now work with any sector source. -- **`Disc::scan_image()`** — scan ISO images or any SectorReader. Full title/stream/label/AACS pipeline, no drive required. -- **`resolve_encryption()`** — single function handles AACS 1.0, 2.0, or none. Uses whatever path works (KEYDB VUK, handshake, media key, device key). - -### Stream types - -- **7 stream types** — Disc, ISO, MKV, M2TS, Network, Stdio, Null -- **`IsoStream`** — read/write Blu-ray ISO images. Uses `Disc::scan_image()` for full UDF parsing (not heuristic scanning). -- **`StdioStream`** — stdin/stdout pipe, format-agnostic -- **Strict URL format** — all URLs require `scheme://path`. Bare paths rejected with clear error messages. -- **Validation** — empty paths, missing ports, read-only/write-only direction errors - -### IOStream trait - -- `IOStream` trait for all stream types (Read + Write + info + finish) -- `open_input()` / `open_output()` resolve URL strings to stream instances - -## 0.7.0 (2026-04-11) - -### Stream I/O architecture - -- **5 stream types** — Disc, MKV, M2TS, Network, Null -- **`IOStream` trait** — common interface for all streams -- **URL resolver** — `open_input()` / `open_output()` with scheme://path format -- **FMKV metadata header** — JSON metadata embedded in M2TS and network streams -- **Bidirectional MKV** — MkvStream reads and writes Matroska containers -- **Network streaming** — TCP with metadata header, TCP_NODELAY -- **BD-TS demuxer** — PAT/PMT scanning, PTS duration detection -- **EBML reader** — parse existing MKV files for read-side MkvStream - -## 0.6.0 (2026-04-10) - -### API improvements - -- **`open()` works on all drives** — no profile match required. Unknown drives can scan, read BD/DVD at OEM speed. `init()` is optional and adds features (riplock removal, UHD reads, speed control). -- **`has_profile()`** — check if unlock parameters are available for this drive -- **`find_drives()`** — returns all optical drives, not just profile-matched ones -- **`raw_gc_010c`** on `DriveId` — raw GET_CONFIG 010C response bytes for profile sharing - -### AACS 2.0 - -- **SCSI handshake wired end-to-end** — ECDH key agreement, real Volume ID from drive, read data key for bus decryption -- **Bus decryption active** — UHD discs with bus encryption now decrypted transparently -- **VUK derivation from Media Key + VID** — works for discs not in KEYDB (processing key + device key paths) - -### MKV muxer - -- **15 new files** — EBML writer, TS demuxer, stream assembly pipeline -- **Codec parsers** — H.264, HEVC, AC-3, DTS, TrueHD, PGS, VC-1 -- **`MkvStream`** — builder pattern, wraps any `impl Write`, configurable lookahead buffer - -### Cleanup - -- Removed orphaned `jar.rs` (342 lines) — replaced by `labels/` module -- Error refactor: 40+ sites converted from English strings to typed error codes - -## 0.5.0 (2026-04-09) - -### Read pipeline — 5x speed improvement - -- **Kernel transfer limit detection**: auto-detect `max_hw_sectors_kb` via sysfs, resolve sg→block device. Previously hardcoded to 510 sectors (1MB) which exceeded the 120KB kernel limit, causing all reads to error and fall back to 6KB reads at 4.8 MB/s. Now auto-tunes to 48 sectors (96KB) or whatever the device supports. -- **Result: 12.5 MB/s sustained, 23 MB/s peak** (was 4.8 MB/s) - -### Drive init pipeline - -- **Full drive init pipeline**: unlock, calibrate (256 zones), register reads, status, probe, set_read_speed, keepalive, timing -- **Speed calibration**: 256+ disc surface probes, 64-entry speed table, triple SET_CD_SPEED -- **Platform trait locked down**: `pub(crate)`, 3 methods only (init, set_read_speed, is_ready) -- **Init guard**: prevents double-init, signature mismatch aborts early - -### MPLS parser fixes - -- **PGS in audio slots**: subtitle language read at correct offset (was truncated: "ng " → "eng") -- **Secondary PG entries**: n_pip_pg loop added for correct STN position tracking -- **Secondary stream types**: stream_type 5 (sec audio), 6 (sec video), 7 (DV EL) attribute parsing -- **Empty stream filter**: coding_type 0x00 entries (padding) no longer appear as "Unknown(0)" - -### Profiles - -- **206 profiles with full per-drive data**: ld_microcode (base64), all CDBs, speed tables, signatures -- **Automated pipeline**: profile import → profiles.json (no manual merging) - -## 0.4.0 (2026-04-07) - -### Labels — complete rewrite - -- **Detect-then-parse architecture**: each BD-J authoring format has its own parser module with `detect()` and `parse()` functions. Drop in a new parser with one line in the registry. -- **5 format parsers**: Paramount (`playlists.xml`), Criterion (`streamproperties.xml`), Pixelogic (`bluray_project.bin`), Warner CTRM (`menu_base.prop` / `language_streams.txt`), shared label vocabulary (`vocab.rs`) -- **Raw disc data principle**: label data passes through as-is from disc. Only BD-standard codec identifiers (MLP, AC3, DTS) are mapped to display names. Unknown authoring tool codes (csp, eda, cf) pass through raw. -- **`variant` field**: replaces `region` — language dialect codes from authoring tools, not BD spec regions -- Removed: old `jar` module (superseded by labels) -- Removed: dead label apply functions from disc.rs - -### Drive - -- **`DriveSession::eject()`**: sends PREVENT ALLOW MEDIUM REMOVAL then START STOP UNIT. Works reliably after raw mode unlock. -- **`DiscRegion` enum**: Free, BluRay(A/B/C), Dvd(1-8). UHD always region-free. - -### Capture - -- **Fixed sector range collection**: captures ALL files on disc (only skips STREAM/ video files and >50MB). Previously skipped BACKUP/, DUPLICATE/, and files >10MB which missed JAR content. - -## 0.3.1 - -- Labels module: 4 disc file parsers for stream labels -- Simplified labels API - -## 0.3.0 - -- Initial public release -- SCSI transport (Linux SG_IO, macOS IOKit) -- UDF 2.50 filesystem reader -- MPLS/CLPI parsers with full STN support -- Drive identification + profile matching -- 206 bundled drive profiles -- AACS 1.0 decryption (VUK + unit keys) +- Content keys (CSS disc/title keys, AACS unit/volume keys) are redacted in log + output (logged as `` with a 1-byte fingerprint); a test guards + against any key field being logged with a raw value. +- The macOS SCSI shim uses `posix_spawn` directly instead of `system()` / `sh + -c`, eliminating a command-injection vector on the device-path string. + +## [1.0.0-rc.1] + +First release candidate for 1.0 — the first tagged 1.0 milestone of the core +library. Established the full feature set: multipass sector recovery, content +decryption (CSS, AACS 1.0/2.0) from `keydb.cfg`, disc parsing, and the threaded +mux pipeline (see "Pre-1.0 development" for the consolidated feature list). + +## Pre-1.0 development + +Versions 0.x were the iterative development series leading up to 1.0. The +highlights, condensed: + +- **Multipass recovery engine.** Pass 1 sweeps the whole disc sequentially, + tolerating bad sectors with an adaptive damage-jump algorithm (mark the bad + range, keep going). Pass N retries the bad ranges with per-sector recovery + timeouts, reverse-direction reads, and range bisection. A mapfile tracks + per-sector state across passes so a rip can resume. +- **Drive and SCSI layer.** Single-shot, synchronous SG_IO transport on Linux + (with IOKit on macOS and SPTI on Windows), full SCSI sense decoding, and + drive enumeration / presence probes. Single-shot reads by design — recovery + lives in the multipass orchestration, not inline in the read path. +- **Content decryption.** CSS for DVDs and AACS 1.0/2.0 for Blu-ray and UHD, + with keys read from `keydb.cfg`. A single decrypting decorator wraps the + sector source so decryption is one audited surface, and a resolved key is + verified against disc content before it is applied. +- **Disc parsing.** UDF, MPLS/CLPI (Blu-ray), and IFO (DVD) parsing for title + and extent assembly, with bounds checks on values derived from untrusted disc + input. Canonical main-title selection picks the real feature over a + play-all virtual playlist on branching discs. +- **Mux pipeline (the "highway").** A three-stage threaded pipeline — + read+decrypt, demux, codec parse — with a recycled buffer pool, taking + file-backed mux from ~60 MB/s to several hundred MB/s warm-cache. Codec + parsers for HEVC, H.264, VC-1, MPEG-2, TrueHD, DTS(-HD), and PGS feed an + EBML/Matroska writer. +- **I/O stack.** Bounded-cache writeback (`sync_file_range` + + `posix_fadvise(DONTNEED)`) keeps the kernel dirty-page cache bounded on long + sequential writes, and time-batched mapfile persistence keeps NFS-staged rips + fast. +- **Library hygiene.** No user-facing English in the library — all errors are + numeric codes handled by the application layer. A large spec-grounded, + mutation-verified test suite guards the silent-corruption surfaces. Rust 2024 + edition; release builds use thin LTO. diff --git a/README.md b/README.md index 6395307..b2ace11 100644 --- a/README.md +++ b/README.md @@ -125,9 +125,9 @@ Streams implement a single unified `pes::Stream` trait (re-exported as `PesStrea ### Keys -DVDs (CSS) decrypt out of the box. The library uses 31 compiled-in 1999-era public player keys (for disc-key cracking) combined with Frank Stevenson's known-plaintext title-key attack — no external key file needed. +DVDs (CSS) decrypt out of the box, with no external key file needed. -Blu-rays and UHD (AACS) require a `keydb.cfg` at `~/.config/freemkv/keydb.cfg` (or passed via `ScanOptions`). The file holds all DKs, PKs, host certs, and per-disc VUKs. No AACS key material is compiled into the binary. +Blu-rays and UHD (AACS) require a `keydb.cfg` at `~/.config/freemkv/keydb.cfg` (or passed via `ScanOptions`). No AACS key material is compiled into the binary. ## Architecture diff --git a/docs/aacs.md b/docs/aacs.md index 3f27f9d..daba46c 100644 --- a/docs/aacs.md +++ b/docs/aacs.md @@ -2,191 +2,45 @@ ## Overview -AACS (Advanced Access Content System) is the encryption layer used by Blu-ray and UHD 4K discs to protect content. libfreemkv implements AACS decryption to enable transparent disc access. +AACS (Advanced Access Content System) is the encryption layer used by Blu-ray +and UHD 4K discs to protect content. libfreemkv implements AACS decryption so +disc access is transparent to the application. There are two major versions: -- **AACS 1.0** -- Used by standard Blu-ray discs. Relies on a custom 160-bit elliptic curve for bus authentication and AES-128 for content encryption. Processing keys and device keys can derive the media key from the disc's Media Key Block (MKB). - -- **AACS 2.0** -- Used by UHD 4K Blu-ray discs. Adds a per-sector bus encryption layer (read_data_key) on top of the standard content encryption. Uses P-256/SHA-256 for its native handshake, though drives accept AACS 1.0 host certificates for backward compatibility. - -Both versions use AES-128-CBC for content decryption with a fixed initialization vector. The fundamental key hierarchy is the same: a Volume Unique Key (VUK) decrypts per-title unit keys, which in turn decrypt the content stream. - - -## Architecture - -AACS support is split across two modules: - -### `aacs.rs` -- Keys and Decryption - -Handles everything related to key resolution and content decryption: - -- KEYDB.cfg parsing (device keys, processing keys, host certificates, per-disc entries) -- Disc hash computation (SHA-1 of `Unit_Key_RO.inf`) -- VUK resolution chain (4 paths, described below) -- MKB record parsing and media key derivation -- Subset-difference tree traversal (AACS-G3 key derivation) -- Unit_Key_RO.inf parsing and unit key decryption -- Content Certificate parsing (AACS version detection) -- Aligned unit decryption (AES-128-CBC) -- Bus decryption (AACS 2.0 read_data_key layer) - -### `aacs_handshake.rs` -- SCSI Authentication - -Handles the drive-level SCSI authentication protocol: - -- ECDH key agreement on the AACS 160-bit curve -- ECDSA signing and verification -- Bus key derivation -- AGID management (allocate/invalidate) -- Volume ID retrieval (encrypted with bus key, verified by AES-CMAC) -- Read Data Key retrieval (for AACS 2.0 bus decryption) -- AACS LA public key certificate verification - - -## Key Resolution Chain - -When a disc is scanned, `resolve_keys()` attempts four paths in priority order. The first path that succeeds is used. - -### Path 1: KEYDB VUK Lookup (fastest) - -``` -Unit_Key_RO.inf --> SHA-1 --> disc_hash --> KEYDB lookup --> VUK -``` - -The disc hash is computed as the SHA-1 digest of the raw `Unit_Key_RO.inf` file from the disc's `/AACS/` directory. This hash is used as the lookup key in `KEYDB.cfg`. If a matching entry contains a VUK (`V` field), it is used directly. - -This is the fast path and resolves the vast majority of discs in a well-maintained KEYDB. - -### Path 2: KEYDB Media Key + Volume ID - -``` -KEYDB media_key + Volume ID (from SCSI handshake) --> VUK derivation -``` - -If the disc hash is not in the KEYDB but a KEYDB entry has a matching Volume ID (`I` field) and a media key (`M` field), the VUK is derived: - -``` -VUK = AES-128-ECB-DECRYPT(media_key, volume_id) XOR volume_id -``` - -Requires a successful SCSI handshake to obtain the Volume ID. - -### Path 3: MKB + Processing Keys - -``` -MKB (from disc) + processing_keys (from KEYDB) --> media_key --> VUK -``` - -Processing keys are pre-computed keys that work against specific MKB versions. For each processing key, the library: - -1. Parses the MKB to extract the Verify Media Key Record (`mk_dv`), subset-difference index, and conditional values (cvalues). -2. Tries each processing key against each UV/cvalue pair: `mk = AES-DEC(pk, cvalue) XOR cvalue`. -3. Validates the derived media key: `AES-ECB(mk, mk_dv)` must produce 12 leading zero bytes. -4. Derives VUK from the validated media key and Volume ID. - -### Path 4: MKB + Device Keys (Subset-Difference Tree) - -``` -MKB + device_keys --> subset-difference tree traversal --> processing_key --> media_key --> VUK -``` - -The most complex path. Each device key has an associated node number, UV value, and mask parameters that position it in the AACS subset-difference tree. The library: - -1. Finds the subset-difference entry in the MKB that applies to the device key's node. -2. Traverses the tree using AACS-G3 key derivation: `aesg3(key, inc) = AES-DEC(key, seed) XOR seed`, where `seed[15]` is incremented by `inc`. Each tree node produces a left child (inc=0), a processing key (inc=1), and a right child (inc=2). -3. At each level, selects left or right based on the UV bit at the current position. -4. The resulting processing key is validated against the MKB cvalue to derive the media key. -5. VUK is derived from the media key and Volume ID. - - -## Content Decryption - -### Aligned Units - -AACS encrypts content in aligned units of 6144 bytes (3 sectors of 2048 bytes each). The encryption flag is signaled by the copy_permission_indicator bits in byte 0 of the unit (`unit[0] & 0xC0 != 0`). - -### Per-Unit Key Derivation - -Each aligned unit has its own decryption key derived from the CPS unit key: - -1. **Derive**: AES-128-ECB encrypt the first 16 bytes of the unit (plaintext TP_extra_header) with the unit key. -2. **XOR**: XOR the encrypted result with the original 16 bytes to produce the per-unit decryption key. -3. **Decrypt**: AES-128-CBC decrypt bytes 16 through 6143 using the per-unit key and the fixed AACS IV. -4. **Clear flag**: Clear the encryption indicator bits (`unit[0] &= !0xC0`). - -### Fixed IV - -All AES-CBC operations in AACS use the same fixed initialization vector, defined in the AACS specification. - -### Verification - -After decryption, the library verifies correctness by checking for MPEG-TS sync bytes (0x47) at the expected 192-byte packet boundaries within the unit. Blu-ray transport stream packets are 192 bytes: 4-byte TP_extra_header followed by a 188-byte TS packet. - - -## Bus Encryption - -### AACS 1.0 - -Standard Blu-ray discs do not use bus encryption. Content is read directly from the disc and decrypted using the unit key. - -### AACS 2.0 - -UHD 4K discs add a per-sector bus encryption layer. The drive encrypts data as it is read from the disc, and the host must decrypt it before applying AACS content decryption. - -Bus encryption uses a **read_data_key** obtained during the SCSI handshake. For each 2048-byte sector within an aligned unit, bytes 16 through 2047 are AES-128-CBC encrypted with the read_data_key and the fixed AACS IV. The first 16 bytes of each sector remain plaintext. - -The full decryption pipeline for AACS 2.0: - -1. **Bus decrypt**: For each sector, AES-128-CBC decrypt bytes 16..2047 with the read_data_key. -2. **Content decrypt**: Standard per-unit key derivation and AES-128-CBC decryption as described above. - - -## SCSI Handshake - -The AACS SCSI authentication handshake establishes a shared bus key between host and drive, then uses it to securely transfer the Volume ID and read data keys. - -### Protocol Flow - -1. **Invalidate AGIDs**: Send REPORT KEY with format 0x3F for AGIDs 0-3 to clear stale sessions. -2. **Allocate AGID**: REPORT KEY format 0x00 returns a fresh Authentication Grant ID. -3. **Send host credentials**: SEND KEY format 0x01 transmits the host nonce (20 random bytes) and host certificate (92 bytes). -4. **Receive drive credentials**: REPORT KEY format 0x01 returns the drive nonce and drive certificate. -5. **Receive drive key**: REPORT KEY format 0x02 returns the drive's ephemeral EC key point and ECDSA signature over `host_nonce || drive_key_point`. -6. **Verify drive key**: The signature is verified against the drive's public key (extracted from its certificate). AACS 1.0 certificates are verified against the AACS LA public key. -7. **Send host key**: The host generates an ephemeral key pair, signs `drive_nonce || host_key_point` with the host private key, and sends via SEND KEY format 0x02. -8. **Compute bus key**: ECDH shared secret = `host_private_key * drive_key_point`. The bus key is the low 128 bits of the shared point's x-coordinate. - -### Post-Authentication Reads - -- **Volume ID**: REPORT DISC STRUCTURE format 0x80. Returns 16-byte VID encrypted with the bus key, plus an AES-CMAC MAC for integrity verification. -- **Read Data Keys**: REPORT DISC STRUCTURE format 0x84. Returns the read_data_key and write_data_key, each AES-ECB encrypted with the bus key. - -### Elliptic Curve - -AACS 1.0 uses a custom 160-bit Weierstrass curve (`y^2 = x^3 + ax + b mod p`) with 20-byte field elements. The library implements full EC arithmetic: point addition, doubling, scalar multiplication, modular inverse, ECDSA sign/verify, and ECDH key agreement. - - -## AACS 2.0 Status - -AACS 2.0 discs are detected via the Content Certificate file (`Content000.cer` or `Content001.cer`). A certificate type byte of 0x01 indicates AACS 2.0. - -AACS 2.0 drives are identified by their drive certificate type (0x11). These drives natively use P-256/SHA-256, but accept AACS 1.0 host certificates for backward compatibility. - -Current implementation status: - -- AACS 2.0 detection: **implemented** (Content Certificate parsing, drive cert type check) -- AACS 1.0 handshake with AACS 2.0 drives: **implemented** (backward compatibility mode) -- Full P-256 AACS 2.0 handshake: **not yet implemented** (prepared but rarely needed since drives accept AACS 1.0 host certs) -- Bus decryption with read_data_key: **implemented** -- Content decryption: **implemented** (same as AACS 1.0) - -In practice, AACS 2.0 UHD discs work through the backward-compatible AACS 1.0 handshake path, with the addition of read_data_key bus decryption. - +- **AACS 1.0** -- Used by standard Blu-ray discs. +- **AACS 2.0 / 2.1** -- Used by UHD 4K Blu-ray discs. Adds a per-sector bus + encryption layer on top of the standard content encryption. UHD drives accept + AACS 1.0 host credentials for backward compatibility. + +All versions use AES-128 for content decryption. The library reads the keys it +needs from `keydb.cfg`, walks the disc's Media Key Block (MKB) to resolve the +disc's key, and decrypts the content stream. AACS-encrypted discs therefore +require a `keydb.cfg`; CSS-protected DVDs do not (see the CSS notes in the +library docs). + +## How it works (feature level) + +When a disc is scanned, the library: + +1. Reads the disc's AACS key-input files from the `/AACS/` directory. +2. Resolves the disc's key from `keydb.cfg` — either directly from a per-disc + entry, or by walking the MKB with the keys present in the keydb. +3. Performs the drive-level SCSI authentication handshake needed to obtain the + Volume ID and, for UHD, the bus-decryption key. +4. Decrypts the content stream as titles are read. + +A resolved key is verified against actual disc content before it is applied, so +a stale or wrong key fails loudly rather than producing silent garbage. If no +usable key is available for an AACS-encrypted disc, the library surfaces a +specific error (the E70xx family) describing which part of the chain was +missing, and a missing `keydb.cfg` surfaces as `Error::KeydbLoad` with the +sentinel path ``. ## API Usage -AACS decryption is transparent to the application. The `Disc::scan()` method handles everything automatically: +AACS decryption is transparent to the application. `Disc::scan()` handles +everything automatically: ```rust use libfreemkv::{Drive, Disc}; @@ -203,7 +57,6 @@ if disc.encrypted { if let Some(ref aacs) = disc.aacs { println!("AACS {}.0", aacs.version); println!("Key source: {}", aacs.key_source.name()); - println!("Disc hash: {}", aacs.disc_hash); if let Some(mkb_ver) = aacs.mkb_version { println!("MKB version: {}", mkb_ver); } @@ -215,108 +68,41 @@ if disc.encrypted { // Read content -- decryption is automatic let mut reader = disc.open_title(&mut session, 0).unwrap(); while let Some(unit) = reader.read_unit().unwrap() { - // unit is 6144 bytes of decrypted content + // decrypted content } ``` -The application never touches keys, never calls decryption functions, and never manages handshakes. All of that is internal to `Disc::scan()` and `ContentReader::read_unit()`. +The application never touches keys, never calls decryption functions, and never +manages handshakes. All of that is internal to `Disc::scan()` and the content +reader. ### KEYDB Location -`ScanOptions` controls where the KEYDB is loaded from. If no explicit path is set, the library checks: - -1. `~/.config/aacs/KEYDB.cfg` -2. `/etc/aacs/KEYDB.cfg` - -To specify an explicit path: +`ScanOptions` controls where the keydb is loaded from. If no explicit path is +set, the library checks the standard config locations. To specify an explicit +path: ```rust -let opts = ScanOptions::with_keydb("/path/to/KEYDB.cfg"); +let opts = ScanOptions::with_keydb("/path/to/keydb.cfg"); let disc = Disc::scan(&mut session, &opts).unwrap(); ``` ### AacsState -After a successful scan, `disc.aacs` contains an `AacsState` with: +After a successful scan, `disc.aacs` contains an `AacsState`: | Field | Type | Description | |-------|------|-------------| | `version` | `u8` | AACS version (1 or 2) | | `bus_encryption` | `bool` | Whether bus encryption is active | | `mkb_version` | `Option` | MKB version from disc | -| `disc_hash` | `String` | SHA-1 of Unit_Key_RO.inf (hex with 0x prefix) | -| `key_source` | `KeySource` | How keys were resolved | -| `vuk` | `[u8; 16]` | Volume Unique Key | -| `unit_keys` | `Vec<(u32, [u8; 16])>` | Decrypted unit keys (CPS unit number, key) | -| `read_data_key` | `Option<[u8; 16]>` | AACS 2.0 bus decryption key | -| `volume_id` | `[u8; 16]` | Volume ID from SCSI handshake | +| `disc_hash` | `String` | Identifier for the disc's key-input files | +| `key_source` | `KeySource` | How the disc's key was resolved | -### KeySource +## keydb.cfg -| Variant | Description | -|---------|-------------| -| `KeyDb` | VUK found directly in KEYDB by disc hash | -| `KeyDbDerived` | Media key + Volume ID from KEYDB, VUK derived | -| `ProcessingKey` | MKB + processing keys from KEYDB | -| `DeviceKey` | MKB + device keys, subset-difference tree traversal | - - -## KEYDB.cfg Format Reference - -The KEYDB.cfg file contains all cryptographic material needed for AACS decryption. Lines starting with `;` or `#` are comments. - -### Device Keys - -``` -| DK | DEVICE_KEY 0x | DEVICE_NODE 0x | KEY_UV 0x | KEY_U_MASK_SHIFT 0x -``` - -- `key`: 16-byte AES device key (hex) -- `node`: Device node number in the subset-difference tree (hex) -- `uv`: UV value for tree positioning (hex) -- `shift`: U mask shift value (hex) - -### Processing Keys - -``` -| PK | 0x -``` - -- `key`: 16-byte pre-computed processing key (hex) - -### Host Certificate - -``` -| HC | HOST_PRIV_KEY 0x | HOST_CERT 0x -``` - -- `privkey`: 20-byte ECDSA private key (hex) -- `cert`: 92-byte AACS host certificate (hex) - -The host certificate is used for SCSI authentication. It contains the host's public key and is signed by the AACS Licensing Administrator. - -### Disc Entries - -``` -0x = | D | <date> | M | 0x<media_key> | I | 0x<disc_id> | V | 0x<vuk> | U | <unit_keys> -``` - -- `disc_hash`: 20-byte SHA-1 of Unit_Key_RO.inf (hex) -- `title`: Human-readable disc title -- `D`: Date tag, followed by release/rip date -- `M`: Media key tag, followed by 16-byte media key (hex) -- `I`: Disc ID tag, followed by 16-byte Volume ID (hex) -- `V`: VUK tag, followed by 16-byte Volume Unique Key (hex) -- `U`: Unit keys tag, followed by space-separated `<unit_num>-0x<key>` pairs - -All fields after the title are optional. A minimal entry needs only the disc hash and VUK: - -``` -0x<disc_hash> = <title> | V | 0x<vuk> -``` - -Inline comments are supported with `;`: - -``` -0x<disc_hash> = <title> | V | 0x<vuk> ; MKBv77 -``` +`keydb.cfg` is the single source of AACS key material. It is a text file (lines +starting with `;` or `#` are comments) holding the host credentials and per-disc +entries the library uses to resolve a disc. autorip can auto-download and +refresh it from a configured URL. The library does not ship any AACS keys +compiled into the binary.