Breaking: CopyOptions::on_progress + PatchOptions::on_progress now take
Fn(bytes_good, pos, total). Consumers display `pos` for "% swept" — the
true Pass 1 progress that advances through skip-forward bad zones, where
bytes_good (Finished sectors only) freezes. v0.13.14 live trace proved
the existing UI was lying for ~14 minutes about Dune 2 being "stuck at
30%" while Pass 1 was actually 83% through the disc via skip-forward.
PatchOptions::reverse: walk bad ranges from highest LBA to lowest. For
drives that wedge after a forward read of a bad sector, approaching the
post-bad-zone NonTrimmed range from end-of-disc reads good sectors before
the drive sees a bad one. Hypothesis informed by the BU40N + Initio
bridge live data — Pass 2 forward saw zero successful reads in 7 min
while Pass 1's pos walked all the way to end-of-disc.
PatchOptions::wedged_threshold: > 0 → exit early after that many
consecutive failures with zero successes in the same pass. Saves the
wallclock budget for productive grinding when the drive has wedged on
the bad zone for THIS pass; a different direction or block size in the
next pass may still recover. New PatchResult::wedged_exit reports it.
Trace: patch_start (block_sectors, recovery, reverse, wedged_threshold,
num_ranges) and patch_done (blocks_attempted, blocks_read_ok,
blocks_read_failed, wedged_exit, halted, bytes_recovered) at the
freemkv::disc target.
v0.13.12 shipped the async fd_recovery design but a live test on Dune 2
showed Pass 1 sat for 14 minutes with bytes_good=0 — the inner loop iterates
(throttled on_progress log fires every 78s) but each iteration evidently
takes ~60s instead of the microseconds the design promises on fast-fail.
Without trace-level telemetry at the SCSI + Disc::copy boundaries we
can't tell where the time goes.
This release is instrumentation only — no behavior change.
- New dep: tracing 0.1. Per project docs, debug/trace logging is allowed in
libfreemkv (the no-English rule applies to errors). Consumers wire a
tracing subscriber.
- SgIoTransport::execute (Linux): trace at every state transition (entry,
recovery_swap_ok, recovery_pending, write_ok / write_err, poll_done,
timeout_spawn_recovery, scsi_err, read_err, ok). Each event includes
opcode + elapsed timing. The bg recovery thread also traces close_ms +
open_ms so we can see if the kernel really takes 60s to close+open on a
wedged Initio bridge.
- Disc::copy: trace at copy_start, outer_loop, region_enter, every 100
inner-loop iterations (iter_progress with pos/region_end/skip_size/
bytes_good/read_ok_count/read_err_count/last_read_ms/copy_elapsed_ms),
copy_done.
- All trace events use targets `freemkv::scsi` and `freemkv::disc` so
consumers can filter by subsystem (e.g. autorip /api/debug?q=freemkv::scsi).
Next: run the live test on Dune 2 again, read the autorip JSONL log,
diagnose why each iter is slow, fix the actual bug.
Fix 1: delete stall guard from Disc::copy. Pass 1 must sweep end-to-end
per ddrescue model (RIP_DESIGN.md §2.1, §3, §9). The v0.13.9 guard at
disc/mod.rs broke Pass 1 at 30% on Dune 2 with 56 GB still NonTried.
Removed stall_secs field, narrative comment in scsi/linux.rs, and the
broken regression test. Replaced with test_disc_copy_completes_full_disc_
with_failing_reader and test_disc_copy_halts_promptly_on_failing_reader.
Fix 2: async SCSI transport recovery. Added Arc<AtomicI32> fd_recovery
on SgIoTransport. On poll timeout: spawn close + spawn open in
background, return Err immediately. Top of execute() swaps fd from
recovery atomic. Main thread never blocked beyond ~1.5s poll budget
(was up to ~60s per timeout because kernel serialized main-thread
open() against in-flight close()). Drop drains pending recovery fd.
§15.1 cross-platform parity: Windows + macOS now have the same
observable recovery contract. SptiTransport gets try_recover()
(synchronous CloseHandle + CreateFileW; Windows close is fast, no
in-flight CDB drain like Linux). MacScsiTransport gets try_recover()
(release IOKit interface + reacquire via new acquire_device_iface()
helper); stores bsd_name for re-resolution. Drop guards null'd-out
interfaces. Stripped English error strings ("try as root" / "run as
administrator") on Linux + Windows. Fixed Windows TimeOutValue
ms→s ceiling so 1500ms gets 2s (was 1s; broke Drive::read fast path).
Fix 4: instrument Disc::patch arms. PatchResult exposes
blocks_attempted, blocks_read_ok, blocks_read_failed so the v0.13.11
mystery (Dune 2 Pass 2 recovered 0 bytes in 100 min) is diagnosable
from the live device log without re-instrumenting from outside.
Cleanup: honor PatchOptions::full_recovery (was read into _ and
ignored; now routed to read_sectors recovery arg). Updated
CopyOptions::batch_sectors doc to describe the actual production
path (sysfs detect_max_batch_sectors, typically 60 sectors / ~120 KB
on BU40N) rather than the test-only 32-sector internal default.
All four crates clippy-clean and tests green on the host targets
(macOS native + cargo check on Linux). Cross-platform CI watches
Linux + Windows + macOS builds + tests.
Fixes the silent Pass 1 hang observed on Dune 2 with v0.13.8 (drive
grinding through bad sectors at 0 KB/s, errs=0, no error surfaced).
Root cause: SgIoTransport::execute's reopen-after-poll-timeout opened
a fresh /dev/sg* fd on the main thread, which serialized against the
spawned close() of the old fd via the kernel's per-device state lock.
The userspace 1.5s timeout still fired, but the abandon-and-reopen
recovery itself blocked the main thread for as long as close() took.
Net: reads returned slowly, skip-forward fired on every iteration,
bytes_good never advanced.
- SgIoTransport::execute: on poll timeout, spawn close, set fd=-1,
return Err. No reopen on the main thread. The transport is now
invalidated until the consumer creates a fresh Drive.
- Disc::copy: add stall guard. CopyOptions.stall_secs (default 120s).
If bytes_good doesn't advance for the threshold, break 'outer
cleanly with complete=false, bytes_pending > 0 so Pass 2 retries
pick up the NonTrimmed ranges with recovery=true 30s timeouts.
- New regression test: test_disc_copy_stall_detection_triggers_
skip_forward in tests/integration_progress_and_halt.rs.
Audit pass against the project docs "no English text in library code" rule.
Found 9 call sites that violated the contract by stuffing English into
io::Error::new(kind, "…") or by abusing Error::DeviceNotFound { path }
as a free-form description field. Each is now a typed Error variant.
New variants and codes: ScsiInterfaceUnavailable (E1004), DeviceLocked
(E1005), IoKitPluginFailed (E1006), UnsupportedPlatform (E2003),
PlatformNotImplemented (E2004), MapfileInvalid (E6011), DiscUrlNotDirect
(E9009).
labels::apply() previously pushed Commentary/Descriptive/Score/IME and
" (Secondary)" English literals into AudioStream.label, leaking into
MKV titles + autorip UI. AudioStream now exposes structured `purpose:
LabelPurpose`, SubtitleStream `qualifier: LabelQualifier`. Callers
translate to localized text. label keeps codec-formatting only.
API hygiene: 11 mux/* modules dropped from `pub` to `pub(crate)` —
their *types* are still re-exported from lib.rs, but the modules were
leaking low-level EBML/TS/network primitives. Stream trait gets a real
rustdoc explaining read-vs-write split. lib.rs grouped re-exports into
documented sections. ScanOptions::with_keydb() removed (one-method-per-
action rule); use struct literal.
Dead-code sweep: removed lookahead.rs (orphan, never declared as mod),
tsreader.rs (TsDemuxReader unused), ebml::{write_int,read_vint,SEEK_*},
ts::{scan_first/last_pts,scan_duration,SCAN_HEAD/TAIL_SIZE,take/set_
remainder}, MkvMuxer codec_private_slots/filled fields and
fill_codec_private method (deferred-codecPrivate path never used since
the v0.10 PES rewrite). cargo clippy --all-targets -D warnings clean.
Tests: new error::tests for variant codes + Display "no English" guard +
io::ErrorKind mapping. 233 lib tests, all green (was 230).
Breaking: ScanOptions::with_keydb removed; mux/* modules pub(crate);
AudioStream and SubtitleStream gained required fields; UnsupportedDrive
{ product_revision: "Renesas not yet implemented" } no longer produced
(use PlatformNotImplemented).
New primitives for two-stage rip workflows: fast forward pass with
zero-fill on failures, then targeted retries of bad ranges via a
ddrescue-compatible mapfile.
- Disc::copy now takes &CopyOptions (breaking change from positional
args). Always writes a sidecar .mapfile. Opt-in skip_on_error +
skip_forward give ddrescue-style fast sweep: 64 KB blocks,
exponential skip-forward on failure, zero-fill bad blocks. Defaults
preserve pre-0.11.21 behavior (recovery reads, abort on bad sector).
- Disc::patch is new and idempotent. Reads the mapfile, re-reads every
non-finished range with full drive recovery, patches good bytes back
into the ISO at exact offsets. Call N times for N retry attempts.
- disc::mapfile is a new module. ddrescue text format, crash-safe
(flushed on every record()), greppable, human-editable, tool-compatible.
Status chars match ddrescue: ? / * / / / - / +.
- Re-exports FileSectorReader from the crate root.
- freemkv CLI caller (pipe.rs) updated to the new Disc::copy signature
in lockstep — shipped in the 0.11.21 freemkv CLI release.
Part of the 0.11.21 ecosystem sync (libfreemkv + freemkv + bdemu +
autorip all on 0.11.21).
- Implement complete CSS key chain: bus auth → disc key → title key
- Add 31 player keys for disc key decryption
- Read disc key via READ DVD STRUCTURE format 0x02
- Read title key via REPORT KEY format 0x04
- Fix CryptKey round 1: use original scratch for term, not modified tmp1
- Fix decrypt_key: use TAB5 for LFSR1 output, TAB4 for LFSR0^invert
- Fix descramble_sector: use TAB5 for LFSR1, TAB4 for LFSR0 (no invert),
and apply TAB1 permutation to ciphertext before XOR
- Fix title key bus XOR: forward order (bus_key[i]), not reversed
- Two-session auth: disc key and title key need separate AGID sessions
- Fix crack_key: scan across extents for scrambled sectors
- Fix TsDemuxer: dynamic PID table size for DVD PIDs
- Set max read speed after scan for DVD riplock removal
- Disc::copy() hardcoded batch=64 sectors, exceeding BU40N's 60-sector
hw limit. Now accepts batch_sectors param, defaults to 60.
- IFO PGC: playback time at offset 0x04 not 0x02, cell time at cell+4
- DiscStream: set demuxer from content_format (TS for BD, PS for DVD)
- Flush TS/PS demuxers at EOF to avoid losing last PES frame
- M2tsStream: flush demuxer at EOF
- StdioStream: FMKV metadata header for roundtrip compatibility
Architecture:
- One stream per format, bidirectional PES (read/write on same type)
- IsoStream merged into DiscStream (one type, any SectorReader)
- Disc::copy() for disc→ISO raw sector dump
- IOStream trait deleted, all byte-level Read/Write removed
- ContentReader/OpenDisc/open_title/open_input/open_output deleted
- CountingStream wrapper for progress tracking
Error codes:
- All io::Error English strings replaced with Error enum variants
- From<Error> for io::Error conversion
- Unused variants removed, new stream/mux variants added
Deleted: mkvout.rs, pesout.rs, isowriter.rs, mkv-muxer-plan.md
Updated: all docs, README stream table, CHANGELOG
238 tests, 0 clippy warnings.
Design fix: codec_privates are now a field on DiscTitle, not a separate
parameter passed through the pipeline. This eliminates the root cause of
the network codec_private bug (forgot to pass the separate param).
API changes:
- output() takes (url, &DiscTitle) — no separate codec_privates param
- MkvOutputStream::create, M2tsOutputStream::create, NetworkOutputStream::connect
all read codec_privates from title.codec_privates
- M2tsMeta::from_title() takes only &DiscTitle — reads privates from title
- Deleted from_title_with_privates (was the wrong-name duplicate)
- Merged read_header + read_header_from_stream into one read_header(impl Read)
- Deleted finish(self) from TsMuxer, keep only finish(&mut self)
Rule: ONE public method per action. No _with_X, _from_Y, _ref variants.
Previously used all-zeros AES key when unit_key_idx was out of range,
producing silently corrupted output. Now returns DecryptFailed error.
Also wired --raw flag through InputOptions → set_raw() on streams.
- Rename DriveSession → Drive across entire codebase
- find_drives() returns Vec<Drive>, find_drive() returns Option<Drive>
- resolve_device() now pub(crate) — internal only
- StreamUrl is now a typed enum (Disc, Mkv, M2ts, Iso, Network, Stdio, Null)
with scheme() and path_str() accessors, replacing struct of Strings
- Add lock_tray() / unlock_tray() for safe disc access during rips
- Improve reset() with eject cycle that clears LibreDrive stuck state
- Add Send bounds to ScsiTransport and PlatformDriver traits
- DiscOptions uses PathBuf instead of String for device/keydb paths
- Update doc example to use new Drive API
- wait_ready: detects MMkv vendor probe when TUR returns Illegal Request
- scan: capacity fallback to 0 when READ CAPACITY fails
- Still needs: re-init to restore standard SCSI commands, or use raw reads for UDF
- Doc comments on DriveSession, find_drives, all Error variants, Result type
- 24 format! strings inlined (clippy pedantic)
- 25 long hex literals with separators (0xFFFFFFFF → 0xFFFF_FFFF)
- README install example updated to 0.8
MattJackson