The CLI --version, the MKV muxing/writing-application field, and the FVI generator
all derive from one libfreemkv const, so a binary reports the exact same label it
stamps into the files it produces — no split-brain where an MKV claims one version
and the binary another.
keydb I/O moves out of libfreemkv into freemkv-keysources
(KeydbSource::save / ::update). Delete src/keydb.rs entirely (save,
http_get, default_path, write_atomic, UpdateResult) and drop `pub mod
keydb;` — http_get had no real callers. The shared Keydb* Error variants
stay in error.rs (keysources raises them; the every-error-has-a-code
contract depends on them). flate2 is no longer used here, so drop it
from Cargo.toml (zip stays for labels/jar.rs).
CHANGELOG: note the keydb-I/O move; reword the rc.5.2
DefaultDecodedFieldDuration entry to state only the action taken (the
revert) rather than an unverified Windows-fps outcome.
Sibling of Disc::copy specialized to write per-file instead of a whole ISO
image, decrypting on the way out: walk the UDF tree, read each file's extents
through the shared DecryptingSectorSource (AACS unit-aligned, CSS per-VTS),
strip AACS/, sanitize host paths per component, .partial+rename, 1-shot with
per-file loss accounting (no mapfile; recovery stays the iso:// multipass
path). Reuses UdfFs + the decrypt seam; only the per-file orchestration is new.
The AACS unit-alignment gate measured `lba % 3` against absolute disc LBA 0,
but aligned units are anchored at each clip's encrypted-region start. A clip
whose start_lba is not 3-aligned had its readable units wrongly rejected with
"Decryption failed" (the big-title-only failure on some Blu-rays). One
canonical clip-anchored helper (`aacs::is_unit_aligned`) is now the single
source of truth for the decrypt-on-read gate; both mux read paths set the
per-extent `unit_base = start_lba` via a new `SectorSource::set_unit_base`.
Also moves key *mechanism* into the library: the encrypted sample reader
(`read_encrypted_units`) and the candidate-key resolution loop
(`resolve_and_apply`) now live here, so a key source is purely a lookup.
Regression test covers a clip based at a non-3-aligned LBA.
- CSS: unlock scrambled-sector reads on enforcing drives via bus-auth
only; classify sense 6F/03 as CSS-locked; early-bail on a fully locked
scan; gate the AACS handshake off DVD discs.
- DVD first-play menu no longer prepended to the feature: read the title
VOBS base from vtstt_vobs (0xC4), not the menu VOBS vtsm_vobs (0xC0).
- Interlaced field-duration (DefaultDecodedFieldDuration) written as a
direct TrackEntry child rather than inside Video, so Windows reports
the correct frame rate.
- Audio channel count read from the AC-3 bitstream; FieldOrder set to
TFF; per-track BPS tags.
- Structured disc diagnostics at --log-level 3; reduced per-operation
log spam.
libfreemkv must stay firmware-clean for crates.io. Move ALL drive-unlock
knowledge — firmware blobs, WRITE_BUFFER/MODE SELECT upload, unlock CDBs,
the MT1959 variant-A/B handshake, the 800 KB profiles.json database, and
the DriveProfile parsing — out into the freemkv-unlock-ld crate.
libfreemkv now keeps only the seam:
- Unlocker trait (name/matches/unlock) + a process-wide ordered registry
(register_unlocker / route_unlock) in src/unlock.rs
- Drive::init() walks the registry; the first unlocker whose matches(id)
is true runs unlock(scsi, id); if none match the drive is left in
stock mode and the host-cert AACS handshake (the OEM route) carries
the disc.
The unlocker issues its own CDBs through the public ScsiTransport::execute,
so libfreemkv knows nothing about how unlocking happens.
Removed:
- profiles.json
- src/platform/mt1959/{mod,variant_a,variant_b}.rs
- src/profile.rs (DriveProfile, ProfilesFile, find_by_drive_id, ...)
- the PlatformDriver trait
Because the Unlocker seam reports only success/failure (no extended-access
marker), VID acquisition is now always via the cert-based handshake; the
per-drive OEM-VID-CDB shortcut and Drive::is_unlocked() (now const false)
are removed/neutralized. Disc-speed calibration moved into the unlocker's
unlock(); Drive::probe_disc() is a no-op.
git grep over src/ is firmware-blob/profiles/WRITE_BUFFER/mt1959-free.
All tests pass on Rust 1.86 (precommit green).
Library-wide review-and-fix pass: tightened AACS keydb/handshake/variant
handling and trailing-partial-unit policy, corrected MPLS mark offset and
added UDF allocation bounds, hardened the mux/codec framing and M2TS paths,
guarded SCSI READ CAPACITY short transfers and unified error mapping, added
overflow guards on untrusted disc input, and made prefetch shutdown
deterministic. Release profile now builds with thin LTO + single codegen unit.
The library no longer loads a keydb anywhere. The scan path always captures the
disc's AACS inputs (MKB, VID, Unit_Key_RO.inf) and resolves NO key; a caller
resolves a Key from a key source and applies it via Disc::decrypt_with.
- ScanOptions loses keydb_path / unit_key / disable_keydb (and the path search);
it now carries only optional DriveCredentials (host certs) for the live-drive
AACS handshake. do_handshake_cert uses those instead of loading the keydb.
An unlocked / LibreDrive drive takes the OEM Volume-ID path and needs none.
- The mux input() path takes caller-resolved unit_keys instead of a keydb_path,
and applies them via decrypt_with.
- Deleted the now-dead inline resolve_encryption / resolve_encryption_static.
All 700+ lib tests pass.
The library now describes the key-source abstraction it consumes, while the
concrete sources live in a companion crate. A KeySource looks a disc up from
its DiscInputs (disc hash, volume id, MKB, Unit_Key_RO.inf) and returns a Key;
the library derives down and decrypts. Sources do no derivation.
Disc::inputs() exposes those inputs from a scanned disc.
Key::Media now carries a Vec of candidate media keys rather than one: a media
key is MKB-scoped, so a source hands its whole pool and the library picks the
one that verifies against the disc's MKB. That mirrors Device/Processing and
keeps key selection (a derivation step) inside the library.
Disambiguates the key vocabulary: Key (the input handed to decrypt_with),
key sources (the resolver's lookup list), and KeyOrigin (how a key was
resolved). Internal-only rename — no dependents import it.
Add libfreemkv::Key (non_exhaustive; Unit wired) + Disc::decrypt_with(Key) — the
caller resolves a key from any source and hands it in; the library decrypts with
no keydb/network of its own. inject_unit_keys now synthesizes a minimal
ExternalUk AACS state when a scan built none (no keydb at scan time), so a disc
keyed at sweep but scanned without a keydb still decrypts from its mapfile UK
instead of reporting E8005 and deferring the mux forever. Tests cover the
synthesize, replace, empty->None, and decrypt_with paths.
- Introduce DrmScheme enum (Css/Aacs10/Aacs20/Aacs21) + drm module with
uniform detect/load dispatch across all four protection schemes.
- Land AACS 2.1 Media Key Variant framework in aacs::variants: chain
derivation, MKB record types 0x82/0x83, bit-0x02 SoftKCD and bit-0x04
online-challenge detection. Aacs21 dispatcher arm wired but commented
out pending validation against a Variant-scheme disc.
- Replace aacs2: bool with AacsVersion enum across ContentCertificate,
UnitKeyFile, ResolvedKeys. resolve_keys splits into _v1/_v2/_v21.
- Delete the libredrive raw-read VID shortcut from do_handshake; the
drive enforces the AGID requirement regardless of firmware-upload
state, so the shortcut spuriously dispatched E7017 instead of
surfacing the real downstream walls.
* `PrefetchedSectorSource::new_with_events` adds an optional
`event_fn` callback that fires `BytesRead` after every successful
batch from the producer thread. The original `new()` becomes a
thin no-events wrapper. Lets autorip wire the highway and still
get UI progress events without polling the consumer side.
* `build_iso_pipeline` grows an `event_fn` arg so the autorip
multipass mux can pipe BytesRead straight through to its progress
UI.
* Stream trait gains a default `errors() -> u64` method (= 0) so
Box<dyn Stream> callers (autorip's mux loop) can read the
skip-on-error counter without downcasting. `DiscStream` overrides
to return its `errors` field.
* Delete `DiscStream::new_pipeline` and the pipeline-mode fields
(`demux_thread`, `demux_rx`) plus the `read_pipeline` helper.
All pipeline construction now goes through
`PipelinedPesStream` via `build_iso_pipeline`; `DiscStream`
becomes the single-thread-only inline path used by the drive
single-pass read.
* `lib.rs` re-exports `build_iso_pipeline`.
Introduces the freemkv mux throughput highway: a three-stage thread
pipeline that replaces the inline single-thread read path for any
file-backed source (ISO and m2ts file URLs both route through it).
Thread A: read + decrypt (PrefetchedSectorSource / BytePrefetcher)
Thread B: M2TS demux (DemuxThread)
Thread C: codec parse (PipelinedPesStream, on caller thread)
Each handoff uses a bounded crossbeam channel with a recycled buffer
pool — no allocations or memcpys in the steady-state hot loop.
Component map:
* io/byte_prefetcher.rs (new) — std::io::Read producer thread with
recycled Vec<u8> pool. Pairs with PrefetchedSectorSource (sector
side) so demux_thread::spawn_zero_copy can wire either upstream.
* sector/prefetched.rs — recycled buffer pool added; into_channels()
peels off the rx/recycle_tx/shell triple for zero-copy demux.
* mux/demux_thread.rs (new) — owns the TsDemuxer/PsDemuxer, runs
feed() on its thread, ships Vec<PesPacket> batches.
* mux/pipelined_stream.rs (new) — the read-side Stream impl. Pulls
packets from the demux thread and runs codec parse on the caller.
* mux/resolve.rs — build_iso_pipeline (public) / build_m2ts_pipeline
(private) assemble the three stages; iso:// and m2ts:// both
return PipelinedPesStream.
* mux/m2ts.rs — collapsed to a write-only sink (Mode::Read deleted;
the read direction lives on the highway now).
* mux/codec/h264.rs — find_start_code uses memchr SIMD memmem::find.
* mux/codec/hevc.rs — tightened frame_data initial capacity.
* mux/ts.rs — boundary-packet handling avoids the per-batch 16 MiB
remainder copy; PesAssembler starts at 16 KiB to dodge the 64-page
first-touch fault tax that the previous 256 KiB pre-alloc paid on
every PES boundary.
* mux/disc.rs — gains DiscStream::new_pipeline + read_pipeline as
the legacy autorip ingress (drive + multipass paths still need
on_event / skip_errors before they migrate to the highway).
* io/file_sector_source/* — per-OS prefetch() syscall hook
(Linux readahead, macOS F_RDADVISE, Windows/other no-op).
* decrypt.rs — FREEMKV_DECRYPT_THREADS renamed to FREEMKV_THREADS;
pool sized to all cores by default.
Measured on rip1 testbed (Civil War UHD, 62 GiB ISO → null://):
60 → 322 MB/s warm cache (old new_pipeline path)
60 → 660 MB/s warm cache (highway path, this commit)
60 → 126 MB/s sustained disk-bound
The IsoSectorReader baseline reader was deleted in favour of
FileSectorSource so the freemkv CLI and autorip exercise the same
read path.
WO-2 (delete SectorReader trait):
- The 0.18 trait split into SectorSource (read-only) and SectorSink
(write-only) is final; the legacy SectorReader alias was a bridge.
- Renames every internal &mut dyn SectorReader (~25 sites) to
&mut dyn SectorSource. The trait method capacity() becomes
capacity_sectors() with a default of 0 (preserves SectorReader's
default-0 behavior).
- Deletes the SectorReader trait, its blanket-to-Source bridge, and
the FileSectorReader type alias. Adds explicit forwarding impls
for Box<dyn SectorSource> and &mut dyn SectorSource so generic
decorators like DecryptingSectorSource<S: SectorSource> compose.
WO-3a (extract Disc::patch):
- Moves Disc::patch (1230 lines) and bytes_bad_in_title from
disc/mod.rs into disc/patch.rs as a split inherent impl. Zero
behavior change — pure mechanical relocation. disc/mod.rs drops
from 3,945 to 2,714 LOC.
WO-6 (partial):
- Deletes src/labels/png_filenames.rs — was a 72-LOC stub with
detect() returning false, never wired into the PARSERS registry.
project docs doc drift fixes (audited 2026-05-13):
- JUMP_BASE_SECTORS: 256→1024 (64 MB base for UHD, not 8 MB)
- PASSN_DAMAGE_THRESHOLD_PCT: 12→6
- PASSN_SKIP_SECTORS_BASE: 64→32
- MAX_RANGE_SECS=180: replaced by proportional range_sectors × 25,
capped at RANGE_BUDGET_CAP_SECS=1800.
The 0.18 trait split into FrameSource (read-only) and FrameSink
(write-only) was an over-engineered API. Consumers don't think
"frame source backed by MKV" — they think "open MKV for reading".
The split paid a real API-complexity cost (two trait names, two
re-exports, dual impls per bidirectional type, deprecation bridge)
for one marginal property: compile-time direction-safety at the
trait-object boundary. The runtime error path on a wrong-direction
call (StreamReadOnly / StreamWriteOnly) is unambiguous and rare in
practice.
Deletions:
- pes::Stream is no longer #[deprecated]
- pes::FrameSource trait + its blanket-from-Stream bridge
- pes::FrameSink trait + the trampoline impls on every concrete type
- The compile-time-direction-safety test scaffolding
- Crate-root FrameSource / FrameSink re-exports
Additions:
- Stream is now Send-bounded (Stream: Send supertrait). Every
concrete impl was already Send-compliant — Box<dyn Read + Send>
and Box<dyn Write + Send> were already in place on the trait
objects MkvStream / M2tsStream / etc hold internally. Promoting
Send into the trait makes Box<dyn Stream> Send too, which lets
autorip drop its SendStream unsafe newtype.
The public API is now: one Stream trait, one concrete type per
format, two constructors (open/create or input/output). Bidirectional
types route through internal Mode { Read | Write } discriminants.
Net: -347 lines libfreemkv, -38 lines autorip, -5 lines freemkv.
Promotes `mod labels` to `pub mod labels` and adds `analyze()` plus
`LabelAnalysis` (both `#[doc(hidden)]`) so an out-of-tree diagnostic
binary (freemkv-tools labels-analyze) can introspect which BD-J parser
matched a given disc, what JAR files the discriminators saw, and what
labels came out — without going through the production `apply()` path
that mutates DiscTitles.
Also adds `tracing::info!(parser = name, "label parser matched")` /
"no label parser matched" inside `extract()`. Dev-only signal: users
get the same seamless behavior; developers can finally tell whether a
disc hit a real parser or fell through to the codec-name fallback in
fill_defaults().
The new `jar_inventory()` helper deduplicates and sorts filenames
under any `/BDMV/JAR/<x>/` subdirectory — same plumbing the existing
`jar_file_exists()` discriminators use, just enumerated rather than
predicate-tested. Used by `analyze()` to surface unrecognized
parser-source files when no parser matches, which is the input to
"do we need a new parser?" triage.
No behavior change to the production label path. `apply()` and
`extract()` remain functionally identical; the new public surface
exists alongside.
Round 3 step 1: lift the visibility on the two flat rip-phase verbs
so consumers (autorip + freemkv CLI) can call them directly instead
of going through Disc::copy's multipass dispatcher. Also lift their
option/outcome types and re-export at crate root.
- fn sweep -> pub fn sweep (with rustdoc explaining its role)
- fn patch -> pub fn patch (ditto)
- pub(crate) struct SweepOptions -> pub struct SweepOptions
- pub(crate) struct PatchOpts -> pub struct PatchOptions (renamed
for consistency — both are 'Options')
- pub(crate) struct PatchOutcome -> pub struct PatchOutcome
- libfreemkv::{SweepOptions, PatchOptions, PatchOutcome} re-exports
at crate root.
Disc::copy still exists and still calls Disc::sweep / Disc::patch
through the now-private sweep_internal / patch_internal wrappers.
Migration of the two autorip callers + the freemkv CLI's
disc_to_iso to direct sweep/patch is a follow-up; once those land
Disc::copy + CopyOptions + CopyResult delete in the same commit.
See (internal)/memory/0_18_redesign.md and
0_18_round3_migration_audit.md.
Single contributor: MattJackson.
Round 2 #1 landed `Pipeline` / `Sink` / `Flow` / `DEFAULT_PIPELINE_DEPTH`
in `crate::io::pipeline` but only re-exported them through
`crate::io` (which is `pub(crate)`), so no out-of-tree consumer could
reach them. autorip's round 2 #2 (lifting the mux loop onto Pipeline +
MuxSink) is the first such consumer; surface the primitives at the
crate root for ergonomic access.
No behaviour change — the items themselves are unchanged from round
2 #1; this is just `pub use` plumbing.
Single contributor: MattJackson.
Splits the unidirectional read trait from a (planned) write trait at
the sector level, eliminating runtime "wrong direction" potential.
Keeps SectorReader alive as a pre-deprecation alias via blanket impl
so existing callers compile unchanged through the migration window.
Adds DecryptingSectorSource decorator: wrap any SectorSource in this
to get plaintext sectors out. Replaces the duplicate decrypt code
paths in sweep_pipeline and DiscStream (those migrations are
follow-up commits).
The formal #[deprecated] attribute on SectorReader is held back to a
follow-up commit because internal call sites in disc/, udf/, mux/,
and verify/ still go through the legacy trait, and the CI gauntlet
treats deprecation lints as errors. Behavioural intent — "this trait
is going away" — is documented on the trait itself.
See (internal)/memory/0_18_redesign.md.
Single contributor: MattJackson.
Splits the bidirectional pes::Stream into one-direction traits so
calling read() on a write-only sink is a compile error instead of
runtime E9001. Keeps Stream alive as deprecated through 0.18 with a
blanket FrameSource impl so existing concrete types compile unchanged.
FrameSink can't be blanket-impl'd from Stream (different finish
signature), so concrete impls migrate per-type in a follow-up.
Concrete `impl pes::Stream for X` blocks in mux/* and the existing
tests gain a one-line `#[allow(deprecated)]` to keep `-D warnings`
clean during the deprecation window — no behavior changes.
See (internal)/memory/0_18_redesign.md.
Single contributor: MattJackson.
One-bit cooperative cancellation flag. Replaces ad-hoc Arc<AtomicBool>
patterns scattered across libfreemkv (DiscStream::set_halt) and the
HALT_FLAGS global registry in autorip. See
(internal)/memory/0_18_redesign.md.
Single contributor: MattJackson (no attribution trailers).
Pass 1 sweep speed on a healthy disc previously dipped from ~15 MB/s
to ~1 MB/s every ~30 s on a host with default Linux dirty-page
settings. Empirical cause: the kernel's vm.dirty_ratio (~20% of RAM)
lets hundreds of MB of dirty pages accumulate, then bursts a flush at
99% disk utilisation that blocks app writes for ~1 s. Confirmed on
the BU40N test bed — dirty pages grew 112 → 563 MB between bursts;
lowering vm.dirty_bytes to 64 MB at the host sysctl level eliminated
the dips. Shipping the equivalent inside libfreemkv so users do not
need to tune the host kernel.
- New crate::io::Writer: drop-in File wrapper (impl Write + Seek).
Wraps a per-platform WritebackPipeline that on Linux schedules
sync_file_range(WRITE) + lagging sync_file_range(WAIT_AFTER) +
posix_fadvise(DONTNEED) in 32 MB chunks, bounding dirty cache at
~64 MB. macOS and Windows ship a no-op stub.
- Disc::sweep wraps its output File in Writer. Loop body unchanged.
- Module is purpose-built so any large sequential output (patch,
mux) can adopt the same wrapper as a one-line change later.
bytes_pending was an opaque aggregate of NonTried + NonTrimmed +
NonScraped. UIs that wanted a "will retry in Pass 2-N" bucket were
stuck showing the entire unread disc as Maybe at pct=0.
Adds two granular fields to MapStats:
bytes_nontried — Pass 1 hasn't read these yet
bytes_retryable — NonTrimmed + NonScraped, Pass 2-N will retry
bytes_pending stays for back-compat (= bytes_nontried + bytes_retryable).
Also picks up the cargo fmt --check lint that's been red on main CI
since v0.13.18 (rustfmt fold differences on a few long format-string
layouts; functional no-op).
Through the entire 0.13.x line, every CHECK CONDITION reply from the
drive (the standard way SCSI tells you why a sector failed) was being
collapsed into a synthetic status=0xFF, sense_key=0 transport-wedge
sentinel and the actual sense data was thrown away. Confirmed live on
the BU40N reading Dune 2 on 2026-04-27: drive returned host_status=0,
driver_status=8, status=2, exec_elapsed_ms=1416 on every bad sector
— a clean CHECK CONDITION carrying full sense data — and Disc::copy
was bailing on it as if the bridge had wedged.
Root cause: scsi/linux.rs's wedge check was
`host_status != 0 || driver_status != 0`
SG's DRIVER_SENSE bit (0x08) is set on every CHECK CONDITION reply
just to flag "sense buffer is populated" — it's not a transport
failure on its own. Pre-fix we conflated the two and silently lost
every drive-reported error reason. macOS and Windows backends had
the same shape: they extracted sense_key only, dropping ASC/ASCQ.
API restructure (clean separation):
Error::ScsiError {
opcode: u8,
status: u8, // 0xFF = synthetic transport-failure
sense: Option<ScsiSense>, // None ⇔ no sense delivered
}
pub struct ScsiSense { sense_key: u8, asc: u8, ascq: u8 }
impl ScsiSense {
pub fn is_marginal(&self) -> bool // keys 0/1/3/B
pub fn is_medium_error(&self) -> bool
pub fn is_hardware_error(&self) -> bool
pub fn is_unit_attention(&self) -> bool
pub fn is_data_protect(&self) -> bool
pub fn is_not_ready(&self) -> bool
pub fn is_illegal_request(&self) -> bool
pub fn is_aborted_command(&self) -> bool
}
impl Error {
pub fn scsi_sense(&self) -> Option<&ScsiSense>
pub fn is_scsi_transport_failure(&self) -> bool
pub fn is_marginal_read(&self) -> bool
}
SCSI protocol constants (SCSI_STATUS_*, SENSE_KEY_*) moved from
error.rs to scsi/mod.rs where they belong alongside SCSI_INQUIRY,
SCSI_READ_10, etc. parse_sense replaces parse_sense_key (returns the
full triple, not just the key); inline tests now exercise ASC/ASCQ
extraction at the right offsets for both descriptor (0x72/0x73) and
fixed (0x70/0x71) sense formats.
Disc::copy + Disc::patch sense-aware dispatch:
- marginal sense (MEDIUM ERROR / ABORTED COMMAND / RECOVERED ERROR
/ NO SENSE) → engage hysteresis (Block→Single, bpt=1)
- non-marginal sense (HARDWARE / DATA PROTECT / UNIT ATTENTION /
NOT READY / ILLEGAL REQUEST / transport failure / kernel
IoError) → bail with full sense info preserved; caller (autorip)
surfaces "physical replug" / "drive failing" / "media changed"
Pre-fix: every CHECK CONDITION → 0xFF synthetic → Disc::copy bailed
→ bytes_good froze at the bad zone. The hysteresis from v0.13.22
was correct but never got to run. This release unblocks it.
Disc::patch's wedged_threshold (50 consecutive failures) stays as
defense-in-depth for chains of marginal failures; a single
non-marginal sense now short-circuits it.
New phase=bail trace event records the bail reason with the sense
triple. phase=transport_err remains for genuine bridge wedges /
kernel timeouts; phase=scsi_err carries the parsed sense_key, asc,
ascq for drive-reported errors.
All 350 tests pass. Clippy clean across all targets.
The v0.13.21 bisect-on-fail recovery was correct (100% of recoverable
sectors picked up) but slow on dense damage clusters. Live test on
Dune 2 v0.13.21 burned ~30 s per damaged 60-block — paying a ~5 s
kernel ABORT/timeout at every level of a log₂(60) ≈ 6 deep DFS, on
the failing branch each time.
Replaced with a two-state hysteresis machine in Disc::copy:
Block(batch):
read(batch) ok → write, advance, stay Block
read(batch) fail → switch to Single, retry SAME range at bpt=1
Single:
read(1) ok → write, consecutive_good++
if consecutive_good >= BPT1_EXIT_THRESHOLD:
switch to Block, reset counter
read(1) fail → mark NonTrimmed, consecutive_good = 0
BPT1_EXIT_THRESHOLD = 10_000 sectors (= 20 MB clean run). Calibrated
from the 2026-04-26 BU40N empirical probe data; tunable.
Per-block math on a damaged 60-block with 1 truly bad sector:
Bisect (v0.13.21): ~30 s (5 s × 6 levels)
Hysteresis (v0.13.22): ~10 s (5 s bpt=batch fail
+ 59 × 1 ms good
+ 1 × 5 s bad)
Inside a damaged cluster spanning many 60-blocks the win compounds:
hysteresis pays the bpt=batch fail cost ONCE on entry, then stays at
bpt=1 across the cluster; bisection re-paid it every 60 sectors. For
Dune 2's ~1248-sector boundary cluster that's ~21 fewer 5-sec
kernel timeouts ≈ 100 s saved per pass.
Telemetry: new phase=mode_change trace event with from, to, lba, and
consecutive_good. Replaces v0.13.21's phase=bisect. Worklist DFS is
gone — single iterative for s in 0..count on the failure path.
Test rename, same fixture and same 100% recovery expectation:
test_disc_copy_bisect_recovers_via_single_sector_reads
→ test_disc_copy_hysteresis_recovers_via_single_sector_reads
Also adds DamageSeverity (Clean / Cosmetic / Moderate / Serious) +
classify_damage(bad_sectors, lost_ms), re-exported from libfreemkv,
so applications can render structured severity instead of formatting
their own from raw counters.
- scsi/linux.rs: full rewrite from async write/poll/read+1.5s timeout+
close-on-timeout to one synchronous ioctl(fd, SG_IO, &hdr). Kernel
honors hdr.timeout and runs its own ABORT/RESET escalation. Errors
check host_status and driver_status (both 0xFF-synthesised) plus
status. Sense-key parser handles descriptor (0x72/0x73) + fixed
(0x70/0x71) formats. Deleted fd_recovery, bg close+open thread, fd
swap dance. -331/+155 lines.
- scsi/macos.rs: try_recover() removed (userspace handle-recovery on
task failure was the same anti-pattern stripped from Linux). bsd_name
field deleted. Errors bubble up directly.
- scsi/windows.rs: try_recover() removed, wide_path field deleted,
INVALID_HANDLE guard removed.
- scsi/mod.rs: parse_sense_key() helper extracted (used by all three
platforms now — single canonical sense-key parse rather than three
inlined copies). +10 unit tests covering descriptor format, fixed
format, truncated buffers, unknown response codes.
- drive/mod.rs: Drive::reset() deleted (escalating eject + STOP/START +
reinit recovery — per audit, kernel handles its own escalation;
userspace shouldn't).
pub fn find_drives() -> Vec<Drive> deleted (opened N drives just to
throw most away). find_drive() now uses discover_drives() directly.
wait_ready() simplified — drops the reset path on sense_key=5,
just keeps polling TUR for 60 iterations.
- lib.rs: find_drives re-export removed.
- benches/sgio_read.rs: switched to find_drive() (no longer iterates a
drive list).
Net: 9 files changed, 226 insertions(+), 473 deletions(-). 329 tests
pass, clippy -D warnings clean. No consumer breakage (CLI, autorip,
bdemu compile + test green).
Architecture decision documented in
(internal)/docs/audits/2026-04-26-scsi-architecture-research.md
(primary-source survey of MakeMKV, sg_dd, ddrescue, and the kernel
mid-layer's own scsi_eh.rst escalation ladder).
Pre-0.13.16 the rip API leaked internal mapfile concepts (pos,
bytes_good, work_done, bytes_pending, Finished/NonTrimmed) into per-pass
positional callbacks Fn(u64, u64, u64). Consumers reinvented the math
each time, and v0.13.15's UI bug surfaced exactly because of this —
autorip's web JS computed pct from bytes_good while the backend
computed from pos, silent drift, frozen UI bar.
This release replaces both Disc::copy::on_progress and
Disc::patch::on_progress callbacks with a single Progress trait +
PassProgress struct (new progress module).
pub struct PassProgress {
pub kind: PassKind, // Sweep | Trim {reverse} | Scrape {reverse} | Mux
pub work_done: u64,
pub work_total: u64,
pub bytes_good_total: u64,
pub bytes_total_disc: u64,
}
pub trait Progress {
fn report(&self, p: &PassProgress);
}
impl<F: Fn(&PassProgress)> Progress for F { ... } // closures work directly
CopyOptions::on_progress and PatchOptions::on_progress are renamed to
progress: Option<&dyn Progress>. Closure callers update trivially via
the blanket impl; struct callers gain a clean named-field shape with no
positional-arg confusion.
PassKind carries the semantic (sweep vs trim vs scrape vs mux) so
consumers can label phases without reinventing detection logic.
Disc::patch reports Trim {reverse} for retry passes with block_sectors
>= 2 and Scrape {reverse} when block_sectors == 1. Direction comes
through reverse: bool. Mux variant is reserved for v0.13.17 when the
mux pipeline emits progress.
Tests + clippy clean across all 4 crates.
Architectural cleanup. autorip + freemkv CLI were reimplementing drive
discovery (sysfs walking, type-5 filtering, sg-path construction) and
calling SCSI reset primitives directly. All of that hardware-aware code
moves into libfreemkv with two cheap public probes:
- DriveInfo + list_drives() — multi-OS enumeration (Linux/macOS/Windows)
with peripheral-type-5 filtering and INQUIRY identity. Cheap.
- drive_has_disc(path) — single TUR with internal wedge recovery
escalation (SCSI reset → USB reset → retry) hidden from callers.
USB-layer reset (USBDEVFS_RESET / IOUSBDeviceInterface::ResetDevice /
storport's combined reset) wired across all three platforms.
Visibility tightening — scsi::reset, scsi::usb_reset, and the timeout
constants are now pub(crate). Compile-time guarantee that no consumer
crate can issue SCSI commands directly.
233 lib tests pass; clippy clean.
Audit pass against the project docs "no English text in library code" rule.
Found 9 call sites that violated the contract by stuffing English into
io::Error::new(kind, "…") or by abusing Error::DeviceNotFound { path }
as a free-form description field. Each is now a typed Error variant.
New variants and codes: ScsiInterfaceUnavailable (E1004), DeviceLocked
(E1005), IoKitPluginFailed (E1006), UnsupportedPlatform (E2003),
PlatformNotImplemented (E2004), MapfileInvalid (E6011), DiscUrlNotDirect
(E9009).
labels::apply() previously pushed Commentary/Descriptive/Score/IME and
" (Secondary)" English literals into AudioStream.label, leaking into
MKV titles + autorip UI. AudioStream now exposes structured `purpose:
LabelPurpose`, SubtitleStream `qualifier: LabelQualifier`. Callers
translate to localized text. label keeps codec-formatting only.
API hygiene: 11 mux/* modules dropped from `pub` to `pub(crate)` —
their *types* are still re-exported from lib.rs, but the modules were
leaking low-level EBML/TS/network primitives. Stream trait gets a real
rustdoc explaining read-vs-write split. lib.rs grouped re-exports into
documented sections. ScanOptions::with_keydb() removed (one-method-per-
action rule); use struct literal.
Dead-code sweep: removed lookahead.rs (orphan, never declared as mod),
tsreader.rs (TsDemuxReader unused), ebml::{write_int,read_vint,SEEK_*},
ts::{scan_first/last_pts,scan_duration,SCAN_HEAD/TAIL_SIZE,take/set_
remainder}, MkvMuxer codec_private_slots/filled fields and
fill_codec_private method (deferred-codecPrivate path never used since
the v0.10 PES rewrite). cargo clippy --all-targets -D warnings clean.
Tests: new error::tests for variant codes + Display "no English" guard +
io::ErrorKind mapping. 233 lib tests, all green (was 230).
Breaking: ScanOptions::with_keydb removed; mux/* modules pub(crate);
AudioStream and SubtitleStream gained required fields; UnsupportedDrive
{ product_revision: "Renesas not yet implemented" } no longer produced
(use PlatformNotImplemented).
New primitives for two-stage rip workflows: fast forward pass with
zero-fill on failures, then targeted retries of bad ranges via a
ddrescue-compatible mapfile.
- Disc::copy now takes &CopyOptions (breaking change from positional
args). Always writes a sidecar .mapfile. Opt-in skip_on_error +
skip_forward give ddrescue-style fast sweep: 64 KB blocks,
exponential skip-forward on failure, zero-fill bad blocks. Defaults
preserve pre-0.11.21 behavior (recovery reads, abort on bad sector).
- Disc::patch is new and idempotent. Reads the mapfile, re-reads every
non-finished range with full drive recovery, patches good bytes back
into the ISO at exact offsets. Call N times for N retry attempts.
- disc::mapfile is a new module. ddrescue text format, crash-safe
(flushed on every record()), greppable, human-editable, tool-compatible.
Status chars match ddrescue: ? / * / / / - / +.
- Re-exports FileSectorReader from the crate root.
- freemkv CLI caller (pipe.rs) updated to the new Disc::copy signature
in lockstep — shipped in the 0.11.21 freemkv CLI release.
Part of the 0.11.21 ecosystem sync (libfreemkv + freemkv + bdemu +
autorip all on 0.11.21).
Architecture:
- One stream per format, bidirectional PES (read/write on same type)
- IsoStream merged into DiscStream (one type, any SectorReader)
- Disc::copy() for disc→ISO raw sector dump
- IOStream trait deleted, all byte-level Read/Write removed
- ContentReader/OpenDisc/open_title/open_input/open_output deleted
- CountingStream wrapper for progress tracking
Error codes:
- All io::Error English strings replaced with Error enum variants
- From<Error> for io::Error conversion
- Unused variants removed, new stream/mux variants added
Deleted: mkvout.rs, pesout.rs, isowriter.rs, mkv-muxer-plan.md
Updated: all docs, README stream table, CHANGELOG
238 tests, 0 clippy warnings.
Stream trait: read() returns PesFrame, write() accepts PesFrame.
A stream is a stream — you read from it or write to it.
No separate Input/Output traits.
API: libfreemkv::input(url) and libfreemkv::output(url, title, codecs)
Returns Box<dyn Stream>.
- TsMuxer: PES frames → BD-TS packets (new, reverse of TsDemuxer)
- M2tsOutputStream: PES → TsMuxer → file
- NetworkOutputStream: PES → TsMuxer → TCP
- StdioOutputStream: PES frames → stdout
- NullOutputStream: discard
- MkvOutputStream: PES → MKV mux
- All outputs via open_pes_output()
- All inputs via open_pes_input() (ISO, disc)
- No byte-level fallback — everything is PES
Matthew Jackson