Round-5 findings from the 10-phase release audit:
- collect_es routed EVERY extended-stream-id (0xFD) PES into the video ES
buffer, so a 0xFD HD-audio sub-stream (MLP/TrueHD) could pollute the
video sample and — if it preceded the video PES — stamp the video track
with the audio PID, losing the video. Only the VC-1 extension (0x55) is
now treated as video; routing 0xFD audio to its own track is deferred to
the HD-DVD program-chain follow-up.
- The sparse-PTS reorder now carries its calibrated per-frame duration onto
each frame, so the muxer emits a BlockDuration and the back-patched
Segment Duration covers the final frame instead of understating it.
- Add regression tests for the MAX_MARKS and MAX_VTI_HITS caps (promote
MAX_VTI_HITS to module scope); make the differential-test factory array a
named type; drop an identity-op in a reorder test.
Round-4 findings from the 10-phase release audit (the first fully clean
round; it dug into the new #22/#18 refactor code):
- AuAssembler closed each AU from only the FRONT mark's fields, so when
one PES fragment carried the source and a later fragment of the same AU
carried the PTS, the second field was dropped — a regression vs the old
separate pts/source mark deques. Now merge the first Some of each field
across all in-range marks.
- parse_vti_clip_order picked the largest residue bucket with
HashMap::into_values().max_by_key(), nondeterministic on a size tie
(randomized HashMap iteration) — could select a different clip table
run-to-run. Break ties by smallest offset.
- Bound the marks/disc_marks deques (MAX_MARKS): the buf-size cap prunes
marks only when bytes accumulate, so a run of zero-length timed
fragments could grow them without bound on hostile input.
- Add push_owned so the PS path moves the PES payload into a passthrough
AU with no copy (MPEG-2 video + all audio), removing a per-PES
malloc+memcpy the refactor had introduced on the DVD path.
- Back-patch the MKV duration from the block END (start + its own
duration) so it covers the final frame instead of understating by one.
- Add direct tests for the MKB record-framing walker; drop a stale
drain_complete_aus doc comment left on process_au.
Round-2 findings from the 10-phase release audit:
- parse_vti_clip_order bucketed hits by residue with an O(stride*hits)
rescan and no hit cap, so a crafted HD-DVD VTI packed with millions of
`.EVO` tokens (up to the 64 MiB UDF read cap) could burn seconds of CPU
on a routine scan. Bucket in a single O(hits) pass and cap collected
hits at MAX_VTI_HITS (a real table holds a few dozen).
- Fix the stale `super::keys::…` intra-doc links left by the aacs module
rename: the referenced fns live in `super::derive`.
Round-1 findings from the 10-phase release audit:
- SparsePtsReorder buffered its current GOP with no bound, draining only on
a keyframe — an open-GOP or crafted program stream that never signals one
could hold the whole title in RAM. Force-complete the GOP at
MAX_GOP_FRAMES, matching the MPEG-2 parser's backstop.
- inject_unit_keys labelled a 2.1 FMTS disc as AACS 1.0 / bus-encryption
off; FMTS is UHD-family, so synthesize the UHD version + bus encryption.
- The compiled Key Correction Data was a non-zero 16-byte constant fed into
the Media Key derivation. Per the no-compiled-keys rule it is now all-zero;
the chain still cannot complete on a real disc (documented), so this is
behaviour-neutral — all variant tests pass unchanged.
- Fix stale doc references (broken `super::variants` intra-doc links, and
`aacs::keys` comments) left by the module rename.
HD-DVD Standard Content splits the main feature across clips at the
layer break (FEATURE_1/FEATURE_2, or feature/feature_Divide). The scanner
enumerated one title per .evo, so main-title selection picked only part 1
(e.g. Shaun's 11 GB FEATURE_1, missing the 6.8 GB FEATURE_2).
Parse the HVA*.VTI navigation file's clip table — a fixed-stride record
list naming every clip in authored order, isolated by residue-mod-stride
rather than the imprecise header pointer — and concatenate the feature
clips (matched by the feature* naming convention) into one title whose
extents run in authored order. Every other clip stays its own title.
Falls back to one-title-per-clip when the VTI is absent or unparseable,
so nothing regresses on a disc with no readable navigation.
Validated on real discs: Shaun 17.8 GB / Anchorman 20.1 GB / Harry Potter
24.4 GB now enumerate as one 2-clip FEATURE title (largest = the movie).
VC-1 HD-DVDs (e.g. Shaun of the Dead) carry video on MPEG-PS extended
stream id 0xFD, with the real stream selector in stream_id_extension
inside the PES extension. Parse that field so the video routes to a
distinct track (pid 0xFD00|ext) instead of being dropped.
Reframe VC-1 access units in AuAssembler with a dedicated Mode::Vc1:
an AU is delimited by the next frame BDU (0x0D) once a frame has already
been seen, so the sequence (0x0F) and entry-point (0x0E) headers that
precede an I-frame stay attached to the frame they describe. The old
single-start-code split stranded those headers on the prior AU, which
the decoder reported as bits-overconsumption and hard decode failures.
hddvd probe now tracks the video pid it detects and emits VC-1 on 0xFD.
- aacs/resolve: a media-keys-only provider missing the VID classifies as
VidUnavailable, not NoMaterial (an MK derives the VUK once the VID
arrives).
- disc/bluray: mark a clip seen only after its .clpi parses, so a
transient parse failure on the first PlayItem cannot suppress the
clip's extents for a later PlayItem referencing it that succeeds.
- disc/patch: log rather than swallow mapfile record/flush failures on a
reverify downgrade, so a failed persist cannot silently mismark a bad
unit good on resume.
- mux/ts: flag a discontinuity when a partial PES is dropped, matching
the other partial-drop paths.
- mux/demux_thread: the no-demuxer branch forwards an empty batch for
early consumer-disconnect detection instead of reading the whole disc.
- io/pipeline: correct the send-timing log (as_secs_f64, not as_micros
printed as ms).
- aacs/derive, aacs/variant, disc/read_error, keysource: comment/doc
accuracy. sector/prefetched, udf: remove dead fields/functions.
- mux/disc: assert unit-aligned read counts in the test.
Replace the AACS-specific inline key-fetch in the decrypt decorator with
a scheme-neutral recovery seam: the input stream (L3) installs a Recover
closure (none / AACS key-fetch) and the decorator (L2) runs it at the
single decrypt-miss point. FMTS (AACS 2.1) forensic-segment units that no
key opens are just undecryptable units, concealed and counted as ordinary
decrypt loss with no FMTS-specific branch ("a loss is a loss"), so the
separate bytes_undecryptable bucket collapses into one loss count.
- sector/recovery.rs: the seam (MissOutcome, none/key_fetch factories),
naming no encryption scheme in its type.
- FMTS: segment routing primitives + BYPASS_FMTS_KEY, and an upfront
ensure_forensic_segments_decryptable gate (Error::FmtsKeyMissing) in
the mux input path, parallel to the unit-key gate.
- CSS descramble/rekey moves from decrypt_sectors into
css::descramble_region: CSS self-recovers from the data itself, so it
stays OFF the seam (which is only for external inputs).
- disc/mod.rs also: main-title selection aligned to largest physical
size; is_regular read from the open file handle, not metadata(path),
fixing a swallowed sync_all on a fresh-rip ISO. decrypt_threads()
resolved once via OnceLock off the per-buffer hot path.
canonical_title_order keyed on clip-count ascending before duration, so a short 1-clip bonus reel outranked the real feature — which is often chaptered into many clips (one per chapter). Make physical size the primary key: the main feature is the largest non-oversize title; decoy 'play-all' playlists run long but tiny (reused clips) and self-eliminate.
Validated across 23 UHD/BD discs: fixes t1 on F9, Fast Five, Fast & Furious 6, and Furious 7 (feature was ranked #13-36); no regressions on the 19 already correct.
Add DiscFormat::Fmts (AACS 2.1) and DiscFormat::HdDvd as first-class peers. Format derives from the AACS MKB generation (mkb_type().generation(): V10=BD, V20=UHD, V21=FMTS), reusing existing AACS code, and from the on-disc tree for HD-DVD/DVD. One detector (detect_disc_format) shared by the coarse DiscId probe and the full scan — no more 'default BluRay, defer to full scan'.
FMTS is a BD-tree stream variant: parse_playlist resolves the clip stream via CLIP_STREAM_EXTS (.m2ts -> .fmts -> .ssif), so the .fmts main feature yields real extents (previously silently empty). HD-DVD is a tree-level peer with its own enumerator (disc/hddvd.rs): HVDVD_TS/*.evo -> MpegPs titles with real extents (playlist/stream parsing honestly stubbed).
Sample selection for key resolution now uses the authoritative AACS CPI flag (aacs_unit_encrypted, byte-0 & 0xC0) not the ts_sync_destroyed heuristic — container-agnostic (M2TS/FMTS/EVO; TS-sync is meaningless on HD-DVD program streams) and stops the decode-server '0 encrypted units' rejection.
Tests live with each format (bluray/hddvd/mod); generic UDF fixture builders extracted to a shared udf::fixture module.
Adopt freemkv-unlock's split Unlocker trait: run_features drives the drive-prep capability, run_bus the content bus removal, each iterating unlockers until one doesn't decline (NotApplicable = try next; Ok or a real error stops).
unlocker_matrix now reports which drive-prep unlocker actually ran — LibreDrive removes the bus at the drive; Renesas unlocks features but leaves the bus to the cert. Wire product_id through to fu::DriveId.
Bump to 1.2.3.
aacs::boil was a thin newtype veneer over aacs::derive — a duplicate layer.
Delete it: move the shared key newtypes (Vid, MediaKey, Vuk, ProcessingKey,
UnitKey) into aacs::types, and expose resolve_candidate + KeyCandidate from
aacs::derive directly. Downstream (keysource, disc::encrypt) now import from
aacs::{derive,types}. Pure API consolidation, no behaviour change; full test
suite green on Rust 1.86.
Relocate the shared low-level primitives into a single crypto module:
aes_ecb_encrypt/decrypt, aes_cbc_decrypt, aes_g (from content/variant) and
aesg3 + AESG3_SEED (from keys), plus AACS_IV. Fixes the scatter where AES-G
lived in the 2.1 file and AES-G3 in keys. Relocation only — no rename, no
logic change (logic-hash identical to baseline; 277 items; 2210 tests green).
Pure file+module-path rename. 'content' names the AACS unit-decrypt layer
(distinct from the top-level sector-decrypt driver crate::decrypt), and
'variant' (singular, spec term 'Media Key Variant') names the 2.1 chain.
Logic-hash identical to baseline; 277 items intact; tests green.
The tier ladder (bucket->mug->thimble) forces tier 0 to attempt ALL
ranges before tier 1, and tier 1 all before tier 2 — so on a hardened
residual (late resume, flood already gone) the marginal specialists are
unreachable for hours: you bail near-empty buckets over hundreds of dead
ranges before a thimble ever touches water.
FREEMKV_PATCH_FLAT collapses the 3 tiers into ONE flat pool of all 16
handler configs. run_handlers already sorts best-first by the live
decayed-yield scorecard, so this is a data-driven bandit: the first
ranges try them all (explore), the ranking floats whatever is actually
landing sectors to the front (exploit), re-measured per range; a handler
that doesn't fit stays last but is never dropped (floor -> can revive).
The new handlers self-limit (yield after 4 dead reads), so the flat
scheduler does NOT reintroduce the old depth-first per-range starvation.
Opt-in; unset keeps the proven tier ladder as the default for 1.2.0.
Add a third breadth-first tier (PATCH_TIERS 2->3) that runs the marginal
specialists on the hardened residual tiers 0-1 leave: SlowSpin (Linear
fwd+rev @ min), FuaRetry (Linear fwd+rev+Bisect @ FUA), SlowFua (Linear @
min+FUA), CachePrime, Oscillate (@ max and @ min), SpeedSweep. Every read is
a wedge-safe read_span, so they inherit wedge-abort / unproductive-yield /
deadline for free. All are new configs, so the EWMA scorecard calibrates each
once then ranks by decayed rate — a specialist that doesn't fit self-
deprioritises. Tiers 0-1 (fast scouts, slow-deep) are unchanged; this is
purely additive. Also switch the scorecard log sort to sort_by_key.
cargo test -p libfreemkv green (2200 passed).
CachePrime reads the good run immediately preceding a residual island to lock
the servo/PLL, then reads the island warm (boundary sectors the drive can't
cold-seek). Oscillate reads each residual sector by alternating approach —
forward-into then reverse-into — for direction-dependent tracking. Both go
through the wedge-safe read_span (primes included). FakeDisc models a
direction-dependent sector and a servo-primed boundary sector; fixtures prove
a forward/cold Linear misses each while Oscillate/CachePrime recover them.
FuaRetry (Linear fwd+rev+Bisect at {max,fua,deep}) and SlowFua (Linear at
{min,fua,deep}) are Linear/Bisect at FUA params, no new struct. FakeDisc now
models a stochastic sector that lands only on the Nth physical (FUA) read (a
cached re-read keeps missing) and a hardest sector needing BOTH min speed and
FUA. Fixtures prove cached reads keep missing while the FUA group lands the
stochastic sector, and that neither lever alone recovers the hardest sector —
only the min+FUA combination does.
SlowSpin is Linear pinned to min speed (no new struct). SpeedSweep is a new
per-sector handler that tries Max->Min until one reads (speed resonance).
FakeDisc now models a sector that reads ONLY at min speed; fixtures prove a
max-speed Linear misses it while SlowSpin and SpeedSweep recover it (and that
SpeedSweep tries fast-then-slow).
Grade handlers by an exponentially-decayed recovery rate (bytes/second,
alpha=0.5) instead of a cumulative rate. The residual hardens mid-pass, so
the best technique changes: cumulative froze the early winner in the lead
forever; the EWMA re-prices continuously — a handler that stops earning
decays down, a late-starting specialist climbs. Keeps rank()'s untried->top
one-shot calibration and attempted-but-zero-time->bottom.
New flip fixture proves a handler that recovers a lot early then nothing
loses its lead to one that starts recovering later.
Add ReadParams { speed: SpeedPref, fua: bool, timeout: TimeoutPref } and
thread it through read_span so every wedge-safe handler read can request a
spindle speed (SET CD SPEED issued only on change, restored to max when the
handler exits), set the READ(10) FUA bit, and pick the 10s vs 60s timeout.
- SectorSource gains read_sectors_fua (default ignores fua); Drive sets the
CDB bit, DecryptingSectorSource threads fua to its inner read.
- recovery_read gains a fua param.
- Linear becomes { direction, params }; Bisect/Jump take params. Existing
tier-0/1 instances keep identical behavior (max speed, no FUA, fast/deep).
- Scoreboard keys on the full-config String name (linear:fwd:max:fast, ...).
- FakeDisc observes speed + FUA + approach so specialist techniques are
provably exercised in later commits.
cargo test -p libfreemkv green (2193 passed).
The private_stream_1 sub-stream id low nibble is the DVD audio-stream
NUMBER (0-7), shared across codecs, not a per-codec ordinal. A DTS (or
LPCM) track that isn't the disc's first audio stream got a sub-id one
too low, so the demux routing key (0xBD00 | sub_id) never matched and
every packet was dropped -> present-but-silent track. AC-3 at position
0 coincidentally worked (ordinal==position). Route by positional index.
Fixes silent DTS 5.0 track on The Punisher (AC-3 5.1 @ pos0 plays,
DTS 5.0 @ pos1 was 0xBD88, real wire id 0xBD89).
Delete src/verify.rs (standalone 'read every sector, classify health'
module) + its lib.rs pub mod/pub use. It was old, unused code. The
decrypt-verify gate (disc::verify / UnitVerifier) is a DIFFERENT module and
stays. Relocated the pure chapter_at_offset helper (used by locate_ranges +
autorip's done-card) into disc/mod.rs as a pub fn.
- Live located drilldown (sweep + patch progress snapshots) now excludes
NonTried, matching the one-shot path. Including the unread remainder made
main_at_risk_ms show ~full-movie at sweep start and melt to 0 as it
progressed — unread is unknown, not damage.
- Wedge abort now requires the failure to be FAST (< WEDGE_FASTFAIL_MS,
500ms) as well as wedge-family sense: a real uncorrectable sector on
Hardware-error media spends ECC-recovery time before failing, so it no
longer false-trips the fast-fail wedge abort. New regression test.
- Removed the always-zero blocks_attempted/read_ok/read_failed/
unreadable_count from PatchOutcome + PatchLoopState (dead residue from
the old grind loop; the HandlerScoreboard supersedes them) so the
patch.done log no longer emits misleading zeros.
yes now means the unlocker actually ran and did its job: LibreDrive from
the runtime firmware-unlock success, AACS host-cert only when LibreDrive
didn't do the bus (stock-drive fallback), CSS from the crack succeeding.
On a LibreDrive UHD that correctly reads LibreDrive: yes, AACS: no (LD
removed the bus, the cert route never ran) instead of the misleading
AACS: yes. Names stay registry-driven; runtime logic lives here.
Returns each registered unlocker's name + whether it applies to this
drive+disc (via the unlock bridge over all_unlockers()). Disc crypto kind
derived in the library so the CLI and autorip render an identical, always-
current report with no hardcoded names. Also adds a Drive::unlocker_name()
runtime getter.
Handlers (section_recover.rs):
- Bisect expand loops now honor ctx.halted() (were deadline-only, so a
Stop could hang up to 60s vacuuming a readable island).
- read_span: explicit Transport arm so a bus-abort read isn't counted as
unproductive grinding; debug_assert the sector-aligned span invariant.
- Scoreboard rank: an attempted-but-zero-time handler (e.g. returned Halted
on its first check) now ranks BOTTOM, not top — it no longer crowds out
proven performers.
- Document the wedge tier-size coupling + new regression test that a
2-handler (tier-1) chain still catches a wedge via cross-section streak.
Pass-N engine (patch.rs):
- Rebuild PatchOutcome stats AFTER the post-read re-verify downgrade flush
(was snapshotting before it, over-reporting bytes_good / recovered and
risking a 'perfect rip' verdict on an imperfect one).
- Progress 'recovered' composes the still-bad set to MATCH work_total
(subtract NonTried, add Unreadable) so the bar can't pin at 0 on a
partially-swept disc or run backward on the Unreadable→NonTrimmed relabel.
- Remove dead work_done field; rewrite the stale 'adaptive batching' comment
to describe the handler chain and mark block_sectors/full_recovery as
informational-only.
Sweep (disc/mod.rs):
- Saturating arithmetic at the damage-jump position math (honor the
read_error side's documented defence-in-depth guarantee).
Deferred (noted, need focused passes): fast_capture re-introduction,
Pass-1 halt-misclassified-as-jump, bytes_good display inflation, the
always-zero blocks_* telemetry, Pass-1 jump-on-first-error policy.
The 2026-07-01 overnight rip ground a wedged BU40N for 28 minutes at
0 B/s. Root cause: the handler chain's read_span split errors only into
Transport (bus gone) vs Bad (everything else), so the drive's fast-fail
wedge sense (ILLEGAL REQUEST 0x05/0x24 — it rejects every CDB in <100ms
without attempting recovery) was treated as an ordinary bad sector. The
chain hopped to the next section forever, and the existing wedge detector
in read_error.rs never saw it because the chain reads through
recovery_read directly.
Add a pass-level wedge streak: read_span counts consecutive wedge-family
(Hardware/IllegalRequest) senses; at WEDGE_ABORT_STREAK (16) it escalates
the read to Transport, which every handler already propagates as
TransportFault — aborting the whole pass and setting wedged_exit so the
caller spin-cycles. The streak is carried across sections via PatchCtx
(seeded into and read back from each per-section HandlerCtx) so a wedge is
caught even when every bad sub-range is smaller than the streak. Any Good
or non-wedge (medium-error) read resets it, so scattered bad sectors on
real media never trip it. New fixture test: a wholly-wedged 1000-sector
section aborts in <100 reads, not 1000.
The expand loops broke out of the batch sweep on the first failing read,
overshooting the good/dead boundary by up to a batch and leaving the
readable sectors adjacent to a dead edge for the re-bisect to re-pin. On
top of that, early-yield counted those boundary-probe failures as a
stall, so the forward expand quit early AND the backward expand inherited
a poisoned unproductive counter and yielded instantly.
Now on a failed expand batch we halve the step (down to a single sector)
to recover right up to the dead edge in ~log2(batch) reads, drive the
expand loops off the deadline only (they self-terminate via halving), and
clear the unproductive streak once an island is located. patch_recovers_
good_middle_of_a_bad_range now recovers 50/50 good-middle sectors.
Scorecard: grade each handler by recovery rate (MB/s) per rip, order
best-first on later sections, log the ranking at pass end. Untried
handlers rank top so each is calibrated once before ranking narrows to
the winners. Ephemeral (reset per pass), no persistence.
Tier 0 scouts are now [Bisect, Jump, Linear-fast x2], scorecard-ordered.
Bisect leads: probing the MIDDLE of a range lands on a readable island in
one read where a linear scan grinds the dead front to reach it. Jump now
jumps to the middle of the REMAINING span (proportional) instead of a
fixed 8 MiB that leapt clean over small ranges and missed their readable
middles. Tier 1 is slow deep-recovery on the residue.
Scorecard (ephemeral, reset each pass, no persistence): grades every
handler by recovery rate (bytes/sec). run_handlers orders handlers
best-first by that rate; an untried handler ranks top so each is
calibrated once, then the ranking narrows to the winners. Logged at pass
end (phase=scorecard) so the operator sees which handler is pulling the
weight on this drive/disc and which is a dud.
Tier 0 scouts are now [Bisect, Jump, Linear-fast x2], scorecard-ordered.
Bisect leads because probing a range's MIDDLE lands on a readable island
in one read, where Jump (linear from the front, big skip) can grind the
dead front or overshoot a small range entirely. The scorecard confirms
or overturns that order with real per-disc data.
Each dead probe read costs the drive's full ~10s timeout, so a large dead
region took a dozen escalating probes. Starting the jump at 8 MiB (vs
1 MiB) clears it in a handful; a skipped span stays bad for Bisect to
reclaim readable islands, so an over-jump loses nothing.
Tier 0 is now a single fast Jump scout: it streams the big readable
ranges back and skips dead runs in seconds, so the pass reaches every
section fast and converges to the small genuine-dead residue instead of
grinding three handlers x 60s on each dead fragment. Tier 1 (fast
mop-up + slow deep reads + Bisect) works only that residue.
Bisect now expands: on a good probe it reads outward forward and
backward in full batches until a read fails, recovering the whole
readable island in large reads; the two failing ends become smaller bad
sub-ranges it bisects again. One huge bad range becomes many precisely
located small dead clusters.
Progress heartbeat: HandlerCtx gains a throttled tick (250ms) called
from every read, pushing a fresh snapshot to the reporter DURING a
handler. The bar and speed now move continuously as recovery happens
instead of jumping once per section (the reason speed read 0 B/s and the
% looked frozen between range boundaries).
Jump: on sustained batch failures skip ahead an escalating distance
(1 MiB doubling to 256 MiB) to find where readable data resumes, leaving
the skipped span for Bisect to pin — mirrors the Pass-1 damage-jump. It
leads the fast tier so a large dead run is skipped in seconds instead of
the linear sweeps grinding every dead batch (10 s each) first; on a
readable range it just streams it back. Recovers readable data buried
behind a big dead front (the 192 MB Dune range).
Progress %: report bytes RECOVERED (initial-bad minus still-pending)
instead of a per-range counter that only advanced on the final tier — so
the bar reflects the readable bulk recovered during tier 0 the instant it
lands, matching the 'MB remaining' number.
The per-range walk was depth-first: each bad range ran the full handler
chain (fast + slow deep-recovery + bisect) before the next range was
touched. So a handful of tiny dead fragments at one end of the disc
burned the whole pass and the big NonTrimmed ranges elsewhere — usually
sweep-jump over-marks that read straight back — were never attempted.
Now recovery runs in two breadth-first tiers over ALL sections:
- Tier 0 gives every section one fast full-batch attempt (fast reads
only), largest ranges first, so the recoverable bulk of the disc comes
back in the first minutes.
- Tier 1 deep-recovers only the residue tier 0 could not pull.
Per-section still-bad sets persist across tiers. Largest-first ordering
means a big readable region is reached before time is spent on tiny dead
fragments.
Linear no longer collapses a failed batch to count=1 single-sector reads
(live probing: a marginal sector recovers in a large read, not a lone
one) — a failed 32-batch stays 32 and is re-attempted at full size by the
next handler/pass; Bisect salvages readable islands.
Adds a handler-start trace line so the debug log shows which handler is
running and the hand-off to the next.
Pass-N recovery is now a chain of time-bounded recovery handlers instead
of one monolithic per-range loop that could grind the front of a bad
range for up to 30 min, wedge the drive, and abort the whole pass.
A bad range is a SubRanges set; recovery is an ordered list of
SectionHandlers (Linear{reverse,fast} covering back/forward x fast/slow,
and Bisect). A coordinator runs each handler with a hard per-handler
deadline: a handler recovers what it can (removing it from the still-bad
set) and hands the rest to the next handler; whatever is still bad after
the chain becomes NonTrimmed residue and we move on to the next range.
Guarantees, now structural rather than bolted-on:
- never hangs: every handler is deadline-bounded; the loop always drains
to recovered-or-residue.
- always moves on: a range that cannot be finished leaves residue and
advances; only a genuine transport fault or user halt ends the pass.
- extensible: a new recovery idea is one SectionHandler impl added to the
chain; a proven-ineffective one is removed. The engine never changes.
Removes ~1.9k lines of the old inner loop (watchdogs, skip escalation,
NOT_READY grind, wedge counters) and their tests. fast_capture is now
inert (the chain supersedes it); breadth-first ordering becomes a future
scheduler concern. New module: disc/section_recover.rs (8 fixture tests,
injectable clock — bounded/never-hang proven without touching a drive).
Two A/B tests updated to the chain's strictly-better recovery counts.
A fast_capture pass defers every failed block to a granular pass anyway, so
spending the drive's 60s deep-recovery timeout on it here only freezes the
breadth-first sweep on a pure-bad cluster (~25s per 32-block, incl. NOT_READY
grind). Read with recovery=false in fast_capture so a bad block fails on the
short timeout (~10s) and the sweep steps on; the granular passes
(fast_capture=false) do the deep recovery on what's left. No recovery lost —
the block stays NonTrimmed for the granular retry.
A PatchOptions.fast_capture pass reads each bad range ONCE at the full batch and
leaves every FAILED block NonTrimmed for a later pass — no bisect, no per-sector
grind, no retry. This lets a first retry pass grab the readable blocks (the
sweep's good skip-ahead overshoot) of EVERY section quickly, before any single
section's slow per-sector recovery — instead of grinding section 1 to exhaustion
before even touching section 2. A later pass (fast_capture = false) does the
granular bisect/retry on what's left.
Load-bearing invariant (fixture test): NO data is dropped. A failed block becomes
NonTrimmed (pending, retried by a granular pass), NEVER Unreadable. The test pins
that the readable half of a range recovers, the bad half stays NonTrimmed (not
Unreadable), and the bad block is marked in ONE batch read with zero bisection.
Disc::copy's internal patch keeps fast_capture=false (single-call full recovery).
The library now offers a one-shot PassProgress snapshot built from a mapfile on
disk + title, so a client gets the fully-rendered drilldown without parsing the
mapfile itself — used for autorip's pass-boundary and mux-entry paints (no live
callback yet at those points). Adds a locate_ranges test covering in-feature vs
out-of-feature at-risk time (ported from autorip's removed from_map tests).
Add LocatedRange + LocatedProgress to the progress contract and a 'located'
field on PassProgress, populated by the sweep + patch emitters from the
in-memory bad-range set + title. Move the range->chapter/time annotation
(locate_ranges, range_chapter, byte_offset_in_title) into the library so a
client renders the disc map + at-risk movie time straight from PassProgress
and never reads the mapfile itself — if the mapfile becomes a mapdb, this type
and its producer change, clients don't.
PassProgress is no longer Copy (located carries a Vec); it's built once per
throttled emission and passed by reference. Non-locating phases (verify,
extract) emit LocatedProgress::default(). Adds consts::MILLIS_PER_SEC.
Consumer-side wiring (autorip drops Mapfile::load) follows.
Live drive probing (cold-single vs in-stream batch vs centered window,
one clean uncached read per sector) showed recovery RATE is identical
across approach: 3/5 every mode. The drive's per-sector ECC is
media-bound and partly stochastic, NOT approach-bound. So scatter
(seek-away recalibration + far-anchor re-read) does not improve recovery
and is dead weight — removed (function, read_good_sectors, SCATTER_*
consts, and its tests).
Add SubRanges: the still-bad (pos,len) sub-ranges of a section with
remove()/split, the work-list foundation for the upcoming per-section
recovery decomposition (recover_section -> stream/bisect/retry phase
helpers). Unit-tested; unused until the decomposition lands.
25 patch + 8 AB profile tests green; precommit clean on Rust 1.86.
Live probing on the BU40N/UHD testbed (rprobe/dprobe) settled the speed
question empirically: a marginal sector reads ~12x FASTER at MAX speed
than at slow, and slow NEVER recovered one MAX didn't. Cleanly-tested
(one uncached read per sector) the recovery RATE is identical across
speed/direction/window/batch — the drive's per-sector ECC is media-bound,
not approach-bound. The only host lever that matters is read SPEED.
So the old 'drop to 0x0000 on first failure and hold it for the whole
range' only slowed the GOOD sectors of a bad range — measured ~3x slower
overall. Now the range stays at MAX; the first failure just re-attempts
once (stochastic media) and falls through. Live A/B: 11 KB/s -> 33 KB/s,
bytes_lost unchanged.
Also: batch adaptation is now halve-on-failure / double-on-success
(geometric), so a bisected batch climbs back through clean runs from any
size (the old count==1-only upscale left it stuck at mid sizes). Removed
the dead consecutive_singles_ok counter and the inert batch-retry probe.
25 patch tests green; precommit clean on Rust 1.86.
Two patch-recovery speed fixes, grounded in live rip1 telemetry on the
Dune UHD bad regions.
1. Pull prime_cache. In the reverse walk it issued throwaway reads of
lba-3..lba-1 — i.e. straight into the not-yet-visited dead L-EC core —
grinding each ~7-9s at the fast timeout, then discarding the result.
The live trace showed a good target read (read_duration_ms=1) taking
9s wall-clock purely from the prime grind ahead of it. The dead-zone
skip (Tier 1) + scatter's recalibration read (read_good_sectors)
cover the cache-warm intent without grinding the dead zone, so prime
is redundant harm — removed (function + CACHE_PRIME_SECTORS + call).
2. Bisect on batch failure. handle_read_failure used to collapse a
failed count>1 batch straight to count=1 and single-walk the entire
batch. Now it halves (count/2) and retries the same start: a good
half recovers in BULK, the bad sector is isolated in O(log n) reads
instead of O(n). Invariant preserved — only a count==1 failure marks
NonTrimmed, so no good sector is lumped into a bad mark.
These are Tier 2 (bisect) of the progressive recovery model
(fast-mass-skip -> bisect -> scatter); composable so order can be tuned.
25 lib + 8 AB fixture tests green; precommit clean on Rust 1.86.
The 'reset, read good data, come back for one sector' technique. A
genuinely-damaged sector makes the drive grind its full recovery timeout
and still fail; re-reading consecutive bad LBAs at identical conditions
re-fails AND is the rapid-failure cadence that drops the BU40N into a
firmware fast-fail wedge (CLAUDE.md hard-rule #2).
scatter_recover: on a stuck single sector, make up to SCATTER_MAX_ATTEMPTS
fresh tries, each preceded by read_good_sectors() — a seek to a far
known-good anchor + a sustained read that re-seats the head/servo and
breaks the failure cadence. The fresh re-read uses the FAST timeout
(recovery=false) so a recalibrated marginal sector reads quick and a
truly-dead one fails fast instead of burning 60s per attempt. The seek +
good read IS the settle (matches ddrescue/MakeMKV) — no idle sleep.
Gated to genuine single-sector MEDIUM_ERROR (0x03): transport faults
still abort the pass, NOT_READY keeps its own retry path, wedge-family
senses are left for cooldown/eject. A scatter recovery is recorded
exactly like a normal read success (write-then-mark-Finished); nothing
marks a range good without a successful read + write.
Logs anchor_ms/reread_ms per attempt for live tuning. 5 fixture tests
(recover-marginal, give-up-dead, skip-non-medium, skip-batch, AACS
widen); 2 AB read-count guards raised for scatter's bounded extra reads.
Precommit green on Rust 1.86.
Pure structural refactor of the Pass-N recovery path, no behavior change.
- PatchCtx<R> holds the one-pass context (reader, pipe, shared, opts,
reused read buffer, cross-region state) and owns orchestration.
- PatchCtx::run() walks the ordered bad ranges, applies the inter-range
cooldown (armed on a grind, consumed at the next range's entry), and
stops the pass the moment a range reports halt/wedge/transport-fault.
- PatchCtx::patch_region() is the focused single-range loop: read ->
success/failure -> damage-skip -> per-range watchdog. Every former
break / break 'outer is now a typed RegionOutcome (Completed,
SkipLimit, BudgetExceeded, Wedged, Halted, TransportFault) that run()
maps back to next-range vs stop-pass exactly as before.
- recovery_read() extracts the AACS mid-unit-window widening + plain
read into one reusable helper (the building block the upcoming
scatter re-read will share).
- Logging unified onto a single dotted patch.* taxonomy (patch.region.*,
patch.read.*, patch.probe.*, patch.wedge.*, patch.batch.*,
patch.mapfile.*, patch.skip.*, patch.stall, patch.speed); the
freemkv::scan pass-boundary markers are left intact.
20/20 disc::patch tests green; precommit (fmt + clippy -D warnings +
tests) clean on Rust 1.86.
Audit fixes (v1.1.0..HEAD regressions in the unlock migration + adaptive patch
speed):
- unlock_bridge ScsiAdapter: libfreemkv's transport returns Err on ANY non-zero
SCSI status (a normal CHECK CONDITION), not only transport faults. The adapter
was collapsing every such Err to { status: 0xFF, sense: None }, which discarded
the parsed sense and defeated the AACS handshake's ILLEGAL_REQUEST wedge guard
(so it kept hammering the drive — hard-rule #2) and inverted its
transport-vs-rejection diagnosis. Now reconstruct status + the 32-byte sense
buffer (sense_key@2, asc@12, ascq@13) and only emit 0xFF/None for a genuine
transport fault.
- Drive::init: a genuine transport fault during the drive-prep unlock means the
bus is dead — propagate it (the v1.1.0 invariant) instead of silently
swallowing it via `if let Ok`. Other errors (no matching unlocker) still fall
through to stock mode. SET CD SPEED max now runs only when the bus is alive.
- disc::patch: on the first read failure in a range, drop to slow recovery speed
and RE-ATTEMPT the same position at slow speed before marking it. A
single-sector range's first failing sector was being marked from a MAX-speed
read it never got to recover.
- docs: lib.rs architecture diagram (handshake → host_certs) and README (stale
pluggable-unlock-seam / register-unlocker / crates.io / docs.rs references).
Two recovery-path fixes:
- patch: log the per-range speed transitions (INFO, phase="patch_speed") — each
range enters at 0xFFFF (max) and drops to 0x0000 (slow recovery) on its first
read failure. Previously the adaptive-speed behavior was invisible in the logs.
- drive init: issue the generic SET CD SPEED max UNCONDITIONALLY at drive-open,
not only when a firmware unlocker matched. A stock-mode BD/UHD drive (no
firmware unlock) was left riplocked because the call sat inside the
unlocker-matched branch.
Rewire the three unlock dispatch points through the freemkv-unlock crate via a
private `unlock_bridge`: drive-prep (kind=Unknown) at `Drive::init`, AACS cert
(kind=Aacs) at `do_handshake_cert`, CSS bus-auth (kind=Css) at scan. The bridge
news up `all_unlockers()` and runs the first matching one, mapping its
`Unlocked` result to the bus-key gate. After a successful drive unlock,
libfreemkv issues a generic SET CD SPEED (max) itself — the old per-unlocker
trait method is gone.
Delete the in-tree unlock code now owned by freemkv-unlock: the AACS cert
handshake (`aacs/handshake.rs`), the CSS bus-auth (`css/auth.rs`), and the
unlock registry (`unlock.rs`). Host-cert collection (a keysource concern) stays
in a small `aacs/host_certs.rs`. No public unlock surface remains — clients
touch libfreemkv only, oblivious to unlockers (as they are to SCSI). 2277 tests
pass.
Pull the wedge-guarded cert loop and host-cert collection out of the in-tree
AacsCertUnlocker into public aacs::handshake primitives (run_cert_handshake +
CertHandshake, collect_host_certs). The in-tree path now delegates to them, so
the external freemkv-unlock-aacs plugin runs the IDENTICAL cert handshake — one
implementation, two callers. Pure refactor of the live AACS path; the existing
handshake + collect_host_certs unit tests validate it unchanged.
Matthew Jackson