Commit Graph
100 Commits
Author SHA1 Message Date
Matthew Jackson 129c34b002 wip(3d): read-only 3D-structure probe (SSIF/m2ts + MPLS ExtensionData)
Dev tool for the 3D build: uses the library UDF reader (7z cannot parse these
ISOs) to dump base .m2ts vs SSIF sizes and the MPLS ExtensionData entries
(STN_table_SS + SS sub-path). Not product code.
2026-07-12 20:38:37 -07:00
Matthew Jackson ebf30a679e fix(udf): follow allocation-descriptor continuation past the AED header
read_icb_extents parsed a continuation block's allocation descriptors from
offset 0, but a continuation block begins with a 24-byte Allocation Extent
Descriptor (ECMA-167 4/14.5): 16-byte descriptor tag + prev-loc(4) +
length_of_allocation_descriptors(4); the real ADs start at offset 24. Reading
from 0 mis-parsed the AED tag as a descriptor -> one garbage extent, then an
unknown-type break, silently truncating any file whose extents spill into a
continuation block.

Few-extent files store their ADs inline and never hit this path, so it stayed
hidden; a heavily fragmented file (~1600 fragments) truncated to 113 extents +
a bogus non-unit-aligned fragment that then aborted the mux
(ExtentNotUnitAligned). Skip the 24-byte AED header, read l_ad bytes of
descriptors from offset 24. Adds a regression test with a non-zero AED tag.
2026-07-12 20:37:55 -07:00
Matthew Jackson 09a8dd183d restore freemkv-unlock path dep for local dev (post-v1.3.2) 2026-07-10 14:26:11 -07:00
Matthew Jackson 057c878831 v1.3.2: bump version (freemkv-unlock git-pinned for the tag) 2026-07-10 14:26:08 -07:00
Matthew Jackson 43cbfa07f5 1.3.2: AACS 2.1 FMTS variant-decode foundation
Add UnitKey.variant_number (0 = ordinary, 1..32 = forensic variant) with new/variant constructors, and aacs::variant_select — resolve a disc's single variant and classify each aligned unit (default / variant / drop foreign / conceal keyless). Correct IndividualSegment.tbl: the per-record field is the variant (cycles 1..32 on a retail disc), not a segment number — Segment.number -> Segment.variant.
2026-07-10 14:00:28 -07:00
Matthew Jackson 0e0967795e changelog: trim the 1.3.1 relicense note to just the relicense fact 2026-07-10 12:53:17 -07:00
Matthew Jackson 68b5372415 restore freemkv-unlock path dep for local dev (post-v1.3.1) 2026-07-10 12:35:12 -07:00
Matthew Jackson 9e8f196b20 v1.3.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-10 12:35:09 -07:00
Matthew Jackson b8f0af9ef5 1.3.1: relicense to MIT (clean-room CSS + drop copyleft-lib references)
Relicensed from AGPL-3.0 to MIT, effective 1.3.1 (<=1.3.0 remain AGPL). The CSS
content cipher and Stevenson title-key attack are attributed to their published
cryptanalysis (not libdvdcss); all libaacs/libbluray/libdvdread/libdvdnav name
references were dropped from comments while keeping the standard format/spec
descriptions. Also bumps to 1.3.1.
2026-07-10 12:31:19 -07:00
Matthew Jackson 24e2bc33cf hddvd: authoritative title composition from the Advanced-Content playlist
Parse ADV_OBJ/VPLST000.XPL (the real HD-DVD player playlist) with roxmltree
into one DiscTitle per <Title>: its PrimaryAudioVideoClip clips in order (EVO
via the .MAP sidecar), the titleDuration, the displayName, and the ChapterList.
A layer-break split (FEATURE_1+FEATURE_2, feature/feature_Divide) composes into
ONE title with the two parts as clips, each carrying its title-time in/out
points (45kHz ticks) for seamless-join splicing. Falls back to the clip-name
heuristic when no playlist is present.

Fixes unknown runtimes (real durations), poor names (FEATURE -> 'Main Movie'),
and gives authoritative composition + chapters. Validated on ANCHORMAN (97m),
SHAUN (99m), HARRY_POTTER (152m).
2026-07-10 10:28:18 -07:00
Matthew Jackson a7df91b92b ci: drop the dead crates.io publish job (libfreemkv is git-tag-only)
libfreemkv is publish=false (it git-deps the firmware crate freemkv-unlock,
never on crates.io), so the Release workflow's 'cargo publish' step failed
hard on every tag. Consumers git-tag-pin libfreemkv; the git tag is the
release artifact. Removed the publish job.
2026-07-10 08:52:57 -07:00
Matthew Jackson 0eb0188ba7 restore freemkv-unlock path dep for local dev (post-v1.3.0) 2026-07-10 08:43:47 -07:00
Matthew Jackson e2f595d558 v1.3.0: bump version (freemkv-unlock git-pinned for the tag) 2026-07-10 08:43:42 -07:00
Matthew Jackson 18082d0df1 audit: void empty-timeline duration, cover sniff overlap
Round-11 findings from the 10-phase release audit (no real HIGH):

- When a no-declared-duration source (HD-DVD) muxes a degenerate single
  frame at tick 0 with no per-frame duration, max_block_ticks stays 0 and
  the reserved DURATION placeholder was left as a literal 0.0 (players read
  that as a zero-length file). Void the element instead, so the Segment
  omits DURATION as an unknown-duration source did before the back-patch.
- Add a regression test for the sniff_video_codec overlap fix (a
  picture_start_code whose payload begins 00 00 followed by a real start
  code) so the i+=4 marker skip can't silently regress to i+=3.
2026-07-09 20:40:54 -07:00
Matthew Jackson 640502d5a8 audit: lock DTS rate table, fix sniff overflow-scan, cover decrypt loss
Round-10 findings from the 10-phase release audit:

- A finder claimed the DTS SFREQ→rate table was wrong at 11/12; verified
  it against ffmpeg's avpriv_dca_sample_rates (12k/24k/48k/96k/192k at
  11-15) — the table is CORRECT. Added a test that locks the full table so
  it can't be mis-"fixed".
- sniff_video_codec advanced 3 bytes after a matched start code, re-reading
  the code byte as an overlapping start code; skip the full 4-byte marker.
- Guard the HD-DVD next_id title counter with saturating_add so a crafted
  disc with >65536 clips can't overflow (panic in debug).
- Add a test that an undecryptable unit (DecryptFailed) is zero-filled and
  counted as loss through ExtractResult (complete=false, bytes_lost>0) —
  the recovery-seam consolidation folded that bucket into bytes_unreadable.
2026-07-09 20:17:12 -07:00
Matthew Jackson 270f9d88b3 audit: drop dead DTS marks cap, lazy passthrough buf, doc corrections
Round-9 findings from the 10-phase release audit (no HIGH):

- Remove the MAX_PTS_MARKS backstop and its tautological test: an empty
  DTS PES returns before recording a mark, and a non-empty run is already
  bounded by the MAX_AU_BYTES buffer clear (which clears pts_marks) — so
  the deque cannot grow unbounded and the cap was dead code.
- AuAssembler::for_codec no longer reserves 256 KiB for a Passthrough
  stream (audio/subtitle, and every TS/BD stream) whose buf is never
  written; only the reassembling modes reserve.
- Correct the scan comment that claimed region is computed (it is a
  Region-free stub until region detection lands) and drop a public-repo
  reference to internal "private refactor notes" in the mkb module doc.
2026-07-09 19:50:54 -07:00
Matthew Jackson 6a0e61d415 audit: clamp BD format fallback, running GOP byte counter, O(1) DTS marks
Round-8 findings from the 10-phase release audit:

- detect_disc_format's BDMV fallback passed detect_format's result
  through unchanged, so an SD bonus/menu title could tag a BD-tree disc as
  DVD (mis-sizing the ECC sweep) — violating its own "never below Blu-ray"
  invariant. Clamp anything but UHD up to Blu-ray.
- Track the MPEG-2 GOP byte total incrementally instead of re-summing the
  whole gop_buf on every pushed picture (was O(pictures²) on any MPEG-2
  disc, not just adversarial input).
- Back the DTS pts_marks deque with a VecDeque so the over-cap prune is an
  O(1) pop_front, not an O(n) Vec::remove(0).
- Add a test exercising parse_stream_id_extension's PTS/DTS skip branches
  (the real AU-opening 0xFD video PES path) — previously untested.
2026-07-09 19:27:01 -07:00
Matthew Jackson 92e3b41468 audit: bound DTS marks, align disc-format tree order, doc/test cleanups
Round-7 findings from the 10-phase release audit (no HIGH; convergence):

- Cap DtsParser.pts_marks (MAX_PTS_MARKS): a run of zero-length timed PES
  packets grew no buffer bytes, so the drain_front mark-prune never ran —
  the deque could accumulate without bound on hostile PS input.
- detect_disc_format tested HVDVD_TS before BDMV while the title-scan
  dispatch tests BDMV first, so a disc with both trees would be classified
  HD-DVD but enumerated as Blu-ray. Align both to BDMV → HVDVD_TS →
  VIDEO_TS.
- Document why the DTS new-PES re-base can emit a locally-decreasing PTS
  (the muxer's block_ts applies the strictly-monotonic audio nudge, tested
  in mkv.rs) — this is by design, not a mux defect.
- Fix stale aacs/keys.rs comment references (functions moved to
  aacs/inf.rs / aacs::resolve/derive in the module split).
2026-07-09 19:01:56 -07:00
Matthew Jackson 7d852419b5 audit: byte caps on GOP buffers, opener-scan resume, honest video codec
Round-6 findings from the 10-phase release audit:

- Wire the documented MAX_PENDING_BYTES byte cap into the MPEG-2 GOP
  buffer (it was dead code) and add an equivalent MAX_GOP_BYTES cap to the
  sparse-PTS reorder, so a crafted stream of few-but-huge access units
  cannot over-allocate — both were bounded only by frame count before.
- probe_evo_streams defaulted an unsniffable HD-DVD video stream to H.264,
  which mis-parses a VC-1 (or still-encrypted) clip into a corrupt track.
  Emit the video stream only when the codec is actually identified — the
  honest outcome, matching the audio path (a real clear clip always carries
  its sequence header at the head).
- Resume the AU-opener search from a cursor (like the boundary search), so
  a long unsynced junk run is O(bytes), not O(buffer) per push.
- Mark mpeg2's now-dead MAX_AU_BUFFER test-only; restore #[doc(hidden)] on
  the aacs probe harness module.
- Add regression tests: the 0xFD video-routing guard, the FMTS-is-UHD key
  state, and the GOP byte caps.
2026-07-09 18:31:47 -07:00
Matthew Jackson 9066433c29 audit: guard 0xFD video routing, carry frame duration, add cap tests
Round-5 findings from the 10-phase release audit:

- collect_es routed EVERY extended-stream-id (0xFD) PES into the video ES
  buffer, so a 0xFD HD-audio sub-stream (MLP/TrueHD) could pollute the
  video sample and — if it preceded the video PES — stamp the video track
  with the audio PID, losing the video. Only the VC-1 extension (0x55) is
  now treated as video; routing 0xFD audio to its own track is deferred to
  the HD-DVD program-chain follow-up.
- The sparse-PTS reorder now carries its calibrated per-frame duration onto
  each frame, so the muxer emits a BlockDuration and the back-patched
  Segment Duration covers the final frame instead of understating it.
- Add regression tests for the MAX_MARKS and MAX_VTI_HITS caps (promote
  MAX_VTI_HITS to module scope); make the differential-test factory array a
  named type; drop an identity-op in a reorder test.
2026-07-09 17:59:04 -07:00
Matthew Jackson c81a6e05cd audit: fix AU mark-field loss, VTI tie determinism, and mark/perf issues
Round-4 findings from the 10-phase release audit (the first fully clean
round; it dug into the new #22/#18 refactor code):

- AuAssembler closed each AU from only the FRONT mark's fields, so when
  one PES fragment carried the source and a later fragment of the same AU
  carried the PTS, the second field was dropped — a regression vs the old
  separate pts/source mark deques. Now merge the first Some of each field
  across all in-range marks.
- parse_vti_clip_order picked the largest residue bucket with
  HashMap::into_values().max_by_key(), nondeterministic on a size tie
  (randomized HashMap iteration) — could select a different clip table
  run-to-run. Break ties by smallest offset.
- Bound the marks/disc_marks deques (MAX_MARKS): the buf-size cap prunes
  marks only when bytes accumulate, so a run of zero-length timed
  fragments could grow them without bound on hostile input.
- Add push_owned so the PS path moves the PES payload into a passthrough
  AU with no copy (MPEG-2 video + all audio), removing a per-PES
  malloc+memcpy the refactor had introduced on the DVD path.
- Back-patch the MKV duration from the block END (start + its own
  duration) so it covers the final frame instead of understating by one.
- Add direct tests for the MKB record-framing walker; drop a stale
  drain_complete_aus doc comment left on process_au.
2026-07-09 17:30:41 -07:00
Matthew Jackson 0a9bdf08f6 docs: drop Renesas from changelog (detection-only stub) 2026-07-09 16:36:47 -07:00
Matthew Jackson 2b74a9b21f docs: soften HD-DVD changelog to partial support (PGC parse deferred) 2026-07-09 16:33:40 -07:00
Matthew Jackson b5a5138569 mux: resume AU-boundary scans from a cursor (O(n) not O(n²))
AuAssembler::drain rescanned the whole buffered access unit from a fixed
offset on every push, so reassembling one AU split across N program-stream
PES fragments cost O(bytes²/fragment) — amplified on the HD-DVD PS path
where H.264/HEVC/VC-1 frames are large and now flow through this shared
assembler (unlike the TS path, which delivers one AU per PES).

Carry a scan_pos cursor (and, for the stateful VC-1/MPEG-2 rules, a
seen_unit flag) so each push resumes the boundary search where the last
one stopped instead of restarting. Total scan work for one AU is now
O(AU bytes). The from-scratch scanners are retained as a #[cfg(test)]
oracle; a new differential test asserts the resumable path yields
byte-identical AUs at every fragment granularity for all three modes.
2026-07-09 16:22:54 -07:00
Matthew Jackson 26423187d3 audit: bound the VTI clip-table scan; fix stale aacs doc links
Round-2 findings from the 10-phase release audit:

- parse_vti_clip_order bucketed hits by residue with an O(stride*hits)
  rescan and no hit cap, so a crafted HD-DVD VTI packed with millions of
  `.EVO` tokens (up to the 64 MiB UDF read cap) could burn seconds of CPU
  on a routine scan. Bucket in a single O(hits) pass and cap collected
  hits at MAX_VTI_HITS (a real table holds a few dozen).
- Fix the stale `super::keys::…` intra-doc links left by the aacs module
  rename: the referenced fns live in `super::derive`.
2026-07-09 14:41:49 -07:00
Matthew Jackson a94f78d090 audit: cap the sparse-PTS reorder buffer, FMTS key state, zero KCD
Round-1 findings from the 10-phase release audit:

- SparsePtsReorder buffered its current GOP with no bound, draining only on
  a keyframe — an open-GOP or crafted program stream that never signals one
  could hold the whole title in RAM. Force-complete the GOP at
  MAX_GOP_FRAMES, matching the MPEG-2 parser's backstop.
- inject_unit_keys labelled a 2.1 FMTS disc as AACS 1.0 / bus-encryption
  off; FMTS is UHD-family, so synthesize the UHD version + bus encryption.
- The compiled Key Correction Data was a non-zero 16-byte constant fed into
  the Media Key derivation. Per the no-compiled-keys rule it is now all-zero;
  the chain still cannot complete on a real disc (documented), so this is
  behaviour-neutral — all variant tests pass unchanged.
- Fix stale doc references (broken `super::variants` intra-doc links, and
  `aacs::keys` comments) left by the module rename.
2026-07-09 14:14:20 -07:00
Matthew Jackson 14c4227292 mux: back-patch the MKV duration from the timeline when the source has none
A title whose scan yields no duration (HD-DVD — its `.MAP` timemaps are
not parsed, so DiscTitle.duration_secs is 0) produced an MKV with no
Segment Duration element, so players/MediaInfo reported an unknown
runtime.

Reserve a DURATION placeholder when the source declares none, track the
highest block timestamp written, and back-patch the placeholder at
finish() with the real muxed runtime (also enabling the per-track BPS
tags for these titles). Gated on duration_secs == 0, so BD/UHD/DVD —
which carry a real mpls/IFO duration — write it up-front exactly as
before, unchanged.
2026-07-09 12:52:48 -07:00
Matthew Jackson fc3e1dd003 docs: changelog for HD-DVD feature composition + MPEG-2 AuAssembler 2026-07-09 09:12:56 -07:00
Matthew Jackson 5090ddab6c disc: compose the HD-DVD feature from its VTI clip table
HD-DVD Standard Content splits the main feature across clips at the
layer break (FEATURE_1/FEATURE_2, or feature/feature_Divide). The scanner
enumerated one title per .evo, so main-title selection picked only part 1
(e.g. Shaun's 11 GB FEATURE_1, missing the 6.8 GB FEATURE_2).

Parse the HVA*.VTI navigation file's clip table — a fixed-stride record
list naming every clip in authored order, isolated by residue-mod-stride
rather than the imprecise header pointer — and concatenate the feature
clips (matched by the feature* naming convention) into one title whose
extents run in authored order. Every other clip stays its own title.
Falls back to one-title-per-clip when the VTI is absent or unparseable,
so nothing regresses on a disc with no readable navigation.

Validated on real discs: Shaun 17.8 GB / Anchorman 20.1 GB / Harry Potter
24.4 GB now enumerate as one 2-clip FEATURE title (largest = the movie).
2026-07-09 09:12:13 -07:00
Matthew Jackson 3633882d6c mux: reassemble MPEG-2 access units via the shared AuAssembler
The MPEG-2 parser hand-rolled its own PES reassembly — a byte buffer plus
parallel PTS / source / discontinuity mark queues keyed by absolute
offset — duplicating what AuAssembler already does for H.264/HEVC/VC-1.

Add a Mode::Mpeg2 to AuAssembler (picture 0x00 with preceding sequence
0xB3 / GOP 0xB8 headers — the same headers-precede-picture shape as the
VC-1 mode) and have the MPEG-2 parser own one via AuAssembler::mpeg2().
parse() now feeds fragments to the assembler and processes each complete
access unit; the buffer, base offset, and three mark queues are gone. The
GOP-buffered temporal_reference reorder and PTS origin-locking are
unchanged. The parser's external contract is unchanged, so all existing
MPEG-2 parser tests pass as-is; new AuAssembler tests cover the MPEG-2
boundary rule directly.
2026-07-09 08:44:01 -07:00
Matthew Jackson ae27a097b9 docs: changelog for HD-DVD VC-1, sparse-PTS, variant layout (1.3.0) 2026-07-09 08:21:09 -07:00
Matthew Jackson 5fdff5664f mux: reconstruct display-order PTS for sparse-PTS program streams
HD-DVD EVO (and DVD VOB) program streams timestamp video at GOP
granularity: only one access unit per GOP carries a PES PTS. The H.264 /
HEVC / VC-1 parsers collapsed a missing PTS to 0, so on such a source
every non-anchor frame landed on the same block timestamp and a decoder
reported "non monotonically increasing dts".

Add a shared SparsePtsReorder that rebuilds a display-order PTS per frame
from the coded picture type (I/P/B) plus the sparse anchor PTS, with a
per-frame duration self-calibrated from the spacing between consecutive
GOP anchors (no external frame-rate needed). Display order is derived via
the classic single-anchor-delay rule (an anchor displays only after the
previously-held anchor; a B displays immediately), exact for the
non-hierarchical GOP structures HD-DVD H.264/VC-1 use. It mirrors the
MPEG-2 parser's GOP-buffered origin-locking.

Gated to the program-stream path only: the three parsers enable it via
with_ps_reorder(is_dvd_ps), so the BD/UHD transport path (per-frame PTS)
is byte-identical and untouched.
2026-07-08 21:54:09 -07:00
Matthew Jackson f8bea78db5 aacs(2.1): pin variant record layout against reference MKBs
Correct the variant Media Key chain against two real variant MKBs
(Zombieland v70, Stand By Me v70): C for Kmp is the per-slot block of
the 0x0c cvalue table indexed by the matched subset-difference, not the
0x2d head. The 0x2d record is the VARIANTS table (leading body-16 bytes,
one big-endian u16 per subset-difference) followed by a trailing 16-byte
Nonce, with no leading header. VKD stays at 0x2f.

Route record-type selection through the named REC_* consts in mkb rather
than bare hex, and document that Key Correction Data is per-licensee: no
universal constant exists and none is compiled in, so on a real disc the
chain yields a wrong Media Key that the Verify-Media-Key gate rejects
rather than emitting a bad key. This is a key-acquisition gap, not a
code gap.
2026-07-08 21:01:57 -07:00
Matthew Jackson 48bec4cc03 mux: HD-DVD VC-1 demux via extended stream id 0xFD
VC-1 HD-DVDs (e.g. Shaun of the Dead) carry video on MPEG-PS extended
stream id 0xFD, with the real stream selector in stream_id_extension
inside the PES extension. Parse that field so the video routes to a
distinct track (pid 0xFD00|ext) instead of being dropped.

Reframe VC-1 access units in AuAssembler with a dedicated Mode::Vc1:
an AU is delimited by the next frame BDU (0x0D) once a frame has already
been seen, so the sequence (0x0F) and entry-point (0x0E) headers that
precede an I-frame stay attached to the frame they describe. The old
single-start-code split stranded those headers on the prior AU, which
the decoder reported as bits-overconsumption and hard decode failures.

hddvd probe now tracks the video pid it detects and emits VC-1 on 0xFD.
2026-07-08 21:01:48 -07:00
Matthew Jackson bfe88d2673 release: prepare 1.3.0
Bump to 1.3.0 and add the 1.3.0 changelog entry (FMTS/HD-DVD formats, AACS 2.1 variant chain, recovery seam, aacs module split, main-title-by-size, and the fixes since v1.2.2). Not tagged or pushed.
2026-07-08 14:48:50 -07:00
Matthew Jackson 0d587d1154 fix: assorted correctness fixes and dead-code cleanup
- aacs/resolve: a media-keys-only provider missing the VID classifies as
  VidUnavailable, not NoMaterial (an MK derives the VUK once the VID
  arrives).
- disc/bluray: mark a clip seen only after its .clpi parses, so a
  transient parse failure on the first PlayItem cannot suppress the
  clip's extents for a later PlayItem referencing it that succeeds.
- disc/patch: log rather than swallow mapfile record/flush failures on a
  reverify downgrade, so a failed persist cannot silently mismark a bad
  unit good on resume.
- mux/ts: flag a discontinuity when a partial PES is dropped, matching
  the other partial-drop paths.
- mux/demux_thread: the no-demuxer branch forwards an empty batch for
  early consumer-disconnect detection instead of reading the whole disc.
- io/pipeline: correct the send-timing log (as_secs_f64, not as_micros
  printed as ms).
- aacs/derive, aacs/variant, disc/read_error, keysource: comment/doc
  accuracy. sector/prefetched, udf: remove dead fields/functions.
- mux/disc: assert unit-aligned read counts in the test.
2026-07-08 14:44:15 -07:00
Matthew Jackson 67aba17173 sector: generic recovery seam; FMTS forensic segments as decrypt loss
Replace the AACS-specific inline key-fetch in the decrypt decorator with
a scheme-neutral recovery seam: the input stream (L3) installs a Recover
closure (none / AACS key-fetch) and the decorator (L2) runs it at the
single decrypt-miss point. FMTS (AACS 2.1) forensic-segment units that no
key opens are just undecryptable units, concealed and counted as ordinary
decrypt loss with no FMTS-specific branch ("a loss is a loss"), so the
separate bytes_undecryptable bucket collapses into one loss count.

- sector/recovery.rs: the seam (MissOutcome, none/key_fetch factories),
  naming no encryption scheme in its type.
- FMTS: segment routing primitives + BYPASS_FMTS_KEY, and an upfront
  ensure_forensic_segments_decryptable gate (Error::FmtsKeyMissing) in
  the mux input path, parallel to the unit-key gate.
- CSS descramble/rekey moves from decrypt_sectors into
  css::descramble_region: CSS self-recovers from the data itself, so it
  stays OFF the seam (which is only for external inputs).
- disc/mod.rs also: main-title selection aligned to largest physical
  size; is_regular read from the open file handle, not metadata(path),
  fixing a swallowed sync_all on a fresh-rip ISO. decrypt_threads()
  resolved once via OnceLock off the per-buffer hot path.
2026-07-08 14:44:03 -07:00
Matthew Jackson 45c12fc5ce labels: reader-backed detection, Criterion fix, menu-language fallback
- DetectFn now takes a SectorSource so a parser can inspect a jar's
  central directory in detect() instead of firing on "any BD-J jar".
  dbp/deluxe do the real com/<vendor>/ prefix check up front, so each
  claims only its own discs (foundational for scaling the registry).
- criterion: treat a stream-map value of 0 as unmapped and synthesize a
  real 1-based number, so a 0 can't shadow or collide with a genuine
  stream 1 (with regression tests).
- png_filenames: new Low-confidence, last-resort parser reading menu
  language from {title}_UHD01_{LANG}_Composite artwork; sits below the
  MPLS floor so a real framework parser always wins.
- vocab: add menu_lang() for 639-2/B to 639-2/T menu-token normalization.
2026-07-08 14:43:52 -07:00
Matthew Jackson 3da8228068 aacs: parse FMTS SegmentKey.tbl container (16-bit variant-indexed)
New aacs::segment_key: parses the AACS 2.1 SegmentKeyNNNNN.tbl container, confirmed against a retail disc as an 8-byte header + 65536 records of 536 bytes, indexed by the 16-bit variant selector the Media Key Variant chain produces. This is the confirmed link between the two 2.1 variant layers (selector picks the device's per-segment variant). The per-record 528-byte payload layout is not yet reversed.
2026-07-07 19:18:33 -07:00
Matthew Jackson 974e886742 aacs: parse FMTS forensic segment map (IndividualSegment.tbl)
New aacs::segment: parses the AACS 2.1 IndividualSegment.tbl into the source-packet ranges of the forensic variant segments (validated against a retail disc: 792 segments, 2560 packets each). First piece of the FMTS variant decoder — the segments' variants are encrypted under segment keys, not the unit key, so a unit-key rip corrupts them (broken HEVC refs).
2026-07-07 18:59:09 -07:00
Matthew Jackson 1f3f52d225 disc: select main title by largest size, not clip count
canonical_title_order keyed on clip-count ascending before duration, so a short 1-clip bonus reel outranked the real feature — which is often chaptered into many clips (one per chapter). Make physical size the primary key: the main feature is the largest non-oversize title; decoy 'play-all' playlists run long but tiny (reused clips) and self-eliminate.

Validated across 23 UHD/BD discs: fixes t1 on F9, Fast Five, Fast & Furious 6, and Furious 7 (feature was ranked #13-36); no regressions on the 19 already correct.
2026-07-07 18:36:36 -07:00
Matthew Jackson 85347597cc disc: first-class FMTS + HD-DVD formats; CPI sample selection
Add DiscFormat::Fmts (AACS 2.1) and DiscFormat::HdDvd as first-class peers. Format derives from the AACS MKB generation (mkb_type().generation(): V10=BD, V20=UHD, V21=FMTS), reusing existing AACS code, and from the on-disc tree for HD-DVD/DVD. One detector (detect_disc_format) shared by the coarse DiscId probe and the full scan — no more 'default BluRay, defer to full scan'.

FMTS is a BD-tree stream variant: parse_playlist resolves the clip stream via CLIP_STREAM_EXTS (.m2ts -> .fmts -> .ssif), so the .fmts main feature yields real extents (previously silently empty). HD-DVD is a tree-level peer with its own enumerator (disc/hddvd.rs): HVDVD_TS/*.evo -> MpegPs titles with real extents (playlist/stream parsing honestly stubbed).

Sample selection for key resolution now uses the authoritative AACS CPI flag (aacs_unit_encrypted, byte-0 & 0xC0) not the ts_sync_destroyed heuristic — container-agnostic (M2TS/FMTS/EVO; TS-sync is meaningless on HD-DVD program streams) and stops the decode-server '0 encrypted units' rejection.

Tests live with each format (bluray/hddvd/mod); generic UDF fixture builders extracted to a shared udf::fixture module.
2026-07-07 18:16:04 -07:00
Matthew Jackson bc04ee7bd2 unlock_bridge: run_features/run_bus dispatch; report LibreDrive vs Renesas
Adopt freemkv-unlock's split Unlocker trait: run_features drives the drive-prep capability, run_bus the content bus removal, each iterating unlockers until one doesn't decline (NotApplicable = try next; Ok or a real error stops).

unlocker_matrix now reports which drive-prep unlocker actually ran — LibreDrive removes the bus at the drive; Renesas unlocks features but leaves the bus to the cert. Wire product_id through to fu::DriveId.

Bump to 1.2.3.
2026-07-07 16:36:05 -07:00
Matthew Jackson 122a03b23d aacs: remove boil veneer; move key newtypes to aacs::types
leak-guard / leak-guard (push) Successful in 8s
aacs::boil was a thin newtype veneer over aacs::derive — a duplicate layer.
Delete it: move the shared key newtypes (Vid, MediaKey, Vuk, ProcessingKey,
UnitKey) into aacs::types, and expose resolve_candidate + KeyCandidate from
aacs::derive directly. Downstream (keysource, disc::encrypt) now import from
aacs::{derive,types}. Pure API consolidation, no behaviour change; full test
suite green on Rust 1.86.
2026-07-05 12:09:39 -07:00
Matthew Jackson dcc553a716 aacs: keep flat re-exports for the typed key primitives
leak-guard / leak-guard (push) Successful in 8s
The module structure is the public API, but the typed key primitives
(MediaKey/UnitKey/Vid/Vuk, mk_from_dk/mk_from_pk/uk_from_vuk/vuk_from_mk),
derive_vuk, the aligned-unit decrypt entry points, and DeviceKey/HostCert
are load-bearing names that downstream key-source crates import through the
flat `aacs::` path. Re-export them here so those crates can track the
module refactor without a lockstep re-pin.
2026-07-04 22:19:39 -07:00
Matthew Jackson 3b06a4c844 aacs(2.1): variant Media Key as a clean PK -> Km primitive
leak-guard / leak-guard (push) Successful in 19s
Redesign derive_media_key_variant to the minimal derivation surface:

  derive_media_key_variant(mkb_records, pk) -> Km

- PK-input only. Deriving Kp from device keys (DK -> PK) is a separate
  concern (walk_processing_key); a leaked 2.1 key is a PK, and the chain
  starts at Kp. A bare PK arrives without its subset-difference slot, so
  the primitive tries it against every slot and returns the Km for the
  slot that passes the MKB's Verify-Media-Key record -- mirroring the
  classical bare-PK derive_media_key_from_pk, gated by the chain's own
  verify so an unverified key is never returned.
- VID-free: the Media Key is MKB-scoped. VUK stays the separate
  derive_vuk(Km, VID) step.
- KCD is a fixed algorithm constant compiled in, not a caller parameter
  (removes the kcd argument, the placeholder const, and KcdNotProvided).
- Soft-correction / online-challenge slots are treated as non-covering;
  surfaced over the generic miss so a disc needing those modes is
  distinguishable from a non-covering key.

resolve_keys_v21 updated to walk DK -> PK first, then call the primitive
and derive the VUK from Km + VID. Module + helper docs refreshed to the
pinned record layout; tests reworked for the PK-only signature.
2026-07-04 19:16:40 -07:00
Matthew Jackson f9d112e481 aacs(2.1): wire variants_for_uv from the 0x2d record; chain runs end-to-end
variants_for_uv previously returned None (placeholder), dead-stopping the
Media Key Variant chain at VariantsTableUnavailable on any real disc. Layout
now pinned against two real 2.1 variant MKBs (Zombieland v70, Stand By Me v70):
the 0x2d Encrypted-Media-Key-Variant-Data body is sd_count u16 VARIANTS entries
(1:1 with 0x0c cvalues / 0x04 subset-diffs) followed by the 16-byte per-disc
Nonce at the tail. variants_for_uv reads the sd_slot_index-th u16.

With this the chain runs fully: Kmp -> Kpnew(=Kmp^KCD, extracted CyberLink
constant) -> VKD(0x2f) -> Km -> Kvu. The only remaining input is a covering
2.1 Processing Key to validate against a known answer; until then the final
Verify-Media-Key (0x86) gate rejects any wrong layout pick, so a bad key can
never be emitted. Tests updated to reflect the wired lookup; 1.86 precommit
green (fmt+clippy+tests).
2026-07-04 18:26:09 -07:00
Matthew Jackson f4fe651cb9 aacs: point integration tests at real module paths
The facade removal (5ff0464) left tests/ calling aacs::disc_hash,
aacs::decrypt_unit, aacs::AacsVersion, etc. at the old flat paths,
so the branch did not compile its integration tests. Repoint each
to its real module (inf/derive/content/mkb). No logic change.
2026-07-04 18:25:59 -07:00
Matthew Jackson 5ff04649ba aacs: remove the mod.rs facade — module paths are the public API
Delete the pub-use re-export facade; consumers now import from the owning
module (aacs::content::decrypt_unit, aacs::mkb::MkbType, aacs::derive::derive_vuk,
aacs::boil::mk_from_dk, aacs::resolve::resolve_keys_v2, ...). Internal callers
repointed accordingly. Path-only change; logic hash identical (95fb9924); 2210
tests green.
2026-07-04 14:19:40 -07:00
Matthew Jackson 84aaceceb7 aacs: merge media_key + volume_key into derive.rs
The volume_key module was only 34 lines and is just the tail of the same
DK/PK -> MK -> VUK -> UK derivation ladder as media_key. Fold both into one
derive module so every aacs module is a substantial, distinct responsibility
(crypto/mkb/derive/inf/content/variant/resolve). Relocation only; logic hash
identical (95fb9924); 2210 tests green.
2026-07-04 14:14:56 -07:00
Matthew Jackson cdee9739fd aacs: split keys.rs god-module into media_key/volume_key/inf/resolve
Break the 2800-line keys.rs into four responsibility-scoped modules:
- media_key.rs: DK/PK -> Media Key subset-difference walk (+ probe harness)
- volume_key.rs: VUK derivation, unit-key unwrap
- inf.rs: Unit_Key_RO.inf parsing, disc_hash, content cert, in-drive MKB read
- resolve.rs: the resolve_keys_* orchestration (keys.rs renamed)

Relocation only; the (white-box) test suite stays in resolve.rs and pulls
the moved items via glob imports. Proven byte-identical to the pre-refactor
state via the logic hash (95fb9924); 2210 tests green.
2026-07-04 14:06:57 -07:00
Matthew Jackson f55f11d043 aacs: extract mkb.rs (MKB record format, MkbType, finders)
Relocate the shared MKB machinery into a single mkb module: the record
framing walker + MkbRecord view (from variant), the MkbType/AacsVersion
classification, the MKB-file utilities, and the record-body finders (from
keys). Fixes the inversion where the MKB parser lived in the 2.1-only
variant module. variant.rs keeps its local MkbRecord-based mkb_find_mk_dv
(name collision with the raw one; unified in the dedup follow-up).

Relocation only. Proven byte-identical to the pre-refactor state via the
function-body logic hash (imports normalized out); 2210 tests green.
2026-07-04 13:55:06 -07:00
Matthew Jackson f55d8f7acd aacs: extract crypto.rs (shared AES primitives + constants)
Relocate the shared low-level primitives into a single crypto module:
aes_ecb_encrypt/decrypt, aes_cbc_decrypt, aes_g (from content/variant) and
aesg3 + AESG3_SEED (from keys), plus AACS_IV. Fixes the scatter where AES-G
lived in the 2.1 file and AES-G3 in keys. Relocation only — no rename, no
logic change (logic-hash identical to baseline; 277 items; 2210 tests green).
2026-07-04 13:37:53 -07:00
Matthew Jackson 31b0ba323a aacs: rename decrypt.rs->content.rs, variants.rs->variant.rs (no logic change)
Pure file+module-path rename. 'content' names the AACS unit-decrypt layer
(distinct from the top-level sector-decrypt driver crate::decrypt), and
'variant' (singular, spec term 'Media Key Variant') names the 2.1 chain.
Logic-hash identical to baseline; 277 items intact; tests green.
2026-07-04 13:24:26 -07:00
Matthew Jackson 188baced39 aacs: annotate crypto with AACS spec section citations
Add [C]/[PR]/[BD]/[libaacs] §x.y provenance markers across the AACS
crypto so each primitive links to the spec section it implements, with a
source-tag legend in mod.rs. Doc-comments only — no logic, constant, or
signature changes.

Also: correct two stale record-type comments in variants.rs (0x82/0x83 →
the real 0x2d/0x2f) and document the Variant Number width (spec lsb_10 vs
the 2.1 chain's lsb_16, driven by the 65,535-entry VKD table).
2026-07-04 13:15:53 -07:00
Matthew Jackson bce11a2de0 restore freemkv-unlock path dep for local dev (post-v1.2.2) 2026-07-03 20:50:17 -07:00
Matthew Jackson 7a79577343 v1.2.2: bump version (freemkv-unlock git-pinned for the tag) 2026-07-03 20:50:14 -07:00
Matthew Jackson f07251c2d4 changelog: 1.2.2 (AACS 2.1 variant MKB, mk_from_dk fix, PK-path perf, autorip online-outage + move-error clear) 2026-07-03 20:48:04 -07:00
Matthew Jackson eb23ab4586 aacs: target real AACS 2.1 Variant MKB records (0x2d/0x2f/0x0c)
The Media Key Variant scheme is detected and parsed via the actual MKB
record types found on a real variant disc:
  - 0x2d  Encrypted Media Key Variant Data (C)
  - 0x2f  Variant Key Data table (65,535 x 16)
  - 0x0c  variant cvalues (one per 0x04 subset-difference slot)

Replaces the earlier placeholder 0x82/0x83 record types, which were a
guess and appear on no real MKB. is_variant_mkb, the record finders, and
the subset-difference cvalue source (now 0x0c, falling back to 0x07/0x05)
are updated accordingly, along with the V20->V21 upgrade detection in
resolve_keys_v2 and its fixtures.

The variant chain still halts at variants_for_uv (the VARIANTS[uv] /
Nonce sub-field offsets need a covering key to confirm end-to-end
against the 0x86 verify), so a best-effort offset is never silently
trusted. All 27 variant tests pass on Rust 1.86.
2026-07-03 13:25:01 -07:00
Matthew Jackson 61b6070f03 aacs: make the PK path terminal (no SD-tree walk)
A Processing Key is the key at its Subset-Difference node — one AES-G from
the Media Key — so it is tried directly against the MKB cvalue tables,
matching libaacs _calc_mk_pks (iterate PKs × cvalues). The prior code
treated every PK entry as a device-node label at unknown depth and BFS-walked
the SD tree (depth 3, capped 5), which was both wrong for terminal PKs and
~15x slower on a large UHD MKB (~181k cvalues): PK derivation on UHD dropped
from ~37s to ~2.4s.

The Subset-Difference tree walk now lives solely in the device-key path
(derive_media_key_from_dk), which owns per-node path bits; the PK path never
descends. Removed PK_WALK_MAX_DEPTH / _CAP / _walked / walk_pk_against_tables_impl;
renamed the core scan try_pk_against_tables and its probe test.

Precommit (Rust 1.86): fmt + clippy + tests green.
2026-07-02 23:17:20 -07:00
Matthew Jackson 741c1ea11d restore freemkv-unlock path dep for local dev (post-v1.2.1) 2026-07-01 19:08:56 -07:00
Matthew Jackson e713b26b87 v1.2.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-01 19:08:53 -07:00
Matthew Jackson fcbd667add changelog: describe the shipped re-base DTS fix + faithful-passthrough note 2026-07-01 19:02:04 -07:00
Matthew Jackson 88c03152e2 mux/dts: re-base PTS per PES instead of a running clock (fix long-title drift)
The first cut used a global running clock (max(next, own-PES PTS) + advance),
which fixed the same-PES collision but DRIFTED: once accumulated frame
durations exceeded the PES-timestamp spacing, it never re-based, so a
feature-long DVD DTS track ran minutes past its real length (2h44 for a 2h03
film) while AC-3 from the same source stayed exact.

Match the AC-3 path: re-base to each PES's own container timestamp, and advance
by one frame duration ONLY within a run of AUs sharing one PES. Fixes the DVD
multi-frame-per-PES collision without drift; the UHD DTS-HD MA per-PES
attribution (da85f56) is preserved (each AU still takes its own core PES's PTS).
Adds new_pes_rebases_to_its_own_pts_no_drift; full mux suite green (905).
2026-07-01 18:21:25 -07:00
Matthew Jackson c8e7ad5e56 v1.2.1: bump version + changelog (DVD DTS timestamp fix) 2026-07-01 17:30:25 -07:00
Matthew Jackson f122f08628 mux/dts: monotonic per-frame PTS + real frame duration (fixes DVD DTS)
DVD packs several DTS core frames into one PES; the parser stamped every
access unit with that single PES PTS and duration_ns=None, so consecutive
frames collided on one timestamp — ffmpeg rejected the output as 'non
monotonically increasing dts to muxer: X >= X' (deep-decode = corrupt,
e.g. The Punisher). The UHD DTS-HD MA path (one AU per PES, distinct PTS)
was unaffected, which is why this only surfaced on DVD.

Parse the DTS core header for samples ((NBLKS+1)*32) and sample rate
(SFREQ, 48kHz fallback) to derive each AU's duration, and stamp a running
monotonic PTS: max(next_clock, own-core-PES PTS), then advance by the
frame duration. A later PES whose PTS is ahead of the clock still wins
(preserves the UHD per-PES attribution from da85f56/c49a180); frames
sharing one PES advance frame-by-frame instead of colliding.

Tests: the 3 that encoded 'same PES -> same PTS' now assert monotonic
advance; new dvd_many_cores_one_pes_are_strictly_monotonic reproduces the
Punisher bug; duration/SFREQ-fallback unit tests added.
2026-07-01 17:17:35 -07:00
Matthew Jackson b2c3540989 restore freemkv-unlock path dep for local dev (post-v1.2.0) 2026-07-01 16:53:06 -07:00
Matthew Jackson 3a4307def6 v1.2.0: bump version (freemkv-unlock git-pinned for the tag) 2026-07-01 16:53:03 -07:00
Matthew Jackson 917026d566 changelog: DVD DTS/LPCM silent-audio fix + Pass-N recovery specialists + flat scheduler 2026-07-01 16:50:08 -07:00
Matthew Jackson 6b0bcbb43f recovery: flat-pool bandit scheduler (FREEMKV_PATCH_FLAT, opt-in)
The tier ladder (bucket->mug->thimble) forces tier 0 to attempt ALL
ranges before tier 1, and tier 1 all before tier 2 — so on a hardened
residual (late resume, flood already gone) the marginal specialists are
unreachable for hours: you bail near-empty buckets over hundreds of dead
ranges before a thimble ever touches water.

FREEMKV_PATCH_FLAT collapses the 3 tiers into ONE flat pool of all 16
handler configs. run_handlers already sorts best-first by the live
decayed-yield scorecard, so this is a data-driven bandit: the first
ranges try them all (explore), the ranking floats whatever is actually
landing sectors to the front (exploit), re-measured per range; a handler
that doesn't fit stays last but is never dropped (floor -> can revive).
The new handlers self-limit (yield after 4 dead reads), so the flat
scheduler does NOT reintroduce the old depth-first per-range starvation.

Opt-in; unset keeps the proven tier ladder as the default for 1.2.0.
2026-07-01 16:44:29 -07:00
Matthew Jackson 516ff4e581 test: widen profile_03 efficiency guard for tier-2 roster
The tier-2 marginal specialists now additionally probe permanently-bad
residual sectors before leaving them NonTrimmed, so the alternating-bad
profile's finite read count rose past the old tier-0/1 bound. Guard
still catches runaway; behavior asserts unchanged.
2026-07-01 13:55:20 -07:00
Matthew Jackson c27009d443 recovery: wire tier 2 marginal specialists into the Pass-N chain
Add a third breadth-first tier (PATCH_TIERS 2->3) that runs the marginal
specialists on the hardened residual tiers 0-1 leave: SlowSpin (Linear
fwd+rev @ min), FuaRetry (Linear fwd+rev+Bisect @ FUA), SlowFua (Linear @
min+FUA), CachePrime, Oscillate (@ max and @ min), SpeedSweep. Every read is
a wedge-safe read_span, so they inherit wedge-abort / unproductive-yield /
deadline for free. All are new configs, so the EWMA scorecard calibrates each
once then ranks by decayed rate — a specialist that doesn't fit self-
deprioritises. Tiers 0-1 (fast scouts, slow-deep) are unchanged; this is
purely additive. Also switch the scorecard log sort to sort_by_key.

cargo test -p libfreemkv green (2200 passed).
2026-07-01 13:50:35 -07:00
Matthew Jackson 789d988314 recovery: CachePrime + Oscillate handlers + fixtures
CachePrime reads the good run immediately preceding a residual island to lock
the servo/PLL, then reads the island warm (boundary sectors the drive can't
cold-seek). Oscillate reads each residual sector by alternating approach —
forward-into then reverse-into — for direction-dependent tracking. Both go
through the wedge-safe read_span (primes included). FakeDisc models a
direction-dependent sector and a servo-primed boundary sector; fixtures prove
a forward/cold Linear misses each while Oscillate/CachePrime recover them.
2026-07-01 13:50:35 -07:00
Matthew Jackson 1e0dc4c514 recovery: FuaRetry + SlowFua fixtures (FUA / min+FUA levers)
FuaRetry (Linear fwd+rev+Bisect at {max,fua,deep}) and SlowFua (Linear at
{min,fua,deep}) are Linear/Bisect at FUA params, no new struct. FakeDisc now
models a stochastic sector that lands only on the Nth physical (FUA) read (a
cached re-read keeps missing) and a hardest sector needing BOTH min speed and
FUA. Fixtures prove cached reads keep missing while the FUA group lands the
stochastic sector, and that neither lever alone recovers the hardest sector —
only the min+FUA combination does.
2026-07-01 13:50:35 -07:00
Matthew Jackson 629ed32e9e recovery: SpeedSweep handler + slow-spindle fixtures (SlowSpin/SpeedSweep)
SlowSpin is Linear pinned to min speed (no new struct). SpeedSweep is a new
per-sector handler that tries Max->Min until one reads (speed resonance).
FakeDisc now models a sector that reads ONLY at min speed; fixtures prove a
max-speed Linear misses it while SlowSpin and SpeedSweep recover it (and that
SpeedSweep tries fast-then-slow).
2026-07-01 13:50:35 -07:00
Matthew Jackson ac3b3fcfa4 recovery: decayed (EWMA) handler scorecard so the ranking flips
Grade handlers by an exponentially-decayed recovery rate (bytes/second,
alpha=0.5) instead of a cumulative rate. The residual hardens mid-pass, so
the best technique changes: cumulative froze the early winner in the lead
forever; the EWMA re-prices continuously — a handler that stops earning
decays down, a late-starting specialist climbs. Keeps rank()'s untried->top
one-shot calibration and attempted-but-zero-time->bottom.

New flip fixture proves a handler that recovers a lot early then nothing
loses its lead to one that starts recovering later.
2026-07-01 13:50:35 -07:00
Matthew Jackson 0c8153304e recovery: parameterize the read primitive with ReadParams (speed/FUA/timeout)
Add ReadParams { speed: SpeedPref, fua: bool, timeout: TimeoutPref } and
thread it through read_span so every wedge-safe handler read can request a
spindle speed (SET CD SPEED issued only on change, restored to max when the
handler exits), set the READ(10) FUA bit, and pick the 10s vs 60s timeout.

- SectorSource gains read_sectors_fua (default ignores fua); Drive sets the
  CDB bit, DecryptingSectorSource threads fua to its inner read.
- recovery_read gains a fua param.
- Linear becomes { direction, params }; Bisect/Jump take params. Existing
  tier-0/1 instances keep identical behavior (max speed, no FUA, fast/deep).
- Scoreboard keys on the full-config String name (linear:fwd:max:fast, ...).
- FakeDisc observes speed + FUA + approach so specialist techniques are
  provably exercised in later commits.

cargo test -p libfreemkv green (2193 passed).
2026-07-01 13:50:35 -07:00
Matthew Jackson e94319c099 dvd: route private_stream_1 audio sub-ids by positional stream number
The private_stream_1 sub-stream id low nibble is the DVD audio-stream
NUMBER (0-7), shared across codecs, not a per-codec ordinal. A DTS (or
LPCM) track that isn't the disc's first audio stream got a sub-id one
too low, so the demux routing key (0xBD00 | sub_id) never matched and
every packet was dropped -> present-but-silent track. AC-3 at position
0 coincidentally worked (ordinal==position). Route by positional index.

Fixes silent DTS 5.0 track on The Punisher (AC-3 5.1 @ pos0 plays,
DTS 5.0 @ pos1 was 0xBD88, real wire id 0xBD89).
2026-07-01 13:49:55 -07:00
Matthew Jackson a93da78621 lib: remove standalone sector-health verify feature
Delete src/verify.rs (standalone 'read every sector, classify health'
module) + its lib.rs pub mod/pub use. It was old, unused code. The
decrypt-verify gate (disc::verify / UnitVerifier) is a DIFFERENT module and
stays. Relocated the pure chapter_at_offset helper (used by locate_ranges +
autorip's done-card) into disc/mod.rs as a pub fn.
2026-07-01 13:09:21 -07:00
Matthew Jackson cf13838f12 recovery: audit fixes — live at-risk excludes NonTried, latency-gated wedge, drop dead block counters
- Live located drilldown (sweep + patch progress snapshots) now excludes
  NonTried, matching the one-shot path. Including the unread remainder made
  main_at_risk_ms show ~full-movie at sweep start and melt to 0 as it
  progressed — unread is unknown, not damage.
- Wedge abort now requires the failure to be FAST (< WEDGE_FASTFAIL_MS,
  500ms) as well as wedge-family sense: a real uncorrectable sector on
  Hardware-error media spends ECC-recovery time before failing, so it no
  longer false-trips the fast-fail wedge abort. New regression test.
- Removed the always-zero blocks_attempted/read_ok/read_failed/
  unreadable_count from PatchOutcome + PatchLoopState (dead residue from
  the old grind loop; the HandlerScoreboard supersedes them) so the
  patch.done log no longer emits misleading zeros.
2026-07-01 12:07:08 -07:00
Matthew Jackson 9519628954 docs: 1.2.0 changelog — Pass-N handler chain, DVD drive-prep unlock fix, unlocker matrix 2026-07-01 11:05:31 -07:00
Matthew Jackson 21f8cec419 disc: unlocker_matrix reports 'did work this rip', not 'matched'
yes now means the unlocker actually ran and did its job: LibreDrive from
the runtime firmware-unlock success, AACS host-cert only when LibreDrive
didn't do the bus (stock-drive fallback), CSS from the crack succeeding.
On a LibreDrive UHD that correctly reads LibreDrive: yes, AACS: no (LD
removed the bus, the cert route never ran) instead of the misleading
AACS: yes. Names stay registry-driven; runtime logic lives here.
2026-07-01 10:44:39 -07:00
Matthew Jackson d5d72c4e22 drive: disable FUA on READ(10) — the ~10x sequential-read regression
Forcing FUA (Force Unit Access) on EVERY READ(10) bypassed the drive's
readahead/streaming cache on the bulk Pass-1 sweep, collapsing sequential
throughput ~10x (UHD 15-25 → ~2 MB/s, DVD → ~0.5 MB/s), disc-type-
agnostic — the cache IS the streaming throughput. This was the real speed
regression (not unlock/riplock, which the earlier chase suspected). Clear
byte-1 bit 0x08. FUA will return as a dedicated Pass-N recovery handler
that sets/clears it per marginal-sector re-read, where cache-masking of a
stochastic sector actually matters — never blanket on the bulk path (#55).
2026-07-01 10:26:56 -07:00
Matthew Jackson c781fb7193 disc: Disc::unlocker_matrix() — registry-driven unlocker report
Returns each registered unlocker's name + whether it applies to this
drive+disc (via the unlock bridge over all_unlockers()). Disc crypto kind
derived in the library so the CLI and autorip render an identical, always-
current report with no hardcoded names. Also adds a Drive::unlocker_name()
runtime getter.
2026-07-01 10:00:59 -07:00
Matthew Jackson 541ca2139f drive: run drive-prep unlock for EVERY disc, incl. DVD (fix riplock regression)
The v1.0.0-rc.1 unlocker refactor added an `if disc_is_dvd() { return }`
early-return in Drive::init() that skipped the whole drive-prep unlock for
DVDs. That firmware unlock is what lifts riplock and readies max read
speed — a DRIVE-level, disc-independent feature — so skipping it left
every DVD stuck at stock/riplock speed (~0.4x, 3h ETA). UHD was unaffected
because it flows through the unlock.

Remove the skip: init() now runs the identity-keyed drive unlocker for all
discs (disc kind is Unknown at init, so only the drive unlocker matches;
the AACS host-cert handshake and CSS bus-auth still run later, gated on the
real disc kind, on top of the unlocked drive). Speed stays where it
belongs — SET CD SPEED(0xFFFF) at pass-1 start (disc/mod.rs) — not in the
unlocker. Also drop the matching probe_disc DVD skip.

Reverts the SET STREAMING stopgap (separate freemkv-unlock revert): that
treated the symptom; the real bug was the skipped unlock. Drive features
come from the firmware unlock, not a stock speed CDB.
2026-07-01 09:38:51 -07:00
Matthew Jackson 2dd98c3e32 recovery: fresh-eyes audit fixes (handlers + Pass-N engine + sweep)
Handlers (section_recover.rs):
- Bisect expand loops now honor ctx.halted() (were deadline-only, so a
  Stop could hang up to 60s vacuuming a readable island).
- read_span: explicit Transport arm so a bus-abort read isn't counted as
  unproductive grinding; debug_assert the sector-aligned span invariant.
- Scoreboard rank: an attempted-but-zero-time handler (e.g. returned Halted
  on its first check) now ranks BOTTOM, not top — it no longer crowds out
  proven performers.
- Document the wedge tier-size coupling + new regression test that a
  2-handler (tier-1) chain still catches a wedge via cross-section streak.

Pass-N engine (patch.rs):
- Rebuild PatchOutcome stats AFTER the post-read re-verify downgrade flush
  (was snapshotting before it, over-reporting bytes_good / recovered and
  risking a 'perfect rip' verdict on an imperfect one).
- Progress 'recovered' composes the still-bad set to MATCH work_total
  (subtract NonTried, add Unreadable) so the bar can't pin at 0 on a
  partially-swept disc or run backward on the Unreadable→NonTrimmed relabel.
- Remove dead work_done field; rewrite the stale 'adaptive batching' comment
  to describe the handler chain and mark block_sectors/full_recovery as
  informational-only.

Sweep (disc/mod.rs):
- Saturating arithmetic at the damage-jump position math (honor the
  read_error side's documented defence-in-depth guarantee).

Deferred (noted, need focused passes): fast_capture re-introduction,
Pass-1 halt-misclassified-as-jump, bytes_good display inflation, the
always-zero blocks_* telemetry, Pass-1 jump-on-first-error policy.
2026-07-01 09:14:32 -07:00
Matthew Jackson ceaa1da369 drive: apply drive-features for every disc, incl. DVD (fix riplock)
DVD ran the drive at ~0.4x stock speed: init() early-returned for DVD
before any speed was set, and its only speed command was the sweep's lone
legacy SET CD SPEED, which this BU40N ignores for DVD. BD/UHD got up to
speed only via the firmware unlocker's calibration — a path a DVD can't
take (it breaks stock CSS).

Call the new unlock-crate drive-features capability at init() for ALL disc
kinds, before the DVD stock-mode early-return, via a bridge shim. Stock
MMC only (SET STREAMING + SET CD SPEED), no bus unlock, so CSS is
undisturbed.
2026-07-01 08:56:19 -07:00
Matthew Jackson fe14a2d5e5 Pass-N: detect drive fast-fail wedge and abort instead of grinding
The 2026-07-01 overnight rip ground a wedged BU40N for 28 minutes at
0 B/s. Root cause: the handler chain's read_span split errors only into
Transport (bus gone) vs Bad (everything else), so the drive's fast-fail
wedge sense (ILLEGAL REQUEST 0x05/0x24 — it rejects every CDB in <100ms
without attempting recovery) was treated as an ordinary bad sector. The
chain hopped to the next section forever, and the existing wedge detector
in read_error.rs never saw it because the chain reads through
recovery_read directly.

Add a pass-level wedge streak: read_span counts consecutive wedge-family
(Hardware/IllegalRequest) senses; at WEDGE_ABORT_STREAK (16) it escalates
the read to Transport, which every handler already propagates as
TransportFault — aborting the whole pass and setting wedged_exit so the
caller spin-cycles. The streak is carried across sections via PatchCtx
(seeded into and read back from each per-section HandlerCtx) so a wedge is
caught even when every bad sub-range is smaller than the streak. Any Good
or non-wedge (medium-error) read resets it, so scattered bad sectors on
real media never trip it. New fixture test: a wholly-wedged 1000-sector
section aborts in <100 reads, not 1000.
2026-07-01 08:02:52 -07:00
Matthew Jackson 8f6a92ccd4 Bisect: halve at dead boundary + exempt boundary probes from stall
The expand loops broke out of the batch sweep on the first failing read,
overshooting the good/dead boundary by up to a batch and leaving the
readable sectors adjacent to a dead edge for the re-bisect to re-pin. On
top of that, early-yield counted those boundary-probe failures as a
stall, so the forward expand quit early AND the backward expand inherited
a poisoned unproductive counter and yielded instantly.

Now on a failed expand batch we halve the step (down to a single sector)
to recover right up to the dead edge in ~log2(batch) reads, drive the
expand loops off the deadline only (they self-terminate via halving), and
clear the unproductive streak once an island is located. patch_recovers_
good_middle_of_a_bad_range now recovers 50/50 good-middle sectors.
2026-06-30 22:06:04 -07:00
Matthew Jackson 29f76aad68 section_recover: per-rip handler scorecard + Bisect-leads-scouts + proportional Jump
Scorecard: grade each handler by recovery rate (MB/s) per rip, order
best-first on later sections, log the ranking at pass end. Untried
handlers rank top so each is calibrated once before ranking narrows to
the winners. Ephemeral (reset per pass), no persistence.

Tier 0 scouts are now [Bisect, Jump, Linear-fast x2], scorecard-ordered.
Bisect leads: probing the MIDDLE of a range lands on a readable island in
one read where a linear scan grinds the dead front to reach it. Jump now
jumps to the middle of the REMAINING span (proportional) instead of a
fixed 8 MiB that leapt clean over small ranges and missed their readable
middles. Tier 1 is slow deep-recovery on the residue.
2026-06-30 21:26:35 -07:00
Matthew Jackson 840ba8390c section_recover: per-rip handler scorecard + Bisect leads the scouts
Scorecard (ephemeral, reset each pass, no persistence): grades every
handler by recovery rate (bytes/sec). run_handlers orders handlers
best-first by that rate; an untried handler ranks top so each is
calibrated once, then the ranking narrows to the winners. Logged at pass
end (phase=scorecard) so the operator sees which handler is pulling the
weight on this drive/disc and which is a dud.

Tier 0 scouts are now [Bisect, Jump, Linear-fast x2], scorecard-ordered.
Bisect leads because probing a range's MIDDLE lands on a readable island
in one read, where Jump (linear from the front, big skip) can grind the
dead front or overshoot a small range entirely. The scorecard confirms
or overturns that order with real per-disc data.
2026-06-30 21:25:39 -07:00
Matthew Jackson e803905265 section_recover: Jump starts at 8 MiB base to clear big dead spots faster
Each dead probe read costs the drive's full ~10s timeout, so a large dead
region took a dozen escalating probes. Starting the jump at 8 MiB (vs
1 MiB) clears it in a handful; a skipped span stays bad for Bisect to
reclaim readable islands, so an over-jump loses nothing.
2026-06-30 21:12:12 -07:00
Matthew Jackson 11d4c33477 patch: single Jump-scout tier 0, expand-Bisect, live in-handler progress
Tier 0 is now a single fast Jump scout: it streams the big readable
ranges back and skips dead runs in seconds, so the pass reaches every
section fast and converges to the small genuine-dead residue instead of
grinding three handlers x 60s on each dead fragment. Tier 1 (fast
mop-up + slow deep reads + Bisect) works only that residue.

Bisect now expands: on a good probe it reads outward forward and
backward in full batches until a read fails, recovering the whole
readable island in large reads; the two failing ends become smaller bad
sub-ranges it bisects again. One huge bad range becomes many precisely
located small dead clusters.

Progress heartbeat: HandlerCtx gains a throttled tick (250ms) called
from every read, pushing a fresh snapshot to the reporter DURING a
handler. The bar and speed now move continuously as recovery happens
instead of jumping once per section (the reason speed read 0 B/s and the
% looked frozen between range boundaries).
2026-06-30 20:57:36 -07:00
Matthew Jackson 8d775cd341 patch: add Jump handler (lead fast tier) + recovery-based progress %
Jump: on sustained batch failures skip ahead an escalating distance
(1 MiB doubling to 256 MiB) to find where readable data resumes, leaving
the skipped span for Bisect to pin — mirrors the Pass-1 damage-jump. It
leads the fast tier so a large dead run is skipped in seconds instead of
the linear sweeps grinding every dead batch (10 s each) first; on a
readable range it just streams it back. Recovers readable data buried
behind a big dead front (the 192 MB Dune range).

Progress %: report bytes RECOVERED (initial-bad minus still-pending)
instead of a per-range counter that only advanced on the final tier — so
the bar reflects the readable bulk recovered during tier 0 the instant it
lands, matching the 'MB remaining' number.
2026-06-30 20:40:22 -07:00
Matthew Jackson bc07011bcb patch: breadth-first two-tier recovery, largest ranges first
The per-range walk was depth-first: each bad range ran the full handler
chain (fast + slow deep-recovery + bisect) before the next range was
touched. So a handful of tiny dead fragments at one end of the disc
burned the whole pass and the big NonTrimmed ranges elsewhere — usually
sweep-jump over-marks that read straight back — were never attempted.

Now recovery runs in two breadth-first tiers over ALL sections:
- Tier 0 gives every section one fast full-batch attempt (fast reads
  only), largest ranges first, so the recoverable bulk of the disc comes
  back in the first minutes.
- Tier 1 deep-recovers only the residue tier 0 could not pull.
Per-section still-bad sets persist across tiers. Largest-first ordering
means a big readable region is reached before time is spent on tiny dead
fragments.

Linear no longer collapses a failed batch to count=1 single-sector reads
(live probing: a marginal sector recovers in a large read, not a lone
one) — a failed 32-batch stays 32 and is re-attempted at full size by the
next handler/pass; Bisect salvages readable islands.

Adds a handler-start trace line so the debug log shows which handler is
running and the hand-off to the next.
2026-06-30 20:25:48 -07:00
Matthew Jackson d65b776a8e patch: replace grind-until-wedge loop with bounded handler chain
Pass-N recovery is now a chain of time-bounded recovery handlers instead
of one monolithic per-range loop that could grind the front of a bad
range for up to 30 min, wedge the drive, and abort the whole pass.

A bad range is a SubRanges set; recovery is an ordered list of
SectionHandlers (Linear{reverse,fast} covering back/forward x fast/slow,
and Bisect). A coordinator runs each handler with a hard per-handler
deadline: a handler recovers what it can (removing it from the still-bad
set) and hands the rest to the next handler; whatever is still bad after
the chain becomes NonTrimmed residue and we move on to the next range.

Guarantees, now structural rather than bolted-on:
- never hangs: every handler is deadline-bounded; the loop always drains
  to recovered-or-residue.
- always moves on: a range that cannot be finished leaves residue and
  advances; only a genuine transport fault or user halt ends the pass.
- extensible: a new recovery idea is one SectionHandler impl added to the
  chain; a proven-ineffective one is removed. The engine never changes.

Removes ~1.9k lines of the old inner loop (watchdogs, skip escalation,
NOT_READY grind, wedge counters) and their tests. fast_capture is now
inert (the chain supersedes it); breadth-first ordering becomes a future
scheduler concern. New module: disc/section_recover.rs (8 fixture tests,
injectable clock — bounded/never-hang proven without touching a drive).
Two A/B tests updated to the chain's strictly-better recovery counts.
2026-06-30 19:55:37 -07:00
Matthew Jackson ff7f3028a5 drive: always set FUA on READ(10) to bypass the drive cache
Every Drive::read now forces the read from physical media instead of
letting the drive satisfy it from cache. A recovery tool must not trust
the cache: on a re-read of a marginal sector the BU40N can return a
cached miss (~4 ms) instead of giving the surface a fresh physical
attempt (~40-250 ms), masking a sector that would otherwise read on a
real media hit. FUA (READ(10) byte 1 bit 3) is validated on the
BU40N/Initio bridge.
2026-06-30 18:49:14 -07:00
Matthew Jackson 25a7b131b4 drive: Drive::spin_cycle() — soft un-wedge via START STOP UNIT (no eject)
Spin the disc down (START=0, LOEJ=0) then back up (START=1) to clear the
BU40N/Initio fast-fail wedge state a run of HARDWARE_ERROR reads leaves the drive
in — the non-eject power-cycle our notes say the wedge needs. The disc stays
loaded; we never eject (slot-loading — a human eject is a product failure for an
unattended service). Validated live 2026-07-01: took the drive from
failing-every-read back to reading at MB/s.
2026-06-30 17:54:51 -07:00
Matthew Jackson 0cab32a08a disc/patch: fast-capture reads fail-fast (no deep recovery)
A fast_capture pass defers every failed block to a granular pass anyway, so
spending the drive's 60s deep-recovery timeout on it here only freezes the
breadth-first sweep on a pure-bad cluster (~25s per 32-block, incl. NOT_READY
grind). Read with recovery=false in fast_capture so a bad block fails on the
short timeout (~10s) and the sweep steps on; the granular passes
(fast_capture=false) do the deep recovery on what's left. No recovery lost —
the block stays NonTrimmed for the granular retry.
2026-06-30 17:08:26 -07:00
Matthew Jackson 22f0f5eb6f docs: 1.2.0 changelog — fast_capture breadth-first patch recovery 2026-06-30 16:38:56 -07:00