Compare commits

..
808 Commits
Author SHA1 Message Date
Matthew Jackson e99cb95bd4 v1.7.3: bump version (freemkv-unlock git-pinned for the tag) 2026-09-19 05:31:10 +00:00
Matthew Jackson 1794f5c406 merge: record v1.7.2 release into dev (-s ours; keep path deps + dev's next-cycle work) 2026-09-18 21:39:15 -07:00
Matthew Jackson 9379507c15 docs: backtick item[10]/item[0..5] in mpls module doc so rustdoc doesn't read them as intra-doc links (doc-links CI) 2026-09-18 21:30:25 -07:00
Matthew Jackson 8457efb5da changelog: 1.7.3 — multi-angle MPLS STN fix (#45) + read_structure_files for --share 2026-09-18 21:11:59 -07:00
Matthew Jackson dbc135bee0 disc: add read_structure_files() to capture BDMV/DVD index metadata for --share diagnostics 2026-09-18 21:08:28 -07:00
Matthew Jackson 48fb9b2cf6 mpls: locate STN table past the multi-angle angle block (issue #45)
The PlayItem STN (stream-number) table offset was hard-coded to 32, which
is only correct for a single-angle PlayItem. A multi-angle PlayItem inserts
number_of_angles(1) + flags(1) + (number_of_angles-1) angle references (10
bytes each) right after still_time, so its STN begins at 32 + 2 +
(number_of_angles-1)*10. The fixed offset lands inside that angle block and
misreads the stream counts as garbage (e.g. n_video=51 on Spider-Man 3 UHD
00245/00246.mpls, number_of_angles=2 => real offset 44), dropping every
stream including the primary video. That makes DiscTitle::has_video() return
false, which drops a multi-angle/seamless-branch feature from the disc's nav
candidate set and loses it in main-title selection.

Compute the STN offset from is_multi_angle (bit 4 of item[10]) + number_of
_angles. Add a regression test building a 2-angle PlayItem and asserting the
video stream is found past the angle block.

Validated on a 92-disc real UHD/BD corpus (offline scan): 0 main-feature
pick changes vs baseline, 3 multi-angle titles (Spider-Man 3 244/245/246)
recover has_video. Adds examples/corpus_titledump.rs as the regression tool.
2026-09-18 16:38:45 -07:00
Matthew Jackson 013f27eede fix(disc): SAFETY — spare seamless-branch feature from wrapper-composite demotion (#45)
The fallback composite gate in rank_titles flagged title P as a "play-all
wrapper" whenever some other title Q's clip set was a proper subset of P's,
Q had video, and Q.size >= 0.5 * P.size. main_feature_order then ranks
standalone-before-composite, demoting P below the decoy Q.

On a seamless-branch feature this inverts the pick: the real 00800 (2h03,
4 clips = body + intro/credits/seamless segments) is a SUPERSET of the
single-clip body decoy 00700 (1h50), and 700 >= 50% of 800's size, so 800
was mis-flagged composite and lost to the shorter 700 — the wrong title
gets ripped.

Add a duration-first wrapper-shape gate: P is only a composite when the
subset Q is EITHER a much shorter cut (Q.duration < 0.85 * P.duration, a
concat/play-all padding Q's runtime) OR nearly size-equal (Q.size >= 0.90 *
P.size, a bumper/outro wrapper). A near-full-length but markedly smaller
body subset (Q runs ~0.89 of P's duration yet holds far fewer bytes,
because P's branch segments add bytes not runtime) is a real seamless
branch, so P is kept. Uses per-title duration_secs/size_bytes, stays
capacity-independent and feature-gate-safe.

Preserves the real play-all avoidance: the 00245 bumper-wrapper decoy
(Q ~0.96 of both duration and size) stays demoted via the size limb, and
a concat play-all (subset < half the runtime) via the duration limb.

Add regression tests: seamless_branch_feature_not_demoted_issue_45 and
true_play_all_concat_wrapper_still_demoted. All existing selection tests
pass.
2026-09-18 13:34:50 -07:00
Matthew Jackson 9f0e10a524 restore freemkv-unlock path dep for local dev (post-v1.7.2) 2026-09-18 18:37:24 +00:00
Matthew Jackson b65124728c v1.7.2: bump version (freemkv-unlock git-pinned for the tag) 2026-09-18 18:37:21 +00:00
Matthew Jackson 10d073286c changelog: 1.7.2 — record the SET data-phase unlock fix + READ CAPACITY/empty-ISO hardening 2026-09-18 06:55:00 -07:00
Matthew Jackson 89fd1a6e4d merge: record v1.7.1 main into dev (dev already at 1.7.1; keep path deps) 2026-09-17 23:08:09 -07:00
Matthew Jackson 5c9e2fefe5 comment-guard: trim over-cap comment blocks to caps (comments only, no code change)
Bring every flagged comment block within its cap (inline // <=3 lines,
/// item-doc prose <=8): tighten wording, or convert an over-long inline
block that documents the item below it into a /// doc, preserving all
technical facts. No code, string literals, or logic changed.
2026-09-17 22:47:32 -07:00
Matthew Jackson 041ac5876f test: make case-collision test host-FS-independent
Probe dir_is_case_insensitive against the real target volume by creating
the dir first. Previously the test probed a not-yet-created dir, which
returns a conservative true; that disagreed with extract_tree's own
post-mkdir probe on a case-sensitive host (Linux CI), panicking the test.
Both branches (collision on insensitive, clean coexist on sensitive) still
verified.
2026-09-17 22:37:17 -07:00
Matthew Jackson e0875b630c docs: fix broken intra-doc links
- decrypt.rs: three refs to the #[cfg(test)]-only decrypt_sectors_mapped
  wrapper can't resolve in a non-test doc build; de-link to inline code
  (keeps the name, no broken link).
- Fix DecryptingSectorSource::with_content_ranges link to its full
  crate::sector path so it resolves.
2026-09-17 22:37:17 -07:00
Matthew Jackson d5902ff9f9 deps: sha1 0.10→0.11 (collapse RustCrypto digest/block-buffer/crypto-common dup); drop unused rand 2026-09-17 22:18:07 -07:00
Matthew Jackson 01cd85f514 changelog: open 1.7.2 — UNRELEASED (unified release; firmware ABI v2 in freemkv-unlock) 2026-09-17 14:35:17 -07:00
Matthew Jackson 6de61484a3 docs: point the freemkv-unlock path-dep note at release-orchestrate.yml (release.sh is gone) 2026-09-17 14:24:49 -07:00
Matthew Jackson 764519de43 aacs/disc: test coverage for VID key-derivation + handshake route selection; diag: print unknown mkb_version 2026-09-15 20:37:18 -07:00
Matthew Jackson 2fb7d78b29 libfreemkv: retry READ CAPACITY for ISO sizing; hard-error if unavailable
READ CAPACITY is the only authoritative whole-disc size for an image
(the UDF partition is a subset — measured 288 sectors / ~576 KB short on
a real BD, which would truncate the backup anchor). Its field failures
are sporadic (5/5 reliable on the target drive), so retry it a few times
rather than substituting a short value. On persistent failure the shared
scan/identify path stays lenient (capacity 0; MKV reads by extent) and
image_read_sectors() returns EmptyImage so an ISO fails cleanly instead
of writing a truncated or empty file.
2026-09-15 16:45:48 -07:00
Matthew Jackson 18024ea5f5 scsi/macos: fix io_service_t leak, honor bsd_name in fallback, fix FFI doc
shim_list_drives: reorder the drain loop condition so IOIteratorNext is
not called on the exit iteration, releasing every obtained io_service_t.

shim_open_exclusive: the fallback matcher ignored the requested bsd_name
and could open a different drive; iterate and accept only the candidate
whose IOMedia BSD name matches, else fail. Correct the misleading da_hold
TOCTOU comment.
2026-09-15 14:40:54 -07:00
Matthew Jackson 2287bb4449 labels/pes/disc: overflow + robustness fixes
class_reader: checked pc + size add (guards 32-bit overflow) and remove
the dead BadInstruction variant.

criterion: dedup duplicate stream numbers instead of emitting a number
twice. deluxe: continue past a bad studio attribute instead of aborting
the whole parse. bdmt: treat mixed-content XML (embedded markup) as XML
so it is dropped, without over-rejecting a bare '<' in prose.

pes: grow the frame buffer incrementally instead of an eager
MAX_FRAME_SIZE allocation. extract: give the case-probe a unique random
suffix so concurrent probes cannot collide (TOCTOU-safe).
2026-09-15 14:40:47 -07:00
Matthew Jackson 013cc3ce8d net/error/hex: NAT64 SSRF arm, Dir Display arms, reject leading +
network: add the NAT64 (64:ff9b::/96) arm to the IP classifier -
extract the embedded IPv4 from the last 32 bits and re-check it, matching
the keysources canonical copy, so a NAT64-wrapped private/special
address is blocked like the bare IPv4.

error: add Display arms for DirInsufficientSpace (E9027), DirNameCollision
(E9028), DirWriteFailed (E9029) so their fields render (codes-only, like
the sibling Dir variants).

hex: reject a leading + in the hex int parser.
2026-09-15 14:40:40 -07:00
Matthew Jackson ab2416d07a aacs: propagate MKB walk errors, add bounds/truncation guards, fix FMTS carry
inf: read_mkb_from_drive used .is_ok() after the first pack, silently
truncating the MKB on a mid-walk transport error; propagate the error
instead. Add checked offset guards (94/117) and reject a truncated title
table (161) like the key list already does.

segment: the FMTS key-range carry assumed 2048-aligned segment starts;
use the floor-difference of the actual LBAs so a non-2048-aligned
segment carries correctly.

Annotate css::keyless::recover_title_key as a test-only helper and
correct the clpi_audit dedup-by-PID doc.
2026-09-15 14:40:33 -07:00
Matthew Jackson b735fb3730 mux/codec: PTS carry, buffer cap, EOS core drop, PPS-by-id, dead code
lpcm: carry last PTS + DTS fallback (add last_pts_ns) and propagate
pes.discontinuity instead of hardcoding false, matching adts/flac/
mpegaudio.

pesbuf: add an internal MAX_BUFFERED_BYTES cap (defense-in-depth like
reorder.rs's MAX_GOP) so a misbehaving caller can't grow it unbounded.

dts: drop a truncated trailing core at end-of-stream instead of emitting
it, while keeping genuine trailing extensions.

hevc: resolve num_extra_slice_header_bits from the PPS the slice
references (slice_pic_parameter_set_id), not the last-active PPS.

crc: remove the dead, misnamed crc8_atm (impl was CRC-8/SMBUS, no
callers). Add the missing drop-gate mismatch test; fix the startcode
read_bits contract doc and the h264 SEI comment.
2026-09-15 14:40:26 -07:00
Matthew Jackson f1b7a73b93 mux: harden MKV child loops, read real mp4 sample rate, fix pre-video PTS origin
mkvstream: add the consumed > remaining => MkvSourceInvalid guard (as
BLOCK_GROUP already had) to the INFO/TRACKS/TrackEntry/Video/Audio child
loops, and route the DURATION/SAMPLING_FREQUENCY float reads through the
checked_size cap instead of truncating cs as usize.

mp4/read: audio sample_rate was taken from the mdhd timescale, ignoring
the AudioSampleEntry 16.16 samplerate at the stsd body; read the real
field and fall back to timescale only when it is 0. Drop the unreachable
MAX_TRACKS break (the box scan is already capped) for a debug_assert.

tsmux + m2ts_mux: PTS origin was seeded on video only, so audio
preceding the first video (or an audio-only stream) rebased onto its own
PTS and lost A/V offset; seed the origin from the first frame of any
kind.
2026-09-15 14:40:18 -07:00
Matthew Jackson fbe91895c4 mux: fix filtered-export drop gate, log skipped key-map, restore json fields
demux_sink: dropped_total() folded drops across ALL tracks while
frames_mapped now counts only persisted frames, so a filtered
(sub://audio://) multi-clip export could false-trip SeamPlanDroppedMost /
SinkWroteNothing. Compare drops and mapped frames over the same
persisted-track set via TimelineContinuity::dropped_for; update the
frames_mapped doc to the persisted-only semantics.

resolve: resolve_mux_key_map(...).ok() swallowed the error silently
(TrueHD channel correction skipped with no trace); log it like the
sibling reopen() fix before skipping.

meta_sink: TitleProfile::from_title dropped json:// stream fields
(pid, color/aspect/dimensions/interlaced/mvc, audio secondary/
sample_rate/purpose, subtitle pid/descriptive_service); restore them at
the json document level.
2026-09-15 14:40:09 -07:00
Matthew Jackson 679894e5ea aacs/disc/drive: fix bus-key gate credit, shift guard, tray-unlock log
disc/encrypt: on the disc-keyed cert route, drive_unlocked was derived
from is_drive_unlocker(matched), which is provably always false -
run_bus only ever dispatches the disc unlockers (AACS/DVD), never a
firmware/drive unlocker. Hardcode false with an honest comment; the
genuine drive-unlock case (drive_unlocked, no read_data_key) is the
OEM-VID short-circuit and the bus-key gate already credits it. Gate the
now test-only is_drive_unlocker behind cfg(test) and add a regression
test through resolve_vid_only.

aacs/derive: resolve_dk_node looped 0..u_mask_shift shifting 1u32 << b,
which panics/wraps when the disc/keydb-controlled u_mask_shift is >= 32.
Cap the search at the 32 real bit positions.

drive: unlock_tray dropped its SCSI result with let _; warn! on failure
so a stuck medium-removal PREVENT lock is visible. Fix the stale
probe_disc doc (now a no-op; calibration moved into the unlocker).
2026-09-15 14:39:53 -07:00
Matthew Jackson b7914c5995 decrypt: honor with_content_ranges in the mapped AACS read path
DecryptingSectorSource::with_content_ranges was a silent no-op: the
proactive AACS key_map path early-returned before consulting the content
extents, and decrypt_span ignored its _content argument. A whole-disc
reader (sweep/patch) that installed both a key map and content ranges
therefore decrypted/verified clear UDF/BDMV-nav units, and its
misaligned filesystem reads tripped the AACS unit-alignment gate.

Thread the content extents through decrypt_sectors_mapped_in_content ->
decrypt_span -> apply_aacs_map, skipping any unit whose LBA falls outside
every content range (never decrypted, verified, or counted as loss), and
gate the alignment check on span_touches_content so out-of-content
filesystem reads pass through untouched.

Also drop the redundant is_multiple_of clause in the prefetched
unit-tail guard (remaining > 0 is already guaranteed).
2026-09-15 14:39:31 -07:00
Matthew Jackson 747aeba4a4 error: drop premature E7031 KeyServiceConfig variant
Reverts the cross-repo E7031 addition: it forced enumerating the code across
freemkv test_support + freemkv-i18n's 29 locales for a LOW config-fault nicety.
keysources config faults keep borrowing the transient key-service code until a
coordinated i18n release; decrypt_bus-pub and the SSRF-parity additions stay.
2026-09-15 13:46:06 -07:00
Matthew Jackson 034a8b145b fix: cross-repo integration — widen decrypt_bus, SSRF parity, KeyServiceConfig
- aacs/content: widen decrypt_bus to pub so freemkv-drivetest can invert
  AACS 2.0 bus encryption out-of-tree (was pub(crate), broke that build).
- mux/network: port the 6to4 (2002::/16) and Teredo (2001:0000::/32)
  tunnel-decode arms and the 198.18.0.0/15 + 192.0.0.0/24 blocks from the
  canonical keysources SSRF classifier so both cover the same set.
- error: add distinct KeyServiceConfig variant (E_KEY_SERVICE_CONFIG 7031)
  for the standing key-service misconfiguration fault, separate from the
  transient KeyServiceUnavailable; keysources' online.rs adopts it later.
2026-09-15 13:36:47 -07:00
Matthew Jackson 6720ccdf8b style: cargo fmt the audit changes 2026-09-15 11:02:53 -07:00
Matthew Jackson 24ab408559 test/labels: cover dirimage nesting cap, long_ad dir, paramount feature hint
- dirimage: add a MAX_DEPTH nesting-cap test that walks a real
  deeper-than-cap directory chain and expects DirImageTooLarge.
- udf: add a long_ad (ICB flags = 1) directory read test — the LBA sits
  at AD offset 4, not 12 like extended_ad.
- labels/paramount: derive the feature hint's id and filename from one
  parsed number (canonical 5-digit NNNNN.mpls) so they can't disagree,
  the same fix already applied to fox, and add a hint-parse test.
2026-09-15 11:02:04 -07:00
Matthew Jackson 6e427e2eac test(scsi/linux): cover the fd-reopen path helpers
Add unit tests for to_c_path (single trailing NUL, path bytes preserved
for libc::open) and resolve_to_sg (sg passthrough plus the no-filename
and non-sr/sg fallbacks that need no sysfs) — the helpers the open /
drive_has_disc / recovery reopen path relies on. Verified compiling for
x86_64-unknown-linux-gnu (this module is Linux-only).
2026-09-15 10:58:38 -07:00
Matthew Jackson e130528fea test: cover drive tray/speed rejection, bdnav Call ops, mkv interlace patch
- drive: SET CD SPEED and PREVENT/ALLOW MEDIUM REMOVAL are best-effort; a
  drive rejection must warn, not fail the rip. New test drives a failing
  transport through set_speed/lock_tray/unlock_tray.
- bdnav: CallObject/CallTitle follow the same play path as JumpObject/
  JumpTitle for feature resolution; add tests for both.
- mkv: FlagInterlaced is corrected at finish() to the whole-stream scan
  majority; add a test that a progressive leader on an interlaced feature
  is patched from progressive to interlaced.
2026-09-15 10:56:26 -07:00
Matthew Jackson 0de2e598ef dvdnav(test): cover Vm set-ops (incl divide/mod by zero) and First-Play SetSystem
Add a direct Vm::set test exercising mov/add/sub/mul/div/mod/and/or/xor
and the divide- and modulo-by-zero guards (register left intact, no
panic) plus an unmodelled-op no-op, and a First-Play test that a
SetSystem instruction keeps resolution running so a following JumpTT
still resolves.
2026-09-15 10:51:23 -07:00
Matthew Jackson d0a5a90261 cleanup: remove dead MOBJ cmd cap, add early-break and doc clarifications
- bdnav/mobj: drop the unreachable MAX_CMDS cap (num_cmds is a u16 and the
  span is already bounded against the input) and its flagged NOTE.
- drive/wait_ready: break out immediately on a dead bus (transport
  failure) instead of polling a phantom for the full ~30 s; add a test.
- css/is_scrambled_pack: document that the 0x11 stream_id offset assumes
  zero pack stuffing (true for DVD-Video VOB packs) and why a stuffed pack
  degrades safely.
- scsi/macos_shim: document why releasing g_handle_lock before da_hold is
  TOCTOU-safe (da_hold re-validates the handle under the lock).
2026-09-15 10:49:16 -07:00
Matthew Jackson be296b3901 mux(timeline): tag provisionals with a monotonic epoch counter
passive_offset tagged a provisional offset with epoch_offsets.len() as
the epoch identity, but that length is capped at MAX_EPOCHS. Past the cap
the ring's remove+push leaves len() constant, so a provisional captured
then compared equal to the current identity forever and never retired,
drifting that track off every other one for the rest of the title. Add a
monotonic epoch_seq bumped once per open_epoch and tag provisionals with
it. Add a past-the-cap retirement test.
2026-09-15 10:45:35 -07:00
Matthew Jackson b30b06e04e mpls/ifo: fix STN gate off-by-one and pin the PGC palette chroma order
mpls: the STN table gate required item.len() > STN_OFFSET + 16, one byte
more than the 16-byte header actually spans, skipping a first play item
whose STN header ends exactly at the item boundary. Use >=, with a
boundary test.

ifo: the subtitle palette layout [padding, Y, Cr, Cb] (Cr BEFORE Cb) was
correct but under-documented; harden the comment against a future
byte-swap and add a test that cross-checks the parse against the DvdSub
ycbcr_to_rgb consumer so the two read Cr/Cb the same way.
2026-09-15 10:42:37 -07:00
Matthew Jackson e6d86b3582 labels: fix fox hint field disagreement and deluxe bare studio anchor
fox: playlist_id and filename were derived independently from the raw id,
so a 5-digit id above u16::MAX left playlist_id None while the filename
stayed Some — a half-hint naming different things. Derive both from one
parsed number via a feature_hint() helper with the canonical NNNNN.mpls
filename, and test that the fields agree.

deluxe: parse_studio_attr anchored on a bare find("studio"), which also
matched substudio=/studioId= and the word 'studio' inside another
attribute's value, then read a later attribute's quotes as the studio.
Require a word boundary before it and '=' after, and add anchoring tests.
2026-09-15 10:38:40 -07:00
Matthew Jackson 2a701a217a dirimage: refuse a read that begins past the image's capacity
read_sectors fabricated zeros and reported success for any LBA, including
one at or beyond the image's own capacity — an oversized/crafted offset
that a real file/ISO source rejects. Refuse a read whose START is past
capacity (as read_exact does past EOF) while still zero-padding a batch
that starts in range and overshoots the end, which the mux's final batch
and the FS scan's fixed-LBA probes rely on. Split the read-range test to
pin both behaviours.
2026-09-15 10:33:26 -07:00
Matthew Jackson 90e8e99152 mux(resolve): surface the reopen error instead of swallowing it
The TrueHD channel-correction probe's reopen closure returned Option<S>
via .ok(), collapsing a real io::Error into an indistinguishable None.
Make reopen return io::Result<S> and log the actual error on the failure
arm so a failed re-open is diagnosable, not silent.
2026-09-15 10:33:26 -07:00
Matthew Jackson 1edacf294f unlock: derive unlocker names/classification from the real unlockers
unlocker_names() was a hand-maintained list that could drift from the
unlockers that actually dispatch, and the cert route hardcoded
drive_unlocked:false. Factor the firmware and disc unlocker sets into
single constructors, derive unlocker_names() and a new is_drive_unlocker()
from their real .name(), and set HandshakeResult.drive_unlocked from
whichever unlocker claimed the drive so the bus-key gate credits a drive
unlock even on the cert dispatch path. Add derivation and classification
tests.
2026-09-15 10:27:09 -07:00
Matthew Jackson 7350577d91 disc(extract): fold host-path collisions by the target volume's case rule
The extract collision check always folded names to lowercase, so two
disc files differing only by case (Movie/movie) aborted the extract even
on a case-sensitive volume where they are distinct, coexisting files.
Probe the real target directory's case-sensitivity and fold only when the
host would (APFS/NTFS), defaulting to case-insensitive when the probe
can't run. Update the case-collision test to track the real volume, and
drop a dead duplicate pack-start write in the CSS cross-contamination test.
2026-09-15 10:24:37 -07:00
Matthew Jackson b985618fb5 mux(dts): continue the timeline for a PTS-less PES instead of resetting
When a PTS-less PES arrived with pending_pts invalidated (e.g. after a
forced flush), the fallback fell straight to 0, snapping the timeline
backwards. Add continuation_base_ns() to reach for the projected next-AU
start, then the last PES front, before conceding 0, and stop stashing a
drained buffer's stale front mark as pending_pts. Add a fallback test.
2026-09-15 10:22:04 -07:00
Matthew Jackson d3c4ce4a60 udf: read every FID extent of a directory, honouring the AD chain
read_directory read only the FIRST allocation descriptor of an out-of-
line directory, so a FID list spanning multiple extents was truncated at
the first boundary, and a type-3 continuation descriptor was mis-read as
FID data. Walk the whole descriptor list (types 0/1/2 as allocated FID
extents with the 2-bit type masked out of the length; type 3 followed as
a continuation block, bounded by MAX_AD_BLOCKS), gathering every extent
into one buffer while tracking the declared FID-list length separately
from the sector-rounded buffer. Bound the FID walk and the name read by
that declared length, and read a FID whose 38-byte header ends exactly at
the boundary (<=, not <) while still rejecting one whose name spills past
it. Add a multi-extent directory test.
2026-09-15 10:18:51 -07:00
Matthew Jackson f4a48480df mux(h264): gate open-GOP promotion on all-intra slices and base view
The open-GOP keyframe promotion trusted only the first slice's coding
type and wasn't gated on the base view. A picture whose first slice is I
but whose later slices are P/B is not a clean random-access point, and an
intra MVC dependent view is never an independent anchor. Track every VCL
slice's coding type (all_vcl_intra) and require a real VCL slice, then
promote only when all slices are intra AND !mvc. Add mixed-slice and MVC
non-promotion tests.
2026-09-15 10:06:20 -07:00
Matthew Jackson aad821c8a2 dirimage: count FID-list sectors in the metadata ceiling
The MAX_META_BYTES guard multiplied (dir_count + file_count) by the
sector size, counting one File Entry per node but omitting each
directory's FID-list data sectors. A wide, shallow tree's directory data
can dwarf the per-node entries, so an oversized folder could slip past
the guard and be materialized in full. Gate on metadata_block_count()
instead, which already sums File Entries and FID-list sectors, and add a
test proving the FID-list sectors are counted.
2026-09-15 10:04:12 -07:00
Matthew Jackson 959bba919d mux(demux): count only persisted frames toward frames_mapped
frames_mapped incremented for every mapped frame, including kind-filtered
ones that write to no track file (e.g. video during an audio:// export).
That inflated the denominator so a seam plan dropping every persisted
frame could ship zero-byte files at exit 0. Move the increment inside the
write block so it counts persisted frames only, and add a test where more
frames are seam-dropped than persisted -> SeamPlanDroppedMost fires.
2026-09-15 10:02:53 -07:00
Matthew Jackson ff3e46a560 error: give ImageTruncated a Display arm carrying have/want
ImageTruncated fell through to the bare-code catch-all, dropping the
have/want byte lengths a truncated-resume report needs. Add an explicit
arm emitting them in the identifier-free E{code} have/want form.
2026-09-15 10:01:14 -07:00
Matthew Jackson 14f97aacd6 aacs: fall through a primary DiscRead to the DUPLICATE copy
read_first propagated every non-UdfNotFound error immediately, so a
DiscRead on the primary Unit_Key_RO.inf aborted instead of trying the
/AACS/DUPLICATE/ backup that AACS ships for exactly this case. Treat a
DiscRead as retriable like an absent candidate: continue to the next
path, remembering the error, and only propagate it once every candidate
has been tried (still more informative than a bare AacsNoKeys). Add
fall-through and propagate-after-exhaustion tests.
2026-09-15 10:01:14 -07:00
Matthew Jackson 5c911d1573 mp4: reject hostile 64-bit largesize in find_boxes_capped
find_boxes_capped used plain `pos + box_size` on an attacker-controlled
64-bit largesize cast to usize. A value near usize::MAX at a nonzero pos
wraps past the length guard, producing a release panic on the payload
slice or an infinite loop on the advance. Mirror read_moov: compute the
end once with checked_add, reject on overflow or overrun, and reuse it
for the guard, slice, and advance. Add the hostile-largesize test.
2026-09-15 09:59:35 -07:00
Matthew Jackson cdc1d17993 v1.7.1: bump version (freemkv-unlock git-pinned for the tag) 2026-09-14 03:34:47 +00:00
Matthew Jackson 7b4a0b31d5 docs(changelog): 1.7.1 release notes (LibreDrive unlock fix) 2026-09-13 20:02:14 -07:00
Matthew Jackson a85764da38 ci: bump actions/checkout v4->v7 in identity-guard (dependabot) 2026-09-14 01:30:58 +00:00
Matthew Jackson 5af336776c chore: bump version to 1.7.1 (unified release lockstep) 2026-09-14 01:22:18 +00:00
Matthew Jackson 86c2c3c501 Add Sponsor badge to README 2026-09-06 23:29:33 +00:00
Matthew Jackson c280478ef9 Add FUNDING.yml 2026-09-06 23:25:41 +00:00
Matthew Jackson e7bcef0c45 docs(changelog): record the scsi-only Windows build fix in 1.7.0
The scsi::windows::list_drives fix (build under default-features=false,
features=["scsi"]) ships in 1.7.0; add it to the changelog's Fixed list.
2026-09-05 18:14:36 -07:00
Matthew Jackson 695569d29a scsi(windows): make list_drives self-contained (build under scsi-only)
scsi::windows::list_drives delegated to crate::drive::windows::find_drives(),
but the drive and identity modules are #[cfg(feature = "rip")]. A scsi-only
build (default-features = false, features = ["scsi"]) — which is exactly how
freemkv-firmware consumes libfreemkv — therefore failed to compile: 'could not
find drive in the crate root'. The --all-features CI never saw it (rip is on).

Enumerate directly at the SCSI layer via scsi::open + scsi::inquiry, mirroring
the Linux and macOS backends, which already build their DriveInfo without the
rip-gated modules. The rip-side drive::windows::find_drives() is untouched.

Verified: cargo check --target x86_64-pc-windows-msvc passes both scsi-only and
with default features.
2026-09-05 15:54:45 -07:00
Matthew Jackson 77e4bca234 docs: reconcile unlock-bridge with the current single-unlock dispatch 2026-09-05 12:35:03 -07:00
Matthew Jackson ae8c20f1ca scsi/macos: assert the presence-probe leaves the lock unchanged, not the global bit 2026-09-05 12:35:03 -07:00
Matthew Jackson 086547b78b scsi/linux: cap detached fd-recovery threads on a wedged bridge 2026-09-05 12:35:03 -07:00
Matthew Jackson b09ed299fc drive: log SCSI failures on lock_tray/set_speed instead of dropping them 2026-09-05 12:35:03 -07:00
Matthew Jackson 3f262ba6cb aacs: surface real disc-read errors instead of a blanket AacsNoKeys 2026-09-05 12:35:02 -07:00
Matthew Jackson 3c8fce3031 disc/encrypt: don't fall through to the AACS cert unlocker on a firmware-claimed drive
A drive claimed by a firmware unlocker (freemkv/LD/Renesas) at init() is that
unlocker's sole VID source. When it stashed no VID, do_handshake_cert now returns
(None, None) instead of running the AACS cert route (run_bus's AacsUnlocker) whose
REPORT KEY AGID cycle poisons the drive's AGID selector so every later bare 0xAD
VID read aborts until the medium is reloaded. A key source can still supply the key.

Adds a red-before-green test proving the guard issues zero SCSI on that path.
2026-09-03 07:41:43 -07:00
Matthew Jackson 5578e42961 Cargo: restore dev path-dep for freemkv-unlock
The dev branch tip must path-dep ../freemkv-unlock so CI's sibling checkout
(and local dev) resolves the branch tip; release.sh swaps it to a git tag
only in the tagged commit. dev had been left in the released git-tag
(v1.6.14) form after the last release, so the new 1.7.0 unlock API couldn't
resolve. Restore the path form the comment already describes.
2026-09-02 18:02:26 -07:00
Matthew Jackson a7db00c0a1 unlock_bridge: drive unlockers through the unified single-unlock contract
Adopt freemkv-unlock's new Unlocker::unlock(). The bridge assembles the
firmware set (freemkv / LD / Renesas) for drive-prep and the disc set
(AACS cert + DVD) for bus removal, injecting host certs into the AACS
unlocker at construction — the loop no longer knows any unlocker's guts.

- run_features / run_bus now loop unlock() and return Ok(Option<Unlocked>);
  drive init and do_handshake_cert adapt to the Option.
- unlocker_matrix reports the LD name and credits Renesas as a drive-level
  (raw-read) route, so AACS shows "no" when any firmware route took the bus.
- Cert route sets HandshakeResult.drive_unlocked=false: the bus is removed
  via read_data_key (AKE), not at the drive; the gate credits it that way.
2026-09-02 17:53:33 -07:00
Matthew Jackson 09957e066b ci: identity-guard — block unrelated-corp / non-canonical committer emails 2026-09-02 14:46:16 -07:00
Matthew Jackson a88b7486ac release: 1.7.0
Fixed
- mux/mp4: honour 64-bit largesize (size==1) and size==0 (run-to-end) in the
  capped child-box scan; a largesize sibling previously ended the walk early and
  later boxes vanished from a MOV/MP4.

Version 1.6.15 (unreleased) -> 1.7.0; changelog folded under 1.7.0.

Earlier 1.7.0 work remains as its own commits below this one: the
GHSA-j8ww-f5fg-9pmh sector-recycle soundness fix, CI/lint hardening (#15/#17),
docs, and Rob Ekl's scsi raw_command / fd-handoff change (#16).
2026-09-02 12:40:55 -07:00
Rob Ekl 6957c7251c scsi: route raw_command through checked_cdb_len; add the fd hand-off release edge (#16)
raw_command clamped CDBs with cdb.len().min(16) instead of routing through
the shared checked_cdb_len guard, so an over-length or empty CDB became a
different command rather than a rejected one; it now returns the crate Result
so a rejected CDB is distinguishable from an I/O failure.

The recovery/Drop fd hand-off lacked a release edge: the teardown swap was not
a release op and the recovery CAS was not an acquire load, so a recovery thread
publishing its fd after Drop drained the slot could miss dead==true and leak the
descriptor. Both sides are now AcqRel, with the execute() drain left at Acquire
(sound via the release-sequence rule). A loom model (cfg(loom) dev-dep, its own
CI job) provides the regression coverage cargo test cannot.

Module docs for fd_handoff and timeline now resolve in their own scope, and item
docs stay within comment-guard's caps.

Fixes #16.
2026-09-02 12:40:55 -07:00
Matthew Jackson 6e14fc0c0c libfreemkv: 1.7.0 groundwork — sector-recycle soundness + CI hardening
Security
- Remove the unsafe Vec::set_len over recycled sector buffers — UB reachable
  from safe code (GHSA-j8ww-f5fg-9pmh, low). Uses Vec::resize; pooling unchanged.

CI / tooling
- clippy on stable with the MSRV single-sourced from Cargo.toml; cargo-deny
  dependency-audit gate with first-party git sources allowed (#15); audience-based
  comment-guard (#17).

Docs
- README (keydb ownership, BD-J parser count), drive-access transport/reset
  notes, mux timeline module-doc links.
2026-09-02 12:40:55 -07:00
Matthew Jackson 962aa5d852 v1.6.14: bump version (freemkv-unlock git-pinned for the tag) 2026-08-31 23:04:25 +00:00
Matthew Jackson 719b63e376 chore: reconcile v1.6.13 release commit from main into dev 2026-08-31 15:48:53 -07:00
Matthew Jackson 3cf7922574 docs(changelog): 1.6.14 entry 2026-08-31 15:32:40 -07:00
Matthew Jackson 0c953e9340 chore(msrv): floor to 1.88, pin CI on the floor, allow uninlined_format_args
Declared rust-version and the ci.yml/qa.yml toolchain pins both sit at 1.88 (the
edition-2024 floor once zip 8 and let-chains are accounted for), so CI builds on
exactly the MSRV: an accidental bump to a newer toolchain fails CI by design and
moving the floor takes a deliberate pin edit. The uninlined_format_args allow
reconciles 1.88's clippy with the allow-by-default it became in 1.97+.
2026-08-31 11:06:35 -07:00
Matthew Jackson 2e90a7585d drive: move --share capture orchestration to freemkv-engine
libfreemkv keeps only generic SCSI primitives; the app-specific drive-info
capture (INQUIRY / GET_CONFIG / vendor buffers) now lives in its consumer.
2026-08-31 10:08:36 -07:00
Matthew Jackson 3e11225389 audit: resolve remaining v1.6.10..HEAD findings (parallel fan-out)
Correctness/robustness:
- scsi/macos_shim: guard g_handle / g_da_bsd read-modify-write with a
  process-global mutex (scoped OUT of the 5 s async-claim wait) — closes the
  DiskArbitration global-state TOCTOU on concurrent/re-entrant open+close.
- disc/profile: the public DiscProfile now surfaces `encrypted` and a numeric
  `key_error` (Error::code of aacs/css failure) so a disc with failed key
  resolution no longer serializes identically to a rippable one.

Performance:
- disc/dvd: read VIDEO_TS.IFO once and reuse it for both parse_vmg and
  resolve_main_title (parse_vmg_with / resolve_main_title gain an optional
  vmg_bytes) — eliminates a duplicate VMGI read on every DVD scan.

Tests:
- bdnav: MAX_OBJECTS count-cap rejection, MAX_STEPS non-convergence guard, and
  an end-to-end index.bdmv + MovieObject.bdmv resolver test (all red-green).
- drive/capture: cover the Renesas "SAT"-signature capture gate (match + miss).

Craft/docs:
- css/keyless: factor the duplicated LFSR0 output-tap into one helper.
- drive/capture: name the Renesas vendor CDB constants.
- labels/deluxe: use the named ANEWARRAY/NEWARRAY opcodes, not bare hex.
- dvdnav/vmcmd: tighten pub -> pub(crate). dvdnav/mod: mark the parked stub.
- disc/bluray: document that whole-clip extents are intentional (trimming is
  PTS-level in SeamPlan). disc/mod + error: doc/comment drift corrected.
2026-08-31 08:36:08 -07:00
Matthew Jackson 82511d84b6 audit: resolve macOS shim lifetime bug + close coverage/DRY gaps
- scsi/macos_shim: the DA claim used a stack DAClaimResult passed to an async
  callback and never released its semaphore — on the 5 s timeout that is both a
  use-after-free (the callback outlives da_hold) and a semaphore leak per drive
  open. Heap-allocate it, reference-count it (rc=2), and let whichever side
  finishes last release the semaphore and free. (Compiles; needs on-hardware
  DA validation.)
- bdnav/index: add a regression test that a title count over MAX_TITLES is
  rejected before any record read (untrusted index.bdmv count).
- dvdnav/nav: share ifo::MAX_TT_SRPT_TITLES instead of a second private copy so
  the two 99-title caps can never diverge.
2026-08-31 08:11:26 -07:00
Matthew Jackson 20afbfdccb audit: fixes from the v1.6.10..HEAD codeaudit sweep
Correctness:
- dvd: join the nav main-feature target on the REAL vts_title_num (carried on
  DvdTitle) instead of the title's position in the vec, which desyncs when a
  sibling PGC is dropped as unparseable or TT_SRPT is non-monotonic.
- diag: the "demoted: no-video" dump now gates on has_probable_video (the gate
  main_feature_order actually uses), not the stricter has_video.

Robustness:
- labels/deluxe: clinit_enum_field_names now caps retained bytes
  (MAX_CLINIT_LDC_BYTES), like its sibling, so a malicious BD-J .class with a
  huge CONSTANT_Utf8 field name can't grow the output unbounded.

Mux (FlagInterlaced correction cleanup):
- factor the PGS/FlagInterlaced deferred byte patches through one patch_bytes
  helper; name the InterlacedRewrite record; add ebml::void_element (validated)
  for the demotion Void; bound the muxed_* test scans to the pre-Cluster region.

Tests:
- add the FlagInterlaced majority tie-break test (tie stays progressive).
- de-vacuum the profile subtitle-count and meta_sink channel-count assertions.

Docs:
- clpi module doc described an EP-map/sector-range parser that was never
  implemented; corrected to what the code actually reads.
2026-08-31 07:51:05 -07:00
Matthew Jackson 6d78a36d22 lib: trim feature-gate comment blocks to satisfy comment-guard 2026-08-30 22:41:40 -07:00
Matthew Jackson a9d41f53f8 mux: correct FlagInterlaced from the whole-stream scan majority
FlagInterlaced was written from only the FIRST coded picture, but MPEG-2
progressive_frame is a per-picture flag: a progressive leader on an
interlaced feature (or an interlaced first picture on progressive film)
mislabeled the whole track, so players either deinterlaced progressive
frames or left interlaced frames combed.

Tally every picture's measured scan and correct the reserved FlagInterlaced
byte to the dominant scan at finish() -- the same reserve-then-patch idiom as
PgsForcedFixup and the AC-3/duration fixups. On a demotion to progressive the
now-stale FieldOrder element is Void'd. Also pins the interlaced direction at
the muxed-byte level and adds promote/demote regression tests.
2026-08-30 22:38:51 -07:00
Matthew Jackson c7c53989f7 Feature-gate the crate: scsi (transport-only) vs rip (full)
Split the crate into two additive features so a transport-only consumer
(freemkv-flash) can depend on the SCSI passthrough layer without pulling the
ripping dependency tree (serde, zip, rayon, aes, freemkv-unlock, …):

- scsi = the SCSI passthrough transport (src/scsi/ + error): ScsiTransport,
  CDB builders, per-OS backends + the macOS cc shim, sense parsing. Pulls only
  tracing + the always-on libc target dep.
- rip  = the full ripping tree (drive/disc/UDF/MPLS/CLPI/AACS/CSS/mux/keydb +
  the freemkv-unlock seam) and all its heavy deps. rip implies scsi.
- default = ["rip", "scsi"], so the full build stays byte-for-byte identical for
  the existing consumers; transport-only users set default-features = false,
  features = ["scsi"].

All ripping modules are gated #[cfg(feature = "rip")]; error + scsi stay always
compiled. Two read-path timeout consts are rip-gated so a transport-only build
does not warn on unused.
2026-08-30 22:10:05 -07:00
dependabot[bot] 7ec2dbfbcb build(deps): bump codecov/codecov-action from 5 to 7
Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 5 to 7.
- [Release notes](https://github.com/codecov/codecov-action/releases)
- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)
- [Commits](https://github.com/codecov/codecov-action/compare/v5...v7)

---
updated-dependencies:
- dependency-name: codecov/codecov-action
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-30 21:53:05 -07:00
avon662 16e5b5ee38 Write FlagInterlaced from the measured picture, not the declared resolution
MkvTrack::video sets `interlaced` from the source scan's DECLARED
resolution. An NTSC DVD's IFO declares 480i whether or not its pictures
are coded progressive -- film and animation routinely are -- so
progressive content ships as FlagInterlaced=1, and every player that
honours the flag runs a deinterlacer over progressive frames, softening
every frame of an otherwise bit-exact remux.

The measurement was already there and already trusted for the sibling
property. `apply_coding_to_track` receives the first coded picture's
PictureInfo and derives FieldOrder::Progressive from it
(progressive_sequence / progressive_frame, ISO/IEC 13818-2 6.3.5 /
6.3.10) -- it used that verdict only to clear FieldOrder and then
discarded it. It now also corrects the scan type, exactly as a measured
TFF/BFF already overrides the declaration.

Measured on a real NTSC DVD: ffmpeg's `idet` reports 100% progressive
(multi-frame detection: 0 TFF, 0 BFF, 0 repeated fields) on titles that
shipped FlagInterlaced=1. progressive_frame is set on every picture
coding extension in the stream while progressive_sequence stays 0, which
is the ordinary shape for a film or animation DVD.

Video payload is unaffected -- byte-identical before and after; only the
track header changes.
2026-08-30 21:47:19 -07:00
Matthew Jackson d40540ce9e v1.6.13: bump version (freemkv-unlock git-pinned for the tag) 2026-08-28 15:48:58 +00:00
Matthew Jackson ba14d66c18 Release 1.6.13: capture additional drive buffers during info --share 2026-08-27 21:22:11 -07:00
Matthew Jackson f16abe1cf4 restore freemkv-unlock path dep for local dev (post-v1.6.12) 2026-08-27 22:57:33 +00:00
Matthew Jackson 477c0da6dd v1.6.12: bump version (freemkv-unlock git-pinned for the tag) 2026-08-27 22:57:29 +00:00
Matthew Jackson 41c5b8b785 changelog: 1.6.12 (unreleased) 2026-08-27 13:11:27 -07:00
Matthew Jackson 9f43ae94d3 Comment and Documentation Cleanup
- Cap every inline `//` comment block at 3 lines; distil the rationale
  rather than delete it. Enforced by a new comment-guard CI check.
- Fix stale and orphaned rustdoc: broken intra-doc links (renamed or
  removed items), doc blocks that had merged onto the wrong item (leaving
  the intended item undocumented), and doc-contradicts-code drift.
- Add a doc-links CI check (cargo doc, deny broken intra-doc links) so
  both classes of comment rot are caught going forward.
2026-08-27 12:07:53 -07:00
Matthew Jackson 9ed6f86255 Fix macOS optical E1000: hold a DiskArbitration claim for the session
The macOS SCSI transport took exclusive SCSI access plus a one-shot
`diskutil unmountDisk`, but never claimed the disc from DiskArbitration.
diskarbitrationd was therefore free to remount the volume mid-rip (a
Spotlight re-probe is the usual trigger), yanking the device out from
under an in-progress read; the failure surfaced to the user as E1000.

Hold a DADiskClaim plus a mount-approval dissenter scoped to our disc for
the transport lifetime, so the OS cannot remount it until shim_close().
Link the DiskArbitration framework in build.rs.
2026-08-27 11:17:01 -07:00
Matthew Jackson 70ae7b6c10 restore freemkv-unlock path dep for local dev (post-v1.6.11) 2026-08-25 23:17:11 -07:00
Matthew Jackson 3f03c6db06 v1.6.11: bump version (freemkv-unlock git-pinned for the tag) 2026-08-25 23:17:07 -07:00
Matthew Jackson 562bb8bee5 docs: neutralize tool-comparison and technique wording (opsec) 2026-08-25 22:57:01 -07:00
Matthew Jackson 7c13ed32b5 fix: align drive-prep unlocker token with unlock rename; neutralize product name 2026-08-25 22:54:38 -07:00
Matthew Jackson 574aa2c506 docs: neutralize retail title references (opsec) 2026-08-25 22:45:57 -07:00
Matthew Jackson 3912762b52 docs: neutralize comment provenance and external references (opsec) 2026-08-25 22:31:19 -07:00
Matthew Jackson 028768e71b docs: 1.6.11 changelog 2026-08-25 22:08:48 -07:00
Matthew Jackson bf4e5ac395 docs: standardize README badges (License, CI, codecov) 2026-08-25 20:26:28 -07:00
Matthew Jackson 11948d513b nav: taint control-transfer decisions only, and track SPRM-laundered GPRMs
Refine the First-Play SPRM-taint abstain to fix two defects:

- Over-abstention: a tainted compare that guards a SetGPRM store (not a
  control transfer) no longer forces abstention. The predicate only
  decides whether the store runs; a following unconditional dispatch is
  still resolvable. Taint now gates only JumpTT/Goto (the control
  transfers the resolver follows); a store under a tainted guard taints
  its destination GPRM (unknown post-state) instead of abstaining.

- Under-abstention: an SPRM laundered through a GPRM (SetGPRM g = SPRM,
  then a later GPRM-only compare on g) previously evaded taint and let
  the resolver commit to the cold-machine arm. Add per-GPRM taint bits:
  a store propagates its source's taint (SPRM or tainted GPRM) and a
  concrete/immediate store clears it; eval marks a compare tainted when
  it reads a tainted GPRM.

Adds red-before-green tests for both plus taint-clear guards.
2026-08-25 20:26:03 -07:00
Matthew Jackson 7d5c0a5c41 Fix round-3 audit findings: VM operand decode, MPLS field offset, log escaping, nav SPRM abstain, tests
1. dvdnav/vmcmd: the register operand of the if_v1/if_v3 compare layouts is the
   LOW byte of the reg-or-immediate field (b[5] / b[7]), not the high byte
   (b[4] / b[6]). Fix both layouts and their comment; add register-mode KATs
   with distinct high/low bytes.

2. mpls: connection_condition is the low nibble of PlayItem byte[10]
   (byte[9] is fully reserved), corroborated by the byte-11 stc_id and
   byte-12 in_time reads. Relocate every fixture's encoded value and update the
   regression test to assert the reserved byte does not leak.

3. labels/deluxe: escape the two disc-derived class-name tracing fields via the
   Debug path, matching the other log sites.

4. disc/dvd: add an integration test that drives scan_dvd_titles with a resolved
   First-Play nav target so the 1-based promotion join is verified end-to-end.

5. bdnav/vm: add unit tests for the SET/SWAP immediate gate — full register
   exchange, and that an immediate-flagged operand is never a write target.

6. dvdnav/nav: honor the documented contract — a First-Play compare that reads a
   system parameter is undecidable at scan time, so eval now taints on SPRM reads
   and the executor abstains rather than committing to the cold-power-on arm.
   GPRM reads never taint.

7. labels/deluxe: narrow four now-internal helpers from pub(crate) to private.
2026-08-25 20:26:03 -07:00
Matthew Jackson 1ede962aec ci(coverage): authenticate Codecov upload with org CODECOV_TOKEN 2026-08-25 19:32:30 -07:00
Matthew Jackson 014ea870a4 ci(coverage): add --ignore-run-fail so instrumentation flake never blocks upload 2026-08-25 19:04:41 -07:00
Matthew Jackson bdd2876d78 ci: add non-required Codecov coverage job, config, and badge
Add an additive coverage job to CI that generates lcov via cargo llvm-cov
and uploads to Codecov (tokenless). Mirrors the test job's pinned 1.94
toolchain and dual freemkv-unlock checkout for the path dep. Informational
only: no gate depends on it. Adds codecov.yml and a README badge.
2026-08-25 18:54:23 -07:00
Matthew Jackson 44de7a8d12 harness: fuzz the Deluxe bytecode decoder directly, not through a dead zip open
Audit round-2 (fix-adversary): the round-1 deluxe sweep fed the zip local-file-
header magic, but ZipArchive::new validates the end-of-central-directory at the
buffer's end — which the byte generators (front magic + noise) can never
synthesize — so the archive open always failed and decode_binding was never
reached: false coverage for the very bytecode-decoder panic path it guarded.
Drive decode_binding_class directly on a parsed .class with an empty master-enum
table instead, so the symbolic stack machine is actually exercised.
2026-08-25 18:45:28 -07:00
Matthew Jackson 552076a943 disc/labels/harness: audit round-1 batch — DoS bound, fuzz coverage, API surface
Continuing the /codeaudit HEAD..v1.6.10 round:

- disc/mod.rs rank_titles: bound the O(n^2 * clips) composite scan to a
  sub-second work budget (both title count and per-title clip count are
  untrusted disc metadata); above it, skip composite classification and degrade
  to the has-video + canonical order. Reorder the per-pair checks so the O(len)
  is_subset runs only after the cheap video/size gates. (regression test)
- bdnav/vm.rs SWAP: honour the imm_op1/imm_op2 store gates so a SWAP to an
  immediate operand does not corrupt a register.
- harness.rs: fuzz the previously-uncovered new parsers — the Deluxe bytecode
  decoder, the Fox dcx.xml scanner, and the combined bdnav index+mobj+VM resolve
  path (labels::{deluxe,fox} and bdnav::vm made pub(crate) for in-crate fuzzing).
- dvdnav: pub(crate) the module + submodules (was pub) to match bdnav and stop
  leaking the DVD IFO/VM internals as public API; drop an unused re-export.
- labels: escape disc-derived strings in diagnostics (Debug not Display) so a
  hostile disc cannot inject terminal control sequences into logs.
2026-08-25 18:33:37 -07:00
Matthew Jackson 3ecbeb7eba disc/bdnav: audit round-1 — VM title/PSR faithfulness + main_title never-panics
From the /codeaudit HEAD..v1.6.10 round (each verified against the libbluray
HDMV VM reference, and design-refuted SOUND on two models):

- bdnav/vm.rs title_obj: resolve First Play via the spec sentinel 0xFFFF (was
  num_titles+1). A JumpTitle 0xFFFF now reaches First Play, and an out-of-range
  JumpTitle(N+1) correctly abstains instead of mis-jumping there.
- bdnav/vm.rs wr: refuse stores to PSR-tagged registers (matches _store_reg),
  so an illegal SET/SWAP to a PSR is a no-op and later compares agree with a
  real player instead of diverging.
- disc/profile.rs main_title(): return Option<&TitleProfile> instead of indexing
  titles[0], which panicked on a disc that scanned to zero titles.

Regression tests added, each shown red-before-green.
2026-08-25 18:16:17 -07:00
Matthew Jackson b405154fd0 ci: run the consumers job at the highest dependent MSRV (1.98)
The MSRV split raised the dependents above libfreemkv's own floor — the
bins (freemkv, autorip, bdemu) are 1.98 and freemkv-keysources is 1.97 —
so the consumers job could no longer build them at libfreemkv's 1.94
toolchain (`rustc 1.94.0 is not supported by ... freemkv-keysources
requires rustc 1.97`), leaving dev red since the MSRV change. Pin that one
job to 1.98, which satisfies every dependent's floor. libfreemkv's own
lib/lint/test jobs stay at its 1.94 MSRV.
2026-08-25 13:49:35 -07:00
Matthew Jackson 8a630c8839 disc: payload-plausibility floor on nav/authoring feature promotion
A nav- or authoring-resolved playlist could promote a video-bearing but
negligible-payload branch above the substantial feature it sits beside:
Spotlight (2015) UHD navigates to a Dolby-Vision seamless-branch playlist
(00001, 0.4 GB across 102 tiny clips) instead of the 91.5 GB single-clip
feature (00002). A player plays the branch; a rip wants the feature.

Require a nav/authoring-promoted title to also carry at least 10% of the
largest video-bearing title's bytes; otherwise selection falls through to
the composite/size ranking, which recovers the real feature. Adds the
Spotlight-shaped regression test.

Verified against the 88-disc hoard: title selection is identical to the
prior release on every disc except the one it fixes (Spider-Man 3 UHD,
00245 wrapper decoy -> 00001 feature).

Also clears three 1.97 clippy lints in the HDMV VM and drops two
reference-implementation filename citations from the bdnav module docs.
2026-08-25 13:44:04 -07:00
Matthew Jackson 6c00090308 labels/fox: wire the extracted feature-playlist id into the selection hint 2026-08-25 13:01:25 -07:00
Matthew Jackson 7f9f5a9d3e Merge branch 'label-fox' into integrate-1.6.11 2026-08-25 13:00:06 -07:00
Matthew Jackson b3c23c19ee Merge branch 'label-deluxe' into integrate-1.6.11 2026-08-25 12:59:54 -07:00
Matthew Jackson f3923cf587 Merge branch 'dvd-nav' into integrate-1.6.11 2026-08-25 12:59:48 -07:00
Matthew Jackson 002912abb7 dvdnav: resolve DVD main feature by following First-Play navigation
Implement the DVD navigation feature resolver (#40): read the disc's own
First-Play program out of VMGI (VIDEO_TS.IFO) and execute it on a bounded
register machine to identify which title a player would land on, then prefer
that title in DVD main-feature selection.

- src/dvdnav/nav.rs: new module.
  * Parses the VMGI First-Play PGC command table (FP_PGC pointer at VMGI+0x84,
    PGC command-table pointer at PGC+0xE4, pre-command list) and TT_SRPT
    (title -> VTS/VTS_TTN, pointer at VMGI+0xC4).
  * Executes the pre-command list with the existing vmcmd decoder over 16 GPRMs
    + SPRMs from a cold machine: SetGPRM (mov/add/sub/mul/div/mod/and/or/xor),
    compare predicates, Goto, Break/Exit, and JumpTT (title dispatch). Menu /
    sub-domain jumps and intra-domain links are treated as undecidable.
  * resolve_from_vmg(&[u8]) is pure and total; resolve_main_title(reader, udf)
    reads the IFO and returns the resolved (title, vtsn, vts_ttn) or None.
  * Every offset/count is bounds-checked; a step budget bounds non-convergence.

- src/disc/dvd.rs: scan_dvd_titles now resolves the nav main title and returns
  its playlist_id alongside the titles; test call sites updated for the tuple.
- src/disc/mod.rs: the DVD nav pick feeds sort_titles_by_main_feature as
  nav_feature, the same seam the BD path uses, guarded to a video-bearing
  title. None -> today's leading-cell heuristic stands (strict improvement).
- src/harness.rs: add resolve_from_vmg to the never-panic sweep.

Tests: 11 nav unit tests (unconditional/conditional/set-then-dispatch convergence,
menu/empty/out-of-range/self-loop/truncation/overflow abstention) built to the
IFO spec using the real First-Play command bytes from the existing decoder KATs
(unconditional JumpTT 1; menu entry). Full suite green on 1.94.
2026-08-25 12:55:34 -07:00
Matthew Jackson b5fd193d54 disc: add normalized DiscProfile output surface
Add a thin, format-agnostic typed VIEW over the scanned Disc model so
downstream reads `profile.titles[i].subtitles[j].forced` (and analogous
audio/video flags) for any disc with no per-format conditionals.

New src/disc/profile.rs:
- DiscProfile / TitleProfile / VideoTrack / AudioTrack / SubtitleTrack
  (serde Serialize + Deserialize), re-exported from the disc module.
- DiscProfile::from_disc / From<&Disc>, Disc::profile(), and accessors
  (DiscProfile::main_title, TitleProfile::video/audio/subtitles).
- Splits each title's Stream enum into three typed vectors; flattens
  normalized fields (language default "und", forced/sdh from the subtitle
  qualifier, commentary/descriptive from the audio purpose, label -> name).
- Hoists derived flags so downstream never recomputes: main_title (the
  pre-sorted main feature at index 0), is_main, and the default
  audio/video track (first non-secondary, keep only the first), subtitle
  default always false.

Make the profile types the single serialization source of truth: the
json:// sink now serializes TitleProfile instead of hand-building JSON,
so json:// and any downstream share one schema. Stream arrays are split
by kind and qualifier/purpose decomposed into booleans; json://-only
extras (clips, full chapter_marks list) are spliced back on. meta_sink
tests reconciled to the new schema.

Tests: profile normalization across BdTs + MpegPs titles, typed-vector
split, populated fields, hoisted main/is_main/default flags,
qualifier/purpose mapping, and a serde round-trip.
2026-08-25 12:43:24 -07:00
Matthew Jackson 49011d577f labels/deluxe: decode real per-stream labels, emit High (Universal + others)
The Deluxe parser detected com/bydeluxe discs but emitted nothing, so
mpls_universal won at Low. Grounding the decoder on real Universal discs
(Fast Five / F9 / Bourne, studio="uni") surfaced three defects that
kept the full Phase A-D pipeline from ever resolving a stream on any
real disc, all fixed here:

1. Master enums were keyed by the ZIP ENTRY name ("pd.class") but the
   binding class references enum constants by JVM INTERNAL name
   (getstatic pd.<field>), so Phase C's getstatic count never matched
   the identified enums and found zero binding classes. Key master enums
   by this_class_name instead.
2. MasterEnumTable resolved getstatic operands by enum VALUE string
   ("English") but the operand is the obfuscated static-FIELD name
   ("a"), so every language/purpose ordinal came back unresolved.
   Capture the ordered putstatic field names (clinit_enum_field_names)
   and key the resolver on them.
3. The Language-enum fingerprint demanded 70+-4 values; Universal's
   enum has 65, so Phase A rejected it outright. Give each fingerprint
   its own count window (Language widened to span 65 and 70); the
   ordered 4+-string prefix stays the real discriminator.

Also: model anewarray/newarray so array construction keeps the symbolic
stack aligned, and skip constructors with an array parameter so the
Universal title-wrapper object (oq.<init>(...,[Lnp;,[Lwb;,...)) is not
mistaken for a subtitle binding. Recover SDH/RNIB from the language
display name (Universal encodes it as a distinct Language value, not a
Purpose). Add config.xml <TitleConfig studio="..."> detection. Raise
confidence to High when the binding table decodes >=1 stream; the
mpls_universal floor still back-fills un-named STN slots.

Fixtures: verbatim pd/lp/tl .class files from FastFive.iso decode to
English/French/Spanish + English Commentary (Dolby Digital) audio and
per-title eng/spa/fra subtitles. Validated end to end: labels-analyze on
FastFive/F9/Bourne now reports parser=deluxe High with real per-stream
languages, names, purposes and codecs instead of mpls_universal Low.
2026-08-25 12:25:38 -07:00
Matthew Jackson f8e39abe54 labels: harden class_reader/jar as first-class .class format readers
Treat Java bytecode as a documented format we READ (never execute), a
peer of mpls/clpi. The class reader's bounds checking was already sound
(typed Reader with checked u8/u16/u32/u64 + slice, saturating switch
arithmetic, iterator halt-on-truncation); this adds the untrusted-input
entry points the never-panic harness was missing and the malformed-input
regression tests the format warranted.

- harness: add class_reader (magic CAFEBABE) and jar (zip PK header)
  sweep targets. The class_reader target also drives the accessors a
  label parser exercises (this_class_name, member Code walk, cp_index,
  member_ref, load_constant_display) so their indexing is fuzzed, not
  just the structural parse. The jar target opens the sweep bytes as a
  zip and walks every .class through for_each_class exactly as deluxe/dbp
  do.
- class_reader tests: oversized constant_pool_count and interfaces_count
  reject with UnexpectedEof (u16 counts keep with_capacity bounded);
  out-of-range cp indices return None from every accessor; a Code
  attribute whose declared code_length runs past its body is rejected.
2026-08-25 12:25:19 -07:00
Matthew Jackson 65d719ee5d disc: wire bdnav as the top nav-feature tier in main-feature selection
scan_with runs the bdnav VM for BDMV discs and feeds the video-gated result
as the new top MAIN_FEATURE_ORDER key (nav-feature > authoring-feature >
standalone > has-video). A nav result is only honoured for a video-bearing
title, so a mis-resolve can never reopen the decoy hole; when the VM abstains
(BD-J, non-convergence) the structural/heuristic order decides exactly as
before. Adds nav selection tests and updates the diag decision reason.
2026-08-25 12:19:16 -07:00
Matthew Jackson c3bdbdcfc9 bdnav: faithful HDMV navigation VM (index.bdmv + MovieObject.bdmv)
Add src/bdnav: a libbluray-faithful, bounded, never-panic HDMV navigation VM
that runs First-Play with correct power-on PSR/GPR state to resolve the
playlist the disc itself plays as the feature. One generic VM handles the
obfuscated Sony dispatcher and every other HDMV disc with no vendor
special-casing; it abstains (None) on the BD-J boundary, malformed data, or
non-convergence. index/mobj parsers added to the never-panic harness.
2026-08-25 12:19:16 -07:00
Matthew Jackson 3f4e69f7b8 labels: add Fox dcx.xml vendor parser
Older Fox discs ship a loose plain-XML manifest at
/BDMV/JAR/<id>/dcx.xml (root <dcx>) that names every feature stream's
language, editorial purpose and forced/SDH state in attributes — fully
parseable and previously unowned (2 Fox discs matched no parser, 11 were
mis-owned by the deluxe stub).

New src/labels/fox.rs:
  - detect(): loose dcx.xml under /BDMV/JAR/*/ (cheap dir walk), falling
    back to a com/foxbd/ jar central-directory prefix so newer Fox discs
    are attributed to Fox even though their bytecode form isn't decoded
    yet (Phase 2, sketched in-file).
  - parse(): scopes extraction to one <playlist name="feature"> element
    (the richest, to avoid merging regional 00800/00801 STN tables),
    reusing labels/xml.rs. audio type=rnib -> Descriptive, comment* ->
    Commentary; subtitle form=sdh -> Sdh, type=embed -> Forced; nested
    id="NN" -> 1-based per-type STN slot. High confidence.
  - Feature playlist id recorded via tracing::debug for now; dev's
    ParseResult has no feature_playlist field (that FeaturePlaylistHint
    slot lives on selection-hardening). An in-code note documents wiring
    it at reconciliation, matching paramount::parse.

Registered ahead of mpls_universal, alongside the High-confidence vendor
parsers; registry-order lock test updated.

Tests embed the real Life of Pi dcx.xml (/BDMV/JAR/05001), asserting the
00800 feature selection + id, all 11 audio labels (incl. the eng rnib
descriptive track) and 11 subtitle labels (sdh/forced/none), the
no-cross-playlist-merge scope rule, plus negative-detect cases.
2026-08-25 12:09:37 -07:00
Matthew Jackson 89b03610e7 disc: demote play-all/wrapper composites in main-feature selection
The has-video gate alone fixed Spider-Man 3 UHD only by accident: the
00245 decoy = [bumper][00001 feature clip][outro], and streams come from
PlayItem 0 (the bumper) only, so it reported v=0. That is fragile — a
video-bearing composite [feature][outro] still beat the feature on size
(audit R2), and a real feature with a non-video first PlayItem would be
wrongly disqualified (R6).

Add structural clip-set composite detection: a title whose clip set
properly contains another substantial video title's whole clip set is a
play-all/wrapper composite and is demoted below the standalone title it
wraps — independent of stream counts, physical size, and disc capacity
(which is 0/unknown when READ CAPACITY fails, disabling the oversize
gate). Replace the has-video disqualification with the more-permissive
has_probable_video (real video, or size+duration+clips consistent with
video) so a bumper-lead feature is never disqualified while streamless
menus still are. Corroborate the authoring hint by duration (a hint at a
short branch/trailer can't override the feature) and require real video
streams for it. Precedence: authoring -> standalone-over-composite ->
has-video -> canonical (with a reserved top slot for the future BD-nav
key). Unify the session.rs key-sampling/verdict pickers and
image_crack_extents on has_probable_video.

Tests: video-bearing composite (T2), oversize composite with unknown
capacity (T3), short-branch hint not honoured (T4), bumper-lead feature
not disqualified (T6), all-streamless deterministic + single-source pick
(T7), equal-clip twins not composite, video-only title. T2/T3/T6 fail
under the prior has-video-only logic.
2026-08-25 11:54:04 -07:00
Matthew Jackson a7cbb3afcd disc: harden main-feature selection against streamless decoys
Main-feature selection ranked purely on physical size -> duration ->
audio (canonical_title_order), with no requirement that the chosen title
carry a decodable video stream. A playlist-obfuscation decoy can inflate
a streamless playlist past the real feature on size AND duration (Spider-
Man 3 UHD: 00245.mpls, 78.4GB/8350s/v=0 beat 00001.mpls, 76.5GB/8038s/
v=1), so the mux ran a title with zero streams -> zero frames -> E6008.

- Add Disc::main_feature_order layering two gates atop canonical_title_order:
  authoring-feature (a title the disc's menu authoring designates AND that
  has video) wins outright; then has-video demotes any streamless title
  below every title that has video. scan_with re-sorts with it after labels
  run so titles[0] is the selected feature. New MAIN_FEATURE_ORDER_KEYS +
  DiscTitle::has_video().
- Surface the authoring feature-playlist id: labels::FeaturePlaylistHint,
  captured from Paramount playlists.xml <playlist name="Feature" id=..>,
  threaded through ParseResult/extract; labels::apply now returns it.
- Unify pickers so key-sampling and the structural verdict (session.rs) and
  image_crack_extents prefer the has-video feature, not a streamless decoy.
- diag: decision row logs video= count + main_feature_order reason and a
  tag=decision.demoted line per no-video title (closes the field-diagnosis
  gap this incident exposed).

Tests: main_feature_order_demotes_streamless_decoy (pre-gate vs post-gate
contrast), _honours_authoring_hint, authoring_hint_never_promotes_a_
streamless_title, feature_playlist_hint_matches_on_id_or_filename; diag
reason test updated. QA green on 1.94 (fmt/clippy -D warnings/test).
2026-08-25 11:23:44 -07:00
Matthew Jackson f75766b4d5 chore: lower MSRV to 1.94, extend disc provenance tests, tidy comments
Lower the crate MSRV from 1.97 to 1.94 -- the lowest toolchain on which
`cargo build`, `cargo clippy --all-targets -D warnings`, and
`cargo test --tests` all pass clean. Dependencies floor the build at 1.90
(freemkv-unlock 1.90, zip 1.88), but clippy is only warning-clean from
1.94 (older clippy has a `duplicated_attributes` false positive on
`#![cfg(test)]` and flags a now-documented `let_and_return`). Update
`rust-version` and every `dtolnay/rust-toolchain` CI pin to 1.94.0, and
add a CHANGELOG Unreleased entry.

Extend the byte-provenance coverage added in f7d38ca (currently one
TS-path test) with two PS-path tests driving the real `Mpeg2Parser`:
- `ps_stream_stamps_source_provenance`: the PS path forwards `ps.source`
  onto `PesPacket`, so demuxed frames carry `Some(SourcePos)`.
- `ps_stream_carries_feed_base_across_reads`: the cumulative `fed_bytes`
  base carries ACROSS read buffers -- a frame decoded from the second
  sector is stamped `source.byte >= 2048`, not reset per read.

Also record the f7d38ca provenance fix in the CHANGELOG, inline a
`let_and_return` test binding, and trim a redundant sentence from
`PsDemuxer::feed_at`'s doc comment.
2026-08-25 10:35:47 -07:00
dependabot[bot] 1c313fe882 Bump actions/checkout from 5 to 7
Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v5...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-24 08:52:57 -07:00
Pascal Harprecht f7d38caf0b mux/disc: pass feed base offset to demuxer to stamp byte provenance
On direct disc reads (`disc://...`), `DiscStream` called `demuxer.feed(&self.read_buf[..bytes])`
instead of `feed_at(buf_base, ...)`. Without a running feed base offset, demuxed `PesPacket`s
had `source = None`, stripping byte provenance from downstream `PesFrame`s.

Under a `SeamPlan`, when `src_byte` is `None`, `SeamPlan::place` falls back to a timestamp
heuristic across clips. On multi-clip playlists where a subsequent clip starts at the same
base timestamp as the initial clip (`in_time = 600s`), B-frame PTS reorder dips (where
`raw_ns < last` and `|raw_ns - next_in| <= 250ms`) falsely triggered a `stepped_back` clip
transition on the first reordered frame. Because the subsequent clip's span was short (e.g. 1s),
all subsequent video frames across the rest of the title were rejected and dropped as outside
clip marks. Audio tracks (which have monotonic PTS without reordering) were unaffected.

Track `fed_bytes` in `DiscStream` and route through `feed_at` for both TS and PS demuxers,
forwarding `ps.source` to `PesPacket` on the PS path.

Test: `mux::disc::tests::stream_surface_tests::disc_stream_stamps_source_provenance`
2026-08-24 08:52:19 -07:00
Matthew Jackson e98e98c8fd TrueHD: resync forward to next major sync on a source discontinuity
MLP/TrueHD carries predictor + restart state across access units, so after a source transport-stream discontinuity (continuity-counter gap) the parser was splicing post-gap audio mid-stream, decoding against stale state and producing a decoder-choking seam (a burst of "restart header sync incorrect" / "Invalid blocksize" errors that could flag the whole track corrupt).

Honor the source's own gap signal: drop forward to the next CRC-valid major-sync AU (a clean re-init point) before resuming, guarded so it never arms before a validated baseline exists (no stream-head whole-track drop). Mirrors the video keyframe resync and the ResyncGate design. Corrects the "audio has no inter-frame refs" comments in truehd/dts/resync. Adds a regression test.
2026-08-23 16:30:29 -07:00
Matthew Jackson 7c085db133 restore freemkv-unlock path dep for local dev (post-v1.6.9) 2026-08-22 08:34:31 -07:00
Matthew Jackson 587e26353c v1.6.9: bump version (freemkv-unlock git-pinned for the tag) 2026-08-22 08:34:26 -07:00
Matthew Jackson 3eb3e649b1 changelog: 1.6.9 unified-release entry 2026-08-22 08:24:55 -07:00
Matthew Jackson b01505983c restore freemkv-unlock path dep for local dev (post-v1.6.8) 2026-08-21 12:54:53 -07:00
Matthew Jackson 87bc07dad8 v1.6.8: bump version (freemkv-unlock git-pinned for the tag) 2026-08-21 12:54:48 -07:00
Matthew Jackson 743c5830b6 docs: 1.6.8 changelog (unified release) 2026-08-21 12:47:34 -07:00
Matthew Jackson 80c8e4ca24 ci: require a CHANGELOG entry for the crate version on main
Adds a 'changelog' job that fails when Cargo.toml's version has no matching
'## [X.Y.Z]' heading in CHANGELOG.md, hard on main and PRs into main, warning
on dev/qa — so the unified release can't ship a crate with a blank changelog.
2026-08-21 11:23:45 -07:00
Matthew Jackson f49e79c198 restore freemkv-unlock path dep for local dev (post-v1.6.7) 2026-08-21 09:31:40 -07:00
Matthew Jackson 231cbf3d76 v1.6.7: bump version (freemkv-unlock git-pinned for the tag) 2026-08-21 09:31:31 -07:00
Matthew Jackson 21c7b12e2b docs: 1.6.7 changelog (unified release, autorip-driven) 2026-08-21 09:23:55 -07:00
Matthew Jackson 799039df21 restore freemkv-unlock path dep for local dev (post-v1.6.6) 2026-08-20 20:03:48 -07:00
Matthew Jackson 497b7726dc v1.6.6: bump version (freemkv-unlock git-pinned for the tag) 2026-08-20 20:03:44 -07:00
Matthew Jackson 94b549a9d8 docs: 1.6.6 changelog 2026-08-20 19:56:40 -07:00
Matthew Jackson fc2d35a19c v1.6.5: bump version (freemkv-unlock git-pinned for the tag) 2026-08-20 06:35:01 +00:00
Matthew Jackson 3c51190c32 docs: 1.6.5 changelog 2026-08-19 21:20:42 -07:00
Matthew Jackson dfe0390445 mux/h264: treat open-GOP intra access unit as a resync keyframe
The B1 resync gate (mux/resync.rs) arms on a concealed-gap / clip-seam
discontinuity and drops video frames until it sees a keyframe. The H.264
parser flagged a keyframe only on an IDR slice (NAL type 5), so on a BD
open-GOP tail whose only random-access point is a non-IDR I-frame (the
recovery point) the gate never disarmed and dropped every remaining frame
to EOF — truncating the video by up to a GOP-plus tail and failing the
cli-acceptance timeline_sound strict span check for BD.

Promote an intra-coded access unit (measured coding_type == I) to a
keyframe, alongside the existing IDR-by-NAL-type flag. This is the H.264
equivalent of HEVC's open-GOP IRAP (CRA) handling, which the HEVC parser
already flags by NAL type — H.264 has no such NAL tag, so the measured
intra coding type is the signal. Keying off coding type rather than a
recovery_point SEI covers every signalling: an SEI-tagged recovery point
(which is intra by construction) and a disc that marks the recovery point
by GOP structure alone. Consistent with videomap.rs's model that a
keyframe IS the intra decode-restart point; the open-GOP anchor now also
gets the self-contained SPS/PPS re-assert, mirroring HEVC at CRA. IDR
stays flagged by NAL type, so a slice whose header fails to parse
(coding_type == None) is unaffected. HEVC, DVD/MPEG-2 unchanged.

Tests: open_gop_intra_access_unit_is_a_keyframe (red before this change),
inter_coded_access_unit_is_not_a_keyframe (precision guard).
2026-08-19 13:32:30 -07:00
Matthew Jackson 8678a2bf94 drive: select the platform module once, not per function
find_drives and resolve_device each carried a three-arm cfg(target_os) block in
their bodies. Alias the current platform's module once (`use linux as platform`
under a single cfg) and let both dispatch through `platform::…`, so the function
bodies are cfg-free and a new entry point cannot forget an arm.
2026-08-19 09:10:26 -07:00
Matthew Jackson d953b46023 Stop the HD-DVD VTI read and the extract progress channel from hiding loss
Two audit fixes plus two consistency cleanups, all in the changed 1.6.5 surface:

- hddvd: the VTI clip-order read used `.ok()`, flattening an unreadable
  authored order (a scratched sector under the .vti — the name came from the
  directory, so it is never "absent") into "no order" with no diagnostic. The
  sibling clip-extent arms log every read failure with its code; this one now
  does too, then falls back to the per-clip heuristic exactly as an unauthored
  disc would. Behaviour is otherwise unchanged; the loud line is the point.

- extract: the progress sink hardcoded bytes_unreadable_total: 0 and counted
  every zero-filled hole as good, so a progress-only consumer saw a holed
  extraction climb to a clean 100%. Thread the running unreadable total through
  and report the real good/unreadable split. The authoritative ExtractResult was
  already truthful; only the live channel lied.

- labels: correct a stale comment that described testlog's old lock-based
  capture; it now installs one global subscriber and routes to a thread-local
  sink.

- ps: import SYSTEM_HEADER from consts instead of re-declaring 0xBB, matching
  its sibling stream-id constants and the file's single-source rule.
2026-08-19 09:10:26 -07:00
Matthew Jackson 443ac6bcb1 Fix two tests that flaked under the parallel harness
testlog: capture through a single global subscriber that routes each
event to a thread-local sink and returns Interest::sometimes, so
tracing's global per-callsite interest cache can never short-circuit a
live capture when another thread rebuilds it. The old scoped
with_default lost events at random in the full suite (an empty capture
failed the log-accounting assertions ~1 run in 15).

labels: capture the escaping test through testlog instead of installing
its own process-wide subscriber, which hard-cached every foreign
callsite "off" and poisoned the captures above.

dirimage: name scratch directories from a process-monotonic counter
rather than SystemTime, which resolves to only a microsecond — two of
the parallel tests sharing a tag drew the same value, collided on one
directory, and the first to finish removed it out from under the
other's reads.
2026-08-19 01:18:38 -07:00
Matthew Jackson 794393f24e CSS: fix the decrypted-HD-DVD false E7023 at the detection layer, not the public API
Commit b02c402 ("key the DVD crack on the disc, not on the container") fixed a
real bug — a decrypted HD-DVD hit E7023 (CssKeyMissing) because the per-title
CSS crack keyed on the MPEG-PS container, which DVD and HD-DVD share — but did
it by adding a required `disc_format: DiscFormat` parameter to the PUBLIC
`DiscStream::new` and `build_iso_pipeline`, a `disc_format` field to
`MuxInput::Iso`/`Live`, and `DiscFormat::may_have_css`, threading the axis down
through mux/driver and mux/resolve. That changed the public API and broke every
downstream caller's compilation (freemkv-engine's integration test now needed 8
args, autorip's MuxInput arms a new field). 1.6.4 shipped and worked with these
exact signatures; a bug fix must not reshape them, and needing a whole
disc-format plumb for HD-DVD was a code smell.

Revert all of that plumbing (public signatures restored to their pre-b02c402
form; no `disc_format` parameter or field, no `may_have_css`, anywhere), and fix
the ACTUAL bug where it lives: the scramble-detection heuristic.

Root cause: `is_scrambled_pack` counted a sector as CSS scramble evidence on
pack-start (00 00 01 BA) + bits 4-5 of byte 0x14. Offset 0x14 is only the PES
scrambling-control field when the sector is a genuine elementary-stream pack. An
HD-DVD `.evo` RDI navigation pack is private_stream_2 (stream_id 0xBF), an
MPEG-PS pack exactly like a DVD VOB, whose byte 0x14 is raw nav payload that
routinely has bits 4-5 set. On a decrypted HD-DVD (None keys, MPEG-PS, so it
reaches the crack) those nav packs flipped the scan's `saw_scrambled` flag; the
crack then found no key — there is no CSS on an HD-DVD — and the scan returned
ScrambledUncracked, hard-failing a good disc with E7023.

Fix: exclude the MPEG-PS structural stream_ids CSS never scrambles — system
header (0xBB), padding (0xBE), private_stream_2 (0xBF) — by the stream_id at
offset 0x11. This is the refinement the DVD design notes already called for
("matches CrackTitleKey"). It needs no format plumbing because byte 0x11 lives
in the CSS-clear header (0x00-0x7F, untouched by scrambling), so it is the true
stream_id even on ciphertext. A decrypted HD-DVD now scans to Unencrypted and
muxes cleanly.

The DVD CSS crack is preserved and proven: a genuinely CSS-scrambled DVD sector
is always video (0xE0-0xEF) or private_stream_1 (0xBD), never an excluded id, so
its scrambled packs still set saw_scrambled and still hard-fail an uncrackable
disc — the "ciphertext muxed as plaintext at rc=0" catastrophe cannot slip
through. Red-before-green both directions: dropping the 0x11 exclusion turns the
decrypted-HD-DVD case back into E7023; inverting it (only nav ids count) turns a
real uncrackable DVD into Unencrypted and strands a crackable one. Both mutations
are caught by tests.

Gate (cargo +1.97): fmt, clippy --all-targets -D warnings, 3539 tests green;
freemkv-engine and autorip both compile against this tree again; precommit.sh
libfreemkv clean.
2026-08-18 23:26:22 -07:00
Matthew Jackson c6e2614c4c Close the Ok-but-empty clip hole on HD-DVD; keep it open, and say why, on BD
The same hole, two disc families, two answers — and the asymmetry is now a
decision written into both files instead of an oversight in one.

`file_extents` can return `Ok` and still yield no usable extent: an empty
allocation-descriptor list, or one every entry of which the `sectors > 0 &&
lba > 0` filter discards. An ordinary zero-byte file reaches it; no crafted
disc is needed.

On HD-DVD that was the flagship failure shape. The clip entered neither
`clip_extents` nor `unusable`, and nothing was logged, so the composer's
`any(|n| unusable.contains(..))` guard missed it while the
`filter(|n| clip_extents.contains_key(..))` beside it quietly deleted the
part: a `FEATURE_2.EVO` of size 0 next to a healthy `FEATURE_1.EVO` composed
a FEATURE title out of part one alone, still advertising the whole runtime,
at rc=0, in silence. Half a movie presented as a whole one. Round 1 accounted
for every `Err` from the resolver and left this route open. It now marks the
clip unusable and logs it under its own new code, E6019
(`E_UDF_NO_USABLE_EXTENT`) — deliberately not the neighbouring E6017, which
would file a zero-length file as an authoring hole and send whoever triages
it at the wrong population.

On Blu-ray the identical hole stays open, as previously decided, and the
reasons are now recorded on both sides. BD has no `unusable` set, so closing
it there means inventing a post-loop "every clip_id must appear in `spans`"
invariant that DROPS the title, and it is not settled that an empty-but-Ok
resolve is always a defect; dropping healthy titles is worse than the gap.
The consequence is milder too: on BD the clip is one PlayItem of an otherwise
whole title, on HD-DVD the feature is COMPOSED from parts. Same hole,
different price.

Also in this change:

* bluray: a non-absence SSIF failure that the `.m2ts` fallback papers over is
  logged. `unresolved` had exactly one reader, `if let (None, Some(code))`, so
  when `/BDMV/STREAM/SSIF/<clip>.ssif` failed with DiscRead /
  UdfAdChainTooLong / UdfEmbeddedData and the base view then resolved, the
  code was recorded and thrown away: the title shipped base-view 2D off a 3D
  disc at rc=0 with no log at all. The site's own doctrine is "ABSENCE is the
  only benign failure". Logged, not refused — the base view is a real rip.

* drive: `wait_ready` polled TEST UNIT READY through a bare `execute` and its
  60 x 500 ms loop never read `self.halt`, so a Stop during spin-up did
  nothing for ~30 s while every other drive path returns Halted at the next
  command boundary. `spin_cycle` issued both START STOP UNIT commands outside
  `checked_exec` and slept `SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`
  blind — ~15 s deaf to Stop, from the recovery path, exactly when the
  operator is most likely to press it. Both now use `checked_exec` and
  `sleep_until_halted`, which already lived in this file with four tests and
  was `#[cfg(test)]`, called from nowhere. It is production code again.

* drive: a READ(10) that returns GOOD status with a residual underrun was
  correctly refused and logged NOWHERE, while the sibling `Err` arm warns with
  lba/count/status. A residual-underrunning drive was indistinguishable from a
  scratched disc — two populations with opposite remedies. It now warns with
  transferred vs expected, which is the whole signal.

* error: `all_error_code_constants_are_unique` was a hand-maintained `vec![]`
  naming 109 of the 127 declared codes while its doc claimed to pin them all,
  and an earlier audit trusted that claim while assigning new ones. The list
  is now derived from the declarations by parsing `include_str!("error.rs")`,
  so a new constant is covered the moment it is written. A parser self-test
  cross-checks the count and three known name/value pairs, so it cannot pass
  vacuously.

* testlog: a test-only `tracing` capture (~120 lines, no new dependency) so
  the logging contract is enforced rather than commented. Three sites carry
  long comments insisting they log the error's OWN code; putting a literal
  back broke nothing. They are pinned now, along with the two new log lines.
  Captures are serialised process-wide: `tracing`'s interest cache is global
  while `with_default` is thread-local, and the rebuild on the exiting
  capture can land after the entering one's, leaving the cache at "never"
  while a capture is live. That produced a real empty-event flake.

* disc: `scan_with`'s halt wiring for the BD and DVD enumerators had no test —
  every BD/DVD cancellation test calls the scanners directly, so passing
  `None` on either branch left the suite green while Stop did nothing.

* mux::network: `accept_from_rejects_stream_without_fmkv_header` half-closes
  instead of `Shutdown::Both`, which raced an RST against the server's read
  and returned ConnectionReset instead of InvalidInput under load. The port
  was already ephemeral; that was never the cause.

Gate: fmt, clippy --all-targets -D warnings, and 3439 tests green on 1.97;
precommit.sh libfreemkv clean.
2026-08-18 21:51:55 -07:00
Matthew Jackson b02c402dae CSS: key the DVD crack on the disc, not on the container
`resolve_dvd_title_key` decided whether to run the DVD CSS crack by asking
what CONTAINER it was looking at — `ContentFormat::MpegPs`. MPEG program
stream is what DVD and HD-DVD have in common, so every HD-DVD title was
run through a crack scan for a copy-protection scheme HD-DVD does not
use and cannot carry: AACS is its family, and CSS appears nowhere on the
disc.

Both outcomes of that scan were wrong. The cheap one wasted up to 50,000
sector reads per title. The expensive one returned `CssKeyMissing` —
E7023 — refusing a perfectly good HD-DVD with a CSS error, which is what
a real CI run produced on the HD-DVD fixture. Reading E7023 there sends
whoever triages it looking for a missing DVD key on a disc that never had
one, which is how a routing bug spends a day disguised as a key problem.

The right axis was already in the codebase and already used: `mux/resolve`
asks `disc.format == DiscFormat::Dvd`. This threads the disc format down
to the decision and adds `DiscFormat::may_have_css` to name the question.

The asymmetry decides the default, so it is worth stating. Running CSS on
an HD-DVD costs a wasted scan or a false refusal — visible, recoverable,
annoying. NOT running it on a real DVD muxes scrambled sectors as
plaintext and exits 0, which is the failure-that-looks-like-success class
this project has shipped once already. So `may_have_css` is false ONLY
for the families proven CSS-free, and `DiscFormat::Unknown` — a bare
reader with no scan behind it — still cracks. An `== Dvd` allow-list
would have read as tighter while silently stranding every caller that
cannot name its disc.

Both directions are pinned: an HD-DVD title must never enter the crack,
and an unknown-format title must still enter it. Removing the disc-format
clause fails the first and leaves the second and every DVD test green.
2026-08-18 19:11:20 -07:00
Matthew Jackson 128e3bb120 Propagate cancellation out of every title enumerator; account the .clpi read
A cancelled scan was indistinguishable from a disc that simply holds
fewer titles, in all three tree enumerators. Once the operator presses
Stop, `Drive::checked_exec` fails EVERY subsequent SCSI command with
`Error::Halted` and `Drive::read` deliberately preserves the variant —
so a cancel is not one failed read, it is every remaining read failing.

  * `scan_bluray_titles` returned a bare `Vec<DiscTitle>` and
    `parse_playlist` an `Option`, so neither had a channel to report a
    halt. During a cancel each remaining `.mpls` read merely failed and
    was skipped by an `if let Ok(..)`, and the scan returned a TRUNCATED
    title list at rc=0. Both now return `Result`; `Ok(None)` from
    `parse_playlist` keeps its two benign meanings (unparseable MPLS,
    playlist under 30 s) and `Err` means the scan is over.
  * `scan_dvd_titles` was worse: `Err(_) => return Vec::new()` turned a
    cancel into ZERO titles at rc=0 — a DVD reported as carrying no
    video at all. `ifo::parse_vmg` compounded it, treating a halted
    title set as one of the placeholder TT_SRPT entries it is designed
    to tolerate and continuing to the next; both now propagate.
  * HD-DVD was already correct and is the model this follows, including
    its reasoning: a cancelled enumeration that returned `Ok` would be
    cached, displayed and ripped from as if it were the disc.

The cancel is propagated at ALL FOUR Blu-ray read sites, not just the
obvious one — the playlist read, the `.clpi` read, and both
`file_extents` calls in the clip resolver — and the halt is polled
between playlists and again after the loop, so the final iteration
cannot slip a truncated list through. The resolver's `note` closure
previously EXEMPTED `Halted` from its disc-defect classification, which
was right (a cancel is not an authoring hole) but left it swallowed:
with no exemption and no propagation a cancel landing on `file_extents`
returned a title claiming its full runtime with that clip's bytes
silently missing. Measured with the fix reverted: `Ok(Some((768000,
[])))`.

Second defect, same function: the `.clpi` read and `clpi::parse` sat
inside `if let Ok(..) && let Ok(..)`, with extent resolution nested
inside. A scratched CLIPINF sector, a missing `.clpi` or a malformed one
skipped the whole block — no extents, nothing added to `total_size` —
but `duration_ticks` had ALREADY been summed from the play items, so the
title still advertised the movie's full runtime. The `if let` discarded
the error, so nothing was logged either: a failure that looks like
success, which is this crate's flagship defect class. It is now a
`match` that drops the title and warns with the error's OWN code, the
same classification the extent resolver below it already uses. Logging a
fixed code would account a scratched disc as an authoring hole.

Deliberately NOT added: a post-loop "every unique clip_id must appear in
`spans`" invariant. `file_extents` can also return `Ok(vec![])`, or a
vector whose every entry the `sectors > 0 && lba > 0` filter discards,
and it is not settled that an empty-but-Ok resolve is always a defect
rather than a legitimate healthy-disc state (zero-length placeholder
ADs). Dropping healthy titles is a worse failure than that residual gap,
so the route stays open, with a comment at the site recording the
decision so the next audit finds it instead of re-deriving it.

Tests: `parse_playlist_missing_clpi_yields_no_extent_no_size` asserted a
title WAS returned with `size_bytes == 0` and empty extents, and its
docstring quoted the buggy control flow as the specification — it
blessed the defect. Inverted and renamed. `parse_playlist_keeps_exactly_
30_seconds` encodes correct behaviour but used a fixture wiring no
STREAM/CLIPINF, so the `.clpi` fix would have dropped its title for a
reason unrelated to the boundary it pins; repointed at the fully wired
fixture rather than weakening the assertion. Four new halt-propagation
tests (BD playlist read, BD `.clpi` read, BD extent resolve, DVD title
set + VMG), each proven red before green.
2026-08-18 14:27:10 -07:00
Matthew Jackson dc100f1c80 Audit round 7: account for every clip that cannot be resolved
Ten lenses over v1.6.4..HEAD, every claim read against the code before
it was believed. Seven confirmed; six are here, one is recorded for the
next round. All of these are the same family — a failure wearing the
shape of success — which is the family that once shipped 9 MB of
ciphertext inside a main-movie m2ts at rc=0.

A clip whose extents cannot be resolved is now accounted for, in both
disc readers. Only `UdfUnrecordedExtent` used to count: every other way
`file_extents` can fail — a scratched sector under the clip's ICB
(DiscRead), an allocation-descriptor chain that never terminated, a file
whose data is embedded rather than extent-mapped — fell through to the
ordinary "file absent" path. On Blu-ray that yielded a title advertising
its full runtime with a clip's bytes silently missing, because the size
and the play-item timing had already counted it. On HD-DVD it was worse:
the clip was never added to `unusable`, so a split feature still composed
from FEATURE_1 alone and offered half a movie as the whole thing. Neither
emitted a single log line. Absence is still benign — a 2D disc has no
.ssif and the extension fallback exists for exactly that.

`Halted` is excluded deliberately, and that exclusion is the whole reason
the first version of this fix was wrong. Cancellation makes EVERY drive
command return `Halted`; classifying it as a disc defect would have
dropped each remaining playlist in turn and handed back a truncated title
list at success — the same defect, wearing a cancel. `parse_playlist`
returns Option and has no channel to propagate a halt, so the existing
behaviour is preserved rather than made worse. Propagating it properly is
next round's work.

Both log sites now emit the error's OWN code instead of a hardcoded 6017.
Accounting a scratched disc (E6000) as an authoring hole would send
anyone triaging it looking for the wrong thing entirely.

AD type 3 is embedded data, not a descriptor list (ECMA-167 4/14.6.8).
`read_icb_extents` lumped it in with the reserved values and decoded the
file's own CONTENT as (length, LBA) pairs, manufacturing extents out of
arbitrary bytes and pointing the reader at unrelated sectors. This same
release already taught `read_directory` to honour type 3; this is the
file half of that decision. It is an error rather than an empty list,
because an empty list reaches the caller as a clip that contributed
nothing while its declared duration still counts it — the silent loss
pointed the other way. A legally zero-length embedded file still returns
an empty list. New code E6018: reusing DiscRead would have mislabelled a
deterministic structural property as transient I/O and fed the retry and
NonTrimmed machinery a byte that will never change.

`file_extents_addressing`, `extents_abs_at` and `AbsExtent` drop to
`pub(crate)`. The first hands back unrecorded extents UNFLAGGED, in a
shape identical to the safe call's return; its doc says callers must use
`file_extents` instead, but a doc comment is not a guard. No dependent
crate references any of the three.

Three tests close gaps the audit found, each proven red before green:
a held AC-3 access unit must not resume as a normal frame after its track
poisons; the PS resume cursor must survive a drain that rebases it (three
separate mutants caught); and AD type 3 must be refused rather than
decoded. The first attempt at the HD-DVD test passed with the fix
reverted, which made it worthless — it needed a VTI fixture before the
composition path ran at all.

Also: four error codes were missing from the uniqueness test that claims
to cover every published code, so a new variant reusing 6014, 6016 or
6017 would have passed it.
2026-08-18 13:46:03 -07:00
Matthew Jackson 286b5d4527 ci: stop Dependabot proposing Rust versions that do not exist
`dtolnay/rust-toolchain` is not a dependency in the sense Dependabot
means: it is versioned by the Rust release it installs, and the tag we
pin is the toolchain CI is pinned to on purpose — precommit.sh runs the
same one locally, so a lint that passes on a developer's newer default
cannot pass CI by accident.

Read as semver, those tags produced a 1.97.0 -> 1.100.0 proposal, a Rust
version that does not exist. Every such PR 404s on toolchain download
across all eight repos and regenerates weekly: eight permanently-red PRs
that promote.yml then has to special-case when it decides whether dev is
green.

freemkv already carries this ignore; this is the same block in the other
seven, so the fleet stays uniform. Bumping the toolchain stays a
deliberate, all-eight-repos change made by hand together with
precommit.sh.
2026-08-18 12:14:58 -07:00
Matthew Jackson d9f3f82c65 Audit round 4-6: disc parsing, extents, codecs and drive faults
Squashed from 12 commits. Every fix was proven red-before-green and killed by a
mutation; the reasoning for each is in the private audit record.

UDF and extents
  Honour ICB types rather than assuming a Short AD, so an AD-type-3 directory
  is no longer decoded from FID bytes into a silently empty listing. Carry the
  ECMA-167 recorded flag through to the resolvers: an allocated-but-never-
  written extent used to reach the read plan as ordinary content and splice
  undefined sectors into the rip. file_extents now refuses such a file, and
  only when the hole actually occupies byte space — a zero-length one displaces
  nothing, and refusing on it dropped whole titles off discs that ripped
  correctly. Type-2 sparse extents are kept alongside type-1; they were falling
  into a catch-all that exited the descriptor loop and returned a truncated
  list as complete. merge_ranges no longer claims a sector neither input
  covered. A short skip or an over-long AD chain errors instead of truncating.

HD-DVD and Blu-ray scanning
  Bound the XPL nesting depth, title count, clips and chapters per title, and
  memoize the clip-name fallback probe — four separate amplification axes, each
  of which alone left the worst case unbounded. The clip and title caps are 512,
  ~10x any retail disc, and a test pins the product of cap and probe budget.
  The scan is cancellable: it returned Ok with titles carrying no streams when
  halted, presenting a cancelled scan as a successful one. A clip dropped for an
  unrecorded extent now says so.

Codecs and muxing
  Resume a held E-AC-3 access unit rather than rescanning from its first frame,
  and drop it on a discontinuity — a stale hold indexed past the end of the new
  buffer. Map every ISO 639-1 code instead of collapsing fifteen languages to
  und. Correct the DVD palette order. Detect a skip past EOF.

Drive and I/O
  Classify dead-bus faults so the wedged-drive path can see them; a catch-all
  arm had been flattening the variants before the classifier ran. A prefetch
  producer that dies now reports SourceTerminated instead of Ok(0), which the
  reader legitimately read as a short read and zero-filled — a whole title
  could be fabricated and the pass reported complete.

Also: charge Ok(0) reads to the CSS crack budget, drop the unreachable soft
re-crack, and send disc-derived strings to logs through the debug formatter so
a crafted label cannot paint an operator's terminal.
2026-08-16 13:22:24 -07:00
Matthew Jackson e30209091e restore freemkv-unlock path dep for local dev (post-v1.6.4) 2026-08-15 09:13:30 -07:00
Matthew Jackson ac46e2382f v1.6.4: bump version (freemkv-unlock git-pinned for the tag) 2026-08-15 09:13:26 -07:00
Matthew Jackson 862c1460c3 Trim single-clip titles to their playlist marks (1.6.4) 2026-08-15 08:14:31 -07:00
Matthew Jackson fea64d07e2 Stop an uncrackable VTS borrowing another VTS's title key
`resolve_vts_key` called the `Option`-returning `css::crack_key`, which
collapses "no scrambled sector was seen" with "scrambled sectors were
seen and no key came out", and then fell back to the disc-wide key. A
multi-VTS CSS DVD whose second title set resists the Stevenson scan was
descrambled under the FIRST set's key: corrupt PES behind an intact
header, written out with `complete = true` at exit 0.

`CrackOutcome` exists precisely to keep those two apart, and its doc says
callers must surface the second as a hard error. Every sibling path in
the crate already does — `Disc::decrypt_keys_for_title` and the mux path
both map `ScrambledUncracked` to `CssKeyMissing`. This was the one path
that did not. The ordering comment 15 lines above describes this exact
outcome as the bug it exists to prevent; ordering makes the crack far
more likely to succeed, but it cannot make a failed crack safe.

Also, three things nothing could catch:

- The demux output filename took the stream's language raw while the
  base beside it was sanitised. A language code is three raw STN bytes
  through `from_utf8_lossy`, and `00 00 00` is the ordinary "undefined"
  encoding on real discs — a NUL in a path fails `File::create` with
  InvalidInput, taking the whole export down before one track file
  opened. `sanitize` now maps control characters too; it did not.

- `css::crack_key_scan`'s short-read handling was dead code under test:
  every source in the module returned the full request, so reverting
  `advance` to the requested count, or dropping the `.max(1)`, left the
  suite green. The `.max(1)` is load-bearing — without it a source
  returning `Ok(0)` never moves the cursor and never increments the
  budget, so the scan spins forever. That mutation now HANGS the test
  rather than failing it, which is the honest demonstration.

- `MAX_SUBDIRS`'s const-assert carried `#[cfg(not(test))]` inside a
  `#[cfg(test)] mod tests`, so it compiled in no configuration and could
  never fire — the dead gate the test above it was written to replace.
  Moved to module scope, and verified it now rejects a wrong constant at
  compile time.
2026-08-11 17:47:56 -07:00
Matthew Jackson 6e8da15c34 Bound the third hostile CSV, and stop two docs overclaiming
`forced_sub` was the last unbounded attacker-controlled list in
paramount.rs. `MAX_COM_INDICES` capped the two `*_com1_idx` attributes;
this one had nothing capping it at all, and unlike them it has no value
to filter — a cell is a classification of the position it sits at, so
its bound has to be positional. Extracted as `forced_subs` for the same
reason `com_indices` was extracted: through `labels_from_feature` the
bound is unobservable, because the subtitle loop cannot reach those
cells either, so the assertion could not fail.

The `MAX_COM_INDICES` doc claimed the entry-allocation argument for the
whole constant. It is the VALUE filter that caps the set (values below
the bound, so at most that many distinct entries, however long the
attribute); the `take` caps the WORK. Both are real and they are not
the same bound; the doc now says which is which.

`jar_inventory_dedup_is_not_quadratic` called itself proof by deadline.
It is a hang guard — a return to the linear scan runs for minutes and
would wedge CI rather than fail it — and no assertion in it can tell a
BTreeSet from any other sub-quadratic dedup. Renamed and documented for
what it does. Its margin was measured before keeping it: 0.14s debug /
0.07s release against 10s, ~70x, unlike the 6x that made paramount.rs's
wall-clock test flake.
2026-08-11 16:59:33 -07:00
Matthew Jackson 8898351cb1 Round 2: stop two of round 1's tests claiming more than they prove
Both are mine, and both pass with the bound they "cover" deleted.

`a_hostile_commentary_index_list_is_bounded_not_merely_fast` asserted
only label purposes. The out-of-range filler it feeds is unobservable at
the label level and a HashSet collapses the repeats, so removing
MAX_COM_INDICES entirely leaves it green. What it DOES catch is a bound
set too low — verified at 2, where the real indices stop resolving. Named
and documented for that, and it no longer implies it guards enforcement.

`an_index_that_cannot_address_any_cell_is_not_retained` asserted through
the labels, where retention is by definition unobservable: the loop never
queries a cell that high. It now reads the set through `com_indices`,
where the claim is checkable.

Enforcement was and remains proven by
`distinct_unaddressable_indices_are_refused_not_stored`, which was red at
50,000 retained entries and green at zero. Two tests, two properties;
neither pretends to the other's job now.
2026-08-11 11:54:28 -07:00
Matthew Jackson 7a03be5496 Bound the commentary-index parse, and stop proving it with a clock
The gate caught `commentary_index_lookup_is_not_quadratic` failing, then
passing on a re-run. Measured both ways: 1.62s alone, OVER 10s against
its own 10s deadline while the suite's other 3,347 tests ran
concurrently. A 6x margin against a shared CPU is not a margin, and a
test that gets re-run until it passes is not a test.

It was also measuring the wrong thing. Replacing the linear scan with a
HashSet bounded the LOOKUP; the set was still built from every entry the
disc declared. `playlists.xml` is attacker-controlled and has no length
of its own, so a hostile disc could still force an unbounded allocation
before any lookup happened — the parse, not the query, was the exposure.
The code's own comment said "unbounded parsed input" and only fixed half
of it.

`MAX_COM_INDICES` bounds both halves, at the one value that cannot
change behaviour: an index at or beyond `u16::MAX` can never match a
cell, because the labelling loops break at `u16::try_from(i + 1)`. Real
authoring is nowhere near it — the BD STN table admits 32 streams.

The parse moved into a `com_indices` helper so the bound is OBSERVABLE.
Through `labels_from_feature` it is not: a HashSet collapses repeated
values, and an out-of-range index changes no label, so the obvious test
passes with or without the cap — an assertion that cannot fail, which is
what the first draft of this fix shipped. The test now hands in 50,000
DISTINCT unaddressable indices and asserts they are refused. Proven red
with the bound removed (50,000 kept), green with it.

Applies to both `sub_com1_idx` and `aud_com1_idx`.

NOT done here: `labels/mod.rs:1421`'s `jar_inventory_dedup_is_not_quadratic`
is the same wall-clock shape and has the same flakiness. Named so it is
not lost.
2026-08-11 11:30:53 -07:00
Matthew Jackson de94e12864 Cover the dir:// input door, and correct the CSS scramble-gate docs
Three round-1 audit findings on v1.6.0..HEAD.

Tests: `mux::resolve::input("dir://…")` — the door the CLI rips a folder
through — was never driven with a real folder. The only dir:// input test
uses a missing path, which fails in DirImage::open long before
`image_input`, so dropping the `is_folder` argument (and with it
`session::apply_folder_encryption_verdict`) passed the whole suite. Two
tests now close it, stated as a DIFFERENTIAL between the two doors,
because that is the invariant the shared function exists to hold: a
clear folder that kept its AACS/ directory must scan the same and select
the same extents through `scan_dir` and through `dir://`, and a
scrambled one must be refused through both with E9063. Proven red
against a mutated guard (both fail with E7022, the tree-shape verdict
the probe overrides) and green with it restored.

Docs: css/mod.rs documented `is_scrambled`, a function this line renamed
to `has_scramble_flag_bits`. Eleven stale sites, two of them broken
rustdoc links. Two were not stale names but false statements — they said
the descramble loop keeps the looser raw-flag test, when
`descramble_sector` and `descramble_region` both gate on
`is_scrambled_pack`; that parenthetical is rewritten to say what the
code does and why (the measured VIDEO_TS.IFO case: 38 titles became 10,
silently, at exit 0). The one mention that must stay is the sentence
explaining why the name was rejected.

Constants: io::image_writer and dirimage::encode each re-declared the
2048-byte sector that consts::SECTOR_BYTES already exports; both now
alias it. Note `BATCH_SECTORS = 2048` in image_writer is a different
quantity (sectors per batch, not bytes) and is deliberately left alone.
Bare 2048 literals elsewhere in the crate are out of scope here.
2026-08-11 10:23:37 -07:00
Matthew Jackson 21ec1d3f80 restore freemkv-unlock path dep for local dev (post-v1.6.3) 2026-08-10 09:04:40 -07:00
Matthew Jackson 6658a80819 v1.6.3: bump version (freemkv-unlock git-pinned for the tag) 2026-08-10 09:04:35 -07:00
Matthew Jackson 3db11a192c Carry the same licence and community files as the other repos
These are eight public repos that ship one product on one version, and
they had drifted: freemkv-engine carried no LICENSE at all (GitHub
reported its licence as none), two crates had no code of conduct, four
had no contributing guide, and none had a security policy — so there was
no private route to report a vulnerability in a disc-decryption tool.

SECURITY.md names GitHub Security Advisories on each repo rather than an
address, so there is nothing to keep in sync and no inbox to go stale.
2026-08-09 22:08:22 -07:00
Matthew Jackson a425a71cd5 Keep the cross-platform jobs for the release candidate
dev is where work lands and is meant to be pushed to often, so what runs
there should be the cheap answer to "did I break it": lint, tests and the
Linux build. The macOS and Windows jobs now run on qa and main instead of
on every push to dev.

Nothing is deleted and no platform goes unchecked before a release. qa.yml
already covers macOS and Windows independently, and the jobs that live only
here -- the Intel macOS build, the Windows release build -- still run, on
the branches where a cross-platform break is worth blocking on.

They are SKIPPED on dev via `if`, not left unscheduled. A queued job would
be worse than a slow one: release.sh's CI gate refuses while any run for
the commit is still in progress, so a job that never gets a runner blocks
releases silently, with no error anywhere. That is the failure the
real-media note in qa.yml describes, and it is why this is an `if` on the
job rather than a narrower set of trigger branches.
2026-08-09 20:24:07 -07:00
Matthew Jackson f79745d815 Release 1.6.3
Version bump and changelog for the 1.6.3 sync. All eight crates ship the
same version, so the crates with no functional change this cycle say so
rather than carrying an empty section.
2026-08-09 20:01:33 -07:00
Matthew Jackson 5d8f77e049 Drop six unused crates, and align the rest with the workspace
Two problems, both invisible until the whole graph is looked at together.

DEAD: num-bigint, sha2, num-traits, num-integer, cmac and cbc are
declared here and referenced nowhere -- not in src, tests or benches.
They were being compiled, audited and offered version bumps forever for
no reason. Removing beats bumping.

cbc nearly survived the sweep: a substring search for "cbc" matches 44
occurrences of ycbcr_to_rgb in the DVD subtitle decoder, so it looked
used. Only a word-boundary search exposed it.

SKEW: this crate was the outlier on every shared dependency -- aes 0.8,
rand 0.8, base64 0.22.1 and zip 2 against 0.9 / 0.10 / 0.23 / 8
elsewhere. Cargo cannot unify across a major version, so it compiled
BOTH: 32 duplicated crates in the freemkv binary's graph, including two
complete AES implementations (aes 0.8 + 0.9, cipher 0.4 + 0.5), two
digest stacks and two getrandom. Two crypto stacks in one product is
worth removing on its own.

The aes bump is an API rename -- BlockCipher-prefixed traits, Array for
GenericArray -- and the obvious translation uses Array::from_slice,
which the new version deprecates and clippy's -D warnings would reject.
These use the From<[T; N]> conversion the crate points at instead.

3441 tests pass in debug and release. The AACS crypto here is covered by
known-answer tests, so a byte-order or sizing mistake in that rename
could not have passed.
2026-08-09 18:37:24 -07:00
dependabot[bot] 79a16a9279 Bump softprops/action-gh-release from 2 to 3 (#4)
Bumps [softprops/action-gh-release](https://github.com/softprops/action-gh-release) from 2 to 3.
- [Release notes](https://github.com/softprops/action-gh-release/releases)
- [Changelog](https://github.com/softprops/action-gh-release/blob/master/CHANGELOG.md)
- [Commits](https://github.com/softprops/action-gh-release/compare/v2...v3)

---
updated-dependencies:
- dependency-name: softprops/action-gh-release
  dependency-version: '3'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-09 17:14:30 -07:00
dependabot[bot] 219a77b581 Bump actions/checkout from 5 to 7 (#5)
Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 7.
- [Release notes](https://github.com/actions/checkout/releases)
- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)
- [Commits](https://github.com/actions/checkout/compare/v5...v7)

---
updated-dependencies:
- dependency-name: actions/checkout
  dependency-version: '7'
  dependency-type: direct:production
  update-type: version-update:semver-major
...

Signed-off-by: dependabot[bot] <support@github.com>
2026-08-09 16:58:27 -07:00
Matthew Jackson be378cbc34 Have Dependabot watch the dependencies, and land its PRs on dev
Version updates were never configured here -- only security alerts, which
report but never open a pull request. So the absence of Dependabot PRs was
not "nothing to update", it was "nothing was checking".

Updates target `dev`, never `main`. main is a release pointer that
release.sh moves to each tag, so a bot commit on it would put work there
that no tag contains -- the exact state that aborted the 1.6.2 cascade at
its last step, and pointing a scheduled bot at main would recreate it
weekly.

Minor and patch bumps are grouped into one PR rather than one per crate:
eight repos times a handful of dependencies is a volume nobody reads, and
an unread PR queue is indistinguishable from no updates at all.

The freemkv crates are excluded. They depend on each other by git tag,
re-pinned by release.sh inside the release commit, and Dependabot cannot
see that cascade -- a PR bumping one could pin a version whose tag does
not exist yet.
2026-08-09 16:48:31 -07:00
Matthew Jackson 63200a1d1f Retire the README version bot
It fired on `release: published` and committed to main, the branch the
release points at its tag, so it could only ever leave main ahead of the
tag. release.sh now makes the same edit inside the release commit.

In this repo it was a no-op regardless: the pattern it rewrote does not
appear in this README, so it has been running and changing nothing.
2026-08-09 07:20:44 -07:00
Matthew Jackson 526dd3552b restore freemkv-unlock path dep for local dev (post-v1.6.2) 2026-08-08 22:18:33 -07:00
Matthew Jackson 8d17904459 v1.6.2: bump version (freemkv-unlock git-pinned for the tag) 2026-08-08 22:18:28 -07:00
Matthew Jackson 1bdb8add2f Stop trusting a byte offset when the feed is not what was measured
A clip's span is measured over the title's full extents when the disc is
scanned, and a frame is placed by the offset it was read from. Those only
agree while the mux reads every byte the scan counted.

On a disc carrying a forensic segment it does not: the read plan omits the
units belonging to another device group, so fewer bytes are fed than the
spans describe, and the shortfall grows through the title. Every frame past
the first segment then looks earlier than it is — placed in a clip it did
not come from, or dropped at a join for failing marks it was never inside.
The spans still tile one another perfectly, so the check that asks whether
they can be trusted cannot see any of it.

The plan is now compared against the extents it was built from. When they
differ the spans are dropped and placement falls back to timestamps, which
is what that path is for and what the surrounding comment already promised
would happen when an offset stops meaning anything. Ordinary discs are
untouched: with no forensic segment the plan IS the extents.
2026-08-08 19:42:53 -07:00
Matthew Jackson c9562c07a3 Make four guards testable, and one guard see both spellings
The provenance guard knew one way to lose a frame's source offset —
writing it as absent. Omitting the field entirely does the same thing,
because the frame type fills it in by default, and the guard read straight
past that. A parser rewritten into the second spelling would have kept its
green light while its track silently went back to being placed by guesswork.
It now reports which spelling it found and where.

The title-count clamp was asserted over a fixture too small to hold more
titles than the cap allows, so the walk stopped when the buffer ran out and
the clamp was never what bounded it — the assertion held with the clamp
deleted. The fixture now carries more entries than the cap.

The subdirectory limit was checked by restating the constant's own
definition; the guard itself had never run, and deleting it changed
nothing. The limit is lowered under test so a real folder can exceed it,
and the test now walks one and requires the refusal.

One test also carried two unrelated grounding notes while the test they
described had none, so reading the note above a test told you about a
different one.
2026-08-08 19:06:37 -07:00
Matthew Jackson 7f9dcf1870 Keep both halves of a clip file that two play items share
A playlist may point at the same .m2ts from several play items — a
seamless split, a looped segment, multiple angles. The file has one set of
bytes, so it has one span, and the byte a frame was read from therefore
identifies the FILE, not which play item's range it falls in. Placement
took the first of them and judged every frame against its marks, so
everything past that range's end was treated as material the playlist
excludes and dropped: half the clip missing from the rip, with the
timeline still charged for its full duration.

The offset narrows a frame to the run; only its timestamp can finish the
job, and each play item carries its own marks. A frame is now matched
against the marks of the entry it actually falls in, falling back to the
first when it falls in none, which leaves genuinely-excluded material
dropped as before.

The test for this construction asserted only that such a playlist is
trusted, and never placed a frame from the second range — so the loss it
described in prose was invisible to it. It now places one from every range
and requires them all to survive.

The deferred-mux replay of buffered frames wrote them without their source
offset, quietly sending the head of every such title down the timestamp
heuristic the rest of this work exists to retire. It passes the offset
through now, and the frame writer that omits it is compiled out of the
library entirely: no production path can discard provenance any more.

Also: the AC-3 buffer-reuse test could not see the regression it named.
Feeding equal-sized packets, a fresh allocation per call yields the same
capacity as a reused one. It now feeds a large packet then a small one,
where only a reused buffer keeps the larger capacity.
2026-08-08 17:18:35 -07:00
Matthew Jackson 3c5c681399 Audit round: an ADTS frame that cannot hold its own CRC, and three drifts
An ADTS header that declares a CRC follows must be at least nine bytes —
seven of header plus the two the CRC occupies — because the declared frame
length counts them. The structural gate compared against a flat seven and
never read the bit that says whether a CRC is there at all, so a frame
whose own header describes something impossible was accepted and handed to
the muxer as decodable.

The pipeline's spawn doc named Sweep, and the thread Sweep would have
created, as callers to look for. Neither has been in this crate since the
recovery passes moved out. The same paragraph already records fixing this
once, for a different departed caller — it simply drifted again a sentence
later, so it now says to name callers that live here or name none.

Whether a disc is structurally AACS-encrypted was spelled out by hand in
both the fast identify and the full scan. They agreed today; nothing made
them agree tomorrow, and disagreeing would mean the same disc reported
encrypted by one path and clear by the other. There is one definition now,
and the comment that pointed at it by line number points at its name.

The AC-3 parser built a fresh buffer on every packet — of the order of a
hundred thousand times per title — to work around a borrow it cannot
avoid. The copy stays; the allocation does not. The buffer is now lent out
and handed back, and a test pins that, because reverting it would be
invisible in behaviour.

Left alone deliberately: send/send_with_halt and finish/finish_with_halt
look like one action under two names, and are not. After the consumer
fails, one must still accept items and the other must refuse them; that
difference is what stops a producer reading an entire disc for a write
that died on its first frame. Collapsing them was tried here and the
existing test caught it. Both now say so where the choice is made.
2026-08-08 16:55:55 -07:00
Matthew Jackson 43d9fb583a Add an error for an image shorter than its recovery data
A recovery resumed against an image that has been truncated since the
previous pass — a full disk, an interrupted transfer, a remount — cannot
repair it: the pass only revisits the ranges the recovery data calls bad,
so everything past the cut stays a hole while the counts still describe a
whole disc. There was no way to say that. Reusing the recovery-data error
would have been wrong, because the recovery data is intact; the image is
not.

Registered in the uniqueness list, the code table and the range check, so
a future variant cannot silently reuse 6015 or map to the wrong code.
2026-08-08 13:35:44 -07:00
Matthew Jackson 60ed45c047 changelog: 1.6.2 2026-08-08 08:53:13 -07:00
Matthew Jackson 07cc9fa5f9 ci: run the dependents' test suites, not just a type-check
The `consumers` job checked out all five dependents against this commit and
ran `cargo check --all-targets`. That proves they still COMPILE, which
catches a changed signature and nothing else. The failures worth catching
here keep every signature intact and change behaviour: the library still
builds, the dependent still builds, and the dependent's tests are what go
red. Those never ran.

Run their suites instead. Same checkouts, same patched paths — only the
verb changes.
2026-08-08 08:46:17 -07:00
Matthew Jackson 1e6bb8babd Give a frame that outruns its epoch's video a provisional offset
The demuxer interleaves, so at a boundary the streams do not reset on the
same frame: audio for the next segment can reach the muxer before the video
frame that opens the epoch it belongs to. Only the primary video track may
open one, so those frames rode the just-ended offset and landed a whole
segment in the past. Downstream the strictly-monotonic block nudge then
crushed the entire run onto one instant a tick apart, which is audible.

Such a frame is recognised on its OWN raw PTS, which within an epoch only
advances, so a large backward step is unambiguous — a different signal from
the shared frontier, which is what the old false-positive ratchet keyed on.
It then rides a provisional offset private to its track, computed the same
way the video path computes a real one, and drops it the moment the video
retires an epoch, so the run rejoins with no seam.

The offset is deliberately private: it never writes offset_ns, never
advances the frontier and never retires an epoch, so it cannot move the
video timeline. Letting a passive track open a REAL epoch was tried first
and inflated a 476.776 s title to 656.216 s, because every track meets a
boundary at its own pace and the video path rebased again on top.

Measured on a real DVD title with 8 cell boundaries: audio frames stamped
inside one cadence 120 -> 0, outsized gaps 7 -> 0, subtitle 1 -> 0, and the
video span byte-identical at 476.484. On the HD-DVD title the remaining
boundary gap on its first audio track falls from 0.999 s to 0.275 s.
Acceptance: 76 pass, 0 fail, 0 skip.
2026-08-08 00:48:10 -07:00
Matthew Jackson 64f894c9f3 Place a straggler by its own epoch's end, not the current frontier
A non-video frame can reach the muxer after the video that opened the next
epoch has already been processed. It carries an old-epoch PTS, so adding
the new offset flings it forward by a whole clip. The remap that exists to
catch this compared the frame against the CURRENT frontier and only
accepted it within one backstep, which is the wrong yardstick: by the time
a straggler is seen the frontier has moved on into a new epoch, and how far
below it the frame lands says nothing about where it belongs.

Retiring an epoch now records the frontier it closed at alongside its
offset, and a straggler is judged against the end of the epoch its PTS came
from. Keeping the whole history rather than one previous offset also fixes
the case a single `prev_offset_ns` cannot express at all: a frame from two
or more epochs back.

Measured on a real HD-DVD title whose second audio track put its final
packet at 12834.587 s in a 6434.100 s file. That frame sits 23 s below the
current frontier — outside the old window, so it was refused — but 0.15 s
below the end of the clip it actually came from. It now lands at 6417.216,
one 32 ms cadence step after its neighbour, and the track spans 6416.160
with no outsized gaps. The other two streams are unchanged.

A title that never rebases retires no epoch, so the lookup finds nothing
and the mapping is bit-identical to before.
2026-08-08 00:02:57 -07:00
Matthew Jackson 3820348d2f test: cover the 1.6.1 provenance work where it was assumed, not asserted
Four parsers (dvdsub, flac, lpcm, mpegaudio) stamp a source offset on
every frame but had no test that read one back, so a regression to
`source: None` would have been caught only by the brace-balanced audit
in codec/mod.rs — a lint, not a behavioural check. Each now asserts the
emitted frame carries the offset of the packet that supplied its first
byte.

The Blu-ray feed spans had no direct test at all. Add one that walks a
multi-item playlist and requires the spans to tile the feed with no gap
or overlap; it catches a one-sector-per-clip drift, which is exactly the
error class that would misattribute frames near a seam.

`no_provenance_still_places_by_marks` asserted only that placement
returned something, which passes for a frame placed in the wrong clip.
Its probe timestamp lands in an overlap between two clips in the real
mark table, so pinning one clip would assert a coin-flip; instead
require the offset to be one that a clip actually containing that
timestamp would produce.
2026-08-07 22:16:02 -07:00
Matthew Jackson 102ae82933 Test the DVD title table's sector-offset arithmetic
The TT_SRPT pointer at 0xC4 is a SECTOR offset from the start of
VIDEO_TS.IFO. Reading it as a byte offset lands 1/2048th of the way in and
does not error — it silently returns a different title list, which is how
a disc enumerating 38 titles and an image enumerating 10 looked like a
scan difference rather than a failure.

The table already had tests for its entry layout, dedup and the 99-title
clamp. What it did not have was anything pinning the offset unit itself,
or the grouping of titles by title set, or a truncated table keeping the
rows it can read rather than failing a whole disc.
2026-08-07 22:01:37 -07:00
Matthew Jackson 9d05f54ddb restore freemkv-unlock path dep for local dev (post-v1.6.1) 2026-08-07 19:08:46 -07:00
Matthew Jackson 47abd1270b v1.6.1: bump version (freemkv-unlock git-pinned for the tag) 2026-08-07 19:08:42 -07:00
Matthew Jackson 96c23ffb87 changelog: mark 1.6.1 UNRELEASED so release.sh can date it
release.sh dates the heading when it cuts the tag, and refuses a bare
'## [1.6.1]' because it cannot tell an undated heading from one it has
already stamped. Shipping a tag whose public changelog says UNRELEASED is
permanent and cannot be corrected inside that tag, so the check is right
to stop rather than guess.
2026-08-07 17:57:29 -07:00
Matthew Jackson f2b0e00bea Guard the invariant that provenance is universal
Every emitted frame must carry the source byte offset of the packet it
came from. That held only for video for as long as it existed, and
nothing asserted it, so nothing caught it: ten parsers built frames with
`source: None` and a multi-clip title could not place audio or subtitles
by byte at all.

Finding them took a brace-balanced scan of the tree by hand — a regex
cannot do it, because a Frame literal contains nested braces and a
non-greedy match stops at the first `}`, which is how five sites survived
the first pass. This is that scan, as a test.

A second test fails if a codec module is added and not listed, since an
unchecked parser is exactly how the gap persists. Modules that emit no
Frame are named explicitly rather than skipped silently.

The guard tripped on itself twice while being written — first on its own
doc comment, then on its own string literals — so it strips line comments
and excludes its own module. A check that matches prose about the defect
rather than the defect is the same mistake in a different place.
2026-08-07 13:37:27 -07:00
Matthew Jackson bca886a9c3 changelog: tighten 1.6.1, and record two entries it was missing
The entries had grown into narrative paragraphs; each is now the outcome
first and the mechanism in a sentence or two. The website changelog page
mirrors libfreemkv's, so it inherits this directly.

Two things that shipped in this cycle were not in it at all:

dir:// as a source. It was pulled from 1.6.1 on 2026-08-05 because the
folder reader produced a wrong title list on a real DVD. That defect was
the CSS descramble bug below, fixed since — the acceptance suite now
shows a folder reporting the same 38 titles as its ISO, with matching
streams, languages and runtime, and the CLI has been wired for it all
along. It ships, so it is listed.

The CSS title-table corruption itself: a disc enumerating 38 titles
produced a decrypted image enumerating 10, silently, at exit 0. That is
a data-integrity fix a user needs to know about.
2026-08-07 13:15:36 -07:00
Matthew Jackson 7cda914f29 Stamp the last frames that were still leaving without provenance
A real rip caught these. The warning added for a track placed from
timestamps under a seam plan fired once, on one subtitle track of a
23-clip title, eighteen minutes in — a display set that arrives as a lone
non-PCS segment is emitted straight through rather than accumulated, and
that path still built its frame with source: None.

Auditing the rest the same way found four more: flac, lpcm, mpegaudio and
the passthrough parser. All are one-PES-one-frame, so the packet's own
facts are the unit's facts.

Every Frame construction in every codec parser now carries a source,
checked by walking balanced braces rather than by eye — the earlier
count was taken with a regex that cannot see a block containing nested
braces, which is how these survived the first pass.
2026-08-07 10:01:39 -07:00
Matthew Jackson 11538981de Place a non-advancing mark table by provenance
Marks that do not advance across a title are normal, not a defect: each
clip file carries its own STC, so one clip's IN has no ordering
relationship to the previous clip's. Refusing those tables dropped
exactly the branched titles this type exists for onto the inference path
that cannot read them, and that is the overrun.

Lifting it was tried once before every track carried a source offset, and
nine audio and subtitle tracks with nothing to place them by pinned
themselves to clip 0 and dropped most of the title. Every parser stamps
provenance now, so the clip comes from the byte offset and the marks keep
one job: whether a frame lies inside its own clip's [in, out].

Still gated on spans_trusted. Without usable spans there is no offset to
place by and inference is all that is left, so a table inference cannot
read is still refused rather than silently truncating a title.

A frame reaching the timestamp path under a plan now says so once per
track. It is not expected any more, and it is how a mostly-dropped track
reached a user without a single line in the log.
2026-08-07 07:49:53 -07:00
Matthew Jackson 8edb53942c Every parser now carries provenance, on the one shared buffer
dts, ac3 and truehd assembled access units across PES packets with three
private implementations of the same bookkeeping, and none of them carried
the source byte offset. They now hold a PesBuf, so a unit takes the
timestamp AND the source of the packet covering its first byte, from the
same mark, and no codec can answer that question its own way again.

dts is the reference case: its pts_marks already implemented the rule
correctly, and all 61 of its existing tests -- including the PTS
attribution ones -- pass unchanged on the shared type. That is the
evidence the type preserves the behaviour dts had right.

ac3 kept a single carry-over timestamp and one anchor offset, so it could
only attribute the first unit in a call; it now resolves each unit at its
own offset. truehd had no attribution at all beyond a running clock.

New tests cover the case that motivated this: a unit whose first bytes
arrive in one packet and whose remainder arrives in the next keeps the
FIRST packet's offset. At a clip seam those two packets belong to
different clips, and taking the later one places the audio in the wrong
one.

Clippy on the pinned toolchain caught an empty `if` block left where dts
used to clear stale marks -- restored as an explicit clear, with why it
is still needed once drain keeps the covering mark.
2026-08-07 07:45:32 -07:00
Matthew Jackson c718cd9e9e One rule for which packet a unit's facts come from
A parser that assembles an access unit across PES packets has to answer
one question for every unit it emits: which packet carried this unit's
FIRST byte? Its timestamp comes from that packet, and so does the source
byte offset that says which clip of a multi-clip title it belongs to.
The packets that complete the unit carry later values that must not
override it.

That question was being answered three ways. DTS kept a deque of
(offset, pts) markers and took the one covering offset 0. AC-3 kept a
single carry-over timestamp. TrueHD kept its own. PGS and DVD subtitles
each held a pending unit with just a start time. None of them carried
the source offset at all, which is why provenance existed only for video
and why nine audio and subtitle tracks on a branched title had nothing
to place them by.

So it lives in one place now. PesBuf owns the bytes AND the marks;
PesFacts returns a packet's timestamp, source and discontinuity
together, so a parser cannot take one from one packet and another from
the next, because it does not assemble them itself.

The type answers WHICH packet. How a codec reads a timestamp out of that
packet stayed the codec's business at first, and that turned out to be
the same drift one level down: dvdsub read pts alone and returned 0 for
a packet carrying only dts, while everything else took pts.or(dts). Now
there is one derivation. It is not a choice between two fields — for
audio and subtitles there is no reordering, so dts IS the presentation
time and reading it is reading the same value from whichever field the
packet used. Reordering video never calls it; that path reconstructs
display order instead.

Migrated: adts, pgs, dvdsub. dts, ac3 and truehd follow.
2026-08-07 07:19:58 -07:00
Matthew Jackson e32f874387 Restore the refusal: only the video parsers carry provenance
Accepting a table whose marks do not advance was wrong, and a real
22-clip title showed exactly how. The rip failed the drop-volume gate at
84 percent with more frames dropped than kept.

The cause is a fact the code never stated: provenance is VIDEO-ONLY.
Every audio and subtitle parser -- dts, ac3, adts, truehd, pgs -- builds
its frames with `source: None`, while hevc, h264 and vc1 propagate it.
So on that title the video track placed correctly by byte offset and
nine audio and subtitle tracks arrived with nothing to place them by.

The branch added to keep an unprovenanced frame from stranding did the
stranding itself. It held such a frame on its track's current clip,
reasoning that the cursor only ever advances under provenance and so
could not be wrong -- which is true only for a track that eventually
receives some. A track that never receives any stays pinned to clip 0
for the whole title, and every frame past clip 0's OUT mark is dropped.
That is what the disc showed: nine tracks pinned to clip 0, the first
drop nine milliseconds past its OUT.

So the refusal goes back, now with the real reason recorded, and the
test asserts it rather than asserting the behaviour that failed.

Lifting it needs the audio and subtitle parsers to carry provenance
first, stamped from the PES that STARTED each access unit -- the same
rule au_assembly already applies to video. The diagnostics that found
this are kept.
2026-08-07 03:21:21 -07:00
Matthew Jackson b1f9adc82b Say which frame and which marks caused a provenance drop
The heuristic path has always logged its first drop per track with the
frame's timestamp and the clip marks it was judged against. The
provenance path did not log at all, so a title that dropped MOST of its
frames there failed the volume gate without a single line saying which
frame, which clip, or which marks — a full-length rip produced zero drop
events and an error, which is not a diagnosable failure.

Also pins the all-identical-spans hazard as a test. Every PlayItem
referencing one clip file gives every clip the same span, which passes
the tiling check and is therefore "trusted" while carrying no
information at all about which PlayItem a byte belongs to. That is not
what the discs on hand do — theirs have one distinct span per clip — but
the check conflates "the spans tile" with "the spans distinguish", and
only the second justifies placing a frame by its byte offset.
2026-08-07 03:05:32 -07:00
Matthew Jackson 16a894f835 Say how many DISTINCT feed spans a title's clips share
A seamlessly branched title re-references one clip file from several
PlayItems with different mark ranges, and those references share a single
feed span because the bytes are read once. A byte offset alone cannot
then tell them apart.

Whether that is happening on a given title is the fact that decides
whether provenance can identify a clip on its own, and it was not
observable. distinct_spans < clips says it directly.
2026-08-07 02:45:57 -07:00
Matthew Jackson 9bcd752216 Say which placement strategy a title got
Whether a title was placed by the seam plan or fell back to inference is
the single most useful fact about a branched rip, and it was invisible.
The two looked identical in the output, so telling them apart meant
rebuilding and re-ripping — which is exactly what it cost to find that
most branched discs were silently on the inference path.

Log both flags once at plan construction, and log each refusal with the
reason that caused it.
2026-08-07 02:01:47 -07:00
Matthew Jackson c1cd2e4b06 Place a restarting-clock playlist by provenance instead of refusing it
A playlist whose clips each restart their own STC has marks that all
cover the same low values, so they are not points on one title-wide
clock. from_clips refused those tables outright and fell back to PTS-jump
inference.

That refusal predates provenance, and it is only about inference. With
the feed spans tiling the title the clip comes from the frame's byte
offset and the marks are never read across clips: each clip's offset_ns
maps its own private clock onto the output timeline, which is exactly
the right operation for a restarting STC.

Refusing unconditionally therefore turned the seam plan off on most of
the branched discs on hand — the very titles it was written for. They
ran on inference with a timeline minutes past the title's real length:
one 2h29m title came out with its audio and video both spanning 3h15m.

So the refusal now applies only when the spans cannot be trusted, where
it is still load-bearing: without a usable clock or a usable byte offset
there is nothing to place with.

Accepting these tables opens one hole, and it is the same stranding the
refusal guarded against. A frame arriving with no byte offset would fall
through to the mark heuristics, which on this kind of table are
meaningless and can strand a track on a clip it has already left,
dropping the rest of the title. Such a frame now stays on its track's
current clip; the cursor only ever advances under provenance, which is
never wrong about which clip a byte came from.
2026-08-07 01:17:04 -07:00
Matthew Jackson cbafe71769 Resolve a frame clip by binary search, not a linear scan
The provenance lookup runs once per frame per track. A linear scan over the
clip list is fine for the 11-PlayItem fixture and is not fine for the real
hoard: two discs there carry 900 PlayItems in one title, and several carry
250-450, so a scan would be hundreds of comparisons on every one of millions of
frames.

spans_trusted already guarantees the spans tile the feed contiguously and in
order, which makes this a partition point. A repeated clip reuses its first
reference span, so the search walks back to the FIRST entry sharing it — the
bytes are read once, so the material is emitted once, at that entry offset, and
the answer is stable regardless of which duplicate the partition lands on.

Also adds a test at the real 900-clip scale: every clip places, output stays
monotonic across all 900, an arbitrary byte resolves to the right clip out of
ascending order, and a byte past the end belongs to no clip.
2026-08-06 23:22:38 -07:00
Matthew Jackson 3c1375a245 Identify a frame clip by provenance, not by inferring it from timestamps
Four audit rounds each fixed one rule in SeamPlan::place and broke another,
because the question the rules were answering has no answer. Inside a
seamless-branching overlap clip k OUT comes AFTER clip k+1 IN — 57.8s of
overlap on the real fixture table — so a single timestamp is legitimately
inside two clips, and a clip file is not trimmed to its marks, so it also
carries material from before its own IN. No rule over timestamps can say which
clip a frame came from, and each attempt was right for one disc layout and
silently wrong for another: 65s of rewind, 17 minutes stranded, 28 minutes
dropped, 55s refused.

Frames already carry the byte offset they were read from (PesFrame::source,
stamped by the TS demuxer). Clip now carries the byte span its stream occupies
in the title feed, recorded while the extents are gathered. So the clip is a
LOOKUP: the offset falls in exactly one span. There is no decision to get wrong.

Every track of a clip lives in the same stream file and therefore shares one
span, so video, audio and subtitles agree by construction. Divergence between
them — each track guessing separately under its own tolerance — is how audio
and video ended up on different clips and drifted apart in the first place.

spans_trusted gates the whole path: unless the spans tile the feed contiguously
from zero, an offset means nothing and provenance is ignored in favour of the
mark heuristics, which is the 1.6.0 behaviour. A broken map degrades instead of
confidently selecting a wrong clip for every frame. A clip referenced twice
reuses its first span (the bytes are read once) and is still trusted.

Sources that stamp no provenance — a mkv:// remux, the deserialize hop — take
the heuristics, which is what they have always used and where they have always
been right, because they have no overlapping clips to be ambiguous about.

36 timeline tests, six of them new and covering: the overlap case marks cannot
see, all tracks agreeing, out-of-marks material dropped AND counted, a holed
span map, a discontiguous one, a repeated clip, and no provenance at all.
2026-08-06 23:08:13 -07:00
Matthew Jackson 651bfaaa2e Say when an entry is omitted for not being a plain file
A symlink to a directory — a normal way to keep tens of gigabytes of streams
off the system disk — was dropped from the plan silently. `entry.file_type()`
reports the LINK, so such a subtree never enters `dirs`, and `metadata()` then
follows it and reports a directory, so it fails the is_file() test and was
skipped with nothing said.

The result is the worst class this release exists to close: PLAYLIST and
CLIPINF still synthesize, so the folder scans and enumerates titles, every clip
resolves to no extents, the mux takes its clean-EOF path, and a near-empty MKV
is written at exit 0.

It is still skipped rather than followed — following link targets invites
cycles and escapes from the folder — but at the same volume as the
unrepresentable-name skip eight lines above.
2026-08-06 22:48:27 -07:00
Matthew Jackson cce4e5effe Enforce the no-rewind invariant instead of guessing at clip crossings
Round 7 enumerated the state space of SeamPlan::place and found three more
holes in the mark heuristics — the third consecutive round to find a defect
here, each time in the guard the previous round added:

- a clip file opening ON its own IN mark, after a tail frame past the previous
  clip OUT had already advanced the cursor, read as another crossing and jumped
  a clip too far: 28 minutes of clip 6 dropped;
- the last clip was unguarded entirely (`clip + 1 < len` skipped the check), so
  the 9->10 seam rewound 75.6s with dropped_total() at 0 — the exact round-5
  signature, reachable without any corrupt input;
- a glitch mid-clip satisfies "inside the current clip", so the round-6
  stranding survived for the rest of that clip.

Three patches to the same heuristic, three wrong in a new way. So stop patching
it. Everything above the placement is still heuristics over marks, but the
property they exist to serve is now checked DIRECTLY: a track output must not
run backwards. A placement that would rewind by more than
DISCONTINUITY_BACKSTEP_NS is refused; the scan then looks for a LATER clip
containing the frame that does not rewind, which is exactly what a genuine
overlap crossing looks like; and if none exists the frame is dropped and
counted, because emitting it anywhere moves the track backwards.

The tolerance is the backstep constant because B-frame reorder legitimately
emits out of order by a fraction of a second, while a rewind that matters is
orders of magnitude larger.

The cursor moves ONLY when a later clip actually accepts the frame. My first
version of this advanced on a failed search, which strands a track exactly as
before — caught immediately by the round-6 regression test.

All 29 previous timeline tests still pass, including
continuity_preserves_forward_gap. The three round-7 scenarios now have a test
each, with the real 00801.mpls numbers.
2026-08-06 12:31:23 -07:00
Matthew Jackson 8e6f1511ea Round 6: a large backstep only crosses when it would be misplaced
The round-5 fix advanced the clip cursor on ANY backward step over 3s. That
also fires for a corrupt PTS, and for a legitimate STC discontinuity inside one
clip — and nothing moves the cursor back, because a forward step matches
neither past_out nor stepped_back. Every later frame then sits below the new
clip IN and is dropped: on the fixture table that is ~17 minutes of one track
gone, and the only volume gate compares total drops against ALL tracks frames,
so it exits 0.

The branch now also requires the frame to be INSIDE the current clip marks,
which is the only case that would otherwise be silently placed at the old
offset — the rewind. A frame outside them needs no help: the containment check
drops and counts it, the cursor stays put, and the next good frame is placed
normally. Both behaviours have a test, each confirmed to fail without the guard.

Separately, the dir:// PES input path scanned the folder and never applied the
encryption verdict scan_dir exists to produce, so the same folder ripped
through one door and failed through the other asking for a key it does not
need. That logic now lives in one function, session::apply_folder_encryption_verdict,
called by both.
2026-08-06 11:02:48 -07:00
Matthew Jackson d68d73e68d Retry a short read instead of skipping past it
The previous commit bounded the crack scan inspection by the bytes actually
read, which stopped it examining stale buffer bytes from an earlier batch. But
the cursor still advanced by the REQUESTED count, so those sectors were skipped
outright — trading "scans the wrong data" for "silently scans less than it
thinks", on exactly the damaged media where a title key is hardest to find.

The cursor now advances by what was read, so the next iteration resumes where
the read stopped. Floored at one sector so a source returning Ok(0) cannot spin.

Found by reading the fix again rather than by the next audit round.
2026-08-06 10:15:53 -07:00
Matthew Jackson 648bcce5b6 Scan only the sectors the crack actually read
read_sectors returns the number of bytes written and a source may return Ok
with fewer than asked — a recovery read over a damaged region does exactly
that. The crack loop discarded the count and inspected all n sectors of a
buffer that is REUSED across batches, so the tail still held the previous
batch sectors.

Cracking a key from those means cracking from data belonging to a different
extent, possibly a different VTS, while crack_span records the CURRENT one. A
key that opens nothing in this region is then installed, and its wrong
descrambles are only partly caught by the per-sector crib.
2026-08-06 09:31:37 -07:00
Matthew Jackson 45af2152a8 Make the public per-sector descramble refuse a non-pack sector
descramble_region was fixed to require the MPEG-2 pack start code, but
descramble_sector — the PUBLIC per-sector entry point, and the one the
module-level example tells callers to use — still keyed on the byte 0x14 flag
bits alone. The crate own documented guidance therefore led straight back into
the defect this release exists to fix: a VIDEO_TS.IFO sector holding 0x15 at
0x14 while starting 00 26 00 00 loses 1912 of its 2048 bytes, and because that
sector carries TT_SRPT the disc enumerates 38 titles while an image decrypted
from it enumerates 10, at exit 0.

It has no callers inside the crate, which is exactly why it survived three
rounds: nothing exercised it. Putting the guard inside the function rather than
in each caller is what keeps the safe path the easy one.

The integration test that covers it built its sector from the flag byte alone,
which no real scrambled sector looks like, so it stopped representing the path
it names — the same fixture-realism gap already fixed in four other places this
release.
2026-08-06 09:26:15 -07:00
Matthew Jackson abbc725acf A clip join must never rewind the output timeline
A hole in the seam fix itself, found in audit round 5 and confirmed by
measurement against the real 00801.mpls marks already in the test file.

Nothing upstream trims a clip stream to its marks, so a clip file can open with
material from BEFORE its IN. With the cursor on clip 5 (7708.99..7910.79) a
frame at 7845.00 — clip 6 pre-mark lead-in, 8s below clip 6 IN of 7853.00 —
sits further from that mark than the 250ms tolerance, so neither crossing rule
fires. The cursor stays on clip 5, and 7845.00 IS inside clip 5 range, so the
frame is PLACED with clip 5 offset.

Measured: output went from 3714.51s to 3649.51s. Backwards 65 seconds, with
dropped_total() still 0 — no counter, no gate, nothing noticed — while the
whole 57.8s overlap band was emitted a second time over clip 5 written tail.
That is the collide-and-flatten symptom this type exists to remove.

A track own PTS only ever runs forward inside a clip, so a backward step larger
than DISCONTINUITY_BACKSTEP_NS is always a clip change. Such a frame now
advances to the first clip that could contain it — which terminates on its own,
rather than running to the end of the list, which is what the next_in bound
exists to prevent — and a frame still below that clip IN is material the
playlist excludes, so the containment check drops and COUNTS it.

All 27 pre-existing timeline tests still pass, including
continuity_preserves_forward_gap, which is the one that catches an over-eager
heuristic here. The new test asserts the output never moves backwards at a
join and was confirmed to FAIL before the fix.
2026-08-06 09:20:39 -07:00
Matthew Jackson 2e3ba374f3 Test the demux seam gates too
The same gap as the MKV muxer: DemuxSink builds its timeline with
TimelineContinuity::with_clips, but no test ever handed it a title carrying
usable PlayItem marks, so frames_mapped and dropped_total() were always zero
and both gates in finish() were unreachable.

Deleting either left the whole suite green while a demux:// export of a
seamless-branching title wrote a directory of zero-byte track files beside a
populated chapters document and reported success at exit 0.

The test gives the title two clips with marks that exclude every frame, and was
confirmed to FAIL with the gate removed.
2026-08-06 09:05:30 -07:00
Matthew Jackson 84b4a6ddae Test the seam gate that stops a drift-broken title being skipped
Audit finding: both seam gates in MkvMuxer::finish were dead code under test.
set_clips was never called anywhere in the suite, so continuity.dropped_total()
was always zero and neither branch could be reached.

The ordering of the two zero-frame checks is load-bearing and the comment says
so: a title the seam plan emptied also has a zero frame count, and MkvInvalid
is classified by is_skippable_title_stub as an empty nav/menu stub — so
reporting it that way makes an all-titles rip drop a real feature and finish the
rest at exit 0. Swapping the checks left the entire suite green.

The new test builds a two-clip plan (from_clips needs at least two, strictly
increasing) whose marks exclude every frame, and asserts both that the error is
SinkWroteNothing and that it is NOT classified as skippable. Confirmed to FAIL
with the two checks swapped.
2026-08-06 08:45:58 -07:00
Matthew Jackson 3f4bf3f290 Round 3: fix three defects introduced by the round-2 fixes
Auditing my own fixes found all three. None were in the original code.

The VTS crack sort was byte-wise case-SENSITIVE while the filters that select
those files (vts_group_of / is_title_vob) are case-insensitive. On a
case-sensitive volume a set holding vts_01_1.vob beside VTS_01_2.VOB sorted
part 2 first, because V (0x56) precedes v (0x76) — reintroducing exactly the
budget-exhaustion the ordering exists to prevent. Now sorted on the same
uppercase normalisation the filters apply.

Refusing a name that round-trips to empty aborted the WHOLE plan. A sidecar
folder named with a single emoji made a backup un-rippable that 1.6.0 handled
fine, and reported it as a collision with a file that does not exist. It is now
skipped with a warning: the entry is unaddressable either way, but one
irrelevant file should not cost the user their rip.

The mtime check now applies only to files whose CONTENT the plan read — the
IFOs, whose bytes 0xC0/0xC4 place every VOB. Everything else is planned from
size alone, which is already checked, so comparing mtime there bought nothing
and risked a real false positive: disc backups commonly live on exFAT/FAT32,
which stores local time, so a long rip spanning a DST transition would see a
whole-hour shift on an untouched multi-gigabyte VOB and abort hours in.
2026-08-06 08:25:54 -07:00
Matthew Jackson edb6f0035b Catch names the reader cannot tell apart, and in-place content changes
Two dirimage findings, both silent-wrong-output.

The per-directory uniqueness check compared raw host names, but the reader does
not see raw host names: parse_udf_name trims leading and trailing whitespace
and drops any code unit char::from_u32 rejects — which is every half of the
surrogate pairs the encoder emits for non-BMP characters. So " 00000.m2ts" and
"00000.m2ts", or "A<astral>.m2ts" and "A.m2ts", were two entries at plan time
and ONE name at read time. find/read_file take the first match, so a title
resolved to the wrong file extents and muxed the wrong bytes at exit 0 — the
exact shadowing DirNameCollision exists to prevent.

The key is now derived by round-tripping the name through the very encoder and
parser that will be used, so it cannot drift from them. A name that survives
that round trip as empty is refused outright: it would exist in the image and
be addressable by nothing. The new test builds a real folder, calls plan, and
was confirmed to FAIL with the fix reverted.

Separately, the plan-vs-read revalidation compared file LENGTH only, while the
plan depends on CONTENT: a DVD VOB placement comes from bytes 0xC0/0xC4 of its
IFO, and IFOs occupy a whole number of sectors so an in-place rewrite keeps the
length. A re-authoring tool touching the folder mid-rip would pass the size
check while every title extent pointed at stale sectors. mtime is now compared
alongside size, and only when both sides report one, so a filesystem without
timestamps falls back to the old behaviour rather than failing every read.
2026-08-06 08:00:23 -07:00
Matthew Jackson 5fc28056c3 Anchor the folder encryption probe on a unit boundary
probe_folder_encryption sampled AACS units starting from the largest extent
anywhere in the disc. AACS units are 3 sectors and a unit boundary is only
guaranteed at the START of a clip, so that anchor is only correct when the
largest extent happens to be a clip first extent.

For any clip over ~2 GiB it is not. The planner caps an allocation descriptor
at MAX_AD_BYTES = 524287 sectors, so every full piece of a split file ties on
sector_count and max_by_key returns the LAST tie — an extent beginning
(k-1)*524287 sectors into the file. 524287 % 3 == 1, so that start misses the
unit boundary for two file sizes in three.

The 6144-byte sample windows then begin mid-source-packet and the byte read as
the CPI flag is content. Both verdicts fail in a costly direction: a decrypted
folder is rejected as encrypted (DirImageEncrypted on something perfectly
rippable — the exact case scan_dir was added to rescue), or real ciphertext
reads as clear and the mux writes it out as video at exit 0. is_unit_aligned
cannot catch it: it measures against the same wrong base.

Now the largest TITLE first extent, which is unit-aligned by construction and
is also the more meaningful sample — the main feature rather than whichever
fragment happened to be biggest.

The existing tests could not reach this: their fixture m2ts is 786,432 bytes,
a single extent, which is always its own first.
2026-08-06 07:37:59 -07:00
Matthew Jackson 51de027d15 Order a VTS crack by filename, not by directory order
Round 1 removed a largest-first sort from resolve_vts_key because
largest-first is the 1.5.1 garbage bug. Auditing that fix showed it was only
half right: `planned` comes from walking the UDF directory, which yields File
Identifier Descriptors in on-disc authoring order with nothing sorting them. So
deleting the sort did not restore playback order, it left the order undefined —
whatever the disc happened to list first.

DVD-Video numbers a title sets VOBs in playback order by spec (VTS_xx_1.VOB ..
VTS_xx_9.VOB, single digit), so ascending filename IS playback order and is
deterministic regardless of how the directory is laid out.

Order decides correctness here: crack_key shares one sector budget across the
whole extent list, a CSS DVDs biggest cell opens with a long clear run, and CSS
recovers the title key from scrambled data itself. Starting in the wrong place
can exhaust the budget without ever meeting scrambled data, whereupon the
caller falls back to the disc-wide key and the whole VTS is descrambled wrongly
— corrupt PES behind an intact header, written out at exit 0.
2026-08-06 07:31:35 -07:00
Matthew Jackson 96282ecda7 Audit round 1: playback order, silent title drops, image durability
Four fixes from the first audit round. Every finding was verified against a
pinned tree and read directly before being accepted.

resolve_vts_key sorted a VTS title-VOB extents largest-first. That is the 1.5.1
garbage bug, and it grew back in a new code path: the comment claimed it
"matched the scan heuristic", but that heuristic WAS the bug and had already
been fixed in decrypt_keys_for_title, which documents the rule (PLAYBACK ORDER,
never largest-cell-first) and pins it with a regression test. A CSS DVDs biggest
cell opens with a long clear run and crack_key shares one sector budget across
the extent list, so starting there can exhaust it without ever MEETING
scrambled data — and CSS recovers the key from scrambled data itself. The crack
then returns None, the caller falls back to the disc-wide key, and every VOB in
that VTS is descrambled wrongly: corrupt PES behind an intact header, written
out as a complete extract at exit 0.

parse_pgcit dropped titles silently in THREE places — an unparseable PGC, an
out-of-range PGC index, and a truncated entry table. The finder caught one; the
other two turned up on reading the function. parse_vmg already counts and warns
per skipped title SET for exactly this reason, and this was the last place a
disc could quietly report fewer titles than it has.

write_image called flush() and returned Ok. flush() only pushes bytes into the
page cache and promises nothing about durability, so a 6-90 GB image could be
reported complete while still unwritten — a crash or an unmounted volume then
leaves a truncated file the caller was told was finished. Now into_inner (so a
buffered-write error surfaces instead of being dropped by BufWriter::drop)
followed by sync_all.

timeline used abs() on a saturating_sub result. Every other comparison in that
module is saturating because the timestamps come off a disc and are not
trusted; abs() panics on i64::MIN, which saturating_sub can produce.
2026-08-06 07:11:52 -07:00
Matthew Jackson 6e2ee9e002 Rename is_scrambled to has_scramble_flag_bits
The 1.6.1 DVD fix removed the USE of the loose predicate but left the
predicate sitting there with the better name. Anyone asking "is this sector
scrambled?" finds is_scrambled before is_scrambled_pack, and reintroduces the
defect that destroyed 1912 bytes of a real VIDEO_TS.IFO — the sector carrying
TT_SRPT — so the disc enumerated 38 titles and an image decrypted from it
enumerated 10, silently, at exit 0.

Byte 0x14 only means "scrambling control" inside an MPEG-2 pack. In an IFO,
UDF or ISO 9660 sector it is whatever that format stores there. The new name
says what the function actually tests and nothing more, so the honest question
has the obvious name and the dangerous one has to be asked for deliberately.

Its doc comment also claimed a caller, decrypt::decrypt_sectors, that does not
exist — so the name was an invitation and the documentation was an argument
for accepting it. It has no production callers at all; it stays public because
an integration test asserts the flag extraction directly.
2026-08-05 22:27:04 -07:00
Matthew Jackson 0f8035b15a Scope the qa matrix to the platforms each crate actually supports
The first cut ran release-profile tests on all three platforms everywhere,
which invented coverage no crate had ever claimed. bdemu is Linux-only — its
ci.yml has no macOS or Windows job at all, and its tests call
ExitStatus::signal(), which does not exist on Windows, so they do not compile
there let alone run. The rest follow the policy ci.yml already set: tests
execute on Linux and macOS, Windows compiles them so the cfg(windows) halves
still get codegen before release time.

That policy is now explicit in qa.yml as a windows-build job rather than
implied by which jobs happen to exist.

Recorded, not papered over: running the suites on Windows DOES find real
defects — autorip has five Windows path-handling failures in mover.rs and
freemkv-engine has three tests hardcoding /dev/null. freemkv ships a Windows
GUI, so they matter. They are pre-existing rather than 1.6.1 regressions, and
fixing them is its own piece of work.

Also build the binary before cli-parity.sh. cli-integration.sh builds its own;
cli-parity.sh only checks for one and exits 2 with "build it first", so both
legs died before running a single case.
2026-08-05 21:22:30 -07:00
Matthew Jackson f67fd92b60 Give Windows a free-space gate, and stop two tests timing the scheduler
Three release-profile failures on macOS and Windows, all found by the qa gate
on its first run. Release-profile tests on those platforms had never run
before it existed, so none of these were regressions — they had simply never
been visible.

available_space returned None on every non-unix target. That did not merely
skip a test, it skipped the GATE: a Windows user extracting a disc to a full
volume got a confusing failure part-way through instead of a clear refusal up
front, and Windows is where the GUI ships. GetDiskFreeSpaceExW is declared
directly against kernel32, matching how scsi::windows already reaches Win32
rather than pulling in a binding crate for one call. It asks for
FreeBytesAvailableToCaller, which accounts for per-user quotas — the same
question f_bavail answers on unix.

The other two asserted on wall-clock timing with no margin:

- sleep_until_halted_wakes_mid_sleep bounded the wait at 350 ms and measured
  377 ms on a loaded runner. That bound measures the scheduler, not the wake.
  What the test is for is distinguishing "woke because the flag flipped" from
  "woke because the 10 s timeout expired", and 2 s does that just as well.

- abandon_loses_to_a_close_already_committed released at 600 ms against two
  300 ms grace windows plus a 250 ms poll cadence, so the windows could expire
  first and the caller abandoned — a race, not a defect. The intervals are
  scaled up so jitter is small relative to them; the ordering under test is
  unchanged, only the margin.
2026-08-05 21:20:29 -07:00
Matthew Jackson 7ced0d9b61 Stamp every qa push as a release candidate
Every push to qa now tags v<version>-rc<N>, N incrementing, before the gates
run. That answers "which build is on qa, and is it the one I tested?" without
anyone having to remember it.

The tag lands whether the run goes green or red, deliberately. A red candidate
needs a name more than a green one does: "rc3 failed release-tests on windows"
is a sentence you can act on, "qa is red" is not. Red on qa is the gate doing
its job — the branch saying this is not production worth yet.

release.yml now excludes v*-rc*. Its trigger was v*, which matches the
candidate tags, so without this every push to qa would have built and PUBLISHED
a GitHub release — including for the candidates that failed.
2026-08-05 21:03:01 -07:00
Matthew Jackson c2c00a874c Add the qa gate: dev for speed, qa for proof
dev -> qa -> main, across every public repo.

dev is where work lands and is meant to be pushed to often, so ci.yml stays
the fast answer: fmt, clippy, unit tests, leak-guard. qa is the release
candidate, and qa.yml is the claim that a commit is production worth —
release-profile tests on all three platforms, and the Linux cross-target
clippy that the local precommit gate has always run but CI never did.

Release profile matters as its own gate: overflow checks are off, debug_assert
is compiled out, and inlining changes what the optimiser can prove. A test
that only passes in debug never guarded the binary anyone ships.

qa.yml checks siblings out at qa rather than dev. This is not a monorepo, and
a qa run resolving its dependencies from unreleased dev tips would go green on
a combination that is not the one shipping — the exact mismatch the branch
model exists to prevent. ci.yml now tracks whichever branch triggered it for
the same reason, since it also fires on qa.

The consequence is that repos move to qa together, in dependency order. qa is
backfilled from main so an unchanged crate still presents working code to the
crates built on it.
2026-08-05 20:59:34 -07:00
Matthew Jackson 87463ed3ee Require a pack start code before descrambling a sector
css::is_scrambled reads bits 4-5 of byte 0x14 and nothing else. That is a
sound test once a caller has committed to a title's VOB data, where every
sector is an MPEG-2 PS pack and byte 0x14 always means what it says.
descramble_region is not such a caller: it is handed arbitrary regions of a
disc, so it also sees IFO, UDF and ISO 9660 sectors — raw structures where
byte 0x14 is whatever that format happens to store there.

Measured on a real disc: the second sector of VIDEO_TS.IFO holds 0x15 at
offset 0x14 while starting 00 26 00 00, which is not a pack. The flag test
read it as scrambled, descrambled it, and destroyed 1912 of its 2048 bytes.
That sector carries TT_SRPT, so the title table went with it — the disc
enumerated 38 titles and an image decrypted from it enumerated 10, silently,
at exit 0.

is_scrambled_pack already existed with the right predicate. Use it here. It
costs nothing: a genuinely scrambled VOB sector always carries the pack start
code, and no IFO sector does.

Verified end to end — the decrypted image's `info` output is now identical to
the source disc's, 38 titles both, differing only in the CSS: Encrypted line.

The fixtures moved with it. Four of them built a sector by setting byte 0x14
alone, which no real scrambled sector looks like; they now build packs.
2026-08-05 20:59:14 -07:00
Matthew Jackson 0c5e419b07 Report the resolved extent alongside each IFO read
The diagnostic hashed IFO contents but not where they came from, so a
content difference could not be told apart from a path resolving to a
different place. It now prints file_start_lba and file_extents next to the
hash.

That measurement is what inverted this investigation: both a CSS disc and
its decrypted copy resolve VIDEO_TS.IFO to the same extent, and only one
sector of the fourteen differs — by exactly a descrambled payload with the
scrambling-control bits cleared. So the decrypted copy holds the correct
bytes, and an iso:// scan of a CSS disc is parsing a still-scrambled IFO
sector, because scan_iso opens a plain FileSectorSource.
2026-08-05 20:24:20 -07:00
Matthew Jackson 218cd1f6f8 Diagnostics that localise the decrypted-DVD title loss
Two opt-in dumps, driven by FMKV_IMAGE, that narrow where a decrypted DVD
image loses titles:

dump_title_sets_for_an_image reports what survives parse_vmg. On one disc
the CSS image yields 13 title sets and 38 titles; its decrypted copy
yields 8 and 10. Sets 8, 9, 10, 12 and 13 are dropped outright, set 11
parses but returns no titles at all, and set 7 returns 3 of 5.

dump_vts_ifo_reads_for_an_image reads every VTS IFO and hashes the
CONTENT. All thirteen are byte-identical across the two images, so
parse_vts is handed the same bytes and the same TT_SRPT info and still
fails on one of them — the divergence is in what it reads from the READER
afterwards, which is file_start_lba and the PGCIT.

An earlier version of the second dump compared only length and magic and
so wrongly reported the images as identical; it hashes the bytes now.
2026-08-05 20:14:46 -07:00
Matthew Jackson 2c4851e6c1 Say when a title set is dropped from a DVD scan
parse_vmg skipped any title set whose parse failed, with no log and no
counter. A real disc enumerated 38 titles from one image and 10 from
another, and the 28 discarded failures were invisible — the symptom read
as a scan difference rather than as dropped reads, which is most of why
it took so long to localise.

Behaviour is unchanged: a disc may legitimately carry placeholder TT_SRPT
entries, so one failure is still not fatal. It now warns per skip with the
title set and the error, and once at the end with kept-versus-declared.

Also adds an opt-in diagnostic that reads every VTS IFO from an image, to
separate a read failure from a parse failure. It reports all 13 sets
reading identically from both a CSS image and its decrypted copy, which is
what proves the 38-to-10 loss is downstream of these reads.
2026-08-05 19:41:43 -07:00
Matthew Jackson 7511adb64e Two opt-in scan diagnostics for the DVD decrypted-image defect
dvd_placement_invariant_on_a_real_folder checks, per title set, the sum
ifo.rs relies on: file_start_lba(VTS_nn_0.IFO) + vtstt_vobs must land on
VTS_nn_1.VOB. It reports all 13 sets correct on a real DVD folder, which
is what excluded placement as the cause.

dump_titles_for_an_image prints every title a scan produces with the
numbers canonical_title_order sorts on. It is what showed the real shape:
the same disc scans to 38 titles as a CSS image and 10 once decrypted,
with identical capacity and a byte-complete image.

Both are #[ignore]d and read their target from the environment, so they
cost the gate nothing and are there for whoever picks the defect up.
2026-08-05 19:33:58 -07:00
Matthew Jackson 84a6c0d373 Round 5: fix the gates added in round 4, and two placement holes
The zero-frame check ran before the seam gate, and its error is
classified as a skippable nav stub — so a title the plan dropped
ENTIRELY was reported as an empty stub and an all-titles rip would omit
a real feature and finish the rest at exit 0. The seam case is decided
first now, with a code that is not skippable.

The demux sink read a frame's track kind out of the FILTERED slot, which
is empty for a class the export drops. On an audio:// or sub:// export
the video track was therefore called non-video and handed the permissive
crossing rule — the same defect round 4 fixed for a Dolby Vision layer,
reintroduced one file over. Video tracks are now recorded before the kind
filter, beside the primary-video reference that exists for this reason.

Its frame counter counted frames PLACED, not written, while its name and
doc claimed otherwise. Renamed and documented for what it is, including
that it cannot see a single lost track among many.

Placement: files in a subdirectory of VIDEO_TS were never given data.
They were declared at full size with no extents, so they appeared in the
tree and read as nothing. The same folder under BDMV was always placed
correctly. And the duplicate title-set guard keyed on the constraint maps,
so an IFO declaring no offsets inserted nothing and a colliding second IFO
went undetected — it keys on the groups seen now.

Display for SeamPlanDroppedMost and ShortImageRead discarded their
payloads, and four new variants were missing from the code-uniqueness
test.
2026-08-05 18:40:14 -07:00
Matthew Jackson 7cc5b0c93d Test the image-size cap against a real folder
The cap was added without a test. Its companion — a modest oversize is
honoured as a gap — already existed, so this pins the other side: an
offset past the ceiling is refused rather than grown into, which is what
keeps a rewritten IFO from planning a multi-terabyte image.
2026-08-05 18:30:23 -07:00
Matthew Jackson f4fc730d83 Make the drop count gate the verdict, not just the log
Round 3 counted frames the clip marks excluded and reported them at
finish. Counting is not bounding: the only other gate was a global
zero-frame check, and its error is additionally classified as a skippable
nav stub, so a title whose marks do not line up with its PES clock could
discard almost all of itself and still exit 0 — a two-hour feature
emitting seconds, which is the defect this change set already shipped
once. Dropping more than was kept is never a real join, so it now fails.

The demux sink had no zero-output guard at all, so a fully-dropped title
finished cleanly: a directory of zero-byte track files beside a populated
chapters document. It now refuses, keyed on frames having been OFFERED —
a chapters-only export, or a track class the title does not carry,
legitimately writes none, and two existing tests correctly said so.

A title set's placement group is the parsed number, so VTS_01_0.IFO and
VTS_1_0.IFO land on one key and the second silently overwrote the first's
constraint, placing a VOB where the IFO the reader uses does not point.
Refused rather than resolved by arrival order.
2026-08-05 18:25:47 -07:00
Matthew Jackson db0ca3367f Key the seam-crossing rule on reorder, not on driving epochs
A Dolby Vision enhancement layer is a second video track: it does not
drive epochs, but it does carry B-frame reorder. The crossing rule was
keyed on driving epochs, so the EL took the branch whose premise is that
the track has no reorder — and its ordinary reorder dip near the end of a
clip, which during an overlap also lands inside the next clip's range,
was read as a join. The EL was then placed on the next clip's offset, out
of step with the base-layer frame it must be co-timed with by the width
of the overlap: the same desync the per-track cursor was added to remove,
reintroduced for one track.

The property the rule actually depends on is whether a backward step can
be reorder, so it is now keyed on that. Both sinks derive it from the
track kind rather than from the epoch driver.
2026-08-05 17:52:13 -07:00
Matthew Jackson cc870a4614 Keep the seam plan to Blu-ray, and stop a missed crossing truncating a title
Two findings from the same escalation, both silent-wrong-output.

The plan was built for every multi-clip title. Only a Blu-ray PlayItem's
IN/OUT are positions in the clock the PES PTS runs on. HD-DVD fills the
same fields from the XPL's title-relative times and a DVD's come from
cell tables, so a plan built from them is an identity map with a drop
filter: it suppresses the layer-break rebase inference performs, and
drops whatever falls outside marks the PTS was never measured against. An
earlier reading of this called HD-DVD safe because its marks are
contiguous and every computed offset was zero — true, and irrelevant,
because they were zero in the wrong clock. Gated on the content format,
with a test using an HD-DVD-shaped table that the clock check alone
accepts.

The crossing test was also one-shot. A table whose clips restart their
own bases could miss it, and a missed crossing STRANDS the track: every
later frame falls outside the stranded clip's marks and is dropped for
the rest of the title. Counting drops, which is all the previous round
added, does not bound them. A table that is not one advancing clock is
now refused outright and falls back to inference, which is the documented
safe path for those titles.

Also from the same round: read_sectors added an unchecked lba + i, where
callers deliberately saturate their LBAs — a wrap folds the read back to
a low sector and hands the muxer another file's bytes. classify added 1
to two numbers parsed verbatim out of a filename. read_head used a single
read() where a short read on a network mount silently records no
placement constraint at all. And the page-cache eviction added last round
released only the read that crossed its threshold rather than everything
accumulated, so seven eighths of what was read stayed pinned.
2026-08-05 17:31:43 -07:00
Matthew Jackson 211d293aee Make the name-cap tests exercise the planner, not the constants
The round-1 tests asserted arithmetic about MAX_CS0_NAME_BYTES and never
called plan(), so both would have passed with the guard deleted — which
is the failure mode this audit exists to catch, committed by the audit's
own fix. They now build a real folder containing a 255-byte name and
require the planner to refuse it, plus a companion proving a name at the
cap is still accepted so the guard is not merely refusing everything.

The subdirectory cap keeps its arithmetic-only test — creating 65,535
directories is not reasonable in a unit test — but now says so instead of
implying coverage it does not have.
2026-08-05 17:21:43 -07:00
Matthew Jackson abf47137ef Audit round 2 fixes: an unsafe default, four omissions, and two swallowed errors
The folder encryption probe returned "not encrypted" when it had sampled
nothing at all — a title shorter than one aligned unit skipped the loop
entirely. That verdict CLEARS the structural one an AACS directory
raised, so a genuinely encrypted folder would have been ripped as clear
and written ciphertext as video at exit 0. With no evidence it now keeps
the structural verdict, and its bounds arithmetic no longer trusts
disc-derived values not to wrap.

Reading an IFO header swallowed every I/O error and returned an empty
buffer, which sent each placement offset through unwrap_or(0) and
recorded no constraint at all — a permission error on one file produced a
silently misplaced VOB. The directory walk swallowed the same class while
claiming to skip only vanished files. Both now propagate; only NotFound
is skipped.

Four things the round-1 changes left inconsistent: two new error codes had
no doc comments, were absent from the io::Error mapping, printed no path
in Display, and were missing from the test that proves codes are distinct.
The demux sink dropped frames silently while the MKV muxer reported them.
And set_clips had been inserted INTO write_frame's doc comment, leaving
write_frame undocumented and its paragraphs describing the wrong function.

uid/gid used 0 as "not specified"; UDF's sentinel is 0xFFFFFFFF, and 0 is
root.
2026-08-05 17:16:03 -07:00
Matthew Jackson 00332e4dba Bound the synthesized image, and stop it pinning the page cache
A DVD title set records where its VOBS begins as an offset inside its own
IFO, and the planner honours that offset because honouring it is what
makes a real backup readable. Nothing bounded it: a regenerated .BUP or a
hand-assembled folder naming an offset far past the content grew the
image to wherever it pointed — a u32 sector count reaches ~8.8 TB, and
writing that to an iso:// destination fills a disk with zeros before
anything notices. Capped at 128 GiB, which clears BD-100 with room.

Metadata is materialized up front and held for the life of the image, at
a 2 KiB File Entry per node, so the 100,000-entry cap alone permitted
~205 MB of it for content of no size at all — and the mux holds two
images at once while probing. The module claimed a budget of a few MiB;
that budget is now enforced rather than asserted.

Host reads had no page-cache eviction. The ISO source documents what that
costs, measured: an 85 GB read pins the whole file, starves the writer,
and collapses the mux to 2.7 MB/s against 70 MB/s isolated. A folder
source reads host files the same way, so it now uses the same eviction —
the hints move from private-to-that-module to crate-internal rather than
being reimplemented.
2026-08-05 17:10:19 -07:00
Matthew Jackson 6152028271 The VMG's 0xC4 is TT_SRPT, not VMGM_C_ADT
The placement code skips that field for the Video Manager, which is
right, but said it was skipping it because the field is the menu cell
address table. It is the title search pointer table, and it is an offset
inside the IFO rather than a pointer to another file — which is the
actual reason it constrains nothing. ifo.rs reads the same offset under
the correct name, so the two would have drifted.
2026-08-05 17:03:19 -07:00
Matthew Jackson abe464d799 Audit round 1 fixes: sparse-track joins, silent drops, and two encoder wraps
A sparse passive track — a subtitle with no event near a clip's mark —
was held to the dense-video crossing window, so it stayed on the previous
clip's offset until its PTS passed that clip's OUT and every event in
between was mistimed by the overlap. Video keeps the tight window,
because its backward steps are also B-frame reorder; passive tracks have
no reorder, so any backward step into the next clip's range is a join.

Frames the marks exclude were dropped without a trace. Dropping is right
at a join, but this codebase has shipped complete-looking wrong output
before, so the count is kept per track and reported when the mux
finishes, alongside the pre-cluster counter that exists for the same
reason.

A File Identifier Descriptor records its name length in one byte, and the
length was narrowed with a cast: a 255-byte name — POSIX NAME_MAX,
entirely ordinary — encodes to 256 and wrote zero, which would read every
later entry in that directory from the wrong offset. A directory's link
count is 16 bits and was computed as 1 + subdirectory count, which the
global entry cap alone permits overflowing. Both are refused while
planning, where the tree can still be rejected cleanly.

The module and struct docs described inference as the whole algorithm;
they now say which path decides what.
2026-08-05 16:58:49 -07:00
Matthew Jackson 862c4e90ab changelog: dir:// is not in 1.6.1
The folder reader produces a wrong title list on a real DVD — the title
table is read at a sector offset relative to VIDEO_TS.IFO's start, and a
synthesized layout does not put that file where the offset resolves the
same way. Output is silently wrong at exit 0, so the feature is not
wired to the CLI in this release. The library module stays, unreachable,
and ships once both disc families are verified against real content.
2026-08-05 16:44:23 -07:00
Matthew Jackson 7cfeadbcd3 changelog: dir:// as a source, and the multi-clip Blu-ray timing fix 2026-08-05 15:47:47 -07:00
Matthew Jackson 9742fe00fd Cross a clip join per track, not on the video's frame
The first cut placed every track with the cursor the video had moved. At
an overlap join the previous clip's audio is still arriving after video
has crossed, and those tail frames sit inside both clips' mark ranges —
so they took the new clip's offset, jumped forward by the overlap, and
collided with the new clip's own audio, which the muxer's monotonic nudge
then flattened. A remux confirmed it: the timeline length was already
correct and the original symptom was still there, 169 audio packets on
the tick floor.

A track's PTS only runs forward inside a clip, so its own backward step
to the next clip's IN is its crossing. That is per track, so the cursor
is too.
2026-08-05 15:46:40 -07:00
Matthew Jackson e814310a3e rustfmt the seam-plan tests
Line-wrapping only, no behaviour change.
2026-08-05 15:28:40 -07:00
Matthew Jackson 480c873be1 Merge branch 'fix/mpls-seam-timeline' into dev 2026-08-05 15:24:37 -07:00
Matthew Jackson 5043baaebe Place Blu-ray clips by the playlist's marks, not by guessing at PTS jumps
A seamless-branching title's PlayItems do not chain contiguously: one
clip's OUT can sit after the next clip's IN, where the disc stores the
join twice, or before it, where the playlist skips material. The mux
never saw those marks — its own header said so — and inferred seams from
PTS jumps instead.

Inference cannot recover this. A forward jump is ambiguous: it means the
playlist skipped, or it means frames were lost to damaged media, and
compressing the latter would falsify timing on exactly the rips that most
need it faithful. An overlap smaller than the B-frame reorder threshold
is invisible to inference entirely, and its duplicate content then
collided in the muxer, where the monotonic nudge flattened a run of audio
onto the tick floor and put sound ahead of picture for the rest of the
film.

Measured on one 11-PlayItem title: the file declared 7893.385 s, which is
what the playlist says the title is, and carried packets to 8029.298 s.
Both numbers came from the same program on the same disc. Four skips
totalling 135.9 s became dead timeline, and a 1.79 s overlap put audio
1.8 s ahead at the half-hour mark. Five of forty-seven titles were
affected; every single-clip title was exact.

So the marks are read. Each clip contributes exactly out - in, laid end to
end, so the output runs as long as the playlist says and a join never
rewinds. Titles without usable marks — DVD, HD-DVD, file sources — keep
the inference path unchanged, and clips that already chain contiguously
produce a constant offset, which is pinned by a test.
2026-08-05 15:24:31 -07:00
Matthew Jackson 4d63a11d63 Read a disc folder as an input: dir:// becomes a source
Users keep discs as extracted folders — a DVD VIDEO_TS or a Blu-ray BDMV,
usually a backup that is already decrypted. dir:// could only ever be a
destination, so those folders could be produced and never read back.

Everything above the sector layer wants a UdfFs over a SectorSource, and
every UdfFs read re-reads the ICB off that source at call time, so a
folder has to present itself as sectors. It does: dirimage plans a block
layout over the real files, encodes a UDF 1.02 filesystem for the
metadata, and serves data straight from disk. read_filesystem then parses
it exactly as it parses a disc, so nothing above changes — and the
iso:// arm of input() is now shared rather than duplicated, so dir://
inherits its decrypt gates, title selection and stream pruning.

The encoder is validated by more than its own reader: macOS mounts the
synthesized image and the mounted files compare byte-identical to the
originals. A round-trip through our own parser could not have shown that
— the tag CRC seeds at zero, and a wrong seed would satisfy us and no
real driver.

DVD placement is not free packing: a VTS IFO records where its title
VOBS begins relative to itself, so the VOB has to land exactly there.
Unsatisfiable marks fail loudly rather than misplace the file. 3D folders
are refused for now: the scanner detects SSIF and the planner cannot
alias its extents yet, so accepting them would produce quiet nonsense.

Left for later: metadata capture, HD-DVD, FMTS, encrypted folders.
2026-08-05 15:24:18 -07:00
Matthew Jackson 2310e65f44 Make iso:// a destination for any image source, not just a drive
Writing an iso:// meant a raw sector copy off a drive, so an existing
image could only ever be a source. Decrypting one you already had meant
finding the disc again.

The engine's copy is the recovery path — mapfile, multipass, damage-jump,
auto-resume — and all of it exists because optical media returns read
errors. A file does not, so a non-drive source gets write_image instead:
sectors in, bytes out, once, no recovery machinery. Keeping them apart is
not just tidiness. The mapfile identity check compares AACS unit keys and
the VID, both empty for an already-decrypted source, so identity passes
for any such source — a second run with different input to the same
output path would resume over the previous image and report success.

A short read is an error rather than a zero-fill: padding a truncated
source yields an image that looks complete and is not, which is the worst
outcome for a copy someone means to keep.
2026-08-05 14:45:01 -07:00
AnimeFN 4bd1c9210c Fix NTSC chapter/duration drift: dvd_time_t is a timecode, not seconds
`dvd_time_t` stores H:M:S:F as non-drop-frame timecode with a rate flag,
not elapsed wall-clock time. On NTSC the seconds field advances every 30
frames, but a frame lasts 1001/30000 s, so 30 frames occupy 1.001 s of
real time. Reading H:M:S as literal seconds under-reports real time by
exactly 0.1% — 3.6 s per hour, growing with elapsed time, which is what
made chapter marks drift ~4 s by the 67-minute mark.

Convert the whole timecode through an integer frame count and apply the
exact 1001/30000 fraction once, instead of reading H:M:S literally and
dividing only the frame remainder by a decimal 29.97. Chapter marks sum
per-cell frame counts as integers and convert once per chapter, so every
mark lands on an exact frame boundary instead of accumulating f64
rounding across a long title.

PAL is arithmetically unaffected: 25 frames = 1.000 s exactly, so the
old and new paths agree to within one ULP. Sweeping all 900k H:M:S:F
combinations under 10 hours, 12 differ, by at most 8.9e-16 s — the new
path divides one exact integer instead of adding a rounded fraction to a
large one, so where they differ it is the more accurate of the two.

Five existing NTSC fixtures asserted the old literal-seconds values and
were updated to real seconds (each is exactly 1.001x its old figure);
`bcd_secs` now documents that its argument is timecode, not real time.

Fixes #1
Reported-by: AnimeFN <admin@animefn.com>
2026-08-05 13:48:11 -07:00
Matthew Jackson bfce73e73c changelog: write it for users, not for the people who fixed it
Entries had grown into investigative narratives — root causes, disc
counts, PIDs, percentages, function names — which is the right level for
a post-mortem and the wrong one for a public changelog. Each entry is now
one to three sentences leading with what a user observes.

Halves the file (1538 -> 735 lines) without dropping a version. API
signature changes stay explicit, since a consumer migrates against those.
2026-08-03 08:31:04 -07:00
Matthew Jackson ca4e6f5a13 restore freemkv-unlock path dep for local dev (post-v1.6.0) 2026-08-03 05:57:31 -07:00
Matthew Jackson 0158112c35 v1.6.0: bump version (freemkv-unlock git-pinned for the tag) 2026-08-03 05:57:28 -07:00
Matthew Jackson b55884675c restore freemkv-unlock path dep for local dev (post-v1.6.0) 2026-08-02 22:35:49 -07:00
Matthew Jackson 455d953144 v1.6.0: bump version (freemkv-unlock git-pinned for the tag) 2026-08-02 22:35:47 -07:00
Matthew Jackson c8eafad5ef Merge branch 'fix/label-pid-binding' into dev 2026-08-02 18:56:10 -07:00
Matthew Jackson fe795fb472 Bind borrowed stream labels by the stream they name, not by a slot
A vendor label's `stream_number` is a slot in the one stream table its
config blob describes. The labels merged in from the playlists — to
cover streams the vendor named nothing for — carried a different number
entirely: a dense counter over every distinct stream found while
scanning the whole disc in directory order, related to no playlist's
slot numbering at all. Two coordinate systems, one field name. The
merge matched them by equality and the binder then counted streams
against the result.

Measured over the 44-image corpus: 22 discs merge such labels; of the
566 places one lands on a stream, 443 (78%) are a stream it does not
describe — the label states the PID it read itself from and it is a
different one. 142 of those are stopped by the language check 4da6d3a
added; 301 are applied. Those labels carry no editorial payload, so the
direct damage is confined to codec text — but the polluted list is also
what the anchor gate reads, and on 11 disc/stream-type pairs it is what
decides the anchor, which is how it reaches the vendor's forced and SDH
flags. 53 anchor facts are harvested off a merged slot. The clip-info
orphans had the same shape, numbered from `max + 1` of a list they share
no coordinate system with.

A label now either NAMES its stream — `StreamId { clip, pid }`, read out
of the very table `disc::bluray` builds the stream from — or it does
not, and only the ones that do not are ever reached by counting:

  * `label_at` returns vendor labels only, so the two numberings can no
    longer be confused by construction.
  * Playlist and clip-info labels bind by id. Exact, no anchor, no
    sequence, no ordinal.
  * A named stream outranks a guessed one, so an editorial flag reaches
    a stream only where the disc's own numbering puts it there.
  * The presence of an id is the provenance the anchor gate was missing.
    It reads the vendor's slots alone now, so a slot the vendor never
    named no longer breaks the sequence — under-yield is the normal
    shape of these blobs — and a title with fewer streams than the list
    has slots is no longer eligible to hold it. Ranking prefers the
    title that positively confirms most of the list.
  * An orphan is in no playlist, hence in no title, so it binds to
    nothing rather than to whatever counted its way.

The MPLS floor is one label per physical stream keyed by `(clip, PID)`,
not by `(type, language, codec, pid)`: the same PID in two clips is two
streams, and the old key collapsed them. Its `stream_number` is now the
entry's real slot in its own playlist's table.

41 of 44 images are byte-identical; all three that move lose a label
they should not have had, and no feature title changes on any image. A
dozen featurette playlists stop reporting a feature subtitle's SDH
marking on their own unrelated subtitle; eleven menu and bonus titles
stop advertising the feature's object-audio format on plain stereo; and
on a disc whose every title carries a single audio stream — too short a
table to anchor anything — a regional-variant tag asserted on all
seventeen titles is now asserted on none, in exchange for every title
stating the codec it actually carries, which none of them did.

Six tests had been asserting the invented numbering, including one
pinning the disc-global counter as a deliberate property.

Also fixes a defect in the same family that the corpus work surfaced:
`pgs_forced_probe::apply_verdicts` set `forced` but left `qualifier`,
so a demoted track shipped with a metadata sidecar calling it forced
next to a Matroska header saying it is not. Only a forced claim is
cleared; an SDH marking is not the probe's to touch.
2026-08-02 18:52:40 -07:00
Matthew Jackson 5b095c56b6 Say plainly that the vendor playlist format is not a specification 2026-08-02 18:27:01 -07:00
Matthew Jackson 383adacec3 Merge branch 'fix/paramount-forced-sub-semantics' into dev 2026-08-02 18:22:25 -07:00
Matthew Jackson 5115eb14da Read the vendor forced-subtitle field as the enumeration it is
One vendor's playlists.xml carries a per-subtitle-slot cell that looks
like a boolean, and it was parsed as one: value 1 meant forced, anything
else meant not forced. Across every image in the corpus that uses this
format the cell takes four values, and 1 is not the forced one.

Decoding three of those discs and counting every PGS display set:

  * 1 marks a FULL dialogue track that additionally contains some
    forced-narrative signs. All nine cells bearing it on one disc are
    full tracks of 949-1411 display sets; all seven on another are full
    tracks of 1602-1651. Neither disc has a small track among them.
  * 2 and 3 mark a DEDICATED forced-narrative track, in its own trailing
    stream slot, duplicating a language that already holds a full track.
    The two 2 slots measured are 15 and 10 display sets with every one
    flagged forced; the four 3 slots are 7, 14, 23 and 59 against
    1216-2655 on the tracks they duplicate.

So the old reading was wrong in both directions — it flagged full
dialogue tracks forced, which is how one language came to present as two
identical full subtitle tracks with one of them marked forced, and it
threw away the cells naming the real forced tracks.

Content could not have corrected this afterwards. Clearing a wrong
forced label needs a disc whose authoring sets forced_on_flag, and on
the measured disc carrying four genuine forced tracks not one display
set anywhere sets it — there, the vendor cell is the only evidence there
is. The classification is now explicit: only a dedicated forced slot
earns the flag, an unrecognised value never does, and the
contains-forced-signs value is dropped rather than weakened into a
forced label, since a wrong forced flag on a full dialogue track is the
user-visible defect while a missing hint costs nothing.

Four of the crate's own tests had been asserting the boolean reading;
their subject was positional alignment, so they keep it and now use a
real forced value. The other two parsers that emit a forced qualifier
from vendor metadata were audited and are structurally immune — in both,
the forced marker names a slot of its own rather than hanging off a full
track's entry, so the failure has no encoding there — and each is now
pinned by a test saying so.
2026-08-02 18:21:32 -07:00
Matthew Jackson 4da6d3ab70 Bind vendor stream labels by PID, not by per-title ordinal
A vendor label list describes ONE playlist's stream table, but
apply_labels re-numbered it from 1 inside every title. Where sibling
playlists cover the identical feature clip and enumerate different
subtitle sets, identical ordinals resolve to different PIDs, and the
same physical stream came out flagged forced in one title and not in
the other. On the title such a disc offers as its rip target that put
`forced` on an 873 MB full-dialogue English subtitle track — the
reported "I see English and English (Forced), they are identical".

The blobs are not wrong; the binding was. The list carries no playlist
id, but it carries a language per slot, and that sequence is a
fingerprint: on the corpus exactly one title's per-type language
sequence reproduces the list position for position, and content
confirms that title's binding is the correct one. So binding is now
two-tier:

  * The title whose whole per-type language sequence sits on the list
    (>= 2 streams, longest wins) is the ANCHOR — the table the list is
    describing. Each of its slots yields a `(clip, PID) -> label` fact,
    and a PID is the same elementary stream in every playlist that
    plays that clip, so sibling playlists bind through the map. A slot
    the anchor never showed us is not bound at all.

  * Streams no anchor fact reaches still bind by the STN ordinal, but a
    label whose language contradicts the stream it would land on is
    dropped. Subtitle labels carry nothing but the qualifier, so an
    unverifiable one is all risk and no gain: off the authoritative
    path they additionally require both sides to STATE a language and
    state the same one. Unlabelled beats mislabelled — the muxer's
    demotable() guard can only clear a wrong `forced` on discs whose
    authoring uses forced_on_flag, and half the measured discs never
    set it.

Measured over 44 disc images, 9 change and every cross-title label
conflict goes away: 171 forced flags that contradicted a sibling
playlist are cleared, 61 correct ones are recovered on playlists that
had been missing them, 5 subtitle qualifiers and 1 audio purpose bound
against a contradicting language are dropped. No title gains a label it
did not have.

Known residual: on one disc the featurette playlists keep two forced
flags (down from nine) where a shifted list happens to coincide on
language. Ruling those out needs the list's provenance — which slots
are the vendor's and which were merged in by the MPLS gap-fill — and a
whole-sequence gate without it costs correct flags on discs whose
vendor slots are interleaved with gap-filled ones.
2026-08-02 17:50:20 -07:00
Matthew Jackson 5f0cd2e3fe Stop naming specific commercial discs in the AACS and codec comments
The same scrub as the previous commit, over the files it did not reach:
the variant-MKB layout notes, the 2.1 segment index observations, the
PPS-revert regressions and the playlist-twin tiebreak.

Measurements keep their numbers — "a v70 `0x2d` body = 46_100*2 + 16"
is the useful part, and the title it came from never was.
2026-08-02 17:10:43 -07:00
Matthew Jackson 308063196a Merge branch 'fix/pgs-forced-probe-sampling' into dev 2026-08-02 17:07:47 -07:00
Matthew Jackson b14497dfba Do not call a track forced off one display set of a sample
A forced verdict is an absence claim, and a sampled run sees a fraction
of a track. Measured on real discs: tracks exist that flag about a
quarter of their display sets and leave the rest unflagged, so a sample
that catches one flagged set and nothing else promotes a full dialogue
track to forced -- which players then force on screen. A sampled run now
needs two display sets; a run that read every extent end to end has no
unread gap and may still promote off one.
2026-08-02 17:05:28 -07:00
Matthew Jackson f6e167ff2d Stop naming specific commercial discs in comments and tests
Fifteen references across six files named the discs a defect was first
seen on. The parser leak found earlier was not an isolated slip — the
same habit runs through the mux comments, the changelog and the AACS
content verdict, where a title name was standing in for the shape of
the problem.

Every one is replaced with the property that actually mattered: a
multi-clip title, a UHD Dolby Vision profile 7 dual-layer stream, a
disc carrying an authored-bad TS packet. The comments are more useful
for it — the reader needs to recognise the shape on a disc they have,
not the one we happened to have.

`SEG_MainFeature` stays: the parser matches on that literal, so it is
a format token rather than a title.
2026-08-02 17:00:10 -07:00
Matthew Jackson f982ce1046 Halve the seeks: eight 32 MiB windows, not sixteen 16 MiB ones
Measured on a real title: the same 256 MiB budget cut into sixteen
windows per extent took 41.8s against the head-first read's 6.6s, because
every jump collapsed the source's read batching into three-sector calls.
Expected observations depend on total bytes read, not on how finely they
are cut, and measured subtitle density (a display set every ~30-50 MB of
clip) makes a 32 MiB window about even money on its own.
2026-08-02 16:55:55 -07:00
Matthew Jackson f21cbdef80 End a label section where the next one's stream list begins
The pixelogic walk finds the feature playlist's section by name and ends
it at the next `SEG_`/`SF_`/`FPL_` marker. Those markers are section
NAMES, and a project's trailing sections — the per-language notice,
disclaimer and dub-credit cards — carry none. On 8 of the 11
affected-format discs in the corpus the feature playlist is the last
NAMED section in the blob, so the terminator never fires and the walk
consumes the whole tail of the file as more of the feature's stream
list.

The card names are `{lang3}_{card}`, which passes `is_stream_token`, so
each one advances an STN counter, and a card whose name collides with a
catalogued component emits a label outright. Measured on the worst disc:
95 entries past the end of a 9-audio/21-PG list, five phantom audio
labels at STN 10-14 from `*_AC` notice cards (`AC` reads as the AC-3
codec), and 94 uncatalogued-component occurrences — which also took the
parse from High to Medium confidence and fired the vocabulary-gap
warning on four components that are deliberately not catalogued. A
second disc fabricated one subtitle label from a token in a following
playlist section named `FP_SingAlong`, which `FPL_` does not match.

What every section has, named or not, is a stream list that opens with
its video slots. So a `Video Stream N` entry repeating one this section
already listed is the first entry of the NEXT section, and ends this
one. Distinct video entries are kept, since a section may legitimately
list a secondary video stream; the memo of them is bounded at the BD STN
table's ceiling so disc bytes cannot grow it.

Replaying all 11 blobs through `assign_labels` before and after: the two
discs above lose exactly their phantom labels (11→6 and 5→4), the other
nine are byte-identical.

One residue is pinned rather than papered over: a card's name precedes
its own section's video slot, so a forward-only walk can still count the
FIRST card after the last real slot. It sits at the tail of a list
nothing follows in, so it can renumber nothing — at worst it costs a
parse its High confidence.

No other parser in src/labels/ walks a flat entry sequence with a
terminator set; the rest scope each stream to a structural range or read
its number off the entry itself. paramount and criterion gain immunity
pins for the boundary property specifically: a stream list cannot run
into the next element's, and a missing element boundary shortens the
list rather than extending it.
2026-08-02 16:55:08 -07:00
Matthew Jackson 7ad5cdaf41 Apply the shape test to a mixed track too
A track that flags some of its own display sets and not others proves the
authoring house makes the distinction, so it needs no sibling to
corroborate the flag being in use -- but it still has to look like a full
dialogue track before a forced label is cleared. A small track with a
couple of flagged signs is a forced track, and demoting it is the mistake
the shape test exists to prevent.
2026-08-02 16:44:10 -07:00
Matthew Jackson af64089978 Take a lone sample window from the middle of its extent
A title cut into 50-odd clips gives each extent one window's worth of
budget. At the extent's head, every clip is sampled at the same relative
position and the first clip's window lands on the opening of the feature
-- the one stretch with no subtitles in it.
2026-08-02 16:31:39 -07:00
Matthew Jackson a2e6debac4 Record the label numbering and vocabulary fixes in the changelog 2026-08-02 16:28:02 -07:00
Matthew Jackson fd1918ef08 Catalogue the DUB forced-narrative marker and report vocabulary gaps once
Two halves of the same subtitle-labelling bug class. The numbering half
landed already: an entry the parser could not parse still occupies an STN
slot, so skipping it shifted every later label onto the wrong stream. This
is the other half — an entry the parser counts correctly but cannot
INTERPRET.

`DUB` names the forced-narrative subtitle track authored to accompany a
language's dubbed audio presentation: the signs and on-screen-text pass a
viewer still needs once the dialogue itself is dubbed. It is the same
editorial class as `*_TXT_FOR_`, spelled differently by some authoring
runs. Uncatalogued, it matched no component arm, so the token signalled
neither audio nor subtitle and the whole stream record was dropped at the
domain guard — a genuine forced track left with no forced qualifier even
after the numbering was right.

Evidence, from two independent discs in the corpus: the token appears only
inside the PG list, embedded in an otherwise contiguous run of
`{lang}[_{region}]_TXT_FOR_` siblings — one forced-narrative slot per
localized language — and takes exactly the slot where that language's
forced entry belongs. Both discs also carry that language's FULL subtitle
track as a separate, separately-labelled slot, so the DUB entry is not it.
The stream it lands on is a sparse PG track, the signature of a forced
pass rather than full dialogue. Deliberately not added to vocab::qualifier:
that maps free-form English label text, where a bare "dub" means dubbed
AUDIO. The forced-subtitle reading is specific to this token grammar.

The corpus sweep that found it also produced four components that are
deliberately NOT catalogued. They are per-language notice and disclaimer
clip names that merely collide with the `{lang3}_{component}` token shape;
each occurrence sits in a one-video-stream section next to the disclaimer
entry it names. They carry no editorial meaning, and mapping them would
attach a qualifier to a stream on the strength of a filename.

Which is also why an unmapped component now reports once per parse rather
than once per occurrence. It was a debug line nobody reads, and that is how
this gap survived to a user complaint; but a per-occurrence warn would bury
the signal under dozens of routine collisions on an ordinary disc. One
bounded, deduplicated line names the distinct components and says plainly
that any forced/SDH/commentary meaning they carry went unapplied. The
backing set is capped and truncates by chars, not bytes — the components
come from untrusted disc bytes, and a byte-offset slice can split a
multi-byte sequence and panic.

Two existing tests encoded the wrong behaviour and are corrected: the STN
numbering test expected the forced run to skip the DUB slot, and the
unclassifiable-slot test used DUB as its example of a token with no
meaning. The latter now uses a genuinely uncatalogued component.
2026-08-02 16:25:04 -07:00
Matthew Jackson 8636497505 Note the bounded display-count over-count on a stalled read retry 2026-08-02 16:18:19 -07:00
Matthew Jackson e78c8e9802 Drain a sampled run's tail and merge repeated extent memos
A window's last PES stays open in the demuxer until the next PUSI, which
under sampling is in a different window or nowhere — so the last display
set of every window was discarded, worst where the sample is thinnest.
Flush the demuxer at the end of each window.

A playlist that lists the same clip twice now merges the second read into
that extent's memo (facts OR'd, display count MAXed, coverage the larger
of the two) instead of overwriting it.
2026-08-02 16:16:20 -07:00
Matthew Jackson d6542eda10 Document the forced-probe redesign in the changelog 2026-08-02 16:09:27 -07:00
Matthew Jackson 08bdd51588 Sample PGS across the title instead of reading its head
The content-based forced-subtitle probe spent its whole 256 MiB budget
on the first sectors of a title. A feature's subtitles begin minutes in,
so the probe read the opening logos, hit the budget, observed no display
set at all and contributed nothing to any verdict — the vendor label was
always the only input.

The forced predicate is asymmetric: one non-forced display set disproves
forced permanently, while proving forced needs the whole track, and
genuine forced tracks are tiny where full tracks are huge. So the same
budget is now SPREAD over each extent in ~16 MiB windows placed on the
AACS unit grid, sized in proportion to the extent, ending at the extent's
end. Cost is unchanged; placement is not.

Also:

  * Per-track early exit. A track that is disproven (and whose label
    needs no correcting) stops asking for budget; an extent that owes
    evidence only for such tracks is skipped outright, and evidence
    already in the cache is never demuxed a second time.

  * Content may now DEMOTE a wrong vendor forced flag, in the probe and
    in the muxer, behind one shared guard: absence of forced_on_flag
    only means something if some other track demonstrably uses it, and
    the track must have the shape of a full dialogue track rather than
    of a forced-narrative one. On a disc where no track sets the flag,
    nothing is demotable.

  * A sampled or budget-cut extent's evidence is memoised with the
    COVERAGE behind it. It used to be filed under the extent's full key
    and replayed to playlists that would have read far more of the clip,
    turning a prefix into an absence claim about the whole extent.
2026-08-02 16:07:02 -07:00
Matthew Jackson 9d19781b8a Number vendor label streams by STN slot, not by parsed entry
Sweep of every parser in src/labels/ for the numbering bug fixed in
pixelogic: a blob lists one entry per STN slot, but the parser advances
its per-kind counter only for entries it can use, so every entry it
skips shifts all later labels onto the wrong stream.

Three parsers were affected; the rest key each label off a number the
blob states outright and are immune.

  * paramount — `aud` / `sub` are the STN-ordered stream lists, and
    `forced_sub` / `*_com1_idx` index those same cells. A cell with an
    empty language was skipped without consuming its slot, so every
    label behind it bound one stream early while the vendor's own
    positional indices still pointed at the raw cell. `stream_number`
    is now the cell's 1-based position. The forced flag is the payload
    here, so the shift lands `forced` on a full-dialogue track.

  * mpls_universal — its counters must agree with the stream list
    `disc::bluray` builds from the same STN entries, since that list is
    what `apply_labels` counts against. They disagreed twice: a
    `coding_type == 0` padding entry was counted here and dropped
    there, and a PG coding_type in an audio STN slot (a layout
    `mpls::parse_stream_entry` has a dedicated arm for) was counted as
    audio here and built as a subtitle there. Both rules now live in
    one `label_type_for`.

  * deluxe — a binding construction whose Language `getstatic` did not
    resolve was skipped outright. It is still an STN slot; it just has
    nothing to label. It now advances the counter, with the list it
    belongs to taken from its CodingType argument or, failing that,
    from what its binding type's resolved siblings showed.

Two paramount tests asserted the renumbering as if it were the spec
(`empty_middle_slot_does_not_inflate_stream_number`,
`audio_stream_numbering_skips_empty_slots`) and are rewritten. Immunity
pins added for ctrm, dbp and criterion so the property cannot rot.

Also scrubs two commercial disc titles from the menu-graphic filename
examples in png_filenames and vocab.
2026-08-02 15:51:09 -07:00
Matthew Jackson 030f7c5f34 Stop naming a specific commercial disc in the label parser
Two comments identified the disc the STN-numbering bug was found on by its
vendor project name. The reproduction does not need it: what matters is the
SHAPE of the token list — placeholder slots, region-only tokens, an
uncatalogued component — not which release happened to exhibit it. Both now
describe the shape.

The remaining `SEG_MainFeature` references stay. That is a vendor section name
the parser matches on at pixelogic.rs:88, not a disc identifier — it is the
format's vocabulary, like `FPL_` or the `eng_MLP_` stream tokens beside it, and
removing it would break the parser.

Also drops the last prohibited citation from the changelog: an mp4:// bullet
said "no ffmpeg". The website changelog page is REGENERATED from this file at
release time, so a scrub of the site alone would have been reverted by the next
release.
2026-08-02 15:30:56 -07:00
Matthew Jackson 4dd0848d62 Number pixelogic subtitle labels by STN slot, not by parsed token
A pixelogic feature section lists one entry per STN slot. Only the
entries that parse were advancing the counters, so every surviving
label was renumbered 1..N and applied to the wrong stream.

Three kinds of entry were being skipped:

  * `PG Stream N` placeholders — the subtitle twin of `Audio Stream N`,
    which was already counted. The old comment claimed the corpus showed
    subtitle tokens align without counting them; on a disc that has both
    placeholders and later editorial tokens they do not.
  * region-only tokens (`fra_CF_`, `spa_LS_`) — REGIONS sets `variant`
    but neither `is_audio` nor `is_subtitle`, so the token is dropped.
  * tokens whose distinguishing component is uncatalogued (`jpn_DUB_`).

On UHD_Crime101_WW_150728 the PG list is 18 slots: five placeholders,
four region-only tokens, and `jpn_DUB_`, with the seven `*_TXT_FOR_`
forced-narrative tokens at STN 11-16 and 18. Counting only the ten
that parse put `eng_SDH_` on STN 1 and the seven forced markers on STN
2-8 — the disc's FULL subtitle tracks. `labels::apply_labels` turns a
Forced qualifier into `SubtitleStream::forced`, which the muxer writes
as Matroska `FlagForced`, so the output offered an "English (forced)"
track carrying 1731 display sets of complete English dialogue while the
real 19-set forced track went unflagged.

Unclassifiable entries carry no stream type, so they advance the list
currently being enumerated; sections run video -> audio -> PG and
`domain` follows the last entry whose type was known.
2026-08-02 15:27:28 -07:00
Matthew Jackson 49d5ca2a67 Record the 1.6.0 fixes in the changelog
The fvi provenance fix, the key-service outage codes, the settings BOM
data-loss fix and the per-track-kind picker rows all landed on dev without a
changelog line. A release that ships undocumented fixes is one nobody can
audit later.
2026-08-02 15:08:27 -07:00
Matthew Jackson f39762e09b Merge branch 'fix/key-service-outage-not-missing-key' into dev 2026-08-02 14:53:23 -07:00
Matthew Jackson 126765ca19 Merge branch 'fix/fvi-source-provenance' into dev 2026-08-02 14:53:23 -07:00
Matthew Jackson bbc07ba437 Report a key source that could not answer as its own failure, not as "no key"
The online key service returned HTTP 502 for about seven hours. Every rip in
that window ended with

    key: online > no entry > NO KEY
    Error: E7022 No key source has a decryption key for this disc (id: 422EB...)

which reads as "this disc is not in the key database". Operators went hunting
for a VUK that was never missing; the correct action was to wait.

E7022 is a claim about the WORLD: every source answered, and none holds a key
for this disc. A source that could not be reached made no such claim -- nothing
at all was learned. `resolve_and_apply_traced` collapsed the two anyway, with
its own comment naming the incident and pointing at the fix.

Three codes for the three different operator actions, each a variant with a
number and no English (the library ships none):

    E7028 KeyServiceUnavailable   unreachable / DNS / timeout / 5xx -> wait
    E7029 KeyServiceUnauthorized  401 or 403                        -> fix token
    E7030 KeyServiceRateLimited   429                               -> back off

None carries a payload: the key-service URL and its resolved address are
operator-confidential and must not ride out in a Display an operator pastes
into a bug report.

`resolve_and_apply_traced` now keeps `Err` and `Ok(empty)` apart. A source that
answered and holds nothing still records `KeyNode::NoEntry` -- that is the one
true "I looked, it is not there". A source that FAILED records an empty path,
and its reason is stamped onto `Disc::aacs_error`, the channel
`ensure_decryptable_keys` already reads for the E7017-vs-E7022 split. The gate
gained three arms alongside `AacsVidUnavailable` and raises the source's own
code.

`KeyOutcome` deliberately gains no variant. It is matched exhaustively by every
front-end's trace renderer (freemkv's `pipe::render_resolution_trace`,
autorip's `keysource::render_resolution_trace`), and this fix must not become a
breaking change across four repos to say something the error code already says
precisely. Dropping the false `NoEntry` node is enough for the trace line.

The three codes join `is_disc_level_no_key`: a service that is down, refusing
the token or throttling is down for every title, so a rip loop must stop rather
than issue N doomed requests -- and on 429, dig the hole deeper.

`FetchOutcome::errored`, documented as unreachable-in-production, now fires for
real: the negative-result cache stops memoising a transient outage.

Red before green: `key_source_failure_is_not_reported_as_a_missing_disc_key`
drives KeySource -> resolve -> aacs_error -> ensure_decryptable twice over the
same disc and asserts the verdicts differ. With the old conflation restored it
fails on the trace node, and with that assertion removed it fails
`left: 7022, right: 7028` at the gate.
2026-08-02 11:53:09 -07:00
Matthew Jackson b97b419f76 Record the source, not the destination, in the FVI header
The `fvi://` arm of `output()` passed the destination `.fvi` path as
`FviSink::create`'s `source_path`, so every index named itself as its
own source. `SourceInfo::default()` supplied the rest, making
`source.medium` always "file" and `source.title` always 0 — three
header members wrong, where FVI_FORMAT.md §6.2 defines `source` as
describing the input.

Beyond the wrong data, it made the output unreproducible: two machines
indexing identical bytes emitted different files purely from where
they wrote them, and a local filesystem path leaked into a shareable
file.

`output()` cannot see the source, so thread the provenance down from
the driver, which can: `mux_stream` derives a `SourceInfo` per
`MuxInput` arm and passes it through `drive_mux` to `output()`. Per the
one-method-per-action rule this is a signature change, not an
`output_with_source()` variant; the parameter is `Option<&SourceInfo>`
so a caller with no provenance declares none rather than back-filling
the destination. `SourceInfo`/`Medium` become public API.

What each arm can honestly reach:

- Session: everything — device path, the caller's title index, the
  title's playlist, the scanned volume id.
- Url: the source URL, its scheme's medium, `title_index`, and the
  playlist off the opened stream's scanned title.
- Iso: the image path and playlist. The title index is not in
  `MuxInput::Iso` (it carries a scanned `DiscTitle`, which has no
  index), so it stays 0.
- Live: medium and playlist. The reader is an opaque
  `Box<dyn SectorSource>` with no path, and again no title index.

Unreachable members are left empty rather than guessed — the sink
already omits the empty ones.
2026-08-02 11:19:54 -07:00
Matthew Jackson eac0040b95 Add the missing direct coverage for parse_stss
Every other sample-table parser (stco, stsc, stts, ctts) had a "count lie"
test proving the declared count is bounded by what the box actually holds,
and stco/stsc had their own arithmetic pinned. parse_stss had neither - no
test in this file ever called it with real entries, only with a too-short
buffer. Added the same two: two distinct entries read from their own
offsets (catching the o = 8 + i*4 arithmetic and the per-entry bounds
check), and a declared count of 3 backed by only 2 real entries (catching
the same "trust the box, not the count" contract the other four parsers
already had).
2026-08-01 16:37:10 -07:00
Matthew Jackson a939b5b2e1 Pin the sample-table parsers' short-buffer safety and byte arithmetic
parse_stco, parse_stsc, parse_stts, parse_ctts and parse_stss all open with
the same "if b.len() < 8" guard before reading count = be32(b, 4), but no
fixture anywhere in this file ever called any of them with a buffer shorter
than 8 bytes - every test builds a complete box. A <-to-== mutant of that
guard only rejects a buffer of EXACTLY 8 bytes and lets everything shorter
fall through to an out-of-bounds be32 read, and nothing was exercising that
fall-through to notice. One test now drives all five through every length
from 0 to 8.

Also: co64's 8-byte offsets were never actually built by any co64 fixture
in this file (only the 32-bit stco path was exercised), so its manual
byte-by-byte u64 assembly was unconstrained the same way parse_elst's was.
And sample_offsets's sidx only advanced correctly by coincidence in every
existing fixture, because none of them placed three or more samples in a
single chunk back to back - the only shape where reusing the wrong sample's
size becomes observable.

Documented the <=8 boundary as equivalent across all five parsers (and the
matching start < end in sample_offsets): the per-entry guard immediately
below always breaks on the first entry at that exact boundary, so both
branches converge on the same empty result. Confirmed by re-running each
mutation against the full suite.
2026-08-01 16:33:17 -07:00
Matthew Jackson 2ccbb9d417 Pin parse_esds_asc's two independent boundary checks
asc_len == 0 and end > b.len() reject for different reasons - a useless
zero-length ASC, and a truncated one - and nothing distinguished the || from
an && that would only reject when both are true simultaneously, or the >
from a < that would reject the common case of an esds with bytes after the
ASC (more child boxes, padding) instead of only a genuine truncation.

Documented the | in read_descriptor_len's accumulator as the same
shift-then-mask equivalent already recorded in audio.rs's BitReader::read:
the shift always vacates exactly the bits the mask fills, so | and ^ can't
disagree.
2026-08-01 16:23:10 -07:00
Matthew Jackson 2d06639963 Pin mdhd_language's length boundary and three unasserted stsd codec paths
mdhd_language's guard was written as "b.len() < off + 2" (reject too short)
but nothing distinguished that from "reject anything not exactly off + 2" -
a buffer one byte longer than the minimum has to keep working, and a buffer
missing the field entirely has to return None rather than read past the end.

parse_stsd's mp4a path had no test at all: the AAC codec_private extraction
depends on both codec == Aac and body.len() >= 28 being true together, and
with no mp4a fixture anywhere in this file neither half of that condition,
nor the boundary itself, was constrained. Also added the same header-length
boundary check parse_elst already had (< 8 vs <= 8 - proved equivalent this
time, since the very next guard on the empty slice catches the <= 8 case
too), and one test walking every recognised audio fourcc (ac-3/ec-3/mp4a/the
four dtsX variants) so a deleted match arm for any of them fails loudly
instead of silently dropping that track.
2026-08-01 16:18:20 -07:00
Matthew Jackson 56921af7c9 Pin find_boxes_capped's cap boundary and its size-field byte offsets
Nothing asserted the scan actually STOPS at cap rather than one match past
it, or that the declared box size is decoded from its own four bytes rather
than an adjacent one - every existing fixture used sizes small enough that
all but the last size byte are zero, so an index slip reading the wrong byte
would read the same zero and go unnoticed.
2026-08-01 16:08:11 -07:00
Matthew Jackson 322381c429 Pin Stream::read's MAX_ALLOC_BYTES boundary and that write always rejects
Stream::read has its own s.size > MAX_ALLOC_BYTES cap, a separate call site
from read_moov's over the same policy - checked they agree (both reject
strictly greater than the cap, exact cap allowed) and they do, so this is not
one of tonight's one-policy-two-copies bugs. But the boundary itself and the
one-byte-over case were unasserted, and Mp4Reader::write returning an error
(mp4:// is read-only) had no test either.

Building Mp4Reader directly in the test (its fields are private but visible
within this module) over the existing FakeBigReader avoids a real 256 MiB
backing file for the boundary case.
2026-08-01 16:04:03 -07:00
Matthew Jackson 309997f09f Pin per-track handler routing, PID arithmetic and the shared sample budget
Nothing asserted that a hdlr other than vide/soun gets dropped rather than
folded into the audio branch, that the per-track PID formulas
(0x1011/0x1100 + track_idx) use the right operator and the right operand,
that a sample-less track still advances track_idx for the next one, or that
the cross-track sample_budget is actually decremented (as opposed to grown or
divided) by each track's real count. All four were reachable with a single
track_idx == 0, which made every existing fixture blind to +/-/* confusion on
these sites - track_idx never moved past 0 in any of them.

Also let audio_trak_missing omit stsz, needed to build a sample-less track for
the track_idx test.
2026-08-01 15:59:53 -07:00
Matthew Jackson dc65517f55 Pin elst byte-offset decoding and a zero-timescale boundary in elst_offset_ticks
parse_elst's existing tests used segment_duration/media_time values that were
almost all zero or 0xFF bytes, so an index slip in the version-1 byte
extraction (reading a neighbouring byte, or one outside the entry entirely)
could return the same value by coincidence and the test wouldn't notice. Added
a fixture with every byte distinct and nonzero so any wrong offset is caught.

elst_offset_ticks's `empty_movie_ticks > 0` guard on the Some(mts) arm looked
like a pure optimisation, but shifting its boundary lets an
empty_movie_ticks == 0 call fall into the division instead of skipping it -
and a zero movie timescale (unreachable through from_reader, which filters it,
but not through this function's own contract) makes that division panic.
Pinned the boundary directly so the function stays safe on its own terms.

Documented nine further mutants as equivalent rather than chasing them:
media_edits/odd_rate and the None-arm's empty_movie_ticks check only gate a
tracing::warn!, never the returned offset, and parse_elst's b.len() < 8 vs
<= 8 boundary computes the same empty Vec either way once available = 0 is
worked through. Confirmed by re-running each mutation against the full test
suite.
2026-08-01 15:54:24 -07:00
Matthew Jackson 20fd5c3f63 Close the MEDIUM mutation gaps across transport, labels and codecs
The remaining triage items after tonight's HIGH fixes: 1,290 lines, almost
all tests. Covers disc/mod.rs's DVD scan path (with real minimal VMG/VTS IFO
fixtures rather than mocks), drive/mod.rs, labels/class_reader.rs and
labels/mod.rs — the two biggest untriaged survivor clusters in the crate —
plus hevc.rs and ps.rs.

One production change, and it is an extraction rather than a behaviour
change: MacScsiTransport::open mapped the shim's negative failure sentinels
to typed errors inline, where nothing could reach it without a real IOKit
FFI call. It is now map_shim_open_error, so the mapping can be pinned. It
matters because collapsing -5 into the DeviceNotFound catch-all turns
"another process holds the drive" into "no such drive", and an operator
chasing the wrong problem is worse than a blunt error.

Gate green on the pinned toolchain including the secrets scanner.
2026-08-01 15:00:01 -07:00
Matthew Jackson 284c19f15a Pin read_moov's box-size boundaries and MAX_ALLOC_BYTES exactly
read_moov's forward-progress guard (box_size < header_len, OR'd with the
EOF check) and the MAX_ALLOC_BYTES cap were only exercised on inputs well
away from their boundaries, so a mutation testing pass found the exact
edges unasserted: a size-8 (header-only) moov, a box that overruns the
file by exactly the amount the OR/AND distinction can see, and a payload
of precisely MAX_ALLOC_BYTES. Added a shared FakeBigReader (lifted out of
an existing test's local struct so a new test can reuse it) to exercise
the MAX_ALLOC_BYTES boundary without a multi-hundred-MiB backing file.
2026-08-01 14:39:39 -07:00
Matthew Jackson 1f1a17d23f Document why audio.rs's bit-packing | mutants are equivalent
Every mutation-testing survivor in this file is a | with ^ flip inside a
bitstream packer: BitReader::read's accumulate step, the push closures in
dac3_box/dec3_box/ddts_box, and the multi-field extractions in parse_eac3
and parse_dts. All nine are the same shape: shift an accumulator left by
exactly the width of the next field, then OR it in, so the two operands
never share a set bit and | and ^ agree on every input. Confirmed by
running cargo-mutants against just these nine mutations after the existing
test suite (which already exercises each function's field values) - all
nine still survive, as expected for a genuinely equivalent mutant. Recorded
the reasoning once at BitReader::read so nobody spends time chasing it
site by site.
2026-08-01 14:24:47 -07:00
Matthew Jackson 1ca47ce406 Document the pack_language and detect_rate equivalent mutants
Three pack_language mutants and two detect_rate boundary mutants survive
mutation testing with no test able to close them, and it's not for lack
of trying: they're equivalent by construction. Recording the proofs next
to the code so nobody re-chases them:

- (b[0] - 0x60) as u16, shifted << 10 then truncated to u16, is congruent
  mod 65536 to (b[0] + 0x60) as u16 shifted the same way, because
  0x60 * 2 * 1024 is an exact multiple of 65536. The same swap on the
  second letter (shifted only << 5) is NOT equivalent, which is why only
  the first letter's mutant survives.
- The two | with ^ mutations that OR the three packed fields together
  are equivalent because the fields (a lowercase letter minus 0x60, so
  1..=26) always fit in 5 bits and never share a set bit once shifted
  into their 0/5/10 positions.
- detect_rate's tolerance and tie-break comparisons only diverge from
  their <= mutants on an exact 0.5 fps distance or an exact tie, and a
  brute-force search over every achievable integer-nanosecond median
  found no case that lands on either boundary bit-exactly.
2026-08-01 14:20:13 -07:00
Matthew Jackson 6889e6a032 Merge branch 'mux-mp4' into dev 2026-08-01 14:06:24 -07:00
Matthew Jackson 794e757889 Assert the MP4 timing arithmetic, and name the faststart slack rule
MP4 track timing was numerically unasserted. Every operator in the
PTS-to-ticks, duration, tkhd_dur and ctts chain could be flipped and the
whole suite stayed green, because no test decoded an output file and checked
a concrete number — the existing tests assert box presence and gross
container shape only. That is the crate's worst failure mode: a title muxes
"successfully" with silently wrong A/V sync or total duration, and nothing
above can tell.

The new tests build tracks with known PTS deltas and compare the emitted
stts, ctts and tkhd.duration against computed values.

Also lifted the faststart slack rule out of the match guard into
faststart_fits(). A leftover hole of 1-7 bytes cannot be expressed as any
ISO-BMFF box, since a box header is 8 bytes, so finish() must fall back to
moov-at-end rather than write a free box that lies about its own size. The
condition now has a name and a test instead of being an unexplained
`g == 0 || g >= 8` inside a pattern guard.
2026-08-01 14:06:16 -07:00
Matthew Jackson ddb36e2c15 Assert the undersized-buffer guard on the chunked read path
Drive::read and read_fua split any request larger than the transport's
transfer limit into chunks and slice the caller's buffer by count * 2048.
The up-front length check is the only thing between an undersized buffer and
a "range end index out of range" panic out of a public API, and the comment
above it records that this was once a live panic.

Every existing read test stays on the single-chunk path, where an undersized
buffer is already tolerated and returns Err(DiscRead), so the guard itself
had no coverage at all and a mutation run flipped its arithmetic freely.

The test drives a mock transport with a small transfer limit and asserts the
two paths agree: an undersized buffer is an error either way, and behaviour
on a caller mistake does not depend on the drive's transfer limit. Confirmed
by hand that both the reported * -> + mutation and a < -> > flip now fail.
2026-08-01 13:56:58 -07:00
Matthew Jackson 530b4a1925 Cite the H.264 spec directly, not a reference implementation
The escape-stripper comments named a third-party decoder as the authority
for the cumulative-zero rule. This repo is public and does not cite other
implementations; the rule is specified in ITU-T H.264 §7.3.1, which is the
citation that belongs here anyway.

No behaviour change — comments only. The scan-secrets gate caught it.
2026-08-01 13:49:47 -07:00
Matthew Jackson 6153690a90 Merge branch 'mux-codec' into dev 2026-08-01 13:47:15 -07:00
Matthew Jackson 65dd9ce9dc Merge branch 'mux-mkv' into dev 2026-08-01 13:45:04 -07:00
Matthew Jackson f72a3d59a3 Assert what the MKV mux and demux actually produce
A mutation run over mux/mkv.rs and mux/mkvstream.rs left 148 survivors.
Reading them turned up no wrong code, but a lot of code whose output
nothing ever looked at. Most of that is on the read side: parse_track
had a dedicated arm for Language, TrackName, FlagForced, Video and
Channels and not one of them was checked, so a re-mux could have lost
the audio language, the subtitle forced flag, every track label, the
resolution and the channel layout with the suite still green. Ten of
the eleven CodecID comparisons were unasserted too — only HEVC was
pinned — so any of them could have been mis-wired and the stream would
have gone to the wrong parser. The round-trip test now writes a real
three-track title through the muxer and reads it back through the
reader, and a separate test walks every registered CodecID.

The BPS statistics tag was the worst of the write side. Its test
asserted `file_bytes.contains("800")`, which a wrong bitrate passes
trivially — 80000 contains "800". Both the tag and the back-patched
Segment duration are now decoded and compared to a computed number, on
a title that declares no duration so the whole max_block_ticks →
seconds → bits chain is exercised. Cue points get the same treatment:
their CueTrack and CueClusterPosition were never read back, on either
the keyframe path or the i16-forced-split path, which are two hand-
written copies of the same three fields.

The rest closes arithmetic that only a bad disc reaches: a zero frame
rate or zero display-aspect denominator (both divisions), a
TimestampScale that does not fit an i64, a cluster timestamp of exactly
i64::MAX, a TrackNumber of 65537 that truncates onto the valid track 1,
and the shortest legal Block at both VINT widths. Two tests separate a
clean end of stream from a device failure: swallowing the second one
truncates the output at a bad sector and reports the rip complete.

Also pinned: only the first video and first audio track may be default
(the de-duplication lives in MkvStream::create and had no test at all),
the activation trigger is the first VIDEO track rather than track 0,
the measured field order reaches the file rather than just the helper
that computes it, and a Blu-ray 3D base/dependent pair builds the merge
instead of shipping two unrelated H.264 tracks.

96 of the 148 mutants verified killed by hand. Of the remainder, most
are equivalent — disjoint-bit `|` that `^` cannot change, delete-arm
mutants whose fallback is the same constant, guards on tracing calls —
and the write_frame branch at 1452 is unreachable: a cluster is always
open by the time it can be entered.
2026-08-01 13:44:08 -07:00
Matthew Jackson efae55fa2f Kill codec mutation survivors and unify H.264's duplicated escape stripper
The mux/codec parsers (startcode, h264, hevc, dts) had 300 surviving
mutants between them, and it turned out to be for the reason you'd
fear: the exp-Golomb readers and the AU-boundary bitstream scanners had
essentially no direct unit coverage, only indirect exercise through
full-frame parse() calls that never touched the actual edge cases.

Direct fixes to test gaps:

- The shared BitReader's read_ue truncation guard (`leading_zeros >
  31`) and skip_start_code's 4-byte-vs-3-byte boundary check had no
  test at their exact boundary. Added tests that hit the boundary
  precisely; a `>=`/`==`/`<=` typo either rejects a legal 31-leading-
  zero code or reads one byte past the buffer.
- H.264's private SpsReader duplicates the same read_bits/read_ue
  shapes with no tests of its own at all (only reached through
  multi-field SPS parsing, several fields deep). Added direct tests.
- HEVC's per-AU trailing-zero strip after the last NAL (no start code
  following) walks `end` down to trim padding; a wrong-direction typo
  there walks off the end of the buffer instead of terminating -
  exactly the "loop must make positive progress on malformed input"
  class. Added a test with a zero-padded trailing NAL.
- HEVC's SEI match guards (`sei_mastering.is_none()` /
  `sei_content_light.is_none()`) implement "first HDR10 value in the
  title wins" - untested, and a naive test using both-messages-per-AU
  can't even exercise the guards because the whole-scan early return
  above them already handles that case. Split into single-message-
  per-AU tests that actually reach the arms.
- parse_mastering_display/parse_content_light_level's length guards
  were `< N` with no boundary test; one-byte-short input now confirmed
  to return None instead of indexing out of bounds.
- DTS's drain_front collapses duplicate offset-0 PTS markers after
  rebasing; untested, and the visible effect (front_pts()) can't tell
  a working collapse from a broken one since it already returns the
  right marker either way - the actual defect is unbounded growth of
  pts_marks over a long recording, so the new test asserts the bound
  directly across repeated drains.
- DTS's dts_core_samples/dts_core_sample_rate header-length guard and
  next_core_boundary's syncword-length guard got exact-boundary tests
  the same way; also caught a nblks `<<`/`>>` direction bug candidate
  in the mutant (confirmed the real code is correct, just untested).

Real bug found and fixed, not just a test gap:

H.264's parse_sps_high_profile_ext re-implemented emulation-prevention
byte stripping inline (a window scan: match `00 00 03` at position i,
advance 3, else advance 1) instead of calling the existing
unescape_ebsp_prefix used by slice-header parsing. On a run of 3+ real
zero bytes ahead of an 0x03 - non-conformant, but this is disc bytes,
not a spec-clean encoder - the two disagreed: unescape_ebsp_prefix's
cumulative zero counter (matching the H.264 reference decode process
and libavcodec's RBSP extractor) treats it as an escape and drops the
0x03; the window scan treats it as real payload and keeps it,
corrupting the SPS bits read after it. Extracted the shared rule into
`unescape_ebsp` (parameterized on output length so both the 16-byte
slice-header prefix and the unbounded SPS case can share it) and
pointed both call sites at the one implementation. Added a regression
test pinning the shared function's behaviour on the input that used to
separate them.

All new tests hand-verified against the actual mutation (operator
flipped or guard replaced by hand, confirmed red, then restored) per
the mutation-testing brief, not just written and trusted.
2026-08-01 13:35:43 -07:00
Matthew Jackson ef934ed308 Close mutation-testing gaps in the TS/PS mux (ts.rs, ps.rs, tsmux.rs)
A 12,330-mutant run left 159 survivors across these three files, all from
missing assertions rather than wrong code — every gap here is a test, no
production logic changed.

Two shapes accounted for most of them:

- Buffer-cap constants (MAX_PES_BUFFER_TOTAL, MAX_PS_BUFFER,
  MAX_BD_PES_PAYLOAD, PES_BUFFER_INIT_CAP) were only ever read by tests
  through their own symbol, so a mutated `*`/`-` in the constant's
  definition changes what the symbol itself evaluates to and every
  self-referential assertion still passes. Pinned each against a literal
  computed independently in the test.

- Several `>`/`==` boundary checks on framing lengths (MPEG-2 pack header,
  system header, BD-TS adaptation field) were only ever exercised with
  slack in the buffer, never at the exact byte the check exists for.
  Added exact-fit cases for the pack header, system header, and
  psi_payload_base's AF-consumes-everything boundary.

Real, higher-value gaps closed along the way:

- ts.rs's per-PID discontinuity_flag and the NULL-TS concealment marker
  both require adaptation_field_length > 0 before trusting the AF flags
  byte; neither branch had a test proving af_len == 0 (no flags byte at
  all, ordinary payload underneath) is left alone.
- header_remaining (PES header spillover across TS packets) only had
  single-continuation-packet coverage, which can't distinguish `-=` from
  `+=`/`*=` because the corrupted value never gets read again. Added a
  case spanning two continuations.
- ps.rs's parse_stream_id_extension (used for HD-DVD 0xFD routing) walks
  nine optional PES-header/extension fields with a `pos +=` each; only
  the PTS/DTS pair had ever been exercised. One test now arms every
  field and checks the walk lands on the right byte.
- find_ps_boundary's `sc + 3 >= len` guard had no test at sc == 0 with a
  bare 3-byte start code, the case a `+` -> `-` mutation turns into a
  debug-mode subtract-overflow panic on ordinary tail-of-buffer input.
- tsmux.rs: an oversized video access unit must go out as a single
  unbounded-length PES; the `is_video || small-enough` guard that
  enforces this had no test with a video frame actually over the
  bounded-PES threshold, so a `||` -> `&&` mutant survived (it would
  silently split a keyframe across several look-alike-independent PES
  units). Also pinned the PES-length and PTS big-endian encodes at
  values above 255 / with bit 29+ set, where a `>>`/`<<` swap first
  becomes observable.

Every test above was verified by hand: applied the exact mutation,
confirmed the test fails (or the specific panic fires), then reverted.

Left unclosed, all confirmed equivalent by hand-tracing rather than
just left alone:
- Every `<<8 | byte` PID/length bit-combine (ts.rs pid/PAT/PMT parsing,
  ps.rs dvd_audio_pid/hddvd_extended_pid/parse_pts): the two halves
  never share a bit, so `|` and `^` produce identical output for every
  input - no test can tell them apart.
- ts.rs's `af_len > 183` check in process_packet: fully subsumed by the
  `payload_start >= TS_PACKET_BYTES` check three lines later for every
  af_len that could trip it.
- ts.rs's out-of-range `pid_index` sentinel (-1 vs 1): unreachable, since
  a TS PID is masked to 13 bits (max 8191) and the table is always sized
  to at least 8192.
- A cluster of "push an empty slice on an exact boundary" mutants in
  tsmux.rs's write_pes_chain (offset < hdr_len, af_bytes stuffing
  guards): the guarded write becomes a length-0 write_all, a no-op
  either way.

Not reached this pass, for lack of a clean seam within the time
available - ps.rs's extract_packets bounded-PES-length exact-fit
checks (lines 278/282/303, the `sc+6>len` / `sc+6+pes_len>len` /
force-flush cap arithmetic). The first two need a scenario where
"proceed vs. wait one more byte" is observable in the packet list, and
the third only shows up at a start-code offset (sc) that survives to
the moment the cap check runs - in this code path sc is always 0 once
an unbounded PES buffer starts accumulating, since nothing before it
ever drains. Didn't find a construction in the time available; flagged
rather than papered over with a self-referential assert.
2026-08-01 13:34:48 -07:00
Matthew Jackson 2c3e372e9b Fail the identity probe when INQUIRY returns a short data phase
DriveId::from_drive issues three data-in commands. The two GET CONFIGURATION
calls both clamp on bytes_transferred, with a comment noting it is
device-reported and untrusted. INQUIRY, three lines above them, discarded it
and decoded bytes 8..43 unconditionally.

The buffer is pre-zeroed, so a drive answering GOOD status with a short or
empty data phase — a USB-SATA bridge mid-wedge does exactly this — produced
blank vendor, product and revision strings and a byte 0 of 0x00. Every
platform enumerator gates on raw_inquiry[0] & 0x1F == the optical peripheral
type, and 0x00 is DIRECT ACCESS, so the drive silently disappeared from the
device list instead of reporting that its identity probe had failed. The
operator sees no drive at all rather than an error.

Anything shorter than the SPC-4 standard 36-byte header is now
E9058 DriveInquiryShort, and the buffer is truncated to what actually
arrived so nothing decodes past it. Exactly 36 bytes is still accepted: the
vendor-specific tail is optional.

This is the same defect as the READ CAPACITY short-transfer bug fixed
earlier today, in the same crate, found the same way.
2026-08-01 12:21:15 -07:00
Matthew Jackson 00abead473 Give extract and analyze one label-parser tie-break, not two
mod.rs had two independent implementations of "pick the winning label
parser". select_result owns the rule — highest confidence wins, and on a tie
the earlier entry in PARSERS wins — and carries a regression test for a past
bug where analyze() picked the LAST equal-confidence parser instead of the
first. extract(), the path that actually ships, re-derived the same rule with
its own inline scan and had no test of its own.

The tie-break is load-bearing rather than incidental: array order encodes a
trust ordering, with the hand-vetted parsers registered ahead of the ones
that detect on any BD-J disc. A later parser winning a tie means the disc
gets labels from a less trusted source, silently.

extract now collects its candidates and calls select_result. Confirmed by
flipping the tie-break to prefer the later entry: the shared test fails,
where before the fix that mutation was invisible to the whole suite.

This is the seventh instance tonight of one policy implemented twice with
only one copy hardened, and the second in this file after the chapter
mark_type filter.
2026-08-01 12:15:48 -07:00
Matthew Jackson f655befb51 Make the FMTS phase gate reachable by tests
An FMTS forensic segment interleaves two variants at the unit level: the
disc carries both halves and we hold the key for exactly one parity.
Decrypting the alternate half with our key produces garbage; leaving it as
ciphertext is correct, because the muxer drops untouched ciphertext cleanly.

The decision lived inline in apply_aacs_map's per-unit closure, where no test
could reach it. A mutation run flipped the subtraction to an addition and the
parity comparison, and every test still passed — so the gate that decides
which half of a forensic segment we decrypt was entirely unasserted.

It is now unit_is_our_phase(). The index arithmetic also saturates and
clamps: both inputs come from the key map, so a unit below its own range
start or a zero unit size means a malformed map, and neither may panic on
debug overflow or a divide by zero inside a library a long-running service
depends on.

Four of the five surviving mutants now fail. The fifth, dividing by the unit
size versus multiplying, is equivalent rather than uncovered: aligned offsets
are exact multiples of the unit size, so the two differ only by a factor of
unit_sectors squared, and unit_sectors is the constant 3 — odd, so parity is
preserved on every reachable input. The proof is recorded in the test instead
of a test that pretends to cover it.
2026-08-01 11:48:39 -07:00
Matthew Jackson a6585df923 Walk past empty extents iteratively instead of recursing
fill_extents skipped an exhausted or zero-sector extent by calling itself,
which costs a stack frame per skipped extent. Nothing filters
sector_count == 0 out of a UDF or MPLS extent list, so a malformed disc
declaring a long run of empty extents recursed once per extent before
reading a single sector. Rust does not guarantee tail-call elimination, so
that overflows the stack — which aborts the process rather than returning
an io::Error, taking a long-running service down with it.

The skip is now a loop. The regression test runs on a 256 KiB stack, where
the recursive version dies and the loop finishes immediately.
2026-08-01 11:05:12 -07:00
Matthew Jackson 823ec82872 Reject a short READ CAPACITY reply, and count only entry marks as chapters
Two cases of the same shape: one policy implemented twice, with only one
copy hardened.

Disc::read_capacity decoded buf[0..4] from READ CAPACITY (10) without
checking that the transport actually delivered four bytes, even though its
comment claims to mirror decode_read_capacity — which has exactly that
check, and documents why. A drive answering GOOD with an empty data phase
leaves the buffer zeroed, so last_lba decodes to 0 and the probe reports a
one-sector disc instead of an error. It now calls the shared decoder rather
than re-deriving it.

collect_chapter_summary filtered chapters on mark_type <= 1, counting the
reserved type 0. PlaylistMark's own doc says filters must test == 1, and
disc/bluray.rs did; the labels path did not, inflating the public
chapter_count and letting a playlist whose only marks are reserved pass the
chapter_count == 0 skip. Both sites now share PlaylistMark::is_chapter_mark
so the copies cannot drift again.

Both fixes were confirmed red before green.
2026-08-01 11:00:49 -07:00
Matthew Jackson 5049e6515f ci: make libfreemkv prove its five dependents still compile
Every job above proves libfreemkv builds; none proved anything built on
it does. That gap bit today one level up — an engine signature change
broke autorip and went unnoticed, because consumer CI only fires on a
push to that consumer, and nobody pushed one.

libfreemkv sits underneath all five dependents, so a break here costs
more than a break anywhere else in the project. It is now the place the
question gets asked, since it is the place the change happened.

cargo check --all-targets only: each dependent has its own suite for its
own behaviour. This answers the narrower question that went unanswered.
2026-07-31 18:40:24 -07:00
Matthew Jackson ba3ce1e6a5 Lint the test code, and fix the 74 findings it had been hiding
Every other repo's CI now runs clippy with --all-targets. libfreemkv,
the crate the other seven build against and the one held up as the
reference workflow, was the last one still linting the library only — so
its ~3,000 tests, by far the largest body of test code in the project,
had never been linted at all. Turning the flag on surfaced 74 findings.

Most were mechanical and applied with clippy --fix. The rest, by hand:

- Four discarded Results in decrypt.rs. css::descramble_region returns a
  Result and four CSS tests threw it away, so a descramble that FAILED
  would have surfaced as a confusing buffer-comparison mismatch instead
  of the actual error. They expect() now.
- A dead `kp` field on the PlantedWalk fixture. The test deliberately
  asserts Kp as the explicit AES-G3(dk, 1) relation from [C] §3.2.4
  rather than against a stored value — its doc comment says so — which
  makes the field not just unused but a trap: the obvious "fix" of
  asserting against it would quietly weaken the test to comparing the
  fixture with itself. Removed.
- Two hand-rolled ICB counters in the HD-DVD fixtures, a needless mut,
  three vec!s that only ever needed arrays, a filter_map whose every arm
  was Some, and a Vec::new()+push chain.
- Doc list indentation in mkv.rs and mp4/read.rs, which was mis-rendering
  in the generated docs.
- A five-[u8; 16]-tuple return type named FourLevelParts.

Three lints are allowed at the specific sites, with reasons, because
they are wrong for this domain: the underscores in the bitstream-header
literals mark BITFIELD boundaries, not digit groups, so regrouping them
uniformly would satisfy the lint by destroying the only thing they
encode; and in three table-validation loops the loop variable is the
domain value under test (a DTS SFREQ code, an AMODE value, a palette
entry number), which is what the assertion messages name.
2026-07-31 15:08:37 -07:00
Matthew Jackson 94f3d106f9 Expose error_code so consumers can read a code instead of parsing one
io_error_code was private, so the predicates built on it (is_halt,
is_skippable_title_stub, is_disc_level_no_key) were the only way to ask
anything about an io::Error's origin. A consumer that needs the code
itself — to report WHY a title failed rather than to branch on one of
three known cases — had no route to it: mux_stream returns an io::Error,
the typed Error is gone by then, and only the E<code> string prefix
survives.

That left every front-end to re-implement the prefix parse by hand,
which is precisely the string-matching 1.5.x spent its time removing.
One parser, exported.

No behaviour change: the function is unchanged and the three predicates
still call it.
2026-07-31 14:46:39 -07:00
Matthew Jackson a24bf3b96d ci: make the branch tip actually buildable
CI checked out one repo, but libfreemkv path-deps ../freemkv-unlock on
the branch tip — release.sh swaps that to a git tag only inside the
TAGGED commit, then restores the path dep on the branch. So the branch
tip has never been buildable in CI by construction, and every green run
we have ever had was a tag build. Windows and Linux were first compiled
at release time, which is the worst moment to discover a build error.

Both repos now check out into subdirectories (actions/checkout refuses a
`path:` outside $GITHUB_WORKSPACE, and ../freemkv-unlock is outside), so
the path dep resolves exactly as it does on a developer's machine. Every
cargo step runs with working-directory: libfreemkv, and rust-cache is
pointed at the same workspace.

The sibling is taken from `dev`. On main/tag builds the Cargo.toml in
that commit carries the git-tag dep instead, so the extra checkout is
simply unused there.

This is what makes the new branch policy's "dev must be green"
achievable rather than aspirational.
2026-07-31 13:27:05 -07:00
Matthew Jackson cc35b7af66 ci: build dev as well as main
Work now lands on dev and CI must be green there; main only moves at
release time, to the tagged commit, so a push to main is the release
validation run rather than day-to-day feedback.

leak-guard already runs on every branch (on: [push, pull_request]) and
release.yml stays tag-triggered, so neither needed a change.
2026-07-31 13:16:56 -07:00
Matthew Jackson 1faf2ce51b fix(css): revert the hard-fail — it made real DVDs unrippable
I broke DVD ripping earlier today and the real-media acceptance gate
caught it on its first full run. Greenland.iso failed with E7013
"Decryption failed"; reverting only this change made it rip clean in 8
seconds. That is a regression I introduced, not a pre-existing defect.

WHAT I GOT WRONG.

Round 9's crypto lens reported that descramble_region "descrambles with
a key it just proved wrong" when the crib check rejects the cached key
and the re-crack also fails. I agreed, and made it Error::DecryptFailed
to match the AACS path, on the reasoning that CSS has no external key
source so a failed crack on a readable sector should never happen.

The premise was wrong. `attack_crib` is a HEURISTIC, not a proof: it
finds a periodic run in the unscrambled header and predicts the run
continues past 0x80. When that prediction does not hold, the crib
reports a mismatch even for a CORRECT key — and the re-crack then fails
BECAUSE the crib was never valid. So crib mismatch plus crack failure is
the signature of a crib false positive, not of a stale key. The cached
key is not proven wrong; it remains the best available evidence, and on
a real DVD it is very probably right. Real discs hit this constantly.

The deeper error was treating "no key" as one thing across schemes. An
AACS unit key either opens a unit or it does not — the Verify-Media-Key
relation decides it, and a wrong key is provable. A CSS title key is
recovered from the data itself by an attack whose success varies sector
by sector, so "the crack failed here" says something about THIS SECTOR's
plaintext, not about the key. Unifying the policy was right for the
schemes that can prove a key wrong. CSS cannot, and I folded it in
anyway.

decrypt_span keeps its shape and the cross-scheme test keeps its two
AACS arms, with CSS now explicitly excluded and the reason stated.

Three tests asserted the wrong behaviour and are corrected, including
one I rewrote earlier today to pin exactly this. Every one of them
passed the whole time the code was broken — because none of them had
ever seen a real disc.

The lesson is the one I kept stating and then did not act on: 3,013 unit
tests, ~400 mutants killed and nine audit rounds did not catch this, and
one acceptance run did. Synthetic media cannot reproduce what a real
disc does.
2026-07-30 22:07:32 -07:00
Matthew Jackson d0ca927a9e test: constrain the AC-3/E-AC-3/DTS header decode and the boxes it emits
All 59 measured survivors in mp4/audio.rs: 50 killed, 9 proven
equivalent, none left unaddressed. No production change — every
extraction reads correct against ETSI TS 102 366 (5.3.2, 5.4.2, Annex
E.1.3, F.4, F.6.1) and TS 102 114 5.3.1.

It was a fixture gap, not a code defect, and a specific one: the
existing fixtures gave several fields the SAME value (fscod=0, bsmod=0,
lfeon=1, acmod=7) and asserted only derived channel counts. Nothing
asserted the emitted dac3/dec3/ddts payload BYTES at all, so the
packer's shifts and masks were entirely unconstrained. A wrong mask
there does not crash — it writes a box declaring the wrong channel
configuration, and a player believes it.

Several kills needed fixtures designed to discriminate rather than
merely exercise:

  the reduced-rate branch needed fscod == 3, which no test in the file
  reached — and `== -> !=` survived on an fscod=0 fixture only because
  the reduced table happens to return 48000 there too

  the DTS LFF mask needed a value where XOR and AND differ in MEANING:
  on the 5.1 fixtures LFF flips 1 -> 2 and BOTH codes mean "LFE present"

  the acmod mix-level skips needed three different acmods, because `^`
  is a no-op unless acmod == 4 exactly

  the ddts bitrate needed 44100/512, which does not divide evenly — at
  48000 the rounding is invisible and `/ -> %` on den/2 survives

The 9 equivalents are one pattern: OR-ing a shifted high part with a
masked low part on disjoint bit lanes, where the mask is on the same
line as the OR. Since cargo-mutants applies one mutation at a time, no
single mutant can break both. Each was applied and observed green.

FILED, not fixed: reserved sample-rate codes are silently guessed as
48 kHz (audio.rs:133, :20/:153) while a reserved DTS AMODE is refused
with a comment explaining why. Same silent-wrong-metadata class,
opposite answer. Refusing would make such a stream unmuxable, which is a
product call.
2026-07-30 21:51:26 -07:00
Matthew Jackson 093c90481b refactor(decrypt): one orchestrator owns the no-key decision
There were TWO top-level decrypt paths: decrypt_sectors_impl for CSS and
clear media, whose AACS arm was a bare `return Err` stub, and a wholly
separate decrypt_sectors_mapped for AACS. Each scheme therefore decided
its own answer to "there is no key for these bytes", and nothing held
them to the same one.

They drifted, in opposite directions, within a single release:

  css::descramble_region descrambled with a key the sector's own crib had
  just proven stale — garbage behind an intact clear header, reported Ok.

  the mapped path returned early for any LBA outside every range, before
  ever asking whether those bytes were ciphertext, so an unkeyable
  encrypted unit passed through and extract counted it as good.

Both were fixed individually earlier today. This removes the shape that
allowed them.

decrypt_span is now the single orchestrator: it owns the loop, the
refusal, and the loss count, and each scheme supplies only what
genuinely differs. apply_aacs_map is a scheme step that reports what it
could not open; it no longer decides what that means. The public
wrappers (decrypt_sectors, _in_content, _mapped) are unchanged in
signature and all funnel through it.

Adding a scheme now means adding an arm here, which means answering the
refusal question. That is the point.

The new test asserts ONE verdict across all three schemes — AACS with no
map, AACS with an encrypted unit outside every range, and CSS whose
re-crack failed — plus that clear media is NOT a refusal. A per-scheme
test cannot hold this: each would keep passing while the two disagreed.
Flipping the AACS arm back to pass-through reds it.

Also removes the last of the tracing-capture scaffolding. Serialising
those captures crate-wide did not fix the 1-in-10 flake, and asserting
the predicates directly made the helper, both capture subscribers and
an unrelated dead OrderSink unused. Deleted rather than left behind.
2026-07-30 21:35:33 -07:00
Matthew Jackson cb8cd21b21 fix(session): delete two dead accessors, make into_drive fallible
drive() and drive_mut() had ZERO callers — not in libfreemkv, freemkv,
autorip, bdemu, keysources or kdb. Deleted rather than converted: dead
public API that panics is not an API worth preserving the shape of.

into_drive() had two callers and now returns Result. The empty-slot
state is reachable through ordinary public use — stage_drive_as_reader
moves the drive into the reader slot, and calling into_drive twice moves
it out — so the panic was not guarding a caller error. identify() was
converted for exactly this reason in this same release; the fix went to
one of four public sinks and the other three were left.

I deferred this on the assumption the blast radius was large. It was
three call sites. Checking beats assuming.

Also fixes a REAL FLAKE in the gate, which is worth more than the above.
resolve_vid_only_bus_key_gate_reports_true_has_volume_id... failed about
one full-suite run in ten while passing every time in isolation. It
installed a capturing tracing subscriber to read back the has_volume_id
field of a warn.

That cannot be made reliable: dispatcher::set_default is THREAD-LOCAL
while tracing's callsite-interest cache is GLOBAL. The original author
knew, and called rebuild_interest_cache() — necessary but not
sufficient. I first serialised every capture in the crate behind one
lock (harness::with_captured_tracing, which also removed the same
hand-rolled dance from three other sites). Still 1-in-10, because the
cache can be re-evaluated against the process-default dispatch rather
than the thread-local one.

So the predicate is now a named function, handshake_has_volume_id, and
the test asserts the VALUE. A boolean does not need a subscriber to
check. The gate's hard-error behaviour keeps its own test.

Measured: 14 consecutive full-suite runs, 2994 passed, 0 failed.

A flaky gate is worse than a missing one — every green after it means
less, and this one had been eroding trust in the whole suite.
2026-07-30 21:18:13 -07:00
Matthew Jackson 1fd612954b test(mux): replace a false-green discontinuity test with the property it named
a_signalled_discontinuity_survives_a_backstop_discard asserted that a
SOURCE-signalled discontinuity on discarded bytes still reaches the AU
that follows. It did not test that. Deleting the disc_marks push, or the
mark-retirement loop inside discard_gap_before, left it passing.

The mechanism: the `discontinuity = true` rode the FIRST over-cap push,
which still has the next AU's delimiter at buf[0] — so it force-flushes
as an over-long AU rather than discarding, and THAT AU consumes the
mark. The assertion's `.find(|x| x.data.contains(&0x22))` then filters
it out, and the flag it reads comes entirely from `pending_gap`, set by
the second push's backstop. Behaviourally identical to the test 40 lines
above it, under a name promising something else.

I wrote it this morning, in the same commit that fixed a different test
for having a fixture that never reached the code it named, while
cataloguing that exact shape. Third instance today of writing the bug I
was hunting.

The two mechanisms cannot be isolated in one fixture — a fragment that
trips the backstop sets pending_gap regardless — so they now get one
test each. The replacement drives disc_marks end to end with no backstop
involved: a flagged fragment that carries a complete AU and is emitted,
not discarded. Nothing else pinned that path. Removing the disc_marks
push reds it.

Found by the round-9 opus escalation over test quality, dispatched
because the sonnet pass over the same 17,000 lines of new test code
returned zero findings.
2026-07-30 20:07:11 -07:00
Matthew Jackson 139abb721b fix(io): a halted fsync is recognisable as a halt, not a hard failure
The three bounded-fsync failures returned bare io::ErrorKind values —
TimedOut, Interrupted, Other/EIO. `is_halt()` matches on
`io_error_code(e) == Some(E_HALTED)`, i.e. the "E<code>" prefix that
`From<Error> for io::Error` mints, and that is documented as the ONLY
recognised shape. A bare ErrorKind carries no prefix.

So cancelling a rip while sync_all / finish was inside the bounded fsync
made `is_halt()` return false, and the CLI reported a clean user cancel
as a hard I/O failure at the end of an otherwise complete mux.

All three were also mutually unclassifiable, which is the same
information-loss the numeric-code scheme exists to prevent: a caller
could not tell "cancelled" from "NFS wedged" from "worker died", and
should not retry the third the way it retries the first. And their
Display text is std English — "timed out", "operation interrupted" —
reaching a user from library code, which this crate does not do.

Now Error::Halted, Error::SyncTimeout (E_SYNC_TIMEOUT 9056) and
Error::SyncWorkerLost (E_SYNC_WORKER_LOST 9057), on both platforms.

E_HALTED also now maps to ErrorKind::Interrupted rather than falling into
the 6000..=6999 InvalidData bucket. A stop is an interruption, not
invalid data. Nothing branched on the old kind — every consumer uses
is_halt() — so this is safe as well as more accurate.

Two things worth recording. I first placed the E_HALTED arm AFTER the
6000..=6999 range arm and wrote a comment claiming it preceded it; match
arms are ordered, so the range won and the comment was simply false. The
test caught it. And the macOS test asserted only that each arm was
non-Ok with a particular ErrorKind — it passed throughout the period the
three were indistinguishable. It now asserts they can be TOLD APART,
which is the property that actually matters.

Found by the round-9 opus escalation over the API contract.
2026-07-30 20:04:08 -07:00
Matthew Jackson 2d9b648486 fix(udf): file_start_lba must skip a leading unrecorded extent
Regression from the round-8 change that started RETAINING ECMA-167
4/14.14.1.1 type-1 (allocated, not recorded) descriptors. Retaining them
is correct — dropping one slides every later extent's data down by the
hole's length, corrupting the file silently. But read_icb_extent still
took extents.first(), so the value it returns can now be a hole.

A type-1 extent's lba is where SPACE is allocated, not where bytes live.
IcbExtent's own doc says exactly that. file_start_lba hands the value
out as "the absolute starting LBA of a file's first data extent", and
ifo.rs uses it as the base for every VTS VOB extent:

    file_start_lba(IFO) + vtstt_vobs + cell.first_sector

So a DVD whose IFO's first descriptor is type-1 reads its entire video
title set from the wrong place on disc. No error anywhere — the reads
succeed, they just land on unrelated sectors. Verified: the test reports
2900 instead of 2040 with the old code.

Same shape as file_extents/extents_abs_at dropping the recorded flag,
and the same root cause: one change taught read_icb_extents about a new
extent type and did not visit the accessors that consume its output.
Three of them; two are still open (task filed).

Found by the round-9 opus escalation over the API contract, dispatched
because the sonnet pass over the same scope returned zero findings.
Seven of its eight items were absences rather than wrong lines — the
class a wrong-line scan structurally cannot see.
2026-07-30 20:00:15 -07:00
Matthew Jackson 08da31883b fix(udf): retry the customary VDS location when the recorded one holds nothing
The Main Volume Descriptor Sequence was selected from the anchor's
declared extent whenever that extent's SHAPE was usable — length >= 16
sectors (ECMA-167 3/10.2.1), non-zero location, no address wrap — and
the customary location was tried only when the shape failed.

Shape is a property of the field, not of what is there. An anchor can
pass all three checks and point at nothing: a mastering tool that wrote
the reserve location, a stale anchor on a rewritten volume, or
deliberate corruption on an untrusted disc. The sweep then finds no
Partition Descriptor, partition_start stays 0, and the volume is
rejected as UdfNotFilesystem — while the real sequence sits unread at
the location the old fixed sweep would have found.

So the branch a DAMAGED disc actually takes had no recovery path, which
is backwards: that is the branch recovery exists for.

Now both are candidates and the fallback is retried on OUTCOME. This is
the same principle the Metadata File Location chain in this same
function already uses — treat the recorded value as a candidate, fall
back when it does not pan out — applied one level up. The sibling was
added in 8e70f2a and the asymmetry has been sitting there since.

A read fault inside a sweep no longer aborts before the next candidate
is tried, but it does not vanish either: it is held and returned if no
candidate yields a partition. "Could not read" stays distinct from "read
fine and the bytes say no", because mux::resolve caches the second for
the whole disc.

Not addressed, and pre-existing: the RESERVE sequence at avdp[24..32]
exists in ECMA-167 3/8.4.2 for exactly a damaged main sequence and is
still not consulted. Worth doing; a bigger change than this one.
2026-07-30 19:49:03 -07:00
Matthew Jackson 871471edcf fix(decrypt): an encrypted unit outside every key range fails, not passes
decrypt_sectors_mapped returned early for any LBA no map range covers,
on the reasoning that unmapped means clear filesystem or nav. "The map
has no key here" and "there is nothing to decrypt here" are different
statements, and only the second makes passing the unit through correct.

On a multi-CPS disc an orphan clip — referenced by no playlist, so in no
title extent and therefore in no range — hits the first and was treated
as the second. The early return fired BEFORE the aacs_unit_encrypted
gate below it, so nothing ever asked whether those bytes were
ciphertext. extract_tree then counted them as bytes_good, dropped the
.partial suffix, set complete = true and exited 0: a scrambled file on
disk with a clean bill of health.

FileResult's own doc already stated the intended contract — "unreadable
sectors AND undecryptable units both land here (extract fails a bad
decrypt loud)". The code did not implement it on this path.

The fix is one line of symmetry. The split-unit branch immediately above
already makes exactly this distinction, checking aacs_unit_seed_encrypted
before refusing. This branch did not, so the same question got two
answers eight lines apart — the duplication shape this release keeps
finding.

Deliberately NOT changed: the decrypt decision on an orphan is still to
refuse rather than guess. Blind trial-decrypt is what the keymap-only
model exists to remove, and extract has no CPS/forensic fetch source.
The defect was never that we declined to key it; it was that declining
looked like success.

The test asserts BOTH directions — a clear out-of-range unit must still
pass through byte-identical, because that is the ordinary whole-disc
read and breaking it would trade one silent defect for a loud one.
2026-07-30 19:44:37 -07:00
Matthew Jackson 0890d84d4d fix(mux): resync-gate drops reach errors(), including after the gap resolves
ResyncGate::dropped is zeroed the moment a keyframe disarms the gate,
and the only EOF warning fires for gates STILL armed. So a mid-title gap
that resolves left no trace anywhere — and most gaps do resolve. A rip
with several concealed gaps reported 0 errors and 0 lost bytes while
whole GOPs had been discarded, which disc.rs's own test comment calls
the ONLY channel through which loss is reported.

This is the other half of 41e0982. Arming the gate after an 8 MiB
backstop discard is right — a picture with dangling references must not
ship — but until the drop is counted that trades silent corruption for
silent loss.

The gate now carries dropped_total alongside dropped: per-run answers
"how expensive was this gap", cumulative answers "what did the caller
lose". errors() sums the gates.

Summed in ONE place rather than counted at the three admit call sites.
Three copies of the same increment is how the mux-flush path ends up
counting and the main path not, or the reverse — the duplication shape
this release has been removing. The gate already knows its own total;
the accessor just has to ask.

Found independently by two round-9 lenses, which is what raised it from
plausible to worth acting on.

Both halves are pinned: removing the dropped_total increment reds the
resync test, and removing the sum from errors() reds the DiscStream one.
The second test asserts the ACCESSOR rather than the gate's counter,
because a test on the counter would have passed throughout the entire
period the defect existed.
2026-07-30 19:40:48 -07:00
Matthew Jackson f1c2b2c158 fix(mp4): refuse a video track with no resolved dimensions
Resolution::pixels() returned (0, 0) for Unknown, and the MP4 sink wrote
it verbatim into tkhd (ISO/IEC 14496-12 8.3.2) and VisualSampleEntry
(12.1.3). Both fields are MANDATORY there, so unlike Matroska — which
omits the optional PixelWidth/PixelHeight elements — MP4 has nothing to
leave out. The result was a structurally complete file that passes every
container check, declares a 0x0 video track, cannot be rendered, and is
written with no error anywhere.

WHY IT WAS POSSIBLE, which is the part worth keeping:

pixels() previously fabricated 1920x1080 for Unknown. That was wrong but
playable, so this sink never needed a guard and the absence of one was
invisible. Changing the sentinel to (0, 0) moved the defect instead of
removing it — a zero PAIR still reads as a usable value, so the sink
stored it and serialised it.

The accessor's doc comment then ENUMERATED the callers it believed were
safe: "the Matroska sink omits the optional elements, the VobSub writer
omits its size: line, and no caller divides by either dimension." Two of
those three are true. MP4 was not on the list because MP4 has no guard
at all, and a prose list cannot enforce itself. mkv.rs's own comment
even states the principle — "the check belongs in the one accessor
rather than in each caller that remembered to write it" — and
labels/mod.rs still carried its own duplicate Unknown test long after
the accessor took that job over.

So: pixels() now returns Option. Not because Option is tidier, but
because every caller genuinely needs a DIFFERENT answer and the compiler
is the only thing that reliably makes them choose one. Matroska and the
metadata sinks take unwrap_or((0, 0)) with the reason stated at each
site; the VobSub path degrades to a palette-only .idx; MP4 fails with
E_MP4_UNKNOWN_RESOLUTION (9055).

Six call sites, not the five my first grep showed — I piped it through
`head` and acted on a truncated list. The compiler caught the sixth.
That is the same mistake as trusting a lens that reported silence.
2026-07-30 19:34:54 -07:00
Matthew Jackson a8e7d46400 fix(css): no provable key is a hard failure, matching AACS
descramble_region descrambled with the key a sector's own crib had just
proven stale, whenever the re-crack from that sector also failed. The
clear header is not scrambled, so it survives intact: the sector still
opens with a valid pack start and passes every structural check the PS
demuxer applies. Only the payload is corrupted — exactly where nothing
looks. Ok(0) dropped, exit 0.

CSS has no external key source. The title key comes only from cracking
the data, so on a READABLE sector "no key" is not a missing input, it is
recovery failing on bytes we can see. That should never happen, and when
it does the answer is not to emit something.

Now Error::DecryptFailed — the same verdict the AACS path already gives
for a unit no held key opens. Both alternatives to failing are bad data
reported as success: descrambled with a rejected key it is garbage
behind a valid header, and passed through untouched it is ciphertext
where plaintext is meant to be.

WHY IT WAS POSSIBLE, which matters more than the fix:

There is no single place that owns "what do we do when there is no key".
decrypt_sectors_impl looks like the central dispatch, but its AACS arm
is a `return Err` stub — AACS decrypts entirely through
decrypt_sectors_mapped, a separate top-level path. So CSS decided its
own policy inside css/, AACS decided in decrypt.rs and mux/resolve.rs,
and nothing held them to the same answer. The asymmetry was not an
oversight; it was structurally permitted.

How a disc decrypts is one process — resolve a key for this data, apply
it, refuse if it cannot be proven. Only the resolve-and-apply step is
scheme-specific. Filed as a task: the policy belongs in one orchestrator
with the schemes supplying only what genuinely differs.

Two tests changed rather than added, both of which pinned the old
behaviour: the unit test asserted the sector was descrambled, and the
integration test asserted the scramble flag was cleared, which is what
descrambling-with-any-key does. Neither established that the result was
CORRECT — the fourth bad-test shape.
2026-07-30 19:10:32 -07:00
Matthew Jackson 94f087d5ab fix(udf): a read fault locating the Metadata File is not a non-UDF disc
Regression I introduced in 8e70f2a earlier today. The Metadata File
Location fix replaced

    read_sector(reader, meta_file_lba, &mut meta_icb)?;

with a candidate loop guarded by `.is_ok()`, which discards the error.
A transient read fault — marginal sector, drive re-read, an ECC recovery
that reports failure once — then falls through to the block-0 fallback,
the File Set Descriptor read there finds the wrong tag, and the volume
comes back as Error::UdfNotFilesystem: the deterministic verdict "this
is not a UDF disc" for a retryable I/O event.

mux::resolve MEMOISES that negative for the whole disc, so one flaky
read silently demotes every remaining title to the base-Unit-Key-only
path — the AACS 2.1 forensic units garble and the demux drops them. The
mux completes with less content and no error, which resolve.rs's own
comment says must never happen.

This file already draws the same distinction twice in prose, at the AVDP
check and the FSD check: read fine but the bytes say no is structural;
could not read is transient. The loop erased it.

Two things I got wrong on the way, both worth recording:

The first guard was `meta_tag == 0` — nothing read at all. That misses
the damaging case, which is the RECORDED candidate faulting while block 0
reads fine and holds some other descriptor. meta_tag is then non-zero and
not 266, and the fault is still laundered into a structural negative. The
guard is now `meta_tag != 266`: any unread candidate leaves the verdict
unproven, so the fault wins.

The first test denied BOTH candidate LBAs, and passed WITHOUT the fix —
the block-0 read fails too, so a read error propagates from further down
either way. A test whose fixture never reaches the changed line. That is
the eleventh bad-test shape this audit catalogued, and I walked straight
into it while fixing a defect found by looking for it. The fixture now
denies only the recorded location, and removing the guard reds it.

Found by the round-9 opus escalation over correctness, dispatched
because the sonnet pass over the same scope returned zero findings.
Silence from a cheap model is not evidence.
2026-07-30 18:57:13 -07:00
Matthew Jackson d706d4b49e fix(labels): make deluxe master-enum selection deterministic
identify_master_enums picks, for each fingerprint, the best candidate
class out of CandidatePool. Its tie-break only prefers an exact ldc
count over an inexact one, so two candidates that are BOTH inexact but
both within LDC_COUNT_TOLERANCE are decided purely by iteration order —
and the pool was a HashMap.

Rust seeds HashMap per instance, so this is not merely unstable across
runs: the new test resolves the SAME jar to both Alpha and Beta within a
single process, across 16 iterations. The same disc could emit different
commentary/SDH/descriptive labels on consecutive rips of unchanged
input, with nothing in the output saying the choice was arbitrary.

BTreeMap fixes it by construction rather than by a sort someone can
forget to keep. The pool is capped at MAX_CANDIDATE_CLASSES, so the
ordering cost is irrelevant.

The test runs the whole identification sixteen times and asserts one
distinct winner. A single run cannot distinguish deterministic from
lucky, and the seed does not change within a process — so repetition is
what makes this a test rather than a hope.

Found by the round-9 labels pass, which was dispatched specifically
because every one of the ten lenses had reported leaving deluxe.rs
unread. 1,159 new lines that nobody had opened.
2026-07-30 18:49:59 -07:00
Matthew Jackson d7bf829b01 fix: align the Linux fsync error with macOS, and clear three stale docs
Round 9 findings, triaged and verified against the pinned tree.

writeback_file: a bounded-fsync WorkerLost returned bare ErrorKind::Other
on Linux where macOS returns EIO. Round 8 fixed the Linux arm to return
Err at all — the right fix — but stopped short of matching the value, so
a consumer distinguishing timeout / halt / lost-worker had nothing to
branch on for the third case on one platform. Now EIO on both.

Three doc comments described the pre-fix behaviour, one of them for
longer than the bug existed:

  linux.rs durable_sync still said "all three fallbacks return Ok(())"
  mod.rs sync_all still said Linux silently swallows fsync failures and
    callers must not treat Ok(()) as a durability barrier
  mod.rs SequentialSink::finish repeated the same caveat

All three now say what the code does: a bounded-fsync failure is an Err
on every platform, so Ok(()) IS a durability barrier. A doc that
describes a fixed bug is worse than no doc — it tells a caller to write
a workaround for something that no longer exists.

au_assembly: discard_gap_before duplicated drop_marks_before's
mark-retirement body verbatim and added one statement. Mine, from
earlier today. It now calls it. Two copies of the same retirement loop
is exactly how the two call sites would drift back together.

clpi: ClpiStream's audio_format / audio_rate / video_format / video_rate
are decoded from untrusted on-disc bytes on every parse and read by
nothing. The identically-named fields consumed in disc/bluray.rs belong
to mpls::StreamEntry, not to this struct — checked, because an earlier
round wrongly called a live function dead. Deleted, along with the seven
test assertions that pinned them; the tests that pin pid, coding_type
and language remain. Also removed a section-header comment orphaned by
the get_extents deletion, describing a fixture that no longer exists.
2026-07-30 18:39:47 -07:00
Matthew Jackson cb5a918daa test: salvage the orphaned labels/disc triage, and extract build_labels
Thirteen agents triaging src/labels and src/disc died on a saturated
machine, leaving 5,836 insertions across 28 files uncommitted in a
worktree. Recovered by 3-way apply onto twelve commits of drift; zero
conflicts. The diff was archived to private-repo first, because a
worktree is not a backup and this one had already nearly been lost.

One production change, and it is the right one: mpls_universal::parse
read every playlist off the disc AND converted the entries to labels in
a single function, so the conversion — stream-type mapping, dedup key,
the dense global counters — could only be reached through a synthetic
UDF image. Extracted to build_labels(&[Playlist]), which unit tests can
drive from already-parsed values. Behaviour-preserving: same iteration
order, same skip-on-error.

Two collisions resolved by hand:

A second mod pass_progress_tests, written independently against the
same survivors as the one committed in 337b57e. Kept mine — it covers
the distinct-counters case and the Progress blanket impl, which theirs
does not — but theirs had three clamp tests mine lacked: good_pct,
bad_pct and pending_pct also clamp an overshoot, and I had only tested
that for work_pct. Merged those in as one test and proved each of the
three clamps load-bearing by removing them individually.

An unused_parens warning in a new fixture.

Method note, recorded because it cost real time: git apply --3way
STAGES its result, so `git diff` reads empty and the tree looks
untouched. I nearly concluded the patch had silently failed. Worse, the
first attempt piped through `head -20`, so `echo exit=$?` reported
head's status rather than git's — the same mistake this audit has
already documented once. Check the real exit status, and check
--cached, not just the working tree.
2026-07-30 16:36:13 -07:00
Matthew Jackson 66a3506df3 test: pin five untrusted-input guards in the AACS 2.1 and CSS paths
Second pass over src/aacs and src/css. No production change; the only
non-test edits are two fixture bytes and one test rename.

Five latent panics on untrusted data, every guard correct and none
tested — so each was free to be deleted:

  variant.rs:224  a 0x04 record not a multiple of 5 indexes p_uv[0..4]
                  off a one-byte tail
  variant.rs:269  a 0x0c record shorter than the 0x04 slot count
                  slices past the cvalue table
  stevenson.rs:177  short sector read -> index 138 into a 129-byte slice
  stevenson.rs:208  a crib longer than the 1920-byte encrypted region
                    -> index 2058 into 2048
  stevenson.rs:272  a header periodic all the way to offset 0 ->
                    subtract with overflow

That last one is reachable from ORDINARY DVD data — constant or padding
bytes are periodic. Verified on HEAD: widening the guard to <= 0x80
passes all 64 css tests unmutated.

media_key_variant_from_kp had only a soft-correction test, so every
step past that early return was unexecuted. The new two-slot fixture
puts the covering slot at index 1, so the uvs[1 + 5*idx] and
cvalues[idx*16] strides stop multiplying by zero.

derive.rs:319 + -> - confirmed killable, as the first pass predicted:
p == 0 makes (p-1)..32 underflow. Every prior fixture used a uv whose
lowest set bit was 4, 10 or 11, so trailing_zeros() was never 0.

One fixture bug caught and fixed rather than papered over: a |= mutant
first SURVIVED because mk[14]'s 0x04 bit happened to be set, making OR
and XOR agree. The byte is now clear and an assert_eq! pins it, so the
fixture cannot drift back into agreeing with the mutation it exists to
catch.

walk_mkb_be24_high_byte_is_honored renamed to
walk_mkb_be24_middle_byte_is_honored. Its 0x00_0110 length exercises
the << 8 term only, which is why << 16 -> >> 16 survived it. The name
was the lie; both framings are worth having, and the comment now points
at the genuine high-byte test at 0x01_0004.

derive.rs 146:32 and 154:30 stay untested, now with a proof rather than
a judgement: bit_pos == -1 requires current_v_mask == 0xFFFF_FFFF, and
calc_v_mask can never return that — its loop condition holds at
!v_mask == 0, so it always shifts at least once. Both branches are
reachable only after the walk has gone non-convergent and is heading
for the bounded exit, where the return value is undefined. Termination
is already pinned.

Equivalents proven by observing green, including six more OR/XOR pairs
on provably disjoint bit fields, and the two KEY_CORRECTION_DATA sites
where the constant is the documented all-zero placeholder so x ^ 0 ==
x | 0. Those become killable only if a real per-licensee KCD is wired
in.

A partial confirmation sweep (138 of 415 mutants before the box
saturated) found 135 caught, one timeout that is itself a detection,
and exactly one survivor — the KEY_CORRECTION_DATA equivalent above.
2026-07-30 16:27:03 -07:00
Matthew Jackson 8e70f2a973 fix(udf): read the Metadata File Location from the partition map
read_filesystem hardcoded the Metadata File's File Entry at block 0 of
the physical partition — 'the metadata file ICB is at physical
partition lba 0'. UDF 2.50 2.2.10 records where it actually lives, as a
partition-relative Uint32 at offset 40 of the Metadata Partition Map.
That field is the only thing on the volume that says where the entry
is; block 0 is merely where authoring tools usually put it.

On a conformant volume that recorded it elsewhere, block 0 holds
something that is not a File Entry, metadata_start falls back to
partition_start, the File Set Descriptor read there carries the wrong
tag, and the volume is rejected as UdfNotFilesystem. Worse, a volume
with a decoy file set at block 0 — as a rewritten or dual-structure
volume can have — does not error at all: it mounts a different
filesystem and reports success.

Verified on HEAD: reverting the lookup reds four tests, e.g. the
metadata partition beginning at 2000 where the map records 33754069.

The recorded location is trusted only when the map's partition type
identifier reads '*UDF Metadata Partition'. A Virtual (2.2.8) or
Sparable (2.2.9) map is ALSO ECMA-167 3/10.7.3 Type 2 and records
unrelated fields at offset 40, so its bytes must never be read as a
location. Deleting that guard reds its own test.

Block 0 stays in the candidate chain, so a volume whose map is absent
or wrong but whose Metadata File does sit there keeps mounting exactly
as before. This is additive, not a behaviour swap.

Also 30 tests and ~55 more mutants across read_icb_extents,
read_file_limited, read_inline_data, the prefetch stubs, parse_dstring
and parse_udf_name. The metadata-partition branch — the branch EVERY
real BD-ROM takes — had no test at all; nothing in the crate built a
two-partition-map volume.

Closes the max_bytes gap flagged earlier: 259 > -> == and > -> < now
die on both the declared-size and the inline-ICB paths.

Equivalents proven by application, notably two guards that read as
protective but are unreachable: pm1_len is a single byte so
440 + pm1_len < 2048 always holds, and ad_offset + l_ad <= 2048 is
enforced upstream so off + ad_size never exceeds the block.

Bit 0 (Existence) stays unread, deliberately. ECMA-167 4/14.4.4 makes
it a display hint, not a statement that the file is absent, and UDF
2.50 2.3.4.2 carries it through as the DOS hidden attribute. For a
ripper the consequences are asymmetric: honouring it can silently drop
a real .m2ts from the title list, ignoring it costs an extra name in a
listing.

Known structural limit, not fixed: read_filesystem takes only the FIRST
extent of the Metadata File, so a fragmented metadata partition would
map every sector past that extent to the wrong place. metadata_start
being a single base LBA is what forbids the fix.
2026-07-30 16:09:35 -07:00
Matthew Jackson fc1f4db3a2 test: constrain the AACS key-map gap fill, the PSI walk, and MP4 field offsets
Third pass over src/mux/. 40 survivors killed, no production change.

resolve.rs — the deleted-statement cluster is now fully constrained.
All 14 deletable statements probed; 9 were already caught, 5 survived:

  c.sort_unstable() in fill_base_key_gaps. Every existing case handed
  it cuts already in LBA order, but IndividualSegment.tbl is a record
  list. Verified on HEAD: deleting the sort passes all 54 resolve
  tests. The mutant lays a base-key fill straight over a forensic
  segment.

  last_idx = idx (FMTS gap fill) and last_idx = hit (multi-CPS cache
  hit). An extent with nothing to sample must inherit its neighbour's
  CPS unit; the mutants fall back to the first unit's key. Exactly the
  shape this file's own comments name — wrong key, no error,
  lost_bytes == 0.

  Both check_halt()? polls in probe_fmts_index_keys. These cannot be
  killed by outcome, since a later poll returns Halted too. The tests
  count reads instead, which is what the don't-hammer-a-struggling-
  drive rule actually says: after a Stop the drive is asked for zero
  content sectors.

The four unresolved += 1 arms each got a test, and deleting each fails
exactly one — one-to-one, so no fixture passes for the wrong reason. A
control test pins that the baseline table resolves, so an expect_err
cannot succeed for an unrelated reason.

ts.rs::scan_streams was never entered. Six killed, two of which return
wrong answers that look right: reading the PAT/PMT CRC as a table entry
invents a stream on PID 546 out of CRC bytes, and dropping the
ES_info_length skip decodes a descriptor as an entry and loses the one
after it. Every existing PMT fixture declares ES_info_length = 0; a
real BD PMT carries a registration descriptor on essentially every
entry. Also ISO/IEC 13818-1 2.4.4.3 program_number == 0 is the network
PID, not a program.

mp4/read.rs — 13. Height read as the width beside it; channelcount;
the 4-byte base-128 descriptor varint (every existing esds fixture uses
a single byte); all three optional ES_Descriptor fields, whose loss is
silent (an AAC track just loses its CodecPrivate); first-vs-last media
edit, which is A/V desync of the difference; and the version-1 mvhd
timescale offset, emitted by any writer whose duration exceeds 32 bits.

dts.rs — 7, from a real cargo-mutants run over the file rather than
guesswork. Including a buffer that IS the syncword, which is the state
a sync split across PES packets lands in the moment its last byte
arrives.

Equivalents proven by application, not argued: the sample_encrypted_units
guard pair is mutually redundant by construction (total*p/9 < total for
p <= 8), so either alone is equivalent and both together are not; the
PMT section_len guard is dead code where its PAT twin panics; three of
the seven EXSS_HEADER_MIN_BYTES arithmetic mutants still sum to 10.
2026-07-30 16:05:37 -07:00
Matthew Jackson 41e0982ec9 fix(mux): a backstop discard is a discontinuity; a stream-start trim is not
drop_marks_before retired discontinuity marks alongside timing marks at
both of its call sites. At stream start that is right. At the
MAX_AU_BUFFER backstop it is not, and the two are now separate.

The backstop fires when 8 MiB accumulate with no AU start code in them
— corrupt or hostile input — and throws the run away. There IS a prior
AU in that case, and whatever emits next definitively does not continue
it: a decoder handed that picture resolves its references against
frames separated from it by megabytes of discarded data. Retiring the
flag meant the resync gate (resync.rs, driven from mux/disc.rs) never
armed, so the broken picture went out looking sound. Silent corruption
is the one class of loss this crate refuses to have.

At stream start the opposite holds. Bytes ahead of the first
access-unit delimiter are the tail of an AU that began before sync, and
there is no prior AU to be discontinuous from. Marking it would arm the
gate at the head of every title and drop its opening GOP. That risk is
why this was a decision rather than a fix, and splitting the call sites
is what avoids paying it.

Recorded as a sticky flag, not an offset mark. A mark placed at the new
base is retired moments later by the pre-sync trim that follows resync
— the gap has to outlive the bytes that caused it. I found that by
writing the test first and watching it fail with the mark approach.

The discard is a discontinuity whether or not the source signalled one,
and a signalled one on discarded bytes still reaches the AU that
follows; both directions are tested.

Note the first over-cap run is NOT a discard: the next AU's delimiter
is still at buf[0], so it force-flushes as an over-long access unit and
loses nothing. Only a run with no opener at all reaches the backstop.
The tests push twice for that reason — the single-push version passes
without the fix.

Swapping either call site for the other fails: reverting the backstop
reds the two gap tests, and arming the gate at stream start reds the
third.
2026-07-30 14:52:39 -07:00
Matthew Jackson 0469845c2d test: drive the AACS 2.1 variant chain to a Media Key, and pin AES-G3
163 of 322 surviving mutants across src/aacs and src/css. No production
line changed — every function read correct; the finding was always an
absent test.

Two structural holes, both verified against HEAD before landing.

variant.rs had no test that ever produced a Media Key. Every terminal
assertion in the module was an Err classification — NotVariantMkb,
SoftCorrectionRequired, OnlineChallengeRequired. So the entire 2.1
success path (VARIANTS lookup, VKD selection, Kpnew, the final unwrap,
the verify gate) was pinned by nothing, and that path produces the
Media Key that becomes the VUK that decrypts every byte of a 2.1 disc.
Built the first complete planted variant MKB: the VARIANTS entry is
chosen as Kvn ^ 1 so the real VKD sits behind a decoy at table index 1,
making the lookup load-bearing rather than incidentally correct. That
one fixture kills 23 operator mutants across three functions.

aesg3 — the subset-difference tree node function — was in the survivor
list as replaceable by [0; 16], meaning every device key in the crate
would derive the same Processing Key. It is caught today only as a side
effect of a negative test added after the mutation run; nothing asserted
the relation itself. Pinned now via the spec relation ([C] 3.2.2) using
the FORWARD primitive, with s0 transcribed independently rather than
read back from AESG3_SEED, so the test cannot agree with a mutated
constant.

Same shape in derive.rs: plant_mkb was one slot with zero descent, so
slot indexing was the identity permutation and the ancestor-descent
branch never ran — which is why 39 of recover_dk_position's mutants
survived. Added a 3-slot fixture keyed at index 2 and a four-level
descent fixture whose expected Processing Key is written out as an
explicit aesg3 chain rather than computed by calc_pk_from_dk; a fixture
built by the function under test moves with its own mutations.

Two latent panics on untrusted input now have tests: a 0x05 cvalue
table shorter than the 0x04 slot index, and a drive declaring more
payload than the 32772-byte response buffer holds.

23 equivalents claimed with reasoning, and confirmed empirically where
possible — all eight css/lfsr mutants were run and exactly the seven
disjoint-bit-lane ones survived.

Explicitly NOT claimed equivalent: derive.rs 146:32 and 154:30 are
reachable, but only on the non-convergent bounded-exit path where the
function's sole contract is termination. A test there would pin
defined-but-meaningless output.

Noted for the next pass: the pre-existing walk_mkb_be24_high_byte_is_honored
used total length 0x0110, whose high byte is zero — it exercised the
middle byte only, which is why << 16 -> >> 16 survived it. Left in
place; a real one was added at 0x01_0004.
2026-07-30 14:44:35 -07:00
Matthew Jackson d72084e077 refactor(clpi): delete the unused EP-map to sector-extent path
get_extents had no caller anywhere in the ecosystem, and neither did
anything feeding it. Removed: get_extents, resolved_ep_map, full_pts,
full_spn, parse_cpi, EpCoarse, EpFine, the ep_coarse/ep_fine fields,
the unused version field, and the 35 tests that exercised them.

This reverses the fix in f2ca0e4, deliberately. That commit corrected a
real truncation — out_time past the last EP entry resolved to
last_ep_spn + 1, dropping everything from the final I-frame to EOF —
and the fix stands in history as the record of what was wrong. But the
defect existed for eight audit rounds precisely because the code had no
caller: nothing exercised it, so nothing noticed. Keeping speculative
infrastructure alive on the strength of a comment saying it is reserved
for a path that does not exist is how that happens again.

ClipInfo keeps source_packet_count (read by disc/bluray.rs) and streams
(the CLPI/MPLS cross-validation in labels/clpi_audit.rs). 1,139 lines
out, gate green in debug and release.
2026-07-30 14:39:22 -07:00
Matthew Jackson f2ca0e48b0 fix(clpi): resolve out_time past the last EP entry to the end of the clip
ClipInfo::get_extents fell back to `last EP SPN + 1` whenever out_time
lay past the last entry-point. EP entries mark I-frames (BD-ROM Part 3,
CPI / EP map) and a clip's final GOP lies after the last one, so a
PlayItem covering a whole clip — whose OUT_time is the presentation end
— always lands in that arm. The extent then stopped one source packet
after the last I-frame.

Measured on a fixture with 200,000 source packets and the last EP at
SPN 131,072: sector_count came back 12,289 where covering the clip
needs 18,750. Everything from the last entry point to EOF is outside
the returned extent.

Scope, stated plainly: get_extents has NO callers anywhere in the
ecosystem today — it is #[allow(dead_code)] and documented as reserved
for the timestamp-range read path. Nothing ships this loss. It is
fixed now because a latent truncation in extent arithmetic is far
cheaper to correct before it has callers than after.

The SPN at-or-after an out-of-range out_time is the end of the clip,
source_packet_count, with .max(last + 1) so a disc that under-declares
its own packet count against its own EP map still yields a sane bound.

Also 174 mutants killed across clpi, mpls, ifo and ebml — the first
time any of these four files has been examined. And ebml's 8-byte VINT
back-patch was duplicated verbatim in end_master and end_master_buf
with its top four payload octets unreachable through either (they need
a 16 MiB..256 TiB buffer); extracted to fixed_width_vint8 and tested
across the full 56-bit payload, no behaviour change.

38 of ebml's 46 survivors are one equivalence cluster: every | in
write_size / read_id / read_size / read_uint_val ORs into disjoint bit
lanes, where ^ is the identical operation. Applied all 38 at once —
green — then spot-checked four individually.
2026-07-30 14:24:57 -07:00
Matthew Jackson b79015f822 test: pin FMTS read_plan unit indexing to an unaligned range start
Five surviving mutants in AacsKeyMap::read_plan, all in the arithmetic
that decides which half of a forensic segment this disc's key opens.

The existing coverage used a forensic range starting exactly on the
extent's first unit. Under that shape several wrong formulas agree with
the right one by arithmetic accident: (lba + range_start) / us and
(lba - range_start) * us both produce the correct kept set.

Real ranges are not shaped like that. A range start comes from a source
packet number — start_spn * 192 through clip_byte_to_lba in
mux/resolve.rs — and 192-byte packets bear no relation to the 3-sector
aligned unit, so range_start % 3 is whatever the disc says.

Getting the parity wrong does not crash. It reads and decrypts the
ALTERNATE variant's half: the units this key does not open decrypt to
garbage, the units it does open are skipped. AACS 2.1 forensic marking
is precisely what makes the two halves differ, so the failure is silent
— a full-length rip carrying the wrong variant.

Two fixtures are needed because no single one kills both: an unaligned
range start with the extent beginning on it inverts the halves under
the + form, and an extent offset one unit-remainder from the range
start inverts them under the * form.

Also pinned the short-tail guard.  is for a remnant
SMALLER than a unit — bytes with no following unit to desync. Widened
to <=, the last whole unit of every extent bypasses the phase gate, so
a forensic segment ending at an extent boundary contributes one
alternate-variant unit to the rip.
2026-07-30 14:18:39 -07:00
Matthew Jackson c7e43b2bc4 test: constrain the DiscStream loss surface and the empty-title guards
Second mutation pass over src/mux/. 26 survivors killed, no production
change. Verified on HEAD before landing: each mutation below passes all
1,237 mux tests unmutated-suite.

The priority item was the honest-loss-reporting surface. Both
DiscStream::errors and DiscStream::lost_bytes could return a constant
with nothing failing — a rip that lost sectors would report zero loss
to the caller. This project has already shipped one defect of that
shape (a total decryption failure reported as an empty title, exit 0).
Driven now through two short-read fills so both land on values that are
neither 0 nor 1 and differ from each other; no constant and no field
swap survives.

MkvStream::finish -> Ok(()) also survived. MkvMuxer::finish has the
zero-frame MkvInvalid guard and two tests cover it, but the Stream
wrapper above it could return Ok unconditionally and bypass the guard
entirely — the empty-title defence was one layer thinner than it looked.

au_assembly: pinned au_opener_from behaviourally to the normative byte
values for all four modes, with negative cases for codes that are
explicitly not openers (MPEG-2 slice 0x01..0xAF, user data 0xB2,
extension 0xB5, sequence end 0xB7 per 13818-2 Table 6-1; VC-1
0x0A/0x0B/0x0C; H.264 SPS/PPS/IDR-slice). au_assembly and codec/ hold
independent copies of these constants; they agree today, and comparing
constants would not catch logic drifting apart, so both sides are now
pinned to the spec instead of to each other.

demux_sink::sanitize: every filename component demux:// writes comes
from disc-controlled text, so the path-separator arm is a traversal
guard. Deleting it now fails, including an end-to-end case where
base = "../evil/Title" must produce exactly one file inside the
chosen directory.

stts_and_ctts_expand renamed to stts_expands_runs_to_per_sample_deltas_in_order
and given runs with distinct deltas AND distinct lengths. Its old name
claimed ctts coverage it never had, which is why the composition-time
chain went unconstrained for eight rounds; the doc comment now points
at the tests that do cover ctts.

Correction to the previous pass: codec/truehd.rs flush -> vec![] IS
equivalent. Applied it, full mux suite green. TrueHD buffers across PES
but parse emits every complete unit immediately, so a residual buffer
at EOF is a truncated access unit and is correctly discarded. The
vec![Default::default()] variants are genuinely different and are
killed.

Deliberately not constrained: mkv::set_opening_capture (diagnostics
behind a process-global tracing check, flaky under the parallel
runner), and the three stdio.rs header paths (StdioStream holds
concrete io::Stdin/Stdout and cannot be driven without a production
refactor to injectable Read/Write).
2026-07-30 14:13:33 -07:00
Matthew Jackson 5b81c33c9a test: constrain MP4 composition timing, MLP substream directory, and codec-private absence
Mutation testing over src/mux/. No production change — 49 survivors
killed, all proven red before green.

The MP4 composition-time chain was entirely unconstrained: VideoTiming::ctts,
build_ctts and parse_ctts could each return a constant and the suite
stayed green. Confirmed on HEAD: build_ctts -> vec![] passes all 1,220
mux tests. A demuxed B-frame title presenting in decode order would
have shipped.

The cause is a test whose name asserts coverage its body does not
deliver — stts_and_ctts_expand builds an stts box and never touches
ctts, and write_then_read_round_trip asserts sample sizes and keyframe
flags but not one PTS. Same shape as the set_speed forwarding finding,
different disguise.

mlp_num_substreams / mlp_substr_header_size: every TrueHD fixture in
the crate uses one substream and no extraword, so both could return a
constant and agree with all of them. These position mlp_parity_ok's
window over the AU header, so a constant mis-windows the parity check
on exactly the multi-substream AUs that carry 7.1 and Atmos.

CodecPrivate absent vs empty: mkv.rs writes Some(bytes) verbatim and
omits the element on None (RFC 9559 5.1.4.1.24), so a zero-length Some
emits a track header asserting the config IS empty. Four parsers could
return Some(vec![]) before any frame.

Also: mandatory ISO/IEC 14496-12 boxes (tkhd, vmhd, smhd, dinf, mdhd)
could each build empty; HEVC num_extra_slice_header_bits (H.265 7.3.2.3)
was never non-zero in any fixture, so the slice-type offset skip was
unexercised; chapter names from the disc go straight into
<ChapterString> and the & escape must run first; a stray 0x47 in a
payload must not latch a TS resync.

Documented as equivalent rather than killed: CodecParser::flush and the
three parser flush bodies that differ from the mutant only by a tracing
call, and DropTally::log_summary.
2026-07-30 13:39:02 -07:00
Matthew Jackson 3ccee19101 fix(udf): skip deleted File Identifier Descriptors
read_directory decoded file-characteristics bit 1 (Directory) and bit 3
(Parent) but never bit 2 (Deleted) — ECMA-167 4/14.4.4. It followed the
ICB of a descriptor naming a file that no longer exists.

4/14.4.3 permits a deleted FID's ICB field to specify an extent of
length zero, so it need not point at a File Entry at all. Following it
reads whatever descriptor occupies that metadata LBA:

- deleted DIRECTORY FID: recursion lands on the File Set Descriptor
  (tag 256), hits the not-a-File-Entry arm and returns Err. One stale
  descriptor in one directory fails enumeration of the entire volume.
- deleted FILE FID: read_file_size returns Ok(0) for a non-File-Entry
  tag, so a deleted name is reported as a real zero-byte file.

Both directions of the same shape at once: a recoverable condition
becoming a hard failure, and a non-existent entry becoming a plausible
success.

Bit 0 (Existence) is still unread. Skipping hidden files could hide
real content, so it is left alone deliberately rather than folded in.

Found by mutation testing: 48 of the 51 surviving mutants in
read_directory are killed by the 17 tests added here, covering the
short_ad decode byte by byte (ECMA-167 4/14.14.1), the extent-type
mask, the AD bounds guard at the exact sector end, the tag-261 File
Entry directory arm that no test reached at all, the multi-sector
read offset, the FID stride including L_IU, and the nesting cap.

Three survivors are genuine equivalents and are documented as such:
l_fi > 0 vs >= 0 (the emptiness guard below reaches the same state),
and the << 32 / | in the visited-set key (meta_start is constant
across a walk, and the two halves are disjoint).
2026-07-30 13:32:06 -07:00
Matthew Jackson 337b57e9dc test: constrain SectorSource speed forwarding and PassProgress percentages
Mutation testing left both unconstrained.

sector/mod.rs — set_speed on the Box<dyn> and &mut dyn forwarding impls
could be replaced with an empty body and nothing failed. This one hides
better than the read methods because the trait's own default body is
already a no-op, so a forwarder that swallowed the call is
indistinguishable from a source with no speed control. Consequence is a
silently absent value, not a wrong one: the recovery path lowers read
speed through a damaged region, and a swallowed call leaves the drive
at full speed while the caller believes it slowed down. Routed through
a generic S: SectorSource bound, since a direct call on a &mut dyn
receiver auto-derefs to the vtable and never enters the forwarding body.

progress.rs — 42 survivors. All four percentage accessors could return
a constant, read the wrong byte counter, or have their divide-by-zero
guard inverted. Added exact-value tests (25%, not 'some percentage'),
both sides of each guard, the overshoot clamp, and one test setting all
three disc counters to distinct values at once — without it, a swapped
field still passes every single-counter test.

The Progress blanket impl for closures could return a constant true.
That return value is the cancellation signal, so a constant-true body
makes every closure-based consumer uncancellable.

Each mutation applied, observed red, reverted.
2026-07-30 13:26:31 -07:00
Matthew Jackson 0e50b74211 docs: correct info invocation and read timeouts
TROUBLESHOOTING step 2 said `freemkv info`, which needs a source URL;
the drive route is `freemkv info disc://`.

architecture.md quoted 1.5 s / 30 s for the read timeouts. The
constants are READ_TIMEOUT_MS = 10_000 and READ_RECOVERY_TIMEOUT_MS =
60_000 (src/scsi/mod.rs:72,94).
2026-07-30 13:21:04 -07:00
Matthew Jackson c56b1d0b85 Constrain five behaviours that mutation testing showed nothing constrained
Fifteen surviving mutants killed, from the highest-risk class: functions a
mutant could replace wholesale with a constant while all 2,555 tests
passed. None of the code was wrong. In every case a test was absent, which
is why eight rounds of reading never found any of them.

The one that generalises is in sector/mod.rs. Its existing test READS as
covering `read_sectors` on the `&mut dyn SectorSource` forwarding impl —
it takes a `&mut dyn`, calls the method, checks the spy. But the receiver
auto-derefs and dispatches through the vtable straight to the spy, so the
forwarding body is never entered. An earlier round hit this exact trap on
`set_unit_base` and fixed it with a generic helper; the read path kept the
test that looked right. Verified by stubbing the forwarding impl to Ok(0):
the new test fails, the old one passes. That makes a tenth distinct shape
of bad test in this audit, and the mutation list is how to find the rest —
any forwarding-impl method in it has the same problem.

decrypt.rs's two existing gate tests assert only `dropped == 0`, which is
precisely what the `Ok(0)` mutant returns; one asserts nothing else at all.
A wrapper that decrypts nothing therefore looked correct while the caller
muxed scrambled MPEG. Now pinned by descrambling a real CSS sector and
comparing against the plaintext it was built from — not against a
re-derived descramble, which would only assert the code agrees with
itself.

css/mod.rs's `is_scrambled_uncracked` turns out to have no production
callers at all; the enum is matched directly. Its three tests all assert
only the true direction, which is exactly why the `-> true` mutant
survived. It is public API, so a consumer routing on it would, under that
mutant, refuse to rip every clear DVD.

aacs/inf.rs's MKB drive read had no test whatsoever. Now pinned
byte-for-byte across multi-pack concatenation, the single-pack case, a
genuinely empty response, and error propagation — an unreadable MKB must
surface as an error, not as an empty one.

aacs/derive.rs's nine mutants are killed with planted MKBs built by
inverting the AACS relations, so no real key material is involved. The
assertions land on the derived Media Key rather than the intermediate
positions: a recovered position that does not actually walk to the planted
key is no better than None. A fixture-guard test asserts the planted MKB
parses, since an unparseable one would make every `-> None` body look
right.

2570 lib tests, debug and release.
2026-07-30 12:41:28 -07:00
Matthew Jackson 20e3aa26ca Test that the Media Key verifier actually rejects a wrong key
`km_verifies` is the gate deciding whether a candidate Media Key belongs
to the disc. The MK-pool brute force in resolve.rs runs every candidate
through it, so a version that said yes to everything would accept
whichever candidate it tried first and the rip would continue with a wrong
Media Key — wrong VUK, wrong title keys, garbage plaintext, and no error
raised anywhere.

Nothing tested it. Whole-crate mutation testing reported
`replace km_verifies -> bool with true` as SURVIVING: the body could be
replaced with `true` and all 2,556 tests still passed. A verification
routine whose verification was itself unverified, in the most
safety-critical function in the crate.

The implementation is correct — it matches the AACS relation
`AES-D(km, mk_dv)[0..8] == 01 23 45 67 89 AB CD EF`. Only the defence was
missing.

No real key material is required to test it. That relation means a valid
record for any chosen km is just `AES-E(km, <the constant> || anything)`,
so the fixture is self-contained. The test asserts three things: the key
the record was built for verifies; a key differing by ONE BIT does not,
which is the assertion that kills the mutant and is a near-miss rather
than a random key; and an MKB carrying no verify record does not default
to yes, because unverifiable and verified are different answers.

Confirmed by reintroducing the exact reported mutation and watching this
test fail.

This is the first defect found by mutation testing rather than by reading.
Eight audit rounds and a security lens that read this file in full all
missed it, because it is not a wrong line — it is an absent test, and only
an instrument that asks "would anything notice if this were broken?" can
see that.
2026-07-30 12:14:14 -07:00
Matthew Jackson 7e4c22e36c Bound the BD-J label parsers, and stop a crafted disc hanging the scan
Ten defects in code no previous round had ever scoped. `src/labels/`
identifies a disc's studio by parsing jar archives and JVM class files off
untrusted media, so every byte here is attacker-controllable — and 813 of
its lines were executed by no test at all.

The worst is a non-terminating loop. A fallback stream-number scan
advanced with `saturating_add`, and the comment says why: a crafted XML
"must not overflow (panic in debug, wrap-to-0 in release)". Once the
counter pins at u16::MAX and that number is taken, the loop cannot exit.
So a fix for an overflow panic produced an unbounded hang, which is
strictly worse — a panic is observable and catchable, and catch_unwind
cannot interrupt a live loop. Reachable from about 8 MB of XML.

Where the same overflow appears in the deluxe decoder the fix is
checked_add and stop, NOT saturation — twice wrong there, because
saturating would peg every stream past the ceiling at one number and
apply_labels binds on (type, number), silently mislabelling tracks. A
correctness bug wearing the costume of success.

Round 7 capped the ldc-string retention per class; nothing capped the
aggregate, so a 64 MiB jar held that budget for every class at once. Same
defect one level up, which is the shape that keeps recurring in this
directory. Four other amplifications are bounded the same way, each with a
stated headroom and a paired test proving real media passes untouched —
the tightest is 5x on a label length, the loosest 2000x on the stream
numbering space, against BD's 32-per-type STN_table limit.

Two are not caps at all: a quadratic membership scan became a set, and an
attacker-derived length added to a cursor without saturation now cannot
wrap. Nothing is excluded by either.

A `#[cfg(test)]` hand-copy of a shipping parser was the ninth bad test
this audit has found, and the first proven by mutation rather than
inspection: deleting the guard from the REAL function left all 26 tests
green, including the one named for that guard. Pointed at the real
function, the same mutation fails.

Separately, all three failure arms of the bounded fsync returned Ok(()) on
both macOS and Linux, so sync_all reported success for a durability
barrier that never ran. Only macOS was in scope; the Linux twin is fixed
here too, because a platform disagreeing with its sibling about whether a
failed sync is an error is the class that already produced an over-length
SCSI CDB macOS rejected and the other two truncated. Note the behaviour
change: a mux whose final sync times out on a wedged mount now fails
rather than exiting 0.

Three of the caps are proven by wall-clock deadline rather than an
operation count, with 18-80x margin on the passing side. On a heavily
oversubscribed machine those could flake.
2026-07-30 11:30:24 -07:00
Matthew Jackson d628e24eaa Key each FMTS extent from its own CPS unit, and stop a bad ICB tag reading
as an empty directory

On an AACS 2.1 disc the non-forensic gap fill hardcoded pool slot 0 as
"the" base Unit Key, so every content LBA outside a forensic segment was
keyed with CPS unit 1's key even on a disc carrying several CPS units. It
does not fail loudly — it produces garbage plaintext. The gap fill now
resolves each extent's own base key from its ciphertext, sharing the
sampling and slot-picking the multi-CPS path already had rather than
adding a second copy, and memoised in the existing per-disc cache. A disc
with one base key still short-circuits with zero extra reads, which the
existing probe-cost test pins. Forcing the slot back to a constant fails
four tests, so the choice is load-bearing rather than incidental.

A directory ICB whose descriptor tag is neither File Entry nor Extended
File Entry (ECMA-167 4/14.9, 4/14.17) was turned into a successfully-read
EMPTY directory, indistinguishable from a genuinely empty one, while the
same tag on a file ICB was already a hard error. Fifth instance in this
audit of a failure converted into a plausible success value, and the
second in this very function — round 5 fixed a read error becoming a file
size of zero here.

An unrecorded extent (ECMA-167 4/14.14.1.1: allocated but not recorded,
logically zeros that still occupy file space) was dropped entirely rather
than contributing its length, so every later extent landed at the wrong
file offset. Silent corruption, not an error. Extents now carry a recorded
flag and the hole emits zeros without touching the media.

The Volume Descriptor Sequence was swept at hardcoded sectors 32..64 while
the anchor's own Main VDS Extent pointer was parsed into a comment and
ignored; ECMA-167 3/10.2.1 defines that extent by the field, not by
position, so a conformant volume placing it elsewhere failed to mount.

drive_status decoded byte 5 as Media Status without checking the event
header's NEA bit or notification class (MMC-6 §6.7), so a reply carrying
no media event descriptor decoded as "no disc". The drive is untrusted
input here, and this is the works-on-my-drive class.

Two more tests were found asserting the defects they sit next to — one
requiring unrecorded extents to be dropped, one that four drive fixtures
built non-conformant replies the corrected decoder rightly rejects. Both
rewritten. Combined with the DTS one in the previous commit that makes
three tests this round that locked a bug in as intended behaviour, which
is a different and worse failure than the tautological tests found so far:
a tautology fails to catch a regression, these actively defend the defect.

Not fixed, adjacent: extract_one_file streams extents sequentially and
will now READ an unrecorded extent's sectors rather than writing
guaranteed zeros. Offsets are right, and pressed media reads as zeros
there, but it is not zero-guaranteed the way read_file now is; that needs
a recorded flag through PlannedFile.
2026-07-30 11:15:52 -07:00
Matthew Jackson 9bea6ab52b Feed the CSS crack the canonical extent order, and stop Resolution faking 1080p
Seven defects in the code the test suite executes least — 913 lines of
disc/mod.rs alone are run by no test at all, which is why this round scoped
from coverage rather than from what previous rounds said they had read.

Disc::scan_image kept its own copy of the crack's extent ordering and fed
crack_key_outcome largest-cell-first. That is the fifth instance in this
audit of a local reimplementation drifting from the canonical one, and the
cost here is a key that does not descramble the feature: picking by sector
count bypasses the capacity gate and can select a different VTS entirely.
The copy is gone — which title comes from the canonical order the scan
already applied, and the extents are handed over in playback order,
exactly as decrypt_keys_for_title does. Its doc records why the duplicate
existed so it cannot grow back.

Resolution::pixels returned 1920x1080 for Unknown. That is the FOURTH
instance of one trap and the other three were in this same file, two of
them fixed hours earlier — without sweeping for siblings, which is the
whole reason this one survived. It now returns (0, 0), and the sweep was
done properly this time: every remaining Unknown arm across the crate is
honest, and the two ColorSpace sites that look like fabrication are
emitting H.273 code point 2, which is the spec's own "unspecified". Two
callers carried local Unknown-to-zero workarounds — precisely the cost of
making callers responsible for a lie — and one is now redundant.

BD-ROM Part 3 code 0xA2 is the lossy secondary DTS stream, not lossless
Master Audio. A test asserted the wrong mapping as intended behaviour, so
correcting the code failed it; the test is deleted with a note pointing at
its replacement. That is a NEW failure mode for this audit: not a test
that cannot fail, but one that locks the defect in. There is no
DtsExpress variant to map to, so it takes the lossy DTS-HD member and the
approximation is documented.

Also: DiscSession::identify could panic through drive_mut once the public
API allows an absent drive — two siblings were converted in an earlier
round and this one was missed; an extent end that added without saturating
where the rest of the crate saturates; a diag reason string restating the
comparator's sort keys and drifting from them, now derived from them; and
a short read that advanced the offset by the full request, silently
skipping the gap. That last one existed twice, in two reads with the same
shape, now merged so they cannot drift apart.

The short-read policy is a judgement call I could not derive from a spec:
no skip_errors is a hard error, with skip_errors zero-fills and charges
the loss. It deliberately does not retry mid-unit, because resuming inside
an AACS aligned unit would trade a silent gap for a silent decrypt
desync — the worse of the two.
2026-07-30 11:12:55 -07:00
Matthew Jackson 0a88b61421 Stop AudioChannels and SampleRate fabricating a value for Unknown
Three copies of the same two mappings existed. The canonical accessors
returned 6 channels and 48000 Hz for Unknown; a third copy in diag.rs
returned 0. The honest one was the copy.

A plausible wrong answer is worse than an obvious one. Six channels at
48 kHz is indistinguishable from a real 5.1 track, so every caller became
responsible for remembering to check the variant first — and this crate
walked into exactly that: the json:// sink reported a confident 5.1 for
audio whose neighbouring fields said "unknown". That was fixed at the call
site earlier in this audit; this fixes it at the source.

The accessors now return 0, which is what both in-crate call sites already
coerced Unknown to by hand, so their guards are gone and the behaviour is
unchanged. Zero is also obviously wrong if it ever reaches output, where
six is not.

The diag.rs duplicates are deleted rather than corrected — a fourth copy
would have drifted too. Their only caller was a trace line in the same
file, now on the canonical accessors. Their tests moved across and gained
the Unknown case, which is the point: restoring either fabricated value
fails both.

Found by the round-7 correctness agent while fixing the json:// sink; it
flagged the third copy as out of its scope rather than touching it.
2026-07-30 10:09:01 -07:00
Matthew Jackson ef0cf028e4 Add a seeded robustness harness for the untrusted-input parsers
Five parsers that take bytes straight off a disc are now swept with
generated input asserting one property: they return Ok or Err and never
panic. That is this crate's own hard rule, and the class seven rounds of
reading is worst at.

Written in-crate rather than with cargo-fuzz, which needs a nightly
toolchain this project does not use, and without proptest or arbitrary,
because one dev-dependency is a deliberate posture and the parsers take
plain byte slices. What is given up is coverage-guided mutation, which is
the real loss. What is gained is determinism: the same seed replays the
same cases anywhere, so a CI failure reproduces locally verbatim.

Three generators, and the second is the one that matters. Pure random
bytes die at the magic check and exercise the entry guards only; prefixing
valid magic is what reaches the parser body; mutating a mostly-zero record
is what reaches the offset and count arithmetic a hostile image would lie
about.

That claim is MEASURED, not asserted. A harness whose cases all bounce off
the entry guards is the fuzzing equivalent of a test that cannot fail, so
one test counts how many generated cases parse to completion: 15,606 of
60,000, about 26%. If a future change to a guard drops that to zero, the
test fails rather than continuing to report a meaningless pass. Two further
tests pin that the three generators produce different bytes and that a seed
replays identically.

1.2M cases across all five targets found nothing. On this evidence that is
a real negative rather than an empty one.

The first version of this file was itself broken in the way this audit
keeps finding: its two meta-tests set FREEMKV_HARNESS_CASES and raced,
because the test harness runs them in parallel and env mutation is unsound
there. The budget is a parameter now, and the environment is read once at
the call site.

Two crate-internal parsers widened from private to pub(crate) so the
harness can reach them. No public API change.
2026-07-30 09:45:56 -07:00
Matthew Jackson 766fb68e04 Make five tests capable of failing, and stop the presence probe unmounting the disc
The worst of the five was a regression suite that never touched the code
it guarded: nine batch-count tests called `safe_batch_count` and
`buggy_batch_count`, both defined in the test file itself. The u16
truncation they exist to prevent could be reintroduced in
sector/prefetched.rs with every one of them green. They now drive the real
producer through the public API, and reinstating the truncation fails five
of the nine. Worth recording that the symptom has changed since the
original fix: the unit-alignment clamp below floors a zero batch at three
sectors, so the bug is now a twenty-fold throughput cliff rather than the
stall it once was.

The MP4 reserve test's only numeric case was dominated by the floor and
the buffer, so BYTES_PER_SAMPLE could be zeroed without failing it. It now
has a case where the per-sample term dominates. The zero-count guard in
FileSectorSource was likewise unfalsifiable — seek-past-EOF and a
zero-length read both succeed — so the test now observes the file cursor.
The AACS media-key ambiguity guard had no test at all; the pool scan is
extracted so the verifier can be injected, because a genuine two-key
collision needs one ciphertext decrypting under two AES-128 keys to
plaintexts sharing a 64-bit magic, which is a 2^64 search and not a
fixture.

macOS implemented the documented cheap, side-effect-free presence probe by
building a full exclusive transport — which force-unmounts the disc. Linux
and Windows issue one TEST UNIT READY with no unmount; macOS was the
outlier. It now walks the IOKit registry for the media object instead.

The C shim's registry reads assumed CoreFoundation types the registry does
not guarantee, so a driver publishing a CFNumber where a CFString was
expected aborted the process from inside public API. Types are checked and
a wrong type treated as absent. The unbounded waitpid on the unmount child
is now a polled deadline, and the last-resort match gained the NULL check
its two siblings already had.

The empty-CDB guard existed only on Linux while a shared helper's comment
claimed all three backends had it. Moved into the helper, so the comment
is now true and macOS and Windows are covered.

One finding was REJECTED with evidence rather than fixed. The TrueHD
buffer-cap test was indeed bogus, but MAX_TRUEHD_BUF turns out to be
unreachable by any input: the parser only retains data when the buffer is
shorter than the declared AU, and that declaration is twelve bits, so the
worst case is 8189 bytes against a 256 KiB cap. An exhaustive sweep over
all 65536 AU headers confirmed it. The fixture now sits at the reachable
ceiling and asserts that instead. The cap itself is left in place as
defence, unreachable by construction, matching how the AC-3 resync guard
was handled earlier in this audit.

Two behaviour changes worth naming: Linux's empty-CDB error becomes
InvalidCdbLength rather than a transport failure, and an unknown device
now reports absent media rather than a not-found error, because the
registry cannot tell an empty drive from a missing one. The latter is a
conflation of the kind this audit has fixed three times; it is recorded
for the next round rather than left silent.
2026-07-30 09:26:56 -07:00
Matthew Jackson 45fbaa8ee2 Stop the live rip path muxing Blu-ray 3D differently from the ISO path
Five defects, four of them the same shape: a local reimplementation of
logic the crate already had, which had drifted from it. Each is now fixed
by calling the canonical version rather than by patching the copy.

DiscStream::new — the live disc:// path — built every parser through the
plain codec lookup and never asked whether a video stream was an MVC
dependent view, though resolve::build_demux_state does. The same 3D disc
therefore muxed correctly from an ISO and incorrectly ripped live. The
open-coded loop is gone; both paths now call build_demux_state.

collect_psi_section reimplemented the continuity-counter gap test and
disagreed with process_packet in the same file: it tolerated neither a
duplicate packet nor an adaptation-field-only packet, which per ISO/IEC
13818-1 §2.4.3.3 does not increment the counter. A spec-legal PMT
continuation was read as desync and the title's stream list came back
empty. Both callers now share one `cc_is_gap`, and a duplicate packet's
payload is no longer appended twice — doing so would have corrupted the
section the check exists to protect.

The json:// sink called the channel-count and sample-rate accessors
unconditionally, and both fabricate a concrete value for Unknown, so it
reported a confident 5.1 at 48 kHz for audio whose format was unknown
while its own neighbouring string fields said "unknown". The keys are now
omitted, matching mkv.rs. This matters more than it did: a sample-rate
ladder fixed earlier in this audit means Unknown now reaches consumers
that used to receive a wrong-but-concrete value.

For an audio:// or sub:// sink the reference video track's output is
filtered out, so its first PTS was never recorded and every delay was
computed against zero — baking a wrong DELAY into the filename. The
reference is now recorded whenever a frame is on the reference track,
independent of whether that track has an output, so a normal title gets a
correct delay; where no reference is ever observed the tag is omitted
rather than guessed.

A third copy of the channel/sample-rate mapping exists in src/diag.rs and
was left alone as outside the confirmed set. It is the same drift shape
and is recorded for the next round.
2026-07-30 09:18:33 -07:00
Matthew Jackson 40ac369fb1 Bound three allocations an untrusted disc can drive without limit
The Program Stream demuxer appended every fed byte and enforced its 4 MiB
cap only inside a branch reached once a start code had been found. Input
containing no start code anywhere therefore hit no cap at all, and since a
whole title is fed through this demuxer, a zero-filled or ciphertext VOB
extent buffered the entire title — up to ~90 GB. When the buffer holds no
start code, only a two-byte `00 00` prefix can begin a PS unit on the next
feed, so that is kept and the rest dropped. The bound is exact rather than
a heuristic: a start code can straddle a feed boundary by at most its
first two bytes, so no real byte is discarded, and a test feeding
`FF FF 00 00` then `01 E0 ...` pins that.

The existing test named for this case fed a real start code first, so the
cap it exercised was the in-PES one. Renamed to say what it covers.

The BD-J label path had a different shape to anything found so far: the
cap is on the COMPRESSED size of a disc file while the allocation scales
with the decompressed size. A `.class` gated only by a path prefix
inflates to the 64 MiB ceiling, yielding ~33M retained strings from `ldc`
operands or ~67M pushes onto a symbolic stack whose depth was unbounded
despite the Code attribute's own `max_stack` being parsed and then
ignored. Bounded both, the stack by `max_stack` itself (JVMS §4.7.3).

The VMG TT_SRPT title count is an untrusted u16 with no de-duplication,
so ~800 KB of crafted IFO re-parsed one PGC 65535 times. Capped at 99, the
DVD-Video maximum, so no conformant disc is clipped.

Every cap carries stated headroom against real media, and each has a test
locking that real media still passes.

I rewrote three of the new assertions before landing them. They compared
the result against the very constant under test — `total <= MAX_TT_SRPT_TITLES`
— which passes vacuously the moment someone raises the constant, the most
likely future regression and the seventh instance of this tautology shape
in this audit. They now assert literals derived from the spec.

The TT_SRPT fixture also had to change: with 65535 identical entries the
de-duplication collapsed them on its own and the cap was never what
bounded the result, so the test passed with the cap removed entirely.
Distinct entries defeat dedup and leave the cap as the only guard;
de-duplication now has its own fixture. Verified by raising each of the
three constants and confirming all three tests fail.
2026-07-30 09:17:16 -07:00
Matthew Jackson f5e65cf2da Remove emulation-prevention bytes before reading the H.264 slice header
The bytes after a NAL header are EBSP, not RBSP: ISO/IEC 14496-10 §7.4.1
has the encoder insert 0x03 after any 0x00 0x00, and §7.3.1 removes it
before parsing. The measured-picture-type parse read the raw NAL instead,
on the stated reasoning that slice_type is too early for an escape to
intervene.

That holds only up to a point. first_mb_in_slice is ue(v), so a value of
65535 or more needs sixteen leading zero bits and opens the payload with
0x00 0x00, which an encoder must then escape. A UHD frame is ~32,400
macroblocks, so a conforming Blu-ray never reaches it — but 8K does, and
the disc is untrusted input. Such a stream decoded slice_type against a
byte the encoder had inserted and reported the wrong picture type: a wrong
result rather than an error, which is the class this lens exists for.

The prefix is un-escaped into a 16-octet buffer rather than the whole NAL:
the two ue(v) fields are at most 32 bits each, so nothing longer can be
needed, and it keeps a per-frame allocation proportional to the frame off
the path.

The test pins both directions. It asserts the un-escaped prefix decodes to
first_mb_in_slice = 65535 and slice_type = 2, AND that the raw EBSP does
NOT — without that second assertion the test would pass whether or not the
fix were present, which is the failure mode this audit has now found four
times. The bit string was derived independently rather than by hand: my
first attempt at the fixture was wrong by one nibble and the test caught
it.

Also covers the cases that must NOT be unescaped: a 0x03 not preceded by
00 00 is ordinary payload, and 00 00 03 03 keeps its second 0x03 because
the escape resets the zero run.
2026-07-30 08:49:51 -07:00
Matthew Jackson ade13a35de Log the OS error when a SCSI command fails on Windows and macOS
Both backends discarded the platform's own error code on the execute hot
path — the one every READ(10) of a rip goes through — and collapsed every
cause to the same status-0xFF transport failure.

On Windows, open() and reset() in the same file both capture the Win32
error; execute() did not. That left ERROR_INVALID_PARAMETER (a struct
layout regression, the exact class this file's SDK-layout tests exist to
catch), ERROR_ACCESS_DENIED and ERROR_GEN_FAILURE (a genuinely wedged
drive) indistinguishable, with nothing in the log to tell a code bug from
a hardware one.

On macOS the same, and worse: the file had no tracing calls at all, where
the Linux and Windows backends both log their execute failures. Its open()
carefully decodes the shim's sentinel into typed variants instead of
flattening them, but execute() threw the IOKit return away — so another
process taking exclusive access mid-rip and a real hardware wedge produced
identical, empty diagnostics.

Logged rather than added to the error type: the typed variant is public
API, and the recovery classification is deliberately the same for all of
these. What was missing is the breadcrumb, not the distinction.

Two further findings from the same sweep were rejected. Windows reset()
always returning Ok(()) and macOS ignoring timeout_ms are both already
documented in the code as deliberate, and the reporter flagged them for
completeness rather than as defects.

Neither fix has a test: reaching either branch needs a failing ioctl or a
failing IOKit call, and both files are compiled only on their own platform.
2026-07-30 08:42:02 -07:00
Matthew Jackson 816a41a01f Turn a release-only slice panic into an error, and stop calling 32 kHz 48 kHz
Four round-6 findings.

FileSectorSource::read_sectors guarded its output buffer with a
debug_assert, which is compiled out in release — so an undersized buffer
panicked with 'range end index out of range' instead of returning an
error, out of a public SectorSource impl where the length is caller input.
Drive::read_fua already carries this exact guard, with a comment recording
the same panic being fixed there, and PrefetchedSectorSource has a
regression test for the same case; this impl had been given neither. The
new test is red in release for precisely the predicted reason: 'range end
index 8192 out of range for slice of length 2049'.

parse_track's sample-rate ladder ended in an unconditional S48, so any
SamplingFrequency below 44100 was recorded as 48 kHz. A 32000 Hz AC-3 or
DTS track is legal and common in broadcast-sourced content, and the wrong
rate then propagated into the reconstructed AudioStream. Anything below
the lowest mapped rate is now Unknown, which is what the crate's canonical
SampleRate::from_hz already returned — the ladder disagreed with it. The
ladder itself stays, because the MKV element is a float and wants
tolerance rather than exact equality.

shim_open_exclusive used the mach port from IOMainPort without checking
the return; on failure the port is left untouched and every IOKit call
below ran against an uninitialised value. shim_list_drives in the same
file does check it.

build.rs treated cc and ar as successful if the process merely SPAWNED, so
a genuine compile error in the macOS C shim produced no object file and
surfaced later as an unexplained link failure against a missing symbol.
The shim is macOS-only and is neither linted nor compiled on the other two
platforms, so a mistake in it has exactly one chance to be noticed.

The last two have no test: one needs IOMainPort to fail, the other needs a
deliberately broken C shim, and neither is reachable from the test
harness. Both mirror a correct sibling in the same file, which is the
evidence available.
2026-07-29 22:56:33 -07:00
Matthew Jackson ae33563587 Fix five clippy errors that only appear on the target CI lints
CI's clippy job runs on ubuntu-latest, so cfg(target_os = "linux") code is
what the gate actually compiles — and none of it is built by clippy on a
Mac. Five `-D warnings` errors were sitting in drive/linux.rs,
scsi/linux.rs, io/writeback/linux.rs and the Linux arm of drive/mod.rs:
four collapsible let-chains and one manual `% n == 0`. CI was red on the
lint job while the local gate reported all green.

Collapsed into let-chains, which the declared toolchain supports, and
folded drive/mod.rs's length precondition into its chain so the body no
longer needs a nested block.

Found because an agent working on the SCSI backends reported the lints in
passing while checking that a Windows-only file compiled. Worth noting how
it stayed hidden: every one of these files is cfg-gated to a platform this
machine is not, so no amount of local gating would have surfaced them. The
companion change to the precommit script closes that hole.
2026-07-29 22:37:47 -07:00
Matthew Jackson 9a88efcc84 Reject an over-length CDB on every transport, splice H.264 param sets in place
Two fixes from round 5.

The Linux and Windows backends truncated a CDB longer than 16 bytes
(`cdb.len().min(16)`) where macOS returned InvalidCdbLength. Under SPC-4 a
command's length is fixed by its opcode group code, so a shortened CDB is
not a shorter form of the same command — it is a DIFFERENT command, and
the drive executes it and answers GOOD with data for a request nobody
made. A silently wrong result on the layer everything else sits on.

Rather than mirror the guard a third time it now lives in scsi::mod as
checked_cdb_len, with all three backends routed through it, so it cannot
drift per platform again. That also makes it testable everywhere: each
platform module is cfg-gated to its own host, so a guard inlined into
linux.rs and windows.rs would have had no test coverage on any single
machine. The shared helper is the only place the behaviour can be
asserted on every platform's CI.

The two existing macOS tests were tautological — they replicated the
guard's logic inline instead of calling it, so they would have passed with
the guard deleted. They now call the real helper.

Separately, the H.264 keyframe parameter-set re-assert grew a
few-hundred-byte prefix buffer to the full access-unit size, copied the
whole frame into it, and dropped the presized buffer: one extra
whole-frame allocation and copy per keyframe. A UHD title is ~200,000
frames of 150-400 KB with a keyframe every second or two, so that is
thousands of avoidable multi-hundred-KB copies per title, each large
enough to go through mmap. It now splices into the reserved headroom in
place. This mirrors the identical fix already made in hevc.rs, which the
H.264 path had drifted from.

Byte-for-byte equivalence is pinned by a test whose expected literals
were captured from the pre-change implementation, and which I confirmed
still passes when the old build-and-copy code is restored. The
no-reallocation claim is measured rather than argued: a counter over 30
bare keyframes, which reports 30 of 30 against the old path and 0 with
the splice.

The reallocation test initially passed even with PARAM_REASSERT_HEADROOM
set to zero, because a small parameter set fits in the presize's
incidental slack — it proved the fixture did not reallocate, not that the
headroom prevented it. Its SPS is now large enough that the constant is
load-bearing, so zeroing it fails the test.

Not verified: no runtime behaviour on Linux or Windows: no drive, no
ioctl. Both files were confirmed to compile for their own targets.
2026-07-29 22:34:22 -07:00
Matthew Jackson 719a9077f3 Stop reporting an uncrackable CSS disc as N empty titles
Same shape as the mkv:// conflation fixed earlier in this round, found by
looking for it deliberately. E7023 carried two conditions with opposite
correct responses: one title on a multi-VTS DVD failing its own re-crack,
where skipping it and finishing the rest is right, and the main feature's
crack failing outright, which is disc-wide and dooms every title
identically. Because both raised the same code and that code is in
is_skippable_title_stub, an uncrackable disc walked all N titles printing
"title skipped, it was empty" and exited 0.

The disc-wide condition gets E7027 CssNoDiscKey, mirroring the AACS-side
E7022 NoDiscKey it is the analogue of, and joins is_disc_level_no_key.
The per-title raise keeps E7023 and stays skippable. Because the engine's
classifier already tests is_disc_level_no_key before the skippable
branch, this reaches the right outcome downstream with no change there:
such a disc now stops on the first title and reports no-key instead of
returning success with nothing written.

Disc::css_error deliberately still stores CssKeyMissing — autorip matches
that variant on the field to pick the CSS rather than AACS message, and
what consumers classify on is the gate's returned verdict, which is the
only thing that changed.

Two neighbouring CSS raises were examined and deliberately left alone:
the no-key branch in the same function is genuinely unreachable via
ensure_decryptable and documented as defensive, and resolve_dvd_title_key
is per-title on both of its call paths.

Verified by removing the new code from is_disc_level_no_key, which fails
both new tests; each pins both directions so neither can silently flip.
Not proven end to end against a real uncrackable disc — none available.
2026-07-29 22:29:51 -07:00
Matthew Jackson de97c5a607 Round 5: reject a degenerate fixed lace, bound the pending buffer by bytes
Five fixes. Three are real defects with regression tests; two are bounds
that were expressible but not expressed.

A fixed-size lace (RFC 9559 §10.3.4) whose body is empty declared n
frames and carried none. The divisibility check passed, because 0 % n is
0, and `chunks` yields nothing on an empty slice whatever width it is
given — so the clamp that existed to avoid chunks(0) returned zero frames
where the Lacing Head said n. The whole lace vanished with no error
raised and the caller saw a clean short block. A zero-size frame cannot
be a valid frame, so it is now malformed.

A disc read failure while fetching a directory entry's ICB became a file
size of zero rather than an error. Zero is indistinguishable from a
genuinely empty file, so an unreadable ICB on a damaged disc silently
changed which titles a caller saw as present — read_directory already
fails hard on its entry-budget guard, so propagating is also what the
surrounding code does. read_file_size still returns Ok(0) for an ICB
whose tag is neither File Entry nor Extended File Entry, which is a real
zero and not a failure.

The pending-frame buffer was capped at 4096 frames, which does not bound
memory: frames are arbitrarily large and a UHD video frame runs to a few
hundred KB, so the existing cap permitted over a gigabyte. Now bounded by
bytes as well, at 64 MiB.

round_up_grain overflowed for inputs within one grain of u64::MAX —
div_ceil then multiply — and the wrapped product is small, turning the
largest possible estimate into a negligible reserve. It saturates, and
the reserve is clamped to what a `free` box's 32-bit size field can
actually hold, since writing a larger one truncated the size and left
mdat beyond a box claiming to be far shorter. No real title comes close;
a 90 GB UHD title estimates a few MiB.

The AC-3 resync guard now advances the PTS cadence like both of its
sibling branches, so the three paths out of that block cannot disagree.
This one is defensive and has NO test: reaching it needs input that both
parses frames and leaves a megabyte of residue, and the parser's own
carry rules drop pre-sync junk and cap a partial frame at 8192 bytes, so
no such input was found. Stated here rather than covered by a test that
would pass either way.

Two findings from this round were rejected on inspection. A reported
panic in the .mpls suffix check does not exist: the `.get(..)` on the
line above returns None off a char boundary and `filter` never runs its
closure, so the byte index is unreachable. A test written for it passed
against the unfixed code, which is what surfaced the error.
2026-07-29 22:28:43 -07:00
Matthew Jackson 424b0fa32e Stop reporting a corrupt mkv:// source as an empty title
E6008 meant two unrelated things: "this title produced no muxable
frames", which is a benign stub worth skipping, and "the source file is
malformed", which is not. Because a single code carried both,
is_skippable_title_stub answered yes to the second one — so feeding a
truncated or corrupt mkv:// input made the engine classify it
SkippableStub, print a notice saying the title was empty, and exit 0.
Silent data loss reported as success.

Split into E9053 MkvSourceInvalid for the read path (25 raise sites
across mkvstream.rs and ebml.rs's read primitives) and E9054
MkvUnencodable for the four write-side sites, which are the encoder
refusing to emit a body at or above the 56-bit VINT limit — an output
limit with no input involved, so calling it a corrupt source would be
wrong in the other direction. E6008 keeps only the zero-frame guard it
was documented to mean.

Kept one code for the whole read path rather than one per raise site:
nothing a consumer does differs between a bad VINT, a non-UTF-8 string
element, a truncated body and a child overrunning its parent. E9052 is
the model for when a carve-out earns its keep — laced blocks name one
specific RFC 9559 §10.3 feature with its own diagnosis.

Also fixed meta_sink.rs raising MkvInvalid for a serde_json encode
failure in the json:// sink, where no MKV is involved at all; it now
matches the identical guard in mux/meta.rs.

Reverting the split at the single code() arm reproduces the old
classification: 22 tests fail, including both new assertions. The
opposite direction is pinned too — dropping E6008 from the predicate
fails the genuine-stub test, which drives the real muxer end to end.
2026-07-29 22:11:53 -07:00
Matthew Jackson 81596e7735 Round 4: fix 26 defects across crypto, resource use and codec paths
Twenty-six confirmed findings from the fourth audit round, landed as one
cluster because they were found by agents working over disjoint file sets.

The one worth calling out is a pair of AACS tests that could not fail.
Both asserted CBC behaviour against a hand-rolled expectation that
happened to be IV-independent, so replacing AACS_IV with sixteen zero
bytes left them passing — they were pinning the code's own arithmetic,
not the published constant. Replaced with a literal witness of the
published IV plus the NIST SP 800-38A F.2.2 CBC-AES128 vector, and
verified the other way round: zeroing AACS_IV now fails three tests.

The rest are allocation and correctness work on hot paths: the Annex-B
writer in demux_sink allocated and freed a whole-frame Vec per frame,
which for a UHD title is ~200,000 allocations over the mmap threshold
plus the page faults to first-touch each one; it now reuses a buffer on
the writer, and still takes the NAL prefix width from the configuration
record rather than assuming four.

Six findings whose real fix lives in a consumer crate are recorded for
re-filing rather than patched here.
2026-07-29 22:09:52 -07:00
Matthew Jackson dce6d7abdf Parse MKV lacing, route by real TrackNumber, honour NAL length size and edit lists
Four conformance defects in the read paths, two of them silent corruption.

**Lacing was ignored entirely.** RFC 9559 §10.2 defines Xiph, EBML and
fixed-size lacing, where one Block carries several frames; the reader took the
Block payload verbatim, so a laced Block became a single "frame" consisting of a
lacing header followed by concatenated frames — garbage to the codec parser, no
error. Audio tracks from other muxers commonly use lacing, so an ordinary
foreign MKV was silently mangled.

All three modes are now parsed: Xiph 255-run sizes including the trailing-zero
rule for exact multiples of 255, EBML unsigned first size plus SIGNED VINT deltas
with the 2^((7*n)-1)-1 bias of §10.3.3, and fixed-size even division, with the
last frame's size deduced from the remainder. Laced timestamps follow §10.3.5:
the first frame takes the Block timestamp and the rest are spaced by the track's
DefaultDuration, else BlockDuration/count, else shared with a warn.

Parsing was chosen over refusing because refusal would leave freemkv unable to
remux common foreign audio at all, and each mode is about fifteen lines.

A malformed lacing header now raises a NEW code, E_MKV_LACING_INVALID = 9052,
deliberately NOT MkvInvalid — because is_skippable_title_stub classifies
MkvInvalid as a skippable nav stub, so reusing it would have recreated the exact
conflation that is still open as a separate finding. A test asserts the new code
is not skippable.

**TrackNumber was assumed to be 1..N in TrackEntry order.** RFC 9559 §5.1.4.1.1
only requires it to be non-zero and unique, so sparse or unordered numbers are
legal. Block routing and codec_private both computed track + 1. A real
TrackNumber map is now built, recorded only for TrackEntries that yield a stream
so dropped track types no longer shift the mapping.

Verified red here independently, and the failure mode is worse than mis-routing:
with track + 1 restored, a buttons track's payload was attributed to the AUDIO
stream — wrong payload into the wrong codec parser.

**The NAL length prefix was hardcoded to 4 bytes.** lengthSizeMinusOne lives in
avcC byte 4 and hvcC byte 21 (ISO/IEC 14496-15 §5.3.3.1.2, §8.3.3.1.2) and was
never read, so a source declaring 1- or 2-byte prefixes had its raw prefixed
bytes emitted verbatim with no start codes. All four conversion sites now derive
the width from the track's own configuration record.

**Edit lists were ignored.** No edts/elst was parsed, so the presentation
timeline an edit list defines (ISO/IEC 14496-12 §8.6.5/§8.6.6) was dropped —
which is how encoder delay is normally expressed. Leading empty edits and the
first media edit's media_time are now applied to both dts and pts, with the movie
vs media timescale distinction respected. A list needing more than a constant
shift applies the leading edit and warns rather than presenting the result as
faithful.

17 tests. I reproduced the lacing mutant independently: returning the body whole
kills five of them, including the exact-payload and malformed-header cases.

Still open and deliberately untouched: the MkvInvalid / is_skippable_title_stub
conflation across ~20 reader raise sites. It is a cross-cutting error.rs change
and E_MKV_LACING_INVALID is the template for it.
2026-07-29 21:53:19 -07:00
Matthew Jackson 6e9a53e151 Anchor forensic segments to the forensic clip, and stop CPS branching on resolve order
Two correctness defects in AACS 2.1 / FMTS key-map resolution, both order- or
anchor-dependent, and both able to abort a whole disc or silently garble it.

**The single-CPS short-circuit depended on which title resolved first.**
`pool_len` counted the WHOLE unit-key pool, and resolve_fmts_key_map appends the
disc's forensic index keys to that same caller-owned pool. The count was captured
before THIS call's FMTS branch but not before earlier titles', so once any forensic
title resolved, every later title saw a pool larger than one and fell into
multi-CPS sampling — 8 random reads per extent, and a whole-disc DecryptFailed if
no pooled key opened a menu extent's samples. A disc that ripped fine when a
non-forensic playlist sorted first failed when a forensic one did.

Forensic keys are now tagged FMTS_POOL_TAG_BASE = 1 << 24 and the short-circuit
asks single_base_key_slot(), which excludes them. The old 1000 tag was NOT kept,
and the reasoning is worth recording: base CPS ids are Unit_Key_RO.inf position + 1
and that count is a BE16, so 1000 sits inside a genuinely reachable id space. 1<<24
cannot collide. The id field is cosmetic — decrypt.rs indexes the pool by slot and
reads only the key — so widening the tag is safe.

**Forensic segment SPNs were anchored to the wrong clip.** They live in the
forensic feature clip's byte space, but were mapped through
clip_byte_to_lba(&title.extents, ..), which treats byte 0 as the start of the
title's FIRST extent. Any playlist not beginning with the forensic clip mapped
every segment to the wrong LBA: either the anchor probe sampled the wrong clip and
the whole-disc resolve aborted with FmtsKeyMissing, or — worse — a forensic index
key was applied to non-forensic sectors while the real forensic units kept the base
key, giving silently garbled output with no error at all.

The correct anchor turns out to be a DISC fact, not title data: an AACS 2.1 disc
names its forensic feature BDMV/STREAM/<clip>.fmts, and carries one
IndividualSegment.tbl, so the SPNs are in that one clip's byte space. A new
forensic_clip_extents() finds the unique .fmts in the already-walked UDF tree, and
those extents now drive the segment arithmetic, the addressability filter, and the
index probe — whose title parameter is gone, since its reads were mis-anchored too.
"Does this title carry forensic content" is now "does it read the forensic clip's
sectors" rather than "do the segment bytes land somewhere in the concatenation".

Where the clip is NOT identifiable — no .fmts, or several, making the SPN space
ambiguous — on a disc that does carry a non-empty table, the resolve now fails loud
with FmtsKeyMissing rather than guessing an anchor. That is a deliberate behaviour
change: a hypothetical disc with two .fmts clips hard-fails where it previously
produced a possibly-wrong map. Failing loud beats silently garbled output, and
inventing an anchor was not acceptable.

This site had been flagged independently three times — by the agent that added the
FMTS per-disc memo, by the round-4 correctness lens with a concrete scenario, and
by the round-4 conformance pass.

Verified red here independently: reverting single_base_key_slot to count the whole
pool fails both new tests. The agent's own evidence was probe_reads 48 vs 40 (the
8 extra sampling reads) and an E7013 DecryptFailed whole-disc abort, and for the
anchor an E7026 FmtsKeyMissing on a [trailer, forensic] extent list.

All six pre-existing FMTS tests pass unchanged through the new anchor, including
the exact-cost assertions (40 probe reads, one key-service call per disc, one UDF
walk for 60 titles), so the round-3 memoisation wins are intact.
2026-07-29 21:37:50 -07:00
Matthew Jackson 75bcc71f34 Group whole E-AC-3 frame sets, and keep short reads unit-aligned
Two defects in fixes landed the same day, both found by round 4 auditing round 3's
work rather than trusting it.

**The E-AC-3 grouping ignored substreamid, so the timeline still doubled.**
Confirmed independently by the correctness and conformance lenses and verified by
hand: substreamid appeared only in test helpers, never in the production path. Per
ETSI TS 102 366 (A/52) Annex E a frame set is independent substream 0 — mandatory,
always first — with its dependents, then the OPTIONAL additional independent
substreams 1..7 with theirs, all covering the same time period. Treating an
additional independent substream as a new access unit advanced the clock a second
time for the same 32 ms, which is exactly the doubling the grouping fix existed to
prevent. No fixture caught it because every fixture used substreamid 0.

is_dependent_substream becomes substream_role -> Starts | Extends: strmtyp 1
extends; strmtyp 0/2 with substreamid != 0 now extends (this was the bug); strmtyp
0/2 with substreamid == 0, legacy AC-3, and reserved strmtyp 3 start. Reserved 3
starts regardless of its id bits, because its BSI layout is undefined so those bits
cannot be trusted — an unknown frame is neither merged into an unrelated programme
nor silently discarded.

The frame set stays ONE sample rather than being split into a separate track for
the associated service, and the reasoning is in the module doc: a substream
numbered 1..7 with no substream 0 is not conforming, so extracting one would mean
renumbering ids and rebuilding frame sets — a transcode, not a remux. Programme
selection is the player's job.

A stream joined mid-frame-set (first sync is substreamid 3) is skipped with a debug
and resyncs at the next id-0, mirroring the orphan-dependent rule: its mandatory
id-0 substream was never seen, so it is neither decodable alone nor timeable.

MAX_AC3_BUF 128 KiB -> 1 MiB, because an AU is now a whole frame set: worst case 8
independent x 9 substreams x 8192 B = 576 KiB, which the old cap could have dropped
mid-hold.

**The forced probe's two round-3 fixes cancelled each other.** CHUNK_SECTORS = 1023
exists (with a const assert) so every read starts on a 3-sector AACS aligned-unit
boundary; the short-read fix advanced by actual bytes, making the advance a
non-multiple of 3. Every later read was then misaligned, DecryptingSectorSource
refused it before reading, the stop became ReadFailed, and no verdict was asserted
— so content-based forced detection silently fell back to the vendor label on
exactly the encrypted discs the 1023 change was written for.

A partially-satisfied read now advances only by whole aligned units and re-reads
the <=2 residue sectors from the next boundary, feeding only the aligned prefix so
nothing is double-fed and no partial unit reaches the parsers. A read that fully
satisfies its request still advances by all of it. When less than one aligned unit
comes back the bytes are fed and the same LBA is retried twice before stopping, so
a starved source cannot spin — verified by raising the retry limit and watching the
test hang.

Verified red independently here: reverting substream_role to strmtyp-only fails
eac3_additional_independent_substream_stays_in_the_frame_set (6 access units where
3 are correct — the doubling, literally) and the mid-frame-set resync test.

Reported, not fixed: dec3_box still hardcodes num_ind_sub - 1 = 0 and
num_dep_sub = 0, so it under-declares any stream carrying additional independent
or dependent substreams now that frame sets arrive whole. DolbyConfig has no
fields for either; a real fix needs the parser to surface observed substream
counts. That file is another lens's this round.

Unverified: no real multi-programme DD+ stream exists here, so defect 1 rests on
synthetic Annex-E fixtures. The retail DD+ check (No Time to Die, all substreamid
0) confirms single-programme discs are unaffected.
2026-07-29 21:33:18 -07:00
Matthew Jackson de46268785 Make two public-API panics return errors, and drop two shipped citations
**The session panics are reachable, and my earlier triage of them was wrong.**
`DiscSession::scan` and `resolve_keys` both did
`self.drive.as_mut().expect(..)`. I previously downgraded these to LOW on the
grounds that no shipped consumer calls them after the drive has been staged into
the reader slot. That is the wrong test: `stage_drive_as_reader` is a PUBLIC
method that empties the drive slot, so the public surface permits the sequence,
and a library must not panic from public API regardless of what current callers
happen to do. Both now return Error::DeviceNotReady.

**A shipped doc comment cited a third-party source FILE** as the authority for
the CLPI ProgramInfo layout ("Layout per the BD CLPI spec clpi_parse.c"). Now
cites the Blu-ray Disc Read-Only Format Part 3 CLIPINF specification.

**The CHANGELOG justified a muxer decision by naming a commercial competitor**
("MakeMKV's rip of the same disc omits it", "matching MakeMKV"). Reworded to
stand on its own terms: the element is optional in RFC 9559, nothing requires it
for interlaced SD, and the 40 ms DefaultDuration is the frame rate the source
actually carries.

The leak gate is extended for both new classes — third-party `*_parse.c` /
`*_dec.c` / `*_demux.c` style filenames, and a competitor named as authority.

Narrowing that rule took two attempts, which is worth recording. A bare
`\.(c|cpp|cc)` pattern produced eight false positives: this repo has its own C
shim (`macos_shim.c`) that build.rs and the docs legitimately reference, and the
pattern also matched the Rust field access `p.cc`. It now matches only the
suffixes typical of third-party media-library sources. This is the second
false-positive round on this rule — the first flagged ffmpeg INVOCATIONS in the
test harness — so the lesson is that a hygiene pattern needs testing in both
directions before it lands, exactly like any other code.
2026-07-29 21:21:29 -07:00
Matthew Jackson 22f71bb243 Restore the MP4 conformance fixes I clobbered while landing another agent's work
detect_rate's nearest-match fix, the colr HLG/BT.470 fix and their four tests
were silently reverted. Cause: the r3fix-silent worktree was cut BEFORE the
conformance commit landed, and I landed its work by copying whole files into the
main tree. mp4/mod.rs was in both agents' file sets, so silent's copy — built on
the older base — overwrote the conformance changes wholesale. The gate stayed
green throughout, because reverting a fix and its tests together is perfectly
consistent.

Re-applied the conformance commit's diff for that file with a three-way merge;
both agents' changes to mp4/mod.rs now coexist (final_report, UndescribableAudio
and the max(track_id) id fix are all still present alongside RATE_TOLERANCE_FPS
and the colr resolver delegation).

Only mp4/mod.rs was affected. mp4/audio.rs and mkv.rs were in no other agent's
file set and were intact.

Process lesson, recorded because I would otherwise repeat it: NEVER land a
parallel agent's work by copying whole files, when its worktree was cut at an
older base than HEAD. Apply its DIFF (git apply -3), or rebase its worktree
first. Copying files silently discards anything committed to those files in the
interim, and no test can catch it because the tests disappear with the code.
2026-07-29 21:03:26 -07:00
Matthew Jackson c73d6c6291 Sweep the pinned toolchain to Rust 1.97
The Windows UI needs current winsafe, whose real minimum is 1.89 (its manifest
under-declares 1.87 while it uses NonNull::from_ref). Rather than stop at the
minimum, this goes to current stable and fixes what that costs.

The counter-intuitive result: 1.97 is CHEAPER than 1.89. libfreemkv had 54
clippy errors at 1.89 and 6 at 1.97, because clippy tightened the noisy
collapsible_if lint in between. Stopping at the minimum would have been the most
expensive choice available.

Roughly 47 lints across the eight repos, the large majority auto-fixed:
libfreemkv 6, freemkv-engine 14, bdemu 8, freemkv-keysources 7, autorip 6,
freemkv-unlock 3, freemkv-i18n 3. The hand-fixed ones are a descending sort to
sort_by_key(Reverse), four manual checked-division sites, a loop counter replaced
by enumerate, and a loop whose first let-else became a while-let.

Worth recording for whoever bumps next: clippy is MSRV-AWARE. Those 54 lints only
appear once the crate DECLARES 1.89 or later, because let-chains become
available. A bare `cargo +1.89 clippy` against a manifest still pinned at 1.87
reports clean and is meaningless — gate with the real precommit script, which is
also the only thing that covers build scripts.

The pin still sits below the Mac default, so it keeps doing its job: catching
lint drift locally before CI sees it.
2026-07-29 21:00:55 -07:00
Matthew Jackson 41102bf023 Fix fifteen defects across perf, resource, panics and key hygiene
All 21 findings held up under verification; 15 fixed here, 6 deferred to files
another agent held this round, 0 rejected.

**A defect in my own round-2 probe fix.** CHUNK_SECTORS was 1024, and
1024 % 3 == 1 — verified — so every chunk after the first was misaligned against
the 6144-byte AACS aligned unit and would be REJECTED by
DecryptingSectorSource's alignment gate. On an encrypted disc the forced-subtitle
probe I added last round would have read almost nothing past its first chunk.
Now 1023 sectors (341 aligned units) with a const assertion that fails the build
if it stops dividing, plus set_unit_base per extent so the source's gate is
anchored where the extent actually starts.

**The same probe skipped sectors on a short read**, advancing by the REQUESTED
count rather than the bytes actually returned, so a partial read silently left a
gap in the middle of the evidence. It now advances by n/SECTOR_BYTES and clamps n
to the buffer.

**Its cache key omitted the PGS PID set**, so a playlist declaring an extra
subtitle PID got another playlist's verdict for a track that had never been
probed. And the key was the whole extent list, so partial clip sharing missed
entirely. Both fixed by keying (start_lba, sector_count, pid) — and per-extent
keying was shown SOUND rather than assumed: ForcedTracker is two monotone
booleans, so per-extent evidence composes by field-wise OR, order- and
grouping-independently. Making that honest required per-extent demux state, so an
extent's evidence comes only from its own bytes, and memoising only extents whose
read reached a designed stop.

**A reachable panic in the timeline.** mkvstream::parse_block accepts a
TimestampScale up to i64::MAX, so a video frame can set high_ns = i64::MAX and the
next passive frame panicked adding the backstep. In release it wrapped negative
instead, firing the straggler clamp for essentially every passive frame — audio
and subtitles rewritten onto the wrong point of the output timeline. All four
sites saturate.

**A public constructor divided by zero**: PrefetchedSectorSource::new_with_events
with unit_align == 0. Now InvalidInput, matching its batch_sectors sibling.

**Two Debug impls printed key material.** DiscInputs (volume_id, mkb, unit_key_ro,
samples) and UnitKeyFile both derived Debug. Nothing logs them today — fixed as
prevention, because the next tracing::debug! someone adds is the leak. A doc claim
that DiscInputs "contains no secrets" was false and is corrected.

**An env-var multiply could overflow** in file_sector_source; now bounded at 64 GiB
like its writeback sibling, with the parse split out so the bound is testable
without touching process env.

**The mp4 demuxer allowed one sample per file byte** — ~64x RAM amplification.
Now file_len/16, since only vide/soun tracks are indexed and the shortest legal
AC-3 frame is 128 bytes.

**Two pipeline concurrency defects**: a consumer apply() error was invisible to the
producer, and abandon/finalise had a TOCTOU where a caller could report an
unfinalised output. Both fixed with compare-exchange state rather than a bool.

**Two per-frame copies removed**, both MEASURED rather than reasoned: the AU
assembler now hands its allocation to the frame (same pointer, unchanged capacity,
proven by asserting the pointer) and tsmux reuses one Annex-B buffer across
frames. Both keep capacity deliberately — a naive split_off would have cost more
than it saved.

**A comment pointed at the wrong file** for a mirrored constant; the mirror is now
compiler-enforced with a const assertion converting 90 kHz ticks to ns, so drift
fails the build.

Deferred to another agent's files, all confirmed: detect_rate's fractional-twin
snap, the mp4 reserve's u32 truncation, round_up_grain's overflow, the quadratic
base-key gap fill, and MkvStream's frame cap counting frames rather than bytes.

Every fix verified red by reverting it. Also noted for later:
DecodeSampleSet still derives Debug over multi-MB of on-disc ciphertext.
2026-07-29 20:47:31 -07:00
Matthew Jackson 07cb9ec62c Make six silent mux failures observable
All six confirmed against the code. The governing rule this cluster serves: a
lossy or degraded outcome is never silent, because a corrupt rip the user does
not know about is the worst failure available.

**A 3D MKV re-mux silently lost one eye.** The BlockGroup read path had arms for
BLOCK / BLOCK_DURATION / REFERENCE_BLOCK only, so BLOCK_ADDITIONS fell into the
skip arm — while the writer does emit BlockAdditions > BlockMore > BlockAdditional
for the MVC dependent view. Reconstruction was judged out of scope and the
reasoning is recorded: PesFrame has no side-payload field and the header parser
never reads BlockAdditionMapping, so there is no dependent-view track to route the
AU to. Instead the loss is now LOUD — counted in bytes and events, warned once,
and surfaced through MkvStream's errors()/lost_bytes(), which the driver already
samples into MuxOutcome. One detail in the finding was wrong and is corrected: the
re-mux does NOT still advertise the mvcC mapping, because the header parser
ignores that element, so the output is a plain 2D H.264 track.

**An all-titles rip silently skipped real titles.** The header-buffer-cap
overflow returned Error::MkvInvalid, and is_skippable_title_stub matches exactly
E_MKV_INVALID | E_CSS_KEY_MISSING — verified here — so a 512 MiB-of-frames title
was classified as an empty nav/menu PGC stub and dropped. It now has its own
E9051 / MuxHeaderBufferExceeded { bytes }, outside the skippable set.

**The public pre-mux report contradicted the file.** Mp4Sink::finish() drops an
audio track it cannot describe, which I chose last round over failing an export
whose video is fine — but mp4_fit_report still listed that stream as included, so
the application's plan and the actual output disagreed. Fixed at both levels:
Mp4SkipReason is now non_exhaustive with NoSamples and UndescribableAudio,
Mp4Sink::final_report() describes the FILE rather than the plan, and for the
boxed dyn Stream path a defaulted Stream::undelivered_streams() carries the
information out to MuxOutcome::undelivered_streams with a driver-side warn.

**MP4 track ids could collide.** ids were assigned before the retain that drops
sample-less tracks, while next_id came from the post-retain count, so [1,3]
yielded next_id 3. Now max(track_id) + 1, saturating.

**Stream selection silently skipped its codec_privates prune** when the lists were
not the same length — but codec_privates is consumed POSITIONALLY and trailing
extras are documented as benign, so the length-equality guard was itself the bug.
The prune now runs unconditionally by index.

**The m2ts_mux scaffolding armed params_written on both the absent and the
unparseable codec_private arms** — the same defect already fixed in tsmux.rs.
Split into params_attempted (a latch, since retrying identical bytes cannot help)
and params_emitted, with a warn on each failure arm and an accessor so the
eventual wiring and its test can observe it.

Each fix verified red by mutating back to the prior behaviour: errors() 0 vs 1,
E6008 vs E9051, final_report [0,1] vs [0], next_track_id 3 vs [1,3], and the
selection prune resolving index 1 to the wrong track's record.

API surface deliberately widened: MuxOutcome gains a public field and Mp4Sink
becomes public. Nothing in-repo breaks. Note a behaviour change on the mkv://
input path — a 3D re-mux now reports non-zero loss, so a consumer treating
errors > 0 as disc damage will trip on it. That is intended: the outcome IS
degraded.
2026-07-29 20:46:10 -07:00
Matthew Jackson 455763587c Fix four MP4 conformance defects against the standards
**dec3 declared a 0 kbit/s AC-3 substream.** parse_dolby routes bsid < 11 to
parse_ac3, which leaves data_rate_kbps = 0 and keeps the AC-3 bsid, yet
dolby_sample_entry wrapped that config in ec-3/dec3 for any Codec::Ac3Plus track.
ETSI TS 102 366 Annex F.4 assigns ac-3/dac3 to an AC-3 bitstream and F.6 assigns
ec-3/dec3 to an Enhanced AC-3 one, so the entry now follows the SYNCFRAME that was
actually parsed, not the playlist's codec label. Computing an AC-3 data rate and
keeping ec-3 was rejected: it fixes one field while bit_stream_identification and
num_dep_sub keep misdescribing the stream. The bsid threshold is hoisted into one
constant so parser and entry-chooser cannot drift. Adjacent defect fixed in the
same box: data_rate is 13 bits from a u16 source, so push's mask WRAPPED anything
above 8191 (9000 became 808); it now saturates.

**colr tagged HLG as PQ, and PAL as BT.601.** video_colr carried a second,
drifted copy of the ColorSpace-to-CICP map: transfer 16 (PQ) for every BT.2020
stream with no HdrFormat override, and 6 (BT.601) for Bt470bg. Per ITU-T H.273
Table 3, HLG is 18 and BT.470-6 System B/G is 5 — and mkv::cicp_for_video already
got both right. video_colr now delegates to that shared resolver, so the
duplicated table is gone and cannot drift again. It keeps only its own decision
about WHETHER to emit the box, since an absent colr and an all-unspecified colr
mean the same thing per ISO/IEC 14496-12.

**Exact 24.000 / 30.000 / 60.000 fps was declared 23.976 / 29.97 / 59.94.**
detect_rate took the FIRST STD_RATES entry within 0.5 fps, and every 1000/1001
entry precedes its integer twin 0.024 fps away — a 0.1% error across the whole
track's mdhd and stts. Fixed as nearest-wins rather than by reordering the table:
reordering fixes today's table and re-breaks the moment someone appends a rate,
while nearest-wins is order-independent. Verified red here independently by
reverting to first-match, which fails exactly the two timing tests.

**ddts MultiAssetFlag was set from has_extension.** In the DTSSpecificBox
(ETSI TS 102 114) that flag signals more than one audio ASSET. A DTS-HD MA/HRA
track is one asset whose extension substream carries the XLL/XBR component, so
setting it from "an EXSS sync follows the core" sent a parser looking for a second
asset descriptor while StreamConstruction simultaneously said there was no
extension — the box contradicting itself. This module parses the core header only
and never reads the EXSS asset table, so 0 is the only honest declaration. A
StreamConstruction index for core+EXSS was deliberately NOT invented: that field
is a table lookup that could not be confirmed against the standard, and a wrong
index is worse than an under-declaration.

DTS_AMODE_LAYOUT's masks were independently re-derived against ETSI TS 102 114
§5.3.1 and all 16 are CORRECT — only three adjacent comments were wrong (the
AMODE 2/3/4 annotations were rotated by one, and AMODE 9's said "5.1 with LFE"
when 0x0007 is the 5.0 mask and LFE is OR'd in separately). Comments corrected.

Every one of the eight new tests decodes the field back OUT of the emitted bytes —
data_rate from the dec3 body's leading 13 bits, MultiAssetFlag from bit 48 of the
ddts tail, colr from the nclx payload inside a real stsd, and the frame rate from
mdhd.timescale plus stts.sample_delta of a fully muxed MP4 — rather than
restating arithmetic. This audit has already caught one of my own tests doing the
latter.

NOT fixed, root-caused and recorded at the site instead: an A_PCM/INT/BIG track
ships with no BitDepth, which the Matroska Codec Specifications make a MUST. The
width exists on disc (BD LPCM signals it in the ES header byte 3, DVD in the IFO
audio attribute byte 1) but neither source reaches MkvTrack::audio, and the fix
needs a new AudioStream member plus a deferred setter in files another agent held
this round. Guessing 16 was rejected — it would confidently misdecode every
24-bit disc — as was refusing the track, which would regress the 16-bit majority
that currently plays by accident.
2026-07-29 20:29:17 -07:00
Matthew Jackson 4b58d3db98 Resolve the forensic key map once per disc, not once per playlist
resolve_content_key_map loops every title into resolve_mux_key_map, which called
resolve_fmts_key_map FIRST — before the CpsUnitCache — and on an FMTS disc
returned immediately. So every playlist re-derived facts that belong to the DISC:
a full UDF walk plus /AACS/IndividualSegment.tbl, and on an FMTS disc the anchor
probe, the 32-index phase probe, and a fetch.fmts_indexes round trip.

On a 60-playlist disc, measured on a synthetic fixture: 840 -> 14 metadata reads,
2,400 -> 40 probe reads, and 60 -> 1 key-service calls. Worst case before was up
to 256 probe reads and 32 key-service calls per title. The 60 redundant
key-service round trips are a strong candidate for the keyserver storm seen in
the field.

Two memos behind a pub(crate) DiscKeyCache. The table memo (UDF walk + tbl parse)
is disc-invariant outright — nothing in that path mentions the title — and runs on
EVERY disc, so a plain BD benefits too. Only the deterministic negatives are
memoised as "not FMTS"; a DiscRead fault propagates uncached so a later title
retries.

A blind once-per-disc hoist of the PROBES was rejected as unsafe, and this is the
load-bearing reasoning: the title enters through clip_byte_to_lba, which decides
which segments are addressable and which LBA every probed clip byte reads from, so
two titles with different extent lists probe different physical bytes. A hoist
would serve title B an answer derived from title A's media and could silently turn
a per-title FmtsKeyMissing into a success. The extent list is the ONLY per-title
input, so keying on it is exactly sufficient — matching the ForcedProbeCache
precedent.

Result-identity was proved, not assumed: NEITHER probe reads the key pool.
Verified here independently — probe_fmts_index_keys takes no keys parameter at
all; index keys come from `fetch`, and the anchor's reply feeds the phase probe.
So the pool's growth across titles, the one thing that does change between calls,
cannot move a memoised value, and the result is order-independent. A test resolves
three titles through a shared memo and through fresh memos and asserts both the
per-title ranges and the final key pool (keys, slots, order) are identical.

Not memoised, deliberately: fail-loud FmtsKeyMissing, and any run where an index
hit a read fault — that is a property of a transient drive fault, not of the
extents, and caching it would spread one bad read across 59 playlists.

A fully-memoised title now does zero I/O, which made the old in-loop halt polls
unreachable for it, so a check_halt on entry was added with a test that cancels
after warming the memos.

Also corrects my own overstatement from last round: the CpsUnitCache doc now says
plainly that on an FMTS disc it removes NO reads, because this function returns
before the extent loop ever runs.

Five mutations, all verified red. Pre-existing bug flagged but not fixed:
filter_addressable_segments only checks that a segment's START byte maps to some
LBA in the title, so a play-all playlist can pass the filter while mapping segment
bytes into the wrong clip, whose anchor then returns empty and aborts the sweep.
2026-07-29 20:27:50 -07:00
Matthew Jackson d451d00f72 Group E-AC-3 dependent substreams into one access unit
The AC-3 parser's own module doc stated the assumption: "AC3 frames are
self-contained and always start with syncword 0x0B77". True for legacy AC-3,
false for E-AC-3 above 5.1. Per ETSI TS 102 366 (A/52) Annex E, byte 2 of an
E-AC-3 syncframe is strmtyp(2) | substreamid(3) | frmsiz[10:8], and an access
unit is one INDEPENDENT substream plus every DEPENDENT substream that follows it
until the next independent one. The parser emitted one PES frame per syncframe,
so a decoder saw each dependent substream as a standalone frame with no parent —
including the AC-3-core + E-AC-3-dependent form Blu-ray uses for Dolby Digital
Plus. The extra channels were lost and the timeline ran at 2x.

The bit position is cross-checked against code already in the tree: the existing
frmsiz parse takes byte2 & 0x07 as its high bits, which is only consistent with
strmtyp occupying byte2's top two bits. Legacy AC-3 is excluded by bsid < 11,
where byte 2 is crc1 and reading strmtyp there would be nonsense. Reserved
strmtyp 3 is treated as INDEPENDENT so an unknown type starts a fresh AU rather
than merging into an unrelated one.

The AU carries the INDEPENDENT substream's PTS, and only the independent
substream advances the clock — dependents cover the same time period and add zero
duration. That is what removes the doubled timeline.

A trailing AU that can still grow is HELD across the PES boundary, because the
boundary is unknowable until the next independent sync; the whole AU is re-scanned
next call, so there is no shift and no double-count in the loss tally. Plain AC-3
is never held, which keeps DVD/AC-3 latency and behaviour unchanged.

A latent pre-existing bug surfaced while testing this: a new PES's PTS was
re-stamping an AU that began in an earlier PES, a constant one-frame shift. Fixed
with a PtsAnchor so a PES timestamp applies to the first AU that STARTS in that
PES's own bytes, while a genuine PTS jump is still adopted.

Nine tests. Verified red against five mutations, each killing a specific set:
reverting to the pre-fix behaviour kills 8 while
plain_ac3_frames_are_not_grouped_or_delayed SURVIVES as the no-regression guard —
reproduced independently here. Stamping the dependent's PTS kills 6; not holding
across PES kills 4; holding plain AC-3 too kills 15; neutering the PTS anchor
kills exactly the 2 split-across-PES timing tests.

Three sibling defects found and deliberately NOT fixed, all in mp4/audio.rs:
dec3 hardcodes num_dep_sub = 0 (and a nonzero value changes the box LAYOUT, not
just a field, per Annex F/G); parse_eac3 ignores strmtyp/substreamid entirely;
and a 7.1 DD+ track is still labelled 5.1 because the channel count comes from
the independent substream while the extra channels are described by the
dependent's chanmap, which nothing parses.

Not verified: no real E-AC-3-with-dependents sample exists here, so all evidence
is synthetic frames plus the spec layout. The multi-independent-substream case
(num_ind_sub > 1, main + associated audio in one PID) is deliberately treated as
one AU per independent substream and is untested.
2026-07-29 20:26:51 -07:00
Matthew Jackson f6ba4823d1 Fix three defects in last round's own fixes
Round 3 audited the round-1/2 fix commits rather than trusting them, and found
three defects in that new code. This is why the pin moves each round.

1. LICENCE REGRESSION, and it was mine. Reverting the "distinguish a failed key
   source" commit also restored a verbatim reference-decoder table citation in
   src/mux/codec/dts.rs, because both changes were in that one commit. The MIT
   licence cleanup was silently undone at HEAD and nothing caught it.

   The citation is replaced with ETSI TS 102 114 §5.3.1 again, and — more
   importantly — the rule now lives in the leak gate instead of in my memory.
   scan-secrets.sh gains LICENCE_RE, which flags ff_dca*, dcadec, l-smash,
   libav*, and bare ffmpeg/FFmpeg as REF-IMPL-CITATION. `no ffmpeg` is
   explicitly allowed via negative lookbehind: stating what this project does
   NOT depend on carries no risk and is a genuine selling point. Verified by
   re-introducing the citation (gate fails) and removing it (gate clean).

2. TsMuxer armed params_written even when the avcC/hvcC parser returned None, so
   a track whose codec_private exists but will not parse was muxed to BD-TS with
   no VPS/SPS/PPS ever emitted — undecodable video, reported as success, with no
   log line. Last round's fix corrected WHICH parser is used and left this half
   untouched. Arming the flag is still right (retrying identical bytes cannot
   succeed) but it is no longer silent: it now warns with the track, codec and
   codec_private length.

3. test_aes_cbc_roundtrip defined a LOCAL fn aes_cbc_encrypt that SHADOWED the
   production primitive, so it round-tripped a copy of the algorithm against
   itself and never touched crypto::aes_cbc_encrypt — the function this cycle
   added. Any mutation to the shipped code passed it. The shadow is deleted and
   the test now calls the real primitive; verified by mutating
   crypto::aes_cbc_encrypt, which now fails it and previously would not have.
2026-07-29 20:08:09 -07:00
Matthew Jackson 85f2246b0f Pin the toolchain to Rust 1.87
The Windows UI needs winsafe, whose current release requires rustc 1.87. The
alternative was pinning winsafe back to an older release, which would bake a
stale API surface into a brand-new UI permanently to dodge one minor version.

The pin's purpose is to sit BELOW the Mac default so clippy drift is caught
locally before CI, not to stay on 1.86 specifically, so 1.87 preserves the
discipline exactly.

Verified before moving anything, not after: `cargo +1.87 clippy -- -D warnings`
and `cargo +1.87 fmt --check` are clean across all eight repos, and the full
precommit gate (fmt + clippy + tests) passes on libfreemkv, autorip, bdemu,
freemkv-engine and freemkv-keysources. Zero new lints, zero formatting drift.
2026-07-29 20:02:02 -07:00
Matthew Jackson 61f98535d8 Memoise multi-CPS key-map sampling per extent, not per title
resolve_content_key_map calls resolve_mux_key_map once per title, and on a
multi-CPS disc that path issues 8 random single-unit reads per extent. A disc's
playlists overwhelmingly reference the same few clips — main feature, play-all,
per-chapter and seamless-branch variants — so the same physical extents were
re-sampled from the drive once per playlist. On a 60-playlist / 15-clip disc
that is ~2,400 non-sequential 6144-byte reads, roughly 8 minutes of pure seeking
at 200 ms per seek, before the mux starts. Now ~600 reads.

Keyed per EXTENT — (format, start_lba, sector_count) — rather than per title's
whole extent list, which is finer-grained than the forced-subtitle probe's cache
and strictly better here: a play-all playlist sharing 4 of 5 extents with the
main feature still hits on those 4.

Why a cached pool index is provably identical to a recomputed one, verified
rather than assumed:

  * `pick` iterates the pool IN ORDER and returns the FIRST index whose key
    decrypts a sample to clean.
  * The pool is APPEND-ONLY. Checked across the whole crate: only `push`, with no
    insert/remove/clear/retain/sort/dedup/truncate/drain/swap/reverse anywhere.
    So appended keys can only land AFTER a matched index, and the first match for
    the same samples cannot shift.
  * The samples are a pure function of the three values in the key, read from
    read-only optical media.

Two outcomes are deliberately NOT cached, which is what makes this safe rather
than merely faster:

  * the inherited index (`None if samples.is_empty() => last_idx`) is per-TITLE
    state, not a property of the extent — caching it would let one title's
    carry-in index leak into another title's clear extent, i.e. a WRONG key;
  * the fail-loud DecryptFailed verdict, so a retry after a key source banks the
    missing key re-samples instead of inheriting a stale answer.

Halt is still polled before the cache lookup, so cancellation is unchanged.
resolve_mux_key_map keeps its exact signature and delegates with a fresh cache,
so there is no public API change. ContentFormat gains Eq + Hash (additive).

Four tests, and the two mutants that matter both verified red: disabling the
cache short-circuit fails the hit and recompute-equivalence tests, and wrongly
caching the inherited index fails multi_cps_inherited_index_is_not_cached.
2026-07-29 19:49:55 -07:00
Matthew Jackson 36d0801f14 Build BlockGroups in memory so the hot path never seeks
Every BlockGroup frame back-patched its element size via ebml::end_master, which
does two stream_position() calls and two real seeks. BufWriter does not override
Seek::stream_position, so each position query is seek(Current(0)) = flush_buf +
lseek — and each of those flushed the 4 MiB BufWriter while every position-moving
seek reset WritebackPipeline::last_flush_pos. The buffer never got to do its job.

An MPEG-2 title takes this path for EVERY frame (the parser stamps a per-frame
duration, so I, P and B all become BlockGroups) — roughly 350,000 per feature.

A BlockGroup's size is knowable before writing, so there is no need to back-patch
at all. New seek-free twins start_master_buf / end_master_buf patch a placeholder
by buffer INDEX instead of file offset, and build_block_group assembles the whole
element into a persistent buffer that write_block_group and its MVC sibling take,
fill, write once, and hand back — including on the error path — so the allocation
is made once rather than per frame.

Measured with a counting writer that, like BufWriter, does not override
stream_position, over 200 frames:

              seek calls   position-moving
  plain   before 912              451
  plain   after  112               51
  MVC     before 2516            1253
  MVC     after  116               53

Per-frame cost is now zero; the residual is the header, the per-cluster
back-patch and Cues. For 350k BlockGroups that is 1.4M seek calls removed on the
plain path, 4.2M on an MVC title.

end_master is deliberately NOT changed for its other callers. The Cluster master
genuinely streams — frames are appended to an open cluster over time, so its body
cannot be buffered without holding a whole cluster in memory — and the rest
(EBML header, Tracks, Info, Chapters, Cues) run once, not per frame.

Byte-identity is the safety property, and it is structural: end_master always
patches a FIXED-width 8-byte VINT, and the buffered pair writes and patches
exactly that same placeholder, so the encodings cannot differ. Verified by
capture-then-compare over 200 frames (17 keyframes / 183 non-keyframes, multiple
clusters, BlockDuration present and absent, both reference branches) — output
byte-for-byte identical.

Three tests now pin it permanently: buffered vs seeking output byte-for-byte for
empty/tiny/multi-byte bodies, the same for NESTED masters (the MVC path nests
BlockAdditions > BlockMore, where an index-arithmetic slip would surface), and
end_master_buf erroring rather than panicking on a position outside the buffer.
All three verified red against a deliberately divergent placeholder width.

A note on verifying this kind of change: comparing emitted MKV across two
worktrees at different commits shows a spurious 14-byte difference, because
MuxingApp/WritingApp embed the build's git SHA twice. Compare at the same base.
2026-07-29 19:33:31 -07:00
Matthew Jackson 656c6f36d7 Only assert a forced-subtitle verdict the read actually supports
probe_and_set_forced broke out of its read loop on a read error but still
applied whatever partial observation it had accumulated as an authoritative
verdict, overwriting the vendor-label-derived forced flag. A disc that faults
early could have a correct flag replaced by a guess from a fraction of the data.

The file already got the zero-observation case right — it deliberately leaves
the vendor flag alone rather than "assert not-forced from having seen nothing".
The defect was that a PARTIAL observation cut short by a fault was treated as
complete.

The fix rests on the two verdicts not being symmetric evidence.
settled_not_forced() is POSITIVE evidence — a non-forced display set was
actually seen, and no unread data can retract it. is_forced() is an ABSENCE
claim — display sets were seen and none was non-forced — which is only sound if
the read got far enough for the absence to mean something.

So every loop exit now yields a named StopReason, and absence claims are
asserted only for a designed stop:

  * Exhausted / Budget → conclusive. The budget is deliberately conclusive: the
    natural exit is "every track settled not-forced", which a genuinely forced
    track never satisfies, so the budget is the ONLY path by which a real forced
    verdict is ever reached. Treating it as inconclusive would disable forced
    detection entirely.
  * Halted / ReadFailed → inconclusive. A cancelled probe's cut-off point is as
    arbitrary as a faulted one, so its absence claim is worth no more.

Evaluated per track, matching the existing observed() gate: a track that already
saw a non-forced set keeps its sound verdict even on a truncated run, while a
forced-so-far sibling keeps the vendor flag.

An inconclusive run is also NOT memoised. The cache key is the extent list and a
disc's playlists share clips, so caching a truncated run would replay one read
fault onto every playlist referencing those extents and deny any later title the
chance to re-read them.

Four tests; three of them verified red by forcing absence_is_conclusive() back
to always-true (the old semantics), while the budget test correctly stays green
either way — confirming the guard was not over-corrected.

Also moves the function doc comment back onto probe_and_set_forced; the earlier
probe commit left it attached to the ForcedProbeCache type alias.
2026-07-29 19:26:58 -07:00
Matthew Jackson 7d861f5287 Record that the failed-vs-absent key source arm is unreachable
The `Ok(_) | Err(_)` arm in resolve_and_apply_traced conflates "this source
had no entry for the disc" with "this source failed", and reports both as
KeyNode::NoEntry. An operator whose key server is returning 502s is therefore
told their disc is not in the database.

The conflation is real but LATENT, and fixing it here would change nothing an
operator can see, because no shipped KeySource ever returns Err:
KeydbSource::get_unit_keys maps a load/parse failure to Ok(Vec::new()),
OnlineSource::get_unit_keys is Ok(self.query(ctx)) where query returns empty on
transport error, HTTP status, oversize body and bad JSON alike, and MultiSource
discards inner Errs. Only test doubles return Err. FetchOutcome::errored in
drive_unit_keys / drive_fmts_indexes is dead for the same reason — the right
contract, honoured by no source.

autorip already works around the missing signal by re-probing the service over
HTTP (probe_online_reachability / key_service_transient_status), and its own
comment names the incident: "the online keysource swallows every failure
(transport error, 502, timeout)".

So the fix belongs at the source boundary in freemkv-keysources, with
Disc::aacs_error as the channel the operator actually reads — not in this
trace. Documented here so the next reader does not assume the arm works, and
does not "fix" a dead path as I nearly did twice.
2026-07-29 19:24:58 -07:00
Matthew Jackson 032994be40 Revert "Distinguish a failed key source from one with no entry"
This reverts commit e5e3b863fd.
2026-07-29 19:15:21 -07:00
Matthew Jackson a61503502e Remove reference-decoder citations from a public MIT-licensed repo
This crate is MIT licensed. Comments citing another decoder's internal symbols
and reproducing its tables verbatim create licence risk that no engineering
benefit justifies, so every such citation is replaced with the primary source:
ETSI TS 102 114 §5.3.1.

Eleven sites across src/mux/codec/dts.rs and src/mux/mp4/audio.rs. The technical
substance is unchanged in every case — the deficit-sample-count semantics, the
reserved-field skips, the invalid LFF value, and the 16 legal AMODE codes are all
spec facts and are now attributed as such. Four CHANGELOG entries that named a
validator are reworded; the "no a reference decoder" dependency claim stays, since stating
what this project does NOT depend on carries no risk.

I initially argued this was a false positive on the grounds that the project's
hygiene rules name internal infrastructure and reverse-engineering material, not
open-source citations, and that a channel-count table from a standard is fact
rather than expression. That reasoning missed the point: the exposure is MIT
distributing text derived from GPL/LGPL sources, and that is the maintainer's
risk to weigh, not mine. Reversed in full.
2026-07-29 19:09:10 -07:00
Matthew Jackson e5e3b863fd Distinguish a failed key source from one with no entry
resolve_and_apply_traced collapsed `Ok(_) | Err(_)` into a single
KeyNode::NoEntry step, so a key source that FAILED — server unreachable, keydb
unreadable, malformed entry — was recorded identically to one that simply had no
entry for this disc. The front-end renders that trace, so it told the operator
their disc is not in the database when the real cause was a fixable
infrastructure problem. drive_unit_keys and drive_fmts_indexes were refactored
this cycle to preserve exactly this distinction; this path had not been.

KeyNode gains a SourceFailed variant and the two arms are split. freemkv's
trace renderer matches KeyNode exhaustively with no catch-all, so its arm is
added in the same change — otherwise the consumer would not build.

Also made ETSI TS 102 114 the primary authority for DTS_AMODE_COUNT's comment
rather than a reference decoder internal symbol, and pointed it at this
crate's own cross-checked DTS_AMODE_LAYOUT / DTS_AMODE_CH tables.

A round-2 finding asked for every a reference decoder and a reference decoder citation in the DTS parser
to be stripped as a public-repo hygiene violation. Rejected: the project's rules
(scan-secrets.sh, CLAUDE.md) prohibit internal infrastructure references and
reverse-engineering material, and a reference-decoder citation is neither. The
AMODE channel-count table is a factual table from the standard, not expression
copied from an implementation. Citing the spec plus a corroborating
implementation is how a decodability gate should be justified.
2026-07-29 19:07:15 -07:00
Matthew Jackson 36c1469026 Reference keyframes per track, size the DTS reserve, correct two claims
The ReferenceBlock offset was computed for ANY video track, but the keyframe tick
it measures against was recorded in a single global slot gated to the PRIMARY
video track. On a title with two video tracks — an MVC base plus secondary view,
or a multi-angle disc — a secondary track's non-keyframe therefore referenced a
keyframe on a different track, or 0 (a self-reference) when the primary had not
produced one yet. The tick is now recorded per track, so a non-keyframe can only
reference a keyframe on its own track.

The faststart moov-hole estimate modelled every audio track as (E-)AC-3 at 1536
samples per frame. 1.6.0 added DTS to the writer's carried set, and a DTS core AU
is commonly 512 samples — a third of that — so a DTS track's sample table was
under-reserved threefold and the mux fell back to moov-at-end, losing faststart
on exactly the files 1.6.0 newly supports.

mvc_frame_emits_blockgroup_additional_and_reference asserted only that the
non-keyframe's ReferenceBlock was Some(_). Its non-MVC sibling, added in the same
commit, pins the exact offset; this one now does too, so a mutant emitting a
constant or wrong-signed offset no longer passes.

The comment above the mp4 sample budget claimed file_len stops a crafted file
inflating allocations "past the file's own size". Each indexed sample costs ~52
bytes, so the real ceiling is ~52x file_len (still capped by MAX_SAMPLE_COUNT).
The bound is real; the comment overstated how tight it is.
2026-07-29 19:03:59 -07:00
Matthew Jackson 4aff402e9d Validate stream-selection PIDs per class, not across both
StreamSelection::apply validated a listed PID by scanning ALL streams, so a PID
named in the wrong class's filter passed validation — an audio filter listing a
subtitle PID, say. `keeps` then matched it against the audio streams only, so
the requested track was silently absent from the output. That is precisely the
outcome this validation documents itself as preventing: "fail loud rather than
silently emit an MKV missing a requested track".

Each filter is now checked against its own stream class. The `listed_pids`
helper existed only for the cross-class scan and is removed rather than left
behind as dead code.

Test covers both directions plus the sanity case, and asserts a rejected
selection leaves the title unpruned.
2026-07-29 19:00:53 -07:00
Matthew Jackson fae5282eb9 Halve the per-frame allocation and copy on the m2ts NAL video path
The NAL path called length_prefixed_to_annex_b, which allocates a whole-frame
Vec of its own, then copied the result into a second whole-frame Vec — two
full-frame allocations and two full-frame copies per video frame. The crate
already has append_length_prefixed_as_annex_b, which writes the conversion
straight into a destination buffer; it is the same code path with the
intermediate removed.

The destination is also sized once up front instead of starting from Vec::new(),
which re-grew from zero capacity inside every conversion.

On a UHD HEVC title muxed to m2ts:// — ~200k video frames averaging ~310 KB of
ES at 60 Mb/s — that removes roughly 62 GB of allocation and 62 GB of memcpy.

Behaviour is unchanged: the existing tsmux conversion tests, including the
non-NAL passthrough and Annex-B default pair added last round, all still pass.

A first attempt reused a persistent scratch buffer across frames, which does not
work: the buffer is handed out as Cow::Owned and so can never be returned. A
right-sized single allocation gets most of the win without restructuring the
function around the borrow.
2026-07-29 18:58:39 -07:00
Matthew Jackson 2398a57979 Fix ddts numeric truncation and the decrypt pool's poison asymmetry
ddts CoreSize wrapped to zero on a maximum-size core. core_size is FSIZE + 1 and
FSIZE is itself 14 bits, so the maximum is 16384 — one past what the 14-bit
CoreSize field holds — and push()'s mask turned that into 0, declaring an empty
core frame. Clamped to 16383 instead: one byte short beats telling a decoder
there is no core. Proven red first (the field read back as 0).

ddts avg/max bitrate under-declared every non-integral frame rate. It computed
sample_rate / frame_samples first, so a 512-sample core at 48 kHz truncated
93.75 frames/s to 93. Multiplying before dividing, with round-to-nearest, keeps
the precision.

set_decrypt_threads skipped the pool swap on a poisoned lock while
DECRYPT_THREADS had already been updated, so the new thread count was reported
as taking effect while the stale pool kept serving. decrypt_pool() deliberately
recovers from poisoning for exactly this reason; the setter now does the same.
The pool Arc is immutable once stored, so a prior panic cannot have left it
half-written.

The CoreSize test decodes the value back out of the emitted box rather than
restating the clamp — a first draft asserted the clamp arithmetic against
itself, which would have passed against the unfixed writer.
2026-07-29 18:56:48 -07:00
Matthew Jackson 5ef391f029 Make encrypt_unit report a refused slice, and expand its key once
Two defects in the encrypt_unit promoted to public API last round, both found by
round 2 auditing that new code.

It returned silently without encrypting when the slice was shorter than
ALIGNED_UNIT_LEN. Its own contract requires the caller to set the container's
encrypted flag BEFORE calling — the header is the key seed — so a silent no-op
leaves a unit advertised as encrypted while still carrying plaintext, with
nothing for an authoring caller to check. It now returns bool and is
#[must_use], so ignoring the refusal is a compile-time warning; every call site
was updated to assert on it.

bool rather than Result deliberately: a wrong buffer length is a programming
error at a library boundary, not a disc condition, and a new Error variant would
mean a new numeric code plus its rendering in another repo.

It also drove CBC from the single-block aes_ecb_encrypt, rebuilding the AES key
schedule for each of the 383 blocks in a unit — an order of magnitude slower
than its inverse, which expands the key once via aes_cbc_decrypt. The missing
counterpart aes_cbc_encrypt now exists alongside it, and encrypt_unit calls it,
so the two directions are symmetric in structure as well as in result. For an
authoring caller encrypting a 90 GB image that removes ~5.6 billion redundant
key expansions.

New test pins the boundary: ALIGNED_UNIT_LEN - 1 returns false and leaves the
buffer byte-identical, ALIGNED_UNIT_LEN succeeds. The existing round-trip and
padding-asymmetry tests still pass, so the CBC rewrite is provably the same
transform.
2026-07-29 18:54:19 -07:00
Matthew Jackson ef36213331 Correct six stale comments and doc claims
All six describe code that does something different from what they say, which
is the class of defect that gets a maintainer to write a bug on purpose.

docs/clpi.md presented the CLPI stream-PID entry as byte-aligned 2/2/2/4/4-byte
fields with a 32-bit fine-entry count. It is one 80-bit packed block —
reserved(10) + EP_stream_type(4) + num_EP_coarse(16) + num_EP_fine(18) +
EP_map_start_address(32) — and num_EP_fine is 18 bits. Anyone parsing to the
doc's offsets would read garbage. Replaced with the real bit layout.

docs/udf.md said read_directory()'s recursion cap is 3; MAX_DIR_DEPTH is 8.

TROUBLESHOOTING.md called Pass 1 `recovery::copy`. The engine's `sweep` is
documented as "Pass 1 of a multipass rip"; `copy` is the dispatch verb that
chooses between sweep and patch. This inconsistency was mine, introduced in the
1.6.0 doc rewrite. docs/drive-access.md already said `sweep` and was right — a
round-2 finding claimed the opposite on the grounds that `recovery::sweep`
appears nowhere else in this crate, which it cannot, being in another crate.

io/pipeline.rs cited `disc::patch` as WRITE_THROUGH_DEPTH's caller; that moved
to freemkv-engine in 1.6.0 and no `patch` exists here.

truehd.rs's doc on mlp_major_sync_crc_ok said the trailer is compared
big-endian while the body compares u16::from_le_bytes — and a big-endian
compare was the bug the function was fixed for, so the comment described the
defect rather than the code.

sector/decrypting.rs claimed the decorator owns "the only mutable state (its
call-count cap and spent flag)". DecryptingSectorSource has no such fields and
no KeyFetch field at all in this revision.
2026-07-29 18:50:53 -07:00
Matthew Jackson 4b13fc0a5c Apply stream selection on the live path, and treat a halt as a clean stop
Three defects around MuxOptions in the mux driver.

MuxInput::Live never applied MuxOptions.selection, so a caller's audio/subtitle
selection was silently ignored on the live-drive path while the field's own
documentation said it was applied before the demux pipeline is built. The Iso
and Session arms both apply it; Live now does the same, in the same place —
before resolve_inline_base_map, which is keyed on extents and so unaffected by
pruning the stream list.

The header gate returned Error::MkvInvalid whenever headers had not resolved.
On the prefetch-highway path a halt landing while the pump is blocked in a read
can end the stream as Ok(None) rather than Err(Halted), so the loop breaks with
headers unresolved through no fault of the data. Reporting that as MkvInvalid
tells the consumer its disc is malformed and skips the stop-preserves-staging
path that a clean completed=false triggers. The gate now re-checks halt first.

MuxOptions.selection's doc claimed it was applied without naming the one input
it is not applied to. That exception lived only in an internal comment at the
Url match arm, where a caller reading the public field docs would never see it.
It is now on the field, pointing at InputOptions::selection instead.
2026-07-29 18:48:51 -07:00
Matthew Jackson b2abff9792 Fix the DVD MPEG-audio codec mapping and the fabricated AACS docs
parse_audio_attr mapped DVD audio_coding_mode 2 to Codec::Mpeg1 — the MPEG-1
VIDEO variant. Codec::kind() reports Video for it, so a DVD MPEG-audio stream
was classified and handled as video everywhere downstream. Modes 2 and 3 are
both MPEG audio Layer II (3 adds the MPEG-2 multichannel extension), so both
map to Codec::Mp2. A test now walks every coding mode and asserts each result's
kind() is Audio, so no mode can map to a non-audio codec again.

docs/aacs.md documented an entire keydb-resolving API that does not exist:
ScanOptions::with_keydb, Disc::open_title, reader.read_unit(). None of those
symbols appear anywhere in the crate, and ScanOptions has no keydb field — its
own doc comment says "libfreemkv is lookup-free — it resolves no keys". A
reader following that page would conclude the library reads keydb.cfg, which
inverts the actual design: the caller resolves keys out-of-band through a
KeySource and applies them with Disc::decrypt_with.

The section is rewritten against the real API, and the AacsState table's
`key_source` type corrected from KeySource to KeyOrigin.

Worth recording: the first replacement example I wrote was itself wrong. It
used `input("disc://...")`, which resolve.rs explicitly rejects with
Error::DiscUrlNotDirect — live disc must go through Drive::open + Disc::scan +
DiscStream::new. Every symbol and signature in the committed example was
checked against the source rather than assumed.
2026-07-29 18:47:06 -07:00
Matthew Jackson 77b8e520f8 Stop Mp4Sink losing audio frames and writing an empty sample entry
Mp4Sink::write returned Ok(()) without recording the sample whenever an audio
track's frame would not parse into a sample entry. Two consequences, both
silent: leading audio frames were lost until one frame parsed, and a track
whose frames never parsed disappeared from the output entirely — finish()'s
retain() removed the sample-less trak and the run reported success. That
contradicts this crate's stated policy that a skipped track is never silently
dropped.

The drop was never necessary. audio_entry is read in exactly one place,
build_trak, reached only from build_moov inside finish() — nothing on the write
path consumes it. So write() now records every sample and derives the entry
opportunistically from whichever frame parses first.

That makes build_trak's `audio_entry.unwrap_or_default()` reachable, which
would emit an stsd declaring entry_count=1 around an EMPTY sample entry: a
structurally invalid mp4 returned as success. finish() therefore drops any
audio track it cannot describe, with a tracing::warn! naming the codec and
sample count.

Dropping rather than erroring is deliberate. It matches finish()'s existing
treatment of sample-less tracks, keeps an export whose video is fine from
failing outright, and needs no new error code — a new code would mean a new
i18n key across 29 locale files in another repo, which is not this change's
scope. The track's bytes stay unreferenced in mdat: wasted space in a valid
file, which is the cheaper failure.

Test pins both halves — moov describes only the video track, and the
unparseable audio bytes still reach mdat rather than being discarded at write
time.
2026-07-29 18:44:49 -07:00
Matthew Jackson 5a4631a9e1 Stop the mp4 demuxer dropping tracks silently or inventing sample offsets
Three defects in the mp4:// read path, all of the same family: a damaged
source was remuxed minus a track, or with fabricated data, and the run
reported success.

Silent drops. Eight paths dropped a whole track on malformed input with no
report of any kind, so an mp4:// source missing its audio looked like a clean
run. Each now emits a tracing::warn! naming the track and the missing or
inconsistent table (tracing English is permitted in this crate; the numeric
error codes are unchanged). The non-A/V handler case is debug!, since skipping
a timecode or hint track is normal.

Fabricated offsets. sample_offsets ended with a `while offsets.len() <
sizes.len()` loop that packed unplaced samples after the last known offset.
Those samples have no known location, so the invented offsets made the reader
pull frame data from arbitrary file bytes — the exact "emit garbage" outcome
the stco/stsc presence guards refuse. It now returns the short list and the
caller drops the track.

Short stts. `durations.get(i).unwrap_or(0)` gave every sample past the end of a
short stts a duration of 0, collapsing the whole tail onto one timestamp. That
is the same degenerate timing the `durations.is_empty()` guard was written to
refuse, so the guard now refuses both cases.

Two shared test fixtures were internally inconsistent and only passed because
the reader was lenient: stsz declared 3 samples while stsc placed 1, and the
hostile-stsz fixture's stsc/stts covered a single sample. Both are now
consistent. The hostile fixture keeps its lying stsz count — that lie is what
it tests — but its stsc and stts now cover whatever count survives the
file_len bound, so it exercises the allocation bound rather than the
inconsistency guards.

Two new tests pin the new refusals by mutating the consistent fixture: an stsc
that places 1 of 3 samples, and an stts that covers 1 of 3.
2026-07-29 18:42:38 -07:00
Matthew Jackson 69d96e3f02 Bound the forced-subtitle probe, make it cancellable, and stop re-reading clips
The probe's only natural exit was "every PGS track has shown a non-forced
display set". A genuinely FORCED track never satisfies that, so on the common
authoring — a forced-narrative track for foreign dialogue — the loop read the
title's entire extent set at 2 MiB per call with no byte cap, no time cap and
no halt check. It was also invoked once per title rather than once per distinct
clip, and a disc's playlists overwhelmingly reference the same few clips (main
feature, play-all, seamless-branch variants), so the same physical extents were
re-read 30-150 times. The two defects multiplied: tens of GB, times the
playlist count, off an optical drive.

Reached via ScanOptions::probe_forced_subtitles, whose only consumer is
`freemkv info -v` (freemkv/src/disc_info.rs). The rip path leaves it off. So
the symptom is `info -v` never returning on an ordinary UHD, not a corrupt rip.

Three changes:

  * PROBE_BUDGET_SECTORS caps a probe at 256 MiB. A forced track's display sets
    appear throughout the title, so a bounded prefix classifies it; the budget
    only decides how long we keep looking for a non-forced set before accepting
    the forced verdict.
  * ScanOptions::halt is now plumbed in and checked per chunk, so `info -v` is
    cancellable. The probe previously took no halt at all.
  * A ForcedProbeCache memoises verdicts against the title's exact extent list.
    Keying on byte-identical input means a hit cannot change any result — it
    only removes the re-read.

Verdict application is factored into apply_verdicts so the cached and freshly
probed paths cannot diverge.

Three tests pin the behaviour, each against a reader that counts sectors and
never ends: the budget stops at exactly PROBE_BUDGET_SECTORS, a cancelled halt
reads zero sectors, and a second title with identical extents costs no further
reads while a different extent list still misses the cache.

Also worth recording: the CLI acceptance harness never exercised `info -v`
against an optical drive — it reads ISOs from local SSD, where a full-extent
read is fast enough to hide both defects.
2026-07-29 18:38:25 -07:00
Matthew Jackson 230fc0b322 Give every audio and video codec its own registered Matroska CodecID
MkvTrack::audio's catch-all was `_ => CODEC_AC3`, and ebml.rs defined no
A_AAC, A_MPEG/L2, A_MPEG/L3, A_FLAC or A_OPUS constant at all. A CodecID
names the payload, so any of those codecs was written into the MKV declaring
AC-3 while carrying something else — a player either refuses the track or
decodes noise.

Reachable through two ordinary paths, both verified: ifo.rs:577 maps DVD
audio_coding_mode 3 to Codec::Mp2, so a DVD with MPEG audio muxed to mkv://
produced a track declaring A_AC3 over MP2 bytes; and mp4/read.rs:479 maps the
`mp4a` sample entry to Codec::Aac for an mp4:// source. codec/mod.rs already
has working parsers for MP2, MP3, AAC, FLAC and Opus, so the pipeline carried
these codecs end to end and only the container label was wrong.

MkvTrack::video had the same shape: `_ => CODEC_MPEG2` announced Codec::Mpeg1
and Codec::Av1 as MPEG-2 video. V_MPEG1 and V_AV1 added.

Both catch-alls stay, because MkvTrack::audio/video return Self and have no
error channel, but they are now reachable only by a non-audio/non-video or
Unknown codec routed there in error. Two tests enumerate every real codec of
each kind and cross-check each against Codec::kind(), so a codec added to the
enum later cannot silently inherit another codec's ID.

Both proven red first: Aac declared A_AC3 before the fix.
2026-07-29 18:34:53 -07:00
Matthew Jackson d79cb0d6eb Parse H.264 parameter sets with the avcC parser, not the hvcC one
TsMuxer::write_frame handed every NAL video track's codec_private to
hvcc_to_annex_b. An H.264 track carries an avcC record, whose box layout is
different, so the parser returned None and no parameter sets were emitted —
and params_written was set unconditionally, so it never retried. H.264
muxed to m2ts:// reached the player with no SPS/PPS and was undecodable,
silently: frame_count still advanced and the mux reported success. AVC is
the dominant Blu-ray video codec, so this was not an edge case.

The correct dispatch already existed and was already used by
demux_sink::annexb_param_sets. tsmux simply never got the codec: it knew
only PIDs and a NAL-or-not bool, so it could not tell hvcC from avcC.

Rather than add a second setter, set_nal_video(track, bool) becomes
set_video_codec(track, Codec). One fact decides both the ES framing and the
parameter-set parser, so the two can no longer disagree — and that
disagreement is precisely this defect. The default stays Codec::Hevc, which
is the behaviour the bool's `true` default encoded, so a caller that never
calls it is unaffected.

Proven red first, end-to-end through M2tsStream::create with a real avcC
record: before the fix neither the SPS nor the PPS reached the transport
stream.
2026-07-29 18:32:21 -07:00
Matthew Jackson 77d2198499 Stop documenting the recovery API that 1.6.0 deleted
Disc::sweep, Disc::patch, Disc::copy, SweepOptions and PatchOptions have
zero occurrences in src/ — recovery moved to freemkv-engine — but they were
still documented in 30 places across README.md, TROUBLESHOOTING.md, six
files under docs/, seven src/ doc comments and a Cargo.toml comment.
README.md is the crate's GitHub front page and carried a full multi-pass
code example that cannot compile.

Two of the src/ references were intra-doc LINKS to deleted items
([`disc::Disc::copy`], [`disc::Disc::patch`] in scsi/mod.rs). They produced
no warning on a normal `cargo doc` only because they sit on pub(crate)
items; `--document-private-items` reports both, and they are gone now.

The README example is deleted rather than rewritten against the engine's
API: libfreemkv documenting a downstream crate's API on its own front page
is the drift that produced this, and it cannot even depend on it. The src/
references become plain code spans naming freemkv_engine::recovery::* —
deliberately not links, for the same reason.

docs/rip-recovery.md was 202 lines about relocated code. It now documents
only what this crate owns — Drive::read, SenseFamily, DiscStream's adaptive
batch halving — plus the read-path design constraints, which belong with the
code that enforces them, and points at freemkv-engine/src/recovery/ for the
strategy. api-design.md's module tree is regenerated from the real src/disc/
and src/drive/ layouts instead of hand-patched; it had listed sweep.rs,
patch.rs, mapfile.rs and read_error.rs, none of which exist.

Three stale facts surfaced while rewriting and are corrected: the read
timeouts are 10 s / 60 s, not the documented 1.5 s / 30 s; Drive::reset and
SgIoTransport::reset no longer exist at all, so "no SCSI reset from any read
path" is now stated as the stronger fact it has become; and verify_title,
listed as a progress-emitting operation, was removed entirely.

CHANGELOG.md keeps its references — those are the historical record of the
releases that shipped the API.
2026-07-29 18:04:51 -07:00
Matthew Jackson 95c05d97c9 Assert ReferenceBlock per block instead of scanning for the byte 0xFB
Two tests checked only find_id(&data, ebml::REFERENCE_BLOCK).is_some().
ReferenceBlock's EBML ID is the single byte 0xFB, so that asks whether one
exists ANYWHERE in the output — not which blocks carry one. Inside a
BlockGroup, keyframe-ness is signalled by the ABSENCE of ReferenceBlock, so
"which" is the entire question.

Measured rather than assumed. A mutant emitting a ReferenceBlock on every
BlockGroup, keyframes included — which reintroduces the exact 1.6.0 defect
these tests were added for, every frame reading as a non-keyframe — passed
mvc_frame_emits_blockgroup_additional_and_reference untouched. The
duration-bearing roundtrip test did catch it, via its MkvStream read-back
rather than via the presence check. A mutant corrupting the offset VALUE
was invisible to both.

all_block_groups() parses every BlockGroup in emission order with its
Block flags, relative timestamp, duration and decoded signed ReferenceBlock.
first_block_group() now delegates to it, so there is one walker rather than
two. Both tests assert per block: the keyframe carries NO ReferenceBlock,
each non-keyframe carries one, the offset equals the distance back to the
keyframe, and the SimpleBlock-only 0x80 flag is clear throughout.

Both tests previously took muxer.writer.into_inner() WITHOUT calling
finish(), leaving cluster sizes unwritten — an unparseable file, which is
why they could only byte-scan in the first place. They now write through
SharedWriter and finish(), so the assertions run against a well-formed
Matroska file.

Verified: with the fixes in place, the every-block mutant and the
off-by-one-offset mutant both fail.
2026-07-29 18:00:06 -07:00
Matthew Jackson f40e01de4b Cover the non-NAL video path, including the wiring that selects it
set_nal_video had exactly one production caller and zero test callers. The
branch it gates decides whether a video track's ES goes through
length_prefixed_to_annex_b, and MPEG-2 and VC-1 must not: they are already
start-code ES. Getting it wrong is silent — frame_count still increments,
so the mux reports success while emitting a video-less file.

Five tests, each verified against a real mutant:

  * non-NAL ES passes through byte-for-byte, and the default path still
    converts. Both use a deliberately length-prefix-SHAPED payload so a
    wrongly-applied conversion rewrites the leading four bytes into a start
    code — a payload the converter happened to leave alone would let a
    mutant pass.
  * a non-NAL keyframe arms params_written, so following non-keyframes are
    not dropped by the pre-keyframe guard.
  * set_nal_video rejects an out-of-range track instead of panicking.
  * M2tsStream::create wires a VC-1 track to the non-NAL path.

That last one matters more than it looks. The four TsMuxer-level tests set
the flag themselves, so deleting the set_nal_video loop from
M2tsStream::create left all 2366 tests passing — the exact mutant the
finding named went undetected until a test drove the real wiring. It now
also catches the subtler mutant of widening the matches! arm to include
Vc1.
2026-07-29 17:54:27 -07:00
Matthew Jackson 5172803566 Promote AACS unit encryption to real API, and assert decrypt byte-exactly
encrypt_unit becomes public library API rather than a #[cfg(test)] helper.
Authoring an encrypted disc image is a legitimate use of this crate, and
the capability was already written four times over: a pub(crate) test-only
copy in aacs/content.rs plus three hand-rolled duplicates in decrypt.rs,
sector/decrypting.rs and disc/extract.rs. All four now call one function,
removing ~110 lines of duplicated cipher code that could drift from
decrypt_unit independently.

It mirrors decrypt_unit's purity contract: crypto only, no encrypted-flag
handling, because where that flag lives is container-specific (CPI bits in
byte 0 for BD-TS, elsewhere for HD-DVD-PS). Callers set the flag BEFORE
encrypting — bytes 0..16 are the key seed left in plaintext, so touching a
header byte afterwards changes the key a decryptor derives. That footgun is
documented at the function and at every call site.

Two tests pin it: an exact round trip through both directions, and the one
place the pair is deliberately asymmetric — decrypt_unit restores
all-zero-on-disc packets to zero, and the test proves an all-zero plaintext
packet enciphers to non-zero bytes so it is never mistaken for padding.
That asymmetry was previously only prose.

Two decrypt tests were also weaker than their own names:

  * aacs_clear_trailing_partial_passes_through asserted only is_ok(), so a
    mutant corrupting the clear partial while returning Ok passed. It now
    snapshots the buffer and asserts byte equality, matching the
    none_keys_is_noop pattern already in the file.
  * aacs_decorator_decrypts_encrypted_unit_via_map checked only that 0x47
    reappeared at the 192-byte stride, leaving corruption in the other 6112
    bytes undetected. The plaintext is fully known, so it now asserts
    byte-exact recovery against it.

Both were verified red first by mutating the production path.
2026-07-29 17:50:24 -07:00
Matthew Jackson 0c2dbdba6e Make the ddts speaker mask agree with its own channel count
The ddts box declares both a channel count and a 16-bit speaker mask, and
a decoder may trust either. dts_channel_layout ended in a `_ => 0x0007`
catch-all describing five speakers (C + L/R + Ls/Rs), so for AMODE 6, 7,
and 10 through 15 the box contradicted the count DTS_AMODE_CH declared
alongside it — provoking a downmix or an outright decode error.

AMODE 6 (L + R + S) is the reachable case: its S is a single
centre-surround, not the Ls/Rs pair, so it is three channels and 0x0012,
not four and 0x0006. AMODE 13/14/15 do not occur on retail media, which
ships a 5.1 AMODE-9 core plus an extension substream.

Replace the catch-all with the full 16-entry ETSI TS 102 114 mask table.
Every entry is cross-checked against DTS_AMODE_CH by counting the
speakers its bits imply: sixteen independent constraints, all satisfied,
and that table is itself pinned to the per-AMODE channel counts in ETSI TS 102 114.

AMODE is a 6-bit field, so the reserved 16..=63 are reachable from a
malformed stream. The old `unwrap_or(6)` invented a channel count for
them that no mask could match; parse_dts now refuses those frames rather
than guessing a layout it cannot name.

One existing fixture placed the ext-sync pattern at f[4..8], which
incidentally set f[7]=0x25 → AMODE 20. That test is about where the
pattern sits, not about AMODE, so the frame is now spec-legal (AMODE 9,
48 kHz) with the pattern moved into the payload proper.
2026-07-29 17:43:38 -07:00
Matthew Jackson 1994f42ff8 Drop the comment that promised tests this crate no longer has
The bisect ReadAction regression tests moved to freemkv-engine with the
recovery strategy, but their explanatory block stayed behind — seventeen lines
describing a `let _ = handle_read_error(..)` bug and asserting "the tests below
prove the required ReadAction values are produced". There are no tests below it;
handle_read_error is not even resolvable here any more. Anyone auditing whether
that bug is still guarded would read this and conclude yes. The block moved to
the engine alongside the tests it describes.
2026-07-29 16:36:22 -07:00
Matthew Jackson 4993f5df79 Route conduct reports through GitHub instead of a private address
The enforcement contact was an address on a domain used for internal
infrastructure, which does not belong in a public repository. GitHub needs no
mailbox to exist and is reachable for anyone who can read this file.
2026-07-29 16:07:14 -07:00
Matthew Jackson a10f43841f mux: fix keyframe signalling for BlockGroup frames
Inside a Matroska BlockGroup the SimpleBlock 0x80 keyframe bit is
reserved and is always written as 0; keyframe-ness is carried only by
the presence or absence of a ReferenceBlock child. Both halves of the
round-trip got this wrong:

  - the reader skipped past ReferenceBlock and read the reserved bit,
    so every BlockGroup frame came back as a non-keyframe;
  - write_block_group discarded its `keyframe` argument and never
    emitted a ReferenceBlock, on the assumption that only intra frames
    (PGS subtitles) reached that path.

The MPEG-2 parser stamps a per-frame duration on I, P and B pictures
alike, so all MPEG-2 video is written as a BlockGroup. That makes the
assumption false and left no video frame looking like a keyframe on
read-back. Downstream, mkv:// -> m2ts:// dropped every video frame (the
TS muxer discards non-key video until the first keyframe) while still
reporting success, and mkv:// -> mkv:// and the stdio round-trip failed
E6008, because the MKV muxer opens a cluster only on a track-0 video
keyframe and so wrote nothing at all. HEVC was unaffected: it carries no
per-frame duration, so it takes the SimpleBlock path where the flag bit
is authoritative.

Verified on a real CSS DVD: 841 keyframes out of 11440 video packets
survive a re-mux, matching the I-picture count in the source bitstream.

Also stop running non-NAL video through the Annex-B converter. MPEG-2
and VC-1 elementary streams are already start-code framed, so
length-prefix conversion corrupts them. TsMuxer takes a per-track
nal_video flag, defaulting to the previous behaviour, which M2tsStream
sets from each video stream's codec.
2026-07-29 15:29:48 -07:00
Matthew Jackson 98fe02746b mux: document that the Url arm selects via InputOptions.selection
The Url mux path prunes streams inside input() via InputOptions.selection;
MuxOptions.selection only applies on the File/Session arms. A Url-source caller
must set InputOptions.selection — noting it so a future caller does not put the
selection on MuxOptions and silently keep every track (the bug the GUI hit).
2026-07-28 18:40:47 -07:00
Matthew Jackson 99ac4268b1 1.6.0: remove recovery strategy (moved to freemkv-engine) + trim dead surface
The sweep/patch recovery strategy, mapfile, retry-decision state machine,
section-recover, and damage classification move out of libfreemkv into the
new freemkv-engine crate. libfreemkv keeps the raw single-shot read and
SCSI-fact translation (SenseFamily stays in scsi).

- Delete disc/{sweep,patch,mapfile,read_error,section_recover}.rs, the
  Disc::copy/sweep/patch methods, the Copy/Sweep/Patch option+result types,
  classify_damage/DamageSeverity, progress_snapshot_from_mapfile, and the
  three recovery integration tests.
- Trim public surface the recovery deletion orphaned: delete the dead
  READ_PIPELINE_DEPTH const, the write-side SectorSink/FileSectorSink (no
  consumer), and the DriveSpeed enum (its one live use — set max drive
  speed — becomes Drive::SPEED_MAX_KBPS). Make mapfile_path_for,
  decrypt_sectors_mapped pub(crate); gate NoopEvents to test.
- Version 1.6.0.
2026-07-28 15:35:19 -07:00
Matthew Jackson 8737c5bb65 changelog: 1.6.0 section (layering API, mux fixes, engine relocation, stream selection); date 1.5.2 2026-07-28 13:54:42 -07:00
Matthew Jackson a124be8a76 mux: end-to-end test that stream selection drops excluded-PID frames
A title declaring two audio PIDs, pruned to one via StreamSelection::apply
before build_iso_pipeline, must never surface a frame from the excluded PID.
Proves the declaration-driven seam end-to-end through the full highway (read ->
demux -> codec parse): the demuxer is built from the pruned title.streams, so
the excluded PID is untracked and skipped, and both track headers and frames
follow the pruned list. Builds on the existing synthetic-TS harness.
2026-07-28 13:36:36 -07:00
Matthew Jackson 8cc868ca54 disc: add DiscTitle audio_streams/subtitle_streams/video_streams accessors
Typed iterators over the audio/subtitle/video streams, cleaner than matching on
the Stream enum for the common iterate-the-tracks case (stream selection, the
desktop UI info panel, disc-info). Additive; all lib tests pass on 1.86.
2026-07-28 13:35:12 -07:00
Matthew Jackson e768d1666b mux: StreamSelection primitive + apply sites for per-title stream selection
The demux pipeline is declaration-driven off DiscTitle.streams (build_demux_state,
DiscStream::new, and the MKV writer all key off that list), so 'which streams to
keep' is already a pipeline capability with no public knob. This adds the knob:

- mux/select.rs: StreamSelection { audio, subtitle: PidFilter::All | Only(Vec<u16>) }
  + apply(&mut DiscTitle): keep Video always, keep Audio/Subtitle whose PID the
  filter lists, prune the rest (and the parallel codec_privates in lockstep);
  error SelectionPidUnknown on a listed PID absent from the title (fail loud, not
  a silently-missing track). Pure; 6 unit tests. Re-exported at crate root.
- Error::SelectionPidUnknown (E6014).
- MuxOptions gains  (+ derives Default now) applied in mux_stream's
  Iso/Session arms before the highway/DiscStream builds demux state (and before
  probe_and_remap's DVD AC-3 PID rewrite). InputOptions gains  applied
  in input()'s iso arm right after the title-index bounds check.

PIDs not languages -- language->PID is caller/engine policy. Default All/All is a
no-op (apply gated on !is_all()), so the no-selection path is byte-identical:
nothing below the title-finalization line changes (ts/ps/demux_thread/
pipelined_stream/mkv/disc untouched). All 2488 lib tests pass on 1.86.
2026-07-28 13:14:40 -07:00
Matthew Jackson 1f643cf91a error: add is_disc_level_no_key classifier (re-exported)
Distinguishes a WHOLE-DISC key failure (E_NO_DISC_KEY / E_KEYDB_LOAD /
E_AACS_NO_KEYS -- every title fails identically) from a per-title skippable
stub. The engine's multi-title loop uses it to fail-fast on the first no-key
title instead of iterating all N. Additive.
2026-07-28 12:45:58 -07:00
Matthew Jackson b299e5d8db error: re-export is_halt + is_skippable_title_stub at crate root
The engine's multi-title rip loop classifies per-title mux failures (halt vs
skippable stub vs hard) using these typed classifiers instead of E-code string
matching. Additive; no behavior change.
2026-07-28 12:31:40 -07:00
Matthew Jackson ffc7f93aeb disc: promote locate_ranges to pub (engine multipass reads it)
Small pub promotion + fmt one-lining. The relocated multipass progress
reporting in freemkv-engine needs locate_ranges externally. No behavior change.
2026-07-28 12:09:38 -07:00
Matthew Jackson 3c53eff257 drive: promote extract_scsi_context to pub
Small pure error->(status,sense) introspection helper the relocated
sweep/patch will need externally. Same category as the prior WritebackFile/
resolve_content_key_map promotions -- infra, not policy. No behavior change.
2026-07-28 11:41:49 -07:00
Matthew Jackson f0dca35a91 io: promote WritebackFile to pub
Bounded-cache buffered File replacement used across mux/extract/sweep/patch
-- general I/O infrastructure, not recovery policy. freemkv-engine's
relocated sweep/patch need to construct it directly once those methods
leave this crate. No behavior change.
2026-07-28 11:36:43 -07:00
Matthew Jackson 4e0c0efbc3 scsi: promote SenseFamily to a lib-level SCSI-fact primitive
Moved SenseFamily::from_sense_key + is_wedge_family from disc/read_error.rs
into scsi/mod.rs (with its own tests) and re-exported at the crate root.
This is pure SCSI sense-code classification -- objective hardware fact, zero
recovery-policy opinion -- so it belongs in the library primitives, unlike
the retry-DECISION state machine (ReadCtx/PassSummary/ReadAction/
handle_read_error) built on top of it, which is freemkv's specific recovery
strategy and is moving to freemkv-engine next.

disc/read_error.rs and disc/section_recover.rs now import SenseFamily from
crate::scsi instead of defining/re-exporting their own copy. No behavior
change. Precommit green on Rust 1.86 (fmt+clippy+test).
2026-07-28 11:32:13 -07:00
Matthew Jackson ac51108e95 disc: promote resolve_content_key_map + encrypted_content_ranges to pub
The upcoming freemkv-engine crate needs both to build the multipass
sweep/patch recovery strategy externally over Disc's public API. Everything
else sweep/patch touch on Disc was already pub; these were the only two
gaps. No behavior change -- visibility only.
2026-07-28 11:26:37 -07:00
Matthew Jackson 8c0eb665b3 mux: fix DTS core-header false-drops + close TrueHD/mux gate coverage
DTS core decodability gate (core_header_drop_reason) — full ETSI TS 102 114
spec-conformance sweep against a reference decoder the spec core-header rules and
a reference decoder parse_frame_header:

- deficit_samples: only require ==32 for NORMAL frames (FTYPE==1). A
  TERMINATION frame (FTYPE==0, the last frame of a stream) legitimately
  carries fewer and is fully decodable; the old unconditional check dropped
  it on every stream that ends on one — a guaranteed per-track silence gap.
  Matches a reference decoder (normal_frame && deficit != DCA_PCMBLOCK_SAMPLES) and
  a reference decoder (branches on normal_frame).
- reserved bit (after RATE): both reference decoders SKIP it (a reference decoder
  skip_bits1, a reference decoder bits_skip1 "Reserved field") and never reject on it.
  Rejecting was a false-drop that silenced any real stream whose encoder
  set the bit. Relaxed to read-and-discard; DropReason::ReservedBit removed.

Swept and confirmed spec-correct as-is (no change): npcmblocks multiple-of-8,
frame_size>=96, audio_mode>=16 (a reference decoder-permissive), sample-rate validity
table (matches avpriv_dca_sample_rates incl 96k/192k at 14/15), LFE flag==3
invalid, PCMR bits table (matches a reference decoder sample_res {16,16,20,20,0,24,24,0}).
Bit-read order verified field-by-field against a reference decoder. bit_rate is left
unvalidated (lenient, never-false-drop direction) as before.

Tests: termination frame with small deficit is kept; normal frame with bad
deficit is dropped; reserved-bit-set frame is kept. make_bad_dts_core now
uses an invalid LFE flag (duration-neutral) instead of the relaxed reserved
bit.

TrueHD: add coverage for the EXTENDED major-sync header CRC path (ms[25]&1,
mshdr=28+2+2n) — previously zero-tested, the exact path a shipped endianness
bug once used to silently drop whole 7.1/Atmos tracks. Trailer is an
independently-computed oracle (separate CRC-16/0x2D, anchored to the 0x4FF7
catalogue value, NOT crc16_mlp), stored little-endian; test asserts accept,
body-corruption reject, and big-endian-trailer reject.

mux driver: extract the finish completion mapping into pure mux_run_completed
so the finalize_failed -> completed=false branch (reachable only via real
write-thread wedge timing) is unit-tested; add an out-of-range
MuxInput::Session title_index test asserting a clean Error::MuxTrackRange
(E9011) instead of a panic.
2026-07-24 10:28:21 -07:00
Matthew Jackson 5e33fa4771 Fix read-fault misclassification, DTS AMODE channel table, and untestable guards
- resolve_fmts_key_map: distinguish a genuinely-not-FMTS disc from a
  transient live-drive read fault. read_filesystem now returns the new
  Error::UdfNotFilesystem for a deterministic tag/format mismatch (no AVDP,
  no partition descriptor, no FSD); resolve maps only UdfNotFilesystem (fs)
  and UdfNotFound (.tbl absent) to Ok(None), and PROPAGATES DiscRead / other
  I/O faults so a marginal AACS 2.1 disc fails loud instead of silently
  dropping forensic content under a base-Unit-Key-only map.

- DTS_AMODE_CH (mp4/audio.rs): extend 10→16 entries
  {1,2,2,2,2,3,3,4,4,5,6,6,6,7,8,8} (the spec per-AMODE channel table / ETSI TS 102 114) so
  the spec-legal high AMODEs that now pass the decodability gate declare
  their true channelcount (AMODE 13→7, 14/15→8) instead of a truncated 6.

- session.rs resolve_keys "called before scan" guard is now testable:
  from_parts_for_test takes Option<Disc>; added a test that a disc-less
  session returns a clean DeviceNotReady Err rather than panicking.

- mp4/read.rs: a track with samples but a missing/malformed stts (mandatory
  per ISO/IEC 14496-12) is dropped rather than emitting all-zero timestamps,
  matching the existing stco/stsc guards; all-tracks-dropped → Mp4Invalid.

- Remove the inert MuxInput::Iso.key_map field (the Iso path re-derives its
  map inside build_iso_pipeline); the live path keeps Live.key_map.

All four fixes are mutation-verified.
2026-07-24 09:59:33 -07:00
Matthew Jackson a7b16e3a1e mux: distinguish FMTS phase-probe read fault from wrong key; cover multi-CPS
Fix 1 (correctness): resolve_fmts_key_map's per-index phase probe read a
single representative segment via read_unit (whose read_sectors(...).ok()?
swallows read errors into None) with no fault fallback. A transient live-drive
read fault (e.g. NOT READY 2/04/3E on the BU40N) made every probe read return
None, giving even==0 && odd==0 — indistinguishable from a genuine wrong key —
so resolve_tie_phase returned FmtsKeyMissing and aborted the entire multi-title
rip even though the forensic index keys were valid and in hand.

Extract the probe into probe_index_phase, which returns Phase / WrongKey /
ReadFault. It mirrors the anchor loop's tolerance: it tries multiple same-index
segments and only concludes WrongKey once a read actually succeeded and decrypted
to no clean parity. If every read of every same-index segment faults it returns
ReadFault; the caller then leaves the phase unresolved so the range-builder
defaults to Phase::All (decrypt both parities, demux drops the garbled alternate
half) instead of aborting. A wrong key can never be masked as a read fault:
ReadFault requires that not a single read succeeded, so zero decrypt evidence.

Fix 2 (test coverage): resolve_mux_key_map's multi-CPS branch was only exercised
with an all-zero source, so pick(), the KeyFetch cold path, and the fail-loud
DecryptFailed guard never ran. Add tests over real AACS ciphertext (via
aacs_encrypt_unit_for_test) covering pick() selecting the correct pool index,
the DecryptFailed guard firing when a clean sample matches no held/fetched key,
and the fetch cold path recovering a missing unit key. Mutation-verified.

Fix 3 (doc): move the reader_event_fn EventKind->MuxEvents mapping doc off
session_mux_keys onto reader_event_fn.
2026-07-24 09:28:09 -07:00
Matthew Jackson 298262f6b3 mux: thread halt into live AACS key-map resolution; cover Session arm
Round-2 follow-ups to 6d6e60f (inline base-map resolve on the live
single-pass Session/Live mux arms).

Fix 1 (halt threading) — the inline resolve chain sampled ciphertext off
the LIVE drive with no cancel token, so an operator /api/stop during key
resolution was not honored (the FMTS probe can issue hundreds of reads,
each able to stall to the 60s SCSI recovery timeout — violating the
"don't hammer a struggling live drive" rule). Add an optional
`halt: Option<&Halt>` to `resolve_mux_key_map`, `resolve_fmts_key_map`,
`resolve_inline_base_map`, and `Disc::resolve_content_key_map`, and poll
it at each loop boundary (FMTS anchor + per-index probe loops, multi-CPS
extent loop) — returning Err(Halted) promptly. Live/Session arms pass the
driver's halt; sweep/patch pass their own token (via Halt::from_arc);
file-backed probe/ISO callers pass None. Tested with a pre-cancelled halt
(Err Halted, no extent sampling) and a None-halt no-abort case;
mutation-verified (dropping the extent-loop check → Ok, not Err).

Fix 2 (Session-arm coverage) — the MuxInput::Session arm ran the same
resolve→install→decrypt sequence as Live but had NO end-to-end test
(DiscSession only exposed open(), which needs live hardware). Add a
#[cfg(test)] DiscSession::from_parts_for_test (injected reader + scanned
disc, no Drive), an end-to-end AACS decrypt test through the Session arm
(mutation-verified: dropping with_key_map → mux aborts), and a
missing-reader clean-error (not panic) test.

Fix 3 (cleanups) — io_error_code: remove the unreachable typed-Error
downcast branch (From<Error> for io::Error stringifies; no path builds an
io::Error holding a typed Error), keeping the stringify parse is_halt /
is_skippable_title_stub rely on. Add a resolve_keys_for test covering the
largest-title sampling branch. Document the patch wedge-exit coverage gap
(TODO) in passn_handler_ab.rs.
2026-07-24 09:04:19 -07:00
Matthew Jackson 2999daf1c9 mux: resolve+install AACS key map on live single-pass (Session/Live)
Under the map-only decrypt model an AACS DecryptingSectorSource decrypts
nothing until a key map is installed; with no map the AACS arm fails loud
with DecryptFailed on the first content unit. The two inline live-mux arms
in mux_stream did not install one:

  - MuxInput::Session (freemkv `rip disc://…mkv`) installed NO map at all.
  - MuxInput::Live (autorip non-FMTS single-pass) installed only a
    caller-supplied forensic FMTS map, which is None for a plain AACS disc.

So EVERY plain AACS Blu-ray/UHD ripped via the live single-pass path failed
DecryptFailed on the first content read. This predates the mux_stream
refactor: the bug was introduced with the map-only decrypt model, and the
pre-refactor CLI likewise built DiscStream::new without with_key_map.

Fix: add resolve_inline_base_map, the inline counterpart to what
build_iso_pipeline does for the file highway. Both arms now resolve the
AACS map off the reader (borrow to sample, then move into DiscStream) and
install it via with_key_map before any read. DVD/CSS keeps DecryptKeys::None
(DiscStream's per-title CSS crack owns it); clear/raw resolve to no map.
Session passes session.key_fetch() so a multi-CPS/orphan unit can still be
recovered; a caller-supplied FMTS map (autorip) is used verbatim, never
re-resolved.

Tests: an end-to-end MuxInput::Live mux over a genuinely-AACS-encrypted
synthetic unit now decrypts and finalises (mutation-verified: dropping the
resolve/install makes the mux abort). Adds a pub(crate) test-only AACS
encrypt helper so the mux test can build a real encrypted fixture, and a
gating test for resolve_inline_base_map (AACS→map, CSS/clear/raw→none).
2026-07-24 08:34:51 -07:00
Matthew Jackson 31a87ae994 Fix audit findings: DTS AMODE bound, key-fetch negative memoization, PGS probe coverage
- dts: accept all 16 legal AMODE channel-arrangement codes (0-15), not just
  0-9. Per ETSI TS 102 114 the 6-bit AMODE field has 16 defined arrangements;
  only 16-63 are reserved. a reference decoder the spec per-AMODE channel table confirms 10-15 are
  decodable 6/7/8-channel layouts. The old bound of 10 dropped spec-legal
  multichannel core frames as undecodable, silencing recoverable audio. Add a
  regression test (literal 0..16 range) that fails if the bound reverts to 10.

- keysource: only memoize a NEGATIVE (empty) key-fetch result when every source
  genuinely ran and none held the key — never when a source Err'd (network down,
  unreachable). A transient outage was being cached as a permanent "no key" for
  the fingerprint, permanently dropping a unit that could be recovered once the
  source came back. Thread an `errored` flag out of the drivers and gate the
  cache insert on it. Tests cover both the recover-after-outage case and that a
  genuine absence is still memoized.

- pgs_forced_probe: add happy-path coverage feeding real synthetic BD-TS PGS
  display sets through the full demux -> parse -> observe -> apply path, both a
  forced verdict landing and a non-forced verdict clearing a vendor flag.

- mp4: correct fit_report doc (audio carried is AC-3/E-AC-3 AND DTS/DTS-HD).

- scan_iso test: add independent fixture expectations (volume id) so the parity
  test is no longer purely tautological against a re-run of the same composition.
2026-07-24 08:32:37 -07:00
Matthew Jackson 6be489a10a mux/driver: map a mid-read halt to completed=false (Stop is not a failure)
The finish stage and the highway path already treat Error::Halted as a clean
operator stop (completed=false), but the header pump (`stream.read()?`) and the
frame pump (`Err(e) => return Err(e)`) propagated it as a hard error. Since slow
recovery reads dominate wall-clock, a Stop almost always lands mid-read, so it
surfaced as a mux failure instead of a resumable incomplete outcome — breaking the
driver's own "a clean operator stop is not an error" contract and diverging from
the ISO/Url highway (which returns Ok(None) on halt).

Both read arms now route Error::Halted to the completed=false path. Adds
error::is_halt(&io::Error) (typed, mirrors is_skippable_title_stub) as the check.
Two regression tests cover a halt landing mid-header-read and mid-frame-read.
2026-07-24 07:52:42 -07:00
Matthew Jackson 922a9d4403 mux: make write-pipeline send deadline per-call via MuxOptions
The 60s per-frame send deadline the mux driver applies in drive_mux was
hardcoded. That is correct for autorip (its hard watchdog + container-restart
model wants a wedged sink surfaced as a bounded per-frame timeout), but wrong
for the CLI's interactive stdout:// / network sinks: the old inline
output.write() had no deadline, so a slow-but-alive downstream (paused pager,
backpressured pipe, slow peer) was spuriously reported as an interrupted mux
after 60s of backpressure.

Add MuxOptions.send_deadline: Option<Duration>. Some(d) keeps a hard d timeout;
None means no backpressure timeout (block on a live-but-slow sink). None is
resolved to an effectively-unbounded-but-Instant-safe NO_SEND_DEADLINE at the
effective_send_deadline seam. Ctrl-C / halt stays responsive under None because
send_with_halt slices its wait at POLL_INTERVAL and re-checks the halt token
every slice regardless of the deadline value.

Unit-test the routing seam (Some(d) -> d, None -> unbounded, never collapses to
60s) and that MuxOptions carries the knob.
2026-07-24 02:38:30 -07:00
Matthew Jackson b70da49f7f mux: restore three drive_mux regressions from the mux hoist
The mux drive loop hoisted out of autorip/CLI into libfreemkv's
drive_mux dropped three guarantees the hand-rolled loops enforced:

A. Header-buffer OOM cap. The header pump buffered every PES frame
   until codec_private resolved with no byte accounting, so a title
   whose video header never resolves (damaged/undecryptable HEVC on
   marginal UHD) buffered the whole 30-90 GB title into RAM until
   OOM-kill. Restore the 512 MiB HEADER_BUFFER_CAP_BYTES cap autorip's
   run_mux enforced; once exceeded, fail fast with the same
   Error::MkvInvalid the headers-never-resolved gate already returns.

B. Watchdog feed during the header-frame drain. After headers resolve
   the buffered header frames are flushed to the sink, but
   on_write_progress (the sole feed for autorip's hard watchdog) fired
   only in the steady-state loop, leaving the watchdog unfed during a
   long/slow drain and able to false-escalate. Feed on_write_progress
   per buffered frame during the drain, matching the steady-state path.

C. Per-title CSS key on MuxInput::Session. The Session arm set keys via
   disc.decrypt_keys() unconditionally, which returns the largest
   title's VTS CSS key; muxing a bonus title in a different VTS
   descrambled with the wrong key (DiscStream's per-title crack only
   fires on DecryptKeys::None). Special-case DVD to None so the pipeline
   cracks the correct per-title key, matching resolve.rs::input(). AACS
   and clear discs still resolve from decrypt_keys().

Adds mutation-verified synthetic tests for each (no live drive).
2026-07-24 02:27:57 -07:00
Matthew Jackson c52240b81f mux: add MuxInput::Live for the inline live-drive single-pass path
STEP 4c-ii (layering refactor): give `mux_stream` a live-drive source so
autorip's live single-pass mux stops hand-rolling `DiscStream::new`.

`MuxInput::Live` is the live analogue of `MuxInput::Iso`: it wraps the
raw `Box<dyn SectorSource>` in the INLINE `DiscStream` (the same
constructor the `Session` arm uses — NOT `build_iso_pipeline`, the
prefetch highway), so a consumer's adaptive batch-retry in
`DiscStream::fill_extents` still fires on a bad live-drive sector. Unlike
`Session`, `Live` carries an optional pre-resolved forensic `AacsKeyMap`
that is applied via `DiscStream::with_key_map` before any read — required
for single-pass FMTS correctness (read only our-phase units; decrypt the
forensic segment with the mapped key). The consumer resolves keys as its
own policy and hands the banked material in; the driver re-resolves
nothing.

Unit test drives `MuxInput::Live` with a recording `SectorSource` and a
phased forensic key map, asserting the alternate-phase (odd) units are
never fetched — proof the arm builds the inline `DiscStream` and applies
the key map. Mutation-verified: dropping the `with_key_map` call reads the
dropped odd-phase LBAs and fails the assertion.
2026-07-24 01:53:36 -07:00
Matthew Jackson 16663841d0 mux/driver: split on_progress; stage live drive as session reader
Split the ambiguous MuxEvents::on_progress(bytes, total) into two
callbacks — on_read_progress(bytes_read, total) and
on_write_progress(bytes_written, total) — so a consumer no longer has
to guess which side of the pipeline a progress figure came from. The
CLI drives its bar from the write side; autorip from the read side.
The reader EventFn's BytesRead now maps to on_read_progress; the
per-frame emit in drive_mux to on_write_progress. Both keep empty
defaults; NoopEvents and the driver tests are updated to match.

Add DiscSession::stage_drive_as_reader so the live single-pass path can
run through MuxInput::Session: the owned Drive (itself a SectorSource)
moves into the reader slot for mux_stream to take. The drive is now
held as Option<Drive> with the device path cached up front, so the
mux driver can still name the device after the drive has been staged;
the driver's Session arm uses the new device_path() accessor.
2026-07-24 01:01:34 -07:00
Matthew Jackson 5670e77def mux/driver: forward reader-side events through Arc<dyn MuxEvents>
mux_stream took events as &dyn MuxEvents and passed None for the
constructors' EventFn, because that hook is Box<dyn Fn(Event)+Send+
'static> and a borrowed handle cannot satisfy 'static. The reader-side
events (BytesRead progress, SectorSkipped, BatchSizeChanged, ReadError)
therefore never fired.

Take events as Arc<dyn MuxEvents> (MuxEvents: Send+Sync+'static) and
clone it into a 'static EventFn (reader_event_fn) that translates each
real EventKind into the matching MuxEvents call. Wire it into
build_iso_pipeline (ISO path) and DiscStream::on_event (live path).
The write-side on_progress in drive_mux is unchanged.

Add a translation unit test (all four variants) and an end-to-end
mux_stream ISO test that asserts the read-side BytesRead reaches the
Arc; passing None regresses the latter.
2026-07-24 00:37:03 -07:00
Matthew Jackson 0b44020d08 mux: add high-level mux_stream driver (layering step 4a)
Add the shared decrypt+mux driver both consumers hand-roll today, as a
library-only API. `mux_stream(input, dest_url, opts, halt, events)` runs
the construct -> headers gate -> open sink -> pump -> finish pipeline:

- MuxInput::{Session, Iso, Url} selects the source. The file/ISO path
  calls the untouched build_iso_pipeline highway (zero added copies —
  the driver reads frames exactly where consumers call stream.read());
  the live path builds a DiscStream; a URL source goes through input().
- chapters:// / json:// metadata sinks short-circuit BEFORE the header
  pump/gate. They write their whole file from the scanned title and need
  no codec headers; the CLI placed this after the gate, so a metadata
  export on a title whose video headers never resolved failed with
  MkvInvalid. Fixed by construction.
- The header gate refuses a stream with no resolved codec_private
  (MkvInvalid); the zero-output gate refuses an empty/undecryptable drain
  (NoStreams); a halt mid-pump yields completed=false, never a success.
- Frames are written through a WRITE_PIPELINE_DEPTH consumer pipeline so
  the latency-bound sink write overlaps the next read.

Add error::is_skippable_title_stub(&io::Error) so consumers can drop the
E7023/E6008 string-match, and DiscSession::take_reader for the live arm.

Driver body unit-tested in isolation via a synthetic Stream against
null:// / chapters:// / json:// sinks (short-circuit, header gate,
zero-output gate, halt, happy path); each gate mutation-verified.
Consumers are NOT migrated yet (steps 4b/4c).
2026-07-24 00:27:47 -07:00
Matthew Jackson 4c73256bdd Add DiscSession::resolve_keys and resolve_keys_for for base AACS keys
Hoist the AACS base-unit-key resolution glue the CLI and autorip hand-rolled
around the existing primitives into one place: sample the largest title's
ciphertext, run the ordered key sources first-valid-wins, bank the winning
unit keys onto the disc, and build the read-time KeyFetch.

- resolve_keys_for(reader, disc, sources): the free-function core, over any
  SectorSource (live drive or a scan_iso file reader). Returns the structured
  ResolutionTrace plus the KeyFetch; a non-AACS disc is a no-op (empty trace,
  no fetch).
- DiscSession::resolve_keys(sources): samples through the session's reader
  (staged file reader if present, else the live drive), banks onto the
  scanned disc, and retains the KeyFetch on the session (key_fetch()
  accessor) for a later mux.
- KeySourceFactory: the Arc source factory the consumer supplies (libfreemkv
  builds no key sources itself).

Sampling is skipped when the factory yields no sources (resolution is a miss
regardless) — no wasted disc read. Tests cover banking, the no-key path
still building a fetch, and the CSS/None non-AACS no-op.
2026-07-24 00:09:28 -07:00
Matthew Jackson 925f4d56f6 Add scan_iso entry point for file-backed ISO scans
Introduce libfreemkv::scan_iso(path, opts) -> (Disc, Box<dyn SectorSource>),
the file-backed counterpart to DiscSession::scan. It is the single place
that opens a FileSectorSource, reads its capacity, and runs Disc::scan_image,
returning the scanned Disc plus a reusable reader over the same image so
consumers stop hand-rolling that triple.

Add an integration test that materialises a minimal synthetic UDF image to a
real file, asserts scan_iso matches the manual open+scan_image composition,
and confirms the returned reader is still usable (capacity + sector read).
Also covers open-failure and scan-failure error propagation.
2026-07-23 23:48:50 -07:00
Matthew Jackson 4ba2c728b2 libfreemkv: add DiscSession (drive open/bring-up hoist, step 1)
New src/session.rs providing DiscSession, DeviceTarget, and KeySpec,
re-exported from lib.rs. DiscSession::open runs the drive open +
advisory wait_ready/init/probe_disc bring-up (owning the Drive by
value); identify()/scan() are split so consumers keep their own
UI/TMDB sequencing. KeySpec carries consumer-built credentials +
key_sources which scan() forwards into ScanOptions without clobbering
caller-set fields (the library derives no certs and reads no keydb).
Unit tests cover the KeySpec->ScanOptions forwarding.
2026-07-23 23:37:20 -07:00
Matthew Jackson 6f2e9a74bd libfreemkv: 10-phase release audit fixes (v1.5.2..HEAD)
Multi-round audit of the decrypt/AACS/mux-codec refactor. Fixes, in
descending severity:

- mux/mp4/read.rs: bound untrusted-input allocations. `sample_budget`
  now also capped by file_len (a fixed-size stsz claiming count=u32::MAX
  can't inflate the Vec<SampleRef> past the file's own size); trak scan
  capped at MAX_TRACKS matches; find_box() takes only the first match
  (cap=1) instead of materializing every match. Removes dead find_boxes
  wrapper.
- disc/mod.rs: merge_content_key_ranges now UNIONS same-key overlapping
  ranges (coverage-preserving) instead of dropping the non-overlapping
  tail, which silently left encrypted LBAs uncovered -> ciphertext
  passthrough in the whole-disc sweep/patch map. Different-key overlap
  (malformed) still dropped to keep the set disjoint.
- sector/decrypting.rs: remove dead unit_key_idx field + with_unit_key_idx
  setter (vestigial from the pre-keymap trial-decrypt design; AACS is
  map-only now). Fix stale docs.
- decrypt.rs / resolve.rs / error.rs / extract.rs: doc/comment drift from
  the refactor (AacsKeyMap positive-map semantics, resolve_mux_key_map doc
  reattachment, decrypt_sectors_in_content legacy-alias, E_MP4_INVALID
  meaning, multi-CPS orphan by-design note).

Test coverage (all mutation-verified real):
- DTS NeedMore force-flush buffer bound; FLAC/MPEG-audio PTS carry-forward;
  mp4 mdhd timescale=0 divide-by-zero guard, MAX_TRACKS cap, sample-count
  file_len bound, MAX_ALLOC_BYTES cap under inflated file_len.
- resolve_fmts_key_map: extracted filter_addressable_segments,
  resolve_tie_phase, fill_base_key_gaps as pure behavior-preserving
  helpers, each unit-tested (segment filter, phase-tie arms, gap-fill
  gaplessness over every extent).
2026-07-23 22:09:04 -07:00
Matthew Jackson 071b65a64e changelog: document 1.5.2 TrueHD/FMTS/extract/trailing-partial fixes 2026-07-23 16:09:03 -07:00
Matthew Jackson b943f7511c Cache AacsKeyMap key indices; extract + test whole-disc range merge
- AacsKeyMap now derives its distinct key-index set once at construction
  (from_ranges_phased) instead of re-allocating/sorting it on every
  decrypt batch; key_indices() returns the cached slice.
- Extract the whole-disc content-map range merge out of
  resolve_content_key_map into merge_content_key_ranges and cover it:
  sort/disjoint, shared-clip dedup, overlap drop, adjacent-kept.
2026-07-23 16:00:00 -07:00
Matthew Jackson 8d730f3d27 Test hygiene: real AACS decrypt coverage, drop dead recovery fixtures
- Repoint three tautological AACS tests (which passed only via the
  unconditional-Err AACS arm of decrypt_sectors) at the real shipping
  path decrypt_sectors_mapped: out-of-range mapped key index, empty pool
  vs a non-empty map, and a scrambled encrypted trailing partial (the new
  guard) — plus a clear-trailing-partial pass-through and an explicit
  fail-loud safety-net test for AACS reaching the unmapped wrapper.
- Add the missing end-to-end AACS round-trip through the decorator +
  key map (aacs_decorator_decrypts_encrypted_unit_via_map) and a
  mapless-AACS-fails-loud decorator test — the exact class of bug the
  TrueHD probe shipped. Fills the // TODO: AACS round-trip test gap.
- Delete the now-dead recovery test fixtures (encrypt_aacs_unit_bad,
  AnyLbaUnit) left after the reactive key-fetch path was removed; keep
  encrypt_aacs_unit + FixedUnit, now used by the round-trip test.
2026-07-23 15:45:17 -07:00
Matthew Jackson 6e15dbaec7 Fix FMTS per-title resolve, extract multi-CPS keying, trailing-partial guard
- resolve_fmts_key_map: filter segments to those addressable within THIS
  title's extents; a title with no forensic content (menu/extras playlist,
  or a different clip) returns Ok(None) and takes the base Unit-Key/CPS
  path instead of hard-failing FmtsKeyMissing. Previously the first
  non-forensic title aborted the entire whole-disc sweep
  (resolve_content_key_map iterates every title) and blocked muxing any
  non-main title.
- FMTS phase probe: an even/odd is_clean tie now only fails loud when
  BOTH halves are 0 (no clean decrypt). A both-clean tie is source-zero
  padding (is_clean is true for any key on all-zero content) — the key is
  valid, default Even, never abort the rip on a padding-heavy sample.
- extract_tree: multi-CPS discs now build the exact per-CPS content map
  (resolve_content_key_map) instead of a blanket key-0 map that silently
  mis-decrypted every secondary-CPS file into garbage. Single-CPS keeps
  the blanket key-0 map (one key opens every unit, incl. orphan clips).
- decrypt_sectors_mapped: a trailing partial unit that is inside a mapped
  range AND flagged encrypted in its clear seed now fails loud (a CBC
  fragment split across a boundary can't be decrypted) instead of being
  emitted as clear. New aacs_unit_seed_encrypted reads the flag on a
  partial.
- Correct the stale decrypt_sectors doc (AACS arm now always errors;
  AACS decrypts only via decrypt_sectors_mapped).
2026-07-23 15:36:15 -07:00
Matthew Jackson 2cf42ec9f3 Fix TrueHD channel-correction probe on AACS discs; delete dead reactive key-fetch path
The mux's TrueHD 7.1/Atmos channel-correction probe built its
DecryptingSectorSource with no key map. Since AACS now decrypts only via
the resolved map, the mapless probe read failed loud on the first unit
and the correction was silently skipped on every AACS disc — 7.1/Atmos
stayed understated as the MPLS-declared 5.1. The probe now resolves the
same up-front key map the mux read installs (non-fatal on failure).

Delete the reactive per-read key-fetch recovery path
(DecryptingSectorSource::with_key_fetch, the recovery field/cipher
scratch, and sector/recovery.rs). It was unreachable: AACS resolves its
full key map up front, so a mapless AACS read is a bug, not a
recover-mid-read case. KeyFetch itself stays — it is the up-front
resolver fetch used by resolve_mux_key_map.
2026-07-23 14:03:08 -07:00
Matthew Jackson 63334dcc8b Decrypt is keymap-only: sweep/patch/extract, no AACS trial-decrypt
Every AACS decrypt now goes through the resolved key map (decrypt_sectors_
mapped): the map keys each content unit up front and a missing key fails at
resolve time. The old trial-decrypt path — try each held key per unit, keep
the first-tried plaintext on a miss — is gone; decrypt_sectors_impl's AACS
arm now fails loud (reaching it means a reader was built without its map,
which would silently apply a wrong key). CSS (self-descramble) and the clear
no-op path are unchanged.

Disc::sweep and Disc::patch resolve a whole-disc key map up front for a
decrypting pass (the fetch secures any missing CPS-unit key, fail-loud) and
decrypt via the map — clear nav/filesystem sectors are in no range and pass
through, so the separate content-range gate and the reactive per-unit
key-fetch recovery are no longer needed. extract_tree keys every unit with
the base Unit Key through the map (its encrypted-flag gate skips clear
files). Multipass sweeps stay --raw.

Removes the obsolete non-mapped-AACS trial/gate/recovery tests (the mapped
path and resolve fail-loud are tested directly).
2026-07-23 13:24:36 -07:00
Matthew Jackson 07c0a5300a AacsKeyMap: a positive map — no key for a sector means pass through
The key map is now purely "these sectors use this key": entry_for returns
Option and an LBA in no range is left untouched (no default-decrypt-
everything fallback). resolve_mux_key_map builds explicit content ranges
for every case — single-CPS keys each content extent, multi-CPS keys each
extent with the key that opens it, and FMTS fills the non-segment content
with the base Unit Key so a whole-disc read decrypts content and passes
nav/filesystem through. Combined with the fail-loud resolve, a map can
never silently apply a wrong key, and clear sectors are never scrambled.

decrypt_sectors_mapped skips a unit with no map entry; read_plan keeps an
unmapped unit (pass-through content) and drops only alternate-phase
forensic units. Tests updated to the Option semantics.
2026-07-23 12:58:35 -07:00
Matthew Jackson 8cac6111a8 Harden mux + decrypt paths; fail-loud on unresolvable keys
mp4 demuxer (untrusted input): bound every allocation sized from a box
field (stsz/stco/stsc counts, stts/ctts run-lengths, per-sample and moov
sizes, plus an absolute cap so a sparse file can't inflate file_len);
guard the parse_stsd slice and a zero mdhd timescale; cap track count so
the per-track PID can't overflow; rewrite read_moov to handle size==0 /
size<8 / 64-bit largesize; parse esds/AudioSpecificConfig for AAC; write
tkhd duration in the movie timescale.

decrypt: resolve_mux_key_map now fails loud on an extent no key can
classify instead of inheriting the previous extent's key, so a keymap
never silently carries a wrong key; the sweep/patch key-fetch recovery
fails loud when a unit is still unresolved after the retry.

AACS: reject inverted forensic segments in both range builders; compare
the forensic index in u16 space so an out-of-range value can't truncate
onto a valid u8 index. RECOVERED_ERROR no longer latches the damage zone,
preserving the 30s wedge cooldown for a following hard error.

audio: AAC/MP2/MP3/FLAC carry the last PTS across a PES with no timestamp;
the DTS-HD extension-sync search is bounded to after the core; the MP4
16.16 sample-rate field saturates. demux_sink records the video reference
before the kind filter so audio:// / sub:// keep multi-clip PTS continuity
and the DELAY tag.

Remove a dead error variant and the AACS-unsupported-video code; codec
comments cite the primary format specs; assorted doc/naming fixes and
regression tests throughout.
2026-07-23 12:02:43 -07:00
Matthew Jackson 9c144d1980 restore freemkv-unlock path dep for local dev (post-v1.5.2) 2026-07-22 22:19:42 -07:00
Matthew Jackson b2ebae8f81 v1.5.2: bump version (freemkv-unlock git-pinned for the tag) 2026-07-22 22:19:39 -07:00
Matthew Jackson b2f35a8afc CSS DVD: resolve the per-title key at read time, drop the scan-time crack
Every DVD read path — the file-backed mux highway (build_iso_pipeline) and
the live-drive single-pass DiscStream — now resolves the per-VTS CSS title
key through one shared step, css::resolve_dvd_title_key, cracked keylessly in
playback order from the title's own extents. Removes the earlier design that
reused a single scan-time key (meaningless for a per-VTS scheme) and muxed a
detection-miss disc's scrambled sectors as garbage.

- Disc::scan no longer cracks a key up front; it does only the CSS bus-auth
  read-unlock, hoisted before the UDF prefetch so scrambled small/menu VOBs
  no longer cost a rejected read each (CSS-DVD scan ~25s -> ~6s).
- An uncrackable title hard-fails (E7023) instead of passing ciphertext as
  plaintext; --raw skips the crack entirely; a Stop mid-crack surfaces Halted.
- DiscStream::new is now fallible and threads raw + halt.
- Fix a stale codec-parser doc claim (TrueHD/FLAC/MP2/AAC do gate via DropTally).
2026-07-22 22:15:27 -07:00
Matthew Jackson a6a4a5d7d2 libfreemkv: 1.5.2 changelog reflects the shipped playback-order crack
The prior wording described the round-1 'self-cracking per-sector' approach
that the audit replaced; rewrite it to match what ships (per-title crack in
playback order + scan-key reuse + loud hard-fail on an uncrackable title).
2026-07-22 13:48:56 -07:00
Matthew Jackson 0de6b82a37 libfreemkv: restore freemkv-unlock path dep on the branch tip
The last release left this pinned to the git tag v1.5.1; release.sh expects
a committed path dep on the branch tip (it swaps path -> git tag only in the
tagged commit, matching bdemu). Restores that state so a local full-workspace
build links the sibling freemkv-unlock, and release.sh's path->tag swap works.
2026-07-22 13:43:32 -07:00
Matthew Jackson b9b0a87328 CSS DVD mux: per-title key via scan-key reuse + playback-order crack
A CSS DVD whose main title was mis-detected as unencrypted (the up-front
crack scanned the largest cell first and starved its budget in that cell's
clear prefix) muxed scrambled sectors as plaintext at exit 0. CSS leaves
the pack/PES header clear, so an un-descrambled sector muxes as a
structurally-valid but corrupt PES packet with zero loss reported.

decrypt_keys_for_title resolves a DVD title's CSS key two ways:
- Fast path: reuse the scan's cracked key when its crack_span covers this
  title's VTS (no re-read; on a live drive no second bus-auth).
- Crack: on a detection miss or a different VTS, crack from the title's OWN
  extents in a SINGLE scan in natural PLAYBACK ORDER (never largest-first).
  One scan = one CSS-locked early-bail, so a locked title is not re-hammered
  per cell against a live drive (hard rule #2); the 50k-sector budget is the
  same accepted bound the disc-wide scan uses. Cracked -> key; Unencrypted
  -> clear; ScrambledUncracked -> hard-fail.
ensure_title_decryptable hard-fails an uncrackable DVD title even when
detection missed, and passes a title that resolved its OWN valid key
regardless of the disc-wide css_error. descramble_region is unchanged from
v1.5.1 (validated-key seed).

Also rename the unlocker report's DVD entry CSS -> DVD. Bump 1.5.2.
2026-07-22 13:07:26 -07:00
Matthew Jackson d6cef9f665 v1.5.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-20 17:04:24 -07:00
Matthew Jackson b9b9185633 restore freemkv-unlock path dep for local dev (post-v1.5.1) 2026-07-20 17:02:14 -07:00
Matthew Jackson 66951a32ed v1.5.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-20 17:02:11 -07:00
Matthew Jackson 505a78efd1 mux/codec/truehd: fix MLP major-sync checksum endianness (was dropping the whole TrueHD track)
Regression since the previous release, which added an MLP major-sync checksum
gate to drop genuinely-undecodable audio frames. The checksum itself was computed
with mismatched byte order: `crc16_mlp` is the correct crc_2D table (poly 0x2D,
MSB-first) but returns its two bytes in the OPPOSITE order to libavutil's
`av_crc`, and `mlp_major_sync_crc_ok` then folded in the pre-trailer word
little-endian while comparing the trailer big-endian. The net result never
matched a real major sync, so EVERY major sync was judged corrupt. That armed the
drop-forward on the first AU and, since no major sync ever validated to clear it,
collateral-dropped every following AU forever — the entire TrueHD track was
silently dropped. Its AUs then flushed only at mux end, so the track's blocks
landed physically after all the video: a decoder reading video+TrueHD had to
buffer the whole title to reach the first audio block and spiralled into an
unbounded memory runaway ("decoder ran out of memory"). Every TrueHD title
produced after the gate landed was affected; a title from the release before it
is clean. (The header-size parse — a frequent suspect for extended 7.1/Atmos
headers — is NOT the bug; it already matches ffmpeg's `mlp_get_major_sync_size`
byte-for-byte.)

Fix: compute the checksum exactly as ffmpeg's `ff_mlp_checksum16` —
`crc16_mlp(body).swap_bytes() ^ AV_RL16(word) == AV_RL16(trailer)`. Cross-verified
byte-exact against two real discs (a 7.1/Atmos title and a 5.1 title, independent
32-byte headers both validate). With the checksum correct, major syncs validate
and the drop-forward corruption protection works as intended.

Defence in depth: a major-sync checksum that STILL can't be validated (a genuinely
corrupt or as-yet-unparsed header) no longer arms the drop-forward until we hold a
validated baseline (`num_substreams` from a prior clean major sync) — so a single
bad header can never again silently drop an entire track.

Tests: the `finalize_major_sync` fixture now builds the checksum the corrected way;
a synthetic checksum-failed head major sync is kept, not dropped; the existing
baseline-then-corrupt drop-forward tests still pass. Verified end to end against a
real disc: the TrueHD track demuxes to a full, cleanly-decodable 48 kHz 8-channel
stream, interleaved with the video, instead of 0 bytes.
2026-07-20 16:58:06 -07:00
Matthew Jackson 9b80084220 aacs: parse HD DVD VTKF title keys at the spec's 36-byte stride
The HD DVD Title Key File (VTKF*.AACS) stores 64 title-key entries of 36 bytes
each — 1-byte BIFO + 3 reserved + 16-byte encrypted key + 16-byte binding MAC —
per AACS "HD DVD and DVD Pre-recorded Book" Table 3-8, confirmed byte-exact
against real discs (Freedom VTKF090, Dukes VTKF000: every 36-byte slot has
BIFO=0x80, a clean key, and a 0xFF binding MAC).

The parser used a 32-byte stride (a 12-byte pad in place of the 16-byte binding
MAC) with flag-based termination. That aligns entry #1 (key at offset 132, where
both strides agree) but drifts +4 bytes per entry after it and never terminates
(the previous entry's 0xFF MAC reads as a set present-flag), so it recovered a
correct key only for single-CPS-unit discs and garbage for CPS unit >=2. Every
multi-title HD DVD (Freedom, Harry Potter) was affected.

Fix: 36-byte stride, iterate the fixed 64 slots, take slots whose BIFO AV_FLG
(bit 7) is set, key at offset 4, slot index = CPS unit (skip empty slots rather
than terminate so a gap can't renumber later keys), and never read the trailing
16-byte TKF MAC as a key. Tests rebuilt on the real layout, including a full
64-entry file.

Also correct the VTKF-selection TODO in mod.rs: the AACS HD DVD Book gives the
selector explicitly (match the TKF's PLAYLIST_NAME field to the active
playlist), not the "validate against an encrypted unit" placeholder.

Reconciled against the new HD DVD reference (freemkv.org/docs/hddvd/); the spec
source is archived in private-repo/spec/.
2026-07-20 11:57:13 -07:00
Matthew Jackson b246169e5b aacs: discover HD DVD AACS dir + title-key files instead of hardcoding /ANY!/VTKF000
The HD DVD AACS directory name and title-key filename are chosen by the
authoring house, but the resolver hardcoded a single spelling
(/ANY!/VTKF000.AACS, /ANY!/MKBROM.AACS, /ANY!/CONTENT_CERT.AACS). Real discs
diverge: Freedom (Memory-Tech) names its AACS dir AAC! and ships VTKF090.AACS
+ VTKF100.AACS; Harry Potter carries VTKF000/001/002/099. On such a disc the
hardcoded path finds nothing, so no MKB/title-key/cert is read and decryption
silently can't engage.

Replace the fixed HD DVD path constants with structural discovery:
- find_hddvd_aacs_dir() locates the AACS dir as the root child dir ending in
  '!' that contains MKBROM.AACS (so the ..._BAK mirror is skipped; the dozens
  of decoy advanced-content '!' dirs are excluded by the MKBROM.AACS guard).
- role_paths(udf, role) builds the ordered candidate list per role: the static
  BD/UHD /AACS/ paths first, then the discovered HD DVD files — MKBROM.AACS,
  CONTENT_CERT.AACS, and every VTKF*.AACS (sorted), not just VTKF000.
- read_first() is now generic over &str / String so it takes the Vec<String>.

BD/UHD unaffected (no '!' dir → discovery returns None, list is the /AACS/
constants exactly as before). Verified on real Freedom (AAC!/VTKF090+100) and
Dukes (ANY!/VTKF000) ISOs; unit tests cover both shapes.

Open item (TODO(hddvd-encrypted)): when a disc has multiple VTKF variants the
correct one must be chosen by validating its VUK-derived key against a real
encrypted unit rather than first-that-reads. Blocked on obtaining a genuinely
encrypted HD DVD image — all HD DVD ISOs on hand are already-decrypted rips.
2026-07-20 11:29:28 -07:00
Matthew Jackson 2d96a3e87d release: tee up 1.5.1 (marginal-read reporting fix) 2026-07-19 20:47:52 -07:00
Matthew Jackson bae5a9ea07 drive: report marginal reads (PER=1) so a dirty disc can't pass clean
A read was only ever judged "good" by the drive's SCSI GOOD status — with no
integrity check on the bytes. On dirty/marginal media a drive can silently
return best-effort ECC data (occasionally mis-corrected) with GOOD status, so a
rip could "pass clean" yet decode with errors, pushing corruption downstream to
the mux instead of catching it at the read.

Enable recovered-error REPORTING at drive-prep: MODE SELECT the Read-Write
Error Recovery page with PER=1 (TB on / DTE off so the data still comes back),
preserving the drive's own retry count. Marginal reads now surface as
CHECK CONDITION / RECOVERED ERROR. Best-effort — a drive that doesn't honor it
keeps its defaults (no regression), and on a clean disc nothing changes.

A recovered error is distrusted: it marks just that ECC block NonTrimmed for a
Pass N re-read (a clean re-read wins; a persistent marginal becomes an honest
concealed gap), and is counted in the pass summary. Crucially it takes a
per-block SkipBlock, NOT the damage-jump path — a single marginal sector must
not trigger a 64 MB skip that would discard good data on a lightly-smudged disc.
2026-07-19 20:30:21 -07:00
Matthew Jackson 86ed27508d changelog: 1.5.0 release date + trim to app-level detail 2026-07-19 17:34:21 -07:00
Matthew Jackson f28d79b51b restore freemkv-unlock path dep for local dev (post-v1.5.0) 2026-07-19 15:37:27 -07:00
Matthew Jackson 5b27b9f8a5 v1.5.0: bump version (freemkv-unlock git-pinned for the tag) 2026-07-19 15:37:25 -07:00
Matthew Jackson 405e9d32c3 changelog: drop-on-undecodable audio + forced PGS detection (1.5.0) 2026-07-19 14:29:57 -07:00
Matthew Jackson 21b362c6e8 disc: detect forced PGS subtitles from stream content for info
Give `info` the same forced-subtitle verdict the muxer derives during a
rip, so the two agree. A shared classifier (mux::codec::pgs::ForcedTracker)
folds a PGS track's display sets — forced iff every one carries the
forced_on_flag — and is used by BOTH the MKV writer and a new scan-time
probe that reads the title's PGS streams (reusing the TS demuxer and PGS
parser). The probe only overrides a track it actually observed content
for, so an undecrypted/unread stream keeps its vendor-derived flag. Gated
behind ScanOptions::probe_forced_subtitles (off for the rip path, which
detects forced while muxing without a second read).
2026-07-19 14:23:04 -07:00
Matthew Jackson 9f8df95513 mux/codec/mpegaudio: keep free-format frames
Free-format MPEG-audio (bitrate_index 0) is a legal, decodable mode — the
decoder derives the frame size from the sync spacing. Dropping it was a
false positive on a clean stream, so it now passes the gate.
2026-07-19 14:23:04 -07:00
Matthew Jackson 2f465f4cfe mux/codec/truehd: fix drop-forward poison + rate/resync validation
Three TrueHD state-machine fixes from an adversarial audit:

- A corrupt access unit drops forward to the next major sync, but only
  the individually-verified corruption now feeds the whole-track poison
  verdict; the resync run is collateral. A couple of transient errors can
  no longer poison and discard an otherwise-good multi-hour track. The
  shared drop tally gains a verified/collateral split for this.
- The per-AU PTS rate is refined only from a CRC-validated major sync, so
  a corrupt major sync whose rate nibble decodes to another rate family
  can no longer shift the resumed audio — a drop stays a silence gap.
- The resync clears only on a CRC-validated major sync, never on a runt
  too short to hold and validate its header.
2026-07-19 14:23:04 -07:00
Matthew Jackson 3fdd463794 release: 1.5.0 2026-07-19 13:42:48 -07:00
Matthew Jackson 6a7392ac8c mux: detect forced PGS subtitles from the stream
Flag a PGS subtitle track FlagForced when it displays subtitles and every
one carries the HDMV forced_on_flag (a dedicated forced/narrative track),
independent of the disc's vendor label metadata. The track header
reserves a FlagForced byte up front and it is promoted at finish() from
the accumulated display-set state. Only ever promotes — a track already
forced from the playlist metadata is never demoted.
2026-07-19 13:42:48 -07:00
Matthew Jackson 28bdecd26b mux/codec: drop undecodable audio frames, keep A/V sync
A damaged audio access unit is now dropped rather than muxed as a
decoder-choking glitch. Sync is preserved — a drop becomes a silence
gap, never a shift — and every drop is logged. Detection is per-codec,
each mirroring the format's authoritative integrity check:

  DTS          core-header validity gates
  AC-3/E-AC-3  native frame CRC-16 + bitstream-id range
  FLAC         whole-frame CRC-16 residue
  MP2/MP3      header sanity + free-format reject
  AAC-ADTS     header sanity (raw AAC passes through untouched)
  TrueHD/MLP   major-sync CRC-16 + AU parity; corrupt AUs drop forward
               to the next major sync, since decode state carries
               across access units

LPCM and video are excluded by design (no in-frame integrity data;
inter-frame prediction). A shared DropTally handles counting, logging,
and a whole-track fallback for a mostly-undecodable track.
2026-07-19 13:42:48 -07:00
Matthew Jackson d29d3d71b9 dts: emit clean core alone when the extension boundary is garbage
DTS-HD MA access units are a lossy core frame followed by trailing
extension substreams up to the next core sync. On source-damaged discs
(observed on the Bourne UHDs) the bytes where the XLL extension belongs
are neither a core sync nor an extension sync -- pure garbage -- which
desyncs ffmpeg's XLL decoder and cascades into 'Read past end of XLL
band data' / 'DSYNC check failed' across the whole track.

next_core_boundary now distinguishes three boundary states via a new
ext_clean flag on NextCore::Found:
  - precise/recognized extension sync  -> ext_clean=true  (keep full AU)
  - garbage at the boundary byte       -> ext_clean=false (drop the ext)

When ext_clean is false we emit the DTS core alone (drop the smallest
junk piece, keep the frame and its PTS) and drain past the garbage to
the next core. Recognized-but-unsizeable extensions still ride the
heuristic scan and are kept intact, so lossless tracks are unaffected --
only genuinely corrupt extension bytes are dropped.

Bourne s1.dts: 1606 damaged frames / 691620 (0.232%), 93% isolated
single frames, worst run 3 in a row (~32ms lossy blip).
2026-07-19 09:27:28 -07:00
Matthew Jackson 999898a438 changelog: faststart default + DTS/DTS-HD audio (mp4://) 2026-07-18 22:35:37 -07:00
Matthew Jackson b4470ccd81 mux/mp4: DTS audio (dtsc/dtsh + ddts)
Parse the DTS core header (SFREQ/AMODE/RATE/LFF/NBLKS/FSIZE) → a ddts box
(sample rate, channel layout mask, core size, computed bitrate, LFE);
whole access units (core + DTS-HD extension substreams) pass through, so
an HD decoder finds the extension. dtsh when an extension sync follows the
core, else dtsc. Fit oracle now carries DTS / DTS-HD MA / DTS-HD HR.

Validated on 300 (real DTS-HD MA 7.1): freemkv's mp4 DTS track is
byte-identical to ffmpeg -c copy under ffprobe (dts / 48000 / 8ch / 7.1)
and decodes clean (exit 0). Channel layout correct.
2026-07-18 22:18:35 -07:00
Matthew Jackson 668dd95237 mux/mp4: faststart on by default (reserve-and-fill)
moov now precedes mdat. At create() reserve a moov-sized hole (a free
box) between ftyp and mdat: reserve = round_up_4MB(16 B/sample ×
est_samples) + 4MB buffer, floored at 8MB. Because the hole precedes
mdat, sample offsets are fixed from the start — no rewrite, no offset
patch. finish() writes moov into the hole and pads the slack with a free
box; if the estimate is blown it falls back to moov-at-end. Streams over
HTTP without a pre-fetch. Reserve-math + box-order tests added.
2026-07-18 21:43:44 -07:00
Matthew Jackson 777d9a04a7 changelog: mp4:// input demuxer (1.5.0) 2026-07-18 21:22:41 -07:00
Matthew Jackson b717044ae5 mux/mp4: MP4 demuxer — mp4:// as a source
Read side of mp4://: parse moov/trak/stbl (stsd codecs+hvcC/avcC/channel
info, stsz/stco/co64+stsc → per-sample offsets, stts+ctts → decode/
composition timing, stss → sync), rebuild a DiscTitle, and emit samples
as PesFrames in global decode order. Video NALs are length-prefixed in
MP4 — the exact framing the MKV muxer wants — so no reframing. Wired into
input() so mp4:// flows to every sink (mkv://, audio://, json://, …).
In-memory write→read round-trip test proves symmetry (streams, hvcC,
sample sizes survive). Progressive MP4 only; fragmented (moof) is future.
2026-07-18 21:10:15 -07:00
Matthew Jackson 379767071c mux/mp4: route through WritebackFile (bounded-cache writeback)
Make Mp4Sink generic over a seekable writer; the CLI output() arm wraps
the file in WritebackFile (as mkv:// does) so a UHD-scale mux to slow /
NFS staging avoids the dirty-page burst. The mdat backpatch is an
ordinary seek WritebackFile already handles (seek_then_patch_roundtrip).
Tests switched to in-memory Cursor writers.
2026-07-18 20:57:44 -07:00
Matthew Jackson 6a06443dd8 changelog: mp4:// native MP4 muxer (1.5.0) 2026-07-18 20:44:18 -07:00
Matthew Jackson 42df399156 mux/mp4: M2 — audio tracks + fit oracle (multi-track)
Generalize the sink to N tracks: one video + every MP4-mappable audio
track. Audio sample entries built from the first frame's bitstream —
AC-3 (ac-3/dac3) and E-AC-3 (ec-3/dec3), parsing the (E-)AC-3 BSI for
fscod/bsid/bsmod/acmod/lfeon and the data rate. Per-sample audio
durations from PTS deltas (no reorder → no ctts, all sync). Fit oracle
(mp4_fit_report, exported): video HEVC/H264, audio AC-3/E-AC-3;
TrueHD/DTS/LPCM and bitmap subs are excluded with a typed reason so the
CLI can report exclusions — never a silent drop.

Verified vs ffmpeg -c copy on real discs: AVC+AC3 and HEVC(HDR10)+AC3
both frame-exact on every track (video 2650/4270, audio 5567/3454),
duration and colour identical, clean decode. On a pathological 4-clip
title ffmpeg's OWN output emits the same DTS-monotonicity warnings (more
of them) — source-inherent, not a muxer defect.
2026-07-18 20:37:10 -07:00
Matthew Jackson 9ede4b9afd mux: native progressive MP4 muxer (mp4://) — M1 video track
New mux/mp4: writes ftyp+mdat+moov (moov-at-end), streaming samples into
a 64-bit mdat and building the sample tables in memory, patched at
finish(). Video track (HEVC/AVC): passthrough length-prefixed NALs
(already MP4 framing), full stts/stsz/stsc/co64/stss and signed ctts,
CFR-derived decode timeline (pipeline carries presentation PTS only), and
a colr box for HDR10 colour signalling. hvc1/avc1 sample entry from the
hvcC/avcC codec_private. Fail-loud on codecs with no MP4 mapping.

Verified against ffmpeg -c copy on real discs: AVC-SDR (300) and
HEVC-HDR10 UHD (Dune) both frame-exact (8159 / 8160 frames), colour-exact
(bt2020/smpte2084/bt2020nc), and clean-decoding. Audio + fit oracle land
in M2.
2026-07-18 20:16:19 -07:00
Matthew Jackson 67e3e39499 mux: add video:// sink (video-only per-track elementary streams)
Completes the per-track-class trio with audio:// / sub://. video:// is
the demux path with a TrackKind::Video kind filter — each video track to
its own native elementary stream (.hevc/.h264/.vc1/.m2v/.obu), no audio
or subtitles. Reuses the existing kind_filter machinery and extension
map; scheme/parse/output/write-only-guard arms added symmetrically.
2026-07-18 19:24:59 -07:00
Matthew Jackson 42be09f628 changelog: 1.5.0 — extraction sinks + complete json:// model 2026-07-18 19:13:30 -07:00
Matthew Jackson 8bbbe43724 mux: json:// emits the complete title model (no dropped fields)
The json:// sink previously emitted only codec + language + pid per
stream. Fill it out to the full DiscTitle: video carries resolution
(+ pixel dims, interlaced), frame rate (+ fraction), HDR, colour space,
display aspect, and measured CICP; audio carries channels (+ count),
sample rate (+ Hz), and editorial purpose; subtitles carry the
qualifier. Add the clip list and chapter names. This is everything the
scan resolved, so json:// is a lossless machine-readable view of a
title rather than a summary.
2026-07-18 18:14:11 -07:00
Matthew Jackson 95888f27a4 test: pin audio://sub://chapters://json:// in the scheme round-trip 2026-07-18 17:46:08 -07:00
Matthew Jackson 96dec65040 mux: chapters:// and json:// metadata sinks
Two write-only file sinks that ignore the PES stream and emit the title
metadata at construction (fvi-style, wired through output()):
- chapters:// — chapter markers in the format the extension picks: .xml
  (Matroska, default), .txt/.ogm (OGM simple), .vtt (WebVTT). Reuses the
  existing chapters_xml/chapters_ogm writers; adds a WebVTT writer.
- json:// — one title's model (playlist, duration, size, format, streams,
  chapters) as pretty JSON via serde_json.

New mux/meta_sink.rs; StreamUrl gains Chapters/Json; codec_label +
chapters_xml/ogm promoted to pub(crate) for reuse.

Tests: chapters_format_selected_by_extension, title_json_carries_streams_and_chapters.
2026-07-18 17:45:39 -07:00
Matthew Jackson 378fb226b7 mux: audio:// and sub:// sinks (demux filtered by track class)
Two extraction sinks built on the existing demux machinery: audio:// keeps
only audio tracks (native containers .thd/.dts/.ac3/.eac3/.pcm...), sub://
only subtitle tracks (PGS .sup, VobSub .idx+.sub, text .srt) — one file per
track, no video, no chapters sidecar. DemuxOptions gains a kind_filter; the
DemuxSink writers (already complete) are reused verbatim. StreamUrl gains
Audio/Sub variants; output() builds the filtered DemuxSink.

Test: kind_filter_keeps_only_the_selected_class.
2026-07-18 17:24:28 -07:00
Matthew Jackson 42caa2d90e restore freemkv-unlock path dep for local dev (post-v1.4.5) 2026-07-18 16:08:03 -07:00
Matthew Jackson 41a2493429 v1.4.5: bump version (freemkv-unlock git-pinned for the tag) 2026-07-18 16:08:01 -07:00
Matthew Jackson 0e73650440 changelog: 1.4.5 (FMTS clean single-variant mux + key-Debug redaction + hex fix) 2026-07-18 13:30:36 -07:00
Matthew Jackson bcfd5eb213 mux: FMTS read-plan on the inline live-drive path too (DiscStream::with_key_map)
The read_plan fix landed on the file-backed highway (build_iso_pipeline)
but the inline DiscStream (live single-pass) still passed the alternate
device-group half to the demux — a known FMTS bug on rip_mode="single".

DiscStream::with_key_map installs the proactive map and rewrites the
extent walk to the read plan, so the live single-pass path reads ONLY
our-phase units, exactly like the highway. DecryptingSectorSource gains
set_key_map for the already-constructed decorator. Non-forensic maps
return the extents unchanged, so every non-FMTS disc is byte-identical.

Test: with_key_map_reads_only_our_phase_units — a forensic Even segment
drops exactly its alternate units from the extent walk. Precommit green
on 1.86.
2026-07-18 13:27:48 -07:00
Matthew Jackson 8ce0a66bee mux: FMTS reads only our-phase units (AacsKeyMap::read_plan)
An FMTS forensic segment interleaves our device group's variant with a
foreign group's at the aligned-unit level. The mux was decrypting only
our phase but leaving the alternate (foreign) units in the buffer as
ciphertext, trusting the demux to 'drop untouched ciphertext cleanly'.
It doesn't: random 0x47 bytes at the 192-byte stride hit tracked PIDs,
mis-parse, and trip the demux's concealed-gap keyframe-resync — which
drops GOOD frames of ours around every segment (349 resyncs / ~6391
packets on Stand by Me, visible as playback flaws).

The map already knows which unit each LBA is and, for a forensic
segment, which phase is ours. AacsKeyMap::read_plan turns that into the
title's read plan: every default/CPS unit, plus inside a segment ONLY
our-phase units. The alternate units are never fetched, decrypted, or
handed to the demux — the demux sees one gapless our-variant stream.

- read_plan is general (single-CPS, multi-CPS, FMTS): a map with no
  Even/Odd range returns the extents unchanged, so DVD/CSS, single-CPS
  UHD and multi-CPS Blu-ray read byte-for-byte as before.
- Wired into build_iso_pipeline (the file-backed highway that muxes
  resumed ISOs). Producer re-anchors unit_base per extent, so per-unit
  segment reads stay unit-aligned and decrypt correctly.
- Extent gains PartialEq/Eq for the read_plan tests.

Tests: read_plan non-forensic unchanged; forensic omits exactly the
alternate units, kept units match the decrypt gate unit-for-unit. All
2314 lib tests pass; precommit green on 1.86.

Pending: end-to-end ISO re-mux validation (concealed gaps 349 -> ~0).
2026-07-18 12:06:20 -07:00
Matthew Jackson 7d4d9a3e84 remove user-facing English from the library (KeyOrigin::name, hddvd Title)
Library holds ZERO user-facing English (CLAUDE.md). Removed KeyOrigin::name()'s
English prose — apps map the typed enum (freemkv gets key_origin_label); diag
logs the enum's Debug repr. hddvd unnamed-title fallback 'Title N' -> neutral
TITLE_N identifier (UDF volume-label style).
2026-07-17 21:50:09 -07:00
Matthew Jackson 4dec71a867 narrow leaked-internal pub surface to pub(crate)
Verified zero callers across all consumer crates AND libfreemkv integration tests:
MUX_APP, Disc::{aacs_disc_hash,encrypted_content_ranges,inject_unit_keys},
locate_ranges, mapfile::{MapEntry,entries}, diag::dump_mkv_track,
DiscStream::{errors,lost_bytes} (read via accessors). Removed the unused mux
DemuxSink/FviSink crate-root re-exports (constructed internally by output() via
the direct module path). Staged demux option variants marked allow(dead_code).
2026-07-17 21:45:33 -07:00
Matthew Jackson a9b29c7060 disc: collapse decrypt_keys_for_title twin into one method
Removed the test-only unchecked wrapper and renamed decrypt_keys_for_title_checked
-> decrypt_keys_for_title (one method returning (keys, title_is_clear)). Production
already used the checked form; the _checked suffix had no counterpart. Callers +
docs updated; two tests take .0.
2026-07-17 21:41:10 -07:00
Matthew Jackson 0cb525517b css: remove redundant crack_key_halt wrapper
crack_key_halt had no caller except the crack_key wrapper — a needless middle
layer. crack_key now calls crack_key_scan directly; crack_key (Option) and
crack_key_outcome (full CrackOutcome + halt) remain as the two real entry points.
2026-07-17 21:34:37 -07:00
Matthew Jackson 5b4a3f32dc mux: collapse FviSink::create/create_with_source into one create
create_with_source (full SourceInfo) was only ever called by the create wrapper
with a partial SourceInfo — the extra capability was unused (YAGNI). Inlined the
constructor body into create and deleted the _with_source variant; callers
unchanged.
2026-07-17 21:32:39 -07:00
Matthew Jackson 66b0bc385c mux: collapse MkvStream::create/create_at into one create(.., Option<&Path>)
One method per action (CLAUDE.md), not a foo_with_X pair — create_at was the real
constructor and create was a None wrapper. Callers pass Option<&Path> directly.
2026-07-17 21:31:29 -07:00
Matthew Jackson aaccd83750 hex: canonical hex->integer parsers + public strip_hex_prefix
Adds parse_hex_u16/u32/u8 and exposes strip_hex_prefix so callers stop hand-rolling
from_str_radix(trim_start_matches("0x")) — a case-sensitive strip that this module
exists to prevent. disc::aacs_disc_hash now uses strip_hex_prefix.
2026-07-17 21:25:59 -07:00
Matthew Jackson 40f5b01a12 disc: redact Debug for AacsState/Key/HandshakeResult (test-guarded)
AacsState (public via Disc.aacs) and Key (the decrypt_with key-transport enum)
are crate-root re-exported and carried VUK/unit/read-data keys + volume id on
#[derive(Debug)]; HandshakeResult carried the VID + AACS 2.0 bus key. Manual
Debug impls print shape only, guarded by red->green tests.
2026-07-17 21:10:51 -07:00
Matthew Jackson 82d42b1930 aacs: redact Debug for ResolvedChain/ResolvedKeys/ProcessingKeyMatch (test-guarded)
These carry raw unit-key / VUK / processing-key bytes on their Debug; manual impls
print shape only (unit_keys_len, redacted markers). Each has a red→green test.
2026-07-17 21:03:31 -07:00
Matthew Jackson 913b01b79f css: redact CssState Debug (test-guarded)
CssState is reachable via the public Disc.css field; #[derive(Debug)] leaked the
raw CSS title key on any {:?} of a Disc. Manual Debug prints crack_span only.
2026-07-17 21:00:34 -07:00
Matthew Jackson af7ef59f3e aacs/types: redact Debug for all key-bearing types (test-guarded)
DeviceKey/HostCert/Vid/MediaKey/Vuk/ProcessingKey/UnitKey/DiscEntry carried key
material on #[derive(Debug)] — a stray {:?} would leak device/host-private/media/
volume/unit keys. Manual Debug impls print shape only. redaction_tests asserts no
key bytes appear and a redaction marker is present (fails if a derive returns).
2026-07-17 20:56:38 -07:00
Matthew Jackson ef8e11b6f3 libfreemkv: phase-aware FMTS decode + two-operation KeyFetch/KeySource seam
FMTS (AACS 2.1) now decodes per (LBA, phase): Phase enum + AacsKeyMap::
from_ranges_phased, decrypt only the variant's parity half. resolve_fmts_key_map
does a 2-phase index-1 anchor then per-index phase probe, and sizes the forensic
set to whatever the source returns (no hardcoded 32). KeyFetch is now two explicit
operations (unit_keys / fmts_indexes) and KeySource splits get_uk into
get_unit_keys + get_fmts_indexes. BYPASS_FMTS_KEY gate removed (first-class format).

Teed up for 1.4.5. Local WIP baseline.
2026-07-17 20:51:15 -07:00
Matthew Jackson e1f897fb32 restore freemkv-unlock path dep for local dev (post-v1.4.4) 2026-07-16 21:44:08 -07:00
Matthew Jackson 602e44fde9 v1.4.4: bump version (freemkv-unlock git-pinned for the tag) 2026-07-16 21:44:05 -07:00
Matthew Jackson 622008650c keysource: add DecodeSampleSet — a >=MIN_SAMPLE_UNITS-by-construction sample set so an online request can't be built under-sized 2026-07-16 21:43:02 -07:00
Matthew Jackson c508484451 restore freemkv-unlock path dep for local dev (post-v1.4.3) 2026-07-16 21:06:04 -07:00
Matthew Jackson cf16f66e07 v1.4.3: bump version (freemkv-unlock git-pinned for the tag) 2026-07-16 21:06:01 -07:00
Matthew Jackson adcab75409 aacs: sample only index-1 forensic segments for the online key query; hoist MIN_SAMPLE_UNITS into the base crate 2026-07-16 21:01:16 -07:00
Matthew Jackson 61e4095727 FMTS: resolve the index key map from one forensic keyserver query
The keyserver protocol now returns all 32 index keys as an array for a
forensic content sample (and a single-element array for plain content).
resolve_fmts_key_map sends one forensic batch and maps array element i to
segment index i+1, replacing the per-index blind-probe collection loop
that repeatedly hit the key service. Segment/index parsing and the
aligned-unit content classification are reworked to support this:

- rename variant_select -> index_select (per-index, not per-variant)
- content classification moves to is_clean(buf, ContentFormat) so the
  unit selector emits only units the key service accepts
- segment.rs: parse IndividualSegment.tbl index tags + SPN ranges,
  build contiguous LBA key ranges from the resolved 32-key array
- decrypt/decorator plumbing for the resolved per-index keys

Fail loud (FmtsKeyMissing) when the forensic query returns < 32 keys or
any segment index stays unresolved.
2026-07-16 19:41:44 -07:00
Matthew Jackson 540f84c7d0 restore freemkv-unlock path dep for local dev (post-v1.4.2) 2026-07-15 19:37:54 -07:00
Matthew Jackson 82c4e018d5 v1.4.2: bump version (freemkv-unlock git-pinned for the tag) 2026-07-15 19:37:51 -07:00
Matthew Jackson 08e962694f Mux decrypt/verify redesign, HD DVD first-class, MVC 3D
decrypt:
- decrypt_sectors is now a pure decrypt (apply key, leave plaintext, report
  unverified bytes); TS-structure is a separate primitive (is_clean_ts/ps) used
  only for key selection and read-verify. The mux passes decrypted bytes through
  (the demuxer drops non-conforming packets), ending the NULL-TS conceal loop and
  the per-unit key-server refetch storm. Key-proof floor replaces the 75%
  supermajority.

recovery:
- Removed the post-read decrypt-verify gate (verify.rs) that mis-aligned the
  disc-absolute unit grid against clip-anchored AACS units and false-failed good
  clips (e.g. Dunkirk's orphan-CPS clip). Bad sectors are marked by physical read
  result; decryptability is proven at scan + mux time.

HD DVD (first-class AACS):
- Role-based candidate-list file sourcing so an HD DVD's /ANY!/ files
  (MKBROM.AACS, VTKF000.AACS, CONTENT_CERT.AACS) are found with no disc-type
  branch. parse_vtkf parses VTKF000.AACS into the same UnitKeyFile as a BD
  Unit_Key_RO.inf, so the shared VUK unwrap applies unchanged. set_unit_base
  clip-anchoring. Two decrypt-axis assumptions remain UNVERIFIED-HDDVD-DECRYPT
  (no encrypted disc to test).

mux:
- MVC (Blu-ray 3D) track signals unified into one MVCDecoderConfigurationRecord;
  release-safe track_vint (3-byte VINT) and pid_index (i32) guards.

hardening:
- Container-aware is_clean / encryption detection; bytes_bad_in_title fail-safe
  on a corrupt mapfile; CSS crack gated on DiscFormat::Dvd (HD DVD excluded);
  non-vacuous CSS tests; patch NOT_READY/HARDWARE/ILLEGAL_REQUEST/ABORTED
  sense-path tests.
2026-07-15 19:35:12 -07:00
Matthew Jackson 37e563b8ef Mux: pure decrypt, policy at the caller (no null, no key-server storm)
decrypt_sectors is now a pure decrypt — apply the CPS unit key, leave the
plaintext, report how many bytes did not reach clean TS ("unverified"). It
never restores ciphertext, nulls, or re-fetches. "Did a key produce clean TS?"
is a key-selection / read-verify signal, not the verdict "did we decrypt?": a
correct key can decrypt a bad-encoded region, and broken TS is a muxer concern
(the demuxer drops the packet and resyncs).

Callers own the policy:
- mux (read > decrypt > mux): pass the decrypted bytes to the muxer, whatever
  they are; fail loud only on a genuine can't-decrypt (no key / misaligned).
- sweep/patch (reading from a disc): an unverified unit is a bad read — recover
  a fresh key and retry, or fail loud so disc-recovery re-reads it.

Removes three duplicated decisions — the decrypt-time ciphertext restore, the
mux NULL-TS conceal loop, and the per-unit key-server refetch — plus the dead
aacs_unit_still_ciphertext predicate. Key-fetch recovery now samples the on-disc
ciphertext explicitly (a pure decrypt leaves the buffer plaintext) and lives
only on the rip/verify path, never the mux.

Fixes the 30-90s/region mux stalls and key-server storm on bad-encoded UHD runs
that 1.4.1 left behind (it relaxed the gate but not the surrounding machinery).
2026-07-14 21:04:24 -07:00
Matthew Jackson e659613ab8 restore freemkv-unlock path dep for local dev (post-v1.4.1) 2026-07-14 14:45:33 -07:00
Matthew Jackson eaa7ddc683 v1.4.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-14 14:45:30 -07:00
Matthew Jackson 723c7edf45 Fix mux concealing decryptable video over a single defective packet
AACS content decryption rejected a whole 6144-byte aligned unit unless
EVERY content packet was conformant MPEG-TS. One authored-bad packet (a
pressing/encoding defect or an AACS 2.1 forensic-variant frame) made the
mux conceal the entire unit as NULL TS — destroying up to 31/32 good
packets and tallying them as loss, surfacing as false "corruption" on
otherwise-clean discs (observed across two UHD titles).

decrypt_unit now asks only "did a key OPEN this unit?" — a padding-aware
>=75% supermajority of content packets restoring their 0x47 sync, a gate
no wrong key can reach (uniform-AES noise floor) yet one that tolerates a
minority of authored-bad packets. Opened units pass through VERBATIM; a
non-conforming packet is left for the demuxer to drop on sync-loss and
resync past. TS-sync conformance is a muxer concern, never a decryption
verdict. The post-read verify/sweep gate now shares the same primitive so
it can never disagree with the mux decrypt.

Also unify the MVC (Blu-ray 3D) track signals: the mvcC CodecPrivate
extension, the BlockAdditionMapping, and each per-frame BlockAdditional
all derive from one MVCDecoderConfigurationRecord built once per track, so
a malformed dependent-view parameter set can no longer orphan a BlockAddID.
2026-07-14 14:43:06 -07:00
Matthew Jackson d9881c57c0 restore freemkv-unlock path dep for local dev (post-v1.4.0) 2026-07-13 19:28:55 -07:00
Matthew Jackson 61b4486ae9 v1.4.0: bump version (freemkv-unlock git-pinned for the tag) 2026-07-13 19:28:52 -07:00
Matthew Jackson 72d39544ac Changelog: 1.4.0 (Blu-ray 3D / MVC) 2026-07-13 12:47:03 -07:00
Matthew Jackson 1e394eaeb8 3D MVC mux: audit round 2 (converged)
Second audit round converged (severity collapsed 6 HIGH -> 1; the one
HIGH was a bounded 32-element scan, not a defect; the sole spec MEDIUM
was the same false-positive re-raised — 0xBF matches ISO/IEC 14496-15
§7.6.2 verbatim). One genuine robustness fix plus coverage:

- extract_mvc_params: skip a zero-length NAL instead of abandoning the
  scan, so a stray length prefix before the subset SPS/PPS no longer
  silently drops 3D signalling. Test proves params after a zero-length
  NAL are still found.
- Tests: parser_for_mvc_dependent routes H.264 to a passthrough parser;
  passthrough with an IDR does not re-assert param sets (the keyframe &&
  !mvc branch).
- Document the per-playlist (not per-clip) is_3d latching as a known
  limitation (real main-feature playlists are uniformly 3D).
2026-07-13 11:39:07 -07:00
Matthew Jackson 4d1aa6bbf7 Harden 3D MVC mux: robustness + tests (audit round 1)
Triage of a 10-lens code audit of the 3D branch. Fixes for real defects;
rejected three spec false-positives that matched the ISO/IEC 14496-15
§7.6.2 record verbatim.

Robustness / correctness:
- Never panic when a title's only video is the MVC dependent view: the
  base is now the first NON-dependent video, so a dependent-only title
  sets up no merge (muxed as an ordinary track) instead of hitting an
  `expect` on the skipped track slot.
- Drop a per-frame BlockAdditional (BlockAddID=2) when the track declared
  no mvcC mapping (dependent params not captured before the header) — a
  plain block keeps the file conforming instead of an orphaned add.
- A non-keyframe MVC base frame always carries a ReferenceBlock (fall back
  to a 0 offset in the pre-first-keyframe corner) so it is never mistaken
  for a seek point.
- Reference the last keyframe on the PRIMARY video track only, so a
  secondary video track's keyframe can't become a cross-track reference.
- dep_by_pts overflow: bound BEFORE inserting so the just-arrived
  dependent survives the drift-clear; count a displaced duplicate-PTS
  dependent as an orphan instead of losing it silently.

API / docs:
- Fold write_frame_with_additional into write_frame(..., Option<&[u8]>)
  per the "no foo_with_X" convention.
- Fix mvc_params doc (StereoMode is intentionally not emitted); remove a
  stale PAT/PMT comment describing an approach that was never taken.

Tests: MVCDecoderConfigurationRecord over-length guards; write_int minimal
two's-complement widths; BlockGroup/BlockAdditions/BlockAdditional +
ReferenceBlock emission; additional dropped without a mapping; h264 MVC
passthrough keeps param sets in-band; extract_mvc_params no-panic on
truncated/empty input; pairing window + dep-overflow edges; no-panic on a
dependent-only title.
2026-07-13 11:24:23 -07:00
Matthew Jackson 9ce03b4d1d Mux Blu-ray 3D (MVC) as a single MVC video track
Fold the MVC dependent (right-eye) view into the base H.264 track as a
per-frame BlockAdditional under an mvcC BlockAdditionMapping, so a 3D
title produces one MVC video track instead of two independent H.264
tracks.

- h264: MVC-passthrough parser mode keeps the dependent view's subset
  SPS/PPS in-band, so each emitted frame is a self-contained dependent
  access unit for a BlockAdditional
- resolve: route the dependent stream through the passthrough parser
- mkvstream: detect the dependent view, pair it to the base frame by
  PTS (bounded FIFO), attach it as a BlockAdditional (BlockAddID=2),
  and skip building its own track; build the mvcC
  MVCDecoderConfigurationRecord from the captured subset SPS/PPS and set
  it on the base track at activation
- mkv: emit the mvcC BlockAdditionMapping and BlockGroup/BlockAdditions,
  with a ReferenceBlock on non-keyframe base frames
- ebml: add BlockAdditions/BlockMore/BlockAdditional/BlockAddID/
  BlockAddIDValue/ReferenceBlock elements and a signed-int writer

Verified against a Blu-ray 3D ISO: ffprobe shows a single MVC track,
the mvcC mapping is present, ~144k BlockAdditionals carry the dependent
view (8.7 GB), and the base view decodes cleanly with no regression.
MVCDecoderConfigurationRecord follows ISO/IEC 14496-15 7.6.2; StereoMode
is intentionally omitted (no enum value describes MVC-in-BlockAdditional;
the mvcC mapping is the primary 3D signal per RFC 9559).
2026-07-13 11:00:04 -07:00
Matthew Jackson f099a4a31a feat(3d): read the SSIF and enumerate the MVC dependent view
Blu-ray 3D as a mux-path variant:
- Detect 3D (a clip has STREAM/SSIF/<clip>.ssif — note the SSIF/ subdir, which
  the old .ssif fallback path got wrong) and use the SSIF extents for the mux:
  one transport stream carrying both the base (left) and MVC dependent (right)
  eyes on distinct PIDs, so muxing it captures the full 3D.
- Add the dependent-view video stream. The base STN table lists only the left
  eye; the on-disc PAT/PMT are AACS-encrypted (unreadable pre-key) and the base
  STN omits the dependent view (it lives in the MPLS STN_table_SS), so use the
  BD-3D PID convention: dependent = base video PID + 1 (0x1011 -> 0x1012),
  coding 0x20 -> H.264. The demux routes its packets from the SSIF.

info now reports two video streams for a 3D title; the mux writes both eyes.
2026-07-12 23:41:47 -07:00
Matthew Jackson e606b5e3a2 feat(3d): enumerate MVC dependent view (stream_type 0x20 -> H.264 video)
from_coding_type maps 0x20 (MVC dependent view) to Codec::H264, so the existing
PAT/PMT scan surfaces the SSIF right-eye substream as a second video stream on
its own PID instead of dropping it as Unknown. No new parser: the dependent eye
rides the same demux path as any other TS video stream. Removes the throwaway
3D-structure probes (findings captured in prior commit messages).
2026-07-12 22:16:13 -07:00
Matthew Jackson de809189ec feat(3d): add MVC dependent-view coding type (0x20)
First step of Blu-ray 3D as a mux-path format variant: name stream_type 0x20
(H.264/MVC dependent view, the SSIF right-eye substream). Recognition in the
scan/demux + dependent-stream enumeration follow.
2026-07-12 22:11:01 -07:00
Matthew Jackson 253b0924e3 wip(3d): prove SSIF de-interleave = ssif−base, dependent decrypts on same UK
Empirical (300, clip 00042): base .m2ts (1567 extents, 25.85 GB) is entirely
inside the SSIF LBA span; dependent = SSIF − base = 1567 ranges, 9.59 GB
(exact vs 35.43−25.85). 200/200 sampled dependent-view units decrypt under the
base UK. => de-interleave needs no SSIF parsing (complement of the base extents
we already resolve), and the dependent eye needs no extra key.
2026-07-12 20:40:58 -07:00
Matthew Jackson 98a7892c34 wip(3d): read-only 3D-structure probe (SSIF/m2ts + MPLS ExtensionData)
Dev tool for the 3D build: uses the library UDF reader (7z cannot parse these
ISOs) to dump base .m2ts vs SSIF sizes and the MPLS ExtensionData entries
(STN_table_SS + SS sub-path). Not product code.
2026-07-12 20:38:37 -07:00
Matthew Jackson b82d21756a fix(udf): follow allocation-descriptor continuation past the AED header
read_icb_extents parsed a continuation block's allocation descriptors from
offset 0, but a continuation block begins with a 24-byte Allocation Extent
Descriptor (ECMA-167 4/14.5): 16-byte descriptor tag + prev-loc(4) +
length_of_allocation_descriptors(4); the real ADs start at offset 24. Reading
from 0 mis-parsed the AED tag as a descriptor -> one garbage extent, then an
unknown-type break, silently truncating any file whose extents spill into a
continuation block.

Few-extent files store their ADs inline and never hit this path, so it stayed
hidden; a heavily fragmented file (~1600 fragments) truncated to 113 extents +
a bogus non-unit-aligned fragment that then aborted the mux
(ExtentNotUnitAligned). Skip the 24-byte AED header, read l_ad bytes of
descriptors from offset 24. Adds a regression test with a non-zero AED tag.
2026-07-12 20:37:55 -07:00
Matthew Jackson 02c584a3d9 restore freemkv-unlock path dep for local dev (post-v1.3.2) 2026-07-10 14:26:11 -07:00
Matthew Jackson 564e66fd24 v1.3.2: bump version (freemkv-unlock git-pinned for the tag) 2026-07-10 14:26:08 -07:00
Matthew Jackson 57d40e81a5 1.3.2: AACS 2.1 FMTS variant-decode foundation
Add UnitKey.variant_number (0 = ordinary, 1..32 = forensic variant) with new/variant constructors, and aacs::variant_select — resolve a disc's single variant and classify each aligned unit (default / variant / drop foreign / conceal keyless). Correct IndividualSegment.tbl: the per-record field is the variant (cycles 1..32 on a retail disc), not a segment number — Segment.number -> Segment.variant.
2026-07-10 14:00:28 -07:00
Matthew Jackson 674819d937 changelog: trim the 1.3.1 relicense note to just the relicense fact 2026-07-10 12:53:17 -07:00
Matthew Jackson c5e8421f9f restore freemkv-unlock path dep for local dev (post-v1.3.1) 2026-07-10 12:35:12 -07:00
Matthew Jackson 7dcf9b5a0d v1.3.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-10 12:35:09 -07:00
Matthew Jackson f01c097784 1.3.1: relicense to MIT (clean-room CSS + drop copyleft-lib references)
Relicensed from AGPL-3.0 to MIT, effective 1.3.1 (<=1.3.0 remain AGPL). The CSS
content cipher and Stevenson title-key attack are attributed to their published
cryptanalysis (not libdvdcss); all libaacs/libbluray/libdvdread/libdvdnav name
references were dropped from comments while keeping the standard format/spec
descriptions. Also bumps to 1.3.1.
2026-07-10 12:31:19 -07:00
Matthew Jackson c265fa349e hddvd: authoritative title composition from the Advanced-Content playlist
Parse ADV_OBJ/VPLST000.XPL (the real HD-DVD player playlist) with roxmltree
into one DiscTitle per <Title>: its PrimaryAudioVideoClip clips in order (EVO
via the .MAP sidecar), the titleDuration, the displayName, and the ChapterList.
A layer-break split (FEATURE_1+FEATURE_2, feature/feature_Divide) composes into
ONE title with the two parts as clips, each carrying its title-time in/out
points (45kHz ticks) for seamless-join splicing. Falls back to the clip-name
heuristic when no playlist is present.

Fixes unknown runtimes (real durations), poor names (FEATURE -> 'Main Movie'),
and gives authoritative composition + chapters. Validated on ANCHORMAN (97m),
SHAUN (99m), HARRY_POTTER (152m).
2026-07-10 10:28:18 -07:00
Matthew Jackson 6a59dcb7ba ci: drop the dead crates.io publish job (libfreemkv is git-tag-only)
libfreemkv is publish=false (it git-deps the firmware crate freemkv-unlock,
never on crates.io), so the Release workflow's 'cargo publish' step failed
hard on every tag. Consumers git-tag-pin libfreemkv; the git tag is the
release artifact. Removed the publish job.
2026-07-10 08:52:57 -07:00
Matthew Jackson 0eb1817ec7 restore freemkv-unlock path dep for local dev (post-v1.3.0) 2026-07-10 08:43:47 -07:00
Matthew Jackson 21f3b6adbd v1.3.0: bump version (freemkv-unlock git-pinned for the tag) 2026-07-10 08:43:42 -07:00
Matthew Jackson b32d636c64 audit: void empty-timeline duration, cover sniff overlap
Round-11 findings from the 10-phase release audit (no real HIGH):

- When a no-declared-duration source (HD-DVD) muxes a degenerate single
  frame at tick 0 with no per-frame duration, max_block_ticks stays 0 and
  the reserved DURATION placeholder was left as a literal 0.0 (players read
  that as a zero-length file). Void the element instead, so the Segment
  omits DURATION as an unknown-duration source did before the back-patch.
- Add a regression test for the sniff_video_codec overlap fix (a
  picture_start_code whose payload begins 00 00 followed by a real start
  code) so the i+=4 marker skip can't silently regress to i+=3.
2026-07-09 20:40:54 -07:00
Matthew Jackson d877630c45 audit: lock DTS rate table, fix sniff overflow-scan, cover decrypt loss
Round-10 findings from the 10-phase release audit:

- A finder claimed the DTS SFREQ→rate table was wrong at 11/12; verified
  it against ffmpeg's avpriv_dca_sample_rates (12k/24k/48k/96k/192k at
  11-15) — the table is CORRECT. Added a test that locks the full table so
  it can't be mis-"fixed".
- sniff_video_codec advanced 3 bytes after a matched start code, re-reading
  the code byte as an overlapping start code; skip the full 4-byte marker.
- Guard the HD-DVD next_id title counter with saturating_add so a crafted
  disc with >65536 clips can't overflow (panic in debug).
- Add a test that an undecryptable unit (DecryptFailed) is zero-filled and
  counted as loss through ExtractResult (complete=false, bytes_lost>0) —
  the recovery-seam consolidation folded that bucket into bytes_unreadable.
2026-07-09 20:17:12 -07:00
Matthew Jackson 1895095f49 audit: drop dead DTS marks cap, lazy passthrough buf, doc corrections
Round-9 findings from the 10-phase release audit (no HIGH):

- Remove the MAX_PTS_MARKS backstop and its tautological test: an empty
  DTS PES returns before recording a mark, and a non-empty run is already
  bounded by the MAX_AU_BYTES buffer clear (which clears pts_marks) — so
  the deque cannot grow unbounded and the cap was dead code.
- AuAssembler::for_codec no longer reserves 256 KiB for a Passthrough
  stream (audio/subtitle, and every TS/BD stream) whose buf is never
  written; only the reassembling modes reserve.
- Correct the scan comment that claimed region is computed (it is a
  Region-free stub until region detection lands) and drop a public-repo
  reference to internal "private refactor notes" in the mkb module doc.
2026-07-09 19:50:54 -07:00
Matthew Jackson 5f8c049444 audit: clamp BD format fallback, running GOP byte counter, O(1) DTS marks
Round-8 findings from the 10-phase release audit:

- detect_disc_format's BDMV fallback passed detect_format's result
  through unchanged, so an SD bonus/menu title could tag a BD-tree disc as
  DVD (mis-sizing the ECC sweep) — violating its own "never below Blu-ray"
  invariant. Clamp anything but UHD up to Blu-ray.
- Track the MPEG-2 GOP byte total incrementally instead of re-summing the
  whole gop_buf on every pushed picture (was O(pictures²) on any MPEG-2
  disc, not just adversarial input).
- Back the DTS pts_marks deque with a VecDeque so the over-cap prune is an
  O(1) pop_front, not an O(n) Vec::remove(0).
- Add a test exercising parse_stream_id_extension's PTS/DTS skip branches
  (the real AU-opening 0xFD video PES path) — previously untested.
2026-07-09 19:27:01 -07:00
Matthew Jackson 7d427d0b17 audit: bound DTS marks, align disc-format tree order, doc/test cleanups
Round-7 findings from the 10-phase release audit (no HIGH; convergence):

- Cap DtsParser.pts_marks (MAX_PTS_MARKS): a run of zero-length timed PES
  packets grew no buffer bytes, so the drain_front mark-prune never ran —
  the deque could accumulate without bound on hostile PS input.
- detect_disc_format tested HVDVD_TS before BDMV while the title-scan
  dispatch tests BDMV first, so a disc with both trees would be classified
  HD-DVD but enumerated as Blu-ray. Align both to BDMV → HVDVD_TS →
  VIDEO_TS.
- Document why the DTS new-PES re-base can emit a locally-decreasing PTS
  (the muxer's block_ts applies the strictly-monotonic audio nudge, tested
  in mkv.rs) — this is by design, not a mux defect.
- Fix stale aacs/keys.rs comment references (functions moved to
  aacs/inf.rs / aacs::resolve/derive in the module split).
2026-07-09 19:01:56 -07:00
Matthew Jackson 0a47b22db5 audit: byte caps on GOP buffers, opener-scan resume, honest video codec
Round-6 findings from the 10-phase release audit:

- Wire the documented MAX_PENDING_BYTES byte cap into the MPEG-2 GOP
  buffer (it was dead code) and add an equivalent MAX_GOP_BYTES cap to the
  sparse-PTS reorder, so a crafted stream of few-but-huge access units
  cannot over-allocate — both were bounded only by frame count before.
- probe_evo_streams defaulted an unsniffable HD-DVD video stream to H.264,
  which mis-parses a VC-1 (or still-encrypted) clip into a corrupt track.
  Emit the video stream only when the codec is actually identified — the
  honest outcome, matching the audio path (a real clear clip always carries
  its sequence header at the head).
- Resume the AU-opener search from a cursor (like the boundary search), so
  a long unsynced junk run is O(bytes), not O(buffer) per push.
- Mark mpeg2's now-dead MAX_AU_BUFFER test-only; restore #[doc(hidden)] on
  the aacs probe harness module.
- Add regression tests: the 0xFD video-routing guard, the FMTS-is-UHD key
  state, and the GOP byte caps.
2026-07-09 18:31:47 -07:00
Matthew Jackson 2ebcb60059 audit: guard 0xFD video routing, carry frame duration, add cap tests
Round-5 findings from the 10-phase release audit:

- collect_es routed EVERY extended-stream-id (0xFD) PES into the video ES
  buffer, so a 0xFD HD-audio sub-stream (MLP/TrueHD) could pollute the
  video sample and — if it preceded the video PES — stamp the video track
  with the audio PID, losing the video. Only the VC-1 extension (0x55) is
  now treated as video; routing 0xFD audio to its own track is deferred to
  the HD-DVD program-chain follow-up.
- The sparse-PTS reorder now carries its calibrated per-frame duration onto
  each frame, so the muxer emits a BlockDuration and the back-patched
  Segment Duration covers the final frame instead of understating it.
- Add regression tests for the MAX_MARKS and MAX_VTI_HITS caps (promote
  MAX_VTI_HITS to module scope); make the differential-test factory array a
  named type; drop an identity-op in a reorder test.
2026-07-09 17:59:04 -07:00
Matthew Jackson f46d99a348 audit: fix AU mark-field loss, VTI tie determinism, and mark/perf issues
Round-4 findings from the 10-phase release audit (the first fully clean
round; it dug into the new #22/#18 refactor code):

- AuAssembler closed each AU from only the FRONT mark's fields, so when
  one PES fragment carried the source and a later fragment of the same AU
  carried the PTS, the second field was dropped — a regression vs the old
  separate pts/source mark deques. Now merge the first Some of each field
  across all in-range marks.
- parse_vti_clip_order picked the largest residue bucket with
  HashMap::into_values().max_by_key(), nondeterministic on a size tie
  (randomized HashMap iteration) — could select a different clip table
  run-to-run. Break ties by smallest offset.
- Bound the marks/disc_marks deques (MAX_MARKS): the buf-size cap prunes
  marks only when bytes accumulate, so a run of zero-length timed
  fragments could grow them without bound on hostile input.
- Add push_owned so the PS path moves the PES payload into a passthrough
  AU with no copy (MPEG-2 video + all audio), removing a per-PES
  malloc+memcpy the refactor had introduced on the DVD path.
- Back-patch the MKV duration from the block END (start + its own
  duration) so it covers the final frame instead of understating by one.
- Add direct tests for the MKB record-framing walker; drop a stale
  drain_complete_aus doc comment left on process_au.
2026-07-09 17:30:41 -07:00
Matthew Jackson 4d93d90561 docs: drop Renesas from changelog (detection-only stub) 2026-07-09 16:36:47 -07:00
Matthew Jackson 4eeafd3378 docs: soften HD-DVD changelog to partial support (PGC parse deferred) 2026-07-09 16:33:40 -07:00
Matthew Jackson a73ba98f82 mux: resume AU-boundary scans from a cursor (O(n) not O(n²))
AuAssembler::drain rescanned the whole buffered access unit from a fixed
offset on every push, so reassembling one AU split across N program-stream
PES fragments cost O(bytes²/fragment) — amplified on the HD-DVD PS path
where H.264/HEVC/VC-1 frames are large and now flow through this shared
assembler (unlike the TS path, which delivers one AU per PES).

Carry a scan_pos cursor (and, for the stateful VC-1/MPEG-2 rules, a
seen_unit flag) so each push resumes the boundary search where the last
one stopped instead of restarting. Total scan work for one AU is now
O(AU bytes). The from-scratch scanners are retained as a #[cfg(test)]
oracle; a new differential test asserts the resumable path yields
byte-identical AUs at every fragment granularity for all three modes.
2026-07-09 16:22:54 -07:00
Matthew Jackson 7676f3f2f6 audit: bound the VTI clip-table scan; fix stale aacs doc links
Round-2 findings from the 10-phase release audit:

- parse_vti_clip_order bucketed hits by residue with an O(stride*hits)
  rescan and no hit cap, so a crafted HD-DVD VTI packed with millions of
  `.EVO` tokens (up to the 64 MiB UDF read cap) could burn seconds of CPU
  on a routine scan. Bucket in a single O(hits) pass and cap collected
  hits at MAX_VTI_HITS (a real table holds a few dozen).
- Fix the stale `super::keys::…` intra-doc links left by the aacs module
  rename: the referenced fns live in `super::derive`.
2026-07-09 14:41:49 -07:00
Matthew Jackson e1cfe59dc9 audit: cap the sparse-PTS reorder buffer, FMTS key state, zero KCD
Round-1 findings from the 10-phase release audit:

- SparsePtsReorder buffered its current GOP with no bound, draining only on
  a keyframe — an open-GOP or crafted program stream that never signals one
  could hold the whole title in RAM. Force-complete the GOP at
  MAX_GOP_FRAMES, matching the MPEG-2 parser's backstop.
- inject_unit_keys labelled a 2.1 FMTS disc as AACS 1.0 / bus-encryption
  off; FMTS is UHD-family, so synthesize the UHD version + bus encryption.
- The compiled Key Correction Data was a non-zero 16-byte constant fed into
  the Media Key derivation. Per the no-compiled-keys rule it is now all-zero;
  the chain still cannot complete on a real disc (documented), so this is
  behaviour-neutral — all variant tests pass unchanged.
- Fix stale doc references (broken `super::variants` intra-doc links, and
  `aacs::keys` comments) left by the module rename.
2026-07-09 14:14:20 -07:00
Matthew Jackson c43f23a759 mux: back-patch the MKV duration from the timeline when the source has none
A title whose scan yields no duration (HD-DVD — its `.MAP` timemaps are
not parsed, so DiscTitle.duration_secs is 0) produced an MKV with no
Segment Duration element, so players/MediaInfo reported an unknown
runtime.

Reserve a DURATION placeholder when the source declares none, track the
highest block timestamp written, and back-patch the placeholder at
finish() with the real muxed runtime (also enabling the per-track BPS
tags for these titles). Gated on duration_secs == 0, so BD/UHD/DVD —
which carry a real mpls/IFO duration — write it up-front exactly as
before, unchanged.
2026-07-09 12:52:48 -07:00
Matthew Jackson 770e0ab936 docs: changelog for HD-DVD feature composition + MPEG-2 AuAssembler 2026-07-09 09:12:56 -07:00
Matthew Jackson 5557765a86 disc: compose the HD-DVD feature from its VTI clip table
HD-DVD Standard Content splits the main feature across clips at the
layer break (FEATURE_1/FEATURE_2, or feature/feature_Divide). The scanner
enumerated one title per .evo, so main-title selection picked only part 1
(e.g. Shaun's 11 GB FEATURE_1, missing the 6.8 GB FEATURE_2).

Parse the HVA*.VTI navigation file's clip table — a fixed-stride record
list naming every clip in authored order, isolated by residue-mod-stride
rather than the imprecise header pointer — and concatenate the feature
clips (matched by the feature* naming convention) into one title whose
extents run in authored order. Every other clip stays its own title.
Falls back to one-title-per-clip when the VTI is absent or unparseable,
so nothing regresses on a disc with no readable navigation.

Validated on real discs: Shaun 17.8 GB / Anchorman 20.1 GB / Harry Potter
24.4 GB now enumerate as one 2-clip FEATURE title (largest = the movie).
2026-07-09 09:12:13 -07:00
Matthew Jackson 458b04ae63 mux: reassemble MPEG-2 access units via the shared AuAssembler
The MPEG-2 parser hand-rolled its own PES reassembly — a byte buffer plus
parallel PTS / source / discontinuity mark queues keyed by absolute
offset — duplicating what AuAssembler already does for H.264/HEVC/VC-1.

Add a Mode::Mpeg2 to AuAssembler (picture 0x00 with preceding sequence
0xB3 / GOP 0xB8 headers — the same headers-precede-picture shape as the
VC-1 mode) and have the MPEG-2 parser own one via AuAssembler::mpeg2().
parse() now feeds fragments to the assembler and processes each complete
access unit; the buffer, base offset, and three mark queues are gone. The
GOP-buffered temporal_reference reorder and PTS origin-locking are
unchanged. The parser's external contract is unchanged, so all existing
MPEG-2 parser tests pass as-is; new AuAssembler tests cover the MPEG-2
boundary rule directly.
2026-07-09 08:44:01 -07:00
Matthew Jackson c31ac6b66b docs: changelog for HD-DVD VC-1, sparse-PTS, variant layout (1.3.0) 2026-07-09 08:21:09 -07:00
Matthew Jackson 3e84ecfe42 mux: reconstruct display-order PTS for sparse-PTS program streams
HD-DVD EVO (and DVD VOB) program streams timestamp video at GOP
granularity: only one access unit per GOP carries a PES PTS. The H.264 /
HEVC / VC-1 parsers collapsed a missing PTS to 0, so on such a source
every non-anchor frame landed on the same block timestamp and a decoder
reported "non monotonically increasing dts".

Add a shared SparsePtsReorder that rebuilds a display-order PTS per frame
from the coded picture type (I/P/B) plus the sparse anchor PTS, with a
per-frame duration self-calibrated from the spacing between consecutive
GOP anchors (no external frame-rate needed). Display order is derived via
the classic single-anchor-delay rule (an anchor displays only after the
previously-held anchor; a B displays immediately), exact for the
non-hierarchical GOP structures HD-DVD H.264/VC-1 use. It mirrors the
MPEG-2 parser's GOP-buffered origin-locking.

Gated to the program-stream path only: the three parsers enable it via
with_ps_reorder(is_dvd_ps), so the BD/UHD transport path (per-frame PTS)
is byte-identical and untouched.
2026-07-08 21:54:09 -07:00
Matthew Jackson 998f82dc07 aacs(2.1): pin variant record layout against reference MKBs
Correct the variant Media Key chain against two real variant MKBs
(Zombieland v70, Stand By Me v70): C for Kmp is the per-slot block of
the 0x0c cvalue table indexed by the matched subset-difference, not the
0x2d head. The 0x2d record is the VARIANTS table (leading body-16 bytes,
one big-endian u16 per subset-difference) followed by a trailing 16-byte
Nonce, with no leading header. VKD stays at 0x2f.

Route record-type selection through the named REC_* consts in mkb rather
than bare hex, and document that Key Correction Data is per-licensee: no
universal constant exists and none is compiled in, so on a real disc the
chain yields a wrong Media Key that the Verify-Media-Key gate rejects
rather than emitting a bad key. This is a key-acquisition gap, not a
code gap.
2026-07-08 21:01:57 -07:00
Matthew Jackson b80af33775 mux: HD-DVD VC-1 demux via extended stream id 0xFD
VC-1 HD-DVDs (e.g. Shaun of the Dead) carry video on MPEG-PS extended
stream id 0xFD, with the real stream selector in stream_id_extension
inside the PES extension. Parse that field so the video routes to a
distinct track (pid 0xFD00|ext) instead of being dropped.

Reframe VC-1 access units in AuAssembler with a dedicated Mode::Vc1:
an AU is delimited by the next frame BDU (0x0D) once a frame has already
been seen, so the sequence (0x0F) and entry-point (0x0E) headers that
precede an I-frame stay attached to the frame they describe. The old
single-start-code split stranded those headers on the prior AU, which
the decoder reported as bits-overconsumption and hard decode failures.

hddvd probe now tracks the video pid it detects and emits VC-1 on 0xFD.
2026-07-08 21:01:48 -07:00
Matthew Jackson 054902cefe release: prepare 1.3.0
Bump to 1.3.0 and add the 1.3.0 changelog entry (FMTS/HD-DVD formats, AACS 2.1 variant chain, recovery seam, aacs module split, main-title-by-size, and the fixes since v1.2.2). Not tagged or pushed.
2026-07-08 14:48:50 -07:00
Matthew Jackson aa277224a9 fix: assorted correctness fixes and dead-code cleanup
- aacs/resolve: a media-keys-only provider missing the VID classifies as
  VidUnavailable, not NoMaterial (an MK derives the VUK once the VID
  arrives).
- disc/bluray: mark a clip seen only after its .clpi parses, so a
  transient parse failure on the first PlayItem cannot suppress the
  clip's extents for a later PlayItem referencing it that succeeds.
- disc/patch: log rather than swallow mapfile record/flush failures on a
  reverify downgrade, so a failed persist cannot silently mismark a bad
  unit good on resume.
- mux/ts: flag a discontinuity when a partial PES is dropped, matching
  the other partial-drop paths.
- mux/demux_thread: the no-demuxer branch forwards an empty batch for
  early consumer-disconnect detection instead of reading the whole disc.
- io/pipeline: correct the send-timing log (as_secs_f64, not as_micros
  printed as ms).
- aacs/derive, aacs/variant, disc/read_error, keysource: comment/doc
  accuracy. sector/prefetched, udf: remove dead fields/functions.
- mux/disc: assert unit-aligned read counts in the test.
2026-07-08 14:44:15 -07:00
Matthew Jackson 26154b6977 sector: generic recovery seam; FMTS forensic segments as decrypt loss
Replace the AACS-specific inline key-fetch in the decrypt decorator with
a scheme-neutral recovery seam: the input stream (L3) installs a Recover
closure (none / AACS key-fetch) and the decorator (L2) runs it at the
single decrypt-miss point. FMTS (AACS 2.1) forensic-segment units that no
key opens are just undecryptable units, concealed and counted as ordinary
decrypt loss with no FMTS-specific branch ("a loss is a loss"), so the
separate bytes_undecryptable bucket collapses into one loss count.

- sector/recovery.rs: the seam (MissOutcome, none/key_fetch factories),
  naming no encryption scheme in its type.
- FMTS: segment routing primitives + BYPASS_FMTS_KEY, and an upfront
  ensure_forensic_segments_decryptable gate (Error::FmtsKeyMissing) in
  the mux input path, parallel to the unit-key gate.
- CSS descramble/rekey moves from decrypt_sectors into
  css::descramble_region: CSS self-recovers from the data itself, so it
  stays OFF the seam (which is only for external inputs).
- disc/mod.rs also: main-title selection aligned to largest physical
  size; is_regular read from the open file handle, not metadata(path),
  fixing a swallowed sync_all on a fresh-rip ISO. decrypt_threads()
  resolved once via OnceLock off the per-buffer hot path.
2026-07-08 14:44:03 -07:00
Matthew Jackson bb90991e05 labels: reader-backed detection, Criterion fix, menu-language fallback
- DetectFn now takes a SectorSource so a parser can inspect a jar's
  central directory in detect() instead of firing on "any BD-J jar".
  dbp/deluxe do the real com/<vendor>/ prefix check up front, so each
  claims only its own discs (foundational for scaling the registry).
- criterion: treat a stream-map value of 0 as unmapped and synthesize a
  real 1-based number, so a 0 can't shadow or collide with a genuine
  stream 1 (with regression tests).
- png_filenames: new Low-confidence, last-resort parser reading menu
  language from {title}_UHD01_{LANG}_Composite artwork; sits below the
  MPLS floor so a real framework parser always wins.
- vocab: add menu_lang() for 639-2/B to 639-2/T menu-token normalization.
2026-07-08 14:43:52 -07:00
Matthew Jackson 175f131008 aacs: parse FMTS SegmentKey.tbl container (16-bit variant-indexed)
New aacs::segment_key: parses the AACS 2.1 SegmentKeyNNNNN.tbl container, confirmed against a retail disc as an 8-byte header + 65536 records of 536 bytes, indexed by the 16-bit variant selector the Media Key Variant chain produces. This is the confirmed link between the two 2.1 variant layers (selector picks the device's per-segment variant). The per-record 528-byte payload layout is not yet reversed.
2026-07-07 19:18:33 -07:00
Matthew Jackson af2f750629 aacs: parse FMTS forensic segment map (IndividualSegment.tbl)
New aacs::segment: parses the AACS 2.1 IndividualSegment.tbl into the source-packet ranges of the forensic variant segments (validated against a retail disc: 792 segments, 2560 packets each). First piece of the FMTS variant decoder — the segments' variants are encrypted under segment keys, not the unit key, so a unit-key rip corrupts them (broken HEVC refs).
2026-07-07 18:59:09 -07:00
Matthew Jackson f1517f7313 disc: select main title by largest size, not clip count
canonical_title_order keyed on clip-count ascending before duration, so a short 1-clip bonus reel outranked the real feature — which is often chaptered into many clips (one per chapter). Make physical size the primary key: the main feature is the largest non-oversize title; decoy 'play-all' playlists run long but tiny (reused clips) and self-eliminate.

Validated across 23 UHD/BD discs: fixes t1 on F9, Fast Five, Fast & Furious 6, and Furious 7 (feature was ranked #13-36); no regressions on the 19 already correct.
2026-07-07 18:36:36 -07:00
Matthew Jackson 984774b068 disc: first-class FMTS + HD-DVD formats; CPI sample selection
Add DiscFormat::Fmts (AACS 2.1) and DiscFormat::HdDvd as first-class peers. Format derives from the AACS MKB generation (mkb_type().generation(): V10=BD, V20=UHD, V21=FMTS), reusing existing AACS code, and from the on-disc tree for HD-DVD/DVD. One detector (detect_disc_format) shared by the coarse DiscId probe and the full scan — no more 'default BluRay, defer to full scan'.

FMTS is a BD-tree stream variant: parse_playlist resolves the clip stream via CLIP_STREAM_EXTS (.m2ts -> .fmts -> .ssif), so the .fmts main feature yields real extents (previously silently empty). HD-DVD is a tree-level peer with its own enumerator (disc/hddvd.rs): HVDVD_TS/*.evo -> MpegPs titles with real extents (playlist/stream parsing honestly stubbed).

Sample selection for key resolution now uses the authoritative AACS CPI flag (aacs_unit_encrypted, byte-0 & 0xC0) not the ts_sync_destroyed heuristic — container-agnostic (M2TS/FMTS/EVO; TS-sync is meaningless on HD-DVD program streams) and stops the key-service '0 encrypted units' rejection.

Tests live with each format (bluray/hddvd/mod); generic UDF fixture builders extracted to a shared udf::fixture module.
2026-07-07 18:16:04 -07:00
Matthew Jackson ddc44e1105 unlock_bridge: run_features/run_bus dispatch; report LibreDrive vs Renesas
Adopt freemkv-unlock's split Unlocker trait: run_features drives the drive-prep capability, run_bus the content bus removal, each iterating unlockers until one doesn't decline (NotApplicable = try next; Ok or a real error stops).

unlocker_matrix now reports which drive-prep unlocker actually ran — LibreDrive removes the bus at the drive; Renesas unlocks features but leaves the bus to the cert. Wire product_id through to fu::DriveId.

Bump to 1.2.3.
2026-07-07 16:36:05 -07:00
Matthew Jackson b8a51ef1c7 aacs: remove boil veneer; move key newtypes to aacs::types
aacs::boil was a thin newtype veneer over aacs::derive — a duplicate layer.
Delete it: move the shared key newtypes (Vid, MediaKey, Vuk, ProcessingKey,
UnitKey) into aacs::types, and expose resolve_candidate + KeyCandidate from
aacs::derive directly. Downstream (keysource, disc::encrypt) now import from
aacs::{derive,types}. Pure API consolidation, no behaviour change; full test
suite green on Rust 1.86.
2026-07-05 12:09:39 -07:00
Matthew Jackson 08cb80aac1 aacs: keep flat re-exports for the typed key primitives
The module structure is the public API, but the typed key primitives
(MediaKey/UnitKey/Vid/Vuk, mk_from_dk/mk_from_pk/uk_from_vuk/vuk_from_mk),
derive_vuk, the aligned-unit decrypt entry points, and DeviceKey/HostCert
are load-bearing names that downstream key-source crates import through the
flat `aacs::` path. Re-export them here so those crates can track the
module refactor without a lockstep re-pin.
2026-07-04 22:19:39 -07:00
Matthew Jackson 60d0ebc8a1 aacs(2.1): variant Media Key as a clean PK -> Km primitive
Redesign derive_media_key_variant to the minimal derivation surface:

  derive_media_key_variant(mkb_records, pk) -> Km

- PK-input only. Deriving Kp from device keys (DK -> PK) is a separate
  concern (walk_processing_key); a leaked 2.1 key is a PK, and the chain
  starts at Kp. A bare PK arrives without its subset-difference slot, so
  the primitive tries it against every slot and returns the Km for the
  slot that passes the MKB's Verify-Media-Key record -- mirroring the
  classical bare-PK derive_media_key_from_pk, gated by the chain's own
  verify so an unverified key is never returned.
- VID-free: the Media Key is MKB-scoped. VUK stays the separate
  derive_vuk(Km, VID) step.
- KCD is a fixed algorithm constant compiled in, not a caller parameter
  (removes the kcd argument, the placeholder const, and KcdNotProvided).
- Soft-correction / online-challenge slots are treated as non-covering;
  surfaced over the generic miss so a disc needing those modes is
  distinguishable from a non-covering key.

resolve_keys_v21 updated to walk DK -> PK first, then call the primitive
and derive the VUK from Km + VID. Module + helper docs refreshed to the
pinned record layout; tests reworked for the PK-only signature.
2026-07-04 19:16:40 -07:00
Matthew Jackson 5dee947007 aacs(2.1): wire variants_for_uv from the 0x2d record; chain runs end-to-end
variants_for_uv previously returned None (placeholder), dead-stopping the
Media Key Variant chain at VariantsTableUnavailable on any real disc. Layout
now pinned against two real 2.1 variant MKBs (Zombieland v70, Stand By Me v70):
the 0x2d Encrypted-Media-Key-Variant-Data body is sd_count u16 VARIANTS entries
(1:1 with 0x0c cvalues / 0x04 subset-diffs) followed by the 16-byte per-disc
Nonce at the tail. variants_for_uv reads the sd_slot_index-th u16.

With this the chain runs fully: Kmp -> Kpnew(=Kmp^KCD, extracted CyberLink
constant) -> VKD(0x2f) -> Km -> Kvu. The only remaining input is a covering
2.1 Processing Key to validate against a known answer; until then the final
Verify-Media-Key (0x86) gate rejects any wrong layout pick, so a bad key can
never be emitted. Tests updated to reflect the wired lookup; 1.86 precommit
green (fmt+clippy+tests).
2026-07-04 18:26:09 -07:00
Matthew Jackson a1796e8345 aacs: point integration tests at real module paths
The facade removal (5cf4252) left tests/ calling aacs::disc_hash,
aacs::decrypt_unit, aacs::AacsVersion, etc. at the old flat paths,
so the branch did not compile its integration tests. Repoint each
to its real module (inf/derive/content/mkb). No logic change.
2026-07-04 18:25:59 -07:00
Matthew Jackson 5cf4252e44 aacs: remove the mod.rs facade — module paths are the public API
Delete the pub-use re-export facade; consumers now import from the owning
module (aacs::content::decrypt_unit, aacs::mkb::MkbType, aacs::derive::derive_vuk,
aacs::boil::mk_from_dk, aacs::resolve::resolve_keys_v2, ...). Internal callers
repointed accordingly. Path-only change; logic hash identical (95fb9924); 2210
tests green.
2026-07-04 14:19:40 -07:00
Matthew Jackson b5f9c4e426 aacs: merge media_key + volume_key into derive.rs
The volume_key module was only 34 lines and is just the tail of the same
DK/PK -> MK -> VUK -> UK derivation ladder as media_key. Fold both into one
derive module so every aacs module is a substantial, distinct responsibility
(crypto/mkb/derive/inf/content/variant/resolve). Relocation only; logic hash
identical (95fb9924); 2210 tests green.
2026-07-04 14:14:56 -07:00
Matthew Jackson 82dca307ad aacs: split keys.rs god-module into media_key/volume_key/inf/resolve
Break the 2800-line keys.rs into four responsibility-scoped modules:
- media_key.rs: DK/PK -> Media Key subset-difference walk (+ probe harness)
- volume_key.rs: VUK derivation, unit-key unwrap
- inf.rs: Unit_Key_RO.inf parsing, disc_hash, content cert, in-drive MKB read
- resolve.rs: the resolve_keys_* orchestration (keys.rs renamed)

Relocation only; the (white-box) test suite stays in resolve.rs and pulls
the moved items via glob imports. Proven byte-identical to the pre-refactor
state via the logic hash (95fb9924); 2210 tests green.
2026-07-04 14:06:57 -07:00
Matthew Jackson a9a21d8139 aacs: extract mkb.rs (MKB record format, MkbType, finders)
Relocate the shared MKB machinery into a single mkb module: the record
framing walker + MkbRecord view (from variant), the MkbType/AacsVersion
classification, the MKB-file utilities, and the record-body finders (from
keys). Fixes the inversion where the MKB parser lived in the 2.1-only
variant module. variant.rs keeps its local MkbRecord-based mkb_find_mk_dv
(name collision with the raw one; unified in the dedup follow-up).

Relocation only. Proven byte-identical to the pre-refactor state via the
function-body logic hash (imports normalized out); 2210 tests green.
2026-07-04 13:55:06 -07:00
Matthew Jackson 8dc7075356 aacs: extract crypto.rs (shared AES primitives + constants)
Relocate the shared low-level primitives into a single crypto module:
aes_ecb_encrypt/decrypt, aes_cbc_decrypt, aes_g (from content/variant) and
aesg3 + AESG3_SEED (from keys), plus AACS_IV. Fixes the scatter where AES-G
lived in the 2.1 file and AES-G3 in keys. Relocation only — no rename, no
logic change (logic-hash identical to baseline; 277 items; 2210 tests green).
2026-07-04 13:37:53 -07:00
Matthew Jackson 02218a6511 aacs: rename decrypt.rs->content.rs, variants.rs->variant.rs (no logic change)
Pure file+module-path rename. 'content' names the AACS unit-decrypt layer
(distinct from the top-level sector-decrypt driver crate::decrypt), and
'variant' (singular, spec term 'Media Key Variant') names the 2.1 chain.
Logic-hash identical to baseline; 277 items intact; tests green.
2026-07-04 13:24:26 -07:00
Matthew Jackson e32a3af898 aacs: annotate crypto with AACS spec section citations
Add [C]/[PR]/[BD]/[libaacs] §x.y provenance markers across the AACS
crypto so each primitive links to the spec section it implements, with a
source-tag legend in mod.rs. Doc-comments only — no logic, constant, or
signature changes.

Also: correct two stale record-type comments in variants.rs (0x82/0x83 →
the real 0x2d/0x2f) and document the Variant Number width (spec lsb_10 vs
the 2.1 chain's lsb_16, driven by the 65,535-entry VKD table).
2026-07-04 13:15:53 -07:00
Matthew Jackson 1253db037f restore freemkv-unlock path dep for local dev (post-v1.2.2) 2026-07-03 20:50:17 -07:00
Matthew Jackson 4b878dcc06 v1.2.2: bump version (freemkv-unlock git-pinned for the tag) 2026-07-03 20:50:14 -07:00
Matthew Jackson 7511b0b808 changelog: 1.2.2 (AACS 2.1 variant MKB, mk_from_dk fix, PK-path perf, autorip online-outage + move-error clear) 2026-07-03 20:48:04 -07:00
Matthew Jackson 707989bd68 aacs: target real AACS 2.1 Variant MKB records (0x2d/0x2f/0x0c)
The Media Key Variant scheme is detected and parsed via the actual MKB
record types found on a real variant disc:
  - 0x2d  Encrypted Media Key Variant Data (C)
  - 0x2f  Variant Key Data table (65,535 x 16)
  - 0x0c  variant cvalues (one per 0x04 subset-difference slot)

Replaces the earlier placeholder 0x82/0x83 record types, which were a
guess and appear on no real MKB. is_variant_mkb, the record finders, and
the subset-difference cvalue source (now 0x0c, falling back to 0x07/0x05)
are updated accordingly, along with the V20->V21 upgrade detection in
resolve_keys_v2 and its fixtures.

The variant chain still halts at variants_for_uv (the VARIANTS[uv] /
Nonce sub-field offsets need a covering key to confirm end-to-end
against the 0x86 verify), so a best-effort offset is never silently
trusted. All 27 variant tests pass on Rust 1.86.
2026-07-03 13:25:01 -07:00
Matthew Jackson 57b2f0c4f9 aacs: make the PK path terminal (no SD-tree walk)
A Processing Key is the key at its Subset-Difference node — one AES-G from
the Media Key — so it is tried directly against the MKB cvalue tables,
matching libaacs _calc_mk_pks (iterate PKs × cvalues). The prior code
treated every PK entry as a device-node label at unknown depth and BFS-walked
the SD tree (depth 3, capped 5), which was both wrong for terminal PKs and
~15x slower on a large UHD MKB (~181k cvalues): PK derivation on UHD dropped
from ~37s to ~2.4s.

The Subset-Difference tree walk now lives solely in the device-key path
(derive_media_key_from_dk), which owns per-node path bits; the PK path never
descends. Removed PK_WALK_MAX_DEPTH / _CAP / _walked / walk_pk_against_tables_impl;
renamed the core scan try_pk_against_tables and its probe test.

Precommit (Rust 1.86): fmt + clippy + tests green.
2026-07-02 23:17:20 -07:00
Matthew Jackson 47406a7e32 aacs: full-ladder resolve_candidate + correct mk_from_dk
resolve_candidate(candidate, mkb, unit_key_ro, vid) -> ResolvedChain: one
composed, PURE-DERIVATION boil-down for a candidate key at any ladder rung
(KeyCandidate::{Uk,Vuk,Mk,Pk,Dk}, each carrying the module's newtype). Walks
DK/PK/MK/VUK -> terminal unit keys, parsing Unit_Key_RO.inf at the version
the disc's MKB declares, and returns every CPS unit key as (cps_unit, key)
(matching ResolvedKeys) plus the intermediate chain. No sampling/validation
(that's the caller's unit_key_validates) and no position recovery (Dk is a
positioned DeviceKey; recover_dk_position first). Adds ProcessingKey newtype.
Consumers stop re-composing the ladder; every client hardens one impl.

Correct mk_from_dk: real Subset-Difference walk (derive_media_key_and_pk_from_dk)
instead of the Media-Key-Variant path, which needed integrator KCD absent
in-tree and Err'd for every real disc -- dead for both consumers. Drops the
now-unused vid arg.
2026-07-01 22:34:52 -07:00
Matthew Jackson 03cbda741a restore freemkv-unlock path dep for local dev (post-v1.2.1) 2026-07-01 19:08:56 -07:00
Matthew Jackson ec59ffe105 v1.2.1: bump version (freemkv-unlock git-pinned for the tag) 2026-07-01 19:08:53 -07:00
Matthew Jackson 02132f52b5 changelog: describe the shipped re-base DTS fix + faithful-passthrough note 2026-07-01 19:02:04 -07:00
Matthew Jackson ca8d046f2a mux/dts: re-base PTS per PES instead of a running clock (fix long-title drift)
The first cut used a global running clock (max(next, own-PES PTS) + advance),
which fixed the same-PES collision but DRIFTED: once accumulated frame
durations exceeded the PES-timestamp spacing, it never re-based, so a
feature-long DVD DTS track ran minutes past its real length (2h44 for a 2h03
film) while AC-3 from the same source stayed exact.

Match the AC-3 path: re-base to each PES's own container timestamp, and advance
by one frame duration ONLY within a run of AUs sharing one PES. Fixes the DVD
multi-frame-per-PES collision without drift; the UHD DTS-HD MA per-PES
attribution (da85f56) is preserved (each AU still takes its own core PES's PTS).
Adds new_pes_rebases_to_its_own_pts_no_drift; full mux suite green (905).
2026-07-01 18:21:25 -07:00
Matthew Jackson f33a9f6c22 v1.2.1: bump version + changelog (DVD DTS timestamp fix) 2026-07-01 17:30:25 -07:00
Matthew Jackson a9aee0d5a4 mux/dts: monotonic per-frame PTS + real frame duration (fixes DVD DTS)
DVD packs several DTS core frames into one PES; the parser stamped every
access unit with that single PES PTS and duration_ns=None, so consecutive
frames collided on one timestamp — ffmpeg rejected the output as 'non
monotonically increasing dts to muxer: X >= X' (deep-decode = corrupt,
e.g. The Punisher). The UHD DTS-HD MA path (one AU per PES, distinct PTS)
was unaffected, which is why this only surfaced on DVD.

Parse the DTS core header for samples ((NBLKS+1)*32) and sample rate
(SFREQ, 48kHz fallback) to derive each AU's duration, and stamp a running
monotonic PTS: max(next_clock, own-core-PES PTS), then advance by the
frame duration. A later PES whose PTS is ahead of the clock still wins
(preserves the UHD per-PES attribution from da85f56/571c613); frames
sharing one PES advance frame-by-frame instead of colliding.

Tests: the 3 that encoded 'same PES -> same PTS' now assert monotonic
advance; new dvd_many_cores_one_pes_are_strictly_monotonic reproduces the
Punisher bug; duration/SFREQ-fallback unit tests added.
2026-07-01 17:17:35 -07:00
Matthew Jackson e16f207efa restore freemkv-unlock path dep for local dev (post-v1.2.0) 2026-07-01 16:53:06 -07:00
Matthew Jackson e5540da6d4 v1.2.0: bump version (freemkv-unlock git-pinned for the tag) 2026-07-01 16:53:03 -07:00
Matthew Jackson 9b49beabee changelog: DVD DTS/LPCM silent-audio fix + Pass-N recovery specialists + flat scheduler 2026-07-01 16:50:08 -07:00
Matthew Jackson b0037f2db9 recovery: flat-pool bandit scheduler (FREEMKV_PATCH_FLAT, opt-in)
The tier ladder (bucket->mug->thimble) forces tier 0 to attempt ALL
ranges before tier 1, and tier 1 all before tier 2 — so on a hardened
residual (late resume, flood already gone) the marginal specialists are
unreachable for hours: you bail near-empty buckets over hundreds of dead
ranges before a thimble ever touches water.

FREEMKV_PATCH_FLAT collapses the 3 tiers into ONE flat pool of all 16
handler configs. run_handlers already sorts best-first by the live
decayed-yield scorecard, so this is a data-driven bandit: the first
ranges try them all (explore), the ranking floats whatever is actually
landing sectors to the front (exploit), re-measured per range; a handler
that doesn't fit stays last but is never dropped (floor -> can revive).
The new handlers self-limit (yield after 4 dead reads), so the flat
scheduler does NOT reintroduce the old depth-first per-range starvation.

Opt-in; unset keeps the proven tier ladder as the default for 1.2.0.
2026-07-01 16:44:29 -07:00
Matthew Jackson 90070c81bd test: widen profile_03 efficiency guard for tier-2 roster
The tier-2 marginal specialists now additionally probe permanently-bad
residual sectors before leaving them NonTrimmed, so the alternating-bad
profile's finite read count rose past the old tier-0/1 bound. Guard
still catches runaway; behavior asserts unchanged.
2026-07-01 13:55:20 -07:00
Matthew Jackson b8c2054249 recovery: wire tier 2 marginal specialists into the Pass-N chain
Add a third breadth-first tier (PATCH_TIERS 2->3) that runs the marginal
specialists on the hardened residual tiers 0-1 leave: SlowSpin (Linear
fwd+rev @ min), FuaRetry (Linear fwd+rev+Bisect @ FUA), SlowFua (Linear @
min+FUA), CachePrime, Oscillate (@ max and @ min), SpeedSweep. Every read is
a wedge-safe read_span, so they inherit wedge-abort / unproductive-yield /
deadline for free. All are new configs, so the EWMA scorecard calibrates each
once then ranks by decayed rate — a specialist that doesn't fit self-
deprioritises. Tiers 0-1 (fast scouts, slow-deep) are unchanged; this is
purely additive. Also switch the scorecard log sort to sort_by_key.

cargo test -p libfreemkv green (2200 passed).
2026-07-01 13:50:35 -07:00
Matthew Jackson c4357d9d87 recovery: CachePrime + Oscillate handlers + fixtures
CachePrime reads the good run immediately preceding a residual island to lock
the servo/PLL, then reads the island warm (boundary sectors the drive can't
cold-seek). Oscillate reads each residual sector by alternating approach —
forward-into then reverse-into — for direction-dependent tracking. Both go
through the wedge-safe read_span (primes included). FakeDisc models a
direction-dependent sector and a servo-primed boundary sector; fixtures prove
a forward/cold Linear misses each while Oscillate/CachePrime recover them.
2026-07-01 13:50:35 -07:00
Matthew Jackson 63c1d5db63 recovery: FuaRetry + SlowFua fixtures (FUA / min+FUA levers)
FuaRetry (Linear fwd+rev+Bisect at {max,fua,deep}) and SlowFua (Linear at
{min,fua,deep}) are Linear/Bisect at FUA params, no new struct. FakeDisc now
models a stochastic sector that lands only on the Nth physical (FUA) read (a
cached re-read keeps missing) and a hardest sector needing BOTH min speed and
FUA. Fixtures prove cached reads keep missing while the FUA group lands the
stochastic sector, and that neither lever alone recovers the hardest sector —
only the min+FUA combination does.
2026-07-01 13:50:35 -07:00
Matthew Jackson e91ce25db3 recovery: SpeedSweep handler + slow-spindle fixtures (SlowSpin/SpeedSweep)
SlowSpin is Linear pinned to min speed (no new struct). SpeedSweep is a new
per-sector handler that tries Max->Min until one reads (speed resonance).
FakeDisc now models a sector that reads ONLY at min speed; fixtures prove a
max-speed Linear misses it while SlowSpin and SpeedSweep recover it (and that
SpeedSweep tries fast-then-slow).
2026-07-01 13:50:35 -07:00
Matthew Jackson 2c94ce97c6 recovery: decayed (EWMA) handler scorecard so the ranking flips
Grade handlers by an exponentially-decayed recovery rate (bytes/second,
alpha=0.5) instead of a cumulative rate. The residual hardens mid-pass, so
the best technique changes: cumulative froze the early winner in the lead
forever; the EWMA re-prices continuously — a handler that stops earning
decays down, a late-starting specialist climbs. Keeps rank()'s untried->top
one-shot calibration and attempted-but-zero-time->bottom.

New flip fixture proves a handler that recovers a lot early then nothing
loses its lead to one that starts recovering later.
2026-07-01 13:50:35 -07:00
Matthew Jackson e7998a1376 recovery: parameterize the read primitive with ReadParams (speed/FUA/timeout)
Add ReadParams { speed: SpeedPref, fua: bool, timeout: TimeoutPref } and
thread it through read_span so every wedge-safe handler read can request a
spindle speed (SET CD SPEED issued only on change, restored to max when the
handler exits), set the READ(10) FUA bit, and pick the 10s vs 60s timeout.

- SectorSource gains read_sectors_fua (default ignores fua); Drive sets the
  CDB bit, DecryptingSectorSource threads fua to its inner read.
- recovery_read gains a fua param.
- Linear becomes { direction, params }; Bisect/Jump take params. Existing
  tier-0/1 instances keep identical behavior (max speed, no FUA, fast/deep).
- Scoreboard keys on the full-config String name (linear:fwd:max:fast, ...).
- FakeDisc observes speed + FUA + approach so specialist techniques are
  provably exercised in later commits.

cargo test -p libfreemkv green (2193 passed).
2026-07-01 13:50:35 -07:00
Matthew Jackson ba6ac3ccdb dvd: route private_stream_1 audio sub-ids by positional stream number
The private_stream_1 sub-stream id low nibble is the DVD audio-stream
NUMBER (0-7), shared across codecs, not a per-codec ordinal. A DTS (or
LPCM) track that isn't the disc's first audio stream got a sub-id one
too low, so the demux routing key (0xBD00 | sub_id) never matched and
every packet was dropped -> present-but-silent track. AC-3 at position
0 coincidentally worked (ordinal==position). Route by positional index.

Fixes silent DTS 5.0 track on The Punisher (AC-3 5.1 @ pos0 plays,
DTS 5.0 @ pos1 was 0xBD88, real wire id 0xBD89).
2026-07-01 13:49:55 -07:00
Matthew Jackson 1d1fe984d7 lib: remove standalone sector-health verify feature
Delete src/verify.rs (standalone 'read every sector, classify health'
module) + its lib.rs pub mod/pub use. It was old, unused code. The
decrypt-verify gate (disc::verify / UnitVerifier) is a DIFFERENT module and
stays. Relocated the pure chapter_at_offset helper (used by locate_ranges +
autorip's done-card) into disc/mod.rs as a pub fn.
2026-07-01 13:09:21 -07:00
Matthew Jackson 9298534b4b recovery: audit fixes — live at-risk excludes NonTried, latency-gated wedge, drop dead block counters
- Live located drilldown (sweep + patch progress snapshots) now excludes
  NonTried, matching the one-shot path. Including the unread remainder made
  main_at_risk_ms show ~full-movie at sweep start and melt to 0 as it
  progressed — unread is unknown, not damage.
- Wedge abort now requires the failure to be FAST (< WEDGE_FASTFAIL_MS,
  500ms) as well as wedge-family sense: a real uncorrectable sector on
  Hardware-error media spends ECC-recovery time before failing, so it no
  longer false-trips the fast-fail wedge abort. New regression test.
- Removed the always-zero blocks_attempted/read_ok/read_failed/
  unreadable_count from PatchOutcome + PatchLoopState (dead residue from
  the old grind loop; the HandlerScoreboard supersedes them) so the
  patch.done log no longer emits misleading zeros.
2026-07-01 12:07:08 -07:00
Matthew Jackson 6799890379 docs: 1.2.0 changelog — Pass-N handler chain, DVD drive-prep unlock fix, unlocker matrix 2026-07-01 11:05:31 -07:00
Matthew Jackson f36163ad02 disc: unlocker_matrix reports 'did work this rip', not 'matched'
yes now means the unlocker actually ran and did its job: LibreDrive from
the runtime firmware-unlock success, AACS host-cert only when LibreDrive
didn't do the bus (stock-drive fallback), CSS from the crack succeeding.
On a LibreDrive UHD that correctly reads LibreDrive: yes, AACS: no (LD
removed the bus, the cert route never ran) instead of the misleading
AACS: yes. Names stay registry-driven; runtime logic lives here.
2026-07-01 10:44:39 -07:00
Matthew Jackson 73ae12ed1f drive: disable FUA on READ(10) — the ~10x sequential-read regression
Forcing FUA (Force Unit Access) on EVERY READ(10) bypassed the drive's
readahead/streaming cache on the bulk Pass-1 sweep, collapsing sequential
throughput ~10x (UHD 15-25 → ~2 MB/s, DVD → ~0.5 MB/s), disc-type-
agnostic — the cache IS the streaming throughput. This was the real speed
regression (not unlock/riplock, which the earlier chase suspected). Clear
byte-1 bit 0x08. FUA will return as a dedicated Pass-N recovery handler
that sets/clears it per marginal-sector re-read, where cache-masking of a
stochastic sector actually matters — never blanket on the bulk path (#55).
2026-07-01 10:26:56 -07:00
Matthew Jackson 2f47b552e8 disc: Disc::unlocker_matrix() — registry-driven unlocker report
Returns each registered unlocker's name + whether it applies to this
drive+disc (via the unlock bridge over all_unlockers()). Disc crypto kind
derived in the library so the CLI and autorip render an identical, always-
current report with no hardcoded names. Also adds a Drive::unlocker_name()
runtime getter.
2026-07-01 10:00:59 -07:00
Matthew Jackson 89c4acdf8f drive: run drive-prep unlock for EVERY disc, incl. DVD (fix riplock regression)
The v1.0.0-rc.1 unlocker refactor added an `if disc_is_dvd() { return }`
early-return in Drive::init() that skipped the whole drive-prep unlock for
DVDs. That firmware unlock is what lifts riplock and readies max read
speed — a DRIVE-level, disc-independent feature — so skipping it left
every DVD stuck at stock/riplock speed (~0.4x, 3h ETA). UHD was unaffected
because it flows through the unlock.

Remove the skip: init() now runs the identity-keyed drive unlocker for all
discs (disc kind is Unknown at init, so only the drive unlocker matches;
the AACS host-cert handshake and CSS bus-auth still run later, gated on the
real disc kind, on top of the unlocked drive). Speed stays where it
belongs — SET CD SPEED(0xFFFF) at pass-1 start (disc/mod.rs) — not in the
unlocker. Also drop the matching probe_disc DVD skip.

Reverts the SET STREAMING stopgap (separate freemkv-unlock revert): that
treated the symptom; the real bug was the skipped unlock. Drive features
come from the firmware unlock, not a stock speed CDB.
2026-07-01 09:38:51 -07:00
Matthew Jackson 1d5a730853 recovery: fresh-eyes audit fixes (handlers + Pass-N engine + sweep)
Handlers (section_recover.rs):
- Bisect expand loops now honor ctx.halted() (were deadline-only, so a
  Stop could hang up to 60s vacuuming a readable island).
- read_span: explicit Transport arm so a bus-abort read isn't counted as
  unproductive grinding; debug_assert the sector-aligned span invariant.
- Scoreboard rank: an attempted-but-zero-time handler (e.g. returned Halted
  on its first check) now ranks BOTTOM, not top — it no longer crowds out
  proven performers.
- Document the wedge tier-size coupling + new regression test that a
  2-handler (tier-1) chain still catches a wedge via cross-section streak.

Pass-N engine (patch.rs):
- Rebuild PatchOutcome stats AFTER the post-read re-verify downgrade flush
  (was snapshotting before it, over-reporting bytes_good / recovered and
  risking a 'perfect rip' verdict on an imperfect one).
- Progress 'recovered' composes the still-bad set to MATCH work_total
  (subtract NonTried, add Unreadable) so the bar can't pin at 0 on a
  partially-swept disc or run backward on the Unreadable→NonTrimmed relabel.
- Remove dead work_done field; rewrite the stale 'adaptive batching' comment
  to describe the handler chain and mark block_sectors/full_recovery as
  informational-only.

Sweep (disc/mod.rs):
- Saturating arithmetic at the damage-jump position math (honor the
  read_error side's documented defence-in-depth guarantee).

Deferred (noted, need focused passes): fast_capture re-introduction,
Pass-1 halt-misclassified-as-jump, bytes_good display inflation, the
always-zero blocks_* telemetry, Pass-1 jump-on-first-error policy.
2026-07-01 09:14:32 -07:00
Matthew Jackson 0a0a8f1327 drive: apply drive-features for every disc, incl. DVD (fix riplock)
DVD ran the drive at ~0.4x stock speed: init() early-returned for DVD
before any speed was set, and its only speed command was the sweep's lone
legacy SET CD SPEED, which this BU40N ignores for DVD. BD/UHD got up to
speed only via the firmware unlocker's calibration — a path a DVD can't
take (it breaks stock CSS).

Call the new unlock-crate drive-features capability at init() for ALL disc
kinds, before the DVD stock-mode early-return, via a bridge shim. Stock
MMC only (SET STREAMING + SET CD SPEED), no bus unlock, so CSS is
undisturbed.
2026-07-01 08:56:19 -07:00
Matthew Jackson e396dcaab7 Pass-N: detect drive fast-fail wedge and abort instead of grinding
The 2026-07-01 overnight rip ground a wedged BU40N for 28 minutes at
0 B/s. Root cause: the handler chain's read_span split errors only into
Transport (bus gone) vs Bad (everything else), so the drive's fast-fail
wedge sense (ILLEGAL REQUEST 0x05/0x24 — it rejects every CDB in <100ms
without attempting recovery) was treated as an ordinary bad sector. The
chain hopped to the next section forever, and the existing wedge detector
in read_error.rs never saw it because the chain reads through
recovery_read directly.

Add a pass-level wedge streak: read_span counts consecutive wedge-family
(Hardware/IllegalRequest) senses; at WEDGE_ABORT_STREAK (16) it escalates
the read to Transport, which every handler already propagates as
TransportFault — aborting the whole pass and setting wedged_exit so the
caller spin-cycles. The streak is carried across sections via PatchCtx
(seeded into and read back from each per-section HandlerCtx) so a wedge is
caught even when every bad sub-range is smaller than the streak. Any Good
or non-wedge (medium-error) read resets it, so scattered bad sectors on
real media never trip it. New fixture test: a wholly-wedged 1000-sector
section aborts in <100 reads, not 1000.
2026-07-01 08:02:52 -07:00
Matthew Jackson f6f68cb9dc Bisect: halve at dead boundary + exempt boundary probes from stall
The expand loops broke out of the batch sweep on the first failing read,
overshooting the good/dead boundary by up to a batch and leaving the
readable sectors adjacent to a dead edge for the re-bisect to re-pin. On
top of that, early-yield counted those boundary-probe failures as a
stall, so the forward expand quit early AND the backward expand inherited
a poisoned unproductive counter and yielded instantly.

Now on a failed expand batch we halve the step (down to a single sector)
to recover right up to the dead edge in ~log2(batch) reads, drive the
expand loops off the deadline only (they self-terminate via halving), and
clear the unproductive streak once an island is located. patch_recovers_
good_middle_of_a_bad_range now recovers 50/50 good-middle sectors.
2026-06-30 22:06:04 -07:00
Matthew Jackson bdafec8081 section_recover: per-rip handler scorecard + Bisect-leads-scouts + proportional Jump
Scorecard: grade each handler by recovery rate (MB/s) per rip, order
best-first on later sections, log the ranking at pass end. Untried
handlers rank top so each is calibrated once before ranking narrows to
the winners. Ephemeral (reset per pass), no persistence.

Tier 0 scouts are now [Bisect, Jump, Linear-fast x2], scorecard-ordered.
Bisect leads: probing the MIDDLE of a range lands on a readable island in
one read where a linear scan grinds the dead front to reach it. Jump now
jumps to the middle of the REMAINING span (proportional) instead of a
fixed 8 MiB that leapt clean over small ranges and missed their readable
middles. Tier 1 is slow deep-recovery on the residue.
2026-06-30 21:26:35 -07:00
Matthew Jackson 71f6cb17a8 section_recover: per-rip handler scorecard + Bisect leads the scouts
Scorecard (ephemeral, reset each pass, no persistence): grades every
handler by recovery rate (bytes/sec). run_handlers orders handlers
best-first by that rate; an untried handler ranks top so each is
calibrated once, then the ranking narrows to the winners. Logged at pass
end (phase=scorecard) so the operator sees which handler is pulling the
weight on this drive/disc and which is a dud.

Tier 0 scouts are now [Bisect, Jump, Linear-fast x2], scorecard-ordered.
Bisect leads because probing a range's MIDDLE lands on a readable island
in one read, where Jump (linear from the front, big skip) can grind the
dead front or overshoot a small range entirely. The scorecard confirms
or overturns that order with real per-disc data.
2026-06-30 21:25:39 -07:00
Matthew Jackson 326868ec64 section_recover: Jump starts at 8 MiB base to clear big dead spots faster
Each dead probe read costs the drive's full ~10s timeout, so a large dead
region took a dozen escalating probes. Starting the jump at 8 MiB (vs
1 MiB) clears it in a handful; a skipped span stays bad for Bisect to
reclaim readable islands, so an over-jump loses nothing.
2026-06-30 21:12:12 -07:00
Matthew Jackson ebb3227294 patch: single Jump-scout tier 0, expand-Bisect, live in-handler progress
Tier 0 is now a single fast Jump scout: it streams the big readable
ranges back and skips dead runs in seconds, so the pass reaches every
section fast and converges to the small genuine-dead residue instead of
grinding three handlers x 60s on each dead fragment. Tier 1 (fast
mop-up + slow deep reads + Bisect) works only that residue.

Bisect now expands: on a good probe it reads outward forward and
backward in full batches until a read fails, recovering the whole
readable island in large reads; the two failing ends become smaller bad
sub-ranges it bisects again. One huge bad range becomes many precisely
located small dead clusters.

Progress heartbeat: HandlerCtx gains a throttled tick (250ms) called
from every read, pushing a fresh snapshot to the reporter DURING a
handler. The bar and speed now move continuously as recovery happens
instead of jumping once per section (the reason speed read 0 B/s and the
% looked frozen between range boundaries).
2026-06-30 20:57:36 -07:00
Matthew Jackson e8b3c719e6 patch: add Jump handler (lead fast tier) + recovery-based progress %
Jump: on sustained batch failures skip ahead an escalating distance
(1 MiB doubling to 256 MiB) to find where readable data resumes, leaving
the skipped span for Bisect to pin — mirrors the Pass-1 damage-jump. It
leads the fast tier so a large dead run is skipped in seconds instead of
the linear sweeps grinding every dead batch (10 s each) first; on a
readable range it just streams it back. Recovers readable data buried
behind a big dead front (the 192 MB Dune range).

Progress %: report bytes RECOVERED (initial-bad minus still-pending)
instead of a per-range counter that only advanced on the final tier — so
the bar reflects the readable bulk recovered during tier 0 the instant it
lands, matching the 'MB remaining' number.
2026-06-30 20:40:22 -07:00
Matthew Jackson 4e65f87b71 patch: breadth-first two-tier recovery, largest ranges first
The per-range walk was depth-first: each bad range ran the full handler
chain (fast + slow deep-recovery + bisect) before the next range was
touched. So a handful of tiny dead fragments at one end of the disc
burned the whole pass and the big NonTrimmed ranges elsewhere — usually
sweep-jump over-marks that read straight back — were never attempted.

Now recovery runs in two breadth-first tiers over ALL sections:
- Tier 0 gives every section one fast full-batch attempt (fast reads
  only), largest ranges first, so the recoverable bulk of the disc comes
  back in the first minutes.
- Tier 1 deep-recovers only the residue tier 0 could not pull.
Per-section still-bad sets persist across tiers. Largest-first ordering
means a big readable region is reached before time is spent on tiny dead
fragments.

Linear no longer collapses a failed batch to count=1 single-sector reads
(live probing: a marginal sector recovers in a large read, not a lone
one) — a failed 32-batch stays 32 and is re-attempted at full size by the
next handler/pass; Bisect salvages readable islands.

Adds a handler-start trace line so the debug log shows which handler is
running and the hand-off to the next.
2026-06-30 20:25:48 -07:00
Matthew Jackson c5e4fa7109 patch: replace grind-until-wedge loop with bounded handler chain
Pass-N recovery is now a chain of time-bounded recovery handlers instead
of one monolithic per-range loop that could grind the front of a bad
range for up to 30 min, wedge the drive, and abort the whole pass.

A bad range is a SubRanges set; recovery is an ordered list of
SectionHandlers (Linear{reverse,fast} covering back/forward x fast/slow,
and Bisect). A coordinator runs each handler with a hard per-handler
deadline: a handler recovers what it can (removing it from the still-bad
set) and hands the rest to the next handler; whatever is still bad after
the chain becomes NonTrimmed residue and we move on to the next range.

Guarantees, now structural rather than bolted-on:
- never hangs: every handler is deadline-bounded; the loop always drains
  to recovered-or-residue.
- always moves on: a range that cannot be finished leaves residue and
  advances; only a genuine transport fault or user halt ends the pass.
- extensible: a new recovery idea is one SectionHandler impl added to the
  chain; a proven-ineffective one is removed. The engine never changes.

Removes ~1.9k lines of the old inner loop (watchdogs, skip escalation,
NOT_READY grind, wedge counters) and their tests. fast_capture is now
inert (the chain supersedes it); breadth-first ordering becomes a future
scheduler concern. New module: disc/section_recover.rs (8 fixture tests,
injectable clock — bounded/never-hang proven without touching a drive).
Two A/B tests updated to the chain's strictly-better recovery counts.
2026-06-30 19:55:37 -07:00
Matthew Jackson b67eaab3a8 drive: always set FUA on READ(10) to bypass the drive cache
Every Drive::read now forces the read from physical media instead of
letting the drive satisfy it from cache. A recovery tool must not trust
the cache: on a re-read of a marginal sector the BU40N can return a
cached miss (~4 ms) instead of giving the surface a fresh physical
attempt (~40-250 ms), masking a sector that would otherwise read on a
real media hit. FUA (READ(10) byte 1 bit 3) is validated on the
BU40N/Initio bridge.
2026-06-30 18:49:14 -07:00
Matthew Jackson dfc54891c5 drive: Drive::spin_cycle() — soft un-wedge via START STOP UNIT (no eject)
Spin the disc down (START=0, LOEJ=0) then back up (START=1) to clear the
BU40N/Initio fast-fail wedge state a run of HARDWARE_ERROR reads leaves the drive
in — the non-eject power-cycle our notes say the wedge needs. The disc stays
loaded; we never eject (slot-loading — a human eject is a product failure for an
unattended service). Validated live 2026-07-01: took the drive from
failing-every-read back to reading at MB/s.
2026-06-30 17:54:51 -07:00
Matthew Jackson 4169cf7f4f disc/patch: fast-capture reads fail-fast (no deep recovery)
A fast_capture pass defers every failed block to a granular pass anyway, so
spending the drive's 60s deep-recovery timeout on it here only freezes the
breadth-first sweep on a pure-bad cluster (~25s per 32-block, incl. NOT_READY
grind). Read with recovery=false in fast_capture so a bad block fails on the
short timeout (~10s) and the sweep steps on; the granular passes
(fast_capture=false) do the deep recovery on what's left. No recovery lost —
the block stays NonTrimmed for the granular retry.
2026-06-30 17:08:26 -07:00
Matthew Jackson edd316b8a8 docs: 1.2.0 changelog — fast_capture breadth-first patch recovery 2026-06-30 16:38:56 -07:00
Matthew Jackson c57ac14ad1 disc/patch: fast_capture mode — breadth-first recovery (#50)
A PatchOptions.fast_capture pass reads each bad range ONCE at the full batch and
leaves every FAILED block NonTrimmed for a later pass — no bisect, no per-sector
grind, no retry. This lets a first retry pass grab the readable blocks (the
sweep's good skip-ahead overshoot) of EVERY section quickly, before any single
section's slow per-sector recovery — instead of grinding section 1 to exhaustion
before even touching section 2. A later pass (fast_capture = false) does the
granular bisect/retry on what's left.

Load-bearing invariant (fixture test): NO data is dropped. A failed block becomes
NonTrimmed (pending, retried by a granular pass), NEVER Unreadable. The test pins
that the readable half of a range recovers, the bad half stays NonTrimmed (not
Unreadable), and the bad block is marked in ONE batch read with zero bisection.
Disc::copy's internal patch keeps fast_capture=false (single-call full recovery).
2026-06-30 16:37:59 -07:00
Matthew Jackson 9154ac12d6 docs: 1.2.0 changelog — PassProgress mapfile-free contract + located drilldown 2026-06-30 16:13:05 -07:00
Matthew Jackson ec3d072ec7 disc: add progress_snapshot_from_mapfile one-shot + locate_ranges test
The library now offers a one-shot PassProgress snapshot built from a mapfile on
disk + title, so a client gets the fully-rendered drilldown without parsing the
mapfile itself — used for autorip's pass-boundary and mux-entry paints (no live
callback yet at those points). Adds a locate_ranges test covering in-feature vs
out-of-feature at-risk time (ported from autorip's removed from_map tests).
2026-06-30 16:09:28 -07:00
Matthew Jackson bbf04c3d51 progress: PassProgress carries the located drilldown (emit side)
Add LocatedRange + LocatedProgress to the progress contract and a 'located'
field on PassProgress, populated by the sweep + patch emitters from the
in-memory bad-range set + title. Move the range->chapter/time annotation
(locate_ranges, range_chapter, byte_offset_in_title) into the library so a
client renders the disc map + at-risk movie time straight from PassProgress
and never reads the mapfile itself — if the mapfile becomes a mapdb, this type
and its producer change, clients don't.

PassProgress is no longer Copy (located carries a Vec); it's built once per
throttled emission and passed by reference. Non-locating phases (verify,
extract) emit LocatedProgress::default(). Adds consts::MILLIS_PER_SEC.
Consumer-side wiring (autorip drops Mapfile::load) follows.
2026-06-30 15:26:49 -07:00
Matthew Jackson d4f8c86cbc disc/patch: remove scatter-recovery (ruled out by live probing); add SubRanges
Live drive probing (cold-single vs in-stream batch vs centered window,
one clean uncached read per sector) showed recovery RATE is identical
across approach: 3/5 every mode. The drive's per-sector ECC is
media-bound and partly stochastic, NOT approach-bound. So scatter
(seek-away recalibration + far-anchor re-read) does not improve recovery
and is dead weight — removed (function, read_good_sectors, SCATTER_*
consts, and its tests).

Add SubRanges: the still-bad (pos,len) sub-ranges of a section with
remove()/split, the work-list foundation for the upcoming per-section
recovery decomposition (recover_section -> stream/bisect/retry phase
helpers). Unit-tested; unused until the decomposition lands.

25 patch + 8 AB profile tests green; precommit clean on Rust 1.86.
2026-06-30 13:56:49 -07:00
Matthew Jackson a4da4dbe24 disc/patch: keep MAX speed on recovery reads + geometric batch re-grow
Live probing on the BU40N/UHD testbed (rprobe/dprobe) settled the speed
question empirically: a marginal sector reads ~12x FASTER at MAX speed
than at slow, and slow NEVER recovered one MAX didn't. Cleanly-tested
(one uncached read per sector) the recovery RATE is identical across
speed/direction/window/batch — the drive's per-sector ECC is media-bound,
not approach-bound. The only host lever that matters is read SPEED.

So the old 'drop to 0x0000 on first failure and hold it for the whole
range' only slowed the GOOD sectors of a bad range — measured ~3x slower
overall. Now the range stays at MAX; the first failure just re-attempts
once (stochastic media) and falls through. Live A/B: 11 KB/s -> 33 KB/s,
bytes_lost unchanged.

Also: batch adaptation is now halve-on-failure / double-on-success
(geometric), so a bisected batch climbs back through clean runs from any
size (the old count==1-only upscale left it stuck at mid sizes). Removed
the dead consecutive_singles_ok counter and the inert batch-retry probe.

25 patch tests green; precommit clean on Rust 1.86.
2026-06-30 13:25:07 -07:00
Matthew Jackson e6dfe54e10 disc/patch: drop cache-priming, bisect batch failures instead of count=1
Two patch-recovery speed fixes, grounded in live rip1 telemetry on the
Dune UHD bad regions.

1. Pull prime_cache. In the reverse walk it issued throwaway reads of
   lba-3..lba-1 — i.e. straight into the not-yet-visited dead L-EC core —
   grinding each ~7-9s at the fast timeout, then discarding the result.
   The live trace showed a good target read (read_duration_ms=1) taking
   9s wall-clock purely from the prime grind ahead of it. The dead-zone
   skip (Tier 1) + scatter's recalibration read (read_good_sectors)
   cover the cache-warm intent without grinding the dead zone, so prime
   is redundant harm — removed (function + CACHE_PRIME_SECTORS + call).

2. Bisect on batch failure. handle_read_failure used to collapse a
   failed count>1 batch straight to count=1 and single-walk the entire
   batch. Now it halves (count/2) and retries the same start: a good
   half recovers in BULK, the bad sector is isolated in O(log n) reads
   instead of O(n). Invariant preserved — only a count==1 failure marks
   NonTrimmed, so no good sector is lumped into a bad mark.

These are Tier 2 (bisect) of the progressive recovery model
(fast-mass-skip -> bisect -> scatter); composable so order can be tuned.
25 lib + 8 AB fixture tests green; precommit clean on Rust 1.86.
2026-06-30 11:20:44 -07:00
Matthew Jackson c68d40887a disc/patch: add scatter-recovery for stuck bad sectors
The 'reset, read good data, come back for one sector' technique. A
genuinely-damaged sector makes the drive grind its full recovery timeout
and still fail; re-reading consecutive bad LBAs at identical conditions
re-fails AND is the rapid-failure cadence that drops the BU40N into a
firmware fast-fail wedge (CLAUDE.md hard-rule #2).

scatter_recover: on a stuck single sector, make up to SCATTER_MAX_ATTEMPTS
fresh tries, each preceded by read_good_sectors() — a seek to a far
known-good anchor + a sustained read that re-seats the head/servo and
breaks the failure cadence. The fresh re-read uses the FAST timeout
(recovery=false) so a recalibrated marginal sector reads quick and a
truly-dead one fails fast instead of burning 60s per attempt. The seek +
good read IS the settle (matches ddrescue/MakeMKV) — no idle sleep.

Gated to genuine single-sector MEDIUM_ERROR (0x03): transport faults
still abort the pass, NOT_READY keeps its own retry path, wedge-family
senses are left for cooldown/eject. A scatter recovery is recorded
exactly like a normal read success (write-then-mark-Finished); nothing
marks a range good without a successful read + write.

Logs anchor_ms/reread_ms per attempt for live tuning. 5 fixture tests
(recover-marginal, give-up-dead, skip-non-medium, skip-batch, AACS
widen); 2 AB read-count guards raised for scatter's bounded extra reads.
Precommit green on Rust 1.86.
2026-06-30 10:29:55 -07:00
Matthew Jackson ad9ea4c199 disc/patch: extract PatchCtx/patch_region orchestration + unify log taxonomy
Pure structural refactor of the Pass-N recovery path, no behavior change.

- PatchCtx<R> holds the one-pass context (reader, pipe, shared, opts,
  reused read buffer, cross-region state) and owns orchestration.
- PatchCtx::run() walks the ordered bad ranges, applies the inter-range
  cooldown (armed on a grind, consumed at the next range's entry), and
  stops the pass the moment a range reports halt/wedge/transport-fault.
- PatchCtx::patch_region() is the focused single-range loop: read ->
  success/failure -> damage-skip -> per-range watchdog. Every former
  break / break 'outer is now a typed RegionOutcome (Completed,
  SkipLimit, BudgetExceeded, Wedged, Halted, TransportFault) that run()
  maps back to next-range vs stop-pass exactly as before.
- recovery_read() extracts the AACS mid-unit-window widening + plain
  read into one reusable helper (the building block the upcoming
  scatter re-read will share).
- Logging unified onto a single dotted patch.* taxonomy (patch.region.*,
  patch.read.*, patch.probe.*, patch.wedge.*, patch.batch.*,
  patch.mapfile.*, patch.skip.*, patch.stall, patch.speed); the
  freemkv::scan pass-boundary markers are left intact.

20/20 disc::patch tests green; precommit (fmt + clippy -D warnings +
tests) clean on Rust 1.86.
2026-06-30 10:04:51 -07:00
Matthew Jackson 9422909773 error: treat IoError/DeviceNotFound as SCSI transport failures (dead bus)
is_scsi_transport_failure recognized only the 0xFF SCSI sentinel, so the two
NON-SCSI dead-bus faults the Linux SG_IO transport returns — Error::IoError
(ioctl(SG_IO) == -1: ENODEV/EIO on an unplugged bridge) and Error::DeviceNotFound
(fd gone) — were classified as recoverable bad sectors. Every caller
(sweep read_error, patch, single-pass fill_extents) would then zero-fill /
skip-ahead across a wedged bus instead of aborting the pass and re-enumerating
the bridge — potentially zero-filling a huge span. Add both to the classifier so
a dead bus aborts the pass everywhere. (Companion to the unlock_bridge fix, which
maps the same two faults to 0xFF across the freemkv-unlock seam.)
2026-06-30 07:19:03 -07:00
Matthew Jackson 53e2e9a6ce unlock_bridge: map non-SCSI transport faults (IoError/DeviceNotFound) to 0xFF
Round-2 audit gap in the prior bridge fix: extract_scsi_context only recognizes
Error::ScsiError / DiscRead and collapses every other variant to (0, None). But
the Linux SG_IO transport returns genuine dead-bus faults as non-SCSI variants —
Error::IoError (ioctl(SG_IO) == -1: ENODEV/EIO on an unplugged bridge) and
Error::DeviceNotFound (fd gone) — which were crossing the seam as status 0x00, so
the unlock crate classified neither as a transport fault (needs 0xFF) nor as a
sense-bearing rejection, and kept hammering a dead bus.

Match the error variant in the adapter: a SCSI status (CHECK CONDITION or a
drive-tagged 0xFF) keeps its real status + sense; any other variant is a
non-SCSI transport/IO fault → SCSI_STATUS_TRANSPORT_FAILURE / None. Keying off
`sense.is_none()` would be wrong (a senseless CHECK CONDITION is a rejection).
Add bridge tests covering CHECK-CONDITION sense preservation, 0xFF passthrough,
and IoError/DeviceNotFound → 0xFF.
2026-06-29 22:57:09 -07:00
Matthew Jackson ab289fb5c4 mux: add the B1 drop-to-keyframe resync gate to DiscStream
DiscStream (the live-drive single-pass path) enables AACS decrypt-loss
concealment (NULL-TS fill on an undecryptable unit) but, unlike the file-backed
PipelinedPesStream, had no B1 resync gate — so after a concealed gap it forwarded
inter-coded video frames referencing the now-missing data, producing
decode-broken MKV output (dangling-reference frames).

Mirror PipelinedPesStream: add per-stream ResyncGate + is_video, and route every
codec-frame emit on the TS path (in-stream parse, demuxer EOF flush, parser EOF
flush) through gate.admit(is_video, frame.discontinuity, frame.keyframe) so a
video track drops forward to the next keyframe after a concealment event. Warn
once if a gate is still armed at EOF (a concealed gap in the final GOP with no
trailing keyframe). Audio/subtitle always admit.
2026-06-29 22:27:52 -07:00
Matthew Jackson dc3b7970e7 unlock/patch: preserve SCSI sense across the bridge; init aborts on dead bus; slow-retry first patch failure
Audit fixes (v1.1.0..HEAD regressions in the unlock migration + adaptive patch
speed):

- unlock_bridge ScsiAdapter: libfreemkv's transport returns Err on ANY non-zero
  SCSI status (a normal CHECK CONDITION), not only transport faults. The adapter
  was collapsing every such Err to { status: 0xFF, sense: None }, which discarded
  the parsed sense and defeated the AACS handshake's ILLEGAL_REQUEST wedge guard
  (so it kept hammering the drive — hard-rule #2) and inverted its
  transport-vs-rejection diagnosis. Now reconstruct status + the 32-byte sense
  buffer (sense_key@2, asc@12, ascq@13) and only emit 0xFF/None for a genuine
  transport fault.

- Drive::init: a genuine transport fault during the drive-prep unlock means the
  bus is dead — propagate it (the v1.1.0 invariant) instead of silently
  swallowing it via `if let Ok`. Other errors (no matching unlocker) still fall
  through to stock mode. SET CD SPEED max now runs only when the bus is alive.

- disc::patch: on the first read failure in a range, drop to slow recovery speed
  and RE-ATTEMPT the same position at slow speed before marking it. A
  single-sector range's first failing sector was being marked from a MAX-speed
  read it never got to recover.

- docs: lib.rs architecture diagram (handshake → host_certs) and README (stale
  pluggable-unlock-seam / register-unlocker / crates.io / docs.rs references).
2026-06-29 22:16:28 -07:00
Matthew Jackson 7cac0a1b15 disc: log adaptive patch speed transitions; issue SET CD SPEED max unconditionally
Two recovery-path fixes:
- patch: log the per-range speed transitions (INFO, phase="patch_speed") — each
  range enters at 0xFFFF (max) and drops to 0x0000 (slow recovery) on its first
  read failure. Previously the adaptive-speed behavior was invisible in the logs.
- drive init: issue the generic SET CD SPEED max UNCONDITIONALLY at drive-open,
  not only when a firmware unlocker matched. A stock-mode BD/UHD drive (no
  firmware unlock) was left riplocked because the call sat inside the
  unlocker-matched branch.
2026-06-29 21:47:50 -07:00
Matthew Jackson 5bd7da0f6f README: drop crates.io/docs.rs badges + stale plugin-model line (off crates.io now) 2026-06-29 21:17:06 -07:00
Matthew Jackson dd23f3f3c9 Cargo: publish = false — off crates.io (git-deps the firmware crate)
libfreemkv now git-deps freemkv-unlock (which carries drive firmware and can
never be on crates.io), so libfreemkv itself can only be consumed by git tag.
The freemkv-unlock dep stays an interim path dep for local cross-repo dev; the
release script swaps it to a git tag in the tagged commit.
2026-06-29 21:04:05 -07:00
Matthew Jackson c7925a293d unlock: dispatch via freemkv-unlock; delete in-tree handshake/css-auth/registry
Rewire the three unlock dispatch points through the freemkv-unlock crate via a
private `unlock_bridge`: drive-prep (kind=Unknown) at `Drive::init`, AACS cert
(kind=Aacs) at `do_handshake_cert`, CSS bus-auth (kind=Css) at scan. The bridge
news up `all_unlockers()` and runs the first matching one, mapping its
`Unlocked` result to the bus-key gate. After a successful drive unlock,
libfreemkv issues a generic SET CD SPEED (max) itself — the old per-unlocker
trait method is gone.

Delete the in-tree unlock code now owned by freemkv-unlock: the AACS cert
handshake (`aacs/handshake.rs`), the CSS bus-auth (`css/auth.rs`), and the
unlock registry (`unlock.rs`). Host-cert collection (a keysource concern) stays
in a small `aacs/host_certs.rs`. No public unlock surface remains — clients
touch libfreemkv only, oblivious to unlockers (as they are to SCSI). 2277 tests
pass.
2026-06-29 20:45:00 -07:00
Matthew Jackson 37a8d8f710 stage 4a: bridge libfreemkv to the freemkv-unlock crate
Add the freemkv-unlock dependency (path dep for local dev; re-pinned to a git
tag at release) and the consumption bridge: one generic ScsiAdapter mapping
libfreemkv's transport to the unlock contract, host-cert + DriveId mapping, and
run_unlockers() which news up all_unlockers() and runs the first matching one.
Additive — validates the dep resolves and the adapter compiles; the in-tree
registry/handshake/css path is still active and gets replaced in 4b.
2026-06-29 19:48:50 -07:00
Matthew Jackson 48bea05472 aacs: extract run_cert_handshake + collect_host_certs as shared primitives
Pull the wedge-guarded cert loop and host-cert collection out of the in-tree
AacsCertUnlocker into public aacs::handshake primitives (run_cert_handshake +
CertHandshake, collect_host_certs). The in-tree path now delegates to them, so
the external freemkv-unlock-aacs plugin runs the IDENTICAL cert handshake — one
implementation, two callers. Pure refactor of the live AACS path; the existing
handshake + collect_host_certs unit tests validate it unchanged.
2026-06-29 17:33:26 -07:00
Matthew Jackson 04ae2704f1 unlock: route_unlock returns a structured UnlockRoute; ctx carries opts
Replace route_unlock's Option<(name, Vid)> with a structured UnlockRoute
{ Unlocked(name, Unlocked) | Failed(UnlockError) | NoMatch } so a single
dispatch serves every caller: drive-prep wants "did anything unlock", and the
AACS cert route (next) needs the FAILURE REASON to render "missing keys" vs
"host cert rejected" instead of collapsing it to a bare None. Only a genuine
SCSI transport fault still returns Err (abort). UnlockCtx gains an optional
ScanOptions (the cert route's host-cert source), and read_mkb_from_drive now
takes &mut dyn ScsiTransport — both prerequisites for the cert handshake to
become an external freemkv-unlock-aacs unlocker. Drive-prep + CSS callers fold
the new outcome; no behavior change.
2026-06-29 17:18:11 -07:00
Matthew Jackson 26fc70b91b unlock: Unlocked carries the bus-key gate nuance; route_unlock returns it
Extend Unlocked with drive_unlocked + read_data_key_err so the registry result
carries everything the downstream bus-key gate needs — a firmware unlocker sets
drive_unlocked=true (bus encryption removed at the drive, no bus key), the cert
route will set read_data_key (+ read_data_key_err on a failed bus-key read).
route_unlock now returns the full Unlocked rather than just a Vid, so a single
dispatch can fold the firmware, cert, and CSS results uniformly. Prep for the
AACS cert handshake becoming a registry unlocker.
2026-06-29 16:59:04 -07:00
Matthew Jackson a061694ba0 css: CSS bus-auth becomes a uniform registry Unlocker
Convert the CSS read-unlock into a first-class registry Unlocker (CssUnlocker)
dispatched through route_unlock like every other barrier removal, instead of
a direct call in scan. libfreemkv appends the built-in CSS unlocker (and, next,
the AACS cert handshake) exactly once via ensure_builtins(), AFTER any
client-registered firmware unlocker — so the registry order is firmware → cert
→ css, owned by the lib, not the client.

Defense in depth: the unlocker does NOT trust the caller-declared DiscKind.
matches() filters on the declared kind (Css), but unlock() self-verifies
against the drive's GET CONFIGURATION profile and refuses (UnlockError::
NotApplicable, a new shared "this unlocker doesn't apply" variant) WITHOUT
issuing a single CSS CDB if the drive reports a non-DVD profile — so a
mis-routed Blu-ray is never sent CSS bus-auth. Guard the firmware unlocker the
same structural way (it matches only the drive-prep phase, kind == Unknown).

Tests: CssUnlocker matches only DiscKind::Css; a BD-profile drive yields
NotApplicable with zero CSS CDBs issued.
2026-06-29 16:49:52 -07:00
Matthew Jackson 45c0bc943c unlock: introduce UnlockCtx + DiscKind; trait keys off context
Reshape the Unlocker seam so every unlocker is dispatched at ONE place from
ONE ordered registry — the firmware, cert, and CSS routes are all "remove the
bus-encryption barrier", differing only in what they key off. matches() and
unlock() now take an UnlockCtx { drive_id, kind: DiscKind } instead of a bare
DriveId: a firmware unlocker keys off drive_id (kind irrelevant), the cert
unlocker will match DiscKind::Aacs, the CSS unlocker DiscKind::Css. UnlockCtx
is #[non_exhaustive] so a host-cert source can be added without breaking
external unlockers. Drive-prep dispatch passes DiscKind::Unknown (no disc
probed yet); the cert/CSS registry impls + the single post-probe dispatch
point follow in subsequent commits.
2026-06-29 16:27:30 -07:00
Matthew Jackson c9c64e2cbf css: auth primitives operate on &mut dyn ScsiTransport; clean up bus_auth
Thread &mut dyn ScsiTransport through unlock_css_reads and its bus-auth /
disc-key primitives instead of &mut Drive — they only ever issued SCSI via
drive.scsi_mut(), so this is mechanical and lossless, and it makes the CSS
unlock speak the same transport interface as the registry Unlocker trait
(prerequisite for CSS becoming a uniform unlocker).

Also clean up the badly-named, half-dead bus_auth:
- rename bus_auth -> establish_authenticated_session: it is run for its
  side effect (sets the drive's ASF=1, unlocking scrambled-sector reads),
  which the name now states.
- drop the derived CSS bus key from the return + computation: it had no
  consumer (descrambling is keyless via the Stevenson attack), so it was
  dead crypto computed on every DVD unlock. Return just the negotiated AGID,
  which the caller genuinely needs for the best-effort disc-key REPORT KEY.
2026-06-29 16:01:07 -07:00
Matthew Jackson 8ac18d857d aacs: handshake primitives operate on &mut dyn ScsiTransport
The AACS cert-auth primitives (aacs_authenticate, the AACS 2.0 P-256
variants, read_volume_id, read_data_keys) and their scsi_read/scsi_write
helpers touched the drive ONLY through Drive::scsi_execute — a pure
pass-through to the transport. Thread &mut dyn ScsiTransport instead of
&mut Drive so these primitives are transport-level, matching the firmware
Unlocker seam (which hands out &mut dyn ScsiTransport for testability).
Pure mechanical signature change, no logic change; the cert orchestrator
(do_handshake_cert) keeps &mut Drive for the OEM-VID shortcut and passes
session.scsi_mut() into the primitives. Step toward making the cert
handshake a uniform registry unlocker.
2026-06-29 15:47:30 -07:00
Matthew Jackson 0a006bbfa6 unlock: Unlocker::unlock returns Unlocked { vid, read_data_key }
Generalize the unlock seam toward uniform unlockers (firmware / cert / CSS).
`unlock` now reports what it LEARNED — a Volume ID and/or an AACS bus key —
instead of just a Vid; libfreemkv files those onto the disc/drive in one
place, so a plugin never touches Disc. The hardware side-effect (extended
mode) still happens inside unlock. route_unlock walks the registry in
registration order (the single ordered place that decides run order); no
priority field. Firmware route reports {vid: Some, read_data_key: None}.
2026-06-29 15:09:17 -07:00
Matthew Jackson 11554c1722 disc: credit firmware unlock in the bus-encryption gate
The bus-key gate only credited the cert handshake's read_data_key as proof
bus encryption was removed. A firmware unlocker removes it AT THE DRIVE
(serves clear content) and yields no read_data_key — so a SUCCESSFUL
firmware unlock (VID present, read_data_key None) tripped the gate and
blocked ALL key resolution, including the online source. That was the
root cause of live UHD discs reporting "missing keys" after an unlock.

Now a single predicate answers "is bus encryption gone?": never-had-it ||
file/ISO || firmware-unlocked || cert-bus-key. The gate is just
`if !bus_encryption_removed { error }` — no enumerated cases. HandshakeResult
gains `drive_unlocked`, and the read_data_key failure reason is captured so
the warn says WHY the bus key is missing.

Also: reword the first hardware-sense escalation as "fast-fail escalation"
(it is often transient — the drive recovers), reserving "wedge" for a
persistent run; and scrub the product name from core comments (it belongs
only in the unlocker crate).
2026-06-29 15:09:16 -07:00
Matthew Jackson 321442632a disc: make bus-key unavailability self-explaining in the log
When a bus-encrypted disc's handshake yields no read_data_key, the gate
logged a bare "bus_key_unavailable" with no indication of WHY — turning
every occurrence into archaeology (is the bus key not-attempted, or did
the read fail?).

Now:
- read_data_keys failure is captured (error code) instead of swallowed by
  `.ok()`, and logged at the handshake with its consequence.
- HandshakeResult carries `read_data_key_err: Option<u16>` so the gate
  distinguishes "never attempted" (None — VID-only/OEM path) from "read
  FAILED" (a code), and the bus_key_unavailable warn now reports the code
  plus whether a Volume ID was present.

No behavior change — purely diagnostic. The handshake_ok debug also now
reports has_volume_id.
2026-06-29 13:33:09 -07:00
Matthew Jackson 46797c01a3 mux: harden audio discontinuity handling (audit follow-up)
Two defensive hardenings from the post-fix audit (vs FFmpeg/GStreamer):

1. Move the `pes.discontinuity` partial-drop ABOVE the empty-data guard in
   all three audio parsers (ac3/dts/truehd), so a discontinuity signal can
   never be stranded by an empty post-gap PES. The demuxer only emits
   non-empty PES today; this is defense-in-depth for any future caller.

2. A PES with no PTS must not reset the timeline to 0. ac3 now carries
   `flush_pts_ns`, dts continues from the most recent known base; truehd
   already kept its running cadence on a None PTS. Matches OSS behavior
   (PTS rebases off the next PES that actually carries a PTS).

Adds an ac3 regression test (empty-payload discontinuity PES still drops
the stranded partial). Loss accounting was reviewed: TS-demux CC-gaps are
NOT counted toward lost_video_secs / abort (that is sector-based via
DiscStream::errors / mapfile bytes_unreadable), so a source splice never
inflates loss — no gating needed there.
2026-06-29 12:50:52 -07:00
Matthew Jackson 790a40138b mux: drop truncated partial audio frame on concealed gap
The AC-3, DTS and TrueHD parsers buffer access units across PES
boundaries. At a concealed-loss gap the buffered unit is truncated:
splicing post-gap bytes onto it manufactures a corrupt frame on top of
the real loss (FFmpeg "Failed to decode block code(s)" / "Invalid data
found" at the gap) and, for TrueHD, strands the PTS cadence into the
non-monotonic audio-DTS band seen on multi-clip titles.

The video parsers already handle this via the ResyncGate, but the
discontinuity signal was only wired into video — audio parsers ignored
pes.discontinuity and spliced across the gap.

Now, when pes.discontinuity is set, each audio parser drops the partial
(clears buf, and for DTS its PTS marks / pending base) so the post-gap
PES re-bases a fresh unit. A lost gap degrades to a clean single-frame
drop instead of a corrupt spliced frame. No effect on perfect rips: the
branch only runs when concealment inserted a discontinuity marker.

Adds a per-parser test feeding a partial frame then a discontinuity PES,
asserting the truncated partial is dropped (not spliced) and the post-gap
PTS is adopted.
2026-06-29 11:23:08 -07:00
Matthew Jackson 4e545e6e46 disc: adaptive patch speed — enter each range fast, slow on first failure
Pass N (patch) pinned the slow recovery speed (0x0000) for the whole pass.
But Pass 1's damage-jump overshoots, so most of a jumped range is clean data
the reverse-walk reads first — grinding it at slow speed wastes minutes per
gap. Now each range mirrors Pass 1's model:

- Enter at max speed (0xFFFF) + reset to the initial batch (current_batch
  carries across ranges, so a prior single-sector grind would otherwise start
  the next range slow). Reads the clean overshoot fast.
- First read failure in the range → drop to 0x0000 (once; idempotent SET CD
  SPEED) for the rest of that range — grind only the genuine damage.
- 10 s halt-responsive cooldown between ranges, gated on "this range actually
  grinded" so a many-small-range pass (100+ ranges) doesn't stall ~20 min on
  unconditional pauses.

No change to good/bad classification, the mapfile, or the abort math — purely
read speed + an inter-range cooldown. Synthetic speed-transition test to follow.
2026-06-29 10:34:18 -07:00
Matthew Jackson afe55c7871 docs: 1.2.0 changelog date → 2026-06-29 (release prep); fix neither/nor grammar in 1.1.0 entry 2026-06-29 09:57:29 -07:00
Matthew Jackson 7114e46018 mux/decrypt: audit-nit cleanup + defensive conceal fallback
Non-blocking follow-ups from the 1.2.0 audit:
- conceal loop: if decrypt reported loss but the padding-aware predicate
  matched nothing to conceal (a ~256^-31 contradiction), fall back to the
  strict predicate and conceal whatever it flags, loudly — belt-and-
  suspenders so ciphertext can never reach the mux.
- decrypt_dropped doc: reflect 1.2.0 (mux-path loss is concealed + tallied,
  not silently dropped).
- direct unit test for aacs_unit_still_ciphertext (the padding-aware
  conceal predicate): clear/all-zero/full-decrypted/full-ciphertext/
  decrypted-short-tail.
- fix three stale "v1.1.1" comment refs (the fragment-tail fix ships in
  1.2.0; there is no v1.1.1 release).
2026-06-29 09:51:28 -07:00
Matthew Jackson ccb1b16ffb mux: make B1 concealment decode-clean on every gap shape
Closes the three residual holes where a concealed/lost gap could still let
a dangling-reference frame reach the muxer (degraded/undecryptable-disc
path only; clean rips are byte-identical and untouched). Root cause: the
discontinuity signal was reconstructed from the 4-bit continuity counter
and applied per-PES, both of which are lossy.

Three coordinated changes:

1. CC-INDEPENDENT marker. fill_null_ts_unit now tags its NULL packets with
   an adaptation-field discontinuity_indicator; the demuxer recognises a
   0x1FFF packet carrying it as a concealed gap and forces a discontinuity
   on every tracked PID (the lost unit's PID is unknowable). This survives
   a loss that is an exact multiple of 16 packets (CC aliases to in-sequence
   — hole 3) and a loss at a PID's very start (no prior CC — hole 4); it
   also drops any open, potentially-truncated partial PES.

2. PUSI ATTRIBUTION. A gap landing on a PES boundary now flags the PES
   STARTING after it, not the one flushed at the boundary (hole 1) —
   stamping the pre-gap frame could arm-then-disarm the gate on a keyframe
   and admit the real post-gap inter frame.

3. PER-FRAME signal. codec::Frame gains `discontinuity`; each parser
   propagates it onto the first post-gap frame. MPEG-2 buffers whole GOPs
   asynchronously, so it associates the gap by ES OFFSET (like PTS/source),
   landing it on the exact post-gap picture mid-GOP (hole 2) — a per-PES
   flag stamped the previous picture. consume_ts (and the EOF flush drain)
   gate on frame.discontinuity.

Tests: CC-independent marker with in-sequence CC + leading-loss; PUSI
attribution flags the post-gap PES; MPEG-2 offset-mark stamps the post-gap
picture through GOP reorder, not the previous one. Existing B1 gate + EOF
tests still green (2270 lib tests).
2026-06-29 09:39:03 -07:00
Matthew Jackson a5268d15ca docs: 1.2.0 CHANGELOG — document the breaking public-API changes
Per the release decision: ship the AACS-version-stride API changes under
the 1.2.0 minor bump, with an explicit Breaking section (DiscInputs new
field, DiscInputsCtx::new arity, read_aacs_inputs* 3-tuple return).
2026-06-29 08:41:26 -07:00
Matthew Jackson accf21b03d mux: conceal only genuinely-undecryptable units (padding-aware)
The P3 concealment loop in DecryptingSectorSource::read_sectors keyed on
aacs_unit_needs_decrypt, whose sync check is the majority-vote
ts_sync_destroyed (<=16 of 32 syncs). A successfully padding-aware-
decrypted content-fragment TAIL unit (e.g. 11 content packets + 21 zero
padding) has only 11 syncs, so the majority vote called it "still
encrypted" — and when such a good unit shared a read buffer with a
genuinely-undecryptable one (dropped>0), the loop overwrote the GOOD
decrypted tail with NULL-TS, silently discarding correct video and
over-counting concealed units vs the tallied dropped bytes.

Add aacs_unit_still_ciphertext (padding-aware): encrypted AND at least
one non-zero (non-padding) 192-byte packet missing its 0x47 sync — the
same discriminator decrypt_unit uses to accept a fragment tail. The
conceal loop now uses it, so only genuinely-unrestored ciphertext is
concealed. Full and fully-ciphertext units are unchanged. Regression
test: a decrypted short-padding-tail co-resident with a failed unit is
left byte-for-byte intact while the failed unit is concealed.
2026-06-29 00:29:20 -07:00
Matthew Jackson 07396ec19d v1.2.0: bump version + reconcile CHANGELOG to [1.2.0] 2026-06-28 23:34:19 -07:00
Matthew Jackson da3859d0c5 mux: B1 drop-to-keyframe resync after a concealed gap
Pairs with A2 (read-path NULL-TS concealment). When the demux assembler
sees a TS continuity gap it now stamps `discontinuity` on the next
completed PES; the codec-parse stage carries that onto a per-track
ResyncGate. After a gap on an inter-coded video track the gate drops
forward to the next IRAP/IDR keyframe so no frame with a dangling
reference reaches the muxer (an ffmpeg deep scan would otherwise report
a missing-reference / non-existing-PPS error). Audio and subtitle tracks
have no cross-frame references, so the gate is a no-op there.

- ts.rs: PesPacket gains `discontinuity`; PesAssembler tracks a sticky
  pending_discontinuity flag set on CC gap / discontinuity_indicator and
  carried to the next completed/flushed PES.
- resync.rs (new): ResyncGate — per-track arm-on-gap, drop non-keyframes
  until the next keyframe disarms and resumes. Logs the resync + drop
  count once at the keyframe.
- pipelined_stream.rs: precompute per-track is_video, apply the gate in
  consume_ts. Out-of-range track index emits as-is (defensive).

Tests: ResyncGate unit tests; ts.rs gap-stamps-discontinuity; end-to-end
B1 video-drops-to-keyframe and audio-never-drops through PipelinedPesStream.
2026-06-28 23:01:47 -07:00
Matthew Jackson d334ab5749 1.2.0: mux loss-concealment read path (P3/Edit-2, A2 NULL-TS fill)
Decrypt-verify is a RIP gate, not a MUX gate. On the mux read path an
undecryptable content unit must never abort the mux:

- DecryptingSectorSource gains tolerate_decrypt_loss(): when set, an
  undecryptable in-content unit is tallied, overwritten with valid NULL
  TS packets (PID 0x1FFF) via aacs::fill_null_ts_unit, logged loud with
  its LBA, and the read returns Ok — the stream keeps flowing. The rip
  paths keep the fail-loud DECRYPT_VERIFY_READ decorator (re-read off the
  disc); only the mux opts in.
- Wire it into both mux read paths: the file-backed highway
  (build_iso_pipeline) and the inline DiscStream.
- NULL-TS fill keeps the demuxer byte-synced on the 192-byte stride; the
  lost video/audio PID packets surface as a CC gap the TS assembler
  already drops a partial PES on (the B1 foundation). Ciphertext is never
  passed downstream either way.
- Fix stale resolve_vid_only no-cert test: default is UHD (audit #4).

Tests: conceal-as-NULL-TS, fill well-formedness, fail-loud still holds.
2026-06-28 22:44:19 -07:00
Matthew Jackson 0b2e83e523 1.2.0: single MKB framing walker + AACS resolve hardening
- mkb_records() as the one record-framing iterator; rebuild walk_mkb,
  find_record_body, mkb_find_subdiff_records, mkb_content_len, mkb_version,
  mkb_type_raw, mkb_find_mk_dv on it (D4).
- resolve_vid_only / read_aacs_version default to UHD (+warn) on a missing
  content cert instead of BD; route MKB through bounded read_mkb_content.
- AacsVersion major()/from_major() + AACS_MAJOR_BD/UHD as the stride source;
  table + stride-discriminating regression tests.
- read_encrypted_units probes 8 evenly-spaced points per extent (off-midpoint
  scrambled content now sampled); decrypt source-zero mask uses PKT.
2026-06-28 22:28:09 -07:00
Matthew Jackson abd7af3907 1.2.0: unify hex parsing (one parser) + robust encrypted-unit sampling
- One workspace hex parser (libfreemkv::hex): the keydb / online / mapfile
  parsers had divergent prefix handling (0x vs 0X vs none) — a key written with
  a prefix one parser didn't expect was silently dropped. All three now call it.
- read_encrypted_units probes several points spread across each extent instead
  of only the midpoint-forward window, so a late-starting / sparse encrypted
  body still yields samples (empty samples make decrypt_with skip wrong-key
  validation). A read error at one probe no longer abandons the extent.
2026-06-28 22:12:06 -07:00
Matthew Jackson 7716a2cb17 1.1.1: unify AACS key-input path on Disc::inputs() + named constants
- read_aacs_inputs* now returns the AACS major version; DiscInputs carries it,
  and DiscInputsCtx parses Unit_Key_RO.inf at the disc's own stride (fixes the
  hardcoded-V20 read-time fetch for V10 discs). One source of truth, no version
  argument to drift.
- Disc::inputs() is the single complete AACS-input source (inf/MKB/VID/hash/
  version); the out-of-band duplicate readers go away.
- Named constants for AACS file paths (aacs::PATH_*) and the AACS majors
  (aacs::AACS_MAJOR_*, AacsVersion::major/from_major) replace magic strings/ints.
- push_ranges saturating (corrupt-disc panic guard).
2026-06-28 21:45:58 -07:00
Matthew Jackson 391e1ae39c 1.1.1: AACS decrypt + key-resolution hardening
- decrypt_unit: padding-aware acceptance — recover real video at content-
  fragment tails (the phantom mux-loss class) without weakening wrong-key
  rejection (a full content unit still needs all 32 TS syncs).
- scan: read the MKB via the bounded read_mkb_content so Disc::inputs()
  carries it. Online key resolution was shipping mkb=0 (a full read of the
  ~128 MiB MKB_RO allocation fails) → the decode service 404'd.
- resolve_vid_only: surface an MKB read error instead of silently emptying.
- fetch: a per-sample dry-set replaces the global fetch_spent latch, so a
  second CPS unit's key can still be fetched after the first came back empty.
- verify::push_ranges: saturating arithmetic (corrupt-disc panic guard).
- Tests for all of the above.
2026-06-28 21:12:57 -07:00
Matthew Jackson ee66db8b95 v1.1.0: bump version 2026-06-28 18:14:18 -07:00
Matthew Jackson 60df09f965 changelog: 1.1.0 (drop -beta.1 pre-release tag) 2026-06-28 17:58:46 -07:00
Matthew Jackson 46dc8720be aacs: rustfmt verify-gate files + correct cert/unit-key test fixtures to libaacs-strict layout; 1.1.0-beta.1 changelog 2026-06-28 16:49:16 -07:00
Matthew Jackson 353c8ce1fb verify: container-kind seam (HD-DVD-ready) + skip units with unread sectors
ContainerKind {Ts,Ps} + ClipLayout.container thread the post-decrypt structural check per clip; decryptability() dispatches it (TS: unit_is_clean_ts, PS: unit_is_clean_ps). New decrypt_unit_checked(unit,key,accept) decouples the container-agnostic AACS crypto from the format-specific acceptance (decrypt_unit delegates with the TS check). unit_is_clean_ps is the MPEG-2 PS pack-start check, documented UNVALIDATED for HD-DVD (.evo unit/seed/pack alignment must be confirmed on real media). clip_layouts assigns Ts today; .evo->Ps is the one-line HD-DVD hook.

reverify_iso now takes an is_finished predicate and SKIPS any unit with a non-Finished backing sector: we can't verify what wasn't read (a non-Finished sector is zero-filled because the drive read failed there), and must never waste a key lookup on a block the read already knows is bad. observe() (sweep) was already safe (only fed Good bytes).
2026-06-28 16:05:25 -07:00
Matthew Jackson 14070a2dec verify: post-read decrypt-verify gate + libaacs-strict verify + audit fixes
Post-read verify gate (new src/disc/verify.rs): UnitVerifier buffers/aligns the disc-absolute read stream into clip-file 6144-byte units, then makes one decryptability() decision per unit (CPI gate -> held keys -> key_fetch -> strict TS). POST_READ_VERIFY const kill-switch; fail-safe contract (only ever downgrades units it is confident are undecryptable; every doubt skips). Hooked into Disc::sweep (producer observes ciphertext -> WorkItem::MarkBad after the Good, FIFO-ordered) and Disc::patch (post-loop reverify_iso reads recovered units whole from the patched ISO). extract::clip_layouts enumerates AACS clips for the gate.

Standards-correct AACS verify: aacs::unit_is_clean_ts is a strict port of libaacs _verify_ts (all 32 TS syncs, not a majority vote); decrypt_unit accepts a key only on it; the majority verify_ts is removed. Deleted the Disc::verify_clips post-pass bolt-on (its primitive is absorbed by the read-path gate).

libaacs/DVD audit fixes: content-cert bus_encryption flag now read from bit 7 (was bit 0 - defeated the bus-key fail-loud gate); cc_id read from offset 14; title_cps_unit range-validated + 1->0 index-converted per libaacs. Corrected attack_crib ("functionally-equivalent" not "exact" port) and read_disc_key (READ DVD STRUCTURE 0xAD, not REPORT KEY) doc comments.

Also includes accumulated uncommitted work: key-fetch seam and TrueHD/DTS audio fix.
2026-06-28 15:03:52 -07:00
Matthew Jackson 1c8672dcaf changelog: enrich 1.1.0-beta.1 and rc.5.2 entries with concrete mechanisms (version label SoT, per-extent AACS anchor, E7017/E7022 no-key split, 3-sector unit grid) 2026-06-26 22:26:15 -07:00
Matthew Jackson b9c0a7c208 changelog: add 1.1.0-beta.1 version-stamp consistency, DTS-HD MA core-sync, and TrueHD DTS-monotonicity fixes 2026-06-26 22:16:45 -07:00
Matthew Jackson 3c6351975b docs(fvi): standardize table headers and multi-line dense cells
Use 'Semantics / reference' as the last-column header across all spec
tables, and break enum/option/note cells onto separate lines with <br>
for readability. Formatting only; no semantic changes.
2026-06-26 21:42:31 -07:00
Matthew Jackson fda771a06a mux: single source of truth for the version label (VERSION_LABEL/MUX_APP)
The CLI --version, the MKV muxing/writing-application field, and the FVI generator
all derive from one libfreemkv const, so a binary reports the exact same label it
stamps into the files it produces — no split-brain where an MKV claims one version
and the binary another.
2026-06-26 21:33:19 -07:00
Matthew Jackson 2f1697d587 docs(fvi): pretty-print JSON blocks in FVI format spec
Reformat all three JSON code blocks (section 8 ext example, Appendix A
Header schema, Appendix A Record schema) with consistent 2-space
indentation and valid syntax. No field or semantic changes.
2026-06-26 21:21:26 -07:00
Matthew Jackson fc95d55d57 mux: allow build-time version label override (FREEMKV_BUILD_LABEL)
build.rs emits FREEMKV_VERSION = FREEMKV_BUILD_LABEL when that env is set
(non-empty), else the Cargo package version. The muxing/writing-application
field and the FVI generator tag use it, so a pre-release/test build can be
stamped without bumping Cargo.toml and disturbing the tag-pinned [patch]
version matching. The git short hash is still appended either way.
2026-06-26 19:38:13 -07:00
Matthew Jackson 571c613c39 mux: fix non-monotonic audio DTS (TrueHD + DTS-HD MA) and stamp builds with git hash
TrueHD: when the PES PTS lags the access-unit cadence, resync to the PTS
but never snap the running timestamp backward, so the emitted DTS stays
monotonic across the resync (next_pts_ns = max(next_pts_ns, pts)).

DTS-HD MA: size each EXSS extension substream exactly from its header
(exss_frame_size) and skip it as a unit, so a false 0x7FFE8001 core sync
inside the lossless extension payload can no longer split the access unit
and truncate the extension. Falls back to a bounded scan when the header
is unparseable.

Provenance: build.rs bakes the git short hash into GIT_SUFFIX; the muxing/
writing-application field and the FVI generator tag now record the exact
build (e.g. "freemkv 1.1.0-beta.1 (g835cc99)"), so any output file is
traceable to the revision that produced it.
2026-06-26 19:26:24 -07:00
Matthew Jackson a12f3b6c78 Remove keydb download/save from the library
keydb I/O moves out of libfreemkv into freemkv-keysources
(KeydbSource::save / ::update). Delete src/keydb.rs entirely (save,
http_get, default_path, write_atomic, UpdateResult) and drop `pub mod
keydb;` — http_get had no real callers. The shared Keydb* Error variants
stay in error.rs (keysources raises them; the every-error-has-a-code
contract depends on them). flate2 is no longer used here, so drop it
from Cargo.toml (zip stays for labels/jar.rs).

CHANGELOG: note the keydb-I/O move; reword the rc.5.2
DefaultDecodedFieldDuration entry to state only the action taken (the
revert) rather than an unverified Windows-fps outcome.
2026-06-26 17:33:57 -07:00
Matthew Jackson a6073d6866 DVD vob_start absolute rebase + rc.5.3 audit fixes
- ifo.rs: rebase VTS title VOBS to absolute disc LBA (file_start_lba +
  vtstt_vobs); fixes DVD rips opening on the menu region instead of the
  movie (e.g. SOTL). Adds absolute-placement regression test.
- aacs/boil.rs: add mk_from_pk primitive (PK -> MK via MKB walk).
- dvdnav/: nav-VM command decoder + start-cell resolver seam, parked
  behind USE_NAV_RESOLVER (kept compiled, never executed).
- mux: FVI src.byte within-sector per spec; Unknown colour -> CICP
  unspecified (2,2,2,1); demux clear PCS -> NORMAL; ts.rs feed() base
  reset + boundary provenance fix.
- Assorted audit fixes (doc/comment/test accuracy) across the crate.
2026-06-26 17:03:58 -07:00
Matthew Jackson b4b05df97a Magic-number/taxonomy pass: central wire-format + sector + unit consts
- libfreemkv::consts: coding_type::* (ES coding-type bytes), pes_stream_id::*
  + PAYLOAD_RANGE, SECTOR_BYTES (usize) + SECTOR_BYTES_U64 (offset math)
- replace bare wire-code/sector literals across disc, mpls, clpi, labels,
  m2ts_mux, ps, tsmux, file_sector_source, extract
- remove two unreachable secondary-stream match arms in mpls parse_stream_entry
2026-06-26 13:20:21 -07:00
Matthew Jackson 52b9731453 AACS pipeline reshape + TrueHD metadata + central consts + clippy/fmt clean
- AACS: delete in-lib keydb parser (Step 3); boil-down primitives
  (mk_from_dk/vuk_from_mk/uk_from_vuk) + newtypes; KeySource->get_uk(ctx)+
  ResolveCtx; Unlocker->unlock()->Result<Vid,UnlockError> + AacsCertUnlocker;
  OEM bus-key gate (AacsBusKeyUnavailable); structured ResolutionTrace (Step 4).
- TrueHD: sample-rate from major-sync, Atmos label, 44.1k AU duration.
- consts: central media/format constants module; 17 duplicate const-defs
  centralized (sector/TS-packet/source-packet); mpls stream-entry + category
  codes named.
- clippy --all-targets -D warnings clean (1.86); fmt clean; 2199 lib tests.
2026-06-26 12:19:24 -07:00
Matthew Jackson fd8abc3684 fix(libfreemkv): rc6 hardening pass — mux timeline/colour/PCR, demux panic sentinel, parser robustness + doc accuracy
Surgical fixes (each with a regression test that fails without the change):

mux/mkv.rs, mux/demux_sink.rs: drive the clip-boundary timeline epoch
off the resolved PRIMARY VIDEO track, not the literal stream index 0.
An M2TS/PMT title can list an audio ES before video, so streams[0] may
be audio; a non-video epoch driver ratchets the frontier and inflates
the timeline. mkv cluster-opening falls back to track 0 for audio-only
titles so they still open clusters.

mux/codec/ac3.rs: correct ACMOD_CHANNELS — acmod=5 (3/1) is 4 channels,
not 3 (was undercounting a 3/1 stream); fix the A/52 Table 5.8 doc.

disc/mod.rs: HDMV coding_type 0x91 (Interactive Graphics / menus) no
longer maps to PGS subtitle — it falls through to Unknown so the PMT/STN
walker drops it instead of surfacing a bogus subtitle track.

mux/videomap.rs + mux/mkv.rs: FVI colour now mirrors the MKV muxer's CICP
precedence (measured CICP authoritative; HDR-driven PQ/HLG transfer
override) via a shared cicp_for_video helper, so the two sinks can't
disagree (HDR10 BT.2020 no longer emits SDR transfer 14).

mux/mkvstream.rs: saturating_add on cluster_ts + rel_ts so an adversarial
CLUSTER_TIMESTAMP near i64::MAX can't overflow/panic before the existing
saturating_mul.

mux/timeline.rs: tighten the tail-straggler clamp so a normal new-epoch
non-video frame leading the sparse video frontier by >3s is not demoted
into the previous clip's epoch.

mux/m2ts_mux/mod.rs: re-stamp PCR per video TS packet (mid-PES), not only
at PES boundaries, so a large UHD I-frame can't open a multi-second PCR
gap; modular 33-bit PTS rebasing so a real 90 kHz clock wrap is not
collapsed to PTS 0 (pre-base frames still floor to 0).

io/byte_prefetcher.rs, sector/prefetched.rs: wrap the producer feed loop
in catch_unwind and emit a typed error sentinel on panic, so a mid-stream
producer panic is not read as a clean EOF at the demux boundary (which
would silently truncate the mux).

mux/codec/h264.rs: extend HIGH_PROFILES to the full ISO/IEC 14496-15 set
that mandates the avcC chroma/bit-depth extension (adds 244 et al.).

Doc/comment accuracy: css/mod.rs (50000 sectors, not scrambled-sectors),
aacs/decrypt.rs (decrypt_unit already-clear path), ifo.rs (TT_SRPT at
0xC4), css/lfsr.rs (LFSR0 24-bit; TAB1-then-XOR cipher; real scramble-flag
predicate), disc/read_error.rs (for_sweep does bounded transient retries).

Skipped: keydb.rs SSRF guard (low/latent, no live caller) — a hard
loopback block breaks an existing behavioral test that exercises the
header-EOF path over a loopback server; a clean fix needs a resolver test
seam beyond this surgical pass. The sibling keydb_fetch.rs comment fix is
out of scope (freemkv crate).
2026-06-25 23:39:03 -07:00
Matthew Jackson 2307f0d79a feat(mux): emit HDR10 static metadata from HEVC SEI
Parse the two HDR10 HEVC SEI messages and emit the corresponding
Matroska Colour metadata, only when actually present in the bitstream
(SDR / no-SEI tracks omit it; nothing is fabricated).

Parse (Rec. ITU-T H.265 Annex D):
- Mastering Display Colour Volume SEI, payloadType 137 (D.2.28):
  display_primaries_x/y[3] (SEI order G,B,R), white_point_x/y
  (0.00002 units), max/min_display_mastering_luminance (0.0001 cd/m²).
- Content Light Level Info SEI, payloadType 144 (D.2.35):
  MaxCLL / MaxFALL (cd/m² integers).
HevcParser::scan_sei walks the sei_rbsp ff-extension payloadType/
payloadSize coding and de-emulates (00 00 03) before reading, reusing
the existing strip_emulation_prevention helper. Both SEI are required
before any metadata is surfaced; SEI NALs still pass through unchanged.

Carry: the measured Hdr10Metadata rides PictureInfo (the same per-coded-
picture seam FieldOrder uses), flowing through from_codec_frame onto
PesFrame.coding to the deferred-muxer activate path, where
apply_coding_to_track stamps it on the video track before the header is
written. Set only when both SEI were seen.

Emit (RFC 9559 / Matroska): new Colour children in ebml.rs
(MasteringMetadata 0x55D0, Primary R/G/B + WhitePoint chromaticity
0x55D1..0x55D8, Luminance max/min 0x55D9/0x55DA, MaxCLL 0x55BC,
MaxFALL 0x55BD). write_hdr10 converts chromaticity SEI int × 0.00002 →
Matroska float, luminance SEI int × 0.0001 → cd/m² float; MaxCLL/MaxFALL
are uints verbatim. SEI primary index 0/1/2 (G/B/R) mapped to the
Matroska R/G/B element layout. Emitted only when hdr10 is present.

Tests: SEI parse with exact raw values, requires-both-SEI, SDR omission,
and emulation-prevention stripping (hevc.rs); muxer emit with exact unit
scaling + SDR omission of MasteringMetadata/MaxCLL/MaxFALL (mkv.rs);
apply_coding_to_track HDR10 plumbing (mkvstream.rs).
2026-06-25 21:59:43 -07:00
Matthew Jackson 3d2f115691 test(disc): pin the -t 1 = main-feature contract (DVD); fix 2 test clippy nits
Owner-flagged invariant: freemkv -t 1 ALWAYS selects the main feature because the
CLI's title 1 maps to titles[0] and the list is ordered by canonical_title_order
(main feature first). Adds a DVD-shaped contract pin asserting titles[0] is the
movie after sorting (a regression there is a title-ordering bug, not a remux issue),
complementing the existing branching-UHD / normal-disc order tests.

Also clears two pre-existing test-only clippy nits surfaced under --all-targets:
unused `lba` in ClearStubReader::read_sectors, and an unneeded `mut` on the
h264 population test's closure.
2026-06-25 21:42:52 -07:00
Matthew Jackson 9db9b9bdf6 test: real-executing coverage for mux codecPrivate/DefaultDuration, sweep damage-jump, patch watchdog clock seam, and AACS CBC KAT
mux/mkv: assert emitted CODEC_PRIVATE bytes verbatim for H.264/HEVC/VC-1/
MPEG-2 (direct TrackEntry child, not nested in Video) and DefaultDuration ns
for all eight frame rates, read back out of a real MkvMuxer.

disc/sweep: end-to-end Disc::sweep against a synthetic MockReader with an
injected bad region, asserting the resulting mapfile marks the clean lead
Finished and the failed batch + zero-filled skip-ahead gap NonTrimmed,
proving the Pass-1 damage-jump engaged.

disc/patch: introduce a minimal clock seam (fn() -> Instant on the internal
PatchLoopState, defaulting to Instant::now) so the per-range and whole-pass
watchdogs are deterministically testable; public API and callers unchanged,
production behavior identical. Add tests that advance a fake clock to trip the
range budget and whole-pass stall predicate.

aacs: add an AES-128-CBC known-answer test for aes_cbc_decrypt using the
published NIST SP 800-38A F.2.2 vector (blocks 1..3 exact; block 0 via the
documented fixed-AACS-IV substitution).
2026-06-25 21:37:20 -07:00
Matthew Jackson 3f38ee6fd5 mux: reconcile fvi:// video-index sink onto rc6
Port + adapt the freemkv native per-picture video index (FVI) from the
old feat/fvi-sink branch onto rc6's codec-agnostic PictureInfo model.
This is a surgical adaptation, not a merge.

Adaptations (fvi_sink.rs, videomap.rs, tests/fvi_pipeline.rs):
- Retarget from the removed crate::mux::codec::mpeg2::PictureInfo (raw
  public fields) to rc6's authoritative crate::mux::codec::PictureInfo
  in codec/coding.rs, via its accessors.
- type from coding_type() -> CodingType{I,P,B}; emitted for ANY frame
  that carries coding (every video codec now fills it), with the
  keyframe-flag I/P fallback only when coding is absent.
- Replace the mpeg2-only tff/rff/progressive members with codec-agnostic
  members derived through the accessors: field_order (tff/bff/progressive)
  and progressive, emitted ONLY when the codec measured the signal
  (Option::Some) and omitted otherwise; plus nb_fields.
- Test fixtures rebuilt via PictureInfo::mpeg2(CodingType, Mpeg2Coding{..})
  / coding_type_only(..); added measured_cicp: None to VideoStream
  literals for rc6's struct.

Honesty decision (key / random-access):
- The codec-agnostic PictureInfo carries NO GOP-closure (no closed_gop /
  gop_start), so key is set from the frame's intra / decode-restart flag
  (frame.keyframe == coding.keyframe() for video), NOT a fabricated
  clean-RAP claim. The old gop member is honestly omitted. FVI_FORMAT.md
  is updated to document this as a limitation: key is an intra picture /
  parser-flagged decode-restart point; MPEG-2 open-GOP clean-RAP precision
  (closed_gop) is not currently distinguished. §7.1 rewritten for the
  new field_order/progressive/nb_fields members.

Wiring:
- mux/mod.rs: pub(crate) mod fvi_sink; pub(crate) mod videomap
  (#[allow(dead_code)] on videomap — the VideoMap accumulator is staged
  for side-channel reuse, sink builds records directly); pub use
  fvi_sink::FviSink.
- mux/resolve.rs: add the fvi:// output scheme to StreamUrl, parse_url,
  scheme(), path_str(), input() (write-only reject) and output()
  (constructs FviSink), mirroring the mkv:///demux:// patterns.

Provenance fix surfaced by the end-to-end test:
- pipelined_stream::consume_ps was dropping the PS demuxer's byte-exact
  source stamp (source: None) when rebuilding PesPacket, so PS/DVD-path
  frames reached the mux/index with no provenance (FVI src null). Carry
  ps.source through, matching the TS path; the real-pipeline fvi test now
  sees the stamped src sectors.

Gate: cargo +1.86 fmt + clippy --lib -D warnings clean; cargo +1.86 test
--lib (2182 passed) and --test fvi_pipeline (2 passed); precommit.sh
libfreemkv green.
2026-06-25 21:18:15 -07:00
Matthew Jackson 350df85439 mux/vc1: measure coding type from progressive PTYPE; carry source
Honest PictureInfo population for VC-1 — completes the set (MPEG-2, H.264, HEVC,
VC-1 all populate now).

- Read the advanced-profile sequence header INTERLACE flag (SMPTE 421M §6.1.1,
  bit 41), de-escaping emulation-prevention bytes as parse_vc1_resolution does.
- For a PROGRESSIVE sequence, decode the picture-layer PTYPE VLC (0=P, 10=B,
  110=I, 1110=BI→I, 1111=Skipped→P) from the first bits after the frame start
  code. For INTERLACED (FCM/FPTYPE precede PTYPE) or simple/main/unknown, decline
  → coding stays None: honestly absent, never read at a wrong bit offset.
- Set coding = coding_type_only(...) and source = pes.source; field order is not
  decoded, so field_order() is honestly None.
- Reuses the shared startcode::BitReader (no new primitive).
- Tests: progressive I/P/B from real PTYPE VLCs + source carry + field-order
  absence; interlaced honest-decline.
2026-06-25 20:50:36 -07:00
Matthew Jackson 62a013d906 mux/hevc: measure coding type from slice_type; reuse shared BitReader
Honest PictureInfo population for HEVC, on the same principle as H.264.

- Consolidate the bit reader: hevc.rs had its own BitReader (used by the SPS
  parser); h264 just gained one in startcode. Promote startcode's to the single
  shared pub(crate) reader (adds read_bits) and delete hevc's copy — one proven
  primitive, reused (SPS parse + both slice-type decoders).
- hevc: decode slice_type from the first coded slice's slice_segment_header
  (H.265 §7.3.6.1) → I/P/B (§7.4.7.1). The offset to slice_type depends on
  num_extra_slice_header_bits, which lives in the PPS — so we parse it from the
  ACTIVE PPS (§7.3.2.3) and only measure slice_type when that PPS is known.
  With no active PPS we decline rather than guess: coding stays None, honestly
  absent. Set coding = coding_type_only(...) and source = pes.source; field
  order (pic_struct SEI) is not decoded, so field_order() is honestly None.
- Tests: I/P/B from real slice headers, source carry, field-order absence, and
  the no-PPS honest-omission case.
2026-06-25 20:44:15 -07:00
Matthew Jackson 7db0a154e9 mux/h264: measure coding type from slice_type; carry source provenance
Honest PictureInfo population for H.264 — the foundation that lets the video
index claim a frame's type and be believed (a faked "P" on a B-frame would make
the index confidently wrong).

- startcode: shared minimal MSB-first BitReader with Exp-Golomb ue(v), reused by
  H.264 (and next HEVC). Documents the emulation-prevention caveat: only the
  leading slice-header fields are read, where 00 00 03 cannot intervene.
- h264: decode first_mb_in_slice + slice_type (H.264 §7.3.3) from the first
  coded slice and map to I/P/B (§7.4.3 Table 7-6; SP→P, SI→I). Set
  coding = PictureInfo::coding_type_only(...) and source = pes.source. Field
  order is NOT decoded here, so field_order() stays None — honestly absent,
  never guessed.
- Tests: I/P/B from real ue-encoded slice headers, source carry, field-order
  absence; BitReader Exp-Golomb table + truncation.
2026-06-25 20:38:09 -07:00
Matthew Jackson 2a11844cad mux: codec-agnostic PictureInfo + provenance; measure field order, never guess
Carry per-picture truth and byte-exact source provenance THROUGH the stream so
the muxer (and the upcoming video index) read MEASURED facts instead of
assuming them. Honest data in, honest data out.

- codec/coding.rs: codec-agnostic PictureInfo (CodingType / FieldOrder + the
  accessors field_order/coding_type/nb_fields/progressive/keyframe). Each codec
  folds its raw signals in; consumers use only accessors, never branch on codec.
- mpeg2: builds PictureInfo from the picture coding extension and carries it +
  SourcePos (source_marks, parallel to pts_marks) on every emitted frame.
- pes / codec::Frame: additive `coding` + `source`, forwarded through the
  highway; None for audio/subtitle and the network/stdio deserialize hop.
- mkvstream: DEFER muxer construction until the first coded picture, set the
  video track's FieldOrder from the MEASURED value, THEN write the header —
  right the first time, no guess, no seek-back. An interlaced track that arrives
  with no measured order is LOGGED loudly and left UNDETERMINED, never faked.
- mkv: MkvTrack::video no longer guesses TFF (a bitstream property the scan
  cannot know is UNDETERMINED at build). Removed VideoStream::top_field_first
  (the dead scan-time guess) crate-wide.
- Tests: parser population (every PictureInfo facet + per-PES source carry) and
  mux-stream consumption (measured -> correct; missing -> UNDETERMINED, not
  faked). Two obsolete tests updated only after confirming (their own comments)
  they existed to enforce the deleted hardcoded-TFF.
2026-06-25 20:13:55 -07:00
Matthew Jackson 051e5b5f25 mux/mkv: spec-conformance fixes (field order, CICP, VobSub idx, SeekHead, AC-3)
- FieldOrder now derives from the bitstream's measured top_field_first
  (Some(true)→TFF, Some(false)→BFF) instead of hardcoding TFF for all
  interlaced content; falls back to TFF when unmeasured. Adds
  VideoStream::top_field_first; DVD/BD scan sets None with a precise
  TODO(spec) for parser→title plumbing.
- CICP (matrix/transfer/primaries/range) now prefers measured CICP from
  the bitstream (VideoStream::measured_cicp) over the coarse ColorSpace
  enum, so the container stops assuming a colour space the stream may
  contradict. Enum remains the fallback.
- VobSub S_VOBSUB CodecPrivate now emits a `size: WxH` line ahead of the
  palette per the .idx format so players place/scale subs correctly.
- SeekHead: when zero cues are written, the CUES Seek entry is Voided
  instead of leaving a dangling pointer to the Cues offset (now Tags/EOF).
- AC-3 Channels back-patch offset is captured from the writer instead of
  the hardcoded chan_elem_pos+2 (decoupled from the VINT width choice).
- Hoisted inline CICP codes and the dvcC fourcc to named constants citing
  ITU-T H.273 / RFC 9559; fixed the stale FieldOrder comment.
- DefaultDuration vs pulldown: precise TODO(spec) left (needs the same
  parser→title channel as top_field_first).

Tests: BFF-from-measured-flag, measured-CICP-overrides-enum, VobSub size:
line present/omitted, zero-cue SeekHead Void. precommit (1.86) green.
2026-06-25 18:40:30 -07:00
Matthew Jackson 3e2740d815 wip: rc6 VFR/DVD/CSS base (held for bulletproofing + split) 2026-06-25 18:17:08 -07:00
Matthew Jackson ef5a1b9ff7 demux: solidify sink — reuse canonical primitives, fix 3 bugs
Delete re-implementations in the demux:// sink and wire to proven helpers;
keep only genuinely-new functionality.

- AnnexB reframing: delete the sink's local length_prefixed_to_annexb (it
  break'd on a zero-length NAL, dropping the rest of the access unit) and
  call the canonical append_length_prefixed_as_annex_b in mux::hevc, which
  skips just the empty NAL.
- HEVC param sets: delete hvcc_param_sets; reuse hvcc_to_annex_b.
- avcC param sets: hoist as the new canonical avcc_to_annex_b in mux::hevc,
  next to hvcc_to_annex_b (the symmetry point); the sink calls it.
- PGS .sup: emit a synthetic clear display set (empty PCS + END) at
  pts + duration_ns so subtitles time out instead of lingering to EOF.
- TimelineContinuity: move verbatim into the shared mux::timeline module
  (with the prev_offset straggler-remap intact) and use it from both the
  MKV muxer and the demux sink; delete the sink's drifted TimelineRebase
  copy (which lacked the straggler branch).
- VobSub .idx: emit the conventional 'id: <lang2>, index: 0' line mkvmerge
  reads to assign the subtitle language; palette reuse unchanged.
- output(): seed DemuxOptions.base from title.playlist when non-empty.

New constants for the PGS clear-segment framing and avcC header cite the
public HDMV PGS (BD-ROM Part 3) and ISO/IEC 14496-15 specs.

Tests: a zero-length NAL mid-frame no longer truncates the AU; a frame with
duration_ns produces a .sup clear segment; existing demux tests stay green.
2026-06-25 17:58:49 -07:00
Matthew Jackson 6c55f0cf20 mux: highway-level regression — DVD video keyframe + duration survive the pipelined path
The DVD seek-index concern is that the mux highway
(PsDemuxer -> PipelinedPesStream codec parse -> frame out -> muxer)
might drop the keyframe flag or per-frame duration that Mpeg2Parser sets
on each Frame, which would stop the muxer's cluster/cue open
(keyframe && track 0) from ever firing and leave a DVD MKV with many
clusters and zero cues.

Add dvd_highway_preserves_video_keyframe_and_duration: drives a real
Mpeg2Parser through PipelinedPesStream via DemuxBatch::Ps batches (one
PTS-stamped MPEG-2 PS video PES per GOP, decode-order I + P/B), reads the
frames back through the highway's read(), and asserts the GOP-opening
I-frames arrive as keyframes, every frame keeps its duration (BlockGroup
path), and video routes to track 0. Guards from_codec_frame's
keyframe/duration propagation across the demux-thread + recycled-buffer
highway, which the prior cue tests (codec parser straight into the muxer)
did not exercise.

Verified against the real current-VFR-build output
(Greenland-feature.mkv, today): 3365 clusters / 3365 CuePoints, all
BlockGroups, cues resolve to clusters and track the video — the highway
preserves the flags and the seek index is complete.
2026-06-25 16:58:04 -07:00
Matthew Jackson 95c3c0486b mkv: regression test — DVD VFR/BlockGroup output gets per-cluster cues
The existing cue tests (cue_count_equals_cluster_count,
cue_positions_resolve_to_clusters, cue_times_match_cluster_timestamps)
all feed frames with duration_ns=None, so they exercise only the
SimpleBlock write path (UHD/HEVC). DVD MPEG-2 video is now VFR: every
coded picture carries a per-frame duration_ns=Some(..), so it is written
as a BlockGroup, not a SimpleBlock. That cue path was untested.

Add cue_count_equals_cluster_count_blockgroup_vfr, which drives the real
Mpeg2Parser end-to-end (decode-order frames, non-monotonic B-frame
display PTS, telecine field durations) into the muxer and asserts the
output is wholly BlockGroup (no SimpleBlock) and that the Cues index has
exactly one cue per cluster, with every cue resolving to a real cluster.
Guards the DVD seek index (scrub/fast-forward) against regressing to the
chapter-seek-only, zero-cue state.
2026-06-25 16:32:32 -07:00
Matthew Jackson 73b1bb0509 demux: add demux:// per-track elementary-stream sink
New write-only pes::Stream sink that taps the per-track PesFrame stream
(the seam right before MKV muxing) and writes each track to its own
elementary-stream file, plus chapters and per-audio-track delay metadata.
Purely additive — the MKV mux path is untouched.

- mux/demux_sink.rs: DemuxSink + EsWriter dispatch. Pass-through for
  codecs whose Frame.data is already standalone ES (MPEG-2, VC-1, AC3/
  E-AC3, DTS/DTS-HD, TrueHD, LPCM). Non-trivial writers:
  - AnnexBWriter: reframes hvcC/avcC 4-byte-length-prefixed NALs to
    Annex-B and prepends VPS/SPS/PPS parsed out of the codec_private
    configuration record (HEVC .hevc / H.264 .h264).
  - PgsSupWriter: rebuilds the HDMV 'PG' segment framing the parser
    strips, with 90kHz PTS/DTS (.sup).
  - VobSubWriter: writes raw SPUs to .sub and synthesizes the .idx
    sidecar (palette + per-SPU timestamp/filepos).
  - Delay-in-filename (mkvmerge-readable 'DELAY <n>ms') + chapter XML/OGM
    export. TimelineRebase ports the MKV muxer's seamless-branch epoch
    logic so per-track ES timestamps stay continuous across clip joins.
- mux/resolve.rs: StreamUrl::Demux variant + scheme/path_str/parse_url/
  input(write-only)/output arms.
- mux/mod.rs: module + public type re-exports.

16 unit tests: Annex-B reframing, hvcC/avcC param extraction, delay
sign/rounding + mkvmerge-regex match, PGS .sup framing, VobSub .idx
synthesis, chapter XML/OGM, timeline rebase, and end-to-end file-keying
by track + track selection.
2026-06-25 16:23:37 -07:00
Matthew Jackson 86de7fe931 css: skip clear/uncrackable extra titles instead of failing the whole mux
A genuinely-clear or uncrackable extra title (a tiny menu/nav stub) no
longer poisons a multi-title rip with a false CssKeyMissing (E7023).

- decrypt_keys_for_title_checked: re-crack a non-overlapping VTS via
  crack_key_outcome and report title_is_clear when the title's own
  extents show no scrambling. A genuinely-clear stub on an otherwise-CSS
  disc needs no key.
- ensure_title_decryptable: pass a clear stub without a key; a scrambled-
  but-uncrackable title still hard-fails with CssKeyMissing.
- is_scrambled_pack: hardened scramble-evidence gate for the crack scan —
  requires the MPEG-PS pack-start signature before trusting the 0x14
  scramble bits, so a clear stub with stray 0x14 bits can't flip
  saw_scrambled. The descramble loop keeps the looser is_scrambled.
- mux/resolve: ISO per-title gate routes through the clear-aware check.
2026-06-25 13:43:04 -07:00
Matthew Jackson 0993d60e8c progress: expose bytes_retryable so 'lost' counts only failed reads
Add PassProgress::bytes_retryable_total (NonTrimmed/NonScraped — failed
and awaiting retry), distinct from bytes_pending_total which also folds
in not-yet-attempted (NonTried) bytes. Set it at every construction site
(Sweep from the snapshot, Patch from stats, 0 for sequential/placeholder
paths). The disc-level 'lost' display in the CLI can now use
unreadable+retryable instead of unreadable+pending, so a healthy
in-progress rip no longer reports its unread remainder as lost.
2026-06-25 13:04:37 -07:00
Matthew Jackson b080fb6e1b Resume plain disc->iso copy after interrupt
Disc::copy gated its entire mapfile-resume dispatch behind
`if opts.multipass`, so a plain (non-multipass) `disc:// iso://`
copy always called sweep_internal(resume=false) — wiping the
mapfile + ISO and re-sweeping from sector 0. The CLI help and
rip_iso examples promise "auto-resumes if interrupted", and the
sweep already flushes a crash-safe mapfile per block, so the only
missing piece was consulting it on re-run.

Lift the mapfile-resume dispatch out of the multipass guard:
clean->no-op, size-mismatch->fresh sweep, NonTried tail->resume
sweep now apply to both modes. The patch (Pass N) dispatch on
retryable bytes stays multipass-only; a plain copy with a
fully-attempted mapfile returns a terminal result instead of
restarting. autorip (always multipass=true) is unchanged.

Add a regression test that an interrupted plain copy resumes the
NonTried tail and does NOT re-read the Finished prefix.
2026-06-25 08:24:00 -07:00
Matthew Jackson d87defef9e libfreemkv: align E7021→E7017 in comments/test-names (keys-but-no-VID is AacsVidUnavailable) 2026-06-25 07:59:54 -07:00
Matthew Jackson 8d8f2fa463 libfreemkv: changelog — rc.6 entry (+ backfill rc.5.3, mark rc.5.2 released) 2026-06-24 23:56:41 -07:00
Matthew Jackson db64d6cea5 test: strengthen AACS resolve-reason + unlocker seam coverage (rc.6 WS3)
Targeted tests for the rc.6 surfaces, strengthening (not duplicating) the
regression tests the rc.6 commits already shipped. No production code changes.

aacs/keys.rs — resolve_keys_with_reason / classify_resolve_failure:
The E7021/E7022 split is already proven end-to-end through the
ensure_decryptable gate (disc/mod.rs). These pin the classifier directly at
the keys.rs seam for the branches the gate test does not reach:
  - processing-keys-only + zero VID -> VidUnavailable (the gate test only
    exercises the device-keys arm of has_derivation_material).
  - VID PRESENT + material -> NoMaterial: a non-zero VID must never be
    reported as VidUnavailable however much material is on hand (the has_vid
    short-circuit; the gate test only uses the zero-VID sentinel).
  - VID present + no material -> NoMaterial.
  - version dispatch: version 1 routes the V10 resolver (stamps V10), any
    other value routes the V20->V21 chain; a resolved disc returns Ok, never
    Err(ResolveFailure).

unlock.rs — Unlocker seam introspection + ordering:
  - matching_name reports the first matching unlocker without running it, and
    is None for an unsupported drive; registered_count grows after a
    registration (monotonic check — the registry is process-wide and shared
    across the unlock tests, so no exact-delta assertion).
  - route_unlock first-registered-match-wins: two unlockers matching the same
    identity, the earlier-registered one runs and the later is never consulted.
2026-06-24 23:40:36 -07:00
Matthew Jackson 7f1827565a fix(extract): anchor AACS unit base per-extent + crash-safety hardening
The dir:// extractor set the AACS unit-alignment base ONCE to the first
extent's start, then read every extent against that single base. For a
multi-extent (fragmented / Long-AD / continuation-ICB) file the second
and later extents start at arbitrary LBAs whose offset from the first
extent is generally not a multiple of 3 sectors, so the first read of
each later extent failed the decrypt-on-read gate
(is_unit_aligned(lba, unit_base)), returned DecryptFailed, and recorded
the whole extent as a zero-filled hole even though the data was readable.
Re-anchor the unit base PER extent (matching mux/disc.rs and
sector/prefetched.rs), so each clip's encrypted region gates on its own
unit grid. Same bug class as the rc.5.2 clip-anchor fix.

Also harden the extract + keydb write paths:
- finalize_file: fsync the .partial after set_len (the truncation runs
  on a second handle the content fsync never touched) and fsync the
  parent dir after rename so the new dirent is crash-durable.
- keydb write_atomic: fsync the parent dir after rename (POSIX dirent
  durability), matching the finalize_file pattern.
- AACS tail batch: document that decrypt_sectors' trailing-partial
  contract already handles the short final unit; no math change.
- decrypt-loss delta loads use Acquire (defensive happens-before if
  file extraction is ever parallelised).
- is_windows_reserved: add CONIN$/CONOUT$/CLOCK$; reserved names are
  now substituted (prefix _) instead of aborting the whole tree walk, so
  a legal Linux-authored NUL.cfg extracts.
- http_get header cap: >= MAX_HEADER_BYTES (was > , one byte over).

Regression tests: multi-extent AACS file (Δ4-sector extents) extracts
both extents with zero loss; focused per-extent alignment-arithmetic
test; reserved-name substitution assertions.
2026-06-24 23:20:15 -07:00
Matthew Jackson 787b4ff54c aacs: split no-key resolution reason into E7021 vs E7022
When key resolution had derivation material (device or processing keys)
but no Volume ID was available to derive the unit key, surface
Error::AacsVidUnavailable instead of the generic NoDiscKey. When there
was no usable key material at all, keep NoDiscKey.

resolve_keys_classical / resolve_keys_v21 still return a bare
Option<ResolvedKeys> (all existing callers unchanged); a new
resolve_keys_with_reason wrapper threads the typed ResolveFailure
(VidUnavailable | NoMaterial) out. decrypt_with uses it; the
ensure_decryptable_keys gate maps a captured AacsVidUnavailable reason to
E7021, otherwise E7022. No decryption math, key derivation, or descramble
logic changed -- only the reason reported on a resolution failure.

Adds ensure_decryptable_aacs_vid_unavailable_vs_no_key proving both
branches (device-keys + zero VID -> E7021; no keys -> E7022).
2026-06-24 22:38:54 -07:00
Matthew Jackson 75be0159e6 v1.0.0-rc.5.3: bump version 2026-06-24 21:50:00 -07:00
Matthew Jackson 8c9672819e libfreemkv: plain-English, source-agnostic AACS scan logs
Rewrite the AACS scan/VID trace lines so a reader understands them without
opening the source: name the real thing (AACS host certificate, Volume ID,
decryption key), say "key source" not "keydb", and describe what happened.
The VID flow is unchanged (unlocker OEM VID → cert handshake → continue); a
missing VID is logged, never fatal. All strings are in tracing macros (the
sanctioned debug-log channel) — no English added to any Error.
2026-06-24 21:47:37 -07:00
Matthew Jackson c9f9b1dd08 libfreemkv: keydb default location is local to the executable
default_path() now resolves <dir of current exe>/keydb.cfg with no OS-config
fallback — freemkv is a portable standalone binary, so the keydb lives next to
it. --keydb still overrides.
2026-06-24 20:46:31 -07:00
Matthew Jackson d94e54e52b libfreemkv: dir:// — decrypted file-tree extraction (Disc::extract_tree)
Sibling of Disc::copy specialized to write per-file instead of a whole ISO
image, decrypting on the way out: walk the UDF tree, read each file's extents
through the shared DecryptingSectorSource (AACS unit-aligned, CSS per-VTS),
strip AACS/, sanitize host paths per component, .partial+rename, 1-shot with
per-file loss accounting (no mapfile; recovery stays the iso:// multipass
path). Reuses UdfFs + the decrypt seam; only the per-file orchestration is new.
2026-06-24 20:46:31 -07:00
Matthew Jackson 7ace1aa3c6 libfreemkv: hard-error when decryption is needed but no key is available
Adds Disc::ensure_decryptable / ensure_decryptable_keys, the single decrypt
gate consulted before any copy or mux. When the source is encrypted and no
key resolved (and not --raw), abort with a typed error and write nothing,
instead of silently emitting ciphertext at exit 0. Unifies the prior ad-hoc
CSS/AACS checks.
2026-06-24 19:33:09 -07:00
Matthew Jackson 20c1035730 v1.0.0-rc.5.2: bump version 2026-06-24 17:14:10 -07:00
Matthew Jackson 8db1d01833 libfreemkv: rc.5.2 changelog — audio routing, AACS clip-anchor, recovery, TopGun, keysources move 2026-06-24 17:12:17 -07:00
Matthew Jackson 3bdf490fce libfreemkv: rc.5.2 SOTL video — full Windows fps, opening-GOP proof, self-sufficient log-level 3
Three Silence-of-the-Lambs (R2 PAL SD-DVD) follow-ups for rc.5.2.

SUB-TASK 1 — Windows Explorer showed 12.5 fps (half) for the 576i25 track.
Root cause: the DefaultDecodedFieldDuration (20 ms field) element rc.5.1
added to "fix" Windows fps did the opposite. With FlagInterlaced=1 +
DefaultDuration=40 ms + DefaultDecodedFieldDuration=20 ms, Explorer halved
to 12.5 fps and MediaInfo flipped to VFR. MakeMKV's correct rip omits the
field-duration element, keeps FlagInterlaced=1 + FieldOrder=TFF +
full-frame DefaultDuration (40 ms), and Explorer shows 25 fps / MediaInfo
CFR. Fix: MkvTrack::video now passes field_duration_ns == 0 so the element
is no longer written; the 1/DefaultDuration = 25 fps signal (the only one
tools trust) is the full-frame value. Interlace signalling (FlagInterlaced,
FieldOrder=TFF) is retained — MediaInfo reads scan type from the MPEG-2 ES
picture coding extension, so it still reports Interlaced / Top Field First.
Tests pin the new TrackEntry elements (element present/absent + values).

SUB-TASK 2 — opening "menu"/still-frame video. Traced the MPEG-2
opening-GOP path; the wrong/last seq header and PTS-floor-to-0 hypotheses
are RULED OUT with file:line evidence: codecPrivate is the FIRST sequence
header (read once at headers-ready, mkvstream.rs:115 + pipelined_stream.rs:289),
DVD VOBU structure guarantees each title opens on seq header + I-frame (no
mid-GOP open), the parser back-anchors leading still-frames to the disc's
real timeline (mpeg2.rs:296-303), and the muxer anchors base on the opening
keyframe's real PTS so the t=0 floor (mkv.rs:963) never corrupts it.
Regression tests pin all three (parser + muxer level).

SUB-TASK 3 — make --log-level 3 self-sufficient (diag.rs + minimal hooks).
(a) dump the ACTUAL MKV TrackEntry elements written per track
(tag=mkv.track: FlagInterlaced, FieldOrder, DefaultDuration, field duration,
Display dims, codecPrivate hex) so Windows-fps-class metadata is verifiable
from a log alone. (b) capture the first ~100 coded frames per track (raw)
to <output>.opening.bin with a per-frame summary line (tag=mkv.opening.frame:
track, key/delta, size, PTS) so opening-GOP/menu issues are diagnosable from
a future log without the disc. Both gated to log-level 3; normal runs open
no side file and record nothing.

CI gate (Rust 1.86): fmt --check, clippy -D warnings, and test --tests all
green.
2026-06-24 17:04:56 -07:00
Matthew Jackson cec217741b fix(dvd-audio-probe): read each sub-stream's true max channel count
The DVD AC-3 sub-stream probe recorded the FIRST decodable frame of each
physical 0x8x sub-stream as its channel count. A DVD feature opens with
logos/warnings whose audio is often a thin 2.0 bed on 0x80 before the
real 5.1 main mix begins a fraction of a second later. The probe locked
onto that opening 2.0 frame and reported 0x80=2, missing the 5.1
entirely (confirmed on Greenland: 0x80's head frames are acmod=2, then
acmod=7+lfe). With no 6-channel sub-stream found, channel-match routing
fell back to the ordinal map — harmless on Greenland, but on a disc where
the 5.1 lives on a non-ordinal sub-stream the wrong-substream bug stays
unfixed.

Fix: scan EVERY 0x0B77 frame of each sub-stream in the probe window and
keep the MAXIMUM channel count (the sub-stream's real main-mix
capability), advancing frame-by-frame via ac3_frame_size so a frame body
can't be mistaken for a new sync. Also bump PROBE_SECTORS 512->1024: the
1 MiB head window saw ONLY 0x80; 2 MiB reliably contains a frame of every
physical sub-stream.

Greenland tag=dvd.substream: before 0x80=2 (only); after 0x80=6, 0x81=2,
0x82=2 — matching the IFO and the decoded output.

Adds probe_reads_max_channels_no_cross_contamination regression test.
2026-06-24 16:55:34 -07:00
Matthew Jackson e4f27a0daf libfreemkv: fix rc.5.2 audit code findings
1. HEVC CRA->BLA false-trigger on 33-bit PTS wraparound
   (src/mux/codec/hevc.rs): the clip-boundary auto-detect compared the
   RAW 33-bit PES PTS against the high-water mark, so a single-clip title
   crossing 2^33->0 (~26.5h) false-armed pending_clip_boundary and rewrote
   a legitimate in-clip CRA(21)->BLA_W_LP(16), dropping valid RASL pictures
   (visible corruption) and breaking the single-clip byte-identical
   guarantee. Now unwrap the PTS onto a monotonic 64-bit timeline first
   (a near-full-period backstep is a wrap: add 2^33, update the watermark,
   do not arm). Regression test cra_after_33bit_pts_wrap_not_rewritten;
   the genuine-clip-join test still passes.

2. Single-pass recovery read bypassed the transport-failure abort
   (src/mux/disc.rs): the line-442 short-circuit only inspected the 10s
   read res. A transport failure (status 0xFF, wedged USB bridge) on the
   60s recovery read fell into the skip_errors branch and zero-filled/
   advanced, marching the disc at one bridge-recovery per probe
   (run-forever, hard rule #2). Re-check the recovery error for
   is_scsi_transport_failure() before the skip block and abort with
   Error::DiscRead. Test transport_failure_on_recovery_read_aborts_even_with_skip_errors.

3. Recovery-read SUCCESS branch had no coverage (src/mux/disc.rs tests):
   added RecoverableReader (errors when recovery=false, succeeds when
   recovery=true) and test recovery_read_success_muxes_recovered_data_no_skip
   driving fill_extents to the size-1 bottom-out and asserting the recovered
   data is muxed (counters advance, no skip).

4. TrueHD channel-correction probe omitted set_unit_base
   (src/disc/mod.rs correct_truehd_channels): the probe read via a
   DecryptingSectorSource without anchoring the AACS unit-alignment gate,
   so it degraded to absolute start_lba % 3 and returned DecryptFailed on a
   non-3-aligned extent, silently understating Atmos/7.1 as 5.1. Now call
   set_unit_base(ext.start_lba) before the probe read (no-op for CSS/None).

5. is_unit_aligned lba<unit_base latent trap (src/aacs/decrypt.rs):
   wrapping_sub mis-gated when lba < unit_base (2^32 == 1 mod 3). Switched
   to saturating_sub (clamps offset to 0, a unit boundary) and pinned the
   contract with is_unit_aligned_lba_below_base_is_well_defined plus
   is_unit_aligned_relative_to_base.

cargo +1.86 fmt --check / clippy -D warnings / test --tests all green.
2026-06-24 16:31:28 -07:00
Matthew Jackson 5b1009df71 dvd: route AC-3 audio to the physically-correct sub-stream by probed channel count
Fixes the "Silence of the Lambs" R2 PAL wrong-substream rip: the feature's
IFO declares one 5.1 AC-3 stream, but the scan assigned it the on-wire
sub-stream id 0x80 purely by per-codec ordinal (ifo::assign_audio_sub_stream_ids).
On this disc the physical 0x80 carries the 2.0 down-mix and the 5.1 main mix
lives at a different 0x8x sub-stream, so the rip muxed 2.0 while labelling it
"Dolby Digital 5.1" (the acmod fixup in mkv.rs then corrected only the Channels
element, surfacing the mismatch as the "IFO claimed 6 but acmod says 2" warning
— too late to re-route).

New src/disc/dvd_audio_probe.rs probes each physical AC-3 sub-stream's real
channel count from the head of the feature (the acmod/lfeon of its first frame
after the 0x0B77 sync) and re-routes each IFO-declared AC-3 stream onto the
physical sub-stream whose actual channel count matches the declared count,
instead of trusting the ordinal. Wired into both mux demux paths
(DiscStream::new and resolve::build_iso_pipeline) over the decrypting reader,
so it works on CSS discs and the autorip ISO-remux path alike. Bounded
512-sector best-effort read; an empty/unreadable probe degrades to the original
ordinal mapping (no regression on normal discs).

The cell selection is left unchanged: the feature's cell 0 (cat=0x02, 302.4s)
is chapter 1 of the movie (matches MakeMKV's chapter map and 1h53 duration
exactly), so it must NOT be dropped — the perceived "wrong video at the start"
was the wrong 2.0 audio over the opening, the same root cause.

Diagnostics (--log-level 3): new tag=dvd.substream rows dump the ACTUAL acmod
channel count of each physical 0x8x sub-stream read from the VOB, and the
per-cell tag=dvd.cell verdict now spells out the keep/skip reason. With the
existing tag=dvd.aattr (IFO declared sub_id + channels) a bug log alone now
shows whether the ordinal 0x80 really carries the declared layout — no disc
needed to diagnose this class.

expose ac3::find_ac3_sync as pub(crate) for the probe.
2026-06-24 16:28:21 -07:00
Matthew Jackson 1fb9463eaf mux/hevc: auto-detect non-seamless clip boundary, rewrite splice CRA→BLA
The "TopGun bug" (Top Gun 1986 UHD, DV Profile 7 dual-layer): the auditor
flags the rip `corrupt` with a flood of HEVC "Could not find ref with POC N"
decode errors (POC 114/210/228/234/240/246/252/318/336/354/372/399 — one
cluster per clip join).

Root cause: the title is multiple .m2ts clips joined at non-seamless
boundaries (mpls connection_condition 0x05/0x06), read as one concatenated
stream. Each next clip opens with a CRA whose RASL leading pictures reference
pre-join frames gone after concatenation. hevc.rs already has the spec remedy
(rewrite splice CRA_NUT 21 → BLA_W_LP 16 so a decoder sets NoRaslOutput and
drops the dangling RASL), behind mark_clip_boundary() — but nothing ever
called it: connection_condition is not plumbed through the threaded mux
pipeline, so the whole CRA→BLA mechanism was dead code.

Fix: detect the boundary inside the parser from the bitstream. Each clip
carries its own PES PTS base, so a non-seamless join is a large backward
PTS reset. HevcParser::parse now tracks a PTS high-water mark and, on a
backward step beyond 3 s (270000 ticks @ 90 kHz — mirroring the mux-side
DISCONTINUITY_BACKSTEP_NS), arms the existing CRA→BLA rewrite for the new
clip's first IRAP. Self-contained: works for every mux path, no cross-thread
plumbing. The 3 s threshold sits above any HEVC B-frame reorder dip and far
below any clip duration, so it never false-triggers in-clip; single-clip /
seamless titles never arm it and stay byte-identical.

Adds regression test cra_at_auto_detected_pts_backstep_rewritten_to_bla
(in-clip dip must not trigger; splice CRA after a backward reset must become
BLA; one-shot). hevc suite 55/55, mux suite 780/780.
2026-06-24 15:41:26 -07:00
Matthew Jackson 23414d289d libfreemkv: rc.5.2 DVD test coverage — depth-aware mux, colour codes, CSS scan
Implements the rc.5.2 quick-units list from the DVD coverage audit and
corrects the "passes-but-encodes-the-bug" tests that could not
distinguish correct from wrong behaviour.

New tests (each with the bug it guards):

mux/mkv.rs
- field_duration_is_direct_trackentry_child_not_in_video: depth-aware
  check that DefaultDecodedFieldDuration (and DefaultDuration) are direct
  TrackEntry children, NOT nested in the Video master. Replaces the flat
  find_id byte-scan that passed either way. Adds master_children /
  first_track_entry depth-walking helpers.
- pal_576i_emits_bt470bg_colour_codes / ntsc_480i_emits_smpte170m_colour_codes:
  assert the actual CICP tuples written into the MKV Colour master —
  PAL (5,5,5,1) vs NTSC (6,6,6,1) — not just stream-layer ColorSpace.
- ntsc_480i_field_order_is_tff_and_encoded: pins NTSC 480i hardcoded TFF
  and its ~33.37ms/16.68ms frame/field durations, asserting the encoded
  FlagInterlaced/FieldOrder bytes (480i was never exercised before).

ifo.rs
- video_attr_absolute_bytes_pin_real_layout: drives parse_video_attr with
  HARDCODED real DVD-Video bytes (PAL/NTSC x 4:3/16:9, plus mpeg_version
  in bits 7-6) instead of v_atr_byte, so a co-edit of the shift constants
  can't re-seed the PAL-as-NTSC bug. Anchors that permitted_df bits (1-0)
  are not read as the TV system.

disc/dvd.rs
- scan_dvd_titles_lpcm_routes_to_a0_pid_range: LPCM (coding 4) → sub-id
  0xA0 → PID 0xBDA0, disjoint from the AC-3 0xBD8x space, channels kept.
- scan_dvd_titles_multiple_vobsub_tracks_distinct_pids: three VobSub
  tracks → distinct 0x20+ordinal PIDs, per-language, shared palette.

css/mod.rs
- crack_outcome_reaches_cracked_with_span: drives the full crack scan to
  CrackOutcome::Cracked via a Stevenson-crackable synthetic sector and
  asserts crack_span recording (the Cracked branch was never exercised).
- recrack_succeeds_on_other_vts_extents: per-VTS re-crack SUCCESS path.
- all_locked_synthetic_iso_yields_css_key_missing_signal: all-locked
  multi-extent ISO → ScrambledUncracked, the signal the scan converts to
  css_error = Some(CssKeyMissing).

Corrected fixtures (passes-but-encodes-the-bug):
- scan_dvd_titles_mixed_audio_codecs_distinct_pids: real channel nibbles
  (AC-3 5.1 = 6ch, DTS 2.0 = 2ch) replacing the 1ch placeholders; asserts
  channel counts and exact canonical PIDs (0xBD80 / 0xBD88).
- ebml.rs FieldOrder comment: drop the stale "PAL DVD (576i) is
  bottom-field-first" line that contradicted the TFF-for-all code.
2026-06-24 15:41:26 -07:00
Matthew Jackson 828aed1ecb libfreemkv: clip-anchored AACS unit gate + consolidate key mechanism
The AACS unit-alignment gate measured `lba % 3` against absolute disc LBA 0,
but aligned units are anchored at each clip's encrypted-region start. A clip
whose start_lba is not 3-aligned had its readable units wrongly rejected with
"Decryption failed" (the big-title-only failure on some Blu-rays). One
canonical clip-anchored helper (`aacs::is_unit_aligned`) is now the single
source of truth for the decrypt-on-read gate; both mux read paths set the
per-extent `unit_base = start_lba` via a new `SectorSource::set_unit_base`.

Also moves key *mechanism* into the library: the encrypted sample reader
(`read_encrypted_units`) and the candidate-key resolution loop
(`resolve_and_apply`) now live here, so a key source is purely a lookup.
Regression test covers a clip based at a non-3-aligned LBA.
2026-06-24 15:40:50 -07:00
Matthew Jackson 10709fafbf libfreemkv: single-pass disc->MKV recovers marginal sectors before giving up
Single-pass disc->MKV has no Pass N, so its read bottom-out now issues one
bounded recovery read (recovery=true, ~60s ECC) before skipping or aborting,
matching the multipass patch. Fixes a transient/marginal sector surfaced as a
read failure direct-to-MKV while multipass recovered it. One read, not a loop
(hard rule #2); recovered data is used so no bogus-status hole reopens.
2026-06-24 14:58:32 -07:00
Matthew Jackson c4fda5c873 v1.0.0-rc.5.1: bump version 2026-06-24 14:36:29 -07:00
Matthew Jackson ab18f384c7 libfreemkv: rc.5.1 DVD correctness fixes
- CSS: unlock scrambled-sector reads on enforcing drives via bus-auth
  only; classify sense 6F/03 as CSS-locked; early-bail on a fully locked
  scan; gate the AACS handshake off DVD discs.
- DVD first-play menu no longer prepended to the feature: read the title
  VOBS base from vtstt_vobs (0xC4), not the menu VOBS vtsm_vobs (0xC0).
- Interlaced field-duration (DefaultDecodedFieldDuration) written as a
  direct TrackEntry child rather than inside Video, so Windows reports
  the correct frame rate.
- Audio channel count read from the AC-3 bitstream; FieldOrder set to
  TFF; per-track BPS tags.
- Structured disc diagnostics at --log-level 3; reduced per-operation
  log spam.
2026-06-24 14:34:55 -07:00
Matthew Jackson 18d1f16582 Add Contributor Covenant v2.1 Code of Conduct 2026-06-24 10:44:58 -07:00
Matthew Jackson 1aea0d81a5 v1.0.0-rc.5: bump version 2026-06-24 10:27:30 -07:00
Matthew Jackson c197b5236f Preserve interlaced scan type in label and MKV output
Interlacing is detected upstream (PAL DVD -> R576i) but was dropped in
two places: the video label hardcoded a 'p' suffix, and the muxer never
wrote any scan-type flag, so MediaInfo inferred progressive and reported
576p for a 576i source.

- Add Resolution::is_interlaced() for the R*i variants.
- generate_video_label now branches i/p for the heights that can be
  interlaced (1080, 576, 480) instead of always emitting 'p'.
- MkvTrack carries interlaced + field_order; the video serializer emits
  FlagInterlaced (0x9A; 1=interlaced, 2=progressive) and, for interlaced
  content, FieldOrder (0x9D) - bottom-field-first for PAL 576i,
  top-field-first otherwise. Adds the EBML constants.
2026-06-24 09:47:16 -07:00
Matthew Jackson d868d4e720 Fix AC-3 channel count read from wrong nibble in VTS audio attr
The VTS_AST_ATR byte 1 stores (channels - 1) in the low 3 bits, but we
were reading the high nibble. A normal 2.0 AC-3 track has 0x1 in the low
nibble and 0x0 in the high nibble, so we computed 0+1=1 and mislabeled
every stereo track as mono in the muxed Channels element.

Read the low 3 bits instead, and correct the two audio_attr tests that
had encoded (channels - 1) in the high nibble to match the buggy reader.
2026-06-24 09:39:02 -07:00
Matthew Jackson 241a592323 fix: correct stale FAIL_PAUSE_SECS doc claiming uniform sweep+patch application 2026-06-24 05:57:39 -07:00
Matthew Jackson a4a12859db fix: drop BytePrefetcher channel endpoints before join to prevent deadlock
Drop rx and recycle_tx (now Option fields) before joining the producer
thread in BytePrefetcher::Drop. Without this, a non-EOF source fills the
depth-2 forward channel then spins in send_timeout(POLL_INTERVAL) forever
because rx is never drained; join() deadlocks. Adds a regression test
(drop_endless_prefetcher_joins_cleanly) that directly exercises the path.
2026-06-24 05:50:58 -07:00
Matthew Jackson 7d3f413030 fix: correct stale doc on PatchItem::Unreadable — promotion shipped via Mapfile::record, not PatchSink 2026-06-24 05:02:24 -07:00
Matthew Jackson b993320249 fix: correct stale css::crack comment reference to css::crack_key 2026-06-24 04:57:47 -07:00
Matthew Jackson cb96550ba5 fix: remove stale sweep_pipeline.rs cross-references in patch.rs comments 2026-06-24 04:52:04 -07:00
Matthew Jackson 9d2007274c Revert "fix: discard duplicate TS packets in non-PUSI continuation path"
This reverts commit ec35b2bce1.
2026-06-24 01:59:49 -07:00
Matthew Jackson e58ce237e7 fix: enforce WEDGE_FAMILY_COOLDOWN_SECS == ZONE_ENTRY_COOLDOWN_SECS at compile time 2026-06-24 01:52:43 -07:00
Matthew Jackson fc82d98bf1 fix: correct Profile-07 comment — prime_cache(lba) reads lba-3..lba-1, not lba itself 2026-06-24 01:51:21 -07:00
Matthew Jackson d40733d0b1 fix(vc1): assemble keyframe prefix in fixed seq-then-entry order
When a keyframe AU carried an unchanged seq_header (stripped) but a
redefined entry_point (appended), the old append-then-reassert path
produced [entry_point, seq_header] — entry_point before seq_header,
violating SMPTE 421M which requires seq+entry before every RAP.

Replace the single shared prefix Vec + reassert_active() with per-type
temporaries (redefined_seq / redefined_ep) collected during the scan,
then assembled in canonical seq-then-entry order at keyframe time.
Non-keyframes still emit only genuine redefinitions, also seq-before-ep.
Removes the now-unused reassert_active() helper. Adds a regression test
covering the seq-unchanged / entry-redefined trigger case.
2026-06-24 01:47:51 -07:00
Matthew Jackson ec35b2bce1 fix: discard duplicate TS packets in non-PUSI continuation path 2026-06-24 01:45:39 -07:00
Matthew Jackson 4b6d3dda64 fix: correct comment — patch leaves middle NonTrimmed, not NonTrimmed/Unreadable 2026-06-24 01:11:39 -07:00
Matthew Jackson de3e62e6cc fix: correct doc comment in patch pipeline test to say NonTrimmed not Unreadable 2026-06-24 01:10:51 -07:00
Matthew Jackson 3cbbe148d1 fix: remove stale TDD-red comment from BytesRead emission test 2026-06-24 00:50:21 -07:00
Matthew Jackson d2cf068a60 fix: correct hevc.rs doc comments — non-seamless BD join is connection_condition 0x05/0x06 not 0x01
Comments at lines ~63 and ~102 misidentified 0x01 (first-item/seamless) as the
non-seamless trigger and labelled 0x05/0x06 as seamless — inverted vs the BD-ROM
spec and mpls.rs (which documents 1=seamless, 5/6=non-seamless). Corrected all
affected doc blocks; no logic change.
2026-06-24 00:49:22 -07:00
Matthew Jackson dc18dd9e8e fix: use trim_mkb in resolve_vid_only to avoid zeroing unrecognised MKB 2026-06-24 00:48:02 -07:00
Matthew Jackson 9e552916de fix: correct stale comment in patch.rs work-list ranges_with call
The comment at line 404 claimed "every non-Finished range" but the
immediately-following ranges_with call lists only NonTrimmed,
NonScraped, and Unreadable — deliberately omitting NonTried.
Update the comment to accurately reflect the actual status list and
explain that NonTried is excluded because it is handled by a preceding
sweep pass, not by patch.
2026-06-24 00:09:13 -07:00
Matthew Jackson cfc41086ba fix: correct module doc — only Pass 1 routes through handle_read_error, not Pass N 2026-06-23 23:34:02 -07:00
Matthew Jackson c2f6d04198 fix: update stale doc comment in ReadCtx::for_patch — sync is automatic 2026-06-23 22:51:47 -07:00
Matthew Jackson a1aca559cf fix: correct skip_sectors_for_probe doc comment (8x per index, not 2x per 3) 2026-06-23 22:50:40 -07:00
Matthew Jackson e49f21a45a fix: correct module doc watchdog constant name in patch.rs 2026-06-23 22:49:42 -07:00
Matthew Jackson 06f6d8932f Fix rc5 audit findings: keydb doc, pipeline ordering, hot-loop Arc, tests
- keydb.rs: separate default_path()/no_home_dir() doc blocks; correct the
  false XDG lock-step claim (Linux write path uses $HOME, ignores
  XDG_CONFIG_HOME; read-side search also checks XDG_CONFIG_HOME).
- io/pipeline.rs: use Release/Acquire on the abandoned flag so a leaked
  consumer reliably skips close() on weak memory models (ARM64/POWER),
  not just x86 TSO.
- mux/disc.rs: cache the decrypt-loss Arc at construction; lost_bytes()
  no longer clones an Arc per frame on the mux hot path.
- disc/dvd.rs: assert display_aspect mapping for both 16:9 (PAL test) and
  4:3 (NTSC test).
- mux/resolve.rs: extract css_error_aborts() helper and unit-test the
  scrambled-but-uncracked CSS guard (Fix 6) incl. the --raw exemption.
- aacs/keys.rs: add unit tests for mkb_type_raw/mkb_type/mkb_is_uhd and
  MkbType (Category C 2.0 UHD, prerecorded 1.0, no-0x10-record None).
- release.yml: publish job needs [verify, test] so a failing test suite
  blocks crates.io publication.
2026-06-23 19:11:09 -07:00
Matthew Jackson 15802da1db Bump to 1.0.0-rc.4.3 2026-06-23 15:49:25 -07:00
Matthew Jackson ea5ee9dc6b DVD: correct PAL/NTSC, anamorphic aspect, and SD colour
Fix three DVD video-attribute bugs surfaced by a PAL disc detected as
NTSC:

- PAL/NTSC: parse video_format from VTS_V_ATR bits 5-4, not bits 1-0
  (the old mask read permitted_df, so PAL 576i/25fps was mis-detected
  as NTSC 480i/29.97). Named consts replace the magic bit positions.
- Anamorphic aspect: write MKV DisplayWidth/Height from the disc's
  display_aspect (16:9 720x576 -> 1024x576) instead of square pixels,
  so 16:9 DVDs no longer render as 4:3.
- Colour: stamp SD colorimetry (PAL=BT.470BG, NTSC=SMPTE-170M) instead
  of BT.709 (HD).

Adds VideoStream.display_aspect (threaded through every muxer) plus
TvSystem/DvdAspect/ColorSpace plumbing, with regression tests. Removes
the deprecated Disc mux set_halt bridge (use with_halt).
2026-06-23 15:38:49 -07:00
Matthew Jackson b7a8937786 docs: changelog entry for rc.4.2 Windows fixes 2026-06-23 12:49:08 -07:00
Matthew Jackson 36012e026c v1.0.0-rc.4.2: bump version 2026-06-23 12:41:07 -07:00
Matthew Jackson 872d78d89d io: add platform-aware fsync helpers (dir + durable file sync)
Add an io::fsync module with a per-OS split (posix/windows) mirroring the
writeback_file convention, replacing two duplicated dir-fsync copies:

- dir(): POSIX directory fsync; a no-op on Windows, where std cannot open
  a directory as a File and the failed open logged a spurious warning on
  every mapfile write.
- file_durable(): opens the target read+write before sync_all so the flush
  succeeds on Windows, where FlushFileBuffers rejects a read-only handle
  with ERROR_ACCESS_DENIED.

Point the mapfile writer at the shared dir() helper.
2026-06-23 12:38:02 -07:00
Matthew Jackson cfd71eed3b v1.0.0-rc.4.1: bump version 2026-06-23 10:45:42 -07:00
Matthew Jackson b531763cae test(scsi/windows): cross-check all FFI structs + constants vs SDK headers
Audited every #[repr(C)] struct and IOCTL/flag constant in scsi/windows.rs
against the authoritative Windows SDK headers (ntddscsi.h, winioctl.h,
devioctl.h, winnt.h, fileapi.h). All correct except the already-reverted
ScsiPassThroughDirect packing. Add the missing regression guards:
- StoragePropertyQuery layout (STORAGE_PROPERTY_QUERY: 0/4/8, size 12).
- IOCTL/flag constants, with IOCTLs asserted against an independent CTL_CODE
  re-derivation (not a tautological literal) so a mistyped code is caught.
Validated compiling via cargo xwin check --target x86_64-pc-windows-msvc.
2026-06-23 10:44:21 -07:00
Matthew Jackson 2d9a82743a fix(scsi/windows): revert wrong packed(4) on ScsiPassThroughDirect (rc.4 drive-detection regression)
rc.4 added #[repr(C, packed(4))] to ScsiPassThroughDirect on the false premise
that ntddscsi.h wraps SCSI_PASS_THROUGH_DIRECT in #pragma pack(push, 4). It does
NOT — verified against the Windows SDK ntddscsi.h: the struct has no pragma pack
and uses natural alignment. On 64-bit Windows (LLP64) that puts DataBuffer at
offset 24 and the struct at 56 bytes, which bare #[repr(C)] produces and which
DeviceIoControl expects.

packed(4) instead imposed offset 20 / 48 bytes — the layout of the SDK's
SEPARATE 32-bit thunk struct SCSI_PASS_THROUGH_DIRECT32 (VOID* POINTER_32). Using
that on a 64-bit host malformed every IOCTL_SCSI_PASS_THROUGH_DIRECT, so the
INQUIRY in drive enumeration failed and autorip/CLI reported zero drives
('RC4 no longer detects my drive'). rc.3.1 (bare repr(C)) worked for the same
users; this restores that layout.

Replace the tautological packed-layout test (which asserted the same wrong
offsets the struct produced) with one cross-checked against the SDK header:
DataBuffer@24, SenseInfoOffset@32, Cdb@36, size 56. Verified compiling via
cargo xwin check for x86_64-pc-windows-msvc.
2026-06-23 10:40:32 -07:00
Matthew Jackson 23f5ae3665 ci: cache Rust builds with Swatinem/rust-cache to speed up release + CI 2026-06-23 10:01:37 -07:00
Matthew Jackson ad06f3062e v1.0.0-rc.4: bump version 2026-06-23 09:31:56 -07:00
Matthew Jackson 2fcb7d3b77 keysource: add KeySource::label() for source identification 2026-06-23 09:09:06 -07:00
Matthew Jackson 9e2f5f0109 aacs: add MkbType (BD vs UHD generation) accessor API
Expose the MKB Type field (record 0x10) as a typed MkbType enum with mkb_type()
/ mkb_type_raw() / mkb_is_uhd() helpers, so callers can distinguish AACS 1.0
(Blu-ray) from AACS 2.0/2.1 (UHD) discs without poking raw bytes.
2026-06-23 08:19:40 -07:00
Matthew Jackson b5ac505c9d CHANGELOG: add 1.0.0-rc.4 — decrypt-loss accounting, durable writes, Windows SCSI, truthful error causes 2026-06-23 07:22:05 -07:00
Matthew Jackson a7280eef85 Account for decrypt-time loss so partial AACS/CSS failures can't pass as a perfect rip
When a scrambled AACS unit fails to decrypt under every available key
(a missing/wrong CPS sub-key, or a marginal unit that fails the TS-sync
verify), decrypt_sectors restored the original encrypted bytes and
returned Ok with no signal. Those still-encrypted bytes flowed to the TS
assembler, which silently dropped the non-syncing packets with no loss
counter. The only loss accounting was DiscStream's read-error zero-fill
path, so mux reported lost_video_secs=0 for decrypt-dropped content and
the abort gate accepted the rip even under abort_on_lost_secs=0. A rip
missing real video/audio segments was published as a perfect success.

decrypt_sectors now returns the number of bytes in scrambled units that
no key could decrypt. DecryptingSectorSource accumulates that into a
shared counter exposed via decrypt_loss(); both mux pipelines fold it
into lost_bytes() — the inline DiscStream path directly, and the
file-backed highway via PipelinedPesStream sharing the producer's
counter. Restore-to-original is unchanged, so clear nav-files are never
corrupted; metadata-probe callers that don't read the counter are
unaffected. Adds regression tests at the decrypt and decorator layers.
2026-06-23 06:15:24 -07:00
Matthew Jackson ab4c66e9d4 keydb: correct read_capped_to_string doc for non-UTF-8 case
The doc claimed Error::KeydbInvalid for non-UTF-8 input, but the code
returns Error::KeydbParse (KeydbInvalid is reserved for the size-cap
violation). Correct the doc to match behavior and add a regression test
asserting non-UTF-8 yields KeydbParse.
2026-06-23 05:27:01 -07:00
Matthew Jackson 2fbdd28023 keydb: classify server-dropped connection as KeydbConnect, not KeydbParse
In http_get, when the server closes the TCP connection before the HTTP
header block completes (n == 0 on the byte-by-byte header read), or sends
a header block exceeding 64 KiB, the code returned KeydbParse (E8004).
Both are connection/protocol-level faults from the server, not parse
failures of keydb content — the keydb bytes were never received. Return
KeydbConnect (E8000) instead, which already covers TCP-level exchange
failures. A CLI user hitting a transient drop or a redirect target that
immediately closes now sees the correct 'server hung up' diagnostic
rather than 'the downloaded file was malformed'.

Add a regression test that stands up a loopback listener which accepts
then drops the connection before headers, asserting KeydbConnect.
2026-06-23 05:21:46 -07:00
Matthew Jackson aef72b920f fix(scsi/windows): pack ScsiPassThroughDirect to match ntddscsi.h layout
ntddscsi.h wraps SCSI_PASS_THROUGH_DIRECT in #pragma pack(push, 4),
forcing the PVOID DataBuffer field to 4-byte alignment even on 64-bit
hosts. The Rust struct used bare #[repr(C)], so the compiler applied
natural 8-byte pointer alignment and inserted 4 padding bytes after
TimeOutValue. That shifted DataBuffer to offset 24 (SDK: 20),
SenseInfoOffset to 32 (28), and Cdb to 36 (32), and grew the struct to
56 bytes (48). DeviceIoControl reads at the SDK offsets, so every SPTI
ioctl on 64-bit Windows either got rejected or interpreted garbage as
the CDB and DataBuffer pointer.

Add #[repr(C, packed(4))] to ScsiPassThroughDirect and the companion
SptwbDirect (so offset_of!(SptwbDirect, sense) stays correct for
SenseInfoOffset), plus a layout regression test asserting DataBuffer at
offset 20 and a 48-byte struct size.
2026-06-23 05:16:22 -07:00
Matthew Jackson 1a64ba2790 prefetched: test that event_fn fires BytesRead per batch
The prefetch producer thread fires a BytesRead event after every
batch it reads, carrying a cumulative byte count. Nothing asserted
this callback actually fired, so a consumer that passed None for the
event_fn would silently get no progress events. Add a fixture-based
regression test that drives a finite extent through new_with_events,
captures the events, and asserts the cumulative count is
non-decreasing and reaches the full extent size at EOF.

This locks the contract autorip's mux progress bar and soft-stall
watchdog depend on.
2026-06-23 04:54:40 -07:00
Matthew Jackson 81d3adbbb3 aacs: preserve transport-failure errors through the auth handshake
A SEND KEY / REPORT KEY step in the bus-auth handshake mapped every
SCSI error to a cert/key-specific code (AacsCertRejected, AacsCertRead,
AacsKeyRead, AacsKeyRejected, etc.) via map_err(|_| ...). That discarded
the underlying SCSI error, so a transport-layer wedge (bridge crash / USB
disconnect) mid-handshake was reported as 'drive rejected your host cert',
sending operators down a keydb/host-cert dead end for what is really a
replug/power-cycle situation.

Add a handshake_err() helper that keeps the original error when it is a
transport failure (is_scsi_transport_failure) and only substitutes the
handshake-specific code for genuine SCSI rejections. Apply it at every
SEND KEY / REPORT KEY / REPORT DISC STRUCTURE step in both the AACS 1.0
and AACS 2.0 paths. Add a regression test covering both branches.
2026-06-23 04:20:54 -07:00
Matthew Jackson 3ce5929fa5 disc: warn when READ CAPACITY fails instead of silently using 0 sectors
read_udf treated a READ CAPACITY SCSI failure as a 0-sector disc via
unwrap_or(0) with no diagnostic. capacity=0 then skews the layer
heuristic (always reports 1 layer, even for dual-layer discs) and the
canonical title-ordering sort, with nothing in /api/state or info to
indicate the command actually failed. Emit a tracing::warn carrying the
original error at the fallback site so a transient capacity failure is
visible. Recovery behavior is unchanged: 0 is still used as the
fallback.
2026-06-23 04:11:46 -07:00
Matthew Jackson 77fb07f954 keydb: map missing home dir to NotFound, not keydb-parse error
default_path returned Error::KeydbParse (E8004, rendered as 'failed to
parse the keydb file') when HOME/USERPROFILE was unset. That misreports
an environment failure — a process with no home directory, typically a
stripped container or CI config — as a corrupt keydb file the code never
read. Map it to an IoError(NotFound) in the I/O category instead, so no
display path blames the keydb. Add a regression test.
2026-06-23 04:05:41 -07:00
Matthew Jackson 54b79b0395 Preserve I/O error from read_aacs_inputs ISO open
Disc::read_aacs_inputs opened the ISO via FileSectorSource::open and
mapped any failure to Error::AacsNoKeys (E7000), discarding the real
Error::IoError (E5000) and its OS errno. A missing or unreadable ISO
(ENOENT/EPERM) is an I/O fault, not a key-resolution failure; callers
that dispatch on the error code would wrongly tell the user to check
their keys when the ISO simply does not exist.

Propagate the open error unchanged and add a regression test asserting
a nonexistent ISO yields E_IO_ERROR, not E_AACS_NO_KEYS.
2026-06-23 03:59:55 -07:00
Matthew Jackson 7710fccddd Fix oversized read batch on non-sysfs (Windows) optical drives
detect_max_batch_sectors() is a Linux-sysfs probe with no platform
gate. It derived the device name with rsplit('/'), which never splits a
Windows \.\CdRom0 / \.\D: path, so the whole path became the device
name, no /sys node matched, is_optical fell to false, and the function
returned the 8192-sector block default (16 MiB/request) instead of the
60-sector optical default. That value then took the Some(b) arm in
Disc::copy and bypassed the 510-sector optical clamp that lives only in
the sysfs branch, leaving every Windows rip/verify running ~16x over the
optical cap (coarser bad-sector recovery, 16 MiB UDF reads).

Gate the sysfs probe behind a new sysfs_batch_probe_supported() helper
(Linux-only, requires a '/'-delimited path) and return the optical
default for any path the probe can't handle. Add regression tests for
the \.\ device-path forms.
2026-06-23 03:51:51 -07:00
Matthew Jackson d2ff433fac scsi(windows): fix StorageAdapterDescriptor.BusType width (u8 -> u32)
STORAGE_ADAPTER_DESCRIPTOR.BusType is STORAGE_BUS_TYPE, an int-sized
(4-byte) enum, but the repr(C) struct declared it as u8. The total
size stayed 32 bytes by coincidence of alignment padding, and the two
fields actually read today (MaximumTransferLength at offset 8 and
AlignmentMask at offset 16) sit ahead of BusType, so there was no
runtime impact. But BusMajorVersion and BusMinorVersion landed at
offsets 26 and 28 instead of the SDK's 28 and 30, so any future reader
of those fields would have gotten wrong values.

Widen BusType to u32 so BusMajorVersion/BusMinorVersion fall at the
correct offsets, and add a layout regression test asserting every
field offset and the 32-byte total against the winioctl.h layout.
2026-06-23 03:45:34 -07:00
Matthew Jackson 30b2b86cad aacs: fix stale mkb_version offset doc comment
The doc comment claimed the version was a BE u32 at offset 8 of the
record body (offset 12 from pos), but the code correctly reads pos+8
(body offset 4): a 4-byte record header at pos, the Type field at body
offset 0, then the version at body offset 4. Rewrite the comment to
match the actual read so a maintainer does not 'correct' the offset and
break MKB version parsing. Clarify the matching test comment too.
2026-06-23 02:28:34 -07:00
Matthew Jackson a192e696ac disc: reject partial CPS-unit key coverage in AACS validation gate
aligned_unit_keys_validate accepted a unit-key set as soon as ONE
scrambled sample decrypted. On a multi-CPS-unit disc a set covering
CPS unit 0 but not CPS unit 1 therefore passed: decrypt_with committed
it, the sweep proceeded, and CPS-unit-1 sectors passed through as raw
encrypted bytes into the ISO/MKV with no error surfaced anywhere.

Require every scrambled sample to be descrambled by some unit key.
A sample no key covers now fails the gate, so an incomplete set is
rejected (AacsKeyRejected) and the caller falls through to the next
candidate, ultimately surfacing a key error instead of silently
writing ciphertext. Wholly-wrong-key rejection is unchanged.

Add a regression test for the partial-coverage case.
2026-06-23 01:43:36 -07:00
Matthew Jackson dc2dcdba55 disc: fix misleading bridge-degradation comment, add 04/3E regression test
The comment on the bridge-degradation branch in handle_read_error
claimed it matched the NOT_READY 04/3E sense signature, but
is_bridge_degradation() keys solely on a non-standard SCSI status byte
(anything that is not GOOD/CHECK CONDITION/TRANSPORT FAILURE) and
ignores sense_key/ASC/ASCQ. A real 04/3E bad-sector error arrives as
CHECK CONDITION (0x02), so it never took this branch — it falls through
to the generic NOT_READY retry. Rewrite the comment to describe the
status-byte condition the predicate actually detects, and drop the
parallel misleading note in the not_ready_err test helper.

Add a regression test asserting a NOT_READY 04/3E error is not
classified as bridge degradation and routes to the NOT_READY retry
(3 s pause) rather than the bridge cooldown (15 s pause).
2026-06-23 01:20:04 -07:00
Matthew Jackson fc6d9d42fd scsi(windows): only sleep on successful device reset
SptiTransport::reset() unconditionally slept 2 seconds after sending
IOCTL_STORAGE_RESET_DEVICE, even when the IOCTL failed (e.g.
ERROR_INVALID_FUNCTION on a driver that does not support the reset).
On failure no reset occurred, so there is nothing to settle and the
2-second penalty was pure waste. Gate the settle sleep on the IOCTL
return so it only fires when the drive was actually reset.
2026-06-23 00:59:26 -07:00
Matthew Jackson 03f6e076ab keydb: write keydb.cfg atomically (temp + fsync + rename)
keydb::save() overwrote the live keydb.cfg with a bare in-place
std::fs::write. keydb.cfg is the single source of AACS truth and this
path runs unattended (first-boot download + daily-refresh thread, with
a container restart on every release), so a SIGKILL, OOM-kill, power
loss, or ENOSPC mid-write could leave the file truncated with the prior
good copy already gone. A truncated keydb does not error at write time;
it surfaces later as failed key resolution on every AACS rip.

Factor the write into write_atomic(): create the parent dir, write a
unique sibling temp file, fsync, then rename (atomic within a
filesystem). On any write/fsync/rename failure the temp is removed and
the existing keydb is left untouched. Same pattern already used by the
settings and mover write paths. Add regression tests covering in-place
replacement (no stray temp) and prior-copy preservation on failure.
2026-06-23 00:24:58 -07:00
Matthew Jackson e5e8ee43ec mux: track skipped bytes for accurate loss estimation
DiscStream skips a whole AACS unit (3 sectors = 6144 bytes) per
read-error event, but only the skip-event count was exposed. Loss
estimates built from errors*2048 therefore undercounted AACS loss ~3x.

Add a lost_bytes field that accumulates the actual zero-filled byte
count at each skip, expose it via a new Stream::lost_bytes() accessor
(default 0; DiscStream and CountingStream override), so consumers can
scale lost-video time by real bytes lost rather than the event count.

Regression tests assert the AACS path records 6144 B/event (and
exceeds the errors*2048 undercount) while the align=1 path records
2048 B/event.
2026-06-23 00:12:56 -07:00
Matthew Jackson dd9be0d21d pipeline: leaked consumer must not finalise an abandoned output
When finish_with_halt's grace period expires it detaches from the
consumer thread and returns an error to the caller, but the leaked
consumer kept running to completion: once its wedged write syscall
returned it would fall through to sink.close(). For the mux writer
close() finalises the MKV (Cues block + segment-header patch), so a
leaked consumer could finalise — and keep writing to — an output file
the caller had already reported as failed, racing a fresh rip for the
same device over the same path.

Add a shared abandonment flag the consumer polls in its drain loop and
again before close(). finish_with_grace sets it before dropping the
JoinHandle, so the moment the wedged syscall returns the consumer skips
any further apply and skips close() entirely, then exits. This does not
interrupt the in-flight syscall (only its return or process exit can),
but it bounds the damage to the write already in flight instead of a
full finalise of an abandoned file.

Regression tests cover both sides: a consumer leaked past the grace
period skips close(), while one that finishes inside the grace window
still calls close() and finalises normally.
2026-06-22 23:22:57 -07:00
Matthew Jackson 835429c35d scsi/windows: surface IOCTL_STORAGE_RESET_DEVICE failures
SptiTransport::reset() discarded the DeviceIoControl return value, so a
wrong or unsupported reset IOCTL would fail with ERROR_INVALID_FUNCTION
and silently no-op while the unconditional 2s settle sleep made it look
like a reset happened. That is exactly the regression class the doc block
records for the two earlier (incorrect) code values.

Bind the result and warn (with GetLastError) when the reset fails, debug
on success. Lift IOCTL_STORAGE_RESET_DEVICE to module scope and add a
test recomputing it from the CTL_CODE formula so a wrong value can't slip
back in unnoticed.
2026-06-22 23:08:52 -07:00
Matthew Jackson 804c4cb194 mapfile: fsync parent directory after rename for durable resume checkpoint
Mapfile::flush() wrote the new state to a .tmp sibling, sync_all()'d the
temp file, then rename(2)'d it over the final mapfile path — but never
fsynced the parent directory. After the rename the new dirent lives only
in the directory's page cache, so a crash or power loss in the
rename-commit window (the wide window on NFS, the very case the temp
fsync guards) can lose it: resume then reads a stale or absent mapfile
even though the data bytes were durable, silently discarding multi-pass
recovery progress.

Add a best-effort fsync_dir() on the path's parent after the rename,
mirroring the established dirent-durability pattern in autorip's mover.
A directory that can't be opened or synced is logged and ignored rather
than failing the write, since the file bytes are already durable.

Adds a regression test exercising the parent-fsync branch against a real
subdirectory and asserting the helper is a no-op on a missing directory.
2026-06-22 23:00:04 -07:00
Matthew Jackson fa3d6dabdd release CI: parallelize test/publish/release off the critical path
Tests, crates.io publish, and the GitHub Release no longer serialize
behind each other: each job only needs the version-check (verify) gate.
Binary consumers git-tag-pin libfreemkv, so they start building the
instant the tag exists rather than waiting on the crates.io publish.
crates.io publish runs as an independent job for external consumers and
uses --no-verify (CI already compiled this commit in the test job).
2026-06-22 21:59:31 -07:00
Matthew Jackson bd3664d3c5 Demote per-read Drive::read trace event to TRACE
Drive::read fires hundreds of thousands of times per rip. Logging its
entry at DEBUG floods a diagnostic log and buries the events that
actually matter. Move it to TRACE so a level-3 (debug) bug-report log
stays readable; level-4 (trace) still captures it for deep dives.
2026-06-22 21:39:27 -07:00
Matthew Jackson a5861157de Merge branch 'feat/windows-hardening' 2026-06-22 18:08:41 -07:00
Matthew Jackson 831ca31911 v1.0.0-rc.3.1: silent-failure guards (mux empty/zero-frame, CSS crack-vs-unencrypted), Windows keydb path, AlignmentMask, English errors 2026-06-22 18:07:48 -07:00
Matthew Jackson d127783522 docs: describe the generic Unlocker seam, drop in-tree firmware specifics
The docs still documented the old in-tree firmware unlocker: the MediaTek
MT1959 variant table, the READ BUFFER unlock CDB bytes, the profiles.json
schema (unlock_mode/unlock_buf_id/unlock_cdb), the platform/mt1959 driver
listings, and the 'why unlock is needed' handshake mechanism. None of that
lives in libfreemkv anymore — the core is firmware-clean and ships only the
pluggable Unlocker trait + registry (src/unlock.rs).

Rewrite drive-access, architecture, api-design, disc-to-rip, and the README
to describe only the generic Unlocker seam: the trait, register_unlocker, the
registry routing, and the host-cert fallback when no unlocker matches. Point
readers to the freemkv-unlock repo for concrete unlockers. No source change.
2026-06-22 17:10:51 -07:00
Matthew Jackson 80bf9eba60 v1.0.0-rc.3: bump version 2026-06-22 16:13:07 -07:00
Matthew Jackson be8d555551 scsi/windows: correct IOCTL_STORAGE_RESET_DEVICE to 0x002D5004
The dual-model Windows audit (Sonnet) caught that the prior 'fix' (0x002DD000)
was also wrong: that decodes to the OBSOLETE RESET_BUS code (function 0x400,
R|W access) which class drivers reject. Canonical ntddstor.h:
  IOCTL_STORAGE_RESET_DEVICE = CTL_CODE(0x2D, 0x0401, METHOD_BUFFERED, FILE_READ_ACCESS)
  = (0x2D<<16)|(1<<14)|(0x401<<2)|0 = 0x002D5004.

Only affects the best-effort drive-reset recovery path (return value is
discarded); the normal read/unlock/rip path never calls it, so this does not
change normal Windows operation. Verified correct-by-construction; the
AlignmentMask finding is deferred to rc4 (benign on USB optical bridges,
needs a real Windows SCSI-HBA rig to validate).
2026-06-22 15:53:45 -07:00
Matthew Jackson 2cab62dc22 Fix 3 Opus-audit findings: patch transport-abort, patch AACS align, reset IOCTL
- patch (Pass N) now aborts immediately on transport failure (status=0xFF),
  symmetric with the sweep and single-pass mux. Previously a USB-bridge crash
  was treated as an ordinary bad sector and the pass hammered the crashed
  device sector-by-sector until the per-range watchdog expired. (medium)
- patch AACS recovery reads are now unit-aligned: a mid-unit single-sector
  read on an AACS disc was rejected by the decrypting reader (DecryptFailed)
  and the sector abandoned without asking the drive. The read is now widened
  to the enclosing whole 3-sector unit and the requested window copied out,
  leaving all recovery accounting (pos/block_bytes/cursor) untouched so it
  cannot desync. Only affected CLI decrypt-to-ISO --multipass re-runs. (low)
- IOCTL_STORAGE_RESET_DEVICE corrected 0x002D1004 -> 0x002DD000 (the old value
  decoded to function 0x401 with the access bits cleared, so DeviceIoControl
  would fail ERROR_INVALID_FUNCTION instead of resetting). Windows-only. (low)

Adds a transport-failure classification regression test.
2026-06-22 15:50:22 -07:00
Matthew Jackson b12c1e622d mux/disc: abort single-pass rip on transport failure (USB-bridge crash)
A direct disc://→mkv:// single-pass rip drives fill_extents in
skip_errors mode. On a read failure it shrank the batch, retried, and
once bottomed out zero-filled + skipped the unit and continued. A SCSI
transport failure (status=0xFF) is a USB-bridge crash, NOT a skippable
bad sector: the bridge is wedged and every subsequent read fails the
same way. So the loop marched the entire disc at one ~15s bridge-
recovery per probe, producing no MKV — the user-reported 'hundreds of
0x28/0xff warnings, runs forever, Movie.mkv never created'.

Fix: short-circuit to an error on transport failure before any
shrink/skip, even under skip_errors — mirroring the multipass sweep's
transport-failure rule in read_error::handle_read_error. The CLI
surfaces it so the user power-cycles the drive or switches to multipass
recovery. Regression test asserts exactly one read is issued and no skip
is counted (no infinite march).
2026-06-22 15:18:52 -07:00
Matthew Jackson 4d81affb45 Merge branch 'rc3-topgun' into rc3-integration 2026-06-22 15:08:06 -07:00
Matthew Jackson c73a3dbcb6 Fix Windows multi-drive selection, disk:// alias, and READ chunking
Three fixes for a Windows ASUS Blu-ray drive that failed/spammed errors:

- resolve.rs: accept disk:// as an alias for disc:// (identical behavior;
  empty = auto-detect, path = device). Windows users commonly type
  disk://i: after the drive-letter convention.

- drive::find_drive: prefer a drive that reports media present. Enumerate
  all optical drives, query Drive::drive_status() (GET EVENT STATUS, works
  regardless of firmware), and return the first reporting DiscPresent;
  fall back to the first enumerated drive when none report a disc so
  single-drive / quirky setups don't regress. Selection policy split into
  select_drive_with_media() for unit testing.

- READ chunking: add ScsiTransport::max_transfer_bytes() (default 1 MiB).
  Windows SPTI overrides it with the adapter MaximumTransferLength queried
  via IOCTL_STORAGE_QUERY_PROPERTY / StorageAdapterProperty, clamped to a
  64 KiB floor (fallback on query failure). Drive::read now caps each
  READ(10) to that limit: small reads take the unchanged single-CDB path,
  larger reads loop over read_one() chunks, reporting the failing chunk's
  LBA on error. This stops the 16 MiB single read that exceeded the
  adapter limit, made DeviceIoControl fail, and spammed transport-failure
  warnings with slow tiny-read fallbacks.

Tests added for the disk:// alias, media-preference selection, and READ
chunk decomposition / per-chunk error LBA.
2026-06-22 15:02:24 -07:00
Matthew Jackson 4f606ae9a3 aacs: source OEM host certs from keysource layer
Complete the OEM/AACS cert baseline so host certs are a KeySource output,
never compiled in. With an unlocker present the OEM route is unused
(unlocker_read_volume_id short-circuits); without one, the cert handshake
runs when a keysource supplies a host cert and fails gracefully when none
does.

- KeySource trait gains host_certs() (default empty), reusing the existing
  aacs::HostCert type. A source holds certs as its second kind of AACS
  material alongside decryption keys.
- ScanOptions gains key_sources so the handshake can collect certs across
  the app's keysource layer, unioned with DriveCredentials.
- do_handshake_cert collects certs via collect_host_certs (credentials +
  every key source). Zero certs from any source now returns the new
  graceful Error::AacsNoHostCert (code 7024, sentinel <no host cert>)
  instead of silently skipping; resolution still falls back to the
  path-1 disc-hash -> VUK lookup, which drops the error on a hit.
- error.rs: add E_AACS_NO_HOST_CERT / Error::AacsNoHostCert, wired into
  code(), Display, and the round-trip + sentinel tests.

HandshakeResult { volume_id, read_data_key } unchanged: the cert path
still yields both the VID and the bus key.
2026-06-22 11:23:38 -07:00
Matthew Jackson 25acd09504 unlock: finalize Unlocker 3-capability contract
Rename the trait to a generic, drive-neutral capability contract so future
unlockers don't conform to LibreDrive specifics:

  - unlock(...)   -> unlock_drive(...)        (the one required capability)
  - read_vid(...) -> read_volume_id(...)      (no-op default)
  - add set_max_read_speed(...)               (no-op default)

The trait doc now states the contract in one place: unlockers are optional
drive-capability providers; the AACS layer is the always-present baseline and
falls back to the full cert handshake when no unlocker matches. Implement only
the capabilities your drive supports.

Registry: route_unlock now calls unlock_drive; unlocker_read_vid renamed to
unlocker_read_volume_id; add unlocker_set_max_read_speed (mirrors route_unlock
resolution, first matching unlocker, no-op if none match). drive::init calls
it on a matched drive in the post-unlock path; a speed-set failure is logged
and does not fail the rip. encrypt.rs handshake updated to the new VID helper.

Tests updated for the renames; added a set_max_read_speed routing test
(match invokes, no-match is a safe no-op).
2026-06-22 11:05:21 -07:00
Matthew Jackson 159e967760 unlock: add OEM read_vid capability to Unlocker seam
An Unlocker unlocks drive functionality, not just the disc: unlock() is
one capability, OEM VID retrieval is another. Widen the Unlocker trait
with a default-no-op read_vid(), add an unlocker_read_vid registry helper
that mirrors route_unlock resolution, and consult it in do_handshake_cert
before the cert-based VID read. A matching unlocker that serves a VID via
its OEM path short-circuits the cert handshake — VID is obtained without
the host certificate + HRL (restoring the pre-refactor decoupled OEM VID
path, now living inside the unlocker). Non-matching drives, and unlockers
without an OEM VID path, fall through to cert auth unchanged.

is_unlocked() now reports the honest signal (a registered unlocker matched
this drive) instead of const false.
2026-06-22 10:50:47 -07:00
Matthew Jackson 6dc62bcd84 Extract drive unlock behind pluggable Unlocker seam
libfreemkv must stay firmware-clean for crates.io. Move ALL drive-unlock
knowledge — firmware blobs, WRITE_BUFFER/MODE SELECT upload, unlock CDBs,
the MT1959 variant-A/B handshake, the 800 KB profiles.json database, and
the DriveProfile parsing — out into the freemkv-unlock-ld crate.

libfreemkv now keeps only the seam:
  - Unlocker trait (name/matches/unlock) + a process-wide ordered registry
    (register_unlocker / route_unlock) in src/unlock.rs
  - Drive::init() walks the registry; the first unlocker whose matches(id)
    is true runs unlock(scsi, id); if none match the drive is left in
    stock mode and the host-cert AACS handshake (the OEM route) carries
    the disc.

The unlocker issues its own CDBs through the public ScsiTransport::execute,
so libfreemkv knows nothing about how unlocking happens.

Removed:
  - profiles.json
  - src/platform/mt1959/{mod,variant_a,variant_b}.rs
  - src/profile.rs (DriveProfile, ProfilesFile, find_by_drive_id, ...)
  - the PlatformDriver trait

Because the Unlocker seam reports only success/failure (no extended-access
marker), VID acquisition is now always via the cert-based handshake; the
per-drive OEM-VID-CDB shortcut and Drive::is_unlocked() (now const false)
are removed/neutralized. Disc-speed calibration moved into the unlocker's
unlock(); Drive::probe_disc() is a no-op.

git grep over src/ is firmware-blob/profiles/WRITE_BUFFER/mt1959-free.
All tests pass on Rust 1.86 (precommit green).
2026-06-22 10:31:51 -07:00
Matthew Jackson 9250f5bb30 wip: top gun EL/decrypt follow-up (in progress, rc3) 2026-06-22 10:02:53 -07:00
Matthew Jackson e960c2f1be mux/mkv: video-only timeline epochs + finer 0.1ms TimestampScale
Fixes corrupt MKV seek index on single-clip titles with many
interleaved tracks (Top Gun UHD: 2 video, 11 audio, 32 PGS).

TimelineContinuity previously shared one high_ns frontier + offset_ns
across ALL tracks. A sparse, lagging non-video frame (subtitle/audio)
ratcheted the frontier up; the next normal video frame then sat >3s
below it and was misread as a clip-boundary discontinuity, permanently
bumping offset_ns. On a one-clip title this fired thousands of times
and inflated Cue/cluster timestamps into the billions of ms, destroying
the seek index (ffmpeg then seeked to wrong positions and emitted
spurious 'Could not find ref with POC N' errors).

Now only the VIDEO track drives epoch decisions: video alone advances
the frontier and opens a new epoch on a real backward PTS jump.
Non-video tracks are remapped under the current offset and never touch
the frontier or offset. A lagging non-video tail straggler at a genuine
multi-clip boundary (old-epoch raw PTS under the new offset) is
recognised via the previous offset and remapped to the seam, so it
neither flies forward nor forces a back/forward-dated split cluster.
Genuine multi-clip seamless rebasing is preserved.

Also drop TimestampScale from 1ms to 0.1ms (100_000 ns/tick) so
23.976fps frames and 0.833ms TrueHD AUs stop colliding on a single
tick (the source of the non-monotonic-DTS warnings and the audio
cadence flattening). The finer scale shrinks the i16 block-relative
span to ~3.27s, so: cluster duration is set to 2s nominal (keeps
keyframe-driven clusters within the i16 range for typical GOPs), and
the i16-overflow cluster-split path now emits a Cue for the split
cluster so the seek index has no gaps.

Regression tests: single-clip late-subtitle must not inflate offset_ns;
non-video must not advance the frontier; non-video straggler remapped
to seam at a real boundary; every cluster (incl. i16-split) carries a
Cue. Existing tick/duration assertions updated for the new scale.
2026-06-22 09:45:45 -07:00
Matthew Jackson f74979bdb4 docs: scrub AACS internals + key-coverage claims from changelog; condense pre-1.0 history 2026-06-22 09:39:40 -07:00
Matthew Jackson dc2cac1b5f v1.0.0-rc.2: bump version 2026-06-22 09:06:54 -07:00
Matthew Jackson 337e77951c rc2: macOS cross-compile fix + security/recovery hardening
- build.rs: pass target -arch to cc so macos_shim cross-compiles (x86_64-apple-darwin)
- AACS/CSS: unit-aligned decrypting sweep; per-VTS CSS title keys (hard-fail on wrong VTS);
  reject truncated Unit_Key_RO; AACS 2.0 sig-verify skip; CSS bus-auth random nonce
- recovery: gap-filling mapfile load; sweep/copy resume reconciliation; stale-mapfile abort;
  patch wedge/damage-window range reset
- mux: TS continuity + PSI CC desync guards; HEVC numTemporalLayers clamp; MPEG-2 pending
  byte-cap; PS parse_pts marker-bit validation; HdrFormat strict parse; Unknown-variant metadata
- net/keydb: network:// SSRF parity (IPv4-mapped, CGNAT, 0.0.0.0/8, Class-E); bounded keydb
  header read + size cap + error context
- io: durable mapfile fsync; NFS writeback degrade; sync_file_range error capture;
  Windows SCSI u32 transfer guard
2026-06-22 08:58:10 -07:00
Matthew Jackson 5941c059c6 v1.0.0-rc.1
CSS keyless decrypt (Stevenson), AACS 1.0/2.0/2.1, MPEG-2 DVD, multi-OS SCSI, multipass recovery, mux highway, audit hardening
2026-06-21 21:06:07 -07:00
Matthew Jackson e8bb6225ac libfreemkv 0.31.10: parallelise recover_dk_position slot scan
The subset-difference slots are independent, so rayon find_map_any scans them
in parallel and cancels on first match. UHD MKB no-match scan ~26s -> ~4.6s on
8 cores. Bit-identical result; 58 aacs tests pass.
2026-06-17 19:56:07 -07:00
Matthew Jackson 9c80ef8245 libfreemkv 0.31.9: ~3x fewer AES ops in the subset-difference PK walk
calc_pk_from_dk derived all three children (left/pk/right) at every tree
level but used only the one it descended into; the Processing Key only
matters at the final node. Derive just the descended child per level + the
PK once at the end. Bit-for-bit identical; speeds every DK->MK derivation
(disc decryption + unpositioned-DK recovery). 60.8s -> 22.9s on a UHD
worst-case recovery scan.
2026-06-17 15:43:39 -07:00
Matthew Jackson dc87962e50 libfreemkv 0.31.8: recover unpositioned device key position from an MKB
Add recover_dk_position: boil a position-less device key down against a
disc MKB to its invariant subset-difference position (node/uv/u_mask_shift)
— zero-descent probe + ancestor walk-up, hoisted verify. Consolidate the
SD-walk surface (drop the research-only probe::walk_pk_against_tables;
make derive_media_key_from_pk_walked internal).
2026-06-17 15:29:51 -07:00
Matthew Jackson 4221cd6a86 libfreemkv 0.31.7: expose Processing Key from the DK walk 2026-06-17 13:18:34 -07:00
Matthew Jackson dda4e7482b aacs: unify the SD-walk; derive_media_key_from_dk now exposes the PK
Two byte-identical copies of the subset-difference walk lived in keys.rs
and variants.rs. Consolidate the pure helpers (aesg3, calc_v_mask,
calc_pk_from_dk) into keys.rs (pub(super)); variants.rs imports them.

Add derive_media_key_and_pk_from_dk(mkb, dks) -> Option<(mk, pk)>, which
returns the intermediate Processing Key the walk already computes;
derive_media_key_from_dk becomes a thin wrapper. This lets callers bank
the PK on a DK boil instead of re-deriving it via a second, divergent
walk (the classical-vs-variant cvalues order made that miss silently).

216 AACS tests pass (incl. a new (mk,pk) regression); precommit (1.86) green.
2026-06-17 13:11:03 -07:00
Matthew Jackson f80551f278 udf: read AACS MKB length-aware; honor inline allocation descriptors
read_file rejected the padded ~128 MiB MKB_RO.inf via the 0.31.0
MAX_FILE_BYTES cap, so read_aacs_inputs failed and the online
key-resolve path never contacted the keyserver. Read the MKB's real
record length from its header and read exactly that. Also honor
inline/embedded (AD type 3) files so small AACS .inf files read from
the ICB payload instead of being misparsed as allocation descriptors.

Release 0.31.6.
2026-06-08 19:20:41 -07:00
Matthew Jackson 41a6d89cd1 libfreemkv 0.31.5: do not force monotonic block timestamps on video
B-frame video PTS is legitimately non-monotonic in decode/storage order; the
audio-oriented monotonic nudge was clobbering it to prev+1ms, which decoders
flagged as non-monotonic DTS (thousands per title). Apply the nudge to
audio/subtitle only; video keeps its true PES PTS. + regression test.
2026-06-08 09:00:35 -07:00
Matthew Jackson f79c2a0aa9 libfreemkv 0.31.4: prune 144 vacuous tests (keep spec-grounded subset) 2026-06-08 07:28:55 -07:00
Matthew Jackson d181362460 libfreemkv 0.31.3: use RFC 5737 doc address in network URL test fixture 2026-06-07 22:42:09 -07:00
Matthew Jackson 8000bae177 libfreemkv 0.31.2: comprehensive spec-grounded test suite (~950 tests)
Test-hardening release, no runtime changes. Adds spec-grounded unit tests
across the silent-corruption surfaces — UDF/MPLS/CLPI/IFO parsing, BD/DVD
title + extent assembly, AACS/CSS key handling, TS/PS demux + codec parsers,
MKV/EBML container output, the mux pipeline, sector prefetch + decrypt
decorator, drive/SCSI sense decoding, label extraction, and core I/O. Each
test is grounded in the format spec or real on-disc behavior and verified to
fail under a targeted source mutation. No behavior changed.
2026-06-07 22:28:29 -07:00
Matthew Jackson 2a55bab3ed tests: read_aacs_inputs Long-AD coverage; scrub title names from comments
Add a precommit fixture proving Disc::read_aacs_inputs reads a Long-AD,
multi-extent /AACS/Unit_Key_RO.inf in full — the exact input the online
key-request path depends on (no disc/deploy needed). Make
read_aacs_inputs_from_reader pub(crate) for the test.

Reword internal comments/doc examples to generic descriptions instead of
specific media titles.
2026-06-07 21:25:29 -07:00
Matthew Jackson 222a596c55 libfreemkv 0.31.1: UDF Long-AD stride fix + MKB trim guard + regression tests 2026-06-07 20:49:55 -07:00
Matthew Jackson c1b4f3cbb3 tests: harden UDF allocation-descriptor + bad-sector recovery paths
Spec-grounded unit tests for the silent-corruption surfaces, each verified to
fail under a targeted source mutation (no vacuous tests).

udf (10): Extended-AD 20-byte stride + extent LBA at off+12, type-1 sparse
extents skipped not emitted, zero-length type-0 terminator, continuation-loop
bound (anti-hang), UTF-16BE and 8-bit name decoding, FID L_IU offset, parent
(..) FID skip, d-string length-byte cap. Locks the spec branches a future
allocation-descriptor refactor must not silently break.

recovery (9): Pass-N damage-skip range bounds (forward/reverse cursor stays in
range), one-quarter-of-remaining skip cap, below-threshold no-op, work-done
accounting, and bridge-degradation retry-to-budget fall-through.
2026-06-07 20:46:00 -07:00
Matthew Jackson 06c30aa466 udf: honor ICB allocation-descriptor type (Short/Long/Extended AD)
read_icb_extents hardcoded an 8-byte Short-AD stride for every file. Large
BD-ROM .m2ts streams use 16-byte Long ADs; striding them as Short ADs reads
descriptor #0 correctly (length+lba align) but lands #1 in the middle of the
first Long AD (its zero impl_use bytes). The AD-list terminator (data_len==0
=> break) then fired on that zero and stopped after the first extent, so every
multi-extent title truncated at ~1 GiB. The same reader backs read_file, so
disc AACS-input files (/AACS/*.inf) and the m2ts mux extents were both
affected.

Read the ICB Tag flags (AD type) and stride 8/16/20 bytes for Short/Long/
Extended ADs accordingly; Extended ADs carry the lba at off+12.

aacs: extract trim_mkb and restore its guard so an MKB whose content length
the parser cannot determine (mkb_content_len == 0) is returned intact instead
of truncated to empty.

Regression tests: Long-AD read_icb_extents returns all extents; Long-AD
read_file returns full content; trim_mkb never zeroes an unrecognised MKB.
2026-06-07 19:53:40 -07:00
Matthew Jackson 061f68594a 0.31.0: hardening and correctness pass across mux, codec, AACS/CSS, UDF/MPLS/CLPI, recovery, drive/SCSI, labels, and I/O
Library-wide review-and-fix pass: tightened AACS keydb/handshake/variant
handling and trailing-partial-unit policy, corrected MPLS mark offset and
added UDF allocation bounds, hardened the mux/codec framing and M2TS paths,
guarded SCSI READ CAPACITY short transfers and unified error mapping, added
overflow guards on untrusted disc input, and made prefetch shutdown
deterministic. Release profile now builds with thin LTO + single codegen unit.
2026-06-07 17:37:38 -07:00
391 changed files with 158217 additions and 33315 deletions
+1
View File
@@ -0,0 +1 @@
github: MattJackson
+63
View File
@@ -0,0 +1,63 @@
version: 2
# Dependency updates land on `dev`, never on `main`.
#
# `main` here is a RELEASE POINTER that release.sh moves to each tag. A bot
# commit on it would put work there that no tag contains, which is exactly the
# state that aborted the 1.6.2 cascade at the last step -- so pointing
# Dependabot at main would recreate that failure on a schedule.
updates:
- package-ecosystem: cargo
directory: /
target-branch: dev
schedule:
interval: weekly
open-pull-requests-limit: 5
# One PR per week for the routine bumps instead of one per crate. Eight
# repos times a handful of crates is a volume nobody reads, and an
# unread PR queue is indistinguishable from no updates at all.
groups:
minor-and-patch:
update-types:
- minor
- patch
ignore:
# The freemkv crates depend on each other by GIT TAG, re-pinned by
# release.sh as part of the release commit. Dependabot cannot see that
# cascade, so a PR bumping one of these would fight the release process
# and could pin a version whose tag does not exist yet.
- dependency-name: freemkv-unlock
- dependency-name: libfreemkv
- dependency-name: freemkv-keysources
- dependency-name: freemkv-i18n
- dependency-name: freemkv-engine
# The workflows are now real infrastructure -- the release cascade, the
# cross-platform hash matrix, the disc gate -- so their actions need the same
# attention as the crates.
- package-ecosystem: github-actions
directory: /
target-branch: dev
schedule:
interval: weekly
open-pull-requests-limit: 5
groups:
actions:
update-types:
- minor
- patch
ignore:
# NOT a dependency: `dtolnay/rust-toolchain` is versioned by the RUST
# release it installs, and the tag we pin is the toolchain CI is pinned
# to on purpose -- precommit.sh runs the same one locally so a lint that
# passes on a developer's newer default cannot pass CI by accident.
#
# Dependabot reads those tags as semver and proposed 1.97.0 -> 1.100.0,
# a Rust version that does not exist. Every such PR 404s on toolchain
# download across all eight repos, and they regenerate weekly -- eight
# permanently-red PRs that promote.yml then has to special-case when it
# decides whether dev is green.
#
# Bumping the toolchain is a deliberate, all-eight-repos change, made by
# hand together with precommit.sh. There is nothing here for a bot.
- dependency-name: dtolnay/rust-toolchain
+383 -17
View File
@@ -2,41 +2,407 @@ name: CI
on:
push:
branches: [main]
# dev -> qa -> main. `dev` is where work lands and is meant to be pushed
# to often: these are the FAST checks, so a mistake surfaces in minutes.
# `qa` is the release candidate — it runs these too, plus the expensive
# suite in qa.yml. `main` only ever moves at release time, to a tagged
# commit that was already green on qa.
branches: [main, dev, qa]
pull_request:
jobs:
lint:
# ── Single source of truth for the MSRV ─────────────────────────────────────
#
# `rust-version` in Cargo.toml is the ONE place the MSRV is declared. This job
# reads it; every job that must build/test AT the MSRV consumes
# `needs.resolve-msrv.outputs.v` instead of a hardcoded pin, so the floor
# cannot drift out of sync across workflow files. The `test` job building
# green at this value IS the verification that the declared MSRV is real.
# clippy and the `consumers` job deliberately do NOT use it — they run on
# `stable` (lint debt is caught against current Rust; dependents float highest).
resolve-msrv:
runs-on: ubuntu-latest
outputs:
v: ${{ steps.msrv.outputs.v }}
steps:
- uses: actions/checkout@v7
- id: msrv
shell: bash
run: echo "v=$(grep -m1 '^rust-version' Cargo.toml | sed -E 's/.*\"([^\"]+)\".*/\1/')" >> "$GITHUB_OUTPUT"
# CI owns the changelog: main may never carry a released version without a
# CHANGELOG.md entry for it. The release orchestrator dates an existing
# `## [X.Y.Z]` heading — it does NOT invent one — so a version bump that
# forgot its changelog entry must fail here rather than ship a blank release.
# Hard-fails on main (and PRs targeting main); warns elsewhere so an
# in-progress dev/qa branch whose entry isn't written yet is not blocked.
changelog:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- uses: actions/checkout@v7
- name: Require a CHANGELOG entry for the crate version on main
shell: bash
run: |
set -euo pipefail
version=$(grep -m1 '^version = ' Cargo.toml | sed -E 's/.*"([^"]+)".*/\1/')
if grep -qE "^## \[${version}\]" CHANGELOG.md; then
echo "CHANGELOG.md has an entry for ${version}."
exit 0
fi
msg="CHANGELOG.md has no '## [${version}]' entry"
if [ "${GITHUB_REF_NAME:-}" = "main" ] || [ "${GITHUB_BASE_REF:-}" = "main" ]; then
echo "::error::${msg} — main requires a changelog entry for the released version."
exit 1
fi
echo "::warning::${msg} (allowed on ${GITHUB_REF_NAME:-unknown}; required before it can reach main)."
lint:
needs: resolve-msrv
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
components: clippy, rustfmt
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
# fmt runs on the pinned MSRV toolchain; clippy runs on stable. Third
# parties compile our source on their own toolchain and never run our
# clippy, so the lint toolchain need not match the MSRV — only build/test
# must (see the test/cross jobs). Linting on stable makes new-lint debt a
# BLOCKING gate the pinned version would otherwise hide. The MSRV pin is
# installed second so it stays the default toolchain (fmt uses it
# implicitly); clippy is invoked explicitly with `+stable`.
- uses: dtolnay/rust-toolchain@stable
with:
components: clippy
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
components: rustfmt
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo fmt --check
# libfreemkv is a library — Cargo.lock is gitignored. --locked
# would always fail on a fresh runner because there's no committed
# lockfile to lock against. The binary crates (freemkv, autorip,
# bdemu) track Cargo.lock and DO use --locked.
- run: cargo clippy -- -D warnings
working-directory: libfreemkv
# Cargo.lock is now committed (issue #15) and the `deny` job audits the
# resolved graph, but --locked is still omitted: CI's sibling path-patch
# layout resolves the freemkv crates as path packages, which --locked
# can't satisfy — the same reason the binary crates omit it in ci.yml.
# --all-targets so TEST code is linted too. Without it this crate — the
# reference implementation for the other seven — was the only one whose
# tests had never been linted at all, and it was hiding 74 findings.
- run: cargo +stable clippy --all-targets -- -D warnings
working-directory: libfreemkv
test:
needs: resolve-msrv
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo test --tests
working-directory: libfreemkv
# The memory orderings in `scsi::fd_handoff` — the fd hand-off between a
# recovery thread and `SgIoTransport::drop` — cannot be checked by `cargo
# test`: they compile to the same instructions on x86_64, so the WRONG ones
# pass every run. loom enumerates the executions the C++20 model permits and
# reports the leaked descriptor directly, which makes it the only regression
# coverage that part of the module has. If this job stops running, that
# coverage disappears silently behind a green build.
#
# Its own job rather than a step in `test` because `RUSTFLAGS` is part of the
# rust-cache key: folding it into `test` costs that job a full cache miss on
# every run. It takes the same sibling checkout as the jobs above even though
# `--no-default-features` drops freemkv-unlock from the BUILD — `--features`
# does not exempt an optional path dependency from manifest RESOLUTION, so
# without the sibling this job would fail the moment the branch tip carries
# the path-dep form of that dependency rather than a git tag.
loom:
needs: resolve-msrv
runs-on: ubuntu-latest
env:
RUSTFLAGS: --cfg loom
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
components: clippy
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo test --lib --no-default-features --features scsi fd_handoff
working-directory: libfreemkv
# `lint` runs clippy without `--cfg loom`, so this is the only place the
# model code is linted at all.
- run: cargo clippy --lib --no-default-features --features scsi --profile test -- -D warnings
working-directory: libfreemkv
deny:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@stable
- uses: EmbarkStudios/cargo-deny-action@v2
with:
command: check advisories bans licenses sources
manifest-path: libfreemkv/Cargo.toml
check-macos:
# dev is the FAST lane: this job still runs, but on the release-candidate
# branches rather than on every push to dev. Nothing is deleted and no
# platform stops being checked before a release -- qa.yml independently
# covers macOS and Windows, and the jobs unique to this file (the Intel
# macOS build, the Windows release build) run here on qa and main. A push
# to dev is meant to be cheap and frequent; waiting on three runner pools
# to agree is what a release candidate is for.
#
# `if` SKIPS the job (it does not queue). A queued job would be far worse
# than a slow one: release.sh's CI gate refuses while any run for the
# commit is still in progress, so a never-scheduled job blocks releases
# silently -- see the note on real-media in qa.yml.
needs: resolve-msrv
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: macos-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo check
working-directory: libfreemkv
check-windows:
# dev is the FAST lane: this job still runs, but on the release-candidate
# branches rather than on every push to dev. Nothing is deleted and no
# platform stops being checked before a release -- qa.yml independently
# covers macOS and Windows, and the jobs unique to this file (the Intel
# macOS build, the Windows release build) run here on qa and main. A push
# to dev is meant to be cheap and frequent; waiting on three runner pools
# to agree is what a release candidate is for.
#
# `if` SKIPS the job (it does not queue). A queued job would be far worse
# than a slow one: release.sh's CI gate refuses while any run for the
# commit is still in progress, so a never-scheduled job blocks releases
# silently -- see the note on real-media in qa.yml.
needs: resolve-msrv
if: github.ref_name == 'qa' || github.ref_name == 'main'
runs-on: windows-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- run: cargo check
- uses: actions/checkout@v7
with:
path: libfreemkv
# libfreemkv path-deps ../freemkv-unlock on the BRANCH tip (release.sh
# swaps it to a git tag only inside the tagged commit, then restores the
# path dep). CI checks out one repo, so the branch tip has never been
# buildable here — every green run you have ever seen was a tag build,
# and Windows/Linux were first compiled at release time.
#
# Both repos go into subdirectories because actions/checkout refuses a
# `path:` outside $GITHUB_WORKSPACE, and `../freemkv-unlock` is outside.
# With this layout the path dep resolves exactly as it does locally.
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
# Build the tests (not just `cargo check`): catches errors in test
# code and forces full codegen of the Windows-only SPTI transport
# (src/scsi/windows.rs), which never compiles on the Linux/macOS dev
# hosts. We don't `cargo test` here — the suite needs no drive but the
# extra build is the value; running tests is covered by the Linux job.
- run: cargo build --tests
working-directory: libfreemkv
# ── Did this change break anything downstream? ──────────────────────────────
#
# Every job above proves libfreemkv builds. None proved its DEPENDENTS do,
# and that gap is real: an engine signature change broke autorip today and
# went unnoticed because consumer CI only fires on a push to that consumer.
# libfreemkv sits below all five of them, so a break here is worth strictly
# more than a break anywhere else in the project.
#
# `cargo check --all-targets` only — each dependent owns its own behaviour
# and has its own suite. The question here is just "does everything built on
# me still compile against this commit".
consumers:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with: { path: libfreemkv }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-unlock, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-unlock }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-keysources, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-keysources }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-engine, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-engine }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv-i18n, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv-i18n }
- uses: actions/checkout@v7
with: { repository: freemkv/freemkv, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: freemkv }
- uses: actions/checkout@v7
with: { repository: freemkv/autorip, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: autorip }
- uses: actions/checkout@v7
with: { repository: freemkv/bdemu, ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}", path: bdemu }
- name: Point every dependent at THIS libfreemkv commit
shell: bash
run: |
for c in freemkv-keysources freemkv-engine freemkv autorip bdemu; do
mkdir -p "$c/.cargo"
cat > "$c/.cargo/config.toml" <<'EOF'
[patch.crates-io]
libfreemkv = { path = "../libfreemkv" }
freemkv-keysources = { path = "../freemkv-keysources" }
freemkv-engine = { path = "../freemkv-engine" }
freemkv-i18n = { path = "../freemkv-i18n" }
EOF
done
# The dependents span a higher MSRV than libfreemkv itself — the binaries
# (freemkv/autorip/bdemu) always track the highest supported Rust. Build
# them on `stable` so this job satisfies the highest dependent floor; the
# library's own lower MSRV (verified by the `test` job) cannot compile them.
- uses: dtolnay/rust-toolchain@stable
- uses: Swatinem/rust-cache@v2
with:
workspaces: |
freemkv-keysources
freemkv-engine
freemkv
autorip
bdemu
# `cargo check` alone only proves the dependents still COMPILE against
# this commit. It cannot see a behavioural change — the library keeps its
# signatures and a dependent's tests start failing. That is the shape of
# every defect worth catching here, so run their suites too.
- run: cargo test --tests
working-directory: freemkv-keysources
- run: cargo test --tests
working-directory: freemkv-engine
- run: cargo test --tests
working-directory: freemkv
- run: cargo test --tests
working-directory: autorip
- run: cargo test --tests
working-directory: bdemu
# ── Coverage (non-required, additive) ───────────────────────────────────────
#
# Reports line coverage to Codecov. Deliberately NOT wired into any release
# gate: it is informational only (see codecov.yml) and no other job `needs:`
# it. Mirrors the `test` job's layout exactly — same MSRV toolchain (resolved
# from Cargo.toml), same dual checkout so `../freemkv-unlock` resolves, cache
# workspace and working-directory. cargo-llvm-cov comes as a prebuilt binary
# from taiki-e/install-action, so it is toolchain-independent.
#
# No `--locked`: like the fmt/clippy/test jobs above, the sibling path-patch
# layout resolves the freemkv crates as path packages, which --locked can't satisfy.
coverage:
needs: resolve-msrv
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
components: llvm-tools-preview
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- uses: taiki-e/install-action@cargo-llvm-cov
- run: cargo llvm-cov --workspace --ignore-run-fail --lcov --output-path lcov.info
working-directory: libfreemkv
- uses: codecov/codecov-action@v7
with:
token: ${{ secrets.CODECOV_TOKEN }}
files: libfreemkv/lcov.info
slug: freemkv/libfreemkv
fail_ci_if_error: false
+17
View File
@@ -0,0 +1,17 @@
name: comment-guard
# Caps explanatory prose by COMMENT INTENT (not visibility):
# //! / /*! module docs — unbounded (module-scoped, can't hide per-item text)
# /// / /** */ item docs — 8 prose lines (fenced ``` examples excluded)
# // / /* */ asides — 3 lines (single-blank-separated runs count as one)
# Overflow belongs in the module //! or docs/<topic>.md with a one-line // pointer.
on: [push, pull_request]
jobs:
comment-guard:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
- name: Run comment-guard
run: python3 ci/comment-guard.py
+32
View File
@@ -0,0 +1,32 @@
name: doc-links
# Fails on broken intra-doc links in `///`/`//!` doc comments (e.g. a `[Type]`
# link to a renamed/removed item). Catches doc drift that `cargo build` misses.
#
# libfreemkv path-deps ../freemkv-unlock on the branch tip, so — like ci.yml —
# both repos are checked out into sibling subdirectories for the path dep to
# resolve.
on: [push, pull_request]
jobs:
doc-links:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: "${{ github.ref_name == 'qa' && 'qa' || 'dev' }}"
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@1.98.0
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- name: cargo doc (deny broken intra-doc links)
working-directory: libfreemkv
env:
RUSTDOCFLAGS: "-D rustdoc::broken_intra_doc_links"
run: cargo doc --no-deps --document-private-items --lib --bins
+41
View File
@@ -0,0 +1,41 @@
name: identity-guard
# Prevention: no commit may carry an unrelated-business or non-canonical author/
# committer email. dev@example.com (a dead corp address) once slipped onto the
# release cascade + a batch of local commits; this fails the build if it recurs.
on:
push:
branches: [dev, qa, main]
pull_request:
jobs:
identity:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Reject banned author/committer emails
env:
BEFORE: ${{ github.event.before }}
AFTER: ${{ github.sha }}
EVENT: ${{ github.event_name }}
PR_BASE: ${{ github.event.pull_request.base.sha }}
PR_HEAD: ${{ github.event.pull_request.head.sha }}
run: |
set -euo pipefail
# Unrelated business domains + a personal address normalized to the
# GitHub noreply. Real external contributors are unaffected.
BANNED='@example-a\.invalid|@example-b\.invalid|owner@example\.invalid'
if [ "$EVENT" = "pull_request" ]; then
range="$PR_BASE..$PR_HEAD"
elif [ -n "${BEFORE:-}" ] && [ "$BEFORE" != "0000000000000000000000000000000000000000" ] && git cat-file -e "$BEFORE" 2>/dev/null; then
range="$BEFORE..$AFTER"
else
range="$AFTER~1..$AFTER"
fi
bad=$(git log --format='%an <%ae>%n%cn <%ce>' "$range" 2>/dev/null | sort -u | grep -iE "$BANNED" || true)
if [ -n "$bad" ]; then
echo "::error::commit(s) carry a banned author/committer identity:"; echo "$bad" | sed 's/^/ /'
echo "::error::Commit as your GitHub noreply (1085847+MattJackson@users.noreply.github.com). Fix: git config user.email <noreply>; git commit --amend --reset-author."
exit 1
fi
echo "identity-guard: clean"
+35
View File
@@ -0,0 +1,35 @@
name: leak-guard
# Self-contained public-repo leak gate. Public CI cannot reach the private
# tooling, so this encodes only the generic net: internal-infra references,
# tracked CLAUDE.md/.claude paths, and AI-attribution in commit messages.
# No project-specific reverse-engineering vocabulary lives here.
on: [push, pull_request]
jobs:
leak-guard:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Compute commit range
id: range
run: |
if [ "${{ github.event_name }}" = "pull_request" ]; then
base="${{ github.event.pull_request.base.sha }}"
head="${{ github.event.pull_request.head.sha }}"
echo "range=$base..$head" >> "$GITHUB_OUTPUT"
else
before="${{ github.event.before }}"
after="${{ github.sha }}"
# New branch / first push: github.event.before is all-zeros.
if [ -z "$before" ] || [ "$before" = "0000000000000000000000000000000000000000" ]; then
echo "range=$after" >> "$GITHUB_OUTPUT"
else
echo "range=$before..$after" >> "$GITHUB_OUTPUT"
fi
fi
- name: Run leak-guard
run: bash ci/leak-guard.sh "${{ steps.range.outputs.range }}"
+154
View File
@@ -0,0 +1,154 @@
name: qa
# ── The qa gate: "is this production worth?" ────────────────────────────────
#
# dev -> qa -> main.
#
# `dev` is for committing often. ci.yml answers "is it green" in minutes with
# fmt, clippy and the unit suite, so a mistake surfaces while it is still cheap
# to fix. `qa` is the release-candidate branch, and THIS workflow is the claim
# that a commit is production worth: everything expensive that can run without
# physical media. `main` only ever receives a qa that went green here.
#
# Sibling repos are checked out at `qa`, NOT `dev`. A qa run that resolved its
# dependencies from dev tips would be validating a combination that is not the
# one being released, which is the exact failure this branch exists to prevent.
#
# What this gate CANNOT cover: `disc://` and real `iso://` need physical media,
# and no hosted runner has an optical drive or the image hoard. Those run on a
# self-hosted runner (see the media job at the end) and are the one leg that
# stays on hardware.
on:
push:
branches: [qa]
workflow_dispatch:
jobs:
# Single source of truth for the MSRV — see ci.yml. Jobs that build/test AT
# the MSRV read `needs.resolve-msrv.outputs.v` from Cargo.toml's rust-version;
# clippy (cross-lint) runs on stable instead.
resolve-msrv:
runs-on: ubuntu-latest
outputs:
v: ${{ steps.msrv.outputs.v }}
steps:
- uses: actions/checkout@v7
- id: msrv
shell: bash
run: echo "v=$(grep -m1 '^rust-version' Cargo.toml | sed -E 's/.*\"([^\"]+)\".*/\1/')" >> "$GITHUB_OUTPUT"
# ── Name the candidate ────────────────────────────────────────────────
#
# Every push to `qa` is a release candidate, so every push gets a tag:
# v<version>-rc<N>, N incrementing. That is the answer to "which build is on
# qa right now, and is it the one I tested?" — a question that otherwise gets
# answered from memory.
#
# This runs FIRST and does not depend on the gates, deliberately. A red
# candidate needs a name more than a green one does: "rc3 failed
# release-tests on windows" is a sentence you can act on; "qa is red" is not.
# Red on qa is a working gate, not an incident — it is the branch saying this
# is not production worth yet. Fix on dev, get dev green, push qa again.
#
# release.yml excludes v*-rc* so a candidate never publishes a release.
rc-tag:
runs-on: ubuntu-latest
permissions:
contents: write
steps:
- uses: actions/checkout@v7
with:
fetch-depth: 0
- name: Stamp the next rc
shell: bash
run: |
set -euo pipefail
v=$(sed -n 's/^version = "\(.*\)"/\1/p' Cargo.toml | head -1)
[ -n "$v" ] || { echo "no version in Cargo.toml" >&2; exit 1; }
# Numeric sort on the rc ordinal: -rc10 must beat -rc9, and a plain
# lexical sort gets that backwards from the tenth candidate on.
n=$(git tag -l "v$v-rc*" | sed "s|^v$v-rc||" | sort -n | tail -1)
tag="v$v-rc$(( ${n:-0} + 1 ))"
git tag "$tag"
git push origin "$tag"
echo "### Candidate \`$tag\`" >> "$GITHUB_STEP_SUMMARY"
# The debug suite runs on every dev push. Release is a DIFFERENT build:
# overflow checks are off, debug_assert! is compiled out, and inlining
# changes what the optimiser can prove. A test that only passes in debug is
# a test that never guarded the binary anyone actually ships.
release-tests:
needs: resolve-msrv
strategy:
fail-fast: false
matrix:
os: ['ubuntu-latest', 'macos-latest']
runs-on: ${{ matrix.os }}
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo test --release --tests
working-directory: libfreemkv
# clippy's output is target-dependent: cfg-gated code only gets linted on
# the target it compiles for. Linting solely on the dev machine's host
# target is how a lint that CI rejects reaches a push.
cross-lint:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
# clippy runs on stable (matches ci.yml's lint job); build/test stay on
# the MSRV pin. See ci.yml for why the lint toolchain need not be the MSRV.
- uses: dtolnay/rust-toolchain@stable
with:
components: clippy
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: rustup target add x86_64-unknown-linux-gnu
- run: cargo clippy --all-targets --target x86_64-unknown-linux-gnu -- -D warnings
working-directory: libfreemkv
# Windows compiles the tests but does not run them, matching the policy
# ci.yml already set. The value here is codegen: the #[cfg(windows)] halves
# of the SCSI transport and platform layers compile on no other runner, so
# without this they are first built at release time.
windows-build:
needs: resolve-msrv
runs-on: windows-latest
steps:
- uses: actions/checkout@v7
with:
path: libfreemkv
- uses: actions/checkout@v7
with:
repository: freemkv/freemkv-unlock
ref: qa
path: freemkv-unlock
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ needs.resolve-msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
with:
workspaces: libfreemkv
- run: cargo build --release --tests
working-directory: libfreemkv
+33 -18
View File
@@ -4,6 +4,11 @@ on:
push:
tags:
- 'v*'
# NOT the release-candidate tags. Every push to `qa` stamps a
# v<version>-rc<N> so a run can be named, and 'v*' matches those too —
# which would have this workflow build and PUBLISH a GitHub release for
# every candidate, including the red ones.
- '!v*-rc*'
permissions:
contents: write
@@ -12,7 +17,7 @@ jobs:
verify:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Verify Cargo.toml version matches tag
run: |
CARGO_VER="v$(grep '^version' Cargo.toml | head -1 | sed 's/.*"\(.*\)"/\1/')"
@@ -22,33 +27,43 @@ jobs:
fi
echo "Version match: $CARGO_VER"
# Tests run as a PARALLEL TRIPWIRE: they fail the run if they fail, but the
# publish/release jobs do NOT `needs:` this job. The tag decision was already
# gated by the local precommit (same MSRV, same commit). Binary consumers
# (freemkv/autorip/bdemu) git-tag-pin libfreemkv and therefore start building
# the instant this tag exists — so this test job and the crates.io publish
# below must NOT sit on their critical path.
test:
needs: verify
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
# libfreemkv is a library — Cargo.lock isn't tracked, so --locked
# would always fail (no lockfile to lock against on a fresh runner).
- uses: actions/checkout@v7
- id: msrv
shell: bash
run: echo "v=$(grep -m1 '^rust-version' Cargo.toml | sed -E 's/.*\"([^\"]+)\".*/\1/')" >> "$GITHUB_OUTPUT"
- uses: dtolnay/rust-toolchain@master
with:
toolchain: ${{ steps.msrv.outputs.v }}
- uses: Swatinem/rust-cache@v2
# Cargo.lock is now committed (issue #15), but --locked is still omitted:
# the sibling path-patch layout resolves the freemkv crates as path
# packages, which --locked can't satisfy.
- run: cargo test
publish:
needs: test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: dtolnay/rust-toolchain@1.86.0
- name: Publish to crates.io
run: cargo publish
env:
CARGO_REGISTRY_TOKEN: ${{ secrets.CARGO_REGISTRY_TOKEN }}
# NOTE: there is no crates.io publish job. libfreemkv is git-tag-only
# (`package.publish = false` — it git-deps the firmware crate freemkv-unlock,
# which never ships to crates.io). Every consumer git-tag-pins libfreemkv via
# a committed [patch.crates-io]; the git tag itself IS the release artifact.
# A `cargo publish` here fails hard on `publish = false`, so it was removed.
release:
needs: test
# Only needs `verify`; the GitHub Release can be cut as soon as the version
# check passes, in parallel with test + publish.
needs: verify
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
- uses: actions/checkout@v7
- name: Create GitHub Release
uses: softprops/action-gh-release@v2
uses: softprops/action-gh-release@v3
with:
generate_release_notes: true
-36
View File
@@ -1,36 +0,0 @@
name: Update README version
on:
release:
types: [published]
permissions:
contents: write
jobs:
update:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v5
with:
ref: main
token: ${{ secrets.ORG_DISPATCH_TOKEN }}
- name: Update version in README
run: |
VERSION="${{ github.event.release.tag_name }}"
FILE="README.md"
# Update cargo dependency version (e.g. "0.2" -> "0.3")
MAJOR_MINOR="${VERSION#v}"
MAJOR_MINOR="${MAJOR_MINOR%.*}"
sed -i "s|libfreemkv = \"[0-9]*\.[0-9]*\"|libfreemkv = \"${MAJOR_MINOR}\"|" "$FILE"
- name: Commit and push
run: |
VERSION="${{ github.event.release.tag_name }}"
git config user.name "github-actions[bot]"
git config user.email "github-actions[bot]@users.noreply.github.com"
git add README.md
git diff --cached --quiet || git commit -m "Update to libfreemkv ${VERSION}"
git push
+16 -2
View File
@@ -1,10 +1,24 @@
/target
Cargo.lock
*.swp
*.swo
.DS_Store
.cargo/
.claude/worktrees/
# session scratch — never track (may contain RE breadcrumbs)
scratch/
# stray local build artifact
/rust_out
# internal agent context — never publish (path AND dir; leak-guard blocks both)
CLAUDE.md
.claude/
# Nightly harness output. Written into the repo it audits, and it embeds
# absolute paths from the machine that ran it — which must never reach a public
# repo. Ignored rather than relocated so a run from any working copy is safe.
.nightly/
# cargo-mutants working output: large, machine-specific, never committed
mutants.out/
mutants.out.old/
+952 -2302
View File
File diff suppressed because it is too large Load Diff
+83
View File
@@ -0,0 +1,83 @@
# Contributor Covenant Code of Conduct
## Our Pledge
We as members, contributors, and leaders pledge to make participation in our community a harassment-free experience for everyone, regardless of age, body size, visible or invisible disability, ethnicity, sex characteristics, gender identity and expression, level of experience, education, socio-economic status, nationality, personal appearance, race, caste, color, religion, or sexual identity and orientation.
We pledge to act and interact in ways that contribute to an open, welcoming, diverse, inclusive, and healthy community.
## Our Standards
Examples of behavior that contributes to a positive environment for our community include:
* Demonstrating empathy and kindness toward other people
* Being respectful of differing opinions, viewpoints, and experiences
* Giving and gracefully accepting constructive feedback
* Accepting responsibility and apologizing to those affected by our mistakes, and learning from the experience
* Focusing on what is best not just for us as individuals, but for the overall community
Examples of unacceptable behavior include:
* The use of sexualized language or imagery, and sexual attention or advances of any kind
* Trolling, insulting or derogatory comments, and personal or political attacks
* Public or private harassment
* Publishing others' private information, such as a physical or email address, without their explicit permission
* Other conduct which could reasonably be considered inappropriate in a professional setting
## Enforcement Responsibilities
Community leaders are responsible for clarifying and enforcing our standards of acceptable behavior and will take appropriate and fair corrective action in response to any behavior that they deem inappropriate, threatening, offensive, or harmful.
Community leaders have the right and responsibility to remove, edit, or reject comments, commits, code, wiki edits, issues, and other contributions that are not aligned to this Code of Conduct, and will communicate reasons for moderation decisions when appropriate.
## Scope
This Code of Conduct applies within all community spaces, and also applies when an individual is officially representing the community in public spaces. Examples of representing our community include using an official e-mail address, posting via an official social media account, or acting as an appointed representative at an online or offline event.
## Enforcement
Instances of abusive, harassing, or otherwise unacceptable behavior may be reported privately to the project maintainer via GitHub at https://github.com/MattJackson. All complaints will be reviewed and investigated promptly and fairly.
All community leaders are obligated to respect the privacy and security of the reporter of any incident.
## Enforcement Guidelines
Community leaders will follow these Community Impact Guidelines in determining the consequences for any action they deem in violation of this Code of Conduct:
### 1. Correction
**Community Impact**: Use of inappropriate language or other behavior deemed unprofessional or unwelcome in the community.
**Consequence**: A private, written warning from community leaders, providing clarity around the nature of the violation and an explanation of why the behavior was inappropriate. A public apology may be requested.
### 2. Warning
**Community Impact**: A violation through a single incident or series of actions.
**Consequence**: A warning with consequences for continued behavior. No interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, for a specified period of time. This includes avoiding interactions in community spaces as well as external channels like social media. Violating these terms may lead to a temporary or permanent ban.
### 3. Temporary Ban
**Community Impact**: A serious violation of community standards, including sustained inappropriate behavior.
**Consequence**: A temporary ban from any sort of interaction or public communication with the community for a specified period of time. No public or private interaction with the people involved, including unsolicited interaction with those enforcing the Code of Conduct, is allowed during this period. Violating these terms may lead to a permanent ban.
### 4. Permanent Ban
**Community Impact**: Demonstrating a pattern of violation of community standards, including sustained inappropriate behavior, harassment of an individual, or aggression toward or disparagement of classes of individuals.
**Consequence**: A permanent ban from any sort of public interaction within the community.
## Attribution
This Code of Conduct is adapted from the [Contributor Covenant][homepage], version 2.1, available at [https://www.contributor-covenant.org/version/2/1/code_of_conduct.html][v2.1].
Community Impact Guidelines were inspired by [Mozilla's code of conduct enforcement ladder][Mozilla CoC].
For answers to common questions about this code of conduct, see the FAQ at [https://www.contributor-covenant.org/faq][FAQ]. Translations are available at [https://www.contributor-covenant.org/translations][translations].
[homepage]: https://www.contributor-covenant.org
[v2.1]: https://www.contributor-covenant.org/version/2/1/code_of_conduct.html
[Mozilla CoC]: https://github.com/mozilla/diversity
[FAQ]: https://www.contributor-covenant.org/faq
[translations]: https://www.contributor-covenant.org/translations
+1 -1
View File
@@ -24,4 +24,4 @@ cargo test
## License
By contributing, you agree your code will be licensed under AGPL-3.0.
By contributing, you agree your code will be licensed under MIT.
Generated
+850
View File
@@ -0,0 +1,850 @@
# This file is automatically @generated by Cargo.
# It is not intended for manual editing.
version = 4
[[package]]
name = "aes"
version = "0.9.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8eb277bec05f56a0e0591f155a484cbd0f4f07ff2905051a48c72f004f7ed58"
dependencies = [
"cipher",
"cpubits",
"cpufeatures",
]
[[package]]
name = "aho-corasick"
version = "1.1.5"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c982642fa9e8606056828ee9a8505737230110bb1099153c79efe865c59d12ba"
dependencies = [
"memchr",
]
[[package]]
name = "autocfg"
version = "1.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f2032f911046de80f0a198e0901378627c33f59ea0ac00e363d481118bd70a53"
[[package]]
name = "base64"
version = "0.23.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ac07cdecf99051d9a5238b80f35af32cdeba5b336e55d957b318b50137e18da5"
[[package]]
name = "bitflags"
version = "2.13.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b588b76d00fde79687d7646a9b5bdf3cc0f655e0bbd080335a95d7e96f3587da"
[[package]]
name = "block-buffer"
version = "0.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d2f6c7dbe95a6ed67ad9f18e57daf93a2f034c524b99fd2b76d18fdfeb6660aa"
dependencies = [
"hybrid-array",
]
[[package]]
name = "bumpalo"
version = "3.20.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "72f5acc6cb2ba439de613abc23857ec3d78374d8ed5ac84e9d11336e87da8649"
[[package]]
name = "cc"
version = "1.4.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "0ad534f4357a5264cce5019c989cf66a4f0dc4e0d1b1d15f8aacec0ff7360273"
dependencies = [
"find-msvc-tools",
"shlex",
]
[[package]]
name = "cfg-if"
version = "1.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9330f8b2ff13f34540b44e946ef35111825727b38d33286ef986142615121801"
[[package]]
name = "chacha20"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "65c35e4b699c7e15ccbe7ee35c005e4fc0a278d22238a2857e6ce2dadeda1b06"
dependencies = [
"cfg-if",
"cpufeatures",
"rand_core",
]
[[package]]
name = "cipher"
version = "0.5.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e8cf2a2c93cd704877c0858356ed03480ff301ee950b43f1cbe4573b088bfa6c"
dependencies = [
"crypto-common",
"inout",
]
[[package]]
name = "const-oid"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a6ef517f0926dd24a1582492c791b6a4818a4d94e789a334894aa15b0d12f55c"
[[package]]
name = "cpubits"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "15b85f9c39137c3a891689859392b1bd49812121d0d61c9caf00d46ed5ce06ae"
[[package]]
name = "cpufeatures"
version = "0.3.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5ca28b0ae3115b884660db4118d803791fd6756b6e88f39c0f3f7859060d7566"
dependencies = [
"libc",
]
[[package]]
name = "crc32fast"
version = "1.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8498c871161e1742aaa9d52551b2d6ebdd4c3d45a3be423e3728f33b955be550"
dependencies = [
"cfg-if",
]
[[package]]
name = "crossbeam-channel"
version = "0.5.16"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d85363c37faeca707aef026efa9f3b34d077bce547e48f770770625c6013679e"
dependencies = [
"crossbeam-utils",
]
[[package]]
name = "crossbeam-deque"
version = "0.8.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "5181e0de7b61eb03a81e347d6dd8797bae9da5146707b51077e2d71a54ec0ceb"
dependencies = [
"crossbeam-epoch",
"crossbeam-utils",
]
[[package]]
name = "crossbeam-epoch"
version = "0.9.20"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2d6914041f254d6e9176c01941b21115dcfb7089e55135a35411081bd106ef3f"
dependencies = [
"crossbeam-utils",
]
[[package]]
name = "crossbeam-utils"
version = "0.8.22"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "61803da095bee82a81bb1a452ecc25d3b2f1416d1897eb86430c6159ef717c17"
[[package]]
name = "crypto-common"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ce6e4c961d6cd6c9a86db418387425e8bdeaf05b3c8bc1411e6dca4c252f1453"
dependencies = [
"hybrid-array",
]
[[package]]
name = "digest"
version = "0.11.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f1dd6dbb5841937940781866fa1281a1ff7bd3bf827091440879f9994983d5c2"
dependencies = [
"block-buffer",
"const-oid",
"crypto-common",
]
[[package]]
name = "either"
version = "1.18.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "252afb9ae5eaa683babdc6a068b3f5726eb19e05070c731f9b2a23a7c3e8ed34"
[[package]]
name = "equivalent"
version = "1.0.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "877a4ace8713b0bcf2a4e7eec82529c029f1d0619886d18145fea96c3ffe5c0f"
[[package]]
name = "errno"
version = "0.3.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "39cab71617ae0d63f51a36d69f866391735b51691dbda63cf6f96d042b63efeb"
dependencies = [
"libc",
"windows-sys",
]
[[package]]
name = "fastrand"
version = "2.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "da7c62ceae207dd37ea5b845da6a0696c799f85e97da1ab5b7910be3c1c80223"
[[package]]
name = "find-msvc-tools"
version = "0.1.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d45db016d36b838f563236e9193d0ee6ce38f3f68b6c94e914b4929c96bbb890"
[[package]]
name = "flate2"
version = "1.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6e634e2e0ebac1ee034020da1ca582e17ffe4e0f5e985823721e168928136dcb"
dependencies = [
"zlib-rs",
]
[[package]]
name = "freemkv-unlock"
version = "1.7.1"
dependencies = [
"aes",
"base64",
"num-bigint",
"num-traits",
"rand",
"serde",
"serde_json",
"sha1",
"sha2",
"tracing",
"zeroize",
]
[[package]]
name = "generator"
version = "0.8.9"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b3b854b0e584ead1a33f18b2fcad7cf7be18b3875c78816b753639aa501513ae"
dependencies = [
"cc",
"cfg-if",
"libc",
"log",
"rustversion",
"windows-link",
"windows-result",
]
[[package]]
name = "getrandom"
version = "0.4.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "300e883d756b2e4ec94e02791f39b04b522276138852cfc41d9fb7e904106099"
dependencies = [
"cfg-if",
"libc",
"r-efi",
"rand_core",
]
[[package]]
name = "hashbrown"
version = "0.17.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ed5909b6e89a2db4456e54cd5f673791d7eca6732202bbf2a9cc504fe2f9b84a"
[[package]]
name = "hybrid-array"
version = "0.4.14"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "707114b52a152fa7bdb290cd7cd5912d9467273b6d74e21b8d81aca1f8533f6b"
dependencies = [
"typenum",
]
[[package]]
name = "indexmap"
version = "2.14.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "07aa2048142242915a31d35844fb311e0e53fcca590c3a0a40dcf1b841fa09eb"
dependencies = [
"equivalent",
"hashbrown",
]
[[package]]
name = "inout"
version = "0.2.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4250ce6452e92010fdf7268ccc5d14faa80bb12fc741938534c58f16804e03c7"
dependencies = [
"hybrid-array",
]
[[package]]
name = "itoa"
version = "1.0.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8f42a60cbdf9a97f5d2305f08a87dc4e09308d1276d28c869c684d7777685682"
[[package]]
name = "lazy_static"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "bbd2bcb4c963f2ddae06a2efc7e9f3591312473c50c6685e1f298068316e66fe"
[[package]]
name = "libc"
version = "0.2.189"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3eaf3ede3fee6db1a4c2ee091bf8a8b4dccdc6d17f656fb07896ee72867612f2"
[[package]]
name = "libfreemkv"
version = "1.7.1"
dependencies = [
"aes",
"base64",
"crossbeam-channel",
"freemkv-unlock",
"libc",
"loom",
"memchr",
"rayon",
"roxmltree",
"serde",
"serde_json",
"sha1",
"tempfile",
"tracing",
"zip",
]
[[package]]
name = "linux-raw-sys"
version = "0.12.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32a66949e030da00e8c7d4434b251670a91556f4144941d37452769c25d58a53"
[[package]]
name = "log"
version = "0.4.34"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f9f8bd3e56ce4dfc153cf470fffbfa98c7620958b312ca5c3a4b8d5181fd13c6"
[[package]]
name = "loom"
version = "0.7.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "419e0dc8046cb947daa77eb95ae174acfbddb7673b4151f56d1eed8e93fbfaca"
dependencies = [
"cfg-if",
"generator",
"scoped-tls",
"tracing",
"tracing-subscriber",
]
[[package]]
name = "matchers"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d1525a2a28c7f4fa0fc98bb91ae755d1e2d1505079e05539e35bc876b5d65ae9"
dependencies = [
"regex-automata",
]
[[package]]
name = "memchr"
version = "2.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf8baf1c55e62ffcace7a9f06f4bd9cd3f0c4beb022d3b367256b91b87513d98"
[[package]]
name = "nu-ansi-term"
version = "0.50.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7957b9740744892f114936ab4a57b3f487491bbeafaf8083688b16841a4240e5"
dependencies = [
"windows-sys",
]
[[package]]
name = "num-bigint"
version = "0.5.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "93e7820bc0a80a0238e650327316f929ba18d5be054b647490a3a6a339f3e7c0"
dependencies = [
"num-integer",
"num-traits",
]
[[package]]
name = "num-integer"
version = "0.1.47"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7ce2d95d4b3734dc35aa2f45e1aa22cd416814592a4f9d9205e11affd5b8e10b"
dependencies = [
"num-traits",
]
[[package]]
name = "num-traits"
version = "0.2.19"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "071dfc062690e90b734c0b2273ce72ad0ffa95f0c74596bc250dcfd960262841"
dependencies = [
"autocfg",
]
[[package]]
name = "once_cell"
version = "1.21.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "9f7c3e4beb33f85d45ae3e3a1792185706c8e16d043238c593331cc7cd313b50"
[[package]]
name = "pin-project-lite"
version = "0.2.17"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "a89322df9ebe1c1578d689c92318e070967d1042b512afbe49518723f4e6d5cd"
[[package]]
name = "proc-macro2"
version = "1.0.107"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "985e7ec9bb745e6ce6535b544d84d6cd6f7ad8bd711c398938ae983b91a766d9"
dependencies = [
"unicode-ident",
]
[[package]]
name = "quote"
version = "1.0.47"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1fbf4db142a473a8d80c26bbf18454ed458bf8d26c8219c331daecfdbd079001"
dependencies = [
"proc-macro2",
]
[[package]]
name = "r-efi"
version = "6.0.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8dcc9c7d52a811697d2151c701e0d08956f92b0e24136cf4cf27b57a6a0d9bf"
[[package]]
name = "rand"
version = "0.10.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c7f5fa3a058cd35567ef9bfa5e75732bee0f9e4c55fa90477bef2dfcdbc4be80"
dependencies = [
"chacha20",
"getrandom",
"rand_core",
]
[[package]]
name = "rand_core"
version = "0.10.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "63b8176103e19a2643978565ca18b50549f6101881c443590420e4dc998a3c69"
[[package]]
name = "rayon"
version = "1.12.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "fb39b166781f92d482534ef4b4b1b2568f42613b53e5b6c160e24cfbfa30926d"
dependencies = [
"either",
"rayon-core",
]
[[package]]
name = "rayon-core"
version = "1.13.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "22e18b0f0062d30d4230b2e85ff77fdfe4326feb054b9783a3460d8435c8ab91"
dependencies = [
"crossbeam-deque",
"crossbeam-utils",
]
[[package]]
name = "regex-automata"
version = "0.4.18"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ad8553b9b26413251cbf30e620595c7a41b3887f03da04579c0e6b0d6a06b4b2"
dependencies = [
"aho-corasick",
"memchr",
"regex-syntax",
]
[[package]]
name = "regex-syntax"
version = "0.8.11"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "d6f6ff9a378485b298a5286656da665ba74413d36db0979633275d2e708145d4"
[[package]]
name = "roxmltree"
version = "0.20.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "6c20b6793b5c2fa6553b250154b78d6d0db37e72700ae35fad9387a46f487c97"
[[package]]
name = "rustix"
version = "1.1.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6fe4565b9518b83ef4f91bb47ce29620ca828bd32cb7e408f0062e9930ba190"
dependencies = [
"bitflags",
"errno",
"libc",
"linux-raw-sys",
"windows-sys",
]
[[package]]
name = "rustversion"
version = "1.0.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cf54715a573b99ac80df0bc206da022bcd442c974952c7b9720069370852e21f"
[[package]]
name = "scoped-tls"
version = "1.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e1cf6437eb19a8f4a6cc0f7dca544973b0b78843adbfeb3683d1a94a0024a294"
[[package]]
name = "serde"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "4148590afebada386688f18773da617792bf2ef03ffc1e4cbd2b1d45b023e0ba"
dependencies = [
"serde_core",
"serde_derive",
]
[[package]]
name = "serde_core"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "67dca2c9c51e58a4791a4b1ed58308b39c64224d349a935ab5039aa360942a48"
dependencies = [
"serde_derive",
]
[[package]]
name = "serde_derive"
version = "1.0.229"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e7a5d71263a5a7d47b41f6b3f06ba276f10cc18b0931f1799f710578e2309348"
dependencies = [
"proc-macro2",
"quote",
"syn 3.0.4",
]
[[package]]
name = "serde_json"
version = "1.0.151"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "c841b55ecdae098c80dcae9cf767f6f8a0c2cdb3416bbef72181df4d0fe73f14"
dependencies = [
"itoa",
"memchr",
"serde",
"serde_core",
"zmij",
]
[[package]]
name = "sha1"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "aacc4cc499359472b4abe1bf11d0b12e688af9a805fa5e3016f9a386dc2d0214"
dependencies = [
"cfg-if",
"cpufeatures",
"digest",
]
[[package]]
name = "sha2"
version = "0.11.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "446ba717509524cb3f22f17ecc096f10f4822d76ab5c0b9822c5f9c284e825f4"
dependencies = [
"cfg-if",
"cpufeatures",
"digest",
]
[[package]]
name = "sharded-slab"
version = "0.1.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f40ca3c46823713e0d4209592e8d6e826aa57e928f09752619fc696c499637f6"
dependencies = [
"lazy_static",
]
[[package]]
name = "shlex"
version = "2.0.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f8fadd59c855ef2080decdef8ff161eb6661b86933c9d82e5ba29dc602a55aba"
[[package]]
name = "simd-adler32"
version = "0.3.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3a219298ac11a56ea9a6d2120044824d6f01aeb034955e7af7bc16858527deea"
[[package]]
name = "smallvec"
version = "1.16.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b9be42f50aa861c555654aa3a37f52f4b1074bacf4e48fe0ef7fa584e80f1f0f"
[[package]]
name = "syn"
version = "2.0.119"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "872831b642d1a07999a962a351ed35b955ea2cfc8f3862091e2a240a84f17297"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "syn"
version = "3.0.4"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6275cddf4610d1775e6d1fe9469b2e77d0f39fd98fb7450901b821e0c53649f"
dependencies = [
"proc-macro2",
"quote",
"unicode-ident",
]
[[package]]
name = "tempfile"
version = "3.27.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "32497e9a4c7b38532efcdebeef879707aa9f794296a4f0244f6f69e9bc8574bd"
dependencies = [
"fastrand",
"getrandom",
"once_cell",
"rustix",
"windows-sys",
]
[[package]]
name = "thread_local"
version = "1.1.10"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "1ad99c4c6d32803332c548b1af0540b357b3f5fc0be8f6c6bfe8b2e6ae784070"
dependencies = [
"cfg-if",
]
[[package]]
name = "tracing"
version = "0.1.44"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "63e71662fa4b2a2c3a26f570f037eb95bb1f85397f3cd8076caed2f026a6d100"
dependencies = [
"pin-project-lite",
"tracing-attributes",
"tracing-core",
]
[[package]]
name = "tracing-attributes"
version = "0.1.31"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7490cfa5ec963746568740651ac6781f701c9c5ea257c58e057f3ba8cf69e8da"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.119",
]
[[package]]
name = "tracing-core"
version = "0.1.36"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "db97caf9d906fbde555dd62fa95ddba9eecfd14cb388e4f491a66d74cd5fb79a"
dependencies = [
"once_cell",
"valuable",
]
[[package]]
name = "tracing-log"
version = "0.2.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ee855f1f400bd0e5c02d150ae5de3840039a3f54b025156404e34c23c03f47c3"
dependencies = [
"log",
"once_cell",
"tracing-core",
]
[[package]]
name = "tracing-subscriber"
version = "0.3.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "cb7f578e5945fb242538965c2d0b04418d38ec25c79d160cd279bf0731c8d319"
dependencies = [
"matchers",
"nu-ansi-term",
"once_cell",
"regex-automata",
"sharded-slab",
"smallvec",
"thread_local",
"tracing",
"tracing-core",
"tracing-log",
]
[[package]]
name = "typed-path"
version = "0.12.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "8e28f89b80c87b8fb0cf04ab448d5dd0dd0ade2f8891bae878de66a75a28600e"
[[package]]
name = "typenum"
version = "1.20.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "b6f5e870be6c3b371b77fe0ee0bafb859fa4964b4404c27de1d380043c4dda20"
[[package]]
name = "unicode-ident"
version = "1.0.24"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e6e4313cd5fcd3dad5cafa179702e2b244f760991f45397d14d4ebf38247da75"
[[package]]
name = "valuable"
version = "0.1.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ba73ea9cf16a25df0c8caa16c51acb937d5712a8429db78a3ee29d5dcacd3a65"
[[package]]
name = "windows-link"
version = "0.2.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f0805222e57f7521d6a62e36fa9163bc891acd422f971defe97d64e70d0a4fe5"
[[package]]
name = "windows-result"
version = "0.4.1"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "7781fa89eaf60850ac3d2da7af8e5242a5ea78d1a11c49bf2910bb5a73853eb5"
dependencies = [
"windows-link",
]
[[package]]
name = "windows-sys"
version = "0.61.2"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "ae137229bcbd6cdf0f7b80a31df61766145077ddf49416a728b02cb3921ff3fc"
dependencies = [
"windows-link",
]
[[package]]
name = "zeroize"
version = "1.9.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "e13c156562582aa81c60cb29407084cdb54c4164760106ab78e6c5b0858cf64e"
dependencies = [
"zeroize_derive",
]
[[package]]
name = "zeroize_derive"
version = "1.5.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "3c50655cbb0fe3fc43170059e702f1ce5e19b84cec58dc87b037a09935c2f328"
dependencies = [
"proc-macro2",
"quote",
"syn 2.0.119",
]
[[package]]
name = "zip"
version = "8.6.0"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "2d04a6b5381502aa6087c94c669499eb1602eb9c5e8198e534de571f7154809b"
dependencies = [
"crc32fast",
"flate2",
"indexmap",
"memchr",
"typed-path",
"zopfli",
]
[[package]]
name = "zlib-rs"
version = "0.6.7"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "34b31d188d9d685a4f9c7b46d6e36631b07058d2cfe190267adce54dc230bf12"
[[package]]
name = "zmij"
version = "1.0.23"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "29666d0abbfad1e3dc4dcf6144730dd3a3ab225bbbdac83319345b1b44ccfc1b"
[[package]]
name = "zopfli"
version = "0.8.3"
source = "registry+https://github.com/rust-lang/crates.io-index"
checksum = "f05cd8797d63865425ff89b5c4a48804f35ba0ce8d125800027ad6017d2b5249"
dependencies = [
"bumpalo",
"crc32fast",
"log",
"simd-adler32",
]
[[patch.unused]]
name = "freemkv-engine"
version = "1.7.1"
[[patch.unused]]
name = "freemkv-i18n"
version = "1.7.1"
[[patch.unused]]
name = "freemkv-keysources"
version = "1.7.1"
+96 -24
View File
@@ -1,51 +1,95 @@
[package]
name = "libfreemkv"
version = "0.30.7"
version = "1.7.3"
edition = "2024"
rust-version = "1.86"
license = "AGPL-3.0-only"
rust-version = "1.88"
license = "MIT"
description = "Open source raw disc access library for optical drives"
repository = "https://github.com/freemkv/libfreemkv"
keywords = ["bluray", "uhd", "optical", "scsi", "disc"]
categories = ["hardware-support", "multimedia"]
# Keep internal AI-instruction / private notes out of the published crate.
exclude = []
exclude = ["CLAUDE.md"]
# OFF crates.io: libfreemkv git-deps freemkv-unlock (firmware, never published),
# so libfreemkv itself can only be consumed by git tag. Clients git-tag-pin it.
publish = false
[profile.release]
lto = "thin"
codegen-units = 1
# ── Feature layout ───────────────────────────────────────────────────────────
# `scsi` — the SCSI passthrough transport layer ONLY (`src/scsi/` + `error`):
# ScsiTransport/execute, ScsiResult/ScsiSense/parse_sense, DriveInfo,
# list_drives, drive_has_disc, CDB builders, the per-OS backends and
# the raw-`cc` macOS shim (built by build.rs). Pulls only `tracing`
# (transport instrumentation) plus the always-on `libc` target dep —
# none of the ripping dependency tree.
# `rip` — the full ripping tree (drive lifecycle, disc/UDF/MPLS/CLPI parsing,
# AACS/CSS decrypt, PES/mux pipeline, keydb, freemkv-unlock seam). All
# heavy deps (serde, zip, rayon, roxmltree, rand, crossbeam, memchr,
# sha1, aes, base64, freemkv-unlock, serde_json) live here.
# `default = ["rip", "scsi"]` preserves today's full build byte-for-byte;
# consumers wanting transport only use `default-features = false, features =
# ["scsi"]`. `rip` implies `scsi` because the ripping tree reads through it.
[features]
default = ["rip", "scsi"]
scsi = ["dep:tracing"]
rip = [
"scsi",
"dep:serde",
"dep:serde_json",
"dep:sha1",
"dep:aes",
"dep:freemkv-unlock",
"dep:zip",
"dep:base64",
"dep:roxmltree",
"dep:crossbeam-channel",
"dep:rayon",
"dep:memchr",
]
[dependencies]
serde = { version = "1", features = ["derive"] }
serde_json = "1"
sha1 = "0.10"
sha2 = "0.10"
aes = "0.8"
cbc = "0.1"
flate2 = "1"
num-bigint = "0.4"
num-traits = "0.2"
num-integer = "0.1"
rand = "0.8"
cmac = "0.7"
zip = { version = "2", default-features = false, features = ["deflate"] }
base64 = "0.22.1"
# Trace-level instrumentation for Disc::copy + SgIoTransport::execute. Permitted
# under project docs ("Acceptable strings: debug/trace logging"). Consumers (autorip)
serde = { version = "1", features = ["derive"], optional = true }
serde_json = { version = "1", optional = true }
sha1 = { version = "0.11", optional = true }
aes = { version = "0.9", optional = true }
# Interim path dep for local cross-repo dev; `release-orchestrate.yml` re-pins
# this to `{ git = ".../freemkv-unlock", tag = "vX.Y.Z" }` in the tagged commit
# (so the released tag resolves freemkv-unlock from git, not a sibling path) and
# restores the path dep on the branch tip.
freemkv-unlock = { git = "https://github.com/freemkv/freemkv-unlock", tag = "v1.7.3", optional = true }
zip = { version = "8", default-features = false, features = ["deflate"], optional = true }
base64 = { version = "0.23", optional = true }
# Read-only XML DOM parser (pure Rust, forbid(unsafe_code), entity-expansion
# bounded). Parses the HD-DVD Advanced-Content playlist `ADV_OBJ/VPLST000.XPL`
# — untrusted disc bytes — into authoritative titles/clips/chapters. A real
# parser, not a hand-rolled scanner: the XPL is genuine XML (comments, varied
# attribute order, self-closing tags).
roxmltree = { version = "0.20", optional = true }
# Trace-level instrumentation for the read/transport path (SgIoTransport::execute
# and friends). Part of the `scsi` feature — the transport layer traces here.
# Permitted
# under CLAUDE.md ("Acceptable strings: debug/trace logging"). Consumers (autorip)
# wire a tracing subscriber and pipe events into the JSONL debug log.
tracing = "0.1"
tracing = { version = "0.1", optional = true }
# Bounded MPSC channel with kernel-wakeup send_timeout. Used by `io::pipeline`
# so the halt-aware send/finish loops can BLOCK on consumer drain instead of
# polling — the 50 ms poll cadence of the previous mpsc-based impl capped mux
# throughput at ~1 MB/s (0.21.7).
crossbeam-channel = "0.5"
crossbeam-channel = { version = "0.5", optional = true }
# Persistent work-stealing thread pool for parallel AACS unit
# decryption. Per-call std::thread::scope spawned fresh OS threads
# and that overhead dominated for typical batch sizes (60 units).
# rayon's global pool initialises once on first use.
rayon = "1"
rayon = { version = "1", optional = true }
# SIMD-accelerated bytestring search. Drives the HEVC/H.264 start-code
# scan in `mux::codec::h264::find_start_code` — naive byte-by-byte
# walk is ~500 MB/s single-thread on x86_64; memchr's vectorised
# `memmem::find` for the 3-byte `00 00 01` needle hits ~5 GB/s on
# AVX2-capable hosts.
memchr = "2"
memchr = { version = "2", optional = true }
[target.'cfg(target_os = "linux")'.dependencies]
libc = "0.2"
@@ -56,7 +100,35 @@ libc = "0.2"
[dev-dependencies]
tempfile = "3"
# Permutation model checker for the `scsi::fd_handoff` atomics — the fd
# hand-off between a Linux recovery thread and `SgIoTransport::drop`. A normal
# test cannot see a missing release edge there: the orderings compile to the
# same instructions on x86_64, so the wrong ones pass every time. loom
# enumerates the executions the C++20 model permits and reports the leaked
# descriptor directly.
#
# Behind `cfg(loom)`, which nothing sets unless you ask for it, so this is out
# of the graph for every ordinary build and for every CI job but the dedicated
# `loom` one. Run it with:
#
# RUSTFLAGS="--cfg loom" cargo test --lib --no-default-features \
# --features scsi fd_handoff
#
# `cargo test --lib` specifically: as a DEV-dependency loom is only in scope for
# test targets, which is why `fd_handoff` gates its import on `all(loom, test)`
# and falls back to std elsewhere. Without that, `--cfg loom` would break plain
# `cargo build`/`clippy` with an unresolved-crate error.
[target.'cfg(loom)'.dev-dependencies]
loom = "0.7"
[[bench]]
name = "sgio_read"
harness = false
[lints.rust]
# `loom` is set by RUSTFLAGS for the model checker above, never by a feature.
unexpected_cfgs = { level = "warn", check-cfg = ['cfg(loom)'] }
[lints.clippy]
uninlined_format_args = "allow"
+17 -12
View File
@@ -1,16 +1,21 @@
GNU AFFERO GENERAL PUBLIC LICENSE
Version 3, 19 November 2007
MIT License
Copyright (C) 2026 FreeMKV Contributors
Copyright (c) 2026 Matthew Jackson & Contributors
This program is free software: you can redistribute it and/or modify
it under the terms of the GNU Affero General Public License as published
by the Free Software Foundation, version 3 of the License.
Permission is hereby granted, free of charge, to any person obtaining a copy
of this software and associated documentation files (the "Software"), to deal
in the Software without restriction, including without limitation the rights
to use, copy, modify, merge, publish, distribute, sublicense, and/or sell
copies of the Software, and to permit persons to whom the Software is
furnished to do so, subject to the following conditions:
This program is distributed in the hope that it will be useful,
but WITHOUT ANY WARRANTY; without even the implied warranty of
MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the
GNU Affero General Public License for more details.
The above copyright notice and this permission notice shall be included in all
copies or substantial portions of the Software.
You should have received a copy of the GNU Affero General Public License
along with this program. If not, see <https://www.gnu.org/licenses/>.
THE SOFTWARE IS PROVIDED "AS IS", WITHOUT WARRANTY OF ANY KIND, EXPRESS OR
IMPLIED, INCLUDING BUT NOT LIMITED TO THE WARRANTIES OF MERCHANTABILITY,
FITNESS FOR A PARTICULAR PURPOSE AND NONINFRINGEMENT. IN NO EVENT SHALL THE
AUTHORS OR COPYRIGHT HOLDERS BE LIABLE FOR ANY CLAIM, DAMAGES OR OTHER
LIABILITY, WHETHER IN AN ACTION OF CONTRACT, TORT OR OTHERWISE, ARISING FROM,
OUT OF OR IN CONNECTION WITH THE SOFTWARE OR THE USE OR OTHER DEALINGS IN THE
SOFTWARE.
+8 -2
View File
@@ -1,7 +1,7 @@
# libfreemkv — local dev helper.
# Mirrors the workspace-wide CI checks but scoped to this single crate.
.PHONY: test build check ci clean
.PHONY: test build check loom ci clean
test:
cargo test --tests
@@ -13,7 +13,13 @@ check:
cargo fmt --check
cargo clippy --all-targets -- -D warnings
ci: check build test
# Model-check the scsi::fd_handoff atomics. `cargo test` cannot fail on those
# orderings — they compile to the same instructions on x86_64 — so this is the
# only check on them. Mirrors the `loom` CI job.
loom:
RUSTFLAGS="--cfg loom" cargo test --lib --no-default-features --features scsi fd_handoff
ci: check build test loom
clean:
cargo clean
+37 -68
View File
@@ -1,26 +1,30 @@
[![Crates.io](https://img.shields.io/crates/v/libfreemkv)](https://crates.io/crates/libfreemkv)
[![docs.rs](https://img.shields.io/docsrs/libfreemkv)](https://docs.rs/libfreemkv)
[![License: AGPL-3.0](https://img.shields.io/badge/license-AGPL--3.0-blue)](LICENSE)
[![Sponsor](https://img.shields.io/badge/Sponsor-%E2%9D%A4-ea4aaa?logo=github-sponsors)](https://github.com/sponsors/MattJackson)
[![License: MIT](https://img.shields.io/badge/license-MIT-blue)](LICENSE)
[![CI](https://github.com/freemkv/libfreemkv/actions/workflows/ci.yml/badge.svg)](https://github.com/freemkv/libfreemkv/actions/workflows/ci.yml)
[![codecov](https://codecov.io/gh/freemkv/libfreemkv/branch/dev/graph/badge.svg)](https://codecov.io/gh/freemkv/libfreemkv)
# libfreemkv
Rust library for 4K UHD / Blu-ray / DVD optical drives. Drive access, disc scanning, stream labels, AACS decryption, CSS decryption, KEYDB updates, and content reading in one crate. Bundled drive profiles — no external files needed.
Rust library for 4K UHD / Blu-ray / DVD optical drives. Drive access, disc scanning, stream labels, AACS decryption, CSS decryption, and content reading in one crate. Drive-level unlocking is handled internally; consumers work with disc access and decryption only.
Built-in keys cover DVDs and Blu-rays (AACS 1.0). For UHD (AACS 2.0 / 2.1) discs, an optional `keydb.cfg` supplies disc-specific volume unique keys.
DVDs (CSS) decrypt out of the box. Blu-ray and UHD (AACS) require disc-specific volume unique keys, supplied by the consumer (e.g. via [freemkv-keysources](https://github.com/freemkv/freemkv-keysources), which owns `keydb.cfg` lookup/download); libfreemkv itself never reads `keydb.cfg` or downloads keys, and no AACS key material is compiled in.
**12+ MB/s** sustained read speeds on BD. Full init: unlock, firmware upload, speed calibration — all from pure Rust.
**12+ MB/s** sustained read speeds on BD. Drive prep (`init()`) handles unlocking internally via the `freemkv-unlock` crate — clients never see it; when no drive unlock applies, the library rips via the host-certificate AACS handshake.
Multi-lingual by design — the library outputs structured data and numeric error codes, never English text. Build any UI or localization on top.
**[API Documentation](https://docs.rs/libfreemkv)** · **[Technical Docs](docs/)**
**[Source & API](https://github.com/freemkv/libfreemkv)** · **[Technical Docs](docs/)**
Part of the [freemkv](https://github.com/freemkv) project.
## Install
Consumed by git tag (not published to crates.io):
```toml
[dependencies]
libfreemkv = "0.25"
libfreemkv = { git = "https://github.com/freemkv/libfreemkv", tag = "vX.Y.Z" }
```
## Quick Start
@@ -29,10 +33,10 @@ libfreemkv = "0.25"
use libfreemkv::{Drive, Disc, ScanOptions};
use std::path::Path;
// Open drive — profiles are bundled, auto-identified
// Open drive — identified via INQUIRY
let mut drive = Drive::open(Path::new("/dev/sg4"))?;
drive.wait_ready()?; // wait for disc
drive.init()?; // unlock + firmware upload
drive.init()?; // unlock + prep (handled internally)
drive.probe_disc()?; // probe disc surface for optimal speeds
// Scan disc — UDF, playlists, streams, AACS (all automatic)
@@ -55,57 +59,23 @@ output.finish()?;
### Multi-pass recovery rip
For damaged discs the library exposes two flat verbs — `Disc::sweep` for the
forward Pass 1 and `Disc::patch` for retrying bad ranges. The library never
loops; the multipass policy is the caller's job. See
[`docs/rip-recovery.md`](docs/rip-recovery.md).
Recovery moved OUT of this crate in 1.6.0. The sweep/patch strategy, the
ddrescue mapfile, damage classification and the multipass loop now live in the
`freemkv-engine` crate as `freemkv_engine::recovery::{copy, sweep, patch}`.
```rust
use libfreemkv::{SweepOptions, PatchOptions};
use libfreemkv::disc::{mapfile, mapfile_path_for};
use std::path::Path;
let iso = Path::new("disc.iso");
// Pass 1: disc → ISO. Skip-on-error, zero-fill, write the sidecar mapfile.
disc.sweep(&mut drive, iso, &SweepOptions {
decrypt: true,
resume: false,
batch_sectors: None,
skip_on_error: true,
progress: None,
halt: None,
})?;
// Pass 2..N: retry every non-finished range. Idempotent.
loop {
let map = mapfile::Mapfile::load(&mapfile_path_for(iso))?;
let stats = map.stats();
if stats.bytes_pending + stats.bytes_unreadable == 0 { break; }
let outcome = disc.patch(&mut drive, iso, &PatchOptions {
decrypt: true,
block_sectors: None,
full_recovery: true,
reverse: true,
wedged_threshold: 50,
progress: None,
halt: None,
})?;
if outcome.bytes_recovered_this_pass == 0 { break; }
}
// Mux from the ISO via the normal stream pipeline (no drive involvement).
```
libfreemkv keeps the layers underneath: the raw single-shot read
(`Drive::read`) and the SCSI-fact translation (`SenseFamily`) that the engine's
strategy is built on. The dependency runs engine → libfreemkv, so this crate
cannot call into it; front-ends get recovery from the engine directly. See
[`docs/rip-recovery.md`](docs/rip-recovery.md) for what stayed here.
## What It Does
- **Drive access** — open, identify, unlock, firmware upload, speed calibration, eject
- **Drive access** — open, identify, internal unlock + prep, speed control, eject
- **12+ MB/s reads** — auto-detects kernel transfer limits, sustained full speed
- **Disc scanning** — UDF 2.50 filesystem, MPLS playlists, CLPI clip info
- **Stream labels** — 5 BD-J format parsers (Paramount, Criterion, Pixelogic, CTRM, Deluxe)
- **Stream labels** — 7 BD-J format parsers (Paramount, Criterion, Pixelogic, CTRM, DBP, Deluxe, Fox)
- **AACS decryption** — transparent key resolution and content decrypt (1.0 + 2.0 bus decryption)
- **KEYDB updates** — download, verify, save from any HTTP URL (zero deps, raw TCP)
- **Content reading** — adaptive batch reads with automatic decryption
- **Stream I/O** — unified stream pipeline for reading and writing any format
@@ -114,42 +84,40 @@ loop {
| Stream | Input | Output | Transport |
|--------|-------|--------|-----------|
| DiscStream | Yes | -- | Optical drive via SCSI |
| IsoStream | Yes | -- | Blu-ray ISO image file (read via stream pipeline; written via `Disc::sweep()`) |
| IsoStream | Yes | -- | Blu-ray ISO image file (read via stream pipeline; written by `freemkv_engine::recovery`) |
| MkvStream | Yes | Yes | Matroska container |
| M2tsStream | Yes | Yes | BD transport stream with FMKV metadata header |
| NetworkStream | Yes (listen) | Yes (connect) | TCP with FMKV metadata header |
| StdioStream | Yes (stdin) | Yes (stdout) | Raw byte pipe |
| NullStream | -- | Yes | Discard sink (byte counter for benchmarks) |
Streams implement `FrameSource` (read) and/or `FrameSink` (write); direction is type-checked. `input()` / `output()` resolve URL strings to PES stream instances. All URLs use the `scheme://path` format — bare paths are rejected.
Streams implement a single unified `pes::Stream` trait (re-exported as `PesStream`) exposing `read()` and `write()` on one type. `input()` / `output()` resolve URL strings to PES stream instances. All URLs use the `scheme://path` format — bare paths are rejected.
### Keys
DVDs (CSS) decrypt out of the box — the 1999-era public player keys are compiled into the library.
DVDs (CSS) decrypt out of the box, with no external key file needed.
Blu-rays and UHD (AACS) require a `keydb.cfg` at `~/.config/freemkv/keydb.cfg` (or passed via `ScanOptions`). The file holds all DKs, PKs, host certs, and per-disc VUKs. No AACS key material is compiled into the binary.
Blu-rays and UHD (AACS) require Unit Keys passed in via `ScanOptions`/`KeySpec`. libfreemkv does not read `keydb.cfg` or fetch keys itself — that's the consumer's job (see [freemkv-keysources](https://github.com/freemkv/freemkv-keysources)). No AACS key material is compiled into the binary.
## Architecture
```text
Drive — open, identify, init, unlock, single-shot read
Drive — open, identify, init, single-shot read
├── ScsiTransport — SG_IO (Linux), IOKit (macOS), SPTI (Windows)
├── DriveProfile — per-drive unlock parameters (bundled)
└── PlatformDriver — MediaTek (supported), Renesas (planned)
└── unlock_bridge — private seam to the freemkv-unlock crate
(firmware / AACS cert / CSS bus-auth unlockers)
Disc — scan titles, streams, AACS/CSS state
├── UDF reader — Blu-ray UDF 2.50 with metadata partitions
├── MPLS parser — playlists → titles + clips + streams
├── CLPI parser — clip info → EP map → sector extents
├── IFO parser — DVD title sets, PGC chains, cell addresses
├── Labels — 5 BD-J format parsers (detect + parse)
├── Labels — 7 BD-J format parsers (detect + parse)
├── AACS — key resolution + content decryption
├── CSS — DVD CSS cipher (table-driven, no keys needed)
└── KEYDB — download + verify + save
└── CSS — DVD CSS (bus auth → player-key disc crack → known-plaintext title-key attack)
Streams — unified PES pipeline
├── FrameSource — read() PES frames (direction-typed)
├── FrameSink — write() PES frames (direction-typed)
├── PesStream — pes::Stream: one trait, read()/write() PES frames
├── DiscStream — sectors → decrypt → TS demux → PES
├── IsoStream — ISO file → decrypt → TS demux → PES
├── MkvStream — MKV mux/demux
@@ -175,6 +143,7 @@ All errors are structured with numeric codes. No user-facing English text — ap
| E6xxx | Disc format errors |
| E7xxx | AACS errors |
| E8xxx | KEYDB update errors |
| E9xxx | Stream / mux errors (URL, PES, ISO, pipeline, demux) |
## Platform Support
@@ -186,8 +155,8 @@ All errors are structured with numeric codes. No user-facing English text — ap
## Contributing
Run `freemkv info disc:// --share` with the [freemkv CLI](https://github.com/freemkv/freemkv) to contribute your drive's profile.
Run `freemkv info disc:// --share` with the [freemkv CLI](https://github.com/freemkv/freemkv) to capture your drive's identity for contribution. Drive-unlock profiles are maintained in the [freemkv-unlock](https://github.com/freemkv/freemkv-unlock) repository.
## License
AGPL-3.0-only
MIT
+22
View File
@@ -0,0 +1,22 @@
# Security Policy
## Supported versions
| Version | Supported |
| ------- | --------- |
| 1.6.x | Yes |
| < 1.6 | No |
Only the current 1.6.x line receives security fixes.
## Reporting a vulnerability
Report vulnerabilities privately through GitHub Security Advisories:
https://github.com/freemkv/libfreemkv/security/advisories/new
Do not open a public issue for a security report. Include the affected
version, steps to reproduce, and the impact you believe the issue has.
## Response time
You will get an initial response within 7 days.
+3 -3
View File
@@ -141,8 +141,8 @@ If your machine has a free SATA port, use it.
freemkv uses a three-layer recovery model. See [`docs/rip-recovery.md`](docs/rip-recovery.md) for full details.
- **Pass 1 (Disc::copy):** Fast sweep with 64 KB reads. On failure, zero-fills the block and skips forward. Writes a ddrescue-format mapfile for later retry.
- **Pass 2+ (Disc::patch):** Targeted re-reads of bad ranges with a long 30-second timeout per CDB. The drive firmware performs its own ECC and laser power retries within that window.
- **Pass 1 (`freemkv_engine::recovery::sweep`):** Fast sweep with 64 KB reads. On failure, zero-fills the block and skips forward. Writes a ddrescue-format mapfile for later retry.
- **Pass 2+ (`freemkv_engine::recovery::patch`):** Targeted re-reads of bad ranges with a long 60-second timeout per CDB. The drive firmware performs its own ECC and laser power retries within that window.
- **In-stream (DiscStream):** Adaptive batch halving -- reduces request size on failure to isolate bad sectors within a larger block.
This means a disc with some bad sectors will still produce a usable ISO. The damaged areas are zero-filled in pass 1 and retried in subsequent passes. Structure-protected sectors (deliberate unreadable regions from copy protection) will never yield, which is expected.
@@ -302,7 +302,7 @@ Many external drive enclosures (Vantec NexStar, Sabrent, OWC, etc.) do not adver
When something goes wrong during a rip, gather this information before filing an issue:
1. **freemkv version:** `freemkv --version` or the crate version in `Cargo.toml`.
2. **Drive model:** from the drive label, or from `freemkv info`.
2. **Drive model:** from the drive label, or from `freemkv info disc://`.
3. **Connection type:** USB (with bridge chipset if known) or direct SATA.
4. **Operating system and kernel:** `uname -a`.
5. **Kernel messages during the failure:** `dmesg | tail -50` immediately after the crash.
+73 -4
View File
@@ -1,15 +1,32 @@
fn main() {
emit_git_suffix();
let target = std::env::var("CARGO_CFG_TARGET_OS").unwrap_or_default();
if target == "macos" {
println!("cargo:rustc-link-lib=framework=IOKit");
println!("cargo:rustc-link-lib=framework=CoreFoundation");
println!("cargo:rustc-link-lib=framework=DiskArbitration");
let out_dir = std::env::var("OUT_DIR").unwrap();
let obj = format!("{out_dir}/macos_shim.o");
let lib = format!("{out_dir}/libmacos_scsi.a");
std::process::Command::new("cc")
// Build the shim for the TARGET arch, not the host's. A bare `cc` on an
// Apple-Silicon CI runner defaults to arm64, so cross-building to
// x86_64-apple-darwin would link a host-arch object against x86_64 Rust
// code → "Undefined symbols for architecture x86_64". (Still raw `cc`,
// not the `cc` crate, which breaks IOKit exclusive access.)
let target_arch = std::env::var("CARGO_CFG_TARGET_ARCH").unwrap_or_default();
let clang_arch: &str = if target_arch == "aarch64" {
"arm64"
} else {
&target_arch // x86_64 → x86_64
};
let cc_status = std::process::Command::new("cc")
.args([
"-arch",
clang_arch,
"-c",
"src/scsi/macos_shim.c",
"-o",
@@ -22,15 +39,67 @@ fn main() {
"-O2",
])
.status()
.expect("failed to compile macos_shim.c");
.expect("failed to spawn cc for macos_shim.c");
// `.status()` succeeding only means the process RAN. A real compile error
// exits non-zero, and ignoring that left no object file, which surfaced
// much later as an unexplained link failure against a missing symbol. The
// shim is macOS-only and is neither linted nor compiled on the other two
// platforms, so a mistake in it has exactly one chance to be noticed.
assert!(cc_status.success(), "cc failed to compile macos_shim.c");
std::process::Command::new("ar")
let ar_status = std::process::Command::new("ar")
.args(["rcs", &lib, &obj])
.status()
.expect("failed to create static lib");
.expect("failed to spawn ar");
assert!(ar_status.success(), "ar failed to create the static lib");
println!("cargo:rustc-link-search=native={out_dir}");
println!("cargo:rustc-link-lib=static=macos_scsi");
println!("cargo:rerun-if-changed=src/scsi/macos_shim.c");
}
}
/// Bake the git short hash into the build as `GIT_SUFFIX` so any muxed MKV or
/// FVI index is traceable to the exact source revision (e.g. ` (g835cc99)`).
/// Empty when git or the repo is unavailable (e.g. a crates.io tarball build),
/// leaving just the package version. Always emitted so `env!("GIT_SUFFIX")`
/// resolves on every target.
fn emit_git_suffix() {
// Version label for the muxing-app / FVI generator tag. `FREEMKV_BUILD_LABEL`
// overrides the Cargo package version when set (non-empty) — used to stamp a
// pre-release/test build without bumping Cargo.toml and disturbing the
// tag-pinned [patch] version matching. Unset → the package version.
let version = std::env::var("FREEMKV_BUILD_LABEL")
.ok()
.filter(|s| !s.trim().is_empty())
.or_else(|| std::env::var("CARGO_PKG_VERSION").ok())
.unwrap_or_default();
println!("cargo:rustc-env=FREEMKV_VERSION={version}");
println!("cargo:rerun-if-env-changed=FREEMKV_BUILD_LABEL");
let suffix = git_short_hash()
.map(|h| format!(" (g{h})"))
.unwrap_or_default();
println!("cargo:rustc-env=GIT_SUFFIX={suffix}");
// Re-run when HEAD (or the branch it points at) moves so the stamp stays
// current without a clean rebuild.
println!("cargo:rerun-if-changed=.git/HEAD");
if let Ok(head) = std::fs::read_to_string(".git/HEAD")
&& let Some(ref_path) = head.strip_prefix("ref: ")
{
println!("cargo:rerun-if-changed=.git/{}", ref_path.trim());
}
}
fn git_short_hash() -> Option<String> {
let out = std::process::Command::new("git")
.args(["rev-parse", "--short=7", "HEAD"])
.output()
.ok()?;
if !out.status.success() {
return None;
}
let h = String::from_utf8(out.stdout).ok()?.trim().to_string();
if h.is_empty() { None } else { Some(h) }
}
+252
View File
@@ -0,0 +1,252 @@
#!/usr/bin/env python3
"""comment-guard: cap explanatory prose by COMMENT INTENT, not visibility.
Three tiers, keyed on the syntax the author reached for:
* `//!` / `/*!` — a module (or crate) design doc. UNBOUNDED. There is one per
file, at the top, documenting the module as a whole, so it can't be used to
smuggle per-item rationale the way item docs can. Long correctness / design
essays (e.g. a memory-ordering proof) legitimately live here.
* `///` / `/** */` — item documentation. Up to DOC_CAP *prose* lines. Fenced
``` code blocks do NOT count, so real examples aren't punished. This holds
whether or not the item is `pub`: a `pub(crate)` fn deserves a real doc
paragraph just like a `pub` one. Capping `///` here (rather than exempting
it, as the old guard did — issue #17) is what stops rationale being hidden
behind a doc sigil, while still leaving room for a genuine paragraph.
* `//` / `/* */` — a quick internal aside. Up to INLINE_CAP lines. This is the
shape that sprawls into walls, so it's kept tight: stanzas separated by a
single blank line count CUMULATIVELY, so one essay can't be split into runs.
Overflow belongs in the module `//!`, or in docs/<topic>.md with a one-line `//`
pointer — both show up in review.
Scans src/ and tests/ under the given root (default cwd). `--selftest` runs
built-in cases. Exit non-zero if any block exceeds its cap.
"""
import re
import sys
from pathlib import Path
INLINE_CAP = 3
DOC_CAP = 8
_RAW_OPEN = re.compile(r'r(#*)"') # raw string opener: r"…", r#"…"#, br##"…"## …
def _strip_line_comment(s):
"""Return (kind, body) for a whole-line comment, else (None, None).
kind is 'doc' for `///`/`//!`, 'plain' for other `//`. A trailing comment on
a code line is not a whole-line comment and is ignored (single-line, cheap).
"""
if s.startswith("//!"):
return "doc", s[3:]
if s.startswith("///") and not s.startswith("////"):
return "doc", s[3:]
if s.startswith("//"):
return "plain", s[2:]
return None, None
def _fence_toggle(body):
"""True if this doc body line opens/closes a ``` fenced code block."""
return body.strip().startswith("```")
def _classify(lines):
"""Yield per-line dicts: comment kind ('doc'/'plain'/None), doc-ness of
block comments, and code info. Block comments are tracked across lines.
"""
info = []
in_block = False
block_is_doc = False
raw_hashes = None # None = not in a raw string; int N = open, needs '"' + N*'#'
for raw in lines:
s = raw.lstrip()
rec = {"raw": raw, "s": s, "comment": None, "is_doc": False, "code": False}
# Inside a raw string literal (e.g. an embedded HTML/CSS/JS asset): the
# bytes are string data, not Rust comments, so the guard is blind to them
# until the string closes. Prevents `/* */` or `//` lines in an embedded
# page from being counted as comment blocks.
if raw_hashes is not None:
if ('"' + "#" * raw_hashes) in raw:
raw_hashes = None
info.append(rec)
continue
if in_block:
rec["comment"] = "doc" if block_is_doc else "plain"
rec["is_doc"] = block_is_doc
if "*/" in raw:
in_block = False
info.append(rec)
continue
# A block comment only starts a comment-line if it's the first token.
if s.startswith("/*"):
rec["is_doc"] = s.startswith("/**") or s.startswith("/*!")
rec["comment"] = "doc" if rec["is_doc"] else "plain"
if "*/" not in s[2:]:
in_block = True
block_is_doc = rec["is_doc"]
info.append(rec)
continue
kind, _ = _strip_line_comment(s)
if kind is not None:
rec["comment"] = kind
rec["is_doc"] = kind == "doc"
info.append(rec)
continue
# Code (or blank). Blank = empty after strip.
rec["code"] = s != ""
# A code line may OPEN a raw string that spans following lines. Detected
# only on code (comments were handled above), so `// see r"x"` is safe.
m = _RAW_OPEN.search(raw)
if m and ('"' + "#" * len(m.group(1))) not in raw[m.end():]:
raw_hashes = len(m.group(1))
info.append(rec)
return info
def violations(path):
lines = path.read_text(encoding="utf-8", errors="replace").splitlines()
info = _classify(lines)
n = len(info)
out = []
i = 0
while i < n:
rec = info[i]
if rec["comment"] is None:
i += 1
continue
start = i
if rec["is_doc"]:
# Doc run: contiguous doc lines. `//!` / `/*!` module docs are
# UNBOUNDED; any other doc run caps PROSE (fenced ``` excluded) at
# DOC_CAP, regardless of the documented item's visibility.
j = i
prose = 0
in_fence = False
module_doc = False
while j < n and info[j]["comment"] == "doc":
s = info[j]["s"]
if s.startswith("//!") or s.startswith("/*!"):
module_doc = True
body = (
s[3:] if (s.startswith("///") or s.startswith("//!")) else s
)
if _fence_toggle(body):
in_fence = not in_fence
elif not in_fence and body.strip() != "":
prose += 1
j += 1
if not module_doc and prose > DOC_CAP:
out.append((start + 1, prose, DOC_CAP, "doc prose"))
i = j
continue
# Internal (plain) run, with single-blank bridging (cumulative).
j = i
count = 0
while j < n:
r = info[j]
if r["comment"] == "plain":
count += 1
j += 1
continue
if r["comment"] == "doc":
break
if r["comment"] is None and not r["code"]:
# blank: bridge only a SINGLE blank, else stop
if j + 1 < n and info[j + 1]["comment"] == "plain":
j += 1
continue
break
break # code
if count > INLINE_CAP:
out.append((start + 1, count, INLINE_CAP, "inline comment"))
i = j
return out
def main(argv):
if "--selftest" in argv:
return _selftest()
root = Path(argv[1]) if len(argv) > 1 else Path.cwd()
files = sorted(p for d in ("src", "tests") for p in (root / d).rglob("*.rs"))
total = 0
for f in files:
for start, length, cap, kind in violations(f):
rel = f.relative_to(root)
print(f"{rel}:{start}: {kind} block is {length} lines (max {cap})")
total += 1
if total:
print(f"\ncomment-guard: {total} comment block(s) exceed their cap.")
print(
f"Inline // asides cap at {INLINE_CAP}; /// item-doc prose caps at "
f"{DOC_CAP} (fenced examples excluded); //! module docs are unbounded. "
"Trim, move the rationale into the module //!, or docs/<topic>.md "
"with a // pointer."
)
return 1
return 0
def _selftest():
import tempfile
cases = [
# (rust source, expected number of violations)
# short /// doc -> ok
("/// short doc\npub fn a() {}\n", 0),
# /// on a PRIVATE item now gets the doc cap (8), so 4 lines is fine
("/// l1\n/// l2\n/// l3\n/// l4\nfn priv_a() {}\n", 0),
# ...but /// is NOT unlimited: 9 prose lines on a private item fails
("".join(f"/// l{k}\n" for k in range(9)) + "fn priv_b() {}\n", 1),
# 8 prose /// on a pub item -> ok (boundary)
("".join(f"/// l{k}\n" for k in range(8)) + "pub fn c() {}\n", 0),
# 9 prose /// on a pub item -> violation
("".join(f"/// l{k}\n" for k in range(9)) + "pub fn d() {}\n", 1),
# /// with a long fenced example -> example excluded, ok
(
"/// summary\n/// # Examples\n/// ```\n"
+ "".join("/// code\n" for _ in range(12))
+ "/// ```\npub fn e() {}\n",
0,
),
# pub(crate) gets the SAME doc cap as pub (no pub/private split anymore)
("".join(f"/// l{k}\n" for k in range(6)) + "pub(crate) fn f() {}\n", 0),
# bare inline 4 lines -> violation
("// a\n// b\n// c\n// d\nlet x = 1;\n", 1),
# blank-split dodge: 3 + blank + 3 inline -> cumulative -> violation
("// a\n// b\n// c\n\n// d\n// e\n// f\nfn z() {}\n", 1),
# plain /* */ block, 5 lines -> inline cap -> violation
("/* one\n two\n three\n four\n five */\nfn q() {}\n", 1),
# comment-like lines INSIDE a raw string are string data, not comments
(
'const H: &str = r##"<style>\n'
+ "".join("/* css note */\n" for _ in range(8))
+ '</style>"##;\nfn h() {}\n',
0,
),
# //! module doc may run long -> EXEMPT (no cap)
("".join(f"//! l{k}\n" for k in range(40)) + "\npub fn m() {}\n", 0),
# /*! block module doc may run long -> EXEMPT
("/*! l0\n" + "".join(f" l{k}\n" for k in range(30)) + "*/\nfn n() {}\n", 0),
]
ok = True
with tempfile.TemporaryDirectory() as d:
for idx, (src, want) in enumerate(cases):
p = Path(d) / f"case{idx}.rs"
p.write_text(src)
got = len(violations(p))
flag = "ok" if got == want else "FAIL"
if got != want:
ok = False
print(f" selftest {idx}: want {want} got {got} [{flag}]")
print("selftest PASSED" if ok else "selftest FAILED")
return 0 if ok else 1
if __name__ == "__main__":
sys.exit(main(sys.argv))
+120
View File
@@ -0,0 +1,120 @@
#!/usr/bin/env bash
#
# leak-guard.sh — self-contained public-repo leak gate.
#
# This is the LAST line of defense in CI. It is intentionally self-contained:
# public CI cannot reach the private tooling, so this script encodes ONLY the
# generic net — internal infrastructure references, agent-context files, and
# AI-attribution in commit messages. It deliberately contains NO project-
# specific reverse-engineering vocabulary (those words would themselves be a
# leak). The richer private scanner stays private.
#
# Fails (exit 1) if any of the following appear in the repo:
# 1. a tracked CLAUDE.md or .claude/ path (agent context — never public),
# 2. tracked file content matching the internal-infra net,
# 3. a commit message (in the given range) with AI attribution.
#
# Usage:
# leak-guard.sh [<commit-range>]
# <commit-range> optional git rev-list range to scan commit messages
# (e.g. "abc..def"). If omitted, commit-message scan is
# skipped (path + content checks always run).
set -euo pipefail
# Absolute path to this script, resolved before any cd, so we can exclude it
# from the content scan (it necessarily contains the detection patterns).
SELF_ABS="$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/$(basename "${BASH_SOURCE[0]}")"
REPO="$(git rev-parse --show-toplevel)"
cd "$REPO"
fail=0
note() { printf ' ✗ %s\n' "$1"; fail=1; }
# Internal-infra net — GENERIC ONLY. This script ships in the public repo, so
# the patterns themselves must not name any org-specific identifier (doing so
# would itself leak the infra they guard). We catch the leak *class*:
# - RFC1918 private IPv4 ranges (10/8, 172.16/12, 192.168/16),
# - private/internal/non-routable TLDs (.internal/.local/.lan/.corp/.invalid),
# - docker.internal.
# The full org-specific net (literal hostnames, service names, repo paths,
# vendor tooling, …) lives ONLY in the private scanner and never ships here.
INFRA_RE='\b10\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}|\b172\.(1[6-9]|2[0-9]|3[01])\.[0-9]{1,3}\.[0-9]{1,3}|\b192\.168\.[0-9]{1,3}\.[0-9]{1,3}|\.internal\b|\.local\b|\.lan\b|\.corp\b|\.invalid\b|docker\.internal'
# Home-path net — GENERIC ONLY. Catches an absolute developer home path
# committed into a tracked file (a macOS /Users/<user>/… or Linux /home/<user>/…
# path). This names NO specific user — it matches the leak *class* (any home
# path), so the pattern itself reveals nothing org- or person-specific. A real
# leak (e.g. /Users/alice/Developer/x slipping into a public RELEASE.md) trips
# this regardless of whose machine it came from. The username segment is a
# literal-username class ([A-Za-z0-9._-]) so dynamic/templated paths that build
# the user at runtime — shell `/home/$USER/`, doc `/home/<rip>/`, Rust
# `/home/{user}/` — do NOT false-positive; only a baked-in literal home leaks.
HOMEPATH_RE='/Users/[A-Za-z0-9._-]+/|/home/[A-Za-z0-9._-]+/'
# AI-attribution net (case-insensitive). "claude" matches only as a standalone
# word — NOT preceded by a dot/slash/alnum and NOT followed by .md — so legit
# mentions of CLAUDE.md / .claude/ in a commit message don't false-positive.
ATTR_RE='co-authored-by|generated with|🤖|(?<![.\/A-Za-z0-9])claude(?!\.md)'
echo "── leak-guard: tracked agent-context paths ──"
while IFS= read -r f; do
case "$f" in
CLAUDE.md|*/CLAUDE.md|.claude|.claude/*|*/.claude|*/.claude/*)
note "tracked agent-context file: $f (CLAUDE.md/.claude must never be tracked in a public repo)" ;;
esac
done < <(git ls-files)
# Match a PCRE against a file, emitting "LINE: MATCH". The pattern is passed as
# an argument (not interpolated into a //) so metacharacters like the "/" in a
# path-style token can't break the regex. Reads raw bytes so non-UTF-8 blobs
# don't abort the scan.
pcre_matches() {
perl -e '
my ($file, $re) = @ARGV;
open(my $fh, "<:raw", $file) or exit 0;
my $rx; eval { $rx = qr/$re/i }; exit 0 if $@;
while (my $l = <$fh>) { if ($l =~ /$rx/) { print "$.: $&\n"; } }
' "$1" "$2" 2>/dev/null
}
# This script's own source necessarily contains the detection patterns (e.g.
# the regex tokens in INFRA_RE), so scanning it would always self-flag. Skip it.
SELF="$(git ls-files --full-name -- "$SELF_ABS" 2>/dev/null | head -1)"
echo "── leak-guard: internal-infra references in tracked files ──"
while IFS= read -r f; do
case "$f" in *.png|*.jpg|*.jpeg|*.ico|*.gif|*.bin|*.crate|*.gz|*.zip|*.pdf) continue ;; esac
[ -n "$SELF" ] && [ "$f" = "$SELF" ] && continue
[ -f "$f" ] || continue
while IFS= read -r hit; do
[ -z "$hit" ] && continue
note "internal-infra reference: $f:$hit"
done < <(pcre_matches "$f" "$INFRA_RE")
while IFS= read -r hit; do
[ -z "$hit" ] && continue
note "[HOME-PATH] absolute home path: $f:$hit (no local home path may be committed to a public repo)"
done < <(pcre_matches "$f" "$HOMEPATH_RE")
done < <(git ls-files)
RANGE="${1:-}"
if [ -n "$RANGE" ]; then
echo "── leak-guard: AI-attribution in commit messages ($RANGE) ──"
while IFS= read -r sha; do
[ -z "$sha" ] && continue
msg="$(git log -1 --format='%B' "$sha" 2>/dev/null || true)"
# Pass the pattern as an argument (not interpolated into a //) so the
# lookbehind char class and "/" don't break the regex.
hit="$(printf '%s' "$msg" | perl -e '
my $re = $ARGV[0]; my $rx = qr/$re/i;
while (my $l = <STDIN>) { if ($l =~ /($rx)/) { print "$1\n"; last; } }
' "$ATTR_RE" | head -1 || true)"
[ -n "$hit" ] && note "commit ${sha:0:12}: message contains \"$hit\" (owner rule: zero AI attribution, ever)"
done < <(git rev-list "$RANGE" 2>/dev/null || true)
fi
echo
if [ "$fail" -ne 0 ]; then
echo "✗ leak-guard: blocking finding(s) above — DO NOT MERGE/PUBLISH"
exit 1
fi
echo "✓ leak-guard: clean"
+18
View File
@@ -0,0 +1,18 @@
# Coverage is informational only: it never fails CI or blocks a merge/release.
# The `coverage` job in .github/workflows/ci.yml is non-required and no gate
# depends on it.
coverage:
status:
project:
default:
informational: true
patch:
default:
informational: true
comment: false
ignore:
- "tests"
- "examples"
- "benches"
+42
View File
@@ -0,0 +1,42 @@
# cargo-deny — gates the RESOLVED dependency graph (issue #15).
# Runs in CI against the committed Cargo.lock: RUSTSEC advisories, a permissive
# licence allow-list, and source/ban hygiene. Update the lock (cargo update)
# to clear an advisory rather than adding an `ignore` entry without cause.
[advisories]
yanked = "deny"
# ignore = ["RUSTSEC-0000-0000"] # only with a written justification
[licenses]
# The whole tree is permissive. r-efi's LGPL arm is satisfied by Apache-2.0
# (its expression is "Apache-2.0 OR LGPL-2.1-or-later"), so no copyleft is
# allowed here. Add a licence only when a real dependency introduces it.
allow = [
"MIT",
"Apache-2.0",
"Apache-2.0 WITH LLVM-exception",
"0BSD",
"Unicode-3.0",
"Unlicense",
"Zlib",
"BSD-2-Clause",
"BSD-3-Clause",
"ISC",
"CDLA-Permissive-2.0",
]
confidence-threshold = 0.9
[bans]
multiple-versions = "warn" # transitive dupes (digest 0.10/0.11 etc.) are noise, not a gate
wildcards = "deny"
# The freemkv siblings resolve as path deps under CI's path-patch layout, which
# reads as a wildcard version req. That is internal, not a supply-chain wildcard.
allow-wildcard-paths = true
[sources]
unknown-registry = "deny"
unknown-git = "deny"
# The freemkv siblings are first-party git-tag deps; allow git sources from
# our own org only (CI resolves them as git, not the local path override).
[sources.allow-org]
github = ["freemkv"]
+302
View File
@@ -0,0 +1,302 @@
# FVI — Freemkv Video Index Format
**Specification version:** 1.0 (DRAFT)\
**File extension:** `.fvi`\
**Media type:** `application/vnd.freemkv.fvi+jsonl`\
**Status:** Draft for review. This document is the normative reference for the FVI
format; implementations and downstream tools cite it by section.
---
## 1. Scope and purpose
FVI is an open, codec-agnostic, byte-exact **index of the coded pictures** in a
video bitstream, together with **provenance** back to the source medium.
An FVI document answers, for every picture in a stream, three questions:
1. **Where is it?** — the byte-exact offset of its first byte in the *source*
(the disc/ISO/file), so a reader can extract or seek to any picture without
re-parsing the whole bitstream.
2. **What is it?** — coding type, random-access capability, GOP boundary, and
(where the codec defines them) field/pulldown attributes.
3. **When is it?** — decode and presentation timestamps on a declared timescale.
FVI is **not** a container, a codec, or a copy of the bitstream. It indexes; it
never stores coded samples. It is the serialized form of an indexer's per-picture
truth — carried from the demuxer, **never reconstructed** (§9).
## 2. Conformance
The key words **MUST**, **MUST NOT**, **REQUIRED**, **SHALL**, **SHALL NOT**,
**SHOULD**, **SHOULD NOT**, **MAY**, and **OPTIONAL** are to be interpreted as
described in BCP 14 (RFC 2119, RFC 8174) when, and only when, they appear in all
capitals.
A **conformant writer** MUST emit a document that satisfies §4–§10. A
**conformant reader** MUST accept any such document and MUST ignore unknown
object members (§11) so that forward-compatible extensions do not break it.
## 3. Terminology
- **Picture** — one coded video frame (or pair of fields coded as a frame). The
unit FVI indexes.
- **Access unit (AU)** — the set of bitstream bytes that decode to exactly one
picture (ISO/IEC 14496-10 §3; ISO/IEC 23008-2 §3).
- **Coded order** — the order pictures appear in the bitstream. FVI records are
emitted in coded order.
- **GOP / coded video sequence** — a self-contained run beginning at a
random-access point.
- **Provenance** — the mapping from an AU back to the exact bytes of the physical
source it was read from (§9).
- **Source position (`src`)** — `{ file, sector, byte }`, the provenance anchor of
an AU.
## 4. Encoding
An FVI document is a sequence of **UTF-8** text lines separated by a single LF
(`U+000A`). Each non-empty line is exactly one JSON value (RFC 8259), forming a
**JSON Lines / NDJSON** stream. A writer MUST NOT emit a UTF-8 BOM. A writer MUST
NOT pretty-print: each JSON value occupies exactly one line.
The first line MUST be the **Header** object (§6). Each subsequent line is one
**Picture record** (§7), in coded order.
Rationale: line-delimited JSON is streamable (a writer appends as it indexes; a
reader processes without loading the whole file), line-addressable (picture *n*
is near line *n+1*), append-safe, and parseable by every language without a
custom grammar — while remaining a precisely specified format, not an ad-hoc dump.
A document MAY be concatenated for multiple elementary streams: each stream is its
own header line followed by its records. Readers MUST treat a Header line as the
start of a new stream section.
## 5. Document structure
```
<header> line 1 (exactly one Header object)
<record> line 2 .. N (one Picture record per picture, coded order)
[<header> <record>…] (OPTIONAL further stream sections)
```
## 6. Header object
| Member | JSON type | Req | Semantics / reference |
|---|---|---|---|
| `format` | string | MUST | Constant `"freemkv/video-index"`. Signature: a document begins with these bytes. |
| `fvi_version` | integer | MUST | Document format version. This spec defines `1`. |
| `generator` | string | SHOULD | Producing tool + version, e.g. `"freemkv/1.0.0-rc.6"`. |
| `stream` | object | MUST | The indexed elementary stream (§6.1). |
| `source` | object | MUST | Provenance root (§6.2). |
| `timescale` | integer | MUST | Ticks per second for all `pts`/`dts` (§10). E.g. `90000`. |
| `picture_count` | integer | MAY | Total pictures, if known at header time; OMITTED when streaming. |
### 6.1 `stream` object
| Member | JSON type | Req | Semantics / reference |
|---|---|---|---|
| `codec` | string | MUST | Registered codec id (Appendix B), e.g. `"mpeg2video"`, `"hevc"`. |
| `width`,`height` | integer | MUST | Coded luma dimensions in pixels. |
| `dar` | `[int,int]` | SHOULD | Display aspect ratio as `[num,den]`. |
| `frame_rate` | `[int,int]` | SHOULD | Nominal rate as exact rational `[num,den]` (e.g. `[24000,1001]`). |
| `scan` | string | MUST | `"progressive"`<br>`"interlaced"`<br>`"mbaff"` |
| `colour` | object | SHOULD | CICP per ITU-T H.273: `primaries`, `transfer`, `matrix` (integer CICP codes or registered names)<br>`range`: `"limited"` \| `"full"`<br>HDR: `mastering_display`, `max_cll`, `max_fall` per ITU-T H.273 / SMPTE ST 2086. |
| `language` | string | MAY | BCP 47 tag, if known. |
### 6.2 `source` object
| Member | JSON type | Req | Semantics / reference |
|---|---|---|---|
| `medium` | string | MUST | `"disc"`<br>`"iso"`<br>`"file"`<br>`"stream"` |
| `path` | string | MAY | Source path/label. |
| `title` | integer | MAY | Title/program number. |
| `playlist` | string | MAY | Playlist/PGC identifier. |
| `volume_id` | string | MAY | Disc volume identifier, if read. |
| `sector_size` | integer | SHOULD | Bytes per `src.sector` unit (e.g. `2048`). Lets readers convert `src` to an absolute byte offset. |
## 7. Picture record
One JSON object per coded picture, in coded order.
| Member | JSON type | Req | Semantics / reference |
|---|---|---|---|
| `n` | integer | MUST | Coded-order index, 0-based, contiguous. |
| `src` | object | MUST | Provenance: `{ "file": int?, "sector": uint, "byte": uint }` — the offset of this AU's **first byte** in the source (§9). MUST be carried from demux, never reconstructed. |
| `type` | string | MUST | Coding type:<br>`"I"`<br>`"P"`<br>`"B"`<br>_ISO/IEC 13818-2 §6.3.9; H.264/H.265 slice types collapsed to frame type._ |
| `key` | boolean | MUST | `true` iff this picture is an intra (I) picture / parser-flagged decode-restart point (IDR / IRAP / I-picture).<br>_MPEG-2 open-GOP clean-RAP precision (`closed_gop`) is not currently distinguished — see note below._ |
| `gop` | boolean | SHOULD | `true` iff this picture begins a GOP / coded video sequence.<br>_Omitted when the implementation does not carry a distinct GOP-boundary signal._ |
| `pts` | integer\|null | SHOULD | Presentation timestamp in `timescale` ticks; `null` if unknown. |
| `dts` | integer\|null | MAY | Decode timestamp in `timescale` ticks. |
| `size` | integer | MAY | AU length in bytes; enables byte-range extraction with `src`. |
| `recovered` | boolean | MAY | `true` iff any byte of this AU came from a retried/marginal read (§9.1).<br>_Default `false`._ |
| codec ext | object | MAY | Codec-specific members under the codec's namespace (§8). |
The `type` and `key` members are **codec-agnostic** and MUST be populated for
every codec. `type` is the I/P/B coding type the parser decoded (collapsing
H.264/H.265 slice types to a frame type); where no per-picture coding is carried
(audio / synthetic frames), `type` is `"I"` for a key picture else `"P"`. `key`
is the picture's random-access flag as the codec parser sets it (IDR / IRAP /
I-picture). A writer MUST NOT emit a degraded record (`type:"?"` or `src:null`)
merely because a codec lacks per-picture coding info — those fallbacks are
reserved for a field that is genuinely unavailable (e.g. provenance absent on a
synthetic source).
> **Limitation (honest random-access).** `key` is set from the picture's
> intra / decode-restart flag. The per-picture coding model this index carries
> does **not** distinguish MPEG-2 open-GOP clean random-access points
> (`closed_gop`) from any other I-picture, so `key` is the parser-flagged
> decode-restart point, not a verified clean-RAP claim. A future revision MAY
> tighten `key` for codecs/profiles that carry that signal; readers MUST NOT
> assume present `key` precision beyond "intra / decode-restart point".
### 7.1 Interlace / pulldown fields
Codec-agnostic interlace/pulldown attributes, derived through the indexer's
per-picture coding accessors (MPEG-2: ISO/IEC 13818-2 §6.3.10). Emitted as
top-level members of the record, and ONLY when the codec actually measured the
signal — an OPTIONAL member that is omitted (not defaulted) when unknown:
| Member | JSON type | Req | Semantics / reference |
|---|---|---|---|
| `field_order` | string | MAY | Display field order:<br>`"tff"` — top field first<br>`"bff"` — bottom field first<br>`"progressive"` — no field order applies<br>_Omitted when the codec did not signal it._ |
| `progressive` | boolean | MAY | `true` iff the picture is progressive.<br>_Omitted when the codec did not signal it._ |
| `nb_fields` | integer | MAY | Number of displayed field periods this picture occupies (the soft-telecine / 2:3 pulldown basis):<br>`1` for a single field picture<br>`2` for a normal frame<br>`3`/`4`/`6` for `repeat_first_field` pulldown per §6.3.10 |
Codecs that carry only a coding type (e.g. H.264 / HEVC / VC-1 through this
pipeline) omit `field_order` and `progressive` rather than guessing a default.
## 8. Codec model and extensibility
Core record members (§7) are codec-agnostic and present for every codec.
Codec-specific data is either (a) promoted to top-level members for a small,
registered set per codec profile (e.g. MPEG-2 §7.1), or (b) placed under an
`ext` object keyed by codec id for richer/optional data:
```json
{
"n": 42,
"type": "P",
"key": false,
"src": {
"sector": 17,
"byte": 924
},
"ext": {
"hevc": {
"temporal_id": 0,
"nal_type": 1
}
}
}
```
New codecs and members are added through Appendix B (codec registry) without a
breaking version bump, provided readers continue to ignore unknown members (§11).
## 9. Provenance and recovery semantics
`src` is **byte-exact** to the source as read. `src.sector` counts in
`source.sector_size`-byte units; `src.byte` is the offset within that sector of
the AU's first byte. For multi-file sources, `src.file` indexes a writer-declared
file list. Provenance MUST be the value observed at demux time; an implementation
MUST NOT recompute `src` by re-parsing — the point of FVI is to *carry* the truth.
### 9.1 Recovery
Because FVI is provenance-native, it can record reliability. A record with
`"recovered":true` indicates the AU's source bytes required retry/marginal-read
recovery. This lets downstream tools surface or quarantine pictures whose bytes
are not byte-identical to a clean read — a capability legacy index formats lack.
## 10. Time model
All `pts`/`dts` are integers in units of `1/timescale` seconds. `pts` is
presentation (display) time; `dts` is decode time. Records are in **coded**
(decode) order, so `pts` is not necessarily monotonic across records (B-pictures
reorder); `dts` is non-decreasing. Readers needing display order sort by `pts`.
## 11. Versioning and forward compatibility
- `fvi_version` is the document version; this spec defines `1`.
- **Additive** changes (new OPTIONAL members, new registered codecs) do NOT bump
`fvi_version`. Readers MUST ignore members they do not recognize.
- A change that alters the meaning of an existing member or makes a new member
REQUIRED bumps `fvi_version`.
- A reader encountering a higher `fvi_version` than it implements SHOULD process
the members it understands and MUST NOT reject the document solely for the
version being higher, unless a member it relies on is absent.
## 12. Conformance requirements (summary)
A conformant **writer** MUST: emit a Header first; emit records in coded order
with contiguous `n`; populate `src` from demux; use named/registered codec ids;
encode one JSON value per UTF-8 LF-terminated line.
A conformant **reader** MUST: accept any §4–§10 document; ignore unknown members;
not assume `picture_count`, `pts`, or `size` are present unless required above.
---
## Appendix A — JSON Schema (informative)
Header:
```json
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"required": ["format", "fvi_version", "stream", "source", "timescale"],
"properties": {
"format": { "const": "freemkv/video-index" },
"fvi_version": { "type": "integer", "minimum": 1 },
"timescale": { "type": "integer", "minimum": 1 },
"stream": { "type": "object", "required": ["codec", "width", "height", "scan"] },
"source": { "type": "object", "required": ["medium"] }
}
}
```
Record:
```json
{
"$schema": "https://json-schema.org/draft/2020-12/schema",
"type": "object",
"required": ["n", "src", "type", "key"],
"properties": {
"n": { "type": "integer", "minimum": 0 },
"type": { "enum": ["I", "P", "B"] },
"key": { "type": "boolean" },
"src": {
"type": "object",
"required": ["sector", "byte"],
"properties": {
"file": { "type": "integer" },
"sector": { "type": "integer", "minimum": 0 },
"byte": { "type": "integer", "minimum": 0 }
}
}
}
}
```
## Appendix B — Registered codec identifiers
| `codec` | Bitstream | Field profile |
|---|---|---|
| `mpeg2video` | ISO/IEC 13818-2 | §7.1 (field_order/progressive/nb_fields) |
| `mpeg1video` | ISO/IEC 11172-2 | §7.1 |
| `h264` | ISO/IEC 14496-10 | core + `ext.h264` |
| `hevc` | ISO/IEC 23008-2 | core + `ext.hevc` |
| `vc1` | SMPTE 421M | core |
## Appendix C — Normative references
- RFC 2119, RFC 8174 — Requirement keywords (BCP 14).
- RFC 8259 — JSON.
- ISO/IEC 13818-2 — MPEG-2 video (picture coding, §6.3.9–6.3.10).
- ISO/IEC 14496-10 — H.264/AVC. ISO/IEC 23008-2 — H.265/HEVC.
- ITU-T H.273 — Coding-independent code points (colour primaries/transfer/matrix).
- SMPTE ST 2086 — Mastering display colour volume (HDR).
- BCP 47 — Language tags.
- RFC 9559 — Matroska (alignment of colour/field-order semantics).
+1 -1
View File
@@ -12,7 +12,7 @@ Technical documentation for [libfreemkv](https://github.com/freemkv/libfreemkv),
|----------|---------------|
| [Architecture](architecture.md) | Module map, design principles, error codes, platform support |
| [Drive Access](drive-access.md) | Drive, SCSI transport, profiles, unlock, why raw mode is needed |
| [Rip Recovery](rip-recovery.md) | Three-layer recovery model: Disc::patch, single-shot Drive::read, DiscStream batch halving |
| [Rip Recovery](rip-recovery.md) | What this crate owns of the recovery model: single-shot Drive::read, SenseFamily, DiscStream batch halving (the strategy itself moved to freemkv-engine in 1.6.0) |
| [AACS Encryption](aacs.md) | Key resolution (4 paths), content decryption, bus encryption, SCSI handshake |
| [UDF Filesystem](udf.md) | UDF 2.50 with metadata partitions, pointer chain, how files are read from disc |
| [MPLS Playlists](mpls.md) | Playlist format, play items, STN stream table, coding types |
+190
View File
@@ -0,0 +1,190 @@
# AACS Content Decryption — Implementation Notes
Rationale and background trimmed out of `src/aacs/content.rs` doc comments to
satisfy `ci/comment-guard.py`'s per-audience prose caps. See
[AACS Encryption](aacs.md) for the public-facing key resolution story and
[AACS Crypto Primitives](aacs-crypto.md) for the low-level AES primitives.
## `is_unit_aligned`
AACS aligned units (6144 B / 3 sectors) are anchored at the start of each
clip's encrypted region, so a read must begin a whole number of units past
that base for `decrypt_sectors` (which anchors units at buffer offset 0) to
align the CBC correctly. This is the single source of truth for the
decrypt-on-read gate, the inline and highway mux read paths, and the
key-validation sample reader — all key off this, never absolute `lba % 3`. A
disc whose clip `start_lba` is not itself 3-aligned would otherwise mis-gate
(reject readable units, then report "Decryption failed") on exactly the
titles whose clips land off a 3-boundary.
`saturating_sub` makes the `lba < unit_base` case well-defined — it clamps
the offset to 0, a unit boundary — rather than the latent `wrapping_sub` trap
where an underflow wraps to ~2^32 and, because `2^32 ≡ 1 (mod 3)`,
mis-reports the alignment (e.g. `lba == unit_base - 1` would falsely read as
aligned).
## `aacs_unit_encrypted`
The mux is the principal consumer of this signal for `BdTs` discs (via
[`is_clean`], see below). It is readable WITHOUT a key. CRITICAL: only
meaningful when `unit` is read at the correct clip-FILE-anchored boundary — a
disc-absolute / mis-aligned read makes the flag byte arbitrary mid-stream
data, which is why per-unit checks run clip-anchored, not in the whole-disc
sweep.
## `aacs_unit_needs_decrypt`
Buffer-iterating sites that may run twice over the same `buf` (the post-fetch
re-decrypt, sample collection, failure diagnosis) need an IDEMPOTENT "does
this still need work?" test, so `aacs_unit_encrypted` (the authoritative flag,
which decryption never rewrites) is composed with a "structure restored?"
check that flips once decrypted: TS syncs come back for `BdTs`; valid
`00 00 01 BA` packs come back for `MpegPs`.
## HD-DVD scramble flag (`PS_SCRAMBLE_OFF` / `PS_SCRAMBLE_MASK`)
BD/UHD/FMTS flag encryption uses the Copy Permission Indicator in the top 2
bits of byte 0 (the M2TS `TP_extra_header`). HD-DVD `.evo` is Program Stream —
byte 0 is a `00 00 01 BA` pack_start_code, NOT a CPI — so AACS reuses the
MPEG-2 `PES_scrambling_control` field instead: pack_header (14 bytes) + PES
start-code/`stream_id` (4) + `PES_packet_length` (2) puts the PES flags byte at
offset 20, with `PES_scrambling_control` in bits 5-4 (`& 0x30`). Non-zero =
encrypted. Derived from BackupHDDVD (`Header[20] & 0x30`) cross-checked
against MPEG-2 systems (ISO/IEC 13818-1).
UNVERIFIED against a real ENCRYPTED HD-DVD disc — none available to confirm
byte-exactly. Two open questions a real disc must settle:
1. A pack carrying an MPEG `system_header` (`00 00 01 BB`) before the PES
packet shifts this offset past 20.
2. Whether offset 20 is even readable pre-decrypt. BD keeps only the first 16
bytes clear (the seed) and AES-CBC-encrypts 16..6144 — under that model
offset 20 is ciphertext. BackupHDDVD reading `Header[20]` pre-decrypt
implies HD-DVD instead encrypts PES *payloads* with *clear* pack/PES
headers (per-PES model). If so, `decrypt_unit`'s 16-byte-seed model also
would not fit HD-DVD and needs its own path. TS is unaffected either way.
## `KEY_PROOF_PACKETS`
Four `0x47` syncs are 32 bits of MPEG-TS structure, but the per-UNIT
false-pass risk is NOT 2^-32: `is_clean_ts` accepts ANY four of the ~31
encrypted packets in a 6144-byte aligned unit, so for a wrong key (uniform
AES noise, `0x47` at 1/256 per packet) it is ≈ C(31,4)·256^-4 ≈ 7e-6, i.e.
~1e-5. 1-in-4-billion is the probability for four SPECIFIC packets and
overstates the margin by ~4000x; at `KEY_PROOF_PACKETS = 3` the per-unit rate
is ≈ C(31,3)·256^-3 ≈ 2.6e-4, so do NOT lower it on the strength of slack
that is not there. It is an ABSOLUTE proof floor, NOT a proportion — a unit
the key opened but whose content is bad-encoded (many non-conforming
packets) is proven by ANY four good packets, not rejected for the bad ones.
## `is_clean_ts`
The mux is its principal consumer for `BdTs` discs, reaching it through
`is_clean`: `mux::resolve`'s multi-CPS `pick` closure selects a unit key by
it, `probe_index_phase` reports each FMTS index's interleave parity by it,
and `decrypt::decrypt_sectors_mapped` uses it as the forensic-range verify
net. (The doc used to say "the mux never calls this", which invited a
maintainer to tighten or loosen the proof rule believing only whole-disc read
verification was affected — while it in fact changes which unit key a
multi-CPS disc muxes with and which phase an FMTS index is muxed at.)
Rule — evidence is ABSOLUTE, scaled to the packets that exist. Over the
ENCRYPTED packets (skip packet 0: its `0x47` sits in the clear 16-byte seed,
so it reads `0x47` for ANY key and is never evidence), let `E` = non-padding
content packets and `synced` = those carrying `0x47`. The key opened the unit
iff `E == 0` (nothing encrypted to prove) OR `synced >= min(E, KEY_PROOF_PACKETS)`.
* WRONG key → ~0 synced → fails (reaching 4 by chance ≈ 1e-5/unit, and every
unit of a clip would have to fluke — astronomically safe).
* RIGHT key, bad-encoded content → any 4 good packets pass; the bad ones are
the muxer's problem. (This is the false-negative the old 75% PROPORTION
caused — a mostly-bad unit the key opened was wrongly rejected.)
* `min(E, 4)` handles the end-of-clip fragment TAIL: a unit with only E=1
real packet (then source-zero padding) needs just that one to sync, so a
sparse-but-valid tail is never false-rejected. Padding (all-zero payload)
is excluded throughout.
## `is_clean_ps`
The Program-Stream arm of `is_clean` (HD-DVD `.evo`): every 2048-byte pack
begins with the pack_start_code `00 00 01 BA`; a 6144-byte AACS unit spans
three packs. Pack 0's start sits in the clear 16-byte seed (present
regardless of the key — mirrors `is_clean_ts` skipping packet 0); packs 1
and 2 (offsets 2048 and 4096) are in the encrypted region, so a wrong key
garbles them and this returns false (64 bits of discrimination). Validated
against real decrypted HD-DVD `.evo` (two real retail titles): pack starts
are exactly 2048-aligned, three per aligned unit, at offsets 0 / 2048 / 4096.
UNVERIFIED-HDDVD-DECRYPT: the pack structure here is confirmed on decrypted
rips, but that a real ENCRYPTED `.evo` decrypts to it via the same
6144-byte aligned unit (16-byte seed + AES-CBC over 16..6144) as BD is the
unverified assumption — we have no encrypted HD DVD. If HD-DVD decryption
yields garbage with a known-good key, the unit granularity is the suspect.
## `decrypt_unit` / `encrypt_unit`
Block Key = AES-128E(Kcu, seed) ⊕ seed (`[BD]` §3.10.1 Fig 3-8: encrypt the
clear 16-byte seed under the CPS Unit Key, XOR the seed back in — the
trailing ⊕ seed is load-bearing); then AES-128-CBC decrypt/encrypt bytes
16..6144 under the AACS IV. `encrypt_unit` is the exact inverse of
`decrypt_unit`, for authoring an encrypted disc image (and for building
genuinely-encrypted read-path fixtures); it does NOT set the encrypted flag,
because where that flag lives is container-specific, and keeping it out
keeps the crypto container-agnostic.
Source-zero padding packets (all 192 bytes zero on disc) are restored to
zero by `decrypt_unit`: their decrypted bytes are AES-noise from decrypting
zeros, but the source WAS zero, so writing the true source back is faithful
and gives the demux a tidy gap. Content packets are left EXACTLY as
decrypted — the decrypt path never rewrites content, so an authored-bad
packet passes through verbatim. `encrypt_unit` deliberately does NOT mirror
this: an all-zero plaintext packet encrypts to ciphertext that is not
all-zero, so it is not mistaken for padding on the way back and the round
trip is still exact. Authoring that wants true source-zero padding leaves
those packets unencrypted instead.
`encrypt_unit` returns `false` — encrypting nothing — when `unit` is shorter
than `ALIGNED_UNIT_LEN`. That case MUST be checked: the caller has already
set the container's encrypted flag by then (the contract requires
flag-before-crypto, since the header is the key seed), so ignoring the
result leaves a unit marked encrypted while still carrying plaintext, which
is the worst possible outcome for an authoring tool.
## `decrypt_bus` key-schedule regression (test: `decrypt_bus_expands_the_read_data_key_once_per_unit`)
MEASURED, not reasoned: `decrypt_bus` used to call `aes_cbc_decrypt` once per
2048-byte sector, and each call built its own AES-128 key schedule, so a
6144-byte aligned unit performed THREE key expansions under the same
loop-invariant `read_data_key`. On a 90 GB UHD read on a stock
(non-firmware-unlocked) drive — ~14.6 million aligned units — that is ~29
million redundant expansions on the per-unit decrypt hot path, for a key
that is constant for the whole disc. The counter is incremented inside
`crypto::new_cipher`, the single construction site.
## `IV0_PUBLISHED` (test fixture)
`[C]` §2.1.2 fixes one default CBC IV for every AACS AES-CBC operation. Both
IV tests used to compute their expected value from `crypto::AACS_IV` itself,
so the constant was asserted against itself and NOTHING in the suite pinned
its bytes: swapping `AACS_IV` for `[0u8; 16]` left both tests passing (one
builds its ciphertext with the same value and the other cancels the change
in a triple XOR) while every real AACS disc decrypted to noise — block 0 of
every 6144-byte aligned unit and of every bus-encrypted sector XORed with the
wrong IV. `IV0_PUBLISHED` is the independent witness the tests assert
against instead.
## Encrypted-flag readers (`aacs_unit_seed_encrypted`, `aacs_unit_encrypted`)
`aacs_unit_seed_encrypted` is the flag reader for a PARTIAL unit — the guard
that stops a truncated encrypted fragment from being emitted as clear
content. It reads ONLY the two Copy Permission Indicator bits (`[BD]`
§3.10.2, byte 0 bits 6-7); the remaining six bits are `TP_extra_header`
arrival-timestamp bits and carry no encryption meaning. Both failure
directions are damaging and silent: a reader that answers "encrypted" for a
clear fragment discards good content, and one that answers "clear" for an
encrypted fragment writes ciphertext into the output as if it were video.
`aacs_unit_encrypted` is the AUTHORITATIVE gate and requires a WHOLE
6144-byte aligned unit: on anything shorter the flag byte is not guaranteed
to be the unit's, so it must answer `false` and leave the partial-unit case
to `aacs_unit_seed_encrypted`. A reversed length guard would both classify
fragments off arbitrary mid-stream bytes and, on an empty slice, index out
of bounds.
+89
View File
@@ -0,0 +1,89 @@
# AACS Crypto Primitives — Implementation Notes
Rationale and background for the internal (non-public) primitives in
`src/aacs/crypto.rs`. See [AACS Encryption](aacs.md) for the public-facing
key resolution and decryption story.
## `aes_cbc_encrypt`
Forward direction of `aes_cbc_decrypt`, and its exact inverse (`[C]` §2.1.2,
`AES-128CBCE`). `data.len()` must be a multiple of 16; the `debug_assert!`
documents/enforces that contract.
Constructs the cipher ONCE for the whole slice. Driving this from the
single-block `aes_ecb_encrypt` instead rebuilds the AES key schedule per
16-byte block, which for a 6144-byte aligned unit is 383 redundant key
expansions.
## `aes_cbc_decrypt`
AES-128-CBC decrypt in-place with the fixed AACS IV (`[C]` §2.1.2,
`AES-128CBCD`). `data.len()` must be a multiple of 16; any trailing partial
block is silently ignored. All callers pass aligned regions (6128 and 2032
bytes), and the `debug_assert!` documents/enforces that contract.
This doc was once orphaned onto `aes_cbc_encrypt` above it, when that
function was inserted directly after it with no separating blank line:
rustdoc rendered the crate's only forward-direction AACS primitive as
"decrypt" and cited the spec's DECRYPT clause for it, while this function
had no doc at all. `encrypt_unit_is_the_exact_inverse_of_decrypt_unit` in
`content.rs` pins the directions behaviourally, so a maintainer "fixing" the
contradiction by swapping the two bodies fails the suite instead of shipping
a second decryptor behind an already-set encrypted flag.
## `cbc_decrypt_blocks`
Split out of `aes_cbc_decrypt` so a caller that decrypts several regions
under one loop-invariant key expands the schedule once. `decrypt_bus`
(`super::content::decrypt_bus`) is that caller: bus encryption (`[C]` §4.2 /
the AACS 2.0 Read Data Key) covers bytes 16..2048 of EVERY 2048-byte
sector, so a 6144-byte aligned unit is three regions under one
`read_data_key` — three key schedules where one suffices, on the per-unit
decrypt hot path of a whole 90 GB read.
## `aes_g`
AES-G(x1, x2) = AES-128D(x1, x2) XOR x2 (`[C]` §2.1.3, note: uses
AES-128**D**). The Media Key Variant chain uses AES-G to derive both the
variant number (`Kvn = AES-G(Kp, Nonce)`) and the Volume Unique Key
(`Kvu = AES-G(Km, VID)`). See `super::derive::derive_vuk` for the classical
VUK form — the math is identical, this exposes it as a neutral primitive
for the variant chain.
## `aesg3`
AACS-G3 (`[C]` §3.2.2, Triple AES Generator): `left = D(k,s0)⊕s0` inc 0,
`pk = D(k,s0+1)⊕(s0+1)` inc 1, `right = D(k,s0+2)⊕(s0+2)` inc 2. Shared
with `super::variant` (its variant chain runs the same SD tree); a single
definition keeps the two walks byte-identical.
## Test: `S0`
The AACS-G3 seed `s0`, transcribed independently from `[C]` §3.2.2 rather
than read from `AESG3_SEED` — a test that sourced the seed from the
production constant would assert that constant against itself and would
still pass if it were edited.
## Test: `aesg3_inverts_to_the_spec_seed_under_aes_encrypt`
`aesg3` is the node function of the AACS subset-difference tree: every
Processing Key the DK walk produces (`aesg3(node_key, 1)`) and every descent
step (`aesg3(., 0)` / `aesg3(., 2)`) is one call. A body that returned a
fixed block would make every device key in the crate derive the SAME
Processing Key, and a `^` that became `|` or `&` would derive a
wrong-but-plausible one — in both cases the MKB walk simply stops finding
Media Keys, with no error to say why.
Pinned through the spec relation rather than a re-implementation: `[C]`
§3.2.2 defines `AES-G3` as `AES-128D(k, s) XOR s` for `s = s0 + inc` (added
into the last seed byte), so applying the FORWARD primitive
`aes_ecb_encrypt` — a different function from the one under test — to
`aesg3(k, inc) XOR s` must reproduce `s` exactly.
## Test: `aesg3_yields_three_distinct_subkeys_for_the_three_increments`
The Triple Generator's three outputs (`[C]` §3.2.2: left = inc 0, the
Processing Key = inc 1, right = inc 2) are the two child node keys and the
Processing Key of ONE tree node. They must be three different keys — if
`inc` were ignored, a descent would revisit its own parent and the walk
would derive the same key at every level of the tree.
+68
View File
@@ -0,0 +1,68 @@
# `src/aacs/derive.rs` — key derivation notes
Rationale and background relocated here from doc comments in `derive.rs` to
keep those comments within the repo's comment-length limits.
## `derive_media_key_from_pk` — why the direct PK × cvalue scan
A Processing Key is **terminal**: it is the key at its Subset-Difference
node, one `AES-G` from the Media Key. So the fast path tries each PK
*directly* against the MKB cvalue tables (no tree descent). On a large
AACS 2.x UHD MKB (~181k cvalues) this is ~15x faster than treating a PK as
a device-node label and walking the tree.
MKB record types relevant to this path:
| Type | Meaning |
|------|---------|
| `0x10` | Type and Version Record (has MKB version) |
| `0x81` | Verify Media Key Record, AACS 1.0 (has `mk_dv`) |
| `0x86` | Verify Media Key Record, AACS 2.0/2.1 (has `mk_dv`) |
| `0x04` | Subset-Difference Index (has UVS entries) |
| `0x05` | Media Key Data Record (cvalues, 1:1 with `0x04`) |
| `0x07` | Explicit Subset-Difference Record (NOT cvalues) |
## `recover_dk_position` — search strategy and cost
This finds a device key's tree node empirically: for each MKB
subset-difference record, it tries the device at the record's node AND at
every ancestor v-position (the device may sit one or more levels ABOVE the
record, descending via AES-G3 to reach it), deriving the candidate
Processing Key DIRECTLY (one `calc_pk_from_dk` per candidate, no full
re-walk) and checking it validates against that record's cvalue.
Cost is `O(slots × tree_depth)` — linear in the MKB's subset-difference
index, not the quartic cost of re-deriving per candidate. Slots are
independent so the scan parallelizes (~181k slots, ~26s serial on a UHD
MKB); `find_map_any` returns the first match and cancels the rest.
## `resolve_candidate` — composed primitives and `None` conditions
Composes the raw derivation primitives (`derive_media_key_from_pk`,
`derive_media_key_and_pk_from_dk`, `derive_vuk`, `derive_unit_keys`) and
parses `Unit_Key_RO.inf` at the version the disc's MKB declares, so a
multi-CPS disc yields all its unit keys from the one candidate.
Validate `unit_keys` against a real encrypted unit with `decrypt_unit` +
`is_clean_ts` to prove the candidate actually opens the disc — this function
is pure derivation, no sampling or validation.
Returns `None` only when derivation itself cannot proceed: a PK its MKB
rejects, a `Dk` the MKB can't process, a missing VID on a path that needs
one, or an unparseable/empty `Unit_Key_RO.inf`.
## Test fixtures — inverting the AACS relations
There are no published AACS test vectors, but none are needed: the AACS
relations (`[C]` §3.2.3–§3.2.5) are invertible, so a valid MKB for a CHOSEN
key can be constructed with `aes_ecb_encrypt` and the same `aesg3` the walk
uses as its node function. `plant_mkb` and its siblings in
`position_recovery_tests` do exactly this — no real key material, and the
assertions check the DERIVED Media Key, not any intermediate the code under
test also produces.
`plant_mkb`'s single-slot fixture picks `uv = 0x0400` (lowest set bit 10)
with `u_mask_shift = 12` so a gating device node exists: `resolve_dk_node`
flips one bit `b < 12`, and the walk's gate (`[C]` §3.2.4) needs that bit
inside `v_mask` (`0xFFFF_FFFF << 11`) and outside `u_mask`
(`0xFFFF_FFFF << 12`) — i.e. `b == 11`.
+78
View File
@@ -0,0 +1,78 @@
# `src/aacs/resolve.rs` — internal rationale
Overflow detail relocated here by the comment-guard pass; each section is
pointed to by a short `//` comment at the named site in `resolve.rs`.
## `resolve_keys_classical`
Resolve all AACS keys for a disc using the classical (single-stage Media Key
derivation) paths. Used by both V10 and V20.
Paths run in root-of-trust -> per-disc-leaf order. A match at any path
returns immediately:
1. MKB + device keys -> processing key -> media key -> VUK
2. MKB + processing keys -> media key -> VUK
3. KEYDB MK + matching VID -> derived VUK
4. KEYDB disc-hash -> VUK
5. KEYDB disc-hash -> pre-decrypted unit keys (no VUK)
## `unique_verifying_mk`
The MK-pool selection rule of path 2.5, split out of `resolve_keys_v1` so the
ambiguity guard has a reachable test.
`verifies` is the MKB check -- in production
`AES-D(mk, mk_dv)[..8] == MK_VERIFY_MAGIC`. Returns a Media Key only when
EXACTLY ONE DISTINCT candidate passes. Duplicates of the same key are one
candidate (a pool aggregated across providers routinely repeats a key), but
two DIFFERENT keys that both verify mean the pool cannot say which is this
disc's Km: picking either derives a wrong VUK, and a wrong VUK decrypts to
plausible-looking garbage rather than failing loudly. Bail and let the later
hash/VID paths answer instead.
The predicate is a parameter rather than the inlined AES check because a
genuine two-key multi-hit cannot be synthesised: it needs one ciphertext
that decrypts under two distinct AES-128 keys to plaintexts sharing a 64-bit
prefix -- a 2^64 search. Injecting the verifier is the only way the
ambiguity branch is reachable from a test at all.
## Test: `mk_pool_ambiguity_bails_rather_than_picking_a_media_key`
Path 2.5's ambiguity guard: when MORE THAN ONE DISTINCT pooled Media Key
verifies against the MKB, the resolver must return no key at all rather than
pick one. A wrong Km derives a wrong VUK, and a wrong VUK does not fail
loudly -- it decrypts the title to garbage that muxes and plays as a corrupt
rip.
The real MKB check cannot be forced into a multi-hit: two distinct AES-128
keys decrypting one `mk_dv` to plaintexts that share the 64-bit verify magic
is a 2^64 search, not a fixture. So the rule is tested through
`unique_verifying_mk`, whose verifier is a parameter -- the same function
`resolve_keys_v1` calls, with the same pool semantics.
## Test: `resolve_keys_v21_treats_the_all_zero_volume_id_as_no_vid`
`resolve_keys_v21` gates paths 1 and 3 on `has_vid`, and an all-zero Volume
ID is the crate's "the VID was never read" sentinel -- the SCSI handshake
leaves the buffer zeroed when it does not run or fails.
Both directions matter and both fail silently:
- treating the zero sentinel as a real VID runs path 3 and derives
`Kvu = AES-G(Km, 0...0)`, a perfectly well-formed but WRONG VUK. It
unwraps the title keys to garbage, and nothing downstream errors -- the
rip just decodes to noise.
- treating a real VID as absent skips paths 1 and 3 entirely, so a disc
that could have been resolved from its Media Key reports no key.
Asserted through the final VUK, not through the flag.
## Test: `classify_vid_present_with_material_is_no_material_not_vid`
A NON-zero VID present, but resolution still fails (the providers carry
material that doesn't resolve this disc). The VID is available, so the
failure is NOT "VID unavailable" -- it must classify NoMaterial regardless
of how much derivation material is present, because re-acquiring the VID is
not the fix. This is the `has_vid == true` short-circuit, which no existing
test covers (the gate test only uses the zero-VID sentinel).
+14
View File
@@ -0,0 +1,14 @@
# AACS resolution trace shape
`src/aacs/trace.rs` defines the structured trace of a resolution attempt (the
unlock phase, then the key-resolution phase).
- No user-facing English: every step's STATE is a typed enum variant.
Applications render these into localized text — the library never does.
This module only defines the shape and is wired through the
resolve/handshake return path far enough to compile.
- The `who` of each step is the source's `label()` / unlocker's `name()` — a
stable identifier string (a NAME, like a codec id, NOT user-facing prose),
carried verbatim so an app renderer never has to match an enum back to a
name it already has. Only the OUTCOME / path enums are structured states
the app maps to i18n English.
+150 -256
View File
@@ -2,191 +2,45 @@
## Overview
AACS (Advanced Access Content System) is the encryption layer used by Blu-ray and UHD 4K discs to protect content. libfreemkv implements AACS decryption to enable transparent disc access.
AACS (Advanced Access Content System) is the encryption layer used by Blu-ray
and UHD 4K discs to protect content. libfreemkv implements AACS decryption so
disc access is transparent to the application.
There are two major versions:
- **AACS 1.0** -- Used by standard Blu-ray discs. Relies on a custom 160-bit elliptic curve for bus authentication and AES-128 for content encryption. Processing keys and device keys can derive the media key from the disc's Media Key Block (MKB).
- **AACS 2.0** -- Used by UHD 4K Blu-ray discs. Adds a per-sector bus encryption layer (read_data_key) on top of the standard content encryption. Uses P-256/SHA-256 for its native handshake, though drives accept AACS 1.0 host certificates for backward compatibility.
Both versions use AES-128-CBC for content decryption with a fixed initialization vector. The fundamental key hierarchy is the same: a Volume Unique Key (VUK) decrypts per-title unit keys, which in turn decrypt the content stream.
## Architecture
AACS support is split across two modules:
### `aacs.rs` -- Keys and Decryption
Handles everything related to key resolution and content decryption:
- KEYDB.cfg parsing (device keys, processing keys, host certificates, per-disc entries)
- Disc hash computation (SHA-1 of `Unit_Key_RO.inf`)
- VUK resolution chain (4 paths, described below)
- MKB record parsing and media key derivation
- Subset-difference tree traversal (AACS-G3 key derivation)
- Unit_Key_RO.inf parsing and unit key decryption
- Content Certificate parsing (AACS version detection)
- Aligned unit decryption (AES-128-CBC)
- Bus decryption (AACS 2.0 read_data_key layer)
### `aacs_handshake.rs` -- SCSI Authentication
Handles the drive-level SCSI authentication protocol:
- ECDH key agreement on the AACS 160-bit curve
- ECDSA signing and verification
- Bus key derivation
- AGID management (allocate/invalidate)
- Volume ID retrieval (encrypted with bus key, verified by AES-CMAC)
- Read Data Key retrieval (for AACS 2.0 bus decryption)
- AACS LA public key certificate verification
## Key Resolution Chain
When a disc is scanned, `resolve_keys()` attempts four paths in priority order. The first path that succeeds is used.
### Path 1: KEYDB VUK Lookup (fastest)
```
Unit_Key_RO.inf --> SHA-1 --> disc_hash --> KEYDB lookup --> VUK
```
The disc hash is computed as the SHA-1 digest of the raw `Unit_Key_RO.inf` file from the disc's `/AACS/` directory. This hash is used as the lookup key in `KEYDB.cfg`. If a matching entry contains a VUK (`V` field), it is used directly.
This is the fast path and resolves the vast majority of discs in a well-maintained KEYDB.
### Path 2: KEYDB Media Key + Volume ID
```
KEYDB media_key + Volume ID (from SCSI handshake) --> VUK derivation
```
If the disc hash is not in the KEYDB but a KEYDB entry has a matching Volume ID (`I` field) and a media key (`M` field), the VUK is derived:
```
VUK = AES-128-ECB-DECRYPT(media_key, volume_id) XOR volume_id
```
Requires a successful SCSI handshake to obtain the Volume ID.
### Path 3: MKB + Processing Keys
```
MKB (from disc) + processing_keys (from KEYDB) --> media_key --> VUK
```
Processing keys are pre-computed keys that work against specific MKB versions. For each processing key, the library:
1. Parses the MKB to extract the Verify Media Key Record (`mk_dv`), subset-difference index, and conditional values (cvalues).
2. Tries each processing key against each UV/cvalue pair: `mk = AES-DEC(pk, cvalue) XOR cvalue`.
3. Validates the derived media key: `AES-ECB(mk, mk_dv)` must produce 12 leading zero bytes.
4. Derives VUK from the validated media key and Volume ID.
### Path 4: MKB + Device Keys (Subset-Difference Tree)
```
MKB + device_keys --> subset-difference tree traversal --> processing_key --> media_key --> VUK
```
The most complex path. Each device key has an associated node number, UV value, and mask parameters that position it in the AACS subset-difference tree. The library:
1. Finds the subset-difference entry in the MKB that applies to the device key's node.
2. Traverses the tree using AACS-G3 key derivation: `aesg3(key, inc) = AES-DEC(key, seed) XOR seed`, where `seed[15]` is incremented by `inc`. Each tree node produces a left child (inc=0), a processing key (inc=1), and a right child (inc=2).
3. At each level, selects left or right based on the UV bit at the current position.
4. The resulting processing key is validated against the MKB cvalue to derive the media key.
5. VUK is derived from the media key and Volume ID.
## Content Decryption
### Aligned Units
AACS encrypts content in aligned units of 6144 bytes (3 sectors of 2048 bytes each). The encryption flag is signaled by the copy_permission_indicator bits in byte 0 of the unit (`unit[0] & 0xC0 != 0`).
### Per-Unit Key Derivation
Each aligned unit has its own decryption key derived from the CPS unit key:
1. **Derive**: AES-128-ECB encrypt the first 16 bytes of the unit (plaintext TP_extra_header) with the unit key.
2. **XOR**: XOR the encrypted result with the original 16 bytes to produce the per-unit decryption key.
3. **Decrypt**: AES-128-CBC decrypt bytes 16 through 6143 using the per-unit key and the fixed AACS IV.
4. **Clear flag**: Clear the encryption indicator bits (`unit[0] &= !0xC0`).
### Fixed IV
All AES-CBC operations in AACS use the same fixed initialization vector, defined in the AACS specification.
### Verification
After decryption, the library verifies correctness by checking for MPEG-TS sync bytes (0x47) at the expected 192-byte packet boundaries within the unit. Blu-ray transport stream packets are 192 bytes: 4-byte TP_extra_header followed by a 188-byte TS packet.
## Bus Encryption
### AACS 1.0
Standard Blu-ray discs do not use bus encryption. Content is read directly from the disc and decrypted using the unit key.
### AACS 2.0
UHD 4K discs add a per-sector bus encryption layer. The drive encrypts data as it is read from the disc, and the host must decrypt it before applying AACS content decryption.
Bus encryption uses a **read_data_key** obtained during the SCSI handshake. For each 2048-byte sector within an aligned unit, bytes 16 through 2047 are AES-128-CBC encrypted with the read_data_key and the fixed AACS IV. The first 16 bytes of each sector remain plaintext.
The full decryption pipeline for AACS 2.0:
1. **Bus decrypt**: For each sector, AES-128-CBC decrypt bytes 16..2047 with the read_data_key.
2. **Content decrypt**: Standard per-unit key derivation and AES-128-CBC decryption as described above.
## SCSI Handshake
The AACS SCSI authentication handshake establishes a shared bus key between host and drive, then uses it to securely transfer the Volume ID and read data keys.
### Protocol Flow
1. **Invalidate AGIDs**: Send REPORT KEY with format 0x3F for AGIDs 0-3 to clear stale sessions.
2. **Allocate AGID**: REPORT KEY format 0x00 returns a fresh Authentication Grant ID.
3. **Send host credentials**: SEND KEY format 0x01 transmits the host nonce (20 random bytes) and host certificate (92 bytes).
4. **Receive drive credentials**: REPORT KEY format 0x01 returns the drive nonce and drive certificate.
5. **Receive drive key**: REPORT KEY format 0x02 returns the drive's ephemeral EC key point and ECDSA signature over `host_nonce || drive_key_point`.
6. **Verify drive key**: The signature is verified against the drive's public key (extracted from its certificate). AACS 1.0 certificates are verified against the AACS LA public key.
7. **Send host key**: The host generates an ephemeral key pair, signs `drive_nonce || host_key_point` with the host private key, and sends via SEND KEY format 0x02.
8. **Compute bus key**: ECDH shared secret = `host_private_key * drive_key_point`. The bus key is the low 128 bits of the shared point's x-coordinate.
### Post-Authentication Reads
- **Volume ID**: REPORT DISC STRUCTURE format 0x80. Returns 16-byte VID encrypted with the bus key, plus an AES-CMAC MAC for integrity verification.
- **Read Data Keys**: REPORT DISC STRUCTURE format 0x84. Returns the read_data_key and write_data_key, each AES-ECB encrypted with the bus key.
### Elliptic Curve
AACS 1.0 uses a custom 160-bit Weierstrass curve (`y^2 = x^3 + ax + b mod p`) with 20-byte field elements. The library implements full EC arithmetic: point addition, doubling, scalar multiplication, modular inverse, ECDSA sign/verify, and ECDH key agreement.
## AACS 2.0 Status
AACS 2.0 discs are detected via the Content Certificate file (`Content000.cer` or `Content001.cer`). A certificate type byte of 0x01 indicates AACS 2.0.
AACS 2.0 drives are identified by their drive certificate type (0x11). These drives natively use P-256/SHA-256, but accept AACS 1.0 host certificates for backward compatibility.
Current implementation status:
- AACS 2.0 detection: **implemented** (Content Certificate parsing, drive cert type check)
- AACS 1.0 handshake with AACS 2.0 drives: **implemented** (backward compatibility mode)
- Full P-256 AACS 2.0 handshake: **not yet implemented** (prepared but rarely needed since drives accept AACS 1.0 host certs)
- Bus decryption with read_data_key: **implemented**
- Content decryption: **implemented** (same as AACS 1.0)
In practice, AACS 2.0 UHD discs work through the backward-compatible AACS 1.0 handshake path, with the addition of read_data_key bus decryption.
- **AACS 1.0** -- Used by standard Blu-ray discs.
- **AACS 2.0 / 2.1** -- Used by UHD 4K Blu-ray discs. Adds a per-sector bus
encryption layer on top of the standard content encryption. UHD drives accept
AACS 1.0 host credentials for backward compatibility.
All versions use AES-128 for content decryption. The library reads the keys it
needs from `keydb.cfg`, walks the disc's Media Key Block (MKB) to resolve the
disc's key, and decrypts the content stream. AACS-encrypted discs therefore
require a `keydb.cfg`; CSS-protected DVDs do not (see the CSS notes in the
library docs).
## How it works (feature level)
When a disc is scanned, the library:
1. Reads the disc's AACS key-input files from the `/AACS/` directory.
2. Resolves the disc's key from `keydb.cfg` — either directly from a per-disc
entry, or by walking the MKB with the keys present in the keydb.
3. Performs the drive-level SCSI authentication handshake needed to obtain the
Volume ID and, for UHD, the bus-decryption key.
4. Decrypts the content stream as titles are read.
A resolved key is verified against actual disc content before it is applied, so
a stale or wrong key fails loudly rather than producing silent garbage. If no
usable key is available for an AACS-encrypted disc, the library surfaces a
specific error (the E70xx family) describing which part of the chain was
missing, and a missing `keydb.cfg` surfaces as `Error::KeydbLoad` with the
sentinel path `<no keydb in search paths>`.
## API Usage
AACS decryption is transparent to the application. The `Disc::scan()` method handles everything automatically:
AACS decryption is transparent to the application. `Disc::scan()` handles
everything automatically:
```rust
use libfreemkv::{Drive, Disc};
@@ -203,7 +57,6 @@ if disc.encrypted {
if let Some(ref aacs) = disc.aacs {
println!("AACS {}.0", aacs.version);
println!("Key source: {}", aacs.key_source.name());
println!("Disc hash: {}", aacs.disc_hash);
if let Some(mkb_ver) = aacs.mkb_version {
println!("MKB version: {}", mkb_ver);
}
@@ -212,111 +65,152 @@ if disc.encrypted {
}
}
// Read content -- decryption is automatic
let mut reader = disc.open_title(&mut session, 0).unwrap();
while let Some(unit) = reader.read_unit().unwrap() {
// unit is 6144 bytes of decrypted content
// Read content -- decryption is applied on read by the DiscStream decorator.
// Live disc does NOT go through the URL resolver: `input("disc://...")` returns
// Error::DiscUrlNotDirect by design.
let keys = disc.decrypt_keys();
let mut stream = DiscStream::new(
Box::new(drive),
disc.titles[0].clone(),
keys,
batch_sectors,
disc.titles[0].content_format,
false, // raw: false → decrypt on read
None, // halt
)?;
while let Ok(Some(frame)) = stream.read() {
// decrypted PES frames
}
```
The application never touches keys, never calls decryption functions, and never manages handshakes. All of that is internal to `Disc::scan()` and `ContentReader::read_unit()`.
The application never calls decryption functions and never manages the
drive-level handshake. It DOES own key resolution — see below.
### KEYDB Location
### Key resolution is the caller's job
`ScanOptions` controls where the KEYDB is loaded from. If no explicit path is set, the library checks:
`libfreemkv` is **lookup-free: it resolves no keys and reads no keydb.** There is
no `ScanOptions::with_keydb`, and `ScanOptions` has no keydb field — its only
scan input is the optional drive credentials for the live-drive authenticated
handshake.
1. `~/.config/aacs/KEYDB.cfg`
2. `/etc/aacs/KEYDB.cfg`
To specify an explicit path:
```rust
let opts = ScanOptions::with_keydb("/path/to/KEYDB.cfg");
let disc = Disc::scan(&mut session, &opts).unwrap();
```
The caller resolves a key out-of-band through a key source and applies it with
[`Disc::decrypt_with`]. `freemkv-keysources` is the crate that implements the
keydb and key-server sources; `ScanOptions::key_sources` takes them as
`Box<dyn KeySource>`.
### AacsState
After a successful scan, `disc.aacs` contains an `AacsState` with:
After a successful scan, `disc.aacs` contains an `AacsState`:
| Field | Type | Description |
|-------|------|-------------|
| `version` | `u8` | AACS version (1 or 2) |
| `bus_encryption` | `bool` | Whether bus encryption is active |
| `mkb_version` | `Option<u32>` | MKB version from disc |
| `disc_hash` | `String` | SHA-1 of Unit_Key_RO.inf (hex with 0x prefix) |
| `key_source` | `KeySource` | How keys were resolved |
| `vuk` | `[u8; 16]` | Volume Unique Key |
| `unit_keys` | `Vec<(u32, [u8; 16])>` | Decrypted unit keys (CPS unit number, key) |
| `read_data_key` | `Option<[u8; 16]>` | AACS 2.0 bus decryption key |
| `volume_id` | `[u8; 16]` | Volume ID from SCSI handshake |
| `disc_hash` | `String` | Identifier for the disc's key-input files |
| `key_source` | `KeyOrigin` | How the disc's key was resolved |
### KeySource
## keydb.cfg
| Variant | Description |
|---------|-------------|
| `KeyDb` | VUK found directly in KEYDB by disc hash |
| `KeyDbDerived` | Media key + Volume ID from KEYDB, VUK derived |
| `ProcessingKey` | MKB + processing keys from KEYDB |
| `DeviceKey` | MKB + device keys, subset-difference tree traversal |
`keydb.cfg` is the single source of AACS key material. It is a text file (lines
starting with `;` or `#` are comments) holding the host credentials and per-disc
entries the library uses to resolve a disc. autorip can auto-download and
refresh it from a configured URL. The library does not ship any AACS keys
compiled into the binary.
## `UnitKey::is_default_index` test rationale
## KEYDB.cfg Format Reference
`is_default_index` is the public predicate that separates ordinary (index-0)
content keys from FMTS forensic index keys (see `UnitKey` docs; AACS 2.1
`IndividualSegment.tbl` tagging). A body answering `true` for everything would
present a forensic index key as an ordinary content key -- the caller would
decrypt the bulk of the title with a key that only opens 1/32nd of it;
answering `false` for everything would hide every ordinary key.
The KEYDB.cfg file contains all cryptographic material needed for AACS decryption. Lines starting with `;` or `#` are comments.
`is_default_index_separates_the_two_constructors` is pinned against the two
named constructors, which are the contract: `UnitKey::new` builds the ordinary
key, `UnitKey::forensic` builds an index key for `1..=32`.
### Device Keys
`is_default_index_agrees_with_the_forensic_index_resolver` checks the
predicate agrees with the one consumer of `index_number` in the crate:
`crate::aacs::index_select::resolve_disc_index` resolves the disc's forensic
index from exactly the keys that are NOT default. If the two disagree, a disc
resolves an index whose key the rest of the pipeline treats as ordinary (or
vice versa).
```
| DK | DEVICE_KEY 0x<key> | DEVICE_NODE 0x<node> | KEY_UV 0x<uv> | KEY_U_MASK_SHIFT 0x<shift>
```
## `src/aacs/mod.rs` module notes
- `key`: 16-byte AES device key (hex)
- `node`: Device node number in the subset-difference tree (hex)
- `uv`: UV value for tree positioning (hex)
- `shift`: U mask shift value (hex)
### Processing Keys
```
| PK | 0x<key>
```
- `key`: 16-byte pre-computed processing key (hex)
### Host Certificate
```
| HC | HOST_PRIV_KEY 0x<privkey> | HOST_CERT 0x<cert>
```
- `privkey`: 20-byte ECDSA private key (hex)
- `cert`: 92-byte AACS host certificate (hex)
The host certificate is used for SCSI authentication. It contains the host's public key and is signed by the AACS Licensing Administrator.
### Disc Entries
### KEYDB.cfg line format
```
| DK | DEVICE_KEY 0x... | DEVICE_NODE 0x... | KEY_UV 0x... | KEY_U_MASK_SHIFT 0x...
| PK | 0x...
| HC | HOST_PRIV_KEY 0x... | HOST_CERT 0x...
| HC2 | HOST_PRIV_KEY 0x... | HOST_CERT 0x...
0x<disc_hash> = <title> | D | <date> | M | 0x<media_key> | I | 0x<disc_id> | V | 0x<vuk> | U | <unit_keys>
```
- `disc_hash`: 20-byte SHA-1 of Unit_Key_RO.inf (hex)
- `title`: Human-readable disc title
- `D`: Date tag, followed by release/rip date
- `M`: Media key tag, followed by 16-byte media key (hex)
- `I`: Disc ID tag, followed by 16-byte Volume ID (hex)
- `V`: VUK tag, followed by 16-byte Volume Unique Key (hex)
- `U`: Unit keys tag, followed by space-separated `<unit_num>-0x<key>` pairs
### Spec provenance tags
All fields after the title are optional. A minimal entry needs only the disc hash and VUK:
The crypto in `src/aacs/` carries `[TAG] §x.y` citations back to the published
AACS specification (Final Rev 0.953), so each primitive links to the section it
implements:
```
0x<disc_hash> = <title> | V | 0x<vuk>
```
- `[C]` -- AACS Introduction and Common Cryptographic Elements Book (primitives,
MKB/key-management).
- `[PR]` -- AACS Pre-recorded Video Book (Volume/Title Key layer).
- `[BD]` -- AACS Blu-ray Disc Pre-recorded Book (CPS Unit Key, Aligned Unit,
Block Key).
- `[RE]` -- reverse-engineered from real discs, cited only where the public
spec is silent (the `0x86` verify record and the Category-C MKB type values).
Inline comments are supported with `;`:
### AACS key-input path discovery (`AacsRole`, `role_paths`, `find_hddvd_aacs_dir`)
```
0x<disc_hash> = <title> | V | 0x<vuk> ; MKBv77
```
BD and UHD keep their key material under a fixed `/AACS/...` tree, so those
paths are constants. HD DVD keeps the equivalents in a reserved root directory
whose NAME is authoring-house-specific -- observed `ANY!` (Dukes of Hazzard)
and `AAC!` (Freedom / Memory-Tech), each with a `<name>!_BAK` mirror -- and
whose title-key file is NOT always `VTKF000.AACS` (Freedom ships
`VTKF090.AACS` + `VTKF100.AACS`). So the HD DVD files are DISCOVERED from the
parsed UDF tree (`find_hddvd_aacs_dir` + `role_paths`), never hardcoded.
Each key role (`AacsRole`) resolves to an ordered candidate list -- the BD/UHD
constants first, then whatever the HD DVD directory actually holds -- which
every reader walks with `read_first`, first-that-reads. No reader ever
branches on disc type: a BD/UHD disc has the `/AACS/` files so those win; an
HD DVD has none of them, so it falls through to the discovered entries.
Centralised so `resolve_vid_only`, `read_aacs_inputs`, `read_mkb_content`, and
`read_aacs_version` can never silently diverge the disc_hash / MKB / VID that
another reader feeds a key service.
`find_hddvd_aacs_dir` identifies the HD DVD AACS directory structurally, NOT by
a hardcoded name: the root child directory whose name ends in `!` (so the
`<name>!_BAK` backup mirror, which also ends in a non-`!` char, is not
mistaken for it) and which contains `MKBROM.AACS`. Observed real names:
`ANY!` (Dukes of Hazzard), `AAC!` (Freedom). A BD/UHD disc has no such
directory -> `None`.
`role_paths` builds the ordered candidate list: fixed BD/UHD paths first, then
the discovered HD DVD files. For `AacsRole::UnitKey` every `VTKF*.AACS` in the
directory is appended in sorted name order -- a disc may carry more than one
variant (Freedom: `VTKF090` + `VTKF100`), not just `VTKF000`.
`read_first` walks an AACS role's candidate paths and returns the first that
reads; `read` performs the actual per-path read (full file or bounded prefix),
so callers share the same first-present walk regardless of read style. Returns
`Error::AacsNoKeys` if no candidate is present. Generic over the path element
(`&str` or owned `String`) so it accepts the `Vec<String>` that `role_paths`
builds from the discovered HD DVD directory.
### Test: `a_bang_suffixed_directory_without_mkbrom_is_not_the_aacs_directory`
The `!`-suffix and the `MKBROM.AACS` presence test are BOTH required -- the
discovery is a conjunction, not a disjunction. The existing fixtures only ever
present a directory that satisfies both (`AAC!` with `MKBROM.AACS`) alongside
one that satisfies neither (`AAC!_BAK` -- which contains `MKBROM.AACS` but is
ALSO reached only after the real dir), so either half of the conjunction could
be dropped and the same directory would still be found. This test's fixture
has a directory that satisfies the name half and NOT the contents half: it
must not be picked. If it were, the HD DVD path would resolve `MKBROM.AACS`,
`CONTENT_CERT.AACS` and the title-key file under a directory that holds none
of them -- the disc reports "no AACS key files" and never rips.
+220
View File
@@ -0,0 +1,220 @@
# AC3 / E-AC3 parser notes (`src/mux/codec/ac3.rs`)
## Why a frame set is one sample, never split into separate tracks
A legacy AC-3 syncframe is a complete access unit on its own, but an E-AC-3
access unit (ETSI TS 102 366 / ATSC A/52 Annex E) is a whole FRAME SET: the
mandatory independent substream (`substreamid` 0) plus every DEPENDENT
substream frame that follows it, plus any ADDITIONAL independent substreams
(`substreamid` 1..7) and their own dependents. A decoder needs the whole set
to reconstruct the programme (a 5.1 independent substream plus a dependent
substream carrying the extra channels of a 7.1 programme, and the AC-3-core +
E-AC-3-dependent arrangement used for backwards-compatible Dolby Digital
Plus), and every substream of the frame set covers the SAME time period.
WHY the whole frame set is ONE sample, rather than splitting an additional
independent substream (an associated / commentary service) into a track of
its own: Annex E orders the substreams of a frame set inside a single
elementary stream, all covering one time period, and a decoder handed a frame
set renders the programme it is asked for from it — that is exactly what the
container's E-AC-3 sample and its `dec3`/`num_ind_sub` description denote.
Splitting would mean rewriting the bitstream (re-numbering `substreamid` so
the extracted service becomes substream 0, and rebuilding its frame sets),
because a substream numbered 1..7 with no substream 0 is not a conforming
stream. That is a transcode, not a remux; this parser is lossless, so the
frame set stays whole and player-side programme selection decides what is
heard.
## `SAMPLE_RATES` / fscod2
Sample rates indexed by fscod (0=48kHz, 1=44.1kHz, 2=32kHz). fscod=3 is
reserved in AC-3; in E-AC-3 it signals "fscod2" (reduced rates: 24/22.05/16
kHz, selected by byte-4 bits `[5:4]`). `frame_sample_rate` decodes fscod2 in
the E-AC-3 case; the table's index-3 entry (48 kHz) is only the fallback when
the header is too short to read fscod2.
## `MAX_AC3_BUF` derivation
An AC-3/E-AC-3 syncframe is at most 8192 bytes (the `frame_size > 8192`
reject) and an access unit is a whole frame set: up to 8 independent
substreams, each with up to 8 dependent substreams (ETSI TS 102 366 Annex E)
— 72 syncframes worst case. A cap of one straddling frame set plus slack
therefore has to exceed 72 × 8192 = 576 KiB. If the buffer grows past the cap
without yielding a frame (pathological / never-syncing input) it is dropped
and resync begins, rather than accumulating one PES worth of data per call
for the whole title.
## `scan_access_units`
Per ETSI TS 102 366 (ATSC A/52) Annex E an access unit is one FRAME SET: the
mandatory independent substream (`substreamid` 0) plus every DEPENDENT
substream that follows it, plus any ADDITIONAL independent substreams
(`substreamid` 1..7 — associated/commentary services) with their own
dependents. The access unit therefore closes only when the next
`substreamid`-0 independent substream (or, with `at_eos`, the end of the
stream) is seen, and it carries THAT substream's PTS and duration: every
other substream of the frame set describes the SAME time period and adds no
duration of its own.
`base_pts_ns` times the access unit that begins at `data[0]` — i.e. the
running cadence carried over from the previous call. `anchor` re-anchors the
running PTS to this PES's own timestamp at the first access unit that STARTS
in the newly-appended bytes: a PES timestamp applies to the first access unit
beginning in that PES, never to one that began in an earlier PES and is only
being completed (or was held) here.
Returns the emitted access units, the offset in `data` from which bytes must
be carried over to the next call, and the PTS to stamp on the access unit
that begins that carry-over.
## `substream_role`
Byte 2 of an E-AC-3 syncframe is `strmtyp(2) | substreamid(3) | frmsiz[10:8]`
(ETSI TS 102 366 Annex E BSI). `strmtyp` 0 and 2 are independent substreams
(type 2 being an independent substream that is not the first of the bit
stream); `strmtyp` 1 is the dependent substream.
Annex E defines a FRAME SET as independent substream 0 — mandatory, always
first — with its dependent substreams, followed by the OPTIONAL additional
independent substreams 1..7, each with their own dependents; all of them
carry the same time period. So the access-unit boundary is an independent
substream with `substreamid` == 0, and NOT merely "an independent substream":
an additional independent substream (an associated or commentary service)
sits INSIDE the frame set already open. Keying the boundary on `strmtyp`
alone made every such substream close the access unit and advance the
running PTS a second time over the same ~32 ms, doubling the timeline (about
a second of A/V drift per second of audio).
`strmtyp` 3 is reserved: its BSI layout is not defined, so its `substreamid`
bits cannot be trusted and it is treated as starting a fresh access unit — an
unknown frame is never merged into an unrelated programme, and never
discarded as an orphan either.
Legacy AC-3 (`bsid < 11`) has no substream structure — byte 2 there is the
crc1 field, never `strmtyp` — so it always starts an access unit.
A frame set MAY carry a substream as several consecutive syncframes of fewer
than six blocks each (Annex E allows numblkscod < 3). Those extra syncframes
are `substreamid` 0 too, so each starts its own access unit here rather than
merging into one frame set. That is deliberate: each carries its OWN
numblkscod-derived duration (see `frame_duration_ns`), so the timeline total
stays exact, and the substreams are still delivered in bitstream order.
## `frame_crc_ok`
Per ETSI TS 102 366 (ATSC A/52) the frame carries a CRC-16/ANSI (poly 0x8005,
init 0, non-reflected) over the bytes after the 2-byte syncword — i.e.
`crc16_ansi(&buf[2..]) == 0` covers `frame_size - 2` bytes; the trailing crc
word makes a clean frame's residue zero. A nonzero residue is a
~1-in-65536-certain sign of payload corruption, so the frame is dropped
(silence gap) rather than shipped as a glitch.
## `acmod_channels` bit layout
This is the AUTHORITATIVE channel count for the track header: the DVD IFO
`audio_attr_t.channels` nibble is a well-known unreliable/stale field, so the
muxer prefers this over the IFO-claimed count (the bitstream acmod is
authoritative; the IFO audio nibble is not trusted). LFE adds one channel
(e.g. acmod=7 + lfeon → 6 = 5.1).
Bit layout from the syncword (A/52 §5.3.2 BSI):
```text
byte 5: bsid(5) | bsmod(3)
byte 6: acmod(3) | [cmixlev(2) if acmod has a centre and acmod!=1]
| [surmixlev(2) if acmod has surround]
| [dsurmod(2) if acmod==2] | lfeon(1) | ...
```
`acmod` therefore always occupies byte-6 bits 7-5; `lfeon` follows a variable
number of optional 2-bit fields, so the decoder tracks the bit cursor.
## `ACMOD_CHANNELS` table
Base channel count per AC-3 `acmod` (A/52 Table 5.8), BEFORE the LFE. Index is
the 3-bit acmod value; add 1 when `lfeon` is set.
```text
0 = 1+1 (Ch1, Ch2) -> 2 4 = 2/1 (L,R,S) -> 3
1 = 1/0 (C, mono) -> 1 5 = 3/1 (L,C,R,S) -> 4
2 = 2/0 (L, R) -> 2 6 = 2/2 (L,R,SL,SR) -> 4
3 = 3/0 (L,C,R) -> 3 7 = 3/2 (L,C,R,SL,SR) -> 5
```
## Test helpers and coverage notes
### `finalize_ac3_crc`
Relies on the CRC-16/ANSI residue property: appending `crc16([2..n-2])`
(big-endian) zeroes the register over `[2..n]`.
### `the_working_buffer_is_reused_across_packets_not_reallocated`
`parse` runs once per PES packet — of the order of 10^5 times on a feature's
audio track — and used to build a fresh `Vec` every call. The copy itself
cannot go without restructuring the borrow relationship, but the allocation
can: the buffer is taken out of `self`, filled, and put back with its
capacity intact. Asserting on capacity is deliberately white-box, because
that is exactly what regresses if someone reverts to `to_vec()` — the
behaviour would be identical and no other test would notice.
### `make_eac3_frame` byte layout
ETSI TS 102 366 Annex E: byte 2 is strmtyp(2) | substreamid(3) | frmsiz[10:8],
byte 3 is frmsiz[7:0], and the frame is `(frmsiz + 1) * 2` bytes. byte 4 =
fscod 0 (48 kHz), numblkscod 3 (6 blocks → 1536 samples → 32 ms), acmod 7 +
lfeon (5.1); byte 5 = bsid 16 so the E-AC-3 paths are taken.
### `a_held_access_unit_is_not_rescanned_from_its_first_frame_every_packet`
An access unit closes only at the next `substreamid`-0 independent substream,
so one that keeps gaining dependent substreams stays OPEN across PES
boundaries and its bytes stay in the carry-over. The carry-over must not be
re-scanned — and re-CRCed — from the access unit's first byte on every
packet: the buffer only stops growing at `MAX_AC3_BUF` (1 MiB), and a PES on
a DVD is about 2 KiB, so re-deriving the held access unit costs work
quadratic in the packets fed.
Measured directly, because a work bound has no frame-level shadow:
`frames_scanned` counts the syncframes the scanner sizes and CRC-gates.
Re-scanning from byte 0 examines 1 + 2 + ... + (N+1) frames.
Mutation: pass `None` for `held` in `parse` (or drop the `if let Some(h) =
held` resume) — the count returns to the quadratic figure.
### `a_discontinuity_drops_the_held_access_unit_with_its_bytes`
`parse` clears `self.acc` on a discontinuity because the buffered bytes are a
truncated frame. The held access unit is described by OFFSETS into exactly
those bytes, so it has to go with them. Without `self.held = None`, the next
packet resumes a HeldAu whose `start`/`end` were computed against the
pre-gap buffer but are applied to the unrelated post-gap bytes — splicing
audio across the gap at best, and indexing past the end of the new, shorter
buffer at worst.
The two existing discontinuity tests use plain AC-3 (bsid < 11), which never
holds an access unit open, so neither of them reaches this reset.
### `a_held_access_unit_is_dropped_when_the_track_poisons_before_it_resumes`
Mutation this catches: deleting (or inverting) the resume-path re-check `if
drop_reason.is_none() && self.tally.is_poisoned()` at the top of
`scan_access_units`. The verdict on a held access unit is frozen at the
moment it was OPENED, and `ac3_drop_reason` reads the tally BEFORE the
previous access unit is closed — so the very drop that crosses the poison
threshold lands after the next unit's verdict was already taken as `None`.
Without the re-check that unit is emitted as an ordinary frame after the
track has been judged too damaged to mux: corrupt audio passed through as
success, and worse, it is the ONE frame that escapes a whole-track fallback
whose entire point is that nothing after the verdict ships.
Neither existing held-AU test reaches this: both keep a pristine tally.
The fixture drives the exact interleaving above. `DropTally` poisons once
`verified_dropped * 2 > kept + dropped` past the 200-AU gate, so one PES
carries 200 CRC-failing `substreamid`-0 syncframes followed by a clean one.
Closing corrupt unit #200 (which happens only when the clean frame is
reached) latches the poison — after that clean frame's own verdict was
computed. Being an E-AC-3 unit that can still gain substreams, it is then
HELD across the boundary with `drop_reason: None`, which is precisely the
state the re-check exists for.
+37
View File
@@ -0,0 +1,37 @@
# ADTS decodability gate
## Module rationale (`src/mux/codec/adts.rs`)
Per the ADTS framing defined in ISO/IEC 13818-7 / ISO/IEC 14496-3, a header
is structurally invalid in exactly three ways this gate treats as hard
rejects: syncword != 0xFFF, a reserved `sampling_frequency_index` (the sample
rate table has 13 valid entries, so index >= 13 is reserved), and
`aac_frame_length < 7` (shorter than the fixed+variable header itself). The
optional 16-bit ADTS CRC is not verified here — it is simply skipped. So the
gate enforces those three rejects: a packet that begins with the ADTS sync
but is otherwise malformed is dropped; a packet with no ADTS sync is raw AAC
(e.g. from an MP4 container, which carries no ADTS header) or a continuation
and passes through unchanged — never false-dropped. Raw AAC has no per-frame
integrity data, so like LPCM it cannot be gated.
## Test: `a_crc_present_header_shorter_than_its_own_crc_is_invalid`
A header that CLAIMS a CRC (protection_absent = 0) but declares a frame
length too short to contain one.
`aac_frame_length` counts the header and the CRC, not just the payload,
so with a CRC present the smallest structurally possible frame is 9
bytes: the 7-byte fixed+variable header plus the 16-bit crc_check.
The gate compared against a flat 7 and never read protection_absent at
all, so a frame whose own header says it is impossible was classified
Valid and forwarded to the muxer.
## Test: `dropped_frames_are_counted_but_their_duration_is_not_invented`
A dropped ADTS frame is dropped BECAUSE its header failed validation, so
the very fields a duration would come from (sampling_frequency_index, and
the 1024-samples-per-AAC-frame constant applied to it) are the ones known
to be untrustworthy. This gate therefore reports the drop's duration as
zero rather than deriving a number from a header it has just rejected —
the honest answer, and the one the count alongside it must be read with.
A nonzero constant here would report silence that was never measured.
+12 -10
View File
@@ -170,24 +170,27 @@ libfreemkv/src/
│ ├── writeback_file.rs WritebackFile (was crate::io::Writer)
│ └── writeback.rs sync_file_range pipeline
├── drive/ Drive (open, init, single-shot read)
│ ├── mod.rs Drive struct, init, read (single-shot), reset, eject
│ ├── capture.rs Drive profile capture for contribution
│ ├── mod.rs Drive struct, init, read (single-shot), eject
│ ├── capture.rs Raw drive SCSI capture (INQUIRY/GET_CONFIG) for contribution
│ ├── linux.rs Linux drive discovery
│ ├── macos.rs macOS drive discovery
│ └── windows.rs Windows drive discovery
├── disc/ Disc (scan, titles, AACS setup, sweep, patch)
├── disc/ Disc (scan, titles, AACS setup, per-format parsing)
│ ├── mod.rs Disc struct, scan, titles, formats
│ ├── sweep.rs Disc::sweep (Pass 1 forward sweep)
│ ├── patch.rs Disc::patch (Pass N retry over mapfile)
│ ├── mapfile.rs ddrescue-format mapfile
│ └── read_error.rs ReadCtx / ReadAction state machine
│ ├── bluray.rs Blu-ray / UHD scanning (MPLS/CLPI-driven)
│ ├── dvd.rs DVD-Video scanning (IFO-driven)
│ ├── hddvd.rs HD-DVD scanning
│ ├── extract.rs Per-extent content extraction
│ ├── encrypt.rs Encrypted-range mapping for content reads
│ ├── dvd_audio_probe.rs DVD audio-stream probing
│ └── pgs_forced_probe.rs PGS forced-subtitle probing
├── scsi/ SCSI transport (Linux SG_IO, macOS IOKit, Windows SPTI)
├── platform/ Drive unlock (MT1959 A/B)
├── unlock.rs Unlocker trait + registry (pluggable unlock seam)
├── aacs/ AACS decryption (handshake, keys, keydb, decrypt)
├── css/ DVD CSS cipher
├── decrypt.rs Unified decrypt dispatcher (AACS/CSS/None)
├── pes.rs PES frame types, FrameSource / FrameSink traits
├── sector/ Sector I/O (was sector.rs in 0.17)
├── sector/ Sector I/O
│ ├── mod.rs SectorSource, SectorSink traits
│ ├── file.rs FileSectorSource, FileSectorSink (ISO-backed)
│ └── decrypting.rs DecryptingSectorSource decorator
@@ -198,7 +201,6 @@ libfreemkv/src/
├── labels/ BD-J label extraction (5 format parsers)
├── keydb.rs KEYDB download, parse, save
├── identity.rs DriveId from INQUIRY
├── profile.rs Bundled drive profiles
├── speed.rs DriveSpeed enum
├── mux/
│ ├── mod.rs Public mux exports
+31 -28
View File
@@ -1,11 +1,13 @@
# libfreemkv Architecture
Open source optical drive access library for 4K UHD Blu-ray, Blu-ray, and DVD.
Rust library with no external dependencies at runtime -- profiles are bundled,
AACS keys are derived internally, and all SCSI communication is handled in-process.
Rust library with profiles bundled and all SCSI communication handled in-process.
AACS decryption requires an external `keydb.cfg` (default
`~/.config/freemkv/keydb.cfg`) — the derivation math is internal, but no AACS key
material is compiled in; DVD CSS player keys are the only compiled-in keys.
**Repository:** <https://github.com/freemkv/libfreemkv>
**License:** AGPL-3.0-only
**License:** MIT
---
@@ -15,9 +17,10 @@ AACS keys are derived internally, and all SCSI communication is handled in-proce
format handling live in the library. CLI binaries are thin wrappers that call
`Drive::open()` and `Disc::scan()`.
2. **No external files.** Bundled drive profiles are compiled into the binary via
`include_str!`. No configuration directory, no runtime file lookups for drive
support.
2. **Firmware-clean core.** libfreemkv ships no firmware, no unlock CDBs, and no
drive profiles. Drive-unlock logic is plugged in by an external crate through
the `Unlocker` trait + registry (`register_unlocker`); without one the library
still rips via the host-certificate AACS handshake.
3. **Transparent AACS.** The `ContentReader` decrypts on the fly when keys are
available. Callers read cleartext sectors without knowing whether the disc
@@ -41,11 +44,9 @@ AACS keys are derived internally, and all SCSI communication is handled in-proce
libfreemkv (lib.rs)
│
├── Drive Access
│ ├── drive Drive — open, identify, init, unlock, single-shot read
│ ├── drive Drive — open, identify, init, single-shot read
│ ├── scsi ScsiTransport trait + platform backends (sg async, IOKit, SPTI)
│ ├── platform/ Platform trait — per-chipset command handlers
│ │ └── mt1959 MediaTek MT1959 driver (LG, ASUS, HP)
│ ├── profile DriveProfile loading, matching, bundled JSON
│ ├── unlock Unlocker trait + registry — the pluggable unlock seam
│ ├── identity DriveId from INQUIRY + GET_CONFIG 010C
│ ├── speed DriveSpeed enum, SET CD SPEED CDB builder
│ └── event Event system for drive status callbacks
@@ -65,7 +66,7 @@ libfreemkv (lib.rs)
│
├── Streaming
│ ├── mux/ Stream implementations (Disc, ISO, MKV, M2TS, Network, Stdio, Null)
│ ├── pes PES frame types; FrameSource / FrameSink direction-typed traits
│ ├── pes PES frame types; the unified pes::Stream (PesStream) read/write trait
│ └── sector/ SectorSource / SectorSink traits, FileSector{Source,Sink}, DecryptingSectorSource
│
├── I/O Primitives
@@ -74,8 +75,7 @@ libfreemkv (lib.rs)
│
├── Support
│ ├── keydb KEYDB.cfg download, parse, verify, save
│ ├── error Error enum with numeric codes E1000-E8000
│ └── profile Bundled drive profiles
│ └── error Error enum with numeric codes E1000-E8000
│
└── lib.rs Public API re-exports
```
@@ -89,20 +89,20 @@ Drive::open(Path::new("/dev/sg4"))
│
├─ scsi::open() Open /dev/sg4 (async write/poll/read)
├─ DriveId::from_drive() INQUIRY + GET_CONFIG 010C
├─ profile::find_by_drive_id() Match against bundled profiles
├─ Platform::new() Instantiate chipset driver (Mt1959)
└─ Drive ready for init/unlock/read
└─ Drive ready for init/read
```
After open:
- `init()` -- unlock + firmware upload + speed calibration
- `init()` -- routes to the matching registered unlocker (if any); otherwise
a no-op and the cert handshake carries the disc
- `probe_disc()` -- probe disc surface for optimal speeds
- `read(lba, count, buf, recovery)` -- single-shot read; `recovery` only selects the per-CDB timeout (1.5 s vs. 30 s)
- `read(lba, count, buf, recovery)` -- single-shot read; `recovery` only selects the per-CDB timeout (`READ_TIMEOUT_MS` 10 s vs. `READ_RECOVERY_TIMEOUT_MS` 60 s)
- `wait_ready()` -- wait for disc insertion
- `eject()` -- eject tray
Recovery is layered above `Drive::read`, not inside it. Layer 1
(`Disc::patch`) handles bad-range retry by replaying the ddrescue mapfile.
(`freemkv_engine::recovery::patch`, in the engine crate) handles bad-range
retry by replaying the ddrescue mapfile.
Layer 3 (`DiscStream::fill_extents` adaptive batch sizer) handles in-loop
request-size adaptation. Inline recovery (gentle retry → SCSI reset → retry)
was removed in 0.13.6 — see [`rip-recovery.md`](rip-recovery.md) and
@@ -195,17 +195,20 @@ implementing `execute()` for that OS and wiring it into `scsi::open()`.
---
## Chipset Support
## Drive Unlock
| Chipset | Drives | Status |
|---------|--------|--------|
| MediaTek MT1959 | LG, ASUS, HP | Supported (bundled profiles) |
| Renesas RS8xxx/RS9xxx | Pioneer, some HL-DT-ST | Planned |
libfreemkv carries no drive-unlock mechanism. The `Unlocker` trait + registry
(`src/unlock.rs`) is the seam: an external crate implements `Unlocker` and
registers it once via `register_unlocker(...)`. At drive-prep the registry is
walked in order and the first unlocker whose `matches()` is true is asked to
`unlock_drive()` over the raw `ScsiTransport`. If none match, the drive is left
untouched and the host-certificate AACS handshake carries the disc.
The `Platform` trait abstracts chipset-specific commands. Each chipset implements
handlers (unlock, config, register, calibrate, keepalive, status, probe,
read_sectors, timing). All handlers are accessed via SCSI READ BUFFER with
chipset-specific mode and buffer ID bytes.
The implementor owns everything firmware-specific — drive profiles, vendor CDBs,
variant logic. Concrete unlockers live in the separate
**[freemkv-unlock](https://github.com/freemkv/freemkv-unlock)** repository, never
in libfreemkv. See [`drive-access.md`](drive-access.md#drive-unlock-seam) for the
trait definition and routing.
---
+217
View File
@@ -0,0 +1,217 @@
# `src/mux/au_assembly.rs` — access-unit assembly
## Why this exists
The contract a codec parser converts is `PES → access units (Frames)`. A
*transport* stream hands the parser one AU per PES for free (BD aligns one
access unit per PES; the TS demuxer reassembles to the
`payload_unit_start_indicator`). A *program* stream does not — the PS muxer
chops the elementary stream into fixed-size PES fragments with no AU
alignment, and only the first fragment of an AU carries a PTS. So a parser
that assumes one-AU-per-PES (h264/hevc/vc1, written against TS) mis-frames a
program stream, while `mpeg2` — the DVD/PS codec — must reassemble across PES.
`AuAssembler` is that reassembly, factored out so EVERY program-stream video
parser shares one implementation instead of hand-rolling the buffer. The
h264/hevc/vc1 parsers (`Mode::StartCode` / `Mode::Vc1`) and the MPEG-2 parser
(`Mode::Mpeg2`, via `AuAssembler::mpeg2`) all drive it. It buffers PES-fragment
bytes and emits one AU per codec AU boundary, carrying the AU-start
timing/source forward. Since the boundary is a codec start code, it lives with
the codec parser (which picks the marker); only the generic buffering +
timing-carry is shared here.
This is *inside* the parser, not a pipeline stage: the pipeline stays
`Demuxer → PES → Parser → Frames`, and the demuxer stays codec-agnostic. Every
stream a parser sees runs through one of these — self-framing codecs (MPEG-2,
audio) use `Mode::Passthrough` so the parser code path is uniform.
## `MAX_AU_BUFFER`
Safety cap on a single in-progress access unit. A real coded picture is far
below this; a stream that never yields a second AU boundary is force-flushed
at the cap rather than buffering without bound on hostile/corrupt input.
## `MAX_MARKS`
Cap on buffered timing/discontinuity marks. A real access unit spans a few
hundred PES fragments at most; this bounds the mark deques so a run of
zero-length (or start-code-free) timed fragments — which grow no buffer bytes
and so never trip the `MAX_AU_BUFFER` mark-prune — cannot accumulate marks
without bound on hostile/corrupt disc input.
## `AuAssembler::for_codec`
Video codecs whose parsers assume AU-complete PES (H.264 / HEVC / VC-1) get a
`Mode::StartCode` assembler; MPEG-2 (self-reassembles) and audio/subtitle
codecs (self-framing) get `Mode::Passthrough` so callers can run every stream
through this uniformly.
## `AuAssembler::mpeg2`
The MPEG-2 parser owns one of these directly (rather than hand-rolling the
buffer): the demux layer runs MPEG-2 through `Mode::Passthrough` and hands
each fragment to the parser, which feeds them here to be reframed on picture
boundaries.
## `AuAssembler::push`/`push_owned`
For a self-framing (`Passthrough`) stream the payload is MOVED straight into
the emitted unit with no copy — the common DVD/HD-DVD case (MPEG-2 video, all
audio). A buffering mode copies into `buf` exactly as `push`.
## `take_front`
Detaches `buf[..end]` as the emitted AU's own `Vec` and leaves `buf` holding
the tail.
The AU's bytes are HANDED OVER — `buf`'s allocation becomes the returned `Vec`
and a fresh buffer (pre-sized to the same capacity, so the next AU accumulates
without re-growing) takes its place holding only the short tail.
`buf[..end].to_vec()` + `drain(..end)` instead copied every AU out in full: on
a UHD HEVC title that is a whole-frame memcpy (hundreds of KB) per coded
picture, ~200k times, for bytes that are about to be discarded from `buf`
anyway.
The allocation COUNT is unchanged (one per AU either way — the frame `Vec`
before, the replacement buffer now), so the only difference is the copy that
no longer happens. Nothing depends on `buf` keeping its identity: the only
state tied to `buf[0]`'s position is `base`/`scan_pos`/`opener_pos`, which the
caller updates immediately after.
Falls back to a copy when the buffer's capacity is far larger than the AU (a
small AU after a multi-MB one): handing over would otherwise attach an
oversized idle allocation to a small frame for as long as the frame queues
downstream, trading a copy for resident memory.
That fallback must not become permanent. `buf`'s capacity used to be a one-way
high-water mark — the replacement buffer was created with `cap.max(tail_len)`,
and the copy path's `drain` also preserves `cap` — so once ONE large AU had
been assembled, every later smaller AU satisfied `cap > 2*end` and took the
copy path forever. On a UHD HEVC title the first IDR grows `buf` to ~4-8 MB,
after which each ~200-400 KB P/B AU paid a whole-AU allocation plus a
whole-AU memcpy plus a tail memmove for ~99% of the ~200,000 coded pictures —
tens of GB of exactly the memcpy this handover exists to remove. So the copy
path now also RELEASES the high-water capacity, which re-arms the handover
for the next AU: one copy after a size step down, not one per frame forever.
## `drop_marks_before`
This is the STREAM-START case: bytes ahead of the first AU boundary are the
tail of an access unit that began before we had sync, and there is no prior
AU for them to be discontinuous *from*. Carrying a mark forward here would arm
the resync gate at the head of every title and drop its first GOP.
## `discard_gap_before`
This is the BACKSTOP case: `MAX_AU_BUFFER` bytes accumulated with no AU start
code in them, so the run is unusable and gets thrown away. Unlike the
stream-start trim above, there IS a prior AU here, and whatever follows
definitively does not continue it — a decoder handed the next picture would
resolve its references against frames separated from it by megabytes of
discarded data.
So the discard is itself a discontinuity, whether or not the source signalled
one. It is recorded as a sticky flag rather than an offset mark because a
mark placed at the new base would be retired moments later by the pre-sync
trim that follows resync — the gap has to outlive the bytes that caused it.
It arms the resync gate, which drops to the next keyframe instead of emitting
a picture with dangling references.
Timing marks before `off` are still retired — they describe bytes that no
longer exist, and the AU that eventually emits takes its PTS from the
fragment that actually opened it.
## Tests
### `a_backstop_discard_marks_the_next_au_discontinuous`
The 8 MiB backstop throws away a start-code-free run as unusable. The AU that
eventually emits after that discard MUST be marked discontinuous, whether or
not the source ever signalled a discontinuity: megabytes of the stream are
simply gone, so the next picture cannot resolve its references against the
last one that was emitted.
`discontinuity` is what arms the resync gate downstream (`resync.rs`, driven
from `mux/disc.rs`), which drops to the next keyframe rather than emitting a
picture with dangling references. If the flag is retired with the discarded
bytes, the gate never arms and the broken picture goes out — a silent
corruption, which is the one class of loss this crate refuses to have.
### `a_stream_start_trim_does_not_mark_the_first_au_discontinuous`
The opposite case, and the reason the two call sites are separate. Bytes
ahead of the FIRST access-unit delimiter are the tail of an AU that began
before we had sync. There is no prior AU for them to be discontinuous from, so
retiring the marks there is right — and necessary: marking the first AU of
every title discontinuous would arm the resync gate at the head of each one
and drop its opening GOP.
### `a_source_signalled_discontinuity_reaches_the_au_it_opens`
A source-signalled discontinuity reaches the AU it opens. This is the
`disc_marks` path — the ORIGINAL mechanism, distinct from the sticky
`pending_gap` the backstop sets. Nothing else pins it: the two tests above
drive `pending_gap`, and a mark placed on a fragment that is later discarded
is retired by design.
Deliberately NOT combined with the backstop. A previous version of this test
signalled the discontinuity on the first over-cap push and asserted the flag
on the AU after the discard — but that first run still has the next AU's
delimiter at `buf[0]`, so it force-flushes as an over-long AU, and THAT AU
consumes the mark. The assertion was then satisfied entirely by
`pending_gap`, making the test a duplicate of the one above it under a name
promising something else. The two mechanisms cannot be isolated in one
fixture, so they get one test each.
### `drained_au_takes_over_the_buffer_allocation_without_copying`
MEASURED: a drained AU must be HANDED the accumulation buffer's allocation,
not copied out of it. The emitted `Vec`'s data pointer is the buffer's own
pointer — which is only true if no full-frame copy happened.
(`buf[..end].to_vec()` allocates fresh, so the pointers differ.) One
whole-AU memcpy per coded picture is ~200k memcpys of a few hundred KB each
on a UHD feature.
### `handover_survives_a_large_au_instead_of_copying_every_later_one`
MEASURED: `take_front`'s copy fallback must not become permanent.
`buf`'s capacity used to be a one-way high-water mark, and the copy path's
`drain` preserves it, so after ONE large AU every later smaller AU satisfied
`cap > 2*end` and copied forever. On a UHD HEVC title the first IDR grows
`buf` to multiple MB, after which ~99% of the ~200,000 coded pictures each
paid a whole-AU allocation + whole-AU memcpy + tail memmove — tens of GB of
exactly the copy the handover exists to remove. Counted at the copy path
itself: one copy is expected right after the size step down; a per-frame
copy is the bug.
### `discarded_pre_sync_marks_do_not_time_the_first_access_unit`
After the pre-sync bytes are discarded, the emitted AU must take the timing
of the fragment that ACTUALLY opened it. `drop_marks_before` is what retires
the discarded fragment's marks; a no-op there stamps the first real access
unit with the PTS and source of bytes that were thrown away — a whole-title
A/V sync offset, since every later frame is timed relative to it.
### AU-opener detection: the per-mode start-code rule
`au_opener_from` duplicates a rule each codec parser also encodes; the tests
below pin it to the normative byte values so a drift between copies shows up.
### `au_opener_from_locates_the_real_start_code_per_codec`
The opener offset must be the position of the real start code, never a fixed
0. A constant `Some(0)` makes every pre-sync run of junk bytes look like the
head of an access unit, so the first AU of every stream that does not begin
exactly on a start code is emitted with junk glued to its front.
### `for_codec_routes_each_video_codec_to_its_reassembly_mode`
`for_codec` is the dispatch that decides whether a stream is REASSEMBLED
across PES fragments or passed straight through. Getting it wrong is silent:
an H.264/HEVC/VC-1 stream routed to `Passthrough` on a program source emits
one "frame" per PES fragment — a few hundred bytes of a coded picture, framed
as a whole access unit — and the output plays as corruption, not as an error.
Each mode is identified BEHAVIOURALLY (feed a two-AU stream in two halves and
see whether it reassembles), so the case cannot pass by matching a constant.
+42
View File
@@ -0,0 +1,42 @@
# BDMV disc-library metadata (`bdmt.rs`)
Every commercial Blu-ray carries a disc-library metadata directory with one
XML file per shipped language. The schema is the Blu-ray "disc library
metadata" namespace (`urn:BDA:bdmv;disclibmeta`), conventionally prefixed
`di:`. Fields commonly present:
- `<di:title>` or `<di:name>` — the title string. Vendor practice varies
(Paramount discs tend to use `<di:name>`).
- `<di:description>` — optional synopsis (often absent on retail discs;
common on box sets and special editions).
- `<di:discNumber>` / `<di:numSets>` (or `<di:numberOfSets>`) — set position
for multi-disc releases.
This module is intentionally separate from the BD-J `StreamLabel` parsers
under `labels/*.rs`. The XML here is disc-level (title, description, set
position), not per-stream.
Real-world XML is irregular: missing description elements, multiple title
elements (first one wins), and occasional malformed content. Extraction is
best-effort — a malformed file is treated as "no metadata" (returns `None`
from the helper), and the caller can still get metadata from
sibling-language XML files.
## `MAX_BDMT_BYTES`
The size comes from attacker-controlled UDF metadata; real files are a few
KB, so 1 MiB is generous while preventing a crafted huge-size entry from
triggering an oversized allocation in `read_file`.
## `parse_bdmt_xml`
Title-element preference: `<di:name>` → `<di:title>` →
`<di:tableOfContents>/<di:titleName>` (first match wins, per the
authoring-tool conventions documented above).
## `looks_like_xml`
Rejects candidate description strings that are themselves XML fragments —
observed on a captured disc, where `<di:description>` contained
`<di:thumbnail href="…"/>` child elements and no actual prose. Surfacing
that raw to the JSON output is worse than dropping the field entirely.
+21
View File
@@ -0,0 +1,21 @@
# bdnav — HDMV navigation resolver
Notes relocated from `src/bdnav/mod.rs` doc comments (kept out of the source
to respect the comment-guard's prose caps).
## No per-disc special-casing
There is no per-disc special-casing: one correctly-booted VM resolves
densely-branched dispatchers as well as simple ones. The honest boundary is
BD-J (the feature is chosen by a Java Xlet, not by the HDMV VM); there — and
on any malformed data or non-convergence — the resolver ABSTAINS (`None`)
and selection falls back to the structural/heuristic order.
## `resolve_feature`
Resolves the playlist id the disc's First-Play navigation plays as the
feature, restricted to ids the caller marks as feature candidates
(`is_feature_candidate` — typically video-bearing, non-trivial titles, so a
short logo/pre-roll `PlayPlayList` is skipped). Returns `None` for BD-J
discs, missing/malformed nav data, or when navigation does not converge on
a candidate.
+239
View File
@@ -0,0 +1,239 @@
# src/disc/bluray.rs — extended notes
Overflow prose for comments that were trimmed to the internal 3-line cap in
`src/disc/bluray.rs`. Each section is pointed to by a `// See docs/bluray.md
— <topic>` comment at the corresponding site.
## CLIP_STREAM_EXTS
Stream-file extensions probed for a BD-family playlist clip, in priority
order. A clip is normally `.m2ts`; AACS 2.1 (FMTS) discs name the main
feature `.fmts` (an M2TS transport stream plus forensic variant segments)
and 3D discs use `.ssif`. `.m2ts` is tried first, so a normal clip is
unaffected — the fallback only runs when `.m2ts` is absent (exactly when
`file_extents` errors).
Scope: these are all variants that live in `BDMV/STREAM/` and are reached
through an MPLS playlist. HD-DVD's `.evo` does NOT belong here — HD-DVD is
a different tree (`HVDVD_TS/`) with `.XPL` playlists and needs its own
enumerator (a peer to `parse_playlist`), not another extension in this
list.
## scan_bluray_titles
Cancellation: `halt` is polled between playlists and once more after the
loop, and a read that fails with `Error::Halted` — how a live drive
reports a Stop, since `Drive::checked_exec` fails every command once its
flag is set and `Drive::read` preserves the variant — is propagated rather
than swallowed. `Halted` is the only error this returns from the
enumeration itself; an unreadable or unparseable playlist keeps its
best-effort skip.
It has to be an error and not a short title list, for the same reason
spelled out on `Disc::scan_hddvd_titles`: a cancelled enumeration that
returned `Ok` would be indistinguishable from a disc that genuinely holds
fewer titles, and the caller would cache and act on it. Before this, a Stop
pressed mid-scan failed every remaining `.mpls` read in turn, each one
silently skipped, and the scan returned `Ok(truncated)` at rc=0.
## parse_playlist
Sums PlayItem durations; returns `Ok(None)` if the playlist is under 30
seconds (skips menu / clip-info stub playlists), fails to parse, or names a
clip that cannot be resolved. Physical sector extents are pulled from the
UDF allocation descriptors of each referenced `.m2ts` (deduplicated by
clip_id).
`Ok(None)` keeps its two benign meanings (unparseable MPLS, sub-30s
playlist) plus the deliberate "drop this title" outcomes below. `Err` means
the scan is over: today that is only `Error::Halted`, the operator's Stop,
which must not be reported as a disc that simply holds fewer titles.
## Test: parse_playlist_keeps_exactly_30_seconds
At exactly 30s the playlist is kept (`< 30.0` is strict). The fixture is a
fully wired BDMV (`make_bdmv_fs`), not the bare `make_min_fs` this used to
use. `make_min_fs` lays no STREAM/CLIPINF, so the play item's `.clpi` read
now fails and the title drops — which is correct behaviour for an
unresolvable clip but has nothing to do with the 30-second boundary this
test exists to pin. Wiring the clip keeps the test measuring exactly one
thing.
## make_bdmv_fs_ssif
Full BDMV with a real Blu-ray 3D layout: `.ssif` files under
`BDMV/STREAM/SSIF/<clip>.ssif` (note the SSIF subdirectory, unlike
`make_bdmv_fs_ext`) plus a matching `.clpi` in CLIPINF. Resolving the SSIF
is what latches `is_3d = true` in `parse_playlist`.
## Test: parse_playlist_fmts_clip_resolves_extent
AACS 2.1: the feature clip is `00001.fmts`, NOT `.m2ts`. The
`CLIP_STREAM_EXTS` fallback in `parse_playlist` must still resolve the
physical extent — before the fix the hard-coded `.m2ts` path errored,
yielding empty extents (a silent empty rip and 0 encrypted samples for key
resolution). Size still comes from the `.clpi`, which parses regardless.
## Test: parse_playlist_dedups_repeated_clip_extents_and_size
THE 0.31.0 DEDUP PATH. A playlist that references the SAME clip_id from
multiple PlayItems (seamless split / looped segment) must count the
physical extents and packet bytes exactly once — mux reads extents in
order, so a duplicate would mux the A/V twice and inflate `size_bytes`
(bluray.rs: `first_ref = seen_clips.insert(...)` gates both `total_size +=`
and the `extents.push`). Per-PlayItem Clip entries are still recorded for
both.
## Test: parse_playlist_missing_clpi_yields_no_title
A clip whose `.clpi` is missing must yield no title.
This test used to assert the opposite — a title with `size_bytes == 0` and
empty extents — and its docstring quoted the buggy control flow ("bluray.rs
only fetches extents inside the `if let Ok(clpi_data)`") as if it were the
specification. It was blessing the defect: the title's `duration_secs` is
summed from the PlayItems before the clip is resolved, so the returned
title advertised the movie's full runtime while carrying none of its
bytes, and the discarded error meant not one log line said so. That is the
flagship failure class of this crate — a failure that looks like success —
reached through an ordinary missing or scratched CLIPINF file.
The correct behaviour is the same as for a clip whose extents cannot be
resolved (see `parse_playlist_unreadable_clip_icb_yields_no_title`): drop
the title and log the read's own error code.
That last clause is now asserted, not merely asked for. The site's comment
insists on `e.code()` because a missing `.clpi` (E6003), a scratched one
(E6000) and a malformed one (E6002) are different populations, but nothing
checked it: putting a literal back compiled and passed. Mutation: `"E6017"`
(or any fixed code) in place of `"E{}", e.code()` fails here, and so does
dropping the warn entirely — a silent drop is the same invisible title
loss this test was written for, one step later.
## Test: parse_playlist_unrecorded_extent_yields_no_title
A clip stream whose ICB declares an UNRECORDED (ECMA-167 4/14.14.1.1
type-1) extent must not yield a title at all.
The extent is allocated to the file but was never written, so the file's
content there is zeros while the media holds whatever was left at those
sectors. Neither answer a `(lba, sector_count)` read plan can give is true
— reading it splices undefined sectors into the rip as content, dropping
it slides every later extent's byte space — so the title is refused rather
than mis-ripped. This fixture is the shape a crafted disc uses to get such
a range into a title's extent list.
(The `sectors > 0 && lba > 0` filter below the resolver stays as defence in
depth; a zero-length AD is only reachable as an unrecorded descriptor,
since a zero-length TYPE 0 one terminates the AD list.)
## Test: parse_playlist_unreadable_clip_icb_yields_no_title
An unrecorded extent was never the only way a clip fails to resolve.
RED BEFORE GREEN: this fixture gives the .m2ts an ICB whose descriptor tag
is neither 261 nor 266, so `file_extents` returns `DiscRead` — the same
variant a scratched sector under a real clip's ICB produces, which is the
ordinary way this happens on real media. Before the fix only
`UdfUnrecordedExtent` set the drop flag, so this fell through to the "file
absent" path and `parse_playlist` returned a title: full declared duration
from the play item, `total_size` already counted from the .clpi, and zero
extents — a movie advertising its runtime with the content missing, and
not one log line. The clip must drop the title exactly as an unrecorded
extent does.
## Test: parse_playlist_logs_a_non_absence_ssif_failure_the_m2ts_fallback_hid
A non-absence SSIF failure that the `.m2ts` fallback then papers over must
be logged, with its own code.
`unresolved` had exactly one reader — `if let (None, Some(code))` — so a
code recorded for `/BDMV/STREAM/SSIF/<clip>.ssif` was thrown away whenever
the fallback succeeded. The disc here IS a 3D disc: the SSIF is present and
carries both eyes, and only its ICB is unreadable (a scratched sector, the
ordinary way this happens). The title shipped base-view 2D at rc=0 with
`is_3d` false and not one log line, so the operator's rip is silently
missing the dependent view and the journal cannot tell this disc apart
from one that was only ever 2D.
The title is deliberately still returned — the base view resolved, so
refusing would trade a degraded rip for no rip. The defect being fixed is
the silence, not the fallback.
Mutations this catches: deleting the new `(Some(_), Some(code))` arm, or
restoring the `if let (None, Some(code))` shape, leaves no event to find;
logging a fixed code instead of `e.code()` fails the code assertion;
making the arm `return Ok(None)` fails the title assertion.
## Test: parse_playlist_id_falls_back_to_zero_when_suffix_is_not_mpls
A filename that is long enough (>= 5 bytes) but does NOT end in ".mpls"
must NOT have its last 5 bytes stripped — the whole string is handed to
the numeric parse instead, which fails and falls back to playlist_id 0
(bluray.rs `filename.len() >= 5 && filename[len-5..].eq_ignore_ascii_case(".mpls")`).
## Test: scan_bluray_titles_skips_non_mpls_extension_file
A non-directory PLAYLIST entry whose name does NOT end in ".mpls" must be
skipped even though its content parses as a perfectly good (long) MPLS
playlist — extension gating, not content sniffing, decides eligibility
(bluray.rs `!entry.is_dir && entry.name...ends_with(".mpls")`).
## HaltingReader
A `SectorSource` that fails every read in `halt_range` with
`Error::Halted` — exactly how a live drive behaves once the operator
presses Stop: `Drive::checked_exec` fails every SCSI command with `Halted`
from then on, and `Drive::read` deliberately preserves the variant. Reads
outside the range still succeed, so a test can aim the cancel at one
structure and leave the scan far enough along to have something to
truncate.
## Test: halted_playlist_read_is_not_reported_as_a_shorter_disc
A Stop on a live drive never touches `ScanOptions::halt`: `Drive` has its
own flag and `checked_exec` fails every SCSI command with `Error::Halted`
once it is set. The Blu-ray enumerator must not swallow that into a
successful scan.
RED BEFORE GREEN: with the propagation reverted this returned
`Ok([00800])` — the `if let Ok(mpls_data)` skipped the cancelled read of
00801.mpls, the loop ended, and a half-enumerated disc came back at
success. One title from a two-title disc is indistinguishable from a disc
that genuinely holds one title, and the caller caches and rips from it.
The halt lands on the second playlist deliberately: it is the last
iteration, so nothing after it would poll a flag — only propagating the
read's own error catches it.
## Test: halted_clpi_read_is_not_accounted_as_an_unresolvable_clip
The same cancel landing on a `.clpi` read must not be classified as an
unresolvable clip either.
RED BEFORE GREEN: with the `Err(Error::Halted)` arm removed from the
CLIPINF match, the cancel fell into the generic "clip could not be
resolved" arm, which logs a disc-defect code and drops the title —
accounting an operator Stop as a scratched disc, and (in the scan loop)
dropping every remaining playlist in turn for a truncated `Ok`. With the
fix the cancel is propagated with its own variant intact.
## Test: halted_extent_resolve_is_not_a_title_missing_its_clip
And the same cancel landing on `file_extents` — the clip's ICB, not its
CLIPINF — must propagate too.
RED BEFORE GREEN: `note` used to exempt `Halted` from the unresolved
classification (correctly — a cancel is not an authoring hole) but had no
way to propagate it, so the resolver simply produced no extents and
`parse_playlist` returned a title with the clip's runtime counted, its
`size_bytes` counted from the .clpi, and zero bytes behind it. Measured
with the propagation reverted: `Ok(Some((768000, [])))` — the flagship
defect shape, wearing a cancel.
## Test: read_meta_title_ignores_non_xml_file_regardless_of_content
A non-.xml file must be ignored even if its content looks like a valid
meta XML (contains a `<di:name>`) — extension gating, not content
sniffing, decides eligibility (bluray.rs `!e.is_dir &&
e.name...ends_with(".xml")`).
+61
View File
@@ -0,0 +1,61 @@
# `src/io/bounded.rs` — bounded-syscall primitive
## Why this exists
[`crate::halt::Halt`] is cooperative: callers poll `is_cancelled()`. It
cannot reach inside a syscall the kernel currently owns the thread for —
`libc::sync_file_range`, `libc::fsync`, `File::write` on NFS, and so on.
`/api/stop` from autorip therefore can't unstick a thread sitting in
such a syscall.
`bounded_syscall` is the escape hatch: it runs `op` on a fresh worker
thread, then `recv_timeout`s on a rendezvous channel for the result.
The wait is broken into ~250 ms slices so the calling thread can poll
the supplied `Halt` in between. If the deadline elapses or the halt
fires, the worker is intentionally leaked — the syscall will unwind
whenever the kernel decides, or at process exit, but the caller is
free to fall back to a degraded code path (skip the sync, log loudly,
etc.).
## Trade-offs
- **Thread per call.** Cheap (`std::thread::spawn` is < 100 µs on
Linux/macOS), but not free. Use on coarse-grained finalisation
syscalls (`sync_all`, `sync_file_range(WAIT_AFTER)`), not on hot
inner-loop writes.
- **Leak on timeout.** A wedged syscall keeps a kernel slot and a
user-space thread around for the rest of the process's life. Bounded
by the number of independent rip/mux sessions, which is one per
disc. The alternative — trapping the caller forever — defeats the
entire purpose of `/api/stop`.
- **Halt granularity ~250 ms.** Halt observation is not instant; it's
the worst-case latency of the `recv_timeout` slice. Good enough for
human-driven stop requests; not suitable for hard real-time
deadlines.
## Single source of truth
Do NOT inline this pattern. Every blocking-syscall wrapper in the rip
+ mux pipeline calls this helper, so changes (e.g. swapping the
channel impl, adjusting the poll slice, adding metrics) land in one
place.
## Platform
Pure `std::thread` + `std::sync::mpsc`. No `cfg(target_os)` needed
here — the helper itself is platform-agnostic. Callers that wrap
Linux-only syscalls (`sync_file_range`) still need their own
`#[cfg(target_os = "linux")]` gates; this helper does not.
## `bounded_syscall` contract details
- `halt` is polled at `POLL_INTERVAL` granularity. Pass `None` for
callers that don't (yet) have a halt token plumbed through —
behaviour degrades to deadline-only, matching the 0.20.5
`wait_after_with_timeout` shape this helper generalises.
- `op` returns `R: Send + 'static`. The closure must own everything it
touches because it may outlive this call (timeout / halt cases).
- On `Halted` / `Timeout` the worker thread is intentionally leaked:
the syscall will unwind whenever the kernel decides, or when the
process exits. The calling thread is never trapped inside a kernel
call.
+111
View File
@@ -0,0 +1,111 @@
# `byte_prefetcher` — design notes and test rationale
`BytePrefetcher` (`src/io/byte_prefetcher.rs`) is the `std::io::Read`
analogue of `crate::sector::PrefetchedSectorSource`. It spawns a
producer thread that fills a bounded pool of `Vec<u8>` chunks from the
underlying reader and ships them through a channel; the consumer pulls
filled chunks, uses them, and sends the empty `Vec<u8>` back through a
recycle channel so the producer can re-fill in place. Result: zero
allocations and zero cross-thread frees in the steady-state hot loop.
This is the byte-stream half of the freemkv mux highway —
`BytePrefetcher` feeds `crate::mux::demux_thread::DemuxThread` for
`m2ts://` (the only in-tree caller today, via `crate::mux::resolve`),
and works for any stream whose source is an `io::Read` rather than a
`SectorSource`.
## `PrefetchShell`
Drop blocks the calling thread until the producer exits. To guarantee
a prompt exit, drop the forward receiver and the recycle sender first
so the producer observes channel disconnection (or cancel the `Halt`
passed to `BytePrefetcher::new`, which the producer polls at
`POLL_INTERVAL` granularity even while parked on a channel op).
## Test coverage rationale
- **`recycle_depth_is_forward_depth_plus_one`** — `RECYCLE_DEPTH` must
be one MORE than `FORWARD_DEPTH` per its own doc comment: the
producer needs at least one buffer to fill while the consumer holds
the other `FORWARD_DEPTH`-worth in flight. A `+` -> `*`/`-` mutation
on `FORWARD_DEPTH + 1` would under-size the recycle channel (e.g.
`FORWARD_DEPTH * 1 == FORWARD_DEPTH`, one short), which starves the
producer of a spare buffer.
- **`into_channels_drop_releases_producer`** — the CRITICAL
regression: after `into_channels`, dropping the returned forward
receiver + recycle sender must let the producer observe
disconnection and exit, so dropping the `PrefetchShell` (which
joins the producer) returns promptly. With the old clone+forget the
leaked endpoints kept the producer blocked and this join hung
forever.
- **`delivers_all_bytes_in_order_across_chunks`** — CORE CONTRACT: the
prefetcher must deliver every source byte, in order, exactly once —
never silently truncate or duplicate. Source is 5000 bytes; chunk
size 1024 forces multiple chunks (4 full + 1 short of 904). The
reassembled stream must equal the source. Mutation: replacing
`buf.truncate(n)` with a no-op would over-report bytes on the final
short read and this fails.
- **`short_read_truncates_to_actual_length`** — a reader that returns
fewer bytes than requested per call must NOT leave stale tail bytes
in the delivered chunk. Cursor over 10 bytes with a 4096 chunk
yields a single 10-byte chunk; the consumer must see exactly 10
bytes, not 4096. Grounds `buf.truncate(n)`. Mutation: delete the
truncate and the chunk would carry 4086 zero bytes of padding,
failing the length assert.
- **`empty_source_yields_clean_eof_no_batches`** — EOF semantics: an
empty source (Cursor over `[]`) yields `read() == Ok(0)` on the
first call, which the producer treats as EOF and returns, dropping
tx. The consumer sees RecvError (zero batches), NOT an Err batch and
NOT a zero-length Ok batch. Grounds the `Ok(0) => return` arm.
Mutation: changing `Ok(0) => return` to `Ok(0) => continue` would
spin forever (test times out).
- **`read_error_is_propagated_as_err_batch`** — a reader that fails
mid-stream must surface the `io::Error` as an `Err` batch on the
forward channel, not swallow it. We deliver one good chunk then an
error. The consumer must see the good bytes followed by the error.
Mutation: changing `let _ = tx.send(Err(e)); return;` to a plain
`return` would drop the error silently and this fails.
- **`read_panic_surfaces_as_err_batch_not_clean_eof`** — PANIC
propagation: a reader that PANICS mid-stream must NOT be read as a
clean EOF at the demux boundary. The producer's `catch_unwind` sends
an explicit `Err` sentinel before the thread unwinds, so the
consumer sees the good bytes followed by an error batch — never a
silent truncation. Without the `catch_unwind` the panic would just
drop `tx`, the consumer would see RecvError (== clean EOF) and the
partial output would be finalized as if complete.
- **`recycled_buffer_carries_no_stale_tail`** — recycle-buffer reuse
must NOT leak stale bytes between chunks of different lengths. After
a full chunk, a short read reuses the same recycled buffer; the
regrow-then-truncate sequence must yield only fresh bytes. Verified
by reassembling the full stream. Source: 8 bytes of 0xAA + 3 bytes
of 0xBB, with chunk_bytes=8 → chunk0 = 8×0xAA, chunk1 = 3×0xBB.
- **`exact_multiple_length_no_trailing_empty_batch`** — exact-multiple
boundary: when the source length is an exact multiple of
chunk_bytes, the final non-empty chunk is followed by an `Ok(0)` EOF
read, NOT a spurious empty Ok batch. 12 bytes with chunk_bytes=4 →
three 4-byte chunks then clean EOF. Total bytes must equal 12 and no
zero-length batch may appear.
- **`drop_finite_prefetcher_joins_cleanly`** — dropping the
`BytePrefetcher` directly (without `into_channels`) must join the
producer cleanly when the source is finite. The producer reaches
EOF, drops tx, and exits; Drop's join returns. Grounds the
`BytePrefetcher` `Drop` impl. Mutation: removing the `Ok(0) =>
return` EOF exit would hang this join.
- **`drop_endless_prefetcher_joins_cleanly`** — regression: dropping a
`BytePrefetcher` directly (without `into_channels`) with an ENDLESS
source must not deadlock. Before the fix, `Drop` joined the producer
while `rx`/`recycle_tx` were still alive (sibling field drop order),
so the producer filled the depth-2 forward channel and then spun in
`send_timeout` forever (rx never drained, halt=None). The fix drops
`rx`+`recycle_tx` BEFORE the join so the producer sees
`SendTimeoutError::Disconnected` and exits.
+48
View File
@@ -0,0 +1,48 @@
# `labels::class_reader`
Hand-rolled JVM `.class` file reader, tailored to the subset needed for BD-J
label extraction (Deluxe / dbp / similar frameworks). Implements JVMS §4
(class file format) and §6 (bytecode) only as far as: constant pool,
methods, the `Code` attribute, and a non-allocating bytecode iterator.
No external deps beyond `std`. No `unsafe`. No panics on malformed input —
every parse fault is a typed `Error`. This is shared infrastructure for any
label parser that needs structured access to `.class` files inside a
`/BDMV/JAR/<x>.jar`.
## `ConstantPool::from_entries`
Test-only constructor — builds a constant pool directly from a vector of
entries. Real callers go through `ClassFile::parse`, which builds this from
class-file bytes. Used by parser unit tests (e.g. `labels::deluxe`) that
need to exercise bytecode walkers against synthetic class fixtures without
hand-rolling valid `.class` byte buffers.
Caller is responsible for: prepending a `CpInfo::Empty` at index 0 (the
spec-reserved slot), and inserting a `CpInfo::Empty` after each Long/Double
entry (the 2-slot quirk).
## `decode_modified_utf8`
Decodes JVM "modified UTF-8" (JVMS §4.4.7). Practically identical to
standard UTF-8 for the BMP-printable subset seen in label strings, but with
two notable deviations:
- U+0000 is encoded as the two-byte sequence `0xC0 0x80`, not as `0x00`.
- Supplementary characters (U+10000..) are encoded as a UTF-16 surrogate
pair, each surrogate emitted as 3-byte modified UTF-8.
For label data (mostly ASCII / Latin-1 / CJK in BMP), the simple
implementation here covers everything we'll encounter. The `0xC0 0x80` →
U+0000 case is tolerated explicitly; supplementary characters would need
surrogate-pair stitching, but no label-relevant string uses them.
## `Reader::slice` overflow test
`Reader::slice` takes an attacker-supplied length: a JVMS `u4`
`attribute_length` / `code_length` (§4.7, §4.7.3) or a `u2` Utf8 length
(§4.4.7). Adding it to `pos` without a wrap check panics on overflow in
debug and, in release, wraps to a small end offset that slips past the
bounds check and then panics inside the slice index. Both are panics
escaping a parser whose whole input is untrusted disc bytes; the contract
is an EOF error.
+18
View File
@@ -0,0 +1,18 @@
# CLPI vs MPLS cross-validation diagnostic
`src/labels/clpi_audit.rs` walks both per-clip CLPI program info and
per-playlist MPLS STN tables, normalizes their stream lists by `(PID,
language, coding_type)`, and classifies each PID into one of four
buckets:
1. **CLPI only** — a stream present in a `.clpi` ProgramInfo that no
playlist STN table references (orphan on disc).
2. **MPLS only** — a stream a playlist references that no `.clpi`
ProgramInfo lists (indicates a parser disagreement).
3. **Match** — both sources agree on coding_type and language.
4. **Divergent** — both sources see the PID but disagree on
coding_type or language.
`audit` returns a structured `ClpiVsMplsAudit` report. This is a
diagnostic surface only; it does not feed the label-selection
pipeline.
+52 -6
View File
@@ -69,13 +69,23 @@ Each stream PID entry header (14 bytes):
```
Offset Size Field
------ ---- -----
0 2 Stream PID
2 2 Reserved + EP stream type
4 2 Number of coarse entries
6 4 Number of fine entries (note: 32-bit, can be large)
10 4 EP map start offset (relative to EP map start)
2 2 stream_PID (byte-aligned)
4 10 Bit-packed block, 80 bits total (see below)
```
The stream PID entry is **not** byte-aligned past `stream_PID`. Bytes 4..14 are one
80-bit packed field, read as a `u64` plus a trailing `u16`:
Bits Width Field
---- ----- -----
0-9 10 reserved
10-13 4 EP_stream_type
14-29 16 num_EP_coarse
30-47 18 num_EP_fine
48-79 32 EP_map_start_address (relative to the EP map start)
Note `num_EP_fine` is **18 bits**, not 32. See `parse_cpi` in `src/clpi.rs`.
libfreemkv parses only the first stream (primary video), which is sufficient for sector-level seeking.
### Two-Level Index
@@ -189,7 +199,43 @@ The full ripping pipeline chains three parsers:
2. **CLPI** converts those timestamps to SPN ranges, then to sector extents.
3. **UDF** provides the file's starting LBA on disc for absolute sector addressing.
The `Disc::scan()` method in `src/disc.rs` orchestrates this: for each play item in each playlist, it loads the corresponding CLPI, calls `get_extents()` with the play item's in/out times, and collects the resulting sector ranges into the title's extent list.
The `Disc::scan()` method in `src/disc/mod.rs` orchestrates this: for each play item in each playlist, it loads the corresponding CLPI, calls `get_extents()` with the play item's in/out times, and collects the resulting sector ranges into the title's extent list.
## ProgramInfo Section (Per-Stream Metadata)
`parse_program_info` in `src/clpi.rs` reads per-stream (pid, coding_type,
language, codec sub-fields) from the ProgramInfo section, per the Blu-ray
Disc Read-Only Format Part 3 CLIPINF (CLPI) specification:
```text
ProgramInfo:
length: 4 bytes
reserved: 1 byte
num_programs: 1 byte
for each program:
spn_program_sequence_start: 4 bytes
program_map_pid: 2 bytes
num_streams: 1 byte
num_groups: 1 byte
for each stream:
pid: 2 bytes
stream_coding_info_length: 1 byte
stream_coding_info: (varies by coding_type)
coding_type: 1 byte
per-type bytes (see match arms in parse_program_info)
```
It returns `Vec::new()` on any structural mismatch — errors aren't
propagated because the EP map is the primary CLPI output, and a corrupt
program_info shouldn't break sector-range lookups.
### Test fixture notes
- `prog_info_start_zero_does_not_parse_header_as_program_info`: the
fixture is crafted so that parsing from offset 0 WOULD yield a stream
(num_programs at `[5]`, a second program header whose num_streams
byte at `[20]` is 1, then a well-formed stream record), so an empty
result can only come from the `prog_info_start > 0` gate in `parse`.
## References
+79
View File
@@ -0,0 +1,79 @@
# `src/mux/codec/mod.rs` — extended notes
Relocated prose from doc comments in `mod.rs`, trimmed there to stay within
the comment-guard's prose caps. Each `//`/`///` pointer in the source names
the section here it corresponds to.
## `Frame::discontinuity`
Carried per-FRAME (not per-PES) because buffering parsers decouple the frame
from the PES that carried the gap signal: MPEG-2 emits whole GOPs, H.264/HEVC
lag one access unit behind the PES that produced them. A clean rip leaves
every frame `false`; the flag is only ever set on the degraded/conceal path
(P3/B1), when a frame's data begins after packets the demuxer never received
— an undecryptable unit concealed as NULL-TS upstream, or a continuity break
in a damaged source.
## Drop-on-undecodable policy across codecs ("clean muxes always")
- **Audio with independent access units** (DTS, AC-3/E-AC-3, …) gates each AU
through a per-codec corruption check and drops the ones that fail, keeping
A/V sync (a drop is a silence gap, never a shift) and logging every drop
via the shared `dropgate::DropTally`. DTS validates via its core-frame
header (ETSI TS 102 114); AC-3 uses its native frame CRC.
- **LPCM is excluded on purpose**: raw PCM carries no framing or integrity
data, so a corrupt sample is indistinguishable from a quiet one — there is
nothing to detect, so nothing can be honestly dropped.
- **Video is excluded on purpose**: H.264/HEVC/MPEG-2/VC-1 are inter-frame
predicted, so dropping one frame corrupts every frame that references it
until the next keyframe. Video instead resyncs at GOP/IDR boundaries (the
ResyncGate) and lets the decoder conceal — a fundamentally different model
than per-frame audio dropping.
- TrueHD/MLP, FLAC, MP2/MP3 and AAC-ADTS also gate undecodable frames via a
`DropTally` (poison/drop-forward for MLP's inter-AU restart state on a
major-sync boundary; CRC/sync-verdict drops for the passthrough codecs).
## `codec_private()` — why "absent" is the only truthful answer for some parsers
A codec parser's `codec_private()` feeds `DiscStream::codec_private`, which
the MKV muxer turns directly into the track's `CodecPrivate` element (RFC
9559 §5.1.4.1.24): `Some(bytes)` writes an element holding exactly those
bytes, `None` omits the element entirely. The two are NOT interchangeable —
a zero-length `CodecPrivate` asserts that the codec's initialisation data IS
empty, which is not true of any codec, and a one-byte one asserts a config no
decoder can parse.
The gating parsers (ADTS, MPEG audio, FLAC) and the passthrough parser
extract no configuration at all: they validate and forward frames whose
configuration is carried in band (ADTS headers, MPEG-1 audio frame headers,
FLAC frame headers) or supplied by the source container. Having derived
nothing, the only truthful answer they can give is "absent" — any `Some`
would be a value they invented. Nothing else in the test suite distinguished
the two, so each of these impls could have returned a fabricated `Some` and
produced a malformed track header unnoticed.
## `provenance_guard` test module
Every emitted frame must carry the source byte offset of the packet it came
from. That invariant held only for video for as long as it existed: `dts`,
`ac3`, `adts`, `truehd`, `pgs`, `dvdsub`, `flac`, `lpcm`, `mpegaudio` and the
passthrough parser all built frames with `source: None`, so a multi-clip
title could not place audio or subtitles by byte and fell back to inferring
the clip from timestamps — which is what made branched titles run minutes
long.
Nothing asserted it, so nothing caught it. Finding it took a brace-balanced
scan of the tree by hand, which also turned up five sites a regex had
missed. `provenance_guard` is that scan, as a test.
### `code_only` / `frame_literals` design
`frame_literals` walks `Frame { .. }` literals with balanced braces. A regex
cannot do this — `Frame` blocks contain nested braces (`coding: Some(..)`,
closures) and a non-greedy match stops at the first `}`, which is how five
sites survived the first pass.
`code_only` strips line comments before scanning. A comment that MENTIONS
`source: None` is prose, not a construction — this guard's own doc comment
tripped it on the first run, which is the same mistake as grepping a file
and matching its commentary.
+17
View File
@@ -0,0 +1,17 @@
# `src/mux/codec/coding.rs`
## Spec references
ITU-T H.273 (CICP code points, shared elsewhere), ISO/IEC 13818-2 §6.3.10
(MPEG-2 picture coding extension: `top_field_first`, `repeat_first_field`,
`progressive_frame`), RFC 9559 §5.1.4.1.28 (Matroska `FieldOrder` element
0x9D).
## HDR10 unit scaling
`Hdr10Metadata` values are stored in their raw SEI integer units (not yet
scaled to the Matroska float domain); the muxer applies the H.265 → Matroska
unit conversion at emit time so the scaling lives in exactly one place.
Spec: Rec. ITU-T H.265 D.2.28 (Mastering Display Colour Volume, payloadType
137) and D.2.35 (Content Light Level Info, payloadType 144).
+11
View File
@@ -0,0 +1,11 @@
# `consts::coding_type` — spec citations
`coding_type` is one registry used in two places that share the same value
space: the MPEG-TS PMT `stream_type` (ISO/IEC 13818-1 Table 2-34) and the
Blu-ray STN/CLPI `stream_coding_type` (BD-ROM Part 3).
The standardized video codes (`0x02`, `0x1B`, `0x24`) are ISO assignments
(ISO/IEC 13818-1 Table 2-34); `0xEA` (VC-1) is a BD-ROM convention in the ISO
user-private range. The `0x80..=0xA2` audio/graphics codes also sit in the
user-private range and follow the Blu-ray Disc Association / ATSC A/52
convention.
+14
View File
@@ -0,0 +1,14 @@
# CRC helpers (`src/mux/codec/crc.rs`)
## `crc16_mlp` reversed byte order
MLP's checksum is the "reversed" scheme. `crc16_mlp` emits its two bytes in
the OPPOSITE order to a standard little-endian CRC readout, so the caller
swaps them back and compares against the stored trailer word read
LITTLE-endian — see `truehd::mlp_major_sync_crc_ok`, which is authoritative.
Comparing big-endian instead is precisely the bug that function was fixed
for: it could never validate a real extended major sync, so whole TrueHD
tracks were dropped silently. This comment used to prescribe exactly that,
and to point at a `truehd::mlp_major_sync_ok` that does not exist.
Verified against real MLP/TrueHD bitstreams (225/225 major-sync AUs).
+87
View File
@@ -0,0 +1,87 @@
# `src/labels/criterion.rs` — internal notes
## `assign_stream_numbers`
Assign a 1-based stream number per `StreamInfo`, parallel to `infos`.
A stream mapped in `playbackconfig.xml` (`stream_map`) keeps its
mapped number. Streams with no mapping (absent or incomplete
`playbackconfig.xml`, or an unmatched `StreamInfo_ID`) are numbered
1-based per type — but the fallback counter SKIPS any number already
claimed via the map, so a synthesized number can never collide with a
map-assigned one. (Both numbering domains are 1-based per type, and
`apply_labels` matches on `(type, stream_number)`, so a collision
would mislabel tracks.)
Returns `None` when the 1-based stream-number space is exhausted —
every number in `1..=u16::MAX` for that type is either already
claimed by the map or already synthesized. That is unreachable on
real media: the BD STN_table carries at most 32 primary audio and 32
PG streams per playlist, so the 65535-wide space leaves >2000x
headroom. It IS reachable from a crafted `streamproperties.xml`
listing >65535 stream entries, and the only correct answers there are
"fail the parse" or "emit colliding numbers"; we fail.
The skip search is bounded by the numbering space itself: a `u16`
`saturating_add` here parked the counter at `u16::MAX` forever
whenever the map also claimed `u16::MAX`, turning an overflow guard
into a hang that `apply()`'s `catch_unwind` cannot interrupt. The
counters are therefore widened to `u32` so the skip loop strictly
increases toward a fixed ceiling (guaranteeing termination) and
exhaustion is reported rather than absorbed.
## Test: immunity pin, section-boundary half
Covers `an_unterminated_stream_element_shortens_the_list_it_cannot_extend_it`.
`parse_stream_infos` emits one `StreamInfo` per `*StreamInfos` element
unconditionally — no filter, no `continue` — so an element whose
fields are missing or unrecognized still occupies its position, and
`assign_stream_numbers` still spends a number on it.
That is the property that keeps this parser out of the failure mode
where a skipped entry pulls every later label one stream forward. It
is load-bearing for the fallback path specifically: with no
`playbackconfig.xml` the numbers come purely from position in this
list, so dropping an element there would shift the rest.
Mutation: skip elements with an empty `ID`/`LangInfoID` → the two
real audio streams renumber to 1 and 2.
Each stream here is one closed XML element, and every field is read
out of `&text[start..end]` — the range `xml::find_element` returned —
so one element can never absorb the next one's fields, however the
document is malformed around it. Contrast the flat-string walk in
pixelogic, where a section whose end marker is missing keeps
consuming entries as STN slots.
The missing-boundary case fails closed. An element with no close tag
of its own ends at the NEXT close tag, so it absorbs the element
behind it — the list comes back SHORTER. It cannot come back longer:
nothing outside a returned range is ever read as a stream, and
`find_element` yields `None` rather than a range running to EOF when
no close tag exists at all. A malformed document can cost this parser
a slot; it can never invent one.
Mutation: read fields from the document rather than the element's
range, or let a close-less element run to EOF → the trailing elements
re-enter the list as extra streams.
## Test: `exhausted_numbering_terminates_instead_of_looping`
A crafted `streamproperties.xml` can drive the fallback counter to
the top of the 1-based u16 stream-number space and then present one
more unmapped stream whose successor number is also claimed by the
map.
This must TERMINATE. The bound is the numbering space itself, so the
assertion is on the spec-derived exhaustion behaviour (`None`), not
on any tunable constant. Run on a worker thread with a deadline so a
non-terminating loop fails the test in 20 s instead of hanging CI.
## Test: `full_u16_numbering_space_is_usable_and_unique`
The whole 1-based u16 space must remain usable: 65535 unmapped audio
streams get 65535 distinct numbers with no panic and no wrap. The
literals here are the JVMS-independent, spec-derived size of a u16
1-based numbering domain, not a tunable cap.
+245
View File
@@ -0,0 +1,245 @@
# CSS keyless title-key recovery (`src/css/keyless.rs`)
## The cipher this attacks
The content descrambler ([`super::lfsr::descramble_sector`]) seeds its two
LFSRs **directly** from `key = title_key XOR sector_seed` (seed =
`sector[0x54..0x59]`): LFSR1 from key/seed bytes 0-1, LFSR0 (24-bit) from
bytes 2-4 with the pre-conditioning `r0 = r0*2 + 8 - (r0 & 7)`, and each body
byte recovered as `plain = TAB1[cipher] ^ (keystream & 0xff)`. There is no
title-key mangling on the content path, so the recovery is a single inversion
of the sector cipher.
## The attack
1. **Known plaintext → keystream.** Because descramble applies TAB1 to the
ciphertext, the per-byte keystream is `TAB1[cipher[i]] ^ plain[i]`.
2. **Brute the 16-bit LFSR1 seed.** For each of 2^16 seeds, run LFSR1
forward; for the first four steps deduce the LFSR0 output bytes from the
keystream (carry-tracked), reconstructing LFSR0's state. For the next six
steps clock LFSR0 normally and check it reproduces the keystream — a wrong
LFSR1 seed fails fast.
3. **Back-clock LFSR0.** Run four backward steps (each a 256-way search for
the byte shifted in) to reach the initial state, then undo the
`r0*2 + 8 - (r0 & 7)` pre-conditioning to recover key[2..5].
4. **XOR back the seed.** `key[0..5] ^= sector_seed[0..5]`.
Known plaintext for step 1 comes from the longest periodic run in the
cleartext `sec[0x00..0x80]`, assumed to continue into the encrypted region at
0x80.
## `recover_title_key_from_plain`
Core of the recovery. `crypted` is the ciphertext starting at sector byte
0x80; `decrypted` is the matching known plaintext; `seed` is
`sector[0x54..0x59]`. On success returns the recovered 5-byte title key;
`None` if no LFSR seed reproduces the keystream. At least 10 bytes of
`crypted`/`decrypted` are required (the cipher is iterated 10 times: 4 to
reconstruct LFSR0, 6 to validate).
## `attack_crib`
Scans the clear header `sec[0x00..0x80]` (never scrambled) for the longest
run that repeats with a cycle length in 2..0x2F. If the run is long enough
(`plen > 3` and at least two full cycles), the plaintext at 0x80 is taken to
be that periodic run continuing forward. Returns `None` for an unscrambled
sector or one with no usable run — such a sector can be neither cracked nor
key-validated, only descrambled with an externally-cached key.
The header is untouched by `descramble_sector`, so the crib is identical
before and after descramble: the decrypt path uses it as a per-sector
"did the cached key descramble correctly?" oracle (the predicted plaintext
must reappear at 0x80), and the cracker uses it as its known plaintext.
## Tests
### `synth_sector`
Builds a synthetic scrambled sector for a given title key and seed, with
`plain` placed as the plaintext at byte 0x80, scrambled with EXACTLY the
cipher `descramble_sector` inverts. Returns
`(scrambled_sector, full_plaintext_body)`.
### `synth_periodic_sector`
Builds a synthetic scrambled sector whose CLEARTEXT (0x00..0x80) ends in a
periodic run that continues into the encrypted region — the case
`crack_title_key` is designed to crack.
### `recover_round_trips_known_keys`
MANDATORY round-trip (Task C.1): synthesize a scrambled sector for a known
(title_key, seed), then assert `recover_title_key` returns a key that
descrambles the body back to plaintext. CSS title-key recovery is
well-defined up to keys that scramble identically; we assert the full body
round-trips (the true correctness property), and additionally that the EXACT
key is returned for the common case.
### `descramble_matches_accepts_only_the_key_the_sector_was_scrambled_with`
`descramble_matches` is the ONLY gate between the LFSR search and a key
handed back to the caller: both `recover_title_key` and the crib-driven
`crack_title_key_inner` return a candidate only if this says the key really
descrambles the sector to the known plaintext. A body that always answered
`true` would let the first spurious LFSR-seed match through as the title
key — the ripper would then descramble the whole title with a key that opens
nothing, producing garbage rather than a "no key" error.
Pinned both directions: the genuine key is accepted, and EVERY key one bit
away from it is rejected. The one-bit neighbours are the strongest form of
wrong key — a gate that only rejects wildly different keys would still pass
a near-miss out of the 2^16 seed search.
### `descramble_matches_does_not_disturb_the_caller_s_sector`
The gate is applied to a COPY: verifying a candidate must not modify the
caller's sector. `recover_title_key` runs the gate and then hands the sector
on to be descrambled for real — if verification descrambled in place, that
second descramble would run over already-transformed bytes (and, worse, a
rejected candidate would leave the sector corrupted).
### `recover_rejects_a_sector_that_ends_inside_the_encrypted_region`
A sector buffer that ENDS inside the encrypted region must be refused, not
sliced. `recover_title_key` slices `sector[0x80..0x8A]` unconditionally
after its length guard. The existing short-sector test uses
`SECTOR_BYTES - 1`, which is still long enough for that slice to succeed —
so the guard was never the thing producing the `None`, and dropping it (or
weakening the `||` to `&&`, which a full-length crib satisfies) changed
nothing observable. On a real short read this is an out-of-bounds panic on
the rip thread.
### `a_buffer_longer_than_one_sector_still_yields_its_key`
A buffer LONGER than one sector is still one sector: both entry points read
the first `SECTOR_BYTES` and must recover the key from it. Callers read DVD
data in multi-sector blocks, so an over-long slice is the normal case, not
an exotic one. A length guard that rejected it (`len > SECTOR_BYTES` instead
of `<`) would make every block-read caller silently unable to crack
anything.
### `recover_accepts_more_than_ten_bytes_of_known_plaintext`
`recover_title_key` accepts MORE than ten bytes of known plaintext, and uses
all of it: the extra bytes tighten the `descramble_matches` gate. The
ten-byte figure is a MINIMUM (the cipher is iterated ten times), not an
exact requirement — a guard reading it as an upper bound would reject every
caller that knows a longer crib.
### `a_recoverable_sector_with_the_scramble_bits_cleared_is_still_refused`
The scramble-flag gate on a sector whose BODY really is ciphertext. Both
entry points refuse a sector with `sector[0x14] & 0x30 == 0`: an
unscrambled sector has no title key to recover, and its bytes at 0x80 are
already plaintext. Every prior test of this gate used an all-zero or
all-`0x11` sector, where the recovery would have found nothing anyway — so
widening the mask test (`&` to `|`, which makes it true for EVERY flag byte)
produced the same `None` and went unseen.
Here the sector is genuinely scrambled and its key IS recoverable; only the
cleared flag stands in the way. If the gate stops working, both functions
start returning keys for sectors the disc says are in the clear.
`attack_crib` carries its own copy of the same gate and is the one that
actually stops the crack (`crack_title_key`'s is defensive duplication); it
also doubles as the decrypt path's cached-key oracle.
### `descramble_matches_forces_the_scramble_flag_on_its_own_copy`
The gate must verify a candidate against the sector's CIPHERTEXT regardless
of what the sector's own flag byte says. `descramble_matches` forces `0x10`
on its copy precisely because `super::lfsr::descramble_sector` is a no-op
when the scramble bits are clear — without that, verifying a
scrambled-but-unflagged sector compares raw ciphertext against the crib, and
every candidate key is rejected. Nothing exercised it: every fixture already
had the flag set, where forcing the bit is a no-op.
### `descramble_matches_compares_all_of_the_plaintext_and_no_more_than_the_sector`
The gate compares the WHOLE supplied plaintext, clamped to the encrypted
region. Two properties in one, because they are the two halves of
`plain.len().min(SECTOR_BYTES - ENCRYPTED_START)`:
- it must compare beyond the first sixteen bytes, or a key that opens only
the head of the crib is accepted; and
- it must never compare past the end of the sector — a caller that knows
more plaintext than the 1920-byte encrypted region holds otherwise
indexes off the end of the buffer and panics.
### `sector_with_trailing_run`
Builds a sector whose clear header ends in a `period`-length repeating run
of exactly `run_len` bytes immediately before 0x80. The run is anchored to
ABSOLUTE sector offset (`sec[x] = pat[x % period]`), which is what makes
"the run continues past 0x80" a statement independent of the code under
test: the byte at `0x80 + i` of the underlying plaintext is
`pat[(0x80 + i) % period]`. Everything before the run is `0x00` (the pattern
bytes are all >= 0xD0, so the run cannot be extended backwards by accident),
and the encrypted region is filled with `0xFF` — so a crib that reads past
0x80 into "ciphertext" is immediately visible.
### `attack_crib_predicts_the_periodic_run_continuing_past_0x80`
KNOWN ANSWER: for a run of `run_len` bytes with period 5 ending exactly at
0x80, the crib is the run continued forward — the same ten bytes for every
run length, because the prediction depends only on the pattern and the
phase, never on how many cycles happened to be visible. The short lengths
are the load-bearing ones: at `run_len = 11` the crib window starts at 0x76
and is only 10 bytes from the end of the header, so any drift in
`plain_start`, in `cycles * best_p`, or in the `i % best_p` wrap reads the
0xFF "ciphertext" instead of the run.
### `attack_crib_refuses_a_run_shorter_than_two_cycles`
A run of exactly ONE cycle (plus the trivial tail the detector counts) is
not enough to predict forward: `attack_crib` requires at least two full
cycles. Weakening that guard would let a one-off byte sequence be declared
periodic and produce a confidently wrong crib — which the decrypt path uses
as its "is my cached key still right?" oracle.
### `attack_crib_refuses_a_buffer_shorter_than_a_sector`
`attack_crib` indexes `sector[0x7f - j]` with no per-access bound, so its
own length guard is the only thing between a short buffer and an
out-of-bounds read. Nothing reached it: every caller-level test used a full
sector, and the entry points' guards fire first.
### `attack_crib_survives_a_header_that_is_periodic_to_offset_zero`
A header that is periodic ALL THE WAY to offset 0 must not walk the
backward scan off the front of the sector. The detector counts backwards
from 0x7f while `j < 0x80`. On a fully periodic header the run never
breaks, so `j` reaches 0x7f and the bound is the ONLY thing that stops it —
one step further and `0x7f - j` underflows a `usize` and panics. A constant
or fully-patterned 128-byte header is ordinary DVD data (padding, a run of
zeros), not a crafted input, and every existing fixture had a
filler/run boundary well before offset 0 that stopped the scan early.
### `attack_crib_is_independent_of_the_encrypted_region`
The crib is read from the CLEAR header only. A run that reaches 0x80 must
predict from the header bytes, never from the encrypted region — the
previously-fixed bug this function's doc comment records. Pinned by
rewriting the encrypted region and requiring the crib not to move.
### `recover_title_key_from_plain_refuses_fewer_than_ten_bytes_of_either_input`
`recover_title_key_from_plain` unconditionally builds a 10-byte keystream
buffer from `crypted[0..10]` and `decrypted[0..10]`, so its length guard is
the only thing standing between a short slice and an index-out-of-bounds
PANIC. Nothing reached that guard before: `recover_title_key` rejects
`plain.len() < 10` at its own door and always hands on exactly ten
ciphertext bytes, and `crack_title_key_inner` always passes a fixed
`[u8; 10]` crib. The guard is a live contract for any future caller and was
executed by no test at either boundary.
### `recover_title_key_from_plain_xors_the_sector_seed_back_out`
The seed XOR-back (`recover_title_key_from_plain`'s last step) is what turns
the recovered LFSR key into the TITLE key: `key ^= sector_seed`. Pinned as a
known answer across seeds that differ only in one byte — the same
ciphertext/plaintext pair therefore must yield title keys differing in
exactly that byte. Without this, a body that ORed the seed in (or dropped
the step) still round-trips on any fixture whose seed is zero, and on the
non-zero ones the failure looks like "no key found" rather than a wrong
step.
+311
View File
@@ -0,0 +1,311 @@
# `src/css/mod.rs` — extended notes
Overflow detail relocated from doc comments in `src/css/mod.rs` (kept short
there per the comment-guard caps). Function/item names below match the code.
## `CSS_LOCKED_BAIL`
The bus-auth read gate is global (all-or-nothing), so a run this long of
`05/6F/03` (CSS-locked) reads means the gate is shut and nothing here is
crackable — bail instead of grinding the full 50_000-sector budget. That
grind is what made rc5 appear to hang on a wedged USB bridge.
## `crack_key`
Recovers the CSS title key with no keys, by scanning scrambled sectors and
running the known-plaintext attack (see the `keyless` module). The crib comes
from the periodic-run detector: a scrambled sector's cleartext region
(bytes 0x00..0x80) often ends in a short-period repeating run (stuffing /
constant fill); the attack assumes that run continues across the 0x80
boundary into the encrypted region, giving the known plaintext the 2^16 LFSR
recovery needs. Scans up to 50000 sectors across the extents and returns the
first sector that yields a key — no player keys, no disc-key crack. Works on
a live drive (after bus-auth unlocks reads) and on disc images alike.
This convenience form runs to completion (no cancellation) and returns just
the key; callers needing an operator-Stop / watchdog cancel, or the
three-way `CrackOutcome` (to distinguish "unencrypted" from
"encrypted-but-uncracked"), use `crack_key_outcome`, which takes a `halt`
token.
## `CrackOutcome`
Distinguishes the three cases the bare `Option<CssState>` conflated (and
which caused a silent-failure bug: scrambled-but-uncracked content was
treated as "unencrypted" and muxed as plaintext garbage at exit 0):
- `Cracked` — a scrambled sector yielded a title key.
- `Unencrypted` — NO scrambled sector was seen across the scanned extents
(`is_scrambled_pack` never true): the content is genuinely plaintext, so
proceeding without a key is correct.
- `ScrambledUncracked` — scrambled sectors WERE seen but no key could be
recovered (the recovery found no crackable crib, or the scrambled region
was unreadable). The content is encrypted; muxing it as plaintext would
emit garbage, so callers MUST surface a hard error instead of falling
through to "unencrypted" — the per-title `crate::error::Error::CssKeyMissing`
when it is ONE title's own re-crack that failed (skippable: a sibling VTS
may still crack), or the disc-level `crate::error::Error::CssNoDiscKey`
when it is the disc-wide scan (`Disc::css_error`, every title fails
identically).
## `resolve_dvd_title_key`
The SINGLE place every DVD read path obtains a title key, so the file-backed
mux highway (`crate::build_iso_pipeline`) and the live-drive single-pass
`crate::DiscStream` descramble a DVD identically ("reading is reading"). CSS
keys are per-VTS and crackable from the scrambled data itself, so a
`None`/MPEG-PS title cracks its own key here, in playback order over
`extents`. Everything else is left untouched:
- AACS keys (HD-DVD `.evo` is also MPEG-PS but arrives as `Aacs`) — no CSS.
- a title that already carries a key — nothing to resolve.
- a genuinely clear DVD (no scrambled sector) — stays `None`, a mux no-op.
A scrambled-but-uncrackable title is a hard `crate::error::Error::CssKeyMissing`,
never a silent scrambled-passthrough mux. That code is the PER-TITLE one
(`error::is_skippable_title_stub`), which is correct here: this function
cracks ONE title's own extents, and another VTS on the same disc may still
yield its key, so an all-titles rip skips this title and finishes the rest.
The whole-disc failure is `crate::error::Error::CssNoDiscKey`, raised by
`Disc::ensure_decryptable_keys` from the scan's `css_error`.
## `crack_key_scan` (private)
Tracks a `saw_scrambled` flag so a scrambled-but-uncracked disc is
distinguished from a genuinely-unencrypted one (the `crack_key` `Option`
wrapper collapses both to `None` via `CrackOutcome::into_state`).
## `descramble_sector`
A no-op unless the sector is a scrambled MPEG-2 PS PACK. The pack start code
is checked, not just the byte 0x14 flag bits, for the same reason
`descramble_region` checks it: 0x14 only means "scrambling control" inside a
pack, and in an IFO it is whatever that format stores there. A real
`VIDEO_TS.IFO` sector holds 0x15 there while starting `00 26 00 00`;
descrambling it destroyed 1912 of its 2048 bytes, and because that sector
carries TT_SRPT the disc enumerated 38 titles while an image decrypted from
it enumerated 10, silently, at exit 0.
This function has no callers inside the crate, but the module-level example
prescribes it — so the crate's own documented guidance led straight into
that defect. Making the guard part of the function, rather than something
each caller must remember, is what keeps the safe path the easy one.
## `descramble_region`
Descrambles a whole CSS buffer in place, re-cracking the title key on a VOB
region boundary. `title_key` is a CACHE of the last crack, not a fixed disc
key: it changes per VTS/VOB region, so it is validated on every scrambled
sector and re-cracked on a miss (the standard on-demand per-region rekey).
This CSS key acquisition is intrinsic to the cipher — CSS has no external key
source, the ONLY way to a title key is cracking the data — so it lives with
the CSS primitives and runs inside `decrypt::decrypt_sectors` (a public,
self-contained CSS decrypt), NOT at the post-decrypt recovery seam that AACS
key-fetch and FMTS segment-skip use (those consume external inputs).
The clear header (`<0x80`) is never scrambled, so its periodic crib predicts
the plaintext at `0x80`. Descramble with the cached key; if the crib fails to
reappear the key region changed (or the primed key was wrong) — restore the
ciphertext, re-crack from this very sector, and descramble again. A crib-less
sector (no periodic run) can be neither validated nor cracked, so it rides
the cached key — correct, because it lives in the same region as the nearby
crib sector that set the cache.
This section used to document an `Error::DecryptFailed` for the case where a
sector's crib rejects the cached key and the re-crack from that sector also
fails. That behaviour was tried and REVERTED: crib mismatch plus crack
failure is the signature of a crib FALSE POSITIVE, not of a stale key, and
failing there made real discs unrippable. The code now descrambles with the
cached key and returns `Ok`.
## `has_scramble_flag_bits`
Deliberately NOT called `is_scrambled`. Byte 0x14 only means "scrambling
control" inside an MPEG-2 Program Stream pack; in an IFO, UDF or ISO 9660
sector it is whatever that format stores there. Treating the flag alone as
proof of scrambling is what destroyed 1912 bytes of a real disc's
`VIDEO_TS.IFO` — the sector carrying TT_SRPT — so the disc enumerated 38
titles and an image decrypted from it enumerated 10, silently, at exit 0.
Callers want `is_scrambled_pack`. It asks the same question and also
requires the pack start code, which every genuinely scrambled VOB sector
carries and no IFO sector does.
It stays public only because an integration test asserts the flag
extraction directly. It has no production callers, and its previous doc
comment claimed one (`decrypt::decrypt_sectors`) that did not exist — so the
name was an invitation and the documentation was an argument for accepting
it.
## `is_scrambled_pack`
The HARDENED test the crack scan uses to set its `saw_scrambled` evidence
flag (Fix 3).
`has_scramble_flag_bits` keys solely on bits 4-5 of byte 0x14. That byte is
only meaningful inside a real DVD sector — an MPEG-2 Program Stream pack,
which ALWAYS begins with the 32-bit pack-start code `00 00 01 BA` at offset
0x00. A tiny clear / nav-only stub (a 0.5 s menu loop, an FBI-warning title)
can carry arbitrary bytes that happen to set bits 4-5 of byte 0x14; trusting
byte 0x14 alone there would flip the scan's `saw_scrambled` gate and make a
genuinely-UNENCRYPTED title report `ScrambledUncracked` — a false E7023.
Requiring the pack-start signature FIRST means only a sector that is
structurally a DVD video pack can be counted as scramble evidence. This does
NOT weaken the genuine "encrypted but uncrackable" hard-fail: a real
scrambled feature is made of valid PS packs, so its scrambled sectors still
pass this check and still drive `ScrambledUncracked` when no key cracks.
The pack-start check alone is not enough, and the sector's PES `stream_id`
(offset 0x11, the byte after the 14-byte pack header's `00 00 01` PES
prefix) is the second load-bearing gate. CSS scrambles ONLY elementary
streams — video (`0xE0..=0xEF`) and private_stream_1 audio/subpicture
(`0xBD`) — and it never touches the clear header, so byte 0x11 is the TRUE
stream_id even on a scrambled sector. The MPEG-PS structural packets that
are never CSS-scrambled — system_header (`0xBB`), padding (`0xBE`) and
private_stream_2 (`0xBF`, DVD PCI/DSI navigation) — must be excluded,
because on those the byte at 0x14 is NOT a PES scrambling-control field but
raw payload/structure whose bits 4-5 land set by chance.
This is the DETECTION defect a decrypted HD-DVD tripped: an HD-DVD `.evo` is
MPEG-PS exactly like a DVD `.vob`, and its RDI navigation packs are
private_stream_2 (`0xBF`) whose payload byte at 0x14 routinely has bits 4-5
set. With no `0x11` gate those nav packs flipped the crack scan's
`saw_scrambled` flag on a disc that carries no CSS at all; the crack then
found no key (there is none) and the scan returned `ScrambledUncracked`,
hard-failing a perfectly good HD-DVD with `CssKeyMissing` — E7023. Excluding
`0xBB/0xBE/0xBF` at 0x11 makes the evidence gate match what the crack itself
can act on (the recovery only recovers a key from a scrambled ES pack), so a
decrypted HD-DVD now scans to `Unencrypted` and muxes cleanly.
This does NOT weaken the genuine "encrypted but uncrackable" hard-fail on a
real DVD: a scrambled DVD feature is made of video (`0xE0..`) and
private_stream_1 (`0xBD`) packs, none of which are excluded, so its
scrambled sectors still set `saw_scrambled` and still drive
`ScrambledUncracked` when no key cracks. Byte 0x11 is in the clear header,
so a scrambled DVD pack can never masquerade as `0xBB/0xBE/0xBF`.
The DESCRAMBLE path gates on this same function — `descramble_sector` and
`descramble_region` both call it, not the raw flag test — because the raw
test does not merely mis-skip a sector there: it descrambles one that was
never scrambled and destroys it. The measured case is written up above under
`descramble_region`: a `VIDEO_TS.IFO` sector holding 0x15 at offset 0x14 lost
1912 of its 2048 bytes, taking TT_SRPT with it, and the disc's 38 titles
became 10 — silently, at exit 0. One gate, both paths.
## Tests (`mod tests`)
Detail relocated from test doc comments:
- `a_crib_false_positive_keeps_the_cached_key_rather_than_failing`:
`attack_crib` is a heuristic — it finds a periodic run in the unscrambled
header and predicts the run continues past 0x80. When that prediction does
not hold, the crib reports a mismatch even though the cached key is
correct, and the re-crack then fails BECAUSE the crib was never valid. So
this combination is the signature of a crib false positive, not of a stale
key, and the cached key remains the best available evidence. This test
exists because a prior round read the same code as "descrambling with a
key we just proved stale" and made it `DecryptFailed` to match the AACS
path; real DVDs hit this constantly, and the change made a real disc
unrippable (no unit test caught it — the real-media acceptance gate did).
CSS is not AACS: an AACS unit key either opens a unit or does not, whereas
a CSS title key is recovered from data whose recoverability varies sector
by sector.
- `has_scramble_flag_bits_short_buffer_is_false_no_panic`: grounding —
`sector.len() >= 2048 && (sector[0x14] >> 4) & 0x03 != 0` short-circuits so
a short buffer never reads index 0x14. Mutation: swapping operand order
would panic indexing a 20-byte slice; this test catches it.
- `has_scramble_flag_bits_uses_bits_4_5_only`: grounding —
`(sector[0x14] >> 4) & 0x03`. Mutation: widening the mask to `& 0x0F` makes
0x40 report scrambled; the 0x40 assert catches it.
- `has_scramble_flag_bits_exact_sector_length_accepted`: grounding —
`sector.len() >= 2048`. Mutation: `>= 2048` → `> 2048` makes an exact
2048-byte scrambled sector report false; this catches it.
- `is_scrambled_pack_requires_pack_start_signature`: Fix 3 hardening —
`is_scrambled_pack` requires BOTH the pack-start code and the 0x14 bits, so
a clear/nav-only stub with stray 0x14 bits isn't scramble evidence (else a
genuinely unencrypted title reports the false E7023).
- `is_scrambled_pack_excludes_nav_and_structural_stream_ids`: detection fix —
`is_scrambled_pack` must exclude system_header/padding/private_stream_2 by
`stream_id` at 0x11, since CSS never scrambles those and byte 0x14 there is
raw payload, not a scrambling-control field. This is the exact
decrypted-HD-DVD defect (an `.evo` RDI pack is 0xBF with pack-start and
0x14 bits set); the video/private_stream_1 cases prove the exclusion
doesn't weaken real-DVD detection.
- `MockSource` fields `lock_all`, `crackable`, `short_read`, `stream_id`: model
a CSS-locked drive (`05/6F/03` on every read), a designated crackable LBA
serving a full synthetic scrambled sector, a short-read count (the
`recovery: true` short-read contract, `Some(0)` being the degenerate
spin-guard case), and the PES `stream_id` written at 0x11 (used to model an
HD-DVD `.evo` RDI nav pack via `PRIVATE_STREAM_2`).
- `crackable_sector`: builds a crackable scrambled sector whose cleartext
header (0x59..0x80) carries a periodic run continuing across the 0x80
boundary, mirroring the `synth_periodic_sector` fixture in the keyless
tests but built from the crate-internal `scramble_sector`.
- `a_source_that_returns_zero_sectors_still_obeys_the_scan_budget`: `tried`
increments only per inspected sector inside `for s in 0..usable`; an
`Ok(0)` never runs that loop, so `advance` is forced to 1 to keep the
cursor moving and the extent's full declared length is walked. Mutation:
deleting the `tried` charge in the `usable == 0` arm goes red at 60,000
reads for a misbehaving/adversarial source.
- `crack_key_caps_total_tries_at_50000`: grounding — `max_tries = 50_000`,
`tried += 1` before the read, loop guard `tried < max_tries`. Mutation:
widening the cap, or removing the increment, both go red.
- `crack_key_budget_is_shared_across_extents`: grounding — `tried` is
declared outside the extent loop with a shared `if tried >= max_tries`
break. Mutation: moving the declaration inside the loop would give each
extent its own budget (80,000 total instead of 50,000).
- `crack_key_scans_from_extent_start_lba`: grounding —
`reader.read_sectors(ext.start_lba + i, ...)`. Mutation: dropping
`ext.start_lba +` would start LBAs at 0 instead of the extent's start.
- `crack_key_continues_past_read_errors`: grounding — a read `Err` falls
through to the next sector rather than aborting. Mutation: propagating the
error with `?` would stop the scan after the first failure.
- `crack_key_empty_extent_reads_nothing` / `crack_key_no_extents_is_none`:
grounding — `while i < ext.sector_count` / `for ext in extents` are no-ops
on empty input; an off-by-one (`<=`) or a pre-loop read would break these.
- `crack_outcome_reaches_cracked_with_span`: audit gap "MockSource never
yields a crackable sector" — drives the full `crack_key_scan` over a
synthetic ISO to exercise the `Cracked` branch and `crack_span` recording
end-to-end, previously untested.
- `is_scrambled_uncracked_is_true_for_that_case_and_false_for_the_other_two`:
a prior round introduced the three-way split precisely because conflating
the cases made an uncrackable disc exit 0 with garbage output; earlier
tests only asserted the TRUE direction, so a predicate answering "yes"
unconditionally was indistinguishable from a correct one. All three
outcomes here come from real `crack_key_outcome` scans, not hand-built enum
values.
- `resolve_dvd_title_key_*` tests: `resolve_dvd_title_key` is the SINGLE
shared per-title CSS step both read paths (`build_iso_pipeline` multi-pass
and `DiscStream::new` single-pass) call, so these pin its full contract at
the shared boundary — crack success, hard-fail on scrambled-uncrackable
(never leaving `keys` as `None`, which would mux scrambled bytes as
plaintext), the decrypted-HD-DVD RDI-pack regression (end to end), `--raw`
passthrough, the AACS-keys-untouched gate, the clear-DVD no-op, and the
halt-cancellation-surfaces-as-Halted fix (not a truncated-scan verdict).
- `all_locked_synthetic_iso_yields_css_key_missing_signal`: audit §2/§5 #7 —
pins that an all-locked synthetic ISO across MULTIPLE extents produces
`ScrambledUncracked`, the exact signal `disc/mod.rs` converts into
`disc.css_error = Some(Error::CssKeyMissing)`.
- `recrack_succeeds_on_other_vts_extents`: audit gap "success path missing"
— the prior re-crack test only covered the locked→None path; this proves a
key cracked for one VTS is genuinely re-derived (not reused) for another.
+32
View File
@@ -0,0 +1,32 @@
# Warner CTRM label parser — relocated rationale
Referenced from `src/labels/ctrm.rs` unit tests via `// See docs/ctrm.md` pointers.
## Immunity pin: no analog of the `paramount` forced_sub false positive
Immunity pin against the defect measured in the `paramount` parser, where a
vendor `forced_sub` cell hung off a FULL dialogue track's own slot to say
"this track also contains forced signs", and reading that cell as "this
track is forced" flagged 30 MB dialogue tracks forced.
This format cannot express that. The forced signal is not a flag beside a
track's entry — it IS the entry's stream-kind token, drawn from a closed
vocabulary in which `subtitle_production` (the full dialogue track) and
`subtitle_narrative` (the forced-narrative track) are mutually exclusive
alternatives in the same position. A row is one or the other; there is no
cell a full track can carry to acquire the qualifier, so the paramount
failure mode has no encoding here.
Mutation: give `subtitle_production` a `Forced` qualifier, or add a forced
side-flag that both kinds may carry.
## Immunity pin: stream numbers come from the row, not a counter
`language_streams.txt` states each stream's number in field 3, so a row the
parser cannot use is simply dropped — it can never renumber the rows behind
it. This is the property that keeps this parser out of the STN-slot-shifting
failure mode that bites parsers which count positionally: there, a skipped
entry silently pulls every later label one stream forward.
Mutation: replace `parts[2]` with a running per-type counter → the three
unusable rows in the corresponding test collapse the survivors onto 1/2 and 1.
+39
View File
@@ -0,0 +1,39 @@
# `labels::dbp`
## Implementation
Iterates `CpInfo::Utf8` constant-pool entries rather than raw
byte-scanning each class file. Equivalent label coverage (the literal
`TextField,...` strings live in the CP as Utf8 entries) with no
false-positive risk from method bytecode or attribute names that
happen to contain `TextField,`. Language / purpose / qualifier
classification lives in `super::vocab` so all Java-parser families
share one source of truth.
The parser ignores any prefix before the first `TextField,` occurrence
in a constant string — whatever string-pool ordering placed ahead of
it is irrelevant.
## `MAX_LABEL_BYTES` rationale
The label is an owned copy of a slice of a `CONSTANT_Utf8_info` entry,
whose `length` field is a `u16` (JVMS §4.4.7) — so a single crafted
constant contributes up to 65535 bytes, and the `u16` stream-number
keyspace admits 65536 of them per type.
Headroom: real dbp menu labels are short display names — "English Dolby
Atmos" (19 bytes), "Spanish 5.1 Dolby Digital" (25). The longest plausible
retail string ("Portuguese (Brazilian) 5.1 Dolby Digital Plus") is 45
bytes. 256 leaves >5x headroom over that, and any string past it is menu
geometry or padding, never a language name — `vocab::lang` would not
resolve it anyway.
## `MAX_LABELS_PER_TYPE` rationale
The keys come from `parse::<u16>()` on disc bytes, so all 65536 slots per
type are reachable; paired with `MAX_LABEL_BYTES` this bounds the whole
scan at 2 x 512 x 256 bytes.
Headroom: the BD STN_table admits at most 32 primary audio and 32 PG
streams per playlist, and dbp emits one menu TextField per stream. 512
leaves 16x headroom over the spec maximum.
+500
View File
@@ -0,0 +1,500 @@
# decrypt.rs — relocated rationale
Overflow prose that would not fit under `ci/comment-guard.py`'s caps,
moved here per file with a `// See docs/decrypt.md — <topic>` pointer
left at the original site.
## Parallel AACS decrypt
Each AACS aligned unit (6144 bytes) is decrypted INDEPENDENTLY of every
other unit — per-unit key derivation from the unit_key plus the unit's
own first-16-byte header. There is no cross-unit dependency, so a
buffer of N units can be decrypted on N threads in parallel via a
persistent rayon thread pool.
Small buffers (< `PARALLEL_MIN_UNITS` units) fall through to the serial
path to avoid pool dispatch overhead beating the per-unit AES work.
## Thread-count configuration — three layers
Resolution order (highest wins):
1. The most recent `set_decrypt_threads` call with `n > 0`. Calling
this *replaces* the live thread pool — useful for a settings-page
slider in a long-running daemon.
2. `FREEMKV_THREADS` env var, if set and `> 0`. Single knob covering
decrypt today, intended to also drive any future input-side /
output-side worker pools.
3. Default: all available cores. Algorithm optimisation comes first —
we measure single-thread performance to find serial bottlenecks
before throwing parallelism at it — but once a pool is engaged we
use the whole box. Hard cap at `MAX_THREADS` (rayon stack memory).
## DECRYPT_POOL
Current rayon pool. `RwLock<Option<Arc<...>>>` so that
`set_decrypt_threads` can swap the pool out without leaking the old
one and without blocking ongoing decrypt work (in-flight calls hold an
`Arc` clone via `decrypt_pool` and finish on the old pool; new calls
pick up the new pool).
## decrypt_pool()
Get (or lazily build) the active rayon thread pool. Returns an `Arc`
so in-flight work survives a concurrent `set_decrypt_threads` swap.
Returns `None` if the pool cannot be built (e.g. the OS refuses the
worker threads under a pid/thread limit). The caller falls back to the
serial decrypt path — library code never panics here.
## unit_is_our_phase
Does this unit belong to the phase we hold the key for?
An FMTS forensic segment interleaves two variants at the unit level:
the disc carries both, and we hold the key for exactly one parity.
Decrypting the alternate half with our key produces garbage; leaving
it as ciphertext is correct, because the muxer drops untouched
ciphertext cleanly.
`Phase::All` means the whole range is ours (the non-forensic case).
This lived inline inside `apply_aacs_map`'s per-unit closure, where
nothing could reach it: a mutation run flipped the `-` to `+` and the
`/` to `*` in the index arithmetic and every test still passed.
Getting either wrong silently decrypts the wrong half of a forensic
segment.
## AacsKeyMap (proactive AACS key-selection map)
Which held unit key decrypts each LBA of a title's encrypted content,
decided ONCE before mux from the disc's CPS-unit (and, later, FMTS
segment) structure — never by trial-decrypt-and-check per unit at mux
time.
This is the pivot that ends the mux "key-server storm": the old path
decrypts a unit, checks whether the plaintext looks like clean MPEG-TS,
and — because authored-bad content never reaches that bar — concludes
"wrong key, fetch a fresh one" and re-asks the key service for units it
already holds the correct key for. There is NO per-unit byte pattern
that separates "correctly decrypted but authored-bad" from "still
encrypted", so that check is unanswerable. The map removes the
question: we resolve one key per CPS unit / segment up front (see
`resolve_mux_key_map`), record which LBA ranges each covers, and at mux
time simply "decrypt this LBA with key K" and trust it — bad TS is the
muxer's concern, exactly as for a physically-read clear disc.
Ranges are `[start_lba, end_lba)` → index into the `Aacs { unit_keys }`
pool, sorted and disjoint. The map is a POSITIVE list: an LBA in no
range is passed through untouched (no default key). How a single-CPS
disc is mapped depends on the caller: the whole-disc EXTRACT path uses
one blanket range `(0, u32::MAX, 0)` so every encrypted unit — parsed
title or orphan clip — resolves to key 0; the per-title MUX/sweep path
(`resolve_mux_key_map` → `content_map`) maps only the title's own
extents, so an orphan clip outside them is left as pass-through. Either
way, clear nav/filesystem sectors (encrypted-flag off) pass through.
## AacsKeyMap::read_plan
Build the FMTS **read plan**: the title's aligned units filtered down
to only the units this rip must actually read — every default / CPS
unit, plus, inside each forensic segment, ONLY our-phase (`Phase::Even`
/ `Phase::Odd`) units. The alternate-phase units are a different device
group's variant: a licensed player never reads them, and neither do we.
They are omitted from the plan entirely, so they are never fetched,
decrypted, or handed to the demux — the demux therefore sees one
gapless our-variant stream, with no ciphertext to trip a
concealed-gap resync (the old behaviour that dropped good frames
around every segment).
`extents` are the title's clip extents (unit-aligned in the interior; a
shorter tail is ordinary content and always kept). `unit_sectors` is
the AACS aligned-unit size in sectors (3). Contiguous kept units
coalesce into as few extents as possible so the producer still issues
large sequential reads across default content; only inside a ~480 KB
forensic segment do reads become unit-granular (every other unit). A
map with no forensic (Even/Odd) range returns `extents` unchanged — the
common disc is not touched.
The parity test is byte-identical to the decrypt hot loop
(`(unit_lba - range_start) / unit_sectors`), so a unit kept here is
exactly a unit `decrypt_sectors_mapped` would open, and vice-versa.
## apply_aacs_map
AACS scheme step: apply `map`'s per-unit keys to `buf`, in-place.
`base_lba` is the absolute LBA of `buf`'s first sector; each aligned
unit (3 sectors) is decrypted with the key the map assigns to its LBA.
There is NO key trial and NO `is_clean` verdict: the map already
decided the key from disc structure, so this applies it and moves on —
a unit that decrypts to authored-bad TS passes through for the muxer to
drop, never re-fetched.
A SCHEME, not a policy. It reports what it could not open by returning
`Err(DecryptFailed)`; the decision that an unopenable unit must never
be emitted belongs to `decrypt_span`, which is the one place that
decides it for every scheme. A map index outside the held pool is
likewise a fail-loud `Error::DecryptFailed`: the resolver's job is to
guarantee every selectable index is present, so a gap here is a
resolver bug, not silent loss.
## decrypt_sectors — CSS / clear detail
For CSS: descrambles per 2048-byte sector, self-cracking the title key
from the data (no external input). For `None`: a no-op. For AACS:
**always** returns `Err(DecryptFailed)` — AACS decrypts exclusively
through the resolved key map (`decrypt_sectors_mapped`), which keys
every content unit up front and fails at RESOLVE time when a key is
missing. Reaching this arm with AACS keys means a reader was built
without installing its map (a bug), so it fails loud rather than apply
a guessed key.
`unit_key_idx` is a legacy parameter kept so the CSS / `None` wrapper
signature stays stable; it is ignored (the CSS arm self-gates on its
per-sector scramble flag). Returns `Err` if decryption was expected but
impossible; never produces silently corrupted output. The `usize`
return is a legacy unverified-byte count that is always `0` for the CSS
/ `None` arms.
## decrypt_span — orchestrator rationale
THE decrypt orchestrator. Every path into this crate's decryption goes
through here.
How a disc decrypts is one process — resolve a key for this span,
apply it, and refuse if no key can be proven. Only the resolve-and-apply
step is scheme-specific. This function owns the loop and the refusal;
the schemes below supply only what genuinely differs between AACS, CSS
and clear media.
That split exists because its absence caused six separate defects in
one release. There used to be TWO top-level paths — this one for CSS
and clear, and a wholly separate `decrypt_sectors_mapped` for AACS
whose arm here was a bare `return Err` stub — so each scheme decided
its own answer to "there is no key for these bytes" and nothing held
them to the same one. CSS drifted to descrambling with a key it had
just proven stale; the mapped path drifted to passing an unkeyable
encrypted unit through as ciphertext. Both looked like success to the
caller.
Adding a scheme means adding an arm here, which means answering the
refusal question. That is the point.
## Tests — relocated rationale
### a_forensic_read_plan_drops_units_the_full_extents_include
`DecryptKeys::None` is a no-op even with a content map + scrambled
bytes. A forensic map's read plan is NOT the extents it was given —
and that is the precondition the mux's provenance guard keys on.
A clip's feed span is measured over the FULL extents at scan time,
while the mux reads this reduced plan, so the byte offsets stamped on
frames and the offsets recorded in the spans describe different
streams. The deficit accumulates, so every frame after the first
segment resolves to an earlier clip than it came from. The spans still
tile each other, so the tiling check cannot see it; the mux compares
the plan against the full extents instead and stops trusting
provenance when they differ.
### content_gate_css_keys_is_noop
CSS ignores the content gate (it lives in the AACS arm) and always
reports `0` — confirming the gate is a no-op for CSS and the read
stays scheme-agnostic (the litmus test: adding CSS verify touches only
the CSS arm, never the read).
### content_gate_css_actually_descrambles_the_buffer
`decrypt_sectors_in_content` is the entry point `DecryptingSectorSource`
dispatches to whenever a content map is installed
(`sector/decrypting.rs` line ~211), so it is on the live read path for
every mapped rip. It must actually DECRYPT. The two `_is_noop` tests
above only assert its `usize` return is `0` — which is what a body
replaced by `Ok(0)` also returns, so neither one constrains it at all.
Here a genuinely scrambled CSS sector goes in and the CONSTRUCTED
plaintext must come out. Anything that skips `css::descramble_region` —
including a body that just reports `Ok(0)` — leaves ciphertext in the
buffer and the caller muxes scrambled MPEG at exit 0.
Expected bytes come from the plaintext this test built BEFORE
scrambling (CSS scrambles only 0x80..2048; the header stays clear), not
from re-running any descramble routine.
### content_gate_aacs_keys_fail_loud_not_ok_zero
The AACS arm of the same entry point must fail LOUD. Under the
keymap-only model AACS decrypts exclusively through
`decrypt_sectors_mapped`; reaching this wrapper with AACS keys means a
reader was built without installing its key map, and continuing would
hand the caller ciphertext under an `Ok`. `DecryptFailed` is the
correct verdict per the function's own contract — it must not be
softened into a success with a zero count.
### css_region_change_recracks_the_title_key
CHARACTERIZATION (recovery refactor safety net): the CSS arm's
per-region re-crack (the `title_key` cache is stale for a new VOB
region → restore ciphertext, `crack_title_key` this sector,
re-descramble). Two crackable sectors scrambled under DIFFERENT keys
sit back-to-back; the cache is primed to the FIRST key. Sector 0 rides
the cache (crib matches); sector 1 must trip the crib mismatch and
re-crack to its own key. Both must land correct plaintext, and the
cache must end on region 1's key.
This behaviour currently lives inline in `decrypt_sectors` (the `Css`
arm). It is the delicate logic the recovery refactor will move to the
input-stream seam, so it must stay green byte-for-byte across that
move.
### aacs_scrambled_trailing_partial_is_rejected
Whole leading unit plus a SCRAMBLED trailing partial that is FLAGGED
encrypted in its clear seed (the malformed danger case): an encrypted
unit split across an extent boundary cannot be CBC-decrypted
standalone. The mapped decrypt must fail loud with `DecryptFailed`
rather than emit the ciphertext partial as clear. Exercises the real
shipping path (`decrypt_sectors_mapped`) and its trailing-partial
guard.
### aacs_clear_trailing_partial_passes_through
A CLEAR trailing partial (encrypted flag NOT set) is a legitimate
content tail and must pass through, never trip the guard above.
"Passes through" means byte-for-byte unchanged, not merely `Ok`.
Asserting only `is_ok()` let a mutant that corrupts the clear partial
while still returning `Ok` pass — which is the whole failure this test
names. Mutation: XOR any byte of the tail before returning -> the
snapshot comparison fails.
### none_keys_is_noop
`DecryptKeys::None` is a pure no-op: the buffer must be returned
byte-for-byte unchanged with Ok, regardless of content (even content
that looks scrambled).
Grounding: the `DecryptKeys::None => {}` match arm does nothing.
Mutation: replace the empty arm with a call that mutates buf -> the
unchanged assert fails.
### is_encrypted_matches_variant
`is_encrypted` reflects the variant: None -> false, Css/Aacs -> true.
Grounding: `!matches!(self, DecryptKeys::None)`.
Mutation: invert the `!` -> None reports true, this fails.
### css_descrambles_with_title_key
The CSS path descrambles each 2048-byte sector with the title key. A
scrambled sector run through decrypt_sectors must come back to its
plaintext body (keystream XOR is involutive), proving the title key is
actually applied.
Grounding: `DecryptKeys::Css { title_key } => for chunk in
buf.chunks_mut(2048) { descramble_sector(title_key, chunk) }`.
Mutation: change `chunks_mut(2048)` to `chunks_mut(2049)` or pass a
fixed wrong key -> the body no longer matches the plaintext.
### css_processes_every_sector_in_buffer
The CSS path processes EACH 2048-byte sector independently in a
multi-sector buffer. Two scrambled sectors (with different seeds) in
one buffer must both round-trip — pinning that the loop steps by 2048
and applies the key to every sector, not just the first.
Grounding: `for chunk in buf.chunks_mut(2048)`.
Mutation: change the loop to descramble only the first chunk (e.g.
`.next()`) -> the second sector stays scrambled, assert fails.
### css_rekeys_when_title_key_region_changes
CSS title keys are per-VTS/VOB region: a real disc holds DIFFERENT keys
for different regions and the only way to get each is to crack it. The
decrypt path must re-crack when the cached key stops descrambling (its
crib no longer reappears at 0x80) instead of blindly applying one key
across a region boundary — the bug that pixelated every freemkv DVD
rip.
Two sectors scrambled under DIFFERENT keys, cache primed to ONLY the
first (exactly what the one-shot scan crack leaves). Sector 0 validates
+ descrambles with the cached key; sector 1's cached-key descramble
fails the crib, so the path re-cracks sector 1's own key and recovers
its plaintext. Before the fix (blind single-key apply) sector 1 was
garbage.
Grounding: the CSS arm's `attack_crib` → `chunk[0x80..] != crib` →
`crack_title_key` → `*title_key = fresh` rekey.
Mutation: drop the rekey branch (apply the cached key always) →
sector 1's body no longer matches its plaintext; this fails.
### css_leaves_clear_sector_unchanged
The CSS path leaves UNSCRAMBLED sectors (flag clear) byte-for-byte
untouched — descramble_sector early-returns on a zero flag. A clear
sector mixed into the buffer must not be corrupted.
Grounding: descramble_sector returns immediately when
`(sector[0x14] >> 4) & 0x03 == 0`.
Mutation: remove that early return in lfsr.rs -> a clear sector would
be XORed with a keystream and change; this fails.
### css_empty_buffer_is_ok
CSS decrypt always returns Ok (it cannot fail — descrambling is XOR,
no key validity check), even for an empty buffer.
Grounding: the CSS arm has no `return Err` path; `chunks_mut` over an
empty slice is a no-op; the function ends `Ok(())`.
Mutation: make the CSS arm return Err -> this fails.
### aacs_mapped_out_of_range_key_idx_errors
A map that selects a key index OUTSIDE the held pool must fail loud
with DecryptFailed — never silently apply a wrong key or pass
ciphertext through. This validates `decrypt_sectors_mapped`'s up-front
`key_indices()` bounds check (the real shipping AACS decrypt path).
Mutation: drop the `unit_keys.get(idx).is_none()` guard → the
out-of-range index would not error; this fails.
### aacs_via_unmapped_decrypt_sectors_fails_loud
SAFETY NET: reaching the CSS/`None` wrapper (`decrypt_sectors`) with
AACS keys means a reader was built with no map — a bug. It must fail
loud, never apply a guessed key. (AACS decrypts exclusively via
`decrypt_sectors_mapped`.)
### read_plan_forensic_reads_only_our_phase_units
FMTS: a forensic Even segment drops exactly its alternate (odd) units
from the read plan — they are never fetched — while default content on
either side stays one coalesced sequential run. The kept units are
byte-identical to the ones the decrypt hot loop opens.
### read_plan_phase_parity_is_measured_from_an_unaligned_range_start
A forensic range does NOT start on an aligned-unit boundary. Its start
LBA comes from a source-packet number — `start_spn * 192` put through
`clip_byte_to_lba` (`mux/resolve.rs`) — and 192-byte packets have no
relationship to the 3-sector aligned unit, so `range_start % 3` is
whatever the disc says.
That makes `unit_ix = (lba - range_start) / us` load-bearing in both of
its operations, and the existing coverage used a range starting exactly
on the extent's first unit, where several wrong formulas agree with the
right one by arithmetic accident.
Getting the parity wrong is not a crash. It reads and decrypts the
ALTERNATE variant's half of a forensic segment: the units this disc's
key does not open decrypt to garbage, and the units it does open are
skipped. AACS 2.1 forensic marking is exactly the mechanism that makes
the two halves different, so a phase inversion is silent — it produces
a full-length rip carrying the wrong variant.
### read_plan_gates_the_last_whole_unit_of_an_extent_not_just_the_remnant
An extent whose last whole unit is an alternate-phase unit must still
drop it. The tail guard exists for a REMNANT shorter than a unit —
bytes with no following unit to desync — and an extent ending exactly
on a unit boundary has no remnant at all.
With the guard widened to `remaining <= us`, the final unit of every
extent bypasses the phase gate and is read unconditionally. On a
forensic segment that lands at an extent end, that is one
alternate-variant unit pulled into the rip and decrypted with a key
that does not open it.
### every_scheme_gives_the_same_verdict_when_no_key_can_be_proven
Every scheme that CANNOT prove a key answers the same way.
This is the property `decrypt_span` exists to hold. There used to be
two top-level decrypt paths — one for CSS and clear media, one for
AACS — and each decided its own answer, so they drifted apart in
opposite directions within a single release: CSS descrambled with a
key it had just proven stale, and the AACS path passed an unkeyable
encrypted unit through as ciphertext. Both reported success.
Asserting one verdict across the schemes is what makes a future
divergence a test failure rather than a silent corruption. A
per-scheme test cannot do that: each would still pass while the two
disagreed.
CSS is deliberately NOT in this list. Its title key is recovered from
the data, sector by sector, by a heuristic that false-positives — a
crib mismatch whose re-crack fails means the crib was wrong, not that
the key is stale, so the cached key is kept and used. Round 9 folded
CSS in here on the reasoning that "no key" should mean one thing
everywhere; that made real DVDs unrippable, and the real-media gate
caught it. Uniform policy is right for schemes that can PROVE a key
wrong. CSS cannot.
### an_encrypted_unit_outside_every_key_range_fails_instead_of_passing_through
An ENCRYPTED unit that falls outside every key-map range must fail,
not pass through as ciphertext.
"The map has no key here" and "there is nothing to decrypt here" are
different statements, and only the second makes passing the unit
through correct. On a multi-CPS disc an orphan clip — referenced by no
playlist, so in no title extent and therefore in no range — hits the
first and was treated as the second. `extract_tree` then counted those
bytes as GOOD, dropped the `.partial` suffix, and reported `complete:
true`, exit 0: a scrambled file on disk with a clean bill of health.
A CLEAR unit outside every range is the ordinary case (filesystem and
nav on a whole-disc read) and must still pass through untouched — so
this asserts both directions.
### mapped_key_selection_is_positional_so_pool_order_matters
The mapped descramble indexes the committed key pool POSITIONALLY
(`unit_keys[key_idx].1`), so the ORDER of the `Vec<UnitKey>` a
`KeySource` returns is load-bearing — it is NOT "cosmetic, the decrypt
path strips it and tries every key", as
`keysource::resolve_and_apply_traced`'s doc used to claim. Trial-decrypt
was deliberately deleted; nothing here searches the pool. Reordering
the same two keys therefore sends each range to the WRONG key: the
range that decrypted clean now fails the correct-phase `is_clean` net
loudly (or, off a forensic phase, would decrypt a whole span under a
neighbour's key). Pins the corrected doc.
### decrypt_threads_within_valid_pool_range
The default (auto) decrypt thread count is always a usable pool size:
at least 1 (a 0-thread rayon pool is invalid) and never above
MAX_THREADS (rayon stack-memory cap). This test reads the resolved
value without mutating the process-global override, so it is safe to
run in parallel with other tests.
Grounding: `cores.clamp(1, MAX_THREADS)` in the default branch;
`env.min(MAX_THREADS)` in the env branch.
Mutation: change `.clamp(1, MAX_THREADS)` to `.clamp(0, MAX_THREADS)`
on a 0-core probe (unlikely) — more robustly, change the cap to
`MAX_THREADS * 2` -> on a many-core CI box the upper-bound assert can
fail. The lower-bound (>=1) guard is the load-bearing invariant.
### phase_gate_selects_only_our_parity_of_a_forensic_segment
The FMTS phase gate picks which half of an interleaved forensic
segment we decrypt. Both the `-` and the `/` in its index arithmetic
survived a mutation run, and getting either wrong silently decrypts the
alternate variant into garbage while reporting success.
### phase_gate_does_not_panic_on_a_malformed_map
A malformed key map must not take down a long-running service. A unit
below its own range start, or a zero unit size, are both map bugs —
they must return a defined answer rather than panicking on debug
overflow or dividing by zero.
Every case here asserts the DEFINED answer, not merely the absence of a
panic. The zero-unit-size case used to be written
`assert!(unit_is_our_phase(100, 30, 0, Phase::Even) || true)`, which
accepts both answers and so pinned nothing at all: the guards could
invert and it would still pass. The answer is knowable —
`saturating_sub` gives 70, `max(1)` makes the divisor 1, unit index 70
is even — so pin it.
</content>
+34
View File
@@ -0,0 +1,34 @@
# `DecryptingSectorSource`
## Why this decorator is the single source of truth for decrypt-on-read
Every decrypt-on-read caller (e.g. `DiscStream`) wraps its source in this
decorator rather than calling the cipher helpers directly. The actual cipher
code lives in `crate::aacs` and `crate::css`; the decorator just calls the
existing `crate::decrypt::decrypt_sectors` helper that drives both of them
in-place after each read. Centralizing the call site means callers can wire
the decorator unconditionally and keep their pipeline shape uniform
regardless of encryption state — for `DecryptKeys::None` discs it is simply
a pass-through, so there is no branch in caller code for "is this disc
encrypted?".
## `KeyFetch`: why two explicit operations
`KeyFetch` exposes `unit_keys` and `fmts_indexes` as two separate operations
rather than one generic callback, so the "one base key vs. a whole forensic
set" contract lives in the type instead of a caller guessing at the return
length:
* `unit_keys` resolves the base Unit Key(s) for a CPS unit from real
encrypted samples drawn from it. The non-forensic path: one key per CPS
unit (the pool grows by whatever it returns). Used by the mux's base /
multi-CPS map resolution and by the sweep/patch recovery decorator.
* `fmts_indexes` resolves the disc's AACS 2.1 forensic index keys from an
index-1 single-phase anchor batch. The source hands back the COMPLETE set
(ordered index 1..N); the caller sizes the forensic map to `len()` and
never assumes a fixed N (32 is all we've seen, but the contract is
"whatever the source returns, >= 1, is all of them").
`KeyFetch` is a stateless, shared pair of `Arc<Fn>`, so one instance is built
once and cloned cheaply (two `Arc` bumps) into every read path; it is
`Send + Sync` so it can ride the mux highway's producer thread.
+227
View File
@@ -0,0 +1,227 @@
# Deluxe BD-J framework parser (`src/labels/deluxe.rs`)
Detected on discs whose `/BDMV/JAR/<x>.jar` contains a `com/bydeluxe/`
directory entry.
## What this parser reads
Deluxe-authored discs store stream labels as ordinal references into
enum classes whose names are obfuscated per-disc, so a name-based
match won't work. The label data is instead recovered by matching on
the **shape of each enum's `<clinit>`**, which is framework-stable:
| Enum | Signature |
|---|---|
| Language | 70 `ldc` operations in `<clinit>`, sequence starts `English, French, Spanish, Dutch, ...` |
| Purpose | 8 ldcs starting `Normal, Commentary, PiP, Trivia, ...` |
| VideoFormat | 7 ldcs starting `HD, HDR10 Plus, HD Dolby, ...` |
| Region | 22 ldcs starting `USA_D1, LIC1, LIC2, LIC3, ...` |
| Studio | 6 ldcs in `<clinit>` |
Matching on the shape rather than the class name keeps the parser
working across obfuscation variants. Codec strings come from the
standard BD-J `org/bluray/ti/CodingType` enum referenced directly by
the binding constructors (see `StackVal::CodingType`), not from a
Deluxe-internal enum.
## Implementation phases
- **Phase A** — master enum identification (`identify_master_enums`).
Walks every `.class`'s `<clinit>` ldc sequence and matches against
the framework-stable fingerprints. Output: `Vec<(label, MasterEnum)>`
with full ordinal → string-value tables.
- **Phase C** — binding-class identification (`find_binding_classes`).
The per-stream table is built by some class via repeated
`getstatic` references to the master enums identified in A.
That class has the highest such `getstatic` count in the jar.
Heuristic shape; precise threshold may need tuning.
- **Phase D** — binding-class bytecode decoder (`decode_binding`).
Walks the binding class's `<clinit>` with a tiny symbolic stack
machine. For each `new X / dup / ... / invokespecial X.<init>`
sequence, collects the int values and enum-reference operands
between the `dup` and the constructor call, then emits a
`DecodedStream`. The signal-to-StreamLabel mapping (which arg is
stream index? which is language? audio vs subtitle?) uses a
heuristic — see `interpret_streams` for the mapping rules.
## Confidence
`parse` returns `Some(ParseResult::high(labels))` when Phases A
through D produce at least one stream: the master enums matched their
framework-stable fingerprints (a strong ordered-prefix signature), the
binding class decoded, and at least one per-stream binding resolved to a
real (language, purpose, codec) tuple — the schema was fully recovered,
not guessed. `None` when the disc isn't Deluxe-authored or when decoding
produces zero streams (a recognized-but-broken state that the analyzer
still surfaces via `parsers_detected`).
## Size caps (Phase A candidate pool)
`MAX_CLINIT_LDC_STRINGS` (cap on `ldc` operands retained per class by
`clinit_ldc_strings`): unlike every other count cap in this crate, the
paired byte cap here cannot be the disc-file size — a `.class` entry
gated only by a `com/bydeluxe/` path prefix deflates from ~100 KB up to
the 64 MiB `MAX_CLASS_BYTES` read ceiling, so ~33M two-byte `ldc`
instructions — one retained `String` each — are reachable from a small
crafted disc. The bound has to be on the decompressed work, so it is
applied here. Headroom: the largest framework-stable enum is `Language`
at 70 values (`FINGERPRINTS`), and no fingerprint matches a count more
than `LDC_COUNT_TOLERANCE` away from its expected size, so anything past
~74 can never identify a master enum. 4096 leaves ~55x headroom over the
largest real enum for framework drift.
`MAX_CLINIT_LDC_BYTES` (companion byte cap): the count cap alone still
admits 4096 x 64 KiB of `Utf8` (a JVMS `CONSTANT_Utf8_info` length is a
u16), i.e. ~268 MB per class from repeated `ldc` of one huge constant.
Headroom: master-enum values are short display names ("English",
"HDR10 Plus", "USA_D1") well under 32 bytes, so a real Language enum
retains ~1 KB. 256 KiB admits 4096 values averaging 64 bytes each.
`MAX_CANDIDATE_TOTAL_BYTES` (aggregate companion, bounds retention across
ALL classes at once): `identify_master_enums` holds every class's
retained strings in one map simultaneously, so the per-class cap alone
still admits `classes x 256 KiB`: a 64 MiB jar of minimal `.class`
entries reaches tens of GiB. This is the same bound one level up.
Headroom: the five `FINGERPRINTS` enums together hold ~113 short values
(~1.2 KB). Every other class in a real BD-J jar contributes only
whatever string constants its own `<clinit>` loads — resource paths,
config keys — so a large authored jar lands in the low hundreds of KB.
16 MiB leaves ~40x headroom over a deliberately generous 400 KB estimate
for real media.
`MAX_CANDIDATE_CLASSES` (entry-count companion to
`MAX_CANDIDATE_TOTAL_BYTES`): the byte budget alone still admits ~16M
map entries when every class retains a single one-byte `ldc`, and the
per-entry `HashMap` + `String` overhead is not counted by that budget.
Headroom: a large retail BD-J title ships on the order of 1-3k classes,
and only those with a non-empty `<clinit>` ldc sequence become
candidates. 65536 leaves >20x headroom over the class count of any real
jar.
## Internal helpers
- `clinit_enum_field_names`: collects the static-field names an enum
class's `<clinit>` stores its own instances into, in declaration order
— the ordinal -> field-name mapping that mirrors `clinit_ldc_strings`'s
ordinal -> value mapping. An obfuscated Deluxe enum constant compiles
to `new E; dup; ldc "Value"; invokespecial E.<init>(…); putstatic
E.<field>`, so the `putstatic` whose owning class AND field descriptor
are both this class's own type names the field for that ordinal. The
binding class then references that constant as `getstatic E.<field>`;
without this mapping `MasterEnumTable` cannot turn the obfuscated field
name back into an ordinal. Bounded by `MAX_CLINIT_LDC_STRINGS` like the
value walk.
- `clinit_ldc_strings`: walks `<clinit>` and collects every `ldc`/`ldc_w`
operand that resolves to a `String` or `Utf8` constant, in declaration
order; `None` if the class has no `<clinit>`. Collection stops at
`MAX_CLINIT_LDC_STRINGS` operands or `MAX_CLINIT_LDC_BYTES` of retained
text, whichever comes first: the walk is driven by the decompressed
class, so it isn't bounded by the disc-file size cap the way the rest
of this module's counts are. Truncation cannot lose a real match — a
truncated sequence is far longer than any `FINGERPRINTS` entry's
`expected_count + LDC_COUNT_TOLERANCE`, so it would have been rejected
on count anyway.
- `find_binding_classes`: identifies all binding-class candidates by
getstatic-count to the master enums, using `MIN_GETSTATIC = 4` — the
binding class on a typical disc has 50+ such getstatic references (one
per slot x arity); the low floor lets a small disc with few streams
still qualify, while still filtering out classes that just reference
the language enum once for a config string. Some Deluxe discs split
the per-stream table across two binding classes (one for audio, one
for subtitle — the audio class commonly has the most getstatic refs,
the subtitle class somewhat fewer, both sharing the master Language +
Purpose enums), so this returns top-K candidates ordered by descending
getstatic count, filtered to a minimum concentration of references.
`MAX_CONSTRUCTIONS` (Phase D, cap on `Construction`s retained from a
binding `<clinit>` walk): one entry is appended per matched `new X / dup
/ invokespecial X.<init>` with no other bound — a `.class` gated only by
a `com/bydeluxe/` path prefix deflates to the 64 MiB `MAX_CLASS_BYTES`
ceiling, and the shortest matching sequence is a handful of bytes, so
millions of `Construction`s (each a `String` plus an arg `Vec`) are
reachable from a small crafted disc (~1 GiB). The same cap bounds the
per-class union in `decode_binding_class` (a crafted class may repeat
`<clinit>`, which JVMS §4.6 forbids but this reader tolerates) and the
cross-class union in `parse`, so the whole phase retains at most this
many. Headroom: the BD `STN_table` (BDAV stream-entry counts) admits at
most 32 primary audio and 32 PG streams per playlist, and a Deluxe
binding table covers the disc's playlists — low hundreds of entries on
the largest retail titles. 4096 leaves >20x headroom.
`Decoder::push` (symbolic-stack push during binding `<clinit>` walking):
honours the Code attribute's declared `max_stack`. JVMS 4.7.3 requires
that a method's operand stack never exceed `max_stack` at any point, so
a push past it can only come from bytecode that would fail JVM
verification. Dropping it costs nothing on real bytecode and bounds the
decoder: a `.class` gated only by a `com/bydeluxe/` path prefix deflates
from ~100 KB to the 64 MiB `MAX_CLASS_BYTES` ceiling, i.e. ~67M
single-byte `iconst_0` (~2 GiB of `StackVal`) on an unbounded `Vec`.
Headroom: `max_stack` is exactly what javac computed for the real
binding `<clinit>`, so no real construction can be clipped. The symbolic
stack counts `long`/`double` as one slot where the JVM counts two, so
its depth is never greater than the verified depth.
## interpret_streams: Constructions -> StreamLabels
Converts the per-construction tuples from Phase D into `StreamLabel`s.
Two binding-constructor shapes are handled:
- 5-arg: `BindingType.<init>(I, Lang;, Lpurpose;, I, LCodingType;)V`
- 4-arg: `BindingType.<init>(I, Lang;, Lpurpose;, LCodingType;)V`
Args are identified by **TYPE**, not position:
- First `EnumRef{kind: "Language"}` -> audio/subtitle language
- First `EnumRef{kind: "Purpose"}` -> Deluxe purpose ordinal
- First `CodingType(name)` -> codec field name (translated via
`coding_type_to_codec_hint`)
- First `Int(n)` -> stream index (preserved as ordering hint;
per-type sequential stream_number is what actually goes into the
StreamLabel, since BD spec stream-numbering is anchored on MPLS
data, not the binding code)
Stream type inference:
- Construction has a `CodingType` arg -> audio stream (subtitles on
Deluxe don't carry a CodingType; their codec is implicit PGS via
the BD spec).
- Construction has Language but no CodingType -> subtitle stream.
- Neither, and its binding type never yielded a stream -> not a
stream (skip). See `slot_kind` for why the binding type is
consulted rather than the language alone.
`slot_kinds`: which stream list each binding type enumerates, learned
from the constructions that DID resolve a language. A `<clinit>` walk
emits a `Construction` for every `new X; ...; invokespecial X.<init>`
it sees, so the list mixes real stream bindings with whatever else the
class initializer builds. `binding_type` is the constructed class name,
which is how the two are told apart: the stream bindings all share one
class (Deluxe splits audio and subtitle across two), and that class is
identifiable from the slots that resolved. A binding type that resolved
as both kinds is left out — with no consistent answer, guessing a list
to advance would be worse than not advancing.
`slot_kind`: the stream list an unresolved construction occupies a slot
in, or `None` when it is not a stream binding. A
`org.bluray.ti.CodingType` argument is decisive on its own: nothing but
an audio stream binding is handed one. Otherwise falls back to what the
binding type's resolved siblings showed (`slot_kinds`).
## Confirmed studio variants
- **Universal** (`studio="uni"`): Language enum `pd` (65 values,
`English, French, Spanish, Dutch, …`), Purpose enum `lp`
(`Normal, Commentary, PiP, Trivia, Descriptive, Score`), audio binding
`np.<init>(I, Lpd;, Llp;, Lorg/bluray/ti/CodingType;)`, subtitle binding
`wb.<init>(I, Lpd;, Llp;, Lmi;)`, all built in one binding class's
`<clinit>` alongside a title-wrapper object whose constructor takes the
per-stream arrays (filtered out by the array-parameter guard in the
decoder). SDH/RNIB is encoded as a distinct Language VALUE
("English SDH", "English RNIB"), recovered into the qualifier from the
name. Grounded on fixtures captured from a real Universal Blu-ray release.
- **Disney/Warner**: 70-value Language enum, same binding shape; the
original corpus this parser was written against.
+74
View File
@@ -0,0 +1,74 @@
# `demux://` sink — per-codec ES output
The `demux://` sink is a write-only `crate::pes::Stream`. Instead of muxing
the per-track `PesFrame` stream into a single container (as `MkvStream`
does), it routes each frame's payload to the file for `frame.track`,
post-processing where the internal codec `Frame` form differs from the
standalone on-disk ES form:
- **HEVC / H.264**: the codec parsers emit hvcC/avcC-style 4-byte
length-prefixed NALs (start codes stripped) and carry the parameter sets
out-of-band in `codec_private`. A standalone `.hevc`/`.h264` needs Annex-B
framing with the parameter sets prepended — see `AnnexBWriter`.
- **PGS**: the parser collapses a display/clear PCS pair into one
duration-bearing `Frame` with the raw segment payload but no `PG` magic /
timestamp header. A `.sup` needs the HDMV segment framing rebuilt — see
`PgsSupWriter`.
- **VobSub**: the `.sub` is the raw SPU stream; the `.idx` sidecar is
synthesized from per-SPU PTS + byte offsets — see `VobSubWriter`.
Every other codec writes `frame.data` verbatim (`PassthroughWriter`).
The sink does NOT touch the MKV mux path; it is purely additive.
## `PgsSupWriter`: rebuilding HDMV segment framing
The parser hands the demux sink the concatenated PGS segments of a display
set in `frame.data` (segment_type + segment_size + payload, repeated), with
no `PG` magic and no PTS/DTS. A `.sup` prefixes each segment with a 13-byte
header: `0x50 0x47` ("PG") | PTS u32 BE | DTS u32 BE | (segment_type|size
already present in the payload). When the parser folded a trailing clear
(`duration_ns` set), the writer re-emits it as an empty composition at
`pts + duration` so players time the subtitle out.
## `synthetic_clear_display_set`: synthesizing the clear PCS
Builds a synthetic "clear" display set: an empty PCS (0 composition objects)
followed by an END segment. The parser folds the original clear/end PCS
pair's wipe time into the display frame's `duration_ns` and drops the clear
bytes, so a faithful `.sup` re-emits one here at `display_pts + duration`;
without it every subtitle lingers to EOF. `width`/`height` are carried from
the display set's PCS so the clear PCS advertises the same video geometry —
they don't affect the wipe but keep the segment well-formed. Returned bytes
are concatenated `type(1)+size(2 BE)+payload` segments, the same shape
`emit_segments` consumes.
## Test: `epoch_driver_follows_ref_video_not_track_zero`
Regression for the hardcoded `frame.track == 0` epoch driver: on an M2TS/PMT
title the PMT can list an AUDIO ES before the VIDEO ES, so the video lands
at stream index 1. The sink already resolves the video reference
dynamically (`ref_video_track`); the epoch driver must use that SAME
reference, not the literal 0. In the test, track 0 is audio and track 1 is
video; a video-only clip boundary (track 1) must open a new epoch (bump
`offset_ns`). With the bug (only `frame.track == 0` drives epochs) the
video back-jump would be treated as a passive rider and `offset_ns` would
stay 0 — corrupting every track's rebased timeline.
## Test: `a_demux_export_the_seam_plan_emptied_fails`
Audit finding: this sink builds its timeline with
`TimelineContinuity::with_clips` but no test ever gave it a title with
usable PlayItem marks, so `frames_mapped` and `dropped_total()` were always
zero and BOTH gates in `finish` were unreachable. Deleting either left the
whole suite green while a `demux://` export of a seamless-branching title
wrote zero-byte track files beside a populated chapters document, at exit 0.
## `sanitize`: path-hostile filename characters
Control characters are replaced, NUL above all. Every string this touches is
disc bytes, and a language code is three raw STN bytes run through
`from_utf8_lossy` with no validation — `00 00 00` is the ordinary
"undefined" encoding on real Blu-rays. A NUL in a path aborts
`File::create` with `InvalidInput`, which took the whole demux export down
before a single track file was opened.
+65
View File
@@ -0,0 +1,65 @@
# `DemuxThread`
## Why a second worker thread
With `crate::sector::PrefetchedSectorSource` alone, read+decrypt already
runs on a producer thread; the *consumer* (main) thread still serialises
`ts_demuxer.feed` (M2TS parsing) with the codec parsers. Profiling showed
feed at ~37% and codec parse at ~44% of consumer wall time — i.e. feed is
heavy enough that pipelining it with parse pays for itself.
Splitting them: feed runs in `DemuxThread`; the consumer thread receives
`Vec<PesPacket>` batches and runs codec parse + frame emission only. Total
throughput becomes `1/max(feed, parse)` instead of `1/(feed + parse)`.
## Lifecycle
`DemuxThread::spawn_zero_copy` consumes the prefetch channels and the
demuxer state, returning a handle plus a `Receiver<DemuxBatch>`. Dropping
the handle closes the channel which signals the worker to exit; the join
in `Drop::drop` blocks until the worker observes channel closure and
returns (no timeout — a wedged downstream would block the drop until it
releases the channel).
## `spawn_zero_copy` rationale
Instead of taking a `SectorSource` and memcpy-ing through its
`read_sectors` API, this constructor consumes the prefetch channels
directly: filled buffers come in via `prefetch_rx`, the demux thread
feeds them, then returns them to `recycle_tx` for the producer to
re-fill. Eliminates the 16 MiB memcpy per batch that the SectorSource
adapter incurred (and, with the producer-side recycling pool, also
eliminates the per-batch heap alloc / cross-thread free that was costing
40%+ of demux-thread time before).
## Test: empty-batch disconnect fix (`worker_exits_promptly_on_consumer_drop_during_empty_batches`)
Regression: worker must detect consumer disconnect even when every demux
batch is empty (no matching PIDs / null packets).
Before the fix, `tx.send()` was never called for empty batches so the
worker never observed the consumer drop — it would spin through ALL
remaining extents before exiting, causing `DemuxThread::drop`'s `join()`
to block for minutes on a mostly-untracked disc region.
The watchdog: if the worker doesn't exit within 1s of the consumer drop
the test fails (rather than hanging forever as the bug would).
## Test: spawn-failure drop-order (`channels_disconnected_before_producer_join_on_spawn_failure`)
Regression: on thread-spawn failure the channels must be dropped BEFORE
the producer shell so the upstream producer observes disconnection and
exits, allowing `join()` to complete without hanging.
A true EAGAIN/pids-limit spawn failure cannot be reliably forced in a
unit test without root or ulimit co-operation, so the test exercises the
drop-order contract directly: a mock shell that panics if `join()` is
called while either channel end is still open.
The test constructs a `(prefetch_tx, prefetch_rx)` pair where the tx side
is held by a sentinel that stays alive as long as either channel end is
open, then asserts that the sentinel is gone by the time
`producer_shell`'s join logic would run. Because a real spawn failure
can't be forced, the test instead verifies the helper logic in isolation:
drop `prefetch_rx` and `recycle_tx` first, then observe the producer-side
sender is disconnected, which is the property the fix relies on.
+65
View File
@@ -0,0 +1,65 @@
# diag.rs — scan diagnostics rationale
## Why this module exists
A bug report log must be self-diagnosing: everything needed to explain *why*
freemkv made the choices it did at scan must be in the log, in a compact,
machine-parseable form. `diag.rs` emits one terse line per row (title, cell,
stream, decision) under the `tracing` target `freemkv::diag`, which the CLI
routes to `log.txt` when `--log-level 3` (debug) is set.
## Format conventions (stable, greppable)
- Every line is prefixed by a `tag=` so a log scraper can filter (`disc`,
`title`, `dvd.cell`, `dvd.vattr`, `dvd.aattr`, `bd.clip`, `bd.mark`, `aacs`,
`stream`, `decision`).
- Raw bytes are shown as `0xNN` next to their decode so a wrong decode is
obvious against the raw value.
- This module only READS already-parsed scan state — it never re-reads the
disc and never mutates anything.
## DVD per-cell table vs. Disc-level dump
The DVD per-cell table (with the raw cell-category byte) is emitted from the
IFO scan itself (`dump_dvd_cells`), because the per-cell `ifo::DvdCell` detail
is lowered away before the `Disc` is built. The `Disc`-level dump
(`dump_disc`) covers everything that survives lowering: titles, streams, the
picked main feature, and AACS state.
## `dump_dvd_substream_probe`
Emits the ACTUAL per-physical-sub-stream AC-3 channel counts read off the VOB
during the mux-time sub-stream probe (the Silence-of-the-Lambs wrong-stream
fix). This is the ground truth the IFO nibble is compared against: each row is
`sub_id=0x8x channels=N` for a physical `private_stream_1` AC-3 sub-stream
whose first frame was decoded. An empty probe (scrambled / unreadable / short
VOB) logs a single `probed=0` line so the absence is explicit in a bug log.
Self-sufficiency: with `tag=dvd.aattr` (the IFO's declared sub_id + claimed
channels) and these `tag=dvd.substream` rows (the physical reality), a bug log
alone shows whether the ordinal `0x80` actually carries the declared channel
layout — no disc needed to diagnose a wrong-substream rip.
## `dump_mkv_track`
Emits the MKV `TrackEntry` elements the muxer is about to WRITE for one track
— the Windows-fps-class metadata (FlagInterlaced, FieldOrder,
DefaultDuration, DefaultDecodedFieldDuration, Display dims) plus the
codecPrivate as hex. With this row a bug log alone is enough to verify why
Windows Explorer reports a given frame rate for an interlaced SD track: the
container values that drive its fps derivation are all present, no disc and
no MediaInfo needed.
`track_number` is the 1-based MKV track number; `track` is the built
`crate::mux::mkv::MkvTrack` whose fields map one-to-one onto the emitted
elements (see `MkvMuxer::new`).
## `main_feature_reason`
The `reason=` token on the main-feature decision row is DERIVED from
`Disc::CANONICAL_TITLE_ORDER_KEYS`, which lives beside the comparator that
actually implements them — never restated here. The previous hand-written
copy drifted (it advertised a `fewest-clips` key the comparator had replaced
with largest-physical-size), which made the self-diagnosing log explain the
pick with a rule the code does not apply. A diagnostic that disagrees with the
decision it documents is worse than no diagnostic.
+132
View File
@@ -0,0 +1,132 @@
# Rationale for `src/dirimage/tests.rs`
Longer-form notes for individual tests, referenced from `// See docs/dirimage-tests.md`
pointers in the source so the inline comments stay within the comment-guard caps.
## `Scratch::unique_path`
The path a scratch dir takes is kept separate from directory creation so its
uniqueness is testable without a syscall between draws. A monotonic counter,
NOT a timestamp, is what guarantees it: `SystemTime::now()` resolves to only
a MICROSECOND on macOS, so two of the seven parallel tests that share the
"bdmv" tag routinely read the same value within one microsecond, collide on
the same directory, and the first to finish `remove_dir_all`s it out from
under the others' reads — an intermittent `read_sectors` ENOENT. The counter
is unique regardless of clock granularity; pairing it with the pid keeps it
unique across test-binary processes.
## `scratch_paths_are_unique_even_at_clock_resolution`
Every `Scratch` must own a DISTINCT directory. Seven tests share the "bdmv"
tag and run in parallel; if two land the same path, the first to drop
`remove_dir_all`s it out from under the other's reads, which surfaced as an
intermittent `read_sectors` failure. Regression: with the old
`SystemTime::now()` name this is RED, because macOS's clock advances only
per microsecond, so a tight loop of pure draws — no `create_dir_all` syscall
between them to nudge the clock, mirroring the real cross-thread collision —
hands back the same value many times over. The counter-based name is unique
regardless of clock granularity.
## `a_directory_spanning_many_blocks_reads_back_whole`
A directory with enough children that its FID list spans several 2048-byte
blocks, read back through the production parser.
This is the case the block-alignment question turns on: FIDs are packed
contiguously and a descriptor may straddle a block boundary, because
`read_directory` (`udf.rs:1312`) walks the directory extent as one flat byte
run and BREAKS at the first non-257 tag. Padding each block would truncate
the directory at the first pad. 200 entries also exceeds the 16-handle LRU
several times over, so it exercises handle eviction on the read path.
## `a_file_below_video_ts_is_placed_not_just_declared`
A file inside a SUBDIRECTORY of `VIDEO_TS` must still have its data placed.
Audit finding. The DVD branch places only the files directly inside
`VIDEO_TS`, because only those carry the IFO-relative constraints — and the
follow-up loop skipped that directory entirely, so anything one level deeper
got a File Entry declaring the file's real size with no extents behind it.
It appeared in the tree at full length and read back as nothing, at exit 0.
The identical folder under `BDMV/` was always placed correctly, which is
what made the gap easy to miss.
## `a_vob_offset_past_the_image_cap_is_refused`
A VOBS offset far past the content must be REFUSED, not honoured.
Audit finding. The planner honours a title set's declared VOBS offset
because honouring it is what makes a real backup readable, and nothing
bounded it: a regenerated `.BUP`, a rewritten IFO or a hand-assembled folder
naming a huge offset grew the image to wherever it pointed. A `u32` sector
count reaches roughly 8.8 TB, and writing that to an `iso://` destination
would fill a disk with zeros before anything noticed.
The companion (`an_oversized_vob_offset_leaves_a_gap_rather_than_failing`)
pins that a MODEST oversize is still honoured as a gap, so this cap refuses
only what it must.
## `playable_bdmv`
A BD folder that really enumerates a title, so the AACS content probe has
an extent to sample.
The `.m2ts` is built as real 192-byte BD source packets, because that is
what the probe judges: byte 0 carries the AACS Copy Permission Indicator in
its top two bits, and byte 4 is the MPEG-TS sync. `scrambled` sets the CPI
and withholds the sync — "flagged and not structurally clean", which is
exactly `aacs_unit_needs_decrypt`. An all-zero payload would prove nothing
either way: `is_clean_ts` skips zero payloads as padding.
## `both_doors_agree_on_a_clear_folder_that_kept_its_aacs_directory`
The two doors must AGREE. Same folder, same verdict, same extents.
Stated as a differential rather than a bare `is_ok()` so it cannot go
vacuous: if the fixture ever stops producing an AACS state, a one-sided
`Ok` assertion would still pass while guarding nothing, whereas "both doors
see the same title" is the invariant the shared function actually exists to
hold.
## `a_scrambled_folder_is_refused_through_the_dir_url_door_too`
The other verdict, through the same door: a folder whose content units are
really scrambled is refused with the TYPED code, not muxed into garbage.
This is the load-bearing half. `E_DIR_IMAGE_ENCRYPTED` is produced at
exactly one site in the crate (`session::apply_folder_encryption_verdict`),
reachable from here only through the `is_folder` argument — so this test
cannot pass if that argument is dropped, whatever else changes.
## `write_and_mount_externally`
Write a synthesized image to a real file and ask the OS to mount it.
This is the only check here that is not circular: `read_filesystem` shares
every assumption with the encoder, an operating system's UDF driver shares
none of them. Ignored by default because it shells out to `hdiutil` and
needs a host that can attach an image; run with
`cargo test -- --ignored write_and_mount_externally --nocapture`.
## `dvd_placement_invariant_on_a_real_folder`
Diagnostic (opt-in): verify the DVD placement invariant for every title set
in a REAL folder. `ifo.rs` derives a title's extents as
`file_start_lba(VTS_nn_0.IFO) + vtstt_vobs`, so that sum must land exactly on
`VTS_nn_1.VOB` or the mux reads the wrong sectors for that title.
Run with: `FMKV_DVD_FOLDER=/path/to/tree cargo test --lib
dvd_placement_invariant_on_a_real_folder -- --ignored --nocapture`
## `a_multi_clip_playlist_produces_feed_spans_that_tile_the_feed`
`Clip::feed_span` is the INPUT to the whole provenance feature — it is what
tells the muxer which clip a byte offset belongs to — and it is produced in
exactly one place, `disc/bluray.rs`. Every `SeamPlan` test synthesizes spans
by hand, so the consumer is thoroughly tested against fixtures written from
the same assumptions as the producer, and nothing checks the producer at all.
That is the shape of the original defect: the placement logic was tested and
correct, and the thing feeding it was wrong. `SeamPlan` only trusts spans
that TILE the feed contiguously from zero, so this asserts exactly that,
against the real scanner reading a real synthesized filesystem.
+185
View File
@@ -0,0 +1,185 @@
# `dirimage` — `dir://` as an image-level source
`dirimage` synthesizes a real, minimal, valid UDF 1.02 volume over a host
folder (a DVD `VIDEO_TS/` or Blu-ray `BDMV/` extracted by a MakeMKV-style
backup). Nothing is emulated:
* **Metadata sectors** (anchors, the volume descriptor sequences, the File
Set Descriptor, every File Entry, every directory's FID list) are encoded
into RAM by `encode` — a few MiB even for a large Blu-ray.
* **Data sectors** are not materialized at all. Each one maps to a byte
range of a real file, read on demand.
So `udf::read_filesystem` parses this image by exactly the same code path it
parses a real disc with, and every consumer above it is unchanged. The cost
is that a single-partition synthetic volume never exercises the UDF 2.50
Metadata Partition path (`udf.rs:946-991`) that every real BD-ROM uses —
this module's tests do not cover that block and must not be read as if they
did.
## What this module deliberately does NOT do
* **3D / SSIF** — rejected up front (`Error::DirImageSsifUnsupported`). An
SSIF aliases the same sectors as its base and dependent `.m2ts`; the
planner allocates disjoint extents, so a 3D folder would produce silently
wrong output.
* **HD-DVD `HVDVD_TS/`** — no title enumerator constraint is modelled.
* **Encrypted folders** — a folder whose content is still AACS-scrambled is
rejected by the caller-side probe, not decrypted here.
## `layout.rs` — rationale detail
Full detail for the comments trimmed in `src/dirimage/layout.rs`, in file order.
**Module overview.** Two phases, kept apart because they fail for different
reasons: walk rejects what the image model cannot represent (3D SSIF, an
unrecognized tree, a case collision inside one directory); assign places
metadata first (File Set Descriptor, one File Entry per node, then the
directory FID lists), then file data. Data placement is the part with real
constraints. For a Blu-ray there are none — `.mpls`/`.clpi` address clips by
name, never by LBA — so files are packed sequentially. For a DVD the IFOs
record VOB positions as sector offsets from the IFO's own start, and `ifo.rs`
re-derives every title extent from them, so the placement must reproduce
those offsets exactly or the rip reads the wrong sectors (`place_video_ts`).
**`DATA_FLOOR`.** Well clear of the volume-space descriptors, and — the
load-bearing part — far from zero: `disc/bluray.rs:137` drops any clip extent
whose LBA is 0, so a file that landed at block 0 would vanish from the title
with no error.
**`MAX_AD_BYTES`.** The AD length field is 30 bits (the top two are the
ECMA-167 4/14.14.1.1 extent type), so 0x3FFF_FFFF is the arithmetic ceiling —
but a non-final extent must be a whole number of blocks, so the usable
ceiling is the largest multiple of 2048 below it. Files larger than this are
split across several ADs; `udf.rs` reads them back as a multi-extent file,
the same shape a dual-layer disc produces.
**`MAX_DEPTH`.** Matches `udf.rs`'s `MAX_DIR_DEPTH`: anything deeper would be
recorded but never descended into, so the files under it would be invisible
to every consumer. Refuse instead of silently dropping.
**`MAX_CS0_NAME_BYTES`.** The FID's name-length field is one byte, so 255 is
the ceiling; 254 leaves the encoder no way to produce a value that wraps to
zero.
**`MAX_SUBDIRS`.** A directory File Entry's link count is `u16` and counts
one per child directory plus one for its own entry in the parent, so the
last usable value is `u16::MAX - 1`. Lowered under `cfg(test)` ONLY so the
guard can actually be executed: building a folder with 65534 subdirectories
to reach the real cap is not a test anyone can run, so an earlier test
asserted arithmetic about the constant instead and would have passed with
the guard deleted. The production value is unchanged.
**Module-scope link-count assert.** Must live at module scope, not inside
`mod tests`: it previously sat inside `#[cfg(test)] mod tests` while also
carrying its own `#[cfg(not(test))]`, so it was compiled in NO configuration
and could never fire — exactly the dead gate the test above it was written
to replace.
**`MAX_IMAGE_SECTORS`.** A DVD title set records where its VOBS begins as an
offset in its own IFO, read verbatim out of a file in the folder. A
regenerated `.BUP`, a tool that rewrote an IFO, or a hand-assembled folder
can therefore name an offset far beyond the content — and the planner
honours it, because honouring it is what makes a real backup readable.
Without a ceiling the image grows to wherever that offset points: a `u32`
sector count reaches ~8.8 TB, and writing one to an `iso://` destination
would fill a disk with zeros before anything noticed. 128 GiB clears the
largest real medium (BD-100) with room to spare, so a genuine disc folder
never meets it.
**`MAX_META_BYTES`.** Every node costs a 2 KiB File Entry sector held for the
life of the image, so the entry cap alone permits ~205 MB of metadata for
content of no size at all — and the mux holds two of these at once while
probing. The module documents a budget of "a few MiB even for a large
Blu-ray"; this is what enforces it rather than merely asserting it.
**`is_excluded`.** Dot-files are host artefacts, never disc content: macOS
sprays `.DS_Store` and `._*` resource forks through any folder a Finder
window has touched, and including them would put files on the "disc" that
were never on the disc. `.partial` is freemkv's own in-flight extraction
suffix (see `disc/extract.rs`) — picking one up would mean planning an
extent over a file that is still being written.
**`read_head`.** Reads the first `n` bytes of an IFO so its placement
offsets can be resolved. Errors propagate: returning an empty buffer instead
would send every offset through `unwrap_or(0)`, recording NO placement
constraint, and a VOB placed without its constraint yields an image that
reads at the wrong offset with nothing reported. An IFO that cannot be read
is a folder that cannot be planned.
**`place_video_ts`.** DVD-Video records VOB positions INSIDE the IFOs, as
sector offsets from the IFO file's own first sector: `VIDEO_TS.IFO` + 0xC0
(`VMGM_VOBS`) → `VIDEO_TS.VOB`; `VTS_nn_0.IFO` + 0xC0 (`vtsm_vobs`) →
`VTS_nn_0.VOB` (the VTS menu); `VTS_nn_0.IFO` + 0xC4 (`vtstt_vobs`) →
`VTS_nn_1.VOB` (the title stream). `ifo.rs:554-556` re-derives every title
extent as `file_start_lba(IFO) + vtstt_vobs + cell.first_sector`, so only the
third of those is load-bearing for freemkv itself — but the other two are
load-bearing for DVD PLAYERS, which read the menus freemkv ignores. All
three are honoured. `VTS_nn_1.VOB … VTS_nn_9.VOB` are one logical stream
split at the 1 GB file-size limit, and the cell sector addresses run
continuously across the split, so they are placed back-to-back with no gap;
laying the group out in its canonical on-disc order gives exactly that for
free, so the constraint becomes a CHECK, not a search. The check can fail —
a regenerated `.BUP`, a tool that rewrote an IFO, a folder assembled by hand
— and when it does the required position lies below the end of the file
that must precede it. There is no placement that satisfies it, so it is a
typed error naming the file rather than a silent misplacement.
**`reject_ssif`.** `disc/bluray.rs:124-130` probes
`/BDMV/STREAM/SSIF/{clip}.ssif` and sets `is_3d` the moment one resolves —
unconditionally, before any capability check. On a real 3D disc the `.ssif`
and the base/dependent `.m2ts` files ALIAS the same sectors (the SSIF
interleave IS the two m2ts streams), which this planner cannot express: it
would allocate three disjoint copies, so the clip's extents and the m2ts's
extents would disagree and the rip would read the wrong bytes at exit 0.
Reject the folder instead.
**`plan` — capacity, and what it changes.** `total_sectors` becomes the
`Disc`'s `capacity_sectors` / `capacity_bytes`, and `capacity_bytes` is not
inert: `canonical_title_order`'s oversize gate (`disc/mod.rs:2039-2040`, body
`:2209-2210`) demotes any title whose `size_bytes` exceeds it, which decides
which title sorts first and therefore what `-t 1` selects. The capacity
reported here is **the synthesized image's own size** — what an ISO built
from this folder would report — and nothing else. It is NOT padded up to a
media tier (BD25/BD50/…): that was considered and does not work, since a
folder does not record the tier of the disc it came from, so a 22 GB folder
off a BD50 would pad to BD25 and diverge anyway, and the padding would
inflate any pre-sized output written from the image. The consequence, stated
plainly: **for a folder that is missing files the source disc had — a
selective MakeMKV backup being the normal case — the capacity is smaller
than the source disc's, the oversize gate is therefore stricter, and a
borderline "play-all" composite title that the ISO kept can be demoted here.
`-t 1` can select a different title from `dir://FOLDER` than from an
`iso://` of the same disc.** For a complete folder the two agree, because
the capacities agree. This is chosen over the alternatives because it is the
only one that is self-consistent: `dir://X` and an `iso://` built from `X`
describe the same image and must answer the same way.
**Tests.**
* `a_file_past_the_ad_ceiling_splits_on_a_block_boundary` — a file above the
30-bit AD ceiling must split into MULTIPLE descriptors, and every
non-final one must be a whole number of blocks, or the next extent's bytes
start mid-sector and the file reassembles wrong. Kills a mutant that emits
one oversized AD (whose length would collide with the extent-TYPE bits
`udf.rs:642` reads).
* `two_names_the_reader_cannot_tell_apart_are_refused` — audit finding: the
uniqueness check compared raw host names, while `parse_udf_name` trims
whitespace and drops code units `char::from_u32` rejects. So
" 00000.m2ts" and "00000.m2ts" were two distinct entries at plan time and
ONE name at read time; `find`/`read_file` take the first match, so a title
resolved to the wrong file's extents and muxed the wrong bytes at exit 0.
Calls `plan` on a real folder and fails if the round-trip key is reverted
to comparing host names.
* `an_over_long_name_is_refused_by_the_planner` — audit finding: the length
was narrowed with `as u8`, so a 255-byte ASCII name (POSIX NAME_MAX, legal
on ext4/APFS/NTFS) encoded to 256 bytes with the CS0 compression byte and
wrote a length of ZERO, making every later entry in that directory read
from the wrong offset. An earlier version of this test asserted arithmetic
about the constants and never called `plan`, so it would have passed with
the guard deleted.
* `the_subdir_cap_refuses_a_folder_with_too_many_subdirectories` — pins the
arithmetic relationship the guard relies on and executes the guard rather
than restating the constant: link count (child dirs + 1) is 16 bits, so
exceeding it wraps and the image lies about its own directory structure.
Creating 65,535 directories in a test is not reasonable, so this does NOT
exercise `walk`.
+492
View File
@@ -0,0 +1,492 @@
# src/disc/mod.rs — internal notes
Overflow rationale for internal (non-`pub`) items in `src/disc/mod.rs`,
relocated here by the comment-guard so the in-file comment can stay short.
Each section is pointed to by a one-line `// See docs/disc-mod.md — <topic>`
comment at the corresponding call site.
## aacs_dir_present
THE ONE definition of "is this disc structurally AACS-encrypted". Both
`Disc::identify` (fast path, filesystem only) and `Disc::scan_with` (full
scan) need this, and both used to spell out the same two `find_dir` calls by
hand. They agreed today; nothing made them agree tomorrow — adding a third
AACS location, or excluding an empty placeholder directory, to one copy and
not the other would silently desync the fast identify from the full scan (the
same disc reported encrypted by one and clear by the other).
This is STRUCTURAL, not cryptographic: it says the tree looks like an
encrypted disc, not that any sector actually is. A folder copied verbatim
from a decrypted disc keeps its `AACS/` and answers true here (see
`session::scan_dir`, which corrects for exactly that).
## correct_truehd_channels
Corrects a title's TrueHD audio-stream metadata by probing the first
decrypted access units — channel count, real sample rate, and Atmos detection
in a single major-sync read. The MPLS descriptors declare the BASE layout
(often 5.1, or a container-guessed rate) even for a 7.1/Atmos TrueHD track;
the truth is in the MLP major sync. `reader` must yield DECRYPTED sectors
(the m2ts is AACS-encrypted, so this can only run at mux time, not scan).
Reads a bounded window of the title's first extent.
Corrections, each individually guarded so a malformed field never writes a
wrong header:
- **Channels**: from the presentation channel masks (as before).
- **Sample rate**: from the whitelisted rate nibble; left untouched on an
unknown rate or no major sync.
- **Atmos**: when a 4th substream is detected AND the stream still carries
the basic descriptor label, the label is promoted to the Atmos form;
richer editorial labels (e.g. an existing "Dolby Atmos") are left intact.
## Resolution::pixels
`Unknown` deliberately does NOT fabricate a plausible 1920x1080. This is the
same trap already closed on `AudioChannels::count` (which used to return 6)
and `SampleRate::hz` (48000.0): a plausible wrong answer is indistinguishable
from a real one at every call site, so a caller can forget to check the
variant first — the `json://` sink had already walked into exactly that,
reporting confident dimensions for a stream whose neighbouring `resolution`
field said "unknown".
It also does not return `(0, 0)`: that shape looks like a usable pair, so the
MP4 sink stored it and serialised a 0x0 video track — a structurally complete
file no player can render, written with no error. Before that it returned a
fabricated 1920x1080, which was wrong but at least playable, so the sink had
never needed a guard and the absence of one was invisible. A doc comment
listing which callers are safe cannot hold this: one did not check, and a
third kept its own duplicate `Unknown` test long after the accessor took the
job over. `Option` forces every caller to make its OWN decision: Matroska
omits the optional PixelWidth/PixelHeight elements, the metadata sinks report
the dimensions as absent, and MP4 must refuse per ISO/IEC 14496-12 (width and
height are mandatory in both `tkhd` 8.3.2 and VisualSampleEntry 12.1.3).
## has_probable_video
The exposed `streams` come from the title's FIRST PlayItem only, so a
feature whose first PlayItem is a non-video bumper reports
`has_video() == false` and would be wrongly disqualified (audit R6). This
predicate is deliberately MORE permissive than `has_video` — it only ever
ADMITS a title the stricter gate would reject — so it cannot reopen the
decoy hole: a size-inflated streamless decoy is demoted STRUCTURALLY by
composite detection in `Disc::rank_titles`, not by this gate, while it still
rejects tiny streamless menu/metadata playlists. The size fallback requires
at least one clip, so a clip-less pure-metadata decoy (which cannot be a
real bumper-lead feature — that has the feature clip) is still rejected.
## image_crack_extents
The extents an image-time CSS crack scans, in the crate's CANONICAL order:
the main feature's own extents, in natural playback order. Both halves are
deferred to logic the crate already owns rather than re-derived here:
- WHICH title — `scan_with` has already sorted `titles` with
`Self::canonical_title_order`, so the canonical main feature is the first
title that actually has extents. No local pick.
- WHAT ORDER — playback order, i.e. the extent vector untouched, exactly as
`Self::decrypt_keys_for_title` hands `&title.extents` to
`css::crack_key_outcome`.
**Why this function exists (do not let the copy grow back):** `Disc::scan_image`
used to re-implement both halves inline — it picked the title with the
largest total sector count, then re-sorted that title's extents
LARGEST-CELL-FIRST before handing them to `crack_key_outcome`. Both had
drifted from the canonical rules:
- Largest-cell-first is the 1.5.1 garbage bug. A CSS DVD's biggest cell opens
with a long CLEAR run, and the crack's 50_000-sector budget is shared
across the whole extent list, so starting there can exhaust the budget
without ever reaching a scrambled sector — the crack reports `Unencrypted`
and the mux emits scrambled MPEG as plaintext. Playback order reaches the
scrambled feature body after only the small clear front matter that
precedes it on a real disc (CSS: the title key is recovered from the
scrambled data itself, so the scan must actually MEET scrambled data).
- The sector-count pick ignores `canonical_title_order`'s capacity gate, so
it selects the oversize "play-all" composite (whose declared cells
double-count data shared with other playlists) instead of the real feature
— i.e. a DIFFERENT title, and on a multi-VTS disc a different VTS, whose
CSS title key does not descramble the feature at all.
The result: the same disc cracked as an ISO could disagree with the same
disc cracked from the drive, which is precisely what the duplicate was free
to do. Keep the derivation here, shared, so it cannot recur.
## read_aacs_inputs_from_reader
Reads a disc's AACS key-input files from a sector source: returns
`(Unit_Key_RO.inf, MKB)` raw bytes. Shared body for `Disc::read_aacs_inputs`
(ISO) and `Disc::read_aacs_inputs_from_drive` (live drive). Prefers MKB_RO,
falls back to MKB_RW, then TRIMS to the real record length. Both files are
allocated to a fixed ~128 MiB and zero-padded, so reading either ships up to
~124 MiB of nothing — trim to the record stream so callers send/store a few
MB, not 128 MiB.
## read_aacs_version
AACS major version (`crate::aacs::mkb::AACS_MAJOR_BD` /
`crate::aacs::mkb::AACS_MAJOR_UHD`) from the content certificate. Drives the
`Unit_Key_RO.inf` parse stride (48-byte V10 vs 64-byte V20/V21), so the
out-of-band key-fetch path parses `enc_title_keys` at the right stride (a
server VUK then derives the correct unit keys).
When no content certificate is readable/parseable, defaults to **UHD (V20,
64-byte stride)** — the conservative choice the pre-1.2.0 fetch path
hardcoded — and logs it: a wrong stride here folds a server VUK against
mis-strided title keys (silent wrong unit keys), so a missing cert must not
quietly pick the V10 stride for a UHD disc.
## read_mkb_content
Reads the AACS MKB's real record stream — NOT its zero padding. `MKB_RO.inf`
/ `MKB_RW.inf` are allocated to a fixed ~128 MiB and zero-padded; the actual
record stream is a few MiB. Reads a bounded prefix, finds the record-stream
length via `crate::aacs::mkb::mkb_content_len` and returns exactly that,
growing the prefix if the records run past it. This avoids reading 100+ MiB
of padding on every scan AND avoids the `read_file` `MAX_FILE_BYTES` cap that
(since 0.31.0) rejected the padded 128 MiB MKB outright — which made
`read_aacs_inputs` fail and autorip report "could not read this disc's key
files" without ever contacting the keyserver.
## CANONICAL_TITLE_ORDER_KEYS
**Why not just sort by duration descending?** Branching UHDs (and some BD
authoring) ship a "play-all" virtual playlist that references the same
source clips multiple times for seamless alternate-angle / alternate-ending
playback. Those playlists report an inflated `duration_secs` (often 4+
hours) and an inflated `size_bytes` greater than the disc's physical
capacity. Example seen in the wild — *The Amateur (2025)* UHD, 58.5 GB
BD-100 disc:
| Title | Playlist | Duration | Size | Clips |
|-------|------------|----------|---------|-------|
| 1 | 00020.mpls | 4h 13m | 92.4 GB | 253 |
| 2 | 00800.mpls | 2h 02m | 57.2 GB | 1 |
Title 1's 92.4 GB cannot fit on a 58.5 GB disc unless the same clip data is
referenced multiple times — proof it's a virtual composite. A duration-only
sort would put it at `titles[0]`, so `freemkv -t 1`, `disc.titles.first()`,
and autorip's main-feature picker all grab the 4-hour composite instead of
the 2-hour movie that actually matches TMDB.
**Effect on non-branching discs:** unchanged — the main movie is already the
longest 1-clip title. **Effect on branching UHDs:** the virtual play-all
playlist is pushed to the back, the actual movie surfaces at index 0.
Defined immediately beside the comparator, so a diagnostic can NAME the
ordering instead of restating it. The `freemkv::diag` main-feature decision
row used to carry its own hand-written copy of this list, and it drifted: it
still advertised a `fewest-clips` key long after the comparator replaced
clip-count with largest-physical-size, so the `--log-level 3` bug-report log
explained freemkv's top-level pick with a rule freemkv does not apply. Any
change to the keys must change this list in the same edit.
## probe_forced_subtitles_for_bdts_titles
Pulled out of `scan_with` as its own callable predicate: the decision is
otherwise reachable only by driving the full scan pipeline (tree dispatch ->
title parsing -> this loop), so a test could not pin the gate without also
authoring an entire synthetic disc image. Running it against an HD-DVD/DVD
(`MpegPs`) title's extents would demux the wrong container, since only
`BdTs` titles carry PES-wrapped PGS the shared classifier
(`pgs_forced_probe`) understands.
## MAIN_FEATURE_ORDER_KEYS
- `nav-feature`: the title the disc's own HDMV navigation plays as the
feature, resolved by running First-Play in the `crate::bdnav` VM the way a
real player would. This is the authoritative, size-independent pick; it is
video-gated (a mis-resolve cannot select a streamless playlist) and simply
absent when the VM abstains (BD-J discs, or non-convergence), in which
case the lower keys decide.
- `authoring-feature`: a title the disc's own menu authoring designates as
the feature (has video, is not itself a composite, and is within a sane
fraction of the longest title's duration) outranks everything else — a
size-independent signal an obfuscation decoy cannot forge.
- `standalone`: a title that is a play-all / wrapper COMPOSITE — its clip
set properly contains another substantial video title's whole clip set (a
real UHD title's decoy playlist `00245` = `[bumper][00001 feature
clip][outro]` case) — is demoted below the standalone title it wraps. This
is the STRUCTURAL anti-decoy signal; it does not depend on stream counts,
physical size, or disc capacity, so it survives where those are spoofed or
unknown.
- `has-video`: a title with no plausible video content (see
`DiscTitle::has_probable_video`) is demoted below every title that has
some — a floor that drops streamless menu/metadata playlists.
## rank_titles
This catches both the wrapper decoy (`00245 ⊋ 00001`) and the oversize
play-all — independent of capacity, which the `fits-disc` gate needs and
which is `0`/unknown on drives that fail READ CAPACITY.
### Wrapper-shape gate (issue #45)
A proper-subset relation with `Q.size ≥ 0.5·P.size` is NOT by itself proof
that `P` is a wrapper. A genuine **seamless-branch feature** `P` (body clip
plus intro/credits/alternate-branch segments) is a *superset* of its
single-clip body decoy `Q`: `Q` runs almost as long as `P` (e.g. 1h50 vs
2h03, ~0.89 of the duration) yet is markedly smaller, because `P`'s branch
segments add bytes without adding much played runtime. The old size-only
gate mis-flagged such a `P` as a "play-all wrapper" and demoted the real
2h03 feature below the shorter 1h50 body — a SAFETY bug (wrong title ripped).
So the composite flag additionally requires `P` to *have* a wrapper shape,
via one of two capacity-independent, per-title (`duration_secs`, `size_bytes`)
signals on the subset `Q`:
- **concat / play-all:** `Q.duration < 0.85·P.duration` — `Q` is a much
shorter cut and `P` pads its runtime with other segments (the true
play-all runs ~the SUM of its parts, so each part is well under half).
- **bumper / outro wrapper:** `Q.size ≥ 0.90·P.size` — `Q` already holds
almost all of `P`'s bytes and `P` merely prepends/appends a short
non-feature lead/tail (the `00245 ⊋ 00001` decoy: `Q` is ~0.96 of both the
duration and the size, so the size limb keeps it demoted).
Otherwise — `Q` runs almost as long as `P` (≥0.85) yet is well under 0.90 of
its size — `P` is the fuller branch set and is kept. `0.85` sits above a real
concat's per-part ratio (~0.5) and below a seamless body's (~0.89); `0.90`
sits above a bumper wrapper's near-1.0 size share and below a branch body's.
**Known residual:** a wrapper that pads runtime only modestly AND withholds
most of its bytes from the body clip is ambiguous by shape alone. The corpus
title-selection gate is the backstop across the disc hoard, and
higher-precedence nav/authoring signals override the composite key.
## aligned_unit_keys_validate
Conservative: a sample that is not AACS-scrambled proves nothing, and with
no scrambled sample at all there is nothing to disprove against, so it
returns `true` (accept). It returns `false` when ANY scrambled sample cannot
be restored to clear MPEG-TS by ANY unit key in the set. That covers two
distinct failure shapes: (1) a wholly wrong key (a keydb VK that does not
match this disc) — no sample decrypts; and (2) a *partially* applicable key
set on a multi-CPS-unit disc — the resolved keys cover CPS unit 0 but not
CPS unit 1. Accepting on the first sample that decrypts (the old behaviour)
would commit such a set, after which CPS-unit-1 sectors pass through as raw
encrypted bytes into the ISO/MKV with no error surfaced anywhere. Requiring
every scrambled sample to decrypt rejects the incomplete set so the caller
falls through to the next candidate. Reuses the ecosystem's single
`is_clean` content-clarity predicate and the full (bus + AACS) unit decrypt,
so it agrees with the actual mux decrypt.
## ensure_decryptable
The verdict, in order:
- `raw == true` → `Ok(())`. `--raw` intentionally skips decryption and needs
no key (the caller wants an encrypted image).
- `self.css_error.is_some()` → `Err(Error::CssNoDiscKey)`. The scan saw
scrambled CSS sectors but recovered no title key (`self.css` is `None` yet
the content IS encrypted). Treating `css.is_none()` as "unencrypted" would
mux scrambled MPEG as plaintext garbage. A DISC-LEVEL verdict
(`error::is_disc_level_no_key`) — the main feature's crack failed, so
every title fails the same way and the rip loop must stop rather than
skip each title in turn.
- AACS-encrypted (`self.aacs.is_some()`) with no usable key
(`decrypt_keys()` is `None`) → `Err(Error::NoDiscKey { .. })`, naming the
disc by hash.
- CSS-encrypted (`self.css.is_some()`) with no usable key →
`Err(Error::CssKeyMissing)`. (The disc-wide `decrypt_keys()` yields
`Css{..}` whenever `css.is_some()`, so this is defensive; the live
multi-VTS case is gated by `Self::ensure_decryptable_keys`.)
- otherwise → `Ok(())`. A genuinely unencrypted disc has `None` keys
legitimately, and a CSS disc whose keyless crack succeeded has a key.
## decrypt_keys_for_title
For a DVD the CSS title key MUST be recovered before descrambling: a
scrambled sector without a recoverable crib cannot self-crack, and CSS
leaves the pack/PES header clear, so a sector left un-descrambled would mux
as a structurally-valid but corrupt PES packet with no loss reported. Two
ways to get it:
- **Fast path** — the scan already cracked a key whose LBA span
(`crate::css::CssState::crack_span`) covers this title's VTS: reuse it. No
re-read, and on a live drive no second CSS bus-auth round-trip. CSS title
keys are per-VTS, so an overlapping span is the same key.
- **Crack** — when up-front detection missed (`self.css == None`) or the
title lives in a different VTS: crack the key from this title's OWN
extents in a SINGLE scan, in natural PLAYBACK ORDER (never
largest-cell-first, which starved the crack in a big cell's clear prefix —
the 1.5.1 bug). Playback order reaches the scrambled feature body after
only the small clear front matter (logo / rating card) that precedes it.
One scan = one CSS-locked early-bail, so a locked title is not
re-hammered per cell against a live drive (hard rule #2); its 50k-sector
budget is the same accepted bound the disc-wide scan uses.
Outcomes (`batch_sectors` sizes the crack's batched reads):
- `(Css{title_key}, false)` — descramble with the recovered/reused key.
- `(None, true)` — a genuinely-clear title needs no key; the gate passes it.
- `(None, false)` — scrambled but no key recoverable → hard failure via
`Self::ensure_title_decryptable`, never a silent garbage mux.
## inject_unit_keys
Injects pre-resolved AACS unit keys into a scanned disc — the deferred-mux /
resume path. The keys come from the mapfile's `# freemkv-uk:` header
(persisted at sweep time when the disc was keyed), so the mux decrypts
directly with NO key-service round-trip. Populates `self.aacs.unit_keys` so
`Self::decrypt_keys` returns them and marks the source `ExternalUk`.
If the scan built no AACS state (`self.aacs == None`) — which happens when
the keydb was absent at scan time (`scan_aacs_no_keydb` ->
`aacs_error = KeydbLoad`) — this synthesizes a minimal `ExternalUk` state
for an encrypted AACS disc. A Unit Key is the FINAL per-title decryption
key; the keydb is only needed to *derive* it, and that derivation already
happened at sweep (the UK is in the mapfile). So a UK alone is sufficient to
decrypt the on-disk ISO — AACS 2.0 bus decryption was applied by the drive
at read time, so `read_data_key` is unused for file-backed mux. Without
this, a keyed disc swept without a keydb would recover its UK yet still
report E8005 (no usable `decrypt_keys`) at remux. No-op for an unencrypted
or CSS (DVD) disc.
## scan_with_passes_the_halt_flag_to_the_bluray_enumerator
Every BD and DVD cancellation test in this crate calls
`scan_bluray_titles` / `scan_dvd_titles` DIRECTLY, so the three-line wiring
in `scan_with` that connects the operator's Stop to them was covered by
nothing at all: replacing `opts.halt.as_ref()` with `None` on either branch
left the whole suite green while a Stop during a BD or DVD scan silently did
nothing. Only the HD-DVD branch was pinned.
The fixture is deliberately the smallest disc that takes the BD branch — a
bare `/BDMV` with no PLAYLIST — because the point under test is the
ARGUMENT, not the enumerator's own (separately tested) halt polling. Nothing
between `scan_with`'s entry and the enumerator reads the flag (the disc is
unencrypted, so the AACS path is skipped), so a green here can only mean the
flag arrived. Mutation: `Self::scan_bluray_titles(reader, &udf_fs, None)?`
fails here.
## halted_reads_do_not_report_the_hddvd_scan_as_successful
A Stop on a LIVE DRIVE never touches `ScanOptions::halt`: `Drive` has its
own flag and `checked_exec` fails every SCSI command with `Error::Halted`
once it is set. The HD-DVD enumerator must not swallow that into a
successful scan. Measured before this was fixed: the scan returned `Ok`
with both titles present and ZERO streams on each — a cancelled scan
wearing the shape of a disc whose clips carry no video or audio. Downstream
that is a title list to cache, display and rip from.
## scan_with_capacity_bytes tests
`scan_with`'s `capacity_bytes = capacity as u64 * 2048` feeds
`canonical_title_order`'s "bigger than the whole disc = play-all composite"
threshold.
`..._not_addition`: chosen so `capacity * 2048` clears both titles (neither
is oversize, so the bigger one — OTHER — ranks first), but
`capacity + 2048` lands BETWEEN the two sizes: MAIN stays non-oversize while
OTHER flips to oversize and gets demoted, changing `titles[0]`. A `*` -> `+`
mutation is caught by this ranking flip, not merely a wrong numeric
threshold value.
`..._not_division`: same mechanism, tuned to catch a `*` -> `/` mutation
instead: `capacity * 2048` clears both titles, but `capacity / 2048`
collapses to a threshold far below MAIN's declared size while still leaving
MAIN under it, so only OTHER flips to oversize.
## read_mkb_content_grows_prefix_past_16mib_when_records_run_longer
This fixture's record stream is deliberately built so the first 16 records
land EXACTLY on the 16 MiB boundary (so the 16 MiB prefix alone parses as a
clean, complete record stream — the case a naive "n < buf.len() means done"
check would wrongly accept) and 4 more MiB of records follow, terminated by
the explicit `00 000000` end marker. Only a caller that actually grows past
16 MiB recovers the true 20 MiB content length.
## main_feature_order_demotes_wrapper_composite_decoy
The decoy `00245.mpls` is a WRAPPER COMPOSITE — `[00339 2s bumper][00001 the
whole feature clip][00336 outro]` — so its first-PlayItem STN reports v=0,
and it is the LONGEST and LARGEST title (8350s / 78.4 GB vs the feature's
8038s / 76.5 GB). Muxing its zero frames produced E6008 forever. It must be
demoted STRUCTURALLY (its clip set properly contains the feature's), not by
the fragile first-PlayItem stream count — this is what makes the fix robust
to the video-bearing-composite variant (see
`main_feature_order_demotes_video_composite`).
## key_source_failure_is_not_reported_as_a_missing_disc_key
Drives the real chain end to end — `KeySource` -> `resolve_and_apply_traced`
-> `Disc::aacs_error` -> `ensure_decryptable` — twice over the SAME disc,
and asserts the two operator-visible verdicts differ:
- source returns `Err(KeyServiceUnavailable)` (the HTTP-502 outage) → E7028
"the key service could not answer; retry later",
- source returns `Ok(vec![])` (the service answered, no entry) → E7022 "no
key source has a decryption key for this disc".
The credential and rate-limit verdicts are asserted alongside, since each is
a different operator action (fix the token / back off).
## css_disc_wide_no_key_is_disc_level_while_per_title_stays_skippable
The two CSS no-key conditions are NOT the same verdict and must classify
oppositely through the public predicates:
- **disc-wide** — `css_error` is set: the MAIN feature's crack failed, so
every title of this disc fails identically. Must be
`crate::error::is_disc_level_no_key` (the rip loop fail-fasts) and must
NOT be `crate::error::is_skippable_title_stub`. While both conditions
shared `E_CSS_KEY_MISSING`, an uncrackable CSS disc iterated all N titles
logging "title skipped" and exited 0 — a total failure reported as
success.
- **per-title** — one title's own re-crack failed on a multi-VTS disc
(`title_is_clear == false`, no key): skipping it and finishing the rest is
correct policy, so it must STAY skippable and must NOT be disc-level.
Pinned in both directions so a future change cannot silently flip either.
## correct_truehd_channels_full_correction_and_atmos_promotion
Kills the `was_basic ==`, the channels/rate `!=`/`&&` guards' "already
correct" branch, the `!matches!` per-stream skip, the `is_atmos ==
Some(true)` branch, and the whole-function no-op mutant.
## byte_offset_in_title_accumulates_across_extents
Kills: `lba >= start && lba < end` flipped to `||` (the first extent's
disjunction would trivially match almost any lba and return the wrong,
too-early offset); `cumulative +=` flipped to `*=` (cumulative is seeded at
0, so `*=` freezes it at 0 forever); and `sector_count * SECTOR_BYTES_U64`
flipped to `+` or `/` (wrong per-extent byte length folded into cumulative).
## chapter_at_offset_concrete_arithmetic
Kills: the `/` (time-fraction) flipped to `%` or `*`; the `*` (duration
scale) flipped to `+`; the `<=` chapter-scan comparison flipped to `>`; the
final `chapter_idx + 1` flipped to `-` or `*`; and every whole-function
fixed-tuple replacement (none produce `(2, 60.0)`).
## locate_ranges_positive_bps_duration_and_at_risk
Kills `bps > 0.0` flipped to `==`/`<` (both would take the `else 0.0`
branch here, wrongly reporting 0); `(*size as f64) / bps` flipped to `%` or
`*`; and the trailing `* MILLIS_PER_SEC` flipped to `+` or `/`.
## decrypt_keys_for_title_css_span_reuse_is_half_open
A CSS title key is per-VTS: reusing the scan's cracked key for a title that
lives OUTSIDE the cracked span descrambles that title with the wrong key,
and the mux emits garbage at exit 0. `crack_span` is documented as the
half-open LBA span `[start, end)`, so overlap is `extent.start < span.end &&
span.start < extent.end` — both comparisons STRICT. A title that merely
ABUTS the span (ends exactly where it begins, or begins exactly where it
ends) shares no sector with it and must NOT reuse the key.
The reader serves only clear (all-zero) sectors, so a title that falls
through to its own crack is reported unencrypted — distinguishable from the
reused-key answer both in the key and in the is-clear flag.
## merge_content_key_ranges
Merges per-title AACS key ranges into the sorted, disjoint set the
whole-disc map needs (`crate::decrypt::AacsKeyMap::entry_for` requires
disjoint ranges). Titles that share a clip resolve the SAME physical span
(same LBAs -> same CPS unit -> same key). When a later range overlaps a kept
one that carries the SAME key index and phase, the two are UNIONED (end
extended to the max) — this covers both the exact-duplicate (shared clip)
case and any partial overlap without ever dropping coverage, so no encrypted
LBA is left in no range (which would pass through as ciphertext). A real disc
never produces two DIFFERENT keys for one LBA; if that malformed case ever
appeared, the later range is dropped to keep the set disjoint rather than
extend one key over another key's LBAs.
+9
View File
@@ -0,0 +1,9 @@
# `disc::profile`
Everything in `DiscProfile` is DERIVED from the `Disc` model at construction:
the `Stream` enum is split into three typed vectors (audio/video/subtitle),
the `qualifier` / `purpose` enums are decomposed into booleans, and the
"default track" selection is precomputed so downstream never recomputes it.
Every field is always populated — a sensible default (`"und"` language, empty
`name`, `false` flag) stands in where the model has nothing, so consumers
never handle a bare `Option`.
+17
View File
@@ -0,0 +1,17 @@
# disc_tests: capacity/layer derivation
`scan_image_reports_capacity_in_bytes_and_the_layer_count` covers the scan
reporting the medium's size on both axes it exposes, derived from the one
sector count the caller hands in:
* `capacity_bytes` is that sector count times the 2048-byte logical sector
(ECMA-167 / BD-ROM logical block size). It is what sizes a full-disc image
read and what the progress percentage divides by, so a wrong scale is a
wrong ISO length, not a cosmetic number.
* `layers` distinguishes single- from dual-layer media. The threshold sits
between the two real capacities: a single-layer BD-25 is 12,219,392
sectors (25,025,314,816 bytes / 2048) and a dual-layer BD-50 is 24,438,784
sectors, so BD-25 must report 1 layer and BD-50 must report 2.
`scan_image` takes the sector count as a parameter, so this exercises the
real derivation without a 50 GB fixture.
+9 -9
View File
@@ -10,14 +10,14 @@ Insert disc
│
▼
1. Open drive (drive/mod.rs)
│ INQUIRY → identify drive
│ Match bundled profile → chipset, unlock parameters
│ INQUIRY → identify drive (DriveId)
│
▼
2. Init drive (drive/mod.rs → platform/mt1959)
│ Firmware upload (if needed, 10s recovery wait)
│ Unlock → vendor-specific command activates raw read mode
│ Speed calibration → probe_disc()
2. Init drive (drive/mod.rs → unlock seam)
│ Walk the registered-unlocker registry; first match unlocks the drive
│ (firmware/vendor handshakes are the unlocker's own business)
│ No match → drive untouched; host-cert AACS handshake carries the disc
│ Speed control → probe_disc()
│
▼
3. AACS handshake (aacs/handshake.rs) — optional
@@ -79,7 +79,7 @@ Insert disc
│ Or: read sectors → decrypt → raw bytes (for ISO output)
│ Drive::read() is single-shot. DiscStream::fill_extents adapts the
│ batch size on failure (halve / probe-up). Bad-range retry is layer
│ 1 above this — Disc::patch re-runs against the mapfile.
│ 1 above this — freemkv_engine::recovery::patch re-runs against the mapfile.
│
▼
PES frames → output stream (MKV, M2TS, network, etc.)
@@ -98,7 +98,7 @@ drive.probe_disc()?;
let disc = Disc::scan(&mut drive, &ScanOptions::default())?;
// Stream pipeline — PES frames from any source to any output.
// 0.18: input() returns Box<dyn FrameSource>, output() returns Box<dyn FrameSink>;
// input() returns Box<dyn FrameSource>, output() returns Box<dyn FrameSink>;
// direction is type-checked, so calling .write() on an input is a compile error.
let opts = InputOptions::default();
let mut input = libfreemkv::input("disc:///dev/sg4", &opts)?;
@@ -123,7 +123,7 @@ output.finish()?;
| aacs/ | [aacs.md](aacs.md) | Key resolution + content decrypt + bus handshake |
| css/ | -- | DVD CSS cipher |
| decrypt.rs | -- | Unified decrypt dispatcher (AACS/CSS/None) |
| disc/ | [rip-recovery.md](rip-recovery.md) | Disc::scan + Disc::sweep + Disc::patch + mapfile |
| disc/ | [rip-recovery.md](rip-recovery.md) | Disc::scan (sweep/patch/mapfile moved to freemkv-engine in 1.6.0) |
| labels/ | -- | BD-J stream labels (5 format parsers) |
| mux/ | -- | Stream implementations (7 stream types) |
| pes.rs | -- | PES frame types + FrameSource / FrameSink traits |
+90 -107
View File
@@ -7,8 +7,9 @@ optical drives.
## Drive
`Drive` is the primary API. It owns the SCSI transport, the matched
drive profile, and the chipset-specific platform driver.
`Drive` is the primary API. It owns the SCSI transport and the drive
identity (`DriveId`); any drive-specific unlock logic lives behind the
pluggable [unlock seam](#drive-unlock-seam), not in `Drive` itself.
### Opening a Drive
@@ -16,33 +17,37 @@ drive profile, and the chipset-specific platform driver.
let mut drive = Drive::open(Path::new("/dev/sg4"))?;
```
`open()` performs: open device → send INQUIRY → match profile → instantiate
platform driver. The drive is ready for `wait_ready()` and `init()`.
`open()` performs: open device → send INQUIRY → build `DriveId`. The drive
is ready for `wait_ready()` and `init()` (which routes through the unlock
seam).
### Drive Operations
| Method | Description |
|--------|-------------|
| `wait_ready()` | Wait for disc insertion (30s timeout, TUR polling) |
| `init()` | Firmware upload + unlock + speed calibration |
| `init()` | Route to the matching registered unlocker (if any), then prepare for reads |
| `probe_disc()` | Probe disc surface for optimal speeds |
| `read(lba, count, buf, recovery)` | Read sectors. Single-shot — no inline retries or reset. |
| `reset()` | Eject-cycle escape hatch. Caller-invoked only; not on the read path. |
| `lock_tray()` | Prevent tray ejection during rip |
| `unlock_tray()` | Allow tray ejection (also runs on Drop) |
| `eject()` | Eject disc tray |
| `drive_status()` | Query physical state (disc present, tray open, etc.) |
| `has_profile()` | Whether a bundled profile matched |
| `has_profile()` | Whether a registered unlocker matches this drive |
| `close()` | Consume Drive, cleanup (also runs via Drop) |
### init() Sequence
`init()` orchestrates the full drive unlock:
`init()` routes drive preparation through the unlock seam:
1. Platform driver `run_init()` — sends vendor-specific SCSI commands
2. If firmware upload needed: upload, wait 10s for drive reset, retry
3. Speed calibration after unlock
4. Max 3 attempts before giving up
1. Walk the registered-unlocker registry; the first whose `matches()` is true
is asked to `unlock_drive()` over the raw transport.
2. Whatever that unlocker needs (firmware upload, vendor handshakes, retries)
is the unlocker's own business — libfreemkv only forwards the transport.
3. If no unlocker matches, the drive is left untouched and the library uses
the host-certificate AACS handshake.
See [Drive Unlock Seam](#drive-unlock-seam) for the trait and registry.
### read() — single-shot
@@ -52,15 +57,16 @@ selects the per-CDB timeout:
| `recovery` | Timeout | Used by |
|------------|----------|------------------------------------------|
| `false` | 1.5 s | `Disc::sweep` fast skip-forward pass, `DiscStream::fill_extents` |
| `true` | 30 s | `Disc::patch` retry pass over the mapfile |
| `false` | 10 s | `freemkv_engine::recovery::sweep` fast skip-forward pass, `DiscStream::fill_extents` |
| `true` | 60 s | `freemkv_engine::recovery::patch` retry pass over the mapfile |
On any SCSI failure or timeout, `read` returns `Err(DiscRead)` immediately.
There are no inline retries, no SCSI reset, no Phase 1/2/3 escalation.
Recovery is layered above `Drive::read`:
- **Layer 1 — `Disc::patch`** loops over the ddrescue mapfile and re-issues
- **Layer 1 — `freemkv_engine::recovery::patch`** (in the engine crate, not
here) loops over the ddrescue mapfile and re-issues
`read(.., recovery=true)` against each non-`+` range.
- **Layer 3 — `DiscStream::fill_extents`** halves the request size on
failure, retries at the same LBA, and probes back up on a clean-read
@@ -97,14 +103,14 @@ descriptors or calls ioctls outside of a `ScsiTransport` implementation.
| Platform | Implementation | Device |
|----------|---------------|--------|
| Linux | `SgIoTransport` — async `write`/`poll`/`read` on `/dev/sg*` | `/dev/sg*` |
| Linux | `SgIoTransport` — synchronous `ioctl(fd, SG_IO, &hdr)` on `/dev/sg*` | `/dev/sg*` |
| macOS | `MacScsiTransport` — IOKit SCSITask | IOKit service |
The Linux backend uses the sg driver's asynchronous interface: `write()` submits
the command, `poll()` waits with an enforceable wall-clock timeout, `read()`
retrieves the result. If `poll()` times out, the fd is abandoned (closed in a
background thread) and a fresh fd opened — the kernel's USB error recovery
cannot block us. Opens with `O_RDWR | O_NONBLOCK`.
The Linux backend issues each command as one synchronous `ioctl(fd, SG_IO, &hdr)`
(`linux.rs`); the timeout is the kernel's, set via `hdr.timeout`, not a userspace
`poll()`. Opens with `O_RDWR | O_NONBLOCK`. (Pre-0.13.20 this backend ran an async
`write()` + `poll(1.5s)` + close-on-timeout + background-reopen pattern; it was
reverted in 0.13.20 — see the note at the top of `linux.rs`.)
The macOS backend uses a C shim (`macos_shim.c`) for IOKit exclusive access.
The shim handles:
@@ -120,14 +126,13 @@ The shim handles:
On non-zero SCSI status, the transport parses sense key from the sense buffer
and returns `Error::ScsiError`.
`SgIoTransport::reset` (Linux) does pure userspace state cleanup: an open +
close pair to make the kernel cancel any SG_IO commands queued against a
previous fd, a 2 s sleep to let the kernel finish that cancellation, then a
fresh fd to send ALLOW MEDIUM REMOVAL to clear any stale tray lock. It does
NOT issue `SG_SCSI_RESET` or escalate via STOP+START UNIT. Both were tried
in 0.13.0–0.13.5 against the LG BU40N (Initio USB-SATA bridge); both failed
to recover wedged drives and made the wedge worse. The macOS reset (which
had been a no-op) was removed entirely in 0.13.6, and the top-level
The Linux backend has no `reset`. `SG_SCSI_RESET` and STOP+START UNIT
escalation were tried in 0.13.0–0.13.5 against the LG BU40N (Initio USB-SATA
bridge); both failed to recover wedged drives and made the wedge worse, so no
Linux transport-reset path exists today — recovery is just a fresh fd opened on
the next command. The only `reset` on a transport in the crate is
`SptiTransport::reset` on Windows (`windows.rs`). The macOS reset (which had
been a no-op) was removed entirely in 0.13.6, and the top-level
`scsi::reset()` / `reset_with_timeout()` / `reset_blocking()` wrappers were
removed at the same time (no callers).
@@ -164,100 +169,68 @@ date for drives where Feature 010C is unavailable.
---
## Drive Profiles
## Drive Unlock Seam
Profiles are JSON objects compiled into the binary (`profiles.json`).
Each profile contains:
| Field | Purpose |
|-------|---------|
| `vendor_id`, `product_revision`, `vendor_specific`, `firmware_date` | Matching fields |
| `chipset` | `"mediatek"` or `"renesas"` |
| `unlock_mode`, `unlock_buf_id` | READ BUFFER CDB parameters |
| `signature` | Expected 4-byte response signature |
| `unlock_cdb` | Pre-built unlock CDB (hex-encoded) |
| `register_offsets` | Offsets for hardware register reads |
| `capabilities` | Feature flags: `bd_raw_read`, `dvd_all_regions`, etc. |
Loading:
libfreemkv ships **no firmware, no unlock CDBs, and no drive profiles.** It
knows only the *seam*, never the *mechanism*. The seam is the `Unlocker`
trait plus a small process-wide registry (`src/unlock.rs`):
```rust
// Bundled (compiled-in) -- no file I/O
let profiles = profile::load_bundled()?;
pub trait Unlocker: Send + Sync {
/// Stable, language-neutral identifier (logged).
fn name(&self) -> &str;
// External file
let profiles = profile::load_all(Path::new("/path/to/profiles.json"))?;
/// True if this unlocker handles the given drive.
fn matches(&self, id: &DriveId) -> bool;
/// Put the drive into extended-access mode. The one required capability.
fn unlock_drive(&self, scsi: &mut dyn ScsiTransport, id: &DriveId) -> Result<()>;
/// Read the disc Volume ID via the drive's OEM path. Default: no-op.
fn read_volume_id(&self, _scsi: &mut dyn ScsiTransport, _id: &DriveId)
-> Result<Option<[u8; 16]>> { Ok(None) }
/// Raise the drive to its maximum read speed. Default: no-op.
fn set_max_read_speed(&self, _scsi: &mut dyn ScsiTransport, _id: &DriveId)
-> Result<()> { Ok(()) }
}
```
---
An unlocker is supplied by an **external crate** and registered once at
process start:
## Chipsets
```rust
libfreemkv::register_unlocker(Box::new(some_unlocker::Plugin::new()));
```
### MediaTek MT1959
The implementor owns everything about *how* a particular drive family is
driven — drive identification against its own profile database, firmware
upload, vendor CDBs, variant logic. libfreemkv only hands over the raw
`ScsiTransport` and the `DriveId`.
Covers all LG, ASUS, and HP optical drives. Two sub-variants share identical
logic with different SCSI parameters:
### Routing
| Variant | READ BUFFER mode | Buffer ID |
|---------|------------------|-----------|
| MT1959-A | 0x01 | 0x44 |
| MT1959-B | 0x02 | 0x77 |
At drive-prep the registry is walked in registration order; the first
unlocker whose `matches()` returns true is asked to `unlock_drive()` (and,
when needed, `read_volume_id()` / `set_max_read_speed()`). If no unlocker
matches, the drive is left untouched and the library falls back to the
standard host-certificate AACS handshake (the "OEM route"). The
`register_unlocker(...)` line is the entire plug: drop it (and the unlocker
crate) and libfreemkv still compiles and rips via the cert handshake.
The Platform trait maps to command handlers:
| Handler | Function | Description |
|---------|----------|-------------|
| 0 | `unlock()` | Send READ BUFFER, verify signature + verification bytes |
| 1 | `read_config()` | Read 1888-byte configuration block + 4-byte status |
| 2-3 | `read_register()` | Read hardware registers at profile-specified offsets |
| 4 | `calibrate()` | Probe disc surface, build 64-entry speed table |
| 5 | `keepalive()` | Periodic session maintenance |
| 6 | `status()` | Query current mode and feature flags |
| 7 | `probe()` | Generic READ BUFFER with dynamic parameters |
| 8 | `read_sectors()` | Speed lookup + SET CD SPEED + READ(10) with flag 0x08 |
| 9 | `timing()` | Timing calibration |
### Renesas (Planned)
RS8xxx/RS9xxx chipsets used in Pioneer and some HL-DT-ST drives.
Currently returns `Error::UnsupportedDrive` when a Renesas profile is matched.
---
## Why Unlock Is Needed
Optical drive firmware restricts what applications can read from disc. Without
unlock:
- **READ(10) works for unencrypted filesystem data.** UDF structures, MPLS
playlists, and CLPI clip info are readable without unlock. Standard READ(10)
works on any drive.
- **READ(10) fails for encrypted content sectors.** The drive firmware returns
SCSI errors (sense key 0x05, illegal request) when an application attempts to
read sectors containing encrypted m2ts content without prior AACS
authentication via the bus key.
- **Raw mode bypasses firmware restrictions.** After unlock, the drive accepts
READ(10) with the raw read flag (CDB byte 1 = 0x08) for all sectors,
regardless of encryption status.
### AACS Before Unlock
AACS bus authentication uses standard MMC REPORT KEY / SEND KEY commands.
On some drives these must execute before unlock. The `Disc::scan()` handles
this internally — it manages the handshake/unlock ordering automatically.
Concrete unlockers — including the firmware-unlock profile databases,
variant logic, and vendor CDBs that used to live in-tree — are maintained
in the separate **[freemkv-unlock](https://github.com/freemkv/freemkv-unlock)**
repository, never here.
---
## Speed Control
After `probe_disc()`, the platform driver maintains a speed lookup table
built by probing the disc surface. On each `read()` call, the driver:
1. Looks up the optimal speed for the target LBA.
2. Issues SET CD SPEED (0xBB) if the speed differs from current.
3. Performs the READ(10).
A matching unlocker may raise the drive to its maximum read speed via
`set_max_read_speed()` (a no-op when no unlocker matches or the unlocker
declines). The library issues SET CD SPEED (0xBB) through the generic CDB
builder; the concrete speed policy lives in the unlocker.
Available speeds:
@@ -266,3 +239,13 @@ Available speeds:
| Blu-ray | 1x (4,500 KB/s) through 12x (54,000 KB/s) |
| DVD | 1x (1,385 KB/s) through 16x (22,160 KB/s) |
| Max | 0xFFFF (drive decides) |
---
## Windows Path Normalization
`normalize_path` in `src/drive/windows.rs` converts a device path ("D:",
"D:\\", "\\.\D:", "\\.\CdRom0") to Windows `\\.\X:` form. A near-identical
`normalize_device_path` exists in `scsi::windows`; both are kept because
they live in separate `cfg(windows)` modules that cannot easily share a
helper without introducing cross-module coupling.
+15
View File
@@ -0,0 +1,15 @@
# Linux `sg*` device enumeration
`enumerate_sg_names` (in `src/drive/linux.rs`) lists `/dev/sgN` SCSI-generic
device names to probe for optical drives.
Linux assigns `/dev/sgN` sequentially across *all* SCSI-generic devices
(disks, tape, HBAs, optical), not just optical drives. That means a fixed
`sg0..15` range can miss an optical drive on a host with many other SCSI
targets ahead of it in enumeration order.
To avoid that, the function prefers the exact present-device list from
`/sys/class/scsi_generic/`, which enumerates only the `sg*` nodes that
actually exist. It falls back to a bounded `sg0..15` probe only when sysfs
itself is unreadable, which happens in some minimal containers that lack a
mounted `/sys`.
+142
View File
@@ -0,0 +1,142 @@
# src/drive/mod.rs — internal notes
Long-form rationale relocated from `src/drive/mod.rs` comments that exceeded
the comment-guard's internal-comment cap. Each section is pointed to by a
short `// See docs/drive-mod.md — <topic>` comment at its original site.
## Error-recovery mode page (PER bit)
`MODE_PAGE_ERROR_RECOVERY` (0x01) is the SBC/MMC Read-Write Error Recovery
mode page. We flip the `PER` bit to make the drive REPORT a recovered read
(via CHECK CONDITION + sense key RECOVERED ERROR) instead of silently
returning best-effort data as GOOD status. On marginal/dirty media that
silent-GOOD data can be mis-corrected — a rip that "passed clean" but decoded
with errors. With PER on, freemkv sees the marginal read and re-reads it in
Pass N (a loud miss, never a silent commit). See
`build_error_recovery_select_payload`.
## `sleep_until_halted` — why it exists and why it was dormant
This was `#[cfg(test)]` for two releases, kept alive only by its unit tests,
while the two production paths that actually sleep — the `wait_ready` poll
backoff (60 x 500 ms) and `spin_cycle`'s spin-down/settle pauses
(`SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`) — used a plain
`std::thread::sleep` and so were deaf to the operator's Stop for ~30 s and
~15 s respectively. Every other drive path returns `Halted` at the next
`checked_exec` boundary; a Stop pressed during spin-up simply did nothing
visible until the poll ran out. The primitive existed; the sleeping code just
did not call it. It is production code again.
## `Drive::read` — single-shot contract and the removed inline retries
`recovery=true` uses `crate::scsi::READ_RECOVERY_TIMEOUT_MS` (60 s, matches
sg_dd) for the `freemkv_engine::recovery::patch` pass; `recovery=false` uses
`crate::scsi::READ_TIMEOUT_MS` (10 s) for `freemkv_engine::recovery::copy`'s
fast skip-forward sweep. Both budgets are generous enough that the drive can
finish ECC recovery on a marginal sector — pre-0.13.21 this was 1.5 s on the
fast path which forced the kernel mid-layer to time out and escalate while we
waited anyway. On any failure this returns `Err(DiscRead)` immediately;
orchestration (`freemkv_engine::recovery::patch` multi-pass, `DiscStream`
adaptive batch halving) handles retry policy.
Inline retry phases (5x gentle + reset+reopen + 5x more) were removed in
0.13.6: on some USB-SATA bridges the inline reset wedged drive firmware
without ever recovering a sector. The remaining recovery layers
(`freemkv_engine::recovery::patch` multi-pass, `DiscStream` batch halving) do
not touch the wedge-prone reset path.
## `find_drive` — media-preference selection policy
On a multi-drive system (common on Windows, where an empty/not-ready drive
can enumerate first) returning the first drive blindly can pick a drive with
no disc, dooming the operation. So `find_drive` opens each candidate in
enumeration order, queries `Drive::drive_status` (GET EVENT STATUS, which
works regardless of firmware state), and returns the first drive reporting
`DriveStatus::DiscPresent`. If no drive reports a disc — or `drive_status()`
is unavailable/returns `Unknown` everywhere (single-drive or quirky bridges)
— it falls back to the first drive that opened, preserving historical
behavior so those setups don't regress. For just listing drives without
opening (e.g. UI sidebar), use `scsi::list_drives()` instead — it returns
`DriveInfo` (path + identity) without the cost of running every drive's
profile + identity probe.
## `open_block_device_for_sg` — why no `O_DIRECT`
Resolves a `/dev/sg*` path to the corresponding `/dev/sr*` block device by
walking sysfs, then opens it for read. No `O_DIRECT`:
`posix_fadvise(POSIX_FADV_DONTNEED)` flushes the cache before each pread,
which avoids buffer-alignment requirements while still forcing fresh device
reads. Returns `None` on any error (sysfs not present, no matching block
device, open failed); callers treat that as "no fallback available" and
propagate the original SCSI READ error.
## `resolve_device` — staged, not yet wired
The cross-platform dispatch is kept ready for the caller that will consume
it, so the per-platform implementations below it (and their tests) stay
live. `allow(dead_code)` marks that deliberately — this is not an accidental
orphan.
## Test rationale: recovery/halt/chunking coverage
The `halt_tests` and `command_tests` modules exercise mutation-sensitive
edges (halt-aware sleep timing, chunked-read offset arithmetic, drive-status
byte decoding, spin-cycle/wait-ready halt responsiveness). The following
notes preserve the "why this exact scenario" reasoning that the inline test
doc comments were trimmed from:
- **`disc_is_dvd_matches_only_dvd_profile_family`**: must match the DVD
profile family (0x0010..=0x001F) and ONLY that family. A false positive on
a BD/UHD profile (0x0040+) would skip the drive unlock UHD reads require; a
false negative on a DVD would re-introduce the CSS read failure. Widening
the range to `..=0x0040` makes the BD-ROM assert fire; a failed/short GET
CONFIGURATION must default to NOT-DVD so the unlock still runs.
- **`media_event_reply` / media-status decoding**: MMC-6 §6.7 — byte 5 of the
GET EVENT STATUS reply is a Media Status only when the Event Header says a
media event descriptor follows (NEA clear AND Notification Class == 4).
Decoding byte 5 regardless of those preconditions reads a reserved/zero
byte as Media Status 0 and reports NoDisc on a drive that has a disc loaded
— the classic "works on my drive, not theirs" firmware split. The drive is
untrusted input, so an event-less reply must fall back to the TEST UNIT
READY status instead.
- **Chunked-read tests** (`read_chunks_write_into_correctly_offset_buffer_regions`,
`an_undersized_buffer_errors_on_the_chunked_path_just_like_the_single_one`,
`chunk_lba_near_u32_max_errors_not_overflows`): the multi-chunk path slices
the caller's buffer by `count * 2048` and advances the per-chunk LBA with
`lba + done`, both unchecked. An undersized buffer PANICKED
("range end index out of range") out of the public `read`/`read_fua`
instead of returning `Err(DiscRead)` like the single-chunk path; an LBA
range crossing `u32::MAX` overflowed (debug panic, or release wrap to a low
LBA silently read). These tests pin both to error-not-panic/overflow, and
one writes a distinct marker per chunk to confirm the byte-offset
arithmetic (`done * 2048`, not `done + 2048`) is correct — no other mock
touches the buffer contents, so a `*` -> `+`/`/` mutation would otherwise go
unasserted.
- **`wait_ready`/`spin_cycle` halt tests**: `wait_ready` was the one drive
path that called `self.scsi.as_mut().execute(..)` instead of
`checked_exec`, and its 60 x 500 ms loop never read `self.halt` — a cancel
during spin-up was ignored for ~30 s. `spin_cycle` had the same gap across
two bare `execute` calls and two plain `std::thread::sleep` pauses
(`SPIN_DOWN_IDLE_SECS` + `SPIN_UP_SETTLE_SECS`), and it runs from the
recovery path — precisely when an operator is most likely to press Stop.
The tests pin: a Stop set before the call returns `Halted` immediately with
no command issued; a Stop set mid-poll (on a deterministic Nth call, no
wall-clock race) exits at the next `checked_exec` boundary; a Stop that
lands during the spin-down pause wakes the halt-aware sleep within
~100 ms instead of blocking for the full 5 s.
- **`read_logs_a_good_status_short_transfer`**: a READ(10) that completes
with GOOD status but a residual underrun is correctly refused, but the
refusal used to log nothing — so a drive that residual-underruns on GOOD
status produced the same silent journal as a scratched disc, and the
operator was sent after the wrong remedy (replace the drive vs. clean the
disc). The test pins that the refusal now warns with lba/transferred/
expected/code.
## `open_block_device_for_sg` doc contract (kept short in code)
See the section above for the `O_DIRECT` rationale; the pointer comment near
the function keeps only the API contract.
+69
View File
@@ -0,0 +1,69 @@
# `dropgate` — audio drop bookkeeping rationale
## Keep/drop rule
A clean mux keeps every frame it can and drops the ones it can't — video
always survives (it's inter-frame predicted; a per-frame drop would cascade,
so video resyncs/conceals instead), audio keeps every decodable access unit,
and a damaged audio AU is dropped rather than shipped as a decoder-choking
glitch.
## Detection is per-codec
The DETECTION is inherently per-codec — each format carries its own
authoritative corruption check (DTS: the core sync/header parse per ETSI TS
102 114; AC-3: the header CRC per ETSI TS 102 366; FLAC: the frame CRC-16;
…). `DropTally` only carries the UNIFORM response so every audio parser
behaves identically:
1. **Count** kept vs dropped AUs and the dropped duration.
2. **Log** every drop (fail-loud, never silent) — a per-drop trace plus a
once-per-track aggregate at `warn` so it surfaces without debug logging.
3. **Whole-track fallback**: once a track is judged mostly undecodable, latch
a poison flag so the remainder is dropped too (a track that damaged isn't
worth muxing).
## Sync preservation
**Sync preservation is the caller's responsibility**, not this type's: the
parser must advance its PTS clock across a dropped AU exactly as it would for
an emitted one, so a drop becomes a silence gap and never a shift of the
following audio. See `DtsParser`'s `stamp_pts` call ordering for the pattern.
## `verified_dropped` vs `dropped`
`verified_dropped` counts AUs dropped because they were INDIVIDUALLY verified
undecodable (a failed CRC/header/parity check); only these feed the
whole-track poison verdict. `dropped` also counts *collateral* drops — AUs
discarded as a consequence of one corruption (TrueHD's resync-forward run, or
a poisoned track) — which must NOT amplify a few real errors into a false
whole-track loss.
## `record_collateral_drop`
A collateral drop is caused by another AU's corruption (TrueHD's
resync-forward run to the next major sync, or an already-poisoned track), not
by being individually verified undecodable. It is counted and logged for the
drop report, but deliberately does NOT feed the poison verdict, so one
corruption event can't amplify into a false whole-track loss.
## `maybe_poison`
Whole-track fallback: after enough AUs to judge, if more than half were
dropped the track is too damaged to be worth muxing — latch `poisoned` and
log it loudly once. The minimum-sample gate keeps a short damaged burst from
poisoning an otherwise-good track. Judge on VERIFIED drops vs all AUs seen: a
track is only poisoned when a majority of its access units are individually
undecodable — not when a couple of corruption events forced long collateral
resync runs.
## Test: `interleaved_keeps_are_in_the_poison_denominator`
The poison verdict is a RATIO — verified drops against every AU seen — so
the kept count is half of it. `does_not_poison_a_mostly_good_track` records
its keeps AFTER the single drop, and `maybe_poison` only runs inside
`record_drop`, so the keeps are never in the denominator when the verdict is
actually computed: that test passes even with the kept count never
incremented. Interleaving them puts the kept count on the critical path,
where losing it turns the ratio into "verified drops vs verified drops" —
always >50% — and silently discards a healthy track.
+309
View File
@@ -0,0 +1,309 @@
# DTS / DTS-HD elementary stream parser (`src/mux/codec/dts.rs`)
Internal design notes moved out of source comments (see `ci/comment-guard.py`
prose caps). Each section is pointed to by a short `//` comment at the
corresponding site in `dts.rs`.
## `DTS_HD_EXT_SYNC`
An access unit is delimited by the next CORE sync; the parser locates and
exactly sizes each extension substream (via `exss_frame_size`) so a false
core sync inside the EXSS payload can't split the AU and truncate the
lossless extension.
## `emit_or_drop`
Gates an assembled access unit through the decodability check and either
pushes it or drops it. `au_pts`/`dur_ns` are already stamped on the shared
PTS clock (which the caller advances whether or not the AU survives), so a
drop leaves the following audio on its true timeline — a gap, not a shift.
Every drop is logged (fail-loud, never silent).
## `stamp_pts`
`front` is the AU's own core-PES PTS (from `front_pts`); `dur_ns` its
decoded duration.
The model matches the (correct) AC-3 path: **re-base to each PES's own
container timestamp, and advance by one frame duration ONLY within a run of
AUs that share the same PES.** A new PES (its `front` differs from the
previous AU's) trusts its own timestamp — so the emitted timeline tracks the
container and never drifts. Advancing within a PES is what fixes the DVD
case (several DTS core frames packed in one PES, which otherwise all
collided on that single PES timestamp → "non monotonically increasing dts").
A global running clock was WRONG here: once accumulated frame durations
exceeded the PES spacing it never re-based, drifting the track minutes past
the real length over a feature-long title.
## `CORE_HEADER_MIN_BYTES`
Number of leading bytes that must be buffered before the core `fsize` field
(bytes 5-7) can be decoded. This is a HEADER-LAYOUT minimum — "enough bytes
to read the size field" — and is deliberately distinct from
`MIN_CORE_FRAME_BYTES` (the decoded-frame-size validity floor). They must
not be conflated: this one gates buffer reads of the header, the other
rejects implausible decoded sizes.
## `MIN_CORE_FRAME_BYTES`
Minimum plausible decoded DTS core frame size, per ETSI TS 102 114: the
on-wire FSIZE floor is 95, so a real core frame is at least 96 bytes. A
decoded `core_size` below this means we matched a false/corrupt core sync
(a lucky 0x7FFE8001 in extension-substream payload whose 14-bit `fsize`
decoded to a tiny value) rather than a real frame, so we resync instead of
closing an access unit at a junk boundary and dropping the DTS-HD extension
tail.
## `PTS_UNSET`
Sentinel for "no valid PTS base captured yet". Real PTS-in-ns values are
non-negative (derived from the unsigned 90 kHz PES timestamp), so a
negative value can never collide with a genuine timestamp. Used to mark the
PTS base invalid after a forced flush so the next PES sets it regardless of
buffer state.
## `flush_tail`
Emits the final buffered access unit (the last core + its extension
substreams, which had no following core sync to close it during
streaming), gated through the decodability check. Requires a complete core
frame; drops a bare partial sync tail.
## `SYNCWORD_BYTES`
Byte length shared by the DTS core sync and the DTS-HD EXSS sync (both
32-bit words). `next_core_boundary` finds the next *valid* core sync after
the current core frame, to delimit the access unit: extension-substream
payload can contain byte sequences that match the core syncword, so each
candidate is validated by decoding its core size — a match whose decoded
size is implausible (`< MIN_CORE_FRAME_BYTES` or `> MAX_AU_BYTES`) is a
false sync and is skipped, continuing the search.
## EXSS header field bit widths
`EXSS_USER_DEFINED_BITS`, `EXSS_INDEX_BITS`, `EXSS_HEADER_SIZE_TYPE_BITS`,
etc. are the DTS-HD extension-substream (EXSS) header field bit widths
(ETSI TS 102 114, ExtSS header). `bHeaderSizeType` selects the short form
(`nuExtSSHeaderSize` 8 bits, `nuExtSSFsize` 16 bits) or, for larger
substreams, the long form (12 / 20 bits).
## `exss_frame_size`
DTS-HD extension substream (EXSS) total byte size — INCLUDING the
`0x64582025` syncword — read precisely from its header. `buf` must begin
with `DTS_HD_EXT_SYNC`. Returns `None` when the size fields aren't fully
buffered.
`nuExtSSFsize` is the total frame size in bytes minus one. Parsing it lets
the AU framer skip the extension by its exact length instead of scanning
its (arbitrary) payload for a core sync.
## `next_core_boundary`
Offset where the current access unit ends (the start of the next core
frame). The AU is the core frame plus its trailing DTS-HD extension
substreams, which are skipped PRECISELY by their declared size — so a
chance core syncword inside the XLL lossless payload can never be mistaken
for the next AU boundary (the bug that truncated the extension and produced
the "Failed to decode block code(s)" class). Falls back to the heuristic
core-sync scan only when an extension can't be sized (malformed / truncated
input).
## `scan_for_next_core`
Heuristic fallback (the pre-fix behaviour): scan forward for the next core
syncword whose decoded size is plausible. Used only when precise extension
skipping can't proceed; a chance core syncword in extension payload usually
decodes to an implausible size and is skipped.
## `dts_core_frame_size`
`fsize` is the 14-bit field at bits 46-59 of the header (bytes 5-7). On the
wire `fsize` is the frame length minus one, so this returns `fsize + 1`,
i.e. the core frame length in bytes (range 1..=16384). Callers treat the
result as the actual byte length and the MIN..=MAX range checks assume so.
Returns `0` when `data` is shorter than `CORE_HEADER_MIN_BYTES` — every call
site rejects that via the minimum-frame lower bound, so a `0` is never
mistaken for a valid tiny frame.
## `dts_core_samples`
Samples in one DTS core frame: `(NBLKS + 1) * 32`. `NBLKS` (7 bits) is the
core-header PCM-sample-block count (ETSI TS 102 114) that fixes the frame's
decoded sample count. Bit layout after the 32-bit sync: FTYPE(1) SHORT(5)
CPF(1) **NBLKS(7)** FSIZE(14) …, so NBLKS = byte4 bit0 + byte5 bits7-2.
## `DTS_PCMBLOCK_SAMPLES` / `DTS_SUBBAND_SAMPLES` (core-header validity)
DTS core-header validity constants (ETSI TS 102 114). For a NORMAL frame
`deficit_samples` must equal `DTS_PCMBLOCK_SAMPLES` — a termination frame
may carry fewer; `npcmblocks` must be a multiple of `DTS_SUBBAND_SAMPLES`;
`audio_mode` must be below `DTS_AMODE_COUNT`; `lfe_present ==
DTS_LFE_FLAG_INVALID` is rejected.
## `DTS_AMODE_COUNT`
Number of LEGAL `AMODE` (channel-arrangement) codes. The 6-bit AMODE field
(ETSI TS 102 114 §5.3.1) has 16 defined channel arrangements, codes 0-15;
only 16-63 are reserved/user-defined and undecodable. The per-AMODE channel
counts in ETSI TS 102 114 §5.3.1 cover all 16, confirming they are
decodable — codes 10-15 are the 6/7/8-channel layouts. A frame is dropped
only when `audio_mode >= DTS_AMODE_COUNT` (i.e. a truly reserved 16-63
code); dropping a legal 10-15 multichannel core would silence recoverable
audio.
## `DTS_CORE_SR_VALID`
Sample rate (Hz) per core `SFREQ` code (ETSI TS 102 114 Table 6-4); a `0`
entry marks a reserved code that fails header validation as an invalid
sample rate. Valid entries are locked to the spec by
`dts_core_sfreq_table_matches_the_dca_spec`; the reserved codes are
{0, 4, 5, 9, 10}.
## `core_header_drop_reason`
Decodability gate: the core-frame header validity checks from ETSI TS 102
114. Returns `Some(reason)` when the DTS core-frame header is invalid — in
which case the packet is undecodable ("Invalid data found") and dropping it
loses nothing a decoder could have used. Returns `None` (keep) for a
decodable header OR if the header can't be fully read (never false-drop on
our own buffer underrun; the framer only emits AUs whose core is fully
buffered and ≥ 96 bytes).
The 4-byte core sync is already validated by the framer, so this reads the
header fields that follow it. The 16-bit CPF header CRC is not verified (we
skip past it) and the audio-header/side-info CRCs are likewise not checked,
because decoders treat those bytes as optional/ignored — verifying them
would drop frames that decode fine (false positives).
## Test: `stamp_pts_reuses_front_when_no_running_cursor_yet`
`stamp_pts`'s third arm (`else if front != PTS_UNSET`) is only reached once
the first arm's `front != self.last_front_pts` has already failed (i.e.
`front == self.last_front_pts`) and no within-PES running cursor is
available yet (`next_pts_ns == PTS_UNSET`). In that state the AU must still
be stamped with its own (repeated) front timestamp, not silently collapsed
to 0 — a `!=` -> `==` typo on the guard would only take this arm when
`front` IS the unset sentinel, producing 0 instead here and the sentinel
value itself if front really were unset.
## Test: `drain_front_collapses_offset_zero_markers_instead_of_leaking`
Every drained access unit that shares its PES with the previous one pushes
a new `(0, pts)` marker once rebased; `drain_front` must collapse those down
to the single most-recent one each time, or `pts_marks` grows once per
access unit for the life of the track — an unbounded allocation on a
multi-hour disc. `front_pts()` alone can't observe this: it always finds the
last offset-0 entry regardless of how many duplicates precede it, so the
leak is invisible unless something checks that the collapse actually ran.
## Test: `make_exss`
Builds a real DTS-HD EXSS substream of `total` bytes (short header form),
with an optional false DTS core syncword embedded in its payload (decoding
to a plausible core size) — to prove precise sizing, not a payload scan,
bounds the extension.
## Test: `make_dts_ext`
Builds a minimal DTS-HD extension substream of `size` bytes (just the sync
+ zero-padding). The parser delimits extensions by the next CORE sync, not
by the extension's own size header, so a valid header isn't required — only
that the bytes carry no spurious core sync.
## Test: `bogus_tiny_core_sync`
Builds 4 bytes that look like a DTS core sync but whose `fsize` field
decodes to a tiny `core_size` (< MIN_CORE_FRAME_BYTES). With the dead-code
guards this passed validation and could close an access unit at a junk
boundary; with the fix it must be drained and resynced past.
## Test: `next_core_boundary_exact_syncword_length_is_not_need_more`
Only the 4-byte EXSS syncword is buffered after the core frame — no header
fields at all. The guard `buf.len() < pos + SYNCWORD_BYTES` must be strict
`<`: at exactly `pos + SYNCWORD_BYTES` the sync bytes ARE fully present, so
the function proceeds to identify them as an extension sync (then fails to
size the header and falls back, ending in `NextCore::None` here since
nothing after it looks like a core sync either). A `<=` typo would instead
return `NeedMore` at this exact length without ever inspecting what the 4
buffered bytes are.
## Test: `make_bad_dts_core`
Builds a structurally-framed but UNDECODABLE core: a valid `make_dts_core`
whose LFE flag is set to the reserved value 3 (`DTS_LFE_FLAG_INVALID`). It
still sizes and syncs correctly (so the framer delimits it normally), but
the core-frame header validity check rejects it as an invalid LFE flag
(ETSI TS 102 114 §5.3.1: an LFF value of 3 is invalid). LFE is byte10
bits2-1, and does NOT feed the frame duration (NBLKS + SFREQ only), so a
dropped bad core still carries the same `DTS_CORE_DUR_NS` as its good peers.
## Test: `reparse_real_dts_file`
Real-data fixture (ignored). Re-parses a raw `.dts` elementary stream
through `DtsParser` and writes the emitted access units back out, so the
garbage-extension → core-only drop can be validated against an actual
damaged stream (e.g. an extracted DTS-HD MA track) end-to-end with an
external DTS decoder. Env: `DTS_IN` (input), `DTS_OUT` (output).
cargo test --lib dts::tests::reparse_real_dts_file -- --ignored --nocapture
## Test: `find_sync_matches_a_buffer_that_is_exactly_the_syncword`
`find_sync` scans `0..=len-4`, so a buffer that is EXACTLY the syncword
still matches. The existing tests cover 0, 3 and "longer than 4", which
leaves the `len == 4` boundary open — and that is the case a syncword split
across PES packets lands on the moment its last byte arrives.
## Test: `a_core_header_of_exactly_the_minimum_length_is_decoded_not_deferred`
`CORE_HEADER_MIN_BYTES` is "enough bytes to DECODE the size field", so a
buffer holding exactly that many must be decoded, not deferred. The
distinction is visible precisely at the boundary: with 10 bytes of a core
sync whose decoded size is sub-spec, the parser must recognise the false
sync, drain past it and resync down to the 3-byte carry-over tail. Waiting
instead leaves the false sync sitting at the front of the buffer, where it
blocks every later real core behind it.
## Test: `flush_emits_a_core_that_exactly_fills_the_buffer`
The end-of-stream flush must emit a final access unit whose core is exactly
as long as the buffer — the ordinary case, since the last AU is closed by
end-of-stream rather than by a following core sync. Rejecting it at the
boundary silently drops the last frame of every DTS track.
## Test: `flush_discards_a_long_buffer_that_does_not_begin_with_a_core_sync`
The flush guard is a DISJUNCTION: a buffer that does not BEGIN with a core
sync is discarded whatever its length. Requiring both conditions instead
lets a long run of junk through to `dts_core_frame_size`, which happily
decodes a 14-bit size out of arbitrary bytes — and the flush then emits an
"access unit" that is not DTS at all.
The junk here is sized so that mis-decoding it yields a plausible core size
that the buffer fully covers, which is exactly when the wrong answer looks
like a right one.
## Test: `exss_frame_size_needs_the_worst_case_header_and_no_more`
`EXSS_HEADER_MIN_BYTES` is the WORST-CASE header length — the long form's
43 bits after the syncword, rounded up to 6 bytes, plus the 4-byte sync. It
gates whether an extension substream can be sized precisely, and that is
what keeps a chance core syncword inside XLL payload from being mistaken
for the next AU boundary; too large and every extension falls back to the
payload scan, too small and the reader runs off a truncated header.
Pinned at both sides of the boundary rather than by value, so the field
widths it is summed from stay honest.
## Test: `an_access_unit_carries_the_source_of_the_packet_its_core_arrived_in`
The whole point of the shared buffer: an access unit whose core arrived in
an EARLIER packet keeps that packet's source offset, not the offset of
whichever packet completed it. At a clip boundary the two belong to
different clips, and taking the later one puts the unit in the wrong clip —
which is what left nine audio and subtitle tracks unplaceable.
</content>
</invoke>
+149
View File
@@ -0,0 +1,149 @@
# DVD AC-3 physical sub-stream probing
## Why this exists (a real-disc wrong-substream bug)
A DVD VTS IFO declares its audio streams in a fixed table, and freemkv's
scan assigns each declared stream a `private_stream_1` sub-stream id purely
by per-codec ordinal — the first AC-3 stream becomes `0x80`, the second
`0x81`, and so on (`ifo::assign_audio_sub_stream_ids`). That assumes the
physical sub-stream order on the wire matches the IFO declaration order.
On some discs it does NOT. A real R2 PAL DVD release's feature declares
ONE AC-3 audio stream the IFO nibble marks as 5.1 (6 channels), but the
physical VOB carries the 5.1 main mix and a 2.0 down-mix on DIFFERENT
`0x8x` sub-stream ids, and the 2.0 is the one that happens to land at the
ordinal `0x80` slot. Routing the declared 5.1 stream to `0x80` by ordinal
therefore muxes the 2.0 down-mix while labelling it 5.1 — the wrong
physical track.
The robust fix is data-driven and codec/disc agnostic: read each physical
AC-3 sub-stream's REAL channel count from the VOB (the `acmod`/`lfeon` of
its first frame after the `0x0B77` sync) and route each IFO-declared AC-3
stream to the physical sub-stream whose actual channel count matches the
IFO's declared count — instead of trusting the ordinal. This never
re-reads the disc beyond a bounded head-of-feature probe and degrades to
the original ordinal mapping when the probe yields nothing
(unreadable/short VOB).
## `PROBE_SECTORS` sizing
How many 2048-byte sectors of the first feature extent to probe. The head
of a DVD feature opens with logos/warnings whose audio is frequently a
thin 2.0 bed on the FIRST sub-stream only — the other physical `0x8x`
sub-streams and the main 5.1 mix do not appear until a sector or two
further in. 512 sectors (1 MiB) was too short: on one real disc it saw
ONLY `0x80`, and only its opening 2.0 frames. 1024 sectors (2 MiB)
reliably contains at least one frame of every physical AC-3 sub-stream
AND enough of `0x80` to reach its 5.1 frames. Still bounded so a live
drive is never hammered (see the project "don't hammer the live drive"
rule).
## Why the maximum channel count, not the first frame
The first frame of a sub-stream at the head of a feature is NOT
representative. A DVD opens with logos/warnings, and the main `0x80`
sub-stream there frequently carries a thin 2.0 bed before transitioning
to its real 5.1 main mix a fraction of a second later (observed on a real
disc: `0x80`'s first frames are acmod=2 → 2 channels, then it becomes
acmod=7+lfe → 6 channels within the same 2 MiB window). Recording only
the FIRST frame read `0x80=2` and missed the 5.1 entirely, defeating the
channel-match routing. The 5.1 capability of a sub-stream is the
*maximum* channel count any of its frames carries, so
`probe_ac3_substream_channels` scans them all and keeps the max.
## `remap_audio_pids` conservatism
Conservative — it only ever REASSIGNS among the physical sub-streams the
probe actually saw, and only when a better (exact-channel) match exists
than the stream's current assignment. A stream whose current sub-stream
already matches is left alone; a stream with no matching physical
sub-stream keeps its ordinal assignment. So a normal disc (physical order
== IFO order) is a no-op.
## Test helper notes (`dvd_audio_probe::tests`)
- `ac3_frame`: builds a single, correctly-SIZED AC-3 frame whose
`acmod`/`lfeon` encode a known channel count. `byte4` is
`fscod=0 | frmsizecod=0`, so `ac3_frame_size` reports 128 bytes and the
frame is zero-padded to exactly that — this lets `max_substream_channels`
advance frame-by-frame over a multi-frame payload exactly as it does on
real VOB data. The BSI bits are laid down with a writer so the test
never hand-miscomputes the lfeon offset, matching `acmod_channels`'
reader.
- `ps_ac3_frames`: builds a minimal `private_stream_1` PES carrying
`frames` for `sub_id`, each preceded only by the 4-byte AC-3 sub-header
at the PES head. Mirrors the on-disc layout the PS demux expects: PES
start `0x000001BD`, length, PES header (no PTS), sub-header
`[sub_id, frame_count, ptr_hi, ptr_lo]`, then the concatenated AC-3
frames.
### `probe_reads_max_channels_no_cross_contamination`
Real-disc regression — the probe must read each sub-stream's TRUE
(max-mix) channel count, not be poisoned by an unrepresentative head
frame, and must NOT cross-contaminate between sub-streams. Mirrors the
real on-disc layout that caused the mis-read: the feature head carries
`0x80` opening with a 2.0 frame and THEN a 5.1 frame (its real main mix),
interleaved with `0x81` carrying only 2.0. The old first-frame probe read
`0x80=2` (the logo bed) and missed the 5.1; the max-over-frames probe
must report `0x80=6` and `0x81=2`.
### `max_substream_channels_locates_sync_after_leading_non_sync_bytes`
Regression guard for a hand-checked mutation (`+` → `-` at the
`pos + rel` offset computation): with `pos` starting at 0 and the first
sync found 3 bytes in, `pos - rel` would underflow a `usize` and panic,
or (if it somehow didn't) index the wrong start entirely. `pos + rel` is
the only computation that is always in-bounds, since `rel` is itself
bounded by the length of the slice searched from `pos`.
### Unmappable-size fallback tests
When an AC-3 header's `fscod`/`frmsizecod` is unmappable (reserved
`fscod == 3`), `max_substream_channels` must fall back to stepping
`start + 2` bytes past the sync to re-lock onto the next genuine sync,
and must keep making forward progress doing so (never revisit the same
sync, which would loop forever, and never jump so far that it skips the
very next real frame).
- `max_substream_channels_unmappable_size_steps_forward_by_two` lays a
bogus-sized header at absolute offset 4 (so `start == 4`,
`start + 2 == 6`) immediately followed, at offset 6, by a real, fully
decodable 2.0 frame — the position the `+ 2` fallback must land on
exactly.
- `max_substream_channels_unmappable_size_at_start_steps_forward_not_back`
puts the unmappable-size sync at absolute offset 0 (`start == 0`) so
that stepping backward instead of forward (`start - 2`) would underflow
rather than merely land on the wrong byte. Also proves the real frame is
still found 6 bytes further in, confirming forward progress past the
bogus header.
### `remap_reads_current_substream_via_and_not_or_or_xor`
`remap_audio_pids` must read a stream's CURRENT physical sub-stream id
from the low byte of its PID via `pid & 0x00FF` — not `|` or `^` with
`0x00FF`, both of which force the low byte to `0xFF` regardless of the
real PID and so always miss the "already matches" shortcut. That matters
observably when TWO physical sub-streams share the same probed channel
count: with a correct read, a stream already sitting on a matching
sub-stream is left alone (conservative, per the module's documented
behaviour); with the low byte forced to `0xFF`, `probed.get(&0xFF)` is
always `None`, so the code falls through to the "find any unclaimed
match" path and picks the FIRST (lowest-keyed, BTreeMap-ordered) matching
physical sub-stream instead — which here is a *different* sub-stream
(0x80) than the one the PID already correctly names (0x81), producing a
spurious PID change.
### `probe_and_remap_reroutes_swapped_substream_scenario_end_to_end`
End-to-end `probe_and_remap`: a real-disc-shaped MpegPs title (one
declared 5.1 AC-3 stream ordinally assigned 0x80) whose physical VOB
bytes carry the 2.0 down-mix on 0x80 and the real 5.1 on 0x81. This must
reach the `remap_audio_pids` call and re-route the stream to 0xBD81. It
also, by construction, proves each of the guards along the way lets a
real, positive case through: the content-format check must NOT bail on
`MpegPs` (only on non-`MpegPs`), the AC-3 presence check must NOT bail
when AC-3 IS present, and the sector-count check must NOT bail when the
count is nonzero — any one of those inverted would skip the probe
entirely and leave the PID at its untouched ordinal value (0xBD80), which
the test's assertion would catch.
+177
View File
@@ -0,0 +1,177 @@
# DVD Title Scan (src/disc/dvd.rs) — Internal Notes
Rationale, regression history, and test-fixture layout notes that were
originally inline comments/doc-comments on private items in
`src/disc/dvd.rs`, relocated here to satisfy the comment-guard's internal-
comment cap. Each pointed-to site in the source carries a short `// See
docs/dvd.md — <topic>` line.
## `Disc::scan_dvd_titles` halt handling
Cancellation: `halt` is polled before the IFO tree is read, and an IFO read
that fails with `Error::Halted` — how a live drive reports a Stop, since
`Drive::checked_exec` fails every command once its flag is set — is
propagated rather than swallowed. Every other IFO failure keeps its
best-effort `Ok(vec![])`.
It has to be an error and not an empty title list, for the same reason
spelled out on `Disc::scan_hddvd_titles`: a cancelled enumeration that
returned `Ok` would be indistinguishable from a disc that genuinely holds
fewer titles. This one was the worst of the three enumerators — a bare
`Err(_) => return Vec::new()` turned an operator Stop into ZERO titles at
rc=0, a disc reported as carrying no video at all.
`scan_dvd_titles` returns the scanned titles plus, when the disc's own
First-Play navigation deterministically dispatches to a title, that title's
`playlist_id` — the nav-resolved main feature. The caller feeds it into
`sort_titles_by_main_feature` as the DVD `nav_feature`, the same seam the BD
path uses. It is `None` (existing ranking heuristics stand) whenever the
navigation is menu-only, malformed, or does not converge.
## DefaultDuration / cadence TODO
`measured_cicp` TODO(spec): populate `top_field_first` + measured colour
from `Mpeg2Parser` once tracks are built after frame parsing — needs a
`CodecParser`->title channel like `codec_private`. Until wired, `None`
falls back to TFF / PAL-NTSC guess.
DefaultDuration TODO(spec): comes from IFO `frame_rate`, so a
soft-telecined 23.976-in-29.97 DVD reports 29.97 instead of true film
rate. Cadence is detectable via `nb_fields` (2:3 pulldown); needs the same
channel as `top_field_first` above.
## Test fixture builders: `build_vmg` / `build_vts`
`build_vmg` builds a VIDEO_TS.IFO: magic "DVDVIDEO-VMG"@0, TT_SRPT sector
ptr@0xC4. TT_SRPT lives at `tt_srpt_sector*2048`: `num_titles(u16)`@0, then
12-byte entries from +8. Each entry: `num_chapters(u16)`@+2, `vts_number`@+6,
`vts_title_num`@+7.
`build_vts` builds a VTS_XX_0.IFO. Layout per ifo.rs:
- magic "DVDVIDEO-VTS"@0
- `vtstt_vobs` (Title VOBS start sector, u32 BE)@0xC4 — the production
`vob_start_sector` the cell sectors are relative to. (0xC0 is
`vtsm_vobs`, the menu VOBS, which the scan must NOT use.)
- VTS_PGCIT sector ptr(u32 BE)@0xCC
- video attr byte@0x200
- `num_audio(u16 BE)`@0x202, audio blocks (8B) @0x204
- `num_subs(u16 BE)`@0x254, subtitle blocks (6B) @0x256
PGCIT (at `pgcit_sector*2048`): `num_pgcs(u16)`@0, PGC info entries (8B)
from +8 with PGC byte offset(u32 BE)@+4. PGC: `nr_programs`@0x02,
`nr_cells`@0x03, BCD time@0x04, `pgm_map` ptr@0xE6, `cell_playback` ptr@0xE8
(both u16 BE rel to PGC start).
## `HaltingReader` test double
A `SectorSource` that fails every read at or above `halt_at` with
`Error::Halted` — how a LIVE DRIVE behaves once the operator presses Stop:
`Drive::checked_exec` fails every SCSI command with `Halted` from then on,
and `Drive::read` deliberately preserves the variant. Reads below the
threshold still succeed, so the scan gets far enough to have something to
truncate.
## Test: `halted_ifo_read_is_not_reported_as_a_shorter_disc`
A Stop on a LIVE DRIVE never touches `ScanOptions::halt`: `Drive` has its
own flag and `checked_exec` fails every SCSI command with `Error::Halted`
once it is set. The DVD enumerator must not swallow that into a
successful scan.
This was the worst of the three enumerators. RED BEFORE GREEN, two
distinct swallows, both measured with the fix reverted:
- `ifo::parse_vmg` treats a failed title set as a placeholder entry and
continues, so a cancel landing on VTS_02's IFO returned
`Ok([VTS_01_1.VOB])` — one title from a two-title disc.
- `scan_dvd_titles`'s `Err(_) => return Vec::new()` turned a cancel landing
on VIDEO_TS.IFO itself into ZERO titles at rc=0 — a disc reported as
holding no video at all.
Both are indistinguishable from a real disc, and both are now
`Err(Error::Halted)`.
Fixture: two title sets, VTS_01's IFO at `PART_START+6000`, VTS_02's at
`PART_START+7000`. Both ICBs sit far below, so filesystem metadata
resolves fine and only the second title set's CONTENT read is cancelled —
the truncation case.
## Test: `scan_dvd_titles_uses_title_vobs_not_menu_vobs`
Regression (first-play menu prepended to the feature): `vob_start` must
come from the **Title** VOBS pointer `vtstt_vobs` (VTS_IFO 0xC4), NOT the
**menu** VOBS pointer `vtsm_vobs` (0xC0). On discs with a per-title menu —
e.g. the Universal "the parental level has been set, press yes" first-play
still — `vtsm_vobs` points at that menu VOB, which sits just before the
title VOB. Cell `first_sector` values are relative to `vtstt_vobs`; reading
0xC0 prepended the menu and shifted every extent back by
`vtstt_vobs - vtsm_vobs`, so the rip opened on the parental prompt instead
of the movie (one real NTSC R1 disc: vtsm=44, vtstt=3640).
Here `build_vts` stamps `vtstt_vobs = 3640` (0xC4); the test additionally
stamps a *different* `vtsm_vobs = 44` (0xC0). The extent must resolve
from 3640.
## Test: `scan_dvd_titles_extent_is_absolute_three_term_sum`
ABSOLUTE-REBASE regression (real-disc menu-at-start fix): `ifo::parse_vts`
now sets `vob_start_sector = file_start_lba(IFO) + vtstt_vobs`, so an
extent's `start_lba` must equal the sum of THREE independent terms — the
IFO file's absolute on-disc LBA, the IFO-relative `vtstt_vobs` (0xC4), and
the cell's `first_sector` — none of which may be dropped. The earlier code
used the bare relative `vtstt_vobs`, placing every extent `ifo_lba`
sectors too early (the rip opened in the VMGI/menu region before drifting
into the movie). The other tests fold two of the three terms together
(zero cell offset, or a single combined expectation); this one keeps all
three distinct and non-overlapping so a regression to ANY two-term
combination is caught.
## Test: `scan_dvd_titles_lpcm_routes_to_a0_pid_range`
LPCM SCAN ROUTING (audit §2 / §5 #6): the 0xA0..=0xA7 PID range was never
exercised in the dvd.rs scan. An LPCM stream (coding_mode 4) at audio
position 1 must get `sub_stream_id 0xA1` → PID `0xBDA1` via
`dvd_audio_pid`, distinct from the AC-3 `0xBD80` space, with its real
channel count preserved.
## Test: `scan_dvd_titles_multiple_vobsub_tracks_distinct_pids`
MULTI-VOBSUB SCAN (audit §2 / §5 #6): the single-subtitle test covered one
track; a multi-subtitle VTS must emit one `Stream::Subtitle` per entry
with distinct PIDs (`0x20 + ordinal`) and per-language tags, all sharing
the PGC palette `codec_data`.
## `stamp_first_play_jumptt` test helper
Stamp a First-Play PGC into a VMG whose pre-command list is a single
unconditional `JumpTT ttn`, so `dvdnav::resolve_main_title` resolves that
title through TT_SRPT. Placed in sector 0, clear of the magic / FP_PGC /
TT_SRPT pointers `build_vmg` already writes.
## Test: `scan_dvd_titles_nav_promotes_first_play_target`
The DVD First-Play nav promotion branch (issue #40): when
`resolve_main_title` returns a `(vtsn, vts_ttn)` target, `scan_dvd_titles`
must map it — via the 1-based `vts_ttn == vts_title_idx + 1` join within
the matching `vts_number` — to the running `title_number`, and surface it
as the `nav_feature`. TT_SRPT here maps title 2 → (VTS 2, title-in-set 1);
the First-Play unconditionally `JumpTT 2`, so the promoted feature must be
the SECOND scanned title (VTS_02 → title_number 2). An off-by-one on the
join or a `vtsn`/`vts_ttn` field swap would promote the wrong title or
none.
## Test: `scan_dvd_titles_drops_leading_scene_index_cell`
End-to-end bug-4 fix: a feature PGC that opens with a leading interleaved-
angle sub-block cell (category 0x90 = in-block cell of an angle block)
must have that cell DROPPED from the muxed extents, so the rip starts at
the real feature. Chapters shift earlier by the dropped duration.
## Test: `scan_dvd_titles_mp2_audio_pid_fallback_is_additive`
Audio PID fallback (dvd.rs `Disc::scan_dvd_titles`): when an audio stream
has no on-wire private_stream_1 sub-stream id — MP1/MP2 audio, per
`ifo::assign_audio_sub_stream_ids` — the PID falls back to `0xBD00 + i`
where `i` is the stream's positional index in the IFO audio-attribute
table. Two MPEG-audio (coding_mode 2) streams must land on two DISTINCT,
correctly-offset PIDs: 0xBD00 and 0xBD01. This pins the `+` (not `-`/`*`)
so the second stream doesn't collide with, or wrap under, the first.
+23
View File
@@ -0,0 +1,23 @@
# src/dvdnav/mod.rs
## Module rationale
Byte layout follows the DVD-Video specification (VMGI/VTSI headers,
PGC/cell tables, PCI/HLI button packets); the VM command decoder is
verified against real discs.
Current contents: `vmcmd` — the VM command decoder; `nav` — the
First-Play navigation executor that resolves which title the disc's own
navigation selects as the feature.
## `resolve_feature_start` — parked status (issue #40)
PARKED (issue #40, `USE_NAV_RESOLVER=false`): this is intentionally a stub,
not an accidental dead one. It unconditionally returns `None` and ignores
`reader`/`udf` on purpose — the IFO/PCI parsing + nav executor (built on
`vmcmd`) land incrementally behind that flag. Until wired in, calling this
is behaviour-neutral (the caller always falls back); improvements to the
resolver take effect here without touching the call site. Do not delete.
`reader`/`udf` are the seam inputs the nav executor will consume to read
VIDEO_TS.IFO + the VTS IFOs/menu VOBs. Reserved until that lands.
+28
View File
@@ -0,0 +1,28 @@
# dvdsub — VobSub SPU boundary rule
The presence of a PTS — not merely an open `pending` — is the authoritative
SPU-boundary signal, so a lost continuation or a corrupt SPU_size can't merge
the next subtitle into the stuck unit.
## ycbcr_to_rgb range convention (deliberate)
This uses the **full-range (JFIF) BT.601** coefficients with no 16/235 luma
scaling. DVD IFO palette YCbCr is nominally studio-swing BT.601, so
studio-swing math would be more colorimetrically "correct" in isolation. But
the output here is a VobSub `.idx` `palette:` line, and the entire VobSub
ecosystem (the original tooling, mkvtoolnix, players that read the `.idx`
palette) is built around this full-range formula — it is the de-facto
on-disk convention. Emitting studio-swing-scaled RGB here would make
freemkv's palettes inconsistent with every other tool and wrong in players
that assume the VobSub convention. Do NOT "fix" this to studio-swing without
changing the consuming side in lockstep.
## ycbcr_to_rgb_reads_byte2_as_cr_and_byte3_as_cb test fixture
A saturated RED entry appears on disc as Y=76, Cr=255, Cb=85 (full-range
BT.601 encoding of RGB #FF0000), i.e. bytes `[0x00, 76, 255, 85]`. Reading
byte 2 as Cb and byte 3 as Cr instead turns this entry BLUE, which is the
exact user-visible symptom. The fixture is deliberately NOT built from this
crate's own doc comments: those described the order wrongly for a long time,
and the previous version of this test inherited the error from them and
therefore could not detect it.
+62
View File
@@ -0,0 +1,62 @@
# src/mux/ebml.rs
## `fixed_width_vint8`
Encodes `data_size` as the FIXED-WIDTH 8-octet EBML VINT used to back-patch
a master element's size field: the `0x01` VINT_MARKER octet followed by the
56-bit VINT_DATA payload, big-endian (RFC 8794 section 4.4 — an 8-octet
VINT carries 7 octets of VINT_DATA).
Extracted so the full 56-bit payload can be exercised directly: reaching
the high payload bytes through `end_master` / `end_master_buf` would take
a multi-terabyte buffer, leaving them unconstrained by any test.
`data_size` must be below 2^56; both callers check that first.
## `start_master_buf` / `end_master_buf`
This is the seek-free twin of `start_master`/`end_master` for elements
small enough to assemble whole before hitting the file (a BlockGroup: one
Block plus a couple of tiny elements). Because `end_master` always
back-patches a FIXED-WIDTH 8-byte VINT (`0x01` + 7 payload bytes) rather
than a minimal-width one, the bytes produced by this pair are byte-for-byte
identical to the seek-and-back-patch pair — assembling in memory cannot
change the emitted Matroska.
## `read_exact_bounded`
Reads exactly `len` bytes WITHOUT trusting `len` to size the allocation.
`vec![0u8; len]` on an attacker-controlled EBML size would allocate
gigabytes before the read fails. Instead the reader is capped to `len`
and the buffer grows as bytes actually arrive: a malformed element that
claims a huge length but supplies few bytes allocates only what it
delivers, then errors on the short read.
## Test: `buffered_master_matches_seeking_master_byte_for_byte`
The buffer-based master helpers must produce byte-for-byte what the
seek-based ones produce. `write_block_group` was rewritten onto
`start_master_buf`/`end_master_buf` to eliminate ~4 seeks (and 4 MiB
BufWriter flushes) PER FRAME, which on an MPEG-2 title is ~350k frames — a
change only safe because the two encodings are identical.
Both write a FIXED-width 8-byte VINT placeholder (0x01 + 7 payload bytes)
and patch it in place; neither ever emits a minimal-width size. This test
pins that, so a future "optimisation" of either one to minimal-width sizes
cannot silently desync the two and change emitted Matroska.
## Test: `buffered_master_nests_correctly`
Nested masters must patch correctly too — the MVC BlockGroup nests
BlockAdditions > BlockMore inside the BlockGroup, and in-memory patching
works by index rather than by file offset, so nesting is where an
index-arithmetic slip would show up.
## Test: `fixed_width_vint8_is_big_endian_over_the_full_payload`
The fixed-width 8-octet VINT is `0x01` followed by the 56-bit VINT_DATA
payload in BIG-ENDIAN order (RFC 8794 section 4.4). Every payload octet is
distinct here, so a shifted, reversed or dropped octet is visible; the top
payload octets are unreachable through `end_master` without a
multi-terabyte buffer, which is why this is tested at the encoder.
+51
View File
@@ -0,0 +1,51 @@
# `dirimage::encode` — descriptor encoder notes
Overflow rationale for comments in `src/dirimage/encode.rs` that would
otherwise exceed the comment-guard caps. Each section below is pointed to
by a short `//` comment at the relevant spot in the code.
## Why a pure function
`encode` is a pure function from `Layout` to sectors — nothing here touches
the filesystem. That is what makes it testable against the production
parser in `udf.rs`: the same bytes this module writes are read back and
checked by the real UDF reader, so the encoder and the parser can't drift
apart silently.
## Why UDF 1.02 with a single Type-1 partition map
UDF revision 1.02 with a single Type-1 partition map is deliberate: it is
the DVD-Video profile, it is the shape `read_filesystem` takes when
`num_partition_maps < 2`, and it avoids the UDF 2.50 Metadata Partition
entirely. That also means a synthetic image never exercises the Metadata
Partition path in `udf.rs` (`:946-991`).
## `finish_tag`: why `tag_loc` correctness matters
Getting `tag_loc` wrong (absolute vs. partition-relative) is the classic
reason a hand-built volume mounts nowhere: a driver that validates the tag
location rejects the descriptor outright. `desc_len` is the descriptor's
total length including the tag; the CRC covers `buf[16..desc_len]`.
## `encode_cs0`: why ASCII, not Latin-1, for the 8-bit form
`parse_udf_name` (`udf.rs:1467`) decodes a compression-8 name with
`from_utf8_lossy`, so a 0x80-0xFF byte — legal CS0 under Latin-1 — would
come back as U+FFFD. Every character above 0x7F therefore takes the 16-bit
(UTF-16BE) form, which that parser decodes correctly.
## `file_entry`: why tag 261, not the Extended File Entry (266)
Real BD-ROMs use the Extended File Entry (tag 266), but an EFE requires
UDF 2.00+, and this image declares 1.02, so `file_entry` writes the plain
File Entry (tag 261) instead. `udf.rs` reads both — the 261 field offsets
it uses (`l_ea` 168, `l_ad` 172, ADs at `176 + l_ea`) are the ones written
here.
## `push_fid`: why FIDs may span logical blocks
FIDs are packed with no inter-descriptor padding beyond the 4-byte
alignment the spec mandates, and they are allowed to span logical blocks —
which is also what `read_directory` (`udf.rs:1312`) assumes: it walks the
directory extent as one flat byte run and stops at the first non-257 tag,
so any block-alignment gap would truncate the directory.
+151
View File
@@ -0,0 +1,151 @@
# encrypt.rs — relocated rationale
Overflow prose that would not fit under `ci/comment-guard.py`'s caps,
moved here per file with a `// See docs/encrypt.md — <topic>` pointer
left at the original site.
## HandshakeResult::drive_unlocked
True when the VID came from an unlocker (in `freemkv-unlock`) that
unlocked the drive. Such a drive serves CLEAR content, so AACS bus
encryption is already removed AT THE DRIVE — the same end state a
successful cert handshake's `read_data_key` provides, just via
firmware instead of the AKE. The bus-key gate MUST credit this as a
valid bus-removal: bus encryption is unremovable only when NEITHER the
firmware unlocked the drive NOR the cert handshake yielded a bus key.
Without this, a SUCCESSFUL unlock (VID present, `read_data_key: None`)
paradoxically trips the gate and blocks ALL key resolution (incl. the
online source).
## bus_encryption_removed
Single source of truth for "is AACS bus encryption gone for this
scan?". The gate asks ONLY this — `if !removed { error }` — never
enumerating cases. Bus encryption is gone when ANY of these holds:
the disc never had it; file/ISO reads (content already clear at read
time); an unlocker unlocked the drive (it serves clear content); or
the cert handshake produced the bus key. Add a NEW removal mechanism
HERE, never in the gate.
## AacsCertUnlocker
libfreemkv-side driver for the AACS cert route. It owns the host-cert
collection (a keysource concern that stays in libfreemkv) and then
dispatches the actual mutual-auth to the `freemkv-unlock` AACS
unlocker via the `crate::unlock_bridge`. The firmware (drive-prep) and
CSS routes dispatch the same way at their own call sites; this one
carries the host certs.
## AacsCertUnlocker::authenticate
Runs the host-certificate mutual-auth handshake: collect
non-compiled-in host certs from the key sources + credentials, then
hand them to the AACS unlocker (via the `freemkv-unlock` dispatch),
which tries each cert (wedge-guarded) and on success yields the
Volume ID + `read_data_key` (the AACS 2.0 bus key). Returns a
`CertUnlockFailure` on every no-VID outcome.
## unlock_error_to_error
Maps a `CertUnlockFailure` back to the `Error` variant
`do_handshake_cert` has always surfaced, so `scan_with`'s rendering
and the path-1 disc-hash → VUK fallback are byte-for-byte unchanged.
(`NoHostCert` keeps the `<no host cert>` sentinel.)
## handshake_has_volume_id
Did the cert handshake actually carry a Volume ID? Extracted so it can
be tested as a VALUE. It only ever reaches an operator as the
`has_volume_id` field of the `bus_key_unavailable` warn, and asserting
on a `tracing` field means installing a capturing subscriber — which
is thread-local, while `tracing`'s callsite-interest cache is global.
Those two facts race: the test failed roughly one run in ten under the
full parallel suite while passing every time in isolation, and
serialising the captures was not enough because the cache can be
re-evaluated against the process default rather than the thread-local
dispatch.
A predicate this small does not need a subscriber to verify. The
polarity is the whole point: an `==` here would tell an operator a VID
was absent on exactly the discs where one was present.
## Disc::do_handshake
SCSI handshake — drives the VID-acquisition flow and returns a
structured `HandshakeResult` for downstream key resolution.
VID acquisition runs through `do_handshake_cert`, which first uses the
OEM VID a firmware unlocker may have stashed at drive `init()` (a
drive-functionality capability decoupled from the host cert + HRL) and
falls back to the cert-based mutual-auth handshake (dispatched to the
`freemkv-unlock` AACS unlocker) when none is present. The cert path
also yields `read_data_key`, required for AACS 2.0 bus decryption.
Returns `(handshake, error)`:
- `(Some(_), None)` — VID acquired
- `(None, Some(_))` — specific failure mode (`AacsHostCertRejected` or
`AacsVidUnavailable`)
- `(None, None)` — handshake not attempted (no keydb; resolution will
proceed with VID=zero and rely on path 1 disc-hash → VUK lookup)
## Disc::do_handshake_cert
Cert-based AACS handshake — the cert route for VID acquisition.
Before running the cert mutual-auth, this checks for an OEM Volume ID
a firmware unlocker stashed at drive `init()`. Such an unlocker
unlocks *drive functionality*, not just the disc: VID retrieval via
the drive's OEM CDB is a capability separate from `unlock`. When one
served a VID, we use it and SKIP the cert handshake entirely — the
OEM path gets the VID *without* the host certificate + HRL, decoupling
VID from the cert chain. The OEM path yields no `read_data_key` (no
bus-key is derived); AACS 2.0 content needing read_data_key for bus
decryption must still use the cert path, so an unlocker with no OEM
VID capability returns `None` and we fall through to cert auth
unchanged.
## Disc::collect_host_certs
Collect every AACS host cert the caller carries, from BOTH the
explicit `DriveCredentials` and the key-source layer
(`crate::KeySource::host_certs` across each source), unioned. Host
certs are keysource-served, never compiled in; this is the one place
the OEM cert route gathers them. An empty result is the graceful
no-cert signal (the caller turns it into `Error::AacsNoHostCert`).
`mkb` is the disc's MKB generation when known, forwarded to each
source's `crate::KeySource::host_certs` so a source MAY return only
generation-appropriate certs (the default ignores it).
## Disc::resolve_vid_only
Build a keys-free AACS state that carries only the Volume ID (+
version metadata), for callers that resolve Unit Keys out-of-band and
have disabled the local keydb. The VID is on-disc content read during
the handshake; preserving it here lets the out-of-band path use it. No
keys are present (`unit_keys` empty, `vuk` None), so the disc reports
as "encrypted, no keys" until the caller re-scans with a resolved Unit
Key.
## build_mkb (test helper)
An MKB with one Type-and-Version record (type 0x10) carrying the
version as BE u32 at record offset 8, followed by a recorded EOF
record then trailing zero padding. mkb_content_len walks records and
stops at the first padding (type 0) byte (aacs/inf.rs).
## resolve_vid_only_no_cert_defaults_version_uhd (test)
No content cert at all → version defaults to UHD (major 2), matching
`read_aacs_version` so the scanned `AacsState.version` and the
out-of-band fetch agree on the Unit_Key_RO stride (audit #4: a wrong
BD-vs-UHD guess mis-parses unit keys). bus_encryption false
(unreadable → off).
## Skipped/deferred test coverage
Tests for `read_vid_oem` (response parsing) are skipped in this file
since `Drive` construction requires a live transport; parsing branches
are exercised through `read_vid_oem`'s callers in integration. The
`collect_host_certs` tests union `DriveCredentials` with the
key-source layer; empty means the route fails gracefully
(`AacsNoHostCert`).
+181
View File
@@ -0,0 +1,181 @@
# Error Codes
Overflow detail for `src/error.rs` doc comments that would otherwise exceed
the comment-guard caps. Each section below is pointed to by a short `//` or
`///` comment at the corresponding definition.
## Error Code Ranges
| Range | Category |
|-------|----------|
| E1xxx | Device errors |
| E2xxx | Profile errors |
| E3xxx | Unlock errors |
| E4xxx | SCSI errors |
| E5xxx | I/O errors |
| E6xxx | Disc format errors |
| E7xxx | AACS errors |
| E8xxx | Keydb errors |
| E9xxx | Stream/mux errors |
## E_UDF_NO_USABLE_EXTENT
A file that EXISTS and whose allocation descriptors resolved without error,
yet yields not one usable extent: an empty AD list, or a list every entry of
which is zero-length or points at LBA 0. Reported by the HD-DVD clip resolver
(`disc::hddvd`).
It has no `Error` variant on purpose. `UdfFs::file_extents` returns
`Ok(vec![])` here rather than failing — the emptiness is only a defect in the
eye of a caller that needs bytes — so the condition is DETECTED by the
caller, not returned to it. The code exists so that detection can be
accounted in the log with something other than a neighbouring error's code:
logging it as E6017 would file a zero-length AD list as an authoring hole
and send whoever triages it at the wrong population.
## E_CSS_NO_DISC_KEY / CssNoDiscKey history
While the disc-wide CSS failure and the per-title CSS failure both used
`E_CSS_KEY_MISSING`, an uncrackable CSS disc was iterated title by title,
each one logged as skipped, and the run exited reporting success — a total
failure reported as success. `E_CSS_NO_DISC_KEY` / `Error::CssNoDiscKey` was
split out so `is_disc_level_no_key` classifies it and a multi-title rip loop
fails fast on it instead.
## E_KEY_SERVICE_UNAVAILABLE
A seven-hour run of HTTP 502s reported as `E_NO_DISC_KEY` told operators
their disc was not in the key database and sent them hunting for a VUK that
was never missing; the correct action was to wait. `E_KEY_SERVICE_UNAVAILABLE`
exists so a key-source outage (transport error, DNS failure, timeout, TLS
failure, HTTP 5xx, unreadable reply) is distinguishable from a source that
answered and simply has no key.
## E_MKV_SOURCE_INVALID vs E_MKV_INVALID
`is_skippable_title_stub` classifies `E_MKV_INVALID` as a title that yielded
no muxable frames, which an all-titles rip may skip while finishing the
rest. A corrupt or truncated `mkv://` source file is a FAILURE, not a stub —
reporting it as skippable would let a broken source be silently passed over
by a run that then exits successfully. `E_MKV_SOURCE_INVALID` (and the
sibling codes `E_MUX_HEADER_BUFFER_EXCEEDED`, `E_MKV_LACING_INVALID`) exist
to keep that distinction, and are fatal.
## UdfEmbeddedData
A file that legitimately stores its data this way is tiny (an ICB caps it at
well under 2 KiB — the AACS `*.inf` key files are the usual case), and the
callers that expect one read it via `read_inline_data` long before extents
are ever requested. A stream file cannot be one. `read_directory` already
refuses the same shape for directories; `UdfEmbeddedData` is the file half
of that decision.
## MkvInvalid / MuxHeaderBufferExceeded history
`MuxHeaderBufferExceeded` exists because hundreds of megabytes of real
frames with unresolvable codec-init headers is NOT the same thing as
`MkvInvalid`'s canonical case (an empty nav/menu PGC stub). Reporting the
former as the latter would have `is_skippable_title_stub` silently drop a
main feature from a rip that then exits successfully.
## AacsBusKeyUnavailable
NOT raised for AACS 1.0 BD (no bus encryption, `read_data_key` legitimately
absent) nor for file-backed (ISO) scans, where bus encryption was already
removed at read time and no handshake runs. Only a live-drive AACS 2.0/UHD
scan that obtained the Volume ID but produced no bus key raises this.
## SourceTerminated
The alternative answer is a lie: a dead source that reports `Ok(0)` is
indistinguishable from end-of-stream, and `DiscStream::fill_extents`
legitimately reads a short count as a skippable hole — zero-filling and
advancing over every remaining sector of the title and still returning
success. Unlike a bad sector, this condition cannot be retried at a smaller
size or skipped past.
## error_code / From<Error> for io::Error round-trip
`mux_stream` hands back an `io::Error`, and a front-end reporting *why* a
title failed had no way to recover the code from it — the typed `Error` is
gone by then and only the `E<code>` string prefix survives. Parsing that
prefix is `error_code`'s job; every consumer re-implementing the parse is how
the string-matching this crate spent 1.5.x removing comes back.
`From<Error> for io::Error` boxes the typed value as the payload
(`io::Error::new(kind, e)`), whose `Display` is the same `E<code>[: …]`
string the stringifying version produced — so `error_code`'s parse is
unaffected, and `From<io::Error> for Error` can additionally `downcast` the
payload back to the exact typed error. Errors that did NOT come from this
crate carry no `E<code>` prefix and yield `None`.
## is_skippable_title_stub
This replaces the CLI's `E7023`/`E6008` string-match with a typed check on
the `io::Error` `mux_stream` returns.
`Error::MkvSourceInvalid`, `Error::MkvLacingInvalid`, `Error::MkvUnencodable`
all used to be raised as `Error::MkvInvalid` and therefore landed in the
skippable set, so a corrupt source was reported as a title worth silently
passing over by a run that then exited successfully. They now carry their
own codes and are fatal, as is `Error::MuxHeaderBufferExceeded`.
`Error::CssNoDiscKey` (the disc's CSS crack recovered no key at all) is the
same conflation on the decrypt axis: while it too was raised as
`Error::CssKeyMissing`, every title of an undecryptable disc classified as a
skippable stub, so the rip loop skipped all of them and exited successfully.
It belongs to `is_disc_level_no_key` instead, and is fatal here.
## is_disc_level_no_key
`E_CSS_NO_DISC_KEY` is the CSS side of the disc-level/per-title split, and it
exists because the disc-wide CSS failure used to be raised as the per-title
`E_CSS_KEY_MISSING`: an undecryptable CSS disc landed in
`is_skippable_title_stub`, so the rip loop skipped all N titles with an
"empty stub" notice and exited successfully.
The key-SOURCE failures (`E_KEY_SERVICE_UNAVAILABLE`,
`E_KEY_SERVICE_UNAUTHORIZED`, `E_KEY_SERVICE_RATE_LIMITED`) are classified
here for the same fail-fast reason and NOT because they mean "no key": a
service that is down, refusing the token, or throttling is down for every
title on the disc, so iterating N titles re-issues N doomed requests (and,
on 429, digs the rate-limit hole deeper). They are separate CODES precisely
so the front-end can say "retry later" / "fix the token" instead of
`E_NO_DISC_KEY`'s "no key source has a key for this disc".
## is_marginal_read
- MEDIUM ERROR (sense key 3) — canonical bad-sector signal.
- ABORTED COMMAND (sense key B) — transient; retry usually works.
- NOT READY (sense key 2) — the dominant bad-sector response on the BU40N
(ASC 0x04/ASCQ 0x3E); a pause + retry often recovers.
- RECOVERED ERROR (sense key 1) / NO SENSE (sense key 0) — not classified as
fatal; treat as recoverable.
Caller-agnostic predicate — describes a property of the *error*, not what
one specific call site should do with it. Used by
`freemkv_engine::recovery::copy`'s hysteresis dispatch.
## Test rationale: declared_error_codes / uniqueness
The uniqueness test used to carry a hand-maintained `vec![]` of the error
constants, with a doc comment claiming it "pins all code assignments". It did
not. At the time this was written the file declared 127 `pub const E_*` and
the vector named 109 of them: eighteen codes — `E_DIR_IMAGE_FANOUT`,
`E_DIR_INSUFFICIENT_SPACE`, `E_DIR_MULTIPASS_REJECTED`, `E_DIR_NAME_COLLISION`,
`E_DIR_NAME_TOO_LONG`, `E_DIR_NOT_EMPTY`, `E_DIR_RAW_REJECTED`,
`E_DIR_SOURCE_UNSUPPORTED`, `E_DIR_WRITE_FAILED`, `E_DRIVE_INQUIRY_SHORT`,
`E_EMPTY_IMAGE`, `E_MP4_UNKNOWN_RESOLUTION`, `E_SEAM_PLAN_DROPPED_MOST`,
`E_SHORT_IMAGE_READ`, `E_SINK_WROTE_NOTHING`, `E_SOURCE_TERMINATED`,
`E_SYNC_TIMEOUT`, `E_SYNC_WORKER_LOST` — were outside the guarantee entirely,
so a new variant colliding with any of them passed green. Worse, an earlier
audit READ that doc comment and trusted it while assigning new codes.
The defect is not the eighteen omissions, it is that the list is
hand-maintained at all: adding a constant and forgetting the vector is a
silent no-op, which is the definition of a guarantee that decays. Deriving
the list from the declarations makes forgetting impossible — the only way to
escape the check is to stop declaring the constant. `include_str!` of the
same file is the cheapest seam that does that: no `build.rs`, no proc macro,
no dependency, and — being `#[cfg(test)]` — not one byte of the source
embedded in a release build.
+150
View File
@@ -0,0 +1,150 @@
# `src/disc/extract.rs` — decrypted file-tree extraction
## Relation to the ISO sweep/patch recovery passes
`extract_tree` is a sibling of the disc→ISO sector dump (the sweep/patch
recovery passes, which now live in the `freemkv-engine` crate), specialized
to write **per file** rather than a whole image, applying decryption on the
way out, and **without** any multipass / recovery orchestration — it is
1-shot, decrypt-only.
No new error codes are introduced: it reuses the existing `Error` set
(`error.rs` is numeric-only, no English).
## `extract_tree` contract details
`reader` is consumed for content reads (it is moved into a
`DecryptingSectorSource`); the structure + extent metadata are read first
with the raw reader. `dest` receives the tree STRAIGHT IN (no auto-named
subfolder). The caller must have run the pre-flight decrypt gate
(`Disc::ensure_decryptable`); this method resolves per-VTS CSS keys itself
for DVD.
Bad sectors become recorded zero-filled holes in the affected file (the run
does not abort); undecryptable units are counted as loss. Files are written
`<name>.partial` and renamed on success, so an interrupted run never leaves
a half-written file that looks complete.
## `resolve_vts_key` — why a failed crack must be a hard error
FALLIBLE, because the three outcomes of a crack are not two.
[`crate::css::CrackOutcome`] exists precisely to separate "no scrambled
sector was seen, so there is nothing to decrypt" from "scrambled sectors
were seen and no key came out", and its doc says callers MUST surface the
second as a hard error. An earlier version of this function used the
`Option`-returning `crack_key`, which collapses both into `None`, and then
fell back to the disc-wide key — descrambling this VTS with ANOTHER VTS's
title key. The hazard is described in the ordering note in the function
body, which was written about the same fallback; ordering makes the crack
far more likely to succeed, but it cannot make a failed crack safe.
## `Borrowed` — why a borrowing `SectorSource` wrapper
Lets the decrypting decorator "own" an inner source for its lifetime while
the caller keeps the underlying `&mut dyn SectorSource` (the decorator is a
`DecryptingSectorSource<S>` generic over `S`, so it does NOT require a
`'static` boxed inner — unlike the mux highway, which takes the reader by
value). The decorator is dropped before `extract_tree` returns, so the
borrow never escapes.
## `whole_unit_batch` — the trailing-partial rule
`remaining` is the sectors left in the current extent; the batch is capped
at `READ_BATCH_SECTORS` and rounded DOWN to a whole number of 3-sector units
UNLESS it is the extent's final (possibly short) tail — the tail always
begins on a unit boundary, so a 1-2 sector partial there is handled by
`decrypt_sectors`' trailing-partial contract. Used by `extract_one_file` so
this rounding rule lives in exactly one place.
## `available_space` (windows) — why the free-space gate matters
Windows has no `statvfs`. This once returned `None` unconditionally, which
does not merely skip a test — it skipped the free-space GATE, so a Windows
user extracting a disc to a full volume got a confusing failure part-way
through instead of a clear refusal up front. libfreemkv ships a Windows
GUI, so that is the platform where the friendly error matters most.
`FreeBytesAvailableToCaller` (not `TotalNumberOfFreeBytes`) is used because
it accounts for per-user quotas, matching what `statvfs`'s `f_bavail` gives
on the unix side.
## `sanitize_component` — why reserved names are substituted, not rejected
A name whose base matches a Windows reserved device name (`NUL`, `CON`,
`COM1`..) is **substituted** (prefixed with `_`) rather than rejected — a
Linux-authored disc may legally carry such a file and a single reserved
name must not abort the whole tree walk. An empty result after stripping
is an error.
## `names_differing_only_by_case_are_a_collision` — why this must be an error
Two disc names differing only by CASE are one host file on macOS APFS and
Windows NTFS, both case-insensitive by default. Keyed by the case-preserving
path, the collision map would see two entries, raise nothing, and the
second file extracted would overwrite the first — while both `PlannedFile`s
report `complete: true`. Silent data loss reported as a clean extract is
the one outcome this crate must never produce, so the host-equivalence key
has to model the host's namespace, not the disc's.
## `a_files_partial_path_colliding_with_another_files_final_name_is_an_error`
A disc carrying both `X` and `X.partial` plans two distinct final names, so
nothing collides in the final-name check — but extracting `X` writes
through `X.partial`, the same host path the other file owns. Whichever
lands second truncates the other, and both entries would still be reported
complete without this guard.
## `multi_extent_aacs_anchors_unit_base_per_extent` — the alignment bug
Regression (rc.6 audit, finding #449): a MULTI-EXTENT AACS file must
re-anchor the unit-alignment base PER extent, not once at the first extent.
The two extents in this test start at absolute LBAs whose difference is NOT
a multiple of 3 (`PART_START+5000` vs `PART_START+5004` — a 4-sector delta).
With a single first-extent unit base the second extent's first read is off
the unit grid (offset 4, `4 % 3 == 1`) → `is_unit_aligned` fails →
`DecryptFailed` → the whole extent becomes a zero-filled (false) hole. With
per-extent anchoring both extents read clean: `bytes_unreadable == 0` and
the file content survives. The fixture content is clear (no TS syncs) so
the decrypt step restores each unit verbatim — this isolates the GATE from
the cipher math.
## `extract_tree_zero_fills_an_unrecorded_extent_instead_of_reading_it`
An ECMA-167 4/14.14.1.1 type-1 extent is ALLOCATED BUT NOT RECORDED: the
space belongs to the file and occupies its byte range, but nothing was ever
written there, and the standard defines its contents as zeros.
`read_icb_extents` keeps the flag (`IcbExtent::recorded`) and
`read_file_limited` honours it by emitting zeros WITHOUT touching the
media. The tree extractor reads the same ICBs and must agree: reading those
sectors returns whatever the media happens to hold there — on an AACS disc,
ciphertext that decrypts to noise — and writes it into the extracted file
as if the disc had recorded it.
## `per_extent_base_is_aligned_first_extent_base_is_not`
Focused alignment-computation check underpinning the per-extent fix: when
each extent anchors its OWN start as the unit base, the extent's own batch
starts are always unit-aligned; anchoring a later extent against the FIRST
extent's base mis-aligns whenever the extents' starts differ by a
non-multiple of 3 sectors. This is the exact arithmetic the decrypt-on-read
gate (`aacs::content::is_unit_aligned`) performs.
## `css_two_vts_groups_do_not_cross_contaminate_keys`
Regression: the per-VTS key crack in `resolve_vts_key` must gather ONLY
this VTS's own title-VOB extents. Two VTS groups here carry DISTINCT
scrambling keys; if the group filter (`vts_group_of(..) == Some(vts) &&
is_title_vob(..)`) is loosened (`!=`, or `&&` -> `||`), one VTS's resolve
gathers the OTHER VTS's extents, cracks the wrong key, and that VTS's own
content silently fails to descramble to the expected plaintext.
## `a_scrambled_vts_that_cannot_be_cracked_fails_instead_of_borrowing_a_key`
A VTS that IS scrambled but whose key could not be recovered must FAIL, not
borrow another VTS's key. `crack_key` returns `Option`, which collapses "no
scrambled sector was seen" with "scrambled sectors were seen and no key
came out"; the fallback then descrambled this VTS under the disc-wide key
and `extract_tree` reported `complete = true` at exit 0, because no read
had failed. `CrackOutcome` exists to keep those two apart, and every
sibling path in the crate already honours it. The fixture: VTS_01 is
crackable, VTS_02 is scrambled with NO periodic crib, so its crack
genuinely fails.
+36
View File
@@ -0,0 +1,36 @@
# `file_sector_source::linux` — read-side platform hints
## Why both `hint_sequential` and `drop_window`
`POSIX_FADV_SEQUENTIAL` at open widens the kernel's readahead window so
each pread aggregates into fewer NFS round-trips. `DONTNEED` on the
consumed window (called periodically by the caller) drops the
already-read pages from the page cache so an 85 GB streaming ISO read
doesn't fill memory and starve concurrent writes (the MKV output
during mux). Together they mirror the write-side WritebackPipeline's
policy.
## History
Pre-Phase-1 (0.20.7 baseline) had both. Phase 1's introduction of
`FileSectorSource` silently dropped the read-side DONTNEED, and
0.21.2's revert of `SEQUENTIAL` (mistakenly attributing a regression
to it) removed the hint. Net effect: 85 GB of ISO reads pinned in the
page cache + no readahead widening → mux throughput collapse from
18 MB/s historical to 2.7-8 MB/s on 0.21.x. Restored in 0.21.6.
## `prefetch`
Async-prefetch `len` bytes at `offset` into the page cache. The kernel
`readahead(2)` syscall queues the I/O and returns immediately — it
does NOT wait for completion. Called right after each consumed read
so the next batch's I/O overlaps with the caller's processing of the
current batch (decrypt + demux + mux).
Without this hint, with a synchronous demux consumer running at
~50 MB/s and a single-spindle disk capable of ~150 MB/s, the disk sits
idle ~70% of each iteration because kernel readahead alone (capped at
`/sys/block/<dev>/queue/read_ahead_kb`, default 128 KB) can only
pre-stage a tiny slice of the next batch. An explicit `readahead()` of
the same size as the current batch tells the kernel to queue the full
next-batch read now.
+29
View File
@@ -0,0 +1,29 @@
# file_sector_source::macos
macOS has no direct `POSIX_FADV_SEQUENTIAL` equivalent; the idiomatic hint is
`fcntl(F_RDADVISE, &radvisory)` describing the byte range you intend to read
soon. The open-time sequential hint points it at the whole file (clamped to
`RDADVISE_MAX_BYTES` so a multi-TB ISO doesn't ask the kernel to prefetch
everything at once).
## RDADVISE_MAX_BYTES
Cap on the byte length passed to `F_RDADVISE`. Asking for a multi-GB
readahead window is counterproductive — the OS doesn't have that much cache
to throw at one fd. 64 MiB is generous for our use case (sweep, mux) so the
kernel's prefetch keeps pace with our app-level pipeline depth.
## drop_window
macOS has no direct `POSIX_FADV_DONTNEED` equivalent for a byte range.
`fcntl(F_NOCACHE)` would disable caching globally on the fd (too coarse — we
want the unread region to still benefit). Best approximation: no-op.
macOS's unified buffer cache is generally less prone to the pin-everything
pathology that triggers the regression on Linux NFS clients.
## prefetch
Async-prefetch the byte range `[offset, offset+len)`. macOS uses the same
`fcntl(F_RDADVISE, &radvisory)` primitive as the open-time sequential hint,
just targeted at a moving window instead of the whole file. The kernel
queues I/O for the requested range and returns immediately.
+77
View File
@@ -0,0 +1,77 @@
# `FileSectorSource` design rationale
`FileSectorSource` reads 2048-byte sectors from an ISO file on disk via
direct `seek + read_exact` (`pread`-equivalent) calls, letting the kernel's
own readahead policy manage prefetch.
## Why no app-level buffer
Pre-0.21.3 this source held a 32 MiB (later 4 MiB) read-ahead buffer to
amortise per-sector NFS round-trips. Empirically that buffer hurt: 32 MiB
refills bursted the NFS TCP connection hard enough to starve the concurrent
writer, and even a 4 MiB window gave the kernel less freedom to pipeline
reads with writes. Direct pread per call lets Linux's readahead widen as it
detects the sequential pattern, and naturally interleaves with writeback.
## DONTNEED on the consumed window
Without page-cache eviction an 85 GB streaming ISO read pins the entire
file in memory, starves the concurrent writer, and collapses mux throughput
(observed: 2.7 MB/s mux on 0.21.5 vs. 70 MB/s isolated NFS reads). Every
`READ_DROP_CHUNK_BYTES_DEFAULT` of consumed bytes we call
`posix_fadvise(DONTNEED)` over that window, mirroring the write-side
`crate::io::writeback::WritebackPipeline` policy.
The drop window is accounted by a monotonic forward byte counter, which
matches the sequential streaming pattern the mux highway drives. Under
random or backward access the dropped range no longer lines up with the
bytes actually read — but `DONTNEED` is purely an advisory cache hint with
no correctness impact, so this degrades to a slightly imprecise hint rather
than a bug.
`READ_DROP_CHUNK_BYTES_DEFAULT` (32 MiB) is the empirically tuned value on
a 7200rpm HDD via SATA: smaller windows (8 / 16 MiB) shorten the
kernel-readahead overlap and slow the producer; larger windows (64 / 128
MiB) let the page cache pin enough of the ISO to pressure concurrent
writes. Override via `FREEMKV_READ_DROP_CHUNK_MIB`.
`READ_DROP_CHUNK_MIB_MAX` (64 GiB) bounds `FREEMKV_READ_DROP_CHUNK_MIB`
before the MiB→byte multiply, mirroring `WRITEBACK_CHUNK_MIB_MAX`, whose
identical multiply is bounded for exactly this reason: without the bound,
a value above 2^44 overflows — a panic on the first ISO open in an
overflow-checked build, and in release a wrap to a near-zero window that
fires `drop_window` on every read. Out-of-range values fall back to the
default.
## Platform open hint
On `open()` each platform issues its "sequential access expected" hint so
OS-level readahead widens. The hint and the DONTNEED call live in per-OS
sibling modules (`linux::hint_sequential` et al.) — no inline `#[cfg]` in
this file.
## Read-ahead prefetch
After every consumed read we issue an OS-level prefetch hint for the next
equivalent-sized window (`platform::prefetch`). The kernel queues that I/O
asynchronously and returns immediately, so the next batch's read overlaps
with the caller's processing of the current batch (decrypt + demux + mux).
Without this the disk sits idle ~70% of each iteration because kernel
SEQUENTIAL readahead alone (capped at `read_ahead_kb`, default 128 KB) is
far smaller than our 16 MiB app-level batch.
## Test notes
- `read_sectors_with_an_undersized_buffer_errors_rather_than_panicking`:
this is a public `SectorSource` impl, so buffer length is caller input,
and the guard used to be a `debug_assert!` — compiled out in release,
where the `out[..bytes]` slice then panicked with 'range end index out
of range'. `Drive::read_fua` already carries this exact guard with a
comment recording the same panic being fixed there, and
`PrefetchedSectorSource` has `direct_read_too_small_buffer_errors` for
the same case; this impl had neither.
- `dontneed_eviction_does_not_affect_data`: reads past the default 32 MiB
drop chunk (16384 sectors) so the eviction block fires at least once,
asserting every sector still reads correctly. Avoids mutating
`FREEMKV_READ_DROP_CHUNK_MIB` to sidestep a parallel-test env race with
`drop_chunk_size_env_override`.
+16
View File
@@ -0,0 +1,16 @@
# FLAC elementary-stream decodability gate (`src/mux/codec/flac.rs`)
FLAC frames carry no length field, so a raw stream is delimited only by
sync-scanning + CRC validation. In freemkv, though, FLAC never arrives raw:
it comes from mp4/mkv, where each packet is exactly one container-delimited
FLAC frame (a complete, pre-delimited frame per packet). So this parser
is a per-packet gate, not a framer: every FLAC frame ends with a 16-bit CRC
(poly 0x8005, init 0, non-reflected) computed so the residue over the whole
frame — footer CRC included — is zero (per the FLAC format specification,
RFC 9639, frame footer). A nonzero residue is definitive corruption → drop
the frame (a silence gap, never a shift — each packet keeps its own PTS),
logged via the shared tally.
A packet that does not begin with the FLAC frame sync is not a delimited
frame we can validate, so it is passed through unchanged (never
false-dropped).
+36
View File
@@ -0,0 +1,36 @@
# fMP4 muxer (`src/mux/fmp4/mod.rs`) — stub status and design notes
## Status
**STUB**. The muxer can emit the init segment (`ftyp` + a minimal HEVC
`moov` skeleton with one video track) via
[`Fmp4Mux::write_init_segment`], so the shape and call site are
validated, but media fragments (`moof`/`mdat`) are NOT emitted.
[`Fmp4Mux::write_video`] therefore returns
[`Error::Fmp4Unimplemented`](crate::error::Error::Fmp4Unimplemented)
rather than silently accepting and discarding frames. It buffers
nothing, so it cannot accumulate memory.
## Not yet implemented
- `moof` box: `mfhd` (sequence_number) + `traf` (`tfhd` + `tfdt`
+ `trun` with sample sizes, durations, flags, composition offsets).
- `mdat` box: concatenated sample data.
- Fragment cadence: one fragment per GOP or every N seconds,
whichever comes first.
- HEVC `hvcC` box inside `moov.trak.mdia.minf.stbl.stsd` so the
init segment is self-describing (`stsd` currently has zero entries).
- Sample-flags computation (sync vs. delta, depends_on, etc.).
- Edit lists / fragment_duration for accurate seeking.
Reference: ISO/IEC 14496-12 §8 (Movie Fragments).
## `wrap_box` size-cast rationale
All callers build tiny init-segment boxes (kilobytes at most), so the
`u32` size never overflows; the saturating cast plus the debug assert
documents and guards that invariant rather than silently emitting a
truncated, structurally corrupt size field. `body` is always
internally constructed here, never untrusted input — a future caller
feeding a multi-gigabyte body trips the debug assert instead of
writing a malformed box.
+70
View File
@@ -0,0 +1,70 @@
# Fox — `dcx.xml` label parser
Older Fox authoring (e.g. a real Fox release, `/BDMV/JAR/05001/dcx.xml`) ships a
human-readable XML manifest alongside the BD-J jar. Its root is `<dcx>`, and
under `<disc>` it lists every playlist the disc plays. The main-feature
playlists carry nested per-stream `<audio>`/`<subtitle>` elements naming
language, editorial purpose and forced/SDH state outright — everything a
label parser wants, in attributes, with no bytecode to walk.
NOT A SPECIFICATION. `/BDMV/JAR/` is application-defined space, so this file
is one authoring house's internal metadata that happens to press onto the
disc. Every field meaning below was read off a real disc; treat an
unfamiliar value as unknown rather than guessing.
## Confirmed schema
A real Fox release, `/BDMV/JAR/05001/dcx.xml`:
```xml
<dcx>
<disc>
<playlist id="00001" lang="eng" name="topmenu"/>
...
<playlist id="00800" lang="eng" name="feature" vers="1" durs="7628">
<audio id="01" lang="eng" type="feature"/>
<audio id="02" lang="eng" type="rnib"/>
<audio id="03" lang="spa" dial="lat" type="feature"/>
...
<subtitle id="01" lang="eng" type="feature" form="sdh"/>
<subtitle id="02" lang="spa" dial="lat" type="embed"/>
...
<subtitle id="11" lang="eng" type="text"/>
<properties> ...chapter marks... </properties>
</playlist>
<playlist id="00801" lang="jpn" name="feature" vers="1" durs="7628"> ... </playlist>
</disc>
</dcx>
```
## Field meanings (read off the disc, not documented)
* `<audio type>`: `feature` = a normal program track; `rnib` = a
descriptive/narration track ("Royal National Institute of Blind People"
described-video), mapped to `LabelPurpose::Descriptive`. A `type`
containing "comment" maps to `LabelPurpose::Commentary`.
* `<subtitle form>`: `sdh` marks a subtitles-for-the-deaf-and-hard-of-
hearing track (`LabelQualifier::Sdh`).
* `<subtitle type>`: `embed` marks a dedicated forced/embedded-subtitle
track (`LabelQualifier::Forced`); `feature`/`text` are full tracks.
* `id` on a nested stream is its 1-based STN slot WITHIN ITS TYPE
(audio ids 01..N, subtitle ids 01..N independently), matching the vendor
`stream_number` convention the ordinal binder in `super::apply_labels`
reads.
The `<audio>`/`<subtitle>` elements are NESTED inside one feature playlist,
so extraction is scoped to a single `<playlist name="feature">` element —
never a document-wide `<audio>` scan, which would merge the regional
`00800` (eng) and `00801` (jpn) tables into one and collide their slots.
## `select_feature_playlist` scoping rationale
Among all `<playlist name="feature">` elements — Fox presses one per
regional variant (`00800` eng, `00801` jpn) — the one carrying the most
`<audio>`/`<subtitle>` streams wins, ties to the first.
Returning ONE element is the whole point: each feature playlist is its own
STN table with its own 1-based slot ids, so merging two would put two
different streams on slot `01`. The richest table is the fullest label set
and anchors to its matching title by language sequence in
`super::apply_labels`.
+13
View File
@@ -0,0 +1,13 @@
# Linux `f_type` classification
`classify_f_type` in `src/platform/fs_type/linux.rs` is the single source of
truth for the magic-number comparisons used by both the path-based
(`detect_impl`) and fd-based (`detect_fd_impl`) entry points.
## Why the `i64` cast
`statfs::f_type` is signed `__fsword_t` on glibc and unsigned `c_ulong` on
musl; a portable comparison needs a common type, so the value is cast to
`i64`. On glibc x86_64 both are already `i64` — clippy flags the cast as
unnecessary on that target only, but the cast is required for musl, hence
the `#[allow(clippy::unnecessary_cast)]`.
+12
View File
@@ -0,0 +1,12 @@
# fs_type module
The buffering architecture (Phase 2) selects different output sinks
for local vs network filesystems: NFS gets the adaptive
`WritebackFile` machinery on Linux; local disks get `LocalFileSink`
and rely on the kernel's default writeback policy. This module
provides the construction-site primitive that picks which one.
Per the per-OS file-split convention, the actual `statfs` call lives
in the matching platform file (`linux.rs`, `macos.rs`, `windows.rs`,
`other.rs`); this `mod.rs` exposes only the cross-platform enum and
the `detect` entry point.
+20
View File
@@ -0,0 +1,20 @@
# `io::fsync` — platform-aware crash-durability primitives
Two flush operations need OS-specific handling to make a write survive a
crash / power loss:
- **`dir`** — fsync a directory so a prior `rename(2)` into it is durable.
After a crash a renamed file's dirent can otherwise be lost even though
the rename returned, because it is still page-cache-only. This is a POSIX
concept: on Windows std cannot even open a directory as a `File` (it does
not set `FILE_FLAG_BACKUP_SEMANTICS`), and NTFS/ReFS commit the rename's
dirent without an explicit directory flush — so it is a no-op there
rather than a failed open that logs on every marker write.
- **`file_durable`** — fsync a file's contents + metadata. Opens the file
**read+write**: on Windows `File::sync_all` maps to `FlushFileBuffers`,
which requires a handle with write access and returns
`ERROR_ACCESS_DENIED` (os error 5) on a read-only handle. (A read-only
`File::open` + `sync_all` is legal on POSIX, which is why that bug only
bit Windows.) The open mode is platform-uniform, so this lives here with
no dispatch.
+37
View File
@@ -0,0 +1,37 @@
# `fvi://` sink notes
Rationale and design notes for `src/mux/fvi_sink.rs` that don't belong in the
public rustdoc contract.
## Relationship to `VideoMap`
The sink is a thin consumer of the reusable, pure-data
[`VideoMap`](crate::mux::videomap) model
([`MapHeader`](crate::mux::videomap::MapHeader) /
[`PictureRecord`](crate::mux::videomap::PictureRecord)): it builds the header
from the title, then writes one record per video
[`PesFrame`](crate::pes::PesFrame) straight to disk. Nothing here re-parses
the elementary stream, and the whole index is never buffered.
Serialization is inlined in the sink itself.
## Extensibility
A different output format would be a DIFFERENT sink reusing the same
`VideoMap` model (e.g. a future `fvi2://`), not a pluggable encoder —
extensibility is by adding a sink, like every other sink in this crate.
## `write_fvi_record`: coding-derived members (§7.1)
Coding-derived members (§7.1) are produced via the codec-agnostic
`PictureInfo` accessors, never the raw bitstream: `field_order` is emitted
only when measured (OMITTED on codec-type-only HEVC/H.264/VC-1),
`progressive` only when signalled.
## `write_fvi_record`: `dts` / future `recovered` member
`dts` is MAY per the format spec — the highway carries no DTS on a frame, so
it is always omitted.
TODO(provenance→recovery join): a `recovered` MAY member (sweep/patch mapfile
overlap with this AU's `src`) belongs here; unreachable at `PesFrame`.
+189
View File
@@ -0,0 +1,189 @@
# H.264 codec parser notes
## Slice-type mapping
`h264_slice_coding_type` maps an H.264 `slice_type` (Rec. ITU-T H.264 §7.4.3,
Table 7-6) to a coding type. Values 5..=9 repeat 0..=4 (the "all slices of
this type" forms), so `slice_type % 5`: 0 = P, 1 = B, 2 = I, 3 = SP
(predicted → P), 4 = SI (intra → I). Returns `None` for values outside 0..=9
(malformed header).
## `unescape_ebsp_prefix`
Copies the leading bytes of an EBSP with emulation-prevention bytes removed
(ISO/IEC 14496-10 §7.3.1: a 0x03 following 0x00 0x00 is not part of the
RBSP). Only a short prefix is produced, because the sole caller decodes two
ue(v) fields from the very start of a slice header. `first_mb_in_slice` and
`slice_type` are each at most 32 bits, so 16 octets is ample and keeps this
off the per-frame allocation path for anything larger.
## `unescape_ebsp`
Copies `ebsp` with emulation-prevention bytes removed, stopping after at most
`max_octets` output bytes (pass `ebsp.len()` for "no cap").
The zero run-length counter is cumulative across the whole scan and is reset
only when an escape byte is actually dropped — matching the reference
decoding process in ITU-T H.264 §7.3.1, which discards a 0x03 following ANY
run of two-or-more 0x00 bytes, not just an exact `00 00 03` sliding window. A
window-based scanner that advances by 3 on a match and by 1 otherwise
disagrees with this on a run of 3+ real zero bytes ahead of an 0x03:
`00 00 00 03` decodes here as `00 00 00` (the run is escaped, matching every
other H.264 decoder), where a window scan starting fresh after each
non-match would keep the 0x03 as real payload. Only one implementation of
this rule may exist in this module — see `parse_sps_high_profile_ext`, which
used to keep its own window-scan copy that disagreed with this one on
exactly that input.
## `handle_param_set`
Handles an SPS/PPS NAL (mirrors the HEVC fix). The strip/emit decision is
made against the currently-ACTIVE set `cur`, NOT the codecPrivate copy
`first`: a streaming MKV decoder applies avcC once at init and thereafter
updates a parameter set only from an in-band NAL, so a switch BACK to the
first-seen body (== codecPrivate) is still a change the decoder must be told
about. Stripping on `== first` silently dropped that revert.
- First of its type → seeds codecPrivate; stripped (decoder gets it from avcC).
- Equal to the active set `cur` → redundant; stripped.
- Different from `cur` (a change in EITHER direction) → emitted in-band and
`cur` updated.
Returns `true` when the NAL was emitted in-band into `frame_data`.
## `reassert_active`
Appends the active parameter set `cur` to `prefix` (length-prefixed) so
every keyframe is SELF-CONTAINED: it carries the active SPS/PPS in-band
ahead of its slices. Skipped only when this access unit ALREADY carried the
NAL in-band (`emitted` — avoids a duplicate) or no active set exists yet.
Unconditional (not only when the active set differs from codecPrivate): a
streaming decoder applies the avcC param sets once at init, then relies on
in-band repetition. Some sources stop repeating a param set at later IDRs
even though its body is unchanged; if the decoder then drops it (a reset
event), nothing re-sends it and every subsequent slice fails (param-set id
out of range) until the next genuine change. Re-asserting at EVERY keyframe
— what compliant muxers do at every IDR — makes streaming decode
self-healing. Re-sending an identical param set is benign; cost is a few
bytes per keyframe. This strictly supersets the change-only re-assert, so
the param-set-revert fix is unaffected.
## `parse_sps_high_profile_ext`
Parses `(chroma_format_idc, bit_depth_luma_minus8, bit_depth_chroma_minus8)`
from a High-Profile SPS NAL (profile_idc ∈ `HIGH_PROFILES` — the 14
chroma/bit-depth extended profiles `codec_private` invokes this for).
SPS RBSP layout (ITU-T H.264 §7.3.2.1.1) up to the fields we need:
```
byte 0 NAL header (already known to be type 7)
byte 1 profile_idc
byte 2 constraint_set_flags / reserved
byte 3 level_idc
ue(v) seq_parameter_set_id
— High-profile branch —
ue(v) chroma_format_idc
if chroma_format_idc == 3: u(1) separate_colour_plane_flag
ue(v) bit_depth_luma_minus8
ue(v) bit_depth_chroma_minus8
```
RBSP emulation-prevention bytes (0x00 0x00 0x03 → 0x00 0x00) are removed
before bit-parsing so the bit reader sees clean RBSP data.
Returns `None` if the SPS is too short or malformed (Exp-Golomb code
overflows 32 bits, leading-zero count > 31, etc.). The caller silently omits
the extension in that case.
## Test notes
### `open_gop_intra_access_unit_is_a_keyframe`
Open-GOP resync fix: a non-IDR intra-coded access unit MUST be flagged a
keyframe so the B1 resync gate (`mux/resync.rs`) can disarm on a BD
open-GOP tail instead of dropping every frame to EOF and truncating the
video. BD H.264 titles reach random access at a non-IDR I-frame (the
recovery point); HEVC gets this free from the CRA NAL type, H.264 does not,
so the measured intra coding type is the signal. Red before the intra
promotion in `parse` — a non-IDR I-slice used to yield `keyframe == false`.
### `inter_coded_access_unit_is_not_a_keyframe`
Guards the promotion's precision: a non-intra (P) access unit must NOT be
promoted to a keyframe, or the resync gate would resume on a frame that
carries inter-references to dropped data and the MKV cue index would gain
bogus random-access points.
### `slice_header_parse_removes_emulation_prevention_bytes`
The slice-header parse must remove emulation-prevention bytes first
(ISO/IEC 14496-10 §7.3.1/§7.4.1): the payload after the NAL header is EBSP,
and a 0x03 following 0x00 0x00 is not part of the RBSP.
Reachable when `first_mb_in_slice` is 65535 or more — a ue(v) that needs
sixteen leading zero bits, so the payload opens 0x00 0x00 and an encoder
must escape it. A UHD frame is ~32,400 macroblocks so conforming Blu-ray
never gets there, but 8K does, and the disc is untrusted input. Reading the
raw NAL decoded `slice_type` against the inserted byte and produced the
wrong picture type.
### `unescape_ebsp_drops_escape_after_a_run_of_three_zeros`
Regression: `parse_sps_high_profile_ext` used to re-derive the
emulation-prevention rule with its own window scanner (match `00 00 03` at
position i, advance by 3; else advance by 1) instead of calling the shared
`unescape_ebsp`. On a run of 3+ real 0x00 bytes ahead of an 0x03 —
non-conformant, but this is untrusted disc input, not a spec-clean encoder —
the two disagreed: `unescape_ebsp`'s cumulative zero counter (the rule the
H.264 reference decoding process in §7.3.1 specifies) drops the 0x03 as an
escape, while a fresh 3-byte window starting right after the non-matching
first byte kept it as real payload. Pin the shared function's behaviour
here so a second hand-rolled copy doesn't quietly reappear.
### `keyframe_param_reassert_emits_exact_bytes`
The keyframe SPS/PPS re-assert must produce EXACTLY these bytes: the active
SPS, then the active PPS, then the access unit's own NALs, each as a 4-byte
big-endian length prefix followed by the NAL body (ISO/IEC 14496-15
length-prefixed form, lengthSizeMinusOne = 3).
This pins the full byte string, not just its length, so the in-place
`splice` that replaced the build-a-new-buffer-and-copy approach is proven
byte-for-byte equivalent — including the ORDER (parameter sets ahead of the
slices, which is what makes the keyframe self-contained). The literals were
captured from the pre-splice implementation's output.
### `keyframe_param_reassert_does_not_reallocate_the_frame`
MEASURED: the keyframe parameter-set re-assert must be spliced into the
front of the already-assembled access unit IN PLACE, not built as a fresh
full-size buffer.
It used to `prefix.extend_from_slice(&frame_data)` and then replace
`frame_data` with `prefix`, which grew a few-hundred-byte `prefix` to the
FULL access-unit size — a fresh multi-hundred-KB allocation — memcpy'd the
whole frame into it, and dropped the presized buffer. One extra whole-frame
allocation plus one extra whole-frame copy per keyframe: a UHD title is
~200,000 frames of 150-400 KB with a keyframe every 1-2 s, i.e. thousands of
avoidable multi-hundred-KB copies per title, each large enough to go through
mmap. `PARAM_REASSERT_HEADROOM` exists so the splice never reallocates; this
counts the reallocations that happen, which must be zero. Mirrors the HEVC
parser's test of the same name.
### `avcc_profile_244_appends_extension_bytes`
ISO 14496-15 §5.3.3.1.2 regression: profile_idc=244 (High 4:4:4 Predictive)
ALSO mandates the chroma/bit-depth extension. It was missing from
HIGH_PROFILES, so a 244 stream took the Baseline/Main path and emitted an
avcC with NO extension bytes — non-conforming, and strict parsers then
assume 8-bit 4:2:0. The extension must be appended.
### `sps_reader_read_ue_thirty_one_leading_zeros_is_still_valid`
`SpsReader::read_ue`'s truncation guard is `leading_zeros > 31`: 31 leading
zero bits is the longest legal code and must decode, not abort. Mirrors the
equivalent guard in the shared `BitReader` in `startcode.rs`, kept here
because `SpsReader` is a separate, unshared implementation used only for SPS
parsing.
+16
View File
@@ -0,0 +1,16 @@
# halt
## Why `Ordering::Relaxed`
`Ordering::Relaxed` is sufficient on both load and store because this flag
is purely advisory — no other memory operations piggyback on it for
happens-before ordering. Callers that need to publish data across threads
do so via channels or other synchronization, not via this bit.
## Why 250ms for `POLL_INTERVAL`
250 ms is the sweet spot between responsiveness (operator presses Stop,
sees it take effect within ~quarter-second) and waste (atomic load + clock
read is cheap but not free at thousands of hertz). It's centralised here
so the half-dozen halt-polling loops across `io` can't drift apart
silently.
+35
View File
@@ -0,0 +1,35 @@
# Seeded robustness harness (`src/harness.rs`)
Every parser reached from here takes bytes that came off a disc, and this
crate's primary boundary is that the disc is untrusted: a malformed, damaged
or hostile image must never crash the library. These tests assert exactly
that one property — the parser returns `Ok` or `Err`, and never panics.
## Why this exists rather than `cargo-fuzz`
`cargo-fuzz` needs a nightly toolchain (`-Zsanitizer` plus SanitizerCoverage
for libFuzzer's coverage feedback) and this project pins stable. So the
generator lives here instead. It gives up coverage-guided mutation — the real
loss — and keeps everything else: millions of cases, structure-aware input,
and a crash corpus. It also gains determinism, which a fuzzer does not have:
the same seed replays the same cases on any machine.
## Why no `proptest` or `arbitrary`
This crate has exactly one dev-dependency. That is a deliberate posture, and
a randomness crate is not worth ten transitive dependencies when the parsers
take plain `&[u8]` and a good enough generator is forty lines.
## Budget
`FREEMKV_HARNESS_CASES` sets cases per generator per target (default 256, low
enough that the per-commit gate stays under a second). The overnight run sets
it to millions. `FREEMKV_HARNESS_SEED` overrides the seed; the default is
fixed so a failure in CI reproduces locally verbatim.
## On failure
The panic message carries the seed, generator and case index. Re-run with
`FREEMKV_HARNESS_SEED=<seed>` to reproduce, then write the offending bytes
into `tests/corpus/` as a permanent regression fixture — discovery happens
here, defence happens there.
+465
View File
@@ -0,0 +1,465 @@
# HD-DVD title scanning (`src/disc/hddvd.rs`)
HD-DVD is a **tree-level peer** of DVD and Blu-ray (not a stream variant like
FMTS): its content lives in `HVDVD_TS/` as `.evo` clips — Enhanced VOB, an
MPEG **program** stream — each with a small `.map` timemap sidecar, navigated
by `.xpl`/`.ifo` playlists in `HVDVD_TS/` and `ADV_OBJ/`. Because it is a
different tree with a different playlist format, it gets its OWN scanner
(this file), a peer to `Disc::scan_bluray_titles` — the two-format design
rule: a genuinely different format is a new enumerator, not an extension
bolted into the BD path.
## Title composition (authoritative path)
Title composition is authoritative, from the Advanced-Content playlist. HD-DVD
ships a real player playlist at `ADV_OBJ/VPLST000.XPL` (DVD-Forum
`HDDVDVideo/Playlist` XML). The scanner parses it (with a real XML parser,
`roxmltree`) into one `DiscTitle` per `<Title>`: its `<PrimaryAudioVideoClip>`
clips in playback order (each an EVO, referenced via its `.MAP` sidecar), the
`titleDuration`, the `displayName`, and the `<ChapterList>`. A layer-break
split (`FEATURE_1` + `FEATURE_2`, or `feature`/`feature_Divide`) is composed
into ONE title with the two parts as clips, carrying each clip's title-time
in/out points (45 kHz ticks) so a seamless join can be spliced onto one
timeline. Container is `ContentFormat::MpegPs`, so the existing PS mux path
handles it. Per-clip streams are enumerated by demuxing the clip head and
building one `Stream` per distinct elementary stream (video + DD+ audio),
codec sniffed from the ES bytes.
## Fallback path
When no playlist is present (or it fails to parse), the scanner falls back to
the older clip-name heuristic: parse the `HVA*.VTI` clip table, join the
`feature*`-named clips into one title, and emit every other clip on its own.
## Known gaps
Not parsed yet: subtitles (8-bit RLC on `0xBD` sub `0x20..=0x3F`) and per-track
audio languages (the XPL carries `<Audio description=...>` but they are not yet
wired onto the streams). Extents and size are real (the ripper images the
clips).
## VTI parsing
`parse_vti_clip_order` parses the `ADVANCED-VTS` VTI clip-name table. The
table is a run of `VTI_CLIP_ENTRY_STRIDE`-spaced records, each carrying a
NUL-terminated `<name>.EVO`. Rather than trust the (imprecise) header pointer,
it collects every NUL-terminated `*.EVO` name and keeps the largest group
sharing one residue modulo the stride — the clip table — in offset order.
Returns empty for a non-VTI blob or one with no recognizable table.
## Codec sniffing
`sniff_video_codec` reads MPEG / Annex-B start codes: `00 00 01 B3` → MPEG-2
(sequence_header), `00 00 01 0F` → VC-1 (BD/HD-DVD sequence-header BDU),
`00 00 01 [x7]` H.264 SPS NAL (type 7, forbidden_zero_bit clear) → H.264.
Returns `None` when no recognizable start code is present. The scan prefers
the unambiguous MPEG-2 / VC-1 sequence headers; H.264 is inferred from an SPS
NAL so a stray slice/picture code can't be mistaken for a different codec.
`sniff_audio_codec` recognizes only Dolby Digital Plus (E-AC-3) today, via its
`0x0B77` syncword — what real retail HD-DVD titles carry on sub-ids
`0xC0..=0xC7`. Returns `None` for an unrecognized sample so the caller drops
the stream rather than mislabeling it.
## probe_evo_streams
Demuxes the head of an `.evo` clip (through the disc's `SectorSource`) and
builds one `Stream` per distinct elementary stream found: the video track
(mapped to the canonical `DVD_VIDEO_PID` for plain 0xE0-0xEF video, or the
extended-stream-id PID for HD-DVD VC-1) and every DD+ audio sub-stream
(mapped via `dvd_audio_pid`). Codec is sniffed from the demuxed ES bytes.
Mirrors the stream construction in `Disc::scan_dvd_titles`; resolution /
language / channels use sane HD-DVD defaults (the muxer reads the true pixel
dimensions from the H.264 SPS, and E-AC-3 channel counts are not decoded
here). Returns an empty vec when the clip cannot be read or carries no
recognizable stream (e.g. an AACS-encrypted clip probed as ciphertext).
Cancellation is checked before every chunk read and is the ONE condition that
returns `Err` rather than an empty vec: a probe cut short by a Stop has not
established that the clip carries no streams, and reporting it as if it had
would enumerate a stream-less title as a scanned fact. A read that fails for
any other reason keeps the existing best-effort behaviour.
## collect_es
VC-1 video rides extended-stream-id `0xFD` with `stream_id_extension` `0x55`.
HD audio (MLP/TrueHD) can also use `0xFD` with other extensions — routing
those to their own audio tracks is deferred (see the HD-DVD program-chain
follow-up); until then only the VC-1 extension is treated as video, so an
audio `0xFD` sub-stream can never mis-stamp the video PID.
## XPL amplification bounds
The XPL parser guards against a crafted `ADV_OBJ/VPLST000.XPL` amplifying a
small file into an enormous amount of work or memory. Four separate axes are
bounded, each independently, because they are separate attacks:
- **`MAX_XPL_DEPTH` (32)** bounds element NESTING. A real `VPLST000.XPL` is
about six levels deep at its deepest
(`Playlist`/`TitleSet`/`Title`/`PrimaryAudioVideoClip`/`Video`, or
`.../ChapterList/Chapter`); 32 leaves a 5x margin for authoring tools that
wrap extra `ApplicationSegment`/`ObjectMappingList` layers, while staying
trivially stack-safe.
- **`MAX_XPL_TITLES` (512)** bounds how many `<Title>` elements one playlist
may declare — BREADTH, separate from the depth guard: a flat playlist
passes the depth check trivially and can still declare a title per 51
bytes, so a 64 MiB `VPLST000.XPL` (exactly `udf::MAX_FILE_BYTES`) reaches
~1.3 million of them. This matters here and not on the Blu-ray side because
of where the count comes from: `bluray.rs` learns its playlists from the
MPLS files actually present in the UDF directory, so the medium bounds it —
an XPL DECLARES its titles, so nothing does. The cost is drive time, not
memory: `compose_xpl_titles` runs `probe_evo_streams` once per title, and
each probe reads real sectors, so an uncapped count turns a `Disc::scan()`
the operator expects to take seconds into tens of terabytes of optical
reads on a disc that is only pretending to be large. The bound that matters
is `cap x EVO_PROBE_SECTORS`, since `compose_xpl_titles` memoizes on the
title's RESOLVED extent list, and distinct titles built from different
combinations of the same clips have different lists, so each one misses the
memo — at 4096 that product was 64 GiB of optical reads, the same worst
case, and the same contradiction with "seconds", as the sibling
`MAX_HDDVD_CLIPS` path. 512 keeps it at 8 GiB and remains far above any
real disc (retail HD-DVDs carry tens of titles).
- **`MAX_HDDVD_CLIPS` (512)** caps `.evo` clips taken from the `HVDVD_TS/`
directory listing. `MAX_XPL_TITLES` and `xpl_depth_within_limit` bound the
`ADV_OBJ/*.XPL` path, but a crafted disc simply omits `/ADV_OBJ`; control
then reaches the clip-name fallback in `Disc::scan_hddvd_titles`, which
emits one title per directory entry and probes each — the same unbounded-
probe cost, reached without the playlist. The directory is the only bound
the medium supplies, and `udf::MAX_DIR_BYTES` (1 MiB) still leaves room for
~24,000 FIDs. Memoizing the probe (`EvoProbeCache`) collapses entries that
resolve to the SAME extents, closing the many-names-one-File-Entry case —
but not this one: a File Entry is a single sector, so an attacker can give
every FID its own, each declaring its own extent, for ~48 MiB of image;
every probe then misses the memo and costs a full `EVO_PROBE_SECTORS`
(16 MiB) read, hundreds of GiB on a scan the operator expects to take
seconds. The cap's job is to bound `cap x EVO_PROBE_SECTORS`; it was 4096
(matching `MAX_XPL_TITLES`, which bounds a different, no-per-item-read-cost
path) — 4096 x 16 MiB is 64 GiB, more than dual-layer HD-DVD media
physically holds (reachable anyway, since distinct extent lists may overlap
on the medium). 512 keeps the product at 8 GiB and still leaves a wide
margin over any real disc (retail HD-DVDs carry TENS of `.evo` clips, ~10x
the real-world maximum, the sizing convention `MAX_XPL_CHAPTERS_PER_TITLE`
also uses against the 99-chapter authoring ceiling). Pinned by
`the_clip_cap_and_probe_budget_bound_a_scans_worst_case_read_volume`, which
asserts the PRODUCT rather than either constant, so raising one without
re-examining the other fails.
- **`MAX_XPL_CLIPS_PER_TITLE` (256)** bounds how many
`<PrimaryAudioVideoClip>` elements a SINGLE `<Title>` may contribute — the
fourth amplification axis, the one the other three guards leave open. A
64 MiB XPL holds ~1.7 million `<PrimaryAudioVideoClip src="A.EVO"/>`
elements at ~38 bytes each, and the collector is `descendants()`, not
`children()`, so a clip nested inside N ancestor `<Title>` elements is
collected by EVERY one of them — the depth cap still permits 32 such
ancestors (measured: a bare `<Title>` root nests 32 deep and still parses,
because the guard counts the self-closing clip element as non-nesting).
The product is ~1.7M x 32 ≈ 56 million heap-allocated `XplClip`s, each with
its own `String`, and then as many `Clip`s again in `compose_xpl_titles`.
Unlike the title cap, the cost here is MEMORY rather than drive time:
`compose_xpl_titles` de-duplicates by `.evo` name before probing
(`seen_evos`), so repeats cost no extra sector reads, but the `Vec`s are
built and held regardless. 256 is far above any real disc — a feature is
one clip, or two across a layer break; even seamless-branching/multi-angle
is tens — and keeps the aggregate bounded at `MAX_XPL_TITLES` x 256 clips
rather than the tens of millions above.
- **`MAX_XPL_CHAPTERS_PER_TITLE` (1024)** bounds `<Chapter>` elements per
`<Title>` — the same `descendants()` amplification, on the second unbounded
`collect()` in the same loop, bounded separately so capping clips doesn't
simply move the attack next door. A chapter is a bare `f64` rather than a
`String`-carrying struct so it's cheaper, but a
`<Chapter titleTimeBegin="…"/>` element is also smaller, so the element
count an XPL can reach is comparable. 1024 is far above any real disc: the
DVD/HD-DVD authoring convention tops out at 99 chapters per title, a 10x
margin.
## scan_hddvd_titles
Scans HD-DVD titles from the `HVDVD_TS/` `.evo` clips. The main feature is
authored as one or more `.evo` clips (a layer-break split —
`FEATURE_1`/`FEATURE_2` or `feature`/`feature_Divide`). The `HVA*.VTI`
navigation file names every clip in authored order; this parses it to
concatenate the feature clips into ONE title (so the largest-title pick gets
the whole movie, not just part 1), emitting every other clip as its own
title. Falls back to one title per clip when the VTI is absent or
unparseable, so a disc with no readable navigation still enumerates.
`chapters`/duration are left empty pending deeper VTI parsing.
Cancellation: `halt` is polled once per clip and again before every probe
chunk, and a read that fails with `Error::Halted` — how a live drive reports
a Stop, since `Drive::checked_exec` fails every command once its flag is set
— is propagated rather than swallowed. `Halted` is the only error this
returns; every other read failure keeps its best-effort behaviour. It has to
be an error and not a short title list: a cancelled enumeration that
returned `Ok` would be indistinguishable from a disc that genuinely holds
fewer titles, and the caller would cache and act on it.
## xpl_depth_within_limit
Rejects a disc-supplied XML blob whose element nesting exceeds
`MAX_XPL_DEPTH`, BEFORE it reaches the parser. `roxmltree` is
recursive-descent and its depth-10 limit applies only to ENTITY expansion, so
element nesting is unbounded: a few hundred KB of well-formed XML (far under
the read cap) overflows the thread stack. That is a process ABORT, not an
`Err` and not an unwind, so neither the `Document::parse` fallback nor
`catch_unwind` can contain it; the only fix is to not hand the document over
at all. A byte-size cap is deliberately NOT added: file size is already
bounded by the UDF read path, and a size cap does not close this class (a
180 KB document already aborts) — depth is the bound that matters.
This is a conservative scan, not a validator: it tracks `<name …>` /
`</name>` while skipping comments, CDATA, processing instructions and
declarations, and treats `<… />` as non-nesting. A miscount can only cost a
pathological document its fast path, and the failure mode is the existing
one — fall back to the clip-name heuristic.
## Test rationale
Detailed rationale for tests in `src/disc/hddvd.rs`'s `#[cfg(test)]` module,
kept here so the in-code doc can stay within the internal-comment cap.
**`scan_hddvd_feature_composition_saturates_absurd_disc_declared_sizes`** —
The SECOND size accumulator. `compose_xpl_titles` sums an XPL's clip sizes;
this one sums the composed FEATURE title's parts, and it reads the same
disc-declared `u64` Information Length from the same `clip_extents` map. A
split feature whose two parts each declare a size near `u64::MAX` overflows
it exactly as the XPL path did. Nothing reconciles the declared size against
the extents that back it — here each part declares `u64::MAX` while
occupying ten sectors — because the allocation descriptor's length field is
only 32 bits wide, so the disc can always claim more than it holds.
Mutation: restore `size_bytes += *size` in the feature-composition path and
this goes red (debug: attempt to add with overflow).
**`scan_hddvd_does_not_compose_a_feature_over_an_unrecorded_part`** — A split
feature (FEATURE_1 + FEATURE_2) where ONE part carries an unrecorded extent
must not be composed into a FEATURE title at all. Composing around the
unusable part is the dangerous outcome: the title would play through as if
it were the whole movie while silently missing that part's runtime, and its
size still counts it. The other clips remain available as their own titles
— refusing the composition is not the same as refusing the disc.
**`scan_hddvd_logs_an_unrecorded_feature_part_with_its_own_code`** — The
refusal above must also be ACCOUNTED, with the unrecorded extent's own code.
This warn site carried a hardcoded `code = 6017` literal — in the very file
whose sibling arm was changed to stop doing exactly that — and nothing
tested it, so neither the literal nor its absence broke anything. Refusing a
feature and saying nothing leaves the operator with a disc that scanned
"fine" and is quietly missing its main title. Mutations: deleting the
`tracing::warn!` (no event to find); restoring the literal `6017` still
passes here BY VALUE, which is the point — what is pinned is
`E_UDF_UNRECORDED_EXTENT`'s value reaching the log, so a renumbering that
the literal could not follow goes red; logging a neighbouring code (e.g.
`E_UDF_NO_USABLE_EXTENT`) goes red immediately.
**`scan_hddvd_does_not_compose_a_feature_over_a_part_with_no_usable_extent`**
— The `Ok`-but-empty twin of the test above, and the one that actually bites
on an ordinary disc: a feature part whose `file_extents` call SUCCEEDS and
yields nothing usable — here a zero-byte `FEATURE_2.EVO`, no crafted ICB
required — must refuse the composition too, and must say so in the log.
Before this, the empty-`Ok` clip entered neither `clip_extents` nor
`unusable`, so the `any(|n| unusable.contains(..))` guard missed it and the
`filter(|n| clip_extents.contains_key(..))` beside it quietly DELETED the
part: the scan composed a `FEATURE` title out of part one alone, still
advertising itself as the feature, at rc=0, with no diagnostic anywhere.
Half a movie presented as a whole one — strictly worse than the `Err` case
the guard was built for, and reachable without a hostile disc. Mutations
this catches, all of which the pre-fix code exhibited: dropping the
`extents.is_empty()` branch entirely (a `FEATURE` title appears with one
extent); inserting into `clip_extents` anyway (same); marking the clip
unusable but NOT logging (the log assertion fails — absence of a log is
itself the defect); logging the neighbouring `E_UDF_UNRECORDED_EXTENT`
instead of `E_UDF_NO_USABLE_EXTENT` (the code assertion fails, because a
zero-length file is not an authoring hole and triaging the two together
sends whoever reads it at the wrong population).
**`parse_xpl_titles_caps_a_playlist_declaring_absurdly_many_titles`** — A
disc-supplied playlist nested far deeper than any real one must be REFUSED
BEFORE the XML parser sees it. `roxmltree` is recursive-descent and its
depth-10 limit covers only ENTITY expansion, so unbounded element nesting
blows the thread stack — an ABORT, which no `catch_unwind` and no `Err`
fallback can contain. ~50k levels is a few hundred KB of well-formed XML,
far under the read cap. The correct outcome is the same as any other
unusable playlist: an empty `Vec`, so the caller falls back to the clip-name
heuristic. Separately, this test also exercises BREADTH, not depth: a flat
playlist passes the nesting guard trivially and can still declare a title
per ~51 bytes, so a 64 MiB XPL (exactly `udf::MAX_FILE_BYTES`) reaches ~1.3
million of them. The cost is drive time, not memory: `compose_xpl_titles`
runs `probe_evo_streams` once per title and each probe reads real sectors,
so an uncapped count turns a scan the operator expects to take seconds into
tens of terabytes of optical reads. `bluray.rs` is not exposed to this
because its playlist count comes from the MPLS files present in the UDF
directory; an XPL DECLARES its titles, so nothing bounds them. Mutation:
delete the `.take(MAX_XPL_TITLES)` and this goes red at the declared count.
**`nested_title_xpl`** — Builds a playlist that nests `nesting` `<Title>`
elements inside one another and puts `clips` self-closing
`<PrimaryAudioVideoClip>` elements at the innermost level, plus `chapters`
`<Chapter>` elements. `parse_xpl_titles` collects a title's clips with
`descendants()`, not `children()`, so EVERY one of the `nesting` ancestors
collects the SAME innermost clip list — the amplification this exercises.
**`parse_xpl_titles_caps_clips_per_title_against_descendant_amplification`**
— CLIPS PER TITLE is the axis the title cap, the depth cap and the
directory cap all leave open. `MAX_XPL_TITLES` bounds how many titles a
playlist may declare and `MAX_XPL_DEPTH` bounds how deeply it may nest, but
NEITHER bounds how many `<PrimaryAudioVideoClip>` elements a single title
collects. Worse, the collector is `descendants()`, so one clip nested inside
N ancestor `<Title>`s is collected N times over — the clip count MULTIPLIES
by the nesting the depth guard still permits. A 64 MiB XPL
(`udf::MAX_FILE_BYTES`) holds ~1.7M clip elements at ~38 bytes each; with
the 32 `<Title>` ancestors the depth cap still permits, that is ~56 million
heap-allocated `XplClip`s, and then as many `Clip`s again in
`compose_xpl_titles`. This test uses small numbers that exercise the same
multiplication without a 64 MiB fixture: pre-fix it collected 756 clips per
title, 18,900 across the 25 nested titles, from a document declaring only
756 clip elements — an exact 25x, the nesting count. Mutation: delete the
`clips.len() >= MAX_XPL_CLIPS_PER_TITLE` break and this goes red at the
unamplified per-title count.
**`parse_xpl_titles_caps_chapters_per_title`** — CHAPTERS PER TITLE is the
same `descendants()` amplification on the second unbounded `collect()` in
the same loop. Bounded separately so closing the clip axis does not simply
move the attack next door. Mutation: delete the
`.take(MAX_XPL_CHAPTERS_PER_TITLE)` and this goes red at the declared count.
**`parse_xpl_titles_keeps_every_clip_of_a_realistic_title`** — The control: a
realistic multi-clip title still resolves EVERY one of its clips and
chapters. `SYNTH_XPL`'s main movie is a layer-break split — two clips on one
timeline — plus two chapters; losing either to the cap would cost a genuine
disc half its feature. Mutation: set `MAX_XPL_CLIPS_PER_TITLE` to 1 and this
goes red.
**`parse_xpl_titles_accepts_real_world_nesting_depth`** — The depth guard
must not reject real discs. A genuine `VPLST000.XPL` is a handful of levels
deep; this is the same document the parsing test uses, plus self-closing
tags, a comment and a processing instruction that the pre-parse scanner has
to account for without inflating its depth count.
**`scan_hddvd_composes_titles_from_xpl_playlist`** — The authoritative path:
when a VPLST000.XPL is present, titles come from the playlist — the
layer-break split (FEATURE_1 + FEATURE_2) is composed into ONE title with
the real duration, name, chapters, and per-clip title-time offsets — not the
clip-name heuristic.
**`compose_xpl_titles_saturates_absurd_disc_declared_clip_sizes`** — A
title's `size_bytes` is a running sum of DISC-DECLARED clip sizes: the `u64`
UDF File Entry Information Length, which nothing cross-checks against the
extent list that file actually occupies. Two clips can therefore each
declare a size near `u64::MAX`. Summed with a plain `+=`, that overflows: a
panic in a debug build — a crash on untrusted disc content, which this
library must never do — and a wrap to a small garbage total in release,
which misreports the title's size while looking entirely normal. Mutation:
restore `size_bytes += *size` in `compose_xpl_titles` and this goes red
(debug: attempt to add with overflow).
**`compose_xpl_titles_dedups_repeated_clip_references_by_evo`** — A crafted
Advanced-Content playlist can name the SAME `.evo` in
`<PrimaryAudioVideoClip>` many times over (the XML has no fixed element
count, unlike MPLS's binary PlayItem count). Each repeat must NOT push
another copy of that clip's extent list onto the title — mirroring
bluray.rs's `first_ref = seen_clips.insert(...)` gate, which pushes a clip's
extents only the first time its id is seen. Here the analogous key is the
`.evo` filename (this file has no clip_id; `.evo` is what `clip_extents` is
keyed by). Without the gate, extents grows by the referenced clip's full
extent list on EVERY repetition — unbounded by the number of on-disc files,
bounded only by playlist size.
**`scan_hddvd_titles_drops_a_split_feature_whose_part_cannot_be_read`** — A
clip whose extents cannot be resolved must drop the SPLIT FEATURE. Catches
reverting the `Err(e) => { warn; unusable.insert(..) }` arm in
`Disc::scan_hddvd_titles` back to the bare `Err(_) => {}` it replaced. RED
BEFORE GREEN, and TWO earlier attempts at this test did NOT go red. The
first asserted "no title names the broken clip" — that passes either way,
because a clip that resolves to no extents is never inserted into
`clip_extents` and so yields no per-clip title regardless. The second fixed
that but shipped no `.vti`: `order` is built solely from
`parse_vti_clip_order` of the navigation file, so with no `.vti` it is
EMPTY, `feature` is empty, and no composed title is ever built — the
assertion held vacuously with the fix reverted. Hence the synthetic
`HVA00001.VTI` fixture: it is what makes the composer run at all. The
defect lives one level up from the per-clip titles, in the composed
feature: `unusable` is what tells the composer that a part is MISSING
rather than merely absent, and the old bare `Err(_) => {}` populated it for
nothing but an unrecorded extent. So a scratched sector under FEATURE_2's
ICB (`Error::DiscRead`) left FEATURE_1 composing a title named "FEATURE" by
itself — half a movie offered as the whole one.
**`scan_hddvd_titles_excludes_a_clip_with_zero_sectors`** — A clip whose
file has a zero-byte size (a degenerate/empty allocation: its ICB's
allocation descriptor has `data_len == 0`, the UDF AD-list terminator, so
`file_extents` yields no extent at all) must not produce a title. NOTE: this
exercises the *upstream* `data_len == 0` terminator path in
`crate::udf::UdfFs::file_extents`, not the `sectors > 0 && lba > 0` guard in
`scan_hddvd_titles` itself — with this fixture (`file_extents` never returns
a `(lba, 0)` tuple, and `PART_START` unconditionally makes every resolved
`lba` positive) that guard is unreachable in a divergent way; kept here as a
regression check on the zero-byte-file behavior in its own right.
**`scan_hddvd_titles_probes_once_for_entries_resolving_to_identical_extents`**
— THE THIRD AMPLIFICATION AXIS on HD-DVD title composition, past both
existing guards. `xpl_depth_within_limit` bounds XML NESTING;
`MAX_XPL_TITLES` + the `.evo` de-dup bound what an XPL can DECLARE. Both
live on the playlist path — and an attacker simply omits `/ADV_OBJ`, so
`read_adv_obj_xpl` returns `None` and neither guard is ever consulted.
Control then reaches the clip-name FALLBACK, which emits one title per
`.evo` DIRECTORY ENTRY and probes each. Nothing de-duplicates a FID's ICB
LBA, so every name in a 1 MiB directory (`udf::MAX_DIR_BYTES`, ~24,000 FIDs)
can point at ONE File Entry. Each resolves to the SAME extents and each
costs a full `EVO_PROBE_SECTORS` (16 MiB) probe: ~375 GiB of optical reads
from a directory that describes a single file. Memoizing the probe on the
resolved extent list collapses that to ONE probe, because all of those
names resolve to one extent list. Mutation: drop the memo lookup in
`EvoProbeCache::streams` (always probe) and this goes red at `ENTRIES`
probes.
**`the_clip_cap_and_probe_budget_bound_a_scans_worst_case_read_volume`** —
The clip cap and the per-probe read budget are ONE bound, not two, and only
their product is meaningful: `MAX_HDDVD_CLIPS` probes each cost up to
`EVO_PROBE_SECTORS`, so the pair fixes the worst-case drive time of a
`Disc::scan()`. `EvoProbeCache` does not help against a crafted disc here:
it keys on the RESOLVED extent list, and distinct-but-overlapping extent
lists all reading one physical region miss the memo every time while every
read succeeds. So the product is genuinely reachable, and it is the number
that has to stay within what a scan can absorb. 8 GiB is the ceiling
asserted here. It is far above any genuine disc — retail HD-DVDs carry tens
of `.evo` clips, i.e. well under 1 GiB of probing — and far below the 64 GiB
the pair used to permit, which exceeded the capacity of the dual-layer media
this format tops out at and took tens of minutes of optical reads. This is
a relationship, not a magic number: raising EITHER constant without
re-examining the other trips it.
**`scan_hddvd_titles_still_probes_each_distinct_clip`** — CONTROL:
memoization must not silently collapse DISTINCT clips. A legitimate disc
with several different `.evo` files still probes each one, and each title
keeps the streams of ITS OWN clip. Mutation: make `EvoProbeCache`'s key a
constant (ignore the extents) and this goes red — the junk clip inherits
the first clip's H.264 streams.
**`scan_hddvd_titles_caps_a_directory_declaring_absurdly_many_clips`** —
Memoization alone is NOT the whole fix. Collapsing identical extent lists
bounds the one-File-Entry attack, but distinct extent lists are only as
bounded as the File Entries an attacker cares to lay down — and a File
Entry is ONE sector. A 1 MiB directory's ~24,000 FIDs can each point at
their own 1-sector File Entry (a ~48 MiB image) declaring its own huge
extent, so every probe misses the memo and the amplification is back.
`MAX_HDDVD_CLIPS` is the bound that closes that. Mutation: drop the
`clips.len() < MAX_HDDVD_CLIPS` gate in the directory scan and this goes
red at `ENTRIES`.
## EvoProbeCache
`EvoProbeCache` memoizes `probe_evo_streams`, keyed on the RESOLVED extent
list. Probing is the expensive half of title composition: each pass reads up
to `EVO_PROBE_SECTORS` (16 MiB) off the medium. Both composition paths can
reach the same physical clip many times over — an XPL naming one `.evo` from
many `<Title>`s, or a directory whose FIDs all point at one File Entry — and
a probe is a pure function of the extents it reads, so the second and later
passes over an identical extent list are re-reads of bytes already seen. The
key is the extent list itself, NOT the clip name or title: two names for one
File Entry share a key (one probe), while two genuinely different clips have
different `start_lba`s and so keep their own probes and streams. Cache size
is bounded by the caller's title/clip cap (`MAX_XPL_TITLES` /
`MAX_HDDVD_CLIPS`).
+99
View File
@@ -0,0 +1,99 @@
# HEVC Annex B muxer (`src/mux/hevc/mod.rs`) — extended notes
Overflow detail for comments capped by `ci/comment-guard.py`. Each section is
pointed to from a short `//` comment at the named item.
## Module overview
Writes a raw `.hevc` / `.h265` Annex B byte stream:
`00 00 00 01 | NAL_unit | 00 00 00 01 | NAL_unit | …` with no container
framing. On the first frame the muxer emits the codec_private's VPS, SPS, PPS
(parsed from a `HEVCDecoderConfigurationRecord` in `length-prefixed-in-hvcC`
form), then converts each PES frame's length-prefixed NAL units to Annex B
and writes them.
## `write_frame`
Input may be either:
- Length-prefixed: `[u32-BE len][NAL bytes]` repeated. This is the form
emitted by libfreemkv's HEVC parser (the MKV-native layout). Converted to
Annex B.
- Already Annex B: a buffer beginning with a `00 00 00 01` or `00 00 01`
start code. Passed through unchanged.
## `hvcc_to_annex_b`
Layout (per ISO/IEC 14496-15 §8.3.3.1.2):
- 22-byte fixed header
- byte 22 = `numOfArrays`
- each array: `array_completeness:1 | reserved:1 | NAL_unit_type:6`,
`numNalus:u16-BE`, then `numNalus` × `(nalUnitLength:u16-BE + NAL bytes)`.
We don't filter on NAL type — VPS (32), SPS (33), PPS (34), and any SEI
arrays included in hvcC all get the same Annex B treatment.
## `length_prefixed_to_annex_b`
Already-Annex-B input (a buffer beginning with a `00 00 00 01` or `00 00 01`
start code) is detected up front and passed through unchanged — some
upstream paths (raw HEVC ES from disc) hand Annex B straight through the
PES layer, and a genuine start code would otherwise be misread as a u32-BE
length prefix.
Truncation policy (single source of truth across all muxers): if a length
prefix runs past the end of the buffer (e.g. a NAL truncated by a bad disc
sector), the truncated trailing NAL is dropped and only the valid Annex-B
prefix accumulated so far is emitted. We never emit a half-NAL nor leak raw
length-prefixed bytes into the Annex-B stream.
## `nal_length_size`
Number of octets each NAL length prefix occupies in the elementary data of a
track described by `record` — the `lengthSizeMinusOne + 1` field of the
decoder configuration record (ISO/IEC 14496-15).
- avcC (`AVCDecoderConfigurationRecord`, §5.3.3.1.2): byte 4 is
`bit(6) reserved | unsigned int(2) lengthSizeMinusOne`.
- hvcC (`HEVCDecoderConfigurationRecord`, §8.3.3.1.2): byte 21 is
`constantFrameRate(2) | numTemporalLayers(3) | temporalIdNested(1) |
lengthSizeMinusOne(2)`.
The spec permits only 1, 2 or 4 octets (`lengthSizeMinusOne` of 0, 1 or 3);
a declared 3 is non-conformant but is decoded rather than rejected, since
reading N octets is the same operation for every N. A record too short to
carry the field, or a codec with no such record, falls back to
`DEFAULT_NAL_LENGTH_SIZE` — the width every freemkv parser emits.
This exists because assuming 4 is silent corruption for a legal source:
reading a 2-octet-prefixed frame as u32-BE yields an absurd first length,
the conversion loop bails with nothing parsed, and the raw length-prefixed
bytes are passed through as though they were already Annex B — a stream
with no start codes at all, and no error anywhere.
## `append_length_prefixed_as_annex_b`
Same conversion as `length_prefixed_to_annex_b` but writes directly into a
caller-owned buffer, avoiding an intermediate allocation on hot paths (e.g.
per-frame video muxing). If `data` doesn't parse as length-prefixed (no
NALs extracted), it's appended unchanged on the assumption it's already
Annex B.
## `append_length_prefixed_as_annex_b_sized`
For a source whose NAL length prefixes are `length_size` octets wide rather
than the 4 this crate's own parsers emit. Derive `length_size` from the
track's configuration record with `nal_length_size` — ISO/IEC 14496-15 lets
a legal avcC/hvcC declare 1 or 2 octet prefixes, and reading those as
u32-BE mangles the frame.
`length_size` outside `1..=4` is clamped to `DEFAULT_NAL_LENGTH_SIZE`; the
field it comes from is 2 bits wide, so that is unreachable from real input.
## `avcc_to_annex_b`
Layout (per ISO/IEC 14496-15 §5.3.3.1.2):
- 5-byte fixed header
- byte 5 = `[reserved:3 | numOfSequenceParameterSets:5]`
- `numOfSPS` × `(sequenceParameterSetLength:u16-BE + SPS bytes)`
- 1 byte = `numOfPictureParameterSets`
- `numOfPPS` × `(pictureParameterSetLength:u16-BE + PPS bytes)`
+265
View File
@@ -0,0 +1,265 @@
# HEVC parser (`src/mux/codec/hevc.rs`) — extended notes
Overflow detail for comments capped by `ci/comment-guard.py`. Each section is
pointed to from a short `//` comment at the named item.
## `hevc_num_extra_slice_header_bits`
`num_extra_slice_header_bits` from a HEVC PPS NAL (H.265 §7.3.2.3): after the
2-byte NAL header, skip `pps_pic_parameter_set_id` + `pps_seq_parameter_set_id`
(both `ue(v)`) and `dependent_slice_segments_enabled_flag` +
`output_flag_present_flag` (`u(1)` each), then read `u(3)`. `None` if the PPS
is too short to parse — the caller then declines to guess a slice type.
## `hevc_first_slice_coding_type`
Measures the coding type from the FIRST coded slice of an access unit
(H.265 §7.3.6.1 `slice_segment_header`). Reads only the leading fields of the
first slice segment: `first_slice_segment_in_pic_flag` u(1), the IRAP
`no_output_of_prior_pics_flag` u(1), `slice_pic_parameter_set_id` ue(v), the
`num_extra_slice_header_bits` reserved bits, then `slice_type` ue(v). Returns
`None` for a non-first slice or on truncation — never a guess. `num_extra`
MUST come from the active PPS so the bit offset to `slice_type` is exact.
## `HevcParser::hdr10`
Combines the accumulated mastering-display and content-light SEI into a
complete `Hdr10Metadata`, or `None` until BOTH HDR10 SEI messages have been
seen. Requiring both means an SDR / partially-signalled stream never emits a
half-populated (confidently-wrong) HDR10 record.
## `HevcParser::scan_sei`
Scans an SEI NAL (`[2-byte NAL header][RBSP]`) for the two HDR10 payload
types and captures each the FIRST time it appears (per-stream constants).
RBSP structure (Rec. ITU-T H.265 D.2 `sei_rbsp` / `sei_message`): a sequence
of messages, each `payloadType` then `payloadSize` encoded as a run of 0xFF
bytes plus a final <0xFF byte (the "ff-extension" coding), followed by
`payloadSize` payload bytes. Emulation-prevention (00 00 03) is stripped
before reading — unlike a slice header, an SEI payload can be deep enough
that an emulation byte falls inside the fields we read. Unknown payload
types are skipped by their size so a later HDR10 message in the same NAL is
still reached.
## `HevcParser::mark_clip_boundary`
MPLS `connection_condition` 0x05 and 0x06 are the non-seamless values (per
the BD-ROM spec: 0x01 = first item / seamless, 0x05/0x06 = non-seamless).
The first CRA at/after this call is rewritten CRA_NUT (21) → BLA_W_LP (16)
so a linear decoder sets NoRaslOutput and discards the now-dangling RASL
leading pictures with no "could not find ref" error.
It is a no-op for the rewrite unless a CRA actually follows: an
IDR/IDR_W_RADL boundary needs no fix (it carries no cross-splice
references), and the flag is cleared by the first IRAP-class NAL it reaches.
SAFETY: never call this for connection_condition 0x01 (seamless/first item)
or within a single-clip title — doing so could convert a legitimate
mid-content CRA to BLA. The default (never called) path leaves output
byte-identical.
## `handle_param_set`
Handles a VPS/SPS/PPS NAL: decides whether to strip it (the decoder already
has the value) or emit it in-band, and tracks the currently-active body.
The decision MUST be made against the currently-active set (`cur`), NOT the
codecPrivate copy (`first`). The two player behaviours for hvcC-in-MKV
diverge exactly here:
- A *seek-capable / Annex-B* player (one that converts hvcC to Annex-B by
inserting the parameter sets) re-applies the hvcC sets at every keyframe.
`reassert_active` handles it.
- A *streaming* decode (a decoder consuming the MKV directly — what most
integrity checkers do) applies hvcC ONCE at init and thereafter updates a
parameter set ONLY from an in-band NAL.
So when a title redefines a set mid-stream (id 0 body A → B) and later
switches BACK to A (== codecPrivate), the change to A must STILL be emitted
in-band: the streaming decoder is sitting on B and will never revert
otherwise, decoding the whole A-segment against B → CABAC/cu_qp_delta
desync. Stripping on `== first` (the old behaviour) dropped exactly that
revert and corrupted every "switch back to the first body" segment.
Rules:
- First of its type → seeds codecPrivate; stripped (the decoder gets it from
hvcC at init).
- Equal to the active set `cur` → redundant; stripped.
- Different from `cur` (a change, in EITHER direction) → emitted in-band and
`cur` updated.
Returns `true` when the NAL was emitted in-band into `frame_data`.
## `reassert_active`
Appends the active parameter set `cur` to `prefix` (length-prefixed) so
every keyframe is SELF-CONTAINED: it carries the active VPS/SPS/PPS in-band
ahead of its slices. Skipped only when this access unit ALREADY carried the
NAL in-band (`emitted` — avoids a duplicate) or no active set exists yet.
Why unconditional (not only when the active set differs from codecPrivate):
a streaming decoder applies the hvcC param sets once at init, then relies on
in-band repetition. Some sources stop repeating a param set at later IRAPs
even though its body is unchanged; if the decoder then drops it (a CRA reset
or SPS event), nothing re-sends it and every subsequent slice fails with
"PPS id out of range" until the next genuine change (observed as a ~24 min
corrupt band on one dual-layer UHD title). Re-asserting the active set at
EVERY keyframe — what compliant Matroska muxers do at every IRAP — makes
streaming decode self-healing. Re-sending an identical param set is benign
(decoders expect it at IRAPs); cost is a few hundred bytes per keyframe.
This strictly supersets the earlier change-only re-assert, so the
param-set-revert fix is unaffected.
## `parse_mastering_display`
Parses a Mastering Display Colour Volume SEI payload (Rec. ITU-T H.265
D.2.28 / semantics D.3.28). Layout — 24 bytes total, all big-endian:
`display_primaries_x[c]` u(16), `display_primaries_y[c]` u(16) for c=0,1,2
(SEI primary order is c=0 Green, c=1 Blue, c=2 Red); white_point_x u(16),
white_point_y u(16); max_display_mastering_luminance u(32);
min_display_mastering_luminance u(32). Returns `None` if the payload is
shorter than 24 bytes (malformed → ignored, never partially populated).
## Test notes
**`hevc_hdr10_sei_keeps_the_first_value_and_ignores_later_repeats`** — the
`scan_sei` match arms are guarded with `self.sei_mastering.is_none()` /
`self.sei_content_light.is_none()`, so the FIRST mastering-display and
content-light SEI a title carries wins and later repeats (every AU of a real
HDR10 stream repeats both) are ignored. Each AU below carries only ONE of
the two messages, so the whole-scan early return
(`if self.sei_mastering.is_some() && self.sei_content_light.is_some()`)
never fires early and the per-arm guards are actually exercised.
**`sei_nal` (test helper)** — wraps one or more SEI messages in a prefix-SEI
NAL (type 39) preceded by an Annex-B start code. The assembled message bytes
are emulation-prevented (as a conforming encoder would) so they never form a
false start code; the 0x80 RBSP trailing-bits byte is appended.
**`scan_sei_stops_copying_once_both_hdr10_messages_are_captured`** — MEASURED,
not reasoned: an HDR10 stream carries a prefix SEI per access unit, and
`scan_sei` used to allocate + byte-copy the whole SEI RBSP through
`strip_emulation_prevention` on EVERY one, including after both HDR10
messages were already captured. On a ~200,000-frame UHD title that is
~200,000 wasted allocations/copies. Counted at the single
`strip_emulation_prevention` call site.
**`nonzero_num_extra_slice_header_bits_shifts_the_slice_type_offset`** —
`num_extra_slice_header_bits` (H.265 §7.3.2.3) is a PPS field, and the
`slice_reserved_flag[i]` bits it counts sit BETWEEN
`slice_pic_parameter_set_id` and `slice_type` in the slice segment header
(§7.3.6.1). Every earlier fixture used a PPS with the field == 0, so the
skip was never exercised: a parser that ignored the field entirely agreed
with all of them, and would then read `slice_type` from the wrong bit offset
on any real stream that sets it — mislabelling every picture's coding type.
**`reasserts_active_pps_at_bare_keyframe`** — regression (UHD banded
corruption): a stream redefines PPS id 0 mid-title, then a later keyframe
arrives WITHOUT repeating it (the source relies on the decoder retaining the
redefinition — valid for a raw bitstream). An hvcC player re-applies the
FIRST (codecPrivate) PPS at every keyframe, so the active redefinition must
be re-asserted in-band at that bare keyframe or the whole segment decodes
against the wrong parameter set.
**`keyframe_param_reassert_does_not_reallocate_the_frame`** — MEASURED: the
keyframe parameter-set re-assert must be spliced into the front of the
already-assembled access unit IN PLACE, not built as a fresh full-size
buffer. It used to `prefix.extend_from_slice(&frame_data)` and then replace
`frame_data` with `prefix`, which grew a few-hundred-byte `prefix` to the
FULL access-unit size — a fresh multi-MB allocation — memcpy'd the whole
frame into it, and dropped the presized buffer. One extra whole-frame
allocation plus one extra whole-frame copy per keyframe: a 2 h UHD title at
24 fps with a 1 s GOP is ~7,200 keyframes, ~14-28 GB of avoidable memcpy per
title. `PARAM_REASSERT_HEADROOM` exists so the splice never reallocates;
this test counts the reallocations that happen, which must be zero.
**`emits_switch_back_to_codecprivate_pps`** — regression (a UHD title, the
real bug): id 0 is body A (→ hvcC), then redefined to B, then the title
switches BACK to A. A streaming decoder (hvcC at init, in-band updates only)
is sitting on B; the switch back to A must be emitted IN-BAND even though A
== codecPrivate, or the whole A-segment decodes against B (cu_qp_delta
desync). Stripping on `== hvcC` dropped this revert.
**`cra_at_auto_detected_pts_backstep_rewritten_to_bla`** — regression for a
UHD Dolby Vision profile 7 dual-layer title: a multi-clip title is read as
one concatenated stream and the mpls connection_condition is never plumbed
to the parser, so the splice CRA opening the next clip kept its dangling
RASL leading pictures and a linear decoder flooded "Could not find ref with
POC N". The parser must AUTO-DETECT the boundary from the backward PES-PTS
reset between clips (each .m2ts has its own PTS base) and rewrite that
splice CRA → BLA_W_LP with no explicit `mark_clip_boundary` call.
**`cra_after_33bit_pts_wrap_not_rewritten`** — regression for the 33-bit PTS
wraparound false-trigger (rc.5.2 audit #1): a SINGLE clip whose raw 90 kHz
PES PTS crosses the 2^33 counter wrap (~26.5 h) must NOT be mistaken for a
non-seamless clip join. Before the fix the raw 2^33→0 backward step armed
`pending_clip_boundary` and the next in-clip CRA was wrongly rewritten
CRA→BLA_W_LP (dropping valid RASL pictures — visible corruption). After
unwrapping onto a monotonic timeline the wrap is absorbed and the CRA stays
CRA.
**`cra_splice_detected_at_large_pts_magnitude_not_masked_by_wrap_logic`** —
the wrap-vs-backstep test above keeps `high_pts` near the very top of the
33-bit range, close enough to the `PTS_WRAP_PERIOD / 2` threshold that
`high - unwrapped` and a hand-flipped `high + unwrapped` land on the SAME
side of the threshold at every step — it doesn't actually distinguish the
two. This test uses PTS magnitudes around 3e9 (order 2^32, well below the
wrap threshold but reached in well under an hour), where the subtraction and
the addition diverge: ordinary forward progression keeps `high - unwrapped`
small, but `high + unwrapped` would already exceed `PTS_WRAP_PERIOD / 2` on
the very next frame, which would pollute `pts_wrap_offset` by a full period
and mask the genuine two-clip splice that follows.
**`non_cra_nals_never_rewritten_at_boundary`** — non-CRA NALs are never
rewritten even when a boundary IS marked. IDR (19), RASL (8/9), VPS/SPS/PPS,
and a trailing slice all pass through unmodified; the IDR clears the pending
boundary so no later CRA is wrongly converted.
**`no_boundary_marker_is_byte_identical`** — a frame stream with NO boundary
marker is BYTE-IDENTICAL to a parser that has no splice-rewrite field at all
(the UHD-safety guarantee): asserts byte-equality of every emitted frame
across a multi-AU stream containing CRAs, IDRs, RASLs, VPS/SPS/PPS, and
trailing slices — none of which is ever marked.
**`seamless_boundary_no_rewrite`** — a SEAMLESS boundary (connection_condition
0x05/0x06) is expressed by NOT calling `mark_clip_boundary`, so a CRA across
a seamless join is left unchanged. This encodes the contract: only
non-seamless joins call `mark_clip_boundary`; seamless ones never do, so no
rewrite occurs.
**`redefined_pps_emitted_inline`** — a parameter set REDEFINED mid-stream
(same id, different body) must be emitted INLINE so the decoder
re-activates it. Some discs redefine PPS id 0 partway through the title;
the old parser kept only the first PPS, so the second segment decoded
against the wrong PPS (CABAC desync).
**`make_sps_full` (test helper)** — builds a stored SPS NAL with sub-layers
and a conformance window, so the parser must skip sub-layer PTL and the 4
conformance-window ue(v) fields before reaching the bit depths.
`max_sub_layers_minus1` controls the sub-layer loop.
**`hvcc_array_length_round_trips_above_256_bytes`** — the hvcC array length
is a 16-bit big-endian field written as two separate `push`es: `(len >> 8)
as u8` then `len as u8`. Every VPS/SPS/PPS the rest of the suite feeds is
well under 256 bytes, so the high byte is always 0 and a `>>` -> `<<`
mutation (which also always truncates to 0 for those inputs) is
unobservable there. A real HEVC SPS with an extended VUI/HRD block can
exceed 256 bytes, so this test uses 300+ byte VPS/SPS/PPS and decodes the
16-bit length fields back to confirm they round-trip to the exact NAL
length, not merely a byte that happens to be 0.
**`hevc_ps_reorder_is_installed_and_flush_drains_its_real_frames`** — HEVC
counterpart of `h264_ps_reorder_reconstructs_distinct_display_pts`. A
DVD/HD-DVD program stream stamps a PTS only on each GOP anchor, so the
parser must reconstruct display-order timestamps for the rest. Two things
are pinned here that nothing else constrains: `with_ps_reorder(true)` must
actually INSTALL the reorderer (a builder that returned a
default-constructed parser would silently leave the transport-stream path
in place on every DVD title), and `flush()` must drain the reorderer's real
buffered frames at EOF — not nothing, and not a manufactured empty frame at
PTS 0 (a zero-length Block whose timestamp jumps back behind every cluster
already written, RFC 9559 §5.1.3.2). Slice bodies follow H.265 §7.3.6.1:
`first_slice_segment_in_pic_flag`, the IRAP-only
`no_output_of_prior_pics_flag`, `ue(pps_id)`, `ue(slice_type)` (Table 7-7:
0 = B, 1 = P, 2 = I). PPS body 0xC0 sets `num_extra_slice_header_bits = 0`.
+26
View File
@@ -0,0 +1,26 @@
# hex
The single hex → bytes parser for the whole workspace.
## Why one parser
Key material arrives as hex from three third-party sources — the keydb, an
online key service, and the mapfile's `# freemkv-vid:` comment — and each
used to parse it slightly differently (one stripped `0x`/`0X`, one stripped
nothing, one stripped `0x` only). A key written with a prefix one parser
didn't expect was silently dropped → "can't decrypt" with no error. This is
the one parser they all call, so the prefix/case/validation rules live in
exactly one place.
## Byte-oriented, not char-oriented
Operates on BYTES, not `&str` char indices: the inputs are untrusted, so a
multi-byte UTF-8 scalar must reject as malformed, never panic on a
mid-codepoint slice.
## `byte()` internals
Combines two ASCII hex-digit bytes into one byte. `as char` is intentional:
for a non-ASCII byte it produces a Latin-1 scalar that `to_digit(16)` then
rejects — so non-hex (incl. `+`/`-` sign chars) and multi-byte input fail
cleanly rather than slipping through `from_str_radix`'s sign handling.
+75
View File
@@ -0,0 +1,75 @@
# identity.rs — drive identity test notes
## `OversizedCountTransport` (test double)
Returns the requested data length but reports a `bytes_transferred` larger
than the caller's buffer — models a drive that lies about its transfer
count. The old slicing code panicked on this; the clamps in `from_drive`
must keep it from indexing out of range.
## `inquiry_with_a_short_data_phase_fails_instead_of_reporting_a_blank_drive`
`ascii_field` with a buffer shorter than `start` returns empty string
rather than panicking.
- Spec: SPC-4 §6.4.2 — bytes[8:16] are vendor ID; a truncated buffer (e.g.
a device that reports fewer than 8 bytes) must not panic.
- Mutation: removing the `data.len() > start` guard makes it panic on
short inputs.
A drive that answers INQUIRY with GOOD status but a short or empty data
phase must fail the probe, not present as a blank drive.
The buffer is pre-zeroed, so decoding it unconditionally yielded empty
vendor/product/revision strings and a byte 0 of 0x00. Every platform
enumerator gates on `raw_inquiry[0] & 0x1F == SCSI_PERIPHERAL_TYPE_OPTICAL`,
and 0x00 is DIRECT ACCESS — so the drive silently vanished from the device
list rather than reporting that its identity probe failed. A USB-SATA
bridge mid-wedge does exactly this.
The two GET CONFIGURATION calls in the same function already clamped on
`bytes_transferred`, with a comment calling it untrusted; INQUIRY, three
lines above them, discarded it.
## `ascii_field_boundary_len_equals_start_is_empty`
`ascii_field`'s guard is `data.len() > start` (strictly greater), not
`>=`: a buffer whose length is exactly `start` has NO byte at that offset,
so it must still yield empty, not attempt to slice.
Mutation: `>` -> `>=` would try to slice `data[start..]` when
`data.len() == start`, which panics (empty range at the very end is fine,
but the guard's job is the `< start` case below it — pinning the exact
boundary catches an off-by-one either direction).
## `display_formats_trimmed_fields_space_separated`
`Display` renders the four trimmed identity fields space-separated — the
human-readable counterpart of `match_key`'s pipe-separated form. Not
exercised anywhere else in this test module.
Mutation: replacing the `fmt` body with `Ok(Default::default())` writes
nothing at all, so formatting any `DriveId` yields "".
## `FixedGcCountTransport` (test double) / `from_drive_gc_failure_yields_empty_firmware_date`
GET CONFIGURATION failure (transport error) must not abort the identity
probe — firmware_date is empty, raw_gc_010c is empty. Mutation:
propagating the GET_CONFIGURATION error with `?` aborts `from_drive`.
`FixedGcCountTransport` is a transport whose GET CONFIGURATION responses
report an exact, caller-chosen `bytes_transferred` for each of the two GC
features (010Ch firmware date / 0108h serial), so the `end > 12` / `> 12`
boundary guards can be pinned precisely. INQUIRY always succeeds.
## `from_drive_firmware_date_boundary_exactly_12_is_empty`
`end > 12` in the firmware-date branch (`from_drive`) is a strict
inequality: `bytes_transferred == 12` reports the field absent (offset 12
is the first byte of the 12-char date; a count of exactly 12 covers bytes
0..12, none of which is the date), so `firmware_date` must be empty, not
the mutant's off-by-one read.
Mutation: `>` -> `>=` would try `gc[12..12]` at the boundary — an empty
but non-panicking slice — silently reporting "present" data that is
actually all outside the transferred count.

Some files were not shown because too many files have changed in this diff Show More